- Restrict fallback_models to only reference preset names in model_presets.
- Add schema validation to reject unknown preset names in fallback_models.
- Remove build_provider_for_model() since bare model fallback is no longer supported.
- Simplify make_provider_factory() to only look up presets by name.
- Update onboard UI to remove "Add custom model" option from fallback chain.
- Update tests to use preset names instead of bare model strings in fallback chains.
- Fix test imports referencing deleted _make_provider function.
When the primary model fails (finish_reason="error" after exhausting
provider-level retries), automatically try each model in the configured
fallback_models list. Supports cross-provider fallback via a cached
provider_factory that resolves the correct provider for each model string.
Config:
agents.defaults.fallback_models: ["model-b", "provider/model-c"]
Changes:
- AgentDefaults: add fallback_models field
- AgentRunSpec: add fallback_models field
- AgentRunner: add provider_factory, _call_provider, _resolve_fallback_provider
- AgentLoop: accept and forward fallback_models + provider_factory
- nanobot.py: extract _make_provider_for_model, add _make_provider_factory
- cli/commands.py: add _make_cli_provider_factory, wire all AgentLoop sites
- tests/agent/test_runner_fallback.py: 8 test cases covering primary success,
single/multi fallback, cross-provider, no-factory reuse, caching
Made-with: Cursor
Add ModelPresetConfig schema and model_presets dictionary to config,
enabling named bundles of model parameters (model, temperature,
max_tokens, reasoning_effort, context_window_tokens) that can be
switched atomically at runtime via the self tool.
Fixes two related input-handling bugs in the onboard wizard:
1. _input_text treated "" as None, preventing users from clearing
optional string fields or entering empty strings intentionally.
2. _input_model_with_autocomplete used `if value else None`, which
discarded falsy values such as empty strings or 0.
To support clearing optional string fields, add _is_str_or_none() and
normalize empty strings to None inside _configure_pydantic_model only
when the field annotation is `str | None`. Required str fields keep
"" as a valid value.
Also included:
- Remember last selected item in provider/channel/model menus for
better UX when configuring multiple items.
- Rename _SIMPLE_TYPES and _MENU_DISPATCH to lowercase to follow
Python naming conventions (they are local variables, not constants).
- Remove unused imports in test file.
Extracted from PR #3358.
Reference hermes-agent#17228 / #18100 / PR#18105.
iLink returns ret=-2 / errcode=-2 for two different reasons:
- stale context_token: errmsg is empty/None or "unknown error"
- genuine rate limit: errmsg is populated (e.g. "frequency limit")
Previously we swallowed all ret=-2 responses, which caused silent
message drops when the context_token was stale.
Changes:
- Add _is_stale_session_ret() to detect empty/"unknown error" errmsg
- _send_text/_send_media_file retry once without context_token on stale
session signal, then raise on persistent failure so ChannelManager
can retry with backoff
- Remove error-swallowing behavior
- Update tests to expect raises and add TestIsStaleSessionRet coverage
The iLink sendmessage API frequently returns ret=-2 (parameter error / rate
limit / expired token) even when HTTP status is 200. The openclaw reference
plugin ignores the JSON body for sendmessage entirely and only checks HTTP
status. Our previous strict ret checking turned ret=-2 into RuntimeError,
causing ChannelManager retries which only made things worse.
Changes:
- _send_text: swallow ret=-2 after one retry without context_token.
Log request body + response at warning level for diagnostics.
- _send_media_file: same ret=-2 swallowing.
- _generate_client_id: change format to ``nanobot:{timestamp}-{hex}`` to
match openclaw-weixin ``{prefix}:{Date.now()}-{hex}``.
- Update tests to expect swallowing instead of raising for ret=-2.
When the iLink API returns ret=-2 (parameter error), it is often caused
by an expired context_token rather than a malformed payload. After a
gateway restart, the cached token can become stale within ~90 seconds if
no new inbound message refreshes it, causing all outbound replies to fail
silently.
Changes:
- _send_text: retry once without context_token when ret=-2 and a token
was present; if the retry succeeds, clear the expired token from cache.
- Remove leftover @staticmethod on _check_response_error so self.logger
and the body parameter work correctly.
- Bump WEIXIN_CHANNEL_VERSION from 2.1.1 -> 2.1.7 to match the reference
openclaw-weixin plugin.
- Add tests covering the ret=-2 retry path, failure path, and no-token
path.
References:
- openclaw/openclaw#61174 (context_token expiry after long agent turns)
- hermes-agent#21011 (ret=-2 rate limiting / parameter error)
The iLink API signals failures through either `ret` or `errcode`.
`_poll_once` already checked both, but `_send_text` and `_send_media_file`
only checked `errcode`. When the API returned `ret != 0` with
`errcode == 0`, the send appeared successful but the message was never
delivered, causing the "still losing messages" issue.
- Add `_check_response_error` helper that validates both fields
- Use it in `_send_text` and `_send_media_file`
- Add debug log after successful text send for observability
- Add test for nonzero ret with zero errcode
Refs: previous inbound fix (suppress -> explicit try/except)
Replace `with suppress(Exception)` in `_poll_once` message processing
and the `start()` poll loop with explicit `try/except` blocks that
log errors via `logger.exception`. Previously, any exception during
message processing (e.g. in `_handle_message`) was swallowed silently,
causing inbound messages to disappear without a trace.
Also add tests verifying that:
- `_poll_once` logs and continues when `_process_message` fails
- the poll loop logs and continues when `_poll_once` fails
Add a session-scoped slash command palette sourced from backend command metadata, and keep welcome-page quick actions localized across all WebUI languages.
Co-authored-by: Cursor <cursoragent@cursor.com>
_send_text() swallowed API errors (non-zero errcode) with just a
warning log, and send() had three silent return paths (no client,
session paused, no context_token). Neither triggered ChannelManager's
retry logic, causing persistent message loss until a new inbound
message refreshed the context_token.
Now all failure paths raise RuntimeError, matching BaseChannel's
contract and enabling proper retry behavior.
When host is set to 0.0.0.0, the gateway now enforces that either token
or token_issue_secret must be configured — it refuses to start otherwise.
Bootstrap endpoint behavior:
- token_issue_secret configured: always validate regardless of source IP
(handles reverse-proxy scenarios where all connections appear as localhost)
- No secret: only localhost can bootstrap (local dev mode)
The frontend shows an authentication form when bootstrap returns 401/403,
persists the secret in localStorage, and retries automatically on reload.
The previous LAN-access fix (PR #3656) relaxed the bootstrap localhost
check when host was 0.0.0.0, but did not require any authentication —
any device on the network could obtain a token without credentials.
New behavior:
- token_issue_secret configured: always validate, regardless of source
IP (handles reverse-proxy scenarios where all connections appear as
localhost).
- No secret configured: only localhost can bootstrap (local dev mode).
This supersedes the host-based check from PR #3656.
The webui bootstrap endpoint (/webui/bootstrap) rejected all non-localhost
connections with HTTP 403, preventing the embedded webui from working when
accessed from another device on the LAN — even when host was set to 0.0.0.0.
Skip the localhost check when the server is explicitly bound to 0.0.0.0 or ::,
since that signals intent to accept external connections.
Align the WebUI sidebar and chat chrome with the updated design, and generate WebUI session titles asynchronously without blocking turns.
Co-authored-by: Cursor <cursoragent@cursor.com>
The is_path branch in _fmt_known was not passing max_length to
abbreviate_path, so read_file, write_file, edit, list_dir, and
web_fetch always truncated paths at 40 chars regardless of config.
Now all three branches (is_path, is_command, fallback) honor the
configured toolHintMaxLength.
Add to config (default: 40, range: 20-500).
Controls how many characters of tool hints are shown in progress updates
(e.g. '$ cd …/project && npm test').
Set to 120+ to see full commands instead of truncated hints:
```json
{
"agents": {
"defaults": {
"toolHintMaxLength": 120
}
}
}
```
- Thread max_length through format_tool_hints → _fmt_known/_fmt_mcp/_fmt_fallback
- Make path abbreviation in _abbreviate_command proportional to max_length
- Add TestToolHintMaxLength test class with 5 tests
- All 41 existing tests pass
- Correct api_key type hint to str | None in _post_transcription_with_retry
- Remove unreachable final return ""
- Fix test_openai_missing_api_key_short_circuits to actually test
missing-key path (use audio_file fixture so file exists)
- Fix PermissionError patch for Windows (patch class method instead
of instance attribute)
A single transient failure between the agent and an OpenAI/Groq Whisper
endpoint currently vanishes as `return ""` in transcribe(). The voice
message arrives as the empty string and there is no way to tell real
silence apart from a failed upload. A malformed but successful response
body is even worse: the JSON-decode error escapes the helper unhandled.
Add a shared `_post_transcription_with_retry` used by both providers.
Retry behaviour:
- exponential backoff 1s -> 2s -> 4s, up to 3 retries (4 attempts)
- retryable HTTP statuses: 408, 429, 500, 502, 503, 504
- retryable exceptions: TimeoutException, ConnectError, ReadError,
WriteError, RemoteProtocolError
Non-transient failures short-circuit to "" on the first attempt --
retrying a misconfigured key or a broken upload only burns rate-limit
quota. Branches that short-circuit:
- missing API key, missing audio file
- file-read errors (PermissionError, OSError) on the audio path,
preserving the nightly contract for direct provider callers
- HTTP auth/4xx body issues via raise_for_status()
- response.json() parse failures
- non-dict JSON payloads
Sharing one helper means OpenAI and Groq cannot drift apart silently.
Thread `language` through the helper. The multipart files dict is rebuilt
inside the per-attempt loop, so when a caller sets self.language the
`language` field is sent on every attempt -- not just the first.
Tests cover:
- every advertised retryable status and exception, parameterized
- language present on attempts 1 and 2 of a 503->200 sequence
- language absent when unset; present when set (both providers)
- malformed JSON body and non-dict JSON body short-circuit to ""
- PermissionError on file read short-circuits with no HTTP attempt
- max-attempts give-up, exponential-backoff schedule, auth no-retry,
missing-key / missing-file short-circuit
Test stub fix: the _StubResponse in tests/channels/test_channel_plugins.py
declared no status_code, which the new helper reads for retry classification.
Set status_code = 200 so the stub advertises the successful response that
those tests already simulate. Also moved the two transcription-provider
imports to the top of that file (previously placed mid-file) so the file
is ruff-clean (E402).
- Correct api_key type hint to str | None in _post_transcription_with_retry
- Remove unreachable final return ""
- Fix test_openai_missing_api_key_short_circuits to actually test
missing-key path (use audio_file fixture so file exists)
- Fix PermissionError patch for Windows (patch class method instead
of instance attribute)
A single transient failure between the agent and an OpenAI/Groq Whisper
endpoint currently vanishes as `return ""` in transcribe(). The voice
message arrives as the empty string and there is no way to tell real
silence apart from a failed upload. A malformed but successful response
body is even worse: the JSON-decode error escapes the helper unhandled.
Add a shared `_post_transcription_with_retry` used by both providers.
Retry behaviour:
- exponential backoff 1s -> 2s -> 4s, up to 3 retries (4 attempts)
- retryable HTTP statuses: 408, 429, 500, 502, 503, 504
- retryable exceptions: TimeoutException, ConnectError, ReadError,
WriteError, RemoteProtocolError
Non-transient failures short-circuit to "" on the first attempt --
retrying a misconfigured key or a broken upload only burns rate-limit
quota. Branches that short-circuit:
- missing API key, missing audio file
- file-read errors (PermissionError, OSError) on the audio path,
preserving the nightly contract for direct provider callers
- HTTP auth/4xx body issues via raise_for_status()
- response.json() parse failures
- non-dict JSON payloads
Sharing one helper means OpenAI and Groq cannot drift apart silently.
Thread `language` through the helper. The multipart files dict is rebuilt
inside the per-attempt loop, so when a caller sets self.language the
`language` field is sent on every attempt -- not just the first.
Tests cover:
- every advertised retryable status and exception, parameterized
- language present on attempts 1 and 2 of a 503->200 sequence
- language absent when unset; present when set (both providers)
- malformed JSON body and non-dict JSON body short-circuit to ""
- PermissionError on file read short-circuits with no HTTP attempt
- max-attempts give-up, exponential-backoff schedule, auth no-retry,
missing-key / missing-file short-circuit
Test stub fix: the _StubResponse in tests/channels/test_channel_plugins.py
declared no status_code, which the new helper reads for retry classification.
Set status_code = 200 so the stub advertises the successful response that
those tests already simulate. Also moved the two transcription-provider
imports to the top of that file (previously placed mid-file) so the file
is ruff-clean (E402).
The is_path branch in _fmt_known was not passing max_length to
abbreviate_path, so read_file, write_file, edit, list_dir, and
web_fetch always truncated paths at 40 chars regardless of config.
Now all three branches (is_path, is_command, fallback) honor the
configured toolHintMaxLength.
Add to config (default: 40, range: 20-500).
Controls how many characters of tool hints are shown in progress updates
(e.g. '$ cd …/project && npm test').
Set to 120+ to see full commands instead of truncated hints:
```json
{
"agents": {
"defaults": {
"toolHintMaxLength": 120
}
}
}
```
- Thread max_length through format_tool_hints → _fmt_known/_fmt_mcp/_fmt_fallback
- Make path abbreviation in _abbreviate_command proportional to max_length
- Add TestToolHintMaxLength test class with 5 tests
- All 41 existing tests pass
Disable the externally updated cron job before yielding to the event loop so slow Windows CI cannot run the short-interval job before the test writes the update.
Keep private URL access blocked at the tool boundary, but return a clear non-retryable hint so the agent can recover conversationally instead of aborting the turn.
Co-authored-by: Cursor <cursoragent@cursor.com>
``Nanobot.run()`` has always documented ``RunResult.tools_used`` and
``RunResult.messages`` but actually returned ``[]`` for both, so SDK
consumers could never inspect which tools fired or what the final
message list looked like — the only useful field was ``content``.
This threads the data out via a tiny ``_SDKCaptureHook`` that installs
alongside any user-supplied hooks. The capture hook accumulates tool
names across iterations and snapshots the message list on each
``after_iteration`` call; the last snapshot reflects end-of-turn state.
Only the SDK facade is touched: ``AgentLoop.process_direct`` and
``AgentRunner`` signatures are unchanged, so channels / CLI / API paths
are unaffected.
Replace the asyncio.Semaphore queueing approach with a simple count
check in SpawnTool.execute(). When the concurrency limit is reached,
the tool returns an error string so the agent can perceive the reason
and adjust its behavior instead of silently queueing.
- Remove max_concurrent_subagents parameter threading through
AgentLoop, commands.py, and nanobot.py
- SubagentManager reads the limit directly from AgentDefaults
- SpawnTool checks get_running_count() before calling spawn()
- Simplify tests to verify rejection behavior
Replace the asyncio.Semaphore queueing approach with a simple count
check in SpawnTool.execute(). When the concurrency limit is reached,
the tool returns an error string so the agent can perceive the reason
and adjust its behavior instead of silently queueing.
- Remove max_concurrent_subagents parameter threading through
AgentLoop, commands.py, and nanobot.py
- SubagentManager reads the limit directly from AgentDefaults
- SpawnTool checks get_running_count() before calling spawn()
- Simplify tests to verify rejection behavior
``Nanobot.run()`` has always documented ``RunResult.tools_used`` and
``RunResult.messages`` but actually returned ``[]`` for both, so SDK
consumers could never inspect which tools fired or what the final
message list looked like — the only useful field was ``content``.
This threads the data out via a tiny ``_SDKCaptureHook`` that installs
alongside any user-supplied hooks. The capture hook accumulates tool
names across iterations and snapshots the message list on each
``after_iteration`` call; the last snapshot reflects end-of-turn state.
Only the SDK facade is touched: ``AgentLoop.process_direct`` and
``AgentRunner`` signatures are unchanged, so channels / CLI / API paths
are unaffected.
Logout previously claimed to support github-copilot in --help text but had
no registered handler, so `provider logout github-copilot` failed with
"Logout not implemented". Add the handler, sharing token deletion with the
codex flow via `_delete_oauth_files`. Tighten handler-table types, fix the
codex test fixture filename, and cover github-copilot plus the unknown
provider path.
- Implement \
anobot provider logout <provider>\ to clear OAuth credentials.
- Add \_LOGOUT_HANDLERS\ registration mechanism mirroring login.
- Implement logout for \openai-codex\ by deleting local \oauth-cli-kit\ token and lock files.
- Fallback gracefully when attempting to logout from providers lacking local credentials or implementations.
- Fixes#2665
Keep the /dev workspace guard exception scoped to the known benign device paths already handled by ExecTool, and add coverage that non-benign /dev targets still get blocked. Also add a streaming regression for tool_error responses so fatal tool failures are delivered by channels instead of being marked as already streamed.
Co-authored-by: Cursor <cursoragent@cursor.com>
Three independent fixes for issues exposed by PR #3493:
1. shell.py: allow /dev/* paths in workspace guard
Commands like `rm file.txt 2>/dev/null` were blocked because
_extract_absolute_paths captured /dev/null as a path outside
the workspace. Allow /dev like media_path is already allowed.
2. shell.py: remove | from home_paths regex prefix
Loki query operator `|~` was misinterpreted as pipe + home
directory, causing false workspace violation errors.
3. loop.py: change _streamed from blacklist to whitelist
stop_reason "tool_error" was not in the exclusion set
{"ask_user", "error"}, so _streamed=True was set on fatal
errors. channel manager then skipped channel.send() because
it assumed the content was already streamed — but it never
was. Whitelist to only {"stop", "end_turn", "max_tokens"}.
Also fixes a pre-existing Windows bug in _spawn where
create_subprocess_exec + list2cmdline breaks commands with
paths containing spaces (e.g. D:\Program Files\python.exe).
Closes: #3599, #3605
Keep the workspace-boundary changes easier to review by trimming long explanatory comments down to short local notes. Also make the #3599 POSIX command regression skip on Windows and normalize workspace violation signatures to POSIX separators so the throttle tests are platform-stable.
Tests:
- uv run pytest tests/tools/test_exec_security.py tests/utils/test_workspace_violation_throttle.py -q
- uv run pytest -q
Co-authored-by: Cursor <cursoragent@cursor.com>
Replaces PR #3493's blanket fatal abort with a "tell the model + throttle
the bypass loop" policy. Workspace-bound rejections are now ordinary
recoverable tool errors enriched with a structured "this is a hard policy
boundary" instruction; SSRF stays the only marker that aborts the turn.
Why the fatal-abort approach broke
----------------------------------
PR #3493 promoted every shell `_guard_command` and filesystem path-resolution
rejection to a turn-fatal RuntimeError. Two of those messages (`path
outside working dir` and `path traversal detected`) are heuristic substring
scans on the raw command, so legitimate commands like `rm <ws>/x.txt
2>/dev/null` or `find . -type f` killed the user's turn (#3599). On
channels with outbound dedupe (Telegram) the user just saw silence (#3605),
and the noise polluted the LLM's context until it started hallucinating
guard rejections on plain relative paths (#3597).
Why we still need *some* throttle
---------------------------------
The original #3493 pain point was real: the LLM, refused once, would
swap tools and try again -- read_file -> exec cat -> exec cp -> bash -c
-> ln -sf -> python -c open(...). Just removing the fatal escape lets
that loop run wild until max_iterations.
What this commit does
---------------------
- `nanobot/utils/runtime.py`: add `workspace_violation_signature` and
`repeated_workspace_violation_error`. The signature normalizes
filesystem `path` arguments and the first absolute path inside an
exec command, so swapping tools against the same outside target hits
the same throttle bucket. Two soft attempts are allowed; the third
attempt's tool result is replaced with a hard "stop trying to bypass"
message that quotes the target path and tells the model to ask the
user for help.
- `nanobot/agent/runner.py`: split classification into `_is_ssrf_violation`
(still fatal) and `_is_workspace_violation` (now soft). All three
failure branches in `_run_tool` (prep_error / exception / Error
result) route through a shared `_classify_violation` that bumps the
per-turn workspace_violation_counts dict and either keeps the tool's
own message or substitutes the throttle escalation. `_execute_tools`
now threads that dict alongside the existing external_lookup_counts.
- `nanobot/agent/tools/shell.py`: append a structured boundary note to
every workspace-bound guard rejection (`working_dir could not be
resolved`, `working_dir is outside`, `path outside working dir`,
`path traversal detected`). SSRF errors stay short and direct so the
model doesn't try to "phrase around" them. Existing `2>/dev/null`
allow-list and benign device passthrough from the previous commit
remain.
- `nanobot/agent/tools/filesystem.py`: append the same boundary note to
the `outside allowed directory` PermissionError so read_file / write_file
/ list_dir errors give the LLM the same explicit hint.
Tests
-----
- `tests/utils/test_workspace_violation_throttle.py` (new): signature
collapses across read_file/exec/python -c against the same path,
different paths get independent budgets, escalation only fires after
the third attempt.
- `tests/agent/test_runner.py`:
- `test_runner_does_not_abort_on_workspace_violation_anymore` -- v2
contract: filesystem PermissionError is now soft, runner moves to
the next iteration and finalizes cleanly.
- `test_is_ssrf_violation_remains_fatal` + the existing
`test_runner_aborts_on_ssrf_violation` -- SSRF still aborts on the
first attempt.
- `test_runner_lets_llm_recover_from_shell_guard_path_outside` -- end
to end recovery from `path outside working dir`.
- `test_runner_throttles_repeated_workspace_bypass_attempts` -- four
bypass attempts against the same outside target produce at least
one `workspace_violation_escalated` event and the run completes
naturally without aborting the turn.
- The two `_execute_tools` direct-call tests now pass the new
workspace_violation_counts dict.
- `tests/tools/test_tool_validation.py`: relax three `==` assertions
to `startswith` + "hard policy boundary" substring check to match
the new structured error messages.
- `tests/tools/test_exec_security.py` keeps the prior `2>/dev/null`
regression and the `> /etc/issue` negative case from the previous
commit on this branch -- they still pass under the new policy.
Coverage status: full pytest 2648 passed / 2 skipped (was 2638 / 2
on origin/main). Ruff is clean for every file touched in this commit.
Co-authored-by: Cursor <cursoragent@cursor.com>
PR #3493 promoted every shell `_guard_command` rejection to a turn-fatal
RuntimeError. The two heuristic outputs in that list -- `path outside
working dir` and `path traversal detected` -- routinely false-positive on
benign constructs (e.g. `2>/dev/null`, quoted `..` arguments to sed/find,
absolute paths inside inline scripts), so legitimate workspace commands
silently kill the user's turn (#3599) and the agent never gets a chance
to retry with a different approach (#3605).
Two changes, both narrowly scoped:
- `ExecTool._guard_command` now skips a small allow-list of kernel device
files (`/dev/null`, the standard streams, `/dev/random`, `/dev/fd/N`,
...) before the workspace path check, matched against the pre-resolve
string so symlinks like `/dev/stderr -> /proc/self/fd/2` still hit the
allow-list. Real outside writes such as `> /etc/issue` remain blocked.
- `AgentRunner._WORKSPACE_BLOCK_MARKERS` keeps only the four hard
path-resolution errors from filesystem.py / shell.py and the SSRF
marker. The two heuristic substrings move out of the fatal list, so
the LLM sees them as ordinary tool errors and can self-correct in the
next iteration. SSRF stays fatal because retrying an internal URL
with a different phrasing would defeat the safety boundary.
Tests:
- `tests/tools/test_exec_security.py`: parametrized regression for the
exact #3599 command sample plus other stdio redirects and device
reads; explicit negative case asserts `> /etc/issue` is still blocked.
- `tests/agent/test_runner.py`: `_is_workspace_violation` no longer
fatals on the two heuristic markers, plus an end-to-end case proving
the runner hands the guard error back to the LLM and finalizes the
next turn cleanly.
Logout previously claimed to support github-copilot in --help text but had
no registered handler, so `provider logout github-copilot` failed with
"Logout not implemented". Add the handler, sharing token deletion with the
codex flow via `_delete_oauth_files`. Tighten handler-table types, fix the
codex test fixture filename, and cover github-copilot plus the unknown
provider path.
- Implement \
anobot provider logout <provider>\ to clear OAuth credentials.
- Add \_LOGOUT_HANDLERS\ registration mechanism mirroring login.
- Implement logout for \openai-codex\ by deleting local \oauth-cli-kit\ token and lock files.
- Fallback gracefully when attempting to logout from providers lacking local credentials or implementations.
- Fixes#2665
