Compare commits

..
Author SHA1 Message Date
chengyongru f1250f0d16 feat(dream): add skill ownership guard with dream_managed frontmatter
- Dream-created skills get  in SKILL.md frontmatter
-  tags each skill as [dream]/[user]/[builtin]
- New config  (default False): when False Dream
  can only modify [dream] skills; when True [user] skills are also editable
- [builtin] skills are never editable
- dream.md prompt uses Jinja conditional to render edit policy
- WebUI types and test fixtures updated with new config field
2026-05-26 00:50:45 +08:00
chengyongru 376d82fec7 fix(agent): prevent runner from exiting while sustained goal is active (cherry-pick) 2026-05-26 00:43:40 +08:00
chengyongru 8dd35c373a feat(dream): single-phase consolidation (cherry-pick from feat/single-phase-dream) 2026-05-26 00:38:04 +08:00
Xubin RenandGitHub 418cb23da2 feat(apps): unify CLI apps and MCP (#3991)
* refactor(cli): load bundled apps from catalog

* feat(plugins): unify CLI and MCP settings

* feat(plugins): add settings category filter

* style(plugins): refine settings catalog

* refactor(cli): load nanobot apps from repo catalog

* feat(store): add capability store entry

* feat(apps): rename capability store

* fix(apps): verify clean app removal

* fix(apps): keep main sidebar on apps view

* feat(apps): add shared app manifest protocol

* fix(apps): dismiss app status message

* refactor(apps): move CLI adapter under apps

* refactor(apps): drop legacy cli apps package
2026-05-25 20:07:02 +08:00
moranandXubin Ren 179acfe104 feat(providers): add Step Plan support
Document how to use StepFun's Step Plan subscription endpoint with the
existing `stepfun` provider by overriding `apiBase`, following the same
pattern as the `zhipu` provider's coding plan documentation.

- **Base URL**: `https://api.stepfun.com/step_plan/v1` (dedicated endpoint)
- **API Key**: same `STEPFUN_API_KEY` as the regular `stepfun` provider
- **Models**: `step-3.5-flash`, `step-3.5-flash-2603`, `step-router-v1`

Changes:
- `docs/configuration.md` — provider tip, and config example showing
  `apiBase` override on the existing `stepfun` provider

Test: 488/488 provider tests passed.
2026-05-25 18:57:36 +08:00
FelixandXubin Ren cfabc29f74 fix(agent): propagate maxConcurrentSubagents config to SubagentManager
The maxConcurrentSubagents field in AgentDefaults was never wired
through AgentLoop.from_config() → AgentLoop.__init__() →
SubagentManager.__init__(), causing it to always fall back to the
hardcoded default of 1 regardless of the user's config.
2026-05-25 16:35:57 +08:00
outlook84andXubin Ren 92f2ff3a33 test: Add test to ensure responses API is used regardless of circuit breaker state 2026-05-25 01:23:36 +08:00
outlook84andXubin Ren c433d60681 feat: Enhance OpenAI provider configuration with extraBody support and apiType validation 2026-05-25 01:23:36 +08:00
outlook84andXubin Ren d472595417 feat: Add OpenAI API type configuration and update provider settings 2026-05-25 01:23:36 +08:00
Xubin Ren 92915ea424 feat(webui): improve slash command actions 2026-05-24 21:24:54 +08:00
Yuxin LouandXubin Ren 3f0098839e fix(provider): preserve OpenAI-compatible tool call ids 2026-05-24 20:53:14 +08:00
Xubin Ren c4e2fcaf0c fix(webui): preserve activity duration on replay 2026-05-24 19:43:20 +08:00
Xubin Ren 8fedee276b fix(webui): auto-collapse completed activity 2026-05-24 19:43:20 +08:00
Xubin Ren 547f81e4aa fix(webui): baseline-align activity diff counts 2026-05-24 19:43:20 +08:00
Xubin Ren 00a6e720dc fix(webui): align inline file references with text 2026-05-24 19:43:20 +08:00
Xubin Ren 6ea7a6a2ac refactor(webui): prune unused legacy components 2026-05-24 19:43:20 +08:00
Xubin Ren 704ac558f6 feat(mcp): add preset setup and capability mentions 2026-05-24 19:43:20 +08:00
Xubin Ren 8be258212e fix(webui): handle final stream image rewrites 2026-05-24 19:43:20 +08:00
Xubin Ren c9ff64fc0f fix(webui): render local CLI image artifacts 2026-05-24 19:43:20 +08:00
Xubin Ren 9efdce276f fix(cli): refresh installed apps after settings changes 2026-05-24 19:43:20 +08:00
04cbandXubin Ren 7a6cc657db feat(spawn): allow per-subagent sampling temperature (#3969) 2026-05-24 13:54:37 +08:00
Xubin Ren ec99232208 docs: fix Xiaomi MiMo token plan env key 2026-05-23 22:56:24 +08:00
honjiaxuanandXubin Ren 43a1784c5f docs: use xiaomi_mimo provider for MiMo token plan
Replace standalone 'Token Plan' section with general Xiaomi MiMo
section using the built-in xiaomi_mimo provider. Token plan becomes
a note within the section, since it's just an apiBase override.

Key changes:
- Use xiaomi_mimo provider (auto-matches via 'mimo' keyword in model name)
- Drop redundant provider field (auto-detected)
- Add token plan tip to provider tips block
- Restructure as general Xiaomi MiMo section with token plan as note
2026-05-23 22:56:24 +08:00
Xubin Ren 3d3ef586e7 docs(config): clarify exec timeout and transcription apiBase 2026-05-23 17:32:59 +08:00
04cbandXubin Ren ef2ef4f789 fix(transcription): normalize chat-style apiBase to audio endpoint (#3637) 2026-05-23 17:32:59 +08:00
04cbandXubin Ren 5b71f61f55 fix(exec): uncap config exec timeout; 0 means no limit (#3595) 2026-05-23 17:32:59 +08:00
Xubin Ren 5937236f9d test(image-generation): tighten zhipu provider coverage 2026-05-23 17:06:36 +08:00
Hermes AgentandXubin Ren 192d2af19d fix(zhipu): raise error on reference images and ensure client cleanup in finally 2026-05-23 17:06:36 +08:00
Jiajun XieandXubin Ren 3e6f9907fe feat: Add Zhipu (智谱) image generation provider 2026-05-23 17:06:36 +08:00
Xubin Ren c0d4f012c8 test(cli): cover CLI Apps on Windows CI 2026-05-23 00:47:28 +08:00
Xubin Ren e2d00ffc8f feat: add CLI Apps settings MVP 2026-05-23 00:33:31 +08:00
Xubin Ren a5a956d9af fix(webui): preserve localized chat show-more copy 2026-05-23 00:01:52 +08:00
Stellar鱼andXubin Ren 8c5acea3b0 chore: fill remaining webui locale keys 2026-05-23 00:01:52 +08:00
Xubin Ren 545294c62c fix(web): keep safe fetch preflight streaming 2026-05-22 23:10:13 +08:00
hinotoi-agentandXubin Ren 25d00b1ea4 fix(web): support redirect handling in fake responses 2026-05-22 23:10:13 +08:00
hinotoi-agentandXubin Ren ff173045fe fix(web): validate redirect targets before fetching 2026-05-22 23:10:13 +08:00
yu-xin-candXubin Ren b1140f6aee chore: fill zh-TW and ja locale keys 2026-05-22 22:38:34 +08:00
Xubin RenandGitHub 782d761b81 Merge PR #3929: Unify image provider HTTP handling and document Gemini image base URLs
Unify image provider HTTP handling and document Gemini image base URLs
2026-05-22 22:31:27 +08:00
Xubin Ren c1073f2986 fix(image-generation): keep image presence helper stable 2026-05-22 22:19:32 +08:00
Xubin Ren 143224e25a Merge remote-tracking branch 'origin/main' into codex/review-pr-3929 2026-05-22 22:15:46 +08:00
Yuxin LouandXubin Ren 055c9be359 fix: dedupe Responses replay item ids
Ensure converted Responses API input items use unique replay ids when restoring assistant messages and function calls. This prevents Codex from rejecting resumed conversations with duplicate rs_* item ids while preserving call_id-based tool result linkage.
2026-05-22 22:14:07 +08:00
Xubin RenandGitHub ddfe5c3bdf Merge PR #3946: Add Ollama image generation support
Add Ollama image generation support
2026-05-22 22:06:28 +08:00
Xubin Ren f5534bcaa0 Merge origin/main into fix-ollama-image-generation 2026-05-22 21:15:42 +08:00
Xubin Ren 8c0b2c1a29 fix(image-generation): clamp OpenAI sizes by model family 2026-05-22 17:42:01 +08:00
ZegWeandXubin Ren ffd85a8611 fix image generation provider settings 2026-05-22 17:42:01 +08:00
ZegWeandXubin Ren 65dff4f3a5 fix(providers): preserve codex text deltas 2026-05-22 17:42:01 +08:00
3483141ed7 feat(providers): add OpenAI and OpenAI Codex image generation providers
Add two new image generation providers:

- `openai` — uses the standalone OpenAI Images API
  (`/v1/images/generations`) with an API key. Supports DALL-E
  and gpt-image-* models, with automatic parameter adjustment
  (gpt-image models don't accept response_format or n).

- `openai_codex` — uses the Codex Responses API with the
  `image_generation` tool, authenticated via OAuth subscription
  token. The same mechanism ChatGPT uses internally.

Also remove the API key pre-check in ImageGenerationTool so
providers that handle their own auth fallback (like Codex OAuth)
can work without a configured key.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:42:01 +08:00
Xubin Ren b0d3069621 fix(apply-patch): tighten edits-only boundaries 2026-05-22 17:25:45 +08:00
chengyongruandXubin Ren 3d9f50a0cc refactor(apply_patch): remove deprecated patch mode, keep edits-only
Drop the legacy unified-diff patch parameter and all related parsing/
generation logic (_parse_patch, _generate_patch, _apply_hunks, etc.).
The tool now accepts only the structured `edits` array, eliminating the
intermediate diff-string round-trip.

Also update file_edit_events tracking and tests to work exclusively
with edits.

Benchmark (zhipu glm-5.1, edits mode): 15/15 cases passed.
2026-05-22 17:25:45 +08:00
Xubin Ren effc1efd92 fix(webui): avoid misleading file edit counters 2026-05-22 13:58:09 +08:00
A.G. BocsardiandXubin Ren 9b2f452b6e fix: drop redundant reasoning_effort for Kimi thinking models
Moonshot's API rejects requests that carry both 'reasoning_effort'
(top-level kwarg) and 'thinking' (extra_body) at the same time.
After the unified thinking-style injection loop injects the native
'thinking' param for kimi models, pop 'reasoning_effort' from kwargs
since it is redundant and causes a 400 error.

Uses _model_slug() + _KIMI_THINKING_MODELS lookup to stay consistent
with the refactored code (the old _is_kimi_thinking_model helper was
removed in 4f895e63).

Existing kimi tests updated to assert 'reasoning_effort' is absent.
Xiaomi MiMo models are unaffected — their API accepts both params.

Closes #3939
2026-05-22 03:36:28 +08:00
Xubin Ren d660573b18 feat(webui): improve sidebar performance 2026-05-22 03:35:20 +08:00
Xubin Ren cb7daa77db feat(webui): refine collapsible sidebar 2026-05-22 00:34:42 +08:00
Xubin Ren 8281cd1946 test(providers): cover Novita gateway fallback 2026-05-21 16:16:32 +08:00
Alex-wuhuandXubin Ren e5476573f4 test(providers): align Novita provider coverage 2026-05-21 16:16:32 +08:00
Alex-wuhuandXubin Ren 0d1d23b5fb feat: add Novita AI provider 2026-05-21 16:16:32 +08:00
Xubin Ren 835bab5f5a fix(exec): stabilize Windows shell tests 2026-05-21 16:10:09 +08:00
Xubin RenandGitHub ccbc0bb6e3 Merge PR #3923: feat(tools): optimize coding workflows
feat(tools): optimize coding workflows
2026-05-21 15:55:13 +08:00
Xubin Ren 722b760eae feat(webui): stream apply patch edit progress 2026-05-21 15:44:01 +08:00
Xubin Ren 23d5148a57 fix(provider): dedupe repeated tool ids in history 2026-05-21 15:33:49 +08:00
Xubin Ren d29fcaf5d1 refactor(agent): internalize tool contract prompt 2026-05-21 15:21:39 +08:00
Haisam Abbas 84603f4cf2 Add Ollama image generation support 2026-05-21 12:06:08 +05:00
Xubin Ren 581faa34f7 Merge remote-tracking branch 'origin/main' into codex/coding-tooling-optimization 2026-05-21 14:44:56 +08:00
Xubin Ren 7e3af8c38b docs(tools): add general tool workflow contract 2026-05-21 14:44:34 +08:00
Haisam AbbasandXubin Ren e645fbcb34 fix shell guard url path detection 2026-05-21 14:42:11 +08:00
Xubin Ren 4f895e6307 refactor(providers): centralize gateway reasoning control 2026-05-21 14:41:50 +08:00
olgagagaandXubin Ren 0cd2f626c0 fix(providers): inject OpenRouter reasoning.effort for thinking models
Follow-up to #3851: that PR added `extra_body.thinking={type: disabled}`
for MiMo via OpenRouter, but OR doesn't forward provider-specific
thinking shapes to upstream — it strips unknown extra_body fields and
uses its own unified `reasoning` parameter. So MiMo via OR kept
thinking despite the injection (reproduced by @ClearPlume on #3851
with identical kwargs but provider switched from openrouter → xiaomi_mimo).

For known thinking-capable models (Kimi, MiMo) routed via the
openrouter spec, also inject `extra_body.reasoning = {effort: <effort>}`
in OR's documented enum ("none"|"minimal"|"low"|"medium"|"high"|"xhigh").
OR translates this to the upstream model's native shape.

Existing tests updated to expect both fields on the OR path. The direct
xiaomi_mimo and moonshot paths are unchanged (the new branch is gated
on spec.name == "openrouter"). Flash and non-MiMo models on OR continue
to receive no injection.
2026-05-21 14:41:50 +08:00
Xubin Ren 44ef697aac docs(tools): clarify coding tool guidance 2026-05-21 14:28:39 +08:00
chengyongruandXubin Ren e2b51fa5dc fix(weixin): prevent silent message drops from poll exceptions and expired tokens
- Remove suppress(Exception) from poll loop and message processing; add
  logger.exception so inbound errors are visible.
- Check both ret and errcode on send to avoid silent drops when iLink
  returns ret != 0 with errcode == 0.
- Proactively refresh context_token via getconfig before sending if the
  cached token is older than 60s. This prevents message loss on long
  agent turns and cron pushes without relying on complex retry logic.

Refs: openclaw/openclaw#61174, NousResearch/hermes-agent#21011
2026-05-21 13:41:05 +08:00
Xubin Ren 7e122d6e49 chore(tools): merge main and resolve conflicts 2026-05-21 12:53:42 +08:00
hanyuanlingandXubin Ren de0a8f5e41 fix(webui): keep new chat during session refresh 2026-05-21 12:42:56 +08:00
Xubin Ren 3d3ebf1110 test(provider): cover duplicate streaming tool call ids 2026-05-21 12:28:24 +08:00
chengyongruandXubin Ren 77ec55bf8e fix(provider): deduplicate streaming tool_call_ids for parallel calls 2026-05-21 12:28:24 +08:00
Xubin Ren 8141df0d3f fix(tools): stabilize session output test 2026-05-21 01:32:27 +08:00
Xubin Ren 5f0ba05de5 feat(tools): tighten patch and session workflows 2026-05-21 01:25:20 +08:00
chengyongruandXubin Ren 886e7e43d5 fix(signal): bypass base is_allowed for policy-approved messages
Override _handle_message to publish directly to the bus for messages
that have already passed _check_inbound_policy. The denied DM pairing
path calls super()._handle_message() to issue pairing codes via the
base class. This avoids cross-policy leakage where e.g. group open
policy would cause is_allowed to incorrectly allow denied DM senders.

Also includes:
- SSE: strip one optional leading space after 'data:' per spec
- Convert 20+ f-string log calls to loguru lazy formatting
- Add end-to-end tests for DM/group routing through the full chain
- Add cross-policy test (dm allowlist + group open) for pairing
- Add Signal channel documentation to docs/chat-apps.md
2026-05-21 01:00:36 +08:00
Kaloyan TenchovandXubin Ren b3d0d24a52 fix(signal): consult pairing store in is_allowed
BaseChannel.is_allowed ORs is_approved (the pairing store) into the
allow decision; the signal override dropped that step and only looked
at config.allow_from. With the new DM-pairing flow in place, an
approved-via-pairing sender's next message would have failed the
allow check and triggered another pairing code in a loop.

OR in a normalized check against the pairing store: walk each part of
the pipe-joined sender_id through _normalize_signal_id and call
is_approved for each variant, so an approval stored under one form
(phone with/without "+", UUID/ACI) still matches when the next inbound
uses a different form. Mirrors how slack.py:643 handles it.

Also tightens the empty-allowlist warning to only fire when nothing
else granted access, since pairing-store hits are now a valid path.

Not part of the original review, but Comments 2 and 3 turn this latent
gap into a broken round-trip — included so the pairing UX actually
works.
2026-05-21 01:00:36 +08:00
Kaloyan TenchovandXubin Ren 82dfe8c1f7 fix(signal): join multi-line SSE data with newline per spec
Per the SSE spec, multiple data: lines within a single event must be
joined with \n before parsing. signal-cli emits single-line JSON so
this was latent, but the joining was wrong.

Addresses review comment on PR #3852.
2026-05-21 01:00:36 +08:00
Kaloyan TenchovandXubin Ren dc33247671 fix(signal): route denied DMs through _handle_message for pairing code
Previously _check_inbound_policy returned (False, chat_id) for DMs
that failed the allowlist and the caller dropped them — so unapproved
DM senders never saw a pairing code. Mirror Slack: when the policy
gate denies a DM but dm.enabled is true, still call
_handle_message(content="", is_dm=True) so BaseChannel can issue the
pairing reply. Group denials stay a hard drop.

Combined with the previous is_dm forwarding, unapproved DM senders
now receive a pairing code through the standard flow.

Addresses review comment on PR #3852.
2026-05-21 01:00:36 +08:00
Kaloyan TenchovandXubin Ren d376ec129d fix(signal): pass is_dm to _handle_message so DM pairing flow runs
BaseChannel._handle_message uses is_dm to decide whether to issue a
pairing code when is_allowed rejects the sender. Without it the base
class treats every denied message as a group message and silently
drops it. Forward is_dm=not is_group_message so unapproved DM users
get a pairing code through the standard flow.

This change only takes effect once denied DMs actually reach
_handle_message (next commit); on its own it is a no-op since the
policy gate still short-circuits before this call.

Addresses review comment on PR #3852.
2026-05-21 01:00:36 +08:00
Kaloyan TenchovandXubin Ren d653f23aba fix(signal): raise on signal-cli error response so send is retriable
_send_http_request collapses every exception path into a {"error": ...}
dict, so the if "error" in response branch inside send() is the only
place where send failures surface. Logging-only there meant the
ChannelManager retry mechanism never fired. Raise RuntimeError so the
base-class retry path is exercised; the outer try/except already
re-raises into the caller.

Addresses review comment on PR #3852.
2026-05-21 01:00:36 +08:00
Kaloyan TenchovandXubin Ren 96767ca179 Cleanup 2026-05-21 01:00:36 +08:00
b300ea495f fix(signal): normalize composite sender_ids in is_allowed too
The base BaseChannel.is_allowed() does a literal ``sender_id in allow_from``
check, but Signal's sender_id is a pipe-joined composite of phone/UUID
parts. After splitting an allowlist entry like ``+phone|uuid`` into two
separate entries, the per-DM gate accepted it but the base gate still
denied because the composite sender string wasn't literally in the list.

Override is_allowed on SignalChannel to delegate to
_sender_matches_allowlist, which already splits both sides on ``|`` and
normalizes each part. _sender_matches_allowlist itself now also splits
allowlist entries on ``|`` so legacy composite entries keep working too.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:00:36 +08:00
632f41e418 test(signal): cover markdown adjacency, nesting, and malformed input
The existing markdown suite was strong on UTF-16 offsets and chunk
redistribution but had no coverage for nested or adjacent styles, no test
that an unmatched opener round-trips as plain text, and no test for the
blockquote/inline-code interaction. Add six cases including the
documented contiguous-BOLD output for `# **wrap** me`, which Signal
renders as one visual span.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:00:36 +08:00
9c486b90d5 test(signal): consolidate channel-capture setup into one factory
Two test classes (TestHandleDataMessageDM, TestHandleDataMessageGroup)
plus three TestCommandHandling tests each repeated the same handful of
lines: build a channel, mock _handle_message to record kwargs, replace
_start_typing with a no-op, paper over the assignment with type: ignore.

Hoist the pattern into _make_channel_with_capture and call it from all
five sites. Drops 30+ lines of duplication and 7 type: ignore comments.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:00:36 +08:00
590ac99c8a test(signal): cover SSE receive loop and the empty-phone start guard
Previously the SSE loop and the empty-phone-number short-circuit in start()
had zero coverage. Both now have tests: a fake httpx stream feeds canned
SSE lines, exercising the valid-frame, invalid-JSON, non-200, and
no-http-client paths; start() with an empty phone number is asserted to
return without entering the HTTP loop.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:00:36 +08:00
7733a7840e refactor(signal): split _handle_data_message into policy and assembly helpers
The receive-path handler was ~165 lines deep into nested DM/group policy
checks, buffer mutations, mention stripping, attachment downloads, and
final bus forwarding. Pull the policy gate out into _check_inbound_policy
(returns (allow, chat_id), still appends to the group buffer once allowed)
and the text+media construction into _assemble_inbound_content. The
top-level method now reads as orchestration only.

Add TestCheckInboundPolicy that exercises the helper directly across the
DM/group policy permutations, including the buffer side effect, so the
new seam is locked in.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:00:36 +08:00
83aed43682 feat(signal): make signal-cli attachments directory configurable
The inbound attachment loop hardcoded ~/.local/share/signal-cli/attachments
as the source path. That is the daemon's default on Linux but not on macOS
or Windows, and breaks if the daemon was launched with XDG_DATA_HOME set.

Add SignalConfig.attachments_dir as an optional override. When unset the
behavior is unchanged; when set the value is run through Path.expanduser()
so ~ is honored.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:00:36 +08:00
ad7c1ac381 refactor(signal): wrap top-level receive handler with _safe_handle
Replace the inline try/except at the end of _handle_receive_notification
with a small async context manager that swallows the exception, logs
self.logger.error with the offending payload's repr (bounded to 200 chars),
and attaches the traceback via logger.opt(exception=True).

The previous log line only carried `e`, so diagnosing a bad envelope from
production logs required correlating timestamps. The wrapper is generic so
future receive/dispatch sites can adopt it; for now only this site uses it.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:00:36 +08:00
882d4139d7 fix(signal): normalize identifiers when matching DM allowlist
The DM allowlist check split sender_id on '|' and looked for raw membership
in the allow_from list. Senders carry their phone number with a leading
'+' but admins routinely write allowlist entries without it (or vice
versa), and UUID/ACI matches were case-sensitive. Both forms now flow
through _normalize_signal_id, so an entry like 19995550001 matches a
sender +19995550001 and a UUID matches case-insensitively.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:00:36 +08:00
ca72f6b6c9 refactor(signal): hygiene cleanups around constants, typing, and config
- Hoist the cell-strip patterns to module level so they match the rest of
  the module's regex style and aren't reparsed on every call.
- Type the markdown transform callback and the mention id walker so the
  inline Callable signature is no longer an untyped Any.
- Add _HTTP_TIMEOUT_SECONDS alongside the other class-level tunables.
- Reject group_message_buffer_size <= 0 in a Pydantic field_validator
  rather than silently disabling the buffer at write time.
- Mark SignalConfig.allow_from as a computed_field so it shows up in
  model_dump() instead of being invisible to serialization.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:00:36 +08:00
96eb3b7194 fix(signal): redistribute textStyle ranges across split message chunks
split_message can break a long Signal payload into multiple JSON-RPC sends,
but the previous code attached the full textStyle list only to chunk 0.
Style ranges in later chunks were dropped, and ranges whose offsets pointed
past chunk 0's end were sent as invalid metadata against chunk 0.

Add _partition_styles, which rebases each range against the chunk it lives
in (in UTF-16 code units, matching the markdown converter) and splits
boundary-spanning ranges across the chunks they touch. Whitespace trimmed
by split_message's lstrip is skipped so offsets stay aligned.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:00:36 +08:00
8f6b7611a2 fix(signal): emit textStyle offsets in UTF-16 code units
Signal's BodyRange (via signal-cli's textStyle) interprets start/length as
UTF-16 code units, but the Phase-3 assembly used Python's len(), which counts
code points. A single non-BMP character (e.g. an emoji) earlier in a message
shifted every subsequent styled span left by one unit, dropping the last
letter of bold/italic words.

Track a running UTF-16 offset in the assembly loop and add regression tests
covering emojis, supplementary CJK, ZWJ sequences, and a multi-section
message that mirrors the reported failure.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 01:00:36 +08:00
Kaloyan TenchovandXubin Ren 1a6fe093e7 fix(signal): drop duplicate self in unconfigured-account log call
Addresses review feedback on HKUDS/nanobot#3852: self.self.logger.error
would crash if the phone_number guard ever fired.
2026-05-21 01:00:36 +08:00
Kaloyan TenchovandXubin Ren 8ec1025193 feat(signal): add Signal channel support
Integrates signal-cli daemon via HTTP JSON-RPC as a nanobot channel.
Supports DMs and group chats with open/allowlist access policies,
markdown→Signal text style conversion, typing indicators, attachment
handling, group message context buffering, and automatic reconnect
with exponential backoff.

Includes unit tests for channel lifecycle, message routing, mention
detection, markdown conversion, and message splitting.

Originally based on https://github.com/HKUDS/nanobot/pull/601.
2026-05-21 01:00:36 +08:00
Xubin Ren 480ca28a2d feat(tools): improve coding workflow recovery 2026-05-21 00:58:05 +08:00
Xubin Ren 3e154bb5cf fix(tools): align exec platform test doubles 2026-05-20 23:42:55 +08:00
Xubin Ren 6851fa57a6 feat(tools): optimize coding workflows 2026-05-20 23:08:21 +08:00
chengyongruandXubin Ren 09a692be6f docs(readme): add multi-language doc site links
Link nanobot.wiki documentation in 10 languages from README header:
English, 简体中文, 繁體中文, Español, Français, Bahasa Indonesia,
日本語, 한국어, Русский, Tiếng Việt.
2026-05-20 22:37:11 +08:00
Haisam Abbas 3f789bd9f9 Revert "fix shell guard url path detection"
This reverts commit 65cecc01fb.
2026-05-20 17:21:34 +05:00
Haisam Abbas 65cecc01fb fix shell guard url path detection 2026-05-20 17:16:53 +05:00
Haisam Abbas a7b34422f3 fix Gemini image base and provider docs 2026-05-20 14:06:55 +05:00
Haisam Abbas 72f999f8f7 refactor image provider HTTP handling 2026-05-20 13:56:43 +05:00
Haisam Abbas e6587a8d8e Fix image mime detection for MiniMax 2026-05-20 12:18:18 +05:00
176 changed files with 27756 additions and 4152 deletions
+1 -1
View File
@@ -20,7 +20,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: ${{ github.event_name == 'pull_request' && fromJSON('["ubuntu-latest"]') || fromJSON('["ubuntu-latest","windows-latest"]') }}
os: ${{ fromJSON('["ubuntu-latest","windows-latest"]') }}
# CI concentrates on newer runtimes (3.11/3.12 still supported per pyproject requires-python).
python-version: ${{ fromJSON('["3.13","3.14"]') }}
+1
View File
@@ -98,3 +98,4 @@ tmp/
temp/
*.tmp
exp/
.playwright-mcp/
+13 -1
View File
@@ -1,6 +1,18 @@
![cover-v5-optimized](./images/GitHub_README.png)
<div align="center">
<p>
<a href="https://nanobot.wiki/docs/latest/getting-started/nanobot-overview">English</a> |
<a href="https://nanobot.wiki/cn/docs/latest/getting-started/nanobot-overview">简体中文</a> |
<a href="https://nanobot.wiki/zh-Hant/docs/latest/getting-started/nanobot-overview">繁體中文</a> |
<a href="https://nanobot.wiki/es/docs/latest/getting-started/nanobot-overview">Español</a> |
<a href="https://nanobot.wiki/fr/docs/latest/getting-started/nanobot-overview">Français</a> |
<a href="https://nanobot.wiki/id/docs/latest/getting-started/nanobot-overview">Bahasa Indonesia</a> |
<a href="https://nanobot.wiki/ja/docs/latest/getting-started/nanobot-overview">日本語</a> |
<a href="https://nanobot.wiki/ko/docs/latest/getting-started/nanobot-overview">한국어</a> |
<a href="https://nanobot.wiki/ru/docs/latest/getting-started/nanobot-overview">Русский</a> |
<a href="https://nanobot.wiki/vi/docs/latest/getting-started/nanobot-overview">Tiếng Việt</a>
</p>
<p>
<a href="https://pypi.org/project/nanobot-ai/"><img src="https://img.shields.io/pypi/v/nanobot-ai" alt="PyPI"></a>
<a href="https://pepy.tech/project/nanobot-ai"><img src="https://static.pepy.tech/badge/nanobot-ai" alt="Downloads"></a>
@@ -61,7 +73,7 @@
- **2026-04-13** 🛡️ Agent turn hardened — user messages persisted early, auto-compact skips active tasks.
- **2026-04-12** 🔒 Lark global domain support, Dream learns discovered skills, shell sandbox tightened.
- **2026-04-11** ⚡ Context compact shrinks sessions on the fly; Kagi web search; QQ & WeCom full media.
- **2026-04-10** 📓 Notebook editing tool, multiple MCP servers, Feishu streaming & done-emoji.
- **2026-04-10** 📓 Multiple MCP servers, Feishu streaming & done-emoji.
- **2026-04-09** 🔌 WebSocket channel, unified cross-channel session, `disabled_skills` config.
- **2026-04-08** 📤 API file uploads, OpenAI reasoning auto-routing with Responses fallback.
- **2026-04-07** 🧠 Anthropic adaptive thinking, MCP resources & prompts exposed as tools.
+67
View File
@@ -17,6 +17,7 @@ Connect nanobot to your favorite chat platform. Want to build your own? See the
| **Wecom** | Bot ID + Bot Secret |
| **Microsoft Teams** | App ID + App Password + public HTTPS endpoint |
| **Mochat** | Claw token (auto-setup available) |
| **Signal** | signal-cli daemon + phone number |
<details>
<summary><b>Telegram</b> (Recommended)</summary>
@@ -669,3 +670,69 @@ nanobot gateway
```
</details>
<details>
<summary><b>Signal</b></summary>
Uses **signal-cli** daemon in HTTP mode — receive messages via SSE, send via JSON-RPC.
**1. Install signal-cli**
Install [signal-cli](https://github.com/AsamK/signal-cli) and register a phone number:
```bash
signal-cli -u +1234567890 register
signal-cli -u +1234567890 verify <CODE>
```
Start the daemon:
```bash
signal-cli -a +1234567890 daemon --http localhost:8080
```
**2. Configure**
```json
{
"channels": {
"signal": {
"enabled": true,
"phoneNumber": "+1234567890",
"daemonHost": "localhost",
"daemonPort": 8080,
"dm": {
"enabled": true,
"policy": "open"
},
"group": {
"enabled": true,
"policy": "open",
"requireMention": true
}
}
}
}
```
> - `phoneNumber`: Your registered Signal phone number.
> - `daemonHost` / `daemonPort`: Where signal-cli daemon is listening (default `localhost:8080`).
> - `dm.policy`: `"open"` (anyone can DM) or `"allowlist"` (only listed numbers/UUIDs). When `"allowlist"`, unlisted DM senders receive a pairing code.
> - `dm.allowFrom`: List of allowed phone numbers or UUIDs (used when policy is `"allowlist"`).
> - `group.policy`: `"open"` (all groups) or `"allowlist"` (only listed group IDs).
> - `group.requireMention`: When `true` (default), the bot only responds in groups when @mentioned.
> - `group.allowFrom`: List of allowed group IDs (used when group policy is `"allowlist"`).
> - `attachmentsDir`: Override the directory where signal-cli stores inbound attachments. Defaults to `~/.local/share/signal-cli/attachments` (the Linux default). Set this if signal-cli runs with a custom `XDG_DATA_HOME` or on macOS/Windows.
> - `groupMessageBufferSize`: Number of recent group messages kept for context (default `20`, must be > 0).
**3. Run**
```bash
nanobot gateway
```
> [!TIP]
> The channel automatically reconnects to the signal-cli daemon with exponential backoff if the connection drops.
> Markdown in bot replies is automatically converted to Signal text styles (bold, italic, code, etc.).
</details>
+105 -2
View File
@@ -126,8 +126,10 @@ ANTHROPIC_API_KEY="$(bw get password api/anthropic)" nanobot agent
> - **VolcEngine / BytePlus Coding Plan**: Use dedicated providers `volcengineCodingPlan` or `byteplusCodingPlan` instead of the pay-per-use `volcengine` / `byteplus` providers.
> - **Zhipu Coding Plan**: If you're on Zhipu's coding plan, set `"apiBase": "https://open.bigmodel.cn/api/coding/paas/v4"` in your zhipu provider config.
> - **Alibaba Cloud BaiLian**: If you're using Alibaba Cloud BaiLian's OpenAI-compatible endpoint, set `"apiBase": "https://dashscope.aliyuncs.com/compatible-mode/v1"` in your dashscope provider config.
> - **StepFun Step Plan**: If you're on StepFun's Step Plan subscription, set `"apiBase": "https://api.stepfun.com/step_plan/v1"` in your stepfun provider config. Supported models include `step-3.5-flash`, `step-3.5-flash-2603`, and `step-router-v1`.
> - **Step Fun (Mainland China)**: If your API key is from Step Fun's mainland China platform (stepfun.com), set `"apiBase": "https://api.stepfun.com/v1"` in your stepfun provider config.
> - **Xiaomi MiMo thinking mode**: MiMo models (e.g. `mimo-v2.5-pro`) default to enabled thinking. Use `agents.defaults.reasoningEffort: "none"` to disable it, or `"low"` / `"medium"` / `"high"` to keep it on. Omitting the field preserves the provider's per-model default.
> - **Xiaomi MiMo Token Plan**: If you're on MiMo's token plan, set `"apiBase": "https://token-plan-sgp.xiaomimimo.com/v1"` in your xiaomi_mimo provider config.
| Provider | Purpose | Get API Key |
|----------|---------|-------------|
@@ -148,6 +150,7 @@ ANTHROPIC_API_KEY="$(bw get password api/anthropic)" nanobot agent
| `gemini` | LLM (Gemini direct) | [aistudio.google.com](https://aistudio.google.com) |
| `aihubmix` | LLM (API gateway, access to all models) | [aihubmix.com](https://aihubmix.com) |
| `siliconflow` | LLM (SiliconFlow/硅基流动) | [siliconflow.cn](https://siliconflow.cn) |
| `novita` | LLM (Novita AI OpenAI-compatible gateway) | [novita.ai](https://novita.ai) |
| `dashscope` | LLM (Qwen) | [dashscope.console.aliyun.com](https://dashscope.console.aliyun.com) |
| `moonshot` | LLM (Moonshot/Kimi) | [platform.moonshot.cn](https://platform.moonshot.cn) |
| `zhipu` | LLM (Zhipu GLM) | [open.bigmodel.cn](https://open.bigmodel.cn) |
@@ -165,6 +168,43 @@ ANTHROPIC_API_KEY="$(bw get password api/anthropic)" nanobot agent
| `github_copilot` | LLM (GitHub Copilot, OAuth) | `nanobot provider login github-copilot` |
| `qianfan` | LLM (Baidu Qianfan) | [cloud.baidu.com](https://cloud.baidu.com/doc/qianfan/s/Hmh4suq26) |
<details>
<summary><b>OpenAI</b></summary>
By default, OpenAI uses `apiType: "auto"`: nanobot calls Chat Completions normally and routes GPT-5/o-series or explicit `reasoningEffort` requests through the Responses API when useful. You can force a specific API surface:
```json
{
"providers": {
"openai": {
"apiKey": "${OPENAI_API_KEY}",
"apiType": "chat_completions"
}
}
}
```
Valid `apiType` values are exactly `auto`, `chat_completions`, and `responses`.
`extraBody` follows the selected OpenAI API surface. With Chat Completions, nanobot passes it through as the SDK `extra_body` value. With Responses, configure it in Responses API body shape; nanobot merges ordinary top-level fields into the Responses request body, appends `extraBody.tools` after generated function tools, and merges `extraBody.include` without duplicates:
```json
{
"providers": {
"openai": {
"apiKey": "${OPENAI_API_KEY}",
"apiType": "responses",
"extraBody": {
"tools": [{ "type": "web_search" }],
"include": ["web_search_call.action.sources"]
}
}
}
}
```
</details>
<details>
<summary><b>Skywork / APIFree</b></summary>
@@ -476,6 +516,68 @@ Official model names include `LongCat-Flash-Chat`, `LongCat-Flash-Thinking`,
</details>
<details>
<summary><b>Xiaomi MiMo</b></summary>
Xiaomi MiMo models are automatically detected by the `xiaomi_mimo` provider when
the model name contains `mimo`. The default API base is
`https://api.xiaomimimo.com/v1`.
> **Token Plan**: If you're using MiMo's token plan, override `apiBase` with the
> dedicated endpoint:
>
> ```json
> {
> "providers": {
> "xiaomi_mimo": {
> "apiKey": "${XIAOMIMIMO_API_KEY}",
> "apiBase": "https://token-plan-sgp.xiaomimimo.com/v1"
> }
> },
> "agents": {
> "defaults": {
> "model": "xiaomi/mimo-v2.5-pro"
> }
> }
> }
> ```
>
> No need to set `provider` explicitly — the model name contains `mimo`, which
> auto-matches to the `xiaomi_mimo` provider spec. Use an API key from the MiMo
> token plan console and check the MiMo platform for the latest supported model
> names.
</details>
<details>
<summary><b>StepFun Step Plan (subscription)</b></summary>
Step Plan is StepFun's subscription-based service for high-frequency AI developers.
If you're on a Step Plan subscription, override `apiBase` in the existing `stepfun`
provider config to point to the dedicated Step Plan endpoint.
```json
{
"providers": {
"stepfun": {
"apiKey": "${STEPFUN_API_KEY}",
"apiBase": "https://api.stepfun.com/step_plan/v1"
}
},
"agents": {
"defaults": {
"provider": "stepfun",
"model": "step-3.5-flash"
}
}
}
```
Supported models include `step-3.5-flash`, `step-3.5-flash-2603`, and
`step-router-v1`.
</details>
<details>
<summary><b>Ant Ling (OpenAI-compatible)</b></summary>
@@ -955,7 +1057,7 @@ Global settings that apply to all channels. Configure under the `channels` secti
| `sendToolHints` | `false` | Stream tool-call hints (e.g. `read_file("…")`) |
| `showReasoning` | `true` | Allow channels to surface model reasoning/thinking content (DeepSeek-R1 `reasoning_content`, Anthropic `thinking_blocks`, inline `<think>` tags). Reasoning flows as a dedicated stream with `_reasoning_delta` / `_reasoning_end` markers — channels override `send_reasoning_delta` / `send_reasoning_end` to render in-place updates. Even with `true`, channels without those overrides stay no-op silently. Currently surfaced on CLI and WebSocket/WebUI (italic shimmer header, auto-collapses after the stream ends); Telegram / Slack / Discord / Feishu / WeChat / Matrix keep the base no-op until their bubble UI is adapted. Independent of `sendProgress`. |
| `sendMaxRetries` | `3` | Max delivery attempts per outbound message, including the initial send (0-10 configured, minimum 1 actual attempt) |
| `transcriptionProvider` | `"groq"` | Voice transcription backend: `"groq"` (free tier, default) or `"openai"`. API key is auto-resolved from the matching provider config. |
| `transcriptionProvider` | `"groq"` | Voice transcription backend: `"groq"` (free tier, default) or `"openai"`. API key and optional `apiBase` are auto-resolved from the matching provider config. Chat-style bases such as `https://api.groq.com/openai/v1` are normalized to the audio transcription endpoint. |
| `transcriptionLanguage` | `null` | Optional ISO-639-1 language hint for audio transcription, e.g. `"en"`, `"ko"`, `"ja"`. |
`sendProgress` and `sendToolHints` can also be overridden per channel. The
@@ -1194,7 +1296,7 @@ If you want to always use the local conversion, you can force it using:
## Image Generation
Image generation is configured under `tools.imageGeneration` and uses provider credentials from `providers.openrouter` or `providers.aihubmix`.
Image generation is configured under `tools.imageGeneration` and uses credentials from the selected provider's `providers.<name>` block.
See [Image Generation](./image-generation.md) for WebUI usage, provider examples, artifact storage, and troubleshooting.
@@ -1287,6 +1389,7 @@ For API keys, tokens, and other secrets, see [Environment Variables for Secrets]
| `tools.restrictToWorkspace` | `false` | When `true`, restricts **all** agent tools (shell, file read/write/edit, list) to the workspace directory. Prevents path traversal and out-of-scope access. |
| `tools.exec.sandbox` | `""` | Sandbox backend for shell commands. Set to `"bwrap"` to wrap exec calls in a [bubblewrap](https://github.com/containers/bubblewrap) sandbox — the process can only see the workspace (read-write) and media directory (read-only); config files and API keys are hidden. Automatically enables `restrictToWorkspace` for file tools. **Linux only** — requires `bwrap` installed (`apt install bubblewrap`; pre-installed in the Docker image). Not available on macOS or Windows (bwrap depends on Linux kernel namespaces). |
| `tools.exec.enable` | `true` | When `false`, the shell `exec` tool is not registered at all. Use this to completely disable shell command execution. |
| `tools.exec.timeout` | `60` | Default hard timeout in seconds for shell commands. Config values may exceed the per-call tool cap; set `0` to disable the hard timeout for trusted long-running commands. |
| `tools.exec.pathAppend` | `""` | Extra directories to append to `PATH` when running shell commands (e.g. `/usr/sbin` for `ufw`). |
| `channels.*.allowFrom` | omitted | Access control per channel. Omit to use pairing-only mode; set `["*"]` to allow everyone; or list specific user IDs. See [Pairing](#pairing) for details. |
+53 -4
View File
@@ -23,7 +23,7 @@ The feature is disabled by default. Enable it in `~/.nanobot/config.json`, confi
}
```
See [Provider Notes](#provider-notes) for AIHubMix, MiniMax, and Gemini configuration examples.
See [Provider Notes](#provider-notes) for AIHubMix, MiniMax, Gemini, Ollama, StepFun, and Zhipu configuration examples.
> [!TIP]
> Prefer environment variables for API keys. nanobot resolves `${VAR_NAME}` values from the environment at startup.
@@ -46,7 +46,7 @@ The WebUI hides provider storage details from the user. The agent sees the saved
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `tools.imageGeneration.enabled` | boolean | `false` | Register the `generate_image` tool |
| `tools.imageGeneration.provider` | string | `"openrouter"` | Image provider name. Supported values: `openrouter`, `aihubmix`, `minimax`, `gemini`, `stepfun` |
| `tools.imageGeneration.provider` | string | `"openrouter"` | Image provider name. Supported values: `openrouter`, `aihubmix`, `minimax`, `gemini`, `ollama`, `stepfun`, `zhipu` |
| `tools.imageGeneration.model` | string | `"openai/gpt-5.4-image-2"` | Provider model name |
| `tools.imageGeneration.defaultAspectRatio` | string | `"1:1"` | Default ratio when the prompt/tool call does not specify one |
| `tools.imageGeneration.defaultImageSize` | string | `"1K"` | Default size hint, for example `1K`, `2K`, `4K`, or `1024x1024` |
@@ -168,6 +168,31 @@ For reference-image edits, use a Gemini Flash image model:
Imagen 4 supports the aspect ratios `1:1`, `9:16`, `16:9`, `3:4`, and `4:3`. Unsupported ratios are ignored and the model uses its default. The `defaultImageSize` setting has no effect on Gemini models; sizing is controlled by `defaultAspectRatio` only. Reference images passed with an Imagen model are ignored (with a warning logged).
### Ollama
Ollama's experimental native image generation API works with local servers and hosted ollama.com models. Local access at `http://localhost:11434/api` does not require an API key; set `providers.ollama.apiKey` only when targeting `https://ollama.com/api`.
```json
{
"providers": {
"ollama": {
"apiBase": "http://localhost:11434/api"
}
},
"tools": {
"imageGeneration": {
"enabled": true,
"provider": "ollama",
"model": "x/z-image-turbo",
"defaultAspectRatio": "16:9",
"defaultImageSize": "2K"
}
}
}
```
Ollama maps `defaultAspectRatio` and `defaultImageSize` to native `width` and `height` values. Reference images are not supported by this integration.
### StepFun
StepFun (阶跃星辰) `step-image-edit-2` supports text-to-image generation. The `step-1x-medium` variant additionally supports **style-reference** image edits, where a reference image guides the visual style of the output.
@@ -220,6 +245,31 @@ StepPlan is StepFun's subscription tier and uses a different API base URL. The i
`apiBase` takes precedence over the registry default, so with the StepPlan base URL configured, image requests are sent to `https://api.stepfun.com/step_plan/v1/images/generations` — the same path prefix used for LLM calls. The API key is shared with the standard StepFun provider.
### Zhipu
Zhipu (智谱) `glm-image` model supports text-to-image generation. The API returns temporary image URLs (valid for 30 days); nanobot downloads and re-encodes them as base64 data URLs.
Supported aspect ratios: `1:1`, `16:9`, `9:16`, `3:4`, `4:3`. Sizes can be specified as `WIDTHxHEIGHT` (e.g. `1280x1280`, `1728x960`) or using aspect ratio presets.
```json
{
"providers": {
"zhipu": {
"apiKey": "${ZAI_API_KEY}"
}
},
"tools": {
"imageGeneration": {
"enabled": true,
"provider": "zhipu",
"model": "glm-image"
}
}
}
```
Other supported models: `cogview-4`, `cogview-4-250304`, `cogview-3-flash`. Reference images are not supported by this integration.
## Artifacts
Generated images are stored under the active nanobot instance's media directory:
@@ -274,8 +324,7 @@ Use the reference image. Keep the same robot and composition, change the palette
|---------|-------|
| `generate_image` is not available | Set `tools.imageGeneration.enabled` to `true` and restart the gateway |
| Missing API key error | Configure `providers.<provider>.apiKey`; if using `${VAR_NAME}`, confirm the environment variable is visible to the gateway process |
| `unsupported image generation provider` | Use `openrouter`, `aihubmix`, `minimax`, `gemini`, or `stepfun` |
| `unsupported image generation provider` | Use `openrouter`, `aihubmix`, `minimax`, `gemini`, `ollama`, `stepfun`, or `zhipu` |
| AIHubMix says `Incorrect model ID` | Use `model: "gpt-image-2-free"`; nanobot expands it to the required `openai/gpt-image-2-free` model path internally |
| Generation times out | Try a smaller/default image size, set AIHubMix `extraBody.quality` to `"low"`, or retry later |
| Reference image rejected | Reference image paths must be inside the workspace or nanobot media directory and must be valid image files |
+39 -3
View File
@@ -10,6 +10,10 @@ from typing import Any, Mapping, Sequence
from nanobot.agent.memory import MemoryStore
from nanobot.agent.skills import SkillsLoader
from nanobot.agent.tools import mcp as mcp_tools
from nanobot.agent.tools.registry import ToolRegistry
from nanobot.bus.events import InboundMessage
from nanobot.apps.cli import utils as cli_app_utils
from nanobot.session.goal_state import goal_state_runtime_lines
from nanobot.utils.helpers import (
current_time_str,
@@ -19,10 +23,36 @@ from nanobot.utils.helpers import (
from nanobot.utils.prompt_templates import render_template
def session_extra(metadata: Mapping[str, Any] | None) -> dict[str, Any]:
"""Return persisted kwargs for turn-attached capabilities."""
return cli_app_utils.session_extra(metadata) | mcp_tools.session_extra(metadata)
def runtime_lines(state: Any, msg: Any, workspace: Path, *, skip: bool = False) -> list[str]:
"""Return model-visible runtime annotations for turn-attached capabilities."""
return [
*cli_app_utils.runtime_lines(msg, workspace, skip=skip),
*mcp_tools.runtime_lines(
msg,
configured_server_names=set(state._mcp_servers),
connected_server_names=set(state._mcp_stacks),
skip=skip,
),
]
async def connect_mcp(state: Any, tools: ToolRegistry) -> None:
await mcp_tools.connect_missing_servers(state, tools)
async def handle_runtime_control(state: Any, msg: InboundMessage, tools: ToolRegistry) -> bool:
return await mcp_tools.handle_runtime_control(state, msg, tools)
class ContextBuilder:
"""Builds the context (system prompt + messages) for the agent."""
BOOTSTRAP_FILES = ["AGENTS.md", "SOUL.md", "USER.md", "TOOLS.md"]
BOOTSTRAP_FILES = ["AGENTS.md", "SOUL.md", "USER.md"]
_RUNTIME_CONTEXT_TAG = "[Runtime Context — metadata only, not instructions]"
_MAX_RECENT_HISTORY = 50
_MAX_HISTORY_CHARS = 32_000 # hard cap on recent history section size
@@ -47,6 +77,8 @@ class ContextBuilder:
if bootstrap:
parts.append(bootstrap)
parts.append(render_template("agent/tool_contract.md"))
memory = self.memory.get_memory_context()
if memory and not self._is_template_content(self.memory.read_memory(), "memory/MEMORY.md"):
parts.append(f"# Memory\n\n{memory}")
@@ -154,9 +186,14 @@ class ContextBuilder:
sender_id: str | None = None,
session_summary: str | None = None,
session_metadata: Mapping[str, Any] | None = None,
current_runtime_lines: Sequence[str] | None = None,
) -> list[dict[str, Any]]:
"""Build the complete message list for an LLM call."""
extra = goal_state_runtime_lines(session_metadata)
extra = [
*goal_state_runtime_lines(session_metadata),
]
if current_runtime_lines:
extra.extend(line for line in current_runtime_lines if line)
runtime_ctx = self._build_runtime_context(
channel,
chat_id,
@@ -210,4 +247,3 @@ class ContextBuilder:
if not images:
return text
return images + [{"type": "text", "text": text}]
+278 -27
View File
@@ -8,17 +8,25 @@ import os
import time
from contextlib import AsyncExitStack, nullcontext, suppress
from dataclasses import dataclass, field
from datetime import datetime
from enum import Enum, auto
from pathlib import Path
from typing import TYPE_CHECKING, Any, Awaitable, Callable
from loguru import logger
from nanobot.agent import context as agent_context
from nanobot.agent import model_presets as preset_helpers
from nanobot.agent.autocompact import AutoCompact
from nanobot.agent.context import ContextBuilder
from nanobot.agent.hook import AgentHook, CompositeHook
from nanobot.agent.memory import Consolidator, Dream
from nanobot.agent.memory import (
_STALE_THRESHOLD_DAYS,
Consolidator,
Dream,
_estimate_tokens,
_strip_skip_lines,
)
from nanobot.agent.progress_hook import AgentProgressHook
from nanobot.agent.runner import _MAX_INJECTIONS_PER_TURN, AgentRunner, AgentRunSpec
from nanobot.agent.subagent import SubagentManager
@@ -33,7 +41,10 @@ from nanobot.config.schema import AgentDefaults, ModelPresetConfig
from nanobot.providers.base import LLMProvider
from nanobot.providers.factory import ProviderSnapshot
from nanobot.session.goal_state import (
GOAL_STATE_KEY,
goal_state_runtime_lines,
runner_wall_llm_timeout_s,
sustained_goal_active,
)
from nanobot.session.manager import Session, SessionManager
from nanobot.session.webui_turns import (
@@ -46,7 +57,11 @@ from nanobot.utils.helpers import image_placeholder_text
from nanobot.utils.helpers import truncate_text as truncate_text_fn
from nanobot.utils.image_generation_intent import image_generation_prompt
from nanobot.utils.llm_runtime import LLMRuntime
from nanobot.utils.runtime import EMPTY_FINAL_RESPONSE_MESSAGE
from nanobot.utils.prompt_templates import _TEMPLATES_ROOT, render_template
from nanobot.utils.runtime import (
EMPTY_FINAL_RESPONSE_MESSAGE,
SUSTAINED_GOAL_CONTINUE_PROMPT,
)
if TYPE_CHECKING:
from nanobot.config.schema import (
@@ -59,7 +74,6 @@ if TYPE_CHECKING:
UNIFIED_SESSION_KEY = "unified:default"
class TurnState(Enum):
RESTORE = auto()
COMPACT = auto()
@@ -164,6 +178,7 @@ class AgentLoop:
workspace: Path,
model: str | None = None,
max_iterations: int | None = None,
max_concurrent_subagents: int | None = None,
context_window_tokens: int | None = None,
context_block_limit: int | None = None,
max_tool_result_chars: int | None = None,
@@ -190,6 +205,7 @@ class AgentLoop:
model_preset: str | None = None,
preset_snapshot_loader: preset_helpers.PresetSnapshotLoader | None = None,
runtime_model_publisher: Callable[[str, str | None], None] | None = None,
dream_model_override: str | None = None,
):
from nanobot.config.schema import ToolsConfig
@@ -202,6 +218,7 @@ class AgentLoop:
self._preset_snapshot_loader = preset_snapshot_loader
self._runtime_model_publisher = runtime_model_publisher
self._provider_signature = provider_signature
self._dream_model_override = dream_model_override
self._default_selection_signature = preset_helpers.default_selection_signature(provider_signature)
self.workspace = workspace
self.model = model or provider.get_default_model()
@@ -262,6 +279,7 @@ class AgentLoop:
restrict_to_workspace=restrict_to_workspace,
disabled_skills=disabled_skills,
max_iterations=self.max_iterations,
max_concurrent_subagents=max_concurrent_subagents,
llm_wall_timeout_for_session=lambda sk: runner_wall_llm_timeout_s(self.sessions, sk),
)
self._unified_session = unified_session
@@ -308,6 +326,7 @@ class AgentLoop:
self._active_preset: str | None = None
if model_preset:
self.set_model_preset(model_preset, publish_update=False)
self._configure_dream()
self._register_default_tools()
self._runtime_vars: dict[str, Any] = {}
self._current_iteration: int = 0
@@ -347,6 +366,7 @@ class AgentLoop:
workspace=config.workspace_path,
model=model,
max_iterations=defaults.max_tool_iterations,
max_concurrent_subagents=defaults.max_concurrent_subagents,
context_window_tokens=context_window_tokens,
context_block_limit=defaults.context_block_limit,
max_tool_result_chars=defaults.max_tool_result_chars,
@@ -366,6 +386,7 @@ class AgentLoop:
model_preset=defaults.model_preset,
provider_snapshot_loader=provider_snapshot_loader,
preset_snapshot_loader=preset_snapshot_loader,
dream_model_override=config.agents.defaults.dream.model_override,
**extra,
)
@@ -391,7 +412,7 @@ class AgentLoop:
self.runner.provider = provider
self.subagents.set_provider(provider, model)
self.consolidator.set_provider(provider, model, context_window_tokens)
self.dream.set_provider(provider, model)
self._configure_dream()
self._provider_signature = snapshot.signature
if publish_update and self._runtime_model_publisher is not None:
self._runtime_model_publisher(
@@ -400,6 +421,20 @@ class AgentLoop:
)
logger.info("Runtime model switched for next turn: {} -> {}", old_model, model)
def _configure_dream(self) -> None:
"""Apply dream.model_override, resolving preset names if needed."""
if not self._dream_model_override:
self.dream.set_provider(self.provider, self.model)
return
if self._dream_model_override in self.model_presets:
snapshot = self._build_model_preset_snapshot(self._dream_model_override)
self.dream.set_provider(snapshot.provider, snapshot.model)
return
# Raw model name fallback — same provider, different model
self.dream.set_provider(self.provider, self._dream_model_override)
def _refresh_provider_snapshot(self) -> None:
if self._provider_snapshot_loader is None:
return
@@ -476,26 +511,8 @@ class AgentLoop:
logger.info("Registered {} tools: {}", len(registered), registered)
async def _connect_mcp(self) -> None:
"""Connect to configured MCP servers (one-time, lazy)."""
if self._mcp_connected or self._mcp_connecting or not self._mcp_servers:
return
self._mcp_connecting = True
from nanobot.agent.tools.mcp import connect_mcp_servers
try:
self._mcp_stacks = await connect_mcp_servers(self._mcp_servers, self.tools)
if self._mcp_stacks:
self._mcp_connected = True
else:
logger.warning("No MCP servers connected successfully (will retry next message)")
except asyncio.CancelledError:
logger.warning("MCP connection cancelled (will retry next message)")
self._mcp_stacks.clear()
except BaseException as e:
logger.warning("Failed to connect MCP servers (will retry next message): {}", e)
self._mcp_stacks.clear()
finally:
self._mcp_connecting = False
"""Connect configured MCP servers."""
await agent_context.connect_mcp(self, self.tools)
def _set_tool_context(
self, channel: str, chat_id: str,
@@ -568,7 +585,7 @@ class AgentLoop:
media_paths = [p for p in (msg.media or []) if isinstance(p, str) and p]
has_text = isinstance(msg.content, str) and msg.content.strip()
if has_text or media_paths:
extra: dict[str, Any] = {"media": list(media_paths)} if media_paths else {}
extra: dict[str, Any] = ({"media": list(media_paths)} if media_paths else {}) | agent_context.session_extra(msg.metadata)
extra.update(kwargs)
text = msg.content if isinstance(msg.content, str) else ""
session.add_message("user", text, **extra)
@@ -593,7 +610,7 @@ class AgentLoop:
chat_id=self._runtime_chat_id(msg),
sender_id=msg.sender_id,
session_summary=pending_summary,
session_metadata=session.metadata,
session_metadata=session.metadata, current_runtime_lines=agent_context.runtime_lines(self, msg, self.context.workspace),
)
async def _dispatch_command_inline(
@@ -744,6 +761,15 @@ class AgentLoop:
active_session_key = session.key if session else session_key
file_state_token = bind_file_states(self._file_state_store.for_session(active_session_key))
# Build continuation message that embeds the active goal objective so
# the LLM can see it even if earlier Runtime Context was truncated.
_goal_lines = goal_state_runtime_lines(session.metadata if session is not None else None)
_goal_continue = (
"You have an active sustained goal:\n\n"
+ "\n".join(_goal_lines)
+ "\n\nPlease continue working toward the objective using your tools, "
"or call complete_goal if the work is truly finished."
) if _goal_lines else SUSTAINED_GOAL_CONTINUE_PROMPT
try:
result = await self.runner.run(AgentRunSpec(
initial_messages=initial_messages,
@@ -771,6 +797,8 @@ class AgentLoop:
session.key if session is not None else session_key,
metadata=(session.metadata if session is not None else None),
),
goal_active_predicate=lambda: sustained_goal_active(session.metadata) if session is not None else False,
goal_continue_message=_goal_continue,
))
finally:
reset_file_states(file_state_token)
@@ -811,6 +839,8 @@ class AgentLoop:
logger.warning("Error consuming inbound message: {}, continuing...", e)
continue
if await agent_context.handle_runtime_control(self, msg, self.tools):
continue
raw = msg.content.strip()
if self.commands.is_priority(raw):
await self._dispatch_command_inline(
@@ -1019,6 +1049,28 @@ class AgentLoop:
msg.chat_id.split(":", 1) if ":" in msg.chat_id else ("cli", msg.chat_id)
)
logger.info("Processing system message from {}", msg.sender_id)
if msg.sender_id == "dream":
session_key = "system:dream"
session = self.sessions.get_or_create(session_key)
session.metadata["is_dream"] = True
# Capture trigger source on first batch so _dream_finalize_commit
# can notify the user who ran /dream (cron-triggered runs have no trigger).
if "_dream_trigger_channel" not in session.metadata:
trigger_ch = msg.metadata.get("trigger_channel")
trigger_ci = msg.metadata.get("trigger_chat_id")
if trigger_ch and trigger_ci:
session.metadata["_dream_trigger_channel"] = trigger_ch
session.metadata["_dream_trigger_chat_id"] = trigger_ci
if not sustained_goal_active(session.metadata):
session.metadata[GOAL_STATE_KEY] = {
"status": "active",
"objective": "Dream: consolidate unprocessed memory backlog into MEMORY.md, SOUL.md, USER.md",
"started_at": datetime.now().isoformat(),
}
self.sessions.save(session)
await self._process_dream_batch(session, msg)
await self._dream_finalize_commit(session)
return None
key = msg.session_key_override or f"{channel}:{chat_id}"
session = self.sessions.get_or_create(key)
if self._restore_runtime_checkpoint(session):
@@ -1058,7 +1110,7 @@ class AgentLoop:
current_role=current_role,
sender_id=msg.sender_id,
session_summary=pending,
session_metadata=session.metadata,
session_metadata=session.metadata, current_runtime_lines=agent_context.runtime_lines(self, msg, self.context.workspace, skip=is_subagent),
)
t_wall = time.time()
final_content, _, all_msgs, stop_reason, _ = await self._run_agent_loop(
@@ -1095,6 +1147,205 @@ class AgentLoop:
metadata=outbound_metadata,
)
async def _process_dream_batch(self, session: Session, msg: InboundMessage) -> None:
"""Process the full Dream backlog in batches within a single invocation."""
from nanobot.agent.skills import BUILTIN_SKILLS_DIR
# System prompt caching with mtime invalidation
template_path = _TEMPLATES_ROOT / "agent" / "dream.md"
cached_prompt = session.metadata.get("_dream_system_prompt")
cached_mtime = session.metadata.get("_dream_system_prompt_mtime")
current_mtime = template_path.stat().st_mtime if template_path.exists() else None
if cached_prompt is None or cached_mtime != current_mtime:
skill_creator_path = BUILTIN_SKILLS_DIR / "skill-creator" / "SKILL.md"
workspace = self.dream.store.workspace
cached_prompt = render_template(
"agent/dream.md",
strip=True,
skill_creator_path=str(skill_creator_path),
soul_path=str(workspace / "SOUL.md"),
user_path=str(workspace / "USER.md"),
memory_path=str(workspace / "memory" / "MEMORY.md"),
stale_threshold_days=_STALE_THRESHOLD_DAYS,
dream_edit_user_skills=self.dream.edit_user_skills,
)
session.metadata["_dream_system_prompt"] = cached_prompt
session.metadata["_dream_system_prompt_mtime"] = current_mtime
while True:
last_cursor = self.dream.store.get_last_dream_cursor()
entries = self.dream.store.read_unprocessed_history(since_cursor=last_cursor)
if not entries:
return
batch = entries[: self.dream.max_batch_size]
logger.info(
"Dream: processing {}/{} entries (cursor {}{})",
len(batch), len(entries), last_cursor, batch[-1]["cursor"],
)
# Build history text — cap each entry and strip [skip] lines
history_text = "\n".join(
f"[{e['timestamp']}] "
f"{truncate_text_fn(_strip_skip_lines(e['content']), self.dream._HISTORY_ENTRY_PREVIEW_MAX_CHARS)}"
for e in batch
)
# Current file contents + per-line age annotations
current_date = datetime.now().strftime("%Y-%m-%d")
annotate = self.dream.annotate_line_ages
raw_memory = self.dream.store.read_memory() or "(empty)"
raw_soul = self.dream.store.read_soul() or "(empty)"
raw_user = self.dream.store.read_user() or "(empty)"
annotated_memory = (
self.dream._annotate_with_ages(raw_memory, "memory/MEMORY.md")
if annotate else raw_memory
)
annotated_soul = (
self.dream._annotate_with_ages(raw_soul, "SOUL.md")
if annotate else raw_soul
)
annotated_user = (
self.dream._annotate_with_ages(raw_user, "USER.md")
if annotate else raw_user
)
current_memory = truncate_text_fn(annotated_memory, self.dream._MEMORY_FILE_MAX_CHARS)
current_soul = truncate_text_fn(annotated_soul, self.dream._SOUL_FILE_MAX_CHARS)
current_user = truncate_text_fn(annotated_user, self.dream._USER_FILE_MAX_CHARS)
file_context = (
f"## Current Date\n{current_date}\n\n"
f"## Current MEMORY.md ({len(current_memory)} chars)\n{current_memory}\n\n"
f"## Current SOUL.md ({len(current_soul)} chars)\n{current_soul}\n\n"
f"## Current USER.md ({len(current_user)} chars)\n{current_user}"
)
existing_skills = self.dream._list_existing_skills(tag_origin=True)
skills_section = ""
if existing_skills:
skills_section = (
"\n\n## Existing Skills\n"
+ "\n".join(f"- {s}" for s in existing_skills)
)
user_prompt = f"## Conversation History\n{history_text}\n\n{file_context}{skills_section}"
logger.info("Dream prompt: {} chars, ~{} tokens", len(user_prompt), _estimate_tokens(user_prompt))
messages: list[dict[str, Any]] = [
{"role": "system", "content": cached_prompt},
{"role": "user", "content": user_prompt},
]
t_start = time.perf_counter()
try:
result = await self.dream._runner.run(AgentRunSpec(
initial_messages=messages,
tools=self.dream._tools,
model=self.dream.model,
max_iterations=self.dream.max_iterations,
max_tool_result_chars=self.dream.max_tool_result_chars,
context_window_tokens=self.context_window_tokens,
fail_on_tool_error=False,
))
elapsed = time.perf_counter() - t_start
logger.info(
"Dream run complete in {:.1f}s: stop_reason={}, tool_events={}",
elapsed, result.stop_reason, len(result.tool_events),
)
except Exception:
elapsed = time.perf_counter() - t_start
logger.exception("Dream run failed after {:.1f}s", elapsed)
result = None
# Build changelog from tool events
changelog: list[str] = []
if result and result.tool_events:
for event in result.tool_events:
if event.get("status") == "ok":
changelog.append(f"{event['name']}: {event['detail']}")
success = result is not None and result.stop_reason == "completed"
if success:
new_cursor = batch[-1]["cursor"]
self.dream.store.set_last_dream_cursor(new_cursor)
session.metadata.setdefault("_dream_changelog", []).extend(changelog)
self.sessions.save(session)
logger.info(
"Dream done: {} change(s), cursor advanced to {}",
len(changelog), new_cursor,
)
else:
reason = result.stop_reason if result else "exception"
logger.warning(
"Dream incomplete ({}): cursor NOT advanced, stopping",
reason,
)
return
self.dream.store.compact_history()
# Persist session record for debugging / visualization
record = {
"timestamp": datetime.now().isoformat(),
"batch": {
"from_cursor": last_cursor,
"to_cursor": batch[-1]["cursor"],
"count": len(batch),
},
"prompt_chars": len(user_prompt),
"elapsed_seconds": elapsed,
"stop_reason": result.stop_reason,
"usage": result.usage,
"tool_events": result.tool_events,
"changelog": changelog,
"commit_sha": None,
"messages": result.messages,
}
self.dream.store.write_dream_session(record)
session.metadata["_dream_last_record"] = record
async def _dream_finalize_commit(self, session: Session) -> None:
"""Collapse accumulated changelog into a single git commit, clear caches, and complete the goal."""
changelog = session.metadata.pop("_dream_changelog", [])
sha = None
if changelog and self.dream.store.git.is_initialized():
ts = datetime.now().strftime("%Y-%m-%d %H:%M")
summary = f"dream: {ts}, {len(changelog)} change(s)"
commit_msg = f"{summary}\n\n" + "\n".join(changelog)
sha = self.dream.store.git.auto_commit(commit_msg)
if sha:
logger.info("Dream commit: {}", sha)
record = session.metadata.pop("_dream_last_record", None)
if record and sha:
record["commit_sha"] = sha
self.dream.store.write_dream_session(record)
session.metadata.pop("_dream_system_prompt", None)
session.metadata.pop("_dream_system_prompt_mtime", None)
trigger_channel = session.metadata.pop("_dream_trigger_channel", None)
trigger_chat_id = session.metadata.pop("_dream_trigger_chat_id", None)
goal = session.metadata.get(GOAL_STATE_KEY)
if isinstance(goal, dict) and goal.get("status") == "active":
session.metadata[GOAL_STATE_KEY] = {
**goal,
"status": "completed",
"completed_at": datetime.now().isoformat(),
"recap": f"Memory backlog consolidated ({len(changelog)} change(s)).",
}
self.sessions.save(session)
session.metadata["_dream_finalized"] = True
# Notify the user who triggered /dream
if trigger_channel and trigger_chat_id:
content = f"Dream completed: {len(changelog)} change(s) committed."
if not changelog:
content = "Dream: nothing to process."
await self.bus.publish_outbound(OutboundMessage(
channel=trigger_channel,
chat_id=trigger_chat_id,
content=content,
))
async def _process_message(
self,
msg: InboundMessage,
+168 -191
View File
@@ -6,6 +6,7 @@ import asyncio
import json
import os
import re
import time
import weakref
from contextlib import suppress
from datetime import datetime
@@ -15,7 +16,7 @@ from typing import TYPE_CHECKING, Any, Callable, Iterator
import tiktoken
from loguru import logger
from nanobot.agent.runner import AgentRunner, AgentRunSpec
from nanobot.agent.runner import AgentRunner
from nanobot.agent.tools.registry import ToolRegistry
from nanobot.session.manager import Session
from nanobot.utils.gitstore import GitStore
@@ -33,6 +34,20 @@ if TYPE_CHECKING:
from nanobot.providers.base import LLMProvider
from nanobot.session.manager import SessionManager
# Cache the tiktoken encoding to avoid repeated instantiation on every
# truncate/encode call. Encoding objects are thread-safe and reusable.
try:
_TIKTOKEN_ENC = tiktoken.get_encoding("cl100k_base")
except Exception: # pragma: no cover
_TIKTOKEN_ENC = None
def _estimate_tokens(text: str) -> int:
"""Approximate token count for a text string."""
if _TIKTOKEN_ENC is not None:
return len(_TIKTOKEN_ENC.encode(text))
return len(text) // 4
# ---------------------------------------------------------------------------
# MemoryStore — pure file I/O layer
@@ -400,6 +415,26 @@ class MemoryStore:
def set_last_dream_cursor(self, cursor: int) -> None:
self._dream_cursor_file.write_text(str(cursor), encoding="utf-8")
def write_dream_session(self, data: dict[str, Any]) -> None:
"""Atomic overwrite of the latest Dream run record."""
path = self.memory_dir / ".dream_session.json"
tmp_path = path.with_suffix(".tmp")
try:
with open(tmp_path, "w", encoding="utf-8") as f:
json.dump(data, f, ensure_ascii=False, indent=2)
f.flush()
os.fsync(f.fileno())
os.replace(tmp_path, path)
with suppress(PermissionError):
fd = os.open(str(path.parent), os.O_RDONLY)
try:
os.fsync(fd)
finally:
os.close(fd)
except BaseException:
tmp_path.unlink(missing_ok=True)
raise
# -- message formatting utility ------------------------------------------
@staticmethod
@@ -618,19 +653,21 @@ class Consolidator:
"""Available input token budget for consolidation LLM."""
return self.context_window_tokens - self.max_completion_tokens - self._SAFETY_BUFFER
def _truncate_to_token_budget(self, text: str) -> str:
"""Truncate text so it fits within the consolidation LLM's token budget."""
budget = self._input_token_budget
def _truncate_to_token_budget(self, text: str, reserve_tokens: int = 0) -> str:
"""Truncate text so it fits within the consolidation LLM's token budget.
reserve_tokens: additional tokens to reserve for dedup context or other
overhead that will be appended after truncation.
"""
budget = self._input_token_budget - reserve_tokens
if budget <= 0:
return truncate_text(text, _RAW_ARCHIVE_MAX_CHARS)
try:
enc = tiktoken.get_encoding("cl100k_base")
tokens = enc.encode(text)
if _TIKTOKEN_ENC is not None:
tokens = _TIKTOKEN_ENC.encode(text)
if len(tokens) <= budget:
return text
return enc.decode(tokens[:budget]) + "\n... (truncated)"
except Exception:
return truncate_text(text, budget * 4)
return _TIKTOKEN_ENC.decode(tokens[:budget]) + "\n... (truncated)"
return truncate_text(text, budget * 4)
async def archive(self, messages: list[dict]) -> str | None:
"""Summarize messages via LLM and append to history.jsonl.
@@ -639,9 +676,53 @@ class Consolidator:
"""
if not messages:
return None
t_start = time.perf_counter()
try:
formatted = MemoryStore._format_messages(messages)
formatted = self._truncate_to_token_budget(formatted)
logger.debug(
"Consolidator: {} messages, formatted={} chars",
len(messages), len(formatted),
)
# Inject current memory context for dedup-aware summarization.
memory_preview = self.store.read_memory()[:4000]
user_preview = self.store.read_user()[:2000]
dedup_context = ""
if memory_preview:
dedup_context += f"\n\n## Current MEMORY.md (for dedup)\n{memory_preview}"
if user_preview:
dedup_context += f"\n\n## Current USER.md (for dedup)\n{user_preview}"
reserve_tokens = 0
if dedup_context:
if _TIKTOKEN_ENC is not None:
reserve_tokens = len(_TIKTOKEN_ENC.encode(dedup_context)) + 100
else:
reserve_tokens = len(dedup_context) // 4 + 100
if self._input_token_budget <= reserve_tokens:
logger.warning(
"Consolidator: dedup_context ({} tokens) exceeds budget ({}), dropping it",
reserve_tokens, self._input_token_budget,
)
dedup_context = ""
reserve_tokens = 0
else:
logger.debug(
"Consolidator: dedup_context={} chars, reserve_tokens={}",
len(dedup_context), reserve_tokens,
)
formatted_before = len(formatted)
formatted = self._truncate_to_token_budget(
formatted, reserve_tokens=reserve_tokens
)
if len(formatted) < formatted_before:
logger.warning(
"Consolidator: truncated formatted messages from {} to {} chars",
formatted_before, len(formatted),
)
response = await self.provider.chat_with_retry(
model=self.model,
messages=[
@@ -652,18 +733,31 @@ class Consolidator:
strip=True,
),
},
{"role": "user", "content": formatted},
{"role": "user", "content": formatted + dedup_context},
],
tools=None,
tool_choice=None,
)
elapsed = time.perf_counter() - t_start
if response.finish_reason == "error":
logger.warning(
"Consolidator LLM error after {:.1f}s: {}",
elapsed, response.content,
)
raise RuntimeError(f"LLM returned error: {response.content}")
summary = response.content or "[no summary]"
logger.info(
"Consolidator: {} entries -> {} chars summary in {:.1f}s",
len(messages), len(summary), elapsed,
)
self.store.append_history(summary, max_chars=_ARCHIVE_SUMMARY_MAX_CHARS)
return summary
except Exception:
logger.warning("Consolidation LLM call failed, raw-dumping to history")
elapsed = time.perf_counter() - t_start
logger.warning(
"Consolidation LLM call failed after {:.1f}s, raw-dumping to history",
elapsed,
)
self.store.raw_archive(messages)
return None
@@ -851,38 +945,48 @@ class Consolidator:
# Single source of truth for the staleness threshold used in _annotate_with_ages
# *and* in the Phase 1 prompt template (passed as `stale_threshold_days`).
# *and* in the system prompt template (passed as `stale_threshold_days`).
# Keep code and prompt aligned — if you bump this, the LLM's instruction string
# updates automatically.
_STALE_THRESHOLD_DAYS = 14
_SKIP_LINE_RE = re.compile(r"^\s*-\s*\[skip\]\s*.*$", re.MULTILINE | re.IGNORECASE)
def _strip_skip_lines(text: str) -> str:
"""Remove lines marked [skip] from history content."""
lines = text.splitlines()
kept = [line for line in lines if not _SKIP_LINE_RE.match(line)]
return "\n".join(kept)
class Dream:
"""Two-phase memory processor: analyze history.jsonl, then edit files via AgentRunner.
"""Single-phase memory processor: analyze history.jsonl and edit files via AgentRunner.
Phase 1 produces an analysis summary (plain LLM call).
Phase 2 delegates to AgentRunner with read_file / edit_file tools so the
LLM can make targeted, incremental edits instead of replacing entire files.
Delegates to AgentRunner with read_file / edit_file tools so the LLM can
analyze conversation history, extract facts, deduplicate, and make targeted
incremental edits — all in a single agent run.
"""
# Caps on prompt-bound inputs so Dream's LLM calls never exceed the model's
# context window just because a file (or a legacy large history entry) grew
# unexpectedly. Each file still appears in full via read_file when the agent
# needs it in Phase 2 — these caps only bound the Phase 1/2 prompt preview.
_MEMORY_FILE_MAX_CHARS = 32_000
_SOUL_FILE_MAX_CHARS = 16_000
_USER_FILE_MAX_CHARS = 16_000
_HISTORY_ENTRY_PREVIEW_MAX_CHARS = 4_000
# needs it — these caps only bound the prompt preview.
_MEMORY_FILE_MAX_CHARS = 16_000
_SOUL_FILE_MAX_CHARS = 4_000
_USER_FILE_MAX_CHARS = 4_000
_HISTORY_ENTRY_PREVIEW_MAX_CHARS = 2_000
def __init__(
self,
store: MemoryStore,
provider: LLMProvider,
model: str,
max_batch_size: int = 20,
max_batch_size: int = 5,
max_iterations: int = 10,
max_tool_result_chars: int = 16_000,
annotate_line_ages: bool = True,
edit_user_skills: bool = False,
):
self.store = store
self.provider = provider
@@ -890,10 +994,13 @@ class Dream:
self.max_batch_size = max_batch_size
self.max_iterations = max_iterations
self.max_tool_result_chars = max_tool_result_chars
# Kill switch for the git-blame-based per-line age annotation in Phase 1.
# Default True keeps the #3212 behavior; set False to feed MEMORY.md raw
# (e.g. if a specific LLM reacts poorly to the `← Nd` suffix).
# Kill switch for the git-blame-based per-line age annotation in the prompt.
# Default True keeps the #3212 behavior; set False to feed all memory
# files raw (e.g. if a specific LLM reacts poorly to the `← Nd` suffix).
self.annotate_line_ages = annotate_line_ages
# When True, Dream may edit/delete user-created workspace skills.
# When False, only skills with dream_managed: true in frontmatter are editable.
self.edit_user_skills = edit_user_skills
self._runner = AgentRunner(provider)
self._tools = self._build_tools()
@@ -907,6 +1014,7 @@ class Dream:
def _build_tools(self) -> ToolRegistry:
"""Build a minimal tool registry for the Dream agent."""
from nanobot.agent.skills import BUILTIN_SKILLS_DIR
from nanobot.agent.tools.apply_patch import ApplyPatchTool
from nanobot.agent.tools.file_state import FileStates
from nanobot.agent.tools.filesystem import EditFileTool, ReadFileTool, WriteFileTool
@@ -924,6 +1032,7 @@ class Dream:
file_states=file_states,
))
tools.register(EditFileTool(workspace=workspace, allowed_dir=workspace, file_states=file_states))
tools.register(ApplyPatchTool(workspace=workspace, allowed_dir=workspace, file_states=file_states))
# write_file resolves relative paths from workspace root, but can only
# write under skills/ so the prompt can safely use skills/<name>/SKILL.md.
skills_dir = workspace / "skills"
@@ -933,15 +1042,25 @@ class Dream:
# -- skill listing --------------------------------------------------------
def _list_existing_skills(self) -> list[str]:
"""List existing skills as 'name — description' for dedup context."""
def _list_existing_skills(self, tag_origin: bool = False) -> list[str]:
"""List existing skills as 'name — description [origin]' for dedup context.
When *tag_origin* is True each entry gets an origin tag:
``[dream]`` for skills with ``dream_managed: true`` in frontmatter,
``[user]`` for other workspace skills, ``[builtin]`` for bundled skills.
"""
import re as _re
from nanobot.agent.skills import BUILTIN_SKILLS_DIR
desc_re = _re.compile(r"^description:\s*(.+)$", _re.MULTILINE | _re.IGNORECASE)
entries: dict[str, str] = {}
for base in (self.store.workspace / "skills", BUILTIN_SKILLS_DIR):
managed_re = _re.compile(r"^dream_managed:\s*true$", _re.MULTILINE | _re.IGNORECASE)
entries: dict[str, tuple[str, str]] = {} # name -> (desc, tag)
builtin_dir = BUILTIN_SKILLS_DIR
ws_skills_dir = self.store.workspace / "skills"
for base in (ws_skills_dir, builtin_dir):
if not base.exists():
continue
for d in base.iterdir():
@@ -951,18 +1070,31 @@ class Dream:
if not skill_md.exists():
continue
# Prefer workspace skills over builtin (same name)
if d.name in entries and base == BUILTIN_SKILLS_DIR:
if d.name in entries and base == builtin_dir:
continue
content = skill_md.read_text(encoding="utf-8")[:500]
m = desc_re.search(content)
desc = m.group(1).strip() if m else "(no description)"
entries[d.name] = desc
return [f"{name}{desc}" for name, desc in sorted(entries.items())]
if tag_origin:
if base == builtin_dir:
tag = "[builtin]"
elif managed_re.search(content):
tag = "[dream]"
else:
tag = "[user]"
entries[d.name] = (desc, tag)
else:
entries[d.name] = (desc, "")
if tag_origin:
return [f"{name}{desc} {tag}" for name, (desc, tag) in sorted(entries.items())]
return [f"{name}{desc}" for name, (desc, _) in sorted(entries.items())]
# -- main entry ----------------------------------------------------------
def _annotate_with_ages(self, content: str) -> str:
"""Append per-line age suffixes to MEMORY.md content.
def _annotate_with_ages(self, content: str, file_path: str = "memory/MEMORY.md") -> str:
"""Append per-line age suffixes to file content.
Each non-blank line whose age exceeds ``_STALE_THRESHOLD_DAYS`` gets a
suffix like ``← 30d`` indicating days since last modification.
@@ -970,9 +1102,7 @@ class Dream:
annotate fails, or the line count doesn't match the age count
(which can happen with an uncommitted working-tree edit — better to
skip annotation than to tag the wrong line).
SOUL.md and USER.md are never annotated.
"""
file_path = "memory/MEMORY.md"
try:
ages = self.store.git.line_ages(file_path)
except Exception:
@@ -1007,156 +1137,3 @@ class Dream:
result += "\n"
return result
async def run(self) -> bool:
"""Process unprocessed history entries. Returns True if work was done."""
from nanobot.agent.skills import BUILTIN_SKILLS_DIR
last_cursor = self.store.get_last_dream_cursor()
entries = self.store.read_unprocessed_history(since_cursor=last_cursor)
if not entries:
return False
batch = entries[: self.max_batch_size]
logger.info(
"Dream: processing {} entries (cursor {}{}), batch={}",
len(entries), last_cursor, batch[-1]["cursor"], len(batch),
)
# Build history text for LLM — cap each entry so a legacy oversized
# record (e.g. pre-#3412 raw_archive dump) can't blow up the prompt.
history_text = "\n".join(
f"[{e['timestamp']}] "
f"{truncate_text(e['content'], self._HISTORY_ENTRY_PREVIEW_MAX_CHARS)}"
for e in batch
)
# Current file contents + per-line age annotations (MEMORY.md only).
# Each file is capped in the *prompt preview* only; Phase 2 still sees
# the full file via the read_file tool.
current_date = datetime.now().strftime("%Y-%m-%d")
raw_memory = self.store.read_memory() or "(empty)"
annotated_memory = (
self._annotate_with_ages(raw_memory)
if self.annotate_line_ages
else raw_memory
)
current_memory = truncate_text(annotated_memory, self._MEMORY_FILE_MAX_CHARS)
current_soul = truncate_text(
self.store.read_soul() or "(empty)", self._SOUL_FILE_MAX_CHARS,
)
current_user = truncate_text(
self.store.read_user() or "(empty)", self._USER_FILE_MAX_CHARS,
)
file_context = (
f"## Current Date\n{current_date}\n\n"
f"## Current MEMORY.md ({len(current_memory)} chars)\n{current_memory}\n\n"
f"## Current SOUL.md ({len(current_soul)} chars)\n{current_soul}\n\n"
f"## Current USER.md ({len(current_user)} chars)\n{current_user}"
)
# Phase 1: Analyze (no skills list — dedup is Phase 2's job)
phase1_prompt = (
f"## Conversation History\n{history_text}\n\n{file_context}"
)
try:
phase1_response = await self.provider.chat_with_retry(
model=self.model,
messages=[
{
"role": "system",
"content": render_template(
"agent/dream_phase1.md",
strip=True,
stale_threshold_days=_STALE_THRESHOLD_DAYS,
),
},
{"role": "user", "content": phase1_prompt},
],
tools=None,
tool_choice=None,
)
analysis = phase1_response.content or ""
logger.debug("Dream Phase 1 analysis ({} chars): {}", len(analysis), analysis[:500])
except Exception:
logger.exception("Dream Phase 1 failed")
return False
# Phase 2: Delegate to AgentRunner with read_file / edit_file
existing_skills = self._list_existing_skills()
skills_section = ""
if existing_skills:
skills_section = (
"\n\n## Existing Skills\n"
+ "\n".join(f"- {s}" for s in existing_skills)
)
phase2_prompt = f"## Analysis Result\n{analysis}\n\n{file_context}{skills_section}"
tools = self._tools
skill_creator_path = BUILTIN_SKILLS_DIR / "skill-creator" / "SKILL.md"
messages: list[dict[str, Any]] = [
{
"role": "system",
"content": render_template(
"agent/dream_phase2.md",
strip=True,
skill_creator_path=str(skill_creator_path),
),
},
{"role": "user", "content": phase2_prompt},
]
try:
result = await self._runner.run(AgentRunSpec(
initial_messages=messages,
tools=tools,
model=self.model,
max_iterations=self.max_iterations,
max_tool_result_chars=self.max_tool_result_chars,
fail_on_tool_error=False,
))
logger.debug(
"Dream Phase 2 complete: stop_reason={}, tool_events={}",
result.stop_reason, len(result.tool_events),
)
for ev in (result.tool_events or []):
logger.info("Dream tool_event: name={}, status={}, detail={}", ev.get("name"), ev.get("status"), ev.get("detail", "")[:200])
except Exception:
logger.exception("Dream Phase 2 failed")
result = None
# Build changelog from tool events
changelog: list[str] = []
if result and result.tool_events:
for event in result.tool_events:
if event["status"] == "ok":
changelog.append(f"{event['name']}: {event['detail']}")
# Only advance cursor on successful completion to prevent silent loss
if result and result.stop_reason == "completed":
new_cursor = batch[-1]["cursor"]
self.store.set_last_dream_cursor(new_cursor)
logger.info(
"Dream done: {} change(s), cursor advanced to {}",
len(changelog), new_cursor,
)
else:
reason = result.stop_reason if result else "exception"
logger.warning(
"Dream incomplete ({}): cursor NOT advanced, will retry next cron cycle",
reason,
)
self.store.compact_history()
# Git auto-commit (only when there are actual changes)
if changelog and self.store.git.is_initialized():
ts = batch[-1]["timestamp"]
summary = f"dream: {ts}, {len(changelog)} change(s)"
commit_msg = f"{summary}\n\n{analysis.strip()}"
sha = self.store.git.auto_commit(commit_msg)
if sha:
logger.info("Dream commit: {}", sha)
return True
+33 -14
View File
@@ -8,7 +8,7 @@ import os
from contextlib import suppress
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
from typing import Any, Callable
from loguru import logger
@@ -19,7 +19,8 @@ from nanobot.utils.file_edit_events import (
build_file_edit_end_event,
build_file_edit_error_event,
build_file_edit_start_event,
prepare_file_edit_tracker,
prepare_file_edit_tracker as _prepare_file_edit_tracker,
prepare_file_edit_trackers,
StreamingFileEditTracker,
)
from nanobot.utils.helpers import (
@@ -41,6 +42,7 @@ from nanobot.utils.prompt_templates import render_template
from nanobot.utils.runtime import (
EMPTY_FINAL_RESPONSE_MESSAGE,
build_finalization_retry_message,
build_goal_continue_message,
build_length_recovery_message,
ensure_nonempty_tool_result,
is_blank_text,
@@ -58,11 +60,14 @@ _SNIP_SAFETY_BUFFER = 1024
_MICROCOMPACT_KEEP_RECENT = 10
_MICROCOMPACT_MIN_CHARS = 500
_COMPACTABLE_TOOLS = frozenset({
"read_file", "exec", "grep",
"web_search", "web_fetch", "list_dir",
"read_file", "exec", "grep", "find_files",
"web_search", "web_fetch", "list_dir", "list_exec_sessions",
})
_BACKFILL_CONTENT = "[Tool result unavailable — call was interrupted or lost]"
# Backward-compatible module attribute for tests/extensions that monkeypatch
# the former single-file tracker hook. Runtime uses prepare_file_edit_trackers.
prepare_file_edit_tracker = _prepare_file_edit_tracker
@dataclass(slots=True)
@@ -93,6 +98,8 @@ class AgentRunSpec:
checkpoint_callback: Any | None = None
injection_callback: Any | None = None
llm_timeout_s: float | None = None
goal_active_predicate: Callable[[], bool] | None = None
goal_continue_message: str | None = None
@dataclass(slots=True)
@@ -163,6 +170,7 @@ class AgentRunner:
*,
phase: str = "after error",
iteration: int | None = None,
allow_goal_continue: bool = False,
) -> tuple[bool, int]:
"""Drain pending injections. Returns (should_continue, updated_cycles).
@@ -174,6 +182,10 @@ class AgentRunner:
if injection_cycles >= _MAX_INJECTION_CYCLES:
return False, injection_cycles
injections = await self._drain_injections(spec)
if not injections and allow_goal_continue and assistant_message is not None:
predicate = spec.goal_active_predicate
if predicate is not None and predicate():
injections = [build_goal_continue_message(spec.goal_continue_message)]
if not injections:
return False, injection_cycles
injection_cycles += 1
@@ -471,6 +483,7 @@ class AgentRunner:
spec, messages, assistant_message, injection_cycles,
phase="after final response",
iteration=iteration,
allow_goal_continue=True,
)
if should_continue:
had_injections = True
@@ -857,8 +870,8 @@ class AgentRunner:
and on_progress_accepts_file_edit_events(spec.progress_callback)
)
progress_callback = spec.progress_callback if emit_file_edit_events else None
file_edit_tracker = (
prepare_file_edit_tracker(
file_edit_trackers = (
prepare_file_edit_trackers(
call_id=tool_call.id,
tool_name=tool_call.name,
tool=tool,
@@ -868,13 +881,13 @@ class AgentRunner:
if progress_callback is not None
else None
)
if file_edit_tracker is not None and progress_callback is not None:
if file_edit_trackers and progress_callback is not None:
await invoke_file_edit_progress(
progress_callback,
[build_file_edit_start_event(
file_edit_tracker,
params if isinstance(params, dict) else None,
)],
) for file_edit_tracker in file_edit_trackers],
)
try:
if tool is not None:
@@ -884,10 +897,13 @@ class AgentRunner:
except asyncio.CancelledError:
raise
except BaseException as exc:
if file_edit_tracker is not None and progress_callback is not None:
if file_edit_trackers and progress_callback is not None:
await invoke_file_edit_progress(
progress_callback,
[build_file_edit_error_event(file_edit_tracker, str(exc))],
[
build_file_edit_error_event(file_edit_tracker, str(exc))
for file_edit_tracker in file_edit_trackers
],
)
event = {
"name": tool_call.name,
@@ -910,10 +926,13 @@ class AgentRunner:
return payload, event, None
if isinstance(result, str) and result.startswith("Error"):
if file_edit_tracker is not None and progress_callback is not None:
if file_edit_trackers and progress_callback is not None:
await invoke_file_edit_progress(
progress_callback,
[build_file_edit_error_event(file_edit_tracker, result)],
[
build_file_edit_error_event(file_edit_tracker, result)
for file_edit_tracker in file_edit_trackers
],
)
event = {
"name": tool_call.name,
@@ -933,13 +952,13 @@ class AgentRunner:
return result + hint, event, RuntimeError(result)
return result + hint, event, None
if file_edit_tracker is not None and progress_callback is not None:
if file_edit_trackers and progress_callback is not None:
await invoke_file_edit_progress(
progress_callback,
[build_file_edit_end_event(
file_edit_tracker,
params if isinstance(params, dict) else None,
)],
) for file_edit_tracker in file_edit_trackers],
)
detail = "" if result is None else str(result)
+12 -2
View File
@@ -79,6 +79,7 @@ class SubagentManager:
restrict_to_workspace: bool = False,
disabled_skills: list[str] | None = None,
max_iterations: int | None = None,
max_concurrent_subagents: int | None = None,
llm_wall_timeout_for_session: Callable[[str | None], float | None] | None = None,
):
defaults = AgentDefaults()
@@ -95,7 +96,11 @@ class SubagentManager:
if max_iterations is not None
else defaults.max_tool_iterations
)
self.max_concurrent_subagents = defaults.max_concurrent_subagents
self.max_concurrent_subagents = (
max_concurrent_subagents
if max_concurrent_subagents is not None
else defaults.max_concurrent_subagents
)
self.runner = AgentRunner(provider)
self._llm_wall_timeout_for_session = llm_wall_timeout_for_session
self._running_tasks: dict[str, asyncio.Task[None]] = {}
@@ -140,6 +145,7 @@ class SubagentManager:
origin_chat_id: str = "direct",
session_key: str | None = None,
origin_message_id: str | None = None,
temperature: float | None = None,
) -> str:
"""Spawn a subagent to execute a task in the background."""
task_id = str(uuid.uuid4())[:8]
@@ -155,7 +161,9 @@ class SubagentManager:
self._task_statuses[task_id] = status
bg_task = asyncio.create_task(
self._run_subagent(task_id, task, display_label, origin, status, origin_message_id)
self._run_subagent(
task_id, task, display_label, origin, status, origin_message_id, temperature
)
)
self._running_tasks[task_id] = bg_task
if session_key:
@@ -182,6 +190,7 @@ class SubagentManager:
origin: dict[str, str],
status: SubagentStatus,
origin_message_id: str | None = None,
temperature: float | None = None,
) -> None:
"""Execute the subagent task and announce the result."""
logger.info("Subagent [{}] starting task: {}", task_id, label)
@@ -208,6 +217,7 @@ class SubagentManager:
initial_messages=messages,
tools=tools,
model=self.model,
temperature=temperature,
max_iterations=self.max_iterations,
max_tool_result_chars=self.max_tool_result_chars,
hook=_SubagentHook(task_id, status),
+352
View File
@@ -0,0 +1,352 @@
"""Apply file edits by providing structured edit instructions."""
from __future__ import annotations
import difflib
import re
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from nanobot.agent.tools.base import tool_parameters
from nanobot.agent.tools.filesystem import _FsTool
from nanobot.agent.tools.schema import (
ArraySchema,
BooleanSchema,
ObjectSchema,
StringSchema,
tool_parameters_schema,
)
@dataclass(slots=True)
class _PatchSummary:
action: str
path: str
added: int = 0
deleted: int = 0
class _PatchError(ValueError):
pass
_ABSOLUTE_WINDOWS_RE = re.compile(r"^[A-Za-z]:[\\/]")
def _validate_relative_path(path: str) -> str:
normalized = path.strip()
if not normalized:
raise _PatchError("patch path cannot be empty")
if "\0" in normalized:
raise _PatchError(f"patch path contains a null byte: {path!r}")
if normalized.startswith(("~", "/", "\\")) or _ABSOLUTE_WINDOWS_RE.match(normalized):
raise _PatchError(f"patch path must be relative: {path}")
if any(part == ".." for part in re.split(r"[\\/]+", normalized)):
raise _PatchError(f"patch path must not contain '..': {path}")
return normalized
def _lines_to_text(lines: list[str]) -> str:
if not lines:
return ""
return "\n".join(lines) + "\n"
def _text_line_count(text: str) -> int:
if not text:
return 0
return len(text.splitlines())
def _line_diff_stats(before: str, after: str) -> tuple[int, int]:
before_lines = before.replace("\r\n", "\n").splitlines()
after_lines = after.replace("\r\n", "\n").splitlines()
added = 0
deleted = 0
matcher = difflib.SequenceMatcher(a=before_lines, b=after_lines, autojunk=False)
for tag, i1, i2, j1, j2 in matcher.get_opcodes():
if tag == "equal":
continue
if tag in ("replace", "delete"):
deleted += i2 - i1
if tag in ("replace", "insert"):
added += j2 - j1
return added, deleted
def _format_summary(summary: _PatchSummary) -> str:
stats = ""
if summary.added or summary.deleted:
stats = f" (+{summary.added}/-{summary.deleted})"
return f"- {summary.action} {summary.path}{stats}"
@tool_parameters(
tool_parameters_schema(
edits=ArraySchema(
items=ObjectSchema(
path=StringSchema("Relative path to the file to edit."),
action=StringSchema(
"Operation type: replace (find and replace text), add (append new content or create file), delete (remove text).",
enum=["replace", "add", "delete"],
),
old_text=StringSchema(
"Exact text to search for in the file. Required for replace and delete.",
nullable=True,
),
new_text=StringSchema(
"Text to replace with or append. Required for replace and add.",
nullable=True,
),
required=["path", "action"],
),
description="List of edits to apply. Each edit specifies a file and the change to make.",
min_items=1,
max_items=20,
),
dry_run=BooleanSchema(
description="Validate and summarize the patch without writing files.",
default=False,
),
required=["edits"],
)
)
class ApplyPatchTool(_FsTool):
"""Apply file edits by providing structured edit instructions."""
_scopes = {"core", "subagent"}
@property
def name(self) -> str:
return "apply_patch"
@property
def description(self) -> str:
return (
"Default tool for code edits. Supports multi-file changes in a single call. "
"Provide a list of structured edits, each specifying a file path, action (replace/add/delete), and the text to change. "
"Paths must be relative. Set dry_run=true to validate and preview without writing files. "
"Use edit_file only for small exact replacements on a single file."
)
async def execute(
self,
edits: list[dict] | None = None,
dry_run: bool = False,
**kwargs: Any,
) -> str:
try:
if not edits:
raise _PatchError("must provide edits")
writes: dict[Path, str] = {}
deletes: set[Path] = set()
summaries: list[_PatchSummary] = []
for edit in edits:
if not isinstance(edit, dict):
raise _PatchError("each edit must be an object")
raw_path = edit.get("path")
if not isinstance(raw_path, str):
raise _PatchError("path required for edit")
path = _validate_relative_path(raw_path)
action = edit.get("action")
if not isinstance(action, str):
raise _PatchError(f"action required for edit: {path}")
source = self._resolve(path)
if action == "add":
new_text = edit.get("new_text")
if new_text is None:
raise _PatchError(f"new_text required for add: {path}")
pending = writes.get(source)
if pending is not None:
content = pending
exists = True
elif source.exists():
raw = source.read_bytes()
try:
content = raw.decode("utf-8")
except UnicodeDecodeError:
raise _PatchError(f"file is not UTF-8 text: {path}")
exists = True
else:
content = ""
exists = False
if exists:
uses_crlf = "\r\n" in content
new_norm = content.replace("\r\n", "\n") + new_text.replace("\r\n", "\n")
if new_norm and not new_norm.endswith("\n"):
new_norm += "\n"
if uses_crlf:
new_norm = new_norm.replace("\n", "\r\n")
writes[source] = new_norm
deletes.discard(source)
added, deleted = _line_diff_stats(content, new_norm)
action_name = "update"
else:
new_norm = new_text.replace("\r\n", "\n")
if new_norm and not new_norm.endswith("\n"):
new_norm += "\n"
writes[source] = new_norm
deletes.discard(source)
added = _text_line_count(new_norm)
deleted = 0
action_name = "add"
summaries.append(
_PatchSummary(
action=action_name, path=path, added=added, deleted=deleted
)
)
elif action == "replace":
old_text = edit.get("old_text") or ""
if not old_text:
raise _PatchError(f"old_text required for replace: {path}")
new_text = edit.get("new_text")
if new_text is None:
raise _PatchError(f"new_text required for replace: {path}")
pending = writes.get(source)
if pending is not None:
content = pending
elif source.exists():
raw = source.read_bytes()
try:
content = raw.decode("utf-8")
except UnicodeDecodeError:
raise _PatchError(f"file is not UTF-8 text: {path}")
else:
raise _PatchError(f"file to update does not exist: {path}")
if pending is None and not source.is_file():
raise _PatchError(f"path to update is not a file: {path}")
uses_crlf = "\r\n" in content
norm_content = content.replace("\r\n", "\n")
norm_old = old_text.replace("\r\n", "\n")
pos = norm_content.find(norm_old)
if pos < 0:
raise _PatchError(f"old_text not found in {path}")
if norm_content.find(norm_old, pos + 1) >= 0:
raise _PatchError(f"old_text appears multiple times in {path}")
new_norm = (
norm_content[:pos]
+ new_text.replace("\r\n", "\n")
+ norm_content[pos + len(norm_old) :]
)
if new_norm and not new_norm.endswith("\n"):
new_norm += "\n"
if uses_crlf:
new_norm = new_norm.replace("\n", "\r\n")
writes[source] = new_norm
deletes.discard(source)
added, deleted = _line_diff_stats(content, new_norm)
summaries.append(
_PatchSummary(
action="update", path=path, added=added, deleted=deleted
)
)
elif action == "delete":
old_text = edit.get("old_text") or ""
if not old_text:
raise _PatchError(f"old_text required for delete: {path}")
pending = writes.get(source)
if pending is not None:
content = pending
elif source.exists():
raw = source.read_bytes()
try:
content = raw.decode("utf-8")
except UnicodeDecodeError:
raise _PatchError(f"file is not UTF-8 text: {path}")
else:
raise _PatchError(f"file to update does not exist: {path}")
if pending is None and not source.is_file():
raise _PatchError(f"path to update is not a file: {path}")
uses_crlf = "\r\n" in content
norm_content = content.replace("\r\n", "\n")
norm_old = old_text.replace("\r\n", "\n")
pos = norm_content.find(norm_old)
if pos < 0:
raise _PatchError(f"old_text not found in {path}")
if norm_content.find(norm_old, pos + 1) >= 0:
raise _PatchError(f"old_text appears multiple times in {path}")
if norm_old == norm_content:
deletes.add(source)
writes.pop(source, None)
added, deleted = 0, _text_line_count(content)
summaries.append(
_PatchSummary(
action="delete", path=path, added=added, deleted=deleted
)
)
else:
new_norm = (
norm_content[:pos] + norm_content[pos + len(norm_old) :]
)
if new_norm and not new_norm.endswith("\n"):
new_norm += "\n"
if uses_crlf:
new_norm = new_norm.replace("\n", "\r\n")
writes[source] = new_norm
deletes.discard(source)
added, deleted = _line_diff_stats(content, new_norm)
summaries.append(
_PatchSummary(
action="update", path=path, added=added, deleted=deleted
)
)
else:
raise _PatchError(f"unknown action: {action}")
if dry_run:
return "Patch dry-run succeeded:\n" + "\n".join(
_format_summary(summary) for summary in summaries
)
backups: dict[Path, bytes | None] = {}
for path in set(writes) | deletes:
backups[path] = path.read_bytes() if path.exists() else None
try:
for path in deletes:
if path.exists():
path.unlink()
for path, content in writes.items():
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8", newline="")
except Exception:
for path, data in backups.items():
if data is None:
if path.exists():
path.unlink()
else:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(data)
raise
for path in set(writes) | deletes:
self._file_states.record_write(path)
return "Patch applied:\n" + "\n".join(
_format_summary(summary) for summary in summaries
)
except PermissionError as exc:
return f"Error: {exc}"
except _PatchError as exc:
return f"Error applying patch: {exc}"
except Exception as exc:
return f"Error applying patch: {exc}"
+127
View File
@@ -0,0 +1,127 @@
"""Controlled runner for installed CLI Apps."""
from __future__ import annotations
from pathlib import Path
from typing import Any
from pydantic import Field
from nanobot.agent.tools.base import Tool, tool_parameters
from nanobot.agent.tools.schema import ArraySchema, BooleanSchema, IntegerSchema, StringSchema, tool_parameters_schema
from nanobot.apps.cli import CliAppError, CliAppManager, CliAppsRuntimeConfig
from nanobot.config.schema import Base
class CliAppsToolConfig(Base):
"""CLI Apps tool configuration."""
enable: bool = True
install_timeout: int = Field(default=300, ge=1, le=3600)
run_timeout: int = Field(default=60, ge=1, le=600)
catalog_ttl_seconds: int = Field(default=3600, ge=60, le=86_400)
@tool_parameters(
tool_parameters_schema(
required=["name"],
name=StringSchema("Installed CLI app registry name, for example gimp, safari, or obsidian."),
args=ArraySchema(
StringSchema("One command-line argument."),
description="Arguments to pass to the CLI entry point. Do not include the entry point itself.",
nullable=True,
),
json=BooleanSchema(
description="Whether to prepend --json when supported by the CLI.",
default=False,
nullable=True,
),
working_dir=StringSchema("Optional working directory for the CLI call.", nullable=True),
timeout=IntegerSchema(
description="Timeout in seconds for this CLI call.",
minimum=1,
maximum=600,
nullable=True,
),
)
)
class CliAppsTool(Tool):
"""Run an installed CLI-Anything or public CLI app through a controlled argv subprocess."""
config_key = "cli_apps"
_scopes = {"core", "subagent"}
@classmethod
def config_cls(cls):
return CliAppsToolConfig
@classmethod
def enabled(cls, ctx: Any) -> bool:
return ctx.config.cli_apps.enable
@classmethod
def create(cls, ctx: Any) -> Tool:
cfg = ctx.config.cli_apps
return cls(
workspace=Path(ctx.workspace),
restrict_to_workspace=ctx.config.restrict_to_workspace,
runtime=CliAppsRuntimeConfig(
install_timeout=cfg.install_timeout,
run_timeout=cfg.run_timeout,
catalog_ttl_seconds=cfg.catalog_ttl_seconds,
),
)
def __init__(
self,
*,
workspace: Path,
restrict_to_workspace: bool = False,
runtime: CliAppsRuntimeConfig | None = None,
) -> None:
self.workspace = workspace
self.restrict_to_workspace = restrict_to_workspace
self.runtime = runtime or CliAppsRuntimeConfig()
@property
def name(self) -> str:
return "run_cli_app"
@property
def description(self) -> str:
try:
installed = CliAppManager(workspace=self.workspace, runtime=self.runtime).installed_names()
except Exception:
installed = []
installed_note = (
f" Installed Settings CLI Apps: {', '.join(installed)}."
if installed
else " No Settings CLI Apps are currently installed."
)
return (
"Run a CLI App that the user explicitly installed in Settings or attached as @app. "
"Do not use this for ordinary system CLIs such as git, gh, python, npm, or brew; "
"unknown names are rejected. Execution uses argv, not shell."
+ installed_note
)
async def execute(
self,
name: str,
args: list[str] | None = None,
json: bool | None = False,
working_dir: str | None = None,
timeout: int | None = None,
) -> str:
manager = CliAppManager(workspace=self.workspace, runtime=self.runtime)
try:
return manager.run(
name,
args=args or [],
json_output=bool(json),
working_dir=working_dir,
timeout=timeout,
restrict_to_workspace=self.restrict_to_workspace,
)
except CliAppError as exc:
return f"Error: {exc.message}"
+592
View File
@@ -0,0 +1,592 @@
"""Session support for long-running exec workflows."""
from __future__ import annotations
import asyncio
import shutil
import time
import uuid
from contextlib import suppress
from dataclasses import dataclass
from typing import Any
from nanobot.agent.tools.base import Tool, tool_parameters
from nanobot.agent.tools.schema import BooleanSchema, IntegerSchema, StringSchema, tool_parameters_schema
DEFAULT_YIELD_MS = 1000
MAX_YIELD_MS = 30_000
DEFAULT_WAIT_FOR_MS = 10_000
MAX_WAIT_FOR_MS = 120_000
DEFAULT_MAX_OUTPUT_CHARS = 10_000
MAX_OUTPUT_CHARS = 50_000
@dataclass(slots=True)
class _SessionPoll:
output: str
done: bool
exit_code: int | None
elapsed_s: float = 0.0
timed_out: bool = False
terminated: bool = False
stdin_closed: bool = False
truncated_chars: int = 0
@dataclass(slots=True)
class ExecSessionInfo:
session_id: str
command: str
cwd: str
elapsed_s: float
idle_s: float
remaining_s: float
returncode: int | None
class _ExecSession:
def __init__(
self,
*,
session_id: str,
process: asyncio.subprocess.Process,
command: str,
cwd: str,
timeout: int | None,
) -> None:
self.session_id = session_id
self.process = process
self.command = command
self.cwd = cwd
self.started_at = time.monotonic()
# timeout None/0 means no limit; an infinite deadline is never reached.
self.deadline = time.monotonic() + timeout if timeout else float("inf")
self.last_access = time.monotonic()
self._chunks: list[str] = []
self._lock = asyncio.Lock()
self._timed_out = False
self._stdout_task = asyncio.create_task(self._read_stream(process.stdout, ""))
self._stderr_task = asyncio.create_task(self._read_stream(process.stderr, "STDERR:\n"))
async def _read_stream(
self,
stream: asyncio.StreamReader | None,
prefix: str,
) -> None:
if stream is None:
return
first = True
while True:
chunk = await stream.read(4096)
if not chunk:
break
text = chunk.decode("utf-8", errors="replace")
if prefix and first:
text = prefix + text
first = False
async with self._lock:
self._chunks.append(text)
async def write(self, chars: str) -> str | None:
if self.process.returncode is not None:
return "session has already exited"
if self.process.stdin is None:
return "session stdin is not available"
try:
self.process.stdin.write(chars.encode("utf-8"))
await self.process.stdin.drain()
except (BrokenPipeError, ConnectionResetError):
return "session stdin is closed"
return None
async def close_stdin(self) -> str | None:
if self.process.returncode is not None:
return "session has already exited"
if self.process.stdin is None:
return "session stdin is not available"
self.process.stdin.close()
with suppress(BrokenPipeError, ConnectionResetError):
await self.process.stdin.wait_closed()
return None
async def poll(
self,
yield_time_ms: int,
max_output_chars: int,
*,
terminated: bool = False,
stdin_closed: bool = False,
) -> _SessionPoll:
self.last_access = time.monotonic()
if yield_time_ms > 0 and self.process.returncode is None:
await asyncio.sleep(min(yield_time_ms, MAX_YIELD_MS) / 1000)
if self.process.returncode is None and time.monotonic() >= self.deadline:
self._timed_out = True
await self.kill()
if self.process.returncode is not None:
with suppress(asyncio.TimeoutError):
await asyncio.wait_for(
asyncio.gather(self._stdout_task, self._stderr_task),
timeout=2.0,
)
async with self._lock:
output = "".join(self._chunks)
self._chunks.clear()
output, truncated = _truncate_output(output, max_output_chars)
return _SessionPoll(
output=output,
done=self.process.returncode is not None,
exit_code=self.process.returncode,
elapsed_s=max(0.0, time.monotonic() - self.started_at),
timed_out=self._timed_out,
terminated=terminated,
stdin_closed=stdin_closed,
truncated_chars=truncated,
)
async def kill(self) -> None:
if self.process.returncode is not None:
return
self.process.kill()
with suppress(asyncio.TimeoutError):
await asyncio.wait_for(self.process.wait(), timeout=5.0)
class ExecSessionManager:
def __init__(self, *, max_sessions: int = 8, idle_timeout: int = 1800) -> None:
self.max_sessions = max_sessions
self.idle_timeout = idle_timeout
self._sessions: dict[str, _ExecSession] = {}
self._lock = asyncio.Lock()
async def start(
self,
*,
command: str,
cwd: str,
env: dict[str, str],
timeout: int | None,
shell_program: str | None,
login: bool,
yield_time_ms: int,
max_output_chars: int,
) -> tuple[str, _SessionPoll]:
async with self._lock:
await self._cleanup_locked()
if len(self._sessions) >= self.max_sessions:
raise RuntimeError(f"maximum exec sessions reached ({self.max_sessions})")
process = await self._spawn(command, cwd, env, shell_program, login)
session_id = uuid.uuid4().hex[:12]
session = _ExecSession(
session_id=session_id,
process=process,
command=command,
cwd=cwd,
timeout=timeout,
)
self._sessions[session_id] = session
poll = await session.poll(yield_time_ms, max_output_chars)
if poll.done:
async with self._lock:
self._sessions.pop(session_id, None)
return session_id, poll
async def write(
self,
*,
session_id: str,
chars: str | None,
close_stdin: bool,
terminate: bool,
yield_time_ms: int,
max_output_chars: int,
) -> _SessionPoll:
async with self._lock:
await self._cleanup_locked()
session = self._sessions.get(session_id)
if session is None:
raise KeyError(session_id)
if chars:
error = await session.write(chars)
if error:
raise RuntimeError(error)
stdin_closed = False
if close_stdin:
error = await session.close_stdin()
if error:
raise RuntimeError(error)
stdin_closed = True
if terminate:
await session.kill()
poll = await session.poll(
yield_time_ms,
max_output_chars,
terminated=terminate,
stdin_closed=stdin_closed,
)
if poll.done:
async with self._lock:
self._sessions.pop(session_id, None)
return poll
async def list(self) -> list[ExecSessionInfo]:
async with self._lock:
await self._cleanup_locked()
now = time.monotonic()
return [
ExecSessionInfo(
session_id=session_id,
command=session.command,
cwd=session.cwd,
elapsed_s=max(0.0, now - session.started_at),
idle_s=max(0.0, now - session.last_access),
remaining_s=max(0.0, session.deadline - now),
returncode=session.process.returncode,
)
for session_id, session in sorted(self._sessions.items())
]
async def _cleanup_locked(self) -> None:
now = time.monotonic()
stale = [
session_id
for session_id, session in self._sessions.items()
if now - session.last_access > self.idle_timeout
]
for session_id in stale:
session = self._sessions.pop(session_id)
await session.kill()
async def _spawn(
self,
command: str,
cwd: str,
env: dict[str, str],
shell_program: str | None,
login: bool,
) -> asyncio.subprocess.Process:
from nanobot.agent.tools import shell
if shell._IS_WINDOWS:
return await asyncio.create_subprocess_shell(
command,
stdin=asyncio.subprocess.PIPE,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
cwd=cwd,
env=env,
)
shell_program = shell_program or shutil.which("bash") or "/bin/bash"
args = [shell_program]
if login and shell_program.rsplit("/", 1)[-1] in {"bash", "zsh"}:
args.append("-l")
args.extend(["-c", command])
return await asyncio.create_subprocess_exec(
*args,
stdin=asyncio.subprocess.PIPE,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
cwd=cwd,
env=env,
)
DEFAULT_EXEC_SESSION_MANAGER = ExecSessionManager()
def clamp_session_int(value: int | None, default: int, minimum: int, maximum: int) -> int:
if value is None:
return default
return min(max(value, minimum), maximum)
def _truncate_output(output: str, max_output_chars: int) -> tuple[str, int]:
if len(output) <= max_output_chars:
return output, 0
half = max_output_chars // 2
omitted = len(output) - max_output_chars
return (
output[:half]
+ f"\n\n... ({omitted:,} chars truncated) ...\n\n"
+ output[-half:],
omitted,
)
def format_session_poll(session_id: str, poll: _SessionPoll) -> str:
parts = [poll.output] if poll.output else []
if poll.truncated_chars:
parts.append(f"(output truncated by {poll.truncated_chars:,} chars)")
if poll.timed_out:
parts.append("Error: Command timed out; session was terminated.")
if poll.terminated and not poll.timed_out:
parts.append("Session terminated.")
if poll.stdin_closed:
parts.append("Stdin closed.")
if poll.done:
parts.append(f"Exit code: {poll.exit_code}")
else:
parts.append(f"Process running. session_id: {session_id}")
parts.append(f"Elapsed: {poll.elapsed_s:.1f}s")
return "\n".join(parts) if parts else "(no output yet)"
@tool_parameters(
tool_parameters_schema(
session_id=StringSchema("Session id returned by exec when yield_time_ms is used."),
chars=StringSchema(
"Bytes/text to write to stdin. Omit or pass an empty string to only poll recent output.",
nullable=True,
),
close_stdin=BooleanSchema(
description="Close stdin after writing chars. Useful for commands waiting for EOF.",
default=False,
),
terminate=BooleanSchema(
description="Terminate the running exec session.",
default=False,
),
yield_time_ms=IntegerSchema(
DEFAULT_YIELD_MS,
description="Milliseconds to wait before returning recent output (default 1000, max 30000).",
minimum=0,
maximum=MAX_YIELD_MS,
),
wait_for=StringSchema(
"Optional text to wait for in output before returning. "
"Useful for interactive commands and dev servers.",
nullable=True,
),
wait_timeout_ms=IntegerSchema(
DEFAULT_WAIT_FOR_MS,
description="Maximum milliseconds to wait for wait_for text (default 10000, max 120000).",
minimum=0,
maximum=MAX_WAIT_FOR_MS,
nullable=True,
),
max_output_chars=IntegerSchema(
DEFAULT_MAX_OUTPUT_CHARS,
description="Maximum output characters to return from this poll (default 10000, max 50000).",
minimum=1000,
maximum=MAX_OUTPUT_CHARS,
),
max_output_tokens=IntegerSchema(
DEFAULT_MAX_OUTPUT_CHARS,
description="Compatibility alias for max_output_chars. The current runtime uses a character budget.",
minimum=1000,
maximum=MAX_OUTPUT_CHARS,
nullable=True,
),
required=["session_id"],
)
)
class WriteStdinTool(Tool):
"""Write to or poll a running exec session."""
_scopes = {"core", "subagent"}
config_key = "exec"
@classmethod
def config_cls(cls):
from nanobot.agent.tools.shell import ExecToolConfig
return ExecToolConfig
@classmethod
def enabled(cls, ctx: Any) -> bool:
return ctx.config.exec.enable
def __init__(
self,
*,
manager: ExecSessionManager | None = None,
) -> None:
self._manager = manager or DEFAULT_EXEC_SESSION_MANAGER
@classmethod
def create(cls, ctx: Any) -> Tool:
return cls()
@property
def exclusive(self) -> bool:
return True
@property
def name(self) -> str:
return "write_stdin"
@property
def description(self) -> str:
return (
"Interact with a running exec session created by exec with "
"yield_time_ms. Use chars='' to poll without writing, chars to send "
"stdin, close_stdin=true to send EOF, or terminate=true to stop the "
"process. Use wait_for with wait_timeout_ms for dev servers, test "
"watchers, and prompts where you need to wait for expected output. "
"Do not use this to start new commands; start them with exec."
)
async def execute(
self,
session_id: str,
chars: str | None = None,
close_stdin: bool = False,
terminate: bool = False,
yield_time_ms: int | None = None,
wait_for: str | None = None,
wait_timeout_ms: int | None = None,
max_output_chars: int | None = None,
max_output_tokens: int | None = None,
**kwargs: Any,
) -> str:
try:
if max_output_chars is None:
max_output_chars = max_output_tokens
output_limit = clamp_session_int(
max_output_chars,
DEFAULT_MAX_OUTPUT_CHARS,
1000,
MAX_OUTPUT_CHARS,
)
if wait_for:
return await self._wait_for_output(
session_id=session_id,
chars=chars,
close_stdin=close_stdin,
terminate=terminate,
wait_for=wait_for,
wait_timeout_ms=clamp_session_int(
wait_timeout_ms,
DEFAULT_WAIT_FOR_MS,
0,
MAX_WAIT_FOR_MS,
),
max_output_chars=output_limit,
)
poll = await self._manager.write(
session_id=session_id,
chars=chars,
close_stdin=close_stdin,
terminate=terminate,
yield_time_ms=clamp_session_int(yield_time_ms, DEFAULT_YIELD_MS, 0, MAX_YIELD_MS),
max_output_chars=output_limit,
)
return format_session_poll(session_id, poll)
except KeyError:
return f"Error: exec session not found: {session_id}"
except Exception as exc:
return f"Error writing to exec session: {exc}"
async def _wait_for_output(
self,
*,
session_id: str,
chars: str | None,
close_stdin: bool,
terminate: bool,
wait_for: str,
wait_timeout_ms: int,
max_output_chars: int,
) -> str:
deadline = time.monotonic() + (wait_timeout_ms / 1000)
aggregate: list[str] = []
first = True
poll: _SessionPoll | None = None
while True:
remaining_ms = max(0, int((deadline - time.monotonic()) * 1000))
step_ms = min(500, remaining_ms)
poll = await self._manager.write(
session_id=session_id,
chars=chars if first else None,
close_stdin=close_stdin if first else False,
terminate=terminate if first else False,
yield_time_ms=step_ms,
max_output_chars=max_output_chars,
)
first = False
if poll.output:
aggregate.append(poll.output)
joined = "".join(aggregate)
if wait_for in joined:
poll.output = joined
return format_session_poll(session_id, poll)
if poll.done or remaining_ms <= 0:
poll.output = "".join(aggregate)
result = format_session_poll(session_id, poll)
if wait_for not in poll.output:
result += f"\nWait target not observed: {wait_for!r}"
return result
@tool_parameters(tool_parameters_schema())
class ListExecSessionsTool(Tool):
"""List active exec sessions."""
_scopes = {"core", "subagent"}
config_key = "exec"
@classmethod
def config_cls(cls):
from nanobot.agent.tools.shell import ExecToolConfig
return ExecToolConfig
@classmethod
def enabled(cls, ctx: Any) -> bool:
return ctx.config.exec.enable
def __init__(
self,
*,
manager: ExecSessionManager | None = None,
) -> None:
self._manager = manager or DEFAULT_EXEC_SESSION_MANAGER
@classmethod
def create(cls, ctx: Any) -> Tool:
return cls()
@property
def name(self) -> str:
return "list_exec_sessions"
@property
def description(self) -> str:
return (
"List active long-running exec sessions, including session_id, cwd, "
"elapsed time, idle time, remaining timeout, and command preview. "
"Use this to recover a session_id after context shifts before "
"polling, writing stdin, or terminating with write_stdin."
)
@property
def read_only(self) -> bool:
return True
async def execute(self, **kwargs: Any) -> str:
try:
sessions = await self._manager.list()
if not sessions:
return "No active exec sessions."
lines = []
for info in sessions:
command = " ".join(info.command.split())
if len(command) > 120:
command = command[:119] + "..."
status = "exited" if info.returncode is not None else "running"
lines.append(
f"{info.session_id} | {status} | elapsed={info.elapsed_s:.1f}s "
f"| idle={info.idle_s:.1f}s | remaining={info.remaining_s:.1f}s "
f"| cwd={info.cwd} | {command}"
)
return "\n".join(lines)
except Exception as exc:
return f"Error listing exec sessions: {exc}"
+106 -22
View File
@@ -132,6 +132,10 @@ def _parse_page_range(pages: str, total: int) -> tuple[int, int]:
minimum=1,
),
pages=StringSchema("Page range for PDF files, e.g. '1-5' (default: all, max 20 pages)"),
force=BooleanSchema(
description="Bypass same-file read deduplication and return content again.",
default=False,
),
required=["path"],
)
)
@@ -154,7 +158,11 @@ class ReadFileTool(_FsTool):
"Text output format: LINE_NUM|CONTENT. "
"Images return visual content for analysis. "
"Supports PDF, DOCX, XLSX, PPTX documents. "
"Use find_files/list_dir first when the path is uncertain. "
"Read the relevant range before editing so replacements or patches "
"are based on current content. "
"Use offset and limit for large text files. "
"Use force=true to re-read content even if unchanged. "
"Reads exceeding ~128K chars are truncated."
)
@@ -162,7 +170,15 @@ class ReadFileTool(_FsTool):
def read_only(self) -> bool:
return True
async def execute(self, path: str | None = None, offset: int = 1, limit: int | None = None, pages: str | None = None, **kwargs: Any) -> Any:
async def execute(
self,
path: str | None = None,
offset: int = 1,
limit: int | None = None,
pages: str | None = None,
force: bool = False,
**kwargs: Any,
) -> Any:
try:
if not path:
return "Error reading file: Unknown path"
@@ -202,7 +218,13 @@ class ReadFileTool(_FsTool):
current_mtime = os.path.getmtime(fp)
except OSError:
current_mtime = 0.0
if entry and entry.can_dedup and entry.offset == offset and entry.limit == limit:
if (
not force
and entry
and entry.can_dedup
and entry.offset == offset
and entry.limit == limit
):
if current_mtime != entry.mtime:
# File was modified externally - force full read and mark as not dedupable
entry.can_dedup = False
@@ -365,9 +387,10 @@ class WriteFileTool(_FsTool):
@property
def description(self) -> str:
return (
"Write content to a file. Overwrites if the file already exists; "
"creates parent directories as needed. "
"For partial edits, prefer edit_file instead."
"Create a new file or intentionally replace an entire file with "
"the provided content. Overwrites existing files and creates parent "
"directories as needed. For code changes or partial edits, prefer "
"apply_patch; use edit_file only for small exact replacements."
)
async def execute(self, path: str | None = None, content: str | None = None, **kwargs: Any) -> str:
@@ -657,6 +680,24 @@ def _find_match(content: str, old_text: str) -> tuple[str | None, int]:
old_text=StringSchema("The text to find and replace"),
new_text=StringSchema("The text to replace with"),
replace_all=BooleanSchema(description="Replace all occurrences (default false)"),
occurrence=IntegerSchema(
1,
description="Optional 1-based occurrence to replace when old_text appears multiple times.",
minimum=1,
nullable=True,
),
line_hint=IntegerSchema(
1,
description="Optional 1-based line hint used to choose the nearest match.",
minimum=1,
nullable=True,
),
expected_replacements=IntegerSchema(
1,
description="Optional guard for the number of replacements that must be made.",
minimum=1,
nullable=True,
),
required=["path", "old_text", "new_text"],
)
)
@@ -674,10 +715,13 @@ class EditFileTool(_FsTool):
@property
def description(self) -> str:
return (
"Edit a file by replacing old_text with new_text. "
"Tolerates minor whitespace/indentation differences and curly/straight quote mismatches. "
"If old_text matches multiple times, you must provide more context "
"or set replace_all=true. Shows a diff of the closest match on failure."
"Perform a small, exact replacement in one file by replacing "
"old_text with new_text. Use this for narrow text substitutions "
"with old_text copied from read_file. For multi-file, structural, "
"or generated code edits, prefer apply_patch. If old_text matches "
"multiple times, provide more context or set occurrence, line_hint, "
"replace_all, and expected_replacements. Shows closest-match "
"diagnostics on failure."
)
@staticmethod
@@ -688,7 +732,8 @@ class EditFileTool(_FsTool):
async def execute(
self, path: str | None = None, old_text: str | None = None,
new_text: str | None = None,
replace_all: bool = False, **kwargs: Any,
replace_all: bool = False, occurrence: int | None = None,
line_hint: int | None = None, expected_replacements: int | None = None, **kwargs: Any,
) -> str:
try:
if not path:
@@ -697,10 +742,12 @@ class EditFileTool(_FsTool):
raise ValueError("Unknown old_text")
if new_text is None:
raise ValueError("Unknown new_text")
# .ipynb detection
if path.endswith(".ipynb"):
return "Error: This is a Jupyter notebook. Use the notebook_edit tool instead of edit_file."
if occurrence is not None and occurrence < 1:
return "Error: occurrence must be >= 1."
if line_hint is not None and line_hint < 1:
return "Error: line_hint must be >= 1."
if expected_replacements is not None and expected_replacements < 1:
return "Error: expected_replacements must be >= 1."
fp = self._resolve(path)
@@ -743,15 +790,42 @@ class EditFileTool(_FsTool):
if not matches:
return self._not_found_msg(old_text, content, path)
count = len(matches)
if replace_all and occurrence is not None:
return "Error: occurrence cannot be used with replace_all=true."
if replace_all and line_hint is not None:
return "Error: line_hint cannot be used with replace_all=true."
if occurrence is not None and line_hint is not None:
return "Error: line_hint cannot be used with occurrence."
if count > 1 and not replace_all:
line_numbers = [match.line for match in matches]
preview = ", ".join(f"line {n}" for n in line_numbers[:3])
if len(line_numbers) > 3:
preview += ", ..."
location_hint = f" at {preview}" if preview else ""
if occurrence is not None:
if occurrence > count:
return (
f"Error: occurrence {occurrence} is out of range; "
f"old_text appears {count} times."
)
elif line_hint is not None:
nearest = min(matches, key=lambda match: abs(match.line - line_hint))
distance = abs(nearest.line - line_hint)
if sum(1 for match in matches if abs(match.line - line_hint) == distance) > 1:
return (
f"Error: line_hint {line_hint} is ambiguous; "
f"old_text appears {count} times."
)
else:
line_numbers = [match.line for match in matches]
preview = ", ".join(f"line {n}" for n in line_numbers[:3])
if len(line_numbers) > 3:
preview += ", ..."
location_hint = f" at {preview}" if preview else ""
return (
f"Warning: old_text appears {count} times{location_hint}. "
"Provide more context, set occurrence to choose one match, "
"or set replace_all=true."
)
elif occurrence is not None and occurrence > count:
return (
f"Warning: old_text appears {count} times{location_hint}. "
"Provide more context to make it unique, or set replace_all=true."
f"Error: occurrence {occurrence} is out of range; "
f"old_text appears {count} time."
)
norm_new = new_text.replace("\r\n", "\n")
@@ -760,7 +834,17 @@ class EditFileTool(_FsTool):
if fp.suffix.lower() not in self._MARKDOWN_EXTS:
norm_new = self._strip_trailing_ws(norm_new)
selected = matches if replace_all else matches[:1]
if replace_all:
selected = matches
elif line_hint is not None:
selected = [min(matches, key=lambda match: abs(match.line - line_hint))]
else:
selected = [matches[occurrence - 1 if occurrence else 0]]
if expected_replacements is not None and len(selected) != expected_replacements:
return (
f"Error: expected {expected_replacements} replacements but "
f"would make {len(selected)}."
)
new_content = content
for match in reversed(selected):
replacement = _preserve_quote_style(norm_old, match.text, norm_new)
-9
View File
@@ -130,12 +130,6 @@ class ImageGenerationTool(Tool):
}
return cls(**kwargs)
def _missing_api_key_error(self) -> str:
cls = get_image_gen_provider(self.config.provider)
if cls and cls.missing_key_message:
return f"Error: {cls.missing_key_message}"
return f"Error: {self.config.provider} API key is not configured."
def _resolve_reference_image(self, value: str) -> str:
raw_path = Path(value).expanduser()
path = raw_path if raw_path.is_absolute() else self.workspace / raw_path
@@ -173,9 +167,6 @@ class ImageGenerationTool(Tool):
client = self._provider_client()
if client is None:
return f"Error: unsupported image generation provider '{self.config.provider}'"
provider = self._provider_config()
if not provider or not provider.api_key:
return self._missing_api_key_error()
requested = count or 1
if requested > self.config.max_images_per_turn:
+279 -1
View File
@@ -6,13 +6,20 @@ import re
import shutil
import urllib.parse
from contextlib import AsyncExitStack, suppress
from typing import Any
from typing import Any, Mapping
from weakref import WeakKeyDictionary
import httpx
from loguru import logger
from nanobot.agent.tools.base import Tool
from nanobot.agent.tools.registry import ToolRegistry
from nanobot.bus.events import (
INBOUND_META_RUNTIME_CONTROL,
RUNTIME_CONTROL_ACK,
RUNTIME_CONTROL_MCP_RELOAD,
InboundMessage,
)
# Transient connection errors that warrant a single retry.
# These typically happen when an MCP server restarts or a network
@@ -33,6 +40,7 @@ _WINDOWS_SHELL_LAUNCHERS: frozenset[str] = frozenset(("npx", "npm", "pnpm", "yar
# Characters allowed in tool names by model providers (Anthropic, OpenAI, etc.).
# Replace anything outside [a-zA-Z0-9_-] with underscore and collapse runs.
_SANITIZE_RE = re.compile(r"_+")
_RELOAD_LOCKS: WeakKeyDictionary[Any, asyncio.Lock] = WeakKeyDictionary()
def _sanitize_name(name: str) -> str:
@@ -503,6 +511,7 @@ async def connect_mcp_servers(
command=command,
args=args,
env=env,
cwd=cfg.cwd or None,
)
read, write = await server_stack.enter_async_context(stdio_client(params))
elif transport_type == "sse":
@@ -662,3 +671,272 @@ async def connect_mcp_servers(
server_stacks[result[0]] = result[1]
return server_stacks
def session_extra(metadata: Mapping[str, Any] | None) -> dict[str, Any]:
"""Return persisted session kwargs for MCP preset attachments."""
mcp_presets = metadata.get("mcp_presets") if isinstance(metadata, Mapping) else None
return {"mcp_presets": mcp_presets} if isinstance(mcp_presets, list) and mcp_presets else {}
def runtime_lines(
message: Any,
*,
available_server_names: set[str] | None = None,
configured_server_names: set[str] | None = None,
connected_server_names: set[str] | None = None,
skip: bool = False,
) -> list[str]:
"""Return model-visible MCP preset annotations for the current turn."""
if skip:
return []
if configured_server_names is None:
configured_server_names = available_server_names
if connected_server_names is None:
connected_server_names = available_server_names
metadata = message.metadata if isinstance(getattr(message, "metadata", None), Mapping) else None
structured = metadata.get("mcp_presets") if isinstance(metadata, Mapping) else None
if not isinstance(structured, list):
return []
lines: list[str] = []
for item in structured[:8]:
if not isinstance(item, Mapping):
continue
raw_name = str(item.get("name") or "").strip().lower()
if not raw_name:
continue
display = str(item.get("display_name") or raw_name).strip() or raw_name
transport = str(item.get("transport") or "mcp").strip() or "mcp"
prefix = f"mcp_{raw_name}_"
if configured_server_names is not None and raw_name not in configured_server_names:
lines.append(
"MCP Preset Attachment: "
f"@{raw_name} ({display}; transport={transport}) is configured in WebUI Settings, "
"but this gateway has not loaded the latest MCP settings yet. "
f"Tools with prefix `{prefix}` may not be available yet; if they are missing, "
"tell the user to restart nanobot."
)
continue
if connected_server_names is not None and raw_name not in connected_server_names:
lines.append(
"MCP Preset Attachment: "
f"@{raw_name} ({display}; transport={transport}) is configured, "
"but its MCP connection is not currently live. "
f"Tools with prefix `{prefix}` may be unavailable; tell the user to open Settings, "
"run the preset test, and restart nanobot only if hot reload is unavailable."
)
continue
lines.append(
"MCP Preset Attachment: "
f"@{raw_name} ({display}; transport={transport}; tool_prefix={prefix}). "
f"Prefer available tools whose names start with `{prefix}` for this request; "
"do not substitute shell commands for this MCP integration unless the user asks."
)
return lines
async def connect_missing_servers(state: Any, registry: ToolRegistry) -> None:
"""Connect configured MCP servers that are not currently live."""
missing_servers = {
name: cfg for name, cfg in state._mcp_servers.items() if name not in state._mcp_stacks
}
if state._mcp_connecting or not missing_servers:
return
state._mcp_connecting = True
try:
connected = await connect_mcp_servers(missing_servers, registry)
state._mcp_stacks.update(connected)
state._mcp_connected = bool(state._mcp_stacks)
if connected:
logger.info("MCP connected servers: {}", sorted(connected))
else:
logger.warning("No MCP servers connected successfully (will retry next message)")
except asyncio.CancelledError:
logger.warning("MCP connection cancelled (will retry next message)")
state._mcp_connected = bool(state._mcp_stacks)
except BaseException as e:
logger.warning("Failed to connect MCP servers (will retry next message): {}", e)
state._mcp_connected = bool(state._mcp_stacks)
finally:
state._mcp_connecting = False
async def reload_servers(state: Any, registry: ToolRegistry) -> dict[str, Any]:
"""Reconcile live MCP connections with the current config file."""
async with _reload_lock(state):
try:
from nanobot.config.loader import (load_config,
resolve_config_env_vars)
config = resolve_config_env_vars(load_config())
next_servers = dict(config.tools.mcp_servers)
except Exception as exc:
logger.warning("MCP hot reload could not read config: {}", exc)
return {
"ok": False,
"message": "Could not reload MCP config. Restart nanobot to pick up changes.",
"requires_restart": True,
"error": str(exc),
}
current_servers = dict(state._mcp_servers)
current_names = set(current_servers)
next_names = set(next_servers)
removed = sorted(current_names - next_names)
added = sorted(next_names - current_names)
changed = sorted(
name
for name in current_names & next_names
if _server_signature(current_servers[name]) != _server_signature(next_servers[name])
)
tools_removed = 0
for name in [*removed, *changed]:
tools_removed += _unregister_server_tools(state, registry, name)
await _close_server(state, name)
state._mcp_servers = next_servers
retry_missing = sorted(
name
for name in next_names
if name not in state._mcp_stacks and name not in set(added) | set(changed)
)
to_connect_names = sorted(set(added) | set(changed) | set(retry_missing))
to_connect = {name: next_servers[name] for name in to_connect_names}
connected: dict[str, AsyncExitStack] = {}
if to_connect:
connected = await connect_mcp_servers(to_connect, registry)
state._mcp_stacks.update(connected)
state._mcp_connected = bool(state._mcp_stacks)
failed = sorted(set(to_connect) - set(connected))
unchanged = not removed and not added and not changed and not retry_missing
ok = not failed
if failed:
message = "MCP config reloaded, but some servers did not connect: " + ", ".join(failed)
elif unchanged:
message = "MCP config is already live."
elif retry_missing and not added and not changed and not removed:
message = "MCP connections refreshed without restarting nanobot."
else:
message = "MCP config reloaded without restarting nanobot."
logger.info(
"MCP hot reload: added={} changed={} removed={} retried={} connected={} failed={} tools_removed={}",
added,
changed,
removed,
retry_missing,
sorted(connected),
failed,
tools_removed,
)
return {
"ok": ok,
"message": message,
"added": added,
"changed": changed,
"removed": removed,
"retried": retry_missing,
"connected": sorted(state._mcp_stacks),
"configured": sorted(state._mcp_servers),
"failed": failed,
"tools_removed": tools_removed,
"requires_restart": False,
}
async def request_mcp_reload(bus: Any, *, timeout: float = 15.0) -> dict[str, Any]:
"""Ask the running agent loop to reconcile live MCP connections."""
loop = asyncio.get_running_loop()
ack: asyncio.Future[dict[str, Any]] = loop.create_future()
await bus.publish_inbound(
InboundMessage(
channel="system",
sender_id="webui-settings",
chat_id="runtime",
content=RUNTIME_CONTROL_MCP_RELOAD,
metadata={
INBOUND_META_RUNTIME_CONTROL: RUNTIME_CONTROL_MCP_RELOAD,
RUNTIME_CONTROL_ACK: ack,
},
)
)
try:
result = await asyncio.wait_for(ack, timeout=timeout)
except asyncio.TimeoutError:
return {
"ok": False,
"message": "MCP hot reload timed out. Restart nanobot to pick up changes.",
"requires_restart": True,
}
return result if isinstance(result, dict) else {
"ok": False,
"message": "MCP hot reload returned an unexpected response.",
"requires_restart": True,
}
async def handle_runtime_control(state: Any, msg: InboundMessage, registry: ToolRegistry) -> bool:
metadata = msg.metadata if isinstance(msg.metadata, dict) else {}
control = metadata.get(INBOUND_META_RUNTIME_CONTROL)
if control != RUNTIME_CONTROL_MCP_RELOAD:
return False
ack = metadata.get(RUNTIME_CONTROL_ACK)
try:
result = await reload_servers(state, registry)
except Exception as exc:
logger.exception("MCP hot reload failed")
result = {
"ok": False,
"message": "MCP hot reload failed. Restart nanobot to pick up changes.",
"requires_restart": True,
"error": str(exc),
}
if isinstance(ack, asyncio.Future) and not ack.done():
ack.set_result(result)
return True
def _reload_lock(state: Any) -> asyncio.Lock:
try:
return _RELOAD_LOCKS[state]
except KeyError:
lock = asyncio.Lock()
_RELOAD_LOCKS[state] = lock
return lock
def _server_signature(cfg: Any) -> Any:
if hasattr(cfg, "model_dump"):
return cfg.model_dump(mode="json")
return cfg
def _tool_prefix(server_name: str) -> str:
safe_name = "".join(ch if ch.isalnum() or ch in {"_", "-"} else "_" for ch in server_name)
while "__" in safe_name:
safe_name = safe_name.replace("__", "_")
return f"mcp_{safe_name}_"
def _unregister_server_tools(state: Any, registry: ToolRegistry, server_name: str) -> int:
prefix = _tool_prefix(server_name)
removed = 0
for tool_name in list(registry.tool_names):
if tool_name.startswith(prefix):
registry.unregister(tool_name)
removed += 1
return removed
async def _close_server(state: Any, server_name: str) -> None:
stack = state._mcp_stacks.pop(server_name, None)
if stack is None:
return
try:
await stack.aclose()
except (RuntimeError, BaseExceptionGroup):
logger.debug("MCP server '{}' cleanup error (can be ignored)", server_name)
-162
View File
@@ -1,162 +0,0 @@
"""NotebookEditTool — edit Jupyter .ipynb notebooks."""
from __future__ import annotations
import json
import uuid
from typing import Any
from nanobot.agent.tools.base import tool_parameters
from nanobot.agent.tools.schema import IntegerSchema, StringSchema, tool_parameters_schema
from nanobot.agent.tools.filesystem import _FsTool
def _new_cell(source: str, cell_type: str = "code", generate_id: bool = False) -> dict:
cell: dict[str, Any] = {
"cell_type": cell_type,
"source": source,
"metadata": {},
}
if cell_type == "code":
cell["outputs"] = []
cell["execution_count"] = None
if generate_id:
cell["id"] = uuid.uuid4().hex[:8]
return cell
def _make_empty_notebook() -> dict:
return {
"nbformat": 4,
"nbformat_minor": 5,
"metadata": {
"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"},
"language_info": {"name": "python"},
},
"cells": [],
}
@tool_parameters(
tool_parameters_schema(
path=StringSchema("Path to the .ipynb notebook file"),
cell_index=IntegerSchema(0, description="0-based index of the cell to edit", minimum=0),
new_source=StringSchema("New source content for the cell"),
cell_type=StringSchema(
"Cell type: 'code' or 'markdown' (default: code)",
enum=["code", "markdown"],
),
edit_mode=StringSchema(
"Mode: 'replace' (default), 'insert' (after target), or 'delete'",
enum=["replace", "insert", "delete"],
),
required=["path", "cell_index"],
)
)
class NotebookEditTool(_FsTool):
"""Edit Jupyter notebook cells: replace, insert, or delete."""
_scopes = {"core"}
_VALID_CELL_TYPES = frozenset({"code", "markdown"})
_VALID_EDIT_MODES = frozenset({"replace", "insert", "delete"})
@property
def name(self) -> str:
return "notebook_edit"
@property
def description(self) -> str:
return (
"Edit a Jupyter notebook (.ipynb) cell. "
"Modes: replace (default) replaces cell content, "
"insert adds a new cell after the target index, "
"delete removes the cell at the index. "
"cell_index is 0-based."
)
async def execute(
self,
path: str | None = None,
cell_index: int = 0,
new_source: str = "",
cell_type: str = "code",
edit_mode: str = "replace",
**kwargs: Any,
) -> str:
try:
if not path:
return "Error: path is required"
if not path.endswith(".ipynb"):
return "Error: notebook_edit only works on .ipynb files. Use edit_file for other files."
if edit_mode not in self._VALID_EDIT_MODES:
return (
f"Error: Invalid edit_mode '{edit_mode}'. "
"Use one of: replace, insert, delete."
)
if cell_type not in self._VALID_CELL_TYPES:
return (
f"Error: Invalid cell_type '{cell_type}'. "
"Use one of: code, markdown."
)
fp = self._resolve(path)
# Create new notebook if file doesn't exist and mode is insert
if not fp.exists():
if edit_mode != "insert":
return f"Error: File not found: {path}"
nb = _make_empty_notebook()
cell = _new_cell(new_source, cell_type, generate_id=True)
nb["cells"].append(cell)
fp.parent.mkdir(parents=True, exist_ok=True)
fp.write_text(json.dumps(nb, indent=1, ensure_ascii=False), encoding="utf-8")
return f"Successfully created {fp} with 1 cell"
try:
nb = json.loads(fp.read_text(encoding="utf-8"))
except (json.JSONDecodeError, UnicodeDecodeError) as e:
return f"Error: Failed to parse notebook: {e}"
cells = nb.get("cells", [])
nbformat_minor = nb.get("nbformat_minor", 0)
generate_id = nb.get("nbformat", 0) >= 4 and nbformat_minor >= 5
if edit_mode == "delete":
if cell_index < 0 or cell_index >= len(cells):
return f"Error: cell_index {cell_index} out of range (notebook has {len(cells)} cells)"
cells.pop(cell_index)
nb["cells"] = cells
fp.write_text(json.dumps(nb, indent=1, ensure_ascii=False), encoding="utf-8")
return f"Successfully deleted cell {cell_index} from {fp}"
if edit_mode == "insert":
insert_at = min(cell_index + 1, len(cells))
cell = _new_cell(new_source, cell_type, generate_id=generate_id)
cells.insert(insert_at, cell)
nb["cells"] = cells
fp.write_text(json.dumps(nb, indent=1, ensure_ascii=False), encoding="utf-8")
return f"Successfully inserted cell at index {insert_at} in {fp}"
# Default: replace
if cell_index < 0 or cell_index >= len(cells):
return f"Error: cell_index {cell_index} out of range (notebook has {len(cells)} cells)"
cells[cell_index]["source"] = new_source
if cell_type and cells[cell_index].get("cell_type") != cell_type:
cells[cell_index]["cell_type"] = cell_type
if cell_type == "code":
cells[cell_index].setdefault("outputs", [])
cells[cell_index].setdefault("execution_count", None)
elif "outputs" in cells[cell_index]:
del cells[cell_index]["outputs"]
cells[cell_index].pop("execution_count", None)
nb["cells"] = cells
fp.write_text(json.dumps(nb, indent=1, ensure_ascii=False), encoding="utf-8")
return f"Successfully edited cell {cell_index} in {fp}"
except PermissionError as e:
return f"Error: {e}"
except Exception as e:
return f"Error editing notebook: {e}"
+169 -2
View File
@@ -1,4 +1,4 @@
"""Search tools: grep."""
"""Search tools: file discovery and grep."""
from __future__ import annotations
@@ -12,6 +12,7 @@ from typing import Any, Iterable, TypeVar
from nanobot.agent.tools.filesystem import ListDirTool, _FsTool
_DEFAULT_HEAD_LIMIT = 250
_DEFAULT_FILE_HEAD_LIMIT = 200
T = TypeVar("T")
_TYPE_GLOB_MAP = {
"py": ("*.py", "*.pyi"),
@@ -88,6 +89,14 @@ def _matches_type(name: str, file_type: str | None) -> bool:
return any(fnmatch.fnmatch(name.lower(), pattern.lower()) for pattern in patterns)
def _matches_query(rel_path: str, query: str | None) -> bool:
if not query:
return True
haystack = rel_path.lower()
terms = [part for part in query.lower().split() if part]
return all(term in haystack for term in terms)
class _SearchTool(_FsTool):
_IGNORE_DIRS = set(ListDirTool._IGNORE_DIRS)
@@ -109,6 +118,163 @@ class _SearchTool(_FsTool):
yield current / filename
class FindFilesTool(_SearchTool):
"""Find files by path fragment, glob, or type."""
_scopes = {"core", "subagent"}
@property
def name(self) -> str:
return "find_files"
@property
def description(self) -> str:
return (
"Find files by path fragment, glob, or file type. "
"Use this before read_file when you need to locate files, and "
"prefer it over shell find/ls for ordinary workspace discovery. "
"Returns workspace-relative paths and skips common dependency/build "
"directories."
)
@property
def read_only(self) -> bool:
return True
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"path": {
"type": "string",
"description": "Directory or file to search in (default '.')",
},
"query": {
"type": "string",
"description": (
"Optional case-insensitive path fragment search. "
"Whitespace-separated terms must all be present."
),
},
"glob": {
"type": "string",
"description": "Optional file filter, e.g. '*.py' or 'tests/**/test_*.py'",
},
"type": {
"type": "string",
"description": "Optional file type shorthand, e.g. 'py', 'ts', 'md', 'json'",
},
"include_dirs": {
"type": "boolean",
"description": "Include matching directories as well as files (default false)",
},
"sort": {
"type": "string",
"enum": ["path", "modified"],
"description": "Sort by path or most recently modified first (default path)",
},
"head_limit": {
"type": "integer",
"description": "Maximum number of paths to return (default 200, 0 for all, max 1000)",
"minimum": 0,
"maximum": 1000,
},
"offset": {
"type": "integer",
"description": "Skip the first N results before applying head_limit",
"minimum": 0,
"maximum": 100000,
},
},
}
def _iter_paths(self, root: Path, *, include_dirs: bool) -> Iterable[Path]:
if root.is_file():
yield root
return
if include_dirs:
yield root
for dirpath, dirnames, filenames in os.walk(root):
dirnames[:] = sorted(d for d in dirnames if d not in self._IGNORE_DIRS)
current = Path(dirpath)
if include_dirs and current != root:
yield current
for filename in sorted(filenames):
yield current / filename
async def execute(
self,
path: str = ".",
query: str | None = None,
glob: str | None = None,
type: str | None = None,
include_dirs: bool = False,
sort: str = "path",
head_limit: int | None = None,
offset: int = 0,
**kwargs: Any,
) -> str:
try:
target = self._resolve(path or ".")
if not target.exists():
return f"Error: Path not found: {path}"
if not (target.is_dir() or target.is_file()):
return f"Error: Unsupported path: {path}"
if sort not in {"path", "modified"}:
return "Error: sort must be 'path' or 'modified'"
limit = (
_DEFAULT_FILE_HEAD_LIMIT
if head_limit is None
else None if head_limit == 0 else head_limit
)
root = target if target.is_dir() else target.parent
matches: list[tuple[str, float]] = []
for candidate in self._iter_paths(target, include_dirs=include_dirs):
if candidate.is_dir() and not include_dirs:
continue
rel_path = candidate.relative_to(root).as_posix()
display_path = self._display_path(candidate, root)
name = candidate.name
if glob and not _match_glob(rel_path, name, glob):
continue
if candidate.is_file() and not _matches_type(name, type):
continue
if candidate.is_dir() and type:
continue
if not _matches_query(display_path, query):
continue
try:
mtime = candidate.stat().st_mtime
except OSError:
mtime = 0.0
suffix = "/" if candidate.is_dir() else ""
matches.append((display_path + suffix, mtime))
if sort == "modified":
matches.sort(key=lambda item: (-item[1], item[0]))
else:
matches.sort(key=lambda item: item[0])
paths = [item[0] for item in matches]
paged, truncated = _paginate(paths, limit, offset)
if not paged:
return "No files found"
result = "\n".join(paged)
note = _pagination_note(limit, offset, truncated)
if note:
result += "\n\n" + note
return result
except PermissionError as e:
return f"Error: {e}"
except Exception as e:
return f"Error finding files: {e}"
class GrepTool(_SearchTool):
"""Search file contents using a regex-like pattern."""
_scopes = {"core", "subagent"}
@@ -125,7 +291,8 @@ class GrepTool(_SearchTool):
return (
"Search file contents with a regex pattern. "
"Default output_mode is files_with_matches (file paths only); "
"use content mode for matching lines with context. "
"use content mode for matching lines with context. Prefer this "
"over shell grep for ordinary workspace searches. "
"Skips binary and files >2 MB. Supports glob/type filtering."
)
+237 -53
View File
@@ -8,6 +8,7 @@ import re
import shutil
import sys
from contextlib import suppress
from dataclasses import dataclass
from pathlib import Path
from typing import Any
@@ -15,8 +16,17 @@ from loguru import logger
from pydantic import Field
from nanobot.agent.tools.base import Tool, tool_parameters
from nanobot.agent.tools.exec_session import (
DEFAULT_MAX_OUTPUT_CHARS,
DEFAULT_YIELD_MS,
DEFAULT_EXEC_SESSION_MANAGER,
MAX_OUTPUT_CHARS,
MAX_YIELD_MS,
clamp_session_int,
format_session_poll,
)
from nanobot.agent.tools.sandbox import wrap_command
from nanobot.agent.tools.schema import IntegerSchema, StringSchema, tool_parameters_schema
from nanobot.agent.tools.schema import BooleanSchema, IntegerSchema, StringSchema, tool_parameters_schema
from nanobot.config.paths import get_media_dir
from nanobot.config.schema import Base
@@ -36,7 +46,7 @@ _WORKSPACE_BOUNDARY_NOTE = (
class ExecToolConfig(Base):
"""Shell exec tool configuration."""
enable: bool = True
timeout: int = 60
timeout: int = Field(default=60, ge=0) # Hard timeout (s); 0 = no limit. Not capped by the per-call max.
path_append: str = ""
sandbox: str = ""
allowed_env_keys: list[str] = Field(default_factory=list)
@@ -44,10 +54,22 @@ class ExecToolConfig(Base):
deny_patterns: list[str] = Field(default_factory=list)
@dataclass(slots=True)
class _PreparedCommand:
command: str
cwd: str
env: dict[str, str]
timeout: int | None
shell_program: str | None
login: bool
@tool_parameters(
tool_parameters_schema(
command=StringSchema("The shell command to execute"),
cmd=StringSchema("Compatibility alias for command"),
working_dir=StringSchema("Optional working directory for the command"),
workdir=StringSchema("Compatibility alias for working_dir"),
timeout=IntegerSchema(
60,
description=(
@@ -57,7 +79,44 @@ class ExecToolConfig(Base):
minimum=1,
maximum=600,
),
required=["command"],
shell=StringSchema(
"Optional shell binary to launch. On Unix, supports sh, bash, or zsh.",
nullable=True,
),
login=BooleanSchema(
description="Whether to run bash/zsh with login shell semantics (default true).",
default=True,
nullable=True,
),
yield_time_ms=IntegerSchema(
description=(
"Optional milliseconds to wait before returning output. "
"When set, a still-running command returns a session_id that "
"can be polled or written to with write_stdin. Omit this field "
"to keep one-shot exec behavior."
),
minimum=0,
maximum=MAX_YIELD_MS,
nullable=True,
),
max_output_chars=IntegerSchema(
description=(
"Maximum output characters to return when yield_time_ms is used "
"(default 10000, max 50000)."
),
minimum=1000,
maximum=MAX_OUTPUT_CHARS,
nullable=True,
),
max_output_tokens=IntegerSchema(
description=(
"Compatibility alias for max_output_chars. The current runtime "
"uses a character budget."
),
minimum=1000,
maximum=MAX_OUTPUT_CHARS,
nullable=True,
),
)
)
class ExecTool(Tool):
@@ -98,6 +157,7 @@ class ExecTool(Tool):
sandbox: str = "",
path_append: str = "",
allowed_env_keys: list[str] | None = None,
session_manager: Any | None = None,
):
self.timeout = timeout
self.working_dir = working_dir
@@ -125,6 +185,7 @@ class ExecTool(Tool):
self.restrict_to_workspace = restrict_to_workspace
self.path_append = path_append
self.allowed_env_keys = allowed_env_keys or []
self._session_manager = session_manager or DEFAULT_EXEC_SESSION_MANAGER
@property
def name(self) -> str:
@@ -150,10 +211,15 @@ class ExecTool(Tool):
def description(self) -> str:
return (
"Execute a shell command and return its output. "
"Prefer read_file/write_file/edit_file over cat/echo/sed, "
"and grep/glob over shell find/grep. "
"Use this for tests, builds, package commands, git commands, and "
"other process execution. Prefer read_file/find_files/grep for "
"inspection and apply_patch/write_file/edit_file for file changes "
"instead of cat, shell find/grep, echo, or sed. "
"Use -y or --yes flags to avoid interactive prompts. "
"Output is truncated at 10 000 chars; timeout defaults to 60s."
"For long-running or interactive commands, pass yield_time_ms; "
"if the command keeps running, exec returns a session_id that can "
"be polled or written to with write_stdin. Output is truncated at "
"10 000 chars; timeout defaults to 60s."
)
@property
@@ -161,9 +227,125 @@ class ExecTool(Tool):
return True
async def execute(
self, command: str, working_dir: str | None = None,
timeout: int | None = None, **kwargs: Any,
self, command: str | None = None, cmd: str | None = None,
working_dir: str | None = None, workdir: str | None = None,
timeout: int | None = None, shell: str | None = None,
login: bool | None = None, yield_time_ms: int | None = None,
max_output_chars: int | None = None,
max_output_tokens: int | None = None,
**kwargs: Any,
) -> str:
command = command or cmd
working_dir = working_dir or workdir
if not command:
return "Error: Missing command. Provide command or cmd."
if max_output_chars is None:
max_output_chars = max_output_tokens
prepared = self._prepare_command(command, working_dir, timeout, shell, login)
if isinstance(prepared, str):
return prepared
if yield_time_ms is not None:
return await self._execute_session(prepared, yield_time_ms, max_output_chars)
try:
process = await self._spawn(
prepared.command,
prepared.cwd,
prepared.env,
prepared.shell_program,
prepared.login,
)
try:
stdout, stderr = await asyncio.wait_for(
process.communicate(),
timeout=prepared.timeout,
)
except asyncio.TimeoutError:
await self._kill_process(process)
return f"Error: Command timed out after {prepared.timeout} seconds"
except asyncio.CancelledError:
await self._kill_process(process)
raise
output_parts = []
if stdout:
output_parts.append(stdout.decode("utf-8", errors="replace"))
if stderr:
stderr_text = stderr.decode("utf-8", errors="replace")
if stderr_text.strip():
output_parts.append(f"STDERR:\n{stderr_text}")
output_parts.append(f"\nExit code: {process.returncode}")
result = "\n".join(output_parts) if output_parts else "(no output)"
max_len = clamp_session_int(max_output_chars, self._MAX_OUTPUT, 1000, MAX_OUTPUT_CHARS)
if len(result) > max_len:
half = max_len // 2
result = (
result[:half]
+ f"\n\n... ({len(result) - max_len:,} chars truncated) ...\n\n"
+ result[-half:]
)
return result
except Exception as e:
return f"Error executing command: {str(e)}"
async def _execute_session(
self,
prepared: _PreparedCommand,
yield_time_ms: int | None,
max_output_chars: int | None,
) -> str:
try:
session_id, poll = await self._session_manager.start(
command=prepared.command,
cwd=prepared.cwd,
env=prepared.env,
timeout=prepared.timeout,
shell_program=prepared.shell_program,
login=prepared.login,
yield_time_ms=clamp_session_int(yield_time_ms, DEFAULT_YIELD_MS, 0, MAX_YIELD_MS),
max_output_chars=clamp_session_int(
max_output_chars,
DEFAULT_MAX_OUTPUT_CHARS,
1000,
MAX_OUTPUT_CHARS,
),
)
return format_session_poll(session_id, poll)
except Exception as exc:
return f"Error executing command: {exc}"
def _resolve_timeout(self, timeout: int | None) -> int | None:
"""Resolve the effective hard timeout in seconds (None = no limit).
A per-call timeout supplied by the model stays capped at _MAX_TIMEOUT so
the LLM cannot request unbounded execution. The config-level default
(self.timeout) may exceed that cap, and 0 disables the limit entirely
for trusted long-running tasks (#3595).
"""
if timeout:
return min(timeout, self._MAX_TIMEOUT)
if self.timeout and self.timeout > 0:
return self.timeout
return None
def _prepare_command(
self,
command: str,
working_dir: str | None = None,
timeout: int | None = None,
shell: str | None = None,
login: bool | None = None,
) -> _PreparedCommand | str:
cwd = working_dir or self.working_dir or os.getcwd()
# Prevent an LLM-supplied working_dir from escaping the configured
@@ -201,7 +383,7 @@ class ExecTool(Tool):
command = wrap_command(self.sandbox, command, workspace, cwd)
cwd = str(Path(workspace).resolve())
effective_timeout = min(timeout or self.timeout, self._MAX_TIMEOUT)
effective_timeout = self._resolve_timeout(timeout)
env = self._build_env()
if self.path_append:
@@ -211,52 +393,24 @@ class ExecTool(Tool):
env["NANOBOT_PATH_APPEND"] = self.path_append
command = f'export PATH="$PATH{os.pathsep}$NANOBOT_PATH_APPEND"; {command}'
try:
process = await self._spawn(command, cwd, env)
shell_program, shell_error = self._resolve_shell(shell)
if shell_error:
return shell_error
try:
stdout, stderr = await asyncio.wait_for(
process.communicate(),
timeout=effective_timeout,
)
except asyncio.TimeoutError:
await self._kill_process(process)
return f"Error: Command timed out after {effective_timeout} seconds"
except asyncio.CancelledError:
await self._kill_process(process)
raise
output_parts = []
if stdout:
output_parts.append(stdout.decode("utf-8", errors="replace"))
if stderr:
stderr_text = stderr.decode("utf-8", errors="replace")
if stderr_text.strip():
output_parts.append(f"STDERR:\n{stderr_text}")
output_parts.append(f"\nExit code: {process.returncode}")
result = "\n".join(output_parts) if output_parts else "(no output)"
max_len = self._MAX_OUTPUT
if len(result) > max_len:
half = max_len // 2
result = (
result[:half]
+ f"\n\n... ({len(result) - max_len:,} chars truncated) ...\n\n"
+ result[-half:]
)
return result
except Exception as e:
return f"Error executing command: {str(e)}"
return _PreparedCommand(
command=command,
cwd=cwd,
env=env,
timeout=effective_timeout,
shell_program=shell_program,
login=True if login is None else login,
)
@staticmethod
async def _spawn(
command: str, cwd: str, env: dict[str, str],
shell_program: str | None = None,
login: bool = True,
) -> asyncio.subprocess.Process:
"""Launch *command* in a platform-appropriate shell."""
if _IS_WINDOWS:
@@ -272,9 +426,14 @@ class ExecTool(Tool):
cwd=cwd,
env=env,
)
bash = shutil.which("bash") or "/bin/bash"
shell_program = shell_program or shutil.which("bash") or "/bin/bash"
args = [shell_program]
shell_name = Path(shell_program).name.lower()
if login and shell_name in {"bash", "bash.exe", "zsh", "zsh.exe"}:
args.append("-l")
args.extend(["-c", command])
return await asyncio.create_subprocess_exec(
bash, "-l", "-c", command,
*args,
stdin=asyncio.subprocess.DEVNULL,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
@@ -282,6 +441,31 @@ class ExecTool(Tool):
env=env,
)
@staticmethod
def _resolve_shell(shell: str | None) -> tuple[str | None, str | None]:
if not shell:
return None, None
if _IS_WINDOWS:
return None, "Error: shell parameter is not supported on Windows"
if "\0" in shell or "\n" in shell or "\r" in shell:
return None, "Error: shell contains invalid characters"
allowed = {"sh", "bash", "zsh"}
path = Path(shell).expanduser()
if path.is_absolute():
if path.name not in allowed:
return None, f"Error: unsupported shell {shell!r}. Allowed: bash, sh, zsh"
if not path.is_file() or not os.access(path, os.X_OK):
return None, f"Error: shell is not executable: {shell}"
return str(path), None
if "/" in shell or "\\" in shell:
return None, "Error: shell must be a shell name or absolute path"
if shell not in allowed:
return None, f"Error: unsupported shell {shell!r}. Allowed: bash, sh, zsh"
resolved = shutil.which(shell)
if not resolved:
return None, f"Error: shell not found: {shell}"
return resolved, None
@staticmethod
async def _kill_process(process: asyncio.subprocess.Process) -> None:
"""Kill a subprocess and reap it to prevent zombies."""
@@ -418,7 +602,7 @@ class ExecTool(Tool):
# Windows: match drive-root paths like `C:\` as well as `C:\path\to\file`, and UNC paths like `\\server\share`
# NOTE: `*` is required so `C:\` (nothing after the slash) is still extracted.
win_paths = re.findall(
r"(?:[A-Za-z]:[^\s\"'|><;]*|\\\\[^\s\"'|><;]+(?:\\[^\s\"'|><;]+)*)",
r"(?<![A-Za-z])(?:[A-Za-z]:[^\s\"'|><;]*|\\\\[^\s\"'|><;]+(?:\\[^\s\"'|><;]+)*)",
command
)
posix_paths = re.findall(r"(?:^|[\s|>'\"])(/[^\s\"'>;|<]+)", command) # POSIX: /absolute only
+18 -2
View File
@@ -7,7 +7,7 @@ from typing import TYPE_CHECKING, Any
from nanobot.agent.tools.base import Tool, tool_parameters
from nanobot.agent.tools.context import ContextAware, RequestContext
from nanobot.agent.tools.schema import StringSchema, tool_parameters_schema
from nanobot.agent.tools.schema import NumberSchema, StringSchema, tool_parameters_schema
if TYPE_CHECKING:
from nanobot.agent.subagent import SubagentManager
@@ -17,6 +17,15 @@ if TYPE_CHECKING:
tool_parameters_schema(
task=StringSchema("The task for the subagent to complete"),
label=StringSchema("Optional short label for the task (for display)"),
temperature=NumberSchema(
description=(
"Optional sampling temperature for the subagent "
"(0.0 = deterministic, higher = more creative). "
"Defaults to the provider's configured temperature."
),
minimum=0.0,
maximum=2.0,
),
required=["task"],
)
)
@@ -58,7 +67,13 @@ class SpawnTool(Tool, ContextAware):
"and use a dedicated subdirectory when helpful."
)
async def execute(self, task: str, label: str | None = None, **kwargs: Any) -> str:
async def execute(
self,
task: str,
label: str | None = None,
temperature: float | None = None,
**kwargs: Any,
) -> str:
"""Spawn a subagent to execute the given task."""
running = self._manager.get_running_count()
limit = self._manager.max_concurrent_subagents
@@ -75,4 +90,5 @@ class SpawnTool(Tool, ContextAware):
origin_chat_id=self._origin_chat_id.get(),
session_key=self._session_key.get(),
origin_message_id=self._origin_message_id.get(),
temperature=temperature,
)
+99 -18
View File
@@ -8,7 +8,7 @@ import json
import os
import re
from typing import Any, Callable
from urllib.parse import quote, urlparse
from urllib.parse import quote, urljoin, urlparse
import httpx
from loguru import logger
@@ -78,9 +78,82 @@ def _validate_url(url: str) -> tuple[bool, str]:
def _validate_url_safe(url: str) -> tuple[bool, str]:
"""Validate URL with SSRF protection: scheme, domain, and resolved IP check."""
from nanobot.security.network import validate_url_target
return validate_url_target(url)
async def _get_with_safe_redirects(
client: httpx.AsyncClient,
url: str,
headers: dict[str, str] | None = None,
) -> tuple[httpx.Response | None, str | None]:
"""GET a URL while validating every redirect target before requesting it."""
current_url = url
for _ in range(MAX_REDIRECTS + 1):
is_valid, error_msg = _validate_url_safe(current_url)
if not is_valid:
return None, f"Redirect blocked: {error_msg}"
response = await client.get(current_url, headers=headers, follow_redirects=False)
is_redirect = 300 <= response.status_code < 400
if not is_redirect:
return response, None
location = response.headers.get("location")
if not location:
return response, None
next_url = urljoin(str(response.url), location)
is_valid, error_msg = _validate_url_safe(next_url)
if not is_valid:
await response.aclose()
return None, f"Redirect blocked: {error_msg}"
await response.aclose()
current_url = next_url
return None, f"Too many redirects: exceeded limit of {MAX_REDIRECTS}"
async def _stream_with_safe_redirects(
client: httpx.AsyncClient,
url: str,
headers: dict[str, str] | None = None,
) -> tuple[httpx.Response | None, Any | None, str | None]:
"""Open a streamed response while validating every redirect target first."""
current_url = url
for _ in range(MAX_REDIRECTS + 1):
is_valid, error_msg = _validate_url_safe(current_url)
if not is_valid:
return None, None, f"Redirect blocked: {error_msg}"
stream = client.stream(
"GET",
current_url,
headers=headers,
follow_redirects=False,
)
response = await stream.__aenter__()
is_redirect = 300 <= response.status_code < 400
if not is_redirect:
return response, stream, None
location = response.headers.get("location")
if not location:
return response, stream, None
next_url = urljoin(str(response.url), location)
is_valid, error_msg = _validate_url_safe(next_url)
if not is_valid:
await stream.__aexit__(None, None, None)
return None, None, f"Redirect blocked: {error_msg}"
await stream.__aexit__(None, None, None)
current_url = next_url
return None, None, f"Too many redirects: exceeded limit of {MAX_REDIRECTS}"
def _format_results(query: str, items: list[dict[str, Any]], n: int) -> str:
"""Format provider results into shared plaintext output."""
if not items:
@@ -488,19 +561,26 @@ class WebFetchTool(Tool):
# Detect and fetch images directly to avoid Jina's textual image captioning
try:
async with httpx.AsyncClient(proxy=self.proxy, follow_redirects=True, max_redirects=MAX_REDIRECTS, timeout=15.0) as client:
async with client.stream("GET", url, headers={"User-Agent": self.user_agent}) as r:
from nanobot.security.network import validate_resolved_url
redir_ok, redir_err = validate_resolved_url(str(r.url))
if not redir_ok:
return json.dumps({"error": f"Redirect blocked: {redir_err}", "url": url}, ensure_ascii=False)
async with httpx.AsyncClient(proxy=self.proxy, timeout=15.0) as client:
r, stream, redirect_error = await _stream_with_safe_redirects(
client,
url,
headers={"User-Agent": self.user_agent},
)
if redirect_error:
return json.dumps({"error": redirect_error, "url": url}, ensure_ascii=False)
if r is None:
return json.dumps({"error": "Fetch failed", "url": url}, ensure_ascii=False)
try:
ctype = r.headers.get("content-type", "")
if ctype.startswith("image/"):
r.raise_for_status()
raw = await r.aread()
return build_image_content_blocks(raw, ctype, url, f"(Image fetched from: {url})")
finally:
if stream is not None:
await stream.__aexit__(None, None, None)
except Exception as e:
logger.debug("Pre-fetch image detection failed for {}: {}", url, e)
@@ -549,23 +629,22 @@ class WebFetchTool(Tool):
async def _fetch_readability(self, url: str, extract_mode: str, max_chars: int) -> Any:
"""Local fallback using readability-lxml."""
from readability import Document
try:
async with httpx.AsyncClient(
follow_redirects=True,
max_redirects=MAX_REDIRECTS,
timeout=30.0,
proxy=self.proxy,
) as client:
r = await client.get(url, headers={"User-Agent": self.user_agent})
r, redirect_error = await _get_with_safe_redirects(
client,
url,
headers={"User-Agent": self.user_agent},
)
if redirect_error:
return json.dumps({"error": redirect_error, "url": url}, ensure_ascii=False)
if r is None:
return json.dumps({"error": "Fetch failed", "url": url}, ensure_ascii=False)
r.raise_for_status()
from nanobot.security.network import validate_resolved_url
redir_ok, redir_err = validate_resolved_url(str(r.url))
if not redir_ok:
return json.dumps({"error": f"Redirect blocked: {redir_err}", "url": url}, ensure_ascii=False)
ctype = r.headers.get("content-type", "")
if ctype.startswith("image/"):
return build_image_content_blocks(r.content, ctype, url, f"(Image fetched from: {url})")
@@ -573,6 +652,8 @@ class WebFetchTool(Tool):
if "application/json" in ctype:
text, extractor = json.dumps(r.json(), indent=2, ensure_ascii=False), "json"
elif "text/html" in ctype or r.text[:256].lower().startswith(("<!doctype", "<html")):
from readability import Document
doc = Document(r.text)
content = self._to_markdown(doc.summary()) if extract_mode == "markdown" else _strip_tags(doc.summary())
text = f"# {doc.title()}\n\n{content}" if doc.title() else content
+5
View File
@@ -0,0 +1,5 @@
"""Shared app protocol helpers."""
from nanobot.apps.protocol import APP_PROTOCOL_SCHEMA, app_manifest
__all__ = ["APP_PROTOCOL_SCHEMA", "app_manifest"]
+13
View File
@@ -0,0 +1,13 @@
"""CLI app adapter for the unified Apps domain."""
from nanobot.apps.cli.service import (
CliAppError,
CliAppManager,
CliAppsRuntimeConfig,
)
__all__ = [
"CliAppError",
"CliAppManager",
"CliAppsRuntimeConfig",
]
File diff suppressed because it is too large Load Diff
+62
View File
@@ -0,0 +1,62 @@
"""CLI Apps helpers shared by the agent loop and settings surfaces."""
from __future__ import annotations
from pathlib import Path
from typing import Any, Mapping
def session_extra(metadata: Mapping[str, Any] | None) -> dict[str, Any]:
"""Return persisted session kwargs for CLI app attachments."""
cli_apps = metadata.get("cli_apps") if isinstance(metadata, Mapping) else None
return {"cli_apps": cli_apps} if isinstance(cli_apps, list) and cli_apps else {}
def runtime_lines(message: Any, workspace: Path, *, skip: bool = False) -> list[str]:
"""Return model-visible CLI app annotations for the current turn."""
if skip:
return []
text = message.content if isinstance(getattr(message, "content", None), str) else ""
metadata = message.metadata if isinstance(getattr(message, "metadata", None), Mapping) else None
return _cli_app_runtime_lines(text, metadata, workspace)
def _cli_app_runtime_lines(
text: str,
metadata: Mapping[str, Any] | None,
workspace: Path,
) -> list[str]:
structured = metadata.get("cli_apps") if isinstance(metadata, Mapping) else None
if isinstance(structured, list):
mentions = [
item for item in structured
if isinstance(item, Mapping) and isinstance(item.get("name"), str)
]
if mentions:
return [
"CLI App Attachment: "
f"@{str(item['name']).strip().lower()} "
f"(installed; tool=run_cli_app; "
f"entry_point={str(item.get('entry_point') or 'unknown')}; "
f"skill=skills/cli-app-{str(item['name']).strip().lower()}/SKILL.md). "
"Read the skill when useful, then run this app with `run_cli_app`; do not bypass it with shell."
for item in mentions
if str(item.get("name") or "").strip()
]
if "@" not in text:
return []
try:
from nanobot.apps.cli import CliAppManager
mentions = CliAppManager(workspace=workspace).mentioned_installed_apps(text)
except Exception:
return []
return [
"CLI App Mention: "
f"@{item['name']} "
f"(installed; tool={item['tool']}; "
f"entry_point={item['entry_point'] or 'unknown'}; "
f"skill={item['skill']}). "
"Read the skill when useful, then run this app with `run_cli_app`; do not bypass it with shell."
for item in mentions
]
+56
View File
@@ -0,0 +1,56 @@
"""Neutral manifest shape for settings-managed agent apps.
The manifest is intentionally descriptive. Installers still live in their
own adapters, while this protocol gives the WebUI and future registries one
small vocabulary for capabilities, trust, and verified install/remove plans.
"""
from __future__ import annotations
from typing import Any
APP_PROTOCOL_SCHEMA = "agent-app.v1"
def compact_dict(values: dict[str, Any]) -> dict[str, Any]:
"""Drop empty optional values while preserving explicit booleans and zeros."""
return {
key: value
for key, value in values.items()
if value is not None and value != "" and value != [] and value != {}
}
def app_manifest(
*,
app_id: str,
display_name: str,
description: str,
category: str,
source: str,
capabilities: list[dict[str, Any]],
install: dict[str, Any],
remove: dict[str, Any],
trust: dict[str, Any],
version: str | None = None,
logo_url: str | None = None,
brand_color: str | None = None,
docs_url: str | None = None,
) -> dict[str, Any]:
"""Build a stable app manifest dictionary."""
return compact_dict({
"schema": APP_PROTOCOL_SCHEMA,
"id": app_id,
"display_name": display_name,
"version": version,
"description": description,
"category": category,
"source": source,
"logo_url": logo_url,
"brand_color": brand_color,
"docs_url": docs_url,
"capabilities": capabilities,
"install": install,
"remove": remove,
"trust": trust,
})
+6 -1
View File
@@ -9,6 +9,12 @@ from typing import Any
# render it and other channels may ignore unknown keys.
OUTBOUND_META_AGENT_UI = "_agent_ui"
# Internal-only inbound metadata used by in-process channels to ask the agent
# loop to update runtime state without going through a user session.
INBOUND_META_RUNTIME_CONTROL = "_runtime_control"
RUNTIME_CONTROL_ACK = "_ack"
RUNTIME_CONTROL_MCP_RELOAD = "mcp_reload"
@dataclass
class InboundMessage:
@@ -45,4 +51,3 @@ class OutboundMessage:
media: list[str] = field(default_factory=list)
metadata: dict[str, Any] = field(default_factory=dict)
buttons: list[list[str]] = field(default_factory=list)
+1
View File
@@ -110,6 +110,7 @@ class ChannelManager:
static_path = _default_webui_dist()
if static_path is not None:
kwargs["static_dist_path"] = static_path
kwargs["workspace_path"] = self.config.workspace_path
if self._webui_runtime_model_name is not None:
kwargs["runtime_model_name"] = self._webui_runtime_model_name
channel = cls(section, self.bus, **kwargs)
File diff suppressed because it is too large Load Diff
+184 -6
View File
@@ -30,11 +30,12 @@ from websockets.exceptions import ConnectionClosed
from websockets.http11 import Request as WsRequest
from websockets.http11 import Response
from nanobot.agent.tools.mcp import request_mcp_reload
from nanobot.bus.events import OUTBOUND_META_AGENT_UI, OutboundMessage
from nanobot.bus.queue import MessageBus
from nanobot.channels.base import BaseChannel
from nanobot.command.builtin import builtin_command_palette
from nanobot.config.paths import get_media_dir
from nanobot.config.paths import get_media_dir, get_workspace_path
from nanobot.config.schema import Base
from nanobot.session.goal_state import goal_state_ws_blob
from nanobot.session.webui_turns import websocket_turn_wall_started_at
@@ -46,18 +47,44 @@ from nanobot.utils.media_decode import (
from nanobot.utils.subagent_channel_display import scrub_subagent_messages_for_channel
from nanobot.webui.settings_api import (
WebUISettingsError,
create_model_configuration,
settings_payload,
update_agent_settings,
update_image_generation_settings,
update_provider_settings,
update_web_search_settings,
)
from nanobot.webui.cli_apps_api import (
cli_apps_action,
cli_apps_payload,
normalize_cli_app_mentions,
)
from nanobot.webui.mcp_presets_api import (
mcp_presets_settings_action,
normalize_mcp_preset_mentions,
)
from nanobot.webui.sidebar_state import (
read_webui_sidebar_state,
write_webui_sidebar_state,
)
from nanobot.webui.thread_disk import delete_webui_thread
from nanobot.webui.transcript import append_transcript_object, build_webui_thread_response
from nanobot.webui.transcript import (
append_transcript_object,
build_webui_thread_response,
rewrite_local_markdown_images,
)
_MCP_PRESET_ACTIONS_BY_PATH = {
"/api/settings/mcp-presets/enable": "enable",
"/api/settings/mcp-presets/remove": "remove",
"/api/settings/mcp-presets/test": "test",
"/api/settings/mcp-presets/custom": "custom",
"/api/settings/mcp-presets/import": "import",
"/api/settings/mcp-presets/import-cursor": "import-cursor",
"/api/settings/mcp-presets/tools": "tools",
}
_MCP_VALUES_HEADER = "X-Nanobot-MCP-Values"
_MCP_VALUES_HEADER_MAX_BYTES = 64 * 1024
if TYPE_CHECKING:
from nanobot.session.manager import SessionManager
@@ -228,6 +255,34 @@ def _parse_query(path_with_query: str) -> dict[str, list[str]]:
return _parse_request_path(path_with_query)[1]
def _parse_mcp_settings_query(request: WsRequest) -> dict[str, list[str]]:
query = _parse_query(request.path)
raw = request.headers.get(_MCP_VALUES_HEADER)
if not raw:
return query
if len(raw.encode("utf-8")) > _MCP_VALUES_HEADER_MAX_BYTES:
raise WebUISettingsError("MCP settings payload is too large")
try:
payload = json.loads(raw)
except json.JSONDecodeError as exc:
raise WebUISettingsError("invalid MCP settings payload") from exc
if not isinstance(payload, dict):
raise WebUISettingsError("MCP settings payload must be a JSON object")
merged = {key: list(values) for key, values in query.items()}
for key, value in payload.items():
if not isinstance(key, str) or not key:
raise WebUISettingsError("MCP settings payload contains an invalid key")
if value is None:
continue
if isinstance(value, str):
text = value.strip()
else:
text = json.dumps(value, ensure_ascii=False, separators=(",", ":"))
if text:
merged[key] = [text]
return merged
def _query_first(query: dict[str, list[str]], key: str) -> str | None:
"""Return the first value for *key*, or None."""
values = query.get(key)
@@ -420,8 +475,6 @@ _MEDIA_ALLOWED_MIMES: frozenset[str] = frozenset({
"video/webm",
"video/quicktime",
})
def _issue_route_secret_matches(headers: Any, configured_secret: str) -> bool:
"""Return True if the token-issue HTTP request carries credentials matching ``token_issue_secret``."""
if not configured_secret:
@@ -449,6 +502,7 @@ class WebSocketChannel(BaseChannel):
*,
session_manager: "SessionManager | None" = None,
static_dist_path: Path | None = None,
workspace_path: Path | None = None,
runtime_model_name: Callable[[], str | None] | None = None,
):
if isinstance(config, dict):
@@ -471,8 +525,14 @@ class WebSocketChannel(BaseChannel):
self._static_dist_path: Path | None = (
static_dist_path.resolve() if static_dist_path is not None else None
)
self._workspace_path = (
Path(workspace_path).expanduser()
if workspace_path is not None
else get_workspace_path()
).resolve(strict=False)
self._runtime_model_name = runtime_model_name
self._settings_restart_sections: set[str] = set()
self._stream_text_buffers: dict[tuple[str, str], list[str]] = {}
# Process-local secret used to HMAC-sign media URLs. The signed URL is
# the capability — anyone who holds a valid URL can fetch that one
# file, nothing else. The secret regenerates on restart so links
@@ -644,6 +704,9 @@ class WebSocketChannel(BaseChannel):
if got == "/api/settings/update":
return self._handle_settings_update(request)
if got == "/api/settings/model-configurations/create":
return self._handle_settings_model_configuration_create(request)
if got == "/api/settings/provider/update":
return self._handle_settings_provider_update(request)
@@ -653,6 +716,28 @@ class WebSocketChannel(BaseChannel):
if got == "/api/settings/image-generation/update":
return self._handle_settings_image_generation_update(request)
if got == "/api/settings/cli-apps":
return self._handle_settings_cli_apps(request)
if got == "/api/settings/cli-apps/install":
return await self._handle_settings_cli_apps_action(request, "install")
if got == "/api/settings/cli-apps/update":
return await self._handle_settings_cli_apps_action(request, "update")
if got == "/api/settings/cli-apps/uninstall":
return await self._handle_settings_cli_apps_action(request, "uninstall")
if got == "/api/settings/cli-apps/test":
return await self._handle_settings_cli_apps_action(request, "test")
if got == "/api/settings/mcp-presets":
return await self._handle_settings_mcp_presets(request)
mcp_action = _MCP_PRESET_ACTIONS_BY_PATH.get(got)
if mcp_action is not None:
return await self._handle_settings_mcp_presets(request, mcp_action)
m = re.match(r"^/api/sessions/([^/]+)/messages$", got)
if m:
return self._handle_session_messages(request, m.group(1))
@@ -844,6 +929,16 @@ class WebSocketChannel(BaseChannel):
self._with_settings_restart_state(payload, section="runtime")
)
def _handle_settings_model_configuration_create(self, request: WsRequest) -> Response:
if not self._check_api_token(request):
return _http_error(401, "Unauthorized")
query = _parse_query(request.path)
try:
payload = create_model_configuration(query)
except WebUISettingsError as e:
return _http_error(e.status, e.message)
return _http_json_response(self._with_settings_restart_state(payload))
def _handle_settings_provider_update(self, request: WsRequest) -> Response:
if not self._check_api_token(request):
return _http_error(401, "Unauthorized")
@@ -874,6 +969,57 @@ class WebSocketChannel(BaseChannel):
return _http_error(e.status, e.message)
return _http_json_response(self._with_settings_restart_state(payload, section="image"))
def _handle_settings_cli_apps(self, request: WsRequest) -> Response:
if not self._check_api_token(request):
return _http_error(401, "Unauthorized")
try:
payload = cli_apps_payload()
except Exception:
self.logger.exception("failed to load CLI Apps payload")
return _http_error(500, "failed to load CLI Apps")
return _http_json_response(payload)
async def _handle_settings_cli_apps_action(self, request: WsRequest, action: str) -> Response:
if not self._check_api_token(request):
return _http_error(401, "Unauthorized")
query = _parse_query(request.path)
try:
payload = await asyncio.to_thread(cli_apps_action, action, query)
except WebUISettingsError as e:
return _http_error(e.status, e.message)
except Exception as e:
status = getattr(e, "status", 500)
message = getattr(e, "message", str(e))
if status >= 500:
self.logger.exception("CLI Apps action '{}' failed", action)
return _http_error(status, message)
return _http_json_response(payload)
async def _handle_settings_mcp_presets(
self,
request: WsRequest,
action: str | None = None,
) -> Response:
if not self._check_api_token(request):
return _http_error(401, "Unauthorized")
try:
payload = await mcp_presets_settings_action(
action,
_parse_mcp_settings_query(request),
reload_mcp=lambda: request_mcp_reload(self.bus),
)
except Exception as e:
status = getattr(e, "status", 500)
message = getattr(e, "message", str(e))
if status >= 500:
self.logger.exception("MCP preset action '{}' failed", action or "list")
return _http_error(status, message)
if action is None:
return _http_json_response(payload)
return _http_json_response(
self._with_settings_restart_state(payload, section="runtime")
)
@staticmethod
def _is_websocket_channel_session_key(key: str) -> bool:
"""True when *key* is a ``websocket:…`` session exposed on this HTTP surface."""
@@ -915,6 +1061,7 @@ class WebSocketChannel(BaseChannel):
data = build_webui_thread_response(
decoded_key,
augment_user_media=self._augment_transcript_user_media,
augment_assistant_text=self._rewrite_local_markdown_images,
)
if data is None:
return _http_error(404, "webui thread not found")
@@ -961,6 +1108,12 @@ class WebSocketChannel(BaseChannel):
}
if media:
user_obj["media_paths"] = list(media)
cli_apps = meta.get("cli_apps")
if isinstance(cli_apps, list) and cli_apps:
user_obj["cli_apps"] = cli_apps
mcp_presets = meta.get("mcp_presets")
if isinstance(mcp_presets, list) and mcp_presets:
user_obj["mcp_presets"] = mcp_presets
self._try_append_webui_transcript(chat_id, user_obj)
await super()._handle_message(
sender_id,
@@ -1050,6 +1203,13 @@ class WebSocketChannel(BaseChannel):
return None
return {"url": signed, "name": path.name}
def _rewrite_local_markdown_images(self, text: str) -> str:
return rewrite_local_markdown_images(
text,
workspace_path=self._workspace_path,
sign_path=self._sign_or_stage_media_path,
)
def _handle_media_fetch(self, sig: str, payload: str) -> Response:
"""Serve a single media file previously signed via
:meth:`_sign_media_path`. Validates the signature, decodes the
@@ -1421,6 +1581,12 @@ class WebSocketChannel(BaseChannel):
metadata: dict[str, Any] = {"remote": getattr(connection, "remote_address", None)}
if envelope.get("webui") is True:
metadata["webui"] = True
cli_apps = normalize_cli_app_mentions(envelope.get("cli_apps"))
if cli_apps:
metadata["cli_apps"] = cli_apps
mcp_presets = normalize_mcp_preset_mentions(envelope.get("mcp_presets"))
if mcp_presets:
metadata["mcp_presets"] = mcp_presets
image_generation = envelope.get("image_generation")
if isinstance(image_generation, dict) and image_generation.get("enabled") is True:
aspect_ratio = image_generation.get("aspect_ratio")
@@ -1532,10 +1698,11 @@ class WebSocketChannel(BaseChannel):
await self._safe_send_to(connection, raw, label=" ")
return
text = msg.content
wire_text = self._rewrite_local_markdown_images(text)
payload: dict[str, Any] = {
"event": "message",
"chat_id": msg.chat_id,
"text": text,
"text": wire_text,
}
if msg.media:
payload["media"] = msg.media
@@ -1563,7 +1730,9 @@ class WebSocketChannel(BaseChannel):
payload["kind"] = "tool_hint"
elif msg.metadata.get("_progress"):
payload["kind"] = "progress"
self._try_append_webui_transcript(msg.chat_id, payload)
transcript_payload = dict(payload)
transcript_payload["text"] = text
self._try_append_webui_transcript(msg.chat_id, transcript_payload)
raw = json.dumps(payload, ensure_ascii=False)
for connection in conns:
await self._safe_send_to(connection, raw, label=" ")
@@ -1628,14 +1797,23 @@ class WebSocketChannel(BaseChannel):
if not conns:
return
meta = metadata or {}
stream_key = (chat_id, str(meta.get("_stream_id") or ""))
if meta.get("_stream_end"):
body: dict[str, Any] = {"event": "stream_end", "chat_id": chat_id}
buffered = self._stream_text_buffers.pop(stream_key, [])
if delta:
buffered.append(delta)
full_text = "".join(buffered)
rewritten = self._rewrite_local_markdown_images(full_text)
if rewritten != full_text:
body["text"] = rewritten
else:
body = {
"event": "delta",
"chat_id": chat_id,
"text": delta,
}
self._stream_text_buffers.setdefault(stream_key, []).append(delta)
if meta.get("_stream_id") is not None:
body["stream_id"] = meta["_stream_id"]
self._try_append_webui_transcript(chat_id, body)
+163 -6
View File
@@ -79,6 +79,12 @@ BASE_INFO: dict[str, str] = {"channel_version": WEIXIN_CHANNEL_VERSION}
ERRCODE_SESSION_EXPIRED = -14
SESSION_PAUSE_DURATION_S = 60 * 60
# iLink context_token is observed to expire server-side after ~90-160s of
# agent inactivity (openclaw/openclaw#61174). Proactively refresh before
# sending if the cached token is older than this threshold.
CONTEXT_TOKEN_MAX_AGE_S = 60
# Retry constants (matching the reference plugin's monitor.ts)
MAX_CONSECUTIVE_FAILURES = 3
BACKOFF_DELAY_S = 30
@@ -159,6 +165,8 @@ class WeixinChannel(BaseChannel):
self._session_pause_until: float = 0.0
self._typing_tasks: dict[str, asyncio.Task] = {}
self._typing_tickets: dict[str, dict[str, Any]] = {}
self._context_token_at: dict[str, float] = {}
self._pending_tool_hints: dict[str, list[str]] = {}
# ------------------------------------------------------------------
# State persistence
@@ -486,6 +494,7 @@ class WeixinChannel(BaseChannel):
except Exception:
if not self._running:
break
self.logger.exception("WeChat poll loop error")
consecutive_failures += 1
if consecutive_failures >= MAX_CONSECUTIVE_FAILURES:
consecutive_failures = 0
@@ -495,6 +504,7 @@ class WeixinChannel(BaseChannel):
async def stop(self) -> None:
self._running = False
self._pending_tool_hints.clear()
if self._poll_task and not self._poll_task.done():
self._poll_task.cancel()
for chat_id in list(self._typing_tasks):
@@ -545,6 +555,7 @@ class WeixinChannel(BaseChannel):
# Check for API-level errors (monitor.ts checks both ret and errcode)
ret = data.get("ret", 0)
errcode = data.get("errcode", 0)
is_error = (ret is not None and ret != 0) or (errcode is not None and errcode != 0)
if is_error:
@@ -575,8 +586,10 @@ class WeixinChannel(BaseChannel):
# Process messages (WeixinMessage[] from types.ts)
msgs: list[dict] = data.get("msgs", []) or []
for msg in msgs:
with suppress(Exception):
try:
await self._process_message(msg)
except Exception:
self.logger.exception("Failed to process WeChat message")
# ------------------------------------------------------------------
# Inbound message processing (matches inbound.ts + process-message.ts)
@@ -610,6 +623,7 @@ class WeixinChannel(BaseChannel):
ctx_token = msg.get("context_token", "")
if ctx_token:
self._context_tokens[from_user_id] = ctx_token
self._context_token_at[from_user_id] = time.time()
self._save_state()
# Parse item_list (WeixinMessage.item_list — types.ts:161)
@@ -915,6 +929,99 @@ class WeixinChannel(BaseChannel):
}
return ""
async def _refresh_context_token_if_stale(
self, chat_id: str, context_token: str
) -> str:
"""Return a fresh context_token if the cached one is too old.
iLink context_token expires server-side after a short idle period
(empirically ~90s). Proactively refreshing before sending prevents
silent message loss on long agent turns or cron pushes.
"""
if not context_token:
return context_token
now = time.time()
cached_at = self._context_token_at.get(chat_id, 0)
age = now - cached_at
if age < CONTEXT_TOKEN_MAX_AGE_S:
return context_token
self.logger.debug(
"WeChat context_token for {} is {:.0f}s old; refreshing via getconfig",
chat_id,
age,
)
body: dict[str, Any] = {
"ilink_user_id": chat_id,
"context_token": context_token,
"base_info": BASE_INFO,
}
try:
data = await self._api_post("ilink/bot/getconfig", body)
except Exception as e:
self.logger.warning("WeChat getconfig failed for {}: {}", chat_id, e)
return context_token
if data.get("ret", 0) != 0:
self.logger.warning(
"WeChat getconfig returned ret={} for {}: {}",
data.get("ret"),
chat_id,
data.get("errmsg", ""),
)
return context_token
new_token = str(data.get("context_token", "") or "")
if new_token and new_token != context_token:
self.logger.info(
"WeChat context_token refreshed for {} (age {:.0f}s -> fresh)",
chat_id,
age,
)
self._context_tokens[chat_id] = new_token
self._context_token_at[chat_id] = now
self._save_state()
return new_token
return context_token
async def _flush_tool_hints(self, chat_id: str) -> None:
"""Send any buffered tool hints for *chat_id* as a single message.
Tool hints are coalesced to reduce message count and avoid hitting the
WeChat iLink rate limit (~7 msgs / 5 min). Failures are logged but
not raised so that the main message send is never blocked.
"""
hints = self._pending_tool_hints.pop(chat_id, None)
if not hints:
return
self.logger.info(
"Flushing {} buffered tool hint(s) for {}",
len(hints),
chat_id,
)
ctx_token = self._context_tokens.get(chat_id, "")
ctx_token = await self._refresh_context_token_if_stale(chat_id, ctx_token)
if not ctx_token:
self.logger.warning(
"Dropped {} buffered tool hint(s) for {}: no context_token",
len(hints),
chat_id,
)
return
try:
await self._send_text(chat_id, "\n\n".join(hints), ctx_token)
except Exception:
self.logger.exception(
"Failed to flush buffered tool hints for {}", chat_id
)
async def _send_typing(self, user_id: str, typing_ticket: str, status: int) -> None:
"""Best-effort sendtyping wrapper."""
if not typing_ticket:
@@ -944,11 +1051,47 @@ class WeixinChannel(BaseChannel):
self._assert_session_active()
is_progress = bool((msg.metadata or {}).get("_progress", False))
# Buffer tool hints to coalesce consecutive ones and avoid burning
# WeChat iLink rate-limit quota (~7 msgs / 5 min).
if is_progress and (msg.metadata or {}).get("_tool_hint"):
if not self.send_tool_hints:
return
self._pending_tool_hints.setdefault(msg.chat_id, []).append(msg.content)
self.logger.debug(
"Buffered tool hint for {} (count={})",
msg.chat_id,
len(self._pending_tool_hints[msg.chat_id]),
)
return
# Reasoning deltas are invisible in WeChat (there is no reasoning
# UI). Skip them entirely — do not send and do not flush buffer.
if is_progress and (msg.metadata or {}).get("_reasoning_delta"):
self.logger.debug(
"Dropped invisible reasoning delta for {}", msg.chat_id
)
return
content = msg.content.strip()
# Empty progress messages (e.g. after_iteration tool_events) must
# NOT act as separators — they have no visible content.
if is_progress and not content and not (msg.media or []):
self.logger.debug(
"Skipped empty progress message for {} (no visible content)",
msg.chat_id,
)
return
# Flush buffered hints before sending any visible message.
await self._flush_tool_hints(msg.chat_id)
if not is_progress:
await self._stop_typing(msg.chat_id, clear_remote=True)
content = msg.content.strip()
ctx_token = self._context_tokens.get(msg.chat_id, "")
ctx_token = await self._refresh_context_token_if_stale(msg.chat_id, ctx_token)
if not ctx_token:
raise RuntimeError(
f"WeChat context_token missing for chat_id={msg.chat_id}, cannot send"
@@ -1037,6 +1180,18 @@ class WeixinChannel(BaseChannel):
with suppress(Exception):
await self._send_typing(msg.chat_id, typing_ticket, TYPING_STATUS_CANCEL)
async def send_delta(
self, chat_id: str, delta: str, metadata: dict[str, Any] | None = None
) -> None:
"""Weixin iLink does not support native streaming deltas.
We only hook ``_stream_end`` so buffered tool hints are flushed even
when the final answer carries the ``_streamed`` flag and bypasses
:meth:`send`.
"""
if metadata and metadata.get("_stream_end"):
await self._flush_tool_hints(chat_id)
async def _start_typing(self, chat_id: str, context_token: str = "") -> None:
"""Start typing indicator immediately when a message is received."""
if not self._client or not self._token or not chat_id:
@@ -1120,10 +1275,11 @@ class WeixinChannel(BaseChannel):
}
data = await self._api_post("ilink/bot/sendmessage", body)
ret = data.get("ret", 0)
errcode = data.get("errcode", 0)
if errcode and errcode != 0:
if (ret is not None and ret != 0) or (errcode is not None and errcode != 0):
raise RuntimeError(
f"WeChat send text error (code {errcode}): {data.get('errmsg', '')}"
f"WeChat send text error (ret={ret}, errcode={errcode}): {data.get('errmsg', '')}"
)
async def _send_media_file(
@@ -1270,10 +1426,11 @@ class WeixinChannel(BaseChannel):
}
data = await self._api_post("ilink/bot/sendmessage", body)
ret = data.get("ret", 0)
errcode = data.get("errcode", 0)
if errcode and errcode != 0:
if (ret is not None and ret != 0) or (errcode is not None and errcode != 0):
raise RuntimeError(
f"WeChat send media error (code {errcode}): {data.get('errmsg', '')}"
f"WeChat send media error (ret={ret}, errcode={errcode}): {data.get('errmsg', '')}"
)
+11 -172
View File
@@ -1,14 +1,12 @@
"""CLI commands for nanobot."""
import asyncio
import json
import os
import select
import signal
import sys
from collections.abc import Callable
from contextlib import nullcontext, suppress
from inspect import signature
from pathlib import Path
from typing import Any
@@ -764,7 +762,7 @@ def _run_gateway(
)
from nanobot.agent.loop import UNIFIED_SESSION_KEY
from nanobot.bus.events import OutboundMessage
from nanobot.bus.events import InboundMessage, OutboundMessage
def _channel_session_key(channel: str, chat_id: str) -> str:
return (
@@ -812,13 +810,13 @@ def _run_gateway(
# Set cron callback (needs agent)
async def on_cron_job(job: CronJob) -> str | None:
"""Execute a cron job through the agent."""
# Dream is an internal job — run directly, not through the agent loop.
if job.name == "dream":
try:
await agent.dream.run()
logger.info("Dream cron job completed")
except Exception:
logger.exception("Dream cron job failed")
await bus.publish_inbound(InboundMessage(
channel="system",
sender_id="dream",
chat_id="dream",
content="",
))
return None
from nanobot.utils.evaluator import evaluate_response
@@ -1029,11 +1027,10 @@ def _run_gateway(
await server.serve_forever()
# Register Dream system job (always-on, idempotent on restart)
dream_cfg = config.agents.defaults.dream
if dream_cfg.model_override:
agent.dream.model = dream_cfg.model_override
agent.dream.max_batch_size = dream_cfg.max_batch_size
agent.dream.max_iterations = dream_cfg.max_iterations
agent.dream.annotate_line_ages = dream_cfg.annotate_line_ages
agent.dream.edit_user_skills = dream_cfg.dream_edit_user_skills
from nanobot.cron.types import CronJob, CronPayload
cron.register_system_job(CronJob(
id="dream",
@@ -1529,106 +1526,6 @@ def status():
console.print(f"{spec.label}: {'[green]✓[/green]' if has_key else '[dim]not set[/dim]'}")
# ============================================================================
# Config Commands
# ============================================================================
config_app = typer.Typer(help="Manage configuration")
app.add_typer(config_app, name="config")
@config_app.command("set")
def config_set(
path: str = typer.Argument(..., help="Dot path, e.g. agents.defaults.model"),
value: str = typer.Argument(..., help="Value. Use null/true/false or JSON for structured values."),
config_path: str | None = typer.Option(None, "--config", "-c", help="Path to config file"),
):
"""Set one config value by dot path."""
from pydantic import ValidationError
from nanobot.config.loader import get_config_path, load_config, save_config, set_config_path
from nanobot.config.schema import Config
resolved_path = Path(config_path).expanduser().resolve() if config_path else get_config_path()
if config_path:
set_config_path(resolved_path)
config = load_config(resolved_path)
parsed = _parse_config_cli_value(value)
try:
_set_config_cli_value(config, path, parsed)
validated = Config.model_validate(config.model_dump(mode="json", by_alias=True))
except (AttributeError, KeyError, TypeError, ValueError, ValidationError) as exc:
console.print(f"[red]Could not set config value:[/red] {exc}")
raise typer.Exit(1)
save_config(validated, resolved_path)
console.print(f"[green]✓[/green] Set [cyan]{path}[/cyan] = [bold]{value}[/bold]")
console.print(f"[dim]Config: {resolved_path}[/dim]")
if path in {"agents.defaults.provider", "agents.defaults.model"} and validated.agents.defaults.model_preset:
console.print(
"[yellow]! agents.defaults.model_preset is set and may override this. "
"Clear it with: nanobot config set agents.defaults.model_preset null[/yellow]"
)
def _parse_config_cli_value(raw: str) -> Any:
lowered = raw.strip().lower()
if lowered == "null":
return None
if lowered == "true":
return True
if lowered == "false":
return False
with suppress(Exception):
return json.loads(raw)
return raw
def _resolve_config_field(obj: Any, key: str) -> str:
from pydantic import BaseModel
from pydantic.alias_generators import to_camel, to_snake
if not isinstance(obj, BaseModel):
return key
fields = type(obj).model_fields
if key in fields:
return key
normalized = to_snake(key.replace("-", "_"))
if normalized in fields:
return normalized
for name, field in fields.items():
aliases = {
to_camel(name),
str(field.alias) if field.alias else "",
str(field.serialization_alias) if field.serialization_alias else "",
}
if key in aliases:
return name
raise AttributeError(f"Unknown config path segment {key!r}")
def _set_config_cli_value(config: Any, path: str, value: Any) -> None:
parts = [part for part in path.split(".") if part]
if not parts:
raise ValueError("Config path cannot be empty.")
current = config
for raw_part in parts[:-1]:
if isinstance(current, dict):
current = current.setdefault(raw_part, {})
continue
part = _resolve_config_field(current, raw_part)
current = getattr(current, part)
leaf = parts[-1]
if isinstance(current, dict):
current[leaf] = value
return
leaf = _resolve_config_field(current, leaf)
setattr(current, leaf, value)
# ============================================================================
# OAuth Login
# ============================================================================
@@ -1643,7 +1540,6 @@ _LOGOUT_HANDLERS: dict[str, Callable[[], None]] = {}
_PROVIDER_DISPLAY: dict[str, str] = {
"openai_codex": "OpenAI Codex",
"github_copilot": "GitHub Copilot",
"xai_oauth": "xAI Grok OAuth",
}
@@ -1679,9 +1575,7 @@ def _resolve_oauth_provider(provider: str):
@provider_app.command("login")
def provider_login(
provider: str = typer.Argument(..., help="OAuth provider (e.g. 'openai-codex', 'github-copilot', 'xai-oauth')"),
no_browser: bool = typer.Option(False, "--no-browser", help="Print the auth URL instead of opening a browser when supported."),
manual_paste: bool = typer.Option(False, "--manual-paste", help="Prompt for a callback URL or fallback code when supported."),
provider: str = typer.Argument(..., help="OAuth provider (e.g. 'openai-codex', 'github-copilot')"),
):
"""Authenticate with an OAuth provider."""
spec = _resolve_oauth_provider(provider)
@@ -1692,18 +1586,12 @@ def provider_login(
raise typer.Exit(1)
console.print(f"{__logo__} OAuth Login - {spec.label}\n")
params = signature(handler).parameters
kwargs: dict[str, bool] = {}
if "no_browser" in params:
kwargs["no_browser"] = no_browser
if "manual_paste" in params:
kwargs["manual_paste"] = manual_paste
handler(**kwargs)
handler()
@provider_app.command("logout")
def provider_logout(
provider: str = typer.Argument(..., help="OAuth provider (e.g. 'openai-codex', 'github-copilot', 'xai-oauth')"),
provider: str = typer.Argument(..., help="OAuth provider (e.g. 'openai-codex', 'github-copilot')"),
):
"""Log out from an OAuth provider."""
spec = _resolve_oauth_provider(provider)
@@ -1767,24 +1655,6 @@ def _logout_github_copilot() -> None:
_delete_oauth_files(storage.get_token_path(), _PROVIDER_DISPLAY["github_copilot"])
@_register_logout("xai_oauth")
def _logout_xai_oauth() -> None:
"""Clear local OAuth credentials for xAI Grok OAuth."""
try:
from nanobot.providers.xai_oauth_provider import delete_xai_oauth_credentials
except ImportError:
console.print("[red]xAI Grok OAuth provider unavailable.[/red]")
raise typer.Exit(1)
removed_paths = delete_xai_oauth_credentials()
if not removed_paths:
console.print(f"[yellow]! No local OAuth credentials found for {_PROVIDER_DISPLAY['xai_oauth']}[/yellow]")
return
console.print(f"[green]✓ Logged out from {_PROVIDER_DISPLAY['xai_oauth']}[/green]")
for path in removed_paths:
console.print(f"[dim]Removed: {path}[/dim]")
def _delete_oauth_files(token_path: Path, provider_label: str) -> None:
"""Delete OAuth token and lock files, reporting the result."""
removed_paths: list[Path] = []
@@ -1828,36 +1698,5 @@ def _login_github_copilot() -> None:
raise typer.Exit(1)
@_register_login("xai_oauth")
def _login_xai_oauth(
*,
no_browser: bool = False,
manual_paste: bool = False,
) -> None:
try:
from nanobot.providers.xai_oauth_provider import login_xai_oauth_interactive
from nanobot.providers.xai_oauth_provider import DEFAULT_XAI_MODEL
console.print("[cyan]Starting xAI Grok OAuth login...[/cyan]\n")
credential = login_xai_oauth_interactive(
print_fn=lambda s: console.print(s),
prompt_fn=lambda s: typer.prompt(s),
open_browser=not no_browser,
manual_paste=manual_paste,
)
account = credential.account_id or "xAI"
storage = "OS keychain" if credential.storage == "keyring" else "private file"
console.print(f"[green]✓ Authenticated with xAI Grok OAuth[/green] [dim]{account} · {storage}[/dim]")
console.print("[dim]To use it for chat:[/dim]")
console.print("[dim] nanobot config set agents.defaults.model_preset null[/dim]")
console.print("[dim] nanobot config set agents.defaults.provider xai-oauth[/dim]")
console.print(f"[dim] nanobot config set agents.defaults.model {DEFAULT_XAI_MODEL}[/dim]")
console.print("[dim]Hosted X Search is enabled by default for xAI OAuth.[/dim]")
console.print("[dim]To disable it: nanobot config set providers.xai_oauth.x_search.enable false[/dim]")
except Exception as e:
console.print(f"[red]Authentication error: {e}[/red]")
raise typer.Exit(1)
if __name__ == "__main__":
app()
+33 -24
View File
@@ -299,30 +299,22 @@ async def cmd_model(ctx: CommandContext) -> OutboundMessage:
async def cmd_dream(ctx: CommandContext) -> OutboundMessage:
"""Manually trigger a Dream consolidation run."""
import time
from nanobot.bus.events import InboundMessage
loop = ctx.loop
msg = ctx.msg
async def _run_dream():
t0 = time.monotonic()
try:
did_work = await loop.dream.run()
elapsed = time.monotonic() - t0
if did_work:
content = f"Dream completed in {elapsed:.1f}s."
else:
content = "Dream: nothing to process."
except Exception as e:
elapsed = time.monotonic() - t0
content = f"Dream failed after {elapsed:.1f}s: {e}"
await loop.bus.publish_outbound(OutboundMessage(
channel=msg.channel, chat_id=msg.chat_id, content=content,
))
asyncio.create_task(_run_dream())
await ctx.loop.bus.publish_inbound(InboundMessage(
channel="system",
sender_id="dream",
chat_id="dream",
content="",
metadata={
"trigger_channel": ctx.msg.channel,
"trigger_chat_id": ctx.msg.chat_id,
},
))
return OutboundMessage(
channel=msg.channel, chat_id=msg.chat_id, content="Dreaming...",
channel=ctx.msg.channel,
chat_id=ctx.msg.chat_id,
content="Dream started. It will process memory backlog and report when done.",
)
@@ -355,6 +347,18 @@ def _format_changed_files(diff: str) -> str:
def _format_dream_log_content(commit, diff: str, *, requested_sha: str | None = None) -> str:
files_line = _format_changed_files(diff)
msg_lines = commit.message.splitlines() if commit.message else []
msg_summary = msg_lines[0] if msg_lines else ""
msg_body = []
in_body = False
for line in msg_lines[1:]:
if not in_body:
if not line:
in_body = True
continue
msg_body.append(line)
body_text = "\n".join(msg_body).strip()
lines = [
"## Dream Update",
"",
@@ -362,8 +366,12 @@ def _format_dream_log_content(commit, diff: str, *, requested_sha: str | None =
"",
f"- Commit: `{commit.sha}`",
f"- Time: {commit.timestamp}",
f"- Changed files: {files_line}",
]
if msg_summary:
lines.append(f"- Summary: {msg_summary}")
lines.append(f"- Changed files: {files_line}")
if body_text:
lines.extend(["", "### Analysis", "", body_text])
if diff:
lines.extend([
"",
@@ -389,7 +397,8 @@ def _format_dream_restore_list(commits: list) -> str:
"",
]
for c in commits:
lines.append(f"- `{c.sha}` {c.timestamp} - {c.message.splitlines()[0]}")
summary = c.message.splitlines()[0] if c.message else "(no message)"
lines.append(f"- `{c.sha}` {c.timestamp} - {summary}")
lines.extend([
"",
"Preview a version with `/dream-log <sha>` before restoring it.",
+7 -1
View File
@@ -10,10 +10,11 @@ import pydantic
from loguru import logger
from pydantic import BaseModel
from nanobot.config.schema import Config
from nanobot.config.schema import Config, _resolve_tool_config_refs
# Global variable to store current config path (for multi-instance support)
_current_config_path: Path | None = None
_schema_refs_ready = False
def set_config_path(path: Path) -> None:
@@ -39,6 +40,11 @@ def load_config(config_path: Path | None = None) -> Config:
Returns:
Loaded configuration object.
"""
global _schema_refs_ready
if not _schema_refs_ready:
_resolve_tool_config_refs()
_schema_refs_ready = True
path = config_path or get_config_path()
config = Config()
+29 -34
View File
@@ -11,6 +11,7 @@ from pydantic_settings import BaseSettings
from nanobot.cron.types import CronSchedule
if TYPE_CHECKING:
from nanobot.agent.tools.cli_apps import CliAppsToolConfig
from nanobot.agent.tools.image_generation import ImageGenerationToolConfig
from nanobot.agent.tools.self import MyToolConfig
from nanobot.agent.tools.shell import ExecToolConfig
@@ -51,14 +52,17 @@ class DreamConfig(Base):
model_override: str | None = Field(
default=None,
validation_alias=AliasChoices("modelOverride", "model", "model_override"),
) # Optional Dream-specific model override
max_batch_size: int = Field(default=20, ge=1) # Max history entries per run
# Bumped from 10 to 15 in #3212 (exp002: +30% dedup, no accuracy loss; >15 plateaus).
max_iterations: int = Field(default=15, ge=1) # Max tool calls per Phase 2
# Per-line git-blame age annotation in Phase 1 prompt (see #3212). Default
# on — set to False to feed MEMORY.md raw if a specific LLM reacts poorly
# to the `← Nd` suffix or you want deterministic, git-independent prompts.
) # Optional Dream-specific model override. Supports preset names (resolved against model_presets) or raw model identifiers.
max_batch_size: int = Field(default=5, ge=1) # Max history entries per run
max_iterations: int = Field(default=15, ge=1) # Max tool calls per Dream run
# Per-line git-blame age annotation in the Dream prompt (see #3212). Default
# on — set to False to feed all memory files raw if a specific LLM reacts
# poorly to the `← Nd` suffix or you want deterministic, git-independent prompts.
annotate_line_ages: bool = True
# When False (default), Dream may only modify skills it created (marked
# dream_managed in frontmatter). When True, Dream may also edit user-created
# workspace skills. Builtin skills are never editable.
dream_edit_user_skills: bool = False
def build_schedule(self, timezone: str) -> CronSchedule:
"""Build the runtime schedule, preferring the legacy cron override if present."""
@@ -91,6 +95,7 @@ FallbackCandidate = str | InlineFallbackConfig
class ModelPresetConfig(Base):
"""A named set of model + generation parameters for quick switching."""
label: str | None = None
model: str
provider: str = "auto"
max_tokens: int = 8192
@@ -169,8 +174,9 @@ class ProviderConfig(Base):
api_key: str | None = None
api_base: str | None = None
api_type: Literal["auto", "chat_completions", "responses"] = "auto" # Request API surface
extra_headers: dict[str, str] | None = None # Custom headers (e.g. APP-Code for AiHubMix)
extra_body: dict[str, Any] | None = None # Extra fields merged into every request body
extra_body: dict[str, Any] | None = None # Extra provider request fields; shape depends on provider/API surface
class BedrockProviderConfig(ProviderConfig):
@@ -180,28 +186,6 @@ class BedrockProviderConfig(ProviderConfig):
profile: str | None = None # Optional AWS shared config profile
class XaiOAuthXSearchConfig(Base):
"""xAI hosted X Search configuration."""
enable: bool = True
allowed_x_handles: list[str] | None = None
excluded_x_handles: list[str] | None = None
from_date: str | None = None
to_date: str | None = None
enable_image_understanding: bool = False
enable_video_understanding: bool = False
class XaiOAuthProviderConfig(ProviderConfig):
"""xAI OAuth provider configuration."""
x_search: XaiOAuthXSearchConfig = Field(default_factory=XaiOAuthXSearchConfig)
def _is_default_xai_oauth_config(value: Any) -> bool:
return isinstance(value, XaiOAuthProviderConfig) and value == XaiOAuthProviderConfig()
class ProvidersConfig(Base):
"""Configuration for LLM providers."""
@@ -233,19 +217,26 @@ class ProvidersConfig(Base):
ant_ling: ProviderConfig = Field(default_factory=ProviderConfig) # Ant Ling
aihubmix: ProviderConfig = Field(default_factory=ProviderConfig) # AiHubMix API gateway
siliconflow: ProviderConfig = Field(default_factory=ProviderConfig) # SiliconFlow (硅基流动)
novita: ProviderConfig = Field(default_factory=ProviderConfig) # Novita AI
volcengine: ProviderConfig = Field(default_factory=ProviderConfig) # VolcEngine (火山引擎)
volcengine_coding_plan: ProviderConfig = Field(default_factory=ProviderConfig) # VolcEngine Coding Plan
byteplus: ProviderConfig = Field(default_factory=ProviderConfig) # BytePlus (VolcEngine international)
byteplus_coding_plan: ProviderConfig = Field(default_factory=ProviderConfig) # BytePlus Coding Plan
openai_codex: ProviderConfig = Field(default_factory=ProviderConfig, exclude=True) # OpenAI Codex (OAuth)
github_copilot: ProviderConfig = Field(default_factory=ProviderConfig, exclude=True) # Github Copilot (OAuth)
xai_oauth: XaiOAuthProviderConfig = Field(
default_factory=XaiOAuthProviderConfig,
exclude_if=_is_default_xai_oauth_config,
) # xAI Grok OAuth
qianfan: ProviderConfig = Field(default_factory=ProviderConfig) # Qianfan (百度千帆)
nvidia: ProviderConfig = Field(default_factory=ProviderConfig) # NVIDIA NIM (nvapi- keys)
@model_validator(mode="after")
def _validate_api_type_scope(self) -> "ProvidersConfig":
for name in self.__class__.model_fields:
if name == "openai":
continue
provider = getattr(self, name, None)
if isinstance(provider, ProviderConfig) and provider.api_type != "auto":
raise ValueError("providers.<name>.api_type is only supported for providers.openai")
return self
class HeartbeatConfig(Base):
"""Heartbeat service configuration."""
@@ -278,6 +269,7 @@ class MCPServerConfig(Base):
command: str = "" # Stdio: command to run (e.g. "npx")
args: list[str] = Field(default_factory=list) # Stdio: command arguments
env: dict[str, str] = Field(default_factory=dict) # Stdio: extra env vars
cwd: str = "" # Stdio: working directory for MCP server runtime artifacts
url: str = "" # HTTP/SSE: endpoint URL
headers: dict[str, str] = Field(default_factory=dict) # HTTP/SSE: custom headers
tool_timeout: int = 30 # seconds before a tool call is cancelled
@@ -301,6 +293,7 @@ class ToolsConfig(Base):
web: WebToolsConfig = Field(default_factory=lambda: _lazy_default("nanobot.agent.tools.web", "WebToolsConfig"))
exec: ExecToolConfig = Field(default_factory=lambda: _lazy_default("nanobot.agent.tools.shell", "ExecToolConfig"))
cli_apps: CliAppsToolConfig = Field(default_factory=lambda: _lazy_default("nanobot.agent.tools.cli_apps", "CliAppsToolConfig"))
my: MyToolConfig = Field(default_factory=lambda: _lazy_default("nanobot.agent.tools.self", "MyToolConfig"))
image_generation: ImageGenerationToolConfig = Field(
default_factory=lambda: _lazy_default("nanobot.agent.tools.image_generation", "ImageGenerationToolConfig"),
@@ -487,6 +480,7 @@ def _resolve_tool_config_refs() -> None:
"""
import sys
from nanobot.agent.tools.cli_apps import CliAppsToolConfig
from nanobot.agent.tools.image_generation import ImageGenerationToolConfig
from nanobot.agent.tools.self import MyToolConfig
from nanobot.agent.tools.shell import ExecToolConfig
@@ -495,6 +489,7 @@ def _resolve_tool_config_refs() -> None:
# Re-export into this module's namespace
mod = sys.modules[__name__]
mod.ExecToolConfig = ExecToolConfig # type: ignore[attr-defined]
mod.CliAppsToolConfig = CliAppsToolConfig # type: ignore[attr-defined]
mod.WebToolsConfig = WebToolsConfig # type: ignore[attr-defined]
mod.WebSearchConfig = WebSearchConfig # type: ignore[attr-defined]
mod.WebFetchConfig = WebFetchConfig # type: ignore[attr-defined]
+4 -24
View File
@@ -14,7 +14,6 @@ __all__ = [
"OpenAICompatProvider",
"OpenAICodexProvider",
"GitHubCopilotProvider",
"XaiOAuthProvider",
"AzureOpenAIProvider",
"BedrockProvider",
]
@@ -24,23 +23,10 @@ _LAZY_IMPORTS = {
"OpenAICompatProvider": ".openai_compat_provider",
"OpenAICodexProvider": ".openai_codex_provider",
"GitHubCopilotProvider": ".github_copilot_provider",
"XaiOAuthProvider": ".xai_oauth_provider",
"AzureOpenAIProvider": ".azure_openai_provider",
"BedrockProvider": ".bedrock_provider",
}
_LAZY_SUBMODULES = {
"anthropic_provider": ".anthropic_provider",
"openai_compat_provider": ".openai_compat_provider",
"openai_codex_provider": ".openai_codex_provider",
"github_copilot_provider": ".github_copilot_provider",
"xai_oauth_provider": ".xai_oauth_provider",
"azure_openai_provider": ".azure_openai_provider",
"bedrock_provider": ".bedrock_provider",
"factory": ".factory",
"registry": ".registry",
}
if TYPE_CHECKING:
from nanobot.providers.anthropic_provider import AnthropicProvider
from nanobot.providers.azure_openai_provider import AzureOpenAIProvider
@@ -48,18 +34,12 @@ if TYPE_CHECKING:
from nanobot.providers.github_copilot_provider import GitHubCopilotProvider
from nanobot.providers.openai_compat_provider import OpenAICompatProvider
from nanobot.providers.openai_codex_provider import OpenAICodexProvider
from nanobot.providers.xai_oauth_provider import XaiOAuthProvider
def __getattr__(name: str):
"""Lazily expose provider implementations without importing all backends up front."""
module_name = _LAZY_IMPORTS.get(name)
if module_name is not None:
module = import_module(module_name, __name__)
return getattr(module, name)
module_name = _LAZY_SUBMODULES.get(name)
if module_name is not None:
module = import_module(module_name, __name__)
globals()[name] = module
return module
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
if module_name is None:
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
module = import_module(module_name, __name__)
return getattr(module, name)
+3 -4
View File
@@ -68,10 +68,6 @@ def _make_provider_core(
from nanobot.providers.github_copilot_provider import GitHubCopilotProvider
provider = GitHubCopilotProvider(default_model=model)
elif backend == "xai_oauth":
from nanobot.providers.xai_oauth_provider import XaiOAuthProvider
provider = XaiOAuthProvider(default_model=model, config=p)
elif backend == "anthropic":
from nanobot.providers.anthropic_provider import AnthropicProvider
@@ -102,6 +98,7 @@ def _make_provider_core(
extra_headers=p.extra_headers if p else None,
spec=spec,
extra_body=p.extra_body if p else None,
api_type=p.api_type if p and provider_name == "openai" else "auto",
)
provider.generation = resolved.to_generation_settings()
@@ -187,6 +184,7 @@ def provider_signature(
config.get_api_base(fallback.model, preset=fallback),
fp.extra_headers if fp else None,
fp.extra_body if fp else None,
fp.api_type if fp else "auto",
getattr(fp, "region", None) if fp else None,
getattr(fp, "profile", None) if fp else None,
fallback.max_tokens,
@@ -203,6 +201,7 @@ def provider_signature(
config.get_api_base(resolved.model, preset=resolved),
p.extra_headers if p else None,
p.extra_body if p else None,
p.api_type if p else "auto",
getattr(p, "region", None) if p else None,
getattr(p, "profile", None) if p else None,
resolved.max_tokens,
+727 -14
View File
@@ -2,8 +2,10 @@
from __future__ import annotations
import asyncio
import base64
import binascii
import re
from abc import ABC, abstractmethod
from dataclasses import dataclass
from pathlib import Path
@@ -31,6 +33,14 @@ _AIHUBMIX_ASPECT_RATIO_SIZES = {
}
_GEMINI_DEFAULT_TIMEOUT_S = 120.0
_GEMINI_IMAGEN_ASPECT_RATIOS = {"1:1", "9:16", "16:9", "3:4", "4:3"}
_OLLAMA_DEFAULT_SIDE = 1024
_OLLAMA_SIZE_PRESETS = {
"1K": 1024,
"2K": 2048,
"4K": 4096,
}
_OLLAMA_EXPLICIT_SIZE_RE = re.compile(r"^\s*(\d+)\s*[xX]\s*(\d+)\s*$")
_OLLAMA_ASPECT_RATIO_RE = re.compile(r"^\s*(\d+)\s*:\s*(\d+)\s*$")
class ImageGenerationError(RuntimeError):
@@ -129,6 +139,11 @@ _IMAGE_GEN_PROVIDERS: dict[str, type[ImageGenerationProvider]] = {}
def register_image_gen_provider(cls: type[ImageGenerationProvider]) -> None:
"""Register an image provider at import time only.
The registry is populated by module side effects so provider discovery
stays lazy and consistent across the process.
"""
name = cls.provider_name
if not name:
raise ValueError(f"{cls.__name__} must set provider_name")
@@ -219,7 +234,10 @@ class ImageGenerationProvider(ABC):
*,
headers: dict[str, str],
body: dict[str, Any],
client: httpx.AsyncClient | None = None,
) -> httpx.Response:
if client is not None:
return await client.post(url, headers=headers, json=body)
if self._client is not None:
return await self._client.post(url, headers=headers, json=body)
async with httpx.AsyncClient(timeout=self.timeout) as c:
@@ -390,10 +408,11 @@ class AIHubMixImageGenerationClient(ImageGenerationProvider):
model_path = _aihubmix_model_path(model)
url = f"{self.api_base}/models/{model_path}/predictions"
try:
response = await client.post(
response = await self._http_post(
url,
headers={**headers, "Content-Type": "application/json"},
json=body,
body=body,
client=client,
)
except httpx.TimeoutException as exc:
raise ImageGenerationError("AIHubMix image generation timed out") from exc
@@ -429,6 +448,139 @@ def _http_error_detail(response: httpx.Response) -> str:
return response.text[:500] or "<empty response body>"
def _round_to_multiple(value: float, multiple: int = 8) -> int:
rounded = int(round(value / multiple) * multiple)
return max(multiple, rounded)
def _ollama_dimensions(aspect_ratio: str | None, image_size: str | None) -> tuple[int, int]:
if image_size:
size = image_size.strip()
explicit = _OLLAMA_EXPLICIT_SIZE_RE.fullmatch(size)
if explicit:
return int(explicit.group(1)), int(explicit.group(2))
long_side = _OLLAMA_SIZE_PRESETS.get(size.upper(), _OLLAMA_DEFAULT_SIDE)
else:
long_side = _OLLAMA_DEFAULT_SIDE
if not aspect_ratio:
return long_side, long_side
ratio = _OLLAMA_ASPECT_RATIO_RE.fullmatch(aspect_ratio.strip())
if ratio is None:
return long_side, long_side
width_ratio = int(ratio.group(1))
height_ratio = int(ratio.group(2))
if width_ratio <= 0 or height_ratio <= 0:
return long_side, long_side
if width_ratio >= height_ratio:
width = long_side
height = _round_to_multiple(long_side * height_ratio / width_ratio)
else:
height = long_side
width = _round_to_multiple(long_side * width_ratio / height_ratio)
return max(8, width), max(8, height)
def _ollama_image_data_url(value: str) -> str:
if value.startswith("data:image/"):
return value
return _b64_image_data_url(value)
def _ollama_images_from_payload(payload: dict[str, Any]) -> list[str]:
images: list[str] = []
def collect(value: Any) -> None:
if isinstance(value, str) and value:
images.append(_ollama_image_data_url(value))
elif isinstance(value, list):
for item in value:
collect(item)
collect(payload.get("image"))
collect(payload.get("images"))
return images
class OllamaImageGenerationClient(ImageGenerationProvider):
"""Async client for Ollama native image generation models."""
provider_name = "ollama"
default_timeout = 300.0
def _default_base_url(self) -> str:
return "http://localhost:11434/api"
def _resolve_base_url(self, api_base: str | None) -> str:
if api_base:
base = api_base.rstrip("/")
if base.endswith("/v1"):
return f"{base[:-3]}/api"
return base
return self._default_base_url()
async def generate(
self,
*,
prompt: str,
model: str,
reference_images: list[str] | None = None,
aspect_ratio: str | None = None,
image_size: str | None = None,
) -> GeneratedImageResponse:
if reference_images:
raise ImageGenerationError(
"Ollama image generation does not support reference images"
)
width, height = _ollama_dimensions(aspect_ratio, image_size)
body: dict[str, Any] = {
"model": model,
"prompt": prompt,
"width": width,
"height": height,
"steps": 0,
}
body.update(self.extra_body)
body["stream"] = False
headers = {
"Content-Type": "application/json",
**self.extra_headers,
}
if self.api_key:
headers["Authorization"] = f"Bearer {self.api_key}"
url = f"{self.api_base}/generate"
response = await self._http_post(url, headers=headers, body=body)
try:
response.raise_for_status()
except httpx.HTTPStatusError as exc:
detail = _http_error_detail(response)
logger.error(
"Ollama image generation failed (HTTP {}): {}",
response.status_code,
detail,
)
raise ImageGenerationError(
f"Ollama image generation failed (HTTP {response.status_code}): {detail}"
) from exc
data = response.json()
images = _ollama_images_from_payload(data)
self._require_images(images, data)
response_text = data.get("response")
content = response_text if isinstance(response_text, str) else ""
return GeneratedImageResponse(images=images, content=content, raw=data)
class GeminiImageGenerationClient(ImageGenerationProvider):
"""Async client for Gemini/Imagen image generation via the Generative Language API."""
@@ -442,9 +594,9 @@ class GeminiImageGenerationClient(ImageGenerationProvider):
return "https://generativelanguage.googleapis.com/v1beta"
def _resolve_base_url(self, api_base: str | None) -> str:
# The Gemini provider's registry default_api_base is the OpenAI-compat
# shim (.../v1beta/openai/), which has no image endpoints.
# Skip the registry lookup and use the native API base directly.
# Gemini chat completions use the registry's OpenAI-compatible shim.
# Image generation must hit the native Generative Language API, so we
# intentionally bypass the shared registry lookup here.
if api_base:
return api_base.rstrip("/")
return self._default_base_url()
@@ -706,22 +858,16 @@ class MiniMaxImageGenerationClient(ImageGenerationProvider):
body.update(self.extra_body)
client = self._client or httpx.AsyncClient(timeout=self.timeout)
try:
return await self._generate_with_client(client, body, headers)
finally:
if self._client is None:
await client.aclose()
return await self._generate_with_client(body, headers)
async def _generate_with_client(
self,
client: httpx.AsyncClient,
body: dict[str, Any],
headers: dict[str, str],
) -> GeneratedImageResponse:
url = f"{self.api_base}/image_generation"
try:
response = await client.post(url, headers=headers, json=body)
response = await self._http_post(url, headers=headers, body=body)
except httpx.TimeoutException as exc:
raise ImageGenerationError("MiniMax image generation timed out") from exc
except httpx.RequestError as exc:
@@ -756,6 +902,426 @@ def _minimax_images_from_payload(payload: dict[str, Any]) -> list[str]:
return images
# ---------------------------------------------------------------------------
# OpenAI image generation
# ---------------------------------------------------------------------------
_OPENAI_DALLE2_SUPPORTED_SIZES = {"256x256", "512x512", "1024x1024"}
_OPENAI_DALLE3_SUPPORTED_SIZES = {"1024x1024", "1792x1024", "1024x1792"}
_OPENAI_GPT_IMAGE_SUPPORTED_SIZES = {
"1024x1024",
"1536x1024",
"1024x1536",
"auto",
}
_OPENAI_DALLE2_ASPECT_RATIO_SIZES = {
"1:1": "1024x1024",
"16:9": "1024x1024",
"9:16": "1024x1024",
"3:4": "1024x1024",
"4:3": "1024x1024",
}
_OPENAI_DALLE3_ASPECT_RATIO_SIZES = {
"1:1": "1024x1024",
"16:9": "1792x1024",
"9:16": "1024x1792",
"3:4": "1024x1792",
"4:3": "1792x1024",
}
_OPENAI_GPT_IMAGE_ASPECT_RATIO_SIZES = {
"1:1": "1024x1024",
"16:9": "1536x1024",
"9:16": "1024x1536",
"3:4": "1024x1536",
"4:3": "1536x1024",
}
class OpenAIImageGenerationClient(ImageGenerationProvider):
"""OpenAI Images API using an API key (``providers.openai.apiKey``)."""
provider_name = "openai"
missing_key_message = (
"OpenAI API key is not configured. Set providers.openai.apiKey."
)
def _default_base_url(self) -> str:
return "https://api.openai.com/v1"
@staticmethod
def _strip_model_prefix(model: str) -> str:
"""Remove ``openai/`` prefix if present (OpenRouter convention)."""
if model.startswith("openai/") or model.startswith("openai_codex/"):
return model.split("/", 1)[1]
return model
async def generate(
self,
*,
prompt: str,
model: str,
reference_images: list[str] | None = None,
aspect_ratio: str | None = None,
image_size: str | None = None,
) -> GeneratedImageResponse:
if not self.api_key:
raise ImageGenerationError(self.missing_key_message)
if reference_images:
logger.warning(
"DALL-E models do not support reference images; "
"ignoring {} reference image(s) for {}",
len(reference_images),
model,
)
headers = {
"Authorization": f"Bearer {self.api_key}",
"Content-Type": "application/json",
**self.extra_headers,
}
clean_model = self._strip_model_prefix(model)
body: dict[str, Any] = {
"model": clean_model,
"prompt": prompt,
}
if not _openai_is_gpt_image_model(clean_model):
body["response_format"] = "b64_json"
body["n"] = 1
size = _openai_size(clean_model, aspect_ratio, image_size)
if size:
body["size"] = size
body.update(self.extra_body)
logger.info("OpenAI Images API request: POST {}/images/generations body={}", self.api_base, body)
response = await self._http_post(
f"{self.api_base}/images/generations",
headers=headers,
body=body,
)
try:
response.raise_for_status()
except httpx.HTTPStatusError as exc:
detail = response.text[:1000]
logger.error("OpenAI Images API error ({}): {}", response.status_code, detail)
raise ImageGenerationError(
f"OpenAI image generation failed (HTTP {response.status_code}): {detail}"
) from exc
payload = response.json()
logger.info("OpenAI Images API response ({}): {}", response.status_code,
{k: v for k, v in payload.items() if k != "data"})
client = self._client
owns_client = client is None
if owns_client:
client = httpx.AsyncClient(timeout=self.timeout)
try:
images = await _openai_images_from_payload(client, payload)
finally:
if owns_client:
await client.aclose()
self._require_images(images, payload)
return GeneratedImageResponse(images=images, content="", raw=payload)
# ---------------------------------------------------------------------------
# OpenAI Codex image generation
# ---------------------------------------------------------------------------
class CodexImageGenerationClient(ImageGenerationProvider):
"""OpenAI image generation via Codex subscription OAuth.
Uses the Codex Responses API with the ``image_generation`` tool
(the same mechanism ChatGPT uses internally). No API key required
the Codex OAuth token from ``oauth_cli_kit`` is used instead.
"""
provider_name = "openai_codex"
missing_key_message = (
"Codex OAuth token is unavailable. "
"Log in with Codex subscription first."
)
def _default_base_url(self) -> str:
return "https://chatgpt.com/backend-api"
def _codex_model(self, model: str) -> str:
"""Strip the ``openai-codex/`` prefix if present."""
if model.startswith(("openai-codex/", "openai_codex/")):
return model.split("/", 1)[1]
return model
async def generate(
self,
*,
prompt: str,
model: str,
reference_images: list[str] | None = None,
aspect_ratio: str | None = None,
image_size: str | None = None,
) -> GeneratedImageResponse:
try:
from oauth_cli_kit import get_token as get_codex_token
except ImportError:
raise ImageGenerationError(self.missing_key_message)
try:
token = await asyncio.to_thread(get_codex_token)
except Exception as exc:
raise ImageGenerationError(self.missing_key_message) from exc
if not token or not token.access:
raise ImageGenerationError(self.missing_key_message)
logger.info(
"Using Codex OAuth token for image generation (account: {})",
token.account_id,
)
if reference_images:
logger.warning(
"Codex image generation does not support reference images; "
"ignoring {} reference image(s)",
len(reference_images),
)
headers = {
"Authorization": f"Bearer {token.access}",
"chatgpt-account-id": token.account_id,
"OpenAI-Beta": "responses=experimental",
"originator": "nanobot",
"User-Agent": "nanobot (python)",
"Content-Type": "application/json",
**self.extra_headers,
}
body: dict[str, Any] = {
"model": self._codex_model(model),
"instructions": "Generate an image based on the user's request.",
"input": [{"role": "user", "content": prompt}],
"tools": [{"type": "image_generation"}],
"tool_choice": "auto",
"stream": True,
"store": False,
}
body.update(self.extra_body)
logger.info("Codex Responses API request: POST {}/codex/responses body={}",
self.api_base, {k: v for k, v in body.items() if k != "input"})
response = await self._http_post(
f"{self.api_base}/codex/responses",
headers=headers,
body=body,
)
try:
response.raise_for_status()
except httpx.HTTPStatusError as exc:
detail = response.text[:1000]
logger.error("Codex Responses API error ({}): {}", response.status_code, detail)
raise ImageGenerationError(
f"Codex image generation failed (HTTP {response.status_code}): {detail}"
) from exc
images, content_text = await _parse_codex_sse_images(response)
raw = {"status": "completed"}
self._require_images(images, raw)
return GeneratedImageResponse(images=images, content=content_text, raw=raw)
def _openai_size(
model: str,
aspect_ratio: str | None,
image_size: str | None,
) -> str:
"""Resolve aspect ratio or image_size to an OpenAI Images API size string."""
sizes, supported_sizes = _openai_size_options(model)
explicit_size = _normalize_openai_image_size(image_size)
if explicit_size and _openai_explicit_size_supported(
explicit_size,
supported_sizes=supported_sizes,
):
return explicit_size
if explicit_size:
logger.warning(
"OpenAI image size '{}' is not supported by {}; using aspect ratio/default size",
explicit_size,
model,
)
if aspect_ratio and aspect_ratio in sizes:
return sizes[aspect_ratio]
return "1024x1024"
def _openai_is_gpt_image_model(model: str) -> bool:
normalized = model.lower()
return normalized.startswith(("gpt-image", "chatgpt-image"))
def _openai_size_options(model: str) -> tuple[dict[str, str], set[str] | None]:
normalized = model.lower()
if normalized.startswith("dall-e-2"):
return _OPENAI_DALLE2_ASPECT_RATIO_SIZES, _OPENAI_DALLE2_SUPPORTED_SIZES
if normalized.startswith("dall-e-3"):
return _OPENAI_DALLE3_ASPECT_RATIO_SIZES, _OPENAI_DALLE3_SUPPORTED_SIZES
if normalized.startswith("gpt-image-2"):
return _OPENAI_GPT_IMAGE_ASPECT_RATIO_SIZES, None
return _OPENAI_GPT_IMAGE_ASPECT_RATIO_SIZES, _OPENAI_GPT_IMAGE_SUPPORTED_SIZES
def _normalize_openai_image_size(image_size: str | None) -> str | None:
if not image_size:
return None
normalized = image_size.strip().lower()
return normalized or None
def _openai_explicit_size_supported(
size: str,
*,
supported_sizes: set[str] | None,
) -> bool:
if supported_sizes is not None:
return size in supported_sizes
width, sep, height = size.partition("x")
return bool(sep and width.isdecimal() and height.isdecimal())
async def _openai_images_from_payload(
client: httpx.AsyncClient,
payload: dict[str, Any],
) -> list[str]:
"""Extract images from OpenAI Images API response.
Handles both ``b64_json`` (preferred) and ``url`` (downloaded) formats.
"""
images: list[str] = []
for item in payload.get("data") or []:
if not isinstance(item, dict):
continue
b64 = item.get("b64_json")
if isinstance(b64, str) and b64:
images.append(_b64_image_data_url(b64))
continue
url = item.get("url")
if isinstance(url, str) and url:
images.append(await _download_image_data_url(client, url))
return images
def _codex_responses_images_from_payload(payload: dict[str, Any]) -> list[str]:
"""Extract images from Codex Responses API ``image_generation_call`` output."""
images: list[str] = []
for item in payload.get("output") or []:
if not isinstance(item, dict):
continue
if item.get("type") != "image_generation_call":
continue
result = item.get("result")
if isinstance(result, str):
images.append(result if result.startswith("data:image/") else _b64_image_data_url(result))
continue
if isinstance(result, dict):
image_url = result.get("image_url") or result.get("image") or ""
if isinstance(image_url, str):
images.append(image_url if image_url.startswith("data:image/") else _b64_image_data_url(image_url))
return images
async def _parse_codex_sse_images(
response: httpx.Response,
) -> tuple[list[str], str]:
"""Parse a Codex Responses API SSE stream for image generation output.
Returns ``(images, content_text)``.
"""
import json as _json
images: list[str] = []
text_parts: list[str] = []
buffer: list[str] = []
async for line_bytes in response.aiter_lines():
line = line_bytes.strip()
if line == "":
if buffer:
data_lines = []
for bl in buffer:
if bl.startswith("data:"):
data_lines.append(bl[5:].strip())
buffer.clear()
if data_lines:
raw = "".join(data_lines)
if raw == "[DONE]":
break
try:
event = _json.loads(raw)
except Exception:
continue
ev_type = event.get("type", "")
if ev_type in ("error", "response.failed"):
logger.error("Codex SSE failure: {}", raw[:2000])
_collect_images_from_sse_event(event, images)
_collect_text_from_sse_event(event, text_parts)
continue
buffer.append(line)
# flush remaining
if buffer:
data_lines = [bl[5:].strip() for bl in buffer if bl.startswith("data:")]
raw = "".join(data_lines)
if raw and raw != "[DONE]":
try:
event = _json.loads(raw)
except Exception:
pass
else:
_collect_images_from_sse_event(event, images)
_collect_text_from_sse_event(event, text_parts)
return images, "".join(text_parts).strip()
def _collect_images_from_sse_event(event: dict[str, Any], images: list[str]) -> None:
if event.get("type") != "response.output_item.done":
return
item = event.get("item") or {}
if item.get("type") != "image_generation_call":
return
result = item.get("result")
if isinstance(result, str):
if result.startswith("data:image/"):
images.append(result)
else:
images.append(_b64_image_data_url(result))
elif isinstance(result, dict):
image_url = result.get("image_url") or result.get("image") or ""
if isinstance(image_url, str):
if image_url.startswith("data:image/"):
images.append(image_url)
else:
images.append(_b64_image_data_url(image_url))
def _collect_text_from_sse_event(event: dict[str, Any], text_parts: list[str]) -> None:
if event.get("type") == "response.output_text.delta":
delta = event.get("delta")
if isinstance(delta, str) and delta:
text_parts.append(delta)
# ---------------------------------------------------------------------------
# StepFun (阶跃星辰) image generation
# ---------------------------------------------------------------------------
@@ -879,12 +1445,159 @@ def _stepfun_images_from_payload(payload: dict[str, Any]) -> list[str]:
return images
# ---------------------------------------------------------------------------
# Zhipu (智谱) image generation
# ---------------------------------------------------------------------------
_ZHIPU_TIMEOUT_S = 300.0
_ZHIPU_ASPECT_RATIO_SIZES = {
"1:1": "1280x1280",
"16:9": "1728x960",
"9:16": "960x1728",
"3:4": "1088x1472",
"4:3": "1472x1088",
}
class ZhipuImageGenerationClient(ImageGenerationProvider):
"""Async client for Zhipu (智谱) image generation API.
Supports:
- Text-to-image via glm-image, cogview-4, cogview-3-flash, etc.
- Aspect ratio selection
- Watermark control
"""
provider_name = "zhipu"
missing_key_message = "Zhipu API key is not configured. Set providers.zhipu.apiKey."
default_timeout = _ZHIPU_TIMEOUT_S
def _default_base_url(self) -> str:
return "https://open.bigmodel.cn/api/paas/v4"
async def generate(
self,
*,
prompt: str,
model: str,
reference_images: list[str] | None = None,
aspect_ratio: str | None = None,
image_size: str | None = None,
) -> GeneratedImageResponse:
if not self.api_key:
raise ImageGenerationError(self.missing_key_message)
if reference_images:
raise ImageGenerationError(
"Zhipu image generation does not support reference images"
)
headers = {
"Authorization": f"Bearer {self.api_key}",
"Content-Type": "application/json",
**self.extra_headers,
}
body: dict[str, Any] = {
"model": model,
"prompt": prompt,
}
size = _zhipu_size(aspect_ratio, image_size)
if size:
body["size"] = size
body.update(self.extra_body)
url = f"{self.api_base}/images/generations"
client = self._client or httpx.AsyncClient(timeout=self.timeout)
try:
return await self._generate_with_client(
client,
headers=headers,
body=body,
url=url,
)
finally:
if self._client is None:
await client.aclose()
async def _generate_with_client(
self,
client: httpx.AsyncClient,
*,
headers: dict[str, str],
body: dict[str, Any],
url: str,
) -> GeneratedImageResponse:
try:
response = await self._http_post(url, headers=headers, body=body, client=client)
except httpx.TimeoutException as exc:
raise ImageGenerationError("Zhipu image generation timed out") from exc
except httpx.RequestError as exc:
raise ImageGenerationError(f"Zhipu image generation request failed: {exc}") from exc
try:
response.raise_for_status()
except httpx.HTTPStatusError as exc:
detail = response.text[:500]
raise ImageGenerationError(f"Zhipu image generation failed: {detail}") from exc
payload = response.json()
images = await _zhipu_images_from_payload(client, payload)
self._require_images(images, payload)
return GeneratedImageResponse(images=images, content="", raw=payload)
def _zhipu_size(
aspect_ratio: str | None,
image_size: str | None,
) -> str:
"""Resolve aspect ratio / image_size to Zhipu size string.
Zhipu glm-image model supports: 1280x1280 (default), 1568x1056,
1056x1568, 1472x1088, 1088x1472, 1728x960, 960x1728.
"""
if image_size and "x" in image_size.lower():
return image_size
if aspect_ratio and aspect_ratio in _ZHIPU_ASPECT_RATIO_SIZES:
return _ZHIPU_ASPECT_RATIO_SIZES[aspect_ratio]
return "1280x1280"
async def _zhipu_images_from_payload(
client: httpx.AsyncClient,
payload: dict[str, Any],
) -> list[str]:
"""Extract image data URLs from Zhipu API response.
Zhipu returns images as temporary URLs that expire after 30 days.
We download and re-encode as base64 data URLs.
"""
images: list[str] = []
for item in payload.get("data") or []:
if not isinstance(item, dict):
continue
url = item.get("url")
if isinstance(url, str) and url:
images.append(await _download_image_data_url(client, url))
return images
# ---------------------------------------------------------------------------
# Provider registration
# ---------------------------------------------------------------------------
register_image_gen_provider(OpenRouterImageGenerationClient)
register_image_gen_provider(AIHubMixImageGenerationClient)
register_image_gen_provider(CodexImageGenerationClient)
register_image_gen_provider(GeminiImageGenerationClient)
register_image_gen_provider(OllamaImageGenerationClient)
register_image_gen_provider(MiniMaxImageGenerationClient)
register_image_gen_provider(OpenAIImageGenerationClient)
register_image_gen_provider(OpenRouterImageGenerationClient)
register_image_gen_provider(StepFunImageGenerationClient)
register_image_gen_provider(ZhipuImageGenerationClient)
+174 -69
View File
@@ -11,6 +11,7 @@ import secrets
import string
import time
import uuid
from collections import deque
from collections.abc import Awaitable, Callable
from ipaddress import ip_address
from typing import TYPE_CHECKING, Any
@@ -74,41 +75,43 @@ _THINKING_STYLE_MAP: dict[str, Any] = {
"enable_thinking": lambda on: {"enable_thinking": on},
"reasoning_split": lambda on: {"reasoning_split": on},
}
_GATEWAY_REASONING_STYLE_MAP: dict[str, Any] = {
"reasoning_effort": lambda effort: {"reasoning": {"effort": effort}},
}
_MODEL_THINKING_STYLES: dict[str, str] = {
**dict.fromkeys(_KIMI_THINKING_MODELS, "thinking_type"),
**dict.fromkeys(_MIMO_THINKING_MODELS, "thinking_type"),
}
def _is_kimi_thinking_model(model_name: str) -> bool:
"""Return True if model_name refers to a Kimi thinking-capable model.
Supports two forms:
- Exact match: e.g. kimi-k2.5 / kimi-k2.6 in _KIMI_THINKING_MODELS
- Slug match: moonshotai/kimi-k2.5 -> the part after the last "/"
is checked against _KIMI_THINKING_MODELS
This covers both the native Moonshot provider (bare slug) and
OpenRouter-style names (``"publisher/slug"``).
"""
name = model_name.lower()
if name in _KIMI_THINKING_MODELS:
return True
if "/" in name and name.rsplit("/", 1)[1] in _KIMI_THINKING_MODELS:
return True
return False
def _model_slug(model_name: str) -> str:
return model_name.lower().rsplit("/", 1)[-1]
def _is_mimo_thinking_model(model_name: str) -> bool:
"""Return True if model_name refers to a MiMo thinking-capable model.
def _model_thinking_style(model_name: str) -> str:
return _MODEL_THINKING_STYLES.get(_model_slug(model_name), "")
Mirrors _is_kimi_thinking_model: gateway providers (e.g. OpenRouter
routing ``xiaomi/mimo-v2.5-pro``) have no ``thinking_style`` on their
spec, so the spec-driven branch in _build_kwargs misses them. The
model-name path catches those cases.
"""
name = model_name.lower()
if name in _MIMO_THINKING_MODELS:
return True
if "/" in name and name.rsplit("/", 1)[1] in _MIMO_THINKING_MODELS:
return True
return False
def _thinking_styles_for(spec: ProviderSpec | None, model_name: str) -> list[str]:
styles: list[str] = []
if spec and spec.thinking_style:
styles.append(spec.thinking_style)
model_style = _model_thinking_style(model_name)
if model_style and model_style not in styles:
styles.append(model_style)
return styles
def _thinking_extra_body(style: str, thinking_enabled: bool) -> dict[str, Any] | None:
builder = _THINKING_STYLE_MAP.get(style)
return builder(thinking_enabled) if builder else None
def _gateway_reasoning_extra_body(style: str, effort: str | None) -> dict[str, Any] | None:
if not effort:
return None
builder = _GATEWAY_REASONING_STYLE_MAP.get(style)
return builder(effort) if builder else None
def _openai_compat_timeout_s() -> float:
@@ -271,6 +274,47 @@ def _deep_merge(base: dict[str, Any], override: dict[str, Any]) -> dict[str, Any
return merged
def _merge_unique_list(base: Any, override: Any) -> Any:
"""Append list values while preserving order and removing duplicates."""
if not isinstance(base, list) or not isinstance(override, list):
return override
result: list[Any] = []
seen: set[str] = set()
for value in [*base, *override]:
try:
key = json.dumps(value, sort_keys=True, ensure_ascii=False)
except Exception:
key = repr(value)
if key in seen:
continue
seen.add(key)
result.append(value)
return result
def _merge_responses_extra_body(
body: dict[str, Any],
extra_body: dict[str, Any],
) -> dict[str, Any]:
"""Merge configured Responses API body fields without clobbering tools."""
reserved = {"include", "tools"}
regular_extra = {key: value for key, value in extra_body.items() if key not in reserved}
merged = _deep_merge(body, regular_extra)
if "include" in extra_body:
merged["include"] = _merge_unique_list(body.get("include"), extra_body["include"])
if "tools" in extra_body:
current_tools = body.get("tools")
configured_tools = extra_body["tools"]
if isinstance(current_tools, list) and isinstance(configured_tools, list):
merged["tools"] = [*current_tools, *configured_tools]
else:
merged["tools"] = configured_tools
return merged
class OpenAICompatProvider(LLMProvider):
"""Unified provider for all OpenAI-compatible APIs.
@@ -286,12 +330,14 @@ class OpenAICompatProvider(LLMProvider):
extra_headers: dict[str, str] | None = None,
spec: ProviderSpec | None = None,
extra_body: dict[str, Any] | None = None,
api_type: str = "auto",
):
super().__init__(api_key, api_base)
self.default_model = default_model
self.extra_headers = extra_headers or {}
self._spec = spec
self._extra_body = extra_body or {}
self._api_type = api_type if spec and spec.name == "openai" else "auto"
if api_key and spec and spec.env_key:
self._setup_env(api_key, api_base)
@@ -425,6 +471,10 @@ class OpenAICompatProvider(LLMProvider):
return tool_call_id
return hashlib.sha1(tool_call_id.encode()).hexdigest()[:9]
def _should_normalize_tool_call_ids(self) -> bool:
"""Return True for providers that reject normal OpenAI tool call IDs."""
return bool(self._spec and self._spec.name == "mistral")
@staticmethod
def _normalize_tool_call_arguments(arguments: Any) -> str:
"""Force function.arguments into a valid JSON object string."""
@@ -461,22 +511,60 @@ class OpenAICompatProvider(LLMProvider):
"""Strip non-standard keys, normalize tool_call IDs."""
sanitized = LLMProvider._sanitize_request_messages(messages, _ALLOWED_MSG_KEYS)
id_map: dict[str, str] = {}
pending_tool_ids: dict[str, deque[str]] = {}
force_string_content = bool(self._spec and self._spec.name == "deepseek")
normalize_tool_ids = self._should_normalize_tool_call_ids()
def map_id(value: Any) -> Any:
if not isinstance(value, str):
return value
if not normalize_tool_ids:
return value
return id_map.setdefault(value, self._normalize_tool_call_id(value))
def unique_tool_id(value: Any, used_ids: set[str], idx: int) -> str:
if isinstance(value, str) and value:
base = map_id(value)
else:
base = _short_tool_id()
if not isinstance(base, str) or not base:
base = _short_tool_id()
if base not in used_ids:
return base
seed = value if isinstance(value, str) and value else base
salt = 1
while True:
candidate = self._normalize_tool_call_id(f"{seed}:{idx}:{salt}")
if isinstance(candidate, str) and candidate not in used_ids:
return candidate
salt += 1
def map_tool_result_id(value: Any) -> Any:
if not isinstance(value, str):
return value
queue = pending_tool_ids.get(value)
if queue:
mapped = queue.popleft()
if not queue:
pending_tool_ids.pop(value, None)
return mapped
return map_id(value)
for clean in sanitized:
if isinstance(clean.get("tool_calls"), list):
normalized = []
for tc in clean["tool_calls"]:
used_ids: set[str] = set()
for idx, tc in enumerate(clean["tool_calls"]):
if not isinstance(tc, dict):
normalized.append(tc)
continue
tc_clean = dict(tc)
tc_clean["id"] = map_id(tc_clean.get("id"))
raw_id = tc_clean.get("id")
mapped_id = unique_tool_id(raw_id, used_ids, idx)
tc_clean["id"] = mapped_id
used_ids.add(mapped_id)
if isinstance(raw_id, str) and raw_id:
pending_tool_ids.setdefault(raw_id, deque()).append(mapped_id)
function = tc_clean.get("function")
if isinstance(function, dict):
function_clean = dict(function)
@@ -494,7 +582,7 @@ class OpenAICompatProvider(LLMProvider):
# that mix non-empty content with tool_calls.
clean["content"] = None
if "tool_call_id" in clean and clean["tool_call_id"]:
clean["tool_call_id"] = map_id(clean["tool_call_id"])
clean["tool_call_id"] = map_tool_result_id(clean["tool_call_id"])
if (
force_string_content
and not (clean.get("role") == "assistant" and clean.get("tool_calls"))
@@ -581,39 +669,27 @@ class OpenAICompatProvider(LLMProvider):
if wire_effort and semantic_effort != "none":
kwargs["reasoning_effort"] = wire_effort
# Provider-specific thinking parameters.
# Only sent when reasoning_effort is explicitly configured so that
# the provider default is preserved otherwise.
# The mapping is driven by ProviderSpec.thinking_style so that adding
# a new provider never requires touching this function.
if spec and spec.thinking_style and reasoning_effort is not None:
# Only send thinking controls when reasoning_effort is explicit so
# omitting the config preserves each provider's default.
if reasoning_effort is not None:
thinking_enabled = semantic_effort not in ("none", "minimal")
extra = _THINKING_STYLE_MAP.get(spec.thinking_style, lambda _: None)(thinking_enabled)
if extra:
kwargs.setdefault("extra_body", {}).update(extra)
for thinking_style in _thinking_styles_for(spec, model_name):
extra = _thinking_extra_body(thinking_style, thinking_enabled)
if extra:
kwargs.setdefault("extra_body", {}).update(extra)
gateway_style = getattr(spec, "gateway_reasoning_style", "") if spec else ""
if gateway_style and _model_thinking_style(model_name):
extra = _gateway_reasoning_extra_body(gateway_style, semantic_effort)
if extra:
kwargs.setdefault("extra_body", {}).update(extra)
# Model-level thinking injection for Kimi thinking-capable models.
# Strip any provider prefix (e.g. "moonshotai/") before the set lookup
# so that OpenRouter-style names like "moonshotai/kimi-k2.5" are handled
# identically to bare names like "kimi-k2.5".
if reasoning_effort is not None and _is_kimi_thinking_model(model_name):
thinking_enabled = semantic_effort not in ("none", "minimal")
kwargs.setdefault("extra_body", {}).update(
{"thinking": {"type": "enabled" if thinking_enabled else "disabled"}}
)
# Model-level thinking injection for MiMo thinking-capable models.
# Same shape as Kimi: gateway providers (OpenRouter, etc.) lack the
# xiaomi_mimo spec's thinking_style, so the spec-driven branch above
# misses them — match by model name to catch "xiaomi/mimo-v2.5-pro"
# and friends. (Direct xiaomi_mimo requests are also covered here;
# both branches write the same payload, so the dict update is a
# safe no-op for already-handled cases.)
if reasoning_effort is not None and _is_mimo_thinking_model(model_name):
thinking_enabled = semantic_effort not in ("none", "minimal")
kwargs.setdefault("extra_body", {}).update(
{"thinking": {"type": "enabled" if thinking_enabled else "disabled"}}
)
# Moonshot rejects requests that carry both 'reasoning_effort'
# and the native 'thinking' param. We already expressed the
# user's intent via the provider-native shape, so drop the
# redundant wire-level kwarg. Only kimi models need this —
# Xiaomi's API accepts both params.
if _model_slug(model_name) in _KIMI_THINKING_MODELS:
kwargs.pop("reasoning_effort", None)
if tools:
kwargs["tools"] = tools
@@ -628,8 +704,7 @@ class OpenAICompatProvider(LLMProvider):
and semantic_effort not in ("none", "minimal")
and (
(spec and spec.thinking_style)
or _is_kimi_thinking_model(model_name)
or _is_mimo_thinking_model(model_name)
or _model_thinking_style(model_name)
)
)
implicit_deepseek_thinking = (
@@ -660,8 +735,14 @@ class OpenAICompatProvider(LLMProvider):
reasoning_effort: str | None,
) -> bool:
"""Use Responses API only for direct OpenAI requests that benefit from it."""
if self._api_type == "chat_completions":
return False
if self._spec and self._spec.name not in ("openai", "github_copilot"):
return False
if self._api_type == "responses":
# Explicit configuration means Responses is mandatory; do not
# consult the circuit breaker or fall back to Chat Completions.
return True
if self._spec is None or self._spec.name != "github_copilot":
if not _is_direct_openai_base(self._effective_base):
return False
@@ -675,7 +756,14 @@ class OpenAICompatProvider(LLMProvider):
if not wants:
return False
# Circuit breaker: skip after repeated failures, probe periodically.
return self._responses_circuit_allows_probe(model, reasoning_effort)
def _responses_circuit_allows_probe(
self,
model: str | None,
reasoning_effort: str | None,
) -> bool:
"""Return False when the Responses API circuit breaker is open."""
key = _responses_circuit_key(model, self.default_model, reasoning_effort)
failures = self._responses_failures.get(key, 0)
if failures >= _RESPONSES_FAILURE_THRESHOLD:
@@ -767,6 +855,10 @@ class OpenAICompatProvider(LLMProvider):
body["tools"] = convert_tools(tools)
body["tool_choice"] = tool_choice or "auto"
extra_body = getattr(self, "_extra_body", {})
if extra_body:
body = _merge_responses_extra_body(body, extra_body)
return body
# ------------------------------------------------------------------
@@ -931,7 +1023,7 @@ class OpenAICompatProvider(LLMProvider):
args = json_repair.loads(args)
ec, prov, fn_prov = _extract_tc_extras(tc)
parsed_tool_calls.append(ToolCallRequest(
id=_short_tool_id(),
id=str(tc_map.get("id") or _short_tool_id()),
name=str(fn.get("name") or ""),
arguments=args if isinstance(args, dict) else {},
extra_content=ec,
@@ -974,7 +1066,7 @@ class OpenAICompatProvider(LLMProvider):
args = json_repair.loads(args)
ec, prov, fn_prov = _extract_tc_extras(tc)
tool_calls.append(ToolCallRequest(
id=_short_tool_id(),
id=str(getattr(tc, "id", None) or _short_tool_id()),
name=tc.function.name,
arguments=args,
extra_content=ec,
@@ -1097,6 +1189,15 @@ class OpenAICompatProvider(LLMProvider):
if delta:
_accum_legacy_function_call(getattr(delta, "function_call", None))
# Some providers (e.g. Zhipu/GLM) reuse the same tool_call id for
# parallel tool calls in streaming mode. Deduplicate before building
# the response so downstream tool messages don't collide.
_seen_tc_ids: set[str] = set()
for b in tc_bufs.values():
if not b["id"] or b["id"] in _seen_tc_ids:
b["id"] = _short_tool_id()
_seen_tc_ids.add(b["id"])
return LLMResponse(
content="".join(content_parts) or None,
tool_calls=[
@@ -1228,6 +1329,8 @@ class OpenAICompatProvider(LLMProvider):
# falling back to /chat/completions cannot succeed and would
# hide the real error.
raise
if self._api_type == "responses":
raise
if not self._should_fallback_from_responses_error(responses_error):
raise
self._record_responses_failure(model, reasoning_effort)
@@ -1301,6 +1404,8 @@ class OpenAICompatProvider(LLMProvider):
# falling back to /chat/completions cannot succeed and would
# hide the real error.
raise
if self._api_type == "responses":
raise
if not self._should_fallback_from_responses_error(responses_error):
raise
self._record_responses_failure(model, reasoning_effort)
@@ -15,6 +15,7 @@ def convert_messages(messages: list[dict[str, Any]]) -> tuple[str, list[dict[str
"""
system_prompt = ""
input_items: list[dict[str, Any]] = []
used_item_ids: set[str] = set()
for idx, msg in enumerate(messages):
role = msg.get("role")
@@ -30,17 +31,19 @@ def convert_messages(messages: list[dict[str, Any]]) -> tuple[str, list[dict[str
if role == "assistant":
if isinstance(content, str) and content:
message_id = _unique_item_id(f"msg_{idx}", used_item_ids)
input_items.append({
"type": "message", "role": "assistant",
"content": [{"type": "output_text", "text": content}],
"status": "completed", "id": f"msg_{idx}",
"status": "completed", "id": message_id,
})
for tool_call in msg.get("tool_calls", []) or []:
fn = tool_call.get("function") or {}
call_id, item_id = split_tool_call_id(tool_call.get("id"))
response_item_id = _unique_item_id(item_id or f"fc_{idx}", used_item_ids)
input_items.append({
"type": "function_call",
"id": item_id or f"fc_{idx}",
"id": response_item_id,
"call_id": call_id or f"call_{idx}",
"name": fn.get("name"),
"arguments": fn.get("arguments") or "{}",
@@ -97,6 +100,20 @@ def convert_tools(tools: list[dict[str, Any]]) -> list[dict[str, Any]]:
return converted
def _unique_item_id(item_id: str, used: set[str]) -> str:
"""Return a Responses input item id that is unique within one request."""
if item_id not in used:
used.add(item_id)
return item_id
suffix = 2
while f"{item_id}_{suffix}" in used:
suffix += 1
unique = f"{item_id}_{suffix}"
used.add(unique)
return unique
def split_tool_call_id(tool_call_id: Any) -> tuple[str, str | None]:
"""Split a compound ``call_id|item_id`` string.
+19 -13
View File
@@ -34,7 +34,7 @@ class ProviderSpec:
display_name: str = "" # shown in `nanobot status`
# which provider implementation to use
# "openai_compat" | "anthropic" | "azure_openai" | "openai_codex" | "github_copilot" | "xai_oauth" | "bedrock"
# "openai_compat" | "anthropic" | "azure_openai" | "openai_codex" | "github_copilot" | "bedrock"
backend: str = "openai_compat"
# extra env vars, e.g. (("ZHIPUAI_API_KEY", "{api_key}"),)
@@ -71,6 +71,11 @@ class ProviderSpec:
# "reasoning_split" — {"reasoning_split": true/false} (MiniMax)
thinking_style: str = ""
# Gateway-native reasoning control to pair with model-level thinking styles.
# "reasoning_effort" — {"reasoning": {"effort": <none|minimal|...>}}
# (OpenRouter)
gateway_reasoning_style: str = ""
# When True, treat the "reasoning" response field as formal content
# when "content" is empty. Only set this for providers (e.g. StepFun)
# whose API returns the actual answer in "reasoning" instead of "content".
@@ -142,6 +147,7 @@ PROVIDERS: tuple[ProviderSpec, ...] = (
detect_by_base_keyword="openrouter",
default_api_base="https://openrouter.ai/api/v1",
supports_prompt_caching=True,
gateway_reasoning_style="reasoning_effort",
),
# Hugging Face Inference Providers: OpenAI-compatible router for chat models.
ProviderSpec(
@@ -193,6 +199,18 @@ PROVIDERS: tuple[ProviderSpec, ...] = (
default_api_base="https://api.siliconflow.cn/v1",
),
# Novita AI: OpenAI-compatible gateway for hosted model APIs.
ProviderSpec(
name="novita",
keywords=("novita",),
env_key="NOVITA_API_KEY",
display_name="Novita AI",
backend="openai_compat",
is_gateway=True,
detect_by_base_keyword="novita",
default_api_base="https://api.novita.ai/openai",
),
# VolcEngine (火山引擎): OpenAI-compatible gateway, pay-per-use models
ProviderSpec(
name="volcengine",
@@ -291,18 +309,6 @@ PROVIDERS: tuple[ProviderSpec, ...] = (
is_oauth=True,
supports_max_completion_tokens=True,
),
# xAI Grok OAuth: SuperGrok subscription-backed Responses API provider
ProviderSpec(
name="xai_oauth",
keywords=("xai-oauth", "grok-oauth", "x-ai-oauth", "xai-grok-oauth"),
env_key="",
display_name="xAI Grok OAuth",
backend="xai_oauth",
default_api_base="https://api.x.ai/v1",
strip_model_prefix=True,
is_oauth=True,
supports_max_completion_tokens=True,
),
# DeepSeek: OpenAI-compatible at api.deepseek.com
ProviderSpec(
name="deepseek",
+27 -8
View File
@@ -7,6 +7,25 @@ from pathlib import Path
import httpx
from loguru import logger
_TRANSCRIPTIONS_PATH = "audio/transcriptions"
def _resolve_transcription_url(api_base: str | None, default_url: str) -> str:
"""Resolve the full transcription endpoint URL.
Accepts either a chat-style base (e.g. ``https://api.groq.com/openai/v1``)
or a complete URL already ending in ``/audio/transcriptions``. A chat-style
base the form users naturally copy from their LLM provider config gets
the path appended instead of being POSTed verbatim and 404ing (#3637).
"""
if not api_base:
return default_url
base = api_base.rstrip("/")
if base.endswith(_TRANSCRIPTIONS_PATH):
return base
return f"{base}/{_TRANSCRIPTIONS_PATH}"
# Up to 3 retries (4 attempts total) with exponential backoff on transient
# failures. Whisper endpoints occasionally return 502/503 under load, and
# mobile-network transcription callers hit sporadic connect/read errors.
@@ -127,12 +146,12 @@ class OpenAITranscriptionProvider:
language: str | None = None,
):
self.api_key = api_key or os.environ.get("OPENAI_API_KEY")
self.api_url = (
api_base
or os.environ.get("OPENAI_TRANSCRIPTION_BASE_URL")
or "https://api.openai.com/v1/audio/transcriptions"
self.api_url = _resolve_transcription_url(
api_base or os.environ.get("OPENAI_TRANSCRIPTION_BASE_URL"),
"https://api.openai.com/v1/audio/transcriptions",
)
self.language = language or None
logger.debug("OpenAI transcription endpoint: {}", self.api_url)
async def transcribe(self, file_path: str | Path) -> str:
if not self.api_key:
@@ -166,12 +185,12 @@ class GroqTranscriptionProvider:
language: str | None = None,
):
self.api_key = api_key or os.environ.get("GROQ_API_KEY")
self.api_url = (
api_base
or os.environ.get("GROQ_BASE_URL")
or "https://api.groq.com/openai/v1/audio/transcriptions"
self.api_url = _resolve_transcription_url(
api_base or os.environ.get("GROQ_BASE_URL"),
"https://api.groq.com/openai/v1/audio/transcriptions",
)
self.language = language or None
logger.debug("Groq transcription endpoint: {}", self.api_url)
async def transcribe(self, file_path: str | Path) -> str:
"""
-768
View File
@@ -1,768 +0,0 @@
"""xAI Grok OAuth credential flow and Responses provider."""
from __future__ import annotations
import asyncio
import base64
import json
import os
import secrets
import time
import webbrowser
from collections.abc import Awaitable, Callable
from contextlib import suppress
from dataclasses import dataclass
from hashlib import sha256
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from threading import Event, Thread
from typing import Any
from urllib.parse import parse_qs, urlencode, urlparse
import httpx
from filelock import FileLock
from nanobot.providers.base import LLMProvider, LLMResponse, ToolCallRequest
from nanobot.providers.openai_responses import consume_sse, convert_messages, convert_tools
DEFAULT_XAI_API_BASE = "https://api.x.ai/v1"
DEFAULT_XAI_AUTH_ISSUER = "https://auth.x.ai"
DEFAULT_XAI_DISCOVERY_URL = f"{DEFAULT_XAI_AUTH_ISSUER}/.well-known/openid-configuration"
DEFAULT_XAI_REDIRECT_URI = "http://127.0.0.1:56121/callback"
DEFAULT_XAI_CLIENT_ID = "b1a00492-073a-47ea-816f-4c329264a828"
DEFAULT_XAI_SCOPE = "openid profile email offline_access grok-cli:access api:access"
_SERVICE_NAME = "nanobot.xai_oauth"
_SECRET_USERNAME = "default"
_TOKEN_SKEW_SECONDS = 60
_LOGIN_TIMEOUT_SECONDS = 300
@dataclass(frozen=True)
class XaiOAuthEndpoints:
authorization_endpoint: str
token_endpoint: str
@dataclass(frozen=True)
class XaiOAuthCredential:
access_token: str
refresh_token: str = ""
expires_at: float | None = None
account_id: str | None = None
token_type: str = "Bearer"
api_base: str = DEFAULT_XAI_API_BASE
storage: str = "unknown"
@property
def is_expiring(self) -> bool:
return self.expires_at is not None and self.expires_at <= time.time() + _TOKEN_SKEW_SECONDS
def _nanobot_home() -> Path:
override = os.environ.get("NANOBOT_HOME")
if override:
return Path(override).expanduser()
from nanobot.config.loader import get_config_path
return get_config_path().parent
def _auth_dir() -> Path:
return _nanobot_home() / "auth"
def get_xai_oauth_metadata_path() -> Path:
"""Return the non-secret xAI OAuth metadata path."""
return _auth_dir() / "xai-oauth.json"
def _lock_path() -> Path:
return get_xai_oauth_metadata_path().with_suffix(".lock")
def _write_private_json(path: Path, payload: dict[str, Any]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
with suppress(OSError):
path.parent.chmod(0o700)
tmp = path.with_suffix(path.suffix + ".tmp")
tmp.write_text(json.dumps(payload, indent=2, ensure_ascii=False), encoding="utf-8")
with suppress(OSError):
tmp.chmod(0o600)
tmp.replace(path)
with suppress(OSError):
path.chmod(0o600)
def _read_json(path: Path) -> dict[str, Any]:
return json.loads(path.read_text(encoding="utf-8"))
def _keyring_set(tokens: dict[str, Any]) -> bool:
try:
import keyring # type: ignore[import-not-found]
keyring.set_password(_SERVICE_NAME, _SECRET_USERNAME, json.dumps(tokens))
return True
except Exception:
return False
def _keyring_get() -> dict[str, Any] | None:
try:
import keyring # type: ignore[import-not-found]
raw = keyring.get_password(_SERVICE_NAME, _SECRET_USERNAME)
except Exception:
return None
if not raw:
return None
try:
payload = json.loads(raw)
except json.JSONDecodeError:
return None
return payload if isinstance(payload, dict) else None
def _keyring_delete() -> None:
try:
import keyring # type: ignore[import-not-found]
keyring.delete_password(_SERVICE_NAME, _SECRET_USERNAME)
except Exception:
pass
def _token_payload(credential: XaiOAuthCredential) -> dict[str, Any]:
return {
"access_token": credential.access_token,
"refresh_token": credential.refresh_token,
"expires_at": credential.expires_at,
"token_type": credential.token_type,
}
def save_xai_oauth_credential(credential: XaiOAuthCredential) -> XaiOAuthCredential:
"""Persist xAI OAuth tokens, preferring OS keychain storage."""
with FileLock(str(_lock_path())):
tokens = _token_payload(credential)
metadata: dict[str, Any] = {
"provider": "xai_oauth",
"api_base": credential.api_base,
"account_id": credential.account_id,
"expires_at": credential.expires_at,
"updated_at": int(time.time()),
}
if _keyring_set(tokens):
metadata["storage"] = "keyring"
else:
metadata["storage"] = "file"
metadata["tokens"] = tokens
_write_private_json(get_xai_oauth_metadata_path(), metadata)
return XaiOAuthCredential(
access_token=credential.access_token,
refresh_token=credential.refresh_token,
expires_at=credential.expires_at,
account_id=credential.account_id,
token_type=credential.token_type,
api_base=credential.api_base,
storage=str(metadata["storage"]),
)
def load_xai_oauth_credential() -> XaiOAuthCredential | None:
"""Load xAI OAuth credentials from keyring or the private file fallback."""
path = get_xai_oauth_metadata_path()
if not path.exists():
return None
with FileLock(str(_lock_path())):
try:
metadata = _read_json(path)
except (OSError, json.JSONDecodeError):
return None
storage = str(metadata.get("storage") or "file")
tokens = _keyring_get() if storage == "keyring" else metadata.get("tokens")
if not isinstance(tokens, dict):
return None
access_token = str(tokens.get("access_token") or "")
if not access_token:
return None
return XaiOAuthCredential(
access_token=access_token,
refresh_token=str(tokens.get("refresh_token") or ""),
expires_at=_as_float(tokens.get("expires_at") or metadata.get("expires_at")),
account_id=_as_str(metadata.get("account_id")),
token_type=str(tokens.get("token_type") or "Bearer"),
api_base=str(metadata.get("api_base") or DEFAULT_XAI_API_BASE),
storage=storage,
)
def delete_xai_oauth_credentials() -> list[Path]:
"""Delete persisted xAI OAuth credentials and return removed local paths."""
removed: list[Path] = []
path = get_xai_oauth_metadata_path()
lock_path = _lock_path()
with FileLock(str(lock_path)):
_keyring_delete()
try:
path.unlink()
removed.append(path)
except FileNotFoundError:
pass
try:
lock_path.unlink()
except FileNotFoundError:
pass
return removed
def get_xai_oauth_login_status() -> XaiOAuthCredential | None:
return load_xai_oauth_credential()
def pkce_challenge(verifier: str) -> str:
digest = sha256(verifier.encode("ascii")).digest()
return base64.urlsafe_b64encode(digest).decode("ascii").rstrip("=")
def _new_pkce_verifier() -> str:
return base64.urlsafe_b64encode(secrets.token_bytes(48)).decode("ascii").rstrip("=")
def build_xai_authorization_url(
endpoints: XaiOAuthEndpoints,
*,
verifier: str,
state: str,
nonce: str | None = None,
redirect_uri: str = DEFAULT_XAI_REDIRECT_URI,
) -> str:
params = {
"response_type": "code",
"client_id": DEFAULT_XAI_CLIENT_ID,
"redirect_uri": redirect_uri,
"scope": DEFAULT_XAI_SCOPE,
"code_challenge": pkce_challenge(verifier),
"code_challenge_method": "S256",
"state": state,
"nonce": nonce or secrets.token_urlsafe(16),
"plan": "generic",
"referrer": "nanobot",
}
return f"{endpoints.authorization_endpoint}?{urlencode(params)}"
def discover_xai_oauth_endpoints() -> XaiOAuthEndpoints:
try:
with httpx.Client(timeout=20.0, follow_redirects=True, trust_env=True) as client:
response = client.get(DEFAULT_XAI_DISCOVERY_URL)
response.raise_for_status()
payload = response.json()
except Exception:
payload = {}
endpoints = XaiOAuthEndpoints(
authorization_endpoint=str(
payload.get("authorization_endpoint")
or f"{DEFAULT_XAI_AUTH_ISSUER}/authorize"
),
token_endpoint=str(
payload.get("token_endpoint")
or f"{DEFAULT_XAI_AUTH_ISSUER}/oauth/token"
),
)
_validate_xai_endpoint(endpoints.authorization_endpoint, "authorization_endpoint")
_validate_xai_endpoint(endpoints.token_endpoint, "token_endpoint")
return endpoints
def _validate_xai_endpoint(url: str, label: str) -> None:
parsed = urlparse(url)
host = parsed.hostname or ""
if parsed.scheme != "https" or not (host == "x.ai" or host.endswith(".x.ai")):
raise RuntimeError(f"Refusing non-xAI OAuth {label}: {url}")
def _parse_callback_value(raw: str) -> tuple[str, str | None]:
raw = raw.strip()
parsed = urlparse(raw)
if parsed.scheme and parsed.netloc:
params = parse_qs(parsed.query)
code = (params.get("code") or [""])[0]
state = (params.get("state") or [None])[0]
if not code:
raise RuntimeError("OAuth callback URL did not contain a code.")
return code, state
if raw.startswith("?") or "=" in raw:
params = parse_qs(raw.lstrip("?"))
code = (params.get("code") or [""])[0]
state = (params.get("state") or [None])[0]
if not code:
raise RuntimeError("OAuth callback query did not contain a code.")
return code, state
if raw:
return raw, None
raise RuntimeError("No OAuth code provided.")
def _decode_jwt_payload(token: str) -> dict[str, Any]:
parts = token.split(".")
if len(parts) < 2:
return {}
data = parts[1] + "=" * (-len(parts[1]) % 4)
try:
decoded = base64.urlsafe_b64decode(data.encode("ascii"))
payload = json.loads(decoded)
except Exception:
return {}
return payload if isinstance(payload, dict) else {}
def _credential_from_token_response(payload: dict[str, Any], previous: XaiOAuthCredential | None = None) -> XaiOAuthCredential:
access_token = str(payload.get("access_token") or "")
if not access_token:
raise RuntimeError("xAI token response did not include an access token.")
claims = _decode_jwt_payload(access_token)
id_claims = _decode_jwt_payload(str(payload.get("id_token") or ""))
expires_at = _as_float(payload.get("expires_at"))
if expires_at is None:
expires_in = _as_float(payload.get("expires_in"))
expires_at = time.time() + expires_in if expires_in else _as_float(claims.get("exp"))
account_id = (
_as_str(id_claims.get("email"))
or _as_str(id_claims.get("preferred_username"))
or _as_str(id_claims.get("sub"))
or _as_str(claims.get("sub"))
or (previous.account_id if previous else None)
)
refresh_token = str(payload.get("refresh_token") or (previous.refresh_token if previous else ""))
return XaiOAuthCredential(
access_token=access_token,
refresh_token=refresh_token,
expires_at=expires_at,
account_id=account_id,
token_type=str(payload.get("token_type") or (previous.token_type if previous else "Bearer")),
api_base=previous.api_base if previous else DEFAULT_XAI_API_BASE,
)
def exchange_xai_oauth_code(
code: str,
*,
verifier: str,
endpoints: XaiOAuthEndpoints | None = None,
redirect_uri: str = DEFAULT_XAI_REDIRECT_URI,
) -> XaiOAuthCredential:
endpoints = endpoints or discover_xai_oauth_endpoints()
challenge = pkce_challenge(verifier)
with httpx.Client(timeout=30.0, follow_redirects=True, trust_env=True) as client:
response = client.post(
endpoints.token_endpoint,
headers={"Accept": "application/json"},
data={
"grant_type": "authorization_code",
"client_id": DEFAULT_XAI_CLIENT_ID,
"code": code,
"redirect_uri": redirect_uri,
"code_verifier": verifier,
"code_challenge": challenge,
"code_challenge_method": "S256",
},
)
if response.status_code >= 400:
raise RuntimeError(f"xAI token exchange failed: HTTP {response.status_code}: {response.text[:500]}")
return _credential_from_token_response(response.json())
def refresh_xai_oauth_credential(credential: XaiOAuthCredential | None = None) -> XaiOAuthCredential:
credential = credential or load_xai_oauth_credential()
if not credential or not credential.refresh_token:
raise RuntimeError("xAI Grok OAuth is not logged in. Run: nanobot provider login xai-oauth")
endpoints = discover_xai_oauth_endpoints()
with httpx.Client(timeout=30.0, follow_redirects=True, trust_env=True) as client:
response = client.post(
endpoints.token_endpoint,
headers={"Accept": "application/json"},
data={
"grant_type": "refresh_token",
"client_id": DEFAULT_XAI_CLIENT_ID,
"refresh_token": credential.refresh_token,
},
)
if response.status_code >= 400:
raise RuntimeError(f"xAI token refresh failed: HTTP {response.status_code}: {response.text[:500]}")
return save_xai_oauth_credential(_credential_from_token_response(response.json(), previous=credential))
def resolve_xai_oauth_credential(*, force_refresh: bool = False) -> XaiOAuthCredential:
credential = load_xai_oauth_credential()
if not credential:
raise RuntimeError("xAI Grok OAuth is not logged in. Run: nanobot provider login xai-oauth")
if force_refresh or credential.is_expiring:
credential = refresh_xai_oauth_credential(credential)
return credential
def login_xai_oauth_interactive(
print_fn: Callable[[str], None] | None = None,
prompt_fn: Callable[[str], str] | None = None,
open_browser: bool = True,
manual_paste: bool = False,
timeout_seconds: int = _LOGIN_TIMEOUT_SECONDS,
) -> XaiOAuthCredential:
"""Run browser PKCE login and persist xAI OAuth credentials."""
printer = print_fn or print
prompt = prompt_fn or input
endpoints = discover_xai_oauth_endpoints()
verifier = _new_pkce_verifier()
state = secrets.token_urlsafe(24)
nonce = secrets.token_urlsafe(24)
authorize_url = build_xai_authorization_url(
endpoints,
verifier=verifier,
state=state,
nonce=nonce,
)
callback = _LoopbackCallback()
server_started = False if manual_paste else callback.start()
printer(f"Open: {authorize_url}")
if open_browser:
with suppress(Exception):
webbrowser.open(authorize_url)
result: dict[str, str] | None = None
if manual_paste:
printer("Paste the callback URL or xAI fallback code after authorization.")
elif server_started:
try:
result = callback.wait(timeout_seconds)
finally:
callback.stop()
else:
printer("Loopback port 56121 is unavailable; paste the callback URL or xAI fallback code.")
if result:
code = result.get("code") or ""
returned_state = result.get("state")
else:
pasted = prompt("Paste callback URL or fallback code")
code, returned_state = _parse_callback_value(pasted)
if not code:
raise RuntimeError("OAuth login did not return a code.")
if returned_state and returned_state != state:
raise RuntimeError("OAuth state mismatch. Please retry login.")
credential = exchange_xai_oauth_code(code, verifier=verifier, endpoints=endpoints)
return save_xai_oauth_credential(credential)
class _LoopbackCallback:
def __init__(self) -> None:
self._event = Event()
self._result: dict[str, str] = {}
self._server: ThreadingHTTPServer | None = None
self._thread: Thread | None = None
def start(self) -> bool:
owner = self
class Handler(BaseHTTPRequestHandler):
def do_GET(self) -> None: # noqa: N802 - stdlib callback name
parsed = urlparse(self.path)
params = parse_qs(parsed.query)
code = (params.get("code") or [""])[0]
state = (params.get("state") or [""])[0]
if parsed.path != "/callback" or not code:
self.send_response(404)
self.end_headers()
return
owner._result = {"code": code, "state": state}
owner._event.set()
self.send_response(200)
self.send_header("Content-Type", "text/html; charset=utf-8")
self.end_headers()
self.wfile.write(b"<html><body>nanobot xAI OAuth complete. You may close this tab.</body></html>")
def log_message(self, format: str, *args: Any) -> None: # noqa: A002
return
class Server(ThreadingHTTPServer):
allow_reuse_address = True
daemon_threads = True
try:
self._server = Server(("127.0.0.1", 56121), Handler)
except OSError:
return False
self._thread = Thread(target=self._server.serve_forever, daemon=True)
self._thread.start()
return True
def wait(self, timeout_seconds: int) -> dict[str, str] | None:
if self._event.wait(timeout_seconds):
return dict(self._result)
return None
def stop(self) -> None:
if self._server:
self._server.shutdown()
self._server.server_close()
if self._thread:
self._thread.join(timeout=1)
def _as_float(value: Any) -> float | None:
try:
return float(value)
except (TypeError, ValueError):
return None
def _as_str(value: Any) -> str | None:
return value if isinstance(value, str) and value else None
DEFAULT_XAI_MODEL = "xai-oauth/grok-4.3"
class XaiOAuthProvider(LLMProvider):
"""Use a SuperGrok OAuth session to call xAI's Responses API."""
supports_progress_deltas = True
def __init__(self, default_model: str = DEFAULT_XAI_MODEL, config: Any | None = None):
super().__init__(api_key=None, api_base=DEFAULT_XAI_API_BASE)
self.default_model = default_model
self.config = config
async def _call_xai(
self,
messages: list[dict[str, Any]],
tools: list[dict[str, Any]] | None,
model: str | None,
max_tokens: int,
temperature: float,
reasoning_effort: str | None,
tool_choice: str | dict[str, Any] | None,
on_content_delta: Callable[[str], Awaitable[None]] | None = None,
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
) -> LLMResponse:
body = _build_xai_responses_body(
messages=messages,
tools=tools,
model=model or self.default_model,
max_tokens=max_tokens,
temperature=temperature,
reasoning_effort=reasoning_effort,
tool_choice=tool_choice,
hosted_x_search=getattr(self.config, "x_search", None),
)
try:
credential = await asyncio.to_thread(resolve_xai_oauth_credential)
try:
content, tool_calls, finish_reason = await _request_xai(
credential,
body,
on_content_delta=on_content_delta,
on_tool_call_delta=on_tool_call_delta,
)
except _XaiHTTPError as exc:
if exc.status_code != 401:
raise
credential = await asyncio.to_thread(resolve_xai_oauth_credential, force_refresh=True)
content, tool_calls, finish_reason = await _request_xai(
credential,
body,
on_content_delta=on_content_delta,
on_tool_call_delta=on_tool_call_delta,
)
return LLMResponse(content=content, tool_calls=tool_calls, finish_reason=finish_reason)
except Exception as exc:
msg = f"Error calling xAI Grok OAuth: {exc}"
retry_after = getattr(exc, "retry_after", None) or self._extract_retry_after(msg)
return LLMResponse(content=msg, finish_reason="error", retry_after=retry_after)
async def chat(
self,
messages: list[dict[str, Any]],
tools: list[dict[str, Any]] | None = None,
model: str | None = None,
max_tokens: int = 4096,
temperature: float = 0.7,
reasoning_effort: str | None = None,
tool_choice: str | dict[str, Any] | None = None,
) -> LLMResponse:
return await self._call_xai(
messages,
tools,
model,
max_tokens,
temperature,
reasoning_effort,
tool_choice,
)
async def chat_stream(
self,
messages: list[dict[str, Any]],
tools: list[dict[str, Any]] | None = None,
model: str | None = None,
max_tokens: int = 4096,
temperature: float = 0.7,
reasoning_effort: str | None = None,
tool_choice: str | dict[str, Any] | None = None,
on_content_delta: Callable[[str], Awaitable[None]] | None = None,
on_thinking_delta: Callable[[str], Awaitable[None]] | None = None,
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
) -> LLMResponse:
_ = on_thinking_delta
return await self._call_xai(
messages,
tools,
model,
max_tokens,
temperature,
reasoning_effort,
tool_choice,
on_content_delta,
on_tool_call_delta,
)
def get_default_model(self) -> str:
return self.default_model
def _strip_model_prefix(model: str) -> str:
for prefix in ("xai-oauth/", "xai_oauth/", "grok-oauth/", "grok_oauth/"):
if model.startswith(prefix):
return model.split("/", 1)[1]
return model
def _build_xai_responses_body(
*,
messages: list[dict[str, Any]],
tools: list[dict[str, Any]] | None,
model: str,
max_tokens: int,
temperature: float,
reasoning_effort: str | None,
tool_choice: str | dict[str, Any] | None,
hosted_x_search: Any | None = None,
) -> dict[str, Any]:
system_prompt, input_items = convert_messages(LLMProvider._sanitize_empty_content(messages))
if system_prompt:
input_items = [
{"role": "system", "content": [{"type": "input_text", "text": system_prompt}]},
*input_items,
]
body: dict[str, Any] = {
"model": _strip_model_prefix(model),
"store": False,
"stream": True,
"input": input_items,
"tool_choice": tool_choice or "auto",
"parallel_tool_calls": True,
}
if max_tokens:
body["max_output_tokens"] = max_tokens
if temperature is not None:
body["temperature"] = temperature
if reasoning_effort and reasoning_effort.lower() != "none":
body["reasoning"] = {"effort": reasoning_effort}
converted_tools = convert_tools(tools) if tools else []
hosted_tool = _build_xai_hosted_x_search_tool(hosted_x_search)
if hosted_tool:
converted_tools.append(hosted_tool)
if converted_tools:
body["tools"] = converted_tools
return body
def _clean_x_handles(handles: list[str] | None) -> list[str] | None:
if not handles:
return None
cleaned = [str(handle).strip().lstrip("@") for handle in handles if str(handle).strip()]
return cleaned[:10] or None
def _build_xai_hosted_x_search_tool(config: Any | None) -> dict[str, Any] | None:
if not config or not getattr(config, "enable", False):
return None
allowed = _clean_x_handles(getattr(config, "allowed_x_handles", None))
excluded = _clean_x_handles(getattr(config, "excluded_x_handles", None))
if allowed and excluded:
raise ValueError("providers.xai_oauth.x_search cannot set both allowed_x_handles and excluded_x_handles")
tool: dict[str, Any] = {"type": "x_search"}
if allowed:
tool["allowed_x_handles"] = allowed
if excluded:
tool["excluded_x_handles"] = excluded
if getattr(config, "from_date", None):
tool["from_date"] = config.from_date
if getattr(config, "to_date", None):
tool["to_date"] = config.to_date
if getattr(config, "enable_image_understanding", False):
tool["enable_image_understanding"] = True
if getattr(config, "enable_video_understanding", False):
tool["enable_video_understanding"] = True
return tool
class _XaiHTTPError(RuntimeError):
def __init__(self, message: str, *, status_code: int, retry_after: float | None = None):
super().__init__(message)
self.status_code = status_code
self.retry_after = retry_after
async def _request_xai(
credential: XaiOAuthCredential,
body: dict[str, Any],
on_content_delta: Callable[[str], Awaitable[None]] | None = None,
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
) -> tuple[str, list[ToolCallRequest], str]:
url = credential.api_base.rstrip("/") + "/responses"
headers = {
"Authorization": f"Bearer {credential.access_token}",
"Accept": "text/event-stream",
"Content-Type": "application/json",
"User-Agent": "nanobot (python)",
}
timeout = httpx.Timeout(120.0, connect=20.0)
async with httpx.AsyncClient(timeout=timeout, follow_redirects=True, trust_env=True) as client:
async with client.stream("POST", url, headers=headers, json=body) as response:
if response.status_code != 200:
raw = await response.aread()
retry_after = LLMProvider._extract_retry_after_from_headers(response.headers)
raise _XaiHTTPError(
_friendly_error(response.status_code, raw.decode("utf-8", "ignore")),
status_code=response.status_code,
retry_after=retry_after,
)
return await consume_sse(response, on_content_delta, on_tool_call_delta)
def _friendly_error(status_code: int, raw: str) -> str:
if status_code == 401:
return "xAI OAuth session expired or was revoked. Run: nanobot provider login xai-oauth"
if status_code == 403:
return (
"xAI accepted the OAuth token, but this account is not entitled for the requested "
"Grok API capability yet. Check the active Grok subscription and selected model."
)
if status_code == 429:
return "xAI Grok subscription quota or rate limit was reached. Please try again later."
return f"HTTP {status_code}: {raw[:500]}"
+50
View File
@@ -27,6 +27,8 @@ _MESSAGE_TIME_PREFIX_RE = re.compile(r"^\[Message Time: [^\]]+\]\n?")
_LOCAL_IMAGE_BREADCRUMB_RE = re.compile(r"^\[image: (?:/|~)[^\]]+\]\s*$")
_TOOL_CALL_ECHO_RE = re.compile(r'^\s*(?:generate_image|message)\([^)]*\)\s*$')
_SESSION_PREVIEW_MAX_CHARS = 120
_SESSION_LIST_PREVIEW_MAX_RECORDS = 200
_SESSION_LIST_PREVIEW_MAX_CHARS = 1_000_000
def _sanitize_assistant_replay_text(content: str) -> str:
@@ -165,6 +167,45 @@ class Session:
image_placeholder_text(p) for p in media if isinstance(p, str) and p
)
content = f"{content}\n{breadcrumbs}" if content else breadcrumbs
cli_apps = message.get("cli_apps")
if role == "user" and isinstance(cli_apps, list) and cli_apps and isinstance(content, str):
cli_lines: list[str] = []
for item in cli_apps[:8]:
if not isinstance(item, dict):
continue
name = str(item.get("name") or "").strip().lower()
if not name:
continue
entry = str(item.get("entry_point") or "unknown").strip() or "unknown"
cli_lines.append(
f"[CLI App Attachment: @{name}; tool=run_cli_app; entry_point={entry}; "
f"skill=skills/cli-app-{name}/SKILL.md]"
)
if cli_lines:
breadcrumbs = "\n".join(cli_lines)
content = f"{content}\n{breadcrumbs}" if content else breadcrumbs
mcp_presets = message.get("mcp_presets")
if (
role == "user"
and isinstance(mcp_presets, list)
and mcp_presets
and isinstance(content, str)
):
mcp_lines: list[str] = []
for item in mcp_presets[:8]:
if not isinstance(item, dict):
continue
name = str(item.get("name") or "").strip().lower()
if not name:
continue
transport = str(item.get("transport") or "mcp").strip() or "mcp"
mcp_lines.append(
f"[MCP Preset Attachment: @{name}; tool_prefix=mcp_{name}_; "
f"transport={transport}]"
)
if mcp_lines:
breadcrumbs = "\n".join(mcp_lines)
content = f"{content}\n{breadcrumbs}" if content else breadcrumbs
if include_timestamps:
content = self._annotate_message_time(message, content)
if role == "assistant" and isinstance(content, str) and not content.strip():
@@ -604,9 +645,18 @@ class SessionManager:
title = metadata.get("title") if isinstance(metadata, dict) else None
preview = ""
fallback_preview = ""
scanned_records = 0
scanned_chars = 0
for line in f:
if not line.strip():
continue
scanned_records += 1
scanned_chars += len(line)
if (
scanned_records > _SESSION_LIST_PREVIEW_MAX_RECORDS
or scanned_chars > _SESSION_LIST_PREVIEW_MAX_CHARS
):
break
item = json.loads(line)
if item.get("_type") == "metadata":
continue
+2
View File
@@ -34,3 +34,5 @@ Examples (replace `keyword`):
- **Do NOT edit SOUL.md, USER.md, or MEMORY.md.** They are automatically managed by Dream.
- If you notice outdated information, it will be corrected when Dream runs next.
- Users can view Dream's activity with the `/dream-log` command.
- Dream runs as a `system` session inside the AgentLoop, triggered by the `/dream` command or cron. Each turn processes one batch; if backlog remains, Dream automatically chains additional turns until complete. All changes are committed in a single git commit.
- Dream can use a different model than the main agent via `agents.defaults.dream.modelOverride`. Supports preset names or raw model identifiers.
+8 -4
View File
@@ -1,5 +1,9 @@
# Agent Instructions
## Workspace Guidance
Use this file for project-specific preferences, recurring workflow conventions, and instructions you want the agent to remember for this workspace. Keep durable facts about the user in `USER.md`, personality/style guidance in `SOUL.md`, and long-term memory in `memory/MEMORY.md`.
## Scheduled Reminders
Before scheduling reminders, check available skills and follow skill guidance first.
@@ -10,10 +14,10 @@ Get USER_ID and CHANNEL from the current session (e.g., `8281248569` and `telegr
## Heartbeat Tasks
`HEARTBEAT.md` is checked on the configured heartbeat interval. Use file tools to manage periodic tasks:
`HEARTBEAT.md` is checked on the configured heartbeat interval. Use file tools to manage periodic tasks.
- **Add**: `edit_file` to append new tasks
- **Remove**: `edit_file` to delete completed tasks
- **Rewrite**: `write_file` to replace all tasks
- Use `apply_patch` for normal task-list updates, especially when adding, removing, or changing multiple lines.
- Use `edit_file` only for small exact replacements copied from the current `HEARTBEAT.md`.
- Use `write_file` for first creation or intentional full-file rewrites.
When the user asks for a recurring/periodic task, update `HEARTBEAT.md` instead of creating a one-time cron reminder.
-28
View File
@@ -1,28 +0,0 @@
# Tool Usage Notes
Tool signatures are provided automatically via function calling.
This file documents non-obvious constraints and usage patterns.
## exec — Safety Limits
- Commands have a configurable timeout (default 60s)
- Dangerous commands are blocked (rm -rf, format, dd, shutdown, etc.)
- Output is truncated at 10,000 characters
- `restrictToWorkspace` config can limit file access to the workspace
## grep — Content Search
- Use `grep` to search file contents inside the workspace
- Default behavior returns only matching file paths (`output_mode="files_with_matches"`)
- Supports optional `glob` filtering (e.g. `glob="*.py"`) plus `context_before` / `context_after`
- Supports `type="py"`, `type="ts"`, `type="md"` and similar shorthand filters
- Use `fixed_strings=true` for literal keywords containing regex characters
- Use `output_mode="files_with_matches"` to get only matching file paths
- Use `output_mode="count"` to size a search before reading full matches
- Use `head_limit` and `offset` to page across results
- Prefer this over `exec` for code and history searches
- Binary or oversized files may be skipped to keep results readable
## cron — Scheduled Reminders
- Please refer to cron skill for usage.
@@ -1,13 +1,27 @@
Extract key facts from this conversation. Only output items matching these categories, skip everything else:
- User facts: personal info, preferences, stated opinions, habits
- Decisions: choices made, conclusions reached
- Solutions: working approaches discovered through trial and error, especially non-obvious methods that succeeded after failed attempts
- Events: plans, deadlines, notable occurrences
- Preferences: communication style, tool preferences
Extract key facts from this conversation. For each fact, annotate its memory attributes.
Priority: user corrections and preferences > solutions > decisions > events > environment facts. The most valuable memory prevents the user from having to repeat themselves.
Only SNIP facts deserve a non-[skip] mark:
- Signal: would the user need to repeat this if forgotten?
- Novel: not already in MEMORY.md or USER.md (check context below)
- Important: prevents rework or captures preferences / rules
- Persistent: still relevant after 2 weeks
Skip: code patterns derivable from source, git history, or anything already captured in existing memory.
Output one fact per line in this format:
- [mark] fact content
Marks (choose the best match):
- [permanent] Core preferences, personal traits, habits — never becomes stale
- [durable] Technical discoveries, project knowledge, config details — valid for months
- [ephemeral] Active task state, temporary decisions — may change in weeks
- [correction] Correction to a previous memory — must state what it replaces
- [skip] Does not meet SNIP criteria — still written to history.jsonl for audit, but Dream will ignore it
Categories to capture: people/roles, decisions/rationale, solutions, events/dates, preferences.
Decisions must include their motivation.
Write densely. Prefer 'X=A, Y=B' over separate bullets for tightly coupled facts.
Priority: user corrections > decisions with rationale > solutions > specific events > general context.
Output in the same language as the input conversation.
CRITICAL: Never drop person names, team names, or project names.
Skip: code patterns derivable from source, git history, or anything already in existing memory.
Output as concise bullet points, one fact per line. No preamble, no commentary.
If nothing noteworthy happened, output: (nothing)
+65
View File
@@ -0,0 +1,65 @@
Update memory files by analyzing conversation history and editing files directly.
Prune before adding — removing stale content is as important as adding new facts.
## File routing
Do NOT guess paths. Route each fact to its canonical file:
| File | Full path | Content |
|------|------|---------|
| SOUL.md | `{{ soul_path }}` | Agent behavior, guardrails, tone, interaction patterns |
| USER.md | `{{ user_path }}` | Personal info, preferences, habits, work context, communication style |
| MEMORY.md | `{{ memory_path }}` | Technical knowledge, project context, infrastructure, accounts |
| SKILL.md | `skills/<name>/SKILL.md` | Reusable workflow templates ([SKILL] entries only) |
Cross-boundary rule: no technical configs in USER.md, no user facts in SOUL.md, no preferences in MEMORY.md. If a fact fits multiple files, keep the most specific copy and remove the rest.
## Delete-or-keep
**Always delete:**
- Same fact at multiple locations — keep canonical copy only
- Merged/closed PR notes, resolved incidents, superseded info
- Verbose entries restatable in fewer words
- Overlapping or nested sections covering the same topic
**Likely delete** (apply judgment):
- Same fact at different detail levels — keep most complete version only
- Debugging steps unlikely to recur
- Ephemeral facts past their useful life
- Tool/service details documented upstream
- Lines with ``← Nd`` where N>{{ stale_threshold_days }} — closer review, not automatic removal
**Never delete:**
- User preferences and personality traits (permanent regardless of age)
- Active project context still referenced in conversations
- Behavioral rules in SOUL.md
When removing: prefer deleting individual items over entire sections.
## Fact extraction
- Atomic facts: "has a cat named Luna" not "discussed pet care"
- Corrections: edit the existing entry, don't append a new one
- Capture confirmed approaches the user validated
## Skill discovery & creation
Flag [SKILL] only when ALL are true: repeatable workflow appeared 2+ times, involves clear steps (not vague preferences), substantial enough for its own instruction set. Check existing skills to avoid redundancy.
For [SKILL] entries:
- Use write_file to create skills/<name>/SKILL.md; read_file `{{ skill_creator_path }}` for format reference
- YAML frontmatter must include name, description, **and `dream_managed: true`** (marks this skill as Dream-created)
- Under 2000 words: when to use, steps, output format, example
- Do NOT overwrite existing skills — if overlapping, merge delta into the existing skill
- Skills are instruction sets, not code. Keep concrete values in MEMORY.md; skills use placeholders
## Skill edit policy
Each skill in the Existing Skills list is tagged with an origin:
- **[dream]** — Dream-created (has `dream_managed: true` in frontmatter). You MAY edit these.
- **[user]** — User-created workspace skill. {% if dream_edit_user_skills %}You MAY edit these.{% else %}You MUST NOT modify, rename, or delete these — you can only read them for context.{% endif %}
- **[builtin]** — Bundled with nanobot. You MUST NEVER modify these.
## Editing
- Default tool: apply_patch. Use edit_file only for small exact replacements.
- File contents provided below — no read_file needed for initial edits.
- Batch all changes into a single apply_patch call. Surgical edits only.
- dry_run=true to preview. If nothing to update, stop without calling tools.
Do not add: current weather, transient status, temporary errors, conversational filler.
-40
View File
@@ -1,40 +0,0 @@
You have TWO equally important tasks:
1. Extract new facts from conversation history
2. Deduplicate existing memory files — find and flag redundant, overlapping, or stale content even if NOT mentioned in history
Output one line per finding:
[FILE] atomic fact (not already in memory)
[FILE-REMOVE] reason for removal
[SKILL] kebab-case-name: one-line description of the reusable pattern
Files: USER (identity, preferences), SOUL (bot behavior, tone), MEMORY (knowledge, project context)
Rules:
- Atomic facts: "has a cat named Luna" not "discussed pet care"
- Corrections: [USER] location is Tokyo, not Osaka
- Capture confirmed approaches the user validated
Deduplication — scan ALL memory files for these redundancy patterns:
- Same fact stated in multiple places (e.g., "communicates in Chinese" in both USER.md and multiple MEMORY.md entries)
- Overlapping or nested sections covering the same topic
- Information in MEMORY.md that is already captured in USER.md or SOUL.md (MEMORY.md should not duplicate permanent-file content)
- Verbose entries that can be condensed without losing information
For each duplicate found, output [FILE-REMOVE] for the less authoritative copy (prefer keeping facts in their canonical location)
Staleness — MEMORY.md lines may have a ``← Nd`` suffix showing days since last modification:
- SOUL.md and USER.md have no age annotations — they are permanent, only update with corrections
- Age only indicates when content was last touched, not whether it should be removed
- Use content judgment: user habits/preferences/personality traits are permanent regardless of age
- Only prune content that is objectively outdated: passed events, resolved tracking, superseded approaches
- Lines with ``← Nd`` (N>{{ stale_threshold_days }}) deserve closer review but are NOT automatically removable
- When removing: prefer deleting individual items over entire sections
Skill discovery — flag [SKILL] when ALL of these are true:
- A specific, repeatable workflow appeared 2+ times in the conversation history
- It involves clear steps (not vague preferences like "likes concise answers")
- It is substantial enough to warrant its own instruction set (not trivial like "read a file")
- Do not worry about duplicates — the next phase will check against existing skills
Do not add: current weather, transient status, temporary errors, conversational filler.
[SKIP] if nothing needs updating.
-37
View File
@@ -1,37 +0,0 @@
Update memory files based on the analysis below.
- [FILE] entries: add the described content to the appropriate file
- [FILE-REMOVE] entries: delete the corresponding content from memory files
- [SKILL] entries: create a new skill under skills/<name>/SKILL.md using write_file
## File paths (relative to workspace root)
- SOUL.md
- USER.md
- memory/MEMORY.md
- skills/<name>/SKILL.md (for [SKILL] entries only)
Do NOT guess paths.
## Editing rules
- Edit directly — file contents provided below, no read_file needed
- Use exact text as old_text, include surrounding blank lines for unique match
- Batch changes to the same file into one edit_file call
- For deletions: section header + all bullets as old_text, new_text empty
- Surgical edits only — never rewrite entire files
- If nothing to update, stop without calling tools
## Skill creation rules (for [SKILL] entries)
- Use write_file to create skills/<name>/SKILL.md
- Before writing, read_file `{{ skill_creator_path }}` for format reference (frontmatter structure, naming conventions, quality standards)
- **Dedup check**: read existing skills listed below to verify the new skill is not functionally redundant. Skip creation if an existing skill already covers the same workflow.
- Include YAML frontmatter with name and description fields
- Keep SKILL.md under 2000 words — concise and actionable
- Include: when to use, steps, output format, at least one example
- Do NOT overwrite existing skills — skip if the skill directory already exists
- Reference specific tools the agent has access to (read_file, write_file, exec, web_search, etc.)
- Skills are instruction sets, not code — do not include implementation code
## Quality
- Every line must carry standalone value
- Concise bullets under clear headers
- When reducing (not deleting): keep essential facts, drop verbose details
- If uncertain whether to delete, keep but add "(verify currency)"
+67
View File
@@ -0,0 +1,67 @@
# Tool Usage Notes
Tool signatures are provided automatically via function calling. This section
documents the general tool contract and non-obvious usage patterns.
## General Tool Contract
- Use the narrowest structured tool that directly matches the task.
- Use read-only discovery before writes when state is uncertain.
- Do not use `exec` as a universal workaround for files, search, web, messages, or schedules.
- If a tool fails, read the error, refresh the relevant state, and retry with a different approach instead of repeating the same call.
- After meaningful changes, verify with the smallest reliable check: re-read changed state, run targeted tests, or inspect command output.
- Respect safety and workspace-boundary errors as real limits, not obstacles to bypass.
## Discovery and Reading
- Use `find_files` or `list_dir` to locate workspace paths before `read_file` when a path is uncertain.
- Use `grep` for content search inside the workspace; prefer it over shell grep for ordinary searches.
- `grep` defaults to `output_mode="files_with_matches"`; use `output_mode="content"` for matching lines with context.
- Use `fixed_strings=true` for literal keywords containing regex characters.
- Use `output_mode="count"` to size a broad search before reading full matches.
- Use `head_limit` and `offset` to page across large result sets.
- Binary or oversized files may be skipped to keep results readable.
## File and Coding Workflows
- For code or config changes, the default loop is: locate (`find_files`/`grep`), inspect (`read_file`), edit (`apply_patch`), then verify (`exec` or re-read).
- Use `apply_patch` as the default code editing tool, especially for multi-file changes, structural edits, generated code, moves, adds, or deletes.
- Use `apply_patch dry_run=true` when the patch is uncertain and you want validation plus a change summary before writing.
- Use `edit_file` only for small exact replacements in one file, with `old_text` copied from `read_file`; add `occurrence`, `line_hint`, or `expected_replacements` when ambiguity matters.
- Use `write_file` for new files or intentional full-file rewrites, not routine partial edits.
- If `apply_patch` or `edit_file` fails, re-read with `force=true`, narrow the context, and try a smaller patch rather than switching to shell `sed` or `echo`.
## Process Execution
- Use `exec` for tests, builds, package commands, git commands, and other process execution.
- Prefer dedicated file/search tools over `cat`, shell `find`, shell `grep`, `sed`, or `echo` for ordinary workspace inspection and edits.
- Use non-interactive flags such as `-y` or `--yes` when available.
- Commands have a configurable timeout (default 60s), dangerous commands are blocked, and output is truncated.
- For long-running or interactive commands, pass `yield_time_ms`; if the process keeps running, continue with `write_stdin`.
- Use `write_stdin` to poll, provide stdin, close stdin, wait for expected output with `wait_for`, or terminate an existing exec session.
- Use `list_exec_sessions` to recover active session IDs after context shifts.
## CLI App Attachments
- When Runtime Context lists a `CLI App Attachment` or `CLI App Mention`, treat the `@name` as an app capability the user intentionally attached to the current turn.
- If the task may need app-specific behavior, read the listed skill first, then call `run_cli_app` with that `name`.
- Do not run an attached CLI app through shell or generic process tools unless the user explicitly asks for that lower-level path.
- If the app CLI is missing, lacks local desktop/app/API prerequisites, or cannot complete the requested action, explain that concrete blocker and what was attempted.
## Web and External Information
- Use web tools when the user asks for current information, a specific URL, or information likely to have changed.
- Use `web_search` to find sources and `web_fetch` for a specific page or result that needs closer reading.
- Do not invent freshness-sensitive facts when tools can verify them.
## Messaging and Media
- Use `message` to send content or local media to the user/channel.
- `read_file` only reads content for your analysis; it does not deliver a file to the user.
- When sending an existing local file, attach it through the message/media mechanism instead of pasting file contents unless the user asked for text.
## Scheduling and Background Work
- Use `cron` for scheduled reminders or recurring jobs; do not run `nanobot cron` through `exec`.
- For heartbeat tasks, update `HEARTBEAT.md` according to the agent instructions.
- Do not write reminders only to memory files when the user expects an actual notification.
+269 -88
View File
@@ -3,15 +3,13 @@
from __future__ import annotations
import difflib
import json
import re
import time
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Awaitable, Callable
TRACKED_FILE_EDIT_TOOLS = frozenset({"write_file", "edit_file", "notebook_edit"})
TRACKED_FILE_EDIT_TOOLS = frozenset({"write_file", "edit_file", "apply_patch"})
_MAX_SNAPSHOT_BYTES = 2 * 1024 * 1024
_LIVE_EMIT_INTERVAL_S = 0.18
_LIVE_EMIT_LINE_STEP = 24
@@ -154,19 +152,108 @@ def prepare_file_edit_tracker(
workspace: Path | None,
params: dict[str, Any] | None,
) -> FileEditTracker | None:
trackers = prepare_file_edit_trackers(
call_id=call_id,
tool_name=tool_name,
tool=tool,
workspace=workspace,
params=params,
)
return trackers[0] if trackers else None
def prepare_file_edit_trackers(
*,
call_id: str,
tool_name: str,
tool: Any,
workspace: Path | None,
params: dict[str, Any] | None,
) -> list[FileEditTracker]:
if not is_file_edit_tool(tool_name):
return None
return []
paths = resolve_file_edit_paths(tool_name, tool, workspace, params)
trackers: list[FileEditTracker] = []
seen: set[Path] = set()
for path in paths:
try:
resolved = path.resolve()
except Exception:
resolved = path
if resolved in seen:
continue
seen.add(resolved)
before = read_file_snapshot(path)
trackers.append(FileEditTracker(
call_id=str(call_id or ""),
tool=tool_name,
path=path,
display_path=display_file_edit_path(path, workspace),
before=before,
))
return trackers
def resolve_file_edit_paths(
tool_name: str,
tool: Any,
workspace: Path | None,
params: dict[str, Any] | None,
) -> list[Path]:
if tool_name == "apply_patch":
return _resolve_apply_patch_paths(tool, workspace, params)
path = resolve_file_edit_path(tool, workspace, params)
if path is None:
return None
before = read_file_snapshot(path)
return FileEditTracker(
call_id=str(call_id or ""),
tool=tool_name,
path=path,
display_path=display_file_edit_path(path, workspace),
before=before,
)
return []
return [path]
def _resolve_apply_patch_paths(
tool: Any,
workspace: Path | None,
params: dict[str, Any] | None,
) -> list[Path]:
if not isinstance(params, dict):
return []
edits = params.get("edits")
if not isinstance(edits, list) or not edits:
return []
if params.get("dry_run") is True:
return []
resolved: list[Path] = []
seen: set[Path] = set()
for edit in edits:
if not isinstance(edit, dict):
continue
raw_path = edit.get("path")
if not isinstance(raw_path, str) or not raw_path.strip():
continue
path = _resolve_raw_file_edit_path(tool, workspace, raw_path)
if path is not None and path not in seen:
seen.add(path)
resolved.append(path)
return resolved
def _resolve_raw_file_edit_path(
tool: Any,
workspace: Path | None,
raw_path: str,
) -> Path | None:
resolver = getattr(tool, "_resolve", None)
if callable(resolver):
try:
resolved = resolver(raw_path)
if isinstance(resolved, Path):
return resolved
if resolved:
return Path(resolved)
except Exception:
return None
if workspace is None:
return Path(raw_path).expanduser().resolve()
return (workspace / raw_path).expanduser().resolve()
def build_file_edit_start_event(
@@ -304,6 +391,9 @@ class StreamingFileEditTracker:
self._states[key] = state
state.apply_delta(payload)
if state.name == "apply_patch":
await self._update_apply_patch(state)
return
if state.name not in {"write_file", "edit_file"}:
return
if state.path is None:
@@ -343,10 +433,80 @@ class StreamingFileEditTracker:
deleted=deleted,
)])
async def _update_apply_patch(self, state: _StreamingFileEditState) -> None:
if _json_bool_true(state.arguments, "dry_run"):
return
tool = self._tools.get("apply_patch") if hasattr(self._tools, "get") else None
events: list[dict[str, Any]] = []
now = time.monotonic()
path_matches = list(re.finditer(r'"path"\s*:\s*"([^"]+)"', state.arguments))
if not path_matches:
return
for i, m in enumerate(path_matches):
raw_path = m.group(1)
path = _resolve_raw_file_edit_path(tool, self._workspace, raw_path)
if path is None:
continue
segment_start = m.start()
segment_end = path_matches[i + 1].start() if i + 1 < len(path_matches) else len(state.arguments)
segment = state.arguments[segment_start:segment_end]
action_match = re.search(r'"action"\s*:\s*"(replace|add|delete)"', segment)
action = action_match.group(1) if action_match else "replace"
old_text = _extract_json_string_prefix(segment, "old_text") or ""
new_text = _extract_json_string_prefix(segment, "new_text") or ""
added = _text_line_count(new_text) if action in ("replace", "add") else 0
deleted = _text_line_count(old_text) if action in ("replace", "delete") else 0
delete_file = action == "delete"
file_state = state.patch_files.get(raw_path)
if file_state is None:
tracker = FileEditTracker(
call_id=state.call_id or state.key,
tool="apply_patch",
path=path,
display_path=display_file_edit_path(path, self._workspace),
before=read_file_snapshot(path),
)
file_state = _StreamingPatchFileState(tracker=tracker)
state.patch_files[raw_path] = file_state
if delete_file and added == 0 and deleted == 0 and file_state.tracker.before.countable:
deleted = _text_line_count(file_state.tracker.before.text or "")
if not file_state.should_emit(added, deleted, now):
continue
file_state.mark_emitted(added, deleted, now)
events.append(build_file_edit_live_event(
file_state.tracker,
added=added,
deleted=deleted,
))
if events:
await self._emit(events)
async def flush(self) -> None:
events: list[dict[str, Any]] = []
now = time.monotonic()
for state in self._states.values():
for file_state in state.patch_files.values():
added, deleted = file_state.last_added, file_state.last_deleted
if not file_state.emitted_once:
continue
if (
file_state.last_emitted_added == added
and file_state.last_emitted_deleted == deleted
):
continue
file_state.mark_emitted(added, deleted, now)
events.append(build_file_edit_live_event(
file_state.tracker,
added=added,
deleted=deleted,
))
if state.tracker is None:
continue
added, deleted = state.live_diff_counts()
@@ -367,12 +527,14 @@ class StreamingFileEditTracker:
def apply_final_call_ids(self, final_tool_calls: list[Any]) -> None:
"""Keep final start/end events keyed to any earlier streamed placeholder."""
used_canonicals: set[str] = set()
for tool_call in final_tool_calls:
canonical = self.canonical_call_id_for(tool_call)
if canonical:
if canonical and canonical not in used_canonicals:
try:
tool_call.id = canonical
except Exception:
used_canonicals.add(canonical)
except (AttributeError, TypeError):
pass
def canonical_call_id_for(self, tool_call: Any) -> str | None:
@@ -389,6 +551,10 @@ class StreamingFileEditTracker:
"""Mark streamed edits as failed when no final tool call will run."""
events: list[dict[str, Any]] = []
for state in self._states.values():
for file_state in state.patch_files.values():
if any(state.matches_final_tool_call(tool_call) for tool_call in final_tool_calls):
continue
events.append(build_file_edit_error_event(file_state.tracker, error))
if state.tracker is None:
continue
if any(state.matches_final_tool_call(tool_call) for tool_call in final_tool_calls):
@@ -492,6 +658,39 @@ class _StreamingJsonStringField:
self.last_char_cr = False
@dataclass(slots=True)
class _StreamingPatchFileState:
tracker: FileEditTracker
emitted_once: bool = False
last_emitted_added: int = -1
last_emitted_deleted: int = -1
last_emit_at: float = 0.0
last_added: int = 0
last_deleted: int = 0
def should_emit(self, added: int, deleted: int, now: float) -> bool:
self.last_added = added
self.last_deleted = deleted
if not self.emitted_once:
return True
if added == self.last_emitted_added and deleted == self.last_emitted_deleted:
return False
if max(
abs(added - self.last_emitted_added),
abs(deleted - self.last_emitted_deleted),
) >= _LIVE_EMIT_LINE_STEP:
return True
return now - self.last_emit_at >= _LIVE_EMIT_INTERVAL_S
def mark_emitted(self, added: int, deleted: int, now: float) -> None:
self.emitted_once = True
self.last_added = added
self.last_deleted = deleted
self.last_emitted_added = added
self.last_emitted_deleted = deleted
self.last_emit_at = now
@dataclass(slots=True)
class _StreamingFileEditState:
key: str
@@ -509,6 +708,7 @@ class _StreamingFileEditState:
new_text: _StreamingJsonStringField = field(
default_factory=lambda: _StreamingJsonStringField("new_text")
)
patch_files: dict[str, _StreamingPatchFileState] = field(default_factory=dict)
emitted_once: bool = False
last_emitted_added: int = -1
last_emitted_deleted: int = -1
@@ -531,6 +731,7 @@ class _StreamingFileEditState:
self.content.reset()
self.old_text.reset()
self.new_text.reset()
self.patch_files.clear()
return
delta = payload.get("arguments_delta")
if isinstance(delta, str) and delta:
@@ -590,6 +791,14 @@ class _StreamingFileEditState:
name = getattr(tool_call, "name", None)
if name != self.name:
return False
if self.name == "apply_patch":
arguments = getattr(tool_call, "arguments", None)
if not isinstance(arguments, dict):
return False
edits = arguments.get("edits")
if not isinstance(edits, list):
return False
return '"edits"' in self.arguments
arguments = getattr(tool_call, "arguments", None)
if not isinstance(arguments, dict):
return False
@@ -612,6 +821,51 @@ def _stream_key(payload: dict[str, Any]) -> str:
return ""
def _json_bool_true(source: str, key: str) -> bool:
return re.search(rf'"{re.escape(key)}"\s*:\s*true\b', source) is not None
def _extract_json_string_prefix(source: str, key: str) -> str | None:
match = re.search(rf'"{re.escape(key)}"\s*:\s*"', source)
if match is None:
return None
out: list[str] = []
i = match.end()
escape = False
while i < len(source):
ch = source[i]
if escape:
escape = False
if ch == "n":
out.append("\n")
elif ch == "r":
out.append("\r")
elif ch == "t":
out.append("\t")
elif ch == "u":
digits = source[i + 1:i + 5]
if len(digits) < 4:
break
try:
out.append(chr(int(digits, 16)))
except ValueError:
break
i += 4
else:
out.append(ch)
i += 1
continue
if ch == "\\":
escape = True
i += 1
continue
if ch == '"':
return "".join(out)
out.append(ch)
i += 1
return "".join(out)
def _extract_complete_json_string(source: str, key: str) -> str | None:
match = re.search(rf'"{re.escape(key)}"\s*:\s*"', source)
if match is None:
@@ -704,77 +958,4 @@ def _predict_after_text(
return before_text.replace(old_text, new_text)
return before_text.replace(old_text, new_text, 1)
return None
if tool_name == "notebook_edit":
return _predict_notebook_after_text(params, before_text)
return None
def _predict_notebook_after_text(params: dict[str, Any], before_text: str) -> str | None:
try:
nb = json.loads(before_text) if before_text.strip() else _empty_notebook()
except Exception:
return None
cells = nb.get("cells")
if not isinstance(cells, list):
return None
try:
cell_index = int(params.get("cell_index", 0))
except (TypeError, ValueError):
return None
new_source = params.get("new_source")
source = new_source if isinstance(new_source, str) else ""
cell_type = (
params.get("cell_type") if params.get("cell_type") in ("code", "markdown") else "code"
)
mode = (
params.get("edit_mode")
if params.get("edit_mode") in ("replace", "insert", "delete")
else "replace"
)
if mode == "delete":
if 0 <= cell_index < len(cells):
cells.pop(cell_index)
else:
return None
elif mode == "insert":
insert_at = min(max(cell_index + 1, 0), len(cells))
cells.insert(insert_at, _new_notebook_cell(source, str(cell_type)))
else:
if not (0 <= cell_index < len(cells)):
return None
cell = cells[cell_index]
if not isinstance(cell, dict):
return None
cell["source"] = source
cell["cell_type"] = cell_type
if cell_type == "code":
cell.setdefault("outputs", [])
cell.setdefault("execution_count", None)
else:
cell.pop("outputs", None)
cell.pop("execution_count", None)
nb["cells"] = cells
try:
return json.dumps(nb, indent=1, ensure_ascii=False)
except Exception:
return None
def _empty_notebook() -> dict[str, Any]:
return {
"nbformat": 4,
"nbformat_minor": 5,
"metadata": {
"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"},
"language_info": {"name": "python"},
},
"cells": [],
}
def _new_notebook_cell(source: str, cell_type: str) -> dict[str, Any]:
cell: dict[str, Any] = {"cell_type": cell_type, "source": source, "metadata": {}}
if cell_type == "code":
cell["outputs"] = []
cell["execution_count"] = None
return cell
+2 -1
View File
@@ -19,7 +19,8 @@ class CommitInfo:
def format(self, diff: str = "") -> str:
"""Format this commit for display, optionally with a diff."""
header = f"## {self.message.splitlines()[0]}\n`{self.sha}` — {self.timestamp}\n"
summary = self.message.splitlines()[0] if self.message else "(no message)"
header = f"## {summary}\n`{self.sha}` — {self.timestamp}\n"
if diff:
return f"{header}\n```diff\n{diff}\n```"
return f"{header}\n(no file changes)"
+3 -2
View File
@@ -576,7 +576,7 @@ def build_status_content(
def sync_workspace_templates(workspace: Path, silent: bool = False) -> list[str]:
"""Sync bundled templates to workspace. Only creates missing files."""
"""Sync bundled templates to workspace. Creates missing files without overwriting user files."""
from importlib.resources import files as pkg_files
try:
@@ -589,10 +589,11 @@ def sync_workspace_templates(workspace: Path, silent: bool = False) -> list[str]
added: list[str] = []
def _write(src, dest: Path):
content = src.read_text(encoding="utf-8") if src else ""
if dest.exists():
return
dest.parent.mkdir(parents=True, exist_ok=True)
dest.write_text(src.read_text(encoding="utf-8") if src else "", encoding="utf-8")
dest.write_text(content, encoding="utf-8")
added.append(str(dest.relative_to(workspace)))
for item in tpl.iterdir():
+10
View File
@@ -29,6 +29,11 @@ LENGTH_RECOVERY_PROMPT = (
"— no recap, no apology. Break remaining work into smaller steps if needed."
)
SUSTAINED_GOAL_CONTINUE_PROMPT = (
"You have an active sustained goal. Please continue working toward the "
"objective using your tools, or call complete_goal if the work is truly finished."
)
def empty_tool_result_message(tool_name: str) -> str:
"""Short prompt-safe marker for tools that completed without visible output."""
@@ -65,6 +70,11 @@ def build_length_recovery_message() -> dict[str, str]:
return {"role": "user", "content": LENGTH_RECOVERY_PROMPT}
def build_goal_continue_message(custom: str | None = None) -> dict[str, str]:
"""Prompt the model to continue when a sustained goal is still active."""
return {"role": "user", "content": custom or SUSTAINED_GOAL_CONTINUE_PROMPT}
def external_lookup_signature(tool_name: str, arguments: dict[str, Any]) -> str | None:
"""Stable signature for repeated external lookups we want to throttle."""
if tool_name == "web_fetch":
+4
View File
@@ -11,8 +11,10 @@ _TOOL_FORMATS: dict[str, tuple[list[str], str, bool, bool]] = {
"read_file": (["path", "file_path"], "read {}", True, False),
"write_file": (["path", "file_path"], "write {}", True, False),
"edit": (["file_path", "path"], "edit {}", True, False),
"find_files": (["query", "glob", "path"], "find {}", False, False),
"grep": (["pattern"], 'grep "{}"', False, False),
"exec": (["command"], "$ {}", False, True),
"list_exec_sessions": ([], "exec sessions", False, False),
"web_search": (["query"], 'search "{}"', False, False),
"web_fetch": (["url"], "fetch {}", True, False),
"list_dir": (["path"], "ls {}", True, False),
@@ -81,6 +83,8 @@ def _extract_arg(tc, key_args: list[str]) -> str | None:
def _fmt_known(tc, fmt: tuple, max_length: int = 40) -> str:
"""Format a registered tool using its template."""
if not fmt[0] and "{}" not in fmt[1]:
return fmt[1]
val = _extract_arg(tc, fmt[0])
if val is None:
return tc.name
+93
View File
@@ -0,0 +1,93 @@
"""CLI Apps helpers for the WebUI HTTP and message surfaces."""
from __future__ import annotations
import re
from typing import Any
from nanobot.apps.cli import CliAppError, CliAppManager, CliAppsRuntimeConfig
from nanobot.config.loader import load_config
QueryParams = dict[str, list[str]]
_CLI_APP_NAME_RE = re.compile(r"^[a-z0-9][a-z0-9_-]{0,63}$", re.IGNORECASE)
_CLI_APP_ATTACHMENT_KEYS = (
"name",
"display_name",
"category",
"entry_point",
"logo_url",
"brand_color",
)
def _clip_ws_string(value: Any, limit: int = 240) -> str | None:
if not isinstance(value, str):
return None
text = value.strip()
if not text:
return None
return text[:limit]
def normalize_cli_app_mentions(raw: Any) -> list[dict[str, str]]:
"""Sanitize structured CLI app mentions sent by the WebUI."""
if not isinstance(raw, list):
return []
out: list[dict[str, str]] = []
seen: set[str] = set()
for item in raw[:8]:
if not isinstance(item, dict):
continue
name = _clip_ws_string(item.get("name"), 64)
if not name or _CLI_APP_NAME_RE.match(name) is None:
continue
key = name.lower()
if key in seen:
continue
seen.add(key)
row: dict[str, str] = {"name": key}
for field in _CLI_APP_ATTACHMENT_KEYS[1:]:
value = _clip_ws_string(item.get(field), 512 if field == "logo_url" else 160)
if value:
row[field] = value
out.append(row)
return out
def _query_first(query: QueryParams, key: str) -> str | None:
values = query.get(key)
return values[0] if values else None
def _manager() -> CliAppManager:
config = load_config()
cli_cfg = config.tools.cli_apps
return CliAppManager(
workspace=config.workspace_path,
runtime=CliAppsRuntimeConfig(
install_timeout=cli_cfg.install_timeout,
run_timeout=cli_cfg.run_timeout,
catalog_ttl_seconds=cli_cfg.catalog_ttl_seconds,
),
)
def cli_apps_payload() -> dict[str, Any]:
return _manager().payload()
def cli_apps_action(action: str, query: QueryParams) -> dict[str, Any]:
name = (_query_first(query, "name") or "").strip()
if not name:
raise CliAppError("missing CLI app name")
manager = _manager()
if action == "install":
return manager.install(name)
if action == "update":
return manager.update(name)
if action == "uninstall":
return manager.uninstall(name)
if action == "test":
return manager.test(name)
raise CliAppError(f"unknown CLI app action '{action}'", status=404)
File diff suppressed because it is too large Load Diff
+5
View File
@@ -0,0 +1,5 @@
"""Compatibility exports for WebUI-attached MCP preset annotations."""
from nanobot.agent.tools.mcp import runtime_lines, session_extra
__all__ = ["runtime_lines", "session_extra"]
+93 -21
View File
@@ -6,10 +6,12 @@ settings payload shape and the allowlisted config mutations exposed to WebUI.
from __future__ import annotations
import re
from typing import Any
from zoneinfo import ZoneInfo
from nanobot.config.loader import get_config_path, load_config, save_config
from nanobot.config.schema import ModelPresetConfig
from nanobot.providers.image_generation import (
get_image_gen_provider,
image_gen_provider_names,
@@ -41,6 +43,7 @@ _IMAGE_GENERATION_ASPECT_RATIOS = {
"2:3",
"21:9",
}
_MODEL_CONFIGURATION_SLUG_RE = re.compile(r"[^a-z0-9_-]+")
class WebUISettingsError(ValueError):
@@ -73,12 +76,16 @@ def _mask_secret_hint(secret: str | None) -> str | None:
def _provider_requires_api_key(spec: Any) -> bool:
if spec.backend == "azure_openai":
return True
if spec.is_oauth:
return False
if spec.is_local or spec.is_direct:
return False
return True
def _provider_configured_for_settings(spec: Any, provider_config: Any) -> bool:
if spec.is_oauth:
return True
if _provider_requires_api_key(spec):
return bool(provider_config.api_key)
return bool(
@@ -96,6 +103,32 @@ def _parse_bool(value: str, field: str) -> bool:
return normalized in {"1", "true", "yes"}
def _model_configuration_slug(label: str) -> str:
normalized = _MODEL_CONFIGURATION_SLUG_RE.sub("-", label.strip().lower())
normalized = normalized.strip("-_")
if not normalized:
raise WebUISettingsError("configuration name is required")
if normalized == "default":
raise WebUISettingsError("configuration name is reserved")
if len(normalized) > 48:
normalized = normalized[:48].rstrip("-_")
return normalized
def _validate_configured_provider(config: Any, provider: str) -> None:
if provider == "auto":
return
spec = find_by_name(provider)
if spec is None:
raise WebUISettingsError("unknown provider")
provider_config = getattr(config.providers, provider, None)
if (
provider_config is None
or not _provider_configured_for_settings(spec, provider_config)
):
raise WebUISettingsError("provider is not configured")
def _image_generation_provider_rows(config: Any) -> list[dict[str, Any]]:
rows: list[dict[str, Any]] = []
for name in image_gen_provider_names():
@@ -148,17 +181,18 @@ def settings_payload(*, requires_restart: bool = False) -> dict[str, Any]:
provider_config = getattr(config.providers, spec.name, None)
if provider_config is None or spec.is_oauth:
continue
providers.append(
{
"name": spec.name,
"label": spec.label,
"configured": _provider_configured_for_settings(spec, provider_config),
"api_key_required": _provider_requires_api_key(spec),
"api_key_hint": _mask_secret_hint(provider_config.api_key),
"api_base": provider_config.api_base,
"default_api_base": spec.default_api_base or None,
}
)
row = {
"name": spec.name,
"label": spec.label,
"configured": _provider_configured_for_settings(spec, provider_config),
"api_key_required": _provider_requires_api_key(spec),
"api_key_hint": _mask_secret_hint(provider_config.api_key),
"api_base": provider_config.api_base,
"default_api_base": spec.default_api_base or None,
}
if spec.name == "openai":
row["api_type"] = provider_config.api_type
providers.append(row)
search_config = config.tools.web.search
image_config = config.tools.image_generation
@@ -194,7 +228,7 @@ def settings_payload(*, requires_restart: bool = False) -> dict[str, Any]:
model_presets.append(
{
"name": name,
"label": name,
"label": preset.label or name,
"active": active_preset_name == name,
"is_default": False,
"model": preset.model,
@@ -273,6 +307,7 @@ def settings_payload(*, requires_restart: bool = False) -> dict[str, Any]:
"max_batch_size": defaults.dream.max_batch_size,
"max_iterations": defaults.dream.max_iterations,
"annotate_line_ages": defaults.dream.annotate_line_ages,
"dream_edit_user_skills": defaults.dream.dream_edit_user_skills,
},
"unified_session": defaults.unified_session,
},
@@ -317,15 +352,7 @@ def update_agent_settings(query: QueryParams) -> dict[str, Any]:
provider = provider.strip()
if not provider:
raise WebUISettingsError("provider is required")
spec = find_by_name(provider)
if spec is None:
raise WebUISettingsError("unknown provider")
provider_config = getattr(config.providers, provider, None)
if (
provider_config is None
or not _provider_configured_for_settings(spec, provider_config)
):
raise WebUISettingsError("provider is not configured")
_validate_configured_provider(config, provider)
if defaults.provider != provider:
defaults.provider = provider
changed = True
@@ -384,6 +411,40 @@ def update_agent_settings(query: QueryParams) -> dict[str, Any]:
return settings_payload(requires_restart=restart_required)
def create_model_configuration(query: QueryParams) -> dict[str, Any]:
label = (_query_first_alias(query, "label", "displayName") or "").strip()
raw_name = (_query_first(query, "name") or label).strip()
model = (_query_first(query, "model") or "").strip()
provider = (_query_first(query, "provider") or "").strip()
if not label:
label = raw_name
if not model:
raise WebUISettingsError("model is required")
if not provider:
raise WebUISettingsError("provider is required")
name = _model_configuration_slug(raw_name or label)
config = load_config()
if name in config.model_presets:
raise WebUISettingsError("configuration already exists", status=409)
_validate_configured_provider(config, provider)
base = config.resolve_default_preset()
config.model_presets[name] = ModelPresetConfig(
label=label,
model=model,
provider=provider,
max_tokens=base.max_tokens,
context_window_tokens=base.context_window_tokens,
temperature=base.temperature,
reasoning_effort=base.reasoning_effort,
)
config.agents.defaults.model_preset = name
save_config(config)
return settings_payload()
def update_provider_settings(query: QueryParams) -> dict[str, Any]:
provider_name = (_query_first(query, "provider") or "").strip()
if not provider_name:
@@ -412,6 +473,17 @@ def update_provider_settings(query: QueryParams) -> dict[str, Any]:
provider_config.api_base = api_base
changed = True
if "api_type" in query:
if spec.name == "openai":
api_type = (_query_first(query, "api_type") or "").strip()
try:
parsed_api_type = type(provider_config)(api_type=api_type).api_type
except Exception:
raise WebUISettingsError("api_type must be auto, chat_completions, or responses") from None
if provider_config.api_type != parsed_api_type:
provider_config.api_type = parsed_api_type
changed = True
if changed:
save_config(config)
image_config = config.tools.image_generation
+154 -4
View File
@@ -4,10 +4,12 @@ from __future__ import annotations
import json
import os
import re
import time
import uuid
from pathlib import Path
from typing import Any, Callable
from typing import Any, Callable, Mapping
from urllib.parse import unquote, urlparse
from loguru import logger
@@ -16,6 +18,61 @@ from nanobot.session.manager import SessionManager
WEBUI_TRANSCRIPT_SCHEMA_VERSION = 3
_MAX_TRANSCRIPT_FILE_BYTES = 8 * 1024 * 1024
_MARKDOWN_LOCAL_IMAGE_RE = re.compile(
r"!\[([^\]]*)\]\((<[^>]+>|[^)\s]+)(\s+(?:\"[^\"]*\"|'[^']*'))?\)"
)
_INLINE_MARKDOWN_IMAGE_EXTS: frozenset[str] = frozenset({
".png",
".jpg",
".jpeg",
".webp",
".gif",
})
def rewrite_local_markdown_images(
text: str,
*,
workspace_path: Path,
sign_path: Callable[[Path], Mapping[str, Any] | None],
) -> str:
"""Rewrite markdown image paths inside the workspace to signed WebUI media URLs."""
if "![" not in text:
return text
def resolve_url(raw_url: str) -> str | None:
url = raw_url.strip()
if url.startswith("<") and url.endswith(">"):
url = url[1:-1].strip()
if not url or url.startswith(("/api/media/", "#")):
return None
parsed = urlparse(url)
if parsed.scheme or parsed.netloc or parsed.query or parsed.fragment:
return None
path_text = unquote(url)
if Path(path_text).suffix.lower() not in _INLINE_MARKDOWN_IMAGE_EXTS:
return None
candidate = Path(path_text).expanduser()
if not candidate.is_absolute():
candidate = workspace_path / candidate
try:
resolved = candidate.resolve(strict=False)
resolved.relative_to(workspace_path)
except (OSError, ValueError):
return None
if not resolved.is_file():
return None
signed = sign_path(resolved)
return str(signed.get("url")) if signed and signed.get("url") else None
def replace(match: re.Match[str]) -> str:
signed_url = resolve_url(match.group(2))
if not signed_url:
return match.group(0)
title = match.group(3) or ""
return f"![{match.group(1)}]({signed_url}{title})"
return _MARKDOWN_LOCAL_IMAGE_RE.sub(replace, text)
def webui_transcript_path(session_key: str) -> Path:
@@ -116,6 +173,55 @@ def tool_trace_lines_from_events(events: Any) -> list[str]:
return lines
_PHASE_RANK = {"start": 1, "end": 2, "error": 3}
def _normalize_tool_events(events: Any) -> list[dict[str, Any]]:
if not isinstance(events, list):
return []
out: list[dict[str, Any]] = []
for event in events:
if not event or not isinstance(event, dict):
continue
if event.get("phase") not in {"start", "end", "error"}:
continue
if not isinstance(event.get("name"), str):
fn = event.get("function")
if not (isinstance(fn, dict) and isinstance(fn.get("name"), str)):
continue
out.append(dict(event))
return out
def _tool_event_key(event: dict[str, Any]) -> str:
call_id = event.get("call_id")
if isinstance(call_id, str) and call_id:
return f"call:{call_id}"
return _format_tool_call_trace(event) or json.dumps(event, sort_keys=True, ensure_ascii=False)
def _merge_tool_events(previous: Any, incoming: list[dict[str, Any]]) -> list[dict[str, Any]]:
if not isinstance(previous, list) or not previous:
return incoming
if not incoming:
return [dict(event) for event in previous if isinstance(event, dict)]
merged = [dict(event) for event in previous if isinstance(event, dict)]
index_by_key = {_tool_event_key(event): idx for idx, event in enumerate(merged)}
for event in incoming:
key = _tool_event_key(event)
existing_index = index_by_key.get(key)
if existing_index is None:
index_by_key[key] = len(merged)
merged.append(event)
continue
existing = merged[existing_index]
incoming_rank = _PHASE_RANK.get(str(event.get("phase")), 0)
existing_rank = _PHASE_RANK.get(str(existing.get("phase")), 0)
if incoming_rank >= existing_rank:
merged[existing_index] = {**existing, **event}
return merged
def _merge_unique_tool_trace_lines(
previous_traces: list[str],
lines: list[str],
@@ -136,6 +242,7 @@ def replay_transcript_to_ui_messages(
lines: list[dict[str, Any]],
*,
augment_user_media: Callable[[list[str]], list[dict[str, Any]]] | None = None,
augment_assistant_text: Callable[[str], str] | None = None,
) -> list[dict[str, Any]]:
"""Fold JSONL records into ``UIMessage``-shaped dicts for the WebUI.
@@ -405,6 +512,14 @@ def replay_transcript_to_ui_messages(
row["media"] = media_att
if all(m.get("kind") == "image" for m in media_att):
row["images"] = [{"url": m.get("url"), "name": m.get("name")} for m in media_att]
cli_apps = rec.get("cli_apps")
if isinstance(cli_apps, list) and cli_apps:
row["cliApps"] = [dict(app) for app in cli_apps if isinstance(app, dict)]
mcp_presets = rec.get("mcp_presets")
if isinstance(mcp_presets, list) and mcp_presets:
row["mcpPresets"] = [
dict(preset) for preset in mcp_presets if isinstance(preset, dict)
]
messages.append(row)
continue
@@ -449,6 +564,24 @@ def replay_transcript_to_ui_messages(
buffer_message_id = None
buffer_parts = []
continue
final_text = rec.get("text")
if isinstance(final_text, str):
if buffer_message_id is None:
buffer_message_id = _new_id("buf", idx)
messages.append(
{
"id": buffer_message_id,
"role": "assistant",
"content": final_text,
"isStreaming": True,
"createdAt": _ts_base + idx,
},
)
else:
for i, m in enumerate(messages):
if m.get("id") == buffer_message_id:
messages[i] = {**m, "content": final_text, "isStreaming": True}
break
buffer_message_id = None
buffer_parts = []
continue
@@ -486,6 +619,7 @@ def replay_transcript_to_ui_messages(
close_reasoning(messages)
continue
if kind in ("tool_hint", "progress"):
structured_events = _normalize_tool_events(rec.get("tool_events"))
structured = tool_trace_lines_from_events(rec.get("tool_events"))
text = rec.get("text")
trace_lines = structured if structured else ([text] if isinstance(text, str) and text else [])
@@ -502,7 +636,7 @@ def replay_transcript_to_ui_messages(
prev_traces = list(last.get("traces") or [last.get("content")])
if structured:
merged_traces, added = _merge_unique_tool_trace_lines(prev_traces, structured)
if not added:
if not added and not structured_events:
continue
else:
merged_traces = prev_traces + trace_lines
@@ -510,6 +644,9 @@ def replay_transcript_to_ui_messages(
**last,
"traces": merged_traces,
"content": merged_traces[-1],
"toolEvents": _merge_tool_events(last.get("toolEvents"), structured_events)
if structured_events
else last.get("toolEvents"),
"activitySegmentId": last.get("activitySegmentId") or segment,
}
messages[-1] = merged
@@ -521,6 +658,7 @@ def replay_transcript_to_ui_messages(
"kind": "trace",
"content": trace_lines[-1],
"traces": trace_lines,
**({"toolEvents": structured_events} if structured_events else {}),
"activitySegmentId": segment,
"createdAt": _ts_base + idx,
},
@@ -569,7 +707,14 @@ def replay_transcript_to_ui_messages(
buffer_parts = []
continue
for m in messages:
for i, m in enumerate(messages):
if (
augment_assistant_text is not None
and m.get("role") == "assistant"
and m.get("kind") != "trace"
and isinstance(m.get("content"), str)
):
messages[i] = {**m, "content": augment_assistant_text(m["content"])}
m.pop("isStreaming", None)
m.pop("reasoningStreaming", None)
return messages
@@ -579,12 +724,17 @@ def build_webui_thread_response(
session_key: str,
*,
augment_user_media: Callable[[list[str]], list[dict[str, Any]]] | None = None,
augment_assistant_text: Callable[[str], str] | None = None,
) -> dict[str, Any] | None:
"""Return a payload compatible with ``WebuiThreadPersistedPayload``."""
lines = read_transcript_lines(session_key)
if not lines:
return None
msgs = replay_transcript_to_ui_messages(lines, augment_user_media=augment_user_media)
msgs = replay_transcript_to_ui_messages(
lines,
augment_user_media=augment_user_media,
augment_assistant_text=augment_assistant_text,
)
return {
"schemaVersion": WEBUI_TRANSCRIPT_SCHEMA_VERSION,
"sessionKey": session_key,
-1
View File
@@ -61,7 +61,6 @@ dependencies = [
"openpyxl>=3.1.0,<4.0.0",
"python-pptx>=1.0.0,<2.0.0",
"filelock>=3.25.2",
"keyring>=25.0.0,<26.0.0",
"boto3>=1.43.0",
]
+53
View File
@@ -139,6 +139,13 @@ class TestLoadBootstrapFiles:
for name in ContextBuilder.BOOTSTRAP_FILES:
assert f"## {name}" in result
def test_legacy_tools_md_is_not_bootstrapped(self, tmp_path):
(tmp_path / "TOOLS.md").write_text("workspace tool notes", encoding="utf-8")
builder = _builder(tmp_path)
result = builder._load_bootstrap_files()
assert "TOOLS.md" not in result
assert "workspace tool notes" not in result
def test_utf8_content(self, tmp_path):
(tmp_path / "AGENTS.md").write_text("用中文回复", encoding="utf-8")
builder = _builder(tmp_path)
@@ -171,6 +178,37 @@ class TestIsTemplateContent:
assert ContextBuilder._is_template_content("totally different", "memory/MEMORY.md") is False
# ---------------------------------------------------------------------------
# Bundled bootstrap templates
# ---------------------------------------------------------------------------
class TestBundledToolContract:
def test_tool_contract_balances_general_and_coding_workflows(self):
from importlib.resources import files as pkg_files
tpl = pkg_files("nanobot") / "templates" / "agent" / "tool_contract.md"
content = tpl.read_text(encoding="utf-8")
assert "## General Tool Contract" in content
assert "Use the narrowest structured tool" in content
assert "Do not use `exec` as a universal workaround" in content
assert "## File and Coding Workflows" in content
assert "apply_patch" in content
assert "## Web and External Information" in content
assert "## Messaging and Media" in content
assert "## Scheduling and Background Work" in content
assert "pure coding" not in content.lower()
def test_tool_contract_is_injected_without_workspace_file(self, tmp_path):
builder = _builder(tmp_path)
prompt = builder.build_system_prompt()
assert "# Tool Usage Notes" in prompt
assert "## General Tool Contract" in prompt
assert "Do not use `exec` as a universal workaround" in prompt
# ---------------------------------------------------------------------------
# _build_user_content
# ---------------------------------------------------------------------------
@@ -324,6 +362,21 @@ class TestBuildMessages:
assert "Other chat goal." not in str(without_goal[-1]["content"])
assert "Goal (active):" not in str(without_goal[-1]["content"])
def test_current_runtime_lines_are_injected(self, tmp_path):
builder = _builder(tmp_path)
messages = builder.build_messages(
[],
"please use @zoom tonight",
current_runtime_lines=[
"CLI App Attachment: @zoom (installed; tool=run_cli_app; entry_point=cli-anything-zoom).",
],
)
user_msg = str(messages[-1]["content"])
assert "CLI App Attachment: @zoom" in user_msg
assert "tool=run_cli_app" in user_msg
assert "entry_point=cli-anything-zoom" in user_msg
def test_consecutive_same_role_merged(self, tmp_path):
builder = _builder(tmp_path)
history = [{"role": "user", "content": "previous user message"}]
+397 -216
View File
@@ -1,19 +1,32 @@
"""Tests for the Dream class — two-phase memory consolidation via AgentRunner."""
"""Tests for Dream driven through AgentLoop._process_system_message."""
import json
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from unittest.mock import AsyncMock, MagicMock, patch
from nanobot.agent.memory import Dream, MemoryStore
from nanobot.agent.loop import AgentLoop
from nanobot.agent.runner import AgentRunResult
from nanobot.agent.skills import BUILTIN_SKILLS_DIR
from nanobot.bus.events import InboundMessage
from nanobot.bus.queue import MessageBus
from nanobot.utils.gitstore import LineAge
def _provider(default_model: str, max_tokens: int = 123) -> MagicMock:
provider = MagicMock()
provider.get_default_model.return_value = default_model
provider.generation = SimpleNamespace(
max_tokens=max_tokens, temperature=0.1, reasoning_effort=None
)
return provider
@pytest.fixture
def store(tmp_path):
from nanobot.agent.memory import MemoryStore
s = MemoryStore(tmp_path)
s.write_soul("# Soul\n- Helpful")
s.write_user("# User\n- Developer")
@@ -23,9 +36,7 @@ def store(tmp_path):
@pytest.fixture
def mock_provider():
p = MagicMock()
p.chat_with_retry = AsyncMock()
return p
return _provider("test-model")
@pytest.fixture
@@ -34,10 +45,16 @@ def mock_runner():
@pytest.fixture
def dream(store, mock_provider, mock_runner):
d = Dream(store=store, provider=mock_provider, model="test-model", max_batch_size=5)
d._runner = mock_runner
return d
def loop(tmp_path, mock_provider, mock_runner):
loop = AgentLoop(
bus=MessageBus(),
provider=mock_provider,
workspace=tmp_path,
model="test-model",
context_window_tokens=1000,
)
loop.dream._runner = mock_runner
return loop
def _make_run_result(
@@ -56,254 +73,418 @@ def _make_run_result(
)
class TestDreamRun:
async def test_noop_when_no_unprocessed_history(self, dream, mock_provider, mock_runner, store):
"""Dream should not call LLM when there's nothing to process."""
result = await dream.run()
assert result is False
mock_provider.chat_with_retry.assert_not_called()
class TestDreamAgentLoopIntegration:
async def test_completes_goal_state_after_full_backlog(self, loop, mock_runner, store):
"""Goal should be completed after processing all backlog in internal loop."""
for i in range(6):
store.append_history(f"event {i}")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
session = loop.sessions.get_or_create("system:dream")
goal = session.metadata.get("goal_state")
assert isinstance(goal, dict)
assert goal["status"] == "completed"
assert store.get_last_dream_cursor() == 6
async def test_completes_goal_state_on_finish(self, loop, mock_runner, store):
"""Goal should be marked completed when backlog is fully processed."""
store.append_history("event 1")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
session = loop.sessions.get_or_create("system:dream")
goal = session.metadata.get("goal_state")
assert goal["status"] == "completed"
assert "completed_at" in goal
assert "recap" in goal
async def test_noop_when_no_unprocessed_history(self, loop, mock_runner):
"""Dream should not call runner when there's nothing to process."""
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
result = await loop._process_system_message(msg)
assert result is None
mock_runner.run.assert_not_called()
async def test_calls_runner_for_unprocessed_entries(self, dream, mock_provider, mock_runner, store):
async def test_calls_runner_for_unprocessed_entries(self, loop, mock_runner, store):
"""Dream should call AgentRunner when there are unprocessed history entries."""
store.append_history("User prefers dark mode")
mock_provider.chat_with_retry.return_value = MagicMock(content="New fact")
mock_runner.run = AsyncMock(return_value=_make_run_result(
tool_events=[{"name": "edit_file", "status": "ok", "detail": "memory/MEMORY.md"}],
))
result = await dream.run()
assert result is True
mock_runner.run = AsyncMock(
return_value=_make_run_result(
tool_events=[
{"name": "edit_file", "status": "ok", "detail": "memory/MEMORY.md"}
],
)
)
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
mock_runner.run.assert_called_once()
spec = mock_runner.run.call_args[0][0]
assert spec.max_iterations == 10
assert spec.fail_on_tool_error is False
async def test_advances_dream_cursor(self, dream, mock_provider, mock_runner, store):
async def test_advances_dream_cursor(self, loop, mock_runner, store):
"""Dream should advance the cursor after processing."""
store.append_history("event 1")
store.append_history("event 2")
mock_provider.chat_with_retry.return_value = MagicMock(content="Nothing new")
mock_runner.run = AsyncMock(return_value=_make_run_result())
await dream.run()
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
assert store.get_last_dream_cursor() == 2
async def test_compacts_processed_history(self, dream, mock_provider, mock_runner, store):
async def test_compacts_processed_history(self, loop, mock_runner, store):
"""Dream should compact history after processing."""
store.append_history("event 1")
store.append_history("event 2")
store.append_history("event 3")
mock_provider.chat_with_retry.return_value = MagicMock(content="Nothing new")
mock_runner.run = AsyncMock(return_value=_make_run_result())
await dream.run()
# After Dream, cursor is advanced and 3, compact keeps last max_history_entries
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
entries = store.read_unprocessed_history(since_cursor=0)
assert all(e["cursor"] > 0 for e in entries)
async def test_skill_phase_uses_builtin_skill_creator_path(self, dream, mock_provider, mock_runner, store):
"""Dream should point skill creation guidance at the builtin skill-creator template."""
async def test_processes_full_backlog_in_one_call(self, loop, mock_runner, store):
"""Backlog larger than max_batch_size should be fully processed in one call."""
for i in range(12):
store.append_history(f"event {i}")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
assert store.get_last_dream_cursor() == 12
assert mock_runner.run.call_count == 3 # 5 + 5 + 2
async def test_single_git_commit_for_multi_batch(self, loop, mock_runner, store):
"""Multi-batch run should collapse into exactly one git commit."""
store.git.init()
store.git.auto_commit("initial")
for i in range(12):
store.append_history(f"event {i}")
mock_runner.run = AsyncMock(
return_value=_make_run_result(
tool_events=[
{"name": "edit_file", "status": "ok", "detail": "memory/MEMORY.md"}
],
)
)
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
commits = store.git.log()
dream_commits = [c for c in commits if c.message.startswith("dream:")]
assert len(dream_commits) == 1
async def test_system_prompt_cached(self, loop, mock_runner, store):
"""Batches within one run should reuse cached system prompt when template mtime unchanged."""
for i in range(6):
store.append_history(f"event {i}")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
# Two batches (5 + 1), both should use the same cached prompt
assert mock_runner.run.call_count == 2
first_prompt = mock_runner.run.call_args_list[0][0][0].initial_messages[0]["content"]
second_prompt = mock_runner.run.call_args_list[1][0][0].initial_messages[0]["content"]
assert second_prompt is first_prompt
async def test_noop_when_empty_backlog(self, loop, mock_runner, store):
"""Empty backlog should not advance cursor or create a commit."""
store.git.init()
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
assert store.get_last_dream_cursor() == 0
commits = store.git.log()
assert len([c for c in commits if c.message.startswith("dream:")]) == 0
class TestDreamPrompt:
async def test_prompt_contains_mece_rules(self, loop, mock_runner, store):
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
system_prompt = spec.initial_messages[0]["content"]
assert "Do NOT guess paths" in system_prompt
assert "SOUL.md" in system_prompt
assert "USER.md" in system_prompt
assert "MEMORY.md" in system_prompt
async def test_skill_phase_uses_builtin_skill_creator_path(self, loop, mock_runner, store):
store.append_history("Repeated workflow one")
store.append_history("Repeated workflow two")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKILL] test-skill: test description")
mock_runner.run = AsyncMock(return_value=_make_run_result())
await dream.run()
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
system_prompt = spec.initial_messages[0]["content"]
expected = str(BUILTIN_SKILLS_DIR / "skill-creator" / "SKILL.md")
assert expected in system_prompt
async def test_skill_write_tool_accepts_workspace_relative_skill_path(self, dream, store):
"""Dream skill creation should allow skills/<name>/SKILL.md relative to workspace root."""
write_tool = dream._tools.get("write_file")
assert write_tool is not None
result = await write_tool.execute(
path="skills/test-skill/SKILL.md",
content="---\nname: test-skill\ndescription: Test\n---\n",
async def test_system_prompt_uses_threshold_from_template_var(self, loop, mock_runner, store):
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
assert "Successfully wrote" in result
assert (store.workspace / "skills" / "test-skill" / "SKILL.md").exists()
async def test_phase1_prompt_includes_line_age_annotations(self, dream, mock_provider, mock_runner, store):
"""Phase 1 prompt should have per-line age suffixes in MEMORY.md when git is available."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
# Init git so line_ages works
store.git.init()
store.git.auto_commit("initial memory state")
await dream.run()
# The MEMORY.md section should not crash and should contain the memory content
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
assert "## Current MEMORY.md" in user_msg
async def test_phase1_annotates_only_memory_not_soul_or_user(self, dream, mock_provider, mock_runner, store):
"""SOUL.md and USER.md should never have age annotations — they are permanent."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
store.git.init()
store.git.auto_commit("initial state")
await dream.run()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
# The ← suffix should only appear in MEMORY.md section
memory_section = user_msg.split("## Current MEMORY.md")[1].split("## Current SOUL.md")[0]
soul_section = user_msg.split("## Current SOUL.md")[1].split("## Current USER.md")[0]
user_section = user_msg.split("## Current USER.md")[1]
# SOUL and USER should not contain age arrows
assert "\u2190" not in soul_section
assert "\u2190" not in user_section
async def test_phase1_prompt_works_without_git(self, dream, mock_provider, mock_runner, store):
"""Phase 1 should work fine even if git is not initialized (no age annotations)."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
await dream.run()
# Should still succeed — just without age annotations
mock_provider.chat_with_retry.assert_called_once()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
assert "## Current MEMORY.md" in user_msg
async def test_phase1_prompt_carries_age_suffix_for_stale_lines(
self, dream, mock_provider, mock_runner, store,
):
"""End-to-end: ages >14d must appear verbatim in the LLM prompt, ages ≤14d must not."""
# MEMORY.md fixture has 2 non-blank lines ("# Memory" and "- Project X active").
# Inject four ages to cover threshold boundaries: >14 suffix, ==14 no suffix, <14 no suffix.
store.write_memory("# Memory\n- Project X active\n- fresh item\n- edge case line")
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
fake_ages = [
LineAge(age_days=30), # "# Memory" → should get ← 30d
LineAge(age_days=20), # "- Project X..." → should get ← 20d
LineAge(age_days=14), # "- fresh item" → ==14, threshold is strictly >14, no suffix
LineAge(age_days=5), # "- edge case..." → no suffix
]
with patch.object(store.git, "line_ages", return_value=fake_ages):
await dream.run()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split("## Current SOUL.md")[0]
assert "\u2190 30d" in memory_section
assert "\u2190 20d" in memory_section
assert "\u2190 14d" not in memory_section
assert "\u2190 5d" not in memory_section
async def test_phase1_skips_annotation_when_disabled(
self, dream, mock_provider, mock_runner, store,
):
"""`annotate_line_ages=False` must bypass the git lookup entirely and keep MEMORY.md raw."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
dream.annotate_line_ages = False
# line_ages must be bypassed entirely — verify with a spy rather than a
# raising side_effect, because _annotate_with_ages catches Exception
# (which swallows AssertionError) and would hide an accidental call.
with patch.object(store.git, "line_ages") as mock_line_ages:
await dream.run()
mock_line_ages.assert_not_called()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
assert "\u2190" not in user_msg
async def test_phase1_skips_annotation_on_line_ages_length_mismatch(
self, dream, mock_provider, mock_runner, store,
):
"""If ages length != lines length (dirty working tree), skip annotation instead of mis-tagging."""
# MEMORY.md has 2 non-blank lines but we hand back only 1 age → mismatch.
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
with patch.object(store.git, "line_ages", return_value=[LineAge(age_days=999)]):
await dream.run()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split("## Current SOUL.md")[0]
# No age arrow at all — we refused to annotate rather than tag the wrong line.
assert "\u2190" not in memory_section
async def test_phase1_prompt_uses_threshold_from_template_var(
self, dream, mock_provider, mock_runner, store,
):
"""System prompt should reference the stale-threshold constant, not a hardcoded 14."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
await dream.run()
system_msg = mock_provider.chat_with_retry.call_args.kwargs["messages"][0]["content"]
# The template renders with stale_threshold_days=14 → LLM must see "N>14"
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
system_msg = spec.initial_messages[0]["content"]
assert "N>14" in system_msg
class TestDreamPromptCaps:
"""Dream's Phase 1/2 prompt must not be poisoned by a legacy oversized
history entry or a runaway MEMORY.md. Without caps, a single pre-#3412
raw_archive dump in history.jsonl would make every subsequent Dream run
exceed the context window and silently advance the cursor past real work.
"""
async def test_phase1_caps_huge_memory_file(
self, dream, mock_provider, mock_runner, store,
):
"""A MEMORY.md much larger than _MEMORY_FILE_MAX_CHARS must be truncated
in the prompt preview (full content is still reachable via read_file)."""
store.write_memory("M" * (dream._MEMORY_FILE_MAX_CHARS * 5))
async def test_caps_huge_memory_file(self, loop, mock_runner, store):
store.write_memory("M" * (loop.dream._MEMORY_FILE_MAX_CHARS * 5))
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split(
"## Current SOUL.md"
)[0]
assert len(memory_section) < loop.dream._MEMORY_FILE_MAX_CHARS + 500
await dream.run()
user_msg = mock_provider.chat_with_retry.call_args.kwargs["messages"][1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split("## Current SOUL.md")[0]
assert len(memory_section) < dream._MEMORY_FILE_MAX_CHARS + 500
async def test_phase1_caps_huge_history_entry(
self, dream, mock_provider, mock_runner, store,
):
"""A legacy oversized history entry (e.g. pre-#3412 raw_archive dump)
must not explode the Phase 1 prompt each entry is capped in the
preview, even though the JSONL record itself stays full-size."""
# Bypass the append_history cap by writing directly, simulating a
# record that was written by an older nanobot build before any caps.
async def test_caps_huge_history_entry(self, loop, mock_runner, store):
store.history_file.write_text(
json.dumps({
"cursor": 1,
"timestamp": "2026-04-01 10:00",
"content": "H" * (dream._HISTORY_ENTRY_PREVIEW_MAX_CHARS * 8),
}) + "\n",
json.dumps(
{
"cursor": 1,
"timestamp": "2026-04-01 10:00",
"content": "H" * (loop.dream._HISTORY_ENTRY_PREVIEW_MAX_CHARS * 8),
}
)
+ "\n",
encoding="utf-8",
)
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
history_section = user_msg.split("## Conversation History\n")[1].split(
"\n\n## Current Date"
)[0]
assert len(history_section) < loop.dream._HISTORY_ENTRY_PREVIEW_MAX_CHARS + 500
await dream.run()
user_msg = mock_provider.chat_with_retry.call_args.kwargs["messages"][1]["content"]
history_section = user_msg.split("## Conversation History\n")[1].split("\n\n## Current Date")[0]
assert len(history_section) < dream._HISTORY_ENTRY_PREVIEW_MAX_CHARS + 500
class TestDreamTools:
def test_apply_patch_tool_registered(self, loop):
tool = loop.dream._tools.get("apply_patch")
assert tool is not None
class TestDreamCaps:
def test_batch_size_default_is_5(self):
from nanobot.config.schema import DreamConfig
assert DreamConfig().max_batch_size == 5
def test_memory_cap_is_16k(self, loop):
assert loop.dream._MEMORY_FILE_MAX_CHARS == 16_000
class TestDreamSkipFiltering:
async def test_skip_entries_removed_from_prompt(self, loop, mock_runner, store):
store.append_history("- [skip] greeting\n- [permanent] User prefers dark mode")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
assert "User prefers dark mode" in user_msg
assert "[skip]" not in user_msg
assert "greeting" not in user_msg
class TestDreamAgeAnnotations:
async def test_prompt_includes_line_age_annotations(self, loop, mock_runner, store):
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
store.git.init()
store.git.auto_commit("initial memory state")
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
assert "## Current MEMORY.md" in user_msg
async def test_annotates_only_memory_not_soul_or_user(self, loop, mock_runner, store):
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
store.git.init()
store.git.auto_commit("initial state")
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
soul_section = user_msg.split("## Current SOUL.md")[1].split(
"## Current USER.md"
)[0]
user_section = user_msg.split("## Current USER.md")[1]
assert "" not in soul_section
assert "" not in user_section
async def test_prompt_works_without_git(self, loop, mock_runner, store):
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
mock_runner.run.assert_called_once()
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
assert "## Current MEMORY.md" in user_msg
async def test_prompt_carries_age_suffix_for_stale_lines(self, loop, mock_runner, store):
store.write_memory(
"# Memory\n- Project X active\n- fresh item\n- edge case line"
)
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
fake_ages = [
LineAge(age_days=30),
LineAge(age_days=20),
LineAge(age_days=14),
LineAge(age_days=5),
]
with patch.object(loop.dream.store.git, "line_ages", return_value=fake_ages):
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split(
"## Current SOUL.md"
)[0]
assert "← 30d" in memory_section
assert "← 20d" in memory_section
assert "← 14d" not in memory_section
assert "← 5d" not in memory_section
async def test_skips_annotation_when_disabled(self, loop, mock_runner, store):
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
loop.dream.annotate_line_ages = False
with patch.object(loop.dream.store.git, "line_ages") as mock_line_ages:
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
mock_line_ages.assert_not_called()
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
assert "" not in user_msg
async def test_skips_annotation_on_line_ages_length_mismatch(self, loop, mock_runner, store):
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
with patch.object(
loop.dream.store.git, "line_ages", return_value=[LineAge(age_days=999)]
):
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split(
"## Current SOUL.md"
)[0]
assert "" not in memory_section
class TestDreamSessionPersistence:
async def test_writes_session_on_success(self, loop, mock_runner, store):
store.append_history("event one")
store.append_history("event two")
mock_runner.run = AsyncMock(
return_value=_make_run_result(
tool_events=[
{"name": "edit_file", "status": "ok", "detail": "memory/MEMORY.md"}
],
)
)
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
session_path = store.memory_dir / ".dream_session.json"
assert session_path.exists()
data = json.loads(session_path.read_text(encoding="utf-8"))
assert data["batch"]["from_cursor"] == 0
assert data["batch"]["to_cursor"] == 2
assert data["batch"]["count"] == 2
assert data["stop_reason"] == "completed"
assert data["changelog"] == ["edit_file: memory/MEMORY.md"]
assert "timestamp" in data
assert "elapsed_seconds" in data
assert "messages" in data
async def test_no_session_record_on_failure(self, loop, mock_runner, store):
"""Failed batch should not write a session record (cursor stays put for retry)."""
store.append_history("event one")
mock_runner.run = AsyncMock(side_effect=RuntimeError("LLM error"))
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
session_path = store.memory_dir / ".dream_session.json"
assert not session_path.exists()
assert store.get_last_dream_cursor() == 0
async def test_session_contains_full_messages(self, loop, mock_runner, store):
store.append_history("event one")
messages = [
{"role": "system", "content": "you are a memory bot"},
{"role": "user", "content": "history here"},
{"role": "assistant", "content": "I will edit MEMORY.md"},
]
result = _make_run_result()
result.messages = messages
mock_runner.run = AsyncMock(return_value=result)
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
session_path = store.memory_dir / ".dream_session.json"
data = json.loads(session_path.read_text(encoding="utf-8"))
assert data["messages"] == messages
assert data["prompt_chars"] > 0
assert data["commit_sha"] is None
+176
View File
@@ -2,12 +2,39 @@
from __future__ import annotations
import asyncio
from contextlib import AsyncExitStack
from typing import Any
from unittest.mock import MagicMock
import pytest
from nanobot.agent.loop import AgentLoop
from nanobot.agent.tools import mcp as mcp_runtime
from nanobot.agent.tools.base import Tool
from nanobot.bus.queue import MessageBus
from nanobot.config.loader import load_config, save_config
from nanobot.config.schema import MCPServerConfig
class _FakeMcpTool(Tool):
def __init__(self, name: str) -> None:
self._name = name
@property
def name(self) -> str:
return self._name
@property
def description(self) -> str:
return "fake MCP tool"
@property
def parameters(self) -> dict[str, Any]:
return {"type": "object", "properties": {}}
async def execute(self, **_kwargs: Any) -> str:
return "ok"
def _make_loop(tmp_path, *, mcp_servers: dict | None = None) -> AgentLoop:
@@ -42,3 +69,152 @@ async def test_connect_mcp_retries_when_no_servers_connect(tmp_path, monkeypatch
assert attempts == 2
assert loop._mcp_connected is False
assert loop._mcp_stacks == {}
@pytest.mark.asyncio
async def test_reload_mcp_servers_adds_and_removes_tools_without_restart(
tmp_path,
monkeypatch: pytest.MonkeyPatch,
):
config_path = tmp_path / "config.json"
monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path)
config = load_config()
config.tools.mcp_servers["browserbase"] = MCPServerConfig(
type="stdio",
command="browserbase-mcp",
)
save_config(config)
closed: list[str] = []
async def _mark_closed(name: str) -> None:
closed.append(name)
async def _fake_connect(servers, registry):
stacks = {}
for name in servers:
registry.register(_FakeMcpTool(f"mcp_{name}_navigate"))
stack = AsyncExitStack()
await stack.__aenter__()
stack.push_async_callback(_mark_closed, name)
stacks[name] = stack
return stacks
monkeypatch.setattr("nanobot.agent.tools.mcp.connect_mcp_servers", _fake_connect)
loop = _make_loop(tmp_path, mcp_servers={})
added = await mcp_runtime.reload_servers(loop, loop.tools)
assert added["ok"] is True
assert added["added"] == ["browserbase"]
assert loop.tools.has("mcp_browserbase_navigate")
assert "browserbase" in loop._mcp_stacks
config = load_config()
del config.tools.mcp_servers["browserbase"]
save_config(config)
removed = await mcp_runtime.reload_servers(loop, loop.tools)
assert removed["ok"] is True
assert removed["removed"] == ["browserbase"]
assert not loop.tools.has("mcp_browserbase_navigate")
assert "browserbase" not in loop._mcp_stacks
assert closed == ["browserbase"]
@pytest.mark.asyncio
async def test_request_mcp_reload_reaches_runtime_control_without_restart(
tmp_path,
monkeypatch: pytest.MonkeyPatch,
):
config_path = tmp_path / "config.json"
monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path)
config = load_config()
config.tools.mcp_servers["browserbase"] = MCPServerConfig(
type="stdio",
command="browserbase-mcp",
)
save_config(config)
closed: list[str] = []
async def _mark_closed(name: str) -> None:
closed.append(name)
async def _fake_connect(servers, registry):
stacks = {}
for name in servers:
registry.register(_FakeMcpTool(f"mcp_{name}_navigate"))
stack = AsyncExitStack()
await stack.__aenter__()
stack.push_async_callback(_mark_closed, name)
stacks[name] = stack
return stacks
monkeypatch.setattr("nanobot.agent.tools.mcp.connect_mcp_servers", _fake_connect)
loop = _make_loop(tmp_path, mcp_servers={})
async def _handle_one_runtime_control() -> None:
msg = await loop.bus.consume_inbound()
handled = await mcp_runtime.handle_runtime_control(loop, msg, loop.tools)
assert handled is True
consumer = asyncio.create_task(_handle_one_runtime_control())
result = await mcp_runtime.request_mcp_reload(loop.bus, timeout=2.0)
await consumer
assert result["ok"] is True
assert result["added"] == ["browserbase"]
assert result["requires_restart"] is False
assert loop.tools.has("mcp_browserbase_navigate")
config = load_config()
del config.tools.mcp_servers["browserbase"]
save_config(config)
consumer = asyncio.create_task(_handle_one_runtime_control())
result = await mcp_runtime.request_mcp_reload(loop.bus, timeout=2.0)
await consumer
assert result["ok"] is True
assert result["removed"] == ["browserbase"]
assert result["requires_restart"] is False
assert not loop.tools.has("mcp_browserbase_navigate")
assert closed == ["browserbase"]
@pytest.mark.asyncio
async def test_reload_mcp_servers_retries_configured_server_without_live_stack(
tmp_path,
monkeypatch: pytest.MonkeyPatch,
):
config_path = tmp_path / "config.json"
monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path)
config = load_config()
config.tools.mcp_servers["browserbase"] = MCPServerConfig(
type="stdio",
command="browserbase-mcp",
)
save_config(config)
async def _fake_connect(servers, registry):
stacks = {}
for name in servers:
registry.register(_FakeMcpTool(f"mcp_{name}_navigate"))
stack = AsyncExitStack()
await stack.__aenter__()
stacks[name] = stack
return stacks
monkeypatch.setattr("nanobot.agent.tools.mcp.connect_mcp_servers", _fake_connect)
loop = _make_loop(tmp_path, mcp_servers={"browserbase": config.tools.mcp_servers["browserbase"]})
result = await mcp_runtime.reload_servers(loop, loop.tools)
assert result["ok"] is True
assert result["added"] == []
assert result["changed"] == []
assert result["retried"] == ["browserbase"]
assert loop.tools.has("mcp_browserbase_navigate")
await loop.close_mcp()
+125
View File
@@ -0,0 +1,125 @@
"""Tests for memory system: Consolidator, token estimation, truncation."""
from unittest.mock import AsyncMock, MagicMock
import pytest
from nanobot.agent.memory import _TIKTOKEN_ENC, Consolidator, MemoryStore, _estimate_tokens
@pytest.fixture
def store(tmp_path):
s = MemoryStore(tmp_path)
s.write_soul("# Soul\n- Helpful")
s.write_user("# User\n- Developer")
s.write_memory("# Memory\n- Project X active")
return s
@pytest.fixture
def mock_provider():
p = MagicMock()
p.chat_with_retry = AsyncMock()
p.generation.max_tokens = 4096
return p
@pytest.fixture
def mock_sessions():
return MagicMock()
@pytest.fixture
def mock_build_messages():
return MagicMock(return_value=[])
@pytest.fixture
def mock_get_tool_definitions():
return MagicMock(return_value=[])
@pytest.fixture
def consolidator(store, mock_provider, mock_sessions, mock_build_messages, mock_get_tool_definitions):
return Consolidator(
store=store,
provider=mock_provider,
model="test-model",
sessions=mock_sessions,
context_window_tokens=128_000,
build_messages=mock_build_messages,
get_tool_definitions=mock_get_tool_definitions,
)
class TestEstimateTokens:
def test_estimate_tokens_returns_positive(self):
assert _estimate_tokens("hello world") > 0
def test_estimate_tokens_english_approximate(self):
# English is roughly 1 token per 4 chars as fallback
text = "a " * 100
if _TIKTOKEN_ENC is not None:
expected = len(_TIKTOKEN_ENC.encode(text))
else:
expected = len(text) // 4
assert _estimate_tokens(text) == expected
class TestTruncateToTokenBudget:
def test_reserve_tokens_reduces_budget(self, consolidator):
long_text = "word " * 200_000
# Without reserve, more text survives
no_reserve = consolidator._truncate_to_token_budget(long_text, reserve_tokens=0)
with_reserve = consolidator._truncate_to_token_budget(long_text, reserve_tokens=500)
assert len(with_reserve) < len(no_reserve)
def test_reserve_tokens_zero_default(self, consolidator):
text = "hello world"
result = consolidator._truncate_to_token_budget(text)
assert result == text
class TestConsolidatorPrompt:
def test_prompt_contains_snip(self):
from nanobot.utils.prompt_templates import render_template
text = render_template("agent/consolidator_archive.md", strip=True)
assert "SNIP" in text
assert "[permanent]" in text
assert "[skip]" in text
class TestConsolidatorArchive:
async def test_archive_injects_dedup_context(self, consolidator, mock_provider, store):
store.write_memory("- User prefers dark mode")
store.write_user("- Developer")
messages = [{"role": "user", "content": "hello", "timestamp": "2026-01-01 10:00"}]
mock_provider.chat_with_retry.return_value = MagicMock(
content="(nothing)", finish_reason="stop"
)
await consolidator.archive(messages)
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs["messages"][1]["content"]
assert "## Current MEMORY.md (for dedup)" in user_msg
assert "User prefers dark mode" in user_msg
assert "## Current USER.md (for dedup)" in user_msg
assert "Developer" in user_msg
async def test_archive_skips_dedup_when_budget_exhausted(self, consolidator, mock_provider, store):
# Shrink token budget so dedup context (always capped at ~6000 chars)
# exceeds the available room.
consolidator.context_window_tokens = 6_000
store.write_memory("word " * 10_000)
messages = [{"role": "user", "content": "hello", "timestamp": "2026-01-01 10:00"}]
mock_provider.chat_with_retry.return_value = MagicMock(
content="(nothing)", finish_reason="stop"
)
await consolidator.archive(messages)
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs["messages"][1]["content"]
# Should not contain dedup context when budget is exhausted
assert "## Current MEMORY.md (for dedup)" not in user_msg
+20
View File
@@ -346,6 +346,26 @@ class TestSyncWorkspaceTemplates:
content = (workspace / "AGENTS.md").read_text()
assert content == "existing content"
def test_does_not_create_tools_md(self, tmp_path):
"""Tool contract is injected internally, not copied into user workspaces."""
workspace = tmp_path / "workspace"
added = sync_workspace_templates(workspace, silent=True)
assert "TOOLS.md" not in added
assert not (workspace / "TOOLS.md").exists()
def test_preserves_existing_tools_md_without_overwriting(self, tmp_path):
"""Legacy user workspaces may have TOOLS.md; sync should leave it untouched."""
workspace = tmp_path / "workspace"
workspace.mkdir(parents=True)
tools_path = workspace / "TOOLS.md"
tools_path.write_text("custom tool notes", encoding="utf-8")
sync_workspace_templates(workspace, silent=True)
assert tools_path.read_text(encoding="utf-8") == "custom tool notes"
def test_creates_memory_directory(self, tmp_path):
"""Should create memory directory structure."""
workspace = tmp_path / "workspace"
+1 -1
View File
@@ -241,7 +241,7 @@ def test_inline_fallback_reasoning_effort_does_not_inherit_primary() -> None:
signature = provider_signature(config)
fallback_signatures = signature[-1]
assert fallback_signatures[0][11] is None
assert fallback_signatures[0][12] is None
# -- FallbackProvider tests --
+184
View File
@@ -0,0 +1,184 @@
"""Tests for sustained-goal continuation in AgentRunner.
When a goal_active_predicate returns True, the runner must not exit with
stop_reason="completed" after a plain-text final response. Instead it should
inject a continuation message and keep looping (similar to mid-turn injection).
"""
from __future__ import annotations
from unittest.mock import AsyncMock, MagicMock
import pytest
from nanobot.config.schema import AgentDefaults
from nanobot.providers.base import LLMProvider, LLMResponse
_MAX_TOOL_RESULT_CHARS = AgentDefaults().max_tool_result_chars
@pytest.mark.asyncio
async def test_runner_exits_normally_without_predicate():
"""Baseline: no predicate, runner exits with completed on final text."""
from nanobot.agent.runner import AgentRunner, AgentRunSpec
provider = MagicMock(spec=LLMProvider)
provider.chat_with_retry = AsyncMock(return_value=LLMResponse(
content="all done", tool_calls=[], usage={},
))
tools = MagicMock()
tools.get_definitions.return_value = []
runner = AgentRunner(provider)
result = await runner.run(AgentRunSpec(
initial_messages=[{"role": "user", "content": "do task"}],
tools=tools,
model="test-model",
max_iterations=2,
max_tool_result_chars=_MAX_TOOL_RESULT_CHARS,
))
assert result.stop_reason == "completed"
assert result.final_content == "all done"
@pytest.mark.asyncio
async def test_runner_exits_normally_with_inactive_goal():
"""Predicate returns False, runner should exit normally."""
from nanobot.agent.runner import AgentRunner, AgentRunSpec
provider = MagicMock(spec=LLMProvider)
provider.chat_with_retry = AsyncMock(return_value=LLMResponse(
content="all done", tool_calls=[], usage={},
))
tools = MagicMock()
tools.get_definitions.return_value = []
runner = AgentRunner(provider)
result = await runner.run(AgentRunSpec(
initial_messages=[{"role": "user", "content": "do task"}],
tools=tools,
model="test-model",
max_iterations=2,
max_tool_result_chars=_MAX_TOOL_RESULT_CHARS,
goal_active_predicate=lambda: False,
))
assert result.stop_reason == "completed"
assert result.final_content == "all done"
@pytest.mark.asyncio
async def test_runner_forces_continue_when_goal_active():
"""Predicate returns True on final text → runner injects continuation and loops.
We set max_iterations=3 and let the provider return final text every time.
Without the fix this would exit on the first iteration with stop_reason
"completed". With the fix the runner is forced to continue until
max_iterations is hit.
"""
from nanobot.agent.runner import AgentRunner, AgentRunSpec
provider = MagicMock(spec=LLMProvider)
provider.chat_with_retry = AsyncMock(return_value=LLMResponse(
content="still working", tool_calls=[], usage={},
))
tools = MagicMock()
tools.get_definitions.return_value = []
runner = AgentRunner(provider)
result = await runner.run(AgentRunSpec(
initial_messages=[{"role": "user", "content": "do task"}],
tools=tools,
model="test-model",
max_iterations=3,
max_tool_result_chars=_MAX_TOOL_RESULT_CHARS,
goal_active_predicate=lambda: True,
))
# Because the predicate keeps returning True, the runner should never
# naturally complete. It loops until max_iterations is exhausted.
assert result.stop_reason == "max_iterations"
# The injected continuation message should be present in the message list.
user_msgs = [m for m in result.messages if m.get("role") == "user"]
assert any("active sustained goal" in str(m.get("content", "")) for m in user_msgs)
@pytest.mark.asyncio
async def test_runner_respects_max_iterations_even_with_active_goal():
"""A single iteration with active goal still hits max_iterations."""
from nanobot.agent.runner import AgentRunner, AgentRunSpec
provider = MagicMock(spec=LLMProvider)
provider.chat_with_retry = AsyncMock(return_value=LLMResponse(
content="still working", tool_calls=[], usage={},
))
tools = MagicMock()
tools.get_definitions.return_value = []
runner = AgentRunner(provider)
result = await runner.run(AgentRunSpec(
initial_messages=[{"role": "user", "content": "do task"}],
tools=tools,
model="test-model",
max_iterations=1,
max_tool_result_chars=_MAX_TOOL_RESULT_CHARS,
goal_active_predicate=lambda: True,
))
assert result.stop_reason == "max_iterations"
@pytest.mark.asyncio
async def test_runner_does_not_force_continue_on_error():
"""Even with active goal, an LLM error should exit with stop_reason="error"."""
from nanobot.agent.runner import AgentRunner, AgentRunSpec
provider = MagicMock(spec=LLMProvider)
provider.chat_with_retry = AsyncMock(return_value=LLMResponse(
content=None, tool_calls=[], usage={},
finish_reason="error",
))
tools = MagicMock()
tools.get_definitions.return_value = []
runner = AgentRunner(provider)
result = await runner.run(AgentRunSpec(
initial_messages=[{"role": "user", "content": "do task"}],
tools=tools,
model="test-model",
max_iterations=2,
max_tool_result_chars=_MAX_TOOL_RESULT_CHARS,
goal_active_predicate=lambda: True,
))
assert result.stop_reason == "error"
@pytest.mark.asyncio
async def test_runner_uses_custom_goal_continue_message():
"""Custom goal_continue_message should be injected instead of the default."""
from nanobot.agent.runner import AgentRunner, AgentRunSpec
provider = MagicMock(spec=LLMProvider)
provider.chat_with_retry = AsyncMock(return_value=LLMResponse(
content="still working", tool_calls=[], usage={},
))
tools = MagicMock()
tools.get_definitions.return_value = []
custom_msg = "CUSTOM_CONTINUE_PLEASE"
runner = AgentRunner(provider)
result = await runner.run(AgentRunSpec(
initial_messages=[{"role": "user", "content": "do task"}],
tools=tools,
model="test-model",
max_iterations=2,
max_tool_result_chars=_MAX_TOOL_RESULT_CHARS,
goal_active_predicate=lambda: True,
goal_continue_message=custom_msg,
))
user_msgs = [m for m in result.messages if m.get("role") == "user"]
assert any(custom_msg in str(m.get("content", "")) for m in user_msgs)
+92
View File
@@ -292,3 +292,95 @@ def test_from_config_static_preset_loader_does_not_enable_hot_reload(tmp_path) -
loop = AgentLoop.from_config(config)
assert loop._provider_snapshot_loader is None
assert loop._preset_snapshot_loader is not None
class TestDreamModelOverride:
def test_dream_follows_main_when_no_override(self, tmp_path) -> None:
provider = _provider("base-model")
loop = AgentLoop(
bus=MessageBus(),
provider=provider,
workspace=tmp_path,
model="base-model",
context_window_tokens=1000,
)
assert loop.dream.model == "base-model"
assert loop.dream.provider is provider
def test_dream_raw_model_override(self, tmp_path) -> None:
provider = _provider("base-model")
loop = AgentLoop(
bus=MessageBus(),
provider=provider,
workspace=tmp_path,
model="base-model",
context_window_tokens=1000,
dream_model_override="custom-model-v2",
)
assert loop.dream.model == "custom-model-v2"
assert loop.dream.provider is provider
def test_dream_preset_override(self, tmp_path) -> None:
cheap_provider = _provider("openai/gpt-4.1-mini", max_tokens=2048)
preset = ModelPresetConfig(
model="openai/gpt-4.1-mini",
provider="openai",
max_tokens=2048,
context_window_tokens=128_000,
)
loop = AgentLoop(
bus=MessageBus(),
provider=_provider("base-model"),
workspace=tmp_path,
model="base-model",
context_window_tokens=1000,
model_presets={"cheap": preset},
dream_model_override="cheap",
preset_snapshot_loader=lambda _name: ProviderSnapshot(
provider=cheap_provider,
model=preset.model,
context_window_tokens=preset.context_window_tokens,
signature=("cheap", preset.model),
),
)
assert loop.dream.model == "openai/gpt-4.1-mini"
assert loop.dream.provider is cheap_provider
assert loop.dream._runner.provider is cheap_provider
def test_dream_override_survives_main_preset_switch(self, tmp_path) -> None:
base_provider = _provider("base-model")
fast_provider = _provider("openai/gpt-4.1", max_tokens=4096)
cheap_provider = _provider("openai/gpt-4.1-mini", max_tokens=2048)
loop = AgentLoop(
bus=MessageBus(),
provider=base_provider,
workspace=tmp_path,
model="base-model",
context_window_tokens=1000,
model_presets={
"fast": ModelPresetConfig(model="openai/gpt-4.1"),
"cheap": ModelPresetConfig(model="openai/gpt-4.1-mini"),
},
dream_model_override="cheap",
preset_snapshot_loader=lambda name: ProviderSnapshot(
provider=fast_provider if name == "fast" else cheap_provider,
model="openai/gpt-4.1" if name == "fast" else "openai/gpt-4.1-mini",
context_window_tokens=32_768 if name == "fast" else 128_000,
signature=(name, "model"),
),
)
# Initially dream is on cheap
assert loop.dream.model == "openai/gpt-4.1-mini"
assert loop.dream.provider is cheap_provider
# Switch main preset to fast
loop.set_model_preset("fast")
# Main agent should be on fast
assert loop.model == "openai/gpt-4.1"
assert loop.provider is fast_provider
# Dream should still be on cheap override
assert loop.dream.model == "openai/gpt-4.1-mini"
assert loop.dream.provider is cheap_provider
assert loop.dream._runner.provider is cheap_provider
@@ -56,6 +56,20 @@ def test_list_sessions_includes_user_preview(tmp_path):
assert rows[0]["preview"] == "帮我总结一下 OpenAI 的最新硬件计划"
def test_list_sessions_bounds_preview_scan(tmp_path):
manager = SessionManager(tmp_path)
session = manager.get_or_create("websocket:chat-long-preview")
for index in range(220):
session.add_message("assistant", f"assistant trace {index}")
session.add_message("user", "this should not force a full sidebar scan")
manager.save(session)
rows = manager.list_sessions()
assert rows[0]["key"] == "websocket:chat-long-preview"
assert rows[0]["preview"] == "assistant trace 0"
# --- Original regression test (from PR 2075) ---
def test_get_history_drops_orphan_tool_results_when_window_cuts_tool_calls():
@@ -359,6 +373,31 @@ def test_get_history_synthesizes_breadcrumb_for_image_only_turn():
assert history[0] == {"role": "user", "content": "[image: /m/pic.png]"}
def test_get_history_synthesizes_cli_app_attachment_breadcrumb():
session = Session(key="test:cli-app")
session.messages.append(
{
"role": "user",
"content": "please use @drawio",
"cli_apps": [{
"name": "drawio",
"entry_point": "cli-anything-drawio",
}],
}
)
history = session.get_history(max_messages=500)
assert history == [{
"role": "user",
"content": (
"please use @drawio\n"
"[CLI App Attachment: @drawio; tool=run_cli_app; "
"entry_point=cli-anything-drawio; skill=skills/cli-app-drawio/SKILL.md]"
),
}]
def test_get_history_ignores_media_kwarg_on_non_user_rows():
"""``media`` only ever appears on user entries in practice, but the
synthesizer must be defensive: assistants / tools with list content
+33
View File
@@ -94,6 +94,39 @@ async def test_subagent_uses_configured_max_iterations(tmp_path):
mgr.runner.run.assert_awaited_once()
@pytest.mark.asyncio
async def test_spawn_forwards_temperature_to_run_spec(tmp_path):
"""A temperature passed to spawn() should reach the AgentRunSpec."""
from nanobot.agent.subagent import SubagentManager
from nanobot.bus.queue import MessageBus
bus = MessageBus()
provider = MagicMock()
provider.get_default_model.return_value = "test-model"
mgr = SubagentManager(
provider=provider,
workspace=tmp_path,
bus=bus,
max_tool_result_chars=_MAX_TOOL_RESULT_CHARS,
)
mgr._announce_result = AsyncMock()
seen = {}
async def fake_run(spec):
seen["temperature"] = spec.temperature
return SimpleNamespace(
stop_reason="done", final_content="done", error=None, tool_events=[],
)
mgr.runner.run = AsyncMock(side_effect=fake_run)
await mgr.spawn(task="do task", temperature=0.9)
await asyncio.gather(*mgr._running_tasks.values(), return_exceptions=True)
assert seen["temperature"] == 0.9
@pytest.mark.asyncio
async def test_spawn_tool_rejects_when_at_concurrency_limit(tmp_path):
"""SpawnTool should return an error string when the concurrency limit is reached."""
File diff suppressed because it is too large Load Diff
+525
View File
@@ -0,0 +1,525 @@
"""Unit tests for the Signal markdown → plain text + textStyle converter."""
from nanobot.channels.signal import _markdown_to_signal, _partition_styles
from nanobot.utils.helpers import split_message
def _utf16_len(s: str) -> int:
return len(s.encode("utf-16-le")) // 2
def styles_for(plain: str, text_styles: list[str]) -> dict[str, list[str]]:
"""Return a dict mapping each styled substring to its style list."""
result: dict[str, list[str]] = {}
for entry in text_styles:
start_s, length_s, style = entry.split(":", 2)
start, length = int(start_s), int(length_s)
span = plain[start : start + length]
result.setdefault(span, []).append(style)
return result
def utf16_styles_for(plain: str, text_styles: list[str]) -> dict[str, list[str]]:
"""Like styles_for, but slices `plain` using UTF-16 offsets (Signal's units)."""
encoded = plain.encode("utf-16-le")
result: dict[str, list[str]] = {}
for entry in text_styles:
start_s, length_s, style = entry.split(":", 2)
start, length = int(start_s), int(length_s)
span = encoded[start * 2 : (start + length) * 2].decode("utf-16-le")
result.setdefault(span, []).append(style)
return result
# ---------------------------------------------------------------------------
# Basic cases
# ---------------------------------------------------------------------------
def test_empty():
plain, styles = _markdown_to_signal("")
assert plain == ""
assert styles == []
def test_plain_text():
plain, styles = _markdown_to_signal("hello world")
assert plain == "hello world"
assert styles == []
def test_bold_stars():
plain, styles = _markdown_to_signal("say **hello** now")
assert plain == "say hello now"
assert styles_for(plain, styles) == {"hello": ["BOLD"]}
def test_bold_underscores():
plain, styles = _markdown_to_signal("say __hello__ now")
assert plain == "say hello now"
assert styles_for(plain, styles) == {"hello": ["BOLD"]}
def test_italic_star():
plain, styles = _markdown_to_signal("say *hello* now")
assert plain == "say hello now"
assert styles_for(plain, styles) == {"hello": ["ITALIC"]}
def test_italic_underscore():
plain, styles = _markdown_to_signal("say _hello_ now")
assert plain == "say hello now"
assert styles_for(plain, styles) == {"hello": ["ITALIC"]}
def test_strikethrough():
plain, styles = _markdown_to_signal("say ~~hello~~ now")
assert plain == "say hello now"
assert styles_for(plain, styles) == {"hello": ["STRIKETHROUGH"]}
# ---------------------------------------------------------------------------
# Code
# ---------------------------------------------------------------------------
def test_inline_code():
plain, styles = _markdown_to_signal("run `ls -la` here")
assert plain == "run ls -la here"
assert styles_for(plain, styles) == {"ls -la": ["MONOSPACE"]}
def test_code_block():
plain, styles = _markdown_to_signal("```\nprint('hi')\n```")
assert "print('hi')" in plain
assert styles_for(plain, styles).get("print('hi')\n") == ["MONOSPACE"] or "MONOSPACE" in str(
styles_for(plain, styles)
)
def test_code_block_with_lang():
plain, styles = _markdown_to_signal("```python\ncode\n```")
assert "code" in plain
assert any("MONOSPACE" in s for s in styles)
def test_code_block_not_processed_further():
"""Markdown inside a code block must not be styled."""
plain, styles = _markdown_to_signal("```\n**not bold**\n```")
assert "**not bold**" in plain
# Only MONOSPACE should be applied, no BOLD
for entry in styles:
assert "BOLD" not in entry
def test_inline_code_not_processed_further():
"""Markdown inside inline code must not be styled."""
plain, styles = _markdown_to_signal("use `**raw**` please")
assert "**raw**" in plain
for entry in styles:
assert "BOLD" not in entry
# ---------------------------------------------------------------------------
# Headers
# ---------------------------------------------------------------------------
def test_header_becomes_bold():
plain, styles = _markdown_to_signal("# My Title")
assert plain == "My Title"
assert styles_for(plain, styles) == {"My Title": ["BOLD"]}
def test_h2_becomes_bold():
plain, styles = _markdown_to_signal("## Sub-section")
assert plain == "Sub-section"
assert styles_for(plain, styles) == {"Sub-section": ["BOLD"]}
# ---------------------------------------------------------------------------
# Blockquotes
# ---------------------------------------------------------------------------
def test_blockquote_strips_marker():
plain, styles = _markdown_to_signal("> some quote")
assert plain == "some quote"
assert styles == []
# ---------------------------------------------------------------------------
# Lists
# ---------------------------------------------------------------------------
def test_bullet_dash():
plain, styles = _markdown_to_signal("- item one")
assert plain == "• item one"
def test_bullet_star():
plain, styles = _markdown_to_signal("* item two")
assert plain == "• item two"
def test_numbered_list():
plain, styles = _markdown_to_signal("1. first\n2. second")
assert "1. first" in plain
assert "2. second" in plain
# ---------------------------------------------------------------------------
# Links
# ---------------------------------------------------------------------------
def test_link_text_differs_from_url():
plain, styles = _markdown_to_signal("[Click here](https://example.com)")
assert plain == "Click here (https://example.com)"
assert styles == []
def test_link_text_equals_url():
plain, styles = _markdown_to_signal("[https://example.com](https://example.com)")
assert plain == "https://example.com"
assert styles == []
def test_link_text_equals_url_without_scheme():
plain, styles = _markdown_to_signal("[example.com](https://example.com)")
assert plain == "https://example.com"
# ---------------------------------------------------------------------------
# Mixed / nesting
# ---------------------------------------------------------------------------
def test_bold_and_italic_adjacent():
plain, styles = _markdown_to_signal("**bold** and *italic*")
assert plain == "bold and italic"
sd = styles_for(plain, styles)
assert sd.get("bold") == ["BOLD"]
assert sd.get("italic") == ["ITALIC"]
def test_header_with_inline_code():
"""Header becomes BOLD; code inside becomes MONOSPACE (not double-BOLD)."""
plain, styles = _markdown_to_signal("# Use `grep`")
assert plain == "Use grep"
sd = styles_for(plain, styles)
assert "BOLD" in sd.get("Use ", []) or "BOLD" in str(styles)
assert "MONOSPACE" in sd.get("grep", [])
def test_multiline_mixed():
md = "**Title**\n\nSome *italic* text.\n\n- bullet\n- another"
plain, styles = _markdown_to_signal(md)
assert "Title" in plain
assert "italic" in plain
assert "• bullet" in plain
sd = styles_for(plain, styles)
assert "BOLD" in sd.get("Title", [])
assert "ITALIC" in sd.get("italic", [])
# ---------------------------------------------------------------------------
# Table rendering
# ---------------------------------------------------------------------------
def test_table_rendered_as_monospace():
md = "| A | B |\n| - | - |\n| 1 | 2 |"
plain, styles = _markdown_to_signal(md)
assert "A" in plain and "B" in plain
assert any("MONOSPACE" in s for s in styles)
# ---------------------------------------------------------------------------
# Style range format
# ---------------------------------------------------------------------------
def test_style_range_format():
"""Each style entry must be 'start:length:STYLE'."""
_, styles = _markdown_to_signal("**bold** text")
for entry in styles:
parts = entry.split(":")
assert len(parts) == 3
assert parts[0].isdigit()
assert parts[1].isdigit()
assert parts[2] in {"BOLD", "ITALIC", "STRIKETHROUGH", "MONOSPACE", "SPOILER"}
def test_style_ranges_are_within_bounds():
text = "hello **world** end"
plain, styles = _markdown_to_signal(text)
for entry in styles:
start_s, length_s, _ = entry.split(":", 2)
start, length = int(start_s), int(length_s)
assert start >= 0
assert start + length <= len(plain)
# ---------------------------------------------------------------------------
# Non-BMP / UTF-16 offsets
#
# Signal's BodyRange (and signal-cli's textStyle) interprets start/length in
# UTF-16 code units. Python's len() counts code points, so characters outside
# the BMP (emojis, supplementary CJK) shift offsets by +1 per occurrence.
# ---------------------------------------------------------------------------
def assert_within_utf16_bounds(plain: str, styles: list[str]) -> None:
limit = _utf16_len(plain)
for entry in styles:
start_s, length_s, _ = entry.split(":", 2)
start, length = int(start_s), int(length_s)
assert start >= 0
assert start + length <= limit, f"range {entry} exceeds utf-16 length {limit} of {plain!r}"
def test_bold_with_emoji_inside():
plain, styles = _markdown_to_signal("**hi 🎉 bye**")
assert plain == "hi 🎉 bye"
assert utf16_styles_for(plain, styles) == {"hi 🎉 bye": ["BOLD"]}
assert_within_utf16_bounds(plain, styles)
def test_italic_with_trailing_emoji():
plain, styles = _markdown_to_signal("*bye 🎉*")
assert plain == "bye 🎉"
assert utf16_styles_for(plain, styles) == {"bye 🎉": ["ITALIC"]}
assert_within_utf16_bounds(plain, styles)
def test_bold_after_emoji_prefix():
plain, styles = _markdown_to_signal("🎉 **bold**")
assert plain == "🎉 bold"
assert utf16_styles_for(plain, styles) == {"bold": ["BOLD"]}
assert_within_utf16_bounds(plain, styles)
def test_bold_after_and_inside_emoji():
plain, styles = _markdown_to_signal("🎉 **a 🎊 b**")
assert plain == "🎉 a 🎊 b"
assert utf16_styles_for(plain, styles) == {"a 🎊 b": ["BOLD"]}
assert_within_utf16_bounds(plain, styles)
def test_supplementary_cjk_in_bold():
"""Non-BMP CJK (U+20BB7) proves the bug is UTF-16, not emoji-specific."""
plain, styles = _markdown_to_signal("**𠮷野家**")
assert plain == "𠮷野家"
assert utf16_styles_for(plain, styles) == {"𠮷野家": ["BOLD"]}
assert_within_utf16_bounds(plain, styles)
def test_zwj_emoji_in_bold():
"""ZWJ family sequence = multiple surrogate pairs + BMP ZWJs."""
plain, styles = _markdown_to_signal("**hi 👨‍👩‍👧 bye**")
assert plain == "hi 👨‍👩‍👧 bye"
assert utf16_styles_for(plain, styles) == {"hi 👨‍👩‍👧 bye": ["BOLD"]}
assert_within_utf16_bounds(plain, styles)
def test_ascii_offsets_unchanged():
"""ASCII-only path must produce the same offsets as before the UTF-16 fix."""
plain, styles = _markdown_to_signal("**bold** plain *it*")
assert plain == "bold plain it"
assert sorted(styles) == sorted(["0:4:BOLD", "11:2:ITALIC"])
def test_reported_daily_brief_pattern():
"""Regression for the reported bug: a single non-BMP emoji shifts every
subsequent styled span left by 1 UTF-16 unit, lopping off the last letter.
"""
md = (
"**Weather**\n"
"- Conditions: 🌩️ Thunderstorms\n\n"
"**News**\n"
"*World*\n"
"*Local*\n\n"
"**Quote of the Day**"
)
plain, styles = _markdown_to_signal(md)
sd = utf16_styles_for(plain, styles)
assert sd.get("Weather") == ["BOLD"]
assert sd.get("News") == ["BOLD"]
assert sd.get("World") == ["ITALIC"]
assert sd.get("Local") == ["ITALIC"]
assert sd.get("Quote of the Day") == ["BOLD"]
assert_within_utf16_bounds(plain, styles)
# ---------------------------------------------------------------------------
# Chunk redistribution
#
# split_message can break a long Signal payload into multiple chunks. The
# style ranges from _markdown_to_signal are anchored to the full text, so
# they must be redistributed per-chunk with rebased offsets — otherwise
# styles for chunks 1..N are silently lost.
# ---------------------------------------------------------------------------
def _resolve_chunk_styles(text: str, max_len: int) -> tuple[list[str], list[list[str]]]:
"""Helper: full markdown → signal pipeline, including chunking."""
plain, styles = _markdown_to_signal(text)
chunks = split_message(plain, max_len) if plain else [""]
return chunks, _partition_styles(plain, chunks, styles)
def test_partition_styles_single_chunk_passthrough():
plain, styles = _markdown_to_signal("**bold** plain *it*")
parts = _partition_styles(plain, [plain], styles)
assert parts == [styles]
def test_partition_styles_no_styles():
plain = "hello world"
assert _partition_styles(plain, [plain], []) == [[]]
assert _partition_styles(plain, ["hello", "world"], []) == [[], []]
def test_partition_styles_drops_styles_outside_chunks():
"""Whitespace trimmed by split_message must not carry a style range."""
plain = "a b"
# Fake a style spanning the trimmed whitespace only.
chunks = ["a", "b"]
parts = _partition_styles(plain, chunks, ["1:3:BOLD"])
assert parts == [[], []]
def test_partition_styles_long_message_preserves_chunk_one_styles():
"""A bold span deep in the message must follow the message into chunk 1."""
# Two ~30-char paragraphs separated by a blank line, then **tail**.
line_a = "alpha " * 5 # 30 chars, ends with space
line_b = "beta " * 5
md = f"{line_a.strip()}\n\n{line_b.strip()}\n\n**tail**"
plain, styles = _markdown_to_signal(md)
# Force a split between the paragraphs.
max_len = len(line_a.strip()) + 2 # fits paragraph A + the "\n\n"
chunks = split_message(plain, max_len)
assert len(chunks) >= 2, "test setup must produce a split"
parts = _partition_styles(plain, chunks, styles)
# The bold "tail" should land in the last chunk, with chunk-relative offset.
final_chunk = chunks[-1]
final_styles = parts[-1]
assert any("BOLD" in s for s in final_styles)
for entry in final_styles:
s, ln, _ = entry.split(":", 2)
start, length = int(s), int(ln)
slice_ = final_chunk.encode("utf-16-le")[start * 2 : (start + length) * 2].decode(
"utf-16-le"
)
assert slice_ == "tail"
def test_partition_styles_chunk_zero_styles_unchanged():
"""Styles entirely in chunk 0 keep their original offsets."""
md = "**head** middle and **tail**"
plain, styles = _markdown_to_signal(md)
# Split so chunk 0 contains "head" and part of the rest, chunk 1 contains "tail".
chunks = split_message(plain, 12)
assert len(chunks) >= 2
parts = _partition_styles(plain, chunks, styles)
# "head" lives in chunk 0; assert its offset is unchanged (chunk 0 starts at 0).
head_entries = [s for s in parts[0] if "BOLD" in s]
assert any(s.startswith("0:4:") for s in head_entries)
def test_partition_styles_with_non_bmp_chunk_offset():
"""Chunk-start offsets must be expressed in UTF-16 code units."""
# Emoji in chunk 0, bold in chunk 1.
md = "🎉 alpha beta gamma\n\n**tail**"
plain, styles = _markdown_to_signal(md)
chunks = split_message(plain, 18)
assert len(chunks) >= 2
parts = _partition_styles(plain, chunks, styles)
final_styles = parts[-1]
assert any("BOLD" in s for s in final_styles)
final_chunk = chunks[-1]
for entry in final_styles:
s, ln, _ = entry.split(":", 2)
start, length = int(s), int(ln)
slice_ = final_chunk.encode("utf-16-le")[start * 2 : (start + length) * 2].decode(
"utf-16-le"
)
assert slice_ == "tail"
def test_partition_styles_range_spanning_chunks_is_split():
"""A style range that straddles a chunk boundary gets sliced into both chunks."""
# Construct manually: plain = "abc def", style covers "abc def" (whole thing).
plain = "abc def"
chunks = split_message(plain, 4) # "abc" / "def"
assert chunks == ["abc", "def"]
parts = _partition_styles(plain, chunks, ["0:7:BOLD"])
# Chunk 0 holds 0:3:BOLD, chunk 1 holds 0:3:BOLD (length=3 each, "def" only
# since the space was trimmed by lstrip).
assert parts[0] == ["0:3:BOLD"]
assert parts[1] == ["0:3:BOLD"]
# ---------------------------------------------------------------------------
# Adjacency, nesting, and malformed input
# ---------------------------------------------------------------------------
def test_bold_italic_combo_outer_bold_inner_italic():
"""`**_combo_**` carries both BOLD and ITALIC over the same span."""
plain, styles = _markdown_to_signal("**_combo_**")
assert plain == "combo"
sd = styles_for(plain, styles)
assert set(sd.get("combo", [])) == {"BOLD", "ITALIC"}
def test_bold_and_italic_adjacent_no_separator():
"""`**bold***italic*` produces BOLD on `bold` and ITALIC on `italic`."""
plain, styles = _markdown_to_signal("**bold***italic*")
assert plain == "bolditalic"
sd = styles_for(plain, styles)
assert sd.get("bold") == ["BOLD"]
assert sd.get("italic") == ["ITALIC"]
def test_unclosed_bold_falls_through_as_plain():
"""An unmatched `**` opener round-trips as literal text with no style."""
plain, styles = _markdown_to_signal("**bold")
assert plain == "**bold"
assert styles == []
def test_unclosed_inline_code_falls_through_as_plain():
"""An unmatched backtick round-trips as literal text with no style."""
plain, styles = _markdown_to_signal("use `grep")
assert plain == "use `grep"
assert styles == []
def test_inline_code_inside_blockquote():
"""Blockquote prefix is stripped; inline code becomes MONOSPACE."""
plain, styles = _markdown_to_signal("> use `grep`")
assert plain == "use grep"
sd = styles_for(plain, styles)
assert sd.get("grep") == ["MONOSPACE"]
def test_header_with_inner_bold_produces_contiguous_bold_ranges():
"""`# **wrap** me` — header forces BOLD over the whole line; the inner `**`
splits the run, yielding two contiguous BOLD ranges that together cover
"wrap me". This is intentional Signal renders adjacent same-style ranges
as a single visual span.
"""
plain, styles = _markdown_to_signal("# **wrap** me")
assert plain == "wrap me"
# Both ranges are BOLD; collectively they cover the whole "wrap me".
bold_ranges = [s for s in styles if s.endswith(":BOLD")]
assert len(bold_ranges) == 2
covered = set()
for entry in bold_ranges:
start, length, _ = entry.split(":", 2)
for i in range(int(start), int(start) + int(length)):
covered.add(i)
assert covered == set(range(len(plain)))
+127 -2
View File
@@ -30,7 +30,7 @@ from nanobot.channels.websocket import (
)
from nanobot.config.loader import load_config, save_config
from nanobot.config.schema import Config, ModelPresetConfig
from nanobot.webui.settings_api import settings_payload
from nanobot.webui.settings_api import settings_payload, update_provider_settings
# -- Shared helpers (aligned with test_websocket_integration.py) ---------------
@@ -480,6 +480,72 @@ async def test_send_delta_emits_delta_and_stream_end() -> None:
assert second["stream_id"] == "sid"
@pytest.mark.asyncio
async def test_send_delta_stream_end_rewrites_local_markdown_image(monkeypatch, tmp_path) -> None:
bus = MagicMock()
workspace = tmp_path / "workspace"
workspace.mkdir()
(workspace / "diagram.png").write_bytes(b"\x89PNG\r\n\x1a\nimage")
media = tmp_path / "media"
def fake_media_dir(channel: str | None = None):
path = media / channel if channel else media
path.mkdir(parents=True, exist_ok=True)
return path
monkeypatch.setattr("nanobot.channels.websocket.get_media_dir", fake_media_dir)
channel = WebSocketChannel(
{"enabled": True, "allowFrom": ["*"], "streaming": True},
bus,
workspace_path=workspace,
)
mock_ws = AsyncMock()
channel._attach(mock_ws, "chat-1")
await channel.send_delta("chat-1", "![Diagram](", {"_stream_delta": True, "_stream_id": "sid"})
await channel.send_delta("chat-1", "diagram.png)", {"_stream_delta": True, "_stream_id": "sid"})
await channel.send_delta("chat-1", "", {"_stream_end": True, "_stream_id": "sid"})
assert mock_ws.send.await_count == 3
final = json.loads(mock_ws.send.call_args_list[2][0][0])
assert final["event"] == "stream_end"
assert final["text"].startswith("![Diagram](/api/media/")
@pytest.mark.asyncio
async def test_send_delta_stream_end_rewrites_inline_final_text(monkeypatch, tmp_path) -> None:
bus = MagicMock()
workspace = tmp_path / "workspace"
workspace.mkdir()
(workspace / "diagram.png").write_bytes(b"\x89PNG\r\n\x1a\nimage")
media = tmp_path / "media"
def fake_media_dir(channel: str | None = None):
path = media / channel if channel else media
path.mkdir(parents=True, exist_ok=True)
return path
monkeypatch.setattr("nanobot.channels.websocket.get_media_dir", fake_media_dir)
channel = WebSocketChannel(
{"enabled": True, "allowFrom": ["*"], "streaming": True},
bus,
workspace_path=workspace,
)
mock_ws = AsyncMock()
channel._attach(mock_ws, "chat-1")
await channel.send_delta(
"chat-1",
"![Diagram](diagram.png)",
{"_stream_delta": True, "_stream_end": True, "_stream_id": "sid"},
)
mock_ws.send.assert_awaited_once()
final = json.loads(mock_ws.send.await_args.args[0])
assert final["event"] == "stream_end"
assert final["text"].startswith("![Diagram](/api/media/")
@pytest.mark.asyncio
async def test_send_reasoning_delta_emits_streaming_frame() -> None:
bus = MagicMock()
@@ -1055,6 +1121,7 @@ async def test_settings_api_returns_safe_subset_and_updates_whitelist(
}
assert image_providers["openrouter"]["label"] == "OpenRouter"
assert image_providers["openrouter"]["configured"] is False
assert image_providers["openai_codex"]["configured"] is True
assert image_providers["gemini"]["label"] == "Gemini"
assert body["runtime"]["config_path"] == str(config_path)
workspace_path = body["runtime"]["workspace_path"].replace("\\", "/")
@@ -1121,6 +1188,30 @@ async def test_settings_api_returns_safe_subset_and_updates_whitelist(
)
assert bad_preset.status_code == 400
created_preset = await _http_get(
"http://127.0.0.1:"
f"{port}/api/settings/model-configurations/create"
"?label=Fast%20writing&provider=openai&model=openai%2Fgpt-4.1-mini",
headers={"Authorization": "Bearer tok"},
)
assert created_preset.status_code == 200
created_body = created_preset.json()
assert created_body["agent"]["model_preset"] == "fast-writing"
assert created_body["agent"]["model"] == "openai/gpt-4.1-mini"
created_presets = {
preset["name"]: preset for preset in created_body["model_presets"]
}
assert created_presets["fast-writing"]["label"] == "Fast writing"
assert created_presets["fast-writing"]["provider"] == "openai"
duplicate_preset = await _http_get(
"http://127.0.0.1:"
f"{port}/api/settings/model-configurations/create"
"?label=Fast%20writing&provider=openai&model=openai%2Fgpt-4.1-mini",
headers={"Authorization": "Bearer tok"},
)
assert duplicate_preset.status_code == 409
search_updated = await _http_get(
"http://127.0.0.1:"
f"{port}/api/settings/web-search/update?provider=searxng"
@@ -1188,7 +1279,10 @@ async def test_settings_api_returns_safe_subset_and_updates_whitelist(
saved = load_config(config_path)
assert saved.agents.defaults.model == "atomic_chat/test"
assert saved.agents.defaults.provider == "atomic_chat"
assert saved.agents.defaults.model_preset == "deep"
assert saved.agents.defaults.model_preset == "fast-writing"
assert saved.model_presets["fast-writing"].label == "Fast writing"
assert saved.model_presets["fast-writing"].model == "openai/gpt-4.1-mini"
assert saved.model_presets["fast-writing"].provider == "openai"
assert saved.agents.defaults.timezone == "Asia/Shanghai"
assert saved.agents.defaults.bot_name == "Nano"
assert saved.agents.defaults.bot_icon == "N"
@@ -1257,6 +1351,37 @@ def test_settings_payload_normalizes_camel_case_provider(
assert body["agent"]["provider"] == "minimax_anthropic"
def test_settings_payload_exposes_api_type_only_for_openai(monkeypatch, tmp_path) -> None:
config_path = tmp_path / "config.json"
config = Config()
config.providers.openai.api_type = "responses"
save_config(config, config_path)
monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path)
body = settings_payload()
providers = {provider["name"]: provider for provider in body["providers"]}
assert providers["openai"]["api_type"] == "responses"
assert "api_type" not in providers["custom"]
def test_update_provider_settings_ignores_api_type_for_non_openai(monkeypatch, tmp_path) -> None:
config_path = tmp_path / "config.json"
save_config(Config(), config_path)
monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path)
body = update_provider_settings({
"provider": ["custom"],
"api_base": ["https://example.test/v1"],
"api_type": ["responses"],
})
assert body["providers"]
config = load_config(config_path)
assert config.providers.custom.api_base == "https://example.test/v1"
assert config.providers.custom.api_type == "auto"
@pytest.mark.asyncio
async def test_end_to_end_server_pushes_streaming_deltas_to_client(bus: MagicMock) -> None:
port = 29880
@@ -105,6 +105,43 @@ async def test_message_without_media_backward_compatible() -> None:
assert call.kwargs["media"] is None
@pytest.mark.asyncio
async def test_message_forwards_normalized_cli_app_attachments() -> None:
channel = _make_channel()
mock_conn = AsyncMock()
envelope = {
"type": "message",
"chat_id": "abc123",
"content": "please use @drawio",
"webui": True,
"cli_apps": [
{
"name": "DrawIO",
"display_name": "Draw.io",
"category": "diagram",
"entry_point": "cli-anything-drawio",
"logo_url": "https://example.invalid/drawio.svg",
"brand_color": "#F08705",
},
{"name": "bad name", "entry_point": "nope"},
],
}
await channel._dispatch_envelope(mock_conn, "client-1", envelope)
channel._handle_message.assert_awaited_once()
metadata = channel._handle_message.call_args.kwargs["metadata"]
assert metadata["webui"] is True
assert metadata["cli_apps"] == [{
"name": "drawio",
"display_name": "Draw.io",
"category": "diagram",
"entry_point": "cli-anything-drawio",
"logo_url": "https://example.invalid/drawio.svg",
"brand_color": "#F08705",
}]
@pytest.mark.asyncio
async def test_message_with_single_image_forwards_saved_path(tmp_path) -> None:
channel = _make_channel()
@@ -140,6 +140,225 @@ async def test_sessions_routes_require_bearer_token(
await server_task
@pytest.mark.asyncio
async def test_cli_apps_routes_require_token_and_return_payload(
bus: MagicMock,
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(
"nanobot.channels.websocket.cli_apps_payload",
lambda: {
"apps": [
{
"name": "gimp",
"display_name": "GIMP",
"category": "image",
"description": "Image editing",
"requires": "Python",
"source": "harness",
"entry_point": "cli-anything-gimp",
"install_supported": True,
"installed": False,
"available": False,
"status": "not_installed",
"logo_url": None,
"brand_color": None,
"skill_installed": False,
}
],
"installed_count": 0,
"catalog_updated_at": "2026-04-18",
},
)
monkeypatch.setattr(
"nanobot.channels.websocket.cli_apps_action",
lambda action, query: {
"apps": [],
"installed_count": 1,
"catalog_updated_at": "2026-04-18",
"last_action": {"ok": True, "message": f"{action}:{query['name'][0]}"},
},
)
channel = _ch(bus, session_manager=_seed_session(tmp_path), port=29912)
server_task = asyncio.create_task(channel.start())
await asyncio.sleep(0.3)
try:
deny = await _http_get("http://127.0.0.1:29912/api/settings/cli-apps")
assert deny.status_code == 401
boot = await _http_get("http://127.0.0.1:29912/webui/bootstrap")
token = boot.json()["token"]
auth = {"Authorization": f"Bearer {token}"}
catalog = await _http_get(
"http://127.0.0.1:29912/api/settings/cli-apps",
headers=auth,
)
assert catalog.status_code == 200
assert catalog.json()["apps"][0]["name"] == "gimp"
installed = await _http_get(
"http://127.0.0.1:29912/api/settings/cli-apps/install?name=gimp",
headers=auth,
)
assert installed.status_code == 200
assert installed.json()["last_action"]["message"] == "install:gimp"
finally:
await channel.stop()
await server_task
@pytest.mark.asyncio
async def test_mcp_presets_routes_require_token_and_return_payload(
bus: MagicMock,
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(
"nanobot.webui.mcp_presets_api.mcp_presets_payload",
lambda: {
"presets": [
{
"name": "browserbase",
"display_name": "Browserbase",
"category": "browser",
"description": "Cloud browser automation",
"docs_url": "https://docs.browserbase.com/integrations/mcp/configuration",
"transport": "streamableHttp",
"requires": "Browserbase API key",
"note": "",
"install_supported": True,
"installed": False,
"configured": False,
"available": False,
"status": "not_installed",
"logo_url": None,
"brand_color": "#111827",
"required_fields": [],
"connection_summary": "",
}
],
"installed_count": 0,
},
)
preset_queries: list[tuple[str, dict[str, list[str]]]] = []
custom_queries: list[tuple[str, dict[str, list[str]]]] = []
def _mcp_preset_action(action: str, query: dict[str, list[str]]) -> dict[str, Any]:
preset_queries.append((action, query))
return {
"presets": [],
"installed_count": 1,
"requires_restart": action != "test",
"last_action": {"ok": True, "message": f"{action}:{query['name'][0]}"},
}
def _custom_action(action: str, query: dict[str, list[str]]) -> dict[str, Any]:
custom_queries.append((action, query))
return {
"presets": [],
"installed_count": 1,
"requires_restart": True,
"last_action": {
"ok": True,
"message": f"{action}:{query.get('name', ['config'])[0]}",
},
}
monkeypatch.setattr(
"nanobot.webui.mcp_presets_api.mcp_presets_action",
_mcp_preset_action,
)
monkeypatch.setattr(
"nanobot.webui.mcp_presets_api.custom_mcp_action",
_custom_action,
)
async def _hot_reload(_bus):
return {"ok": True, "message": "MCP config reloaded.", "requires_restart": False}
monkeypatch.setattr(
"nanobot.channels.websocket.request_mcp_reload",
_hot_reload,
)
channel = _ch(bus, session_manager=_seed_session(tmp_path), port=29913)
server_task = asyncio.create_task(channel.start())
await asyncio.sleep(0.3)
try:
deny = await _http_get("http://127.0.0.1:29913/api/settings/mcp-presets")
assert deny.status_code == 401
boot = await _http_get("http://127.0.0.1:29913/webui/bootstrap")
token = boot.json()["token"]
auth = {"Authorization": f"Bearer {token}"}
catalog = await _http_get(
"http://127.0.0.1:29913/api/settings/mcp-presets",
headers=auth,
)
assert catalog.status_code == 200
assert catalog.json()["presets"][0]["name"] == "browserbase"
enabled = await _http_get(
"http://127.0.0.1:29913/api/settings/mcp-presets/enable?name=browserbase",
headers={
**auth,
"X-Nanobot-MCP-Values": json.dumps(
{"browserbase_api_key": "bb_live_secret"}
),
},
)
assert enabled.status_code == 200
assert preset_queries[-1][1]["browserbase_api_key"] == ["bb_live_secret"]
body = enabled.json()
assert "bb_live_secret" not in enabled.text
assert body["last_action"]["message"] == "enable:browserbase MCP config reloaded."
assert body["hot_reload"]["ok"] is True
assert body["restart_required_sections"] == []
bad_header = await _http_get(
"http://127.0.0.1:29913/api/settings/mcp-presets/enable?name=browserbase",
headers={**auth, "X-Nanobot-MCP-Values": "[]"},
)
assert bad_header.status_code == 400
custom = await _http_get(
"http://127.0.0.1:29913/api/settings/mcp-presets/custom",
headers={
**auth,
"X-Nanobot-MCP-Values": json.dumps(
{"name": "docs", "command": "npx"}
),
},
)
assert custom.status_code == 200
assert custom_queries[-1][1]["command"] == ["npx"]
assert custom.json()["last_action"]["message"] == "custom:docs MCP config reloaded."
imported = await _http_get(
"http://127.0.0.1:29913/api/settings/mcp-presets/import",
headers={**auth, "X-Nanobot-MCP-Values": json.dumps({"config": "{}"})},
)
assert imported.status_code == 200
assert imported.json()["last_action"]["message"] == "import:config MCP config reloaded."
tools = await _http_get(
"http://127.0.0.1:29913/api/settings/mcp-presets/tools",
headers={
**auth,
"X-Nanobot-MCP-Values": json.dumps(
{"name": "docs", "enabled_tools": []}
),
},
)
assert tools.status_code == 200
assert tools.json()["last_action"]["message"] == "tools:docs MCP config reloaded."
finally:
await channel.stop()
await server_task
@pytest.mark.asyncio
async def test_sessions_list_only_returns_websocket_sessions_by_default(
bus: MagicMock, tmp_path: Path
@@ -44,6 +44,7 @@ def _ch(
bus: Any,
*,
session_manager: SessionManager | None = None,
workspace_path: Path | None = None,
port: int,
) -> WebSocketChannel:
return WebSocketChannel(
@@ -57,6 +58,7 @@ def _ch(
},
bus,
session_manager=session_manager,
workspace_path=workspace_path,
)
@@ -67,6 +69,15 @@ def bus() -> MagicMock:
return b
def _fake_media_dir(root: Path):
def inner(channel: str | None = None) -> Path:
path = root / channel if channel else root
path.mkdir(parents=True, exist_ok=True)
return path
return inner
async def _http_get(
url: str, headers: dict[str, str] | None = None
) -> httpx.Response:
@@ -123,6 +134,45 @@ def test_sign_media_path_round_trips_via_hmac(
assert _b64url_decode(payload).decode() == "a.png"
def test_local_markdown_image_is_staged_and_rewritten(
bus: MagicMock,
tmp_path: Path,
) -> None:
workspace = tmp_path / "workspace"
workspace.mkdir()
(workspace / "demo_arch.png").write_bytes(_PNG_BYTES)
media = tmp_path / "media"
channel = _ch(bus, workspace_path=workspace, port=0)
with patch("nanobot.channels.websocket.get_media_dir", side_effect=_fake_media_dir(media)):
rewritten = channel._rewrite_local_markdown_images(
"The result:\n![Cloud Architecture Diagram](demo_arch.png)"
)
assert "![Cloud Architecture Diagram](/api/media/" in rewritten
staged = list((media / "websocket").iterdir())
assert len(staged) == 1
assert staged[0].read_bytes() == _PNG_BYTES
def test_local_markdown_image_rejects_workspace_escape(
bus: MagicMock,
tmp_path: Path,
) -> None:
workspace = tmp_path / "workspace"
workspace.mkdir()
outside = tmp_path / "outside.png"
outside.write_bytes(_PNG_BYTES)
media = tmp_path / "media"
channel = _ch(bus, workspace_path=workspace, port=0)
text = "![nope](../outside.png)"
with patch("nanobot.channels.websocket.get_media_dir", side_effect=_fake_media_dir(media)):
assert channel._rewrite_local_markdown_images(text) == text
assert not (media / "websocket").exists()
# ---------------------------------------------------------------------------
# /api/media/<sig>/<payload>: the serving handler
# ---------------------------------------------------------------------------
+526
View File
@@ -1,6 +1,7 @@
import asyncio
import json
import tempfile
import time
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import AsyncMock
@@ -374,6 +375,7 @@ async def test_send_uses_typing_start_and_cancel_when_ticket_available() -> None
channel._client = object()
channel._token = "token"
channel._context_tokens["wx-user"] = "ctx-typing"
channel._context_token_at["wx-user"] = time.time()
channel._send_text = AsyncMock()
channel._api_post = AsyncMock(
side_effect=[
@@ -402,6 +404,7 @@ async def test_send_still_sends_text_when_typing_ticket_missing() -> None:
channel._client = object()
channel._token = "token"
channel._context_tokens["wx-user"] = "ctx-no-ticket"
channel._context_token_at["wx-user"] = time.time()
channel._send_text = AsyncMock()
channel._api_post = AsyncMock(return_value={"ret": 1, "errmsg": "no config"})
@@ -1254,3 +1257,526 @@ async def test_send_text_succeeds_on_zero_errcode() -> None:
await channel._send_text("wx-user", "hello", "ctx-ok")
channel._api_post.assert_awaited_once()
@pytest.mark.asyncio
async def test_send_text_raises_on_nonzero_ret_even_when_errcode_zero() -> None:
"""_send_text must raise when the API returns ret != 0, even if errcode is 0.
The iLink API signals failure through either field. Checking only errcode
caused silent message drops (responses generated but never delivered).
"""
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel._api_post = AsyncMock(
return_value={"ret": -100, "errcode": 0, "errmsg": "internal error"}
)
with pytest.raises(RuntimeError, match="WeChat send text error.*ret=-100.*errcode=0"):
await channel._send_text("wx-user", "hello", "ctx-ok")
channel._api_post.assert_awaited_once()
# ---------------------------------------------------------------------------
# Tests for _poll_once not silently dropping messages on processing errors
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_poll_once_logs_exception_on_process_message_failure(monkeypatch) -> None:
"""When _process_message raises, _poll_once must log the error and continue
processing remaining messages instead of silently swallowing the exception."""
channel, _bus = _make_channel()
channel._client = SimpleNamespace(timeout=None)
channel._token = "token"
channel._get_updates_buf = "old-buf"
calls = []
logged_messages: list[str] = []
async def _failing_process(msg: dict) -> None:
calls.append(msg.get("message_id"))
if msg.get("message_id") == "msg-1":
raise RuntimeError("processing failed")
channel._process_message = _failing_process # type: ignore[method-assign]
monkeypatch.setattr(
channel.logger,
"exception",
lambda message, *args, **kwargs: logged_messages.append(str(message)),
)
channel._api_post = AsyncMock( # type: ignore[method-assign]
return_value={
"ret": 0,
"errcode": 0,
"get_updates_buf": "new-buf",
"msgs": [
{"message_id": "msg-1", "message_type": 1},
{"message_id": "msg-2", "message_type": 1},
],
}
)
await channel._poll_once()
# Both messages should have been attempted
assert calls == ["msg-1", "msg-2"]
# Buffer should still advance (already updated before processing)
assert channel._get_updates_buf == "new-buf"
# Error should be logged
assert any("Failed to process WeChat message" in m for m in logged_messages)
@pytest.mark.asyncio
async def test_poll_loop_logs_exception_and_continues_on_poll_failure(monkeypatch) -> None:
"""When _poll_once raises a non-timeout exception, the start() loop must log
the error and continue polling instead of exiting silently."""
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel.config.token = "token" # skip QR login in start()
channel._running = True
call_count = 0
logged_messages: list[str] = []
async def _failing_poll() -> None:
nonlocal call_count
call_count += 1
if call_count == 1:
raise RuntimeError("poll exploded")
channel._running = False # Stop after second call
channel._poll_once = _failing_poll # type: ignore[method-assign]
monkeypatch.setattr(
channel.logger,
"exception",
lambda message, *args, **kwargs: logged_messages.append(str(message)),
)
# Use a tiny retry delay so the test finishes quickly
original_retry = weixin_mod.RETRY_DELAY_S
weixin_mod.RETRY_DELAY_S = 0.01
try:
await channel.start()
finally:
weixin_mod.RETRY_DELAY_S = original_retry
assert call_count == 2
assert any("WeChat poll loop error" in m for m in logged_messages)
# ---------------------------------------------------------------------------
# Tool-hint buffering
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_buffer_single_tool_hint_not_sent_immediately() -> None:
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel.send_tool_hints = True
channel._context_tokens["wx-user"] = "ctx-1"
channel._context_token_at["wx-user"] = time.time()
channel._send_text = AsyncMock()
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "Using tool",
"media": [],
"metadata": {"_progress": True, "_tool_hint": True},
},
)()
)
channel._send_text.assert_not_awaited()
assert channel._pending_tool_hints["wx-user"] == ["Using tool"]
@pytest.mark.asyncio
async def test_buffer_multiple_tool_hints_flushed_on_final_answer() -> None:
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel.send_tool_hints = True
channel._context_tokens["wx-user"] = "ctx-1"
channel._context_token_at["wx-user"] = time.time()
channel._send_text = AsyncMock()
for hint in ["tool1", "tool2"]:
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": hint,
"media": [],
"metadata": {"_progress": True, "_tool_hint": True},
},
)()
)
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "Done",
"media": [],
"metadata": {},
},
)()
)
assert channel._send_text.await_count == 2
channel._send_text.assert_any_await("wx-user", "tool1\n\ntool2", "ctx-1")
channel._send_text.assert_any_await("wx-user", "Done", "ctx-1")
assert "wx-user" not in channel._pending_tool_hints
@pytest.mark.asyncio
async def test_thought_progress_flushes_tool_hints() -> None:
"""Thoughts are visible progress messages and must act as separators,
flushing buffered tool hints before they are sent."""
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel.send_tool_hints = True
channel._context_tokens["wx-user"] = "ctx-1"
channel._context_token_at["wx-user"] = time.time()
channel._send_text = AsyncMock()
# Buffer a tool hint
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "search 'foo'",
"media": [],
"metadata": {"_progress": True, "_tool_hint": True},
},
)()
)
# Send a thought — progress but not a tool_hint.
# It must act as a separator and flush the buffered hint.
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "Let me think...",
"media": [],
"metadata": {"_progress": True},
},
)()
)
# The buffered hint was flushed before the thought was sent.
channel._send_text.assert_any_await("wx-user", "search 'foo'", "ctx-1")
channel._send_text.assert_any_await("wx-user", "Let me think...", "ctx-1")
assert "wx-user" not in channel._pending_tool_hints
# Final answer arrives with nothing left to flush.
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "Done",
"media": [],
"metadata": {},
},
)()
)
assert channel._send_text.await_count == 3
channel._send_text.assert_any_await("wx-user", "Done", "ctx-1")
@pytest.mark.asyncio
async def test_reasoning_delta_does_not_flush_tool_hints() -> None:
"""Reasoning deltas are invisible in WeChat and must NOT flush buffered
tool hints otherwise hints separated only by hidden reasoning would
fail to coalesce."""
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel.send_tool_hints = True
channel._context_tokens["wx-user"] = "ctx-1"
channel._context_token_at["wx-user"] = time.time()
channel._send_text = AsyncMock()
# Buffer a tool hint
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "search 'foo'",
"media": [],
"metadata": {"_progress": True, "_tool_hint": True},
},
)()
)
# Send a reasoning delta — invisible in WeChat, must NOT flush
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "Thinking step 1...",
"media": [],
"metadata": {"_progress": True, "_reasoning_delta": True},
},
)()
)
# Reasoning is invisible; hint stays buffered, _send_text not called
channel._send_text.assert_not_awaited()
assert channel._pending_tool_hints["wx-user"] == ["search 'foo'"]
# Final answer flushes the buffered hint
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "Done",
"media": [],
"metadata": {},
},
)()
)
channel._send_text.assert_any_await("wx-user", "search 'foo'", "ctx-1")
channel._send_text.assert_any_await("wx-user", "Done", "ctx-1")
assert "wx-user" not in channel._pending_tool_hints
@pytest.mark.asyncio
async def test_empty_progress_message_does_not_flush_tool_hints() -> None:
"""Empty progress messages (e.g. after_iteration tool_events) have no
visible content and must NOT act as separators."""
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel.send_tool_hints = True
channel._context_tokens["wx-user"] = "ctx-1"
channel._context_token_at["wx-user"] = time.time()
channel._send_text = AsyncMock()
# Buffer a tool hint
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "search 'foo'",
"media": [],
"metadata": {"_progress": True, "_tool_hint": True},
},
)()
)
# Send an empty progress message (no content, no media)
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "",
"media": [],
"metadata": {"_progress": True, "_tool_events": [{"phase": "end"}]},
},
)()
)
# Nothing should have been sent yet
channel._send_text.assert_not_awaited()
assert channel._pending_tool_hints["wx-user"] == ["search 'foo'"]
# Final answer flushes the buffered hint
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "Done",
"media": [],
"metadata": {},
},
)()
)
channel._send_text.assert_any_await("wx-user", "search 'foo'", "ctx-1")
channel._send_text.assert_any_await("wx-user", "Done", "ctx-1")
assert "wx-user" not in channel._pending_tool_hints
@pytest.mark.asyncio
async def test_buffer_flush_refreshes_context_token() -> None:
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel.send_tool_hints = True
channel._context_tokens["wx-user"] = "ctx-old"
channel._context_token_at["wx-user"] = time.time()
channel._refresh_context_token_if_stale = AsyncMock(return_value="ctx-refreshed")
channel._send_text = AsyncMock()
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "hint",
"media": [],
"metadata": {"_progress": True, "_tool_hint": True},
},
)()
)
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "Done",
"media": [],
"metadata": {},
},
)()
)
assert channel._refresh_context_token_if_stale.await_count == 2
channel._refresh_context_token_if_stale.assert_any_await("wx-user", "ctx-old")
channel._send_text.assert_any_await("wx-user", "hint", "ctx-refreshed")
@pytest.mark.asyncio
async def test_buffer_flush_failure_does_not_block_final_answer() -> None:
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel.send_tool_hints = True
channel._context_tokens["wx-user"] = "ctx-1"
channel._context_token_at["wx-user"] = time.time()
channel._send_text = AsyncMock(side_effect=[RuntimeError("boom"), None])
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "hint",
"media": [],
"metadata": {"_progress": True, "_tool_hint": True},
},
)()
)
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "Done",
"media": [],
"metadata": {},
},
)()
)
assert channel._send_text.await_count == 2
channel._send_text.assert_any_await("wx-user", "hint", "ctx-1")
channel._send_text.assert_any_await("wx-user", "Done", "ctx-1")
@pytest.mark.asyncio
async def test_buffer_flushed_on_stream_end() -> None:
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel.send_tool_hints = True
channel._context_tokens["wx-user"] = "ctx-1"
channel._context_token_at["wx-user"] = time.time()
channel._send_text = AsyncMock()
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "hint",
"media": [],
"metadata": {"_progress": True, "_tool_hint": True},
},
)()
)
await channel.send_delta("wx-user", "", {"_stream_end": True})
channel._send_text.assert_awaited_once_with("wx-user", "hint", "ctx-1")
assert "wx-user" not in channel._pending_tool_hints
@pytest.mark.asyncio
async def test_stop_clears_buffer() -> None:
channel, _bus = _make_channel()
channel._pending_tool_hints["wx-user"] = ["hint1", "hint2"]
await channel.stop()
assert "wx-user" not in channel._pending_tool_hints
@pytest.mark.asyncio
async def test_send_tool_hints_false_drops_tool_hints() -> None:
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel.send_tool_hints = False
channel._send_text = AsyncMock()
await channel.send(
type(
"Msg",
(),
{
"chat_id": "wx-user",
"content": "hint",
"media": [],
"metadata": {"_progress": True, "_tool_hint": True},
},
)()
)
channel._send_text.assert_not_awaited()
assert "wx-user" not in channel._pending_tool_hints
+1 -199
View File
@@ -11,7 +11,7 @@ from typer.testing import CliRunner
from nanobot.bus.events import OutboundMessage
from nanobot.cli.commands import app
from nanobot.providers.factory import make_provider
from nanobot.config.schema import Config, ModelPresetConfig
from nanobot.config.schema import Config
from nanobot.cron.types import CronJob, CronPayload
from nanobot.providers.factory import ProviderSnapshot
from nanobot.providers.openai_codex_provider import _strip_model_prefix
@@ -226,16 +226,6 @@ def test_config_dump_excludes_oauth_provider_blocks():
assert "openaiCodex" not in providers
assert "githubCopilot" not in providers
assert "xaiOauth" not in providers
def test_config_dump_includes_xai_oauth_when_hosted_search_is_disabled():
config = Config()
config.providers.xai_oauth.x_search.enable = False
providers = config.model_dump(by_alias=True)["providers"]
assert providers["xaiOauth"]["xSearch"]["enable"] is False
def test_provider_logout_openai_codex_removes_local_oauth_files(tmp_path, monkeypatch):
@@ -290,175 +280,6 @@ def test_provider_logout_github_copilot_succeeds_when_no_local_oauth_file(monkey
assert "No local OAuth credentials found for GitHub Copilot" in result.stdout
def test_provider_logout_xai_oauth_removes_local_oauth_files(tmp_path, monkeypatch):
token_path = tmp_path / "auth" / "xai-oauth.json"
lock_path = token_path.with_suffix(".lock")
token_path.parent.mkdir(parents=True, exist_ok=True)
token_path.write_text("{}", encoding="utf-8")
lock_path.write_text("", encoding="utf-8")
monkeypatch.setenv("NANOBOT_HOME", str(tmp_path))
monkeypatch.setattr("nanobot.providers.xai_oauth_provider._keyring_delete", lambda: None)
result = runner.invoke(app, ["provider", "logout", "xai-oauth"])
assert result.exit_code == 0
assert not token_path.exists()
assert not lock_path.exists()
assert "Logged out from xAI Grok OAuth" in result.stdout
def test_provider_logout_xai_oauth_succeeds_when_no_local_oauth_file(monkeypatch, tmp_path):
monkeypatch.setenv("NANOBOT_HOME", str(tmp_path))
monkeypatch.setattr("nanobot.providers.xai_oauth_provider._keyring_delete", lambda: None)
result = runner.invoke(app, ["provider", "logout", "xai-oauth"])
assert result.exit_code == 0
assert "No local OAuth credentials found for xAI Grok OAuth" in result.stdout
def test_provider_login_xai_oauth_forwards_manual_options(monkeypatch):
from nanobot.providers.xai_oauth_provider import XaiOAuthCredential
captured: dict[str, object] = {}
def fake_login_xai_oauth_interactive(**kwargs):
captured.update(kwargs)
return XaiOAuthCredential(access_token="access", account_id="acct", storage="keyring")
monkeypatch.setattr(
"nanobot.providers.xai_oauth_provider.login_xai_oauth_interactive",
fake_login_xai_oauth_interactive,
)
result = runner.invoke(app, ["provider", "login", "xai-oauth", "--no-browser", "--manual-paste"])
assert result.exit_code == 0
assert captured["open_browser"] is False
assert captured["manual_paste"] is True
assert "Authenticated with xAI Grok OAuth" in result.stdout
assert "nanobot config set agents.defaults.provider xai-oauth" in result.stdout
def test_config_set_updates_default_model_selection(tmp_path):
config_path = tmp_path / "config.json"
result = runner.invoke(app, [
"config",
"set",
"--config",
str(config_path),
"agents.defaults.model_preset",
"null",
])
assert result.exit_code == 0
result = runner.invoke(app, [
"config",
"set",
"--config",
str(config_path),
"agents.defaults.provider",
"xai-oauth",
])
assert result.exit_code == 0
result = runner.invoke(app, [
"config",
"set",
"--config",
str(config_path),
"agents.defaults.model",
"xai-oauth/grok-4.3",
])
assert result.exit_code == 0
data = json.loads(config_path.read_text(encoding="utf-8"))
config = Config.model_validate(data)
assert config.agents.defaults.model_preset is None
assert config.agents.defaults.provider == "xai-oauth"
assert config.agents.defaults.model == "xai-oauth/grok-4.3"
def test_config_set_warns_when_model_preset_would_override_selection(tmp_path):
config = Config()
config.agents.defaults.model_preset = "fast"
config.model_presets["fast"] = ModelPresetConfig(
provider="openrouter",
model="openrouter/openai/gpt-4o-mini",
)
config_path = tmp_path / "config.json"
config_path.write_text(json.dumps(config.model_dump(mode="json", by_alias=True)), encoding="utf-8")
result = runner.invoke(app, [
"config",
"set",
"--config",
str(config_path),
"agents.defaults.provider",
"xai-oauth",
])
assert result.exit_code == 0
assert "model_preset is set and may override this" in result.stdout
def test_config_set_disables_xai_oauth_hosted_search(tmp_path):
config_path = tmp_path / "config.json"
result = runner.invoke(app, [
"config",
"set",
"--config",
str(config_path),
"providers.xai_oauth.x_search.enable",
"false",
])
assert result.exit_code == 0
data = json.loads(config_path.read_text(encoding="utf-8"))
assert data["providers"]["xaiOauth"]["xSearch"]["enable"] is False
assert Config.model_validate(data).providers.xai_oauth.x_search.enable is False
def test_config_set_rejects_unknown_path(tmp_path):
result = runner.invoke(app, [
"config",
"set",
"--config",
str(tmp_path / "config.json"),
"agents.defaults.not_a_field",
"value",
])
assert result.exit_code == 1
assert "Could not set config value" in result.stdout
def test_provider_login_xai_oauth_does_not_update_config(monkeypatch, tmp_path):
from nanobot.providers.xai_oauth_provider import XaiOAuthCredential
config = Config()
config.agents.defaults.provider = "auto"
config.agents.defaults.model = "anthropic/claude-opus-4-5"
config_path = tmp_path / "config.json"
monkeypatch.setattr(
"nanobot.providers.xai_oauth_provider.login_xai_oauth_interactive",
lambda **_kwargs: XaiOAuthCredential(access_token="access", account_id="acct", storage="keyring"),
)
monkeypatch.setattr("nanobot.config.loader.get_config_path", lambda: config_path)
monkeypatch.setattr("nanobot.config.loader.load_config", lambda _path=None: config)
save_config = MagicMock()
monkeypatch.setattr("nanobot.config.loader.save_config", save_config)
result = runner.invoke(app, ["provider", "login", "xai-oauth"])
assert result.exit_code == 0
save_config.assert_not_called()
assert "nanobot config set agents.defaults.model xai-oauth/grok-4.3" in result.stdout
def test_provider_logout_rejects_unknown_provider():
result = runner.invoke(app, ["provider", "logout", "not-a-real-provider"])
@@ -577,8 +398,6 @@ def test_find_by_name_accepts_camel_case_and_hyphen_aliases():
assert find_by_name("volcengineCodingPlan").name == "volcengine_coding_plan"
assert find_by_name("github-copilot") is not None
assert find_by_name("github-copilot").name == "github_copilot"
assert find_by_name("xai-oauth") is not None
assert find_by_name("xai-oauth").name == "xai_oauth"
assert find_by_name("longcat") is not None
assert find_by_name("longcat").name == "longcat"
assert find_by_name("atomic-chat") is not None
@@ -721,23 +540,6 @@ def test_make_provider_uses_github_copilot_backend():
assert provider.__class__.__name__ == "GitHubCopilotProvider"
def test_make_provider_uses_xai_oauth_backend():
config = Config.model_validate(
{
"agents": {
"defaults": {
"provider": "xai-oauth",
"model": "xai-oauth/grok-4.3",
}
}
}
)
provider = make_provider(config)
assert provider.__class__.__name__ == "XaiOAuthProvider"
def test_github_copilot_provider_strips_prefixed_model_name():
from nanobot.providers.github_copilot_provider import GitHubCopilotProvider
+575
View File
@@ -0,0 +1,575 @@
from __future__ import annotations
import json
import subprocess
import sys
import time
from pathlib import Path
from types import SimpleNamespace
import pytest
from nanobot.apps.cli.service import CliAppError, CliAppManager, CliAppsRuntimeConfig
def _write_cache(path: Path, registry: dict) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(
json.dumps({"_cached_at": time.time(), "data": registry}),
encoding="utf-8",
)
def _manager(tmp_path: Path) -> CliAppManager:
workspace = tmp_path / "workspace"
workspace.mkdir()
return CliAppManager(
workspace=workspace,
data_dir=tmp_path / "data",
runtime=CliAppsRuntimeConfig(catalog_ttl_seconds=3600, install_timeout=5, run_timeout=5),
)
def _seed_catalog(manager: CliAppManager) -> None:
harness = {
"meta": {"updated": "2026-04-16"},
"clis": [
{
"name": "gimp",
"display_name": "GIMP",
"version": "1.0.0",
"description": "Image editing",
"category": "image",
"requires": "Python 3.10+",
"install_cmd": "pip install cli-anything-gimp",
"entry_point": "cli-anything-gimp",
"skill_md": "skills/cli-anything-gimp/SKILL.md",
}
],
}
public = {
"meta": {"updated": "2026-04-18"},
"clis": [
{
"name": "gimp",
"display_name": "GIMP",
"description": "Public duplicate entry",
},
{
"name": "jimeng",
"display_name": "Jimeng",
"version": "latest",
"description": "Script install",
"category": "ai",
"install_strategy": "script",
"install_cmd": "curl -fsSL https://example.invalid/install.sh | bash",
"entry_point": "dreamina",
},
{
"name": "feishu",
"display_name": "Feishu/Lark CLI",
"version": "latest",
"description": "Official Lark CLI",
"category": "communication",
"package_manager": "npm",
"npm_package": "@larksuite/cli",
"install_cmd": "npm install -g @larksuite/cli",
"entry_point": "lark-cli",
},
{
"name": "dify-workflow",
"display_name": "Dify Workflow",
"version": "latest",
"description": "Run Dify workflows",
"category": "ai",
"install_cmd": "pip install cli-anything-dify-workflow",
"entry_point": "cli-anything-dify-workflow",
},
{
"name": "shopify",
"display_name": "Shopify CLI",
"version": "latest",
"description": "Shopify",
"category": "web",
"package_manager": "npm",
"npm_package": "@shopify/cli",
"install_cmd": "npm install -g @shopify/cli",
"entry_point": "shopify",
},
{
"name": "clibrowser",
"display_name": "clibrowser",
"version": "latest",
"description": "Cargo install",
"category": "web",
"install_cmd": "cargo install --git https://example.invalid/clibrowser.git",
"entry_point": "clibrowser",
},
{
"name": "suno",
"display_name": "Suno CLI",
"version": "latest",
"description": "python3 pip install",
"category": "music",
"package_manager": "pip",
"install_strategy": "command",
"install_cmd": "python3 -m pip install git+https://example.invalid/suno-cli.git",
"uninstall_cmd": "python3 -m pip uninstall -y suno-cli",
"entry_point": "suno",
},
],
}
_write_cache(manager._cache_path("harness"), harness)
_write_cache(manager._cache_path("public"), public)
def test_payload_merges_catalog_and_marks_unsupported_installs(tmp_path: Path) -> None:
manager = _manager(tmp_path)
_seed_catalog(manager)
payload = manager.payload()
assert payload["catalog_updated_at"] == "2026-04-18"
apps = {app["name"]: app for app in payload["apps"]}
assert set(apps) == {
"clibrowser",
"dify-workflow",
"feishu",
"gimp",
"jimeng",
"shopify",
"suno",
}
assert apps["gimp"]["install_supported"] is True
assert apps["gimp"]["source"] == "harness+public"
assert apps["gimp"]["description"] == "Public duplicate entry"
assert apps["clibrowser"]["install_supported"] is False
assert apps["jimeng"]["install_supported"] is False
assert apps["suno"]["install_supported"] is True
assert apps["gimp"]["logo_url"]
gimp_manifest = apps["gimp"]["manifest"]
assert gimp_manifest["schema"] == "agent-app.v1"
assert gimp_manifest["id"] == "gimp"
assert gimp_manifest["source"] == "cli-anything:harness+public"
assert gimp_manifest["capabilities"][0]["type"] == "cli"
assert gimp_manifest["capabilities"][0]["entry_point"] == "cli-anything-gimp"
assert gimp_manifest["install"]["verification"] == ["entry_point_available"]
assert "entry_point_absent" in gimp_manifest["remove"]["verification"]
assert gimp_manifest["trust"]["review_status"] == "catalog_entry"
assert apps["dify-workflow"]["logo_url"] == "https://cdn.simpleicons.org/dify/155EEF"
assert apps["feishu"]["logo_url"] == (
"https://www.google.com/s2/favicons?domain=larksuite.com&sz=64"
)
assert apps["jimeng"]["logo_url"] == "https://cdn.simpleicons.org/bytedance/3C8CFF"
assert apps["clibrowser"]["logo_url"] == (
"https://www.google.com/s2/favicons?domain=github.com/allthingssecurity/clibrowser&sz=64"
)
def test_payload_uses_anygen_official_domain_for_logo(tmp_path: Path) -> None:
manager = _manager(tmp_path)
_write_cache(manager._cache_path("harness"), {"meta": {"updated": "2026-04-16"}, "clis": []})
_write_cache(
manager._cache_path("public"),
{
"meta": {"updated": "2026-04-18"},
"clis": [
{
"name": "anygen",
"display_name": "AnyGen",
"description": "Generate docs, slides, websites and more via AnyGen cloud API",
"category": "generation",
"install_cmd": "pip install cli-anything-anygen",
"entry_point": "cli-anything-anygen",
}
],
},
)
payload = manager.payload()
app = payload["apps"][0]
assert app["name"] == "anygen"
assert app["logo_url"] == "https://www.google.com/s2/favicons?domain=anygen.io&sz=64"
def test_install_dispatches_safe_pip_and_installs_skill(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
_seed_catalog(manager)
calls: list[list[str]] = []
def fake_run(argv: list[str], *, timeout: int) -> subprocess.CompletedProcess[str]:
calls.append(argv)
return subprocess.CompletedProcess(argv, 0, stdout="ok", stderr="")
monkeypatch.setattr(manager, "_run_argv", fake_run)
monkeypatch.setattr(
manager,
"_fetch_skill_content",
lambda app: "---\nname: cli-anything-gimp\ndescription: GIMP\n---\n# GIMP\n",
)
payload = manager.install("gimp")
assert calls == [[sys.executable, "-m", "pip", "install", "cli-anything-gimp"]]
assert payload["last_action"]["ok"] is True
assert payload["last_action"]["installed"] is True
assert "state_recorded" in payload["last_action"]["verification"]
installed = json.loads(manager.installed_path.read_text(encoding="utf-8"))["apps"]
assert installed["gimp"]["entry_point"] == "cli-anything-gimp"
skill = manager.workspace / "skills" / "cli-app-gimp" / "SKILL.md"
assert skill.is_file()
assert 'run_cli_app` tool with `name="gimp"' in skill.read_text(encoding="utf-8")
def test_install_records_entry_point_path_and_pip_distribution(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
_seed_catalog(manager)
resolved = tmp_path / "bin" / "cli-anything-gimp"
resolved.parent.mkdir()
resolved.write_text("#!/bin/sh\n", encoding="utf-8")
monkeypatch.setattr(
manager,
"_run_argv",
lambda argv, *, timeout: subprocess.CompletedProcess(argv, 0, stdout="ok", stderr=""),
)
monkeypatch.setattr(
manager,
"_fetch_skill_content",
lambda app: "---\nname: cli-anything-gimp\ndescription: GIMP\n---\n# GIMP\n",
)
monkeypatch.setattr(
"nanobot.apps.cli.service.shutil.which",
lambda command: str(resolved) if command == "cli-anything-gimp" else None,
)
monkeypatch.setattr(
"nanobot.apps.cli.service.importlib_metadata.distributions",
lambda: [
SimpleNamespace(
entry_points=[
SimpleNamespace(group="console_scripts", name="cli-anything-gimp"),
],
metadata={"Name": "cli-anything-gimp"},
)
],
)
manager.install("gimp")
installed = json.loads(manager.installed_path.read_text(encoding="utf-8"))["apps"]
assert installed["gimp"]["entry_point_path"] == str(resolved)
assert installed["gimp"]["pip_distribution"] == "cli-anything-gimp"
def test_installed_state_writes_atomically_without_temp_leftovers(tmp_path: Path) -> None:
manager = _manager(tmp_path)
manager._save_installed({"gimp": {"entry_point": "cli-anything-gimp"}})
manager._save_installed({"zoom": {"entry_point": "cli-anything-zoom"}})
installed = json.loads(manager.installed_path.read_text(encoding="utf-8"))["apps"]
assert set(installed) == {"zoom"}
assert not list(manager.installed_path.parent.glob(".installed.json.*.tmp"))
def test_fetch_skill_content_rejects_untrusted_urls(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
def fail_get(*args, **kwargs):
raise AssertionError("untrusted skill URL should not be fetched")
monkeypatch.setattr("nanobot.apps.cli.service.httpx.get", fail_get)
assert manager._fetch_skill_content({
"name": "evil",
"skill_md": "https://example.com/SKILL.md",
}) is None
assert manager._fetch_skill_content({
"name": "evil",
"skill_md": "skills/../evil/SKILL.md",
}) is None
def test_fetch_skill_content_allows_cli_anything_raw_skill_url(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
seen: list[str] = []
class Response:
text = "---\nname: cli-app-test\ndescription: Test\n---\n# Test\n"
@staticmethod
def raise_for_status() -> None:
return None
def fake_get(url: str, **kwargs):
seen.append(url)
return Response()
monkeypatch.setattr("nanobot.apps.cli.service.httpx.get", fake_get)
content = manager._fetch_skill_content({
"name": "gimp",
"skill_md": "https://raw.githubusercontent.com/HKUDS/CLI-Anything/main/skills/cli-anything-gimp/SKILL.md",
})
assert content and "# Test" in content
assert seen == [
"https://raw.githubusercontent.com/HKUDS/CLI-Anything/main/skills/cli-anything-gimp/SKILL.md"
]
def test_uninstall_removes_installed_state_and_generated_skill(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
_seed_catalog(manager)
manager._save_installed({"gimp": {"entry_point": "cli-anything-gimp"}})
skill_dir = manager.workspace / "skills" / "cli-app-gimp"
skill_dir.mkdir(parents=True)
(skill_dir / "SKILL.md").write_text("# GIMP\n", encoding="utf-8")
monkeypatch.setattr(
manager,
"_run_argv",
lambda argv, *, timeout: subprocess.CompletedProcess(argv, 0, stdout="ok", stderr=""),
)
payload = manager.uninstall("gimp")
assert payload["last_action"]["ok"] is True
assert "gimp" not in json.loads(manager.installed_path.read_text(encoding="utf-8"))["apps"]
assert not skill_dir.exists()
def test_uninstall_uses_safe_python_m_pip_uninstall_command(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
_seed_catalog(manager)
manager._save_installed({"suno": {"entry_point": "suno"}})
calls: list[list[str]] = []
def fake_run(argv: list[str], *, timeout: int) -> subprocess.CompletedProcess[str]:
calls.append(argv)
return subprocess.CompletedProcess(argv, 0, stdout="ok", stderr="")
monkeypatch.setattr(manager, "_run_argv", fake_run)
payload = manager.uninstall("suno")
assert calls == [[sys.executable, "-m", "pip", "uninstall", "-y", "suno-cli"]]
assert payload["last_action"]["ok"] is True
def test_uninstall_uses_recorded_pip_distribution(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
_seed_catalog(manager)
manager._save_installed({
"gimp": {
"entry_point": "cli-anything-gimp",
"pip_distribution": "actual-dist-name",
"entry_point_path": str(tmp_path / "bin" / "cli-anything-gimp"),
}
})
calls: list[list[str]] = []
def fake_run(argv: list[str], *, timeout: int) -> subprocess.CompletedProcess[str]:
calls.append(argv)
return subprocess.CompletedProcess(argv, 0, stdout="ok", stderr="")
monkeypatch.setattr(manager, "_run_argv", fake_run)
payload = manager.uninstall("gimp")
assert calls == [[sys.executable, "-m", "pip", "uninstall", "-y", "actual-dist-name"]]
assert payload["last_action"]["ok"] is True
assert payload["last_action"]["removed"] is True
assert "entry_point_absent" in payload["last_action"]["verification"]
assert "gimp" not in json.loads(manager.installed_path.read_text(encoding="utf-8"))["apps"]
def test_uninstall_keeps_state_when_entry_point_still_available(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
_seed_catalog(manager)
manager._save_installed({"gimp": {"entry_point": "cli-anything-gimp"}})
monkeypatch.setattr(
manager,
"_run_argv",
lambda argv, *, timeout: subprocess.CompletedProcess(argv, 0, stdout="ok", stderr=""),
)
monkeypatch.setattr(
"nanobot.apps.cli.service.shutil.which",
lambda command: "/usr/local/bin/cli-anything-gimp" if command == "cli-anything-gimp" else None,
)
payload = manager.uninstall("gimp")
assert payload["last_action"]["ok"] is False
assert payload["last_action"]["removed"] is False
assert payload["last_action"]["still_available"] is True
assert payload["last_action"]["verification_failed"] == ["entry_point_absent"]
assert "kept it installed" in payload["last_action"]["message"]
assert "gimp" in json.loads(manager.installed_path.read_text(encoding="utf-8"))["apps"]
def test_uninstall_keeps_state_when_recorded_entry_point_still_exists(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
_seed_catalog(manager)
resolved = tmp_path / "bin" / "cli-anything-gimp"
resolved.parent.mkdir()
resolved.write_text("#!/bin/sh\n", encoding="utf-8")
manager._save_installed({
"gimp": {
"entry_point": "cli-anything-gimp",
"entry_point_path": str(resolved),
}
})
monkeypatch.setattr(
manager,
"_run_argv",
lambda argv, *, timeout: subprocess.CompletedProcess(argv, 0, stdout="ok", stderr=""),
)
payload = manager.uninstall("gimp")
assert payload["last_action"]["ok"] is False
assert str(resolved) in payload["last_action"]["message"]
assert "gimp" in json.loads(manager.installed_path.read_text(encoding="utf-8"))["apps"]
def test_mentioned_installed_apps_only_returns_installed_mentions(tmp_path: Path) -> None:
manager = _manager(tmp_path)
manager._save_installed(
{
"gimp": {"entry_point": "cli-anything-gimp", "source": "harness"},
"zoom": {"entry_point": "cli-anything-zoom", "source": "public"},
}
)
mentions = manager.mentioned_installed_apps("use @zoom and @krita, then @GIMP")
assert mentions == [
{
"name": "zoom",
"entry_point": "cli-anything-zoom",
"source": "public",
"skill": "skills/cli-app-zoom/SKILL.md",
"tool": "run_cli_app",
},
{
"name": "gimp",
"entry_point": "cli-anything-gimp",
"source": "harness",
"skill": "skills/cli-app-gimp/SKILL.md",
"tool": "run_cli_app",
},
]
def test_install_rejects_unknown_and_script_strategy(tmp_path: Path) -> None:
manager = _manager(tmp_path)
_seed_catalog(manager)
with pytest.raises(CliAppError, match="not found"):
manager.install("missing")
with pytest.raises(CliAppError, match="unsupported"):
manager.install("jimeng")
def test_run_installed_cli_uses_argv_without_shell(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
_seed_catalog(manager)
resolved = str(tmp_path / "bin" / "cli-anything-gimp")
monkeypatch.setattr(
"nanobot.apps.cli.service.shutil.which",
lambda entry: resolved if entry == "cli-anything-gimp" else None,
)
def fake_run(argv: list[str], **kwargs: object) -> subprocess.CompletedProcess[str]:
assert "shell" not in kwargs or kwargs["shell"] is False
return subprocess.CompletedProcess(
argv,
0,
stdout="ARGS=" + repr(argv[1:]),
stderr="",
)
monkeypatch.setattr("nanobot.apps.cli.service.subprocess.run", fake_run)
manager._save_installed(
{
"gimp": {
"version": "1.0.0",
"entry_point": "cli-anything-gimp",
"source": "harness",
"strategy": "pip",
}
}
)
result = manager.run("gimp", ["project", "list"], json_output=True)
assert "CLI app 'gimp' exited 0" in result
assert "['--json', 'project', 'list']" in result
def test_run_reports_created_artifacts(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
_seed_catalog(manager)
resolved = str(tmp_path / "bin" / "cli-anything-gimp")
monkeypatch.setattr(
"nanobot.apps.cli.service.shutil.which",
lambda entry: resolved if entry == "cli-anything-gimp" else None,
)
def fake_run(argv: list[str], **kwargs: object) -> subprocess.CompletedProcess[str]:
cwd = Path(str(kwargs["cwd"]))
(cwd / "diagram.png").write_bytes(b"\x89PNG\r\n\x1a\nimage")
return subprocess.CompletedProcess(argv, 0, stdout="done", stderr="")
monkeypatch.setattr("nanobot.apps.cli.service.subprocess.run", fake_run)
manager._save_installed({"gimp": {"entry_point": "cli-anything-gimp"}})
result = manager.run("gimp", ["render"])
assert "Artifacts created or updated:" in result
assert "diagram.png (previewable image" in result
assert "![diagram](diagram.png)" in result
def test_run_blocks_working_dir_outside_workspace(tmp_path: Path) -> None:
manager = _manager(tmp_path)
_seed_catalog(manager)
manager._save_installed({"gimp": {"entry_point": "cli-anything-gimp"}})
with pytest.raises(CliAppError, match="outside the configured workspace"):
manager.run("gimp", working_dir="/etc", restrict_to_workspace=True)
+125
View File
@@ -0,0 +1,125 @@
from __future__ import annotations
import asyncio
import json
import subprocess
import time
from pathlib import Path
from nanobot.agent.tools.cli_apps import CliAppsTool
from nanobot.apps.cli.service import CliAppManager, CliAppsRuntimeConfig
def _write_cache(path: Path, registry: dict) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(
json.dumps({"_cached_at": time.time(), "data": registry}),
encoding="utf-8",
)
def test_run_cli_app_uses_installed_registry_app(
tmp_path: Path,
monkeypatch,
) -> None:
workspace = tmp_path / "workspace"
workspace.mkdir()
data_dir = tmp_path / "data"
registry = {
"meta": {"updated": "2026-04-16"},
"clis": [
{
"name": "gimp",
"display_name": "GIMP",
"version": "1.0.0",
"description": "Image editing",
"category": "image",
"install_cmd": "pip install cli-anything-gimp",
"entry_point": "cli-anything-gimp",
}
],
}
_write_cache(data_dir / "harness_registry_cache.json", registry)
_write_cache(data_dir / "public_registry_cache.json", {"meta": {}, "clis": []})
CliAppManager(workspace=workspace, data_dir=data_dir)._save_installed(
{"gimp": {"entry_point": "cli-anything-gimp"}}
)
resolved = str(tmp_path / "bin" / "cli-anything-gimp")
monkeypatch.setattr(
"nanobot.apps.cli.service.shutil.which",
lambda entry: resolved if entry == "cli-anything-gimp" else None,
)
def fake_run(argv: list[str], **kwargs: object) -> subprocess.CompletedProcess[str]:
assert "shell" not in kwargs or kwargs["shell"] is False
return subprocess.CompletedProcess(
argv,
0,
stdout="tool:" + " ".join(argv[1:]),
stderr="",
)
monkeypatch.setattr("nanobot.apps.cli.service.subprocess.run", fake_run)
monkeypatch.setattr("nanobot.apps.cli.service.get_runtime_subdir", lambda _name: data_dir)
tool = CliAppsTool(
workspace=workspace,
restrict_to_workspace=True,
runtime=CliAppsRuntimeConfig(run_timeout=5),
)
assert tool.name == "run_cli_app"
result = asyncio.run(
tool.execute(
name="gimp",
args=["project", "list"],
json=True,
working_dir=str(workspace),
)
)
assert "CLI app 'gimp' exited 0" in result
assert "tool:--json project list" in result
def test_run_cli_app_rejects_uninstalled_app(tmp_path: Path, monkeypatch) -> None:
workspace = tmp_path / "workspace"
workspace.mkdir()
data_dir = tmp_path / "data"
registry = {
"meta": {"updated": "2026-04-16"},
"clis": [
{
"name": "gimp",
"display_name": "GIMP",
"version": "1.0.0",
"description": "Image editing",
"category": "image",
"install_cmd": "pip install cli-anything-gimp",
"entry_point": "cli-anything-gimp",
}
],
}
_write_cache(data_dir / "harness_registry_cache.json", registry)
_write_cache(data_dir / "public_registry_cache.json", {"meta": {}, "clis": []})
monkeypatch.setattr("nanobot.apps.cli.service.get_runtime_subdir", lambda _name: data_dir)
tool = CliAppsTool(workspace=workspace, restrict_to_workspace=True)
result = asyncio.run(tool.execute(name="gimp"))
assert "not installed" in result
def test_run_cli_app_description_names_only_settings_installed_apps(tmp_path: Path, monkeypatch) -> None:
workspace = tmp_path / "workspace"
workspace.mkdir()
data_dir = tmp_path / "data"
CliAppManager(workspace=workspace, data_dir=data_dir)._save_installed(
{"drawio": {"entry_point": "cli-anything-drawio"}}
)
monkeypatch.setattr("nanobot.apps.cli.service.get_runtime_subdir", lambda _name: data_dir)
tool = CliAppsTool(workspace=workspace)
assert "Settings CLI Apps: drawio" in tool.description
assert "ordinary system CLIs such as git, gh" in tool.description
+64
View File
@@ -0,0 +1,64 @@
"""Tests for CLI Apps loop helpers."""
from types import SimpleNamespace
from nanobot.apps.cli.service import CliAppManager
from nanobot.apps.cli.utils import runtime_lines, session_extra
def test_session_extra_returns_cli_apps_only_when_present() -> None:
cli_apps = [{"name": "zoom"}]
assert session_extra({"cli_apps": cli_apps}) == {"cli_apps": cli_apps}
assert session_extra({}) == {}
assert session_extra(None) == {}
def test_cli_app_mentions_inject_runtime_metadata(tmp_path, monkeypatch):
data_dir = tmp_path / "data"
monkeypatch.setattr("nanobot.apps.cli.service.get_runtime_subdir", lambda _name: data_dir)
manager = CliAppManager(workspace=tmp_path)
manager._save_installed(
{
"zoom": {
"entry_point": "cli-anything-zoom",
"source": "harness",
},
"krita": {
"entry_point": "cli-anything-krita",
"source": "harness",
},
}
)
lines = runtime_lines(
SimpleNamespace(content="please use @zoom tonight; ignore @krita?", metadata={}),
tmp_path,
)
joined = "\n".join(lines)
assert "CLI App Mention: @zoom" in joined
assert "tool=run_cli_app" in joined
assert "entry_point=cli-anything-zoom" in joined
assert "skill=skills/cli-app-zoom/SKILL.md" in joined
def test_structured_cli_app_attachment_injects_runtime_metadata(tmp_path):
lines = runtime_lines(
SimpleNamespace(
content="please use @zoom tonight",
metadata={
"cli_apps": [{
"name": "zoom",
"entry_point": "cli-anything-zoom",
"display_name": "Zoom",
}],
},
),
tmp_path,
)
joined = "\n".join(lines)
assert "CLI App Attachment: @zoom" in joined
assert "tool=run_cli_app" in joined
assert "entry_point=cli-anything-zoom" in joined
assert "skill=skills/cli-app-zoom/SKILL.md" in joined
+46
View File
@@ -113,6 +113,52 @@ async def test_dream_restore_lists_versions_with_next_steps() -> None:
assert "Restore a version with `/dream-restore <sha>`." in out.content
@pytest.mark.asyncio
async def test_dream_log_shows_summary_and_analysis() -> None:
commit = CommitInfo(
sha="abcd1234",
message="dream: 2026-04-04, 2 change(s)\n\n[ADD] fact A →USER\n[REMOVE] old fact",
timestamp="2026-04-04 12:00",
)
diff = (
"diff --git a/SOUL.md b/SOUL.md\n"
"--- a/SOUL.md\n"
"+++ b/SOUL.md\n"
"@@ -1 +1 @@\n"
"-old\n"
"+new\n"
)
git = _FakeGit(commits=[commit], diff_map={commit.sha: (commit, diff)})
out = await cmd_dream_log(_make_ctx("/dream-log", git))
assert "## Dream Update" in out.content
assert "- Summary: dream: 2026-04-04, 2 change(s)" in out.content
assert "### Analysis" in out.content
assert "[ADD] fact A →USER" in out.content
assert "[REMOVE] old fact" in out.content
@pytest.mark.asyncio
async def test_dream_log_with_empty_commit_message() -> None:
commit = CommitInfo(sha="abcd1234", message="", timestamp="2026-04-04 12:00")
diff = (
"diff --git a/SOUL.md b/SOUL.md\n"
"--- a/SOUL.md\n"
"+++ b/SOUL.md\n"
"@@ -1 +1 @@\n"
"-old\n"
"+new\n"
)
git = _FakeGit(commits=[commit], diff_map={commit.sha: (commit, diff)})
out = await cmd_dream_log(_make_ctx("/dream-log", git))
assert "## Dream Update" in out.content
assert "- Summary:" not in out.content
assert "### Analysis" not in out.content
@pytest.mark.asyncio
async def test_dream_restore_success_mentions_files_and_followup() -> None:
commit = CommitInfo(sha="abcd1234", message="dream: latest", timestamp="2026-04-04 12:00")
+53
View File
@@ -1,3 +1,5 @@
import pytest
from nanobot.config.schema import Config
@@ -12,6 +14,40 @@ def test_resolve_preset_returns_defaults_when_no_preset() -> None:
assert resolved.reasoning_effort == config.agents.defaults.reasoning_effort
def test_provider_api_type_accepts_exact_values_only() -> None:
config = Config.model_validate({
"providers": {
"openai": {
"apiKey": "sk-test",
"apiType": "responses",
}
}
})
assert config.providers.openai.api_type == "responses"
with pytest.raises(ValueError):
Config.model_validate({
"providers": {
"openai": {
"apiKey": "sk-test",
"apiType": "response",
}
}
})
def test_provider_api_type_is_openai_only() -> None:
with pytest.raises(ValueError, match="only supported"):
Config.model_validate({
"providers": {
"custom": {
"apiBase": "https://example.test/v1",
"apiType": "responses",
}
}
})
def test_legacy_defaults_config_without_presets_still_resolves() -> None:
config = Config.model_validate({
"agents": {
@@ -192,3 +228,20 @@ def test_match_provider_uses_preset_provider_when_forced() -> None:
})
name = config.get_provider_name()
assert name == "anthropic"
def test_match_provider_routes_forced_novita_model_api_models() -> None:
config = Config.model_validate({
"providers": {
"novita": {"apiKey": "sk-test"},
},
"agents": {
"defaults": {
"model": "deepseek-v4-pro",
"provider": "novita",
}
},
})
assert config.get_provider_name() == "novita"
assert config.get_api_base() == "https://api.novita.ai/openai"
+29
View File
@@ -56,6 +56,35 @@ def test_custom_provider_parse_chunks_accepts_plain_text_chunks() -> None:
assert result.content == "hello world"
def test_custom_provider_parse_chunks_deduplicates_parallel_tool_call_ids() -> None:
chunks = [{
"choices": [{
"finish_reason": "tool_calls",
"delta": {
"tool_calls": [
{
"index": 0,
"id": "call_dup",
"function": {"name": "read_file", "arguments": '{"path":"a.txt"}'},
},
{
"index": 1,
"id": "call_dup",
"function": {"name": "read_file", "arguments": '{"path":"b.txt"}'},
},
],
},
}],
}]
result = OpenAICompatProvider._parse_chunks(chunks)
ids = [tool_call.id for tool_call in result.tool_calls or []]
assert ids[0] == "call_dup"
assert len(ids) == 2
assert len(set(ids)) == 2
def test_local_provider_502_error_includes_reachability_hint() -> None:
spec = find_by_name("ollama")
with patch("nanobot.providers.openai_compat_provider.AsyncOpenAI"):
+81
View File
@@ -9,6 +9,7 @@ from nanobot.providers.openai_compat_provider import (
OpenAICompatProvider,
_deep_merge,
)
from nanobot.providers.registry import find_by_name
# ---------------------------------------------------------------------------
# _deep_merge unit tests
@@ -185,6 +186,86 @@ class TestBuildKwargsExtraBody:
assert kwargs["extra_body"]["repetition_penalty"] == 1.15
class TestBuildResponsesBodyExtraBody:
"""Verify extra_body flows into Responses API request bodies."""
def test_responses_extra_body_merges_top_level_fields(self) -> None:
provider = OpenAICompatProvider(
api_key="test-key",
default_model="gpt-5",
spec=find_by_name("openai"),
extra_body={
"metadata": {"source": "test"},
"parallel_tool_calls": False,
},
)
body = provider._build_responses_body(
messages=_simple_messages(),
tools=None, model=None, max_tokens=100,
temperature=0.1, reasoning_effort=None, tool_choice=None,
)
assert body["metadata"] == {"source": "test"}
assert body["parallel_tool_calls"] is False
def test_responses_extra_body_appends_tools(self) -> None:
provider = OpenAICompatProvider(
api_key="test-key",
default_model="gpt-5",
spec=find_by_name("openai"),
extra_body={"tools": [{"type": "web_search"}]},
)
body = provider._build_responses_body(
messages=_simple_messages(),
tools=[{
"type": "function",
"function": {
"name": "read_file",
"description": "Read a file",
"parameters": {"type": "object"},
},
}],
model=None, max_tokens=100, temperature=0.1,
reasoning_effort=None, tool_choice=None,
)
assert body["tools"] == [
{
"type": "function",
"name": "read_file",
"description": "Read a file",
"parameters": {"type": "object"},
},
{"type": "web_search"},
]
def test_responses_extra_body_merges_include_without_duplicates(self) -> None:
provider = OpenAICompatProvider(
api_key="test-key",
default_model="gpt-5",
spec=find_by_name("openai"),
extra_body={
"include": [
"reasoning.encrypted_content",
"web_search_call.action.sources",
],
},
)
body = provider._build_responses_body(
messages=_simple_messages(),
tools=None, model=None, max_tokens=100,
temperature=0.1, reasoning_effort="high", tool_choice=None,
)
assert body["include"] == [
"reasoning.encrypted_content",
"web_search_call.action.sources",
]
# ---------------------------------------------------------------------------
# Schema validation
# ---------------------------------------------------------------------------
@@ -20,6 +20,7 @@ def _make_copilot_provider() -> OpenAICompatProvider:
p.default_model = "github_copilot/gpt-5.4-mini"
p._spec = find_by_name("github_copilot")
p._effective_base = "https://api.githubcopilot.com"
p._api_type = "auto"
p._responses_failures = {}
p._responses_tripped_at = {}
return p
+612
View File
@@ -9,12 +9,16 @@ import pytest
from nanobot.providers.image_generation import (
AIHubMixImageGenerationClient,
CodexImageGenerationClient,
GeminiImageGenerationClient,
GeneratedImageResponse,
ImageGenerationError,
MiniMaxImageGenerationClient,
OllamaImageGenerationClient,
OpenAIImageGenerationClient,
OpenRouterImageGenerationClient,
StepFunImageGenerationClient,
ZhipuImageGenerationClient,
)
PNG_BYTES = (
@@ -36,12 +40,14 @@ class FakeResponse:
payload: dict[str, Any],
status_code: int = 200,
content: bytes = b"",
sse_lines: list[str] | None = None,
) -> None:
self._payload = payload
self.status_code = status_code
self.text = str(payload)
self.content = content
self.request = httpx.Request("POST", "https://openrouter.ai/api/v1/chat/completions")
self._sse_lines = sse_lines
def json(self) -> dict[str, Any]:
return self._payload
@@ -51,6 +57,15 @@ class FakeResponse:
response = httpx.Response(self.status_code, request=self.request, text=self.text)
raise httpx.HTTPStatusError("failed", request=self.request, response=response)
async def aiter_lines(self):
if self._sse_lines is not None:
for line in self._sse_lines:
yield line
return
# Fallback: treat response text as SSE lines
for line in self.text.split("\n"):
yield line
class FakeClient:
def __init__(self, response: FakeResponse) -> None:
@@ -133,6 +148,54 @@ async def test_openrouter_image_generation_requires_api_key() -> None:
await client.generate(prompt="draw", model="model")
@pytest.mark.asyncio
async def test_ollama_image_generation_payload_and_response() -> None:
raw_b64 = PNG_DATA_URL.removeprefix("data:image/png;base64,")
fake = FakeClient(FakeResponse({"image": raw_b64}))
client = OllamaImageGenerationClient(
api_key="ollama-test",
api_base="http://localhost:11434/v1/",
extra_headers={"X-Test": "1"},
extra_body={"seed": 123},
client=fake, # type: ignore[arg-type]
)
response = await client.generate(
prompt="a sunset",
model="x/z-image-turbo",
aspect_ratio="16:9",
image_size="1K",
)
assert response.images == [PNG_DATA_URL]
assert response.content == ""
call = fake.calls[0]
assert call["url"] == "http://localhost:11434/api/generate"
assert call["headers"]["Authorization"] == "Bearer ollama-test"
assert call["headers"]["X-Test"] == "1"
body = call["json"]
assert body["model"] == "x/z-image-turbo"
assert body["prompt"] == "a sunset"
assert body["width"] == 1024
assert body["height"] == 576
assert body["steps"] == 0
assert body["stream"] is False
assert body["seed"] == 123
@pytest.mark.asyncio
async def test_ollama_image_generation_rejects_reference_images() -> None:
client = OllamaImageGenerationClient(api_key=None)
with pytest.raises(ImageGenerationError, match="reference images"):
await client.generate(
prompt="edit this",
model="x/z-image-turbo",
reference_images=["ref.png"],
)
@pytest.mark.asyncio
async def test_aihubmix_image_generation_payload_and_response() -> None:
raw_b64 = PNG_DATA_URL.removeprefix("data:image/png;base64,")
@@ -348,6 +411,11 @@ async def test_gemini_requires_api_key() -> None:
await client.generate(prompt="draw", model="imagen-4.0-generate-001")
def test_gemini_image_client_uses_native_api_base_by_default() -> None:
client = GeminiImageGenerationClient(api_key="AIza-test")
assert client.api_base == "https://generativelanguage.googleapis.com/v1beta"
@pytest.mark.asyncio
async def test_gemini_no_images_raises() -> None:
fake = FakeClient(FakeResponse({"candidates": [{"content": {"parts": [{"text": "sorry"}]}}]}))
@@ -390,6 +458,17 @@ async def test_minimax_payload_and_response_with_reference_image(tmp_path: Path)
assert body["subject_reference"][0]["image_file"].startswith("data:image/png;base64,")
@pytest.mark.asyncio
async def test_minimax_base64_response_uses_detected_mime() -> None:
raw_b64 = base64.b64encode(JPEG_BYTES).decode("ascii")
fake = FakeClient(FakeResponse({"data": {"image_base64": [raw_b64]}}))
client = MiniMaxImageGenerationClient(api_key="sk-mm-test", client=fake) # type: ignore[arg-type]
response = await client.generate(prompt="draw", model="image-01")
assert response.images == [f"data:image/jpeg;base64,{raw_b64}"]
# ---------------------------------------------------------------------------
# StepFun (阶跃星辰)
# ---------------------------------------------------------------------------
@@ -515,3 +594,536 @@ async def test_stepfun_no_images_raises() -> None:
with pytest.raises(ImageGenerationError, match="returned no images"):
await client.generate(prompt="draw", model="step-image-edit-2")
# ---------------------------------------------------------------------------
# OpenAI
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_openai_payload_and_response() -> None:
fake = FakeClient(FakeResponse({"data": [{"b64_json": RAW_B64}]}))
client = OpenAIImageGenerationClient(
api_key="sk-openai-test",
api_base="https://api.openai.com/v1",
extra_headers={"X-Test": "1"},
client=fake, # type: ignore[arg-type]
)
response = await client.generate(
prompt="a cat on the moon",
model="dall-e-3",
aspect_ratio="16:9",
)
assert response.images == [PNG_DATA_URL]
call = fake.calls[0]
assert call["url"] == "https://api.openai.com/v1/images/generations"
assert call["headers"]["Authorization"] == "Bearer sk-openai-test"
assert call["headers"]["X-Test"] == "1"
body = call["json"]
assert body["model"] == "dall-e-3"
assert body["prompt"] == "a cat on the moon"
assert body["response_format"] == "b64_json"
assert body["n"] == 1
assert body["size"] == "1792x1024"
@pytest.mark.asyncio
async def test_openai_b64_json_response_uses_detected_mime() -> None:
raw_b64 = base64.b64encode(JPEG_BYTES).decode("ascii")
fake = FakeClient(FakeResponse({"data": [{"b64_json": raw_b64}]}))
client = OpenAIImageGenerationClient(
api_key="sk-openai-test",
client=fake, # type: ignore[arg-type]
)
response = await client.generate(prompt="draw", model="dall-e-3")
assert response.images == [f"data:image/jpeg;base64,{raw_b64}"]
@pytest.mark.asyncio
async def test_openai_url_download_fallback() -> None:
fake = FakeClient(FakeResponse({"data": [{"url": "https://cdn.example/image.png"}]}))
fake.get_response = FakeResponse({}, content=PNG_BYTES)
client = OpenAIImageGenerationClient(
api_key="sk-openai-test",
client=fake, # type: ignore[arg-type]
)
response = await client.generate(prompt="draw", model="dall-e-3")
assert response.images[0].startswith("data:image/png;base64,")
assert fake.get_calls[0]["url"] == "https://cdn.example/image.png"
@pytest.mark.asyncio
async def test_openai_multiple_images() -> None:
fake = FakeClient(FakeResponse({
"data": [
{"b64_json": RAW_B64},
{"b64_json": RAW_B64},
]
}))
client = OpenAIImageGenerationClient(
api_key="sk-openai-test",
client=fake, # type: ignore[arg-type]
)
response = await client.generate(prompt="draw", model="dall-e-3")
assert len(response.images) == 2
assert response.images == [PNG_DATA_URL, PNG_DATA_URL]
@pytest.mark.asyncio
async def test_openai_aspect_ratio_to_size() -> None:
fake = FakeClient(FakeResponse({"data": [{"b64_json": RAW_B64}]}))
client = OpenAIImageGenerationClient(
api_key="sk-openai-test",
client=fake, # type: ignore[arg-type]
)
await client.generate(prompt="draw", model="dall-e-3", aspect_ratio="1:1")
assert fake.calls[0]["json"]["size"] == "1024x1024"
@pytest.mark.asyncio
async def test_openai_dalle3_uses_supported_orientation_sizes() -> None:
fake = FakeClient(FakeResponse({"data": [{"b64_json": RAW_B64}]}))
client = OpenAIImageGenerationClient(
api_key="sk-openai-test",
client=fake, # type: ignore[arg-type]
)
await client.generate(prompt="draw", model="dall-e-3", aspect_ratio="3:4")
await client.generate(prompt="draw", model="dall-e-3", aspect_ratio="4:3")
assert fake.calls[0]["json"]["size"] == "1024x1792"
assert fake.calls[1]["json"]["size"] == "1792x1024"
@pytest.mark.asyncio
async def test_openai_dalle2_uses_square_size_for_non_square_ratios() -> None:
fake = FakeClient(FakeResponse({"data": [{"b64_json": RAW_B64}]}))
client = OpenAIImageGenerationClient(
api_key="sk-openai-test",
client=fake, # type: ignore[arg-type]
)
await client.generate(prompt="draw", model="dall-e-2", aspect_ratio="16:9")
assert fake.calls[0]["json"]["size"] == "1024x1024"
@pytest.mark.asyncio
async def test_openai_gpt_image_uses_supported_landscape_size() -> None:
fake = FakeClient(FakeResponse({"data": [{"b64_json": RAW_B64}]}))
client = OpenAIImageGenerationClient(
api_key="sk-openai-test",
client=fake, # type: ignore[arg-type]
)
await client.generate(prompt="draw", model="gpt-image-1", aspect_ratio="16:9")
assert fake.calls[0]["json"]["size"] == "1536x1024"
@pytest.mark.asyncio
async def test_openai_gpt_image_uses_supported_orientation_sizes() -> None:
fake = FakeClient(FakeResponse({"data": [{"b64_json": RAW_B64}]}))
client = OpenAIImageGenerationClient(
api_key="sk-openai-test",
client=fake, # type: ignore[arg-type]
)
await client.generate(prompt="draw", model="gpt-image-1", aspect_ratio="3:4")
await client.generate(prompt="draw", model="gpt-image-1", aspect_ratio="4:3")
assert fake.calls[0]["json"]["size"] == "1024x1536"
assert fake.calls[1]["json"]["size"] == "1536x1024"
@pytest.mark.asyncio
async def test_openai_default_size_when_no_aspect_ratio() -> None:
fake = FakeClient(FakeResponse({"data": [{"b64_json": RAW_B64}]}))
client = OpenAIImageGenerationClient(
api_key="sk-openai-test",
client=fake, # type: ignore[arg-type]
)
await client.generate(prompt="draw", model="dall-e-3")
body = fake.calls[0]["json"]
assert body["size"] == "1024x1024"
@pytest.mark.asyncio
async def test_openai_ignores_explicit_size_unsupported_by_model_family() -> None:
fake = FakeClient(FakeResponse({"data": [{"b64_json": RAW_B64}]}))
client = OpenAIImageGenerationClient(
api_key="sk-openai-test",
client=fake, # type: ignore[arg-type]
)
await client.generate(
prompt="draw",
model="dall-e-3",
aspect_ratio="16:9",
image_size="1536x1024",
)
body = fake.calls[0]["json"]
assert body["size"] == "1792x1024"
@pytest.mark.asyncio
async def test_openai_uses_explicit_image_size() -> None:
fake = FakeClient(FakeResponse({"data": [{"b64_json": RAW_B64}]}))
client = OpenAIImageGenerationClient(
api_key="sk-openai-test",
client=fake, # type: ignore[arg-type]
)
await client.generate(
prompt="draw",
model="dall-e-3",
aspect_ratio="16:9",
image_size="1024x1024",
)
body = fake.calls[0]["json"]
assert body["size"] == "1024x1024"
@pytest.mark.asyncio
async def test_openai_requires_api_key() -> None:
client = OpenAIImageGenerationClient(api_key=None)
with pytest.raises(ImageGenerationError, match="API key"):
await client.generate(prompt="draw", model="dall-e-3")
# ---------------------------------------------------------------------------
# OpenAI Codex (Responses API)
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_codex_payload_and_response(monkeypatch) -> None:
import sys
from dataclasses import dataclass
from types import SimpleNamespace
@dataclass
class FakeToken:
account_id: str = "acct-123"
access: str = "oauth-token"
async def fake_to_thread(fn, *args, **kwargs):
return fn(*args, **kwargs)
monkeypatch.setattr("asyncio.to_thread", fake_to_thread)
fake_oauth = SimpleNamespace(get_token=lambda: FakeToken())
monkeypatch.setitem(sys.modules, "oauth_cli_kit", fake_oauth)
sse_lines = [
'data: {"type":"response.output_item.added","item":{"id":"ig_1","type":"image_generation_call","status":"in_progress"}}',
"",
f'data: {{"type":"response.output_item.done","item":{{"id":"ig_1","type":"image_generation_call","result":"{PNG_DATA_URL}","status":"completed"}}}}',
"",
'data: [DONE]',
"",
]
fake = FakeClient(FakeResponse({}, sse_lines=sse_lines))
client = CodexImageGenerationClient(
api_key=None,
api_base="https://chatgpt.com/backend-api",
extra_headers={"X-Test": "1"},
client=fake, # type: ignore[arg-type]
)
response = await client.generate(
prompt="draw a cat",
model="gpt-5.4",
)
assert response.images == [PNG_DATA_URL]
assert response.content == ""
call = fake.calls[0]
assert call["url"] == "https://chatgpt.com/backend-api/codex/responses"
assert call["headers"]["Authorization"] == "Bearer oauth-token"
assert call["headers"]["chatgpt-account-id"] == "acct-123"
assert call["headers"]["OpenAI-Beta"] == "responses=experimental"
assert call["headers"]["X-Test"] == "1"
body = call["json"]
assert body["model"] == "gpt-5.4"
assert body["instructions"] == "Generate an image based on the user's request."
assert body["input"] == [{"role": "user", "content": "draw a cat"}]
assert body["tools"] == [{"type": "image_generation"}]
assert body["tool_choice"] == "auto"
assert body["store"] is False
assert body["stream"] is True
@pytest.mark.asyncio
async def test_codex_strips_model_prefix(monkeypatch) -> None:
import sys
from dataclasses import dataclass
from types import SimpleNamespace
@dataclass
class FakeToken:
account_id: str = "acct-123"
access: str = "oauth-token"
async def fake_to_thread(fn, *args, **kwargs):
return fn(*args, **kwargs)
monkeypatch.setattr("asyncio.to_thread", fake_to_thread)
fake_oauth = SimpleNamespace(get_token=lambda: FakeToken())
monkeypatch.setitem(sys.modules, "oauth_cli_kit", fake_oauth)
fake = FakeClient(FakeResponse({}, sse_lines=[
f'data: {{"type":"response.output_item.done","item":{{"type":"image_generation_call","result":"{PNG_DATA_URL}"}}}}',
"",
'data: [DONE]',
"",
]))
client = CodexImageGenerationClient(
api_key=None, client=fake # type: ignore[arg-type]
)
await client.generate(prompt="draw", model="openai-codex/gpt-5.4")
assert fake.calls[0]["json"]["model"] == "gpt-5.4"
@pytest.mark.asyncio
async def test_codex_requires_oauth(monkeypatch) -> None:
async def fake_to_thread(fn, *args, **kwargs):
raise RuntimeError("no token")
monkeypatch.setattr("asyncio.to_thread", fake_to_thread)
client = CodexImageGenerationClient(api_key=None)
with pytest.raises(ImageGenerationError, match="OAuth token"):
await client.generate(prompt="draw", model="gpt-5.4")
@pytest.mark.asyncio
async def test_codex_no_images_raises(monkeypatch) -> None:
import sys
from dataclasses import dataclass
from types import SimpleNamespace
@dataclass
class FakeToken:
account_id: str = "acct-123"
access: str = "oauth-token"
async def fake_to_thread(fn, *args, **kwargs):
return fn(*args, **kwargs)
monkeypatch.setattr("asyncio.to_thread", fake_to_thread)
fake_oauth = SimpleNamespace(get_token=lambda: FakeToken())
monkeypatch.setitem(sys.modules, "oauth_cli_kit", fake_oauth)
fake = FakeClient(FakeResponse({}, sse_lines=[
'data: {"type":"response.completed","response":{"status":"completed"}}',
"",
'data: [DONE]',
"",
]))
client = CodexImageGenerationClient(
api_key=None, client=fake # type: ignore[arg-type]
)
with pytest.raises(ImageGenerationError, match="returned no images"):
await client.generate(prompt="draw", model="gpt-5.4")
@pytest.mark.asyncio
async def test_codex_extracts_text_content(monkeypatch) -> None:
import sys
from dataclasses import dataclass
from types import SimpleNamespace
@dataclass
class FakeToken:
account_id: str = "acct-123"
access: str = "oauth-token"
async def fake_to_thread(fn, *args, **kwargs):
return fn(*args, **kwargs)
monkeypatch.setattr("asyncio.to_thread", fake_to_thread)
fake_oauth = SimpleNamespace(get_token=lambda: FakeToken())
monkeypatch.setitem(sys.modules, "oauth_cli_kit", fake_oauth)
fake = FakeClient(FakeResponse({}, sse_lines=[
'data: {"type":"response.output_text.delta","delta":"Here "}',
"",
'data: {"type":"response.output_text.delta","delta":"is your cat image."}',
"",
f'data: {{"type":"response.output_item.done","item":{{"type":"image_generation_call","result":"{PNG_DATA_URL}"}}}}',
"",
'data: [DONE]',
"",
]))
client = CodexImageGenerationClient(
api_key=None, client=fake # type: ignore[arg-type]
)
response = await client.generate(prompt="draw a cat", model="gpt-5.4")
assert response.images == [PNG_DATA_URL]
assert response.content == "Here is your cat image."
@pytest.mark.asyncio
async def test_codex_json_result_format(monkeypatch) -> None:
"""image_generation_call result can be a dict with image_url key."""
import sys
from dataclasses import dataclass
from types import SimpleNamespace
@dataclass
class FakeToken:
account_id: str = "acct-123"
access: str = "oauth-token"
async def fake_to_thread(fn, *args, **kwargs):
return fn(*args, **kwargs)
monkeypatch.setattr("asyncio.to_thread", fake_to_thread)
fake_oauth = SimpleNamespace(get_token=lambda: FakeToken())
monkeypatch.setitem(sys.modules, "oauth_cli_kit", fake_oauth)
fake = FakeClient(FakeResponse({}, sse_lines=[
f'data: {{"type":"response.output_item.done","item":{{"type":"image_generation_call","result":{{"image_url":"{PNG_DATA_URL}"}}}}}}',
"",
'data: [DONE]',
"",
]))
client = CodexImageGenerationClient(
api_key=None, client=fake # type: ignore[arg-type]
)
response = await client.generate(prompt="draw", model="gpt-5.4")
assert response.images == [PNG_DATA_URL]
@pytest.mark.asyncio
async def test_openai_no_images_raises() -> None:
fake = FakeClient(FakeResponse({"data": []}))
client = OpenAIImageGenerationClient(
api_key="sk-openai-test",
client=fake, # type: ignore[arg-type]
)
with pytest.raises(ImageGenerationError, match="returned no images"):
await client.generate(prompt="draw", model="dall-e-3")
# ---------------------------------------------------------------------------
# Zhipu
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_zhipu_image_generation_payload_and_response() -> None:
fake = FakeClient(FakeResponse({"data": [{"url": "https://cdn.example/image.png"}]}))
fake.get_response = FakeResponse({}, content=PNG_BYTES)
client = ZhipuImageGenerationClient(
api_key="sk-zhipu-test",
api_base="https://open.bigmodel.cn/api/paas/v4",
extra_headers={"X-Test": "1"},
extra_body={"watermark_enabled": False},
client=fake, # type: ignore[arg-type]
)
response = await client.generate(
prompt="a sunset over the ocean",
model="glm-image",
aspect_ratio="16:9",
image_size="2K",
)
assert response.images[0].startswith("data:image/png;base64,")
call = fake.calls[0]
assert call["url"] == "https://open.bigmodel.cn/api/paas/v4/images/generations"
assert call["headers"]["Authorization"] == "Bearer sk-zhipu-test"
assert call["headers"]["X-Test"] == "1"
body = call["json"]
assert body["model"] == "glm-image"
assert body["prompt"] == "a sunset over the ocean"
assert body["size"] == "1728x960"
assert body["watermark_enabled"] is False
@pytest.mark.asyncio
async def test_zhipu_image_generation_with_explicit_size() -> None:
fake = FakeClient(FakeResponse({"data": [{"url": "https://cdn.example/image.png"}]}))
fake.get_response = FakeResponse({}, content=PNG_BYTES)
client = ZhipuImageGenerationClient(
api_key="sk-zhipu-test",
client=fake, # type: ignore[arg-type]
)
await client.generate(
prompt="a cat",
model="cogview-4",
image_size="1024x1024",
)
body = fake.calls[0]["json"]
assert body["size"] == "1024x1024"
@pytest.mark.asyncio
async def test_zhipu_image_generation_downloads_url_response() -> None:
fake = FakeClient(FakeResponse({"data": [{"url": "https://cdn.example/image.png"}]}))
fake.get_response = FakeResponse({}, content=PNG_BYTES)
client = ZhipuImageGenerationClient(
api_key="sk-zhipu-test",
client=fake, # type: ignore[arg-type]
)
response = await client.generate(prompt="draw", model="glm-image")
assert response.images[0].startswith("data:image/png;base64,")
assert fake.get_calls[0]["url"] == "https://cdn.example/image.png"
@pytest.mark.asyncio
async def test_zhipu_image_generation_requires_api_key() -> None:
client = ZhipuImageGenerationClient(api_key=None)
with pytest.raises(ImageGenerationError, match="API key"):
await client.generate(prompt="draw", model="glm-image")
@pytest.mark.asyncio
async def test_zhipu_image_generation_no_images_raises() -> None:
fake = FakeClient(FakeResponse({"data": [{"text": "sorry"}]}))
client = ZhipuImageGenerationClient(api_key="sk-zhipu-test", client=fake) # type: ignore[arg-type]
with pytest.raises(ImageGenerationError, match="returned no images"):
await client.generate(prompt="draw", model="glm-image")
@pytest.mark.asyncio
async def test_zhipu_image_generation_rejects_reference_images() -> None:
client = ZhipuImageGenerationClient(api_key="sk-zhipu-test")
with pytest.raises(ImageGenerationError, match="reference images"):
await client.generate(
prompt="edit this",
model="glm-image",
reference_images=["ref.png"],
)
+104 -5
View File
@@ -441,6 +441,15 @@ def test_openrouter_spec_is_gateway() -> None:
assert spec.default_api_base == "https://openrouter.ai/api/v1"
def test_novita_spec_uses_openai_compatible_gateway() -> None:
spec = find_by_name("novita")
assert spec is not None
assert spec.is_gateway is True
assert spec.backend == "openai_compat"
assert spec.env_key == "NOVITA_API_KEY"
assert spec.default_api_base == "https://api.novita.ai/openai"
def test_gemma_routes_to_gemini_provider() -> None:
"""gemma models (e.g. gemma-3-27b-it) must auto-route to Gemini when GEMINI_API_KEY is set.
Users running gemma via the Gemini API endpoint expect automatic provider detection."""
@@ -449,6 +458,12 @@ def test_gemma_routes_to_gemini_provider() -> None:
assert "gemma" in spec.keywords
def test_gemini_spec_keeps_openai_compat_base() -> None:
spec = find_by_name("gemini")
assert spec is not None
assert spec.default_api_base == "https://generativelanguage.googleapis.com/v1beta/openai/"
async def test_openrouter_sets_default_attribution_headers() -> None:
spec = find_by_name("openrouter")
with patch("nanobot.providers.openai_compat_provider.AsyncOpenAI") as mock_client_cls:
@@ -587,6 +602,7 @@ async def test_openai_compat_preserves_extra_content_on_tool_calls() -> None:
assert len(result.tool_calls) == 1
tool_call = result.tool_calls[0]
assert tool_call.id == "call_123"
assert tool_call.extra_content == {"google": {"thought_signature": "signed-token"}}
assert tool_call.function_provider_specific_fields == {"inner": "value"}
@@ -979,7 +995,7 @@ def test_deepseek_thinking_keeps_tool_history_with_reasoning_content() -> None:
assert kwargs["messages"][2]["role"] == "tool"
def test_openai_compat_keeps_tool_calls_after_consecutive_assistant_messages() -> None:
def test_openai_compat_preserves_tool_call_ids_after_consecutive_assistant_messages() -> None:
with patch("nanobot.providers.openai_compat_provider.AsyncOpenAI"):
provider = OpenAICompatProvider()
@@ -1001,12 +1017,75 @@ def test_openai_compat_keeps_tool_calls_after_consecutive_assistant_messages() -
{"role": "user", "content": "多少star了呢"},
])
assert sanitized[1]["role"] == "assistant"
assert sanitized[1]["content"] is None
assert sanitized[1]["tool_calls"][0]["id"] == "call_function_akxp3wqzn7ph_1"
assert sanitized[2]["tool_call_id"] == "call_function_akxp3wqzn7ph_1"
def test_mistral_normalizes_tool_call_ids_after_consecutive_assistant_messages() -> None:
with patch("nanobot.providers.openai_compat_provider.AsyncOpenAI"):
provider = OpenAICompatProvider(spec=find_by_name("mistral"))
sanitized = provider._sanitize_messages([
{"role": "user", "content": "不错"},
{"role": "assistant", "content": "对,破 4 万指日可待"},
{
"role": "assistant",
"content": "<think>我再查一下</think>",
"tool_calls": [
{
"id": "call_function_akxp3wqzn7ph_1",
"type": "function",
"function": {"name": "exec", "arguments": "{}"},
}
],
},
{"role": "tool", "tool_call_id": "call_function_akxp3wqzn7ph_1", "name": "exec", "content": "ok"},
{"role": "user", "content": "多少star了呢"},
])
assert sanitized[1]["role"] == "assistant"
assert sanitized[1]["content"] is None
assert sanitized[1]["tool_calls"][0]["id"] == "3ec83c30d"
assert sanitized[2]["tool_call_id"] == "3ec83c30d"
def test_openai_compat_deduplicates_duplicate_tool_call_ids_in_history() -> None:
with patch("nanobot.providers.openai_compat_provider.AsyncOpenAI"):
provider = OpenAICompatProvider()
sanitized = provider._sanitize_messages([
{"role": "user", "content": "check both files"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "ab1b45c2a",
"type": "function",
"function": {"name": "read_file", "arguments": '{"path":"a.txt"}'},
},
{
"id": "ab1b45c2a",
"type": "function",
"function": {"name": "read_file", "arguments": '{"path":"b.txt"}'},
},
],
},
{"role": "tool", "tool_call_id": "ab1b45c2a", "name": "read_file", "content": "a"},
{"role": "tool", "tool_call_id": "ab1b45c2a", "name": "read_file", "content": "b"},
{"role": "user", "content": "continue"},
])
tool_call_ids = [tc["id"] for tc in sanitized[1]["tool_calls"]]
tool_result_ids = [sanitized[2]["tool_call_id"], sanitized[3]["tool_call_id"]]
assert tool_call_ids[0] == "ab1b45c2a"
assert len(tool_call_ids) == len(set(tool_call_ids)) == 2
assert tool_result_ids == tool_call_ids
def test_openai_compat_stringifies_dict_tool_arguments() -> None:
with patch("nanobot.providers.openai_compat_provider.AsyncOpenAI"):
provider = OpenAICompatProvider()
@@ -1376,12 +1455,15 @@ def test_kimi_k25_thinking_enabled() -> None:
"""kimi-k2.5 with reasoning_effort set should opt in to thinking."""
kw = _build_kwargs_for("moonshot", "kimi-k2.5", reasoning_effort="medium")
assert kw.get("extra_body") == {"thinking": {"type": "enabled"}}
# Moonshot rejects both 'reasoning_effort' and 'thinking' (#3939)
assert "reasoning_effort" not in kw
def test_kimi_k25_thinking_disabled_for_minimal() -> None:
"""reasoning_effort='minimal' maps to thinking disabled for kimi-k2.5."""
kw = _build_kwargs_for("moonshot", "kimi-k2.5", reasoning_effort="minimal")
assert kw.get("extra_body") == {"thinking": {"type": "disabled"}}
assert "reasoning_effort" not in kw
def test_kimi_k25_no_extra_body_when_reasoning_effort_none() -> None:
@@ -1391,21 +1473,36 @@ def test_kimi_k25_no_extra_body_when_reasoning_effort_none() -> None:
def test_kimi_k25_thinking_enabled_with_openrouter_prefix() -> None:
"""OpenRouter-style model names like moonshotai/kimi-k2.5 must trigger thinking."""
"""OpenRouter-style model names like moonshotai/kimi-k2.5 must trigger thinking.
OR drops upstream-provider `thinking` fields, so the same intent also has
to go through OR's `reasoning.effort` shape (#3851 follow-up).
"""
kw = _build_kwargs_for("openrouter", "moonshotai/kimi-k2.5", reasoning_effort="medium")
assert kw.get("extra_body") == {"thinking": {"type": "enabled"}}
assert kw.get("extra_body") == {
"thinking": {"type": "enabled"},
"reasoning": {"effort": "medium"},
}
# Even via OR, reasoning_effort wire kwarg is dropped for kimi models
assert "reasoning_effort" not in kw
def test_kimi_k26_thinking_enabled() -> None:
"""kimi-k2.6 with reasoning_effort set should opt in to thinking."""
kw = _build_kwargs_for("moonshot", "kimi-k2.6", reasoning_effort="medium")
assert kw.get("extra_body") == {"thinking": {"type": "enabled"}}
assert "reasoning_effort" not in kw
def test_kimi_k26_thinking_enabled_with_openrouter_prefix() -> None:
"""OpenRouter-style names like moonshotai/kimi-k2.6 must trigger thinking."""
"""OpenRouter-style names like moonshotai/kimi-k2.6 must trigger thinking
via both upstream `thinking` and OR's `reasoning.effort`."""
kw = _build_kwargs_for("openrouter", "moonshotai/kimi-k2.6", reasoning_effort="medium")
assert kw.get("extra_body") == {"thinking": {"type": "enabled"}}
assert kw.get("extra_body") == {
"thinking": {"type": "enabled"},
"reasoning": {"effort": "medium"},
}
assert "reasoning_effort" not in kw
def test_moonshot_kimi_k26_temperature_override() -> None:
@@ -1424,6 +1521,7 @@ def test_kimi_k26_code_preview_thinking_enabled() -> None:
"""k2.6-code-preview also supports thinking; should behave like k2.5."""
kw = _build_kwargs_for("moonshot", "k2.6-code-preview", reasoning_effort="high")
assert kw.get("extra_body") == {"thinking": {"type": "enabled"}}
assert "reasoning_effort" not in kw
def test_kimi_k2_series_no_thinking_injection() -> None:
@@ -1453,6 +1551,7 @@ def test_kimi_k25_thinking_disabled_for_none_string() -> None:
"""reasoning_effort='none' maps to thinking disabled for kimi-k2.5."""
kw = _build_kwargs_for("moonshot", "kimi-k2.5", reasoning_effort="none")
assert kw.get("extra_body") == {"thinking": {"type": "disabled"}}
assert "reasoning_effort" not in kw
def test_dashscope_thinking_disabled_for_none_string() -> None:
+97
View File
@@ -0,0 +1,97 @@
"""Tests for the Novita AI provider registration."""
from unittest.mock import patch
from nanobot.config.schema import Config, ProvidersConfig
from nanobot.providers.openai_compat_provider import OpenAICompatProvider
from nanobot.providers.registry import PROVIDERS, find_by_name
def test_novita_config_field_exists() -> None:
config = ProvidersConfig()
assert hasattr(config, "novita")
def test_novita_provider_in_registry() -> None:
specs = {spec.name: spec for spec in PROVIDERS}
assert "novita" in specs
novita = specs["novita"]
assert novita.backend == "openai_compat"
assert novita.env_key == "NOVITA_API_KEY"
assert novita.display_name == "Novita AI"
assert novita.is_gateway is True
assert novita.detect_by_base_keyword == "novita"
assert novita.default_api_base == "https://api.novita.ai/openai"
assert novita.strip_model_prefix is False
def test_find_by_name_novita() -> None:
spec = find_by_name("novita")
assert spec is not None
assert spec.name == "novita"
def test_novita_forced_provider_uses_default_api_base() -> None:
config = Config.model_validate({
"providers": {
"novita": {
"apiKey": "novita-key",
},
},
"agents": {
"defaults": {
"model": "deepseek-v4-pro",
"provider": "novita",
},
},
})
assert config.get_provider_name("deepseek-v4-pro") == "novita"
assert config.get_api_key("deepseek-v4-pro") == "novita-key"
assert config.get_api_base("deepseek-v4-pro") == "https://api.novita.ai/openai"
def test_novita_gateway_routes_unprefixed_models_when_configured() -> None:
config = Config.model_validate({
"providers": {
"novita": {
"apiKey": "novita-key",
},
},
"agents": {
"defaults": {
"model": "deepseek-v4-pro",
},
},
})
assert config.get_provider_name("deepseek-v4-pro") == "novita"
assert config.get_api_key("deepseek-v4-pro") == "novita-key"
assert config.get_api_base("deepseek-v4-pro") == "https://api.novita.ai/openai"
def test_novita_preserves_model_api_id() -> None:
spec = find_by_name("novita")
with patch("nanobot.providers.openai_compat_provider.AsyncOpenAI"):
provider = OpenAICompatProvider(
api_key="novita-key",
default_model="deepseek-v4-pro",
spec=spec,
)
kwargs = provider._build_kwargs(
messages=[{"role": "user", "content": "hi"}],
tools=None,
model="deepseek-v4-pro",
max_tokens=1024,
temperature=0.7,
reasoning_effort=None,
tool_choice=None,
)
assert kwargs["model"] == "deepseek-v4-pro"
assert kwargs["max_tokens"] == 1024
assert "max_completion_tokens" not in kwargs
+43
View File
@@ -155,6 +155,49 @@ class TestConvertMessages:
assert items[0]["id"] == "fc_1"
assert items[0]["name"] == "get_weather"
def test_duplicate_response_item_ids_are_made_unique(self):
"""Codex rejects replayed Responses input items with duplicate ids."""
_, items = convert_messages([
{
"role": "assistant",
"content": None,
"tool_calls": [{
"id": "call_a|rs_same",
"function": {"name": "first", "arguments": "{}"},
}],
},
{"role": "tool", "tool_call_id": "call_a|rs_same", "content": "ok"},
{
"role": "assistant",
"content": None,
"tool_calls": [{
"id": "call_b|rs_same",
"function": {"name": "second", "arguments": "{}"},
}],
},
{"role": "tool", "tool_call_id": "call_b|rs_same", "content": "ok"},
])
function_call_ids = [
item["id"] for item in items if item.get("type") == "function_call"
]
assert function_call_ids == ["rs_same", "rs_same_2"]
assert len(function_call_ids) == len(set(function_call_ids))
def test_fallback_response_item_ids_are_unique_with_multiple_tool_calls(self):
_, items = convert_messages([{
"role": "assistant",
"content": None,
"tool_calls": [
{"id": "call_a", "function": {"name": "first", "arguments": "{}"}},
{"id": "call_b", "function": {"name": "second", "arguments": "{}"}},
],
}])
function_call_ids = [
item["id"] for item in items if item.get("type") == "function_call"
]
assert function_call_ids == ["fc_0", "fc_0_2"]
assert len(function_call_ids) == len(set(function_call_ids))
def test_assistant_with_tool_calls_no_id(self):
"""Fallback IDs when tool_call.id is missing."""
_, items = convert_messages([{
-9
View File
@@ -12,7 +12,6 @@ def test_importing_providers_package_is_lazy(monkeypatch) -> None:
monkeypatch.delitem(sys.modules, "nanobot.providers.openai_compat_provider", raising=False)
monkeypatch.delitem(sys.modules, "nanobot.providers.openai_codex_provider", raising=False)
monkeypatch.delitem(sys.modules, "nanobot.providers.github_copilot_provider", raising=False)
monkeypatch.delitem(sys.modules, "nanobot.providers.xai_oauth_provider", raising=False)
monkeypatch.delitem(sys.modules, "nanobot.providers.azure_openai_provider", raising=False)
monkeypatch.delitem(sys.modules, "nanobot.providers.bedrock_provider", raising=False)
@@ -22,7 +21,6 @@ def test_importing_providers_package_is_lazy(monkeypatch) -> None:
assert "nanobot.providers.openai_compat_provider" not in sys.modules
assert "nanobot.providers.openai_codex_provider" not in sys.modules
assert "nanobot.providers.github_copilot_provider" not in sys.modules
assert "nanobot.providers.xai_oauth_provider" not in sys.modules
assert "nanobot.providers.azure_openai_provider" not in sys.modules
assert "nanobot.providers.bedrock_provider" not in sys.modules
assert providers.__all__ == [
@@ -32,7 +30,6 @@ def test_importing_providers_package_is_lazy(monkeypatch) -> None:
"OpenAICompatProvider",
"OpenAICodexProvider",
"GitHubCopilotProvider",
"XaiOAuthProvider",
"AzureOpenAIProvider",
"BedrockProvider",
]
@@ -53,9 +50,3 @@ def test_openai_codex_supports_progress_deltas() -> None:
from nanobot.providers.openai_codex_provider import OpenAICodexProvider
assert OpenAICodexProvider.supports_progress_deltas is True
def test_xai_oauth_supports_progress_deltas() -> None:
from nanobot.providers.xai_oauth_provider import XaiOAuthProvider
assert XaiOAuthProvider.supports_progress_deltas is True
@@ -18,6 +18,7 @@ def provider():
p.default_model = "gpt-5"
p._spec = type("Spec", (), {"name": "openai"})()
p._effective_base = "https://api.openai.com/v1"
p._api_type = "auto"
p._responses_failures = {}
p._responses_tripped_at = {}
return p
@@ -27,6 +28,33 @@ def test_responses_api_available_by_default(provider):
assert provider._should_use_responses_api("gpt-5", None) is True
def test_api_type_chat_completions_disables_responses(provider):
provider._api_type = "chat_completions"
assert provider._should_use_responses_api("gpt-5", None) is False
def test_api_type_responses_forces_responses_for_openai(provider):
provider.default_model = "gpt-4o"
provider._api_type = "responses"
assert provider._should_use_responses_api("gpt-4o", None) is True
def test_api_type_responses_ignores_circuit_breaker(provider):
provider.default_model = "gpt-4o"
provider._api_type = "responses"
provider._responses_failures = {"gpt-4o|gpt-4o|": _RESPONSES_FAILURE_THRESHOLD}
provider._responses_tripped_at = {"gpt-4o|gpt-4o|": 0.0}
assert provider._should_use_responses_api("gpt-4o", None) is True
def test_api_type_responses_does_not_force_non_openai(provider):
provider._spec = type("Spec", (), {"name": "custom"})()
provider._api_type = "responses"
assert provider._should_use_responses_api("gpt-4o", None) is False
def test_circuit_opens_after_threshold(provider):
for _ in range(_RESPONSES_FAILURE_THRESHOLD):
provider._record_responses_failure("gpt-5", None)
+39 -1
View File
@@ -8,7 +8,11 @@ from unittest.mock import AsyncMock, patch
import httpx
import pytest
from nanobot.providers.transcription import GroqTranscriptionProvider, OpenAITranscriptionProvider
from nanobot.providers.transcription import (
GroqTranscriptionProvider,
OpenAITranscriptionProvider,
_resolve_transcription_url,
)
@pytest.fixture
@@ -290,3 +294,37 @@ async def test_retries_on_every_advertised_transient_exception(
result = await provider.transcribe(audio_file)
assert result == "recovered"
assert post.await_count == 2
# ---------------------------------------------------------------------------
# apiBase normalization (#3637): a chat-style base must not be POSTed verbatim
# ---------------------------------------------------------------------------
def test_resolve_transcription_url_falls_back_to_default() -> None:
default = "https://api.openai.com/v1/audio/transcriptions"
assert _resolve_transcription_url(None, default) == default
assert _resolve_transcription_url("", default) == default
def test_resolve_transcription_url_appends_path_to_chat_style_base() -> None:
assert (
_resolve_transcription_url("https://api.groq.com/openai/v1", "https://x/audio/transcriptions")
== "https://api.groq.com/openai/v1/audio/transcriptions"
)
# Trailing slash must not produce a doubled separator.
assert (
_resolve_transcription_url("https://api.groq.com/openai/v1/", "https://x/audio/transcriptions")
== "https://api.groq.com/openai/v1/audio/transcriptions"
)
def test_resolve_transcription_url_keeps_full_endpoint() -> None:
full = "https://api.groq.com/openai/v1/audio/transcriptions"
assert _resolve_transcription_url(full, "https://x/audio/transcriptions") == full
def test_groq_provider_normalizes_chat_style_api_base() -> None:
"""Regression for #3637: apiBase set to the v1 base resolves to the audio endpoint."""
provider = GroqTranscriptionProvider(api_key="gsk-test", api_base="https://api.groq.com/openai/v1")
assert provider.api_url == "https://api.groq.com/openai/v1/audio/transcriptions"
-146
View File
@@ -1,146 +0,0 @@
from __future__ import annotations
import json
import os
import stat
from urllib.parse import parse_qs, urlparse
import pytest
import nanobot.providers.xai_oauth_provider as auth
def test_build_xai_authorization_url_includes_pkce_and_grok_scope() -> None:
endpoints = auth.XaiOAuthEndpoints(
authorization_endpoint="https://auth.x.ai/authorize",
token_endpoint="https://auth.x.ai/oauth/token",
)
url = auth.build_xai_authorization_url(
endpoints,
verifier="verifier",
state="state",
nonce="nonce",
)
parsed = urlparse(url)
params = parse_qs(parsed.query)
assert parsed.scheme == "https"
assert parsed.hostname == "auth.x.ai"
assert params["client_id"] == [auth.DEFAULT_XAI_CLIENT_ID]
assert params["code_challenge"] == [auth.pkce_challenge("verifier")]
assert params["code_challenge_method"] == ["S256"]
assert params["scope"] == [auth.DEFAULT_XAI_SCOPE]
assert params["nonce"] == ["nonce"]
assert params["plan"] == ["generic"]
assert params["referrer"] == ["nanobot"]
def test_parse_callback_value_accepts_fallback_shapes() -> None:
assert auth._parse_callback_value("https://localhost/callback?code=abc&state=state") == ("abc", "state")
assert auth._parse_callback_value("?code=abc&state=state") == ("abc", "state")
assert auth._parse_callback_value("code=abc&state=state") == ("abc", "state")
assert auth._parse_callback_value("fallback-code") == ("fallback-code", None)
def test_file_storage_fallback_is_private_and_round_trips(tmp_path, monkeypatch) -> None:
monkeypatch.setenv("NANOBOT_HOME", str(tmp_path))
monkeypatch.setattr(auth, "_keyring_set", lambda _tokens: False)
monkeypatch.setattr(auth, "_keyring_get", lambda: None)
saved = auth.save_xai_oauth_credential(
auth.XaiOAuthCredential(
access_token="access",
refresh_token="refresh",
expires_at=123.0,
account_id="acct",
)
)
path = auth.get_xai_oauth_metadata_path()
payload = json.loads(path.read_text(encoding="utf-8"))
assert saved.storage == "file"
assert payload["storage"] == "file"
assert payload["tokens"]["access_token"] == "access"
if os.name != "nt":
assert stat.S_IMODE(path.stat().st_mode) == 0o600
loaded = auth.load_xai_oauth_credential()
assert loaded is not None
assert loaded.access_token == "access"
assert loaded.refresh_token == "refresh"
assert loaded.account_id == "acct"
assert loaded.storage == "file"
def test_keyring_storage_keeps_tokens_out_of_metadata(tmp_path, monkeypatch) -> None:
monkeypatch.setenv("NANOBOT_HOME", str(tmp_path))
secret: dict[str, object] = {}
def fake_set(tokens: dict[str, object]) -> bool:
secret.update(tokens)
return True
monkeypatch.setattr(auth, "_keyring_set", fake_set)
monkeypatch.setattr(auth, "_keyring_get", lambda: dict(secret))
auth.save_xai_oauth_credential(
auth.XaiOAuthCredential(
access_token="access",
refresh_token="refresh",
expires_at=123.0,
account_id="acct",
)
)
payload = json.loads(auth.get_xai_oauth_metadata_path().read_text(encoding="utf-8"))
assert payload["storage"] == "keyring"
assert "tokens" not in payload
assert auth.load_xai_oauth_credential().access_token == "access"
def test_exchange_xai_oauth_code_sends_required_code_challenge(monkeypatch) -> None:
captured: dict[str, object] = {}
class FakeResponse:
status_code = 200
text = ""
def json(self) -> dict[str, object]:
return {"access_token": "access", "refresh_token": "refresh", "expires_in": 3600}
class FakeClient:
def __init__(self, *args, **kwargs) -> None:
pass
def __enter__(self):
return self
def __exit__(self, *args) -> None:
pass
def post(self, url: str, headers: dict[str, str], data: dict[str, str]) -> FakeResponse:
captured["url"] = url
captured["headers"] = headers
captured["data"] = data
return FakeResponse()
monkeypatch.setattr(auth.httpx, "Client", FakeClient)
endpoints = auth.XaiOAuthEndpoints(
authorization_endpoint="https://auth.x.ai/authorize",
token_endpoint="https://auth.x.ai/oauth/token",
)
credential = auth.exchange_xai_oauth_code("code", verifier="verifier", endpoints=endpoints)
assert credential.access_token == "access"
assert captured["url"] == "https://auth.x.ai/oauth/token"
data = captured["data"]
assert data["code_verifier"] == "verifier"
assert data["code_challenge"] == auth.pkce_challenge("verifier")
assert data["code_challenge_method"] == "S256"
def test_rejects_non_xai_discovery_endpoints() -> None:
with pytest.raises(RuntimeError):
auth._validate_xai_endpoint("https://example.com/oauth/token", "token_endpoint")
-141
View File
@@ -1,141 +0,0 @@
from __future__ import annotations
import asyncio
from nanobot.config.schema import XaiOAuthXSearchConfig
import nanobot.providers.xai_oauth_provider as xai_oauth_provider
from nanobot.providers.xai_oauth_provider import (
XaiOAuthCredential,
XaiOAuthProvider,
_build_xai_responses_body,
_strip_model_prefix,
)
def test_xai_oauth_strip_prefix_supports_aliases() -> None:
assert _strip_model_prefix("xai-oauth/grok-4.3") == "grok-4.3"
assert _strip_model_prefix("xai_oauth/grok-4.3") == "grok-4.3"
assert _strip_model_prefix("grok-oauth/grok-4.3") == "grok-4.3"
assert _strip_model_prefix("grok-4.3") == "grok-4.3"
def test_build_xai_responses_body_keeps_system_prompt_in_input() -> None:
body = _build_xai_responses_body(
messages=[
{"role": "system", "content": "You are nanobot."},
{"role": "user", "content": "hi"},
],
tools=[
{
"type": "function",
"function": {
"name": "ping",
"description": "Ping",
"parameters": {"type": "object", "properties": {}},
},
}
],
model="xai-oauth/grok-4.3",
max_tokens=32,
temperature=0.2,
reasoning_effort="high",
tool_choice=None,
)
assert body["model"] == "grok-4.3"
assert "instructions" not in body
assert body["input"][0] == {
"role": "system",
"content": [{"type": "input_text", "text": "You are nanobot."}],
}
assert body["input"][1]["role"] == "user"
assert body["max_output_tokens"] == 32
assert body["temperature"] == 0.2
assert body["reasoning"] == {"effort": "high"}
assert body["tools"][0]["name"] == "ping"
def test_build_xai_responses_body_attaches_hosted_x_search_by_default() -> None:
body = _build_xai_responses_body(
messages=[{"role": "user", "content": "what is happening on X?"}],
tools=None,
model="xai-oauth/grok-4.3",
max_tokens=32,
temperature=0.2,
reasoning_effort=None,
tool_choice=None,
hosted_x_search=XaiOAuthXSearchConfig(),
)
assert body["tools"] == [{"type": "x_search"}]
def test_build_xai_responses_body_can_customize_hosted_x_search() -> None:
body = _build_xai_responses_body(
messages=[{"role": "user", "content": "what is happening on X?"}],
tools=None,
model="xai-oauth/grok-4.3",
max_tokens=32,
temperature=0.2,
reasoning_effort=None,
tool_choice=None,
hosted_x_search=XaiOAuthXSearchConfig(
allowed_x_handles=["@xai", " nanobot "],
enable_image_understanding=True,
),
)
assert body["tools"] == [
{
"type": "x_search",
"allowed_x_handles": ["xai", "nanobot"],
"enable_image_understanding": True,
}
]
def test_build_xai_responses_body_omits_disabled_hosted_x_search() -> None:
body = _build_xai_responses_body(
messages=[{"role": "user", "content": "hi"}],
tools=None,
model="xai-oauth/grok-4.3",
max_tokens=32,
temperature=0.2,
reasoning_effort=None,
tool_choice=None,
hosted_x_search=XaiOAuthXSearchConfig(enable=False),
)
assert "tools" not in body
def test_xai_oauth_provider_refreshes_once_on_401(monkeypatch) -> None:
async def run() -> None:
response = await provider.chat([{"role": "user", "content": "hi"}])
assert response.content == "ok"
assert response.finish_reason == "stop"
assert calls == [("resolve", False), ("resolve", True)]
provider = XaiOAuthProvider(default_model="xai-oauth/grok-4.3")
credentials = [
XaiOAuthCredential(access_token="expired"),
XaiOAuthCredential(access_token="fresh"),
]
calls: list[tuple[str, bool]] = []
def fake_resolve(*, force_refresh: bool = False) -> XaiOAuthCredential:
calls.append(("resolve", force_refresh))
return credentials.pop(0)
async def fake_request(credential, body, on_content_delta=None, on_tool_call_delta=None):
from nanobot.providers.xai_oauth_provider import _XaiHTTPError
if credential.access_token == "expired":
raise _XaiHTTPError("expired", status_code=401)
return "ok", [], "stop"
monkeypatch.setattr(xai_oauth_provider, "resolve_xai_oauth_credential", fake_resolve)
monkeypatch.setattr(xai_oauth_provider, "_request_xai", fake_request)
asyncio.run(run())
+44 -9
View File
@@ -32,7 +32,7 @@ def _mimo_spec():
def _openrouter_spec():
"""Return the registered OpenRouter ProviderSpec (no thinking_style)."""
"""Return the registered OpenRouter ProviderSpec."""
specs = {s.name: s for s in PROVIDERS}
return specs["openrouter"]
@@ -77,6 +77,13 @@ def test_xiaomi_mimo_uses_thinking_type_style():
assert spec.default_api_base == "https://api.xiaomimimo.com/v1"
def test_openrouter_declares_gateway_reasoning_style():
"""OpenRouter uses its own reasoning.effort field for routed thinking models."""
spec = _openrouter_spec()
assert spec.thinking_style == ""
assert spec.gateway_reasoning_style == "reasoning_effort"
# ---------------------------------------------------------------------------
# _build_kwargs wire-format
# ---------------------------------------------------------------------------
@@ -142,9 +149,11 @@ def test_mimo_reasoning_effort_unset_preserves_provider_default():
def test_mimo_via_openrouter_reasoning_effort_none_disables_thinking():
"""OpenRouter routes MiMo as "xiaomi/mimo-v2.5-pro"; the openrouter spec
has no thinking_style, so the disable signal must come from the
model-name path (#3845)."""
"""OpenRouter routes MiMo as "xiaomi/mimo-v2.5-pro" and does NOT forward
extra_body.thinking to upstream, so a disable signal must also reach OR
in its own `reasoning.effort` shape. Verifies both the upstream-MiMo
payload (#3845) and the OR-native payload (#3851 follow-up) are sent.
"""
provider = _openrouter_provider("xiaomi/mimo-v2.5-pro")
kwargs = provider._build_kwargs(
messages=_simple_messages(),
@@ -152,11 +161,15 @@ def test_mimo_via_openrouter_reasoning_effort_none_disables_thinking():
temperature=0.7, reasoning_effort="none", tool_choice=None,
)
assert "reasoning_effort" not in kwargs
assert kwargs["extra_body"] == {"thinking": {"type": "disabled"}}
assert kwargs["extra_body"] == {
"thinking": {"type": "disabled"},
"reasoning": {"effort": "none"},
}
def test_mimo_via_openrouter_reasoning_effort_medium_enables_thinking():
"""Same as the direct path: any non-none/minimal effort enables thinking."""
"""Non-none/minimal effort enables thinking and the OR `reasoning.effort`
field mirrors the requested effort level."""
provider = _openrouter_provider("xiaomi/mimo-v2.5-pro")
kwargs = provider._build_kwargs(
messages=_simple_messages(),
@@ -164,7 +177,10 @@ def test_mimo_via_openrouter_reasoning_effort_medium_enables_thinking():
temperature=0.7, reasoning_effort="medium", tool_choice=None,
)
assert kwargs.get("reasoning_effort") == "medium"
assert kwargs["extra_body"] == {"thinking": {"type": "enabled"}}
assert kwargs["extra_body"] == {
"thinking": {"type": "enabled"},
"reasoning": {"effort": "medium"},
}
def test_mimo_via_openrouter_bare_slug_also_matches():
@@ -176,12 +192,16 @@ def test_mimo_via_openrouter_bare_slug_also_matches():
tools=None, model=None, max_tokens=100,
temperature=0.7, reasoning_effort="none", tool_choice=None,
)
assert kwargs["extra_body"] == {"thinking": {"type": "disabled"}}
assert kwargs["extra_body"] == {
"thinking": {"type": "disabled"},
"reasoning": {"effort": "none"},
}
def test_mimo_flash_via_openrouter_does_not_inject_thinking():
"""mimo-v2-flash has no thinking mode per Xiaomi docs; the allowlist
excludes it, so no thinking field should be injected on the gateway path."""
excludes it, so neither the upstream `thinking` field nor OR's
`reasoning.effort` should be injected on the gateway path."""
provider = _openrouter_provider("xiaomi/mimo-v2-flash")
kwargs = provider._build_kwargs(
messages=_simple_messages(),
@@ -200,3 +220,18 @@ def test_non_mimo_model_via_openrouter_unaffected():
temperature=0.7, reasoning_effort="none", tool_choice=None,
)
assert "extra_body" not in kwargs
def test_kimi_via_openrouter_also_injects_reasoning_effort():
"""Kimi has the same gateway problem as MiMo: OR drops the upstream
`thinking` field. The same OR-reasoning injection should fire."""
provider = _openrouter_provider("moonshotai/kimi-k2.5")
kwargs = provider._build_kwargs(
messages=_simple_messages(),
tools=None, model=None, max_tokens=100,
temperature=0.7, reasoning_effort="none", tool_choice=None,
)
assert kwargs["extra_body"] == {
"thinking": {"type": "disabled"},
"reasoning": {"effort": "none"},
}

Some files were not shown because too many files have changed in this diff Show More