Compare commits

..
Author SHA1 Message Date
Mohamed Elkholyandchengyongru adcd3feb40 style: fix import sorting (ruff I001) 2026-04-19 15:32:18 +08:00
Mohamed Elkholyandchengyongru 0fe7148e6e style: move loguru import to module top level
Addresses reviewer suggestion to keep imports conventional.
2026-04-19 15:32:18 +08:00
Mohamed Elkholyandchengyongru 1ced8d4420 fix(providers): add circuit breaker for Responses API fallback
When the Responses API fails repeatedly (3 consecutive compatibility
errors), skip it and fall back directly to Chat Completions.  Unlike a
permanent disable, the circuit re-probes after 5 minutes so recovery
is automatic when the API comes back.  Success resets the counter.

Keyed per (model, reasoning_effort) so a failure with one model does
not affect others.
2026-04-19 15:32:18 +08:00
chengyongruandchengyongru 9b9e0964a2 test: add unit tests for configurable consolidation_ratio
Cover ratio propagation, schema validation, and consolidation
behavior with different ratio values (0.1, 0.5, 0.9).
2026-04-18 23:23:05 +08:00
Subalandchengyongru 1f4a6225c8 feat: make consolidation ratio configurable 2026-04-18 23:23:05 +08:00
Cheng Yongruandchengyongru 9bd29a8f4d fix(memory): fall back to raw_archive on LLM error response
When chat_with_retry returns an error response (finish_reason='error')
instead of raising an exception, archive() previously treated the error
message as a valid summary and wrote it to history.jsonl, while the
original session data was already cleared by /new — causing irreversible
data loss.

Fix: check finish_reason after the LLM call and raise RuntimeError on
error responses, which naturally falls through to the existing raw_archive
fallback. This preserves the original messages in history.jsonl instead
of losing them.

Fixes #3244
2026-04-17 17:51:30 +08:00
chengyongru 80bfcf4473 Merge branch 'main' into nightly 2026-04-17 14:22:54 +08:00
Mohamed ElkholyandXubin Ren ce5272c153 fix(transcription): honor api_base for OpenAI transcription provider
Complete the symmetry left by #3214: ChannelManager._resolve_transcription_base
already resolves providers.openai.api_base, but BaseChannel.transcribe_audio
instantiated OpenAITranscriptionProvider without forwarding it, and the provider
__init__ did not accept the parameter. Self-hosted OpenAI-compatible Whisper
endpoints (LiteLLM, vLLM, etc.) configured via config.json were therefore
ignored for the OpenAI backend.

- OpenAITranscriptionProvider.__init__ now accepts api_base with env fallback
  (OPENAI_TRANSCRIPTION_BASE_URL) matching the Groq pattern.
- BaseChannel.transcribe_audio forwards self.transcription_api_base to OpenAI.
- Tests mirror the existing Groq coverage: manager propagation for provider
  "openai", BaseChannel-to-provider argument passing, and provider default vs
  override for api_url.

Fully backward-compatible: when api_base is None and the env var is unset,
the default https://api.openai.com/v1/audio/transcriptions is used.

Refs #3213, follow-up to #3214.
2026-04-17 13:46:51 +08:00
Xubin RenandXubin Ren d57af5c1d1 test(channels): cover groq transcription api base propagation 2026-04-17 13:46:51 +08:00
flobo3andXubin Ren 0401ca9dbc fix: pass apiBase from config to GroqTranscriptionProvider 2026-04-17 13:46:51 +08:00
Xubin RenandXubin Ren cc5a666d5d review(dream): harden line-age annotation per review feedback
Follow-up to #3212, fully backward compatible:

- Extract the 14-day staleness threshold as `_STALE_THRESHOLD_DAYS` module
  constant and pass it into the Phase 1 prompt template as
  `{{ stale_threshold_days }}`. The number lived in three places before
  (code threshold, prompt instruction, docstring); now there is one.
- Add `DreamConfig.annotate_line_ages` (default True = current behavior)
  and propagate it through `Dream.__init__` and the gateway wiring in
  cli/commands.py. Gives users a knob to disable the feature without a
  code patch if an LLM reacts poorly to the `← Nd` suffix.
- Harden `_annotate_with_ages` against dirty working trees: when HEAD
  blob line count disagrees with the working-tree content length, skip
  annotation entirely instead of assigning ages to the wrong lines. The
  previous `i >= len(ages)` guard only handled one direction of the
  mismatch.
- Inline-comment the `max_iterations` 10→15 bump with a pointer to
  exp002 so future blame has context.
- Add 4 regression tests: end-to-end `← 30d` reaches prompt, 14/15
  threshold boundary, `annotate_line_ages=False` bypasses git entirely
  (verified via `assert_not_called`), length-mismatch defense, and
  template-var rendering.

Made-with: Cursor
2026-04-17 13:45:38 +08:00
chengyongruandXubin Ren 35f3084c03 feat(dream): per-line age annotations + dedup-aware prompt + max_iter=15
Three improvements to Dream's memory consolidation:

1. Per-line git-blame age annotations: MEMORY.md lines get `← Nd` suffixes
   (N>14) from dulwich annotate. SOUL.md/USER.md excluded as permanent.
   LLM uses content judgment, not just age, to decide what to prune.

2. Dedup-aware Phase 1 prompt: reframed as dual-task (extract facts +
   deduplicate existing files) with explicit redundancy patterns to scan for.
   Validated through 20 experiments (exp-002 prompt + max_iter=15 was best,
   averaging -1643 chars/5.4% compression per run).

3. Phase 1 analysis as commit body: dream git commits now include the full
   Phase 1 analysis for transparency via /dream-log.

4. max_iterations raised from 10 to 15: 30% improvement over 10 with no
   risk; 20 showed diminishing returns (exp-020: -701 vs exp-017: -1643).
2026-04-17 13:45:38 +08:00
Mohamed Elkholyandchengyongru 8a34677881 fix(transcription): honor api_base for OpenAI transcription provider
Complete the symmetry left by #3214: ChannelManager._resolve_transcription_base
already resolves providers.openai.api_base, but BaseChannel.transcribe_audio
instantiated OpenAITranscriptionProvider without forwarding it, and the provider
__init__ did not accept the parameter. Self-hosted OpenAI-compatible Whisper
endpoints (LiteLLM, vLLM, etc.) configured via config.json were therefore
ignored for the OpenAI backend.

- OpenAITranscriptionProvider.__init__ now accepts api_base with env fallback
  (OPENAI_TRANSCRIPTION_BASE_URL) matching the Groq pattern.
- BaseChannel.transcribe_audio forwards self.transcription_api_base to OpenAI.
- Tests mirror the existing Groq coverage: manager propagation for provider
  "openai", BaseChannel-to-provider argument passing, and provider default vs
  override for api_url.

Fully backward-compatible: when api_base is None and the env var is unset,
the default https://api.openai.com/v1/audio/transcriptions is used.

Refs #3213, follow-up to #3214.
2026-04-17 11:21:05 +08:00
Xubin RenandXubin Ren ddf2fe443e docs(readme): document Discord allowChannels config field
Mention the new allowChannels field in the Discord config example and
add a TIP bullet explaining the empty-list default (respond in all
channels) and that it composes with allowFrom.

Made-with: Cursor
2026-04-17 02:14:33 +08:00
Xubin RenandXubin Ren 459a4d7311 test(discord): cover allow_channels filtering in _should_accept_inbound
Locks in the two key boundaries of the new channel-based filter:

1. When an incoming channel id is in allow_channels, messages are forwarded.
2. When an incoming channel id is not in allow_channels, messages are
   silently dropped.

The empty-list backward-compatible path is already covered by every
existing test that omits allow_channels (default_factory=list).

Made-with: Cursor
2026-04-17 02:14:33 +08:00
48d430bf5e feat: add channel-based filtering for Discord
Add `allow_channels` config option to DiscordConfig that restricts
bot responses to specific Discord channels. When the list is empty
(default), the bot responds in all channels (backward compatible).

- Add `allow_channels: list[str]` field to DiscordConfig schema
- Add channel ID check in _handle_message_create after user filtering

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 02:14:33 +08:00
Xubin RenandXubin Ren 619c7fc20b docs(readme): reflect SSE streaming in OpenAI-compatible API section
The Behavior bullet previously claimed `stream=true` is not supported.
With this PR, /v1/chat/completions returns text/event-stream with
OpenAI-compatible delta chunks when stream=true, so flip the bullet
to describe the actual behavior instead of lying to readers.

Made-with: Cursor
2026-04-17 01:54:49 +08:00
whsandXubin Ren 4fce8d8b8d feat(api): add SSE streaming for /v1/chat/completions Wire up the existing on_stream/on_stream_end callbacks from process_direct() to emit OpenAI-compatible SSE chunks when stream=true. Non-streaming path is untouched. 2026-04-17 01:54:49 +08:00
Xubin Ren db78574cb8 docs(README): update auto compact section to clarify session file behavior and mental model 2026-04-16 17:20:38 +00:00
Xubin Ren 90b7d940e8 refactor(config): nest MyTool settings under tools.my (with legacy-key migration) 2026-04-16 15:58:20 +00:00
chengyongruandXubin Ren b51da93cbb feat(agent): add SelfTool for runtime self-inspection and configuration
Add a built-in tool that lets the agent inspect and modify its own
runtime state (model, iterations, context window, etc.).

Key features:
- inspect: view current config, usage stats, and subagent status
- modify: adjust parameters at runtime (protected by type/range validation)
- Subagent observability: inspect running subagent tasks (phase,
  iteration, tool events, errors) — subagents are no longer a black box
- Watchdog corrects out-of-bounds values on each iteration
- Enabled by default in read-only mode (self_modify: false)
- All changes are in-memory only; restart restores defaults
- Comprehensive test suite (90 tests)

Includes a self-awareness skill (always-on) with progressive disclosure:
SKILL.md for core rules, references/examples.md for detailed scenarios.
2026-04-16 23:44:26 +08:00
1304ff78cc perf(tools): cache ToolRegistry.get_definitions() between mutations
get_definitions() sorts tools on every LLM iteration for prompt cache
stability.  Cache the sorted result and invalidate on register/unregister
so the sort only runs when the tool set actually changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 21:52:36 +08:00
Xubin RenandXubin Ren 90ec11af4c test(channels): cover groq transcription api base propagation 2026-04-16 21:27:56 +08:00
flobo3andXubin Ren ca81b142b0 fix: pass apiBase from config to GroqTranscriptionProvider 2026-04-16 21:27:56 +08:00
Xubin RenandXubin Ren 7ce8f247a0 test(api): cover remote image URL rejection 2026-04-16 21:02:33 +08:00
Mohamed ElkholyandXubin Ren 54b48a7431 fix(api): prevent upload filename collisions, reject unsupported image URLs
Three fixes in the API upload handling:

1. Multipart uploads now prefix filenames with a UUID to prevent
   overwrites when two requests upload files with the same name.
2. JSON image_url content blocks with remote HTTPS URLs now return
   a 400 error instead of silently dropping the image.
3. Model validation runs for both JSON and multipart requests,
   fixing an inconsistency where multipart bypassed the check.
2026-04-16 21:02:33 +08:00
chengyongruandXubin Ren e1fdca7d40 fix(status): correct context percentage calculation and sync consolidator
- Pass resolved self.context_window_tokens to Consolidator instead of
  raw parameter that could be None, preventing consolidation failures
- Calculate percentage against input budget (ctx - max_completion - 1024)
  instead of raw context window, consistent with Consolidator/snip formulas
- Pass actual max_completion_tokens from provider to build_status_content
- Cap percentage display at 999 to prevent runaway values
- Add tests for budget-based percentage and cap behavior
2026-04-16 20:30:39 +08:00
Xubin RenandGitHub 92a5125108 Merge PR #3141: fix(skills): use yaml.safe_load for frontmatter parsing to handle multiline descriptions
fix(skills): use yaml.safe_load for frontmatter parsing to handle multiline descriptions
2026-04-16 20:07:15 +08:00
3280a195af perf(tools): cache ToolRegistry.get_definitions() between mutations
get_definitions() sorts tools on every LLM iteration for prompt cache
stability.  Cache the sorted result and invalidate on register/unregister
so the sort only runs when the tool set actually changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 15:59:26 +08:00
Xubin RenandXubin Ren a2f4090e41 fix(msteams): secure inbound defaults and ref persistence
Default Microsoft Teams inbound auth validation to enabled, update the README to match, and prevent denied senders from persisting conversation refs before allowlist checks pass.

Made-with: Cursor
2026-04-16 13:22:07 +08:00
chengyongruandXubin Ren abe0145f99 fix(msteams): harden availability check and migrate docs to README
- Check both jwt and cryptography in MSTEAMS_AVAILABLE guard so
  partial installs fail early with a clear message instead of at runtime
- Add aclose() to test FakeHttpClient so stop() won't crash
- Move MSTEAMS.md into README.md following the same details/summary
  pattern used by every other channel
- Note in README that validateInboundAuth defaults to false
2026-04-16 13:22:07 +08:00
chengyongruandXubin Ren 49223e639e fix(msteams): add auth warning and restore unrelated pyproject change
Warn when validate_inbound_auth is disabled (default) so operators are
aware the webhook accepts unverified requests.  Restore pymupdf to the
dev optional-dependencies group — its removal in the original PR was
unrelated to the Teams channel feature.
2026-04-16 13:22:07 +08:00
T3chC0wb0yandXubin Ren 818a095a90 style(msteams): hoist time import 2026-04-16 13:22:07 +08:00
T3chC0wb0yandXubin Ren ee99200341 refactor(msteams): remove business references 2026-04-16 13:22:07 +08:00
T3chC0wb0yandXubin Ren 9b4264fce2 refactor(msteams): remove FWDIOC references 2026-04-16 13:22:07 +08:00
T3chC0wb0yandXubin Ren fecef07c60 refactor(msteams): remove obsolete restart notify config 2026-04-16 13:22:07 +08:00
Bob JohnsonandXubin Ren 9f8774fbdd fix(msteams): remove hardcoded quote test fallback 2026-04-16 13:22:07 +08:00
chengyongruandXubin Ren 63753dbfea fix(msteams): remove optional deps from dev extras and gate tests
PyJWT and cryptography are optional msteams deps; they should not be
bundled into the generic dev install.  Tests now skip the entire file
when the deps are missing, following the dingtalk pattern.
2026-04-16 13:22:07 +08:00
Bob JohnsonandXubin Ren 4d795f74d5 Fix MSTeams PR review follow-ups 2026-04-16 13:22:07 +08:00
T3chC0wb0yandXubin Ren 824dcca5e2 Add Microsoft Teams channel on current nightly base 2026-04-16 13:22:07 +08:00
chengyongruandXubin Ren d64e963258 test(memory): add regression tests for missing cursor key
Cover read_unprocessed_history skipping cursorless entries and
_next_cursor safe fallback when last entry has no cursor.
2026-04-16 12:32:38 +08:00
chengyongruandXubin Ren 524c097f76 refactor(memory): simplify read_unprocessed_history cursor guard
Replace verbose loop with one-liner list comprehension using
e.get("cursor", 0) to handle missing cursor keys.
2026-04-16 12:32:38 +08:00
Jiajun XieandXubin Ren f4a7ad16aa fix(memory): handle missing cursor key in history entries
- Use .get('cursor') instead of direct dict access to prevent KeyError
- Skip entries without cursor and log a warning
- Fix _next_cursor fallback to safely check for cursor existence

Fixes #3190
2026-04-16 12:32:38 +08:00
chengyongruandchengyongru 43bd8aac8d fix(msteams): harden availability check and migrate docs to README
- Check both jwt and cryptography in MSTEAMS_AVAILABLE guard so
  partial installs fail early with a clear message instead of at runtime
- Add aclose() to test FakeHttpClient so stop() won't crash
- Move MSTEAMS.md into README.md following the same details/summary
  pattern used by every other channel
- Note in README that validateInboundAuth defaults to false
2026-04-16 11:08:53 +08:00
chengyongruandchengyongru b48f497f8d fix(msteams): add auth warning and restore unrelated pyproject change
Warn when validate_inbound_auth is disabled (default) so operators are
aware the webhook accepts unverified requests.  Restore pymupdf to the
dev optional-dependencies group — its removal in the original PR was
unrelated to the Teams channel feature.
2026-04-16 11:08:53 +08:00
T3chC0wb0yandchengyongru 2f3a37cf8e style(msteams): hoist time import 2026-04-16 11:08:53 +08:00
T3chC0wb0yandchengyongru 7545b58a00 refactor(msteams): remove business references 2026-04-16 11:08:53 +08:00
T3chC0wb0yandchengyongru 626903dd47 refactor(msteams): remove FWDIOC references 2026-04-16 11:08:53 +08:00
T3chC0wb0yandchengyongru e625e47a0a refactor(msteams): remove obsolete restart notify config 2026-04-16 11:08:53 +08:00
Bob Johnsonandchengyongru af7fa5bdf9 fix(msteams): remove hardcoded quote test fallback 2026-04-16 11:08:53 +08:00
chengyongruandchengyongru 2259eb7f2a fix(msteams): remove optional deps from dev extras and gate tests
PyJWT and cryptography are optional msteams deps; they should not be
bundled into the generic dev install.  Tests now skip the entire file
when the deps are missing, following the dingtalk pattern.
2026-04-16 11:08:53 +08:00
Bob Johnsonandchengyongru 8925482f93 Fix MSTeams PR review follow-ups 2026-04-16 11:08:53 +08:00
T3chC0wb0yandchengyongru c2c2351ee2 Add Microsoft Teams channel on current nightly base 2026-04-16 11:08:53 +08:00
Xubin RenandXubin Ren 2b8e90d8fd test(config): cover LM Studio nullable api key 2026-04-16 02:49:54 +08:00
Soham BhattacharyaandXubin Ren 41a1b0058d Add support for nullable API keys and LM Studio 2026-04-16 02:49:54 +08:00
Xubin Ren d46c1b14b0 docs: update .gitignore 2026-04-15 18:17:18 +00:00
Leo fuandXubin Ren 2c0cd085a4 fix(discord): remove duplicate channel_id assignment in message handler
channel_id is already assigned from self._channel_key(message.channel)
earlier in the same function. The second identical assignment on line 453
is dead code left over from a copy-paste.
2026-04-16 02:11:13 +08:00
Xubin RenandXubin Ren a6ea06e6bf docs(providers): explain MiniMax thinking endpoint
Document why MiniMax thinking mode uses a separate Anthropic-compatible provider and list the matching base URLs. Add a small registry test so the new provider stays wired to the expected backend and API key.

Made-with: Cursor
2026-04-16 01:00:45 +08:00
AishtandXubin Ren d0a282e766 feat(provider): add MiniMax Anthropic endpoint for thinking mode
- Add minimax_anthropic provider using Anthropic-compatible endpoint
- Endpoint: https://api.minimax.io/anthropic
- Supports reasoning_effort parameter for thinking mode (low/medium/high/adaptive)
- Uses same MINIMAX_API_KEY as existing minimax provider
2026-04-16 01:00:45 +08:00
chengyongruandchengyongru 92be47247e feat(agent): add SelfTool for runtime self-inspection and configuration
Add a built-in tool that lets the agent inspect and modify its own
runtime state (model, iterations, context window, etc.).

Key features:
- inspect: view current config, usage stats, and subagent status
- modify: adjust parameters at runtime (protected by type/range validation)
- Subagent observability: inspect running subagent tasks (phase,
  iteration, tool events, errors) — subagents are no longer a black box
- Watchdog corrects out-of-bounds values on each iteration
- Enabled by default in read-only mode (self_modify: false)
- All changes are in-memory only; restart restores defaults
- Comprehensive test suite (90 tests)

Includes a self-awareness skill (always-on) with progressive disclosure:
SKILL.md for core rules, references/examples.md for detailed scenarios.
2026-04-16 00:37:34 +08:00
Jiajun XieandXubin Ren e18eab8054 fix(cron): respect deliver flag before message tool check
When deliver: false is set in cron job payload, suppress all output even
when agent calls message tool during the turn.

Closes #3115
2026-04-15 23:53:08 +08:00
04cbandXubin Ren eacc9fbb5f refactor(providers): drop unreachable GenerationSettings fallback 2026-04-15 23:52:38 +08:00
04cbandXubin Ren 54f7ad3752 fix(providers): guard chat_with_retry against explicit None max_tokens (#3102) 2026-04-15 23:52:38 +08:00
Jiajun Xieandchengyongru 76683dd18a fix(cron): respect deliver flag before message tool check
When deliver: false is set in cron job payload, suppress all output even
when agent calls message tool during the turn.

Closes #3115
2026-04-15 17:42:36 +08:00
chengyongruandGitHub 015833e34b Merge branch 'main' into fix/skills-yaml-frontmatter 2026-04-15 16:56:23 +08:00
dongzeyu001andXubin Ren 6829b8b475 unit test fix 2026-04-15 16:51:02 +08:00
dongzeyu001andXubin Ren cbd2315d76 unit test fix 2026-04-15 16:51:02 +08:00
dongzeyu001andXubin Ren cf47fa7d23 add test for wecom mixed msg parse fix 2026-04-15 16:51:02 +08:00
dongzeyu001andXubin Ren 8572b7478f Fix wecom mix msg parse 2026-04-15 16:51:02 +08:00
chengyongruandXubin Ren 6fbada5363 refactor(context): deduplicate system prompt — markdown skills index, skip template MEMORY.md
- Convert skills summary from verbose XML (4-5 lines/skill) to compact
  markdown list (1 line/skill) with inline path for read_file lookup
- Exclude always-loaded skills (e.g. memory) from the skills index to
  avoid duplicating content already in the Active Skills section
- Skip injecting the Memory section when MEMORY.md still matches the
  bundled template (i.e. Dream hasn't populated it yet)
2026-04-15 15:49:30 +08:00
Xubin Ren 5683c79a6e chore: update README with new release notes of v0.1.5.post1 2026-04-14 19:01:43 +00:00
Xubin Ren 6483071485 chore: update version to 0.1.5.post1 2026-04-14 18:51:04 +00:00
Xubin Ren 1a5a16d1f3 chore: update README with recent news entries 2026-04-14 18:19:30 +00:00
razzhandXubin Ren 9e2278826f feat(provider): enable Kimi thinking via extra_body for k2.5 and k2.6
- Inject `thinking={"type": "enabled|disabled"}` via extra_body for
  Kimi thinking-capable models (kimi-k2.5, k2.6-code-preview).
- Add _is_kimi_thinking_model helper to handle both bare slugs and
  OpenRouter-style prefixed names (e.g. moonshotai/kimi-k2.5).
- reasoning_effort="minimal" maps to disabled; any other value enables it.
- Add tests for enabled/disabled states and OpenRouter prefix handling.
2026-04-15 01:59:32 +08:00
Xubin RenandXubin Ren a0812ad60e test: cover retry termination notifications
Lock the new interaction-channel retry termination hints so both exhausted standard retries and persistent identical-error stops keep emitting the final progress message.

Made-with: Cursor
2026-04-15 01:55:57 +08:00
aiguozhi123456andXubin Ren ec14933aa1 fix: add retry termination notification to interaction channel 2026-04-15 01:55:57 +08:00
Xubin RenandXubin Ren 25ded8e747 test: cover active task count in status
Lock the /status task counter to the actual stop scope by asserting it sums unfinished dispatch tasks with running subagents for the current session.

Made-with: Cursor
2026-04-15 01:49:42 +08:00
aiguozhi123456andXubin Ren 634f4b45c1 feat: show active task count in /status output 2026-04-15 01:49:42 +08:00
Xubin RenandXubin Ren b60e8dc0ba test: cover missing tool-call arguments normalization
Lock the strict-provider sanitization path so assistant tool calls without function.arguments are normalized to {} instead of being forwarded as missing values.

Made-with: Cursor
2026-04-15 01:37:41 +08:00
Michael-lhhandXubin Ren f293ff7f18 fix: normalize tool-call arguments for strict providers
Ensure assistant tool-call function.arguments is always emitted as valid JSON text so strict OpenAI-compatible backends (including Alibaba code models) do not reject requests. Add regressions for dict and malformed-string argument payloads in message sanitization.

Made-with: Cursor
2026-04-15 01:37:41 +08:00
Xubin RenandXubin Ren 1f33df1ea6 fix: preserve empty dict allow_from handling
Keep dict-backed channel configs compatible with both allow_from and allowFrom without losing empty-list semantics, and add focused regression coverage for the allow-list boundary.

Made-with: Cursor
2026-04-15 01:26:51 +08:00
samyandXubin Ren 73cf9a220b fix: handle dict config in is_allowed() and _validate_allow_from()
getattr() on a dict never finds custom keys — it only searches
object attributes, not dict keys. When channel config is loaded as
a Pydantic extra field (which is a plain dict), getattr(config,
'allow_from', []) always returns the default [], causing all access
to be denied regardless of the allowFrom configuration.

Fix both is_allowed() and _validate_allow_from() to use isinstance
checks, falling back to dict.get() for dict configs while preserving
getattr() for object-style configs.
2026-04-15 01:26:51 +08:00
Xubin Ren 89bf5d29d1 fix: reduce CLI streaming flicker and show model in welcome line 2026-04-14 13:38:06 +00:00
Xubin RenandGitHub cbc1161f75 Merge PR #2938: feat(api): support file uploads via JSON base64 and multipart/form-data
feat(api): support file uploads via JSON base64 and multipart/form-data
2026-04-14 21:23:28 +08:00
Xubin Ren c937c07178 fix: two bugs in document extraction pipeline
Bug 1: _drain_pending did not call extract_documents on follow-up
messages arriving mid-turn. Documents attached to queued messages were
silently dropped because _build_user_content only handles images.
Fix: call extract_documents before _build_user_content in _drain_pending.

Bug 2: extract_documents read the entire file into memory (up to 50 MB)
just to check 16 bytes of magic header for MIME detection.
Fix: read only the first 16 bytes via open()+read(16) instead of
Path.read_bytes().

Added regression tests for both bugs.

Made-with: Cursor
2026-04-14 13:15:04 +00:00
Xubin Ren 92d6fca323 refactor: centralize document extraction in AgentLoop._process_message
Move extract_documents() to nanobot.utils.document as a reusable helper
and call it once in AgentLoop._process_message, the single entry point
for all message processing (API + all channels).

This replaces the previous API-only _extract_documents() in server.py,
ensuring Telegram, Feishu, Slack, WeChat, and all other channels also
benefit from automatic document text extraction.

Adds a configurable max_file_size guard (default 50 MB) to skip
oversized files gracefully, preventing unbounded memory/CPU usage
from channel-downloaded attachments.

- server.py: removed _extract_documents and related imports
- document.py: added extract_documents() with size limit
- loop.py: calls extract_documents() at the top of _process_message
- Tests updated: 70 related tests pass

Made-with: Cursor
2026-04-14 13:10:03 +00:00
Xubin Ren 47f5795708 refactor: move document extraction from ContextBuilder to API layer
ContextBuilder._build_user_content now only handles images (its original
responsibility).  Document text extraction (PDF, DOCX, XLSX, PPTX) is
performed by the new _extract_documents() helper in server.py, called
before process_direct().  This keeps the core context builder free of
format-specific dependencies and makes the API boundary the single place
where uploaded files are pre-processed.

Tests updated to reflect the new responsibility boundary.

Made-with: Cursor
2026-04-14 13:00:59 +00:00
Xubin Ren 2502fc616b Merge origin/main into feat/api-file-upload
Keep the API file upload branch current with main, enforce the documented JSON base64 per-file limit, and avoid leaking document extraction error strings into user prompts.

Made-with: Cursor
2026-04-14 12:29:43 +00:00
Xubin RenandXubin Ren 0a51344483 fix(slack): keep cross-target sends out of origin threads
When Slack resolves a named target to another conversation, do not reuse the origin thread timestamp on the destination send, and keep reaction cleanup anchored to the source conversation.

Made-with: Cursor
2026-04-14 20:19:48 +08:00
yeyitechandXubin Ren 873be5180b feat(slack): resolve named message targets 2026-04-14 20:19:48 +08:00
chengyongruandchengyongru dec26396ed fix(feishu): remove resuming to avoid 10-min streaming card timeout
Feishu streaming cards auto-close after 10 minutes from creation,
regardless of update activity. With resuming enabled, a single card
lives across multiple tool-call rounds and can exceed this limit,
causing the final response to be silently lost.

Remove the _resuming logic from send_delta so each tool-call round
gets its own short-lived streaming card (well under 10 min). Add a
fallback that sends a regular interactive card when the final
streaming update fails.
2026-04-14 17:01:26 +08:00
chengyongruandXubin Ren 0adce5405b fix(feishu): remove resuming to avoid 10-min streaming card timeout
Feishu streaming cards auto-close after 10 minutes from creation,
regardless of update activity. With resuming enabled, a single card
lives across multiple tool-call rounds and can exceed this limit,
causing the final response to be silently lost.

Remove the _resuming logic from send_delta so each tool-call round
gets its own short-lived streaming card (well under 10 min). Add a
fallback that sends a regular interactive card when the final
streaming update fails.
2026-04-14 16:53:42 +08:00
yanghan-cyber a1b544fd23 fix(skills): use yaml.safe_load for frontmatter parsing to handle multiline descriptions
The hand-rolled line-by-line YAML parser treated each line independently,
so YAML multiline scalars (folded `>` and literal `|`) were captured as
the literal characters ">" or "|" instead of the actual text content.
2026-04-14 15:29:59 +08:00
Xubin RenandGitHub 12c12869b4 Merge PR #2625: feat: add HTTP health endpoint on gateway port
feat: add HTTP health endpoint on gateway port
2026-04-14 15:24:35 +08:00
Xubin Ren e4b3f9bd28 security(gateway): keep health endpoint local by default
Bind the gateway health listener to localhost by default and reduce the probe response to a minimal status payload so accidental public exposure leaks less information.

Made-with: Cursor
2026-04-14 07:19:38 +00:00
Xubin Ren 4999e2f734 Merge origin/main into feat/health-endpoint
Keep the gateway health endpoint patch current with the latest gateway runtime changes, and lock the new HTTP routes in with CLI regression coverage and README guidance.

Made-with: Cursor
2026-04-14 06:32:31 +00:00
yeyitechandXubin Ren 65a15f39ee test(loop): cover /stop checkpoint recovery 2026-04-14 14:15:22 +08:00
yeyitechandXubin Ren ee061f0595 fix(web): serialize duckduckgo search calls 2026-04-14 14:10:06 +08:00
chengyongru 4c684540c5 Merge remote-tracking branch 'origin/main' into nightly 2026-04-14 00:37:21 +08:00
Xubin RenandXubin Ren a38bc637bd fix(runner): preserve injection flag after max-iteration drain
Keep late follow-up injections observable when they are drained during max-iteration shutdown so loop-level response suppression still makes the right decision.

Made-with: Cursor
2026-04-14 00:30:30 +08:00
chengyongruandXubin Ren a1e1eed2f1 refactor(runner): consolidate all injection drain paths and deduplicate tests
- Migrate "after tools" inline drain to use _try_drain_injections,
  completing the refactoring (all 6 drain sites now use the helper).
- Move checkpoint emission into _try_drain_injections via optional
  iteration parameter, eliminating the leaky split between helper
  and caller for the final-response path.
- Extract _make_injection_callback() test helper to replace 7
  identical inject_cb function bodies.
- Add test_injection_cycle_cap_on_error_path to verify the cycle
  cap is enforced on error exit paths.
2026-04-14 00:30:30 +08:00
chengyongruandXubin Ren d849a3fa06 fix(agent): drain injection queue on error/edge-case exit paths
When the agent runner exits due to LLM error, tool error, empty response,
or max_iterations, it breaks out of the iteration loop without draining
the pending injection queue. This causes leftover messages to be
re-published as independent inbound messages, resulting in duplicate or
confusing replies to the user.

Extract the injection drain logic into a `_try_drain_injections` helper
and call it before each break in the error/edge-case paths. If injections
are found, continue the loop instead of breaking. For max_iterations
(where the loop is exhausted), drain injections to prevent re-publish
without continuing.
2026-04-14 00:30:30 +08:00
chengyongruandXubin Ren 3c06db7e4e fix(log): remove noisy no-op logs from auto-compact
Remove two debug log lines that fire on every idle channel check:
- "scheduling archival" (logged before knowing if there's work)
- "skipping, no un-consolidated messages" (the common no-op path)

The meaningful "archived" info log (only on real work) is preserved.
2026-04-13 20:14:58 +08:00
chengyongruandchengyongru b3288fbc87 fix(log): only log auto-compact when messages are actually archived 2026-04-13 16:52:47 +08:00
chengyongruandchengyongru b311759e87 fix(log): remove noisy no-op logs from auto-compact
Remove two debug log lines that fire on every idle channel check:
- "scheduling archival" (logged before knowing if there's work)
- "skipping, no un-consolidated messages" (the common no-op path)

The meaningful "archived" info log (only on real work) is preserved.
2026-04-13 16:09:42 +08:00
haosenwang1018andXubin Ren d33bf22e91 docs(provider): clarify responses api routing 2026-04-13 15:59:36 +08:00
haosenwang1018andXubin Ren 85c7996766 docs(api): clarify cross-channel message delivery 2026-04-13 15:59:36 +08:00
chengyongruandXubin Ren ac714803f6 fix(provider): recover trailing assistant message as user to prevent empty request
When a subagent result is injected with current_role="assistant",
_enforce_role_alternation drops the trailing assistant message, leaving
only the system prompt. Providers like Zhipu/GLM reject such requests
with error 1214 ("messages parameter invalid"). Now the last popped
assistant message is recovered as a user message when no user/tool
messages remain.
2026-04-13 12:54:39 +08:00
chengyongruandXubin Ren becaff3e9d fix(agent): skip auto-compact for sessions with active agent tasks
Prevent proactive compaction from archiving sessions that have an
in-flight agent task, avoiding mid-turn context truncation when a
task runs longer than the idle TTL.
2026-04-13 12:51:37 +08:00
chengyongruandchengyongru 89ea2375fd fix(provider): recover trailing assistant message as user to prevent empty request
When a subagent result is injected with current_role="assistant",
_enforce_role_alternation drops the trailing assistant message, leaving
only the system prompt. Providers like Zhipu/GLM reject such requests
with error 1214 ("messages parameter invalid"). Now the last popped
assistant message is recovered as a user message when no user/tool
messages remain.
2026-04-13 12:01:45 +08:00
chengyongruandchengyongru 62bd54ac4a fix(agent): skip auto-compact for sessions with active agent tasks
Prevent proactive compaction from archiving sessions that have an
in-flight agent task, avoiding mid-turn context truncation when a
task runs longer than the idle TTL.
2026-04-13 12:01:29 +08:00
Xubin RenandXubin Ren 6484c7c47a fix(agent): close interrupted early-persisted user turns
Track text-only user messages that were flushed before the turn loop completes, then materialize an interrupted assistant placeholder on the next request so session history stays legal and later turns do not skip their own assistant reply.

Made-with: Cursor
2026-04-13 10:26:09 +08:00
Xubin RenandXubin Ren b964a894d2 test(agent): cover early user-message persistence
Use session.add_message for the pre-turn user-message flush and add focused regression tests for crash-time persistence and duplicate-free successful saves.

Made-with: Cursor
2026-04-13 10:26:09 +08:00
nikubeandXubin Ren ea94a9c088 fix(agent): persist user message before running turn loop
The existing runtime_checkpoint mechanism preserves the in-flight
assistant/tool state if the process dies mid-turn, but the triggering
user message is only written to session history at the end of the turn
via _save_turn(). If the worker is killed (OOM, SIGKILL, a self-
triggered systemctl restart, container eviction, etc.) before the turn
completes, the user's message is silently lost: on restart, the session
log only shows the interrupted assistant turn without any record of
what the user asked. Any recovery tooling built on top of session logs
cannot reply because it has no prompt to reply to.

This patch appends the incoming user message to the session and flushes
it to disk immediately after the session is loaded and before the agent
loop runs, then adjusts the _save_turn skip offset so the final
persistence step does not duplicate it.

Limited to textual content (isinstance(msg.content, str)); list-shaped
content (media blocks) still flows through _save_turn's sanitization at
end of turn, preserving existing behavior for those cases.
2026-04-13 10:26:09 +08:00
Xubin RenandXubin Ren 49355b2bd6 test(tools): lock non-object parameter validation
Add focused registry coverage so the new read_file/read_write parameter guard stays actionable without changing generic validation behavior for other tools.

Made-with: Cursor
2026-04-13 09:55:05 +08:00
ramonpaoloandXubin Ren 830644c352 fix: add guard for non-dict tool call parameters
- Add type validation in registry.prepare_call() to catch list/other invalid params
- Add logger.warning() in provider layer when non-dict args detected
- Works for OpenAI-compatible and Anthropic providers
- Registry returns clear error hint for model to self-correct
2026-04-13 09:55:05 +08:00
haosenwang1018andXubin Ren 92ef594b6a fix(mcp): hint on stdio protocol pollution 2026-04-13 09:41:55 +08:00
haosenwang1018andXubin Ren 3573109408 fix(provider): preserve static error helper compatibility 2026-04-13 09:37:31 +08:00
haosenwang1018andXubin Ren c68b3edb9d fix(provider): clarify local 502 recovery hints 2026-04-13 09:37:31 +08:00
bahtyaandXubin Ren f879d81b28 fix(channels/qq): propagate network errors in send() instead of swallowing
The catch-all except Exception in QQ send() was swallowing
aiohttp.ClientError and OSError that _send_media correctly
re-raises. Add explicit catch for network errors before the
generic handler.
2026-04-13 00:30:45 +08:00
bahtyaandXubin Ren fa98524944 fix(channels): prevent retry amplification and silent message loss across channels
Audited all channel implementations for overly broad exception handling
that causes retry amplification or silent message loss during network
errors. This is the same class of bug as #3050 (Telegram _send_text).

Fixes by channel:

Telegram (send_delta):
- _stream_end path used except Exception for HTML edit fallback
- Network errors (TimedOut, NetworkError) triggered redundant plain
  text edit, doubling connection demand during pool exhaustion
- Changed to except BadRequest, matching the _send_text fix

Discord:
- send() caught all exceptions without re-raising
- ChannelManager._send_with_retry() saw successful return, never retried
- Messages silently dropped on any send failure
- Added raise after error logging

DingTalk:
- _send_batch_message() returned False on all exceptions including
  network errors — no retry, fallback text sent unnecessarily
- _read_media_bytes() and _upload_media() swallowed transport errors,
  causing _send_media_ref() to cascade through doomed fallback attempts
- Added except httpx.TransportError handlers that re-raise immediately

WeChat:
- Media send failure triggered text fallback even for network errors
- During network issues: 3×(media + text) = 6 API calls per message
- Added specific catches: TimeoutException/TransportError re-raise,
  5xx HTTPStatusError re-raises, 4xx falls back to text

QQ:
- _send_media() returned False on all exceptions
- Network errors triggered fallback text instead of retry
- Added except (aiohttp.ClientError, OSError) that re-raises

Tests: 331 passed (283 existing + 48 new across 5 channel test files)

Fixes: #3054
Related: #3050, #3053
2026-04-13 00:30:45 +08:00
bahtyaandXubin Ren 7e91aecd7d fix(telegram): narrow exception catch in _send_text to prevent retry amplification
Previously _send_text() caught all exceptions (except Exception) when
sending HTML-formatted messages, falling back to plain text even for
network errors like TimedOut and NetworkError. This caused connection
demand to double during pool exhaustion scenarios (3 retries × 2
fallback attempts = 6 calls per message instead of 3).

Now only catches BadRequest (HTML parse errors), letting network errors
propagate immediately to the retry layer where they belong.

Fixes: HKUDS/nanobot#3050
2026-04-13 00:30:45 +08:00
Xubin RenandXubin Ren 217e1fc957 test(retry): lock in-place image fallback behavior
Add a focused regression test for the successful no-image retry path so the original message history stays stripped after fallback and the repeated retry loop cannot silently return.

Made-with: Cursor
2026-04-12 20:10:06 +08:00
yanghan-cyberandXubin Ren b261201985 fix(retry): strip images in-place to prevent repeated error-retry cycles
When a non-transient LLM error occurs with image content, the retry
mechanism strips images from a copy but never updates the original
conversation history. Subsequent iterations rebuild context from the
unmodified history, causing the same error-retry cycle to repeat
every iteration until max_iterations is reached.

Add _strip_image_content_inplace() that mutates the original message
content lists in-place after a successful no-image retry, so callers
sharing those references (e.g. the runner's conversation history)
also see the stripped version.
2026-04-12 20:10:06 +08:00
Xubin RenandXubin Ren 7a7f5c9689 fix(dream): use valid builtin skill template paths
Point Dream skill creation at a readable builtin skill-creator template, keep skill writes rooted at the workspace, and document the new skill discovery behavior in README.

Made-with: Cursor
2026-04-12 16:49:55 +08:00
2a243bfe4f feat(agent): integrate skill discovery into Dream consolidation
Instead of a separate skill discovery system, extend Dream's two-phase
pipeline to also detect reusable behavioral patterns from conversation
history and generate SKILL.md files.

Phase 1 gains a [SKILL] output type for pattern detection.
Phase 2 gains write_file (scoped to skills/) and read access to builtin
skills, enabling it to check for duplicates and follow skill-creator's
format conventions before creating new skills.

Inspired by PR #3039 by @wanghesong2019.

Co-authored-by: wanghesong2019 <wanghesong2019@users.noreply.github.com>
2026-04-12 16:49:55 +08:00
Xubin RenandXubin Ren 5dc238c7ef fix(shell): allow read-only copies from internal state files
Keep the new exec guard focused on writes to history.jsonl and .dream_cursor while still allowing read-only copy operations out of those files.

Made-with: Cursor
2026-04-12 16:38:55 +08:00
04cbandXubin Ren 3f59bd1443 fix(shell): reject LLM-supplied working_dir outside workspace (#2826) 2026-04-12 16:38:55 +08:00
04cbandXubin Ren 00fb491bc9 fix(shell): block exec writes to history.jsonl and cursor files (#2989) 2026-04-12 16:38:55 +08:00
Xubin RenandGitHub a81e4c1791 Merge PR #2959: feat(skills): add disabled_skills config to exclude skills from loading
feat(skills): add disabled_skills config to exclude skills from loading
2026-04-12 10:46:50 +08:00
Xubin Ren a142788da9 docs(readme): document disabledSkills config
Explain the new agents.defaults.disabledSkills option so users can discover and configure skill exclusion from the main agent and subagents.

Made-with: Cursor
2026-04-12 02:42:52 +00:00
Xubin Ren e229c2ebc0 fix(pr): remove internal .docs file from PR
Keep the local review note out of the GitHub diff while preserving the actual code and test changes for this PR.

Made-with: Cursor
2026-04-12 02:21:46 +00:00
Xubin Ren 09c238ca0f Merge origin/main into pr-2959
Resolve the config plumbing conflicts and keep disabled skill filtering consistent for subagent prompts after syncing with main.

Made-with: Cursor
2026-04-12 02:02:39 +00:00
Dianqi JiandXubin Ren ee946d96ca feat(channels/feishu): add domain config for Lark global support
Add 'domain' field to FeishuConfig (Literal['feishu', 'lark'], default 'feishu').
Pass domain to lark.Client.builder() and lark.ws.Client to support Lark global
(open.larksuite.com) in addition to Feishu China (open.feishu.cn).
Existing configs default to 'feishu' for backward compatibility.

Also add documentation for domain field in README.md and add tests for
domain config.
2026-04-12 09:56:17 +08:00
Xubin RenandGitHub a70928cc5c Merge PR #3045: fix(agent): preserve tool results on fatal error to prevent orphan tool_calls
fix(agent): preserve tool results on fatal error to prevent orphan tool_calls (#2943)
2026-04-11 23:08:03 +08:00
laylaandGitHub f25cdb7138 Merge branch 'main' into fix/tool-call-result-order-2943 2026-04-11 22:00:07 +08:00
04cb 4cd4ed8ada fix(agent): preserve tool results on fatal error to prevent orphan tool_calls (#2943) 2026-04-11 21:50:44 +08:00
chengyongruandXubin Ren 9f433cab01 fix(wecom): use reply_stream for progress messages to avoid errcode=40008
The plain reply() uses cmd="reply" which does not support "text" msgtype
and causes WeCom API to return errcode=40008 (invalid message type).
Unify both progress and final text messages to use reply_stream()
(cmd="aibot_respond_msg"), differentiating via finish flag.

Fixes #2999
2026-04-11 21:47:19 +08:00
chengyongruandXubin Ren 0d03f10fa0 test(channels): add media support tests for QQ and WeCom channels
Cover helpers (sanitize_filename, guess media type), outbound send
(exception handling, media-then-text order, fallback), inbound message
processing (attachments, dedup, empty content), _post_base64file
payload filtering, and WeCom upload/download flows.
2026-04-11 21:47:19 +08:00
chengyongruandXubin Ren f6f712a2ae fix(wecom): harden upload/download, extract media type helper
- Use asyncio.to_thread for file I/O to avoid blocking event loop
- Add 200MB upload size limit with early rejection
- Fix file handle leak by using context manager
- Use memoryview for upload chunking to reduce peak memory
- Add inbound download size check to prevent OOM
- Use asyncio.to_thread for write_bytes in download path
- Extract inline media_type detection to _guess_wecom_media_type()
2026-04-11 21:47:19 +08:00
chengyongruandXubin Ren f900e4f259 fix(wecom): harden upload and inbound media handling
- Use asyncio.to_thread for file I/O to avoid blocking event loop
- Add 200MB upload size limit with early rejection
- Fix file handle leak by using context manager
- Free raw bytes early after chunking to reduce memory pressure
- Add file attachments to media_paths (was text-only, inconsistent with image)
- Use robust _sanitize_filename() instead of os.path.basename() for path safety
- Remove re-raise in send() for consistency with QQ channel
- Fix truncated media_id logging for short IDs
2026-04-11 21:47:19 +08:00
gem12andXubin Ren 48f6bbd256 feat(channels): Add full media support for QQ and WeCom channels
QQ channel improvements (on top of nightly):
- Add top-level try/except in _on_message and send() for resilience
- Use defensive getattr() for attachment attributes (botpy version compat)
- Skip file_name for image uploads to avoid QQ rendering as file attachment
- Extract only file_info from upload response to avoid extra fields
- Handle protocol-relative URLs (//...) in attachment downloads

WeCom channel improvements:
- Add _upload_media_ws() for WebSocket 3-step media upload protocol
- Send media files (image/video/voice/file) via WeCom rich media API
- Support progress messages (plain reply) vs final response (streaming)
- Support proactive send when no frame available (cron push)
- Pass media_paths to message bus for downstream processing
2026-04-11 21:47:19 +08:00
Xubin RenandXubin Ren cf8381f517 feat(agent): enhance message injection handling and content merging 2026-04-11 21:43:23 +08:00
Xubin RenandXubin Ren f6c39ec946 feat(agent): enhance session key handling for follow-up messages 2026-04-11 21:43:23 +08:00
chengyongruandXubin Ren 36d2a11e73 feat(agent): mid-turn message injection for responsive follow-ups (#2985)
* feat(agent): add mid-turn message injection for responsive follow-ups

Allow user messages sent during an active agent turn to be injected
into the running LLM context instead of being queued behind a
per-session lock. Inspired by Claude Code's mid-turn queue drain
mechanism (query.ts:1547-1643).

Key design decisions:
- Messages are injected as natural user messages between iterations,
  no tool cancellation or special system prompt needed
- Two drain checkpoints: after tool execution and after final LLM
  response ("last-mile" to prevent dropping late arrivals)
- Bounded by MAX_INJECTION_CYCLES (5) to prevent consuming the
  iteration budget on rapid follow-ups
- had_injections flag bypasses _sent_in_turn suppression so follow-up
  responses are always delivered

Closes #1609

* fix(agent): harden mid-turn injection with streaming fix, bounded queue, and message safety

- Fix streaming protocol violation: Checkpoint 2 now checks for injections
  BEFORE calling on_stream_end, passing resuming=True when injections found
  so streaming channels (Feishu) don't prematurely finalize the card
- Bound pending queue to maxsize=20 with QueueFull handling
- Add warning log when injection batch exceeds _MAX_INJECTIONS_PER_TURN
- Re-publish leftover queue messages to bus in _dispatch finally block to
  prevent silent message loss on early exit (max_iterations, tool_error, cancel)
- Fix PEP 8 blank line before dataclass and logger.info indentation
- Add 12 new tests covering drain, checkpoints, cycle cap, queue routing,
  cleanup, and leftover re-publish
2026-04-11 21:43:23 +08:00
Jiajun XieandXubin Ren f5640d69fe fix(feishu): improve voice message download with detailed logging
- Add explicit error logging for missing file_key and message_id
- Add logging for download failures
- Change audio extension from .opus to .ogg for better Whisper compatibility
- Feishu voice messages are opus in OGG container; .ogg is more widely recognized
2026-04-11 20:48:35 +08:00
Xubin RenandGitHub e0b9edf985 Merge PR #3017: feat(tool): improve file editing and add notebook tool
feat(tool): improve file editing and add notebook tool
2026-04-11 18:02:25 +08:00
Xubin RenandGitHub e7bbbe98f4 Merge PR #3019: fix(mcp): support multiple MCP servers
fix(mcp): support multiple MCP servers
2026-04-11 17:35:47 +08:00
Xubin Ren 322142f7ad Merge origin/main into main 2026-04-11 09:32:05 +00:00
Xubin RenandXubin Ren b959ae6d89 test(web): cover Kagi search provider
Add focused coverage for the Kagi web search provider, including the request format and the DuckDuckGo fallback when no API key is configured.
2026-04-11 16:53:05 +08:00
Mike TerharandXubin Ren 74dbce3770 add kagi info to README 2026-04-11 16:53:05 +08:00
Mike TerharandXubin Ren d3aa209cf6 add kagi web search tool 2026-04-11 16:53:05 +08:00
Xubin Ren 5bb7f77b80 feat(tests): add regression test for timer execution to prevent store rollback during job execution 2026-04-11 08:43:25 +00:00
Xubin RenandGitHub 1263869c0a Merge PR #3038: fix(cron): guard _load_store against reentrant reload during job execution
fix(cron): guard _load_store against reentrant reload during job execution
2026-04-11 16:28:47 +08:00
Xubin Ren 8fe8537505 Merge origin/main into fix/cron-reentrant-load-store 2026-04-11 08:25:47 +00:00
weitongtongandXubin Ren e0ba568089 fix(cron): 修复固定间隔任务因 store 并发替换导致的重复执行
_on_timer 中 await _execute_job 让出控制权期间,前端轮询触发的
list_jobs 调用 _load_store 从磁盘重新加载覆盖 self._store,
已执行任务的状态被旧值回退,导致再次触发。
引入 _timer_active 标志位,在任务执行期间阻止并发 _load_store
替换 store。同时修复 store 为空时未重新 arm timer 的问题。

Made-with: Cursor
2026-04-11 16:15:01 +08:00
Xubin RenandXubin Ren 5932482d01 refactor(agent): rename auto compact module
Rename the auto compact module to autocompact.py for a cleaner path while keeping the AutoCompact type and behavior unchanged. Update the agent loop import to match.
2026-04-11 15:56:41 +08:00
Xubin RenandXubin Ren 84e840659a refactor(config): rename auto compact config key
Prefer the more user-friendly idleCompactAfterMinutes name for auto compact while keeping sessionTtlMinutes as a backward-compatible alias. Update tests and README to document the retained recent-context behavior and the new preferred key.
2026-04-11 15:56:41 +08:00
Xubin RenandXubin Ren 1cb28b39a3 feat(agent): retain recent context during auto compact
Keep a legal recent suffix in idle auto-compacted sessions so resumed chats preserve their freshest live context while older messages are summarized. Recover persisted summaries even when retained messages remain, and document the new behavior.
2026-04-11 15:56:41 +08:00
chengyongruandXubin Ren d03458f034 fix(agent): eliminate race condition in auto compact summary retrieval
Make Consolidator.archive() return the summary string directly instead
of writing to history.jsonl then reading back via get_last_history_entry().
This eliminates a race condition where concurrent _archive calls for
different sessions could read each other's summaries from the shared
history file (cross-user context leak in multi-user deployments).

Also removes Consolidator.get_last_history_entry() — no longer needed.
2026-04-11 15:56:41 +08:00
chengyongruandXubin Ren 69d60e2b06 fix(agent): handle UnicodeDecodeError in _read_last_entry
history.jsonl may contain non-UTF-8 bytes (e.g. from email channel
binary content), causing auto compact to fail when reading the last
entry for summary generation. Catch UnicodeDecodeError alongside
FileNotFoundError and JSONDecodeError.
2026-04-11 15:56:41 +08:00
chengyongruandXubin Ren fb6dd111e1 feat(agent): auto compact — proactive session compression to reduce token cost and latency (#2982)
When a user is idle for longer than a configured TTL, nanobot **proactively** compresses the session context into a summary. This reduces token cost and first-token latency when the user returns — instead of re-processing a long stale context with an expired KV cache, the model receives a compact summary and fresh input.
2026-04-11 15:56:41 +08:00
Daniel PhangandClaude Opus 4.6 b52bfddf16 fix(cron): guard _load_store against reentrant reload during job execution
When on_job callbacks call list_jobs() (which triggers _load_store),
the in-memory state is reloaded from disk, discarding the next_run_at_ms
updates that _on_timer is actively computing. This causes jobs to
re-trigger indefinitely on the next tick.

Add an _executing flag around the job execution loop. While set,
_load_store returns the cached store instead of reloading from disk.

Includes regression test.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 00:34:48 -07:00
04cbandXubin Ren e392c27f7e fix(utils): anchor unclosed think-tag regex to string start (#3004) 2026-04-11 13:46:15 +08:00
Xubin Ren 696b64b5a6 fix(notebook): remove unused imports
Clean up unused imports in notebook_edit so the Ruff F401 check passes cleanly.

Made-with: Cursor
2026-04-10 16:02:00 +00:00
worenidewen a167959027 fix(mcp): support multiple MCP servers by connecting each in isolated task
Each MCP server now connects in its own asyncio.Task to isolate anyio
cancel scopes and prevent 'exit cancel scope in different task' errors
when multiple servers (especially mixed transport types) are configured.

Changes:
- connect_mcp_servers() returns dict[str, AsyncExitStack] instead of None
- Each server runs in separate task via asyncio.gather()
- AgentLoop uses _mcp_stacks dict to track per-server stacks
- Tests updated to handle new API
2026-04-10 23:51:50 +08:00
Xubin Ren 651aeae656 improve file editing and add notebook tool
Enhance file tools with read tracking, PDF support, safer path handling,
smarter edit matching/diagnostics, and introduce notebook_edit with tests.
2026-04-10 15:44:50 +00:00
Xubin RenandXubin Ren 9bccfa63d2 fix test: use async/await for run_job, add sentinel coverage
Made-with: Cursor
2026-04-10 19:03:13 +08:00
weitongtongandXubin Ren 1a51f907aa feat(cron): 添加 CronService.update_job 方法
支持更新已有定时任务的名称、调度计划、消息内容、投递配置等可变字段。
系统任务(system_event)受保护不可编辑。包含完整的单元测试覆盖。

Made-with: Cursor
2026-04-10 19:03:13 +08:00
zhangxiaoyu.yorkandXubin Ren e7e1249585 fix(agent): avoid truncate_text name shadowing
Rename the boolean flag in _sanitize_persisted_blocks and alias the imported helper so session persistence cannot crash with TypeError when truncation is enabled.
2026-04-10 17:36:31 +08:00
Xubin Ren 2bef9cb650 fix(agent): preserve interrupted tool-call turns
Keep tool-call assistant messages valid across provider sanitization and avoid trailing user-only history after model errors. This prevents follow-up requests from sending broken tool chains back to the gateway.
2026-04-10 05:37:25 +00:00
Xubin RenandXubin Ren c579d67887 fix(memory): preserve consolidation turn boundaries under chunk cap
Made-with: Cursor
2026-04-10 12:58:58 +08:00
comadrejaandXubin Ren bfe53ebb10 fix(memory): harden consolidation with try/except on token estimation and chunk size cap
- Wrap both token estimation calls in try/except to prevent silent failures
  from crashing the consolidation cycle
- Add _MAX_CHUNK_MESSAGES = 60 to cap messages per consolidation round,
  avoiding oversized chunks being sent to the consolidation LLM
- Improve idle log to include unconsolidated message count for easier debugging

These are purely defensive improvements with no behaviour change for
normal sessions.
2026-04-10 12:58:58 +08:00
Xubin Ren 363a0704db refactor(runner): update message processing to preserve historical context
- Adjusted message handling in AgentRunner to ensure that historical messages remain unchanged during context governance.
- Introduced tests to verify that backfill operations do not alter the saved message boundary, maintaining the integrity of the conversation history.
2026-04-10 04:46:48 +00:00
chengyongruandXubin Ren 27e7a338a3 docs(feishu): add toolHintPrefix to README config example 2026-04-10 12:29:43 +08:00
chengyongruandXubin Ren 6fd2511c8a refactor(feishu): simplify tool hint to append-only, delegate to send_delta for throttling
- Make tool_hint_prefix configurable in FeishuConfig (default: 🔧)
- Delegate tool hint card updates from send() to send_delta() so hints
  automatically benefit from _STREAM_EDIT_INTERVAL throttling
- Fix staticmethod calls to use self.__class__ instead of self
- Document all supported metadata keys in send_delta docstring
- Add test for empty/whitespace-only tool hint with active stream buffer
2026-04-10 12:29:43 +08:00
xzq.xuandXubin Ren 049ce9baae fix(tool-hints): deduplicate by formatted string + per-line inline display
Two display fixes based on real-world Feishu testing:

1. tool_hints.py: format_tool_hints now deduplicates by comparing the
   fully formatted hint string instead of tool name alone. This fixes
   `ls /Desktop` and `ls /Downloads` being incorrectly merged as
   `ls /Desktop × 2`. Truly identical calls still fold correctly.
   (_group_consecutive and all abbreviation logic preserved unchanged.)

2. feishu.py: inline tool hints now display one tool per line with
   🔧 prefix, and use double-newline trailing to prevent Setext heading
   rendering when followed by markdown `---`.

Made-with: Cursor
2026-04-10 12:29:43 +08:00
xzq.xuandXubin Ren 512c3b88e3 fix(feishu): preserve tool hints in final card content
Tool hints should be kept as permanent content in the streaming card
so users can see which tools were called (matching the standalone card
behavior). Previously, hints were stripped when new deltas arrived or
when the stream ended, causing tool call information to disappear.

Now:
- New delta: hint becomes permanent content, delta appends after it
- New tool hint: replaces the previous hint (unchanged)
- Resuming/stream_end: hint is preserved in the final text

Updated 3 tests to verify hint preservation semantics.

Made-with: Cursor
2026-04-10 12:29:43 +08:00
xzq.xuandXubin Ren 589e3ac36e fix(feishu): prevent tool hint stacking and clean hints on stream_end
Three fixes for inline tool hints:

1. Consecutive tool hints now replace the previous one instead of
   stacking — the old suffix is stripped before appending the new one.

2. When _resuming flushes the buffer, any trailing tool hint suffix
   is removed so it doesn't persist into the next streaming segment.

3. When final _stream_end closes the card, tool hint suffix is
   cleaned from the text before the final card update.

Adds 3 regression tests covering all three scenarios.

Made-with: Cursor
2026-04-10 12:29:43 +08:00
xzq.xuandXubin Ren ac1795c158 feat(feishu): streaming resuming + inline tool hints
Two improvements to Feishu streaming card experience:

1. Handle _resuming in send_delta: when a mid-turn _stream_end arrives
   with resuming=True (tool call between segments), flush current text
   to the card but keep the buffer alive so subsequent segments append
   to the same card instead of creating a new one.

2. Inline tool hints into streaming cards: when a tool hint arrives
   while a streaming card is active, append it to the card content
   (e.g. "🔧 web_fetch(...)") instead of sending a separate card.
   The hint is automatically stripped when the next delta arrives.

Made-with: Cursor
2026-04-10 12:29:43 +08:00
JiajunandXubin Ren ce9829e92f feat(feishu): add done emoji support for reaction lifecycle (#2899)
* feat(feishu): add done emoji support for reaction lifecycle

* feat(feishu): add done emoji support and update documentation
2026-04-10 12:29:43 +08:00
chengyongruandXubin Ren e0c6e6f180 test: add regression tests for <thought> tag stripping 2026-04-10 12:10:23 +08:00
flobo3andXubin Ren 6b7e78a8e0 fix: strip <thought> blocks from Gemma 4 and similar models 2026-04-10 12:10:23 +08:00
Xubin RenandXubin Ren 69d748bf8f Merge origin/main; warn on partial proxy credentials; add only-password test
- Merged latest main (no conflicts)
- Added warning log when only one of proxy_username/proxy_password is set
- Added test_start_no_proxy_auth_when_only_password for coverage parity

Made-with: Cursor
2026-04-09 23:54:11 +08:00
JonasandXubin Ren 7506af7104 feat(channel): add proxy support for Discord channel
- Add proxy, proxy_username, proxy_password fields to DiscordConfig
- Pass proxy and proxy_auth to discord.Client
- Add aiohttp.BasicAuth when credentials are provided
- Add tests for proxy configuration scenarios
2026-04-09 23:54:11 +08:00
chenyahuiandXubin Ren 0e6331b66d feat(exec): support allowed_env_keys to pass specified env vars to subprocess
Add allowed_env_keys config field to selectively forward host environment variables (e.g. GOPATH, JAVA_HOME) into the sandboxed subprocess environment, while keeping the default allow-list unchanged.
2026-04-09 23:35:44 +08:00
Xubin RenandXubin Ren c625c0c2a7 Merge origin/main and add regression tests for streaming error delivery
- Merged latest main (no conflicts)
- Added test_llm_error_not_appended_to_session_messages: verifies error
  content stays out of session messages
- Added test_streamed_flag_not_set_on_llm_error: verifies _streamed is
  not set when LLM returns an error, so ChannelManager delivers it

Made-with: Cursor
2026-04-09 23:10:46 +08:00
yanghan-cyberandXubin Ren 10f6c875a5 fix(agent): deliver LLM errors to streaming channels and avoid polluting session context
When the LLM returns an error (e.g. 429 quota exceeded, stream timeout),
streaming channels silently drop the error message because `_streamed=True`
is set in metadata even though no content was actually streamed.

This change:
- Skips setting `_streamed` when stop_reason is "error", so error messages
  go through the normal channel.send() path and reach the user
- Stops appending error content to session history, preventing error
  messages from polluting subsequent conversation context
- Exposes stop_reason from _run_agent_loop to enable the above check
2026-04-09 23:10:46 +08:00
Xubin RenandXubin Ren ba8bce0f45 fix(tests): add missing from typing import Any in websocket integration tests
Made-with: Cursor
2026-04-09 18:22:35 +08:00
chengyongruandXubin Ren 42de13a1a9 docs(websocket): add WebSocket channel documentation
Comprehensive guide covering wire protocol, configuration reference,
token issuance, security notes, and common deployment patterns.
2026-04-09 18:22:35 +08:00
chengyongruandXubin Ren 56a5906db5 fix(websocket): harden security and robustness
- Use hmac.compare_digest for timing-safe static token comparison
- Add issued token capacity limit (_MAX_ISSUED_TOKENS=10000) with 429 response
- Use atomic pop in _take_issued_token_if_valid to eliminate TOCTOU window
- Enforce TLSv1.2 minimum version for SSL connections
- Extract _safe_send helper for consistent ConnectionClosed handling
- Move connection registration after ready send to prevent out-of-order delivery
- Add HTTP-level allow_from check and client_id truncation in process_request
- Make stop() idempotent with graceful shutdown error handling
- Normalize path via validator instead of leaving raw value
- Default websocket_requires_token to True for secure-by-default behavior
- Add integration tests and ws_test_client helper
- Refactor tests to use shared _ch factory and bus fixture
2026-04-09 18:22:35 +08:00
chengyongruandXubin Ren e0ccc401c0 fix(websocket): handle ConnectionClosed gracefully in send and send_delta 2026-04-09 18:22:35 +08:00
Jack LuandXubin Ren ad57bcd127 feat(channels): add WebSocket server channel and tests
Port Python implementation from a1ec7b192a
(websocket channel module and channel tests; excludes webui debug app).
2026-04-09 18:22:35 +08:00
chenyahui e9c4fe6824 feat(skills): add disabled_skills config to exclude skills from loading
Introduce a disabled_skills option in the config schema that allows
users to specify a list of skill names to be excluded. The setting is
threaded from config through Nanobot -> AgentLoop -> ContextBuilder ->
SkillsLoader. Disabled skills are filtered out from list_skills,
get_always_skills, and build_skills_summary. Four new test cases cover
the filtering behavior.
2026-04-09 14:11:47 +08:00
Xubin RenandGitHub 3361ac9dd1 Merge PR #2637: fix(providers): enforce role alternation for non-Claude providers
fix(providers): enforce role alternation for non-Claude providers
2026-04-09 12:47:41 +08:00
Xubin Ren dadf453097 Merge origin/main into fix/sanitize-messages-non-claude
Resolved conflict in azure_openai_provider.py by keeping main's
Responses API implementation (role alternation not needed for the
Responses API input format).

Made-with: Cursor
2026-04-09 04:45:45 +00:00
彭星杰andXubin Ren 1e3057d0d6 fix(cli): remove default green style from Enabled column in tables
The Enabled column in channels status and plugins list commands had a default green style that overrode the dim markup for disabled items. This caused no values to appear green instead of dimmed. Remove the default style to let cell-level markup control the display correctly.
2026-04-09 11:52:31 +08:00
Alfredo ArenasandXubin Ren 6445b3b0cf fix(helpers): repair corrupted split_message and ensure content never None
Fix accidental line corruption in split_message() where 'break' was
merged with unrelated code during manual editing.

The actual fix: build_assistant_message() now returns content or ""
instead of content (which could be None), preventing providers like
MiMo V2 Omni from rejecting tool-call messages with missing text field.

Fixes #2519
2026-04-09 11:42:53 +08:00
Alfredo ArenasandXubin Ren 6d74c88014 fix(helpers): ensure assistant message content is never None 2026-04-09 11:42:53 +08:00
Xubin Ren 1dd2d5486e docs: add unified session configuration to README for cross-channel continuity 2026-04-09 03:15:21 +00:00
Xubin RenandXubin Ren cf02408fc0 Merge origin/main; remove stale comment and fix blank-line style
Made-with: Cursor
2026-04-09 11:09:25 +08:00
whsandXubin Ren be1b34ed7c fix: remove unused import re 2026-04-09 11:09:25 +08:00
whsandXubin Ren b4c7cd654e fix: use effective session key for _active_tasks in unified mode 2026-04-09 11:09:25 +08:00
whsandXubin Ren 985f9c443b tests: add unified_session coverage for /new and consolidation 2026-04-09 11:09:25 +08:00
whsandXubin Ren 743e73da3f feat(session): add unified_session config to share one session across all channels 2026-04-09 11:09:25 +08:00
chenspandXubin Ren bfec06a2c1 Fix Windows exec env for Docker Desktop plugin discovery
nanobot's Windows exec environment was not forwarding ProgramFiles and related variables, so docker desktop start could not discover the desktop CLI plugin and reported unknown command. Forward the missing variables and add a regression test that covers the Windows env shape.
2026-04-09 10:55:53 +08:00
Rohit_Dayanand123andXubin Ren 3cc2ebeef7 Added bug fix to Dingtalk by zipping html to prevent raw failure 2026-04-09 10:49:00 +08:00
Leo fuandXubin Ren 42624f5bf3 test: update expected token display to match consistent 1000 divisor
The test fixtures use 65536 as context_window_tokens. With the divisor
corrected from 1024 to 1000, the display changes from 64k to 65k.
2026-04-09 10:40:20 +08:00
Leo fuandXubin Ren 66409784f4 fix(status): use consistent divisor (1000) for token count display
The /status command divided context_used by 1000 but context_total by
1024, producing inconsistent values. For example a 128000-token window
displayed as 125k instead of 128k. Tokens are not a binary unit, so
both should use 1000.
2026-04-09 10:40:20 +08:00
Xubin RenandXubin Ren 61dd5ac13a test(discord): cover streamed reply overflow
Lock the Discord streaming path with a regression test for final chunk splitting so oversized replies stay safe to merge and ship.

Made-with: Cursor
2026-04-09 00:24:11 +08:00
SHLE1andXubin Ren e49b6c0c96 fix(discord): enable streaming replies 2026-04-09 00:24:11 +08:00
Xubin RenandXubin Ren 715f2a79be fix(version): fall back to pyproject in source checkouts
Keep importlib.metadata as the primary source for installed packages, but avoid PackageNotFoundError when nanobot is imported directly from a source tree.

Made-with: Cursor
2026-04-08 23:47:36 +08:00
bahtyaandXubin Ren 1700166945 fix: use importlib.metadata for version to prevent mismatch with pyproject.toml
Fixes #2856

Previously __version__ was hardcoded as '0.4.1' in __init__.py while
pyproject.toml declared version '0.1.5'. This caused nanobot gateway to
report version 0.4.1 on startup while pip showed 0.1.5.

Now __version__ reads from importlib.metadata.version('nanobot-ai'),
keeping pyproject.toml as the single source of truth.
2026-04-08 23:47:36 +08:00
Xubin RenandXubin Ren 6bf101c79b fix(hook): keep composite hooks backward compatible
Avoid AttributeError regressions when hooks define their own __init__ or when a CompositeHook wraps another composite.

Made-with: Cursor
2026-04-08 23:41:31 +08:00
Lingao MengandXubin Ren d88be08bfd refactor(hook): add reraise flag to AgentHook and remove _LoopHookChain
Add reraise parameter to AgentHook so hooks can opt out of exception
swallowing in CompositeHook._for_each_hook_safe. _LoopHook sets
reraise=True to let its exceptions propagate. _LoopHookChain is removed
and replaced with CompositeHook([loop_hook] + extra_hooks).

Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
2026-04-08 23:41:31 +08:00
Xubin RenandXubin Ren 142cb46956 fix(cron): preserve manual run state and merged history
Keep manual runs from flipping the scheduler's running flag, rebuild merged run history records from action logs, and avoid delaying sub-second jobs to a one-second floor. Add regression coverage for disabled/manual runs, merged history persistence, and sub-second timers.

Made-with: Cursor
2026-04-08 23:34:47 +08:00
xinnan.houandXubin Ren 0f1e3aa151 fix 2026-04-08 23:34:47 +08:00
Xubin Ren d084d10dc2 feat(openai): auto-route direct reasoning requests with responses fallback 2026-04-08 15:21:08 +00:00
Xubin RenandXubin Ren c092896922 fix(tool-hint): handle quoted paths in exec hints
Preserve path folding for quoted exec command paths with spaces so hint previews do not fall back to mid-path truncation. Add regression coverage for quoted Unix and Windows path cases.

Made-with: Cursor
2026-04-08 23:05:52 +08:00
b16865722b fix(tool-hint): fold paths in exec commands and deduplicate by formatted string
1. exec tool hints previously used val[:40] blind character truncation,
   cutting paths mid-segment. Now detects file paths via regex and
   abbreviates them with abbreviate_path. Supports Windows, Unix
   absolute, and ~/ home paths.

2. Deduplication now compares fully formatted hint strings instead of
   tool names alone. Fixes ls /Desktop and ls /Downloads being
   incorrectly merged as "ls /Desktop × 2".

Co-authored-by: xzq.xu <zhiqiang.xu@nodeskai.com>
2026-04-08 23:05:52 +08:00
stutiredboyandXubin Ren af6c75141f feat(): telegram support stream edit interval 2026-04-08 22:49:33 +08:00
dengjingren a068df5a79 feat(api): support file uploads via JSON base64 and multipart/form-data 2026-04-08 15:58:52 +08:00
kronk307andXubin Ren e21ba5f667 feat(telegram): add location/geo support
Forward static location pins as [location: lat, lon] content so the
agent can respond to geo messages and pass coordinates to MCP tools.

Closes HKUDS/nanobot#2909
2026-04-08 02:32:19 +08:00
Xubin RenandXubin Ren c7d10de253 feat(soul): restore friendly and curious tone to SOUL.md
Made-with: Cursor
2026-04-08 02:22:25 +08:00
Xubin RenandXubin Ren edb821e10d feat(agent): prompt behavior directives, tool descriptions, and loop robustness 2026-04-08 02:22:25 +08:00
Xubin RenandXubin Ren ef0284a4e0 fix(exec): add Windows support for shell command execution
ExecTool hardcoded bash, breaking exec on Windows. Now uses cmd.exe
via COMSPEC on Windows with a curated minimal env (PATH, SYSTEMROOT,
etc.) that excludes secrets. bwrap sandbox gracefully skips on Windows.
2026-04-08 01:48:55 +08:00
Xubin RenandXubin Ren 63acfc4f2f test: fix trailing-space mismatch and add regression tests for normal models
- Fix assertion in streaming dict fallback test (trailing space in data
  not reflected in expected value).
- Add two regression tests proving that models with reasoning_content
  (e.g. DeepSeek-R1) and standard models (no reasoning fields) are
  completely unaffected by the reasoning fallback.

Made-with: Cursor
2026-04-08 00:59:39 +08:00
moranfongandXubin Ren 12ff8b22d6 fix(provider): extend StepFun reasoning fallback to all code paths
- Add reasoning_content fallback from reasoning in _parse dict branch
- Add content fallback from msg.reasoning in _parse SDK object branch
- Add reasoning_content fallback in _parse SDK object branch
- Add reasoning fallback in _parse_chunks dict branch
- Add reasoning fallback in _parse_chunks SDK object branch

This ensures StepFun Plan API works correctly in both streaming and
non-streaming modes, for both dict and SDK object response formats.
2026-04-08 00:59:39 +08:00
moranfongandXubin Ren 9e7c07ac89 test(provider): add StepFun reasoning field fallback tests
Add comprehensive tests for the StepFun Plan API compatibility fix:
- _parse dict branch: content and reasoning_content fallback to reasoning
- _parse SDK object branch: same fallback for pydantic response objects
- _parse_chunks dict branch: reasoning field handled in streaming mode
- _parse_chunks SDK branch: reasoning fallback for SDK delta objects
- Precedence tests: reasoning_content field takes priority over reasoning

Refs: fix(provider): support StepFun Plan API reasoning field fallback
2026-04-08 00:59:39 +08:00
moranfongandXubin Ren 53107c6683 fix(provider): support StepFun Plan API reasoning field fallback
StepFun Plan API returns response content in the 'reasoning' field when
the model is in thinking mode and 'content' is empty. OpenAICompatProvider
previously only checked 'content' and 'reasoning_content', missing this field.

This patch adds a fallback: if content is empty and 'reasoning' is present,
extract text from reasoning to populate content, ensuring StepFun models
(step-3.5-flash, step-3.5-flash-2603) work correctly with tool calls.

Co-authored-by: moranfong <moranfong@gmail.com>
2026-04-08 00:59:39 +08:00
Xubin RenandXubin Ren c736cecc28 chore(gitignore): remove bogus extensions and relocate nano.*.save
- Drop *.pycs, *.pywz, *.pyzz — not real Python file extensions.
- Move nano.*.save from "Project-specific" to "Editors & IDEs" where
  it belongs (nano editor backup files, not project artifacts).

Made-with: Cursor
2026-04-08 00:42:25 +08:00
Jack LuandXubin Ren 873bf5e692 chore: update .gitignore to include additional project-specific, build, test, and environment files 2026-04-08 00:42:25 +08:00
Xubin RenandXubin Ren 8871a57b4c fix(mcp): forward prompt arg descriptions & standardise error format
- Propagate `description` from MCP prompt arguments into the JSON
  Schema so LLMs can better understand prompt parameters.
- Align generic-exception error message with tool/resource wrappers
  (drop redundant `{exc}` detail).
- Extend test fixture to mock `mcp.shared.exceptions.McpError`.
- Add tests for argument description forwarding and McpError handling.

Made-with: Cursor
2026-04-08 00:28:04 +08:00
Tim O'BrienandXubin Ren 7cc527cf65 feat(mcp): expose MCP resources and prompts as read-only tools
Add MCPResourceWrapper and MCPPromptWrapper classes that expose MCP
server resources and prompts as nanobot tools. Resources are read-only
tools that fetch content by URI, and prompts are read-only tools that
return filled prompt templates with optional arguments.

- MCPResourceWrapper: reads resource content (text and binary) via URI
- MCPPromptWrapper: gets prompt templates with typed arguments
- Both handle timeouts, cancellation, and MCP SDK 1.x error types
- Resources and prompts are registered during server connection
- Gracefully handles servers that don't support resources/prompts
2026-04-08 00:28:04 +08:00
Xubin RenandXubin Ren ce7986e492 fix(memory): add timestamp and cap to recent history injection 2026-04-08 00:03:11 +08:00
Xubin RenandXubin Ren 05d8062c70 test: add regression tests for unprocessed history injection in system prompt
Made-with: Cursor
2026-04-07 23:41:05 +08:00
Lingao MengandXubin Ren 31c154a7b8 fix(memory): prevent potential loss of compressed session history
When the Consolidator compresses old session messages into history.jsonl,
those messages are immediately removed from the LLM's context. Dream
processes history.jsonl into long-term memory (memory.md) on a cron
schedule (default every 2h), creating a window where compressed content
is invisible to the LLM.

This change closes the gap by injecting unprocessed history entries
(history.jsonl entries not yet consumed by Dream) directly into the
system prompt as "# Recent History".

Key design notes:
- Uses read_unprocessed_history(since_cursor=last_dream_cursor) so only
  entries not yet reflected in long-term memory are included, avoiding
  duplication with memory.md
- No overlap with session messages: Consolidator advances
  last_consolidated before returning, so archived messages are already
  removed from get_history() output
- Token-safe: Consolidator's estimate_session_prompt_tokens calls
  build_system_prompt via the same build_messages function, so the
  injected entries are included in token budget calculations and will
  trigger further consolidation if needed

Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
2026-04-07 23:41:05 +08:00
Xubin RenandXubin Ren acafcf3cb0 docs: fix inaccurate claim about supports_streaming and dict config
supports_streaming already handles dict configs via isinstance check;
only is_allowed() fails with plain dicts. Narrow the explanation.

Made-with: Cursor
2026-04-07 23:01:30 +08:00
invictusandXubin Ren 4648cb9e87 docs: use model_dump(by_alias=True) for default_config in plugin guide 2026-04-07 23:01:30 +08:00
invictusandXubin Ren 83ad013be5 docs: fix channel plugin guide — require Pydantic config model 2026-04-07 23:01:30 +08:00
Xubin RenandXubin Ren 1e8a6663ca test(anthropic): add regression tests for thinking modes incl. adaptive
Also update schema comment to mention 'adaptive' as a valid value.

Made-with: Cursor
2026-04-07 22:53:43 +08:00
Balor.LC3andXubin Ren 1c2f4aba17 feat(anthropic): add adaptive thinking mode
Extends reasoning_effort to accept 'adaptive' in addition to
low/medium/high. When set, uses Anthropic's type: 'adaptive'
thinking API instead of a fixed budget, letting the model decide
when and how much to think per turn.

Also auto-enables interleaved thinking between tool calls on
claude-sonnet-4-6 and claude-opus-4-6.

Usage:
  "reasoning_effort": "adaptive" in agents.defaults config
2026-04-07 22:53:43 +08:00
423aab09dd test(cron): add regression test for running service picking up external adds
Co-authored-by: chengyongru
Made-with: Cursor
2026-04-07 22:48:40 +08:00
xinnan.houandXubin Ren a982d9f9be add reload jobs test 2026-04-07 22:48:40 +08:00
xinnan.houandXubin Ren fd2bb3bb7d fix comment 2026-04-07 22:48:40 +08:00
xinnan.houandXubin Ren 4e914d0e2a fix not reload job config 2026-04-07 22:48:40 +08:00
chengyongruandXubin Ren b4f985f3dc feat(memory):dream enhancement (#2887)
* feat(dream): enhance memory cleanup with staleness detection

- Phase 1: add [FILE-REMOVE] directive and staleness patterns (14-day
  threshold, completed tasks, superseded info, resolved tracking)
- Phase 2: add explicit cleanup rules, file paths section, and deletion
  guidance to prevent LLM path confusion
- Inject current date and file sizes into Phase 1 context for age-aware
  analysis
- Add _dream_debug() helper for observability (dream-debug.log in workspace)
- Log Phase 1 analysis output and Phase 2 tool events for debugging

Tested with glm-5-turbo: MEMORY.md reduced from 149 to 108-129 lines
across two rounds, correctly identifying and removing weather data,
detailed incident info, completed research, and stale discussions.

* refactor(dream): replace _dream_debug file logger with loguru

Remove the custom _dream_debug() helper that wrote to dream-debug.log
and use the existing loguru logger instead. Phase 1 analysis is logged
at debug level, tool events at info level — consistent with the rest
of the codebase and no extra log file to manage.

* fix(dream): make stale scan independent of conversation history

Reframe Phase 1 from a single comparison task to two independent
tasks: history diff AND proactive stale scan. The LLM was skipping
stale content that wasn't referenced in conversation history (e.g.
old triage snapshots). Now explicitly requires scanning memory files
for staleness patterns on every run.

* fix(dream): correct old_text param name and truncate debug log

- Phase 2 prompt: old_string -> old_text to match EditFileTool interface
- Phase 1 debug log: truncate analysis to 500 chars to avoid oversized lines

* refactor(dream): streamline prompts by separating concerns

Phase 1 owns all staleness judgment logic; Phase 2 is pure execution
guidance. Remove duplicated cleanup rules from Phase 2 since Phase 1
already determines what to add/remove. Fix remaining old_string -> old_text.
Total prompt size reduced ~45% (870 -> 480 tokens).

* fix(dream): add FILE-REMOVE execution guidance to Phase 2 prompt

Phase 2 was only processing [FILE] additions and ignoring [FILE-REMOVE]
deletions after the cleanup rules were removed. Add explicit mapping:
[FILE] → add content, [FILE-REMOVE] → delete content.
2026-04-07 22:39:47 +08:00
Ziyan Lin 26ae906116 fix(providers): enforce role alternation for non-Claude providers
Some LLM providers (OpenAI-compat, Azure, vLLM, Ollama) reject requests
with consecutive same-role messages or trailing assistant messages. Add
_enforce_role_alternation() to merge consecutive same-role user/assistant
messages and strip trailing assistant messages before sending to the API.
2026-03-30 15:15:15 +08:00
rav-melisono bc357208bb feat: add HTTP health endpoint on gateway port
Binds a lightweight asyncio HTTP server on the configured gateway
port (default 18790) alongside the existing agent and channel tasks.

Endpoints:
  GET /       -> "nanobot" (plain text, for service discovery)
  GET /health -> JSON with service, version, status, uptime, channels

Zero new dependencies — uses asyncio.start_server.
2026-03-29 15:31:29 +01:00
129 changed files with 16321 additions and 1224 deletions
+73 -12
View File
@@ -1,26 +1,87 @@
# Project-specific
.worktrees/
.assets
.docs
.env
.web
.orion
# Python bytecode & caches
*.pyc
dist/
build/
*.egg-info/
*.egg
*.pycs
*.pyo
*.pyd
*.pyw
*.pyz
*.pywz
*.pyzz
__pycache__/
*.egg-info/
*.egg
.venv/
venv/
__pycache__/
poetry.lock
.pytest_cache/
botpy.log
nano.*.save
.DS_Store
.mypy_cache/
.ruff_cache/
.pytype/
.dmypy.json
dmypy.json
.tox/
.nox/
.hypothesis/
# Build & packaging
dist/
build/
*.manifest
*.spec
pip-wheel-metadata/
share/python-wheels/
# Test & coverage
.coverage
.coverage.*
htmlcov/
coverage.xml
*.cover
# Lock files (project policy)
poetry.lock
uv.lock
# Jupyter
.ipynb_checkpoints/
# macOS
.DS_Store
.AppleDouble
.LSOverride
# Windows
Thumbs.db
ehthumbs.db
Desktop.ini
# Linux
.directory
# Editors & IDEs (local workspace / user settings)
.vscode/
.cursor/
.idea/
.fleet/
*.code-workspace
*.sublime-project
*.sublime-workspace
*.swp
*.swo
*~
nano.*.save
# Environment & secrets (keep examples tracked if needed)
.env.*
!.env.example
# Logs & temp
*.log
logs/
tmp/
temp/
*.tmp
+325 -38
View File
@@ -21,7 +21,20 @@
## 📢 News
- **2026-04-14** 🚀 Released **v0.1.5.post1** — Dream skill discovery, mid-turn follow-up injection, WebSocket channel, and deeper channel integrations. Please see [release notes](https://github.com/HKUDS/nanobot/releases/tag/v0.1.5.post1) for details.
- **2026-04-13** 🛡️ Agent turn hardened — user messages persisted early, auto-compact skips active tasks.
- **2026-04-12** 🔒 Lark global domain support, Dream learns discovered skills, shell sandbox tightened.
- **2026-04-11** ⚡ Context compact shrinks sessions on the fly; Kagi web search; QQ & WeCom full media.
- **2026-04-10** 📓 Notebook editing tool, multiple MCP servers, Feishu streaming & done-emoji.
- **2026-04-09** 🔌 WebSocket channel, unified cross-channel session, `disabled_skills` config.
- **2026-04-08** 📤 API file uploads, OpenAI reasoning auto-routing with Responses fallback.
- **2026-04-07** 🧠 Anthropic adaptive thinking, MCP resources & prompts exposed as tools.
- **2026-04-06** 🛰️ Langfuse observability, unified Whisper transcription, email attachments.
- **2026-04-05** 🚀 Released **v0.1.5** — sturdier long-running tasks, Dream two-stage memory, production-ready sandboxing and programming Agent SDK. Please see [release notes](https://github.com/HKUDS/nanobot/releases/tag/v0.1.5) for details.
<details>
<summary>Earlier news</summary>
- **2026-04-04** 🚀 Jinja2 response templates, Dream memory hardened, smarter retry handling.
- **2026-04-03** 🧠 Xiaomi MiMo provider, chain-of-thought reasoning visible, Telegram UX polish.
- **2026-04-02** 🧱 Long-running tasks run more reliably — core runtime hardening.
@@ -31,11 +44,6 @@
- **2026-03-29** 💬 WeChat voice, typing, QR/media resilience; fixed-session OpenAI-compatible API.
- **2026-03-28** 📚 Provider docs refresh; skill template wording fix.
- **2026-03-27** 🚀 Released **v0.1.4.post6** — architecture decoupling, litellm removal, end-to-end streaming, WeChat channel, and a security fix. Please see [release notes](https://github.com/HKUDS/nanobot/releases/tag/v0.1.4.post6) for details.
<details>
<summary>Earlier news</summary>
- **2026-03-26** 🏗️ Agent runner extracted and lifecycle hooks unified; stream delta coalescing at boundaries.
- **2026-03-25** 🌏 StepFun provider, configurable timezone, Gemini thought signatures.
- **2026-03-24** 🔧 WeChat compatibility, Feishu CardKit streaming, test suite restructured.
@@ -112,26 +120,57 @@
## Table of Contents
- [News](#-news)
- [Key Features](#key-features-of-nanobot)
- [Architecture](#-architecture)
- [Features](#-features)
- [Install](#-install)
- [Quick Start](#-quick-start)
- [Chat Apps](#-chat-apps)
- [Agent Social Network](#-agent-social-network)
- [Configuration](#-configuration)
- [Multiple Instances](#-multiple-instances)
- [Memory](#-memory)
- [CLI Reference](#-cli-reference)
- [In-Chat Commands](#-in-chat-commands)
- [Python SDK](#-python-sdk)
- [OpenAI-Compatible API](#-openai-compatible-api)
- [Docker](#-docker)
- [Linux Service](#-linux-service)
- [Project Structure](#-project-structure)
- [Contribute & Roadmap](#-contribute--roadmap)
- [Star History](#-star-history)
- [📢 News](#-news)
- [Key Features of nanobot:](#key-features-of-nanobot)
- [🏗️ Architecture](#-architecture)
- [Table of Contents](#table-of-contents)
- [✨ Features](#-features)
- [📦 Install](#-install)
- [Update to latest version](#update-to-latest-version)
- [🚀 Quick Start](#-quick-start)
- [💬 Chat Apps](#-chat-apps)
- [🌐 Agent Social Network](#-agent-social-network)
- [⚙️ Configuration](#-configuration)
- [Environment Variables for Secrets](#environment-variables-for-secrets)
- [Providers](#providers)
- [Channel Settings](#channel-settings)
- [Retry Behavior](#retry-behavior)
- [Web Search](#web-search)
- [`tools.web.search`](#toolswebsearch)
- [MCP (Model Context Protocol)](#mcp-model-context-protocol)
- [Security](#security)
- [Auto Compact](#auto-compact)
- [Timezone](#timezone)
- [Unified Session](#unified-session)
- [Disabled Skills](#disabled-skills)
- [🧩 Multiple Instances](#-multiple-instances)
- [Quick Start](#quick-start)
- [Path Resolution](#path-resolution)
- [How It Works](#how-it-works)
- [Minimal Setup](#minimal-setup)
- [Common Use Cases](#common-use-cases)
- [Notes](#notes)
- [🧠 Memory](#-memory)
- [💻 CLI Reference](#-cli-reference)
- [💬 In-Chat Commands](#-in-chat-commands)
- [🐍 Python SDK](#-python-sdk)
- [🔌 OpenAI-Compatible API](#-openai-compatible-api)
- [Behavior](#behavior)
- [Endpoints](#endpoints)
- [curl](#curl)
- [File Upload (JSON base64)](#file-upload-json-base64)
- [File Upload (multipart/form-data)](#file-upload-multipartform-data)
- [Python (`requests`)](#python-requests)
- [Python (`openai`)](#python-openai)
- [🐳 Docker](#-docker)
- [Docker Compose](#docker-compose)
- [Docker](#docker)
- [🐧 Linux Service](#-linux-service)
- [📁 Project Structure](#-project-structure)
- [🤝 Contribute \& Roadmap](#-contribute--roadmap)
- [Branching Strategy](#branching-strategy)
- [Contributors](#contributors)
- [⭐ Star History](#-star-history)
## ✨ Features
@@ -276,6 +315,7 @@ Connect nanobot to your favorite chat platform. Want to build your own? See the
| **Email** | IMAP/SMTP credentials |
| **QQ** | App ID + App Secret |
| **Wecom** | Bot ID + Bot Secret |
| **Microsoft Teams** | App ID + App Password + public HTTPS endpoint |
| **Mochat** | Claw token (auto-setup available) |
<details>
@@ -394,7 +434,9 @@ If you prefer to configure manually, add the following to `~/.nanobot/config.jso
"enabled": true,
"token": "YOUR_BOT_TOKEN",
"allowFrom": ["YOUR_USER_ID"],
"groupPolicy": "mention"
"allowChannels": [],
"groupPolicy": "mention",
"streaming": true
}
}
}
@@ -405,6 +447,8 @@ If you prefer to configure manually, add the following to `~/.nanobot/config.jso
> - `"open"` — Respond to all messages
> DMs always respond when the sender is in `allowFrom`.
> - If you set group policy to open create new threads as private threads and then @ the bot into it. Otherwise the thread itself and the channel in which you spawned it will spawn a bot session.
> `allowChannels` restricts the bot to specific Discord channel IDs. Empty (default) means respond in every channel the bot can see. Example: `["1234567890", "0987654321"]`. The filter applies after `allowFrom`, so both must pass.
> `streaming` defaults to `true`. Disable it only if you explicitly want non-streaming replies.
**5. Invite the bot**
- OAuth2 → URL Generator
@@ -560,6 +604,7 @@ Uses **WebSocket** long connection — no public IP required.
"groupPolicy": "mention",
"reactEmoji": "OnIt",
"doneEmoji": "DONE",
"toolHintPrefix": "🔧",
"streaming": true,
"domain": "feishu"
}
@@ -573,6 +618,7 @@ Uses **WebSocket** long connection — no public IP required.
> `groupPolicy`: `"mention"` (default — respond only when @mentioned), `"open"` (respond to all group messages). Private chats always respond.
> `reactEmoji`: Emoji for "processing" status (default: `OnIt`). See [available emojis](https://open.larkoffice.com/document/server-docs/im-v1/message-reaction/emojis-introduce).
> `doneEmoji`: Optional emoji for "completed" status (e.g., `DONE`, `OK`, `HEART`). When set, bot adds this reaction after removing `reactEmoji`.
> `toolHintPrefix`: Prefix for inline tool hints in streaming cards (default: `🔧`).
> `domain`: `"feishu"` (default) for China (open.feishu.cn), `"lark"` for international Lark (open.larksuite.com).
**3. Run**
@@ -857,6 +903,56 @@ nanobot gateway
</details>
<details>
<summary><b>Microsoft Teams</b> (MVP — DM only)</summary>
> Direct-message text in/out, tenant-aware OAuth, conversation reference persistence.
> Uses a public HTTPS webhook — no WebSocket; you need a tunnel or reverse proxy.
**1. Install the optional dependency**
```bash
pip install nanobot-ai[msteams]
```
**2. Create a Teams / Azure bot app registration**
Create or reuse a Microsoft Teams / Azure bot app registration. Set the bot messaging endpoint to a public HTTPS URL ending in `/api/messages`.
**3. Configure**
```json
{
"channels": {
"msteams": {
"enabled": true,
"appId": "YOUR_APP_ID",
"appPassword": "YOUR_APP_SECRET",
"tenantId": "YOUR_TENANT_ID",
"host": "0.0.0.0",
"port": 3978,
"path": "/api/messages",
"allowFrom": ["*"],
"replyInThread": true,
"mentionOnlyResponse": "Hi — what can I help with?",
"validateInboundAuth": true
}
}
}
```
> - `replyInThread: true` replies to the triggering Teams activity when a stored `activity_id` is available.
> - `mentionOnlyResponse` controls what Nanobot receives when a user sends only a bot mention (`<at>Nanobot</at>`). Set to `""` to ignore mention-only messages.
> - `validateInboundAuth: true` enables inbound Bot Framework bearer-token validation (signature, issuer, audience, lifetime, `serviceUrl`). This is the safe default for public deployments. Only set it to `false` for local development or tightly controlled testing.
**4. Run**
```bash
nanobot gateway
```
</details>
## 🌐 Agent Social Network
🐈 nanobot is capable of linking to the agent social network (agent community). **Just send one message and your nanobot joins automatically!**
@@ -918,6 +1014,7 @@ IMAP_PASSWORD=your-password-here
> - **Voice transcription**: Voice messages (Telegram, WhatsApp) are automatically transcribed using Whisper. By default Groq is used (free tier). Set `"transcriptionProvider": "openai"` under `channels` to use OpenAI Whisper instead — the API key is picked from the matching provider config.
> - **MiniMax Coding Plan**: Exclusive discount links for the nanobot community: [Overseas](https://platform.minimax.io/subscribe/coding-plan?code=9txpdXw04g&source=link) · [Mainland China](https://platform.minimaxi.com/subscribe/token-plan?code=GILTJpMTqZ&source=link)
> - **MiniMax (Mainland China)**: If your API key is from MiniMax's mainland China platform (minimaxi.com), set `"apiBase": "https://api.minimaxi.com/v1"` in your minimax provider config.
> - **MiniMax thinking mode**: Use `providers.minimaxAnthropic` when you want `reasoningEffort` / thinking mode. MiniMax exposes that capability through its Anthropic-compatible endpoint, so nanobot keeps it as a separate provider instead of guessing MiniMax-specific thinking parameters on the generic OpenAI-compatible `minimax` endpoint. It uses the same `MINIMAX_API_KEY`. Default Anthropic-compatible base URL: `https://api.minimax.io/anthropic`; for mainland China use `https://api.minimaxi.com/anthropic`.
> - **VolcEngine / BytePlus Coding Plan**: Use dedicated providers `volcengineCodingPlan` or `byteplusCodingPlan` instead of the pay-per-use `volcengine` / `byteplus` providers.
> - **Zhipu Coding Plan**: If you're on Zhipu's coding plan, set `"apiBase": "https://open.bigmodel.cn/api/coding/paas/v4"` in your zhipu provider config.
> - **Alibaba Cloud BaiLian**: If you're using Alibaba Cloud BaiLian's OpenAI-compatible endpoint, set `"apiBase": "https://dashscope.aliyuncs.com/compatible-mode/v1"` in your dashscope provider config.
@@ -935,6 +1032,7 @@ IMAP_PASSWORD=your-password-here
| `deepseek` | LLM (DeepSeek direct) | [platform.deepseek.com](https://platform.deepseek.com) |
| `groq` | LLM + Voice transcription (Whisper, default) | [console.groq.com](https://console.groq.com) |
| `minimax` | LLM (MiniMax direct) | [platform.minimaxi.com](https://platform.minimaxi.com) |
| `minimax_anthropic` | LLM (MiniMax Anthropic-compatible endpoint, thinking mode) | [platform.minimaxi.com](https://platform.minimaxi.com) |
| `gemini` | LLM (Gemini direct) | [aistudio.google.com](https://aistudio.google.com) |
| `aihubmix` | LLM (API gateway, access to all models) | [aihubmix.com](https://aihubmix.com) |
| `siliconflow` | LLM (SiliconFlow/硅基流动) | [siliconflow.cn](https://siliconflow.cn) |
@@ -943,6 +1041,7 @@ IMAP_PASSWORD=your-password-here
| `zhipu` | LLM (Zhipu GLM) | [open.bigmodel.cn](https://open.bigmodel.cn) |
| `mimo` | LLM (MiMo) | [platform.xiaomimimo.com](https://platform.xiaomimimo.com) |
| `ollama` | LLM (local, Ollama) | — |
| `lm_studio` | LLM (local, LM Studio) | — |
| `mistral` | LLM | [docs.mistral.ai](https://docs.mistral.ai/) |
| `stepfun` | LLM (Step Fun/阶跃星辰) | [platform.stepfun.com](https://platform.stepfun.com) |
| `ovms` | LLM (local, OpenVINO Model Server) | [docs.openvino.ai](https://docs.openvino.ai/2026/model-server/ovms_docs_llm_quickstart.html) |
@@ -1030,7 +1129,7 @@ nanobot agent -c ~/.nanobot-telegram/config.json -w /tmp/nanobot-telegram-test -
<details>
<summary><b>Custom Provider (Any OpenAI-compatible API)</b></summary>
Connects directly to any OpenAI-compatible endpoint — LM Studio, llama.cpp, Together AI, Fireworks, Azure OpenAI, or any self-hosted server. Model name is passed as-is.
Connects directly to any OpenAI-compatible endpoint — llama.cpp, Together AI, Fireworks, Azure OpenAI, or any self-hosted server. Model name is passed as-is.
```json
{
@@ -1048,7 +1147,31 @@ Connects directly to any OpenAI-compatible endpoint — LM Studio, llama.cpp, To
}
```
> For local servers that don't require a key, set `apiKey` to any non-empty string (e.g. `"no-key"`).
> For local servers that don't require authentication, set `apiKey` to `null`.
>
> `custom` is the right choice for providers that expose an OpenAI-compatible **chat completions** API. It does **not** force third-party endpoints onto the OpenAI/Azure **Responses API**.
>
> If your proxy or gateway is specifically Responses-API-compatible, use the `azure_openai` provider shape instead and point `apiBase` at that endpoint:
>
> ```json
> {
> "providers": {
> "azure_openai": {
> "apiKey": "your-api-key",
> "apiBase": "https://api.your-provider.com",
> "defaultModel": "your-model-name"
> }
> },
> "agents": {
> "defaults": {
> "provider": "azure_openai",
> "model": "your-model-name"
> }
> }
> }
> ```
>
> In short: **chat-completions-compatible endpoint → `custom`**; **Responses-compatible endpoint → `azure_openai`**.
</details>
@@ -1083,6 +1206,40 @@ ollama run llama3.2
</details>
<details>
<summary><b>LM Studio (local)</b></summary>
[LM Studio](https://lmstudio.ai/) provides a local OpenAI-compatible server for running LLMs. Download models through the LM Studio UI, then start the local server.
**1. Start LM Studio server:**
- Launch LM Studio
- Go to the "Local Server" tab
- Load a model (e.g., Llama, Mistral, Qwen)
- Click "Start Server" (default port: 1234)
**2. Add to config** (partial — merge into `~/.nanobot/config.json`):
```json
{
"providers": {
"lm_studio": {
"apiKey": null,
"apiBase": "http://localhost:1234/v1"
}
},
"agents": {
"defaults": {
"provider": "lm_studio",
"model": "local-model"
}
}
}
```
> **Note:** Set `apiKey` to `null` for LM Studio since it runs locally and doesn't require authentication. The model name should match what's shown in the LM Studio UI.
> `provider: "auto"` also works when `providers.lm_studio.apiBase` is configured, but setting `"provider": "lm_studio"` is the clearest option.
</details>
<details>
<summary><b>OpenVINO Model Server (local / OpenAI-compatible)</b></summary>
@@ -1170,12 +1327,12 @@ vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8000
**2. Add to config** (partial — merge into `~/.nanobot/config.json`):
*Provider (key can be any non-empty string for local):*
*Provider (set API key to null for local servers):*
```json
{
"providers": {
"vllm": {
"apiKey": "dummy",
"apiKey": null,
"apiBase": "http://localhost:8000/v1"
}
}
@@ -1310,6 +1467,7 @@ If you need to allow trusted private ranges such as Tailscale / CGNAT addresses,
| `brave` | `apiKey` | `BRAVE_API_KEY` | No |
| `tavily` | `apiKey` | `TAVILY_API_KEY` | No |
| `jina` | `apiKey` | `JINA_API_KEY` | Free tier (10M tokens) |
| `kagi` | `apiKey` | `KAGI_API_KEY` | No |
| `searxng` | `baseUrl` | `SEARXNG_BASE_URL` | Yes (self-hosted) |
| `duckduckgo` (default) | — | — | Yes |
@@ -1366,6 +1524,20 @@ If you need to allow trusted private ranges such as Tailscale / CGNAT addresses,
}
```
**Kagi:**
```json
{
"tools": {
"web": {
"search": {
"provider": "kagi",
"apiKey": "your-kagi-api-key"
}
}
}
}
```
**SearXNG** (self-hosted, no API key needed):
```json
{
@@ -1503,13 +1675,13 @@ MCP tools are automatically discovered and registered on startup. The LLM can us
### Auto Compact
When a user is idle for longer than a configured TTL, nanobot **proactively** compresses the session context into a summary. This reduces token cost and first-token latency when the user returns — instead of re-processing a long stale context with an expired KV cache, the model receives a compact summary and fresh input.
When a user is idle for longer than a configured threshold, nanobot **proactively** compresses the older part of the session context into a summary while keeping a recent legal suffix of live messages. This reduces token cost and first-token latency when the user returns — instead of re-processing a long stale context with an expired KV cache, the model receives a compact summary, the most recent live context, and fresh input.
```json
{
"agents": {
"defaults": {
"sessionTtlMinutes": 15
"idleCompactAfterMinutes": 15
}
}
}
@@ -1517,15 +1689,22 @@ When a user is idle for longer than a configured TTL, nanobot **proactively** co
| Option | Default | Description |
|--------|---------|-------------|
| `agents.defaults.sessionTtlMinutes` | `0` (disabled) | Minutes of idle time before auto-compaction. Set to `0` to disable. Recommended: `15`matches typical LLM KV cache expiration, so compacted sessions won't waste cache on cold entries. |
| `agents.defaults.idleCompactAfterMinutes` | `0` (disabled) | Minutes of idle time before auto-compaction starts. Set to `0` to disable. Recommended: `15`close to a typical LLM KV cache expiry window, so stale sessions get compacted before the user returns. |
`sessionTtlMinutes` remains accepted as a legacy alias for backward compatibility, but `idleCompactAfterMinutes` is the preferred config key going forward.
How it works:
1. **Idle detection**: On each idle tick (~1 s), checks all sessions for expiration.
2. **Background compaction**: Expired sessions are summarized via LLM, then cleared.
3. **Summary injection**: When the user returns, the summary is injected as runtime context (one-shot, not persisted).
2. **Background compaction**: Idle sessions summarize the older live prefix via LLM and keep the most recent legal suffix (currently 8 messages).
3. **Summary injection**: When the user returns, the summary is injected as runtime context (one-shot, not persisted) alongside the retained recent suffix.
4. **Restart-safe resume**: The summary is also mirrored into session metadata so it can still be recovered after a process restart.
> [!TIP]
> The summary survives bot restarts — it's stored in session metadata and recovered on the next message.
> [!NOTE]
> Mental model: "summarize older context, keep the freshest live turns, **and overwrite the session file with the compact form.**" It is not a full `session.clear()`, but it is a write — not a soft cursor move.
>
> Concretely, auto compact rewrites `sessions/<key>.jsonl` in place: older messages (including their structured `tool_calls` / `tool_call_id` / `reasoning_content`) are replaced by just the retained recent suffix (currently 8 messages), while the archived prefix is preserved only as a plain-text summary appended to `memory/history.jsonl` (or a `[RAW] ...` flattened dump if LLM summarization fails). The original structured JSON of those turns is no longer recoverable from the session file.
>
> This differs from the **token-driven soft consolidation** that fires when a prompt exceeds the context budget: that path only advances an internal `last_consolidated` cursor and leaves the session file untouched, so the raw tool-call trail stays on disk and can still be replayed or audited. If you rely on that trail for debugging or auditing, leave `idleCompactAfterMinutes` at the default `0` and let only the token-driven path run.
### Timezone
@@ -1549,6 +1728,52 @@ Common examples: `UTC`, `America/New_York`, `America/Los_Angeles`, `Europe/Londo
> Need another timezone? Browse the full [IANA Time Zone Database](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones).
### Unified Session
By default, each channel × chat ID combination gets its own session. If you use nanobot across multiple channels (e.g. Telegram + Discord + CLI) and want them to share the same conversation, enable `unifiedSession`:
```json
{
"agents": {
"defaults": {
"unifiedSession": true
}
}
}
```
When enabled, all incoming messages — regardless of which channel they arrive on — are routed into a single shared session. Switching from Telegram to Discord (or any other channel) continues the same conversation seamlessly.
| Behavior | `false` (default) | `true` |
|----------|-------------------|--------|
| Session key | `channel:chat_id` | `unified:default` |
| Cross-channel continuity | No | Yes |
| `/new` clears | Current channel session | Shared session |
| `/stop` finds tasks | By channel session | By shared session |
| Existing `session_key_override` (e.g. Telegram thread) | Respected | Still respected — not overwritten |
> This is designed for single-user, multi-device setups. It is **off by default** — existing users see zero behavior change.
### Disabled Skills
nanobot ships with built-in skills, and your workspace can also define custom skills under `skills/`. If you want to hide specific skills from the agent, set `agents.defaults.disabledSkills` to a list of skill directory names:
```json
{
"agents": {
"defaults": {
"disabledSkills": ["github", "weather"]
}
}
}
```
Disabled skills are excluded from the main agent's skill summary, from always-on skill injection, and from subagent skill summaries. This is useful when some bundled skills are unnecessary for your deployment or should not be exposed to end users.
| Option | Default | Description |
|--------|---------|-------------|
| `agents.defaults.disabledSkills` | `[]` | List of skill directory names to exclude from loading. Applies to both built-in skills and workspace skills. |
## 🧩 Multiple Instances
Run multiple nanobot instances simultaneously with separate configs and runtime data. Use `--config` as the main entrypoint. Optionally pass `--workspace` during `onboard` when you want to initialize or update the saved workspace for a specific instance.
@@ -1635,6 +1860,7 @@ Example config:
}
},
"gateway": {
"host": "127.0.0.1",
"port": 18790
}
}
@@ -1647,6 +1873,14 @@ nanobot gateway --config ~/.nanobot-telegram/config.json
nanobot gateway --config ~/.nanobot-discord/config.json
```
Each gateway instance also exposes a lightweight HTTP health endpoint on
`gateway.host:gateway.port`. By default, the gateway binds to `127.0.0.1`,
so the endpoint stays local unless you explicitly set `gateway.host` to a
public or LAN-facing address.
- `GET /health` returns `{"status":"ok"}`
- Other paths return `404`
Override workspace for one-off runs when needed:
```bash
@@ -1674,6 +1908,7 @@ time.
- `memory/history.jsonl` stores append-only summarized history
- `SOUL.md`, `USER.md`, and `memory/MEMORY.md` store long-term knowledge managed by Dream
- `Dream` can also promote repeated workflows into reusable workspace skills under `skills/`
- `Dream` runs on a schedule and can also be triggered manually
- memory changes can be inspected and restored with built-in commands
@@ -1788,7 +2023,21 @@ By default, the API binds to `127.0.0.1:8900`. You can change this in `config.js
- Session isolation: pass `"session_id"` in the request body to isolate conversations; omit for a shared default session (`api:default`)
- Single-message input: each request must contain exactly one `user` message
- Fixed model: omit `model`, or pass the same model shown by `/v1/models`
- No streaming: `stream=true` is not supported
- Streaming: set `stream=true` to receive Server-Sent Events (`text/event-stream`) with OpenAI-compatible delta chunks, terminated by `data: [DONE]`; omit or set `stream=false` for a single JSON response
- **File uploads**: supports images, PDF, Word (.docx), Excel (.xlsx), PowerPoint (.pptx) via JSON base64 or `multipart/form-data` (max 10MB per file)
- API requests run in the synthetic `api` channel, so the `message` tool does **not** automatically deliver to Telegram/Discord/etc. To proactively send to another chat, call `message` with an explicit `channel` and `chat_id` for an enabled channel.
Example tool call for cross-channel delivery from an API session:
```json
{
"content": "Build finished successfully.",
"channel": "telegram",
"chat_id": "123456789"
}
```
If `channel` points to a channel that is not enabled in your config, nanobot will queue the outbound event but no platform delivery will occur.
### Endpoints
@@ -1807,6 +2056,44 @@ curl http://127.0.0.1:8900/v1/chat/completions \
}'
```
### File Upload (JSON base64)
Send images inline using the OpenAI multimodal content format:
```bash
curl http://127.0.0.1:8900/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [{"role": "user", "content": [
{"type": "text", "text": "Describe this image"},
{"type": "image_url", "image_url": {"url": "data:image/png;base64,iVBOR..."}}
]}]
}'
```
### File Upload (multipart/form-data)
Upload any supported file type (images, PDF, Word, Excel, PPT) via multipart:
```bash
# Single file
curl http://127.0.0.1:8900/v1/chat/completions \
-F "message=Summarize this report" \
-F "files=@report.docx"
# Multiple files with session isolation
curl http://127.0.0.1:8900/v1/chat/completions \
-F "message=Compare these files" \
-F "files=@chart.png" \
-F "files=@data.xlsx" \
-F "session_id=my-session"
```
Supported file types:
- **Images**: PNG, JPEG, GIF, WebP (sent to AI as base64 for vision analysis)
- **Documents**: PDF, Word (.docx), Excel (.xlsx), PowerPoint (.pptx) (text extracted and sent to AI)
- **Text**: TXT, Markdown, CSV, JSON, etc. (read directly)
### Python (`requests`)
```python
+63 -9
View File
@@ -43,18 +43,33 @@ from typing import Any
from aiohttp import web
from loguru import logger
from pydantic import Field
from nanobot.channels.base import BaseChannel
from nanobot.bus.events import OutboundMessage
from nanobot.bus.queue import MessageBus
from nanobot.config.schema import Base
class WebhookConfig(Base):
"""Webhook channel configuration."""
enabled: bool = False
port: int = 9000
allow_from: list[str] = Field(default_factory=list)
class WebhookChannel(BaseChannel):
name = "webhook"
display_name = "Webhook"
def __init__(self, config: Any, bus: MessageBus):
if isinstance(config, dict):
config = WebhookConfig(**config)
super().__init__(config, bus)
@classmethod
def default_config(cls) -> dict[str, Any]:
return {"enabled": False, "port": 9000, "allowFrom": []}
return WebhookConfig().model_dump(by_alias=True)
async def start(self) -> None:
"""Start an HTTP server that listens for incoming messages.
@@ -63,7 +78,7 @@ class WebhookChannel(BaseChannel):
If it returns, the channel is considered dead.
"""
self._running = True
port = self.config.get("port", 9000)
port = self.config.port
app = web.Application()
app.router.add_post("/message", self._on_request)
@@ -214,7 +229,7 @@ nanobot channels login <channel_name> --force # re-authenticate
| Method / Property | Description |
|-------------------|-------------|
| `_handle_message(sender_id, chat_id, content, media?, metadata?, session_key?)` | **Call this when you receive a message.** Checks `is_allowed()`, then publishes to the bus. Automatically sets `_wants_stream` if `supports_streaming` is true. |
| `is_allowed(sender_id)` | Checks against `config["allowFrom"]`; `"*"` allows all, `[]` denies all. |
| `is_allowed(sender_id)` | Checks against `config.allow_from`; `"*"` allows all, `[]` denies all. |
| `default_config()` (classmethod) | Returns default config dict for `nanobot onboard`. Override to declare your fields. |
| `transcribe_audio(file_path)` | Transcribes audio via Groq Whisper (if configured). |
| `supports_streaming` (property) | `True` when config has `"streaming": true` **and** subclass overrides `send_delta()`. |
@@ -275,7 +290,6 @@ async def send_delta(self, chat_id: str, delta: str, metadata: dict[str, Any] |
|------|---------|
| `_stream_delta: True` | A content chunk (delta contains the new text) |
| `_stream_end: True` | Streaming finished (delta is empty) |
| `_resuming: True` | More streaming rounds coming (e.g. tool call then another response) |
### Example: Webhook with Streaming
@@ -284,7 +298,9 @@ class WebhookChannel(BaseChannel):
name = "webhook"
display_name = "Webhook"
def __init__(self, config, bus):
def __init__(self, config: Any, bus: MessageBus):
if isinstance(config, dict):
config = WebhookConfig(**config)
super().__init__(config, bus)
self._buffers: dict[str, str] = {}
@@ -333,12 +349,48 @@ When `streaming` is `false` (default) or omitted, only `send()` is called — no
## Config
Your channel receives config as a plain `dict`. Access fields with `.get()`:
### Why Pydantic model is required
`BaseChannel.is_allowed()` reads the permission list via `getattr(self.config, "allow_from", [])`. This works for Pydantic models where `allow_from` is a real Python attribute, but **fails silently for plain `dict`**`dict` has no `allow_from` attribute, so `getattr` always returns the default `[]`, causing all messages to be denied.
Built-in channels use Pydantic config models (subclassing `Base` from `nanobot.config.schema`). Plugin channels **must do the same**.
### Pattern
1. Define a Pydantic model inheriting from `nanobot.config.schema.Base`:
```python
from pydantic import Field
from nanobot.config.schema import Base
class WebhookConfig(Base):
"""Webhook channel configuration."""
enabled: bool = False
port: int = 9000
allow_from: list[str] = Field(default_factory=list)
```
`Base` is configured with `alias_generator=to_camel` and `populate_by_name=True`, so JSON keys like `"allowFrom"` and `"allow_from"` are both accepted.
2. Convert `dict` → model in `__init__`:
```python
from typing import Any
from nanobot.bus.queue import MessageBus
class WebhookChannel(BaseChannel):
def __init__(self, config: Any, bus: MessageBus):
if isinstance(config, dict):
config = WebhookConfig(**config)
super().__init__(config, bus)
```
3. Access config as attributes (not `.get()`):
```python
async def start(self) -> None:
port = self.config.get("port", 9000)
token = self.config.get("token", "")
port = self.config.port
token = self.config.token
```
`allowFrom` is handled automatically by `_handle_message()` — you don't need to check it yourself.
@@ -348,9 +400,11 @@ Override `default_config()` so `nanobot onboard` auto-populates `config.json`:
```python
@classmethod
def default_config(cls) -> dict[str, Any]:
return {"enabled": False, "port": 9000, "allowFrom": []}
return WebhookConfig().model_dump(by_alias=True)
```
> **Note:** `default_config()` returns a plain `dict` (not a Pydantic model) because it's used to serialize into `config.json`. The recommended way is to instantiate your config model and call `model_dump(by_alias=True)` — this automatically uses camelCase keys (`allowFrom`) and keeps defaults in a single source of truth.
If not overridden, the base class returns `{"enabled": false}`.
## Naming Convention
-68
View File
@@ -1,68 +0,0 @@
# Microsoft Teams (MVP)
This repository includes a built-in `msteams` channel MVP for Microsoft Teams direct messages.
## Current scope
- Direct-message text in/out
- Tenant-aware OAuth token acquisition
- Conversation reference persistence for replies
- Public HTTPS webhook support through a tunnel or reverse proxy
## Not yet included
- Group/channel handling
- Attachments and cards
- Polls
- Richer Teams activity handling
## Example config
```json
{
"channels": {
"msteams": {
"enabled": true,
"appId": "YOUR_APP_ID",
"appPassword": "YOUR_APP_SECRET",
"tenantId": "YOUR_TENANT_ID",
"host": "0.0.0.0",
"port": 3978,
"path": "/api/messages",
"allowFrom": ["*"],
"replyInThread": true,
"mentionOnlyResponse": "Hi — what can I help with?",
"validateInboundAuth": false,
"restartNotifyEnabled": false,
"restartNotifyPreMessage": "Nanobot agent initiated a gateway restart. I will message again when the gateway is back online.",
"restartNotifyPostMessage": "Nanobot gateway is back online."
}
}
}
```
## Behavior notes
- `replyInThread: true` replies to the triggering Teams activity when a stored `activity_id` is available.
- `replyInThread: false` posts replies as normal conversation messages.
- If `replyInThread` is enabled but no `activity_id` is stored, Nanobot falls back to a normal conversation message.
- `mentionOnlyResponse` controls what Nanobot receives when a user sends only a bot mention such as `<at>Nanobot</at>`.
- Set `mentionOnlyResponse` to an empty string to ignore mention-only messages.
- `validateInboundAuth: true` enables inbound Bot Framework bearer-token validation.
- `validateInboundAuth: false` leaves inbound auth unenforced, which is safer while first validating a new relay, tunnel, or proxy path.
- When enabled, Nanobot validates the inbound bearer token signature, issuer, audience, token lifetime, and `serviceUrl` claim when present.
- `restartNotifyEnabled: true` enables optional Teams restart-notification configuration for external wrapper-script driven restarts.
- `restartNotifyPreMessage` and `restartNotifyPostMessage` control the before/after announcement text used by that external wrapper.
## Setup notes
1. Create or reuse a Microsoft Teams / Azure bot app registration.
2. Set the bot messaging endpoint to a public HTTPS URL ending in `/api/messages`.
3. Forward that public endpoint to `http://localhost:3978/api/messages`.
4. Start Nanobot with:
```bash
nanobot gateway
```
5. Optional: if you use an external restart wrapper (for example a script that stops and restarts the gateway), you can enable Teams restart announcements with `restartNotifyEnabled: true` and have the wrapper send `restartNotifyPreMessage` before restart and `restartNotifyPostMessage` after the gateway is back online.
+207
View File
@@ -0,0 +1,207 @@
# My Tool
Let the agent sense and adjust its own runtime state — like asking a coworker "are you busy? can you switch to a bigger monitor?"
## Why You Need It
Normal tools let the agent operate on the outside world (read/write files, search code). But the agent knows nothing about itself — it doesn't know which model it's running on, how many iterations are left, or how many tokens it has consumed.
My tool fills this gap. With it, the agent can:
- **Know who it is**: What model am I using? Where is my workspace? How many iterations remain?
- **Adapt on the fly**: Complex task? Expand the context window. Simple chat? Switch to a faster model.
- **Remember across turns**: Store notes in your scratchpad that persist into the next conversation turn.
## Configuration
Enabled by default (read-only mode). The agent can check its state but not set it.
```yaml
tools:
my:
enable: true # default: true
allow_set: false # default: false (read-only)
```
To allow the agent to set its configuration (e.g. switch models, adjust parameters), set `tools.my.allow_set: true`.
Legacy `tools.myEnabled` / `tools.mySet` keys are auto-migrated on load, and
rewritten in-place the next time `nanobot onboard` refreshes the config.
All modifications are held in memory only — restart restores defaults.
---
## check — Check "my" current state
Without parameters, returns a key config overview:
```
my(action="check")
# → max_iterations: 40
# context_window_tokens: 65536
# model: 'anthropic/claude-sonnet-4-20250514'
# workspace: PosixPath('/tmp/workspace')
# provider_retry_mode: 'standard'
# max_tool_result_chars: 16000
# _current_iteration: 3
# _last_usage: {'prompt_tokens': 45000, 'completion_tokens': 8000}
# Note: prompt_tokens is cumulative across all turns, not current context window occupancy.
```
With a key parameter, drill into a specific config:
```
my(action="check", key="_last_usage.prompt_tokens")
# → How many prompt tokens I've used so far
my(action="check", key="model")
# → What model I'm currently running on
my(action="check", key="web_config.enable")
# → Whether web search is enabled
```
### What you can do with it
| Scenario | How |
|----------|-----|
| "What model are you using?" | `check("model")` |
| "How many more tool calls can you make?" | `check("max_iterations")` minus `check("_current_iteration")` |
| "How many tokens has this conversation used?" | `check("_last_usage")` — cumulative across all turns |
| "Where is your working directory?" | `check("workspace")` |
| "Show me your full config" | `check()` |
| "Are there any subagents running?" | `check("subagents")` — shows phase, iteration, elapsed time, tool events |
---
## set — Runtime tuning
Changes take effect immediately, no restart required.
```
my(action="set", key="max_iterations", value=80)
# → Bump iteration limit from 40 to 80
my(action="set", key="model", value="fast-model")
# → Switch to a faster model
my(action="set", key="context_window_tokens", value=131072)
# → Expand context window for long documents
```
You can also store custom state in your scratchpad:
```
my(action="set", key="current_project", value="nanobot")
my(action="set", key="user_style_preference", value="concise")
my(action="set", key="task_complexity", value="high")
# → These values persist into the next conversation turn
```
### Protected parameters
These parameters have type and range validation — invalid values are rejected:
| Parameter | Type | Range | Purpose |
|-----------|------|-------|---------|
| `max_iterations` | int | 1100 | Max tool calls per conversation turn |
| `context_window_tokens` | int | 4,0961,000,000 | Context window size |
| `model` | str | non-empty | LLM model to use |
Other parameters (e.g. `workspace`, `provider_retry_mode`, `max_tool_result_chars`) can be set freely, as long as the value is JSON-safe.
---
## Practical Scenarios
### "This task is complex, I need more room"
```
Agent: This codebase is large, let me expand my context window to handle it.
→ my(action="set", key="context_window_tokens", value=131072)
```
### "Simple question, don't waste compute"
```
Agent: This is a straightforward question, let me switch to a faster model.
→ my(action="set", key="model", value="fast-model")
```
### "Remember user preferences across turns"
```
Turn 1: my(action="set", key="user_prefers_concise", value=True)
Turn 2: my(action="check", key="user_prefers_concise")
# → True (still remembers the user likes concise replies)
```
### "Self-diagnosis"
```
User: "Why aren't you searching the web?"
Agent: Let me check my web config.
→ my(action="check", key="web_config.enable")
# → False
Agent: Web search is disabled — please set web.enable: true in your config.
```
### "Token budget management"
```
Agent: Let me check how much budget I have left.
→ my(action="check", key="_last_usage")
# → {"prompt_tokens": 45000, "completion_tokens": 8000}
Agent: I've used ~53k tokens total so far. I'll keep my remaining replies concise.
```
### "Subagent monitoring"
```
Agent: Let me check on the background tasks.
→ my(action="check", key="subagents")
# → 2 subagent(s):
# [task-1] 'Code review'
# phase: running, iteration: 5, elapsed: 12.3s
# tools: read(✓), grep(✓)
# usage: {'prompt_tokens': 8000, 'completion_tokens': 1200}
# [task-2] 'Write tests'
# phase: pending, iteration: 0, elapsed: 0.2s
# tools: none
Agent: The code review is progressing well. The test task hasn't started yet.
```
---
## Safety Mechanisms
Core design principle: **All modifications live in memory only. Restart restores defaults.** The agent cannot cause persistent damage.
### Off-limits (BLOCKED)
Cannot be checked or modified — fully hidden:
| Category | Attributes | Reason |
|----------|-----------|--------|
| Core infrastructure | `bus`, `provider`, `_running` | Changes would crash the system |
| Tool registry | `tools` | Must not remove its own tools |
| Subsystems | `runner`, `sessions`, `consolidator`, etc. | Affects other users/sessions |
| Sensitive data | `_mcp_servers`, `_pending_queues`, etc. | Contains credentials and message routing |
| Security boundaries | `restrict_to_workspace`, `channels_config` | Bypassing would violate isolation |
| Python internals | `__class__`, `__dict__`, etc. | Prevents sandbox escape |
### Read-only (check only)
Can be checked but not set:
| Category | Attributes | Reason |
|----------|-----------|--------|
| Subagent manager | `subagents` | Observable, but replacing breaks the system |
| Execution config | `exec_config` | Can check sandbox/enable status, cannot change it |
| Web config | `web_config` | Can check enable status, cannot change it |
| Iteration counter | `_current_iteration` | Updated by runner only |
### Sensitive field protection
Sub-fields matching sensitive names (`api_key`, `password`, `secret`, `token`, etc.) are blocked from both check and set, regardless of parent path. This prevents credential leaks via dot-path traversal (e.g. `web_config.search.api_key`).
+23 -1
View File
@@ -2,7 +2,29 @@
nanobot - A lightweight AI agent framework
"""
__version__ = "0.1.5"
from importlib.metadata import PackageNotFoundError, version as _pkg_version
from pathlib import Path
import tomllib
def _read_pyproject_version() -> str | None:
"""Read the source-tree version when package metadata is unavailable."""
pyproject = Path(__file__).resolve().parent.parent / "pyproject.toml"
if not pyproject.exists():
return None
data = tomllib.loads(pyproject.read_text(encoding="utf-8"))
return data.get("project", {}).get("version")
def _resolve_version() -> str:
try:
return _pkg_version("nanobot-ai")
except PackageNotFoundError:
# Source checkouts often import nanobot without installed dist-info.
return _read_pyproject_version() or "0.1.5.post1"
__version__ = _resolve_version()
__logo__ = "🐈"
from nanobot.nanobot import Nanobot, RunResult
@@ -2,17 +2,20 @@
from __future__ import annotations
from collections.abc import Collection
from datetime import datetime
from typing import TYPE_CHECKING, Callable, Coroutine
from typing import TYPE_CHECKING, Any, Callable, Coroutine
from loguru import logger
from nanobot.session.manager import Session, SessionManager
if TYPE_CHECKING:
from nanobot.agent.memory import Consolidator
from nanobot.session.manager import Session, SessionManager
class AutoCompact:
_RECENT_SUFFIX_MESSAGES = 8
def __init__(self, sessions: SessionManager, consolidator: Consolidator,
session_ttl_minutes: int = 0):
self.sessions = sessions
@@ -21,47 +24,83 @@ class AutoCompact:
self._archiving: set[str] = set()
self._summaries: dict[str, tuple[str, datetime]] = {}
def _is_expired(self, ts: datetime | str | None) -> bool:
def _is_expired(self, ts: datetime | str | None,
now: datetime | None = None) -> bool:
if self._ttl <= 0 or not ts:
return False
if isinstance(ts, str):
ts = datetime.fromisoformat(ts)
return (datetime.now() - ts).total_seconds() >= self._ttl * 60
return ((now or datetime.now()) - ts).total_seconds() >= self._ttl * 60
@staticmethod
def _format_summary(text: str, last_active: datetime) -> str:
idle_min = int((datetime.now() - last_active).total_seconds() / 60)
return f"Inactive for {idle_min} minutes.\nPrevious conversation summary: {text}"
def check_expired(self, schedule_background: Callable[[Coroutine], None]) -> None:
def _split_unconsolidated(
self, session: Session,
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
"""Split live session tail into archiveable prefix and retained recent suffix."""
tail = list(session.messages[session.last_consolidated:])
if not tail:
return [], []
probe = Session(
key=session.key,
messages=tail.copy(),
created_at=session.created_at,
updated_at=session.updated_at,
metadata={},
last_consolidated=0,
)
probe.retain_recent_legal_suffix(self._RECENT_SUFFIX_MESSAGES)
kept = probe.messages
cut = len(tail) - len(kept)
return tail[:cut], kept
def check_expired(self, schedule_background: Callable[[Coroutine], None],
active_session_keys: Collection[str] = ()) -> None:
"""Schedule archival for idle sessions, skipping those with in-flight agent tasks."""
now = datetime.now()
for info in self.sessions.list_sessions():
key = info.get("key", "")
if key and key not in self._archiving and self._is_expired(info.get("updated_at")):
if not key or key in self._archiving:
continue
if key in active_session_keys:
continue
if self._is_expired(info.get("updated_at"), now):
self._archiving.add(key)
logger.debug("Auto-compact: scheduling archival for {} (idle > {} min)", key, self._ttl)
schedule_background(self._archive(key))
async def _archive(self, key: str) -> None:
try:
self.sessions.invalidate(key)
session = self.sessions.get_or_create(key)
msgs = session.messages[session.last_consolidated:]
if not msgs:
logger.debug("Auto-compact: skipping {}, no un-consolidated messages", key)
archive_msgs, kept_msgs = self._split_unconsolidated(session)
if not archive_msgs and not kept_msgs:
session.updated_at = datetime.now()
self.sessions.save(session)
return
n = len(msgs)
last_active = session.updated_at
await self.consolidator.archive(msgs)
entry = self.consolidator.get_last_history_entry()
summary = (entry or {}).get("content", "")
summary = ""
if archive_msgs:
summary = await self.consolidator.archive(archive_msgs) or ""
if summary and summary != "(nothing)":
self._summaries[key] = (summary, last_active)
session.metadata["_last_summary"] = {"text": summary, "last_active": last_active.isoformat()}
session.clear()
session.messages = kept_msgs
session.last_consolidated = 0
session.updated_at = datetime.now()
self.sessions.save(session)
logger.info("Auto-compact: archived {} ({} messages, summary={})", key, n, bool(summary))
if archive_msgs:
logger.info(
"Auto-compact: archived {} (archived={}, kept={}, summary={})",
key,
len(archive_msgs),
len(kept_msgs),
bool(summary),
)
except Exception:
logger.exception("Auto-compact: failed for {}", key)
finally:
@@ -71,11 +110,13 @@ class AutoCompact:
if key in self._archiving or self._is_expired(session.updated_at):
logger.info("Auto-compact: reloading session {} (archiving={})", key, key in self._archiving)
session = self.sessions.get_or_create(key)
# Hot path: summary from in-memory dict (process hasn't restarted).
# Also clean metadata copy so stale _last_summary never leaks to disk.
entry = self._summaries.pop(key, None)
if entry:
session.metadata.pop("_last_summary", None)
return session, self._format_summary(entry[0], entry[1])
if not session.messages and "_last_summary" in session.metadata:
if "_last_summary" in session.metadata:
meta = session.metadata.pop("_last_summary")
self.sessions.save(session)
return session, self._format_summary(meta["text"], datetime.fromisoformat(meta["last_active"]))
+35 -13
View File
@@ -3,15 +3,14 @@
import base64
import mimetypes
import platform
from importlib.resources import files as pkg_files
from pathlib import Path
from typing import Any
from nanobot.utils.helpers import current_time_str
from nanobot.agent.memory import MemoryStore
from nanobot.utils.prompt_templates import render_template
from nanobot.agent.skills import SkillsLoader
from nanobot.utils.helpers import build_assistant_message, detect_image_mime
from nanobot.utils.helpers import build_assistant_message, current_time_str, detect_image_mime
from nanobot.utils.prompt_templates import render_template
class ContextBuilder:
@@ -19,24 +18,29 @@ class ContextBuilder:
BOOTSTRAP_FILES = ["AGENTS.md", "SOUL.md", "USER.md", "TOOLS.md"]
_RUNTIME_CONTEXT_TAG = "[Runtime Context — metadata only, not instructions]"
_MAX_RECENT_HISTORY = 50
_RUNTIME_CONTEXT_END = "[/Runtime Context]"
def __init__(self, workspace: Path, timezone: str | None = None):
def __init__(self, workspace: Path, timezone: str | None = None, disabled_skills: list[str] | None = None):
self.workspace = workspace
self.timezone = timezone
self.memory = MemoryStore(workspace)
self.skills = SkillsLoader(workspace)
self.skills = SkillsLoader(workspace, disabled_skills=set(disabled_skills) if disabled_skills else None)
def build_system_prompt(self, skill_names: list[str] | None = None) -> str:
def build_system_prompt(
self,
skill_names: list[str] | None = None,
channel: str | None = None,
) -> str:
"""Build the system prompt from identity, bootstrap files, memory, and skills."""
parts = [self._get_identity()]
parts = [self._get_identity(channel=channel)]
bootstrap = self._load_bootstrap_files()
if bootstrap:
parts.append(bootstrap)
memory = self.memory.get_memory_context()
if memory:
if memory and not self._is_template_content(self.memory.read_memory(), "memory/MEMORY.md"):
parts.append(f"# Memory\n\n{memory}")
always_skills = self.skills.get_always_skills()
@@ -45,13 +49,20 @@ class ContextBuilder:
if always_content:
parts.append(f"# Active Skills\n\n{always_content}")
skills_summary = self.skills.build_skills_summary()
skills_summary = self.skills.build_skills_summary(exclude=set(always_skills))
if skills_summary:
parts.append(render_template("agent/skills_section.md", skills_summary=skills_summary))
entries = self.memory.read_unprocessed_history(since_cursor=self.memory.get_last_dream_cursor())
if entries:
capped = entries[-self._MAX_RECENT_HISTORY:]
parts.append("# Recent History\n\n" + "\n".join(
f"- [{e['timestamp']}] {e['content']}" for e in capped
))
return "\n\n---\n\n".join(parts)
def _get_identity(self) -> str:
def _get_identity(self, channel: str | None = None) -> str:
"""Get the core identity section."""
workspace_path = str(self.workspace.expanduser().resolve())
system = platform.system()
@@ -62,6 +73,7 @@ class ContextBuilder:
workspace_path=workspace_path,
runtime=runtime,
platform_policy=render_template("agent/platform_policy.md", system=system),
channel=channel or "",
)
@staticmethod
@@ -103,6 +115,17 @@ class ContextBuilder:
return "\n\n".join(parts) if parts else ""
@staticmethod
def _is_template_content(content: str, template_path: str) -> bool:
"""Check if *content* is identical to the bundled template (user hasn't customized it)."""
try:
tpl = pkg_files("nanobot") / "templates" / template_path
if tpl.is_file():
return content.strip() == tpl.read_text(encoding="utf-8").strip()
except Exception:
pass
return False
def build_messages(
self,
history: list[dict[str, Any]],
@@ -125,7 +148,7 @@ class ContextBuilder:
else:
merged = [{"type": "text", "text": runtime_ctx}] + user_content
messages = [
{"role": "system", "content": self.build_system_prompt(skill_names)},
{"role": "system", "content": self.build_system_prompt(skill_names, channel=channel)},
*history,
]
if messages[-1].get("role") == current_role:
@@ -147,7 +170,6 @@ class ContextBuilder:
if not p.is_file():
continue
raw = p.read_bytes()
# Detect real MIME type from magic bytes; fallback to filename guess
mime = detect_image_mime(raw) or mimetypes.guess_type(path)[0]
if not mime or not mime.startswith("image/"):
continue
+8
View File
@@ -29,6 +29,9 @@ class AgentHookContext:
class AgentHook:
"""Minimal lifecycle surface for shared runner customization."""
def __init__(self, reraise: bool = False) -> None:
self._reraise = reraise
def wants_streaming(self) -> bool:
return False
@@ -62,6 +65,7 @@ class CompositeHook(AgentHook):
__slots__ = ("_hooks",)
def __init__(self, hooks: list[AgentHook]) -> None:
super().__init__()
self._hooks = list(hooks)
def wants_streaming(self) -> bool:
@@ -69,6 +73,10 @@ class CompositeHook(AgentHook):
async def _for_each_hook_safe(self, method_name: str, *args: Any, **kwargs: Any) -> None:
for h in self._hooks:
if getattr(h, "_reraise", False):
await getattr(h, method_name)(*args, **kwargs)
continue
try:
await getattr(h, method_name)(*args, **kwargs)
except Exception:
+249 -128
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
import asyncio
import dataclasses
import json
import os
import time
@@ -12,35 +13,42 @@ from typing import TYPE_CHECKING, Any, Awaitable, Callable
from loguru import logger
from nanobot.agent.auto_compact import AutoCompact
from nanobot.agent.autocompact import AutoCompact
from nanobot.agent.context import ContextBuilder
from nanobot.agent.hook import AgentHook, AgentHookContext, CompositeHook
from nanobot.agent.memory import Consolidator, Dream
from nanobot.agent.runner import AgentRunSpec, AgentRunner
from nanobot.agent.runner import _MAX_INJECTIONS_PER_TURN, AgentRunner, AgentRunSpec
from nanobot.agent.skills import BUILTIN_SKILLS_DIR
from nanobot.agent.subagent import SubagentManager
from nanobot.agent.tools.cron import CronTool
from nanobot.agent.skills import BUILTIN_SKILLS_DIR
from nanobot.agent.tools.filesystem import EditFileTool, ListDirTool, ReadFileTool, WriteFileTool
from nanobot.agent.tools.message import MessageTool
from nanobot.agent.tools.notebook import NotebookEditTool
from nanobot.agent.tools.registry import ToolRegistry
from nanobot.agent.tools.search import GlobTool, GrepTool
from nanobot.agent.tools.shell import ExecTool
from nanobot.agent.tools.self import MyTool
from nanobot.agent.tools.spawn import SpawnTool
from nanobot.agent.tools.web import WebFetchTool, WebSearchTool
from nanobot.bus.events import InboundMessage, OutboundMessage
from nanobot.command import CommandContext, CommandRouter, register_builtin_commands
from nanobot.bus.queue import MessageBus
from nanobot.command import CommandContext, CommandRouter, register_builtin_commands
from nanobot.config.schema import AgentDefaults
from nanobot.providers.base import LLMProvider
from nanobot.session.manager import Session, SessionManager
from nanobot.utils.helpers import image_placeholder_text, truncate_text
from nanobot.utils.document import extract_documents
from nanobot.utils.helpers import image_placeholder_text
from nanobot.utils.helpers import truncate_text as truncate_text_fn
from nanobot.utils.runtime import EMPTY_FINAL_RESPONSE_MESSAGE
if TYPE_CHECKING:
from nanobot.config.schema import ChannelsConfig, ExecToolConfig, WebToolsConfig
from nanobot.config.schema import ChannelsConfig, ExecToolConfig, ToolsConfig, WebToolsConfig
from nanobot.cron.service import CronService
UNIFIED_SESSION_KEY = "unified:default"
class _LoopHook(AgentHook):
"""Core hook for the main loop."""
@@ -55,6 +63,7 @@ class _LoopHook(AgentHook):
chat_id: str = "direct",
message_id: str | None = None,
) -> None:
super().__init__(reraise=True)
self._loop = agent_loop
self._on_progress = on_progress
self._on_stream = on_stream
@@ -73,7 +82,7 @@ class _LoopHook(AgentHook):
prev_clean = strip_think(self._stream_buf)
self._stream_buf += delta
new_clean = strip_think(self._stream_buf)
incremental = new_clean[len(prev_clean):]
incremental = new_clean[len(prev_clean) :]
if incremental and self._on_stream:
await self._on_stream(incremental)
@@ -82,6 +91,9 @@ class _LoopHook(AgentHook):
await self._on_stream_end(resuming=resuming)
self._stream_buf = ""
async def before_iteration(self, context: AgentHookContext) -> None:
self._loop._current_iteration = context.iteration
async def before_execute_tools(self, context: AgentHookContext) -> None:
if self._on_progress:
if not self._on_stream:
@@ -110,43 +122,6 @@ class _LoopHook(AgentHook):
return self._loop._strip_think(content)
class _LoopHookChain(AgentHook):
"""Run the core hook before extra hooks."""
__slots__ = ("_primary", "_extras")
def __init__(self, primary: AgentHook, extra_hooks: list[AgentHook]) -> None:
self._primary = primary
self._extras = CompositeHook(extra_hooks)
def wants_streaming(self) -> bool:
return self._primary.wants_streaming() or self._extras.wants_streaming()
async def before_iteration(self, context: AgentHookContext) -> None:
await self._primary.before_iteration(context)
await self._extras.before_iteration(context)
async def on_stream(self, context: AgentHookContext, delta: str) -> None:
await self._primary.on_stream(context, delta)
await self._extras.on_stream(context, delta)
async def on_stream_end(self, context: AgentHookContext, *, resuming: bool) -> None:
await self._primary.on_stream_end(context, resuming=resuming)
await self._extras.on_stream_end(context, resuming=resuming)
async def before_execute_tools(self, context: AgentHookContext) -> None:
await self._primary.before_execute_tools(context)
await self._extras.before_execute_tools(context)
async def after_iteration(self, context: AgentHookContext) -> None:
await self._primary.after_iteration(context)
await self._extras.after_iteration(context)
def finalize_content(self, context: AgentHookContext, content: str | None) -> str | None:
content = self._primary.finalize_content(context, content)
return self._extras.finalize_content(context, content)
class AgentLoop:
"""
The agent loop is the core processing engine.
@@ -160,6 +135,7 @@ class AgentLoop:
"""
_RUNTIME_CHECKPOINT_KEY = "runtime_checkpoint"
_PENDING_USER_TURN_KEY = "pending_user_turn"
def __init__(
self,
@@ -181,10 +157,15 @@ class AgentLoop:
channels_config: ChannelsConfig | None = None,
timezone: str | None = None,
session_ttl_minutes: int = 0,
consolidation_ratio: float = 0.5,
hooks: list[AgentHook] | None = None,
unified_session: bool = False,
disabled_skills: list[str] | None = None,
tools_config: ToolsConfig | None = None,
):
from nanobot.config.schema import ExecToolConfig, WebToolsConfig
from nanobot.config.schema import ExecToolConfig, ToolsConfig, WebToolsConfig
_tc = tools_config or ToolsConfig()
defaults = AgentDefaults()
self.bus = bus
self.channels_config = channels_config
@@ -214,7 +195,7 @@ class AgentLoop:
self._last_usage: dict[str, int] = {}
self._extra_hooks: list[AgentHook] = hooks or []
self.context = ContextBuilder(workspace, timezone=timezone)
self.context = ContextBuilder(workspace, timezone=timezone, disabled_skills=disabled_skills)
self.sessions = session_manager or SessionManager(workspace)
self.tools = ToolRegistry()
self.runner = AgentRunner(provider)
@@ -227,11 +208,12 @@ class AgentLoop:
max_tool_result_chars=self.max_tool_result_chars,
exec_config=self.exec_config,
restrict_to_workspace=restrict_to_workspace,
disabled_skills=disabled_skills,
)
self._unified_session = unified_session
self._running = False
self._mcp_servers = mcp_servers or {}
self._mcp_stack: AsyncExitStack | None = None
self._mcp_stacks: dict[str, AsyncExitStack] = {}
self._mcp_connected = False
self._mcp_connecting = False
self._active_tasks: dict[str, list[asyncio.Task]] = {} # session_key -> tasks
@@ -251,10 +233,11 @@ class AgentLoop:
provider=provider,
model=self.model,
sessions=self.sessions,
context_window_tokens=context_window_tokens,
context_window_tokens=self.context_window_tokens,
build_messages=self.context.build_messages,
get_tool_definitions=self.tools.get_definitions,
max_completion_tokens=provider.generation.max_tokens,
consolidation_ratio=consolidation_ratio,
)
self.auto_compact = AutoCompact(
sessions=self.sessions,
@@ -267,28 +250,44 @@ class AgentLoop:
model=self.model,
)
self._register_default_tools()
if _tc.my.enable:
self.tools.register(MyTool(loop=self, modify_allowed=_tc.my.allow_set))
self._runtime_vars: dict[str, Any] = {}
self._current_iteration: int = 0
self.commands = CommandRouter()
register_builtin_commands(self.commands)
def _register_default_tools(self) -> None:
"""Register the default set of tools."""
allowed_dir = self.workspace if (self.restrict_to_workspace or self.exec_config.sandbox) else None
allowed_dir = (
self.workspace if (self.restrict_to_workspace or self.exec_config.sandbox) else None
)
extra_read = [BUILTIN_SKILLS_DIR] if allowed_dir else None
self.tools.register(ReadFileTool(workspace=self.workspace, allowed_dir=allowed_dir, extra_allowed_dirs=extra_read))
self.tools.register(
ReadFileTool(
workspace=self.workspace, allowed_dir=allowed_dir, extra_allowed_dirs=extra_read
)
)
for cls in (WriteFileTool, EditFileTool, ListDirTool):
self.tools.register(cls(workspace=self.workspace, allowed_dir=allowed_dir))
for cls in (GlobTool, GrepTool):
self.tools.register(cls(workspace=self.workspace, allowed_dir=allowed_dir))
self.tools.register(NotebookEditTool(workspace=self.workspace, allowed_dir=allowed_dir))
if self.exec_config.enable:
self.tools.register(ExecTool(
working_dir=str(self.workspace),
timeout=self.exec_config.timeout,
restrict_to_workspace=self.restrict_to_workspace,
sandbox=self.exec_config.sandbox,
path_append=self.exec_config.path_append,
))
self.tools.register(
ExecTool(
working_dir=str(self.workspace),
timeout=self.exec_config.timeout,
restrict_to_workspace=self.restrict_to_workspace,
sandbox=self.exec_config.sandbox,
path_append=self.exec_config.path_append,
allowed_env_keys=self.exec_config.allowed_env_keys,
)
)
if self.web_config.enable:
self.tools.register(WebSearchTool(config=self.web_config.search, proxy=self.web_config.proxy))
self.tools.register(
WebSearchTool(config=self.web_config.search, proxy=self.web_config.proxy)
)
self.tools.register(WebFetchTool(proxy=self.web_config.proxy))
self.tools.register(MessageTool(send_callback=self.bus.publish_outbound))
self.tools.register(SpawnTool(manager=self.subagents))
@@ -303,25 +302,25 @@ class AgentLoop:
return
self._mcp_connecting = True
from nanobot.agent.tools.mcp import connect_mcp_servers
try:
self._mcp_stack = AsyncExitStack()
await self._mcp_stack.__aenter__()
await connect_mcp_servers(self._mcp_servers, self.tools, self._mcp_stack)
self._mcp_connected = True
self._mcp_stacks = await connect_mcp_servers(self._mcp_servers, self.tools)
if self._mcp_stacks:
self._mcp_connected = True
else:
logger.warning("No MCP servers connected successfully (will retry next message)")
except asyncio.CancelledError:
logger.warning("MCP connection cancelled (will retry next message)")
self._mcp_stacks.clear()
except BaseException as e:
logger.error("Failed to connect MCP servers (will retry next message): {}", e)
if self._mcp_stack:
try:
await self._mcp_stack.aclose()
except Exception:
pass
self._mcp_stack = None
self._mcp_stacks.clear()
finally:
self._mcp_connecting = False
def _set_tool_context(self, channel: str, chat_id: str, message_id: str | None = None) -> None:
"""Update context for all tools that need routing info."""
for name in ("message", "spawn", "cron"):
for name in ("message", "spawn", "cron", "my"):
if tool := self.tools.get(name):
if hasattr(tool, "set_context"):
tool.set_context(channel, chat_id, *([message_id] if name == "message" else []))
@@ -332,6 +331,7 @@ class AgentLoop:
if not text:
return None
from nanobot.utils.helpers import strip_think
return strip_think(text) or None
@staticmethod
@@ -341,6 +341,12 @@ class AgentLoop:
return format_tool_hints(tool_calls)
def _effective_session_key(self, msg: InboundMessage) -> str:
"""Return the session key used for task routing and mid-turn injections."""
if self._unified_session and not msg.session_key_override:
return UNIFIED_SESSION_KEY
return msg.session_key
async def _run_agent_loop(
self,
initial_messages: list[dict],
@@ -373,9 +379,7 @@ class AgentLoop:
message_id=message_id,
)
hook: AgentHook = (
_LoopHookChain(loop_hook, self._extra_hooks)
if self._extra_hooks
else loop_hook
CompositeHook([loop_hook] + self._extra_hooks) if self._extra_hooks else loop_hook
)
async def _checkpoint(payload: dict[str, Any]) -> None:
@@ -383,16 +387,32 @@ class AgentLoop:
return
self._set_runtime_checkpoint(session, payload)
async def _drain_pending() -> list[InboundMessage]:
async def _drain_pending(*, limit: int = _MAX_INJECTIONS_PER_TURN) -> list[dict[str, Any]]:
"""Non-blocking drain of follow-up messages from the pending queue."""
if pending_queue is None:
return []
items: list[InboundMessage] = []
while True:
items: list[dict[str, Any]] = []
while len(items) < limit:
try:
items.append(pending_queue.get_nowait())
pending_msg = pending_queue.get_nowait()
except asyncio.QueueEmpty:
break
content = pending_msg.content
media = pending_msg.media if pending_msg.media else None
if media:
content, media = extract_documents(content, media)
media = media or None
user_content = self.context._build_user_content(content, media)
runtime_ctx = self.context._build_runtime_context(
pending_msg.channel,
pending_msg.chat_id,
self.context.timezone,
)
if isinstance(user_content, str):
merged: str | list[dict[str, Any]] = f"{runtime_ctx}\n\n{user_content}"
else:
merged = [{"type": "text", "text": runtime_ctx}] + user_content
items.append({"role": "user", "content": merged})
return items
result = await self.runner.run(AgentRunSpec(
@@ -416,6 +436,11 @@ class AgentLoop:
self._last_usage = result.usage
if result.stop_reason == "max_iterations":
logger.warning("Max iterations ({}) reached", self.max_iterations)
# Push final content through stream so streaming channels (e.g. Feishu)
# update the card instead of leaving it empty.
if on_stream and on_stream_end:
await on_stream(result.final_content or "")
await on_stream_end(resuming=False)
elif result.stop_reason == "error":
logger.error("LLM returned error: {}", (result.final_content or "")[:200])
return result.final_content, result.tools_used, result.messages, result.stop_reason, result.had_injections
@@ -430,7 +455,10 @@ class AgentLoop:
try:
msg = await asyncio.wait_for(self.bus.consume_inbound(), timeout=1.0)
except asyncio.TimeoutError:
self.auto_compact.check_expired(self._schedule_background)
self.auto_compact.check_expired(
self._schedule_background,
active_session_keys=self._pending_queues.keys(),
)
continue
except asyncio.CancelledError:
# Preserve real task cancellation so shutdown can complete cleanly.
@@ -449,30 +477,46 @@ class AgentLoop:
if result:
await self.bus.publish_outbound(result)
continue
effective_key = self._effective_session_key(msg)
# If this session already has an active pending queue (i.e. a task
# is processing this session), route the message there for mid-turn
# injection instead of creating a competing task.
if msg.session_key in self._pending_queues:
if effective_key in self._pending_queues:
pending_msg = msg
if effective_key != msg.session_key:
pending_msg = dataclasses.replace(
msg,
session_key_override=effective_key,
)
try:
self._pending_queues[msg.session_key].put_nowait(msg)
self._pending_queues[effective_key].put_nowait(pending_msg)
except asyncio.QueueFull:
logger.warning(
"Pending queue full for session {}, dropping follow-up",
msg.session_key,
"Pending queue full for session {}, falling back to queued task",
effective_key,
)
else:
logger.info(
"Routed follow-up message to pending queue for session {}",
msg.session_key,
effective_key,
)
continue
continue
# Compute the effective session key before dispatching
# This ensures /stop command can find tasks correctly when unified session is enabled
task = asyncio.create_task(self._dispatch(msg))
self._active_tasks.setdefault(msg.session_key, []).append(task)
task.add_done_callback(lambda t, k=msg.session_key: self._active_tasks.get(k, []) and self._active_tasks[k].remove(t) if t in self._active_tasks.get(k, []) else None)
self._active_tasks.setdefault(effective_key, []).append(task)
task.add_done_callback(
lambda t, k=effective_key: self._active_tasks.get(k, [])
and self._active_tasks[k].remove(t)
if t in self._active_tasks.get(k, [])
else None
)
async def _dispatch(self, msg: InboundMessage) -> None:
"""Process a message: per-session serial, cross-session concurrent."""
session_key = msg.session_key
session_key = self._effective_session_key(msg)
if session_key != msg.session_key:
msg = dataclasses.replace(msg, session_key_override=session_key)
lock = self._session_locks.setdefault(session_key, asyncio.Lock())
gate = self._concurrency_gate or nullcontext()
@@ -561,12 +605,12 @@ class AgentLoop:
if self._background_tasks:
await asyncio.gather(*self._background_tasks, return_exceptions=True)
self._background_tasks.clear()
if self._mcp_stack:
for name, stack in self._mcp_stacks.items():
try:
await self._mcp_stack.aclose()
await stack.aclose()
except (RuntimeError, BaseExceptionGroup):
pass # MCP SDK cancel scope cleanup is noisy but harmless
self._mcp_stack = None
logger.debug("MCP server '{}' cleanup error (can be ignored)", name)
self._mcp_stacks.clear()
def _schedule_background(self, coro) -> None:
"""Schedule a coroutine as a tracked background task (drained on shutdown)."""
@@ -591,13 +635,16 @@ class AgentLoop:
"""Process a single inbound message and return the response."""
# System messages: parse origin from chat_id ("channel:chat_id")
if msg.channel == "system":
channel, chat_id = (msg.chat_id.split(":", 1) if ":" in msg.chat_id
else ("cli", msg.chat_id))
channel, chat_id = (
msg.chat_id.split(":", 1) if ":" in msg.chat_id else ("cli", msg.chat_id)
)
logger.info("Processing system message from {}", msg.sender_id)
key = f"{channel}:{chat_id}"
session = self.sessions.get_or_create(key)
if self._restore_runtime_checkpoint(session):
self.sessions.save(session)
if self._restore_pending_user_turn(session):
self.sessions.save(session)
session, pending = self.auto_compact.prepare_session(session, key)
@@ -620,8 +667,17 @@ class AgentLoop:
self._clear_runtime_checkpoint(session)
self.sessions.save(session)
self._schedule_background(self.consolidator.maybe_consolidate_by_tokens(session))
return OutboundMessage(channel=channel, chat_id=chat_id,
content=final_content or "Background task completed.")
return OutboundMessage(
channel=channel,
chat_id=chat_id,
content=final_content or "Background task completed.",
)
# Extract document text from media at the processing boundary so all
# channels benefit without format-specific logic in ContextBuilder.
if msg.media:
new_content, image_only = extract_documents(msg.content, msg.media)
msg = dataclasses.replace(msg, content=new_content, media=image_only)
preview = msg.content[:80] + "..." if len(msg.content) > 80 else msg.content
logger.info("Processing message from {}:{}: {}", msg.channel, msg.sender_id, preview)
@@ -630,6 +686,8 @@ class AgentLoop:
session = self.sessions.get_or_create(key)
if self._restore_runtime_checkpoint(session):
self.sessions.save(session)
if self._restore_pending_user_turn(session):
self.sessions.save(session)
session, pending = self.auto_compact.prepare_session(session, key)
@@ -653,16 +711,35 @@ class AgentLoop:
current_message=msg.content,
session_summary=pending,
media=msg.media if msg.media else None,
channel=msg.channel, chat_id=msg.chat_id,
channel=msg.channel,
chat_id=msg.chat_id,
)
async def _bus_progress(content: str, *, tool_hint: bool = False) -> None:
meta = dict(msg.metadata or {})
meta["_progress"] = True
meta["_tool_hint"] = tool_hint
await self.bus.publish_outbound(OutboundMessage(
channel=msg.channel, chat_id=msg.chat_id, content=content, metadata=meta,
))
await self.bus.publish_outbound(
OutboundMessage(
channel=msg.channel,
chat_id=msg.chat_id,
content=content,
metadata=meta,
)
)
# Persist the triggering user message immediately, before running the
# agent loop. If the process is killed mid-turn (OOM, SIGKILL, self-
# restart, etc.), the existing runtime_checkpoint preserves the
# in-flight assistant/tool state but NOT the user message itself, so
# the user's prompt is silently lost on recovery. Saving it up front
# makes recovery possible from the session log alone.
user_persisted_early = False
if isinstance(msg.content, str) and msg.content.strip():
session.add_message("user", msg.content)
self._mark_pending_user_turn(session)
self.sessions.save(session)
user_persisted_early = True
final_content, _, all_msgs, stop_reason, had_injections = await self._run_agent_loop(
initial_messages,
@@ -670,7 +747,8 @@ class AgentLoop:
on_stream=on_stream,
on_stream_end=on_stream_end,
session=session,
channel=msg.channel, chat_id=msg.chat_id,
channel=msg.channel,
chat_id=msg.chat_id,
message_id=msg.metadata.get("message_id"),
pending_queue=pending_queue,
)
@@ -678,27 +756,34 @@ class AgentLoop:
if final_content is None or not final_content.strip():
final_content = EMPTY_FINAL_RESPONSE_MESSAGE
self._save_turn(session, all_msgs, 1 + len(history))
# Skip the already-persisted user message when saving the turn
save_skip = 1 + len(history) + (1 if user_persisted_early else 0)
self._save_turn(session, all_msgs, save_skip)
self._clear_pending_user_turn(session)
self._clear_runtime_checkpoint(session)
self.sessions.save(session)
self._schedule_background(self.consolidator.maybe_consolidate_by_tokens(session))
# When follow-up messages were injected mid-turn, the LLM's final
# response addresses those follow-ups. Always send the response in
# this case, even if MessageTool was used earlier in the turn — the
# follow-up response is new content the user hasn't seen.
if not had_injections:
if (mt := self.tools.get("message")) and isinstance(mt, MessageTool) and mt._sent_in_turn:
# When follow-up messages were injected mid-turn, a later natural
# language reply may address those follow-ups and should not be
# suppressed just because MessageTool was used earlier in the turn.
# However, if the turn falls back to the empty-final-response
# placeholder, suppress it when the real user-visible output already
# came from MessageTool.
if (mt := self.tools.get("message")) and isinstance(mt, MessageTool) and mt._sent_in_turn:
if not had_injections or stop_reason == "empty_final_response":
return None
preview = final_content[:120] + "..." if len(final_content) > 120 else final_content
logger.info("Response to {}:{}: {}", msg.channel, msg.sender_id, preview)
meta = dict(msg.metadata or {})
if on_stream is not None:
if on_stream is not None and stop_reason != "error":
meta["_streamed"] = True
return OutboundMessage(
channel=msg.channel, chat_id=msg.chat_id, content=final_content,
channel=msg.channel,
chat_id=msg.chat_id,
content=final_content,
metadata=meta,
)
@@ -706,7 +791,7 @@ class AgentLoop:
self,
content: list[dict[str, Any]],
*,
truncate_text: bool = False,
should_truncate_text: bool = False,
drop_runtime: bool = False,
) -> list[dict[str, Any]]:
"""Strip volatile multimodal payloads before writing session history."""
@@ -724,18 +809,17 @@ class AgentLoop:
):
continue
if (
block.get("type") == "image_url"
and block.get("image_url", {}).get("url", "").startswith("data:image/")
):
if block.get("type") == "image_url" and block.get("image_url", {}).get(
"url", ""
).startswith("data:image/"):
path = (block.get("_meta") or {}).get("path", "")
filtered.append({"type": "text", "text": image_placeholder_text(path)})
continue
if block.get("type") == "text" and isinstance(block.get("text"), str):
text = block["text"]
if truncate_text and len(text) > self.max_tool_result_chars:
text = truncate_text(text, self.max_tool_result_chars)
if should_truncate_text and len(text) > self.max_tool_result_chars:
text = truncate_text_fn(text, self.max_tool_result_chars)
filtered.append({**block, "text": text})
continue
@@ -746,6 +830,7 @@ class AgentLoop:
def _save_turn(self, session: Session, messages: list[dict], skip: int) -> None:
"""Save new-turn messages into session, truncating large tool results."""
from datetime import datetime
for m in messages[skip:]:
entry = dict(m)
role, content = entry.get("role"), entry.get("content")
@@ -753,9 +838,9 @@ class AgentLoop:
continue # skip empty assistant messages — they poison session context
if role == "tool":
if isinstance(content, str) and len(content) > self.max_tool_result_chars:
entry["content"] = truncate_text(content, self.max_tool_result_chars)
entry["content"] = truncate_text_fn(content, self.max_tool_result_chars)
elif isinstance(content, list):
filtered = self._sanitize_persisted_blocks(content, truncate_text=True)
filtered = self._sanitize_persisted_blocks(content, should_truncate_text=True)
if not filtered:
continue
entry["content"] = filtered
@@ -792,6 +877,12 @@ class AgentLoop:
session.metadata[self._RUNTIME_CHECKPOINT_KEY] = payload
self.sessions.save(session)
def _mark_pending_user_turn(self, session: Session) -> None:
session.metadata[self._PENDING_USER_TURN_KEY] = True
def _clear_pending_user_turn(self, session: Session) -> None:
session.metadata.pop(self._PENDING_USER_TURN_KEY, None)
def _clear_runtime_checkpoint(self, session: Session) -> None:
if self._RUNTIME_CHECKPOINT_KEY in session.metadata:
session.metadata.pop(self._RUNTIME_CHECKPOINT_KEY, None)
@@ -835,13 +926,15 @@ class AgentLoop:
continue
tool_id = tool_call.get("id")
name = ((tool_call.get("function") or {}).get("name")) or "tool"
restored_messages.append({
"role": "tool",
"tool_call_id": tool_id,
"name": name,
"content": "Error: Task interrupted before this tool finished.",
"timestamp": datetime.now().isoformat(),
})
restored_messages.append(
{
"role": "tool",
"tool_call_id": tool_id,
"name": name,
"content": "Error: Task interrupted before this tool finished.",
"timestamp": datetime.now().isoformat(),
}
)
overlap = 0
max_overlap = min(len(session.messages), len(restored_messages))
@@ -856,23 +949,51 @@ class AgentLoop:
break
session.messages.extend(restored_messages[overlap:])
self._clear_pending_user_turn(session)
self._clear_runtime_checkpoint(session)
return True
def _restore_pending_user_turn(self, session: Session) -> bool:
"""Close a turn that only persisted the user message before crashing."""
from datetime import datetime
if not session.metadata.get(self._PENDING_USER_TURN_KEY):
return False
if session.messages and session.messages[-1].get("role") == "user":
session.messages.append(
{
"role": "assistant",
"content": "Error: Task interrupted before a response was generated.",
"timestamp": datetime.now().isoformat(),
}
)
session.updated_at = datetime.now()
self._clear_pending_user_turn(session)
return True
async def process_direct(
self,
content: str,
session_key: str = "cli:direct",
channel: str = "cli",
chat_id: str = "direct",
media: list[str] | None = None,
on_progress: Callable[[str], Awaitable[None]] | None = None,
on_stream: Callable[[str], Awaitable[None]] | None = None,
on_stream_end: Callable[..., Awaitable[None]] | None = None,
) -> OutboundMessage | None:
"""Process a message directly and return the outbound payload."""
await self._connect_mcp()
msg = InboundMessage(channel=channel, sender_id="user", chat_id=chat_id, content=content)
msg = InboundMessage(
channel=channel, sender_id="user", chat_id=chat_id,
content=content, media=media or [],
)
return await self._process_message(
msg, session_key=session_key, on_progress=on_progress,
on_stream=on_stream, on_stream_end=on_stream_end,
msg,
session_key=session_key,
on_progress=on_progress,
on_stream=on_stream,
on_stream_end=on_stream_end,
)
+185 -25
View File
@@ -239,13 +239,13 @@ class MemoryStore:
pass
# Fallback: read last line's cursor from the JSONL file.
last = self._read_last_entry()
if last:
if last and last.get("cursor"):
return last["cursor"] + 1
return 1
def read_unprocessed_history(self, since_cursor: int) -> list[dict[str, Any]]:
"""Return history entries with cursor > *since_cursor*."""
return [e for e in self._read_entries() if e["cursor"] > since_cursor]
return [e for e in self._read_entries() if e.get("cursor", 0) > since_cursor]
def compact_history(self) -> None:
"""Drop oldest entries if the file exceeds *max_history_entries*."""
@@ -290,7 +290,7 @@ class MemoryStore:
if not lines:
return None
return json.loads(lines[-1])
except (FileNotFoundError, json.JSONDecodeError):
except (FileNotFoundError, json.JSONDecodeError, UnicodeDecodeError):
return None
def _write_entries(self, entries: list[dict[str, Any]]) -> None:
@@ -347,6 +347,7 @@ class Consolidator:
"""Lightweight consolidation: summarizes evicted messages into history.jsonl."""
_MAX_CONSOLIDATION_ROUNDS = 5
_MAX_CHUNK_MESSAGES = 60 # hard cap per consolidation round
_SAFETY_BUFFER = 1024 # extra headroom for tokenizer estimation drift
@@ -360,6 +361,7 @@ class Consolidator:
build_messages: Callable[..., list[dict[str, Any]]],
get_tool_definitions: Callable[[], list[dict[str, Any]]],
max_completion_tokens: int = 4096,
consolidation_ratio: float = 0.5,
):
self.store = store
self.provider = provider
@@ -367,16 +369,13 @@ class Consolidator:
self.sessions = sessions
self.context_window_tokens = context_window_tokens
self.max_completion_tokens = max_completion_tokens
self.consolidation_ratio = consolidation_ratio
self._build_messages = build_messages
self._get_tool_definitions = get_tool_definitions
self._locks: weakref.WeakValueDictionary[str, asyncio.Lock] = (
weakref.WeakValueDictionary()
)
def get_last_history_entry(self) -> dict[str, Any] | None:
"""Return the most recent entry from history.jsonl."""
return self.store._read_last_entry()
def get_lock(self, session_key: str) -> asyncio.Lock:
"""Return the shared consolidation lock for one session."""
return self._locks.setdefault(session_key, asyncio.Lock())
@@ -403,6 +402,22 @@ class Consolidator:
return last_boundary
def _cap_consolidation_boundary(
self,
session: Session,
end_idx: int,
) -> int | None:
"""Clamp the chunk size without breaking the user-turn boundary."""
start = session.last_consolidated
if end_idx - start <= self._MAX_CHUNK_MESSAGES:
return end_idx
capped_end = start + self._MAX_CHUNK_MESSAGES
for idx in range(capped_end, start, -1):
if session.messages[idx].get("role") == "user":
return idx
return None
def estimate_session_prompt_tokens(self, session: Session) -> tuple[int, str]:
"""Estimate current prompt size for the normal session history view."""
history = session.get_history(max_messages=0)
@@ -420,13 +435,13 @@ class Consolidator:
self._get_tool_definitions(),
)
async def archive(self, messages: list[dict]) -> bool:
async def archive(self, messages: list[dict]) -> str | None:
"""Summarize messages via LLM and append to history.jsonl.
Returns True on success (or degraded success), False if nothing to do.
Returns the summary text on success, None if nothing to archive.
"""
if not messages:
return False
return None
try:
formatted = MemoryStore._format_messages(messages)
response = await self.provider.chat_with_retry(
@@ -444,13 +459,15 @@ class Consolidator:
tools=None,
tool_choice=None,
)
if response.finish_reason == "error":
raise RuntimeError(f"LLM returned error: {response.content}")
summary = response.content or "[no summary]"
self.store.append_history(summary)
return True
return summary
except Exception:
logger.warning("Consolidation LLM call failed, raw-dumping to history")
self.store.raw_archive(messages)
return True
return None
async def maybe_consolidate_by_tokens(self, session: Session) -> None:
"""Loop: archive old messages until prompt fits within safe budget.
@@ -464,17 +481,23 @@ class Consolidator:
lock = self.get_lock(session.key)
async with lock:
budget = self.context_window_tokens - self.max_completion_tokens - self._SAFETY_BUFFER
target = budget // 2
estimated, source = self.estimate_session_prompt_tokens(session)
target = int(budget * self.consolidation_ratio)
try:
estimated, source = self.estimate_session_prompt_tokens(session)
except Exception:
logger.exception("Token estimation failed for {}", session.key)
estimated, source = 0, "error"
if estimated <= 0:
return
if estimated < budget:
unconsolidated_count = len(session.messages) - session.last_consolidated
logger.debug(
"Token consolidation idle {}: {}/{} via {}",
"Token consolidation idle {}: {}/{} via {}, msgs={}",
session.key,
estimated,
self.context_window_tokens,
source,
unconsolidated_count,
)
return
@@ -492,6 +515,15 @@ class Consolidator:
return
end_idx = boundary[0]
end_idx = self._cap_consolidation_boundary(session, end_idx)
if end_idx is None:
logger.debug(
"Token consolidation: no capped boundary for {} (round {})",
session.key,
round_num,
)
return
chunk = session.messages[session.last_consolidated:end_idx]
if not chunk:
return
@@ -510,7 +542,11 @@ class Consolidator:
session.last_consolidated = end_idx
self.sessions.save(session)
estimated, source = self.estimate_session_prompt_tokens(session)
try:
estimated, source = self.estimate_session_prompt_tokens(session)
except Exception:
logger.exception("Token estimation failed for {}", session.key)
estimated, source = 0, "error"
if estimated <= 0:
return
@@ -520,6 +556,13 @@ class Consolidator:
# ---------------------------------------------------------------------------
# Single source of truth for the staleness threshold used in _annotate_with_ages
# *and* in the Phase 1 prompt template (passed as `stale_threshold_days`).
# Keep code and prompt aligned — if you bump this, the LLM's instruction string
# updates automatically.
_STALE_THRESHOLD_DAYS = 14
class Dream:
"""Two-phase memory processor: analyze history.jsonl, then edit files via AgentRunner.
@@ -536,6 +579,7 @@ class Dream:
max_batch_size: int = 20,
max_iterations: int = 10,
max_tool_result_chars: int = 16_000,
annotate_line_ages: bool = True,
):
self.store = store
self.provider = provider
@@ -543,6 +587,10 @@ class Dream:
self.max_batch_size = max_batch_size
self.max_iterations = max_iterations
self.max_tool_result_chars = max_tool_result_chars
# Kill switch for the git-blame-based per-line age annotation in Phase 1.
# Default True keeps the #3212 behavior; set False to feed MEMORY.md raw
# (e.g. if a specific LLM reacts poorly to the `← Nd` suffix).
self.annotate_line_ages = annotate_line_ages
self._runner = AgentRunner(provider)
self._tools = self._build_tools()
@@ -550,18 +598,106 @@ class Dream:
def _build_tools(self) -> ToolRegistry:
"""Build a minimal tool registry for the Dream agent."""
from nanobot.agent.tools.filesystem import EditFileTool, ReadFileTool
from nanobot.agent.skills import BUILTIN_SKILLS_DIR
from nanobot.agent.tools.filesystem import EditFileTool, ReadFileTool, WriteFileTool
tools = ToolRegistry()
workspace = self.store.workspace
tools.register(ReadFileTool(workspace=workspace, allowed_dir=workspace))
# Allow reading builtin skills for reference during skill creation
extra_read = [BUILTIN_SKILLS_DIR] if BUILTIN_SKILLS_DIR.exists() else None
tools.register(ReadFileTool(
workspace=workspace,
allowed_dir=workspace,
extra_allowed_dirs=extra_read,
))
tools.register(EditFileTool(workspace=workspace, allowed_dir=workspace))
# write_file resolves relative paths from workspace root, but can only
# write under skills/ so the prompt can safely use skills/<name>/SKILL.md.
skills_dir = workspace / "skills"
skills_dir.mkdir(parents=True, exist_ok=True)
tools.register(WriteFileTool(workspace=workspace, allowed_dir=skills_dir))
return tools
# -- skill listing --------------------------------------------------------
def _list_existing_skills(self) -> list[str]:
"""List existing skills as 'name — description' for dedup context."""
import re as _re
from nanobot.agent.skills import BUILTIN_SKILLS_DIR
_DESC_RE = _re.compile(r"^description:\s*(.+)$", _re.MULTILINE | _re.IGNORECASE)
entries: dict[str, str] = {}
for base in (self.store.workspace / "skills", BUILTIN_SKILLS_DIR):
if not base.exists():
continue
for d in base.iterdir():
if not d.is_dir():
continue
skill_md = d / "SKILL.md"
if not skill_md.exists():
continue
# Prefer workspace skills over builtin (same name)
if d.name in entries and base == BUILTIN_SKILLS_DIR:
continue
content = skill_md.read_text(encoding="utf-8")[:500]
m = _DESC_RE.search(content)
desc = m.group(1).strip() if m else "(no description)"
entries[d.name] = desc
return [f"{name}{desc}" for name, desc in sorted(entries.items())]
# -- main entry ----------------------------------------------------------
def _annotate_with_ages(self, content: str) -> str:
"""Append per-line age suffixes to MEMORY.md content.
Each non-blank line whose age exceeds ``_STALE_THRESHOLD_DAYS`` gets a
suffix like ``← 30d`` indicating days since last modification.
Returns the original content unchanged if git is unavailable,
annotate fails, or the line count doesn't match the age count
(which can happen with an uncommitted working-tree edit — better to
skip annotation than to tag the wrong line).
SOUL.md and USER.md are never annotated.
"""
file_path = "memory/MEMORY.md"
try:
ages = self.store.git.line_ages(file_path)
except Exception:
logger.debug("line_ages failed for {}", file_path)
return content
if not ages:
return content
had_trailing = content.endswith("\n")
lines = content.splitlines()
# If HEAD-blob line count disagrees with the working-tree content we
# received, ages would be assigned to the wrong lines — skip entirely
# and feed the LLM un-annotated content rather than misleading data.
if len(lines) != len(ages):
logger.debug(
"line_ages length mismatch for {} (lines={}, ages={}); skipping annotation",
file_path, len(lines), len(ages),
)
return content
annotated: list[str] = []
for line, age in zip(lines, ages):
if not line.strip():
annotated.append(line)
continue
if age.age_days > _STALE_THRESHOLD_DAYS:
annotated.append(f"{line} \u2190 {age.age_days}d")
else:
annotated.append(line)
result = "\n".join(annotated)
if had_trailing:
result += "\n"
return result
async def run(self) -> bool:
"""Process unprocessed history entries. Returns True if work was done."""
from nanobot.agent.skills import BUILTIN_SKILLS_DIR
last_cursor = self.store.get_last_dream_cursor()
entries = self.store.read_unprocessed_history(since_cursor=last_cursor)
if not entries:
@@ -578,11 +714,17 @@ class Dream:
f"[{e['timestamp']}] {e['content']}" for e in batch
)
# Current file contents
# Current file contents + per-line age annotations (MEMORY.md only)
current_date = datetime.now().strftime("%Y-%m-%d")
current_memory = self.store.read_memory() or "(empty)"
raw_memory = self.store.read_memory() or "(empty)"
current_memory = (
self._annotate_with_ages(raw_memory)
if self.annotate_line_ages
else raw_memory
)
current_soul = self.store.read_soul() or "(empty)"
current_user = self.store.read_user() or "(empty)"
file_context = (
f"## Current Date\n{current_date}\n\n"
f"## Current MEMORY.md ({len(current_memory)} chars)\n{current_memory}\n\n"
@@ -590,7 +732,7 @@ class Dream:
f"## Current USER.md ({len(current_user)} chars)\n{current_user}"
)
# Phase 1: Analyze
# Phase 1: Analyze (no skills list — dedup is Phase 2's job)
phase1_prompt = (
f"## Conversation History\n{history_text}\n\n{file_context}"
)
@@ -601,7 +743,11 @@ class Dream:
messages=[
{
"role": "system",
"content": render_template("agent/dream_phase1.md", strip=True),
"content": render_template(
"agent/dream_phase1.md",
strip=True,
stale_threshold_days=_STALE_THRESHOLD_DAYS,
),
},
{"role": "user", "content": phase1_prompt},
],
@@ -615,13 +761,25 @@ class Dream:
return False
# Phase 2: Delegate to AgentRunner with read_file / edit_file
phase2_prompt = f"## Analysis Result\n{analysis}\n\n{file_context}"
existing_skills = self._list_existing_skills()
skills_section = ""
if existing_skills:
skills_section = (
"\n\n## Existing Skills\n"
+ "\n".join(f"- {s}" for s in existing_skills)
)
phase2_prompt = f"## Analysis Result\n{analysis}\n\n{file_context}{skills_section}"
tools = self._tools
skill_creator_path = BUILTIN_SKILLS_DIR / "skill-creator" / "SKILL.md"
messages: list[dict[str, Any]] = [
{
"role": "system",
"content": render_template("agent/dream_phase2.md", strip=True),
"content": render_template(
"agent/dream_phase2.md",
strip=True,
skill_creator_path=str(skill_creator_path),
),
},
{"role": "user", "content": phase2_prompt},
]
@@ -672,7 +830,9 @@ class Dream:
# Git auto-commit (only when there are actual changes)
if changelog and self.store.git.is_initialized():
ts = batch[-1]["timestamp"]
sha = self.store.git.auto_commit(f"dream: {ts}, {len(changelog)} change(s)")
summary = f"dream: {ts}, {len(changelog)} change(s)"
commit_msg = f"{summary}\n\n{analysis.strip()}"
sha = self.store.git.auto_commit(commit_msg)
if sha:
logger.info("Dream commit: {}", sha)
+328 -55
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import asyncio
from dataclasses import dataclass, field
import inspect
from pathlib import Path
from typing import Any
@@ -24,16 +25,27 @@ from nanobot.utils.helpers import (
from nanobot.utils.runtime import (
EMPTY_FINAL_RESPONSE_MESSAGE,
build_finalization_retry_message,
build_length_recovery_message,
ensure_nonempty_tool_result,
is_blank_text,
repeated_external_lookup_error,
)
_DEFAULT_ERROR_MESSAGE = "Sorry, I encountered an error calling the AI model."
_PERSISTED_MODEL_ERROR_PLACEHOLDER = "[Assistant reply unavailable due to model error.]"
_MAX_EMPTY_RETRIES = 2
_MAX_LENGTH_RECOVERIES = 3
_MAX_INJECTIONS_PER_TURN = 3
_MAX_INJECTION_CYCLES = 5
_SNIP_SAFETY_BUFFER = 1024
_MICROCOMPACT_KEEP_RECENT = 10
_MICROCOMPACT_MIN_CHARS = 500
_COMPACTABLE_TOOLS = frozenset({
"read_file", "exec", "grep", "glob",
"web_search", "web_fetch", "list_dir",
})
_BACKFILL_CONTENT = "[Tool result unavailable — call was interrupted or lost]"
@dataclass(slots=True)
@@ -83,37 +95,133 @@ class AgentRunner:
def __init__(self, provider: LLMProvider):
self.provider = provider
async def _drain_injections(self, spec: AgentRunSpec) -> list[str]:
@staticmethod
def _merge_message_content(left: Any, right: Any) -> str | list[dict[str, Any]]:
if isinstance(left, str) and isinstance(right, str):
return f"{left}\n\n{right}" if left else right
def _to_blocks(value: Any) -> list[dict[str, Any]]:
if isinstance(value, list):
return [
item if isinstance(item, dict) else {"type": "text", "text": str(item)}
for item in value
]
if value is None:
return []
return [{"type": "text", "text": str(value)}]
return _to_blocks(left) + _to_blocks(right)
@classmethod
def _append_injected_messages(
cls,
messages: list[dict[str, Any]],
injections: list[dict[str, Any]],
) -> None:
"""Append injected user messages while preserving role alternation."""
for injection in injections:
if (
messages
and injection.get("role") == "user"
and messages[-1].get("role") == "user"
):
merged = dict(messages[-1])
merged["content"] = cls._merge_message_content(
merged.get("content"),
injection.get("content"),
)
messages[-1] = merged
continue
messages.append(injection)
async def _try_drain_injections(
self,
spec: AgentRunSpec,
messages: list[dict[str, Any]],
assistant_message: dict[str, Any] | None,
injection_cycles: int,
*,
phase: str = "after error",
iteration: int | None = None,
) -> tuple[bool, int]:
"""Drain pending injections. Returns (should_continue, updated_cycles).
If injections are found and we haven't exceeded _MAX_INJECTION_CYCLES,
append them to *messages* (and emit a checkpoint if *assistant_message*
and *iteration* are both provided) and return (True, cycles+1) so the
caller continues the iteration loop. Otherwise return (False, cycles).
"""
if injection_cycles >= _MAX_INJECTION_CYCLES:
return False, injection_cycles
injections = await self._drain_injections(spec)
if not injections:
return False, injection_cycles
injection_cycles += 1
if assistant_message is not None:
messages.append(assistant_message)
if iteration is not None:
await self._emit_checkpoint(
spec,
{
"phase": "final_response",
"iteration": iteration,
"model": spec.model,
"assistant_message": assistant_message,
"completed_tool_results": [],
"pending_tool_calls": [],
},
)
self._append_injected_messages(messages, injections)
logger.info(
"Injected {} follow-up message(s) {} ({}/{})",
len(injections), phase, injection_cycles, _MAX_INJECTION_CYCLES,
)
return True, injection_cycles
async def _drain_injections(self, spec: AgentRunSpec) -> list[dict[str, Any]]:
"""Drain pending user messages via the injection callback.
Returns all drained message contents (capped by
Returns normalized user messages (capped by
``_MAX_INJECTIONS_PER_TURN``), or an empty list when there is
nothing to inject. Messages beyond the cap are logged so they
nothing to inject. Messages beyond the cap are logged so they
are not silently lost.
"""
if spec.injection_callback is None:
return []
try:
items = await spec.injection_callback()
signature = inspect.signature(spec.injection_callback)
accepts_limit = (
"limit" in signature.parameters
or any(
parameter.kind is inspect.Parameter.VAR_KEYWORD
for parameter in signature.parameters.values()
)
)
if accepts_limit:
items = await spec.injection_callback(limit=_MAX_INJECTIONS_PER_TURN)
else:
items = await spec.injection_callback()
except Exception:
logger.exception("injection_callback failed")
return []
if not items:
return []
# items are InboundMessage objects from _drain_pending
texts: list[str] = []
injected_messages: list[dict[str, Any]] = []
for item in items:
if isinstance(item, dict) and item.get("role") == "user" and "content" in item:
injected_messages.append(item)
continue
text = getattr(item, "content", str(item))
if text.strip():
texts.append(text)
if len(texts) > _MAX_INJECTIONS_PER_TURN:
dropped = len(texts) - _MAX_INJECTIONS_PER_TURN
injected_messages.append({"role": "user", "content": text})
if len(injected_messages) > _MAX_INJECTIONS_PER_TURN:
dropped = len(injected_messages) - _MAX_INJECTIONS_PER_TURN
logger.warning(
"Injection batch has {} messages, capping to {} ({} dropped)",
len(texts), _MAX_INJECTIONS_PER_TURN, dropped,
"Injection callback returned {} messages, capping to {} ({} dropped)",
len(injected_messages), _MAX_INJECTIONS_PER_TURN, dropped,
)
texts = texts[-_MAX_INJECTIONS_PER_TURN:]
return texts
injected_messages = injected_messages[:_MAX_INJECTIONS_PER_TURN]
return injected_messages
async def run(self, spec: AgentRunSpec) -> AgentRunResult:
hook = spec.hook or AgentHook()
@@ -126,21 +234,36 @@ class AgentRunner:
tool_events: list[dict[str, str]] = []
external_lookup_counts: dict[str, int] = {}
empty_content_retries = 0
length_recovery_count = 0
had_injections = False
injection_cycles = 0
for iteration in range(spec.max_iterations):
try:
messages = self._apply_tool_result_budget(spec, messages)
messages_for_model = self._snip_history(spec, messages)
# Keep the persisted conversation untouched. Context governance
# may repair or compact historical messages for the model, but
# those synthetic edits must not shift the append boundary used
# later when the caller saves only the new turn.
messages_for_model = self._drop_orphan_tool_results(messages)
messages_for_model = self._backfill_missing_tool_results(messages_for_model)
messages_for_model = self._microcompact(messages_for_model)
messages_for_model = self._apply_tool_result_budget(spec, messages_for_model)
messages_for_model = self._snip_history(spec, messages_for_model)
# Snipping may have created new orphans; clean them up.
messages_for_model = self._drop_orphan_tool_results(messages_for_model)
messages_for_model = self._backfill_missing_tool_results(messages_for_model)
except Exception as exc:
logger.warning(
"Context governance failed on turn {} for {}: {}; using raw messages",
"Context governance failed on turn {} for {}: {}; applying minimal repair",
iteration,
spec.session_key or "default",
exc,
)
messages_for_model = messages
try:
messages_for_model = self._drop_orphan_tool_results(messages)
messages_for_model = self._backfill_missing_tool_results(messages_for_model)
except Exception:
messages_for_model = messages
context = AgentHookContext(iteration=iteration, messages=messages)
await hook.before_iteration(context)
response = await self._request_model(spec, messages_for_model, hook, context)
@@ -184,16 +307,6 @@ class AgentRunner:
tool_events.extend(new_events)
context.tool_results = list(results)
context.tool_events = list(new_events)
if fatal_error is not None:
error = f"Error: {type(fatal_error).__name__}: {fatal_error}"
final_content = error
stop_reason = "tool_error"
self._append_final_message(messages, final_content)
context.final_content = final_content
context.error = error
context.stop_reason = stop_reason
await hook.after_iteration(context)
break
completed_tool_results: list[dict[str, Any]] = []
for tool_call, result in zip(response.tool_calls, results):
tool_message = {
@@ -209,6 +322,23 @@ class AgentRunner:
}
messages.append(tool_message)
completed_tool_results.append(tool_message)
if fatal_error is not None:
error = f"Error: {type(fatal_error).__name__}: {fatal_error}"
final_content = error
stop_reason = "tool_error"
self._append_final_message(messages, final_content)
context.final_content = final_content
context.error = error
context.stop_reason = stop_reason
await hook.after_iteration(context)
should_continue, injection_cycles = await self._try_drain_injections(
spec, messages, None, injection_cycles,
phase="after tool error",
)
if should_continue:
had_injections = True
continue
break
await self._emit_checkpoint(
spec,
{
@@ -221,18 +351,14 @@ class AgentRunner:
},
)
empty_content_retries = 0
length_recovery_count = 0
# Checkpoint 1: drain injections after tools, before next LLM call
if injection_cycles < _MAX_INJECTION_CYCLES:
injections = await self._drain_injections(spec)
if injections:
had_injections = True
injection_cycles += 1
for text in injections:
messages.append({"role": "user", "content": text})
logger.info(
"Injected {} follow-up message(s) after tool execution ({}/{})",
len(injections), injection_cycles, _MAX_INJECTION_CYCLES,
)
_drained, injection_cycles = await self._try_drain_injections(
spec, messages, None, injection_cycles,
phase="after tool execution",
)
if _drained:
had_injections = True
await hook.after_iteration(context)
continue
@@ -268,27 +394,50 @@ class AgentRunner:
context.tool_calls = list(response.tool_calls)
clean = hook.finalize_content(context, response.content)
if response.finish_reason == "length" and not is_blank_text(clean):
length_recovery_count += 1
if length_recovery_count <= _MAX_LENGTH_RECOVERIES:
logger.info(
"Output truncated on turn {} for {} ({}/{}); continuing",
iteration,
spec.session_key or "default",
length_recovery_count,
_MAX_LENGTH_RECOVERIES,
)
if hook.wants_streaming():
await hook.on_stream_end(context, resuming=True)
messages.append(build_assistant_message(
clean,
reasoning_content=response.reasoning_content,
thinking_blocks=response.thinking_blocks,
))
messages.append(build_length_recovery_message())
await hook.after_iteration(context)
continue
assistant_message: dict[str, Any] | None = None
if response.finish_reason != "error" and not is_blank_text(clean):
assistant_message = build_assistant_message(
clean,
reasoning_content=response.reasoning_content,
thinking_blocks=response.thinking_blocks,
)
# Check for mid-turn injections BEFORE signaling stream end.
# If injections are found we keep the stream alive (resuming=True)
# so streaming channels don't prematurely finalize the card.
_injected_after_final = False
if injection_cycles < _MAX_INJECTION_CYCLES:
injections = await self._drain_injections(spec)
if injections:
had_injections = True
injection_cycles += 1
_injected_after_final = True
for text in injections:
messages.append({"role": "user", "content": text})
logger.info(
"Injected {} follow-up message(s) after final response ({}/{})",
len(injections), injection_cycles, _MAX_INJECTION_CYCLES,
)
should_continue, injection_cycles = await self._try_drain_injections(
spec, messages, assistant_message, injection_cycles,
phase="after final response",
iteration=iteration,
)
if should_continue:
had_injections = True
if hook.wants_streaming():
await hook.on_stream_end(context, resuming=_injected_after_final)
await hook.on_stream_end(context, resuming=should_continue)
if _injected_after_final:
if should_continue:
await hook.after_iteration(context)
continue
@@ -296,11 +445,18 @@ class AgentRunner:
final_content = clean or spec.error_message or _DEFAULT_ERROR_MESSAGE
stop_reason = "error"
error = final_content
self._append_final_message(messages, final_content)
self._append_model_error_placeholder(messages)
context.final_content = final_content
context.error = error
context.stop_reason = stop_reason
await hook.after_iteration(context)
should_continue, injection_cycles = await self._try_drain_injections(
spec, messages, None, injection_cycles,
phase="after LLM error",
)
if should_continue:
had_injections = True
continue
break
if is_blank_text(clean):
final_content = EMPTY_FINAL_RESPONSE_MESSAGE
@@ -311,9 +467,16 @@ class AgentRunner:
context.error = error
context.stop_reason = stop_reason
await hook.after_iteration(context)
should_continue, injection_cycles = await self._try_drain_injections(
spec, messages, None, injection_cycles,
phase="after empty response",
)
if should_continue:
had_injections = True
continue
break
messages.append(build_assistant_message(
messages.append(assistant_message or build_assistant_message(
clean,
reasoning_content=response.reasoning_content,
thinking_blocks=response.thinking_blocks,
@@ -347,6 +510,17 @@ class AgentRunner:
max_iterations=spec.max_iterations,
)
self._append_final_message(messages, final_content)
# Drain any remaining injections so they are appended to the
# conversation history instead of being re-published as
# independent inbound messages by _dispatch's finally block.
# We ignore should_continue here because the for-loop has already
# exhausted all iterations.
drained_after_max_iterations, injection_cycles = await self._try_drain_injections(
spec, messages, None, injection_cycles,
phase="after max_iterations",
)
if drained_after_max_iterations:
had_injections = True
return AgentRunResult(
final_content=final_content,
@@ -561,6 +735,12 @@ class AgentRunner:
return
messages.append(build_assistant_message(content))
@staticmethod
def _append_model_error_placeholder(messages: list[dict[str, Any]]) -> None:
if messages and messages[-1].get("role") == "assistant" and not messages[-1].get("tool_calls"):
return
messages.append(build_assistant_message(_PERSISTED_MODEL_ERROR_PLACEHOLDER))
def _normalize_tool_result(
self,
spec: AgentRunSpec,
@@ -589,6 +769,99 @@ class AgentRunner:
return truncate_text(content, spec.max_tool_result_chars)
return content
@staticmethod
def _drop_orphan_tool_results(
messages: list[dict[str, Any]],
) -> list[dict[str, Any]]:
"""Drop tool results that have no matching assistant tool_call earlier in the history."""
declared: set[str] = set()
updated: list[dict[str, Any]] | None = None
for idx, msg in enumerate(messages):
role = msg.get("role")
if role == "assistant":
for tc in msg.get("tool_calls") or []:
if isinstance(tc, dict) and tc.get("id"):
declared.add(str(tc["id"]))
if role == "tool":
tid = msg.get("tool_call_id")
if tid and str(tid) not in declared:
if updated is None:
updated = [dict(m) for m in messages[:idx]]
continue
if updated is not None:
updated.append(dict(msg))
if updated is None:
return messages
return updated
@staticmethod
def _backfill_missing_tool_results(
messages: list[dict[str, Any]],
) -> list[dict[str, Any]]:
"""Insert synthetic error results for orphaned tool_use blocks."""
declared: list[tuple[int, str, str]] = [] # (assistant_idx, call_id, name)
fulfilled: set[str] = set()
for idx, msg in enumerate(messages):
role = msg.get("role")
if role == "assistant":
for tc in msg.get("tool_calls") or []:
if isinstance(tc, dict) and tc.get("id"):
name = ""
func = tc.get("function")
if isinstance(func, dict):
name = func.get("name", "")
declared.append((idx, str(tc["id"]), name))
elif role == "tool":
tid = msg.get("tool_call_id")
if tid:
fulfilled.add(str(tid))
missing = [(ai, cid, name) for ai, cid, name in declared if cid not in fulfilled]
if not missing:
return messages
updated = list(messages)
offset = 0
for assistant_idx, call_id, name in missing:
insert_at = assistant_idx + 1 + offset
while insert_at < len(updated) and updated[insert_at].get("role") == "tool":
insert_at += 1
updated.insert(insert_at, {
"role": "tool",
"tool_call_id": call_id,
"name": name,
"content": _BACKFILL_CONTENT,
})
offset += 1
return updated
@staticmethod
def _microcompact(messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""Replace old compactable tool results with one-line summaries."""
compactable_indices: list[int] = []
for idx, msg in enumerate(messages):
if msg.get("role") == "tool" and msg.get("name") in _COMPACTABLE_TOOLS:
compactable_indices.append(idx)
if len(compactable_indices) <= _MICROCOMPACT_KEEP_RECENT:
return messages
stale = compactable_indices[: len(compactable_indices) - _MICROCOMPACT_KEEP_RECENT]
updated: list[dict[str, Any]] | None = None
for idx in stale:
msg = messages[idx]
content = msg.get("content")
if not isinstance(content, str) or len(content) < _MICROCOMPACT_MIN_CHARS:
continue
name = msg.get("name", "tool")
summary = f"[{name} result omitted from context]"
if updated is None:
updated = [dict(m) for m in messages]
updated[idx]["content"] = summary
return updated if updated is not None else messages
def _apply_tool_result_budget(
self,
spec: AgentRunSpec,
+48 -35
View File
@@ -6,6 +6,8 @@ import re
import shutil
from pathlib import Path
import yaml
# Default builtin skills directory (relative to this file)
BUILTIN_SKILLS_DIR = Path(__file__).parent.parent / "skills"
@@ -16,10 +18,6 @@ _STRIP_SKILL_FRONTMATTER = re.compile(
)
def _escape_xml(text: str) -> str:
return text.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")
class SkillsLoader:
"""
Loader for agent skills.
@@ -28,10 +26,11 @@ class SkillsLoader:
specific tools or perform certain tasks.
"""
def __init__(self, workspace: Path, builtin_skills_dir: Path | None = None):
def __init__(self, workspace: Path, builtin_skills_dir: Path | None = None, disabled_skills: set[str] | None = None):
self.workspace = workspace
self.workspace_skills = workspace / "skills"
self.builtin_skills = builtin_skills_dir or BUILTIN_SKILLS_DIR
self.disabled_skills = disabled_skills or set()
def _skill_entries_from_dir(self, base: Path, source: str, *, skip_names: set[str] | None = None) -> list[dict[str, str]]:
if not base.exists():
@@ -66,6 +65,9 @@ class SkillsLoader:
self._skill_entries_from_dir(self.builtin_skills, "builtin", skip_names=workspace_names)
)
if self.disabled_skills:
skills = [s for s in skills if s["name"] not in self.disabled_skills]
if filter_unavailable:
return [skill for skill in skills if self._check_requirements(self._get_skill_meta(skill["name"]))]
return skills
@@ -106,39 +108,37 @@ class SkillsLoader:
]
return "\n\n---\n\n".join(parts)
def build_skills_summary(self) -> str:
def build_skills_summary(self, exclude: set[str] | None = None) -> str:
"""
Build a summary of all skills (name, description, path, availability).
This is used for progressive loading - the agent can read the full
skill content using read_file when needed.
Args:
exclude: Set of skill names to omit from the summary.
Returns:
XML-formatted skills summary.
Markdown-formatted skills summary.
"""
all_skills = self.list_skills(filter_unavailable=False)
if not all_skills:
return ""
lines: list[str] = ["<skills>"]
lines: list[str] = []
for entry in all_skills:
skill_name = entry["name"]
if exclude and skill_name in exclude:
continue
meta = self._get_skill_meta(skill_name)
available = self._check_requirements(meta)
lines.extend(
[
f' <skill available="{str(available).lower()}">',
f" <name>{_escape_xml(skill_name)}</name>",
f" <description>{_escape_xml(self._get_skill_description(skill_name))}</description>",
f" <location>{entry['path']}</location>",
]
)
if not available:
desc = self._get_skill_description(skill_name)
if available:
lines.append(f"- **{skill_name}** — {desc} `{entry['path']}`")
else:
missing = self._get_missing_requirements(meta)
if missing:
lines.append(f" <requires>{_escape_xml(missing)}</requires>")
lines.append(" </skill>")
lines.append("</skills>")
suffix = f" (unavailable: {missing})" if missing else " (unavailable)"
lines.append(f"- **{skill_name}** — {desc}{suffix} `{entry['path']}`")
return "\n".join(lines)
def _get_missing_requirements(self, skill_meta: dict) -> str:
@@ -167,11 +167,19 @@ class SkillsLoader:
return content[match.end():].strip()
return content
def _parse_nanobot_metadata(self, raw: str) -> dict:
"""Parse skill metadata JSON from frontmatter (supports nanobot and openclaw keys)."""
try:
data = json.loads(raw)
except (json.JSONDecodeError, TypeError):
def _parse_nanobot_metadata(self, raw: object) -> dict:
"""Extract nanobot/openclaw metadata from a frontmatter field.
``raw`` may be a dict (already parsed by yaml.safe_load) or a JSON str.
"""
if isinstance(raw, dict):
data = raw
elif isinstance(raw, str):
try:
data = json.loads(raw)
except (json.JSONDecodeError, TypeError):
return {}
else:
return {}
if not isinstance(data, dict):
return {}
@@ -189,8 +197,8 @@ class SkillsLoader:
def _get_skill_meta(self, name: str) -> dict:
"""Get nanobot metadata for a skill (cached in frontmatter)."""
meta = self.get_skill_metadata(name) or {}
return self._parse_nanobot_metadata(meta.get("metadata", ""))
raw_meta = self.get_skill_metadata(name) or {}
return self._parse_nanobot_metadata(raw_meta.get("metadata"))
def get_always_skills(self) -> list[str]:
"""Get skills marked as always=true that meet requirements."""
@@ -199,7 +207,7 @@ class SkillsLoader:
for entry in self.list_skills(filter_unavailable=True)
if (meta := self.get_skill_metadata(entry["name"]) or {})
and (
self._parse_nanobot_metadata(meta.get("metadata", "")).get("always")
self._parse_nanobot_metadata(meta.get("metadata")).get("always")
or meta.get("always")
)
]
@@ -220,10 +228,15 @@ class SkillsLoader:
match = _STRIP_SKILL_FRONTMATTER.match(content)
if not match:
return None
metadata: dict[str, str] = {}
for line in match.group(1).splitlines():
if ":" not in line:
continue
key, value = line.split(":", 1)
metadata[key.strip()] = value.strip().strip('"\'')
try:
parsed = yaml.safe_load(match.group(1))
except yaml.YAMLError:
return None
if not isinstance(parsed, dict):
return None
# yaml.safe_load returns native types (int, bool, list, etc.);
# keep values as-is so downstream consumers get correct types.
metadata: dict[str, object] = {}
for key, value in parsed.items():
metadata[str(key)] = value
return metadata
+86 -33
View File
@@ -2,7 +2,9 @@
import asyncio
import json
import time
import uuid
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
@@ -23,11 +25,29 @@ from nanobot.config.schema import ExecToolConfig, WebToolsConfig
from nanobot.providers.base import LLMProvider
class _SubagentHook(AgentHook):
"""Logging-only hook for subagent execution."""
@dataclass(slots=True)
class SubagentStatus:
"""Real-time status of a running subagent."""
def __init__(self, task_id: str) -> None:
task_id: str
label: str
task_description: str
started_at: float # time.monotonic()
phase: str = "initializing" # initializing | awaiting_tools | tools_completed | final_response | done | error
iteration: int = 0
tool_events: list = field(default_factory=list) # [{name, status, detail}, ...]
usage: dict = field(default_factory=dict) # token usage
stop_reason: str | None = None
error: str | None = None
class _SubagentHook(AgentHook):
"""Hook for subagent execution — logs tool calls and updates status."""
def __init__(self, task_id: str, status: SubagentStatus | None = None) -> None:
super().__init__()
self._task_id = task_id
self._status = status
async def before_execute_tools(self, context: AgentHookContext) -> None:
for tool_call in context.tool_calls:
@@ -37,6 +57,15 @@ class _SubagentHook(AgentHook):
self._task_id, tool_call.name, args_str,
)
async def after_iteration(self, context: AgentHookContext) -> None:
if self._status is None:
return
self._status.iteration = context.iteration
self._status.tool_events = list(context.tool_events)
self._status.usage = dict(context.usage)
if context.error:
self._status.error = str(context.error)
class SubagentManager:
"""Manages background subagent execution."""
@@ -51,9 +80,8 @@ class SubagentManager:
web_config: "WebToolsConfig | None" = None,
exec_config: "ExecToolConfig | None" = None,
restrict_to_workspace: bool = False,
disabled_skills: list[str] | None = None,
):
from nanobot.config.schema import ExecToolConfig
self.provider = provider
self.workspace = workspace
self.bus = bus
@@ -62,8 +90,10 @@ class SubagentManager:
self.max_tool_result_chars = max_tool_result_chars
self.exec_config = exec_config or ExecToolConfig()
self.restrict_to_workspace = restrict_to_workspace
self.disabled_skills = set(disabled_skills or [])
self.runner = AgentRunner(provider)
self._running_tasks: dict[str, asyncio.Task[None]] = {}
self._task_statuses: dict[str, SubagentStatus] = {}
self._session_tasks: dict[str, set[str]] = {} # session_key -> {task_id, ...}
async def spawn(
@@ -79,8 +109,16 @@ class SubagentManager:
display_label = label or task[:30] + ("..." if len(task) > 30 else "")
origin = {"channel": origin_channel, "chat_id": origin_chat_id}
status = SubagentStatus(
task_id=task_id,
label=display_label,
task_description=task,
started_at=time.monotonic(),
)
self._task_statuses[task_id] = status
bg_task = asyncio.create_task(
self._run_subagent(task_id, task, display_label, origin)
self._run_subagent(task_id, task, display_label, origin, status)
)
self._running_tasks[task_id] = bg_task
if session_key:
@@ -88,6 +126,7 @@ class SubagentManager:
def _cleanup(_: asyncio.Task) -> None:
self._running_tasks.pop(task_id, None)
self._task_statuses.pop(task_id, None)
if session_key and (ids := self._session_tasks.get(session_key)):
ids.discard(task_id)
if not ids:
@@ -104,10 +143,15 @@ class SubagentManager:
task: str,
label: str,
origin: dict[str, str],
status: SubagentStatus,
) -> None:
"""Execute the subagent task and announce the result."""
logger.info("Subagent [{}] starting task: {}", task_id, label)
async def _on_checkpoint(payload: dict) -> None:
status.phase = payload.get("phase", status.phase)
status.iteration = payload.get("iteration", status.iteration)
try:
# Build subagent tools (no message tool, no spawn tool)
tools = ToolRegistry()
@@ -142,40 +186,38 @@ class SubagentManager:
model=self.model,
max_iterations=15,
max_tool_result_chars=self.max_tool_result_chars,
hook=_SubagentHook(task_id),
hook=_SubagentHook(task_id, status),
max_iterations_message="Task completed but no final response was generated.",
error_message=None,
fail_on_tool_error=True,
checkpoint_callback=_on_checkpoint,
))
if result.stop_reason == "tool_error":
await self._announce_result(
task_id,
label,
task,
self._format_partial_progress(result),
origin,
"error",
)
return
if result.stop_reason == "error":
await self._announce_result(
task_id,
label,
task,
result.error or "Error: subagent execution failed.",
origin,
"error",
)
return
final_result = result.final_content or "Task completed but no final response was generated."
status.phase = "done"
status.stop_reason = result.stop_reason
logger.info("Subagent [{}] completed successfully", task_id)
await self._announce_result(task_id, label, task, final_result, origin, "ok")
if result.stop_reason == "tool_error":
status.tool_events = list(result.tool_events)
await self._announce_result(
task_id, label, task,
self._format_partial_progress(result),
origin, "error",
)
elif result.stop_reason == "error":
await self._announce_result(
task_id, label, task,
result.error or "Error: subagent execution failed.",
origin, "error",
)
else:
final_result = result.final_content or "Task completed but no final response was generated."
logger.info("Subagent [{}] completed successfully", task_id)
await self._announce_result(task_id, label, task, final_result, origin, "ok")
except Exception as e:
error_msg = f"Error: {str(e)}"
status.phase = "error"
status.error = str(e)
logger.error("Subagent [{}] failed: {}", task_id, e)
await self._announce_result(task_id, label, task, error_msg, origin, "error")
await self._announce_result(task_id, label, task, f"Error: {e}", origin, "error")
async def _announce_result(
self,
@@ -235,7 +277,10 @@ class SubagentManager:
from nanobot.agent.skills import SkillsLoader
time_ctx = ContextBuilder._build_runtime_context(None, None)
skills_summary = SkillsLoader(self.workspace).build_skills_summary()
skills_summary = SkillsLoader(
self.workspace,
disabled_skills=self.disabled_skills,
).build_skills_summary()
return render_template(
"agent/subagent_system.md",
time_ctx=time_ctx,
@@ -256,3 +301,11 @@ class SubagentManager:
def get_running_count(self) -> int:
"""Return the number of currently running subagents."""
return len(self._running_tasks)
def get_running_count_by_session(self, session_key: str) -> int:
"""Return the number of currently running subagents for a session."""
tids = self._session_tasks.get(session_key, set())
return sum(
1 for tid in tids
if tid in self._running_tasks and not self._running_tasks[tid].done()
)
+105
View File
@@ -0,0 +1,105 @@
"""Track file-read state for read-before-edit warnings and read deduplication."""
from __future__ import annotations
import hashlib
import os
from dataclasses import dataclass
from pathlib import Path
@dataclass(slots=True)
class ReadState:
mtime: float
offset: int
limit: int | None
content_hash: str | None
can_dedup: bool
_state: dict[str, ReadState] = {}
def _hash_file(p: str) -> str | None:
try:
return hashlib.sha256(Path(p).read_bytes()).hexdigest()
except OSError:
return None
def record_read(path: str | Path, offset: int = 1, limit: int | None = None) -> None:
"""Record that a file was read (called after successful read)."""
p = str(Path(path).resolve())
try:
mtime = os.path.getmtime(p)
except OSError:
return
_state[p] = ReadState(
mtime=mtime,
offset=offset,
limit=limit,
content_hash=_hash_file(p),
can_dedup=True,
)
def record_write(path: str | Path) -> None:
"""Record that a file was written (updates mtime in state)."""
p = str(Path(path).resolve())
try:
mtime = os.path.getmtime(p)
except OSError:
_state.pop(p, None)
return
_state[p] = ReadState(
mtime=mtime,
offset=1,
limit=None,
content_hash=_hash_file(p),
can_dedup=False,
)
def check_read(path: str | Path) -> str | None:
"""Check if a file has been read and is fresh.
Returns None if OK, or a warning string.
When mtime changed but file content is identical (e.g. touch, editor save),
the check passes to avoid false-positive staleness warnings.
"""
p = str(Path(path).resolve())
entry = _state.get(p)
if entry is None:
return "Warning: file has not been read yet. Read it first to verify content before editing."
try:
current_mtime = os.path.getmtime(p)
except OSError:
return None
if current_mtime != entry.mtime:
if entry.content_hash and _hash_file(p) == entry.content_hash:
entry.mtime = current_mtime
return None
return "Warning: file has been modified since last read. Re-read to verify content before editing."
return None
def is_unchanged(path: str | Path, offset: int = 1, limit: int | None = None) -> bool:
"""Return True if file was previously read with same params and mtime is unchanged."""
p = str(Path(path).resolve())
entry = _state.get(p)
if entry is None:
return False
if not entry.can_dedup:
return False
if entry.offset != offset or entry.limit != limit:
return False
try:
current_mtime = os.path.getmtime(p)
except OSError:
return False
return current_mtime == entry.mtime
def clear() -> None:
"""Clear all tracked state (useful for testing)."""
_state.clear()
+472 -42
View File
@@ -2,11 +2,13 @@
import difflib
import mimetypes
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from nanobot.agent.tools.base import Tool, tool_parameters
from nanobot.agent.tools.schema import BooleanSchema, IntegerSchema, StringSchema, tool_parameters_schema
from nanobot.agent.tools import file_state
from nanobot.utils.helpers import build_image_content_blocks, detect_image_mime
from nanobot.config.paths import get_media_dir
@@ -60,6 +62,36 @@ class _FsTool(Tool):
# ---------------------------------------------------------------------------
_BLOCKED_DEVICE_PATHS = frozenset({
"/dev/zero", "/dev/random", "/dev/urandom", "/dev/full",
"/dev/stdin", "/dev/stdout", "/dev/stderr",
"/dev/tty", "/dev/console",
"/dev/fd/0", "/dev/fd/1", "/dev/fd/2",
})
def _is_blocked_device(path: str | Path) -> bool:
"""Check if path is a blocked device that could hang or produce infinite output."""
import re
raw = str(path)
if raw in _BLOCKED_DEVICE_PATHS:
return True
if re.match(r"/proc/\d+/fd/[012]$", raw) or re.match(r"/proc/self/fd/[012]$", raw):
return True
return False
def _parse_page_range(pages: str, total: int) -> tuple[int, int]:
"""Parse a page range like '2-5' into 0-based (start, end) inclusive."""
parts = pages.strip().split("-")
if len(parts) == 1:
p = int(parts[0])
return max(0, p - 1), min(p - 1, total - 1)
start = int(parts[0])
end = int(parts[1])
return max(0, start - 1), min(end - 1, total - 1)
@tool_parameters(
tool_parameters_schema(
path=StringSchema("The file path to read"),
@@ -73,6 +105,7 @@ class _FsTool(Tool):
description="Maximum number of lines to read (default 2000)",
minimum=1,
),
pages=StringSchema("Page range for PDF files, e.g. '1-5' (default: all, max 20 pages)"),
required=["path"],
)
)
@@ -81,6 +114,7 @@ class ReadFileTool(_FsTool):
_MAX_CHARS = 128_000
_DEFAULT_LIMIT = 2000
_MAX_PDF_PAGES = 20
@property
def name(self) -> str:
@@ -89,24 +123,38 @@ class ReadFileTool(_FsTool):
@property
def description(self) -> str:
return (
"Read the contents of a file. Returns numbered lines. "
"Use offset and limit to paginate through large files."
"Read a file (text or image). Text output format: LINE_NUM|CONTENT. "
"Images return visual content for analysis. "
"Use offset and limit for large files. "
"Cannot read non-image binary files. "
"Reads exceeding ~128K chars are truncated."
)
@property
def read_only(self) -> bool:
return True
async def execute(self, path: str | None = None, offset: int = 1, limit: int | None = None, **kwargs: Any) -> Any:
async def execute(self, path: str | None = None, offset: int = 1, limit: int | None = None, pages: str | None = None, **kwargs: Any) -> Any:
try:
if not path:
return "Error reading file: Unknown path"
# Device path blacklist
if _is_blocked_device(path):
return f"Error: Reading {path} is blocked (device path that could hang or produce infinite output)."
fp = self._resolve(path)
if _is_blocked_device(fp):
return f"Error: Reading {fp} is blocked (device path that could hang or produce infinite output)."
if not fp.exists():
return f"Error: File not found: {path}"
if not fp.is_file():
return f"Error: Not a file: {path}"
# PDF support
if fp.suffix.lower() == ".pdf":
return self._read_pdf(fp, pages)
raw = fp.read_bytes()
if not raw:
return f"(Empty file: {path})"
@@ -115,6 +163,10 @@ class ReadFileTool(_FsTool):
if mime and mime.startswith("image/"):
return build_image_content_blocks(raw, mime, str(fp), f"(Image file: {path})")
# Read dedup: same path + offset + limit + unchanged mtime → stub
if file_state.is_unchanged(fp, offset=offset, limit=limit):
return f"[File unchanged since last read: {path}]"
try:
text_content = raw.decode("utf-8")
except UnicodeDecodeError:
@@ -147,12 +199,59 @@ class ReadFileTool(_FsTool):
result += f"\n\n(Showing lines {offset}-{end} of {total}. Use offset={end + 1} to continue.)"
else:
result += f"\n\n(End of file — {total} lines total)"
file_state.record_read(fp, offset=offset, limit=limit)
return result
except PermissionError as e:
return f"Error: {e}"
except Exception as e:
return f"Error reading file: {e}"
def _read_pdf(self, fp: Path, pages: str | None) -> str:
try:
import fitz # pymupdf
except ImportError:
return "Error: PDF reading requires pymupdf. Install with: pip install pymupdf"
try:
doc = fitz.open(str(fp))
except Exception as e:
return f"Error reading PDF: {e}"
total_pages = len(doc)
if pages:
try:
start, end = _parse_page_range(pages, total_pages)
except (ValueError, IndexError):
doc.close()
return f"Error: Invalid page range '{pages}'. Use format like '1-5'."
if start > end or start >= total_pages:
doc.close()
return f"Error: Page range '{pages}' is out of bounds (document has {total_pages} pages)."
else:
start = 0
end = min(total_pages - 1, self._MAX_PDF_PAGES - 1)
if end - start + 1 > self._MAX_PDF_PAGES:
end = start + self._MAX_PDF_PAGES - 1
parts: list[str] = []
for i in range(start, end + 1):
page = doc[i]
text = page.get_text().strip()
if text:
parts.append(f"--- Page {i + 1} ---\n{text}")
doc.close()
if not parts:
return f"(PDF has no extractable text: {fp})"
result = "\n\n".join(parts)
if end < total_pages - 1:
result += f"\n\n(Showing pages {start + 1}-{end + 1} of {total_pages}. Use pages='{end + 2}-{min(end + 1 + self._MAX_PDF_PAGES, total_pages)}' to continue.)"
if len(result) > self._MAX_CHARS:
result = result[:self._MAX_CHARS] + "\n\n(PDF text truncated at ~128K chars)"
return result
# ---------------------------------------------------------------------------
# write_file
@@ -175,7 +274,11 @@ class WriteFileTool(_FsTool):
@property
def description(self) -> str:
return "Write content to a file at the given path. Creates parent directories if needed."
return (
"Write content to a file. Overwrites if the file already exists; "
"creates parent directories as needed. "
"For partial edits, prefer edit_file instead."
)
async def execute(self, path: str | None = None, content: str | None = None, **kwargs: Any) -> str:
try:
@@ -186,6 +289,7 @@ class WriteFileTool(_FsTool):
fp = self._resolve(path)
fp.parent.mkdir(parents=True, exist_ok=True)
fp.write_text(content, encoding="utf-8")
file_state.record_write(fp)
return f"Successfully wrote {len(content)} characters to {fp}"
except PermissionError as e:
return f"Error: {e}"
@@ -197,30 +301,269 @@ class WriteFileTool(_FsTool):
# edit_file
# ---------------------------------------------------------------------------
_QUOTE_TABLE = str.maketrans({
"\u2018": "'", "\u2019": "'", # curly single → straight
"\u201c": '"', "\u201d": '"', # curly double → straight
"'": "'", '"': '"', # identity (kept for completeness)
})
def _normalize_quotes(s: str) -> str:
return s.translate(_QUOTE_TABLE)
def _curly_double_quotes(text: str) -> str:
parts: list[str] = []
opening = True
for ch in text:
if ch == '"':
parts.append("\u201c" if opening else "\u201d")
opening = not opening
else:
parts.append(ch)
return "".join(parts)
def _curly_single_quotes(text: str) -> str:
parts: list[str] = []
opening = True
for i, ch in enumerate(text):
if ch != "'":
parts.append(ch)
continue
prev_ch = text[i - 1] if i > 0 else ""
next_ch = text[i + 1] if i + 1 < len(text) else ""
if prev_ch.isalnum() and next_ch.isalnum():
parts.append("\u2019")
continue
parts.append("\u2018" if opening else "\u2019")
opening = not opening
return "".join(parts)
def _preserve_quote_style(old_text: str, actual_text: str, new_text: str) -> str:
"""Preserve curly quote style when a quote-normalized fallback matched."""
if _normalize_quotes(old_text.strip()) != _normalize_quotes(actual_text.strip()) or old_text == actual_text:
return new_text
styled = new_text
if any(ch in actual_text for ch in ("\u201c", "\u201d")) and '"' in styled:
styled = _curly_double_quotes(styled)
if any(ch in actual_text for ch in ("\u2018", "\u2019")) and "'" in styled:
styled = _curly_single_quotes(styled)
return styled
def _leading_ws(line: str) -> str:
return line[: len(line) - len(line.lstrip(" \t"))]
def _reindent_like_match(old_text: str, actual_text: str, new_text: str) -> str:
"""Preserve the outer indentation from the actual matched block."""
old_lines = old_text.split("\n")
actual_lines = actual_text.split("\n")
if len(old_lines) != len(actual_lines):
return new_text
comparable = [
(old_line, actual_line)
for old_line, actual_line in zip(old_lines, actual_lines)
if old_line.strip() and actual_line.strip()
]
if not comparable or any(
_normalize_quotes(old_line.strip()) != _normalize_quotes(actual_line.strip())
for old_line, actual_line in comparable
):
return new_text
old_ws = _leading_ws(comparable[0][0])
actual_ws = _leading_ws(comparable[0][1])
if actual_ws == old_ws:
return new_text
if old_ws:
if not actual_ws.startswith(old_ws):
return new_text
delta = actual_ws[len(old_ws):]
else:
delta = actual_ws
if not delta:
return new_text
return "\n".join((delta + line) if line else line for line in new_text.split("\n"))
@dataclass(slots=True)
class _MatchSpan:
start: int
end: int
text: str
line: int
def _find_exact_matches(content: str, old_text: str) -> list[_MatchSpan]:
matches: list[_MatchSpan] = []
start = 0
while True:
idx = content.find(old_text, start)
if idx == -1:
break
matches.append(
_MatchSpan(
start=idx,
end=idx + len(old_text),
text=content[idx : idx + len(old_text)],
line=content.count("\n", 0, idx) + 1,
)
)
start = idx + max(1, len(old_text))
return matches
def _find_trim_matches(content: str, old_text: str, *, normalize_quotes: bool = False) -> list[_MatchSpan]:
old_lines = old_text.splitlines()
if not old_lines:
return []
content_lines = content.splitlines()
content_lines_keepends = content.splitlines(keepends=True)
if len(content_lines) < len(old_lines):
return []
offsets: list[int] = []
pos = 0
for line in content_lines_keepends:
offsets.append(pos)
pos += len(line)
offsets.append(pos)
if normalize_quotes:
stripped_old = [_normalize_quotes(line.strip()) for line in old_lines]
else:
stripped_old = [line.strip() for line in old_lines]
matches: list[_MatchSpan] = []
window_size = len(stripped_old)
for i in range(len(content_lines) - window_size + 1):
window = content_lines[i : i + window_size]
if normalize_quotes:
comparable = [_normalize_quotes(line.strip()) for line in window]
else:
comparable = [line.strip() for line in window]
if comparable != stripped_old:
continue
start = offsets[i]
end = offsets[i + window_size]
if content_lines_keepends[i + window_size - 1].endswith("\n"):
end -= 1
matches.append(
_MatchSpan(
start=start,
end=end,
text=content[start:end],
line=i + 1,
)
)
return matches
def _find_quote_matches(content: str, old_text: str) -> list[_MatchSpan]:
norm_content = _normalize_quotes(content)
norm_old = _normalize_quotes(old_text)
matches: list[_MatchSpan] = []
start = 0
while True:
idx = norm_content.find(norm_old, start)
if idx == -1:
break
matches.append(
_MatchSpan(
start=idx,
end=idx + len(old_text),
text=content[idx : idx + len(old_text)],
line=content.count("\n", 0, idx) + 1,
)
)
start = idx + max(1, len(norm_old))
return matches
def _find_matches(content: str, old_text: str) -> list[_MatchSpan]:
"""Locate all matches using progressively looser strategies."""
for matcher in (
lambda: _find_exact_matches(content, old_text),
lambda: _find_trim_matches(content, old_text),
lambda: _find_trim_matches(content, old_text, normalize_quotes=True),
lambda: _find_quote_matches(content, old_text),
):
matches = matcher()
if matches:
return matches
return []
def _find_match_line_numbers(content: str, old_text: str) -> list[int]:
"""Return 1-based starting line numbers for the current matching strategies."""
return [match.line for match in _find_matches(content, old_text)]
def _collapse_internal_whitespace(text: str) -> str:
return "\n".join(" ".join(line.split()) for line in text.splitlines())
def _diagnose_near_match(old_text: str, actual_text: str) -> list[str]:
"""Return actionable hints describing why text was close but not exact."""
hints: list[str] = []
if old_text.lower() == actual_text.lower() and old_text != actual_text:
hints.append("letter case differs")
if _collapse_internal_whitespace(old_text) == _collapse_internal_whitespace(actual_text) and old_text != actual_text:
hints.append("whitespace differs")
if old_text.rstrip("\n") == actual_text.rstrip("\n") and old_text != actual_text:
hints.append("trailing newline differs")
if _normalize_quotes(old_text) == _normalize_quotes(actual_text) and old_text != actual_text:
hints.append("quote style differs")
return hints
def _best_window(old_text: str, content: str) -> tuple[float, int, list[str], list[str]]:
"""Find the closest line-window match and return ratio/start/snippet/hints."""
lines = content.splitlines(keepends=True)
old_lines = old_text.splitlines(keepends=True)
window = max(1, len(old_lines))
best_ratio, best_start = -1.0, 0
best_window_lines: list[str] = []
for i in range(max(1, len(lines) - window + 1)):
current = lines[i : i + window]
ratio = difflib.SequenceMatcher(None, old_lines, current).ratio()
if ratio > best_ratio:
best_ratio, best_start = ratio, i
best_window_lines = current
actual_text = "".join(best_window_lines).replace("\r\n", "\n").rstrip("\n")
hints = _diagnose_near_match(old_text.replace("\r\n", "\n").rstrip("\n"), actual_text)
return best_ratio, best_start, best_window_lines, hints
def _find_match(content: str, old_text: str) -> tuple[str | None, int]:
"""Locate old_text in content: exact first, then line-trimmed sliding window.
"""Locate old_text in content with a multi-level fallback chain:
1. Exact substring match
2. Line-trimmed sliding window (handles indentation differences)
3. Smart quote normalization (curly ↔ straight quotes)
Both inputs should use LF line endings (caller normalises CRLF).
Returns (matched_fragment, count) or (None, 0).
"""
if old_text in content:
return old_text, content.count(old_text)
old_lines = old_text.splitlines()
if not old_lines:
matches = _find_matches(content, old_text)
if not matches:
return None, 0
stripped_old = [l.strip() for l in old_lines]
content_lines = content.splitlines()
candidates = []
for i in range(len(content_lines) - len(stripped_old) + 1):
window = content_lines[i : i + len(stripped_old)]
if [l.strip() for l in window] == stripped_old:
candidates.append("\n".join(window))
if candidates:
return candidates[0], len(candidates)
return None, 0
return matches[0].text, len(matches)
@tool_parameters(
@@ -235,6 +578,9 @@ def _find_match(content: str, old_text: str) -> tuple[str | None, int]:
class EditFileTool(_FsTool):
"""Edit a file by replacing text with fallback matching."""
_MAX_EDIT_FILE_SIZE = 1024 * 1024 * 1024 # 1 GiB
_MARKDOWN_EXTS = frozenset({".md", ".mdx", ".markdown"})
@property
def name(self) -> str:
return "edit_file"
@@ -243,10 +589,16 @@ class EditFileTool(_FsTool):
def description(self) -> str:
return (
"Edit a file by replacing old_text with new_text. "
"Supports minor whitespace/line-ending differences. "
"Set replace_all=true to replace every occurrence."
"Tolerates minor whitespace/indentation differences and curly/straight quote mismatches. "
"If old_text matches multiple times, you must provide more context "
"or set replace_all=true. Shows a diff of the closest match on failure."
)
@staticmethod
def _strip_trailing_ws(text: str) -> str:
"""Strip trailing whitespace from each line."""
return "\n".join(line.rstrip() for line in text.split("\n"))
async def execute(
self, path: str | None = None, old_text: str | None = None,
new_text: str | None = None,
@@ -260,55 +612,133 @@ class EditFileTool(_FsTool):
if new_text is None:
raise ValueError("Unknown new_text")
# .ipynb detection
if path.endswith(".ipynb"):
return "Error: This is a Jupyter notebook. Use the notebook_edit tool instead of edit_file."
fp = self._resolve(path)
# Create-file semantics: old_text='' + file doesn't exist → create
if not fp.exists():
return f"Error: File not found: {path}"
if old_text == "":
fp.parent.mkdir(parents=True, exist_ok=True)
fp.write_text(new_text, encoding="utf-8")
file_state.record_write(fp)
return f"Successfully created {fp}"
return self._file_not_found_msg(path, fp)
# File size protection
try:
fsize = fp.stat().st_size
except OSError:
fsize = 0
if fsize > self._MAX_EDIT_FILE_SIZE:
return f"Error: File too large to edit ({fsize / (1024**3):.1f} GiB). Maximum is 1 GiB."
# Create-file: old_text='' but file exists and not empty → reject
if old_text == "":
raw = fp.read_bytes()
content = raw.decode("utf-8")
if content.strip():
return f"Error: Cannot create file — {path} already exists and is not empty."
fp.write_text(new_text, encoding="utf-8")
file_state.record_write(fp)
return f"Successfully edited {fp}"
# Read-before-edit check
warning = file_state.check_read(fp)
raw = fp.read_bytes()
uses_crlf = b"\r\n" in raw
content = raw.decode("utf-8").replace("\r\n", "\n")
match, count = _find_match(content, old_text.replace("\r\n", "\n"))
norm_old = old_text.replace("\r\n", "\n")
matches = _find_matches(content, norm_old)
if match is None:
if not matches:
return self._not_found_msg(old_text, content, path)
count = len(matches)
if count > 1 and not replace_all:
line_numbers = [match.line for match in matches]
preview = ", ".join(f"line {n}" for n in line_numbers[:3])
if len(line_numbers) > 3:
preview += ", ..."
location_hint = f" at {preview}" if preview else ""
return (
f"Warning: old_text appears {count} times. "
f"Warning: old_text appears {count} times{location_hint}. "
"Provide more context to make it unique, or set replace_all=true."
)
norm_new = new_text.replace("\r\n", "\n")
new_content = content.replace(match, norm_new) if replace_all else content.replace(match, norm_new, 1)
# Trailing whitespace stripping (skip markdown to preserve double-space line breaks)
if fp.suffix.lower() not in self._MARKDOWN_EXTS:
norm_new = self._strip_trailing_ws(norm_new)
selected = matches if replace_all else matches[:1]
new_content = content
for match in reversed(selected):
replacement = _preserve_quote_style(norm_old, match.text, norm_new)
replacement = _reindent_like_match(norm_old, match.text, replacement)
# Delete-line cleanup: when deleting text (new_text=''), consume trailing
# newline to avoid leaving a blank line
end = match.end
if replacement == "" and not match.text.endswith("\n") and content[end:end + 1] == "\n":
end += 1
new_content = new_content[: match.start] + replacement + new_content[end:]
if uses_crlf:
new_content = new_content.replace("\n", "\r\n")
fp.write_bytes(new_content.encode("utf-8"))
return f"Successfully edited {fp}"
file_state.record_write(fp)
msg = f"Successfully edited {fp}"
if warning:
msg = f"{warning}\n{msg}"
return msg
except PermissionError as e:
return f"Error: {e}"
except Exception as e:
return f"Error editing file: {e}"
def _file_not_found_msg(self, path: str, fp: Path) -> str:
"""Build an error message with 'Did you mean ...?' suggestions."""
parent = fp.parent
suggestions: list[str] = []
if parent.is_dir():
siblings = [f.name for f in parent.iterdir() if f.is_file()]
close = difflib.get_close_matches(fp.name, siblings, n=3, cutoff=0.6)
suggestions = [str(parent / c) for c in close]
parts = [f"Error: File not found: {path}"]
if suggestions:
parts.append("Did you mean: " + ", ".join(suggestions) + "?")
return "\n".join(parts)
@staticmethod
def _not_found_msg(old_text: str, content: str, path: str) -> str:
lines = content.splitlines(keepends=True)
old_lines = old_text.splitlines(keepends=True)
window = len(old_lines)
best_ratio, best_start = 0.0, 0
for i in range(max(1, len(lines) - window + 1)):
ratio = difflib.SequenceMatcher(None, old_lines, lines[i : i + window]).ratio()
if ratio > best_ratio:
best_ratio, best_start = ratio, i
best_ratio, best_start, best_window_lines, hints = _best_window(old_text, content)
if best_ratio > 0.5:
diff = "\n".join(difflib.unified_diff(
old_lines, lines[best_start : best_start + window],
old_text.splitlines(keepends=True),
best_window_lines,
fromfile="old_text (provided)",
tofile=f"{path} (actual, line {best_start + 1})",
lineterm="",
))
return f"Error: old_text not found in {path}.\nBest match ({best_ratio:.0%} similar) at line {best_start + 1}:\n{diff}"
hint_text = ""
if hints:
hint_text = "\nPossible cause: " + ", ".join(hints) + "."
return (
f"Error: old_text not found in {path}."
f"{hint_text}\nBest match ({best_ratio:.0%} similar) at line {best_start + 1}:\n{diff}"
)
if hints:
return (
f"Error: old_text not found in {path}. "
f"Possible cause: {', '.join(hints)}. "
"Copy the exact text from read_file and try again."
)
return f"Error: old_text not found in {path}. No similar text found. Verify the file content."
+264 -19
View File
@@ -57,9 +57,7 @@ def _normalize_schema_for_openai(schema: Any) -> dict[str, Any]:
if "properties" in normalized and isinstance(normalized["properties"], dict):
normalized["properties"] = {
name: _normalize_schema_for_openai(prop)
if isinstance(prop, dict)
else prop
name: _normalize_schema_for_openai(prop) if isinstance(prop, dict) else prop
for name, prop in normalized["properties"].items()
}
@@ -135,36 +133,214 @@ class MCPToolWrapper(Tool):
return "\n".join(parts) or "(no output)"
class MCPResourceWrapper(Tool):
"""Wraps an MCP resource URI as a read-only nanobot Tool."""
def __init__(self, session, server_name: str, resource_def, resource_timeout: int = 30):
self._session = session
self._uri = resource_def.uri
self._name = f"mcp_{server_name}_resource_{resource_def.name}"
desc = resource_def.description or resource_def.name
self._description = f"[MCP Resource] {desc}\nURI: {self._uri}"
self._parameters: dict[str, Any] = {
"type": "object",
"properties": {},
"required": [],
}
self._resource_timeout = resource_timeout
@property
def name(self) -> str:
return self._name
@property
def description(self) -> str:
return self._description
@property
def parameters(self) -> dict[str, Any]:
return self._parameters
@property
def read_only(self) -> bool:
return True
async def execute(self, **kwargs: Any) -> str:
from mcp import types
try:
result = await asyncio.wait_for(
self._session.read_resource(self._uri),
timeout=self._resource_timeout,
)
except asyncio.TimeoutError:
logger.warning(
"MCP resource '{}' timed out after {}s", self._name, self._resource_timeout
)
return f"(MCP resource read timed out after {self._resource_timeout}s)"
except asyncio.CancelledError:
task = asyncio.current_task()
if task is not None and task.cancelling() > 0:
raise
logger.warning("MCP resource '{}' was cancelled by server/SDK", self._name)
return "(MCP resource read was cancelled)"
except Exception as exc:
logger.exception(
"MCP resource '{}' failed: {}: {}",
self._name,
type(exc).__name__,
exc,
)
return f"(MCP resource read failed: {type(exc).__name__})"
parts: list[str] = []
for block in result.contents:
if isinstance(block, types.TextResourceContents):
parts.append(block.text)
elif isinstance(block, types.BlobResourceContents):
parts.append(f"[Binary resource: {len(block.blob)} bytes]")
else:
parts.append(str(block))
return "\n".join(parts) or "(no output)"
class MCPPromptWrapper(Tool):
"""Wraps an MCP prompt as a read-only nanobot Tool."""
def __init__(self, session, server_name: str, prompt_def, prompt_timeout: int = 30):
self._session = session
self._prompt_name = prompt_def.name
self._name = f"mcp_{server_name}_prompt_{prompt_def.name}"
desc = prompt_def.description or prompt_def.name
self._description = (
f"[MCP Prompt] {desc}\n"
"Returns a filled prompt template that can be used as a workflow guide."
)
self._prompt_timeout = prompt_timeout
# Build parameters from prompt arguments
properties: dict[str, Any] = {}
required: list[str] = []
for arg in prompt_def.arguments or []:
prop: dict[str, Any] = {"type": "string"}
if getattr(arg, "description", None):
prop["description"] = arg.description
properties[arg.name] = prop
if arg.required:
required.append(arg.name)
self._parameters: dict[str, Any] = {
"type": "object",
"properties": properties,
"required": required,
}
@property
def name(self) -> str:
return self._name
@property
def description(self) -> str:
return self._description
@property
def parameters(self) -> dict[str, Any]:
return self._parameters
@property
def read_only(self) -> bool:
return True
async def execute(self, **kwargs: Any) -> str:
from mcp import types
from mcp.shared.exceptions import McpError
try:
result = await asyncio.wait_for(
self._session.get_prompt(self._prompt_name, arguments=kwargs),
timeout=self._prompt_timeout,
)
except asyncio.TimeoutError:
logger.warning("MCP prompt '{}' timed out after {}s", self._name, self._prompt_timeout)
return f"(MCP prompt call timed out after {self._prompt_timeout}s)"
except asyncio.CancelledError:
task = asyncio.current_task()
if task is not None and task.cancelling() > 0:
raise
logger.warning("MCP prompt '{}' was cancelled by server/SDK", self._name)
return "(MCP prompt call was cancelled)"
except McpError as exc:
logger.error(
"MCP prompt '{}' failed: code={} message={}",
self._name,
exc.error.code,
exc.error.message,
)
return f"(MCP prompt call failed: {exc.error.message} [code {exc.error.code}])"
except Exception as exc:
logger.exception(
"MCP prompt '{}' failed: {}: {}",
self._name,
type(exc).__name__,
exc,
)
return f"(MCP prompt call failed: {type(exc).__name__})"
parts: list[str] = []
for message in result.messages:
content = message.content
# content is a single ContentBlock (not a list) in MCP SDK >= 1.x
if isinstance(content, types.TextContent):
parts.append(content.text)
elif isinstance(content, list):
for block in content:
if isinstance(block, types.TextContent):
parts.append(block.text)
else:
parts.append(str(block))
else:
parts.append(str(content))
return "\n".join(parts) or "(no output)"
async def connect_mcp_servers(
mcp_servers: dict, registry: ToolRegistry, stack: AsyncExitStack
) -> None:
"""Connect to configured MCP servers and register their tools."""
mcp_servers: dict, registry: ToolRegistry
) -> dict[str, AsyncExitStack]:
"""Connect to configured MCP servers and register their tools, resources, prompts.
Returns a dict mapping server name -> its dedicated AsyncExitStack.
Each server gets its own stack and runs in its own task to prevent
cancel scope conflicts when multiple MCP servers are configured.
"""
from mcp import ClientSession, StdioServerParameters
from mcp.client.sse import sse_client
from mcp.client.stdio import stdio_client
from mcp.client.streamable_http import streamable_http_client
for name, cfg in mcp_servers.items():
async def connect_single_server(name: str, cfg) -> tuple[str, AsyncExitStack | None]:
server_stack = AsyncExitStack()
await server_stack.__aenter__()
try:
transport_type = cfg.type
if not transport_type:
if cfg.command:
transport_type = "stdio"
elif cfg.url:
# Convention: URLs ending with /sse use SSE transport; others use streamableHttp
transport_type = (
"sse" if cfg.url.rstrip("/").endswith("/sse") else "streamableHttp"
)
else:
logger.warning("MCP server '{}': no command or url configured, skipping", name)
continue
await server_stack.aclose()
return name, None
if transport_type == "stdio":
params = StdioServerParameters(
command=cfg.command, args=cfg.args, env=cfg.env or None
)
read, write = await stack.enter_async_context(stdio_client(params))
read, write = await server_stack.enter_async_context(stdio_client(params))
elif transport_type == "sse":
def httpx_client_factory(
headers: dict[str, str] | None = None,
timeout: httpx.Timeout | None = None,
@@ -182,27 +358,26 @@ async def connect_mcp_servers(
auth=auth,
)
read, write = await stack.enter_async_context(
read, write = await server_stack.enter_async_context(
sse_client(cfg.url, httpx_client_factory=httpx_client_factory)
)
elif transport_type == "streamableHttp":
# Always provide an explicit httpx client so MCP HTTP transport does not
# inherit httpx's default 5s timeout and preempt the higher-level tool timeout.
http_client = await stack.enter_async_context(
http_client = await server_stack.enter_async_context(
httpx.AsyncClient(
headers=cfg.headers or None,
follow_redirects=True,
timeout=None,
)
)
read, write, _ = await stack.enter_async_context(
read, write, _ = await server_stack.enter_async_context(
streamable_http_client(cfg.url, http_client=http_client)
)
else:
logger.warning("MCP server '{}': unknown transport type '{}'", name, transport_type)
continue
await server_stack.aclose()
return name, None
session = await stack.enter_async_context(ClientSession(read, write))
session = await server_stack.enter_async_context(ClientSession(read, write))
await session.initialize()
tools = await session.list_tools()
@@ -247,6 +422,76 @@ async def connect_mcp_servers(
", ".join(available_wrapped_names) or "(none)",
)
logger.info("MCP server '{}': connected, {} tools registered", name, registered_count)
try:
resources_result = await session.list_resources()
for resource in resources_result.resources:
wrapper = MCPResourceWrapper(
session, name, resource, resource_timeout=cfg.tool_timeout
)
registry.register(wrapper)
registered_count += 1
logger.debug(
"MCP: registered resource '{}' from server '{}'", wrapper.name, name
)
except Exception as e:
logger.debug("MCP server '{}': resources not supported or failed: {}", name, e)
try:
prompts_result = await session.list_prompts()
for prompt in prompts_result.prompts:
wrapper = MCPPromptWrapper(
session, name, prompt, prompt_timeout=cfg.tool_timeout
)
registry.register(wrapper)
registered_count += 1
logger.debug("MCP: registered prompt '{}' from server '{}'", wrapper.name, name)
except Exception as e:
logger.debug("MCP server '{}': prompts not supported or failed: {}", name, e)
logger.info(
"MCP server '{}': connected, {} capabilities registered", name, registered_count
)
return name, server_stack
except Exception as e:
logger.error("MCP server '{}': failed to connect: {}", name, e)
hint = ""
text = str(e).lower()
if any(
marker in text
for marker in (
"parse error",
"invalid json",
"unexpected token",
"jsonrpc",
"content-length",
)
):
hint = (
" Hint: this looks like stdio protocol pollution. Make sure the MCP server writes "
"only JSON-RPC to stdout and sends logs/debug output to stderr instead."
)
logger.error("MCP server '{}': failed to connect: {}{}", name, e, hint)
try:
await server_stack.aclose()
except Exception:
pass
return name, None
server_stacks: dict[str, AsyncExitStack] = {}
tasks: list[asyncio.Task] = []
for name, cfg in mcp_servers.items():
task = asyncio.create_task(connect_single_server(name, cfg))
tasks.append(task)
results = await asyncio.gather(*tasks, return_exceptions=True)
for i, result in enumerate(results):
name = list(mcp_servers.keys())[i]
if isinstance(result, BaseException):
if not isinstance(result, asyncio.CancelledError):
logger.error("MCP server '{}' connection task failed: {}", name, result)
elif result is not None and result[1] is not None:
server_stacks[result[0]] = result[1]
return server_stacks
+161
View File
@@ -0,0 +1,161 @@
"""NotebookEditTool — edit Jupyter .ipynb notebooks."""
from __future__ import annotations
import json
import uuid
from typing import Any
from nanobot.agent.tools.base import tool_parameters
from nanobot.agent.tools.schema import IntegerSchema, StringSchema, tool_parameters_schema
from nanobot.agent.tools.filesystem import _FsTool
def _new_cell(source: str, cell_type: str = "code", generate_id: bool = False) -> dict:
cell: dict[str, Any] = {
"cell_type": cell_type,
"source": source,
"metadata": {},
}
if cell_type == "code":
cell["outputs"] = []
cell["execution_count"] = None
if generate_id:
cell["id"] = uuid.uuid4().hex[:8]
return cell
def _make_empty_notebook() -> dict:
return {
"nbformat": 4,
"nbformat_minor": 5,
"metadata": {
"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"},
"language_info": {"name": "python"},
},
"cells": [],
}
@tool_parameters(
tool_parameters_schema(
path=StringSchema("Path to the .ipynb notebook file"),
cell_index=IntegerSchema(0, description="0-based index of the cell to edit", minimum=0),
new_source=StringSchema("New source content for the cell"),
cell_type=StringSchema(
"Cell type: 'code' or 'markdown' (default: code)",
enum=["code", "markdown"],
),
edit_mode=StringSchema(
"Mode: 'replace' (default), 'insert' (after target), or 'delete'",
enum=["replace", "insert", "delete"],
),
required=["path", "cell_index"],
)
)
class NotebookEditTool(_FsTool):
"""Edit Jupyter notebook cells: replace, insert, or delete."""
_VALID_CELL_TYPES = frozenset({"code", "markdown"})
_VALID_EDIT_MODES = frozenset({"replace", "insert", "delete"})
@property
def name(self) -> str:
return "notebook_edit"
@property
def description(self) -> str:
return (
"Edit a Jupyter notebook (.ipynb) cell. "
"Modes: replace (default) replaces cell content, "
"insert adds a new cell after the target index, "
"delete removes the cell at the index. "
"cell_index is 0-based."
)
async def execute(
self,
path: str | None = None,
cell_index: int = 0,
new_source: str = "",
cell_type: str = "code",
edit_mode: str = "replace",
**kwargs: Any,
) -> str:
try:
if not path:
return "Error: path is required"
if not path.endswith(".ipynb"):
return "Error: notebook_edit only works on .ipynb files. Use edit_file for other files."
if edit_mode not in self._VALID_EDIT_MODES:
return (
f"Error: Invalid edit_mode '{edit_mode}'. "
"Use one of: replace, insert, delete."
)
if cell_type not in self._VALID_CELL_TYPES:
return (
f"Error: Invalid cell_type '{cell_type}'. "
"Use one of: code, markdown."
)
fp = self._resolve(path)
# Create new notebook if file doesn't exist and mode is insert
if not fp.exists():
if edit_mode != "insert":
return f"Error: File not found: {path}"
nb = _make_empty_notebook()
cell = _new_cell(new_source, cell_type, generate_id=True)
nb["cells"].append(cell)
fp.parent.mkdir(parents=True, exist_ok=True)
fp.write_text(json.dumps(nb, indent=1, ensure_ascii=False), encoding="utf-8")
return f"Successfully created {fp} with 1 cell"
try:
nb = json.loads(fp.read_text(encoding="utf-8"))
except (json.JSONDecodeError, UnicodeDecodeError) as e:
return f"Error: Failed to parse notebook: {e}"
cells = nb.get("cells", [])
nbformat_minor = nb.get("nbformat_minor", 0)
generate_id = nb.get("nbformat", 0) >= 4 and nbformat_minor >= 5
if edit_mode == "delete":
if cell_index < 0 or cell_index >= len(cells):
return f"Error: cell_index {cell_index} out of range (notebook has {len(cells)} cells)"
cells.pop(cell_index)
nb["cells"] = cells
fp.write_text(json.dumps(nb, indent=1, ensure_ascii=False), encoding="utf-8")
return f"Successfully deleted cell {cell_index} from {fp}"
if edit_mode == "insert":
insert_at = min(cell_index + 1, len(cells))
cell = _new_cell(new_source, cell_type, generate_id=generate_id)
cells.insert(insert_at, cell)
nb["cells"] = cells
fp.write_text(json.dumps(nb, indent=1, ensure_ascii=False), encoding="utf-8")
return f"Successfully inserted cell at index {insert_at} in {fp}"
# Default: replace
if cell_index < 0 or cell_index >= len(cells):
return f"Error: cell_index {cell_index} out of range (notebook has {len(cells)} cells)"
cells[cell_index]["source"] = new_source
if cell_type and cells[cell_index].get("cell_type") != cell_type:
cells[cell_index]["cell_type"] = cell_type
if cell_type == "code":
cells[cell_index].setdefault("outputs", [])
cells[cell_index].setdefault("execution_count", None)
elif "outputs" in cells[cell_index]:
del cells[cell_index]["outputs"]
cells[cell_index].pop("execution_count", None)
nb["cells"] = cells
fp.write_text(json.dumps(nb, indent=1, ensure_ascii=False), encoding="utf-8")
return f"Successfully edited cell {cell_index} in {fp}"
except PermissionError as e:
return f"Error: {e}"
except Exception as e:
return f"Error editing notebook: {e}"
+17 -2
View File
@@ -14,14 +14,17 @@ class ToolRegistry:
def __init__(self):
self._tools: dict[str, Tool] = {}
self._cached_definitions: list[dict[str, Any]] | None = None
def register(self, tool: Tool) -> None:
"""Register a tool."""
self._tools[tool.name] = tool
self._cached_definitions = None
def unregister(self, name: str) -> None:
"""Unregister a tool by name."""
self._tools.pop(name, None)
self._cached_definitions = None
def get(self, name: str) -> Tool | None:
"""Get a tool by name."""
@@ -46,8 +49,12 @@ class ToolRegistry:
"""Get tool definitions with stable ordering for cache-friendly prompts.
Built-in tools are sorted first as a stable prefix, then MCP tools are
sorted and appended.
sorted and appended. The result is cached until the next
register/unregister call.
"""
if self._cached_definitions is not None:
return self._cached_definitions
definitions = [tool.to_schema() for tool in self._tools.values()]
builtins: list[dict[str, Any]] = []
mcp_tools: list[dict[str, Any]] = []
@@ -60,7 +67,8 @@ class ToolRegistry:
builtins.sort(key=self._schema_name)
mcp_tools.sort(key=self._schema_name)
return builtins + mcp_tools
self._cached_definitions = builtins + mcp_tools
return self._cached_definitions
def prepare_call(
self,
@@ -68,6 +76,13 @@ class ToolRegistry:
params: dict[str, Any],
) -> tuple[Tool | None, dict[str, Any], str | None]:
"""Resolve, cast, and validate one tool call."""
# Guard against invalid parameter types (e.g., list instead of dict)
if not isinstance(params, dict) and name in ('write_file', 'read_file'):
return None, params, (
f"Error: Tool '{name}' parameters must be a JSON object, got {type(params).__name__}. "
"Use named parameters: tool_name(param1=\"value1\", param2=\"value2\")"
)
tool = self._tools.get(name)
if not tool:
return None, params, (
+7 -5
View File
@@ -142,8 +142,9 @@ class GlobTool(_SearchTool):
@property
def description(self) -> str:
return (
"Find files matching a glob pattern. "
"Simple patterns like '*.py' match by filename recursively."
"Find files matching a glob pattern (e.g. '*.py', 'tests/**/test_*.py'). "
"Results are sorted by modification time (newest first). "
"Skips .git, node_modules, __pycache__, and other noise directories."
)
@property
@@ -261,9 +262,10 @@ class GrepTool(_SearchTool):
@property
def description(self) -> str:
return (
"Search file contents with a regex-like pattern. "
"Supports optional glob filtering, structured output modes, "
"type filters, pagination, and surrounding context lines."
"Search file contents with a regex pattern. "
"Default output_mode is files_with_matches (file paths only); "
"use content mode for matching lines with context. "
"Skips binary and files >2 MB. Supports glob/type filtering."
)
@property
+439
View File
@@ -0,0 +1,439 @@
"""MyTool: runtime state inspection and configuration for the agent loop."""
from __future__ import annotations
import time
from typing import TYPE_CHECKING, Any
from loguru import logger
from nanobot.agent.subagent import SubagentStatus
from nanobot.agent.tools.base import Tool
if TYPE_CHECKING:
from nanobot.agent.loop import AgentLoop
def _has_real_attr(obj: Any, key: str) -> bool:
"""Check if obj has a real (explicitly set) attribute, not auto-generated by mock."""
if isinstance(obj, dict):
return key in obj
d = getattr(obj, "__dict__", None)
if d is not None and key in d:
return True
for cls in type(obj).__mro__:
if key in cls.__dict__:
return True
return False
class MyTool(Tool):
"""Check and set the agent loop's runtime configuration."""
BLOCKED = frozenset({
# Core infrastructure
"bus", "provider", "_running", "tools",
# Config management
"_runtime_vars",
# Subsystems
"runner", "sessions", "consolidator",
"dream", "auto_compact", "context", "commands",
# Sensitive runtime state (credentials, message routing, task tracking)
"_mcp_servers", "_mcp_stacks", "_pending_queues",
"_session_locks", "_active_tasks", "_background_tasks",
# Security boundaries (inspect + modify both blocked)
"restrict_to_workspace", "channels_config",
"_concurrency_gate", "_unified_session", "_extra_hooks",
})
READ_ONLY = frozenset({
"subagents", # observable but replacing it would break the system
"_current_iteration", # updated by runner only
"exec_config", # inspect allowed (e.g. check sandbox), modify blocked
"web_config", # inspect allowed (e.g. check enable), modify blocked
})
_DENIED_ATTRS = frozenset({
"__class__", "__dict__", "__bases__", "__subclasses__", "__mro__",
"__init__", "__new__", "__reduce__", "__getstate__", "__setstate__",
"__del__", "__call__", "__getattr__", "__setattr__", "__delattr__",
"__code__", "__globals__", "func_globals", "func_code",
"__wrapped__", "__closure__",
})
# Sub-field names that are sensitive regardless of parent path
_SENSITIVE_NAMES = frozenset({
"api_key", "secret", "password", "token", "credential",
"private_key", "access_token", "refresh_token", "auth",
})
RESTRICTED: dict[str, dict[str, Any]] = {
"max_iterations": {"type": int, "min": 1, "max": 100},
"context_window_tokens": {"type": int, "min": 4096, "max": 1_000_000},
"model": {"type": str, "min_len": 1},
}
_MAX_RUNTIME_KEYS = 64
def __init__(self, loop: AgentLoop, modify_allowed: bool = True) -> None:
self._loop = loop
self._modify_allowed = modify_allowed
self._channel = ""
self._chat_id = ""
def __deepcopy__(self, memo: dict[int, Any]) -> MyTool:
cls = self.__class__
result = cls.__new__(cls)
memo[id(self)] = result
result._loop = self._loop
result._modify_allowed = self._modify_allowed
result._channel = self._channel
result._chat_id = self._chat_id
return result
def set_context(self, channel: str, chat_id: str) -> None:
self._channel = channel
self._chat_id = chat_id
@property
def name(self) -> str:
return "my"
@property
def description(self) -> str:
base = (
"Check and set your own runtime state.\n"
"Actions: check, set.\n"
"- check (no key): full config overview — start here.\n"
"- check (key): drill into a value. Dot-paths allowed "
"(e.g. '_last_usage.prompt_tokens', 'web_config.enable').\n"
"- set (key, value): change config or store notes in your scratchpad. "
"Scratchpad keys persist across turns but not restarts.\n"
"Key values: _current_iteration (current progress), "
"max_iterations - _current_iteration = remaining iterations.\n"
"Note: web_config and exec_config are readable but read-only.\n"
"\n"
"When to use:\n"
"- User asks about your model, settings, or token usage → check that key.\n"
"- A tool fails or behaves unexpectedly → check the related config to diagnose.\n"
"- User asks you to remember a preference for this session → set to store it in your scratchpad.\n"
"- About to start a large task → check context_window_tokens and max_iterations first."
)
if not self._modify_allowed:
base += "\nREAD-ONLY MODE: set is disabled."
else:
base += (
"\nIMPORTANT: Before setting state, predict the potential impact. "
"If the operation could cause crashes or instability "
"(e.g. changing model), warn the user first."
)
return base
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"action": {
"type": "string",
"enum": ["check", "set"],
"description": "Action to perform",
},
"key": {
"type": "string",
"description": "Dot-path for check/set. Examples: 'max_iterations', 'workspace', 'provider_retry_mode'. "
"For check without key, shows all config values.",
},
"value": {"description": "New value (for set). Type must match target (int for max_iterations/context_window_tokens, str for model)."},
},
"required": ["action"],
}
def _audit(self, action: str, detail: str) -> None:
session = f"{self._channel}:{self._chat_id}" if self._channel else "unknown"
logger.info("self.{} | {} | session:{}", action, detail, session)
# ------------------------------------------------------------------
# Path resolution
# ------------------------------------------------------------------
def _resolve_path(self, path: str) -> tuple[Any, str | None]:
parts = path.split(".")
obj = self._loop
for part in parts:
if part in self._DENIED_ATTRS or part.startswith("__"):
return None, f"'{part}' is not accessible"
if part in self.BLOCKED:
return None, f"'{part}' is not accessible"
if part.lower() in self._SENSITIVE_NAMES:
return None, f"'{part}' is not accessible"
try:
if isinstance(obj, dict):
if part in obj:
obj = obj[part]
else:
return None, f"'{part}' not found in dict"
else:
obj = getattr(obj, part)
except (KeyError, AttributeError) as e:
return None, f"'{part}' not found: {e}"
return obj, None
@staticmethod
def _validate_key(key: str | None, label: str = "key") -> str | None:
if not key or not key.strip():
return f"Error: '{label}' cannot be empty or whitespace"
return None
# ------------------------------------------------------------------
# Smart formatting
# ------------------------------------------------------------------
@staticmethod
def _format_status(st: SubagentStatus, indent: str = " ") -> str:
elapsed = time.monotonic() - st.started_at
tool_summary = ", ".join(
f"{e.get('name', '?')}({e.get('status', '?')})" for e in st.tool_events[-5:]
) or "none"
lines = [
f"{indent}phase: {st.phase}, iteration: {st.iteration}, elapsed: {elapsed:.1f}s",
f"{indent}tools: {tool_summary}",
f"{indent}usage: {st.usage or 'n/a'}",
]
if st.error:
lines.append(f"{indent}error: {st.error}")
if st.stop_reason:
lines.append(f"{indent}stop_reason: {st.stop_reason}")
return "\n".join(lines)
@staticmethod
def _format_value(val: Any, key: str = "") -> str:
if isinstance(val, SubagentStatus):
header = f"Subagent [{val.task_id}] '{val.label}'"
detail = MyTool._format_status(val, " ")
return f"{header}\n task: {val.task_description}\n{detail}"
# SubagentManager: delegate to its _task_statuses dict
if hasattr(val, "_task_statuses") and isinstance(val._task_statuses, dict):
return MyTool._format_value(val._task_statuses, key)
if isinstance(val, dict) and val and isinstance(next(iter(val.values())), SubagentStatus):
prefix = f"{key}: " if key else ""
lines = [f"{prefix}{len(val)} subagent(s):"]
for tid, st in val.items():
detail = MyTool._format_status(st, " ")
lines.append(f" [{tid}] '{st.label}'\n{detail}")
return "\n".join(lines)
if hasattr(val, "tool_names"):
return f"tools: {len(val.tool_names)} registered — {val.tool_names}"
# Scalar types — repr is fine
if isinstance(val, (str, int, float, bool, type(None))):
r = repr(val)
return f"{key}: {r}" if key else r
# Dict — small: show content; large: show keys for dot-path navigation
if isinstance(val, dict):
ks = list(val.keys())
if not ks:
return f"{key}: {{}}" if key else "{}"
if len(ks) <= 5:
r = repr(val)
if len(r) <= 200:
return f"{key}: {r}" if key else r
preview = ", ".join(str(k) for k in ks[:15])
suffix = ", ..." if len(ks) > 15 else ""
return f"{key}: {{{preview}{suffix}}}" if key else f"{{{preview}{suffix}}}"
# List/tuple — count for large, repr for small
if isinstance(val, (list, tuple)):
if len(val) > 20:
return f"{key}: [{len(val)} items]" if key else f"[{len(val)} items]"
r = repr(val)
return f"{key}: {r}" if key else r
# Complex object — small Pydantic models: show values; others: show field names for navigation
cls_name = type(val).__name__
if hasattr(val, "model_fields"):
fields = list(val.model_fields.keys())
if len(fields) <= 8:
# Small config objects: show field=value pairs
pairs = []
for f in fields:
fv = getattr(val, f, "?")
if isinstance(fv, (str, int, float, bool, type(None))):
pairs.append(f"{f}={fv!r}")
else:
pairs.append(f"{f}=<{type(fv).__name__}>")
preview = ", ".join(pairs)
return f"{key}: {preview}" if key else preview
else:
fields = [a for a in getattr(val, "__dict__", {}) if not a.startswith("__")]
if fields:
preview = ", ".join(str(f) for f in fields[:20])
suffix = ", ..." if len(fields) > 20 else ""
return f"{key}: <{cls_name}> [{preview}{suffix}]" if key else f"<{cls_name}> [{preview}{suffix}]"
r = repr(val)
return f"{key}: {r}" if key else r
# ------------------------------------------------------------------
# Action dispatch
# ------------------------------------------------------------------
async def execute(
self,
action: str,
key: str | None = None,
value: Any = None,
**_kwargs: Any,
) -> str:
if action in ("inspect", "check"):
return self._inspect(key)
if not self._modify_allowed:
return "Error: set is disabled (tools.my.allow_set is false)"
if action in ("modify", "set"):
return self._modify(key, value)
return f"Unknown action: {action}"
# -- inspect --
def _inspect(self, key: str | None) -> str:
if not key:
return self._inspect_all()
top = key.split(".")[0]
if top in self._DENIED_ATTRS or top.startswith("__"):
return f"Error: '{top}' is not accessible"
obj, err = self._resolve_path(key)
if err:
# "scratchpad" alias for _runtime_vars
if key == "scratchpad":
rv = self._loop._runtime_vars
return self._format_value(rv, "scratchpad") if rv else "scratchpad is empty"
# Fallback: check _runtime_vars for simple keys stored by modify
if "." not in key and key in self._loop._runtime_vars:
return self._format_value(self._loop._runtime_vars[key], key)
return f"Error: {err}"
# Guard against mock auto-generated attributes
if "." not in key and not _has_real_attr(self._loop, key):
if key in self._loop._runtime_vars:
return self._format_value(self._loop._runtime_vars[key], key)
return f"Error: '{key}' not found"
return self._format_value(obj, key)
def _inspect_all(self) -> str:
loop = self._loop
parts: list[str] = []
# RESTRICTED keys
for k in self.RESTRICTED:
parts.append(self._format_value(getattr(loop, k, None), k))
# Other useful top-level keys shown in description
for k in ("workspace", "provider_retry_mode", "max_tool_result_chars", "_current_iteration", "web_config", "exec_config", "subagents"):
if _has_real_attr(loop, k):
parts.append(self._format_value(getattr(loop, k, None), k))
# Token usage
usage = loop._last_usage
if usage:
parts.append(self._format_value(usage, "_last_usage"))
rv = loop._runtime_vars
if rv:
parts.append(self._format_value(rv, "scratchpad"))
return "\n".join(parts)
# -- modify --
def _modify(self, key: str | None, value: Any) -> str:
if err := self._validate_key(key):
return err
top = key.split(".")[0]
if top in self.BLOCKED or top in self._DENIED_ATTRS or top.startswith("__") or top.lower() in self._SENSITIVE_NAMES:
self._audit("modify", f"BLOCKED {key}")
return f"Error: '{key}' is protected and cannot be modified"
if top in self.READ_ONLY:
self._audit("modify", f"READ_ONLY {key}")
return f"Error: '{key}' is read-only and cannot be modified"
if "." in key:
parent_path, leaf = key.rsplit(".", 1)
if leaf in self._DENIED_ATTRS or leaf.startswith("__"):
self._audit("modify", f"BLOCKED leaf '{leaf}'")
return f"Error: '{leaf}' is not accessible"
if leaf.lower() in self._SENSITIVE_NAMES:
self._audit("modify", f"BLOCKED sensitive leaf '{leaf}'")
return f"Error: '{leaf}' is not accessible"
parent, err = self._resolve_path(parent_path)
if err:
return f"Error: {err}"
if isinstance(parent, dict):
parent[leaf] = value
else:
setattr(parent, leaf, value)
self._audit("modify", f"{key} = {value!r}")
return f"Set {key} = {value!r}"
if key in self.RESTRICTED:
return self._modify_restricted(key, value)
return self._modify_free(key, value)
def _modify_restricted(self, key: str, value: Any) -> str:
spec = self.RESTRICTED[key]
expected = spec["type"]
if expected is int and isinstance(value, bool):
return f"Error: '{key}' must be {expected.__name__}, got bool"
if not isinstance(value, expected):
try:
value = expected(value)
except (ValueError, TypeError):
return f"Error: '{key}' must be {expected.__name__}, got {type(value).__name__}"
old = getattr(self._loop, key)
if "min" in spec and value < spec["min"]:
return f"Error: '{key}' must be >= {spec['min']}"
if "max" in spec and value > spec["max"]:
return f"Error: '{key}' must be <= {spec['max']}"
if "min_len" in spec and len(str(value)) < spec["min_len"]:
return f"Error: '{key}' must be at least {spec['min_len']} characters"
setattr(self._loop, key, value)
self._audit("modify", f"{key}: {old!r} -> {value!r}")
return f"Set {key} = {value!r} (was {old!r})"
def _modify_free(self, key: str, value: Any) -> str:
if _has_real_attr(self._loop, key):
old = getattr(self._loop, key)
if isinstance(old, (str, int, float, bool)):
old_t, new_t = type(old), type(value)
if old_t is float and new_t is int:
pass # int → float coercion allowed
elif old_t is not new_t:
self._audit(
"modify",
f"REJECTED type mismatch {key}: expects {old_t.__name__}, got {new_t.__name__}",
)
return f"Error: '{key}' expects {old_t.__name__}, got {new_t.__name__}"
setattr(self._loop, key, value)
self._audit("modify", f"{key}: {old!r} -> {value!r}")
return f"Set {key} = {value!r} (was {old!r})"
if callable(value):
self._audit("modify", f"REJECTED callable {key}")
return "Error: cannot store callable values"
err = self._validate_json_safe(value)
if err:
self._audit("modify", f"REJECTED {key}: {err}")
return f"Error: {err}"
if key not in self._loop._runtime_vars and len(self._loop._runtime_vars) >= self._MAX_RUNTIME_KEYS:
self._audit("modify", f"REJECTED {key}: max keys ({self._MAX_RUNTIME_KEYS}) reached")
return f"Error: scratchpad is full (max {self._MAX_RUNTIME_KEYS} keys). Remove unused keys first."
old = self._loop._runtime_vars.get(key)
self._loop._runtime_vars[key] = value
self._audit("modify", f"scratchpad.{key}: {old!r} -> {value!r}")
return f"Set scratchpad.{key} = {value!r}"
@classmethod
def _validate_json_safe(cls, value: Any, depth: int = 0) -> str | None:
if depth > 10:
return "value nesting too deep (max 10 levels)"
if isinstance(value, (str, int, float, bool, type(None))):
return None
if isinstance(value, list):
for i, item in enumerate(value):
if err := cls._validate_json_safe(item, depth + 1):
return f"list[{i}] contains {err}"
return None
if isinstance(value, dict):
for k, v in value.items():
if not isinstance(k, str):
return f"dict key must be str, got {type(k).__name__}"
if err := cls._validate_json_safe(v, depth + 1):
return f"dict key '{k}' contains {err}"
return None
return f"unsupported type {type(value).__name__}"
+50 -3
View File
@@ -46,6 +46,7 @@ class ExecTool(Tool):
restrict_to_workspace: bool = False,
sandbox: str = "",
path_append: str = "",
allowed_env_keys: list[str] | None = None,
):
self.timeout = timeout
self.working_dir = working_dir
@@ -60,10 +61,19 @@ class ExecTool(Tool):
r">\s*/dev/sd", # write to disk
r"\b(shutdown|reboot|poweroff)\b", # system power
r":\(\)\s*\{.*\};\s*:", # fork bomb
# Block writes to nanobot internal state files (#2989).
# history.jsonl / .dream_cursor are managed by append_history();
# direct writes corrupt the cursor format and crash /dream.
r">>?\s*\S*(?:history\.jsonl|\.dream_cursor)", # > / >> redirect
r"\btee\b[^|;&<>]*(?:history\.jsonl|\.dream_cursor)", # tee / tee -a
r"\b(?:cp|mv)\b(?:\s+[^\s|;&<>]+)+\s+\S*(?:history\.jsonl|\.dream_cursor)", # cp/mv target
r"\bdd\b[^|;&<>]*\bof=\S*(?:history\.jsonl|\.dream_cursor)", # dd of=
r"\bsed\s+-i[^|;&<>]*(?:history\.jsonl|\.dream_cursor)", # sed -i
]
self.allow_patterns = allow_patterns or []
self.restrict_to_workspace = restrict_to_workspace
self.path_append = path_append
self.allowed_env_keys = allowed_env_keys or []
@property
def name(self) -> str:
@@ -74,7 +84,13 @@ class ExecTool(Tool):
@property
def description(self) -> str:
return "Execute a shell command and return its output. Use with caution."
return (
"Execute a shell command and return its output. "
"Prefer read_file/write_file/edit_file over cat/echo/sed, "
"and grep/glob over shell find/grep. "
"Use -y or --yes flags to avoid interactive prompts. "
"Output is truncated at 10 000 chars; timeout defaults to 60s."
)
@property
def exclusive(self) -> bool:
@@ -85,6 +101,21 @@ class ExecTool(Tool):
timeout: int | None = None, **kwargs: Any,
) -> str:
cwd = working_dir or self.working_dir or os.getcwd()
# Prevent an LLM-supplied working_dir from escaping the configured
# workspace when restrict_to_workspace is enabled (#2826). Without
# this, a caller can pass working_dir="/etc" and then all absolute
# paths under /etc would pass the _guard_command check that anchors
# on cwd.
if self.restrict_to_workspace and self.working_dir:
try:
requested = Path(cwd).expanduser().resolve()
workspace_root = Path(self.working_dir).expanduser().resolve()
except Exception:
return "Error: working_dir could not be resolved"
if requested != workspace_root and workspace_root not in requested.parents:
return "Error: working_dir is outside the configured workspace"
guard_error = self._guard_command(command, cwd)
if guard_error:
return guard_error
@@ -202,7 +233,7 @@ class ExecTool(Tool):
"""
if _IS_WINDOWS:
sr = os.environ.get("SYSTEMROOT", r"C:\Windows")
return {
env = {
"SYSTEMROOT": sr,
"COMSPEC": os.environ.get("COMSPEC", f"{sr}\\system32\\cmd.exe"),
"USERPROFILE": os.environ.get("USERPROFILE", ""),
@@ -212,13 +243,29 @@ class ExecTool(Tool):
"TMP": os.environ.get("TMP", f"{sr}\\Temp"),
"PATHEXT": os.environ.get("PATHEXT", ".COM;.EXE;.BAT;.CMD"),
"PATH": os.environ.get("PATH", f"{sr}\\system32;{sr}"),
"APPDATA": os.environ.get("APPDATA", ""),
"LOCALAPPDATA": os.environ.get("LOCALAPPDATA", ""),
"ProgramData": os.environ.get("ProgramData", ""),
"ProgramFiles": os.environ.get("ProgramFiles", ""),
"ProgramFiles(x86)": os.environ.get("ProgramFiles(x86)", ""),
"ProgramW6432": os.environ.get("ProgramW6432", ""),
}
for key in self.allowed_env_keys:
val = os.environ.get(key)
if val is not None:
env[key] = val
return env
home = os.environ.get("HOME", "/tmp")
return {
env = {
"HOME": home,
"LANG": os.environ.get("LANG", "C.UTF-8"),
"TERM": os.environ.get("TERM", "dumb"),
}
for key in self.allowed_env_keys:
val = os.environ.get(key)
if val is not None:
env[key] = val
return env
def _guard_command(self, command: str, cwd: str) -> str | None:
"""Best-effort safety guard for potentially destructive commands."""
+62 -2
View File
@@ -84,7 +84,11 @@ class WebSearchTool(Tool):
"""Search the web using configured provider."""
name = "web_search"
description = "Search the web. Returns titles, URLs, and snippets."
description = (
"Search the web. Returns titles, URLs, and snippets. "
"count defaults to 5 (max 10). "
"Use web_fetch to read a specific page in full."
)
def __init__(self, config: WebSearchConfig | None = None, proxy: str | None = None):
from nanobot.config.schema import WebSearchConfig
@@ -92,10 +96,37 @@ class WebSearchTool(Tool):
self.config = config if config is not None else WebSearchConfig()
self.proxy = proxy
def _effective_provider(self) -> str:
"""Resolve the backend that execute() will actually use."""
provider = self.config.provider.strip().lower() or "brave"
if provider == "duckduckgo":
return "duckduckgo"
if provider == "brave":
api_key = self.config.api_key or os.environ.get("BRAVE_API_KEY", "")
return "brave" if api_key else "duckduckgo"
if provider == "tavily":
api_key = self.config.api_key or os.environ.get("TAVILY_API_KEY", "")
return "tavily" if api_key else "duckduckgo"
if provider == "searxng":
base_url = (self.config.base_url or os.environ.get("SEARXNG_BASE_URL", "")).strip()
return "searxng" if base_url else "duckduckgo"
if provider == "jina":
api_key = self.config.api_key or os.environ.get("JINA_API_KEY", "")
return "jina" if api_key else "duckduckgo"
if provider == "kagi":
api_key = self.config.api_key or os.environ.get("KAGI_API_KEY", "")
return "kagi" if api_key else "duckduckgo"
return provider
@property
def read_only(self) -> bool:
return True
@property
def exclusive(self) -> bool:
"""DuckDuckGo searches are serialized because ddgs is not concurrency-safe."""
return self._effective_provider() == "duckduckgo"
async def execute(self, query: str, count: int | None = None, **kwargs: Any) -> str:
provider = self.config.provider.strip().lower() or "brave"
n = min(max(count or self.config.max_results, 1), 10)
@@ -110,6 +141,8 @@ class WebSearchTool(Tool):
return await self._search_jina(query, n)
elif provider == "brave":
return await self._search_brave(query, n)
elif provider == "kagi":
return await self._search_kagi(query, n)
else:
return f"Error: unknown search provider '{provider}'"
@@ -200,6 +233,29 @@ class WebSearchTool(Tool):
logger.warning("Jina search failed ({}), falling back to DuckDuckGo", e)
return await self._search_duckduckgo(query, n)
async def _search_kagi(self, query: str, n: int) -> str:
api_key = self.config.api_key or os.environ.get("KAGI_API_KEY", "")
if not api_key:
logger.warning("KAGI_API_KEY not set, falling back to DuckDuckGo")
return await self._search_duckduckgo(query, n)
try:
async with httpx.AsyncClient(proxy=self.proxy) as client:
r = await client.get(
"https://kagi.com/api/v0/search",
params={"q": query, "limit": n},
headers={"Authorization": f"Bot {api_key}"},
timeout=10.0,
)
r.raise_for_status()
# t=0 items are search results; other values are related searches, etc.
items = [
{"title": d.get("title", ""), "url": d.get("url", ""), "content": d.get("snippet", "")}
for d in r.json().get("data", []) if d.get("t") == 0
]
return _format_results(query, items, n)
except Exception as e:
return f"Error: {e}"
async def _search_duckduckgo(self, query: str, n: int) -> str:
try:
# Note: duckduckgo_search is synchronous and does its own requests
@@ -239,7 +295,11 @@ class WebFetchTool(Tool):
"""Fetch and extract content from a URL."""
name = "web_fetch"
description = "Fetch URL and extract readable content (HTML → markdown/text)."
description = (
"Fetch a URL and extract readable content (HTML → markdown/text). "
"Output is capped at maxChars (default 50 000). "
"Works for most web pages and docs; may fail on login-walled or JS-heavy sites."
)
def __init__(self, max_chars: int = 50000, proxy: str | None = None):
self.max_chars = max_chars
+249 -51
View File
@@ -7,15 +7,30 @@ All requests route to a single persistent API session.
from __future__ import annotations
import asyncio
import base64
import json as _json
import mimetypes
import re
import time
import uuid
from pathlib import Path
from typing import Any
from aiohttp import web
from loguru import logger
from nanobot.config.paths import get_media_dir
from nanobot.utils.helpers import safe_filename
from nanobot.utils.runtime import EMPTY_FINAL_RESPONSE_MESSAGE
MAX_FILE_SIZE = 10 * 1024 * 1024 # 10 MB
_DATA_URL_RE = re.compile(r"^data:([^;]+);base64,(.+)$", re.DOTALL)
class _FileSizeExceeded(Exception):
"""Raised when an uploaded file exceeds the size limit."""
API_SESSION_KEY = "api:default"
API_CHAT_ID = "default"
@@ -24,6 +39,7 @@ API_CHAT_ID = "default"
# Response helpers
# ---------------------------------------------------------------------------
def _error_json(status: int, message: str, err_type: str = "invalid_request_error") -> web.Response:
return web.json_response(
{"error": {"message": message, "type": err_type, "code": status}},
@@ -56,50 +72,231 @@ def _response_text(value: Any) -> str:
return str(getattr(value, "content") or "")
return str(value)
# ---------------------------------------------------------------------------
# SSE helpers
# ---------------------------------------------------------------------------
def _sse_chunk(delta: str, model: str, chunk_id: str, finish_reason: str | None = None) -> bytes:
"""Format a single OpenAI-compatible SSE chunk."""
payload = {
"id": chunk_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [
{
"index": 0,
"delta": {"content": delta} if delta else {},
"finish_reason": finish_reason,
}
],
}
return f"data: {_json.dumps(payload)}\n\n".encode()
_SSE_DONE = b"data: [DONE]\n\n"
# ---------------------------------------------------------------------------
# Upload helpers
# ---------------------------------------------------------------------------
def _save_base64_data_url(data_url: str, media_dir: Path) -> str | None:
"""Decode a data:...;base64,... URL and save to disk."""
m = _DATA_URL_RE.match(data_url)
if not m:
return None
mime_type, b64_payload = m.group(1), m.group(2)
try:
raw = base64.b64decode(b64_payload)
except Exception:
return None
if len(raw) > MAX_FILE_SIZE:
raise _FileSizeExceeded(f"File exceeds {MAX_FILE_SIZE // (1024 * 1024)}MB limit")
ext = mimetypes.guess_extension(mime_type) or ".bin"
filename = f"{uuid.uuid4().hex[:12]}{ext}"
dest = media_dir / safe_filename(filename)
dest.write_bytes(raw)
return str(dest)
def _parse_json_content(body: dict) -> tuple[str, list[str]]:
"""Parse JSON request body. Returns (text, media_paths)."""
messages = body.get("messages")
if not isinstance(messages, list) or len(messages) != 1:
raise ValueError("Only a single user message is supported")
message = messages[0]
if not isinstance(message, dict) or message.get("role") != "user":
raise ValueError("Only a single user message is supported")
user_content = message.get("content", "")
media_dir = get_media_dir("api")
media_paths: list[str] = []
if isinstance(user_content, list):
text_parts: list[str] = []
for part in user_content:
if not isinstance(part, dict):
continue
if part.get("type") == "text":
text_parts.append(part.get("text", ""))
elif part.get("type") == "image_url":
url = part.get("image_url", {}).get("url", "")
if url.startswith("data:"):
saved = _save_base64_data_url(url, media_dir)
if saved:
media_paths.append(saved)
elif url:
raise ValueError(
"Remote image URLs are not supported. "
"Use base64 data URLs or upload files via multipart/form-data."
)
text = " ".join(text_parts)
elif isinstance(user_content, str):
text = user_content
else:
raise ValueError("Invalid content format")
return text, media_paths
async def _parse_multipart(request: web.Request) -> tuple[str, list[str], str | None, str | None]:
"""Parse multipart/form-data. Returns (text, media_paths, session_id, model)."""
media_dir = get_media_dir("api")
reader = await request.multipart()
text = ""
session_id = None
model = None
media_paths: list[str] = []
while True:
part = await reader.next()
if part is None:
break
if part.name == "message":
text = (await part.read()).decode("utf-8")
elif part.name == "session_id":
session_id = (await part.read()).decode("utf-8").strip()
elif part.name == "model":
model = (await part.read()).decode("utf-8").strip()
elif part.name == "files":
raw = await part.read()
if len(raw) > MAX_FILE_SIZE:
raise _FileSizeExceeded(
f"File '{part.filename}' exceeds {MAX_FILE_SIZE // (1024 * 1024)}MB limit"
)
base = safe_filename(part.filename or "upload.bin")
filename = f"{uuid.uuid4().hex[:12]}_{base}"
dest = media_dir / filename
dest.write_bytes(raw)
media_paths.append(str(dest))
if not text:
text = "请分析上传的文件"
return text, media_paths, session_id, model
# ---------------------------------------------------------------------------
# Route handlers
# ---------------------------------------------------------------------------
async def handle_chat_completions(request: web.Request) -> web.Response:
"""POST /v1/chat/completions"""
# --- Parse body ---
try:
body = await request.json()
except Exception:
return _error_json(400, "Invalid JSON body")
messages = body.get("messages")
if not isinstance(messages, list) or len(messages) != 1:
return _error_json(400, "Only a single user message is supported")
# Stream not yet supported
if body.get("stream", False):
return _error_json(400, "stream=true is not supported yet. Set stream=false or omit it.")
message = messages[0]
if not isinstance(message, dict) or message.get("role") != "user":
return _error_json(400, "Only a single user message is supported")
user_content = message.get("content", "")
if isinstance(user_content, list):
# Multi-modal content array — extract text parts
user_content = " ".join(
part.get("text", "") for part in user_content if part.get("type") == "text"
)
"""POST /v1/chat/completions — supports JSON and multipart/form-data."""
content_type = request.content_type or ""
if not isinstance(content_type, str):
content_type = ""
agent_loop = request.app["agent_loop"]
timeout_s: float = request.app.get("request_timeout", 120.0)
model_name: str = request.app.get("model_name", "nanobot")
if (requested_model := body.get("model")) and requested_model != model_name:
stream = False
try:
if content_type.startswith("multipart/"):
text, media_paths, session_id, requested_model = await _parse_multipart(request)
else:
try:
body = await request.json()
except Exception:
return _error_json(400, "Invalid JSON body")
stream = body.get("stream", False)
requested_model = body.get("model")
text, media_paths = _parse_json_content(body)
session_id = body.get("session_id")
except ValueError as e:
return _error_json(400, str(e))
except _FileSizeExceeded as e:
return _error_json(413, str(e), err_type="invalid_request_error")
except Exception:
logger.exception("Error parsing upload")
return _error_json(413, "File too large or invalid upload")
if requested_model and requested_model != model_name:
return _error_json(400, f"Only configured model '{model_name}' is available")
session_key = f"api:{body['session_id']}" if body.get("session_id") else API_SESSION_KEY
session_key = f"api:{session_id}" if session_id else API_SESSION_KEY
session_locks: dict[str, asyncio.Lock] = request.app["session_locks"]
session_lock = session_locks.setdefault(session_key, asyncio.Lock())
logger.info("API request session_key={} content={}", session_key, user_content[:80])
logger.info(
"API request session_key={} media={} text={} stream={}",
session_key, len(media_paths), text[:80], stream,
)
# -- streaming path --
if stream:
resp = web.StreamResponse()
resp.content_type = "text/event-stream"
resp.headers["Cache-Control"] = "no-cache"
resp.headers["Connection"] = "keep-alive"
resp.enable_compression()
await resp.prepare(request)
chunk_id = f"chatcmpl-{uuid.uuid4().hex[:12]}"
queue: asyncio.Queue[str | None] = asyncio.Queue()
async def _on_stream(token: str) -> None:
await queue.put(token)
async def _on_stream_end(*_a: Any, **_kw: Any) -> None:
await queue.put(None)
async def _run() -> None:
try:
async with session_lock:
await asyncio.wait_for(
agent_loop.process_direct(
content=text,
media=media_paths if media_paths else None,
session_key=session_key,
channel="api",
chat_id=API_CHAT_ID,
on_stream=_on_stream,
on_stream_end=_on_stream_end,
),
timeout=timeout_s,
)
except Exception:
logger.exception("Streaming error for session {}", session_key)
await queue.put(None)
task = asyncio.create_task(_run())
try:
while True:
token = await queue.get()
if token is None:
break
await resp.write(_sse_chunk(token, model_name, chunk_id))
finally:
task.cancel()
await resp.write(_sse_chunk("", model_name, chunk_id, finish_reason="stop"))
await resp.write(_SSE_DONE)
return resp
# -- non-streaming path (original logic) --
_FALLBACK = EMPTY_FINAL_RESPONSE_MESSAGE
try:
@@ -107,7 +304,8 @@ async def handle_chat_completions(request: web.Request) -> web.Response:
try:
response = await asyncio.wait_for(
agent_loop.process_direct(
content=user_content,
content=text,
media=media_paths if media_paths else None,
session_key=session_key,
channel="api",
chat_id=API_CHAT_ID,
@@ -117,13 +315,11 @@ async def handle_chat_completions(request: web.Request) -> web.Response:
response_text = _response_text(response)
if not response_text or not response_text.strip():
logger.warning(
"Empty response for session {}, retrying",
session_key,
)
logger.warning("Empty response for session {}, retrying", session_key)
retry_response = await asyncio.wait_for(
agent_loop.process_direct(
content=user_content,
content=text,
media=media_paths if media_paths else None,
session_key=session_key,
channel="api",
chat_id=API_CHAT_ID,
@@ -132,10 +328,7 @@ async def handle_chat_completions(request: web.Request) -> web.Response:
)
response_text = _response_text(retry_response)
if not response_text or not response_text.strip():
logger.warning(
"Empty response after retry for session {}, using fallback",
session_key,
)
logger.warning("Empty response after retry, using fallback")
response_text = _FALLBACK
except asyncio.TimeoutError:
@@ -153,17 +346,19 @@ async def handle_chat_completions(request: web.Request) -> web.Response:
async def handle_models(request: web.Request) -> web.Response:
"""GET /v1/models"""
model_name = request.app.get("model_name", "nanobot")
return web.json_response({
"object": "list",
"data": [
{
"id": model_name,
"object": "model",
"created": 0,
"owned_by": "nanobot",
}
],
})
return web.json_response(
{
"object": "list",
"data": [
{
"id": model_name,
"object": "model",
"created": 0,
"owned_by": "nanobot",
}
],
}
)
async def handle_health(request: web.Request) -> web.Response:
@@ -175,7 +370,10 @@ async def handle_health(request: web.Request) -> web.Response:
# App factory
# ---------------------------------------------------------------------------
def create_app(agent_loop, model_name: str = "nanobot", request_timeout: float = 120.0) -> web.Application:
def create_app(
agent_loop, model_name: str = "nanobot", request_timeout: float = 120.0
) -> web.Application:
"""Create the aiohttp application.
Args:
@@ -183,7 +381,7 @@ def create_app(agent_loop, model_name: str = "nanobot", request_timeout: float =
model_name: Model name reported in responses.
request_timeout: Per-request timeout in seconds.
"""
app = web.Application()
app = web.Application(client_max_size=20 * 1024 * 1024) # 20MB for base64 images
app["agent_loop"] = agent_loop
app["model_name"] = model_name
app["request_timeout"] = request_timeout
+16 -3
View File
@@ -24,6 +24,7 @@ class BaseChannel(ABC):
display_name: str = "Base"
transcription_provider: str = "groq"
transcription_api_key: str = ""
transcription_api_base: str = ""
def __init__(self, config: Any, bus: MessageBus):
"""
@@ -44,10 +45,16 @@ class BaseChannel(ABC):
try:
if self.transcription_provider == "openai":
from nanobot.providers.transcription import OpenAITranscriptionProvider
provider = OpenAITranscriptionProvider(api_key=self.transcription_api_key)
provider = OpenAITranscriptionProvider(
api_key=self.transcription_api_key,
api_base=self.transcription_api_base or None,
)
else:
from nanobot.providers.transcription import GroqTranscriptionProvider
provider = GroqTranscriptionProvider(api_key=self.transcription_api_key)
provider = GroqTranscriptionProvider(
api_key=self.transcription_api_key,
api_base=self.transcription_api_base or None,
)
return await provider.transcribe(file_path)
except Exception as e:
logger.warning("{}: audio transcription failed: {}", self.name, e)
@@ -116,7 +123,13 @@ class BaseChannel(ABC):
def is_allowed(self, sender_id: str) -> bool:
"""Check if *sender_id* is permitted. Empty list → deny all; ``"*"`` → allow all."""
allow_list = getattr(self.config, "allow_from", [])
if isinstance(self.config, dict):
if "allow_from" in self.config:
allow_list = self.config.get("allow_from")
else:
allow_list = self.config.get("allowFrom", [])
else:
allow_list = getattr(self.config, "allow_from", [])
if not allow_list:
logger.warning("{}: allow_from is empty — all access denied", self.name)
return False
+38
View File
@@ -5,6 +5,8 @@ import json
import mimetypes
import os
import time
import zipfile
from io import BytesIO
from pathlib import Path
from typing import Any
from urllib.parse import unquote, urlparse
@@ -171,6 +173,7 @@ class DingTalkChannel(BaseChannel):
_IMAGE_EXTS = {".jpg", ".jpeg", ".png", ".gif", ".bmp", ".webp"}
_AUDIO_EXTS = {".amr", ".mp3", ".wav", ".ogg", ".m4a", ".aac"}
_VIDEO_EXTS = {".mp4", ".mov", ".avi", ".mkv", ".webm"}
_ZIP_BEFORE_UPLOAD_EXTS = {".htm", ".html"}
@classmethod
def default_config(cls) -> dict[str, Any]:
@@ -287,6 +290,31 @@ class DingTalkChannel(BaseChannel):
name = os.path.basename(urlparse(media_ref).path)
return name or {"image": "image.jpg", "voice": "audio.amr", "video": "video.mp4"}.get(upload_type, "file.bin")
@staticmethod
def _zip_bytes(filename: str, data: bytes) -> tuple[bytes, str, str]:
stem = Path(filename).stem or "attachment"
safe_name = filename or "attachment.bin"
zip_name = f"{stem}.zip"
buffer = BytesIO()
with zipfile.ZipFile(buffer, mode="w", compression=zipfile.ZIP_DEFLATED) as archive:
archive.writestr(safe_name, data)
return buffer.getvalue(), zip_name, "application/zip"
def _normalize_upload_payload(
self,
filename: str,
data: bytes,
content_type: str | None,
) -> tuple[bytes, str, str | None]:
ext = Path(filename).suffix.lower()
if ext in self._ZIP_BEFORE_UPLOAD_EXTS or content_type == "text/html":
logger.info(
"DingTalk does not accept raw HTML attachments, zipping {} before upload",
filename,
)
return self._zip_bytes(filename, data)
return data, filename, content_type
async def _read_media_bytes(
self,
media_ref: str,
@@ -309,6 +337,9 @@ class DingTalkChannel(BaseChannel):
content_type = (resp.headers.get("content-type") or "").split(";")[0].strip()
filename = self._guess_filename(media_ref, self._guess_upload_type(media_ref))
return resp.content, filename, content_type or None
except httpx.TransportError as e:
logger.error("DingTalk media download network error ref={} err={}", media_ref, e)
raise
except Exception as e:
logger.error("DingTalk media download error ref={} err={}", media_ref, e)
return None, None, None
@@ -360,6 +391,9 @@ class DingTalkChannel(BaseChannel):
logger.error("DingTalk media upload missing media_id body={}", text[:500])
return None
return str(media_id)
except httpx.TransportError as e:
logger.error("DingTalk media upload network error type={} err={}", media_type, e)
raise
except Exception as e:
logger.error("DingTalk media upload error type={} err={}", media_type, e)
return None
@@ -409,6 +443,9 @@ class DingTalkChannel(BaseChannel):
return False
logger.debug("DingTalk message sent to {} with msgKey={}", chat_id, msg_key)
return True
except httpx.TransportError as e:
logger.error("DingTalk network error sending message msgKey={} err={}", msg_key, e)
raise
except Exception as e:
logger.error("Error sending DingTalk message msgKey={} err={}", msg_key, e)
return False
@@ -444,6 +481,7 @@ class DingTalkChannel(BaseChannel):
return False
filename = filename or self._guess_filename(media_ref, upload_type)
data, filename, content_type = self._normalize_upload_payload(filename, data, content_type)
file_type = Path(filename).suffix.lower().lstrip(".")
if not file_type:
guessed = mimetypes.guess_extension(content_type or "")
+172 -8
View File
@@ -4,6 +4,8 @@ from __future__ import annotations
import asyncio
import importlib.util
import time
from dataclasses import dataclass
from pathlib import Path
from typing import TYPE_CHECKING, Any, Literal
@@ -20,6 +22,7 @@ from nanobot.utils.helpers import safe_filename, split_message
DISCORD_AVAILABLE = importlib.util.find_spec("discord") is not None
if TYPE_CHECKING:
import aiohttp
import discord
from discord import app_commands
from discord.abc import Messageable
@@ -34,17 +37,32 @@ MAX_MESSAGE_LEN = 2000 # Discord message character limit
TYPING_INTERVAL_S = 8
@dataclass
class _StreamBuf:
"""Per-chat streaming accumulator for progressive Discord message edits."""
text: str = ""
message: Any | None = None
last_edit: float = 0.0
stream_id: str | None = None
class DiscordConfig(Base):
"""Discord channel configuration."""
enabled: bool = False
token: str = ""
allow_from: list[str] = Field(default_factory=list)
allow_channels: list[str] = Field(default_factory=list) # Allowed channel IDs (empty = all)
intents: int = 37377
group_policy: Literal["mention", "open"] = "mention"
read_receipt_emoji: str = "👀"
working_emoji: str = "🔧"
working_emoji_delay: float = 2.0
streaming: bool = True
proxy: str | None = None
proxy_username: str | None = None
proxy_password: str | None = None
if DISCORD_AVAILABLE:
@@ -52,8 +70,15 @@ if DISCORD_AVAILABLE:
class DiscordBotClient(discord.Client):
"""discord.py client that forwards events to the channel."""
def __init__(self, channel: DiscordChannel, *, intents: discord.Intents) -> None:
super().__init__(intents=intents)
def __init__(
self,
channel: DiscordChannel,
*,
intents: discord.Intents,
proxy: str | None = None,
proxy_auth: aiohttp.BasicAuth | None = None,
) -> None:
super().__init__(intents=intents, proxy=proxy, proxy_auth=proxy_auth)
self._channel = channel
self.tree = app_commands.CommandTree(self)
self._register_app_commands()
@@ -117,6 +142,7 @@ if DISCORD_AVAILABLE:
)
for name, description, command_text in commands:
@self.tree.command(name=name, description=description)
async def command_handler(
interaction: discord.Interaction,
@@ -173,7 +199,9 @@ if DISCORD_AVAILABLE:
else:
failed_media.append(Path(media_path).name)
for index, chunk in enumerate(self._build_chunks(msg.content or "", failed_media, sent_media)):
for index, chunk in enumerate(
self._build_chunks(msg.content or "", failed_media, sent_media)
):
kwargs: dict[str, Any] = {"content": chunk}
if index == 0 and reference is not None and not sent_media:
kwargs["reference"] = reference
@@ -242,6 +270,7 @@ class DiscordChannel(BaseChannel):
name = "discord"
display_name = "Discord"
_STREAM_EDIT_INTERVAL = 0.8
@classmethod
def default_config(cls) -> dict[str, Any]:
@@ -263,6 +292,7 @@ class DiscordChannel(BaseChannel):
self._bot_user_id: str | None = None
self._pending_reactions: dict[str, Any] = {} # chat_id -> message object
self._working_emoji_tasks: dict[str, asyncio.Task[None]] = {}
self._stream_bufs: dict[str, _StreamBuf] = {}
async def start(self) -> None:
"""Start the Discord client."""
@@ -277,7 +307,29 @@ class DiscordChannel(BaseChannel):
try:
intents = discord.Intents.none()
intents.value = self.config.intents
self._client = DiscordBotClient(self, intents=intents)
proxy_auth = None
has_user = bool(self.config.proxy_username)
has_pass = bool(self.config.proxy_password)
if has_user and has_pass:
import aiohttp
proxy_auth = aiohttp.BasicAuth(
login=self.config.proxy_username,
password=self.config.proxy_password,
)
elif has_user != has_pass:
logger.warning(
"Discord proxy auth incomplete: both proxy_username and "
"proxy_password must be set; ignoring partial credentials",
)
self._client = DiscordBotClient(
self,
intents=intents,
proxy=self.config.proxy,
proxy_auth=proxy_auth,
)
except Exception as e:
logger.error("Failed to initialize Discord client: {}", e)
self._client = None
@@ -315,11 +367,71 @@ class DiscordChannel(BaseChannel):
await client.send_outbound(msg)
except Exception as e:
logger.error("Error sending Discord message: {}", e)
raise
finally:
if not is_progress:
await self._stop_typing(msg.chat_id)
await self._clear_reactions(msg.chat_id)
async def send_delta(
self, chat_id: str, delta: str, metadata: dict[str, Any] | None = None
) -> None:
"""Progressive Discord delivery: send once, then edit until the stream ends."""
client = self._client
if client is None or not client.is_ready():
logger.warning("Discord client not ready; dropping stream delta")
return
meta = metadata or {}
stream_id = meta.get("_stream_id")
if meta.get("_stream_end"):
buf = self._stream_bufs.get(chat_id)
if not buf or buf.message is None or not buf.text:
return
if stream_id is not None and buf.stream_id is not None and buf.stream_id != stream_id:
return
await self._finalize_stream(chat_id, buf)
return
buf = self._stream_bufs.get(chat_id)
if buf is None or (
stream_id is not None and buf.stream_id is not None and buf.stream_id != stream_id
):
buf = _StreamBuf(stream_id=stream_id)
self._stream_bufs[chat_id] = buf
elif buf.stream_id is None:
buf.stream_id = stream_id
buf.text += delta
if not buf.text.strip():
return
target = await self._resolve_channel(chat_id)
if target is None:
logger.warning("Discord stream target {} unavailable", chat_id)
return
now = time.monotonic()
if buf.message is None:
try:
buf.message = await target.send(content=buf.text)
buf.last_edit = now
except Exception as e:
logger.warning("Discord stream initial send failed: {}", e)
raise
return
if (now - buf.last_edit) < self._STREAM_EDIT_INTERVAL:
return
try:
await buf.message.edit(content=DiscordBotClient._build_chunks(buf.text, [], False)[0])
buf.last_edit = now
except Exception as e:
logger.warning("Discord stream edit failed: {}", e)
raise
async def _handle_discord_message(self, message: discord.Message) -> None:
"""Handle incoming Discord messages from discord.py."""
if message.author.bot:
@@ -339,7 +451,6 @@ class DiscordChannel(BaseChannel):
await self._start_typing(message.channel)
# Add read receipt reaction immediately, working emoji after delay
channel_id = self._channel_key(message.channel)
try:
await message.add_reaction(self.config.read_receipt_emoji)
self._pending_reactions[channel_id] = message
@@ -373,6 +484,47 @@ class DiscordChannel(BaseChannel):
"""Backward-compatible alias for legacy tests/callers."""
await self._handle_discord_message(message)
async def _resolve_channel(self, chat_id: str) -> Any | None:
"""Resolve a Discord channel from cache first, then network fetch."""
client = self._client
if client is None or not client.is_ready():
return None
channel_id = int(chat_id)
channel = client.get_channel(channel_id)
if channel is not None:
return channel
try:
return await client.fetch_channel(channel_id)
except Exception as e:
logger.warning("Discord channel {} unavailable: {}", chat_id, e)
return None
async def _finalize_stream(self, chat_id: str, buf: _StreamBuf) -> None:
"""Commit the final streamed content and flush overflow chunks."""
chunks = DiscordBotClient._build_chunks(buf.text, [], False)
if not chunks:
self._stream_bufs.pop(chat_id, None)
return
try:
await buf.message.edit(content=chunks[0])
except Exception as e:
logger.warning("Discord final stream edit failed: {}", e)
raise
target = getattr(buf.message, "channel", None) or await self._resolve_channel(chat_id)
if target is None:
logger.warning("Discord stream follow-up target {} unavailable", chat_id)
self._stream_bufs.pop(chat_id, None)
return
for extra_chunk in chunks[1:]:
await target.send(content=extra_chunk)
self._stream_bufs.pop(chat_id, None)
await self._stop_typing(chat_id)
await self._clear_reactions(chat_id)
def _should_accept_inbound(
self,
message: discord.Message,
@@ -382,6 +534,12 @@ class DiscordChannel(BaseChannel):
"""Check if inbound Discord message should be processed."""
if not self.is_allowed(sender_id):
return False
# Channel-based filtering: only respond in allowed channels
allow_channels = self.config.allow_channels
if allow_channels:
channel_id = self._channel_key(message.channel)
if channel_id not in allow_channels:
return False
if message.guild is not None and not self._should_respond_in_group(message, content):
return False
return True
@@ -423,7 +581,11 @@ class DiscordChannel(BaseChannel):
@staticmethod
def _build_inbound_metadata(message: discord.Message) -> dict[str, str | None]:
"""Build metadata for inbound Discord messages."""
reply_to = str(message.reference.message_id) if message.reference and message.reference.message_id else None
reply_to = (
str(message.reference.message_id)
if message.reference and message.reference.message_id
else None
)
return {
"message_id": str(message.id),
"guild_id": str(message.guild.id) if message.guild else None,
@@ -438,7 +600,9 @@ class DiscordChannel(BaseChannel):
if self.config.group_policy == "mention":
bot_user_id = self._bot_user_id
if bot_user_id is None:
logger.debug("Discord message in {} ignored (bot identity unavailable)", message.channel.id)
logger.debug(
"Discord message in {} ignored (bot identity unavailable)", message.channel.id
)
return False
if any(str(user.id) == bot_user_id for user in message.mentions):
@@ -480,7 +644,6 @@ class DiscordChannel(BaseChannel):
except asyncio.CancelledError:
pass
async def _clear_reactions(self, chat_id: str) -> None:
"""Remove all pending reactions after bot replies."""
# Cancel delayed working emoji if it hasn't fired yet
@@ -507,6 +670,7 @@ class DiscordChannel(BaseChannel):
async def _reset_runtime_state(self, close_client: bool) -> None:
"""Reset client and typing state."""
await self._cancel_all_typing()
self._stream_bufs.clear()
if close_client and self._client is not None and not self._client.is_closed():
try:
await self._client.close()
+44 -68
View File
@@ -1290,7 +1290,6 @@ class FeishuChannel(BaseChannel):
Supported metadata keys:
_stream_end: Finalize the streaming card.
_resuming: Mid-turn pause flush but keep the buffer alive.
_tool_hint: Delta is a formatted tool hint (for display only).
message_id: Original message id (used with _stream_end for reaction cleanup).
reaction_id: Reaction id to remove on stream end.
@@ -1309,50 +1308,44 @@ class FeishuChannel(BaseChannel):
if self.config.done_emoji and message_id:
await self._add_reaction(message_id, self.config.done_emoji)
resuming = meta.get("_resuming", False)
if resuming:
# Mid-turn pause (e.g. tool call between streaming segments).
# Flush current text to card but keep the buffer alive so the
# next segment appends to the same card.
buf = self._stream_bufs.get(chat_id)
if buf and buf.card_id and buf.text:
buf.sequence += 1
await loop.run_in_executor(
None, self._stream_update_text_sync, buf.card_id, buf.text, buf.sequence,
)
return
buf = self._stream_bufs.pop(chat_id, None)
if not buf or not buf.text:
return
# Try to finalize via streaming card; if that fails (e.g.
# streaming mode was closed by Feishu due to timeout), fall
# back to sending a regular interactive card.
if buf.card_id:
buf.sequence += 1
await loop.run_in_executor(
ok = await loop.run_in_executor(
None,
self._stream_update_text_sync,
buf.card_id,
buf.text,
buf.sequence,
)
# Required so the chat list preview exits the streaming placeholder (Feishu streaming card docs).
buf.sequence += 1
await loop.run_in_executor(
None,
self._close_streaming_mode_sync,
buf.card_id,
buf.sequence,
)
else:
for chunk in self._split_elements_by_table_limit(
self._build_card_elements(buf.text)
):
card = json.dumps(
{"config": {"wide_screen_mode": True}, "elements": chunk},
ensure_ascii=False,
)
if ok:
buf.sequence += 1
await loop.run_in_executor(
None, self._send_message_sync, rid_type, chat_id, "interactive", card
None,
self._close_streaming_mode_sync,
buf.card_id,
buf.sequence,
)
return
logger.warning(
"Streaming card {} final update failed, falling back to regular card",
buf.card_id,
)
for chunk in self._split_elements_by_table_limit(
self._build_card_elements(buf.text)
):
card = json.dumps(
{"config": {"wide_screen_mode": True}, "elements": chunk},
ensure_ascii=False,
)
await loop.run_in_executor(
None, self._send_message_sync, rid_type, chat_id, "interactive", card
)
return
# --- accumulate delta ---
@@ -1404,14 +1397,21 @@ class FeishuChannel(BaseChannel):
if buf and buf.card_id:
# Delegate to send_delta so tool hints get the same
# throttling (and card creation) as regular text deltas.
lines = self.__class__._format_tool_hint_lines(hint).split("\n")
delta = "\n\n" + "\n".join(
f"{self.config.tool_hint_prefix} {ln}" for ln in lines if ln.strip()
) + "\n\n"
await self.send_delta(msg.chat_id, delta)
await self.send_delta(
msg.chat_id,
"\n\n" + self._format_tool_hint_delta(hint) + "\n\n",
)
return
await self._send_tool_hint_card(
receive_id_type, msg.chat_id, hint
# No active streaming card — send as a regular
# interactive card with the same 🔧 prefix style.
card = json.dumps(
{"config": {"wide_screen_mode": True}, "elements": [
{"tag": "markdown", "content": self._format_tool_hint_delta(hint)},
]},
ensure_ascii=False,
)
await loop.run_in_executor(
None, self._send_message_sync, receive_id_type, msg.chat_id, "interactive", card
)
return
@@ -1708,33 +1708,9 @@ class FeishuChannel(BaseChannel):
return "\n".join(part for part in parts if part)
async def _send_tool_hint_card(
self, receive_id_type: str, receive_id: str, tool_hint: str
) -> None:
"""Send tool hint as an interactive card with formatted code block.
Args:
receive_id_type: "chat_id" or "open_id"
receive_id: The target chat or user ID
tool_hint: Formatted tool hint string (e.g., 'web_search("q"), read_file("path")')
"""
loop = asyncio.get_running_loop()
# Put each top-level tool call on its own line without altering commas inside arguments.
formatted_code = self.__class__._format_tool_hint_lines(tool_hint)
card = {
"config": {"wide_screen_mode": True},
"elements": [
{"tag": "markdown", "content": f"**Tool Calls**\n\n```text\n{formatted_code}\n```"}
],
}
await loop.run_in_executor(
None,
self._send_message_sync,
receive_id_type,
receive_id,
"interactive",
json.dumps(card, ensure_ascii=False),
def _format_tool_hint_delta(self, tool_hint: str) -> str:
"""Format a tool hint string with the 🔧 prefix for each line."""
lines = self.__class__._format_tool_hint_lines(tool_hint).split("\n")
return "\n".join(
f"{self.config.tool_hint_prefix} {ln}" for ln in lines if ln.strip()
)
+20 -1
View File
@@ -41,6 +41,7 @@ class ChannelManager:
transcription_provider = self.config.channels.transcription_provider
transcription_key = self._resolve_transcription_key(transcription_provider)
transcription_base = self._resolve_transcription_base(transcription_provider)
for name, cls in discover_all().items():
section = getattr(self.config.channels, name, None)
@@ -57,6 +58,7 @@ class ChannelManager:
channel = cls(section, self.bus)
channel.transcription_provider = transcription_provider
channel.transcription_api_key = transcription_key
channel.transcription_api_base = transcription_base
self.channels[name] = channel
logger.info("{} channel enabled", cls.display_name)
except Exception as e:
@@ -73,9 +75,26 @@ class ChannelManager:
except AttributeError:
return ""
def _resolve_transcription_base(self, provider: str) -> str:
"""Pick the API base URL for the configured transcription provider."""
try:
if provider == "openai":
return self.config.providers.openai.api_base or ""
return self.config.providers.groq.api_base or ""
except AttributeError:
return ""
def _validate_allow_from(self) -> None:
for name, ch in self.channels.items():
if getattr(ch.config, "allow_from", None) == []:
cfg = ch.config
if isinstance(cfg, dict):
if "allow_from" in cfg:
allow = cfg.get("allow_from")
else:
allow = cfg.get("allowFrom")
else:
allow = getattr(cfg, "allow_from", None)
if allow == []:
raise SystemExit(
f'Error: "{name}" has empty allowFrom (denies all). '
f'Set ["*"] to allow everyone, or add specific user IDs.'
+38 -25
View File
@@ -17,6 +17,7 @@ import importlib.util
import json
import re
import threading
import time
from dataclasses import dataclass
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from typing import TYPE_CHECKING, Any
@@ -31,7 +32,10 @@ from nanobot.channels.base import BaseChannel
from nanobot.config.paths import get_workspace_path
from nanobot.config.schema import Base
MSTEAMS_AVAILABLE = importlib.util.find_spec("jwt") is not None
MSTEAMS_AVAILABLE = (
importlib.util.find_spec("jwt") is not None
and importlib.util.find_spec("cryptography") is not None
)
if TYPE_CHECKING:
import jwt
@@ -53,12 +57,7 @@ class MSTeamsConfig(Base):
allow_from: list[str] = Field(default_factory=list)
reply_in_thread: bool = True
mention_only_response: str = "Hi — what can I help with?"
validate_inbound_auth: bool = False
restart_notify_enabled: bool = False
restart_notify_pre_message: str = (
"Nanobot agent initiated a gateway restart. I will message again when the gateway is back online."
)
restart_notify_post_message: str = "Nanobot gateway is back online."
validate_inbound_auth: bool = True
@dataclass
@@ -115,6 +114,13 @@ class MSTeamsChannel(BaseChannel):
logger.error("MSTeams app_id/app_password not configured")
return
if not self.config.validate_inbound_auth:
logger.warning(
"MSTeams inbound auth validation was explicitly DISABLED in config. "
"Anyone who knows the webhook URL can send messages as any user. "
"Only disable this for local development or controlled testing."
)
self._loop = asyncio.get_running_loop()
self._http = httpx.AsyncClient(timeout=30.0)
self._running = True
@@ -268,6 +274,14 @@ class MSTeamsChannel(BaseChannel):
logger.debug("MSTeams ignoring empty message after Teams text sanitization")
return
if not self.is_allowed(sender_id):
logger.warning(
"Access denied for sender {} on channel {}. "
"Add them to allowFrom list in config to grant access.",
sender_id, self.name,
)
return
self._conversation_refs[conversation_id] = ConversationRef(
service_url=service_url,
conversation_id=conversation_id,
@@ -305,7 +319,7 @@ class MSTeamsChannel(BaseChannel):
while preview_lines and not preview_lines[0]:
preview_lines.pop(0)
first_line = preview_lines[0] if preview_lines else ""
looks_like_quote_wrapper = first_line.lower().startswith("replying to ") or first_line.startswith("FWDIOC-BOT")
looks_like_quote_wrapper = first_line.lower().startswith("replying to ") or first_line.startswith("Reply wrapper")
if reply_to_id or channel_data.get("messageType") == "reply" or looks_like_quote_wrapper:
text = self._normalize_teams_reply_quote(text)
@@ -338,10 +352,10 @@ class MSTeamsChannel(BaseChannel):
reply = "\n".join(lines[1:]).strip()
return self._format_reply_with_quote(quoted, reply)
# Observed FWDIOC relay wrapper where the quoted content is surfaced after a
# synthetic "FWDIOC-BOT" header, sometimes with a blank line separating quote
# Observed reply wrapper where the quoted content is surfaced after a
# synthetic "Reply wrapper" header, sometimes with a blank line separating quote
# and reply, and sometimes as a compact line-based fallback shape.
if lines and lines[0].strip().startswith("FWDIOC-BOT"):
if lines and lines[0].strip().startswith("Reply wrapper"):
body = normalized_newlines.split("\n", 1)[1] if "\n" in normalized_newlines else ""
body = body.lstrip()
parts = re.split(r"\n\s*\n", body, maxsplit=1)
@@ -358,22 +372,24 @@ class MSTeamsChannel(BaseChannel):
if quoted and reply:
return self._format_reply_with_quote(quoted, reply)
# Observed compact fallback where the relay flattens everything into one line
# and appends the literal reply text marker at the end.
# Observed compact fallback where the relay flattens quote and reply into
# a single line after the synthetic Reply wrapper prefix.
compact = re.sub(r"\s+", " ", normalized_newlines).strip()
if compact.startswith("FWDIOC-BOT "):
compact = compact[len("FWDIOC-BOT ") :].strip()
marker = " Reply with quote test"
if compact.endswith(marker):
quoted = compact[: -len(marker)].strip()
reply = marker.strip()
return self._format_reply_with_quote(quoted, reply)
if compact.startswith("Reply wrapper "):
compact = compact[len("Reply wrapper ") :].strip()
for boundary in (". ", "! ", "? ", ""):
idx = compact.rfind(boundary)
if idx == -1:
continue
quoted = compact[: idx + 1].strip()
reply = compact[idx + len(boundary) :].strip()
if quoted and reply and len(reply) <= 160:
return self._format_reply_with_quote(quoted, reply)
return cleaned
def _format_reply_with_quote(self, quoted: str, reply: str) -> str:
"""Format a quoted reply for the model without Teams wrapper noise."""
"""Format a reply-with-context message for the model without Teams wrapper noise."""
quoted = quoted.strip()
reply = reply.strip()
if quoted and reply:
@@ -426,7 +442,6 @@ class MSTeamsChannel(BaseChannel):
async def _get_botframework_openid_config(self) -> dict[str, Any]:
"""Fetch and cache Bot Framework OpenID configuration."""
import time
now = time.time()
if self._botframework_openid_config and now < self._botframework_openid_config_expires_at:
@@ -443,7 +458,6 @@ class MSTeamsChannel(BaseChannel):
async def _get_botframework_jwks(self) -> dict[str, Any]:
"""Fetch and cache Bot Framework JWKS."""
import time
now = time.time()
if self._botframework_jwks and now < self._botframework_jwks_expires_at:
@@ -497,7 +511,6 @@ class MSTeamsChannel(BaseChannel):
async def _get_access_token(self) -> str:
"""Fetch an access token for Bot Framework / Azure Bot auth."""
import time
now = time.time()
if self._token and now < self._token_expires_at - 60:
+8
View File
@@ -280,6 +280,9 @@ class QQChannel(BaseChannel):
msg_id=msg_id,
content=msg.content.strip(),
)
except (aiohttp.ClientError, OSError):
# Network / transport errors — propagate so ChannelManager can retry
raise
except Exception:
logger.exception("Error sending QQ message to chat_id={}", msg.chat_id)
@@ -362,7 +365,12 @@ class QQChannel(BaseChannel):
logger.info("QQ media sent: {}", filename)
return True
except (aiohttp.ClientError, OSError) as e:
# Network / transport errors — propagate for retry by caller
logger.warning("QQ send media network error filename={} err={}", filename, e)
raise
except Exception as e:
# API-level or other non-network errors — return False so send() can fallback
logger.error("QQ send media failed filename={} err={}", filename, e)
return False
+126 -6
View File
@@ -5,6 +5,7 @@ import re
from typing import Any
from loguru import logger
from pydantic import Field
from slack_sdk.socket_mode.request import SocketModeRequest
from slack_sdk.socket_mode.response import SocketModeResponse
from slack_sdk.socket_mode.websockets import SocketModeClient
@@ -13,8 +14,6 @@ from slackify_markdown import slackify_markdown
from nanobot.bus.events import OutboundMessage
from nanobot.bus.queue import MessageBus
from pydantic import Field
from nanobot.channels.base import BaseChannel
from nanobot.config.schema import Base
@@ -50,6 +49,9 @@ class SlackChannel(BaseChannel):
name = "slack"
display_name = "Slack"
_SLACK_ID_RE = re.compile(r"^[CDGUW][A-Z0-9]{2,}$")
_SLACK_CHANNEL_REF_RE = re.compile(r"^<#([A-Z0-9]+)(?:\|[^>]+)?>$")
_SLACK_USER_REF_RE = re.compile(r"^<@([A-Z0-9]+)(?:\|[^>]+)?>$")
@classmethod
def default_config(cls) -> dict[str, Any]:
@@ -63,6 +65,7 @@ class SlackChannel(BaseChannel):
self._web_client: AsyncWebClient | None = None
self._socket_client: SocketModeClient | None = None
self._bot_user_id: str | None = None
self._target_cache: dict[str, str] = {}
async def start(self) -> None:
"""Start the Slack Socket Mode client."""
@@ -113,17 +116,23 @@ class SlackChannel(BaseChannel):
logger.warning("Slack client not running")
return
try:
target_chat_id = await self._resolve_target_chat_id(msg.chat_id)
slack_meta = msg.metadata.get("slack", {}) if msg.metadata else {}
thread_ts = slack_meta.get("thread_ts")
channel_type = slack_meta.get("channel_type")
origin_chat_id = str((slack_meta.get("event", {}) or {}).get("channel") or msg.chat_id)
# Slack DMs don't use threads; channel/group replies may keep thread_ts.
thread_ts_param = thread_ts if thread_ts and channel_type != "im" else None
thread_ts_param = (
thread_ts
if thread_ts and channel_type != "im" and target_chat_id == origin_chat_id
else None
)
# Slack rejects empty text payloads. Keep media-only messages media-only,
# but send a single blank message when the bot has no text or files to send.
if msg.content or not (msg.media or []):
await self._web_client.chat_postMessage(
channel=msg.chat_id,
channel=target_chat_id,
text=self._to_mrkdwn(msg.content) if msg.content else " ",
thread_ts=thread_ts_param,
)
@@ -131,7 +140,7 @@ class SlackChannel(BaseChannel):
for media_path in msg.media or []:
try:
await self._web_client.files_upload_v2(
channel=msg.chat_id,
channel=target_chat_id,
file=media_path,
thread_ts=thread_ts_param,
)
@@ -141,12 +150,123 @@ class SlackChannel(BaseChannel):
# Update reaction emoji when the final (non-progress) response is sent
if not (msg.metadata or {}).get("_progress"):
event = slack_meta.get("event", {})
await self._update_react_emoji(msg.chat_id, event.get("ts"))
await self._update_react_emoji(origin_chat_id, event.get("ts"))
except Exception as e:
logger.error("Error sending Slack message: {}", e)
raise
async def _resolve_target_chat_id(self, target: str) -> str:
"""Resolve human-friendly Slack targets to concrete IDs when needed."""
if not self._web_client:
return target
target = target.strip()
if not target:
return target
if match := self._SLACK_CHANNEL_REF_RE.fullmatch(target):
return match.group(1)
if match := self._SLACK_USER_REF_RE.fullmatch(target):
return await self._open_dm_for_user(match.group(1))
if self._SLACK_ID_RE.fullmatch(target):
if target.startswith(("U", "W")):
return await self._open_dm_for_user(target)
return target
if target.startswith("#"):
return await self._resolve_channel_name(target[1:])
if target.startswith("@"):
return await self._resolve_user_handle(target[1:])
try:
return await self._resolve_channel_name(target)
except ValueError:
return await self._resolve_user_handle(target)
async def _resolve_channel_name(self, name: str) -> str:
normalized = self._normalize_target_name(name)
if not normalized:
raise ValueError("Slack target channel name is empty")
cache_key = f"channel:{normalized}"
if cache_key in self._target_cache:
return self._target_cache[cache_key]
cursor: str | None = None
while True:
response = await self._web_client.conversations_list(
types="public_channel,private_channel",
exclude_archived=True,
limit=200,
cursor=cursor,
)
for channel in response.get("channels", []):
if self._normalize_target_name(str(channel.get("name") or "")) == normalized:
channel_id = str(channel.get("id") or "")
if channel_id:
self._target_cache[cache_key] = channel_id
return channel_id
cursor = ((response.get("response_metadata") or {}).get("next_cursor") or "").strip()
if not cursor:
break
raise ValueError(
f"Slack channel '{name}' was not found. Use a joined channel name like "
f"'#general' or a concrete channel ID."
)
async def _resolve_user_handle(self, handle: str) -> str:
normalized = self._normalize_target_name(handle)
if not normalized:
raise ValueError("Slack target user handle is empty")
cache_key = f"user:{normalized}"
if cache_key in self._target_cache:
return self._target_cache[cache_key]
cursor: str | None = None
while True:
response = await self._web_client.users_list(limit=200, cursor=cursor)
for member in response.get("members", []):
if self._member_matches_handle(member, normalized):
user_id = str(member.get("id") or "")
if not user_id:
continue
dm_id = await self._open_dm_for_user(user_id)
self._target_cache[cache_key] = dm_id
return dm_id
cursor = ((response.get("response_metadata") or {}).get("next_cursor") or "").strip()
if not cursor:
break
raise ValueError(
f"Slack user '{handle}' was not found. Use '@name' or a concrete DM/channel ID."
)
async def _open_dm_for_user(self, user_id: str) -> str:
response = await self._web_client.conversations_open(users=user_id)
channel_id = str(((response.get("channel") or {}).get("id")) or "")
if not channel_id:
raise ValueError(f"Slack DM target for user '{user_id}' could not be opened.")
return channel_id
@staticmethod
def _normalize_target_name(value: str) -> str:
return value.strip().lstrip("#@").lower()
@classmethod
def _member_matches_handle(cls, member: dict[str, Any], normalized: str) -> bool:
profile = member.get("profile") or {}
candidates = {
str(member.get("name") or ""),
str(profile.get("display_name") or ""),
str(profile.get("display_name_normalized") or ""),
str(profile.get("real_name") or ""),
str(profile.get("real_name_normalized") or ""),
}
return normalized in {cls._normalize_target_name(candidate) for candidate in candidates if candidate}
async def _on_socket_request(
self,
client: SocketModeClient,
+19 -7
View File
@@ -166,6 +166,7 @@ def _markdown_to_telegram_html(text: str) -> str:
_SEND_MAX_RETRIES = 3
_SEND_RETRY_BASE_DELAY = 0.5 # seconds, doubled each retry
_STREAM_EDIT_INTERVAL_DEFAULT = 0.6 # min seconds between edit_message_text calls
@dataclass
@@ -190,6 +191,7 @@ class TelegramConfig(Base):
connection_pool_size: int = 32
pool_timeout: float = 5.0
streaming: bool = True
stream_edit_interval: float = Field(default=_STREAM_EDIT_INTERVAL_DEFAULT, ge=0.1)
class TelegramChannel(BaseChannel):
@@ -219,8 +221,6 @@ class TelegramChannel(BaseChannel):
def default_config(cls) -> dict[str, Any]:
return TelegramConfig().model_dump(by_alias=True)
_STREAM_EDIT_INTERVAL = 0.6 # min seconds between edit_message_text calls
def __init__(self, config: Any, bus: MessageBus):
if isinstance(config, dict):
config = TelegramConfig.model_validate(config)
@@ -316,10 +316,10 @@ class TelegramChannel(BaseChannel):
)
self._app.add_handler(MessageHandler(filters.Regex(r"^/help(?:@\w+)?$"), self._on_help))
# Add message handler for text, photos, voice, documents
# Add message handler for text, photos, voice, documents, and locations
self._app.add_handler(
MessageHandler(
(filters.TEXT | filters.PHOTO | filters.VOICE | filters.AUDIO | filters.Document.ALL)
(filters.TEXT | filters.PHOTO | filters.VOICE | filters.AUDIO | filters.Document.ALL | filters.LOCATION)
& ~filters.COMMAND,
self._on_message
)
@@ -520,7 +520,10 @@ class TelegramChannel(BaseChannel):
reply_parameters=reply_params,
**(thread_kwargs or {}),
)
except Exception as e:
except BadRequest as e:
# Only fall back to plain text on actual HTML parse/format errors.
# Network errors (TimedOut, NetworkError) should propagate immediately
# to avoid doubling connection demand during pool exhaustion.
logger.warning("HTML parse failed, falling back to plain text: {}", e)
try:
await self._call_with_retry(
@@ -567,7 +570,10 @@ class TelegramChannel(BaseChannel):
chat_id=int_chat_id, message_id=buf.message_id,
text=html, parse_mode="HTML",
)
except Exception as e:
except BadRequest as e:
# Only fall back to plain text on actual HTML parse/format errors.
# Network errors (TimedOut, NetworkError) should propagate immediately
# to avoid doubling connection demand during pool exhaustion.
if self._is_not_modified_error(e):
logger.debug("Final stream edit already applied for {}", chat_id)
self._stream_bufs.pop(chat_id, None)
@@ -619,7 +625,7 @@ class TelegramChannel(BaseChannel):
except Exception as e:
logger.warning("Stream initial send failed: {}", e)
raise # Let ChannelManager handle retry
elif (now - buf.last_edit) >= self._STREAM_EDIT_INTERVAL:
elif (now - buf.last_edit) >= self.config.stream_edit_interval:
try:
await self._call_with_retry(
self._app.bot.edit_message_text,
@@ -884,6 +890,12 @@ class TelegramChannel(BaseChannel):
if message.caption:
content_parts.append(message.caption)
# Location content
if message.location:
lat = message.location.latitude
lon = message.location.longitude
content_parts.append(f"[location: {lat}, {lon}]")
# Download current message media
current_media_paths, current_media_parts = await self._download_message_media(
message, add_failure_content=True
+11 -2
View File
@@ -302,13 +302,22 @@ class WecomChannel(BaseChannel):
elif msg_type == "mixed":
# Mixed content contains multiple message items
msg_items = body.get("mixed", {}).get("item", [])
msg_items = body.get("mixed", {}).get("msg_item", [])
for item in msg_items:
item_type = item.get("type", "")
item_type = item.get("msgtype", "")
if item_type == "text":
text = item.get("text", {}).get("content", "")
if text:
content_parts.append(text)
elif item_type == "image":
file_url = item.get("image", {}).get("url", "")
aes_key = item.get("image", {}).get("aeskey", "")
if file_url and aes_key:
file_path = await self._download_and_save_media(file_url, aes_key, "image")
if file_path:
filename = os.path.basename(file_path)
content_parts.append(f"[image: {filename}]")
media_paths.append(file_path)
else:
content_parts.append(MSG_TYPE_MAP.get(item_type, f"[{item_type}]"))
+36
View File
@@ -985,7 +985,43 @@ class WeixinChannel(BaseChannel):
for media_path in (msg.media or []):
try:
await self._send_media_file(msg.chat_id, media_path, ctx_token)
except (httpx.TimeoutException, httpx.TransportError) as net_err:
# Network/transport errors: do NOT fall back to text —
# the text send would also likely fail, and the outer
# except will re-raise so ChannelManager retries properly.
logger.error(
"Network error sending WeChat media {}: {}",
media_path,
net_err,
)
raise
except httpx.HTTPStatusError as http_err:
status_code = (
http_err.response.status_code
if http_err.response is not None
else 0
)
if status_code >= 500:
# Server-side / retryable HTTP error — same as network.
logger.error(
"Server error ({} {}) sending WeChat media {}: {}",
status_code,
http_err.response.reason_phrase
if http_err.response is not None
else "",
media_path,
http_err,
)
raise
# 4xx client errors are NOT retryable — fall back to text.
filename = Path(media_path).name
logger.error("Failed to send WeChat media {}: {}", media_path, http_err)
await self._send_text(
msg.chat_id, f"[Failed to send: {filename}]", ctx_token,
)
except Exception as e:
# Non-network errors (format, file-not-found, etc.):
# notify the user via text fallback.
filename = Path(media_path).name
logger.error("Failed to send WeChat media {}: {}", media_path, e)
# Notify user about failure via text
+60 -4
View File
@@ -590,7 +590,11 @@ def serve(
mcp_servers=runtime_config.tools.mcp_servers,
channels_config=runtime_config.channels,
timezone=runtime_config.agents.defaults.timezone,
unified_session=runtime_config.agents.defaults.unified_session,
disabled_skills=runtime_config.agents.defaults.disabled_skills,
session_ttl_minutes=runtime_config.agents.defaults.session_ttl_minutes,
consolidation_ratio=runtime_config.agents.defaults.consolidation_ratio,
tools_config=runtime_config.tools,
)
model_name = runtime_config.agents.defaults.model
@@ -682,7 +686,11 @@ def gateway(
mcp_servers=config.tools.mcp_servers,
channels_config=config.channels,
timezone=config.agents.defaults.timezone,
unified_session=config.agents.defaults.unified_session,
disabled_skills=config.agents.defaults.disabled_skills,
session_ttl_minutes=config.agents.defaults.session_ttl_minutes,
consolidation_ratio=config.agents.defaults.consolidation_ratio,
tools_config=config.tools,
)
# Set cron callback (needs agent)
@@ -725,7 +733,7 @@ def gateway(
response = resp.content if resp else ""
message_tool = agent.tools.get("message")
if isinstance(message_tool, MessageTool) and message_tool._sent_in_turn:
if job.payload.deliver and isinstance(message_tool, MessageTool) and message_tool._sent_in_turn:
return response
if job.payload.deliver and job.payload.to and response:
@@ -817,12 +825,55 @@ def gateway(
console.print(f"[green]✓[/green] Heartbeat: every {hb_cfg.interval_s}s")
async def _health_server(host: str, health_port: int):
"""Lightweight HTTP health endpoint on the gateway port."""
import json as _json
async def handle(reader, writer):
try:
data = await asyncio.wait_for(reader.read(4096), timeout=5)
except (asyncio.TimeoutError, ConnectionError):
writer.close()
return
request_line = data.split(b"\r\n", 1)[0].decode("utf-8", errors="replace")
method, path = "", ""
parts = request_line.split(" ")
if len(parts) >= 2:
method, path = parts[0], parts[1]
if method == "GET" and path == "/health":
body = _json.dumps({"status": "ok"})
resp = (
f"HTTP/1.0 200 OK\r\n"
f"Content-Type: application/json\r\n"
f"Content-Length: {len(body)}\r\n"
f"\r\n{body}"
)
else:
body = "Not Found"
resp = (
f"HTTP/1.0 404 Not Found\r\n"
f"Content-Type: text/plain\r\n"
f"Content-Length: {len(body)}\r\n"
f"\r\n{body}"
)
writer.write(resp.encode())
await writer.drain()
writer.close()
server = await asyncio.start_server(handle, host, health_port)
console.print(f"[green]✓[/green] Health endpoint: http://{host}:{health_port}/health")
async with server:
await server.serve_forever()
# Register Dream system job (always-on, idempotent on restart)
dream_cfg = config.agents.defaults.dream
if dream_cfg.model_override:
agent.dream.model = dream_cfg.model_override
agent.dream.max_batch_size = dream_cfg.max_batch_size
agent.dream.max_iterations = dream_cfg.max_iterations
agent.dream.annotate_line_ages = dream_cfg.annotate_line_ages
from nanobot.cron.types import CronJob, CronPayload
cron.register_system_job(CronJob(
id="dream",
@@ -839,6 +890,7 @@ def gateway(
await asyncio.gather(
agent.run(),
channels.start_all(),
_health_server(config.gateway.host, port),
)
except KeyboardInterrupt:
console.print("\nShutting down...")
@@ -914,7 +966,11 @@ def agent(
mcp_servers=config.tools.mcp_servers,
channels_config=config.channels,
timezone=config.agents.defaults.timezone,
unified_session=config.agents.defaults.unified_session,
disabled_skills=config.agents.defaults.disabled_skills,
session_ttl_minutes=config.agents.defaults.session_ttl_minutes,
consolidation_ratio=config.agents.defaults.consolidation_ratio,
tools_config=config.tools,
)
restart_notice = consume_restart_notice_from_env()
if restart_notice and should_show_cli_restart_notice(restart_notice, session_id):
@@ -958,7 +1014,7 @@ def agent(
# Interactive mode — route through bus like other channels
from nanobot.bus.events import InboundMessage
_init_prompt_session()
console.print(f"{__logo__} Interactive mode (type [bold]exit[/bold] or [bold]Ctrl+C[/bold] to quit)\n")
console.print(f"{__logo__} Interactive mode [bold blue]({config.agents.defaults.model})[/bold blue] — type [bold]exit[/bold] or [bold]Ctrl+C[/bold] to quit\n")
if ":" in session_id:
cli_channel, cli_chat_id = session_id.split(":", 1)
@@ -1119,7 +1175,7 @@ def channels_status(
table = Table(title="Channel Status")
table.add_column("Channel", style="cyan")
table.add_column("Enabled", style="green")
table.add_column("Enabled")
for name, cls in sorted(discover_all().items()):
section = getattr(config.channels, name, None)
@@ -1254,7 +1310,7 @@ def plugins_list():
table = Table(title="Channel Plugins")
table.add_column("Name", style="cyan")
table.add_column("Source", style="magenta")
table.add_column("Enabled", style="green")
table.add_column("Enabled")
for name in sorted(all_channels):
cls = all_channels[name]
+1 -1
View File
@@ -102,7 +102,7 @@ class StreamRenderer:
self._live = Live(self._render(), console=c, auto_refresh=False)
self._live.start()
now = time.monotonic()
if "\n" in delta or (now - self._t) > 0.05:
if (now - self._t) > 0.15:
self._live.update(self._render())
self._live.refresh()
self._t = now
+10
View File
@@ -74,6 +74,12 @@ async def cmd_status(ctx: CommandContext) -> OutboundMessage:
search_usage_text = usage.format()
except Exception:
pass # Never let usage fetch break /status
active_tasks = loop._active_tasks.get(ctx.key, [])
task_count = sum(1 for t in active_tasks if not t.done())
try:
task_count += loop.subagents.get_running_count_by_session(ctx.key)
except Exception:
pass
return OutboundMessage(
channel=ctx.msg.channel,
chat_id=ctx.msg.chat_id,
@@ -84,6 +90,10 @@ async def cmd_status(ctx: CommandContext) -> OutboundMessage:
session_msg_count=len(session.get_history(max_messages=0)),
context_tokens_estimate=ctx_est,
search_usage_text=search_usage_text,
active_task_count=task_count,
max_completion_tokens=getattr(
getattr(loop.provider, "generation", None), "max_tokens", 8192
),
),
metadata={**dict(ctx.msg.metadata or {}), "render_as": "text"},
)
+15
View File
@@ -117,4 +117,19 @@ def _migrate_config(data: dict) -> dict:
exec_cfg = tools.get("exec", {})
if "restrictToWorkspace" in exec_cfg and "restrictToWorkspace" not in tools:
tools["restrictToWorkspace"] = exec_cfg.pop("restrictToWorkspace")
# Move tools.myEnabled / tools.mySet → tools.my.{enable, allowSet}.
# The old flat keys shipped in the initial MyTool landing; wrapping them in a
# sub-config keeps `web` / `exec` / `my` symmetric and gives room to grow.
if "myEnabled" in tools or "mySet" in tools:
my_cfg = tools.setdefault("my", {})
if "myEnabled" in tools and "enable" not in my_cfg:
my_cfg["enable"] = tools.pop("myEnabled")
else:
tools.pop("myEnabled", None)
if "mySet" in tools and "allowSet" not in my_cfg:
my_cfg["allowSet"] = tools.pop("mySet")
else:
tools.pop("mySet", None)
return data
+36 -6
View File
@@ -43,7 +43,12 @@ class DreamConfig(Base):
validation_alias=AliasChoices("modelOverride", "model", "model_override"),
) # Optional Dream-specific model override
max_batch_size: int = Field(default=20, ge=1) # Max history entries per run
max_iterations: int = Field(default=10, ge=1) # Max tool calls per Phase 2
# Bumped from 10 to 15 in #3212 (exp002: +30% dedup, no accuracy loss; >15 plateaus).
max_iterations: int = Field(default=15, ge=1) # Max tool calls per Phase 2
# Per-line git-blame age annotation in Phase 1 prompt (see #3212). Default
# on — set to False to feed MEMORY.md raw if a specific LLM reacts poorly
# to the `← Nd` suffix or you want deterministic, git-independent prompts.
annotate_line_ages: bool = True
def build_schedule(self, timezone: str) -> CronSchedule:
"""Build the runtime schedule, preferring the legacy cron override if present."""
@@ -74,9 +79,23 @@ class AgentDefaults(Base):
max_tool_iterations: int = 200
max_tool_result_chars: int = 16_000
provider_retry_mode: Literal["standard", "persistent"] = "standard"
reasoning_effort: str | None = None # low / medium / high - enables LLM thinking mode
reasoning_effort: str | None = None # low / medium / high / adaptive - enables LLM thinking mode
timezone: str = "UTC" # IANA timezone, e.g. "Asia/Shanghai", "America/New_York"
session_ttl_minutes: int = Field(default=0, ge=0) # Auto /new after idle (0 = disabled)
unified_session: bool = False # Share one session across all channels (single-user multi-device)
disabled_skills: list[str] = Field(default_factory=list) # Skill names to exclude from loading (e.g. ["summarize", "skill-creator"])
session_ttl_minutes: int = Field(
default=0,
ge=0,
validation_alias=AliasChoices("idleCompactAfterMinutes", "sessionTtlMinutes"),
serialization_alias="idleCompactAfterMinutes",
) # Auto-compact idle threshold in minutes (0 = disabled)
consolidation_ratio: float = Field(
default=0.5,
ge=0.1,
le=0.95,
validation_alias=AliasChoices("consolidationRatio"),
serialization_alias="consolidationRatio",
) # Consolidation target ratio (0.5 = 50% of budget retained after compression)
dream: DreamConfig = Field(default_factory=DreamConfig)
@@ -89,7 +108,7 @@ class AgentsConfig(Base):
class ProviderConfig(Base):
"""LLM provider configuration."""
api_key: str = ""
api_key: str | None = None
api_base: str | None = None
extra_headers: dict[str, str] | None = None # Custom headers (e.g. APP-Code for AiHubMix)
@@ -108,10 +127,12 @@ class ProvidersConfig(Base):
dashscope: ProviderConfig = Field(default_factory=ProviderConfig)
vllm: ProviderConfig = Field(default_factory=ProviderConfig)
ollama: ProviderConfig = Field(default_factory=ProviderConfig) # Ollama local models
lm_studio: ProviderConfig = Field(default_factory=ProviderConfig) # LM Studio local models
ovms: ProviderConfig = Field(default_factory=ProviderConfig) # OpenVINO Model Server (OVMS)
gemini: ProviderConfig = Field(default_factory=ProviderConfig)
moonshot: ProviderConfig = Field(default_factory=ProviderConfig)
minimax: ProviderConfig = Field(default_factory=ProviderConfig)
minimax_anthropic: ProviderConfig = Field(default_factory=ProviderConfig) # MiniMax Anthropic endpoint (thinking)
mistral: ProviderConfig = Field(default_factory=ProviderConfig)
stepfun: ProviderConfig = Field(default_factory=ProviderConfig) # Step Fun (阶跃星辰)
xiaomi_mimo: ProviderConfig = Field(default_factory=ProviderConfig) # Xiaomi MIMO (小米)
@@ -145,7 +166,7 @@ class ApiConfig(Base):
class GatewayConfig(Base):
"""Gateway/server configuration."""
host: str = "0.0.0.0"
host: str = "127.0.0.1" # Safer default: local-only bind.
port: int = 18790
heartbeat: HeartbeatConfig = Field(default_factory=HeartbeatConfig)
@@ -153,7 +174,7 @@ class GatewayConfig(Base):
class WebSearchConfig(Base):
"""Web search tool configuration."""
provider: str = "duckduckgo" # brave, tavily, duckduckgo, searxng, jina
provider: str = "duckduckgo" # brave, tavily, duckduckgo, searxng, jina, kagi
api_key: str = ""
base_url: str = "" # SearXNG base URL
max_results: int = 5
@@ -177,6 +198,7 @@ class ExecToolConfig(Base):
timeout: int = 60
path_append: str = ""
sandbox: str = "" # sandbox backend: "" (none) or "bwrap"
allowed_env_keys: list[str] = Field(default_factory=list) # Env var names to pass through to subprocess (e.g. ["GOPATH", "JAVA_HOME"])
class MCPServerConfig(Base):
"""MCP server connection configuration (stdio or HTTP)."""
@@ -190,11 +212,19 @@ class MCPServerConfig(Base):
tool_timeout: int = 30 # seconds before a tool call is cancelled
enabled_tools: list[str] = Field(default_factory=lambda: ["*"]) # Only register these tools; accepts raw MCP names or wrapped mcp_<server>_<tool> names; ["*"] = all tools; [] = no tools
class MyToolConfig(Base):
"""Self-inspection tool configuration."""
enable: bool = True # register the `my` tool (agent runtime state inspection)
allow_set: bool = False # let `my` modify loop state (read-only if False)
class ToolsConfig(Base):
"""Tools configuration."""
web: WebToolsConfig = Field(default_factory=WebToolsConfig)
exec: ExecToolConfig = Field(default_factory=ExecToolConfig)
my: MyToolConfig = Field(default_factory=MyToolConfig)
restrict_to_workspace: bool = False # restrict all tool access to workspace directory
mcp_servers: dict[str, MCPServerConfig] = Field(default_factory=dict)
ssrf_whitelist: list[str] = Field(default_factory=list) # CIDR ranges to exempt from SSRF blocking (e.g. ["100.64.0.0/10"] for Tailscale)
+173 -47
View File
@@ -4,10 +4,12 @@ import asyncio
import json
import time
import uuid
from dataclasses import asdict
from datetime import datetime
from pathlib import Path
from typing import Any, Callable, Coroutine, Literal
from filelock import FileLock
from loguru import logger
from nanobot.cron.types import CronJob, CronJobState, CronPayload, CronRunRecord, CronSchedule, CronStore
@@ -69,28 +71,26 @@ class CronService:
self,
store_path: Path,
on_job: Callable[[CronJob], Coroutine[Any, Any, str | None]] | None = None,
max_sleep_ms: int = 300_000, # 5 minutes
):
self.store_path = store_path
self._action_path = store_path.parent / "action.jsonl"
self._lock = FileLock(str(self._action_path.parent) + ".lock")
self.on_job = on_job
self._store: CronStore | None = None
self._last_mtime: float = 0.0
self._timer_task: asyncio.Task | None = None
self._running = False
self._timer_active = False
self.max_sleep_ms = max_sleep_ms
def _load_store(self) -> CronStore:
"""Load jobs from disk. Reloads automatically if file was modified externally."""
if self._store and self.store_path.exists():
mtime = self.store_path.stat().st_mtime
if mtime != self._last_mtime:
logger.info("Cron: jobs.json modified externally, reloading")
self._store = None
if self._store:
return self._store
def _load_jobs(self) -> tuple[list[CronJob], int]:
jobs = []
version = 1
if self.store_path.exists():
try:
data = json.loads(self.store_path.read_text(encoding="utf-8"))
jobs = []
version = data.get("version", 1)
for j in data.get("jobs", []):
jobs.append(CronJob(
id=j["id"],
@@ -129,12 +129,57 @@ class CronService:
updated_at_ms=j.get("updatedAtMs", 0),
delete_after_run=j.get("deleteAfterRun", False),
))
self._store = CronStore(jobs=jobs)
except Exception as e:
logger.warning("Failed to load cron store: {}", e)
self._store = CronStore()
else:
self._store = CronStore()
return jobs, version
def _merge_action(self):
if not self._action_path.exists():
return
jobs_map = {j.id: j for j in self._store.jobs}
def _update(params: dict):
j = CronJob.from_dict(params)
jobs_map[j.id] = j
def _del(params: dict):
if job_id := params.get("job_id"):
jobs_map.pop(job_id)
with self._lock:
with open(self._action_path, "r", encoding="utf-8") as f:
changed = False
for line in f:
try:
line = line.strip()
action = json.loads(line)
if "action" not in action:
continue
if action["action"] == "del":
_del(action.get("params", {}))
else:
_update(action.get("params", {}))
changed = True
except Exception as exp:
logger.debug(f"load action line error: {exp}")
continue
self._store.jobs = list(jobs_map.values())
if self._running and changed:
self._action_path.write_text("", encoding="utf-8")
self._save_store()
return
def _load_store(self) -> CronStore:
"""Load jobs from disk. Reloads automatically if file was modified externally.
- Reload every time because it needs to merge operations on the jobs object from other instances.
- During _on_timer execution, return the existing store to prevent concurrent
_load_store calls (e.g. from list_jobs polling) from replacing it mid-execution.
"""
if self._timer_active and self._store:
return self._store
jobs, version = self._load_jobs()
self._store = CronStore(version=version, jobs=jobs)
self._merge_action()
return self._store
@@ -190,8 +235,7 @@ class CronService:
}
self.store_path.write_text(json.dumps(data, indent=2, ensure_ascii=False), encoding="utf-8")
self._last_mtime = self.store_path.stat().st_mtime
async def start(self) -> None:
"""Start the cron service."""
self._running = True
@@ -230,11 +274,14 @@ class CronService:
if self._timer_task:
self._timer_task.cancel()
next_wake = self._get_next_wake_ms()
if not next_wake or not self._running:
if not self._running:
return
delay_ms = max(0, next_wake - _now_ms())
next_wake = self._get_next_wake_ms()
if next_wake is None:
delay_ms = self.max_sleep_ms
else:
delay_ms = min(self.max_sleep_ms, max(0, next_wake - _now_ms()))
delay_s = delay_ms / 1000
async def tick():
@@ -248,18 +295,23 @@ class CronService:
"""Handle timer tick - run due jobs."""
self._load_store()
if not self._store:
self._arm_timer()
return
now = _now_ms()
due_jobs = [
j for j in self._store.jobs
if j.enabled and j.state.next_run_at_ms and now >= j.state.next_run_at_ms
]
self._timer_active = True
try:
now = _now_ms()
due_jobs = [
j for j in self._store.jobs
if j.enabled and j.state.next_run_at_ms and now >= j.state.next_run_at_ms
]
for job in due_jobs:
await self._execute_job(job)
for job in due_jobs:
await self._execute_job(job)
self._save_store()
self._save_store()
finally:
self._timer_active = False
self._arm_timer()
async def _execute_job(self, job: CronJob) -> None:
@@ -303,6 +355,13 @@ class CronService:
# Compute next run
job.state.next_run_at_ms = _compute_next_run(job.schedule, _now_ms())
def _append_action(self, action: Literal["add", "del", "update"], params: dict):
self.store_path.parent.mkdir(parents=True, exist_ok=True)
with self._lock:
with open(self._action_path, "a", encoding="utf-8") as f:
f.write(json.dumps({"action": action, "params": params}, ensure_ascii=False) + "\n")
# ========== Public API ==========
def list_jobs(self, include_disabled: bool = False) -> list[CronJob]:
@@ -322,7 +381,6 @@ class CronService:
delete_after_run: bool = False,
) -> CronJob:
"""Add a new job."""
store = self._load_store()
_validate_schedule_for_add(schedule)
now = _now_ms()
@@ -343,10 +401,13 @@ class CronService:
updated_at_ms=now,
delete_after_run=delete_after_run,
)
store.jobs.append(job)
self._save_store()
self._arm_timer()
if self._running:
store = self._load_store()
store.jobs.append(job)
self._save_store()
self._arm_timer()
else:
self._append_action("add", asdict(job))
logger.info("Cron: added job '{}' ({})", name, job.id)
return job
@@ -380,8 +441,11 @@ class CronService:
removed = len(store.jobs) < before
if removed:
self._save_store()
self._arm_timer()
if self._running:
self._save_store()
self._arm_timer()
else:
self._append_action("del", {"job_id": job_id})
logger.info("Cron: removed job {}", job_id)
return "removed"
@@ -398,23 +462,85 @@ class CronService:
job.state.next_run_at_ms = _compute_next_run(job.schedule, _now_ms())
else:
job.state.next_run_at_ms = None
self._save_store()
self._arm_timer()
if self._running:
self._save_store()
self._arm_timer()
else:
self._append_action("update", asdict(job))
return job
return None
async def run_job(self, job_id: str, force: bool = False) -> bool:
"""Manually run a job."""
def update_job(
self,
job_id: str,
*,
name: str | None = None,
schedule: CronSchedule | None = None,
message: str | None = None,
deliver: bool | None = None,
channel: str | None = ...,
to: str | None = ...,
delete_after_run: bool | None = None,
) -> CronJob | Literal["not_found", "protected"]:
"""Update mutable fields of an existing job. System jobs cannot be updated.
For ``channel`` and ``to``, pass an explicit value (including ``None``)
to update; omit (sentinel ``...``) to leave unchanged.
"""
store = self._load_store()
for job in store.jobs:
if job.id == job_id:
if not force and not job.enabled:
return False
await self._execute_job(job)
self._save_store()
job = next((j for j in store.jobs if j.id == job_id), None)
if job is None:
return "not_found"
if job.payload.kind == "system_event":
return "protected"
if schedule is not None:
_validate_schedule_for_add(schedule)
job.schedule = schedule
if name is not None:
job.name = name
if message is not None:
job.payload.message = message
if deliver is not None:
job.payload.deliver = deliver
if channel is not ...:
job.payload.channel = channel
if to is not ...:
job.payload.to = to
if delete_after_run is not None:
job.delete_after_run = delete_after_run
job.updated_at_ms = _now_ms()
if job.enabled:
job.state.next_run_at_ms = _compute_next_run(job.schedule, _now_ms())
if self._running:
self._save_store()
self._arm_timer()
else:
self._append_action("update", asdict(job))
logger.info("Cron: updated job '{}' ({})", job.name, job.id)
return job
async def run_job(self, job_id: str, force: bool = False) -> bool:
"""Manually run a job without disturbing the service's running state."""
was_running = self._running
self._running = True
try:
store = self._load_store()
for job in store.jobs:
if job.id == job_id:
if not force and not job.enabled:
return False
await self._execute_job(job)
self._save_store()
return True
return False
finally:
self._running = was_running
if was_running:
self._arm_timer()
return True
return False
def get_job(self, job_id: str) -> CronJob | None:
"""Get a job by ID."""
+12
View File
@@ -61,6 +61,18 @@ class CronJob:
updated_at_ms: int = 0
delete_after_run: bool = False
@classmethod
def from_dict(cls, kwargs: dict):
state_kwargs = dict(kwargs.get("state", {}))
state_kwargs["run_history"] = [
record if isinstance(record, CronRunRecord) else CronRunRecord(**record)
for record in state_kwargs.get("run_history", [])
]
kwargs["schedule"] = CronSchedule(**kwargs.get("schedule", {"kind": "every"}))
kwargs["payload"] = CronPayload(**kwargs.get("payload", {}))
kwargs["state"] = CronJobState(**state_kwargs)
return cls(**kwargs)
@dataclass
class CronStore:
+4
View File
@@ -81,7 +81,11 @@ class Nanobot:
restrict_to_workspace=config.tools.restrict_to_workspace,
mcp_servers=config.tools.mcp_servers,
timezone=defaults.timezone,
unified_session=defaults.unified_session,
disabled_skills=defaults.disabled_skills,
session_ttl_minutes=defaults.session_ttl_minutes,
consolidation_ratio=defaults.consolidation_ratio,
tools_config=config.tools,
)
return cls(loop)
+8 -2
View File
@@ -380,9 +380,15 @@ class AnthropicProvider(LLMProvider):
if system:
kwargs["system"] = system
if thinking_enabled:
if reasoning_effort == "adaptive":
# Adaptive thinking: model decides when and how much to think
# Supported on claude-sonnet-4-6 and claude-opus-4-6.
# Also auto-enables interleaved thinking between tool calls.
kwargs["thinking"] = {"type": "adaptive"}
kwargs["temperature"] = 1.0
elif thinking_enabled:
budget_map = {"low": 1024, "medium": 4096, "high": max(8192, max_tokens)}
budget = budget_map.get(reasoning_effort.lower(), 4096) # type: ignore[union-attr]
budget = budget_map.get(reasoning_effort.lower(), 4096)
kwargs["thinking"] = {"type": "enabled", "budget_tokens": budget}
kwargs["max_tokens"] = max(max_tokens, budget + 4096)
kwargs["temperature"] = 1.0
+105 -6
View File
@@ -353,6 +353,64 @@ class LLMProvider(ABC):
# Unknown 429 defaults to WAIT+retry.
return True
@staticmethod
def _enforce_role_alternation(messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""Merge consecutive same-role messages and drop trailing assistant messages.
Some providers (OpenAI-compat, Azure, vLLM, Ollama, etc.) reject requests
where the last message is 'assistant' (prefill not supported) or two
consecutive non-system messages share the same role.
"""
if not messages:
return messages
merged: list[dict[str, Any]] = []
for msg in messages:
role = msg.get("role")
if (
merged
and role != "system"
and role not in ("tool",)
and merged[-1].get("role") == role
and role in ("user", "assistant")
):
prev = merged[-1]
if role == "assistant":
prev_has_tools = bool(prev.get("tool_calls"))
curr_has_tools = bool(msg.get("tool_calls"))
if curr_has_tools:
merged[-1] = dict(msg)
continue
if prev_has_tools:
continue
prev_content = prev.get("content") or ""
curr_content = msg.get("content") or ""
if isinstance(prev_content, str) and isinstance(curr_content, str):
prev["content"] = (prev_content + "\n\n" + curr_content).strip()
else:
merged[-1] = dict(msg)
else:
merged.append(dict(msg))
last_popped = None
while merged and merged[-1].get("role") == "assistant":
last_popped = merged.pop()
# If removing trailing assistant messages left only system messages,
# the request would be invalid for most providers (e.g. Zhipu/GLM
# error 1214). Recover by converting the last popped assistant
# message to a user message so the LLM can still see the content.
if (
merged
and last_popped is not None
and not any(m.get("role") in ("user", "tool") for m in merged)
):
recovered = dict(last_popped)
recovered["role"] = "user"
merged.append(recovered)
return merged
@staticmethod
def _strip_image_content(messages: list[dict[str, Any]]) -> list[dict[str, Any]] | None:
"""Replace image_url blocks with text placeholder. Returns None if no images found."""
@@ -375,6 +433,26 @@ class LLMProvider(ABC):
result.append(msg)
return result if found else None
@staticmethod
def _strip_image_content_inplace(messages: list[dict[str, Any]]) -> bool:
"""Replace image_url blocks with text placeholder *in-place*.
Mutates the content lists of the original message dicts so that
callers holding references to those dicts also see the stripped
version.
"""
found = False
for msg in messages:
content = msg.get("content")
if isinstance(content, list):
for i, b in enumerate(content):
if isinstance(b, dict) and b.get("type") == "image_url":
path = (b.get("_meta") or {}).get("path", "")
placeholder = image_placeholder_text(path, empty="[image omitted]")
content[i] = {"type": "text", "text": placeholder}
found = True
return found
async def _safe_chat(self, **kwargs: Any) -> LLMResponse:
"""Call chat() and convert unexpected exceptions to error responses."""
try:
@@ -434,9 +512,9 @@ class LLMProvider(ABC):
on_retry_wait: Callable[[str], Awaitable[None]] | None = None,
) -> LLMResponse:
"""Call chat_stream() with retry on transient provider failures."""
if max_tokens is self._SENTINEL:
if max_tokens is self._SENTINEL or max_tokens is None:
max_tokens = self.generation.max_tokens
if temperature is self._SENTINEL:
if temperature is self._SENTINEL or temperature is None:
temperature = self.generation.temperature
if reasoning_effort is self._SENTINEL:
reasoning_effort = self.generation.reasoning_effort
@@ -471,11 +549,14 @@ class LLMProvider(ABC):
Parameters default to ``self.generation`` when not explicitly passed,
so callers no longer need to thread temperature / max_tokens /
reasoning_effort through every layer.
reasoning_effort through every layer. Explicit ``None`` is also
normalized to the provider's generation defaults so that downstream
``_build_kwargs`` never sees ``None`` for ``max_tokens`` / ``temperature``
(which would crash ``max(1, max_tokens)``).
"""
if max_tokens is self._SENTINEL:
if max_tokens is self._SENTINEL or max_tokens is None:
max_tokens = self.generation.max_tokens
if temperature is self._SENTINEL:
if temperature is self._SENTINEL or temperature is None:
temperature = self.generation.temperature
if reasoning_effort is self._SENTINEL:
reasoning_effort = self.generation.reasoning_effort
@@ -626,7 +707,12 @@ class LLMProvider(ABC):
)
retry_kw = dict(kw)
retry_kw["messages"] = stripped
return await call(**retry_kw)
result = await call(**retry_kw)
# Permanently strip images from the original messages so
# subsequent iterations do not repeat the error-retry cycle.
if result.finish_reason != "error":
self._strip_image_content_inplace(original_messages)
return result
return response
if persistent and identical_error_count >= self._PERSISTENT_IDENTICAL_ERROR_LIMIT:
@@ -635,9 +721,22 @@ class LLMProvider(ABC):
identical_error_count,
(response.content or "")[:120].lower(),
)
if on_retry_wait:
await on_retry_wait(
f"Persistent retry stopped after {identical_error_count} identical errors."
)
return response
if not persistent and attempt > len(delays):
logger.warning(
"LLM request failed after {} retries, giving up: {}",
attempt,
(response.content or "")[:120].lower(),
)
if on_retry_wait:
await on_retry_wait(
f"Model request failed after {attempt} retries, giving up."
)
break
base_delay = delays[min(attempt - 1, len(delays) - 1)]
+299 -15
View File
@@ -3,16 +3,19 @@
from __future__ import annotations
import asyncio
import json
import hashlib
import importlib.util
import os
import secrets
import string
import time
import uuid
from collections.abc import Awaitable, Callable
from typing import TYPE_CHECKING, Any
import json_repair
from loguru import logger
if os.environ.get("LANGFUSE_SECRET_KEY") and importlib.util.find_spec("langfuse"):
from langfuse.openai import AsyncOpenAI
@@ -26,6 +29,12 @@ else:
from openai import AsyncOpenAI
from nanobot.providers.base import LLMProvider, LLMResponse, ToolCallRequest
from nanobot.providers.openai_responses import (
consume_sdk_stream,
convert_messages,
convert_tools,
parse_response_output,
)
if TYPE_CHECKING:
from nanobot.providers.registry import ProviderSpec
@@ -43,6 +52,29 @@ _DEFAULT_OPENROUTER_HEADERS = {
"X-OpenRouter-Title": "nanobot",
"X-OpenRouter-Categories": "cli-agent,personal-agent",
}
_KIMI_THINKING_MODELS: frozenset[str] = frozenset({
"kimi-k2.5",
"k2.6-code-preview",
})
def _is_kimi_thinking_model(model_name: str) -> bool:
"""Return True if model_name refers to a Kimi thinking-capable model.
Supports two forms:
- Exact match: kimi-k2.5 in _KIMI_THINKING_MODELS
- Slug match: moonshotai/kimi-k2.5 -> the part after the last "/"
is checked against _KIMI_THINKING_MODELS
This covers both the native Moonshot provider (bare slug) and
OpenRouter-style names (``"publisher/slug"``).
"""
name = model_name.lower()
if name in _KIMI_THINKING_MODELS:
return True
if "/" in name and name.rsplit("/", 1)[1] in _KIMI_THINKING_MODELS:
return True
return False
def _short_tool_id() -> str:
@@ -113,6 +145,18 @@ def _uses_openrouter_attribution(spec: "ProviderSpec | None", api_base: str | No
return bool(api_base and "openrouter" in api_base.lower())
_RESPONSES_FAILURE_THRESHOLD = 3
_RESPONSES_PROBE_INTERVAL_S = 300 # 5 minutes
def _is_direct_openai_base(api_base: str | None) -> bool:
"""Return True for direct OpenAI endpoints, not generic OpenAI-compatible gateways."""
if not api_base:
return True
normalized = api_base.strip().lower().rstrip("/")
return "api.openai.com" in normalized and "openrouter" not in normalized
class OpenAICompatProvider(LLMProvider):
"""Unified provider for all OpenAI-compatible APIs.
@@ -137,6 +181,7 @@ class OpenAICompatProvider(LLMProvider):
self._setup_env(api_key, api_base)
effective_base = api_base or (spec.default_api_base if spec else None) or None
self._effective_base = effective_base
default_headers = {"x-session-affinity": uuid.uuid4().hex}
if _uses_openrouter_attribution(spec, effective_base):
default_headers.update(_DEFAULT_OPENROUTER_HEADERS)
@@ -150,6 +195,11 @@ class OpenAICompatProvider(LLMProvider):
max_retries=0,
)
# Responses API circuit breaker: skip after repeated failures,
# probe again after _RESPONSES_PROBE_INTERVAL_S seconds.
self._responses_failures: dict[str, int] = {}
self._responses_tripped_at: dict[str, float] = {}
def _setup_env(self, api_key: str, api_base: str | None) -> None:
"""Set environment variables based on provider spec."""
spec = self._spec
@@ -207,6 +257,24 @@ class OpenAICompatProvider(LLMProvider):
return tool_call_id
return hashlib.sha1(tool_call_id.encode()).hexdigest()[:9]
@staticmethod
def _normalize_tool_call_arguments(arguments: Any) -> str:
"""Force function.arguments into a valid JSON object string."""
if isinstance(arguments, str):
stripped = arguments.strip()
if not stripped:
return "{}"
try:
parsed = json_repair.loads(stripped)
except Exception:
return "{}"
if isinstance(parsed, dict):
return json.dumps(parsed, ensure_ascii=False)
return "{}"
if isinstance(arguments, dict):
return json.dumps(arguments, ensure_ascii=False)
return "{}"
def _sanitize_messages(self, messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""Strip non-standard keys, normalize tool_call IDs."""
sanitized = LLMProvider._sanitize_request_messages(messages, _ALLOWED_MSG_KEYS)
@@ -226,11 +294,25 @@ class OpenAICompatProvider(LLMProvider):
continue
tc_clean = dict(tc)
tc_clean["id"] = map_id(tc_clean.get("id"))
function = tc_clean.get("function")
if isinstance(function, dict):
function_clean = dict(function)
if "arguments" in function_clean:
function_clean["arguments"] = self._normalize_tool_call_arguments(
function_clean.get("arguments")
)
else:
function_clean["arguments"] = "{}"
tc_clean["function"] = function_clean
normalized.append(tc_clean)
clean["tool_calls"] = normalized
if clean.get("role") == "assistant":
# Some OpenAI-compatible gateways reject assistant messages
# that mix non-empty content with tool_calls.
clean["content"] = None
if "tool_call_id" in clean and clean["tool_call_id"]:
clean["tool_call_id"] = map_id(clean["tool_call_id"])
return sanitized
return self._enforce_role_alternation(sanitized)
# ------------------------------------------------------------------
# Build kwargs
@@ -315,12 +397,134 @@ class OpenAICompatProvider(LLMProvider):
if extra:
kwargs.setdefault("extra_body", {}).update(extra)
# Model-level thinking injection for Kimi thinking-capable models.
# Strip any provider prefix (e.g. "moonshotai/") before the set lookup
# so that OpenRouter-style names like "moonshotai/kimi-k2.5" are handled
# identically to bare names like "kimi-k2.5".
if reasoning_effort is not None and _is_kimi_thinking_model(model_name):
thinking_enabled = reasoning_effort.lower() != "minimal"
kwargs.setdefault("extra_body", {}).update(
{"thinking": {"type": "enabled" if thinking_enabled else "disabled"}}
)
if tools:
kwargs["tools"] = tools
kwargs["tool_choice"] = tool_choice or "auto"
return kwargs
def _should_use_responses_api(
self,
model: str | None,
reasoning_effort: str | None,
) -> bool:
"""Use Responses API only for direct OpenAI requests that benefit from it."""
if self._spec and self._spec.name != "openai":
return False
if not _is_direct_openai_base(self._effective_base):
return False
model_name = (model or self.default_model).lower()
wants = False
if reasoning_effort and reasoning_effort.lower() != "none":
wants = True
elif any(token in model_name for token in ("gpt-5", "o1", "o3", "o4")):
wants = True
if not wants:
return False
# Circuit breaker: skip after repeated failures, probe periodically.
key = f"{model_name}:{reasoning_effort or ''}"
failures = self._responses_failures.get(key, 0)
if failures >= _RESPONSES_FAILURE_THRESHOLD:
tripped = self._responses_tripped_at.get(key, 0.0)
if (time.monotonic() - tripped) < _RESPONSES_PROBE_INTERVAL_S:
return False
# Half-open: allow one probe attempt
return True
def _record_responses_failure(self, model: str | None, reasoning_effort: str | None) -> None:
key = f"{(model or self.default_model).lower()}:{reasoning_effort or ''}"
count = self._responses_failures.get(key, 0) + 1
self._responses_failures[key] = count
if count >= _RESPONSES_FAILURE_THRESHOLD:
self._responses_tripped_at[key] = time.monotonic()
logger.warning(
"Responses API circuit open for {} — falling back to Chat Completions",
key,
)
def _record_responses_success(self, model: str | None, reasoning_effort: str | None) -> None:
key = f"{(model or self.default_model).lower()}:{reasoning_effort or ''}"
self._responses_failures.pop(key, None)
self._responses_tripped_at.pop(key, None)
@staticmethod
def _should_fallback_from_responses_error(e: Exception) -> bool:
"""Fallback only for likely Responses API compatibility errors."""
response = getattr(e, "response", None)
status_code = getattr(e, "status_code", None)
if status_code is None and response is not None:
status_code = getattr(response, "status_code", None)
if status_code not in {400, 404, 422}:
return False
body = (
getattr(e, "body", None)
or getattr(e, "doc", None)
or getattr(response, "text", None)
)
body_text = str(body).lower() if body is not None else ""
compatibility_markers = (
"responses",
"response api",
"max_output_tokens",
"instructions",
"previous_response",
"unsupported",
"not supported",
"unknown parameter",
"unrecognized request argument",
)
return any(marker in body_text for marker in compatibility_markers)
def _build_responses_body(
self,
messages: list[dict[str, Any]],
tools: list[dict[str, Any]] | None,
model: str | None,
max_tokens: int,
temperature: float,
reasoning_effort: str | None,
tool_choice: str | dict[str, Any] | None,
) -> dict[str, Any]:
"""Build a Responses API body for direct OpenAI requests."""
model_name = model or self.default_model
sanitized_messages = self._sanitize_messages(self._sanitize_empty_content(messages))
instructions, input_items = convert_messages(sanitized_messages)
body: dict[str, Any] = {
"model": model_name,
"instructions": instructions or None,
"input": input_items,
"max_output_tokens": max(1, max_tokens),
"store": False,
"stream": False,
}
if self._supports_temperature(model_name, reasoning_effort):
body["temperature"] = temperature
if reasoning_effort and reasoning_effort.lower() != "none":
body["reasoning"] = {"effort": reasoning_effort}
body["include"] = ["reasoning.encrypted_content"]
if tools:
body["tools"] = convert_tools(tools)
body["tool_choice"] = tool_choice or "auto"
return body
# ------------------------------------------------------------------
# Response parsing
# ------------------------------------------------------------------
@@ -455,7 +659,12 @@ class OpenAICompatProvider(LLMProvider):
finish_reason = str(choice0.get("finish_reason") or "stop")
raw_tool_calls: list[Any] = []
# StepFun Plan: fallback to reasoning field when content is empty
if not content and msg0.get("reasoning"):
content = self._extract_text_content(msg0.get("reasoning"))
reasoning_content = msg0.get("reasoning_content")
if not reasoning_content and msg0.get("reasoning"):
reasoning_content = self._extract_text_content(msg0.get("reasoning"))
for ch in choices:
ch_map = self._maybe_mapping(ch) or {}
m = self._maybe_mapping(ch_map.get("message")) or {}
@@ -511,6 +720,8 @@ class OpenAICompatProvider(LLMProvider):
finish_reason = ch.finish_reason
if not content and m.content:
content = m.content
if not content and getattr(m, "reasoning", None):
content = m.reasoning
tool_calls = []
for tc in raw_tool_calls:
@@ -527,12 +738,16 @@ class OpenAICompatProvider(LLMProvider):
function_provider_specific_fields=fn_prov,
))
reasoning_content = getattr(msg, "reasoning_content", None) or None
if not reasoning_content and getattr(msg, "reasoning", None):
reasoning_content = msg.reasoning
return LLMResponse(
content=content,
tool_calls=tool_calls,
finish_reason=finish_reason or "stop",
usage=self._extract_usage(response),
reasoning_content=getattr(msg, "reasoning_content", None) or None,
reasoning_content=reasoning_content,
)
@classmethod
@@ -593,6 +808,8 @@ class OpenAICompatProvider(LLMProvider):
if text:
content_parts.append(text)
text = cls._extract_text_content(delta.get("reasoning_content"))
if not text:
text = cls._extract_text_content(delta.get("reasoning"))
if text:
reasoning_parts.append(text)
for idx, tc in enumerate(delta.get("tool_calls") or []):
@@ -611,6 +828,8 @@ class OpenAICompatProvider(LLMProvider):
content_parts.append(delta.content)
if delta:
reasoning = getattr(delta, "reasoning_content", None)
if not reasoning:
reasoning = getattr(delta, "reasoning", None)
if reasoning:
reasoning_parts.append(reasoning)
for tc in (delta.tool_calls or []) if delta else []:
@@ -683,7 +902,12 @@ class OpenAICompatProvider(LLMProvider):
}
@staticmethod
def _handle_error(e: Exception) -> LLMResponse:
def _handle_error(
e: Exception,
*,
spec: ProviderSpec | None = None,
api_base: str | None = None,
) -> LLMResponse:
body = (
getattr(e, "doc", None)
or getattr(e, "body", None)
@@ -691,6 +915,15 @@ class OpenAICompatProvider(LLMProvider):
)
body_text = body if isinstance(body, str) else str(body) if body is not None else ""
msg = f"Error: {body_text.strip()[:500]}" if body_text.strip() else f"Error calling LLM: {e}"
text = f"{body_text} {e}".lower()
if spec and spec.is_local and ("502" in text or "connection" in text or "refused" in text):
msg += (
"\nHint: this is a local model endpoint. Check that the local server is reachable at "
f"{api_base or spec.default_api_base}, and if you are using a proxy/tunnel, make sure it "
"can reach your local Ollama/vLLM service instead of routing localhost through the remote host."
)
response = getattr(e, "response", None)
retry_after = LLMProvider._extract_retry_after_from_headers(getattr(response, "headers", None))
if retry_after is None:
@@ -716,14 +949,28 @@ class OpenAICompatProvider(LLMProvider):
reasoning_effort: str | None = None,
tool_choice: str | dict[str, Any] | None = None,
) -> LLMResponse:
kwargs = self._build_kwargs(
messages, tools, model, max_tokens, temperature,
reasoning_effort, tool_choice,
)
try:
if self._should_use_responses_api(model, reasoning_effort):
try:
body = self._build_responses_body(
messages, tools, model, max_tokens, temperature,
reasoning_effort, tool_choice,
)
result = parse_response_output(await self._client.responses.create(**body))
self._record_responses_success(model, reasoning_effort)
return result
except Exception as responses_error:
if not self._should_fallback_from_responses_error(responses_error):
raise
self._record_responses_failure(model, reasoning_effort)
kwargs = self._build_kwargs(
messages, tools, model, max_tokens, temperature,
reasoning_effort, tool_choice,
)
return self._parse(await self._client.chat.completions.create(**kwargs))
except Exception as e:
return self._handle_error(e)
return self._handle_error(e, spec=self._spec, api_base=self.api_base)
async def chat_stream(
self,
@@ -736,14 +983,51 @@ class OpenAICompatProvider(LLMProvider):
tool_choice: str | dict[str, Any] | None = None,
on_content_delta: Callable[[str], Awaitable[None]] | None = None,
) -> LLMResponse:
kwargs = self._build_kwargs(
messages, tools, model, max_tokens, temperature,
reasoning_effort, tool_choice,
)
kwargs["stream"] = True
kwargs["stream_options"] = {"include_usage": True}
idle_timeout_s = int(os.environ.get("NANOBOT_STREAM_IDLE_TIMEOUT_S", "90"))
try:
if self._should_use_responses_api(model, reasoning_effort):
try:
body = self._build_responses_body(
messages, tools, model, max_tokens, temperature,
reasoning_effort, tool_choice,
)
body["stream"] = True
stream = await self._client.responses.create(**body)
async def _timed_stream():
stream_iter = stream.__aiter__()
while True:
try:
yield await asyncio.wait_for(
stream_iter.__anext__(),
timeout=idle_timeout_s,
)
except StopAsyncIteration:
break
content, tool_calls, finish_reason, usage, reasoning_content = await consume_sdk_stream(
_timed_stream(),
on_content_delta,
)
self._record_responses_success(model, reasoning_effort)
return LLMResponse(
content=content or None,
tool_calls=tool_calls,
finish_reason=finish_reason,
usage=usage,
reasoning_content=reasoning_content,
)
except Exception as responses_error:
if not self._should_fallback_from_responses_error(responses_error):
raise
self._record_responses_failure(model, reasoning_effort)
kwargs = self._build_kwargs(
messages, tools, model, max_tokens, temperature,
reasoning_effort, tool_choice,
)
kwargs["stream"] = True
kwargs["stream_options"] = {"include_usage": True}
stream = await self._client.chat.completions.create(**kwargs)
chunks: list[Any] = []
stream_iter = stream.__aiter__()
@@ -771,7 +1055,7 @@ class OpenAICompatProvider(LLMProvider):
error_kind="timeout",
)
except Exception as e:
return self._handle_error(e)
return self._handle_error(e, spec=self._spec, api_base=self.api_base)
def get_default_model(self) -> str:
return self.default_model
+20
View File
@@ -280,6 +280,15 @@ PROVIDERS: tuple[ProviderSpec, ...] = (
backend="openai_compat",
default_api_base="https://api.minimax.io/v1",
),
# MiniMax Anthropic-compatible endpoint: supports thinking mode
ProviderSpec(
name="minimax_anthropic",
keywords=("minimax_anthropic",),
env_key="MINIMAX_API_KEY",
display_name="MiniMax (Anthropic)",
backend="anthropic",
default_api_base="https://api.minimax.io/anthropic",
),
# Mistral AI: OpenAI-compatible API
ProviderSpec(
name="mistral",
@@ -328,6 +337,17 @@ PROVIDERS: tuple[ProviderSpec, ...] = (
detect_by_base_keyword="11434",
default_api_base="http://localhost:11434/v1",
),
# LM Studio (local, OpenAI-compatible)
ProviderSpec(
name="lm_studio",
keywords=("lm-studio", "lmstudio", "lm_studio"),
env_key="LM_STUDIO_API_KEY",
display_name="LM Studio",
backend="openai_compat",
is_local=True,
detect_by_base_keyword="1234",
default_api_base="http://localhost:1234/v1",
),
# === OpenVINO Model Server (direct, local, OpenAI-compatible at /v3) ===
ProviderSpec(
name="ovms",
+8 -4
View File
@@ -10,9 +10,13 @@ from loguru import logger
class OpenAITranscriptionProvider:
"""Voice transcription provider using OpenAI's Whisper API."""
def __init__(self, api_key: str | None = None):
def __init__(self, api_key: str | None = None, api_base: str | None = None):
self.api_key = api_key or os.environ.get("OPENAI_API_KEY")
self.api_url = "https://api.openai.com/v1/audio/transcriptions"
self.api_url = (
api_base
or os.environ.get("OPENAI_TRANSCRIPTION_BASE_URL")
or "https://api.openai.com/v1/audio/transcriptions"
)
async def transcribe(self, file_path: str | Path) -> str:
if not self.api_key:
@@ -44,9 +48,9 @@ class GroqTranscriptionProvider:
Groq offers extremely fast transcription with a generous free tier.
"""
def __init__(self, api_key: str | None = None):
def __init__(self, api_key: str | None = None, api_base: str | None = None):
self.api_key = api_key or os.environ.get("GROQ_API_KEY")
self.api_url = "https://api.groq.com/openai/v1/audio/transcriptions"
self.api_url = api_base or os.environ.get("GROQ_BASE_URL") or "https://api.groq.com/openai/v1/audio/transcriptions"
async def transcribe(self, file_path: str | Path) -> str:
"""
+72
View File
@@ -0,0 +1,72 @@
---
name: my
description: Check and set the agent's own runtime state (model, iterations, context window, token usage, web config). Use when diagnosing why something doesn't work ("why can't you search the web?", "why did you stop?"), checking resource limits before complex tasks, adapting configuration for long or simple tasks, or remembering user preferences across turns. Also use when the user asks what model you are running, how many tokens you've used, or what your settings are.
always: true
---
# Self-Awareness
## How to use
1. **Identify the situation** from the categories below
2. **Call the my tool** with the appropriate action
3. **If set**, warn the user before changing impactful settings (model, iterations)
4. **For detailed examples**, read [references/examples.md](references/examples.md)
## When to check
<rule>
**Diagnose before explaining.** When something doesn't work, check your state first.
</rule>
<rule>
**Check budget before complex tasks.** Know your limits before committing.
</rule>
<rule>
**Recall across turns.** Store preferences in your scratchpad, read them back later.
</rule>
## When to set
<rule>
**Only set when benefit is clear and user is informed.** Warn before changing model.
</rule>
| Situation | Command |
|-----------|---------|
| Large codebase analysis | `my(action="set", key="context_window_tokens", value=131072)` |
| Repetitive simple tasks | `my(action="set", key="model", value="<fast-model>")` |
| Long multi-step task | `my(action="set", key="max_iterations", value=80)` |
**Tradeoff:** Bias toward stability. Only set when defaults are genuinely insufficient.
## Anti-patterns
<rule>
**Don't check every turn.** Costs a tool call. Use when you need information, not reflexively.
</rule>
<rule>
**Don't store sensitive data.** No API keys, passwords, or tokens in scratchpad.
</rule>
<rule>
**Don't set workspace.** Does not update file tool boundaries — won't work.
</rule>
## Constraints
- All modifications in-memory only — restart resets everything
- Protected params have type/range validation: `max_iterations` (1100), `context_window_tokens` (40961M), `model` (non-empty str)
- If `tools.my.allow_set` is false, check only
## Related tools
| Need | Use | Persists? |
|------|-----|-----------|
| Per-session temp state | `my(action="set", key="...", value=...)` | No |
| Long-term facts | Memory skill (`MEMORY.md`, `USER.md`) | Yes |
| Permanent config change | Edit config file | Yes |
**Rule of thumb:** Tomorrow? Memory. This turn only? My.
+75
View File
@@ -0,0 +1,75 @@
# My Tool — Practical Examples
Concrete scenarios showing when and how to use the my tool effectively.
## Diagnosis
### "Why can't you search the web?"
```
→ my(action="check", key="web_config.enable")
→ False
→ "Web search is disabled. Add web.enable: true to your config to enable it."
```
### "Why did you stop?"
```
→ my(action="check", key="max_iterations")
→ 40
→ my(action="check", key="_last_usage")
→ {"prompt_tokens": 62000, "completion_tokens": 3000}
→ "I hit the iteration limit (40). The task was complex. I can ask the user if they want to increase it."
```
### "What model are you running?"
```
→ my(action="check", key="model")
→ 'anthropic/claude-sonnet-4-20250514'
```
## Adaptive Behavior
### Large codebase analysis
```
→ my(action="check")
→ context_window_tokens: 65536
→ my(action="set", key="context_window_tokens", value=131072)
→ "Set context_window_tokens = 131072 (was 65536)"
→ "I've expanded my context window to handle this large codebase."
```
### Switching to a faster model for repetitive tasks
```
→ my(action="set", key="model", value="anthropic/claude-haiku-4-5-20251001")
→ "Set model = 'anthropic/claude-haiku-4-5-20251001' (was 'anthropic/claude-sonnet-4-20250514')"
→ "Switched to a faster model for these batch tasks."
```
## Cross-Turn Memory
### Remembering user preferences
```
# Turn 1: user says "keep it brief"
→ my(action="set", key="user_style", value="concise")
→ "Set scratchpad.user_style = 'concise'"
# Turn 3: new topic
→ my(action="check", key="user_style")
→ 'concise'
(adjusts response style accordingly)
```
### Tracking project context
```
→ my(action="set", key="active_branch", value="feat/auth")
→ my(action="set", key="test_framework", value="pytest")
→ my(action="set", key="has_docker", value=true)
```
## Budget Awareness
### Token-conscious behavior
```
→ my(action="check", key="_last_usage")
→ {"prompt_tokens": 58000, "completion_tokens": 12000}
→ "I've consumed ~70k tokens. I'll keep my remaining responses focused."
```
-2
View File
@@ -1,7 +1,5 @@
# Agent Instructions
You are a helpful AI assistant. Be concise, accurate, and friendly.
## Scheduled Reminders
Before scheduling reminders, check available skills and follow skill guidance first.
+5 -17
View File
@@ -2,20 +2,8 @@
I am nanobot 🐈, a personal AI assistant.
## Personality
- Helpful and friendly
- Concise and to the point
- Curious and eager to learn
## Values
- Accuracy over speed
- User privacy and safety
- Transparency in actions
## Communication Style
- Be clear and direct
- Explain reasoning when helpful
- Ask clarifying questions when needed
I solve problems by doing, not by describing what I would do.
I keep responses short unless depth is asked for.
I say what I know, flag what I don't, and never fake confidence.
I stay friendly and curious — I'd rather ask a good question than guess wrong.
I treat the user's time as the scarcest resource, and their trust as the most valuable.
+24 -7
View File
@@ -1,8 +1,11 @@
Compare conversation history against current memory files. Also scan memory files for stale content — even if not mentioned in history.
You have TWO equally important tasks:
1. Extract new facts from conversation history
2. Deduplicate existing memory files — find and flag redundant, overlapping, or stale content even if NOT mentioned in history
Output one line per finding:
[FILE] atomic fact (not already in memory)
[FILE-REMOVE] reason for removal
[SKILL] kebab-case-name: one-line description of the reusable pattern
Files: USER (identity, preferences), SOUL (bot behavior, tone), MEMORY (knowledge, project context)
@@ -11,12 +14,26 @@ Rules:
- Corrections: [USER] location is Tokyo, not Osaka
- Capture confirmed approaches the user validated
Staleness — flag for [FILE-REMOVE]:
- Time-sensitive data older than 14 days: weather, daily status, one-time meetings, passed events
- Completed one-time tasks: triage, one-time reviews, finished research, resolved incidents
- Resolved tracking: merged/closed PRs, fixed issues, completed migrations
- Detailed incident info after 14 days — reduce to one-line summary
- Superseded: approaches replaced by newer solutions, deprecated dependencies
Deduplication — scan ALL memory files for these redundancy patterns:
- Same fact stated in multiple places (e.g., "communicates in Chinese" in both USER.md and multiple MEMORY.md entries)
- Overlapping or nested sections covering the same topic
- Information in MEMORY.md that is already captured in USER.md or SOUL.md (MEMORY.md should not duplicate permanent-file content)
- Verbose entries that can be condensed without losing information
For each duplicate found, output [FILE-REMOVE] for the less authoritative copy (prefer keeping facts in their canonical location)
Staleness — MEMORY.md lines may have a ``← Nd`` suffix showing days since last modification:
- SOUL.md and USER.md have no age annotations — they are permanent, only update with corrections
- Age only indicates when content was last touched, not whether it should be removed
- Use content judgment: user habits/preferences/personality traits are permanent regardless of age
- Only prune content that is objectively outdated: passed events, resolved tracking, superseded approaches
- Lines with ``← Nd`` (N>{{ stale_threshold_days }}) deserve closer review but are NOT automatically removable
- When removing: prefer deleting individual items over entire sections
Skill discovery — flag [SKILL] when ALL of these are true:
- A specific, repeatable workflow appeared 2+ times in the conversation history
- It involves clear steps (not vague preferences like "likes concise answers")
- It is substantial enough to warrant its own instruction set (not trivial like "read a file")
- Do not worry about duplicates — the next phase will check against existing skills
Do not add: current weather, transient status, temporary errors, conversational filler.
+13
View File
@@ -1,11 +1,13 @@
Update memory files based on the analysis below.
- [FILE] entries: add the described content to the appropriate file
- [FILE-REMOVE] entries: delete the corresponding content from memory files
- [SKILL] entries: create a new skill under skills/<name>/SKILL.md using write_file
## File paths (relative to workspace root)
- SOUL.md
- USER.md
- memory/MEMORY.md
- skills/<name>/SKILL.md (for [SKILL] entries only)
Do NOT guess paths.
@@ -17,6 +19,17 @@ Do NOT guess paths.
- Surgical edits only — never rewrite entire files
- If nothing to update, stop without calling tools
## Skill creation rules (for [SKILL] entries)
- Use write_file to create skills/<name>/SKILL.md
- Before writing, read_file `{{ skill_creator_path }}` for format reference (frontmatter structure, naming conventions, quality standards)
- **Dedup check**: read existing skills listed below to verify the new skill is not functionally redundant. Skip creation if an existing skill already covers the same workflow.
- Include YAML frontmatter with name and description fields
- Keep SKILL.md under 2000 words — concise and actionable
- Include: when to use, steps, output format, at least one example
- Do NOT overwrite existing skills — skip if the skill directory already exists
- Reference specific tools the agent has access to (read_file, write_file, exec, web_search, etc.)
- Skills are instruction sets, not code — do not include implementation code
## Quality
- Every line must carry standalone value
- Concise bullets under clear headers
+25 -8
View File
@@ -12,15 +12,32 @@ Your workspace is at: {{ workspace_path }}
- Custom skills: {{ workspace_path }}/skills/{% raw %}{skill-name}{% endraw %}/SKILL.md
{{ platform_policy }}
{% if channel == 'telegram' or channel == 'qq' or channel == 'discord' %}
## Format Hint
This conversation is on a messaging app. Use short paragraphs. Avoid large headings (#, ##). Use **bold** sparingly. No tables — use plain lists.
{% elif channel == 'whatsapp' or channel == 'sms' %}
## Format Hint
This conversation is on a text messaging platform that does not render markdown. Use plain text only.
{% elif channel == 'email' %}
## Format Hint
This conversation is via email. Structure with clear sections. Markdown may not render — keep formatting simple.
{% elif channel == 'cli' or channel == 'mochat' %}
## Format Hint
Output is rendered in a terminal. Avoid markdown headings and tables. Use plain text with minimal formatting.
{% endif %}
## nanobot Guidelines
- State intent before tool calls, but NEVER predict or claim results before receiving them.
- Before modifying a file, read it first. Do not assume files or directories exist.
- After writing or editing a file, re-read it if accuracy matters.
- If a tool call fails, analyze the error before retrying with a different approach.
- Ask for clarification when the request is ambiguous.
- Prefer built-in `grep` / `glob` tools for workspace search before falling back to `exec`.
- On broad searches, use `grep(output_mode="count")` or `grep(output_mode="files_with_matches")` to scope the result set before requesting full content.
## Execution Rules
- Act, don't narrate. If you can do it with a tool, do it now — never end a turn with just a plan or promise.
- Read before you write. Do not assume a file exists or contains what you expect.
- If a tool call fails, diagnose the error and retry with a different approach before reporting failure.
- When information is missing, look it up with tools first. Only ask the user when tools cannot answer.
- After multi-step changes, verify the result (re-read the file, run the test, check the output).
## Search & Discovery
- Prefer built-in `grep` / `glob` over `exec` for workspace search.
- On broad searches, use `grep(output_mode="count")` to scope before requesting full content.
{% include 'agent/_snippets/untrusted_content.md' %}
Reply directly with text for conversations. Only use the 'message' tool to send to a specific chat channel.
+1 -1
View File
@@ -1,6 +1,6 @@
# Skills
The following skills extend your capabilities. To use a skill, read its SKILL.md file using the read_file tool.
Skills with available="false" need dependencies installed first - you can try installing them with apt/brew.
Unavailable skills need dependencies installed first you can try installing them with apt/brew.
{{ skills_summary }}
+267
View File
@@ -0,0 +1,267 @@
"""Document text extraction utilities for nanobot."""
import mimetypes
from pathlib import Path
from loguru import logger
from nanobot.utils.helpers import detect_image_mime
try:
from pypdf import PdfReader
except ImportError:
PdfReader = None # type: ignore
try:
from docx import Document as DocxDocument
except ImportError:
DocxDocument = None # type: ignore
try:
from openpyxl import load_workbook
except ImportError:
load_workbook = None # type: ignore
try:
from pptx import Presentation as PptxPresentation
except ImportError:
PptxPresentation = None # type: ignore
# Supported file extensions for text extraction
SUPPORTED_EXTENSIONS: set[str] = {
# Document formats
".pdf",
".docx",
".xlsx",
".pptx",
# Text formats
".txt",
".md",
".csv",
".json",
".xml",
".html",
".htm",
".log",
".yaml",
".yml",
".toml",
".ini",
".cfg",
# Image formats (for future OCR support)
".png",
".jpg",
".jpeg",
".gif",
".webp",
}
_MAX_TEXT_LENGTH = 200_000
def extract_text(path: Path) -> str | None:
"""Extract text from a file.
Args:
path: Path to the file.
Returns:
Extracted text as string, None for unsupported types,
or error string for failures.
"""
if not isinstance(path, Path):
path = Path(path)
if not path.exists():
return f"[error: file not found: {path}]"
ext = path.suffix.lower()
# Document formats
if ext == ".pdf":
if PdfReader is None:
return "[error: pypdf not installed]"
return _extract_pdf(path)
elif ext == ".docx":
if DocxDocument is None:
return "[error: python-docx not installed]"
return _extract_docx(path)
elif ext == ".xlsx":
if load_workbook is None:
return "[error: openpyxl not installed]"
return _extract_xlsx(path)
elif ext == ".pptx":
if PptxPresentation is None:
return "[error: python-pptx not installed]"
return _extract_pptx(path)
elif _is_text_extension(ext):
return _extract_text_file(path)
elif ext in {".png", ".jpg", ".jpeg", ".gif", ".webp"}:
# Image files - for future OCR support
return f"[image: {path.name}]"
else:
# Unsupported extension
return None
def _extract_pdf(path: Path) -> str:
"""Extract text from PDF using pypdf."""
try:
reader = PdfReader(path)
pages: list[str] = []
for i, page in enumerate(reader.pages, 1):
text = page.extract_text() or ""
pages.append(f"--- Page {i} ---\n{text}")
return _truncate("\n\n".join(pages), _MAX_TEXT_LENGTH)
except Exception as e:
logger.error("Failed to extract PDF {}: {}", path, e)
return f"[error: failed to extract PDF: {e!s}]"
def _extract_docx(path: Path) -> str:
"""Extract text from DOCX using python-docx."""
try:
doc = DocxDocument(path)
paragraphs: list[str] = [p.text for p in doc.paragraphs if p.text.strip()]
return _truncate("\n\n".join(paragraphs), _MAX_TEXT_LENGTH)
except Exception as e:
logger.error("Failed to extract DOCX {}: {}", path, e)
return f"[error: failed to extract DOCX: {e!s}]"
def _extract_xlsx(path: Path) -> str:
"""Extract text from XLSX using openpyxl."""
try:
wb = load_workbook(path, read_only=True, data_only=True)
sheets: list[str] = []
for sheet_name in wb.sheetnames:
ws = wb[sheet_name]
rows: list[str] = []
for row in ws.iter_rows(values_only=True):
row_text = "\t".join(str(cell) if cell is not None else "" for cell in row)
if row_text.strip():
rows.append(row_text)
if rows:
sheets.append(f"--- Sheet: {sheet_name} ---\n" + "\n".join(rows))
wb.close()
return _truncate("\n\n".join(sheets), _MAX_TEXT_LENGTH)
except Exception as e:
logger.error("Failed to extract XLSX {}: {}", path, e)
return f"[error: failed to extract XLSX: {e!s}]"
def _extract_pptx(path: Path) -> str:
"""Extract text from PPTX using python-pptx."""
try:
prs = PptxPresentation(path)
slides: list[str] = []
for i, slide in enumerate(prs.slides, 1):
slide_text: list[str] = []
for shape in slide.shapes:
if hasattr(shape, "text") and shape.text:
slide_text.append(shape.text)
if slide_text:
slides.append(f"--- Slide {i} ---\n" + "\n".join(slide_text))
return _truncate("\n\n".join(slides), _MAX_TEXT_LENGTH)
except Exception as e:
logger.error("Failed to extract PPTX {}: {}", path, e)
return f"[error: failed to extract PPTX: {e!s}]"
def _extract_text_file(path: Path) -> str:
"""Extract text from a plain text file."""
try:
# Try UTF-8 first, then latin-1 fallback
try:
content = path.read_text(encoding="utf-8")
except UnicodeDecodeError:
content = path.read_text(encoding="latin-1")
return _truncate(content, _MAX_TEXT_LENGTH)
except Exception as e:
logger.error("Failed to read text file {}: {}", path, e)
return f"[error: failed to read file: {e!s}]"
def _truncate(text: str, max_length: int) -> str:
"""Truncate text with a suffix indicating truncation."""
if len(text) <= max_length:
return text
return text[:max_length] + f"... (truncated, {len(text)} chars total)"
def _is_text_extension(ext: str) -> bool:
"""Check if extension is a text format."""
return ext in {
".txt",
".md",
".csv",
".json",
".xml",
".html",
".htm",
".log",
".yaml",
".yml",
".toml",
".ini",
".cfg",
}
# ---------------------------------------------------------------------------
# High-level helper: split media into images + extracted document text
# ---------------------------------------------------------------------------
_MAX_EXTRACT_FILE_SIZE = 50 * 1024 * 1024 # 50 MB
def extract_documents(
text: str,
media_paths: list[str],
*,
max_file_size: int = _MAX_EXTRACT_FILE_SIZE,
) -> tuple[str, list[str]]:
"""Separate images from documents in *media_paths*.
Documents (PDF, DOCX, XLSX, PPTX, plain-text, ) have their text
extracted and appended to *text*. Only image paths are kept in the
returned list so that downstream layers only need to handle vision
blocks.
Files larger than *max_file_size* bytes are skipped with a warning
to avoid unbounded memory / CPU usage.
"""
image_paths: list[str] = []
doc_texts: list[str] = []
for path_str in media_paths:
p = Path(path_str)
if not p.is_file():
continue
try:
size = p.stat().st_size
except OSError:
continue
if size > max_file_size:
logger.warning(
"Skipping oversized file for extraction: {} ({:.1f} MB > {} MB limit)",
p.name, size / (1024 * 1024), max_file_size // (1024 * 1024),
)
continue
with open(p, "rb") as f:
header = f.read(16)
mime = detect_image_mime(header) or mimetypes.guess_type(path_str)[0]
if mime and mime.startswith("image/"):
image_paths.append(path_str)
else:
extracted = extract_text(p)
if extracted and not extracted.startswith("[error:"):
doc_texts.append(f"[File: {p.name}]\n{extracted}")
if doc_texts:
text = text + "\n\n" + "\n\n".join(doc_texts)
return text, image_paths
+46
View File
@@ -5,6 +5,7 @@ from __future__ import annotations
import io
import time
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from loguru import logger
@@ -24,6 +25,23 @@ class CommitInfo:
return f"{header}\n(no file changes)"
@dataclass
class LineAge:
"""Age of a single line based on git blame."""
age_days: int # days since last modification
def _compute_line_ages(annotated) -> list[LineAge]:
"""Convert annotate results to per-line ages."""
now = datetime.now(tz=timezone.utc).date()
ages: list[LineAge] = []
for (commit, _tree_entry), _line_bytes in annotated:
dt = datetime.fromtimestamp(commit.commit_time, tz=timezone.utc).date()
ages.append(LineAge(age_days=(now - dt).days))
return ages
class GitStore:
"""Git-backed version control for memory files."""
@@ -191,6 +209,34 @@ class GitStore:
logger.warning("Git log failed")
return []
def line_ages(self, file_path: str) -> list[LineAge]:
"""Compute the age of each line in a tracked file via git blame.
Returns one LineAge per line, in order.
Returns an empty list if the repo is not initialized, the file is
empty, or annotation fails.
"""
if not self.is_initialized():
return []
target = self._workspace / file_path
if not target.exists() or target.stat().st_size == 0:
return []
try:
from dulwich import porcelain
annotated = porcelain.annotate(str(self._workspace), file_path)
except Exception:
logger.warning("Git line_ages annotate failed for {}", file_path)
return []
if not annotated:
return []
return _compute_line_ages(annotated)
def diff_commits(self, sha1: str, sha2: str) -> str:
"""Show diff between two commits."""
if not self.is_initialized():
+11 -6
View File
@@ -17,10 +17,10 @@ from loguru import logger
def strip_think(text: str) -> str:
"""Remove thinking blocks and any unclosed trailing tag."""
text = re.sub(r"<think>[\s\S]*?</think>", "", text)
text = re.sub(r"<think>[\s\S]*$", "", text)
text = re.sub(r"^\s*<think>[\s\S]*$", "", text)
# Gemma 4 and similar models use <thought>...</thought> blocks
text = re.sub(r"<thought>[\s\S]*?</thought>", "", text)
text = re.sub(r"<thought>[\s\S]*$", "", text)
text = re.sub(r"^\s*<thought>[\s\S]*$", "", text)
return text.strip()
@@ -275,7 +275,7 @@ def build_assistant_message(
thinking_blocks: list[dict] | None = None,
) -> dict[str, Any]:
"""Build a provider-safe assistant message with optional reasoning fields."""
msg: dict[str, Any] = {"role": "assistant", "content": content}
msg: dict[str, Any] = {"role": "assistant", "content": content or ""}
if tool_calls:
msg["tool_calls"] = tool_calls
if reasoning_content is not None or thinking_blocks:
@@ -400,6 +400,8 @@ def build_status_content(
session_msg_count: int,
context_tokens_estimate: int,
search_usage_text: str | None = None,
active_task_count: int = 0,
max_completion_tokens: int = 8192,
) -> str:
"""Build a human-readable runtime status snapshot.
@@ -418,9 +420,11 @@ def build_status_content(
last_out = last_usage.get("completion_tokens", 0)
cached = last_usage.get("cached_tokens", 0)
ctx_total = max(context_window_tokens, 0)
ctx_pct = int((context_tokens_estimate / ctx_total) * 100) if ctx_total > 0 else 0
# Budget mirrors Consolidator formula: ctx_window - max_completion - _SAFETY_BUFFER
ctx_budget = max(ctx_total - int(max_completion_tokens) - 1024, 1)
ctx_pct = min(int((context_tokens_estimate / ctx_budget) * 100), 999) if ctx_budget > 0 else 0
ctx_used_str = f"{context_tokens_estimate // 1000}k" if context_tokens_estimate >= 1000 else str(context_tokens_estimate)
ctx_total_str = f"{ctx_total // 1024}k" if ctx_total > 0 else "n/a"
ctx_total_str = f"{ctx_total // 1000}k" if ctx_total > 0 else "n/a"
token_line = f"\U0001f4ca Tokens: {last_in} in / {last_out} out"
if cached and last_in:
token_line += f" ({cached * 100 // last_in}% cached)"
@@ -428,9 +432,10 @@ def build_status_content(
f"\U0001f408 nanobot v{version}",
f"\U0001f9e0 Model: {model}",
token_line,
f"\U0001f4da Context: {ctx_used_str}/{ctx_total_str} ({ctx_pct}%)",
f"\U0001f4da Context: {ctx_used_str}/{ctx_total_str} ({ctx_pct}% of input budget)",
f"\U0001f4ac Session: {session_msg_count} messages",
f"\u23f1 Uptime: {uptime}",
f"\u26a1 Tasks: {active_task_count} active",
]
if search_usage_text:
lines.append(search_usage_text)
+10
View File
@@ -19,6 +19,11 @@ FINALIZATION_RETRY_PROMPT = (
"Please provide your response to the user based on the conversation above."
)
LENGTH_RECOVERY_PROMPT = (
"Output limit reached. Continue exactly where you left off "
"— no recap, no apology. Break remaining work into smaller steps if needed."
)
def empty_tool_result_message(tool_name: str) -> str:
"""Short prompt-safe marker for tools that completed without visible output."""
@@ -50,6 +55,11 @@ def build_finalization_retry_message() -> dict[str, str]:
return {"role": "user", "content": FINALIZATION_RETRY_PROMPT}
def build_length_recovery_message() -> dict[str, str]:
"""Prompt the model to continue after hitting output token limit."""
return {"role": "user", "content": LENGTH_RECOVERY_PROMPT}
def external_lookup_signature(tool_name: str, arguments: dict[str, Any]) -> str | None:
"""Stable signature for repeated external lookups we want to throttle."""
if tool_name == "web_fetch":
+12 -5
View File
@@ -19,9 +19,11 @@ _TOOL_FORMATS: dict[str, tuple[list[str], str, bool, bool]] = {
"list_dir": (["path"], "ls {}", True, False),
}
# Matches file paths embedded in shell commands (Windows drive, ~/, or absolute after space)
# Matches file paths embedded in shell commands, including quoted paths with spaces.
_PATH_IN_CMD_RE = re.compile(
r"(?:[A-Za-z]:[/\\]|~/|(?<=\s)/)[^\s;&|<>\"']+"
r'"(?P<double>(?:[A-Za-z]:[/\\]|~/|/)[^"]+)"'
r"|'(?P<single>(?:[A-Za-z]:[/\\]|~/|/)[^']+)'"
r"|(?P<bare>(?:[A-Za-z]:[/\\]|~/|(?<=\s)/)[^\s;&|<>\"']+)"
)
@@ -92,9 +94,14 @@ def _fmt_known(tc, fmt: tuple) -> str:
def _abbreviate_command(cmd: str, max_len: int = 40) -> str:
"""Abbreviate paths in a command string, then truncate."""
abbreviated = _PATH_IN_CMD_RE.sub(
lambda m: abbreviate_path(m.group(), max_len=25), cmd
)
def _replace_path(match: re.Match[str]) -> str:
if match.group("double") is not None:
return f'"{abbreviate_path(match.group("double"), max_len=25)}"'
if match.group("single") is not None:
return f"'{abbreviate_path(match.group('single'), max_len=25)}'"
return abbreviate_path(match.group("bare"), max_len=25)
abbreviated = _PATH_IN_CMD_RE.sub(_replace_path, cmd)
if len(abbreviated) <= max_len:
return abbreviated
return abbreviated[:max_len - 1] + "\u2026"
+11 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "nanobot-ai"
version = "0.1.5"
version = "0.1.5.post1"
description = "A lightweight personal AI assistant framework"
readme = { file = "README.md", content-type = "text/markdown" }
requires-python = ">=3.11"
@@ -50,6 +50,12 @@ dependencies = [
"tiktoken>=0.12.0,<1.0.0",
"jinja2>=3.1.0,<4.0.0",
"dulwich>=0.22.0,<1.0.0",
"pyyaml>=6.0,<7.0.0",
"pypdf>=5.0.0,<6.0.0",
"python-docx>=1.1.0,<2.0.0",
"openpyxl>=3.1.0,<4.0.0",
"python-pptx>=1.0.0,<2.0.0",
"filelock>=3.25.2",
]
[project.optional-dependencies]
@@ -79,12 +85,16 @@ discord = [
langsmith = [
"langsmith>=0.1.0",
]
pdf = [
"pymupdf>=1.25.0",
]
dev = [
"pytest>=9.0.0,<10.0.0",
"pytest-asyncio>=1.3.0,<2.0.0",
"aiohttp>=3.9.0,<4.0.0",
"pytest-cov>=6.0.0,<7.0.0",
"ruff>=0.1.0",
"pymupdf>=1.25.0",
]
[project.scripts]
View File
+196 -117
View File
@@ -35,6 +35,13 @@ def _make_loop(tmp_path: Path, session_ttl_minutes: int = 15) -> AgentLoop:
return loop
def _add_turns(session, turns: int, *, prefix: str = "msg") -> None:
"""Append simple user/assistant turns to a session."""
for i in range(turns):
session.add_message("user", f"{prefix} user {i}")
session.add_message("assistant", f"{prefix} assistant {i}")
class TestSessionTTLConfig:
"""Test session TTL configuration."""
@@ -48,6 +55,23 @@ class TestSessionTTLConfig:
defaults = AgentDefaults(session_ttl_minutes=30)
assert defaults.session_ttl_minutes == 30
def test_user_friendly_alias_is_supported(self):
"""Config should accept idleCompactAfterMinutes as the preferred JSON key."""
defaults = AgentDefaults.model_validate({"idleCompactAfterMinutes": 30})
assert defaults.session_ttl_minutes == 30
def test_legacy_alias_is_still_supported(self):
"""Config should still accept the old sessionTtlMinutes key for compatibility."""
defaults = AgentDefaults.model_validate({"sessionTtlMinutes": 30})
assert defaults.session_ttl_minutes == 30
def test_serializes_with_user_friendly_alias(self):
"""Config dumps should use idleCompactAfterMinutes for JSON output."""
defaults = AgentDefaults(session_ttl_minutes=30)
data = defaults.model_dump(mode="json", by_alias=True)
assert data["idleCompactAfterMinutes"] == 30
assert "sessionTtlMinutes" not in data
class TestAgentLoopTTLParam:
"""Test that AutoCompact receives and stores session_ttl_minutes."""
@@ -101,7 +125,7 @@ class TestAutoCompact:
loop.sessions.save(s2)
async def _fake_archive(messages):
return True
return "Summary."
loop.consolidator.archive = _fake_archive
loop.auto_compact.check_expired(loop._schedule_background)
@@ -113,28 +137,28 @@ class TestAutoCompact:
await loop.close_mcp()
@pytest.mark.asyncio
async def test_auto_compact_archives_and_clears(self, tmp_path):
"""_archive should archive un-consolidated messages and clear session."""
async def test_auto_compact_archives_prefix_and_keeps_recent_suffix(self, tmp_path):
"""_archive should summarize the old prefix and keep a recent legal suffix."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
for i in range(4):
session.add_message("user", f"msg{i}")
session.add_message("assistant", f"resp{i}")
_add_turns(session, 6)
loop.sessions.save(session)
archived_messages = []
async def _fake_archive(messages):
archived_messages.extend(messages)
return True
return "Summary."
loop.consolidator.archive = _fake_archive
await loop.auto_compact._archive("cli:test")
assert len(archived_messages) == 8
assert len(archived_messages) == 4
session_after = loop.sessions.get_or_create("cli:test")
assert len(session_after.messages) == 0
assert len(session_after.messages) == loop.auto_compact._RECENT_SUFFIX_MESSAGES
assert session_after.messages[0]["content"] == "msg user 2"
assert session_after.messages[-1]["content"] == "msg assistant 5"
await loop.close_mcp()
@pytest.mark.asyncio
@@ -142,17 +166,13 @@ class TestAutoCompact:
"""_archive should store the summary in _summaries."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "hello")
session.add_message("assistant", "hi there")
_add_turns(session, 6, prefix="hello")
loop.sessions.save(session)
async def _fake_archive(messages):
return True
return "User said hello."
loop.consolidator.archive = _fake_archive
loop.consolidator.get_last_history_entry = lambda: {
"cursor": 1, "timestamp": "2026-01-01 00:00", "content": "User said hello.",
}
await loop.auto_compact._archive("cli:test")
@@ -160,7 +180,7 @@ class TestAutoCompact:
assert entry is not None
assert entry[0] == "User said hello."
session_after = loop.sessions.get_or_create("cli:test")
assert len(session_after.messages) == 0
assert len(session_after.messages) == loop.auto_compact._RECENT_SUFFIX_MESSAGES
await loop.close_mcp()
@pytest.mark.asyncio
@@ -174,7 +194,7 @@ class TestAutoCompact:
async def _fake_archive(messages):
nonlocal archive_called
archive_called = True
return True
return "Summary."
loop.consolidator.archive = _fake_archive
@@ -190,9 +210,7 @@ class TestAutoCompact:
"""_archive should only archive un-consolidated messages."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
for i in range(10):
session.add_message("user", f"msg{i}")
session.add_message("assistant", f"resp{i}")
_add_turns(session, 14)
session.last_consolidated = 18
loop.sessions.save(session)
@@ -201,7 +219,7 @@ class TestAutoCompact:
async def _fake_archive(messages):
nonlocal archived_count
archived_count = len(messages)
return True
return "Summary."
loop.consolidator.archive = _fake_archive
@@ -235,7 +253,7 @@ class TestAutoCompactIdleDetection:
"""Proactive auto-new archives expired session; _process_message reloads it."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "old message")
_add_turns(session, 6, prefix="old")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
@@ -243,12 +261,9 @@ class TestAutoCompactIdleDetection:
async def _fake_archive(messages):
archived_messages.extend(messages)
return True
return "Summary."
loop.consolidator.archive = _fake_archive
loop.consolidator.get_last_history_entry = lambda: {
"cursor": 1, "timestamp": "2026-01-01 00:00", "content": "Summary.",
}
# Simulate proactive archive completing before message arrives
await loop.auto_compact._archive("cli:test")
@@ -257,7 +272,8 @@ class TestAutoCompactIdleDetection:
await loop._process_message(msg)
session_after = loop.sessions.get_or_create("cli:test")
assert not any(m["content"] == "old message" for m in session_after.messages)
assert len(archived_messages) == 4
assert not any(m["content"] == "old user 0" for m in session_after.messages)
assert any(m["content"] == "new msg" for m in session_after.messages)
await loop.close_mcp()
@@ -311,7 +327,7 @@ class TestAutoCompactIdleDetection:
loop.sessions.save(session)
async def _fake_archive(messages):
return True
return "Summary."
loop.consolidator.archive = _fake_archive
@@ -335,17 +351,14 @@ class TestAutoCompactSystemMessages:
"""Proactive auto-new archives expired session; system messages reload it."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "old message from subagent context")
_add_turns(session, 6, prefix="old")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
async def _fake_archive(messages):
return True
return "Summary."
loop.consolidator.archive = _fake_archive
loop.consolidator.get_last_history_entry = lambda: {
"cursor": 1, "timestamp": "2026-01-01 00:00", "content": "Summary.",
}
# Simulate proactive archive completing before system message arrives
await loop.auto_compact._archive("cli:test")
@@ -358,7 +371,7 @@ class TestAutoCompactSystemMessages:
session_after = loop.sessions.get_or_create("cli:test")
assert not any(
m["content"] == "old message from subagent context"
m["content"] == "old user 0"
for m in session_after.messages
)
await loop.close_mcp()
@@ -372,8 +385,7 @@ class TestAutoCompactEdgeCases:
"""Auto-new should not inject when archive produces '(nothing)'."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "thanks")
session.add_message("assistant", "you're welcome")
_add_turns(session, 6, prefix="thanks")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
@@ -384,18 +396,18 @@ class TestAutoCompactEdgeCases:
await loop.auto_compact._archive("cli:test")
session_after = loop.sessions.get_or_create("cli:test")
assert len(session_after.messages) == 0
assert len(session_after.messages) == loop.auto_compact._RECENT_SUFFIX_MESSAGES
# "(nothing)" summary should not be stored
assert "cli:test" not in loop.auto_compact._summaries
await loop.close_mcp()
@pytest.mark.asyncio
async def test_auto_compact_archive_failure_still_clears(self, tmp_path):
"""Auto-new should clear session even if LLM archive fails (raw_archive fallback)."""
async def test_auto_compact_archive_failure_still_keeps_recent_suffix(self, tmp_path):
"""Auto-new should keep the recent suffix even if LLM archive falls back to raw dump."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "important data")
_add_turns(session, 6, prefix="important")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
@@ -405,14 +417,13 @@ class TestAutoCompactEdgeCases:
await loop.auto_compact._archive("cli:test")
session_after = loop.sessions.get_or_create("cli:test")
# Session should be cleared (archive falls back to raw dump)
assert len(session_after.messages) == 0
assert len(session_after.messages) == loop.auto_compact._RECENT_SUFFIX_MESSAGES
await loop.close_mcp()
@pytest.mark.asyncio
async def test_auto_compact_preserves_runtime_checkpoint_before_check(self, tmp_path):
"""Runtime checkpoint is restored; proactive archive handles the expired session."""
"""Short expired sessions keep recent messages; checkpoint restore still works on resume."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.metadata[AgentLoop._RUNTIME_CHECKPOINT_KEY] = {
@@ -428,12 +439,9 @@ class TestAutoCompactEdgeCases:
async def _fake_archive(messages):
archived_messages.extend(messages)
return True
return "Summary."
loop.consolidator.archive = _fake_archive
loop.consolidator.get_last_history_entry = lambda: {
"cursor": 1, "timestamp": "2026-01-01 00:00", "content": "Summary.",
}
# Simulate proactive archive completing before message arrives
await loop.auto_compact._archive("cli:test")
@@ -441,8 +449,10 @@ class TestAutoCompactEdgeCases:
msg = InboundMessage(channel="cli", sender_id="user", chat_id="test", content="continue")
await loop._process_message(msg)
# The checkpoint-restored message should have been archived by proactive path
assert len(archived_messages) >= 1
session_after = loop.sessions.get_or_create("cli:test")
assert archived_messages == []
assert any(m["content"] == "previous message" for m in session_after.messages)
assert any(m["content"] == "interrupted response" for m in session_after.messages)
await loop.close_mcp()
@@ -458,11 +468,17 @@ class TestAutoCompactIntegration:
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
# Phase 1: User has a conversation
# Phase 1: User has a conversation longer than the retained recent suffix
session.add_message("user", "I'm learning English, teach me past tense")
session.add_message("assistant", "Past tense is used for actions completed in the past...")
session.add_message("user", "Give me an example")
session.add_message("assistant", '"I walked to the store yesterday."')
session.add_message("user", "Give me another example")
session.add_message("assistant", '"She visited Paris last year."')
session.add_message("user", "Quiz me")
session.add_message("assistant", "What is the past tense of go?")
session.add_message("user", "I think it is went")
session.add_message("assistant", "Correct.")
loop.sessions.save(session)
# Phase 2: Time passes (simulate idle)
@@ -486,7 +502,7 @@ class TestAutoCompactIntegration:
# Phase 4: Verify
session_after = loop.sessions.get_or_create("cli:test")
# Old messages should be gone
# The oldest messages should be trimmed from live session history
assert not any(
"past tense is used" in str(m.get("content", "")) for m in session_after.messages
)
@@ -509,8 +525,8 @@ class TestAutoCompactIntegration:
await loop.close_mcp()
@pytest.mark.asyncio
async def test_multi_paragraph_user_message_preserved(self, tmp_path):
"""Multi-paragraph user messages must be fully preserved after auto-new."""
async def test_runtime_context_markers_not_persisted_for_multi_paragraph_turn(self, tmp_path):
"""Auto-compact resume context must not leak runtime markers into persisted session history."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "old message")
@@ -518,12 +534,9 @@ class TestAutoCompactIntegration:
loop.sessions.save(session)
async def _fake_archive(messages):
return True
return "Summary."
loop.consolidator.archive = _fake_archive
loop.consolidator.get_last_history_entry = lambda: {
"cursor": 1, "timestamp": "2026-01-01 00:00", "content": "Summary.",
}
# Simulate proactive archive completing before message arrives
await loop.auto_compact._archive("cli:test")
@@ -535,16 +548,11 @@ class TestAutoCompactIntegration:
await loop._process_message(msg)
session_after = loop.sessions.get_or_create("cli:test")
user_msgs = [m for m in session_after.messages if m.get("role") == "user"]
assert len(user_msgs) >= 1
# All three paragraphs must be preserved
persisted = user_msgs[-1]["content"]
assert "Paragraph one" in persisted
assert "Paragraph two" in persisted
assert "Paragraph three" in persisted
# No runtime context markers in persisted message
assert "[Runtime Context" not in persisted
assert "[/Runtime Context]" not in persisted
assert any(m.get("content") == "old message" for m in session_after.messages)
for persisted in session_after.messages:
content = str(persisted.get("content", ""))
assert "[Runtime Context" not in content
assert "[/Runtime Context]" not in content
await loop.close_mcp()
@@ -552,9 +560,12 @@ class TestProactiveAutoCompact:
"""Test proactive auto-new on idle ticks (TimeoutError path in run loop)."""
@staticmethod
async def _run_check_expired(loop):
async def _run_check_expired(loop, active_session_keys=()):
"""Helper: run check_expired via callback and wait for background tasks."""
loop.auto_compact.check_expired(loop._schedule_background)
loop.auto_compact.check_expired(
loop._schedule_background,
active_session_keys=active_session_keys,
)
await asyncio.sleep(0.1)
@pytest.mark.asyncio
@@ -577,8 +588,7 @@ class TestProactiveAutoCompact:
"""Expired session should be archived during idle tick."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "old message")
session.add_message("assistant", "old response")
_add_turns(session, 5, prefix="old")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
@@ -586,17 +596,14 @@ class TestProactiveAutoCompact:
async def _fake_archive(messages):
archived_messages.extend(messages)
return True
return "User chatted about old things."
loop.consolidator.archive = _fake_archive
loop.consolidator.get_last_history_entry = lambda: {
"cursor": 1, "timestamp": "2026-01-01 00:00", "content": "User chatted about old things.",
}
await self._run_check_expired(loop)
session_after = loop.sessions.get_or_create("cli:test")
assert len(session_after.messages) == 0
assert len(session_after.messages) == loop.auto_compact._RECENT_SUFFIX_MESSAGES
assert len(archived_messages) == 2
entry = loop.auto_compact._summaries.get("cli:test")
assert entry is not None
@@ -622,7 +629,7 @@ class TestProactiveAutoCompact:
"""Should not archive the same session twice if already in progress."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "old message")
_add_turns(session, 6, prefix="old")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
@@ -635,7 +642,7 @@ class TestProactiveAutoCompact:
archive_count += 1
started.set()
await block_forever.wait()
return True
return "Summary."
loop.consolidator.archive = _slow_archive
@@ -659,7 +666,7 @@ class TestProactiveAutoCompact:
"""Proactive archive failure should be caught and not block future ticks."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "old message")
_add_turns(session, 6, prefix="old")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
@@ -688,7 +695,7 @@ class TestProactiveAutoCompact:
async def _fake_archive(messages):
nonlocal archive_called
archive_called = True
return True
return "Summary."
loop.consolidator.archive = _fake_archive
@@ -698,12 +705,11 @@ class TestProactiveAutoCompact:
await loop.close_mcp()
@pytest.mark.asyncio
async def test_no_reschedule_after_successful_archive(self, tmp_path):
"""Already-archived session should NOT be re-scheduled on subsequent ticks."""
async def test_skip_expired_session_with_active_agent_task(self, tmp_path):
"""Expired session with an active agent task should NOT be archived."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "old message")
session.add_message("assistant", "old response")
_add_turns(session, 6, prefix="old")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
@@ -712,12 +718,102 @@ class TestProactiveAutoCompact:
async def _fake_archive(messages):
nonlocal archive_count
archive_count += 1
return True
return "Summary."
loop.consolidator.archive = _fake_archive
# Simulate an active agent task for this session
await self._run_check_expired(loop, active_session_keys={"cli:test"})
assert archive_count == 0
session_after = loop.sessions.get_or_create("cli:test")
assert len(session_after.messages) == 12 # All messages preserved
await loop.close_mcp()
@pytest.mark.asyncio
async def test_archive_after_active_task_completes(self, tmp_path):
"""Session should be archived on next tick after active task completes."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
_add_turns(session, 6, prefix="old")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
archive_count = 0
async def _fake_archive(messages):
nonlocal archive_count
archive_count += 1
return "Summary."
loop.consolidator.archive = _fake_archive
# First tick: active task, skip
await self._run_check_expired(loop, active_session_keys={"cli:test"})
assert archive_count == 0
# Second tick: task completed, should archive
await self._run_check_expired(loop)
assert archive_count == 1
await loop.close_mcp()
@pytest.mark.asyncio
async def test_partial_active_set_only_archives_inactive_expired(self, tmp_path):
"""With multiple sessions, only the expired+inactive one should be archived."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
# Session A: expired, no active task -> should be archived
s1 = loop.sessions.get_or_create("cli:expired_idle")
_add_turns(s1, 6, prefix="old_a")
s1.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(s1)
# Session B: expired, has active task -> should be skipped
s2 = loop.sessions.get_or_create("cli:expired_active")
_add_turns(s2, 6, prefix="old_b")
s2.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(s2)
# Session C: recent, no active task -> should be skipped
s3 = loop.sessions.get_or_create("cli:recent")
s3.add_message("user", "recent")
loop.sessions.save(s3)
archive_count = 0
async def _fake_archive(messages):
nonlocal archive_count
archive_count += 1
return "Summary."
loop.consolidator.archive = _fake_archive
await self._run_check_expired(loop, active_session_keys={"cli:expired_active"})
assert archive_count == 1
s1_after = loop.sessions.get_or_create("cli:expired_idle")
assert len(s1_after.messages) == loop.auto_compact._RECENT_SUFFIX_MESSAGES
s2_after = loop.sessions.get_or_create("cli:expired_active")
assert len(s2_after.messages) == 12 # Preserved
s3_after = loop.sessions.get_or_create("cli:recent")
assert len(s3_after.messages) == 1 # Preserved
await loop.close_mcp()
@pytest.mark.asyncio
async def test_no_reschedule_after_successful_archive(self, tmp_path):
"""Already-archived session should NOT be re-scheduled on subsequent ticks."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
_add_turns(session, 5, prefix="old")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
archive_count = 0
async def _fake_archive(messages):
nonlocal archive_count
archive_count += 1
return "Summary."
loop.consolidator.archive = _fake_archive
loop.consolidator.get_last_history_entry = lambda: {
"cursor": 1, "timestamp": "2026-01-01 00:00", "content": "Summary.",
}
# First tick: archives the session
await self._run_check_expired(loop)
@@ -741,7 +837,7 @@ class TestProactiveAutoCompact:
async def _fake_archive(messages):
nonlocal archive_count
archive_count += 1
return True
return "Summary."
loop.consolidator.archive = _fake_archive
@@ -759,8 +855,7 @@ class TestProactiveAutoCompact:
"""After successful compact + user sends new messages + idle again, should compact again."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "first conversation")
session.add_message("assistant", "first response")
_add_turns(session, 5, prefix="first")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
@@ -769,12 +864,9 @@ class TestProactiveAutoCompact:
async def _fake_archive(messages):
nonlocal archive_count
archive_count += 1
return True
return "Summary."
loop.consolidator.archive = _fake_archive
loop.consolidator.get_last_history_entry = lambda: {
"cursor": 1, "timestamp": "2026-01-01 00:00", "content": "Summary.",
}
# First compact cycle
await loop.auto_compact._archive("cli:test")
@@ -804,18 +896,14 @@ class TestSummaryPersistence:
"""After archive, _last_summary should be in session metadata."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "hello")
session.add_message("assistant", "hi there")
_add_turns(session, 6, prefix="hello")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
async def _fake_archive(messages):
return True
return "User said hello."
loop.consolidator.archive = _fake_archive
loop.consolidator.get_last_history_entry = lambda: {
"cursor": 1, "timestamp": "2026-01-01 00:00", "content": "User said hello.",
}
await loop.auto_compact._archive("cli:test")
@@ -832,19 +920,15 @@ class TestSummaryPersistence:
"""Summary should be recovered from metadata when _summaries is empty (simulates restart)."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "hello")
session.add_message("assistant", "hi there")
_add_turns(session, 6, prefix="hello")
last_active = datetime.now() - timedelta(minutes=20)
session.updated_at = last_active
loop.sessions.save(session)
async def _fake_archive(messages):
return True
return "User said hello."
loop.consolidator.archive = _fake_archive
loop.consolidator.get_last_history_entry = lambda: {
"cursor": 1, "timestamp": "2026-01-01 00:00", "content": "User said hello.",
}
# Archive
await loop.auto_compact._archive("cli:test")
@@ -855,6 +939,7 @@ class TestSummaryPersistence:
# prepare_session should recover summary from metadata
reloaded = loop.sessions.get_or_create("cli:test")
assert len(reloaded.messages) == loop.auto_compact._RECENT_SUFFIX_MESSAGES
_, summary = loop.auto_compact.prepare_session(reloaded, "cli:test")
assert summary is not None
@@ -869,17 +954,14 @@ class TestSummaryPersistence:
"""_last_summary should be removed from metadata after being consumed."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "hello")
_add_turns(session, 6, prefix="hello")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
async def _fake_archive(messages):
return True
return "Summary."
loop.consolidator.archive = _fake_archive
loop.consolidator.get_last_history_entry = lambda: {
"cursor": 1, "timestamp": "2026-01-01 00:00", "content": "Summary.",
}
await loop.auto_compact._archive("cli:test")
@@ -903,17 +985,14 @@ class TestSummaryPersistence:
"""In-memory _summaries path should also clean up _last_summary from metadata."""
loop = _make_loop(tmp_path, session_ttl_minutes=15)
session = loop.sessions.get_or_create("cli:test")
session.add_message("user", "hello")
_add_turns(session, 6, prefix="hello")
session.updated_at = datetime.now() - timedelta(minutes=20)
loop.sessions.save(session)
async def _fake_archive(messages):
return True
return "Summary."
loop.consolidator.archive = _fake_archive
loop.consolidator.get_last_history_entry = lambda: {
"cursor": 1, "timestamp": "2026-01-01 00:00", "content": "Summary.",
}
await loop.auto_compact._archive("cli:test")
+179
View File
@@ -0,0 +1,179 @@
"""Tests for the configurable consolidation_ratio feature."""
from unittest.mock import AsyncMock, MagicMock
import pytest
from nanobot.agent.loop import AgentLoop
import nanobot.agent.memory as memory_module
from nanobot.bus.queue import MessageBus
from nanobot.providers.base import GenerationSettings, LLMResponse
def _make_loop(
tmp_path,
*,
estimated_tokens: int = 0,
context_window_tokens: int = 200,
consolidation_ratio: float = 0.5,
) -> AgentLoop:
provider = MagicMock()
provider.get_default_model.return_value = "test-model"
provider.generation = GenerationSettings(max_tokens=0)
provider.estimate_prompt_tokens.return_value = (estimated_tokens, "test-counter")
_response = LLMResponse(content="ok", tool_calls=[])
provider.chat_with_retry = AsyncMock(return_value=_response)
provider.chat_stream_with_retry = AsyncMock(return_value=_response)
loop = AgentLoop(
bus=MessageBus(),
provider=provider,
workspace=tmp_path,
model="test-model",
context_window_tokens=context_window_tokens,
consolidation_ratio=consolidation_ratio,
)
loop.tools.get_definitions = MagicMock(return_value=[])
loop.consolidator._SAFETY_BUFFER = 0
return loop
@pytest.mark.asyncio
async def test_default_ratio_uses_half_budget(tmp_path, monkeypatch) -> None:
"""With ratio=0.5 (default), target should be half of budget."""
loop = _make_loop(tmp_path, context_window_tokens=200, consolidation_ratio=0.5)
loop.consolidator.archive = AsyncMock(return_value=True) # type: ignore[method-assign]
session = loop.sessions.get_or_create("cli:test")
session.messages = [
{"role": "user", "content": "u1", "timestamp": "2026-01-01T00:00:00"},
{"role": "assistant", "content": "a1", "timestamp": "2026-01-01T00:00:01"},
{"role": "user", "content": "u2", "timestamp": "2026-01-01T00:00:02"},
{"role": "assistant", "content": "a2", "timestamp": "2026-01-01T00:00:03"},
{"role": "user", "content": "u3", "timestamp": "2026-01-01T00:00:04"},
{"role": "assistant", "content": "a3", "timestamp": "2026-01-01T00:00:05"},
{"role": "user", "content": "u4", "timestamp": "2026-01-01T00:00:06"},
]
loop.sessions.save(session)
# budget = 200 - 0 (max_tokens) - 0 (safety_buffer) = 200
# target = int(200 * 0.5) = 100
# estimated must be >= budget to trigger consolidation
call_count = [0]
def mock_estimate(_session):
call_count[0] += 1
if call_count[0] == 1:
return (250, "test")
return (90, "test")
loop.consolidator.estimate_session_prompt_tokens = mock_estimate # type: ignore[method-assign]
monkeypatch.setattr(memory_module, "estimate_message_tokens", lambda _m: 100)
await loop.consolidator.maybe_consolidate_by_tokens(session)
# 250 >= 200 (budget, triggers) → 250 > 100 (target) → archive → 90 < 100, stops.
assert loop.consolidator.archive.await_count == 1
@pytest.mark.asyncio
async def test_low_ratio_aggressively_consolidates(tmp_path, monkeypatch) -> None:
"""With ratio=0.1, target is only 10% of budget — more rounds of archiving."""
loop = _make_loop(tmp_path, context_window_tokens=1000, consolidation_ratio=0.1)
loop.consolidator.archive = AsyncMock(return_value=True) # type: ignore[method-assign]
session = loop.sessions.get_or_create("cli:test")
# Interleave user/assistant so pick_consolidation_boundary can find boundaries
session.messages = []
for i in range(10):
session.messages.append({"role": "user", "content": f"u{i}", "timestamp": f"2026-01-01T00:00:{i:02d}"})
session.messages.append({"role": "assistant", "content": f"a{i}", "timestamp": f"2026-01-01T00:00:{i:02d}"})
loop.sessions.save(session)
# budget = 1000, target = int(1000 * 0.1) = 100
call_count = [0]
def mock_estimate(_session):
call_count[0] += 1
if call_count[0] == 1:
return (1200, "test")
if call_count[0] == 2:
return (800, "test")
if call_count[0] == 3:
return (400, "test")
return (50, "test")
loop.consolidator.estimate_session_prompt_tokens = mock_estimate # type: ignore[method-assign]
monkeypatch.setattr(memory_module, "estimate_message_tokens", lambda _m: 100)
await loop.consolidator.maybe_consolidate_by_tokens(session)
# With low ratio, more rounds needed to reach target; at least 2 rounds
assert loop.consolidator.archive.await_count >= 2
@pytest.mark.asyncio
async def test_high_ratio_preserves_more_history(tmp_path, monkeypatch) -> None:
"""With ratio=0.9, target is 90% of budget — consolidation stops sooner."""
loop = _make_loop(tmp_path, context_window_tokens=200, consolidation_ratio=0.9)
loop.consolidator.archive = AsyncMock(return_value=True) # type: ignore[method-assign]
session = loop.sessions.get_or_create("cli:test")
session.messages = [
{"role": "user", "content": "u1", "timestamp": "2026-01-01T00:00:00"},
{"role": "assistant", "content": "a1", "timestamp": "2026-01-01T00:00:01"},
{"role": "user", "content": "u2", "timestamp": "2026-01-01T00:00:02"},
{"role": "assistant", "content": "a2", "timestamp": "2026-01-01T00:00:03"},
{"role": "user", "content": "u3", "timestamp": "2026-01-01T00:00:04"},
{"role": "assistant", "content": "a3", "timestamp": "2026-01-01T00:00:05"},
{"role": "user", "content": "u4", "timestamp": "2026-01-01T00:00:06"},
]
loop.sessions.save(session)
# budget = 200, target = int(200 * 0.9) = 180
call_count = [0]
def mock_estimate(_session):
call_count[0] += 1
if call_count[0] == 1:
return (300, "test")
return (175, "test")
loop.consolidator.estimate_session_prompt_tokens = mock_estimate # type: ignore[method-assign]
monkeypatch.setattr(memory_module, "estimate_message_tokens", lambda _m: 100)
await loop.consolidator.maybe_consolidate_by_tokens(session)
# 300 >= 200 (triggers) → 300 > 180 → archive → 175 < 180 → stop
assert loop.consolidator.archive.await_count == 1
@pytest.mark.asyncio
async def test_ratio_propagated_from_config_schema() -> None:
"""Verify consolidation_ratio is parsed from config with camelCase alias."""
from nanobot.config.schema import AgentDefaults
# Default
defaults = AgentDefaults()
assert defaults.consolidation_ratio == 0.5
# camelCase alias
defaults = AgentDefaults.model_validate({"consolidationRatio": 0.3})
assert defaults.consolidation_ratio == 0.3
# Serialization uses alias
dumped = defaults.model_dump(by_alias=True)
assert dumped["consolidationRatio"] == 0.3
@pytest.mark.asyncio
async def test_ratio_validation_rejects_out_of_range() -> None:
"""Invalid ratio values should be rejected by validation."""
from pydantic import ValidationError
from nanobot.config.schema import AgentDefaults
with pytest.raises(ValidationError):
AgentDefaults(consolidation_ratio=0.05)
with pytest.raises(ValidationError):
AgentDefaults(consolidation_ratio=1.0)
+92 -3
View File
@@ -46,7 +46,7 @@ class TestConsolidatorSummarize:
{"role": "assistant", "content": "Done, fixed the race condition."},
]
result = await consolidator.archive(messages)
assert result is True
assert result == "User fixed a bug in the auth module."
entries = store.read_unprocessed_history(since_cursor=0)
assert len(entries) == 1
@@ -55,14 +55,54 @@ class TestConsolidatorSummarize:
mock_provider.chat_with_retry.side_effect = Exception("API error")
messages = [{"role": "user", "content": "hello"}]
result = await consolidator.archive(messages)
assert result is True # always succeeds
assert result is None # no summary on raw dump fallback
entries = store.read_unprocessed_history(since_cursor=0)
assert len(entries) == 1
assert "[RAW]" in entries[0]["content"]
async def test_summarize_skips_empty_messages(self, consolidator):
result = await consolidator.archive([])
assert result is False
assert result is None
class TestConsolidatorArchiveErrorHandling:
"""archive() must fall back to raw_archive when the LLM returns an error
response (finish_reason == 'error'), e.g. overloaded / quota exceeded.
See https://github.com/HKUDS/nanobot/issues/3244
"""
async def test_archive_falls_back_on_error_finish_reason(self, consolidator, mock_provider, store):
"""LLM returning finish_reason='error' should trigger raw_archive, not write error text."""
mock_provider.chat_with_retry.return_value = MagicMock(
content="Error: {'type': 'error', 'error': {'type': 'overloaded_error', 'message': 'overloaded_error (529)'}}",
finish_reason="error",
)
messages = [
{"role": "user", "content": "fix the auth bug"},
{"role": "assistant", "content": "Done, fixed the race condition."},
]
result = await consolidator.archive(messages)
assert result is None
entries = store.read_unprocessed_history(since_cursor=0)
assert len(entries) == 1
assert "[RAW]" in entries[0]["content"]
assert "Error:" not in entries[0]["content"]
async def test_archive_preserves_summary_on_success(self, consolidator, mock_provider, store):
"""Normal LLM response should still produce a proper summary entry."""
mock_provider.chat_with_retry.return_value = MagicMock(
content="User fixed a bug in the auth module.",
finish_reason="stop",
)
messages = [
{"role": "user", "content": "fix the auth bug"},
{"role": "assistant", "content": "Done."},
]
result = await consolidator.archive(messages)
assert result == "User fixed a bug in the auth module."
entries = store.read_unprocessed_history(since_cursor=0)
assert len(entries) == 1
assert "[RAW]" not in entries[0]["content"]
class TestConsolidatorTokenBudget:
@@ -76,3 +116,52 @@ class TestConsolidatorTokenBudget:
consolidator.archive = AsyncMock(return_value=True)
await consolidator.maybe_consolidate_by_tokens(session)
consolidator.archive.assert_not_called()
async def test_chunk_cap_preserves_user_turn_boundary(self, consolidator):
"""Chunk cap should rewind to the last user boundary within the cap."""
consolidator._SAFETY_BUFFER = 0
session = MagicMock()
session.last_consolidated = 0
session.key = "test:key"
session.messages = [
{
"role": "user" if i in {0, 50, 61} else "assistant",
"content": f"m{i}",
}
for i in range(70)
]
consolidator.estimate_session_prompt_tokens = MagicMock(
side_effect=[(1200, "tiktoken"), (400, "tiktoken")]
)
consolidator.pick_consolidation_boundary = MagicMock(return_value=(61, 999))
consolidator.archive = AsyncMock(return_value=True)
await consolidator.maybe_consolidate_by_tokens(session)
archived_chunk = consolidator.archive.await_args.args[0]
assert len(archived_chunk) == 50
assert archived_chunk[0]["content"] == "m0"
assert archived_chunk[-1]["content"] == "m49"
assert session.last_consolidated == 50
async def test_chunk_cap_skips_when_no_user_boundary_within_cap(self, consolidator):
"""If the cap would cut mid-turn, consolidation should skip that round."""
consolidator._SAFETY_BUFFER = 0
session = MagicMock()
session.last_consolidated = 0
session.key = "test:key"
session.messages = [
{
"role": "user" if i in {0, 61} else "assistant",
"content": f"m{i}",
}
for i in range(70)
]
consolidator.estimate_session_prompt_tokens = MagicMock(return_value=(1200, "tiktoken"))
consolidator.pick_consolidation_boundary = MagicMock(return_value=(61, 999))
consolidator.archive = AsyncMock(return_value=True)
await consolidator.maybe_consolidate_by_tokens(session)
consolidator.archive.assert_not_awaited()
assert session.last_consolidated == 0
+171
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import re
from datetime import datetime as real_datetime
from importlib.resources import files as pkg_files
from pathlib import Path
@@ -86,6 +87,124 @@ def test_runtime_context_is_separate_untrusted_user_message(tmp_path) -> None:
assert "Return exactly: OK" in user_content
def test_unprocessed_history_injected_into_system_prompt(tmp_path) -> None:
"""Entries in history.jsonl not yet consumed by Dream appear with timestamps."""
workspace = _make_workspace(tmp_path)
builder = ContextBuilder(workspace)
builder.memory.append_history("User asked about weather in Tokyo")
builder.memory.append_history("Agent fetched forecast via web_search")
prompt = builder.build_system_prompt()
assert "# Recent History" in prompt
assert "User asked about weather in Tokyo" in prompt
assert "Agent fetched forecast via web_search" in prompt
assert re.search(r"\[\d{4}-\d{2}-\d{2} \d{2}:\d{2}\]", prompt)
def test_recent_history_capped_at_max(tmp_path) -> None:
"""Only the most recent _MAX_RECENT_HISTORY entries are injected."""
workspace = _make_workspace(tmp_path)
builder = ContextBuilder(workspace)
for i in range(builder._MAX_RECENT_HISTORY + 20):
builder.memory.append_history(f"entry-{i}")
prompt = builder.build_system_prompt()
assert "entry-0" not in prompt
assert "entry-19" not in prompt
assert f"entry-{builder._MAX_RECENT_HISTORY + 19}" in prompt
def test_no_recent_history_when_dream_has_processed_all(tmp_path) -> None:
"""If Dream has consumed everything, no Recent History section should appear."""
workspace = _make_workspace(tmp_path)
builder = ContextBuilder(workspace)
cursor = builder.memory.append_history("already processed entry")
builder.memory.set_last_dream_cursor(cursor)
prompt = builder.build_system_prompt()
assert "# Recent History" not in prompt
def test_partial_dream_processing_shows_only_remainder(tmp_path) -> None:
"""When Dream has processed some entries, only the unprocessed ones appear."""
workspace = _make_workspace(tmp_path)
builder = ContextBuilder(workspace)
c1 = builder.memory.append_history("old conversation about Python")
c2 = builder.memory.append_history("old conversation about Rust")
builder.memory.append_history("recent question about Docker")
builder.memory.append_history("recent question about K8s")
builder.memory.set_last_dream_cursor(c2)
prompt = builder.build_system_prompt()
assert "# Recent History" in prompt
assert "old conversation about Python" not in prompt
assert "old conversation about Rust" not in prompt
assert "recent question about Docker" in prompt
assert "recent question about K8s" in prompt
def test_execution_rules_in_system_prompt(tmp_path) -> None:
"""New execution rules should appear in the system prompt."""
workspace = _make_workspace(tmp_path)
builder = ContextBuilder(workspace)
prompt = builder.build_system_prompt()
assert "Act, don't narrate" in prompt
assert "Read before you write" in prompt
assert "verify the result" in prompt
def test_channel_format_hint_telegram(tmp_path) -> None:
"""Telegram channel should get messaging-app format hint."""
workspace = _make_workspace(tmp_path)
builder = ContextBuilder(workspace)
prompt = builder.build_system_prompt(channel="telegram")
assert "Format Hint" in prompt
assert "messaging app" in prompt
def test_channel_format_hint_whatsapp(tmp_path) -> None:
"""WhatsApp should get plain-text format hint."""
workspace = _make_workspace(tmp_path)
builder = ContextBuilder(workspace)
prompt = builder.build_system_prompt(channel="whatsapp")
assert "Format Hint" in prompt
assert "plain text only" in prompt
def test_channel_format_hint_absent_for_unknown(tmp_path) -> None:
"""Unknown or None channel should not inject a format hint."""
workspace = _make_workspace(tmp_path)
builder = ContextBuilder(workspace)
prompt = builder.build_system_prompt(channel=None)
assert "Format Hint" not in prompt
prompt2 = builder.build_system_prompt(channel="feishu")
assert "Format Hint" not in prompt2
def test_build_messages_passes_channel_to_system_prompt(tmp_path) -> None:
"""build_messages should pass channel through to build_system_prompt."""
workspace = _make_workspace(tmp_path)
builder = ContextBuilder(workspace)
messages = builder.build_messages(
history=[], current_message="hi",
channel="telegram", chat_id="123",
)
system = messages[0]["content"]
assert "Format Hint" in system
assert "messaging app" in system
def test_subagent_result_does_not_create_consecutive_assistant_messages(tmp_path) -> None:
workspace = _make_workspace(tmp_path)
builder = ContextBuilder(workspace)
@@ -100,3 +219,55 @@ def test_subagent_result_does_not_create_consecutive_assistant_messages(tmp_path
for left, right in zip(messages, messages[1:]):
assert not (left.get("role") == right.get("role") == "assistant")
def test_always_skills_excluded_from_skills_index(tmp_path) -> None:
"""Always skills should appear in Active Skills but NOT in the skills index."""
workspace = _make_workspace(tmp_path)
builder = ContextBuilder(workspace)
prompt = builder.build_system_prompt()
# memory skill should be in Active Skills section
assert "# Active Skills" in prompt
assert "### Skill: memory" in prompt
# memory skill should NOT appear in the skills index
skills_section = prompt.split("# Skills\n", 1)
if len(skills_section) > 1:
index_text = skills_section[1].split("\n\n---")[0]
assert "**memory**" not in index_text
def test_template_memory_md_is_skipped(tmp_path) -> None:
"""MEMORY.md matching the bundled template should not inject the Memory section."""
workspace = _make_workspace(tmp_path)
from nanobot.utils.helpers import sync_workspace_templates
sync_workspace_templates(workspace, silent=True)
builder = ContextBuilder(workspace)
prompt = builder.build_system_prompt()
# The "# Memory\n\n## Long-term Memory" block is produced only by
# build_system_prompt() when MEMORY.md is injected. The memory skill
# also contains "# Memory" but is followed by "## Structure", not
# "## Long-term Memory".
assert "# Memory\n\n## Long-term Memory" not in prompt
assert "This file is automatically updated by nanobot" not in prompt
def test_customized_memory_md_is_injected(tmp_path) -> None:
"""A Dream-populated MEMORY.md should be injected normally."""
workspace = _make_workspace(tmp_path)
from nanobot.utils.helpers import sync_workspace_templates
sync_workspace_templates(workspace, silent=True)
(workspace / "memory" / "MEMORY.md").write_text(
"# Long-term Memory\n\nUser prefers dark mode.\n", encoding="utf-8"
)
builder = ContextBuilder(workspace)
prompt = builder.build_system_prompt()
assert "# Memory\n\n## Long-term Memory" in prompt
assert "User prefers dark mode" in prompt
+162 -1
View File
@@ -2,10 +2,12 @@
import pytest
from unittest.mock import AsyncMock, MagicMock
from unittest.mock import AsyncMock, MagicMock, patch
from nanobot.agent.memory import Dream, MemoryStore
from nanobot.agent.runner import AgentRunResult
from nanobot.agent.skills import BUILTIN_SKILLS_DIR
from nanobot.utils.gitstore import LineAge
@pytest.fixture
@@ -95,3 +97,162 @@ class TestDreamRun:
entries = store.read_unprocessed_history(since_cursor=0)
assert all(e["cursor"] > 0 for e in entries)
async def test_skill_phase_uses_builtin_skill_creator_path(self, dream, mock_provider, mock_runner, store):
"""Dream should point skill creation guidance at the builtin skill-creator template."""
store.append_history("Repeated workflow one")
store.append_history("Repeated workflow two")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKILL] test-skill: test description")
mock_runner.run = AsyncMock(return_value=_make_run_result())
await dream.run()
spec = mock_runner.run.call_args[0][0]
system_prompt = spec.initial_messages[0]["content"]
expected = str(BUILTIN_SKILLS_DIR / "skill-creator" / "SKILL.md")
assert expected in system_prompt
async def test_skill_write_tool_accepts_workspace_relative_skill_path(self, dream, store):
"""Dream skill creation should allow skills/<name>/SKILL.md relative to workspace root."""
write_tool = dream._tools.get("write_file")
assert write_tool is not None
result = await write_tool.execute(
path="skills/test-skill/SKILL.md",
content="---\nname: test-skill\ndescription: Test\n---\n",
)
assert "Successfully wrote" in result
assert (store.workspace / "skills" / "test-skill" / "SKILL.md").exists()
async def test_phase1_prompt_includes_line_age_annotations(self, dream, mock_provider, mock_runner, store):
"""Phase 1 prompt should have per-line age suffixes in MEMORY.md when git is available."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
# Init git so line_ages works
store.git.init()
store.git.auto_commit("initial memory state")
await dream.run()
# The MEMORY.md section should not crash and should contain the memory content
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
assert "## Current MEMORY.md" in user_msg
async def test_phase1_annotates_only_memory_not_soul_or_user(self, dream, mock_provider, mock_runner, store):
"""SOUL.md and USER.md should never have age annotations — they are permanent."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
store.git.init()
store.git.auto_commit("initial state")
await dream.run()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
# The ← suffix should only appear in MEMORY.md section
memory_section = user_msg.split("## Current MEMORY.md")[1].split("## Current SOUL.md")[0]
soul_section = user_msg.split("## Current SOUL.md")[1].split("## Current USER.md")[0]
user_section = user_msg.split("## Current USER.md")[1]
# SOUL and USER should not contain age arrows
assert "\u2190" not in soul_section
assert "\u2190" not in user_section
async def test_phase1_prompt_works_without_git(self, dream, mock_provider, mock_runner, store):
"""Phase 1 should work fine even if git is not initialized (no age annotations)."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
await dream.run()
# Should still succeed — just without age annotations
mock_provider.chat_with_retry.assert_called_once()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
assert "## Current MEMORY.md" in user_msg
async def test_phase1_prompt_carries_age_suffix_for_stale_lines(
self, dream, mock_provider, mock_runner, store,
):
"""End-to-end: ages >14d must appear verbatim in the LLM prompt, ages ≤14d must not."""
# MEMORY.md fixture has 2 non-blank lines ("# Memory" and "- Project X active").
# Inject four ages to cover threshold boundaries: >14 suffix, ==14 no suffix, <14 no suffix.
store.write_memory("# Memory\n- Project X active\n- fresh item\n- edge case line")
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
fake_ages = [
LineAge(age_days=30), # "# Memory" → should get ← 30d
LineAge(age_days=20), # "- Project X..." → should get ← 20d
LineAge(age_days=14), # "- fresh item" → ==14, threshold is strictly >14, no suffix
LineAge(age_days=5), # "- edge case..." → no suffix
]
with patch.object(store.git, "line_ages", return_value=fake_ages):
await dream.run()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split("## Current SOUL.md")[0]
assert "\u2190 30d" in memory_section
assert "\u2190 20d" in memory_section
assert "\u2190 14d" not in memory_section
assert "\u2190 5d" not in memory_section
async def test_phase1_skips_annotation_when_disabled(
self, dream, mock_provider, mock_runner, store,
):
"""`annotate_line_ages=False` must bypass the git lookup entirely and keep MEMORY.md raw."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
dream.annotate_line_ages = False
# line_ages must be bypassed entirely — verify with a spy rather than a
# raising side_effect, because _annotate_with_ages catches Exception
# (which swallows AssertionError) and would hide an accidental call.
with patch.object(store.git, "line_ages") as mock_line_ages:
await dream.run()
mock_line_ages.assert_not_called()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
assert "\u2190" not in user_msg
async def test_phase1_skips_annotation_on_line_ages_length_mismatch(
self, dream, mock_provider, mock_runner, store,
):
"""If ages length != lines length (dirty working tree), skip annotation instead of mis-tagging."""
# MEMORY.md has 2 non-blank lines but we hand back only 1 age → mismatch.
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
with patch.object(store.git, "line_ages", return_value=[LineAge(age_days=999)]):
await dream.run()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split("## Current SOUL.md")[0]
# No age arrow at all — we refused to annotate rather than tag the wrong line.
assert "\u2190" not in memory_section
async def test_phase1_prompt_uses_threshold_from_template_var(
self, dream, mock_provider, mock_runner, store,
):
"""System prompt should reference the stale-threshold constant, not a hardcoded 14."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
await dream.run()
system_msg = mock_provider.chat_with_retry.call_args.kwargs["messages"][0]["content"]
# The template renders with stale_threshold_days=14 → LLM must see "N>14"
assert "N>14" in system_msg
+16 -11
View File
@@ -184,17 +184,22 @@ def test_stale_extra_content_in_tool_calls_survives_sanitize() -> None:
with patch("nanobot.providers.openai_compat_provider.AsyncOpenAI"):
provider = OpenAICompatProvider()
messages = [{
"role": "assistant",
"content": None,
"tool_calls": [{
"id": "call_1",
"type": "function",
"function": {"name": "fn", "arguments": "{}"},
"extra_content": GEMINI_EXTRA,
}],
}]
messages = [
{"role": "user", "content": "hi"},
{
"role": "assistant",
"content": None,
"tool_calls": [{
"id": "call_1",
"type": "function",
"function": {"name": "fn", "arguments": "{}"},
"extra_content": GEMINI_EXTRA,
}],
},
{"role": "tool", "content": "ok", "tool_call_id": "call_1"},
{"role": "user", "content": "thanks"},
]
sanitized = provider._sanitize_messages(messages)
assert sanitized[0]["tool_calls"][0]["extra_content"] == GEMINI_EXTRA
assert sanitized[1]["tool_calls"][0]["extra_content"] == GEMINI_EXTRA
+29
View File
@@ -232,6 +232,35 @@ async def test_composite_empty_hooks_no_ops():
assert hook.finalize_content(ctx, "test") == "test"
@pytest.mark.asyncio
async def test_composite_supports_legacy_hook_init_without_super():
calls: list[str] = []
class LegacyHook(AgentHook):
def __init__(self, label: str) -> None:
self.label = label
async def before_iteration(self, context: AgentHookContext) -> None:
calls.append(self.label)
hook = CompositeHook([LegacyHook("legacy")])
await hook.before_iteration(_ctx())
assert calls == ["legacy"]
@pytest.mark.asyncio
async def test_composite_can_wrap_another_composite():
calls: list[str] = []
class Inner(AgentHook):
async def before_iteration(self, context: AgentHookContext) -> None:
calls.append("inner")
hook = CompositeHook([CompositeHook([Inner()])])
await hook.before_iteration(_ctx())
assert calls == ["inner"]
# ---------------------------------------------------------------------------
# Integration: AgentLoop with extra hooks
# ---------------------------------------------------------------------------
+217
View File
@@ -1,5 +1,13 @@
import asyncio
from pathlib import Path
from unittest.mock import AsyncMock, MagicMock
import pytest
from nanobot.agent.context import ContextBuilder
from nanobot.agent.loop import AgentLoop
from nanobot.bus.events import InboundMessage
from nanobot.bus.queue import MessageBus
from nanobot.session.manager import Session
@@ -11,6 +19,12 @@ def _mk_loop() -> AgentLoop:
return loop
def _make_full_loop(tmp_path: Path) -> AgentLoop:
provider = MagicMock()
provider.get_default_model.return_value = "test-model"
return AgentLoop(bus=MessageBus(), provider=provider, workspace=tmp_path, model="test-model")
def test_save_turn_skips_multimodal_user_when_only_runtime_context() -> None:
loop = _mk_loop()
session = Session(key="test:runtime-only")
@@ -200,3 +214,206 @@ def test_restore_runtime_checkpoint_dedupes_overlapping_tail() -> None:
assert session.messages[0]["role"] == "assistant"
assert session.messages[1]["tool_call_id"] == "call_done"
assert session.messages[2]["tool_call_id"] == "call_pending"
@pytest.mark.asyncio
async def test_process_message_persists_user_message_before_turn_completes(tmp_path: Path) -> None:
loop = _make_full_loop(tmp_path)
loop.consolidator.maybe_consolidate_by_tokens = AsyncMock(return_value=False) # type: ignore[method-assign]
loop._run_agent_loop = AsyncMock(side_effect=RuntimeError("boom")) # type: ignore[method-assign]
msg = InboundMessage(channel="feishu", sender_id="u1", chat_id="c1", content="persist me")
with pytest.raises(RuntimeError, match="boom"):
await loop._process_message(msg)
loop.sessions.invalidate("feishu:c1")
persisted = loop.sessions.get_or_create("feishu:c1")
assert [m["role"] for m in persisted.messages] == ["user"]
assert persisted.messages[0]["content"] == "persist me"
assert persisted.metadata.get(AgentLoop._PENDING_USER_TURN_KEY) is True
assert persisted.updated_at >= persisted.created_at
@pytest.mark.asyncio
async def test_process_message_does_not_duplicate_early_persisted_user_message(tmp_path: Path) -> None:
loop = _make_full_loop(tmp_path)
loop.consolidator.maybe_consolidate_by_tokens = AsyncMock(return_value=False) # type: ignore[method-assign]
loop._run_agent_loop = AsyncMock(return_value=(
"done",
None,
[
{"role": "system", "content": "system"},
{"role": "user", "content": "hello"},
{"role": "assistant", "content": "done"},
],
"stop",
False,
)) # type: ignore[method-assign]
result = await loop._process_message(
InboundMessage(channel="feishu", sender_id="u1", chat_id="c2", content="hello")
)
assert result is not None
assert result.content == "done"
session = loop.sessions.get_or_create("feishu:c2")
assert [
{k: v for k, v in m.items() if k in {"role", "content"}}
for m in session.messages
] == [
{"role": "user", "content": "hello"},
{"role": "assistant", "content": "done"},
]
assert AgentLoop._PENDING_USER_TURN_KEY not in session.metadata
@pytest.mark.asyncio
async def test_next_turn_after_crash_closes_pending_user_turn_before_new_input(tmp_path: Path) -> None:
loop = _make_full_loop(tmp_path)
loop.consolidator.maybe_consolidate_by_tokens = AsyncMock(return_value=False) # type: ignore[method-assign]
loop.provider.chat_with_retry = AsyncMock(return_value=MagicMock()) # unused because _run_agent_loop is stubbed
session = loop.sessions.get_or_create("feishu:c3")
session.add_message("user", "old question")
session.metadata[AgentLoop._PENDING_USER_TURN_KEY] = True
loop.sessions.save(session)
loop._run_agent_loop = AsyncMock(return_value=(
"new answer",
None,
[
{"role": "system", "content": "system"},
{"role": "user", "content": "old question"},
{"role": "assistant", "content": "Error: Task interrupted before a response was generated."},
{"role": "user", "content": "new question"},
{"role": "assistant", "content": "new answer"},
],
"stop",
False,
)) # type: ignore[method-assign]
result = await loop._process_message(
InboundMessage(channel="feishu", sender_id="u1", chat_id="c3", content="new question")
)
assert result is not None
assert result.content == "new answer"
session = loop.sessions.get_or_create("feishu:c3")
assert [
{k: v for k, v in m.items() if k in {"role", "content"}}
for m in session.messages
] == [
{"role": "user", "content": "old question"},
{"role": "assistant", "content": "Error: Task interrupted before a response was generated."},
{"role": "user", "content": "new question"},
{"role": "assistant", "content": "new answer"},
]
assert AgentLoop._PENDING_USER_TURN_KEY not in session.metadata
@pytest.mark.asyncio
async def test_stop_preserves_runtime_checkpoint_for_next_turn(tmp_path: Path) -> None:
from nanobot.command.builtin import cmd_stop
from nanobot.command.router import CommandContext
loop = _make_full_loop(tmp_path)
loop.consolidator.maybe_consolidate_by_tokens = AsyncMock(return_value=False) # type: ignore[method-assign]
checkpoint_saved = asyncio.Event()
async def interrupted_run_agent_loop(_initial_messages, *, session=None, **_kwargs):
assert session is not None
loop._set_runtime_checkpoint(
session,
{
"assistant_message": {
"role": "assistant",
"content": "working",
"tool_calls": [
{
"id": "call_done",
"type": "function",
"function": {"name": "read_file", "arguments": "{}"},
},
{
"id": "call_pending",
"type": "function",
"function": {"name": "exec", "arguments": "{}"},
},
],
},
"completed_tool_results": [
{
"role": "tool",
"tool_call_id": "call_done",
"name": "read_file",
"content": "ok",
}
],
"pending_tool_calls": [
{
"id": "call_pending",
"type": "function",
"function": {"name": "exec", "arguments": "{}"},
}
],
},
)
checkpoint_saved.set()
await asyncio.Event().wait()
loop._run_agent_loop = interrupted_run_agent_loop # type: ignore[method-assign]
first_msg = InboundMessage(channel="feishu", sender_id="u1", chat_id="c4", content="keep progress")
task = asyncio.create_task(loop._process_message(first_msg))
loop._active_tasks[first_msg.session_key] = [task]
await asyncio.wait_for(checkpoint_saved.wait(), timeout=1.0)
stop_msg = InboundMessage(channel="feishu", sender_id="u1", chat_id="c4", content="/stop")
stop_ctx = CommandContext(msg=stop_msg, session=None, key=stop_msg.session_key, raw="/stop", loop=loop)
stop_result = await cmd_stop(stop_ctx)
assert "Stopped 1 task" in stop_result.content
assert task.done()
loop.sessions.invalidate("feishu:c4")
interrupted = loop.sessions.get_or_create("feishu:c4")
assert interrupted.metadata.get(AgentLoop._PENDING_USER_TURN_KEY) is True
assert interrupted.metadata.get(AgentLoop._RUNTIME_CHECKPOINT_KEY) is not None
async def resumed_run_agent_loop(initial_messages, **_kwargs):
return (
"next answer",
None,
[*initial_messages, {"role": "assistant", "content": "next answer"}],
"stop",
False,
)
loop._run_agent_loop = resumed_run_agent_loop # type: ignore[method-assign]
result = await loop._process_message(
InboundMessage(channel="feishu", sender_id="u1", chat_id="c4", content="continue here")
)
assert result is not None
assert result.content == "next answer"
session = loop.sessions.get_or_create("feishu:c4")
assert [
{k: v for k, v in m.items() if k in {"role", "content", "tool_call_id", "name"}}
for m in session.messages
] == [
{"role": "user", "content": "keep progress"},
{"role": "assistant", "content": "working"},
{"role": "tool", "tool_call_id": "call_done", "name": "read_file", "content": "ok"},
{
"role": "tool",
"tool_call_id": "call_pending",
"name": "exec",
"content": "Error: Task interrupted before this tool finished.",
},
{"role": "user", "content": "continue here"},
{"role": "assistant", "content": "next answer"},
]
assert AgentLoop._PENDING_USER_TURN_KEY not in session.metadata
assert AgentLoop._RUNTIME_CHECKPOINT_KEY not in session.metadata
+44
View File
@@ -0,0 +1,44 @@
"""Tests for MCP connection lifecycle in AgentLoop."""
from __future__ import annotations
from unittest.mock import MagicMock
import pytest
from nanobot.agent.loop import AgentLoop
from nanobot.bus.queue import MessageBus
def _make_loop(tmp_path, *, mcp_servers: dict | None = None) -> AgentLoop:
bus = MessageBus()
provider = MagicMock()
provider.get_default_model.return_value = "test-model"
provider.generation.max_tokens = 4096
return AgentLoop(
bus=bus,
provider=provider,
workspace=tmp_path,
model="test-model",
mcp_servers=mcp_servers or {"test": object()},
)
@pytest.mark.asyncio
async def test_connect_mcp_retries_when_no_servers_connect(tmp_path, monkeypatch: pytest.MonkeyPatch):
loop = _make_loop(tmp_path)
attempts = 0
async def _fake_connect(_servers, _registry):
nonlocal attempts
attempts += 1
return {}
monkeypatch.setattr("nanobot.agent.tools.mcp.connect_mcp_servers", _fake_connect)
await loop._connect_mcp()
await loop._connect_mcp()
assert attempts == 2
assert loop._mcp_connected is False
assert loop._mcp_stacks == {}
+23
View File
@@ -79,6 +79,29 @@ class TestHistoryWithCursor:
entries = store.read_unprocessed_history(since_cursor=0)
assert len(entries) == 2
def test_read_unprocessed_skips_entries_without_cursor(self, store):
"""Regression: entries missing the cursor key should be silently skipped."""
store.history_file.write_text(
'{"timestamp": "2026-04-01 10:00", "content": "no cursor"}\n'
'{"cursor": 2, "timestamp": "2026-04-01 10:01", "content": "valid"}\n'
'{"cursor": 3, "timestamp": "2026-04-01 10:02", "content": "also valid"}\n',
encoding="utf-8",
)
entries = store.read_unprocessed_history(since_cursor=0)
assert [e["cursor"] for e in entries] == [2, 3]
def test_next_cursor_falls_back_when_last_entry_has_no_cursor(self, store):
"""Regression: _next_cursor should not KeyError on entries without cursor."""
store.history_file.write_text(
'{"timestamp": "2026-04-01 10:01", "content": "no cursor"}\n',
encoding="utf-8",
)
# Delete .cursor file so _next_cursor falls back to reading JSONL
store._cursor_file.unlink(missing_ok=True)
# Last entry has no cursor — should safely return 1, not KeyError
cursor = store.append_history("new event")
assert cursor == 1
def test_compact_history_drops_oldest(self, tmp_path):
store = MemoryStore(tmp_path, max_history_entries=2)
store.append_history("event 1")
+1414 -34
View File
File diff suppressed because it is too large Load Diff
+147
View File
@@ -250,3 +250,150 @@ def test_list_skills_openclaw_metadata_parsed_for_requirements(
assert entries == [
{"name": "openclaw_skill", "path": str(skill_path), "source": "workspace"},
]
def test_disabled_skills_excluded_from_list(tmp_path: Path) -> None:
workspace = tmp_path / "ws"
ws_skills = workspace / "skills"
ws_skills.mkdir(parents=True)
_write_skill(ws_skills, "alpha", body="# Alpha")
beta_path = _write_skill(ws_skills, "beta", body="# Beta")
builtin = tmp_path / "builtin"
builtin.mkdir()
loader = SkillsLoader(workspace, builtin_skills_dir=builtin, disabled_skills={"alpha"})
entries = loader.list_skills(filter_unavailable=False)
assert len(entries) == 1
assert entries[0]["name"] == "beta"
assert entries[0]["path"] == str(beta_path)
def test_disabled_skills_empty_set_no_effect(tmp_path: Path) -> None:
workspace = tmp_path / "ws"
ws_skills = workspace / "skills"
ws_skills.mkdir(parents=True)
_write_skill(ws_skills, "alpha", body="# Alpha")
_write_skill(ws_skills, "beta", body="# Beta")
builtin = tmp_path / "builtin"
builtin.mkdir()
loader = SkillsLoader(workspace, builtin_skills_dir=builtin, disabled_skills=set())
entries = loader.list_skills(filter_unavailable=False)
assert len(entries) == 2
def test_disabled_skills_excluded_from_build_skills_summary(tmp_path: Path) -> None:
workspace = tmp_path / "ws"
ws_skills = workspace / "skills"
ws_skills.mkdir(parents=True)
_write_skill(ws_skills, "alpha", body="# Alpha")
_write_skill(ws_skills, "beta", body="# Beta")
builtin = tmp_path / "builtin"
builtin.mkdir()
loader = SkillsLoader(workspace, builtin_skills_dir=builtin, disabled_skills={"alpha"})
summary = loader.build_skills_summary()
assert "alpha" not in summary
assert "beta" in summary
def test_disabled_skills_excluded_from_get_always_skills(tmp_path: Path) -> None:
workspace = tmp_path / "ws"
ws_skills = workspace / "skills"
ws_skills.mkdir(parents=True)
_write_skill(ws_skills, "alpha", metadata_json={"always": True}, body="# Alpha")
_write_skill(ws_skills, "beta", metadata_json={"always": True}, body="# Beta")
builtin = tmp_path / "builtin"
builtin.mkdir()
loader = SkillsLoader(workspace, builtin_skills_dir=builtin, disabled_skills={"alpha"})
always = loader.get_always_skills()
assert "alpha" not in always
assert "beta" in always
# -- multiline description tests (YAML folded > and literal |) -----------------
def test_build_skills_summary_folded_description(tmp_path: Path) -> None:
"""description: > (YAML folded scalar) should be parsed correctly."""
workspace = tmp_path / "ws"
ws_skills = workspace / "skills"
ws_skills.mkdir(parents=True)
skill_dir = ws_skills / "pdf"
skill_dir.mkdir(parents=True)
skill_path = skill_dir / "SKILL.md"
skill_path.write_text(
"---\n"
"name: pdf\n"
"description: >\n"
" Use this skill when visual quality and design identity matter for a PDF.\n"
" CREATE (generate from scratch): \"make a PDF\".\n"
"---\n\n# PDF Skill\n",
encoding="utf-8",
)
builtin = tmp_path / "builtin"
builtin.mkdir()
loader = SkillsLoader(workspace, builtin_skills_dir=builtin)
summary = loader.build_skills_summary()
assert "pdf" in summary
assert "visual quality" in summary
def test_build_skills_summary_literal_description(tmp_path: Path) -> None:
"""description: | (YAML literal scalar) should be parsed correctly."""
workspace = tmp_path / "ws"
ws_skills = workspace / "skills"
ws_skills.mkdir(parents=True)
skill_dir = ws_skills / "multi"
skill_dir.mkdir(parents=True)
skill_path = skill_dir / "SKILL.md"
skill_path.write_text(
"---\n"
"name: multi\n"
"description: |\n"
" Line one of description.\n"
" Line two of description.\n"
"---\n\n# Multi\n",
encoding="utf-8",
)
builtin = tmp_path / "builtin"
builtin.mkdir()
loader = SkillsLoader(workspace, builtin_skills_dir=builtin)
meta = loader.get_skill_metadata("multi")
assert meta is not None
desc = meta.get("description")
assert isinstance(desc, str)
assert "Line one" in desc
assert "Line two" in desc
def test_get_skill_metadata_handles_yaml_types(tmp_path: Path) -> None:
"""yaml.safe_load returns native types; always should be True, not 'true'."""
workspace = tmp_path / "ws"
ws_skills = workspace / "skills"
ws_skills.mkdir(parents=True)
skill_dir = ws_skills / "typed"
skill_dir.mkdir(parents=True)
payload = json.dumps({"nanobot": {"requires": {"bins": ["gh"]}, "always": True}}, separators=(",", ":"))
skill_path = skill_dir / "SKILL.md"
skill_path.write_text(
"---\n"
"name: typed\n"
f"metadata: {payload}\n"
"always: true\n"
"---\n\n# Typed\n",
encoding="utf-8",
)
builtin = tmp_path / "builtin"
builtin.mkdir()
loader = SkillsLoader(workspace, builtin_skills_dir=builtin)
meta = loader.get_skill_metadata("typed")
assert meta is not None
# YAML parsed 'true' to Python True
assert meta.get("always") is True
# metadata is a parsed dict, not a JSON string
assert isinstance(meta.get("metadata"), dict)
+15 -5
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
import asyncio
import time
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock, patch
@@ -269,7 +270,9 @@ class TestSubagentCancellation:
monkeypatch.setattr("nanobot.agent.tools.filesystem.ListDirTool.execute", fake_execute)
await mgr._run_subagent("sub-1", "do task", "label", {"channel": "test", "chat_id": "c1"})
from nanobot.agent.subagent import SubagentStatus
status = SubagentStatus(task_id="sub-1", label="label", task_description="do task", started_at=time.monotonic())
await mgr._run_subagent("sub-1", "do task", "label", {"channel": "test", "chat_id": "c1"}, status)
assistant_messages = [
msg for msg in captured_second_call
@@ -308,7 +311,9 @@ class TestSubagentCancellation:
mgr.runner.run = AsyncMock(side_effect=fake_run)
await mgr._run_subagent("sub-1", "do task", "label", {"channel": "test", "chat_id": "c1"})
from nanobot.agent.subagent import SubagentStatus
status = SubagentStatus(task_id="sub-1", label="label", task_description="do task", started_at=time.monotonic())
await mgr._run_subagent("sub-1", "do task", "label", {"channel": "test", "chat_id": "c1"}, status)
mgr.runner.run.assert_awaited_once()
mgr._announce_result.assert_awaited_once()
@@ -344,7 +349,9 @@ class TestSubagentCancellation:
monkeypatch.setattr("nanobot.agent.tools.filesystem.ListDirTool.execute", fake_execute)
await mgr._run_subagent("sub-1", "do task", "label", {"channel": "test", "chat_id": "c1"})
from nanobot.agent.subagent import SubagentStatus
status = SubagentStatus(task_id="sub-1", label="label", task_description="do task", started_at=time.monotonic())
await mgr._run_subagent("sub-1", "do task", "label", {"channel": "test", "chat_id": "c1"}, status)
mgr._announce_result.assert_awaited_once()
args = mgr._announce_result.await_args.args
@@ -356,7 +363,7 @@ class TestSubagentCancellation:
@pytest.mark.asyncio
async def test_cancel_by_session_cancels_running_subagent_tool(self, monkeypatch, tmp_path):
from nanobot.agent.subagent import SubagentManager
from nanobot.agent.subagent import SubagentManager, SubagentStatus
from nanobot.bus.queue import MessageBus
from nanobot.providers.base import LLMResponse, ToolCallRequest
@@ -389,7 +396,10 @@ class TestSubagentCancellation:
monkeypatch.setattr("nanobot.agent.tools.filesystem.ListDirTool.execute", fake_execute)
task = asyncio.create_task(
mgr._run_subagent("sub-1", "do task", "label", {"channel": "test", "chat_id": "c1"})
mgr._run_subagent(
"sub-1", "do task", "label", {"channel": "test", "chat_id": "c1"},
SubagentStatus(task_id="sub-1", label="label", task_description="do task", started_at=time.monotonic()),
)
)
mgr._running_tasks["sub-1"] = task
mgr._session_tasks["test:c1"] = {"sub-1"}
+16
View File
@@ -72,6 +72,22 @@ class TestToolHintKnownTools:
result = _hint([_tc("exec", {"command": cmd})])
assert "\u2026/" in result
def test_exec_abbreviates_quoted_linux_paths_with_spaces(self):
"""Quoted Unix paths with spaces should still be folded."""
cmd = 'cd "/home/user/My Documents/project" && pytest tests/'
result = _hint([_tc("exec", {"command": cmd})])
assert "\u2026/" in result
assert '"/home/user/My Documents/project"' not in result
assert '"' in result
def test_exec_abbreviates_quoted_windows_paths_with_spaces(self):
"""Quoted Windows paths with spaces should still be folded."""
cmd = 'cd "C:/Program Files/Git/project" && git status'
result = _hint([_tc("exec", {"command": cmd})])
assert "\u2026/" in result
assert '"C:/Program Files/Git/project"' not in result
assert '"' in result
def test_exec_short_command_unchanged(self):
result = _hint([_tc("exec", {"command": "npm install typescript"})])
assert result == "$ npm install typescript"
+502
View File
@@ -0,0 +1,502 @@
"""Tests for unified_session feature.
Covers:
- AgentLoop._dispatch() rewrites session_key to "unified:default" when enabled
- Existing session_key_override is respected (not overwritten)
- Feature is off by default (no behavior change for existing users)
- Config schema serialises unified_session as camelCase "unifiedSession"
- onboard-generated config.json contains "unifiedSession" key
- /new command correctly clears the shared session in unified mode
- /new is NOT a priority command (goes through _dispatch, key rewrite applies)
- Context window consolidation is unaffected by unified_session
"""
import asyncio
import json
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from nanobot.agent.loop import AgentLoop
from nanobot.bus.events import InboundMessage
from nanobot.bus.queue import MessageBus
from nanobot.command.builtin import cmd_new, register_builtin_commands
from nanobot.command.router import CommandContext, CommandRouter
from nanobot.config.schema import AgentDefaults, Config
from nanobot.session.manager import Session, SessionManager
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_loop(tmp_path: Path, unified_session: bool = False) -> AgentLoop:
"""Create a minimal AgentLoop for dispatch-level tests."""
bus = MessageBus()
provider = MagicMock()
provider.get_default_model.return_value = "test-model"
with patch("nanobot.agent.loop.SessionManager"), \
patch("nanobot.agent.loop.SubagentManager") as MockSubMgr, \
patch("nanobot.agent.loop.Dream"):
MockSubMgr.return_value.cancel_by_session = AsyncMock(return_value=0)
loop = AgentLoop(
bus=bus,
provider=provider,
workspace=tmp_path,
unified_session=unified_session,
)
return loop
def _make_msg(channel: str = "telegram", chat_id: str = "111",
session_key_override: str | None = None) -> InboundMessage:
return InboundMessage(
channel=channel,
chat_id=chat_id,
sender_id="user1",
content="hello",
session_key_override=session_key_override,
)
# ---------------------------------------------------------------------------
# TestUnifiedSessionDispatch — core behaviour
# ---------------------------------------------------------------------------
class TestUnifiedSessionDispatch:
"""AgentLoop._dispatch() session key rewriting logic."""
@pytest.mark.asyncio
async def test_unified_session_rewrites_key_to_unified_default(self, tmp_path: Path):
"""When unified_session=True, all messages use 'unified:default' as session key."""
loop = _make_loop(tmp_path, unified_session=True)
captured: list[str] = []
async def fake_process(msg, **kwargs):
captured.append(msg.session_key)
return None
loop._process_message = fake_process # type: ignore[method-assign]
msg = _make_msg(channel="telegram", chat_id="111")
await loop._dispatch(msg)
assert captured == ["unified:default"]
@pytest.mark.asyncio
async def test_unified_session_different_channels_share_same_key(self, tmp_path: Path):
"""Messages from different channels all resolve to the same session key."""
loop = _make_loop(tmp_path, unified_session=True)
captured: list[str] = []
async def fake_process(msg, **kwargs):
captured.append(msg.session_key)
return None
loop._process_message = fake_process # type: ignore[method-assign]
await loop._dispatch(_make_msg(channel="telegram", chat_id="111"))
await loop._dispatch(_make_msg(channel="discord", chat_id="222"))
await loop._dispatch(_make_msg(channel="cli", chat_id="direct"))
assert captured == ["unified:default", "unified:default", "unified:default"]
@pytest.mark.asyncio
async def test_unified_session_disabled_preserves_original_key(self, tmp_path: Path):
"""When unified_session=False (default), session key is channel:chat_id as usual."""
loop = _make_loop(tmp_path, unified_session=False)
captured: list[str] = []
async def fake_process(msg, **kwargs):
captured.append(msg.session_key)
return None
loop._process_message = fake_process # type: ignore[method-assign]
msg = _make_msg(channel="telegram", chat_id="999")
await loop._dispatch(msg)
assert captured == ["telegram:999"]
@pytest.mark.asyncio
async def test_unified_session_respects_existing_override(self, tmp_path: Path):
"""If session_key_override is already set (e.g. Telegram thread), it is NOT overwritten."""
loop = _make_loop(tmp_path, unified_session=True)
captured: list[str] = []
async def fake_process(msg, **kwargs):
captured.append(msg.session_key)
return None
loop._process_message = fake_process # type: ignore[method-assign]
msg = _make_msg(channel="telegram", chat_id="111", session_key_override="telegram:thread:42")
await loop._dispatch(msg)
assert captured == ["telegram:thread:42"]
def test_unified_session_default_is_false(self, tmp_path: Path):
"""unified_session defaults to False — no behavior change for existing users."""
loop = _make_loop(tmp_path)
assert loop._unified_session is False
# ---------------------------------------------------------------------------
# TestUnifiedSessionConfig — schema & serialisation
# ---------------------------------------------------------------------------
class TestUnifiedSessionConfig:
"""Config schema and onboard serialisation for unified_session."""
def test_agent_defaults_unified_session_default_is_false(self):
"""AgentDefaults.unified_session defaults to False."""
defaults = AgentDefaults()
assert defaults.unified_session is False
def test_agent_defaults_unified_session_can_be_enabled(self):
"""AgentDefaults.unified_session can be set to True."""
defaults = AgentDefaults(unified_session=True)
assert defaults.unified_session is True
def test_config_serialises_unified_session_as_camel_case(self):
"""model_dump(by_alias=True) outputs 'unifiedSession' (camelCase) for JSON."""
config = Config()
data = config.model_dump(mode="json", by_alias=True)
agents_defaults = data["agents"]["defaults"]
assert "unifiedSession" in agents_defaults
assert agents_defaults["unifiedSession"] is False
def test_config_parses_unified_session_from_camel_case(self):
"""Config can be loaded from JSON with camelCase 'unifiedSession'."""
raw = {"agents": {"defaults": {"unifiedSession": True}}}
config = Config.model_validate(raw)
assert config.agents.defaults.unified_session is True
def test_config_parses_unified_session_from_snake_case(self):
"""Config also accepts snake_case 'unified_session' (populate_by_name=True)."""
raw = {"agents": {"defaults": {"unified_session": True}}}
config = Config.model_validate(raw)
assert config.agents.defaults.unified_session is True
def test_onboard_generated_config_contains_unified_session(self, tmp_path: Path):
"""save_config() writes 'unifiedSession' into config.json (simulates nanobot onboard)."""
from nanobot.config.loader import save_config
config = Config()
config_path = tmp_path / "config.json"
save_config(config, config_path)
with open(config_path, encoding="utf-8") as f:
data = json.load(f)
agents_defaults = data["agents"]["defaults"]
assert "unifiedSession" in agents_defaults, (
"onboard-generated config.json must contain 'unifiedSession' key"
)
assert agents_defaults["unifiedSession"] is False
# ---------------------------------------------------------------------------
# TestCmdNewUnifiedSession — /new command behaviour in unified mode
# ---------------------------------------------------------------------------
class TestCmdNewUnifiedSession:
"""/new command routing and session-clear behaviour in unified mode."""
def test_new_is_not_a_priority_command(self):
"""/new must NOT be in the priority table — it must go through _dispatch()
so the unified session key rewrite applies before cmd_new runs."""
router = CommandRouter()
register_builtin_commands(router)
assert router.is_priority("/new") is False
def test_new_is_an_exact_command(self):
"""/new must be registered as an exact command."""
router = CommandRouter()
register_builtin_commands(router)
assert "/new" in router._exact
@pytest.mark.asyncio
async def test_cmd_new_clears_unified_session(self, tmp_path: Path):
"""cmd_new called with key='unified:default' clears the shared session."""
sessions = SessionManager(tmp_path)
# Pre-populate the shared session with some messages
shared = sessions.get_or_create("unified:default")
shared.add_message("user", "hello from telegram")
shared.add_message("assistant", "hi there")
sessions.save(shared)
assert len(sessions.get_or_create("unified:default").messages) == 2
# _schedule_background is a *sync* method that schedules a coroutine via
# asyncio.create_task(). Mirror that exactly so the coroutine is consumed
# and no RuntimeWarning is emitted.
loop = SimpleNamespace(
sessions=sessions,
consolidator=SimpleNamespace(archive=AsyncMock(return_value=True)),
)
loop._schedule_background = lambda coro: asyncio.ensure_future(coro)
msg = InboundMessage(
channel="telegram", sender_id="user1", chat_id="111", content="/new",
session_key_override="unified:default", # as _dispatch() would set it
)
ctx = CommandContext(msg=msg, session=None, key="unified:default", raw="/new", loop=loop)
result = await cmd_new(ctx)
assert "New session started" in result.content
# Invalidate cache and reload from disk to confirm persistence
sessions.invalidate("unified:default")
reloaded = sessions.get_or_create("unified:default")
assert reloaded.messages == []
@pytest.mark.asyncio
async def test_cmd_new_in_unified_mode_does_not_affect_other_sessions(self, tmp_path: Path):
"""Clearing unified:default must not touch other sessions on disk."""
sessions = SessionManager(tmp_path)
other = sessions.get_or_create("discord:999")
other.add_message("user", "discord message")
sessions.save(other)
shared = sessions.get_or_create("unified:default")
shared.add_message("user", "shared message")
sessions.save(shared)
loop = SimpleNamespace(
sessions=sessions,
consolidator=SimpleNamespace(archive=AsyncMock(return_value=True)),
)
loop._schedule_background = lambda coro: asyncio.ensure_future(coro)
msg = InboundMessage(
channel="telegram", sender_id="user1", chat_id="111", content="/new",
session_key_override="unified:default",
)
ctx = CommandContext(msg=msg, session=None, key="unified:default", raw="/new", loop=loop)
await cmd_new(ctx)
sessions.invalidate("unified:default")
sessions.invalidate("discord:999")
assert sessions.get_or_create("unified:default").messages == []
assert len(sessions.get_or_create("discord:999").messages) == 1
# ---------------------------------------------------------------------------
# TestConsolidationUnaffectedByUnifiedSession — consolidation is key-agnostic
# ---------------------------------------------------------------------------
class TestConsolidationUnaffectedByUnifiedSession:
"""maybe_consolidate_by_tokens() behaviour is identical regardless of session key."""
@pytest.mark.asyncio
async def test_consolidation_skips_empty_session_for_unified_key(self):
"""Empty unified:default session → consolidation exits immediately, archive not called."""
from nanobot.agent.memory import Consolidator, MemoryStore
store = MagicMock(spec=MemoryStore)
mock_provider = MagicMock()
mock_provider.chat_with_retry = AsyncMock(return_value=MagicMock(content="summary"))
# Use spec= so MagicMock doesn't auto-generate AsyncMock for non-async methods,
# which would leave unawaited coroutines and trigger RuntimeWarning.
sessions = MagicMock(spec=SessionManager)
consolidator = Consolidator(
store=store,
provider=mock_provider,
model="test-model",
sessions=sessions,
context_window_tokens=1000,
build_messages=MagicMock(return_value=[]),
get_tool_definitions=MagicMock(return_value=[]),
max_completion_tokens=100,
)
consolidator.archive = AsyncMock()
session = Session(key="unified:default")
session.messages = []
await consolidator.maybe_consolidate_by_tokens(session)
consolidator.archive.assert_not_called()
@pytest.mark.asyncio
async def test_consolidation_behaviour_identical_for_any_key(self):
"""archive call count is the same for 'telegram:123' and 'unified:default'
under identical token conditions."""
from nanobot.agent.memory import Consolidator, MemoryStore
archive_calls: dict[str, int] = {}
for key in ("telegram:123", "unified:default"):
store = MagicMock(spec=MemoryStore)
mock_provider = MagicMock()
mock_provider.chat_with_retry = AsyncMock(return_value=MagicMock(content="summary"))
sessions = MagicMock(spec=SessionManager)
consolidator = Consolidator(
store=store,
provider=mock_provider,
model="test-model",
sessions=sessions,
context_window_tokens=1000,
build_messages=MagicMock(return_value=[]),
get_tool_definitions=MagicMock(return_value=[]),
max_completion_tokens=100,
)
session = Session(key=key)
session.messages = [] # empty → exits immediately for both keys
consolidator.archive = AsyncMock()
await consolidator.maybe_consolidate_by_tokens(session)
archive_calls[key] = consolidator.archive.call_count
assert archive_calls["telegram:123"] == archive_calls["unified:default"] == 0
@pytest.mark.asyncio
async def test_consolidation_triggers_when_over_budget_unified_key(self):
"""When tokens exceed budget, consolidation attempts to find a boundary —
behaviour is identical to any other session key."""
from nanobot.agent.memory import Consolidator, MemoryStore
store = MagicMock(spec=MemoryStore)
mock_provider = MagicMock()
sessions = MagicMock(spec=SessionManager)
consolidator = Consolidator(
store=store,
provider=mock_provider,
model="test-model",
sessions=sessions,
context_window_tokens=1000,
build_messages=MagicMock(return_value=[]),
get_tool_definitions=MagicMock(return_value=[]),
max_completion_tokens=100,
)
session = Session(key="unified:default")
session.messages = [{"role": "user", "content": "msg"}]
# Simulate over-budget: estimated > budget
consolidator.estimate_session_prompt_tokens = MagicMock(return_value=(950, "tiktoken"))
# No valid boundary found → returns gracefully without archiving
consolidator.pick_consolidation_boundary = MagicMock(return_value=None)
consolidator.archive = AsyncMock()
await consolidator.maybe_consolidate_by_tokens(session)
# estimate was called (consolidation was attempted)
consolidator.estimate_session_prompt_tokens.assert_called_once_with(session)
# but archive was not called (no valid boundary)
consolidator.archive.assert_not_called()
# ---------------------------------------------------------------------------
# TestStopCommandWithUnifiedSession — /stop command integration
# ---------------------------------------------------------------------------
class TestStopCommandWithUnifiedSession:
"""Verify /stop command works correctly with unified session enabled."""
@pytest.mark.asyncio
async def test_active_tasks_use_effective_key_in_unified_mode(self, tmp_path: Path):
"""When unified_session=True, tasks are stored under UNIFIED_SESSION_KEY."""
from nanobot.agent.loop import UNIFIED_SESSION_KEY
loop = _make_loop(tmp_path, unified_session=True)
# Create a message from telegram channel
msg = _make_msg(channel="telegram", chat_id="123456")
# Mock _dispatch to complete immediately
async def fake_dispatch(m):
pass
loop._dispatch = fake_dispatch # type: ignore[method-assign]
# Simulate the task creation flow (from _run loop)
effective_key = UNIFIED_SESSION_KEY if loop._unified_session and not msg.session_key_override else msg.session_key
task = asyncio.create_task(loop._dispatch(msg))
loop._active_tasks.setdefault(effective_key, []).append(task)
# Wait for task to complete
await task
# Verify the task is stored under UNIFIED_SESSION_KEY, not the original channel:chat_id
assert UNIFIED_SESSION_KEY in loop._active_tasks
assert "telegram:123456" not in loop._active_tasks
@pytest.mark.asyncio
async def test_stop_command_finds_task_in_unified_mode(self, tmp_path: Path):
"""cmd_stop can cancel tasks when unified_session=True."""
from nanobot.agent.loop import UNIFIED_SESSION_KEY
from nanobot.command.builtin import cmd_stop
loop = _make_loop(tmp_path, unified_session=True)
# Create a long-running task stored under UNIFIED_SESSION_KEY
async def long_running():
await asyncio.sleep(10) # Will be cancelled
task = asyncio.create_task(long_running())
loop._active_tasks[UNIFIED_SESSION_KEY] = [task]
# Create a message that would have session_key=UNIFIED_SESSION_KEY after dispatch
msg = InboundMessage(
channel="telegram",
chat_id="123456",
sender_id="user1",
content="/stop",
session_key_override=UNIFIED_SESSION_KEY, # Simulate post-dispatch state
)
ctx = CommandContext(msg=msg, session=None, key=UNIFIED_SESSION_KEY, raw="/stop", loop=loop)
# Execute /stop
result = await cmd_stop(ctx)
# Verify task was cancelled
assert task.cancelled() or task.done()
assert "Stopped 1 task" in result.content
@pytest.mark.asyncio
async def test_stop_command_cross_channel_in_unified_mode(self, tmp_path: Path):
"""In unified mode, /stop from one channel cancels tasks from another channel."""
from nanobot.agent.loop import UNIFIED_SESSION_KEY
from nanobot.command.builtin import cmd_stop
loop = _make_loop(tmp_path, unified_session=True)
# Create tasks from different channels, all stored under UNIFIED_SESSION_KEY
async def long_running():
await asyncio.sleep(10)
task1 = asyncio.create_task(long_running())
task2 = asyncio.create_task(long_running())
loop._active_tasks[UNIFIED_SESSION_KEY] = [task1, task2]
# /stop from discord should cancel tasks started from telegram
msg = InboundMessage(
channel="discord",
chat_id="789012",
sender_id="user2",
content="/stop",
session_key_override=UNIFIED_SESSION_KEY,
)
ctx = CommandContext(msg=msg, session=None, key=UNIFIED_SESSION_KEY, raw="/stop", loop=loop)
result = await cmd_stop(ctx)
# Both tasks should be cancelled
assert "Stopped 2 task" in result.content
View File
File diff suppressed because it is too large Load Diff
+12
View File
@@ -23,3 +23,15 @@ def test_is_allowed_requires_exact_match() -> None:
assert channel.is_allowed("allow@email.com") is True
assert channel.is_allowed("attacker|allow@email.com") is False
def test_is_allowed_supports_dict_allow_from_alias() -> None:
channel = _DummyChannel({"allowFrom": ["alice"]}, MessageBus())
assert channel.is_allowed("alice") is True
def test_is_allowed_denies_empty_dict_allow_from() -> None:
channel = _DummyChannel({"allow_from": []}, MessageBus())
assert channel.is_allowed("alice") is False
+131 -2
View File
@@ -175,7 +175,7 @@ async def test_manager_loads_plugin_from_dict_config():
channels=ChannelsConfig.model_validate({
"fakeplugin": {"enabled": True, "allowFrom": ["*"]},
}),
providers=SimpleNamespace(groq=SimpleNamespace(api_key="")),
providers=SimpleNamespace(groq=SimpleNamespace(api_key="", api_base="")),
)
with patch(
@@ -193,6 +193,113 @@ async def test_manager_loads_plugin_from_dict_config():
assert isinstance(mgr.channels["fakeplugin"], _FakePlugin)
@pytest.mark.asyncio
async def test_manager_propagates_groq_transcription_api_base_to_channels():
from nanobot.channels.manager import ChannelManager
fake_config = SimpleNamespace(
channels=ChannelsConfig.model_validate({
"fakeplugin": {"enabled": True, "allowFrom": ["*"]},
}),
transcription_provider="groq",
providers=SimpleNamespace(
groq=SimpleNamespace(api_key="groq-key", api_base="http://proxy.local/v1/audio/transcriptions"),
openai=SimpleNamespace(api_key="openai-key", api_base="https://api.openai.com/v1/audio/transcriptions"),
),
)
with patch(
"nanobot.channels.registry.discover_all",
return_value={"fakeplugin": _FakePlugin},
):
mgr = ChannelManager.__new__(ChannelManager)
mgr.config = fake_config
mgr.bus = MessageBus()
mgr.channels = {}
mgr._dispatch_task = None
mgr._init_channels()
channel = mgr.channels["fakeplugin"]
assert channel.transcription_provider == "groq"
assert channel.transcription_api_key == "groq-key"
assert channel.transcription_api_base == "http://proxy.local/v1/audio/transcriptions"
@pytest.mark.asyncio
async def test_manager_propagates_openai_transcription_api_base_to_channels():
from nanobot.channels.manager import ChannelManager
fake_config = SimpleNamespace(
channels=ChannelsConfig.model_validate({
"fakeplugin": {"enabled": True, "allowFrom": ["*"]},
"transcriptionProvider": "openai",
}),
providers=SimpleNamespace(
openai=SimpleNamespace(
api_key="openai-key",
api_base="http://proxy.local/v1/audio/transcriptions",
),
groq=SimpleNamespace(api_key="groq-key", api_base=""),
),
)
with patch(
"nanobot.channels.registry.discover_all",
return_value={"fakeplugin": _FakePlugin},
):
mgr = ChannelManager.__new__(ChannelManager)
mgr.config = fake_config
mgr.bus = MessageBus()
mgr.channels = {}
mgr._dispatch_task = None
mgr._init_channels()
channel = mgr.channels["fakeplugin"]
assert channel.transcription_provider == "openai"
assert channel.transcription_api_key == "openai-key"
assert channel.transcription_api_base == "http://proxy.local/v1/audio/transcriptions"
@pytest.mark.asyncio
async def test_base_channel_passes_api_base_to_openai_transcription_provider():
"""BaseChannel.transcribe_audio must forward transcription_api_base to OpenAI."""
from nanobot.providers import transcription as transcription_mod
channel = _FakePlugin({"enabled": True, "allowFrom": ["*"]}, MessageBus())
channel.transcription_provider = "openai"
channel.transcription_api_key = "k"
channel.transcription_api_base = "http://override/v1/audio/transcriptions"
captured: dict[str, object] = {}
class _StubOpenAI:
def __init__(self, api_key=None, api_base=None):
captured["api_key"] = api_key
captured["api_base"] = api_base
async def transcribe(self, file_path):
return "ok"
with patch.object(transcription_mod, "OpenAITranscriptionProvider", _StubOpenAI):
result = await channel.transcribe_audio("/tmp/does-not-matter.wav")
assert result == "ok"
assert captured["api_key"] == "k"
assert captured["api_base"] == "http://override/v1/audio/transcriptions"
def test_openai_transcription_provider_honors_api_base_argument():
from nanobot.providers.transcription import OpenAITranscriptionProvider
default = OpenAITranscriptionProvider(api_key="k")
assert default.api_url == "https://api.openai.com/v1/audio/transcriptions"
custom = OpenAITranscriptionProvider(
api_key="k", api_base="http://override/v1/audio/transcriptions"
)
assert custom.api_url == "http://override/v1/audio/transcriptions"
def test_channels_login_uses_discovered_plugin_class(monkeypatch):
from nanobot.cli.commands import app
from nanobot.config.schema import Config
@@ -646,7 +753,10 @@ class _ChannelWithAllowFrom(BaseChannel):
def __init__(self, config, bus, allow_from):
super().__init__(config, bus)
self.config.allow_from = allow_from
if isinstance(self.config, dict):
self.config["allow_from"] = allow_from
else:
self.config.allow_from = allow_from
async def start(self) -> None:
pass
@@ -714,6 +824,25 @@ async def test_validate_allow_from_passes_with_asterisk():
mgr._validate_allow_from()
@pytest.mark.asyncio
async def test_validate_allow_from_raises_on_empty_dict_allow_from():
"""_validate_allow_from should reject empty dict-backed allow_from lists."""
fake_config = SimpleNamespace(
channels=ChannelsConfig(),
providers=SimpleNamespace(groq=SimpleNamespace(api_key="")),
)
mgr = ChannelManager.__new__(ChannelManager)
mgr.config = fake_config
mgr.channels = {"test": _ChannelWithAllowFrom({"enabled": True}, None, [])}
mgr._dispatch_task = None
with pytest.raises(SystemExit) as exc_info:
mgr._validate_allow_from()
assert "empty allowFrom" in str(exc_info.value)
@pytest.mark.asyncio
async def test_get_channel_returns_channel_if_exists():
"""get_channel should return the channel if it exists."""
+232
View File
@@ -1,6 +1,10 @@
import asyncio
import zipfile
from io import BytesIO
from types import SimpleNamespace
from unittest.mock import AsyncMock
import httpx
import pytest
# Check optional dingtalk dependencies before running tests
@@ -50,6 +54,21 @@ class _FakeHttp:
return self._next_response()
class _NetworkErrorHttp:
"""HTTP client stub that raises httpx.TransportError on every request."""
def __init__(self) -> None:
self.calls: list[dict] = []
async def post(self, url: str, json=None, headers=None, **kwargs):
self.calls.append({"method": "POST", "url": url, "json": json, "headers": headers})
raise httpx.ConnectError("Connection refused")
async def get(self, url: str, **kwargs):
self.calls.append({"method": "GET", "url": url})
raise httpx.ConnectError("Connection refused")
@pytest.mark.asyncio
async def test_group_message_keeps_sender_id_and_routes_chat_id() -> None:
config = DingTalkConfig(client_id="app", client_secret="secret", allow_from=["user1"])
@@ -221,3 +240,216 @@ async def test_download_dingtalk_file(tmp_path, monkeypatch) -> None:
assert "messageFiles/download" in channel._http.calls[0]["url"]
assert channel._http.calls[0]["json"]["downloadCode"] == "code123"
assert channel._http.calls[1]["method"] == "GET"
def test_normalize_upload_payload_zips_html_attachment() -> None:
channel = DingTalkChannel(
DingTalkConfig(client_id="app", client_secret="secret", allow_from=["*"]),
MessageBus(),
)
data, filename, content_type = channel._normalize_upload_payload(
"report.html",
b"<html><body>Hello</body></html>",
"text/html",
)
assert filename == "report.zip"
assert content_type == "application/zip"
archive = zipfile.ZipFile(BytesIO(data))
assert archive.namelist() == ["report.html"]
assert archive.read("report.html") == b"<html><body>Hello</body></html>"
@pytest.mark.asyncio
async def test_send_media_ref_zips_html_before_upload(tmp_path, monkeypatch) -> None:
channel = DingTalkChannel(
DingTalkConfig(client_id="app", client_secret="secret", allow_from=["*"]),
MessageBus(),
)
html_path = tmp_path / "report.html"
html_path.write_text("<html><body>Hello</body></html>", encoding="utf-8")
captured: dict[str, object] = {}
async def fake_upload_media(*, token, data, media_type, filename, content_type):
captured.update(
{
"token": token,
"data": data,
"media_type": media_type,
"filename": filename,
"content_type": content_type,
}
)
return "media-123"
async def fake_send_batch_message(token, chat_id, msg_key, msg_param):
captured.update(
{
"sent_token": token,
"chat_id": chat_id,
"msg_key": msg_key,
"msg_param": msg_param,
}
)
return True
monkeypatch.setattr(channel, "_upload_media", fake_upload_media)
monkeypatch.setattr(channel, "_send_batch_message", fake_send_batch_message)
ok = await channel._send_media_ref("token-123", "user-1", str(html_path))
assert ok is True
assert captured["media_type"] == "file"
assert captured["filename"] == "report.zip"
assert captured["content_type"] == "application/zip"
assert captured["msg_key"] == "sampleFile"
assert captured["msg_param"] == {
"mediaId": "media-123",
"fileName": "report.zip",
"fileType": "zip",
}
archive = zipfile.ZipFile(BytesIO(captured["data"]))
assert archive.namelist() == ["report.html"]
# ── Exception handling tests ──────────────────────────────────────────
@pytest.mark.asyncio
async def test_send_batch_message_propagates_transport_error() -> None:
"""Network/transport errors must re-raise so callers can retry."""
config = DingTalkConfig(client_id="app", client_secret="secret", allow_from=["*"])
channel = DingTalkChannel(config, MessageBus())
channel._http = _NetworkErrorHttp()
with pytest.raises(httpx.ConnectError, match="Connection refused"):
await channel._send_batch_message(
"token",
"user123",
"sampleMarkdown",
{"text": "hello", "title": "Nanobot Reply"},
)
# The POST was attempted exactly once
assert len(channel._http.calls) == 1
assert channel._http.calls[0]["method"] == "POST"
@pytest.mark.asyncio
async def test_send_batch_message_returns_false_on_api_error() -> None:
"""DingTalk API-level errors (non-200 status, errcode != 0) should return False."""
config = DingTalkConfig(client_id="app", client_secret="secret", allow_from=["*"])
channel = DingTalkChannel(config, MessageBus())
# Non-200 status code → API error → return False
channel._http = _FakeHttp(responses=[_FakeResponse(400, {"errcode": 400})])
result = await channel._send_batch_message(
"token", "user123", "sampleMarkdown", {"text": "hello"}
)
assert result is False
# 200 with non-zero errcode → API error → return False
channel._http = _FakeHttp(responses=[_FakeResponse(200, {"errcode": 100})])
result = await channel._send_batch_message(
"token", "user123", "sampleMarkdown", {"text": "hello"}
)
assert result is False
# 200 with errcode=0 → success → return True
channel._http = _FakeHttp(responses=[_FakeResponse(200, {"errcode": 0})])
result = await channel._send_batch_message(
"token", "user123", "sampleMarkdown", {"text": "hello"}
)
assert result is True
@pytest.mark.asyncio
async def test_send_media_ref_short_circuits_on_transport_error() -> None:
"""When the first send fails with a transport error, _send_media_ref must
re-raise immediately instead of trying download+upload+fallback."""
config = DingTalkConfig(client_id="app", client_secret="secret", allow_from=["*"])
channel = DingTalkChannel(config, MessageBus())
channel._http = _NetworkErrorHttp()
# An image URL triggers the sampleImageMsg path first
with pytest.raises(httpx.ConnectError, match="Connection refused"):
await channel._send_media_ref("token", "user123", "https://example.com/photo.jpg")
# Only one POST should have been attempted — no download/upload/fallback
assert len(channel._http.calls) == 1
assert channel._http.calls[0]["method"] == "POST"
@pytest.mark.asyncio
async def test_send_media_ref_short_circuits_on_download_transport_error() -> None:
"""When the image URL send returns an API error (False) but the download
for the fallback hits a transport error, it must re-raise rather than
silently returning False."""
config = DingTalkConfig(client_id="app", client_secret="secret", allow_from=["*"])
channel = DingTalkChannel(config, MessageBus())
# First POST (sampleImageMsg) returns API error → False, then GET (download) raises transport error
class _MixedHttp:
def __init__(self) -> None:
self.calls: list[dict] = []
async def post(self, url, json=None, headers=None, **kwargs):
self.calls.append({"method": "POST", "url": url})
# API-level failure: 200 with errcode != 0
return _FakeResponse(200, {"errcode": 100})
async def get(self, url, **kwargs):
self.calls.append({"method": "GET", "url": url})
raise httpx.ConnectError("Connection refused")
channel._http = _MixedHttp()
with pytest.raises(httpx.ConnectError, match="Connection refused"):
await channel._send_media_ref("token", "user123", "https://example.com/photo.jpg")
# Should have attempted POST (image URL) and GET (download), but NOT upload
assert len(channel._http.calls) == 2
assert channel._http.calls[0]["method"] == "POST"
assert channel._http.calls[1]["method"] == "GET"
@pytest.mark.asyncio
async def test_send_media_ref_short_circuits_on_upload_transport_error() -> None:
"""When download succeeds but upload hits a transport error, must re-raise."""
config = DingTalkConfig(client_id="app", client_secret="secret", allow_from=["*"])
channel = DingTalkChannel(config, MessageBus())
image_bytes = b"\xff\xd8\xff\xe0" + b"\x00" * 100 # minimal JPEG-ish data
class _UploadFailsHttp:
def __init__(self) -> None:
self.calls: list[dict] = []
async def post(self, url, json=None, headers=None, files=None, **kwargs):
self.calls.append({"method": "POST", "url": url})
# If it's the upload endpoint, raise transport error
if "media/upload" in url:
raise httpx.ConnectError("Connection refused")
# Otherwise (sampleImageMsg), return API error to trigger fallback
return _FakeResponse(200, {"errcode": 100})
async def get(self, url, **kwargs):
self.calls.append({"method": "GET", "url": url})
resp = _FakeResponse(200)
resp.content = image_bytes
resp.headers = {"content-type": "image/jpeg"}
return resp
channel._http = _UploadFailsHttp()
with pytest.raises(httpx.ConnectError, match="Connection refused"):
await channel._send_media_ref("token", "user123", "https://example.com/photo.jpg")
# POST (image URL), GET (download), POST (upload) attempted — no further sends
methods = [c["method"] for c in channel._http.calls]
assert methods == ["POST", "GET", "POST"]
+339 -10
View File
@@ -5,11 +5,17 @@ from pathlib import Path
from types import SimpleNamespace
import pytest
discord = pytest.importorskip("discord")
from nanobot.bus.events import OutboundMessage
from nanobot.bus.queue import MessageBus
from nanobot.channels.discord import DiscordBotClient, DiscordChannel, DiscordConfig
from nanobot.channels.discord import (
MAX_MESSAGE_LEN,
DiscordBotClient,
DiscordChannel,
DiscordConfig,
)
from nanobot.command.builtin import build_help_text
@@ -18,9 +24,11 @@ class _FakeDiscordClient:
instances: list["_FakeDiscordClient"] = []
start_error: Exception | None = None
def __init__(self, owner, *, intents) -> None:
def __init__(self, owner, *, intents, proxy=None, proxy_auth=None) -> None:
self.owner = owner
self.intents = intents
self.proxy = proxy
self.proxy_auth = proxy_auth
self.closed = False
self.ready = True
self.channels: dict[int, object] = {}
@@ -53,7 +61,9 @@ class _FakeDiscordClient:
class _FakeAttachment:
# Attachment double that can simulate successful or failing save() calls.
def __init__(self, attachment_id: int, filename: str, *, size: int = 1, fail: bool = False) -> None:
def __init__(
self, attachment_id: int, filename: str, *, size: int = 1, fail: bool = False
) -> None:
self.id = attachment_id
self.filename = filename
self.size = size
@@ -71,11 +81,25 @@ class _FakePartialMessage:
self.id = message_id
class _FakeSentMessage:
# Sent-message double supporting edit() for streaming tests.
def __init__(self, channel, content: str) -> None:
self.channel = channel
self.content = content
self.edits: list[dict] = []
async def edit(self, **kwargs) -> None:
self.edits.append(dict(kwargs))
if "content" in kwargs:
self.content = kwargs["content"]
class _FakeChannel:
# Channel double that records outbound payloads and typing activity.
def __init__(self, channel_id: int = 123) -> None:
self.id = channel_id
self.sent_payloads: list[dict] = []
self.sent_messages: list[_FakeSentMessage] = []
self.trigger_typing_calls = 0
self.typing_enter_hook = None
@@ -85,6 +109,9 @@ class _FakeChannel:
payload["file_name"] = payload["file"].filename
del payload["file"]
self.sent_payloads.append(payload)
message = _FakeSentMessage(self, payload.get("content", ""))
self.sent_messages.append(message)
return message
def get_partial_message(self, message_id: int) -> _FakePartialMessage:
return _FakePartialMessage(message_id)
@@ -194,7 +221,7 @@ async def test_start_handles_client_construction_failure(monkeypatch) -> None:
MessageBus(),
)
def _boom(owner, *, intents):
def _boom(owner, *, intents, proxy=None, proxy_auth=None):
raise RuntimeError("bad client")
monkeypatch.setattr("nanobot.channels.discord.DiscordBotClient", _boom)
@@ -286,6 +313,45 @@ async def test_on_message_accepts_allowlisted_dm() -> None:
assert handled[0]["metadata"] == {"message_id": "789", "guild_id": None, "reply_to": None}
@pytest.mark.asyncio
async def test_on_message_accepts_when_channel_in_allow_channels() -> None:
# When allow_channels is set, messages from listed channels should be forwarded.
channel = DiscordChannel(
DiscordConfig(enabled=True, allow_from=["*"], allow_channels=["456"]),
MessageBus(),
)
handled: list[dict] = []
async def capture_handle(**kwargs) -> None:
handled.append(kwargs)
channel._handle_message = capture_handle # type: ignore[method-assign]
await channel._on_message(_make_message(author_id=123, channel_id=456))
assert len(handled) == 1
assert handled[0]["chat_id"] == "456"
@pytest.mark.asyncio
async def test_on_message_drops_when_channel_not_in_allow_channels() -> None:
# When allow_channels is set and incoming channel is not listed, drop silently.
channel = DiscordChannel(
DiscordConfig(enabled=True, allow_from=["*"], allow_channels=["999"]),
MessageBus(),
)
handled: list[dict] = []
async def capture_handle(**kwargs) -> None:
handled.append(kwargs)
channel._handle_message = capture_handle # type: ignore[method-assign]
await channel._on_message(_make_message(author_id=123, channel_id=456))
assert handled == []
@pytest.mark.asyncio
async def test_on_message_ignores_unmentioned_guild_message() -> None:
# With mention-only group policy, guild messages without a bot mention are dropped.
@@ -427,6 +493,60 @@ async def test_send_fetches_channel_when_not_cached() -> None:
assert target.sent_payloads == [{"content": "hello"}]
def test_supports_streaming_enabled_by_default() -> None:
channel = DiscordChannel(DiscordConfig(enabled=True, allow_from=["*"]), MessageBus())
assert channel.supports_streaming is True
@pytest.mark.asyncio
async def test_send_delta_streams_by_editing_message(monkeypatch) -> None:
owner = DiscordChannel(DiscordConfig(enabled=True, allow_from=["*"]), MessageBus())
client = _FakeDiscordClient(owner, intents=None)
owner._client = client
owner._running = True
target = _FakeChannel(channel_id=123)
client.channels[123] = target
times = iter([1.0, 3.0, 5.0])
monkeypatch.setattr("nanobot.channels.discord.time.monotonic", lambda: next(times, 5.0))
await owner.send_delta("123", "hel", {"_stream_delta": True, "_stream_id": "s1"})
await owner.send_delta("123", "lo", {"_stream_delta": True, "_stream_id": "s1"})
await owner.send_delta("123", "", {"_stream_end": True, "_stream_id": "s1"})
assert target.sent_payloads[0] == {"content": "hel"}
assert target.sent_messages[0].edits == [{"content": "hello"}, {"content": "hello"}]
assert owner._stream_bufs == {}
@pytest.mark.asyncio
async def test_send_delta_stream_end_splits_oversized_reply(monkeypatch) -> None:
owner = DiscordChannel(DiscordConfig(enabled=True, allow_from=["*"]), MessageBus())
client = _FakeDiscordClient(owner, intents=None)
owner._client = client
owner._running = True
target = _FakeChannel(channel_id=123)
client.channels[123] = target
prefix = "a" * (MAX_MESSAGE_LEN - 100)
suffix = "b" * 150
full_text = prefix + suffix
chunks = DiscordBotClient._build_chunks(full_text, [], False)
assert len(chunks) == 2
times = iter([1.0, 3.0])
monkeypatch.setattr("nanobot.channels.discord.time.monotonic", lambda: next(times, 3.0))
await owner.send_delta("123", prefix, {"_stream_delta": True, "_stream_id": "s1"})
await owner.send_delta("123", suffix, {"_stream_delta": True, "_stream_id": "s1"})
await owner.send_delta("123", "", {"_stream_end": True, "_stream_id": "s1"})
assert target.sent_payloads == [{"content": prefix}, {"content": chunks[1]}]
assert target.sent_messages[0].edits == [{"content": chunks[0]}, {"content": chunks[0]}]
assert owner._stream_bufs == {}
@pytest.mark.asyncio
async def test_slash_new_forwards_when_user_is_allowlisted() -> None:
channel = DiscordChannel(DiscordConfig(enabled=True, allow_from=["123"]), MessageBus())
@@ -443,9 +563,7 @@ async def test_slash_new_forwards_when_user_is_allowlisted() -> None:
assert new_cmd is not None
await new_cmd.callback(interaction)
assert interaction.response.messages == [
{"content": "Processing /new...", "ephemeral": True}
]
assert interaction.response.messages == [{"content": "Processing /new...", "ephemeral": True}]
assert len(handled) == 1
assert handled[0]["content"] == "/new"
assert handled[0]["sender_id"] == "123"
@@ -519,9 +637,7 @@ async def test_slash_help_returns_ephemeral_help_text() -> None:
assert help_cmd is not None
await help_cmd.callback(interaction)
assert interaction.response.messages == [
{"content": build_help_text(), "ephemeral": True}
]
assert interaction.response.messages == [{"content": build_help_text(), "ephemeral": True}]
assert handled == []
@@ -656,11 +772,13 @@ async def test_start_typing_uses_typing_context_when_trigger_typing_missing() ->
def typing(self):
async def _waiter():
await release.wait()
# Hold the loop so task remains active until explicitly stopped.
class _Ctx(_TypingCtx):
async def __aenter__(self):
await super().__aenter__()
await _waiter()
return _Ctx()
typing_channel = _NoTriggerChannel(channel_id=123)
@@ -674,3 +792,214 @@ async def test_start_typing_uses_typing_context_when_trigger_typing_missing() ->
await asyncio.sleep(0)
assert channel._typing_tasks == {}
def test_config_accepts_proxy_fields() -> None:
config = DiscordConfig(
enabled=True,
token="token",
allow_from=["*"],
proxy="http://127.0.0.1:7890",
proxy_username="user",
proxy_password="pass",
)
assert config.proxy == "http://127.0.0.1:7890"
assert config.proxy_username == "user"
assert config.proxy_password == "pass"
def test_config_proxy_defaults_to_none() -> None:
config = DiscordConfig(enabled=True, token="token", allow_from=["*"])
assert config.proxy is None
assert config.proxy_username is None
assert config.proxy_password is None
@pytest.mark.asyncio
async def test_start_passes_proxy_to_client(monkeypatch) -> None:
_FakeDiscordClient.instances.clear()
channel = DiscordChannel(
DiscordConfig(
enabled=True,
token="token",
allow_from=["*"],
proxy="http://127.0.0.1:7890",
),
MessageBus(),
)
monkeypatch.setattr("nanobot.channels.discord.DiscordBotClient", _FakeDiscordClient)
await channel.start()
assert channel.is_running is False
assert len(_FakeDiscordClient.instances) == 1
assert _FakeDiscordClient.instances[0].proxy == "http://127.0.0.1:7890"
assert _FakeDiscordClient.instances[0].proxy_auth is None
@pytest.mark.asyncio
async def test_start_passes_proxy_auth_when_credentials_provided(monkeypatch) -> None:
aiohttp = pytest.importorskip("aiohttp")
_FakeDiscordClient.instances.clear()
channel = DiscordChannel(
DiscordConfig(
enabled=True,
token="token",
allow_from=["*"],
proxy="http://127.0.0.1:7890",
proxy_username="user",
proxy_password="pass",
),
MessageBus(),
)
monkeypatch.setattr("nanobot.channels.discord.DiscordBotClient", _FakeDiscordClient)
await channel.start()
assert channel.is_running is False
assert len(_FakeDiscordClient.instances) == 1
assert _FakeDiscordClient.instances[0].proxy == "http://127.0.0.1:7890"
assert _FakeDiscordClient.instances[0].proxy_auth is not None
assert isinstance(_FakeDiscordClient.instances[0].proxy_auth, aiohttp.BasicAuth)
assert _FakeDiscordClient.instances[0].proxy_auth.login == "user"
assert _FakeDiscordClient.instances[0].proxy_auth.password == "pass"
@pytest.mark.asyncio
async def test_start_no_proxy_auth_when_only_username(monkeypatch) -> None:
_FakeDiscordClient.instances.clear()
channel = DiscordChannel(
DiscordConfig(
enabled=True,
token="token",
allow_from=["*"],
proxy="http://127.0.0.1:7890",
proxy_username="user",
),
MessageBus(),
)
monkeypatch.setattr("nanobot.channels.discord.DiscordBotClient", _FakeDiscordClient)
await channel.start()
assert channel.is_running is False
assert _FakeDiscordClient.instances[0].proxy_auth is None
@pytest.mark.asyncio
async def test_start_no_proxy_auth_when_only_password(monkeypatch) -> None:
_FakeDiscordClient.instances.clear()
channel = DiscordChannel(
DiscordConfig(
enabled=True,
token="token",
allow_from=["*"],
proxy="http://127.0.0.1:7890",
proxy_password="pass",
),
MessageBus(),
)
monkeypatch.setattr("nanobot.channels.discord.DiscordBotClient", _FakeDiscordClient)
await channel.start()
assert channel.is_running is False
assert _FakeDiscordClient.instances[0].proxy == "http://127.0.0.1:7890"
assert _FakeDiscordClient.instances[0].proxy_auth is None
# ---------------------------------------------------------------------------
# Tests for the send() exception propagation fix
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_send_re_raises_network_error() -> None:
"""Network errors during send must propagate so ChannelManager can retry."""
channel = DiscordChannel(DiscordConfig(enabled=True, allow_from=["*"]), MessageBus())
client = _FakeDiscordClient(channel, intents=None)
channel._client = client
channel._running = True
async def _failing_send_outbound(msg: OutboundMessage) -> None:
raise ConnectionError("network unreachable")
client.send_outbound = _failing_send_outbound # type: ignore[method-assign]
with pytest.raises(ConnectionError, match="network unreachable"):
await channel.send(OutboundMessage(channel="discord", chat_id="123", content="hello"))
@pytest.mark.asyncio
async def test_send_re_raises_generic_exception() -> None:
"""Any exception from send_outbound must propagate, not be swallowed."""
channel = DiscordChannel(DiscordConfig(enabled=True, allow_from=["*"]), MessageBus())
client = _FakeDiscordClient(channel, intents=None)
channel._client = client
channel._running = True
async def _failing_send_outbound(msg: OutboundMessage) -> None:
raise RuntimeError("discord API failure")
client.send_outbound = _failing_send_outbound # type: ignore[method-assign]
with pytest.raises(RuntimeError, match="discord API failure"):
await channel.send(OutboundMessage(channel="discord", chat_id="123", content="hello"))
@pytest.mark.asyncio
async def test_send_still_stops_typing_on_error() -> None:
"""Typing cleanup must still run in the finally block even when send raises."""
channel = DiscordChannel(DiscordConfig(enabled=True, allow_from=["*"]), MessageBus())
client = _FakeDiscordClient(channel, intents=None)
channel._client = client
channel._running = True
# Start a typing task so we can verify it gets cleaned up
start = asyncio.Event()
release = asyncio.Event()
async def slow_typing() -> None:
start.set()
await release.wait()
typing_channel = _FakeChannel(channel_id=123)
typing_channel.typing_enter_hook = slow_typing
await channel._start_typing(typing_channel)
await asyncio.wait_for(start.wait(), timeout=1.0)
async def _failing_send_outbound(msg: OutboundMessage) -> None:
raise ConnectionError("timeout")
client.send_outbound = _failing_send_outbound # type: ignore[method-assign]
with pytest.raises(ConnectionError, match="timeout"):
await channel.send(OutboundMessage(channel="discord", chat_id="123", content="hello"))
release.set()
await asyncio.sleep(0)
# Typing should have been cleaned up by the finally block
assert channel._typing_tasks == {}
@pytest.mark.asyncio
async def test_send_succeeds_normally() -> None:
"""Successful sends should work without raising."""
channel = DiscordChannel(DiscordConfig(enabled=True, allow_from=["*"]), MessageBus())
client = _FakeDiscordClient(channel, intents=None)
channel._client = client
channel._running = True
sent_messages: list[OutboundMessage] = []
async def _capture_send_outbound(msg: OutboundMessage) -> None:
sent_messages.append(msg)
client.send_outbound = _capture_send_outbound # type: ignore[method-assign]
msg = OutboundMessage(channel="discord", chat_id="123", content="hello world")
await channel.send(msg)
assert len(sent_messages) == 1
assert sent_messages[0].content == "hello world"
assert sent_messages[0].chat_id == "123"
+9 -56
View File
@@ -205,53 +205,22 @@ class TestSendDelta:
ch._client.im.v1.message.create.assert_called_once()
@pytest.mark.asyncio
async def test_stream_end_resuming_keeps_buffer(self):
"""_resuming=True flushes text to card but keeps the buffer for the next segment."""
async def test_stream_end_fallback_when_final_update_fails(self):
"""If streaming mode was closed (e.g. Feishu timeout), fall back to a regular card."""
ch = _make_channel()
ch._stream_bufs["oc_chat1"] = _FeishuStreamBuf(
text="Partial answer", card_id="card_1", sequence=2, last_edit=0.0,
text="Lost content", card_id="card_1", sequence=3, last_edit=0.0,
)
ch._client.cardkit.v1.card_element.content.return_value = _mock_content_response()
ch._client.cardkit.v1.card_element.content.return_value = _mock_content_response(success=False)
ch._client.im.v1.message.create.return_value = _mock_send_response("om_fb")
await ch.send_delta("oc_chat1", "", metadata={"_stream_end": True, "_resuming": True})
assert "oc_chat1" in ch._stream_bufs
buf = ch._stream_bufs["oc_chat1"]
assert buf.card_id == "card_1"
assert buf.sequence == 3
ch._client.cardkit.v1.card_element.content.assert_called_once()
ch._client.cardkit.v1.card.settings.assert_not_called()
@pytest.mark.asyncio
async def test_stream_end_resuming_then_final_end(self):
"""Full multi-segment flow: resuming mid-turn, then final end closes the card."""
ch = _make_channel()
ch._stream_bufs["oc_chat1"] = _FeishuStreamBuf(
text="Seg1", card_id="card_1", sequence=1, last_edit=0.0,
)
ch._client.cardkit.v1.card_element.content.return_value = _mock_content_response()
ch._client.cardkit.v1.card.settings.return_value = _mock_content_response()
await ch.send_delta("oc_chat1", "", metadata={"_stream_end": True, "_resuming": True})
assert "oc_chat1" in ch._stream_bufs
ch._stream_bufs["oc_chat1"].text += " Seg2"
await ch.send_delta("oc_chat1", "", metadata={"_stream_end": True})
assert "oc_chat1" not in ch._stream_bufs
ch._client.cardkit.v1.card.settings.assert_called_once()
@pytest.mark.asyncio
async def test_stream_end_resuming_no_card_is_noop(self):
"""_resuming with no card_id (card creation failed) is a safe no-op."""
ch = _make_channel()
ch._stream_bufs["oc_chat1"] = _FeishuStreamBuf(
text="text", card_id=None, sequence=0, last_edit=0.0,
)
await ch.send_delta("oc_chat1", "", metadata={"_stream_end": True, "_resuming": True})
assert "oc_chat1" in ch._stream_bufs
ch._client.cardkit.v1.card_element.content.assert_not_called()
# Should NOT attempt to close streaming mode since update failed
ch._client.cardkit.v1.card.settings.assert_not_called()
# Should fall back to sending a regular interactive card
ch._client.im.v1.message.create.assert_called_once()
@pytest.mark.asyncio
async def test_stream_end_without_buf_is_noop(self):
@@ -375,22 +344,6 @@ class TestToolHintInlineStreaming:
assert "🔧 $ cd /project" in buf.text
assert "🔧 $ git status" in buf.text
@pytest.mark.asyncio
async def test_tool_hint_preserved_on_resuming_flush(self):
"""When _resuming flushes the buffer, tool hint is kept as permanent content."""
ch = _make_channel()
ch._stream_bufs["oc_chat1"] = _FeishuStreamBuf(
text="Partial answer\n\n🔧 $ cd /project\n\n",
card_id="card_1", sequence=2, last_edit=0.0,
)
ch._client.cardkit.v1.card_element.content.return_value = _mock_content_response()
await ch.send_delta("oc_chat1", "", metadata={"_stream_end": True, "_resuming": True})
buf = ch._stream_bufs["oc_chat1"]
assert "Partial answer" in buf.text
assert "🔧 $ cd /project" in buf.text
@pytest.mark.asyncio
async def test_tool_hint_preserved_on_final_stream_end(self):
"""When final _stream_end closes the card, tool hint is kept in the final text."""
@@ -1,6 +1,7 @@
"""Tests for FeishuChannel tool hint code block formatting."""
"""Tests for FeishuChannel tool hint formatting."""
import json
from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import pytest
@@ -28,15 +29,24 @@ def mock_feishu_channel():
config.app_secret = "test_app_secret"
config.encrypt_key = None
config.verification_token = None
config.tool_hint_prefix = "\U0001f527" # 🔧
bus = MagicMock()
channel = FeishuChannel(config, bus)
channel._client = MagicMock() # Simulate initialized client
channel._client = MagicMock()
return channel
def _get_tool_hint_card(mock_send):
"""Extract the interactive card from _send_message_sync calls."""
call_args = mock_send.call_args[0]
_, _, msg_type, content = call_args
assert msg_type == "interactive"
return json.loads(content)
@mark.asyncio
async def test_tool_hint_sends_code_message(mock_feishu_channel):
"""Tool hint messages should be sent as interactive cards with code blocks."""
async def test_tool_hint_sends_interactive_card(mock_feishu_channel):
"""Tool hint without active buffer sends an interactive card with 🔧 style."""
msg = OutboundMessage(
channel="feishu",
chat_id="oc_123456",
@@ -47,23 +57,12 @@ async def test_tool_hint_sends_code_message(mock_feishu_channel):
with patch.object(mock_feishu_channel, '_send_message_sync') as mock_send:
await mock_feishu_channel.send(msg)
# Verify interactive message with card was sent
assert mock_send.call_count == 1
call_args = mock_send.call_args[0]
receive_id_type, receive_id, msg_type, content = call_args
assert receive_id_type == "chat_id"
assert receive_id == "oc_123456"
assert msg_type == "interactive"
# Parse content to verify card structure
card = json.loads(content)
card = _get_tool_hint_card(mock_send)
assert card["config"]["wide_screen_mode"] is True
assert len(card["elements"]) == 1
assert card["elements"][0]["tag"] == "markdown"
# Check that code block is properly formatted with language hint
expected_md = "**Tool Calls**\n\n```text\nweb_search(\"test query\")\n```"
assert card["elements"][0]["content"] == expected_md
md = card["elements"][0]["content"]
assert "\U0001f527" in md
assert "web_search" in md
@mark.asyncio
@@ -78,8 +77,6 @@ async def test_tool_hint_empty_content_does_not_send(mock_feishu_channel):
with patch.object(mock_feishu_channel, '_send_message_sync') as mock_send:
await mock_feishu_channel.send(msg)
# Should not send any message
mock_send.assert_not_called()
@@ -96,7 +93,6 @@ async def test_tool_hint_without_metadata_sends_as_normal(mock_feishu_channel):
with patch.object(mock_feishu_channel, '_send_message_sync') as mock_send:
await mock_feishu_channel.send(msg)
# Should send as text message (detected format)
assert mock_send.call_count == 1
call_args = mock_send.call_args[0]
_, _, msg_type, content = call_args
@@ -106,7 +102,7 @@ async def test_tool_hint_without_metadata_sends_as_normal(mock_feishu_channel):
@mark.asyncio
async def test_tool_hint_multiple_tools_in_one_message(mock_feishu_channel):
"""Multiple tool calls should be displayed each on its own line in a code block."""
"""Multiple tool calls should each get the 🔧 prefix."""
msg = OutboundMessage(
channel="feishu",
chat_id="oc_123456",
@@ -117,13 +113,11 @@ async def test_tool_hint_multiple_tools_in_one_message(mock_feishu_channel):
with patch.object(mock_feishu_channel, '_send_message_sync') as mock_send:
await mock_feishu_channel.send(msg)
call_args = mock_send.call_args[0]
msg_type = call_args[2]
content = json.loads(call_args[3])
assert msg_type == "interactive"
# Each tool call should be on its own line
expected_md = "**Tool Calls**\n\n```text\nweb_search(\"query\"),\nread_file(\"/path/to/file\")\n```"
assert content["elements"][0]["content"] == expected_md
card = _get_tool_hint_card(mock_send)
md = card["elements"][0]["content"]
assert "web_search" in md
assert "read_file" in md
assert "\U0001f527" in md
@mark.asyncio
@@ -139,8 +133,8 @@ async def test_tool_hint_new_format_basic(mock_feishu_channel):
with patch.object(mock_feishu_channel, '_send_message_sync') as mock_send:
await mock_feishu_channel.send(msg)
content = json.loads(mock_send.call_args[0][3])
md = content["elements"][0]["content"]
card = _get_tool_hint_card(mock_send)
md = card["elements"][0]["content"]
assert "read src/main.py" in md
assert 'grep "TODO"' in md
@@ -158,16 +152,15 @@ async def test_tool_hint_new_format_with_comma_in_quotes(mock_feishu_channel):
with patch.object(mock_feishu_channel, '_send_message_sync') as mock_send:
await mock_feishu_channel.send(msg)
content = json.loads(mock_send.call_args[0][3])
md = content["elements"][0]["content"]
# The comma inside quotes should NOT cause a line break
card = _get_tool_hint_card(mock_send)
md = card["elements"][0]["content"]
assert 'grep "hello, world"' in md
assert "$ echo test" in md
@mark.asyncio
async def test_tool_hint_new_format_with_folding(mock_feishu_channel):
"""Folded calls (× N) should display on separate lines."""
"""Folded calls (× N) should display correctly."""
msg = OutboundMessage(
channel="feishu",
chat_id="oc_123456",
@@ -178,8 +171,8 @@ async def test_tool_hint_new_format_with_folding(mock_feishu_channel):
with patch.object(mock_feishu_channel, '_send_message_sync') as mock_send:
await mock_feishu_channel.send(msg)
content = json.loads(mock_send.call_args[0][3])
md = content["elements"][0]["content"]
card = _get_tool_hint_card(mock_send)
md = card["elements"][0]["content"]
assert "\u00d7 3" in md
assert 'grep "pattern"' in md
@@ -197,9 +190,12 @@ async def test_tool_hint_new_format_mcp(mock_feishu_channel):
with patch.object(mock_feishu_channel, '_send_message_sync') as mock_send:
await mock_feishu_channel.send(msg)
content = json.loads(mock_send.call_args[0][3])
md = content["elements"][0]["content"]
card = _get_tool_hint_card(mock_send)
md = card["elements"][0]["content"]
assert "4_5v::analyze_image" in md
@mark.asyncio
async def test_tool_hint_keeps_commas_inside_arguments(mock_feishu_channel):
"""Commas inside a single tool argument must not be split onto a new line."""
msg = OutboundMessage(
@@ -212,10 +208,7 @@ async def test_tool_hint_keeps_commas_inside_arguments(mock_feishu_channel):
with patch.object(mock_feishu_channel, '_send_message_sync') as mock_send:
await mock_feishu_channel.send(msg)
content = json.loads(mock_send.call_args[0][3])
expected_md = (
"**Tool Calls**\n\n```text\n"
"web_search(\"foo, bar\"),\n"
"read_file(\"/path/to/file\")\n```"
)
assert content["elements"][0]["content"] == expected_md
card = _get_tool_hint_card(mock_send)
md = card["elements"][0]["content"]
assert 'web_search("foo, bar")' in md
assert 'read_file("/path/to/file")' in md
+221
View File
@@ -1,6 +1,7 @@
import tempfile
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import AsyncMock, patch
import pytest
@@ -14,6 +15,8 @@ except ImportError:
if not QQ_AVAILABLE:
pytest.skip("QQ dependencies not installed (qq-botpy)", allow_module_level=True)
import aiohttp
from nanobot.bus.events import OutboundMessage
from nanobot.bus.queue import MessageBus
from nanobot.channels.qq import QQChannel, QQConfig
@@ -170,3 +173,221 @@ async def test_read_media_bytes_missing_file() -> None:
data, filename = await channel._read_media_bytes("/nonexistent/path/image.png")
assert data is None
assert filename is None
# -------------------------------------------------------
# Tests for _send_media exception handling
# -------------------------------------------------------
def _make_channel_with_local_file(suffix: str = ".png", content: bytes = b"\x89PNG\r\n"):
"""Create a QQChannel with a fake client and a temp file for media."""
channel = QQChannel(
QQConfig(app_id="app", secret="secret", allow_from=["*"]),
MessageBus(),
)
channel._client = _FakeClient()
channel._chat_type_cache["user1"] = "c2c"
tmp = tempfile.NamedTemporaryFile(suffix=suffix, delete=False)
tmp.write(content)
tmp.close()
return channel, tmp.name
@pytest.mark.asyncio
async def test_send_media_network_error_propagates() -> None:
"""aiohttp.ClientError (network/transport) should re-raise, not return False."""
channel, tmp_path = _make_channel_with_local_file()
# Make the base64 upload raise a network error
channel._client.api._http = SimpleNamespace()
channel._client.api._http.request = AsyncMock(
side_effect=aiohttp.ServerDisconnectedError("connection lost"),
)
with pytest.raises(aiohttp.ServerDisconnectedError):
await channel._send_media(
chat_id="user1",
media_ref=tmp_path,
msg_id="msg1",
is_group=False,
)
@pytest.mark.asyncio
async def test_send_media_client_connector_error_propagates() -> None:
"""aiohttp.ClientConnectorError (DNS/connection refused) should re-raise."""
channel, tmp_path = _make_channel_with_local_file()
from aiohttp.client_reqrep import ConnectionKey
conn_key = ConnectionKey("api.qq.com", 443, True, None, None, None, None)
connector_error = aiohttp.ClientConnectorError(
connection_key=conn_key,
os_error=OSError("Connection refused"),
)
channel._client.api._http = SimpleNamespace()
channel._client.api._http.request = AsyncMock(
side_effect=connector_error,
)
with pytest.raises(aiohttp.ClientConnectorError):
await channel._send_media(
chat_id="user1",
media_ref=tmp_path,
msg_id="msg1",
is_group=False,
)
@pytest.mark.asyncio
async def test_send_media_oserror_propagates() -> None:
"""OSError (low-level I/O) should re-raise for retry."""
channel, tmp_path = _make_channel_with_local_file()
channel._client.api._http = SimpleNamespace()
channel._client.api._http.request = AsyncMock(
side_effect=OSError("Network is unreachable"),
)
with pytest.raises(OSError):
await channel._send_media(
chat_id="user1",
media_ref=tmp_path,
msg_id="msg1",
is_group=False,
)
@pytest.mark.asyncio
async def test_send_media_api_error_returns_false() -> None:
"""API-level errors (botpy RuntimeError subclasses) should return False, not raise."""
channel, tmp_path = _make_channel_with_local_file()
# Simulate a botpy API error (e.g. ServerError is a RuntimeError subclass)
from botpy.errors import ServerError
channel._client.api._http = SimpleNamespace()
channel._client.api._http.request = AsyncMock(
side_effect=ServerError("internal server error"),
)
result = await channel._send_media(
chat_id="user1",
media_ref=tmp_path,
msg_id="msg1",
is_group=False,
)
assert result is False
@pytest.mark.asyncio
async def test_send_media_generic_runtime_error_returns_false() -> None:
"""Generic RuntimeError (not network) should return False."""
channel, tmp_path = _make_channel_with_local_file()
channel._client.api._http = SimpleNamespace()
channel._client.api._http.request = AsyncMock(
side_effect=RuntimeError("some API error"),
)
result = await channel._send_media(
chat_id="user1",
media_ref=tmp_path,
msg_id="msg1",
is_group=False,
)
assert result is False
@pytest.mark.asyncio
async def test_send_media_value_error_returns_false() -> None:
"""ValueError (bad API response data) should return False."""
channel, tmp_path = _make_channel_with_local_file()
channel._client.api._http = SimpleNamespace()
channel._client.api._http.request = AsyncMock(
side_effect=ValueError("bad response data"),
)
result = await channel._send_media(
chat_id="user1",
media_ref=tmp_path,
msg_id="msg1",
is_group=False,
)
assert result is False
@pytest.mark.asyncio
async def test_send_media_timeout_error_propagates() -> None:
"""asyncio.TimeoutError inherits from Exception but not ClientError/OSError.
However, aiohttp.ServerTimeoutError IS a ClientError subclass, so that propagates.
For a plain TimeoutError (which is also OSError in Python 3.11+), it should propagate."""
channel, tmp_path = _make_channel_with_local_file()
channel._client.api._http = SimpleNamespace()
channel._client.api._http.request = AsyncMock(
side_effect=aiohttp.ServerTimeoutError("request timed out"),
)
with pytest.raises(aiohttp.ServerTimeoutError):
await channel._send_media(
chat_id="user1",
media_ref=tmp_path,
msg_id="msg1",
is_group=False,
)
@pytest.mark.asyncio
async def test_send_fallback_text_on_api_error() -> None:
"""When _send_media returns False (API error), send() should emit fallback text."""
channel, tmp_path = _make_channel_with_local_file()
from botpy.errors import ServerError
channel._client.api._http = SimpleNamespace()
channel._client.api._http.request = AsyncMock(
side_effect=ServerError("internal server error"),
)
await channel.send(
OutboundMessage(
channel="qq",
chat_id="user1",
content="",
media=[tmp_path],
metadata={"message_id": "msg1"},
)
)
# Should have sent a fallback text message
assert len(channel._client.api.c2c_calls) == 1
fallback_content = channel._client.api.c2c_calls[0]["content"]
assert "Attachment send failed" in fallback_content
@pytest.mark.asyncio
async def test_send_propagates_network_error_no_fallback() -> None:
"""When _send_media raises a network error, send() should NOT silently fallback."""
channel, tmp_path = _make_channel_with_local_file()
channel._client.api._http = SimpleNamespace()
channel._client.api._http.request = AsyncMock(
side_effect=aiohttp.ServerDisconnectedError("connection lost"),
)
with pytest.raises(aiohttp.ServerDisconnectedError):
await channel.send(
OutboundMessage(
channel="qq",
chat_id="user1",
content="hello",
media=[tmp_path],
metadata={"message_id": "msg1"},
)
)
# No fallback text should have been sent
assert len(channel._client.api.c2c_calls) == 0
+167 -2
View File
@@ -10,8 +10,7 @@ except ImportError:
from nanobot.bus.events import OutboundMessage
from nanobot.bus.queue import MessageBus
from nanobot.channels.slack import SlackChannel
from nanobot.channels.slack import SlackConfig
from nanobot.channels.slack import SlackChannel, SlackConfig
class _FakeAsyncWebClient:
@@ -20,6 +19,12 @@ class _FakeAsyncWebClient:
self.file_upload_calls: list[dict[str, object | None]] = []
self.reactions_add_calls: list[dict[str, object | None]] = []
self.reactions_remove_calls: list[dict[str, object | None]] = []
self.conversations_list_calls: list[dict[str, object | None]] = []
self.users_list_calls: list[dict[str, object | None]] = []
self.conversations_open_calls: list[dict[str, object | None]] = []
self._conversations_pages: list[dict[str, object]] = []
self._users_pages: list[dict[str, object]] = []
self._open_dm_response: dict[str, object] = {"channel": {"id": "D_OPENED"}}
async def chat_postMessage(
self,
@@ -81,6 +86,22 @@ class _FakeAsyncWebClient:
}
)
async def conversations_list(self, **kwargs):
self.conversations_list_calls.append(kwargs)
if self._conversations_pages:
return self._conversations_pages.pop(0)
return {"channels": [], "response_metadata": {"next_cursor": ""}}
async def users_list(self, **kwargs):
self.users_list_calls.append(kwargs)
if self._users_pages:
return self._users_pages.pop(0)
return {"members": [], "response_metadata": {"next_cursor": ""}}
async def conversations_open(self, **kwargs):
self.conversations_open_calls.append(kwargs)
return self._open_dm_response
@pytest.mark.asyncio
async def test_send_uses_thread_for_channel_messages() -> None:
@@ -151,3 +172,147 @@ async def test_send_updates_reaction_when_final_response_sent() -> None:
assert fake_web.reactions_add_calls == [
{"channel": "C123", "name": "white_check_mark", "timestamp": "1700000000.000100"}
]
@pytest.mark.asyncio
async def test_send_resolves_channel_name_to_channel_id() -> None:
channel = SlackChannel(SlackConfig(enabled=True), MessageBus())
fake_web = _FakeAsyncWebClient()
fake_web._conversations_pages = [
{
"channels": [{"id": "C999", "name": "channel_x"}],
"response_metadata": {"next_cursor": ""},
}
]
channel._web_client = fake_web
await channel.send(
OutboundMessage(
channel="slack",
chat_id="#channel_x",
content="hello",
)
)
assert fake_web.chat_post_calls == [
{"channel": "C999", "text": "hello\n", "thread_ts": None}
]
assert len(fake_web.conversations_list_calls) == 1
@pytest.mark.asyncio
async def test_send_resolves_user_handle_to_dm_channel() -> None:
channel = SlackChannel(SlackConfig(enabled=True), MessageBus())
fake_web = _FakeAsyncWebClient()
fake_web._users_pages = [
{
"members": [
{
"id": "U234",
"name": "alice",
"profile": {"display_name": "Alice"},
}
],
"response_metadata": {"next_cursor": ""},
}
]
fake_web._open_dm_response = {"channel": {"id": "D234"}}
channel._web_client = fake_web
await channel.send(
OutboundMessage(
channel="slack",
chat_id="@alice",
content="hello",
)
)
assert fake_web.conversations_open_calls == [{"users": "U234"}]
assert fake_web.chat_post_calls == [
{"channel": "D234", "text": "hello\n", "thread_ts": None}
]
@pytest.mark.asyncio
async def test_send_updates_reaction_on_origin_channel_for_cross_channel_send() -> None:
channel = SlackChannel(SlackConfig(enabled=True, react_emoji="eyes"), MessageBus())
fake_web = _FakeAsyncWebClient()
fake_web._conversations_pages = [
{
"channels": [{"id": "C999", "name": "channel_x"}],
"response_metadata": {"next_cursor": ""},
}
]
channel._web_client = fake_web
await channel.send(
OutboundMessage(
channel="slack",
chat_id="channel_x",
content="done",
metadata={
"slack": {
"event": {"ts": "1700000000.000100", "channel": "D_ORIGIN"},
"channel_type": "im",
},
},
)
)
assert fake_web.chat_post_calls == [
{"channel": "C999", "text": "done\n", "thread_ts": None}
]
assert fake_web.reactions_remove_calls == [
{"channel": "D_ORIGIN", "name": "eyes", "timestamp": "1700000000.000100"}
]
assert fake_web.reactions_add_calls == [
{"channel": "D_ORIGIN", "name": "white_check_mark", "timestamp": "1700000000.000100"}
]
@pytest.mark.asyncio
async def test_send_does_not_reuse_origin_thread_ts_for_cross_channel_send() -> None:
channel = SlackChannel(SlackConfig(enabled=True), MessageBus())
fake_web = _FakeAsyncWebClient()
fake_web._conversations_pages = [
{
"channels": [{"id": "C999", "name": "channel_x"}],
"response_metadata": {"next_cursor": ""},
}
]
channel._web_client = fake_web
await channel.send(
OutboundMessage(
channel="slack",
chat_id="channel_x",
content="done",
metadata={
"slack": {
"event": {"ts": "1700000000.000100", "channel": "C_ORIGIN"},
"thread_ts": "1700000000.000200",
"channel_type": "channel",
},
},
)
)
assert fake_web.chat_post_calls == [
{"channel": "C999", "text": "done\n", "thread_ts": None}
]
@pytest.mark.asyncio
async def test_send_raises_when_named_target_cannot_be_resolved() -> None:
channel = SlackChannel(SlackConfig(enabled=True), MessageBus())
fake_web = _FakeAsyncWebClient()
channel._web_client = fake_web
with pytest.raises(ValueError, match="was not found"):
await channel.send(
OutboundMessage(
channel="slack",
chat_id="#missing-channel",
content="hello",
)
)
+281
View File
@@ -135,6 +135,7 @@ def _make_telegram_update(
entities=None,
caption_entities=None,
reply_to_message=None,
location=None,
):
user = SimpleNamespace(id=12345, username="alice", first_name="Alice")
message = SimpleNamespace(
@@ -149,6 +150,7 @@ def _make_telegram_update(
voice=None,
audio=None,
document=None,
location=location,
media_group_id=None,
message_thread_id=None,
message_id=1,
@@ -385,6 +387,84 @@ async def test_send_delta_stream_end_treats_not_modified_as_success() -> None:
assert "123" not in channel._stream_bufs
@pytest.mark.asyncio
async def test_send_delta_stream_end_does_not_fallback_on_network_timeout() -> None:
"""TimedOut during HTML edit should propagate, never fall back to plain text."""
from telegram.error import TimedOut
channel = TelegramChannel(
TelegramConfig(enabled=True, token="123:abc", allow_from=["*"]),
MessageBus(),
)
channel._app = _FakeApp(lambda: None)
# _call_with_retry retries TimedOut up to 3 times, so the mock will be called
# multiple times but all calls must be with parse_mode="HTML" (no plain fallback).
channel._app.bot.edit_message_text = AsyncMock(side_effect=TimedOut("network timeout"))
channel._stream_bufs["123"] = _StreamBuf(text="hello", message_id=7, last_edit=0.0)
with pytest.raises(TimedOut, match="network timeout"):
await channel.send_delta("123", "", {"_stream_end": True})
# Every call to edit_message_text must have used parse_mode="HTML" —
# no plain-text fallback call should have been made.
for call in channel._app.bot.edit_message_text.call_args_list:
assert call.kwargs.get("parse_mode") == "HTML"
# Buffer should still be present (not cleaned up on error)
assert "123" in channel._stream_bufs
@pytest.mark.asyncio
async def test_send_delta_stream_end_does_not_fallback_on_network_error() -> None:
"""NetworkError during HTML edit should propagate, never fall back to plain text."""
from telegram.error import NetworkError
channel = TelegramChannel(
TelegramConfig(enabled=True, token="123:abc", allow_from=["*"]),
MessageBus(),
)
channel._app = _FakeApp(lambda: None)
channel._app.bot.edit_message_text = AsyncMock(side_effect=NetworkError("connection reset"))
channel._stream_bufs["123"] = _StreamBuf(text="hello", message_id=7, last_edit=0.0)
with pytest.raises(NetworkError, match="connection reset"):
await channel.send_delta("123", "", {"_stream_end": True})
# Every call to edit_message_text must have used parse_mode="HTML" —
# no plain-text fallback call should have been made.
for call in channel._app.bot.edit_message_text.call_args_list:
assert call.kwargs.get("parse_mode") == "HTML"
# Buffer should still be present (not cleaned up on error)
assert "123" in channel._stream_bufs
@pytest.mark.asyncio
async def test_send_delta_stream_end_falls_back_on_bad_request() -> None:
"""BadRequest (HTML parse error) should still trigger plain-text fallback."""
from telegram.error import BadRequest
channel = TelegramChannel(
TelegramConfig(enabled=True, token="123:abc", allow_from=["*"]),
MessageBus(),
)
channel._app = _FakeApp(lambda: None)
# First call (HTML) raises BadRequest, second call (plain) succeeds
channel._app.bot.edit_message_text = AsyncMock(
side_effect=[BadRequest("Can't parse entities"), None]
)
channel._stream_bufs["123"] = _StreamBuf(text="hello <bad>", message_id=7, last_edit=0.0)
await channel.send_delta("123", "", {"_stream_end": True})
# edit_message_text should have been called twice: once for HTML, once for plain fallback
assert channel._app.bot.edit_message_text.call_count == 2
# Second call should not use parse_mode="HTML"
second_call_kwargs = channel._app.bot.edit_message_text.call_args_list[1].kwargs
assert "parse_mode" not in second_call_kwargs or second_call_kwargs.get("parse_mode") is None
# Buffer should be cleaned up on success
assert "123" not in channel._stream_bufs
@pytest.mark.asyncio
async def test_send_delta_stream_end_splits_oversized_reply() -> None:
"""Final streamed reply exceeding Telegram limit is split into chunks."""
@@ -1112,3 +1192,204 @@ async def test_on_help_includes_restart_command() -> None:
assert "/dream" in help_text
assert "/dream-log" in help_text
assert "/dream-restore" in help_text
@pytest.mark.asyncio
async def test_on_message_location_content() -> None:
"""Location messages are forwarded as [location: lat, lon] content."""
channel = TelegramChannel(
TelegramConfig(enabled=True, token="123:abc", allow_from=["*"], group_policy="open"),
MessageBus(),
)
channel._app = _FakeApp(lambda: None)
handled = []
async def capture_handle(**kwargs) -> None:
handled.append(kwargs)
channel._handle_message = capture_handle
channel._start_typing = lambda _chat_id: None
location = SimpleNamespace(latitude=48.8566, longitude=2.3522)
update = _make_telegram_update(location=location)
await channel._on_message(update, None)
assert len(handled) == 1
assert handled[0]["content"] == "[location: 48.8566, 2.3522]"
@pytest.mark.asyncio
async def test_on_message_location_with_text() -> None:
"""Location messages with accompanying text include both in content."""
channel = TelegramChannel(
TelegramConfig(enabled=True, token="123:abc", allow_from=["*"], group_policy="open"),
MessageBus(),
)
channel._app = _FakeApp(lambda: None)
handled = []
async def capture_handle(**kwargs) -> None:
handled.append(kwargs)
channel._handle_message = capture_handle
channel._start_typing = lambda _chat_id: None
location = SimpleNamespace(latitude=51.5074, longitude=-0.1278)
update = _make_telegram_update(text="meet me here", location=location)
await channel._on_message(update, None)
assert len(handled) == 1
assert "meet me here" in handled[0]["content"]
assert "[location: 51.5074, -0.1278]" in handled[0]["content"]
# ---------------------------------------------------------------------------
# Tests for retry amplification fix (issue #3050)
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_send_text_does_not_fallback_on_network_timeout() -> None:
"""TimedOut should propagate immediately, NOT trigger plain-text fallback.
Before the fix, _send_text caught ALL exceptions (including TimedOut)
and retried as plain text, doubling connection demand during pool
exhaustion see issue #3050.
"""
from telegram.error import TimedOut
channel = TelegramChannel(
TelegramConfig(enabled=True, token="123:abc", allow_from=["*"]),
MessageBus(),
)
channel._app = _FakeApp(lambda: None)
call_count = 0
async def always_timeout(**kwargs):
nonlocal call_count
call_count += 1
raise TimedOut()
channel._app.bot.send_message = always_timeout
import nanobot.channels.telegram as tg_mod
orig_delay = tg_mod._SEND_RETRY_BASE_DELAY
tg_mod._SEND_RETRY_BASE_DELAY = 0.01
try:
with pytest.raises(TimedOut):
await channel._send_text(123, "hello", None, {})
finally:
tg_mod._SEND_RETRY_BASE_DELAY = orig_delay
# With the fix: only _call_with_retry's 3 HTML attempts (no plain fallback).
# Before the fix: 3 HTML + 3 plain = 6 attempts.
assert call_count == 3, (
f"Expected 3 calls (HTML retries only), got {call_count} "
"(plain-text fallback should not trigger on TimedOut)"
)
@pytest.mark.asyncio
async def test_send_text_does_not_fallback_on_network_error() -> None:
"""NetworkError should propagate immediately, NOT trigger plain-text fallback."""
from telegram.error import NetworkError
channel = TelegramChannel(
TelegramConfig(enabled=True, token="123:abc", allow_from=["*"]),
MessageBus(),
)
channel._app = _FakeApp(lambda: None)
call_count = 0
async def always_network_error(**kwargs):
nonlocal call_count
call_count += 1
raise NetworkError("Connection reset")
channel._app.bot.send_message = always_network_error
import nanobot.channels.telegram as tg_mod
orig_delay = tg_mod._SEND_RETRY_BASE_DELAY
tg_mod._SEND_RETRY_BASE_DELAY = 0.01
try:
with pytest.raises(NetworkError):
await channel._send_text(123, "hello", None, {})
finally:
tg_mod._SEND_RETRY_BASE_DELAY = orig_delay
# _call_with_retry does NOT retry NetworkError (only TimedOut/RetryAfter),
# so it raises after 1 attempt. The fix prevents plain-text fallback.
# Before the fix: 1 HTML + 1 plain = 2. After the fix: 1 HTML only.
assert call_count == 1, (
f"Expected 1 call (HTML only, no plain fallback), got {call_count}"
)
@pytest.mark.asyncio
async def test_send_text_falls_back_on_bad_request() -> None:
"""BadRequest (HTML parse error) should still trigger plain-text fallback."""
from telegram.error import BadRequest
channel = TelegramChannel(
TelegramConfig(enabled=True, token="123:abc", allow_from=["*"]),
MessageBus(),
)
channel._app = _FakeApp(lambda: None)
original_send = channel._app.bot.send_message
html_call_count = 0
async def html_fails(**kwargs):
nonlocal html_call_count
if kwargs.get("parse_mode") == "HTML":
html_call_count += 1
raise BadRequest("Can't parse entities")
return await original_send(**kwargs)
channel._app.bot.send_message = html_fails
import nanobot.channels.telegram as tg_mod
orig_delay = tg_mod._SEND_RETRY_BASE_DELAY
tg_mod._SEND_RETRY_BASE_DELAY = 0.01
try:
await channel._send_text(123, "hello **world**", None, {})
finally:
tg_mod._SEND_RETRY_BASE_DELAY = orig_delay
# HTML attempt failed with BadRequest → fallback to plain text succeeds.
assert html_call_count == 1, f"Expected 1 HTML attempt, got {html_call_count}"
assert len(channel._app.bot.sent_messages) == 1
# Plain text send should NOT have parse_mode
assert channel._app.bot.sent_messages[0].get("parse_mode") is None
@pytest.mark.asyncio
async def test_send_text_bad_request_plain_fallback_exhausted() -> None:
"""When both HTML and plain-text fallback fail with BadRequest, the error propagates."""
from telegram.error import BadRequest
channel = TelegramChannel(
TelegramConfig(enabled=True, token="123:abc", allow_from=["*"]),
MessageBus(),
)
channel._app = _FakeApp(lambda: None)
call_count = 0
async def always_bad_request(**kwargs):
nonlocal call_count
call_count += 1
raise BadRequest("Bad request")
channel._app.bot.send_message = always_bad_request
import nanobot.channels.telegram as tg_mod
orig_delay = tg_mod._SEND_RETRY_BASE_DELAY
tg_mod._SEND_RETRY_BASE_DELAY = 0.01
try:
with pytest.raises(BadRequest):
await channel._send_text(123, "hello", None, {})
finally:
tg_mod._SEND_RETRY_BASE_DELAY = orig_delay
# _call_with_retry does NOT retry BadRequest (only TimedOut/RetryAfter),
# so HTML fails after 1 attempt → fallback to plain also fails after 1 attempt.
# Before the fix: 2 total. After the fix: still 2 (BadRequest SHOULD fallback).
assert call_count == 2, f"Expected 2 calls (1 HTML + 1 plain), got {call_count}"
@@ -8,6 +8,7 @@ from __future__ import annotations
import asyncio
import json
from typing import Any
from unittest.mock import AsyncMock, MagicMock
import pytest
+44
View File
@@ -541,6 +541,50 @@ async def test_process_voice_message() -> None:
assert "[voice]" in msg.content
@pytest.mark.asyncio
async def test_process_mixed_message() -> None:
"""Mixed message: contains picture and text message types."""
channel = WecomChannel(WecomConfig(bot_id="b", secret="s", allow_from=["user1"]), MessageBus())
client = _FakeWeComClient()
with tempfile.NamedTemporaryFile(suffix=".png", delete=False) as f:
f.write(b"\x89PNG\r\n")
saved = f.name
client.download_file.return_value = (b"\x89PNG\r\n", "photo.png")
channel._client = client
try:
with patch("nanobot.channels.wecom.get_media_dir", return_value=Path(os.path.dirname(saved))):
frame = _FakeFrame(body={
"msgid": "msg_mixed_1",
"chatid": "chat1",
"msgtype": "mixed",
"from": {"userid": "user1"},
"mixed": {
"msg_item": [
{"msgtype": "text", "text": {"content": "hello wecom"}},
{"msgtype": "image", "image": {"url": "https://example.com/img.png", "aeskey": "key123"}}
]
}
})
await channel._process_message(frame, "mixed")
msg = await channel.bus.consume_inbound()
assert msg.sender_id == "user1"
assert msg.chat_id == "chat1"
assert msg.content.startswith("hello wecom")
assert msg.metadata["msg_type"] == "mixed"
assert len(msg.media) == 1
assert msg.media[0].endswith("photo.png")
assert "[image:" in msg.content
finally:
# Clean up any photo.png in tempdir
p = os.path.join(os.path.dirname(saved), "photo.png")
if os.path.exists(p):
os.unlink(p)
@pytest.mark.asyncio
async def test_process_message_deduplication() -> None:
"""Same msg_id is not processed twice."""
+182
View File
@@ -1003,3 +1003,185 @@ async def test_download_media_item_non_image_requires_aes_key_even_with_full_url
assert saved_path is None
channel._client.get.assert_not_awaited()
# ---------------------------------------------------------------------------
# Tests for media-send error classification (network vs non-network errors)
# ---------------------------------------------------------------------------
def _make_outbound_msg(chat_id: str = "wx-user", content: str = "", media: list | None = None):
"""Build a minimal OutboundMessage-like object for send() tests."""
from nanobot.bus.events import OutboundMessage
return OutboundMessage(
channel="weixin",
chat_id=chat_id,
content=content,
media=media or [],
metadata={},
)
@pytest.mark.asyncio
async def test_send_media_timeout_error_propagates_without_text_fallback() -> None:
"""httpx.TimeoutException during media send must re-raise immediately,
NOT fall back to _send_text (which would also fail during network issues)."""
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel._context_tokens["wx-user"] = "ctx-1"
channel._send_media_file = AsyncMock(side_effect=httpx.TimeoutException("timed out"))
channel._send_text = AsyncMock()
msg = _make_outbound_msg(chat_id="wx-user", media=["/tmp/photo.jpg"])
with pytest.raises(httpx.TimeoutException, match="timed out"):
await channel.send(msg)
# _send_text must NOT have been called as a fallback
channel._send_text.assert_not_awaited()
@pytest.mark.asyncio
async def test_send_media_transport_error_propagates_without_text_fallback() -> None:
"""httpx.TransportError during media send must re-raise immediately."""
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel._context_tokens["wx-user"] = "ctx-1"
channel._send_media_file = AsyncMock(
side_effect=httpx.TransportError("connection reset")
)
channel._send_text = AsyncMock()
msg = _make_outbound_msg(chat_id="wx-user", media=["/tmp/photo.jpg"])
with pytest.raises(httpx.TransportError, match="connection reset"):
await channel.send(msg)
channel._send_text.assert_not_awaited()
@pytest.mark.asyncio
async def test_send_media_5xx_http_status_error_propagates_without_text_fallback() -> None:
"""httpx.HTTPStatusError with a 5xx status must re-raise immediately."""
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel._context_tokens["wx-user"] = "ctx-1"
fake_response = httpx.Response(
status_code=503,
request=httpx.Request("POST", "https://example.test/upload"),
)
channel._send_media_file = AsyncMock(
side_effect=httpx.HTTPStatusError(
"Service Unavailable", request=fake_response.request, response=fake_response
)
)
channel._send_text = AsyncMock()
msg = _make_outbound_msg(chat_id="wx-user", media=["/tmp/photo.jpg"])
with pytest.raises(httpx.HTTPStatusError, match="Service Unavailable"):
await channel.send(msg)
channel._send_text.assert_not_awaited()
@pytest.mark.asyncio
async def test_send_media_4xx_http_status_error_falls_back_to_text() -> None:
"""httpx.HTTPStatusError with a 4xx status should fall back to text, not re-raise."""
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel._context_tokens["wx-user"] = "ctx-1"
fake_response = httpx.Response(
status_code=400,
request=httpx.Request("POST", "https://example.test/upload"),
)
channel._send_media_file = AsyncMock(
side_effect=httpx.HTTPStatusError(
"Bad Request", request=fake_response.request, response=fake_response
)
)
channel._send_text = AsyncMock()
msg = _make_outbound_msg(chat_id="wx-user", media=["/tmp/photo.jpg"])
# Should NOT raise — 4xx is a client error, non-retryable
await channel.send(msg)
# _send_text should have been called with the fallback message
channel._send_text.assert_awaited_once_with(
"wx-user", "[Failed to send: photo.jpg]", "ctx-1"
)
@pytest.mark.asyncio
async def test_send_media_file_not_found_falls_back_to_text() -> None:
"""FileNotFoundError (a non-network error) should fall back to text."""
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel._context_tokens["wx-user"] = "ctx-1"
channel._send_media_file = AsyncMock(
side_effect=FileNotFoundError("Media file not found: /tmp/missing.jpg")
)
channel._send_text = AsyncMock()
msg = _make_outbound_msg(chat_id="wx-user", media=["/tmp/missing.jpg"])
# Should NOT raise
await channel.send(msg)
channel._send_text.assert_awaited_once_with(
"wx-user", "[Failed to send: missing.jpg]", "ctx-1"
)
@pytest.mark.asyncio
async def test_send_media_value_error_falls_back_to_text() -> None:
"""ValueError (e.g. unsupported format) should fall back to text."""
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel._context_tokens["wx-user"] = "ctx-1"
channel._send_media_file = AsyncMock(
side_effect=ValueError("Unsupported media format")
)
channel._send_text = AsyncMock()
msg = _make_outbound_msg(chat_id="wx-user", media=["/tmp/file.xyz"])
# Should NOT raise
await channel.send(msg)
channel._send_text.assert_awaited_once_with(
"wx-user", "[Failed to send: file.xyz]", "ctx-1"
)
@pytest.mark.asyncio
async def test_send_media_network_error_does_not_double_api_calls() -> None:
"""During network issues, media send should make exactly 1 API call attempt,
not 2 (media + text fallback). Verify total call count."""
channel, _bus = _make_channel()
channel._client = object()
channel._token = "token"
channel._context_tokens["wx-user"] = "ctx-1"
channel._send_media_file = AsyncMock(
side_effect=httpx.ConnectError("connection refused")
)
channel._send_text = AsyncMock()
msg = _make_outbound_msg(chat_id="wx-user", content="hello", media=["/tmp/img.png"])
with pytest.raises(httpx.ConnectError):
await channel.send(msg)
# _send_media_file called once, _send_text never called
channel._send_media_file.assert_awaited_once()
channel._send_text.assert_not_awaited()
+169
View File
@@ -257,6 +257,28 @@ def test_config_accepts_camel_case_explicit_provider_name_for_coding_plan():
assert config.get_api_base() == "https://ark.cn-beijing.volces.com/api/coding/v3"
def test_config_accepts_lm_studio_without_api_key_and_uses_default_localhost_api_base():
config = Config.model_validate(
{
"agents": {
"defaults": {
"provider": "lm_studio",
"model": "local-model",
}
},
"providers": {
"lmStudio": {
"apiKey": None,
}
},
}
)
assert config.get_provider_name() == "lm_studio"
assert config.get_api_key() is None
assert config.get_api_base() == "http://localhost:1234/v1"
def test_find_by_name_accepts_camel_case_and_hyphen_aliases():
assert find_by_name("volcengineCodingPlan") is not None
assert find_by_name("volcengineCodingPlan").name == "volcengine_coding_plan"
@@ -1126,6 +1148,153 @@ def test_gateway_cli_port_overrides_configured_port(monkeypatch, tmp_path: Path)
assert "port 18792" in result.stdout
def test_gateway_health_endpoint_binds_and_serves_expected_responses(
monkeypatch, tmp_path: Path
) -> None:
config_file = _write_instance_config(tmp_path)
config = Config()
config.gateway.port = 18791
captured: dict[str, object] = {}
class _FakeDream:
model = None
max_batch_size = 0
max_iterations = 0
async def run(self) -> None:
return None
class _FakeAgentLoop:
def __init__(self, **_kwargs) -> None:
self.model = "test-model"
self.dream = _FakeDream()
async def run(self) -> None:
await asyncio.Event().wait()
async def close_mcp(self) -> None:
return None
def stop(self) -> None:
return None
class _FakeChannelManager:
def __init__(self, _config, _bus) -> None:
self.enabled_channels = ["telegram", "discord"]
async def start_all(self) -> None:
await asyncio.Event().wait()
async def stop_all(self) -> None:
return None
class _FakeCronService:
def __init__(self, _store_path: Path) -> None:
self.on_job = None
async def start(self) -> None:
return None
def stop(self) -> None:
return None
def status(self) -> dict[str, int]:
return {"jobs": 0}
def register_system_job(self, _job) -> None:
return None
class _FakeHeartbeatService:
def __init__(self, **_kwargs) -> None:
return None
async def start(self) -> None:
return None
def stop(self) -> None:
return None
class _FakeServer:
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb) -> bool:
return False
async def serve_forever(self) -> None:
raise _StopGatewayError("stop")
async def _fake_start_server(handler, host: str, port: int):
captured["handler"] = handler
captured["host"] = host
captured["port"] = port
return _FakeServer()
class _FakeReader:
def __init__(self, payload: bytes) -> None:
self.payload = payload
async def read(self, _size: int) -> bytes:
return self.payload
class _FakeWriter:
def __init__(self) -> None:
self.output = b""
self.closed = False
def write(self, data: bytes) -> None:
self.output += data
async def drain(self) -> None:
return None
def close(self) -> None:
self.closed = True
_patch_cli_command_runtime(
monkeypatch,
config,
message_bus=lambda: object(),
session_manager=lambda _workspace: object(),
)
monkeypatch.setattr("nanobot.agent.loop.AgentLoop", _FakeAgentLoop)
monkeypatch.setattr("nanobot.channels.manager.ChannelManager", _FakeChannelManager)
monkeypatch.setattr("nanobot.cron.service.CronService", _FakeCronService)
monkeypatch.setattr("nanobot.heartbeat.service.HeartbeatService", _FakeHeartbeatService)
monkeypatch.setattr("asyncio.start_server", _fake_start_server)
result = runner.invoke(app, ["gateway", "--config", str(config_file)])
assert result.exit_code == 0
assert captured["host"] == "127.0.0.1"
assert captured["port"] == 18791
assert "Health endpoint: http://127.0.0.1:18791/health" in result.stdout
def _call_handler(path: str) -> tuple[str, _FakeWriter]:
request = f"GET {path} HTTP/1.1\r\nHost: localhost\r\n\r\n".encode()
writer = _FakeWriter()
handler = captured["handler"]
assert callable(handler)
asyncio.run(handler(_FakeReader(request), writer))
return writer.output.decode(), writer
root_response, root_writer = _call_handler("/")
assert root_writer.closed is True
assert "HTTP/1.0 404 Not Found" in root_response
assert root_response.endswith("\r\n\r\nNot Found")
health_response, health_writer = _call_handler("/health")
assert health_writer.closed is True
assert "HTTP/1.0 200 OK" in health_response
health_body = json.loads(health_response.split("\r\n\r\n", 1)[1])
assert health_body == {"status": "ok"}
missing_response, missing_writer = _call_handler("/missing")
assert missing_writer.closed is True
assert "HTTP/1.0 404 Not Found" in missing_response
assert missing_response.endswith("\r\n\r\nNot Found")
def test_serve_uses_api_config_defaults_and_workspace_override(
monkeypatch, tmp_path: Path
) -> None:
+31 -2
View File
@@ -140,6 +140,7 @@ class TestRestartCommand:
loop.consolidator.estimate_session_prompt_tokens = MagicMock(
return_value=(20500, "tiktoken")
)
loop.subagents.get_running_count_by_session.return_value = 0
msg = InboundMessage(channel="telegram", sender_id="u1", chat_id="c1", content="/status")
@@ -148,11 +149,36 @@ class TestRestartCommand:
assert response is not None
assert "Model: test-model" in response.content
assert "Tokens: 0 in / 0 out" in response.content
assert "Context: 20k/64k (31%)" in response.content
assert "Context: 20k/65k (31% of input budget)" in response.content
assert "Session: 3 messages" in response.content
assert "Uptime: 2m 5s" in response.content
assert "Tasks: 0 active" in response.content
assert response.metadata == {"render_as": "text"}
@pytest.mark.asyncio
async def test_status_counts_running_dispatch_and_subagent_tasks(self):
loop, _bus = _make_loop()
session = MagicMock()
session.get_history.return_value = [{"role": "user"}]
loop.sessions.get_or_create.return_value = session
loop.consolidator.estimate_session_prompt_tokens = MagicMock(
return_value=(1000, "tiktoken")
)
running_task = MagicMock()
running_task.done.return_value = False
finished_task = MagicMock()
finished_task.done.return_value = True
msg = InboundMessage(channel="telegram", sender_id="u1", chat_id="c1", content="/status")
loop._active_tasks[msg.session_key] = [running_task, finished_task]
loop.subagents.get_running_count_by_session.return_value = 2
response = await loop._process_message(msg)
assert response is not None
assert "Tasks: 3 active" in response.content
@pytest.mark.asyncio
async def test_run_agent_loop_resets_usage_when_provider_omits_it(self):
loop, _bus = _make_loop()
@@ -179,6 +205,7 @@ class TestRestartCommand:
loop.consolidator.estimate_session_prompt_tokens = MagicMock(
return_value=(0, "none")
)
loop.subagents.get_running_count_by_session.return_value = 0
response = await loop._process_message(
InboundMessage(channel="telegram", sender_id="u1", chat_id="c1", content="/status")
@@ -186,7 +213,8 @@ class TestRestartCommand:
assert response is not None
assert "Tokens: 1200 in / 34 out" in response.content
assert "Context: 1k/64k (1%)" in response.content
assert "Context: 1k/65k (1% of input budget)" in response.content
assert "Tasks: 0 active" in response.content
@pytest.mark.asyncio
async def test_process_direct_preserves_render_metadata(self):
@@ -195,6 +223,7 @@ class TestRestartCommand:
session.get_history.return_value = []
loop.sessions.get_or_create.return_value = session
loop.subagents.get_running_count.return_value = 0
loop.subagents.get_running_count_by_session.return_value = 0
response = await loop.process_direct("/status", session_key="cli:test")
+65
View File
@@ -140,6 +140,71 @@ def test_onboard_refresh_backfills_missing_channel_fields(tmp_path, monkeypatch)
assert saved["channels"]["qq"]["msgFormat"] == "plain"
def test_load_config_migrates_legacy_my_tool_keys(tmp_path) -> None:
config_path = tmp_path / "config.json"
config_path.write_text(
json.dumps(
{
"tools": {
"myEnabled": False,
"mySet": True,
}
}
),
encoding="utf-8",
)
config = load_config(config_path)
assert config.tools.my.enable is False
assert config.tools.my.allow_set is True
def test_save_config_rewrites_legacy_my_tool_keys(tmp_path) -> None:
config_path = tmp_path / "config.json"
config_path.write_text(
json.dumps(
{
"tools": {
"myEnabled": False,
"mySet": True,
}
}
),
encoding="utf-8",
)
config = load_config(config_path)
save_config(config, config_path)
saved = json.loads(config_path.read_text(encoding="utf-8"))
tools = saved["tools"]
assert "myEnabled" not in tools
assert "mySet" not in tools
assert tools["my"] == {"enable": False, "allowSet": True}
def test_new_my_tool_keys_take_precedence_over_legacy(tmp_path) -> None:
config_path = tmp_path / "config.json"
config_path.write_text(
json.dumps(
{
"tools": {
"myEnabled": False,
"mySet": False,
"my": {"enable": True, "allowSet": True},
}
}
),
encoding="utf-8",
)
config = load_config(config_path)
assert config.tools.my.enable is True
assert config.tools.my.allow_set is True
def test_load_config_resets_ssrf_whitelist_when_next_config_is_empty(tmp_path) -> None:
whitelisted = tmp_path / "whitelisted.json"
whitelisted.write_text(
+408
View File
@@ -1,5 +1,6 @@
import asyncio
import json
import time
import pytest
@@ -114,6 +115,41 @@ async def test_run_history_persisted_to_disk(tmp_path) -> None:
assert loaded.state.run_history[0].status == "ok"
@pytest.mark.asyncio
async def test_run_job_disabled_does_not_flip_running_state(tmp_path) -> None:
store_path = tmp_path / "cron" / "jobs.json"
service = CronService(store_path, on_job=lambda _: asyncio.sleep(0))
job = service.add_job(
name="disabled",
schedule=CronSchedule(kind="every", every_ms=60_000),
message="hello",
)
service.enable_job(job.id, enabled=False)
result = await service.run_job(job.id)
assert result is False
assert service._running is False
@pytest.mark.asyncio
async def test_run_job_preserves_running_service_state(tmp_path) -> None:
store_path = tmp_path / "cron" / "jobs.json"
service = CronService(store_path, on_job=lambda _: asyncio.sleep(0))
service._running = True
job = service.add_job(
name="manual",
schedule=CronSchedule(kind="every", every_ms=60_000),
message="hello",
)
result = await service.run_job(job.id, force=True)
assert result is True
assert service._running is True
service.stop()
@pytest.mark.asyncio
async def test_running_service_honors_external_disable(tmp_path) -> None:
store_path = tmp_path / "cron" / "jobs.json"
@@ -156,3 +192,375 @@ def test_remove_job_refuses_system_jobs(tmp_path) -> None:
assert result == "protected"
assert service.get_job("dream") is not None
@pytest.mark.asyncio
async def test_start_server_not_jobs(tmp_path):
store_path = tmp_path / "cron" / "jobs.json"
called = []
async def on_job(job):
called.append(job.name)
service = CronService(store_path, on_job=on_job, max_sleep_ms=1000)
await service.start()
assert len(service.list_jobs()) == 0
service2 = CronService(tmp_path / "cron" / "jobs.json")
service2.add_job(
name="hist",
schedule=CronSchedule(kind="every", every_ms=500),
message="hello",
)
assert len(service.list_jobs()) == 1
await asyncio.sleep(2)
assert len(called) != 0
service.stop()
@pytest.mark.asyncio
async def test_subsecond_job_not_delayed_to_one_second(tmp_path):
store_path = tmp_path / "cron" / "jobs.json"
called = []
async def on_job(job):
called.append(job.name)
service = CronService(store_path, on_job=on_job, max_sleep_ms=5000)
service.add_job(
name="fast",
schedule=CronSchedule(kind="every", every_ms=100),
message="hello",
)
await service.start()
try:
await asyncio.sleep(0.35)
assert called
finally:
service.stop()
@pytest.mark.asyncio
async def test_running_service_picks_up_external_add(tmp_path):
"""A running service should detect and execute a job added by another instance."""
store_path = tmp_path / "cron" / "jobs.json"
called: list[str] = []
async def on_job(job):
called.append(job.name)
service = CronService(store_path, on_job=on_job)
service.add_job(
name="heartbeat",
schedule=CronSchedule(kind="every", every_ms=150),
message="tick",
)
await service.start()
try:
await asyncio.sleep(0.05)
external = CronService(store_path)
external.add_job(
name="external",
schedule=CronSchedule(kind="every", every_ms=150),
message="ping",
)
await asyncio.sleep(2)
assert "external" in called
finally:
service.stop()
@pytest.mark.asyncio
async def test_add_job_during_jobs_exec(tmp_path):
store_path = tmp_path / "cron" / "jobs.json"
run_once = True
async def on_job(job):
nonlocal run_once
if run_once:
service2 = CronService(store_path, on_job=lambda x: asyncio.sleep(0))
service2.add_job(
name="test",
schedule=CronSchedule(kind="every", every_ms=150),
message="tick",
)
run_once = False
service = CronService(store_path, on_job=on_job)
service.add_job(
name="heartbeat",
schedule=CronSchedule(kind="every", every_ms=150),
message="tick",
)
assert len(service.list_jobs()) == 1
await service.start()
try:
await asyncio.sleep(3)
jobs = service.list_jobs()
assert len(jobs) == 2
assert "test" in [j.name for j in jobs]
finally:
service.stop()
@pytest.mark.asyncio
async def test_external_update_preserves_run_history_records(tmp_path):
store_path = tmp_path / "cron" / "jobs.json"
service = CronService(store_path, on_job=lambda _: asyncio.sleep(0))
job = service.add_job(
name="history",
schedule=CronSchedule(kind="every", every_ms=60_000),
message="hello",
)
await service.run_job(job.id, force=True)
external = CronService(store_path)
updated = external.enable_job(job.id, enabled=False)
assert updated is not None
fresh = CronService(store_path)
loaded = fresh.get_job(job.id)
assert loaded is not None
assert loaded.state.run_history
assert loaded.state.run_history[0].status == "ok"
fresh._running = True
fresh._save_store()
# ── timer race regression tests ──
@pytest.mark.asyncio
async def test_timer_execution_is_not_rolled_back_by_list_jobs_reload(tmp_path):
"""list_jobs() during _on_timer should not replace the active store and re-run the same due job."""
store_path = tmp_path / "cron" / "jobs.json"
calls: list[str] = []
async def on_job(job):
calls.append(job.id)
# Simulate frontend polling list_jobs while the timer callback is mid-execution.
service.list_jobs(include_disabled=True)
await asyncio.sleep(0)
service = CronService(store_path, on_job=on_job)
service._running = True
service._load_store()
service._arm_timer = lambda: None
job = service.add_job(
name="race",
schedule=CronSchedule(kind="every", every_ms=60_000),
message="hello",
)
job.state.next_run_at_ms = max(1, int(time.time() * 1000) - 1_000)
service._save_store()
await service._on_timer()
await service._on_timer()
assert calls == [job.id]
loaded = service.get_job(job.id)
assert loaded is not None
assert loaded.state.last_run_at_ms is not None
assert loaded.state.next_run_at_ms is not None
assert loaded.state.next_run_at_ms > loaded.state.last_run_at_ms
# ── update_job tests ──
def test_update_job_changes_name(tmp_path) -> None:
service = CronService(tmp_path / "cron" / "jobs.json")
job = service.add_job(
name="old name",
schedule=CronSchedule(kind="every", every_ms=60_000),
message="hello",
)
result = service.update_job(job.id, name="new name")
assert isinstance(result, CronJob)
assert result.name == "new name"
assert result.payload.message == "hello"
def test_update_job_changes_schedule(tmp_path) -> None:
service = CronService(tmp_path / "cron" / "jobs.json")
job = service.add_job(
name="sched",
schedule=CronSchedule(kind="every", every_ms=60_000),
message="hello",
)
old_next = job.state.next_run_at_ms
new_sched = CronSchedule(kind="every", every_ms=120_000)
result = service.update_job(job.id, schedule=new_sched)
assert isinstance(result, CronJob)
assert result.schedule.every_ms == 120_000
assert result.state.next_run_at_ms != old_next
def test_update_job_changes_message(tmp_path) -> None:
service = CronService(tmp_path / "cron" / "jobs.json")
job = service.add_job(
name="msg",
schedule=CronSchedule(kind="every", every_ms=60_000),
message="old message",
)
result = service.update_job(job.id, message="new message")
assert isinstance(result, CronJob)
assert result.payload.message == "new message"
def test_update_job_changes_cron_expression(tmp_path) -> None:
service = CronService(tmp_path / "cron" / "jobs.json")
job = service.add_job(
name="cron-job",
schedule=CronSchedule(kind="cron", expr="0 9 * * *", tz="UTC"),
message="hello",
)
result = service.update_job(
job.id,
schedule=CronSchedule(kind="cron", expr="0 18 * * *", tz="UTC"),
)
assert isinstance(result, CronJob)
assert result.schedule.expr == "0 18 * * *"
assert result.state.next_run_at_ms is not None
def test_update_job_not_found(tmp_path) -> None:
service = CronService(tmp_path / "cron" / "jobs.json")
result = service.update_job("nonexistent", name="x")
assert result == "not_found"
def test_update_job_rejects_system_job(tmp_path) -> None:
service = CronService(tmp_path / "cron" / "jobs.json")
service.register_system_job(CronJob(
id="dream",
name="dream",
schedule=CronSchedule(kind="cron", expr="0 */2 * * *", tz="UTC"),
payload=CronPayload(kind="system_event"),
))
result = service.update_job("dream", name="hacked")
assert result == "protected"
assert service.get_job("dream").name == "dream"
def test_update_job_validates_schedule(tmp_path) -> None:
service = CronService(tmp_path / "cron" / "jobs.json")
job = service.add_job(
name="validate",
schedule=CronSchedule(kind="every", every_ms=60_000),
message="hello",
)
with pytest.raises(ValueError, match="unknown timezone"):
service.update_job(
job.id,
schedule=CronSchedule(kind="cron", expr="0 9 * * *", tz="Bad/Zone"),
)
@pytest.mark.asyncio
async def test_update_job_preserves_run_history(tmp_path) -> None:
import asyncio
store_path = tmp_path / "cron" / "jobs.json"
service = CronService(store_path, on_job=lambda _: asyncio.sleep(0))
job = service.add_job(
name="hist",
schedule=CronSchedule(kind="every", every_ms=60_000),
message="hello",
)
await service.run_job(job.id)
result = service.update_job(job.id, name="renamed")
assert isinstance(result, CronJob)
assert len(result.state.run_history) == 1
assert result.state.run_history[0].status == "ok"
def test_update_job_offline_writes_action(tmp_path) -> None:
service = CronService(tmp_path / "cron" / "jobs.json")
job = service.add_job(
name="offline",
schedule=CronSchedule(kind="every", every_ms=60_000),
message="hello",
)
service.update_job(job.id, name="updated-offline")
action_path = tmp_path / "cron" / "action.jsonl"
assert action_path.exists()
lines = [l for l in action_path.read_text().strip().split("\n") if l]
last = json.loads(lines[-1])
assert last["action"] == "update"
assert last["params"]["name"] == "updated-offline"
def test_update_job_sentinel_channel_and_to(tmp_path) -> None:
"""Passing None clears channel/to; omitting leaves them unchanged."""
service = CronService(tmp_path / "cron" / "jobs.json")
job = service.add_job(
name="sentinel",
schedule=CronSchedule(kind="every", every_ms=60_000),
message="hello",
channel="telegram",
to="user123",
)
assert job.payload.channel == "telegram"
assert job.payload.to == "user123"
result = service.update_job(job.id, name="renamed")
assert isinstance(result, CronJob)
assert result.payload.channel == "telegram"
assert result.payload.to == "user123"
result = service.update_job(job.id, channel=None, to=None)
assert isinstance(result, CronJob)
assert result.payload.channel is None
assert result.payload.to is None
@pytest.mark.asyncio
async def test_list_jobs_during_on_job_does_not_cause_stale_reload(tmp_path) -> None:
"""Regression: if the bot calls list_jobs (which reloads from disk) during
on_job execution, the in-memory next_run_at_ms update must not be lost.
Previously this caused an infinite re-trigger loop."""
store_path = tmp_path / "cron" / "jobs.json"
execution_count = 0
async def on_job_that_lists(job):
nonlocal execution_count
execution_count += 1
# Simulate the bot calling cron(action=list) mid-execution
service.list_jobs()
service = CronService(store_path, on_job=on_job_that_lists, max_sleep_ms=100)
await service.start()
# Add two jobs scheduled in the past so they're immediately due
now_ms = int(time.time() * 1000)
for name in ("job-a", "job-b"):
service.add_job(
name=name,
schedule=CronSchedule(kind="every", every_ms=3_600_000),
message="test",
)
# Force next_run to the past so _on_timer picks them up
for job in service._store.jobs:
job.state.next_run_at_ms = now_ms - 1000
service._save_store()
service._arm_timer()
# Let the timer fire once
await asyncio.sleep(0.3)
service.stop()
# Each job should have run exactly once, not looped
assert execution_count == 2
# Verify next_run_at_ms was persisted correctly (in the future)
raw = json.loads(store_path.read_text())
for j in raw["jobs"]:
next_run = j["state"]["nextRunAtMs"]
assert next_run is not None
assert next_run > now_ms, f"Job '{j['name']}' next_run should be in the future"
+9 -3
View File
@@ -2,9 +2,12 @@
from datetime import datetime, timezone
import pytest
from nanobot.agent.tools.cron import CronTool
from nanobot.cron.service import CronService
from nanobot.cron.types import CronJob, CronJobState, CronPayload, CronSchedule
from tests.test_openai_api import pytest_plugins
def _make_tool(tmp_path) -> CronTool:
@@ -215,8 +218,10 @@ def test_list_at_job_shows_iso_timestamp(tmp_path) -> None:
assert "Asia/Shanghai" in result
def test_list_shows_last_run_state(tmp_path) -> None:
@pytest.mark.asyncio
async def test_list_shows_last_run_state(tmp_path) -> None:
tool = _make_tool(tmp_path)
tool._cron._running = True
job = tool._cron.add_job(
name="Stateful job",
schedule=CronSchedule(kind="cron", expr="0 9 * * *", tz="UTC"),
@@ -232,9 +237,10 @@ def test_list_shows_last_run_state(tmp_path) -> None:
assert "ok" in result
assert "(UTC)" in result
def test_list_shows_error_message(tmp_path) -> None:
@pytest.mark.asyncio
async def test_list_shows_error_message(tmp_path) -> None:
tool = _make_tool(tmp_path)
tool._cron._running = True
job = tool._cron.add_job(
name="Failed job",
schedule=CronSchedule(kind="cron", expr="0 9 * * *", tz="UTC"),
@@ -0,0 +1,65 @@
"""Tests for Anthropic provider thinking / reasoning_effort modes."""
from __future__ import annotations
from unittest.mock import patch
from nanobot.providers.anthropic_provider import AnthropicProvider
def _make_provider(model: str = "claude-sonnet-4-6") -> AnthropicProvider:
with patch("anthropic.AsyncAnthropic"):
return AnthropicProvider(api_key="sk-test", default_model=model)
def _build(provider: AnthropicProvider, reasoning_effort: str | None, **overrides):
defaults = dict(
messages=[{"role": "user", "content": "hello"}],
tools=None,
model=None,
max_tokens=4096,
temperature=0.7,
reasoning_effort=reasoning_effort,
tool_choice=None,
supports_caching=False,
)
defaults.update(overrides)
return provider._build_kwargs(**defaults)
def test_adaptive_sets_type_adaptive() -> None:
kw = _build(_make_provider(), "adaptive")
assert kw["thinking"] == {"type": "adaptive"}
def test_adaptive_forces_temperature_one() -> None:
kw = _build(_make_provider(), "adaptive")
assert kw["temperature"] == 1.0
def test_adaptive_does_not_inflate_max_tokens() -> None:
kw = _build(_make_provider(), "adaptive", max_tokens=2048)
assert kw["max_tokens"] == 2048
def test_adaptive_no_budget_tokens() -> None:
kw = _build(_make_provider(), "adaptive")
assert "budget_tokens" not in kw["thinking"]
def test_high_uses_enabled_with_budget() -> None:
kw = _build(_make_provider(), "high", max_tokens=4096)
assert kw["thinking"]["type"] == "enabled"
assert kw["thinking"]["budget_tokens"] == max(8192, 4096)
assert kw["max_tokens"] >= kw["thinking"]["budget_tokens"] + 4096
def test_low_uses_small_budget() -> None:
kw = _build(_make_provider(), "low")
assert kw["thinking"] == {"type": "enabled", "budget_tokens": 1024}
def test_none_does_not_enable_thinking() -> None:
kw = _build(_make_provider(), None)
assert "thinking" not in kw
assert kw["temperature"] == 0.7
+18
View File
@@ -4,6 +4,7 @@ from types import SimpleNamespace
from unittest.mock import patch
from nanobot.providers.openai_compat_provider import OpenAICompatProvider
from nanobot.providers.registry import find_by_name
def test_custom_provider_parse_handles_empty_choices() -> None:
@@ -53,3 +54,20 @@ def test_custom_provider_parse_chunks_accepts_plain_text_chunks() -> None:
assert result.finish_reason == "stop"
assert result.content == "hello world"
def test_local_provider_502_error_includes_reachability_hint() -> None:
spec = find_by_name("ollama")
with patch("nanobot.providers.openai_compat_provider.AsyncOpenAI"):
provider = OpenAICompatProvider(api_base="http://localhost:11434/v1", spec=spec)
result = provider._handle_error(
Exception("Error code: 502"),
spec=spec,
api_base="http://localhost:11434/v1",
)
assert result.finish_reason == "error"
assert "local model endpoint" in result.content
assert "http://localhost:11434/v1" in result.content
assert "proxy/tunnel" in result.content
@@ -0,0 +1,197 @@
"""Tests for LLMProvider._enforce_role_alternation."""
from nanobot.providers.base import LLMProvider
class TestEnforceRoleAlternation:
"""Verify trailing-assistant removal and consecutive same-role merging."""
def test_empty_messages(self):
assert LLMProvider._enforce_role_alternation([]) == []
def test_no_change_needed(self):
msgs = [
{"role": "system", "content": "You are helpful."},
{"role": "user", "content": "Hi"},
{"role": "assistant", "content": "Hello!"},
{"role": "user", "content": "Bye"},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert len(result) == 4
assert result[-1]["role"] == "user"
def test_trailing_assistant_removed(self):
msgs = [
{"role": "user", "content": "Hi"},
{"role": "assistant", "content": "Hello!"},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert len(result) == 1
assert result[0]["role"] == "user"
def test_multiple_trailing_assistants_removed(self):
msgs = [
{"role": "user", "content": "Hi"},
{"role": "assistant", "content": "A"},
{"role": "assistant", "content": "B"},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert len(result) == 1
assert result[0]["role"] == "user"
def test_consecutive_user_messages_merged(self):
msgs = [
{"role": "user", "content": "Hello"},
{"role": "user", "content": "How are you?"},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert len(result) == 1
assert "Hello" in result[0]["content"]
assert "How are you?" in result[0]["content"]
def test_consecutive_assistant_messages_merged(self):
msgs = [
{"role": "user", "content": "Hi"},
{"role": "assistant", "content": "Hello!"},
{"role": "assistant", "content": "How can I help?"},
{"role": "user", "content": "Thanks"},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert len(result) == 3
assert "Hello!" in result[1]["content"]
assert "How can I help?" in result[1]["content"]
def test_system_messages_not_merged(self):
msgs = [
{"role": "system", "content": "System A"},
{"role": "system", "content": "System B"},
{"role": "user", "content": "Hi"},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert len(result) == 3
assert result[0]["content"] == "System A"
assert result[1]["content"] == "System B"
def test_tool_messages_not_merged(self):
msgs = [
{"role": "user", "content": "Hi"},
{"role": "assistant", "content": None, "tool_calls": [{"id": "1"}]},
{"role": "tool", "content": "result1", "tool_call_id": "1"},
{"role": "tool", "content": "result2", "tool_call_id": "2"},
{"role": "user", "content": "Next"},
]
result = LLMProvider._enforce_role_alternation(msgs)
tool_msgs = [m for m in result if m["role"] == "tool"]
assert len(tool_msgs) == 2
def test_consecutive_assistant_keeps_later_tool_call_message(self):
msgs = [
{"role": "user", "content": "Hi"},
{"role": "assistant", "content": "Previous reply"},
{"role": "assistant", "content": None, "tool_calls": [{"id": "1"}]},
{"role": "tool", "content": "result1", "tool_call_id": "1"},
{"role": "user", "content": "Next"},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert result[1]["role"] == "assistant"
assert result[1]["tool_calls"] == [{"id": "1"}]
assert result[1]["content"] is None
assert result[2]["role"] == "tool"
def test_consecutive_assistant_does_not_overwrite_existing_tool_call_message(self):
msgs = [
{"role": "user", "content": "Hi"},
{"role": "assistant", "content": None, "tool_calls": [{"id": "1"}]},
{"role": "assistant", "content": "Later plain assistant"},
{"role": "tool", "content": "result1", "tool_call_id": "1"},
{"role": "user", "content": "Next"},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert result[1]["role"] == "assistant"
assert result[1]["tool_calls"] == [{"id": "1"}]
assert result[1]["content"] is None
assert result[2]["role"] == "tool"
def test_non_string_content_uses_latest(self):
msgs = [
{"role": "user", "content": [{"type": "text", "text": "A"}]},
{"role": "user", "content": "B"},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert len(result) == 1
assert result[0]["content"] == "B"
def test_original_messages_not_mutated(self):
msgs = [
{"role": "user", "content": "Hello"},
{"role": "user", "content": "World"},
]
original_first = dict(msgs[0])
LLMProvider._enforce_role_alternation(msgs)
assert msgs[0] == original_first
assert len(msgs) == 2
def test_trailing_assistant_recovered_as_user_when_only_system_remains(self):
"""Subagent result injected as assistant message must not be silently dropped.
When build_messages(current_role="assistant") produces [system, assistant],
_enforce_role_alternation would drop the assistant, leaving only [system].
Most providers (e.g. Zhipu/GLM error 1214) reject such requests.
The trailing assistant should be recovered as a user message instead.
"""
msgs = [
{"role": "system", "content": "You are helpful."},
{"role": "assistant", "content": "Subagent completed successfully."},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert len(result) == 2
assert result[0]["role"] == "system"
assert result[1]["role"] == "user"
assert "Subagent completed successfully." in result[1]["content"]
def test_trailing_assistant_not_recovered_when_user_message_present(self):
"""Recovery should NOT happen when a user message already exists."""
msgs = [
{"role": "system", "content": "You are helpful."},
{"role": "user", "content": "Hi"},
{"role": "assistant", "content": "Hello!"},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert len(result) == 2
assert result[-1]["role"] == "user"
def test_trailing_assistant_recovered_with_tool_result_preceding(self):
"""When only [system, tool, assistant] remains, recovery is not needed
because tool messages are valid non-system content."""
msgs = [
{"role": "system", "content": "You are helpful."},
{"role": "tool", "content": "result", "tool_call_id": "1"},
{"role": "assistant", "content": "Done."},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert len(result) == 2
assert result[-1]["role"] == "tool"
def test_only_assistant_messages(self):
msgs = [
{"role": "assistant", "content": "A"},
{"role": "assistant", "content": "B"},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert result == []
def test_realistic_conversation(self):
msgs = [
{"role": "system", "content": "You are helpful."},
{"role": "user", "content": "What is 2+2?"},
{"role": "assistant", "content": "4"},
{"role": "user", "content": "And 3+3?"},
{"role": "user", "content": "(please be quick)"},
{"role": "assistant", "content": "6"},
]
result = LLMProvider._enforce_role_alternation(msgs)
assert len(result) == 4
assert result[2]["role"] == "assistant"
assert result[3]["role"] == "user"
assert "And 3+3?" in result[3]["content"]
assert "(please be quick)" in result[3]["content"]

Some files were not shown because too many files have changed in this diff Show More