Compare commits

..
Author SHA1 Message Date
chengyongru f1250f0d16 feat(dream): add skill ownership guard with dream_managed frontmatter
- Dream-created skills get  in SKILL.md frontmatter
-  tags each skill as [dream]/[user]/[builtin]
- New config  (default False): when False Dream
  can only modify [dream] skills; when True [user] skills are also editable
- [builtin] skills are never editable
- dream.md prompt uses Jinja conditional to render edit policy
- WebUI types and test fixtures updated with new config field
2026-05-26 00:50:45 +08:00
chengyongru 376d82fec7 fix(agent): prevent runner from exiting while sustained goal is active (cherry-pick) 2026-05-26 00:43:40 +08:00
chengyongru 8dd35c373a feat(dream): single-phase consolidation (cherry-pick from feat/single-phase-dream) 2026-05-26 00:38:04 +08:00
204 changed files with 5660 additions and 19798 deletions
+4
View File
@@ -31,6 +31,10 @@ Tool descriptions, skills, and replayed session history also shape model behavio
Anything written into memory, session history, or prompt inputs can be replayed into future LLM calls. Metadata such as timestamps, local media paths, tool-call echoes, and raw fallback dumps must be bounded and sanitized before they become examples for the model to imitate. Anything written into memory, session history, or prompt inputs can be replayed into future LLM calls. Metadata such as timestamps, local media paths, tool-call echoes, and raw fallback dumps must be bounded and sanitized before they become examples for the model to imitate.
## Heartbeat Virtual Tool Call
The heartbeat service (`heartbeat/service.py`) does not parse free-text LLM output. Instead, it injects a virtual `heartbeat` tool with `action: skip | run` into the conversation. Phase 1 is a structured decision; Phase 2 executes only on `run`. When adding new periodic background checks, follow this virtual-tool-call pattern rather than string matching.
## Skills as Extension Point ## Skills as Extension Point
Built-in skills live in `nanobot/skills/` (markdown + YAML frontmatter format). Agent capabilities that are "know-how" rather than code should be added as skills, not hardcoded into the agent loop. External skills can be published to and installed from ClawHub. Built-in skills live in `nanobot/skills/` (markdown + YAML frontmatter format). Agent capabilities that are "know-how" rather than code should be added as skills, not hardcoded into the agent loop. External skills can be published to and installed from ClawHub.
-2
View File
@@ -6,8 +6,6 @@
.env .env
.web .web
.orion .orion
nanobot-desktop/
desktop/
# Claude / AI assistant artifacts # Claude / AI assistant artifacts
docs/superpowers/ docs/superpowers/
+1 -1
View File
@@ -47,7 +47,7 @@ Messages flow through an async `MessageBus` (`nanobot/bus/queue.py`) that decoup
- **WebUI** (`webui/`): Vite-based React SPA that talks to the gateway over a WebSocket multiplex protocol. The dev server proxies `/api`, `/webui`, `/auth`, and WebSocket traffic to the gateway. - **WebUI** (`webui/`): Vite-based React SPA that talks to the gateway over a WebSocket multiplex protocol. The dev server proxies `/api`, `/webui`, `/auth`, and WebSocket traffic to the gateway.
- **API Server** (`nanobot/api/server.py`): OpenAI-compatible HTTP API (`/v1/chat/completions`, `/v1/models`) for programmatic access. - **API Server** (`nanobot/api/server.py`): OpenAI-compatible HTTP API (`/v1/chat/completions`, `/v1/models`) for programmatic access.
- **Command Router** (`nanobot/command/`): Slash command routing and built-in command handlers. - **Command Router** (`nanobot/command/`): Slash command routing and built-in command handlers.
- **Heartbeat** (`nanobot/templates/HEARTBEAT.md`): Periodic task list checked via `cron` jobs (legacy dedicated service removed). - **Heartbeat** (`nanobot/heartbeat/`): Periodic agent wake-up service for scheduled task checking.
- **Pairing** (`nanobot/pairing/`): DM sender approval store with persistent pairing codes per channel. - **Pairing** (`nanobot/pairing/`): DM sender approval store with persistent pairing codes per channel.
- **Skills** (`nanobot/skills/`): Built-in skill definitions (long-goal, cron, github, image-generation, etc.) loaded into agent context. - **Skills** (`nanobot/skills/`): Built-in skill definitions (long-goal, cron, github, image-generation, etc.) loaded into agent context.
- **Security** (`nanobot/security/`): PTH file guard and other security measures activated at CLI entry. - **Security** (`nanobot/security/`): PTH file guard and other security measures activated at CLI entry.
-2
View File
@@ -12,8 +12,6 @@ software together: with care, clarity, and respect for the next person reading t
## Maintainers ## Maintainers
Maintainers are community stewards who help review, organize, and maintain the project. The list below describes each maintainer's current open-source project responsibilities.
| Maintainer | Focus | | Maintainer | Focus |
|------------|-------| |------------|-------|
| [@re-bin](https://github.com/re-bin) | Project lead, `main` branch | | [@re-bin](https://github.com/re-bin) | Project lead, `main` branch |
+3 -1
View File
@@ -46,15 +46,17 @@ core_agent=$(count_top_level_py_lines "nanobot/agent")
core_bus=$(count_top_level_py_lines "nanobot/bus") core_bus=$(count_top_level_py_lines "nanobot/bus")
core_config=$(count_top_level_py_lines "nanobot/config") core_config=$(count_top_level_py_lines "nanobot/config")
core_cron=$(count_top_level_py_lines "nanobot/cron") core_cron=$(count_top_level_py_lines "nanobot/cron")
core_heartbeat=$(count_top_level_py_lines "nanobot/heartbeat")
core_session=$(count_top_level_py_lines "nanobot/session") core_session=$(count_top_level_py_lines "nanobot/session")
print_row "agent/" "$core_agent" print_row "agent/" "$core_agent"
print_row "bus/" "$core_bus" print_row "bus/" "$core_bus"
print_row "config/" "$core_config" print_row "config/" "$core_config"
print_row "cron/" "$core_cron" print_row "cron/" "$core_cron"
print_row "heartbeat/" "$core_heartbeat"
print_row "session/" "$core_session" print_row "session/" "$core_session"
core_total=$((core_agent + core_bus + core_config + core_cron + core_session)) core_total=$((core_agent + core_bus + core_config + core_cron + core_heartbeat + core_session))
echo "" echo ""
echo "Separate buckets" echo "Separate buckets"
-37
View File
@@ -51,43 +51,6 @@ Connect nanobot to your favorite chat platform. Want to build your own? See the
nanobot gateway nanobot gateway
``` ```
**Webhook mode (optional)**
Telegram uses long polling by default. To receive updates through a webhook, expose
a public HTTPS URL that forwards to nanobot's local listener and set `mode` to
`webhook`:
```json
{
"channels": {
"telegram": {
"enabled": true,
"token": "YOUR_BOT_TOKEN",
"mode": "webhook",
"webhookUrl": "https://example.com/telegram",
"webhookListenHost": "127.0.0.1",
"webhookListenPort": 8081,
"webhookPath": "/telegram",
"webhookSecretToken": "CHANGE_ME_RANDOM_SECRET",
"webhookMaxConnections": 4,
"allowFrom": ["YOUR_USER_ID"]
}
}
}
```
> `webhookSecretToken` is required in webhook mode. Do not expose the local
> webhook listener directly to the public internet without a reverse proxy or
> tunnel in front of it. TLS/Host policy is handled by your proxy; nanobot only
> listens on `webhookListenHost:webhookListenPort` and validates Telegram's
> webhook secret token. `webhookMaxConnections` defaults to `4`; nanobot
> still serializes Telegram updates per conversation before forwarding them to
> the agent.
>
> `webhookUrl` is the public HTTPS URL registered with Telegram.
> `webhookPath` is the local path nanobot listens on. They often use the same
> path, but may differ when a reverse proxy or tunnel rewrites the request path.
</details> </details>
<details> <details>
+1 -3
View File
@@ -1043,7 +1043,6 @@ Global settings that apply to all channels. Configure under the `channels` secti
"channels": { "channels": {
"sendProgress": true, "sendProgress": true,
"sendToolHints": false, "sendToolHints": false,
"extractDocumentText": true,
"sendMaxRetries": 3, "sendMaxRetries": 3,
"transcriptionProvider": "groq", "transcriptionProvider": "groq",
"transcriptionLanguage": null, "transcriptionLanguage": null,
@@ -1057,7 +1056,6 @@ Global settings that apply to all channels. Configure under the `channels` secti
| `sendProgress` | `true` | Stream agent's text progress to the channel | | `sendProgress` | `true` | Stream agent's text progress to the channel |
| `sendToolHints` | `false` | Stream tool-call hints (e.g. `read_file("…")`) | | `sendToolHints` | `false` | Stream tool-call hints (e.g. `read_file("…")`) |
| `showReasoning` | `true` | Allow channels to surface model reasoning/thinking content (DeepSeek-R1 `reasoning_content`, Anthropic `thinking_blocks`, inline `<think>` tags). Reasoning flows as a dedicated stream with `_reasoning_delta` / `_reasoning_end` markers — channels override `send_reasoning_delta` / `send_reasoning_end` to render in-place updates. Even with `true`, channels without those overrides stay no-op silently. Currently surfaced on CLI and WebSocket/WebUI (italic shimmer header, auto-collapses after the stream ends); Telegram / Slack / Discord / Feishu / WeChat / Matrix keep the base no-op until their bubble UI is adapted. Independent of `sendProgress`. | | `showReasoning` | `true` | Allow channels to surface model reasoning/thinking content (DeepSeek-R1 `reasoning_content`, Anthropic `thinking_blocks`, inline `<think>` tags). Reasoning flows as a dedicated stream with `_reasoning_delta` / `_reasoning_end` markers — channels override `send_reasoning_delta` / `send_reasoning_end` to render in-place updates. Even with `true`, channels without those overrides stay no-op silently. Currently surfaced on CLI and WebSocket/WebUI (italic shimmer header, auto-collapses after the stream ends); Telegram / Slack / Discord / Feishu / WeChat / Matrix keep the base no-op until their bubble UI is adapted. Independent of `sendProgress`. |
| `extractDocumentText` | `true` | Extract supported document/text attachments into the model prompt. Set to `false` to keep document content out of the prompt and include attachment path references instead. |
| `sendMaxRetries` | `3` | Max delivery attempts per outbound message, including the initial send (0-10 configured, minimum 1 actual attempt) | | `sendMaxRetries` | `3` | Max delivery attempts per outbound message, including the initial send (0-10 configured, minimum 1 actual attempt) |
| `transcriptionProvider` | `"groq"` | Voice transcription backend: `"groq"` (free tier, default) or `"openai"`. API key and optional `apiBase` are auto-resolved from the matching provider config. Chat-style bases such as `https://api.groq.com/openai/v1` are normalized to the audio transcription endpoint. | | `transcriptionProvider` | `"groq"` | Voice transcription backend: `"groq"` (free tier, default) or `"openai"`. API key and optional `apiBase` are auto-resolved from the matching provider config. Chat-style bases such as `https://api.groq.com/openai/v1` are normalized to the audio transcription endpoint. |
| `transcriptionLanguage` | `null` | Optional ISO-639-1 language hint for audio transcription, e.g. `"en"`, `"ko"`, `"ja"`. | | `transcriptionLanguage` | `null` | Optional ISO-639-1 language hint for audio transcription, e.g. `"en"`, `"ko"`, `"ja"`. |
@@ -1534,7 +1532,7 @@ By default, nanobot uses `UTC` for runtime time context. If you want the agent t
} }
``` ```
This affects runtime time strings shown to the model, such as runtime context. It also becomes the default timezone for cron schedules when a cron expression omits `tz`, and for one-shot `at` times when the ISO datetime has no explicit offset. This affects runtime time strings shown to the model, such as runtime context and heartbeat prompts. It also becomes the default timezone for cron schedules when a cron expression omits `tz`, and for one-shot `at` times when the ISO datetime has no explicit offset.
Common examples: `UTC`, `America/New_York`, `America/Los_Angeles`, `Europe/London`, `Europe/Berlin`, `Asia/Tokyo`, `Asia/Shanghai`, `Asia/Singapore`, `Australia/Sydney`. Common examples: `UTC`, `America/New_York`, `America/Los_Angeles`, `Europe/London`, `Europe/Berlin`, `Asia/Tokyo`, `Asia/Shanghai`, `Asia/Singapore`, `Australia/Sydney`.
+14 -31
View File
@@ -3,6 +3,8 @@
import base64 import base64
import mimetypes import mimetypes
import platform import platform
from contextlib import suppress
from importlib.resources import files as pkg_files
from pathlib import Path from pathlib import Path
from typing import Any, Mapping, Sequence from typing import Any, Mapping, Sequence
@@ -10,13 +12,12 @@ from nanobot.agent.memory import MemoryStore
from nanobot.agent.skills import SkillsLoader from nanobot.agent.skills import SkillsLoader
from nanobot.agent.tools import mcp as mcp_tools from nanobot.agent.tools import mcp as mcp_tools
from nanobot.agent.tools.registry import ToolRegistry from nanobot.agent.tools.registry import ToolRegistry
from nanobot.apps.cli import utils as cli_app_utils
from nanobot.bus.events import InboundMessage from nanobot.bus.events import InboundMessage
from nanobot.apps.cli import utils as cli_app_utils
from nanobot.session.goal_state import goal_state_runtime_lines from nanobot.session.goal_state import goal_state_runtime_lines
from nanobot.utils.helpers import ( from nanobot.utils.helpers import (
current_time_str, current_time_str,
detect_image_mime, detect_image_mime,
load_bundled_template,
truncate_text, truncate_text,
) )
from nanobot.utils.prompt_templates import render_template from nanobot.utils.prompt_templates import render_template
@@ -68,13 +69,11 @@ class ContextBuilder:
skill_names: list[str] | None = None, skill_names: list[str] | None = None,
channel: str | None = None, channel: str | None = None,
session_summary: str | None = None, session_summary: str | None = None,
workspace: Path | None = None,
) -> str: ) -> str:
"""Build the system prompt from identity, bootstrap files, memory, and skills.""" """Build the system prompt from identity, bootstrap files, memory, and skills."""
root = workspace or self.workspace parts = [self._get_identity(channel=channel)]
parts = [self._get_identity(channel=channel, workspace=root)]
bootstrap = self._load_bootstrap_files(root) bootstrap = self._load_bootstrap_files()
if bootstrap: if bootstrap:
parts.append(bootstrap) parts.append(bootstrap)
@@ -108,10 +107,9 @@ class ContextBuilder:
return "\n\n---\n\n".join(parts) return "\n\n---\n\n".join(parts)
def _get_identity(self, channel: str | None = None, workspace: Path | None = None) -> str: def _get_identity(self, channel: str | None = None) -> str:
"""Get the core identity section.""" """Get the core identity section."""
root = workspace or self.workspace workspace_path = str(self.workspace.expanduser().resolve())
workspace_path = str(root.expanduser().resolve())
system = platform.system() system = platform.system()
runtime = f"{'macOS' if system == 'Darwin' else system} {platform.machine()}, Python {platform.python_version()}" runtime = f"{'macOS' if system == 'Darwin' else system} {platform.machine()}, Python {platform.python_version()}"
@@ -155,13 +153,12 @@ class ContextBuilder:
return _to_blocks(left) + _to_blocks(right) return _to_blocks(left) + _to_blocks(right)
def _load_bootstrap_files(self, workspace: Path | None = None) -> str: def _load_bootstrap_files(self) -> str:
"""Load all bootstrap files from workspace.""" """Load all bootstrap files from workspace."""
parts = [] parts = []
root = workspace or self.workspace
for filename in self.BOOTSTRAP_FILES: for filename in self.BOOTSTRAP_FILES:
file_path = root / filename file_path = self.workspace / filename
if file_path.exists(): if file_path.exists():
content = file_path.read_text(encoding="utf-8") content = file_path.read_text(encoding="utf-8")
parts.append(f"## {filename}\n\n{content}") parts.append(f"## {filename}\n\n{content}")
@@ -171,9 +168,10 @@ class ContextBuilder:
@staticmethod @staticmethod
def _is_template_content(content: str, template_path: str) -> bool: def _is_template_content(content: str, template_path: str) -> bool:
"""Check if *content* is identical to the bundled template (user hasn't customized it).""" """Check if *content* is identical to the bundled template (user hasn't customized it)."""
tpl = load_bundled_template(template_path) with suppress(Exception):
if tpl is not None: tpl = pkg_files("nanobot") / "templates" / template_path
return content.strip() == tpl.strip() if tpl.is_file():
return content.strip() == tpl.read_text(encoding="utf-8").strip()
return False return False
def build_messages( def build_messages(
@@ -189,18 +187,11 @@ class ContextBuilder:
session_summary: str | None = None, session_summary: str | None = None,
session_metadata: Mapping[str, Any] | None = None, session_metadata: Mapping[str, Any] | None = None,
current_runtime_lines: Sequence[str] | None = None, current_runtime_lines: Sequence[str] | None = None,
workspace: Path | None = None,
runtime_state: Any | None = None,
inbound_message: Any | None = None,
skip_runtime_lines: bool = False,
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
"""Build the complete message list for an LLM call.""" """Build the complete message list for an LLM call."""
root = workspace or self.workspace
extra = [ extra = [
*goal_state_runtime_lines(session_metadata), *goal_state_runtime_lines(session_metadata),
] ]
if runtime_state is not None and inbound_message is not None:
extra.extend(runtime_lines(runtime_state, inbound_message, root, skip=skip_runtime_lines))
if current_runtime_lines: if current_runtime_lines:
extra.extend(line for line in current_runtime_lines if line) extra.extend(line for line in current_runtime_lines if line)
runtime_ctx = self._build_runtime_context( runtime_ctx = self._build_runtime_context(
@@ -221,15 +212,7 @@ class ContextBuilder:
else: else:
merged = user_content + [{"type": "text", "text": runtime_ctx}] merged = user_content + [{"type": "text", "text": runtime_ctx}]
messages = [ messages = [
{ {"role": "system", "content": self.build_system_prompt(skill_names, channel=channel, session_summary=session_summary)},
"role": "system",
"content": self.build_system_prompt(
skill_names,
channel=channel,
session_summary=session_summary,
workspace=root,
),
},
*history, *history,
] ]
if messages[-1].get("role") == current_role: if messages[-1].get("role") == current_role:
+295 -116
View File
@@ -8,6 +8,7 @@ import os
import time import time
from contextlib import AsyncExitStack, nullcontext, suppress from contextlib import AsyncExitStack, nullcontext, suppress
from dataclasses import dataclass, field from dataclasses import dataclass, field
from datetime import datetime
from enum import Enum, auto from enum import Enum, auto
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING, Any, Awaitable, Callable from typing import TYPE_CHECKING, Any, Awaitable, Callable
@@ -19,11 +20,16 @@ from nanobot.agent import model_presets as preset_helpers
from nanobot.agent.autocompact import AutoCompact from nanobot.agent.autocompact import AutoCompact
from nanobot.agent.context import ContextBuilder from nanobot.agent.context import ContextBuilder
from nanobot.agent.hook import AgentHook, CompositeHook from nanobot.agent.hook import AgentHook, CompositeHook
from nanobot.agent.memory import Consolidator, Dream from nanobot.agent.memory import (
_STALE_THRESHOLD_DAYS,
Consolidator,
Dream,
_estimate_tokens,
_strip_skip_lines,
)
from nanobot.agent.progress_hook import AgentProgressHook from nanobot.agent.progress_hook import AgentProgressHook
from nanobot.agent.runner import _MAX_INJECTIONS_PER_TURN, AgentRunner, AgentRunSpec from nanobot.agent.runner import _MAX_INJECTIONS_PER_TURN, AgentRunner, AgentRunSpec
from nanobot.agent.subagent import SubagentManager from nanobot.agent.subagent import SubagentManager
from nanobot.agent.tools.context import RequestContext, bind_request_context, reset_request_context
from nanobot.agent.tools.file_state import FileStateStore, bind_file_states, reset_file_states from nanobot.agent.tools.file_state import FileStateStore, bind_file_states, reset_file_states
from nanobot.agent.tools.message import MessageTool from nanobot.agent.tools.message import MessageTool
from nanobot.agent.tools.registry import ToolRegistry from nanobot.agent.tools.registry import ToolRegistry
@@ -34,12 +40,8 @@ from nanobot.command import CommandContext, CommandRouter, register_builtin_comm
from nanobot.config.schema import AgentDefaults, ModelPresetConfig from nanobot.config.schema import AgentDefaults, ModelPresetConfig
from nanobot.providers.base import LLMProvider from nanobot.providers.base import LLMProvider
from nanobot.providers.factory import ProviderSnapshot from nanobot.providers.factory import ProviderSnapshot
from nanobot.security.workspace_access import (
WorkspaceScopeResolver,
bind_workspace_scope,
reset_workspace_scope,
)
from nanobot.session.goal_state import ( from nanobot.session.goal_state import (
GOAL_STATE_KEY,
goal_state_runtime_lines, goal_state_runtime_lines,
runner_wall_llm_timeout_s, runner_wall_llm_timeout_s,
sustained_goal_active, sustained_goal_active,
@@ -50,11 +52,12 @@ from nanobot.session.webui_turns import (
build_bus_progress_callback, build_bus_progress_callback,
mark_webui_session, mark_webui_session,
) )
from nanobot.utils.document import extract_documents, reference_non_image_attachments from nanobot.utils.document import extract_documents
from nanobot.utils.helpers import image_placeholder_text from nanobot.utils.helpers import image_placeholder_text
from nanobot.utils.helpers import truncate_text as truncate_text_fn from nanobot.utils.helpers import truncate_text as truncate_text_fn
from nanobot.utils.image_generation_intent import image_generation_prompt from nanobot.utils.image_generation_intent import image_generation_prompt
from nanobot.utils.llm_runtime import LLMRuntime from nanobot.utils.llm_runtime import LLMRuntime
from nanobot.utils.prompt_templates import _TEMPLATES_ROOT, render_template
from nanobot.utils.runtime import ( from nanobot.utils.runtime import (
EMPTY_FINAL_RESPONSE_MESSAGE, EMPTY_FINAL_RESPONSE_MESSAGE,
SUSTAINED_GOAL_CONTINUE_PROMPT, SUSTAINED_GOAL_CONTINUE_PROMPT,
@@ -120,6 +123,7 @@ class TurnContext:
pending_queue: asyncio.Queue | None = None pending_queue: asyncio.Queue | None = None
pending_summary: str | None = None pending_summary: str | None = None
turn_wall_started_at: float = field(default_factory=time.time) turn_wall_started_at: float = field(default_factory=time.time)
turn_latency_ms: int | None = None turn_latency_ms: int | None = None
@@ -201,6 +205,7 @@ class AgentLoop:
model_preset: str | None = None, model_preset: str | None = None,
preset_snapshot_loader: preset_helpers.PresetSnapshotLoader | None = None, preset_snapshot_loader: preset_helpers.PresetSnapshotLoader | None = None,
runtime_model_publisher: Callable[[str, str | None], None] | None = None, runtime_model_publisher: Callable[[str, str | None], None] | None = None,
dream_model_override: str | None = None,
): ):
from nanobot.config.schema import ToolsConfig from nanobot.config.schema import ToolsConfig
@@ -213,6 +218,7 @@ class AgentLoop:
self._preset_snapshot_loader = preset_snapshot_loader self._preset_snapshot_loader = preset_snapshot_loader
self._runtime_model_publisher = runtime_model_publisher self._runtime_model_publisher = runtime_model_publisher
self._provider_signature = provider_signature self._provider_signature = provider_signature
self._dream_model_override = dream_model_override
self._default_selection_signature = preset_helpers.default_selection_signature(provider_signature) self._default_selection_signature = preset_helpers.default_selection_signature(provider_signature)
self.workspace = workspace self.workspace = workspace
self.model = model or provider.get_default_model() self.model = model or provider.get_default_model()
@@ -246,10 +252,6 @@ class AgentLoop:
self._image_generation_provider_configs["openrouter"] = image_generation_provider_config self._image_generation_provider_configs["openrouter"] = image_generation_provider_config
self.cron_service = cron_service self.cron_service = cron_service
self.restrict_to_workspace = restrict_to_workspace self.restrict_to_workspace = restrict_to_workspace
self.workspace_scopes = WorkspaceScopeResolver(
default_workspace=workspace,
default_restrict_to_workspace=restrict_to_workspace,
)
self._start_time = time.time() self._start_time = time.time()
self._last_usage: dict[str, int] = {} self._last_usage: dict[str, int] = {}
self._pending_turn_latency_ms: dict[str, int] = {} self._pending_turn_latency_ms: dict[str, int] = {}
@@ -324,6 +326,7 @@ class AgentLoop:
self._active_preset: str | None = None self._active_preset: str | None = None
if model_preset: if model_preset:
self.set_model_preset(model_preset, publish_update=False) self.set_model_preset(model_preset, publish_update=False)
self._configure_dream()
self._register_default_tools() self._register_default_tools()
self._runtime_vars: dict[str, Any] = {} self._runtime_vars: dict[str, Any] = {}
self._current_iteration: int = 0 self._current_iteration: int = 0
@@ -383,6 +386,7 @@ class AgentLoop:
model_preset=defaults.model_preset, model_preset=defaults.model_preset,
provider_snapshot_loader=provider_snapshot_loader, provider_snapshot_loader=provider_snapshot_loader,
preset_snapshot_loader=preset_snapshot_loader, preset_snapshot_loader=preset_snapshot_loader,
dream_model_override=config.agents.defaults.dream.model_override,
**extra, **extra,
) )
@@ -408,7 +412,7 @@ class AgentLoop:
self.runner.provider = provider self.runner.provider = provider
self.subagents.set_provider(provider, model) self.subagents.set_provider(provider, model)
self.consolidator.set_provider(provider, model, context_window_tokens) self.consolidator.set_provider(provider, model, context_window_tokens)
self.dream.set_provider(provider, model) self._configure_dream()
self._provider_signature = snapshot.signature self._provider_signature = snapshot.signature
if publish_update and self._runtime_model_publisher is not None: if publish_update and self._runtime_model_publisher is not None:
self._runtime_model_publisher( self._runtime_model_publisher(
@@ -417,6 +421,20 @@ class AgentLoop:
) )
logger.info("Runtime model switched for next turn: {} -> {}", old_model, model) logger.info("Runtime model switched for next turn: {} -> {}", old_model, model)
def _configure_dream(self) -> None:
"""Apply dream.model_override, resolving preset names if needed."""
if not self._dream_model_override:
self.dream.set_provider(self.provider, self.model)
return
if self._dream_model_override in self.model_presets:
snapshot = self._build_model_preset_snapshot(self._dream_model_override)
self.dream.set_provider(snapshot.provider, snapshot.model)
return
# Raw model name fallback — same provider, different model
self.dream.set_provider(self.provider, self._dream_model_override)
def _refresh_provider_snapshot(self) -> None: def _refresh_provider_snapshot(self) -> None:
if self._provider_snapshot_loader is None: if self._provider_snapshot_loader is None:
return return
@@ -479,7 +497,6 @@ class AgentLoop:
provider_snapshot_loader=self._provider_snapshot_loader, provider_snapshot_loader=self._provider_snapshot_loader,
image_generation_provider_configs=self._image_generation_provider_configs, image_generation_provider_configs=self._image_generation_provider_configs,
timezone=self.context.timezone or "UTC", timezone=self.context.timezone or "UTC",
workspace_sandbox=self.workspace_scopes.sandbox_status,
) )
loader = ToolLoader() loader = ToolLoader()
registered = loader.load(ctx, self.tools) registered = loader.load(ctx, self.tools)
@@ -503,7 +520,7 @@ class AgentLoop:
session_key: str | None = None, session_key: str | None = None,
) -> None: ) -> None:
"""Update context for all tools that need routing info.""" """Update context for all tools that need routing info."""
from nanobot.agent.tools.context import ContextAware from nanobot.agent.tools.context import ContextAware, RequestContext
if session_key is not None: if session_key is not None:
effective_key = session_key effective_key = session_key
@@ -585,7 +602,6 @@ class AgentLoop:
pending_summary: str | None, pending_summary: str | None,
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
"""Build the initial message list for the LLM turn.""" """Build the initial message list for the LLM turn."""
scope = self.workspace_scopes.for_message(msg, session.metadata)
return self.context.build_messages( return self.context.build_messages(
history=history, history=history,
current_message=image_generation_prompt(msg.content, msg.metadata), current_message=image_generation_prompt(msg.content, msg.metadata),
@@ -594,10 +610,7 @@ class AgentLoop:
chat_id=self._runtime_chat_id(msg), chat_id=self._runtime_chat_id(msg),
sender_id=msg.sender_id, sender_id=msg.sender_id,
session_summary=pending_summary, session_summary=pending_summary,
session_metadata=session.metadata, session_metadata=session.metadata, current_runtime_lines=agent_context.runtime_lines(self, msg, self.context.workspace),
workspace=scope.project_path,
runtime_state=self,
inbound_message=msg,
) )
async def _dispatch_command_inline( async def _dispatch_command_inline(
@@ -711,7 +724,7 @@ class AgentLoop:
content = pending_msg.content content = pending_msg.content
media = pending_msg.media if pending_msg.media else None media = pending_msg.media if pending_msg.media else None
if media: if media:
content, media = self._prepare_message_media(content, media) content, media = extract_documents(content, media)
media = media or None media = media or None
user_content = self.context._build_user_content(content, media) user_content = self.context._build_user_content(content, media)
return {"role": "user", "content": user_content} return {"role": "user", "content": user_content}
@@ -747,21 +760,7 @@ class AgentLoop:
return items return items
active_session_key = session.key if session else session_key active_session_key = session.key if session else session_key
effective_scope = self.workspace_scopes.for_turn(
channel=channel,
message_metadata=metadata,
session_metadata=session.metadata if session is not None else None,
)
request_ctx = RequestContext(
channel=channel,
chat_id=chat_id,
message_id=message_id,
session_key=active_session_key,
metadata=dict(metadata or {}),
)
file_state_token = bind_file_states(self._file_state_store.for_session(active_session_key)) file_state_token = bind_file_states(self._file_state_store.for_session(active_session_key))
request_token = bind_request_context(request_ctx)
workspace_token = bind_workspace_scope(effective_scope)
# Build continuation message that embeds the active goal objective so # Build continuation message that embeds the active goal objective so
# the LLM can see it even if earlier Runtime Context was truncated. # the LLM can see it even if earlier Runtime Context was truncated.
_goal_lines = goal_state_runtime_lines(session.metadata if session is not None else None) _goal_lines = goal_state_runtime_lines(session.metadata if session is not None else None)
@@ -781,7 +780,7 @@ class AgentLoop:
hook=hook, hook=hook,
error_message="Sorry, I encountered an error calling the AI model.", error_message="Sorry, I encountered an error calling the AI model.",
concurrent_tools=True, concurrent_tools=True,
workspace=effective_scope.project_path, workspace=self.workspace,
session_key=session.key if session else None, session_key=session.key if session else None,
context_window_tokens=self.context_window_tokens, context_window_tokens=self.context_window_tokens,
context_block_limit=self.context_block_limit, context_block_limit=self.context_block_limit,
@@ -802,8 +801,6 @@ class AgentLoop:
goal_continue_message=_goal_continue, goal_continue_message=_goal_continue,
)) ))
finally: finally:
reset_workspace_scope(workspace_token)
reset_request_context(request_token)
reset_file_states(file_state_token) reset_file_states(file_state_token)
self._last_usage = result.usage self._last_usage = result.usage
if result.stop_reason == "max_iterations": if result.stop_reason == "max_iterations":
@@ -842,16 +839,16 @@ class AgentLoop:
logger.warning("Error consuming inbound message: {}, continuing...", e) logger.warning("Error consuming inbound message: {}, continuing...", e)
continue continue
raw = msg.content.strip()
effective_key = self._effective_session_key(msg)
if await agent_context.handle_runtime_control(self, msg, self.tools): if await agent_context.handle_runtime_control(self, msg, self.tools):
continue continue
raw = msg.content.strip()
if self.commands.is_priority(raw): if self.commands.is_priority(raw):
await self._dispatch_command_inline( await self._dispatch_command_inline(
msg, effective_key, raw, msg, msg.session_key, raw,
self.commands.dispatch_priority, self.commands.dispatch_priority,
) )
continue continue
effective_key = self._effective_session_key(msg)
# If this session already has an active pending queue (i.e. a task # If this session already has an active pending queue (i.e. a task
# is processing this session), route the message there for mid-turn # is processing this session), route the message there for mid-turn
# injection instead of creating a competing task. # injection instead of creating a competing task.
@@ -902,13 +899,13 @@ class AgentLoop:
lock = self._session_locks.setdefault(session_key, asyncio.Lock()) lock = self._session_locks.setdefault(session_key, asyncio.Lock())
gate = self._concurrency_gate or nullcontext() gate = self._concurrency_gate or nullcontext()
pending: asyncio.Queue | None = None # Register a pending queue so follow-up messages for this session are
# routed here (mid-turn injection) instead of spawning a new task.
pending = asyncio.Queue(maxsize=20)
self._pending_queues[session_key] = pending
try: try:
async with lock, gate: async with lock, gate:
# Only the task that owns the session lock may publish the
# active mid-turn injection queue for this session.
pending = asyncio.Queue(maxsize=20)
self._pending_queues[session_key] = pending
try: try:
on_stream = on_stream_end = None on_stream = on_stream_end = None
if msg.metadata.get("_wants_stream"): if msg.metadata.get("_wants_stream"):
@@ -992,39 +989,28 @@ class AgentLoop:
channel=msg.channel, chat_id=msg.chat_id, channel=msg.channel, chat_id=msg.chat_id,
content="Sorry, I encountered an error.", content="Sorry, I encountered an error.",
)) ))
finally:
# Drain any messages still in the pending queue and re-publish
# them to the bus so they are processed as fresh inbound messages
# rather than silently lost. Only remove our own queue; a
# later task waiting on the lock must not be able to steal
# cleanup ownership.
queue = None
if self._pending_queues.get(session_key) is pending:
queue = self._pending_queues.pop(session_key, None)
else:
queue = pending
if queue is not None:
leftover = 0
while True:
try:
item = queue.get_nowait()
except asyncio.QueueEmpty:
break
await self.bus.publish_inbound(item)
leftover += 1
if leftover:
logger.info(
"Re-published {} leftover message(s) to bus for session {}",
leftover, session_key,
)
await self._webui_turns.publish_run_status(msg, "idle")
self._pending_turn_latency_ms.pop(session_key, None)
self._webui_turns.discard(session_key)
finally: finally:
if pending is None: # Drain any messages still in the pending queue and re-publish
await self._webui_turns.publish_run_status(msg, "idle") # them to the bus so they are processed as fresh inbound messages
self._pending_turn_latency_ms.pop(session_key, None) # rather than silently lost.
self._webui_turns.discard(session_key) queue = self._pending_queues.pop(session_key, None)
if queue is not None:
leftover = 0
while True:
try:
item = queue.get_nowait()
except asyncio.QueueEmpty:
break
await self.bus.publish_inbound(item)
leftover += 1
if leftover:
logger.info(
"Re-published {} leftover message(s) to bus for session {}",
leftover, session_key,
)
await self._webui_turns.publish_run_status(msg, "idle")
self._pending_turn_latency_ms.pop(session_key, None)
self._webui_turns.discard(session_key)
async def close_mcp(self) -> None: async def close_mcp(self) -> None:
"""Drain pending background archives, then close MCP connections.""" """Drain pending background archives, then close MCP connections."""
@@ -1063,6 +1049,28 @@ class AgentLoop:
msg.chat_id.split(":", 1) if ":" in msg.chat_id else ("cli", msg.chat_id) msg.chat_id.split(":", 1) if ":" in msg.chat_id else ("cli", msg.chat_id)
) )
logger.info("Processing system message from {}", msg.sender_id) logger.info("Processing system message from {}", msg.sender_id)
if msg.sender_id == "dream":
session_key = "system:dream"
session = self.sessions.get_or_create(session_key)
session.metadata["is_dream"] = True
# Capture trigger source on first batch so _dream_finalize_commit
# can notify the user who ran /dream (cron-triggered runs have no trigger).
if "_dream_trigger_channel" not in session.metadata:
trigger_ch = msg.metadata.get("trigger_channel")
trigger_ci = msg.metadata.get("trigger_chat_id")
if trigger_ch and trigger_ci:
session.metadata["_dream_trigger_channel"] = trigger_ch
session.metadata["_dream_trigger_chat_id"] = trigger_ci
if not sustained_goal_active(session.metadata):
session.metadata[GOAL_STATE_KEY] = {
"status": "active",
"objective": "Dream: consolidate unprocessed memory backlog into MEMORY.md, SOUL.md, USER.md",
"started_at": datetime.now().isoformat(),
}
self.sessions.save(session)
await self._process_dream_batch(session, msg)
await self._dream_finalize_commit(session)
return None
key = msg.session_key_override or f"{channel}:{chat_id}" key = msg.session_key_override or f"{channel}:{chat_id}"
session = self.sessions.get_or_create(key) session = self.sessions.get_or_create(key)
if self._restore_runtime_checkpoint(session): if self._restore_runtime_checkpoint(session):
@@ -1093,7 +1101,6 @@ class AgentLoop:
} }
history = session.get_history(**_hist_kwargs) history = session.get_history(**_hist_kwargs)
current_role = "assistant" if is_subagent else "user" current_role = "assistant" if is_subagent else "user"
workspace_scope = self.workspace_scopes.for_message(msg, session.metadata)
messages = self.context.build_messages( messages = self.context.build_messages(
history=history, history=history,
@@ -1103,11 +1110,7 @@ class AgentLoop:
current_role=current_role, current_role=current_role,
sender_id=msg.sender_id, sender_id=msg.sender_id,
session_summary=pending, session_summary=pending,
session_metadata=session.metadata, session_metadata=session.metadata, current_runtime_lines=agent_context.runtime_lines(self, msg, self.context.workspace, skip=is_subagent),
workspace=workspace_scope.project_path,
runtime_state=self,
inbound_message=msg,
skip_runtime_lines=is_subagent,
) )
t_wall = time.time() t_wall = time.time()
final_content, _, all_msgs, stop_reason, _ = await self._run_agent_loop( final_content, _, all_msgs, stop_reason, _ = await self._run_agent_loop(
@@ -1144,6 +1147,205 @@ class AgentLoop:
metadata=outbound_metadata, metadata=outbound_metadata,
) )
async def _process_dream_batch(self, session: Session, msg: InboundMessage) -> None:
"""Process the full Dream backlog in batches within a single invocation."""
from nanobot.agent.skills import BUILTIN_SKILLS_DIR
# System prompt caching with mtime invalidation
template_path = _TEMPLATES_ROOT / "agent" / "dream.md"
cached_prompt = session.metadata.get("_dream_system_prompt")
cached_mtime = session.metadata.get("_dream_system_prompt_mtime")
current_mtime = template_path.stat().st_mtime if template_path.exists() else None
if cached_prompt is None or cached_mtime != current_mtime:
skill_creator_path = BUILTIN_SKILLS_DIR / "skill-creator" / "SKILL.md"
workspace = self.dream.store.workspace
cached_prompt = render_template(
"agent/dream.md",
strip=True,
skill_creator_path=str(skill_creator_path),
soul_path=str(workspace / "SOUL.md"),
user_path=str(workspace / "USER.md"),
memory_path=str(workspace / "memory" / "MEMORY.md"),
stale_threshold_days=_STALE_THRESHOLD_DAYS,
dream_edit_user_skills=self.dream.edit_user_skills,
)
session.metadata["_dream_system_prompt"] = cached_prompt
session.metadata["_dream_system_prompt_mtime"] = current_mtime
while True:
last_cursor = self.dream.store.get_last_dream_cursor()
entries = self.dream.store.read_unprocessed_history(since_cursor=last_cursor)
if not entries:
return
batch = entries[: self.dream.max_batch_size]
logger.info(
"Dream: processing {}/{} entries (cursor {}{})",
len(batch), len(entries), last_cursor, batch[-1]["cursor"],
)
# Build history text — cap each entry and strip [skip] lines
history_text = "\n".join(
f"[{e['timestamp']}] "
f"{truncate_text_fn(_strip_skip_lines(e['content']), self.dream._HISTORY_ENTRY_PREVIEW_MAX_CHARS)}"
for e in batch
)
# Current file contents + per-line age annotations
current_date = datetime.now().strftime("%Y-%m-%d")
annotate = self.dream.annotate_line_ages
raw_memory = self.dream.store.read_memory() or "(empty)"
raw_soul = self.dream.store.read_soul() or "(empty)"
raw_user = self.dream.store.read_user() or "(empty)"
annotated_memory = (
self.dream._annotate_with_ages(raw_memory, "memory/MEMORY.md")
if annotate else raw_memory
)
annotated_soul = (
self.dream._annotate_with_ages(raw_soul, "SOUL.md")
if annotate else raw_soul
)
annotated_user = (
self.dream._annotate_with_ages(raw_user, "USER.md")
if annotate else raw_user
)
current_memory = truncate_text_fn(annotated_memory, self.dream._MEMORY_FILE_MAX_CHARS)
current_soul = truncate_text_fn(annotated_soul, self.dream._SOUL_FILE_MAX_CHARS)
current_user = truncate_text_fn(annotated_user, self.dream._USER_FILE_MAX_CHARS)
file_context = (
f"## Current Date\n{current_date}\n\n"
f"## Current MEMORY.md ({len(current_memory)} chars)\n{current_memory}\n\n"
f"## Current SOUL.md ({len(current_soul)} chars)\n{current_soul}\n\n"
f"## Current USER.md ({len(current_user)} chars)\n{current_user}"
)
existing_skills = self.dream._list_existing_skills(tag_origin=True)
skills_section = ""
if existing_skills:
skills_section = (
"\n\n## Existing Skills\n"
+ "\n".join(f"- {s}" for s in existing_skills)
)
user_prompt = f"## Conversation History\n{history_text}\n\n{file_context}{skills_section}"
logger.info("Dream prompt: {} chars, ~{} tokens", len(user_prompt), _estimate_tokens(user_prompt))
messages: list[dict[str, Any]] = [
{"role": "system", "content": cached_prompt},
{"role": "user", "content": user_prompt},
]
t_start = time.perf_counter()
try:
result = await self.dream._runner.run(AgentRunSpec(
initial_messages=messages,
tools=self.dream._tools,
model=self.dream.model,
max_iterations=self.dream.max_iterations,
max_tool_result_chars=self.dream.max_tool_result_chars,
context_window_tokens=self.context_window_tokens,
fail_on_tool_error=False,
))
elapsed = time.perf_counter() - t_start
logger.info(
"Dream run complete in {:.1f}s: stop_reason={}, tool_events={}",
elapsed, result.stop_reason, len(result.tool_events),
)
except Exception:
elapsed = time.perf_counter() - t_start
logger.exception("Dream run failed after {:.1f}s", elapsed)
result = None
# Build changelog from tool events
changelog: list[str] = []
if result and result.tool_events:
for event in result.tool_events:
if event.get("status") == "ok":
changelog.append(f"{event['name']}: {event['detail']}")
success = result is not None and result.stop_reason == "completed"
if success:
new_cursor = batch[-1]["cursor"]
self.dream.store.set_last_dream_cursor(new_cursor)
session.metadata.setdefault("_dream_changelog", []).extend(changelog)
self.sessions.save(session)
logger.info(
"Dream done: {} change(s), cursor advanced to {}",
len(changelog), new_cursor,
)
else:
reason = result.stop_reason if result else "exception"
logger.warning(
"Dream incomplete ({}): cursor NOT advanced, stopping",
reason,
)
return
self.dream.store.compact_history()
# Persist session record for debugging / visualization
record = {
"timestamp": datetime.now().isoformat(),
"batch": {
"from_cursor": last_cursor,
"to_cursor": batch[-1]["cursor"],
"count": len(batch),
},
"prompt_chars": len(user_prompt),
"elapsed_seconds": elapsed,
"stop_reason": result.stop_reason,
"usage": result.usage,
"tool_events": result.tool_events,
"changelog": changelog,
"commit_sha": None,
"messages": result.messages,
}
self.dream.store.write_dream_session(record)
session.metadata["_dream_last_record"] = record
async def _dream_finalize_commit(self, session: Session) -> None:
"""Collapse accumulated changelog into a single git commit, clear caches, and complete the goal."""
changelog = session.metadata.pop("_dream_changelog", [])
sha = None
if changelog and self.dream.store.git.is_initialized():
ts = datetime.now().strftime("%Y-%m-%d %H:%M")
summary = f"dream: {ts}, {len(changelog)} change(s)"
commit_msg = f"{summary}\n\n" + "\n".join(changelog)
sha = self.dream.store.git.auto_commit(commit_msg)
if sha:
logger.info("Dream commit: {}", sha)
record = session.metadata.pop("_dream_last_record", None)
if record and sha:
record["commit_sha"] = sha
self.dream.store.write_dream_session(record)
session.metadata.pop("_dream_system_prompt", None)
session.metadata.pop("_dream_system_prompt_mtime", None)
trigger_channel = session.metadata.pop("_dream_trigger_channel", None)
trigger_chat_id = session.metadata.pop("_dream_trigger_chat_id", None)
goal = session.metadata.get(GOAL_STATE_KEY)
if isinstance(goal, dict) and goal.get("status") == "active":
session.metadata[GOAL_STATE_KEY] = {
**goal,
"status": "completed",
"completed_at": datetime.now().isoformat(),
"recap": f"Memory backlog consolidated ({len(changelog)} change(s)).",
}
self.sessions.save(session)
session.metadata["_dream_finalized"] = True
# Notify the user who triggered /dream
if trigger_channel and trigger_chat_id:
content = f"Dream completed: {len(changelog)} change(s) committed."
if not changelog:
content = "Dream: nothing to process."
await self.bus.publish_outbound(OutboundMessage(
channel=trigger_channel,
chat_id=trigger_chat_id,
content=content,
))
async def _process_message( async def _process_message(
self, self,
msg: InboundMessage, msg: InboundMessage,
@@ -1271,7 +1473,7 @@ class AgentLoop:
msg = ctx.msg msg = ctx.msg
if msg.media: if msg.media:
new_content, image_only = self._prepare_message_media(msg.content, msg.media) new_content, image_only = extract_documents(msg.content, msg.media)
ctx.msg = dataclasses.replace(msg, content=new_content, media=image_only) ctx.msg = dataclasses.replace(msg, content=new_content, media=image_only)
msg = ctx.msg msg = ctx.msg
@@ -1283,7 +1485,6 @@ class AgentLoop:
if ctx.session is None: if ctx.session is None:
ctx.session = self.sessions.get_or_create(ctx.session_key) ctx.session = self.sessions.get_or_create(ctx.session_key)
mark_webui_session(ctx.session, msg.metadata) mark_webui_session(ctx.session, msg.metadata)
self.workspace_scopes.persist_message_scope(ctx.session, msg)
if self._restore_runtime_checkpoint(ctx.session): if self._restore_runtime_checkpoint(ctx.session):
self.sessions.save(ctx.session) self.sessions.save(ctx.session)
@@ -1292,16 +1493,6 @@ class AgentLoop:
return "ok" return "ok"
def _prepare_message_media(self, content: str, media: list[str]) -> tuple[str, list[str]]:
if self._should_extract_document_text():
return extract_documents(content, media)
return reference_non_image_attachments(content, media)
def _should_extract_document_text(self) -> bool:
if self.channels_config is None:
return True
return self.channels_config.extract_document_text
async def _state_compact(self, ctx: TurnContext) -> str: async def _state_compact(self, ctx: TurnContext) -> str:
ctx.session, pending = self.auto_compact.prepare_session(ctx.session, ctx.session_key) ctx.session, pending = self.auto_compact.prepare_session(ctx.session, ctx.session_key)
ctx.pending_summary = pending ctx.pending_summary = pending
@@ -1361,10 +1552,7 @@ class AgentLoop:
) )
ctx.initial_messages = self._build_initial_messages( ctx.initial_messages = self._build_initial_messages(
ctx.msg, ctx.msg, ctx.session, ctx.history, ctx.pending_summary
ctx.session,
ctx.history,
ctx.pending_summary,
) )
ctx.user_persisted_early = self._persist_user_message_early( ctx.user_persisted_early = self._persist_user_message_early(
ctx.msg, ctx.session ctx.msg, ctx.session
@@ -1667,19 +1855,10 @@ class AgentLoop:
channel=channel, sender_id="user", chat_id=chat_id, channel=channel, sender_id="user", chat_id=chat_id,
content=content, media=media or [], content=content, media=media or [],
) )
# Share the dispatch lock so direct calls serialize with bus turns. return await self._process_message(
lock = self._session_locks.setdefault(session_key, asyncio.Lock()) msg,
try: session_key=session_key,
async with lock: on_progress=on_progress,
return await self._process_message( on_stream=on_stream,
msg, on_stream_end=on_stream_end,
session_key=session_key, )
on_progress=on_progress,
on_stream=on_stream,
on_stream_end=on_stream_end,
)
finally:
if channel == "websocket":
await self._webui_turns.publish_run_status(msg, "idle")
self._pending_turn_latency_ms.pop(session_key, None)
self._webui_turns.discard(session_key)
+168 -191
View File
@@ -6,6 +6,7 @@ import asyncio
import json import json
import os import os
import re import re
import time
import weakref import weakref
from contextlib import suppress from contextlib import suppress
from datetime import datetime from datetime import datetime
@@ -15,7 +16,7 @@ from typing import TYPE_CHECKING, Any, Callable, Iterator
import tiktoken import tiktoken
from loguru import logger from loguru import logger
from nanobot.agent.runner import AgentRunner, AgentRunSpec from nanobot.agent.runner import AgentRunner
from nanobot.agent.tools.registry import ToolRegistry from nanobot.agent.tools.registry import ToolRegistry
from nanobot.session.manager import Session from nanobot.session.manager import Session
from nanobot.utils.gitstore import GitStore from nanobot.utils.gitstore import GitStore
@@ -33,6 +34,20 @@ if TYPE_CHECKING:
from nanobot.providers.base import LLMProvider from nanobot.providers.base import LLMProvider
from nanobot.session.manager import SessionManager from nanobot.session.manager import SessionManager
# Cache the tiktoken encoding to avoid repeated instantiation on every
# truncate/encode call. Encoding objects are thread-safe and reusable.
try:
_TIKTOKEN_ENC = tiktoken.get_encoding("cl100k_base")
except Exception: # pragma: no cover
_TIKTOKEN_ENC = None
def _estimate_tokens(text: str) -> int:
"""Approximate token count for a text string."""
if _TIKTOKEN_ENC is not None:
return len(_TIKTOKEN_ENC.encode(text))
return len(text) // 4
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# MemoryStore — pure file I/O layer # MemoryStore — pure file I/O layer
@@ -400,6 +415,26 @@ class MemoryStore:
def set_last_dream_cursor(self, cursor: int) -> None: def set_last_dream_cursor(self, cursor: int) -> None:
self._dream_cursor_file.write_text(str(cursor), encoding="utf-8") self._dream_cursor_file.write_text(str(cursor), encoding="utf-8")
def write_dream_session(self, data: dict[str, Any]) -> None:
"""Atomic overwrite of the latest Dream run record."""
path = self.memory_dir / ".dream_session.json"
tmp_path = path.with_suffix(".tmp")
try:
with open(tmp_path, "w", encoding="utf-8") as f:
json.dump(data, f, ensure_ascii=False, indent=2)
f.flush()
os.fsync(f.fileno())
os.replace(tmp_path, path)
with suppress(PermissionError):
fd = os.open(str(path.parent), os.O_RDONLY)
try:
os.fsync(fd)
finally:
os.close(fd)
except BaseException:
tmp_path.unlink(missing_ok=True)
raise
# -- message formatting utility ------------------------------------------ # -- message formatting utility ------------------------------------------
@staticmethod @staticmethod
@@ -618,19 +653,21 @@ class Consolidator:
"""Available input token budget for consolidation LLM.""" """Available input token budget for consolidation LLM."""
return self.context_window_tokens - self.max_completion_tokens - self._SAFETY_BUFFER return self.context_window_tokens - self.max_completion_tokens - self._SAFETY_BUFFER
def _truncate_to_token_budget(self, text: str) -> str: def _truncate_to_token_budget(self, text: str, reserve_tokens: int = 0) -> str:
"""Truncate text so it fits within the consolidation LLM's token budget.""" """Truncate text so it fits within the consolidation LLM's token budget.
budget = self._input_token_budget
reserve_tokens: additional tokens to reserve for dedup context or other
overhead that will be appended after truncation.
"""
budget = self._input_token_budget - reserve_tokens
if budget <= 0: if budget <= 0:
return truncate_text(text, _RAW_ARCHIVE_MAX_CHARS) return truncate_text(text, _RAW_ARCHIVE_MAX_CHARS)
try: if _TIKTOKEN_ENC is not None:
enc = tiktoken.get_encoding("cl100k_base") tokens = _TIKTOKEN_ENC.encode(text)
tokens = enc.encode(text)
if len(tokens) <= budget: if len(tokens) <= budget:
return text return text
return enc.decode(tokens[:budget]) + "\n... (truncated)" return _TIKTOKEN_ENC.decode(tokens[:budget]) + "\n... (truncated)"
except Exception: return truncate_text(text, budget * 4)
return truncate_text(text, budget * 4)
async def archive(self, messages: list[dict]) -> str | None: async def archive(self, messages: list[dict]) -> str | None:
"""Summarize messages via LLM and append to history.jsonl. """Summarize messages via LLM and append to history.jsonl.
@@ -639,9 +676,53 @@ class Consolidator:
""" """
if not messages: if not messages:
return None return None
t_start = time.perf_counter()
try: try:
formatted = MemoryStore._format_messages(messages) formatted = MemoryStore._format_messages(messages)
formatted = self._truncate_to_token_budget(formatted) logger.debug(
"Consolidator: {} messages, formatted={} chars",
len(messages), len(formatted),
)
# Inject current memory context for dedup-aware summarization.
memory_preview = self.store.read_memory()[:4000]
user_preview = self.store.read_user()[:2000]
dedup_context = ""
if memory_preview:
dedup_context += f"\n\n## Current MEMORY.md (for dedup)\n{memory_preview}"
if user_preview:
dedup_context += f"\n\n## Current USER.md (for dedup)\n{user_preview}"
reserve_tokens = 0
if dedup_context:
if _TIKTOKEN_ENC is not None:
reserve_tokens = len(_TIKTOKEN_ENC.encode(dedup_context)) + 100
else:
reserve_tokens = len(dedup_context) // 4 + 100
if self._input_token_budget <= reserve_tokens:
logger.warning(
"Consolidator: dedup_context ({} tokens) exceeds budget ({}), dropping it",
reserve_tokens, self._input_token_budget,
)
dedup_context = ""
reserve_tokens = 0
else:
logger.debug(
"Consolidator: dedup_context={} chars, reserve_tokens={}",
len(dedup_context), reserve_tokens,
)
formatted_before = len(formatted)
formatted = self._truncate_to_token_budget(
formatted, reserve_tokens=reserve_tokens
)
if len(formatted) < formatted_before:
logger.warning(
"Consolidator: truncated formatted messages from {} to {} chars",
formatted_before, len(formatted),
)
response = await self.provider.chat_with_retry( response = await self.provider.chat_with_retry(
model=self.model, model=self.model,
messages=[ messages=[
@@ -652,18 +733,31 @@ class Consolidator:
strip=True, strip=True,
), ),
}, },
{"role": "user", "content": formatted}, {"role": "user", "content": formatted + dedup_context},
], ],
tools=None, tools=None,
tool_choice=None, tool_choice=None,
) )
elapsed = time.perf_counter() - t_start
if response.finish_reason == "error": if response.finish_reason == "error":
logger.warning(
"Consolidator LLM error after {:.1f}s: {}",
elapsed, response.content,
)
raise RuntimeError(f"LLM returned error: {response.content}") raise RuntimeError(f"LLM returned error: {response.content}")
summary = response.content or "[no summary]" summary = response.content or "[no summary]"
logger.info(
"Consolidator: {} entries -> {} chars summary in {:.1f}s",
len(messages), len(summary), elapsed,
)
self.store.append_history(summary, max_chars=_ARCHIVE_SUMMARY_MAX_CHARS) self.store.append_history(summary, max_chars=_ARCHIVE_SUMMARY_MAX_CHARS)
return summary return summary
except Exception: except Exception:
logger.warning("Consolidation LLM call failed, raw-dumping to history") elapsed = time.perf_counter() - t_start
logger.warning(
"Consolidation LLM call failed after {:.1f}s, raw-dumping to history",
elapsed,
)
self.store.raw_archive(messages) self.store.raw_archive(messages)
return None return None
@@ -851,38 +945,48 @@ class Consolidator:
# Single source of truth for the staleness threshold used in _annotate_with_ages # Single source of truth for the staleness threshold used in _annotate_with_ages
# *and* in the Phase 1 prompt template (passed as `stale_threshold_days`). # *and* in the system prompt template (passed as `stale_threshold_days`).
# Keep code and prompt aligned — if you bump this, the LLM's instruction string # Keep code and prompt aligned — if you bump this, the LLM's instruction string
# updates automatically. # updates automatically.
_STALE_THRESHOLD_DAYS = 14 _STALE_THRESHOLD_DAYS = 14
_SKIP_LINE_RE = re.compile(r"^\s*-\s*\[skip\]\s*.*$", re.MULTILINE | re.IGNORECASE)
def _strip_skip_lines(text: str) -> str:
"""Remove lines marked [skip] from history content."""
lines = text.splitlines()
kept = [line for line in lines if not _SKIP_LINE_RE.match(line)]
return "\n".join(kept)
class Dream: class Dream:
"""Two-phase memory processor: analyze history.jsonl, then edit files via AgentRunner. """Single-phase memory processor: analyze history.jsonl and edit files via AgentRunner.
Phase 1 produces an analysis summary (plain LLM call). Delegates to AgentRunner with read_file / edit_file tools so the LLM can
Phase 2 delegates to AgentRunner with read_file / edit_file tools so the analyze conversation history, extract facts, deduplicate, and make targeted
LLM can make targeted, incremental edits instead of replacing entire files. incremental edits — all in a single agent run.
""" """
# Caps on prompt-bound inputs so Dream's LLM calls never exceed the model's # Caps on prompt-bound inputs so Dream's LLM calls never exceed the model's
# context window just because a file (or a legacy large history entry) grew # context window just because a file (or a legacy large history entry) grew
# unexpectedly. Each file still appears in full via read_file when the agent # unexpectedly. Each file still appears in full via read_file when the agent
# needs it in Phase 2 — these caps only bound the Phase 1/2 prompt preview. # needs it — these caps only bound the prompt preview.
_MEMORY_FILE_MAX_CHARS = 32_000 _MEMORY_FILE_MAX_CHARS = 16_000
_SOUL_FILE_MAX_CHARS = 16_000 _SOUL_FILE_MAX_CHARS = 4_000
_USER_FILE_MAX_CHARS = 16_000 _USER_FILE_MAX_CHARS = 4_000
_HISTORY_ENTRY_PREVIEW_MAX_CHARS = 4_000 _HISTORY_ENTRY_PREVIEW_MAX_CHARS = 2_000
def __init__( def __init__(
self, self,
store: MemoryStore, store: MemoryStore,
provider: LLMProvider, provider: LLMProvider,
model: str, model: str,
max_batch_size: int = 20, max_batch_size: int = 5,
max_iterations: int = 10, max_iterations: int = 10,
max_tool_result_chars: int = 16_000, max_tool_result_chars: int = 16_000,
annotate_line_ages: bool = True, annotate_line_ages: bool = True,
edit_user_skills: bool = False,
): ):
self.store = store self.store = store
self.provider = provider self.provider = provider
@@ -890,10 +994,13 @@ class Dream:
self.max_batch_size = max_batch_size self.max_batch_size = max_batch_size
self.max_iterations = max_iterations self.max_iterations = max_iterations
self.max_tool_result_chars = max_tool_result_chars self.max_tool_result_chars = max_tool_result_chars
# Kill switch for the git-blame-based per-line age annotation in Phase 1. # Kill switch for the git-blame-based per-line age annotation in the prompt.
# Default True keeps the #3212 behavior; set False to feed MEMORY.md raw # Default True keeps the #3212 behavior; set False to feed all memory
# (e.g. if a specific LLM reacts poorly to the `← Nd` suffix). # files raw (e.g. if a specific LLM reacts poorly to the `← Nd` suffix).
self.annotate_line_ages = annotate_line_ages self.annotate_line_ages = annotate_line_ages
# When True, Dream may edit/delete user-created workspace skills.
# When False, only skills with dream_managed: true in frontmatter are editable.
self.edit_user_skills = edit_user_skills
self._runner = AgentRunner(provider) self._runner = AgentRunner(provider)
self._tools = self._build_tools() self._tools = self._build_tools()
@@ -907,6 +1014,7 @@ class Dream:
def _build_tools(self) -> ToolRegistry: def _build_tools(self) -> ToolRegistry:
"""Build a minimal tool registry for the Dream agent.""" """Build a minimal tool registry for the Dream agent."""
from nanobot.agent.skills import BUILTIN_SKILLS_DIR from nanobot.agent.skills import BUILTIN_SKILLS_DIR
from nanobot.agent.tools.apply_patch import ApplyPatchTool
from nanobot.agent.tools.file_state import FileStates from nanobot.agent.tools.file_state import FileStates
from nanobot.agent.tools.filesystem import EditFileTool, ReadFileTool, WriteFileTool from nanobot.agent.tools.filesystem import EditFileTool, ReadFileTool, WriteFileTool
@@ -924,6 +1032,7 @@ class Dream:
file_states=file_states, file_states=file_states,
)) ))
tools.register(EditFileTool(workspace=workspace, allowed_dir=workspace, file_states=file_states)) tools.register(EditFileTool(workspace=workspace, allowed_dir=workspace, file_states=file_states))
tools.register(ApplyPatchTool(workspace=workspace, allowed_dir=workspace, file_states=file_states))
# write_file resolves relative paths from workspace root, but can only # write_file resolves relative paths from workspace root, but can only
# write under skills/ so the prompt can safely use skills/<name>/SKILL.md. # write under skills/ so the prompt can safely use skills/<name>/SKILL.md.
skills_dir = workspace / "skills" skills_dir = workspace / "skills"
@@ -933,15 +1042,25 @@ class Dream:
# -- skill listing -------------------------------------------------------- # -- skill listing --------------------------------------------------------
def _list_existing_skills(self) -> list[str]: def _list_existing_skills(self, tag_origin: bool = False) -> list[str]:
"""List existing skills as 'name — description' for dedup context.""" """List existing skills as 'name — description [origin]' for dedup context.
When *tag_origin* is True each entry gets an origin tag:
``[dream]`` for skills with ``dream_managed: true`` in frontmatter,
``[user]`` for other workspace skills, ``[builtin]`` for bundled skills.
"""
import re as _re import re as _re
from nanobot.agent.skills import BUILTIN_SKILLS_DIR from nanobot.agent.skills import BUILTIN_SKILLS_DIR
desc_re = _re.compile(r"^description:\s*(.+)$", _re.MULTILINE | _re.IGNORECASE) desc_re = _re.compile(r"^description:\s*(.+)$", _re.MULTILINE | _re.IGNORECASE)
entries: dict[str, str] = {} managed_re = _re.compile(r"^dream_managed:\s*true$", _re.MULTILINE | _re.IGNORECASE)
for base in (self.store.workspace / "skills", BUILTIN_SKILLS_DIR):
entries: dict[str, tuple[str, str]] = {} # name -> (desc, tag)
builtin_dir = BUILTIN_SKILLS_DIR
ws_skills_dir = self.store.workspace / "skills"
for base in (ws_skills_dir, builtin_dir):
if not base.exists(): if not base.exists():
continue continue
for d in base.iterdir(): for d in base.iterdir():
@@ -951,18 +1070,31 @@ class Dream:
if not skill_md.exists(): if not skill_md.exists():
continue continue
# Prefer workspace skills over builtin (same name) # Prefer workspace skills over builtin (same name)
if d.name in entries and base == BUILTIN_SKILLS_DIR: if d.name in entries and base == builtin_dir:
continue continue
content = skill_md.read_text(encoding="utf-8")[:500] content = skill_md.read_text(encoding="utf-8")[:500]
m = desc_re.search(content) m = desc_re.search(content)
desc = m.group(1).strip() if m else "(no description)" desc = m.group(1).strip() if m else "(no description)"
entries[d.name] = desc
return [f"{name}{desc}" for name, desc in sorted(entries.items())] if tag_origin:
if base == builtin_dir:
tag = "[builtin]"
elif managed_re.search(content):
tag = "[dream]"
else:
tag = "[user]"
entries[d.name] = (desc, tag)
else:
entries[d.name] = (desc, "")
if tag_origin:
return [f"{name}{desc} {tag}" for name, (desc, tag) in sorted(entries.items())]
return [f"{name}{desc}" for name, (desc, _) in sorted(entries.items())]
# -- main entry ---------------------------------------------------------- # -- main entry ----------------------------------------------------------
def _annotate_with_ages(self, content: str) -> str: def _annotate_with_ages(self, content: str, file_path: str = "memory/MEMORY.md") -> str:
"""Append per-line age suffixes to MEMORY.md content. """Append per-line age suffixes to file content.
Each non-blank line whose age exceeds ``_STALE_THRESHOLD_DAYS`` gets a Each non-blank line whose age exceeds ``_STALE_THRESHOLD_DAYS`` gets a
suffix like ``← 30d`` indicating days since last modification. suffix like ``← 30d`` indicating days since last modification.
@@ -970,9 +1102,7 @@ class Dream:
annotate fails, or the line count doesn't match the age count annotate fails, or the line count doesn't match the age count
(which can happen with an uncommitted working-tree edit — better to (which can happen with an uncommitted working-tree edit — better to
skip annotation than to tag the wrong line). skip annotation than to tag the wrong line).
SOUL.md and USER.md are never annotated.
""" """
file_path = "memory/MEMORY.md"
try: try:
ages = self.store.git.line_ages(file_path) ages = self.store.git.line_ages(file_path)
except Exception: except Exception:
@@ -1007,156 +1137,3 @@ class Dream:
result += "\n" result += "\n"
return result return result
async def run(self) -> bool:
"""Process unprocessed history entries. Returns True if work was done."""
from nanobot.agent.skills import BUILTIN_SKILLS_DIR
last_cursor = self.store.get_last_dream_cursor()
entries = self.store.read_unprocessed_history(since_cursor=last_cursor)
if not entries:
return False
batch = entries[: self.max_batch_size]
logger.info(
"Dream: processing {} entries (cursor {}{}), batch={}",
len(entries), last_cursor, batch[-1]["cursor"], len(batch),
)
# Build history text for LLM — cap each entry so a legacy oversized
# record (e.g. pre-#3412 raw_archive dump) can't blow up the prompt.
history_text = "\n".join(
f"[{e['timestamp']}] "
f"{truncate_text(e['content'], self._HISTORY_ENTRY_PREVIEW_MAX_CHARS)}"
for e in batch
)
# Current file contents + per-line age annotations (MEMORY.md only).
# Each file is capped in the *prompt preview* only; Phase 2 still sees
# the full file via the read_file tool.
current_date = datetime.now().strftime("%Y-%m-%d")
raw_memory = self.store.read_memory() or "(empty)"
annotated_memory = (
self._annotate_with_ages(raw_memory)
if self.annotate_line_ages
else raw_memory
)
current_memory = truncate_text(annotated_memory, self._MEMORY_FILE_MAX_CHARS)
current_soul = truncate_text(
self.store.read_soul() or "(empty)", self._SOUL_FILE_MAX_CHARS,
)
current_user = truncate_text(
self.store.read_user() or "(empty)", self._USER_FILE_MAX_CHARS,
)
file_context = (
f"## Current Date\n{current_date}\n\n"
f"## Current MEMORY.md ({len(current_memory)} chars)\n{current_memory}\n\n"
f"## Current SOUL.md ({len(current_soul)} chars)\n{current_soul}\n\n"
f"## Current USER.md ({len(current_user)} chars)\n{current_user}"
)
# Phase 1: Analyze (no skills list — dedup is Phase 2's job)
phase1_prompt = (
f"## Conversation History\n{history_text}\n\n{file_context}"
)
try:
phase1_response = await self.provider.chat_with_retry(
model=self.model,
messages=[
{
"role": "system",
"content": render_template(
"agent/dream_phase1.md",
strip=True,
stale_threshold_days=_STALE_THRESHOLD_DAYS,
),
},
{"role": "user", "content": phase1_prompt},
],
tools=None,
tool_choice=None,
)
analysis = phase1_response.content or ""
logger.debug("Dream Phase 1 analysis ({} chars): {}", len(analysis), analysis[:500])
except Exception:
logger.exception("Dream Phase 1 failed")
return False
# Phase 2: Delegate to AgentRunner with read_file / edit_file
existing_skills = self._list_existing_skills()
skills_section = ""
if existing_skills:
skills_section = (
"\n\n## Existing Skills\n"
+ "\n".join(f"- {s}" for s in existing_skills)
)
phase2_prompt = f"## Analysis Result\n{analysis}\n\n{file_context}{skills_section}"
tools = self._tools
skill_creator_path = BUILTIN_SKILLS_DIR / "skill-creator" / "SKILL.md"
messages: list[dict[str, Any]] = [
{
"role": "system",
"content": render_template(
"agent/dream_phase2.md",
strip=True,
skill_creator_path=str(skill_creator_path),
),
},
{"role": "user", "content": phase2_prompt},
]
try:
result = await self._runner.run(AgentRunSpec(
initial_messages=messages,
tools=tools,
model=self.model,
max_iterations=self.max_iterations,
max_tool_result_chars=self.max_tool_result_chars,
fail_on_tool_error=False,
))
logger.debug(
"Dream Phase 2 complete: stop_reason={}, tool_events={}",
result.stop_reason, len(result.tool_events),
)
for ev in (result.tool_events or []):
logger.info("Dream tool_event: name={}, status={}, detail={}", ev.get("name"), ev.get("status"), ev.get("detail", "")[:200])
except Exception:
logger.exception("Dream Phase 2 failed")
result = None
# Build changelog from tool events
changelog: list[str] = []
if result and result.tool_events:
for event in result.tool_events:
if event["status"] == "ok":
changelog.append(f"{event['name']}: {event['detail']}")
# Only advance cursor on successful completion to prevent silent loss
if result and result.stop_reason == "completed":
new_cursor = batch[-1]["cursor"]
self.store.set_last_dream_cursor(new_cursor)
logger.info(
"Dream done: {} change(s), cursor advanced to {}",
len(changelog), new_cursor,
)
else:
reason = result.stop_reason if result else "exception"
logger.warning(
"Dream incomplete ({}): cursor NOT advanced, will retry next cron cycle",
reason,
)
self.store.compact_history()
# Git auto-commit (only when there are actual changes)
if changelog and self.store.git.is_initialized():
ts = batch[-1]["timestamp"]
summary = f"dream: {ts}, {len(changelog)} change(s)"
commit_msg = f"{summary}\n\n{analysis.strip()}"
sha = self.store.git.auto_commit(commit_msg)
if sha:
logger.info("Dream commit: {}", sha)
return True
+12 -33
View File
@@ -16,14 +16,12 @@ from nanobot.agent.hook import AgentHook, AgentHookContext
from nanobot.agent.tools.registry import ToolRegistry from nanobot.agent.tools.registry import ToolRegistry
from nanobot.providers.base import LLMProvider, LLMResponse, ToolCallRequest from nanobot.providers.base import LLMProvider, LLMResponse, ToolCallRequest
from nanobot.utils.file_edit_events import ( from nanobot.utils.file_edit_events import (
StreamingFileEditTracker,
build_file_edit_end_event, build_file_edit_end_event,
build_file_edit_error_event, build_file_edit_error_event,
build_file_edit_start_event, build_file_edit_start_event,
prepare_file_edit_trackers,
)
from nanobot.utils.file_edit_events import (
prepare_file_edit_tracker as _prepare_file_edit_tracker, prepare_file_edit_tracker as _prepare_file_edit_tracker,
prepare_file_edit_trackers,
StreamingFileEditTracker,
) )
from nanobot.utils.helpers import ( from nanobot.utils.helpers import (
IncrementalThinkExtractor, IncrementalThinkExtractor,
@@ -53,10 +51,6 @@ from nanobot.utils.runtime import (
) )
_DEFAULT_ERROR_MESSAGE = "Sorry, I encountered an error calling the AI model." _DEFAULT_ERROR_MESSAGE = "Sorry, I encountered an error calling the AI model."
_ARREARAGE_ERROR_MESSAGE = (
"The AI provider rejected the request because the API key is out of quota or the "
"account is in arrears. Please top up / check the billing status of your API key and try again."
)
_PERSISTED_MODEL_ERROR_PLACEHOLDER = "[Assistant reply unavailable due to model error.]" _PERSISTED_MODEL_ERROR_PLACEHOLDER = "[Assistant reply unavailable due to model error.]"
_MAX_EMPTY_RETRIES = 2 _MAX_EMPTY_RETRIES = 2
_MAX_LENGTH_RECOVERIES = 3 _MAX_LENGTH_RECOVERIES = 3
@@ -185,19 +179,16 @@ class AgentRunner:
and *iteration* are both provided) and return (True, cycles+1) so the and *iteration* are both provided) and return (True, cycles+1) so the
caller continues the iteration loop. Otherwise return (False, cycles). caller continues the iteration loop. Otherwise return (False, cycles).
""" """
injections: list[dict[str, Any]] = [] if injection_cycles >= _MAX_INJECTION_CYCLES:
real_injection = False return False, injection_cycles
if injection_cycles < _MAX_INJECTION_CYCLES: injections = await self._drain_injections(spec)
injections = await self._drain_injections(spec)
real_injection = bool(injections)
if not injections and allow_goal_continue and assistant_message is not None: if not injections and allow_goal_continue and assistant_message is not None:
predicate = spec.goal_active_predicate predicate = spec.goal_active_predicate
if predicate is not None and predicate(): if predicate is not None and predicate():
injections = [build_goal_continue_message(spec.goal_continue_message)] injections = [build_goal_continue_message(spec.goal_continue_message)]
if not injections: if not injections:
return False, injection_cycles return False, injection_cycles
if real_injection: injection_cycles += 1
injection_cycles += 1
if assistant_message is not None: if assistant_message is not None:
messages.append(assistant_message) messages.append(assistant_message)
if iteration is not None: if iteration is not None:
@@ -213,13 +204,10 @@ class AgentRunner:
}, },
) )
self._append_injected_messages(messages, injections) self._append_injected_messages(messages, injections)
if real_injection: logger.info(
logger.info( "Injected {} follow-up message(s) {} ({}/{})",
"Injected {} follow-up message(s) {} ({}/{})", len(injections), phase, injection_cycles, _MAX_INJECTION_CYCLES,
len(injections), phase, injection_cycles, _MAX_INJECTION_CYCLES, )
)
else:
logger.info("Injected sustained-goal continuation {}", phase)
return True, injection_cycles return True, injection_cycles
async def _drain_injections(self, spec: AgentRunSpec) -> list[dict[str, Any]]: async def _drain_injections(self, spec: AgentRunSpec) -> list[dict[str, Any]]:
@@ -508,10 +496,7 @@ class AgentRunner:
continue continue
if response.finish_reason == "error": if response.finish_reason == "error":
if LLMProvider.is_arrearage_response(response): final_content = clean or spec.error_message or _DEFAULT_ERROR_MESSAGE
final_content = _ARREARAGE_ERROR_MESSAGE
else:
final_content = clean or spec.error_message or _DEFAULT_ERROR_MESSAGE
stop_reason = "error" stop_reason = "error"
error = final_content error = final_content
self._append_model_error_placeholder(messages) self._append_model_error_placeholder(messages)
@@ -1280,13 +1265,7 @@ class AgentRunner:
return messages return messages
system_tokens = sum(estimate_message_tokens(msg) for msg in system_messages) system_tokens = sum(estimate_message_tokens(msg) for msg in system_messages)
fixed_tokens, _ = estimate_prompt_tokens_chain( remaining_budget = max(128, budget - system_tokens)
self.provider,
spec.model,
system_messages,
spec.tools.get_definitions(),
)
remaining_budget = max(0, budget - max(system_tokens, fixed_tokens))
kept: list[dict[str, Any]] = [] kept: list[dict[str, Any]] = []
kept_tokens = 0 kept_tokens = 0
for message in reversed(non_system): for message in reversed(non_system):
+21 -52
View File
@@ -16,12 +16,6 @@ from nanobot.agent.tools.context import ToolContext
from nanobot.agent.tools.file_state import FileStates from nanobot.agent.tools.file_state import FileStates
from nanobot.agent.tools.loader import ToolLoader from nanobot.agent.tools.loader import ToolLoader
from nanobot.agent.tools.registry import ToolRegistry from nanobot.agent.tools.registry import ToolRegistry
from nanobot.security.workspace_access import (
WorkspaceScope,
bind_workspace_scope,
reset_workspace_scope,
workspace_sandbox_status,
)
from nanobot.bus.events import InboundMessage from nanobot.bus.events import InboundMessage
from nanobot.bus.queue import MessageBus from nanobot.bus.queue import MessageBus
from nanobot.config.schema import AgentDefaults, ToolsConfig from nanobot.config.schema import AgentDefaults, ToolsConfig
@@ -134,10 +128,6 @@ class SubagentManager:
config=cfg, config=cfg,
workspace=str(root.resolve()), workspace=str(root.resolve()),
file_state_store=FileStates(), file_state_store=FileStates(),
workspace_sandbox=workspace_sandbox_status(
restrict_to_workspace=cfg.restrict_to_workspace,
workspace=root,
),
) )
ToolLoader().load(ctx, registry, scope="subagent") ToolLoader().load(ctx, registry, scope="subagent")
return registry return registry
@@ -156,7 +146,6 @@ class SubagentManager:
session_key: str | None = None, session_key: str | None = None,
origin_message_id: str | None = None, origin_message_id: str | None = None,
temperature: float | None = None, temperature: float | None = None,
workspace_scope: WorkspaceScope | None = None,
) -> str: ) -> str:
"""Spawn a subagent to execute a task in the background.""" """Spawn a subagent to execute a task in the background."""
task_id = str(uuid.uuid4())[:8] task_id = str(uuid.uuid4())[:8]
@@ -173,14 +162,7 @@ class SubagentManager:
bg_task = asyncio.create_task( bg_task = asyncio.create_task(
self._run_subagent( self._run_subagent(
task_id, task_id, task, display_label, origin, status, origin_message_id, temperature
task,
display_label,
origin,
status,
origin_message_id,
temperature,
workspace_scope,
) )
) )
self._running_tasks[task_id] = bg_task self._running_tasks[task_id] = bg_task
@@ -209,7 +191,6 @@ class SubagentManager:
status: SubagentStatus, status: SubagentStatus,
origin_message_id: str | None = None, origin_message_id: str | None = None,
temperature: float | None = None, temperature: float | None = None,
workspace_scope: WorkspaceScope | None = None,
) -> None: ) -> None:
"""Execute the subagent task and announce the result.""" """Execute the subagent task and announce the result."""
logger.info("Subagent [{}] starting task: {}", task_id, label) logger.info("Subagent [{}] starting task: {}", task_id, label)
@@ -219,13 +200,8 @@ class SubagentManager:
status.iteration = payload.get("iteration", status.iteration) status.iteration = payload.get("iteration", status.iteration)
try: try:
root = workspace_scope.project_path if workspace_scope is not None else self.workspace tools = self._build_tools()
cfg = None system_prompt = self._build_subagent_prompt()
if workspace_scope is not None:
cfg = self._subagent_tools_config()
cfg.restrict_to_workspace = workspace_scope.restrict_to_workspace
tools = self._build_tools(workspace=root, tools_config=cfg)
system_prompt = self._build_subagent_prompt(workspace=root)
messages: list[dict[str, Any]] = [ messages: list[dict[str, Any]] = [
{"role": "system", "content": system_prompt}, {"role": "system", "content": system_prompt},
{"role": "user", "content": task}, {"role": "user", "content": task},
@@ -237,27 +213,21 @@ class SubagentManager:
if self._llm_wall_timeout_for_session if self._llm_wall_timeout_for_session
else None else None
) )
token = bind_workspace_scope(workspace_scope) if workspace_scope is not None else None result = await self.runner.run(AgentRunSpec(
try: initial_messages=messages,
result = await self.runner.run(AgentRunSpec( tools=tools,
initial_messages=messages, model=self.model,
tools=tools, temperature=temperature,
model=self.model, max_iterations=self.max_iterations,
temperature=temperature, max_tool_result_chars=self.max_tool_result_chars,
max_iterations=self.max_iterations, hook=_SubagentHook(task_id, status),
max_tool_result_chars=self.max_tool_result_chars, max_iterations_message="Task completed but no final response was generated.",
hook=_SubagentHook(task_id, status), error_message=None,
max_iterations_message="Task completed but no final response was generated.", fail_on_tool_error=True,
error_message=None, checkpoint_callback=_on_checkpoint,
fail_on_tool_error=True, session_key=sess_key,
checkpoint_callback=_on_checkpoint, llm_timeout_s=llm_timeout,
session_key=sess_key, ))
workspace=root,
llm_timeout_s=llm_timeout,
))
finally:
if token is not None:
reset_workspace_scope(token)
status.phase = "done" status.phase = "done"
status.stop_reason = result.stop_reason status.stop_reason = result.stop_reason
@@ -351,21 +321,20 @@ class SubagentManager:
lines.append(f"- {result.error}") lines.append(f"- {result.error}")
return "\n".join(lines) or (result.error or "Error: subagent execution failed.") return "\n".join(lines) or (result.error or "Error: subagent execution failed.")
def _build_subagent_prompt(self, workspace: Path | None = None) -> str: def _build_subagent_prompt(self) -> str:
"""Build a focused system prompt for the subagent.""" """Build a focused system prompt for the subagent."""
from nanobot.agent.context import ContextBuilder from nanobot.agent.context import ContextBuilder
from nanobot.agent.skills import SkillsLoader from nanobot.agent.skills import SkillsLoader
time_ctx = ContextBuilder._build_runtime_context(None, None) time_ctx = ContextBuilder._build_runtime_context(None, None)
root = workspace or self.workspace
skills_summary = SkillsLoader( skills_summary = SkillsLoader(
root, self.workspace,
disabled_skills=self.disabled_skills, disabled_skills=self.disabled_skills,
).build_skills_summary() ).build_skills_summary()
return render_template( return render_template(
"agent/subagent_system.md", "agent/subagent_system.md",
time_ctx=time_ctx, time_ctx=time_ctx,
workspace=str(root), workspace=str(self.workspace),
skills_summary=skills_summary or "", skills_summary=skills_summary or "",
) )
+69 -7
View File
@@ -88,11 +88,11 @@ def _format_summary(summary: _PatchSummary) -> str:
items=ObjectSchema( items=ObjectSchema(
path=StringSchema("Relative path to the file to edit."), path=StringSchema("Relative path to the file to edit."),
action=StringSchema( action=StringSchema(
"Operation type: replace or add.", "Operation type: replace (find and replace text), add (append new content or create file), delete (remove text).",
enum=["replace", "add"], enum=["replace", "add", "delete"],
), ),
old_text=StringSchema( old_text=StringSchema(
"Exact text to search for in the file. Required for replace.", "Exact text to search for in the file. Required for replace and delete.",
nullable=True, nullable=True,
), ),
new_text=StringSchema( new_text=StringSchema(
@@ -124,8 +124,7 @@ class ApplyPatchTool(_FsTool):
def description(self) -> str: def description(self) -> str:
return ( return (
"Default tool for code edits. Supports multi-file changes in a single call. " "Default tool for code edits. Supports multi-file changes in a single call. "
"Provide a list of structured edits, each specifying a file path, action " "Provide a list of structured edits, each specifying a file path, action (replace/add/delete), and the text to change. "
"(replace/add), and the exact text to change. "
"Paths must be relative. Set dry_run=true to validate and preview without writing files. " "Paths must be relative. Set dry_run=true to validate and preview without writing files. "
"Use edit_file only for small exact replacements on a single file." "Use edit_file only for small exact replacements on a single file."
) )
@@ -141,6 +140,7 @@ class ApplyPatchTool(_FsTool):
raise _PatchError("must provide edits") raise _PatchError("must provide edits")
writes: dict[Path, str] = {} writes: dict[Path, str] = {}
deletes: set[Path] = set()
summaries: list[_PatchSummary] = [] summaries: list[_PatchSummary] = []
for edit in edits: for edit in edits:
@@ -183,6 +183,7 @@ class ApplyPatchTool(_FsTool):
if uses_crlf: if uses_crlf:
new_norm = new_norm.replace("\n", "\r\n") new_norm = new_norm.replace("\n", "\r\n")
writes[source] = new_norm writes[source] = new_norm
deletes.discard(source)
added, deleted = _line_diff_stats(content, new_norm) added, deleted = _line_diff_stats(content, new_norm)
action_name = "update" action_name = "update"
else: else:
@@ -190,6 +191,7 @@ class ApplyPatchTool(_FsTool):
if new_norm and not new_norm.endswith("\n"): if new_norm and not new_norm.endswith("\n"):
new_norm += "\n" new_norm += "\n"
writes[source] = new_norm writes[source] = new_norm
deletes.discard(source)
added = _text_line_count(new_norm) added = _text_line_count(new_norm)
deleted = 0 deleted = 0
action_name = "add" action_name = "add"
@@ -244,6 +246,7 @@ class ApplyPatchTool(_FsTool):
new_norm = new_norm.replace("\n", "\r\n") new_norm = new_norm.replace("\n", "\r\n")
writes[source] = new_norm writes[source] = new_norm
deletes.discard(source)
added, deleted = _line_diff_stats(content, new_norm) added, deleted = _line_diff_stats(content, new_norm)
summaries.append( summaries.append(
_PatchSummary( _PatchSummary(
@@ -251,6 +254,62 @@ class ApplyPatchTool(_FsTool):
) )
) )
elif action == "delete":
old_text = edit.get("old_text") or ""
if not old_text:
raise _PatchError(f"old_text required for delete: {path}")
pending = writes.get(source)
if pending is not None:
content = pending
elif source.exists():
raw = source.read_bytes()
try:
content = raw.decode("utf-8")
except UnicodeDecodeError:
raise _PatchError(f"file is not UTF-8 text: {path}")
else:
raise _PatchError(f"file to update does not exist: {path}")
if pending is None and not source.is_file():
raise _PatchError(f"path to update is not a file: {path}")
uses_crlf = "\r\n" in content
norm_content = content.replace("\r\n", "\n")
norm_old = old_text.replace("\r\n", "\n")
pos = norm_content.find(norm_old)
if pos < 0:
raise _PatchError(f"old_text not found in {path}")
if norm_content.find(norm_old, pos + 1) >= 0:
raise _PatchError(f"old_text appears multiple times in {path}")
if norm_old == norm_content:
deletes.add(source)
writes.pop(source, None)
added, deleted = 0, _text_line_count(content)
summaries.append(
_PatchSummary(
action="delete", path=path, added=added, deleted=deleted
)
)
else:
new_norm = (
norm_content[:pos] + norm_content[pos + len(norm_old) :]
)
if new_norm and not new_norm.endswith("\n"):
new_norm += "\n"
if uses_crlf:
new_norm = new_norm.replace("\n", "\r\n")
writes[source] = new_norm
deletes.discard(source)
added, deleted = _line_diff_stats(content, new_norm)
summaries.append(
_PatchSummary(
action="update", path=path, added=added, deleted=deleted
)
)
else: else:
raise _PatchError(f"unknown action: {action}") raise _PatchError(f"unknown action: {action}")
@@ -260,10 +319,13 @@ class ApplyPatchTool(_FsTool):
) )
backups: dict[Path, bytes | None] = {} backups: dict[Path, bytes | None] = {}
for path in writes: for path in set(writes) | deletes:
backups[path] = path.read_bytes() if path.exists() else None backups[path] = path.read_bytes() if path.exists() else None
try: try:
for path in deletes:
if path.exists():
path.unlink()
for path, content in writes.items(): for path, content in writes.items():
path.parent.mkdir(parents=True, exist_ok=True) path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8", newline="") path.write_text(content, encoding="utf-8", newline="")
@@ -277,7 +339,7 @@ class ApplyPatchTool(_FsTool):
path.write_bytes(data) path.write_bytes(data)
raise raise
for path in writes: for path in set(writes) | deletes:
self._file_states.record_write(path) self._file_states.record_write(path)
return "Patch applied:\n" + "\n".join( return "Patch applied:\n" + "\n".join(
_format_summary(summary) for summary in summaries _format_summary(summary) for summary in summaries
+2 -8
View File
@@ -9,7 +9,6 @@ from pydantic import Field
from nanobot.agent.tools.base import Tool, tool_parameters from nanobot.agent.tools.base import Tool, tool_parameters
from nanobot.agent.tools.schema import ArraySchema, BooleanSchema, IntegerSchema, StringSchema, tool_parameters_schema from nanobot.agent.tools.schema import ArraySchema, BooleanSchema, IntegerSchema, StringSchema, tool_parameters_schema
from nanobot.security.workspace_access import current_tool_workspace
from nanobot.apps.cli import CliAppError, CliAppManager, CliAppsRuntimeConfig from nanobot.apps.cli import CliAppError, CliAppManager, CliAppsRuntimeConfig
from nanobot.config.schema import Base from nanobot.config.schema import Base
@@ -114,12 +113,7 @@ class CliAppsTool(Tool):
working_dir: str | None = None, working_dir: str | None = None,
timeout: int | None = None, timeout: int | None = None,
) -> str: ) -> str:
access = current_tool_workspace( manager = CliAppManager(workspace=self.workspace, runtime=self.runtime)
self.workspace,
restrict_to_workspace=self.restrict_to_workspace,
)
workspace = access.project_path or self.workspace
manager = CliAppManager(workspace=workspace, runtime=self.runtime)
try: try:
return manager.run( return manager.run(
name, name,
@@ -127,7 +121,7 @@ class CliAppsTool(Tool):
json_output=bool(json), json_output=bool(json),
working_dir=working_dir, working_dir=working_dir,
timeout=timeout, timeout=timeout,
restrict_to_workspace=access.restrict_to_workspace, restrict_to_workspace=self.restrict_to_workspace,
) )
except CliAppError as exc: except CliAppError as exc:
return f"Error: {exc.message}" return f"Error: {exc.message}"
-24
View File
@@ -1,15 +1,9 @@
"""Runtime context for tool construction.""" """Runtime context for tool construction."""
from __future__ import annotations from __future__ import annotations
from contextvars import ContextVar, Token
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Any, Callable, Protocol, runtime_checkable from typing import Any, Callable, Protocol, runtime_checkable
_CURRENT_REQUEST_CONTEXT: ContextVar["RequestContext | None"] = ContextVar(
"nanobot_tool_request_context",
default=None,
)
@dataclass(frozen=True) @dataclass(frozen=True)
class RequestContext: class RequestContext:
@@ -27,23 +21,6 @@ class ContextAware(Protocol):
... ...
def bind_request_context(ctx: RequestContext) -> Token[RequestContext | None]:
return _CURRENT_REQUEST_CONTEXT.set(ctx)
def reset_request_context(token: Token[RequestContext | None]) -> None:
_CURRENT_REQUEST_CONTEXT.reset(token)
def current_request_context() -> RequestContext | None:
return _CURRENT_REQUEST_CONTEXT.get()
def current_request_session_key() -> str | None:
ctx = current_request_context()
return ctx.session_key if ctx else None
@dataclass @dataclass
class ToolContext: class ToolContext:
config: Any config: Any
@@ -56,4 +33,3 @@ class ToolContext:
provider_snapshot_loader: Callable[[], Any] | None = None provider_snapshot_loader: Callable[[], Any] | None = None
image_generation_provider_configs: dict[str, Any] | None = None image_generation_provider_configs: dict[str, Any] | None = None
timezone: str = "UTC" timezone: str = "UTC"
workspace_sandbox: Any | None = None
+26 -32
View File
@@ -3,6 +3,7 @@
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
import shutil
import time import time
import uuid import uuid
from contextlib import suppress from contextlib import suppress
@@ -10,13 +11,8 @@ from dataclasses import dataclass
from typing import Any from typing import Any
from nanobot.agent.tools.base import Tool, tool_parameters from nanobot.agent.tools.base import Tool, tool_parameters
from nanobot.agent.tools.context import current_request_session_key from nanobot.agent.tools.schema import BooleanSchema, IntegerSchema, StringSchema, tool_parameters_schema
from nanobot.agent.tools.schema import (
BooleanSchema,
IntegerSchema,
StringSchema,
tool_parameters_schema,
)
DEFAULT_YIELD_MS = 1000 DEFAULT_YIELD_MS = 1000
MAX_YIELD_MS = 30_000 MAX_YIELD_MS = 30_000
@@ -47,7 +43,6 @@ class ExecSessionInfo:
idle_s: float idle_s: float
remaining_s: float remaining_s: float
returncode: int | None returncode: int | None
owner_session_key: str | None = None
class _ExecSession: class _ExecSession:
@@ -59,13 +54,11 @@ class _ExecSession:
command: str, command: str,
cwd: str, cwd: str,
timeout: int | None, timeout: int | None,
owner_session_key: str | None = None,
) -> None: ) -> None:
self.session_id = session_id self.session_id = session_id
self.process = process self.process = process
self.command = command self.command = command
self.cwd = cwd self.cwd = cwd
self.owner_session_key = owner_session_key
self.started_at = time.monotonic() self.started_at = time.monotonic()
# timeout None/0 means no limit; an infinite deadline is never reached. # timeout None/0 means no limit; an infinite deadline is never reached.
self.deadline = time.monotonic() + timeout if timeout else float("inf") self.deadline = time.monotonic() + timeout if timeout else float("inf")
@@ -182,7 +175,6 @@ class ExecSessionManager:
login: bool, login: bool,
yield_time_ms: int, yield_time_ms: int,
max_output_chars: int, max_output_chars: int,
owner_session_key: str | None = None,
) -> tuple[str, _SessionPoll]: ) -> tuple[str, _SessionPoll]:
async with self._lock: async with self._lock:
await self._cleanup_locked() await self._cleanup_locked()
@@ -196,7 +188,6 @@ class ExecSessionManager:
command=command, command=command,
cwd=cwd, cwd=cwd,
timeout=timeout, timeout=timeout,
owner_session_key=owner_session_key,
) )
self._sessions[session_id] = session self._sessions[session_id] = session
@@ -215,19 +206,12 @@ class ExecSessionManager:
terminate: bool, terminate: bool,
yield_time_ms: int, yield_time_ms: int,
max_output_chars: int, max_output_chars: int,
owner_session_key: str | None = None,
) -> _SessionPoll: ) -> _SessionPoll:
async with self._lock: async with self._lock:
await self._cleanup_locked() await self._cleanup_locked()
session = self._sessions.get(session_id) session = self._sessions.get(session_id)
if session is None: if session is None:
raise KeyError(session_id) raise KeyError(session_id)
if (
owner_session_key
and session.owner_session_key
and session.owner_session_key != owner_session_key
):
raise KeyError(session_id)
if chars: if chars:
error = await session.write(chars) error = await session.write(chars)
@@ -252,7 +236,7 @@ class ExecSessionManager:
self._sessions.pop(session_id, None) self._sessions.pop(session_id, None)
return poll return poll
async def list(self, *, owner_session_key: str | None = None) -> list[ExecSessionInfo]: async def list(self) -> list[ExecSessionInfo]:
async with self._lock: async with self._lock:
await self._cleanup_locked() await self._cleanup_locked()
now = time.monotonic() now = time.monotonic()
@@ -265,12 +249,8 @@ class ExecSessionManager:
idle_s=max(0.0, now - session.last_access), idle_s=max(0.0, now - session.last_access),
remaining_s=max(0.0, session.deadline - now), remaining_s=max(0.0, session.deadline - now),
returncode=session.process.returncode, returncode=session.process.returncode,
owner_session_key=session.owner_session_key,
) )
for session_id, session in sorted(self._sessions.items()) for session_id, session in sorted(self._sessions.items())
if not owner_session_key
or not session.owner_session_key
or session.owner_session_key == owner_session_key
] ]
async def _cleanup_locked(self) -> None: async def _cleanup_locked(self) -> None:
@@ -292,11 +272,29 @@ class ExecSessionManager:
shell_program: str | None, shell_program: str | None,
login: bool, login: bool,
) -> asyncio.subprocess.Process: ) -> asyncio.subprocess.Process:
from nanobot.agent.tools.shell import ExecTool from nanobot.agent.tools import shell
return await ExecTool._spawn( if shell._IS_WINDOWS:
command, cwd, env, shell_program, login, return await asyncio.create_subprocess_shell(
command,
stdin=asyncio.subprocess.PIPE,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
cwd=cwd,
env=env,
)
shell_program = shell_program or shutil.which("bash") or "/bin/bash"
args = [shell_program]
if login and shell_program.rsplit("/", 1)[-1] in {"bash", "zsh"}:
args.append("-l")
args.extend(["-c", command])
return await asyncio.create_subprocess_exec(
*args,
stdin=asyncio.subprocess.PIPE, stdin=asyncio.subprocess.PIPE,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
cwd=cwd,
env=env,
) )
@@ -479,7 +477,6 @@ class WriteStdinTool(Tool):
terminate=terminate, terminate=terminate,
yield_time_ms=clamp_session_int(yield_time_ms, DEFAULT_YIELD_MS, 0, MAX_YIELD_MS), yield_time_ms=clamp_session_int(yield_time_ms, DEFAULT_YIELD_MS, 0, MAX_YIELD_MS),
max_output_chars=output_limit, max_output_chars=output_limit,
owner_session_key=current_request_session_key(),
) )
return format_session_poll(session_id, poll) return format_session_poll(session_id, poll)
except KeyError: except KeyError:
@@ -513,7 +510,6 @@ class WriteStdinTool(Tool):
terminate=terminate if first else False, terminate=terminate if first else False,
yield_time_ms=step_ms, yield_time_ms=step_ms,
max_output_chars=max_output_chars, max_output_chars=max_output_chars,
owner_session_key=current_request_session_key(),
) )
first = False first = False
if poll.output: if poll.output:
@@ -577,9 +573,7 @@ class ListExecSessionsTool(Tool):
async def execute(self, **kwargs: Any) -> str: async def execute(self, **kwargs: Any) -> str:
try: try:
sessions = await self._manager.list( sessions = await self._manager.list()
owner_session_key=current_request_session_key(),
)
if not sessions: if not sessions:
return "No active exec sessions." return "No active exec sessions."
lines = [] lines = []
+3 -23
View File
@@ -10,7 +10,6 @@ from typing import Any
from nanobot.agent.tools.base import Tool, tool_parameters from nanobot.agent.tools.base import Tool, tool_parameters
from nanobot.agent.tools.file_state import FileStates, _hash_file, current_file_states from nanobot.agent.tools.file_state import FileStates, _hash_file, current_file_states
from nanobot.agent.tools.path_utils import resolve_workspace_path from nanobot.agent.tools.path_utils import resolve_workspace_path
from nanobot.security.workspace_access import current_tool_workspace
from nanobot.agent.tools.schema import ( from nanobot.agent.tools.schema import (
BooleanSchema, BooleanSchema,
IntegerSchema, IntegerSchema,
@@ -29,18 +28,10 @@ class _FsTool(Tool):
allowed_dir: Path | None = None, allowed_dir: Path | None = None,
extra_allowed_dirs: list[Path] | None = None, extra_allowed_dirs: list[Path] | None = None,
file_states: FileStates | None = None, file_states: FileStates | None = None,
restrict_to_workspace: bool | None = None,
sandbox_restricts_workspace: bool = False,
): ):
self._workspace = workspace self._workspace = workspace
self._allowed_dir = allowed_dir self._allowed_dir = allowed_dir
self._extra_allowed_dirs = extra_allowed_dirs self._extra_allowed_dirs = extra_allowed_dirs
self._restrict_to_workspace = (
bool(restrict_to_workspace)
if restrict_to_workspace is not None
else allowed_dir is not None
)
self._sandbox_restricts_workspace = sandbox_restricts_workspace
# Explicit state is used by isolated runners like Dream/subagents. # Explicit state is used by isolated runners like Dream/subagents.
# Main AgentLoop tools leave this unset and resolve state from the # Main AgentLoop tools leave this unset and resolve state from the
# current async task, which keeps shared tool instances session-safe. # current async task, which keeps shared tool instances session-safe.
@@ -55,16 +46,13 @@ class _FsTool(Tool):
ctx.config.restrict_to_workspace ctx.config.restrict_to_workspace
or ctx.config.exec.sandbox or ctx.config.exec.sandbox
) )
sandbox_restricts = bool(ctx.config.exec.sandbox)
allowed_dir = Path(ctx.workspace) if restrict else None allowed_dir = Path(ctx.workspace) if restrict else None
extra_read = [BUILTIN_SKILLS_DIR] extra_read = [BUILTIN_SKILLS_DIR] if allowed_dir else None
return cls( return cls(
workspace=Path(ctx.workspace), workspace=Path(ctx.workspace),
allowed_dir=allowed_dir, allowed_dir=allowed_dir,
extra_allowed_dirs=extra_read, extra_allowed_dirs=extra_read,
file_states=ctx.file_state_store, file_states=ctx.file_state_store,
restrict_to_workspace=ctx.config.restrict_to_workspace,
sandbox_restricts_workspace=sandbox_restricts,
) )
@property @property
@@ -74,21 +62,13 @@ class _FsTool(Tool):
return current_file_states(self._fallback_file_states) return current_file_states(self._fallback_file_states)
def _resolve(self, path: str) -> Path: def _resolve(self, path: str) -> Path:
access = current_tool_workspace(
self._workspace,
restrict_to_workspace=self._restrict_to_workspace,
sandbox_restricts_workspace=self._sandbox_restricts_workspace,
)
return resolve_workspace_path( return resolve_workspace_path(
path, path,
access.project_path, self._workspace,
access.allowed_root, self._allowed_dir,
self._extra_allowed_dirs, self._extra_allowed_dirs,
) )
def _display_workspace(self) -> Path | None:
return current_tool_workspace(self._workspace).project_path
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# read_file # read_file
+17 -15
View File
@@ -14,7 +14,6 @@ from nanobot.agent.tools.schema import (
StringSchema, StringSchema,
tool_parameters_schema, tool_parameters_schema,
) )
from nanobot.security.workspace_access import current_tool_workspace
from nanobot.config.paths import get_media_dir from nanobot.config.paths import get_media_dir
from nanobot.config.schema import Base from nanobot.config.schema import Base
from nanobot.providers.image_generation import ( from nanobot.providers.image_generation import (
@@ -22,7 +21,6 @@ from nanobot.providers.image_generation import (
ImageGenerationProvider, ImageGenerationProvider,
get_image_gen_provider, get_image_gen_provider,
) )
from nanobot.security.workspace_policy import WorkspaceBoundaryError, resolve_allowed_path
from nanobot.utils.artifacts import ( from nanobot.utils.artifacts import (
ArtifactError, ArtifactError,
generated_image_tool_result, generated_image_tool_result,
@@ -133,22 +131,18 @@ class ImageGenerationTool(Tool):
return cls(**kwargs) return cls(**kwargs)
def _resolve_reference_image(self, value: str) -> str: def _resolve_reference_image(self, value: str) -> str:
access = current_tool_workspace(self.workspace, restrict_to_workspace=True) raw_path = Path(value).expanduser()
workspace = access.project_path or self.workspace path = raw_path if raw_path.is_absolute() else self.workspace / raw_path
try: try:
resolved = resolve_allowed_path( resolved = path.resolve(strict=True)
value,
workspace=workspace,
allowed_root=access.allowed_root,
extra_allowed_roots=[get_media_dir()] if access.allowed_root is not None else None,
strict=True,
)
except WorkspaceBoundaryError as exc:
raise ImageGenerationError(
"reference_images must be inside the workspace or nanobot media directory"
) from exc
except OSError as exc: except OSError as exc:
raise ImageGenerationError(f"reference image not found: {value}") from exc raise ImageGenerationError(f"reference image not found: {value}") from exc
allowed_roots = [self.workspace.resolve(), get_media_dir().resolve()]
if not any(_is_relative_to(resolved, root) for root in allowed_roots):
raise ImageGenerationError(
"reference_images must be inside the workspace or nanobot media directory"
)
if not resolved.is_file(): if not resolved.is_file():
raise ImageGenerationError(f"reference image is not a file: {value}") raise ImageGenerationError(f"reference image is not a file: {value}")
raw = resolved.read_bytes() raw = resolved.read_bytes()
@@ -207,3 +201,11 @@ class ImageGenerationTool(Tool):
return generated_image_tool_result(artifacts) return generated_image_tool_result(artifacts)
except (ArtifactError, ImageGenerationError, OSError) as exc: except (ArtifactError, ImageGenerationError, OSError) as exc:
return f"Error: {exc}" return f"Error: {exc}"
def _is_relative_to(path: Path, root: Path) -> bool:
try:
path.relative_to(root)
except ValueError:
return False
return True
+6 -13
View File
@@ -16,7 +16,6 @@ There is **no** sub-agent orchestrator and **no** special WebSocket ``agent_ui``
from __future__ import annotations from __future__ import annotations
from contextvars import ContextVar
from datetime import datetime from datetime import datetime
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
@@ -46,22 +45,15 @@ class _GoalToolsMixin(ContextAware):
def __init__(self, sessions: SessionManager, bus: Any | None = None) -> None: def __init__(self, sessions: SessionManager, bus: Any | None = None) -> None:
self._sessions = sessions self._sessions = sessions
self._bus = bus self._bus = bus
# Each subclass gets its own ContextVar so concurrent tasks across self._request_ctx: RequestContext | None = None
# different tool types (LongTaskTool vs CompleteGoalTool) do not
# interfere with each other.
self._request_ctx: ContextVar[RequestContext | None] = ContextVar(
f"{self.__class__.__name__}_request_ctx",
default=None,
)
def set_context(self, ctx: RequestContext) -> None: def set_context(self, ctx: RequestContext) -> None:
self._request_ctx.set(ctx) self._request_ctx = ctx
def _session(self): def _session(self):
request_ctx = self._request_ctx.get() if self._request_ctx is None:
if request_ctx is None:
return None return None
key = request_ctx.session_key key = self._request_ctx.session_key
if not key: if not key:
return None return None
return self._sessions.get_or_create(key) return self._sessions.get_or_create(key)
@@ -69,7 +61,7 @@ class _GoalToolsMixin(ContextAware):
async def _publish_goal_state_ws(self, metadata: dict[str, Any]) -> None: async def _publish_goal_state_ws(self, metadata: dict[str, Any]) -> None:
"""Fan-out authoritative goal snapshot for this WebSocket chat only.""" """Fan-out authoritative goal snapshot for this WebSocket chat only."""
bus = self._bus bus = self._bus
rc = self._request_ctx.get() rc = self._request_ctx
if bus is None or rc is None or rc.channel != "websocket": if bus is None or rc is None or rc.channel != "websocket":
return return
cid = (rc.chat_id or "").strip() cid = (rc.chat_id or "").strip()
@@ -232,3 +224,4 @@ class CompleteGoalTool(Tool, _GoalToolsMixin):
if tail: if tail:
return f"Goal marked complete ({ended}). Recap:\n{tail}" return f"Goal marked complete ({ended}). Recap:\n{tail}"
return f"Goal marked complete ({ended})." return f"Goal marked complete ({ended})."
+4 -24
View File
@@ -8,7 +8,6 @@ from nanobot.agent.tools.base import Tool, tool_parameters
from nanobot.agent.tools.context import ContextAware, RequestContext from nanobot.agent.tools.context import ContextAware, RequestContext
from nanobot.agent.tools.path_utils import resolve_workspace_path from nanobot.agent.tools.path_utils import resolve_workspace_path
from nanobot.agent.tools.schema import ArraySchema, StringSchema, tool_parameters_schema from nanobot.agent.tools.schema import ArraySchema, StringSchema, tool_parameters_schema
from nanobot.security.workspace_access import current_tool_workspace
from nanobot.bus.events import OutboundMessage from nanobot.bus.events import OutboundMessage
from nanobot.config.paths import get_workspace_path from nanobot.config.paths import get_workspace_path
@@ -83,10 +82,6 @@ class MessageTool(Tool, ContextAware):
"message_record_channel_delivery", "message_record_channel_delivery",
default=False, default=False,
) )
self._suppress_delivery_var: ContextVar[bool] = ContextVar(
"message_suppress_delivery",
default=False,
)
@classmethod @classmethod
def create(cls, ctx: Any) -> Tool: def create(cls, ctx: Any) -> Tool:
@@ -125,14 +120,6 @@ class MessageTool(Tool, ContextAware):
"""Restore previous proactive delivery recording state.""" """Restore previous proactive delivery recording state."""
self._record_channel_delivery_var.reset(token) self._record_channel_delivery_var.reset(token)
def set_suppress_delivery(self, active: bool):
"""Temporarily suppress real channel delivery for internal checks."""
return self._suppress_delivery_var.set(active)
def reset_suppress_delivery(self, token) -> None:
"""Restore previous channel delivery suppression state."""
self._suppress_delivery_var.reset(token)
@property @property
def _sent_in_turn(self) -> bool: def _sent_in_turn(self) -> bool:
return self._sent_in_turn_var.get() return self._sent_in_turn_var.get()
@@ -162,19 +149,15 @@ class MessageTool(Tool, ContextAware):
def _resolve_media(self, media: list[str]) -> list[str]: def _resolve_media(self, media: list[str]) -> list[str]:
"""Resolve local media attachments and enforce workspace restriction when enabled.""" """Resolve local media attachments and enforce workspace restriction when enabled."""
resolved: list[str] = [] resolved: list[str] = []
access = current_tool_workspace( allowed_dir = self._workspace if self._restrict_to_workspace else None
self._workspace,
restrict_to_workspace=self._restrict_to_workspace,
)
workspace = access.project_path or self._workspace
for p in media: for p in media:
if p.startswith(("http://", "https://")): if p.startswith(("http://", "https://")):
resolved.append(p) resolved.append(p)
elif not access.restrict_to_workspace: elif not self._restrict_to_workspace:
path = Path(p).expanduser() path = Path(p).expanduser()
resolved.append(p if path.is_absolute() else str(workspace / path)) resolved.append(p if path.is_absolute() else str(self._workspace / path))
else: else:
resolved.append(str(resolve_workspace_path(p, workspace, access.allowed_root))) resolved.append(str(resolve_workspace_path(p, self._workspace, allowed_dir)))
return resolved return resolved
async def execute( async def execute(
@@ -229,9 +212,6 @@ class MessageTool(Tool, ContextAware):
if not channel or not chat_id: if not channel or not chat_id:
return "Error: No target channel/chat specified" return "Error: No target channel/chat specified"
if self._suppress_delivery_var.get():
return "Message suppressed during internal check"
if not self._send_callback: if not self._send_callback:
return "Error: Message sending not configured" return "Error: Message sending not configured"
+23 -11
View File
@@ -3,15 +3,21 @@
from pathlib import Path from pathlib import Path
from nanobot.config.paths import get_media_dir from nanobot.config.paths import get_media_dir
from nanobot.security.workspace_policy import (
is_path_within, WORKSPACE_BOUNDARY_NOTE = (
resolve_allowed_path, " (this is a hard policy boundary, not a transient failure; "
"do not retry with shell tricks or alternative tools, and ask "
"the user how to proceed if the resource is genuinely required)"
) )
def is_under(path: Path, directory: Path) -> bool: def is_under(path: Path, directory: Path) -> bool:
"""Return True when path resolves under directory.""" """Return True when path resolves under directory."""
return is_path_within(path, directory) try:
path.relative_to(directory.resolve())
return True
except ValueError:
return False
def resolve_workspace_path( def resolve_workspace_path(
@@ -21,10 +27,16 @@ def resolve_workspace_path(
extra_allowed_dirs: list[Path] | None = None, extra_allowed_dirs: list[Path] | None = None,
) -> Path: ) -> Path:
"""Resolve path against workspace and enforce allowed directory containment.""" """Resolve path against workspace and enforce allowed directory containment."""
extra_roots = [get_media_dir(), *(extra_allowed_dirs or [])] if allowed_dir else None p = Path(path).expanduser()
return resolve_allowed_path( if not p.is_absolute() and workspace:
path, p = workspace / p
workspace=workspace, resolved = p.resolve()
allowed_root=allowed_dir, if allowed_dir:
extra_allowed_roots=extra_roots, media_path = get_media_dir().resolve()
) all_dirs = [allowed_dir, media_path, *(extra_allowed_dirs or [])]
if not any(is_under(resolved, d) for d in all_dirs):
raise PermissionError(
f"Path {path} is outside allowed directory {allowed_dir}"
+ WORKSPACE_BOUNDARY_NOTE
)
return resolved
-3
View File
@@ -42,9 +42,6 @@ class RuntimeState(Protocol):
@property @property
def exec_config(self) -> Any: ... def exec_config(self) -> Any: ...
@property
def workspace_sandbox(self) -> Any: ...
@property @property
def subagents(self) -> Any: ... def subagents(self) -> Any: ...
+2 -3
View File
@@ -101,10 +101,9 @@ class _SearchTool(_FsTool):
_IGNORE_DIRS = set(ListDirTool._IGNORE_DIRS) _IGNORE_DIRS = set(ListDirTool._IGNORE_DIRS)
def _display_path(self, target: Path, root: Path) -> str: def _display_path(self, target: Path, root: Path) -> str:
workspace = self._display_workspace() if self._workspace:
if workspace:
with suppress(ValueError): with suppress(ValueError):
return target.relative_to(workspace).as_posix() return target.relative_to(self._workspace).as_posix()
return target.relative_to(root).as_posix() return target.relative_to(root).as_posix()
def _iter_files(self, root: Path) -> Iterable[Path]: def _iter_files(self, root: Path) -> Iterable[Path]:
+6 -15
View File
@@ -3,18 +3,16 @@
from __future__ import annotations from __future__ import annotations
import time import time
from typing import TYPE_CHECKING, Any from typing import Any
from loguru import logger from loguru import logger
from nanobot.agent.subagent import SubagentStatus
from nanobot.agent.tools.base import Tool from nanobot.agent.tools.base import Tool
from nanobot.agent.tools.context import ContextAware, RequestContext from nanobot.agent.tools.context import ContextAware, RequestContext
from nanobot.agent.tools.runtime_state import RuntimeState from nanobot.agent.tools.runtime_state import RuntimeState
from nanobot.config.schema import Base from nanobot.config.schema import Base
if TYPE_CHECKING:
from nanobot.agent.subagent import SubagentStatus
class MyToolConfig(Base): class MyToolConfig(Base):
"""Self-inspection tool configuration.""" """Self-inspection tool configuration."""
@@ -35,12 +33,6 @@ def _has_real_attr(obj: Any, key: str) -> bool:
return False return False
def _is_subagent_status(value: Any) -> bool:
from nanobot.agent.subagent import SubagentStatus
return isinstance(value, SubagentStatus)
class MyTool(Tool, ContextAware): class MyTool(Tool, ContextAware):
"""Check and set the agent loop's runtime configuration.""" """Check and set the agent loop's runtime configuration."""
@@ -76,7 +68,6 @@ class MyTool(Tool, ContextAware):
"_current_iteration", # updated by runner only "_current_iteration", # updated by runner only
"exec_config", # inspect allowed (e.g. check sandbox), modify blocked "exec_config", # inspect allowed (e.g. check sandbox), modify blocked
"web_config", # inspect allowed (e.g. check enable), modify blocked "web_config", # inspect allowed (e.g. check enable), modify blocked
"workspace_sandbox", # read-only view of workspace enforcement level
}) })
_DENIED_ATTRS = frozenset({ _DENIED_ATTRS = frozenset({
@@ -223,7 +214,7 @@ class MyTool(Tool, ContextAware):
# ------------------------------------------------------------------ # ------------------------------------------------------------------
@staticmethod @staticmethod
def _format_status(st: "SubagentStatus", indent: str = " ") -> str: def _format_status(st: SubagentStatus, indent: str = " ") -> str:
elapsed = time.monotonic() - st.started_at elapsed = time.monotonic() - st.started_at
tool_summary = ", ".join( tool_summary = ", ".join(
f"{e.get('name', '?')}({e.get('status', '?')})" for e in st.tool_events[-5:] f"{e.get('name', '?')}({e.get('status', '?')})" for e in st.tool_events[-5:]
@@ -241,14 +232,14 @@ class MyTool(Tool, ContextAware):
@staticmethod @staticmethod
def _format_value(val: Any, key: str = "") -> str: def _format_value(val: Any, key: str = "") -> str:
if _is_subagent_status(val): if isinstance(val, SubagentStatus):
header = f"Subagent [{val.task_id}] '{val.label}'" header = f"Subagent [{val.task_id}] '{val.label}'"
detail = MyTool._format_status(val, " ") detail = MyTool._format_status(val, " ")
return f"{header}\n task: {val.task_description}\n{detail}" return f"{header}\n task: {val.task_description}\n{detail}"
# SubagentManager: delegate to its _task_statuses dict # SubagentManager: delegate to its _task_statuses dict
if hasattr(val, "_task_statuses") and isinstance(val._task_statuses, dict): if hasattr(val, "_task_statuses") and isinstance(val._task_statuses, dict):
return MyTool._format_value(val._task_statuses, key) return MyTool._format_value(val._task_statuses, key)
if isinstance(val, dict) and val and _is_subagent_status(next(iter(val.values()))): if isinstance(val, dict) and val and isinstance(next(iter(val.values())), SubagentStatus):
prefix = f"{key}: " if key else "" prefix = f"{key}: " if key else ""
lines = [f"{prefix}{len(val)} subagent(s):"] lines = [f"{prefix}{len(val)} subagent(s):"]
for tid, st in val.items(): for tid, st in val.items():
@@ -358,7 +349,7 @@ class MyTool(Tool, ContextAware):
parts.append(self._format_value(getattr(state, k, None), k)) parts.append(self._format_value(getattr(state, k, None), k))
parts.append(self._format_value(state.model_preset, "model_preset")) parts.append(self._format_value(state.model_preset, "model_preset"))
# Other useful top-level keys shown in description # Other useful top-level keys shown in description
for k in ("workspace", "provider_retry_mode", "max_tool_result_chars", "_current_iteration", "web_config", "exec_config", "workspace_sandbox", "subagents"): for k in ("workspace", "provider_retry_mode", "max_tool_result_chars", "_current_iteration", "web_config", "exec_config", "subagents"):
if _has_real_attr(state, k): if _has_real_attr(state, k):
parts.append(self._format_value(getattr(state, k, None), k)) parts.append(self._format_value(getattr(state, k, None), k))
# Token usage # Token usage
+22 -64
View File
@@ -16,27 +16,19 @@ from loguru import logger
from pydantic import Field from pydantic import Field
from nanobot.agent.tools.base import Tool, tool_parameters from nanobot.agent.tools.base import Tool, tool_parameters
from nanobot.agent.tools.context import current_request_session_key
from nanobot.agent.tools.exec_session import ( from nanobot.agent.tools.exec_session import (
DEFAULT_EXEC_SESSION_MANAGER,
DEFAULT_MAX_OUTPUT_CHARS, DEFAULT_MAX_OUTPUT_CHARS,
DEFAULT_YIELD_MS, DEFAULT_YIELD_MS,
DEFAULT_EXEC_SESSION_MANAGER,
MAX_OUTPUT_CHARS, MAX_OUTPUT_CHARS,
MAX_YIELD_MS, MAX_YIELD_MS,
clamp_session_int, clamp_session_int,
format_session_poll, format_session_poll,
) )
from nanobot.agent.tools.sandbox import wrap_command from nanobot.agent.tools.sandbox import wrap_command
from nanobot.agent.tools.schema import ( from nanobot.agent.tools.schema import BooleanSchema, IntegerSchema, StringSchema, tool_parameters_schema
BooleanSchema,
IntegerSchema,
StringSchema,
tool_parameters_schema,
)
from nanobot.config.paths import get_media_dir from nanobot.config.paths import get_media_dir
from nanobot.config.schema import Base from nanobot.config.schema import Base
from nanobot.security.workspace_access import current_scope_allows_loopback, current_tool_workspace
from nanobot.security.workspace_policy import is_path_within
_IS_WINDOWS = sys.platform == "win32" _IS_WINDOWS = sys.platform == "win32"
@@ -148,7 +140,6 @@ class ExecTool(Tool):
working_dir=ctx.workspace, working_dir=ctx.workspace,
timeout=cfg.timeout, timeout=cfg.timeout,
restrict_to_workspace=ctx.config.restrict_to_workspace, restrict_to_workspace=ctx.config.restrict_to_workspace,
webui_allow_local_service_access=ctx.config.webui_allow_local_service_access,
sandbox=cfg.sandbox, sandbox=cfg.sandbox,
path_append=cfg.path_append, path_append=cfg.path_append,
allowed_env_keys=cfg.allowed_env_keys, allowed_env_keys=cfg.allowed_env_keys,
@@ -163,8 +154,6 @@ class ExecTool(Tool):
deny_patterns: list[str] | None = None, deny_patterns: list[str] | None = None,
allow_patterns: list[str] | None = None, allow_patterns: list[str] | None = None,
restrict_to_workspace: bool = False, restrict_to_workspace: bool = False,
webui_allow_local_service_access: bool = True,
allow_local_preview_access: bool | None = None,
sandbox: str = "", sandbox: str = "",
path_append: str = "", path_append: str = "",
allowed_env_keys: list[str] | None = None, allowed_env_keys: list[str] | None = None,
@@ -194,9 +183,6 @@ class ExecTool(Tool):
] ]
self.allow_patterns = allow_patterns or [] self.allow_patterns = allow_patterns or []
self.restrict_to_workspace = restrict_to_workspace self.restrict_to_workspace = restrict_to_workspace
if allow_local_preview_access is not None:
webui_allow_local_service_access = allow_local_preview_access
self.webui_allow_local_service_access = webui_allow_local_service_access
self.path_append = path_append self.path_append = path_append
self.allowed_env_keys = allowed_env_keys or [] self.allowed_env_keys = allowed_env_keys or []
self._session_manager = session_manager or DEFAULT_EXEC_SESSION_MANAGER self._session_manager = session_manager or DEFAULT_EXEC_SESSION_MANAGER
@@ -327,7 +313,6 @@ class ExecTool(Tool):
shell_program=prepared.shell_program, shell_program=prepared.shell_program,
login=prepared.login, login=prepared.login,
yield_time_ms=clamp_session_int(yield_time_ms, DEFAULT_YIELD_MS, 0, MAX_YIELD_MS), yield_time_ms=clamp_session_int(yield_time_ms, DEFAULT_YIELD_MS, 0, MAX_YIELD_MS),
owner_session_key=current_request_session_key(),
max_output_chars=clamp_session_int( max_output_chars=clamp_session_int(
max_output_chars, max_output_chars,
DEFAULT_MAX_OUTPUT_CHARS, DEFAULT_MAX_OUTPUT_CHARS,
@@ -361,39 +346,29 @@ class ExecTool(Tool):
shell: str | None = None, shell: str | None = None,
login: bool | None = None, login: bool | None = None,
) -> _PreparedCommand | str: ) -> _PreparedCommand | str:
access = current_tool_workspace( cwd = working_dir or self.working_dir or os.getcwd()
self.working_dir,
restrict_to_workspace=self.restrict_to_workspace,
sandbox_restricts_workspace=bool(self.sandbox),
)
workspace_root = str(access.project_path) if access.project_path is not None else self.working_dir
cwd = working_dir or workspace_root or os.getcwd()
# Prevent an LLM-supplied working_dir from escaping the configured # Prevent an LLM-supplied working_dir from escaping the configured
# workspace when restrict_to_workspace is enabled (#2826). Without # workspace when restrict_to_workspace is enabled (#2826). Without
# this, a caller can pass working_dir="/etc" and then all absolute # this, a caller can pass working_dir="/etc" and then all absolute
# paths under /etc would pass the _guard_command check that anchors # paths under /etc would pass the _guard_command check that anchors
# on cwd. # on cwd.
if access.restrict_to_workspace and workspace_root: if self.restrict_to_workspace and self.working_dir:
try: try:
requested = Path(cwd).expanduser().resolve() requested = Path(cwd).expanduser().resolve()
resolved_root = Path(workspace_root).expanduser().resolve() workspace_root = Path(self.working_dir).expanduser().resolve()
except Exception: except Exception:
return ( return (
"Error: working_dir could not be resolved" "Error: working_dir could not be resolved"
+ _WORKSPACE_BOUNDARY_NOTE + _WORKSPACE_BOUNDARY_NOTE
) )
if not is_path_within(requested, resolved_root): if requested != workspace_root and workspace_root not in requested.parents:
return ( return (
"Error: working_dir is outside the configured workspace" "Error: working_dir is outside the configured workspace"
+ _WORKSPACE_BOUNDARY_NOTE + _WORKSPACE_BOUNDARY_NOTE
) )
guard_error = self._guard_command( guard_error = self._guard_command(command, cwd)
command,
cwd,
restrict_to_workspace=access.restrict_to_workspace,
)
if guard_error: if guard_error:
return guard_error return guard_error
@@ -404,7 +379,7 @@ class ExecTool(Tool):
self.sandbox, self.sandbox,
) )
else: else:
workspace = workspace_root or cwd workspace = self.working_dir or cwd
command = wrap_command(self.sandbox, command, workspace, cwd) command = wrap_command(self.sandbox, command, workspace, cwd)
cwd = str(Path(workspace).resolve()) cwd = str(Path(workspace).resolve())
@@ -436,23 +411,16 @@ class ExecTool(Tool):
command: str, cwd: str, env: dict[str, str], command: str, cwd: str, env: dict[str, str],
shell_program: str | None = None, shell_program: str | None = None,
login: bool = True, login: bool = True,
*,
stdin: int = asyncio.subprocess.DEVNULL,
) -> asyncio.subprocess.Process: ) -> asyncio.subprocess.Process:
"""Launch *command* in a platform-appropriate shell.""" """Launch *command* in a platform-appropriate shell."""
if _IS_WINDOWS: if _IS_WINDOWS:
if "\n" in command: # create_subprocess_exec re-quotes args via list2cmdline, which
return await asyncio.create_subprocess_exec( # breaks commands containing paths with spaces (e.g. "D:\Program
"powershell", "-NoProfile", "-Command", command, # Files\python.exe" "script.py"). create_subprocess_shell passes
stdin=stdin, # the raw command string to COMSPEC without re-quoting.
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
cwd=cwd,
env=env,
)
return await asyncio.create_subprocess_shell( return await asyncio.create_subprocess_shell(
command, command,
stdin=stdin, stdin=asyncio.subprocess.DEVNULL,
stdout=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE,
cwd=cwd, cwd=cwd,
@@ -466,7 +434,7 @@ class ExecTool(Tool):
args.extend(["-c", command]) args.extend(["-c", command])
return await asyncio.create_subprocess_exec( return await asyncio.create_subprocess_exec(
*args, *args,
stdin=stdin, stdin=asyncio.subprocess.DEVNULL,
stdout=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE,
cwd=cwd, cwd=cwd,
@@ -560,13 +528,7 @@ class ExecTool(Tool):
env[key] = val env[key] = val
return env return env
def _guard_command( def _guard_command(self, command: str, cwd: str) -> str | None:
self,
command: str,
cwd: str,
*,
restrict_to_workspace: bool | None = None,
) -> str | None:
"""Best-effort safety guard for potentially destructive commands.""" """Best-effort safety guard for potentially destructive commands."""
cmd = command.strip() cmd = command.strip()
lower = cmd.lower() lower = cmd.lower()
@@ -586,17 +548,11 @@ class ExecTool(Tool):
return "Error: Command blocked by allowlist filter (not in allowlist)" return "Error: Command blocked by allowlist filter (not in allowlist)"
from nanobot.security.network import contains_internal_url from nanobot.security.network import contains_internal_url
if contains_internal_url( if contains_internal_url(cmd):
cmd,
allow_loopback=current_scope_allows_loopback(
enabled=self.webui_allow_local_service_access,
),
):
# The runner turns this marker into a non-retryable security hint. # The runner turns this marker into a non-retryable security hint.
return "Error: Command blocked by safety guard (internal/private URL detected)" return "Error: Command blocked by safety guard (internal/private URL detected)"
should_restrict = self.restrict_to_workspace if restrict_to_workspace is None else restrict_to_workspace if self.restrict_to_workspace:
if should_restrict:
if "..\\" in cmd or "../" in cmd: if "..\\" in cmd or "../" in cmd:
return ( return (
"Error: Command blocked by safety guard (path traversal detected)" "Error: Command blocked by safety guard (path traversal detected)"
@@ -621,9 +577,11 @@ class ExecTool(Tool):
continue continue
media_path = get_media_dir().resolve() media_path = get_media_dir().resolve()
if p.is_absolute() and not ( if (p.is_absolute()
is_path_within(p, cwd_path) and cwd_path not in p.parents
or is_path_within(p, media_path) and p != cwd_path
and media_path not in p.parents
and p != media_path
): ):
return ( return (
"Error: Command blocked by safety guard (path outside working dir)" "Error: Command blocked by safety guard (path outside working dir)"
-2
View File
@@ -8,7 +8,6 @@ from typing import TYPE_CHECKING, Any
from nanobot.agent.tools.base import Tool, tool_parameters from nanobot.agent.tools.base import Tool, tool_parameters
from nanobot.agent.tools.context import ContextAware, RequestContext from nanobot.agent.tools.context import ContextAware, RequestContext
from nanobot.agent.tools.schema import NumberSchema, StringSchema, tool_parameters_schema from nanobot.agent.tools.schema import NumberSchema, StringSchema, tool_parameters_schema
from nanobot.security.workspace_access import current_workspace_scope
if TYPE_CHECKING: if TYPE_CHECKING:
from nanobot.agent.subagent import SubagentManager from nanobot.agent.subagent import SubagentManager
@@ -92,5 +91,4 @@ class SpawnTool(Tool, ContextAware):
session_key=self._session_key.get(), session_key=self._session_key.get(),
origin_message_id=self._origin_message_id.get(), origin_message_id=self._origin_message_id.get(),
temperature=temperature, temperature=temperature,
workspace_scope=current_workspace_scope(),
) )
+6 -5
View File
@@ -455,16 +455,17 @@ class WebSearchTool(Tool):
return await self._search_duckduckgo(query, n) return await self._search_duckduckgo(query, n)
try: try:
async with httpx.AsyncClient(proxy=self.proxy) as client: async with httpx.AsyncClient(proxy=self.proxy) as client:
r = await client.post( r = await client.get(
"https://kagi.com/api/v1/search", "https://kagi.com/api/v0/search",
json={"query": query, "limit": n}, params={"q": query, "limit": n},
headers={"Authorization": f"Bearer {api_key}", "User-Agent": self.user_agent}, headers={"Authorization": f"Bot {api_key}", "User-Agent": self.user_agent},
timeout=10.0, timeout=10.0,
) )
r.raise_for_status() r.raise_for_status()
# t=0 items are search results; other values are related searches, etc.
items = [ items = [
{"title": d.get("title", ""), "url": d.get("url", ""), "content": d.get("snippet", "")} {"title": d.get("title", ""), "url": d.get("url", ""), "content": d.get("snippet", "")}
for d in r.json().get("data", {}).get("search", []) for d in r.json().get("data", []) if d.get("t") == 0
] ]
return _format_results(query, items, n) return _format_results(query, items, n)
except Exception as e: except Exception as e:
+30 -108
View File
@@ -20,27 +20,18 @@ import httpx
from nanobot.apps.protocol import app_manifest, compact_dict from nanobot.apps.protocol import app_manifest, compact_dict
from nanobot.config.paths import get_runtime_subdir from nanobot.config.paths import get_runtime_subdir
from nanobot.security.workspace_policy import is_path_within
CLI_ANYTHING_REGISTRY_URL = "https://hkuds.github.io/CLI-Anything/registry.json" CLI_ANYTHING_REGISTRY_URL = "https://hkuds.github.io/CLI-Anything/registry.json"
CLI_ANYTHING_PUBLIC_REGISTRY_URL = "https://hkuds.github.io/CLI-Anything/public_registry.json" CLI_ANYTHING_PUBLIC_REGISTRY_URL = "https://hkuds.github.io/CLI-Anything/public_registry.json"
CLI_ANYTHING_RAW_BASE = "https://raw.githubusercontent.com/HKUDS/CLI-Anything/main" CLI_ANYTHING_RAW_BASE = "https://raw.githubusercontent.com/HKUDS/CLI-Anything/main"
NANOBOT_EXTENSION_REGISTRY_URL = "https://raw.githubusercontent.com/Re-bin/nanobot-extension/main/registry.json" CLI_ANYTHING_RAW_SKILLS_BASE = f"{CLI_ANYTHING_RAW_BASE}/skills/"
NANOBOT_EXTENSION_RAW_BASE = "https://raw.githubusercontent.com/Re-bin/nanobot-extension/main"
_CATALOG_SOURCES = (
("harness", CLI_ANYTHING_REGISTRY_URL, CLI_ANYTHING_RAW_BASE, True),
("public", CLI_ANYTHING_PUBLIC_REGISTRY_URL, CLI_ANYTHING_RAW_BASE, True),
("extensions", NANOBOT_EXTENSION_REGISTRY_URL, NANOBOT_EXTENSION_RAW_BASE, False),
)
_MAX_TOOL_OUTPUT_CHARS = 12_000 _MAX_TOOL_OUTPUT_CHARS = 12_000
_MAX_ARTIFACT_SCAN_PATHS = 4_000 _MAX_ARTIFACT_SCAN_PATHS = 4_000
_MAX_ARTIFACT_REPORT = 12 _MAX_ARTIFACT_REPORT = 12
_SAFE_NAME_RE = re.compile(r"[^a-z0-9_-]+") _SAFE_NAME_RE = re.compile(r"[^a-z0-9_-]+")
_SAFE_NPM_DIR_RE = re.compile(r"^[a-z0-9._-]+$", re.IGNORECASE)
_MENTION_RE = re.compile(r"(^|[\s([{])@([a-z0-9_-]+)\b", re.IGNORECASE) _MENTION_RE = re.compile(r"(^|[\s([{])@([a-z0-9_-]+)\b", re.IGNORECASE)
_SHELL_META_CHARS = ("|", "&&", "||", ";", "$(", "`", ">", "<") _SHELL_META_CHARS = ("|", "&&", "||", ";", "$(", "`", ">", "<")
_ENDORSEMENT_WORD_RE = re.compile(r"\bofficial\s+", re.IGNORECASE)
_ARTIFACT_EXTENSIONS = frozenset({ _ARTIFACT_EXTENSIONS = frozenset({
".csv", ".csv",
".drawio", ".drawio",
@@ -303,11 +294,6 @@ def _brand_candidates(app: dict[str, Any]) -> list[str]:
def _brand_payload(app: dict[str, Any]) -> tuple[str | None, str | None]: def _brand_payload(app: dict[str, Any]) -> tuple[str | None, str | None]:
declared_logo = str(app.get("logo_url") or "").strip()
if declared_logo.startswith(("https://", "/")):
declared_color = str(app.get("brand_color") or "").strip()
return declared_logo, declared_color or None
brand = None brand = None
domain_brand = None domain_brand = None
for candidate in _brand_candidates(app): for candidate in _brand_candidates(app):
@@ -356,17 +342,16 @@ def _safe_skill_path(value: str) -> str | None:
return value if parts[-1] == "SKILL.md" else None return value if parts[-1] == "SKILL.md" else None
def _skill_content_url(skill_md: str, *, raw_base: str = CLI_ANYTHING_RAW_BASE) -> str | None: def _skill_content_url(skill_md: str) -> str | None:
safe_path = _safe_skill_path(skill_md) safe_path = _safe_skill_path(skill_md)
if safe_path: if safe_path:
return f"{raw_base.rstrip('/')}/{safe_path}" return f"{CLI_ANYTHING_RAW_BASE}/{safe_path}"
parsed = urlparse(skill_md) parsed = urlparse(skill_md)
if parsed.scheme != "https" or parsed.netloc != "raw.githubusercontent.com": if parsed.scheme != "https" or parsed.netloc != "raw.githubusercontent.com":
return None return None
raw_prefix = raw_base.rstrip("/") + "/" if not skill_md.startswith(CLI_ANYTHING_RAW_SKILLS_BASE):
if not skill_md.startswith(raw_prefix):
return None return None
suffix = skill_md.removeprefix(raw_prefix) suffix = skill_md.removeprefix(f"{CLI_ANYTHING_RAW_BASE}/")
return skill_md if _safe_skill_path(suffix) else None return skill_md if _safe_skill_path(suffix) else None
@@ -377,12 +362,6 @@ def _truncate(text: str, limit: int = _MAX_TOOL_OUTPUT_CHARS) -> str:
return text[:limit] + f"\n\n... truncated {omitted} characters ..." return text[:limit] + f"\n\n... truncated {omitted} characters ..."
def _catalog_description(app: dict[str, Any]) -> str:
"""Return catalog copy without implying vendor endorsement."""
description = str(app.get("description") or "")
return _ENDORSEMENT_WORD_RE.sub("", description).strip()
class CliAppManager: class CliAppManager:
"""Manage CLI-Anything registry entries and local install state.""" """Manage CLI-Anything registry entries and local install state."""
@@ -448,22 +427,27 @@ class CliAppManager:
return data return data
def catalog(self, *, force_refresh: bool = False) -> tuple[list[dict[str, Any]], str | None]: def catalog(self, *, force_refresh: bool = False) -> tuple[list[dict[str, Any]], str | None]:
registries: list[tuple[str, str, dict[str, Any]]] = [] registries = [
for source, url, raw_base, required in _CATALOG_SOURCES: (
try: "harness",
registry = self._fetch_registry( self._fetch_registry(
url, CLI_ANYTHING_REGISTRY_URL,
self._cache_path(source), self._cache_path("harness"),
force_refresh=force_refresh, force_refresh=force_refresh,
) ),
except Exception: ),
if required: (
raise "public",
continue self._fetch_registry(
registries.append((source, raw_base, registry)) CLI_ANYTHING_PUBLIC_REGISTRY_URL,
self._cache_path("public"),
force_refresh=force_refresh,
),
),
]
apps_by_name: dict[str, dict[str, Any]] = {} apps_by_name: dict[str, dict[str, Any]] = {}
updated_values: list[str] = [] updated_values: list[str] = []
for source, raw_base, registry in registries: for source, registry in registries:
meta = registry.get("meta") meta = registry.get("meta")
if isinstance(meta, dict) and isinstance(meta.get("updated"), str): if isinstance(meta, dict) and isinstance(meta.get("updated"), str):
updated_values.append(meta["updated"]) updated_values.append(meta["updated"])
@@ -472,7 +456,6 @@ class CliAppManager:
continue continue
entry = dict(row) entry = dict(row)
entry["_source"] = source entry["_source"] = source
entry["_raw_base"] = raw_base
key = str(entry["name"]).lower() key = str(entry["name"]).lower()
previous = apps_by_name.get(key) previous = apps_by_name.get(key)
if previous: if previous:
@@ -485,15 +468,6 @@ class CliAppManager:
apps_by_name[key] = entry apps_by_name[key] = entry
return list(apps_by_name.values()), max(updated_values) if updated_values else None return list(apps_by_name.values()), max(updated_values) if updated_values else None
def _manifest_source(self, app: dict[str, Any]) -> str:
source = str(app.get("_source") or "harness")
if source == "extensions":
return "nanobot-extension"
return f"cli-anything:{source}"
def _trust_registry(self, app: dict[str, Any]) -> str:
return "nanobot-extension" if str(app.get("_source") or "") == "extensions" else "cli-anything"
def get_app(self, name: str, *, force_refresh: bool = False) -> dict[str, Any]: def get_app(self, name: str, *, force_refresh: bool = False) -> dict[str, Any]:
wanted = name.lower() wanted = name.lower()
for app in self.catalog(force_refresh=force_refresh)[0]: for app in self.catalog(force_refresh=force_refresh)[0]:
@@ -580,7 +554,7 @@ class CliAppManager:
"name": name, "name": name,
"display_name": app.get("display_name") or name, "display_name": app.get("display_name") or name,
"category": app.get("category") or "uncategorized", "category": app.get("category") or "uncategorized",
"description": _catalog_description(app), "description": app.get("description") or "",
"requires": app.get("requires") or "", "requires": app.get("requires") or "",
"source": app.get("_source") or "harness", "source": app.get("_source") or "harness",
"entry_point": entry_point, "entry_point": entry_point,
@@ -656,16 +630,16 @@ class CliAppManager:
app_id=name, app_id=name,
display_name=str(app.get("display_name") or name), display_name=str(app.get("display_name") or name),
version=str(app.get("version") or ""), version=str(app.get("version") or ""),
description=_catalog_description(app), description=str(app.get("description") or ""),
category=str(app.get("category") or "uncategorized"), category=str(app.get("category") or "uncategorized"),
source=self._manifest_source(app), source=f"cli-anything:{app.get('_source') or 'harness'}",
logo_url=logo_url, logo_url=logo_url,
brand_color=brand_color, brand_color=brand_color,
capabilities=capabilities, capabilities=capabilities,
install=install, install=install,
remove=remove, remove=remove,
trust={ trust={
"registry": self._trust_registry(app), "registry": "cli-anything",
"level": "catalog", "level": "catalog",
"review_status": "catalog_entry", "review_status": "catalog_entry",
}, },
@@ -741,45 +715,6 @@ class CliAppManager:
return [npm, "install", "-g", package + "@latest"] return [npm, "install", "-g", package + "@latest"]
return [npm, "uninstall", "-g", package] return [npm, "uninstall", "-g", package]
def _cleanup_stale_npm_install(self, app: dict[str, Any]) -> bool:
npm = shutil.which("npm")
package = str(app.get("npm_package") or "").strip()
if not npm or not package or "/" in package or _SAFE_NPM_DIR_RE.match(package) is None:
return False
result = self._run_argv([npm, "root", "-g"], timeout=min(self.runtime.install_timeout, 30))
if result.returncode != 0:
return False
root = Path(result.stdout.strip()).expanduser()
try:
root = root.resolve(strict=True)
except OSError:
return False
targets = [root / package, *root.glob(f".{package}-*")]
removed = False
for target in targets:
try:
resolved = target.resolve(strict=False)
if not is_path_within(resolved, root) or not target.is_dir():
continue
shutil.rmtree(target)
removed = True
except OSError:
continue
return removed
def _retry_stale_npm_install(
self,
app: dict[str, Any],
argv: list[str],
result: subprocess.CompletedProcess[str],
) -> subprocess.CompletedProcess[str]:
output = f"{result.stderr}\n{result.stdout}"
if "ENOTEMPTY" not in output or "rename" not in output:
return result
if not self._cleanup_stale_npm_install(app):
return result
return self._run_argv(argv, timeout=self.runtime.install_timeout)
def _split_safe_command(self, app: dict[str, Any], key: str, expected: str) -> list[str]: def _split_safe_command(self, app: dict[str, Any], key: str, expected: str) -> list[str]:
command = str(app.get(key) or "") command = str(app.get(key) or "")
if not command: if not command:
@@ -850,7 +785,7 @@ class CliAppManager:
skill_md = str(app.get("skill_md") or "").strip() skill_md = str(app.get("skill_md") or "").strip()
if not skill_md: if not skill_md:
return None return None
url = _skill_content_url(skill_md, raw_base=str(app.get("_raw_base") or CLI_ANYTHING_RAW_BASE)) url = _skill_content_url(skill_md)
if not url: if not url:
return None return None
try: try:
@@ -867,7 +802,7 @@ class CliAppManager:
name = str(app.get("name") or "unknown") name = str(app.get("name") or "unknown")
display = str(app.get("display_name") or name) display = str(app.get("display_name") or name)
entry = str(app.get("entry_point") or f"cli-anything-{name}") entry = str(app.get("entry_point") or f"cli-anything-{name}")
description = _catalog_description(app) or f"Use {display} from nanobot." description = str(app.get("description") or f"Use {display} from nanobot.")
return f"""--- return f"""---
name: {_safe_skill_name(name)} name: {_safe_skill_name(name)}
description: >- description: >-
@@ -933,17 +868,6 @@ Use the `run_cli_app` tool with `name="{name}"` for command execution. Do not in
if not self._install_supported(app): if not self._install_supported(app):
raise CliAppError("this CLI app uses an unsupported install strategy") raise CliAppError("this CLI app uses an unsupported install strategy")
strategy = self._strategy(app) strategy = self._strategy(app)
entry_point = str(app.get("entry_point") or "")
if entry_point and shutil.which(entry_point):
self._record_installed(app)
return self.payload() | {
"last_action": {
"ok": True,
"message": f"CLI for {app['display_name']} is already available.",
"installed": True,
"verification": ["entry_point_available", "state_recorded", "managed_paths_present"],
}
}
if strategy == "bundled": if strategy == "bundled":
detect_cmd = str(app.get("detect_cmd") or app.get("entry_point") or "") detect_cmd = str(app.get("detect_cmd") or app.get("entry_point") or "")
if detect_cmd and _command_exists(detect_cmd): if detect_cmd and _command_exists(detect_cmd):
@@ -961,8 +885,6 @@ Use the `run_cli_app` tool with `name="{name}"` for command execution. Do not in
argv = self._argv_for_action(app, "install") argv = self._argv_for_action(app, "install")
assert argv is not None assert argv is not None
result = self._run_argv(argv, timeout=self.runtime.install_timeout) result = self._run_argv(argv, timeout=self.runtime.install_timeout)
if strategy == "npm" and result.returncode != 0:
result = self._retry_stale_npm_install(app, argv, result)
if result.returncode != 0: if result.returncode != 0:
raise CliAppError(_truncate(result.stderr or result.stdout or "install failed"), status=500) raise CliAppError(_truncate(result.stderr or result.stdout or "install failed"), status=500)
self._record_installed(app) self._record_installed(app)
@@ -1096,7 +1018,7 @@ Use the `run_cli_app` tool with `name="{name}"` for command execution. Do not in
cwd = Path(working_dir).expanduser() if working_dir else self.workspace cwd = Path(working_dir).expanduser() if working_dir else self.workspace
cwd = cwd.resolve(strict=False) cwd = cwd.resolve(strict=False)
workspace = self.workspace.resolve(strict=False) workspace = self.workspace.resolve(strict=False)
if restrict_to_workspace and not is_path_within(cwd, workspace): if restrict_to_workspace and cwd != workspace and not cwd.is_relative_to(workspace):
raise CliAppError("working_dir is outside the configured workspace") raise CliAppError("working_dir is outside the configured workspace")
return cwd return cwd
-10
View File
@@ -207,16 +207,6 @@ if DISCORD_AVAILABLE:
) -> None: ) -> None:
await self._forward_slash_command(interaction, _command_text) await self._forward_slash_command(interaction, _command_text)
@self.tree.command(name="model", description="Show or switch runtime model preset")
@app_commands.describe(preset="Optional model preset name, such as default")
async def model_command(
interaction: discord.Interaction,
preset: str | None = None,
) -> None:
preset = (preset or "").strip()
command_text = f"/model {preset}" if preset else "/model"
await self._forward_slash_command(interaction, command_text)
@self.tree.command(name="help", description="Show available commands") @self.tree.command(name="help", description="Show available commands")
async def help_command(interaction: discord.Interaction) -> None: async def help_command(interaction: discord.Interaction) -> None:
sender_id = str(interaction.user.id) sender_id = str(interaction.user.id)
+1 -10
View File
@@ -57,17 +57,11 @@ class ChannelManager:
*, *,
session_manager: "SessionManager | None" = None, session_manager: "SessionManager | None" = None,
webui_runtime_model_name: Callable[[], str | None] | None = None, webui_runtime_model_name: Callable[[], str | None] | None = None,
webui_static_dist: bool = True,
webui_runtime_surface: str = "browser",
webui_runtime_capabilities: dict[str, Any] | None = None,
): ):
self.config = config self.config = config
self.bus = bus self.bus = bus
self._session_manager = session_manager self._session_manager = session_manager
self._webui_runtime_model_name = webui_runtime_model_name self._webui_runtime_model_name = webui_runtime_model_name
self._webui_static_dist = webui_static_dist
self._webui_runtime_surface = webui_runtime_surface
self._webui_runtime_capabilities = dict(webui_runtime_capabilities or {})
self.channels: dict[str, BaseChannel] = {} self.channels: dict[str, BaseChannel] = {}
self._dispatch_task: asyncio.Task | None = None self._dispatch_task: asyncio.Task | None = None
self._origin_reply_fingerprints: dict[tuple[str, str, str], str] = {} self._origin_reply_fingerprints: dict[tuple[str, str, str], str] = {}
@@ -113,15 +107,12 @@ class ChannelManager:
if cls.name == "websocket": if cls.name == "websocket":
if self._session_manager is not None: if self._session_manager is not None:
kwargs["session_manager"] = self._session_manager kwargs["session_manager"] = self._session_manager
static_path = _default_webui_dist() if self._webui_static_dist else None static_path = _default_webui_dist()
if static_path is not None: if static_path is not None:
kwargs["static_dist_path"] = static_path kwargs["static_dist_path"] = static_path
kwargs["workspace_path"] = self.config.workspace_path kwargs["workspace_path"] = self.config.workspace_path
kwargs["restrict_to_workspace"] = self.config.tools.restrict_to_workspace
if self._webui_runtime_model_name is not None: if self._webui_runtime_model_name is not None:
kwargs["runtime_model_name"] = self._webui_runtime_model_name kwargs["runtime_model_name"] = self._webui_runtime_model_name
kwargs["runtime_surface"] = self._webui_runtime_surface
kwargs["runtime_capabilities_overrides"] = self._webui_runtime_capabilities
channel = cls(section, self.bus, **kwargs) channel = cls(section, self.bus, **kwargs)
channel.transcription_provider = transcription_provider channel.transcription_provider = transcription_provider
channel.transcription_api_key = transcription_key channel.transcription_api_key = transcription_key
+28 -60
View File
@@ -8,23 +8,21 @@ from contextlib import suppress
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import Any, Literal, TypeAlias from typing import Any, Literal, TypeAlias
from urllib.parse import quote, urlparse
from pydantic import Field from pydantic import Field
from nanobot.security.workspace_policy import is_path_within
try: try:
import aiohttp
import nh3 import nh3
from mistune import create_markdown from mistune import create_markdown
from nio import ( from nio import (
AsyncClient, AsyncClient,
AsyncClientConfig, AsyncClientConfig,
DownloadError,
InviteEvent, InviteEvent,
JoinError, JoinError,
LoginResponse, LoginResponse,
MatrixRoom, MatrixRoom,
MemoryDownloadResponse,
RoomEncryptedMedia, RoomEncryptedMedia,
RoomMessage, RoomMessage,
RoomMessageMedia, RoomMessageMedia,
@@ -64,10 +62,6 @@ _MSGTYPE_MAP = {"m.image": "image", "m.audio": "audio", "m.video": "video", "m.f
MATRIX_MEDIA_EVENT_FILTER = (RoomMessageMedia, RoomEncryptedMedia) MATRIX_MEDIA_EVENT_FILTER = (RoomMessageMedia, RoomEncryptedMedia)
MatrixMediaEvent: TypeAlias = RoomMessageMedia | RoomEncryptedMedia MatrixMediaEvent: TypeAlias = RoomMessageMedia | RoomEncryptedMedia
class _MediaTooLargeError(Exception):
"""Raised when an inbound Matrix media download exceeds the configured cap."""
MATRIX_MARKDOWN = create_markdown( MATRIX_MARKDOWN = create_markdown(
escape=True, escape=True,
plugins=["table", "strikethrough", "url", "superscript", "subscript"], plugins=["table", "strikethrough", "url", "superscript", "subscript"],
@@ -196,7 +190,6 @@ class MatrixConfig(Base):
e2ee_enabled: bool = Field(default=True, alias="e2eeEnabled") e2ee_enabled: bool = Field(default=True, alias="e2eeEnabled")
sync_stop_grace_seconds: int = 2 sync_stop_grace_seconds: int = 2
max_media_bytes: int = 20 * 1024 * 1024 max_media_bytes: int = 20 * 1024 * 1024
max_concurrent_media_downloads: int = 2
allow_from: list[str] = Field(default_factory=list) allow_from: list[str] = Field(default_factory=list)
group_policy: Literal["open", "mention", "allowlist"] = "open" group_policy: Literal["open", "mention", "allowlist"] = "open"
group_allow_from: list[str] = Field(default_factory=list) group_allow_from: list[str] = Field(default_factory=list)
@@ -238,9 +231,6 @@ class MatrixChannel(BaseChannel):
self._server_upload_limit_checked = False self._server_upload_limit_checked = False
self._stream_bufs: dict[str, _StreamBuf] = {} self._stream_bufs: dict[str, _StreamBuf] = {}
self._started_at_ms: int = 0 self._started_at_ms: int = 0
self._media_download_semaphore = asyncio.Semaphore(
max(1, int(self.config.max_concurrent_media_downloads))
)
async def start(self) -> None: async def start(self) -> None:
@@ -354,7 +344,11 @@ class MatrixChannel(BaseChannel):
"""Check path is inside workspace (when restriction enabled).""" """Check path is inside workspace (when restriction enabled)."""
if not self._restrict_to_workspace or not self._workspace: if not self._restrict_to_workspace or not self._workspace:
return True return True
return is_path_within(path, self._workspace) try:
path.resolve(strict=False).relative_to(self._workspace)
return True
except ValueError:
return False
def _collect_outbound_media_candidates(self, media: list[str]) -> list[Path]: def _collect_outbound_media_candidates(self, media: list[str]) -> list[Path]:
"""Deduplicate and resolve outbound attachment paths.""" """Deduplicate and resolve outbound attachment paths."""
@@ -749,7 +743,7 @@ class MatrixChannel(BaseChannel):
def _event_declared_size_bytes(self, event: MatrixMediaEvent) -> int | None: def _event_declared_size_bytes(self, event: MatrixMediaEvent) -> int | None:
info = self._event_source_content(event).get("info") info = self._event_source_content(event).get("info")
size = info.get("size") if isinstance(info, dict) else None size = info.get("size") if isinstance(info, dict) else None
return size if type(size) is int and size >= 0 else None return size if isinstance(size, int) and size >= 0 else None
def _event_mime(self, event: MatrixMediaEvent) -> str | None: def _event_mime(self, event: MatrixMediaEvent) -> str | None:
info = self._event_source_content(event).get("info") info = self._event_source_content(event).get("info")
@@ -778,48 +772,26 @@ class MatrixChannel(BaseChannel):
event_prefix = (event_id[:24] or "evt").strip("_") event_prefix = (event_id[:24] or "evt").strip("_")
return self._media_dir() / f"{event_prefix}_{stem}{suffix}" return self._media_dir() / f"{event_prefix}_{stem}{suffix}"
async def _download_media_bytes(self, mxc_url: str, limit_bytes: int) -> bytes | None: async def _download_media_bytes(self, mxc_url: str) -> bytes | None:
if not self.client or limit_bytes <= 0: if not self.client:
raise _MediaTooLargeError
parsed = urlparse(mxc_url)
if parsed.scheme != "mxc" or not parsed.netloc or not parsed.path.strip("/"):
return None return None
response = await self.client.download(mxc=mxc_url)
homeserver = str(getattr(self.client, "homeserver", "") or self.config.homeserver).rstrip("/") if isinstance(response, DownloadError):
media_url = ( self.logger.warning("download failed for {}: {}", mxc_url, response)
f"{homeserver}/_matrix/client/v1/media/download/"
f"{quote(parsed.netloc, safe='')}/{quote(parsed.path.strip('/'), safe='')}"
)
token = getattr(self.client, "access_token", None) or self.config.access_token
headers = {"Authorization": f"Bearer {token}"} if token else None
timeout = aiohttp.ClientTimeout(total=None)
try:
async with aiohttp.ClientSession(timeout=timeout, headers=headers) as session:
async with session.get(media_url, params={"allow_remote": "true"}) as response:
if response.status >= 400:
self.logger.warning("download failed for {}: HTTP {}", mxc_url, response.status)
return None
content_length = response.headers.get("Content-Length")
if content_length is not None:
try:
if int(content_length) > limit_bytes:
raise _MediaTooLargeError
except ValueError:
pass
chunks = bytearray()
async for chunk in response.content.iter_chunked(64 * 1024):
chunks.extend(chunk)
if len(chunks) > limit_bytes:
raise _MediaTooLargeError
return bytes(chunks)
except _MediaTooLargeError:
raise
except (aiohttp.ClientError, asyncio.TimeoutError, OSError):
self.logger.warning("download failed for {}", mxc_url, exc_info=True)
return None return None
body = getattr(response, "body", None)
if isinstance(body, (bytes, bytearray)):
return bytes(body)
if isinstance(response, MemoryDownloadResponse):
return bytes(response.body)
if isinstance(body, (str, Path)):
path = Path(body)
if path.is_file():
try:
return path.read_bytes()
except OSError:
return None
return None
def _decrypt_media_bytes(self, event: MatrixMediaEvent, ciphertext: bytes) -> bytes | None: def _decrypt_media_bytes(self, event: MatrixMediaEvent, ciphertext: bytes) -> bytes | None:
key_obj, hashes, iv = getattr(event, "key", None), getattr(event, "hashes", None), getattr(event, "iv", None) key_obj, hashes, iv = getattr(event, "key", None), getattr(event, "hashes", None), getattr(event, "iv", None)
@@ -848,14 +820,10 @@ class MatrixChannel(BaseChannel):
limit_bytes = await self._effective_media_limit_bytes() limit_bytes = await self._effective_media_limit_bytes()
declared = self._event_declared_size_bytes(event) declared = self._event_declared_size_bytes(event)
if declared is None or declared > limit_bytes: if declared is not None and declared > limit_bytes:
return None, _ATTACH_TOO_LARGE.format(filename) return None, _ATTACH_TOO_LARGE.format(filename)
try: downloaded = await self._download_media_bytes(mxc_url)
async with self._media_download_semaphore:
downloaded = await self._download_media_bytes(mxc_url, limit_bytes)
except _MediaTooLargeError:
return None, _ATTACH_TOO_LARGE.format(filename)
if downloaded is None: if downloaded is None:
return None, fail return None, fail
-49
View File
@@ -53,13 +53,6 @@ if MSTEAMS_AVAILABLE:
MSTEAMS_REF_TTL_DAYS = 30 MSTEAMS_REF_TTL_DAYS = 30
MSTEAMS_WEBCHAT_HOST = "webchat.botframework.com" MSTEAMS_WEBCHAT_HOST = "webchat.botframework.com"
MSTEAMS_DEFAULT_TRUSTED_SERVICE_URL_HOSTS = [
"smba.trafficmanager.net",
"smba.infra.gcc.teams.microsoft.com",
"smba.infra.gov.teams.microsoft.us",
"smba.infra.dod.teams.microsoft.us",
"*.botframework.com",
]
MSTEAMS_REF_META_FILENAME = "msteams_conversations_meta.json" MSTEAMS_REF_META_FILENAME = "msteams_conversations_meta.json"
MSTEAMS_REF_LOCK_FILENAME = "msteams_conversations.lock" MSTEAMS_REF_LOCK_FILENAME = "msteams_conversations.lock"
MSTEAMS_REF_TOUCH_INTERVAL_S = 300 MSTEAMS_REF_TOUCH_INTERVAL_S = 300
@@ -83,9 +76,6 @@ class MSTeamsConfig(Base):
prune_web_chat_refs: bool = True prune_web_chat_refs: bool = True
prune_non_personal_refs: bool = True prune_non_personal_refs: bool = True
ref_touch_interval_s: int = Field(default=MSTEAMS_REF_TOUCH_INTERVAL_S, ge=0) ref_touch_interval_s: int = Field(default=MSTEAMS_REF_TOUCH_INTERVAL_S, ge=0)
trusted_service_url_hosts: list[str] = Field(
default_factory=lambda: MSTEAMS_DEFAULT_TRUSTED_SERVICE_URL_HOSTS.copy()
)
@dataclass @dataclass
@@ -252,11 +242,6 @@ class MSTeamsChannel(BaseChannel):
if not ref: if not ref:
raise RuntimeError(f"MSTeams conversation ref not found for chat_id={msg.chat_id}") raise RuntimeError(f"MSTeams conversation ref not found for chat_id={msg.chat_id}")
if not self._is_trusted_service_url(ref.service_url):
raise RuntimeError(
f"MSTeams conversation ref has untrusted service_url for chat_id={msg.chat_id}"
)
token = await self._get_access_token() token = await self._get_access_token()
base_url = f"{ref.service_url.rstrip('/')}/v3/conversations/{ref.conversation_id}/activities" base_url = f"{ref.service_url.rstrip('/')}/v3/conversations/{ref.conversation_id}/activities"
use_thread_reply = self.config.reply_in_thread and bool(ref.activity_id) use_thread_reply = self.config.reply_in_thread and bool(ref.activity_id)
@@ -299,13 +284,6 @@ class MSTeamsChannel(BaseChannel):
if not sender_id or not conversation_id or not service_url: if not sender_id or not conversation_id or not service_url:
return return
if not self._is_trusted_service_url(service_url):
self.logger.warning(
"Ignoring MSTeams activity with untrusted serviceUrl host: {}",
service_url,
)
return
if recipient.get("id") and from_user.get("id") == recipient.get("id"): if recipient.get("id") and from_user.get("id") == recipient.get("id"):
return return
@@ -648,29 +626,6 @@ class MSTeamsChannel(BaseChannel):
return host == MSTEAMS_WEBCHAT_HOST or host.endswith(f".{MSTEAMS_WEBCHAT_HOST}") return host == MSTEAMS_WEBCHAT_HOST or host.endswith(f".{MSTEAMS_WEBCHAT_HOST}")
return MSTEAMS_WEBCHAT_HOST in normalized.lower() return MSTEAMS_WEBCHAT_HOST in normalized.lower()
def _is_trusted_service_url(self, service_url: str) -> bool:
"""Return True for HTTPS Bot Framework service URLs trusted for bearer replies."""
parsed = urlparse(service_url.strip())
if parsed.scheme.lower() != "https":
return False
host = (parsed.hostname or "").strip().lower().rstrip(".")
if not host:
return False
for pattern in self.config.trusted_service_url_hosts:
trusted_host = str(pattern or "").strip().lower().rstrip(".")
if not trusted_host:
continue
if trusted_host.startswith("*."):
suffix = trusted_host[1:]
if host.endswith(suffix) and host != suffix.lstrip("."):
return True
continue
if host == trusted_host:
return True
return False
def _prune_conversation_refs(self, *, now: float | None = None) -> bool: def _prune_conversation_refs(self, *, now: float | None = None) -> bool:
"""Remove stale and unsupported conversation refs from memory.""" """Remove stale and unsupported conversation refs from memory."""
if not self._conversation_refs: if not self._conversation_refs:
@@ -682,10 +637,6 @@ class MSTeamsChannel(BaseChannel):
keys_to_drop: list[str] = [] keys_to_drop: list[str] = []
for key, ref in self._conversation_refs.items(): for key, ref in self._conversation_refs.items():
if not self._is_trusted_service_url(ref.service_url):
keys_to_drop.append(key)
continue
if self.config.prune_web_chat_refs and self._is_webchat_service_url(ref.service_url): if self.config.prune_web_chat_refs and self._is_webchat_service_url(ref.service_url):
keys_to_drop.append(key) keys_to_drop.append(key)
continue continue
+12 -165
View File
@@ -10,9 +10,8 @@ from contextlib import suppress
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import Any, Literal from typing import Any, Literal
from urllib.parse import urlparse
from pydantic import Field, field_validator, model_validator from pydantic import Field
from telegram import ( from telegram import (
BotCommand, BotCommand,
InlineKeyboardButton, InlineKeyboardButton,
@@ -226,22 +225,11 @@ class _StreamBuf:
stream_id: str | None = None stream_id: str | None = None
@dataclass
class _QueuedTelegramUpdate:
"""Telegram update staged for per-session ordered processing."""
kind: Literal["command", "message"]
update: Update
context: Any
sort_key: tuple[int, int]
class TelegramConfig(Base): class TelegramConfig(Base):
"""Telegram channel configuration.""" """Telegram channel configuration."""
enabled: bool = False enabled: bool = False
token: str = "" token: str = ""
mode: Literal["polling", "webhook"] = "polling"
allow_from: list[str] = Field(default_factory=list) allow_from: list[str] = Field(default_factory=list)
proxy: str | None = None proxy: str | None = None
reply_to_message: bool = False reply_to_message: bool = False
@@ -253,48 +241,13 @@ class TelegramConfig(Base):
# Enable inline keyboard buttons in Telegram messages. # Enable inline keyboard buttons in Telegram messages.
inline_keyboards: bool = False inline_keyboards: bool = False
stream_edit_interval: float = Field(default=_STREAM_EDIT_INTERVAL_DEFAULT, ge=0.1) stream_edit_interval: float = Field(default=_STREAM_EDIT_INTERVAL_DEFAULT, ge=0.1)
webhook_url: str = ""
webhook_listen_host: str = "127.0.0.1"
webhook_listen_port: int = Field(default=8081, ge=1, le=65535)
webhook_path: str = "/telegram"
webhook_secret_token: str = ""
webhook_max_connections: int = Field(default=4, ge=1, le=100)
@field_validator("webhook_path")
@classmethod
def webhook_path_must_start_with_slash(cls, value: str) -> str:
value = value.strip() or "/telegram"
if not value.startswith("/"):
raise ValueError('webhook_path must start with "/"')
return value
@model_validator(mode="after")
def validate_webhook_config(self) -> "TelegramConfig":
if self.mode != "webhook":
return self
url = self.webhook_url.strip()
if not url:
raise ValueError("webhook_url is required when Telegram mode is webhook")
parsed = urlparse(url)
if parsed.scheme != "https" or not parsed.netloc:
raise ValueError("webhook_url must be a public HTTPS URL")
secret = self.webhook_secret_token.strip()
if not secret:
raise ValueError("webhook_secret_token is required when Telegram mode is webhook")
if len(secret) > 256 or re.match(r"^[A-Za-z0-9_-]+$", secret) is None:
raise ValueError(
"webhook_secret_token must be 1-256 characters using only A-Z, a-z, 0-9, _ and -"
)
return self
class TelegramChannel(BaseChannel): class TelegramChannel(BaseChannel):
""" """
Telegram channel using long polling or webhook mode. Telegram channel using long polling.
Long polling is the default. Webhook mode requires a public HTTPS URL and a Simple and reliable - no webhook/public IP needed.
Telegram secret token.
""" """
name = "telegram" name = "telegram"
@@ -341,8 +294,6 @@ class TelegramChannel(BaseChannel):
self._bot_user_id: int | None = None self._bot_user_id: int | None = None
self._bot_username: str | None = None self._bot_username: str | None = None
self._stream_bufs: dict[str, _StreamBuf] = {} # chat_id -> streaming state self._stream_bufs: dict[str, _StreamBuf] = {} # chat_id -> streaming state
self._inbound_buffers: dict[str, list[_QueuedTelegramUpdate]] = {}
self._inbound_workers: dict[str, asyncio.Task] = {}
def is_allowed(self, sender_id: str) -> bool: def is_allowed(self, sender_id: str) -> bool:
"""Preserve Telegram's legacy id|username allowlist matching.""" """Preserve Telegram's legacy id|username allowlist matching."""
@@ -375,7 +326,7 @@ class TelegramChannel(BaseChannel):
return content return content
async def start(self) -> None: async def start(self) -> None:
"""Start the Telegram bot.""" """Start the Telegram bot with long polling."""
if not self.config.token: if not self.config.token:
self.logger.error("bot token not configured") self.logger.error("bot token not configured")
return return
@@ -443,12 +394,9 @@ class TelegramChannel(BaseChannel):
else: else:
allowed_updates = ["message"] allowed_updates = ["message"]
if self.config.mode == "webhook": self.logger.info("Starting bot (polling mode)...")
self.logger.info("Starting bot (webhook mode)...")
else:
self.logger.info("Starting bot (polling mode)...")
# Initialize and start receiving updates # Initialize and start polling
await self._app.initialize() await self._app.initialize()
await self._app.start() await self._app.start()
@@ -464,26 +412,12 @@ class TelegramChannel(BaseChannel):
except Exception as e: except Exception as e:
self.logger.warning("Failed to register bot commands: {}", e) self.logger.warning("Failed to register bot commands: {}", e)
if self.config.mode == "webhook": # Start polling (this runs until stopped)
# ``url_path`` is the local HTTP route. ``webhook_url`` is the await self._app.updater.start_polling(
# public HTTPS URL Telegram calls; reverse proxies may rewrite it. allowed_updates=allowed_updates,
await self._app.updater.start_webhook( drop_pending_updates=False, # Process pending messages on startup
listen=self.config.webhook_listen_host, error_callback=self._on_polling_error,
port=self.config.webhook_listen_port, )
url_path=self.config.webhook_path.lstrip("/"),
webhook_url=self.config.webhook_url.strip(),
allowed_updates=allowed_updates,
drop_pending_updates=False,
secret_token=self.config.webhook_secret_token.strip(),
max_connections=self.config.webhook_max_connections,
)
else:
# Start polling (this runs until stopped)
await self._app.updater.start_polling(
allowed_updates=allowed_updates,
drop_pending_updates=False, # Process pending messages on startup
error_callback=self._on_polling_error,
)
# Keep running until stopped # Keep running until stopped
while self._running: while self._running:
@@ -502,11 +436,6 @@ class TelegramChannel(BaseChannel):
self._media_group_tasks.clear() self._media_group_tasks.clear()
self._media_group_buffers.clear() self._media_group_buffers.clear()
for task in self._inbound_workers.values():
task.cancel()
self._inbound_workers.clear()
self._inbound_buffers.clear()
if self._app: if self._app:
self.logger.info("Stopping bot...") self.logger.info("Stopping bot...")
await self._app.updater.stop() await self._app.updater.stop()
@@ -1066,85 +995,10 @@ class TelegramChannel(BaseChannel):
if len(self._message_threads) > 1000: if len(self._message_threads) > 1000:
self._message_threads.pop(next(iter(self._message_threads))) self._message_threads.pop(next(iter(self._message_threads)))
@staticmethod
def _queue_key_for_message(message) -> str:
"""Return the final nanobot session key used for ordered Telegram ingress."""
return TelegramChannel._derive_topic_session_key(message) or f"telegram:{message.chat_id}"
@staticmethod
def _sort_key_for_update(update: Update) -> tuple[int, int]:
"""Sort by chat message id first, then Telegram update id."""
message = getattr(update, "message", None)
message_id = int(getattr(message, "message_id", 0) or 0)
update_id = int(getattr(update, "update_id", 0) or 0)
return (message_id, update_id)
def _enqueue_ordered_update(
self,
*,
kind: Literal["command", "message"],
update: Update,
context: ContextTypes.DEFAULT_TYPE,
) -> None:
"""Stage a Telegram update behind a short per-session reorder window."""
message = update.message
key = self._queue_key_for_message(message)
self._inbound_buffers.setdefault(key, []).append(
_QueuedTelegramUpdate(
kind=kind,
update=update,
context=context,
sort_key=self._sort_key_for_update(update),
)
)
if key not in self._inbound_workers:
self._inbound_workers[key] = asyncio.create_task(
self._drain_ordered_updates(key)
)
async def _drain_ordered_updates(self, key: str) -> None:
"""Drain one Telegram session buffer in stable message order."""
try:
while self._running:
await asyncio.sleep(0.2)
batch = self._inbound_buffers.get(key, [])
if not batch:
break
self._inbound_buffers[key] = []
batch.sort(key=lambda item: item.sort_key)
for item in batch:
try:
if item.kind == "command":
await self._process_forward_command(item.update, item.context)
else:
await self._process_message_update(item.update, item.context)
except Exception as e:
self.logger.warning(
"Telegram queued update handling failed for {}: {}",
key,
e,
)
if not self._inbound_buffers.get(key):
self._inbound_buffers.pop(key, None)
except asyncio.CancelledError:
raise
except Exception as e:
self.logger.warning("Telegram ordered update worker failed for {}: {}", key, e)
finally:
if not self._inbound_buffers.get(key):
self._inbound_workers.pop(key, None)
async def _forward_command(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None: async def _forward_command(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
"""Forward slash commands to the bus for unified handling in AgentLoop.""" """Forward slash commands to the bus for unified handling in AgentLoop."""
if not update.message or not update.effective_user: if not update.message or not update.effective_user:
return return
if not self._running:
await self._process_forward_command(update, context)
return
self._enqueue_ordered_update(kind="command", update=update, context=context)
async def _process_forward_command(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
"""Process a queued slash command."""
message = update.message message = update.message
user = update.effective_user user = update.effective_user
sender_id = self._sender_id(user) sender_id = self._sender_id(user)
@@ -1173,13 +1027,6 @@ class TelegramChannel(BaseChannel):
"""Handle incoming messages (text, photos, voice, documents).""" """Handle incoming messages (text, photos, voice, documents)."""
if not update.message or not update.effective_user: if not update.message or not update.effective_user:
return return
if not self._running:
await self._process_message_update(update, context)
return
self._enqueue_ordered_update(kind="message", update=update, context=context)
async def _process_message_update(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
"""Process a queued Telegram message update."""
message = update.message message = update.message
user = update.effective_user user = update.effective_user
+365 -365
View File
@@ -3,34 +3,34 @@
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
import base64
import binascii
import email.utils import email.utils
import hashlib
import hmac import hmac
import http import http
import json import json
import mimetypes import mimetypes
import re import re
import secrets import secrets
import shutil
import ssl import ssl
import time import time
import uuid import uuid
from collections.abc import Callable from collections.abc import Callable
from contextlib import suppress
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING, Any, Self from typing import TYPE_CHECKING, Any, Self
from urllib.parse import parse_qs, unquote, urlparse from urllib.parse import parse_qs, unquote, urlparse
from loguru import logger from loguru import logger
from pydantic import Field, field_validator, model_validator from pydantic import Field, field_validator, model_validator
from websockets.asyncio.server import ServerConnection, serve, unix_serve from websockets.asyncio.server import ServerConnection, serve
from websockets.datastructures import Headers from websockets.datastructures import Headers
from websockets.exceptions import ConnectionClosed from websockets.exceptions import ConnectionClosed
from websockets.http11 import Request as WsRequest from websockets.http11 import Request as WsRequest
from websockets.http11 import Response from websockets.http11 import Response
from nanobot.security.workspace_access import ( from nanobot.agent.tools.mcp import request_mcp_reload
WORKSPACE_SCOPE_METADATA_KEY,
WorkspaceScopeError,
)
from nanobot.bus.events import OUTBOUND_META_AGENT_UI, OutboundMessage from nanobot.bus.events import OUTBOUND_META_AGENT_UI, OutboundMessage
from nanobot.bus.queue import MessageBus from nanobot.bus.queue import MessageBus
from nanobot.channels.base import BaseChannel from nanobot.channels.base import BaseChannel
@@ -39,20 +39,30 @@ from nanobot.config.paths import get_media_dir, get_workspace_path
from nanobot.config.schema import Base from nanobot.config.schema import Base
from nanobot.session.goal_state import goal_state_ws_blob from nanobot.session.goal_state import goal_state_ws_blob
from nanobot.session.webui_turns import websocket_turn_wall_started_at from nanobot.session.webui_turns import websocket_turn_wall_started_at
from nanobot.utils.helpers import safe_filename
from nanobot.utils.media_decode import ( from nanobot.utils.media_decode import (
FileSizeExceeded, FileSizeExceeded,
save_base64_data_url, save_base64_data_url,
) )
from nanobot.utils.subagent_channel_display import scrub_subagent_messages_for_channel from nanobot.utils.subagent_channel_display import scrub_subagent_messages_for_channel
from nanobot.webui.settings_api import runtime_capabilities from nanobot.webui.settings_api import (
from nanobot.webui.cli_apps_api import normalize_cli_app_mentions WebUISettingsError,
from nanobot.webui.media_api import ( create_model_configuration,
serve_signed_media, settings_payload,
sign_media_path, update_agent_settings,
sign_or_stage_media_path, update_image_generation_settings,
update_provider_settings,
update_web_search_settings,
)
from nanobot.webui.cli_apps_api import (
cli_apps_action,
cli_apps_payload,
normalize_cli_app_mentions,
)
from nanobot.webui.mcp_presets_api import (
mcp_presets_settings_action,
normalize_mcp_preset_mentions,
) )
from nanobot.webui.mcp_presets_api import normalize_mcp_preset_mentions
from nanobot.webui.settings_routes import WebUISettingsRouter
from nanobot.webui.sidebar_state import ( from nanobot.webui.sidebar_state import (
read_webui_sidebar_state, read_webui_sidebar_state,
write_webui_sidebar_state, write_webui_sidebar_state,
@@ -63,9 +73,18 @@ from nanobot.webui.transcript import (
build_webui_thread_response, build_webui_thread_response,
rewrite_local_markdown_images, rewrite_local_markdown_images,
) )
from nanobot.webui.workspaces import (
WebUIWorkspaceController, _MCP_PRESET_ACTIONS_BY_PATH = {
) "/api/settings/mcp-presets/enable": "enable",
"/api/settings/mcp-presets/remove": "remove",
"/api/settings/mcp-presets/test": "test",
"/api/settings/mcp-presets/custom": "custom",
"/api/settings/mcp-presets/import": "import",
"/api/settings/mcp-presets/import-cursor": "import-cursor",
"/api/settings/mcp-presets/tools": "tools",
}
_MCP_VALUES_HEADER = "X-Nanobot-MCP-Values"
_MCP_VALUES_HEADER_MAX_BYTES = 64 * 1024
if TYPE_CHECKING: if TYPE_CHECKING:
from nanobot.session.manager import SessionManager from nanobot.session.manager import SessionManager
@@ -81,41 +100,6 @@ def _normalize_config_path(path: str) -> str:
return _strip_trailing_slash(path) return _strip_trailing_slash(path)
def _case_insensitive_header(headers: Any, key: str) -> str:
"""Read a header from websockets/http test stubs without assuming casing."""
try:
value = headers.get(key)
except Exception:
value = None
if value is None:
try:
value = headers.get(key.lower())
except Exception:
value = None
return str(value or "").strip()
def _safe_host_header(value: str) -> str:
"""Return a safe Host header value, or empty when it should not be echoed."""
value = value.strip()
if not value:
return ""
if re.fullmatch(r"\[[0-9A-Fa-f:.]+\](?::\d{1,5})?", value):
return value
if re.fullmatch(r"[A-Za-z0-9.-]+(?::\d{1,5})?", value):
return value
return ""
def _host_for_url(host: str, port: int) -> str:
host = host.strip()
if host in ("0.0.0.0", "::"):
host = "127.0.0.1"
if ":" in host and not host.startswith("["):
host = f"[{host}]"
return f"{host}:{port}"
class WebSocketConfig(Base): class WebSocketConfig(Base):
"""WebSocket server channel configuration. """WebSocket server channel configuration.
@@ -139,7 +123,6 @@ class WebSocketConfig(Base):
enabled: bool = False enabled: bool = False
host: str = "127.0.0.1" host: str = "127.0.0.1"
port: int = 8765 port: int = 8765
unix_socket_path: str = ""
path: str = "/" path: str = "/"
token: str = "" token: str = ""
token_issue_path: str = "" token_issue_path: str = ""
@@ -158,19 +141,6 @@ class WebSocketConfig(Base):
ssl_certfile: str = "" ssl_certfile: str = ""
ssl_keyfile: str = "" ssl_keyfile: str = ""
@field_validator("unix_socket_path")
@classmethod
def unix_socket_path_format(cls, value: str) -> str:
value = value.strip()
if not value:
return ""
if "\x00" in value:
raise ValueError("unix_socket_path must not contain NUL bytes")
path = Path(value).expanduser()
if not path.is_absolute():
raise ValueError("unix_socket_path must be an absolute path")
return str(path)
@field_validator("path") @field_validator("path")
@classmethod @classmethod
def path_must_start_with_slash(cls, value: str) -> str: def path_must_start_with_slash(cls, value: str) -> str:
@@ -285,6 +255,34 @@ def _parse_query(path_with_query: str) -> dict[str, list[str]]:
return _parse_request_path(path_with_query)[1] return _parse_request_path(path_with_query)[1]
def _parse_mcp_settings_query(request: WsRequest) -> dict[str, list[str]]:
query = _parse_query(request.path)
raw = request.headers.get(_MCP_VALUES_HEADER)
if not raw:
return query
if len(raw.encode("utf-8")) > _MCP_VALUES_HEADER_MAX_BYTES:
raise WebUISettingsError("MCP settings payload is too large")
try:
payload = json.loads(raw)
except json.JSONDecodeError as exc:
raise WebUISettingsError("invalid MCP settings payload") from exc
if not isinstance(payload, dict):
raise WebUISettingsError("MCP settings payload must be a JSON object")
merged = {key: list(values) for key, values in query.items()}
for key, value in payload.items():
if not isinstance(key, str) or not key:
raise WebUISettingsError("MCP settings payload contains an invalid key")
if value is None:
continue
if isinstance(value, str):
text = value.strip()
else:
text = json.dumps(value, ensure_ascii=False, separators=(",", ":"))
if text:
merged[key] = [text]
return merged
def _query_first(query: dict[str, list[str]], key: str) -> str | None: def _query_first(query: dict[str, list[str]], key: str) -> str | None:
"""Return the first value for *key*, or None.""" """Return the first value for *key*, or None."""
values = query.get(key) values = query.get(key)
@@ -453,6 +451,30 @@ def _is_websocket_upgrade(request: WsRequest) -> bool:
return True return True
def _b64url_encode(data: bytes) -> str:
"""URL-safe base64 without padding — compact + friendly in URL paths."""
return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
def _b64url_decode(s: str) -> bytes:
"""Reverse of :func:`_b64url_encode`; caller handles ``ValueError``."""
pad = "=" * (-len(s) % 4)
return base64.urlsafe_b64decode(s + pad)
# Allowed MIME types we actually serve from the media endpoint. Anything
# outside this set is degraded to ``application/octet-stream`` so an
# attacker who somehow gets a signed URL for an unexpected file type can't
# trick the browser into sniffing executable content.
_MEDIA_ALLOWED_MIMES: frozenset[str] = frozenset({
"image/png",
"image/jpeg",
"image/webp",
"image/gif",
"video/mp4",
"video/webm",
"video/quicktime",
})
def _issue_route_secret_matches(headers: Any, configured_secret: str) -> bool: def _issue_route_secret_matches(headers: Any, configured_secret: str) -> bool:
"""Return True if the token-issue HTTP request carries credentials matching ``token_issue_secret``.""" """Return True if the token-issue HTTP request carries credentials matching ``token_issue_secret``."""
if not configured_secret: if not configured_secret:
@@ -481,10 +503,7 @@ class WebSocketChannel(BaseChannel):
session_manager: "SessionManager | None" = None, session_manager: "SessionManager | None" = None,
static_dist_path: Path | None = None, static_dist_path: Path | None = None,
workspace_path: Path | None = None, workspace_path: Path | None = None,
restrict_to_workspace: bool = False,
runtime_model_name: Callable[[], str | None] | None = None, runtime_model_name: Callable[[], str | None] | None = None,
runtime_surface: str = "browser",
runtime_capabilities_overrides: dict[str, Any] | None = None,
): ):
if isinstance(config, dict): if isinstance(config, dict):
config = WebSocketConfig.model_validate(config) config = WebSocketConfig.model_validate(config)
@@ -511,30 +530,8 @@ class WebSocketChannel(BaseChannel):
if workspace_path is not None if workspace_path is not None
else get_workspace_path() else get_workspace_path()
).resolve(strict=False) ).resolve(strict=False)
self._default_restrict_to_workspace = restrict_to_workspace
self._webui_workspaces = WebUIWorkspaceController(
session_manager=self._session_manager,
default_workspace=self._workspace_path,
default_restrict_to_workspace=self._default_restrict_to_workspace,
)
self._runtime_model_name = runtime_model_name self._runtime_model_name = runtime_model_name
self._runtime_surface = ( self._settings_restart_sections: set[str] = set()
"native" if runtime_surface in {"native", "desktop"} else "browser"
)
self._runtime_capabilities = runtime_capabilities(
self._runtime_surface,
runtime_capabilities_overrides,
)
self._settings_routes = WebUISettingsRouter(
bus=self.bus,
logger=self.logger,
check_api_token=self._check_api_token,
parse_query=_parse_query,
json_response=_http_json_response,
error_response=_http_error,
runtime_surface=self._runtime_surface,
runtime_capabilities=self._runtime_capabilities,
)
self._stream_text_buffers: dict[tuple[str, str], list[str]] = {} self._stream_text_buffers: dict[tuple[str, str], list[str]] = {}
# Process-local secret used to HMAC-sign media URLs. The signed URL is # Process-local secret used to HMAC-sign media URLs. The signed URL is
# the capability — anyone who holds a valid URL can fetch that one # the capability — anyone who holds a valid URL can fetch that one
@@ -678,91 +675,69 @@ class WebSocketChannel(BaseChannel):
"""Route an inbound HTTP request to a handler or to the WS upgrade path.""" """Route an inbound HTTP request to a handler or to the WS upgrade path."""
got, query = _parse_request_path(request.path) got, query = _parse_request_path(request.path)
# 1. Token issue endpoint (legacy, optional, gated by configured secret).
if self.config.token_issue_path: if self.config.token_issue_path:
issue_expected = _normalize_config_path(self.config.token_issue_path) issue_expected = _normalize_config_path(self.config.token_issue_path)
if got == issue_expected: if got == issue_expected:
return self._handle_token_issue_http(connection, request) return self._handle_token_issue_http(connection, request)
# 2. Bootstrap (`/webui/bootstrap`): mint WS/API tokens + shared session metadata.
if got == "/webui/bootstrap": if got == "/webui/bootstrap":
return self._handle_bootstrap(connection, request) return self._handle_bootstrap(connection, request)
api_response = await self._dispatch_api_route(connection, request, got) # 3. REST handlers co-located with this channel (sessions, settings, …).
if api_response is not None:
return api_response
ws_matched, ws_response = self._dispatch_websocket_upgrade(
connection, request, got, query
)
if ws_matched:
return ws_response
# API clients should never receive the SPA shell for an unknown route.
# Returning HTML here makes the WebUI fail with "Unexpected token <"
# when a dev server is pointed at an older gateway.
if got.startswith("/api/"):
return _http_error(404, "API route not found")
if self._static_dist_path is not None:
response = self._serve_static(got)
if response is not None:
return response
return connection.respond(404, "Not Found")
async def _dispatch_api_route(
self,
connection: Any,
request: WsRequest,
got: str,
) -> Any | None:
"""Route REST-ish WebUI requests served beside the WebSocket endpoint."""
response = await self._dispatch_settings_api_route(request, got)
if response is not None:
return response
response = self._dispatch_session_api_route(request, got)
if response is not None:
return response
response = self._dispatch_media_api_route(request, got)
if response is not None:
return response
return self._dispatch_misc_api_route(connection, request, got)
def _dispatch_misc_api_route(
self,
connection: Any,
request: WsRequest,
got: str,
) -> Response | None:
"""Route small API endpoints that do not belong to a larger route group."""
if got == "/api/sessions": if got == "/api/sessions":
return self._handle_sessions_list(request) return self._handle_sessions_list(request)
if got == "/api/settings":
return self._handle_settings(request)
if got == "/api/commands": if got == "/api/commands":
return self._handle_commands(request) return self._handle_commands(request)
if got == "/api/workspaces":
return self._handle_workspaces(connection, request)
if got == "/api/webui/sidebar-state": if got == "/api/webui/sidebar-state":
return self._handle_webui_sidebar_state(request) return self._handle_webui_sidebar_state(request)
if got == "/api/webui/sidebar-state/update": if got == "/api/webui/sidebar-state/update":
return self._handle_webui_sidebar_state_update(request) return self._handle_webui_sidebar_state_update(request)
return None if got == "/api/settings/update":
return self._handle_settings_update(request)
async def _dispatch_settings_api_route( if got == "/api/settings/model-configurations/create":
self, return self._handle_settings_model_configuration_create(request)
request: WsRequest,
got: str, if got == "/api/settings/provider/update":
) -> Response | None: return self._handle_settings_provider_update(request)
return await self._settings_routes.dispatch(request, got)
if got == "/api/settings/web-search/update":
return self._handle_settings_web_search_update(request)
if got == "/api/settings/image-generation/update":
return self._handle_settings_image_generation_update(request)
if got == "/api/settings/cli-apps":
return self._handle_settings_cli_apps(request)
if got == "/api/settings/cli-apps/install":
return await self._handle_settings_cli_apps_action(request, "install")
if got == "/api/settings/cli-apps/update":
return await self._handle_settings_cli_apps_action(request, "update")
if got == "/api/settings/cli-apps/uninstall":
return await self._handle_settings_cli_apps_action(request, "uninstall")
if got == "/api/settings/cli-apps/test":
return await self._handle_settings_cli_apps_action(request, "test")
if got == "/api/settings/mcp-presets":
return await self._handle_settings_mcp_presets(request)
mcp_action = _MCP_PRESET_ACTIONS_BY_PATH.get(got)
if mcp_action is not None:
return await self._handle_settings_mcp_presets(request, mcp_action)
def _dispatch_session_api_route(
self,
request: WsRequest,
got: str,
) -> Response | None:
m = re.match(r"^/api/sessions/([^/]+)/messages$", got) m = re.match(r"^/api/sessions/([^/]+)/messages$", got)
if m: if m:
return self._handle_session_messages(request, m.group(1)) return self._handle_session_messages(request, m.group(1))
@@ -777,36 +752,34 @@ class WebSocketChannel(BaseChannel):
if m: if m:
return self._handle_session_delete(request, m.group(1)) return self._handle_session_delete(request, m.group(1))
return None # Signed media fetch: ``<sig>`` is an HMAC over ``<payload>``; the
# payload decodes to a path inside :func:`get_media_dir`. See
def _dispatch_media_api_route( # :meth:`_sign_media_path` for the inverse direction used to build
self, # these URLs when replaying a session.
request: WsRequest,
got: str,
) -> Response | None:
m = re.match(r"^/api/media/([A-Za-z0-9_-]+)/([A-Za-z0-9_-]+)$", got) m = re.match(r"^/api/media/([A-Za-z0-9_-]+)/([A-Za-z0-9_-]+)$", got)
if m: if m:
return self._handle_media_fetch(m.group(1), m.group(2), request) return self._handle_media_fetch(m.group(1), m.group(2))
return None # 4. WebSocket upgrade (the channel's primary purpose). Only run the
# handshake gate on requests that actually ask to upgrade; otherwise
def _dispatch_websocket_upgrade( # a bare ``GET /`` from the browser would be rejected as an
self, # unauthorized WS handshake instead of serving the SPA's index.html.
connection: Any,
request: WsRequest,
got: str,
query: dict[str, list[str]],
) -> tuple[bool, Any | None]:
"""Authorize only real WS upgrade requests for the configured path."""
expected_ws = self._expected_path() expected_ws = self._expected_path()
if got != expected_ws or not _is_websocket_upgrade(request): if got == expected_ws and _is_websocket_upgrade(request):
return False, None client_id = _query_first(query, "client_id") or ""
client_id = _query_first(query, "client_id") or "" if len(client_id) > 128:
if len(client_id) > 128: client_id = client_id[:128]
client_id = client_id[:128] if not self.is_allowed(client_id):
if not self.is_allowed(client_id): return connection.respond(403, "Forbidden")
return True, connection.respond(403, "Forbidden") return self._authorize_websocket_handshake(connection, query)
return True, self._authorize_websocket_handshake(connection, query)
# 5. Static SPA serving (only if a build directory was wired in).
if self._static_dist_path is not None:
response = self._serve_static(got)
if response is not None:
return response
return connection.respond(404, "Not Found")
# -- HTTP route handlers ------------------------------------------------ # -- HTTP route handlers ------------------------------------------------
@@ -859,32 +832,15 @@ class WebSocketChannel(BaseChannel):
# while the REST surface keeps validating the other until TTL expiry. # while the REST surface keeps validating the other until TTL expiry.
self._issued_tokens[token] = expiry self._issued_tokens[token] = expiry
self._api_tokens[token] = expiry self._api_tokens[token] = expiry
ws_url = self._bootstrap_ws_url(request)
return _http_json_response( return _http_json_response(
{ {
"token": token, "token": token,
"ws_path": self._expected_path(), "ws_path": self._expected_path(),
"ws_url": ws_url,
"expires_in": self.config.token_ttl_s, "expires_in": self.config.token_ttl_s,
"model_name": _resolve_bootstrap_model_name(self._runtime_model_name), "model_name": _resolve_bootstrap_model_name(self._runtime_model_name),
"runtime_surface": self._runtime_surface,
"runtime_capabilities": self._runtime_capabilities,
} }
) )
def _bootstrap_ws_url(self, request: Any) -> str:
"""Absolute WS URL clients should prefer over a dev-server proxy."""
headers = getattr(request, "headers", {}) or {}
host = _safe_host_header(_case_insensitive_header(headers, "Host"))
if not host:
host = _host_for_url(self.config.host, self.config.port)
proto = _case_insensitive_header(headers, "X-Forwarded-Proto")
proto = proto.split(",", 1)[0].strip().lower()
secure = proto in {"https", "wss"} or bool(self.config.ssl_certfile.strip())
scheme = "wss" if secure else "ws"
return f"{scheme}://{host}{self._expected_path()}"
def _handle_sessions_list(self, request: WsRequest) -> Response: def _handle_sessions_list(self, request: WsRequest) -> Response:
if not self._check_api_token(request): if not self._check_api_token(request):
return _http_error(401, "Unauthorized") return _http_error(401, "Unauthorized")
@@ -903,17 +859,31 @@ class WebSocketChannel(BaseChannel):
started_at = websocket_turn_wall_started_at(chat_id) started_at = websocket_turn_wall_started_at(chat_id)
if started_at is not None: if started_at is not None:
row["run_started_at"] = started_at row["run_started_at"] = started_at
scope = self._webui_workspaces.scope_for_session_key(key)
row["workspace_scope"] = scope.payload()
cleaned.append(row) cleaned.append(row)
return _http_json_response({"sessions": cleaned}) return _http_json_response({"sessions": cleaned})
def _handle_workspaces(self, connection: Any, request: WsRequest) -> Response: def _handle_settings(self, request: WsRequest) -> Response:
if not self._check_api_token(request): if not self._check_api_token(request):
return _http_error(401, "Unauthorized") return _http_error(401, "Unauthorized")
return _http_json_response( return _http_json_response(self._with_settings_restart_state(settings_payload()))
self._webui_workspaces.payload(controls_available=_is_localhost(connection))
) def _with_settings_restart_state(
self,
payload: dict[str, Any],
*,
section: str | None = None,
) -> dict[str, Any]:
"""Keep restart-required state alive for this gateway process."""
if section and payload.get("requires_restart"):
self._settings_restart_sections.add(section)
if self._settings_restart_sections:
payload = dict(payload)
payload["requires_restart"] = True
payload["restart_required_sections"] = sorted(self._settings_restart_sections)
else:
payload = dict(payload)
payload["restart_required_sections"] = []
return payload
def _handle_commands(self, request: WsRequest) -> Response: def _handle_commands(self, request: WsRequest) -> Response:
if not self._check_api_token(request): if not self._check_api_token(request):
@@ -947,7 +917,108 @@ class WebSocketChannel(BaseChannel):
return _http_error(500, "failed to write sidebar state") return _http_error(500, "failed to write sidebar state")
return _http_json_response(state) return _http_json_response(state)
# -- Session replay, transcript, and signed media ---------------------- def _handle_settings_update(self, request: WsRequest) -> Response:
if not self._check_api_token(request):
return _http_error(401, "Unauthorized")
query = _parse_query(request.path)
try:
payload = update_agent_settings(query)
except WebUISettingsError as e:
return _http_error(e.status, e.message)
return _http_json_response(
self._with_settings_restart_state(payload, section="runtime")
)
def _handle_settings_model_configuration_create(self, request: WsRequest) -> Response:
if not self._check_api_token(request):
return _http_error(401, "Unauthorized")
query = _parse_query(request.path)
try:
payload = create_model_configuration(query)
except WebUISettingsError as e:
return _http_error(e.status, e.message)
return _http_json_response(self._with_settings_restart_state(payload))
def _handle_settings_provider_update(self, request: WsRequest) -> Response:
if not self._check_api_token(request):
return _http_error(401, "Unauthorized")
query = _parse_query(request.path)
try:
payload = update_provider_settings(query)
except WebUISettingsError as e:
return _http_error(e.status, e.message)
return _http_json_response(self._with_settings_restart_state(payload, section="image"))
def _handle_settings_web_search_update(self, request: WsRequest) -> Response:
if not self._check_api_token(request):
return _http_error(401, "Unauthorized")
query = _parse_query(request.path)
try:
payload = update_web_search_settings(query)
except WebUISettingsError as e:
return _http_error(e.status, e.message)
return _http_json_response(self._with_settings_restart_state(payload, section="web"))
def _handle_settings_image_generation_update(self, request: WsRequest) -> Response:
if not self._check_api_token(request):
return _http_error(401, "Unauthorized")
query = _parse_query(request.path)
try:
payload = update_image_generation_settings(query)
except WebUISettingsError as e:
return _http_error(e.status, e.message)
return _http_json_response(self._with_settings_restart_state(payload, section="image"))
def _handle_settings_cli_apps(self, request: WsRequest) -> Response:
if not self._check_api_token(request):
return _http_error(401, "Unauthorized")
try:
payload = cli_apps_payload()
except Exception:
self.logger.exception("failed to load CLI Apps payload")
return _http_error(500, "failed to load CLI Apps")
return _http_json_response(payload)
async def _handle_settings_cli_apps_action(self, request: WsRequest, action: str) -> Response:
if not self._check_api_token(request):
return _http_error(401, "Unauthorized")
query = _parse_query(request.path)
try:
payload = await asyncio.to_thread(cli_apps_action, action, query)
except WebUISettingsError as e:
return _http_error(e.status, e.message)
except Exception as e:
status = getattr(e, "status", 500)
message = getattr(e, "message", str(e))
if status >= 500:
self.logger.exception("CLI Apps action '{}' failed", action)
return _http_error(status, message)
return _http_json_response(payload)
async def _handle_settings_mcp_presets(
self,
request: WsRequest,
action: str | None = None,
) -> Response:
if not self._check_api_token(request):
return _http_error(401, "Unauthorized")
try:
payload = await mcp_presets_settings_action(
action,
_parse_mcp_settings_query(request),
reload_mcp=lambda: request_mcp_reload(self.bus),
)
except Exception as e:
status = getattr(e, "status", 500)
message = getattr(e, "message", str(e))
if status >= 500:
self.logger.exception("MCP preset action '{}' failed", action or "list")
return _http_error(status, message)
if action is None:
return _http_json_response(payload)
return _http_json_response(
self._with_settings_restart_state(payload, section="runtime")
)
@staticmethod @staticmethod
def _is_websocket_channel_session_key(key: str) -> bool: def _is_websocket_channel_session_key(key: str) -> bool:
@@ -987,19 +1058,13 @@ class WebSocketChannel(BaseChannel):
return _http_error(400, "invalid session key") return _http_error(400, "invalid session key")
if not self._is_websocket_channel_session_key(decoded_key): if not self._is_websocket_channel_session_key(decoded_key):
return _http_error(404, "session not found") return _http_error(404, "session not found")
scope = self._webui_workspaces.scope_for_session_key(decoded_key)
data = build_webui_thread_response( data = build_webui_thread_response(
decoded_key, decoded_key,
augment_user_media=self._augment_transcript_user_media, augment_user_media=self._augment_transcript_user_media,
augment_assistant_text=lambda text: rewrite_local_markdown_images( augment_assistant_text=self._rewrite_local_markdown_images,
text,
workspace_path=scope.project_path,
sign_path=self._sign_or_stage_media_path,
),
) )
if data is None: if data is None:
return _http_error(404, "webui thread not found") return _http_error(404, "webui thread not found")
data["workspace_scope"] = scope.payload()
return _http_json_response(data) return _http_json_response(data)
def _try_append_webui_transcript(self, chat_id: str, wire: dict[str, Any]) -> None: def _try_append_webui_transcript(self, chat_id: str, wire: dict[str, Any]) -> None:
@@ -1100,11 +1165,16 @@ class WebSocketChannel(BaseChannel):
be fetched. The returned path is relative to the server origin; the be fetched. The returned path is relative to the server origin; the
client joins it against this server's HTTP origin (same host as WS). client joins it against this server's HTTP origin (same host as WS).
""" """
return sign_media_path( try:
abs_path, media_root = get_media_dir().resolve()
secret=self._media_secret, rel = abs_path.resolve().relative_to(media_root)
media_dir=lambda channel=None: get_media_dir(channel), except (OSError, ValueError):
) return None
payload = _b64url_encode(rel.as_posix().encode("utf-8"))
mac = hmac.new(
self._media_secret, payload.encode("ascii"), hashlib.sha256
).digest()[:16]
return f"/api/media/{_b64url_encode(mac)}/{payload}"
def _sign_or_stage_media_path(self, path: Path) -> dict[str, str] | None: def _sign_or_stage_media_path(self, path: Path) -> dict[str, str] | None:
"""Return a signed media URL payload for *path*. """Return a signed media URL payload for *path*.
@@ -1115,12 +1185,23 @@ class WebSocketChannel(BaseChannel):
can fetch them through the existing signed media route without can fetch them through the existing signed media route without
exposing arbitrary filesystem paths. exposing arbitrary filesystem paths.
""" """
return sign_or_stage_media_path( signed = self._sign_media_path(path)
path, if signed is not None:
secret=self._media_secret, return {"url": signed, "name": path.name}
media_dir=lambda channel=None: get_media_dir(channel), try:
logger=self.logger, if not path.is_file():
) return None
media_dir = get_media_dir("websocket")
safe_name = safe_filename(path.name) or "attachment"
staged = media_dir / f"{uuid.uuid4().hex[:12]}-{safe_name}"
shutil.copyfile(path, staged)
except OSError as exc:
self.logger.warning("failed to stage outbound media {}: {}", path, exc)
return None
signed = self._sign_media_path(staged)
if signed is None:
return None
return {"url": signed, "name": path.name}
def _rewrite_local_markdown_images(self, text: str) -> str: def _rewrite_local_markdown_images(self, text: str) -> str:
return rewrite_local_markdown_images( return rewrite_local_markdown_images(
@@ -1129,20 +1210,52 @@ class WebSocketChannel(BaseChannel):
sign_path=self._sign_or_stage_media_path, sign_path=self._sign_or_stage_media_path,
) )
def _handle_media_fetch( def _handle_media_fetch(self, sig: str, payload: str) -> Response:
self, sig: str, payload: str, request: WsRequest | None = None
) -> Response:
"""Serve a single media file previously signed via """Serve a single media file previously signed via
:meth:`_sign_media_path`. Validates the signature, decodes the :meth:`_sign_media_path`. Validates the signature, decodes the
payload to a relative path, and streams the file bytes with a payload to a relative path, and streams the file bytes with a
long-lived immutable cache header (the URL already encodes the long-lived immutable cache header (the URL already encodes the
file identity, so caches can be aggressive).""" file identity, so caches can be aggressive)."""
return serve_signed_media( try:
sig, provided_mac = _b64url_decode(sig)
payload, except (ValueError, binascii.Error):
secret=self._media_secret, return _http_error(401, "invalid signature")
request=request, expected_mac = hmac.new(
media_dir=lambda channel=None: get_media_dir(channel), self._media_secret, payload.encode("ascii"), hashlib.sha256
).digest()[:16]
if not hmac.compare_digest(expected_mac, provided_mac):
return _http_error(401, "invalid signature")
try:
rel_bytes = _b64url_decode(payload)
rel_str = rel_bytes.decode("utf-8")
except (ValueError, binascii.Error, UnicodeDecodeError):
return _http_error(400, "invalid payload")
# An attacker who somehow bypassed the HMAC check would still need
# the resolved path to escape the media root; guard defensively.
try:
media_root = get_media_dir().resolve()
candidate = (media_root / rel_str).resolve()
candidate.relative_to(media_root)
except (OSError, ValueError):
return _http_error(404, "not found")
if not candidate.is_file():
return _http_error(404, "not found")
try:
body = candidate.read_bytes()
except OSError:
return _http_error(500, "read error")
mime, _ = mimetypes.guess_type(candidate.name)
if mime not in _MEDIA_ALLOWED_MIMES:
mime = "application/octet-stream"
return _http_response(
body,
content_type=mime,
extra_headers=[
("Cache-Control", "private, max-age=31536000, immutable"),
# Paired with the MIME whitelist above: prevents browsers from
# MIME-sniffing an octet-stream fallback into executable HTML.
("X-Content-Type-Options", "nosniff"),
],
) )
def _handle_session_delete(self, request: WsRequest, key: str) -> Response: def _handle_session_delete(self, request: WsRequest, key: str) -> Response:
@@ -1161,8 +1274,6 @@ class WebSocketChannel(BaseChannel):
delete_webui_thread(decoded_key) delete_webui_thread(decoded_key)
return _http_json_response({"deleted": bool(deleted)}) return _http_json_response({"deleted": bool(deleted)})
# -- Static files and WebSocket handshake ------------------------------
def _serve_static(self, request_path: str) -> Response | None: def _serve_static(self, request_path: str) -> Response | None:
"""Resolve *request_path* against the built SPA directory; SPA fallback to index.html.""" """Resolve *request_path* against the built SPA directory; SPA fallback to index.html."""
assert self._static_dist_path is not None assert self._static_dist_path is not None
@@ -1227,8 +1338,6 @@ class WebSocketChannel(BaseChannel):
self._take_issued_token_if_valid(supplied) self._take_issued_token_if_valid(supplied)
return None return None
# -- Server lifecycle and connection ingress ---------------------------
async def start(self) -> None: async def start(self) -> None:
from nanobot.utils.logging_bridge import redirect_lib_logging from nanobot.utils.logging_bridge import redirect_lib_logging
@@ -1250,63 +1359,34 @@ class WebSocketChannel(BaseChannel):
await self._connection_loop(connection) await self._connection_loop(connection)
self.logger.info( self.logger.info(
"WebSocket server listening on {}", "WebSocket server listening on {}://{}:{}{}",
( scheme,
f"unix:{self.config.unix_socket_path}{self.config.path}" self.config.host,
if self.config.unix_socket_path self.config.port,
else f"{scheme}://{self.config.host}:{self.config.port}{self.config.path}" self.config.path,
),
) )
if self.config.token_issue_path: if self.config.token_issue_path:
self.logger.info( self.logger.info(
"WebSocket token issue route: {}", "WebSocket token issue route: {}://{}:{}{}",
( scheme,
f"unix:{self.config.unix_socket_path}{_normalize_config_path(self.config.token_issue_path)}" self.config.host,
if self.config.unix_socket_path self.config.port,
else ( _normalize_config_path(self.config.token_issue_path),
f"{scheme}://{self.config.host}:{self.config.port}"
f"{_normalize_config_path(self.config.token_issue_path)}"
)
),
) )
async def runner() -> None: async def runner() -> None:
socket_path = self.config.unix_socket_path async with serve(
if socket_path: handler,
path_obj = Path(socket_path) self.config.host,
path_obj.parent.mkdir(parents=True, exist_ok=True) self.config.port,
with suppress(FileNotFoundError): process_request=process_request,
path_obj.unlink() max_size=self.config.max_message_bytes,
server = await unix_serve( ping_interval=self.config.ping_interval_s,
handler, ping_timeout=self.config.ping_timeout_s,
socket_path, ssl=ssl_context,
process_request=process_request, ):
max_size=self.config.max_message_bytes,
ping_interval=self.config.ping_interval_s,
ping_timeout=self.config.ping_timeout_s,
)
with suppress(OSError):
path_obj.chmod(0o600)
else:
server = await serve(
handler,
self.config.host,
self.config.port,
process_request=process_request,
max_size=self.config.max_message_bytes,
ping_interval=self.config.ping_interval_s,
ping_timeout=self.config.ping_timeout_s,
ssl=ssl_context,
)
try:
assert self._stop_event is not None assert self._stop_event is not None
await self._stop_event.wait() await self._stop_event.wait()
finally:
server.close()
await server.wait_closed()
if socket_path:
with suppress(FileNotFoundError):
Path(socket_path).unlink()
self._server_task = asyncio.create_task(runner()) self._server_task = asyncio.create_task(runner())
await self._server_task await self._server_task
@@ -1372,8 +1452,6 @@ class WebSocketChannel(BaseChannel):
finally: finally:
self._cleanup_connection(connection) self._cleanup_connection(connection)
# -- Inbound WebSocket envelopes ---------------------------------------
def _save_envelope_media( def _save_envelope_media(
self, self,
media: list[Any], media: list[Any],
@@ -1452,25 +1530,8 @@ class WebSocketChannel(BaseChannel):
t = envelope.get("type") t = envelope.get("type")
if t == "new_chat": if t == "new_chat":
new_id = str(uuid.uuid4()) new_id = str(uuid.uuid4())
scope = await self._workspace_scope_or_error(
connection,
lambda: self._webui_workspaces.scope_for_new_chat(
envelope,
controls_available=_is_localhost(connection),
),
)
if scope is None:
return
self._webui_workspaces.persist_scope(new_id, scope)
self._attach(connection, new_id) self._attach(connection, new_id)
await self._send_event(connection, "attached", chat_id=new_id) await self._send_event(connection, "attached", chat_id=new_id)
await self._send_event(
connection,
"session_updated",
chat_id=new_id,
scope="metadata",
workspace_scope=scope.payload(),
)
await self._hydrate_after_subscribe(new_id) await self._hydrate_after_subscribe(new_id)
return return
if t == "attach": if t == "attach":
@@ -1482,32 +1543,6 @@ class WebSocketChannel(BaseChannel):
await self._send_event(connection, "attached", chat_id=cid) await self._send_event(connection, "attached", chat_id=cid)
await self._hydrate_after_subscribe(cid) await self._hydrate_after_subscribe(cid)
return return
if t == "set_workspace_scope":
cid = envelope.get("chat_id")
if not _is_valid_chat_id(cid):
await self._send_event(connection, "error", detail="invalid chat_id")
return
scope = await self._workspace_scope_or_error(
connection,
lambda: self._webui_workspaces.scope_for_set_request(
envelope,
chat_id=cid,
chat_running=websocket_turn_wall_started_at(cid) is not None,
controls_available=_is_localhost(connection),
),
chat_id=cid,
)
if scope is None:
return
self._webui_workspaces.persist_scope(cid, scope)
await self._send_event(
connection,
"session_updated",
chat_id=cid,
scope="metadata",
workspace_scope=scope.payload(),
)
return
if t == "message": if t == "message":
cid = envelope.get("chat_id") cid = envelope.get("chat_id")
content = envelope.get("content") content = envelope.get("content")
@@ -1539,18 +1574,6 @@ class WebSocketChannel(BaseChannel):
if not content.strip() and not media_paths: if not content.strip() and not media_paths:
await self._send_event(connection, "error", detail="missing content") await self._send_event(connection, "error", detail="missing content")
return return
scope = await self._workspace_scope_or_error(
connection,
lambda: self._webui_workspaces.scope_for_message(
envelope,
chat_id=cid,
chat_running=websocket_turn_wall_started_at(cid) is not None,
controls_available=_is_localhost(connection),
),
chat_id=cid,
)
if scope is None:
return
# Auto-attach on first use so clients can one-shot without a separate attach. # Auto-attach on first use so clients can one-shot without a separate attach.
self._attach(connection, cid) self._attach(connection, cid)
@@ -1564,8 +1587,6 @@ class WebSocketChannel(BaseChannel):
mcp_presets = normalize_mcp_preset_mentions(envelope.get("mcp_presets")) mcp_presets = normalize_mcp_preset_mentions(envelope.get("mcp_presets"))
if mcp_presets: if mcp_presets:
metadata["mcp_presets"] = mcp_presets metadata["mcp_presets"] = mcp_presets
metadata[WORKSPACE_SCOPE_METADATA_KEY] = scope.metadata()
self._webui_workspaces.persist_scope(cid, scope)
image_generation = envelope.get("image_generation") image_generation = envelope.get("image_generation")
if isinstance(image_generation, dict) and image_generation.get("enabled") is True: if isinstance(image_generation, dict) and image_generation.get("enabled") is True:
aspect_ratio = image_generation.get("aspect_ratio") aspect_ratio = image_generation.get("aspect_ratio")
@@ -1584,27 +1605,6 @@ class WebSocketChannel(BaseChannel):
return return
await self._send_event(connection, "error", detail=f"unknown type: {t!r}") await self._send_event(connection, "error", detail=f"unknown type: {t!r}")
async def _workspace_scope_or_error(
self,
connection: Any,
resolver: Callable[[], Any],
*,
chat_id: str | None = None,
) -> Any | None:
try:
return resolver()
except WorkspaceScopeError as exc:
await self._send_event(
connection,
"error",
detail="workspace_scope_rejected",
reason=exc.message,
**({"chat_id": chat_id} if chat_id else {}),
)
return None
# -- Outbound WebSocket events -----------------------------------------
async def stop(self) -> None: async def stop(self) -> None:
if not self._running: if not self._running:
return return
+89 -254
View File
@@ -1,7 +1,6 @@
"""CLI commands for nanobot.""" """CLI commands for nanobot."""
import asyncio import asyncio
import functools
import os import os
import select import select
import signal import signal
@@ -76,7 +75,6 @@ class SafeFileHistory(FileHistory):
from nanobot.cli.stream import StreamRenderer, ThinkingSpinner from nanobot.cli.stream import StreamRenderer, ThinkingSpinner
from nanobot.config.paths import get_workspace_path, is_default_workspace from nanobot.config.paths import get_workspace_path, is_default_workspace
from nanobot.config.schema import Config from nanobot.config.schema import Config
from nanobot.utils.evaluator import evaluate_response
from nanobot.utils.helpers import sync_workspace_templates from nanobot.utils.helpers import sync_workspace_templates
from nanobot.utils.restart import ( from nanobot.utils.restart import (
consume_restart_notice_from_env, consume_restart_notice_from_env,
@@ -96,20 +94,6 @@ EXIT_COMMANDS = {"exit", "quit", "/exit", "/quit", ":q"}
_REASONING_SENTENCE_ENDINGS = (".", "!", "?", "", "", "") _REASONING_SENTENCE_ENDINGS = (".", "!", "?", "", "", "")
_REASONING_FLUSH_CHARS = 60 _REASONING_FLUSH_CHARS = 60
_HEARTBEAT_PREAMBLE = (
"[Your response will be delivered directly to the user's messaging app. "
"Output ONLY the final user-facing message. Never reference internal "
"files (HEARTBEAT.md, AWARENESS.md, etc.), your instructions, or your "
"decision process. If nothing needs reporting, respond with a brief "
"no-op status and nothing else.]\n\n"
)
@functools.lru_cache(maxsize=None)
def _heartbeat_template() -> str | None:
from nanobot.utils.helpers import load_bundled_template
return load_bundled_template("HEARTBEAT.md")
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# CLI input: prompt_toolkit for editing, paste, history, and display # CLI input: prompt_toolkit for editing, paste, history, and display
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -720,144 +704,11 @@ def gateway(
_run_gateway(cfg, port=port) _run_gateway(cfg, port=port)
def _load_or_create_desktop_config(config: str | None, workspace: str | None) -> Config:
"""Load the desktop-owned config, creating it on first launch."""
from nanobot.config.loader import (
get_config_path,
load_config,
resolve_config_env_vars,
save_config,
set_config_path,
)
from nanobot.config.schema import Config as NanobotConfig
config_path = Path(config).expanduser().resolve() if config else get_config_path()
set_config_path(config_path)
created = False
if config_path.exists():
try:
loaded = resolve_config_env_vars(load_config(config_path))
except ValueError as e:
console.print(f"[red]Error: {e}[/red]")
raise typer.Exit(1)
else:
loaded = NanobotConfig()
created = True
if workspace:
workspace_path = Path(workspace).expanduser()
loaded.agents.defaults.workspace = str(workspace_path)
created = True
if created:
save_config(loaded, config_path)
return loaded
def _configure_desktop_gateway(
config: Config,
*,
webui_port: int,
webui_socket: str | None,
token_issue_secret: str,
) -> None:
"""Force a local WebSocket-only gateway for the desktop app process."""
config.gateway.host = "127.0.0.1"
config.gateway.port = webui_port
config.gateway.heartbeat.enabled = False
extras = dict(getattr(config.channels, "__pydantic_extra__", None) or {})
for name, section in list(extras.items()):
if name == "websocket":
continue
if isinstance(section, dict):
extras[name] = {**section, "enabled": False}
else:
with suppress(Exception):
setattr(section, "enabled", False)
extras[name] = section
websocket_cfg = extras.get("websocket")
if not isinstance(websocket_cfg, dict):
websocket_cfg = {}
websocket_cfg.update(
{
"enabled": True,
"host": "127.0.0.1",
"port": webui_port,
"unix_socket_path": webui_socket or "",
"path": "/",
"token_issue_secret": token_issue_secret,
"websocket_requires_token": True,
"allow_from": ["*"],
"streaming": True,
}
)
extras["websocket"] = websocket_cfg
config.channels.__pydantic_extra__ = extras
@app.command("desktop-gateway", hidden=True)
def desktop_gateway(
webui_port: int = typer.Option(0, "--webui-port", min=0, max=65535),
webui_socket: str | None = typer.Option(None, "--webui-socket", help="Unix socket path for desktop IPC"),
token_issue_secret: str = typer.Option(..., "--token-issue-secret"),
workspace: str | None = typer.Option(None, "--workspace", "-w", help="Desktop workspace directory"),
config: str | None = typer.Option(None, "--config", "-c", help="Desktop config file"),
verbose: bool = typer.Option(False, "--verbose", "-v", help="Verbose output"),
):
"""Start the private local gateway used by nanobot Desktop."""
if not token_issue_secret.strip():
console.print("[red]Error: --token-issue-secret is required[/red]")
raise typer.Exit(1)
if webui_port <= 0 and not (webui_socket or "").strip():
console.print("[red]Error: --webui-port or --webui-socket is required[/red]")
raise typer.Exit(1)
if verbose:
logger.remove(_log_handler_id)
logger.add(
sys.stderr,
format=(
"<green>{time:YYYY-MM-DD HH:mm:ss}</green> | "
"<level>{level: <5}</level> | "
"<cyan>{extra[channel]}</cyan> | "
"<level>{message}</level>"
),
level="DEBUG",
colorize=None,
filter=lambda record: record["extra"].setdefault("channel", "-") or True,
)
cfg = _load_or_create_desktop_config(config, workspace)
_configure_desktop_gateway(
cfg,
webui_port=webui_port,
webui_socket=webui_socket,
token_issue_secret=token_issue_secret,
)
_run_gateway(
cfg,
port=webui_port,
webui_static_dist=False,
webui_runtime_surface="native",
webui_runtime_capabilities={
"can_restart_engine": True,
"can_pick_folder": True,
"can_open_logs": True,
"can_export_diagnostics": True,
},
health_server_enabled=False,
)
def _run_gateway( def _run_gateway(
config: Config, config: Config,
*, *,
port: int | None = None, port: int | None = None,
open_browser_url: str | None = None, open_browser_url: str | None = None,
webui_static_dist: bool = True,
webui_runtime_surface: str = "browser",
webui_runtime_capabilities: dict[str, Any] | None = None,
health_server_enabled: bool = True,
) -> None: ) -> None:
"""Shared gateway runtime; ``open_browser_url`` opens a tab once channels are up.""" """Shared gateway runtime; ``open_browser_url`` opens a tab once channels are up."""
from nanobot.agent.tools.cron import CronTool from nanobot.agent.tools.cron import CronTool
@@ -867,6 +718,7 @@ def _run_gateway(
from nanobot.channels.websocket import publish_runtime_model_update from nanobot.channels.websocket import publish_runtime_model_update
from nanobot.cron.service import CronService from nanobot.cron.service import CronService
from nanobot.cron.types import CronJob from nanobot.cron.types import CronJob
from nanobot.heartbeat.service import HeartbeatService
from nanobot.providers.factory import build_provider_snapshot, load_provider_snapshot from nanobot.providers.factory import build_provider_snapshot, load_provider_snapshot
from nanobot.providers.image_generation import image_gen_provider_configs from nanobot.providers.image_generation import image_gen_provider_configs
from nanobot.session.manager import SessionManager from nanobot.session.manager import SessionManager
@@ -910,7 +762,7 @@ def _run_gateway(
) )
from nanobot.agent.loop import UNIFIED_SESSION_KEY from nanobot.agent.loop import UNIFIED_SESSION_KEY
from nanobot.bus.events import OutboundMessage from nanobot.bus.events import InboundMessage, OutboundMessage
def _channel_session_key(channel: str, chat_id: str) -> str: def _channel_session_key(channel: str, chat_id: str) -> str:
return ( return (
@@ -958,76 +810,16 @@ def _run_gateway(
# Set cron callback (needs agent) # Set cron callback (needs agent)
async def on_cron_job(job: CronJob) -> str | None: async def on_cron_job(job: CronJob) -> str | None:
"""Execute a cron job through the agent.""" """Execute a cron job through the agent."""
async def _silent(*_args, **_kwargs):
pass
# Dream is an internal job — run directly, not through the agent loop.
if job.name == "dream": if job.name == "dream":
try: await bus.publish_inbound(InboundMessage(
await agent.dream.run() channel="system",
logger.info("Dream cron job completed") sender_id="dream",
except Exception: chat_id="dream",
logger.exception("Dream cron job failed") content="",
))
return None return None
# Heartbeat is a system job that checks HEARTBEAT.md for active tasks. from nanobot.utils.evaluator import evaluate_response
if job.name == "heartbeat":
heartbeat_file = config.workspace_path / "HEARTBEAT.md"
try:
content = heartbeat_file.read_text(encoding="utf-8")
except OSError:
logger.debug("Heartbeat: HEARTBEAT.md missing")
return None
if not content or content == _heartbeat_template():
logger.debug("Heartbeat: HEARTBEAT.md empty or identical to template")
return None
channel, chat_id = _pick_heartbeat_target()
if channel == "cli":
return None
prompt = (
_HEARTBEAT_PREAMBLE
+ f"Review the following HEARTBEAT.md and report any active tasks:\n\n{content}"
)
message_suppress_token = None
if isinstance(message_tool, MessageTool):
message_suppress_token = message_tool.set_suppress_delivery(True)
try:
resp = await agent.process_direct(
prompt,
session_key="heartbeat",
channel=channel,
chat_id=chat_id,
on_progress=_silent,
)
finally:
if isinstance(message_tool, MessageTool) and message_suppress_token is not None:
message_tool.reset_suppress_delivery(message_suppress_token)
response = resp.content if resp else ""
# Keep a small tail of heartbeat history so the loop stays bounded.
session = agent.sessions.get_or_create("heartbeat")
session.retain_recent_legal_suffix(hb_cfg.keep_recent_messages)
agent.sessions.save(session)
if not response:
return None
should_notify = await evaluate_response(
response, prompt, agent.provider, agent.model, default_notify=False,
)
if should_notify:
logger.info("Heartbeat: completed, delivering response")
await _deliver_to_channel(
OutboundMessage(channel=channel, chat_id=chat_id, content=response),
record=True,
)
else:
logger.info("Heartbeat: silenced by post-run evaluation")
return response
reminder_note = ( reminder_note = (
"The scheduled time has arrived. Deliver this reminder to the user now, " "The scheduled time has arrived. Deliver this reminder to the user now, "
@@ -1042,6 +834,9 @@ def _run_gateway(
if isinstance(cron_tool, CronTool): if isinstance(cron_tool, CronTool):
cron_token = cron_tool.set_cron_context(True) cron_token = cron_tool.set_cron_context(True)
async def _silent(*_args, **_kwargs):
pass
message_record_token = None message_record_token = None
if isinstance(message_tool, MessageTool): if isinstance(message_tool, MessageTool):
message_record_token = message_tool.set_record_channel_delivery(True) message_record_token = message_tool.set_record_channel_delivery(True)
@@ -1098,14 +893,12 @@ def _run_gateway(
bus, bus,
session_manager=session_manager, session_manager=session_manager,
webui_runtime_model_name=_webui_runtime_model_name, webui_runtime_model_name=_webui_runtime_model_name,
webui_static_dist=webui_static_dist,
webui_runtime_surface=webui_runtime_surface,
webui_runtime_capabilities=webui_runtime_capabilities,
) )
def _pick_heartbeat_target() -> tuple[str, str]: def _pick_heartbeat_target() -> tuple[str, str]:
"""Pick a routable channel/chat target for heartbeat-triggered messages.""" """Pick a routable channel/chat target for heartbeat-triggered messages."""
enabled = set(channels.enabled_channels) enabled = set(channels.enabled_channels)
# Prefer the most recently updated non-internal session on an enabled channel.
for item in session_manager.list_sessions(): for item in session_manager.list_sessions():
key = item.get("key") or "" key = item.get("key") or ""
if ":" not in key: if ":" not in key:
@@ -1115,8 +908,70 @@ def _run_gateway(
continue continue
if channel in enabled and chat_id: if channel in enabled and chat_id:
return channel, chat_id return channel, chat_id
# Fallback keeps prior behavior but remains explicit.
return "cli", "direct" return "cli", "direct"
# Create heartbeat service
heartbeat_preamble = (
"[Your response will be delivered directly to the user's messaging app. "
"Output ONLY the final user-facing message. Never reference internal "
"files (HEARTBEAT.md, AWARENESS.md, etc.), your instructions, or your "
"decision process. If nothing needs reporting, respond with just "
"'All clear.' and nothing else.]\n\n"
)
async def on_heartbeat_execute(tasks: str) -> str:
"""Phase 2: execute heartbeat tasks through the full agent loop."""
channel, chat_id = _pick_heartbeat_target()
async def _silent(*_args, **_kwargs):
pass
resp = await agent.process_direct(
heartbeat_preamble + tasks,
session_key="heartbeat",
channel=channel,
chat_id=chat_id,
on_progress=_silent,
)
# Keep a small tail of heartbeat history so the loop stays bounded
# without losing all short-term context between runs.
session = agent.sessions.get_or_create("heartbeat")
session.retain_recent_legal_suffix(hb_cfg.keep_recent_messages)
agent.sessions.save(session)
return resp.content if resp else ""
async def on_heartbeat_notify(response: str) -> None:
"""Deliver a heartbeat response to the user's channel.
In addition to publishing the outbound message, this injects the
delivered text as an assistant turn into the *target channel's*
session. Without this, a user reply on the channel (e.g. "Sure")
lands in a session that has no context about the heartbeat message
and the agent cannot follow through.
"""
channel, chat_id = _pick_heartbeat_target()
if channel == "cli":
return # No external channel available to deliver to
await _deliver_to_channel(
OutboundMessage(channel=channel, chat_id=chat_id, content=response),
record=True,
)
hb_cfg = config.gateway.heartbeat
heartbeat = HeartbeatService(
workspace=config.workspace_path,
llm_runtime=agent.llm_runtime,
on_execute=on_heartbeat_execute,
on_notify=on_heartbeat_notify,
interval_s=hb_cfg.interval_s,
enabled=hb_cfg.enabled,
timezone=config.agents.defaults.timezone,
)
if channels.enabled_channels: if channels.enabled_channels:
console.print(f"[green]✓[/green] Channels enabled: {', '.join(channels.enabled_channels)}") console.print(f"[green]✓[/green] Channels enabled: {', '.join(channels.enabled_channels)}")
else: else:
@@ -1126,11 +981,7 @@ def _run_gateway(
if cron_status["jobs"] > 0: if cron_status["jobs"] > 0:
console.print(f"[green]✓[/green] Cron: {cron_status['jobs']} scheduled jobs") console.print(f"[green]✓[/green] Cron: {cron_status['jobs']} scheduled jobs")
hb_cfg = config.gateway.heartbeat console.print(f"[green]✓[/green] Heartbeat: every {hb_cfg.interval_s}s")
if hb_cfg.enabled:
console.print(f"[green]✓[/green] Heartbeat: every {hb_cfg.interval_s}s")
else:
console.print("[yellow]✗[/yellow] Heartbeat: disabled")
async def _health_server(host: str, health_port: int): async def _health_server(host: str, health_port: int):
"""Lightweight HTTP health endpoint on the gateway port.""" """Lightweight HTTP health endpoint on the gateway port."""
@@ -1174,37 +1025,20 @@ def _run_gateway(
console.print(f"[green]✓[/green] Health endpoint: http://{host}:{health_port}/health") console.print(f"[green]✓[/green] Health endpoint: http://{host}:{health_port}/health")
async with server: async with server:
await server.serve_forever() await server.serve_forever()
# Register Dream system job (idempotent on restart) # Register Dream system job (always-on, idempotent on restart)
dream_cfg = config.agents.defaults.dream dream_cfg = config.agents.defaults.dream
if dream_cfg.model_override:
agent.dream.model = dream_cfg.model_override
agent.dream.max_batch_size = dream_cfg.max_batch_size agent.dream.max_batch_size = dream_cfg.max_batch_size
agent.dream.max_iterations = dream_cfg.max_iterations agent.dream.max_iterations = dream_cfg.max_iterations
agent.dream.annotate_line_ages = dream_cfg.annotate_line_ages agent.dream.annotate_line_ages = dream_cfg.annotate_line_ages
from nanobot.cron.types import CronJob, CronPayload, CronSchedule agent.dream.edit_user_skills = dream_cfg.dream_edit_user_skills
if dream_cfg.enabled: from nanobot.cron.types import CronJob, CronPayload
cron.register_system_job(CronJob( cron.register_system_job(CronJob(
id="dream", id="dream",
name="dream", name="dream",
schedule=dream_cfg.build_schedule(config.agents.defaults.timezone), schedule=dream_cfg.build_schedule(config.agents.defaults.timezone),
payload=CronPayload(kind="system_event"), payload=CronPayload(kind="system_event"),
)) ))
console.print(f"[green]✓[/green] Dream: {dream_cfg.describe_schedule()}") console.print(f"[green]✓[/green] Dream: {dream_cfg.describe_schedule()}")
else:
console.print("[yellow]○[/yellow] Dream: disabled")
# Register Heartbeat system job (idempotent on restart)
if hb_cfg.enabled:
cron.register_system_job(CronJob(
id="heartbeat",
name="heartbeat",
schedule=CronSchedule(
kind="every",
every_ms=hb_cfg.interval_s * 1000,
tz=config.agents.defaults.timezone,
),
payload=CronPayload(kind="system_event"),
))
async def _open_browser_when_ready() -> None: async def _open_browser_when_ready() -> None:
"""Wait for the gateway to bind, then point the user's browser at the webui.""" """Wait for the gateway to bind, then point the user's browser at the webui."""
@@ -1232,12 +1066,12 @@ def _run_gateway(
async def run(): async def run():
try: try:
await cron.start() await cron.start()
await heartbeat.start()
tasks = [ tasks = [
agent.run(), agent.run(),
channels.start_all(), channels.start_all(),
_health_server(config.gateway.host, port),
] ]
if health_server_enabled:
tasks.append(_health_server(config.gateway.host, port))
if open_browser_url: if open_browser_url:
tasks.append(_open_browser_when_ready()) tasks.append(_open_browser_when_ready())
await asyncio.gather(*tasks) await asyncio.gather(*tasks)
@@ -1250,6 +1084,7 @@ def _run_gateway(
console.print(traceback.format_exc()) console.print(traceback.format_exc())
finally: finally:
await agent.close_mcp() await agent.close_mcp()
heartbeat.stop()
cron.stop() cron.stop()
agent.stop() agent.stop()
await channels.stop_all() await channels.stop_all()
+1 -1
View File
@@ -1155,7 +1155,7 @@ _SETTINGS_SECTIONS: dict[str, tuple[str, str, set[str] | None]] = {
"Agent Settings": ("Agent Defaults", "Configure default model, temperature, and behavior", None), "Agent Settings": ("Agent Defaults", "Configure default model, temperature, and behavior", None),
"Channel Common": ("Channel Common", "Configure cross-channel behavior: progress, tool hints, retries", None), "Channel Common": ("Channel Common", "Configure cross-channel behavior: progress, tool hints, retries", None),
"API Server": ("API Server", "Configure OpenAI-compatible API endpoint", None), "API Server": ("API Server", "Configure OpenAI-compatible API endpoint", None),
"Gateway": ("Gateway Settings", "Configure server host, port", None), "Gateway": ("Gateway Settings", "Configure server host, port, and heartbeat", None),
"Tools": ("Tools Settings", "Configure web search, shell exec, and other tools", {"mcp_servers"}), "Tools": ("Tools Settings", "Configure web search, shell exec, and other tools", {"mcp_servers"}),
} }
+34 -25
View File
@@ -123,7 +123,7 @@ async def cmd_stop(ctx: CommandContext) -> OutboundMessage:
"""Cancel all active tasks and subagents for the session.""" """Cancel all active tasks and subagents for the session."""
loop = ctx.loop loop = ctx.loop
msg = ctx.msg msg = ctx.msg
total = await loop._cancel_active_tasks(ctx.key) total = await loop._cancel_active_tasks(msg.session_key)
content = f"Stopped {total} task(s)." if total else "No active task to stop." content = f"Stopped {total} task(s)." if total else "No active task to stop."
return OutboundMessage( return OutboundMessage(
channel=msg.channel, chat_id=msg.chat_id, content=content, channel=msg.channel, chat_id=msg.chat_id, content=content,
@@ -299,30 +299,22 @@ async def cmd_model(ctx: CommandContext) -> OutboundMessage:
async def cmd_dream(ctx: CommandContext) -> OutboundMessage: async def cmd_dream(ctx: CommandContext) -> OutboundMessage:
"""Manually trigger a Dream consolidation run.""" """Manually trigger a Dream consolidation run."""
import time from nanobot.bus.events import InboundMessage
loop = ctx.loop await ctx.loop.bus.publish_inbound(InboundMessage(
msg = ctx.msg channel="system",
sender_id="dream",
async def _run_dream(): chat_id="dream",
t0 = time.monotonic() content="",
try: metadata={
did_work = await loop.dream.run() "trigger_channel": ctx.msg.channel,
elapsed = time.monotonic() - t0 "trigger_chat_id": ctx.msg.chat_id,
if did_work: },
content = f"Dream completed in {elapsed:.1f}s." ))
else:
content = "Dream: nothing to process."
except Exception as e:
elapsed = time.monotonic() - t0
content = f"Dream failed after {elapsed:.1f}s: {e}"
await loop.bus.publish_outbound(OutboundMessage(
channel=msg.channel, chat_id=msg.chat_id, content=content,
))
asyncio.create_task(_run_dream())
return OutboundMessage( return OutboundMessage(
channel=msg.channel, chat_id=msg.chat_id, content="Dreaming...", channel=ctx.msg.channel,
chat_id=ctx.msg.chat_id,
content="Dream started. It will process memory backlog and report when done.",
) )
@@ -355,6 +347,18 @@ def _format_changed_files(diff: str) -> str:
def _format_dream_log_content(commit, diff: str, *, requested_sha: str | None = None) -> str: def _format_dream_log_content(commit, diff: str, *, requested_sha: str | None = None) -> str:
files_line = _format_changed_files(diff) files_line = _format_changed_files(diff)
msg_lines = commit.message.splitlines() if commit.message else []
msg_summary = msg_lines[0] if msg_lines else ""
msg_body = []
in_body = False
for line in msg_lines[1:]:
if not in_body:
if not line:
in_body = True
continue
msg_body.append(line)
body_text = "\n".join(msg_body).strip()
lines = [ lines = [
"## Dream Update", "## Dream Update",
"", "",
@@ -362,8 +366,12 @@ def _format_dream_log_content(commit, diff: str, *, requested_sha: str | None =
"", "",
f"- Commit: `{commit.sha}`", f"- Commit: `{commit.sha}`",
f"- Time: {commit.timestamp}", f"- Time: {commit.timestamp}",
f"- Changed files: {files_line}",
] ]
if msg_summary:
lines.append(f"- Summary: {msg_summary}")
lines.append(f"- Changed files: {files_line}")
if body_text:
lines.extend(["", "### Analysis", "", body_text])
if diff: if diff:
lines.extend([ lines.extend([
"", "",
@@ -389,7 +397,8 @@ def _format_dream_restore_list(commits: list) -> str:
"", "",
] ]
for c in commits: for c in commits:
lines.append(f"- `{c.sha}` {c.timestamp} - {c.message.splitlines()[0]}") summary = c.message.splitlines()[0] if c.message else "(no message)"
lines.append(f"- `{c.sha}` {c.timestamp} - {summary}")
lines.extend([ lines.extend([
"", "",
"Preview a version with `/dream-log <sha>` before restoring it.", "Preview a version with `/dream-log <sha>` before restoring it.",
+12 -25
View File
@@ -37,7 +37,6 @@ class ChannelsConfig(Base):
send_progress: bool = True # stream agent's text progress to the channel send_progress: bool = True # stream agent's text progress to the channel
send_tool_hints: bool = False # stream tool-call hints (e.g. read_file("…")) send_tool_hints: bool = False # stream tool-call hints (e.g. read_file("…"))
show_reasoning: bool = True # surface model reasoning when channel implements it show_reasoning: bool = True # surface model reasoning when channel implements it
extract_document_text: bool = True # extract text from document attachments before sending to the model
send_max_retries: int = Field(default=3, ge=0, le=10) # Max delivery attempts (initial send included) send_max_retries: int = Field(default=3, ge=0, le=10) # Max delivery attempts (initial send included)
transcription_provider: str = "groq" # Voice transcription backend: "groq" or "openai" transcription_provider: str = "groq" # Voice transcription backend: "groq" or "openai"
transcription_language: str | None = Field(default=None, pattern=r"^[a-z]{2,3}$") # Optional ISO-639-1 hint for audio transcription transcription_language: str | None = Field(default=None, pattern=r"^[a-z]{2,3}$") # Optional ISO-639-1 hint for audio transcription
@@ -48,20 +47,22 @@ class DreamConfig(Base):
_HOUR_MS = 3_600_000 _HOUR_MS = 3_600_000
enabled: bool = True # Register the periodic Dream consolidation job on startup
interval_h: int = Field(default=2, ge=1) # Every 2 hours by default interval_h: int = Field(default=2, ge=1) # Every 2 hours by default
cron: str | None = Field(default=None, exclude=True) # Legacy compatibility override cron: str | None = Field(default=None, exclude=True) # Legacy compatibility override
model_override: str | None = Field( model_override: str | None = Field(
default=None, default=None,
validation_alias=AliasChoices("modelOverride", "model", "model_override"), validation_alias=AliasChoices("modelOverride", "model", "model_override"),
) # Optional Dream-specific model override ) # Optional Dream-specific model override. Supports preset names (resolved against model_presets) or raw model identifiers.
max_batch_size: int = Field(default=20, ge=1) # Max history entries per run max_batch_size: int = Field(default=5, ge=1) # Max history entries per run
# Bumped from 10 to 15 in #3212 (exp002: +30% dedup, no accuracy loss; >15 plateaus). max_iterations: int = Field(default=15, ge=1) # Max tool calls per Dream run
max_iterations: int = Field(default=15, ge=1) # Max tool calls per Phase 2 # Per-line git-blame age annotation in the Dream prompt (see #3212). Default
# Per-line git-blame age annotation in Phase 1 prompt (see #3212). Default # on — set to False to feed all memory files raw if a specific LLM reacts
# on — set to False to feed MEMORY.md raw if a specific LLM reacts poorly # poorly to the `← Nd` suffix or you want deterministic, git-independent prompts.
# to the `← Nd` suffix or you want deterministic, git-independent prompts.
annotate_line_ages: bool = True annotate_line_ages: bool = True
# When False (default), Dream may only modify skills it created (marked
# dream_managed in frontmatter). When True, Dream may also edit user-created
# workspace skills. Builtin skills are never editable.
dream_edit_user_skills: bool = False
def build_schedule(self, timezone: str) -> CronSchedule: def build_schedule(self, timezone: str) -> CronSchedule:
"""Build the runtime schedule, preferring the legacy cron override if present.""" """Build the runtime schedule, preferring the legacy cron override if present."""
@@ -238,7 +239,7 @@ class ProvidersConfig(Base):
class HeartbeatConfig(Base): class HeartbeatConfig(Base):
"""Heartbeat service configuration (now backed by cron).""" """Heartbeat service configuration."""
enabled: bool = True enabled: bool = True
interval_s: int = 30 * 60 # 30 minutes interval_s: int = 30 * 60 # 30 minutes
@@ -297,16 +298,7 @@ class ToolsConfig(Base):
image_generation: ImageGenerationToolConfig = Field( image_generation: ImageGenerationToolConfig = Field(
default_factory=lambda: _lazy_default("nanobot.agent.tools.image_generation", "ImageGenerationToolConfig"), default_factory=lambda: _lazy_default("nanobot.agent.tools.image_generation", "ImageGenerationToolConfig"),
) )
restrict_to_workspace: bool = False # policy intent: keep tool access inside workspace when possible restrict_to_workspace: bool = False # restrict all tool access to workspace directory
webui_allow_local_service_access: bool = Field(
default=True,
validation_alias=AliasChoices(
"webuiAllowLocalServiceAccess",
"webui_allow_local_service_access",
"allowLocalPreviewAccess",
"allow_local_preview_access",
),
) # allow WebUI Full Access shell checks against localhost services; legacy allowLocalPreviewAccess still reads
mcp_servers: dict[str, MCPServerConfig] = Field(default_factory=dict) mcp_servers: dict[str, MCPServerConfig] = Field(default_factory=dict)
ssrf_whitelist: list[str] = Field(default_factory=list) # CIDR ranges to exempt from SSRF blocking (e.g. ["100.64.0.0/10"] for Tailscale) ssrf_whitelist: list[str] = Field(default_factory=list) # CIDR ranges to exempt from SSRF blocking (e.g. ["100.64.0.0/10"] for Tailscale)
@@ -325,11 +317,6 @@ class Config(BaseSettings):
validation_alias=AliasChoices("modelPresets", "model_presets"), validation_alias=AliasChoices("modelPresets", "model_presets"),
) )
def __init__(self, **values: Any) -> None:
if not type(self).__pydantic_complete__:
_resolve_tool_config_refs()
super().__init__(**values)
@model_validator(mode="after") @model_validator(mode="after")
def _validate_model_preset(self) -> "Config": def _validate_model_preset(self) -> "Config":
if "default" in self.model_presets: if "default" in self.model_presets:
+5
View File
@@ -0,0 +1,5 @@
"""Heartbeat service for periodic agent wake-ups."""
from nanobot.heartbeat.service import HeartbeatService
__all__ = ["HeartbeatService"]
+243
View File
@@ -0,0 +1,243 @@
"""Heartbeat service - periodic agent wake-up to check for tasks."""
from __future__ import annotations
import asyncio
from pathlib import Path
from typing import Any, Callable, Coroutine
from loguru import logger
from nanobot.providers.base import LLMProvider
from nanobot.utils.llm_runtime import LLMRuntimeResolver, static_llm_runtime
_HEARTBEAT_TOOL = [
{
"type": "function",
"function": {
"name": "heartbeat",
"description": "Report heartbeat decision after reviewing tasks.",
"parameters": {
"type": "object",
"properties": {
"action": {
"type": "string",
"enum": ["skip", "run"],
"description": "skip = nothing to do, run = has active tasks",
},
"tasks": {
"type": "string",
"description": "Natural-language summary of active tasks (required for run)",
},
},
"required": ["action"],
},
},
}
]
class HeartbeatService:
"""
Periodic heartbeat service that wakes the agent to check for tasks.
Phase 1 (decision): reads HEARTBEAT.md and asks the LLM via a virtual
tool call whether there are active tasks. This avoids free-text parsing
and the unreliable HEARTBEAT_OK token.
Phase 2 (execution): only triggered when Phase 1 returns ``run``. The
``on_execute`` callback runs the task through the full agent loop and
returns the result to deliver.
"""
def __init__(
self,
workspace: Path,
provider: LLMProvider | None = None,
model: str | None = None,
on_execute: Callable[[str], Coroutine[Any, Any, str]] | None = None,
on_notify: Callable[[str], Coroutine[Any, Any, None]] | None = None,
interval_s: int = 30 * 60,
enabled: bool = True,
timezone: str | None = None,
llm_runtime: LLMRuntimeResolver | None = None,
):
self.workspace = workspace
if llm_runtime is None:
if provider is None or model is None:
raise ValueError("HeartbeatService requires either llm_runtime or provider/model")
llm_runtime = static_llm_runtime(provider, model)
self._llm_runtime = llm_runtime
self.on_execute = on_execute
self.on_notify = on_notify
self.interval_s = interval_s
self.enabled = enabled
self.timezone = timezone
self._running = False
self._task: asyncio.Task | None = None
@property
def heartbeat_file(self) -> Path:
return self.workspace / "HEARTBEAT.md"
def _read_heartbeat_file(self) -> str | None:
if self.heartbeat_file.exists():
try:
return self.heartbeat_file.read_text(encoding="utf-8")
except Exception:
return None
return None
async def _decide(self, content: str) -> tuple[str, str]:
"""Phase 1: ask LLM to decide skip/run via virtual tool call.
Returns (action, tasks) where action is 'skip' or 'run'.
"""
from nanobot.utils.helpers import current_time_str
llm = self._llm_runtime()
response = await llm.provider.chat_with_retry(
messages=[
{"role": "system", "content": "You are a heartbeat agent. Call the heartbeat tool to report your decision."},
{"role": "user", "content": (
f"Current Time: {current_time_str(self.timezone)}\n\n"
"Review the following HEARTBEAT.md and decide whether there are active tasks.\n\n"
f"{content}"
)},
],
tools=_HEARTBEAT_TOOL,
model=llm.model,
)
if not response.should_execute_tools:
if response.has_tool_calls:
logger.warning(
"Ignoring heartbeat tool calls under finish_reason='{}'",
response.finish_reason,
)
return "skip", ""
args = response.tool_calls[0].arguments
return args.get("action", "skip"), args.get("tasks", "")
async def start(self) -> None:
"""Start the heartbeat service."""
if not self.enabled:
logger.info("Heartbeat disabled")
return
if self._running:
logger.warning("Heartbeat already running")
return
self._running = True
self._task = asyncio.create_task(self._run_loop())
logger.info("Heartbeat started (every {}s)", self.interval_s)
def stop(self) -> None:
"""Stop the heartbeat service."""
self._running = False
if self._task:
self._task.cancel()
self._task = None
async def _run_loop(self) -> None:
"""Main heartbeat loop."""
while self._running:
try:
await asyncio.sleep(self.interval_s)
if self._running:
await self._tick()
except asyncio.CancelledError:
break
except Exception:
logger.exception("Heartbeat error")
@staticmethod
def _is_deliverable(response: str) -> bool:
"""Check if a heartbeat response is suitable for user delivery.
Filters out two classes of bad output before the evaluator runs:
1. **Finalization fallback** the runner hit empty-response retries
and produced a canned error message. For heartbeat, empty output
is a valid "nothing to report" outcome, not a failure.
2. **Leaked reasoning** the model reflected internal file names,
decision logic, or meta-commentary instead of a user-facing report.
"""
text = response.lower()
# Runner finalization fallback
if "couldn't produce a final answer" in text:
return False
# Leaked internal reasoning patterns
leaked_patterns = [
"heartbeat.md",
"awareness.md",
"judgment call:",
"decision logic",
"valid options are",
"my instructions",
"i am supposed to",
"strict heartbeat interpretation",
]
if any(pattern in text for pattern in leaked_patterns):
return False
return True
async def _tick(self) -> None:
"""Execute a single heartbeat tick."""
from nanobot.utils.evaluator import evaluate_response
content = self._read_heartbeat_file()
if not content:
logger.debug("Heartbeat: HEARTBEAT.md missing or empty")
return
logger.info("Heartbeat: checking for tasks...")
try:
action, tasks = await self._decide(content)
if action != "run":
logger.info("Heartbeat: OK (nothing to report)")
return
logger.info("Heartbeat: tasks found, executing...")
if self.on_execute:
response = await self.on_execute(tasks)
if not response:
logger.info("Heartbeat: no response from execution")
return
if not self._is_deliverable(response):
logger.info(
"Heartbeat: suppressed non-deliverable response ({})",
response[:80],
)
return
llm = self._llm_runtime()
should_notify = await evaluate_response(
response, tasks, llm.provider, llm.model,
)
if should_notify and self.on_notify:
logger.info("Heartbeat: completed, delivering response")
await self.on_notify(response)
else:
logger.info("Heartbeat: silenced by post-run evaluation")
except Exception:
logger.exception("Heartbeat execution failed")
async def trigger_now(self) -> str | None:
"""Manually trigger a heartbeat."""
content = self._read_heartbeat_file()
if not content:
return None
action, tasks = await self._decide(content)
if action != "run" or not self.on_execute:
return None
return await self.on_execute(tasks)
+1 -16
View File
@@ -45,21 +45,13 @@ class AnthropicProvider(LLMProvider):
if api_key: if api_key:
client_kw["api_key"] = api_key client_kw["api_key"] = api_key
if api_base: if api_base:
client_kw["base_url"] = self._normalize_base_url(api_base) client_kw["base_url"] = api_base
if extra_headers: if extra_headers:
client_kw["default_headers"] = extra_headers client_kw["default_headers"] = extra_headers
# Keep retries centralized in LLMProvider._run_with_retry to avoid retry amplification. # Keep retries centralized in LLMProvider._run_with_retry to avoid retry amplification.
client_kw["max_retries"] = 0 client_kw["max_retries"] = 0
self._client = AsyncAnthropic(**client_kw) self._client = AsyncAnthropic(**client_kw)
@staticmethod
def _normalize_base_url(api_base: str) -> str:
"""Anthropic SDK appends /v1 to request paths internally."""
normalized = api_base.rstrip("/")
if normalized.endswith("/v1"):
return normalized[: -len("/v1")]
return normalized
@classmethod @classmethod
def _handle_error(cls, e: Exception) -> LLMResponse: def _handle_error(cls, e: Exception) -> LLMResponse:
response = getattr(e, "response", None) response = getattr(e, "response", None)
@@ -236,13 +228,6 @@ class AnthropicProvider(LLMProvider):
if converted: if converted:
result.append(converted) result.append(converted)
continue continue
if not item.get("type"):
# Anthropic requires every content block to declare a "type".
# A tool that returned a bare dict (or a list of dicts) lands
# here; coerce it to a text block instead of emitting a block
# the API rejects with "content.0.type: Field required".
result.append({"type": "text", "text": str(item)})
continue
result.append(item) result.append(item)
return result or "(empty)" return result or "(empty)"
+1 -40
View File
@@ -315,29 +315,6 @@ class LLMProvider(ABC):
return cls._is_transient_error(response.content) return cls._is_transient_error(response.content)
@classmethod
def is_arrearage_response(cls, response: LLMResponse) -> bool:
"""Detect API-key arrearage / quota / billing errors that won't clear on retry.
These surface as HTTP 402 or as billing semantic tokens (e.g.
``insufficient_quota``, ``payment_required``); reuses the same token and
text markers the 429 retry policy treats as non-retryable.
"""
if response.error_status_code is not None and int(response.error_status_code) == 402:
return True
type_token = cls._normalize_error_token(response.error_type)
code_token = cls._normalize_error_token(response.error_code)
if any(
token in cls._NON_RETRYABLE_429_ERROR_TOKENS
for token in (type_token, code_token)
if token is not None
):
return True
content = (response.content or "").lower()
return any(marker in content for marker in cls._NON_RETRYABLE_429_TEXT_MARKERS)
@staticmethod @staticmethod
def _normalize_error_token(value: Any) -> str | None: def _normalize_error_token(value: Any) -> str | None:
if value is None: if value is None:
@@ -580,20 +557,11 @@ class LLMProvider(ABC):
if reasoning_effort is self._SENTINEL: if reasoning_effort is self._SENTINEL:
reasoning_effort = self.generation.reasoning_effort reasoning_effort = self.generation.reasoning_effort
has_streamed_content = False
async def _tracking_delta(text: str) -> None:
nonlocal has_streamed_content
if text:
has_streamed_content = True
if on_content_delta:
await on_content_delta(text)
kw: dict[str, Any] = dict( kw: dict[str, Any] = dict(
messages=messages, tools=tools, model=model, messages=messages, tools=tools, model=model,
max_tokens=max_tokens, temperature=temperature, max_tokens=max_tokens, temperature=temperature,
reasoning_effort=reasoning_effort, tool_choice=tool_choice, reasoning_effort=reasoning_effort, tool_choice=tool_choice,
on_content_delta=_tracking_delta if on_content_delta is not None else None, on_content_delta=on_content_delta,
on_thinking_delta=on_thinking_delta, on_thinking_delta=on_thinking_delta,
on_tool_call_delta=on_tool_call_delta, on_tool_call_delta=on_tool_call_delta,
) )
@@ -603,7 +571,6 @@ class LLMProvider(ABC):
messages, messages,
retry_mode=retry_mode, retry_mode=retry_mode,
on_retry_wait=on_retry_wait, on_retry_wait=on_retry_wait,
should_retry_guard=lambda: not has_streamed_content,
) )
async def chat_with_retry( async def chat_with_retry(
@@ -750,7 +717,6 @@ class LLMProvider(ABC):
*, *,
retry_mode: str, retry_mode: str,
on_retry_wait: Callable[[str], Awaitable[None]] | None, on_retry_wait: Callable[[str], Awaitable[None]] | None,
should_retry_guard: Callable[[], bool] | None = None,
) -> LLMResponse: ) -> LLMResponse:
attempt = 0 attempt = 0
delays = list(self._CHAT_RETRY_DELAYS) delays = list(self._CHAT_RETRY_DELAYS)
@@ -764,11 +730,6 @@ class LLMProvider(ABC):
if response.finish_reason != "error": if response.finish_reason != "error":
return response return response
last_response = response last_response = response
if should_retry_guard is not None and not should_retry_guard():
logger.warning(
"LLM stream failed after content was emitted; skipping retry"
)
return response
error_key = ((response.content or "").strip().lower() or None) error_key = ((response.content or "").strip().lower() or None)
if error_key and error_key == last_error_key: if error_key and error_key == last_error_key:
identical_error_count += 1 identical_error_count += 1
+16 -162
View File
@@ -5,7 +5,6 @@ from __future__ import annotations
import asyncio import asyncio
import hashlib import hashlib
import json import json
import os
from collections.abc import Awaitable, Callable from collections.abc import Awaitable, Callable
from typing import Any from typing import Any
@@ -15,7 +14,7 @@ from oauth_cli_kit import get_token as get_codex_token
from nanobot.providers.base import LLMProvider, LLMResponse, ToolCallRequest from nanobot.providers.base import LLMProvider, LLMResponse, ToolCallRequest
from nanobot.providers.openai_responses import ( from nanobot.providers.openai_responses import (
consume_sse_with_reasoning, consume_sse,
convert_messages, convert_messages,
convert_tools, convert_tools,
) )
@@ -41,7 +40,6 @@ class OpenAICodexProvider(LLMProvider):
reasoning_effort: str | None, reasoning_effort: str | None,
tool_choice: str | dict[str, Any] | None, tool_choice: str | dict[str, Any] | None,
on_content_delta: Callable[[str], Awaitable[None]] | None = None, on_content_delta: Callable[[str], Awaitable[None]] | None = None,
on_thinking_delta: Callable[[str], Awaitable[None]] | None = None,
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None, on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
) -> LLMResponse: ) -> LLMResponse:
"""Shared request logic for both chat() and chat_stream().""" """Shared request logic for both chat() and chat_stream()."""
@@ -63,52 +61,32 @@ class OpenAICodexProvider(LLMProvider):
"tool_choice": tool_choice or "auto", "tool_choice": tool_choice or "auto",
"parallel_tool_calls": True, "parallel_tool_calls": True,
} }
reasoning_options = _build_reasoning_options(reasoning_effort) if reasoning_effort and reasoning_effort.lower() != "none":
if reasoning_options: body["reasoning"] = {"effort": reasoning_effort}
body["reasoning"] = reasoning_options
if tools: if tools:
body["tools"] = convert_tools(tools) body["tools"] = convert_tools(tools)
try: try:
try: try:
content, tool_calls, finish_reason, reasoning_content = await _request_codex( content, tool_calls, finish_reason = await _request_codex(
DEFAULT_CODEX_URL, headers, body, verify=True, DEFAULT_CODEX_URL, headers, body, verify=True,
on_content_delta=on_content_delta, on_content_delta=on_content_delta,
on_thinking_delta=on_thinking_delta,
on_tool_call_delta=on_tool_call_delta, on_tool_call_delta=on_tool_call_delta,
) )
except Exception as e: except Exception as e:
if "CERTIFICATE_VERIFY_FAILED" not in str(e): if "CERTIFICATE_VERIFY_FAILED" not in str(e):
raise raise
logger.warning("SSL verification failed for Codex API; retrying with verify=False") logger.warning("SSL verification failed for Codex API; retrying with verify=False")
content, tool_calls, finish_reason, reasoning_content = await _request_codex( content, tool_calls, finish_reason = await _request_codex(
DEFAULT_CODEX_URL, headers, body, verify=False, DEFAULT_CODEX_URL, headers, body, verify=False,
on_content_delta=on_content_delta, on_content_delta=on_content_delta,
on_thinking_delta=on_thinking_delta,
on_tool_call_delta=on_tool_call_delta, on_tool_call_delta=on_tool_call_delta,
) )
return LLMResponse( return LLMResponse(content=content, tool_calls=tool_calls, finish_reason=finish_reason)
content=content,
tool_calls=tool_calls,
finish_reason=finish_reason,
reasoning_content=reasoning_content,
)
except Exception as e: except Exception as e:
response = _codex_error_response(e) msg = f"Error calling Codex: {e}"
exc_type = "CodexHTTPError" if isinstance(e, _CodexHTTPError) else type(e).__name__ retry_after = getattr(e, "retry_after", None) or self._extract_retry_after(msg)
logger.warning( return LLMResponse(content=msg, finish_reason="error", retry_after=retry_after)
"Codex API request failed: type={} kind={} retryable={} status={} "
"error_type={} error_code={} retry_after={} summary={}",
exc_type,
response.error_kind,
response.error_should_retry,
response.error_status_code,
response.error_type,
response.error_code,
response.retry_after,
_codex_log_summary(exc_type, response),
)
return response
async def chat( async def chat(
self, messages: list[dict[str, Any]], tools: list[dict[str, Any]] | None = None, self, messages: list[dict[str, Any]], tools: list[dict[str, Any]] | None = None,
@@ -127,6 +105,7 @@ class OpenAICodexProvider(LLMProvider):
on_thinking_delta: Callable[[str], Awaitable[None]] | None = None, on_thinking_delta: Callable[[str], Awaitable[None]] | None = None,
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None, on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
) -> LLMResponse: ) -> LLMResponse:
_ = on_thinking_delta
return await self._call_codex( return await self._call_codex(
messages, messages,
tools, tools,
@@ -134,7 +113,6 @@ class OpenAICodexProvider(LLMProvider):
reasoning_effort, reasoning_effort,
tool_choice, tool_choice,
on_content_delta, on_content_delta,
on_thinking_delta,
on_tool_call_delta, on_tool_call_delta,
) )
@@ -148,16 +126,6 @@ def _strip_model_prefix(model: str) -> str:
return model return model
def _build_reasoning_options(reasoning_effort: str | None) -> dict[str, str] | None:
"""Opt in to visible summaries without changing provider-default effort."""
if reasoning_effort and reasoning_effort.lower() == "none":
return {"effort": "none"}
options = {"summary": "auto"}
if reasoning_effort:
options["effort"] = reasoning_effort
return options
def _build_headers(account_id: str, token: str) -> dict[str, str]: def _build_headers(account_id: str, token: str) -> dict[str, str]:
return { return {
"Authorization": f"Bearer {token}", "Authorization": f"Bearer {token}",
@@ -171,22 +139,9 @@ def _build_headers(account_id: str, token: str) -> dict[str, str]:
class _CodexHTTPError(RuntimeError): class _CodexHTTPError(RuntimeError):
def __init__( def __init__(self, message: str, retry_after: float | None = None):
self,
message: str,
*,
status_code: int | None = None,
retry_after: float | None = None,
error_type: str | None = None,
error_code: str | None = None,
should_retry: bool | None = None,
):
super().__init__(message) super().__init__(message)
self.status_code = status_code
self.retry_after = retry_after self.retry_after = retry_after
self.error_type = error_type
self.error_code = error_code
self.should_retry = should_retry
async def _request_codex( async def _request_codex(
@@ -195,31 +150,18 @@ async def _request_codex(
body: dict[str, Any], body: dict[str, Any],
verify: bool, verify: bool,
on_content_delta: Callable[[str], Awaitable[None]] | None = None, on_content_delta: Callable[[str], Awaitable[None]] | None = None,
on_thinking_delta: Callable[[str], Awaitable[None]] | None = None,
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None, on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
) -> tuple[str, list[ToolCallRequest], str, str | None]: ) -> tuple[str, list[ToolCallRequest], str]:
idle_timeout_s = int(os.environ.get("NANOBOT_STREAM_IDLE_TIMEOUT_S", "90")) async with httpx.AsyncClient(timeout=60.0, verify=verify) as client:
async with httpx.AsyncClient(timeout=idle_timeout_s, verify=verify) as client:
async with client.stream("POST", url, headers=headers, json=body) as response: async with client.stream("POST", url, headers=headers, json=body) as response:
if response.status_code != 200: if response.status_code != 200:
text = await response.aread() text = await response.aread()
raw = text.decode("utf-8", "ignore")
retry_after = LLMProvider._extract_retry_after_from_headers(response.headers) retry_after = LLMProvider._extract_retry_after_from_headers(response.headers)
error_type, error_code = LLMProvider._extract_error_type_code(raw)
raise _CodexHTTPError( raise _CodexHTTPError(
_friendly_error(response.status_code, raw), _friendly_error(response.status_code, text.decode("utf-8", "ignore")),
status_code=response.status_code,
retry_after=retry_after, retry_after=retry_after,
error_type=error_type,
error_code=error_code,
should_retry=_should_retry_status(response.status_code, error_type, error_code, raw),
) )
return await consume_sse_with_reasoning( return await consume_sse(response, on_content_delta, on_tool_call_delta)
response,
on_content_delta=on_content_delta,
on_tool_call_delta=on_tool_call_delta,
on_reasoning_delta=on_thinking_delta,
)
def _prompt_cache_key(messages: list[dict[str, Any]]) -> str: def _prompt_cache_key(messages: list[dict[str, Any]]) -> str:
@@ -228,94 +170,6 @@ def _prompt_cache_key(messages: list[dict[str, Any]]) -> str:
def _friendly_error(status_code: int, raw: str) -> str: def _friendly_error(status_code: int, raw: str) -> str:
_ = raw
if status_code == 429: if status_code == 429:
return "ChatGPT usage quota exceeded or rate limit triggered. Please try again later." return "ChatGPT usage quota exceeded or rate limit triggered. Please try again later."
return f"HTTP {status_code}: Codex API request failed" return f"HTTP {status_code}: {raw}"
def _codex_error_response(exc: Exception) -> LLMResponse:
"""Convert Codex transport/API failures into actionable, retryable metadata."""
exc_type = "CodexHTTPError" if isinstance(exc, _CodexHTTPError) else type(exc).__name__
detail = str(exc).strip()
status_code = getattr(exc, "status_code", None)
error_kind: str | None = None
default_detail: str | None = None
should_retry: bool | None = getattr(exc, "should_retry", None)
if isinstance(exc, (httpx.TimeoutException, asyncio.TimeoutError)):
error_kind = "timeout"
default_detail = "timed out waiting for response"
should_retry = True if should_retry is None else should_retry
elif isinstance(exc, httpx.RemoteProtocolError):
error_kind = "connection"
default_detail = "network protocol error while reading response"
should_retry = True if should_retry is None else should_retry
elif isinstance(exc, (httpx.NetworkError, httpx.TransportError)):
error_kind = "connection"
default_detail = "network connection failed"
should_retry = True if should_retry is None else should_retry
elif isinstance(exc, _CodexHTTPError):
error_kind = "http"
default_detail = "HTTP request failed"
if status_code is not None and should_retry is None:
retry_content = None if int(status_code) == 429 and isinstance(exc, _CodexHTTPError) else detail
should_retry = _should_retry_status(
int(status_code),
getattr(exc, "error_type", None),
getattr(exc, "error_code", None),
retry_content,
)
detail = detail or default_detail or "unexpected error"
message = f"Error calling Codex ({exc_type}): {detail}"
retry_after = getattr(exc, "retry_after", None) or LLMProvider._extract_retry_after(message)
return LLMResponse(
content=message,
finish_reason="error",
retry_after=retry_after,
error_status_code=int(status_code) if status_code is not None else None,
error_kind=error_kind,
error_type=getattr(exc, "error_type", None),
error_code=getattr(exc, "error_code", None),
error_retry_after_s=retry_after,
error_should_retry=should_retry,
)
def _codex_log_summary(exc_type: str, response: LLMResponse) -> str:
"""Return a bounded diagnostic summary without request body or raw upstream payload."""
if response.error_status_code is not None:
parts = [f"HTTP {response.error_status_code}"]
if response.error_type:
parts.append(f"type={response.error_type}")
if response.error_code:
parts.append(f"code={response.error_code}")
return " ".join(parts)
kind = (response.error_kind or "").strip()
if kind:
return f"{exc_type} {kind}"
return exc_type
def _should_retry_status(
status_code: int,
error_type: str | None,
error_code: str | None,
content: str | None,
) -> bool:
if status_code == 429:
return LLMProvider._is_retryable_429_response(
LLMResponse(
content=content or "",
finish_reason="error",
error_status_code=status_code,
error_type=error_type,
error_code=error_code,
)
)
return status_code in LLMProvider._RETRYABLE_STATUS_CODES or status_code >= 500
@@ -10,7 +10,6 @@ from nanobot.providers.openai_responses.parsing import (
FINISH_REASON_MAP, FINISH_REASON_MAP,
consume_sdk_stream, consume_sdk_stream,
consume_sse, consume_sse,
consume_sse_with_reasoning,
iter_sse, iter_sse,
map_finish_reason, map_finish_reason,
parse_response_output, parse_response_output,
@@ -23,7 +22,6 @@ __all__ = [
"split_tool_call_id", "split_tool_call_id",
"iter_sse", "iter_sse",
"consume_sse", "consume_sse",
"consume_sse_with_reasoning",
"consume_sdk_stream", "consume_sdk_stream",
"map_finish_reason", "map_finish_reason",
"parse_response_output", "parse_response_output",
+4 -103
View File
@@ -65,28 +65,10 @@ async def consume_sse(
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None, on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
) -> tuple[str, list[ToolCallRequest], str]: ) -> tuple[str, list[ToolCallRequest], str]:
"""Consume a Responses API SSE stream into ``(content, tool_calls, finish_reason)``.""" """Consume a Responses API SSE stream into ``(content, tool_calls, finish_reason)``."""
content, tool_calls, finish_reason, _ = await consume_sse_with_reasoning(
response,
on_content_delta=on_content_delta,
on_tool_call_delta=on_tool_call_delta,
)
return content, tool_calls, finish_reason
async def consume_sse_with_reasoning(
response: httpx.Response,
on_content_delta: Callable[[str], Awaitable[None]] | None = None,
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
on_reasoning_delta: Callable[[str], Awaitable[None]] | None = None,
) -> tuple[str, list[ToolCallRequest], str, str | None]:
"""Consume a Responses API SSE stream, including visible reasoning summaries."""
content = "" content = ""
tool_calls: list[ToolCallRequest] = [] tool_calls: list[ToolCallRequest] = []
tool_call_buffers: dict[str, dict[str, Any]] = {} tool_call_buffers: dict[str, dict[str, Any]] = {}
tool_call_args_emitted: set[str] = set()
finish_reason = "stop" finish_reason = "stop"
reasoning_content: str | None = None
streamed_reasoning = False
async for event in iter_sse(response): async for event in iter_sse(response):
event_type = event.get("type") event_type = event.get("type")
@@ -112,26 +94,6 @@ async def consume_sse_with_reasoning(
content += delta_text content += delta_text
if on_content_delta and delta_text: if on_content_delta and delta_text:
await on_content_delta(delta_text) await on_content_delta(delta_text)
elif event_type == "response.reasoning_summary_text.delta":
delta_text = event.get("delta") or ""
if delta_text:
reasoning_content = (reasoning_content or "") + delta_text
streamed_reasoning = True
if on_reasoning_delta:
await on_reasoning_delta(delta_text)
elif event_type == "response.reasoning_summary_text.done":
text = event.get("text") or ""
if text and not streamed_reasoning and not reasoning_content:
reasoning_content = text
if on_reasoning_delta:
await on_reasoning_delta(text)
elif event_type == "response.reasoning_summary_part.done":
part = event.get("part") or {}
text = part.get("text") if part.get("type") == "summary_text" else None
if text and not streamed_reasoning and not reasoning_content:
reasoning_content = text
if on_reasoning_delta:
await on_reasoning_delta(text)
elif event_type == "response.function_call_arguments.delta": elif event_type == "response.function_call_arguments.delta":
call_id = event.get("call_id") call_id = event.get("call_id")
if call_id and call_id in tool_call_buffers: if call_id and call_id in tool_call_buffers:
@@ -146,15 +108,7 @@ async def consume_sse_with_reasoning(
elif event_type == "response.function_call_arguments.done": elif event_type == "response.function_call_arguments.done":
call_id = event.get("call_id") call_id = event.get("call_id")
if call_id and call_id in tool_call_buffers: if call_id and call_id in tool_call_buffers:
arguments = event.get("arguments") or "" tool_call_buffers[call_id]["arguments"] = event.get("arguments") or ""
tool_call_buffers[call_id]["arguments"] = arguments
if on_tool_call_delta:
tool_call_args_emitted.add(str(call_id))
await on_tool_call_delta({
"call_id": str(call_id),
"name": str(tool_call_buffers[call_id].get("name") or ""),
"arguments": str(arguments),
})
elif event_type == "response.output_item.done": elif event_type == "response.output_item.done":
item = event.get("item") or {} item = event.get("item") or {}
if item.get("type") == "function_call": if item.get("type") == "function_call":
@@ -163,13 +117,6 @@ async def consume_sse_with_reasoning(
continue continue
buf = tool_call_buffers.get(call_id) or {} buf = tool_call_buffers.get(call_id) or {}
args_raw = buf.get("arguments") or item.get("arguments") or "{}" args_raw = buf.get("arguments") or item.get("arguments") or "{}"
if on_tool_call_delta and str(call_id) not in tool_call_args_emitted:
tool_call_args_emitted.add(str(call_id))
await on_tool_call_delta({
"call_id": str(call_id),
"name": str(buf.get("name") or item.get("name") or ""),
"arguments": str(args_raw),
})
try: try:
args = json.loads(args_raw) args = json.loads(args_raw)
except Exception: except Exception:
@@ -188,44 +135,14 @@ async def consume_sse_with_reasoning(
arguments=args, arguments=args,
) )
) )
elif item.get("type") == "reasoning" and not reasoning_content:
summary = _extract_reasoning_summary_from_output([item])
if summary:
reasoning_content = summary
if on_reasoning_delta:
await on_reasoning_delta(summary)
elif event_type == "response.completed": elif event_type == "response.completed":
response_obj = event.get("response") or {} status = (event.get("response") or {}).get("status")
status = response_obj.get("status")
finish_reason = map_finish_reason(status) finish_reason = map_finish_reason(status)
if not reasoning_content:
summary = _extract_reasoning_summary_from_output(response_obj.get("output") or [])
if summary:
reasoning_content = summary
if on_reasoning_delta:
await on_reasoning_delta(summary)
elif event_type in {"error", "response.failed"}: elif event_type in {"error", "response.failed"}:
detail = event.get("error") or event.get("message") or event detail = event.get("error") or event.get("message") or event
raise RuntimeError(f"Response failed: {str(detail)[:500]}") raise RuntimeError(f"Response failed: {str(detail)[:500]}")
return content, tool_calls, finish_reason, reasoning_content return content, tool_calls, finish_reason
def _extract_reasoning_summary_from_output(output: Any) -> str | None:
parts: list[str] = []
for item in output or []:
if not isinstance(item, dict):
dump = getattr(item, "model_dump", None)
item = dump() if callable(dump) else vars(item)
if item.get("type") != "reasoning":
continue
for summary in item.get("summary") or []:
if not isinstance(summary, dict):
dump = getattr(summary, "model_dump", None)
summary = dump() if callable(dump) else vars(summary)
if summary.get("type") == "summary_text" and summary.get("text"):
parts.append(summary["text"])
return "".join(parts) or None
def parse_response_output(response: Any) -> LLMResponse: def parse_response_output(response: Any) -> LLMResponse:
@@ -313,7 +230,6 @@ async def consume_sdk_stream(
content = "" content = ""
tool_calls: list[ToolCallRequest] = [] tool_calls: list[ToolCallRequest] = []
tool_call_buffers: dict[str, dict[str, Any]] = {} tool_call_buffers: dict[str, dict[str, Any]] = {}
tool_call_args_emitted: set[str] = set()
finish_reason = "stop" finish_reason = "stop"
usage: dict[str, int] = {} usage: dict[str, int] = {}
reasoning_content: str | None = None reasoning_content: str | None = None
@@ -356,15 +272,7 @@ async def consume_sdk_stream(
elif event_type == "response.function_call_arguments.done": elif event_type == "response.function_call_arguments.done":
call_id = getattr(event, "call_id", None) call_id = getattr(event, "call_id", None)
if call_id and call_id in tool_call_buffers: if call_id and call_id in tool_call_buffers:
arguments = getattr(event, "arguments", "") or "" tool_call_buffers[call_id]["arguments"] = getattr(event, "arguments", "") or ""
tool_call_buffers[call_id]["arguments"] = arguments
if on_tool_call_delta:
tool_call_args_emitted.add(str(call_id))
await on_tool_call_delta({
"call_id": str(call_id),
"name": str(tool_call_buffers[call_id].get("name") or ""),
"arguments": str(arguments),
})
elif event_type == "response.output_item.done": elif event_type == "response.output_item.done":
item = getattr(event, "item", None) item = getattr(event, "item", None)
if item and getattr(item, "type", None) == "function_call": if item and getattr(item, "type", None) == "function_call":
@@ -373,13 +281,6 @@ async def consume_sdk_stream(
continue continue
buf = tool_call_buffers.get(call_id) or {} buf = tool_call_buffers.get(call_id) or {}
args_raw = buf.get("arguments") or getattr(item, "arguments", None) or "{}" args_raw = buf.get("arguments") or getattr(item, "arguments", None) or "{}"
if on_tool_call_delta and str(call_id) not in tool_call_args_emitted:
tool_call_args_emitted.add(str(call_id))
await on_tool_call_delta({
"call_id": str(call_id),
"name": str(buf.get("name") or getattr(item, "name", None) or ""),
"arguments": str(args_raw),
})
try: try:
args = json.loads(args_raw) args = json.loads(args_raw)
except Exception: except Exception:
+5 -45
View File
@@ -36,36 +36,15 @@ def configure_ssrf_whitelist(cidrs: list[str]) -> None:
_allowed_networks = nets _allowed_networks = nets
def _normalize_addr(
addr: ipaddress.IPv4Address | ipaddress.IPv6Address,
) -> ipaddress.IPv4Address | ipaddress.IPv6Address:
"""Normalize IPv6-mapped IPv4 addresses to their IPv4 form.
``::ffff:127.0.0.1`` is semantically identical to ``127.0.0.1`` but
Python's ipaddress treats it as an IPv6Address that matches neither
``127.0.0.0/8`` nor ``::1/128``. Converting it to IPv4 ensures
blocklist/allowlist checks work correctly.
"""
if isinstance(addr, ipaddress.IPv6Address) and addr.ipv4_mapped is not None:
return addr.ipv4_mapped
return addr
def _is_private(addr: ipaddress.IPv4Address | ipaddress.IPv6Address) -> bool: def _is_private(addr: ipaddress.IPv4Address | ipaddress.IPv6Address) -> bool:
normalized = _normalize_addr(addr) if _allowed_networks and any(addr in net for net in _allowed_networks):
if _allowed_networks and any(normalized in net for net in _allowed_networks):
return False return False
return any(normalized in net for net in _BLOCKED_NETWORKS) return any(addr in net for net in _BLOCKED_NETWORKS)
def validate_url_target(url: str, *, allow_loopback: bool = False) -> tuple[bool, str]: def validate_url_target(url: str) -> tuple[bool, str]:
"""Validate a URL is safe to fetch: scheme, hostname, and resolved IPs. """Validate a URL is safe to fetch: scheme, hostname, and resolved IPs.
``allow_loopback`` is intentionally narrow: it only permits literal
loopback hosts (localhost, 127.0.0.0/8, ::1) when every resolved address is
loopback. It does not allow RFC1918, link-local, metadata, or public DNS
names that happen to resolve to loopback.
Returns (ok, error_message). When ok is True, error_message is empty. Returns (ok, error_message). When ok is True, error_message is empty.
""" """
try: try:
@@ -87,16 +66,11 @@ def validate_url_target(url: str, *, allow_loopback: bool = False) -> tuple[bool
except socket.gaierror: except socket.gaierror:
return False, f"Cannot resolve hostname: {hostname}" return False, f"Cannot resolve hostname: {hostname}"
addrs: list[ipaddress.IPv4Address | ipaddress.IPv6Address] = []
for info in infos: for info in infos:
try: try:
addr = ipaddress.ip_address(info[4][0]) addr = ipaddress.ip_address(info[4][0])
except ValueError: except ValueError:
continue continue
addrs.append(addr)
if allow_loopback and _is_allowed_loopback_target(hostname, addrs):
return True, ""
for addr in addrs:
if _is_private(addr): if _is_private(addr):
return False, f"Blocked: {hostname} resolves to private/internal address {addr}" return False, f"Blocked: {hostname} resolves to private/internal address {addr}"
@@ -135,25 +109,11 @@ def validate_resolved_url(url: str) -> tuple[bool, str]:
return True, "" return True, ""
def contains_internal_url(command: str, *, allow_loopback: bool = False) -> bool: def contains_internal_url(command: str) -> bool:
"""Return True if the command string contains a URL targeting an internal/private address.""" """Return True if the command string contains a URL targeting an internal/private address."""
for m in _URL_RE.finditer(command): for m in _URL_RE.finditer(command):
url = m.group(0) url = m.group(0)
ok, _ = validate_url_target(url, allow_loopback=allow_loopback) ok, _ = validate_url_target(url)
if not ok: if not ok:
return True return True
return False return False
def _is_allowed_loopback_target(
hostname: str,
addrs: list[ipaddress.IPv4Address | ipaddress.IPv6Address],
) -> bool:
if not addrs or not all(_normalize_addr(addr).is_loopback for addr in addrs):
return False
normalized = hostname.rstrip(".").lower()
if normalized == "localhost":
return True
with suppress(ValueError):
return ipaddress.ip_address(hostname).is_loopback
return False
-430
View File
@@ -1,430 +0,0 @@
"""Workspace access scope and sandbox capability helpers."""
from __future__ import annotations
import os
from contextvars import ContextVar, Token
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Literal
WorkspaceAccessMode = Literal["restricted", "full"]
WORKSPACE_SCOPE_METADATA_KEY = "workspace_scope"
_ACCESS_MODES = {"restricted", "full"}
_TRUE_VALUES = {"1", "true", "yes", "on", "enabled"}
_FALSE_VALUES = {"0", "false", "no", "off", "disabled", ""}
_PROVIDER_LABELS = {
"none": "None",
"unknown": "Unknown system sandbox",
"macos_app_sandbox": "macOS App Sandbox",
"bwrap": "Bubblewrap",
}
_CURRENT_WORKSPACE_SCOPE: ContextVar["WorkspaceScope | None"] = ContextVar(
"nanobot_workspace_scope",
default=None,
)
class WorkspaceScopeError(ValueError):
"""Raised when a requested WebUI workspace scope is invalid."""
status = 400
def __init__(self, message: str, *, status: int = 400) -> None:
super().__init__(message)
self.message = message
self.status = status
@dataclass(frozen=True)
class WorkspaceSandboxStatus:
"""Resolved workspace sandbox state for runtime display and tooling."""
restrict_to_workspace: bool
workspace_root: str
level: str
enforced: bool
provider: str
provider_label: str
summary: str
def as_dict(self) -> dict[str, object]:
return {
"restrict_to_workspace": self.restrict_to_workspace,
"workspace_root": self.workspace_root,
"level": self.level,
"enforced": self.enforced,
"provider": self.provider,
"provider_label": self.provider_label,
"summary": self.summary,
}
@dataclass(frozen=True)
class WorkspaceScope:
"""Effective project root and access mode for one agent turn."""
project_path: Path
access_mode: WorkspaceAccessMode
restrict_to_workspace: bool
sandbox_status: WorkspaceSandboxStatus
source_channel: str | None = None
@property
def project_name(self) -> str:
return self.project_path.name or str(self.project_path)
def metadata(self) -> dict[str, str]:
return {
"project_path": str(self.project_path),
"access_mode": self.access_mode,
}
def payload(self) -> dict[str, Any]:
return {
**self.metadata(),
"project_name": self.project_name,
"restrict_to_workspace": self.restrict_to_workspace,
"sandbox_status": self.sandbox_status.as_dict(),
}
@dataclass(frozen=True)
class ToolWorkspace:
"""Workspace policy resolved for a tool call."""
project_path: Path | None
restrict_to_workspace: bool
scope: WorkspaceScope | None = None
@property
def allowed_root(self) -> Path | None:
if self.restrict_to_workspace and self.project_path is not None:
return self.project_path
return None
@dataclass(frozen=True)
class WorkspaceScopeResolver:
"""Resolve the effective workspace scope at an agent turn boundary."""
default_workspace: str | Path
default_restrict_to_workspace: bool
scoped_channel: str = "websocket"
@property
def sandbox_status(self) -> WorkspaceSandboxStatus:
return self.default().sandbox_status
def default(self) -> WorkspaceScope:
return default_workspace_scope(
self.default_workspace,
self.default_restrict_to_workspace,
)
def for_message(
self,
msg: Any,
session_metadata: Any,
) -> WorkspaceScope:
return self.for_turn(
channel=getattr(msg, "channel", None),
message_metadata=getattr(msg, "metadata", None),
session_metadata=session_metadata,
)
def for_turn(
self,
*,
channel: str | None,
message_metadata: Any,
session_metadata: Any,
) -> WorkspaceScope:
if channel != self.scoped_channel:
return self.default()
return resolve_effective_workspace_scope(
message_metadata=message_metadata,
session_metadata=session_metadata,
default_workspace=self.default_workspace,
default_restrict_to_workspace=self.default_restrict_to_workspace,
source_channel=channel,
)
def persist_message_scope(self, session: Any, msg: Any) -> None:
if getattr(msg, "channel", None) != self.scoped_channel:
return
metadata = getattr(msg, "metadata", None)
if not isinstance(metadata, dict):
return
raw = metadata.get(WORKSPACE_SCOPE_METADATA_KEY)
if isinstance(raw, dict):
session.metadata[WORKSPACE_SCOPE_METADATA_KEY] = dict(raw)
def workspace_sandbox_status(
*,
restrict_to_workspace: bool,
workspace: str | Path,
environ: dict[str, str] | None = None,
) -> WorkspaceSandboxStatus:
"""Return how workspace restriction is enforced in the current host."""
workspace_root = str(Path(workspace).expanduser().resolve(strict=False))
provider = _env_system_provider(environ)
if not restrict_to_workspace:
return WorkspaceSandboxStatus(
restrict_to_workspace=False,
workspace_root=workspace_root,
level="off",
enforced=False,
provider="none",
provider_label=_provider_label("none"),
summary="Workspace restriction is disabled.",
)
if provider:
label = _provider_label(provider)
return WorkspaceSandboxStatus(
restrict_to_workspace=True,
workspace_root=workspace_root,
level="system",
enforced=True,
provider=provider,
provider_label=label,
summary=f"Workspace restriction is system-enforced by {label}.",
)
return WorkspaceSandboxStatus(
restrict_to_workspace=True,
workspace_root=workspace_root,
level="application",
enforced=False,
provider="none",
provider_label=_provider_label("none"),
summary="Workspace restriction uses nanobot application-level guards.",
)
def default_access_mode(restrict_to_workspace: bool) -> WorkspaceAccessMode:
return "restricted" if restrict_to_workspace else "full"
def build_workspace_scope(
project_path: str | Path,
access_mode: str,
*,
source_channel: str | None = None,
) -> WorkspaceScope:
mode = _normalize_access_mode(access_mode)
root = Path(project_path).expanduser().resolve(strict=False)
restrict = mode == "restricted"
return WorkspaceScope(
project_path=root,
access_mode=mode,
restrict_to_workspace=restrict,
sandbox_status=workspace_sandbox_status(
restrict_to_workspace=restrict,
workspace=root,
),
source_channel=source_channel,
)
def default_workspace_scope(
workspace: str | Path,
restrict_to_workspace: bool,
*,
source_channel: str | None = None,
) -> WorkspaceScope:
return build_workspace_scope(
workspace,
default_access_mode(restrict_to_workspace),
source_channel=source_channel,
)
def validate_workspace_scope_payload(
raw: Any,
*,
default_workspace: str | Path,
default_restrict_to_workspace: bool,
source_channel: str | None = None,
) -> WorkspaceScope:
"""Validate a client-requested workspace scope."""
if raw is None:
return default_workspace_scope(
default_workspace,
default_restrict_to_workspace,
source_channel=source_channel,
)
if not isinstance(raw, dict):
raise WorkspaceScopeError("workspace_scope must be an object")
raw_path = raw.get("project_path") or raw.get("path")
if raw_path is None or raw_path == "":
raw_path = str(Path(default_workspace).expanduser().resolve(strict=False))
if not isinstance(raw_path, str):
raise WorkspaceScopeError("project_path must be a string")
if "\0" in raw_path:
raise WorkspaceScopeError("project_path contains invalid characters")
project = Path(raw_path).expanduser()
if not project.is_absolute():
raise WorkspaceScopeError("project_path must be absolute")
project = project.resolve(strict=False)
if not project.is_dir():
raise WorkspaceScopeError("project_path must be an existing directory")
raw_mode = raw.get("access_mode")
if raw_mode is None:
raw_mode = default_access_mode(default_restrict_to_workspace)
if not isinstance(raw_mode, str):
raise WorkspaceScopeError("access_mode must be a string")
return build_workspace_scope(project, raw_mode, source_channel=source_channel)
def workspace_scope_from_metadata(
metadata: Any,
*,
default_workspace: str | Path,
default_restrict_to_workspace: bool,
source_channel: str | None = None,
) -> WorkspaceScope:
"""Resolve persisted metadata, falling back safely for old or stale sessions."""
if not isinstance(metadata, dict):
return default_workspace_scope(
default_workspace,
default_restrict_to_workspace,
source_channel=source_channel,
)
try:
return validate_workspace_scope_payload(
metadata.get(WORKSPACE_SCOPE_METADATA_KEY),
default_workspace=default_workspace,
default_restrict_to_workspace=default_restrict_to_workspace,
source_channel=source_channel,
)
except WorkspaceScopeError:
return default_workspace_scope(
default_workspace,
default_restrict_to_workspace,
source_channel=source_channel,
)
def resolve_effective_workspace_scope(
*,
message_metadata: Any,
session_metadata: Any,
default_workspace: str | Path,
default_restrict_to_workspace: bool,
source_channel: str | None = None,
) -> WorkspaceScope:
if isinstance(message_metadata, dict) and WORKSPACE_SCOPE_METADATA_KEY in message_metadata:
return workspace_scope_from_metadata(
message_metadata,
default_workspace=default_workspace,
default_restrict_to_workspace=default_restrict_to_workspace,
source_channel=source_channel,
)
return workspace_scope_from_metadata(
session_metadata,
default_workspace=default_workspace,
default_restrict_to_workspace=default_restrict_to_workspace,
source_channel=source_channel,
)
def bind_workspace_scope(scope: WorkspaceScope) -> Token[WorkspaceScope | None]:
return _CURRENT_WORKSPACE_SCOPE.set(scope)
def reset_workspace_scope(token: Token[WorkspaceScope | None]) -> None:
_CURRENT_WORKSPACE_SCOPE.reset(token)
def current_workspace_scope() -> WorkspaceScope | None:
return _CURRENT_WORKSPACE_SCOPE.get()
def current_tool_workspace(
default_workspace: str | Path | None,
*,
restrict_to_workspace: bool = False,
sandbox_restricts_workspace: bool = False,
) -> ToolWorkspace:
"""Return the workspace/access policy for the current tool call."""
scope = current_workspace_scope()
project_path = (
scope.project_path
if scope is not None
else Path(default_workspace).expanduser() if default_workspace is not None else None
)
restrict = (
scope.restrict_to_workspace
if scope is not None
else bool(restrict_to_workspace)
) or sandbox_restricts_workspace
return ToolWorkspace(
project_path=project_path,
restrict_to_workspace=restrict,
scope=scope,
)
def current_scope_allows_loopback(*, enabled: bool) -> bool:
"""Return True when the current WebUI Full Access turn may touch loopback URLs."""
scope = current_workspace_scope()
return bool(
enabled
and scope is not None
and scope.source_channel == "websocket"
and scope.access_mode == "full"
and not scope.restrict_to_workspace
)
def _env_system_provider(environ: dict[str, str] | None = None) -> str | None:
env = environ if environ is not None else os.environ
explicit_provider = env.get("NANOBOT_WORKSPACE_SANDBOX_PROVIDER")
enforced = env.get("NANOBOT_WORKSPACE_SANDBOX_ENFORCED")
compatibility = env.get("NANOBOT_SANDBOX_ENFORCED")
marker = enforced if enforced is not None else compatibility
if marker is None:
return None
normalized_marker = marker.strip().lower()
if normalized_marker in _FALSE_VALUES:
return None
if normalized_marker in _TRUE_VALUES:
return _normalize_provider(explicit_provider)
return _normalize_provider(marker)
def _normalize_provider(value: str | None) -> str:
if not value:
return "unknown"
normalized = value.strip().lower().replace("-", "_").replace(" ", "_")
return normalized or "unknown"
def _provider_label(provider: str) -> str:
if provider in _PROVIDER_LABELS:
return _PROVIDER_LABELS[provider]
return provider.replace("_", " ").title()
def _normalize_access_mode(value: str) -> WorkspaceAccessMode:
mode = value.strip().lower().replace("_", "-")
if mode == "restrict":
mode = "restricted"
if mode == "full-access":
mode = "full"
if mode not in _ACCESS_MODES:
raise WorkspaceScopeError("access_mode must be restricted or full")
return mode # type: ignore[return-value]
-85
View File
@@ -1,85 +0,0 @@
"""Workspace path boundary helpers.
These helpers are application-level guards. They make path decisions
consistent across tools, but they are not a replacement for an OS sandbox.
"""
from __future__ import annotations
from pathlib import Path
from typing import Iterable
WORKSPACE_BOUNDARY_NOTE = (
" (this is a hard policy boundary, not a transient failure; "
"do not retry with shell tricks or alternative tools, and ask "
"the user how to proceed if the resource is genuinely required)"
)
class WorkspaceBoundaryError(PermissionError):
"""Raised when a requested path escapes an allowed workspace boundary."""
def resolve_path(path: str | Path, workspace: str | Path | None = None, *, strict: bool = False) -> Path:
"""Resolve *path*, interpreting relative paths against *workspace* when set."""
candidate = Path(path).expanduser()
if not candidate.is_absolute() and workspace is not None:
candidate = Path(workspace).expanduser() / candidate
return candidate.resolve(strict=strict)
def is_path_within(path: str | Path, root: str | Path) -> bool:
"""Return True when *path* resolves to *root* or a descendant of *root*."""
try:
resolved_path = Path(path).expanduser().resolve(strict=False)
resolved_root = Path(root).expanduser().resolve(strict=False)
resolved_path.relative_to(resolved_root)
return True
except (OSError, RuntimeError, TypeError, ValueError):
return False
def is_path_allowed(path: str | Path, roots: Iterable[str | Path]) -> bool:
"""Return True when *path* is inside any allowed root."""
return any(is_path_within(path, root) for root in roots)
def require_path_within(
path: str | Path,
root: str | Path,
*,
message: str | None = None,
) -> Path:
"""Resolve *path* and require it to be inside *root*."""
resolved = Path(path).expanduser().resolve(strict=False)
if not is_path_within(resolved, root):
raise WorkspaceBoundaryError(
message
or f"Path {path} is outside allowed directory {Path(root).expanduser()}"
+ WORKSPACE_BOUNDARY_NOTE
)
return resolved
def resolve_allowed_path(
path: str | Path,
*,
workspace: str | Path | None = None,
allowed_root: str | Path | None = None,
extra_allowed_roots: Iterable[str | Path] | None = None,
strict: bool = False,
) -> Path:
"""Resolve a path and enforce containment in allowed roots when configured."""
resolved = resolve_path(path, workspace, strict=False)
if allowed_root is None:
return resolve_path(path, workspace, strict=strict) if strict else resolved
roots = [allowed_root, *(extra_allowed_roots or [])]
if not is_path_allowed(resolved, roots):
raise WorkspaceBoundaryError(
f"Path {path} is outside allowed directory {Path(allowed_root).expanduser()}"
+ WORKSPACE_BOUNDARY_NOTE
)
if strict:
return resolve_path(path, workspace, strict=True)
return resolved
+7 -15
View File
@@ -19,7 +19,6 @@ from nanobot.utils.helpers import (
find_legal_message_start, find_legal_message_start,
image_placeholder_text, image_placeholder_text,
safe_filename, safe_filename,
strip_think,
) )
from nanobot.utils.subagent_channel_display import scrub_subagent_announce_body from nanobot.utils.subagent_channel_display import scrub_subagent_announce_body
@@ -77,17 +76,6 @@ def _message_preview_text(message: dict[str, Any]) -> str:
return _text_preview(content) return _text_preview(content)
def _metadata_title(metadata: Any) -> str:
if not isinstance(metadata, dict):
return ""
title = metadata.get("title")
if not isinstance(title, str):
return ""
if metadata.get("title_user_edited") is True:
return title
return strip_think(title)
@dataclass @dataclass
class Session: class Session:
"""A conversation session.""" """A conversation session."""
@@ -654,7 +642,7 @@ class SessionManager:
if data.get("_type") == "metadata": if data.get("_type") == "metadata":
key = data.get("key") or path.stem.replace("_", ":", 1) key = data.get("key") or path.stem.replace("_", ":", 1)
metadata = data.get("metadata", {}) metadata = data.get("metadata", {})
title = _metadata_title(metadata) title = metadata.get("title") if isinstance(metadata, dict) else None
preview = "" preview = ""
fallback_preview = "" fallback_preview = ""
scanned_records = 0 scanned_records = 0
@@ -685,7 +673,7 @@ class SessionManager:
"key": key, "key": key,
"created_at": data.get("created_at"), "created_at": data.get("created_at"),
"updated_at": data.get("updated_at"), "updated_at": data.get("updated_at"),
"title": title, "title": title if isinstance(title, str) else "",
"preview": preview, "preview": preview,
"path": str(path) "path": str(path)
}) })
@@ -696,7 +684,11 @@ class SessionManager:
"key": repaired.key, "key": repaired.key,
"created_at": repaired.created_at.isoformat(), "created_at": repaired.created_at.isoformat(),
"updated_at": repaired.updated_at.isoformat(), "updated_at": repaired.updated_at.isoformat(),
"title": _metadata_title(repaired.metadata), "title": (
repaired.metadata.get("title")
if isinstance(repaired.metadata.get("title"), str)
else ""
),
"preview": next( "preview": next(
( (
text text
+2 -12
View File
@@ -19,7 +19,7 @@ from nanobot.bus.queue import MessageBus
from nanobot.providers.base import LLMProvider from nanobot.providers.base import LLMProvider
from nanobot.session.goal_state import goal_state_ws_blob from nanobot.session.goal_state import goal_state_ws_blob
from nanobot.session.manager import Session, SessionManager from nanobot.session.manager import Session, SessionManager
from nanobot.utils.helpers import strip_think, truncate_text from nanobot.utils.helpers import truncate_text
from nanobot.utils.llm_runtime import LLMRuntime from nanobot.utils.llm_runtime import LLMRuntime
WEBUI_SESSION_METADATA_KEY = "webui" WEBUI_SESSION_METADATA_KEY = "webui"
@@ -48,7 +48,6 @@ def clean_generated_title(raw: str | None) -> str:
return "" return ""
text = re.sub(r"^\s*(title|标题)\s*[:]\s*", "", text, flags=re.IGNORECASE) text = re.sub(r"^\s*(title|标题)\s*[:]\s*", "", text, flags=re.IGNORECASE)
text = text.strip().strip("\"'`“”‘’") text = text.strip().strip("\"'`“”‘’")
text = strip_think(text)
text = re.sub(r"\s+", " ", text).strip() text = re.sub(r"\s+", " ", text).strip()
text = text.rstrip("。.!?,;:") text = text.rstrip("。.!?,;:")
if len(text) > TITLE_MAX_CHARS: if len(text) > TITLE_MAX_CHARS:
@@ -66,9 +65,6 @@ def _title_inputs(session: Session) -> tuple[str, str]:
content = message.get("content") content = message.get("content")
if not isinstance(content, str) or not content.strip(): if not isinstance(content, str) or not content.strip():
continue continue
content = strip_think(content)
if not content:
continue
if role == "user" and not user_text: if role == "user" and not user_text:
user_text = content.strip() user_text = content.strip()
elif role == "assistant" and not assistant_text: elif role == "assistant" and not assistant_text:
@@ -93,13 +89,7 @@ async def maybe_generate_webui_title(
return False return False
current_title = session.metadata.get(WEBUI_TITLE_METADATA_KEY) current_title = session.metadata.get(WEBUI_TITLE_METADATA_KEY)
if isinstance(current_title, str) and current_title.strip(): if isinstance(current_title, str) and current_title.strip():
cleaned_current_title = clean_generated_title(current_title) return False
if cleaned_current_title:
if cleaned_current_title != current_title:
session.metadata[WEBUI_TITLE_METADATA_KEY] = cleaned_current_title
sessions.save(session)
return False
session.metadata.pop(WEBUI_TITLE_METADATA_KEY, None)
user_text, assistant_text = _title_inputs(session) user_text, assistant_text = _title_inputs(session)
if not user_text: if not user_text:
+2
View File
@@ -34,3 +34,5 @@ Examples (replace `keyword`):
- **Do NOT edit SOUL.md, USER.md, or MEMORY.md.** They are automatically managed by Dream. - **Do NOT edit SOUL.md, USER.md, or MEMORY.md.** They are automatically managed by Dream.
- If you notice outdated information, it will be corrected when Dream runs next. - If you notice outdated information, it will be corrected when Dream runs next.
- Users can view Dream's activity with the `/dream-log` command. - Users can view Dream's activity with the `/dream-log` command.
- Dream runs as a `system` session inside the AgentLoop, triggered by the `/dream` command or cron. Each turn processes one batch; if backlog remains, Dream automatically chains additional turns until complete. All changes are committed in a single git commit.
- Dream can use a different model than the main agent via `agents.defaults.dream.modelOverride`. Supports preset names or raw model identifiers.
+2 -2
View File
@@ -14,10 +14,10 @@ Get USER_ID and CHANNEL from the current session (e.g., `8281248569` and `telegr
## Heartbeat Tasks ## Heartbeat Tasks
`HEARTBEAT.md` is checked periodically when registered as a cron job. Use the built-in `cron` tool to schedule it (e.g. `cron add --name heartbeat --schedule "every 30m" --message "Check HEARTBEAT.md"`). `HEARTBEAT.md` is checked on the configured heartbeat interval. Use file tools to manage periodic tasks.
- Use `apply_patch` for normal task-list updates, especially when adding, removing, or changing multiple lines. - Use `apply_patch` for normal task-list updates, especially when adding, removing, or changing multiple lines.
- Use `edit_file` only for small exact replacements copied from the current `HEARTBEAT.md`. - Use `edit_file` only for small exact replacements copied from the current `HEARTBEAT.md`.
- Use `write_file` for first creation or intentional full-file rewrites. - Use `write_file` for first creation or intentional full-file rewrites.
When the user asks for a recurring/periodic task, update `HEARTBEAT.md` and register it via `cron` instead of creating a one-time reminder. When the user asks for a recurring/periodic task, update `HEARTBEAT.md` instead of creating a one-time cron reminder.
+3 -3
View File
@@ -1,9 +1,9 @@
# Heartbeat Tasks # Heartbeat Tasks
This file is checked periodically by your nanobot agent. This file is checked every 30 minutes by your nanobot agent.
Register it as a cron job (e.g. `cron add --name heartbeat --schedule "every 30m" --message "Check HEARTBEAT.md"`) to get the same behavior as the legacy heartbeat service. Add tasks below that you want the agent to work on periodically.
If this file has no tasks (only headers and comments), the agent will skip it. If this file has no tasks (only headers and comments), the agent will skip the heartbeat.
## Active Tasks ## Active Tasks
@@ -1,13 +1,27 @@
Extract key facts from this conversation. Only output items matching these categories, skip everything else: Extract key facts from this conversation. For each fact, annotate its memory attributes.
- User facts: personal info, preferences, stated opinions, habits
- Decisions: choices made, conclusions reached
- Solutions: working approaches discovered through trial and error, especially non-obvious methods that succeeded after failed attempts
- Events: plans, deadlines, notable occurrences
- Preferences: communication style, tool preferences
Priority: user corrections and preferences > solutions > decisions > events > environment facts. The most valuable memory prevents the user from having to repeat themselves. Only SNIP facts deserve a non-[skip] mark:
- Signal: would the user need to repeat this if forgotten?
- Novel: not already in MEMORY.md or USER.md (check context below)
- Important: prevents rework or captures preferences / rules
- Persistent: still relevant after 2 weeks
Skip: code patterns derivable from source, git history, or anything already captured in existing memory. Output one fact per line in this format:
- [mark] fact content
Marks (choose the best match):
- [permanent] Core preferences, personal traits, habits — never becomes stale
- [durable] Technical discoveries, project knowledge, config details — valid for months
- [ephemeral] Active task state, temporary decisions — may change in weeks
- [correction] Correction to a previous memory — must state what it replaces
- [skip] Does not meet SNIP criteria — still written to history.jsonl for audit, but Dream will ignore it
Categories to capture: people/roles, decisions/rationale, solutions, events/dates, preferences.
Decisions must include their motivation.
Write densely. Prefer 'X=A, Y=B' over separate bullets for tightly coupled facts.
Priority: user corrections > decisions with rationale > solutions > specific events > general context.
Output in the same language as the input conversation.
CRITICAL: Never drop person names, team names, or project names.
Skip: code patterns derivable from source, git history, or anything already in existing memory.
Output as concise bullet points, one fact per line. No preamble, no commentary.
If nothing noteworthy happened, output: (nothing) If nothing noteworthy happened, output: (nothing)
+65
View File
@@ -0,0 +1,65 @@
Update memory files by analyzing conversation history and editing files directly.
Prune before adding — removing stale content is as important as adding new facts.
## File routing
Do NOT guess paths. Route each fact to its canonical file:
| File | Full path | Content |
|------|------|---------|
| SOUL.md | `{{ soul_path }}` | Agent behavior, guardrails, tone, interaction patterns |
| USER.md | `{{ user_path }}` | Personal info, preferences, habits, work context, communication style |
| MEMORY.md | `{{ memory_path }}` | Technical knowledge, project context, infrastructure, accounts |
| SKILL.md | `skills/<name>/SKILL.md` | Reusable workflow templates ([SKILL] entries only) |
Cross-boundary rule: no technical configs in USER.md, no user facts in SOUL.md, no preferences in MEMORY.md. If a fact fits multiple files, keep the most specific copy and remove the rest.
## Delete-or-keep
**Always delete:**
- Same fact at multiple locations — keep canonical copy only
- Merged/closed PR notes, resolved incidents, superseded info
- Verbose entries restatable in fewer words
- Overlapping or nested sections covering the same topic
**Likely delete** (apply judgment):
- Same fact at different detail levels — keep most complete version only
- Debugging steps unlikely to recur
- Ephemeral facts past their useful life
- Tool/service details documented upstream
- Lines with ``← Nd`` where N>{{ stale_threshold_days }} — closer review, not automatic removal
**Never delete:**
- User preferences and personality traits (permanent regardless of age)
- Active project context still referenced in conversations
- Behavioral rules in SOUL.md
When removing: prefer deleting individual items over entire sections.
## Fact extraction
- Atomic facts: "has a cat named Luna" not "discussed pet care"
- Corrections: edit the existing entry, don't append a new one
- Capture confirmed approaches the user validated
## Skill discovery & creation
Flag [SKILL] only when ALL are true: repeatable workflow appeared 2+ times, involves clear steps (not vague preferences), substantial enough for its own instruction set. Check existing skills to avoid redundancy.
For [SKILL] entries:
- Use write_file to create skills/<name>/SKILL.md; read_file `{{ skill_creator_path }}` for format reference
- YAML frontmatter must include name, description, **and `dream_managed: true`** (marks this skill as Dream-created)
- Under 2000 words: when to use, steps, output format, example
- Do NOT overwrite existing skills — if overlapping, merge delta into the existing skill
- Skills are instruction sets, not code. Keep concrete values in MEMORY.md; skills use placeholders
## Skill edit policy
Each skill in the Existing Skills list is tagged with an origin:
- **[dream]** — Dream-created (has `dream_managed: true` in frontmatter). You MAY edit these.
- **[user]** — User-created workspace skill. {% if dream_edit_user_skills %}You MAY edit these.{% else %}You MUST NOT modify, rename, or delete these — you can only read them for context.{% endif %}
- **[builtin]** — Bundled with nanobot. You MUST NEVER modify these.
## Editing
- Default tool: apply_patch. Use edit_file only for small exact replacements.
- File contents provided below — no read_file needed for initial edits.
- Batch all changes into a single apply_patch call. Surgical edits only.
- dry_run=true to preview. If nothing to update, stop without calling tools.
Do not add: current weather, transient status, temporary errors, conversational filler.
-40
View File
@@ -1,40 +0,0 @@
You have TWO equally important tasks:
1. Extract new facts from conversation history
2. Deduplicate existing memory files — find and flag redundant, overlapping, or stale content even if NOT mentioned in history
Output one line per finding:
[FILE] atomic fact (not already in memory)
[FILE-REMOVE] reason for removal
[SKILL] kebab-case-name: one-line description of the reusable pattern
Files: USER (identity, preferences), SOUL (bot behavior, tone), MEMORY (knowledge, project context)
Rules:
- Atomic facts: "has a cat named Luna" not "discussed pet care"
- Corrections: [USER] location is Tokyo, not Osaka
- Capture confirmed approaches the user validated
Deduplication — scan ALL memory files for these redundancy patterns:
- Same fact stated in multiple places (e.g., "communicates in Chinese" in both USER.md and multiple MEMORY.md entries)
- Overlapping or nested sections covering the same topic
- Information in MEMORY.md that is already captured in USER.md or SOUL.md (MEMORY.md should not duplicate permanent-file content)
- Verbose entries that can be condensed without losing information
For each duplicate found, output [FILE-REMOVE] for the less authoritative copy (prefer keeping facts in their canonical location)
Staleness — MEMORY.md lines may have a ``← Nd`` suffix showing days since last modification:
- SOUL.md and USER.md have no age annotations — they are permanent, only update with corrections
- Age only indicates when content was last touched, not whether it should be removed
- Use content judgment: user habits/preferences/personality traits are permanent regardless of age
- Only prune content that is objectively outdated: passed events, resolved tracking, superseded approaches
- Lines with ``← Nd`` (N>{{ stale_threshold_days }}) deserve closer review but are NOT automatically removable
- When removing: prefer deleting individual items over entire sections
Skill discovery — flag [SKILL] when ALL of these are true:
- A specific, repeatable workflow appeared 2+ times in the conversation history
- It involves clear steps (not vague preferences like "likes concise answers")
- It is substantial enough to warrant its own instruction set (not trivial like "read a file")
- Do not worry about duplicates — the next phase will check against existing skills
Do not add: current weather, transient status, temporary errors, conversational filler.
[SKIP] if nothing needs updating.
-37
View File
@@ -1,37 +0,0 @@
Update memory files based on the analysis below.
- [FILE] entries: add the described content to the appropriate file
- [FILE-REMOVE] entries: delete the corresponding content from memory files
- [SKILL] entries: create a new skill under skills/<name>/SKILL.md using write_file
## File paths (relative to workspace root)
- SOUL.md
- USER.md
- memory/MEMORY.md
- skills/<name>/SKILL.md (for [SKILL] entries only)
Do NOT guess paths.
## Editing rules
- Edit directly — file contents provided below, no read_file needed
- Use exact text as old_text, include surrounding blank lines for unique match
- Batch changes to the same file into one edit_file call
- For deletions: section header + all bullets as old_text, new_text empty
- Surgical edits only — never rewrite entire files
- If nothing to update, stop without calling tools
## Skill creation rules (for [SKILL] entries)
- Use write_file to create skills/<name>/SKILL.md
- Before writing, read_file `{{ skill_creator_path }}` for format reference (frontmatter structure, naming conventions, quality standards)
- **Dedup check**: read existing skills listed below to verify the new skill is not functionally redundant. Skip creation if an existing skill already covers the same workflow.
- Include YAML frontmatter with name and description fields
- Keep SKILL.md under 2000 words — concise and actionable
- Include: when to use, steps, output format, at least one example
- Do NOT overwrite existing skills — skip if the skill directory already exists
- Reference specific tools the agent has access to (read_file, write_file, exec, web_search, etc.)
- Skills are instruction sets, not code — do not include implementation code
## Quality
- Every line must carry standalone value
- Concise bullets under clear headers
- When reducing (not deleting): keep essential facts, drop verbose details
- If uncertain whether to delete, keep but add "(verify currency)"
+1 -1
View File
@@ -63,5 +63,5 @@ documents the general tool contract and non-obvious usage patterns.
## Scheduling and Background Work ## Scheduling and Background Work
- Use `cron` for scheduled reminders or recurring jobs; do not run `nanobot cron` through `exec`. - Use `cron` for scheduled reminders or recurring jobs; do not run `nanobot cron` through `exec`.
- For heartbeat tasks, register `HEARTBEAT.md` as a cron job according to the agent instructions. - For heartbeat tasks, update `HEARTBEAT.md` according to the agent instructions.
- Do not write reminders only to memory files when the user expects an actual notification. - Do not write reminders only to memory files when the user expects an actual notification.
+5 -41
View File
@@ -7,6 +7,7 @@ from loguru import logger
from nanobot.utils.helpers import detect_image_mime from nanobot.utils.helpers import detect_image_mime
# Supported file extensions for text extraction # Supported file extensions for text extraction
SUPPORTED_EXTENSIONS: set[str] = { SUPPORTED_EXTENSIONS: set[str] = {
# Document formats # Document formats
@@ -231,46 +232,6 @@ def _is_text_extension(ext: str) -> bool:
_MAX_EXTRACT_FILE_SIZE = 50 * 1024 * 1024 # 50 MB _MAX_EXTRACT_FILE_SIZE = 50 * 1024 * 1024 # 50 MB
def is_image_file(path: str) -> bool:
"""Check whether *path* looks like an image file.
Uses magic-byte detection (reads first 16 bytes) with a ``mimetypes``
extension-based fallback.
"""
p = Path(path)
mime: str | None = None
if p.is_file():
try:
with p.open("rb") as f:
mime = detect_image_mime(f.read(16))
except OSError:
mime = None
if not mime:
mime = mimetypes.guess_type(path)[0]
return bool(mime and mime.startswith("image/"))
def reference_non_image_attachments(
content: str, media: list[str],
) -> tuple[str, list[str]]:
"""Separate images from non-image attachments without reading file content.
Image paths are preserved for downstream vision-block construction.
Non-image paths are appended as ``[Attachment: path]`` references.
"""
image_paths: list[str] = []
attachment_refs: list[str] = []
for path in media:
if is_image_file(path):
image_paths.append(path)
else:
attachment_refs.append(f"[Attachment: {path}]")
if attachment_refs:
suffix = "\n".join(attachment_refs)
content = f"{content}\n\n{suffix}" if content else suffix
return content, image_paths
def extract_documents( def extract_documents(
text: str, text: str,
media_paths: list[str], media_paths: list[str],
@@ -306,7 +267,10 @@ def extract_documents(
) )
continue continue
if is_image_file(path_str): with open(p, "rb") as f:
header = f.read(16)
mime = detect_image_mime(header) or mimetypes.guess_type(path_str)[0]
if mime and mime.startswith("image/"):
image_paths.append(path_str) image_paths.append(path_str)
else: else:
extracted = extract_text(p) extracted = extract_text(p)
+9 -16
View File
@@ -44,15 +44,12 @@ async def evaluate_response(
task_context: str, task_context: str,
provider: LLMProvider, provider: LLMProvider,
model: str, model: str,
*,
default_notify: bool = True,
) -> bool: ) -> bool:
"""Decide whether a background-task result should be delivered to the user. """Decide whether a background-task result should be delivered to the user.
Uses a lightweight tool-call LLM request. ``default_notify`` controls Uses a lightweight tool-call LLM request (same pattern as heartbeat
the fallback path when the evaluator cannot produce a valid decision: ``_decide()``). Falls back to ``True`` (notify) on any failure so
user-scheduled reminders stay fail-open, while internal checks such as that important messages are never silently dropped.
heartbeat can fail closed.
""" """
try: try:
llm_response = await provider.chat_with_retry( llm_response = await provider.chat_with_retry(
@@ -74,23 +71,19 @@ async def evaluate_response(
if not llm_response.should_execute_tools: if not llm_response.should_execute_tools:
if llm_response.has_tool_calls: if llm_response.has_tool_calls:
logger.warning( logger.warning(
"evaluate_response: ignoring tool calls under finish_reason='{}', defaulting to notify={}", "evaluate_response: ignoring tool calls under finish_reason='{}', defaulting to notify",
llm_response.finish_reason, llm_response.finish_reason,
default_notify,
) )
else: else:
logger.warning( logger.warning("evaluate_response: no tool call returned, defaulting to notify")
"evaluate_response: no tool call returned, defaulting to notify={}", return True
default_notify,
)
return default_notify
args = llm_response.tool_calls[0].arguments args = llm_response.tool_calls[0].arguments
should_notify = args.get("should_notify", default_notify) should_notify = args.get("should_notify", True)
reason = args.get("reason", "") reason = args.get("reason", "")
logger.info("evaluate_response: should_notify={}, reason={}", should_notify, reason) logger.info("evaluate_response: should_notify={}, reason={}", should_notify, reason)
return bool(should_notify) return bool(should_notify)
except Exception: except Exception:
logger.exception("evaluate_response failed, defaulting to notify={}", default_notify) logger.exception("evaluate_response failed, defaulting to notify")
return default_notify return True
+5 -8
View File
@@ -299,7 +299,6 @@ def build_file_edit_end_event(
deleted=deleted, deleted=deleted,
approximate=False, approximate=False,
binary=(after.binary or after.oversized or after.unreadable) and not counted, binary=(after.binary or after.oversized or after.unreadable) and not counted,
operation="delete" if tracker.before.exists and not after.exists else None,
) )
@@ -325,7 +324,6 @@ def build_file_edit_live_event(
*, *,
added: int, added: int,
deleted: int = 0, deleted: int = 0,
operation: str | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
"""Build an approximate in-progress event while tool-call arguments stream.""" """Build an approximate in-progress event while tool-call arguments stream."""
return _event_payload( return _event_payload(
@@ -335,7 +333,6 @@ def build_file_edit_live_event(
added=added, added=added,
deleted=deleted, deleted=deleted,
approximate=True, approximate=True,
operation=operation,
) )
@@ -457,14 +454,15 @@ class StreamingFileEditTracker:
segment_end = path_matches[i + 1].start() if i + 1 < len(path_matches) else len(state.arguments) segment_end = path_matches[i + 1].start() if i + 1 < len(path_matches) else len(state.arguments)
segment = state.arguments[segment_start:segment_end] segment = state.arguments[segment_start:segment_end]
action_match = re.search(r'"action"\s*:\s*"(replace|add)"', segment) action_match = re.search(r'"action"\s*:\s*"(replace|add|delete)"', segment)
action = action_match.group(1) if action_match else "replace" action = action_match.group(1) if action_match else "replace"
old_text = _extract_json_string_prefix(segment, "old_text") or "" old_text = _extract_json_string_prefix(segment, "old_text") or ""
new_text = _extract_json_string_prefix(segment, "new_text") or "" new_text = _extract_json_string_prefix(segment, "new_text") or ""
added = _text_line_count(new_text) if action in ("replace", "add") else 0 added = _text_line_count(new_text) if action in ("replace", "add") else 0
deleted = _text_line_count(old_text) if action == "replace" else 0 deleted = _text_line_count(old_text) if action in ("replace", "delete") else 0
delete_file = action == "delete"
file_state = state.patch_files.get(raw_path) file_state = state.patch_files.get(raw_path)
if file_state is None: if file_state is None:
@@ -477,6 +475,8 @@ class StreamingFileEditTracker:
) )
file_state = _StreamingPatchFileState(tracker=tracker) file_state = _StreamingPatchFileState(tracker=tracker)
state.patch_files[raw_path] = file_state state.patch_files[raw_path] = file_state
if delete_file and added == 0 and deleted == 0 and file_state.tracker.before.countable:
deleted = _text_line_count(file_state.tracker.before.text or "")
if not file_state.should_emit(added, deleted, now): if not file_state.should_emit(added, deleted, now):
continue continue
file_state.mark_emitted(added, deleted, now) file_state.mark_emitted(added, deleted, now)
@@ -916,7 +916,6 @@ def _event_payload(
deleted: int, deleted: int,
approximate: bool, approximate: bool,
binary: bool = False, binary: bool = False,
operation: str | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
payload: dict[str, Any] = { payload: dict[str, Any] = {
"version": 1, "version": 1,
@@ -932,8 +931,6 @@ def _event_payload(
} }
if binary: if binary:
payload["binary"] = True payload["binary"] = True
if operation:
payload["operation"] = operation
return payload return payload
+2 -1
View File
@@ -19,7 +19,8 @@ class CommitInfo:
def format(self, diff: str = "") -> str: def format(self, diff: str = "") -> str:
"""Format this commit for display, optionally with a diff.""" """Format this commit for display, optionally with a diff."""
header = f"## {self.message.splitlines()[0]}\n`{self.sha}` — {self.timestamp}\n" summary = self.message.splitlines()[0] if self.message else "(no message)"
header = f"## {summary}\n`{self.sha}` — {self.timestamp}\n"
if diff: if diff:
return f"{header}\n```diff\n{diff}\n```" return f"{header}\n```diff\n{diff}\n```"
return f"{header}\n(no file changes)" return f"{header}\n(no file changes)"
-11
View File
@@ -626,14 +626,3 @@ def sync_workspace_templates(workspace: Path, silent: bool = False) -> list[str]
logger.exception("Failed to initialize git store for {}", workspace) logger.exception("Failed to initialize git store for {}", workspace)
return added return added
def load_bundled_template(template_name: str) -> str | None:
"""Read a bundled template file from the nanobot package."""
from importlib.resources import files as pkg_files
with suppress(Exception):
tpl = pkg_files("nanobot") / "templates" / template_name
if tpl.is_file():
return tpl.read_text(encoding="utf-8")
return None
+4 -4
View File
@@ -124,7 +124,7 @@ MCP_PRESETS: tuple[McpPreset, ...] = (
name="playwright", name="playwright",
display_name="Playwright", display_name="Playwright",
category="browser", category="browser",
description="Local browser inspection and automation with Playwright's MCP server.", description="Local browser inspection and automation with the official Playwright MCP server.",
docs_url="https://playwright.dev/docs/getting-started-mcp", docs_url="https://playwright.dev/docs/getting-started-mcp",
transport="stdio", transport="stdio",
install_supported=True, install_supported=True,
@@ -216,7 +216,7 @@ MCP_PRESETS: tuple[McpPreset, ...] = (
name="microsoft-learn", name="microsoft-learn",
display_name="Microsoft Learn", display_name="Microsoft Learn",
category="docs", category="docs",
description="Search and fetch Microsoft Learn documentation through Microsoft's hosted MCP server.", description="Search and fetch official Microsoft Learn documentation through Microsoft's hosted MCP server.",
docs_url="https://learn.microsoft.com/en-us/training/support/mcp", docs_url="https://learn.microsoft.com/en-us/training/support/mcp",
transport="streamableHttp", transport="streamableHttp",
install_supported=True, install_supported=True,
@@ -307,7 +307,7 @@ MCP_PRESETS: tuple[McpPreset, ...] = (
name="figma", name="figma",
display_name="Figma", display_name="Figma",
category="design", category="design",
description="Read design context from Figma using the local Dev Mode MCP server.", description="Read design context from Figma using the official local Dev Mode MCP server.",
docs_url="https://help.figma.com/hc/en-us/articles/32132100833559-Guide-to-the-Figma-MCP-server", docs_url="https://help.figma.com/hc/en-us/articles/32132100833559-Guide-to-the-Figma-MCP-server",
transport="streamableHttp", transport="streamableHttp",
install_supported=True, install_supported=True,
@@ -325,7 +325,7 @@ MCP_PRESETS: tuple[McpPreset, ...] = (
name="github", name="github",
display_name="GitHub", display_name="GitHub",
category="code", category="code",
description="Repository, issue, and pull request workflows via GitHub's MCP server.", description="Repository, issue, and pull request workflows via GitHub's official MCP server.",
docs_url="https://github.com/github/github-mcp-server", docs_url="https://github.com/github/github-mcp-server",
transport="stdio", transport="stdio",
install_supported=True, install_supported=True,
-255
View File
@@ -1,255 +0,0 @@
"""Signed media helpers for the WebUI HTTP surface."""
from __future__ import annotations
import base64
import binascii
import email.utils
import hashlib
import hmac
import http
import mimetypes
import re
import shutil
import uuid
from collections.abc import Callable
from pathlib import Path
from typing import Any
from websockets.datastructures import Headers
from websockets.http11 import Request as WsRequest
from websockets.http11 import Response
from nanobot.config.paths import get_media_dir
from nanobot.utils.helpers import safe_filename
MediaDirProvider = Callable[[str | None], Path]
def b64url_encode(data: bytes) -> str:
"""URL-safe base64 without padding."""
return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
def b64url_decode(value: str) -> bytes:
"""Reverse of :func:`b64url_encode`; caller handles decode errors."""
pad = "=" * (-len(value) % 4)
return base64.urlsafe_b64decode(value + pad)
def _default_media_dir(channel: str | None = None) -> Path:
return get_media_dir(channel)
# Allowed MIME types we actually serve from the media endpoint. Anything
# outside this set is degraded to ``application/octet-stream`` so an
# attacker who somehow gets a signed URL for an unexpected file type can't
# trick the browser into sniffing executable content.
_MEDIA_ALLOWED_MIMES: frozenset[str] = frozenset({
"image/png",
"image/jpeg",
"image/webp",
"image/gif",
"image/svg+xml",
"video/mp4",
"video/webm",
"video/quicktime",
})
_SVG_MEDIA_HEADERS: tuple[tuple[str, str], ...] = (
(
"Content-Security-Policy",
"default-src 'none'; img-src 'self' data:; style-src 'unsafe-inline'; sandbox",
),
)
_BYTE_RANGE_RE = re.compile(r"^bytes=(\d*)-(\d*)$")
def _http_response(
body: bytes,
*,
status: int = 200,
content_type: str = "text/plain; charset=utf-8",
extra_headers: list[tuple[str, str]] | None = None,
) -> Response:
headers = [
("Date", email.utils.formatdate(usegmt=True)),
("Connection", "close"),
("Content-Length", str(len(body))),
("Content-Type", content_type),
]
if extra_headers:
headers.extend(extra_headers)
reason = http.HTTPStatus(status).phrase
return Response(status, reason, Headers(headers), body)
def _http_error(status: int, message: str | None = None) -> Response:
body = (message or http.HTTPStatus(status).phrase).encode("utf-8")
return _http_response(body, status=status)
def _case_insensitive_header(headers: Any, key: str) -> str:
try:
value = headers.get(key)
except Exception:
value = None
if value is None:
try:
value = headers.get(key.lower())
except Exception:
value = None
return str(value or "").strip()
def _parse_single_byte_range(range_header: str, size: int) -> tuple[int, int]:
"""Parse a single HTTP byte range for signed media responses."""
if size <= 0 or "," in range_header:
raise ValueError("invalid byte range")
m = _BYTE_RANGE_RE.fullmatch(range_header.strip())
if m is None:
raise ValueError("invalid byte range")
start_text, end_text = m.groups()
if not start_text and not end_text:
raise ValueError("invalid byte range")
if not start_text:
suffix_length = int(end_text)
if suffix_length <= 0:
raise ValueError("invalid byte range")
start = max(size - suffix_length, 0)
end = size - 1
else:
start = int(start_text)
end = int(end_text) if end_text else size - 1
if start >= size or start > end:
raise ValueError("invalid byte range")
end = min(end, size - 1)
return start, end
def sign_media_path(
abs_path: Path,
*,
secret: bytes,
media_dir: MediaDirProvider = _default_media_dir,
) -> str | None:
"""Return a signed ``/api/media/<sig>/<payload>`` URL for a media-root path."""
try:
media_root = media_dir(None).resolve()
rel = abs_path.resolve().relative_to(media_root)
except (OSError, ValueError):
return None
payload = b64url_encode(rel.as_posix().encode("utf-8"))
mac = hmac.new(secret, payload.encode("ascii"), hashlib.sha256).digest()[:16]
return f"/api/media/{b64url_encode(mac)}/{payload}"
def sign_or_stage_media_path(
path: Path,
*,
secret: bytes,
media_dir: MediaDirProvider = _default_media_dir,
logger: Any | None = None,
) -> dict[str, str] | None:
"""Sign an existing media-root path, or stage an arbitrary file before signing."""
signed = sign_media_path(path, secret=secret, media_dir=media_dir)
if signed is not None:
return {"url": signed, "name": path.name}
try:
if not path.is_file():
return None
target_dir = media_dir("websocket")
safe_name = safe_filename(path.name) or "attachment"
staged = target_dir / f"{uuid.uuid4().hex[:12]}-{safe_name}"
shutil.copyfile(path, staged)
except OSError as exc:
if logger is not None:
logger.warning("failed to stage outbound media {}: {}", path, exc)
return None
signed = sign_media_path(staged, secret=secret, media_dir=media_dir)
if signed is None:
return None
return {"url": signed, "name": path.name}
def serve_signed_media(
sig: str,
payload: str,
*,
secret: bytes,
request: WsRequest | None = None,
media_dir: MediaDirProvider = _default_media_dir,
) -> Response:
"""Serve a signed media URL, including browser-friendly byte ranges."""
try:
provided_mac = b64url_decode(sig)
except (ValueError, binascii.Error):
return _http_error(401, "invalid signature")
expected_mac = hmac.new(secret, payload.encode("ascii"), hashlib.sha256).digest()[:16]
if not hmac.compare_digest(expected_mac, provided_mac):
return _http_error(401, "invalid signature")
try:
rel_bytes = b64url_decode(payload)
rel_str = rel_bytes.decode("utf-8")
except (ValueError, binascii.Error, UnicodeDecodeError):
return _http_error(400, "invalid payload")
try:
media_root = media_dir(None).resolve()
candidate = (media_root / rel_str).resolve()
candidate.relative_to(media_root)
except (OSError, ValueError):
return _http_error(404, "not found")
if not candidate.is_file():
return _http_error(404, "not found")
mime, _ = mimetypes.guess_type(candidate.name)
if mime not in _MEDIA_ALLOWED_MIMES:
mime = "application/octet-stream"
common_headers = [
("Accept-Ranges", "bytes"),
("Cache-Control", "private, max-age=31536000, immutable"),
("X-Content-Type-Options", "nosniff"),
]
if mime == "image/svg+xml":
common_headers.extend(_SVG_MEDIA_HEADERS)
try:
size = candidate.stat().st_size
except OSError:
return _http_error(500, "read error")
range_header = _case_insensitive_header(request.headers, "Range") if request else ""
if range_header:
try:
start, end = _parse_single_byte_range(range_header, size)
except ValueError:
return _http_response(
b"range not satisfiable",
status=416,
extra_headers=[
("Accept-Ranges", "bytes"),
("Content-Range", f"bytes */{size}"),
("X-Content-Type-Options", "nosniff"),
],
)
try:
length = end - start + 1
with candidate.open("rb") as fh:
fh.seek(start)
body = fh.read(length)
except OSError:
return _http_error(500, "read error")
return _http_response(
body,
status=206,
content_type=mime,
extra_headers=[
*common_headers,
("Content-Range", f"bytes {start}-{end}/{size}"),
],
)
try:
body = candidate.read_bytes()
except OSError:
return _http_error(500, "read error")
return _http_response(body, content_type=mime, extra_headers=common_headers)
+7 -631
View File
@@ -6,15 +6,10 @@ settings payload shape and the allowlisted config mutations exposed to WebUI.
from __future__ import annotations from __future__ import annotations
import os
import re import re
import time from typing import Any
from contextlib import suppress
from typing import Any, Literal
from zoneinfo import ZoneInfo from zoneinfo import ZoneInfo
import httpx
from nanobot.config.loader import get_config_path, load_config, save_config from nanobot.config.loader import get_config_path, load_config, save_config
from nanobot.config.schema import ModelPresetConfig from nanobot.config.schema import ModelPresetConfig
from nanobot.providers.image_generation import ( from nanobot.providers.image_generation import (
@@ -22,48 +17,8 @@ from nanobot.providers.image_generation import (
image_gen_provider_names, image_gen_provider_names,
) )
from nanobot.providers.registry import PROVIDERS, find_by_name from nanobot.providers.registry import PROVIDERS, find_by_name
from nanobot.security.workspace_access import workspace_sandbox_status
from nanobot.webui.workspaces import (
read_webui_default_access_mode,
write_webui_default_access_mode,
)
QueryParams = dict[str, list[str]] QueryParams = dict[str, list[str]]
RuntimeSurface = Literal["browser", "native"]
_RUNTIME_CAPABILITIES = {
"can_restart_engine": False,
"can_pick_folder": False,
"can_open_logs": False,
"can_export_diagnostics": False,
}
_NATIVE_RUNTIME_CAPABILITIES = {
**_RUNTIME_CAPABILITIES,
"can_restart_engine": True,
"can_pick_folder": True,
"can_open_logs": True,
"can_export_diagnostics": True,
}
_BROWSER_RESTART_BEHAVIOR_BY_SECTION = {
"appearance": "none",
"models": "none",
"providers": "none",
"runtime": "engineRestart",
"browser": "engineRestart",
"image": "engineRestart",
"apps": "engineRestart",
"advanced": "appRestart",
}
_NATIVE_RESTART_BEHAVIOR_BY_SECTION = {
**_BROWSER_RESTART_BEHAVIOR_BY_SECTION,
"runtime": "engineRestart",
"browser": "engineRestart",
"image": "engineRestart",
"apps": "engineRestart",
}
_WEB_SEARCH_PROVIDER_OPTIONS: tuple[dict[str, str], ...] = ( _WEB_SEARCH_PROVIDER_OPTIONS: tuple[dict[str, str], ...] = (
{"name": "duckduckgo", "label": "DuckDuckGo", "credential": "none"}, {"name": "duckduckgo", "label": "DuckDuckGo", "credential": "none"},
@@ -88,49 +43,7 @@ _IMAGE_GENERATION_ASPECT_RATIOS = {
"2:3", "2:3",
"21:9", "21:9",
} }
_CONTEXT_WINDOW_TOKEN_OPTIONS = {65_536, 262_144}
_MODEL_CONFIGURATION_SLUG_RE = re.compile(r"[^a-z0-9_-]+") _MODEL_CONFIGURATION_SLUG_RE = re.compile(r"[^a-z0-9_-]+")
_ENV_REF_RE = re.compile(r"\$\{([A-Za-z_][A-Za-z0-9_]*)\}")
_MODEL_LIST_UNSUPPORTED_BACKENDS = {
"anthropic",
"azure_openai",
"bedrock",
"github_copilot",
"openai_codex",
}
_MODEL_LIST_CATALOG_PROVIDERS = {
"aihubmix",
"byteplus",
"byteplus_coding_plan",
"huggingface",
"novita",
"openrouter",
"siliconflow",
"volcengine",
"volcengine_coding_plan",
}
_MODEL_LIST_OFFICIAL_PROVIDERS = {
"ant_ling",
"dashscope",
"deepseek",
"gemini",
"groq",
"longcat",
"minimax",
"minimax_anthropic",
"mistral",
"moonshot",
"nvidia",
"openai",
"qianfan",
"skywork",
"stepfun",
"xiaomi_mimo",
"zhipu",
}
class WebUISettingsError(ValueError): class WebUISettingsError(ValueError):
@@ -142,70 +55,6 @@ class WebUISettingsError(ValueError):
self.status = status self.status = status
def _normalize_surface(surface: str | None) -> RuntimeSurface:
return "native" if surface in {"native", "desktop"} else "browser"
def runtime_capabilities(
surface: str | None = "browser",
overrides: dict[str, Any] | None = None,
) -> dict[str, bool]:
"""Return the capability flags exposed to the WebUI runtime."""
base = (
_NATIVE_RUNTIME_CAPABILITIES
if _normalize_surface(surface) == "native"
else _RUNTIME_CAPABILITIES
)
result = dict(base)
for key, value in (overrides or {}).items():
if key in result:
result[key] = bool(value)
return result
def restart_behavior_by_section(surface: str | None = "browser") -> dict[str, str]:
return dict(
_NATIVE_RESTART_BEHAVIOR_BY_SECTION
if _normalize_surface(surface) == "native"
else _BROWSER_RESTART_BEHAVIOR_BY_SECTION
)
def decorate_settings_payload(
payload: dict[str, Any],
*,
surface: str | None = "browser",
runtime_capability_overrides: dict[str, Any] | None = None,
restart_required_sections: list[str] | None = None,
apply_state: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Attach runtime-surface metadata without changing the core settings shape."""
surface_value = _normalize_surface(surface)
sections = restart_required_sections
if sections is None:
raw_sections = payload.get("restart_required_sections") or []
sections = [str(section) for section in raw_sections if isinstance(section, str)]
sections = sorted(dict.fromkeys(sections))
result = dict(payload)
result["surface"] = surface_value
result["runtime_surface"] = surface_value
result["runtime_capabilities"] = runtime_capabilities(
surface_value,
runtime_capability_overrides,
)
result["restart_behavior_by_section"] = restart_behavior_by_section(surface_value)
result["restart_required_sections"] = sections
if sections:
result["requires_restart"] = True
else:
result["requires_restart"] = bool(result.get("requires_restart", False))
result["apply_state"] = apply_state or {
"status": "pending" if result["requires_restart"] else "idle",
"sections": sections,
}
return result
def _query_first(query: QueryParams, key: str) -> str | None: def _query_first(query: QueryParams, key: str) -> str | None:
values = query.get(key) values = query.get(key)
return values[0] if values else None return values[0] if values else None
@@ -224,25 +73,6 @@ def _mask_secret_hint(secret: str | None) -> str | None:
return f"{secret[:4]}••••{secret[-4:]}" return f"{secret[:4]}••••{secret[-4:]}"
def _resolve_env_placeholders(value: str | None) -> str | None:
if not value:
return None
missing = False
def replace(match: re.Match[str]) -> str:
nonlocal missing
env_value = os.environ.get(match.group(1))
if env_value is None:
missing = True
return ""
return env_value
resolved = _ENV_REF_RE.sub(replace, value).strip()
if missing and not resolved:
return None
return resolved or None
def _provider_requires_api_key(spec: Any) -> bool: def _provider_requires_api_key(spec: Any) -> bool:
if spec.backend == "azure_openai": if spec.backend == "azure_openai":
return True return True
@@ -253,57 +83,9 @@ def _provider_requires_api_key(spec: Any) -> bool:
return True return True
def _oauth_provider_status(spec: Any) -> dict[str, Any]:
if not getattr(spec, "is_oauth", False):
return {"configured": False, "account": None, "expires_at": None, "login_supported": False}
if spec.name == "openai_codex":
try:
from oauth_cli_kit import get_token as get_codex_token
except Exception:
return {
"configured": False,
"account": None,
"expires_at": None,
"login_supported": False,
}
token = None
with suppress(Exception):
token = get_codex_token()
expires_at = getattr(token, "expires", None) if token else None
return {
"configured": bool(token and token.access),
"account": getattr(token, "account_id", None) if token else None,
"expires_at": expires_at,
"login_supported": True,
}
if spec.name == "github_copilot":
try:
from nanobot.providers.github_copilot_provider import get_github_copilot_login_status
except Exception:
return {
"configured": False,
"account": None,
"expires_at": None,
"login_supported": False,
}
token = None
with suppress(Exception):
token = get_github_copilot_login_status()
return {
"configured": bool(token and token.access and token.expires > int(time.time() * 1000)),
"account": getattr(token, "account_id", None) if token else None,
"expires_at": getattr(token, "expires", None) if token else None,
"login_supported": True,
}
return {"configured": False, "account": None, "expires_at": None, "login_supported": False}
def _provider_configured_for_settings(spec: Any, provider_config: Any) -> bool: def _provider_configured_for_settings(spec: Any, provider_config: Any) -> bool:
if spec.is_oauth: if spec.is_oauth:
return bool(_oauth_provider_status(spec)["configured"]) return True
if _provider_requires_api_key(spec): if _provider_requires_api_key(spec):
return bool(provider_config.api_key) return bool(provider_config.api_key)
return bool( return bool(
@@ -314,191 +96,6 @@ def _provider_configured_for_settings(spec: Any, provider_config: Any) -> bool:
) )
def _model_catalog_kind(spec: Any) -> str:
if spec.name in _MODEL_LIST_CATALOG_PROVIDERS:
return "catalog"
if spec.name in _MODEL_LIST_OFFICIAL_PROVIDERS:
return "official"
if spec.is_local:
return "local"
if spec.is_direct:
return "custom"
if spec.is_gateway:
return "catalog"
return "official"
def _model_id_from_row(row: Any) -> str | None:
if isinstance(row, str):
return row.strip() or None
if not isinstance(row, dict):
return None
for key in ("id", "name", "model"):
value = row.get(key)
if isinstance(value, str) and value.strip():
return value.strip()
return None
def _model_context_window(row: Any) -> int | None:
if not isinstance(row, dict):
return None
for key in (
"context_window",
"context_length",
"max_context_length",
"max_model_len",
"max_input_tokens",
):
value = row.get(key)
if isinstance(value, int) and value > 0:
return value
if isinstance(value, float) and value > 0:
return int(value)
return None
def _model_row_payload(row: Any) -> dict[str, Any] | None:
model_id = _model_id_from_row(row)
if not model_id:
return None
label: str | None = None
owned_by: str | None = None
if isinstance(row, dict):
raw_label = row.get("display_name") or row.get("label") or row.get("name")
if isinstance(raw_label, str) and raw_label.strip() and raw_label.strip() != model_id:
label = raw_label.strip()
raw_owner = row.get("owned_by") or row.get("owner") or row.get("organization")
if isinstance(raw_owner, str) and raw_owner.strip():
owned_by = raw_owner.strip()
return {
"id": model_id,
"label": label,
"owned_by": owned_by,
"context_window": _model_context_window(row),
}
def _extract_model_rows(body: Any) -> list[dict[str, Any]]:
raw_rows = body.get("data") if isinstance(body, dict) else body
if not isinstance(raw_rows, list):
return []
rows: list[dict[str, Any]] = []
seen: set[str] = set()
for raw_row in raw_rows:
row = _model_row_payload(raw_row)
if row is None or row["id"] in seen:
continue
seen.add(row["id"])
rows.append(row)
return rows
def provider_models_payload(query: QueryParams) -> dict[str, Any]:
"""Fetch an OpenAI-compatible provider's model list for Settings.
The result is advisory only: users can always type a custom model id. This
helper deliberately avoids mutating config so probing model lists never
changes runtime behavior.
"""
provider_name = (_query_first(query, "provider") or "").strip()
if not provider_name:
raise WebUISettingsError("provider is required")
spec = find_by_name(provider_name)
if spec is None:
raise WebUISettingsError("unknown provider")
base_payload: dict[str, Any] = {
"provider": spec.name,
"label": spec.label,
"catalog_kind": _model_catalog_kind(spec),
"models": [],
"model_count": 0,
"message": None,
"fetched_at": time.time(),
}
if (
spec.backend in _MODEL_LIST_UNSUPPORTED_BACKENDS
and spec.name != "minimax_anthropic"
) or spec.is_oauth:
return {
**base_payload,
"status": "unsupported",
"catalog_kind": "unsupported",
"message": "Model list is not available for this provider. Type a model ID manually.",
}
config = load_config()
provider_config = getattr(config.providers, spec.name, None)
if provider_config is None:
raise WebUISettingsError("unknown provider")
api_base = _resolve_env_placeholders(provider_config.api_base) or spec.default_api_base
if spec.name == "openai" and not api_base:
api_base = "https://api.openai.com/v1"
if not api_base:
return {
**base_payload,
"status": "missing_api_base",
"message": "Configure an API base URL to load models.",
}
api_key = _resolve_env_placeholders(provider_config.api_key)
if _provider_requires_api_key(spec) and not api_key:
return {
**base_payload,
"status": "not_configured",
"message": "Configure this provider before loading models.",
}
headers = {"Accept": "application/json"}
if api_key:
if spec.name == "minimax_anthropic":
headers["X-Api-Key"] = api_key
else:
headers["Authorization"] = f"Bearer {api_key}"
models_url = f"{api_base.rstrip('/')}/models"
if spec.name == "minimax_anthropic" and not api_base.rstrip("/").endswith("/v1"):
models_url = f"{api_base.rstrip('/')}/v1/models"
try:
response = httpx.get(
models_url,
headers=headers,
timeout=10.0,
follow_redirects=False,
)
response.raise_for_status()
rows = _extract_model_rows(response.json())
except httpx.HTTPStatusError as exc:
status = exc.response.status_code
if status in {401, 403}:
return {
**base_payload,
"status": "not_configured",
"message": "The provider rejected the configured credential.",
}
return {
**base_payload,
"status": "error",
"message": f"Model list request failed with HTTP {status}.",
}
except (httpx.HTTPError, ValueError) as exc:
return {
**base_payload,
"status": "error",
"message": f"Could not load models: {exc}",
}
return {
**base_payload,
"status": "available",
"models": rows,
"model_count": len(rows),
}
def _parse_bool(value: str, field: str) -> bool: def _parse_bool(value: str, field: str) -> bool:
normalized = value.strip().lower() normalized = value.strip().lower()
if normalized not in {"1", "0", "true", "false", "yes", "no"}: if normalized not in {"1", "0", "true", "false", "yes", "no"}:
@@ -506,18 +103,6 @@ def _parse_bool(value: str, field: str) -> bool:
return normalized in {"1", "true", "yes"} return normalized in {"1", "true", "yes"}
def _parse_context_window_tokens(value: str | None) -> int | None:
if value is None:
return None
try:
parsed = int(value)
except ValueError:
raise WebUISettingsError("context_window_tokens must be an integer") from None
if parsed not in _CONTEXT_WINDOW_TOKEN_OPTIONS:
raise WebUISettingsError("context_window_tokens must be 65536 or 262144")
return parsed
def _model_configuration_slug(label: str) -> str: def _model_configuration_slug(label: str) -> str:
normalized = _MODEL_CONFIGURATION_SLUG_RE.sub("-", label.strip().lower()) normalized = _MODEL_CONFIGURATION_SLUG_RE.sub("-", label.strip().lower())
normalized = normalized.strip("-_") normalized = normalized.strip("-_")
@@ -559,7 +144,6 @@ def _image_generation_provider_rows(config: Any) -> list[dict[str, Any]]:
"name": name, "name": name,
"label": spec.label if spec is not None else name, "label": spec.label if spec is not None else name,
"configured": configured, "configured": configured,
"auth_type": "oauth" if spec is not None and spec.is_oauth else "api_key",
"api_key_hint": _mask_secret_hint( "api_key_hint": _mask_secret_hint(
getattr(provider_config, "api_key", None) getattr(provider_config, "api_key", None)
), ),
@@ -572,14 +156,7 @@ def _image_generation_provider_rows(config: Any) -> list[dict[str, Any]]:
return rows return rows
def settings_payload( def settings_payload(*, requires_restart: bool = False) -> dict[str, Any]:
*,
requires_restart: bool = False,
surface: str | None = "browser",
runtime_capability_overrides: dict[str, Any] | None = None,
restart_required_sections: list[str] | None = None,
apply_state: dict[str, Any] | None = None,
) -> dict[str, Any]:
config = load_config() config = load_config()
defaults = config.agents.defaults defaults = config.agents.defaults
active_preset_name = defaults.model_preset or "default" active_preset_name = defaults.model_preset or "default"
@@ -602,27 +179,17 @@ def settings_payload(
providers = [] providers = []
for spec in PROVIDERS: for spec in PROVIDERS:
provider_config = getattr(config.providers, spec.name, None) provider_config = getattr(config.providers, spec.name, None)
if provider_config is None: if provider_config is None or spec.is_oauth:
continue continue
oauth_status = _oauth_provider_status(spec) if spec.is_oauth else None
row = { row = {
"name": spec.name, "name": spec.name,
"label": spec.label, "label": spec.label,
"configured": ( "configured": _provider_configured_for_settings(spec, provider_config),
bool(oauth_status["configured"])
if oauth_status is not None
else _provider_configured_for_settings(spec, provider_config)
),
"auth_type": "oauth" if spec.is_oauth else "api_key",
"api_key_required": _provider_requires_api_key(spec), "api_key_required": _provider_requires_api_key(spec),
"api_key_hint": _mask_secret_hint(provider_config.api_key), "api_key_hint": _mask_secret_hint(provider_config.api_key),
"api_base": provider_config.api_base, "api_base": provider_config.api_base,
"default_api_base": spec.default_api_base or None, "default_api_base": spec.default_api_base or None,
} }
if oauth_status is not None:
row["oauth_account"] = oauth_status["account"]
row["oauth_expires_at"] = oauth_status["expires_at"]
row["oauth_login_supported"] = oauth_status["login_supported"]
if spec.name == "openai": if spec.name == "openai":
row["api_type"] = provider_config.api_type row["api_type"] = provider_config.api_type
providers.append(row) providers.append(row)
@@ -674,11 +241,7 @@ def settings_payload(
) )
exec_config = config.tools.exec exec_config = config.tools.exec
sandbox_status = workspace_sandbox_status( return {
restrict_to_workspace=config.tools.restrict_to_workspace,
workspace=config.workspace_path,
)
payload = {
"agent": { "agent": {
"model": effective_preset.model, "model": effective_preset.model,
"provider": selected_provider, "provider": selected_provider,
@@ -744,16 +307,12 @@ def settings_payload(
"max_batch_size": defaults.dream.max_batch_size, "max_batch_size": defaults.dream.max_batch_size,
"max_iterations": defaults.dream.max_iterations, "max_iterations": defaults.dream.max_iterations,
"annotate_line_ages": defaults.dream.annotate_line_ages, "annotate_line_ages": defaults.dream.annotate_line_ages,
"dream_edit_user_skills": defaults.dream.dream_edit_user_skills,
}, },
"unified_session": defaults.unified_session, "unified_session": defaults.unified_session,
}, },
"advanced": { "advanced": {
"restrict_to_workspace": config.tools.restrict_to_workspace, "restrict_to_workspace": config.tools.restrict_to_workspace,
"workspace_sandbox": sandbox_status.as_dict(),
"webui_allow_local_service_access": config.tools.webui_allow_local_service_access,
"allow_local_preview_access": config.tools.webui_allow_local_service_access,
"webui_default_access_mode": read_webui_default_access_mode(),
"private_service_protection_enabled": True,
"ssrf_whitelist_count": len(config.tools.ssrf_whitelist), "ssrf_whitelist_count": len(config.tools.ssrf_whitelist),
"mcp_server_count": len(config.tools.mcp_servers), "mcp_server_count": len(config.tools.mcp_servers),
"exec_enabled": exec_config.enable, "exec_enabled": exec_config.enable,
@@ -762,13 +321,6 @@ def settings_payload(
}, },
"requires_restart": requires_restart, "requires_restart": requires_restart,
} }
return decorate_settings_payload(
payload,
surface=surface,
runtime_capability_overrides=runtime_capability_overrides,
restart_required_sections=restart_required_sections,
apply_state=apply_state,
)
def update_agent_settings(query: QueryParams) -> dict[str, Any]: def update_agent_settings(query: QueryParams) -> dict[str, Any]:
@@ -805,16 +357,6 @@ def update_agent_settings(query: QueryParams) -> dict[str, Any]:
defaults.provider = provider defaults.provider = provider
changed = True changed = True
context_window_tokens = _parse_context_window_tokens(
_query_first_alias(query, "context_window_tokens", "contextWindowTokens")
)
if (
context_window_tokens is not None
and defaults.context_window_tokens != context_window_tokens
):
defaults.context_window_tokens = context_window_tokens
changed = True
timezone = _query_first(query, "timezone") timezone = _query_first(query, "timezone")
if timezone is not None: if timezone is not None:
timezone = timezone.strip() timezone = timezone.strip()
@@ -903,64 +445,6 @@ def create_model_configuration(query: QueryParams) -> dict[str, Any]:
return settings_payload() return settings_payload()
def update_model_configuration(query: QueryParams) -> dict[str, Any]:
name = (_query_first(query, "name") or "").strip()
if not name or name == "default":
raise WebUISettingsError("model configuration is required")
config = load_config()
preset = config.model_presets.get(name)
if preset is None:
raise WebUISettingsError("unknown model configuration")
changed = False
label = _query_first_alias(query, "label", "displayName")
if label is not None:
label = label.strip()
if not label:
raise WebUISettingsError("label is required")
if preset.label != label:
preset.label = label
changed = True
model = _query_first(query, "model")
if model is not None:
model = model.strip()
if not model:
raise WebUISettingsError("model is required")
if preset.model != model:
preset.model = model
changed = True
provider = _query_first(query, "provider")
if provider is not None:
provider = provider.strip()
if not provider:
raise WebUISettingsError("provider is required")
_validate_configured_provider(config, provider)
if preset.provider != provider:
preset.provider = provider
changed = True
context_window_tokens = _parse_context_window_tokens(
_query_first_alias(query, "context_window_tokens", "contextWindowTokens")
)
if (
context_window_tokens is not None
and preset.context_window_tokens != context_window_tokens
):
preset.context_window_tokens = context_window_tokens
changed = True
if config.agents.defaults.model_preset != name:
config.agents.defaults.model_preset = name
changed = True
if changed:
save_config(config)
return settings_payload()
def update_provider_settings(query: QueryParams) -> dict[str, Any]: def update_provider_settings(query: QueryParams) -> dict[str, Any]:
provider_name = (_query_first(query, "provider") or "").strip() provider_name = (_query_first(query, "provider") or "").strip()
if not provider_name: if not provider_name:
@@ -1012,114 +496,6 @@ def update_provider_settings(query: QueryParams) -> dict[str, Any]:
return settings_payload(requires_restart=restart_required) return settings_payload(requires_restart=restart_required)
def login_oauth_provider(query: QueryParams) -> dict[str, Any]:
provider_name = (_query_first(query, "provider") or "").strip()
if not provider_name:
raise WebUISettingsError("provider is required")
spec = find_by_name(provider_name)
if spec is None or not spec.is_oauth:
raise WebUISettingsError("unknown OAuth provider")
if spec.name == "openai_codex":
try:
from oauth_cli_kit import get_token, login_oauth_interactive
except ImportError:
raise WebUISettingsError("oauth_cli_kit is not installed", status=500) from None
token = None
with suppress(Exception):
token = get_token()
if not (token and token.access):
messages: list[str] = []
token = login_oauth_interactive(
print_fn=lambda message: messages.append(str(message)),
prompt_fn=lambda _prompt: "",
)
if not (token and token.access):
raise WebUISettingsError("OAuth login failed", status=401)
return settings_payload()
if spec.name == "github_copilot":
try:
from nanobot.providers.github_copilot_provider import (
get_github_copilot_login_status,
login_github_copilot,
)
except ImportError:
raise WebUISettingsError("GitHub Copilot OAuth support is unavailable", status=500) from None
token = get_github_copilot_login_status()
if not token:
token = login_github_copilot(print_fn=lambda _message: None)
if not (token and token.access):
raise WebUISettingsError("OAuth login failed", status=401)
return settings_payload()
raise WebUISettingsError("OAuth login is not supported for this provider")
def logout_oauth_provider(query: QueryParams) -> dict[str, Any]:
provider_name = (_query_first(query, "provider") or "").strip()
if not provider_name:
raise WebUISettingsError("provider is required")
spec = find_by_name(provider_name)
if spec is None or not spec.is_oauth:
raise WebUISettingsError("unknown OAuth provider")
if spec.name == "openai_codex":
try:
from oauth_cli_kit.providers import OPENAI_CODEX_PROVIDER
from oauth_cli_kit.storage import FileTokenStorage
except ImportError:
raise WebUISettingsError("oauth_cli_kit is not installed", status=500) from None
token_path = FileTokenStorage(token_filename=OPENAI_CODEX_PROVIDER.token_filename).get_token_path()
elif spec.name == "github_copilot":
try:
from nanobot.providers.github_copilot_provider import get_storage
except ImportError:
raise WebUISettingsError("GitHub Copilot OAuth support is unavailable", status=500) from None
token_path = get_storage().get_token_path()
else:
raise WebUISettingsError("OAuth logout is not supported for this provider")
for path in (token_path, token_path.with_suffix(".lock")):
with suppress(FileNotFoundError):
path.unlink()
return settings_payload()
def update_network_safety_settings(query: QueryParams) -> dict[str, Any]:
raw_allow = (
_query_first_alias(query, "webui_allow_local_service_access", "webuiAllowLocalServiceAccess")
or _query_first_alias(query, "allow_local_preview_access", "allowLocalPreviewAccess")
)
raw_default_access_mode = _query_first_alias(query, "webui_default_access_mode", "webuiDefaultAccessMode")
if raw_allow is None and raw_default_access_mode is None:
raise WebUISettingsError("webui_allow_local_service_access or webui_default_access_mode is required")
config = load_config()
changed = False
if raw_allow is not None:
webui_allow_local_service_access = _parse_bool(raw_allow, "webui_allow_local_service_access")
if config.tools.webui_allow_local_service_access != webui_allow_local_service_access:
config.tools.webui_allow_local_service_access = webui_allow_local_service_access
changed = True
if changed:
save_config(config)
if raw_default_access_mode is not None:
default_access_mode = raw_default_access_mode.strip().lower()
if default_access_mode == "restricted":
default_access_mode = "default"
if default_access_mode not in {"default", "full"}:
raise WebUISettingsError("webui_default_access_mode must be default or full")
try:
write_webui_default_access_mode(default_access_mode)
except ValueError as exc:
raise WebUISettingsError(str(exc)) from exc
return settings_payload(requires_restart=changed)
def update_web_search_settings(query: QueryParams) -> dict[str, Any]: def update_web_search_settings(query: QueryParams) -> dict[str, Any]:
provider_name = (_query_first(query, "provider") or "").strip().lower() provider_name = (_query_first(query, "provider") or "").strip().lower()
provider_option = _WEB_SEARCH_PROVIDER_BY_NAME.get(provider_name) provider_option = _WEB_SEARCH_PROVIDER_BY_NAME.get(provider_name)
-329
View File
@@ -1,329 +0,0 @@
"""HTTP route adapter for WebUI Settings APIs.
Keep WebUI Settings route handlers here, not in ``channels/websocket.py``.
The websocket channel owns transport concerns; this module owns WebUI Settings
request mapping and response shaping.
"""
from __future__ import annotations
import asyncio
import json
from collections.abc import Callable
from typing import Any
from websockets.http11 import Request as WsRequest
from websockets.http11 import Response
from nanobot.agent.tools.mcp import request_mcp_reload
from nanobot.bus.queue import MessageBus
from nanobot.webui.cli_apps_api import cli_apps_action, cli_apps_payload
from nanobot.webui.mcp_presets_api import mcp_presets_settings_action
from nanobot.webui.settings_api import (
WebUISettingsError,
create_model_configuration,
decorate_settings_payload,
login_oauth_provider,
logout_oauth_provider,
provider_models_payload,
settings_payload,
update_agent_settings,
update_image_generation_settings,
update_model_configuration,
update_network_safety_settings,
update_provider_settings,
update_web_search_settings,
)
QueryParams = dict[str, list[str]]
_MCP_VALUES_HEADER = "X-Nanobot-MCP-Values"
_MCP_VALUES_HEADER_MAX_BYTES = 64 * 1024
_MCP_PRESET_ACTIONS_BY_PATH = {
"/api/settings/mcp-presets/enable": "enable",
"/api/settings/mcp-presets/remove": "remove",
"/api/settings/mcp-presets/test": "test",
"/api/settings/mcp-presets/custom": "custom",
"/api/settings/mcp-presets/import": "import",
"/api/settings/mcp-presets/import-cursor": "import-cursor",
"/api/settings/mcp-presets/tools": "tools",
}
class WebUISettingsRouter:
"""Route WebUI Settings HTTP requests behind a transport-neutral boundary."""
def __init__(
self,
*,
bus: MessageBus,
logger: Any,
check_api_token: Callable[[WsRequest], bool],
parse_query: Callable[[str], QueryParams],
json_response: Callable[[dict[str, Any]], Response],
error_response: Callable[[int, str | None], Response],
runtime_surface: str,
runtime_capabilities: dict[str, Any],
) -> None:
self.bus = bus
self.logger = logger
self._check_api_token = check_api_token
self._parse_query = parse_query
self._json_response = json_response
self._error_response = error_response
self._runtime_surface = runtime_surface
self._runtime_capabilities = runtime_capabilities
self._restart_sections: set[str] = set()
async def dispatch(self, request: WsRequest, path: str) -> Response | None:
if path == "/api/settings":
return self._handle_settings(request)
if path == "/api/settings/update":
return self._handle_settings_update(request)
if path == "/api/settings/model-configurations/create":
return self._handle_settings_model_configuration_create(request)
if path == "/api/settings/model-configurations/update":
return self._handle_settings_model_configuration_update(request)
if path == "/api/settings/provider/update":
return self._handle_settings_provider_update(request)
if path == "/api/settings/provider-models":
return await self._handle_settings_provider_models(request)
if path == "/api/settings/provider/oauth-login":
return await self._handle_settings_provider_oauth(request, "login")
if path == "/api/settings/provider/oauth-logout":
return await self._handle_settings_provider_oauth(request, "logout")
if path == "/api/settings/web-search/update":
return self._handle_settings_web_search_update(request)
if path == "/api/settings/image-generation/update":
return self._handle_settings_image_generation_update(request)
if path == "/api/settings/network-safety/update":
return self._handle_settings_network_safety_update(request)
if path == "/api/settings/cli-apps":
return self._handle_settings_cli_apps(request)
if path == "/api/settings/cli-apps/install":
return await self._handle_settings_cli_apps_action(request, "install")
if path == "/api/settings/cli-apps/update":
return await self._handle_settings_cli_apps_action(request, "update")
if path == "/api/settings/cli-apps/uninstall":
return await self._handle_settings_cli_apps_action(request, "uninstall")
if path == "/api/settings/cli-apps/test":
return await self._handle_settings_cli_apps_action(request, "test")
if path == "/api/settings/mcp-presets":
return await self._handle_settings_mcp_presets(request)
mcp_action = _MCP_PRESET_ACTIONS_BY_PATH.get(path)
if mcp_action is not None:
return await self._handle_settings_mcp_presets(request, mcp_action)
return None
def _query(self, request: WsRequest) -> QueryParams:
return self._parse_query(request.path)
def _authorized(self, request: WsRequest) -> bool:
return self._check_api_token(request)
def _unauthorized(self) -> Response:
return self._error_response(401, "Unauthorized")
def _with_restart_state(
self,
payload: dict[str, Any],
*,
section: str | None = None,
) -> dict[str, Any]:
"""Keep restart-required state alive for this gateway process."""
if section and payload.get("requires_restart"):
self._restart_sections.add(section)
sections = sorted(self._restart_sections)
payload = dict(payload)
if sections:
payload["requires_restart"] = True
return decorate_settings_payload(
payload,
surface=self._runtime_surface,
runtime_capability_overrides=self._runtime_capabilities,
restart_required_sections=sections,
)
def _parse_mcp_settings_query(self, request: WsRequest) -> QueryParams:
query = self._query(request)
raw = request.headers.get(_MCP_VALUES_HEADER)
if not raw:
return query
if len(raw.encode("utf-8")) > _MCP_VALUES_HEADER_MAX_BYTES:
raise WebUISettingsError("MCP settings payload is too large")
try:
payload = json.loads(raw)
except json.JSONDecodeError as exc:
raise WebUISettingsError("invalid MCP settings payload") from exc
if not isinstance(payload, dict):
raise WebUISettingsError("MCP settings payload must be a JSON object")
merged = {key: list(values) for key, values in query.items()}
for key, value in payload.items():
if not isinstance(key, str) or not key:
raise WebUISettingsError("MCP settings payload contains an invalid key")
if value is None:
continue
if isinstance(value, str):
text = value.strip()
else:
text = json.dumps(value, ensure_ascii=False, separators=(",", ":"))
if text:
merged[key] = [text]
return merged
def _handle_settings(self, request: WsRequest) -> Response:
if not self._authorized(request):
return self._unauthorized()
return self._json_response(
self._with_restart_state(
settings_payload(
surface=self._runtime_surface,
runtime_capability_overrides=self._runtime_capabilities,
)
)
)
def _handle_settings_update(self, request: WsRequest) -> Response:
if not self._authorized(request):
return self._unauthorized()
try:
payload = update_agent_settings(self._query(request))
except WebUISettingsError as e:
return self._error_response(e.status, e.message)
return self._json_response(self._with_restart_state(payload, section="runtime"))
def _handle_settings_model_configuration_create(self, request: WsRequest) -> Response:
if not self._authorized(request):
return self._unauthorized()
try:
payload = create_model_configuration(self._query(request))
except WebUISettingsError as e:
return self._error_response(e.status, e.message)
return self._json_response(self._with_restart_state(payload))
def _handle_settings_model_configuration_update(self, request: WsRequest) -> Response:
if not self._authorized(request):
return self._unauthorized()
try:
payload = update_model_configuration(self._query(request))
except WebUISettingsError as e:
return self._error_response(e.status, e.message)
return self._json_response(self._with_restart_state(payload))
def _handle_settings_provider_update(self, request: WsRequest) -> Response:
if not self._authorized(request):
return self._unauthorized()
try:
payload = update_provider_settings(self._query(request))
except WebUISettingsError as e:
return self._error_response(e.status, e.message)
return self._json_response(self._with_restart_state(payload, section="image"))
async def _handle_settings_provider_models(self, request: WsRequest) -> Response:
if not self._authorized(request):
return self._unauthorized()
try:
payload = await asyncio.to_thread(provider_models_payload, self._query(request))
except WebUISettingsError as e:
return self._error_response(e.status, e.message)
except Exception:
self.logger.exception("failed to load provider model list")
return self._error_response(500, "failed to load provider model list")
return self._json_response(payload)
async def _handle_settings_provider_oauth(
self,
request: WsRequest,
action: str,
) -> Response:
if not self._authorized(request):
return self._unauthorized()
query = self._query(request)
try:
if action == "login":
payload = await asyncio.to_thread(login_oauth_provider, query)
else:
payload = await asyncio.to_thread(logout_oauth_provider, query)
except WebUISettingsError as e:
return self._error_response(e.status, e.message)
return self._json_response(self._with_restart_state(payload))
def _handle_settings_web_search_update(self, request: WsRequest) -> Response:
if not self._authorized(request):
return self._unauthorized()
try:
payload = update_web_search_settings(self._query(request))
except WebUISettingsError as e:
return self._error_response(e.status, e.message)
return self._json_response(self._with_restart_state(payload, section="browser"))
def _handle_settings_image_generation_update(self, request: WsRequest) -> Response:
if not self._authorized(request):
return self._unauthorized()
try:
payload = update_image_generation_settings(self._query(request))
except WebUISettingsError as e:
return self._error_response(e.status, e.message)
return self._json_response(self._with_restart_state(payload, section="image"))
def _handle_settings_network_safety_update(self, request: WsRequest) -> Response:
if not self._authorized(request):
return self._unauthorized()
try:
payload = update_network_safety_settings(self._query(request))
except WebUISettingsError as e:
return self._error_response(e.status, e.message)
return self._json_response(self._with_restart_state(payload, section="runtime"))
def _handle_settings_cli_apps(self, request: WsRequest) -> Response:
if not self._authorized(request):
return self._unauthorized()
try:
payload = cli_apps_payload()
except Exception:
self.logger.exception("failed to load CLI Apps payload")
return self._error_response(500, "failed to load CLI Apps")
return self._json_response(payload)
async def _handle_settings_cli_apps_action(
self,
request: WsRequest,
action: str,
) -> Response:
if not self._authorized(request):
return self._unauthorized()
try:
payload = await asyncio.to_thread(cli_apps_action, action, self._query(request))
except WebUISettingsError as e:
return self._error_response(e.status, e.message)
except Exception as e:
status = getattr(e, "status", 500)
message = getattr(e, "message", str(e))
if status >= 500:
self.logger.exception("CLI Apps action '{}' failed", action)
return self._error_response(status, message)
return self._json_response(payload)
async def _handle_settings_mcp_presets(
self,
request: WsRequest,
action: str | None = None,
) -> Response:
if not self._authorized(request):
return self._unauthorized()
try:
payload = await mcp_presets_settings_action(
action,
self._parse_mcp_settings_query(request),
reload_mcp=lambda: request_mcp_reload(self.bus),
)
except Exception as e:
status = getattr(e, "status", 500)
message = getattr(e, "message", str(e))
if status >= 500:
self.logger.exception("MCP preset action '{}' failed", action or "list")
return self._error_response(status, message)
if action is None:
return self._json_response(payload)
return self._json_response(self._with_restart_state(payload, section="runtime"))
+1 -4
View File
@@ -38,7 +38,6 @@ def default_webui_sidebar_state() -> dict[str, Any]:
"pinned_keys": [], "pinned_keys": [],
"archived_keys": [], "archived_keys": [],
"title_overrides": {}, "title_overrides": {},
"project_name_overrides": {},
"tags_by_key": {}, "tags_by_key": {},
"collapsed_groups": {}, "collapsed_groups": {},
"view": { "view": {
@@ -137,9 +136,6 @@ def normalize_webui_sidebar_state(raw: Any) -> dict[str, Any]:
state["pinned_keys"] = _clean_string_list(raw.get("pinned_keys")) state["pinned_keys"] = _clean_string_list(raw.get("pinned_keys"))
state["archived_keys"] = _clean_string_list(raw.get("archived_keys")) state["archived_keys"] = _clean_string_list(raw.get("archived_keys"))
state["title_overrides"] = _clean_title_overrides(raw.get("title_overrides")) state["title_overrides"] = _clean_title_overrides(raw.get("title_overrides"))
state["project_name_overrides"] = _clean_title_overrides(
raw.get("project_name_overrides")
)
state["tags_by_key"] = _clean_tags_by_key(raw.get("tags_by_key")) state["tags_by_key"] = _clean_tags_by_key(raw.get("tags_by_key"))
state["collapsed_groups"] = _clean_bool_map(raw.get("collapsed_groups")) state["collapsed_groups"] = _clean_bool_map(raw.get("collapsed_groups"))
state["view"] = _clean_view(raw.get("view")) state["view"] = _clean_view(raw.get("view"))
@@ -194,3 +190,4 @@ def write_webui_sidebar_state(raw: dict[str, Any]) -> dict[str, Any]:
finally: finally:
os.close(dir_fd) os.close(dir_fd)
return state return state
+23 -187
View File
@@ -27,18 +27,6 @@ _INLINE_MARKDOWN_IMAGE_EXTS: frozenset[str] = frozenset({
".jpeg", ".jpeg",
".webp", ".webp",
".gif", ".gif",
".svg",
})
_INLINE_MARKDOWN_VIDEO_EXTS: frozenset[str] = frozenset({
".mp4",
".mov",
".webm",
})
_INLINE_MARKDOWN_MEDIA_EXTS = _INLINE_MARKDOWN_IMAGE_EXTS | _INLINE_MARKDOWN_VIDEO_EXTS
_FILE_EDIT_TOOL_NAMES: frozenset[str] = frozenset({
"write_file",
"edit_file",
"apply_patch",
}) })
@@ -48,7 +36,7 @@ def rewrite_local_markdown_images(
workspace_path: Path, workspace_path: Path,
sign_path: Callable[[Path], Mapping[str, Any] | None], sign_path: Callable[[Path], Mapping[str, Any] | None],
) -> str: ) -> str:
"""Rewrite markdown media paths inside the workspace to signed WebUI media URLs.""" """Rewrite markdown image paths inside the workspace to signed WebUI media URLs."""
if "![" not in text: if "![" not in text:
return text return text
@@ -62,7 +50,7 @@ def rewrite_local_markdown_images(
if parsed.scheme or parsed.netloc or parsed.query or parsed.fragment: if parsed.scheme or parsed.netloc or parsed.query or parsed.fragment:
return None return None
path_text = unquote(url) path_text = unquote(url)
if Path(path_text).suffix.lower() not in _INLINE_MARKDOWN_MEDIA_EXTS: if Path(path_text).suffix.lower() not in _INLINE_MARKDOWN_IMAGE_EXTS:
return None return None
candidate = Path(path_text).expanduser() candidate = Path(path_text).expanduser()
if not candidate.is_absolute(): if not candidate.is_absolute():
@@ -87,15 +75,6 @@ def rewrite_local_markdown_images(
return _MARKDOWN_LOCAL_IMAGE_RE.sub(replace, text) return _MARKDOWN_LOCAL_IMAGE_RE.sub(replace, text)
def _media_kind_from_name(name: str) -> str:
ext = Path(name).suffix.lower()
if ext in _INLINE_MARKDOWN_IMAGE_EXTS:
return "image"
if ext in _INLINE_MARKDOWN_VIDEO_EXTS:
return "video"
return "file"
def webui_transcript_path(session_key: str) -> Path: def webui_transcript_path(session_key: str) -> Path:
stem = SessionManager.safe_key(session_key) stem = SessionManager.safe_key(session_key)
return get_webui_dir() / f"{stem}.jsonl" return get_webui_dir() / f"{stem}.jsonl"
@@ -221,19 +200,6 @@ def _tool_event_key(event: dict[str, Any]) -> str:
return _format_tool_call_trace(event) or json.dumps(event, sort_keys=True, ensure_ascii=False) return _format_tool_call_trace(event) or json.dumps(event, sort_keys=True, ensure_ascii=False)
def _tool_event_file_edit_key(event: dict[str, Any]) -> str | None:
call_id = event.get("call_id")
if not isinstance(call_id, str) or not call_id:
return None
name = event.get("name")
if not isinstance(name, str) or not name:
fn = event.get("function")
name = fn.get("name") if isinstance(fn, dict) else ""
if not isinstance(name, str) or name not in _FILE_EDIT_TOOL_NAMES:
return None
return f"{call_id}|{name}"
def _merge_tool_events(previous: Any, incoming: list[dict[str, Any]]) -> list[dict[str, Any]]: def _merge_tool_events(previous: Any, incoming: list[dict[str, Any]]) -> list[dict[str, Any]]:
if not isinstance(previous, list) or not previous: if not isinstance(previous, list) or not previous:
return incoming return incoming
@@ -256,87 +222,6 @@ def _merge_tool_events(previous: Any, incoming: list[dict[str, Any]]) -> list[di
return merged return merged
def _file_edit_key(edit: dict[str, Any]) -> str:
call_id = str(edit.get("call_id") or "")
tool = str(edit.get("tool") or "")
if call_id:
return f"{call_id}|{tool}"
return f"{tool}|{edit.get('path') or ''}"
def _message_has_file_edit_for_tool_event(
message: dict[str, Any],
event: dict[str, Any],
) -> bool:
key = _tool_event_file_edit_key(event)
if not key:
return False
edits = message.get("fileEdits")
if not isinstance(edits, list):
return False
return any(isinstance(edit, dict) and _file_edit_key(edit) == key for edit in edits)
def _filter_covered_file_edit_tool_events(
messages: list[dict[str, Any]],
events: list[dict[str, Any]],
) -> list[dict[str, Any]]:
if not events:
return events
return [
event
for event in events
if not any(_message_has_file_edit_for_tool_event(message, event) for message in messages)
]
def _strip_covered_file_edit_tool_hints(
message: dict[str, Any],
edits: list[dict[str, Any]],
) -> dict[str, Any]:
incoming_keys = {
_file_edit_key(edit)
for edit in edits
if isinstance(edit, dict)
}
events = message.get("toolEvents")
if not incoming_keys or not isinstance(events, list):
return message
kept_events: list[dict[str, Any]] = []
removed_trace_lines: set[str] = set()
changed = False
for event in events:
if not isinstance(event, dict):
continue
key = _tool_event_file_edit_key(event)
if key and key in incoming_keys:
changed = True
removed_trace_lines.update(tool_trace_lines_from_events([event]))
continue
kept_events.append(event)
if not changed:
return message
raw_traces = message.get("traces")
if isinstance(raw_traces, list):
previous_traces = [trace for trace in raw_traces if isinstance(trace, str)]
else:
content = message.get("content")
previous_traces = [content] if isinstance(content, str) and content else []
next_traces = [trace for trace in previous_traces if trace not in removed_trace_lines]
next_message = {
**message,
"traces": next_traces,
"content": next_traces[-1] if next_traces else "",
}
if kept_events:
next_message["toolEvents"] = kept_events
else:
next_message.pop("toolEvents", None)
return next_message
def _merge_unique_tool_trace_lines( def _merge_unique_tool_trace_lines(
previous_traces: list[str], previous_traces: list[str],
lines: list[str], lines: list[str],
@@ -458,40 +343,6 @@ def replay_transcript_to_ui_messages(
return None return None
return str(last.get("id")) return str(last.get("id"))
def demote_interrupted_assistant(segment: str) -> None:
nonlocal buffer_message_id, buffer_parts
for i in range(len(messages) - 1, -1, -1):
candidate = messages[i]
if candidate.get("role") == "user":
break
content = candidate.get("content")
if (
candidate.get("role") != "assistant"
or candidate.get("kind") == "trace"
or not candidate.get("isStreaming")
or not isinstance(content, str)
or not content.strip()
or candidate.get("media")
):
continue
reasoning_parts = [
part
for part in (candidate.get("reasoning"), content)
if isinstance(part, str) and part.strip()
]
messages[i] = {
**candidate,
"content": "",
"reasoning": "\n\n".join(reasoning_parts),
"reasoningStreaming": False,
"isStreaming": False,
"activitySegmentId": candidate.get("activitySegmentId") or segment,
}
if buffer_message_id == candidate.get("id"):
buffer_message_id = None
buffer_parts = []
return
def close_reasoning(prev: list[dict[str, Any]]) -> None: def close_reasoning(prev: list[dict[str, Any]]) -> None:
for i in range(len(prev) - 1, -1, -1): for i in range(len(prev) - 1, -1, -1):
if prev[i].get("reasoningStreaming"): if prev[i].get("reasoningStreaming"):
@@ -553,6 +404,13 @@ def replay_transcript_to_ui_messages(
active_activity_segment_id = None active_activity_segment_id = None
active_file_edit_segment_id = None active_file_edit_segment_id = None
def _file_edit_key(edit: dict[str, Any]) -> str:
call_id = str(edit.get("call_id") or "")
tool = str(edit.get("tool") or "")
if call_id:
return f"{call_id}|{tool}"
return f"{tool}|{edit.get('path') or ''}"
def find_file_edit_trace_index( def find_file_edit_trace_index(
segment: str | None, segment: str | None,
edits: list[dict[str, Any]], edits: list[dict[str, Any]],
@@ -562,23 +420,16 @@ def replay_transcript_to_ui_messages(
candidate = messages[i] candidate = messages[i]
if candidate.get("role") == "user": if candidate.get("role") == "user":
break break
if candidate.get("kind") != "trace": if candidate.get("kind") != "trace" or not candidate.get("fileEdits"):
continue continue
if segment and candidate.get("activitySegmentId") == segment: if segment and candidate.get("activitySegmentId") == segment:
return i return i
existing_edits = candidate.get("fileEdits") existing_edits = candidate.get("fileEdits")
if isinstance(existing_edits, list): if not isinstance(existing_edits, list):
for existing in existing_edits: continue
if isinstance(existing, dict) and _file_edit_key(existing) in incoming_keys: for existing in existing_edits:
return i if isinstance(existing, dict) and _file_edit_key(existing) in incoming_keys:
existing_tool_events = candidate.get("toolEvents") return i
if isinstance(existing_tool_events, list):
for event in existing_tool_events:
if not isinstance(event, dict):
continue
key = _tool_event_file_edit_key(event)
if key and key in incoming_keys:
return i
return None return None
def upsert_file_edits(edits: list[dict[str, Any]], idx: int) -> None: def upsert_file_edits(edits: list[dict[str, Any]], idx: int) -> None:
@@ -586,16 +437,11 @@ def replay_transcript_to_ui_messages(
if not edits: if not edits:
return return
segment = active_file_edit_segment_id segment = active_file_edit_segment_id
if not segment:
segment = _new_activity_segment(activate=False)
active_file_edit_segment_id = segment
demote_interrupted_assistant(segment)
target_index = find_file_edit_trace_index(segment, edits) target_index = find_file_edit_trace_index(segment, edits)
if target_index is not None: if target_index is not None:
last = messages[target_index] last = messages[target_index]
segment = str(last.get("activitySegmentId") or segment or _new_activity_segment(activate=False)) segment = str(last.get("activitySegmentId") or segment or _new_activity_segment(activate=False))
active_file_edit_segment_id = segment active_file_edit_segment_id = segment
last = _strip_covered_file_edit_tool_hints(last, edits)
else: else:
if not segment: if not segment:
segment = _new_activity_segment(activate=False) segment = _new_activity_segment(activate=False)
@@ -774,21 +620,12 @@ def replay_transcript_to_ui_messages(
continue continue
if kind in ("tool_hint", "progress"): if kind in ("tool_hint", "progress"):
structured_events = _normalize_tool_events(rec.get("tool_events")) structured_events = _normalize_tool_events(rec.get("tool_events"))
visible_structured_events = _filter_covered_file_edit_tool_events(messages, structured_events) structured = tool_trace_lines_from_events(rec.get("tool_events"))
structured = tool_trace_lines_from_events(visible_structured_events)
text = rec.get("text") text = rec.get("text")
if structured: trace_lines = structured if structured else ([text] if isinstance(text, str) and text else [])
trace_lines = structured
elif structured_events:
trace_lines = []
elif isinstance(text, str) and text:
trace_lines = [text]
else:
trace_lines = []
if not trace_lines: if not trace_lines:
continue continue
segment = _ensure_activity_segment() segment = _ensure_activity_segment()
demote_interrupted_assistant(segment)
last = messages[-1] if messages else None last = messages[-1] if messages else None
if ( if (
last last
@@ -799,7 +636,7 @@ def replay_transcript_to_ui_messages(
prev_traces = list(last.get("traces") or [last.get("content")]) prev_traces = list(last.get("traces") or [last.get("content")])
if structured: if structured:
merged_traces, added = _merge_unique_tool_trace_lines(prev_traces, structured) merged_traces, added = _merge_unique_tool_trace_lines(prev_traces, structured)
if not added and not visible_structured_events: if not added and not structured_events:
continue continue
else: else:
merged_traces = prev_traces + trace_lines merged_traces = prev_traces + trace_lines
@@ -807,8 +644,8 @@ def replay_transcript_to_ui_messages(
**last, **last,
"traces": merged_traces, "traces": merged_traces,
"content": merged_traces[-1], "content": merged_traces[-1],
"toolEvents": _merge_tool_events(last.get("toolEvents"), visible_structured_events) "toolEvents": _merge_tool_events(last.get("toolEvents"), structured_events)
if visible_structured_events if structured_events
else last.get("toolEvents"), else last.get("toolEvents"),
"activitySegmentId": last.get("activitySegmentId") or segment, "activitySegmentId": last.get("activitySegmentId") or segment,
} }
@@ -821,7 +658,7 @@ def replay_transcript_to_ui_messages(
"kind": "trace", "kind": "trace",
"content": trace_lines[-1], "content": trace_lines[-1],
"traces": trace_lines, "traces": trace_lines,
**({"toolEvents": visible_structured_events} if visible_structured_events else {}), **({"toolEvents": structured_events} if structured_events else {}),
"activitySegmentId": segment, "activitySegmentId": segment,
"createdAt": _ts_base + idx, "createdAt": _ts_base + idx,
}, },
@@ -837,12 +674,11 @@ def replay_transcript_to_ui_messages(
if isinstance(media_urls, list): if isinstance(media_urls, list):
for m in media_urls: for m in media_urls:
if isinstance(m, dict) and m.get("url"): if isinstance(m, dict) and m.get("url"):
name = str(m.get("name") or "")
media.append( media.append(
{ {
"kind": _media_kind_from_name(name), "kind": "image",
"url": str(m["url"]), "url": str(m["url"]),
"name": name, "name": str(m.get("name") or ""),
}, },
) )
extra: dict[str, Any] = {"content": content_s} extra: dict[str, Any] = {"content": content_s}
-283
View File
@@ -1,283 +0,0 @@
"""Persisted WebUI project workspace state."""
from __future__ import annotations
import json
import os
import time
from pathlib import Path
from typing import Any
from loguru import logger
from nanobot.config.paths import get_webui_dir
from nanobot.security.workspace_access import (
WORKSPACE_SCOPE_METADATA_KEY,
WorkspaceScope,
WorkspaceScopeError,
build_workspace_scope,
default_workspace_scope,
validate_workspace_scope_payload,
)
WEBUI_WORKSPACE_STATE_SCHEMA_VERSION = 1
_MAX_STATE_FILE_BYTES = 128 * 1024
_DEFAULT_ACCESS_MODES = {"default", "full"}
_LEGACY_RESTRICTED_DEFAULT_ACCESS_MODE = "restricted"
_WEBUI_SCOPE_CHANNEL = "websocket"
def webui_workspace_state_path() -> Path:
return get_webui_dir() / "workspace-state.json"
def default_webui_workspace_state() -> dict[str, Any]:
return {
"schema_version": WEBUI_WORKSPACE_STATE_SCHEMA_VERSION,
"default_access_mode": "default",
"updated_at": None,
}
def normalize_webui_workspace_state(raw: Any) -> dict[str, Any]:
if not isinstance(raw, dict):
raw = {}
state = default_webui_workspace_state()
updated_at = raw.get("updated_at")
state["updated_at"] = updated_at if isinstance(updated_at, str) else None
default_access_mode = raw.get("default_access_mode")
if default_access_mode in _DEFAULT_ACCESS_MODES:
state["default_access_mode"] = default_access_mode
return state
def read_webui_workspace_state() -> dict[str, Any]:
path = webui_workspace_state_path()
if not path.is_file():
return default_webui_workspace_state()
try:
if path.stat().st_size > _MAX_STATE_FILE_BYTES:
logger.warning("webui workspace state too large, ignoring: {}", path)
return default_webui_workspace_state()
with open(path, encoding="utf-8") as f:
raw = json.load(f)
except (OSError, json.JSONDecodeError) as e:
logger.warning("read webui workspace state failed {}: {}", path, e)
return default_webui_workspace_state()
return normalize_webui_workspace_state(raw)
def write_webui_workspace_state(raw: dict[str, Any]) -> dict[str, Any]:
state = normalize_webui_workspace_state(raw)
state["updated_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
encoded = json.dumps(
state,
ensure_ascii=False,
indent=2,
sort_keys=True,
).encode("utf-8")
if len(encoded) > _MAX_STATE_FILE_BYTES:
raise ValueError("workspace state is too large")
path = webui_workspace_state_path()
path.parent.mkdir(parents=True, exist_ok=True)
tmp = path.with_suffix(".json.tmp")
with open(tmp, "wb") as f:
f.write(encoded)
f.write(b"\n")
f.flush()
os.fsync(f.fileno())
os.replace(tmp, path)
try:
dir_fd = os.open(path.parent, os.O_RDONLY)
except OSError:
return state
try:
os.fsync(dir_fd)
finally:
os.close(dir_fd)
return state
def read_webui_default_access_mode() -> str:
state = read_webui_workspace_state()
mode = state.get("default_access_mode")
return mode if mode in _DEFAULT_ACCESS_MODES else "default"
def write_webui_default_access_mode(mode: str) -> bool:
if mode == _LEGACY_RESTRICTED_DEFAULT_ACCESS_MODE:
mode = "default"
if mode not in _DEFAULT_ACCESS_MODES:
raise ValueError("default access mode must be default or full")
state = read_webui_workspace_state()
changed = state.get("default_access_mode") != mode
if changed:
state["default_access_mode"] = mode
write_webui_workspace_state(state)
return changed
def default_scope_for_webui(
default_workspace: Path,
default_restrict_to_workspace: bool,
) -> WorkspaceScope:
mode = read_webui_default_access_mode()
if mode == "default":
return default_workspace_scope(
default_workspace,
default_restrict_to_workspace,
source_channel=_WEBUI_SCOPE_CHANNEL,
)
return build_workspace_scope(default_workspace, mode, source_channel=_WEBUI_SCOPE_CHANNEL)
def workspaces_payload(
*,
default_workspace: Path,
default_restrict_to_workspace: bool,
controls_available: bool,
) -> dict[str, Any]:
default_access_mode = read_webui_default_access_mode()
default_scope = (
default_workspace_scope(
default_workspace,
default_restrict_to_workspace,
source_channel=_WEBUI_SCOPE_CHANNEL,
)
if default_access_mode == "default"
else build_workspace_scope(default_workspace, default_access_mode, source_channel=_WEBUI_SCOPE_CHANNEL)
)
return {
"schema_version": WEBUI_WORKSPACE_STATE_SCHEMA_VERSION,
"default_access_mode": default_access_mode,
"default_scope": default_scope.payload(),
"controls": {
"can_change_project": controls_available,
"can_use_full_access": controls_available,
},
}
class WebUIWorkspaceController:
"""Own WebUI project scope persistence and validation."""
def __init__(
self,
*,
session_manager: Any | None,
default_workspace: Path,
default_restrict_to_workspace: bool,
) -> None:
self._sessions = session_manager
self._default_workspace = default_workspace
self._default_restrict_to_workspace = default_restrict_to_workspace
def default_scope(self) -> WorkspaceScope:
return default_scope_for_webui(
self._default_workspace,
self._default_restrict_to_workspace,
)
def scope_for_session_key(self, session_key: str) -> WorkspaceScope:
if self._sessions is None:
return self.default_scope()
data = self._sessions.read_session_file(session_key)
metadata = data.get("metadata", {}) if isinstance(data, dict) else {}
if not isinstance(metadata, dict) or WORKSPACE_SCOPE_METADATA_KEY not in metadata:
return self.default_scope()
try:
return validate_workspace_scope_payload(
metadata.get(WORKSPACE_SCOPE_METADATA_KEY),
default_workspace=self._default_workspace,
default_restrict_to_workspace=self._default_restrict_to_workspace,
source_channel=_WEBUI_SCOPE_CHANNEL,
)
except WorkspaceScopeError:
return self.default_scope()
def payload(self, *, controls_available: bool) -> dict[str, Any]:
return workspaces_payload(
default_workspace=self._default_workspace,
default_restrict_to_workspace=self._default_restrict_to_workspace,
controls_available=controls_available,
)
def scope_from_envelope(
self,
envelope: dict[str, Any],
*,
session_key: str | None,
controls_available: bool,
) -> WorkspaceScope:
raw = envelope.get(WORKSPACE_SCOPE_METADATA_KEY)
if raw is None and session_key:
scope = self.scope_for_session_key(session_key)
elif raw is None:
scope = self.default_scope()
else:
scope = validate_workspace_scope_payload(
raw,
default_workspace=self._default_workspace,
default_restrict_to_workspace=self._default_restrict_to_workspace,
source_channel=_WEBUI_SCOPE_CHANNEL,
)
if not controls_available and scope.metadata() != self.default_scope().metadata():
raise WorkspaceScopeError("workspace controls are localhost-only", status=403)
return scope
def scope_for_new_chat(
self,
envelope: dict[str, Any],
*,
controls_available: bool,
) -> WorkspaceScope:
return self.scope_from_envelope(
envelope,
session_key=None,
controls_available=controls_available,
)
def scope_for_set_request(
self,
envelope: dict[str, Any],
*,
chat_id: str,
chat_running: bool,
controls_available: bool,
) -> WorkspaceScope:
if chat_running:
raise WorkspaceScopeError("chat_running", status=409)
return self.scope_from_envelope(
envelope,
session_key=f"websocket:{chat_id}",
controls_available=controls_available,
)
def scope_for_message(
self,
envelope: dict[str, Any],
*,
chat_id: str,
chat_running: bool,
controls_available: bool,
) -> WorkspaceScope:
scope = self.scope_from_envelope(
envelope,
session_key=f"websocket:{chat_id}",
controls_available=controls_available,
)
if (
WORKSPACE_SCOPE_METADATA_KEY in envelope
and chat_running
and scope.metadata() != self.scope_for_session_key(f"websocket:{chat_id}").metadata()
):
raise WorkspaceScopeError("chat_running", status=409)
return scope
def persist_scope(self, chat_id: str, scope: WorkspaceScope) -> None:
if self._sessions is not None:
session = self._sessions.get_or_create(f"websocket:{chat_id}")
session.metadata["webui"] = True
session.metadata[WORKSPACE_SCOPE_METADATA_KEY] = scope.metadata()
self._sessions.save(session)
+1 -2
View File
@@ -37,7 +37,7 @@ dependencies = [
"rich>=14.0.0,<15.0.0", "rich>=14.0.0,<15.0.0",
"croniter>=6.0.0,<7.0.0", "croniter>=6.0.0,<7.0.0",
"dingtalk-stream>=0.24.0,<1.0.0", "dingtalk-stream>=0.24.0,<1.0.0",
"python-telegram-bot[socks,webhooks]>=22.6,<23.0", "python-telegram-bot[socks]>=22.6,<23.0",
"lark-oapi>=1.5.0,<2.0.0", "lark-oapi>=1.5.0,<2.0.0",
"socksio>=1.0.0,<2.0.0", "socksio>=1.0.0,<2.0.0",
"python-socketio>=5.16.0,<6.0.0", "python-socketio>=5.16.0,<6.0.0",
@@ -82,7 +82,6 @@ msteams = [
matrix = [ matrix = [
"matrix-nio[e2e]>=0.25.2; sys_platform != 'win32'", "matrix-nio[e2e]>=0.25.2; sys_platform != 'win32'",
"aiohttp>=3.9.0,<4.0.0",
"mistune>=3.0.0,<4.0.0", "mistune>=3.0.0,<4.0.0",
"nh3>=0.2.17,<1.0.0", "nh3>=0.2.17,<1.0.0",
] ]
@@ -1,169 +0,0 @@
import asyncio
import base64
from pathlib import Path
from unittest.mock import AsyncMock, MagicMock
import pytest
from nanobot.agent.loop import AgentLoop, TurnContext, TurnState
from nanobot.bus.events import InboundMessage
from nanobot.bus.queue import MessageBus
from nanobot.config.schema import ChannelsConfig
from nanobot.providers.base import LLMResponse
from nanobot.utils.document import reference_non_image_attachments
def _make_loop(tmp_path: Path, channels_config: ChannelsConfig | None = None) -> AgentLoop:
provider = MagicMock()
provider.get_default_model.return_value = "test-model"
provider.chat_with_retry = AsyncMock(return_value=LLMResponse(content="ok"))
return AgentLoop(
bus=MessageBus(),
provider=provider,
workspace=tmp_path,
model="test-model",
channels_config=channels_config,
)
@pytest.mark.asyncio
async def test_state_restore_extracts_documents_by_default(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
loop = _make_loop(tmp_path)
doc_path = tmp_path / "report.txt"
doc_path.write_text("Quarterly revenue is $5M", encoding="utf-8")
calls: list[tuple[str, list[str]]] = []
def fake_extract_documents(content: str, media: list[str]) -> tuple[str, list[str]]:
calls.append((content, media))
return f"{content}\n\n[File: report.txt]\nQuarterly revenue is $5M", []
monkeypatch.setattr("nanobot.agent.loop.extract_documents", fake_extract_documents)
ctx = TurnContext(
msg=InboundMessage(
channel="cli",
sender_id="u",
chat_id="c",
content="summarize",
media=[str(doc_path)],
),
session_key="cli:c",
state=TurnState.RESTORE,
turn_id="turn-1",
)
assert await loop._state_restore(ctx) == "ok"
assert calls == [("summarize", [str(doc_path)])]
assert "Quarterly revenue" in ctx.msg.content
assert ctx.msg.media == []
@pytest.mark.asyncio
async def test_state_restore_references_documents_when_extraction_disabled(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
loop = _make_loop(tmp_path, ChannelsConfig(extract_document_text=False))
doc_path = tmp_path / "report.txt"
doc_path.write_text("Quarterly revenue is $5M", encoding="utf-8")
def fail_extract_documents(content: str, media: list[str]) -> tuple[str, list[str]]:
raise AssertionError("document extraction should be disabled")
monkeypatch.setattr("nanobot.agent.loop.extract_documents", fail_extract_documents)
ctx = TurnContext(
msg=InboundMessage(
channel="cli",
sender_id="u",
chat_id="c",
content="summarize",
media=[str(doc_path)],
),
session_key="cli:c",
state=TurnState.RESTORE,
turn_id="turn-1",
)
assert await loop._state_restore(ctx) == "ok"
assert "Quarterly revenue" not in ctx.msg.content
assert f"[Attachment: {doc_path}]" in ctx.msg.content
assert ctx.msg.media == []
@pytest.mark.asyncio
async def test_pending_followup_references_documents_when_extraction_disabled(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
doc_path = tmp_path / "followup.txt"
doc_path.write_text("Do not inject this file body", encoding="utf-8")
captured_messages: list[list[dict]] = []
call_count = {"n": 0}
async def chat_with_retry(*, messages: list[dict], **kwargs: object) -> LLMResponse:
call_count["n"] += 1
captured_messages.append([dict(message) for message in messages])
return LLMResponse(content=f"answer-{call_count['n']}", tool_calls=[], usage={})
loop = _make_loop(tmp_path, ChannelsConfig(extract_document_text=False))
loop.provider.chat_with_retry = chat_with_retry
loop.tools.get_definitions = MagicMock(return_value=[])
def fail_extract_documents(content: str, media: list[str]) -> tuple[str, list[str]]:
raise AssertionError("document extraction should be disabled")
monkeypatch.setattr("nanobot.agent.loop.extract_documents", fail_extract_documents)
pending_queue: asyncio.Queue[InboundMessage] = asyncio.Queue()
await pending_queue.put(
InboundMessage(
channel="cli",
sender_id="u",
chat_id="c",
content="check this",
media=[str(doc_path)],
)
)
final_content, _, _, _, had_injections = await loop._run_agent_loop(
[{"role": "user", "content": "hello"}],
channel="cli",
chat_id="c",
pending_queue=pending_queue,
)
assert final_content == "answer-2"
assert had_injections is True
injected_user_content = [
message["content"]
for message in captured_messages[-1]
if message.get("role") == "user" and isinstance(message.get("content"), str)
][-1]
assert "check this" in injected_user_content
assert f"[Attachment: {doc_path}]" in injected_user_content
assert "Do not inject this file body" not in injected_user_content
def test_document_extraction_disabled_still_preserves_images(tmp_path: Path) -> None:
image_path = tmp_path / "chart.png"
image_path.write_bytes(
base64.b64decode(
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+yF9kAAAAASUVORK5CYII="
)
)
doc_path = tmp_path / "report.txt"
doc_path.write_text("manual extraction target", encoding="utf-8")
content, media = reference_non_image_attachments(
"review these",
[str(image_path), str(doc_path)],
)
assert media == [str(image_path)]
assert f"[Attachment: {doc_path}]" in content
+397 -216
View File
@@ -1,19 +1,32 @@
"""Tests for the Dream class — two-phase memory consolidation via AgentRunner.""" """Tests for Dream driven through AgentLoop._process_system_message."""
import json import json
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock, patch
import pytest import pytest
from unittest.mock import AsyncMock, MagicMock, patch from nanobot.agent.loop import AgentLoop
from nanobot.agent.memory import Dream, MemoryStore
from nanobot.agent.runner import AgentRunResult from nanobot.agent.runner import AgentRunResult
from nanobot.agent.skills import BUILTIN_SKILLS_DIR from nanobot.agent.skills import BUILTIN_SKILLS_DIR
from nanobot.bus.events import InboundMessage
from nanobot.bus.queue import MessageBus
from nanobot.utils.gitstore import LineAge from nanobot.utils.gitstore import LineAge
def _provider(default_model: str, max_tokens: int = 123) -> MagicMock:
provider = MagicMock()
provider.get_default_model.return_value = default_model
provider.generation = SimpleNamespace(
max_tokens=max_tokens, temperature=0.1, reasoning_effort=None
)
return provider
@pytest.fixture @pytest.fixture
def store(tmp_path): def store(tmp_path):
from nanobot.agent.memory import MemoryStore
s = MemoryStore(tmp_path) s = MemoryStore(tmp_path)
s.write_soul("# Soul\n- Helpful") s.write_soul("# Soul\n- Helpful")
s.write_user("# User\n- Developer") s.write_user("# User\n- Developer")
@@ -23,9 +36,7 @@ def store(tmp_path):
@pytest.fixture @pytest.fixture
def mock_provider(): def mock_provider():
p = MagicMock() return _provider("test-model")
p.chat_with_retry = AsyncMock()
return p
@pytest.fixture @pytest.fixture
@@ -34,10 +45,16 @@ def mock_runner():
@pytest.fixture @pytest.fixture
def dream(store, mock_provider, mock_runner): def loop(tmp_path, mock_provider, mock_runner):
d = Dream(store=store, provider=mock_provider, model="test-model", max_batch_size=5) loop = AgentLoop(
d._runner = mock_runner bus=MessageBus(),
return d provider=mock_provider,
workspace=tmp_path,
model="test-model",
context_window_tokens=1000,
)
loop.dream._runner = mock_runner
return loop
def _make_run_result( def _make_run_result(
@@ -56,254 +73,418 @@ def _make_run_result(
) )
class TestDreamRun: class TestDreamAgentLoopIntegration:
async def test_noop_when_no_unprocessed_history(self, dream, mock_provider, mock_runner, store): async def test_completes_goal_state_after_full_backlog(self, loop, mock_runner, store):
"""Dream should not call LLM when there's nothing to process.""" """Goal should be completed after processing all backlog in internal loop."""
result = await dream.run() for i in range(6):
assert result is False store.append_history(f"event {i}")
mock_provider.chat_with_retry.assert_not_called() mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
session = loop.sessions.get_or_create("system:dream")
goal = session.metadata.get("goal_state")
assert isinstance(goal, dict)
assert goal["status"] == "completed"
assert store.get_last_dream_cursor() == 6
async def test_completes_goal_state_on_finish(self, loop, mock_runner, store):
"""Goal should be marked completed when backlog is fully processed."""
store.append_history("event 1")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
session = loop.sessions.get_or_create("system:dream")
goal = session.metadata.get("goal_state")
assert goal["status"] == "completed"
assert "completed_at" in goal
assert "recap" in goal
async def test_noop_when_no_unprocessed_history(self, loop, mock_runner):
"""Dream should not call runner when there's nothing to process."""
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
result = await loop._process_system_message(msg)
assert result is None
mock_runner.run.assert_not_called() mock_runner.run.assert_not_called()
async def test_calls_runner_for_unprocessed_entries(self, dream, mock_provider, mock_runner, store): async def test_calls_runner_for_unprocessed_entries(self, loop, mock_runner, store):
"""Dream should call AgentRunner when there are unprocessed history entries.""" """Dream should call AgentRunner when there are unprocessed history entries."""
store.append_history("User prefers dark mode") store.append_history("User prefers dark mode")
mock_provider.chat_with_retry.return_value = MagicMock(content="New fact") mock_runner.run = AsyncMock(
mock_runner.run = AsyncMock(return_value=_make_run_result( return_value=_make_run_result(
tool_events=[{"name": "edit_file", "status": "ok", "detail": "memory/MEMORY.md"}], tool_events=[
)) {"name": "edit_file", "status": "ok", "detail": "memory/MEMORY.md"}
result = await dream.run() ],
assert result is True )
)
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
mock_runner.run.assert_called_once() mock_runner.run.assert_called_once()
spec = mock_runner.run.call_args[0][0] spec = mock_runner.run.call_args[0][0]
assert spec.max_iterations == 10 assert spec.max_iterations == 10
assert spec.fail_on_tool_error is False assert spec.fail_on_tool_error is False
async def test_advances_dream_cursor(self, dream, mock_provider, mock_runner, store): async def test_advances_dream_cursor(self, loop, mock_runner, store):
"""Dream should advance the cursor after processing.""" """Dream should advance the cursor after processing."""
store.append_history("event 1") store.append_history("event 1")
store.append_history("event 2") store.append_history("event 2")
mock_provider.chat_with_retry.return_value = MagicMock(content="Nothing new")
mock_runner.run = AsyncMock(return_value=_make_run_result()) mock_runner.run = AsyncMock(return_value=_make_run_result())
await dream.run() msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
assert store.get_last_dream_cursor() == 2 assert store.get_last_dream_cursor() == 2
async def test_compacts_processed_history(self, dream, mock_provider, mock_runner, store): async def test_compacts_processed_history(self, loop, mock_runner, store):
"""Dream should compact history after processing.""" """Dream should compact history after processing."""
store.append_history("event 1") store.append_history("event 1")
store.append_history("event 2") store.append_history("event 2")
store.append_history("event 3") store.append_history("event 3")
mock_provider.chat_with_retry.return_value = MagicMock(content="Nothing new")
mock_runner.run = AsyncMock(return_value=_make_run_result()) mock_runner.run = AsyncMock(return_value=_make_run_result())
await dream.run() msg = InboundMessage(
# After Dream, cursor is advanced and 3, compact keeps last max_history_entries channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
entries = store.read_unprocessed_history(since_cursor=0) entries = store.read_unprocessed_history(since_cursor=0)
assert all(e["cursor"] > 0 for e in entries) assert all(e["cursor"] > 0 for e in entries)
async def test_skill_phase_uses_builtin_skill_creator_path(self, dream, mock_provider, mock_runner, store): async def test_processes_full_backlog_in_one_call(self, loop, mock_runner, store):
"""Dream should point skill creation guidance at the builtin skill-creator template.""" """Backlog larger than max_batch_size should be fully processed in one call."""
for i in range(12):
store.append_history(f"event {i}")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
assert store.get_last_dream_cursor() == 12
assert mock_runner.run.call_count == 3 # 5 + 5 + 2
async def test_single_git_commit_for_multi_batch(self, loop, mock_runner, store):
"""Multi-batch run should collapse into exactly one git commit."""
store.git.init()
store.git.auto_commit("initial")
for i in range(12):
store.append_history(f"event {i}")
mock_runner.run = AsyncMock(
return_value=_make_run_result(
tool_events=[
{"name": "edit_file", "status": "ok", "detail": "memory/MEMORY.md"}
],
)
)
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
commits = store.git.log()
dream_commits = [c for c in commits if c.message.startswith("dream:")]
assert len(dream_commits) == 1
async def test_system_prompt_cached(self, loop, mock_runner, store):
"""Batches within one run should reuse cached system prompt when template mtime unchanged."""
for i in range(6):
store.append_history(f"event {i}")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
# Two batches (5 + 1), both should use the same cached prompt
assert mock_runner.run.call_count == 2
first_prompt = mock_runner.run.call_args_list[0][0][0].initial_messages[0]["content"]
second_prompt = mock_runner.run.call_args_list[1][0][0].initial_messages[0]["content"]
assert second_prompt is first_prompt
async def test_noop_when_empty_backlog(self, loop, mock_runner, store):
"""Empty backlog should not advance cursor or create a commit."""
store.git.init()
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
assert store.get_last_dream_cursor() == 0
commits = store.git.log()
assert len([c for c in commits if c.message.startswith("dream:")]) == 0
class TestDreamPrompt:
async def test_prompt_contains_mece_rules(self, loop, mock_runner, store):
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
system_prompt = spec.initial_messages[0]["content"]
assert "Do NOT guess paths" in system_prompt
assert "SOUL.md" in system_prompt
assert "USER.md" in system_prompt
assert "MEMORY.md" in system_prompt
async def test_skill_phase_uses_builtin_skill_creator_path(self, loop, mock_runner, store):
store.append_history("Repeated workflow one") store.append_history("Repeated workflow one")
store.append_history("Repeated workflow two") store.append_history("Repeated workflow two")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKILL] test-skill: test description")
mock_runner.run = AsyncMock(return_value=_make_run_result()) mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
await dream.run() channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0] spec = mock_runner.run.call_args[0][0]
system_prompt = spec.initial_messages[0]["content"] system_prompt = spec.initial_messages[0]["content"]
expected = str(BUILTIN_SKILLS_DIR / "skill-creator" / "SKILL.md") expected = str(BUILTIN_SKILLS_DIR / "skill-creator" / "SKILL.md")
assert expected in system_prompt assert expected in system_prompt
async def test_skill_write_tool_accepts_workspace_relative_skill_path(self, dream, store): async def test_system_prompt_uses_threshold_from_template_var(self, loop, mock_runner, store):
"""Dream skill creation should allow skills/<name>/SKILL.md relative to workspace root.""" store.append_history("some event")
write_tool = dream._tools.get("write_file") mock_runner.run = AsyncMock(return_value=_make_run_result())
assert write_tool is not None msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
result = await write_tool.execute(
path="skills/test-skill/SKILL.md",
content="---\nname: test-skill\ndescription: Test\n---\n",
) )
await loop._process_system_message(msg)
assert "Successfully wrote" in result spec = mock_runner.run.call_args[0][0]
assert (store.workspace / "skills" / "test-skill" / "SKILL.md").exists() system_msg = spec.initial_messages[0]["content"]
async def test_phase1_prompt_includes_line_age_annotations(self, dream, mock_provider, mock_runner, store):
"""Phase 1 prompt should have per-line age suffixes in MEMORY.md when git is available."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
# Init git so line_ages works
store.git.init()
store.git.auto_commit("initial memory state")
await dream.run()
# The MEMORY.md section should not crash and should contain the memory content
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
assert "## Current MEMORY.md" in user_msg
async def test_phase1_annotates_only_memory_not_soul_or_user(self, dream, mock_provider, mock_runner, store):
"""SOUL.md and USER.md should never have age annotations — they are permanent."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
store.git.init()
store.git.auto_commit("initial state")
await dream.run()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
# The ← suffix should only appear in MEMORY.md section
memory_section = user_msg.split("## Current MEMORY.md")[1].split("## Current SOUL.md")[0]
soul_section = user_msg.split("## Current SOUL.md")[1].split("## Current USER.md")[0]
user_section = user_msg.split("## Current USER.md")[1]
# SOUL and USER should not contain age arrows
assert "\u2190" not in soul_section
assert "\u2190" not in user_section
async def test_phase1_prompt_works_without_git(self, dream, mock_provider, mock_runner, store):
"""Phase 1 should work fine even if git is not initialized (no age annotations)."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
await dream.run()
# Should still succeed — just without age annotations
mock_provider.chat_with_retry.assert_called_once()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
assert "## Current MEMORY.md" in user_msg
async def test_phase1_prompt_carries_age_suffix_for_stale_lines(
self, dream, mock_provider, mock_runner, store,
):
"""End-to-end: ages >14d must appear verbatim in the LLM prompt, ages ≤14d must not."""
# MEMORY.md fixture has 2 non-blank lines ("# Memory" and "- Project X active").
# Inject four ages to cover threshold boundaries: >14 suffix, ==14 no suffix, <14 no suffix.
store.write_memory("# Memory\n- Project X active\n- fresh item\n- edge case line")
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
fake_ages = [
LineAge(age_days=30), # "# Memory" → should get ← 30d
LineAge(age_days=20), # "- Project X..." → should get ← 20d
LineAge(age_days=14), # "- fresh item" → ==14, threshold is strictly >14, no suffix
LineAge(age_days=5), # "- edge case..." → no suffix
]
with patch.object(store.git, "line_ages", return_value=fake_ages):
await dream.run()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split("## Current SOUL.md")[0]
assert "\u2190 30d" in memory_section
assert "\u2190 20d" in memory_section
assert "\u2190 14d" not in memory_section
assert "\u2190 5d" not in memory_section
async def test_phase1_skips_annotation_when_disabled(
self, dream, mock_provider, mock_runner, store,
):
"""`annotate_line_ages=False` must bypass the git lookup entirely and keep MEMORY.md raw."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
dream.annotate_line_ages = False
# line_ages must be bypassed entirely — verify with a spy rather than a
# raising side_effect, because _annotate_with_ages catches Exception
# (which swallows AssertionError) and would hide an accidental call.
with patch.object(store.git, "line_ages") as mock_line_ages:
await dream.run()
mock_line_ages.assert_not_called()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
assert "\u2190" not in user_msg
async def test_phase1_skips_annotation_on_line_ages_length_mismatch(
self, dream, mock_provider, mock_runner, store,
):
"""If ages length != lines length (dirty working tree), skip annotation instead of mis-tagging."""
# MEMORY.md has 2 non-blank lines but we hand back only 1 age → mismatch.
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
with patch.object(store.git, "line_ages", return_value=[LineAge(age_days=999)]):
await dream.run()
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs.get("messages", call_args[1].get("messages"))[1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split("## Current SOUL.md")[0]
# No age arrow at all — we refused to annotate rather than tag the wrong line.
assert "\u2190" not in memory_section
async def test_phase1_prompt_uses_threshold_from_template_var(
self, dream, mock_provider, mock_runner, store,
):
"""System prompt should reference the stale-threshold constant, not a hardcoded 14."""
store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result())
await dream.run()
system_msg = mock_provider.chat_with_retry.call_args.kwargs["messages"][0]["content"]
# The template renders with stale_threshold_days=14 → LLM must see "N>14"
assert "N>14" in system_msg assert "N>14" in system_msg
class TestDreamPromptCaps: class TestDreamPromptCaps:
"""Dream's Phase 1/2 prompt must not be poisoned by a legacy oversized async def test_caps_huge_memory_file(self, loop, mock_runner, store):
history entry or a runaway MEMORY.md. Without caps, a single pre-#3412 store.write_memory("M" * (loop.dream._MEMORY_FILE_MAX_CHARS * 5))
raw_archive dump in history.jsonl would make every subsequent Dream run
exceed the context window and silently advance the cursor past real work.
"""
async def test_phase1_caps_huge_memory_file(
self, dream, mock_provider, mock_runner, store,
):
"""A MEMORY.md much larger than _MEMORY_FILE_MAX_CHARS must be truncated
in the prompt preview (full content is still reachable via read_file)."""
store.write_memory("M" * (dream._MEMORY_FILE_MAX_CHARS * 5))
store.append_history("some event") store.append_history("some event")
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result()) mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split(
"## Current SOUL.md"
)[0]
assert len(memory_section) < loop.dream._MEMORY_FILE_MAX_CHARS + 500
await dream.run() async def test_caps_huge_history_entry(self, loop, mock_runner, store):
user_msg = mock_provider.chat_with_retry.call_args.kwargs["messages"][1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split("## Current SOUL.md")[0]
assert len(memory_section) < dream._MEMORY_FILE_MAX_CHARS + 500
async def test_phase1_caps_huge_history_entry(
self, dream, mock_provider, mock_runner, store,
):
"""A legacy oversized history entry (e.g. pre-#3412 raw_archive dump)
must not explode the Phase 1 prompt each entry is capped in the
preview, even though the JSONL record itself stays full-size."""
# Bypass the append_history cap by writing directly, simulating a
# record that was written by an older nanobot build before any caps.
store.history_file.write_text( store.history_file.write_text(
json.dumps({ json.dumps(
"cursor": 1, {
"timestamp": "2026-04-01 10:00", "cursor": 1,
"content": "H" * (dream._HISTORY_ENTRY_PREVIEW_MAX_CHARS * 8), "timestamp": "2026-04-01 10:00",
}) + "\n", "content": "H" * (loop.dream._HISTORY_ENTRY_PREVIEW_MAX_CHARS * 8),
}
)
+ "\n",
encoding="utf-8", encoding="utf-8",
) )
mock_provider.chat_with_retry.return_value = MagicMock(content="[SKIP]")
mock_runner.run = AsyncMock(return_value=_make_run_result()) mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
history_section = user_msg.split("## Conversation History\n")[1].split(
"\n\n## Current Date"
)[0]
assert len(history_section) < loop.dream._HISTORY_ENTRY_PREVIEW_MAX_CHARS + 500
await dream.run()
user_msg = mock_provider.chat_with_retry.call_args.kwargs["messages"][1]["content"] class TestDreamTools:
history_section = user_msg.split("## Conversation History\n")[1].split("\n\n## Current Date")[0] def test_apply_patch_tool_registered(self, loop):
assert len(history_section) < dream._HISTORY_ENTRY_PREVIEW_MAX_CHARS + 500 tool = loop.dream._tools.get("apply_patch")
assert tool is not None
class TestDreamCaps:
def test_batch_size_default_is_5(self):
from nanobot.config.schema import DreamConfig
assert DreamConfig().max_batch_size == 5
def test_memory_cap_is_16k(self, loop):
assert loop.dream._MEMORY_FILE_MAX_CHARS == 16_000
class TestDreamSkipFiltering:
async def test_skip_entries_removed_from_prompt(self, loop, mock_runner, store):
store.append_history("- [skip] greeting\n- [permanent] User prefers dark mode")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
assert "User prefers dark mode" in user_msg
assert "[skip]" not in user_msg
assert "greeting" not in user_msg
class TestDreamAgeAnnotations:
async def test_prompt_includes_line_age_annotations(self, loop, mock_runner, store):
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
store.git.init()
store.git.auto_commit("initial memory state")
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
assert "## Current MEMORY.md" in user_msg
async def test_annotates_only_memory_not_soul_or_user(self, loop, mock_runner, store):
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
store.git.init()
store.git.auto_commit("initial state")
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
soul_section = user_msg.split("## Current SOUL.md")[1].split(
"## Current USER.md"
)[0]
user_section = user_msg.split("## Current USER.md")[1]
assert "" not in soul_section
assert "" not in user_section
async def test_prompt_works_without_git(self, loop, mock_runner, store):
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
mock_runner.run.assert_called_once()
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
assert "## Current MEMORY.md" in user_msg
async def test_prompt_carries_age_suffix_for_stale_lines(self, loop, mock_runner, store):
store.write_memory(
"# Memory\n- Project X active\n- fresh item\n- edge case line"
)
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
fake_ages = [
LineAge(age_days=30),
LineAge(age_days=20),
LineAge(age_days=14),
LineAge(age_days=5),
]
with patch.object(loop.dream.store.git, "line_ages", return_value=fake_ages):
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split(
"## Current SOUL.md"
)[0]
assert "← 30d" in memory_section
assert "← 20d" in memory_section
assert "← 14d" not in memory_section
assert "← 5d" not in memory_section
async def test_skips_annotation_when_disabled(self, loop, mock_runner, store):
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
loop.dream.annotate_line_ages = False
with patch.object(loop.dream.store.git, "line_ages") as mock_line_ages:
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
mock_line_ages.assert_not_called()
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
assert "" not in user_msg
async def test_skips_annotation_on_line_ages_length_mismatch(self, loop, mock_runner, store):
store.append_history("some event")
mock_runner.run = AsyncMock(return_value=_make_run_result())
with patch.object(
loop.dream.store.git, "line_ages", return_value=[LineAge(age_days=999)]
):
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
spec = mock_runner.run.call_args[0][0]
user_msg = spec.initial_messages[1]["content"]
memory_section = user_msg.split("## Current MEMORY.md")[1].split(
"## Current SOUL.md"
)[0]
assert "" not in memory_section
class TestDreamSessionPersistence:
async def test_writes_session_on_success(self, loop, mock_runner, store):
store.append_history("event one")
store.append_history("event two")
mock_runner.run = AsyncMock(
return_value=_make_run_result(
tool_events=[
{"name": "edit_file", "status": "ok", "detail": "memory/MEMORY.md"}
],
)
)
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
session_path = store.memory_dir / ".dream_session.json"
assert session_path.exists()
data = json.loads(session_path.read_text(encoding="utf-8"))
assert data["batch"]["from_cursor"] == 0
assert data["batch"]["to_cursor"] == 2
assert data["batch"]["count"] == 2
assert data["stop_reason"] == "completed"
assert data["changelog"] == ["edit_file: memory/MEMORY.md"]
assert "timestamp" in data
assert "elapsed_seconds" in data
assert "messages" in data
async def test_no_session_record_on_failure(self, loop, mock_runner, store):
"""Failed batch should not write a session record (cursor stays put for retry)."""
store.append_history("event one")
mock_runner.run = AsyncMock(side_effect=RuntimeError("LLM error"))
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
session_path = store.memory_dir / ".dream_session.json"
assert not session_path.exists()
assert store.get_last_dream_cursor() == 0
async def test_session_contains_full_messages(self, loop, mock_runner, store):
store.append_history("event one")
messages = [
{"role": "system", "content": "you are a memory bot"},
{"role": "user", "content": "history here"},
{"role": "assistant", "content": "I will edit MEMORY.md"},
]
result = _make_run_result()
result.messages = messages
mock_runner.run = AsyncMock(return_value=result)
msg = InboundMessage(
channel="system", sender_id="dream", chat_id="dream", content=""
)
await loop._process_system_message(msg)
session_path = store.memory_dir / ".dream_session.json"
data = json.loads(session_path.read_text(encoding="utf-8"))
assert data["messages"] == messages
assert data["prompt_chars"] > 0
assert data["commit_sha"] is None
-30
View File
@@ -56,38 +56,8 @@ async def test_fallback_on_error() -> None:
assert result is True assert result is True
@pytest.mark.asyncio
async def test_fallback_can_fail_closed() -> None:
class FailingProvider(DummyProvider):
async def chat(self, *args, **kwargs) -> LLMResponse:
raise RuntimeError("provider down")
provider = FailingProvider([])
result = await evaluate_response(
"some response",
"some task",
provider,
"m",
default_notify=False,
)
assert result is False
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_no_tool_call_fallback() -> None: async def test_no_tool_call_fallback() -> None:
provider = DummyProvider([LLMResponse(content="I think you should notify", tool_calls=[])]) provider = DummyProvider([LLMResponse(content="I think you should notify", tool_calls=[])])
result = await evaluate_response("some response", "some task", provider, "m") result = await evaluate_response("some response", "some task", provider, "m")
assert result is True assert result is True
@pytest.mark.asyncio
async def test_no_tool_call_can_fail_closed() -> None:
provider = DummyProvider([LLMResponse(content="I think you should notify", tool_calls=[])])
result = await evaluate_response(
"some response",
"some task",
provider,
"m",
default_notify=False,
)
assert result is False
+336
View File
@@ -0,0 +1,336 @@
import asyncio
import pytest
from nanobot.heartbeat.service import HeartbeatService
from nanobot.providers.base import LLMProvider, LLMResponse, ToolCallRequest
from nanobot.utils.llm_runtime import LLMRuntime
class DummyProvider(LLMProvider):
def __init__(self, responses: list[LLMResponse]):
super().__init__()
self._responses = list(responses)
self.calls = 0
self.models: list[str | None] = []
async def chat(self, *args, **kwargs) -> LLMResponse:
self.calls += 1
self.models.append(kwargs.get("model"))
if self._responses:
return self._responses.pop(0)
return LLMResponse(content="", tool_calls=[])
def get_default_model(self) -> str:
return "test-model"
@pytest.mark.asyncio
async def test_start_is_idempotent(tmp_path) -> None:
provider = DummyProvider([])
service = HeartbeatService(
workspace=tmp_path,
provider=provider,
model="openai/gpt-4o-mini",
interval_s=9999,
enabled=True,
)
await service.start()
first_task = service._task
await service.start()
assert service._task is first_task
service.stop()
await asyncio.sleep(0)
@pytest.mark.asyncio
async def test_decide_returns_skip_when_no_tool_call(tmp_path) -> None:
provider = DummyProvider([LLMResponse(content="no tool call", tool_calls=[])])
service = HeartbeatService(
workspace=tmp_path,
provider=provider,
model="openai/gpt-4o-mini",
)
action, tasks = await service._decide("heartbeat content")
assert action == "skip"
assert tasks == ""
@pytest.mark.asyncio
async def test_trigger_now_executes_when_decision_is_run(tmp_path) -> None:
(tmp_path / "HEARTBEAT.md").write_text("- [ ] do thing", encoding="utf-8")
provider = DummyProvider([
LLMResponse(
content="",
tool_calls=[
ToolCallRequest(
id="hb_1",
name="heartbeat",
arguments={"action": "run", "tasks": "check open tasks"},
)
],
)
])
called_with: list[str] = []
async def _on_execute(tasks: str) -> str:
called_with.append(tasks)
return "done"
service = HeartbeatService(
workspace=tmp_path,
provider=provider,
model="openai/gpt-4o-mini",
on_execute=_on_execute,
)
result = await service.trigger_now()
assert result == "done"
assert called_with == ["check open tasks"]
@pytest.mark.asyncio
async def test_trigger_now_returns_none_when_decision_is_skip(tmp_path) -> None:
(tmp_path / "HEARTBEAT.md").write_text("- [ ] do thing", encoding="utf-8")
provider = DummyProvider([
LLMResponse(
content="",
tool_calls=[
ToolCallRequest(
id="hb_1",
name="heartbeat",
arguments={"action": "skip"},
)
],
)
])
async def _on_execute(tasks: str) -> str:
return tasks
service = HeartbeatService(
workspace=tmp_path,
provider=provider,
model="openai/gpt-4o-mini",
on_execute=_on_execute,
)
assert await service.trigger_now() is None
@pytest.mark.asyncio
async def test_tick_notifies_when_evaluator_says_yes(tmp_path, monkeypatch) -> None:
"""Phase 1 run -> Phase 2 execute -> Phase 3 evaluate=notify -> on_notify called."""
(tmp_path / "HEARTBEAT.md").write_text("- [ ] check deployments", encoding="utf-8")
provider = DummyProvider([
LLMResponse(
content="",
tool_calls=[
ToolCallRequest(
id="hb_1",
name="heartbeat",
arguments={"action": "run", "tasks": "check deployments"},
)
],
),
])
executed: list[str] = []
notified: list[str] = []
async def _on_execute(tasks: str) -> str:
executed.append(tasks)
return "deployment failed on staging"
async def _on_notify(response: str) -> None:
notified.append(response)
service = HeartbeatService(
workspace=tmp_path,
provider=provider,
model="openai/gpt-4o-mini",
on_execute=_on_execute,
on_notify=_on_notify,
)
async def _eval_notify(*a, **kw):
return True
monkeypatch.setattr("nanobot.utils.evaluator.evaluate_response", _eval_notify)
await service._tick()
assert executed == ["check deployments"]
assert notified == ["deployment failed on staging"]
@pytest.mark.asyncio
async def test_tick_suppresses_when_evaluator_says_no(tmp_path, monkeypatch) -> None:
"""Phase 1 run -> Phase 2 execute -> Phase 3 evaluate=silent -> on_notify NOT called."""
(tmp_path / "HEARTBEAT.md").write_text("- [ ] check status", encoding="utf-8")
provider = DummyProvider([
LLMResponse(
content="",
tool_calls=[
ToolCallRequest(
id="hb_1",
name="heartbeat",
arguments={"action": "run", "tasks": "check status"},
)
],
),
])
executed: list[str] = []
notified: list[str] = []
async def _on_execute(tasks: str) -> str:
executed.append(tasks)
return "everything is fine, no issues"
async def _on_notify(response: str) -> None:
notified.append(response)
service = HeartbeatService(
workspace=tmp_path,
provider=provider,
model="openai/gpt-4o-mini",
on_execute=_on_execute,
on_notify=_on_notify,
)
async def _eval_silent(*a, **kw):
return False
monkeypatch.setattr("nanobot.utils.evaluator.evaluate_response", _eval_silent)
await service._tick()
assert executed == ["check status"]
assert notified == []
def test_tick_uses_runtime_provider_and_model(tmp_path, monkeypatch) -> None:
"""Preset changes must apply to heartbeat decision and post-run evaluation."""
(tmp_path / "HEARTBEAT.md").write_text("- [ ] check runtime model", encoding="utf-8")
runtime_provider = DummyProvider([
LLMResponse(
content="",
tool_calls=[
ToolCallRequest(
id="hb_1",
name="heartbeat",
arguments={"action": "run", "tasks": "check runtime model"},
)
],
),
])
runtime_model = "openai/gpt-4.1"
executed: list[str] = []
evaluated: list[tuple[LLMProvider, str]] = []
async def _on_execute(tasks: str) -> str:
executed.append(tasks)
return "runtime model produced a user-facing update"
async def _eval_capture(response, tasks, provider, model):
evaluated.append((provider, model))
return False
service = HeartbeatService(
workspace=tmp_path,
llm_runtime=lambda: LLMRuntime(runtime_provider, runtime_model),
on_execute=_on_execute,
)
monkeypatch.setattr("nanobot.utils.evaluator.evaluate_response", _eval_capture)
asyncio.run(service._tick())
assert runtime_provider.calls == 1
assert runtime_provider.models == [runtime_model]
assert executed == ["check runtime model"]
assert evaluated == [(runtime_provider, runtime_model)]
@pytest.mark.asyncio
async def test_decide_retries_transient_error_then_succeeds(tmp_path, monkeypatch) -> None:
provider = DummyProvider([
LLMResponse(content="429 rate limit", finish_reason="error"),
LLMResponse(
content="",
tool_calls=[
ToolCallRequest(
id="hb_1",
name="heartbeat",
arguments={"action": "run", "tasks": "check open tasks"},
)
],
),
])
delays: list[int] = []
async def _fake_sleep(delay: int) -> None:
delays.append(delay)
monkeypatch.setattr(asyncio, "sleep", _fake_sleep)
service = HeartbeatService(
workspace=tmp_path,
provider=provider,
model="openai/gpt-4o-mini",
)
action, tasks = await service._decide("heartbeat content")
assert action == "run"
assert tasks == "check open tasks"
assert provider.calls == 2
assert delays == [1]
@pytest.mark.asyncio
async def test_decide_prompt_includes_current_time(tmp_path) -> None:
"""Phase 1 user prompt must contain current time so the LLM can judge task urgency."""
captured_messages: list[dict] = []
class CapturingProvider(LLMProvider):
async def chat(self, *, messages=None, **kwargs) -> LLMResponse:
if messages:
captured_messages.extend(messages)
return LLMResponse(
content="",
tool_calls=[
ToolCallRequest(
id="hb_1", name="heartbeat",
arguments={"action": "skip"},
)
],
)
def get_default_model(self) -> str:
return "test-model"
service = HeartbeatService(
workspace=tmp_path,
provider=CapturingProvider(),
model="test-model",
)
await service._decide("- [ ] check servers at 10:00 UTC")
user_msg = captured_messages[1]
assert user_msg["role"] == "user"
assert "Current Time:" in user_msg["content"]
@@ -1,91 +0,0 @@
import asyncio
from unittest.mock import AsyncMock, MagicMock
import pytest
from nanobot.agent.loop import AgentLoop
from nanobot.bus.events import OutboundMessage
from nanobot.bus.queue import MessageBus
from nanobot.providers.base import GenerationSettings, LLMResponse
def _make_loop(tmp_path):
bus = MessageBus()
provider = MagicMock()
provider.get_default_model.return_value = "test-model"
provider.generation = GenerationSettings(max_tokens=0)
provider.estimate_prompt_tokens.return_value = (0, "test-counter")
response = LLMResponse(content="done", tool_calls=[])
provider.chat_with_retry = AsyncMock(return_value=response)
provider.chat_stream_with_retry = AsyncMock(return_value=response)
loop = AgentLoop(
bus=bus,
provider=provider,
workspace=tmp_path,
model="test-model",
)
loop.tools.get_definitions = MagicMock(return_value=[])
return loop
@pytest.mark.asyncio
async def test_process_direct_websocket_clears_run_status(tmp_path) -> None:
loop = _make_loop(tmp_path)
response = await loop.process_direct(
"deliver reminder",
session_key="cron:reminder-1",
channel="websocket",
chat_id="chat-1",
)
assert response is not None
assert response.content == "done"
events = []
while loop.bus.outbound_size:
events.append(await loop.bus.consume_outbound())
statuses = [
event.metadata
for event in events
if event.metadata.get("_goal_status") is True
]
assert [status["goal_status"] for status in statuses] == ["running", "idle"]
assert isinstance(statuses[0].get("started_at"), float)
assert "started_at" not in statuses[1]
@pytest.mark.asyncio
async def test_process_direct_reuses_existing_session_lock(tmp_path) -> None:
loop = _make_loop(tmp_path)
loop._connect_mcp = AsyncMock()
session_key = "api:fixed"
lock = loop._session_locks.setdefault(session_key, asyncio.Lock())
await lock.acquire()
entered = asyncio.Event()
async def _process_message(msg, **_kwargs):
entered.set()
return OutboundMessage(channel=msg.channel, chat_id=msg.chat_id, content=msg.content)
loop._process_message = _process_message
task = asyncio.create_task(loop.process_direct("direct", session_key=session_key))
try:
await asyncio.sleep(0)
assert not entered.is_set()
lock.release()
response = await asyncio.wait_for(task, timeout=1.0)
assert entered.is_set()
assert response is not None
assert response.content == "direct"
finally:
if lock.locked():
lock.release()
if not task.done():
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
+7 -13
View File
@@ -17,7 +17,6 @@ from nanobot.session.webui_turns import (
WEBUI_SESSION_METADATA_KEY, WEBUI_SESSION_METADATA_KEY,
WEBUI_TITLE_METADATA_KEY, WEBUI_TITLE_METADATA_KEY,
WebuiTurnCoordinator, WebuiTurnCoordinator,
clean_generated_title,
maybe_generate_webui_title, maybe_generate_webui_title,
) )
from nanobot.utils.llm_runtime import LLMRuntime from nanobot.utils.llm_runtime import LLMRuntime
@@ -54,11 +53,6 @@ def test_agent_loop_llm_runtime_reflects_current_provider_and_model(tmp_path: Pa
assert runtime.model == "next-model" assert runtime.model == "next-model"
def test_clean_generated_title_strips_reasoning_tags() -> None:
assert clean_generated_title("<think>reasoning</think> WebUI polish") == "WebUI polish"
assert clean_generated_title("Title: <think> The user said hello") == ""
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_generate_webui_title_only_for_marked_webui_sessions(tmp_path: Path) -> None: async def test_generate_webui_title_only_for_marked_webui_sessions(tmp_path: Path) -> None:
loop = _make_full_loop(tmp_path) loop = _make_full_loop(tmp_path)
@@ -608,17 +602,17 @@ async def test_process_message_uses_explicit_session_metadata_for_goal_context(
chat_session = loop.sessions.get_or_create("websocket:chat-with-goal") chat_session = loop.sessions.get_or_create("websocket:chat-with-goal")
chat_session.metadata[GOAL_STATE_KEY] = { chat_session.metadata[GOAL_STATE_KEY] = {
"status": "active", "status": "active",
"objective": "This chat goal must not leak into system.", "objective": "This chat goal must not leak into heartbeat.",
} }
loop.sessions.save(chat_session) loop.sessions.save(chat_session)
system_session = loop.sessions.get_or_create("system") system_session = loop.sessions.get_or_create("heartbeat")
system_session.metadata = {} system_session.metadata = {}
loop.sessions.save(system_session) loop.sessions.save(system_session)
loop.context.build_messages = MagicMock( # type: ignore[method-assign] loop.context.build_messages = MagicMock( # type: ignore[method-assign]
return_value=[ return_value=[
{"role": "system", "content": "system"}, {"role": "system", "content": "system"},
{"role": "user", "content": "runtime + system"}, {"role": "user", "content": "runtime + heartbeat"},
] ]
) )
loop._run_agent_loop = AsyncMock(return_value=( # type: ignore[method-assign] loop._run_agent_loop = AsyncMock(return_value=( # type: ignore[method-assign]
@@ -626,7 +620,7 @@ async def test_process_message_uses_explicit_session_metadata_for_goal_context(
[], [],
[ [
{"role": "system", "content": "system"}, {"role": "system", "content": "system"},
{"role": "user", "content": "runtime + system"}, {"role": "user", "content": "runtime + heartbeat"},
{"role": "assistant", "content": "ok"}, {"role": "assistant", "content": "ok"},
], ],
"stop", "stop",
@@ -636,11 +630,11 @@ async def test_process_message_uses_explicit_session_metadata_for_goal_context(
result = await loop._process_message( result = await loop._process_message(
InboundMessage( InboundMessage(
channel="websocket", channel="websocket",
sender_id="system", sender_id="heartbeat",
chat_id="chat-with-goal", chat_id="chat-with-goal",
content="system work", content="heartbeat work",
), ),
session_key="system", session_key="heartbeat",
) )
assert result is not None assert result is not None
+125
View File
@@ -0,0 +1,125 @@
"""Tests for memory system: Consolidator, token estimation, truncation."""
from unittest.mock import AsyncMock, MagicMock
import pytest
from nanobot.agent.memory import _TIKTOKEN_ENC, Consolidator, MemoryStore, _estimate_tokens
@pytest.fixture
def store(tmp_path):
s = MemoryStore(tmp_path)
s.write_soul("# Soul\n- Helpful")
s.write_user("# User\n- Developer")
s.write_memory("# Memory\n- Project X active")
return s
@pytest.fixture
def mock_provider():
p = MagicMock()
p.chat_with_retry = AsyncMock()
p.generation.max_tokens = 4096
return p
@pytest.fixture
def mock_sessions():
return MagicMock()
@pytest.fixture
def mock_build_messages():
return MagicMock(return_value=[])
@pytest.fixture
def mock_get_tool_definitions():
return MagicMock(return_value=[])
@pytest.fixture
def consolidator(store, mock_provider, mock_sessions, mock_build_messages, mock_get_tool_definitions):
return Consolidator(
store=store,
provider=mock_provider,
model="test-model",
sessions=mock_sessions,
context_window_tokens=128_000,
build_messages=mock_build_messages,
get_tool_definitions=mock_get_tool_definitions,
)
class TestEstimateTokens:
def test_estimate_tokens_returns_positive(self):
assert _estimate_tokens("hello world") > 0
def test_estimate_tokens_english_approximate(self):
# English is roughly 1 token per 4 chars as fallback
text = "a " * 100
if _TIKTOKEN_ENC is not None:
expected = len(_TIKTOKEN_ENC.encode(text))
else:
expected = len(text) // 4
assert _estimate_tokens(text) == expected
class TestTruncateToTokenBudget:
def test_reserve_tokens_reduces_budget(self, consolidator):
long_text = "word " * 200_000
# Without reserve, more text survives
no_reserve = consolidator._truncate_to_token_budget(long_text, reserve_tokens=0)
with_reserve = consolidator._truncate_to_token_budget(long_text, reserve_tokens=500)
assert len(with_reserve) < len(no_reserve)
def test_reserve_tokens_zero_default(self, consolidator):
text = "hello world"
result = consolidator._truncate_to_token_budget(text)
assert result == text
class TestConsolidatorPrompt:
def test_prompt_contains_snip(self):
from nanobot.utils.prompt_templates import render_template
text = render_template("agent/consolidator_archive.md", strip=True)
assert "SNIP" in text
assert "[permanent]" in text
assert "[skip]" in text
class TestConsolidatorArchive:
async def test_archive_injects_dedup_context(self, consolidator, mock_provider, store):
store.write_memory("- User prefers dark mode")
store.write_user("- Developer")
messages = [{"role": "user", "content": "hello", "timestamp": "2026-01-01 10:00"}]
mock_provider.chat_with_retry.return_value = MagicMock(
content="(nothing)", finish_reason="stop"
)
await consolidator.archive(messages)
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs["messages"][1]["content"]
assert "## Current MEMORY.md (for dedup)" in user_msg
assert "User prefers dark mode" in user_msg
assert "## Current USER.md (for dedup)" in user_msg
assert "Developer" in user_msg
async def test_archive_skips_dedup_when_budget_exhausted(self, consolidator, mock_provider, store):
# Shrink token budget so dedup context (always capped at ~6000 chars)
# exceeds the available room.
consolidator.context_window_tokens = 6_000
store.write_memory("word " * 10_000)
messages = [{"role": "user", "content": "hello", "timestamp": "2026-01-01 10:00"}]
mock_provider.chat_with_retry.return_value = MagicMock(
content="(nothing)", finish_reason="stop"
)
await consolidator.archive(messages)
call_args = mock_provider.chat_with_retry.call_args
user_msg = call_args.kwargs["messages"][1]["content"]
# Should not contain dedup context when budget is exhausted
assert "## Current MEMORY.md (for dedup)" not in user_msg
-25
View File
@@ -78,31 +78,6 @@ async def test_llm_error_not_appended_to_session_messages():
assert assistant_msgs[-1]["content"] == _PERSISTED_MODEL_ERROR_PLACEHOLDER assert assistant_msgs[-1]["content"] == _PERSISTED_MODEL_ERROR_PLACEHOLDER
@pytest.mark.asyncio
async def test_llm_arrearage_error_surfaces_clear_message():
"""Arrearage errors yield a clear user-facing message, not a raw dump (#3006)."""
from nanobot.agent.runner import AgentRunSpec, AgentRunner, _ARREARAGE_ERROR_MESSAGE
provider = MagicMock(spec=LLMProvider)
provider.chat_with_retry = AsyncMock(return_value=LLMResponse(
content="HTTP 402 insufficient_quota", finish_reason="error", error_status_code=402,
))
tools = MagicMock()
tools.get_definitions.return_value = []
runner = AgentRunner(provider)
result = await runner.run(AgentRunSpec(
initial_messages=[{"role": "user", "content": "hello"}],
tools=tools,
model="test-model",
max_iterations=5,
max_tool_result_chars=_MAX_TOOL_RESULT_CHARS,
))
assert result.stop_reason == "error"
assert result.final_content == _ARREARAGE_ERROR_MESSAGE
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_runner_tool_error_sets_final_content(): async def test_runner_tool_error_sets_final_content():
from nanobot.agent.runner import AgentRunSpec, AgentRunner from nanobot.agent.runner import AgentRunSpec, AgentRunner
-27
View File
@@ -129,33 +129,6 @@ async def test_runner_respects_max_iterations_even_with_active_goal():
assert result.stop_reason == "max_iterations" assert result.stop_reason == "max_iterations"
@pytest.mark.asyncio
async def test_runner_goal_continue_not_limited_by_injection_cycle_cap():
"""Synthetic goal continuation should be governed by max_iterations."""
from nanobot.agent.runner import _MAX_INJECTION_CYCLES, AgentRunner, AgentRunSpec
provider = MagicMock(spec=LLMProvider)
provider.chat_with_retry = AsyncMock(return_value=LLMResponse(
content="still working", tool_calls=[], usage={},
))
tools = MagicMock()
tools.get_definitions.return_value = []
max_iterations = _MAX_INJECTION_CYCLES + 3
runner = AgentRunner(provider)
result = await runner.run(AgentRunSpec(
initial_messages=[{"role": "user", "content": "do task"}],
tools=tools,
model="test-model",
max_iterations=max_iterations,
max_tool_result_chars=_MAX_TOOL_RESULT_CHARS,
goal_active_predicate=lambda: True,
))
assert result.stop_reason == "max_iterations"
assert provider.chat_with_retry.await_count == max_iterations
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_runner_does_not_force_continue_on_error(): async def test_runner_does_not_force_continue_on_error():
"""Even with active goal, an LLM error should exit with stop_reason="error".""" """Even with active goal, an LLM error should exit with stop_reason="error"."""
-54
View File
@@ -105,60 +105,6 @@ def test_snip_history_drops_orphaned_tool_results_from_trimmed_slice(monkeypatch
assert trimmed[0]["role"] == "system" assert trimmed[0]["role"] == "system"
non_system = [m for m in trimmed if m["role"] != "system"] non_system = [m for m in trimmed if m["role"] != "system"]
assert non_system[0]["role"] == "user", f"Expected user after system, got {non_system[0]['role']}" assert non_system[0]["role"] == "user", f"Expected user after system, got {non_system[0]['role']}"
def test_snip_history_reserves_budget_for_tool_definitions(monkeypatch):
from nanobot.agent.runner import AgentRunSpec, AgentRunner
provider = MagicMock()
tools = MagicMock()
tools.get_definitions.return_value = [{"type": "function", "function": {"name": "large_tool"}}]
runner = AgentRunner(provider)
messages = [
{"role": "system", "content": "system"},
{"role": "user", "content": "old user"},
{"role": "assistant", "content": "old assistant"},
{"role": "user", "content": "recent one"},
{"role": "assistant", "content": "recent answer"},
{"role": "user", "content": "recent two"},
]
spec = AgentRunSpec(
initial_messages=messages,
tools=tools,
model="test-model",
max_iterations=1,
max_tool_result_chars=_MAX_TOOL_RESULT_CHARS,
context_window_tokens=2000,
context_block_limit=500,
)
def _estimate(_provider, _model, estimate_messages, estimate_tools):
if estimate_messages == messages:
return 1000, None
assert estimate_messages == [{"role": "system", "content": "system"}]
assert estimate_tools == tools.get_definitions.return_value
return 350, None
monkeypatch.setattr("nanobot.agent.runner.estimate_prompt_tokens_chain", _estimate)
token_sizes = {
"system": 50,
"old user": 200,
"old assistant": 200,
"recent one": 200,
"recent answer": 200,
"recent two": 200,
}
monkeypatch.setattr(
"nanobot.agent.runner.estimate_message_tokens",
lambda msg: token_sizes.get(str(msg.get("content")), 40),
)
trimmed = runner._snip_history(spec, messages)
contents = [message.get("content") for message in trimmed]
assert contents == ["system", "recent two"]
async def test_backfill_missing_tool_results_inserts_error(): async def test_backfill_missing_tool_results_inserts_error():
"""Orphaned tool_use (no matching tool_result) should get a synthetic error.""" """Orphaned tool_use (no matching tool_result) should get a synthetic error."""
from nanobot.agent.runner import AgentRunner, _BACKFILL_CONTENT from nanobot.agent.runner import AgentRunner, _BACKFILL_CONTENT
-36
View File
@@ -554,42 +554,6 @@ async def test_pending_queue_cleanup_on_dispatch(tmp_path):
assert msg.session_key not in loop._pending_queues assert msg.session_key not in loop._pending_queues
@pytest.mark.asyncio
async def test_waiting_dispatch_does_not_replace_active_pending_queue(tmp_path):
"""A queued dispatch must not steal the active task's injection queue."""
from nanobot.bus.events import InboundMessage
loop = _make_loop(tmp_path)
session_key = "cli:c"
lock = loop._session_locks.setdefault(session_key, asyncio.Lock())
await lock.acquire()
active_pending = asyncio.Queue(maxsize=1)
loop._pending_queues[session_key] = active_pending
waiting_at_lock = asyncio.Event()
original_acquire = asyncio.Lock.acquire
async def _patched_acquire(self, *args, **kwargs):
if self is lock:
waiting_at_lock.set()
return await original_acquire(self, *args, **kwargs)
with patch.object(asyncio.Lock, "acquire", _patched_acquire):
waiting = asyncio.create_task(
loop._dispatch(
InboundMessage(channel="cli", sender_id="u", chat_id="c", content="queued")
)
)
await asyncio.wait_for(waiting_at_lock.wait(), timeout=2.0)
assert loop._pending_queues[session_key] is active_pending
waiting.cancel()
with pytest.raises(asyncio.CancelledError):
await waiting
lock.release()
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_followup_routed_to_pending_queue(tmp_path): async def test_followup_routed_to_pending_queue(tmp_path):
"""Unified-session follow-ups should route into the active pending queue.""" """Unified-session follow-ups should route into the active pending queue."""
+1 -31
View File
@@ -4,10 +4,7 @@ from unittest.mock import MagicMock
from nanobot.agent.loop import AgentLoop from nanobot.agent.loop import AgentLoop
from nanobot.bus.queue import MessageBus from nanobot.bus.queue import MessageBus
from nanobot.config.loader import save_config from nanobot.providers.factory import ProviderSnapshot
from nanobot.config.schema import Config
from nanobot.providers.factory import ProviderSnapshot, load_provider_snapshot
from nanobot.webui.settings_api import update_agent_settings
def _provider(default_model: str, max_tokens: int = 123) -> MagicMock: def _provider(default_model: str, max_tokens: int = 123) -> MagicMock:
@@ -75,30 +72,3 @@ def test_llm_runtime_refreshes_provider_snapshot(tmp_path: Path) -> None:
assert runtime.model == "new-model" assert runtime.model == "new-model"
assert loop.provider is new_provider assert loop.provider is new_provider
assert loop.runner.provider is new_provider assert loop.runner.provider is new_provider
def test_settings_context_window_refreshes_runtime_state(
tmp_path: Path,
monkeypatch,
) -> None:
config_path = tmp_path / "config.json"
config = Config()
config.agents.defaults.workspace = str(tmp_path / "workspace")
config.agents.defaults.model = "openai/gpt-4o"
config.agents.defaults.provider = "openai"
config.agents.defaults.context_window_tokens = 65_536
config.providers.openai.api_key = "sk-test"
save_config(config, config_path)
monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path)
def loader(*, preset_name: str | None = None) -> ProviderSnapshot:
return load_provider_snapshot(config_path, preset_name=preset_name)
loop = AgentLoop.from_config(config, provider_snapshot_loader=loader)
payload = update_agent_settings({"context_window_tokens": ["262144"]})
loop._refresh_provider_snapshot()
assert payload["requires_restart"] is False
assert loop.context_window_tokens == 262_144
assert loop.consolidator.context_window_tokens == 262_144
+92
View File
@@ -292,3 +292,95 @@ def test_from_config_static_preset_loader_does_not_enable_hot_reload(tmp_path) -
loop = AgentLoop.from_config(config) loop = AgentLoop.from_config(config)
assert loop._provider_snapshot_loader is None assert loop._provider_snapshot_loader is None
assert loop._preset_snapshot_loader is not None assert loop._preset_snapshot_loader is not None
class TestDreamModelOverride:
def test_dream_follows_main_when_no_override(self, tmp_path) -> None:
provider = _provider("base-model")
loop = AgentLoop(
bus=MessageBus(),
provider=provider,
workspace=tmp_path,
model="base-model",
context_window_tokens=1000,
)
assert loop.dream.model == "base-model"
assert loop.dream.provider is provider
def test_dream_raw_model_override(self, tmp_path) -> None:
provider = _provider("base-model")
loop = AgentLoop(
bus=MessageBus(),
provider=provider,
workspace=tmp_path,
model="base-model",
context_window_tokens=1000,
dream_model_override="custom-model-v2",
)
assert loop.dream.model == "custom-model-v2"
assert loop.dream.provider is provider
def test_dream_preset_override(self, tmp_path) -> None:
cheap_provider = _provider("openai/gpt-4.1-mini", max_tokens=2048)
preset = ModelPresetConfig(
model="openai/gpt-4.1-mini",
provider="openai",
max_tokens=2048,
context_window_tokens=128_000,
)
loop = AgentLoop(
bus=MessageBus(),
provider=_provider("base-model"),
workspace=tmp_path,
model="base-model",
context_window_tokens=1000,
model_presets={"cheap": preset},
dream_model_override="cheap",
preset_snapshot_loader=lambda _name: ProviderSnapshot(
provider=cheap_provider,
model=preset.model,
context_window_tokens=preset.context_window_tokens,
signature=("cheap", preset.model),
),
)
assert loop.dream.model == "openai/gpt-4.1-mini"
assert loop.dream.provider is cheap_provider
assert loop.dream._runner.provider is cheap_provider
def test_dream_override_survives_main_preset_switch(self, tmp_path) -> None:
base_provider = _provider("base-model")
fast_provider = _provider("openai/gpt-4.1", max_tokens=4096)
cheap_provider = _provider("openai/gpt-4.1-mini", max_tokens=2048)
loop = AgentLoop(
bus=MessageBus(),
provider=base_provider,
workspace=tmp_path,
model="base-model",
context_window_tokens=1000,
model_presets={
"fast": ModelPresetConfig(model="openai/gpt-4.1"),
"cheap": ModelPresetConfig(model="openai/gpt-4.1-mini"),
},
dream_model_override="cheap",
preset_snapshot_loader=lambda name: ProviderSnapshot(
provider=fast_provider if name == "fast" else cheap_provider,
model="openai/gpt-4.1" if name == "fast" else "openai/gpt-4.1-mini",
context_window_tokens=32_768 if name == "fast" else 128_000,
signature=(name, "model"),
),
)
# Initially dream is on cheap
assert loop.dream.model == "openai/gpt-4.1-mini"
assert loop.dream.provider is cheap_provider
# Switch main preset to fast
loop.set_model_preset("fast")
# Main agent should be on fast
assert loop.model == "openai/gpt-4.1"
assert loop.provider is fast_provider
# Dream should still be on cheap override
assert loop.dream.model == "openai/gpt-4.1-mini"
assert loop.dream.provider is cheap_provider
assert loop.dream._runner.provider is cheap_provider
@@ -43,32 +43,6 @@ def test_list_sessions_includes_metadata_title(tmp_path):
assert rows[0]["title"] == "自动生成标题" assert rows[0]["title"] == "自动生成标题"
def test_list_sessions_hides_generated_think_title(tmp_path):
manager = SessionManager(tmp_path)
session = manager.get_or_create("websocket:chat-think-title")
session.metadata["title"] = "<think> The user said hello and assistant replied"
session.add_message("user", "hello")
manager.save(session)
rows = manager.list_sessions()
assert rows[0]["key"] == "websocket:chat-think-title"
assert rows[0]["title"] == ""
assert rows[0]["preview"] == "hello"
def test_list_sessions_keeps_user_edited_think_title(tmp_path):
manager = SessionManager(tmp_path)
session = manager.get_or_create("websocket:chat-user-title")
session.metadata["title"] = "<think> literally discussed"
session.metadata["title_user_edited"] = True
manager.save(session)
rows = manager.list_sessions()
assert rows[0]["title"] == "<think> literally discussed"
def test_list_sessions_includes_user_preview(tmp_path): def test_list_sessions_includes_user_preview(tmp_path):
manager = SessionManager(tmp_path) manager = SessionManager(tmp_path)
session = manager.get_or_create("websocket:chat-preview") session = manager.get_or_create("websocket:chat-preview")
+1 -27
View File
@@ -474,32 +474,6 @@ class TestStopCommandWithUnifiedSession:
assert task.cancelled() or task.done() assert task.cancelled() or task.done()
assert "Stopped 1 task" in result.content assert "Stopped 1 task" in result.content
@pytest.mark.asyncio
async def test_stop_command_uses_effective_key_without_session_override(self, tmp_path: Path):
"""Priority /stop must cancel the unified session even before dispatch rewrites the message."""
from nanobot.agent.loop import UNIFIED_SESSION_KEY
from nanobot.command.builtin import cmd_stop
loop = _make_loop(tmp_path, unified_session=True)
async def long_running():
await asyncio.sleep(10)
task = asyncio.create_task(long_running())
loop._active_tasks[UNIFIED_SESSION_KEY] = [task]
msg = InboundMessage(
channel="telegram",
chat_id="123456",
sender_id="user1",
content="/stop",
)
ctx = CommandContext(msg=msg, session=None, key=UNIFIED_SESSION_KEY, raw="/stop", loop=loop)
result = await cmd_stop(ctx)
assert task.cancelled() or task.done()
assert "Stopped 1 task" in result.content
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_stop_command_cross_channel_in_unified_mode(self, tmp_path: Path): async def test_stop_command_cross_channel_in_unified_mode(self, tmp_path: Path):
"""In unified mode, /stop from one channel cancels tasks from another channel.""" """In unified mode, /stop from one channel cancels tasks from another channel."""
@@ -530,4 +504,4 @@ class TestStopCommandWithUnifiedSession:
result = await cmd_stop(ctx) result = await cmd_stop(ctx)
# Both tasks should be cancelled # Both tasks should be cancelled
assert "Stopped 2 task" in result.content assert "Stopped 2 task" in result.content
-348
View File
@@ -1,348 +0,0 @@
import json
import time
from pathlib import Path
from types import SimpleNamespace
import pytest
from nanobot.agent.tools.cli_apps import CliAppsTool
from nanobot.agent.tools.filesystem import ReadFileTool
from nanobot.agent.tools.image_generation import ImageGenerationError, ImageGenerationTool
from nanobot.agent.tools.message import MessageTool
from nanobot.agent.tools.shell import ExecTool
from nanobot.agent.tools.spawn import SpawnTool
from nanobot.security.workspace_access import (
WORKSPACE_SCOPE_METADATA_KEY,
WorkspaceScopeError,
bind_workspace_scope,
default_workspace_scope,
reset_workspace_scope,
validate_workspace_scope_payload,
workspace_scope_from_metadata,
)
from nanobot.apps.cli.service import CliAppManager, CliAppsRuntimeConfig
from nanobot.config.schema import ImageGenerationToolConfig, ProviderConfig
PNG_BYTES = (
b"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x01"
b"\x00\x00\x00\x01\x08\x04\x00\x00\x00\xb5\x1c\x0c\x02"
b"\x00\x00\x00\x0bIDATx\xdacd\xfc\xff\x1f\x00\x03\x03"
b"\x02\x00\xef\xbf\xa7\xdb\x00\x00\x00\x00IEND\xaeB`\x82"
)
def test_workspace_scope_defaults_match_legacy_config(tmp_path: Path) -> None:
unrestricted = default_workspace_scope(tmp_path, restrict_to_workspace=False)
restricted = default_workspace_scope(tmp_path, restrict_to_workspace=True)
assert unrestricted.project_path == tmp_path.resolve()
assert unrestricted.access_mode == "full"
assert unrestricted.restrict_to_workspace is False
assert restricted.access_mode == "restricted"
assert restricted.restrict_to_workspace is True
def test_workspace_scope_rejects_invalid_project_path(tmp_path: Path) -> None:
with pytest.raises(WorkspaceScopeError, match="absolute"):
validate_workspace_scope_payload(
{"project_path": "relative/project", "access_mode": "restricted"},
default_workspace=tmp_path,
default_restrict_to_workspace=False,
)
with pytest.raises(WorkspaceScopeError, match="existing directory"):
validate_workspace_scope_payload(
{"project_path": str(tmp_path / "missing"), "access_mode": "restricted"},
default_workspace=tmp_path,
default_restrict_to_workspace=False,
)
def test_workspace_scope_accepts_home_relative_project_path(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
home = tmp_path / "home"
project = home / "Desktop" / "Photos"
project.mkdir(parents=True)
monkeypatch.setenv("HOME", str(home))
monkeypatch.setenv("USERPROFILE", str(home))
scope = validate_workspace_scope_payload(
{"project_path": "~/Desktop/Photos", "access_mode": "restricted"},
default_workspace=tmp_path,
default_restrict_to_workspace=False,
)
assert scope.project_path == project.resolve()
assert scope.metadata()["project_path"] == str(project.resolve())
def test_workspace_scope_metadata_falls_back_for_stale_session(tmp_path: Path) -> None:
scope = workspace_scope_from_metadata(
{
WORKSPACE_SCOPE_METADATA_KEY: {
"project_path": str(tmp_path / "missing"),
"access_mode": "restricted",
}
},
default_workspace=tmp_path,
default_restrict_to_workspace=False,
)
assert scope.project_path == tmp_path.resolve()
assert scope.access_mode == "full"
@pytest.mark.asyncio
async def test_filesystem_tool_uses_current_restricted_workspace_scope(tmp_path: Path) -> None:
project = tmp_path / "project"
project.mkdir()
outside = tmp_path / "outside.txt"
outside.write_text("nope")
inside = project / "inside.txt"
inside.write_text("ok")
tool = ReadFileTool(workspace=tmp_path, restrict_to_workspace=False)
scope = validate_workspace_scope_payload(
{"project_path": str(project), "access_mode": "restricted"},
default_workspace=tmp_path,
default_restrict_to_workspace=False,
)
token = bind_workspace_scope(scope)
try:
assert "ok" in await tool.execute(path="inside.txt")
assert "outside allowed directory" in await tool.execute(path=str(outside))
finally:
reset_workspace_scope(token)
@pytest.mark.asyncio
async def test_exec_tool_uses_scope_project_as_default_cwd(tmp_path: Path) -> None:
project = tmp_path / "project"
project.mkdir()
tool = ExecTool(working_dir=str(tmp_path), restrict_to_workspace=False, timeout=5)
scope = validate_workspace_scope_payload(
{"project_path": str(project), "access_mode": "restricted"},
default_workspace=tmp_path,
default_restrict_to_workspace=False,
)
token = bind_workspace_scope(scope)
try:
result = await tool.execute(command="printf ok > scoped-marker.txt")
finally:
reset_workspace_scope(token)
assert "Exit code: 0" in result
assert (project / "scoped-marker.txt").read_text() == "ok"
@pytest.mark.asyncio
async def test_exec_full_scope_allows_explicit_cwd_outside_project(tmp_path: Path) -> None:
project = tmp_path / "project"
outside = tmp_path / "outside"
project.mkdir()
outside.mkdir()
tool = ExecTool(working_dir=str(tmp_path), restrict_to_workspace=True, timeout=5)
scope = validate_workspace_scope_payload(
{"project_path": str(project), "access_mode": "full"},
default_workspace=tmp_path,
default_restrict_to_workspace=True,
)
token = bind_workspace_scope(scope)
try:
result = await tool.execute(command="printf ok > outside-marker.txt", working_dir=str(outside))
finally:
reset_workspace_scope(token)
assert "Exit code: 0" in result
assert (outside / "outside-marker.txt").read_text() == "ok"
def test_image_reference_scope_restricted_blocks_outside_and_full_allows(tmp_path: Path) -> None:
project = tmp_path / "project"
outside = tmp_path / "outside"
project.mkdir()
outside.mkdir()
ref = outside / "ref.png"
ref.write_bytes(PNG_BYTES)
tool = ImageGenerationTool(
workspace=tmp_path,
config=ImageGenerationToolConfig(enabled=True),
provider_config=ProviderConfig(api_key="sk-test"),
)
restricted = validate_workspace_scope_payload(
{"project_path": str(project), "access_mode": "restricted"},
default_workspace=tmp_path,
default_restrict_to_workspace=False,
)
token = bind_workspace_scope(restricted)
try:
with pytest.raises(ImageGenerationError, match="inside the workspace"):
tool._resolve_reference_image(str(ref))
finally:
reset_workspace_scope(token)
full = validate_workspace_scope_payload(
{"project_path": str(project), "access_mode": "full"},
default_workspace=tmp_path,
default_restrict_to_workspace=True,
)
token = bind_workspace_scope(full)
try:
assert tool._resolve_reference_image(str(ref)) == str(ref.resolve())
finally:
reset_workspace_scope(token)
def test_message_media_scope_restricted_blocks_outside_and_full_allows(tmp_path: Path) -> None:
project = tmp_path / "project"
outside = tmp_path / "outside"
project.mkdir()
outside.mkdir()
media = outside / "shot.png"
media.write_bytes(PNG_BYTES)
tool = MessageTool(workspace=tmp_path, restrict_to_workspace=True)
restricted = validate_workspace_scope_payload(
{"project_path": str(project), "access_mode": "restricted"},
default_workspace=tmp_path,
default_restrict_to_workspace=False,
)
token = bind_workspace_scope(restricted)
try:
with pytest.raises(PermissionError):
tool._resolve_media([str(media)])
finally:
reset_workspace_scope(token)
full = validate_workspace_scope_payload(
{"project_path": str(project), "access_mode": "full"},
default_workspace=tmp_path,
default_restrict_to_workspace=True,
)
token = bind_workspace_scope(full)
try:
assert tool._resolve_media([str(media)]) == [str(media)]
finally:
reset_workspace_scope(token)
@pytest.mark.asyncio
async def test_cli_app_scope_controls_working_dir(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
project = tmp_path / "project"
outside = tmp_path / "outside"
data_dir = tmp_path / "data"
project.mkdir()
outside.mkdir()
registry = {
"meta": {},
"clis": [
{
"name": "demo",
"display_name": "Demo",
"version": "1.0",
"description": "demo",
"category": "test",
"install_cmd": "pip install demo",
"entry_point": "demo-cli",
}
],
}
data_dir.mkdir()
(data_dir / "harness_registry_cache.json").write_text(
json.dumps({"_cached_at": time.time(), "data": registry}),
encoding="utf-8",
)
(data_dir / "public_registry_cache.json").write_text(
json.dumps({"_cached_at": time.time(), "data": {"meta": {}, "clis": []}}),
encoding="utf-8",
)
(data_dir / "extensions_registry_cache.json").write_text(
json.dumps({"_cached_at": time.time(), "data": {"meta": {}, "clis": []}}),
encoding="utf-8",
)
CliAppManager(workspace=project, data_dir=data_dir)._save_installed(
{"demo": {"entry_point": "demo-cli"}}
)
monkeypatch.setattr("nanobot.apps.cli.service.get_runtime_subdir", lambda _name: data_dir)
monkeypatch.setattr(
"nanobot.apps.cli.service.shutil.which",
lambda entry: "/usr/bin/demo-cli" if entry == "demo-cli" else None,
)
seen: dict[str, str] = {}
def fake_run(argv, **kwargs):
seen["cwd"] = kwargs["cwd"]
return SimpleNamespace(returncode=0, stdout="ok", stderr="")
monkeypatch.setattr("nanobot.apps.cli.service.subprocess.run", fake_run)
tool = CliAppsTool(
workspace=tmp_path,
restrict_to_workspace=True,
runtime=CliAppsRuntimeConfig(run_timeout=5),
)
restricted = validate_workspace_scope_payload(
{"project_path": str(project), "access_mode": "restricted"},
default_workspace=tmp_path,
default_restrict_to_workspace=False,
)
token = bind_workspace_scope(restricted)
try:
blocked = await tool.execute(name="demo", working_dir=str(outside))
finally:
reset_workspace_scope(token)
assert "outside the configured workspace" in blocked
full = validate_workspace_scope_payload(
{"project_path": str(project), "access_mode": "full"},
default_workspace=tmp_path,
default_restrict_to_workspace=True,
)
token = bind_workspace_scope(full)
try:
result = await tool.execute(name="demo", working_dir=str(outside))
finally:
reset_workspace_scope(token)
assert "CLI app 'demo' exited 0" in result
assert seen["cwd"] == str(outside.resolve())
@pytest.mark.asyncio
async def test_spawn_tool_forwards_current_workspace_scope(tmp_path: Path) -> None:
project = tmp_path / "project"
project.mkdir()
scope = validate_workspace_scope_payload(
{"project_path": str(project), "access_mode": "restricted"},
default_workspace=tmp_path,
default_restrict_to_workspace=False,
)
class Manager:
max_concurrent_subagents = 4
def __init__(self) -> None:
self.seen = None
def get_running_count(self) -> int:
return 0
async def spawn(self, **kwargs):
self.seen = kwargs
return "spawned"
manager = Manager()
tool = SpawnTool(manager) # type: ignore[arg-type]
token = bind_workspace_scope(scope)
try:
result = await tool.execute(task="inspect")
finally:
reset_workspace_scope(token)
assert result == "spawned"
assert manager.seen["workspace_scope"] == scope
-44
View File
@@ -2,7 +2,6 @@
from __future__ import annotations from __future__ import annotations
import asyncio
from unittest.mock import AsyncMock, MagicMock from unittest.mock import AsyncMock, MagicMock
import pytest import pytest
@@ -73,49 +72,6 @@ async def test_complete_goal_closes_active_goal(tmp_path):
assert blob["recap"] == "Done." assert blob["recap"] == "Done."
@pytest.mark.asyncio
async def test_goal_tools_keep_request_context_per_task(tmp_path):
sm = SessionManager(tmp_path)
lt = LongTaskTool(sessions=sm)
cg = CompleteGoalTool(sessions=sm)
ctx_a = RequestContext(channel="websocket", chat_id="a", session_key="websocket:a")
ctx_b = RequestContext(channel="websocket", chat_id="b", session_key="websocket:b")
lt.set_context(ctx_a)
task_a = asyncio.create_task(lt.execute(goal="Goal A"))
lt.set_context(ctx_b)
task_b = asyncio.create_task(lt.execute(goal="Goal B"))
await asyncio.gather(task_a, task_b)
assert sm.get_or_create("websocket:a").metadata[GOAL_STATE_KEY]["objective"] == "Goal A"
assert sm.get_or_create("websocket:b").metadata[GOAL_STATE_KEY]["objective"] == "Goal B"
cg.set_context(ctx_a)
done_a = asyncio.create_task(cg.execute(recap="Done A"))
cg.set_context(ctx_b)
done_b = asyncio.create_task(cg.execute(recap="Done B"))
await asyncio.gather(done_a, done_b)
assert sm.get_or_create("websocket:a").metadata[GOAL_STATE_KEY]["recap"] == "Done A"
assert sm.get_or_create("websocket:b").metadata[GOAL_STATE_KEY]["recap"] == "Done B"
@pytest.mark.asyncio
async def test_goal_tools_context_isolated_across_tool_types(tmp_path):
"""LongTaskTool and CompleteGoalTool must not share routing context."""
sm = SessionManager(tmp_path)
lt = LongTaskTool(sessions=sm)
cg = CompleteGoalTool(sessions=sm)
ctx = RequestContext(channel="websocket", chat_id="a", session_key="websocket:a")
lt.set_context(ctx)
assert cg._request_ctx.get() is None
cg.set_context(ctx)
assert lt._request_ctx.get() is ctx
assert cg._request_ctx.get() is ctx
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_long_task_publishes_goal_state_ws_after_save(tmp_path): async def test_long_task_publishes_goal_state_ws_after_save(tmp_path):
bus = MagicMock() bus = MagicMock()
-7
View File
@@ -91,13 +91,6 @@ def test_channels_config_builtin_fields_removed():
assert not hasattr(cfg, "telegram") assert not hasattr(cfg, "telegram")
assert cfg.send_progress is True assert cfg.send_progress is True
assert cfg.send_tool_hints is False assert cfg.send_tool_hints is False
assert cfg.extract_document_text is True
def test_channels_config_extract_document_text_accepts_camel_alias():
cfg = ChannelsConfig.model_validate({"extractDocumentText": False})
assert cfg.extract_document_text is False
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
+1 -26
View File
@@ -865,7 +865,7 @@ async def test_slash_new_is_blocked_for_disallowed_user() -> None:
assert handled == [] assert handled == []
@pytest.mark.parametrize("slash_name", ["stop", "restart", "status", "history", "model"]) @pytest.mark.parametrize("slash_name", ["stop", "restart", "status", "history"])
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_slash_commands_forward_via_handle_message(slash_name: str) -> None: async def test_slash_commands_forward_via_handle_message(slash_name: str) -> None:
channel = DiscordChannel(DiscordConfig(enabled=True, allow_from=["*"]), MessageBus()) channel = DiscordChannel(DiscordConfig(enabled=True, allow_from=["*"]), MessageBus())
@@ -891,31 +891,6 @@ async def test_slash_commands_forward_via_handle_message(slash_name: str) -> Non
assert handled[0]["metadata"]["is_slash_command"] is True assert handled[0]["metadata"]["is_slash_command"] is True
@pytest.mark.asyncio
async def test_slash_model_forwards_optional_preset() -> None:
channel = DiscordChannel(DiscordConfig(enabled=True, allow_from=["*"]), MessageBus())
handled: list[dict] = []
async def capture_handle(**kwargs) -> None:
handled.append(kwargs)
channel._handle_message = capture_handle # type: ignore[method-assign]
client = DiscordBotClient(channel, intents=discord.Intents.none())
interaction = _make_interaction()
interaction.command.qualified_name = "model"
model_cmd = client.tree.get_command("model")
assert model_cmd is not None
await model_cmd.callback(interaction, preset="fast")
assert interaction.response.messages == [
{"content": "Processing /model fast...", "ephemeral": True}
]
assert len(handled) == 1
assert handled[0]["content"] == "/model fast"
assert handled[0]["metadata"]["is_slash_command"] is True
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_slash_help_returns_ephemeral_help_text() -> None: async def test_slash_help_returns_ephemeral_help_text() -> None:
channel = DiscordChannel(DiscordConfig(enabled=True, allow_from=["*"]), MessageBus()) channel = DiscordChannel(DiscordConfig(enabled=True, allow_from=["*"]), MessageBus())
+8 -145
View File
@@ -9,6 +9,8 @@ pytest.importorskip("nh3")
pytest.importorskip("mistune") pytest.importorskip("mistune")
from nio import RoomSendResponse, SyncError from nio import RoomSendResponse, SyncError
from nanobot.channels.matrix import _build_matrix_text_content
import nanobot.channels.matrix as matrix_module import nanobot.channels.matrix as matrix_module
from nanobot.bus.events import OutboundMessage from nanobot.bus.events import OutboundMessage
from nanobot.bus.queue import MessageBus from nanobot.bus.queue import MessageBus
@@ -16,9 +18,8 @@ from nanobot.channels.matrix import (
MATRIX_HTML_FORMAT, MATRIX_HTML_FORMAT,
TYPING_NOTICE_TIMEOUT_MS, TYPING_NOTICE_TIMEOUT_MS,
MatrixChannel, MatrixChannel,
MatrixConfig,
_build_matrix_text_content,
) )
from nanobot.channels.matrix import MatrixConfig
_ROOM_SEND_UNSET = object() _ROOM_SEND_UNSET = object()
@@ -692,13 +693,6 @@ async def test_on_media_message_downloads_attachment_and_sets_metadata(
client.download_bytes = b"image" client.download_bytes = b"image"
channel.client = client channel.client = client
async def _download_media_bytes(mxc_url: str, limit_bytes: int) -> bytes:
client.download_calls.append(mxc_url)
assert limit_bytes >= len(client.download_bytes)
return client.download_bytes
monkeypatch.setattr(channel, "_download_media_bytes", _download_media_bytes)
handled: list[dict[str, object]] = [] handled: list[dict[str, object]] = []
async def _fake_handle_message(**kwargs) -> None: async def _fake_handle_message(**kwargs) -> None:
@@ -863,14 +857,9 @@ async def test_on_media_message_handles_download_error(monkeypatch, tmp_path) ->
channel = MatrixChannel(_make_config(), MessageBus()) channel = MatrixChannel(_make_config(), MessageBus())
client = _FakeAsyncClient("", "", "", None) client = _FakeAsyncClient("", "", "", None)
client.download_response = matrix_module.DownloadError("download failed")
channel.client = client channel.client = client
async def _download_media_bytes(mxc_url: str, _limit_bytes: int):
client.download_calls.append(mxc_url)
return None
monkeypatch.setattr(channel, "_download_media_bytes", _download_media_bytes)
handled: list[dict[str, object]] = [] handled: list[dict[str, object]] = []
async def _fake_handle_message(**kwargs) -> None: async def _fake_handle_message(**kwargs) -> None:
@@ -884,7 +873,7 @@ async def test_on_media_message_handles_download_error(monkeypatch, tmp_path) ->
body="photo.png", body="photo.png",
url="mxc://example.org/mediaid", url="mxc://example.org/mediaid",
event_id="$event3", event_id="$event3",
source={"content": {"msgtype": "m.image", "info": {"size": 5}}}, source={"content": {"msgtype": "m.image"}},
) )
await channel._on_media_message(room, event) await channel._on_media_message(room, event)
@@ -910,13 +899,6 @@ async def test_on_media_message_decrypts_encrypted_media(monkeypatch, tmp_path)
client.download_bytes = b"cipher" client.download_bytes = b"cipher"
channel.client = client channel.client = client
async def _download_media_bytes(mxc_url: str, limit_bytes: int) -> bytes:
client.download_calls.append(mxc_url)
assert limit_bytes >= len(client.download_bytes)
return client.download_bytes
monkeypatch.setattr(channel, "_download_media_bytes", _download_media_bytes)
handled: list[dict[str, object]] = [] handled: list[dict[str, object]] = []
async def _fake_handle_message(**kwargs) -> None: async def _fake_handle_message(**kwargs) -> None:
@@ -960,13 +942,6 @@ async def test_on_media_message_handles_decrypt_error(monkeypatch, tmp_path) ->
client.download_bytes = b"cipher" client.download_bytes = b"cipher"
channel.client = client channel.client = client
async def _download_media_bytes(mxc_url: str, limit_bytes: int) -> bytes:
client.download_calls.append(mxc_url)
assert limit_bytes >= len(client.download_bytes)
return client.download_bytes
monkeypatch.setattr(channel, "_download_media_bytes", _download_media_bytes)
handled: list[dict[str, object]] = [] handled: list[dict[str, object]] = []
async def _fake_handle_message(**kwargs) -> None: async def _fake_handle_message(**kwargs) -> None:
@@ -983,7 +958,7 @@ async def test_on_media_message_handles_decrypt_error(monkeypatch, tmp_path) ->
key={"k": "key"}, key={"k": "key"},
hashes={"sha256": "hash"}, hashes={"sha256": "hash"},
iv="iv", iv="iv",
source={"content": {"msgtype": "m.file", "info": {"size": 6}}}, source={"content": {"msgtype": "m.file"}},
) )
await channel._on_media_message(room, event) await channel._on_media_message(room, event)
@@ -1781,7 +1756,7 @@ async def test_send_delta_on_error_stops_typing(monkeypatch) -> None:
assert "!room:matrix.org" in channel._stream_bufs assert "!room:matrix.org" in channel._stream_bufs
assert channel._stream_bufs["!room:matrix.org"].text == "Hello" assert channel._stream_bufs["!room:matrix.org"].text == "Hello"
assert len(client.room_send_calls) == 1 assert len(client.room_send_calls) == 1
assert len(client.typing_calls) == 1 assert len(client.typing_calls) == 1
@@ -1798,116 +1773,4 @@ async def test_send_delta_ignores_whitespace_only_delta(monkeypatch) -> None:
assert "!room:matrix.org" in channel._stream_bufs assert "!room:matrix.org" in channel._stream_bufs
assert channel._stream_bufs["!room:matrix.org"].text == " " assert channel._stream_bufs["!room:matrix.org"].text == " "
assert client.room_send_calls == []
@pytest.mark.asyncio
async def test_fetch_media_rejects_missing_declared_size(monkeypatch, tmp_path) -> None:
channel = MatrixChannel(_make_config(max_media_bytes=8), MessageBus())
client = _FakeAsyncClient("https://matrix.org", "", "", None)
channel.client = client
monkeypatch.setattr("nanobot.channels.matrix.get_media_dir", lambda _name: tmp_path)
async def _download_should_not_run(*_args, **_kwargs):
raise AssertionError("download should be rejected before fetching bytes")
monkeypatch.setattr(channel, "_download_media_bytes", _download_should_not_run)
event = SimpleNamespace(
sender="@alice:matrix.org",
event_id="$event1",
body="payload.bin",
url="mxc://example.org/media",
source={"content": {"msgtype": "m.file"}},
)
attachment, marker = await channel._fetch_media_attachment(
SimpleNamespace(room_id="!room:matrix.org"),
event,
)
assert attachment is None
assert marker == "[attachment: payload.bin - too large]"
@pytest.mark.asyncio
async def test_fetch_media_rejects_bool_declared_size(monkeypatch, tmp_path) -> None:
channel = MatrixChannel(_make_config(max_media_bytes=8), MessageBus())
client = _FakeAsyncClient("https://matrix.org", "", "", None)
channel.client = client
monkeypatch.setattr("nanobot.channels.matrix.get_media_dir", lambda _name: tmp_path)
async def _download_should_not_run(*_args, **_kwargs):
raise AssertionError("bool size should be rejected before fetching bytes")
monkeypatch.setattr(channel, "_download_media_bytes", _download_should_not_run)
event = SimpleNamespace(
sender="@alice:matrix.org",
event_id="$event1",
body="payload.bin",
url="mxc://example.org/media",
source={"content": {"msgtype": "m.file", "info": {"size": True}}},
)
attachment, marker = await channel._fetch_media_attachment(
SimpleNamespace(room_id="!room:matrix.org"),
event,
)
assert attachment is None
assert marker == "[attachment: payload.bin - too large]"
@pytest.mark.asyncio
async def test_fetch_media_rejects_declared_oversized_before_download(monkeypatch, tmp_path) -> None:
channel = MatrixChannel(_make_config(max_media_bytes=8), MessageBus())
client = _FakeAsyncClient("https://matrix.org", "", "", None)
channel.client = client
monkeypatch.setattr("nanobot.channels.matrix.get_media_dir", lambda _name: tmp_path)
async def _download_should_not_run(*_args, **_kwargs):
raise AssertionError("download should be rejected before fetching bytes")
monkeypatch.setattr(channel, "_download_media_bytes", _download_should_not_run)
event = SimpleNamespace(
sender="@alice:matrix.org",
event_id="$event1",
body="payload.bin",
url="mxc://example.org/media",
source={"content": {"msgtype": "m.file", "info": {"size": 9}}},
)
attachment, marker = await channel._fetch_media_attachment(
SimpleNamespace(room_id="!room:matrix.org"),
event,
)
assert attachment is None
assert marker == "[attachment: payload.bin - too large]"
@pytest.mark.asyncio
async def test_fetch_media_maps_streaming_cap_to_too_large(monkeypatch, tmp_path) -> None:
channel = MatrixChannel(_make_config(max_media_bytes=8), MessageBus())
client = _FakeAsyncClient("https://matrix.org", "", "", None)
channel.client = client
monkeypatch.setattr("nanobot.channels.matrix.get_media_dir", lambda _name: tmp_path)
async def _download_too_large(_mxc_url: str, _limit_bytes: int):
raise matrix_module._MediaTooLargeError
monkeypatch.setattr(channel, "_download_media_bytes", _download_too_large)
event = SimpleNamespace(
sender="@alice:matrix.org",
event_id="$event1",
body="payload.bin",
url="mxc://example.org/media",
source={"content": {"msgtype": "m.file", "info": {"size": 8}}},
)
attachment, marker = await channel._fetch_media_attachment(
SimpleNamespace(room_id="!room:matrix.org"),
event,
)
assert attachment is None
assert marker == "[attachment: payload.bin - too large]"
assert client.room_send_calls == []
-107
View File
@@ -1,4 +1,3 @@
import asyncio
from pathlib import Path from pathlib import Path
from types import SimpleNamespace from types import SimpleNamespace
from unittest.mock import AsyncMock from unittest.mock import AsyncMock
@@ -37,19 +36,11 @@ class _FakeUpdater:
def __init__(self, on_start_polling) -> None: def __init__(self, on_start_polling) -> None:
self._on_start_polling = on_start_polling self._on_start_polling = on_start_polling
self.start_polling_kwargs = None self.start_polling_kwargs = None
self.start_webhook_kwargs = None
async def start_polling(self, **kwargs) -> None: async def start_polling(self, **kwargs) -> None:
self.start_polling_kwargs = kwargs self.start_polling_kwargs = kwargs
self._on_start_polling() self._on_start_polling()
async def start_webhook(self, **kwargs) -> None:
self.start_webhook_kwargs = kwargs
self._on_start_polling()
async def stop(self) -> None:
pass
class _FakeBot: class _FakeBot:
def __init__(self) -> None: def __init__(self) -> None:
@@ -112,12 +103,6 @@ class _FakeApp:
async def start(self) -> None: async def start(self) -> None:
pass pass
async def stop(self) -> None:
pass
async def shutdown(self) -> None:
pass
class _FakeBuilder: class _FakeBuilder:
def __init__(self, app: _FakeApp) -> None: def __init__(self, app: _FakeApp) -> None:
@@ -247,98 +232,6 @@ async def test_start_respects_custom_pool_config(monkeypatch) -> None:
assert poll_req.kwargs["pool_timeout"] == 10.0 assert poll_req.kwargs["pool_timeout"] == 10.0
def test_webhook_config_requires_https_url_and_secret() -> None:
with pytest.raises(ValueError, match="webhook_url is required"):
TelegramConfig(enabled=True, token="123:abc", mode="webhook")
with pytest.raises(ValueError, match="public HTTPS URL"):
TelegramConfig(
enabled=True,
token="123:abc",
mode="webhook",
webhook_url="http://example.com/telegram",
webhook_secret_token="secret",
)
with pytest.raises(ValueError, match="webhook_secret_token is required"):
TelegramConfig(
enabled=True,
token="123:abc",
mode="webhook",
webhook_url="https://example.com/telegram",
)
@pytest.mark.asyncio
async def test_start_webhook_mode(monkeypatch) -> None:
_FakeHTTPXRequest.clear()
config = TelegramConfig(
enabled=True,
token="123:abc",
allow_from=["*"],
mode="webhook",
webhook_url="https://example.com/telegram",
webhook_listen_host="127.0.0.1",
webhook_listen_port=8081,
webhook_path="/telegram",
webhook_secret_token="secret-token",
webhook_max_connections=1,
)
bus = MessageBus()
channel = TelegramChannel(config, bus)
app = _FakeApp(lambda: setattr(channel, "_running", False))
builder = _FakeBuilder(app)
monkeypatch.setattr("nanobot.channels.telegram.HTTPXRequest", _FakeHTTPXRequest)
monkeypatch.setattr(
"nanobot.channels.telegram.Application",
SimpleNamespace(builder=lambda: builder),
)
await channel.start()
assert app.updater.start_polling_kwargs is None
assert app.updater.start_webhook_kwargs == {
"listen": "127.0.0.1",
"port": 8081,
"url_path": "telegram",
"webhook_url": "https://example.com/telegram",
"allowed_updates": ["message"],
"drop_pending_updates": False,
"secret_token": "secret-token",
"max_connections": 1,
}
@pytest.mark.asyncio
async def test_running_message_handler_reorders_same_session_updates() -> None:
channel = TelegramChannel(
TelegramConfig(enabled=True, token="123:abc", allow_from=["*"]),
MessageBus(),
)
seen: list[int] = []
async def fake_process(update, context) -> None:
seen.append(update.message.message_id)
channel._process_message_update = fake_process
channel._running = True
first = _make_telegram_update(text="first")
first.update_id = 100
first.message.message_id = 1
second = _make_telegram_update(text="second")
second.update_id = 101
second.message.message_id = 2
await channel._on_message(second, None)
await channel._on_message(first, None)
await asyncio.sleep(0.3)
channel._running = False
assert seen == [1, 2]
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_send_text_retries_on_timeout() -> None: async def test_send_text_retries_on_timeout() -> None:
"""_send_text retries on TimedOut before succeeding.""" """_send_text retries on TimedOut before succeeding."""
+10 -458
View File
@@ -30,8 +30,6 @@ from nanobot.channels.websocket import (
) )
from nanobot.config.loader import load_config, save_config from nanobot.config.loader import load_config, save_config
from nanobot.config.schema import Config, ModelPresetConfig from nanobot.config.schema import Config, ModelPresetConfig
from nanobot.session import webui_turns as wth
from nanobot.session.manager import SessionManager
from nanobot.webui.settings_api import settings_payload, update_provider_settings from nanobot.webui.settings_api import settings_payload, update_provider_settings
# -- Shared helpers (aligned with test_websocket_integration.py) --------------- # -- Shared helpers (aligned with test_websocket_integration.py) ---------------
@@ -59,14 +57,6 @@ def bus() -> MagicMock:
return b return b
@pytest.fixture(autouse=True)
def isolate_webui_workspace_state(tmp_path, monkeypatch) -> None:
monkeypatch.setattr(
"nanobot.webui.workspaces.get_webui_dir",
lambda: tmp_path / "webui",
)
async def _http_get(url: str, headers: dict[str, str] | None = None) -> httpx.Response: async def _http_get(url: str, headers: dict[str, str] | None = None) -> httpx.Response:
"""Run GET in a thread to avoid blocking the asyncio loop shared with websockets.""" """Run GET in a thread to avoid blocking the asyncio loop shared with websockets."""
return await asyncio.to_thread( return await asyncio.to_thread(
@@ -74,15 +64,6 @@ async def _http_get(url: str, headers: dict[str, str] | None = None) -> httpx.Re
) )
async def _recv_ws_event(client: Any, event: str) -> dict[str, Any]:
"""Receive until a specific websocket event appears."""
for _ in range(10):
payload = json.loads(await client.recv())
if payload.get("event") == event:
return payload
raise AssertionError(f"websocket event {event!r} was not received")
def test_normalize_http_path_strips_trailing_slash_except_root() -> None: def test_normalize_http_path_strips_trailing_slash_except_root() -> None:
assert _normalize_http_path("/chat/") == "/chat" assert _normalize_http_path("/chat/") == "/chat"
assert _normalize_http_path("/chat?x=1") == "/chat" assert _normalize_http_path("/chat?x=1") == "/chat"
@@ -100,19 +81,6 @@ def test_normalize_config_path_matches_request() -> None:
assert _normalize_config_path("/") == "/" assert _normalize_config_path("/") == "/"
def test_websocket_config_accepts_absolute_unix_socket(tmp_path) -> None:
socket_path = tmp_path / "engine.sock"
cfg = WebSocketConfig(unix_socket_path=str(socket_path))
assert cfg.unix_socket_path == str(socket_path)
def test_websocket_config_rejects_relative_unix_socket() -> None:
with pytest.raises(ValueError, match="absolute path"):
WebSocketConfig(unix_socket_path="engine.sock")
def test_parse_query_extracts_token_and_client_id() -> None: def test_parse_query_extracts_token_and_client_id() -> None:
query = _parse_query("/?token=secret&client_id=u1") query = _parse_query("/?token=secret&client_id=u1")
assert query.get("token") == ["secret"] assert query.get("token") == ["secret"]
@@ -236,291 +204,6 @@ async def test_plain_websocket_message_does_not_mark_webui(bus: MagicMock) -> No
assert "webui" not in msg.metadata assert "webui" not in msg.metadata
@pytest.mark.asyncio
async def test_webui_message_scope_inherits_persisted_session_scope(
bus: MagicMock,
tmp_path,
) -> None:
default_workspace = tmp_path / "default"
project = tmp_path / "project"
default_workspace.mkdir()
project.mkdir()
sessions = SessionManager(tmp_path / "sessions")
channel = WebSocketChannel(
{"enabled": True, "allowFrom": ["*"], "host": "127.0.0.1"},
bus,
session_manager=sessions,
workspace_path=default_workspace,
restrict_to_workspace=True,
)
conn = AsyncMock()
conn.remote_address = ("127.0.0.1", 50123)
await channel._dispatch_envelope(
conn,
"webui-client",
{
"type": "set_workspace_scope",
"chat_id": "chat-scope",
"workspace_scope": {
"project_path": str(project),
"access_mode": "full",
},
},
)
await channel._dispatch_envelope(
conn,
"webui-client",
{"type": "message", "chat_id": "chat-scope", "content": "hello", "webui": True},
)
msg = bus.publish_inbound.await_args.args[0]
assert msg.metadata["workspace_scope"] == {
"project_path": str(project.resolve()),
"access_mode": "full",
}
@pytest.mark.asyncio
async def test_webui_scope_expands_home_project_path(
bus: MagicMock,
tmp_path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
default_workspace = tmp_path / "default"
home = tmp_path / "home"
project = home / "Desktop" / "Photos"
default_workspace.mkdir()
project.mkdir(parents=True)
monkeypatch.setenv("HOME", str(home))
monkeypatch.setenv("USERPROFILE", str(home))
channel = WebSocketChannel(
{"enabled": True, "allowFrom": ["*"], "host": "127.0.0.1"},
bus,
session_manager=SessionManager(tmp_path / "sessions"),
workspace_path=default_workspace,
restrict_to_workspace=True,
)
conn = AsyncMock()
conn.remote_address = ("127.0.0.1", 50123)
await channel._dispatch_envelope(
conn,
"webui-client",
{
"type": "set_workspace_scope",
"chat_id": "chat-scope",
"workspace_scope": {
"project_path": "~/Desktop/Photos",
"access_mode": "restricted",
},
},
)
await channel._dispatch_envelope(
conn,
"webui-client",
{"type": "message", "chat_id": "chat-scope", "content": "hello", "webui": True},
)
msg = bus.publish_inbound.await_args.args[0]
assert msg.metadata["workspace_scope"] == {
"project_path": str(project.resolve()),
"access_mode": "restricted",
}
@pytest.mark.asyncio
async def test_webui_scope_rejects_missing_project_path(bus: MagicMock, tmp_path) -> None:
default_workspace = tmp_path / "default"
default_workspace.mkdir()
channel = WebSocketChannel(
{"enabled": True, "allowFrom": ["*"], "host": "127.0.0.1"},
bus,
session_manager=SessionManager(tmp_path / "sessions"),
workspace_path=default_workspace,
)
conn = AsyncMock()
conn.remote_address = ("127.0.0.1", 50123)
await channel._dispatch_envelope(
conn,
"webui-client",
{
"type": "set_workspace_scope",
"chat_id": "chat-scope",
"workspace_scope": {
"project_path": str(tmp_path / "missing"),
"access_mode": "restricted",
},
},
)
conn.send.assert_awaited()
payload = json.loads(conn.send.await_args.args[0])
assert payload["event"] == "error"
assert payload["detail"] == "workspace_scope_rejected"
bus.publish_inbound.assert_not_awaited()
@pytest.mark.asyncio
async def test_webui_scope_rejects_running_scope_change(bus: MagicMock, tmp_path) -> None:
default_workspace = tmp_path / "default"
project = tmp_path / "project"
other = tmp_path / "other"
default_workspace.mkdir()
project.mkdir()
other.mkdir()
sessions = SessionManager(tmp_path / "sessions")
channel = WebSocketChannel(
{"enabled": True, "allowFrom": ["*"], "host": "127.0.0.1"},
bus,
session_manager=sessions,
workspace_path=default_workspace,
restrict_to_workspace=True,
)
conn = AsyncMock()
conn.remote_address = ("127.0.0.1", 50123)
await channel._dispatch_envelope(
conn,
"webui-client",
{
"type": "set_workspace_scope",
"chat_id": "chat-running",
"workspace_scope": {
"project_path": str(project),
"access_mode": "restricted",
},
},
)
wth._WEBSOCKET_TURN_WALL_STARTED_AT["chat-running"] = 123.0
try:
await channel._dispatch_envelope(
conn,
"webui-client",
{
"type": "message",
"chat_id": "chat-running",
"content": "hello",
"webui": True,
"workspace_scope": {
"project_path": str(other),
"access_mode": "full",
},
},
)
finally:
wth._WEBSOCKET_TURN_WALL_STARTED_AT.clear()
payload = json.loads(conn.send.await_args.args[0])
assert payload["event"] == "error"
assert payload["detail"] == "workspace_scope_rejected"
assert payload["reason"] == "chat_running"
assert payload["chat_id"] == "chat-running"
bus.publish_inbound.assert_not_awaited()
@pytest.mark.asyncio
async def test_webui_set_workspace_scope_rejects_running_chat(bus: MagicMock, tmp_path) -> None:
default_workspace = tmp_path / "default"
project = tmp_path / "project"
other = tmp_path / "other"
default_workspace.mkdir()
project.mkdir()
other.mkdir()
sessions = SessionManager(tmp_path / "sessions")
channel = WebSocketChannel(
{"enabled": True, "allowFrom": ["*"], "host": "127.0.0.1"},
bus,
session_manager=sessions,
workspace_path=default_workspace,
restrict_to_workspace=True,
)
conn = AsyncMock()
conn.remote_address = ("127.0.0.1", 50123)
await channel._dispatch_envelope(
conn,
"webui-client",
{
"type": "set_workspace_scope",
"chat_id": "chat-running",
"workspace_scope": {
"project_path": str(project),
"access_mode": "restricted",
},
},
)
conn.send.reset_mock()
wth._WEBSOCKET_TURN_WALL_STARTED_AT["chat-running"] = 123.0
try:
await channel._dispatch_envelope(
conn,
"webui-client",
{
"type": "set_workspace_scope",
"chat_id": "chat-running",
"workspace_scope": {
"project_path": str(other),
"access_mode": "full",
},
},
)
finally:
wth._WEBSOCKET_TURN_WALL_STARTED_AT.clear()
payload = json.loads(conn.send.await_args.args[0])
assert payload["event"] == "error"
assert payload["detail"] == "workspace_scope_rejected"
assert payload["reason"] == "chat_running"
assert payload["chat_id"] == "chat-running"
saved = sessions.read_session_file("websocket:chat-running")
assert saved["metadata"]["workspace_scope"] == {
"project_path": str(project.resolve()),
"access_mode": "restricted",
}
@pytest.mark.asyncio
async def test_webui_scope_rejects_non_loopback_custom_scope(bus: MagicMock, tmp_path) -> None:
default_workspace = tmp_path / "default"
project = tmp_path / "project"
default_workspace.mkdir()
project.mkdir()
sessions = SessionManager(tmp_path / "sessions")
channel = WebSocketChannel(
{"enabled": True, "allowFrom": ["*"], "host": "127.0.0.1"},
bus,
session_manager=sessions,
workspace_path=default_workspace,
restrict_to_workspace=True,
)
conn = AsyncMock()
conn.remote_address = ("203.0.113.8", 50123)
await channel._dispatch_envelope(
conn,
"webui-client",
{
"type": "set_workspace_scope",
"chat_id": "chat-remote",
"workspace_scope": {
"project_path": str(project),
"access_mode": "full",
},
},
)
payload = json.loads(conn.send.await_args.args[0])
assert payload["event"] == "error"
assert payload["detail"] == "workspace_scope_rejected"
assert payload["reason"] == "workspace controls are localhost-only"
assert payload["chat_id"] == "chat-remote"
assert sessions.read_session_file("websocket:chat-remote") is None
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_send_delivers_json_message_with_media_and_reply() -> None: async def test_send_delivers_json_message_with_media_and_reply() -> None:
bus = MagicMock() bus = MagicMock()
@@ -1384,15 +1067,6 @@ async def test_settings_api_returns_safe_subset_and_updates_whitelist(
config.tools.web.search.api_key = "brave-secret" config.tools.web.search.api_key = "brave-secret"
save_config(config, config_path) save_config(config, config_path)
monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path) monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path)
monkeypatch.setattr(
"nanobot.webui.settings_api._oauth_provider_status",
lambda _spec: {
"configured": False,
"account": None,
"expires_at": None,
"login_supported": True,
},
)
channel = _ch(bus, port=port) channel = _ch(bus, port=port)
channel._api_tokens["tok"] = time.monotonic() + 300 channel._api_tokens["tok"] = time.monotonic() + 300
@@ -1429,8 +1103,6 @@ async def test_settings_api_returns_safe_subset_and_updates_whitelist(
assert providers["atomic_chat"]["configured"] is False assert providers["atomic_chat"]["configured"] is False
assert providers["atomic_chat"]["api_key_required"] is False assert providers["atomic_chat"]["api_key_required"] is False
assert providers["atomic_chat"]["default_api_base"] == "http://localhost:1337/v1" assert providers["atomic_chat"]["default_api_base"] == "http://localhost:1337/v1"
assert providers["openai_codex"]["auth_type"] == "oauth"
assert providers["openai_codex"]["configured"] is False
assert body["agent"]["has_api_key"] is True assert body["agent"]["has_api_key"] is True
assert body["web_search"]["provider"] == "brave" assert body["web_search"]["provider"] == "brave"
assert body["web_search"]["api_key_hint"] == "brav••••cret" assert body["web_search"]["api_key_hint"] == "brav••••cret"
@@ -1449,29 +1121,18 @@ async def test_settings_api_returns_safe_subset_and_updates_whitelist(
} }
assert image_providers["openrouter"]["label"] == "OpenRouter" assert image_providers["openrouter"]["label"] == "OpenRouter"
assert image_providers["openrouter"]["configured"] is False assert image_providers["openrouter"]["configured"] is False
assert image_providers["openai_codex"]["auth_type"] == "oauth" assert image_providers["openai_codex"]["configured"] is True
assert image_providers["openai_codex"]["configured"] is False
assert image_providers["gemini"]["label"] == "Gemini" assert image_providers["gemini"]["label"] == "Gemini"
assert body["runtime"]["config_path"] == str(config_path) assert body["runtime"]["config_path"] == str(config_path)
workspace_path = body["runtime"]["workspace_path"].replace("\\", "/") workspace_path = body["runtime"]["workspace_path"].replace("\\", "/")
assert workspace_path.endswith("/.nanobot/workspace") assert workspace_path.endswith("/.nanobot/workspace")
assert body["runtime"]["gateway_port"] == 18790 assert body["runtime"]["gateway_port"] == 18790
assert body["advanced"]["exec_enabled"] is True assert body["advanced"]["exec_enabled"] is True
assert body["advanced"]["webui_allow_local_service_access"] is True
assert body["advanced"]["webui_default_access_mode"] == "default"
assert body["advanced"]["private_service_protection_enabled"] is True
assert body["advanced"]["mcp_server_count"] == 0 assert body["advanced"]["mcp_server_count"] == 0
assert body["restart_required_sections"] == [] assert body["restart_required_sections"] == []
assert "secret-key" not in settings.text assert "secret-key" not in settings.text
assert "brave-secret" not in settings.text assert "brave-secret" not in settings.text
unknown_api = await _http_get(
f"http://127.0.0.1:{port}/api/settings/model-configurations/missing",
headers={"Authorization": "Bearer tok"},
)
assert unknown_api.status_code == 404
assert "<!doctype html>" not in unknown_api.text.lower()
provider_updated = await _http_get( provider_updated = await _http_get(
"http://127.0.0.1:" "http://127.0.0.1:"
f"{port}/api/settings/provider/update?provider=openrouter" f"{port}/api/settings/provider/update?provider=openrouter"
@@ -1543,21 +1204,6 @@ async def test_settings_api_returns_safe_subset_and_updates_whitelist(
assert created_presets["fast-writing"]["label"] == "Fast writing" assert created_presets["fast-writing"]["label"] == "Fast writing"
assert created_presets["fast-writing"]["provider"] == "openai" assert created_presets["fast-writing"]["provider"] == "openai"
updated_preset = await _http_get(
"http://127.0.0.1:"
f"{port}/api/settings/model-configurations/update"
"?name=fast-writing&label=Codex&provider=openai&model=openai%2Fgpt-5.5",
headers={"Authorization": "Bearer tok"},
)
assert updated_preset.status_code == 200
updated_preset_body = updated_preset.json()
assert updated_preset_body["agent"]["model_preset"] == "fast-writing"
assert updated_preset_body["agent"]["model"] == "openai/gpt-5.5"
updated_presets = {
preset["name"]: preset for preset in updated_preset_body["model_presets"]
}
assert updated_presets["fast-writing"]["label"] == "Codex"
duplicate_preset = await _http_get( duplicate_preset = await _http_get(
"http://127.0.0.1:" "http://127.0.0.1:"
f"{port}/api/settings/model-configurations/create" f"{port}/api/settings/model-configurations/create"
@@ -1576,26 +1222,13 @@ async def test_settings_api_returns_safe_subset_and_updates_whitelist(
assert search_updated.status_code == 200 assert search_updated.status_code == 200
search_body = search_updated.json() search_body = search_updated.json()
assert search_body["requires_restart"] is True assert search_body["requires_restart"] is True
assert search_body["restart_required_sections"] == ["browser", "runtime"] assert search_body["restart_required_sections"] == ["runtime", "web"]
assert search_body["web_search"]["provider"] == "searxng" assert search_body["web_search"]["provider"] == "searxng"
assert search_body["web_search"]["api_key_hint"] is None assert search_body["web_search"]["api_key_hint"] is None
assert search_body["web_search"]["base_url"] == "https://search.example.com" assert search_body["web_search"]["base_url"] == "https://search.example.com"
assert search_body["web_search"]["max_results"] == 8 assert search_body["web_search"]["max_results"] == 8
assert search_body["web"]["fetch"]["use_jina_reader"] is False assert search_body["web"]["fetch"]["use_jina_reader"] is False
network_safety_updated = await _http_get(
"http://127.0.0.1:"
f"{port}/api/settings/network-safety/update?webui_allow_local_service_access=false&webui_default_access_mode=full",
headers={"Authorization": "Bearer tok"},
)
assert network_safety_updated.status_code == 200
network_safety_body = network_safety_updated.json()
assert network_safety_body["requires_restart"] is True
assert network_safety_body["restart_required_sections"] == ["browser", "runtime"]
assert network_safety_body["advanced"]["webui_allow_local_service_access"] is False
assert network_safety_body["advanced"]["webui_default_access_mode"] == "full"
assert network_safety_body["advanced"]["private_service_protection_enabled"] is True
image_updated = await _http_get( image_updated = await _http_get(
"http://127.0.0.1:" "http://127.0.0.1:"
f"{port}/api/settings/image-generation/update?enabled=true" f"{port}/api/settings/image-generation/update?enabled=true"
@@ -1607,7 +1240,7 @@ async def test_settings_api_returns_safe_subset_and_updates_whitelist(
assert image_updated.status_code == 200 assert image_updated.status_code == 200
image_body = image_updated.json() image_body = image_updated.json()
assert image_body["requires_restart"] is True assert image_body["requires_restart"] is True
assert image_body["restart_required_sections"] == ["browser", "image", "runtime"] assert image_body["restart_required_sections"] == ["image", "runtime", "web"]
assert image_body["image_generation"]["enabled"] is True assert image_body["image_generation"]["enabled"] is True
assert image_body["image_generation"]["model"] == "openai/gpt-image-1" assert image_body["image_generation"]["model"] == "openai/gpt-image-1"
assert image_body["image_generation"]["default_aspect_ratio"] == "16:9" assert image_body["image_generation"]["default_aspect_ratio"] == "16:9"
@@ -1623,9 +1256,9 @@ async def test_settings_api_returns_safe_subset_and_updates_whitelist(
assert image_provider_updated.status_code == 200 assert image_provider_updated.status_code == 200
assert image_provider_updated.json()["requires_restart"] is True assert image_provider_updated.json()["requires_restart"] is True
assert image_provider_updated.json()["restart_required_sections"] == [ assert image_provider_updated.json()["restart_required_sections"] == [
"browser",
"image", "image",
"runtime", "runtime",
"web",
] ]
assert "sk-or-next" not in image_provider_updated.text assert "sk-or-next" not in image_provider_updated.text
@@ -1647,8 +1280,8 @@ async def test_settings_api_returns_safe_subset_and_updates_whitelist(
assert saved.agents.defaults.model == "atomic_chat/test" assert saved.agents.defaults.model == "atomic_chat/test"
assert saved.agents.defaults.provider == "atomic_chat" assert saved.agents.defaults.provider == "atomic_chat"
assert saved.agents.defaults.model_preset == "fast-writing" assert saved.agents.defaults.model_preset == "fast-writing"
assert saved.model_presets["fast-writing"].label == "Codex" assert saved.model_presets["fast-writing"].label == "Fast writing"
assert saved.model_presets["fast-writing"].model == "openai/gpt-5.5" assert saved.model_presets["fast-writing"].model == "openai/gpt-4.1-mini"
assert saved.model_presets["fast-writing"].provider == "openai" assert saved.model_presets["fast-writing"].provider == "openai"
assert saved.agents.defaults.timezone == "Asia/Shanghai" assert saved.agents.defaults.timezone == "Asia/Shanghai"
assert saved.agents.defaults.bot_name == "Nano" assert saved.agents.defaults.bot_name == "Nano"
@@ -1663,7 +1296,6 @@ async def test_settings_api_returns_safe_subset_and_updates_whitelist(
assert saved.tools.web.search.max_results == 8 assert saved.tools.web.search.max_results == 8
assert saved.tools.web.search.timeout == 45 assert saved.tools.web.search.timeout == 45
assert saved.tools.web.fetch.use_jina_reader is False assert saved.tools.web.fetch.use_jina_reader is False
assert saved.tools.webui_allow_local_service_access is False
assert saved.tools.image_generation.enabled is True assert saved.tools.image_generation.enabled is True
assert saved.tools.image_generation.provider == "openrouter" assert saved.tools.image_generation.provider == "openrouter"
assert saved.tools.image_generation.model == "openai/gpt-image-1" assert saved.tools.image_generation.model == "openai/gpt-image-1"
@@ -1703,43 +1335,6 @@ async def test_commands_api_returns_slash_command_metadata(bus: MagicMock) -> No
await server_task await server_task
@pytest.mark.asyncio
async def test_bootstrap_exposes_native_surface(bus: MagicMock) -> None:
port = 29893
channel = WebSocketChannel(
{
"enabled": True,
"allowFrom": ["*"],
"host": "127.0.0.1",
"port": port,
"path": "/ws",
"tokenIssueSecret": "native-secret",
"websocketRequiresToken": True,
},
bus,
runtime_surface="native",
runtime_capabilities_overrides={"can_pick_folder": True},
)
server_task = asyncio.create_task(channel.start())
await asyncio.sleep(0.3)
try:
response = await _http_get(
f"http://127.0.0.1:{port}/webui/bootstrap",
headers={"X-Nanobot-Auth": "native-secret"},
)
assert response.status_code == 200
body = response.json()
assert body["runtime_surface"] == "native"
assert body["runtime_capabilities"]["can_pick_folder"] is True
assert body["runtime_capabilities"]["can_restart_engine"] is True
assert body["token"].startswith("nbwt_")
finally:
await channel.stop()
await server_task
def test_settings_payload_normalizes_camel_case_provider( def test_settings_payload_normalizes_camel_case_provider(
bus: MagicMock, bus: MagicMock,
monkeypatch, monkeypatch,
@@ -1770,44 +1365,6 @@ def test_settings_payload_exposes_api_type_only_for_openai(monkeypatch, tmp_path
assert "api_type" not in providers["custom"] assert "api_type" not in providers["custom"]
def test_settings_payload_reports_workspace_sandbox(monkeypatch, tmp_path) -> None:
config_path = tmp_path / "config.json"
config = Config()
config.tools.restrict_to_workspace = True
save_config(config, config_path)
monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path)
monkeypatch.setenv("NANOBOT_SANDBOX_ENFORCED", "macos_app_sandbox")
body = settings_payload()
sandbox = body["advanced"]["workspace_sandbox"]
assert sandbox["restrict_to_workspace"] is True
assert sandbox["level"] == "system"
assert sandbox["enforced"] is True
assert sandbox["provider"] == "macos_app_sandbox"
assert sandbox["provider_label"] == "macOS App Sandbox"
def test_settings_payload_includes_native_runtime_surface(monkeypatch, tmp_path) -> None:
config_path = tmp_path / "config.json"
save_config(Config(), config_path)
monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path)
body = settings_payload(
surface="native",
runtime_capability_overrides={"can_open_logs": True},
restart_required_sections=["runtime"],
)
assert body["surface"] == "native"
assert body["runtime_surface"] == "native"
assert body["runtime_capabilities"]["can_open_logs"] is True
assert body["runtime_capabilities"]["can_restart_engine"] is True
assert body["restart_behavior_by_section"]["runtime"] == "engineRestart"
assert body["requires_restart"] is True
assert body["apply_state"] == {"status": "pending", "sections": ["runtime"]}
def test_update_provider_settings_ignores_api_type_for_non_openai(monkeypatch, tmp_path) -> None: def test_update_provider_settings_ignores_api_type_for_non_openai(monkeypatch, tmp_path) -> None:
config_path = tmp_path / "config.json" config_path = tmp_path / "config.json"
save_config(Config(), config_path) save_config(Config(), config_path)
@@ -2114,8 +1671,6 @@ async def test_multiplex_new_chat_roundtrip(bus: MagicMock) -> None:
OutboundMessage(channel="websocket", chat_id=new_chat, content="ok") OutboundMessage(channel="websocket", chat_id=new_chat, content="ok")
) )
reply = json.loads(await client.recv()) reply = json.loads(await client.recv())
if reply["event"] == "session_updated":
reply = json.loads(await client.recv())
assert reply["event"] == "message" assert reply["event"] == "message"
assert reply["chat_id"] == new_chat assert reply["chat_id"] == new_chat
assert reply["text"] == "ok" assert reply["text"] == "ok"
@@ -2136,16 +1691,16 @@ async def test_multiplex_two_chats_isolated(bus: MagicMock) -> None:
await client.recv() # ready await client.recv() # ready
await client.send(json.dumps({"type": "new_chat"})) await client.send(json.dumps({"type": "new_chat"}))
chat_a = (await _recv_ws_event(client, "attached"))["chat_id"] chat_a = json.loads(await client.recv())["chat_id"]
await client.send(json.dumps({"type": "new_chat"})) await client.send(json.dumps({"type": "new_chat"}))
chat_b = (await _recv_ws_event(client, "attached"))["chat_id"] chat_b = json.loads(await client.recv())["chat_id"]
assert chat_a != chat_b assert chat_a != chat_b
# Push A → client sees A only (FIFO over the single WS). # Push A → client sees A only (FIFO over the single WS).
await channel.send( await channel.send(
OutboundMessage(channel="websocket", chat_id=chat_a, content="for-A") OutboundMessage(channel="websocket", chat_id=chat_a, content="for-A")
) )
msg_a = await _recv_ws_event(client, "message") msg_a = json.loads(await client.recv())
assert msg_a["chat_id"] == chat_a assert msg_a["chat_id"] == chat_a
assert msg_a["text"] == "for-A" assert msg_a["text"] == "for-A"
@@ -2153,7 +1708,7 @@ async def test_multiplex_two_chats_isolated(bus: MagicMock) -> None:
await channel.send( await channel.send(
OutboundMessage(channel="websocket", chat_id=chat_b, content="for-B") OutboundMessage(channel="websocket", chat_id=chat_b, content="for-B")
) )
msg_b = await _recv_ws_event(client, "message") msg_b = json.loads(await client.recv())
assert msg_b["chat_id"] == chat_b assert msg_b["chat_id"] == chat_b
assert msg_b["text"] == "for-B" assert msg_b["text"] == "for-B"
finally: finally:
@@ -2275,9 +1830,6 @@ def test_sessions_list_includes_active_run_started_at() -> None:
assert resp.status_code == 200 assert resp.status_code == 200
body = json.loads(resp.body.decode()) body = json.loads(resp.body.decode())
workspace_scope = body["sessions"][0].pop("workspace_scope")
assert workspace_scope["project_path"] == str(channel._workspace_path)
assert workspace_scope["access_mode"] in {"restricted", "full"}
assert body["sessions"] == [ assert body["sessions"] == [
{ {
"key": "websocket:chat-1", "key": "websocket:chat-1",
+3 -15
View File
@@ -95,7 +95,6 @@ async def test_bootstrap_returns_token_for_localhost(
body = resp.json() body = resp.json()
assert body["token"].startswith("nbwt_") assert body["token"].startswith("nbwt_")
assert body["ws_path"] == "/" assert body["ws_path"] == "/"
assert body["ws_url"] == "ws://127.0.0.1:29901/"
assert body["expires_in"] > 0 assert body["expires_in"] > 0
assert isinstance(body.get("model_name"), str) assert isinstance(body.get("model_name"), str)
finally: finally:
@@ -148,7 +147,7 @@ async def test_cli_apps_routes_require_token_and_return_payload(
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
) -> None: ) -> None:
monkeypatch.setattr( monkeypatch.setattr(
"nanobot.webui.settings_routes.cli_apps_payload", "nanobot.channels.websocket.cli_apps_payload",
lambda: { lambda: {
"apps": [ "apps": [
{ {
@@ -173,7 +172,7 @@ async def test_cli_apps_routes_require_token_and_return_payload(
}, },
) )
monkeypatch.setattr( monkeypatch.setattr(
"nanobot.webui.settings_routes.cli_apps_action", "nanobot.channels.websocket.cli_apps_action",
lambda action, query: { lambda action, query: {
"apps": [], "apps": [],
"installed_count": 1, "installed_count": 1,
@@ -280,7 +279,7 @@ async def test_mcp_presets_routes_require_token_and_return_payload(
return {"ok": True, "message": "MCP config reloaded.", "requires_restart": False} return {"ok": True, "message": "MCP config reloaded.", "requires_restart": False}
monkeypatch.setattr( monkeypatch.setattr(
"nanobot.webui.settings_routes.request_mcp_reload", "nanobot.channels.websocket.request_mcp_reload",
_hot_reload, _hot_reload,
) )
channel = _ch(bus, session_manager=_seed_session(tmp_path), port=29913) channel = _ch(bus, session_manager=_seed_session(tmp_path), port=29913)
@@ -735,17 +734,6 @@ def test_bootstrap_accepts_static_token_as_secret(bus: MagicMock) -> None:
assert body["token"].startswith("nbwt_") assert body["token"].startswith("nbwt_")
def test_bootstrap_ws_url_uses_forwarded_https_host(bus: MagicMock) -> None:
channel = _ch(bus, host="127.0.0.1", port=29931)
resp = channel._handle_bootstrap(
_LOCAL,
_FakeReq({"Host": "nanobot.example", "X-Forwarded-Proto": "https"}),
)
assert resp.status_code == 200
body = json.loads(resp.body)
assert body["ws_url"] == "wss://nanobot.example/"
def test_localhost_without_auth_is_valid(bus: MagicMock) -> None: def test_localhost_without_auth_is_valid(bus: MagicMock) -> None:
channel = _ch(bus, host="127.0.0.1") channel = _ch(bus, host="127.0.0.1")
resp = channel._handle_bootstrap(_LOCAL, _NO_HEADERS) resp = channel._handle_bootstrap(_LOCAL, _NO_HEADERS)
+11 -155
View File
@@ -21,10 +21,10 @@ from unittest.mock import AsyncMock, MagicMock, patch
import httpx import httpx
import pytest import pytest
from nanobot.channels.websocket import WebSocketChannel from nanobot.channels.websocket import (
from nanobot.webui.media_api import ( WebSocketChannel,
b64url_decode, _b64url_decode,
b64url_encode, _b64url_encode,
) )
from nanobot.session.manager import Session, SessionManager from nanobot.session.manager import Session, SessionManager
@@ -129,9 +129,9 @@ def test_sign_media_path_round_trips_via_hmac(
expected = hmac.new( expected = hmac.new(
channel._media_secret, payload.encode("ascii"), hashlib.sha256 channel._media_secret, payload.encode("ascii"), hashlib.sha256
).digest()[:16] ).digest()[:16]
assert b64url_decode(sig) == expected assert _b64url_decode(sig) == expected
# The payload decodes back to the *relative* path — no absolute-path leaks. # The payload decodes back to the *relative* path — no absolute-path leaks.
assert b64url_decode(payload).decode() == "a.png" assert _b64url_decode(payload).decode() == "a.png"
def test_local_markdown_image_is_staged_and_rewritten( def test_local_markdown_image_is_staged_and_rewritten(
@@ -155,28 +155,6 @@ def test_local_markdown_image_is_staged_and_rewritten(
assert staged[0].read_bytes() == _PNG_BYTES assert staged[0].read_bytes() == _PNG_BYTES
def test_local_markdown_video_is_staged_and_rewritten(
bus: MagicMock,
tmp_path: Path,
) -> None:
workspace = tmp_path / "workspace"
workspace.mkdir()
video_bytes = b"fake mp4"
(workspace / "nanobot-intro.mp4").write_bytes(video_bytes)
media = tmp_path / "media"
channel = _ch(bus, workspace_path=workspace, port=0)
with patch("nanobot.channels.websocket.get_media_dir", side_effect=_fake_media_dir(media)):
rewritten = channel._rewrite_local_markdown_images(
"The result:\n![nanobot-intro.mp4](nanobot-intro.mp4)"
)
assert "![nanobot-intro.mp4](/api/media/" in rewritten
staged = list((media / "websocket").iterdir())
assert len(staged) == 1
assert staged[0].read_bytes() == video_bytes
def test_local_markdown_image_rejects_workspace_escape( def test_local_markdown_image_rejects_workspace_escape(
bus: MagicMock, bus: MagicMock,
tmp_path: Path, tmp_path: Path,
@@ -227,103 +205,10 @@ async def test_media_route_serves_signed_file(
assert resp.headers["content-type"].startswith("image/png") assert resp.headers["content-type"].startswith("image/png")
# Immutable cache header lets the browser skip round-trips on replay. # Immutable cache header lets the browser skip round-trips on replay.
assert "immutable" in resp.headers.get("cache-control", "") assert "immutable" in resp.headers.get("cache-control", "")
# Video players rely on byte ranges; images get the header for consistency.
assert resp.headers.get("accept-ranges") == "bytes"
# nosniff keeps the browser from second-guessing our Content-Type. # nosniff keeps the browser from second-guessing our Content-Type.
assert resp.headers.get("x-content-type-options") == "nosniff" assert resp.headers.get("x-content-type-options") == "nosniff"
@pytest.mark.asyncio
async def test_media_route_serves_video_byte_ranges(
bus: MagicMock, tmp_path: Path
) -> None:
"""MP4 playback needs HTTP Range support for mid-stream reads and seeking."""
media = tmp_path / "media"
media.mkdir()
target = media / "clip.mp4"
target.write_bytes(b"0123456789")
channel = _ch(bus, port=29927)
with patch("nanobot.channels.websocket.get_media_dir", return_value=media):
url_path = channel._sign_media_path(target)
assert url_path is not None
server_task = asyncio.create_task(channel.start())
await asyncio.sleep(0.3)
try:
resp = await _http_get(
f"http://127.0.0.1:29927{url_path}",
headers={"Range": "bytes=2-5"},
)
finally:
await channel.stop()
await server_task
assert resp.status_code == 206
assert resp.content == b"2345"
assert resp.headers["content-type"].startswith("video/mp4")
assert resp.headers.get("accept-ranges") == "bytes"
assert resp.headers.get("content-range") == "bytes 2-5/10"
assert resp.headers.get("content-length") == "4"
@pytest.mark.asyncio
async def test_media_route_serves_suffix_video_byte_ranges(
bus: MagicMock, tmp_path: Path
) -> None:
media = tmp_path / "media"
media.mkdir()
target = media / "clip.mp4"
target.write_bytes(b"0123456789")
channel = _ch(bus, port=29928)
with patch("nanobot.channels.websocket.get_media_dir", return_value=media):
url_path = channel._sign_media_path(target)
assert url_path is not None
server_task = asyncio.create_task(channel.start())
await asyncio.sleep(0.3)
try:
resp = await _http_get(
f"http://127.0.0.1:29928{url_path}",
headers={"Range": "bytes=-3"},
)
finally:
await channel.stop()
await server_task
assert resp.status_code == 206
assert resp.content == b"789"
assert resp.headers.get("content-range") == "bytes 7-9/10"
@pytest.mark.asyncio
async def test_media_route_rejects_unsatisfiable_byte_range(
bus: MagicMock, tmp_path: Path
) -> None:
media = tmp_path / "media"
media.mkdir()
target = media / "clip.mp4"
target.write_bytes(b"0123456789")
channel = _ch(bus, port=29929)
with patch("nanobot.channels.websocket.get_media_dir", return_value=media):
url_path = channel._sign_media_path(target)
assert url_path is not None
server_task = asyncio.create_task(channel.start())
await asyncio.sleep(0.3)
try:
resp = await _http_get(
f"http://127.0.0.1:29929{url_path}",
headers={"Range": "bytes=100-200"},
)
finally:
await channel.stop()
await server_task
assert resp.status_code == 416
assert resp.headers.get("accept-ranges") == "bytes"
assert resp.headers.get("content-range") == "bytes */10"
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_media_route_rejects_bad_signature( async def test_media_route_rejects_bad_signature(
bus: MagicMock, tmp_path: Path bus: MagicMock, tmp_path: Path
@@ -346,7 +231,7 @@ async def test_media_route_rejects_bad_signature(
forged_mac = hmac.new( forged_mac = hmac.new(
b"\x00" * 32, payload.encode("ascii"), hashlib.sha256 b"\x00" * 32, payload.encode("ascii"), hashlib.sha256
).digest()[:16] ).digest()[:16]
forged = f"/api/media/{b64url_encode(forged_mac)}/{payload}" forged = f"/api/media/{_b64url_encode(forged_mac)}/{payload}"
server_task = asyncio.create_task(channel.start()) server_task = asyncio.create_task(channel.start())
await asyncio.sleep(0.3) await asyncio.sleep(0.3)
@@ -375,11 +260,11 @@ async def test_media_route_rejects_path_traversal_payload(
channel = _ch(bus, port=29922) channel = _ch(bus, port=29922)
# Hand-craft a traversal payload the legit signer would refuse to mint. # Hand-craft a traversal payload the legit signer would refuse to mint.
payload = b64url_encode(b"../secret.txt") payload = _b64url_encode(b"../secret.txt")
mac = hmac.new( mac = hmac.new(
channel._media_secret, payload.encode("ascii"), hashlib.sha256 channel._media_secret, payload.encode("ascii"), hashlib.sha256
).digest()[:16] ).digest()[:16]
url = f"/api/media/{b64url_encode(mac)}/{payload}" url = f"/api/media/{_b64url_encode(mac)}/{payload}"
with patch("nanobot.channels.websocket.get_media_dir", return_value=media): with patch("nanobot.channels.websocket.get_media_dir", return_value=media):
server_task = asyncio.create_task(channel.start()) server_task = asyncio.create_task(channel.start())
@@ -434,11 +319,11 @@ async def test_media_route_degrades_non_image_to_octet_stream(
channel = _ch(bus, port=29924) channel = _ch(bus, port=29924)
with patch("nanobot.channels.websocket.get_media_dir", return_value=media): with patch("nanobot.channels.websocket.get_media_dir", return_value=media):
payload = b64url_encode(b"scary.html") payload = _b64url_encode(b"scary.html")
mac = hmac.new( mac = hmac.new(
channel._media_secret, payload.encode("ascii"), hashlib.sha256 channel._media_secret, payload.encode("ascii"), hashlib.sha256
).digest()[:16] ).digest()[:16]
url = f"/api/media/{b64url_encode(mac)}/{payload}" url = f"/api/media/{_b64url_encode(mac)}/{payload}"
server_task = asyncio.create_task(channel.start()) server_task = asyncio.create_task(channel.start())
await asyncio.sleep(0.3) await asyncio.sleep(0.3)
try: try:
@@ -453,35 +338,6 @@ async def test_media_route_degrades_non_image_to_octet_stream(
assert resp.headers.get("x-content-type-options") == "nosniff" assert resp.headers.get("x-content-type-options") == "nosniff"
@pytest.mark.asyncio
async def test_media_route_serves_svg_with_strict_csp(
bus: MagicMock, tmp_path: Path
) -> None:
"""Generated SVG can preview as an image without becoming executable HTML."""
media = tmp_path / "media"
media.mkdir()
target = media / "chart.svg"
target.write_text("<svg xmlns='http://www.w3.org/2000/svg'><script>alert(1)</script></svg>")
channel = _ch(bus, port=29928)
with patch("nanobot.channels.websocket.get_media_dir", return_value=media):
url_path = channel._sign_media_path(target)
assert url_path is not None
server_task = asyncio.create_task(channel.start())
await asyncio.sleep(0.3)
try:
resp = await _http_get(f"http://127.0.0.1:29928{url_path}")
finally:
await channel.stop()
await server_task
assert resp.status_code == 200
assert resp.headers["content-type"].startswith("image/svg+xml")
assert resp.headers.get("x-content-type-options") == "nosniff"
assert "default-src 'none'" in resp.headers.get("content-security-policy", "")
assert "sandbox" in resp.headers.get("content-security-policy", "")
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# /api/sessions/<key>/messages: media_urls hydration on session read # /api/sessions/<key>/messages: media_urls hydration on session read
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
+13 -185
View File
@@ -469,28 +469,6 @@ def test_config_auto_detects_xiaomi_mimo_from_model_keyword():
assert config.get_api_base() == "https://api.xiaomimimo.com/v1" assert config.get_api_base() == "https://api.xiaomimimo.com/v1"
def test_config_explicit_minimax_anthropic_provider_uses_default_api_base():
config = Config.model_validate(
{
"agents": {
"defaults": {
"provider": "minimax_anthropic",
"model": "MiniMax-M2.7-highspeed",
}
},
"providers": {
"minimaxAnthropic": {
"apiKey": "test-key",
}
},
}
)
assert config.get_provider_name() == "minimax_anthropic"
assert config.get_api_key() == "test-key"
assert config.get_api_base() == "https://api.minimax.io/anthropic"
def test_config_auto_detects_ollama_from_local_api_base(): def test_config_auto_detects_ollama_from_local_api_base():
config = Config.model_validate( config = Config.model_validate(
{ {
@@ -1232,7 +1210,7 @@ def test_gateway_cron_evaluator_receives_scheduled_reminder_context(
monkeypatch.setattr("nanobot.cli.commands.AgentLoop", _FakeAgentLoop) monkeypatch.setattr("nanobot.cli.commands.AgentLoop", _FakeAgentLoop)
monkeypatch.setattr("nanobot.channels.manager.ChannelManager", _StopAfterCronSetup) monkeypatch.setattr("nanobot.channels.manager.ChannelManager", _StopAfterCronSetup)
monkeypatch.setattr( monkeypatch.setattr(
"nanobot.cli.commands.evaluate_response", "nanobot.utils.evaluator.evaluate_response",
_capture_evaluate_response, _capture_evaluate_response,
) )
@@ -1364,7 +1342,7 @@ def test_gateway_cron_job_suppresses_intermediate_progress(
monkeypatch.setattr("nanobot.cli.commands.AgentLoop", _FakeAgentLoop) monkeypatch.setattr("nanobot.cli.commands.AgentLoop", _FakeAgentLoop)
monkeypatch.setattr("nanobot.channels.manager.ChannelManager", _StopAfterCronSetup) monkeypatch.setattr("nanobot.channels.manager.ChannelManager", _StopAfterCronSetup)
monkeypatch.setattr( monkeypatch.setattr(
"nanobot.cli.commands.evaluate_response", "nanobot.utils.evaluator.evaluate_response",
_always_reject, _always_reject,
) )
@@ -1394,138 +1372,6 @@ def test_gateway_cron_job_suppresses_intermediate_progress(
bus.publish_outbound.assert_not_awaited() bus.publish_outbound.assert_not_awaited()
def test_gateway_heartbeat_fails_closed_and_suppresses_message_tool(
monkeypatch, tmp_path: Path
) -> None:
"""Heartbeat only delivers after an explicit positive evaluation, and
internal checks cannot bypass that gate with the proactive message tool."""
from nanobot.agent.tools.message import MessageTool
config_file = tmp_path / "instance" / "config.json"
config_file.parent.mkdir(parents=True)
config_file.write_text("{}")
config = Config()
config.agents.defaults.workspace = str(tmp_path / "config-workspace")
config.workspace_path.mkdir(parents=True)
(config.workspace_path / "HEARTBEAT.md").write_text(
"Check whether anything needs attention.",
encoding="utf-8",
)
bus = MagicMock()
bus.publish_outbound = AsyncMock()
seen: dict[str, object] = {}
class _FakeSession:
def retain_recent_legal_suffix(self, _keep: int) -> None:
seen["retained"] = True
class _FakeSessionManager:
def __init__(self, _workspace: Path) -> None:
self.session = _FakeSession()
def list_sessions(self) -> list[dict[str, object]]:
return [{"key": "lark:chat-1", "updated_at": "2026-05-30T00:00:00"}]
def get_or_create(self, key: str) -> _FakeSession:
seen["session_key"] = key
return self.session
def save(self, session: _FakeSession) -> None:
seen["saved"] = session
class _FakeCron:
def __init__(self, _store_path: Path) -> None:
self.on_job = None
seen["cron"] = self
def status(self) -> dict[str, int]:
return {"jobs": 0}
def register_system_job(self, job: CronJob) -> CronJob:
if job.name == "heartbeat":
seen["heartbeat_job"] = job
raise _StopGatewayError("stop")
return job
class _FakeDream:
model = None
max_batch_size = 0
max_iterations = 0
annotate_line_ages = False
async def run(self) -> None:
return None
class _FakeAgentLoop:
@classmethod
def from_config(cls, config, bus=None, **extra):
return cls(bus=bus, **extra)
def __init__(self, bus=None, **kwargs) -> None:
self.model = "test-model"
self.provider = object()
self.sessions = kwargs["session_manager"]
self.dream = _FakeDream()
self.tools = {
"message": MessageTool(send_callback=bus.publish_outbound),
}
async def process_direct(self, *_args, **_kwargs):
result = await self.tools["message"].execute(
content="All clear.",
channel="lark",
chat_id="chat-1",
)
seen["message_tool_result"] = result
return OutboundMessage(
channel="lark",
chat_id="chat-1",
content="All clear.",
)
async def close_mcp(self) -> None:
return None
def stop(self) -> None:
return None
class _FakeChannels:
enabled_channels = ["lark"]
async def _capture_evaluate(*_args, **kwargs) -> bool:
seen["default_notify"] = kwargs.get("default_notify")
return False
_patch_cli_command_runtime(
monkeypatch,
config,
message_bus=lambda: bus,
session_manager=_FakeSessionManager,
cron_service=_FakeCron,
)
monkeypatch.setattr("nanobot.cli.commands.AgentLoop", _FakeAgentLoop)
monkeypatch.setattr(
"nanobot.channels.manager.ChannelManager",
lambda *_args, **_kwargs: _FakeChannels(),
)
monkeypatch.setattr("nanobot.cli.commands.evaluate_response", _capture_evaluate)
result = runner.invoke(app, ["gateway", "--config", str(config_file)])
assert isinstance(result.exception, _StopGatewayError)
cron = seen["cron"]
response = asyncio.run(cron.on_job(seen["heartbeat_job"]))
assert response == "All clear."
assert seen["message_tool_result"] == "Message suppressed during internal check"
assert seen["default_notify"] is False
assert seen["session_key"] == "heartbeat"
assert seen["retained"] is True
bus.publish_outbound.assert_not_awaited()
def test_gateway_workspace_override_does_not_migrate_legacy_cron( def test_gateway_workspace_override_does_not_migrate_legacy_cron(
monkeypatch, tmp_path: Path monkeypatch, tmp_path: Path
) -> None: ) -> None:
@@ -1675,35 +1521,6 @@ def test_gateway_cli_port_overrides_configured_port(monkeypatch, tmp_path: Path)
assert "port 18792" in result.stdout assert "port 18792" in result.stdout
def test_configure_desktop_gateway_forces_local_websocket_only() -> None:
from nanobot.cli.commands import _configure_desktop_gateway
config = Config()
config.channels.__pydantic_extra__ = {
"telegram": {"enabled": True, "token": "x"},
"websocket": {"enabled": False, "port": 8765},
}
_configure_desktop_gateway(
config,
webui_port=29888,
webui_socket="/tmp/nanobot-test.sock",
token_issue_secret="secret",
)
extras = config.channels.__pydantic_extra__ or {}
assert config.gateway.host == "127.0.0.1"
assert config.gateway.port == 29888
assert config.gateway.heartbeat.enabled is False
assert extras["telegram"]["enabled"] is False
assert extras["websocket"]["enabled"] is True
assert extras["websocket"]["host"] == "127.0.0.1"
assert extras["websocket"]["port"] == 29888
assert extras["websocket"]["unix_socket_path"] == "/tmp/nanobot-test.sock"
assert extras["websocket"]["token_issue_secret"] == "secret"
assert extras["websocket"]["websocket_requires_token"] is True
def test_gateway_health_endpoint_binds_and_serves_expected_responses( def test_gateway_health_endpoint_binds_and_serves_expected_responses(
monkeypatch, tmp_path: Path monkeypatch, tmp_path: Path
) -> None: ) -> None:
@@ -1772,6 +1589,16 @@ def test_gateway_health_endpoint_binds_and_serves_expected_responses(
def register_system_job(self, _job) -> None: def register_system_job(self, _job) -> None:
return None return None
class _FakeHeartbeatService:
def __init__(self, **_kwargs) -> None:
return None
async def start(self) -> None:
return None
def stop(self) -> None:
return None
class _FakeServer: class _FakeServer:
async def __aenter__(self): async def __aenter__(self):
return self return self
@@ -1818,6 +1645,7 @@ def test_gateway_health_endpoint_binds_and_serves_expected_responses(
monkeypatch.setattr("nanobot.cli.commands.AgentLoop", _FakeAgentLoop) monkeypatch.setattr("nanobot.cli.commands.AgentLoop", _FakeAgentLoop)
monkeypatch.setattr("nanobot.channels.manager.ChannelManager", _FakeChannelManager) monkeypatch.setattr("nanobot.channels.manager.ChannelManager", _FakeChannelManager)
monkeypatch.setattr("nanobot.cron.service.CronService", _FakeCronService) monkeypatch.setattr("nanobot.cron.service.CronService", _FakeCronService)
monkeypatch.setattr("nanobot.heartbeat.service.HeartbeatService", _FakeHeartbeatService)
monkeypatch.setattr("asyncio.start_server", _fake_start_server) monkeypatch.setattr("asyncio.start_server", _fake_start_server)
result = runner.invoke(app, ["gateway", "--config", str(config_file)]) result = runner.invoke(app, ["gateway", "--config", str(config_file)])
-204
View File
@@ -121,7 +121,6 @@ def _seed_catalog(manager: CliAppManager) -> None:
} }
_write_cache(manager._cache_path("harness"), harness) _write_cache(manager._cache_path("harness"), harness)
_write_cache(manager._cache_path("public"), public) _write_cache(manager._cache_path("public"), public)
_write_cache(manager._cache_path("extensions"), {"meta": {}, "clis": []})
def test_payload_merges_catalog_and_marks_unsupported_installs(tmp_path: Path) -> None: def test_payload_merges_catalog_and_marks_unsupported_installs(tmp_path: Path) -> None:
@@ -144,8 +143,6 @@ def test_payload_merges_catalog_and_marks_unsupported_installs(tmp_path: Path) -
assert apps["gimp"]["install_supported"] is True assert apps["gimp"]["install_supported"] is True
assert apps["gimp"]["source"] == "harness+public" assert apps["gimp"]["source"] == "harness+public"
assert apps["gimp"]["description"] == "Public duplicate entry" assert apps["gimp"]["description"] == "Public duplicate entry"
assert apps["feishu"]["description"] == "Lark CLI"
assert apps["feishu"]["manifest"]["description"] == "Lark CLI"
assert apps["clibrowser"]["install_supported"] is False assert apps["clibrowser"]["install_supported"] is False
assert apps["jimeng"]["install_supported"] is False assert apps["jimeng"]["install_supported"] is False
assert apps["suno"]["install_supported"] is True assert apps["suno"]["install_supported"] is True
@@ -188,7 +185,6 @@ def test_payload_uses_anygen_official_domain_for_logo(tmp_path: Path) -> None:
], ],
}, },
) )
_write_cache(manager._cache_path("extensions"), {"meta": {}, "clis": []})
payload = manager.payload() payload = manager.payload()
@@ -197,79 +193,6 @@ def test_payload_uses_anygen_official_domain_for_logo(tmp_path: Path) -> None:
assert app["logo_url"] == "https://www.google.com/s2/favicons?domain=anygen.io&sz=64" assert app["logo_url"] == "https://www.google.com/s2/favicons?domain=anygen.io&sz=64"
def test_payload_includes_nanobot_extension_registry(tmp_path: Path) -> None:
manager = _manager(tmp_path)
_write_cache(manager._cache_path("harness"), {"meta": {"updated": "2026-04-16"}, "clis": []})
_write_cache(manager._cache_path("public"), {"meta": {"updated": "2026-04-18"}, "clis": []})
_write_cache(
manager._cache_path("extensions"),
{
"meta": {"updated": "2026-05-29"},
"clis": [
{
"name": "hyperframes",
"display_name": "HyperFrames",
"version": "latest",
"description": "HTML-to-MP4 motion graphics CLI",
"category": "video",
"package_manager": "npm",
"npm_package": "hyperframes",
"install_cmd": "npm install -g hyperframes",
"entry_point": "hyperframes",
"logo_url": "https://raw.githubusercontent.com/heygen-com/hyperframes/main/assets/logo.png",
"brand_color": "#111827",
"skill_md": "skills/hyperframes/SKILL.md",
}
],
},
)
payload = manager.payload()
assert payload["catalog_updated_at"] == "2026-05-29"
app = payload["apps"][0]
assert app["name"] == "hyperframes"
assert app["source"] == "extensions"
assert app["logo_url"] == "https://raw.githubusercontent.com/heygen-com/hyperframes/main/assets/logo.png"
assert app["brand_color"] == "#111827"
assert app["install_supported"] is True
assert app["manifest"]["source"] == "nanobot-extension"
assert app["manifest"]["trust"]["registry"] == "nanobot-extension"
def test_optional_extension_registry_failure_does_not_break_payload(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
_write_cache(
manager._cache_path("harness"),
{
"meta": {"updated": "2026-04-16"},
"clis": [
{
"name": "gimp",
"display_name": "GIMP",
"description": "Image editing",
"install_cmd": "pip install cli-anything-gimp",
"entry_point": "cli-anything-gimp",
}
],
},
)
_write_cache(manager._cache_path("public"), {"meta": {"updated": "2026-04-18"}, "clis": []})
def fail_get(*args, **kwargs):
raise RuntimeError("network unavailable")
monkeypatch.setattr("nanobot.apps.cli.service.httpx.get", fail_get)
payload = manager.payload()
assert payload["catalog_updated_at"] == "2026-04-18"
assert [app["name"] for app in payload["apps"]] == ["gimp"]
def test_install_dispatches_safe_pip_and_installs_skill( def test_install_dispatches_safe_pip_and_installs_skill(
tmp_path: Path, tmp_path: Path,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
@@ -302,101 +225,6 @@ def test_install_dispatches_safe_pip_and_installs_skill(
assert 'run_cli_app` tool with `name="gimp"' in skill.read_text(encoding="utf-8") assert 'run_cli_app` tool with `name="gimp"' in skill.read_text(encoding="utf-8")
def test_install_records_available_cli_without_reinstalling(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
_seed_catalog(manager)
resolved = tmp_path / "bin" / "lark-cli"
resolved.parent.mkdir()
resolved.write_text("#!/bin/sh\n", encoding="utf-8")
def fail_run(argv: list[str], *, timeout: int) -> subprocess.CompletedProcess[str]:
raise AssertionError(f"unexpected install command: {argv}")
monkeypatch.setattr(manager, "_run_argv", fail_run)
monkeypatch.setattr(
"nanobot.apps.cli.service.shutil.which",
lambda command: str(resolved) if command == "lark-cli" else None,
)
payload = manager.install("feishu")
assert payload["last_action"]["ok"] is True
assert payload["last_action"]["installed"] is True
assert "entry_point_available" in payload["last_action"]["verification"]
installed = json.loads(manager.installed_path.read_text(encoding="utf-8"))["apps"]
assert installed["feishu"]["entry_point_path"] == str(resolved)
skill = manager.workspace / "skills" / "cli-app-feishu" / "SKILL.md"
assert skill.is_file()
assert 'run_cli_app` tool with `name="feishu"' in skill.read_text(encoding="utf-8")
def test_install_recovers_stale_npm_global_directory(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
_write_cache(manager._cache_path("harness"), {"meta": {"updated": "2026-04-16"}, "clis": []})
_write_cache(manager._cache_path("public"), {"meta": {"updated": "2026-04-18"}, "clis": []})
_write_cache(
manager._cache_path("extensions"),
{
"meta": {"updated": "2026-05-29"},
"clis": [
{
"name": "hyperframes",
"display_name": "HyperFrames",
"package_manager": "npm",
"npm_package": "hyperframes",
"install_cmd": "npm install -g hyperframes",
"entry_point": "hyperframes",
"skill_md": "skills/hyperframes/SKILL.md",
}
],
},
)
npm = str(tmp_path / "bin" / "npm")
global_root = tmp_path / "global"
stale_package = global_root / "hyperframes"
stale_temp = global_root / ".hyperframes-broken"
stale_package.mkdir(parents=True)
stale_temp.mkdir()
install_attempts = 0
def fake_run(argv: list[str], *, timeout: int) -> subprocess.CompletedProcess[str]:
nonlocal install_attempts
if argv == [npm, "root", "-g"]:
return subprocess.CompletedProcess(argv, 0, stdout=str(global_root), stderr="")
if argv == [npm, "install", "-g", "hyperframes"]:
install_attempts += 1
if install_attempts == 1:
return subprocess.CompletedProcess(
argv,
1,
stdout="",
stderr="npm error ENOTEMPTY\nnpm error syscall rename",
)
return subprocess.CompletedProcess(argv, 0, stdout="ok", stderr="")
raise AssertionError(f"unexpected command: {argv}")
monkeypatch.setattr(manager, "_run_argv", fake_run)
monkeypatch.setattr(
"nanobot.apps.cli.service.shutil.which",
lambda command: npm if command == "npm" else None,
)
payload = manager.install("hyperframes")
assert install_attempts == 2
assert not stale_package.exists()
assert not stale_temp.exists()
assert payload["last_action"]["ok"] is True
installed = json.loads(manager.installed_path.read_text(encoding="utf-8"))["apps"]
assert installed["hyperframes"]["strategy"] == "npm"
def test_install_records_entry_point_path_and_pip_distribution( def test_install_records_entry_point_path_and_pip_distribution(
tmp_path: Path, tmp_path: Path,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
@@ -503,38 +331,6 @@ def test_fetch_skill_content_allows_cli_anything_raw_skill_url(
] ]
def test_fetch_skill_content_uses_extension_raw_base_for_relative_skills(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
manager = _manager(tmp_path)
seen: list[str] = []
class Response:
text = "---\nname: hyperframes\ndescription: HyperFrames\n---\n# HyperFrames\n"
@staticmethod
def raise_for_status() -> None:
return None
def fake_get(url: str, **kwargs):
seen.append(url)
return Response()
monkeypatch.setattr("nanobot.apps.cli.service.httpx.get", fake_get)
content = manager._fetch_skill_content({
"name": "hyperframes",
"skill_md": "skills/hyperframes/SKILL.md",
"_raw_base": "https://raw.githubusercontent.com/Re-bin/nanobot-extension/main",
})
assert content and "# HyperFrames" in content
assert seen == [
"https://raw.githubusercontent.com/Re-bin/nanobot-extension/main/skills/hyperframes/SKILL.md"
]
def test_uninstall_removes_installed_state_and_generated_skill( def test_uninstall_removes_installed_state_and_generated_skill(
tmp_path: Path, tmp_path: Path,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
-2
View File
@@ -41,7 +41,6 @@ def test_run_cli_app_uses_installed_registry_app(
} }
_write_cache(data_dir / "harness_registry_cache.json", registry) _write_cache(data_dir / "harness_registry_cache.json", registry)
_write_cache(data_dir / "public_registry_cache.json", {"meta": {}, "clis": []}) _write_cache(data_dir / "public_registry_cache.json", {"meta": {}, "clis": []})
_write_cache(data_dir / "extensions_registry_cache.json", {"meta": {}, "clis": []})
CliAppManager(workspace=workspace, data_dir=data_dir)._save_installed( CliAppManager(workspace=workspace, data_dir=data_dir)._save_installed(
{"gimp": {"entry_point": "cli-anything-gimp"}} {"gimp": {"entry_point": "cli-anything-gimp"}}
) )
@@ -103,7 +102,6 @@ def test_run_cli_app_rejects_uninstalled_app(tmp_path: Path, monkeypatch) -> Non
} }
_write_cache(data_dir / "harness_registry_cache.json", registry) _write_cache(data_dir / "harness_registry_cache.json", registry)
_write_cache(data_dir / "public_registry_cache.json", {"meta": {}, "clis": []}) _write_cache(data_dir / "public_registry_cache.json", {"meta": {}, "clis": []})
_write_cache(data_dir / "extensions_registry_cache.json", {"meta": {}, "clis": []})
monkeypatch.setattr("nanobot.apps.cli.service.get_runtime_subdir", lambda _name: data_dir) monkeypatch.setattr("nanobot.apps.cli.service.get_runtime_subdir", lambda _name: data_dir)
tool = CliAppsTool(workspace=workspace, restrict_to_workspace=True) tool = CliAppsTool(workspace=workspace, restrict_to_workspace=True)
+46
View File
@@ -113,6 +113,52 @@ async def test_dream_restore_lists_versions_with_next_steps() -> None:
assert "Restore a version with `/dream-restore <sha>`." in out.content assert "Restore a version with `/dream-restore <sha>`." in out.content
@pytest.mark.asyncio
async def test_dream_log_shows_summary_and_analysis() -> None:
commit = CommitInfo(
sha="abcd1234",
message="dream: 2026-04-04, 2 change(s)\n\n[ADD] fact A →USER\n[REMOVE] old fact",
timestamp="2026-04-04 12:00",
)
diff = (
"diff --git a/SOUL.md b/SOUL.md\n"
"--- a/SOUL.md\n"
"+++ b/SOUL.md\n"
"@@ -1 +1 @@\n"
"-old\n"
"+new\n"
)
git = _FakeGit(commits=[commit], diff_map={commit.sha: (commit, diff)})
out = await cmd_dream_log(_make_ctx("/dream-log", git))
assert "## Dream Update" in out.content
assert "- Summary: dream: 2026-04-04, 2 change(s)" in out.content
assert "### Analysis" in out.content
assert "[ADD] fact A →USER" in out.content
assert "[REMOVE] old fact" in out.content
@pytest.mark.asyncio
async def test_dream_log_with_empty_commit_message() -> None:
commit = CommitInfo(sha="abcd1234", message="", timestamp="2026-04-04 12:00")
diff = (
"diff --git a/SOUL.md b/SOUL.md\n"
"--- a/SOUL.md\n"
"+++ b/SOUL.md\n"
"@@ -1 +1 @@\n"
"-old\n"
"+new\n"
)
git = _FakeGit(commits=[commit], diff_map={commit.sha: (commit, diff)})
out = await cmd_dream_log(_make_ctx("/dream-log", git))
assert "## Dream Update" in out.content
assert "- Summary:" not in out.content
assert "### Analysis" not in out.content
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_dream_restore_success_mentions_files_and_followup() -> None: async def test_dream_restore_success_mentions_files_and_followup() -> None:
commit = CommitInfo(sha="abcd1234", message="dream: latest", timestamp="2026-04-04 12:00") commit = CommitInfo(sha="abcd1234", message="dream: latest", timestamp="2026-04-04 12:00")
-21
View File
@@ -223,24 +223,3 @@ def test_load_config_resets_ssrf_whitelist_when_next_config_is_empty(tmp_path) -
with patch("nanobot.security.network.socket.getaddrinfo", _fake_resolve("ts.local", ["100.100.1.1"])): with patch("nanobot.security.network.socket.getaddrinfo", _fake_resolve("ts.local", ["100.100.1.1"])):
ok, _ = validate_url_target("http://ts.local/api") ok, _ = validate_url_target("http://ts.local/api")
assert not ok assert not ok
def test_load_config_defaults_local_service_access_to_enabled(tmp_path) -> None:
config_path = tmp_path / "config.json"
config_path.write_text(json.dumps({"tools": {}}), encoding="utf-8")
config = load_config(config_path)
assert config.tools.webui_allow_local_service_access is True
def test_load_config_accepts_legacy_local_preview_access(tmp_path) -> None:
config_path = tmp_path / "config.json"
config_path.write_text(
json.dumps({"tools": {"allowLocalPreviewAccess": False}}),
encoding="utf-8",
)
config = load_config(config_path)
assert config.tools.webui_allow_local_service_access is False

Some files were not shown because too many files have changed in this diff Show More