mirror of
https://github.com/HKUDS/nanobot.git
synced 2026-08-08 05:18:49 +03:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5922c4ebea |
@@ -31,6 +31,10 @@ Tool descriptions, skills, and replayed session history also shape model behavio
|
|||||||
|
|
||||||
Anything written into memory, session history, or prompt inputs can be replayed into future LLM calls. Metadata such as timestamps, local media paths, tool-call echoes, and raw fallback dumps must be bounded and sanitized before they become examples for the model to imitate.
|
Anything written into memory, session history, or prompt inputs can be replayed into future LLM calls. Metadata such as timestamps, local media paths, tool-call echoes, and raw fallback dumps must be bounded and sanitized before they become examples for the model to imitate.
|
||||||
|
|
||||||
|
## Heartbeat Virtual Tool Call
|
||||||
|
|
||||||
|
The heartbeat service (`heartbeat/service.py`) does not parse free-text LLM output. Instead, it injects a virtual `heartbeat` tool with `action: skip | run` into the conversation. Phase 1 is a structured decision; Phase 2 executes only on `run`. When adding new periodic background checks, follow this virtual-tool-call pattern rather than string matching.
|
||||||
|
|
||||||
## Skills as Extension Point
|
## Skills as Extension Point
|
||||||
|
|
||||||
Built-in skills live in `nanobot/skills/` (markdown + YAML frontmatter format). Agent capabilities that are "know-how" rather than code should be added as skills, not hardcoded into the agent loop. External skills can be published to and installed from ClawHub.
|
Built-in skills live in `nanobot/skills/` (markdown + YAML frontmatter format). Agent capabilities that are "know-how" rather than code should be added as skills, not hardcoded into the agent loop. External skills can be published to and installed from ClawHub.
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ jobs:
|
|||||||
strategy:
|
strategy:
|
||||||
fail-fast: false
|
fail-fast: false
|
||||||
matrix:
|
matrix:
|
||||||
os: ${{ fromJSON('["ubuntu-latest","windows-latest"]') }}
|
os: ${{ github.event_name == 'pull_request' && fromJSON('["ubuntu-latest"]') || fromJSON('["ubuntu-latest","windows-latest"]') }}
|
||||||
# CI concentrates on newer runtimes (3.11/3.12 still supported per pyproject requires-python).
|
# CI concentrates on newer runtimes (3.11/3.12 still supported per pyproject requires-python).
|
||||||
python-version: ${{ fromJSON('["3.13","3.14"]') }}
|
python-version: ${{ fromJSON('["3.13","3.14"]') }}
|
||||||
|
|
||||||
|
|||||||
@@ -6,8 +6,6 @@
|
|||||||
.env
|
.env
|
||||||
.web
|
.web
|
||||||
.orion
|
.orion
|
||||||
nanobot-desktop/
|
|
||||||
desktop/
|
|
||||||
|
|
||||||
# Claude / AI assistant artifacts
|
# Claude / AI assistant artifacts
|
||||||
docs/superpowers/
|
docs/superpowers/
|
||||||
@@ -100,4 +98,3 @@ tmp/
|
|||||||
temp/
|
temp/
|
||||||
*.tmp
|
*.tmp
|
||||||
exp/
|
exp/
|
||||||
.playwright-mcp/
|
|
||||||
|
|||||||
@@ -47,7 +47,7 @@ Messages flow through an async `MessageBus` (`nanobot/bus/queue.py`) that decoup
|
|||||||
- **WebUI** (`webui/`): Vite-based React SPA that talks to the gateway over a WebSocket multiplex protocol. The dev server proxies `/api`, `/webui`, `/auth`, and WebSocket traffic to the gateway.
|
- **WebUI** (`webui/`): Vite-based React SPA that talks to the gateway over a WebSocket multiplex protocol. The dev server proxies `/api`, `/webui`, `/auth`, and WebSocket traffic to the gateway.
|
||||||
- **API Server** (`nanobot/api/server.py`): OpenAI-compatible HTTP API (`/v1/chat/completions`, `/v1/models`) for programmatic access.
|
- **API Server** (`nanobot/api/server.py`): OpenAI-compatible HTTP API (`/v1/chat/completions`, `/v1/models`) for programmatic access.
|
||||||
- **Command Router** (`nanobot/command/`): Slash command routing and built-in command handlers.
|
- **Command Router** (`nanobot/command/`): Slash command routing and built-in command handlers.
|
||||||
- **Heartbeat** (`nanobot/templates/HEARTBEAT.md`): Periodic task list checked via `cron` jobs (legacy dedicated service removed).
|
- **Heartbeat** (`nanobot/heartbeat/`): Periodic agent wake-up service for scheduled task checking.
|
||||||
- **Pairing** (`nanobot/pairing/`): DM sender approval store with persistent pairing codes per channel.
|
- **Pairing** (`nanobot/pairing/`): DM sender approval store with persistent pairing codes per channel.
|
||||||
- **Skills** (`nanobot/skills/`): Built-in skill definitions (long-goal, cron, github, image-generation, etc.) loaded into agent context.
|
- **Skills** (`nanobot/skills/`): Built-in skill definitions (long-goal, cron, github, image-generation, etc.) loaded into agent context.
|
||||||
- **Security** (`nanobot/security/`): PTH file guard and other security measures activated at CLI entry.
|
- **Security** (`nanobot/security/`): PTH file guard and other security measures activated at CLI entry.
|
||||||
|
|||||||
@@ -12,8 +12,6 @@ software together: with care, clarity, and respect for the next person reading t
|
|||||||
|
|
||||||
## Maintainers
|
## Maintainers
|
||||||
|
|
||||||
Maintainers are community stewards who help review, organize, and maintain the project. The list below describes each maintainer's current open-source project responsibilities.
|
|
||||||
|
|
||||||
| Maintainer | Focus |
|
| Maintainer | Focus |
|
||||||
|------------|-------|
|
|------------|-------|
|
||||||
| [@re-bin](https://github.com/re-bin) | Project lead, `main` branch |
|
| [@re-bin](https://github.com/re-bin) | Project lead, `main` branch |
|
||||||
|
|||||||
@@ -1,18 +1,6 @@
|
|||||||

|

|
||||||
|
|
||||||
<div align="center">
|
<div align="center">
|
||||||
<p>
|
|
||||||
<a href="https://nanobot.wiki/docs/latest/getting-started/nanobot-overview">English</a> |
|
|
||||||
<a href="https://nanobot.wiki/cn/docs/latest/getting-started/nanobot-overview">简体中文</a> |
|
|
||||||
<a href="https://nanobot.wiki/zh-Hant/docs/latest/getting-started/nanobot-overview">繁體中文</a> |
|
|
||||||
<a href="https://nanobot.wiki/es/docs/latest/getting-started/nanobot-overview">Español</a> |
|
|
||||||
<a href="https://nanobot.wiki/fr/docs/latest/getting-started/nanobot-overview">Français</a> |
|
|
||||||
<a href="https://nanobot.wiki/id/docs/latest/getting-started/nanobot-overview">Bahasa Indonesia</a> |
|
|
||||||
<a href="https://nanobot.wiki/ja/docs/latest/getting-started/nanobot-overview">日本語</a> |
|
|
||||||
<a href="https://nanobot.wiki/ko/docs/latest/getting-started/nanobot-overview">한국어</a> |
|
|
||||||
<a href="https://nanobot.wiki/ru/docs/latest/getting-started/nanobot-overview">Русский</a> |
|
|
||||||
<a href="https://nanobot.wiki/vi/docs/latest/getting-started/nanobot-overview">Tiếng Việt</a>
|
|
||||||
</p>
|
|
||||||
<p>
|
<p>
|
||||||
<a href="https://pypi.org/project/nanobot-ai/"><img src="https://img.shields.io/pypi/v/nanobot-ai" alt="PyPI"></a>
|
<a href="https://pypi.org/project/nanobot-ai/"><img src="https://img.shields.io/pypi/v/nanobot-ai" alt="PyPI"></a>
|
||||||
<a href="https://pepy.tech/project/nanobot-ai"><img src="https://static.pepy.tech/badge/nanobot-ai" alt="Downloads"></a>
|
<a href="https://pepy.tech/project/nanobot-ai"><img src="https://static.pepy.tech/badge/nanobot-ai" alt="Downloads"></a>
|
||||||
@@ -73,7 +61,7 @@
|
|||||||
- **2026-04-13** 🛡️ Agent turn hardened — user messages persisted early, auto-compact skips active tasks.
|
- **2026-04-13** 🛡️ Agent turn hardened — user messages persisted early, auto-compact skips active tasks.
|
||||||
- **2026-04-12** 🔒 Lark global domain support, Dream learns discovered skills, shell sandbox tightened.
|
- **2026-04-12** 🔒 Lark global domain support, Dream learns discovered skills, shell sandbox tightened.
|
||||||
- **2026-04-11** ⚡ Context compact shrinks sessions on the fly; Kagi web search; QQ & WeCom full media.
|
- **2026-04-11** ⚡ Context compact shrinks sessions on the fly; Kagi web search; QQ & WeCom full media.
|
||||||
- **2026-04-10** 📓 Multiple MCP servers, Feishu streaming & done-emoji.
|
- **2026-04-10** 📓 Notebook editing tool, multiple MCP servers, Feishu streaming & done-emoji.
|
||||||
- **2026-04-09** 🔌 WebSocket channel, unified cross-channel session, `disabled_skills` config.
|
- **2026-04-09** 🔌 WebSocket channel, unified cross-channel session, `disabled_skills` config.
|
||||||
- **2026-04-08** 📤 API file uploads, OpenAI reasoning auto-routing with Responses fallback.
|
- **2026-04-08** 📤 API file uploads, OpenAI reasoning auto-routing with Responses fallback.
|
||||||
- **2026-04-07** 🧠 Anthropic adaptive thinking, MCP resources & prompts exposed as tools.
|
- **2026-04-07** 🧠 Anthropic adaptive thinking, MCP resources & prompts exposed as tools.
|
||||||
|
|||||||
+3
-1
@@ -46,15 +46,17 @@ core_agent=$(count_top_level_py_lines "nanobot/agent")
|
|||||||
core_bus=$(count_top_level_py_lines "nanobot/bus")
|
core_bus=$(count_top_level_py_lines "nanobot/bus")
|
||||||
core_config=$(count_top_level_py_lines "nanobot/config")
|
core_config=$(count_top_level_py_lines "nanobot/config")
|
||||||
core_cron=$(count_top_level_py_lines "nanobot/cron")
|
core_cron=$(count_top_level_py_lines "nanobot/cron")
|
||||||
|
core_heartbeat=$(count_top_level_py_lines "nanobot/heartbeat")
|
||||||
core_session=$(count_top_level_py_lines "nanobot/session")
|
core_session=$(count_top_level_py_lines "nanobot/session")
|
||||||
|
|
||||||
print_row "agent/" "$core_agent"
|
print_row "agent/" "$core_agent"
|
||||||
print_row "bus/" "$core_bus"
|
print_row "bus/" "$core_bus"
|
||||||
print_row "config/" "$core_config"
|
print_row "config/" "$core_config"
|
||||||
print_row "cron/" "$core_cron"
|
print_row "cron/" "$core_cron"
|
||||||
|
print_row "heartbeat/" "$core_heartbeat"
|
||||||
print_row "session/" "$core_session"
|
print_row "session/" "$core_session"
|
||||||
|
|
||||||
core_total=$((core_agent + core_bus + core_config + core_cron + core_session))
|
core_total=$((core_agent + core_bus + core_config + core_cron + core_heartbeat + core_session))
|
||||||
|
|
||||||
echo ""
|
echo ""
|
||||||
echo "Separate buckets"
|
echo "Separate buckets"
|
||||||
|
|||||||
@@ -17,7 +17,6 @@ Connect nanobot to your favorite chat platform. Want to build your own? See the
|
|||||||
| **Wecom** | Bot ID + Bot Secret |
|
| **Wecom** | Bot ID + Bot Secret |
|
||||||
| **Microsoft Teams** | App ID + App Password + public HTTPS endpoint |
|
| **Microsoft Teams** | App ID + App Password + public HTTPS endpoint |
|
||||||
| **Mochat** | Claw token (auto-setup available) |
|
| **Mochat** | Claw token (auto-setup available) |
|
||||||
| **Signal** | signal-cli daemon + phone number |
|
|
||||||
|
|
||||||
<details>
|
<details>
|
||||||
<summary><b>Telegram</b> (Recommended)</summary>
|
<summary><b>Telegram</b> (Recommended)</summary>
|
||||||
@@ -51,43 +50,6 @@ Connect nanobot to your favorite chat platform. Want to build your own? See the
|
|||||||
nanobot gateway
|
nanobot gateway
|
||||||
```
|
```
|
||||||
|
|
||||||
**Webhook mode (optional)**
|
|
||||||
|
|
||||||
Telegram uses long polling by default. To receive updates through a webhook, expose
|
|
||||||
a public HTTPS URL that forwards to nanobot's local listener and set `mode` to
|
|
||||||
`webhook`:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"channels": {
|
|
||||||
"telegram": {
|
|
||||||
"enabled": true,
|
|
||||||
"token": "YOUR_BOT_TOKEN",
|
|
||||||
"mode": "webhook",
|
|
||||||
"webhookUrl": "https://example.com/telegram",
|
|
||||||
"webhookListenHost": "127.0.0.1",
|
|
||||||
"webhookListenPort": 8081,
|
|
||||||
"webhookPath": "/telegram",
|
|
||||||
"webhookSecretToken": "CHANGE_ME_RANDOM_SECRET",
|
|
||||||
"webhookMaxConnections": 4,
|
|
||||||
"allowFrom": ["YOUR_USER_ID"]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
> `webhookSecretToken` is required in webhook mode. Do not expose the local
|
|
||||||
> webhook listener directly to the public internet without a reverse proxy or
|
|
||||||
> tunnel in front of it. TLS/Host policy is handled by your proxy; nanobot only
|
|
||||||
> listens on `webhookListenHost:webhookListenPort` and validates Telegram's
|
|
||||||
> webhook secret token. `webhookMaxConnections` defaults to `4`; nanobot
|
|
||||||
> still serializes Telegram updates per conversation before forwarding them to
|
|
||||||
> the agent.
|
|
||||||
>
|
|
||||||
> `webhookUrl` is the public HTTPS URL registered with Telegram.
|
|
||||||
> `webhookPath` is the local path nanobot listens on. They often use the same
|
|
||||||
> path, but may differ when a reverse proxy or tunnel rewrites the request path.
|
|
||||||
|
|
||||||
</details>
|
</details>
|
||||||
|
|
||||||
<details>
|
<details>
|
||||||
@@ -707,69 +669,3 @@ nanobot gateway
|
|||||||
```
|
```
|
||||||
|
|
||||||
</details>
|
</details>
|
||||||
|
|
||||||
<details>
|
|
||||||
<summary><b>Signal</b></summary>
|
|
||||||
|
|
||||||
Uses **signal-cli** daemon in HTTP mode — receive messages via SSE, send via JSON-RPC.
|
|
||||||
|
|
||||||
**1. Install signal-cli**
|
|
||||||
|
|
||||||
Install [signal-cli](https://github.com/AsamK/signal-cli) and register a phone number:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
signal-cli -u +1234567890 register
|
|
||||||
signal-cli -u +1234567890 verify <CODE>
|
|
||||||
```
|
|
||||||
|
|
||||||
Start the daemon:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
signal-cli -a +1234567890 daemon --http localhost:8080
|
|
||||||
```
|
|
||||||
|
|
||||||
**2. Configure**
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"channels": {
|
|
||||||
"signal": {
|
|
||||||
"enabled": true,
|
|
||||||
"phoneNumber": "+1234567890",
|
|
||||||
"daemonHost": "localhost",
|
|
||||||
"daemonPort": 8080,
|
|
||||||
"dm": {
|
|
||||||
"enabled": true,
|
|
||||||
"policy": "open"
|
|
||||||
},
|
|
||||||
"group": {
|
|
||||||
"enabled": true,
|
|
||||||
"policy": "open",
|
|
||||||
"requireMention": true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
> - `phoneNumber`: Your registered Signal phone number.
|
|
||||||
> - `daemonHost` / `daemonPort`: Where signal-cli daemon is listening (default `localhost:8080`).
|
|
||||||
> - `dm.policy`: `"open"` (anyone can DM) or `"allowlist"` (only listed numbers/UUIDs). When `"allowlist"`, unlisted DM senders receive a pairing code.
|
|
||||||
> - `dm.allowFrom`: List of allowed phone numbers or UUIDs (used when policy is `"allowlist"`).
|
|
||||||
> - `group.policy`: `"open"` (all groups) or `"allowlist"` (only listed group IDs).
|
|
||||||
> - `group.requireMention`: When `true` (default), the bot only responds in groups when @mentioned.
|
|
||||||
> - `group.allowFrom`: List of allowed group IDs (used when group policy is `"allowlist"`).
|
|
||||||
> - `attachmentsDir`: Override the directory where signal-cli stores inbound attachments. Defaults to `~/.local/share/signal-cli/attachments` (the Linux default). Set this if signal-cli runs with a custom `XDG_DATA_HOME` or on macOS/Windows.
|
|
||||||
> - `groupMessageBufferSize`: Number of recent group messages kept for context (default `20`, must be > 0).
|
|
||||||
|
|
||||||
**3. Run**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
nanobot gateway
|
|
||||||
```
|
|
||||||
|
|
||||||
> [!TIP]
|
|
||||||
> The channel automatically reconnects to the signal-cli daemon with exponential backoff if the connection drops.
|
|
||||||
> Markdown in bot replies is automatically converted to Signal text styles (bold, italic, code, etc.).
|
|
||||||
|
|
||||||
</details>
|
|
||||||
|
|||||||
+3
-108
@@ -126,10 +126,8 @@ ANTHROPIC_API_KEY="$(bw get password api/anthropic)" nanobot agent
|
|||||||
> - **VolcEngine / BytePlus Coding Plan**: Use dedicated providers `volcengineCodingPlan` or `byteplusCodingPlan` instead of the pay-per-use `volcengine` / `byteplus` providers.
|
> - **VolcEngine / BytePlus Coding Plan**: Use dedicated providers `volcengineCodingPlan` or `byteplusCodingPlan` instead of the pay-per-use `volcengine` / `byteplus` providers.
|
||||||
> - **Zhipu Coding Plan**: If you're on Zhipu's coding plan, set `"apiBase": "https://open.bigmodel.cn/api/coding/paas/v4"` in your zhipu provider config.
|
> - **Zhipu Coding Plan**: If you're on Zhipu's coding plan, set `"apiBase": "https://open.bigmodel.cn/api/coding/paas/v4"` in your zhipu provider config.
|
||||||
> - **Alibaba Cloud BaiLian**: If you're using Alibaba Cloud BaiLian's OpenAI-compatible endpoint, set `"apiBase": "https://dashscope.aliyuncs.com/compatible-mode/v1"` in your dashscope provider config.
|
> - **Alibaba Cloud BaiLian**: If you're using Alibaba Cloud BaiLian's OpenAI-compatible endpoint, set `"apiBase": "https://dashscope.aliyuncs.com/compatible-mode/v1"` in your dashscope provider config.
|
||||||
> - **StepFun Step Plan**: If you're on StepFun's Step Plan subscription, set `"apiBase": "https://api.stepfun.com/step_plan/v1"` in your stepfun provider config. Supported models include `step-3.5-flash`, `step-3.5-flash-2603`, and `step-router-v1`.
|
|
||||||
> - **Step Fun (Mainland China)**: If your API key is from Step Fun's mainland China platform (stepfun.com), set `"apiBase": "https://api.stepfun.com/v1"` in your stepfun provider config.
|
> - **Step Fun (Mainland China)**: If your API key is from Step Fun's mainland China platform (stepfun.com), set `"apiBase": "https://api.stepfun.com/v1"` in your stepfun provider config.
|
||||||
> - **Xiaomi MiMo thinking mode**: MiMo models (e.g. `mimo-v2.5-pro`) default to enabled thinking. Use `agents.defaults.reasoningEffort: "none"` to disable it, or `"low"` / `"medium"` / `"high"` to keep it on. Omitting the field preserves the provider's per-model default.
|
> - **Xiaomi MiMo thinking mode**: MiMo models (e.g. `mimo-v2.5-pro`) default to enabled thinking. Use `agents.defaults.reasoningEffort: "none"` to disable it, or `"low"` / `"medium"` / `"high"` to keep it on. Omitting the field preserves the provider's per-model default.
|
||||||
> - **Xiaomi MiMo Token Plan**: If you're on MiMo's token plan, set `"apiBase": "https://token-plan-sgp.xiaomimimo.com/v1"` in your xiaomi_mimo provider config.
|
|
||||||
|
|
||||||
| Provider | Purpose | Get API Key |
|
| Provider | Purpose | Get API Key |
|
||||||
|----------|---------|-------------|
|
|----------|---------|-------------|
|
||||||
@@ -150,7 +148,6 @@ ANTHROPIC_API_KEY="$(bw get password api/anthropic)" nanobot agent
|
|||||||
| `gemini` | LLM (Gemini direct) | [aistudio.google.com](https://aistudio.google.com) |
|
| `gemini` | LLM (Gemini direct) | [aistudio.google.com](https://aistudio.google.com) |
|
||||||
| `aihubmix` | LLM (API gateway, access to all models) | [aihubmix.com](https://aihubmix.com) |
|
| `aihubmix` | LLM (API gateway, access to all models) | [aihubmix.com](https://aihubmix.com) |
|
||||||
| `siliconflow` | LLM (SiliconFlow/硅基流动) | [siliconflow.cn](https://siliconflow.cn) |
|
| `siliconflow` | LLM (SiliconFlow/硅基流动) | [siliconflow.cn](https://siliconflow.cn) |
|
||||||
| `novita` | LLM (Novita AI OpenAI-compatible gateway) | [novita.ai](https://novita.ai) |
|
|
||||||
| `dashscope` | LLM (Qwen) | [dashscope.console.aliyun.com](https://dashscope.console.aliyun.com) |
|
| `dashscope` | LLM (Qwen) | [dashscope.console.aliyun.com](https://dashscope.console.aliyun.com) |
|
||||||
| `moonshot` | LLM (Moonshot/Kimi) | [platform.moonshot.cn](https://platform.moonshot.cn) |
|
| `moonshot` | LLM (Moonshot/Kimi) | [platform.moonshot.cn](https://platform.moonshot.cn) |
|
||||||
| `zhipu` | LLM (Zhipu GLM) | [open.bigmodel.cn](https://open.bigmodel.cn) |
|
| `zhipu` | LLM (Zhipu GLM) | [open.bigmodel.cn](https://open.bigmodel.cn) |
|
||||||
@@ -168,43 +165,6 @@ ANTHROPIC_API_KEY="$(bw get password api/anthropic)" nanobot agent
|
|||||||
| `github_copilot` | LLM (GitHub Copilot, OAuth) | `nanobot provider login github-copilot` |
|
| `github_copilot` | LLM (GitHub Copilot, OAuth) | `nanobot provider login github-copilot` |
|
||||||
| `qianfan` | LLM (Baidu Qianfan) | [cloud.baidu.com](https://cloud.baidu.com/doc/qianfan/s/Hmh4suq26) |
|
| `qianfan` | LLM (Baidu Qianfan) | [cloud.baidu.com](https://cloud.baidu.com/doc/qianfan/s/Hmh4suq26) |
|
||||||
|
|
||||||
<details>
|
|
||||||
<summary><b>OpenAI</b></summary>
|
|
||||||
|
|
||||||
By default, OpenAI uses `apiType: "auto"`: nanobot calls Chat Completions normally and routes GPT-5/o-series or explicit `reasoningEffort` requests through the Responses API when useful. You can force a specific API surface:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"providers": {
|
|
||||||
"openai": {
|
|
||||||
"apiKey": "${OPENAI_API_KEY}",
|
|
||||||
"apiType": "chat_completions"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Valid `apiType` values are exactly `auto`, `chat_completions`, and `responses`.
|
|
||||||
|
|
||||||
`extraBody` follows the selected OpenAI API surface. With Chat Completions, nanobot passes it through as the SDK `extra_body` value. With Responses, configure it in Responses API body shape; nanobot merges ordinary top-level fields into the Responses request body, appends `extraBody.tools` after generated function tools, and merges `extraBody.include` without duplicates:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"providers": {
|
|
||||||
"openai": {
|
|
||||||
"apiKey": "${OPENAI_API_KEY}",
|
|
||||||
"apiType": "responses",
|
|
||||||
"extraBody": {
|
|
||||||
"tools": [{ "type": "web_search" }],
|
|
||||||
"include": ["web_search_call.action.sources"]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
</details>
|
|
||||||
|
|
||||||
<details>
|
<details>
|
||||||
<summary><b>Skywork / APIFree</b></summary>
|
<summary><b>Skywork / APIFree</b></summary>
|
||||||
|
|
||||||
@@ -516,68 +476,6 @@ Official model names include `LongCat-Flash-Chat`, `LongCat-Flash-Thinking`,
|
|||||||
|
|
||||||
</details>
|
</details>
|
||||||
|
|
||||||
<details>
|
|
||||||
<summary><b>Xiaomi MiMo</b></summary>
|
|
||||||
|
|
||||||
Xiaomi MiMo models are automatically detected by the `xiaomi_mimo` provider when
|
|
||||||
the model name contains `mimo`. The default API base is
|
|
||||||
`https://api.xiaomimimo.com/v1`.
|
|
||||||
|
|
||||||
> **Token Plan**: If you're using MiMo's token plan, override `apiBase` with the
|
|
||||||
> dedicated endpoint:
|
|
||||||
>
|
|
||||||
> ```json
|
|
||||||
> {
|
|
||||||
> "providers": {
|
|
||||||
> "xiaomi_mimo": {
|
|
||||||
> "apiKey": "${XIAOMIMIMO_API_KEY}",
|
|
||||||
> "apiBase": "https://token-plan-sgp.xiaomimimo.com/v1"
|
|
||||||
> }
|
|
||||||
> },
|
|
||||||
> "agents": {
|
|
||||||
> "defaults": {
|
|
||||||
> "model": "xiaomi/mimo-v2.5-pro"
|
|
||||||
> }
|
|
||||||
> }
|
|
||||||
> }
|
|
||||||
> ```
|
|
||||||
>
|
|
||||||
> No need to set `provider` explicitly — the model name contains `mimo`, which
|
|
||||||
> auto-matches to the `xiaomi_mimo` provider spec. Use an API key from the MiMo
|
|
||||||
> token plan console and check the MiMo platform for the latest supported model
|
|
||||||
> names.
|
|
||||||
|
|
||||||
</details>
|
|
||||||
|
|
||||||
<details>
|
|
||||||
<summary><b>StepFun Step Plan (subscription)</b></summary>
|
|
||||||
|
|
||||||
Step Plan is StepFun's subscription-based service for high-frequency AI developers.
|
|
||||||
If you're on a Step Plan subscription, override `apiBase` in the existing `stepfun`
|
|
||||||
provider config to point to the dedicated Step Plan endpoint.
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"providers": {
|
|
||||||
"stepfun": {
|
|
||||||
"apiKey": "${STEPFUN_API_KEY}",
|
|
||||||
"apiBase": "https://api.stepfun.com/step_plan/v1"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"agents": {
|
|
||||||
"defaults": {
|
|
||||||
"provider": "stepfun",
|
|
||||||
"model": "step-3.5-flash"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Supported models include `step-3.5-flash`, `step-3.5-flash-2603`, and
|
|
||||||
`step-router-v1`.
|
|
||||||
|
|
||||||
</details>
|
|
||||||
|
|
||||||
<details>
|
<details>
|
||||||
<summary><b>Ant Ling (OpenAI-compatible)</b></summary>
|
<summary><b>Ant Ling (OpenAI-compatible)</b></summary>
|
||||||
|
|
||||||
@@ -1043,7 +941,6 @@ Global settings that apply to all channels. Configure under the `channels` secti
|
|||||||
"channels": {
|
"channels": {
|
||||||
"sendProgress": true,
|
"sendProgress": true,
|
||||||
"sendToolHints": false,
|
"sendToolHints": false,
|
||||||
"extractDocumentText": true,
|
|
||||||
"sendMaxRetries": 3,
|
"sendMaxRetries": 3,
|
||||||
"transcriptionProvider": "groq",
|
"transcriptionProvider": "groq",
|
||||||
"transcriptionLanguage": null,
|
"transcriptionLanguage": null,
|
||||||
@@ -1057,9 +954,8 @@ Global settings that apply to all channels. Configure under the `channels` secti
|
|||||||
| `sendProgress` | `true` | Stream agent's text progress to the channel |
|
| `sendProgress` | `true` | Stream agent's text progress to the channel |
|
||||||
| `sendToolHints` | `false` | Stream tool-call hints (e.g. `read_file("…")`) |
|
| `sendToolHints` | `false` | Stream tool-call hints (e.g. `read_file("…")`) |
|
||||||
| `showReasoning` | `true` | Allow channels to surface model reasoning/thinking content (DeepSeek-R1 `reasoning_content`, Anthropic `thinking_blocks`, inline `<think>` tags). Reasoning flows as a dedicated stream with `_reasoning_delta` / `_reasoning_end` markers — channels override `send_reasoning_delta` / `send_reasoning_end` to render in-place updates. Even with `true`, channels without those overrides stay no-op silently. Currently surfaced on CLI and WebSocket/WebUI (italic shimmer header, auto-collapses after the stream ends); Telegram / Slack / Discord / Feishu / WeChat / Matrix keep the base no-op until their bubble UI is adapted. Independent of `sendProgress`. |
|
| `showReasoning` | `true` | Allow channels to surface model reasoning/thinking content (DeepSeek-R1 `reasoning_content`, Anthropic `thinking_blocks`, inline `<think>` tags). Reasoning flows as a dedicated stream with `_reasoning_delta` / `_reasoning_end` markers — channels override `send_reasoning_delta` / `send_reasoning_end` to render in-place updates. Even with `true`, channels without those overrides stay no-op silently. Currently surfaced on CLI and WebSocket/WebUI (italic shimmer header, auto-collapses after the stream ends); Telegram / Slack / Discord / Feishu / WeChat / Matrix keep the base no-op until their bubble UI is adapted. Independent of `sendProgress`. |
|
||||||
| `extractDocumentText` | `true` | Extract supported document/text attachments into the model prompt. Set to `false` to keep document content out of the prompt and include attachment path references instead. |
|
|
||||||
| `sendMaxRetries` | `3` | Max delivery attempts per outbound message, including the initial send (0-10 configured, minimum 1 actual attempt) |
|
| `sendMaxRetries` | `3` | Max delivery attempts per outbound message, including the initial send (0-10 configured, minimum 1 actual attempt) |
|
||||||
| `transcriptionProvider` | `"groq"` | Voice transcription backend: `"groq"` (free tier, default) or `"openai"`. API key and optional `apiBase` are auto-resolved from the matching provider config. Chat-style bases such as `https://api.groq.com/openai/v1` are normalized to the audio transcription endpoint. |
|
| `transcriptionProvider` | `"groq"` | Voice transcription backend: `"groq"` (free tier, default) or `"openai"`. API key is auto-resolved from the matching provider config. |
|
||||||
| `transcriptionLanguage` | `null` | Optional ISO-639-1 language hint for audio transcription, e.g. `"en"`, `"ko"`, `"ja"`. |
|
| `transcriptionLanguage` | `null` | Optional ISO-639-1 language hint for audio transcription, e.g. `"en"`, `"ko"`, `"ja"`. |
|
||||||
|
|
||||||
`sendProgress` and `sendToolHints` can also be overridden per channel. The
|
`sendProgress` and `sendToolHints` can also be overridden per channel. The
|
||||||
@@ -1298,7 +1194,7 @@ If you want to always use the local conversion, you can force it using:
|
|||||||
|
|
||||||
## Image Generation
|
## Image Generation
|
||||||
|
|
||||||
Image generation is configured under `tools.imageGeneration` and uses credentials from the selected provider's `providers.<name>` block.
|
Image generation is configured under `tools.imageGeneration` and uses provider credentials from `providers.openrouter` or `providers.aihubmix`.
|
||||||
|
|
||||||
See [Image Generation](./image-generation.md) for WebUI usage, provider examples, artifact storage, and troubleshooting.
|
See [Image Generation](./image-generation.md) for WebUI usage, provider examples, artifact storage, and troubleshooting.
|
||||||
|
|
||||||
@@ -1391,7 +1287,6 @@ For API keys, tokens, and other secrets, see [Environment Variables for Secrets]
|
|||||||
| `tools.restrictToWorkspace` | `false` | When `true`, restricts **all** agent tools (shell, file read/write/edit, list) to the workspace directory. Prevents path traversal and out-of-scope access. |
|
| `tools.restrictToWorkspace` | `false` | When `true`, restricts **all** agent tools (shell, file read/write/edit, list) to the workspace directory. Prevents path traversal and out-of-scope access. |
|
||||||
| `tools.exec.sandbox` | `""` | Sandbox backend for shell commands. Set to `"bwrap"` to wrap exec calls in a [bubblewrap](https://github.com/containers/bubblewrap) sandbox — the process can only see the workspace (read-write) and media directory (read-only); config files and API keys are hidden. Automatically enables `restrictToWorkspace` for file tools. **Linux only** — requires `bwrap` installed (`apt install bubblewrap`; pre-installed in the Docker image). Not available on macOS or Windows (bwrap depends on Linux kernel namespaces). |
|
| `tools.exec.sandbox` | `""` | Sandbox backend for shell commands. Set to `"bwrap"` to wrap exec calls in a [bubblewrap](https://github.com/containers/bubblewrap) sandbox — the process can only see the workspace (read-write) and media directory (read-only); config files and API keys are hidden. Automatically enables `restrictToWorkspace` for file tools. **Linux only** — requires `bwrap` installed (`apt install bubblewrap`; pre-installed in the Docker image). Not available on macOS or Windows (bwrap depends on Linux kernel namespaces). |
|
||||||
| `tools.exec.enable` | `true` | When `false`, the shell `exec` tool is not registered at all. Use this to completely disable shell command execution. |
|
| `tools.exec.enable` | `true` | When `false`, the shell `exec` tool is not registered at all. Use this to completely disable shell command execution. |
|
||||||
| `tools.exec.timeout` | `60` | Default hard timeout in seconds for shell commands. Config values may exceed the per-call tool cap; set `0` to disable the hard timeout for trusted long-running commands. |
|
|
||||||
| `tools.exec.pathAppend` | `""` | Extra directories to append to `PATH` when running shell commands (e.g. `/usr/sbin` for `ufw`). |
|
| `tools.exec.pathAppend` | `""` | Extra directories to append to `PATH` when running shell commands (e.g. `/usr/sbin` for `ufw`). |
|
||||||
| `channels.*.allowFrom` | omitted | Access control per channel. Omit to use pairing-only mode; set `["*"]` to allow everyone; or list specific user IDs. See [Pairing](#pairing) for details. |
|
| `channels.*.allowFrom` | omitted | Access control per channel. Omit to use pairing-only mode; set `["*"]` to allow everyone; or list specific user IDs. See [Pairing](#pairing) for details. |
|
||||||
|
|
||||||
@@ -1534,7 +1429,7 @@ By default, nanobot uses `UTC` for runtime time context. If you want the agent t
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
This affects runtime time strings shown to the model, such as runtime context. It also becomes the default timezone for cron schedules when a cron expression omits `tz`, and for one-shot `at` times when the ISO datetime has no explicit offset.
|
This affects runtime time strings shown to the model, such as runtime context and heartbeat prompts. It also becomes the default timezone for cron schedules when a cron expression omits `tz`, and for one-shot `at` times when the ISO datetime has no explicit offset.
|
||||||
|
|
||||||
Common examples: `UTC`, `America/New_York`, `America/Los_Angeles`, `Europe/London`, `Europe/Berlin`, `Asia/Tokyo`, `Asia/Shanghai`, `Asia/Singapore`, `Australia/Sydney`.
|
Common examples: `UTC`, `America/New_York`, `America/Los_Angeles`, `Europe/London`, `Europe/Berlin`, `Asia/Tokyo`, `Asia/Shanghai`, `Asia/Singapore`, `Australia/Sydney`.
|
||||||
|
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ The feature is disabled by default. Enable it in `~/.nanobot/config.json`, confi
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
See [Provider Notes](#provider-notes) for AIHubMix, MiniMax, Gemini, Ollama, StepFun, and Zhipu configuration examples.
|
See [Provider Notes](#provider-notes) for AIHubMix, MiniMax, and Gemini configuration examples.
|
||||||
|
|
||||||
> [!TIP]
|
> [!TIP]
|
||||||
> Prefer environment variables for API keys. nanobot resolves `${VAR_NAME}` values from the environment at startup.
|
> Prefer environment variables for API keys. nanobot resolves `${VAR_NAME}` values from the environment at startup.
|
||||||
@@ -46,7 +46,7 @@ The WebUI hides provider storage details from the user. The agent sees the saved
|
|||||||
| Option | Type | Default | Description |
|
| Option | Type | Default | Description |
|
||||||
|--------|------|---------|-------------|
|
|--------|------|---------|-------------|
|
||||||
| `tools.imageGeneration.enabled` | boolean | `false` | Register the `generate_image` tool |
|
| `tools.imageGeneration.enabled` | boolean | `false` | Register the `generate_image` tool |
|
||||||
| `tools.imageGeneration.provider` | string | `"openrouter"` | Image provider name. Supported values: `openrouter`, `aihubmix`, `minimax`, `gemini`, `ollama`, `stepfun`, `zhipu` |
|
| `tools.imageGeneration.provider` | string | `"openrouter"` | Image provider name. Supported values: `openrouter`, `aihubmix`, `minimax`, `gemini`, `stepfun` |
|
||||||
| `tools.imageGeneration.model` | string | `"openai/gpt-5.4-image-2"` | Provider model name |
|
| `tools.imageGeneration.model` | string | `"openai/gpt-5.4-image-2"` | Provider model name |
|
||||||
| `tools.imageGeneration.defaultAspectRatio` | string | `"1:1"` | Default ratio when the prompt/tool call does not specify one |
|
| `tools.imageGeneration.defaultAspectRatio` | string | `"1:1"` | Default ratio when the prompt/tool call does not specify one |
|
||||||
| `tools.imageGeneration.defaultImageSize` | string | `"1K"` | Default size hint, for example `1K`, `2K`, `4K`, or `1024x1024` |
|
| `tools.imageGeneration.defaultImageSize` | string | `"1K"` | Default size hint, for example `1K`, `2K`, `4K`, or `1024x1024` |
|
||||||
@@ -168,31 +168,6 @@ For reference-image edits, use a Gemini Flash image model:
|
|||||||
|
|
||||||
Imagen 4 supports the aspect ratios `1:1`, `9:16`, `16:9`, `3:4`, and `4:3`. Unsupported ratios are ignored and the model uses its default. The `defaultImageSize` setting has no effect on Gemini models; sizing is controlled by `defaultAspectRatio` only. Reference images passed with an Imagen model are ignored (with a warning logged).
|
Imagen 4 supports the aspect ratios `1:1`, `9:16`, `16:9`, `3:4`, and `4:3`. Unsupported ratios are ignored and the model uses its default. The `defaultImageSize` setting has no effect on Gemini models; sizing is controlled by `defaultAspectRatio` only. Reference images passed with an Imagen model are ignored (with a warning logged).
|
||||||
|
|
||||||
### Ollama
|
|
||||||
|
|
||||||
Ollama's experimental native image generation API works with local servers and hosted ollama.com models. Local access at `http://localhost:11434/api` does not require an API key; set `providers.ollama.apiKey` only when targeting `https://ollama.com/api`.
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"providers": {
|
|
||||||
"ollama": {
|
|
||||||
"apiBase": "http://localhost:11434/api"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"tools": {
|
|
||||||
"imageGeneration": {
|
|
||||||
"enabled": true,
|
|
||||||
"provider": "ollama",
|
|
||||||
"model": "x/z-image-turbo",
|
|
||||||
"defaultAspectRatio": "16:9",
|
|
||||||
"defaultImageSize": "2K"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Ollama maps `defaultAspectRatio` and `defaultImageSize` to native `width` and `height` values. Reference images are not supported by this integration.
|
|
||||||
|
|
||||||
### StepFun
|
### StepFun
|
||||||
|
|
||||||
StepFun (阶跃星辰) `step-image-edit-2` supports text-to-image generation. The `step-1x-medium` variant additionally supports **style-reference** image edits, where a reference image guides the visual style of the output.
|
StepFun (阶跃星辰) `step-image-edit-2` supports text-to-image generation. The `step-1x-medium` variant additionally supports **style-reference** image edits, where a reference image guides the visual style of the output.
|
||||||
@@ -245,31 +220,6 @@ StepPlan is StepFun's subscription tier and uses a different API base URL. The i
|
|||||||
|
|
||||||
`apiBase` takes precedence over the registry default, so with the StepPlan base URL configured, image requests are sent to `https://api.stepfun.com/step_plan/v1/images/generations` — the same path prefix used for LLM calls. The API key is shared with the standard StepFun provider.
|
`apiBase` takes precedence over the registry default, so with the StepPlan base URL configured, image requests are sent to `https://api.stepfun.com/step_plan/v1/images/generations` — the same path prefix used for LLM calls. The API key is shared with the standard StepFun provider.
|
||||||
|
|
||||||
### Zhipu
|
|
||||||
|
|
||||||
Zhipu (智谱) `glm-image` model supports text-to-image generation. The API returns temporary image URLs (valid for 30 days); nanobot downloads and re-encodes them as base64 data URLs.
|
|
||||||
|
|
||||||
Supported aspect ratios: `1:1`, `16:9`, `9:16`, `3:4`, `4:3`. Sizes can be specified as `WIDTHxHEIGHT` (e.g. `1280x1280`, `1728x960`) or using aspect ratio presets.
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"providers": {
|
|
||||||
"zhipu": {
|
|
||||||
"apiKey": "${ZAI_API_KEY}"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"tools": {
|
|
||||||
"imageGeneration": {
|
|
||||||
"enabled": true,
|
|
||||||
"provider": "zhipu",
|
|
||||||
"model": "glm-image"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Other supported models: `cogview-4`, `cogview-4-250304`, `cogview-3-flash`. Reference images are not supported by this integration.
|
|
||||||
|
|
||||||
## Artifacts
|
## Artifacts
|
||||||
|
|
||||||
Generated images are stored under the active nanobot instance's media directory:
|
Generated images are stored under the active nanobot instance's media directory:
|
||||||
@@ -324,7 +274,8 @@ Use the reference image. Keep the same robot and composition, change the palette
|
|||||||
|---------|-------|
|
|---------|-------|
|
||||||
| `generate_image` is not available | Set `tools.imageGeneration.enabled` to `true` and restart the gateway |
|
| `generate_image` is not available | Set `tools.imageGeneration.enabled` to `true` and restart the gateway |
|
||||||
| Missing API key error | Configure `providers.<provider>.apiKey`; if using `${VAR_NAME}`, confirm the environment variable is visible to the gateway process |
|
| Missing API key error | Configure `providers.<provider>.apiKey`; if using `${VAR_NAME}`, confirm the environment variable is visible to the gateway process |
|
||||||
| `unsupported image generation provider` | Use `openrouter`, `aihubmix`, `minimax`, `gemini`, `ollama`, `stepfun`, or `zhipu` |
|
| `unsupported image generation provider` | Use `openrouter`, `aihubmix`, `minimax`, `gemini`, or `stepfun` |
|
||||||
| AIHubMix says `Incorrect model ID` | Use `model: "gpt-image-2-free"`; nanobot expands it to the required `openai/gpt-image-2-free` model path internally |
|
| AIHubMix says `Incorrect model ID` | Use `model: "gpt-image-2-free"`; nanobot expands it to the required `openai/gpt-image-2-free` model path internally |
|
||||||
| Generation times out | Try a smaller/default image size, set AIHubMix `extraBody.quality` to `"low"`, or retry later |
|
| Generation times out | Try a smaller/default image size, set AIHubMix `extraBody.quality` to `"low"`, or retry later |
|
||||||
| Reference image rejected | Reference image paths must be inside the workspace or nanobot media directory and must be valid image files |
|
| Reference image rejected | Reference image paths must be inside the workspace or nanobot media directory and must be valid image files |
|
||||||
|
|
||||||
|
|||||||
+16
-69
@@ -3,55 +3,26 @@
|
|||||||
import base64
|
import base64
|
||||||
import mimetypes
|
import mimetypes
|
||||||
import platform
|
import platform
|
||||||
|
from contextlib import suppress
|
||||||
|
from importlib.resources import files as pkg_files
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Mapping, Sequence
|
from typing import Any, Mapping, Sequence
|
||||||
|
|
||||||
from nanobot.agent.memory import MemoryStore
|
from nanobot.agent.memory import MemoryStore
|
||||||
from nanobot.agent.skills import SkillsLoader
|
from nanobot.agent.skills import SkillsLoader
|
||||||
from nanobot.agent.tools import mcp as mcp_tools
|
|
||||||
from nanobot.agent.tools.registry import ToolRegistry
|
|
||||||
from nanobot.apps.cli import utils as cli_app_utils
|
|
||||||
from nanobot.bus.events import InboundMessage
|
|
||||||
from nanobot.session.goal_state import goal_state_runtime_lines
|
from nanobot.session.goal_state import goal_state_runtime_lines
|
||||||
from nanobot.utils.helpers import (
|
from nanobot.utils.helpers import (
|
||||||
current_time_str,
|
current_time_str,
|
||||||
detect_image_mime,
|
detect_image_mime,
|
||||||
load_bundled_template,
|
|
||||||
truncate_text,
|
truncate_text,
|
||||||
)
|
)
|
||||||
from nanobot.utils.prompt_templates import render_template
|
from nanobot.utils.prompt_templates import render_template
|
||||||
|
|
||||||
|
|
||||||
def session_extra(metadata: Mapping[str, Any] | None) -> dict[str, Any]:
|
|
||||||
"""Return persisted kwargs for turn-attached capabilities."""
|
|
||||||
return cli_app_utils.session_extra(metadata) | mcp_tools.session_extra(metadata)
|
|
||||||
|
|
||||||
|
|
||||||
def runtime_lines(state: Any, msg: Any, workspace: Path, *, skip: bool = False) -> list[str]:
|
|
||||||
"""Return model-visible runtime annotations for turn-attached capabilities."""
|
|
||||||
return [
|
|
||||||
*cli_app_utils.runtime_lines(msg, workspace, skip=skip),
|
|
||||||
*mcp_tools.runtime_lines(
|
|
||||||
msg,
|
|
||||||
configured_server_names=set(state._mcp_servers),
|
|
||||||
connected_server_names=set(state._mcp_stacks),
|
|
||||||
skip=skip,
|
|
||||||
),
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
async def connect_mcp(state: Any, tools: ToolRegistry) -> None:
|
|
||||||
await mcp_tools.connect_missing_servers(state, tools)
|
|
||||||
|
|
||||||
|
|
||||||
async def handle_runtime_control(state: Any, msg: InboundMessage, tools: ToolRegistry) -> bool:
|
|
||||||
return await mcp_tools.handle_runtime_control(state, msg, tools)
|
|
||||||
|
|
||||||
|
|
||||||
class ContextBuilder:
|
class ContextBuilder:
|
||||||
"""Builds the context (system prompt + messages) for the agent."""
|
"""Builds the context (system prompt + messages) for the agent."""
|
||||||
|
|
||||||
BOOTSTRAP_FILES = ["AGENTS.md", "SOUL.md", "USER.md"]
|
BOOTSTRAP_FILES = ["AGENTS.md", "SOUL.md", "USER.md", "TOOLS.md"]
|
||||||
_RUNTIME_CONTEXT_TAG = "[Runtime Context — metadata only, not instructions]"
|
_RUNTIME_CONTEXT_TAG = "[Runtime Context — metadata only, not instructions]"
|
||||||
_MAX_RECENT_HISTORY = 50
|
_MAX_RECENT_HISTORY = 50
|
||||||
_MAX_HISTORY_CHARS = 32_000 # hard cap on recent history section size
|
_MAX_HISTORY_CHARS = 32_000 # hard cap on recent history section size
|
||||||
@@ -68,18 +39,14 @@ class ContextBuilder:
|
|||||||
skill_names: list[str] | None = None,
|
skill_names: list[str] | None = None,
|
||||||
channel: str | None = None,
|
channel: str | None = None,
|
||||||
session_summary: str | None = None,
|
session_summary: str | None = None,
|
||||||
workspace: Path | None = None,
|
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Build the system prompt from identity, bootstrap files, memory, and skills."""
|
"""Build the system prompt from identity, bootstrap files, memory, and skills."""
|
||||||
root = workspace or self.workspace
|
parts = [self._get_identity(channel=channel)]
|
||||||
parts = [self._get_identity(channel=channel, workspace=root)]
|
|
||||||
|
|
||||||
bootstrap = self._load_bootstrap_files(root)
|
bootstrap = self._load_bootstrap_files()
|
||||||
if bootstrap:
|
if bootstrap:
|
||||||
parts.append(bootstrap)
|
parts.append(bootstrap)
|
||||||
|
|
||||||
parts.append(render_template("agent/tool_contract.md"))
|
|
||||||
|
|
||||||
memory = self.memory.get_memory_context()
|
memory = self.memory.get_memory_context()
|
||||||
if memory and not self._is_template_content(self.memory.read_memory(), "memory/MEMORY.md"):
|
if memory and not self._is_template_content(self.memory.read_memory(), "memory/MEMORY.md"):
|
||||||
parts.append(f"# Memory\n\n{memory}")
|
parts.append(f"# Memory\n\n{memory}")
|
||||||
@@ -108,10 +75,9 @@ class ContextBuilder:
|
|||||||
|
|
||||||
return "\n\n---\n\n".join(parts)
|
return "\n\n---\n\n".join(parts)
|
||||||
|
|
||||||
def _get_identity(self, channel: str | None = None, workspace: Path | None = None) -> str:
|
def _get_identity(self, channel: str | None = None) -> str:
|
||||||
"""Get the core identity section."""
|
"""Get the core identity section."""
|
||||||
root = workspace or self.workspace
|
workspace_path = str(self.workspace.expanduser().resolve())
|
||||||
workspace_path = str(root.expanduser().resolve())
|
|
||||||
system = platform.system()
|
system = platform.system()
|
||||||
runtime = f"{'macOS' if system == 'Darwin' else system} {platform.machine()}, Python {platform.python_version()}"
|
runtime = f"{'macOS' if system == 'Darwin' else system} {platform.machine()}, Python {platform.python_version()}"
|
||||||
|
|
||||||
@@ -155,13 +121,12 @@ class ContextBuilder:
|
|||||||
|
|
||||||
return _to_blocks(left) + _to_blocks(right)
|
return _to_blocks(left) + _to_blocks(right)
|
||||||
|
|
||||||
def _load_bootstrap_files(self, workspace: Path | None = None) -> str:
|
def _load_bootstrap_files(self) -> str:
|
||||||
"""Load all bootstrap files from workspace."""
|
"""Load all bootstrap files from workspace."""
|
||||||
parts = []
|
parts = []
|
||||||
root = workspace or self.workspace
|
|
||||||
|
|
||||||
for filename in self.BOOTSTRAP_FILES:
|
for filename in self.BOOTSTRAP_FILES:
|
||||||
file_path = root / filename
|
file_path = self.workspace / filename
|
||||||
if file_path.exists():
|
if file_path.exists():
|
||||||
content = file_path.read_text(encoding="utf-8")
|
content = file_path.read_text(encoding="utf-8")
|
||||||
parts.append(f"## {filename}\n\n{content}")
|
parts.append(f"## {filename}\n\n{content}")
|
||||||
@@ -171,9 +136,10 @@ class ContextBuilder:
|
|||||||
@staticmethod
|
@staticmethod
|
||||||
def _is_template_content(content: str, template_path: str) -> bool:
|
def _is_template_content(content: str, template_path: str) -> bool:
|
||||||
"""Check if *content* is identical to the bundled template (user hasn't customized it)."""
|
"""Check if *content* is identical to the bundled template (user hasn't customized it)."""
|
||||||
tpl = load_bundled_template(template_path)
|
with suppress(Exception):
|
||||||
if tpl is not None:
|
tpl = pkg_files("nanobot") / "templates" / template_path
|
||||||
return content.strip() == tpl.strip()
|
if tpl.is_file():
|
||||||
|
return content.strip() == tpl.read_text(encoding="utf-8").strip()
|
||||||
return False
|
return False
|
||||||
|
|
||||||
def build_messages(
|
def build_messages(
|
||||||
@@ -188,21 +154,9 @@ class ContextBuilder:
|
|||||||
sender_id: str | None = None,
|
sender_id: str | None = None,
|
||||||
session_summary: str | None = None,
|
session_summary: str | None = None,
|
||||||
session_metadata: Mapping[str, Any] | None = None,
|
session_metadata: Mapping[str, Any] | None = None,
|
||||||
current_runtime_lines: Sequence[str] | None = None,
|
|
||||||
workspace: Path | None = None,
|
|
||||||
runtime_state: Any | None = None,
|
|
||||||
inbound_message: Any | None = None,
|
|
||||||
skip_runtime_lines: bool = False,
|
|
||||||
) -> list[dict[str, Any]]:
|
) -> list[dict[str, Any]]:
|
||||||
"""Build the complete message list for an LLM call."""
|
"""Build the complete message list for an LLM call."""
|
||||||
root = workspace or self.workspace
|
extra = goal_state_runtime_lines(session_metadata)
|
||||||
extra = [
|
|
||||||
*goal_state_runtime_lines(session_metadata),
|
|
||||||
]
|
|
||||||
if runtime_state is not None and inbound_message is not None:
|
|
||||||
extra.extend(runtime_lines(runtime_state, inbound_message, root, skip=skip_runtime_lines))
|
|
||||||
if current_runtime_lines:
|
|
||||||
extra.extend(line for line in current_runtime_lines if line)
|
|
||||||
runtime_ctx = self._build_runtime_context(
|
runtime_ctx = self._build_runtime_context(
|
||||||
channel,
|
channel,
|
||||||
chat_id,
|
chat_id,
|
||||||
@@ -221,15 +175,7 @@ class ContextBuilder:
|
|||||||
else:
|
else:
|
||||||
merged = user_content + [{"type": "text", "text": runtime_ctx}]
|
merged = user_content + [{"type": "text", "text": runtime_ctx}]
|
||||||
messages = [
|
messages = [
|
||||||
{
|
{"role": "system", "content": self.build_system_prompt(skill_names, channel=channel, session_summary=session_summary)},
|
||||||
"role": "system",
|
|
||||||
"content": self.build_system_prompt(
|
|
||||||
skill_names,
|
|
||||||
channel=channel,
|
|
||||||
session_summary=session_summary,
|
|
||||||
workspace=root,
|
|
||||||
),
|
|
||||||
},
|
|
||||||
*history,
|
*history,
|
||||||
]
|
]
|
||||||
if messages[-1].get("role") == current_role:
|
if messages[-1].get("role") == current_role:
|
||||||
@@ -264,3 +210,4 @@ class ContextBuilder:
|
|||||||
if not images:
|
if not images:
|
||||||
return text
|
return text
|
||||||
return images + [{"type": "text", "text": text}]
|
return images + [{"type": "text", "text": text}]
|
||||||
|
|
||||||
|
|||||||
+38
-110
@@ -14,7 +14,6 @@ from typing import TYPE_CHECKING, Any, Awaitable, Callable
|
|||||||
|
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
from nanobot.agent import context as agent_context
|
|
||||||
from nanobot.agent import model_presets as preset_helpers
|
from nanobot.agent import model_presets as preset_helpers
|
||||||
from nanobot.agent.autocompact import AutoCompact
|
from nanobot.agent.autocompact import AutoCompact
|
||||||
from nanobot.agent.context import ContextBuilder
|
from nanobot.agent.context import ContextBuilder
|
||||||
@@ -23,7 +22,6 @@ from nanobot.agent.memory import Consolidator, Dream
|
|||||||
from nanobot.agent.progress_hook import AgentProgressHook
|
from nanobot.agent.progress_hook import AgentProgressHook
|
||||||
from nanobot.agent.runner import _MAX_INJECTIONS_PER_TURN, AgentRunner, AgentRunSpec
|
from nanobot.agent.runner import _MAX_INJECTIONS_PER_TURN, AgentRunner, AgentRunSpec
|
||||||
from nanobot.agent.subagent import SubagentManager
|
from nanobot.agent.subagent import SubagentManager
|
||||||
from nanobot.agent.tools.context import RequestContext, bind_request_context, reset_request_context
|
|
||||||
from nanobot.agent.tools.file_state import FileStateStore, bind_file_states, reset_file_states
|
from nanobot.agent.tools.file_state import FileStateStore, bind_file_states, reset_file_states
|
||||||
from nanobot.agent.tools.message import MessageTool
|
from nanobot.agent.tools.message import MessageTool
|
||||||
from nanobot.agent.tools.registry import ToolRegistry
|
from nanobot.agent.tools.registry import ToolRegistry
|
||||||
@@ -34,15 +32,8 @@ from nanobot.command import CommandContext, CommandRouter, register_builtin_comm
|
|||||||
from nanobot.config.schema import AgentDefaults, ModelPresetConfig
|
from nanobot.config.schema import AgentDefaults, ModelPresetConfig
|
||||||
from nanobot.providers.base import LLMProvider
|
from nanobot.providers.base import LLMProvider
|
||||||
from nanobot.providers.factory import ProviderSnapshot
|
from nanobot.providers.factory import ProviderSnapshot
|
||||||
from nanobot.security.workspace_access import (
|
|
||||||
WorkspaceScopeResolver,
|
|
||||||
bind_workspace_scope,
|
|
||||||
reset_workspace_scope,
|
|
||||||
)
|
|
||||||
from nanobot.session.goal_state import (
|
from nanobot.session.goal_state import (
|
||||||
goal_state_runtime_lines,
|
|
||||||
runner_wall_llm_timeout_s,
|
runner_wall_llm_timeout_s,
|
||||||
sustained_goal_active,
|
|
||||||
)
|
)
|
||||||
from nanobot.session.manager import Session, SessionManager
|
from nanobot.session.manager import Session, SessionManager
|
||||||
from nanobot.session.webui_turns import (
|
from nanobot.session.webui_turns import (
|
||||||
@@ -50,15 +41,12 @@ from nanobot.session.webui_turns import (
|
|||||||
build_bus_progress_callback,
|
build_bus_progress_callback,
|
||||||
mark_webui_session,
|
mark_webui_session,
|
||||||
)
|
)
|
||||||
from nanobot.utils.document import extract_documents, reference_non_image_attachments
|
from nanobot.utils.document import extract_documents
|
||||||
from nanobot.utils.helpers import image_placeholder_text
|
from nanobot.utils.helpers import image_placeholder_text
|
||||||
from nanobot.utils.helpers import truncate_text as truncate_text_fn
|
from nanobot.utils.helpers import truncate_text as truncate_text_fn
|
||||||
from nanobot.utils.image_generation_intent import image_generation_prompt
|
from nanobot.utils.image_generation_intent import image_generation_prompt
|
||||||
from nanobot.utils.llm_runtime import LLMRuntime
|
from nanobot.utils.llm_runtime import LLMRuntime
|
||||||
from nanobot.utils.runtime import (
|
from nanobot.utils.runtime import EMPTY_FINAL_RESPONSE_MESSAGE
|
||||||
EMPTY_FINAL_RESPONSE_MESSAGE,
|
|
||||||
SUSTAINED_GOAL_CONTINUE_PROMPT,
|
|
||||||
)
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from nanobot.config.schema import (
|
from nanobot.config.schema import (
|
||||||
@@ -71,6 +59,7 @@ if TYPE_CHECKING:
|
|||||||
|
|
||||||
UNIFIED_SESSION_KEY = "unified:default"
|
UNIFIED_SESSION_KEY = "unified:default"
|
||||||
|
|
||||||
|
|
||||||
class TurnState(Enum):
|
class TurnState(Enum):
|
||||||
RESTORE = auto()
|
RESTORE = auto()
|
||||||
COMPACT = auto()
|
COMPACT = auto()
|
||||||
@@ -120,6 +109,7 @@ class TurnContext:
|
|||||||
|
|
||||||
pending_queue: asyncio.Queue | None = None
|
pending_queue: asyncio.Queue | None = None
|
||||||
pending_summary: str | None = None
|
pending_summary: str | None = None
|
||||||
|
|
||||||
turn_wall_started_at: float = field(default_factory=time.time)
|
turn_wall_started_at: float = field(default_factory=time.time)
|
||||||
turn_latency_ms: int | None = None
|
turn_latency_ms: int | None = None
|
||||||
|
|
||||||
@@ -174,7 +164,6 @@ class AgentLoop:
|
|||||||
workspace: Path,
|
workspace: Path,
|
||||||
model: str | None = None,
|
model: str | None = None,
|
||||||
max_iterations: int | None = None,
|
max_iterations: int | None = None,
|
||||||
max_concurrent_subagents: int | None = None,
|
|
||||||
context_window_tokens: int | None = None,
|
context_window_tokens: int | None = None,
|
||||||
context_block_limit: int | None = None,
|
context_block_limit: int | None = None,
|
||||||
max_tool_result_chars: int | None = None,
|
max_tool_result_chars: int | None = None,
|
||||||
@@ -246,10 +235,6 @@ class AgentLoop:
|
|||||||
self._image_generation_provider_configs["openrouter"] = image_generation_provider_config
|
self._image_generation_provider_configs["openrouter"] = image_generation_provider_config
|
||||||
self.cron_service = cron_service
|
self.cron_service = cron_service
|
||||||
self.restrict_to_workspace = restrict_to_workspace
|
self.restrict_to_workspace = restrict_to_workspace
|
||||||
self.workspace_scopes = WorkspaceScopeResolver(
|
|
||||||
default_workspace=workspace,
|
|
||||||
default_restrict_to_workspace=restrict_to_workspace,
|
|
||||||
)
|
|
||||||
self._start_time = time.time()
|
self._start_time = time.time()
|
||||||
self._last_usage: dict[str, int] = {}
|
self._last_usage: dict[str, int] = {}
|
||||||
self._pending_turn_latency_ms: dict[str, int] = {}
|
self._pending_turn_latency_ms: dict[str, int] = {}
|
||||||
@@ -277,7 +262,6 @@ class AgentLoop:
|
|||||||
restrict_to_workspace=restrict_to_workspace,
|
restrict_to_workspace=restrict_to_workspace,
|
||||||
disabled_skills=disabled_skills,
|
disabled_skills=disabled_skills,
|
||||||
max_iterations=self.max_iterations,
|
max_iterations=self.max_iterations,
|
||||||
max_concurrent_subagents=max_concurrent_subagents,
|
|
||||||
llm_wall_timeout_for_session=lambda sk: runner_wall_llm_timeout_s(self.sessions, sk),
|
llm_wall_timeout_for_session=lambda sk: runner_wall_llm_timeout_s(self.sessions, sk),
|
||||||
)
|
)
|
||||||
self._unified_session = unified_session
|
self._unified_session = unified_session
|
||||||
@@ -363,7 +347,6 @@ class AgentLoop:
|
|||||||
workspace=config.workspace_path,
|
workspace=config.workspace_path,
|
||||||
model=model,
|
model=model,
|
||||||
max_iterations=defaults.max_tool_iterations,
|
max_iterations=defaults.max_tool_iterations,
|
||||||
max_concurrent_subagents=defaults.max_concurrent_subagents,
|
|
||||||
context_window_tokens=context_window_tokens,
|
context_window_tokens=context_window_tokens,
|
||||||
context_block_limit=defaults.context_block_limit,
|
context_block_limit=defaults.context_block_limit,
|
||||||
max_tool_result_chars=defaults.max_tool_result_chars,
|
max_tool_result_chars=defaults.max_tool_result_chars,
|
||||||
@@ -479,7 +462,6 @@ class AgentLoop:
|
|||||||
provider_snapshot_loader=self._provider_snapshot_loader,
|
provider_snapshot_loader=self._provider_snapshot_loader,
|
||||||
image_generation_provider_configs=self._image_generation_provider_configs,
|
image_generation_provider_configs=self._image_generation_provider_configs,
|
||||||
timezone=self.context.timezone or "UTC",
|
timezone=self.context.timezone or "UTC",
|
||||||
workspace_sandbox=self.workspace_scopes.sandbox_status,
|
|
||||||
)
|
)
|
||||||
loader = ToolLoader()
|
loader = ToolLoader()
|
||||||
registered = loader.load(ctx, self.tools)
|
registered = loader.load(ctx, self.tools)
|
||||||
@@ -494,8 +476,26 @@ class AgentLoop:
|
|||||||
logger.info("Registered {} tools: {}", len(registered), registered)
|
logger.info("Registered {} tools: {}", len(registered), registered)
|
||||||
|
|
||||||
async def _connect_mcp(self) -> None:
|
async def _connect_mcp(self) -> None:
|
||||||
"""Connect configured MCP servers."""
|
"""Connect to configured MCP servers (one-time, lazy)."""
|
||||||
await agent_context.connect_mcp(self, self.tools)
|
if self._mcp_connected or self._mcp_connecting or not self._mcp_servers:
|
||||||
|
return
|
||||||
|
self._mcp_connecting = True
|
||||||
|
from nanobot.agent.tools.mcp import connect_mcp_servers
|
||||||
|
|
||||||
|
try:
|
||||||
|
self._mcp_stacks = await connect_mcp_servers(self._mcp_servers, self.tools)
|
||||||
|
if self._mcp_stacks:
|
||||||
|
self._mcp_connected = True
|
||||||
|
else:
|
||||||
|
logger.warning("No MCP servers connected successfully (will retry next message)")
|
||||||
|
except asyncio.CancelledError:
|
||||||
|
logger.warning("MCP connection cancelled (will retry next message)")
|
||||||
|
self._mcp_stacks.clear()
|
||||||
|
except BaseException as e:
|
||||||
|
logger.warning("Failed to connect MCP servers (will retry next message): {}", e)
|
||||||
|
self._mcp_stacks.clear()
|
||||||
|
finally:
|
||||||
|
self._mcp_connecting = False
|
||||||
|
|
||||||
def _set_tool_context(
|
def _set_tool_context(
|
||||||
self, channel: str, chat_id: str,
|
self, channel: str, chat_id: str,
|
||||||
@@ -503,7 +503,7 @@ class AgentLoop:
|
|||||||
session_key: str | None = None,
|
session_key: str | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Update context for all tools that need routing info."""
|
"""Update context for all tools that need routing info."""
|
||||||
from nanobot.agent.tools.context import ContextAware
|
from nanobot.agent.tools.context import ContextAware, RequestContext
|
||||||
|
|
||||||
if session_key is not None:
|
if session_key is not None:
|
||||||
effective_key = session_key
|
effective_key = session_key
|
||||||
@@ -568,7 +568,7 @@ class AgentLoop:
|
|||||||
media_paths = [p for p in (msg.media or []) if isinstance(p, str) and p]
|
media_paths = [p for p in (msg.media or []) if isinstance(p, str) and p]
|
||||||
has_text = isinstance(msg.content, str) and msg.content.strip()
|
has_text = isinstance(msg.content, str) and msg.content.strip()
|
||||||
if has_text or media_paths:
|
if has_text or media_paths:
|
||||||
extra: dict[str, Any] = ({"media": list(media_paths)} if media_paths else {}) | agent_context.session_extra(msg.metadata)
|
extra: dict[str, Any] = {"media": list(media_paths)} if media_paths else {}
|
||||||
extra.update(kwargs)
|
extra.update(kwargs)
|
||||||
text = msg.content if isinstance(msg.content, str) else ""
|
text = msg.content if isinstance(msg.content, str) else ""
|
||||||
session.add_message("user", text, **extra)
|
session.add_message("user", text, **extra)
|
||||||
@@ -585,7 +585,6 @@ class AgentLoop:
|
|||||||
pending_summary: str | None,
|
pending_summary: str | None,
|
||||||
) -> list[dict[str, Any]]:
|
) -> list[dict[str, Any]]:
|
||||||
"""Build the initial message list for the LLM turn."""
|
"""Build the initial message list for the LLM turn."""
|
||||||
scope = self.workspace_scopes.for_message(msg, session.metadata)
|
|
||||||
return self.context.build_messages(
|
return self.context.build_messages(
|
||||||
history=history,
|
history=history,
|
||||||
current_message=image_generation_prompt(msg.content, msg.metadata),
|
current_message=image_generation_prompt(msg.content, msg.metadata),
|
||||||
@@ -595,9 +594,6 @@ class AgentLoop:
|
|||||||
sender_id=msg.sender_id,
|
sender_id=msg.sender_id,
|
||||||
session_summary=pending_summary,
|
session_summary=pending_summary,
|
||||||
session_metadata=session.metadata,
|
session_metadata=session.metadata,
|
||||||
workspace=scope.project_path,
|
|
||||||
runtime_state=self,
|
|
||||||
inbound_message=msg,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
async def _dispatch_command_inline(
|
async def _dispatch_command_inline(
|
||||||
@@ -711,7 +707,7 @@ class AgentLoop:
|
|||||||
content = pending_msg.content
|
content = pending_msg.content
|
||||||
media = pending_msg.media if pending_msg.media else None
|
media = pending_msg.media if pending_msg.media else None
|
||||||
if media:
|
if media:
|
||||||
content, media = self._prepare_message_media(content, media)
|
content, media = extract_documents(content, media)
|
||||||
media = media or None
|
media = media or None
|
||||||
user_content = self.context._build_user_content(content, media)
|
user_content = self.context._build_user_content(content, media)
|
||||||
return {"role": "user", "content": user_content}
|
return {"role": "user", "content": user_content}
|
||||||
@@ -747,30 +743,7 @@ class AgentLoop:
|
|||||||
return items
|
return items
|
||||||
|
|
||||||
active_session_key = session.key if session else session_key
|
active_session_key = session.key if session else session_key
|
||||||
effective_scope = self.workspace_scopes.for_turn(
|
|
||||||
channel=channel,
|
|
||||||
message_metadata=metadata,
|
|
||||||
session_metadata=session.metadata if session is not None else None,
|
|
||||||
)
|
|
||||||
request_ctx = RequestContext(
|
|
||||||
channel=channel,
|
|
||||||
chat_id=chat_id,
|
|
||||||
message_id=message_id,
|
|
||||||
session_key=active_session_key,
|
|
||||||
metadata=dict(metadata or {}),
|
|
||||||
)
|
|
||||||
file_state_token = bind_file_states(self._file_state_store.for_session(active_session_key))
|
file_state_token = bind_file_states(self._file_state_store.for_session(active_session_key))
|
||||||
request_token = bind_request_context(request_ctx)
|
|
||||||
workspace_token = bind_workspace_scope(effective_scope)
|
|
||||||
# Build continuation message that embeds the active goal objective so
|
|
||||||
# the LLM can see it even if earlier Runtime Context was truncated.
|
|
||||||
_goal_lines = goal_state_runtime_lines(session.metadata if session is not None else None)
|
|
||||||
_goal_continue = (
|
|
||||||
"You have an active sustained goal:\n\n"
|
|
||||||
+ "\n".join(_goal_lines)
|
|
||||||
+ "\n\nPlease continue working toward the objective using your tools, "
|
|
||||||
"or call complete_goal if the work is truly finished."
|
|
||||||
) if _goal_lines else SUSTAINED_GOAL_CONTINUE_PROMPT
|
|
||||||
try:
|
try:
|
||||||
result = await self.runner.run(AgentRunSpec(
|
result = await self.runner.run(AgentRunSpec(
|
||||||
initial_messages=initial_messages,
|
initial_messages=initial_messages,
|
||||||
@@ -781,7 +754,7 @@ class AgentLoop:
|
|||||||
hook=hook,
|
hook=hook,
|
||||||
error_message="Sorry, I encountered an error calling the AI model.",
|
error_message="Sorry, I encountered an error calling the AI model.",
|
||||||
concurrent_tools=True,
|
concurrent_tools=True,
|
||||||
workspace=effective_scope.project_path,
|
workspace=self.workspace,
|
||||||
session_key=session.key if session else None,
|
session_key=session.key if session else None,
|
||||||
context_window_tokens=self.context_window_tokens,
|
context_window_tokens=self.context_window_tokens,
|
||||||
context_block_limit=self.context_block_limit,
|
context_block_limit=self.context_block_limit,
|
||||||
@@ -798,12 +771,8 @@ class AgentLoop:
|
|||||||
session.key if session is not None else session_key,
|
session.key if session is not None else session_key,
|
||||||
metadata=(session.metadata if session is not None else None),
|
metadata=(session.metadata if session is not None else None),
|
||||||
),
|
),
|
||||||
goal_active_predicate=lambda: sustained_goal_active(session.metadata) if session is not None else False,
|
|
||||||
goal_continue_message=_goal_continue,
|
|
||||||
))
|
))
|
||||||
finally:
|
finally:
|
||||||
reset_workspace_scope(workspace_token)
|
|
||||||
reset_request_context(request_token)
|
|
||||||
reset_file_states(file_state_token)
|
reset_file_states(file_state_token)
|
||||||
self._last_usage = result.usage
|
self._last_usage = result.usage
|
||||||
if result.stop_reason == "max_iterations":
|
if result.stop_reason == "max_iterations":
|
||||||
@@ -843,15 +812,13 @@ class AgentLoop:
|
|||||||
continue
|
continue
|
||||||
|
|
||||||
raw = msg.content.strip()
|
raw = msg.content.strip()
|
||||||
effective_key = self._effective_session_key(msg)
|
|
||||||
if await agent_context.handle_runtime_control(self, msg, self.tools):
|
|
||||||
continue
|
|
||||||
if self.commands.is_priority(raw):
|
if self.commands.is_priority(raw):
|
||||||
await self._dispatch_command_inline(
|
await self._dispatch_command_inline(
|
||||||
msg, effective_key, raw,
|
msg, msg.session_key, raw,
|
||||||
self.commands.dispatch_priority,
|
self.commands.dispatch_priority,
|
||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
|
effective_key = self._effective_session_key(msg)
|
||||||
# If this session already has an active pending queue (i.e. a task
|
# If this session already has an active pending queue (i.e. a task
|
||||||
# is processing this session), route the message there for mid-turn
|
# is processing this session), route the message there for mid-turn
|
||||||
# injection instead of creating a competing task.
|
# injection instead of creating a competing task.
|
||||||
@@ -902,13 +869,13 @@ class AgentLoop:
|
|||||||
lock = self._session_locks.setdefault(session_key, asyncio.Lock())
|
lock = self._session_locks.setdefault(session_key, asyncio.Lock())
|
||||||
gate = self._concurrency_gate or nullcontext()
|
gate = self._concurrency_gate or nullcontext()
|
||||||
|
|
||||||
pending: asyncio.Queue | None = None
|
# Register a pending queue so follow-up messages for this session are
|
||||||
try:
|
# routed here (mid-turn injection) instead of spawning a new task.
|
||||||
async with lock, gate:
|
|
||||||
# Only the task that owns the session lock may publish the
|
|
||||||
# active mid-turn injection queue for this session.
|
|
||||||
pending = asyncio.Queue(maxsize=20)
|
pending = asyncio.Queue(maxsize=20)
|
||||||
self._pending_queues[session_key] = pending
|
self._pending_queues[session_key] = pending
|
||||||
|
|
||||||
|
try:
|
||||||
|
async with lock, gate:
|
||||||
try:
|
try:
|
||||||
on_stream = on_stream_end = None
|
on_stream = on_stream_end = None
|
||||||
if msg.metadata.get("_wants_stream"):
|
if msg.metadata.get("_wants_stream"):
|
||||||
@@ -995,14 +962,8 @@ class AgentLoop:
|
|||||||
finally:
|
finally:
|
||||||
# Drain any messages still in the pending queue and re-publish
|
# Drain any messages still in the pending queue and re-publish
|
||||||
# them to the bus so they are processed as fresh inbound messages
|
# them to the bus so they are processed as fresh inbound messages
|
||||||
# rather than silently lost. Only remove our own queue; a
|
# rather than silently lost.
|
||||||
# later task waiting on the lock must not be able to steal
|
|
||||||
# cleanup ownership.
|
|
||||||
queue = None
|
|
||||||
if self._pending_queues.get(session_key) is pending:
|
|
||||||
queue = self._pending_queues.pop(session_key, None)
|
queue = self._pending_queues.pop(session_key, None)
|
||||||
else:
|
|
||||||
queue = pending
|
|
||||||
if queue is not None:
|
if queue is not None:
|
||||||
leftover = 0
|
leftover = 0
|
||||||
while True:
|
while True:
|
||||||
@@ -1020,11 +981,6 @@ class AgentLoop:
|
|||||||
await self._webui_turns.publish_run_status(msg, "idle")
|
await self._webui_turns.publish_run_status(msg, "idle")
|
||||||
self._pending_turn_latency_ms.pop(session_key, None)
|
self._pending_turn_latency_ms.pop(session_key, None)
|
||||||
self._webui_turns.discard(session_key)
|
self._webui_turns.discard(session_key)
|
||||||
finally:
|
|
||||||
if pending is None:
|
|
||||||
await self._webui_turns.publish_run_status(msg, "idle")
|
|
||||||
self._pending_turn_latency_ms.pop(session_key, None)
|
|
||||||
self._webui_turns.discard(session_key)
|
|
||||||
|
|
||||||
async def close_mcp(self) -> None:
|
async def close_mcp(self) -> None:
|
||||||
"""Drain pending background archives, then close MCP connections."""
|
"""Drain pending background archives, then close MCP connections."""
|
||||||
@@ -1093,7 +1049,6 @@ class AgentLoop:
|
|||||||
}
|
}
|
||||||
history = session.get_history(**_hist_kwargs)
|
history = session.get_history(**_hist_kwargs)
|
||||||
current_role = "assistant" if is_subagent else "user"
|
current_role = "assistant" if is_subagent else "user"
|
||||||
workspace_scope = self.workspace_scopes.for_message(msg, session.metadata)
|
|
||||||
|
|
||||||
messages = self.context.build_messages(
|
messages = self.context.build_messages(
|
||||||
history=history,
|
history=history,
|
||||||
@@ -1104,10 +1059,6 @@ class AgentLoop:
|
|||||||
sender_id=msg.sender_id,
|
sender_id=msg.sender_id,
|
||||||
session_summary=pending,
|
session_summary=pending,
|
||||||
session_metadata=session.metadata,
|
session_metadata=session.metadata,
|
||||||
workspace=workspace_scope.project_path,
|
|
||||||
runtime_state=self,
|
|
||||||
inbound_message=msg,
|
|
||||||
skip_runtime_lines=is_subagent,
|
|
||||||
)
|
)
|
||||||
t_wall = time.time()
|
t_wall = time.time()
|
||||||
final_content, _, all_msgs, stop_reason, _ = await self._run_agent_loop(
|
final_content, _, all_msgs, stop_reason, _ = await self._run_agent_loop(
|
||||||
@@ -1271,7 +1222,7 @@ class AgentLoop:
|
|||||||
msg = ctx.msg
|
msg = ctx.msg
|
||||||
|
|
||||||
if msg.media:
|
if msg.media:
|
||||||
new_content, image_only = self._prepare_message_media(msg.content, msg.media)
|
new_content, image_only = extract_documents(msg.content, msg.media)
|
||||||
ctx.msg = dataclasses.replace(msg, content=new_content, media=image_only)
|
ctx.msg = dataclasses.replace(msg, content=new_content, media=image_only)
|
||||||
msg = ctx.msg
|
msg = ctx.msg
|
||||||
|
|
||||||
@@ -1283,7 +1234,6 @@ class AgentLoop:
|
|||||||
if ctx.session is None:
|
if ctx.session is None:
|
||||||
ctx.session = self.sessions.get_or_create(ctx.session_key)
|
ctx.session = self.sessions.get_or_create(ctx.session_key)
|
||||||
mark_webui_session(ctx.session, msg.metadata)
|
mark_webui_session(ctx.session, msg.metadata)
|
||||||
self.workspace_scopes.persist_message_scope(ctx.session, msg)
|
|
||||||
|
|
||||||
if self._restore_runtime_checkpoint(ctx.session):
|
if self._restore_runtime_checkpoint(ctx.session):
|
||||||
self.sessions.save(ctx.session)
|
self.sessions.save(ctx.session)
|
||||||
@@ -1292,16 +1242,6 @@ class AgentLoop:
|
|||||||
|
|
||||||
return "ok"
|
return "ok"
|
||||||
|
|
||||||
def _prepare_message_media(self, content: str, media: list[str]) -> tuple[str, list[str]]:
|
|
||||||
if self._should_extract_document_text():
|
|
||||||
return extract_documents(content, media)
|
|
||||||
return reference_non_image_attachments(content, media)
|
|
||||||
|
|
||||||
def _should_extract_document_text(self) -> bool:
|
|
||||||
if self.channels_config is None:
|
|
||||||
return True
|
|
||||||
return self.channels_config.extract_document_text
|
|
||||||
|
|
||||||
async def _state_compact(self, ctx: TurnContext) -> str:
|
async def _state_compact(self, ctx: TurnContext) -> str:
|
||||||
ctx.session, pending = self.auto_compact.prepare_session(ctx.session, ctx.session_key)
|
ctx.session, pending = self.auto_compact.prepare_session(ctx.session, ctx.session_key)
|
||||||
ctx.pending_summary = pending
|
ctx.pending_summary = pending
|
||||||
@@ -1361,10 +1301,7 @@ class AgentLoop:
|
|||||||
)
|
)
|
||||||
|
|
||||||
ctx.initial_messages = self._build_initial_messages(
|
ctx.initial_messages = self._build_initial_messages(
|
||||||
ctx.msg,
|
ctx.msg, ctx.session, ctx.history, ctx.pending_summary
|
||||||
ctx.session,
|
|
||||||
ctx.history,
|
|
||||||
ctx.pending_summary,
|
|
||||||
)
|
)
|
||||||
ctx.user_persisted_early = self._persist_user_message_early(
|
ctx.user_persisted_early = self._persist_user_message_early(
|
||||||
ctx.msg, ctx.session
|
ctx.msg, ctx.session
|
||||||
@@ -1667,10 +1604,6 @@ class AgentLoop:
|
|||||||
channel=channel, sender_id="user", chat_id=chat_id,
|
channel=channel, sender_id="user", chat_id=chat_id,
|
||||||
content=content, media=media or [],
|
content=content, media=media or [],
|
||||||
)
|
)
|
||||||
# Share the dispatch lock so direct calls serialize with bus turns.
|
|
||||||
lock = self._session_locks.setdefault(session_key, asyncio.Lock())
|
|
||||||
try:
|
|
||||||
async with lock:
|
|
||||||
return await self._process_message(
|
return await self._process_message(
|
||||||
msg,
|
msg,
|
||||||
session_key=session_key,
|
session_key=session_key,
|
||||||
@@ -1678,8 +1611,3 @@ class AgentLoop:
|
|||||||
on_stream=on_stream,
|
on_stream=on_stream,
|
||||||
on_stream_end=on_stream_end,
|
on_stream_end=on_stream_end,
|
||||||
)
|
)
|
||||||
finally:
|
|
||||||
if channel == "websocket":
|
|
||||||
await self._webui_turns.publish_run_status(msg, "idle")
|
|
||||||
self._pending_turn_latency_ms.pop(session_key, None)
|
|
||||||
self._webui_turns.discard(session_key)
|
|
||||||
|
|||||||
+18
-58
@@ -8,7 +8,7 @@ import os
|
|||||||
from contextlib import suppress
|
from contextlib import suppress
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Callable
|
from typing import Any
|
||||||
|
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
@@ -16,14 +16,11 @@ from nanobot.agent.hook import AgentHook, AgentHookContext
|
|||||||
from nanobot.agent.tools.registry import ToolRegistry
|
from nanobot.agent.tools.registry import ToolRegistry
|
||||||
from nanobot.providers.base import LLMProvider, LLMResponse, ToolCallRequest
|
from nanobot.providers.base import LLMProvider, LLMResponse, ToolCallRequest
|
||||||
from nanobot.utils.file_edit_events import (
|
from nanobot.utils.file_edit_events import (
|
||||||
StreamingFileEditTracker,
|
|
||||||
build_file_edit_end_event,
|
build_file_edit_end_event,
|
||||||
build_file_edit_error_event,
|
build_file_edit_error_event,
|
||||||
build_file_edit_start_event,
|
build_file_edit_start_event,
|
||||||
prepare_file_edit_trackers,
|
prepare_file_edit_tracker,
|
||||||
)
|
StreamingFileEditTracker,
|
||||||
from nanobot.utils.file_edit_events import (
|
|
||||||
prepare_file_edit_tracker as _prepare_file_edit_tracker,
|
|
||||||
)
|
)
|
||||||
from nanobot.utils.helpers import (
|
from nanobot.utils.helpers import (
|
||||||
IncrementalThinkExtractor,
|
IncrementalThinkExtractor,
|
||||||
@@ -44,7 +41,6 @@ from nanobot.utils.prompt_templates import render_template
|
|||||||
from nanobot.utils.runtime import (
|
from nanobot.utils.runtime import (
|
||||||
EMPTY_FINAL_RESPONSE_MESSAGE,
|
EMPTY_FINAL_RESPONSE_MESSAGE,
|
||||||
build_finalization_retry_message,
|
build_finalization_retry_message,
|
||||||
build_goal_continue_message,
|
|
||||||
build_length_recovery_message,
|
build_length_recovery_message,
|
||||||
ensure_nonempty_tool_result,
|
ensure_nonempty_tool_result,
|
||||||
is_blank_text,
|
is_blank_text,
|
||||||
@@ -53,10 +49,6 @@ from nanobot.utils.runtime import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
_DEFAULT_ERROR_MESSAGE = "Sorry, I encountered an error calling the AI model."
|
_DEFAULT_ERROR_MESSAGE = "Sorry, I encountered an error calling the AI model."
|
||||||
_ARREARAGE_ERROR_MESSAGE = (
|
|
||||||
"The AI provider rejected the request because the API key is out of quota or the "
|
|
||||||
"account is in arrears. Please top up / check the billing status of your API key and try again."
|
|
||||||
)
|
|
||||||
_PERSISTED_MODEL_ERROR_PLACEHOLDER = "[Assistant reply unavailable due to model error.]"
|
_PERSISTED_MODEL_ERROR_PLACEHOLDER = "[Assistant reply unavailable due to model error.]"
|
||||||
_MAX_EMPTY_RETRIES = 2
|
_MAX_EMPTY_RETRIES = 2
|
||||||
_MAX_LENGTH_RECOVERIES = 3
|
_MAX_LENGTH_RECOVERIES = 3
|
||||||
@@ -66,14 +58,11 @@ _SNIP_SAFETY_BUFFER = 1024
|
|||||||
_MICROCOMPACT_KEEP_RECENT = 10
|
_MICROCOMPACT_KEEP_RECENT = 10
|
||||||
_MICROCOMPACT_MIN_CHARS = 500
|
_MICROCOMPACT_MIN_CHARS = 500
|
||||||
_COMPACTABLE_TOOLS = frozenset({
|
_COMPACTABLE_TOOLS = frozenset({
|
||||||
"read_file", "exec", "grep", "find_files",
|
"read_file", "exec", "grep",
|
||||||
"web_search", "web_fetch", "list_dir", "list_exec_sessions",
|
"web_search", "web_fetch", "list_dir",
|
||||||
})
|
})
|
||||||
_BACKFILL_CONTENT = "[Tool result unavailable — call was interrupted or lost]"
|
_BACKFILL_CONTENT = "[Tool result unavailable — call was interrupted or lost]"
|
||||||
|
|
||||||
# Backward-compatible module attribute for tests/extensions that monkeypatch
|
|
||||||
# the former single-file tracker hook. Runtime uses prepare_file_edit_trackers.
|
|
||||||
prepare_file_edit_tracker = _prepare_file_edit_tracker
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
@dataclass(slots=True)
|
||||||
@@ -104,8 +93,6 @@ class AgentRunSpec:
|
|||||||
checkpoint_callback: Any | None = None
|
checkpoint_callback: Any | None = None
|
||||||
injection_callback: Any | None = None
|
injection_callback: Any | None = None
|
||||||
llm_timeout_s: float | None = None
|
llm_timeout_s: float | None = None
|
||||||
goal_active_predicate: Callable[[], bool] | None = None
|
|
||||||
goal_continue_message: str | None = None
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
@dataclass(slots=True)
|
||||||
@@ -176,7 +163,6 @@ class AgentRunner:
|
|||||||
*,
|
*,
|
||||||
phase: str = "after error",
|
phase: str = "after error",
|
||||||
iteration: int | None = None,
|
iteration: int | None = None,
|
||||||
allow_goal_continue: bool = False,
|
|
||||||
) -> tuple[bool, int]:
|
) -> tuple[bool, int]:
|
||||||
"""Drain pending injections. Returns (should_continue, updated_cycles).
|
"""Drain pending injections. Returns (should_continue, updated_cycles).
|
||||||
|
|
||||||
@@ -185,18 +171,11 @@ class AgentRunner:
|
|||||||
and *iteration* are both provided) and return (True, cycles+1) so the
|
and *iteration* are both provided) and return (True, cycles+1) so the
|
||||||
caller continues the iteration loop. Otherwise return (False, cycles).
|
caller continues the iteration loop. Otherwise return (False, cycles).
|
||||||
"""
|
"""
|
||||||
injections: list[dict[str, Any]] = []
|
if injection_cycles >= _MAX_INJECTION_CYCLES:
|
||||||
real_injection = False
|
return False, injection_cycles
|
||||||
if injection_cycles < _MAX_INJECTION_CYCLES:
|
|
||||||
injections = await self._drain_injections(spec)
|
injections = await self._drain_injections(spec)
|
||||||
real_injection = bool(injections)
|
|
||||||
if not injections and allow_goal_continue and assistant_message is not None:
|
|
||||||
predicate = spec.goal_active_predicate
|
|
||||||
if predicate is not None and predicate():
|
|
||||||
injections = [build_goal_continue_message(spec.goal_continue_message)]
|
|
||||||
if not injections:
|
if not injections:
|
||||||
return False, injection_cycles
|
return False, injection_cycles
|
||||||
if real_injection:
|
|
||||||
injection_cycles += 1
|
injection_cycles += 1
|
||||||
if assistant_message is not None:
|
if assistant_message is not None:
|
||||||
messages.append(assistant_message)
|
messages.append(assistant_message)
|
||||||
@@ -213,13 +192,10 @@ class AgentRunner:
|
|||||||
},
|
},
|
||||||
)
|
)
|
||||||
self._append_injected_messages(messages, injections)
|
self._append_injected_messages(messages, injections)
|
||||||
if real_injection:
|
|
||||||
logger.info(
|
logger.info(
|
||||||
"Injected {} follow-up message(s) {} ({}/{})",
|
"Injected {} follow-up message(s) {} ({}/{})",
|
||||||
len(injections), phase, injection_cycles, _MAX_INJECTION_CYCLES,
|
len(injections), phase, injection_cycles, _MAX_INJECTION_CYCLES,
|
||||||
)
|
)
|
||||||
else:
|
|
||||||
logger.info("Injected sustained-goal continuation {}", phase)
|
|
||||||
return True, injection_cycles
|
return True, injection_cycles
|
||||||
|
|
||||||
async def _drain_injections(self, spec: AgentRunSpec) -> list[dict[str, Any]]:
|
async def _drain_injections(self, spec: AgentRunSpec) -> list[dict[str, Any]]:
|
||||||
@@ -495,7 +471,6 @@ class AgentRunner:
|
|||||||
spec, messages, assistant_message, injection_cycles,
|
spec, messages, assistant_message, injection_cycles,
|
||||||
phase="after final response",
|
phase="after final response",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
allow_goal_continue=True,
|
|
||||||
)
|
)
|
||||||
if should_continue:
|
if should_continue:
|
||||||
had_injections = True
|
had_injections = True
|
||||||
@@ -508,9 +483,6 @@ class AgentRunner:
|
|||||||
continue
|
continue
|
||||||
|
|
||||||
if response.finish_reason == "error":
|
if response.finish_reason == "error":
|
||||||
if LLMProvider.is_arrearage_response(response):
|
|
||||||
final_content = _ARREARAGE_ERROR_MESSAGE
|
|
||||||
else:
|
|
||||||
final_content = clean or spec.error_message or _DEFAULT_ERROR_MESSAGE
|
final_content = clean or spec.error_message or _DEFAULT_ERROR_MESSAGE
|
||||||
stop_reason = "error"
|
stop_reason = "error"
|
||||||
error = final_content
|
error = final_content
|
||||||
@@ -885,8 +857,8 @@ class AgentRunner:
|
|||||||
and on_progress_accepts_file_edit_events(spec.progress_callback)
|
and on_progress_accepts_file_edit_events(spec.progress_callback)
|
||||||
)
|
)
|
||||||
progress_callback = spec.progress_callback if emit_file_edit_events else None
|
progress_callback = spec.progress_callback if emit_file_edit_events else None
|
||||||
file_edit_trackers = (
|
file_edit_tracker = (
|
||||||
prepare_file_edit_trackers(
|
prepare_file_edit_tracker(
|
||||||
call_id=tool_call.id,
|
call_id=tool_call.id,
|
||||||
tool_name=tool_call.name,
|
tool_name=tool_call.name,
|
||||||
tool=tool,
|
tool=tool,
|
||||||
@@ -896,13 +868,13 @@ class AgentRunner:
|
|||||||
if progress_callback is not None
|
if progress_callback is not None
|
||||||
else None
|
else None
|
||||||
)
|
)
|
||||||
if file_edit_trackers and progress_callback is not None:
|
if file_edit_tracker is not None and progress_callback is not None:
|
||||||
await invoke_file_edit_progress(
|
await invoke_file_edit_progress(
|
||||||
progress_callback,
|
progress_callback,
|
||||||
[build_file_edit_start_event(
|
[build_file_edit_start_event(
|
||||||
file_edit_tracker,
|
file_edit_tracker,
|
||||||
params if isinstance(params, dict) else None,
|
params if isinstance(params, dict) else None,
|
||||||
) for file_edit_tracker in file_edit_trackers],
|
)],
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
if tool is not None:
|
if tool is not None:
|
||||||
@@ -912,13 +884,10 @@ class AgentRunner:
|
|||||||
except asyncio.CancelledError:
|
except asyncio.CancelledError:
|
||||||
raise
|
raise
|
||||||
except BaseException as exc:
|
except BaseException as exc:
|
||||||
if file_edit_trackers and progress_callback is not None:
|
if file_edit_tracker is not None and progress_callback is not None:
|
||||||
await invoke_file_edit_progress(
|
await invoke_file_edit_progress(
|
||||||
progress_callback,
|
progress_callback,
|
||||||
[
|
[build_file_edit_error_event(file_edit_tracker, str(exc))],
|
||||||
build_file_edit_error_event(file_edit_tracker, str(exc))
|
|
||||||
for file_edit_tracker in file_edit_trackers
|
|
||||||
],
|
|
||||||
)
|
)
|
||||||
event = {
|
event = {
|
||||||
"name": tool_call.name,
|
"name": tool_call.name,
|
||||||
@@ -941,13 +910,10 @@ class AgentRunner:
|
|||||||
return payload, event, None
|
return payload, event, None
|
||||||
|
|
||||||
if isinstance(result, str) and result.startswith("Error"):
|
if isinstance(result, str) and result.startswith("Error"):
|
||||||
if file_edit_trackers and progress_callback is not None:
|
if file_edit_tracker is not None and progress_callback is not None:
|
||||||
await invoke_file_edit_progress(
|
await invoke_file_edit_progress(
|
||||||
progress_callback,
|
progress_callback,
|
||||||
[
|
[build_file_edit_error_event(file_edit_tracker, result)],
|
||||||
build_file_edit_error_event(file_edit_tracker, result)
|
|
||||||
for file_edit_tracker in file_edit_trackers
|
|
||||||
],
|
|
||||||
)
|
)
|
||||||
event = {
|
event = {
|
||||||
"name": tool_call.name,
|
"name": tool_call.name,
|
||||||
@@ -967,13 +933,13 @@ class AgentRunner:
|
|||||||
return result + hint, event, RuntimeError(result)
|
return result + hint, event, RuntimeError(result)
|
||||||
return result + hint, event, None
|
return result + hint, event, None
|
||||||
|
|
||||||
if file_edit_trackers and progress_callback is not None:
|
if file_edit_tracker is not None and progress_callback is not None:
|
||||||
await invoke_file_edit_progress(
|
await invoke_file_edit_progress(
|
||||||
progress_callback,
|
progress_callback,
|
||||||
[build_file_edit_end_event(
|
[build_file_edit_end_event(
|
||||||
file_edit_tracker,
|
file_edit_tracker,
|
||||||
params if isinstance(params, dict) else None,
|
params if isinstance(params, dict) else None,
|
||||||
) for file_edit_tracker in file_edit_trackers],
|
)],
|
||||||
)
|
)
|
||||||
|
|
||||||
detail = "" if result is None else str(result)
|
detail = "" if result is None else str(result)
|
||||||
@@ -1280,13 +1246,7 @@ class AgentRunner:
|
|||||||
return messages
|
return messages
|
||||||
|
|
||||||
system_tokens = sum(estimate_message_tokens(msg) for msg in system_messages)
|
system_tokens = sum(estimate_message_tokens(msg) for msg in system_messages)
|
||||||
fixed_tokens, _ = estimate_prompt_tokens_chain(
|
remaining_budget = max(128, budget - system_tokens)
|
||||||
self.provider,
|
|
||||||
spec.model,
|
|
||||||
system_messages,
|
|
||||||
spec.tools.get_definitions(),
|
|
||||||
)
|
|
||||||
remaining_budget = max(0, budget - max(system_tokens, fixed_tokens))
|
|
||||||
kept: list[dict[str, Any]] = []
|
kept: list[dict[str, Any]] = []
|
||||||
kept_tokens = 0
|
kept_tokens = 0
|
||||||
for message in reversed(non_system):
|
for message in reversed(non_system):
|
||||||
|
|||||||
@@ -16,12 +16,6 @@ from nanobot.agent.tools.context import ToolContext
|
|||||||
from nanobot.agent.tools.file_state import FileStates
|
from nanobot.agent.tools.file_state import FileStates
|
||||||
from nanobot.agent.tools.loader import ToolLoader
|
from nanobot.agent.tools.loader import ToolLoader
|
||||||
from nanobot.agent.tools.registry import ToolRegistry
|
from nanobot.agent.tools.registry import ToolRegistry
|
||||||
from nanobot.security.workspace_access import (
|
|
||||||
WorkspaceScope,
|
|
||||||
bind_workspace_scope,
|
|
||||||
reset_workspace_scope,
|
|
||||||
workspace_sandbox_status,
|
|
||||||
)
|
|
||||||
from nanobot.bus.events import InboundMessage
|
from nanobot.bus.events import InboundMessage
|
||||||
from nanobot.bus.queue import MessageBus
|
from nanobot.bus.queue import MessageBus
|
||||||
from nanobot.config.schema import AgentDefaults, ToolsConfig
|
from nanobot.config.schema import AgentDefaults, ToolsConfig
|
||||||
@@ -85,7 +79,6 @@ class SubagentManager:
|
|||||||
restrict_to_workspace: bool = False,
|
restrict_to_workspace: bool = False,
|
||||||
disabled_skills: list[str] | None = None,
|
disabled_skills: list[str] | None = None,
|
||||||
max_iterations: int | None = None,
|
max_iterations: int | None = None,
|
||||||
max_concurrent_subagents: int | None = None,
|
|
||||||
llm_wall_timeout_for_session: Callable[[str | None], float | None] | None = None,
|
llm_wall_timeout_for_session: Callable[[str | None], float | None] | None = None,
|
||||||
):
|
):
|
||||||
defaults = AgentDefaults()
|
defaults = AgentDefaults()
|
||||||
@@ -102,11 +95,7 @@ class SubagentManager:
|
|||||||
if max_iterations is not None
|
if max_iterations is not None
|
||||||
else defaults.max_tool_iterations
|
else defaults.max_tool_iterations
|
||||||
)
|
)
|
||||||
self.max_concurrent_subagents = (
|
self.max_concurrent_subagents = defaults.max_concurrent_subagents
|
||||||
max_concurrent_subagents
|
|
||||||
if max_concurrent_subagents is not None
|
|
||||||
else defaults.max_concurrent_subagents
|
|
||||||
)
|
|
||||||
self.runner = AgentRunner(provider)
|
self.runner = AgentRunner(provider)
|
||||||
self._llm_wall_timeout_for_session = llm_wall_timeout_for_session
|
self._llm_wall_timeout_for_session = llm_wall_timeout_for_session
|
||||||
self._running_tasks: dict[str, asyncio.Task[None]] = {}
|
self._running_tasks: dict[str, asyncio.Task[None]] = {}
|
||||||
@@ -134,10 +123,6 @@ class SubagentManager:
|
|||||||
config=cfg,
|
config=cfg,
|
||||||
workspace=str(root.resolve()),
|
workspace=str(root.resolve()),
|
||||||
file_state_store=FileStates(),
|
file_state_store=FileStates(),
|
||||||
workspace_sandbox=workspace_sandbox_status(
|
|
||||||
restrict_to_workspace=cfg.restrict_to_workspace,
|
|
||||||
workspace=root,
|
|
||||||
),
|
|
||||||
)
|
)
|
||||||
ToolLoader().load(ctx, registry, scope="subagent")
|
ToolLoader().load(ctx, registry, scope="subagent")
|
||||||
return registry
|
return registry
|
||||||
@@ -155,8 +140,6 @@ class SubagentManager:
|
|||||||
origin_chat_id: str = "direct",
|
origin_chat_id: str = "direct",
|
||||||
session_key: str | None = None,
|
session_key: str | None = None,
|
||||||
origin_message_id: str | None = None,
|
origin_message_id: str | None = None,
|
||||||
temperature: float | None = None,
|
|
||||||
workspace_scope: WorkspaceScope | None = None,
|
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Spawn a subagent to execute a task in the background."""
|
"""Spawn a subagent to execute a task in the background."""
|
||||||
task_id = str(uuid.uuid4())[:8]
|
task_id = str(uuid.uuid4())[:8]
|
||||||
@@ -172,16 +155,7 @@ class SubagentManager:
|
|||||||
self._task_statuses[task_id] = status
|
self._task_statuses[task_id] = status
|
||||||
|
|
||||||
bg_task = asyncio.create_task(
|
bg_task = asyncio.create_task(
|
||||||
self._run_subagent(
|
self._run_subagent(task_id, task, display_label, origin, status, origin_message_id)
|
||||||
task_id,
|
|
||||||
task,
|
|
||||||
display_label,
|
|
||||||
origin,
|
|
||||||
status,
|
|
||||||
origin_message_id,
|
|
||||||
temperature,
|
|
||||||
workspace_scope,
|
|
||||||
)
|
|
||||||
)
|
)
|
||||||
self._running_tasks[task_id] = bg_task
|
self._running_tasks[task_id] = bg_task
|
||||||
if session_key:
|
if session_key:
|
||||||
@@ -208,8 +182,6 @@ class SubagentManager:
|
|||||||
origin: dict[str, str],
|
origin: dict[str, str],
|
||||||
status: SubagentStatus,
|
status: SubagentStatus,
|
||||||
origin_message_id: str | None = None,
|
origin_message_id: str | None = None,
|
||||||
temperature: float | None = None,
|
|
||||||
workspace_scope: WorkspaceScope | None = None,
|
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Execute the subagent task and announce the result."""
|
"""Execute the subagent task and announce the result."""
|
||||||
logger.info("Subagent [{}] starting task: {}", task_id, label)
|
logger.info("Subagent [{}] starting task: {}", task_id, label)
|
||||||
@@ -219,13 +191,8 @@ class SubagentManager:
|
|||||||
status.iteration = payload.get("iteration", status.iteration)
|
status.iteration = payload.get("iteration", status.iteration)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
root = workspace_scope.project_path if workspace_scope is not None else self.workspace
|
tools = self._build_tools()
|
||||||
cfg = None
|
system_prompt = self._build_subagent_prompt()
|
||||||
if workspace_scope is not None:
|
|
||||||
cfg = self._subagent_tools_config()
|
|
||||||
cfg.restrict_to_workspace = workspace_scope.restrict_to_workspace
|
|
||||||
tools = self._build_tools(workspace=root, tools_config=cfg)
|
|
||||||
system_prompt = self._build_subagent_prompt(workspace=root)
|
|
||||||
messages: list[dict[str, Any]] = [
|
messages: list[dict[str, Any]] = [
|
||||||
{"role": "system", "content": system_prompt},
|
{"role": "system", "content": system_prompt},
|
||||||
{"role": "user", "content": task},
|
{"role": "user", "content": task},
|
||||||
@@ -237,13 +204,10 @@ class SubagentManager:
|
|||||||
if self._llm_wall_timeout_for_session
|
if self._llm_wall_timeout_for_session
|
||||||
else None
|
else None
|
||||||
)
|
)
|
||||||
token = bind_workspace_scope(workspace_scope) if workspace_scope is not None else None
|
|
||||||
try:
|
|
||||||
result = await self.runner.run(AgentRunSpec(
|
result = await self.runner.run(AgentRunSpec(
|
||||||
initial_messages=messages,
|
initial_messages=messages,
|
||||||
tools=tools,
|
tools=tools,
|
||||||
model=self.model,
|
model=self.model,
|
||||||
temperature=temperature,
|
|
||||||
max_iterations=self.max_iterations,
|
max_iterations=self.max_iterations,
|
||||||
max_tool_result_chars=self.max_tool_result_chars,
|
max_tool_result_chars=self.max_tool_result_chars,
|
||||||
hook=_SubagentHook(task_id, status),
|
hook=_SubagentHook(task_id, status),
|
||||||
@@ -252,12 +216,8 @@ class SubagentManager:
|
|||||||
fail_on_tool_error=True,
|
fail_on_tool_error=True,
|
||||||
checkpoint_callback=_on_checkpoint,
|
checkpoint_callback=_on_checkpoint,
|
||||||
session_key=sess_key,
|
session_key=sess_key,
|
||||||
workspace=root,
|
|
||||||
llm_timeout_s=llm_timeout,
|
llm_timeout_s=llm_timeout,
|
||||||
))
|
))
|
||||||
finally:
|
|
||||||
if token is not None:
|
|
||||||
reset_workspace_scope(token)
|
|
||||||
status.phase = "done"
|
status.phase = "done"
|
||||||
status.stop_reason = result.stop_reason
|
status.stop_reason = result.stop_reason
|
||||||
|
|
||||||
@@ -351,21 +311,20 @@ class SubagentManager:
|
|||||||
lines.append(f"- {result.error}")
|
lines.append(f"- {result.error}")
|
||||||
return "\n".join(lines) or (result.error or "Error: subagent execution failed.")
|
return "\n".join(lines) or (result.error or "Error: subagent execution failed.")
|
||||||
|
|
||||||
def _build_subagent_prompt(self, workspace: Path | None = None) -> str:
|
def _build_subagent_prompt(self) -> str:
|
||||||
"""Build a focused system prompt for the subagent."""
|
"""Build a focused system prompt for the subagent."""
|
||||||
from nanobot.agent.context import ContextBuilder
|
from nanobot.agent.context import ContextBuilder
|
||||||
from nanobot.agent.skills import SkillsLoader
|
from nanobot.agent.skills import SkillsLoader
|
||||||
|
|
||||||
time_ctx = ContextBuilder._build_runtime_context(None, None)
|
time_ctx = ContextBuilder._build_runtime_context(None, None)
|
||||||
root = workspace or self.workspace
|
|
||||||
skills_summary = SkillsLoader(
|
skills_summary = SkillsLoader(
|
||||||
root,
|
self.workspace,
|
||||||
disabled_skills=self.disabled_skills,
|
disabled_skills=self.disabled_skills,
|
||||||
).build_skills_summary()
|
).build_skills_summary()
|
||||||
return render_template(
|
return render_template(
|
||||||
"agent/subagent_system.md",
|
"agent/subagent_system.md",
|
||||||
time_ctx=time_ctx,
|
time_ctx=time_ctx,
|
||||||
workspace=str(root),
|
workspace=str(self.workspace),
|
||||||
skills_summary=skills_summary or "",
|
skills_summary=skills_summary or "",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -1,290 +0,0 @@
|
|||||||
"""Apply file edits by providing structured edit instructions."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import difflib
|
|
||||||
import re
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from nanobot.agent.tools.base import tool_parameters
|
|
||||||
from nanobot.agent.tools.filesystem import _FsTool
|
|
||||||
from nanobot.agent.tools.schema import (
|
|
||||||
ArraySchema,
|
|
||||||
BooleanSchema,
|
|
||||||
ObjectSchema,
|
|
||||||
StringSchema,
|
|
||||||
tool_parameters_schema,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
|
||||||
class _PatchSummary:
|
|
||||||
action: str
|
|
||||||
path: str
|
|
||||||
added: int = 0
|
|
||||||
deleted: int = 0
|
|
||||||
|
|
||||||
|
|
||||||
class _PatchError(ValueError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
_ABSOLUTE_WINDOWS_RE = re.compile(r"^[A-Za-z]:[\\/]")
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_relative_path(path: str) -> str:
|
|
||||||
normalized = path.strip()
|
|
||||||
if not normalized:
|
|
||||||
raise _PatchError("patch path cannot be empty")
|
|
||||||
if "\0" in normalized:
|
|
||||||
raise _PatchError(f"patch path contains a null byte: {path!r}")
|
|
||||||
if normalized.startswith(("~", "/", "\\")) or _ABSOLUTE_WINDOWS_RE.match(normalized):
|
|
||||||
raise _PatchError(f"patch path must be relative: {path}")
|
|
||||||
if any(part == ".." for part in re.split(r"[\\/]+", normalized)):
|
|
||||||
raise _PatchError(f"patch path must not contain '..': {path}")
|
|
||||||
return normalized
|
|
||||||
|
|
||||||
|
|
||||||
def _lines_to_text(lines: list[str]) -> str:
|
|
||||||
if not lines:
|
|
||||||
return ""
|
|
||||||
return "\n".join(lines) + "\n"
|
|
||||||
|
|
||||||
|
|
||||||
def _text_line_count(text: str) -> int:
|
|
||||||
if not text:
|
|
||||||
return 0
|
|
||||||
return len(text.splitlines())
|
|
||||||
|
|
||||||
|
|
||||||
def _line_diff_stats(before: str, after: str) -> tuple[int, int]:
|
|
||||||
before_lines = before.replace("\r\n", "\n").splitlines()
|
|
||||||
after_lines = after.replace("\r\n", "\n").splitlines()
|
|
||||||
added = 0
|
|
||||||
deleted = 0
|
|
||||||
matcher = difflib.SequenceMatcher(a=before_lines, b=after_lines, autojunk=False)
|
|
||||||
for tag, i1, i2, j1, j2 in matcher.get_opcodes():
|
|
||||||
if tag == "equal":
|
|
||||||
continue
|
|
||||||
if tag in ("replace", "delete"):
|
|
||||||
deleted += i2 - i1
|
|
||||||
if tag in ("replace", "insert"):
|
|
||||||
added += j2 - j1
|
|
||||||
return added, deleted
|
|
||||||
|
|
||||||
|
|
||||||
def _format_summary(summary: _PatchSummary) -> str:
|
|
||||||
stats = ""
|
|
||||||
if summary.added or summary.deleted:
|
|
||||||
stats = f" (+{summary.added}/-{summary.deleted})"
|
|
||||||
return f"- {summary.action} {summary.path}{stats}"
|
|
||||||
|
|
||||||
|
|
||||||
@tool_parameters(
|
|
||||||
tool_parameters_schema(
|
|
||||||
edits=ArraySchema(
|
|
||||||
items=ObjectSchema(
|
|
||||||
path=StringSchema("Relative path to the file to edit."),
|
|
||||||
action=StringSchema(
|
|
||||||
"Operation type: replace or add.",
|
|
||||||
enum=["replace", "add"],
|
|
||||||
),
|
|
||||||
old_text=StringSchema(
|
|
||||||
"Exact text to search for in the file. Required for replace.",
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
new_text=StringSchema(
|
|
||||||
"Text to replace with or append. Required for replace and add.",
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
required=["path", "action"],
|
|
||||||
),
|
|
||||||
description="List of edits to apply. Each edit specifies a file and the change to make.",
|
|
||||||
min_items=1,
|
|
||||||
max_items=20,
|
|
||||||
),
|
|
||||||
dry_run=BooleanSchema(
|
|
||||||
description="Validate and summarize the patch without writing files.",
|
|
||||||
default=False,
|
|
||||||
),
|
|
||||||
required=["edits"],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
class ApplyPatchTool(_FsTool):
|
|
||||||
"""Apply file edits by providing structured edit instructions."""
|
|
||||||
_scopes = {"core", "subagent"}
|
|
||||||
|
|
||||||
@property
|
|
||||||
def name(self) -> str:
|
|
||||||
return "apply_patch"
|
|
||||||
|
|
||||||
@property
|
|
||||||
def description(self) -> str:
|
|
||||||
return (
|
|
||||||
"Default tool for code edits. Supports multi-file changes in a single call. "
|
|
||||||
"Provide a list of structured edits, each specifying a file path, action "
|
|
||||||
"(replace/add), and the exact text to change. "
|
|
||||||
"Paths must be relative. Set dry_run=true to validate and preview without writing files. "
|
|
||||||
"Use edit_file only for small exact replacements on a single file."
|
|
||||||
)
|
|
||||||
|
|
||||||
async def execute(
|
|
||||||
self,
|
|
||||||
edits: list[dict] | None = None,
|
|
||||||
dry_run: bool = False,
|
|
||||||
**kwargs: Any,
|
|
||||||
) -> str:
|
|
||||||
try:
|
|
||||||
if not edits:
|
|
||||||
raise _PatchError("must provide edits")
|
|
||||||
|
|
||||||
writes: dict[Path, str] = {}
|
|
||||||
summaries: list[_PatchSummary] = []
|
|
||||||
|
|
||||||
for edit in edits:
|
|
||||||
if not isinstance(edit, dict):
|
|
||||||
raise _PatchError("each edit must be an object")
|
|
||||||
raw_path = edit.get("path")
|
|
||||||
if not isinstance(raw_path, str):
|
|
||||||
raise _PatchError("path required for edit")
|
|
||||||
path = _validate_relative_path(raw_path)
|
|
||||||
action = edit.get("action")
|
|
||||||
if not isinstance(action, str):
|
|
||||||
raise _PatchError(f"action required for edit: {path}")
|
|
||||||
source = self._resolve(path)
|
|
||||||
|
|
||||||
if action == "add":
|
|
||||||
new_text = edit.get("new_text")
|
|
||||||
if new_text is None:
|
|
||||||
raise _PatchError(f"new_text required for add: {path}")
|
|
||||||
|
|
||||||
pending = writes.get(source)
|
|
||||||
if pending is not None:
|
|
||||||
content = pending
|
|
||||||
exists = True
|
|
||||||
elif source.exists():
|
|
||||||
raw = source.read_bytes()
|
|
||||||
try:
|
|
||||||
content = raw.decode("utf-8")
|
|
||||||
except UnicodeDecodeError:
|
|
||||||
raise _PatchError(f"file is not UTF-8 text: {path}")
|
|
||||||
exists = True
|
|
||||||
else:
|
|
||||||
content = ""
|
|
||||||
exists = False
|
|
||||||
|
|
||||||
if exists:
|
|
||||||
uses_crlf = "\r\n" in content
|
|
||||||
new_norm = content.replace("\r\n", "\n") + new_text.replace("\r\n", "\n")
|
|
||||||
if new_norm and not new_norm.endswith("\n"):
|
|
||||||
new_norm += "\n"
|
|
||||||
if uses_crlf:
|
|
||||||
new_norm = new_norm.replace("\n", "\r\n")
|
|
||||||
writes[source] = new_norm
|
|
||||||
added, deleted = _line_diff_stats(content, new_norm)
|
|
||||||
action_name = "update"
|
|
||||||
else:
|
|
||||||
new_norm = new_text.replace("\r\n", "\n")
|
|
||||||
if new_norm and not new_norm.endswith("\n"):
|
|
||||||
new_norm += "\n"
|
|
||||||
writes[source] = new_norm
|
|
||||||
added = _text_line_count(new_norm)
|
|
||||||
deleted = 0
|
|
||||||
action_name = "add"
|
|
||||||
|
|
||||||
summaries.append(
|
|
||||||
_PatchSummary(
|
|
||||||
action=action_name, path=path, added=added, deleted=deleted
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
elif action == "replace":
|
|
||||||
old_text = edit.get("old_text") or ""
|
|
||||||
if not old_text:
|
|
||||||
raise _PatchError(f"old_text required for replace: {path}")
|
|
||||||
new_text = edit.get("new_text")
|
|
||||||
if new_text is None:
|
|
||||||
raise _PatchError(f"new_text required for replace: {path}")
|
|
||||||
|
|
||||||
pending = writes.get(source)
|
|
||||||
if pending is not None:
|
|
||||||
content = pending
|
|
||||||
elif source.exists():
|
|
||||||
raw = source.read_bytes()
|
|
||||||
try:
|
|
||||||
content = raw.decode("utf-8")
|
|
||||||
except UnicodeDecodeError:
|
|
||||||
raise _PatchError(f"file is not UTF-8 text: {path}")
|
|
||||||
else:
|
|
||||||
raise _PatchError(f"file to update does not exist: {path}")
|
|
||||||
|
|
||||||
if pending is None and not source.is_file():
|
|
||||||
raise _PatchError(f"path to update is not a file: {path}")
|
|
||||||
|
|
||||||
uses_crlf = "\r\n" in content
|
|
||||||
norm_content = content.replace("\r\n", "\n")
|
|
||||||
norm_old = old_text.replace("\r\n", "\n")
|
|
||||||
|
|
||||||
pos = norm_content.find(norm_old)
|
|
||||||
if pos < 0:
|
|
||||||
raise _PatchError(f"old_text not found in {path}")
|
|
||||||
if norm_content.find(norm_old, pos + 1) >= 0:
|
|
||||||
raise _PatchError(f"old_text appears multiple times in {path}")
|
|
||||||
|
|
||||||
new_norm = (
|
|
||||||
norm_content[:pos]
|
|
||||||
+ new_text.replace("\r\n", "\n")
|
|
||||||
+ norm_content[pos + len(norm_old) :]
|
|
||||||
)
|
|
||||||
if new_norm and not new_norm.endswith("\n"):
|
|
||||||
new_norm += "\n"
|
|
||||||
if uses_crlf:
|
|
||||||
new_norm = new_norm.replace("\n", "\r\n")
|
|
||||||
|
|
||||||
writes[source] = new_norm
|
|
||||||
added, deleted = _line_diff_stats(content, new_norm)
|
|
||||||
summaries.append(
|
|
||||||
_PatchSummary(
|
|
||||||
action="update", path=path, added=added, deleted=deleted
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
else:
|
|
||||||
raise _PatchError(f"unknown action: {action}")
|
|
||||||
|
|
||||||
if dry_run:
|
|
||||||
return "Patch dry-run succeeded:\n" + "\n".join(
|
|
||||||
_format_summary(summary) for summary in summaries
|
|
||||||
)
|
|
||||||
|
|
||||||
backups: dict[Path, bytes | None] = {}
|
|
||||||
for path in writes:
|
|
||||||
backups[path] = path.read_bytes() if path.exists() else None
|
|
||||||
|
|
||||||
try:
|
|
||||||
for path, content in writes.items():
|
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
path.write_text(content, encoding="utf-8", newline="")
|
|
||||||
except Exception:
|
|
||||||
for path, data in backups.items():
|
|
||||||
if data is None:
|
|
||||||
if path.exists():
|
|
||||||
path.unlink()
|
|
||||||
else:
|
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
path.write_bytes(data)
|
|
||||||
raise
|
|
||||||
|
|
||||||
for path in writes:
|
|
||||||
self._file_states.record_write(path)
|
|
||||||
return "Patch applied:\n" + "\n".join(
|
|
||||||
_format_summary(summary) for summary in summaries
|
|
||||||
)
|
|
||||||
except PermissionError as exc:
|
|
||||||
return f"Error: {exc}"
|
|
||||||
except _PatchError as exc:
|
|
||||||
return f"Error applying patch: {exc}"
|
|
||||||
except Exception as exc:
|
|
||||||
return f"Error applying patch: {exc}"
|
|
||||||
@@ -1,133 +0,0 @@
|
|||||||
"""Controlled runner for installed CLI Apps."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from pydantic import Field
|
|
||||||
|
|
||||||
from nanobot.agent.tools.base import Tool, tool_parameters
|
|
||||||
from nanobot.agent.tools.schema import ArraySchema, BooleanSchema, IntegerSchema, StringSchema, tool_parameters_schema
|
|
||||||
from nanobot.security.workspace_access import current_tool_workspace
|
|
||||||
from nanobot.apps.cli import CliAppError, CliAppManager, CliAppsRuntimeConfig
|
|
||||||
from nanobot.config.schema import Base
|
|
||||||
|
|
||||||
|
|
||||||
class CliAppsToolConfig(Base):
|
|
||||||
"""CLI Apps tool configuration."""
|
|
||||||
|
|
||||||
enable: bool = True
|
|
||||||
install_timeout: int = Field(default=300, ge=1, le=3600)
|
|
||||||
run_timeout: int = Field(default=60, ge=1, le=600)
|
|
||||||
catalog_ttl_seconds: int = Field(default=3600, ge=60, le=86_400)
|
|
||||||
|
|
||||||
|
|
||||||
@tool_parameters(
|
|
||||||
tool_parameters_schema(
|
|
||||||
required=["name"],
|
|
||||||
name=StringSchema("Installed CLI app registry name, for example gimp, safari, or obsidian."),
|
|
||||||
args=ArraySchema(
|
|
||||||
StringSchema("One command-line argument."),
|
|
||||||
description="Arguments to pass to the CLI entry point. Do not include the entry point itself.",
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
json=BooleanSchema(
|
|
||||||
description="Whether to prepend --json when supported by the CLI.",
|
|
||||||
default=False,
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
working_dir=StringSchema("Optional working directory for the CLI call.", nullable=True),
|
|
||||||
timeout=IntegerSchema(
|
|
||||||
description="Timeout in seconds for this CLI call.",
|
|
||||||
minimum=1,
|
|
||||||
maximum=600,
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
class CliAppsTool(Tool):
|
|
||||||
"""Run an installed CLI-Anything or public CLI app through a controlled argv subprocess."""
|
|
||||||
|
|
||||||
config_key = "cli_apps"
|
|
||||||
_scopes = {"core", "subagent"}
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def config_cls(cls):
|
|
||||||
return CliAppsToolConfig
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def enabled(cls, ctx: Any) -> bool:
|
|
||||||
return ctx.config.cli_apps.enable
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def create(cls, ctx: Any) -> Tool:
|
|
||||||
cfg = ctx.config.cli_apps
|
|
||||||
return cls(
|
|
||||||
workspace=Path(ctx.workspace),
|
|
||||||
restrict_to_workspace=ctx.config.restrict_to_workspace,
|
|
||||||
runtime=CliAppsRuntimeConfig(
|
|
||||||
install_timeout=cfg.install_timeout,
|
|
||||||
run_timeout=cfg.run_timeout,
|
|
||||||
catalog_ttl_seconds=cfg.catalog_ttl_seconds,
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
workspace: Path,
|
|
||||||
restrict_to_workspace: bool = False,
|
|
||||||
runtime: CliAppsRuntimeConfig | None = None,
|
|
||||||
) -> None:
|
|
||||||
self.workspace = workspace
|
|
||||||
self.restrict_to_workspace = restrict_to_workspace
|
|
||||||
self.runtime = runtime or CliAppsRuntimeConfig()
|
|
||||||
|
|
||||||
@property
|
|
||||||
def name(self) -> str:
|
|
||||||
return "run_cli_app"
|
|
||||||
|
|
||||||
@property
|
|
||||||
def description(self) -> str:
|
|
||||||
try:
|
|
||||||
installed = CliAppManager(workspace=self.workspace, runtime=self.runtime).installed_names()
|
|
||||||
except Exception:
|
|
||||||
installed = []
|
|
||||||
installed_note = (
|
|
||||||
f" Installed Settings CLI Apps: {', '.join(installed)}."
|
|
||||||
if installed
|
|
||||||
else " No Settings CLI Apps are currently installed."
|
|
||||||
)
|
|
||||||
return (
|
|
||||||
"Run a CLI App that the user explicitly installed in Settings or attached as @app. "
|
|
||||||
"Do not use this for ordinary system CLIs such as git, gh, python, npm, or brew; "
|
|
||||||
"unknown names are rejected. Execution uses argv, not shell."
|
|
||||||
+ installed_note
|
|
||||||
)
|
|
||||||
|
|
||||||
async def execute(
|
|
||||||
self,
|
|
||||||
name: str,
|
|
||||||
args: list[str] | None = None,
|
|
||||||
json: bool | None = False,
|
|
||||||
working_dir: str | None = None,
|
|
||||||
timeout: int | None = None,
|
|
||||||
) -> str:
|
|
||||||
access = current_tool_workspace(
|
|
||||||
self.workspace,
|
|
||||||
restrict_to_workspace=self.restrict_to_workspace,
|
|
||||||
)
|
|
||||||
workspace = access.project_path or self.workspace
|
|
||||||
manager = CliAppManager(workspace=workspace, runtime=self.runtime)
|
|
||||||
try:
|
|
||||||
return manager.run(
|
|
||||||
name,
|
|
||||||
args=args or [],
|
|
||||||
json_output=bool(json),
|
|
||||||
working_dir=working_dir,
|
|
||||||
timeout=timeout,
|
|
||||||
restrict_to_workspace=access.restrict_to_workspace,
|
|
||||||
)
|
|
||||||
except CliAppError as exc:
|
|
||||||
return f"Error: {exc.message}"
|
|
||||||
@@ -1,15 +1,9 @@
|
|||||||
"""Runtime context for tool construction."""
|
"""Runtime context for tool construction."""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from contextvars import ContextVar, Token
|
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import Any, Callable, Protocol, runtime_checkable
|
from typing import Any, Callable, Protocol, runtime_checkable
|
||||||
|
|
||||||
_CURRENT_REQUEST_CONTEXT: ContextVar["RequestContext | None"] = ContextVar(
|
|
||||||
"nanobot_tool_request_context",
|
|
||||||
default=None,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class RequestContext:
|
class RequestContext:
|
||||||
@@ -27,23 +21,6 @@ class ContextAware(Protocol):
|
|||||||
...
|
...
|
||||||
|
|
||||||
|
|
||||||
def bind_request_context(ctx: RequestContext) -> Token[RequestContext | None]:
|
|
||||||
return _CURRENT_REQUEST_CONTEXT.set(ctx)
|
|
||||||
|
|
||||||
|
|
||||||
def reset_request_context(token: Token[RequestContext | None]) -> None:
|
|
||||||
_CURRENT_REQUEST_CONTEXT.reset(token)
|
|
||||||
|
|
||||||
|
|
||||||
def current_request_context() -> RequestContext | None:
|
|
||||||
return _CURRENT_REQUEST_CONTEXT.get()
|
|
||||||
|
|
||||||
|
|
||||||
def current_request_session_key() -> str | None:
|
|
||||||
ctx = current_request_context()
|
|
||||||
return ctx.session_key if ctx else None
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class ToolContext:
|
class ToolContext:
|
||||||
config: Any
|
config: Any
|
||||||
@@ -56,4 +33,3 @@ class ToolContext:
|
|||||||
provider_snapshot_loader: Callable[[], Any] | None = None
|
provider_snapshot_loader: Callable[[], Any] | None = None
|
||||||
image_generation_provider_configs: dict[str, Any] | None = None
|
image_generation_provider_configs: dict[str, Any] | None = None
|
||||||
timezone: str = "UTC"
|
timezone: str = "UTC"
|
||||||
workspace_sandbox: Any | None = None
|
|
||||||
|
|||||||
@@ -1,598 +0,0 @@
|
|||||||
"""Session support for long-running exec workflows."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import time
|
|
||||||
import uuid
|
|
||||||
from contextlib import suppress
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from nanobot.agent.tools.base import Tool, tool_parameters
|
|
||||||
from nanobot.agent.tools.context import current_request_session_key
|
|
||||||
from nanobot.agent.tools.schema import (
|
|
||||||
BooleanSchema,
|
|
||||||
IntegerSchema,
|
|
||||||
StringSchema,
|
|
||||||
tool_parameters_schema,
|
|
||||||
)
|
|
||||||
|
|
||||||
DEFAULT_YIELD_MS = 1000
|
|
||||||
MAX_YIELD_MS = 30_000
|
|
||||||
DEFAULT_WAIT_FOR_MS = 10_000
|
|
||||||
MAX_WAIT_FOR_MS = 120_000
|
|
||||||
DEFAULT_MAX_OUTPUT_CHARS = 10_000
|
|
||||||
MAX_OUTPUT_CHARS = 50_000
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
|
||||||
class _SessionPoll:
|
|
||||||
output: str
|
|
||||||
done: bool
|
|
||||||
exit_code: int | None
|
|
||||||
elapsed_s: float = 0.0
|
|
||||||
timed_out: bool = False
|
|
||||||
terminated: bool = False
|
|
||||||
stdin_closed: bool = False
|
|
||||||
truncated_chars: int = 0
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
|
||||||
class ExecSessionInfo:
|
|
||||||
session_id: str
|
|
||||||
command: str
|
|
||||||
cwd: str
|
|
||||||
elapsed_s: float
|
|
||||||
idle_s: float
|
|
||||||
remaining_s: float
|
|
||||||
returncode: int | None
|
|
||||||
owner_session_key: str | None = None
|
|
||||||
|
|
||||||
|
|
||||||
class _ExecSession:
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
session_id: str,
|
|
||||||
process: asyncio.subprocess.Process,
|
|
||||||
command: str,
|
|
||||||
cwd: str,
|
|
||||||
timeout: int | None,
|
|
||||||
owner_session_key: str | None = None,
|
|
||||||
) -> None:
|
|
||||||
self.session_id = session_id
|
|
||||||
self.process = process
|
|
||||||
self.command = command
|
|
||||||
self.cwd = cwd
|
|
||||||
self.owner_session_key = owner_session_key
|
|
||||||
self.started_at = time.monotonic()
|
|
||||||
# timeout None/0 means no limit; an infinite deadline is never reached.
|
|
||||||
self.deadline = time.monotonic() + timeout if timeout else float("inf")
|
|
||||||
self.last_access = time.monotonic()
|
|
||||||
self._chunks: list[str] = []
|
|
||||||
self._lock = asyncio.Lock()
|
|
||||||
self._timed_out = False
|
|
||||||
self._stdout_task = asyncio.create_task(self._read_stream(process.stdout, ""))
|
|
||||||
self._stderr_task = asyncio.create_task(self._read_stream(process.stderr, "STDERR:\n"))
|
|
||||||
|
|
||||||
async def _read_stream(
|
|
||||||
self,
|
|
||||||
stream: asyncio.StreamReader | None,
|
|
||||||
prefix: str,
|
|
||||||
) -> None:
|
|
||||||
if stream is None:
|
|
||||||
return
|
|
||||||
first = True
|
|
||||||
while True:
|
|
||||||
chunk = await stream.read(4096)
|
|
||||||
if not chunk:
|
|
||||||
break
|
|
||||||
text = chunk.decode("utf-8", errors="replace")
|
|
||||||
if prefix and first:
|
|
||||||
text = prefix + text
|
|
||||||
first = False
|
|
||||||
async with self._lock:
|
|
||||||
self._chunks.append(text)
|
|
||||||
|
|
||||||
async def write(self, chars: str) -> str | None:
|
|
||||||
if self.process.returncode is not None:
|
|
||||||
return "session has already exited"
|
|
||||||
if self.process.stdin is None:
|
|
||||||
return "session stdin is not available"
|
|
||||||
try:
|
|
||||||
self.process.stdin.write(chars.encode("utf-8"))
|
|
||||||
await self.process.stdin.drain()
|
|
||||||
except (BrokenPipeError, ConnectionResetError):
|
|
||||||
return "session stdin is closed"
|
|
||||||
return None
|
|
||||||
|
|
||||||
async def close_stdin(self) -> str | None:
|
|
||||||
if self.process.returncode is not None:
|
|
||||||
return "session has already exited"
|
|
||||||
if self.process.stdin is None:
|
|
||||||
return "session stdin is not available"
|
|
||||||
self.process.stdin.close()
|
|
||||||
with suppress(BrokenPipeError, ConnectionResetError):
|
|
||||||
await self.process.stdin.wait_closed()
|
|
||||||
return None
|
|
||||||
|
|
||||||
async def poll(
|
|
||||||
self,
|
|
||||||
yield_time_ms: int,
|
|
||||||
max_output_chars: int,
|
|
||||||
*,
|
|
||||||
terminated: bool = False,
|
|
||||||
stdin_closed: bool = False,
|
|
||||||
) -> _SessionPoll:
|
|
||||||
self.last_access = time.monotonic()
|
|
||||||
if yield_time_ms > 0 and self.process.returncode is None:
|
|
||||||
await asyncio.sleep(min(yield_time_ms, MAX_YIELD_MS) / 1000)
|
|
||||||
|
|
||||||
if self.process.returncode is None and time.monotonic() >= self.deadline:
|
|
||||||
self._timed_out = True
|
|
||||||
await self.kill()
|
|
||||||
|
|
||||||
if self.process.returncode is not None:
|
|
||||||
with suppress(asyncio.TimeoutError):
|
|
||||||
await asyncio.wait_for(
|
|
||||||
asyncio.gather(self._stdout_task, self._stderr_task),
|
|
||||||
timeout=2.0,
|
|
||||||
)
|
|
||||||
|
|
||||||
async with self._lock:
|
|
||||||
output = "".join(self._chunks)
|
|
||||||
self._chunks.clear()
|
|
||||||
|
|
||||||
output, truncated = _truncate_output(output, max_output_chars)
|
|
||||||
return _SessionPoll(
|
|
||||||
output=output,
|
|
||||||
done=self.process.returncode is not None,
|
|
||||||
exit_code=self.process.returncode,
|
|
||||||
elapsed_s=max(0.0, time.monotonic() - self.started_at),
|
|
||||||
timed_out=self._timed_out,
|
|
||||||
terminated=terminated,
|
|
||||||
stdin_closed=stdin_closed,
|
|
||||||
truncated_chars=truncated,
|
|
||||||
)
|
|
||||||
|
|
||||||
async def kill(self) -> None:
|
|
||||||
if self.process.returncode is not None:
|
|
||||||
return
|
|
||||||
self.process.kill()
|
|
||||||
with suppress(asyncio.TimeoutError):
|
|
||||||
await asyncio.wait_for(self.process.wait(), timeout=5.0)
|
|
||||||
|
|
||||||
|
|
||||||
class ExecSessionManager:
|
|
||||||
def __init__(self, *, max_sessions: int = 8, idle_timeout: int = 1800) -> None:
|
|
||||||
self.max_sessions = max_sessions
|
|
||||||
self.idle_timeout = idle_timeout
|
|
||||||
self._sessions: dict[str, _ExecSession] = {}
|
|
||||||
self._lock = asyncio.Lock()
|
|
||||||
|
|
||||||
async def start(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
command: str,
|
|
||||||
cwd: str,
|
|
||||||
env: dict[str, str],
|
|
||||||
timeout: int | None,
|
|
||||||
shell_program: str | None,
|
|
||||||
login: bool,
|
|
||||||
yield_time_ms: int,
|
|
||||||
max_output_chars: int,
|
|
||||||
owner_session_key: str | None = None,
|
|
||||||
) -> tuple[str, _SessionPoll]:
|
|
||||||
async with self._lock:
|
|
||||||
await self._cleanup_locked()
|
|
||||||
if len(self._sessions) >= self.max_sessions:
|
|
||||||
raise RuntimeError(f"maximum exec sessions reached ({self.max_sessions})")
|
|
||||||
process = await self._spawn(command, cwd, env, shell_program, login)
|
|
||||||
session_id = uuid.uuid4().hex[:12]
|
|
||||||
session = _ExecSession(
|
|
||||||
session_id=session_id,
|
|
||||||
process=process,
|
|
||||||
command=command,
|
|
||||||
cwd=cwd,
|
|
||||||
timeout=timeout,
|
|
||||||
owner_session_key=owner_session_key,
|
|
||||||
)
|
|
||||||
self._sessions[session_id] = session
|
|
||||||
|
|
||||||
poll = await session.poll(yield_time_ms, max_output_chars)
|
|
||||||
if poll.done:
|
|
||||||
async with self._lock:
|
|
||||||
self._sessions.pop(session_id, None)
|
|
||||||
return session_id, poll
|
|
||||||
|
|
||||||
async def write(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
session_id: str,
|
|
||||||
chars: str | None,
|
|
||||||
close_stdin: bool,
|
|
||||||
terminate: bool,
|
|
||||||
yield_time_ms: int,
|
|
||||||
max_output_chars: int,
|
|
||||||
owner_session_key: str | None = None,
|
|
||||||
) -> _SessionPoll:
|
|
||||||
async with self._lock:
|
|
||||||
await self._cleanup_locked()
|
|
||||||
session = self._sessions.get(session_id)
|
|
||||||
if session is None:
|
|
||||||
raise KeyError(session_id)
|
|
||||||
if (
|
|
||||||
owner_session_key
|
|
||||||
and session.owner_session_key
|
|
||||||
and session.owner_session_key != owner_session_key
|
|
||||||
):
|
|
||||||
raise KeyError(session_id)
|
|
||||||
|
|
||||||
if chars:
|
|
||||||
error = await session.write(chars)
|
|
||||||
if error:
|
|
||||||
raise RuntimeError(error)
|
|
||||||
stdin_closed = False
|
|
||||||
if close_stdin:
|
|
||||||
error = await session.close_stdin()
|
|
||||||
if error:
|
|
||||||
raise RuntimeError(error)
|
|
||||||
stdin_closed = True
|
|
||||||
if terminate:
|
|
||||||
await session.kill()
|
|
||||||
poll = await session.poll(
|
|
||||||
yield_time_ms,
|
|
||||||
max_output_chars,
|
|
||||||
terminated=terminate,
|
|
||||||
stdin_closed=stdin_closed,
|
|
||||||
)
|
|
||||||
if poll.done:
|
|
||||||
async with self._lock:
|
|
||||||
self._sessions.pop(session_id, None)
|
|
||||||
return poll
|
|
||||||
|
|
||||||
async def list(self, *, owner_session_key: str | None = None) -> list[ExecSessionInfo]:
|
|
||||||
async with self._lock:
|
|
||||||
await self._cleanup_locked()
|
|
||||||
now = time.monotonic()
|
|
||||||
return [
|
|
||||||
ExecSessionInfo(
|
|
||||||
session_id=session_id,
|
|
||||||
command=session.command,
|
|
||||||
cwd=session.cwd,
|
|
||||||
elapsed_s=max(0.0, now - session.started_at),
|
|
||||||
idle_s=max(0.0, now - session.last_access),
|
|
||||||
remaining_s=max(0.0, session.deadline - now),
|
|
||||||
returncode=session.process.returncode,
|
|
||||||
owner_session_key=session.owner_session_key,
|
|
||||||
)
|
|
||||||
for session_id, session in sorted(self._sessions.items())
|
|
||||||
if not owner_session_key
|
|
||||||
or not session.owner_session_key
|
|
||||||
or session.owner_session_key == owner_session_key
|
|
||||||
]
|
|
||||||
|
|
||||||
async def _cleanup_locked(self) -> None:
|
|
||||||
now = time.monotonic()
|
|
||||||
stale = [
|
|
||||||
session_id
|
|
||||||
for session_id, session in self._sessions.items()
|
|
||||||
if now - session.last_access > self.idle_timeout
|
|
||||||
]
|
|
||||||
for session_id in stale:
|
|
||||||
session = self._sessions.pop(session_id)
|
|
||||||
await session.kill()
|
|
||||||
|
|
||||||
async def _spawn(
|
|
||||||
self,
|
|
||||||
command: str,
|
|
||||||
cwd: str,
|
|
||||||
env: dict[str, str],
|
|
||||||
shell_program: str | None,
|
|
||||||
login: bool,
|
|
||||||
) -> asyncio.subprocess.Process:
|
|
||||||
from nanobot.agent.tools.shell import ExecTool
|
|
||||||
|
|
||||||
return await ExecTool._spawn(
|
|
||||||
command, cwd, env, shell_program, login,
|
|
||||||
stdin=asyncio.subprocess.PIPE,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
DEFAULT_EXEC_SESSION_MANAGER = ExecSessionManager()
|
|
||||||
|
|
||||||
|
|
||||||
def clamp_session_int(value: int | None, default: int, minimum: int, maximum: int) -> int:
|
|
||||||
if value is None:
|
|
||||||
return default
|
|
||||||
return min(max(value, minimum), maximum)
|
|
||||||
|
|
||||||
|
|
||||||
def _truncate_output(output: str, max_output_chars: int) -> tuple[str, int]:
|
|
||||||
if len(output) <= max_output_chars:
|
|
||||||
return output, 0
|
|
||||||
half = max_output_chars // 2
|
|
||||||
omitted = len(output) - max_output_chars
|
|
||||||
return (
|
|
||||||
output[:half]
|
|
||||||
+ f"\n\n... ({omitted:,} chars truncated) ...\n\n"
|
|
||||||
+ output[-half:],
|
|
||||||
omitted,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def format_session_poll(session_id: str, poll: _SessionPoll) -> str:
|
|
||||||
parts = [poll.output] if poll.output else []
|
|
||||||
if poll.truncated_chars:
|
|
||||||
parts.append(f"(output truncated by {poll.truncated_chars:,} chars)")
|
|
||||||
if poll.timed_out:
|
|
||||||
parts.append("Error: Command timed out; session was terminated.")
|
|
||||||
if poll.terminated and not poll.timed_out:
|
|
||||||
parts.append("Session terminated.")
|
|
||||||
if poll.stdin_closed:
|
|
||||||
parts.append("Stdin closed.")
|
|
||||||
if poll.done:
|
|
||||||
parts.append(f"Exit code: {poll.exit_code}")
|
|
||||||
else:
|
|
||||||
parts.append(f"Process running. session_id: {session_id}")
|
|
||||||
parts.append(f"Elapsed: {poll.elapsed_s:.1f}s")
|
|
||||||
return "\n".join(parts) if parts else "(no output yet)"
|
|
||||||
|
|
||||||
|
|
||||||
@tool_parameters(
|
|
||||||
tool_parameters_schema(
|
|
||||||
session_id=StringSchema("Session id returned by exec when yield_time_ms is used."),
|
|
||||||
chars=StringSchema(
|
|
||||||
"Bytes/text to write to stdin. Omit or pass an empty string to only poll recent output.",
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
close_stdin=BooleanSchema(
|
|
||||||
description="Close stdin after writing chars. Useful for commands waiting for EOF.",
|
|
||||||
default=False,
|
|
||||||
),
|
|
||||||
terminate=BooleanSchema(
|
|
||||||
description="Terminate the running exec session.",
|
|
||||||
default=False,
|
|
||||||
),
|
|
||||||
yield_time_ms=IntegerSchema(
|
|
||||||
DEFAULT_YIELD_MS,
|
|
||||||
description="Milliseconds to wait before returning recent output (default 1000, max 30000).",
|
|
||||||
minimum=0,
|
|
||||||
maximum=MAX_YIELD_MS,
|
|
||||||
),
|
|
||||||
wait_for=StringSchema(
|
|
||||||
"Optional text to wait for in output before returning. "
|
|
||||||
"Useful for interactive commands and dev servers.",
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
wait_timeout_ms=IntegerSchema(
|
|
||||||
DEFAULT_WAIT_FOR_MS,
|
|
||||||
description="Maximum milliseconds to wait for wait_for text (default 10000, max 120000).",
|
|
||||||
minimum=0,
|
|
||||||
maximum=MAX_WAIT_FOR_MS,
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
max_output_chars=IntegerSchema(
|
|
||||||
DEFAULT_MAX_OUTPUT_CHARS,
|
|
||||||
description="Maximum output characters to return from this poll (default 10000, max 50000).",
|
|
||||||
minimum=1000,
|
|
||||||
maximum=MAX_OUTPUT_CHARS,
|
|
||||||
),
|
|
||||||
max_output_tokens=IntegerSchema(
|
|
||||||
DEFAULT_MAX_OUTPUT_CHARS,
|
|
||||||
description="Compatibility alias for max_output_chars. The current runtime uses a character budget.",
|
|
||||||
minimum=1000,
|
|
||||||
maximum=MAX_OUTPUT_CHARS,
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
required=["session_id"],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
class WriteStdinTool(Tool):
|
|
||||||
"""Write to or poll a running exec session."""
|
|
||||||
|
|
||||||
_scopes = {"core", "subagent"}
|
|
||||||
config_key = "exec"
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def config_cls(cls):
|
|
||||||
from nanobot.agent.tools.shell import ExecToolConfig
|
|
||||||
|
|
||||||
return ExecToolConfig
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def enabled(cls, ctx: Any) -> bool:
|
|
||||||
return ctx.config.exec.enable
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
manager: ExecSessionManager | None = None,
|
|
||||||
) -> None:
|
|
||||||
self._manager = manager or DEFAULT_EXEC_SESSION_MANAGER
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def create(cls, ctx: Any) -> Tool:
|
|
||||||
return cls()
|
|
||||||
|
|
||||||
@property
|
|
||||||
def exclusive(self) -> bool:
|
|
||||||
return True
|
|
||||||
|
|
||||||
@property
|
|
||||||
def name(self) -> str:
|
|
||||||
return "write_stdin"
|
|
||||||
|
|
||||||
@property
|
|
||||||
def description(self) -> str:
|
|
||||||
return (
|
|
||||||
"Interact with a running exec session created by exec with "
|
|
||||||
"yield_time_ms. Use chars='' to poll without writing, chars to send "
|
|
||||||
"stdin, close_stdin=true to send EOF, or terminate=true to stop the "
|
|
||||||
"process. Use wait_for with wait_timeout_ms for dev servers, test "
|
|
||||||
"watchers, and prompts where you need to wait for expected output. "
|
|
||||||
"Do not use this to start new commands; start them with exec."
|
|
||||||
)
|
|
||||||
|
|
||||||
async def execute(
|
|
||||||
self,
|
|
||||||
session_id: str,
|
|
||||||
chars: str | None = None,
|
|
||||||
close_stdin: bool = False,
|
|
||||||
terminate: bool = False,
|
|
||||||
yield_time_ms: int | None = None,
|
|
||||||
wait_for: str | None = None,
|
|
||||||
wait_timeout_ms: int | None = None,
|
|
||||||
max_output_chars: int | None = None,
|
|
||||||
max_output_tokens: int | None = None,
|
|
||||||
**kwargs: Any,
|
|
||||||
) -> str:
|
|
||||||
try:
|
|
||||||
if max_output_chars is None:
|
|
||||||
max_output_chars = max_output_tokens
|
|
||||||
output_limit = clamp_session_int(
|
|
||||||
max_output_chars,
|
|
||||||
DEFAULT_MAX_OUTPUT_CHARS,
|
|
||||||
1000,
|
|
||||||
MAX_OUTPUT_CHARS,
|
|
||||||
)
|
|
||||||
if wait_for:
|
|
||||||
return await self._wait_for_output(
|
|
||||||
session_id=session_id,
|
|
||||||
chars=chars,
|
|
||||||
close_stdin=close_stdin,
|
|
||||||
terminate=terminate,
|
|
||||||
wait_for=wait_for,
|
|
||||||
wait_timeout_ms=clamp_session_int(
|
|
||||||
wait_timeout_ms,
|
|
||||||
DEFAULT_WAIT_FOR_MS,
|
|
||||||
0,
|
|
||||||
MAX_WAIT_FOR_MS,
|
|
||||||
),
|
|
||||||
max_output_chars=output_limit,
|
|
||||||
)
|
|
||||||
poll = await self._manager.write(
|
|
||||||
session_id=session_id,
|
|
||||||
chars=chars,
|
|
||||||
close_stdin=close_stdin,
|
|
||||||
terminate=terminate,
|
|
||||||
yield_time_ms=clamp_session_int(yield_time_ms, DEFAULT_YIELD_MS, 0, MAX_YIELD_MS),
|
|
||||||
max_output_chars=output_limit,
|
|
||||||
owner_session_key=current_request_session_key(),
|
|
||||||
)
|
|
||||||
return format_session_poll(session_id, poll)
|
|
||||||
except KeyError:
|
|
||||||
return f"Error: exec session not found: {session_id}"
|
|
||||||
except Exception as exc:
|
|
||||||
return f"Error writing to exec session: {exc}"
|
|
||||||
|
|
||||||
async def _wait_for_output(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
session_id: str,
|
|
||||||
chars: str | None,
|
|
||||||
close_stdin: bool,
|
|
||||||
terminate: bool,
|
|
||||||
wait_for: str,
|
|
||||||
wait_timeout_ms: int,
|
|
||||||
max_output_chars: int,
|
|
||||||
) -> str:
|
|
||||||
deadline = time.monotonic() + (wait_timeout_ms / 1000)
|
|
||||||
aggregate: list[str] = []
|
|
||||||
first = True
|
|
||||||
poll: _SessionPoll | None = None
|
|
||||||
|
|
||||||
while True:
|
|
||||||
remaining_ms = max(0, int((deadline - time.monotonic()) * 1000))
|
|
||||||
step_ms = min(500, remaining_ms)
|
|
||||||
poll = await self._manager.write(
|
|
||||||
session_id=session_id,
|
|
||||||
chars=chars if first else None,
|
|
||||||
close_stdin=close_stdin if first else False,
|
|
||||||
terminate=terminate if first else False,
|
|
||||||
yield_time_ms=step_ms,
|
|
||||||
max_output_chars=max_output_chars,
|
|
||||||
owner_session_key=current_request_session_key(),
|
|
||||||
)
|
|
||||||
first = False
|
|
||||||
if poll.output:
|
|
||||||
aggregate.append(poll.output)
|
|
||||||
joined = "".join(aggregate)
|
|
||||||
if wait_for in joined:
|
|
||||||
poll.output = joined
|
|
||||||
return format_session_poll(session_id, poll)
|
|
||||||
if poll.done or remaining_ms <= 0:
|
|
||||||
poll.output = "".join(aggregate)
|
|
||||||
result = format_session_poll(session_id, poll)
|
|
||||||
if wait_for not in poll.output:
|
|
||||||
result += f"\nWait target not observed: {wait_for!r}"
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
@tool_parameters(tool_parameters_schema())
|
|
||||||
class ListExecSessionsTool(Tool):
|
|
||||||
"""List active exec sessions."""
|
|
||||||
|
|
||||||
_scopes = {"core", "subagent"}
|
|
||||||
config_key = "exec"
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def config_cls(cls):
|
|
||||||
from nanobot.agent.tools.shell import ExecToolConfig
|
|
||||||
|
|
||||||
return ExecToolConfig
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def enabled(cls, ctx: Any) -> bool:
|
|
||||||
return ctx.config.exec.enable
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
manager: ExecSessionManager | None = None,
|
|
||||||
) -> None:
|
|
||||||
self._manager = manager or DEFAULT_EXEC_SESSION_MANAGER
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def create(cls, ctx: Any) -> Tool:
|
|
||||||
return cls()
|
|
||||||
|
|
||||||
@property
|
|
||||||
def name(self) -> str:
|
|
||||||
return "list_exec_sessions"
|
|
||||||
|
|
||||||
@property
|
|
||||||
def description(self) -> str:
|
|
||||||
return (
|
|
||||||
"List active long-running exec sessions, including session_id, cwd, "
|
|
||||||
"elapsed time, idle time, remaining timeout, and command preview. "
|
|
||||||
"Use this to recover a session_id after context shifts before "
|
|
||||||
"polling, writing stdin, or terminating with write_stdin."
|
|
||||||
)
|
|
||||||
|
|
||||||
@property
|
|
||||||
def read_only(self) -> bool:
|
|
||||||
return True
|
|
||||||
|
|
||||||
async def execute(self, **kwargs: Any) -> str:
|
|
||||||
try:
|
|
||||||
sessions = await self._manager.list(
|
|
||||||
owner_session_key=current_request_session_key(),
|
|
||||||
)
|
|
||||||
if not sessions:
|
|
||||||
return "No active exec sessions."
|
|
||||||
lines = []
|
|
||||||
for info in sessions:
|
|
||||||
command = " ".join(info.command.split())
|
|
||||||
if len(command) > 120:
|
|
||||||
command = command[:119] + "..."
|
|
||||||
status = "exited" if info.returncode is not None else "running"
|
|
||||||
lines.append(
|
|
||||||
f"{info.session_id} | {status} | elapsed={info.elapsed_s:.1f}s "
|
|
||||||
f"| idle={info.idle_s:.1f}s | remaining={info.remaining_s:.1f}s "
|
|
||||||
f"| cwd={info.cwd} | {command}"
|
|
||||||
)
|
|
||||||
return "\n".join(lines)
|
|
||||||
except Exception as exc:
|
|
||||||
return f"Error listing exec sessions: {exc}"
|
|
||||||
@@ -10,7 +10,6 @@ from typing import Any
|
|||||||
from nanobot.agent.tools.base import Tool, tool_parameters
|
from nanobot.agent.tools.base import Tool, tool_parameters
|
||||||
from nanobot.agent.tools.file_state import FileStates, _hash_file, current_file_states
|
from nanobot.agent.tools.file_state import FileStates, _hash_file, current_file_states
|
||||||
from nanobot.agent.tools.path_utils import resolve_workspace_path
|
from nanobot.agent.tools.path_utils import resolve_workspace_path
|
||||||
from nanobot.security.workspace_access import current_tool_workspace
|
|
||||||
from nanobot.agent.tools.schema import (
|
from nanobot.agent.tools.schema import (
|
||||||
BooleanSchema,
|
BooleanSchema,
|
||||||
IntegerSchema,
|
IntegerSchema,
|
||||||
@@ -29,18 +28,10 @@ class _FsTool(Tool):
|
|||||||
allowed_dir: Path | None = None,
|
allowed_dir: Path | None = None,
|
||||||
extra_allowed_dirs: list[Path] | None = None,
|
extra_allowed_dirs: list[Path] | None = None,
|
||||||
file_states: FileStates | None = None,
|
file_states: FileStates | None = None,
|
||||||
restrict_to_workspace: bool | None = None,
|
|
||||||
sandbox_restricts_workspace: bool = False,
|
|
||||||
):
|
):
|
||||||
self._workspace = workspace
|
self._workspace = workspace
|
||||||
self._allowed_dir = allowed_dir
|
self._allowed_dir = allowed_dir
|
||||||
self._extra_allowed_dirs = extra_allowed_dirs
|
self._extra_allowed_dirs = extra_allowed_dirs
|
||||||
self._restrict_to_workspace = (
|
|
||||||
bool(restrict_to_workspace)
|
|
||||||
if restrict_to_workspace is not None
|
|
||||||
else allowed_dir is not None
|
|
||||||
)
|
|
||||||
self._sandbox_restricts_workspace = sandbox_restricts_workspace
|
|
||||||
# Explicit state is used by isolated runners like Dream/subagents.
|
# Explicit state is used by isolated runners like Dream/subagents.
|
||||||
# Main AgentLoop tools leave this unset and resolve state from the
|
# Main AgentLoop tools leave this unset and resolve state from the
|
||||||
# current async task, which keeps shared tool instances session-safe.
|
# current async task, which keeps shared tool instances session-safe.
|
||||||
@@ -55,16 +46,13 @@ class _FsTool(Tool):
|
|||||||
ctx.config.restrict_to_workspace
|
ctx.config.restrict_to_workspace
|
||||||
or ctx.config.exec.sandbox
|
or ctx.config.exec.sandbox
|
||||||
)
|
)
|
||||||
sandbox_restricts = bool(ctx.config.exec.sandbox)
|
|
||||||
allowed_dir = Path(ctx.workspace) if restrict else None
|
allowed_dir = Path(ctx.workspace) if restrict else None
|
||||||
extra_read = [BUILTIN_SKILLS_DIR]
|
extra_read = [BUILTIN_SKILLS_DIR] if allowed_dir else None
|
||||||
return cls(
|
return cls(
|
||||||
workspace=Path(ctx.workspace),
|
workspace=Path(ctx.workspace),
|
||||||
allowed_dir=allowed_dir,
|
allowed_dir=allowed_dir,
|
||||||
extra_allowed_dirs=extra_read,
|
extra_allowed_dirs=extra_read,
|
||||||
file_states=ctx.file_state_store,
|
file_states=ctx.file_state_store,
|
||||||
restrict_to_workspace=ctx.config.restrict_to_workspace,
|
|
||||||
sandbox_restricts_workspace=sandbox_restricts,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
@@ -74,21 +62,13 @@ class _FsTool(Tool):
|
|||||||
return current_file_states(self._fallback_file_states)
|
return current_file_states(self._fallback_file_states)
|
||||||
|
|
||||||
def _resolve(self, path: str) -> Path:
|
def _resolve(self, path: str) -> Path:
|
||||||
access = current_tool_workspace(
|
|
||||||
self._workspace,
|
|
||||||
restrict_to_workspace=self._restrict_to_workspace,
|
|
||||||
sandbox_restricts_workspace=self._sandbox_restricts_workspace,
|
|
||||||
)
|
|
||||||
return resolve_workspace_path(
|
return resolve_workspace_path(
|
||||||
path,
|
path,
|
||||||
access.project_path,
|
self._workspace,
|
||||||
access.allowed_root,
|
self._allowed_dir,
|
||||||
self._extra_allowed_dirs,
|
self._extra_allowed_dirs,
|
||||||
)
|
)
|
||||||
|
|
||||||
def _display_workspace(self) -> Path | None:
|
|
||||||
return current_tool_workspace(self._workspace).project_path
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# read_file
|
# read_file
|
||||||
@@ -152,10 +132,6 @@ def _parse_page_range(pages: str, total: int) -> tuple[int, int]:
|
|||||||
minimum=1,
|
minimum=1,
|
||||||
),
|
),
|
||||||
pages=StringSchema("Page range for PDF files, e.g. '1-5' (default: all, max 20 pages)"),
|
pages=StringSchema("Page range for PDF files, e.g. '1-5' (default: all, max 20 pages)"),
|
||||||
force=BooleanSchema(
|
|
||||||
description="Bypass same-file read deduplication and return content again.",
|
|
||||||
default=False,
|
|
||||||
),
|
|
||||||
required=["path"],
|
required=["path"],
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
@@ -178,11 +154,7 @@ class ReadFileTool(_FsTool):
|
|||||||
"Text output format: LINE_NUM|CONTENT. "
|
"Text output format: LINE_NUM|CONTENT. "
|
||||||
"Images return visual content for analysis. "
|
"Images return visual content for analysis. "
|
||||||
"Supports PDF, DOCX, XLSX, PPTX documents. "
|
"Supports PDF, DOCX, XLSX, PPTX documents. "
|
||||||
"Use find_files/list_dir first when the path is uncertain. "
|
|
||||||
"Read the relevant range before editing so replacements or patches "
|
|
||||||
"are based on current content. "
|
|
||||||
"Use offset and limit for large text files. "
|
"Use offset and limit for large text files. "
|
||||||
"Use force=true to re-read content even if unchanged. "
|
|
||||||
"Reads exceeding ~128K chars are truncated."
|
"Reads exceeding ~128K chars are truncated."
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -190,15 +162,7 @@ class ReadFileTool(_FsTool):
|
|||||||
def read_only(self) -> bool:
|
def read_only(self) -> bool:
|
||||||
return True
|
return True
|
||||||
|
|
||||||
async def execute(
|
async def execute(self, path: str | None = None, offset: int = 1, limit: int | None = None, pages: str | None = None, **kwargs: Any) -> Any:
|
||||||
self,
|
|
||||||
path: str | None = None,
|
|
||||||
offset: int = 1,
|
|
||||||
limit: int | None = None,
|
|
||||||
pages: str | None = None,
|
|
||||||
force: bool = False,
|
|
||||||
**kwargs: Any,
|
|
||||||
) -> Any:
|
|
||||||
try:
|
try:
|
||||||
if not path:
|
if not path:
|
||||||
return "Error reading file: Unknown path"
|
return "Error reading file: Unknown path"
|
||||||
@@ -238,13 +202,7 @@ class ReadFileTool(_FsTool):
|
|||||||
current_mtime = os.path.getmtime(fp)
|
current_mtime = os.path.getmtime(fp)
|
||||||
except OSError:
|
except OSError:
|
||||||
current_mtime = 0.0
|
current_mtime = 0.0
|
||||||
if (
|
if entry and entry.can_dedup and entry.offset == offset and entry.limit == limit:
|
||||||
not force
|
|
||||||
and entry
|
|
||||||
and entry.can_dedup
|
|
||||||
and entry.offset == offset
|
|
||||||
and entry.limit == limit
|
|
||||||
):
|
|
||||||
if current_mtime != entry.mtime:
|
if current_mtime != entry.mtime:
|
||||||
# File was modified externally - force full read and mark as not dedupable
|
# File was modified externally - force full read and mark as not dedupable
|
||||||
entry.can_dedup = False
|
entry.can_dedup = False
|
||||||
@@ -407,10 +365,9 @@ class WriteFileTool(_FsTool):
|
|||||||
@property
|
@property
|
||||||
def description(self) -> str:
|
def description(self) -> str:
|
||||||
return (
|
return (
|
||||||
"Create a new file or intentionally replace an entire file with "
|
"Write content to a file. Overwrites if the file already exists; "
|
||||||
"the provided content. Overwrites existing files and creates parent "
|
"creates parent directories as needed. "
|
||||||
"directories as needed. For code changes or partial edits, prefer "
|
"For partial edits, prefer edit_file instead."
|
||||||
"apply_patch; use edit_file only for small exact replacements."
|
|
||||||
)
|
)
|
||||||
|
|
||||||
async def execute(self, path: str | None = None, content: str | None = None, **kwargs: Any) -> str:
|
async def execute(self, path: str | None = None, content: str | None = None, **kwargs: Any) -> str:
|
||||||
@@ -700,24 +657,6 @@ def _find_match(content: str, old_text: str) -> tuple[str | None, int]:
|
|||||||
old_text=StringSchema("The text to find and replace"),
|
old_text=StringSchema("The text to find and replace"),
|
||||||
new_text=StringSchema("The text to replace with"),
|
new_text=StringSchema("The text to replace with"),
|
||||||
replace_all=BooleanSchema(description="Replace all occurrences (default false)"),
|
replace_all=BooleanSchema(description="Replace all occurrences (default false)"),
|
||||||
occurrence=IntegerSchema(
|
|
||||||
1,
|
|
||||||
description="Optional 1-based occurrence to replace when old_text appears multiple times.",
|
|
||||||
minimum=1,
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
line_hint=IntegerSchema(
|
|
||||||
1,
|
|
||||||
description="Optional 1-based line hint used to choose the nearest match.",
|
|
||||||
minimum=1,
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
expected_replacements=IntegerSchema(
|
|
||||||
1,
|
|
||||||
description="Optional guard for the number of replacements that must be made.",
|
|
||||||
minimum=1,
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
required=["path", "old_text", "new_text"],
|
required=["path", "old_text", "new_text"],
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
@@ -735,13 +674,10 @@ class EditFileTool(_FsTool):
|
|||||||
@property
|
@property
|
||||||
def description(self) -> str:
|
def description(self) -> str:
|
||||||
return (
|
return (
|
||||||
"Perform a small, exact replacement in one file by replacing "
|
"Edit a file by replacing old_text with new_text. "
|
||||||
"old_text with new_text. Use this for narrow text substitutions "
|
"Tolerates minor whitespace/indentation differences and curly/straight quote mismatches. "
|
||||||
"with old_text copied from read_file. For multi-file, structural, "
|
"If old_text matches multiple times, you must provide more context "
|
||||||
"or generated code edits, prefer apply_patch. If old_text matches "
|
"or set replace_all=true. Shows a diff of the closest match on failure."
|
||||||
"multiple times, provide more context or set occurrence, line_hint, "
|
|
||||||
"replace_all, and expected_replacements. Shows closest-match "
|
|
||||||
"diagnostics on failure."
|
|
||||||
)
|
)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
@@ -752,8 +688,7 @@ class EditFileTool(_FsTool):
|
|||||||
async def execute(
|
async def execute(
|
||||||
self, path: str | None = None, old_text: str | None = None,
|
self, path: str | None = None, old_text: str | None = None,
|
||||||
new_text: str | None = None,
|
new_text: str | None = None,
|
||||||
replace_all: bool = False, occurrence: int | None = None,
|
replace_all: bool = False, **kwargs: Any,
|
||||||
line_hint: int | None = None, expected_replacements: int | None = None, **kwargs: Any,
|
|
||||||
) -> str:
|
) -> str:
|
||||||
try:
|
try:
|
||||||
if not path:
|
if not path:
|
||||||
@@ -762,12 +697,10 @@ class EditFileTool(_FsTool):
|
|||||||
raise ValueError("Unknown old_text")
|
raise ValueError("Unknown old_text")
|
||||||
if new_text is None:
|
if new_text is None:
|
||||||
raise ValueError("Unknown new_text")
|
raise ValueError("Unknown new_text")
|
||||||
if occurrence is not None and occurrence < 1:
|
|
||||||
return "Error: occurrence must be >= 1."
|
# .ipynb detection
|
||||||
if line_hint is not None and line_hint < 1:
|
if path.endswith(".ipynb"):
|
||||||
return "Error: line_hint must be >= 1."
|
return "Error: This is a Jupyter notebook. Use the notebook_edit tool instead of edit_file."
|
||||||
if expected_replacements is not None and expected_replacements < 1:
|
|
||||||
return "Error: expected_replacements must be >= 1."
|
|
||||||
|
|
||||||
fp = self._resolve(path)
|
fp = self._resolve(path)
|
||||||
|
|
||||||
@@ -810,28 +743,7 @@ class EditFileTool(_FsTool):
|
|||||||
if not matches:
|
if not matches:
|
||||||
return self._not_found_msg(old_text, content, path)
|
return self._not_found_msg(old_text, content, path)
|
||||||
count = len(matches)
|
count = len(matches)
|
||||||
if replace_all and occurrence is not None:
|
|
||||||
return "Error: occurrence cannot be used with replace_all=true."
|
|
||||||
if replace_all and line_hint is not None:
|
|
||||||
return "Error: line_hint cannot be used with replace_all=true."
|
|
||||||
if occurrence is not None and line_hint is not None:
|
|
||||||
return "Error: line_hint cannot be used with occurrence."
|
|
||||||
if count > 1 and not replace_all:
|
if count > 1 and not replace_all:
|
||||||
if occurrence is not None:
|
|
||||||
if occurrence > count:
|
|
||||||
return (
|
|
||||||
f"Error: occurrence {occurrence} is out of range; "
|
|
||||||
f"old_text appears {count} times."
|
|
||||||
)
|
|
||||||
elif line_hint is not None:
|
|
||||||
nearest = min(matches, key=lambda match: abs(match.line - line_hint))
|
|
||||||
distance = abs(nearest.line - line_hint)
|
|
||||||
if sum(1 for match in matches if abs(match.line - line_hint) == distance) > 1:
|
|
||||||
return (
|
|
||||||
f"Error: line_hint {line_hint} is ambiguous; "
|
|
||||||
f"old_text appears {count} times."
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
line_numbers = [match.line for match in matches]
|
line_numbers = [match.line for match in matches]
|
||||||
preview = ", ".join(f"line {n}" for n in line_numbers[:3])
|
preview = ", ".join(f"line {n}" for n in line_numbers[:3])
|
||||||
if len(line_numbers) > 3:
|
if len(line_numbers) > 3:
|
||||||
@@ -839,13 +751,7 @@ class EditFileTool(_FsTool):
|
|||||||
location_hint = f" at {preview}" if preview else ""
|
location_hint = f" at {preview}" if preview else ""
|
||||||
return (
|
return (
|
||||||
f"Warning: old_text appears {count} times{location_hint}. "
|
f"Warning: old_text appears {count} times{location_hint}. "
|
||||||
"Provide more context, set occurrence to choose one match, "
|
"Provide more context to make it unique, or set replace_all=true."
|
||||||
"or set replace_all=true."
|
|
||||||
)
|
|
||||||
elif occurrence is not None and occurrence > count:
|
|
||||||
return (
|
|
||||||
f"Error: occurrence {occurrence} is out of range; "
|
|
||||||
f"old_text appears {count} time."
|
|
||||||
)
|
)
|
||||||
|
|
||||||
norm_new = new_text.replace("\r\n", "\n")
|
norm_new = new_text.replace("\r\n", "\n")
|
||||||
@@ -854,17 +760,7 @@ class EditFileTool(_FsTool):
|
|||||||
if fp.suffix.lower() not in self._MARKDOWN_EXTS:
|
if fp.suffix.lower() not in self._MARKDOWN_EXTS:
|
||||||
norm_new = self._strip_trailing_ws(norm_new)
|
norm_new = self._strip_trailing_ws(norm_new)
|
||||||
|
|
||||||
if replace_all:
|
selected = matches if replace_all else matches[:1]
|
||||||
selected = matches
|
|
||||||
elif line_hint is not None:
|
|
||||||
selected = [min(matches, key=lambda match: abs(match.line - line_hint))]
|
|
||||||
else:
|
|
||||||
selected = [matches[occurrence - 1 if occurrence else 0]]
|
|
||||||
if expected_replacements is not None and len(selected) != expected_replacements:
|
|
||||||
return (
|
|
||||||
f"Error: expected {expected_replacements} replacements but "
|
|
||||||
f"would make {len(selected)}."
|
|
||||||
)
|
|
||||||
new_content = content
|
new_content = content
|
||||||
for match in reversed(selected):
|
for match in reversed(selected):
|
||||||
replacement = _preserve_quote_style(norm_old, match.text, norm_new)
|
replacement = _preserve_quote_style(norm_old, match.text, norm_new)
|
||||||
|
|||||||
@@ -14,7 +14,6 @@ from nanobot.agent.tools.schema import (
|
|||||||
StringSchema,
|
StringSchema,
|
||||||
tool_parameters_schema,
|
tool_parameters_schema,
|
||||||
)
|
)
|
||||||
from nanobot.security.workspace_access import current_tool_workspace
|
|
||||||
from nanobot.config.paths import get_media_dir
|
from nanobot.config.paths import get_media_dir
|
||||||
from nanobot.config.schema import Base
|
from nanobot.config.schema import Base
|
||||||
from nanobot.providers.image_generation import (
|
from nanobot.providers.image_generation import (
|
||||||
@@ -22,7 +21,6 @@ from nanobot.providers.image_generation import (
|
|||||||
ImageGenerationProvider,
|
ImageGenerationProvider,
|
||||||
get_image_gen_provider,
|
get_image_gen_provider,
|
||||||
)
|
)
|
||||||
from nanobot.security.workspace_policy import WorkspaceBoundaryError, resolve_allowed_path
|
|
||||||
from nanobot.utils.artifacts import (
|
from nanobot.utils.artifacts import (
|
||||||
ArtifactError,
|
ArtifactError,
|
||||||
generated_image_tool_result,
|
generated_image_tool_result,
|
||||||
@@ -132,23 +130,25 @@ class ImageGenerationTool(Tool):
|
|||||||
}
|
}
|
||||||
return cls(**kwargs)
|
return cls(**kwargs)
|
||||||
|
|
||||||
|
def _missing_api_key_error(self) -> str:
|
||||||
|
cls = get_image_gen_provider(self.config.provider)
|
||||||
|
if cls and cls.missing_key_message:
|
||||||
|
return f"Error: {cls.missing_key_message}"
|
||||||
|
return f"Error: {self.config.provider} API key is not configured."
|
||||||
|
|
||||||
def _resolve_reference_image(self, value: str) -> str:
|
def _resolve_reference_image(self, value: str) -> str:
|
||||||
access = current_tool_workspace(self.workspace, restrict_to_workspace=True)
|
raw_path = Path(value).expanduser()
|
||||||
workspace = access.project_path or self.workspace
|
path = raw_path if raw_path.is_absolute() else self.workspace / raw_path
|
||||||
try:
|
try:
|
||||||
resolved = resolve_allowed_path(
|
resolved = path.resolve(strict=True)
|
||||||
value,
|
|
||||||
workspace=workspace,
|
|
||||||
allowed_root=access.allowed_root,
|
|
||||||
extra_allowed_roots=[get_media_dir()] if access.allowed_root is not None else None,
|
|
||||||
strict=True,
|
|
||||||
)
|
|
||||||
except WorkspaceBoundaryError as exc:
|
|
||||||
raise ImageGenerationError(
|
|
||||||
"reference_images must be inside the workspace or nanobot media directory"
|
|
||||||
) from exc
|
|
||||||
except OSError as exc:
|
except OSError as exc:
|
||||||
raise ImageGenerationError(f"reference image not found: {value}") from exc
|
raise ImageGenerationError(f"reference image not found: {value}") from exc
|
||||||
|
|
||||||
|
allowed_roots = [self.workspace.resolve(), get_media_dir().resolve()]
|
||||||
|
if not any(_is_relative_to(resolved, root) for root in allowed_roots):
|
||||||
|
raise ImageGenerationError(
|
||||||
|
"reference_images must be inside the workspace or nanobot media directory"
|
||||||
|
)
|
||||||
if not resolved.is_file():
|
if not resolved.is_file():
|
||||||
raise ImageGenerationError(f"reference image is not a file: {value}")
|
raise ImageGenerationError(f"reference image is not a file: {value}")
|
||||||
raw = resolved.read_bytes()
|
raw = resolved.read_bytes()
|
||||||
@@ -173,6 +173,9 @@ class ImageGenerationTool(Tool):
|
|||||||
client = self._provider_client()
|
client = self._provider_client()
|
||||||
if client is None:
|
if client is None:
|
||||||
return f"Error: unsupported image generation provider '{self.config.provider}'"
|
return f"Error: unsupported image generation provider '{self.config.provider}'"
|
||||||
|
provider = self._provider_config()
|
||||||
|
if not provider or not provider.api_key:
|
||||||
|
return self._missing_api_key_error()
|
||||||
|
|
||||||
requested = count or 1
|
requested = count or 1
|
||||||
if requested > self.config.max_images_per_turn:
|
if requested > self.config.max_images_per_turn:
|
||||||
@@ -207,3 +210,11 @@ class ImageGenerationTool(Tool):
|
|||||||
return generated_image_tool_result(artifacts)
|
return generated_image_tool_result(artifacts)
|
||||||
except (ArtifactError, ImageGenerationError, OSError) as exc:
|
except (ArtifactError, ImageGenerationError, OSError) as exc:
|
||||||
return f"Error: {exc}"
|
return f"Error: {exc}"
|
||||||
|
|
||||||
|
|
||||||
|
def _is_relative_to(path: Path, root: Path) -> bool:
|
||||||
|
try:
|
||||||
|
path.relative_to(root)
|
||||||
|
except ValueError:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|||||||
@@ -16,7 +16,6 @@ There is **no** sub-agent orchestrator and **no** special WebSocket ``agent_ui``
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from contextvars import ContextVar
|
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
@@ -46,22 +45,15 @@ class _GoalToolsMixin(ContextAware):
|
|||||||
def __init__(self, sessions: SessionManager, bus: Any | None = None) -> None:
|
def __init__(self, sessions: SessionManager, bus: Any | None = None) -> None:
|
||||||
self._sessions = sessions
|
self._sessions = sessions
|
||||||
self._bus = bus
|
self._bus = bus
|
||||||
# Each subclass gets its own ContextVar so concurrent tasks across
|
self._request_ctx: RequestContext | None = None
|
||||||
# different tool types (LongTaskTool vs CompleteGoalTool) do not
|
|
||||||
# interfere with each other.
|
|
||||||
self._request_ctx: ContextVar[RequestContext | None] = ContextVar(
|
|
||||||
f"{self.__class__.__name__}_request_ctx",
|
|
||||||
default=None,
|
|
||||||
)
|
|
||||||
|
|
||||||
def set_context(self, ctx: RequestContext) -> None:
|
def set_context(self, ctx: RequestContext) -> None:
|
||||||
self._request_ctx.set(ctx)
|
self._request_ctx = ctx
|
||||||
|
|
||||||
def _session(self):
|
def _session(self):
|
||||||
request_ctx = self._request_ctx.get()
|
if self._request_ctx is None:
|
||||||
if request_ctx is None:
|
|
||||||
return None
|
return None
|
||||||
key = request_ctx.session_key
|
key = self._request_ctx.session_key
|
||||||
if not key:
|
if not key:
|
||||||
return None
|
return None
|
||||||
return self._sessions.get_or_create(key)
|
return self._sessions.get_or_create(key)
|
||||||
@@ -69,7 +61,7 @@ class _GoalToolsMixin(ContextAware):
|
|||||||
async def _publish_goal_state_ws(self, metadata: dict[str, Any]) -> None:
|
async def _publish_goal_state_ws(self, metadata: dict[str, Any]) -> None:
|
||||||
"""Fan-out authoritative goal snapshot for this WebSocket chat only."""
|
"""Fan-out authoritative goal snapshot for this WebSocket chat only."""
|
||||||
bus = self._bus
|
bus = self._bus
|
||||||
rc = self._request_ctx.get()
|
rc = self._request_ctx
|
||||||
if bus is None or rc is None or rc.channel != "websocket":
|
if bus is None or rc is None or rc.channel != "websocket":
|
||||||
return
|
return
|
||||||
cid = (rc.chat_id or "").strip()
|
cid = (rc.chat_id or "").strip()
|
||||||
@@ -232,3 +224,4 @@ class CompleteGoalTool(Tool, _GoalToolsMixin):
|
|||||||
if tail:
|
if tail:
|
||||||
return f"Goal marked complete ({ended}). Recap:\n{tail}"
|
return f"Goal marked complete ({ended}). Recap:\n{tail}"
|
||||||
return f"Goal marked complete ({ended})."
|
return f"Goal marked complete ({ended})."
|
||||||
|
|
||||||
|
|||||||
+1
-279
@@ -6,20 +6,13 @@ import re
|
|||||||
import shutil
|
import shutil
|
||||||
import urllib.parse
|
import urllib.parse
|
||||||
from contextlib import AsyncExitStack, suppress
|
from contextlib import AsyncExitStack, suppress
|
||||||
from typing import Any, Mapping
|
from typing import Any
|
||||||
from weakref import WeakKeyDictionary
|
|
||||||
|
|
||||||
import httpx
|
import httpx
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
from nanobot.agent.tools.base import Tool
|
from nanobot.agent.tools.base import Tool
|
||||||
from nanobot.agent.tools.registry import ToolRegistry
|
from nanobot.agent.tools.registry import ToolRegistry
|
||||||
from nanobot.bus.events import (
|
|
||||||
INBOUND_META_RUNTIME_CONTROL,
|
|
||||||
RUNTIME_CONTROL_ACK,
|
|
||||||
RUNTIME_CONTROL_MCP_RELOAD,
|
|
||||||
InboundMessage,
|
|
||||||
)
|
|
||||||
|
|
||||||
# Transient connection errors that warrant a single retry.
|
# Transient connection errors that warrant a single retry.
|
||||||
# These typically happen when an MCP server restarts or a network
|
# These typically happen when an MCP server restarts or a network
|
||||||
@@ -40,7 +33,6 @@ _WINDOWS_SHELL_LAUNCHERS: frozenset[str] = frozenset(("npx", "npm", "pnpm", "yar
|
|||||||
# Characters allowed in tool names by model providers (Anthropic, OpenAI, etc.).
|
# Characters allowed in tool names by model providers (Anthropic, OpenAI, etc.).
|
||||||
# Replace anything outside [a-zA-Z0-9_-] with underscore and collapse runs.
|
# Replace anything outside [a-zA-Z0-9_-] with underscore and collapse runs.
|
||||||
_SANITIZE_RE = re.compile(r"_+")
|
_SANITIZE_RE = re.compile(r"_+")
|
||||||
_RELOAD_LOCKS: WeakKeyDictionary[Any, asyncio.Lock] = WeakKeyDictionary()
|
|
||||||
|
|
||||||
|
|
||||||
def _sanitize_name(name: str) -> str:
|
def _sanitize_name(name: str) -> str:
|
||||||
@@ -511,7 +503,6 @@ async def connect_mcp_servers(
|
|||||||
command=command,
|
command=command,
|
||||||
args=args,
|
args=args,
|
||||||
env=env,
|
env=env,
|
||||||
cwd=cfg.cwd or None,
|
|
||||||
)
|
)
|
||||||
read, write = await server_stack.enter_async_context(stdio_client(params))
|
read, write = await server_stack.enter_async_context(stdio_client(params))
|
||||||
elif transport_type == "sse":
|
elif transport_type == "sse":
|
||||||
@@ -671,272 +662,3 @@ async def connect_mcp_servers(
|
|||||||
server_stacks[result[0]] = result[1]
|
server_stacks[result[0]] = result[1]
|
||||||
|
|
||||||
return server_stacks
|
return server_stacks
|
||||||
|
|
||||||
|
|
||||||
def session_extra(metadata: Mapping[str, Any] | None) -> dict[str, Any]:
|
|
||||||
"""Return persisted session kwargs for MCP preset attachments."""
|
|
||||||
mcp_presets = metadata.get("mcp_presets") if isinstance(metadata, Mapping) else None
|
|
||||||
return {"mcp_presets": mcp_presets} if isinstance(mcp_presets, list) and mcp_presets else {}
|
|
||||||
|
|
||||||
|
|
||||||
def runtime_lines(
|
|
||||||
message: Any,
|
|
||||||
*,
|
|
||||||
available_server_names: set[str] | None = None,
|
|
||||||
configured_server_names: set[str] | None = None,
|
|
||||||
connected_server_names: set[str] | None = None,
|
|
||||||
skip: bool = False,
|
|
||||||
) -> list[str]:
|
|
||||||
"""Return model-visible MCP preset annotations for the current turn."""
|
|
||||||
if skip:
|
|
||||||
return []
|
|
||||||
if configured_server_names is None:
|
|
||||||
configured_server_names = available_server_names
|
|
||||||
if connected_server_names is None:
|
|
||||||
connected_server_names = available_server_names
|
|
||||||
metadata = message.metadata if isinstance(getattr(message, "metadata", None), Mapping) else None
|
|
||||||
structured = metadata.get("mcp_presets") if isinstance(metadata, Mapping) else None
|
|
||||||
if not isinstance(structured, list):
|
|
||||||
return []
|
|
||||||
|
|
||||||
lines: list[str] = []
|
|
||||||
for item in structured[:8]:
|
|
||||||
if not isinstance(item, Mapping):
|
|
||||||
continue
|
|
||||||
raw_name = str(item.get("name") or "").strip().lower()
|
|
||||||
if not raw_name:
|
|
||||||
continue
|
|
||||||
display = str(item.get("display_name") or raw_name).strip() or raw_name
|
|
||||||
transport = str(item.get("transport") or "mcp").strip() or "mcp"
|
|
||||||
prefix = f"mcp_{raw_name}_"
|
|
||||||
if configured_server_names is not None and raw_name not in configured_server_names:
|
|
||||||
lines.append(
|
|
||||||
"MCP Preset Attachment: "
|
|
||||||
f"@{raw_name} ({display}; transport={transport}) is configured in WebUI Settings, "
|
|
||||||
"but this gateway has not loaded the latest MCP settings yet. "
|
|
||||||
f"Tools with prefix `{prefix}` may not be available yet; if they are missing, "
|
|
||||||
"tell the user to restart nanobot."
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
if connected_server_names is not None and raw_name not in connected_server_names:
|
|
||||||
lines.append(
|
|
||||||
"MCP Preset Attachment: "
|
|
||||||
f"@{raw_name} ({display}; transport={transport}) is configured, "
|
|
||||||
"but its MCP connection is not currently live. "
|
|
||||||
f"Tools with prefix `{prefix}` may be unavailable; tell the user to open Settings, "
|
|
||||||
"run the preset test, and restart nanobot only if hot reload is unavailable."
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
lines.append(
|
|
||||||
"MCP Preset Attachment: "
|
|
||||||
f"@{raw_name} ({display}; transport={transport}; tool_prefix={prefix}). "
|
|
||||||
f"Prefer available tools whose names start with `{prefix}` for this request; "
|
|
||||||
"do not substitute shell commands for this MCP integration unless the user asks."
|
|
||||||
)
|
|
||||||
return lines
|
|
||||||
|
|
||||||
|
|
||||||
async def connect_missing_servers(state: Any, registry: ToolRegistry) -> None:
|
|
||||||
"""Connect configured MCP servers that are not currently live."""
|
|
||||||
missing_servers = {
|
|
||||||
name: cfg for name, cfg in state._mcp_servers.items() if name not in state._mcp_stacks
|
|
||||||
}
|
|
||||||
if state._mcp_connecting or not missing_servers:
|
|
||||||
return
|
|
||||||
state._mcp_connecting = True
|
|
||||||
try:
|
|
||||||
connected = await connect_mcp_servers(missing_servers, registry)
|
|
||||||
state._mcp_stacks.update(connected)
|
|
||||||
state._mcp_connected = bool(state._mcp_stacks)
|
|
||||||
if connected:
|
|
||||||
logger.info("MCP connected servers: {}", sorted(connected))
|
|
||||||
else:
|
|
||||||
logger.warning("No MCP servers connected successfully (will retry next message)")
|
|
||||||
except asyncio.CancelledError:
|
|
||||||
logger.warning("MCP connection cancelled (will retry next message)")
|
|
||||||
state._mcp_connected = bool(state._mcp_stacks)
|
|
||||||
except BaseException as e:
|
|
||||||
logger.warning("Failed to connect MCP servers (will retry next message): {}", e)
|
|
||||||
state._mcp_connected = bool(state._mcp_stacks)
|
|
||||||
finally:
|
|
||||||
state._mcp_connecting = False
|
|
||||||
|
|
||||||
|
|
||||||
async def reload_servers(state: Any, registry: ToolRegistry) -> dict[str, Any]:
|
|
||||||
"""Reconcile live MCP connections with the current config file."""
|
|
||||||
async with _reload_lock(state):
|
|
||||||
try:
|
|
||||||
from nanobot.config.loader import (load_config,
|
|
||||||
resolve_config_env_vars)
|
|
||||||
|
|
||||||
config = resolve_config_env_vars(load_config())
|
|
||||||
next_servers = dict(config.tools.mcp_servers)
|
|
||||||
except Exception as exc:
|
|
||||||
logger.warning("MCP hot reload could not read config: {}", exc)
|
|
||||||
return {
|
|
||||||
"ok": False,
|
|
||||||
"message": "Could not reload MCP config. Restart nanobot to pick up changes.",
|
|
||||||
"requires_restart": True,
|
|
||||||
"error": str(exc),
|
|
||||||
}
|
|
||||||
|
|
||||||
current_servers = dict(state._mcp_servers)
|
|
||||||
current_names = set(current_servers)
|
|
||||||
next_names = set(next_servers)
|
|
||||||
removed = sorted(current_names - next_names)
|
|
||||||
added = sorted(next_names - current_names)
|
|
||||||
changed = sorted(
|
|
||||||
name
|
|
||||||
for name in current_names & next_names
|
|
||||||
if _server_signature(current_servers[name]) != _server_signature(next_servers[name])
|
|
||||||
)
|
|
||||||
|
|
||||||
tools_removed = 0
|
|
||||||
for name in [*removed, *changed]:
|
|
||||||
tools_removed += _unregister_server_tools(state, registry, name)
|
|
||||||
await _close_server(state, name)
|
|
||||||
|
|
||||||
state._mcp_servers = next_servers
|
|
||||||
retry_missing = sorted(
|
|
||||||
name
|
|
||||||
for name in next_names
|
|
||||||
if name not in state._mcp_stacks and name not in set(added) | set(changed)
|
|
||||||
)
|
|
||||||
to_connect_names = sorted(set(added) | set(changed) | set(retry_missing))
|
|
||||||
to_connect = {name: next_servers[name] for name in to_connect_names}
|
|
||||||
connected: dict[str, AsyncExitStack] = {}
|
|
||||||
if to_connect:
|
|
||||||
connected = await connect_mcp_servers(to_connect, registry)
|
|
||||||
state._mcp_stacks.update(connected)
|
|
||||||
|
|
||||||
state._mcp_connected = bool(state._mcp_stacks)
|
|
||||||
failed = sorted(set(to_connect) - set(connected))
|
|
||||||
unchanged = not removed and not added and not changed and not retry_missing
|
|
||||||
ok = not failed
|
|
||||||
if failed:
|
|
||||||
message = "MCP config reloaded, but some servers did not connect: " + ", ".join(failed)
|
|
||||||
elif unchanged:
|
|
||||||
message = "MCP config is already live."
|
|
||||||
elif retry_missing and not added and not changed and not removed:
|
|
||||||
message = "MCP connections refreshed without restarting nanobot."
|
|
||||||
else:
|
|
||||||
message = "MCP config reloaded without restarting nanobot."
|
|
||||||
|
|
||||||
logger.info(
|
|
||||||
"MCP hot reload: added={} changed={} removed={} retried={} connected={} failed={} tools_removed={}",
|
|
||||||
added,
|
|
||||||
changed,
|
|
||||||
removed,
|
|
||||||
retry_missing,
|
|
||||||
sorted(connected),
|
|
||||||
failed,
|
|
||||||
tools_removed,
|
|
||||||
)
|
|
||||||
return {
|
|
||||||
"ok": ok,
|
|
||||||
"message": message,
|
|
||||||
"added": added,
|
|
||||||
"changed": changed,
|
|
||||||
"removed": removed,
|
|
||||||
"retried": retry_missing,
|
|
||||||
"connected": sorted(state._mcp_stacks),
|
|
||||||
"configured": sorted(state._mcp_servers),
|
|
||||||
"failed": failed,
|
|
||||||
"tools_removed": tools_removed,
|
|
||||||
"requires_restart": False,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
async def request_mcp_reload(bus: Any, *, timeout: float = 15.0) -> dict[str, Any]:
|
|
||||||
"""Ask the running agent loop to reconcile live MCP connections."""
|
|
||||||
loop = asyncio.get_running_loop()
|
|
||||||
ack: asyncio.Future[dict[str, Any]] = loop.create_future()
|
|
||||||
await bus.publish_inbound(
|
|
||||||
InboundMessage(
|
|
||||||
channel="system",
|
|
||||||
sender_id="webui-settings",
|
|
||||||
chat_id="runtime",
|
|
||||||
content=RUNTIME_CONTROL_MCP_RELOAD,
|
|
||||||
metadata={
|
|
||||||
INBOUND_META_RUNTIME_CONTROL: RUNTIME_CONTROL_MCP_RELOAD,
|
|
||||||
RUNTIME_CONTROL_ACK: ack,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
result = await asyncio.wait_for(ack, timeout=timeout)
|
|
||||||
except asyncio.TimeoutError:
|
|
||||||
return {
|
|
||||||
"ok": False,
|
|
||||||
"message": "MCP hot reload timed out. Restart nanobot to pick up changes.",
|
|
||||||
"requires_restart": True,
|
|
||||||
}
|
|
||||||
return result if isinstance(result, dict) else {
|
|
||||||
"ok": False,
|
|
||||||
"message": "MCP hot reload returned an unexpected response.",
|
|
||||||
"requires_restart": True,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
async def handle_runtime_control(state: Any, msg: InboundMessage, registry: ToolRegistry) -> bool:
|
|
||||||
metadata = msg.metadata if isinstance(msg.metadata, dict) else {}
|
|
||||||
control = metadata.get(INBOUND_META_RUNTIME_CONTROL)
|
|
||||||
if control != RUNTIME_CONTROL_MCP_RELOAD:
|
|
||||||
return False
|
|
||||||
|
|
||||||
ack = metadata.get(RUNTIME_CONTROL_ACK)
|
|
||||||
try:
|
|
||||||
result = await reload_servers(state, registry)
|
|
||||||
except Exception as exc:
|
|
||||||
logger.exception("MCP hot reload failed")
|
|
||||||
result = {
|
|
||||||
"ok": False,
|
|
||||||
"message": "MCP hot reload failed. Restart nanobot to pick up changes.",
|
|
||||||
"requires_restart": True,
|
|
||||||
"error": str(exc),
|
|
||||||
}
|
|
||||||
if isinstance(ack, asyncio.Future) and not ack.done():
|
|
||||||
ack.set_result(result)
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
def _reload_lock(state: Any) -> asyncio.Lock:
|
|
||||||
try:
|
|
||||||
return _RELOAD_LOCKS[state]
|
|
||||||
except KeyError:
|
|
||||||
lock = asyncio.Lock()
|
|
||||||
_RELOAD_LOCKS[state] = lock
|
|
||||||
return lock
|
|
||||||
|
|
||||||
|
|
||||||
def _server_signature(cfg: Any) -> Any:
|
|
||||||
if hasattr(cfg, "model_dump"):
|
|
||||||
return cfg.model_dump(mode="json")
|
|
||||||
return cfg
|
|
||||||
|
|
||||||
|
|
||||||
def _tool_prefix(server_name: str) -> str:
|
|
||||||
safe_name = "".join(ch if ch.isalnum() or ch in {"_", "-"} else "_" for ch in server_name)
|
|
||||||
while "__" in safe_name:
|
|
||||||
safe_name = safe_name.replace("__", "_")
|
|
||||||
return f"mcp_{safe_name}_"
|
|
||||||
|
|
||||||
|
|
||||||
def _unregister_server_tools(state: Any, registry: ToolRegistry, server_name: str) -> int:
|
|
||||||
prefix = _tool_prefix(server_name)
|
|
||||||
removed = 0
|
|
||||||
for tool_name in list(registry.tool_names):
|
|
||||||
if tool_name.startswith(prefix):
|
|
||||||
registry.unregister(tool_name)
|
|
||||||
removed += 1
|
|
||||||
return removed
|
|
||||||
|
|
||||||
|
|
||||||
async def _close_server(state: Any, server_name: str) -> None:
|
|
||||||
stack = state._mcp_stacks.pop(server_name, None)
|
|
||||||
if stack is None:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
await stack.aclose()
|
|
||||||
except (RuntimeError, BaseExceptionGroup):
|
|
||||||
logger.debug("MCP server '{}' cleanup error (can be ignored)", server_name)
|
|
||||||
|
|||||||
@@ -8,7 +8,6 @@ from nanobot.agent.tools.base import Tool, tool_parameters
|
|||||||
from nanobot.agent.tools.context import ContextAware, RequestContext
|
from nanobot.agent.tools.context import ContextAware, RequestContext
|
||||||
from nanobot.agent.tools.path_utils import resolve_workspace_path
|
from nanobot.agent.tools.path_utils import resolve_workspace_path
|
||||||
from nanobot.agent.tools.schema import ArraySchema, StringSchema, tool_parameters_schema
|
from nanobot.agent.tools.schema import ArraySchema, StringSchema, tool_parameters_schema
|
||||||
from nanobot.security.workspace_access import current_tool_workspace
|
|
||||||
from nanobot.bus.events import OutboundMessage
|
from nanobot.bus.events import OutboundMessage
|
||||||
from nanobot.config.paths import get_workspace_path
|
from nanobot.config.paths import get_workspace_path
|
||||||
|
|
||||||
@@ -83,10 +82,6 @@ class MessageTool(Tool, ContextAware):
|
|||||||
"message_record_channel_delivery",
|
"message_record_channel_delivery",
|
||||||
default=False,
|
default=False,
|
||||||
)
|
)
|
||||||
self._suppress_delivery_var: ContextVar[bool] = ContextVar(
|
|
||||||
"message_suppress_delivery",
|
|
||||||
default=False,
|
|
||||||
)
|
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def create(cls, ctx: Any) -> Tool:
|
def create(cls, ctx: Any) -> Tool:
|
||||||
@@ -125,14 +120,6 @@ class MessageTool(Tool, ContextAware):
|
|||||||
"""Restore previous proactive delivery recording state."""
|
"""Restore previous proactive delivery recording state."""
|
||||||
self._record_channel_delivery_var.reset(token)
|
self._record_channel_delivery_var.reset(token)
|
||||||
|
|
||||||
def set_suppress_delivery(self, active: bool):
|
|
||||||
"""Temporarily suppress real channel delivery for internal checks."""
|
|
||||||
return self._suppress_delivery_var.set(active)
|
|
||||||
|
|
||||||
def reset_suppress_delivery(self, token) -> None:
|
|
||||||
"""Restore previous channel delivery suppression state."""
|
|
||||||
self._suppress_delivery_var.reset(token)
|
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def _sent_in_turn(self) -> bool:
|
def _sent_in_turn(self) -> bool:
|
||||||
return self._sent_in_turn_var.get()
|
return self._sent_in_turn_var.get()
|
||||||
@@ -162,19 +149,15 @@ class MessageTool(Tool, ContextAware):
|
|||||||
def _resolve_media(self, media: list[str]) -> list[str]:
|
def _resolve_media(self, media: list[str]) -> list[str]:
|
||||||
"""Resolve local media attachments and enforce workspace restriction when enabled."""
|
"""Resolve local media attachments and enforce workspace restriction when enabled."""
|
||||||
resolved: list[str] = []
|
resolved: list[str] = []
|
||||||
access = current_tool_workspace(
|
allowed_dir = self._workspace if self._restrict_to_workspace else None
|
||||||
self._workspace,
|
|
||||||
restrict_to_workspace=self._restrict_to_workspace,
|
|
||||||
)
|
|
||||||
workspace = access.project_path or self._workspace
|
|
||||||
for p in media:
|
for p in media:
|
||||||
if p.startswith(("http://", "https://")):
|
if p.startswith(("http://", "https://")):
|
||||||
resolved.append(p)
|
resolved.append(p)
|
||||||
elif not access.restrict_to_workspace:
|
elif not self._restrict_to_workspace:
|
||||||
path = Path(p).expanduser()
|
path = Path(p).expanduser()
|
||||||
resolved.append(p if path.is_absolute() else str(workspace / path))
|
resolved.append(p if path.is_absolute() else str(self._workspace / path))
|
||||||
else:
|
else:
|
||||||
resolved.append(str(resolve_workspace_path(p, workspace, access.allowed_root)))
|
resolved.append(str(resolve_workspace_path(p, self._workspace, allowed_dir)))
|
||||||
return resolved
|
return resolved
|
||||||
|
|
||||||
async def execute(
|
async def execute(
|
||||||
@@ -229,9 +212,6 @@ class MessageTool(Tool, ContextAware):
|
|||||||
if not channel or not chat_id:
|
if not channel or not chat_id:
|
||||||
return "Error: No target channel/chat specified"
|
return "Error: No target channel/chat specified"
|
||||||
|
|
||||||
if self._suppress_delivery_var.get():
|
|
||||||
return "Message suppressed during internal check"
|
|
||||||
|
|
||||||
if not self._send_callback:
|
if not self._send_callback:
|
||||||
return "Error: Message sending not configured"
|
return "Error: Message sending not configured"
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,162 @@
|
|||||||
|
"""NotebookEditTool — edit Jupyter .ipynb notebooks."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import uuid
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from nanobot.agent.tools.base import tool_parameters
|
||||||
|
from nanobot.agent.tools.schema import IntegerSchema, StringSchema, tool_parameters_schema
|
||||||
|
from nanobot.agent.tools.filesystem import _FsTool
|
||||||
|
|
||||||
|
|
||||||
|
def _new_cell(source: str, cell_type: str = "code", generate_id: bool = False) -> dict:
|
||||||
|
cell: dict[str, Any] = {
|
||||||
|
"cell_type": cell_type,
|
||||||
|
"source": source,
|
||||||
|
"metadata": {},
|
||||||
|
}
|
||||||
|
if cell_type == "code":
|
||||||
|
cell["outputs"] = []
|
||||||
|
cell["execution_count"] = None
|
||||||
|
if generate_id:
|
||||||
|
cell["id"] = uuid.uuid4().hex[:8]
|
||||||
|
return cell
|
||||||
|
|
||||||
|
|
||||||
|
def _make_empty_notebook() -> dict:
|
||||||
|
return {
|
||||||
|
"nbformat": 4,
|
||||||
|
"nbformat_minor": 5,
|
||||||
|
"metadata": {
|
||||||
|
"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"},
|
||||||
|
"language_info": {"name": "python"},
|
||||||
|
},
|
||||||
|
"cells": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@tool_parameters(
|
||||||
|
tool_parameters_schema(
|
||||||
|
path=StringSchema("Path to the .ipynb notebook file"),
|
||||||
|
cell_index=IntegerSchema(0, description="0-based index of the cell to edit", minimum=0),
|
||||||
|
new_source=StringSchema("New source content for the cell"),
|
||||||
|
cell_type=StringSchema(
|
||||||
|
"Cell type: 'code' or 'markdown' (default: code)",
|
||||||
|
enum=["code", "markdown"],
|
||||||
|
),
|
||||||
|
edit_mode=StringSchema(
|
||||||
|
"Mode: 'replace' (default), 'insert' (after target), or 'delete'",
|
||||||
|
enum=["replace", "insert", "delete"],
|
||||||
|
),
|
||||||
|
required=["path", "cell_index"],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
class NotebookEditTool(_FsTool):
|
||||||
|
"""Edit Jupyter notebook cells: replace, insert, or delete."""
|
||||||
|
_scopes = {"core"}
|
||||||
|
|
||||||
|
_VALID_CELL_TYPES = frozenset({"code", "markdown"})
|
||||||
|
_VALID_EDIT_MODES = frozenset({"replace", "insert", "delete"})
|
||||||
|
|
||||||
|
@property
|
||||||
|
def name(self) -> str:
|
||||||
|
return "notebook_edit"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def description(self) -> str:
|
||||||
|
return (
|
||||||
|
"Edit a Jupyter notebook (.ipynb) cell. "
|
||||||
|
"Modes: replace (default) replaces cell content, "
|
||||||
|
"insert adds a new cell after the target index, "
|
||||||
|
"delete removes the cell at the index. "
|
||||||
|
"cell_index is 0-based."
|
||||||
|
)
|
||||||
|
|
||||||
|
async def execute(
|
||||||
|
self,
|
||||||
|
path: str | None = None,
|
||||||
|
cell_index: int = 0,
|
||||||
|
new_source: str = "",
|
||||||
|
cell_type: str = "code",
|
||||||
|
edit_mode: str = "replace",
|
||||||
|
**kwargs: Any,
|
||||||
|
) -> str:
|
||||||
|
try:
|
||||||
|
if not path:
|
||||||
|
return "Error: path is required"
|
||||||
|
|
||||||
|
if not path.endswith(".ipynb"):
|
||||||
|
return "Error: notebook_edit only works on .ipynb files. Use edit_file for other files."
|
||||||
|
|
||||||
|
if edit_mode not in self._VALID_EDIT_MODES:
|
||||||
|
return (
|
||||||
|
f"Error: Invalid edit_mode '{edit_mode}'. "
|
||||||
|
"Use one of: replace, insert, delete."
|
||||||
|
)
|
||||||
|
|
||||||
|
if cell_type not in self._VALID_CELL_TYPES:
|
||||||
|
return (
|
||||||
|
f"Error: Invalid cell_type '{cell_type}'. "
|
||||||
|
"Use one of: code, markdown."
|
||||||
|
)
|
||||||
|
|
||||||
|
fp = self._resolve(path)
|
||||||
|
|
||||||
|
# Create new notebook if file doesn't exist and mode is insert
|
||||||
|
if not fp.exists():
|
||||||
|
if edit_mode != "insert":
|
||||||
|
return f"Error: File not found: {path}"
|
||||||
|
nb = _make_empty_notebook()
|
||||||
|
cell = _new_cell(new_source, cell_type, generate_id=True)
|
||||||
|
nb["cells"].append(cell)
|
||||||
|
fp.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
fp.write_text(json.dumps(nb, indent=1, ensure_ascii=False), encoding="utf-8")
|
||||||
|
return f"Successfully created {fp} with 1 cell"
|
||||||
|
|
||||||
|
try:
|
||||||
|
nb = json.loads(fp.read_text(encoding="utf-8"))
|
||||||
|
except (json.JSONDecodeError, UnicodeDecodeError) as e:
|
||||||
|
return f"Error: Failed to parse notebook: {e}"
|
||||||
|
|
||||||
|
cells = nb.get("cells", [])
|
||||||
|
nbformat_minor = nb.get("nbformat_minor", 0)
|
||||||
|
generate_id = nb.get("nbformat", 0) >= 4 and nbformat_minor >= 5
|
||||||
|
|
||||||
|
if edit_mode == "delete":
|
||||||
|
if cell_index < 0 or cell_index >= len(cells):
|
||||||
|
return f"Error: cell_index {cell_index} out of range (notebook has {len(cells)} cells)"
|
||||||
|
cells.pop(cell_index)
|
||||||
|
nb["cells"] = cells
|
||||||
|
fp.write_text(json.dumps(nb, indent=1, ensure_ascii=False), encoding="utf-8")
|
||||||
|
return f"Successfully deleted cell {cell_index} from {fp}"
|
||||||
|
|
||||||
|
if edit_mode == "insert":
|
||||||
|
insert_at = min(cell_index + 1, len(cells))
|
||||||
|
cell = _new_cell(new_source, cell_type, generate_id=generate_id)
|
||||||
|
cells.insert(insert_at, cell)
|
||||||
|
nb["cells"] = cells
|
||||||
|
fp.write_text(json.dumps(nb, indent=1, ensure_ascii=False), encoding="utf-8")
|
||||||
|
return f"Successfully inserted cell at index {insert_at} in {fp}"
|
||||||
|
|
||||||
|
# Default: replace
|
||||||
|
if cell_index < 0 or cell_index >= len(cells):
|
||||||
|
return f"Error: cell_index {cell_index} out of range (notebook has {len(cells)} cells)"
|
||||||
|
cells[cell_index]["source"] = new_source
|
||||||
|
if cell_type and cells[cell_index].get("cell_type") != cell_type:
|
||||||
|
cells[cell_index]["cell_type"] = cell_type
|
||||||
|
if cell_type == "code":
|
||||||
|
cells[cell_index].setdefault("outputs", [])
|
||||||
|
cells[cell_index].setdefault("execution_count", None)
|
||||||
|
elif "outputs" in cells[cell_index]:
|
||||||
|
del cells[cell_index]["outputs"]
|
||||||
|
cells[cell_index].pop("execution_count", None)
|
||||||
|
nb["cells"] = cells
|
||||||
|
fp.write_text(json.dumps(nb, indent=1, ensure_ascii=False), encoding="utf-8")
|
||||||
|
return f"Successfully edited cell {cell_index} in {fp}"
|
||||||
|
|
||||||
|
except PermissionError as e:
|
||||||
|
return f"Error: {e}"
|
||||||
|
except Exception as e:
|
||||||
|
return f"Error editing notebook: {e}"
|
||||||
@@ -3,15 +3,21 @@
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from nanobot.config.paths import get_media_dir
|
from nanobot.config.paths import get_media_dir
|
||||||
from nanobot.security.workspace_policy import (
|
|
||||||
is_path_within,
|
WORKSPACE_BOUNDARY_NOTE = (
|
||||||
resolve_allowed_path,
|
" (this is a hard policy boundary, not a transient failure; "
|
||||||
|
"do not retry with shell tricks or alternative tools, and ask "
|
||||||
|
"the user how to proceed if the resource is genuinely required)"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def is_under(path: Path, directory: Path) -> bool:
|
def is_under(path: Path, directory: Path) -> bool:
|
||||||
"""Return True when path resolves under directory."""
|
"""Return True when path resolves under directory."""
|
||||||
return is_path_within(path, directory)
|
try:
|
||||||
|
path.relative_to(directory.resolve())
|
||||||
|
return True
|
||||||
|
except ValueError:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
def resolve_workspace_path(
|
def resolve_workspace_path(
|
||||||
@@ -21,10 +27,16 @@ def resolve_workspace_path(
|
|||||||
extra_allowed_dirs: list[Path] | None = None,
|
extra_allowed_dirs: list[Path] | None = None,
|
||||||
) -> Path:
|
) -> Path:
|
||||||
"""Resolve path against workspace and enforce allowed directory containment."""
|
"""Resolve path against workspace and enforce allowed directory containment."""
|
||||||
extra_roots = [get_media_dir(), *(extra_allowed_dirs or [])] if allowed_dir else None
|
p = Path(path).expanduser()
|
||||||
return resolve_allowed_path(
|
if not p.is_absolute() and workspace:
|
||||||
path,
|
p = workspace / p
|
||||||
workspace=workspace,
|
resolved = p.resolve()
|
||||||
allowed_root=allowed_dir,
|
if allowed_dir:
|
||||||
extra_allowed_roots=extra_roots,
|
media_path = get_media_dir().resolve()
|
||||||
|
all_dirs = [allowed_dir, media_path, *(extra_allowed_dirs or [])]
|
||||||
|
if not any(is_under(resolved, d) for d in all_dirs):
|
||||||
|
raise PermissionError(
|
||||||
|
f"Path {path} is outside allowed directory {allowed_dir}"
|
||||||
|
+ WORKSPACE_BOUNDARY_NOTE
|
||||||
)
|
)
|
||||||
|
return resolved
|
||||||
|
|||||||
@@ -42,9 +42,6 @@ class RuntimeState(Protocol):
|
|||||||
@property
|
@property
|
||||||
def exec_config(self) -> Any: ...
|
def exec_config(self) -> Any: ...
|
||||||
|
|
||||||
@property
|
|
||||||
def workspace_sandbox(self) -> Any: ...
|
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def subagents(self) -> Any: ...
|
def subagents(self) -> Any: ...
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
"""Search tools: file discovery and grep."""
|
"""Search tools: grep."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -12,7 +12,6 @@ from typing import Any, Iterable, TypeVar
|
|||||||
from nanobot.agent.tools.filesystem import ListDirTool, _FsTool
|
from nanobot.agent.tools.filesystem import ListDirTool, _FsTool
|
||||||
|
|
||||||
_DEFAULT_HEAD_LIMIT = 250
|
_DEFAULT_HEAD_LIMIT = 250
|
||||||
_DEFAULT_FILE_HEAD_LIMIT = 200
|
|
||||||
T = TypeVar("T")
|
T = TypeVar("T")
|
||||||
_TYPE_GLOB_MAP = {
|
_TYPE_GLOB_MAP = {
|
||||||
"py": ("*.py", "*.pyi"),
|
"py": ("*.py", "*.pyi"),
|
||||||
@@ -89,22 +88,13 @@ def _matches_type(name: str, file_type: str | None) -> bool:
|
|||||||
return any(fnmatch.fnmatch(name.lower(), pattern.lower()) for pattern in patterns)
|
return any(fnmatch.fnmatch(name.lower(), pattern.lower()) for pattern in patterns)
|
||||||
|
|
||||||
|
|
||||||
def _matches_query(rel_path: str, query: str | None) -> bool:
|
|
||||||
if not query:
|
|
||||||
return True
|
|
||||||
haystack = rel_path.lower()
|
|
||||||
terms = [part for part in query.lower().split() if part]
|
|
||||||
return all(term in haystack for term in terms)
|
|
||||||
|
|
||||||
|
|
||||||
class _SearchTool(_FsTool):
|
class _SearchTool(_FsTool):
|
||||||
_IGNORE_DIRS = set(ListDirTool._IGNORE_DIRS)
|
_IGNORE_DIRS = set(ListDirTool._IGNORE_DIRS)
|
||||||
|
|
||||||
def _display_path(self, target: Path, root: Path) -> str:
|
def _display_path(self, target: Path, root: Path) -> str:
|
||||||
workspace = self._display_workspace()
|
if self._workspace:
|
||||||
if workspace:
|
|
||||||
with suppress(ValueError):
|
with suppress(ValueError):
|
||||||
return target.relative_to(workspace).as_posix()
|
return target.relative_to(self._workspace).as_posix()
|
||||||
return target.relative_to(root).as_posix()
|
return target.relative_to(root).as_posix()
|
||||||
|
|
||||||
def _iter_files(self, root: Path) -> Iterable[Path]:
|
def _iter_files(self, root: Path) -> Iterable[Path]:
|
||||||
@@ -119,163 +109,6 @@ class _SearchTool(_FsTool):
|
|||||||
yield current / filename
|
yield current / filename
|
||||||
|
|
||||||
|
|
||||||
class FindFilesTool(_SearchTool):
|
|
||||||
"""Find files by path fragment, glob, or type."""
|
|
||||||
_scopes = {"core", "subagent"}
|
|
||||||
|
|
||||||
@property
|
|
||||||
def name(self) -> str:
|
|
||||||
return "find_files"
|
|
||||||
|
|
||||||
@property
|
|
||||||
def description(self) -> str:
|
|
||||||
return (
|
|
||||||
"Find files by path fragment, glob, or file type. "
|
|
||||||
"Use this before read_file when you need to locate files, and "
|
|
||||||
"prefer it over shell find/ls for ordinary workspace discovery. "
|
|
||||||
"Returns workspace-relative paths and skips common dependency/build "
|
|
||||||
"directories."
|
|
||||||
)
|
|
||||||
|
|
||||||
@property
|
|
||||||
def read_only(self) -> bool:
|
|
||||||
return True
|
|
||||||
|
|
||||||
@property
|
|
||||||
def parameters(self) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"path": {
|
|
||||||
"type": "string",
|
|
||||||
"description": "Directory or file to search in (default '.')",
|
|
||||||
},
|
|
||||||
"query": {
|
|
||||||
"type": "string",
|
|
||||||
"description": (
|
|
||||||
"Optional case-insensitive path fragment search. "
|
|
||||||
"Whitespace-separated terms must all be present."
|
|
||||||
),
|
|
||||||
},
|
|
||||||
"glob": {
|
|
||||||
"type": "string",
|
|
||||||
"description": "Optional file filter, e.g. '*.py' or 'tests/**/test_*.py'",
|
|
||||||
},
|
|
||||||
"type": {
|
|
||||||
"type": "string",
|
|
||||||
"description": "Optional file type shorthand, e.g. 'py', 'ts', 'md', 'json'",
|
|
||||||
},
|
|
||||||
"include_dirs": {
|
|
||||||
"type": "boolean",
|
|
||||||
"description": "Include matching directories as well as files (default false)",
|
|
||||||
},
|
|
||||||
"sort": {
|
|
||||||
"type": "string",
|
|
||||||
"enum": ["path", "modified"],
|
|
||||||
"description": "Sort by path or most recently modified first (default path)",
|
|
||||||
},
|
|
||||||
"head_limit": {
|
|
||||||
"type": "integer",
|
|
||||||
"description": "Maximum number of paths to return (default 200, 0 for all, max 1000)",
|
|
||||||
"minimum": 0,
|
|
||||||
"maximum": 1000,
|
|
||||||
},
|
|
||||||
"offset": {
|
|
||||||
"type": "integer",
|
|
||||||
"description": "Skip the first N results before applying head_limit",
|
|
||||||
"minimum": 0,
|
|
||||||
"maximum": 100000,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
def _iter_paths(self, root: Path, *, include_dirs: bool) -> Iterable[Path]:
|
|
||||||
if root.is_file():
|
|
||||||
yield root
|
|
||||||
return
|
|
||||||
if include_dirs:
|
|
||||||
yield root
|
|
||||||
for dirpath, dirnames, filenames in os.walk(root):
|
|
||||||
dirnames[:] = sorted(d for d in dirnames if d not in self._IGNORE_DIRS)
|
|
||||||
current = Path(dirpath)
|
|
||||||
if include_dirs and current != root:
|
|
||||||
yield current
|
|
||||||
for filename in sorted(filenames):
|
|
||||||
yield current / filename
|
|
||||||
|
|
||||||
async def execute(
|
|
||||||
self,
|
|
||||||
path: str = ".",
|
|
||||||
query: str | None = None,
|
|
||||||
glob: str | None = None,
|
|
||||||
type: str | None = None,
|
|
||||||
include_dirs: bool = False,
|
|
||||||
sort: str = "path",
|
|
||||||
head_limit: int | None = None,
|
|
||||||
offset: int = 0,
|
|
||||||
**kwargs: Any,
|
|
||||||
) -> str:
|
|
||||||
try:
|
|
||||||
target = self._resolve(path or ".")
|
|
||||||
if not target.exists():
|
|
||||||
return f"Error: Path not found: {path}"
|
|
||||||
if not (target.is_dir() or target.is_file()):
|
|
||||||
return f"Error: Unsupported path: {path}"
|
|
||||||
|
|
||||||
if sort not in {"path", "modified"}:
|
|
||||||
return "Error: sort must be 'path' or 'modified'"
|
|
||||||
|
|
||||||
limit = (
|
|
||||||
_DEFAULT_FILE_HEAD_LIMIT
|
|
||||||
if head_limit is None
|
|
||||||
else None if head_limit == 0 else head_limit
|
|
||||||
)
|
|
||||||
root = target if target.is_dir() else target.parent
|
|
||||||
matches: list[tuple[str, float]] = []
|
|
||||||
|
|
||||||
for candidate in self._iter_paths(target, include_dirs=include_dirs):
|
|
||||||
if candidate.is_dir() and not include_dirs:
|
|
||||||
continue
|
|
||||||
rel_path = candidate.relative_to(root).as_posix()
|
|
||||||
display_path = self._display_path(candidate, root)
|
|
||||||
name = candidate.name
|
|
||||||
|
|
||||||
if glob and not _match_glob(rel_path, name, glob):
|
|
||||||
continue
|
|
||||||
if candidate.is_file() and not _matches_type(name, type):
|
|
||||||
continue
|
|
||||||
if candidate.is_dir() and type:
|
|
||||||
continue
|
|
||||||
if not _matches_query(display_path, query):
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
mtime = candidate.stat().st_mtime
|
|
||||||
except OSError:
|
|
||||||
mtime = 0.0
|
|
||||||
suffix = "/" if candidate.is_dir() else ""
|
|
||||||
matches.append((display_path + suffix, mtime))
|
|
||||||
|
|
||||||
if sort == "modified":
|
|
||||||
matches.sort(key=lambda item: (-item[1], item[0]))
|
|
||||||
else:
|
|
||||||
matches.sort(key=lambda item: item[0])
|
|
||||||
|
|
||||||
paths = [item[0] for item in matches]
|
|
||||||
paged, truncated = _paginate(paths, limit, offset)
|
|
||||||
if not paged:
|
|
||||||
return "No files found"
|
|
||||||
|
|
||||||
result = "\n".join(paged)
|
|
||||||
note = _pagination_note(limit, offset, truncated)
|
|
||||||
if note:
|
|
||||||
result += "\n\n" + note
|
|
||||||
return result
|
|
||||||
except PermissionError as e:
|
|
||||||
return f"Error: {e}"
|
|
||||||
except Exception as e:
|
|
||||||
return f"Error finding files: {e}"
|
|
||||||
|
|
||||||
|
|
||||||
class GrepTool(_SearchTool):
|
class GrepTool(_SearchTool):
|
||||||
"""Search file contents using a regex-like pattern."""
|
"""Search file contents using a regex-like pattern."""
|
||||||
_scopes = {"core", "subagent"}
|
_scopes = {"core", "subagent"}
|
||||||
@@ -292,8 +125,7 @@ class GrepTool(_SearchTool):
|
|||||||
return (
|
return (
|
||||||
"Search file contents with a regex pattern. "
|
"Search file contents with a regex pattern. "
|
||||||
"Default output_mode is files_with_matches (file paths only); "
|
"Default output_mode is files_with_matches (file paths only); "
|
||||||
"use content mode for matching lines with context. Prefer this "
|
"use content mode for matching lines with context. "
|
||||||
"over shell grep for ordinary workspace searches. "
|
|
||||||
"Skips binary and files >2 MB. Supports glob/type filtering."
|
"Skips binary and files >2 MB. Supports glob/type filtering."
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -3,18 +3,16 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import time
|
import time
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import Any
|
||||||
|
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
|
from nanobot.agent.subagent import SubagentStatus
|
||||||
from nanobot.agent.tools.base import Tool
|
from nanobot.agent.tools.base import Tool
|
||||||
from nanobot.agent.tools.context import ContextAware, RequestContext
|
from nanobot.agent.tools.context import ContextAware, RequestContext
|
||||||
from nanobot.agent.tools.runtime_state import RuntimeState
|
from nanobot.agent.tools.runtime_state import RuntimeState
|
||||||
from nanobot.config.schema import Base
|
from nanobot.config.schema import Base
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from nanobot.agent.subagent import SubagentStatus
|
|
||||||
|
|
||||||
|
|
||||||
class MyToolConfig(Base):
|
class MyToolConfig(Base):
|
||||||
"""Self-inspection tool configuration."""
|
"""Self-inspection tool configuration."""
|
||||||
@@ -35,12 +33,6 @@ def _has_real_attr(obj: Any, key: str) -> bool:
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
def _is_subagent_status(value: Any) -> bool:
|
|
||||||
from nanobot.agent.subagent import SubagentStatus
|
|
||||||
|
|
||||||
return isinstance(value, SubagentStatus)
|
|
||||||
|
|
||||||
|
|
||||||
class MyTool(Tool, ContextAware):
|
class MyTool(Tool, ContextAware):
|
||||||
"""Check and set the agent loop's runtime configuration."""
|
"""Check and set the agent loop's runtime configuration."""
|
||||||
|
|
||||||
@@ -76,7 +68,6 @@ class MyTool(Tool, ContextAware):
|
|||||||
"_current_iteration", # updated by runner only
|
"_current_iteration", # updated by runner only
|
||||||
"exec_config", # inspect allowed (e.g. check sandbox), modify blocked
|
"exec_config", # inspect allowed (e.g. check sandbox), modify blocked
|
||||||
"web_config", # inspect allowed (e.g. check enable), modify blocked
|
"web_config", # inspect allowed (e.g. check enable), modify blocked
|
||||||
"workspace_sandbox", # read-only view of workspace enforcement level
|
|
||||||
})
|
})
|
||||||
|
|
||||||
_DENIED_ATTRS = frozenset({
|
_DENIED_ATTRS = frozenset({
|
||||||
@@ -223,7 +214,7 @@ class MyTool(Tool, ContextAware):
|
|||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _format_status(st: "SubagentStatus", indent: str = " ") -> str:
|
def _format_status(st: SubagentStatus, indent: str = " ") -> str:
|
||||||
elapsed = time.monotonic() - st.started_at
|
elapsed = time.monotonic() - st.started_at
|
||||||
tool_summary = ", ".join(
|
tool_summary = ", ".join(
|
||||||
f"{e.get('name', '?')}({e.get('status', '?')})" for e in st.tool_events[-5:]
|
f"{e.get('name', '?')}({e.get('status', '?')})" for e in st.tool_events[-5:]
|
||||||
@@ -241,14 +232,14 @@ class MyTool(Tool, ContextAware):
|
|||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _format_value(val: Any, key: str = "") -> str:
|
def _format_value(val: Any, key: str = "") -> str:
|
||||||
if _is_subagent_status(val):
|
if isinstance(val, SubagentStatus):
|
||||||
header = f"Subagent [{val.task_id}] '{val.label}'"
|
header = f"Subagent [{val.task_id}] '{val.label}'"
|
||||||
detail = MyTool._format_status(val, " ")
|
detail = MyTool._format_status(val, " ")
|
||||||
return f"{header}\n task: {val.task_description}\n{detail}"
|
return f"{header}\n task: {val.task_description}\n{detail}"
|
||||||
# SubagentManager: delegate to its _task_statuses dict
|
# SubagentManager: delegate to its _task_statuses dict
|
||||||
if hasattr(val, "_task_statuses") and isinstance(val._task_statuses, dict):
|
if hasattr(val, "_task_statuses") and isinstance(val._task_statuses, dict):
|
||||||
return MyTool._format_value(val._task_statuses, key)
|
return MyTool._format_value(val._task_statuses, key)
|
||||||
if isinstance(val, dict) and val and _is_subagent_status(next(iter(val.values()))):
|
if isinstance(val, dict) and val and isinstance(next(iter(val.values())), SubagentStatus):
|
||||||
prefix = f"{key}: " if key else ""
|
prefix = f"{key}: " if key else ""
|
||||||
lines = [f"{prefix}{len(val)} subagent(s):"]
|
lines = [f"{prefix}{len(val)} subagent(s):"]
|
||||||
for tid, st in val.items():
|
for tid, st in val.items():
|
||||||
@@ -358,7 +349,7 @@ class MyTool(Tool, ContextAware):
|
|||||||
parts.append(self._format_value(getattr(state, k, None), k))
|
parts.append(self._format_value(getattr(state, k, None), k))
|
||||||
parts.append(self._format_value(state.model_preset, "model_preset"))
|
parts.append(self._format_value(state.model_preset, "model_preset"))
|
||||||
# Other useful top-level keys shown in description
|
# Other useful top-level keys shown in description
|
||||||
for k in ("workspace", "provider_retry_mode", "max_tool_result_chars", "_current_iteration", "web_config", "exec_config", "workspace_sandbox", "subagents"):
|
for k in ("workspace", "provider_retry_mode", "max_tool_result_chars", "_current_iteration", "web_config", "exec_config", "subagents"):
|
||||||
if _has_real_attr(state, k):
|
if _has_real_attr(state, k):
|
||||||
parts.append(self._format_value(getattr(state, k, None), k))
|
parts.append(self._format_value(getattr(state, k, None), k))
|
||||||
# Token usage
|
# Token usage
|
||||||
|
|||||||
+73
-299
@@ -8,7 +8,6 @@ import re
|
|||||||
import shutil
|
import shutil
|
||||||
import sys
|
import sys
|
||||||
from contextlib import suppress
|
from contextlib import suppress
|
||||||
from dataclasses import dataclass
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
@@ -16,27 +15,10 @@ from loguru import logger
|
|||||||
from pydantic import Field
|
from pydantic import Field
|
||||||
|
|
||||||
from nanobot.agent.tools.base import Tool, tool_parameters
|
from nanobot.agent.tools.base import Tool, tool_parameters
|
||||||
from nanobot.agent.tools.context import current_request_session_key
|
|
||||||
from nanobot.agent.tools.exec_session import (
|
|
||||||
DEFAULT_EXEC_SESSION_MANAGER,
|
|
||||||
DEFAULT_MAX_OUTPUT_CHARS,
|
|
||||||
DEFAULT_YIELD_MS,
|
|
||||||
MAX_OUTPUT_CHARS,
|
|
||||||
MAX_YIELD_MS,
|
|
||||||
clamp_session_int,
|
|
||||||
format_session_poll,
|
|
||||||
)
|
|
||||||
from nanobot.agent.tools.sandbox import wrap_command
|
from nanobot.agent.tools.sandbox import wrap_command
|
||||||
from nanobot.agent.tools.schema import (
|
from nanobot.agent.tools.schema import IntegerSchema, StringSchema, tool_parameters_schema
|
||||||
BooleanSchema,
|
|
||||||
IntegerSchema,
|
|
||||||
StringSchema,
|
|
||||||
tool_parameters_schema,
|
|
||||||
)
|
|
||||||
from nanobot.config.paths import get_media_dir
|
from nanobot.config.paths import get_media_dir
|
||||||
from nanobot.config.schema import Base
|
from nanobot.config.schema import Base
|
||||||
from nanobot.security.workspace_access import current_scope_allows_loopback, current_tool_workspace
|
|
||||||
from nanobot.security.workspace_policy import is_path_within
|
|
||||||
|
|
||||||
_IS_WINDOWS = sys.platform == "win32"
|
_IS_WINDOWS = sys.platform == "win32"
|
||||||
|
|
||||||
@@ -54,7 +36,7 @@ _WORKSPACE_BOUNDARY_NOTE = (
|
|||||||
class ExecToolConfig(Base):
|
class ExecToolConfig(Base):
|
||||||
"""Shell exec tool configuration."""
|
"""Shell exec tool configuration."""
|
||||||
enable: bool = True
|
enable: bool = True
|
||||||
timeout: int = Field(default=60, ge=0) # Hard timeout (s); 0 = no limit. Not capped by the per-call max.
|
timeout: int = 60
|
||||||
path_append: str = ""
|
path_append: str = ""
|
||||||
sandbox: str = ""
|
sandbox: str = ""
|
||||||
allowed_env_keys: list[str] = Field(default_factory=list)
|
allowed_env_keys: list[str] = Field(default_factory=list)
|
||||||
@@ -62,22 +44,10 @@ class ExecToolConfig(Base):
|
|||||||
deny_patterns: list[str] = Field(default_factory=list)
|
deny_patterns: list[str] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
|
||||||
class _PreparedCommand:
|
|
||||||
command: str
|
|
||||||
cwd: str
|
|
||||||
env: dict[str, str]
|
|
||||||
timeout: int | None
|
|
||||||
shell_program: str | None
|
|
||||||
login: bool
|
|
||||||
|
|
||||||
|
|
||||||
@tool_parameters(
|
@tool_parameters(
|
||||||
tool_parameters_schema(
|
tool_parameters_schema(
|
||||||
command=StringSchema("The shell command to execute"),
|
command=StringSchema("The shell command to execute"),
|
||||||
cmd=StringSchema("Compatibility alias for command"),
|
|
||||||
working_dir=StringSchema("Optional working directory for the command"),
|
working_dir=StringSchema("Optional working directory for the command"),
|
||||||
workdir=StringSchema("Compatibility alias for working_dir"),
|
|
||||||
timeout=IntegerSchema(
|
timeout=IntegerSchema(
|
||||||
60,
|
60,
|
||||||
description=(
|
description=(
|
||||||
@@ -87,44 +57,7 @@ class _PreparedCommand:
|
|||||||
minimum=1,
|
minimum=1,
|
||||||
maximum=600,
|
maximum=600,
|
||||||
),
|
),
|
||||||
shell=StringSchema(
|
required=["command"],
|
||||||
"Optional shell binary to launch. On Unix, supports sh, bash, or zsh.",
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
login=BooleanSchema(
|
|
||||||
description="Whether to run bash/zsh with login shell semantics (default true).",
|
|
||||||
default=True,
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
yield_time_ms=IntegerSchema(
|
|
||||||
description=(
|
|
||||||
"Optional milliseconds to wait before returning output. "
|
|
||||||
"When set, a still-running command returns a session_id that "
|
|
||||||
"can be polled or written to with write_stdin. Omit this field "
|
|
||||||
"to keep one-shot exec behavior."
|
|
||||||
),
|
|
||||||
minimum=0,
|
|
||||||
maximum=MAX_YIELD_MS,
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
max_output_chars=IntegerSchema(
|
|
||||||
description=(
|
|
||||||
"Maximum output characters to return when yield_time_ms is used "
|
|
||||||
"(default 10000, max 50000)."
|
|
||||||
),
|
|
||||||
minimum=1000,
|
|
||||||
maximum=MAX_OUTPUT_CHARS,
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
max_output_tokens=IntegerSchema(
|
|
||||||
description=(
|
|
||||||
"Compatibility alias for max_output_chars. The current runtime "
|
|
||||||
"uses a character budget."
|
|
||||||
),
|
|
||||||
minimum=1000,
|
|
||||||
maximum=MAX_OUTPUT_CHARS,
|
|
||||||
nullable=True,
|
|
||||||
),
|
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
class ExecTool(Tool):
|
class ExecTool(Tool):
|
||||||
@@ -148,7 +81,6 @@ class ExecTool(Tool):
|
|||||||
working_dir=ctx.workspace,
|
working_dir=ctx.workspace,
|
||||||
timeout=cfg.timeout,
|
timeout=cfg.timeout,
|
||||||
restrict_to_workspace=ctx.config.restrict_to_workspace,
|
restrict_to_workspace=ctx.config.restrict_to_workspace,
|
||||||
webui_allow_local_service_access=ctx.config.webui_allow_local_service_access,
|
|
||||||
sandbox=cfg.sandbox,
|
sandbox=cfg.sandbox,
|
||||||
path_append=cfg.path_append,
|
path_append=cfg.path_append,
|
||||||
allowed_env_keys=cfg.allowed_env_keys,
|
allowed_env_keys=cfg.allowed_env_keys,
|
||||||
@@ -163,12 +95,9 @@ class ExecTool(Tool):
|
|||||||
deny_patterns: list[str] | None = None,
|
deny_patterns: list[str] | None = None,
|
||||||
allow_patterns: list[str] | None = None,
|
allow_patterns: list[str] | None = None,
|
||||||
restrict_to_workspace: bool = False,
|
restrict_to_workspace: bool = False,
|
||||||
webui_allow_local_service_access: bool = True,
|
|
||||||
allow_local_preview_access: bool | None = None,
|
|
||||||
sandbox: str = "",
|
sandbox: str = "",
|
||||||
path_append: str = "",
|
path_append: str = "",
|
||||||
allowed_env_keys: list[str] | None = None,
|
allowed_env_keys: list[str] | None = None,
|
||||||
session_manager: Any | None = None,
|
|
||||||
):
|
):
|
||||||
self.timeout = timeout
|
self.timeout = timeout
|
||||||
self.working_dir = working_dir
|
self.working_dir = working_dir
|
||||||
@@ -194,12 +123,8 @@ class ExecTool(Tool):
|
|||||||
]
|
]
|
||||||
self.allow_patterns = allow_patterns or []
|
self.allow_patterns = allow_patterns or []
|
||||||
self.restrict_to_workspace = restrict_to_workspace
|
self.restrict_to_workspace = restrict_to_workspace
|
||||||
if allow_local_preview_access is not None:
|
|
||||||
webui_allow_local_service_access = allow_local_preview_access
|
|
||||||
self.webui_allow_local_service_access = webui_allow_local_service_access
|
|
||||||
self.path_append = path_append
|
self.path_append = path_append
|
||||||
self.allowed_env_keys = allowed_env_keys or []
|
self.allowed_env_keys = allowed_env_keys or []
|
||||||
self._session_manager = session_manager or DEFAULT_EXEC_SESSION_MANAGER
|
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def name(self) -> str:
|
def name(self) -> str:
|
||||||
@@ -225,15 +150,10 @@ class ExecTool(Tool):
|
|||||||
def description(self) -> str:
|
def description(self) -> str:
|
||||||
return (
|
return (
|
||||||
"Execute a shell command and return its output. "
|
"Execute a shell command and return its output. "
|
||||||
"Use this for tests, builds, package commands, git commands, and "
|
"Prefer read_file/write_file/edit_file over cat/echo/sed, "
|
||||||
"other process execution. Prefer read_file/find_files/grep for "
|
"and grep/glob over shell find/grep. "
|
||||||
"inspection and apply_patch/write_file/edit_file for file changes "
|
|
||||||
"instead of cat, shell find/grep, echo, or sed. "
|
|
||||||
"Use -y or --yes flags to avoid interactive prompts. "
|
"Use -y or --yes flags to avoid interactive prompts. "
|
||||||
"For long-running or interactive commands, pass yield_time_ms; "
|
"Output is truncated at 10 000 chars; timeout defaults to 60s."
|
||||||
"if the command keeps running, exec returns a session_id that can "
|
|
||||||
"be polled or written to with write_stdin. Output is truncated at "
|
|
||||||
"10 000 chars; timeout defaults to 60s."
|
|
||||||
)
|
)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
@@ -241,45 +161,67 @@ class ExecTool(Tool):
|
|||||||
return True
|
return True
|
||||||
|
|
||||||
async def execute(
|
async def execute(
|
||||||
self, command: str | None = None, cmd: str | None = None,
|
self, command: str, working_dir: str | None = None,
|
||||||
working_dir: str | None = None, workdir: str | None = None,
|
timeout: int | None = None, **kwargs: Any,
|
||||||
timeout: int | None = None, shell: str | None = None,
|
|
||||||
login: bool | None = None, yield_time_ms: int | None = None,
|
|
||||||
max_output_chars: int | None = None,
|
|
||||||
max_output_tokens: int | None = None,
|
|
||||||
**kwargs: Any,
|
|
||||||
) -> str:
|
) -> str:
|
||||||
command = command or cmd
|
cwd = working_dir or self.working_dir or os.getcwd()
|
||||||
working_dir = working_dir or workdir
|
|
||||||
if not command:
|
|
||||||
return "Error: Missing command. Provide command or cmd."
|
|
||||||
if max_output_chars is None:
|
|
||||||
max_output_chars = max_output_tokens
|
|
||||||
|
|
||||||
prepared = self._prepare_command(command, working_dir, timeout, shell, login)
|
# Prevent an LLM-supplied working_dir from escaping the configured
|
||||||
if isinstance(prepared, str):
|
# workspace when restrict_to_workspace is enabled (#2826). Without
|
||||||
return prepared
|
# this, a caller can pass working_dir="/etc" and then all absolute
|
||||||
|
# paths under /etc would pass the _guard_command check that anchors
|
||||||
|
# on cwd.
|
||||||
|
if self.restrict_to_workspace and self.working_dir:
|
||||||
|
try:
|
||||||
|
requested = Path(cwd).expanduser().resolve()
|
||||||
|
workspace_root = Path(self.working_dir).expanduser().resolve()
|
||||||
|
except Exception:
|
||||||
|
return (
|
||||||
|
"Error: working_dir could not be resolved"
|
||||||
|
+ _WORKSPACE_BOUNDARY_NOTE
|
||||||
|
)
|
||||||
|
if requested != workspace_root and workspace_root not in requested.parents:
|
||||||
|
return (
|
||||||
|
"Error: working_dir is outside the configured workspace"
|
||||||
|
+ _WORKSPACE_BOUNDARY_NOTE
|
||||||
|
)
|
||||||
|
|
||||||
if yield_time_ms is not None:
|
guard_error = self._guard_command(command, cwd)
|
||||||
return await self._execute_session(prepared, yield_time_ms, max_output_chars)
|
if guard_error:
|
||||||
|
return guard_error
|
||||||
|
|
||||||
|
if self.sandbox:
|
||||||
|
if _IS_WINDOWS:
|
||||||
|
logger.warning(
|
||||||
|
"Sandbox '{}' is not supported on Windows; running unsandboxed",
|
||||||
|
self.sandbox,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
workspace = self.working_dir or cwd
|
||||||
|
command = wrap_command(self.sandbox, command, workspace, cwd)
|
||||||
|
cwd = str(Path(workspace).resolve())
|
||||||
|
|
||||||
|
effective_timeout = min(timeout or self.timeout, self._MAX_TIMEOUT)
|
||||||
|
env = self._build_env()
|
||||||
|
|
||||||
|
if self.path_append:
|
||||||
|
if _IS_WINDOWS:
|
||||||
|
env["PATH"] = env.get("PATH", "") + os.pathsep + self.path_append
|
||||||
|
else:
|
||||||
|
env["NANOBOT_PATH_APPEND"] = self.path_append
|
||||||
|
command = f'export PATH="$PATH{os.pathsep}$NANOBOT_PATH_APPEND"; {command}'
|
||||||
|
|
||||||
try:
|
try:
|
||||||
process = await self._spawn(
|
process = await self._spawn(command, cwd, env)
|
||||||
prepared.command,
|
|
||||||
prepared.cwd,
|
|
||||||
prepared.env,
|
|
||||||
prepared.shell_program,
|
|
||||||
prepared.login,
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
stdout, stderr = await asyncio.wait_for(
|
stdout, stderr = await asyncio.wait_for(
|
||||||
process.communicate(),
|
process.communicate(),
|
||||||
timeout=prepared.timeout,
|
timeout=effective_timeout,
|
||||||
)
|
)
|
||||||
except asyncio.TimeoutError:
|
except asyncio.TimeoutError:
|
||||||
await self._kill_process(process)
|
await self._kill_process(process)
|
||||||
return f"Error: Command timed out after {prepared.timeout} seconds"
|
return f"Error: Command timed out after {effective_timeout} seconds"
|
||||||
except asyncio.CancelledError:
|
except asyncio.CancelledError:
|
||||||
await self._kill_process(process)
|
await self._kill_process(process)
|
||||||
raise
|
raise
|
||||||
@@ -298,7 +240,7 @@ class ExecTool(Tool):
|
|||||||
|
|
||||||
result = "\n".join(output_parts) if output_parts else "(no output)"
|
result = "\n".join(output_parts) if output_parts else "(no output)"
|
||||||
|
|
||||||
max_len = clamp_session_int(max_output_chars, self._MAX_OUTPUT, 1000, MAX_OUTPUT_CHARS)
|
max_len = self._MAX_OUTPUT
|
||||||
if len(result) > max_len:
|
if len(result) > max_len:
|
||||||
half = max_len // 2
|
half = max_len // 2
|
||||||
result = (
|
result = (
|
||||||
@@ -312,192 +254,34 @@ class ExecTool(Tool):
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
return f"Error executing command: {str(e)}"
|
return f"Error executing command: {str(e)}"
|
||||||
|
|
||||||
async def _execute_session(
|
|
||||||
self,
|
|
||||||
prepared: _PreparedCommand,
|
|
||||||
yield_time_ms: int | None,
|
|
||||||
max_output_chars: int | None,
|
|
||||||
) -> str:
|
|
||||||
try:
|
|
||||||
session_id, poll = await self._session_manager.start(
|
|
||||||
command=prepared.command,
|
|
||||||
cwd=prepared.cwd,
|
|
||||||
env=prepared.env,
|
|
||||||
timeout=prepared.timeout,
|
|
||||||
shell_program=prepared.shell_program,
|
|
||||||
login=prepared.login,
|
|
||||||
yield_time_ms=clamp_session_int(yield_time_ms, DEFAULT_YIELD_MS, 0, MAX_YIELD_MS),
|
|
||||||
owner_session_key=current_request_session_key(),
|
|
||||||
max_output_chars=clamp_session_int(
|
|
||||||
max_output_chars,
|
|
||||||
DEFAULT_MAX_OUTPUT_CHARS,
|
|
||||||
1000,
|
|
||||||
MAX_OUTPUT_CHARS,
|
|
||||||
),
|
|
||||||
)
|
|
||||||
return format_session_poll(session_id, poll)
|
|
||||||
except Exception as exc:
|
|
||||||
return f"Error executing command: {exc}"
|
|
||||||
|
|
||||||
def _resolve_timeout(self, timeout: int | None) -> int | None:
|
|
||||||
"""Resolve the effective hard timeout in seconds (None = no limit).
|
|
||||||
|
|
||||||
A per-call timeout supplied by the model stays capped at _MAX_TIMEOUT so
|
|
||||||
the LLM cannot request unbounded execution. The config-level default
|
|
||||||
(self.timeout) may exceed that cap, and 0 disables the limit entirely
|
|
||||||
for trusted long-running tasks (#3595).
|
|
||||||
"""
|
|
||||||
if timeout:
|
|
||||||
return min(timeout, self._MAX_TIMEOUT)
|
|
||||||
if self.timeout and self.timeout > 0:
|
|
||||||
return self.timeout
|
|
||||||
return None
|
|
||||||
|
|
||||||
def _prepare_command(
|
|
||||||
self,
|
|
||||||
command: str,
|
|
||||||
working_dir: str | None = None,
|
|
||||||
timeout: int | None = None,
|
|
||||||
shell: str | None = None,
|
|
||||||
login: bool | None = None,
|
|
||||||
) -> _PreparedCommand | str:
|
|
||||||
access = current_tool_workspace(
|
|
||||||
self.working_dir,
|
|
||||||
restrict_to_workspace=self.restrict_to_workspace,
|
|
||||||
sandbox_restricts_workspace=bool(self.sandbox),
|
|
||||||
)
|
|
||||||
workspace_root = str(access.project_path) if access.project_path is not None else self.working_dir
|
|
||||||
cwd = working_dir or workspace_root or os.getcwd()
|
|
||||||
|
|
||||||
# Prevent an LLM-supplied working_dir from escaping the configured
|
|
||||||
# workspace when restrict_to_workspace is enabled (#2826). Without
|
|
||||||
# this, a caller can pass working_dir="/etc" and then all absolute
|
|
||||||
# paths under /etc would pass the _guard_command check that anchors
|
|
||||||
# on cwd.
|
|
||||||
if access.restrict_to_workspace and workspace_root:
|
|
||||||
try:
|
|
||||||
requested = Path(cwd).expanduser().resolve()
|
|
||||||
resolved_root = Path(workspace_root).expanduser().resolve()
|
|
||||||
except Exception:
|
|
||||||
return (
|
|
||||||
"Error: working_dir could not be resolved"
|
|
||||||
+ _WORKSPACE_BOUNDARY_NOTE
|
|
||||||
)
|
|
||||||
if not is_path_within(requested, resolved_root):
|
|
||||||
return (
|
|
||||||
"Error: working_dir is outside the configured workspace"
|
|
||||||
+ _WORKSPACE_BOUNDARY_NOTE
|
|
||||||
)
|
|
||||||
|
|
||||||
guard_error = self._guard_command(
|
|
||||||
command,
|
|
||||||
cwd,
|
|
||||||
restrict_to_workspace=access.restrict_to_workspace,
|
|
||||||
)
|
|
||||||
if guard_error:
|
|
||||||
return guard_error
|
|
||||||
|
|
||||||
if self.sandbox:
|
|
||||||
if _IS_WINDOWS:
|
|
||||||
logger.warning(
|
|
||||||
"Sandbox '{}' is not supported on Windows; running unsandboxed",
|
|
||||||
self.sandbox,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
workspace = workspace_root or cwd
|
|
||||||
command = wrap_command(self.sandbox, command, workspace, cwd)
|
|
||||||
cwd = str(Path(workspace).resolve())
|
|
||||||
|
|
||||||
effective_timeout = self._resolve_timeout(timeout)
|
|
||||||
env = self._build_env()
|
|
||||||
|
|
||||||
if self.path_append:
|
|
||||||
if _IS_WINDOWS:
|
|
||||||
env["PATH"] = env.get("PATH", "") + os.pathsep + self.path_append
|
|
||||||
else:
|
|
||||||
env["NANOBOT_PATH_APPEND"] = self.path_append
|
|
||||||
command = f'export PATH="$PATH{os.pathsep}$NANOBOT_PATH_APPEND"; {command}'
|
|
||||||
|
|
||||||
shell_program, shell_error = self._resolve_shell(shell)
|
|
||||||
if shell_error:
|
|
||||||
return shell_error
|
|
||||||
|
|
||||||
return _PreparedCommand(
|
|
||||||
command=command,
|
|
||||||
cwd=cwd,
|
|
||||||
env=env,
|
|
||||||
timeout=effective_timeout,
|
|
||||||
shell_program=shell_program,
|
|
||||||
login=True if login is None else login,
|
|
||||||
)
|
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
async def _spawn(
|
async def _spawn(
|
||||||
command: str, cwd: str, env: dict[str, str],
|
command: str, cwd: str, env: dict[str, str],
|
||||||
shell_program: str | None = None,
|
|
||||||
login: bool = True,
|
|
||||||
*,
|
|
||||||
stdin: int = asyncio.subprocess.DEVNULL,
|
|
||||||
) -> asyncio.subprocess.Process:
|
) -> asyncio.subprocess.Process:
|
||||||
"""Launch *command* in a platform-appropriate shell."""
|
"""Launch *command* in a platform-appropriate shell."""
|
||||||
if _IS_WINDOWS:
|
if _IS_WINDOWS:
|
||||||
if "\n" in command:
|
# create_subprocess_exec re-quotes args via list2cmdline, which
|
||||||
return await asyncio.create_subprocess_exec(
|
# breaks commands containing paths with spaces (e.g. "D:\Program
|
||||||
"powershell", "-NoProfile", "-Command", command,
|
# Files\python.exe" "script.py"). create_subprocess_shell passes
|
||||||
stdin=stdin,
|
# the raw command string to COMSPEC without re-quoting.
|
||||||
stdout=asyncio.subprocess.PIPE,
|
|
||||||
stderr=asyncio.subprocess.PIPE,
|
|
||||||
cwd=cwd,
|
|
||||||
env=env,
|
|
||||||
)
|
|
||||||
return await asyncio.create_subprocess_shell(
|
return await asyncio.create_subprocess_shell(
|
||||||
command,
|
command,
|
||||||
stdin=stdin,
|
stdin=asyncio.subprocess.DEVNULL,
|
||||||
stdout=asyncio.subprocess.PIPE,
|
stdout=asyncio.subprocess.PIPE,
|
||||||
stderr=asyncio.subprocess.PIPE,
|
stderr=asyncio.subprocess.PIPE,
|
||||||
cwd=cwd,
|
cwd=cwd,
|
||||||
env=env,
|
env=env,
|
||||||
)
|
)
|
||||||
shell_program = shell_program or shutil.which("bash") or "/bin/bash"
|
bash = shutil.which("bash") or "/bin/bash"
|
||||||
args = [shell_program]
|
|
||||||
shell_name = Path(shell_program).name.lower()
|
|
||||||
if login and shell_name in {"bash", "bash.exe", "zsh", "zsh.exe"}:
|
|
||||||
args.append("-l")
|
|
||||||
args.extend(["-c", command])
|
|
||||||
return await asyncio.create_subprocess_exec(
|
return await asyncio.create_subprocess_exec(
|
||||||
*args,
|
bash, "-l", "-c", command,
|
||||||
stdin=stdin,
|
stdin=asyncio.subprocess.DEVNULL,
|
||||||
stdout=asyncio.subprocess.PIPE,
|
stdout=asyncio.subprocess.PIPE,
|
||||||
stderr=asyncio.subprocess.PIPE,
|
stderr=asyncio.subprocess.PIPE,
|
||||||
cwd=cwd,
|
cwd=cwd,
|
||||||
env=env,
|
env=env,
|
||||||
)
|
)
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _resolve_shell(shell: str | None) -> tuple[str | None, str | None]:
|
|
||||||
if not shell:
|
|
||||||
return None, None
|
|
||||||
if _IS_WINDOWS:
|
|
||||||
return None, "Error: shell parameter is not supported on Windows"
|
|
||||||
if "\0" in shell or "\n" in shell or "\r" in shell:
|
|
||||||
return None, "Error: shell contains invalid characters"
|
|
||||||
allowed = {"sh", "bash", "zsh"}
|
|
||||||
path = Path(shell).expanduser()
|
|
||||||
if path.is_absolute():
|
|
||||||
if path.name not in allowed:
|
|
||||||
return None, f"Error: unsupported shell {shell!r}. Allowed: bash, sh, zsh"
|
|
||||||
if not path.is_file() or not os.access(path, os.X_OK):
|
|
||||||
return None, f"Error: shell is not executable: {shell}"
|
|
||||||
return str(path), None
|
|
||||||
if "/" in shell or "\\" in shell:
|
|
||||||
return None, "Error: shell must be a shell name or absolute path"
|
|
||||||
if shell not in allowed:
|
|
||||||
return None, f"Error: unsupported shell {shell!r}. Allowed: bash, sh, zsh"
|
|
||||||
resolved = shutil.which(shell)
|
|
||||||
if not resolved:
|
|
||||||
return None, f"Error: shell not found: {shell}"
|
|
||||||
return resolved, None
|
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
async def _kill_process(process: asyncio.subprocess.Process) -> None:
|
async def _kill_process(process: asyncio.subprocess.Process) -> None:
|
||||||
"""Kill a subprocess and reap it to prevent zombies."""
|
"""Kill a subprocess and reap it to prevent zombies."""
|
||||||
@@ -560,13 +344,7 @@ class ExecTool(Tool):
|
|||||||
env[key] = val
|
env[key] = val
|
||||||
return env
|
return env
|
||||||
|
|
||||||
def _guard_command(
|
def _guard_command(self, command: str, cwd: str) -> str | None:
|
||||||
self,
|
|
||||||
command: str,
|
|
||||||
cwd: str,
|
|
||||||
*,
|
|
||||||
restrict_to_workspace: bool | None = None,
|
|
||||||
) -> str | None:
|
|
||||||
"""Best-effort safety guard for potentially destructive commands."""
|
"""Best-effort safety guard for potentially destructive commands."""
|
||||||
cmd = command.strip()
|
cmd = command.strip()
|
||||||
lower = cmd.lower()
|
lower = cmd.lower()
|
||||||
@@ -586,17 +364,11 @@ class ExecTool(Tool):
|
|||||||
return "Error: Command blocked by allowlist filter (not in allowlist)"
|
return "Error: Command blocked by allowlist filter (not in allowlist)"
|
||||||
|
|
||||||
from nanobot.security.network import contains_internal_url
|
from nanobot.security.network import contains_internal_url
|
||||||
if contains_internal_url(
|
if contains_internal_url(cmd):
|
||||||
cmd,
|
|
||||||
allow_loopback=current_scope_allows_loopback(
|
|
||||||
enabled=self.webui_allow_local_service_access,
|
|
||||||
),
|
|
||||||
):
|
|
||||||
# The runner turns this marker into a non-retryable security hint.
|
# The runner turns this marker into a non-retryable security hint.
|
||||||
return "Error: Command blocked by safety guard (internal/private URL detected)"
|
return "Error: Command blocked by safety guard (internal/private URL detected)"
|
||||||
|
|
||||||
should_restrict = self.restrict_to_workspace if restrict_to_workspace is None else restrict_to_workspace
|
if self.restrict_to_workspace:
|
||||||
if should_restrict:
|
|
||||||
if "..\\" in cmd or "../" in cmd:
|
if "..\\" in cmd or "../" in cmd:
|
||||||
return (
|
return (
|
||||||
"Error: Command blocked by safety guard (path traversal detected)"
|
"Error: Command blocked by safety guard (path traversal detected)"
|
||||||
@@ -621,9 +393,11 @@ class ExecTool(Tool):
|
|||||||
continue
|
continue
|
||||||
|
|
||||||
media_path = get_media_dir().resolve()
|
media_path = get_media_dir().resolve()
|
||||||
if p.is_absolute() and not (
|
if (p.is_absolute()
|
||||||
is_path_within(p, cwd_path)
|
and cwd_path not in p.parents
|
||||||
or is_path_within(p, media_path)
|
and p != cwd_path
|
||||||
|
and media_path not in p.parents
|
||||||
|
and p != media_path
|
||||||
):
|
):
|
||||||
return (
|
return (
|
||||||
"Error: Command blocked by safety guard (path outside working dir)"
|
"Error: Command blocked by safety guard (path outside working dir)"
|
||||||
@@ -644,7 +418,7 @@ class ExecTool(Tool):
|
|||||||
# Windows: match drive-root paths like `C:\` as well as `C:\path\to\file`, and UNC paths like `\\server\share`
|
# Windows: match drive-root paths like `C:\` as well as `C:\path\to\file`, and UNC paths like `\\server\share`
|
||||||
# NOTE: `*` is required so `C:\` (nothing after the slash) is still extracted.
|
# NOTE: `*` is required so `C:\` (nothing after the slash) is still extracted.
|
||||||
win_paths = re.findall(
|
win_paths = re.findall(
|
||||||
r"(?<![A-Za-z])(?:[A-Za-z]:[^\s\"'|><;]*|\\\\[^\s\"'|><;]+(?:\\[^\s\"'|><;]+)*)",
|
r"(?:[A-Za-z]:[^\s\"'|><;]*|\\\\[^\s\"'|><;]+(?:\\[^\s\"'|><;]+)*)",
|
||||||
command
|
command
|
||||||
)
|
)
|
||||||
posix_paths = re.findall(r"(?:^|[\s|>'\"])(/[^\s\"'>;|<]+)", command) # POSIX: /absolute only
|
posix_paths = re.findall(r"(?:^|[\s|>'\"])(/[^\s\"'>;|<]+)", command) # POSIX: /absolute only
|
||||||
|
|||||||
@@ -7,8 +7,7 @@ from typing import TYPE_CHECKING, Any
|
|||||||
|
|
||||||
from nanobot.agent.tools.base import Tool, tool_parameters
|
from nanobot.agent.tools.base import Tool, tool_parameters
|
||||||
from nanobot.agent.tools.context import ContextAware, RequestContext
|
from nanobot.agent.tools.context import ContextAware, RequestContext
|
||||||
from nanobot.agent.tools.schema import NumberSchema, StringSchema, tool_parameters_schema
|
from nanobot.agent.tools.schema import StringSchema, tool_parameters_schema
|
||||||
from nanobot.security.workspace_access import current_workspace_scope
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from nanobot.agent.subagent import SubagentManager
|
from nanobot.agent.subagent import SubagentManager
|
||||||
@@ -18,15 +17,6 @@ if TYPE_CHECKING:
|
|||||||
tool_parameters_schema(
|
tool_parameters_schema(
|
||||||
task=StringSchema("The task for the subagent to complete"),
|
task=StringSchema("The task for the subagent to complete"),
|
||||||
label=StringSchema("Optional short label for the task (for display)"),
|
label=StringSchema("Optional short label for the task (for display)"),
|
||||||
temperature=NumberSchema(
|
|
||||||
description=(
|
|
||||||
"Optional sampling temperature for the subagent "
|
|
||||||
"(0.0 = deterministic, higher = more creative). "
|
|
||||||
"Defaults to the provider's configured temperature."
|
|
||||||
),
|
|
||||||
minimum=0.0,
|
|
||||||
maximum=2.0,
|
|
||||||
),
|
|
||||||
required=["task"],
|
required=["task"],
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
@@ -68,13 +58,7 @@ class SpawnTool(Tool, ContextAware):
|
|||||||
"and use a dedicated subdirectory when helpful."
|
"and use a dedicated subdirectory when helpful."
|
||||||
)
|
)
|
||||||
|
|
||||||
async def execute(
|
async def execute(self, task: str, label: str | None = None, **kwargs: Any) -> str:
|
||||||
self,
|
|
||||||
task: str,
|
|
||||||
label: str | None = None,
|
|
||||||
temperature: float | None = None,
|
|
||||||
**kwargs: Any,
|
|
||||||
) -> str:
|
|
||||||
"""Spawn a subagent to execute the given task."""
|
"""Spawn a subagent to execute the given task."""
|
||||||
running = self._manager.get_running_count()
|
running = self._manager.get_running_count()
|
||||||
limit = self._manager.max_concurrent_subagents
|
limit = self._manager.max_concurrent_subagents
|
||||||
@@ -91,6 +75,4 @@ class SpawnTool(Tool, ContextAware):
|
|||||||
origin_chat_id=self._origin_chat_id.get(),
|
origin_chat_id=self._origin_chat_id.get(),
|
||||||
session_key=self._session_key.get(),
|
session_key=self._session_key.get(),
|
||||||
origin_message_id=self._origin_message_id.get(),
|
origin_message_id=self._origin_message_id.get(),
|
||||||
temperature=temperature,
|
|
||||||
workspace_scope=current_workspace_scope(),
|
|
||||||
)
|
)
|
||||||
|
|||||||
+24
-104
@@ -8,7 +8,7 @@ import json
|
|||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
from typing import Any, Callable
|
from typing import Any, Callable
|
||||||
from urllib.parse import quote, urljoin, urlparse
|
from urllib.parse import quote, urlparse
|
||||||
|
|
||||||
import httpx
|
import httpx
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
@@ -78,82 +78,9 @@ def _validate_url(url: str) -> tuple[bool, str]:
|
|||||||
def _validate_url_safe(url: str) -> tuple[bool, str]:
|
def _validate_url_safe(url: str) -> tuple[bool, str]:
|
||||||
"""Validate URL with SSRF protection: scheme, domain, and resolved IP check."""
|
"""Validate URL with SSRF protection: scheme, domain, and resolved IP check."""
|
||||||
from nanobot.security.network import validate_url_target
|
from nanobot.security.network import validate_url_target
|
||||||
|
|
||||||
return validate_url_target(url)
|
return validate_url_target(url)
|
||||||
|
|
||||||
|
|
||||||
async def _get_with_safe_redirects(
|
|
||||||
client: httpx.AsyncClient,
|
|
||||||
url: str,
|
|
||||||
headers: dict[str, str] | None = None,
|
|
||||||
) -> tuple[httpx.Response | None, str | None]:
|
|
||||||
"""GET a URL while validating every redirect target before requesting it."""
|
|
||||||
current_url = url
|
|
||||||
for _ in range(MAX_REDIRECTS + 1):
|
|
||||||
is_valid, error_msg = _validate_url_safe(current_url)
|
|
||||||
if not is_valid:
|
|
||||||
return None, f"Redirect blocked: {error_msg}"
|
|
||||||
|
|
||||||
response = await client.get(current_url, headers=headers, follow_redirects=False)
|
|
||||||
is_redirect = 300 <= response.status_code < 400
|
|
||||||
if not is_redirect:
|
|
||||||
return response, None
|
|
||||||
|
|
||||||
location = response.headers.get("location")
|
|
||||||
if not location:
|
|
||||||
return response, None
|
|
||||||
|
|
||||||
next_url = urljoin(str(response.url), location)
|
|
||||||
is_valid, error_msg = _validate_url_safe(next_url)
|
|
||||||
if not is_valid:
|
|
||||||
await response.aclose()
|
|
||||||
return None, f"Redirect blocked: {error_msg}"
|
|
||||||
|
|
||||||
await response.aclose()
|
|
||||||
current_url = next_url
|
|
||||||
|
|
||||||
return None, f"Too many redirects: exceeded limit of {MAX_REDIRECTS}"
|
|
||||||
|
|
||||||
|
|
||||||
async def _stream_with_safe_redirects(
|
|
||||||
client: httpx.AsyncClient,
|
|
||||||
url: str,
|
|
||||||
headers: dict[str, str] | None = None,
|
|
||||||
) -> tuple[httpx.Response | None, Any | None, str | None]:
|
|
||||||
"""Open a streamed response while validating every redirect target first."""
|
|
||||||
current_url = url
|
|
||||||
for _ in range(MAX_REDIRECTS + 1):
|
|
||||||
is_valid, error_msg = _validate_url_safe(current_url)
|
|
||||||
if not is_valid:
|
|
||||||
return None, None, f"Redirect blocked: {error_msg}"
|
|
||||||
|
|
||||||
stream = client.stream(
|
|
||||||
"GET",
|
|
||||||
current_url,
|
|
||||||
headers=headers,
|
|
||||||
follow_redirects=False,
|
|
||||||
)
|
|
||||||
response = await stream.__aenter__()
|
|
||||||
is_redirect = 300 <= response.status_code < 400
|
|
||||||
if not is_redirect:
|
|
||||||
return response, stream, None
|
|
||||||
|
|
||||||
location = response.headers.get("location")
|
|
||||||
if not location:
|
|
||||||
return response, stream, None
|
|
||||||
|
|
||||||
next_url = urljoin(str(response.url), location)
|
|
||||||
is_valid, error_msg = _validate_url_safe(next_url)
|
|
||||||
if not is_valid:
|
|
||||||
await stream.__aexit__(None, None, None)
|
|
||||||
return None, None, f"Redirect blocked: {error_msg}"
|
|
||||||
|
|
||||||
await stream.__aexit__(None, None, None)
|
|
||||||
current_url = next_url
|
|
||||||
|
|
||||||
return None, None, f"Too many redirects: exceeded limit of {MAX_REDIRECTS}"
|
|
||||||
|
|
||||||
|
|
||||||
def _format_results(query: str, items: list[dict[str, Any]], n: int) -> str:
|
def _format_results(query: str, items: list[dict[str, Any]], n: int) -> str:
|
||||||
"""Format provider results into shared plaintext output."""
|
"""Format provider results into shared plaintext output."""
|
||||||
if not items:
|
if not items:
|
||||||
@@ -455,16 +382,17 @@ class WebSearchTool(Tool):
|
|||||||
return await self._search_duckduckgo(query, n)
|
return await self._search_duckduckgo(query, n)
|
||||||
try:
|
try:
|
||||||
async with httpx.AsyncClient(proxy=self.proxy) as client:
|
async with httpx.AsyncClient(proxy=self.proxy) as client:
|
||||||
r = await client.post(
|
r = await client.get(
|
||||||
"https://kagi.com/api/v1/search",
|
"https://kagi.com/api/v0/search",
|
||||||
json={"query": query, "limit": n},
|
params={"q": query, "limit": n},
|
||||||
headers={"Authorization": f"Bearer {api_key}", "User-Agent": self.user_agent},
|
headers={"Authorization": f"Bot {api_key}", "User-Agent": self.user_agent},
|
||||||
timeout=10.0,
|
timeout=10.0,
|
||||||
)
|
)
|
||||||
r.raise_for_status()
|
r.raise_for_status()
|
||||||
|
# t=0 items are search results; other values are related searches, etc.
|
||||||
items = [
|
items = [
|
||||||
{"title": d.get("title", ""), "url": d.get("url", ""), "content": d.get("snippet", "")}
|
{"title": d.get("title", ""), "url": d.get("url", ""), "content": d.get("snippet", "")}
|
||||||
for d in r.json().get("data", {}).get("search", [])
|
for d in r.json().get("data", []) if d.get("t") == 0
|
||||||
]
|
]
|
||||||
return _format_results(query, items, n)
|
return _format_results(query, items, n)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -560,26 +488,19 @@ class WebFetchTool(Tool):
|
|||||||
|
|
||||||
# Detect and fetch images directly to avoid Jina's textual image captioning
|
# Detect and fetch images directly to avoid Jina's textual image captioning
|
||||||
try:
|
try:
|
||||||
async with httpx.AsyncClient(proxy=self.proxy, timeout=15.0) as client:
|
async with httpx.AsyncClient(proxy=self.proxy, follow_redirects=True, max_redirects=MAX_REDIRECTS, timeout=15.0) as client:
|
||||||
r, stream, redirect_error = await _stream_with_safe_redirects(
|
async with client.stream("GET", url, headers={"User-Agent": self.user_agent}) as r:
|
||||||
client,
|
from nanobot.security.network import validate_resolved_url
|
||||||
url,
|
|
||||||
headers={"User-Agent": self.user_agent},
|
redir_ok, redir_err = validate_resolved_url(str(r.url))
|
||||||
)
|
if not redir_ok:
|
||||||
if redirect_error:
|
return json.dumps({"error": f"Redirect blocked: {redir_err}", "url": url}, ensure_ascii=False)
|
||||||
return json.dumps({"error": redirect_error, "url": url}, ensure_ascii=False)
|
|
||||||
if r is None:
|
|
||||||
return json.dumps({"error": "Fetch failed", "url": url}, ensure_ascii=False)
|
|
||||||
|
|
||||||
try:
|
|
||||||
ctype = r.headers.get("content-type", "")
|
ctype = r.headers.get("content-type", "")
|
||||||
if ctype.startswith("image/"):
|
if ctype.startswith("image/"):
|
||||||
r.raise_for_status()
|
r.raise_for_status()
|
||||||
raw = await r.aread()
|
raw = await r.aread()
|
||||||
return build_image_content_blocks(raw, ctype, url, f"(Image fetched from: {url})")
|
return build_image_content_blocks(raw, ctype, url, f"(Image fetched from: {url})")
|
||||||
finally:
|
|
||||||
if stream is not None:
|
|
||||||
await stream.__aexit__(None, None, None)
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.debug("Pre-fetch image detection failed for {}: {}", url, e)
|
logger.debug("Pre-fetch image detection failed for {}: {}", url, e)
|
||||||
|
|
||||||
@@ -628,22 +549,23 @@ class WebFetchTool(Tool):
|
|||||||
|
|
||||||
async def _fetch_readability(self, url: str, extract_mode: str, max_chars: int) -> Any:
|
async def _fetch_readability(self, url: str, extract_mode: str, max_chars: int) -> Any:
|
||||||
"""Local fallback using readability-lxml."""
|
"""Local fallback using readability-lxml."""
|
||||||
|
from readability import Document
|
||||||
|
|
||||||
try:
|
try:
|
||||||
async with httpx.AsyncClient(
|
async with httpx.AsyncClient(
|
||||||
|
follow_redirects=True,
|
||||||
|
max_redirects=MAX_REDIRECTS,
|
||||||
timeout=30.0,
|
timeout=30.0,
|
||||||
proxy=self.proxy,
|
proxy=self.proxy,
|
||||||
) as client:
|
) as client:
|
||||||
r, redirect_error = await _get_with_safe_redirects(
|
r = await client.get(url, headers={"User-Agent": self.user_agent})
|
||||||
client,
|
|
||||||
url,
|
|
||||||
headers={"User-Agent": self.user_agent},
|
|
||||||
)
|
|
||||||
if redirect_error:
|
|
||||||
return json.dumps({"error": redirect_error, "url": url}, ensure_ascii=False)
|
|
||||||
if r is None:
|
|
||||||
return json.dumps({"error": "Fetch failed", "url": url}, ensure_ascii=False)
|
|
||||||
r.raise_for_status()
|
r.raise_for_status()
|
||||||
|
|
||||||
|
from nanobot.security.network import validate_resolved_url
|
||||||
|
redir_ok, redir_err = validate_resolved_url(str(r.url))
|
||||||
|
if not redir_ok:
|
||||||
|
return json.dumps({"error": f"Redirect blocked: {redir_err}", "url": url}, ensure_ascii=False)
|
||||||
|
|
||||||
ctype = r.headers.get("content-type", "")
|
ctype = r.headers.get("content-type", "")
|
||||||
if ctype.startswith("image/"):
|
if ctype.startswith("image/"):
|
||||||
return build_image_content_blocks(r.content, ctype, url, f"(Image fetched from: {url})")
|
return build_image_content_blocks(r.content, ctype, url, f"(Image fetched from: {url})")
|
||||||
@@ -651,8 +573,6 @@ class WebFetchTool(Tool):
|
|||||||
if "application/json" in ctype:
|
if "application/json" in ctype:
|
||||||
text, extractor = json.dumps(r.json(), indent=2, ensure_ascii=False), "json"
|
text, extractor = json.dumps(r.json(), indent=2, ensure_ascii=False), "json"
|
||||||
elif "text/html" in ctype or r.text[:256].lower().startswith(("<!doctype", "<html")):
|
elif "text/html" in ctype or r.text[:256].lower().startswith(("<!doctype", "<html")):
|
||||||
from readability import Document
|
|
||||||
|
|
||||||
doc = Document(r.text)
|
doc = Document(r.text)
|
||||||
content = self._to_markdown(doc.summary()) if extract_mode == "markdown" else _strip_tags(doc.summary())
|
content = self._to_markdown(doc.summary()) if extract_mode == "markdown" else _strip_tags(doc.summary())
|
||||||
text = f"# {doc.title()}\n\n{content}" if doc.title() else content
|
text = f"# {doc.title()}\n\n{content}" if doc.title() else content
|
||||||
|
|||||||
@@ -1,5 +0,0 @@
|
|||||||
"""Shared app protocol helpers."""
|
|
||||||
|
|
||||||
from nanobot.apps.protocol import APP_PROTOCOL_SCHEMA, app_manifest
|
|
||||||
|
|
||||||
__all__ = ["APP_PROTOCOL_SCHEMA", "app_manifest"]
|
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
"""CLI app adapter for the unified Apps domain."""
|
|
||||||
|
|
||||||
from nanobot.apps.cli.service import (
|
|
||||||
CliAppError,
|
|
||||||
CliAppManager,
|
|
||||||
CliAppsRuntimeConfig,
|
|
||||||
)
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"CliAppError",
|
|
||||||
"CliAppManager",
|
|
||||||
"CliAppsRuntimeConfig",
|
|
||||||
]
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,62 +0,0 @@
|
|||||||
"""CLI Apps helpers shared by the agent loop and settings surfaces."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Mapping
|
|
||||||
|
|
||||||
|
|
||||||
def session_extra(metadata: Mapping[str, Any] | None) -> dict[str, Any]:
|
|
||||||
"""Return persisted session kwargs for CLI app attachments."""
|
|
||||||
cli_apps = metadata.get("cli_apps") if isinstance(metadata, Mapping) else None
|
|
||||||
return {"cli_apps": cli_apps} if isinstance(cli_apps, list) and cli_apps else {}
|
|
||||||
|
|
||||||
|
|
||||||
def runtime_lines(message: Any, workspace: Path, *, skip: bool = False) -> list[str]:
|
|
||||||
"""Return model-visible CLI app annotations for the current turn."""
|
|
||||||
if skip:
|
|
||||||
return []
|
|
||||||
text = message.content if isinstance(getattr(message, "content", None), str) else ""
|
|
||||||
metadata = message.metadata if isinstance(getattr(message, "metadata", None), Mapping) else None
|
|
||||||
return _cli_app_runtime_lines(text, metadata, workspace)
|
|
||||||
|
|
||||||
|
|
||||||
def _cli_app_runtime_lines(
|
|
||||||
text: str,
|
|
||||||
metadata: Mapping[str, Any] | None,
|
|
||||||
workspace: Path,
|
|
||||||
) -> list[str]:
|
|
||||||
structured = metadata.get("cli_apps") if isinstance(metadata, Mapping) else None
|
|
||||||
if isinstance(structured, list):
|
|
||||||
mentions = [
|
|
||||||
item for item in structured
|
|
||||||
if isinstance(item, Mapping) and isinstance(item.get("name"), str)
|
|
||||||
]
|
|
||||||
if mentions:
|
|
||||||
return [
|
|
||||||
"CLI App Attachment: "
|
|
||||||
f"@{str(item['name']).strip().lower()} "
|
|
||||||
f"(installed; tool=run_cli_app; "
|
|
||||||
f"entry_point={str(item.get('entry_point') or 'unknown')}; "
|
|
||||||
f"skill=skills/cli-app-{str(item['name']).strip().lower()}/SKILL.md). "
|
|
||||||
"Read the skill when useful, then run this app with `run_cli_app`; do not bypass it with shell."
|
|
||||||
for item in mentions
|
|
||||||
if str(item.get("name") or "").strip()
|
|
||||||
]
|
|
||||||
if "@" not in text:
|
|
||||||
return []
|
|
||||||
try:
|
|
||||||
from nanobot.apps.cli import CliAppManager
|
|
||||||
|
|
||||||
mentions = CliAppManager(workspace=workspace).mentioned_installed_apps(text)
|
|
||||||
except Exception:
|
|
||||||
return []
|
|
||||||
return [
|
|
||||||
"CLI App Mention: "
|
|
||||||
f"@{item['name']} "
|
|
||||||
f"(installed; tool={item['tool']}; "
|
|
||||||
f"entry_point={item['entry_point'] or 'unknown'}; "
|
|
||||||
f"skill={item['skill']}). "
|
|
||||||
"Read the skill when useful, then run this app with `run_cli_app`; do not bypass it with shell."
|
|
||||||
for item in mentions
|
|
||||||
]
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
"""Neutral manifest shape for settings-managed agent apps.
|
|
||||||
|
|
||||||
The manifest is intentionally descriptive. Installers still live in their
|
|
||||||
own adapters, while this protocol gives the WebUI and future registries one
|
|
||||||
small vocabulary for capabilities, trust, and verified install/remove plans.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
APP_PROTOCOL_SCHEMA = "agent-app.v1"
|
|
||||||
|
|
||||||
|
|
||||||
def compact_dict(values: dict[str, Any]) -> dict[str, Any]:
|
|
||||||
"""Drop empty optional values while preserving explicit booleans and zeros."""
|
|
||||||
return {
|
|
||||||
key: value
|
|
||||||
for key, value in values.items()
|
|
||||||
if value is not None and value != "" and value != [] and value != {}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def app_manifest(
|
|
||||||
*,
|
|
||||||
app_id: str,
|
|
||||||
display_name: str,
|
|
||||||
description: str,
|
|
||||||
category: str,
|
|
||||||
source: str,
|
|
||||||
capabilities: list[dict[str, Any]],
|
|
||||||
install: dict[str, Any],
|
|
||||||
remove: dict[str, Any],
|
|
||||||
trust: dict[str, Any],
|
|
||||||
version: str | None = None,
|
|
||||||
logo_url: str | None = None,
|
|
||||||
brand_color: str | None = None,
|
|
||||||
docs_url: str | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Build a stable app manifest dictionary."""
|
|
||||||
return compact_dict({
|
|
||||||
"schema": APP_PROTOCOL_SCHEMA,
|
|
||||||
"id": app_id,
|
|
||||||
"display_name": display_name,
|
|
||||||
"version": version,
|
|
||||||
"description": description,
|
|
||||||
"category": category,
|
|
||||||
"source": source,
|
|
||||||
"logo_url": logo_url,
|
|
||||||
"brand_color": brand_color,
|
|
||||||
"docs_url": docs_url,
|
|
||||||
"capabilities": capabilities,
|
|
||||||
"install": install,
|
|
||||||
"remove": remove,
|
|
||||||
"trust": trust,
|
|
||||||
})
|
|
||||||
@@ -9,12 +9,6 @@ from typing import Any
|
|||||||
# render it and other channels may ignore unknown keys.
|
# render it and other channels may ignore unknown keys.
|
||||||
OUTBOUND_META_AGENT_UI = "_agent_ui"
|
OUTBOUND_META_AGENT_UI = "_agent_ui"
|
||||||
|
|
||||||
# Internal-only inbound metadata used by in-process channels to ask the agent
|
|
||||||
# loop to update runtime state without going through a user session.
|
|
||||||
INBOUND_META_RUNTIME_CONTROL = "_runtime_control"
|
|
||||||
RUNTIME_CONTROL_ACK = "_ack"
|
|
||||||
RUNTIME_CONTROL_MCP_RELOAD = "mcp_reload"
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class InboundMessage:
|
class InboundMessage:
|
||||||
@@ -51,3 +45,4 @@ class OutboundMessage:
|
|||||||
media: list[str] = field(default_factory=list)
|
media: list[str] = field(default_factory=list)
|
||||||
metadata: dict[str, Any] = field(default_factory=dict)
|
metadata: dict[str, Any] = field(default_factory=dict)
|
||||||
buttons: list[list[str]] = field(default_factory=list)
|
buttons: list[list[str]] = field(default_factory=list)
|
||||||
|
|
||||||
|
|||||||
@@ -207,16 +207,6 @@ if DISCORD_AVAILABLE:
|
|||||||
) -> None:
|
) -> None:
|
||||||
await self._forward_slash_command(interaction, _command_text)
|
await self._forward_slash_command(interaction, _command_text)
|
||||||
|
|
||||||
@self.tree.command(name="model", description="Show or switch runtime model preset")
|
|
||||||
@app_commands.describe(preset="Optional model preset name, such as default")
|
|
||||||
async def model_command(
|
|
||||||
interaction: discord.Interaction,
|
|
||||||
preset: str | None = None,
|
|
||||||
) -> None:
|
|
||||||
preset = (preset or "").strip()
|
|
||||||
command_text = f"/model {preset}" if preset else "/model"
|
|
||||||
await self._forward_slash_command(interaction, command_text)
|
|
||||||
|
|
||||||
@self.tree.command(name="help", description="Show available commands")
|
@self.tree.command(name="help", description="Show available commands")
|
||||||
async def help_command(interaction: discord.Interaction) -> None:
|
async def help_command(interaction: discord.Interaction) -> None:
|
||||||
sender_id = str(interaction.user.id)
|
sender_id = str(interaction.user.id)
|
||||||
|
|||||||
@@ -57,17 +57,11 @@ class ChannelManager:
|
|||||||
*,
|
*,
|
||||||
session_manager: "SessionManager | None" = None,
|
session_manager: "SessionManager | None" = None,
|
||||||
webui_runtime_model_name: Callable[[], str | None] | None = None,
|
webui_runtime_model_name: Callable[[], str | None] | None = None,
|
||||||
webui_static_dist: bool = True,
|
|
||||||
webui_runtime_surface: str = "browser",
|
|
||||||
webui_runtime_capabilities: dict[str, Any] | None = None,
|
|
||||||
):
|
):
|
||||||
self.config = config
|
self.config = config
|
||||||
self.bus = bus
|
self.bus = bus
|
||||||
self._session_manager = session_manager
|
self._session_manager = session_manager
|
||||||
self._webui_runtime_model_name = webui_runtime_model_name
|
self._webui_runtime_model_name = webui_runtime_model_name
|
||||||
self._webui_static_dist = webui_static_dist
|
|
||||||
self._webui_runtime_surface = webui_runtime_surface
|
|
||||||
self._webui_runtime_capabilities = dict(webui_runtime_capabilities or {})
|
|
||||||
self.channels: dict[str, BaseChannel] = {}
|
self.channels: dict[str, BaseChannel] = {}
|
||||||
self._dispatch_task: asyncio.Task | None = None
|
self._dispatch_task: asyncio.Task | None = None
|
||||||
self._origin_reply_fingerprints: dict[tuple[str, str, str], str] = {}
|
self._origin_reply_fingerprints: dict[tuple[str, str, str], str] = {}
|
||||||
@@ -113,15 +107,11 @@ class ChannelManager:
|
|||||||
if cls.name == "websocket":
|
if cls.name == "websocket":
|
||||||
if self._session_manager is not None:
|
if self._session_manager is not None:
|
||||||
kwargs["session_manager"] = self._session_manager
|
kwargs["session_manager"] = self._session_manager
|
||||||
static_path = _default_webui_dist() if self._webui_static_dist else None
|
static_path = _default_webui_dist()
|
||||||
if static_path is not None:
|
if static_path is not None:
|
||||||
kwargs["static_dist_path"] = static_path
|
kwargs["static_dist_path"] = static_path
|
||||||
kwargs["workspace_path"] = self.config.workspace_path
|
|
||||||
kwargs["restrict_to_workspace"] = self.config.tools.restrict_to_workspace
|
|
||||||
if self._webui_runtime_model_name is not None:
|
if self._webui_runtime_model_name is not None:
|
||||||
kwargs["runtime_model_name"] = self._webui_runtime_model_name
|
kwargs["runtime_model_name"] = self._webui_runtime_model_name
|
||||||
kwargs["runtime_surface"] = self._webui_runtime_surface
|
|
||||||
kwargs["runtime_capabilities_overrides"] = self._webui_runtime_capabilities
|
|
||||||
channel = cls(section, self.bus, **kwargs)
|
channel = cls(section, self.bus, **kwargs)
|
||||||
channel.transcription_provider = transcription_provider
|
channel.transcription_provider = transcription_provider
|
||||||
channel.transcription_api_key = transcription_key
|
channel.transcription_api_key = transcription_key
|
||||||
|
|||||||
+26
-58
@@ -8,23 +8,21 @@ from contextlib import suppress
|
|||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Literal, TypeAlias
|
from typing import Any, Literal, TypeAlias
|
||||||
from urllib.parse import quote, urlparse
|
|
||||||
|
|
||||||
from pydantic import Field
|
from pydantic import Field
|
||||||
|
|
||||||
from nanobot.security.workspace_policy import is_path_within
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
import aiohttp
|
|
||||||
import nh3
|
import nh3
|
||||||
from mistune import create_markdown
|
from mistune import create_markdown
|
||||||
from nio import (
|
from nio import (
|
||||||
AsyncClient,
|
AsyncClient,
|
||||||
AsyncClientConfig,
|
AsyncClientConfig,
|
||||||
|
DownloadError,
|
||||||
InviteEvent,
|
InviteEvent,
|
||||||
JoinError,
|
JoinError,
|
||||||
LoginResponse,
|
LoginResponse,
|
||||||
MatrixRoom,
|
MatrixRoom,
|
||||||
|
MemoryDownloadResponse,
|
||||||
RoomEncryptedMedia,
|
RoomEncryptedMedia,
|
||||||
RoomMessage,
|
RoomMessage,
|
||||||
RoomMessageMedia,
|
RoomMessageMedia,
|
||||||
@@ -64,10 +62,6 @@ _MSGTYPE_MAP = {"m.image": "image", "m.audio": "audio", "m.video": "video", "m.f
|
|||||||
MATRIX_MEDIA_EVENT_FILTER = (RoomMessageMedia, RoomEncryptedMedia)
|
MATRIX_MEDIA_EVENT_FILTER = (RoomMessageMedia, RoomEncryptedMedia)
|
||||||
MatrixMediaEvent: TypeAlias = RoomMessageMedia | RoomEncryptedMedia
|
MatrixMediaEvent: TypeAlias = RoomMessageMedia | RoomEncryptedMedia
|
||||||
|
|
||||||
|
|
||||||
class _MediaTooLargeError(Exception):
|
|
||||||
"""Raised when an inbound Matrix media download exceeds the configured cap."""
|
|
||||||
|
|
||||||
MATRIX_MARKDOWN = create_markdown(
|
MATRIX_MARKDOWN = create_markdown(
|
||||||
escape=True,
|
escape=True,
|
||||||
plugins=["table", "strikethrough", "url", "superscript", "subscript"],
|
plugins=["table", "strikethrough", "url", "superscript", "subscript"],
|
||||||
@@ -196,7 +190,6 @@ class MatrixConfig(Base):
|
|||||||
e2ee_enabled: bool = Field(default=True, alias="e2eeEnabled")
|
e2ee_enabled: bool = Field(default=True, alias="e2eeEnabled")
|
||||||
sync_stop_grace_seconds: int = 2
|
sync_stop_grace_seconds: int = 2
|
||||||
max_media_bytes: int = 20 * 1024 * 1024
|
max_media_bytes: int = 20 * 1024 * 1024
|
||||||
max_concurrent_media_downloads: int = 2
|
|
||||||
allow_from: list[str] = Field(default_factory=list)
|
allow_from: list[str] = Field(default_factory=list)
|
||||||
group_policy: Literal["open", "mention", "allowlist"] = "open"
|
group_policy: Literal["open", "mention", "allowlist"] = "open"
|
||||||
group_allow_from: list[str] = Field(default_factory=list)
|
group_allow_from: list[str] = Field(default_factory=list)
|
||||||
@@ -238,9 +231,6 @@ class MatrixChannel(BaseChannel):
|
|||||||
self._server_upload_limit_checked = False
|
self._server_upload_limit_checked = False
|
||||||
self._stream_bufs: dict[str, _StreamBuf] = {}
|
self._stream_bufs: dict[str, _StreamBuf] = {}
|
||||||
self._started_at_ms: int = 0
|
self._started_at_ms: int = 0
|
||||||
self._media_download_semaphore = asyncio.Semaphore(
|
|
||||||
max(1, int(self.config.max_concurrent_media_downloads))
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
async def start(self) -> None:
|
async def start(self) -> None:
|
||||||
@@ -354,7 +344,11 @@ class MatrixChannel(BaseChannel):
|
|||||||
"""Check path is inside workspace (when restriction enabled)."""
|
"""Check path is inside workspace (when restriction enabled)."""
|
||||||
if not self._restrict_to_workspace or not self._workspace:
|
if not self._restrict_to_workspace or not self._workspace:
|
||||||
return True
|
return True
|
||||||
return is_path_within(path, self._workspace)
|
try:
|
||||||
|
path.resolve(strict=False).relative_to(self._workspace)
|
||||||
|
return True
|
||||||
|
except ValueError:
|
||||||
|
return False
|
||||||
|
|
||||||
def _collect_outbound_media_candidates(self, media: list[str]) -> list[Path]:
|
def _collect_outbound_media_candidates(self, media: list[str]) -> list[Path]:
|
||||||
"""Deduplicate and resolve outbound attachment paths."""
|
"""Deduplicate and resolve outbound attachment paths."""
|
||||||
@@ -749,7 +743,7 @@ class MatrixChannel(BaseChannel):
|
|||||||
def _event_declared_size_bytes(self, event: MatrixMediaEvent) -> int | None:
|
def _event_declared_size_bytes(self, event: MatrixMediaEvent) -> int | None:
|
||||||
info = self._event_source_content(event).get("info")
|
info = self._event_source_content(event).get("info")
|
||||||
size = info.get("size") if isinstance(info, dict) else None
|
size = info.get("size") if isinstance(info, dict) else None
|
||||||
return size if type(size) is int and size >= 0 else None
|
return size if isinstance(size, int) and size >= 0 else None
|
||||||
|
|
||||||
def _event_mime(self, event: MatrixMediaEvent) -> str | None:
|
def _event_mime(self, event: MatrixMediaEvent) -> str | None:
|
||||||
info = self._event_source_content(event).get("info")
|
info = self._event_source_content(event).get("info")
|
||||||
@@ -778,47 +772,25 @@ class MatrixChannel(BaseChannel):
|
|||||||
event_prefix = (event_id[:24] or "evt").strip("_")
|
event_prefix = (event_id[:24] or "evt").strip("_")
|
||||||
return self._media_dir() / f"{event_prefix}_{stem}{suffix}"
|
return self._media_dir() / f"{event_prefix}_{stem}{suffix}"
|
||||||
|
|
||||||
async def _download_media_bytes(self, mxc_url: str, limit_bytes: int) -> bytes | None:
|
async def _download_media_bytes(self, mxc_url: str) -> bytes | None:
|
||||||
if not self.client or limit_bytes <= 0:
|
if not self.client:
|
||||||
raise _MediaTooLargeError
|
|
||||||
|
|
||||||
parsed = urlparse(mxc_url)
|
|
||||||
if parsed.scheme != "mxc" or not parsed.netloc or not parsed.path.strip("/"):
|
|
||||||
return None
|
return None
|
||||||
|
response = await self.client.download(mxc=mxc_url)
|
||||||
homeserver = str(getattr(self.client, "homeserver", "") or self.config.homeserver).rstrip("/")
|
if isinstance(response, DownloadError):
|
||||||
media_url = (
|
self.logger.warning("download failed for {}: {}", mxc_url, response)
|
||||||
f"{homeserver}/_matrix/client/v1/media/download/"
|
|
||||||
f"{quote(parsed.netloc, safe='')}/{quote(parsed.path.strip('/'), safe='')}"
|
|
||||||
)
|
|
||||||
token = getattr(self.client, "access_token", None) or self.config.access_token
|
|
||||||
headers = {"Authorization": f"Bearer {token}"} if token else None
|
|
||||||
timeout = aiohttp.ClientTimeout(total=None)
|
|
||||||
|
|
||||||
try:
|
|
||||||
async with aiohttp.ClientSession(timeout=timeout, headers=headers) as session:
|
|
||||||
async with session.get(media_url, params={"allow_remote": "true"}) as response:
|
|
||||||
if response.status >= 400:
|
|
||||||
self.logger.warning("download failed for {}: HTTP {}", mxc_url, response.status)
|
|
||||||
return None
|
return None
|
||||||
content_length = response.headers.get("Content-Length")
|
body = getattr(response, "body", None)
|
||||||
if content_length is not None:
|
if isinstance(body, (bytes, bytearray)):
|
||||||
|
return bytes(body)
|
||||||
|
if isinstance(response, MemoryDownloadResponse):
|
||||||
|
return bytes(response.body)
|
||||||
|
if isinstance(body, (str, Path)):
|
||||||
|
path = Path(body)
|
||||||
|
if path.is_file():
|
||||||
try:
|
try:
|
||||||
if int(content_length) > limit_bytes:
|
return path.read_bytes()
|
||||||
raise _MediaTooLargeError
|
except OSError:
|
||||||
except ValueError:
|
return None
|
||||||
pass
|
|
||||||
|
|
||||||
chunks = bytearray()
|
|
||||||
async for chunk in response.content.iter_chunked(64 * 1024):
|
|
||||||
chunks.extend(chunk)
|
|
||||||
if len(chunks) > limit_bytes:
|
|
||||||
raise _MediaTooLargeError
|
|
||||||
return bytes(chunks)
|
|
||||||
except _MediaTooLargeError:
|
|
||||||
raise
|
|
||||||
except (aiohttp.ClientError, asyncio.TimeoutError, OSError):
|
|
||||||
self.logger.warning("download failed for {}", mxc_url, exc_info=True)
|
|
||||||
return None
|
return None
|
||||||
|
|
||||||
def _decrypt_media_bytes(self, event: MatrixMediaEvent, ciphertext: bytes) -> bytes | None:
|
def _decrypt_media_bytes(self, event: MatrixMediaEvent, ciphertext: bytes) -> bytes | None:
|
||||||
@@ -848,14 +820,10 @@ class MatrixChannel(BaseChannel):
|
|||||||
|
|
||||||
limit_bytes = await self._effective_media_limit_bytes()
|
limit_bytes = await self._effective_media_limit_bytes()
|
||||||
declared = self._event_declared_size_bytes(event)
|
declared = self._event_declared_size_bytes(event)
|
||||||
if declared is None or declared > limit_bytes:
|
if declared is not None and declared > limit_bytes:
|
||||||
return None, _ATTACH_TOO_LARGE.format(filename)
|
return None, _ATTACH_TOO_LARGE.format(filename)
|
||||||
|
|
||||||
try:
|
downloaded = await self._download_media_bytes(mxc_url)
|
||||||
async with self._media_download_semaphore:
|
|
||||||
downloaded = await self._download_media_bytes(mxc_url, limit_bytes)
|
|
||||||
except _MediaTooLargeError:
|
|
||||||
return None, _ATTACH_TOO_LARGE.format(filename)
|
|
||||||
if downloaded is None:
|
if downloaded is None:
|
||||||
return None, fail
|
return None, fail
|
||||||
|
|
||||||
|
|||||||
@@ -53,13 +53,6 @@ if MSTEAMS_AVAILABLE:
|
|||||||
|
|
||||||
MSTEAMS_REF_TTL_DAYS = 30
|
MSTEAMS_REF_TTL_DAYS = 30
|
||||||
MSTEAMS_WEBCHAT_HOST = "webchat.botframework.com"
|
MSTEAMS_WEBCHAT_HOST = "webchat.botframework.com"
|
||||||
MSTEAMS_DEFAULT_TRUSTED_SERVICE_URL_HOSTS = [
|
|
||||||
"smba.trafficmanager.net",
|
|
||||||
"smba.infra.gcc.teams.microsoft.com",
|
|
||||||
"smba.infra.gov.teams.microsoft.us",
|
|
||||||
"smba.infra.dod.teams.microsoft.us",
|
|
||||||
"*.botframework.com",
|
|
||||||
]
|
|
||||||
MSTEAMS_REF_META_FILENAME = "msteams_conversations_meta.json"
|
MSTEAMS_REF_META_FILENAME = "msteams_conversations_meta.json"
|
||||||
MSTEAMS_REF_LOCK_FILENAME = "msteams_conversations.lock"
|
MSTEAMS_REF_LOCK_FILENAME = "msteams_conversations.lock"
|
||||||
MSTEAMS_REF_TOUCH_INTERVAL_S = 300
|
MSTEAMS_REF_TOUCH_INTERVAL_S = 300
|
||||||
@@ -83,9 +76,6 @@ class MSTeamsConfig(Base):
|
|||||||
prune_web_chat_refs: bool = True
|
prune_web_chat_refs: bool = True
|
||||||
prune_non_personal_refs: bool = True
|
prune_non_personal_refs: bool = True
|
||||||
ref_touch_interval_s: int = Field(default=MSTEAMS_REF_TOUCH_INTERVAL_S, ge=0)
|
ref_touch_interval_s: int = Field(default=MSTEAMS_REF_TOUCH_INTERVAL_S, ge=0)
|
||||||
trusted_service_url_hosts: list[str] = Field(
|
|
||||||
default_factory=lambda: MSTEAMS_DEFAULT_TRUSTED_SERVICE_URL_HOSTS.copy()
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
@@ -252,11 +242,6 @@ class MSTeamsChannel(BaseChannel):
|
|||||||
if not ref:
|
if not ref:
|
||||||
raise RuntimeError(f"MSTeams conversation ref not found for chat_id={msg.chat_id}")
|
raise RuntimeError(f"MSTeams conversation ref not found for chat_id={msg.chat_id}")
|
||||||
|
|
||||||
if not self._is_trusted_service_url(ref.service_url):
|
|
||||||
raise RuntimeError(
|
|
||||||
f"MSTeams conversation ref has untrusted service_url for chat_id={msg.chat_id}"
|
|
||||||
)
|
|
||||||
|
|
||||||
token = await self._get_access_token()
|
token = await self._get_access_token()
|
||||||
base_url = f"{ref.service_url.rstrip('/')}/v3/conversations/{ref.conversation_id}/activities"
|
base_url = f"{ref.service_url.rstrip('/')}/v3/conversations/{ref.conversation_id}/activities"
|
||||||
use_thread_reply = self.config.reply_in_thread and bool(ref.activity_id)
|
use_thread_reply = self.config.reply_in_thread and bool(ref.activity_id)
|
||||||
@@ -299,13 +284,6 @@ class MSTeamsChannel(BaseChannel):
|
|||||||
if not sender_id or not conversation_id or not service_url:
|
if not sender_id or not conversation_id or not service_url:
|
||||||
return
|
return
|
||||||
|
|
||||||
if not self._is_trusted_service_url(service_url):
|
|
||||||
self.logger.warning(
|
|
||||||
"Ignoring MSTeams activity with untrusted serviceUrl host: {}",
|
|
||||||
service_url,
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
if recipient.get("id") and from_user.get("id") == recipient.get("id"):
|
if recipient.get("id") and from_user.get("id") == recipient.get("id"):
|
||||||
return
|
return
|
||||||
|
|
||||||
@@ -648,29 +626,6 @@ class MSTeamsChannel(BaseChannel):
|
|||||||
return host == MSTEAMS_WEBCHAT_HOST or host.endswith(f".{MSTEAMS_WEBCHAT_HOST}")
|
return host == MSTEAMS_WEBCHAT_HOST or host.endswith(f".{MSTEAMS_WEBCHAT_HOST}")
|
||||||
return MSTEAMS_WEBCHAT_HOST in normalized.lower()
|
return MSTEAMS_WEBCHAT_HOST in normalized.lower()
|
||||||
|
|
||||||
def _is_trusted_service_url(self, service_url: str) -> bool:
|
|
||||||
"""Return True for HTTPS Bot Framework service URLs trusted for bearer replies."""
|
|
||||||
parsed = urlparse(service_url.strip())
|
|
||||||
if parsed.scheme.lower() != "https":
|
|
||||||
return False
|
|
||||||
|
|
||||||
host = (parsed.hostname or "").strip().lower().rstrip(".")
|
|
||||||
if not host:
|
|
||||||
return False
|
|
||||||
|
|
||||||
for pattern in self.config.trusted_service_url_hosts:
|
|
||||||
trusted_host = str(pattern or "").strip().lower().rstrip(".")
|
|
||||||
if not trusted_host:
|
|
||||||
continue
|
|
||||||
if trusted_host.startswith("*."):
|
|
||||||
suffix = trusted_host[1:]
|
|
||||||
if host.endswith(suffix) and host != suffix.lstrip("."):
|
|
||||||
return True
|
|
||||||
continue
|
|
||||||
if host == trusted_host:
|
|
||||||
return True
|
|
||||||
return False
|
|
||||||
|
|
||||||
def _prune_conversation_refs(self, *, now: float | None = None) -> bool:
|
def _prune_conversation_refs(self, *, now: float | None = None) -> bool:
|
||||||
"""Remove stale and unsupported conversation refs from memory."""
|
"""Remove stale and unsupported conversation refs from memory."""
|
||||||
if not self._conversation_refs:
|
if not self._conversation_refs:
|
||||||
@@ -682,10 +637,6 @@ class MSTeamsChannel(BaseChannel):
|
|||||||
keys_to_drop: list[str] = []
|
keys_to_drop: list[str] = []
|
||||||
|
|
||||||
for key, ref in self._conversation_refs.items():
|
for key, ref in self._conversation_refs.items():
|
||||||
if not self._is_trusted_service_url(ref.service_url):
|
|
||||||
keys_to_drop.append(key)
|
|
||||||
continue
|
|
||||||
|
|
||||||
if self.config.prune_web_chat_refs and self._is_webchat_service_url(ref.service_url):
|
if self.config.prune_web_chat_refs and self._is_webchat_service_url(ref.service_url):
|
||||||
keys_to_drop.append(key)
|
keys_to_drop.append(key)
|
||||||
continue
|
continue
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -10,9 +10,8 @@ from contextlib import suppress
|
|||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Literal
|
from typing import Any, Literal
|
||||||
from urllib.parse import urlparse
|
|
||||||
|
|
||||||
from pydantic import Field, field_validator, model_validator
|
from pydantic import Field
|
||||||
from telegram import (
|
from telegram import (
|
||||||
BotCommand,
|
BotCommand,
|
||||||
InlineKeyboardButton,
|
InlineKeyboardButton,
|
||||||
@@ -226,22 +225,11 @@ class _StreamBuf:
|
|||||||
stream_id: str | None = None
|
stream_id: str | None = None
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class _QueuedTelegramUpdate:
|
|
||||||
"""Telegram update staged for per-session ordered processing."""
|
|
||||||
|
|
||||||
kind: Literal["command", "message"]
|
|
||||||
update: Update
|
|
||||||
context: Any
|
|
||||||
sort_key: tuple[int, int]
|
|
||||||
|
|
||||||
|
|
||||||
class TelegramConfig(Base):
|
class TelegramConfig(Base):
|
||||||
"""Telegram channel configuration."""
|
"""Telegram channel configuration."""
|
||||||
|
|
||||||
enabled: bool = False
|
enabled: bool = False
|
||||||
token: str = ""
|
token: str = ""
|
||||||
mode: Literal["polling", "webhook"] = "polling"
|
|
||||||
allow_from: list[str] = Field(default_factory=list)
|
allow_from: list[str] = Field(default_factory=list)
|
||||||
proxy: str | None = None
|
proxy: str | None = None
|
||||||
reply_to_message: bool = False
|
reply_to_message: bool = False
|
||||||
@@ -253,48 +241,13 @@ class TelegramConfig(Base):
|
|||||||
# Enable inline keyboard buttons in Telegram messages.
|
# Enable inline keyboard buttons in Telegram messages.
|
||||||
inline_keyboards: bool = False
|
inline_keyboards: bool = False
|
||||||
stream_edit_interval: float = Field(default=_STREAM_EDIT_INTERVAL_DEFAULT, ge=0.1)
|
stream_edit_interval: float = Field(default=_STREAM_EDIT_INTERVAL_DEFAULT, ge=0.1)
|
||||||
webhook_url: str = ""
|
|
||||||
webhook_listen_host: str = "127.0.0.1"
|
|
||||||
webhook_listen_port: int = Field(default=8081, ge=1, le=65535)
|
|
||||||
webhook_path: str = "/telegram"
|
|
||||||
webhook_secret_token: str = ""
|
|
||||||
webhook_max_connections: int = Field(default=4, ge=1, le=100)
|
|
||||||
|
|
||||||
@field_validator("webhook_path")
|
|
||||||
@classmethod
|
|
||||||
def webhook_path_must_start_with_slash(cls, value: str) -> str:
|
|
||||||
value = value.strip() or "/telegram"
|
|
||||||
if not value.startswith("/"):
|
|
||||||
raise ValueError('webhook_path must start with "/"')
|
|
||||||
return value
|
|
||||||
|
|
||||||
@model_validator(mode="after")
|
|
||||||
def validate_webhook_config(self) -> "TelegramConfig":
|
|
||||||
if self.mode != "webhook":
|
|
||||||
return self
|
|
||||||
|
|
||||||
url = self.webhook_url.strip()
|
|
||||||
if not url:
|
|
||||||
raise ValueError("webhook_url is required when Telegram mode is webhook")
|
|
||||||
parsed = urlparse(url)
|
|
||||||
if parsed.scheme != "https" or not parsed.netloc:
|
|
||||||
raise ValueError("webhook_url must be a public HTTPS URL")
|
|
||||||
secret = self.webhook_secret_token.strip()
|
|
||||||
if not secret:
|
|
||||||
raise ValueError("webhook_secret_token is required when Telegram mode is webhook")
|
|
||||||
if len(secret) > 256 or re.match(r"^[A-Za-z0-9_-]+$", secret) is None:
|
|
||||||
raise ValueError(
|
|
||||||
"webhook_secret_token must be 1-256 characters using only A-Z, a-z, 0-9, _ and -"
|
|
||||||
)
|
|
||||||
return self
|
|
||||||
|
|
||||||
|
|
||||||
class TelegramChannel(BaseChannel):
|
class TelegramChannel(BaseChannel):
|
||||||
"""
|
"""
|
||||||
Telegram channel using long polling or webhook mode.
|
Telegram channel using long polling.
|
||||||
|
|
||||||
Long polling is the default. Webhook mode requires a public HTTPS URL and a
|
Simple and reliable - no webhook/public IP needed.
|
||||||
Telegram secret token.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
name = "telegram"
|
name = "telegram"
|
||||||
@@ -341,8 +294,6 @@ class TelegramChannel(BaseChannel):
|
|||||||
self._bot_user_id: int | None = None
|
self._bot_user_id: int | None = None
|
||||||
self._bot_username: str | None = None
|
self._bot_username: str | None = None
|
||||||
self._stream_bufs: dict[str, _StreamBuf] = {} # chat_id -> streaming state
|
self._stream_bufs: dict[str, _StreamBuf] = {} # chat_id -> streaming state
|
||||||
self._inbound_buffers: dict[str, list[_QueuedTelegramUpdate]] = {}
|
|
||||||
self._inbound_workers: dict[str, asyncio.Task] = {}
|
|
||||||
|
|
||||||
def is_allowed(self, sender_id: str) -> bool:
|
def is_allowed(self, sender_id: str) -> bool:
|
||||||
"""Preserve Telegram's legacy id|username allowlist matching."""
|
"""Preserve Telegram's legacy id|username allowlist matching."""
|
||||||
@@ -375,7 +326,7 @@ class TelegramChannel(BaseChannel):
|
|||||||
return content
|
return content
|
||||||
|
|
||||||
async def start(self) -> None:
|
async def start(self) -> None:
|
||||||
"""Start the Telegram bot."""
|
"""Start the Telegram bot with long polling."""
|
||||||
if not self.config.token:
|
if not self.config.token:
|
||||||
self.logger.error("bot token not configured")
|
self.logger.error("bot token not configured")
|
||||||
return
|
return
|
||||||
@@ -443,12 +394,9 @@ class TelegramChannel(BaseChannel):
|
|||||||
else:
|
else:
|
||||||
allowed_updates = ["message"]
|
allowed_updates = ["message"]
|
||||||
|
|
||||||
if self.config.mode == "webhook":
|
|
||||||
self.logger.info("Starting bot (webhook mode)...")
|
|
||||||
else:
|
|
||||||
self.logger.info("Starting bot (polling mode)...")
|
self.logger.info("Starting bot (polling mode)...")
|
||||||
|
|
||||||
# Initialize and start receiving updates
|
# Initialize and start polling
|
||||||
await self._app.initialize()
|
await self._app.initialize()
|
||||||
await self._app.start()
|
await self._app.start()
|
||||||
|
|
||||||
@@ -464,20 +412,6 @@ class TelegramChannel(BaseChannel):
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
self.logger.warning("Failed to register bot commands: {}", e)
|
self.logger.warning("Failed to register bot commands: {}", e)
|
||||||
|
|
||||||
if self.config.mode == "webhook":
|
|
||||||
# ``url_path`` is the local HTTP route. ``webhook_url`` is the
|
|
||||||
# public HTTPS URL Telegram calls; reverse proxies may rewrite it.
|
|
||||||
await self._app.updater.start_webhook(
|
|
||||||
listen=self.config.webhook_listen_host,
|
|
||||||
port=self.config.webhook_listen_port,
|
|
||||||
url_path=self.config.webhook_path.lstrip("/"),
|
|
||||||
webhook_url=self.config.webhook_url.strip(),
|
|
||||||
allowed_updates=allowed_updates,
|
|
||||||
drop_pending_updates=False,
|
|
||||||
secret_token=self.config.webhook_secret_token.strip(),
|
|
||||||
max_connections=self.config.webhook_max_connections,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
# Start polling (this runs until stopped)
|
# Start polling (this runs until stopped)
|
||||||
await self._app.updater.start_polling(
|
await self._app.updater.start_polling(
|
||||||
allowed_updates=allowed_updates,
|
allowed_updates=allowed_updates,
|
||||||
@@ -502,11 +436,6 @@ class TelegramChannel(BaseChannel):
|
|||||||
self._media_group_tasks.clear()
|
self._media_group_tasks.clear()
|
||||||
self._media_group_buffers.clear()
|
self._media_group_buffers.clear()
|
||||||
|
|
||||||
for task in self._inbound_workers.values():
|
|
||||||
task.cancel()
|
|
||||||
self._inbound_workers.clear()
|
|
||||||
self._inbound_buffers.clear()
|
|
||||||
|
|
||||||
if self._app:
|
if self._app:
|
||||||
self.logger.info("Stopping bot...")
|
self.logger.info("Stopping bot...")
|
||||||
await self._app.updater.stop()
|
await self._app.updater.stop()
|
||||||
@@ -1066,85 +995,10 @@ class TelegramChannel(BaseChannel):
|
|||||||
if len(self._message_threads) > 1000:
|
if len(self._message_threads) > 1000:
|
||||||
self._message_threads.pop(next(iter(self._message_threads)))
|
self._message_threads.pop(next(iter(self._message_threads)))
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _queue_key_for_message(message) -> str:
|
|
||||||
"""Return the final nanobot session key used for ordered Telegram ingress."""
|
|
||||||
return TelegramChannel._derive_topic_session_key(message) or f"telegram:{message.chat_id}"
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _sort_key_for_update(update: Update) -> tuple[int, int]:
|
|
||||||
"""Sort by chat message id first, then Telegram update id."""
|
|
||||||
message = getattr(update, "message", None)
|
|
||||||
message_id = int(getattr(message, "message_id", 0) or 0)
|
|
||||||
update_id = int(getattr(update, "update_id", 0) or 0)
|
|
||||||
return (message_id, update_id)
|
|
||||||
|
|
||||||
def _enqueue_ordered_update(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
kind: Literal["command", "message"],
|
|
||||||
update: Update,
|
|
||||||
context: ContextTypes.DEFAULT_TYPE,
|
|
||||||
) -> None:
|
|
||||||
"""Stage a Telegram update behind a short per-session reorder window."""
|
|
||||||
message = update.message
|
|
||||||
key = self._queue_key_for_message(message)
|
|
||||||
self._inbound_buffers.setdefault(key, []).append(
|
|
||||||
_QueuedTelegramUpdate(
|
|
||||||
kind=kind,
|
|
||||||
update=update,
|
|
||||||
context=context,
|
|
||||||
sort_key=self._sort_key_for_update(update),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
if key not in self._inbound_workers:
|
|
||||||
self._inbound_workers[key] = asyncio.create_task(
|
|
||||||
self._drain_ordered_updates(key)
|
|
||||||
)
|
|
||||||
|
|
||||||
async def _drain_ordered_updates(self, key: str) -> None:
|
|
||||||
"""Drain one Telegram session buffer in stable message order."""
|
|
||||||
try:
|
|
||||||
while self._running:
|
|
||||||
await asyncio.sleep(0.2)
|
|
||||||
batch = self._inbound_buffers.get(key, [])
|
|
||||||
if not batch:
|
|
||||||
break
|
|
||||||
self._inbound_buffers[key] = []
|
|
||||||
batch.sort(key=lambda item: item.sort_key)
|
|
||||||
for item in batch:
|
|
||||||
try:
|
|
||||||
if item.kind == "command":
|
|
||||||
await self._process_forward_command(item.update, item.context)
|
|
||||||
else:
|
|
||||||
await self._process_message_update(item.update, item.context)
|
|
||||||
except Exception as e:
|
|
||||||
self.logger.warning(
|
|
||||||
"Telegram queued update handling failed for {}: {}",
|
|
||||||
key,
|
|
||||||
e,
|
|
||||||
)
|
|
||||||
if not self._inbound_buffers.get(key):
|
|
||||||
self._inbound_buffers.pop(key, None)
|
|
||||||
except asyncio.CancelledError:
|
|
||||||
raise
|
|
||||||
except Exception as e:
|
|
||||||
self.logger.warning("Telegram ordered update worker failed for {}: {}", key, e)
|
|
||||||
finally:
|
|
||||||
if not self._inbound_buffers.get(key):
|
|
||||||
self._inbound_workers.pop(key, None)
|
|
||||||
|
|
||||||
async def _forward_command(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
|
async def _forward_command(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
|
||||||
"""Forward slash commands to the bus for unified handling in AgentLoop."""
|
"""Forward slash commands to the bus for unified handling in AgentLoop."""
|
||||||
if not update.message or not update.effective_user:
|
if not update.message or not update.effective_user:
|
||||||
return
|
return
|
||||||
if not self._running:
|
|
||||||
await self._process_forward_command(update, context)
|
|
||||||
return
|
|
||||||
self._enqueue_ordered_update(kind="command", update=update, context=context)
|
|
||||||
|
|
||||||
async def _process_forward_command(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
|
|
||||||
"""Process a queued slash command."""
|
|
||||||
message = update.message
|
message = update.message
|
||||||
user = update.effective_user
|
user = update.effective_user
|
||||||
sender_id = self._sender_id(user)
|
sender_id = self._sender_id(user)
|
||||||
@@ -1173,13 +1027,6 @@ class TelegramChannel(BaseChannel):
|
|||||||
"""Handle incoming messages (text, photos, voice, documents)."""
|
"""Handle incoming messages (text, photos, voice, documents)."""
|
||||||
if not update.message or not update.effective_user:
|
if not update.message or not update.effective_user:
|
||||||
return
|
return
|
||||||
if not self._running:
|
|
||||||
await self._process_message_update(update, context)
|
|
||||||
return
|
|
||||||
self._enqueue_ordered_update(kind="message", update=update, context=context)
|
|
||||||
|
|
||||||
async def _process_message_update(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
|
|
||||||
"""Process a queued Telegram message update."""
|
|
||||||
|
|
||||||
message = update.message
|
message = update.message
|
||||||
user = update.effective_user
|
user = update.effective_user
|
||||||
|
|||||||
+221
-399
@@ -3,69 +3,61 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
|
import base64
|
||||||
|
import binascii
|
||||||
import email.utils
|
import email.utils
|
||||||
|
import hashlib
|
||||||
import hmac
|
import hmac
|
||||||
import http
|
import http
|
||||||
import json
|
import json
|
||||||
import mimetypes
|
import mimetypes
|
||||||
import re
|
import re
|
||||||
import secrets
|
import secrets
|
||||||
|
import shutil
|
||||||
import ssl
|
import ssl
|
||||||
import time
|
import time
|
||||||
import uuid
|
import uuid
|
||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
from contextlib import suppress
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import TYPE_CHECKING, Any, Self
|
from typing import TYPE_CHECKING, Any, Self
|
||||||
from urllib.parse import parse_qs, unquote, urlparse
|
from urllib.parse import parse_qs, unquote, urlparse
|
||||||
|
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
from pydantic import Field, field_validator, model_validator
|
from pydantic import Field, field_validator, model_validator
|
||||||
from websockets.asyncio.server import ServerConnection, serve, unix_serve
|
from websockets.asyncio.server import ServerConnection, serve
|
||||||
from websockets.datastructures import Headers
|
from websockets.datastructures import Headers
|
||||||
from websockets.exceptions import ConnectionClosed
|
from websockets.exceptions import ConnectionClosed
|
||||||
from websockets.http11 import Request as WsRequest
|
from websockets.http11 import Request as WsRequest
|
||||||
from websockets.http11 import Response
|
from websockets.http11 import Response
|
||||||
|
|
||||||
from nanobot.security.workspace_access import (
|
|
||||||
WORKSPACE_SCOPE_METADATA_KEY,
|
|
||||||
WorkspaceScopeError,
|
|
||||||
)
|
|
||||||
from nanobot.bus.events import OUTBOUND_META_AGENT_UI, OutboundMessage
|
from nanobot.bus.events import OUTBOUND_META_AGENT_UI, OutboundMessage
|
||||||
from nanobot.bus.queue import MessageBus
|
from nanobot.bus.queue import MessageBus
|
||||||
from nanobot.channels.base import BaseChannel
|
from nanobot.channels.base import BaseChannel
|
||||||
from nanobot.command.builtin import builtin_command_palette
|
from nanobot.command.builtin import builtin_command_palette
|
||||||
from nanobot.config.paths import get_media_dir, get_workspace_path
|
from nanobot.config.paths import get_media_dir
|
||||||
from nanobot.config.schema import Base
|
from nanobot.config.schema import Base
|
||||||
from nanobot.session.goal_state import goal_state_ws_blob
|
from nanobot.session.goal_state import goal_state_ws_blob
|
||||||
from nanobot.session.webui_turns import websocket_turn_wall_started_at
|
from nanobot.session.webui_turns import websocket_turn_wall_started_at
|
||||||
|
from nanobot.utils.helpers import safe_filename
|
||||||
from nanobot.utils.media_decode import (
|
from nanobot.utils.media_decode import (
|
||||||
FileSizeExceeded,
|
FileSizeExceeded,
|
||||||
save_base64_data_url,
|
save_base64_data_url,
|
||||||
)
|
)
|
||||||
from nanobot.utils.subagent_channel_display import scrub_subagent_messages_for_channel
|
from nanobot.utils.subagent_channel_display import scrub_subagent_messages_for_channel
|
||||||
from nanobot.webui.settings_api import runtime_capabilities
|
from nanobot.webui.settings_api import (
|
||||||
from nanobot.webui.cli_apps_api import normalize_cli_app_mentions
|
WebUISettingsError,
|
||||||
from nanobot.webui.media_api import (
|
settings_payload,
|
||||||
serve_signed_media,
|
update_agent_settings,
|
||||||
sign_media_path,
|
update_image_generation_settings,
|
||||||
sign_or_stage_media_path,
|
update_provider_settings,
|
||||||
|
update_web_search_settings,
|
||||||
)
|
)
|
||||||
from nanobot.webui.mcp_presets_api import normalize_mcp_preset_mentions
|
|
||||||
from nanobot.webui.settings_routes import WebUISettingsRouter
|
|
||||||
from nanobot.webui.sidebar_state import (
|
from nanobot.webui.sidebar_state import (
|
||||||
read_webui_sidebar_state,
|
read_webui_sidebar_state,
|
||||||
write_webui_sidebar_state,
|
write_webui_sidebar_state,
|
||||||
)
|
)
|
||||||
from nanobot.webui.thread_disk import delete_webui_thread
|
from nanobot.webui.thread_disk import delete_webui_thread
|
||||||
from nanobot.webui.transcript import (
|
from nanobot.webui.transcript import append_transcript_object, build_webui_thread_response
|
||||||
append_transcript_object,
|
|
||||||
build_webui_thread_response,
|
|
||||||
rewrite_local_markdown_images,
|
|
||||||
)
|
|
||||||
from nanobot.webui.workspaces import (
|
|
||||||
WebUIWorkspaceController,
|
|
||||||
)
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from nanobot.session.manager import SessionManager
|
from nanobot.session.manager import SessionManager
|
||||||
@@ -81,41 +73,6 @@ def _normalize_config_path(path: str) -> str:
|
|||||||
return _strip_trailing_slash(path)
|
return _strip_trailing_slash(path)
|
||||||
|
|
||||||
|
|
||||||
def _case_insensitive_header(headers: Any, key: str) -> str:
|
|
||||||
"""Read a header from websockets/http test stubs without assuming casing."""
|
|
||||||
try:
|
|
||||||
value = headers.get(key)
|
|
||||||
except Exception:
|
|
||||||
value = None
|
|
||||||
if value is None:
|
|
||||||
try:
|
|
||||||
value = headers.get(key.lower())
|
|
||||||
except Exception:
|
|
||||||
value = None
|
|
||||||
return str(value or "").strip()
|
|
||||||
|
|
||||||
|
|
||||||
def _safe_host_header(value: str) -> str:
|
|
||||||
"""Return a safe Host header value, or empty when it should not be echoed."""
|
|
||||||
value = value.strip()
|
|
||||||
if not value:
|
|
||||||
return ""
|
|
||||||
if re.fullmatch(r"\[[0-9A-Fa-f:.]+\](?::\d{1,5})?", value):
|
|
||||||
return value
|
|
||||||
if re.fullmatch(r"[A-Za-z0-9.-]+(?::\d{1,5})?", value):
|
|
||||||
return value
|
|
||||||
return ""
|
|
||||||
|
|
||||||
|
|
||||||
def _host_for_url(host: str, port: int) -> str:
|
|
||||||
host = host.strip()
|
|
||||||
if host in ("0.0.0.0", "::"):
|
|
||||||
host = "127.0.0.1"
|
|
||||||
if ":" in host and not host.startswith("["):
|
|
||||||
host = f"[{host}]"
|
|
||||||
return f"{host}:{port}"
|
|
||||||
|
|
||||||
|
|
||||||
class WebSocketConfig(Base):
|
class WebSocketConfig(Base):
|
||||||
"""WebSocket server channel configuration.
|
"""WebSocket server channel configuration.
|
||||||
|
|
||||||
@@ -139,7 +96,6 @@ class WebSocketConfig(Base):
|
|||||||
enabled: bool = False
|
enabled: bool = False
|
||||||
host: str = "127.0.0.1"
|
host: str = "127.0.0.1"
|
||||||
port: int = 8765
|
port: int = 8765
|
||||||
unix_socket_path: str = ""
|
|
||||||
path: str = "/"
|
path: str = "/"
|
||||||
token: str = ""
|
token: str = ""
|
||||||
token_issue_path: str = ""
|
token_issue_path: str = ""
|
||||||
@@ -158,19 +114,6 @@ class WebSocketConfig(Base):
|
|||||||
ssl_certfile: str = ""
|
ssl_certfile: str = ""
|
||||||
ssl_keyfile: str = ""
|
ssl_keyfile: str = ""
|
||||||
|
|
||||||
@field_validator("unix_socket_path")
|
|
||||||
@classmethod
|
|
||||||
def unix_socket_path_format(cls, value: str) -> str:
|
|
||||||
value = value.strip()
|
|
||||||
if not value:
|
|
||||||
return ""
|
|
||||||
if "\x00" in value:
|
|
||||||
raise ValueError("unix_socket_path must not contain NUL bytes")
|
|
||||||
path = Path(value).expanduser()
|
|
||||||
if not path.is_absolute():
|
|
||||||
raise ValueError("unix_socket_path must be an absolute path")
|
|
||||||
return str(path)
|
|
||||||
|
|
||||||
@field_validator("path")
|
@field_validator("path")
|
||||||
@classmethod
|
@classmethod
|
||||||
def path_must_start_with_slash(cls, value: str) -> str:
|
def path_must_start_with_slash(cls, value: str) -> str:
|
||||||
@@ -453,6 +396,32 @@ def _is_websocket_upgrade(request: WsRequest) -> bool:
|
|||||||
return True
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def _b64url_encode(data: bytes) -> str:
|
||||||
|
"""URL-safe base64 without padding — compact + friendly in URL paths."""
|
||||||
|
return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
|
||||||
|
|
||||||
|
|
||||||
|
def _b64url_decode(s: str) -> bytes:
|
||||||
|
"""Reverse of :func:`_b64url_encode`; caller handles ``ValueError``."""
|
||||||
|
pad = "=" * (-len(s) % 4)
|
||||||
|
return base64.urlsafe_b64decode(s + pad)
|
||||||
|
|
||||||
|
|
||||||
|
# Allowed MIME types we actually serve from the media endpoint. Anything
|
||||||
|
# outside this set is degraded to ``application/octet-stream`` so an
|
||||||
|
# attacker who somehow gets a signed URL for an unexpected file type can't
|
||||||
|
# trick the browser into sniffing executable content.
|
||||||
|
_MEDIA_ALLOWED_MIMES: frozenset[str] = frozenset({
|
||||||
|
"image/png",
|
||||||
|
"image/jpeg",
|
||||||
|
"image/webp",
|
||||||
|
"image/gif",
|
||||||
|
"video/mp4",
|
||||||
|
"video/webm",
|
||||||
|
"video/quicktime",
|
||||||
|
})
|
||||||
|
|
||||||
|
|
||||||
def _issue_route_secret_matches(headers: Any, configured_secret: str) -> bool:
|
def _issue_route_secret_matches(headers: Any, configured_secret: str) -> bool:
|
||||||
"""Return True if the token-issue HTTP request carries credentials matching ``token_issue_secret``."""
|
"""Return True if the token-issue HTTP request carries credentials matching ``token_issue_secret``."""
|
||||||
if not configured_secret:
|
if not configured_secret:
|
||||||
@@ -480,11 +449,7 @@ class WebSocketChannel(BaseChannel):
|
|||||||
*,
|
*,
|
||||||
session_manager: "SessionManager | None" = None,
|
session_manager: "SessionManager | None" = None,
|
||||||
static_dist_path: Path | None = None,
|
static_dist_path: Path | None = None,
|
||||||
workspace_path: Path | None = None,
|
|
||||||
restrict_to_workspace: bool = False,
|
|
||||||
runtime_model_name: Callable[[], str | None] | None = None,
|
runtime_model_name: Callable[[], str | None] | None = None,
|
||||||
runtime_surface: str = "browser",
|
|
||||||
runtime_capabilities_overrides: dict[str, Any] | None = None,
|
|
||||||
):
|
):
|
||||||
if isinstance(config, dict):
|
if isinstance(config, dict):
|
||||||
config = WebSocketConfig.model_validate(config)
|
config = WebSocketConfig.model_validate(config)
|
||||||
@@ -506,36 +471,8 @@ class WebSocketChannel(BaseChannel):
|
|||||||
self._static_dist_path: Path | None = (
|
self._static_dist_path: Path | None = (
|
||||||
static_dist_path.resolve() if static_dist_path is not None else None
|
static_dist_path.resolve() if static_dist_path is not None else None
|
||||||
)
|
)
|
||||||
self._workspace_path = (
|
|
||||||
Path(workspace_path).expanduser()
|
|
||||||
if workspace_path is not None
|
|
||||||
else get_workspace_path()
|
|
||||||
).resolve(strict=False)
|
|
||||||
self._default_restrict_to_workspace = restrict_to_workspace
|
|
||||||
self._webui_workspaces = WebUIWorkspaceController(
|
|
||||||
session_manager=self._session_manager,
|
|
||||||
default_workspace=self._workspace_path,
|
|
||||||
default_restrict_to_workspace=self._default_restrict_to_workspace,
|
|
||||||
)
|
|
||||||
self._runtime_model_name = runtime_model_name
|
self._runtime_model_name = runtime_model_name
|
||||||
self._runtime_surface = (
|
self._settings_restart_sections: set[str] = set()
|
||||||
"native" if runtime_surface in {"native", "desktop"} else "browser"
|
|
||||||
)
|
|
||||||
self._runtime_capabilities = runtime_capabilities(
|
|
||||||
self._runtime_surface,
|
|
||||||
runtime_capabilities_overrides,
|
|
||||||
)
|
|
||||||
self._settings_routes = WebUISettingsRouter(
|
|
||||||
bus=self.bus,
|
|
||||||
logger=self.logger,
|
|
||||||
check_api_token=self._check_api_token,
|
|
||||||
parse_query=_parse_query,
|
|
||||||
json_response=_http_json_response,
|
|
||||||
error_response=_http_error,
|
|
||||||
runtime_surface=self._runtime_surface,
|
|
||||||
runtime_capabilities=self._runtime_capabilities,
|
|
||||||
)
|
|
||||||
self._stream_text_buffers: dict[tuple[str, str], list[str]] = {}
|
|
||||||
# Process-local secret used to HMAC-sign media URLs. The signed URL is
|
# Process-local secret used to HMAC-sign media URLs. The signed URL is
|
||||||
# the capability — anyone who holds a valid URL can fetch that one
|
# the capability — anyone who holds a valid URL can fetch that one
|
||||||
# file, nothing else. The secret regenerates on restart so links
|
# file, nothing else. The secret regenerates on restart so links
|
||||||
@@ -678,91 +615,44 @@ class WebSocketChannel(BaseChannel):
|
|||||||
"""Route an inbound HTTP request to a handler or to the WS upgrade path."""
|
"""Route an inbound HTTP request to a handler or to the WS upgrade path."""
|
||||||
got, query = _parse_request_path(request.path)
|
got, query = _parse_request_path(request.path)
|
||||||
|
|
||||||
|
# 1. Token issue endpoint (legacy, optional, gated by configured secret).
|
||||||
if self.config.token_issue_path:
|
if self.config.token_issue_path:
|
||||||
issue_expected = _normalize_config_path(self.config.token_issue_path)
|
issue_expected = _normalize_config_path(self.config.token_issue_path)
|
||||||
if got == issue_expected:
|
if got == issue_expected:
|
||||||
return self._handle_token_issue_http(connection, request)
|
return self._handle_token_issue_http(connection, request)
|
||||||
|
|
||||||
|
# 2. Bootstrap (`/webui/bootstrap`): mint WS/API tokens + shared session metadata.
|
||||||
if got == "/webui/bootstrap":
|
if got == "/webui/bootstrap":
|
||||||
return self._handle_bootstrap(connection, request)
|
return self._handle_bootstrap(connection, request)
|
||||||
|
|
||||||
api_response = await self._dispatch_api_route(connection, request, got)
|
# 3. REST handlers co-located with this channel (sessions, settings, …).
|
||||||
if api_response is not None:
|
|
||||||
return api_response
|
|
||||||
|
|
||||||
ws_matched, ws_response = self._dispatch_websocket_upgrade(
|
|
||||||
connection, request, got, query
|
|
||||||
)
|
|
||||||
if ws_matched:
|
|
||||||
return ws_response
|
|
||||||
|
|
||||||
# API clients should never receive the SPA shell for an unknown route.
|
|
||||||
# Returning HTML here makes the WebUI fail with "Unexpected token <"
|
|
||||||
# when a dev server is pointed at an older gateway.
|
|
||||||
if got.startswith("/api/"):
|
|
||||||
return _http_error(404, "API route not found")
|
|
||||||
|
|
||||||
if self._static_dist_path is not None:
|
|
||||||
response = self._serve_static(got)
|
|
||||||
if response is not None:
|
|
||||||
return response
|
|
||||||
|
|
||||||
return connection.respond(404, "Not Found")
|
|
||||||
|
|
||||||
async def _dispatch_api_route(
|
|
||||||
self,
|
|
||||||
connection: Any,
|
|
||||||
request: WsRequest,
|
|
||||||
got: str,
|
|
||||||
) -> Any | None:
|
|
||||||
"""Route REST-ish WebUI requests served beside the WebSocket endpoint."""
|
|
||||||
response = await self._dispatch_settings_api_route(request, got)
|
|
||||||
if response is not None:
|
|
||||||
return response
|
|
||||||
response = self._dispatch_session_api_route(request, got)
|
|
||||||
if response is not None:
|
|
||||||
return response
|
|
||||||
response = self._dispatch_media_api_route(request, got)
|
|
||||||
if response is not None:
|
|
||||||
return response
|
|
||||||
return self._dispatch_misc_api_route(connection, request, got)
|
|
||||||
|
|
||||||
def _dispatch_misc_api_route(
|
|
||||||
self,
|
|
||||||
connection: Any,
|
|
||||||
request: WsRequest,
|
|
||||||
got: str,
|
|
||||||
) -> Response | None:
|
|
||||||
"""Route small API endpoints that do not belong to a larger route group."""
|
|
||||||
if got == "/api/sessions":
|
if got == "/api/sessions":
|
||||||
return self._handle_sessions_list(request)
|
return self._handle_sessions_list(request)
|
||||||
|
|
||||||
|
if got == "/api/settings":
|
||||||
|
return self._handle_settings(request)
|
||||||
|
|
||||||
if got == "/api/commands":
|
if got == "/api/commands":
|
||||||
return self._handle_commands(request)
|
return self._handle_commands(request)
|
||||||
|
|
||||||
if got == "/api/workspaces":
|
|
||||||
return self._handle_workspaces(connection, request)
|
|
||||||
|
|
||||||
if got == "/api/webui/sidebar-state":
|
if got == "/api/webui/sidebar-state":
|
||||||
return self._handle_webui_sidebar_state(request)
|
return self._handle_webui_sidebar_state(request)
|
||||||
|
|
||||||
if got == "/api/webui/sidebar-state/update":
|
if got == "/api/webui/sidebar-state/update":
|
||||||
return self._handle_webui_sidebar_state_update(request)
|
return self._handle_webui_sidebar_state_update(request)
|
||||||
|
|
||||||
return None
|
if got == "/api/settings/update":
|
||||||
|
return self._handle_settings_update(request)
|
||||||
|
|
||||||
async def _dispatch_settings_api_route(
|
if got == "/api/settings/provider/update":
|
||||||
self,
|
return self._handle_settings_provider_update(request)
|
||||||
request: WsRequest,
|
|
||||||
got: str,
|
if got == "/api/settings/web-search/update":
|
||||||
) -> Response | None:
|
return self._handle_settings_web_search_update(request)
|
||||||
return await self._settings_routes.dispatch(request, got)
|
|
||||||
|
if got == "/api/settings/image-generation/update":
|
||||||
|
return self._handle_settings_image_generation_update(request)
|
||||||
|
|
||||||
def _dispatch_session_api_route(
|
|
||||||
self,
|
|
||||||
request: WsRequest,
|
|
||||||
got: str,
|
|
||||||
) -> Response | None:
|
|
||||||
m = re.match(r"^/api/sessions/([^/]+)/messages$", got)
|
m = re.match(r"^/api/sessions/([^/]+)/messages$", got)
|
||||||
if m:
|
if m:
|
||||||
return self._handle_session_messages(request, m.group(1))
|
return self._handle_session_messages(request, m.group(1))
|
||||||
@@ -777,36 +667,34 @@ class WebSocketChannel(BaseChannel):
|
|||||||
if m:
|
if m:
|
||||||
return self._handle_session_delete(request, m.group(1))
|
return self._handle_session_delete(request, m.group(1))
|
||||||
|
|
||||||
return None
|
# Signed media fetch: ``<sig>`` is an HMAC over ``<payload>``; the
|
||||||
|
# payload decodes to a path inside :func:`get_media_dir`. See
|
||||||
def _dispatch_media_api_route(
|
# :meth:`_sign_media_path` for the inverse direction used to build
|
||||||
self,
|
# these URLs when replaying a session.
|
||||||
request: WsRequest,
|
|
||||||
got: str,
|
|
||||||
) -> Response | None:
|
|
||||||
m = re.match(r"^/api/media/([A-Za-z0-9_-]+)/([A-Za-z0-9_-]+)$", got)
|
m = re.match(r"^/api/media/([A-Za-z0-9_-]+)/([A-Za-z0-9_-]+)$", got)
|
||||||
if m:
|
if m:
|
||||||
return self._handle_media_fetch(m.group(1), m.group(2), request)
|
return self._handle_media_fetch(m.group(1), m.group(2))
|
||||||
|
|
||||||
return None
|
# 4. WebSocket upgrade (the channel's primary purpose). Only run the
|
||||||
|
# handshake gate on requests that actually ask to upgrade; otherwise
|
||||||
def _dispatch_websocket_upgrade(
|
# a bare ``GET /`` from the browser would be rejected as an
|
||||||
self,
|
# unauthorized WS handshake instead of serving the SPA's index.html.
|
||||||
connection: Any,
|
|
||||||
request: WsRequest,
|
|
||||||
got: str,
|
|
||||||
query: dict[str, list[str]],
|
|
||||||
) -> tuple[bool, Any | None]:
|
|
||||||
"""Authorize only real WS upgrade requests for the configured path."""
|
|
||||||
expected_ws = self._expected_path()
|
expected_ws = self._expected_path()
|
||||||
if got != expected_ws or not _is_websocket_upgrade(request):
|
if got == expected_ws and _is_websocket_upgrade(request):
|
||||||
return False, None
|
|
||||||
client_id = _query_first(query, "client_id") or ""
|
client_id = _query_first(query, "client_id") or ""
|
||||||
if len(client_id) > 128:
|
if len(client_id) > 128:
|
||||||
client_id = client_id[:128]
|
client_id = client_id[:128]
|
||||||
if not self.is_allowed(client_id):
|
if not self.is_allowed(client_id):
|
||||||
return True, connection.respond(403, "Forbidden")
|
return connection.respond(403, "Forbidden")
|
||||||
return True, self._authorize_websocket_handshake(connection, query)
|
return self._authorize_websocket_handshake(connection, query)
|
||||||
|
|
||||||
|
# 5. Static SPA serving (only if a build directory was wired in).
|
||||||
|
if self._static_dist_path is not None:
|
||||||
|
response = self._serve_static(got)
|
||||||
|
if response is not None:
|
||||||
|
return response
|
||||||
|
|
||||||
|
return connection.respond(404, "Not Found")
|
||||||
|
|
||||||
# -- HTTP route handlers ------------------------------------------------
|
# -- HTTP route handlers ------------------------------------------------
|
||||||
|
|
||||||
@@ -859,32 +747,15 @@ class WebSocketChannel(BaseChannel):
|
|||||||
# while the REST surface keeps validating the other until TTL expiry.
|
# while the REST surface keeps validating the other until TTL expiry.
|
||||||
self._issued_tokens[token] = expiry
|
self._issued_tokens[token] = expiry
|
||||||
self._api_tokens[token] = expiry
|
self._api_tokens[token] = expiry
|
||||||
ws_url = self._bootstrap_ws_url(request)
|
|
||||||
return _http_json_response(
|
return _http_json_response(
|
||||||
{
|
{
|
||||||
"token": token,
|
"token": token,
|
||||||
"ws_path": self._expected_path(),
|
"ws_path": self._expected_path(),
|
||||||
"ws_url": ws_url,
|
|
||||||
"expires_in": self.config.token_ttl_s,
|
"expires_in": self.config.token_ttl_s,
|
||||||
"model_name": _resolve_bootstrap_model_name(self._runtime_model_name),
|
"model_name": _resolve_bootstrap_model_name(self._runtime_model_name),
|
||||||
"runtime_surface": self._runtime_surface,
|
|
||||||
"runtime_capabilities": self._runtime_capabilities,
|
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
|
||||||
def _bootstrap_ws_url(self, request: Any) -> str:
|
|
||||||
"""Absolute WS URL clients should prefer over a dev-server proxy."""
|
|
||||||
headers = getattr(request, "headers", {}) or {}
|
|
||||||
host = _safe_host_header(_case_insensitive_header(headers, "Host"))
|
|
||||||
if not host:
|
|
||||||
host = _host_for_url(self.config.host, self.config.port)
|
|
||||||
|
|
||||||
proto = _case_insensitive_header(headers, "X-Forwarded-Proto")
|
|
||||||
proto = proto.split(",", 1)[0].strip().lower()
|
|
||||||
secure = proto in {"https", "wss"} or bool(self.config.ssl_certfile.strip())
|
|
||||||
scheme = "wss" if secure else "ws"
|
|
||||||
return f"{scheme}://{host}{self._expected_path()}"
|
|
||||||
|
|
||||||
def _handle_sessions_list(self, request: WsRequest) -> Response:
|
def _handle_sessions_list(self, request: WsRequest) -> Response:
|
||||||
if not self._check_api_token(request):
|
if not self._check_api_token(request):
|
||||||
return _http_error(401, "Unauthorized")
|
return _http_error(401, "Unauthorized")
|
||||||
@@ -903,17 +774,31 @@ class WebSocketChannel(BaseChannel):
|
|||||||
started_at = websocket_turn_wall_started_at(chat_id)
|
started_at = websocket_turn_wall_started_at(chat_id)
|
||||||
if started_at is not None:
|
if started_at is not None:
|
||||||
row["run_started_at"] = started_at
|
row["run_started_at"] = started_at
|
||||||
scope = self._webui_workspaces.scope_for_session_key(key)
|
|
||||||
row["workspace_scope"] = scope.payload()
|
|
||||||
cleaned.append(row)
|
cleaned.append(row)
|
||||||
return _http_json_response({"sessions": cleaned})
|
return _http_json_response({"sessions": cleaned})
|
||||||
|
|
||||||
def _handle_workspaces(self, connection: Any, request: WsRequest) -> Response:
|
def _handle_settings(self, request: WsRequest) -> Response:
|
||||||
if not self._check_api_token(request):
|
if not self._check_api_token(request):
|
||||||
return _http_error(401, "Unauthorized")
|
return _http_error(401, "Unauthorized")
|
||||||
return _http_json_response(
|
return _http_json_response(self._with_settings_restart_state(settings_payload()))
|
||||||
self._webui_workspaces.payload(controls_available=_is_localhost(connection))
|
|
||||||
)
|
def _with_settings_restart_state(
|
||||||
|
self,
|
||||||
|
payload: dict[str, Any],
|
||||||
|
*,
|
||||||
|
section: str | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Keep restart-required state alive for this gateway process."""
|
||||||
|
if section and payload.get("requires_restart"):
|
||||||
|
self._settings_restart_sections.add(section)
|
||||||
|
if self._settings_restart_sections:
|
||||||
|
payload = dict(payload)
|
||||||
|
payload["requires_restart"] = True
|
||||||
|
payload["restart_required_sections"] = sorted(self._settings_restart_sections)
|
||||||
|
else:
|
||||||
|
payload = dict(payload)
|
||||||
|
payload["restart_required_sections"] = []
|
||||||
|
return payload
|
||||||
|
|
||||||
def _handle_commands(self, request: WsRequest) -> Response:
|
def _handle_commands(self, request: WsRequest) -> Response:
|
||||||
if not self._check_api_token(request):
|
if not self._check_api_token(request):
|
||||||
@@ -947,7 +832,47 @@ class WebSocketChannel(BaseChannel):
|
|||||||
return _http_error(500, "failed to write sidebar state")
|
return _http_error(500, "failed to write sidebar state")
|
||||||
return _http_json_response(state)
|
return _http_json_response(state)
|
||||||
|
|
||||||
# -- Session replay, transcript, and signed media ----------------------
|
def _handle_settings_update(self, request: WsRequest) -> Response:
|
||||||
|
if not self._check_api_token(request):
|
||||||
|
return _http_error(401, "Unauthorized")
|
||||||
|
query = _parse_query(request.path)
|
||||||
|
try:
|
||||||
|
payload = update_agent_settings(query)
|
||||||
|
except WebUISettingsError as e:
|
||||||
|
return _http_error(e.status, e.message)
|
||||||
|
return _http_json_response(
|
||||||
|
self._with_settings_restart_state(payload, section="runtime")
|
||||||
|
)
|
||||||
|
|
||||||
|
def _handle_settings_provider_update(self, request: WsRequest) -> Response:
|
||||||
|
if not self._check_api_token(request):
|
||||||
|
return _http_error(401, "Unauthorized")
|
||||||
|
query = _parse_query(request.path)
|
||||||
|
try:
|
||||||
|
payload = update_provider_settings(query)
|
||||||
|
except WebUISettingsError as e:
|
||||||
|
return _http_error(e.status, e.message)
|
||||||
|
return _http_json_response(self._with_settings_restart_state(payload, section="image"))
|
||||||
|
|
||||||
|
def _handle_settings_web_search_update(self, request: WsRequest) -> Response:
|
||||||
|
if not self._check_api_token(request):
|
||||||
|
return _http_error(401, "Unauthorized")
|
||||||
|
query = _parse_query(request.path)
|
||||||
|
try:
|
||||||
|
payload = update_web_search_settings(query)
|
||||||
|
except WebUISettingsError as e:
|
||||||
|
return _http_error(e.status, e.message)
|
||||||
|
return _http_json_response(self._with_settings_restart_state(payload, section="web"))
|
||||||
|
|
||||||
|
def _handle_settings_image_generation_update(self, request: WsRequest) -> Response:
|
||||||
|
if not self._check_api_token(request):
|
||||||
|
return _http_error(401, "Unauthorized")
|
||||||
|
query = _parse_query(request.path)
|
||||||
|
try:
|
||||||
|
payload = update_image_generation_settings(query)
|
||||||
|
except WebUISettingsError as e:
|
||||||
|
return _http_error(e.status, e.message)
|
||||||
|
return _http_json_response(self._with_settings_restart_state(payload, section="image"))
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _is_websocket_channel_session_key(key: str) -> bool:
|
def _is_websocket_channel_session_key(key: str) -> bool:
|
||||||
@@ -987,19 +912,12 @@ class WebSocketChannel(BaseChannel):
|
|||||||
return _http_error(400, "invalid session key")
|
return _http_error(400, "invalid session key")
|
||||||
if not self._is_websocket_channel_session_key(decoded_key):
|
if not self._is_websocket_channel_session_key(decoded_key):
|
||||||
return _http_error(404, "session not found")
|
return _http_error(404, "session not found")
|
||||||
scope = self._webui_workspaces.scope_for_session_key(decoded_key)
|
|
||||||
data = build_webui_thread_response(
|
data = build_webui_thread_response(
|
||||||
decoded_key,
|
decoded_key,
|
||||||
augment_user_media=self._augment_transcript_user_media,
|
augment_user_media=self._augment_transcript_user_media,
|
||||||
augment_assistant_text=lambda text: rewrite_local_markdown_images(
|
|
||||||
text,
|
|
||||||
workspace_path=scope.project_path,
|
|
||||||
sign_path=self._sign_or_stage_media_path,
|
|
||||||
),
|
|
||||||
)
|
)
|
||||||
if data is None:
|
if data is None:
|
||||||
return _http_error(404, "webui thread not found")
|
return _http_error(404, "webui thread not found")
|
||||||
data["workspace_scope"] = scope.payload()
|
|
||||||
return _http_json_response(data)
|
return _http_json_response(data)
|
||||||
|
|
||||||
def _try_append_webui_transcript(self, chat_id: str, wire: dict[str, Any]) -> None:
|
def _try_append_webui_transcript(self, chat_id: str, wire: dict[str, Any]) -> None:
|
||||||
@@ -1043,12 +961,6 @@ class WebSocketChannel(BaseChannel):
|
|||||||
}
|
}
|
||||||
if media:
|
if media:
|
||||||
user_obj["media_paths"] = list(media)
|
user_obj["media_paths"] = list(media)
|
||||||
cli_apps = meta.get("cli_apps")
|
|
||||||
if isinstance(cli_apps, list) and cli_apps:
|
|
||||||
user_obj["cli_apps"] = cli_apps
|
|
||||||
mcp_presets = meta.get("mcp_presets")
|
|
||||||
if isinstance(mcp_presets, list) and mcp_presets:
|
|
||||||
user_obj["mcp_presets"] = mcp_presets
|
|
||||||
self._try_append_webui_transcript(chat_id, user_obj)
|
self._try_append_webui_transcript(chat_id, user_obj)
|
||||||
await super()._handle_message(
|
await super()._handle_message(
|
||||||
sender_id,
|
sender_id,
|
||||||
@@ -1100,11 +1012,16 @@ class WebSocketChannel(BaseChannel):
|
|||||||
be fetched. The returned path is relative to the server origin; the
|
be fetched. The returned path is relative to the server origin; the
|
||||||
client joins it against this server's HTTP origin (same host as WS).
|
client joins it against this server's HTTP origin (same host as WS).
|
||||||
"""
|
"""
|
||||||
return sign_media_path(
|
try:
|
||||||
abs_path,
|
media_root = get_media_dir().resolve()
|
||||||
secret=self._media_secret,
|
rel = abs_path.resolve().relative_to(media_root)
|
||||||
media_dir=lambda channel=None: get_media_dir(channel),
|
except (OSError, ValueError):
|
||||||
)
|
return None
|
||||||
|
payload = _b64url_encode(rel.as_posix().encode("utf-8"))
|
||||||
|
mac = hmac.new(
|
||||||
|
self._media_secret, payload.encode("ascii"), hashlib.sha256
|
||||||
|
).digest()[:16]
|
||||||
|
return f"/api/media/{_b64url_encode(mac)}/{payload}"
|
||||||
|
|
||||||
def _sign_or_stage_media_path(self, path: Path) -> dict[str, str] | None:
|
def _sign_or_stage_media_path(self, path: Path) -> dict[str, str] | None:
|
||||||
"""Return a signed media URL payload for *path*.
|
"""Return a signed media URL payload for *path*.
|
||||||
@@ -1115,34 +1032,70 @@ class WebSocketChannel(BaseChannel):
|
|||||||
can fetch them through the existing signed media route without
|
can fetch them through the existing signed media route without
|
||||||
exposing arbitrary filesystem paths.
|
exposing arbitrary filesystem paths.
|
||||||
"""
|
"""
|
||||||
return sign_or_stage_media_path(
|
signed = self._sign_media_path(path)
|
||||||
path,
|
if signed is not None:
|
||||||
secret=self._media_secret,
|
return {"url": signed, "name": path.name}
|
||||||
media_dir=lambda channel=None: get_media_dir(channel),
|
try:
|
||||||
logger=self.logger,
|
if not path.is_file():
|
||||||
)
|
return None
|
||||||
|
media_dir = get_media_dir("websocket")
|
||||||
|
safe_name = safe_filename(path.name) or "attachment"
|
||||||
|
staged = media_dir / f"{uuid.uuid4().hex[:12]}-{safe_name}"
|
||||||
|
shutil.copyfile(path, staged)
|
||||||
|
except OSError as exc:
|
||||||
|
self.logger.warning("failed to stage outbound media {}: {}", path, exc)
|
||||||
|
return None
|
||||||
|
signed = self._sign_media_path(staged)
|
||||||
|
if signed is None:
|
||||||
|
return None
|
||||||
|
return {"url": signed, "name": path.name}
|
||||||
|
|
||||||
def _rewrite_local_markdown_images(self, text: str) -> str:
|
def _handle_media_fetch(self, sig: str, payload: str) -> Response:
|
||||||
return rewrite_local_markdown_images(
|
|
||||||
text,
|
|
||||||
workspace_path=self._workspace_path,
|
|
||||||
sign_path=self._sign_or_stage_media_path,
|
|
||||||
)
|
|
||||||
|
|
||||||
def _handle_media_fetch(
|
|
||||||
self, sig: str, payload: str, request: WsRequest | None = None
|
|
||||||
) -> Response:
|
|
||||||
"""Serve a single media file previously signed via
|
"""Serve a single media file previously signed via
|
||||||
:meth:`_sign_media_path`. Validates the signature, decodes the
|
:meth:`_sign_media_path`. Validates the signature, decodes the
|
||||||
payload to a relative path, and streams the file bytes with a
|
payload to a relative path, and streams the file bytes with a
|
||||||
long-lived immutable cache header (the URL already encodes the
|
long-lived immutable cache header (the URL already encodes the
|
||||||
file identity, so caches can be aggressive)."""
|
file identity, so caches can be aggressive)."""
|
||||||
return serve_signed_media(
|
try:
|
||||||
sig,
|
provided_mac = _b64url_decode(sig)
|
||||||
payload,
|
except (ValueError, binascii.Error):
|
||||||
secret=self._media_secret,
|
return _http_error(401, "invalid signature")
|
||||||
request=request,
|
expected_mac = hmac.new(
|
||||||
media_dir=lambda channel=None: get_media_dir(channel),
|
self._media_secret, payload.encode("ascii"), hashlib.sha256
|
||||||
|
).digest()[:16]
|
||||||
|
if not hmac.compare_digest(expected_mac, provided_mac):
|
||||||
|
return _http_error(401, "invalid signature")
|
||||||
|
try:
|
||||||
|
rel_bytes = _b64url_decode(payload)
|
||||||
|
rel_str = rel_bytes.decode("utf-8")
|
||||||
|
except (ValueError, binascii.Error, UnicodeDecodeError):
|
||||||
|
return _http_error(400, "invalid payload")
|
||||||
|
# An attacker who somehow bypassed the HMAC check would still need
|
||||||
|
# the resolved path to escape the media root; guard defensively.
|
||||||
|
try:
|
||||||
|
media_root = get_media_dir().resolve()
|
||||||
|
candidate = (media_root / rel_str).resolve()
|
||||||
|
candidate.relative_to(media_root)
|
||||||
|
except (OSError, ValueError):
|
||||||
|
return _http_error(404, "not found")
|
||||||
|
if not candidate.is_file():
|
||||||
|
return _http_error(404, "not found")
|
||||||
|
try:
|
||||||
|
body = candidate.read_bytes()
|
||||||
|
except OSError:
|
||||||
|
return _http_error(500, "read error")
|
||||||
|
mime, _ = mimetypes.guess_type(candidate.name)
|
||||||
|
if mime not in _MEDIA_ALLOWED_MIMES:
|
||||||
|
mime = "application/octet-stream"
|
||||||
|
return _http_response(
|
||||||
|
body,
|
||||||
|
content_type=mime,
|
||||||
|
extra_headers=[
|
||||||
|
("Cache-Control", "private, max-age=31536000, immutable"),
|
||||||
|
# Paired with the MIME whitelist above: prevents browsers from
|
||||||
|
# MIME-sniffing an octet-stream fallback into executable HTML.
|
||||||
|
("X-Content-Type-Options", "nosniff"),
|
||||||
|
],
|
||||||
)
|
)
|
||||||
|
|
||||||
def _handle_session_delete(self, request: WsRequest, key: str) -> Response:
|
def _handle_session_delete(self, request: WsRequest, key: str) -> Response:
|
||||||
@@ -1161,8 +1114,6 @@ class WebSocketChannel(BaseChannel):
|
|||||||
delete_webui_thread(decoded_key)
|
delete_webui_thread(decoded_key)
|
||||||
return _http_json_response({"deleted": bool(deleted)})
|
return _http_json_response({"deleted": bool(deleted)})
|
||||||
|
|
||||||
# -- Static files and WebSocket handshake ------------------------------
|
|
||||||
|
|
||||||
def _serve_static(self, request_path: str) -> Response | None:
|
def _serve_static(self, request_path: str) -> Response | None:
|
||||||
"""Resolve *request_path* against the built SPA directory; SPA fallback to index.html."""
|
"""Resolve *request_path* against the built SPA directory; SPA fallback to index.html."""
|
||||||
assert self._static_dist_path is not None
|
assert self._static_dist_path is not None
|
||||||
@@ -1227,8 +1178,6 @@ class WebSocketChannel(BaseChannel):
|
|||||||
self._take_issued_token_if_valid(supplied)
|
self._take_issued_token_if_valid(supplied)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
# -- Server lifecycle and connection ingress ---------------------------
|
|
||||||
|
|
||||||
async def start(self) -> None:
|
async def start(self) -> None:
|
||||||
from nanobot.utils.logging_bridge import redirect_lib_logging
|
from nanobot.utils.logging_bridge import redirect_lib_logging
|
||||||
|
|
||||||
@@ -1250,45 +1199,23 @@ class WebSocketChannel(BaseChannel):
|
|||||||
await self._connection_loop(connection)
|
await self._connection_loop(connection)
|
||||||
|
|
||||||
self.logger.info(
|
self.logger.info(
|
||||||
"WebSocket server listening on {}",
|
"WebSocket server listening on {}://{}:{}{}",
|
||||||
(
|
scheme,
|
||||||
f"unix:{self.config.unix_socket_path}{self.config.path}"
|
self.config.host,
|
||||||
if self.config.unix_socket_path
|
self.config.port,
|
||||||
else f"{scheme}://{self.config.host}:{self.config.port}{self.config.path}"
|
self.config.path,
|
||||||
),
|
|
||||||
)
|
)
|
||||||
if self.config.token_issue_path:
|
if self.config.token_issue_path:
|
||||||
self.logger.info(
|
self.logger.info(
|
||||||
"WebSocket token issue route: {}",
|
"WebSocket token issue route: {}://{}:{}{}",
|
||||||
(
|
scheme,
|
||||||
f"unix:{self.config.unix_socket_path}{_normalize_config_path(self.config.token_issue_path)}"
|
self.config.host,
|
||||||
if self.config.unix_socket_path
|
self.config.port,
|
||||||
else (
|
_normalize_config_path(self.config.token_issue_path),
|
||||||
f"{scheme}://{self.config.host}:{self.config.port}"
|
|
||||||
f"{_normalize_config_path(self.config.token_issue_path)}"
|
|
||||||
)
|
|
||||||
),
|
|
||||||
)
|
)
|
||||||
|
|
||||||
async def runner() -> None:
|
async def runner() -> None:
|
||||||
socket_path = self.config.unix_socket_path
|
async with serve(
|
||||||
if socket_path:
|
|
||||||
path_obj = Path(socket_path)
|
|
||||||
path_obj.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
with suppress(FileNotFoundError):
|
|
||||||
path_obj.unlink()
|
|
||||||
server = await unix_serve(
|
|
||||||
handler,
|
|
||||||
socket_path,
|
|
||||||
process_request=process_request,
|
|
||||||
max_size=self.config.max_message_bytes,
|
|
||||||
ping_interval=self.config.ping_interval_s,
|
|
||||||
ping_timeout=self.config.ping_timeout_s,
|
|
||||||
)
|
|
||||||
with suppress(OSError):
|
|
||||||
path_obj.chmod(0o600)
|
|
||||||
else:
|
|
||||||
server = await serve(
|
|
||||||
handler,
|
handler,
|
||||||
self.config.host,
|
self.config.host,
|
||||||
self.config.port,
|
self.config.port,
|
||||||
@@ -1297,16 +1224,9 @@ class WebSocketChannel(BaseChannel):
|
|||||||
ping_interval=self.config.ping_interval_s,
|
ping_interval=self.config.ping_interval_s,
|
||||||
ping_timeout=self.config.ping_timeout_s,
|
ping_timeout=self.config.ping_timeout_s,
|
||||||
ssl=ssl_context,
|
ssl=ssl_context,
|
||||||
)
|
):
|
||||||
try:
|
|
||||||
assert self._stop_event is not None
|
assert self._stop_event is not None
|
||||||
await self._stop_event.wait()
|
await self._stop_event.wait()
|
||||||
finally:
|
|
||||||
server.close()
|
|
||||||
await server.wait_closed()
|
|
||||||
if socket_path:
|
|
||||||
with suppress(FileNotFoundError):
|
|
||||||
Path(socket_path).unlink()
|
|
||||||
|
|
||||||
self._server_task = asyncio.create_task(runner())
|
self._server_task = asyncio.create_task(runner())
|
||||||
await self._server_task
|
await self._server_task
|
||||||
@@ -1372,8 +1292,6 @@ class WebSocketChannel(BaseChannel):
|
|||||||
finally:
|
finally:
|
||||||
self._cleanup_connection(connection)
|
self._cleanup_connection(connection)
|
||||||
|
|
||||||
# -- Inbound WebSocket envelopes ---------------------------------------
|
|
||||||
|
|
||||||
def _save_envelope_media(
|
def _save_envelope_media(
|
||||||
self,
|
self,
|
||||||
media: list[Any],
|
media: list[Any],
|
||||||
@@ -1452,25 +1370,8 @@ class WebSocketChannel(BaseChannel):
|
|||||||
t = envelope.get("type")
|
t = envelope.get("type")
|
||||||
if t == "new_chat":
|
if t == "new_chat":
|
||||||
new_id = str(uuid.uuid4())
|
new_id = str(uuid.uuid4())
|
||||||
scope = await self._workspace_scope_or_error(
|
|
||||||
connection,
|
|
||||||
lambda: self._webui_workspaces.scope_for_new_chat(
|
|
||||||
envelope,
|
|
||||||
controls_available=_is_localhost(connection),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
if scope is None:
|
|
||||||
return
|
|
||||||
self._webui_workspaces.persist_scope(new_id, scope)
|
|
||||||
self._attach(connection, new_id)
|
self._attach(connection, new_id)
|
||||||
await self._send_event(connection, "attached", chat_id=new_id)
|
await self._send_event(connection, "attached", chat_id=new_id)
|
||||||
await self._send_event(
|
|
||||||
connection,
|
|
||||||
"session_updated",
|
|
||||||
chat_id=new_id,
|
|
||||||
scope="metadata",
|
|
||||||
workspace_scope=scope.payload(),
|
|
||||||
)
|
|
||||||
await self._hydrate_after_subscribe(new_id)
|
await self._hydrate_after_subscribe(new_id)
|
||||||
return
|
return
|
||||||
if t == "attach":
|
if t == "attach":
|
||||||
@@ -1482,32 +1383,6 @@ class WebSocketChannel(BaseChannel):
|
|||||||
await self._send_event(connection, "attached", chat_id=cid)
|
await self._send_event(connection, "attached", chat_id=cid)
|
||||||
await self._hydrate_after_subscribe(cid)
|
await self._hydrate_after_subscribe(cid)
|
||||||
return
|
return
|
||||||
if t == "set_workspace_scope":
|
|
||||||
cid = envelope.get("chat_id")
|
|
||||||
if not _is_valid_chat_id(cid):
|
|
||||||
await self._send_event(connection, "error", detail="invalid chat_id")
|
|
||||||
return
|
|
||||||
scope = await self._workspace_scope_or_error(
|
|
||||||
connection,
|
|
||||||
lambda: self._webui_workspaces.scope_for_set_request(
|
|
||||||
envelope,
|
|
||||||
chat_id=cid,
|
|
||||||
chat_running=websocket_turn_wall_started_at(cid) is not None,
|
|
||||||
controls_available=_is_localhost(connection),
|
|
||||||
),
|
|
||||||
chat_id=cid,
|
|
||||||
)
|
|
||||||
if scope is None:
|
|
||||||
return
|
|
||||||
self._webui_workspaces.persist_scope(cid, scope)
|
|
||||||
await self._send_event(
|
|
||||||
connection,
|
|
||||||
"session_updated",
|
|
||||||
chat_id=cid,
|
|
||||||
scope="metadata",
|
|
||||||
workspace_scope=scope.payload(),
|
|
||||||
)
|
|
||||||
return
|
|
||||||
if t == "message":
|
if t == "message":
|
||||||
cid = envelope.get("chat_id")
|
cid = envelope.get("chat_id")
|
||||||
content = envelope.get("content")
|
content = envelope.get("content")
|
||||||
@@ -1539,18 +1414,6 @@ class WebSocketChannel(BaseChannel):
|
|||||||
if not content.strip() and not media_paths:
|
if not content.strip() and not media_paths:
|
||||||
await self._send_event(connection, "error", detail="missing content")
|
await self._send_event(connection, "error", detail="missing content")
|
||||||
return
|
return
|
||||||
scope = await self._workspace_scope_or_error(
|
|
||||||
connection,
|
|
||||||
lambda: self._webui_workspaces.scope_for_message(
|
|
||||||
envelope,
|
|
||||||
chat_id=cid,
|
|
||||||
chat_running=websocket_turn_wall_started_at(cid) is not None,
|
|
||||||
controls_available=_is_localhost(connection),
|
|
||||||
),
|
|
||||||
chat_id=cid,
|
|
||||||
)
|
|
||||||
if scope is None:
|
|
||||||
return
|
|
||||||
|
|
||||||
# Auto-attach on first use so clients can one-shot without a separate attach.
|
# Auto-attach on first use so clients can one-shot without a separate attach.
|
||||||
self._attach(connection, cid)
|
self._attach(connection, cid)
|
||||||
@@ -1558,14 +1421,6 @@ class WebSocketChannel(BaseChannel):
|
|||||||
metadata: dict[str, Any] = {"remote": getattr(connection, "remote_address", None)}
|
metadata: dict[str, Any] = {"remote": getattr(connection, "remote_address", None)}
|
||||||
if envelope.get("webui") is True:
|
if envelope.get("webui") is True:
|
||||||
metadata["webui"] = True
|
metadata["webui"] = True
|
||||||
cli_apps = normalize_cli_app_mentions(envelope.get("cli_apps"))
|
|
||||||
if cli_apps:
|
|
||||||
metadata["cli_apps"] = cli_apps
|
|
||||||
mcp_presets = normalize_mcp_preset_mentions(envelope.get("mcp_presets"))
|
|
||||||
if mcp_presets:
|
|
||||||
metadata["mcp_presets"] = mcp_presets
|
|
||||||
metadata[WORKSPACE_SCOPE_METADATA_KEY] = scope.metadata()
|
|
||||||
self._webui_workspaces.persist_scope(cid, scope)
|
|
||||||
image_generation = envelope.get("image_generation")
|
image_generation = envelope.get("image_generation")
|
||||||
if isinstance(image_generation, dict) and image_generation.get("enabled") is True:
|
if isinstance(image_generation, dict) and image_generation.get("enabled") is True:
|
||||||
aspect_ratio = image_generation.get("aspect_ratio")
|
aspect_ratio = image_generation.get("aspect_ratio")
|
||||||
@@ -1584,27 +1439,6 @@ class WebSocketChannel(BaseChannel):
|
|||||||
return
|
return
|
||||||
await self._send_event(connection, "error", detail=f"unknown type: {t!r}")
|
await self._send_event(connection, "error", detail=f"unknown type: {t!r}")
|
||||||
|
|
||||||
async def _workspace_scope_or_error(
|
|
||||||
self,
|
|
||||||
connection: Any,
|
|
||||||
resolver: Callable[[], Any],
|
|
||||||
*,
|
|
||||||
chat_id: str | None = None,
|
|
||||||
) -> Any | None:
|
|
||||||
try:
|
|
||||||
return resolver()
|
|
||||||
except WorkspaceScopeError as exc:
|
|
||||||
await self._send_event(
|
|
||||||
connection,
|
|
||||||
"error",
|
|
||||||
detail="workspace_scope_rejected",
|
|
||||||
reason=exc.message,
|
|
||||||
**({"chat_id": chat_id} if chat_id else {}),
|
|
||||||
)
|
|
||||||
return None
|
|
||||||
|
|
||||||
# -- Outbound WebSocket events -----------------------------------------
|
|
||||||
|
|
||||||
async def stop(self) -> None:
|
async def stop(self) -> None:
|
||||||
if not self._running:
|
if not self._running:
|
||||||
return
|
return
|
||||||
@@ -1698,11 +1532,10 @@ class WebSocketChannel(BaseChannel):
|
|||||||
await self._safe_send_to(connection, raw, label=" ")
|
await self._safe_send_to(connection, raw, label=" ")
|
||||||
return
|
return
|
||||||
text = msg.content
|
text = msg.content
|
||||||
wire_text = self._rewrite_local_markdown_images(text)
|
|
||||||
payload: dict[str, Any] = {
|
payload: dict[str, Any] = {
|
||||||
"event": "message",
|
"event": "message",
|
||||||
"chat_id": msg.chat_id,
|
"chat_id": msg.chat_id,
|
||||||
"text": wire_text,
|
"text": text,
|
||||||
}
|
}
|
||||||
if msg.media:
|
if msg.media:
|
||||||
payload["media"] = msg.media
|
payload["media"] = msg.media
|
||||||
@@ -1730,9 +1563,7 @@ class WebSocketChannel(BaseChannel):
|
|||||||
payload["kind"] = "tool_hint"
|
payload["kind"] = "tool_hint"
|
||||||
elif msg.metadata.get("_progress"):
|
elif msg.metadata.get("_progress"):
|
||||||
payload["kind"] = "progress"
|
payload["kind"] = "progress"
|
||||||
transcript_payload = dict(payload)
|
self._try_append_webui_transcript(msg.chat_id, payload)
|
||||||
transcript_payload["text"] = text
|
|
||||||
self._try_append_webui_transcript(msg.chat_id, transcript_payload)
|
|
||||||
raw = json.dumps(payload, ensure_ascii=False)
|
raw = json.dumps(payload, ensure_ascii=False)
|
||||||
for connection in conns:
|
for connection in conns:
|
||||||
await self._safe_send_to(connection, raw, label=" ")
|
await self._safe_send_to(connection, raw, label=" ")
|
||||||
@@ -1797,23 +1628,14 @@ class WebSocketChannel(BaseChannel):
|
|||||||
if not conns:
|
if not conns:
|
||||||
return
|
return
|
||||||
meta = metadata or {}
|
meta = metadata or {}
|
||||||
stream_key = (chat_id, str(meta.get("_stream_id") or ""))
|
|
||||||
if meta.get("_stream_end"):
|
if meta.get("_stream_end"):
|
||||||
body: dict[str, Any] = {"event": "stream_end", "chat_id": chat_id}
|
body: dict[str, Any] = {"event": "stream_end", "chat_id": chat_id}
|
||||||
buffered = self._stream_text_buffers.pop(stream_key, [])
|
|
||||||
if delta:
|
|
||||||
buffered.append(delta)
|
|
||||||
full_text = "".join(buffered)
|
|
||||||
rewritten = self._rewrite_local_markdown_images(full_text)
|
|
||||||
if rewritten != full_text:
|
|
||||||
body["text"] = rewritten
|
|
||||||
else:
|
else:
|
||||||
body = {
|
body = {
|
||||||
"event": "delta",
|
"event": "delta",
|
||||||
"chat_id": chat_id,
|
"chat_id": chat_id,
|
||||||
"text": delta,
|
"text": delta,
|
||||||
}
|
}
|
||||||
self._stream_text_buffers.setdefault(stream_key, []).append(delta)
|
|
||||||
if meta.get("_stream_id") is not None:
|
if meta.get("_stream_id") is not None:
|
||||||
body["stream_id"] = meta["_stream_id"]
|
body["stream_id"] = meta["_stream_id"]
|
||||||
self._try_append_webui_transcript(chat_id, body)
|
self._try_append_webui_transcript(chat_id, body)
|
||||||
|
|||||||
+6
-163
@@ -79,12 +79,6 @@ BASE_INFO: dict[str, str] = {"channel_version": WEIXIN_CHANNEL_VERSION}
|
|||||||
ERRCODE_SESSION_EXPIRED = -14
|
ERRCODE_SESSION_EXPIRED = -14
|
||||||
SESSION_PAUSE_DURATION_S = 60 * 60
|
SESSION_PAUSE_DURATION_S = 60 * 60
|
||||||
|
|
||||||
# iLink context_token is observed to expire server-side after ~90-160s of
|
|
||||||
# agent inactivity (openclaw/openclaw#61174). Proactively refresh before
|
|
||||||
# sending if the cached token is older than this threshold.
|
|
||||||
CONTEXT_TOKEN_MAX_AGE_S = 60
|
|
||||||
|
|
||||||
|
|
||||||
# Retry constants (matching the reference plugin's monitor.ts)
|
# Retry constants (matching the reference plugin's monitor.ts)
|
||||||
MAX_CONSECUTIVE_FAILURES = 3
|
MAX_CONSECUTIVE_FAILURES = 3
|
||||||
BACKOFF_DELAY_S = 30
|
BACKOFF_DELAY_S = 30
|
||||||
@@ -165,8 +159,6 @@ class WeixinChannel(BaseChannel):
|
|||||||
self._session_pause_until: float = 0.0
|
self._session_pause_until: float = 0.0
|
||||||
self._typing_tasks: dict[str, asyncio.Task] = {}
|
self._typing_tasks: dict[str, asyncio.Task] = {}
|
||||||
self._typing_tickets: dict[str, dict[str, Any]] = {}
|
self._typing_tickets: dict[str, dict[str, Any]] = {}
|
||||||
self._context_token_at: dict[str, float] = {}
|
|
||||||
self._pending_tool_hints: dict[str, list[str]] = {}
|
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# State persistence
|
# State persistence
|
||||||
@@ -494,7 +486,6 @@ class WeixinChannel(BaseChannel):
|
|||||||
except Exception:
|
except Exception:
|
||||||
if not self._running:
|
if not self._running:
|
||||||
break
|
break
|
||||||
self.logger.exception("WeChat poll loop error")
|
|
||||||
consecutive_failures += 1
|
consecutive_failures += 1
|
||||||
if consecutive_failures >= MAX_CONSECUTIVE_FAILURES:
|
if consecutive_failures >= MAX_CONSECUTIVE_FAILURES:
|
||||||
consecutive_failures = 0
|
consecutive_failures = 0
|
||||||
@@ -504,7 +495,6 @@ class WeixinChannel(BaseChannel):
|
|||||||
|
|
||||||
async def stop(self) -> None:
|
async def stop(self) -> None:
|
||||||
self._running = False
|
self._running = False
|
||||||
self._pending_tool_hints.clear()
|
|
||||||
if self._poll_task and not self._poll_task.done():
|
if self._poll_task and not self._poll_task.done():
|
||||||
self._poll_task.cancel()
|
self._poll_task.cancel()
|
||||||
for chat_id in list(self._typing_tasks):
|
for chat_id in list(self._typing_tasks):
|
||||||
@@ -555,7 +545,6 @@ class WeixinChannel(BaseChannel):
|
|||||||
# Check for API-level errors (monitor.ts checks both ret and errcode)
|
# Check for API-level errors (monitor.ts checks both ret and errcode)
|
||||||
ret = data.get("ret", 0)
|
ret = data.get("ret", 0)
|
||||||
errcode = data.get("errcode", 0)
|
errcode = data.get("errcode", 0)
|
||||||
|
|
||||||
is_error = (ret is not None and ret != 0) or (errcode is not None and errcode != 0)
|
is_error = (ret is not None and ret != 0) or (errcode is not None and errcode != 0)
|
||||||
|
|
||||||
if is_error:
|
if is_error:
|
||||||
@@ -586,10 +575,8 @@ class WeixinChannel(BaseChannel):
|
|||||||
# Process messages (WeixinMessage[] from types.ts)
|
# Process messages (WeixinMessage[] from types.ts)
|
||||||
msgs: list[dict] = data.get("msgs", []) or []
|
msgs: list[dict] = data.get("msgs", []) or []
|
||||||
for msg in msgs:
|
for msg in msgs:
|
||||||
try:
|
with suppress(Exception):
|
||||||
await self._process_message(msg)
|
await self._process_message(msg)
|
||||||
except Exception:
|
|
||||||
self.logger.exception("Failed to process WeChat message")
|
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# Inbound message processing (matches inbound.ts + process-message.ts)
|
# Inbound message processing (matches inbound.ts + process-message.ts)
|
||||||
@@ -623,7 +610,6 @@ class WeixinChannel(BaseChannel):
|
|||||||
ctx_token = msg.get("context_token", "")
|
ctx_token = msg.get("context_token", "")
|
||||||
if ctx_token:
|
if ctx_token:
|
||||||
self._context_tokens[from_user_id] = ctx_token
|
self._context_tokens[from_user_id] = ctx_token
|
||||||
self._context_token_at[from_user_id] = time.time()
|
|
||||||
self._save_state()
|
self._save_state()
|
||||||
|
|
||||||
# Parse item_list (WeixinMessage.item_list — types.ts:161)
|
# Parse item_list (WeixinMessage.item_list — types.ts:161)
|
||||||
@@ -929,99 +915,6 @@ class WeixinChannel(BaseChannel):
|
|||||||
}
|
}
|
||||||
return ""
|
return ""
|
||||||
|
|
||||||
async def _refresh_context_token_if_stale(
|
|
||||||
self, chat_id: str, context_token: str
|
|
||||||
) -> str:
|
|
||||||
"""Return a fresh context_token if the cached one is too old.
|
|
||||||
|
|
||||||
iLink context_token expires server-side after a short idle period
|
|
||||||
(empirically ~90s). Proactively refreshing before sending prevents
|
|
||||||
silent message loss on long agent turns or cron pushes.
|
|
||||||
"""
|
|
||||||
if not context_token:
|
|
||||||
return context_token
|
|
||||||
|
|
||||||
now = time.time()
|
|
||||||
cached_at = self._context_token_at.get(chat_id, 0)
|
|
||||||
age = now - cached_at
|
|
||||||
|
|
||||||
if age < CONTEXT_TOKEN_MAX_AGE_S:
|
|
||||||
return context_token
|
|
||||||
|
|
||||||
self.logger.debug(
|
|
||||||
"WeChat context_token for {} is {:.0f}s old; refreshing via getconfig",
|
|
||||||
chat_id,
|
|
||||||
age,
|
|
||||||
)
|
|
||||||
|
|
||||||
body: dict[str, Any] = {
|
|
||||||
"ilink_user_id": chat_id,
|
|
||||||
"context_token": context_token,
|
|
||||||
"base_info": BASE_INFO,
|
|
||||||
}
|
|
||||||
try:
|
|
||||||
data = await self._api_post("ilink/bot/getconfig", body)
|
|
||||||
except Exception as e:
|
|
||||||
self.logger.warning("WeChat getconfig failed for {}: {}", chat_id, e)
|
|
||||||
return context_token
|
|
||||||
|
|
||||||
if data.get("ret", 0) != 0:
|
|
||||||
self.logger.warning(
|
|
||||||
"WeChat getconfig returned ret={} for {}: {}",
|
|
||||||
data.get("ret"),
|
|
||||||
chat_id,
|
|
||||||
data.get("errmsg", ""),
|
|
||||||
)
|
|
||||||
return context_token
|
|
||||||
|
|
||||||
new_token = str(data.get("context_token", "") or "")
|
|
||||||
if new_token and new_token != context_token:
|
|
||||||
self.logger.info(
|
|
||||||
"WeChat context_token refreshed for {} (age {:.0f}s -> fresh)",
|
|
||||||
chat_id,
|
|
||||||
age,
|
|
||||||
)
|
|
||||||
self._context_tokens[chat_id] = new_token
|
|
||||||
self._context_token_at[chat_id] = now
|
|
||||||
self._save_state()
|
|
||||||
return new_token
|
|
||||||
|
|
||||||
return context_token
|
|
||||||
|
|
||||||
async def _flush_tool_hints(self, chat_id: str) -> None:
|
|
||||||
"""Send any buffered tool hints for *chat_id* as a single message.
|
|
||||||
|
|
||||||
Tool hints are coalesced to reduce message count and avoid hitting the
|
|
||||||
WeChat iLink rate limit (~7 msgs / 5 min). Failures are logged but
|
|
||||||
not raised so that the main message send is never blocked.
|
|
||||||
"""
|
|
||||||
hints = self._pending_tool_hints.pop(chat_id, None)
|
|
||||||
if not hints:
|
|
||||||
return
|
|
||||||
|
|
||||||
self.logger.info(
|
|
||||||
"Flushing {} buffered tool hint(s) for {}",
|
|
||||||
len(hints),
|
|
||||||
chat_id,
|
|
||||||
)
|
|
||||||
|
|
||||||
ctx_token = self._context_tokens.get(chat_id, "")
|
|
||||||
ctx_token = await self._refresh_context_token_if_stale(chat_id, ctx_token)
|
|
||||||
if not ctx_token:
|
|
||||||
self.logger.warning(
|
|
||||||
"Dropped {} buffered tool hint(s) for {}: no context_token",
|
|
||||||
len(hints),
|
|
||||||
chat_id,
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
try:
|
|
||||||
await self._send_text(chat_id, "\n\n".join(hints), ctx_token)
|
|
||||||
except Exception:
|
|
||||||
self.logger.exception(
|
|
||||||
"Failed to flush buffered tool hints for {}", chat_id
|
|
||||||
)
|
|
||||||
|
|
||||||
async def _send_typing(self, user_id: str, typing_ticket: str, status: int) -> None:
|
async def _send_typing(self, user_id: str, typing_ticket: str, status: int) -> None:
|
||||||
"""Best-effort sendtyping wrapper."""
|
"""Best-effort sendtyping wrapper."""
|
||||||
if not typing_ticket:
|
if not typing_ticket:
|
||||||
@@ -1051,47 +944,11 @@ class WeixinChannel(BaseChannel):
|
|||||||
self._assert_session_active()
|
self._assert_session_active()
|
||||||
|
|
||||||
is_progress = bool((msg.metadata or {}).get("_progress", False))
|
is_progress = bool((msg.metadata or {}).get("_progress", False))
|
||||||
|
|
||||||
# Buffer tool hints to coalesce consecutive ones and avoid burning
|
|
||||||
# WeChat iLink rate-limit quota (~7 msgs / 5 min).
|
|
||||||
if is_progress and (msg.metadata or {}).get("_tool_hint"):
|
|
||||||
if not self.send_tool_hints:
|
|
||||||
return
|
|
||||||
self._pending_tool_hints.setdefault(msg.chat_id, []).append(msg.content)
|
|
||||||
self.logger.debug(
|
|
||||||
"Buffered tool hint for {} (count={})",
|
|
||||||
msg.chat_id,
|
|
||||||
len(self._pending_tool_hints[msg.chat_id]),
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
# Reasoning deltas are invisible in WeChat (there is no reasoning
|
|
||||||
# UI). Skip them entirely — do not send and do not flush buffer.
|
|
||||||
if is_progress and (msg.metadata or {}).get("_reasoning_delta"):
|
|
||||||
self.logger.debug(
|
|
||||||
"Dropped invisible reasoning delta for {}", msg.chat_id
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
content = msg.content.strip()
|
|
||||||
|
|
||||||
# Empty progress messages (e.g. after_iteration tool_events) must
|
|
||||||
# NOT act as separators — they have no visible content.
|
|
||||||
if is_progress and not content and not (msg.media or []):
|
|
||||||
self.logger.debug(
|
|
||||||
"Skipped empty progress message for {} (no visible content)",
|
|
||||||
msg.chat_id,
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
# Flush buffered hints before sending any visible message.
|
|
||||||
await self._flush_tool_hints(msg.chat_id)
|
|
||||||
|
|
||||||
if not is_progress:
|
if not is_progress:
|
||||||
await self._stop_typing(msg.chat_id, clear_remote=True)
|
await self._stop_typing(msg.chat_id, clear_remote=True)
|
||||||
|
|
||||||
|
content = msg.content.strip()
|
||||||
ctx_token = self._context_tokens.get(msg.chat_id, "")
|
ctx_token = self._context_tokens.get(msg.chat_id, "")
|
||||||
ctx_token = await self._refresh_context_token_if_stale(msg.chat_id, ctx_token)
|
|
||||||
if not ctx_token:
|
if not ctx_token:
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
f"WeChat context_token missing for chat_id={msg.chat_id}, cannot send"
|
f"WeChat context_token missing for chat_id={msg.chat_id}, cannot send"
|
||||||
@@ -1180,18 +1037,6 @@ class WeixinChannel(BaseChannel):
|
|||||||
with suppress(Exception):
|
with suppress(Exception):
|
||||||
await self._send_typing(msg.chat_id, typing_ticket, TYPING_STATUS_CANCEL)
|
await self._send_typing(msg.chat_id, typing_ticket, TYPING_STATUS_CANCEL)
|
||||||
|
|
||||||
async def send_delta(
|
|
||||||
self, chat_id: str, delta: str, metadata: dict[str, Any] | None = None
|
|
||||||
) -> None:
|
|
||||||
"""Weixin iLink does not support native streaming deltas.
|
|
||||||
|
|
||||||
We only hook ``_stream_end`` so buffered tool hints are flushed even
|
|
||||||
when the final answer carries the ``_streamed`` flag and bypasses
|
|
||||||
:meth:`send`.
|
|
||||||
"""
|
|
||||||
if metadata and metadata.get("_stream_end"):
|
|
||||||
await self._flush_tool_hints(chat_id)
|
|
||||||
|
|
||||||
async def _start_typing(self, chat_id: str, context_token: str = "") -> None:
|
async def _start_typing(self, chat_id: str, context_token: str = "") -> None:
|
||||||
"""Start typing indicator immediately when a message is received."""
|
"""Start typing indicator immediately when a message is received."""
|
||||||
if not self._client or not self._token or not chat_id:
|
if not self._client or not self._token or not chat_id:
|
||||||
@@ -1275,11 +1120,10 @@ class WeixinChannel(BaseChannel):
|
|||||||
}
|
}
|
||||||
|
|
||||||
data = await self._api_post("ilink/bot/sendmessage", body)
|
data = await self._api_post("ilink/bot/sendmessage", body)
|
||||||
ret = data.get("ret", 0)
|
|
||||||
errcode = data.get("errcode", 0)
|
errcode = data.get("errcode", 0)
|
||||||
if (ret is not None and ret != 0) or (errcode is not None and errcode != 0):
|
if errcode and errcode != 0:
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
f"WeChat send text error (ret={ret}, errcode={errcode}): {data.get('errmsg', '')}"
|
f"WeChat send text error (code {errcode}): {data.get('errmsg', '')}"
|
||||||
)
|
)
|
||||||
|
|
||||||
async def _send_media_file(
|
async def _send_media_file(
|
||||||
@@ -1426,11 +1270,10 @@ class WeixinChannel(BaseChannel):
|
|||||||
}
|
}
|
||||||
|
|
||||||
data = await self._api_post("ilink/bot/sendmessage", body)
|
data = await self._api_post("ilink/bot/sendmessage", body)
|
||||||
ret = data.get("ret", 0)
|
|
||||||
errcode = data.get("errcode", 0)
|
errcode = data.get("errcode", 0)
|
||||||
if (ret is not None and ret != 0) or (errcode is not None and errcode != 0):
|
if errcode and errcode != 0:
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
f"WeChat send media error (ret={ret}, errcode={errcode}): {data.get('errmsg', '')}"
|
f"WeChat send media error (code {errcode}): {data.get('errmsg', '')}"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+236
-240
@@ -1,13 +1,14 @@
|
|||||||
"""CLI commands for nanobot."""
|
"""CLI commands for nanobot."""
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import functools
|
import json
|
||||||
import os
|
import os
|
||||||
import select
|
import select
|
||||||
import signal
|
import signal
|
||||||
import sys
|
import sys
|
||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
from contextlib import nullcontext, suppress
|
from contextlib import nullcontext, suppress
|
||||||
|
from inspect import signature
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
@@ -76,7 +77,6 @@ class SafeFileHistory(FileHistory):
|
|||||||
from nanobot.cli.stream import StreamRenderer, ThinkingSpinner
|
from nanobot.cli.stream import StreamRenderer, ThinkingSpinner
|
||||||
from nanobot.config.paths import get_workspace_path, is_default_workspace
|
from nanobot.config.paths import get_workspace_path, is_default_workspace
|
||||||
from nanobot.config.schema import Config
|
from nanobot.config.schema import Config
|
||||||
from nanobot.utils.evaluator import evaluate_response
|
|
||||||
from nanobot.utils.helpers import sync_workspace_templates
|
from nanobot.utils.helpers import sync_workspace_templates
|
||||||
from nanobot.utils.restart import (
|
from nanobot.utils.restart import (
|
||||||
consume_restart_notice_from_env,
|
consume_restart_notice_from_env,
|
||||||
@@ -96,20 +96,6 @@ EXIT_COMMANDS = {"exit", "quit", "/exit", "/quit", ":q"}
|
|||||||
_REASONING_SENTENCE_ENDINGS = (".", "!", "?", "。", "!", "?")
|
_REASONING_SENTENCE_ENDINGS = (".", "!", "?", "。", "!", "?")
|
||||||
_REASONING_FLUSH_CHARS = 60
|
_REASONING_FLUSH_CHARS = 60
|
||||||
|
|
||||||
_HEARTBEAT_PREAMBLE = (
|
|
||||||
"[Your response will be delivered directly to the user's messaging app. "
|
|
||||||
"Output ONLY the final user-facing message. Never reference internal "
|
|
||||||
"files (HEARTBEAT.md, AWARENESS.md, etc.), your instructions, or your "
|
|
||||||
"decision process. If nothing needs reporting, respond with a brief "
|
|
||||||
"no-op status and nothing else.]\n\n"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@functools.lru_cache(maxsize=None)
|
|
||||||
def _heartbeat_template() -> str | None:
|
|
||||||
from nanobot.utils.helpers import load_bundled_template
|
|
||||||
return load_bundled_template("HEARTBEAT.md")
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# CLI input: prompt_toolkit for editing, paste, history, and display
|
# CLI input: prompt_toolkit for editing, paste, history, and display
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -720,144 +706,11 @@ def gateway(
|
|||||||
_run_gateway(cfg, port=port)
|
_run_gateway(cfg, port=port)
|
||||||
|
|
||||||
|
|
||||||
def _load_or_create_desktop_config(config: str | None, workspace: str | None) -> Config:
|
|
||||||
"""Load the desktop-owned config, creating it on first launch."""
|
|
||||||
from nanobot.config.loader import (
|
|
||||||
get_config_path,
|
|
||||||
load_config,
|
|
||||||
resolve_config_env_vars,
|
|
||||||
save_config,
|
|
||||||
set_config_path,
|
|
||||||
)
|
|
||||||
from nanobot.config.schema import Config as NanobotConfig
|
|
||||||
|
|
||||||
config_path = Path(config).expanduser().resolve() if config else get_config_path()
|
|
||||||
set_config_path(config_path)
|
|
||||||
created = False
|
|
||||||
if config_path.exists():
|
|
||||||
try:
|
|
||||||
loaded = resolve_config_env_vars(load_config(config_path))
|
|
||||||
except ValueError as e:
|
|
||||||
console.print(f"[red]Error: {e}[/red]")
|
|
||||||
raise typer.Exit(1)
|
|
||||||
else:
|
|
||||||
loaded = NanobotConfig()
|
|
||||||
created = True
|
|
||||||
|
|
||||||
if workspace:
|
|
||||||
workspace_path = Path(workspace).expanduser()
|
|
||||||
loaded.agents.defaults.workspace = str(workspace_path)
|
|
||||||
created = True
|
|
||||||
|
|
||||||
if created:
|
|
||||||
save_config(loaded, config_path)
|
|
||||||
return loaded
|
|
||||||
|
|
||||||
|
|
||||||
def _configure_desktop_gateway(
|
|
||||||
config: Config,
|
|
||||||
*,
|
|
||||||
webui_port: int,
|
|
||||||
webui_socket: str | None,
|
|
||||||
token_issue_secret: str,
|
|
||||||
) -> None:
|
|
||||||
"""Force a local WebSocket-only gateway for the desktop app process."""
|
|
||||||
config.gateway.host = "127.0.0.1"
|
|
||||||
config.gateway.port = webui_port
|
|
||||||
config.gateway.heartbeat.enabled = False
|
|
||||||
|
|
||||||
extras = dict(getattr(config.channels, "__pydantic_extra__", None) or {})
|
|
||||||
for name, section in list(extras.items()):
|
|
||||||
if name == "websocket":
|
|
||||||
continue
|
|
||||||
if isinstance(section, dict):
|
|
||||||
extras[name] = {**section, "enabled": False}
|
|
||||||
else:
|
|
||||||
with suppress(Exception):
|
|
||||||
setattr(section, "enabled", False)
|
|
||||||
extras[name] = section
|
|
||||||
|
|
||||||
websocket_cfg = extras.get("websocket")
|
|
||||||
if not isinstance(websocket_cfg, dict):
|
|
||||||
websocket_cfg = {}
|
|
||||||
websocket_cfg.update(
|
|
||||||
{
|
|
||||||
"enabled": True,
|
|
||||||
"host": "127.0.0.1",
|
|
||||||
"port": webui_port,
|
|
||||||
"unix_socket_path": webui_socket or "",
|
|
||||||
"path": "/",
|
|
||||||
"token_issue_secret": token_issue_secret,
|
|
||||||
"websocket_requires_token": True,
|
|
||||||
"allow_from": ["*"],
|
|
||||||
"streaming": True,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
extras["websocket"] = websocket_cfg
|
|
||||||
config.channels.__pydantic_extra__ = extras
|
|
||||||
|
|
||||||
|
|
||||||
@app.command("desktop-gateway", hidden=True)
|
|
||||||
def desktop_gateway(
|
|
||||||
webui_port: int = typer.Option(0, "--webui-port", min=0, max=65535),
|
|
||||||
webui_socket: str | None = typer.Option(None, "--webui-socket", help="Unix socket path for desktop IPC"),
|
|
||||||
token_issue_secret: str = typer.Option(..., "--token-issue-secret"),
|
|
||||||
workspace: str | None = typer.Option(None, "--workspace", "-w", help="Desktop workspace directory"),
|
|
||||||
config: str | None = typer.Option(None, "--config", "-c", help="Desktop config file"),
|
|
||||||
verbose: bool = typer.Option(False, "--verbose", "-v", help="Verbose output"),
|
|
||||||
):
|
|
||||||
"""Start the private local gateway used by nanobot Desktop."""
|
|
||||||
if not token_issue_secret.strip():
|
|
||||||
console.print("[red]Error: --token-issue-secret is required[/red]")
|
|
||||||
raise typer.Exit(1)
|
|
||||||
if webui_port <= 0 and not (webui_socket or "").strip():
|
|
||||||
console.print("[red]Error: --webui-port or --webui-socket is required[/red]")
|
|
||||||
raise typer.Exit(1)
|
|
||||||
if verbose:
|
|
||||||
logger.remove(_log_handler_id)
|
|
||||||
logger.add(
|
|
||||||
sys.stderr,
|
|
||||||
format=(
|
|
||||||
"<green>{time:YYYY-MM-DD HH:mm:ss}</green> | "
|
|
||||||
"<level>{level: <5}</level> | "
|
|
||||||
"<cyan>{extra[channel]}</cyan> | "
|
|
||||||
"<level>{message}</level>"
|
|
||||||
),
|
|
||||||
level="DEBUG",
|
|
||||||
colorize=None,
|
|
||||||
filter=lambda record: record["extra"].setdefault("channel", "-") or True,
|
|
||||||
)
|
|
||||||
cfg = _load_or_create_desktop_config(config, workspace)
|
|
||||||
_configure_desktop_gateway(
|
|
||||||
cfg,
|
|
||||||
webui_port=webui_port,
|
|
||||||
webui_socket=webui_socket,
|
|
||||||
token_issue_secret=token_issue_secret,
|
|
||||||
)
|
|
||||||
_run_gateway(
|
|
||||||
cfg,
|
|
||||||
port=webui_port,
|
|
||||||
webui_static_dist=False,
|
|
||||||
webui_runtime_surface="native",
|
|
||||||
webui_runtime_capabilities={
|
|
||||||
"can_restart_engine": True,
|
|
||||||
"can_pick_folder": True,
|
|
||||||
"can_open_logs": True,
|
|
||||||
"can_export_diagnostics": True,
|
|
||||||
},
|
|
||||||
health_server_enabled=False,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _run_gateway(
|
def _run_gateway(
|
||||||
config: Config,
|
config: Config,
|
||||||
*,
|
*,
|
||||||
port: int | None = None,
|
port: int | None = None,
|
||||||
open_browser_url: str | None = None,
|
open_browser_url: str | None = None,
|
||||||
webui_static_dist: bool = True,
|
|
||||||
webui_runtime_surface: str = "browser",
|
|
||||||
webui_runtime_capabilities: dict[str, Any] | None = None,
|
|
||||||
health_server_enabled: bool = True,
|
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Shared gateway runtime; ``open_browser_url`` opens a tab once channels are up."""
|
"""Shared gateway runtime; ``open_browser_url`` opens a tab once channels are up."""
|
||||||
from nanobot.agent.tools.cron import CronTool
|
from nanobot.agent.tools.cron import CronTool
|
||||||
@@ -867,6 +720,7 @@ def _run_gateway(
|
|||||||
from nanobot.channels.websocket import publish_runtime_model_update
|
from nanobot.channels.websocket import publish_runtime_model_update
|
||||||
from nanobot.cron.service import CronService
|
from nanobot.cron.service import CronService
|
||||||
from nanobot.cron.types import CronJob
|
from nanobot.cron.types import CronJob
|
||||||
|
from nanobot.heartbeat.service import HeartbeatService
|
||||||
from nanobot.providers.factory import build_provider_snapshot, load_provider_snapshot
|
from nanobot.providers.factory import build_provider_snapshot, load_provider_snapshot
|
||||||
from nanobot.providers.image_generation import image_gen_provider_configs
|
from nanobot.providers.image_generation import image_gen_provider_configs
|
||||||
from nanobot.session.manager import SessionManager
|
from nanobot.session.manager import SessionManager
|
||||||
@@ -958,9 +812,6 @@ def _run_gateway(
|
|||||||
# Set cron callback (needs agent)
|
# Set cron callback (needs agent)
|
||||||
async def on_cron_job(job: CronJob) -> str | None:
|
async def on_cron_job(job: CronJob) -> str | None:
|
||||||
"""Execute a cron job through the agent."""
|
"""Execute a cron job through the agent."""
|
||||||
async def _silent(*_args, **_kwargs):
|
|
||||||
pass
|
|
||||||
|
|
||||||
# Dream is an internal job — run directly, not through the agent loop.
|
# Dream is an internal job — run directly, not through the agent loop.
|
||||||
if job.name == "dream":
|
if job.name == "dream":
|
||||||
try:
|
try:
|
||||||
@@ -970,64 +821,7 @@ def _run_gateway(
|
|||||||
logger.exception("Dream cron job failed")
|
logger.exception("Dream cron job failed")
|
||||||
return None
|
return None
|
||||||
|
|
||||||
# Heartbeat is a system job that checks HEARTBEAT.md for active tasks.
|
from nanobot.utils.evaluator import evaluate_response
|
||||||
if job.name == "heartbeat":
|
|
||||||
heartbeat_file = config.workspace_path / "HEARTBEAT.md"
|
|
||||||
try:
|
|
||||||
content = heartbeat_file.read_text(encoding="utf-8")
|
|
||||||
except OSError:
|
|
||||||
logger.debug("Heartbeat: HEARTBEAT.md missing")
|
|
||||||
return None
|
|
||||||
if not content or content == _heartbeat_template():
|
|
||||||
logger.debug("Heartbeat: HEARTBEAT.md empty or identical to template")
|
|
||||||
return None
|
|
||||||
|
|
||||||
channel, chat_id = _pick_heartbeat_target()
|
|
||||||
if channel == "cli":
|
|
||||||
return None
|
|
||||||
|
|
||||||
prompt = (
|
|
||||||
_HEARTBEAT_PREAMBLE
|
|
||||||
+ f"Review the following HEARTBEAT.md and report any active tasks:\n\n{content}"
|
|
||||||
)
|
|
||||||
|
|
||||||
message_suppress_token = None
|
|
||||||
if isinstance(message_tool, MessageTool):
|
|
||||||
message_suppress_token = message_tool.set_suppress_delivery(True)
|
|
||||||
|
|
||||||
try:
|
|
||||||
resp = await agent.process_direct(
|
|
||||||
prompt,
|
|
||||||
session_key="heartbeat",
|
|
||||||
channel=channel,
|
|
||||||
chat_id=chat_id,
|
|
||||||
on_progress=_silent,
|
|
||||||
)
|
|
||||||
finally:
|
|
||||||
if isinstance(message_tool, MessageTool) and message_suppress_token is not None:
|
|
||||||
message_tool.reset_suppress_delivery(message_suppress_token)
|
|
||||||
response = resp.content if resp else ""
|
|
||||||
|
|
||||||
# Keep a small tail of heartbeat history so the loop stays bounded.
|
|
||||||
session = agent.sessions.get_or_create("heartbeat")
|
|
||||||
session.retain_recent_legal_suffix(hb_cfg.keep_recent_messages)
|
|
||||||
agent.sessions.save(session)
|
|
||||||
|
|
||||||
if not response:
|
|
||||||
return None
|
|
||||||
|
|
||||||
should_notify = await evaluate_response(
|
|
||||||
response, prompt, agent.provider, agent.model, default_notify=False,
|
|
||||||
)
|
|
||||||
if should_notify:
|
|
||||||
logger.info("Heartbeat: completed, delivering response")
|
|
||||||
await _deliver_to_channel(
|
|
||||||
OutboundMessage(channel=channel, chat_id=chat_id, content=response),
|
|
||||||
record=True,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
logger.info("Heartbeat: silenced by post-run evaluation")
|
|
||||||
return response
|
|
||||||
|
|
||||||
reminder_note = (
|
reminder_note = (
|
||||||
"The scheduled time has arrived. Deliver this reminder to the user now, "
|
"The scheduled time has arrived. Deliver this reminder to the user now, "
|
||||||
@@ -1042,6 +836,9 @@ def _run_gateway(
|
|||||||
if isinstance(cron_tool, CronTool):
|
if isinstance(cron_tool, CronTool):
|
||||||
cron_token = cron_tool.set_cron_context(True)
|
cron_token = cron_tool.set_cron_context(True)
|
||||||
|
|
||||||
|
async def _silent(*_args, **_kwargs):
|
||||||
|
pass
|
||||||
|
|
||||||
message_record_token = None
|
message_record_token = None
|
||||||
if isinstance(message_tool, MessageTool):
|
if isinstance(message_tool, MessageTool):
|
||||||
message_record_token = message_tool.set_record_channel_delivery(True)
|
message_record_token = message_tool.set_record_channel_delivery(True)
|
||||||
@@ -1098,14 +895,12 @@ def _run_gateway(
|
|||||||
bus,
|
bus,
|
||||||
session_manager=session_manager,
|
session_manager=session_manager,
|
||||||
webui_runtime_model_name=_webui_runtime_model_name,
|
webui_runtime_model_name=_webui_runtime_model_name,
|
||||||
webui_static_dist=webui_static_dist,
|
|
||||||
webui_runtime_surface=webui_runtime_surface,
|
|
||||||
webui_runtime_capabilities=webui_runtime_capabilities,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
def _pick_heartbeat_target() -> tuple[str, str]:
|
def _pick_heartbeat_target() -> tuple[str, str]:
|
||||||
"""Pick a routable channel/chat target for heartbeat-triggered messages."""
|
"""Pick a routable channel/chat target for heartbeat-triggered messages."""
|
||||||
enabled = set(channels.enabled_channels)
|
enabled = set(channels.enabled_channels)
|
||||||
|
# Prefer the most recently updated non-internal session on an enabled channel.
|
||||||
for item in session_manager.list_sessions():
|
for item in session_manager.list_sessions():
|
||||||
key = item.get("key") or ""
|
key = item.get("key") or ""
|
||||||
if ":" not in key:
|
if ":" not in key:
|
||||||
@@ -1115,8 +910,70 @@ def _run_gateway(
|
|||||||
continue
|
continue
|
||||||
if channel in enabled and chat_id:
|
if channel in enabled and chat_id:
|
||||||
return channel, chat_id
|
return channel, chat_id
|
||||||
|
# Fallback keeps prior behavior but remains explicit.
|
||||||
return "cli", "direct"
|
return "cli", "direct"
|
||||||
|
|
||||||
|
# Create heartbeat service
|
||||||
|
heartbeat_preamble = (
|
||||||
|
"[Your response will be delivered directly to the user's messaging app. "
|
||||||
|
"Output ONLY the final user-facing message. Never reference internal "
|
||||||
|
"files (HEARTBEAT.md, AWARENESS.md, etc.), your instructions, or your "
|
||||||
|
"decision process. If nothing needs reporting, respond with just "
|
||||||
|
"'All clear.' and nothing else.]\n\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
async def on_heartbeat_execute(tasks: str) -> str:
|
||||||
|
"""Phase 2: execute heartbeat tasks through the full agent loop."""
|
||||||
|
channel, chat_id = _pick_heartbeat_target()
|
||||||
|
|
||||||
|
async def _silent(*_args, **_kwargs):
|
||||||
|
pass
|
||||||
|
|
||||||
|
resp = await agent.process_direct(
|
||||||
|
heartbeat_preamble + tasks,
|
||||||
|
session_key="heartbeat",
|
||||||
|
channel=channel,
|
||||||
|
chat_id=chat_id,
|
||||||
|
on_progress=_silent,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Keep a small tail of heartbeat history so the loop stays bounded
|
||||||
|
# without losing all short-term context between runs.
|
||||||
|
session = agent.sessions.get_or_create("heartbeat")
|
||||||
|
session.retain_recent_legal_suffix(hb_cfg.keep_recent_messages)
|
||||||
|
agent.sessions.save(session)
|
||||||
|
|
||||||
|
return resp.content if resp else ""
|
||||||
|
|
||||||
|
async def on_heartbeat_notify(response: str) -> None:
|
||||||
|
"""Deliver a heartbeat response to the user's channel.
|
||||||
|
|
||||||
|
In addition to publishing the outbound message, this injects the
|
||||||
|
delivered text as an assistant turn into the *target channel's*
|
||||||
|
session. Without this, a user reply on the channel (e.g. "Sure")
|
||||||
|
lands in a session that has no context about the heartbeat message
|
||||||
|
and the agent cannot follow through.
|
||||||
|
"""
|
||||||
|
channel, chat_id = _pick_heartbeat_target()
|
||||||
|
if channel == "cli":
|
||||||
|
return # No external channel available to deliver to
|
||||||
|
|
||||||
|
await _deliver_to_channel(
|
||||||
|
OutboundMessage(channel=channel, chat_id=chat_id, content=response),
|
||||||
|
record=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
hb_cfg = config.gateway.heartbeat
|
||||||
|
heartbeat = HeartbeatService(
|
||||||
|
workspace=config.workspace_path,
|
||||||
|
llm_runtime=agent.llm_runtime,
|
||||||
|
on_execute=on_heartbeat_execute,
|
||||||
|
on_notify=on_heartbeat_notify,
|
||||||
|
interval_s=hb_cfg.interval_s,
|
||||||
|
enabled=hb_cfg.enabled,
|
||||||
|
timezone=config.agents.defaults.timezone,
|
||||||
|
)
|
||||||
|
|
||||||
if channels.enabled_channels:
|
if channels.enabled_channels:
|
||||||
console.print(f"[green]✓[/green] Channels enabled: {', '.join(channels.enabled_channels)}")
|
console.print(f"[green]✓[/green] Channels enabled: {', '.join(channels.enabled_channels)}")
|
||||||
else:
|
else:
|
||||||
@@ -1126,11 +983,7 @@ def _run_gateway(
|
|||||||
if cron_status["jobs"] > 0:
|
if cron_status["jobs"] > 0:
|
||||||
console.print(f"[green]✓[/green] Cron: {cron_status['jobs']} scheduled jobs")
|
console.print(f"[green]✓[/green] Cron: {cron_status['jobs']} scheduled jobs")
|
||||||
|
|
||||||
hb_cfg = config.gateway.heartbeat
|
|
||||||
if hb_cfg.enabled:
|
|
||||||
console.print(f"[green]✓[/green] Heartbeat: every {hb_cfg.interval_s}s")
|
console.print(f"[green]✓[/green] Heartbeat: every {hb_cfg.interval_s}s")
|
||||||
else:
|
|
||||||
console.print("[yellow]✗[/yellow] Heartbeat: disabled")
|
|
||||||
|
|
||||||
async def _health_server(host: str, health_port: int):
|
async def _health_server(host: str, health_port: int):
|
||||||
"""Lightweight HTTP health endpoint on the gateway port."""
|
"""Lightweight HTTP health endpoint on the gateway port."""
|
||||||
@@ -1174,15 +1027,14 @@ def _run_gateway(
|
|||||||
console.print(f"[green]✓[/green] Health endpoint: http://{host}:{health_port}/health")
|
console.print(f"[green]✓[/green] Health endpoint: http://{host}:{health_port}/health")
|
||||||
async with server:
|
async with server:
|
||||||
await server.serve_forever()
|
await server.serve_forever()
|
||||||
# Register Dream system job (idempotent on restart)
|
# Register Dream system job (always-on, idempotent on restart)
|
||||||
dream_cfg = config.agents.defaults.dream
|
dream_cfg = config.agents.defaults.dream
|
||||||
if dream_cfg.model_override:
|
if dream_cfg.model_override:
|
||||||
agent.dream.model = dream_cfg.model_override
|
agent.dream.model = dream_cfg.model_override
|
||||||
agent.dream.max_batch_size = dream_cfg.max_batch_size
|
agent.dream.max_batch_size = dream_cfg.max_batch_size
|
||||||
agent.dream.max_iterations = dream_cfg.max_iterations
|
agent.dream.max_iterations = dream_cfg.max_iterations
|
||||||
agent.dream.annotate_line_ages = dream_cfg.annotate_line_ages
|
agent.dream.annotate_line_ages = dream_cfg.annotate_line_ages
|
||||||
from nanobot.cron.types import CronJob, CronPayload, CronSchedule
|
from nanobot.cron.types import CronJob, CronPayload
|
||||||
if dream_cfg.enabled:
|
|
||||||
cron.register_system_job(CronJob(
|
cron.register_system_job(CronJob(
|
||||||
id="dream",
|
id="dream",
|
||||||
name="dream",
|
name="dream",
|
||||||
@@ -1190,21 +1042,6 @@ def _run_gateway(
|
|||||||
payload=CronPayload(kind="system_event"),
|
payload=CronPayload(kind="system_event"),
|
||||||
))
|
))
|
||||||
console.print(f"[green]✓[/green] Dream: {dream_cfg.describe_schedule()}")
|
console.print(f"[green]✓[/green] Dream: {dream_cfg.describe_schedule()}")
|
||||||
else:
|
|
||||||
console.print("[yellow]○[/yellow] Dream: disabled")
|
|
||||||
|
|
||||||
# Register Heartbeat system job (idempotent on restart)
|
|
||||||
if hb_cfg.enabled:
|
|
||||||
cron.register_system_job(CronJob(
|
|
||||||
id="heartbeat",
|
|
||||||
name="heartbeat",
|
|
||||||
schedule=CronSchedule(
|
|
||||||
kind="every",
|
|
||||||
every_ms=hb_cfg.interval_s * 1000,
|
|
||||||
tz=config.agents.defaults.timezone,
|
|
||||||
),
|
|
||||||
payload=CronPayload(kind="system_event"),
|
|
||||||
))
|
|
||||||
|
|
||||||
async def _open_browser_when_ready() -> None:
|
async def _open_browser_when_ready() -> None:
|
||||||
"""Wait for the gateway to bind, then point the user's browser at the webui."""
|
"""Wait for the gateway to bind, then point the user's browser at the webui."""
|
||||||
@@ -1232,12 +1069,12 @@ def _run_gateway(
|
|||||||
async def run():
|
async def run():
|
||||||
try:
|
try:
|
||||||
await cron.start()
|
await cron.start()
|
||||||
|
await heartbeat.start()
|
||||||
tasks = [
|
tasks = [
|
||||||
agent.run(),
|
agent.run(),
|
||||||
channels.start_all(),
|
channels.start_all(),
|
||||||
|
_health_server(config.gateway.host, port),
|
||||||
]
|
]
|
||||||
if health_server_enabled:
|
|
||||||
tasks.append(_health_server(config.gateway.host, port))
|
|
||||||
if open_browser_url:
|
if open_browser_url:
|
||||||
tasks.append(_open_browser_when_ready())
|
tasks.append(_open_browser_when_ready())
|
||||||
await asyncio.gather(*tasks)
|
await asyncio.gather(*tasks)
|
||||||
@@ -1250,6 +1087,7 @@ def _run_gateway(
|
|||||||
console.print(traceback.format_exc())
|
console.print(traceback.format_exc())
|
||||||
finally:
|
finally:
|
||||||
await agent.close_mcp()
|
await agent.close_mcp()
|
||||||
|
heartbeat.stop()
|
||||||
cron.stop()
|
cron.stop()
|
||||||
agent.stop()
|
agent.stop()
|
||||||
await channels.stop_all()
|
await channels.stop_all()
|
||||||
@@ -1691,6 +1529,106 @@ def status():
|
|||||||
console.print(f"{spec.label}: {'[green]✓[/green]' if has_key else '[dim]not set[/dim]'}")
|
console.print(f"{spec.label}: {'[green]✓[/green]' if has_key else '[dim]not set[/dim]'}")
|
||||||
|
|
||||||
|
|
||||||
|
# ============================================================================
|
||||||
|
# Config Commands
|
||||||
|
# ============================================================================
|
||||||
|
|
||||||
|
config_app = typer.Typer(help="Manage configuration")
|
||||||
|
app.add_typer(config_app, name="config")
|
||||||
|
|
||||||
|
|
||||||
|
@config_app.command("set")
|
||||||
|
def config_set(
|
||||||
|
path: str = typer.Argument(..., help="Dot path, e.g. agents.defaults.model"),
|
||||||
|
value: str = typer.Argument(..., help="Value. Use null/true/false or JSON for structured values."),
|
||||||
|
config_path: str | None = typer.Option(None, "--config", "-c", help="Path to config file"),
|
||||||
|
):
|
||||||
|
"""Set one config value by dot path."""
|
||||||
|
from pydantic import ValidationError
|
||||||
|
|
||||||
|
from nanobot.config.loader import get_config_path, load_config, save_config, set_config_path
|
||||||
|
from nanobot.config.schema import Config
|
||||||
|
|
||||||
|
resolved_path = Path(config_path).expanduser().resolve() if config_path else get_config_path()
|
||||||
|
if config_path:
|
||||||
|
set_config_path(resolved_path)
|
||||||
|
|
||||||
|
config = load_config(resolved_path)
|
||||||
|
parsed = _parse_config_cli_value(value)
|
||||||
|
try:
|
||||||
|
_set_config_cli_value(config, path, parsed)
|
||||||
|
validated = Config.model_validate(config.model_dump(mode="json", by_alias=True))
|
||||||
|
except (AttributeError, KeyError, TypeError, ValueError, ValidationError) as exc:
|
||||||
|
console.print(f"[red]Could not set config value:[/red] {exc}")
|
||||||
|
raise typer.Exit(1)
|
||||||
|
|
||||||
|
save_config(validated, resolved_path)
|
||||||
|
console.print(f"[green]✓[/green] Set [cyan]{path}[/cyan] = [bold]{value}[/bold]")
|
||||||
|
console.print(f"[dim]Config: {resolved_path}[/dim]")
|
||||||
|
if path in {"agents.defaults.provider", "agents.defaults.model"} and validated.agents.defaults.model_preset:
|
||||||
|
console.print(
|
||||||
|
"[yellow]! agents.defaults.model_preset is set and may override this. "
|
||||||
|
"Clear it with: nanobot config set agents.defaults.model_preset null[/yellow]"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_config_cli_value(raw: str) -> Any:
|
||||||
|
lowered = raw.strip().lower()
|
||||||
|
if lowered == "null":
|
||||||
|
return None
|
||||||
|
if lowered == "true":
|
||||||
|
return True
|
||||||
|
if lowered == "false":
|
||||||
|
return False
|
||||||
|
with suppress(Exception):
|
||||||
|
return json.loads(raw)
|
||||||
|
return raw
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_config_field(obj: Any, key: str) -> str:
|
||||||
|
from pydantic import BaseModel
|
||||||
|
from pydantic.alias_generators import to_camel, to_snake
|
||||||
|
|
||||||
|
if not isinstance(obj, BaseModel):
|
||||||
|
return key
|
||||||
|
fields = type(obj).model_fields
|
||||||
|
if key in fields:
|
||||||
|
return key
|
||||||
|
normalized = to_snake(key.replace("-", "_"))
|
||||||
|
if normalized in fields:
|
||||||
|
return normalized
|
||||||
|
for name, field in fields.items():
|
||||||
|
aliases = {
|
||||||
|
to_camel(name),
|
||||||
|
str(field.alias) if field.alias else "",
|
||||||
|
str(field.serialization_alias) if field.serialization_alias else "",
|
||||||
|
}
|
||||||
|
if key in aliases:
|
||||||
|
return name
|
||||||
|
raise AttributeError(f"Unknown config path segment {key!r}")
|
||||||
|
|
||||||
|
|
||||||
|
def _set_config_cli_value(config: Any, path: str, value: Any) -> None:
|
||||||
|
parts = [part for part in path.split(".") if part]
|
||||||
|
if not parts:
|
||||||
|
raise ValueError("Config path cannot be empty.")
|
||||||
|
|
||||||
|
current = config
|
||||||
|
for raw_part in parts[:-1]:
|
||||||
|
if isinstance(current, dict):
|
||||||
|
current = current.setdefault(raw_part, {})
|
||||||
|
continue
|
||||||
|
part = _resolve_config_field(current, raw_part)
|
||||||
|
current = getattr(current, part)
|
||||||
|
|
||||||
|
leaf = parts[-1]
|
||||||
|
if isinstance(current, dict):
|
||||||
|
current[leaf] = value
|
||||||
|
return
|
||||||
|
leaf = _resolve_config_field(current, leaf)
|
||||||
|
setattr(current, leaf, value)
|
||||||
|
|
||||||
|
|
||||||
# ============================================================================
|
# ============================================================================
|
||||||
# OAuth Login
|
# OAuth Login
|
||||||
# ============================================================================
|
# ============================================================================
|
||||||
@@ -1705,6 +1643,7 @@ _LOGOUT_HANDLERS: dict[str, Callable[[], None]] = {}
|
|||||||
_PROVIDER_DISPLAY: dict[str, str] = {
|
_PROVIDER_DISPLAY: dict[str, str] = {
|
||||||
"openai_codex": "OpenAI Codex",
|
"openai_codex": "OpenAI Codex",
|
||||||
"github_copilot": "GitHub Copilot",
|
"github_copilot": "GitHub Copilot",
|
||||||
|
"xai_oauth": "xAI Grok OAuth",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -1740,7 +1679,9 @@ def _resolve_oauth_provider(provider: str):
|
|||||||
|
|
||||||
@provider_app.command("login")
|
@provider_app.command("login")
|
||||||
def provider_login(
|
def provider_login(
|
||||||
provider: str = typer.Argument(..., help="OAuth provider (e.g. 'openai-codex', 'github-copilot')"),
|
provider: str = typer.Argument(..., help="OAuth provider (e.g. 'openai-codex', 'github-copilot', 'xai-oauth')"),
|
||||||
|
no_browser: bool = typer.Option(False, "--no-browser", help="Print the auth URL instead of opening a browser when supported."),
|
||||||
|
manual_paste: bool = typer.Option(False, "--manual-paste", help="Prompt for a callback URL or fallback code when supported."),
|
||||||
):
|
):
|
||||||
"""Authenticate with an OAuth provider."""
|
"""Authenticate with an OAuth provider."""
|
||||||
spec = _resolve_oauth_provider(provider)
|
spec = _resolve_oauth_provider(provider)
|
||||||
@@ -1751,12 +1692,18 @@ def provider_login(
|
|||||||
raise typer.Exit(1)
|
raise typer.Exit(1)
|
||||||
|
|
||||||
console.print(f"{__logo__} OAuth Login - {spec.label}\n")
|
console.print(f"{__logo__} OAuth Login - {spec.label}\n")
|
||||||
handler()
|
params = signature(handler).parameters
|
||||||
|
kwargs: dict[str, bool] = {}
|
||||||
|
if "no_browser" in params:
|
||||||
|
kwargs["no_browser"] = no_browser
|
||||||
|
if "manual_paste" in params:
|
||||||
|
kwargs["manual_paste"] = manual_paste
|
||||||
|
handler(**kwargs)
|
||||||
|
|
||||||
|
|
||||||
@provider_app.command("logout")
|
@provider_app.command("logout")
|
||||||
def provider_logout(
|
def provider_logout(
|
||||||
provider: str = typer.Argument(..., help="OAuth provider (e.g. 'openai-codex', 'github-copilot')"),
|
provider: str = typer.Argument(..., help="OAuth provider (e.g. 'openai-codex', 'github-copilot', 'xai-oauth')"),
|
||||||
):
|
):
|
||||||
"""Log out from an OAuth provider."""
|
"""Log out from an OAuth provider."""
|
||||||
spec = _resolve_oauth_provider(provider)
|
spec = _resolve_oauth_provider(provider)
|
||||||
@@ -1820,6 +1767,24 @@ def _logout_github_copilot() -> None:
|
|||||||
_delete_oauth_files(storage.get_token_path(), _PROVIDER_DISPLAY["github_copilot"])
|
_delete_oauth_files(storage.get_token_path(), _PROVIDER_DISPLAY["github_copilot"])
|
||||||
|
|
||||||
|
|
||||||
|
@_register_logout("xai_oauth")
|
||||||
|
def _logout_xai_oauth() -> None:
|
||||||
|
"""Clear local OAuth credentials for xAI Grok OAuth."""
|
||||||
|
try:
|
||||||
|
from nanobot.providers.xai_oauth_provider import delete_xai_oauth_credentials
|
||||||
|
except ImportError:
|
||||||
|
console.print("[red]xAI Grok OAuth provider unavailable.[/red]")
|
||||||
|
raise typer.Exit(1)
|
||||||
|
|
||||||
|
removed_paths = delete_xai_oauth_credentials()
|
||||||
|
if not removed_paths:
|
||||||
|
console.print(f"[yellow]! No local OAuth credentials found for {_PROVIDER_DISPLAY['xai_oauth']}[/yellow]")
|
||||||
|
return
|
||||||
|
console.print(f"[green]✓ Logged out from {_PROVIDER_DISPLAY['xai_oauth']}[/green]")
|
||||||
|
for path in removed_paths:
|
||||||
|
console.print(f"[dim]Removed: {path}[/dim]")
|
||||||
|
|
||||||
|
|
||||||
def _delete_oauth_files(token_path: Path, provider_label: str) -> None:
|
def _delete_oauth_files(token_path: Path, provider_label: str) -> None:
|
||||||
"""Delete OAuth token and lock files, reporting the result."""
|
"""Delete OAuth token and lock files, reporting the result."""
|
||||||
removed_paths: list[Path] = []
|
removed_paths: list[Path] = []
|
||||||
@@ -1863,5 +1828,36 @@ def _login_github_copilot() -> None:
|
|||||||
raise typer.Exit(1)
|
raise typer.Exit(1)
|
||||||
|
|
||||||
|
|
||||||
|
@_register_login("xai_oauth")
|
||||||
|
def _login_xai_oauth(
|
||||||
|
*,
|
||||||
|
no_browser: bool = False,
|
||||||
|
manual_paste: bool = False,
|
||||||
|
) -> None:
|
||||||
|
try:
|
||||||
|
from nanobot.providers.xai_oauth_provider import login_xai_oauth_interactive
|
||||||
|
from nanobot.providers.xai_oauth_provider import DEFAULT_XAI_MODEL
|
||||||
|
|
||||||
|
console.print("[cyan]Starting xAI Grok OAuth login...[/cyan]\n")
|
||||||
|
credential = login_xai_oauth_interactive(
|
||||||
|
print_fn=lambda s: console.print(s),
|
||||||
|
prompt_fn=lambda s: typer.prompt(s),
|
||||||
|
open_browser=not no_browser,
|
||||||
|
manual_paste=manual_paste,
|
||||||
|
)
|
||||||
|
account = credential.account_id or "xAI"
|
||||||
|
storage = "OS keychain" if credential.storage == "keyring" else "private file"
|
||||||
|
console.print(f"[green]✓ Authenticated with xAI Grok OAuth[/green] [dim]{account} · {storage}[/dim]")
|
||||||
|
console.print("[dim]To use it for chat:[/dim]")
|
||||||
|
console.print("[dim] nanobot config set agents.defaults.model_preset null[/dim]")
|
||||||
|
console.print("[dim] nanobot config set agents.defaults.provider xai-oauth[/dim]")
|
||||||
|
console.print(f"[dim] nanobot config set agents.defaults.model {DEFAULT_XAI_MODEL}[/dim]")
|
||||||
|
console.print("[dim]Hosted X Search is enabled by default for xAI OAuth.[/dim]")
|
||||||
|
console.print("[dim]To disable it: nanobot config set providers.xai_oauth.x_search.enable false[/dim]")
|
||||||
|
except Exception as e:
|
||||||
|
console.print(f"[red]Authentication error: {e}[/red]")
|
||||||
|
raise typer.Exit(1)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
app()
|
app()
|
||||||
|
|||||||
@@ -1155,7 +1155,7 @@ _SETTINGS_SECTIONS: dict[str, tuple[str, str, set[str] | None]] = {
|
|||||||
"Agent Settings": ("Agent Defaults", "Configure default model, temperature, and behavior", None),
|
"Agent Settings": ("Agent Defaults", "Configure default model, temperature, and behavior", None),
|
||||||
"Channel Common": ("Channel Common", "Configure cross-channel behavior: progress, tool hints, retries", None),
|
"Channel Common": ("Channel Common", "Configure cross-channel behavior: progress, tool hints, retries", None),
|
||||||
"API Server": ("API Server", "Configure OpenAI-compatible API endpoint", None),
|
"API Server": ("API Server", "Configure OpenAI-compatible API endpoint", None),
|
||||||
"Gateway": ("Gateway Settings", "Configure server host, port", None),
|
"Gateway": ("Gateway Settings", "Configure server host, port, and heartbeat", None),
|
||||||
"Tools": ("Tools Settings", "Configure web search, shell exec, and other tools", {"mcp_servers"}),
|
"Tools": ("Tools Settings", "Configure web search, shell exec, and other tools", {"mcp_servers"}),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -123,7 +123,7 @@ async def cmd_stop(ctx: CommandContext) -> OutboundMessage:
|
|||||||
"""Cancel all active tasks and subagents for the session."""
|
"""Cancel all active tasks and subagents for the session."""
|
||||||
loop = ctx.loop
|
loop = ctx.loop
|
||||||
msg = ctx.msg
|
msg = ctx.msg
|
||||||
total = await loop._cancel_active_tasks(ctx.key)
|
total = await loop._cancel_active_tasks(msg.session_key)
|
||||||
content = f"Stopped {total} task(s)." if total else "No active task to stop."
|
content = f"Stopped {total} task(s)." if total else "No active task to stop."
|
||||||
return OutboundMessage(
|
return OutboundMessage(
|
||||||
channel=msg.channel, chat_id=msg.chat_id, content=content,
|
channel=msg.channel, chat_id=msg.chat_id, content=content,
|
||||||
|
|||||||
@@ -10,11 +10,10 @@ import pydantic
|
|||||||
from loguru import logger
|
from loguru import logger
|
||||||
from pydantic import BaseModel
|
from pydantic import BaseModel
|
||||||
|
|
||||||
from nanobot.config.schema import Config, _resolve_tool_config_refs
|
from nanobot.config.schema import Config
|
||||||
|
|
||||||
# Global variable to store current config path (for multi-instance support)
|
# Global variable to store current config path (for multi-instance support)
|
||||||
_current_config_path: Path | None = None
|
_current_config_path: Path | None = None
|
||||||
_schema_refs_ready = False
|
|
||||||
|
|
||||||
|
|
||||||
def set_config_path(path: Path) -> None:
|
def set_config_path(path: Path) -> None:
|
||||||
@@ -40,11 +39,6 @@ def load_config(config_path: Path | None = None) -> Config:
|
|||||||
Returns:
|
Returns:
|
||||||
Loaded configuration object.
|
Loaded configuration object.
|
||||||
"""
|
"""
|
||||||
global _schema_refs_ready
|
|
||||||
if not _schema_refs_ready:
|
|
||||||
_resolve_tool_config_refs()
|
|
||||||
_schema_refs_ready = True
|
|
||||||
|
|
||||||
path = config_path or get_config_path()
|
path = config_path or get_config_path()
|
||||||
|
|
||||||
config = Config()
|
config = Config()
|
||||||
|
|||||||
+29
-37
@@ -11,7 +11,6 @@ from pydantic_settings import BaseSettings
|
|||||||
from nanobot.cron.types import CronSchedule
|
from nanobot.cron.types import CronSchedule
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from nanobot.agent.tools.cli_apps import CliAppsToolConfig
|
|
||||||
from nanobot.agent.tools.image_generation import ImageGenerationToolConfig
|
from nanobot.agent.tools.image_generation import ImageGenerationToolConfig
|
||||||
from nanobot.agent.tools.self import MyToolConfig
|
from nanobot.agent.tools.self import MyToolConfig
|
||||||
from nanobot.agent.tools.shell import ExecToolConfig
|
from nanobot.agent.tools.shell import ExecToolConfig
|
||||||
@@ -37,7 +36,6 @@ class ChannelsConfig(Base):
|
|||||||
send_progress: bool = True # stream agent's text progress to the channel
|
send_progress: bool = True # stream agent's text progress to the channel
|
||||||
send_tool_hints: bool = False # stream tool-call hints (e.g. read_file("…"))
|
send_tool_hints: bool = False # stream tool-call hints (e.g. read_file("…"))
|
||||||
show_reasoning: bool = True # surface model reasoning when channel implements it
|
show_reasoning: bool = True # surface model reasoning when channel implements it
|
||||||
extract_document_text: bool = True # extract text from document attachments before sending to the model
|
|
||||||
send_max_retries: int = Field(default=3, ge=0, le=10) # Max delivery attempts (initial send included)
|
send_max_retries: int = Field(default=3, ge=0, le=10) # Max delivery attempts (initial send included)
|
||||||
transcription_provider: str = "groq" # Voice transcription backend: "groq" or "openai"
|
transcription_provider: str = "groq" # Voice transcription backend: "groq" or "openai"
|
||||||
transcription_language: str | None = Field(default=None, pattern=r"^[a-z]{2,3}$") # Optional ISO-639-1 hint for audio transcription
|
transcription_language: str | None = Field(default=None, pattern=r"^[a-z]{2,3}$") # Optional ISO-639-1 hint for audio transcription
|
||||||
@@ -48,7 +46,6 @@ class DreamConfig(Base):
|
|||||||
|
|
||||||
_HOUR_MS = 3_600_000
|
_HOUR_MS = 3_600_000
|
||||||
|
|
||||||
enabled: bool = True # Register the periodic Dream consolidation job on startup
|
|
||||||
interval_h: int = Field(default=2, ge=1) # Every 2 hours by default
|
interval_h: int = Field(default=2, ge=1) # Every 2 hours by default
|
||||||
cron: str | None = Field(default=None, exclude=True) # Legacy compatibility override
|
cron: str | None = Field(default=None, exclude=True) # Legacy compatibility override
|
||||||
model_override: str | None = Field(
|
model_override: str | None = Field(
|
||||||
@@ -94,7 +91,6 @@ FallbackCandidate = str | InlineFallbackConfig
|
|||||||
class ModelPresetConfig(Base):
|
class ModelPresetConfig(Base):
|
||||||
"""A named set of model + generation parameters for quick switching."""
|
"""A named set of model + generation parameters for quick switching."""
|
||||||
|
|
||||||
label: str | None = None
|
|
||||||
model: str
|
model: str
|
||||||
provider: str = "auto"
|
provider: str = "auto"
|
||||||
max_tokens: int = 8192
|
max_tokens: int = 8192
|
||||||
@@ -173,9 +169,8 @@ class ProviderConfig(Base):
|
|||||||
|
|
||||||
api_key: str | None = None
|
api_key: str | None = None
|
||||||
api_base: str | None = None
|
api_base: str | None = None
|
||||||
api_type: Literal["auto", "chat_completions", "responses"] = "auto" # Request API surface
|
|
||||||
extra_headers: dict[str, str] | None = None # Custom headers (e.g. APP-Code for AiHubMix)
|
extra_headers: dict[str, str] | None = None # Custom headers (e.g. APP-Code for AiHubMix)
|
||||||
extra_body: dict[str, Any] | None = None # Extra provider request fields; shape depends on provider/API surface
|
extra_body: dict[str, Any] | None = None # Extra fields merged into every request body
|
||||||
|
|
||||||
|
|
||||||
class BedrockProviderConfig(ProviderConfig):
|
class BedrockProviderConfig(ProviderConfig):
|
||||||
@@ -185,6 +180,28 @@ class BedrockProviderConfig(ProviderConfig):
|
|||||||
profile: str | None = None # Optional AWS shared config profile
|
profile: str | None = None # Optional AWS shared config profile
|
||||||
|
|
||||||
|
|
||||||
|
class XaiOAuthXSearchConfig(Base):
|
||||||
|
"""xAI hosted X Search configuration."""
|
||||||
|
|
||||||
|
enable: bool = True
|
||||||
|
allowed_x_handles: list[str] | None = None
|
||||||
|
excluded_x_handles: list[str] | None = None
|
||||||
|
from_date: str | None = None
|
||||||
|
to_date: str | None = None
|
||||||
|
enable_image_understanding: bool = False
|
||||||
|
enable_video_understanding: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class XaiOAuthProviderConfig(ProviderConfig):
|
||||||
|
"""xAI OAuth provider configuration."""
|
||||||
|
|
||||||
|
x_search: XaiOAuthXSearchConfig = Field(default_factory=XaiOAuthXSearchConfig)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_default_xai_oauth_config(value: Any) -> bool:
|
||||||
|
return isinstance(value, XaiOAuthProviderConfig) and value == XaiOAuthProviderConfig()
|
||||||
|
|
||||||
|
|
||||||
class ProvidersConfig(Base):
|
class ProvidersConfig(Base):
|
||||||
"""Configuration for LLM providers."""
|
"""Configuration for LLM providers."""
|
||||||
|
|
||||||
@@ -216,29 +233,22 @@ class ProvidersConfig(Base):
|
|||||||
ant_ling: ProviderConfig = Field(default_factory=ProviderConfig) # Ant Ling
|
ant_ling: ProviderConfig = Field(default_factory=ProviderConfig) # Ant Ling
|
||||||
aihubmix: ProviderConfig = Field(default_factory=ProviderConfig) # AiHubMix API gateway
|
aihubmix: ProviderConfig = Field(default_factory=ProviderConfig) # AiHubMix API gateway
|
||||||
siliconflow: ProviderConfig = Field(default_factory=ProviderConfig) # SiliconFlow (硅基流动)
|
siliconflow: ProviderConfig = Field(default_factory=ProviderConfig) # SiliconFlow (硅基流动)
|
||||||
novita: ProviderConfig = Field(default_factory=ProviderConfig) # Novita AI
|
|
||||||
volcengine: ProviderConfig = Field(default_factory=ProviderConfig) # VolcEngine (火山引擎)
|
volcengine: ProviderConfig = Field(default_factory=ProviderConfig) # VolcEngine (火山引擎)
|
||||||
volcengine_coding_plan: ProviderConfig = Field(default_factory=ProviderConfig) # VolcEngine Coding Plan
|
volcengine_coding_plan: ProviderConfig = Field(default_factory=ProviderConfig) # VolcEngine Coding Plan
|
||||||
byteplus: ProviderConfig = Field(default_factory=ProviderConfig) # BytePlus (VolcEngine international)
|
byteplus: ProviderConfig = Field(default_factory=ProviderConfig) # BytePlus (VolcEngine international)
|
||||||
byteplus_coding_plan: ProviderConfig = Field(default_factory=ProviderConfig) # BytePlus Coding Plan
|
byteplus_coding_plan: ProviderConfig = Field(default_factory=ProviderConfig) # BytePlus Coding Plan
|
||||||
openai_codex: ProviderConfig = Field(default_factory=ProviderConfig, exclude=True) # OpenAI Codex (OAuth)
|
openai_codex: ProviderConfig = Field(default_factory=ProviderConfig, exclude=True) # OpenAI Codex (OAuth)
|
||||||
github_copilot: ProviderConfig = Field(default_factory=ProviderConfig, exclude=True) # Github Copilot (OAuth)
|
github_copilot: ProviderConfig = Field(default_factory=ProviderConfig, exclude=True) # Github Copilot (OAuth)
|
||||||
|
xai_oauth: XaiOAuthProviderConfig = Field(
|
||||||
|
default_factory=XaiOAuthProviderConfig,
|
||||||
|
exclude_if=_is_default_xai_oauth_config,
|
||||||
|
) # xAI Grok OAuth
|
||||||
qianfan: ProviderConfig = Field(default_factory=ProviderConfig) # Qianfan (百度千帆)
|
qianfan: ProviderConfig = Field(default_factory=ProviderConfig) # Qianfan (百度千帆)
|
||||||
nvidia: ProviderConfig = Field(default_factory=ProviderConfig) # NVIDIA NIM (nvapi- keys)
|
nvidia: ProviderConfig = Field(default_factory=ProviderConfig) # NVIDIA NIM (nvapi- keys)
|
||||||
|
|
||||||
@model_validator(mode="after")
|
|
||||||
def _validate_api_type_scope(self) -> "ProvidersConfig":
|
|
||||||
for name in self.__class__.model_fields:
|
|
||||||
if name == "openai":
|
|
||||||
continue
|
|
||||||
provider = getattr(self, name, None)
|
|
||||||
if isinstance(provider, ProviderConfig) and provider.api_type != "auto":
|
|
||||||
raise ValueError("providers.<name>.api_type is only supported for providers.openai")
|
|
||||||
return self
|
|
||||||
|
|
||||||
|
|
||||||
class HeartbeatConfig(Base):
|
class HeartbeatConfig(Base):
|
||||||
"""Heartbeat service configuration (now backed by cron)."""
|
"""Heartbeat service configuration."""
|
||||||
|
|
||||||
enabled: bool = True
|
enabled: bool = True
|
||||||
interval_s: int = 30 * 60 # 30 minutes
|
interval_s: int = 30 * 60 # 30 minutes
|
||||||
@@ -268,7 +278,6 @@ class MCPServerConfig(Base):
|
|||||||
command: str = "" # Stdio: command to run (e.g. "npx")
|
command: str = "" # Stdio: command to run (e.g. "npx")
|
||||||
args: list[str] = Field(default_factory=list) # Stdio: command arguments
|
args: list[str] = Field(default_factory=list) # Stdio: command arguments
|
||||||
env: dict[str, str] = Field(default_factory=dict) # Stdio: extra env vars
|
env: dict[str, str] = Field(default_factory=dict) # Stdio: extra env vars
|
||||||
cwd: str = "" # Stdio: working directory for MCP server runtime artifacts
|
|
||||||
url: str = "" # HTTP/SSE: endpoint URL
|
url: str = "" # HTTP/SSE: endpoint URL
|
||||||
headers: dict[str, str] = Field(default_factory=dict) # HTTP/SSE: custom headers
|
headers: dict[str, str] = Field(default_factory=dict) # HTTP/SSE: custom headers
|
||||||
tool_timeout: int = 30 # seconds before a tool call is cancelled
|
tool_timeout: int = 30 # seconds before a tool call is cancelled
|
||||||
@@ -292,21 +301,11 @@ class ToolsConfig(Base):
|
|||||||
|
|
||||||
web: WebToolsConfig = Field(default_factory=lambda: _lazy_default("nanobot.agent.tools.web", "WebToolsConfig"))
|
web: WebToolsConfig = Field(default_factory=lambda: _lazy_default("nanobot.agent.tools.web", "WebToolsConfig"))
|
||||||
exec: ExecToolConfig = Field(default_factory=lambda: _lazy_default("nanobot.agent.tools.shell", "ExecToolConfig"))
|
exec: ExecToolConfig = Field(default_factory=lambda: _lazy_default("nanobot.agent.tools.shell", "ExecToolConfig"))
|
||||||
cli_apps: CliAppsToolConfig = Field(default_factory=lambda: _lazy_default("nanobot.agent.tools.cli_apps", "CliAppsToolConfig"))
|
|
||||||
my: MyToolConfig = Field(default_factory=lambda: _lazy_default("nanobot.agent.tools.self", "MyToolConfig"))
|
my: MyToolConfig = Field(default_factory=lambda: _lazy_default("nanobot.agent.tools.self", "MyToolConfig"))
|
||||||
image_generation: ImageGenerationToolConfig = Field(
|
image_generation: ImageGenerationToolConfig = Field(
|
||||||
default_factory=lambda: _lazy_default("nanobot.agent.tools.image_generation", "ImageGenerationToolConfig"),
|
default_factory=lambda: _lazy_default("nanobot.agent.tools.image_generation", "ImageGenerationToolConfig"),
|
||||||
)
|
)
|
||||||
restrict_to_workspace: bool = False # policy intent: keep tool access inside workspace when possible
|
restrict_to_workspace: bool = False # restrict all tool access to workspace directory
|
||||||
webui_allow_local_service_access: bool = Field(
|
|
||||||
default=True,
|
|
||||||
validation_alias=AliasChoices(
|
|
||||||
"webuiAllowLocalServiceAccess",
|
|
||||||
"webui_allow_local_service_access",
|
|
||||||
"allowLocalPreviewAccess",
|
|
||||||
"allow_local_preview_access",
|
|
||||||
),
|
|
||||||
) # allow WebUI Full Access shell checks against localhost services; legacy allowLocalPreviewAccess still reads
|
|
||||||
mcp_servers: dict[str, MCPServerConfig] = Field(default_factory=dict)
|
mcp_servers: dict[str, MCPServerConfig] = Field(default_factory=dict)
|
||||||
ssrf_whitelist: list[str] = Field(default_factory=list) # CIDR ranges to exempt from SSRF blocking (e.g. ["100.64.0.0/10"] for Tailscale)
|
ssrf_whitelist: list[str] = Field(default_factory=list) # CIDR ranges to exempt from SSRF blocking (e.g. ["100.64.0.0/10"] for Tailscale)
|
||||||
|
|
||||||
@@ -325,11 +324,6 @@ class Config(BaseSettings):
|
|||||||
validation_alias=AliasChoices("modelPresets", "model_presets"),
|
validation_alias=AliasChoices("modelPresets", "model_presets"),
|
||||||
)
|
)
|
||||||
|
|
||||||
def __init__(self, **values: Any) -> None:
|
|
||||||
if not type(self).__pydantic_complete__:
|
|
||||||
_resolve_tool_config_refs()
|
|
||||||
super().__init__(**values)
|
|
||||||
|
|
||||||
@model_validator(mode="after")
|
@model_validator(mode="after")
|
||||||
def _validate_model_preset(self) -> "Config":
|
def _validate_model_preset(self) -> "Config":
|
||||||
if "default" in self.model_presets:
|
if "default" in self.model_presets:
|
||||||
@@ -493,7 +487,6 @@ def _resolve_tool_config_refs() -> None:
|
|||||||
"""
|
"""
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
from nanobot.agent.tools.cli_apps import CliAppsToolConfig
|
|
||||||
from nanobot.agent.tools.image_generation import ImageGenerationToolConfig
|
from nanobot.agent.tools.image_generation import ImageGenerationToolConfig
|
||||||
from nanobot.agent.tools.self import MyToolConfig
|
from nanobot.agent.tools.self import MyToolConfig
|
||||||
from nanobot.agent.tools.shell import ExecToolConfig
|
from nanobot.agent.tools.shell import ExecToolConfig
|
||||||
@@ -502,7 +495,6 @@ def _resolve_tool_config_refs() -> None:
|
|||||||
# Re-export into this module's namespace
|
# Re-export into this module's namespace
|
||||||
mod = sys.modules[__name__]
|
mod = sys.modules[__name__]
|
||||||
mod.ExecToolConfig = ExecToolConfig # type: ignore[attr-defined]
|
mod.ExecToolConfig = ExecToolConfig # type: ignore[attr-defined]
|
||||||
mod.CliAppsToolConfig = CliAppsToolConfig # type: ignore[attr-defined]
|
|
||||||
mod.WebToolsConfig = WebToolsConfig # type: ignore[attr-defined]
|
mod.WebToolsConfig = WebToolsConfig # type: ignore[attr-defined]
|
||||||
mod.WebSearchConfig = WebSearchConfig # type: ignore[attr-defined]
|
mod.WebSearchConfig = WebSearchConfig # type: ignore[attr-defined]
|
||||||
mod.WebFetchConfig = WebFetchConfig # type: ignore[attr-defined]
|
mod.WebFetchConfig = WebFetchConfig # type: ignore[attr-defined]
|
||||||
|
|||||||
@@ -0,0 +1,5 @@
|
|||||||
|
"""Heartbeat service for periodic agent wake-ups."""
|
||||||
|
|
||||||
|
from nanobot.heartbeat.service import HeartbeatService
|
||||||
|
|
||||||
|
__all__ = ["HeartbeatService"]
|
||||||
@@ -0,0 +1,243 @@
|
|||||||
|
"""Heartbeat service - periodic agent wake-up to check for tasks."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Callable, Coroutine
|
||||||
|
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from nanobot.providers.base import LLMProvider
|
||||||
|
from nanobot.utils.llm_runtime import LLMRuntimeResolver, static_llm_runtime
|
||||||
|
|
||||||
|
_HEARTBEAT_TOOL = [
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "heartbeat",
|
||||||
|
"description": "Report heartbeat decision after reviewing tasks.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"action": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": ["skip", "run"],
|
||||||
|
"description": "skip = nothing to do, run = has active tasks",
|
||||||
|
},
|
||||||
|
"tasks": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Natural-language summary of active tasks (required for run)",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": ["action"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class HeartbeatService:
|
||||||
|
"""
|
||||||
|
Periodic heartbeat service that wakes the agent to check for tasks.
|
||||||
|
|
||||||
|
Phase 1 (decision): reads HEARTBEAT.md and asks the LLM — via a virtual
|
||||||
|
tool call — whether there are active tasks. This avoids free-text parsing
|
||||||
|
and the unreliable HEARTBEAT_OK token.
|
||||||
|
|
||||||
|
Phase 2 (execution): only triggered when Phase 1 returns ``run``. The
|
||||||
|
``on_execute`` callback runs the task through the full agent loop and
|
||||||
|
returns the result to deliver.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
workspace: Path,
|
||||||
|
provider: LLMProvider | None = None,
|
||||||
|
model: str | None = None,
|
||||||
|
on_execute: Callable[[str], Coroutine[Any, Any, str]] | None = None,
|
||||||
|
on_notify: Callable[[str], Coroutine[Any, Any, None]] | None = None,
|
||||||
|
interval_s: int = 30 * 60,
|
||||||
|
enabled: bool = True,
|
||||||
|
timezone: str | None = None,
|
||||||
|
llm_runtime: LLMRuntimeResolver | None = None,
|
||||||
|
):
|
||||||
|
self.workspace = workspace
|
||||||
|
if llm_runtime is None:
|
||||||
|
if provider is None or model is None:
|
||||||
|
raise ValueError("HeartbeatService requires either llm_runtime or provider/model")
|
||||||
|
llm_runtime = static_llm_runtime(provider, model)
|
||||||
|
self._llm_runtime = llm_runtime
|
||||||
|
self.on_execute = on_execute
|
||||||
|
self.on_notify = on_notify
|
||||||
|
self.interval_s = interval_s
|
||||||
|
self.enabled = enabled
|
||||||
|
self.timezone = timezone
|
||||||
|
self._running = False
|
||||||
|
self._task: asyncio.Task | None = None
|
||||||
|
|
||||||
|
@property
|
||||||
|
def heartbeat_file(self) -> Path:
|
||||||
|
return self.workspace / "HEARTBEAT.md"
|
||||||
|
|
||||||
|
def _read_heartbeat_file(self) -> str | None:
|
||||||
|
if self.heartbeat_file.exists():
|
||||||
|
try:
|
||||||
|
return self.heartbeat_file.read_text(encoding="utf-8")
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def _decide(self, content: str) -> tuple[str, str]:
|
||||||
|
"""Phase 1: ask LLM to decide skip/run via virtual tool call.
|
||||||
|
|
||||||
|
Returns (action, tasks) where action is 'skip' or 'run'.
|
||||||
|
"""
|
||||||
|
from nanobot.utils.helpers import current_time_str
|
||||||
|
|
||||||
|
llm = self._llm_runtime()
|
||||||
|
|
||||||
|
response = await llm.provider.chat_with_retry(
|
||||||
|
messages=[
|
||||||
|
{"role": "system", "content": "You are a heartbeat agent. Call the heartbeat tool to report your decision."},
|
||||||
|
{"role": "user", "content": (
|
||||||
|
f"Current Time: {current_time_str(self.timezone)}\n\n"
|
||||||
|
"Review the following HEARTBEAT.md and decide whether there are active tasks.\n\n"
|
||||||
|
f"{content}"
|
||||||
|
)},
|
||||||
|
],
|
||||||
|
tools=_HEARTBEAT_TOOL,
|
||||||
|
model=llm.model,
|
||||||
|
)
|
||||||
|
|
||||||
|
if not response.should_execute_tools:
|
||||||
|
if response.has_tool_calls:
|
||||||
|
logger.warning(
|
||||||
|
"Ignoring heartbeat tool calls under finish_reason='{}'",
|
||||||
|
response.finish_reason,
|
||||||
|
)
|
||||||
|
return "skip", ""
|
||||||
|
|
||||||
|
args = response.tool_calls[0].arguments
|
||||||
|
return args.get("action", "skip"), args.get("tasks", "")
|
||||||
|
|
||||||
|
async def start(self) -> None:
|
||||||
|
"""Start the heartbeat service."""
|
||||||
|
if not self.enabled:
|
||||||
|
logger.info("Heartbeat disabled")
|
||||||
|
return
|
||||||
|
if self._running:
|
||||||
|
logger.warning("Heartbeat already running")
|
||||||
|
return
|
||||||
|
|
||||||
|
self._running = True
|
||||||
|
self._task = asyncio.create_task(self._run_loop())
|
||||||
|
logger.info("Heartbeat started (every {}s)", self.interval_s)
|
||||||
|
|
||||||
|
def stop(self) -> None:
|
||||||
|
"""Stop the heartbeat service."""
|
||||||
|
self._running = False
|
||||||
|
if self._task:
|
||||||
|
self._task.cancel()
|
||||||
|
self._task = None
|
||||||
|
|
||||||
|
async def _run_loop(self) -> None:
|
||||||
|
"""Main heartbeat loop."""
|
||||||
|
while self._running:
|
||||||
|
try:
|
||||||
|
await asyncio.sleep(self.interval_s)
|
||||||
|
if self._running:
|
||||||
|
await self._tick()
|
||||||
|
except asyncio.CancelledError:
|
||||||
|
break
|
||||||
|
except Exception:
|
||||||
|
logger.exception("Heartbeat error")
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _is_deliverable(response: str) -> bool:
|
||||||
|
"""Check if a heartbeat response is suitable for user delivery.
|
||||||
|
|
||||||
|
Filters out two classes of bad output before the evaluator runs:
|
||||||
|
|
||||||
|
1. **Finalization fallback** — the runner hit empty-response retries
|
||||||
|
and produced a canned error message. For heartbeat, empty output
|
||||||
|
is a valid "nothing to report" outcome, not a failure.
|
||||||
|
2. **Leaked reasoning** — the model reflected internal file names,
|
||||||
|
decision logic, or meta-commentary instead of a user-facing report.
|
||||||
|
"""
|
||||||
|
text = response.lower()
|
||||||
|
|
||||||
|
# Runner finalization fallback
|
||||||
|
if "couldn't produce a final answer" in text:
|
||||||
|
return False
|
||||||
|
|
||||||
|
# Leaked internal reasoning patterns
|
||||||
|
leaked_patterns = [
|
||||||
|
"heartbeat.md",
|
||||||
|
"awareness.md",
|
||||||
|
"judgment call:",
|
||||||
|
"decision logic",
|
||||||
|
"valid options are",
|
||||||
|
"my instructions",
|
||||||
|
"i am supposed to",
|
||||||
|
"strict heartbeat interpretation",
|
||||||
|
]
|
||||||
|
if any(pattern in text for pattern in leaked_patterns):
|
||||||
|
return False
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
|
async def _tick(self) -> None:
|
||||||
|
"""Execute a single heartbeat tick."""
|
||||||
|
from nanobot.utils.evaluator import evaluate_response
|
||||||
|
|
||||||
|
content = self._read_heartbeat_file()
|
||||||
|
if not content:
|
||||||
|
logger.debug("Heartbeat: HEARTBEAT.md missing or empty")
|
||||||
|
return
|
||||||
|
|
||||||
|
logger.info("Heartbeat: checking for tasks...")
|
||||||
|
|
||||||
|
try:
|
||||||
|
action, tasks = await self._decide(content)
|
||||||
|
|
||||||
|
if action != "run":
|
||||||
|
logger.info("Heartbeat: OK (nothing to report)")
|
||||||
|
return
|
||||||
|
|
||||||
|
logger.info("Heartbeat: tasks found, executing...")
|
||||||
|
if self.on_execute:
|
||||||
|
response = await self.on_execute(tasks)
|
||||||
|
|
||||||
|
if not response:
|
||||||
|
logger.info("Heartbeat: no response from execution")
|
||||||
|
return
|
||||||
|
|
||||||
|
if not self._is_deliverable(response):
|
||||||
|
logger.info(
|
||||||
|
"Heartbeat: suppressed non-deliverable response ({})",
|
||||||
|
response[:80],
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
llm = self._llm_runtime()
|
||||||
|
should_notify = await evaluate_response(
|
||||||
|
response, tasks, llm.provider, llm.model,
|
||||||
|
)
|
||||||
|
if should_notify and self.on_notify:
|
||||||
|
logger.info("Heartbeat: completed, delivering response")
|
||||||
|
await self.on_notify(response)
|
||||||
|
else:
|
||||||
|
logger.info("Heartbeat: silenced by post-run evaluation")
|
||||||
|
except Exception:
|
||||||
|
logger.exception("Heartbeat execution failed")
|
||||||
|
|
||||||
|
async def trigger_now(self) -> str | None:
|
||||||
|
"""Manually trigger a heartbeat."""
|
||||||
|
content = self._read_heartbeat_file()
|
||||||
|
if not content:
|
||||||
|
return None
|
||||||
|
action, tasks = await self._decide(content)
|
||||||
|
if action != "run" or not self.on_execute:
|
||||||
|
return None
|
||||||
|
return await self.on_execute(tasks)
|
||||||
@@ -14,6 +14,7 @@ __all__ = [
|
|||||||
"OpenAICompatProvider",
|
"OpenAICompatProvider",
|
||||||
"OpenAICodexProvider",
|
"OpenAICodexProvider",
|
||||||
"GitHubCopilotProvider",
|
"GitHubCopilotProvider",
|
||||||
|
"XaiOAuthProvider",
|
||||||
"AzureOpenAIProvider",
|
"AzureOpenAIProvider",
|
||||||
"BedrockProvider",
|
"BedrockProvider",
|
||||||
]
|
]
|
||||||
@@ -23,10 +24,23 @@ _LAZY_IMPORTS = {
|
|||||||
"OpenAICompatProvider": ".openai_compat_provider",
|
"OpenAICompatProvider": ".openai_compat_provider",
|
||||||
"OpenAICodexProvider": ".openai_codex_provider",
|
"OpenAICodexProvider": ".openai_codex_provider",
|
||||||
"GitHubCopilotProvider": ".github_copilot_provider",
|
"GitHubCopilotProvider": ".github_copilot_provider",
|
||||||
|
"XaiOAuthProvider": ".xai_oauth_provider",
|
||||||
"AzureOpenAIProvider": ".azure_openai_provider",
|
"AzureOpenAIProvider": ".azure_openai_provider",
|
||||||
"BedrockProvider": ".bedrock_provider",
|
"BedrockProvider": ".bedrock_provider",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
_LAZY_SUBMODULES = {
|
||||||
|
"anthropic_provider": ".anthropic_provider",
|
||||||
|
"openai_compat_provider": ".openai_compat_provider",
|
||||||
|
"openai_codex_provider": ".openai_codex_provider",
|
||||||
|
"github_copilot_provider": ".github_copilot_provider",
|
||||||
|
"xai_oauth_provider": ".xai_oauth_provider",
|
||||||
|
"azure_openai_provider": ".azure_openai_provider",
|
||||||
|
"bedrock_provider": ".bedrock_provider",
|
||||||
|
"factory": ".factory",
|
||||||
|
"registry": ".registry",
|
||||||
|
}
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from nanobot.providers.anthropic_provider import AnthropicProvider
|
from nanobot.providers.anthropic_provider import AnthropicProvider
|
||||||
from nanobot.providers.azure_openai_provider import AzureOpenAIProvider
|
from nanobot.providers.azure_openai_provider import AzureOpenAIProvider
|
||||||
@@ -34,12 +48,18 @@ if TYPE_CHECKING:
|
|||||||
from nanobot.providers.github_copilot_provider import GitHubCopilotProvider
|
from nanobot.providers.github_copilot_provider import GitHubCopilotProvider
|
||||||
from nanobot.providers.openai_compat_provider import OpenAICompatProvider
|
from nanobot.providers.openai_compat_provider import OpenAICompatProvider
|
||||||
from nanobot.providers.openai_codex_provider import OpenAICodexProvider
|
from nanobot.providers.openai_codex_provider import OpenAICodexProvider
|
||||||
|
from nanobot.providers.xai_oauth_provider import XaiOAuthProvider
|
||||||
|
|
||||||
|
|
||||||
def __getattr__(name: str):
|
def __getattr__(name: str):
|
||||||
"""Lazily expose provider implementations without importing all backends up front."""
|
"""Lazily expose provider implementations without importing all backends up front."""
|
||||||
module_name = _LAZY_IMPORTS.get(name)
|
module_name = _LAZY_IMPORTS.get(name)
|
||||||
if module_name is None:
|
if module_name is not None:
|
||||||
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
|
|
||||||
module = import_module(module_name, __name__)
|
module = import_module(module_name, __name__)
|
||||||
return getattr(module, name)
|
return getattr(module, name)
|
||||||
|
module_name = _LAZY_SUBMODULES.get(name)
|
||||||
|
if module_name is not None:
|
||||||
|
module = import_module(module_name, __name__)
|
||||||
|
globals()[name] = module
|
||||||
|
return module
|
||||||
|
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
|
||||||
|
|||||||
@@ -45,21 +45,13 @@ class AnthropicProvider(LLMProvider):
|
|||||||
if api_key:
|
if api_key:
|
||||||
client_kw["api_key"] = api_key
|
client_kw["api_key"] = api_key
|
||||||
if api_base:
|
if api_base:
|
||||||
client_kw["base_url"] = self._normalize_base_url(api_base)
|
client_kw["base_url"] = api_base
|
||||||
if extra_headers:
|
if extra_headers:
|
||||||
client_kw["default_headers"] = extra_headers
|
client_kw["default_headers"] = extra_headers
|
||||||
# Keep retries centralized in LLMProvider._run_with_retry to avoid retry amplification.
|
# Keep retries centralized in LLMProvider._run_with_retry to avoid retry amplification.
|
||||||
client_kw["max_retries"] = 0
|
client_kw["max_retries"] = 0
|
||||||
self._client = AsyncAnthropic(**client_kw)
|
self._client = AsyncAnthropic(**client_kw)
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _normalize_base_url(api_base: str) -> str:
|
|
||||||
"""Anthropic SDK appends /v1 to request paths internally."""
|
|
||||||
normalized = api_base.rstrip("/")
|
|
||||||
if normalized.endswith("/v1"):
|
|
||||||
return normalized[: -len("/v1")]
|
|
||||||
return normalized
|
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def _handle_error(cls, e: Exception) -> LLMResponse:
|
def _handle_error(cls, e: Exception) -> LLMResponse:
|
||||||
response = getattr(e, "response", None)
|
response = getattr(e, "response", None)
|
||||||
@@ -236,13 +228,6 @@ class AnthropicProvider(LLMProvider):
|
|||||||
if converted:
|
if converted:
|
||||||
result.append(converted)
|
result.append(converted)
|
||||||
continue
|
continue
|
||||||
if not item.get("type"):
|
|
||||||
# Anthropic requires every content block to declare a "type".
|
|
||||||
# A tool that returned a bare dict (or a list of dicts) lands
|
|
||||||
# here; coerce it to a text block instead of emitting a block
|
|
||||||
# the API rejects with "content.0.type: Field required".
|
|
||||||
result.append({"type": "text", "text": str(item)})
|
|
||||||
continue
|
|
||||||
result.append(item)
|
result.append(item)
|
||||||
return result or "(empty)"
|
return result or "(empty)"
|
||||||
|
|
||||||
|
|||||||
@@ -315,29 +315,6 @@ class LLMProvider(ABC):
|
|||||||
|
|
||||||
return cls._is_transient_error(response.content)
|
return cls._is_transient_error(response.content)
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def is_arrearage_response(cls, response: LLMResponse) -> bool:
|
|
||||||
"""Detect API-key arrearage / quota / billing errors that won't clear on retry.
|
|
||||||
|
|
||||||
These surface as HTTP 402 or as billing semantic tokens (e.g.
|
|
||||||
``insufficient_quota``, ``payment_required``); reuses the same token and
|
|
||||||
text markers the 429 retry policy treats as non-retryable.
|
|
||||||
"""
|
|
||||||
if response.error_status_code is not None and int(response.error_status_code) == 402:
|
|
||||||
return True
|
|
||||||
|
|
||||||
type_token = cls._normalize_error_token(response.error_type)
|
|
||||||
code_token = cls._normalize_error_token(response.error_code)
|
|
||||||
if any(
|
|
||||||
token in cls._NON_RETRYABLE_429_ERROR_TOKENS
|
|
||||||
for token in (type_token, code_token)
|
|
||||||
if token is not None
|
|
||||||
):
|
|
||||||
return True
|
|
||||||
|
|
||||||
content = (response.content or "").lower()
|
|
||||||
return any(marker in content for marker in cls._NON_RETRYABLE_429_TEXT_MARKERS)
|
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _normalize_error_token(value: Any) -> str | None:
|
def _normalize_error_token(value: Any) -> str | None:
|
||||||
if value is None:
|
if value is None:
|
||||||
@@ -580,20 +557,11 @@ class LLMProvider(ABC):
|
|||||||
if reasoning_effort is self._SENTINEL:
|
if reasoning_effort is self._SENTINEL:
|
||||||
reasoning_effort = self.generation.reasoning_effort
|
reasoning_effort = self.generation.reasoning_effort
|
||||||
|
|
||||||
has_streamed_content = False
|
|
||||||
|
|
||||||
async def _tracking_delta(text: str) -> None:
|
|
||||||
nonlocal has_streamed_content
|
|
||||||
if text:
|
|
||||||
has_streamed_content = True
|
|
||||||
if on_content_delta:
|
|
||||||
await on_content_delta(text)
|
|
||||||
|
|
||||||
kw: dict[str, Any] = dict(
|
kw: dict[str, Any] = dict(
|
||||||
messages=messages, tools=tools, model=model,
|
messages=messages, tools=tools, model=model,
|
||||||
max_tokens=max_tokens, temperature=temperature,
|
max_tokens=max_tokens, temperature=temperature,
|
||||||
reasoning_effort=reasoning_effort, tool_choice=tool_choice,
|
reasoning_effort=reasoning_effort, tool_choice=tool_choice,
|
||||||
on_content_delta=_tracking_delta if on_content_delta is not None else None,
|
on_content_delta=on_content_delta,
|
||||||
on_thinking_delta=on_thinking_delta,
|
on_thinking_delta=on_thinking_delta,
|
||||||
on_tool_call_delta=on_tool_call_delta,
|
on_tool_call_delta=on_tool_call_delta,
|
||||||
)
|
)
|
||||||
@@ -603,7 +571,6 @@ class LLMProvider(ABC):
|
|||||||
messages,
|
messages,
|
||||||
retry_mode=retry_mode,
|
retry_mode=retry_mode,
|
||||||
on_retry_wait=on_retry_wait,
|
on_retry_wait=on_retry_wait,
|
||||||
should_retry_guard=lambda: not has_streamed_content,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
async def chat_with_retry(
|
async def chat_with_retry(
|
||||||
@@ -750,7 +717,6 @@ class LLMProvider(ABC):
|
|||||||
*,
|
*,
|
||||||
retry_mode: str,
|
retry_mode: str,
|
||||||
on_retry_wait: Callable[[str], Awaitable[None]] | None,
|
on_retry_wait: Callable[[str], Awaitable[None]] | None,
|
||||||
should_retry_guard: Callable[[], bool] | None = None,
|
|
||||||
) -> LLMResponse:
|
) -> LLMResponse:
|
||||||
attempt = 0
|
attempt = 0
|
||||||
delays = list(self._CHAT_RETRY_DELAYS)
|
delays = list(self._CHAT_RETRY_DELAYS)
|
||||||
@@ -764,11 +730,6 @@ class LLMProvider(ABC):
|
|||||||
if response.finish_reason != "error":
|
if response.finish_reason != "error":
|
||||||
return response
|
return response
|
||||||
last_response = response
|
last_response = response
|
||||||
if should_retry_guard is not None and not should_retry_guard():
|
|
||||||
logger.warning(
|
|
||||||
"LLM stream failed after content was emitted; skipping retry"
|
|
||||||
)
|
|
||||||
return response
|
|
||||||
error_key = ((response.content or "").strip().lower() or None)
|
error_key = ((response.content or "").strip().lower() or None)
|
||||||
if error_key and error_key == last_error_key:
|
if error_key and error_key == last_error_key:
|
||||||
identical_error_count += 1
|
identical_error_count += 1
|
||||||
|
|||||||
@@ -68,6 +68,10 @@ def _make_provider_core(
|
|||||||
from nanobot.providers.github_copilot_provider import GitHubCopilotProvider
|
from nanobot.providers.github_copilot_provider import GitHubCopilotProvider
|
||||||
|
|
||||||
provider = GitHubCopilotProvider(default_model=model)
|
provider = GitHubCopilotProvider(default_model=model)
|
||||||
|
elif backend == "xai_oauth":
|
||||||
|
from nanobot.providers.xai_oauth_provider import XaiOAuthProvider
|
||||||
|
|
||||||
|
provider = XaiOAuthProvider(default_model=model, config=p)
|
||||||
elif backend == "anthropic":
|
elif backend == "anthropic":
|
||||||
from nanobot.providers.anthropic_provider import AnthropicProvider
|
from nanobot.providers.anthropic_provider import AnthropicProvider
|
||||||
|
|
||||||
@@ -98,7 +102,6 @@ def _make_provider_core(
|
|||||||
extra_headers=p.extra_headers if p else None,
|
extra_headers=p.extra_headers if p else None,
|
||||||
spec=spec,
|
spec=spec,
|
||||||
extra_body=p.extra_body if p else None,
|
extra_body=p.extra_body if p else None,
|
||||||
api_type=p.api_type if p and provider_name == "openai" else "auto",
|
|
||||||
)
|
)
|
||||||
|
|
||||||
provider.generation = resolved.to_generation_settings()
|
provider.generation = resolved.to_generation_settings()
|
||||||
@@ -184,7 +187,6 @@ def provider_signature(
|
|||||||
config.get_api_base(fallback.model, preset=fallback),
|
config.get_api_base(fallback.model, preset=fallback),
|
||||||
fp.extra_headers if fp else None,
|
fp.extra_headers if fp else None,
|
||||||
fp.extra_body if fp else None,
|
fp.extra_body if fp else None,
|
||||||
fp.api_type if fp else "auto",
|
|
||||||
getattr(fp, "region", None) if fp else None,
|
getattr(fp, "region", None) if fp else None,
|
||||||
getattr(fp, "profile", None) if fp else None,
|
getattr(fp, "profile", None) if fp else None,
|
||||||
fallback.max_tokens,
|
fallback.max_tokens,
|
||||||
@@ -201,7 +203,6 @@ def provider_signature(
|
|||||||
config.get_api_base(resolved.model, preset=resolved),
|
config.get_api_base(resolved.model, preset=resolved),
|
||||||
p.extra_headers if p else None,
|
p.extra_headers if p else None,
|
||||||
p.extra_body if p else None,
|
p.extra_body if p else None,
|
||||||
p.api_type if p else "auto",
|
|
||||||
getattr(p, "region", None) if p else None,
|
getattr(p, "region", None) if p else None,
|
||||||
getattr(p, "profile", None) if p else None,
|
getattr(p, "profile", None) if p else None,
|
||||||
resolved.max_tokens,
|
resolved.max_tokens,
|
||||||
|
|||||||
@@ -2,10 +2,8 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import base64
|
import base64
|
||||||
import binascii
|
import binascii
|
||||||
import re
|
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
@@ -33,14 +31,6 @@ _AIHUBMIX_ASPECT_RATIO_SIZES = {
|
|||||||
}
|
}
|
||||||
_GEMINI_DEFAULT_TIMEOUT_S = 120.0
|
_GEMINI_DEFAULT_TIMEOUT_S = 120.0
|
||||||
_GEMINI_IMAGEN_ASPECT_RATIOS = {"1:1", "9:16", "16:9", "3:4", "4:3"}
|
_GEMINI_IMAGEN_ASPECT_RATIOS = {"1:1", "9:16", "16:9", "3:4", "4:3"}
|
||||||
_OLLAMA_DEFAULT_SIDE = 1024
|
|
||||||
_OLLAMA_SIZE_PRESETS = {
|
|
||||||
"1K": 1024,
|
|
||||||
"2K": 2048,
|
|
||||||
"4K": 4096,
|
|
||||||
}
|
|
||||||
_OLLAMA_EXPLICIT_SIZE_RE = re.compile(r"^\s*(\d+)\s*[xX]\s*(\d+)\s*$")
|
|
||||||
_OLLAMA_ASPECT_RATIO_RE = re.compile(r"^\s*(\d+)\s*:\s*(\d+)\s*$")
|
|
||||||
|
|
||||||
|
|
||||||
class ImageGenerationError(RuntimeError):
|
class ImageGenerationError(RuntimeError):
|
||||||
@@ -139,11 +129,6 @@ _IMAGE_GEN_PROVIDERS: dict[str, type[ImageGenerationProvider]] = {}
|
|||||||
|
|
||||||
|
|
||||||
def register_image_gen_provider(cls: type[ImageGenerationProvider]) -> None:
|
def register_image_gen_provider(cls: type[ImageGenerationProvider]) -> None:
|
||||||
"""Register an image provider at import time only.
|
|
||||||
|
|
||||||
The registry is populated by module side effects so provider discovery
|
|
||||||
stays lazy and consistent across the process.
|
|
||||||
"""
|
|
||||||
name = cls.provider_name
|
name = cls.provider_name
|
||||||
if not name:
|
if not name:
|
||||||
raise ValueError(f"{cls.__name__} must set provider_name")
|
raise ValueError(f"{cls.__name__} must set provider_name")
|
||||||
@@ -234,10 +219,7 @@ class ImageGenerationProvider(ABC):
|
|||||||
*,
|
*,
|
||||||
headers: dict[str, str],
|
headers: dict[str, str],
|
||||||
body: dict[str, Any],
|
body: dict[str, Any],
|
||||||
client: httpx.AsyncClient | None = None,
|
|
||||||
) -> httpx.Response:
|
) -> httpx.Response:
|
||||||
if client is not None:
|
|
||||||
return await client.post(url, headers=headers, json=body)
|
|
||||||
if self._client is not None:
|
if self._client is not None:
|
||||||
return await self._client.post(url, headers=headers, json=body)
|
return await self._client.post(url, headers=headers, json=body)
|
||||||
async with httpx.AsyncClient(timeout=self.timeout) as c:
|
async with httpx.AsyncClient(timeout=self.timeout) as c:
|
||||||
@@ -408,11 +390,10 @@ class AIHubMixImageGenerationClient(ImageGenerationProvider):
|
|||||||
model_path = _aihubmix_model_path(model)
|
model_path = _aihubmix_model_path(model)
|
||||||
url = f"{self.api_base}/models/{model_path}/predictions"
|
url = f"{self.api_base}/models/{model_path}/predictions"
|
||||||
try:
|
try:
|
||||||
response = await self._http_post(
|
response = await client.post(
|
||||||
url,
|
url,
|
||||||
headers={**headers, "Content-Type": "application/json"},
|
headers={**headers, "Content-Type": "application/json"},
|
||||||
body=body,
|
json=body,
|
||||||
client=client,
|
|
||||||
)
|
)
|
||||||
except httpx.TimeoutException as exc:
|
except httpx.TimeoutException as exc:
|
||||||
raise ImageGenerationError("AIHubMix image generation timed out") from exc
|
raise ImageGenerationError("AIHubMix image generation timed out") from exc
|
||||||
@@ -448,139 +429,6 @@ def _http_error_detail(response: httpx.Response) -> str:
|
|||||||
return response.text[:500] or "<empty response body>"
|
return response.text[:500] or "<empty response body>"
|
||||||
|
|
||||||
|
|
||||||
def _round_to_multiple(value: float, multiple: int = 8) -> int:
|
|
||||||
rounded = int(round(value / multiple) * multiple)
|
|
||||||
return max(multiple, rounded)
|
|
||||||
|
|
||||||
|
|
||||||
def _ollama_dimensions(aspect_ratio: str | None, image_size: str | None) -> tuple[int, int]:
|
|
||||||
if image_size:
|
|
||||||
size = image_size.strip()
|
|
||||||
explicit = _OLLAMA_EXPLICIT_SIZE_RE.fullmatch(size)
|
|
||||||
if explicit:
|
|
||||||
return int(explicit.group(1)), int(explicit.group(2))
|
|
||||||
long_side = _OLLAMA_SIZE_PRESETS.get(size.upper(), _OLLAMA_DEFAULT_SIDE)
|
|
||||||
else:
|
|
||||||
long_side = _OLLAMA_DEFAULT_SIDE
|
|
||||||
|
|
||||||
if not aspect_ratio:
|
|
||||||
return long_side, long_side
|
|
||||||
|
|
||||||
ratio = _OLLAMA_ASPECT_RATIO_RE.fullmatch(aspect_ratio.strip())
|
|
||||||
if ratio is None:
|
|
||||||
return long_side, long_side
|
|
||||||
|
|
||||||
width_ratio = int(ratio.group(1))
|
|
||||||
height_ratio = int(ratio.group(2))
|
|
||||||
if width_ratio <= 0 or height_ratio <= 0:
|
|
||||||
return long_side, long_side
|
|
||||||
|
|
||||||
if width_ratio >= height_ratio:
|
|
||||||
width = long_side
|
|
||||||
height = _round_to_multiple(long_side * height_ratio / width_ratio)
|
|
||||||
else:
|
|
||||||
height = long_side
|
|
||||||
width = _round_to_multiple(long_side * width_ratio / height_ratio)
|
|
||||||
return max(8, width), max(8, height)
|
|
||||||
|
|
||||||
|
|
||||||
def _ollama_image_data_url(value: str) -> str:
|
|
||||||
if value.startswith("data:image/"):
|
|
||||||
return value
|
|
||||||
return _b64_image_data_url(value)
|
|
||||||
|
|
||||||
|
|
||||||
def _ollama_images_from_payload(payload: dict[str, Any]) -> list[str]:
|
|
||||||
images: list[str] = []
|
|
||||||
|
|
||||||
def collect(value: Any) -> None:
|
|
||||||
if isinstance(value, str) and value:
|
|
||||||
images.append(_ollama_image_data_url(value))
|
|
||||||
elif isinstance(value, list):
|
|
||||||
for item in value:
|
|
||||||
collect(item)
|
|
||||||
|
|
||||||
collect(payload.get("image"))
|
|
||||||
collect(payload.get("images"))
|
|
||||||
return images
|
|
||||||
|
|
||||||
|
|
||||||
class OllamaImageGenerationClient(ImageGenerationProvider):
|
|
||||||
"""Async client for Ollama native image generation models."""
|
|
||||||
|
|
||||||
provider_name = "ollama"
|
|
||||||
default_timeout = 300.0
|
|
||||||
|
|
||||||
def _default_base_url(self) -> str:
|
|
||||||
return "http://localhost:11434/api"
|
|
||||||
|
|
||||||
def _resolve_base_url(self, api_base: str | None) -> str:
|
|
||||||
if api_base:
|
|
||||||
base = api_base.rstrip("/")
|
|
||||||
if base.endswith("/v1"):
|
|
||||||
return f"{base[:-3]}/api"
|
|
||||||
return base
|
|
||||||
return self._default_base_url()
|
|
||||||
|
|
||||||
async def generate(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
prompt: str,
|
|
||||||
model: str,
|
|
||||||
reference_images: list[str] | None = None,
|
|
||||||
aspect_ratio: str | None = None,
|
|
||||||
image_size: str | None = None,
|
|
||||||
) -> GeneratedImageResponse:
|
|
||||||
if reference_images:
|
|
||||||
raise ImageGenerationError(
|
|
||||||
"Ollama image generation does not support reference images"
|
|
||||||
)
|
|
||||||
|
|
||||||
width, height = _ollama_dimensions(aspect_ratio, image_size)
|
|
||||||
body: dict[str, Any] = {
|
|
||||||
"model": model,
|
|
||||||
"prompt": prompt,
|
|
||||||
"width": width,
|
|
||||||
"height": height,
|
|
||||||
"steps": 0,
|
|
||||||
}
|
|
||||||
body.update(self.extra_body)
|
|
||||||
body["stream"] = False
|
|
||||||
|
|
||||||
headers = {
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
**self.extra_headers,
|
|
||||||
}
|
|
||||||
if self.api_key:
|
|
||||||
headers["Authorization"] = f"Bearer {self.api_key}"
|
|
||||||
|
|
||||||
url = f"{self.api_base}/generate"
|
|
||||||
response = await self._http_post(url, headers=headers, body=body)
|
|
||||||
|
|
||||||
try:
|
|
||||||
response.raise_for_status()
|
|
||||||
except httpx.HTTPStatusError as exc:
|
|
||||||
detail = _http_error_detail(response)
|
|
||||||
logger.error(
|
|
||||||
"Ollama image generation failed (HTTP {}): {}",
|
|
||||||
response.status_code,
|
|
||||||
detail,
|
|
||||||
)
|
|
||||||
raise ImageGenerationError(
|
|
||||||
f"Ollama image generation failed (HTTP {response.status_code}): {detail}"
|
|
||||||
) from exc
|
|
||||||
|
|
||||||
data = response.json()
|
|
||||||
images = _ollama_images_from_payload(data)
|
|
||||||
|
|
||||||
self._require_images(images, data)
|
|
||||||
|
|
||||||
response_text = data.get("response")
|
|
||||||
content = response_text if isinstance(response_text, str) else ""
|
|
||||||
|
|
||||||
return GeneratedImageResponse(images=images, content=content, raw=data)
|
|
||||||
|
|
||||||
|
|
||||||
class GeminiImageGenerationClient(ImageGenerationProvider):
|
class GeminiImageGenerationClient(ImageGenerationProvider):
|
||||||
"""Async client for Gemini/Imagen image generation via the Generative Language API."""
|
"""Async client for Gemini/Imagen image generation via the Generative Language API."""
|
||||||
|
|
||||||
@@ -594,9 +442,9 @@ class GeminiImageGenerationClient(ImageGenerationProvider):
|
|||||||
return "https://generativelanguage.googleapis.com/v1beta"
|
return "https://generativelanguage.googleapis.com/v1beta"
|
||||||
|
|
||||||
def _resolve_base_url(self, api_base: str | None) -> str:
|
def _resolve_base_url(self, api_base: str | None) -> str:
|
||||||
# Gemini chat completions use the registry's OpenAI-compatible shim.
|
# The Gemini provider's registry default_api_base is the OpenAI-compat
|
||||||
# Image generation must hit the native Generative Language API, so we
|
# shim (.../v1beta/openai/), which has no image endpoints.
|
||||||
# intentionally bypass the shared registry lookup here.
|
# Skip the registry lookup and use the native API base directly.
|
||||||
if api_base:
|
if api_base:
|
||||||
return api_base.rstrip("/")
|
return api_base.rstrip("/")
|
||||||
return self._default_base_url()
|
return self._default_base_url()
|
||||||
@@ -858,16 +706,22 @@ class MiniMaxImageGenerationClient(ImageGenerationProvider):
|
|||||||
|
|
||||||
body.update(self.extra_body)
|
body.update(self.extra_body)
|
||||||
|
|
||||||
return await self._generate_with_client(body, headers)
|
client = self._client or httpx.AsyncClient(timeout=self.timeout)
|
||||||
|
try:
|
||||||
|
return await self._generate_with_client(client, body, headers)
|
||||||
|
finally:
|
||||||
|
if self._client is None:
|
||||||
|
await client.aclose()
|
||||||
|
|
||||||
async def _generate_with_client(
|
async def _generate_with_client(
|
||||||
self,
|
self,
|
||||||
|
client: httpx.AsyncClient,
|
||||||
body: dict[str, Any],
|
body: dict[str, Any],
|
||||||
headers: dict[str, str],
|
headers: dict[str, str],
|
||||||
) -> GeneratedImageResponse:
|
) -> GeneratedImageResponse:
|
||||||
url = f"{self.api_base}/image_generation"
|
url = f"{self.api_base}/image_generation"
|
||||||
try:
|
try:
|
||||||
response = await self._http_post(url, headers=headers, body=body)
|
response = await client.post(url, headers=headers, json=body)
|
||||||
except httpx.TimeoutException as exc:
|
except httpx.TimeoutException as exc:
|
||||||
raise ImageGenerationError("MiniMax image generation timed out") from exc
|
raise ImageGenerationError("MiniMax image generation timed out") from exc
|
||||||
except httpx.RequestError as exc:
|
except httpx.RequestError as exc:
|
||||||
@@ -902,426 +756,6 @@ def _minimax_images_from_payload(payload: dict[str, Any]) -> list[str]:
|
|||||||
return images
|
return images
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# OpenAI image generation
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
_OPENAI_DALLE2_SUPPORTED_SIZES = {"256x256", "512x512", "1024x1024"}
|
|
||||||
_OPENAI_DALLE3_SUPPORTED_SIZES = {"1024x1024", "1792x1024", "1024x1792"}
|
|
||||||
_OPENAI_GPT_IMAGE_SUPPORTED_SIZES = {
|
|
||||||
"1024x1024",
|
|
||||||
"1536x1024",
|
|
||||||
"1024x1536",
|
|
||||||
"auto",
|
|
||||||
}
|
|
||||||
_OPENAI_DALLE2_ASPECT_RATIO_SIZES = {
|
|
||||||
"1:1": "1024x1024",
|
|
||||||
"16:9": "1024x1024",
|
|
||||||
"9:16": "1024x1024",
|
|
||||||
"3:4": "1024x1024",
|
|
||||||
"4:3": "1024x1024",
|
|
||||||
}
|
|
||||||
_OPENAI_DALLE3_ASPECT_RATIO_SIZES = {
|
|
||||||
"1:1": "1024x1024",
|
|
||||||
"16:9": "1792x1024",
|
|
||||||
"9:16": "1024x1792",
|
|
||||||
"3:4": "1024x1792",
|
|
||||||
"4:3": "1792x1024",
|
|
||||||
}
|
|
||||||
_OPENAI_GPT_IMAGE_ASPECT_RATIO_SIZES = {
|
|
||||||
"1:1": "1024x1024",
|
|
||||||
"16:9": "1536x1024",
|
|
||||||
"9:16": "1024x1536",
|
|
||||||
"3:4": "1024x1536",
|
|
||||||
"4:3": "1536x1024",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class OpenAIImageGenerationClient(ImageGenerationProvider):
|
|
||||||
"""OpenAI Images API using an API key (``providers.openai.apiKey``)."""
|
|
||||||
|
|
||||||
provider_name = "openai"
|
|
||||||
missing_key_message = (
|
|
||||||
"OpenAI API key is not configured. Set providers.openai.apiKey."
|
|
||||||
)
|
|
||||||
|
|
||||||
def _default_base_url(self) -> str:
|
|
||||||
return "https://api.openai.com/v1"
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _strip_model_prefix(model: str) -> str:
|
|
||||||
"""Remove ``openai/`` prefix if present (OpenRouter convention)."""
|
|
||||||
if model.startswith("openai/") or model.startswith("openai_codex/"):
|
|
||||||
return model.split("/", 1)[1]
|
|
||||||
return model
|
|
||||||
|
|
||||||
async def generate(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
prompt: str,
|
|
||||||
model: str,
|
|
||||||
reference_images: list[str] | None = None,
|
|
||||||
aspect_ratio: str | None = None,
|
|
||||||
image_size: str | None = None,
|
|
||||||
) -> GeneratedImageResponse:
|
|
||||||
if not self.api_key:
|
|
||||||
raise ImageGenerationError(self.missing_key_message)
|
|
||||||
|
|
||||||
if reference_images:
|
|
||||||
logger.warning(
|
|
||||||
"DALL-E models do not support reference images; "
|
|
||||||
"ignoring {} reference image(s) for {}",
|
|
||||||
len(reference_images),
|
|
||||||
model,
|
|
||||||
)
|
|
||||||
|
|
||||||
headers = {
|
|
||||||
"Authorization": f"Bearer {self.api_key}",
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
**self.extra_headers,
|
|
||||||
}
|
|
||||||
|
|
||||||
clean_model = self._strip_model_prefix(model)
|
|
||||||
body: dict[str, Any] = {
|
|
||||||
"model": clean_model,
|
|
||||||
"prompt": prompt,
|
|
||||||
}
|
|
||||||
|
|
||||||
if not _openai_is_gpt_image_model(clean_model):
|
|
||||||
body["response_format"] = "b64_json"
|
|
||||||
body["n"] = 1
|
|
||||||
|
|
||||||
size = _openai_size(clean_model, aspect_ratio, image_size)
|
|
||||||
if size:
|
|
||||||
body["size"] = size
|
|
||||||
|
|
||||||
body.update(self.extra_body)
|
|
||||||
|
|
||||||
logger.info("OpenAI Images API request: POST {}/images/generations body={}", self.api_base, body)
|
|
||||||
|
|
||||||
response = await self._http_post(
|
|
||||||
f"{self.api_base}/images/generations",
|
|
||||||
headers=headers,
|
|
||||||
body=body,
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
response.raise_for_status()
|
|
||||||
except httpx.HTTPStatusError as exc:
|
|
||||||
detail = response.text[:1000]
|
|
||||||
logger.error("OpenAI Images API error ({}): {}", response.status_code, detail)
|
|
||||||
raise ImageGenerationError(
|
|
||||||
f"OpenAI image generation failed (HTTP {response.status_code}): {detail}"
|
|
||||||
) from exc
|
|
||||||
|
|
||||||
payload = response.json()
|
|
||||||
logger.info("OpenAI Images API response ({}): {}", response.status_code,
|
|
||||||
{k: v for k, v in payload.items() if k != "data"})
|
|
||||||
|
|
||||||
client = self._client
|
|
||||||
owns_client = client is None
|
|
||||||
if owns_client:
|
|
||||||
client = httpx.AsyncClient(timeout=self.timeout)
|
|
||||||
try:
|
|
||||||
images = await _openai_images_from_payload(client, payload)
|
|
||||||
finally:
|
|
||||||
if owns_client:
|
|
||||||
await client.aclose()
|
|
||||||
|
|
||||||
self._require_images(images, payload)
|
|
||||||
|
|
||||||
return GeneratedImageResponse(images=images, content="", raw=payload)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# OpenAI Codex image generation
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class CodexImageGenerationClient(ImageGenerationProvider):
|
|
||||||
"""OpenAI image generation via Codex subscription OAuth.
|
|
||||||
|
|
||||||
Uses the Codex Responses API with the ``image_generation`` tool
|
|
||||||
(the same mechanism ChatGPT uses internally). No API key required —
|
|
||||||
the Codex OAuth token from ``oauth_cli_kit`` is used instead.
|
|
||||||
"""
|
|
||||||
|
|
||||||
provider_name = "openai_codex"
|
|
||||||
missing_key_message = (
|
|
||||||
"Codex OAuth token is unavailable. "
|
|
||||||
"Log in with Codex subscription first."
|
|
||||||
)
|
|
||||||
|
|
||||||
def _default_base_url(self) -> str:
|
|
||||||
return "https://chatgpt.com/backend-api"
|
|
||||||
|
|
||||||
def _codex_model(self, model: str) -> str:
|
|
||||||
"""Strip the ``openai-codex/`` prefix if present."""
|
|
||||||
if model.startswith(("openai-codex/", "openai_codex/")):
|
|
||||||
return model.split("/", 1)[1]
|
|
||||||
return model
|
|
||||||
|
|
||||||
async def generate(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
prompt: str,
|
|
||||||
model: str,
|
|
||||||
reference_images: list[str] | None = None,
|
|
||||||
aspect_ratio: str | None = None,
|
|
||||||
image_size: str | None = None,
|
|
||||||
) -> GeneratedImageResponse:
|
|
||||||
try:
|
|
||||||
from oauth_cli_kit import get_token as get_codex_token
|
|
||||||
except ImportError:
|
|
||||||
raise ImageGenerationError(self.missing_key_message)
|
|
||||||
|
|
||||||
try:
|
|
||||||
token = await asyncio.to_thread(get_codex_token)
|
|
||||||
except Exception as exc:
|
|
||||||
raise ImageGenerationError(self.missing_key_message) from exc
|
|
||||||
if not token or not token.access:
|
|
||||||
raise ImageGenerationError(self.missing_key_message)
|
|
||||||
|
|
||||||
logger.info(
|
|
||||||
"Using Codex OAuth token for image generation (account: {})",
|
|
||||||
token.account_id,
|
|
||||||
)
|
|
||||||
|
|
||||||
if reference_images:
|
|
||||||
logger.warning(
|
|
||||||
"Codex image generation does not support reference images; "
|
|
||||||
"ignoring {} reference image(s)",
|
|
||||||
len(reference_images),
|
|
||||||
)
|
|
||||||
|
|
||||||
headers = {
|
|
||||||
"Authorization": f"Bearer {token.access}",
|
|
||||||
"chatgpt-account-id": token.account_id,
|
|
||||||
"OpenAI-Beta": "responses=experimental",
|
|
||||||
"originator": "nanobot",
|
|
||||||
"User-Agent": "nanobot (python)",
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
**self.extra_headers,
|
|
||||||
}
|
|
||||||
|
|
||||||
body: dict[str, Any] = {
|
|
||||||
"model": self._codex_model(model),
|
|
||||||
"instructions": "Generate an image based on the user's request.",
|
|
||||||
"input": [{"role": "user", "content": prompt}],
|
|
||||||
"tools": [{"type": "image_generation"}],
|
|
||||||
"tool_choice": "auto",
|
|
||||||
"stream": True,
|
|
||||||
"store": False,
|
|
||||||
}
|
|
||||||
body.update(self.extra_body)
|
|
||||||
|
|
||||||
logger.info("Codex Responses API request: POST {}/codex/responses body={}",
|
|
||||||
self.api_base, {k: v for k, v in body.items() if k != "input"})
|
|
||||||
|
|
||||||
response = await self._http_post(
|
|
||||||
f"{self.api_base}/codex/responses",
|
|
||||||
headers=headers,
|
|
||||||
body=body,
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
response.raise_for_status()
|
|
||||||
except httpx.HTTPStatusError as exc:
|
|
||||||
detail = response.text[:1000]
|
|
||||||
logger.error("Codex Responses API error ({}): {}", response.status_code, detail)
|
|
||||||
raise ImageGenerationError(
|
|
||||||
f"Codex image generation failed (HTTP {response.status_code}): {detail}"
|
|
||||||
) from exc
|
|
||||||
|
|
||||||
images, content_text = await _parse_codex_sse_images(response)
|
|
||||||
|
|
||||||
raw = {"status": "completed"}
|
|
||||||
self._require_images(images, raw)
|
|
||||||
|
|
||||||
return GeneratedImageResponse(images=images, content=content_text, raw=raw)
|
|
||||||
|
|
||||||
|
|
||||||
def _openai_size(
|
|
||||||
model: str,
|
|
||||||
aspect_ratio: str | None,
|
|
||||||
image_size: str | None,
|
|
||||||
) -> str:
|
|
||||||
"""Resolve aspect ratio or image_size to an OpenAI Images API size string."""
|
|
||||||
sizes, supported_sizes = _openai_size_options(model)
|
|
||||||
explicit_size = _normalize_openai_image_size(image_size)
|
|
||||||
if explicit_size and _openai_explicit_size_supported(
|
|
||||||
explicit_size,
|
|
||||||
supported_sizes=supported_sizes,
|
|
||||||
):
|
|
||||||
return explicit_size
|
|
||||||
if explicit_size:
|
|
||||||
logger.warning(
|
|
||||||
"OpenAI image size '{}' is not supported by {}; using aspect ratio/default size",
|
|
||||||
explicit_size,
|
|
||||||
model,
|
|
||||||
)
|
|
||||||
if aspect_ratio and aspect_ratio in sizes:
|
|
||||||
return sizes[aspect_ratio]
|
|
||||||
return "1024x1024"
|
|
||||||
|
|
||||||
|
|
||||||
def _openai_is_gpt_image_model(model: str) -> bool:
|
|
||||||
normalized = model.lower()
|
|
||||||
return normalized.startswith(("gpt-image", "chatgpt-image"))
|
|
||||||
|
|
||||||
|
|
||||||
def _openai_size_options(model: str) -> tuple[dict[str, str], set[str] | None]:
|
|
||||||
normalized = model.lower()
|
|
||||||
if normalized.startswith("dall-e-2"):
|
|
||||||
return _OPENAI_DALLE2_ASPECT_RATIO_SIZES, _OPENAI_DALLE2_SUPPORTED_SIZES
|
|
||||||
if normalized.startswith("dall-e-3"):
|
|
||||||
return _OPENAI_DALLE3_ASPECT_RATIO_SIZES, _OPENAI_DALLE3_SUPPORTED_SIZES
|
|
||||||
if normalized.startswith("gpt-image-2"):
|
|
||||||
return _OPENAI_GPT_IMAGE_ASPECT_RATIO_SIZES, None
|
|
||||||
return _OPENAI_GPT_IMAGE_ASPECT_RATIO_SIZES, _OPENAI_GPT_IMAGE_SUPPORTED_SIZES
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_openai_image_size(image_size: str | None) -> str | None:
|
|
||||||
if not image_size:
|
|
||||||
return None
|
|
||||||
normalized = image_size.strip().lower()
|
|
||||||
return normalized or None
|
|
||||||
|
|
||||||
|
|
||||||
def _openai_explicit_size_supported(
|
|
||||||
size: str,
|
|
||||||
*,
|
|
||||||
supported_sizes: set[str] | None,
|
|
||||||
) -> bool:
|
|
||||||
if supported_sizes is not None:
|
|
||||||
return size in supported_sizes
|
|
||||||
width, sep, height = size.partition("x")
|
|
||||||
return bool(sep and width.isdecimal() and height.isdecimal())
|
|
||||||
|
|
||||||
|
|
||||||
async def _openai_images_from_payload(
|
|
||||||
client: httpx.AsyncClient,
|
|
||||||
payload: dict[str, Any],
|
|
||||||
) -> list[str]:
|
|
||||||
"""Extract images from OpenAI Images API response.
|
|
||||||
|
|
||||||
Handles both ``b64_json`` (preferred) and ``url`` (downloaded) formats.
|
|
||||||
"""
|
|
||||||
images: list[str] = []
|
|
||||||
for item in payload.get("data") or []:
|
|
||||||
if not isinstance(item, dict):
|
|
||||||
continue
|
|
||||||
b64 = item.get("b64_json")
|
|
||||||
if isinstance(b64, str) and b64:
|
|
||||||
images.append(_b64_image_data_url(b64))
|
|
||||||
continue
|
|
||||||
url = item.get("url")
|
|
||||||
if isinstance(url, str) and url:
|
|
||||||
images.append(await _download_image_data_url(client, url))
|
|
||||||
return images
|
|
||||||
|
|
||||||
|
|
||||||
def _codex_responses_images_from_payload(payload: dict[str, Any]) -> list[str]:
|
|
||||||
"""Extract images from Codex Responses API ``image_generation_call`` output."""
|
|
||||||
images: list[str] = []
|
|
||||||
for item in payload.get("output") or []:
|
|
||||||
if not isinstance(item, dict):
|
|
||||||
continue
|
|
||||||
if item.get("type") != "image_generation_call":
|
|
||||||
continue
|
|
||||||
result = item.get("result")
|
|
||||||
if isinstance(result, str):
|
|
||||||
images.append(result if result.startswith("data:image/") else _b64_image_data_url(result))
|
|
||||||
continue
|
|
||||||
if isinstance(result, dict):
|
|
||||||
image_url = result.get("image_url") or result.get("image") or ""
|
|
||||||
if isinstance(image_url, str):
|
|
||||||
images.append(image_url if image_url.startswith("data:image/") else _b64_image_data_url(image_url))
|
|
||||||
return images
|
|
||||||
|
|
||||||
|
|
||||||
async def _parse_codex_sse_images(
|
|
||||||
response: httpx.Response,
|
|
||||||
) -> tuple[list[str], str]:
|
|
||||||
"""Parse a Codex Responses API SSE stream for image generation output.
|
|
||||||
|
|
||||||
Returns ``(images, content_text)``.
|
|
||||||
"""
|
|
||||||
import json as _json
|
|
||||||
|
|
||||||
images: list[str] = []
|
|
||||||
text_parts: list[str] = []
|
|
||||||
|
|
||||||
buffer: list[str] = []
|
|
||||||
async for line_bytes in response.aiter_lines():
|
|
||||||
line = line_bytes.strip()
|
|
||||||
if line == "":
|
|
||||||
if buffer:
|
|
||||||
data_lines = []
|
|
||||||
for bl in buffer:
|
|
||||||
if bl.startswith("data:"):
|
|
||||||
data_lines.append(bl[5:].strip())
|
|
||||||
buffer.clear()
|
|
||||||
if data_lines:
|
|
||||||
raw = "".join(data_lines)
|
|
||||||
if raw == "[DONE]":
|
|
||||||
break
|
|
||||||
try:
|
|
||||||
event = _json.loads(raw)
|
|
||||||
except Exception:
|
|
||||||
continue
|
|
||||||
ev_type = event.get("type", "")
|
|
||||||
if ev_type in ("error", "response.failed"):
|
|
||||||
logger.error("Codex SSE failure: {}", raw[:2000])
|
|
||||||
_collect_images_from_sse_event(event, images)
|
|
||||||
_collect_text_from_sse_event(event, text_parts)
|
|
||||||
continue
|
|
||||||
buffer.append(line)
|
|
||||||
|
|
||||||
# flush remaining
|
|
||||||
if buffer:
|
|
||||||
data_lines = [bl[5:].strip() for bl in buffer if bl.startswith("data:")]
|
|
||||||
raw = "".join(data_lines)
|
|
||||||
if raw and raw != "[DONE]":
|
|
||||||
try:
|
|
||||||
event = _json.loads(raw)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
else:
|
|
||||||
_collect_images_from_sse_event(event, images)
|
|
||||||
_collect_text_from_sse_event(event, text_parts)
|
|
||||||
|
|
||||||
return images, "".join(text_parts).strip()
|
|
||||||
|
|
||||||
|
|
||||||
def _collect_images_from_sse_event(event: dict[str, Any], images: list[str]) -> None:
|
|
||||||
if event.get("type") != "response.output_item.done":
|
|
||||||
return
|
|
||||||
item = event.get("item") or {}
|
|
||||||
if item.get("type") != "image_generation_call":
|
|
||||||
return
|
|
||||||
result = item.get("result")
|
|
||||||
if isinstance(result, str):
|
|
||||||
if result.startswith("data:image/"):
|
|
||||||
images.append(result)
|
|
||||||
else:
|
|
||||||
images.append(_b64_image_data_url(result))
|
|
||||||
elif isinstance(result, dict):
|
|
||||||
image_url = result.get("image_url") or result.get("image") or ""
|
|
||||||
if isinstance(image_url, str):
|
|
||||||
if image_url.startswith("data:image/"):
|
|
||||||
images.append(image_url)
|
|
||||||
else:
|
|
||||||
images.append(_b64_image_data_url(image_url))
|
|
||||||
|
|
||||||
|
|
||||||
def _collect_text_from_sse_event(event: dict[str, Any], text_parts: list[str]) -> None:
|
|
||||||
if event.get("type") == "response.output_text.delta":
|
|
||||||
delta = event.get("delta")
|
|
||||||
if isinstance(delta, str) and delta:
|
|
||||||
text_parts.append(delta)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# StepFun (阶跃星辰) image generation
|
# StepFun (阶跃星辰) image generation
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -1445,159 +879,12 @@ def _stepfun_images_from_payload(payload: dict[str, Any]) -> list[str]:
|
|||||||
return images
|
return images
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Zhipu (智谱) image generation
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
_ZHIPU_TIMEOUT_S = 300.0
|
|
||||||
|
|
||||||
_ZHIPU_ASPECT_RATIO_SIZES = {
|
|
||||||
"1:1": "1280x1280",
|
|
||||||
"16:9": "1728x960",
|
|
||||||
"9:16": "960x1728",
|
|
||||||
"3:4": "1088x1472",
|
|
||||||
"4:3": "1472x1088",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class ZhipuImageGenerationClient(ImageGenerationProvider):
|
|
||||||
"""Async client for Zhipu (智谱) image generation API.
|
|
||||||
|
|
||||||
Supports:
|
|
||||||
- Text-to-image via glm-image, cogview-4, cogview-3-flash, etc.
|
|
||||||
- Aspect ratio selection
|
|
||||||
- Watermark control
|
|
||||||
"""
|
|
||||||
|
|
||||||
provider_name = "zhipu"
|
|
||||||
missing_key_message = "Zhipu API key is not configured. Set providers.zhipu.apiKey."
|
|
||||||
default_timeout = _ZHIPU_TIMEOUT_S
|
|
||||||
|
|
||||||
def _default_base_url(self) -> str:
|
|
||||||
return "https://open.bigmodel.cn/api/paas/v4"
|
|
||||||
|
|
||||||
async def generate(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
prompt: str,
|
|
||||||
model: str,
|
|
||||||
reference_images: list[str] | None = None,
|
|
||||||
aspect_ratio: str | None = None,
|
|
||||||
image_size: str | None = None,
|
|
||||||
) -> GeneratedImageResponse:
|
|
||||||
if not self.api_key:
|
|
||||||
raise ImageGenerationError(self.missing_key_message)
|
|
||||||
|
|
||||||
if reference_images:
|
|
||||||
raise ImageGenerationError(
|
|
||||||
"Zhipu image generation does not support reference images"
|
|
||||||
)
|
|
||||||
|
|
||||||
headers = {
|
|
||||||
"Authorization": f"Bearer {self.api_key}",
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
**self.extra_headers,
|
|
||||||
}
|
|
||||||
|
|
||||||
body: dict[str, Any] = {
|
|
||||||
"model": model,
|
|
||||||
"prompt": prompt,
|
|
||||||
}
|
|
||||||
|
|
||||||
size = _zhipu_size(aspect_ratio, image_size)
|
|
||||||
if size:
|
|
||||||
body["size"] = size
|
|
||||||
|
|
||||||
body.update(self.extra_body)
|
|
||||||
|
|
||||||
url = f"{self.api_base}/images/generations"
|
|
||||||
|
|
||||||
client = self._client or httpx.AsyncClient(timeout=self.timeout)
|
|
||||||
try:
|
|
||||||
return await self._generate_with_client(
|
|
||||||
client,
|
|
||||||
headers=headers,
|
|
||||||
body=body,
|
|
||||||
url=url,
|
|
||||||
)
|
|
||||||
finally:
|
|
||||||
if self._client is None:
|
|
||||||
await client.aclose()
|
|
||||||
|
|
||||||
async def _generate_with_client(
|
|
||||||
self,
|
|
||||||
client: httpx.AsyncClient,
|
|
||||||
*,
|
|
||||||
headers: dict[str, str],
|
|
||||||
body: dict[str, Any],
|
|
||||||
url: str,
|
|
||||||
) -> GeneratedImageResponse:
|
|
||||||
try:
|
|
||||||
response = await self._http_post(url, headers=headers, body=body, client=client)
|
|
||||||
except httpx.TimeoutException as exc:
|
|
||||||
raise ImageGenerationError("Zhipu image generation timed out") from exc
|
|
||||||
except httpx.RequestError as exc:
|
|
||||||
raise ImageGenerationError(f"Zhipu image generation request failed: {exc}") from exc
|
|
||||||
|
|
||||||
try:
|
|
||||||
response.raise_for_status()
|
|
||||||
except httpx.HTTPStatusError as exc:
|
|
||||||
detail = response.text[:500]
|
|
||||||
raise ImageGenerationError(f"Zhipu image generation failed: {detail}") from exc
|
|
||||||
|
|
||||||
payload = response.json()
|
|
||||||
images = await _zhipu_images_from_payload(client, payload)
|
|
||||||
|
|
||||||
self._require_images(images, payload)
|
|
||||||
|
|
||||||
return GeneratedImageResponse(images=images, content="", raw=payload)
|
|
||||||
|
|
||||||
|
|
||||||
def _zhipu_size(
|
|
||||||
aspect_ratio: str | None,
|
|
||||||
image_size: str | None,
|
|
||||||
) -> str:
|
|
||||||
"""Resolve aspect ratio / image_size to Zhipu size string.
|
|
||||||
|
|
||||||
Zhipu glm-image model supports: 1280x1280 (default), 1568x1056,
|
|
||||||
1056x1568, 1472x1088, 1088x1472, 1728x960, 960x1728.
|
|
||||||
"""
|
|
||||||
if image_size and "x" in image_size.lower():
|
|
||||||
return image_size
|
|
||||||
if aspect_ratio and aspect_ratio in _ZHIPU_ASPECT_RATIO_SIZES:
|
|
||||||
return _ZHIPU_ASPECT_RATIO_SIZES[aspect_ratio]
|
|
||||||
return "1280x1280"
|
|
||||||
|
|
||||||
|
|
||||||
async def _zhipu_images_from_payload(
|
|
||||||
client: httpx.AsyncClient,
|
|
||||||
payload: dict[str, Any],
|
|
||||||
) -> list[str]:
|
|
||||||
"""Extract image data URLs from Zhipu API response.
|
|
||||||
|
|
||||||
Zhipu returns images as temporary URLs that expire after 30 days.
|
|
||||||
We download and re-encode as base64 data URLs.
|
|
||||||
"""
|
|
||||||
images: list[str] = []
|
|
||||||
for item in payload.get("data") or []:
|
|
||||||
if not isinstance(item, dict):
|
|
||||||
continue
|
|
||||||
url = item.get("url")
|
|
||||||
if isinstance(url, str) and url:
|
|
||||||
images.append(await _download_image_data_url(client, url))
|
|
||||||
return images
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Provider registration
|
# Provider registration
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
register_image_gen_provider(AIHubMixImageGenerationClient)
|
|
||||||
register_image_gen_provider(CodexImageGenerationClient)
|
|
||||||
register_image_gen_provider(GeminiImageGenerationClient)
|
|
||||||
register_image_gen_provider(OllamaImageGenerationClient)
|
|
||||||
register_image_gen_provider(MiniMaxImageGenerationClient)
|
|
||||||
register_image_gen_provider(OpenAIImageGenerationClient)
|
|
||||||
register_image_gen_provider(OpenRouterImageGenerationClient)
|
register_image_gen_provider(OpenRouterImageGenerationClient)
|
||||||
|
register_image_gen_provider(AIHubMixImageGenerationClient)
|
||||||
|
register_image_gen_provider(GeminiImageGenerationClient)
|
||||||
|
register_image_gen_provider(MiniMaxImageGenerationClient)
|
||||||
register_image_gen_provider(StepFunImageGenerationClient)
|
register_image_gen_provider(StepFunImageGenerationClient)
|
||||||
register_image_gen_provider(ZhipuImageGenerationClient)
|
|
||||||
|
|||||||
@@ -5,7 +5,6 @@ from __future__ import annotations
|
|||||||
import asyncio
|
import asyncio
|
||||||
import hashlib
|
import hashlib
|
||||||
import json
|
import json
|
||||||
import os
|
|
||||||
from collections.abc import Awaitable, Callable
|
from collections.abc import Awaitable, Callable
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
@@ -15,7 +14,7 @@ from oauth_cli_kit import get_token as get_codex_token
|
|||||||
|
|
||||||
from nanobot.providers.base import LLMProvider, LLMResponse, ToolCallRequest
|
from nanobot.providers.base import LLMProvider, LLMResponse, ToolCallRequest
|
||||||
from nanobot.providers.openai_responses import (
|
from nanobot.providers.openai_responses import (
|
||||||
consume_sse_with_reasoning,
|
consume_sse,
|
||||||
convert_messages,
|
convert_messages,
|
||||||
convert_tools,
|
convert_tools,
|
||||||
)
|
)
|
||||||
@@ -41,7 +40,6 @@ class OpenAICodexProvider(LLMProvider):
|
|||||||
reasoning_effort: str | None,
|
reasoning_effort: str | None,
|
||||||
tool_choice: str | dict[str, Any] | None,
|
tool_choice: str | dict[str, Any] | None,
|
||||||
on_content_delta: Callable[[str], Awaitable[None]] | None = None,
|
on_content_delta: Callable[[str], Awaitable[None]] | None = None,
|
||||||
on_thinking_delta: Callable[[str], Awaitable[None]] | None = None,
|
|
||||||
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
|
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
|
||||||
) -> LLMResponse:
|
) -> LLMResponse:
|
||||||
"""Shared request logic for both chat() and chat_stream()."""
|
"""Shared request logic for both chat() and chat_stream()."""
|
||||||
@@ -63,52 +61,32 @@ class OpenAICodexProvider(LLMProvider):
|
|||||||
"tool_choice": tool_choice or "auto",
|
"tool_choice": tool_choice or "auto",
|
||||||
"parallel_tool_calls": True,
|
"parallel_tool_calls": True,
|
||||||
}
|
}
|
||||||
reasoning_options = _build_reasoning_options(reasoning_effort)
|
if reasoning_effort and reasoning_effort.lower() != "none":
|
||||||
if reasoning_options:
|
body["reasoning"] = {"effort": reasoning_effort}
|
||||||
body["reasoning"] = reasoning_options
|
|
||||||
if tools:
|
if tools:
|
||||||
body["tools"] = convert_tools(tools)
|
body["tools"] = convert_tools(tools)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
try:
|
try:
|
||||||
content, tool_calls, finish_reason, reasoning_content = await _request_codex(
|
content, tool_calls, finish_reason = await _request_codex(
|
||||||
DEFAULT_CODEX_URL, headers, body, verify=True,
|
DEFAULT_CODEX_URL, headers, body, verify=True,
|
||||||
on_content_delta=on_content_delta,
|
on_content_delta=on_content_delta,
|
||||||
on_thinking_delta=on_thinking_delta,
|
|
||||||
on_tool_call_delta=on_tool_call_delta,
|
on_tool_call_delta=on_tool_call_delta,
|
||||||
)
|
)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
if "CERTIFICATE_VERIFY_FAILED" not in str(e):
|
if "CERTIFICATE_VERIFY_FAILED" not in str(e):
|
||||||
raise
|
raise
|
||||||
logger.warning("SSL verification failed for Codex API; retrying with verify=False")
|
logger.warning("SSL verification failed for Codex API; retrying with verify=False")
|
||||||
content, tool_calls, finish_reason, reasoning_content = await _request_codex(
|
content, tool_calls, finish_reason = await _request_codex(
|
||||||
DEFAULT_CODEX_URL, headers, body, verify=False,
|
DEFAULT_CODEX_URL, headers, body, verify=False,
|
||||||
on_content_delta=on_content_delta,
|
on_content_delta=on_content_delta,
|
||||||
on_thinking_delta=on_thinking_delta,
|
|
||||||
on_tool_call_delta=on_tool_call_delta,
|
on_tool_call_delta=on_tool_call_delta,
|
||||||
)
|
)
|
||||||
return LLMResponse(
|
return LLMResponse(content=content, tool_calls=tool_calls, finish_reason=finish_reason)
|
||||||
content=content,
|
|
||||||
tool_calls=tool_calls,
|
|
||||||
finish_reason=finish_reason,
|
|
||||||
reasoning_content=reasoning_content,
|
|
||||||
)
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
response = _codex_error_response(e)
|
msg = f"Error calling Codex: {e}"
|
||||||
exc_type = "CodexHTTPError" if isinstance(e, _CodexHTTPError) else type(e).__name__
|
retry_after = getattr(e, "retry_after", None) or self._extract_retry_after(msg)
|
||||||
logger.warning(
|
return LLMResponse(content=msg, finish_reason="error", retry_after=retry_after)
|
||||||
"Codex API request failed: type={} kind={} retryable={} status={} "
|
|
||||||
"error_type={} error_code={} retry_after={} summary={}",
|
|
||||||
exc_type,
|
|
||||||
response.error_kind,
|
|
||||||
response.error_should_retry,
|
|
||||||
response.error_status_code,
|
|
||||||
response.error_type,
|
|
||||||
response.error_code,
|
|
||||||
response.retry_after,
|
|
||||||
_codex_log_summary(exc_type, response),
|
|
||||||
)
|
|
||||||
return response
|
|
||||||
|
|
||||||
async def chat(
|
async def chat(
|
||||||
self, messages: list[dict[str, Any]], tools: list[dict[str, Any]] | None = None,
|
self, messages: list[dict[str, Any]], tools: list[dict[str, Any]] | None = None,
|
||||||
@@ -127,6 +105,7 @@ class OpenAICodexProvider(LLMProvider):
|
|||||||
on_thinking_delta: Callable[[str], Awaitable[None]] | None = None,
|
on_thinking_delta: Callable[[str], Awaitable[None]] | None = None,
|
||||||
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
|
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
|
||||||
) -> LLMResponse:
|
) -> LLMResponse:
|
||||||
|
_ = on_thinking_delta
|
||||||
return await self._call_codex(
|
return await self._call_codex(
|
||||||
messages,
|
messages,
|
||||||
tools,
|
tools,
|
||||||
@@ -134,7 +113,6 @@ class OpenAICodexProvider(LLMProvider):
|
|||||||
reasoning_effort,
|
reasoning_effort,
|
||||||
tool_choice,
|
tool_choice,
|
||||||
on_content_delta,
|
on_content_delta,
|
||||||
on_thinking_delta,
|
|
||||||
on_tool_call_delta,
|
on_tool_call_delta,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -148,16 +126,6 @@ def _strip_model_prefix(model: str) -> str:
|
|||||||
return model
|
return model
|
||||||
|
|
||||||
|
|
||||||
def _build_reasoning_options(reasoning_effort: str | None) -> dict[str, str] | None:
|
|
||||||
"""Opt in to visible summaries without changing provider-default effort."""
|
|
||||||
if reasoning_effort and reasoning_effort.lower() == "none":
|
|
||||||
return {"effort": "none"}
|
|
||||||
options = {"summary": "auto"}
|
|
||||||
if reasoning_effort:
|
|
||||||
options["effort"] = reasoning_effort
|
|
||||||
return options
|
|
||||||
|
|
||||||
|
|
||||||
def _build_headers(account_id: str, token: str) -> dict[str, str]:
|
def _build_headers(account_id: str, token: str) -> dict[str, str]:
|
||||||
return {
|
return {
|
||||||
"Authorization": f"Bearer {token}",
|
"Authorization": f"Bearer {token}",
|
||||||
@@ -171,22 +139,9 @@ def _build_headers(account_id: str, token: str) -> dict[str, str]:
|
|||||||
|
|
||||||
|
|
||||||
class _CodexHTTPError(RuntimeError):
|
class _CodexHTTPError(RuntimeError):
|
||||||
def __init__(
|
def __init__(self, message: str, retry_after: float | None = None):
|
||||||
self,
|
|
||||||
message: str,
|
|
||||||
*,
|
|
||||||
status_code: int | None = None,
|
|
||||||
retry_after: float | None = None,
|
|
||||||
error_type: str | None = None,
|
|
||||||
error_code: str | None = None,
|
|
||||||
should_retry: bool | None = None,
|
|
||||||
):
|
|
||||||
super().__init__(message)
|
super().__init__(message)
|
||||||
self.status_code = status_code
|
|
||||||
self.retry_after = retry_after
|
self.retry_after = retry_after
|
||||||
self.error_type = error_type
|
|
||||||
self.error_code = error_code
|
|
||||||
self.should_retry = should_retry
|
|
||||||
|
|
||||||
|
|
||||||
async def _request_codex(
|
async def _request_codex(
|
||||||
@@ -195,31 +150,18 @@ async def _request_codex(
|
|||||||
body: dict[str, Any],
|
body: dict[str, Any],
|
||||||
verify: bool,
|
verify: bool,
|
||||||
on_content_delta: Callable[[str], Awaitable[None]] | None = None,
|
on_content_delta: Callable[[str], Awaitable[None]] | None = None,
|
||||||
on_thinking_delta: Callable[[str], Awaitable[None]] | None = None,
|
|
||||||
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
|
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
|
||||||
) -> tuple[str, list[ToolCallRequest], str, str | None]:
|
) -> tuple[str, list[ToolCallRequest], str]:
|
||||||
idle_timeout_s = int(os.environ.get("NANOBOT_STREAM_IDLE_TIMEOUT_S", "90"))
|
async with httpx.AsyncClient(timeout=60.0, verify=verify) as client:
|
||||||
async with httpx.AsyncClient(timeout=idle_timeout_s, verify=verify) as client:
|
|
||||||
async with client.stream("POST", url, headers=headers, json=body) as response:
|
async with client.stream("POST", url, headers=headers, json=body) as response:
|
||||||
if response.status_code != 200:
|
if response.status_code != 200:
|
||||||
text = await response.aread()
|
text = await response.aread()
|
||||||
raw = text.decode("utf-8", "ignore")
|
|
||||||
retry_after = LLMProvider._extract_retry_after_from_headers(response.headers)
|
retry_after = LLMProvider._extract_retry_after_from_headers(response.headers)
|
||||||
error_type, error_code = LLMProvider._extract_error_type_code(raw)
|
|
||||||
raise _CodexHTTPError(
|
raise _CodexHTTPError(
|
||||||
_friendly_error(response.status_code, raw),
|
_friendly_error(response.status_code, text.decode("utf-8", "ignore")),
|
||||||
status_code=response.status_code,
|
|
||||||
retry_after=retry_after,
|
retry_after=retry_after,
|
||||||
error_type=error_type,
|
|
||||||
error_code=error_code,
|
|
||||||
should_retry=_should_retry_status(response.status_code, error_type, error_code, raw),
|
|
||||||
)
|
|
||||||
return await consume_sse_with_reasoning(
|
|
||||||
response,
|
|
||||||
on_content_delta=on_content_delta,
|
|
||||||
on_tool_call_delta=on_tool_call_delta,
|
|
||||||
on_reasoning_delta=on_thinking_delta,
|
|
||||||
)
|
)
|
||||||
|
return await consume_sse(response, on_content_delta, on_tool_call_delta)
|
||||||
|
|
||||||
|
|
||||||
def _prompt_cache_key(messages: list[dict[str, Any]]) -> str:
|
def _prompt_cache_key(messages: list[dict[str, Any]]) -> str:
|
||||||
@@ -228,94 +170,6 @@ def _prompt_cache_key(messages: list[dict[str, Any]]) -> str:
|
|||||||
|
|
||||||
|
|
||||||
def _friendly_error(status_code: int, raw: str) -> str:
|
def _friendly_error(status_code: int, raw: str) -> str:
|
||||||
_ = raw
|
|
||||||
if status_code == 429:
|
if status_code == 429:
|
||||||
return "ChatGPT usage quota exceeded or rate limit triggered. Please try again later."
|
return "ChatGPT usage quota exceeded or rate limit triggered. Please try again later."
|
||||||
return f"HTTP {status_code}: Codex API request failed"
|
return f"HTTP {status_code}: {raw}"
|
||||||
|
|
||||||
|
|
||||||
def _codex_error_response(exc: Exception) -> LLMResponse:
|
|
||||||
"""Convert Codex transport/API failures into actionable, retryable metadata."""
|
|
||||||
exc_type = "CodexHTTPError" if isinstance(exc, _CodexHTTPError) else type(exc).__name__
|
|
||||||
detail = str(exc).strip()
|
|
||||||
|
|
||||||
status_code = getattr(exc, "status_code", None)
|
|
||||||
error_kind: str | None = None
|
|
||||||
default_detail: str | None = None
|
|
||||||
should_retry: bool | None = getattr(exc, "should_retry", None)
|
|
||||||
|
|
||||||
if isinstance(exc, (httpx.TimeoutException, asyncio.TimeoutError)):
|
|
||||||
error_kind = "timeout"
|
|
||||||
default_detail = "timed out waiting for response"
|
|
||||||
should_retry = True if should_retry is None else should_retry
|
|
||||||
elif isinstance(exc, httpx.RemoteProtocolError):
|
|
||||||
error_kind = "connection"
|
|
||||||
default_detail = "network protocol error while reading response"
|
|
||||||
should_retry = True if should_retry is None else should_retry
|
|
||||||
elif isinstance(exc, (httpx.NetworkError, httpx.TransportError)):
|
|
||||||
error_kind = "connection"
|
|
||||||
default_detail = "network connection failed"
|
|
||||||
should_retry = True if should_retry is None else should_retry
|
|
||||||
elif isinstance(exc, _CodexHTTPError):
|
|
||||||
error_kind = "http"
|
|
||||||
default_detail = "HTTP request failed"
|
|
||||||
|
|
||||||
if status_code is not None and should_retry is None:
|
|
||||||
retry_content = None if int(status_code) == 429 and isinstance(exc, _CodexHTTPError) else detail
|
|
||||||
should_retry = _should_retry_status(
|
|
||||||
int(status_code),
|
|
||||||
getattr(exc, "error_type", None),
|
|
||||||
getattr(exc, "error_code", None),
|
|
||||||
retry_content,
|
|
||||||
)
|
|
||||||
|
|
||||||
detail = detail or default_detail or "unexpected error"
|
|
||||||
message = f"Error calling Codex ({exc_type}): {detail}"
|
|
||||||
retry_after = getattr(exc, "retry_after", None) or LLMProvider._extract_retry_after(message)
|
|
||||||
return LLMResponse(
|
|
||||||
content=message,
|
|
||||||
finish_reason="error",
|
|
||||||
retry_after=retry_after,
|
|
||||||
error_status_code=int(status_code) if status_code is not None else None,
|
|
||||||
error_kind=error_kind,
|
|
||||||
error_type=getattr(exc, "error_type", None),
|
|
||||||
error_code=getattr(exc, "error_code", None),
|
|
||||||
error_retry_after_s=retry_after,
|
|
||||||
error_should_retry=should_retry,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _codex_log_summary(exc_type: str, response: LLMResponse) -> str:
|
|
||||||
"""Return a bounded diagnostic summary without request body or raw upstream payload."""
|
|
||||||
if response.error_status_code is not None:
|
|
||||||
parts = [f"HTTP {response.error_status_code}"]
|
|
||||||
if response.error_type:
|
|
||||||
parts.append(f"type={response.error_type}")
|
|
||||||
if response.error_code:
|
|
||||||
parts.append(f"code={response.error_code}")
|
|
||||||
return " ".join(parts)
|
|
||||||
|
|
||||||
kind = (response.error_kind or "").strip()
|
|
||||||
if kind:
|
|
||||||
return f"{exc_type} {kind}"
|
|
||||||
|
|
||||||
return exc_type
|
|
||||||
|
|
||||||
|
|
||||||
def _should_retry_status(
|
|
||||||
status_code: int,
|
|
||||||
error_type: str | None,
|
|
||||||
error_code: str | None,
|
|
||||||
content: str | None,
|
|
||||||
) -> bool:
|
|
||||||
if status_code == 429:
|
|
||||||
return LLMProvider._is_retryable_429_response(
|
|
||||||
LLMResponse(
|
|
||||||
content=content or "",
|
|
||||||
finish_reason="error",
|
|
||||||
error_status_code=status_code,
|
|
||||||
error_type=error_type,
|
|
||||||
error_code=error_code,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
return status_code in LLMProvider._RETRYABLE_STATUS_CODES or status_code >= 500
|
|
||||||
|
|||||||
@@ -11,7 +11,6 @@ import secrets
|
|||||||
import string
|
import string
|
||||||
import time
|
import time
|
||||||
import uuid
|
import uuid
|
||||||
from collections import deque
|
|
||||||
from collections.abc import Awaitable, Callable
|
from collections.abc import Awaitable, Callable
|
||||||
from ipaddress import ip_address
|
from ipaddress import ip_address
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import TYPE_CHECKING, Any
|
||||||
@@ -75,43 +74,41 @@ _THINKING_STYLE_MAP: dict[str, Any] = {
|
|||||||
"enable_thinking": lambda on: {"enable_thinking": on},
|
"enable_thinking": lambda on: {"enable_thinking": on},
|
||||||
"reasoning_split": lambda on: {"reasoning_split": on},
|
"reasoning_split": lambda on: {"reasoning_split": on},
|
||||||
}
|
}
|
||||||
_GATEWAY_REASONING_STYLE_MAP: dict[str, Any] = {
|
|
||||||
"reasoning_effort": lambda effort: {"reasoning": {"effort": effort}},
|
|
||||||
}
|
|
||||||
_MODEL_THINKING_STYLES: dict[str, str] = {
|
|
||||||
**dict.fromkeys(_KIMI_THINKING_MODELS, "thinking_type"),
|
|
||||||
**dict.fromkeys(_MIMO_THINKING_MODELS, "thinking_type"),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _model_slug(model_name: str) -> str:
|
def _is_kimi_thinking_model(model_name: str) -> bool:
|
||||||
return model_name.lower().rsplit("/", 1)[-1]
|
"""Return True if model_name refers to a Kimi thinking-capable model.
|
||||||
|
|
||||||
|
Supports two forms:
|
||||||
|
- Exact match: e.g. kimi-k2.5 / kimi-k2.6 in _KIMI_THINKING_MODELS
|
||||||
|
- Slug match: moonshotai/kimi-k2.5 -> the part after the last "/"
|
||||||
|
is checked against _KIMI_THINKING_MODELS
|
||||||
|
|
||||||
|
This covers both the native Moonshot provider (bare slug) and
|
||||||
|
OpenRouter-style names (``"publisher/slug"``).
|
||||||
|
"""
|
||||||
|
name = model_name.lower()
|
||||||
|
if name in _KIMI_THINKING_MODELS:
|
||||||
|
return True
|
||||||
|
if "/" in name and name.rsplit("/", 1)[1] in _KIMI_THINKING_MODELS:
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
def _model_thinking_style(model_name: str) -> str:
|
def _is_mimo_thinking_model(model_name: str) -> bool:
|
||||||
return _MODEL_THINKING_STYLES.get(_model_slug(model_name), "")
|
"""Return True if model_name refers to a MiMo thinking-capable model.
|
||||||
|
|
||||||
|
Mirrors _is_kimi_thinking_model: gateway providers (e.g. OpenRouter
|
||||||
def _thinking_styles_for(spec: ProviderSpec | None, model_name: str) -> list[str]:
|
routing ``xiaomi/mimo-v2.5-pro``) have no ``thinking_style`` on their
|
||||||
styles: list[str] = []
|
spec, so the spec-driven branch in _build_kwargs misses them. The
|
||||||
if spec and spec.thinking_style:
|
model-name path catches those cases.
|
||||||
styles.append(spec.thinking_style)
|
"""
|
||||||
model_style = _model_thinking_style(model_name)
|
name = model_name.lower()
|
||||||
if model_style and model_style not in styles:
|
if name in _MIMO_THINKING_MODELS:
|
||||||
styles.append(model_style)
|
return True
|
||||||
return styles
|
if "/" in name and name.rsplit("/", 1)[1] in _MIMO_THINKING_MODELS:
|
||||||
|
return True
|
||||||
|
return False
|
||||||
def _thinking_extra_body(style: str, thinking_enabled: bool) -> dict[str, Any] | None:
|
|
||||||
builder = _THINKING_STYLE_MAP.get(style)
|
|
||||||
return builder(thinking_enabled) if builder else None
|
|
||||||
|
|
||||||
|
|
||||||
def _gateway_reasoning_extra_body(style: str, effort: str | None) -> dict[str, Any] | None:
|
|
||||||
if not effort:
|
|
||||||
return None
|
|
||||||
builder = _GATEWAY_REASONING_STYLE_MAP.get(style)
|
|
||||||
return builder(effort) if builder else None
|
|
||||||
|
|
||||||
|
|
||||||
def _openai_compat_timeout_s() -> float:
|
def _openai_compat_timeout_s() -> float:
|
||||||
@@ -274,47 +271,6 @@ def _deep_merge(base: dict[str, Any], override: dict[str, Any]) -> dict[str, Any
|
|||||||
return merged
|
return merged
|
||||||
|
|
||||||
|
|
||||||
def _merge_unique_list(base: Any, override: Any) -> Any:
|
|
||||||
"""Append list values while preserving order and removing duplicates."""
|
|
||||||
if not isinstance(base, list) or not isinstance(override, list):
|
|
||||||
return override
|
|
||||||
result: list[Any] = []
|
|
||||||
seen: set[str] = set()
|
|
||||||
for value in [*base, *override]:
|
|
||||||
try:
|
|
||||||
key = json.dumps(value, sort_keys=True, ensure_ascii=False)
|
|
||||||
except Exception:
|
|
||||||
key = repr(value)
|
|
||||||
if key in seen:
|
|
||||||
continue
|
|
||||||
seen.add(key)
|
|
||||||
result.append(value)
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def _merge_responses_extra_body(
|
|
||||||
body: dict[str, Any],
|
|
||||||
extra_body: dict[str, Any],
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Merge configured Responses API body fields without clobbering tools."""
|
|
||||||
reserved = {"include", "tools"}
|
|
||||||
regular_extra = {key: value for key, value in extra_body.items() if key not in reserved}
|
|
||||||
merged = _deep_merge(body, regular_extra)
|
|
||||||
|
|
||||||
if "include" in extra_body:
|
|
||||||
merged["include"] = _merge_unique_list(body.get("include"), extra_body["include"])
|
|
||||||
|
|
||||||
if "tools" in extra_body:
|
|
||||||
current_tools = body.get("tools")
|
|
||||||
configured_tools = extra_body["tools"]
|
|
||||||
if isinstance(current_tools, list) and isinstance(configured_tools, list):
|
|
||||||
merged["tools"] = [*current_tools, *configured_tools]
|
|
||||||
else:
|
|
||||||
merged["tools"] = configured_tools
|
|
||||||
|
|
||||||
return merged
|
|
||||||
|
|
||||||
|
|
||||||
class OpenAICompatProvider(LLMProvider):
|
class OpenAICompatProvider(LLMProvider):
|
||||||
"""Unified provider for all OpenAI-compatible APIs.
|
"""Unified provider for all OpenAI-compatible APIs.
|
||||||
|
|
||||||
@@ -330,14 +286,12 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
extra_headers: dict[str, str] | None = None,
|
extra_headers: dict[str, str] | None = None,
|
||||||
spec: ProviderSpec | None = None,
|
spec: ProviderSpec | None = None,
|
||||||
extra_body: dict[str, Any] | None = None,
|
extra_body: dict[str, Any] | None = None,
|
||||||
api_type: str = "auto",
|
|
||||||
):
|
):
|
||||||
super().__init__(api_key, api_base)
|
super().__init__(api_key, api_base)
|
||||||
self.default_model = default_model
|
self.default_model = default_model
|
||||||
self.extra_headers = extra_headers or {}
|
self.extra_headers = extra_headers or {}
|
||||||
self._spec = spec
|
self._spec = spec
|
||||||
self._extra_body = extra_body or {}
|
self._extra_body = extra_body or {}
|
||||||
self._api_type = api_type if spec and spec.name == "openai" else "auto"
|
|
||||||
|
|
||||||
if api_key and spec and spec.env_key:
|
if api_key and spec and spec.env_key:
|
||||||
self._setup_env(api_key, api_base)
|
self._setup_env(api_key, api_base)
|
||||||
@@ -471,10 +425,6 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
return tool_call_id
|
return tool_call_id
|
||||||
return hashlib.sha1(tool_call_id.encode()).hexdigest()[:9]
|
return hashlib.sha1(tool_call_id.encode()).hexdigest()[:9]
|
||||||
|
|
||||||
def _should_normalize_tool_call_ids(self) -> bool:
|
|
||||||
"""Return True for providers that reject normal OpenAI tool call IDs."""
|
|
||||||
return bool(self._spec and self._spec.name == "mistral")
|
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _normalize_tool_call_arguments(arguments: Any) -> str:
|
def _normalize_tool_call_arguments(arguments: Any) -> str:
|
||||||
"""Force function.arguments into a valid JSON object string."""
|
"""Force function.arguments into a valid JSON object string."""
|
||||||
@@ -511,60 +461,22 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
"""Strip non-standard keys, normalize tool_call IDs."""
|
"""Strip non-standard keys, normalize tool_call IDs."""
|
||||||
sanitized = LLMProvider._sanitize_request_messages(messages, _ALLOWED_MSG_KEYS)
|
sanitized = LLMProvider._sanitize_request_messages(messages, _ALLOWED_MSG_KEYS)
|
||||||
id_map: dict[str, str] = {}
|
id_map: dict[str, str] = {}
|
||||||
pending_tool_ids: dict[str, deque[str]] = {}
|
|
||||||
force_string_content = bool(self._spec and self._spec.name == "deepseek")
|
force_string_content = bool(self._spec and self._spec.name == "deepseek")
|
||||||
normalize_tool_ids = self._should_normalize_tool_call_ids()
|
|
||||||
|
|
||||||
def map_id(value: Any) -> Any:
|
def map_id(value: Any) -> Any:
|
||||||
if not isinstance(value, str):
|
if not isinstance(value, str):
|
||||||
return value
|
return value
|
||||||
if not normalize_tool_ids:
|
|
||||||
return value
|
|
||||||
return id_map.setdefault(value, self._normalize_tool_call_id(value))
|
return id_map.setdefault(value, self._normalize_tool_call_id(value))
|
||||||
|
|
||||||
def unique_tool_id(value: Any, used_ids: set[str], idx: int) -> str:
|
|
||||||
if isinstance(value, str) and value:
|
|
||||||
base = map_id(value)
|
|
||||||
else:
|
|
||||||
base = _short_tool_id()
|
|
||||||
if not isinstance(base, str) or not base:
|
|
||||||
base = _short_tool_id()
|
|
||||||
if base not in used_ids:
|
|
||||||
return base
|
|
||||||
seed = value if isinstance(value, str) and value else base
|
|
||||||
salt = 1
|
|
||||||
while True:
|
|
||||||
candidate = self._normalize_tool_call_id(f"{seed}:{idx}:{salt}")
|
|
||||||
if isinstance(candidate, str) and candidate not in used_ids:
|
|
||||||
return candidate
|
|
||||||
salt += 1
|
|
||||||
|
|
||||||
def map_tool_result_id(value: Any) -> Any:
|
|
||||||
if not isinstance(value, str):
|
|
||||||
return value
|
|
||||||
queue = pending_tool_ids.get(value)
|
|
||||||
if queue:
|
|
||||||
mapped = queue.popleft()
|
|
||||||
if not queue:
|
|
||||||
pending_tool_ids.pop(value, None)
|
|
||||||
return mapped
|
|
||||||
return map_id(value)
|
|
||||||
|
|
||||||
for clean in sanitized:
|
for clean in sanitized:
|
||||||
if isinstance(clean.get("tool_calls"), list):
|
if isinstance(clean.get("tool_calls"), list):
|
||||||
normalized = []
|
normalized = []
|
||||||
used_ids: set[str] = set()
|
for tc in clean["tool_calls"]:
|
||||||
for idx, tc in enumerate(clean["tool_calls"]):
|
|
||||||
if not isinstance(tc, dict):
|
if not isinstance(tc, dict):
|
||||||
normalized.append(tc)
|
normalized.append(tc)
|
||||||
continue
|
continue
|
||||||
tc_clean = dict(tc)
|
tc_clean = dict(tc)
|
||||||
raw_id = tc_clean.get("id")
|
tc_clean["id"] = map_id(tc_clean.get("id"))
|
||||||
mapped_id = unique_tool_id(raw_id, used_ids, idx)
|
|
||||||
tc_clean["id"] = mapped_id
|
|
||||||
used_ids.add(mapped_id)
|
|
||||||
if isinstance(raw_id, str) and raw_id:
|
|
||||||
pending_tool_ids.setdefault(raw_id, deque()).append(mapped_id)
|
|
||||||
function = tc_clean.get("function")
|
function = tc_clean.get("function")
|
||||||
if isinstance(function, dict):
|
if isinstance(function, dict):
|
||||||
function_clean = dict(function)
|
function_clean = dict(function)
|
||||||
@@ -582,7 +494,7 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
# that mix non-empty content with tool_calls.
|
# that mix non-empty content with tool_calls.
|
||||||
clean["content"] = None
|
clean["content"] = None
|
||||||
if "tool_call_id" in clean and clean["tool_call_id"]:
|
if "tool_call_id" in clean and clean["tool_call_id"]:
|
||||||
clean["tool_call_id"] = map_tool_result_id(clean["tool_call_id"])
|
clean["tool_call_id"] = map_id(clean["tool_call_id"])
|
||||||
if (
|
if (
|
||||||
force_string_content
|
force_string_content
|
||||||
and not (clean.get("role") == "assistant" and clean.get("tool_calls"))
|
and not (clean.get("role") == "assistant" and clean.get("tool_calls"))
|
||||||
@@ -669,27 +581,39 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
if wire_effort and semantic_effort != "none":
|
if wire_effort and semantic_effort != "none":
|
||||||
kwargs["reasoning_effort"] = wire_effort
|
kwargs["reasoning_effort"] = wire_effort
|
||||||
|
|
||||||
# Only send thinking controls when reasoning_effort is explicit so
|
# Provider-specific thinking parameters.
|
||||||
# omitting the config preserves each provider's default.
|
# Only sent when reasoning_effort is explicitly configured so that
|
||||||
if reasoning_effort is not None:
|
# the provider default is preserved otherwise.
|
||||||
|
# The mapping is driven by ProviderSpec.thinking_style so that adding
|
||||||
|
# a new provider never requires touching this function.
|
||||||
|
if spec and spec.thinking_style and reasoning_effort is not None:
|
||||||
thinking_enabled = semantic_effort not in ("none", "minimal")
|
thinking_enabled = semantic_effort not in ("none", "minimal")
|
||||||
for thinking_style in _thinking_styles_for(spec, model_name):
|
extra = _THINKING_STYLE_MAP.get(spec.thinking_style, lambda _: None)(thinking_enabled)
|
||||||
extra = _thinking_extra_body(thinking_style, thinking_enabled)
|
|
||||||
if extra:
|
|
||||||
kwargs.setdefault("extra_body", {}).update(extra)
|
|
||||||
gateway_style = getattr(spec, "gateway_reasoning_style", "") if spec else ""
|
|
||||||
if gateway_style and _model_thinking_style(model_name):
|
|
||||||
extra = _gateway_reasoning_extra_body(gateway_style, semantic_effort)
|
|
||||||
if extra:
|
if extra:
|
||||||
kwargs.setdefault("extra_body", {}).update(extra)
|
kwargs.setdefault("extra_body", {}).update(extra)
|
||||||
|
|
||||||
# Moonshot rejects requests that carry both 'reasoning_effort'
|
# Model-level thinking injection for Kimi thinking-capable models.
|
||||||
# and the native 'thinking' param. We already expressed the
|
# Strip any provider prefix (e.g. "moonshotai/") before the set lookup
|
||||||
# user's intent via the provider-native shape, so drop the
|
# so that OpenRouter-style names like "moonshotai/kimi-k2.5" are handled
|
||||||
# redundant wire-level kwarg. Only kimi models need this —
|
# identically to bare names like "kimi-k2.5".
|
||||||
# Xiaomi's API accepts both params.
|
if reasoning_effort is not None and _is_kimi_thinking_model(model_name):
|
||||||
if _model_slug(model_name) in _KIMI_THINKING_MODELS:
|
thinking_enabled = semantic_effort not in ("none", "minimal")
|
||||||
kwargs.pop("reasoning_effort", None)
|
kwargs.setdefault("extra_body", {}).update(
|
||||||
|
{"thinking": {"type": "enabled" if thinking_enabled else "disabled"}}
|
||||||
|
)
|
||||||
|
|
||||||
|
# Model-level thinking injection for MiMo thinking-capable models.
|
||||||
|
# Same shape as Kimi: gateway providers (OpenRouter, etc.) lack the
|
||||||
|
# xiaomi_mimo spec's thinking_style, so the spec-driven branch above
|
||||||
|
# misses them — match by model name to catch "xiaomi/mimo-v2.5-pro"
|
||||||
|
# and friends. (Direct xiaomi_mimo requests are also covered here;
|
||||||
|
# both branches write the same payload, so the dict update is a
|
||||||
|
# safe no-op for already-handled cases.)
|
||||||
|
if reasoning_effort is not None and _is_mimo_thinking_model(model_name):
|
||||||
|
thinking_enabled = semantic_effort not in ("none", "minimal")
|
||||||
|
kwargs.setdefault("extra_body", {}).update(
|
||||||
|
{"thinking": {"type": "enabled" if thinking_enabled else "disabled"}}
|
||||||
|
)
|
||||||
|
|
||||||
if tools:
|
if tools:
|
||||||
kwargs["tools"] = tools
|
kwargs["tools"] = tools
|
||||||
@@ -704,7 +628,8 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
and semantic_effort not in ("none", "minimal")
|
and semantic_effort not in ("none", "minimal")
|
||||||
and (
|
and (
|
||||||
(spec and spec.thinking_style)
|
(spec and spec.thinking_style)
|
||||||
or _model_thinking_style(model_name)
|
or _is_kimi_thinking_model(model_name)
|
||||||
|
or _is_mimo_thinking_model(model_name)
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
implicit_deepseek_thinking = (
|
implicit_deepseek_thinking = (
|
||||||
@@ -735,14 +660,8 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
reasoning_effort: str | None,
|
reasoning_effort: str | None,
|
||||||
) -> bool:
|
) -> bool:
|
||||||
"""Use Responses API only for direct OpenAI requests that benefit from it."""
|
"""Use Responses API only for direct OpenAI requests that benefit from it."""
|
||||||
if self._api_type == "chat_completions":
|
|
||||||
return False
|
|
||||||
if self._spec and self._spec.name not in ("openai", "github_copilot"):
|
if self._spec and self._spec.name not in ("openai", "github_copilot"):
|
||||||
return False
|
return False
|
||||||
if self._api_type == "responses":
|
|
||||||
# Explicit configuration means Responses is mandatory; do not
|
|
||||||
# consult the circuit breaker or fall back to Chat Completions.
|
|
||||||
return True
|
|
||||||
if self._spec is None or self._spec.name != "github_copilot":
|
if self._spec is None or self._spec.name != "github_copilot":
|
||||||
if not _is_direct_openai_base(self._effective_base):
|
if not _is_direct_openai_base(self._effective_base):
|
||||||
return False
|
return False
|
||||||
@@ -756,14 +675,7 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
if not wants:
|
if not wants:
|
||||||
return False
|
return False
|
||||||
|
|
||||||
return self._responses_circuit_allows_probe(model, reasoning_effort)
|
# Circuit breaker: skip after repeated failures, probe periodically.
|
||||||
|
|
||||||
def _responses_circuit_allows_probe(
|
|
||||||
self,
|
|
||||||
model: str | None,
|
|
||||||
reasoning_effort: str | None,
|
|
||||||
) -> bool:
|
|
||||||
"""Return False when the Responses API circuit breaker is open."""
|
|
||||||
key = _responses_circuit_key(model, self.default_model, reasoning_effort)
|
key = _responses_circuit_key(model, self.default_model, reasoning_effort)
|
||||||
failures = self._responses_failures.get(key, 0)
|
failures = self._responses_failures.get(key, 0)
|
||||||
if failures >= _RESPONSES_FAILURE_THRESHOLD:
|
if failures >= _RESPONSES_FAILURE_THRESHOLD:
|
||||||
@@ -855,10 +767,6 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
body["tools"] = convert_tools(tools)
|
body["tools"] = convert_tools(tools)
|
||||||
body["tool_choice"] = tool_choice or "auto"
|
body["tool_choice"] = tool_choice or "auto"
|
||||||
|
|
||||||
extra_body = getattr(self, "_extra_body", {})
|
|
||||||
if extra_body:
|
|
||||||
body = _merge_responses_extra_body(body, extra_body)
|
|
||||||
|
|
||||||
return body
|
return body
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
@@ -1023,7 +931,7 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
args = json_repair.loads(args)
|
args = json_repair.loads(args)
|
||||||
ec, prov, fn_prov = _extract_tc_extras(tc)
|
ec, prov, fn_prov = _extract_tc_extras(tc)
|
||||||
parsed_tool_calls.append(ToolCallRequest(
|
parsed_tool_calls.append(ToolCallRequest(
|
||||||
id=str(tc_map.get("id") or _short_tool_id()),
|
id=_short_tool_id(),
|
||||||
name=str(fn.get("name") or ""),
|
name=str(fn.get("name") or ""),
|
||||||
arguments=args if isinstance(args, dict) else {},
|
arguments=args if isinstance(args, dict) else {},
|
||||||
extra_content=ec,
|
extra_content=ec,
|
||||||
@@ -1066,7 +974,7 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
args = json_repair.loads(args)
|
args = json_repair.loads(args)
|
||||||
ec, prov, fn_prov = _extract_tc_extras(tc)
|
ec, prov, fn_prov = _extract_tc_extras(tc)
|
||||||
tool_calls.append(ToolCallRequest(
|
tool_calls.append(ToolCallRequest(
|
||||||
id=str(getattr(tc, "id", None) or _short_tool_id()),
|
id=_short_tool_id(),
|
||||||
name=tc.function.name,
|
name=tc.function.name,
|
||||||
arguments=args,
|
arguments=args,
|
||||||
extra_content=ec,
|
extra_content=ec,
|
||||||
@@ -1189,15 +1097,6 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
if delta:
|
if delta:
|
||||||
_accum_legacy_function_call(getattr(delta, "function_call", None))
|
_accum_legacy_function_call(getattr(delta, "function_call", None))
|
||||||
|
|
||||||
# Some providers (e.g. Zhipu/GLM) reuse the same tool_call id for
|
|
||||||
# parallel tool calls in streaming mode. Deduplicate before building
|
|
||||||
# the response so downstream tool messages don't collide.
|
|
||||||
_seen_tc_ids: set[str] = set()
|
|
||||||
for b in tc_bufs.values():
|
|
||||||
if not b["id"] or b["id"] in _seen_tc_ids:
|
|
||||||
b["id"] = _short_tool_id()
|
|
||||||
_seen_tc_ids.add(b["id"])
|
|
||||||
|
|
||||||
return LLMResponse(
|
return LLMResponse(
|
||||||
content="".join(content_parts) or None,
|
content="".join(content_parts) or None,
|
||||||
tool_calls=[
|
tool_calls=[
|
||||||
@@ -1329,8 +1228,6 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
# falling back to /chat/completions cannot succeed and would
|
# falling back to /chat/completions cannot succeed and would
|
||||||
# hide the real error.
|
# hide the real error.
|
||||||
raise
|
raise
|
||||||
if self._api_type == "responses":
|
|
||||||
raise
|
|
||||||
if not self._should_fallback_from_responses_error(responses_error):
|
if not self._should_fallback_from_responses_error(responses_error):
|
||||||
raise
|
raise
|
||||||
self._record_responses_failure(model, reasoning_effort)
|
self._record_responses_failure(model, reasoning_effort)
|
||||||
@@ -1404,8 +1301,6 @@ class OpenAICompatProvider(LLMProvider):
|
|||||||
# falling back to /chat/completions cannot succeed and would
|
# falling back to /chat/completions cannot succeed and would
|
||||||
# hide the real error.
|
# hide the real error.
|
||||||
raise
|
raise
|
||||||
if self._api_type == "responses":
|
|
||||||
raise
|
|
||||||
if not self._should_fallback_from_responses_error(responses_error):
|
if not self._should_fallback_from_responses_error(responses_error):
|
||||||
raise
|
raise
|
||||||
self._record_responses_failure(model, reasoning_effort)
|
self._record_responses_failure(model, reasoning_effort)
|
||||||
|
|||||||
@@ -10,7 +10,6 @@ from nanobot.providers.openai_responses.parsing import (
|
|||||||
FINISH_REASON_MAP,
|
FINISH_REASON_MAP,
|
||||||
consume_sdk_stream,
|
consume_sdk_stream,
|
||||||
consume_sse,
|
consume_sse,
|
||||||
consume_sse_with_reasoning,
|
|
||||||
iter_sse,
|
iter_sse,
|
||||||
map_finish_reason,
|
map_finish_reason,
|
||||||
parse_response_output,
|
parse_response_output,
|
||||||
@@ -23,7 +22,6 @@ __all__ = [
|
|||||||
"split_tool_call_id",
|
"split_tool_call_id",
|
||||||
"iter_sse",
|
"iter_sse",
|
||||||
"consume_sse",
|
"consume_sse",
|
||||||
"consume_sse_with_reasoning",
|
|
||||||
"consume_sdk_stream",
|
"consume_sdk_stream",
|
||||||
"map_finish_reason",
|
"map_finish_reason",
|
||||||
"parse_response_output",
|
"parse_response_output",
|
||||||
|
|||||||
@@ -15,7 +15,6 @@ def convert_messages(messages: list[dict[str, Any]]) -> tuple[str, list[dict[str
|
|||||||
"""
|
"""
|
||||||
system_prompt = ""
|
system_prompt = ""
|
||||||
input_items: list[dict[str, Any]] = []
|
input_items: list[dict[str, Any]] = []
|
||||||
used_item_ids: set[str] = set()
|
|
||||||
|
|
||||||
for idx, msg in enumerate(messages):
|
for idx, msg in enumerate(messages):
|
||||||
role = msg.get("role")
|
role = msg.get("role")
|
||||||
@@ -31,19 +30,17 @@ def convert_messages(messages: list[dict[str, Any]]) -> tuple[str, list[dict[str
|
|||||||
|
|
||||||
if role == "assistant":
|
if role == "assistant":
|
||||||
if isinstance(content, str) and content:
|
if isinstance(content, str) and content:
|
||||||
message_id = _unique_item_id(f"msg_{idx}", used_item_ids)
|
|
||||||
input_items.append({
|
input_items.append({
|
||||||
"type": "message", "role": "assistant",
|
"type": "message", "role": "assistant",
|
||||||
"content": [{"type": "output_text", "text": content}],
|
"content": [{"type": "output_text", "text": content}],
|
||||||
"status": "completed", "id": message_id,
|
"status": "completed", "id": f"msg_{idx}",
|
||||||
})
|
})
|
||||||
for tool_call in msg.get("tool_calls", []) or []:
|
for tool_call in msg.get("tool_calls", []) or []:
|
||||||
fn = tool_call.get("function") or {}
|
fn = tool_call.get("function") or {}
|
||||||
call_id, item_id = split_tool_call_id(tool_call.get("id"))
|
call_id, item_id = split_tool_call_id(tool_call.get("id"))
|
||||||
response_item_id = _unique_item_id(item_id or f"fc_{idx}", used_item_ids)
|
|
||||||
input_items.append({
|
input_items.append({
|
||||||
"type": "function_call",
|
"type": "function_call",
|
||||||
"id": response_item_id,
|
"id": item_id or f"fc_{idx}",
|
||||||
"call_id": call_id or f"call_{idx}",
|
"call_id": call_id or f"call_{idx}",
|
||||||
"name": fn.get("name"),
|
"name": fn.get("name"),
|
||||||
"arguments": fn.get("arguments") or "{}",
|
"arguments": fn.get("arguments") or "{}",
|
||||||
@@ -100,20 +97,6 @@ def convert_tools(tools: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
|||||||
return converted
|
return converted
|
||||||
|
|
||||||
|
|
||||||
def _unique_item_id(item_id: str, used: set[str]) -> str:
|
|
||||||
"""Return a Responses input item id that is unique within one request."""
|
|
||||||
if item_id not in used:
|
|
||||||
used.add(item_id)
|
|
||||||
return item_id
|
|
||||||
|
|
||||||
suffix = 2
|
|
||||||
while f"{item_id}_{suffix}" in used:
|
|
||||||
suffix += 1
|
|
||||||
unique = f"{item_id}_{suffix}"
|
|
||||||
used.add(unique)
|
|
||||||
return unique
|
|
||||||
|
|
||||||
|
|
||||||
def split_tool_call_id(tool_call_id: Any) -> tuple[str, str | None]:
|
def split_tool_call_id(tool_call_id: Any) -> tuple[str, str | None]:
|
||||||
"""Split a compound ``call_id|item_id`` string.
|
"""Split a compound ``call_id|item_id`` string.
|
||||||
|
|
||||||
|
|||||||
@@ -65,28 +65,10 @@ async def consume_sse(
|
|||||||
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
|
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
|
||||||
) -> tuple[str, list[ToolCallRequest], str]:
|
) -> tuple[str, list[ToolCallRequest], str]:
|
||||||
"""Consume a Responses API SSE stream into ``(content, tool_calls, finish_reason)``."""
|
"""Consume a Responses API SSE stream into ``(content, tool_calls, finish_reason)``."""
|
||||||
content, tool_calls, finish_reason, _ = await consume_sse_with_reasoning(
|
|
||||||
response,
|
|
||||||
on_content_delta=on_content_delta,
|
|
||||||
on_tool_call_delta=on_tool_call_delta,
|
|
||||||
)
|
|
||||||
return content, tool_calls, finish_reason
|
|
||||||
|
|
||||||
|
|
||||||
async def consume_sse_with_reasoning(
|
|
||||||
response: httpx.Response,
|
|
||||||
on_content_delta: Callable[[str], Awaitable[None]] | None = None,
|
|
||||||
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
|
|
||||||
on_reasoning_delta: Callable[[str], Awaitable[None]] | None = None,
|
|
||||||
) -> tuple[str, list[ToolCallRequest], str, str | None]:
|
|
||||||
"""Consume a Responses API SSE stream, including visible reasoning summaries."""
|
|
||||||
content = ""
|
content = ""
|
||||||
tool_calls: list[ToolCallRequest] = []
|
tool_calls: list[ToolCallRequest] = []
|
||||||
tool_call_buffers: dict[str, dict[str, Any]] = {}
|
tool_call_buffers: dict[str, dict[str, Any]] = {}
|
||||||
tool_call_args_emitted: set[str] = set()
|
|
||||||
finish_reason = "stop"
|
finish_reason = "stop"
|
||||||
reasoning_content: str | None = None
|
|
||||||
streamed_reasoning = False
|
|
||||||
|
|
||||||
async for event in iter_sse(response):
|
async for event in iter_sse(response):
|
||||||
event_type = event.get("type")
|
event_type = event.get("type")
|
||||||
@@ -112,26 +94,6 @@ async def consume_sse_with_reasoning(
|
|||||||
content += delta_text
|
content += delta_text
|
||||||
if on_content_delta and delta_text:
|
if on_content_delta and delta_text:
|
||||||
await on_content_delta(delta_text)
|
await on_content_delta(delta_text)
|
||||||
elif event_type == "response.reasoning_summary_text.delta":
|
|
||||||
delta_text = event.get("delta") or ""
|
|
||||||
if delta_text:
|
|
||||||
reasoning_content = (reasoning_content or "") + delta_text
|
|
||||||
streamed_reasoning = True
|
|
||||||
if on_reasoning_delta:
|
|
||||||
await on_reasoning_delta(delta_text)
|
|
||||||
elif event_type == "response.reasoning_summary_text.done":
|
|
||||||
text = event.get("text") or ""
|
|
||||||
if text and not streamed_reasoning and not reasoning_content:
|
|
||||||
reasoning_content = text
|
|
||||||
if on_reasoning_delta:
|
|
||||||
await on_reasoning_delta(text)
|
|
||||||
elif event_type == "response.reasoning_summary_part.done":
|
|
||||||
part = event.get("part") or {}
|
|
||||||
text = part.get("text") if part.get("type") == "summary_text" else None
|
|
||||||
if text and not streamed_reasoning and not reasoning_content:
|
|
||||||
reasoning_content = text
|
|
||||||
if on_reasoning_delta:
|
|
||||||
await on_reasoning_delta(text)
|
|
||||||
elif event_type == "response.function_call_arguments.delta":
|
elif event_type == "response.function_call_arguments.delta":
|
||||||
call_id = event.get("call_id")
|
call_id = event.get("call_id")
|
||||||
if call_id and call_id in tool_call_buffers:
|
if call_id and call_id in tool_call_buffers:
|
||||||
@@ -146,15 +108,7 @@ async def consume_sse_with_reasoning(
|
|||||||
elif event_type == "response.function_call_arguments.done":
|
elif event_type == "response.function_call_arguments.done":
|
||||||
call_id = event.get("call_id")
|
call_id = event.get("call_id")
|
||||||
if call_id and call_id in tool_call_buffers:
|
if call_id and call_id in tool_call_buffers:
|
||||||
arguments = event.get("arguments") or ""
|
tool_call_buffers[call_id]["arguments"] = event.get("arguments") or ""
|
||||||
tool_call_buffers[call_id]["arguments"] = arguments
|
|
||||||
if on_tool_call_delta:
|
|
||||||
tool_call_args_emitted.add(str(call_id))
|
|
||||||
await on_tool_call_delta({
|
|
||||||
"call_id": str(call_id),
|
|
||||||
"name": str(tool_call_buffers[call_id].get("name") or ""),
|
|
||||||
"arguments": str(arguments),
|
|
||||||
})
|
|
||||||
elif event_type == "response.output_item.done":
|
elif event_type == "response.output_item.done":
|
||||||
item = event.get("item") or {}
|
item = event.get("item") or {}
|
||||||
if item.get("type") == "function_call":
|
if item.get("type") == "function_call":
|
||||||
@@ -163,13 +117,6 @@ async def consume_sse_with_reasoning(
|
|||||||
continue
|
continue
|
||||||
buf = tool_call_buffers.get(call_id) or {}
|
buf = tool_call_buffers.get(call_id) or {}
|
||||||
args_raw = buf.get("arguments") or item.get("arguments") or "{}"
|
args_raw = buf.get("arguments") or item.get("arguments") or "{}"
|
||||||
if on_tool_call_delta and str(call_id) not in tool_call_args_emitted:
|
|
||||||
tool_call_args_emitted.add(str(call_id))
|
|
||||||
await on_tool_call_delta({
|
|
||||||
"call_id": str(call_id),
|
|
||||||
"name": str(buf.get("name") or item.get("name") or ""),
|
|
||||||
"arguments": str(args_raw),
|
|
||||||
})
|
|
||||||
try:
|
try:
|
||||||
args = json.loads(args_raw)
|
args = json.loads(args_raw)
|
||||||
except Exception:
|
except Exception:
|
||||||
@@ -188,44 +135,14 @@ async def consume_sse_with_reasoning(
|
|||||||
arguments=args,
|
arguments=args,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
elif item.get("type") == "reasoning" and not reasoning_content:
|
|
||||||
summary = _extract_reasoning_summary_from_output([item])
|
|
||||||
if summary:
|
|
||||||
reasoning_content = summary
|
|
||||||
if on_reasoning_delta:
|
|
||||||
await on_reasoning_delta(summary)
|
|
||||||
elif event_type == "response.completed":
|
elif event_type == "response.completed":
|
||||||
response_obj = event.get("response") or {}
|
status = (event.get("response") or {}).get("status")
|
||||||
status = response_obj.get("status")
|
|
||||||
finish_reason = map_finish_reason(status)
|
finish_reason = map_finish_reason(status)
|
||||||
if not reasoning_content:
|
|
||||||
summary = _extract_reasoning_summary_from_output(response_obj.get("output") or [])
|
|
||||||
if summary:
|
|
||||||
reasoning_content = summary
|
|
||||||
if on_reasoning_delta:
|
|
||||||
await on_reasoning_delta(summary)
|
|
||||||
elif event_type in {"error", "response.failed"}:
|
elif event_type in {"error", "response.failed"}:
|
||||||
detail = event.get("error") or event.get("message") or event
|
detail = event.get("error") or event.get("message") or event
|
||||||
raise RuntimeError(f"Response failed: {str(detail)[:500]}")
|
raise RuntimeError(f"Response failed: {str(detail)[:500]}")
|
||||||
|
|
||||||
return content, tool_calls, finish_reason, reasoning_content
|
return content, tool_calls, finish_reason
|
||||||
|
|
||||||
|
|
||||||
def _extract_reasoning_summary_from_output(output: Any) -> str | None:
|
|
||||||
parts: list[str] = []
|
|
||||||
for item in output or []:
|
|
||||||
if not isinstance(item, dict):
|
|
||||||
dump = getattr(item, "model_dump", None)
|
|
||||||
item = dump() if callable(dump) else vars(item)
|
|
||||||
if item.get("type") != "reasoning":
|
|
||||||
continue
|
|
||||||
for summary in item.get("summary") or []:
|
|
||||||
if not isinstance(summary, dict):
|
|
||||||
dump = getattr(summary, "model_dump", None)
|
|
||||||
summary = dump() if callable(dump) else vars(summary)
|
|
||||||
if summary.get("type") == "summary_text" and summary.get("text"):
|
|
||||||
parts.append(summary["text"])
|
|
||||||
return "".join(parts) or None
|
|
||||||
|
|
||||||
|
|
||||||
def parse_response_output(response: Any) -> LLMResponse:
|
def parse_response_output(response: Any) -> LLMResponse:
|
||||||
@@ -313,7 +230,6 @@ async def consume_sdk_stream(
|
|||||||
content = ""
|
content = ""
|
||||||
tool_calls: list[ToolCallRequest] = []
|
tool_calls: list[ToolCallRequest] = []
|
||||||
tool_call_buffers: dict[str, dict[str, Any]] = {}
|
tool_call_buffers: dict[str, dict[str, Any]] = {}
|
||||||
tool_call_args_emitted: set[str] = set()
|
|
||||||
finish_reason = "stop"
|
finish_reason = "stop"
|
||||||
usage: dict[str, int] = {}
|
usage: dict[str, int] = {}
|
||||||
reasoning_content: str | None = None
|
reasoning_content: str | None = None
|
||||||
@@ -356,15 +272,7 @@ async def consume_sdk_stream(
|
|||||||
elif event_type == "response.function_call_arguments.done":
|
elif event_type == "response.function_call_arguments.done":
|
||||||
call_id = getattr(event, "call_id", None)
|
call_id = getattr(event, "call_id", None)
|
||||||
if call_id and call_id in tool_call_buffers:
|
if call_id and call_id in tool_call_buffers:
|
||||||
arguments = getattr(event, "arguments", "") or ""
|
tool_call_buffers[call_id]["arguments"] = getattr(event, "arguments", "") or ""
|
||||||
tool_call_buffers[call_id]["arguments"] = arguments
|
|
||||||
if on_tool_call_delta:
|
|
||||||
tool_call_args_emitted.add(str(call_id))
|
|
||||||
await on_tool_call_delta({
|
|
||||||
"call_id": str(call_id),
|
|
||||||
"name": str(tool_call_buffers[call_id].get("name") or ""),
|
|
||||||
"arguments": str(arguments),
|
|
||||||
})
|
|
||||||
elif event_type == "response.output_item.done":
|
elif event_type == "response.output_item.done":
|
||||||
item = getattr(event, "item", None)
|
item = getattr(event, "item", None)
|
||||||
if item and getattr(item, "type", None) == "function_call":
|
if item and getattr(item, "type", None) == "function_call":
|
||||||
@@ -373,13 +281,6 @@ async def consume_sdk_stream(
|
|||||||
continue
|
continue
|
||||||
buf = tool_call_buffers.get(call_id) or {}
|
buf = tool_call_buffers.get(call_id) or {}
|
||||||
args_raw = buf.get("arguments") or getattr(item, "arguments", None) or "{}"
|
args_raw = buf.get("arguments") or getattr(item, "arguments", None) or "{}"
|
||||||
if on_tool_call_delta and str(call_id) not in tool_call_args_emitted:
|
|
||||||
tool_call_args_emitted.add(str(call_id))
|
|
||||||
await on_tool_call_delta({
|
|
||||||
"call_id": str(call_id),
|
|
||||||
"name": str(buf.get("name") or getattr(item, "name", None) or ""),
|
|
||||||
"arguments": str(args_raw),
|
|
||||||
})
|
|
||||||
try:
|
try:
|
||||||
args = json.loads(args_raw)
|
args = json.loads(args_raw)
|
||||||
except Exception:
|
except Exception:
|
||||||
|
|||||||
@@ -34,7 +34,7 @@ class ProviderSpec:
|
|||||||
display_name: str = "" # shown in `nanobot status`
|
display_name: str = "" # shown in `nanobot status`
|
||||||
|
|
||||||
# which provider implementation to use
|
# which provider implementation to use
|
||||||
# "openai_compat" | "anthropic" | "azure_openai" | "openai_codex" | "github_copilot" | "bedrock"
|
# "openai_compat" | "anthropic" | "azure_openai" | "openai_codex" | "github_copilot" | "xai_oauth" | "bedrock"
|
||||||
backend: str = "openai_compat"
|
backend: str = "openai_compat"
|
||||||
|
|
||||||
# extra env vars, e.g. (("ZHIPUAI_API_KEY", "{api_key}"),)
|
# extra env vars, e.g. (("ZHIPUAI_API_KEY", "{api_key}"),)
|
||||||
@@ -71,11 +71,6 @@ class ProviderSpec:
|
|||||||
# "reasoning_split" — {"reasoning_split": true/false} (MiniMax)
|
# "reasoning_split" — {"reasoning_split": true/false} (MiniMax)
|
||||||
thinking_style: str = ""
|
thinking_style: str = ""
|
||||||
|
|
||||||
# Gateway-native reasoning control to pair with model-level thinking styles.
|
|
||||||
# "reasoning_effort" — {"reasoning": {"effort": <none|minimal|...>}}
|
|
||||||
# (OpenRouter)
|
|
||||||
gateway_reasoning_style: str = ""
|
|
||||||
|
|
||||||
# When True, treat the "reasoning" response field as formal content
|
# When True, treat the "reasoning" response field as formal content
|
||||||
# when "content" is empty. Only set this for providers (e.g. StepFun)
|
# when "content" is empty. Only set this for providers (e.g. StepFun)
|
||||||
# whose API returns the actual answer in "reasoning" instead of "content".
|
# whose API returns the actual answer in "reasoning" instead of "content".
|
||||||
@@ -147,7 +142,6 @@ PROVIDERS: tuple[ProviderSpec, ...] = (
|
|||||||
detect_by_base_keyword="openrouter",
|
detect_by_base_keyword="openrouter",
|
||||||
default_api_base="https://openrouter.ai/api/v1",
|
default_api_base="https://openrouter.ai/api/v1",
|
||||||
supports_prompt_caching=True,
|
supports_prompt_caching=True,
|
||||||
gateway_reasoning_style="reasoning_effort",
|
|
||||||
),
|
),
|
||||||
# Hugging Face Inference Providers: OpenAI-compatible router for chat models.
|
# Hugging Face Inference Providers: OpenAI-compatible router for chat models.
|
||||||
ProviderSpec(
|
ProviderSpec(
|
||||||
@@ -199,18 +193,6 @@ PROVIDERS: tuple[ProviderSpec, ...] = (
|
|||||||
default_api_base="https://api.siliconflow.cn/v1",
|
default_api_base="https://api.siliconflow.cn/v1",
|
||||||
),
|
),
|
||||||
|
|
||||||
# Novita AI: OpenAI-compatible gateway for hosted model APIs.
|
|
||||||
ProviderSpec(
|
|
||||||
name="novita",
|
|
||||||
keywords=("novita",),
|
|
||||||
env_key="NOVITA_API_KEY",
|
|
||||||
display_name="Novita AI",
|
|
||||||
backend="openai_compat",
|
|
||||||
is_gateway=True,
|
|
||||||
detect_by_base_keyword="novita",
|
|
||||||
default_api_base="https://api.novita.ai/openai",
|
|
||||||
),
|
|
||||||
|
|
||||||
# VolcEngine (火山引擎): OpenAI-compatible gateway, pay-per-use models
|
# VolcEngine (火山引擎): OpenAI-compatible gateway, pay-per-use models
|
||||||
ProviderSpec(
|
ProviderSpec(
|
||||||
name="volcengine",
|
name="volcengine",
|
||||||
@@ -309,6 +291,18 @@ PROVIDERS: tuple[ProviderSpec, ...] = (
|
|||||||
is_oauth=True,
|
is_oauth=True,
|
||||||
supports_max_completion_tokens=True,
|
supports_max_completion_tokens=True,
|
||||||
),
|
),
|
||||||
|
# xAI Grok OAuth: SuperGrok subscription-backed Responses API provider
|
||||||
|
ProviderSpec(
|
||||||
|
name="xai_oauth",
|
||||||
|
keywords=("xai-oauth", "grok-oauth", "x-ai-oauth", "xai-grok-oauth"),
|
||||||
|
env_key="",
|
||||||
|
display_name="xAI Grok OAuth",
|
||||||
|
backend="xai_oauth",
|
||||||
|
default_api_base="https://api.x.ai/v1",
|
||||||
|
strip_model_prefix=True,
|
||||||
|
is_oauth=True,
|
||||||
|
supports_max_completion_tokens=True,
|
||||||
|
),
|
||||||
# DeepSeek: OpenAI-compatible at api.deepseek.com
|
# DeepSeek: OpenAI-compatible at api.deepseek.com
|
||||||
ProviderSpec(
|
ProviderSpec(
|
||||||
name="deepseek",
|
name="deepseek",
|
||||||
|
|||||||
@@ -7,25 +7,6 @@ from pathlib import Path
|
|||||||
import httpx
|
import httpx
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
_TRANSCRIPTIONS_PATH = "audio/transcriptions"
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_transcription_url(api_base: str | None, default_url: str) -> str:
|
|
||||||
"""Resolve the full transcription endpoint URL.
|
|
||||||
|
|
||||||
Accepts either a chat-style base (e.g. ``https://api.groq.com/openai/v1``)
|
|
||||||
or a complete URL already ending in ``/audio/transcriptions``. A chat-style
|
|
||||||
base — the form users naturally copy from their LLM provider config — gets
|
|
||||||
the path appended instead of being POSTed verbatim and 404ing (#3637).
|
|
||||||
"""
|
|
||||||
if not api_base:
|
|
||||||
return default_url
|
|
||||||
base = api_base.rstrip("/")
|
|
||||||
if base.endswith(_TRANSCRIPTIONS_PATH):
|
|
||||||
return base
|
|
||||||
return f"{base}/{_TRANSCRIPTIONS_PATH}"
|
|
||||||
|
|
||||||
|
|
||||||
# Up to 3 retries (4 attempts total) with exponential backoff on transient
|
# Up to 3 retries (4 attempts total) with exponential backoff on transient
|
||||||
# failures. Whisper endpoints occasionally return 502/503 under load, and
|
# failures. Whisper endpoints occasionally return 502/503 under load, and
|
||||||
# mobile-network transcription callers hit sporadic connect/read errors.
|
# mobile-network transcription callers hit sporadic connect/read errors.
|
||||||
@@ -146,12 +127,12 @@ class OpenAITranscriptionProvider:
|
|||||||
language: str | None = None,
|
language: str | None = None,
|
||||||
):
|
):
|
||||||
self.api_key = api_key or os.environ.get("OPENAI_API_KEY")
|
self.api_key = api_key or os.environ.get("OPENAI_API_KEY")
|
||||||
self.api_url = _resolve_transcription_url(
|
self.api_url = (
|
||||||
api_base or os.environ.get("OPENAI_TRANSCRIPTION_BASE_URL"),
|
api_base
|
||||||
"https://api.openai.com/v1/audio/transcriptions",
|
or os.environ.get("OPENAI_TRANSCRIPTION_BASE_URL")
|
||||||
|
or "https://api.openai.com/v1/audio/transcriptions"
|
||||||
)
|
)
|
||||||
self.language = language or None
|
self.language = language or None
|
||||||
logger.debug("OpenAI transcription endpoint: {}", self.api_url)
|
|
||||||
|
|
||||||
async def transcribe(self, file_path: str | Path) -> str:
|
async def transcribe(self, file_path: str | Path) -> str:
|
||||||
if not self.api_key:
|
if not self.api_key:
|
||||||
@@ -185,12 +166,12 @@ class GroqTranscriptionProvider:
|
|||||||
language: str | None = None,
|
language: str | None = None,
|
||||||
):
|
):
|
||||||
self.api_key = api_key or os.environ.get("GROQ_API_KEY")
|
self.api_key = api_key or os.environ.get("GROQ_API_KEY")
|
||||||
self.api_url = _resolve_transcription_url(
|
self.api_url = (
|
||||||
api_base or os.environ.get("GROQ_BASE_URL"),
|
api_base
|
||||||
"https://api.groq.com/openai/v1/audio/transcriptions",
|
or os.environ.get("GROQ_BASE_URL")
|
||||||
|
or "https://api.groq.com/openai/v1/audio/transcriptions"
|
||||||
)
|
)
|
||||||
self.language = language or None
|
self.language = language or None
|
||||||
logger.debug("Groq transcription endpoint: {}", self.api_url)
|
|
||||||
|
|
||||||
async def transcribe(self, file_path: str | Path) -> str:
|
async def transcribe(self, file_path: str | Path) -> str:
|
||||||
"""
|
"""
|
||||||
|
|||||||
@@ -0,0 +1,768 @@
|
|||||||
|
"""xAI Grok OAuth credential flow and Responses provider."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import base64
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import secrets
|
||||||
|
import time
|
||||||
|
import webbrowser
|
||||||
|
from collections.abc import Awaitable, Callable
|
||||||
|
from contextlib import suppress
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from hashlib import sha256
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
from pathlib import Path
|
||||||
|
from threading import Event, Thread
|
||||||
|
from typing import Any
|
||||||
|
from urllib.parse import parse_qs, urlencode, urlparse
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from filelock import FileLock
|
||||||
|
|
||||||
|
from nanobot.providers.base import LLMProvider, LLMResponse, ToolCallRequest
|
||||||
|
from nanobot.providers.openai_responses import consume_sse, convert_messages, convert_tools
|
||||||
|
|
||||||
|
DEFAULT_XAI_API_BASE = "https://api.x.ai/v1"
|
||||||
|
DEFAULT_XAI_AUTH_ISSUER = "https://auth.x.ai"
|
||||||
|
DEFAULT_XAI_DISCOVERY_URL = f"{DEFAULT_XAI_AUTH_ISSUER}/.well-known/openid-configuration"
|
||||||
|
DEFAULT_XAI_REDIRECT_URI = "http://127.0.0.1:56121/callback"
|
||||||
|
DEFAULT_XAI_CLIENT_ID = "b1a00492-073a-47ea-816f-4c329264a828"
|
||||||
|
DEFAULT_XAI_SCOPE = "openid profile email offline_access grok-cli:access api:access"
|
||||||
|
|
||||||
|
_SERVICE_NAME = "nanobot.xai_oauth"
|
||||||
|
_SECRET_USERNAME = "default"
|
||||||
|
_TOKEN_SKEW_SECONDS = 60
|
||||||
|
_LOGIN_TIMEOUT_SECONDS = 300
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class XaiOAuthEndpoints:
|
||||||
|
authorization_endpoint: str
|
||||||
|
token_endpoint: str
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class XaiOAuthCredential:
|
||||||
|
access_token: str
|
||||||
|
refresh_token: str = ""
|
||||||
|
expires_at: float | None = None
|
||||||
|
account_id: str | None = None
|
||||||
|
token_type: str = "Bearer"
|
||||||
|
api_base: str = DEFAULT_XAI_API_BASE
|
||||||
|
storage: str = "unknown"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_expiring(self) -> bool:
|
||||||
|
return self.expires_at is not None and self.expires_at <= time.time() + _TOKEN_SKEW_SECONDS
|
||||||
|
|
||||||
|
|
||||||
|
def _nanobot_home() -> Path:
|
||||||
|
override = os.environ.get("NANOBOT_HOME")
|
||||||
|
if override:
|
||||||
|
return Path(override).expanduser()
|
||||||
|
from nanobot.config.loader import get_config_path
|
||||||
|
|
||||||
|
return get_config_path().parent
|
||||||
|
|
||||||
|
|
||||||
|
def _auth_dir() -> Path:
|
||||||
|
return _nanobot_home() / "auth"
|
||||||
|
|
||||||
|
|
||||||
|
def get_xai_oauth_metadata_path() -> Path:
|
||||||
|
"""Return the non-secret xAI OAuth metadata path."""
|
||||||
|
return _auth_dir() / "xai-oauth.json"
|
||||||
|
|
||||||
|
|
||||||
|
def _lock_path() -> Path:
|
||||||
|
return get_xai_oauth_metadata_path().with_suffix(".lock")
|
||||||
|
|
||||||
|
|
||||||
|
def _write_private_json(path: Path, payload: dict[str, Any]) -> None:
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with suppress(OSError):
|
||||||
|
path.parent.chmod(0o700)
|
||||||
|
tmp = path.with_suffix(path.suffix + ".tmp")
|
||||||
|
tmp.write_text(json.dumps(payload, indent=2, ensure_ascii=False), encoding="utf-8")
|
||||||
|
with suppress(OSError):
|
||||||
|
tmp.chmod(0o600)
|
||||||
|
tmp.replace(path)
|
||||||
|
with suppress(OSError):
|
||||||
|
path.chmod(0o600)
|
||||||
|
|
||||||
|
|
||||||
|
def _read_json(path: Path) -> dict[str, Any]:
|
||||||
|
return json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def _keyring_set(tokens: dict[str, Any]) -> bool:
|
||||||
|
try:
|
||||||
|
import keyring # type: ignore[import-not-found]
|
||||||
|
|
||||||
|
keyring.set_password(_SERVICE_NAME, _SECRET_USERNAME, json.dumps(tokens))
|
||||||
|
return True
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _keyring_get() -> dict[str, Any] | None:
|
||||||
|
try:
|
||||||
|
import keyring # type: ignore[import-not-found]
|
||||||
|
|
||||||
|
raw = keyring.get_password(_SERVICE_NAME, _SECRET_USERNAME)
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
if not raw:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
payload = json.loads(raw)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
return None
|
||||||
|
return payload if isinstance(payload, dict) else None
|
||||||
|
|
||||||
|
|
||||||
|
def _keyring_delete() -> None:
|
||||||
|
try:
|
||||||
|
import keyring # type: ignore[import-not-found]
|
||||||
|
|
||||||
|
keyring.delete_password(_SERVICE_NAME, _SECRET_USERNAME)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _token_payload(credential: XaiOAuthCredential) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"access_token": credential.access_token,
|
||||||
|
"refresh_token": credential.refresh_token,
|
||||||
|
"expires_at": credential.expires_at,
|
||||||
|
"token_type": credential.token_type,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def save_xai_oauth_credential(credential: XaiOAuthCredential) -> XaiOAuthCredential:
|
||||||
|
"""Persist xAI OAuth tokens, preferring OS keychain storage."""
|
||||||
|
with FileLock(str(_lock_path())):
|
||||||
|
tokens = _token_payload(credential)
|
||||||
|
metadata: dict[str, Any] = {
|
||||||
|
"provider": "xai_oauth",
|
||||||
|
"api_base": credential.api_base,
|
||||||
|
"account_id": credential.account_id,
|
||||||
|
"expires_at": credential.expires_at,
|
||||||
|
"updated_at": int(time.time()),
|
||||||
|
}
|
||||||
|
if _keyring_set(tokens):
|
||||||
|
metadata["storage"] = "keyring"
|
||||||
|
else:
|
||||||
|
metadata["storage"] = "file"
|
||||||
|
metadata["tokens"] = tokens
|
||||||
|
_write_private_json(get_xai_oauth_metadata_path(), metadata)
|
||||||
|
return XaiOAuthCredential(
|
||||||
|
access_token=credential.access_token,
|
||||||
|
refresh_token=credential.refresh_token,
|
||||||
|
expires_at=credential.expires_at,
|
||||||
|
account_id=credential.account_id,
|
||||||
|
token_type=credential.token_type,
|
||||||
|
api_base=credential.api_base,
|
||||||
|
storage=str(metadata["storage"]),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def load_xai_oauth_credential() -> XaiOAuthCredential | None:
|
||||||
|
"""Load xAI OAuth credentials from keyring or the private file fallback."""
|
||||||
|
path = get_xai_oauth_metadata_path()
|
||||||
|
if not path.exists():
|
||||||
|
return None
|
||||||
|
with FileLock(str(_lock_path())):
|
||||||
|
try:
|
||||||
|
metadata = _read_json(path)
|
||||||
|
except (OSError, json.JSONDecodeError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
storage = str(metadata.get("storage") or "file")
|
||||||
|
tokens = _keyring_get() if storage == "keyring" else metadata.get("tokens")
|
||||||
|
if not isinstance(tokens, dict):
|
||||||
|
return None
|
||||||
|
access_token = str(tokens.get("access_token") or "")
|
||||||
|
if not access_token:
|
||||||
|
return None
|
||||||
|
|
||||||
|
return XaiOAuthCredential(
|
||||||
|
access_token=access_token,
|
||||||
|
refresh_token=str(tokens.get("refresh_token") or ""),
|
||||||
|
expires_at=_as_float(tokens.get("expires_at") or metadata.get("expires_at")),
|
||||||
|
account_id=_as_str(metadata.get("account_id")),
|
||||||
|
token_type=str(tokens.get("token_type") or "Bearer"),
|
||||||
|
api_base=str(metadata.get("api_base") or DEFAULT_XAI_API_BASE),
|
||||||
|
storage=storage,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def delete_xai_oauth_credentials() -> list[Path]:
|
||||||
|
"""Delete persisted xAI OAuth credentials and return removed local paths."""
|
||||||
|
removed: list[Path] = []
|
||||||
|
path = get_xai_oauth_metadata_path()
|
||||||
|
lock_path = _lock_path()
|
||||||
|
with FileLock(str(lock_path)):
|
||||||
|
_keyring_delete()
|
||||||
|
try:
|
||||||
|
path.unlink()
|
||||||
|
removed.append(path)
|
||||||
|
except FileNotFoundError:
|
||||||
|
pass
|
||||||
|
try:
|
||||||
|
lock_path.unlink()
|
||||||
|
except FileNotFoundError:
|
||||||
|
pass
|
||||||
|
return removed
|
||||||
|
|
||||||
|
|
||||||
|
def get_xai_oauth_login_status() -> XaiOAuthCredential | None:
|
||||||
|
return load_xai_oauth_credential()
|
||||||
|
|
||||||
|
|
||||||
|
def pkce_challenge(verifier: str) -> str:
|
||||||
|
digest = sha256(verifier.encode("ascii")).digest()
|
||||||
|
return base64.urlsafe_b64encode(digest).decode("ascii").rstrip("=")
|
||||||
|
|
||||||
|
|
||||||
|
def _new_pkce_verifier() -> str:
|
||||||
|
return base64.urlsafe_b64encode(secrets.token_bytes(48)).decode("ascii").rstrip("=")
|
||||||
|
|
||||||
|
|
||||||
|
def build_xai_authorization_url(
|
||||||
|
endpoints: XaiOAuthEndpoints,
|
||||||
|
*,
|
||||||
|
verifier: str,
|
||||||
|
state: str,
|
||||||
|
nonce: str | None = None,
|
||||||
|
redirect_uri: str = DEFAULT_XAI_REDIRECT_URI,
|
||||||
|
) -> str:
|
||||||
|
params = {
|
||||||
|
"response_type": "code",
|
||||||
|
"client_id": DEFAULT_XAI_CLIENT_ID,
|
||||||
|
"redirect_uri": redirect_uri,
|
||||||
|
"scope": DEFAULT_XAI_SCOPE,
|
||||||
|
"code_challenge": pkce_challenge(verifier),
|
||||||
|
"code_challenge_method": "S256",
|
||||||
|
"state": state,
|
||||||
|
"nonce": nonce or secrets.token_urlsafe(16),
|
||||||
|
"plan": "generic",
|
||||||
|
"referrer": "nanobot",
|
||||||
|
}
|
||||||
|
return f"{endpoints.authorization_endpoint}?{urlencode(params)}"
|
||||||
|
|
||||||
|
|
||||||
|
def discover_xai_oauth_endpoints() -> XaiOAuthEndpoints:
|
||||||
|
try:
|
||||||
|
with httpx.Client(timeout=20.0, follow_redirects=True, trust_env=True) as client:
|
||||||
|
response = client.get(DEFAULT_XAI_DISCOVERY_URL)
|
||||||
|
response.raise_for_status()
|
||||||
|
payload = response.json()
|
||||||
|
except Exception:
|
||||||
|
payload = {}
|
||||||
|
|
||||||
|
endpoints = XaiOAuthEndpoints(
|
||||||
|
authorization_endpoint=str(
|
||||||
|
payload.get("authorization_endpoint")
|
||||||
|
or f"{DEFAULT_XAI_AUTH_ISSUER}/authorize"
|
||||||
|
),
|
||||||
|
token_endpoint=str(
|
||||||
|
payload.get("token_endpoint")
|
||||||
|
or f"{DEFAULT_XAI_AUTH_ISSUER}/oauth/token"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
_validate_xai_endpoint(endpoints.authorization_endpoint, "authorization_endpoint")
|
||||||
|
_validate_xai_endpoint(endpoints.token_endpoint, "token_endpoint")
|
||||||
|
return endpoints
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_xai_endpoint(url: str, label: str) -> None:
|
||||||
|
parsed = urlparse(url)
|
||||||
|
host = parsed.hostname or ""
|
||||||
|
if parsed.scheme != "https" or not (host == "x.ai" or host.endswith(".x.ai")):
|
||||||
|
raise RuntimeError(f"Refusing non-xAI OAuth {label}: {url}")
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_callback_value(raw: str) -> tuple[str, str | None]:
|
||||||
|
raw = raw.strip()
|
||||||
|
parsed = urlparse(raw)
|
||||||
|
if parsed.scheme and parsed.netloc:
|
||||||
|
params = parse_qs(parsed.query)
|
||||||
|
code = (params.get("code") or [""])[0]
|
||||||
|
state = (params.get("state") or [None])[0]
|
||||||
|
if not code:
|
||||||
|
raise RuntimeError("OAuth callback URL did not contain a code.")
|
||||||
|
return code, state
|
||||||
|
if raw.startswith("?") or "=" in raw:
|
||||||
|
params = parse_qs(raw.lstrip("?"))
|
||||||
|
code = (params.get("code") or [""])[0]
|
||||||
|
state = (params.get("state") or [None])[0]
|
||||||
|
if not code:
|
||||||
|
raise RuntimeError("OAuth callback query did not contain a code.")
|
||||||
|
return code, state
|
||||||
|
if raw:
|
||||||
|
return raw, None
|
||||||
|
raise RuntimeError("No OAuth code provided.")
|
||||||
|
|
||||||
|
|
||||||
|
def _decode_jwt_payload(token: str) -> dict[str, Any]:
|
||||||
|
parts = token.split(".")
|
||||||
|
if len(parts) < 2:
|
||||||
|
return {}
|
||||||
|
data = parts[1] + "=" * (-len(parts[1]) % 4)
|
||||||
|
try:
|
||||||
|
decoded = base64.urlsafe_b64decode(data.encode("ascii"))
|
||||||
|
payload = json.loads(decoded)
|
||||||
|
except Exception:
|
||||||
|
return {}
|
||||||
|
return payload if isinstance(payload, dict) else {}
|
||||||
|
|
||||||
|
|
||||||
|
def _credential_from_token_response(payload: dict[str, Any], previous: XaiOAuthCredential | None = None) -> XaiOAuthCredential:
|
||||||
|
access_token = str(payload.get("access_token") or "")
|
||||||
|
if not access_token:
|
||||||
|
raise RuntimeError("xAI token response did not include an access token.")
|
||||||
|
|
||||||
|
claims = _decode_jwt_payload(access_token)
|
||||||
|
id_claims = _decode_jwt_payload(str(payload.get("id_token") or ""))
|
||||||
|
expires_at = _as_float(payload.get("expires_at"))
|
||||||
|
if expires_at is None:
|
||||||
|
expires_in = _as_float(payload.get("expires_in"))
|
||||||
|
expires_at = time.time() + expires_in if expires_in else _as_float(claims.get("exp"))
|
||||||
|
|
||||||
|
account_id = (
|
||||||
|
_as_str(id_claims.get("email"))
|
||||||
|
or _as_str(id_claims.get("preferred_username"))
|
||||||
|
or _as_str(id_claims.get("sub"))
|
||||||
|
or _as_str(claims.get("sub"))
|
||||||
|
or (previous.account_id if previous else None)
|
||||||
|
)
|
||||||
|
refresh_token = str(payload.get("refresh_token") or (previous.refresh_token if previous else ""))
|
||||||
|
|
||||||
|
return XaiOAuthCredential(
|
||||||
|
access_token=access_token,
|
||||||
|
refresh_token=refresh_token,
|
||||||
|
expires_at=expires_at,
|
||||||
|
account_id=account_id,
|
||||||
|
token_type=str(payload.get("token_type") or (previous.token_type if previous else "Bearer")),
|
||||||
|
api_base=previous.api_base if previous else DEFAULT_XAI_API_BASE,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def exchange_xai_oauth_code(
|
||||||
|
code: str,
|
||||||
|
*,
|
||||||
|
verifier: str,
|
||||||
|
endpoints: XaiOAuthEndpoints | None = None,
|
||||||
|
redirect_uri: str = DEFAULT_XAI_REDIRECT_URI,
|
||||||
|
) -> XaiOAuthCredential:
|
||||||
|
endpoints = endpoints or discover_xai_oauth_endpoints()
|
||||||
|
challenge = pkce_challenge(verifier)
|
||||||
|
with httpx.Client(timeout=30.0, follow_redirects=True, trust_env=True) as client:
|
||||||
|
response = client.post(
|
||||||
|
endpoints.token_endpoint,
|
||||||
|
headers={"Accept": "application/json"},
|
||||||
|
data={
|
||||||
|
"grant_type": "authorization_code",
|
||||||
|
"client_id": DEFAULT_XAI_CLIENT_ID,
|
||||||
|
"code": code,
|
||||||
|
"redirect_uri": redirect_uri,
|
||||||
|
"code_verifier": verifier,
|
||||||
|
"code_challenge": challenge,
|
||||||
|
"code_challenge_method": "S256",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if response.status_code >= 400:
|
||||||
|
raise RuntimeError(f"xAI token exchange failed: HTTP {response.status_code}: {response.text[:500]}")
|
||||||
|
return _credential_from_token_response(response.json())
|
||||||
|
|
||||||
|
|
||||||
|
def refresh_xai_oauth_credential(credential: XaiOAuthCredential | None = None) -> XaiOAuthCredential:
|
||||||
|
credential = credential or load_xai_oauth_credential()
|
||||||
|
if not credential or not credential.refresh_token:
|
||||||
|
raise RuntimeError("xAI Grok OAuth is not logged in. Run: nanobot provider login xai-oauth")
|
||||||
|
|
||||||
|
endpoints = discover_xai_oauth_endpoints()
|
||||||
|
with httpx.Client(timeout=30.0, follow_redirects=True, trust_env=True) as client:
|
||||||
|
response = client.post(
|
||||||
|
endpoints.token_endpoint,
|
||||||
|
headers={"Accept": "application/json"},
|
||||||
|
data={
|
||||||
|
"grant_type": "refresh_token",
|
||||||
|
"client_id": DEFAULT_XAI_CLIENT_ID,
|
||||||
|
"refresh_token": credential.refresh_token,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if response.status_code >= 400:
|
||||||
|
raise RuntimeError(f"xAI token refresh failed: HTTP {response.status_code}: {response.text[:500]}")
|
||||||
|
return save_xai_oauth_credential(_credential_from_token_response(response.json(), previous=credential))
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_xai_oauth_credential(*, force_refresh: bool = False) -> XaiOAuthCredential:
|
||||||
|
credential = load_xai_oauth_credential()
|
||||||
|
if not credential:
|
||||||
|
raise RuntimeError("xAI Grok OAuth is not logged in. Run: nanobot provider login xai-oauth")
|
||||||
|
if force_refresh or credential.is_expiring:
|
||||||
|
credential = refresh_xai_oauth_credential(credential)
|
||||||
|
return credential
|
||||||
|
|
||||||
|
|
||||||
|
def login_xai_oauth_interactive(
|
||||||
|
print_fn: Callable[[str], None] | None = None,
|
||||||
|
prompt_fn: Callable[[str], str] | None = None,
|
||||||
|
open_browser: bool = True,
|
||||||
|
manual_paste: bool = False,
|
||||||
|
timeout_seconds: int = _LOGIN_TIMEOUT_SECONDS,
|
||||||
|
) -> XaiOAuthCredential:
|
||||||
|
"""Run browser PKCE login and persist xAI OAuth credentials."""
|
||||||
|
printer = print_fn or print
|
||||||
|
prompt = prompt_fn or input
|
||||||
|
endpoints = discover_xai_oauth_endpoints()
|
||||||
|
verifier = _new_pkce_verifier()
|
||||||
|
state = secrets.token_urlsafe(24)
|
||||||
|
nonce = secrets.token_urlsafe(24)
|
||||||
|
authorize_url = build_xai_authorization_url(
|
||||||
|
endpoints,
|
||||||
|
verifier=verifier,
|
||||||
|
state=state,
|
||||||
|
nonce=nonce,
|
||||||
|
)
|
||||||
|
|
||||||
|
callback = _LoopbackCallback()
|
||||||
|
server_started = False if manual_paste else callback.start()
|
||||||
|
printer(f"Open: {authorize_url}")
|
||||||
|
if open_browser:
|
||||||
|
with suppress(Exception):
|
||||||
|
webbrowser.open(authorize_url)
|
||||||
|
|
||||||
|
result: dict[str, str] | None = None
|
||||||
|
if manual_paste:
|
||||||
|
printer("Paste the callback URL or xAI fallback code after authorization.")
|
||||||
|
elif server_started:
|
||||||
|
try:
|
||||||
|
result = callback.wait(timeout_seconds)
|
||||||
|
finally:
|
||||||
|
callback.stop()
|
||||||
|
else:
|
||||||
|
printer("Loopback port 56121 is unavailable; paste the callback URL or xAI fallback code.")
|
||||||
|
|
||||||
|
if result:
|
||||||
|
code = result.get("code") or ""
|
||||||
|
returned_state = result.get("state")
|
||||||
|
else:
|
||||||
|
pasted = prompt("Paste callback URL or fallback code")
|
||||||
|
code, returned_state = _parse_callback_value(pasted)
|
||||||
|
|
||||||
|
if not code:
|
||||||
|
raise RuntimeError("OAuth login did not return a code.")
|
||||||
|
if returned_state and returned_state != state:
|
||||||
|
raise RuntimeError("OAuth state mismatch. Please retry login.")
|
||||||
|
|
||||||
|
credential = exchange_xai_oauth_code(code, verifier=verifier, endpoints=endpoints)
|
||||||
|
return save_xai_oauth_credential(credential)
|
||||||
|
|
||||||
|
|
||||||
|
class _LoopbackCallback:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._event = Event()
|
||||||
|
self._result: dict[str, str] = {}
|
||||||
|
self._server: ThreadingHTTPServer | None = None
|
||||||
|
self._thread: Thread | None = None
|
||||||
|
|
||||||
|
def start(self) -> bool:
|
||||||
|
owner = self
|
||||||
|
|
||||||
|
class Handler(BaseHTTPRequestHandler):
|
||||||
|
def do_GET(self) -> None: # noqa: N802 - stdlib callback name
|
||||||
|
parsed = urlparse(self.path)
|
||||||
|
params = parse_qs(parsed.query)
|
||||||
|
code = (params.get("code") or [""])[0]
|
||||||
|
state = (params.get("state") or [""])[0]
|
||||||
|
if parsed.path != "/callback" or not code:
|
||||||
|
self.send_response(404)
|
||||||
|
self.end_headers()
|
||||||
|
return
|
||||||
|
owner._result = {"code": code, "state": state}
|
||||||
|
owner._event.set()
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Type", "text/html; charset=utf-8")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(b"<html><body>nanobot xAI OAuth complete. You may close this tab.</body></html>")
|
||||||
|
|
||||||
|
def log_message(self, format: str, *args: Any) -> None: # noqa: A002
|
||||||
|
return
|
||||||
|
|
||||||
|
class Server(ThreadingHTTPServer):
|
||||||
|
allow_reuse_address = True
|
||||||
|
daemon_threads = True
|
||||||
|
|
||||||
|
try:
|
||||||
|
self._server = Server(("127.0.0.1", 56121), Handler)
|
||||||
|
except OSError:
|
||||||
|
return False
|
||||||
|
self._thread = Thread(target=self._server.serve_forever, daemon=True)
|
||||||
|
self._thread.start()
|
||||||
|
return True
|
||||||
|
|
||||||
|
def wait(self, timeout_seconds: int) -> dict[str, str] | None:
|
||||||
|
if self._event.wait(timeout_seconds):
|
||||||
|
return dict(self._result)
|
||||||
|
return None
|
||||||
|
|
||||||
|
def stop(self) -> None:
|
||||||
|
if self._server:
|
||||||
|
self._server.shutdown()
|
||||||
|
self._server.server_close()
|
||||||
|
if self._thread:
|
||||||
|
self._thread.join(timeout=1)
|
||||||
|
|
||||||
|
|
||||||
|
def _as_float(value: Any) -> float | None:
|
||||||
|
try:
|
||||||
|
return float(value)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _as_str(value: Any) -> str | None:
|
||||||
|
return value if isinstance(value, str) and value else None
|
||||||
|
|
||||||
|
|
||||||
|
DEFAULT_XAI_MODEL = "xai-oauth/grok-4.3"
|
||||||
|
|
||||||
|
|
||||||
|
class XaiOAuthProvider(LLMProvider):
|
||||||
|
"""Use a SuperGrok OAuth session to call xAI's Responses API."""
|
||||||
|
|
||||||
|
supports_progress_deltas = True
|
||||||
|
|
||||||
|
def __init__(self, default_model: str = DEFAULT_XAI_MODEL, config: Any | None = None):
|
||||||
|
super().__init__(api_key=None, api_base=DEFAULT_XAI_API_BASE)
|
||||||
|
self.default_model = default_model
|
||||||
|
self.config = config
|
||||||
|
|
||||||
|
async def _call_xai(
|
||||||
|
self,
|
||||||
|
messages: list[dict[str, Any]],
|
||||||
|
tools: list[dict[str, Any]] | None,
|
||||||
|
model: str | None,
|
||||||
|
max_tokens: int,
|
||||||
|
temperature: float,
|
||||||
|
reasoning_effort: str | None,
|
||||||
|
tool_choice: str | dict[str, Any] | None,
|
||||||
|
on_content_delta: Callable[[str], Awaitable[None]] | None = None,
|
||||||
|
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
|
||||||
|
) -> LLMResponse:
|
||||||
|
body = _build_xai_responses_body(
|
||||||
|
messages=messages,
|
||||||
|
tools=tools,
|
||||||
|
model=model or self.default_model,
|
||||||
|
max_tokens=max_tokens,
|
||||||
|
temperature=temperature,
|
||||||
|
reasoning_effort=reasoning_effort,
|
||||||
|
tool_choice=tool_choice,
|
||||||
|
hosted_x_search=getattr(self.config, "x_search", None),
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
credential = await asyncio.to_thread(resolve_xai_oauth_credential)
|
||||||
|
try:
|
||||||
|
content, tool_calls, finish_reason = await _request_xai(
|
||||||
|
credential,
|
||||||
|
body,
|
||||||
|
on_content_delta=on_content_delta,
|
||||||
|
on_tool_call_delta=on_tool_call_delta,
|
||||||
|
)
|
||||||
|
except _XaiHTTPError as exc:
|
||||||
|
if exc.status_code != 401:
|
||||||
|
raise
|
||||||
|
credential = await asyncio.to_thread(resolve_xai_oauth_credential, force_refresh=True)
|
||||||
|
content, tool_calls, finish_reason = await _request_xai(
|
||||||
|
credential,
|
||||||
|
body,
|
||||||
|
on_content_delta=on_content_delta,
|
||||||
|
on_tool_call_delta=on_tool_call_delta,
|
||||||
|
)
|
||||||
|
return LLMResponse(content=content, tool_calls=tool_calls, finish_reason=finish_reason)
|
||||||
|
except Exception as exc:
|
||||||
|
msg = f"Error calling xAI Grok OAuth: {exc}"
|
||||||
|
retry_after = getattr(exc, "retry_after", None) or self._extract_retry_after(msg)
|
||||||
|
return LLMResponse(content=msg, finish_reason="error", retry_after=retry_after)
|
||||||
|
|
||||||
|
async def chat(
|
||||||
|
self,
|
||||||
|
messages: list[dict[str, Any]],
|
||||||
|
tools: list[dict[str, Any]] | None = None,
|
||||||
|
model: str | None = None,
|
||||||
|
max_tokens: int = 4096,
|
||||||
|
temperature: float = 0.7,
|
||||||
|
reasoning_effort: str | None = None,
|
||||||
|
tool_choice: str | dict[str, Any] | None = None,
|
||||||
|
) -> LLMResponse:
|
||||||
|
return await self._call_xai(
|
||||||
|
messages,
|
||||||
|
tools,
|
||||||
|
model,
|
||||||
|
max_tokens,
|
||||||
|
temperature,
|
||||||
|
reasoning_effort,
|
||||||
|
tool_choice,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def chat_stream(
|
||||||
|
self,
|
||||||
|
messages: list[dict[str, Any]],
|
||||||
|
tools: list[dict[str, Any]] | None = None,
|
||||||
|
model: str | None = None,
|
||||||
|
max_tokens: int = 4096,
|
||||||
|
temperature: float = 0.7,
|
||||||
|
reasoning_effort: str | None = None,
|
||||||
|
tool_choice: str | dict[str, Any] | None = None,
|
||||||
|
on_content_delta: Callable[[str], Awaitable[None]] | None = None,
|
||||||
|
on_thinking_delta: Callable[[str], Awaitable[None]] | None = None,
|
||||||
|
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
|
||||||
|
) -> LLMResponse:
|
||||||
|
_ = on_thinking_delta
|
||||||
|
return await self._call_xai(
|
||||||
|
messages,
|
||||||
|
tools,
|
||||||
|
model,
|
||||||
|
max_tokens,
|
||||||
|
temperature,
|
||||||
|
reasoning_effort,
|
||||||
|
tool_choice,
|
||||||
|
on_content_delta,
|
||||||
|
on_tool_call_delta,
|
||||||
|
)
|
||||||
|
|
||||||
|
def get_default_model(self) -> str:
|
||||||
|
return self.default_model
|
||||||
|
|
||||||
|
|
||||||
|
def _strip_model_prefix(model: str) -> str:
|
||||||
|
for prefix in ("xai-oauth/", "xai_oauth/", "grok-oauth/", "grok_oauth/"):
|
||||||
|
if model.startswith(prefix):
|
||||||
|
return model.split("/", 1)[1]
|
||||||
|
return model
|
||||||
|
|
||||||
|
|
||||||
|
def _build_xai_responses_body(
|
||||||
|
*,
|
||||||
|
messages: list[dict[str, Any]],
|
||||||
|
tools: list[dict[str, Any]] | None,
|
||||||
|
model: str,
|
||||||
|
max_tokens: int,
|
||||||
|
temperature: float,
|
||||||
|
reasoning_effort: str | None,
|
||||||
|
tool_choice: str | dict[str, Any] | None,
|
||||||
|
hosted_x_search: Any | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
system_prompt, input_items = convert_messages(LLMProvider._sanitize_empty_content(messages))
|
||||||
|
if system_prompt:
|
||||||
|
input_items = [
|
||||||
|
{"role": "system", "content": [{"type": "input_text", "text": system_prompt}]},
|
||||||
|
*input_items,
|
||||||
|
]
|
||||||
|
|
||||||
|
body: dict[str, Any] = {
|
||||||
|
"model": _strip_model_prefix(model),
|
||||||
|
"store": False,
|
||||||
|
"stream": True,
|
||||||
|
"input": input_items,
|
||||||
|
"tool_choice": tool_choice or "auto",
|
||||||
|
"parallel_tool_calls": True,
|
||||||
|
}
|
||||||
|
if max_tokens:
|
||||||
|
body["max_output_tokens"] = max_tokens
|
||||||
|
if temperature is not None:
|
||||||
|
body["temperature"] = temperature
|
||||||
|
if reasoning_effort and reasoning_effort.lower() != "none":
|
||||||
|
body["reasoning"] = {"effort": reasoning_effort}
|
||||||
|
converted_tools = convert_tools(tools) if tools else []
|
||||||
|
hosted_tool = _build_xai_hosted_x_search_tool(hosted_x_search)
|
||||||
|
if hosted_tool:
|
||||||
|
converted_tools.append(hosted_tool)
|
||||||
|
if converted_tools:
|
||||||
|
body["tools"] = converted_tools
|
||||||
|
return body
|
||||||
|
|
||||||
|
|
||||||
|
def _clean_x_handles(handles: list[str] | None) -> list[str] | None:
|
||||||
|
if not handles:
|
||||||
|
return None
|
||||||
|
cleaned = [str(handle).strip().lstrip("@") for handle in handles if str(handle).strip()]
|
||||||
|
return cleaned[:10] or None
|
||||||
|
|
||||||
|
|
||||||
|
def _build_xai_hosted_x_search_tool(config: Any | None) -> dict[str, Any] | None:
|
||||||
|
if not config or not getattr(config, "enable", False):
|
||||||
|
return None
|
||||||
|
|
||||||
|
allowed = _clean_x_handles(getattr(config, "allowed_x_handles", None))
|
||||||
|
excluded = _clean_x_handles(getattr(config, "excluded_x_handles", None))
|
||||||
|
if allowed and excluded:
|
||||||
|
raise ValueError("providers.xai_oauth.x_search cannot set both allowed_x_handles and excluded_x_handles")
|
||||||
|
|
||||||
|
tool: dict[str, Any] = {"type": "x_search"}
|
||||||
|
if allowed:
|
||||||
|
tool["allowed_x_handles"] = allowed
|
||||||
|
if excluded:
|
||||||
|
tool["excluded_x_handles"] = excluded
|
||||||
|
if getattr(config, "from_date", None):
|
||||||
|
tool["from_date"] = config.from_date
|
||||||
|
if getattr(config, "to_date", None):
|
||||||
|
tool["to_date"] = config.to_date
|
||||||
|
if getattr(config, "enable_image_understanding", False):
|
||||||
|
tool["enable_image_understanding"] = True
|
||||||
|
if getattr(config, "enable_video_understanding", False):
|
||||||
|
tool["enable_video_understanding"] = True
|
||||||
|
return tool
|
||||||
|
|
||||||
|
|
||||||
|
class _XaiHTTPError(RuntimeError):
|
||||||
|
def __init__(self, message: str, *, status_code: int, retry_after: float | None = None):
|
||||||
|
super().__init__(message)
|
||||||
|
self.status_code = status_code
|
||||||
|
self.retry_after = retry_after
|
||||||
|
|
||||||
|
|
||||||
|
async def _request_xai(
|
||||||
|
credential: XaiOAuthCredential,
|
||||||
|
body: dict[str, Any],
|
||||||
|
on_content_delta: Callable[[str], Awaitable[None]] | None = None,
|
||||||
|
on_tool_call_delta: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
|
||||||
|
) -> tuple[str, list[ToolCallRequest], str]:
|
||||||
|
url = credential.api_base.rstrip("/") + "/responses"
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Bearer {credential.access_token}",
|
||||||
|
"Accept": "text/event-stream",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"User-Agent": "nanobot (python)",
|
||||||
|
}
|
||||||
|
timeout = httpx.Timeout(120.0, connect=20.0)
|
||||||
|
async with httpx.AsyncClient(timeout=timeout, follow_redirects=True, trust_env=True) as client:
|
||||||
|
async with client.stream("POST", url, headers=headers, json=body) as response:
|
||||||
|
if response.status_code != 200:
|
||||||
|
raw = await response.aread()
|
||||||
|
retry_after = LLMProvider._extract_retry_after_from_headers(response.headers)
|
||||||
|
raise _XaiHTTPError(
|
||||||
|
_friendly_error(response.status_code, raw.decode("utf-8", "ignore")),
|
||||||
|
status_code=response.status_code,
|
||||||
|
retry_after=retry_after,
|
||||||
|
)
|
||||||
|
return await consume_sse(response, on_content_delta, on_tool_call_delta)
|
||||||
|
|
||||||
|
|
||||||
|
def _friendly_error(status_code: int, raw: str) -> str:
|
||||||
|
if status_code == 401:
|
||||||
|
return "xAI OAuth session expired or was revoked. Run: nanobot provider login xai-oauth"
|
||||||
|
if status_code == 403:
|
||||||
|
return (
|
||||||
|
"xAI accepted the OAuth token, but this account is not entitled for the requested "
|
||||||
|
"Grok API capability yet. Check the active Grok subscription and selected model."
|
||||||
|
)
|
||||||
|
if status_code == 429:
|
||||||
|
return "xAI Grok subscription quota or rate limit was reached. Please try again later."
|
||||||
|
return f"HTTP {status_code}: {raw[:500]}"
|
||||||
@@ -36,36 +36,15 @@ def configure_ssrf_whitelist(cidrs: list[str]) -> None:
|
|||||||
_allowed_networks = nets
|
_allowed_networks = nets
|
||||||
|
|
||||||
|
|
||||||
def _normalize_addr(
|
|
||||||
addr: ipaddress.IPv4Address | ipaddress.IPv6Address,
|
|
||||||
) -> ipaddress.IPv4Address | ipaddress.IPv6Address:
|
|
||||||
"""Normalize IPv6-mapped IPv4 addresses to their IPv4 form.
|
|
||||||
|
|
||||||
``::ffff:127.0.0.1`` is semantically identical to ``127.0.0.1`` but
|
|
||||||
Python's ipaddress treats it as an IPv6Address that matches neither
|
|
||||||
``127.0.0.0/8`` nor ``::1/128``. Converting it to IPv4 ensures
|
|
||||||
blocklist/allowlist checks work correctly.
|
|
||||||
"""
|
|
||||||
if isinstance(addr, ipaddress.IPv6Address) and addr.ipv4_mapped is not None:
|
|
||||||
return addr.ipv4_mapped
|
|
||||||
return addr
|
|
||||||
|
|
||||||
|
|
||||||
def _is_private(addr: ipaddress.IPv4Address | ipaddress.IPv6Address) -> bool:
|
def _is_private(addr: ipaddress.IPv4Address | ipaddress.IPv6Address) -> bool:
|
||||||
normalized = _normalize_addr(addr)
|
if _allowed_networks and any(addr in net for net in _allowed_networks):
|
||||||
if _allowed_networks and any(normalized in net for net in _allowed_networks):
|
|
||||||
return False
|
return False
|
||||||
return any(normalized in net for net in _BLOCKED_NETWORKS)
|
return any(addr in net for net in _BLOCKED_NETWORKS)
|
||||||
|
|
||||||
|
|
||||||
def validate_url_target(url: str, *, allow_loopback: bool = False) -> tuple[bool, str]:
|
def validate_url_target(url: str) -> tuple[bool, str]:
|
||||||
"""Validate a URL is safe to fetch: scheme, hostname, and resolved IPs.
|
"""Validate a URL is safe to fetch: scheme, hostname, and resolved IPs.
|
||||||
|
|
||||||
``allow_loopback`` is intentionally narrow: it only permits literal
|
|
||||||
loopback hosts (localhost, 127.0.0.0/8, ::1) when every resolved address is
|
|
||||||
loopback. It does not allow RFC1918, link-local, metadata, or public DNS
|
|
||||||
names that happen to resolve to loopback.
|
|
||||||
|
|
||||||
Returns (ok, error_message). When ok is True, error_message is empty.
|
Returns (ok, error_message). When ok is True, error_message is empty.
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
@@ -87,16 +66,11 @@ def validate_url_target(url: str, *, allow_loopback: bool = False) -> tuple[bool
|
|||||||
except socket.gaierror:
|
except socket.gaierror:
|
||||||
return False, f"Cannot resolve hostname: {hostname}"
|
return False, f"Cannot resolve hostname: {hostname}"
|
||||||
|
|
||||||
addrs: list[ipaddress.IPv4Address | ipaddress.IPv6Address] = []
|
|
||||||
for info in infos:
|
for info in infos:
|
||||||
try:
|
try:
|
||||||
addr = ipaddress.ip_address(info[4][0])
|
addr = ipaddress.ip_address(info[4][0])
|
||||||
except ValueError:
|
except ValueError:
|
||||||
continue
|
continue
|
||||||
addrs.append(addr)
|
|
||||||
if allow_loopback and _is_allowed_loopback_target(hostname, addrs):
|
|
||||||
return True, ""
|
|
||||||
for addr in addrs:
|
|
||||||
if _is_private(addr):
|
if _is_private(addr):
|
||||||
return False, f"Blocked: {hostname} resolves to private/internal address {addr}"
|
return False, f"Blocked: {hostname} resolves to private/internal address {addr}"
|
||||||
|
|
||||||
@@ -135,25 +109,11 @@ def validate_resolved_url(url: str) -> tuple[bool, str]:
|
|||||||
return True, ""
|
return True, ""
|
||||||
|
|
||||||
|
|
||||||
def contains_internal_url(command: str, *, allow_loopback: bool = False) -> bool:
|
def contains_internal_url(command: str) -> bool:
|
||||||
"""Return True if the command string contains a URL targeting an internal/private address."""
|
"""Return True if the command string contains a URL targeting an internal/private address."""
|
||||||
for m in _URL_RE.finditer(command):
|
for m in _URL_RE.finditer(command):
|
||||||
url = m.group(0)
|
url = m.group(0)
|
||||||
ok, _ = validate_url_target(url, allow_loopback=allow_loopback)
|
ok, _ = validate_url_target(url)
|
||||||
if not ok:
|
if not ok:
|
||||||
return True
|
return True
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
def _is_allowed_loopback_target(
|
|
||||||
hostname: str,
|
|
||||||
addrs: list[ipaddress.IPv4Address | ipaddress.IPv6Address],
|
|
||||||
) -> bool:
|
|
||||||
if not addrs or not all(_normalize_addr(addr).is_loopback for addr in addrs):
|
|
||||||
return False
|
|
||||||
normalized = hostname.rstrip(".").lower()
|
|
||||||
if normalized == "localhost":
|
|
||||||
return True
|
|
||||||
with suppress(ValueError):
|
|
||||||
return ipaddress.ip_address(hostname).is_loopback
|
|
||||||
return False
|
|
||||||
|
|||||||
@@ -1,430 +0,0 @@
|
|||||||
"""Workspace access scope and sandbox capability helpers."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
from contextvars import ContextVar, Token
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Literal
|
|
||||||
|
|
||||||
WorkspaceAccessMode = Literal["restricted", "full"]
|
|
||||||
WORKSPACE_SCOPE_METADATA_KEY = "workspace_scope"
|
|
||||||
_ACCESS_MODES = {"restricted", "full"}
|
|
||||||
|
|
||||||
_TRUE_VALUES = {"1", "true", "yes", "on", "enabled"}
|
|
||||||
_FALSE_VALUES = {"0", "false", "no", "off", "disabled", ""}
|
|
||||||
_PROVIDER_LABELS = {
|
|
||||||
"none": "None",
|
|
||||||
"unknown": "Unknown system sandbox",
|
|
||||||
"macos_app_sandbox": "macOS App Sandbox",
|
|
||||||
"bwrap": "Bubblewrap",
|
|
||||||
}
|
|
||||||
|
|
||||||
_CURRENT_WORKSPACE_SCOPE: ContextVar["WorkspaceScope | None"] = ContextVar(
|
|
||||||
"nanobot_workspace_scope",
|
|
||||||
default=None,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class WorkspaceScopeError(ValueError):
|
|
||||||
"""Raised when a requested WebUI workspace scope is invalid."""
|
|
||||||
|
|
||||||
status = 400
|
|
||||||
|
|
||||||
def __init__(self, message: str, *, status: int = 400) -> None:
|
|
||||||
super().__init__(message)
|
|
||||||
self.message = message
|
|
||||||
self.status = status
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class WorkspaceSandboxStatus:
|
|
||||||
"""Resolved workspace sandbox state for runtime display and tooling."""
|
|
||||||
|
|
||||||
restrict_to_workspace: bool
|
|
||||||
workspace_root: str
|
|
||||||
level: str
|
|
||||||
enforced: bool
|
|
||||||
provider: str
|
|
||||||
provider_label: str
|
|
||||||
summary: str
|
|
||||||
|
|
||||||
def as_dict(self) -> dict[str, object]:
|
|
||||||
return {
|
|
||||||
"restrict_to_workspace": self.restrict_to_workspace,
|
|
||||||
"workspace_root": self.workspace_root,
|
|
||||||
"level": self.level,
|
|
||||||
"enforced": self.enforced,
|
|
||||||
"provider": self.provider,
|
|
||||||
"provider_label": self.provider_label,
|
|
||||||
"summary": self.summary,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class WorkspaceScope:
|
|
||||||
"""Effective project root and access mode for one agent turn."""
|
|
||||||
|
|
||||||
project_path: Path
|
|
||||||
access_mode: WorkspaceAccessMode
|
|
||||||
restrict_to_workspace: bool
|
|
||||||
sandbox_status: WorkspaceSandboxStatus
|
|
||||||
source_channel: str | None = None
|
|
||||||
|
|
||||||
@property
|
|
||||||
def project_name(self) -> str:
|
|
||||||
return self.project_path.name or str(self.project_path)
|
|
||||||
|
|
||||||
def metadata(self) -> dict[str, str]:
|
|
||||||
return {
|
|
||||||
"project_path": str(self.project_path),
|
|
||||||
"access_mode": self.access_mode,
|
|
||||||
}
|
|
||||||
|
|
||||||
def payload(self) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
**self.metadata(),
|
|
||||||
"project_name": self.project_name,
|
|
||||||
"restrict_to_workspace": self.restrict_to_workspace,
|
|
||||||
"sandbox_status": self.sandbox_status.as_dict(),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class ToolWorkspace:
|
|
||||||
"""Workspace policy resolved for a tool call."""
|
|
||||||
|
|
||||||
project_path: Path | None
|
|
||||||
restrict_to_workspace: bool
|
|
||||||
scope: WorkspaceScope | None = None
|
|
||||||
|
|
||||||
@property
|
|
||||||
def allowed_root(self) -> Path | None:
|
|
||||||
if self.restrict_to_workspace and self.project_path is not None:
|
|
||||||
return self.project_path
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class WorkspaceScopeResolver:
|
|
||||||
"""Resolve the effective workspace scope at an agent turn boundary."""
|
|
||||||
|
|
||||||
default_workspace: str | Path
|
|
||||||
default_restrict_to_workspace: bool
|
|
||||||
scoped_channel: str = "websocket"
|
|
||||||
|
|
||||||
@property
|
|
||||||
def sandbox_status(self) -> WorkspaceSandboxStatus:
|
|
||||||
return self.default().sandbox_status
|
|
||||||
|
|
||||||
def default(self) -> WorkspaceScope:
|
|
||||||
return default_workspace_scope(
|
|
||||||
self.default_workspace,
|
|
||||||
self.default_restrict_to_workspace,
|
|
||||||
)
|
|
||||||
|
|
||||||
def for_message(
|
|
||||||
self,
|
|
||||||
msg: Any,
|
|
||||||
session_metadata: Any,
|
|
||||||
) -> WorkspaceScope:
|
|
||||||
return self.for_turn(
|
|
||||||
channel=getattr(msg, "channel", None),
|
|
||||||
message_metadata=getattr(msg, "metadata", None),
|
|
||||||
session_metadata=session_metadata,
|
|
||||||
)
|
|
||||||
|
|
||||||
def for_turn(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
channel: str | None,
|
|
||||||
message_metadata: Any,
|
|
||||||
session_metadata: Any,
|
|
||||||
) -> WorkspaceScope:
|
|
||||||
if channel != self.scoped_channel:
|
|
||||||
return self.default()
|
|
||||||
return resolve_effective_workspace_scope(
|
|
||||||
message_metadata=message_metadata,
|
|
||||||
session_metadata=session_metadata,
|
|
||||||
default_workspace=self.default_workspace,
|
|
||||||
default_restrict_to_workspace=self.default_restrict_to_workspace,
|
|
||||||
source_channel=channel,
|
|
||||||
)
|
|
||||||
|
|
||||||
def persist_message_scope(self, session: Any, msg: Any) -> None:
|
|
||||||
if getattr(msg, "channel", None) != self.scoped_channel:
|
|
||||||
return
|
|
||||||
metadata = getattr(msg, "metadata", None)
|
|
||||||
if not isinstance(metadata, dict):
|
|
||||||
return
|
|
||||||
raw = metadata.get(WORKSPACE_SCOPE_METADATA_KEY)
|
|
||||||
if isinstance(raw, dict):
|
|
||||||
session.metadata[WORKSPACE_SCOPE_METADATA_KEY] = dict(raw)
|
|
||||||
|
|
||||||
|
|
||||||
def workspace_sandbox_status(
|
|
||||||
*,
|
|
||||||
restrict_to_workspace: bool,
|
|
||||||
workspace: str | Path,
|
|
||||||
environ: dict[str, str] | None = None,
|
|
||||||
) -> WorkspaceSandboxStatus:
|
|
||||||
"""Return how workspace restriction is enforced in the current host."""
|
|
||||||
|
|
||||||
workspace_root = str(Path(workspace).expanduser().resolve(strict=False))
|
|
||||||
provider = _env_system_provider(environ)
|
|
||||||
if not restrict_to_workspace:
|
|
||||||
return WorkspaceSandboxStatus(
|
|
||||||
restrict_to_workspace=False,
|
|
||||||
workspace_root=workspace_root,
|
|
||||||
level="off",
|
|
||||||
enforced=False,
|
|
||||||
provider="none",
|
|
||||||
provider_label=_provider_label("none"),
|
|
||||||
summary="Workspace restriction is disabled.",
|
|
||||||
)
|
|
||||||
|
|
||||||
if provider:
|
|
||||||
label = _provider_label(provider)
|
|
||||||
return WorkspaceSandboxStatus(
|
|
||||||
restrict_to_workspace=True,
|
|
||||||
workspace_root=workspace_root,
|
|
||||||
level="system",
|
|
||||||
enforced=True,
|
|
||||||
provider=provider,
|
|
||||||
provider_label=label,
|
|
||||||
summary=f"Workspace restriction is system-enforced by {label}.",
|
|
||||||
)
|
|
||||||
|
|
||||||
return WorkspaceSandboxStatus(
|
|
||||||
restrict_to_workspace=True,
|
|
||||||
workspace_root=workspace_root,
|
|
||||||
level="application",
|
|
||||||
enforced=False,
|
|
||||||
provider="none",
|
|
||||||
provider_label=_provider_label("none"),
|
|
||||||
summary="Workspace restriction uses nanobot application-level guards.",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def default_access_mode(restrict_to_workspace: bool) -> WorkspaceAccessMode:
|
|
||||||
return "restricted" if restrict_to_workspace else "full"
|
|
||||||
|
|
||||||
|
|
||||||
def build_workspace_scope(
|
|
||||||
project_path: str | Path,
|
|
||||||
access_mode: str,
|
|
||||||
*,
|
|
||||||
source_channel: str | None = None,
|
|
||||||
) -> WorkspaceScope:
|
|
||||||
mode = _normalize_access_mode(access_mode)
|
|
||||||
root = Path(project_path).expanduser().resolve(strict=False)
|
|
||||||
restrict = mode == "restricted"
|
|
||||||
return WorkspaceScope(
|
|
||||||
project_path=root,
|
|
||||||
access_mode=mode,
|
|
||||||
restrict_to_workspace=restrict,
|
|
||||||
sandbox_status=workspace_sandbox_status(
|
|
||||||
restrict_to_workspace=restrict,
|
|
||||||
workspace=root,
|
|
||||||
),
|
|
||||||
source_channel=source_channel,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def default_workspace_scope(
|
|
||||||
workspace: str | Path,
|
|
||||||
restrict_to_workspace: bool,
|
|
||||||
*,
|
|
||||||
source_channel: str | None = None,
|
|
||||||
) -> WorkspaceScope:
|
|
||||||
return build_workspace_scope(
|
|
||||||
workspace,
|
|
||||||
default_access_mode(restrict_to_workspace),
|
|
||||||
source_channel=source_channel,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def validate_workspace_scope_payload(
|
|
||||||
raw: Any,
|
|
||||||
*,
|
|
||||||
default_workspace: str | Path,
|
|
||||||
default_restrict_to_workspace: bool,
|
|
||||||
source_channel: str | None = None,
|
|
||||||
) -> WorkspaceScope:
|
|
||||||
"""Validate a client-requested workspace scope."""
|
|
||||||
if raw is None:
|
|
||||||
return default_workspace_scope(
|
|
||||||
default_workspace,
|
|
||||||
default_restrict_to_workspace,
|
|
||||||
source_channel=source_channel,
|
|
||||||
)
|
|
||||||
if not isinstance(raw, dict):
|
|
||||||
raise WorkspaceScopeError("workspace_scope must be an object")
|
|
||||||
|
|
||||||
raw_path = raw.get("project_path") or raw.get("path")
|
|
||||||
if raw_path is None or raw_path == "":
|
|
||||||
raw_path = str(Path(default_workspace).expanduser().resolve(strict=False))
|
|
||||||
if not isinstance(raw_path, str):
|
|
||||||
raise WorkspaceScopeError("project_path must be a string")
|
|
||||||
if "\0" in raw_path:
|
|
||||||
raise WorkspaceScopeError("project_path contains invalid characters")
|
|
||||||
|
|
||||||
project = Path(raw_path).expanduser()
|
|
||||||
if not project.is_absolute():
|
|
||||||
raise WorkspaceScopeError("project_path must be absolute")
|
|
||||||
project = project.resolve(strict=False)
|
|
||||||
if not project.is_dir():
|
|
||||||
raise WorkspaceScopeError("project_path must be an existing directory")
|
|
||||||
|
|
||||||
raw_mode = raw.get("access_mode")
|
|
||||||
if raw_mode is None:
|
|
||||||
raw_mode = default_access_mode(default_restrict_to_workspace)
|
|
||||||
if not isinstance(raw_mode, str):
|
|
||||||
raise WorkspaceScopeError("access_mode must be a string")
|
|
||||||
return build_workspace_scope(project, raw_mode, source_channel=source_channel)
|
|
||||||
|
|
||||||
|
|
||||||
def workspace_scope_from_metadata(
|
|
||||||
metadata: Any,
|
|
||||||
*,
|
|
||||||
default_workspace: str | Path,
|
|
||||||
default_restrict_to_workspace: bool,
|
|
||||||
source_channel: str | None = None,
|
|
||||||
) -> WorkspaceScope:
|
|
||||||
"""Resolve persisted metadata, falling back safely for old or stale sessions."""
|
|
||||||
if not isinstance(metadata, dict):
|
|
||||||
return default_workspace_scope(
|
|
||||||
default_workspace,
|
|
||||||
default_restrict_to_workspace,
|
|
||||||
source_channel=source_channel,
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
return validate_workspace_scope_payload(
|
|
||||||
metadata.get(WORKSPACE_SCOPE_METADATA_KEY),
|
|
||||||
default_workspace=default_workspace,
|
|
||||||
default_restrict_to_workspace=default_restrict_to_workspace,
|
|
||||||
source_channel=source_channel,
|
|
||||||
)
|
|
||||||
except WorkspaceScopeError:
|
|
||||||
return default_workspace_scope(
|
|
||||||
default_workspace,
|
|
||||||
default_restrict_to_workspace,
|
|
||||||
source_channel=source_channel,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_effective_workspace_scope(
|
|
||||||
*,
|
|
||||||
message_metadata: Any,
|
|
||||||
session_metadata: Any,
|
|
||||||
default_workspace: str | Path,
|
|
||||||
default_restrict_to_workspace: bool,
|
|
||||||
source_channel: str | None = None,
|
|
||||||
) -> WorkspaceScope:
|
|
||||||
if isinstance(message_metadata, dict) and WORKSPACE_SCOPE_METADATA_KEY in message_metadata:
|
|
||||||
return workspace_scope_from_metadata(
|
|
||||||
message_metadata,
|
|
||||||
default_workspace=default_workspace,
|
|
||||||
default_restrict_to_workspace=default_restrict_to_workspace,
|
|
||||||
source_channel=source_channel,
|
|
||||||
)
|
|
||||||
return workspace_scope_from_metadata(
|
|
||||||
session_metadata,
|
|
||||||
default_workspace=default_workspace,
|
|
||||||
default_restrict_to_workspace=default_restrict_to_workspace,
|
|
||||||
source_channel=source_channel,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def bind_workspace_scope(scope: WorkspaceScope) -> Token[WorkspaceScope | None]:
|
|
||||||
return _CURRENT_WORKSPACE_SCOPE.set(scope)
|
|
||||||
|
|
||||||
|
|
||||||
def reset_workspace_scope(token: Token[WorkspaceScope | None]) -> None:
|
|
||||||
_CURRENT_WORKSPACE_SCOPE.reset(token)
|
|
||||||
|
|
||||||
|
|
||||||
def current_workspace_scope() -> WorkspaceScope | None:
|
|
||||||
return _CURRENT_WORKSPACE_SCOPE.get()
|
|
||||||
|
|
||||||
|
|
||||||
def current_tool_workspace(
|
|
||||||
default_workspace: str | Path | None,
|
|
||||||
*,
|
|
||||||
restrict_to_workspace: bool = False,
|
|
||||||
sandbox_restricts_workspace: bool = False,
|
|
||||||
) -> ToolWorkspace:
|
|
||||||
"""Return the workspace/access policy for the current tool call."""
|
|
||||||
|
|
||||||
scope = current_workspace_scope()
|
|
||||||
project_path = (
|
|
||||||
scope.project_path
|
|
||||||
if scope is not None
|
|
||||||
else Path(default_workspace).expanduser() if default_workspace is not None else None
|
|
||||||
)
|
|
||||||
restrict = (
|
|
||||||
scope.restrict_to_workspace
|
|
||||||
if scope is not None
|
|
||||||
else bool(restrict_to_workspace)
|
|
||||||
) or sandbox_restricts_workspace
|
|
||||||
return ToolWorkspace(
|
|
||||||
project_path=project_path,
|
|
||||||
restrict_to_workspace=restrict,
|
|
||||||
scope=scope,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def current_scope_allows_loopback(*, enabled: bool) -> bool:
|
|
||||||
"""Return True when the current WebUI Full Access turn may touch loopback URLs."""
|
|
||||||
|
|
||||||
scope = current_workspace_scope()
|
|
||||||
return bool(
|
|
||||||
enabled
|
|
||||||
and scope is not None
|
|
||||||
and scope.source_channel == "websocket"
|
|
||||||
and scope.access_mode == "full"
|
|
||||||
and not scope.restrict_to_workspace
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _env_system_provider(environ: dict[str, str] | None = None) -> str | None:
|
|
||||||
env = environ if environ is not None else os.environ
|
|
||||||
explicit_provider = env.get("NANOBOT_WORKSPACE_SANDBOX_PROVIDER")
|
|
||||||
enforced = env.get("NANOBOT_WORKSPACE_SANDBOX_ENFORCED")
|
|
||||||
compatibility = env.get("NANOBOT_SANDBOX_ENFORCED")
|
|
||||||
|
|
||||||
marker = enforced if enforced is not None else compatibility
|
|
||||||
if marker is None:
|
|
||||||
return None
|
|
||||||
|
|
||||||
normalized_marker = marker.strip().lower()
|
|
||||||
if normalized_marker in _FALSE_VALUES:
|
|
||||||
return None
|
|
||||||
if normalized_marker in _TRUE_VALUES:
|
|
||||||
return _normalize_provider(explicit_provider)
|
|
||||||
return _normalize_provider(marker)
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_provider(value: str | None) -> str:
|
|
||||||
if not value:
|
|
||||||
return "unknown"
|
|
||||||
normalized = value.strip().lower().replace("-", "_").replace(" ", "_")
|
|
||||||
return normalized or "unknown"
|
|
||||||
|
|
||||||
|
|
||||||
def _provider_label(provider: str) -> str:
|
|
||||||
if provider in _PROVIDER_LABELS:
|
|
||||||
return _PROVIDER_LABELS[provider]
|
|
||||||
return provider.replace("_", " ").title()
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_access_mode(value: str) -> WorkspaceAccessMode:
|
|
||||||
mode = value.strip().lower().replace("_", "-")
|
|
||||||
if mode == "restrict":
|
|
||||||
mode = "restricted"
|
|
||||||
if mode == "full-access":
|
|
||||||
mode = "full"
|
|
||||||
if mode not in _ACCESS_MODES:
|
|
||||||
raise WorkspaceScopeError("access_mode must be restricted or full")
|
|
||||||
return mode # type: ignore[return-value]
|
|
||||||
@@ -1,85 +0,0 @@
|
|||||||
"""Workspace path boundary helpers.
|
|
||||||
|
|
||||||
These helpers are application-level guards. They make path decisions
|
|
||||||
consistent across tools, but they are not a replacement for an OS sandbox.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Iterable
|
|
||||||
|
|
||||||
WORKSPACE_BOUNDARY_NOTE = (
|
|
||||||
" (this is a hard policy boundary, not a transient failure; "
|
|
||||||
"do not retry with shell tricks or alternative tools, and ask "
|
|
||||||
"the user how to proceed if the resource is genuinely required)"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class WorkspaceBoundaryError(PermissionError):
|
|
||||||
"""Raised when a requested path escapes an allowed workspace boundary."""
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_path(path: str | Path, workspace: str | Path | None = None, *, strict: bool = False) -> Path:
|
|
||||||
"""Resolve *path*, interpreting relative paths against *workspace* when set."""
|
|
||||||
candidate = Path(path).expanduser()
|
|
||||||
if not candidate.is_absolute() and workspace is not None:
|
|
||||||
candidate = Path(workspace).expanduser() / candidate
|
|
||||||
return candidate.resolve(strict=strict)
|
|
||||||
|
|
||||||
|
|
||||||
def is_path_within(path: str | Path, root: str | Path) -> bool:
|
|
||||||
"""Return True when *path* resolves to *root* or a descendant of *root*."""
|
|
||||||
try:
|
|
||||||
resolved_path = Path(path).expanduser().resolve(strict=False)
|
|
||||||
resolved_root = Path(root).expanduser().resolve(strict=False)
|
|
||||||
resolved_path.relative_to(resolved_root)
|
|
||||||
return True
|
|
||||||
except (OSError, RuntimeError, TypeError, ValueError):
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def is_path_allowed(path: str | Path, roots: Iterable[str | Path]) -> bool:
|
|
||||||
"""Return True when *path* is inside any allowed root."""
|
|
||||||
return any(is_path_within(path, root) for root in roots)
|
|
||||||
|
|
||||||
|
|
||||||
def require_path_within(
|
|
||||||
path: str | Path,
|
|
||||||
root: str | Path,
|
|
||||||
*,
|
|
||||||
message: str | None = None,
|
|
||||||
) -> Path:
|
|
||||||
"""Resolve *path* and require it to be inside *root*."""
|
|
||||||
resolved = Path(path).expanduser().resolve(strict=False)
|
|
||||||
if not is_path_within(resolved, root):
|
|
||||||
raise WorkspaceBoundaryError(
|
|
||||||
message
|
|
||||||
or f"Path {path} is outside allowed directory {Path(root).expanduser()}"
|
|
||||||
+ WORKSPACE_BOUNDARY_NOTE
|
|
||||||
)
|
|
||||||
return resolved
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_allowed_path(
|
|
||||||
path: str | Path,
|
|
||||||
*,
|
|
||||||
workspace: str | Path | None = None,
|
|
||||||
allowed_root: str | Path | None = None,
|
|
||||||
extra_allowed_roots: Iterable[str | Path] | None = None,
|
|
||||||
strict: bool = False,
|
|
||||||
) -> Path:
|
|
||||||
"""Resolve a path and enforce containment in allowed roots when configured."""
|
|
||||||
resolved = resolve_path(path, workspace, strict=False)
|
|
||||||
if allowed_root is None:
|
|
||||||
return resolve_path(path, workspace, strict=strict) if strict else resolved
|
|
||||||
|
|
||||||
roots = [allowed_root, *(extra_allowed_roots or [])]
|
|
||||||
if not is_path_allowed(resolved, roots):
|
|
||||||
raise WorkspaceBoundaryError(
|
|
||||||
f"Path {path} is outside allowed directory {Path(allowed_root).expanduser()}"
|
|
||||||
+ WORKSPACE_BOUNDARY_NOTE
|
|
||||||
)
|
|
||||||
if strict:
|
|
||||||
return resolve_path(path, workspace, strict=True)
|
|
||||||
return resolved
|
|
||||||
@@ -19,7 +19,6 @@ from nanobot.utils.helpers import (
|
|||||||
find_legal_message_start,
|
find_legal_message_start,
|
||||||
image_placeholder_text,
|
image_placeholder_text,
|
||||||
safe_filename,
|
safe_filename,
|
||||||
strip_think,
|
|
||||||
)
|
)
|
||||||
from nanobot.utils.subagent_channel_display import scrub_subagent_announce_body
|
from nanobot.utils.subagent_channel_display import scrub_subagent_announce_body
|
||||||
|
|
||||||
@@ -28,8 +27,6 @@ _MESSAGE_TIME_PREFIX_RE = re.compile(r"^\[Message Time: [^\]]+\]\n?")
|
|||||||
_LOCAL_IMAGE_BREADCRUMB_RE = re.compile(r"^\[image: (?:/|~)[^\]]+\]\s*$")
|
_LOCAL_IMAGE_BREADCRUMB_RE = re.compile(r"^\[image: (?:/|~)[^\]]+\]\s*$")
|
||||||
_TOOL_CALL_ECHO_RE = re.compile(r'^\s*(?:generate_image|message)\([^)]*\)\s*$')
|
_TOOL_CALL_ECHO_RE = re.compile(r'^\s*(?:generate_image|message)\([^)]*\)\s*$')
|
||||||
_SESSION_PREVIEW_MAX_CHARS = 120
|
_SESSION_PREVIEW_MAX_CHARS = 120
|
||||||
_SESSION_LIST_PREVIEW_MAX_RECORDS = 200
|
|
||||||
_SESSION_LIST_PREVIEW_MAX_CHARS = 1_000_000
|
|
||||||
|
|
||||||
|
|
||||||
def _sanitize_assistant_replay_text(content: str) -> str:
|
def _sanitize_assistant_replay_text(content: str) -> str:
|
||||||
@@ -77,17 +74,6 @@ def _message_preview_text(message: dict[str, Any]) -> str:
|
|||||||
return _text_preview(content)
|
return _text_preview(content)
|
||||||
|
|
||||||
|
|
||||||
def _metadata_title(metadata: Any) -> str:
|
|
||||||
if not isinstance(metadata, dict):
|
|
||||||
return ""
|
|
||||||
title = metadata.get("title")
|
|
||||||
if not isinstance(title, str):
|
|
||||||
return ""
|
|
||||||
if metadata.get("title_user_edited") is True:
|
|
||||||
return title
|
|
||||||
return strip_think(title)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class Session:
|
class Session:
|
||||||
"""A conversation session."""
|
"""A conversation session."""
|
||||||
@@ -179,45 +165,6 @@ class Session:
|
|||||||
image_placeholder_text(p) for p in media if isinstance(p, str) and p
|
image_placeholder_text(p) for p in media if isinstance(p, str) and p
|
||||||
)
|
)
|
||||||
content = f"{content}\n{breadcrumbs}" if content else breadcrumbs
|
content = f"{content}\n{breadcrumbs}" if content else breadcrumbs
|
||||||
cli_apps = message.get("cli_apps")
|
|
||||||
if role == "user" and isinstance(cli_apps, list) and cli_apps and isinstance(content, str):
|
|
||||||
cli_lines: list[str] = []
|
|
||||||
for item in cli_apps[:8]:
|
|
||||||
if not isinstance(item, dict):
|
|
||||||
continue
|
|
||||||
name = str(item.get("name") or "").strip().lower()
|
|
||||||
if not name:
|
|
||||||
continue
|
|
||||||
entry = str(item.get("entry_point") or "unknown").strip() or "unknown"
|
|
||||||
cli_lines.append(
|
|
||||||
f"[CLI App Attachment: @{name}; tool=run_cli_app; entry_point={entry}; "
|
|
||||||
f"skill=skills/cli-app-{name}/SKILL.md]"
|
|
||||||
)
|
|
||||||
if cli_lines:
|
|
||||||
breadcrumbs = "\n".join(cli_lines)
|
|
||||||
content = f"{content}\n{breadcrumbs}" if content else breadcrumbs
|
|
||||||
mcp_presets = message.get("mcp_presets")
|
|
||||||
if (
|
|
||||||
role == "user"
|
|
||||||
and isinstance(mcp_presets, list)
|
|
||||||
and mcp_presets
|
|
||||||
and isinstance(content, str)
|
|
||||||
):
|
|
||||||
mcp_lines: list[str] = []
|
|
||||||
for item in mcp_presets[:8]:
|
|
||||||
if not isinstance(item, dict):
|
|
||||||
continue
|
|
||||||
name = str(item.get("name") or "").strip().lower()
|
|
||||||
if not name:
|
|
||||||
continue
|
|
||||||
transport = str(item.get("transport") or "mcp").strip() or "mcp"
|
|
||||||
mcp_lines.append(
|
|
||||||
f"[MCP Preset Attachment: @{name}; tool_prefix=mcp_{name}_; "
|
|
||||||
f"transport={transport}]"
|
|
||||||
)
|
|
||||||
if mcp_lines:
|
|
||||||
breadcrumbs = "\n".join(mcp_lines)
|
|
||||||
content = f"{content}\n{breadcrumbs}" if content else breadcrumbs
|
|
||||||
if include_timestamps:
|
if include_timestamps:
|
||||||
content = self._annotate_message_time(message, content)
|
content = self._annotate_message_time(message, content)
|
||||||
if role == "assistant" and isinstance(content, str) and not content.strip():
|
if role == "assistant" and isinstance(content, str) and not content.strip():
|
||||||
@@ -654,21 +601,12 @@ class SessionManager:
|
|||||||
if data.get("_type") == "metadata":
|
if data.get("_type") == "metadata":
|
||||||
key = data.get("key") or path.stem.replace("_", ":", 1)
|
key = data.get("key") or path.stem.replace("_", ":", 1)
|
||||||
metadata = data.get("metadata", {})
|
metadata = data.get("metadata", {})
|
||||||
title = _metadata_title(metadata)
|
title = metadata.get("title") if isinstance(metadata, dict) else None
|
||||||
preview = ""
|
preview = ""
|
||||||
fallback_preview = ""
|
fallback_preview = ""
|
||||||
scanned_records = 0
|
|
||||||
scanned_chars = 0
|
|
||||||
for line in f:
|
for line in f:
|
||||||
if not line.strip():
|
if not line.strip():
|
||||||
continue
|
continue
|
||||||
scanned_records += 1
|
|
||||||
scanned_chars += len(line)
|
|
||||||
if (
|
|
||||||
scanned_records > _SESSION_LIST_PREVIEW_MAX_RECORDS
|
|
||||||
or scanned_chars > _SESSION_LIST_PREVIEW_MAX_CHARS
|
|
||||||
):
|
|
||||||
break
|
|
||||||
item = json.loads(line)
|
item = json.loads(line)
|
||||||
if item.get("_type") == "metadata":
|
if item.get("_type") == "metadata":
|
||||||
continue
|
continue
|
||||||
@@ -685,7 +623,7 @@ class SessionManager:
|
|||||||
"key": key,
|
"key": key,
|
||||||
"created_at": data.get("created_at"),
|
"created_at": data.get("created_at"),
|
||||||
"updated_at": data.get("updated_at"),
|
"updated_at": data.get("updated_at"),
|
||||||
"title": title,
|
"title": title if isinstance(title, str) else "",
|
||||||
"preview": preview,
|
"preview": preview,
|
||||||
"path": str(path)
|
"path": str(path)
|
||||||
})
|
})
|
||||||
@@ -696,7 +634,11 @@ class SessionManager:
|
|||||||
"key": repaired.key,
|
"key": repaired.key,
|
||||||
"created_at": repaired.created_at.isoformat(),
|
"created_at": repaired.created_at.isoformat(),
|
||||||
"updated_at": repaired.updated_at.isoformat(),
|
"updated_at": repaired.updated_at.isoformat(),
|
||||||
"title": _metadata_title(repaired.metadata),
|
"title": (
|
||||||
|
repaired.metadata.get("title")
|
||||||
|
if isinstance(repaired.metadata.get("title"), str)
|
||||||
|
else ""
|
||||||
|
),
|
||||||
"preview": next(
|
"preview": next(
|
||||||
(
|
(
|
||||||
text
|
text
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ from nanobot.bus.queue import MessageBus
|
|||||||
from nanobot.providers.base import LLMProvider
|
from nanobot.providers.base import LLMProvider
|
||||||
from nanobot.session.goal_state import goal_state_ws_blob
|
from nanobot.session.goal_state import goal_state_ws_blob
|
||||||
from nanobot.session.manager import Session, SessionManager
|
from nanobot.session.manager import Session, SessionManager
|
||||||
from nanobot.utils.helpers import strip_think, truncate_text
|
from nanobot.utils.helpers import truncate_text
|
||||||
from nanobot.utils.llm_runtime import LLMRuntime
|
from nanobot.utils.llm_runtime import LLMRuntime
|
||||||
|
|
||||||
WEBUI_SESSION_METADATA_KEY = "webui"
|
WEBUI_SESSION_METADATA_KEY = "webui"
|
||||||
@@ -48,7 +48,6 @@ def clean_generated_title(raw: str | None) -> str:
|
|||||||
return ""
|
return ""
|
||||||
text = re.sub(r"^\s*(title|标题)\s*[::]\s*", "", text, flags=re.IGNORECASE)
|
text = re.sub(r"^\s*(title|标题)\s*[::]\s*", "", text, flags=re.IGNORECASE)
|
||||||
text = text.strip().strip("\"'`“”‘’")
|
text = text.strip().strip("\"'`“”‘’")
|
||||||
text = strip_think(text)
|
|
||||||
text = re.sub(r"\s+", " ", text).strip()
|
text = re.sub(r"\s+", " ", text).strip()
|
||||||
text = text.rstrip("。.!!??,,;;:")
|
text = text.rstrip("。.!!??,,;;:")
|
||||||
if len(text) > TITLE_MAX_CHARS:
|
if len(text) > TITLE_MAX_CHARS:
|
||||||
@@ -66,9 +65,6 @@ def _title_inputs(session: Session) -> tuple[str, str]:
|
|||||||
content = message.get("content")
|
content = message.get("content")
|
||||||
if not isinstance(content, str) or not content.strip():
|
if not isinstance(content, str) or not content.strip():
|
||||||
continue
|
continue
|
||||||
content = strip_think(content)
|
|
||||||
if not content:
|
|
||||||
continue
|
|
||||||
if role == "user" and not user_text:
|
if role == "user" and not user_text:
|
||||||
user_text = content.strip()
|
user_text = content.strip()
|
||||||
elif role == "assistant" and not assistant_text:
|
elif role == "assistant" and not assistant_text:
|
||||||
@@ -93,13 +89,7 @@ async def maybe_generate_webui_title(
|
|||||||
return False
|
return False
|
||||||
current_title = session.metadata.get(WEBUI_TITLE_METADATA_KEY)
|
current_title = session.metadata.get(WEBUI_TITLE_METADATA_KEY)
|
||||||
if isinstance(current_title, str) and current_title.strip():
|
if isinstance(current_title, str) and current_title.strip():
|
||||||
cleaned_current_title = clean_generated_title(current_title)
|
|
||||||
if cleaned_current_title:
|
|
||||||
if cleaned_current_title != current_title:
|
|
||||||
session.metadata[WEBUI_TITLE_METADATA_KEY] = cleaned_current_title
|
|
||||||
sessions.save(session)
|
|
||||||
return False
|
return False
|
||||||
session.metadata.pop(WEBUI_TITLE_METADATA_KEY, None)
|
|
||||||
|
|
||||||
user_text, assistant_text = _title_inputs(session)
|
user_text, assistant_text = _title_inputs(session)
|
||||||
if not user_text:
|
if not user_text:
|
||||||
|
|||||||
@@ -1,9 +1,5 @@
|
|||||||
# Agent Instructions
|
# Agent Instructions
|
||||||
|
|
||||||
## Workspace Guidance
|
|
||||||
|
|
||||||
Use this file for project-specific preferences, recurring workflow conventions, and instructions you want the agent to remember for this workspace. Keep durable facts about the user in `USER.md`, personality/style guidance in `SOUL.md`, and long-term memory in `memory/MEMORY.md`.
|
|
||||||
|
|
||||||
## Scheduled Reminders
|
## Scheduled Reminders
|
||||||
|
|
||||||
Before scheduling reminders, check available skills and follow skill guidance first.
|
Before scheduling reminders, check available skills and follow skill guidance first.
|
||||||
@@ -14,10 +10,10 @@ Get USER_ID and CHANNEL from the current session (e.g., `8281248569` and `telegr
|
|||||||
|
|
||||||
## Heartbeat Tasks
|
## Heartbeat Tasks
|
||||||
|
|
||||||
`HEARTBEAT.md` is checked periodically when registered as a cron job. Use the built-in `cron` tool to schedule it (e.g. `cron add --name heartbeat --schedule "every 30m" --message "Check HEARTBEAT.md"`).
|
`HEARTBEAT.md` is checked on the configured heartbeat interval. Use file tools to manage periodic tasks:
|
||||||
|
|
||||||
- Use `apply_patch` for normal task-list updates, especially when adding, removing, or changing multiple lines.
|
- **Add**: `edit_file` to append new tasks
|
||||||
- Use `edit_file` only for small exact replacements copied from the current `HEARTBEAT.md`.
|
- **Remove**: `edit_file` to delete completed tasks
|
||||||
- Use `write_file` for first creation or intentional full-file rewrites.
|
- **Rewrite**: `write_file` to replace all tasks
|
||||||
|
|
||||||
When the user asks for a recurring/periodic task, update `HEARTBEAT.md` and register it via `cron` instead of creating a one-time reminder.
|
When the user asks for a recurring/periodic task, update `HEARTBEAT.md` instead of creating a one-time cron reminder.
|
||||||
|
|||||||
@@ -1,9 +1,9 @@
|
|||||||
# Heartbeat Tasks
|
# Heartbeat Tasks
|
||||||
|
|
||||||
This file is checked periodically by your nanobot agent.
|
This file is checked every 30 minutes by your nanobot agent.
|
||||||
Register it as a cron job (e.g. `cron add --name heartbeat --schedule "every 30m" --message "Check HEARTBEAT.md"`) to get the same behavior as the legacy heartbeat service.
|
Add tasks below that you want the agent to work on periodically.
|
||||||
|
|
||||||
If this file has no tasks (only headers and comments), the agent will skip it.
|
If this file has no tasks (only headers and comments), the agent will skip the heartbeat.
|
||||||
|
|
||||||
## Active Tasks
|
## Active Tasks
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# Tool Usage Notes
|
||||||
|
|
||||||
|
Tool signatures are provided automatically via function calling.
|
||||||
|
This file documents non-obvious constraints and usage patterns.
|
||||||
|
|
||||||
|
## exec — Safety Limits
|
||||||
|
|
||||||
|
- Commands have a configurable timeout (default 60s)
|
||||||
|
- Dangerous commands are blocked (rm -rf, format, dd, shutdown, etc.)
|
||||||
|
- Output is truncated at 10,000 characters
|
||||||
|
- `restrictToWorkspace` config can limit file access to the workspace
|
||||||
|
|
||||||
|
## grep — Content Search
|
||||||
|
|
||||||
|
- Use `grep` to search file contents inside the workspace
|
||||||
|
- Default behavior returns only matching file paths (`output_mode="files_with_matches"`)
|
||||||
|
- Supports optional `glob` filtering (e.g. `glob="*.py"`) plus `context_before` / `context_after`
|
||||||
|
- Supports `type="py"`, `type="ts"`, `type="md"` and similar shorthand filters
|
||||||
|
- Use `fixed_strings=true` for literal keywords containing regex characters
|
||||||
|
- Use `output_mode="files_with_matches"` to get only matching file paths
|
||||||
|
- Use `output_mode="count"` to size a search before reading full matches
|
||||||
|
- Use `head_limit` and `offset` to page across results
|
||||||
|
- Prefer this over `exec` for code and history searches
|
||||||
|
- Binary or oversized files may be skipped to keep results readable
|
||||||
|
|
||||||
|
## cron — Scheduled Reminders
|
||||||
|
|
||||||
|
- Please refer to cron skill for usage.
|
||||||
@@ -1,67 +0,0 @@
|
|||||||
# Tool Usage Notes
|
|
||||||
|
|
||||||
Tool signatures are provided automatically via function calling. This section
|
|
||||||
documents the general tool contract and non-obvious usage patterns.
|
|
||||||
|
|
||||||
## General Tool Contract
|
|
||||||
|
|
||||||
- Use the narrowest structured tool that directly matches the task.
|
|
||||||
- Use read-only discovery before writes when state is uncertain.
|
|
||||||
- Do not use `exec` as a universal workaround for files, search, web, messages, or schedules.
|
|
||||||
- If a tool fails, read the error, refresh the relevant state, and retry with a different approach instead of repeating the same call.
|
|
||||||
- After meaningful changes, verify with the smallest reliable check: re-read changed state, run targeted tests, or inspect command output.
|
|
||||||
- Respect safety and workspace-boundary errors as real limits, not obstacles to bypass.
|
|
||||||
|
|
||||||
## Discovery and Reading
|
|
||||||
|
|
||||||
- Use `find_files` or `list_dir` to locate workspace paths before `read_file` when a path is uncertain.
|
|
||||||
- Use `grep` for content search inside the workspace; prefer it over shell grep for ordinary searches.
|
|
||||||
- `grep` defaults to `output_mode="files_with_matches"`; use `output_mode="content"` for matching lines with context.
|
|
||||||
- Use `fixed_strings=true` for literal keywords containing regex characters.
|
|
||||||
- Use `output_mode="count"` to size a broad search before reading full matches.
|
|
||||||
- Use `head_limit` and `offset` to page across large result sets.
|
|
||||||
- Binary or oversized files may be skipped to keep results readable.
|
|
||||||
|
|
||||||
## File and Coding Workflows
|
|
||||||
|
|
||||||
- For code or config changes, the default loop is: locate (`find_files`/`grep`), inspect (`read_file`), edit (`apply_patch`), then verify (`exec` or re-read).
|
|
||||||
- Use `apply_patch` as the default code editing tool, especially for multi-file changes, structural edits, generated code, moves, adds, or deletes.
|
|
||||||
- Use `apply_patch dry_run=true` when the patch is uncertain and you want validation plus a change summary before writing.
|
|
||||||
- Use `edit_file` only for small exact replacements in one file, with `old_text` copied from `read_file`; add `occurrence`, `line_hint`, or `expected_replacements` when ambiguity matters.
|
|
||||||
- Use `write_file` for new files or intentional full-file rewrites, not routine partial edits.
|
|
||||||
- If `apply_patch` or `edit_file` fails, re-read with `force=true`, narrow the context, and try a smaller patch rather than switching to shell `sed` or `echo`.
|
|
||||||
|
|
||||||
## Process Execution
|
|
||||||
|
|
||||||
- Use `exec` for tests, builds, package commands, git commands, and other process execution.
|
|
||||||
- Prefer dedicated file/search tools over `cat`, shell `find`, shell `grep`, `sed`, or `echo` for ordinary workspace inspection and edits.
|
|
||||||
- Use non-interactive flags such as `-y` or `--yes` when available.
|
|
||||||
- Commands have a configurable timeout (default 60s), dangerous commands are blocked, and output is truncated.
|
|
||||||
- For long-running or interactive commands, pass `yield_time_ms`; if the process keeps running, continue with `write_stdin`.
|
|
||||||
- Use `write_stdin` to poll, provide stdin, close stdin, wait for expected output with `wait_for`, or terminate an existing exec session.
|
|
||||||
- Use `list_exec_sessions` to recover active session IDs after context shifts.
|
|
||||||
|
|
||||||
## CLI App Attachments
|
|
||||||
|
|
||||||
- When Runtime Context lists a `CLI App Attachment` or `CLI App Mention`, treat the `@name` as an app capability the user intentionally attached to the current turn.
|
|
||||||
- If the task may need app-specific behavior, read the listed skill first, then call `run_cli_app` with that `name`.
|
|
||||||
- Do not run an attached CLI app through shell or generic process tools unless the user explicitly asks for that lower-level path.
|
|
||||||
- If the app CLI is missing, lacks local desktop/app/API prerequisites, or cannot complete the requested action, explain that concrete blocker and what was attempted.
|
|
||||||
|
|
||||||
## Web and External Information
|
|
||||||
|
|
||||||
- Use web tools when the user asks for current information, a specific URL, or information likely to have changed.
|
|
||||||
- Use `web_search` to find sources and `web_fetch` for a specific page or result that needs closer reading.
|
|
||||||
- Do not invent freshness-sensitive facts when tools can verify them.
|
|
||||||
|
|
||||||
## Messaging and Media
|
|
||||||
|
|
||||||
- Use `message` to send content or local media to the user/channel.
|
|
||||||
- `read_file` only reads content for your analysis; it does not deliver a file to the user.
|
|
||||||
- When sending an existing local file, attach it through the message/media mechanism instead of pasting file contents unless the user asked for text.
|
|
||||||
|
|
||||||
## Scheduling and Background Work
|
|
||||||
|
|
||||||
- Use `cron` for scheduled reminders or recurring jobs; do not run `nanobot cron` through `exec`.
|
|
||||||
- For heartbeat tasks, register `HEARTBEAT.md` as a cron job according to the agent instructions.
|
|
||||||
- Do not write reminders only to memory files when the user expects an actual notification.
|
|
||||||
@@ -7,6 +7,7 @@ from loguru import logger
|
|||||||
|
|
||||||
from nanobot.utils.helpers import detect_image_mime
|
from nanobot.utils.helpers import detect_image_mime
|
||||||
|
|
||||||
|
|
||||||
# Supported file extensions for text extraction
|
# Supported file extensions for text extraction
|
||||||
SUPPORTED_EXTENSIONS: set[str] = {
|
SUPPORTED_EXTENSIONS: set[str] = {
|
||||||
# Document formats
|
# Document formats
|
||||||
@@ -231,46 +232,6 @@ def _is_text_extension(ext: str) -> bool:
|
|||||||
_MAX_EXTRACT_FILE_SIZE = 50 * 1024 * 1024 # 50 MB
|
_MAX_EXTRACT_FILE_SIZE = 50 * 1024 * 1024 # 50 MB
|
||||||
|
|
||||||
|
|
||||||
def is_image_file(path: str) -> bool:
|
|
||||||
"""Check whether *path* looks like an image file.
|
|
||||||
|
|
||||||
Uses magic-byte detection (reads first 16 bytes) with a ``mimetypes``
|
|
||||||
extension-based fallback.
|
|
||||||
"""
|
|
||||||
p = Path(path)
|
|
||||||
mime: str | None = None
|
|
||||||
if p.is_file():
|
|
||||||
try:
|
|
||||||
with p.open("rb") as f:
|
|
||||||
mime = detect_image_mime(f.read(16))
|
|
||||||
except OSError:
|
|
||||||
mime = None
|
|
||||||
if not mime:
|
|
||||||
mime = mimetypes.guess_type(path)[0]
|
|
||||||
return bool(mime and mime.startswith("image/"))
|
|
||||||
|
|
||||||
|
|
||||||
def reference_non_image_attachments(
|
|
||||||
content: str, media: list[str],
|
|
||||||
) -> tuple[str, list[str]]:
|
|
||||||
"""Separate images from non-image attachments without reading file content.
|
|
||||||
|
|
||||||
Image paths are preserved for downstream vision-block construction.
|
|
||||||
Non-image paths are appended as ``[Attachment: path]`` references.
|
|
||||||
"""
|
|
||||||
image_paths: list[str] = []
|
|
||||||
attachment_refs: list[str] = []
|
|
||||||
for path in media:
|
|
||||||
if is_image_file(path):
|
|
||||||
image_paths.append(path)
|
|
||||||
else:
|
|
||||||
attachment_refs.append(f"[Attachment: {path}]")
|
|
||||||
if attachment_refs:
|
|
||||||
suffix = "\n".join(attachment_refs)
|
|
||||||
content = f"{content}\n\n{suffix}" if content else suffix
|
|
||||||
return content, image_paths
|
|
||||||
|
|
||||||
|
|
||||||
def extract_documents(
|
def extract_documents(
|
||||||
text: str,
|
text: str,
|
||||||
media_paths: list[str],
|
media_paths: list[str],
|
||||||
@@ -306,7 +267,10 @@ def extract_documents(
|
|||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
if is_image_file(path_str):
|
with open(p, "rb") as f:
|
||||||
|
header = f.read(16)
|
||||||
|
mime = detect_image_mime(header) or mimetypes.guess_type(path_str)[0]
|
||||||
|
if mime and mime.startswith("image/"):
|
||||||
image_paths.append(path_str)
|
image_paths.append(path_str)
|
||||||
else:
|
else:
|
||||||
extracted = extract_text(p)
|
extracted = extract_text(p)
|
||||||
|
|||||||
@@ -44,15 +44,12 @@ async def evaluate_response(
|
|||||||
task_context: str,
|
task_context: str,
|
||||||
provider: LLMProvider,
|
provider: LLMProvider,
|
||||||
model: str,
|
model: str,
|
||||||
*,
|
|
||||||
default_notify: bool = True,
|
|
||||||
) -> bool:
|
) -> bool:
|
||||||
"""Decide whether a background-task result should be delivered to the user.
|
"""Decide whether a background-task result should be delivered to the user.
|
||||||
|
|
||||||
Uses a lightweight tool-call LLM request. ``default_notify`` controls
|
Uses a lightweight tool-call LLM request (same pattern as heartbeat
|
||||||
the fallback path when the evaluator cannot produce a valid decision:
|
``_decide()``). Falls back to ``True`` (notify) on any failure so
|
||||||
user-scheduled reminders stay fail-open, while internal checks such as
|
that important messages are never silently dropped.
|
||||||
heartbeat can fail closed.
|
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
llm_response = await provider.chat_with_retry(
|
llm_response = await provider.chat_with_retry(
|
||||||
@@ -74,23 +71,19 @@ async def evaluate_response(
|
|||||||
if not llm_response.should_execute_tools:
|
if not llm_response.should_execute_tools:
|
||||||
if llm_response.has_tool_calls:
|
if llm_response.has_tool_calls:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"evaluate_response: ignoring tool calls under finish_reason='{}', defaulting to notify={}",
|
"evaluate_response: ignoring tool calls under finish_reason='{}', defaulting to notify",
|
||||||
llm_response.finish_reason,
|
llm_response.finish_reason,
|
||||||
default_notify,
|
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
logger.warning(
|
logger.warning("evaluate_response: no tool call returned, defaulting to notify")
|
||||||
"evaluate_response: no tool call returned, defaulting to notify={}",
|
return True
|
||||||
default_notify,
|
|
||||||
)
|
|
||||||
return default_notify
|
|
||||||
|
|
||||||
args = llm_response.tool_calls[0].arguments
|
args = llm_response.tool_calls[0].arguments
|
||||||
should_notify = args.get("should_notify", default_notify)
|
should_notify = args.get("should_notify", True)
|
||||||
reason = args.get("reason", "")
|
reason = args.get("reason", "")
|
||||||
logger.info("evaluate_response: should_notify={}, reason={}", should_notify, reason)
|
logger.info("evaluate_response: should_notify={}, reason={}", should_notify, reason)
|
||||||
return bool(should_notify)
|
return bool(should_notify)
|
||||||
|
|
||||||
except Exception:
|
except Exception:
|
||||||
logger.exception("evaluate_response failed, defaulting to notify={}", default_notify)
|
logger.exception("evaluate_response failed, defaulting to notify")
|
||||||
return default_notify
|
return True
|
||||||
|
|||||||
@@ -3,13 +3,15 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import difflib
|
import difflib
|
||||||
|
import json
|
||||||
import re
|
import re
|
||||||
import time
|
import time
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Awaitable, Callable
|
from typing import Any, Awaitable, Callable
|
||||||
|
|
||||||
TRACKED_FILE_EDIT_TOOLS = frozenset({"write_file", "edit_file", "apply_patch"})
|
|
||||||
|
TRACKED_FILE_EDIT_TOOLS = frozenset({"write_file", "edit_file", "notebook_edit"})
|
||||||
_MAX_SNAPSHOT_BYTES = 2 * 1024 * 1024
|
_MAX_SNAPSHOT_BYTES = 2 * 1024 * 1024
|
||||||
_LIVE_EMIT_INTERVAL_S = 0.18
|
_LIVE_EMIT_INTERVAL_S = 0.18
|
||||||
_LIVE_EMIT_LINE_STEP = 24
|
_LIVE_EMIT_LINE_STEP = 24
|
||||||
@@ -152,108 +154,19 @@ def prepare_file_edit_tracker(
|
|||||||
workspace: Path | None,
|
workspace: Path | None,
|
||||||
params: dict[str, Any] | None,
|
params: dict[str, Any] | None,
|
||||||
) -> FileEditTracker | None:
|
) -> FileEditTracker | None:
|
||||||
trackers = prepare_file_edit_trackers(
|
|
||||||
call_id=call_id,
|
|
||||||
tool_name=tool_name,
|
|
||||||
tool=tool,
|
|
||||||
workspace=workspace,
|
|
||||||
params=params,
|
|
||||||
)
|
|
||||||
return trackers[0] if trackers else None
|
|
||||||
|
|
||||||
|
|
||||||
def prepare_file_edit_trackers(
|
|
||||||
*,
|
|
||||||
call_id: str,
|
|
||||||
tool_name: str,
|
|
||||||
tool: Any,
|
|
||||||
workspace: Path | None,
|
|
||||||
params: dict[str, Any] | None,
|
|
||||||
) -> list[FileEditTracker]:
|
|
||||||
if not is_file_edit_tool(tool_name):
|
if not is_file_edit_tool(tool_name):
|
||||||
return []
|
return None
|
||||||
paths = resolve_file_edit_paths(tool_name, tool, workspace, params)
|
path = resolve_file_edit_path(tool, workspace, params)
|
||||||
trackers: list[FileEditTracker] = []
|
if path is None:
|
||||||
seen: set[Path] = set()
|
return None
|
||||||
for path in paths:
|
|
||||||
try:
|
|
||||||
resolved = path.resolve()
|
|
||||||
except Exception:
|
|
||||||
resolved = path
|
|
||||||
if resolved in seen:
|
|
||||||
continue
|
|
||||||
seen.add(resolved)
|
|
||||||
before = read_file_snapshot(path)
|
before = read_file_snapshot(path)
|
||||||
trackers.append(FileEditTracker(
|
return FileEditTracker(
|
||||||
call_id=str(call_id or ""),
|
call_id=str(call_id or ""),
|
||||||
tool=tool_name,
|
tool=tool_name,
|
||||||
path=path,
|
path=path,
|
||||||
display_path=display_file_edit_path(path, workspace),
|
display_path=display_file_edit_path(path, workspace),
|
||||||
before=before,
|
before=before,
|
||||||
))
|
)
|
||||||
return trackers
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_file_edit_paths(
|
|
||||||
tool_name: str,
|
|
||||||
tool: Any,
|
|
||||||
workspace: Path | None,
|
|
||||||
params: dict[str, Any] | None,
|
|
||||||
) -> list[Path]:
|
|
||||||
if tool_name == "apply_patch":
|
|
||||||
return _resolve_apply_patch_paths(tool, workspace, params)
|
|
||||||
path = resolve_file_edit_path(tool, workspace, params)
|
|
||||||
if path is None:
|
|
||||||
return []
|
|
||||||
return [path]
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_apply_patch_paths(
|
|
||||||
tool: Any,
|
|
||||||
workspace: Path | None,
|
|
||||||
params: dict[str, Any] | None,
|
|
||||||
) -> list[Path]:
|
|
||||||
if not isinstance(params, dict):
|
|
||||||
return []
|
|
||||||
edits = params.get("edits")
|
|
||||||
if not isinstance(edits, list) or not edits:
|
|
||||||
return []
|
|
||||||
if params.get("dry_run") is True:
|
|
||||||
return []
|
|
||||||
|
|
||||||
resolved: list[Path] = []
|
|
||||||
seen: set[Path] = set()
|
|
||||||
for edit in edits:
|
|
||||||
if not isinstance(edit, dict):
|
|
||||||
continue
|
|
||||||
raw_path = edit.get("path")
|
|
||||||
if not isinstance(raw_path, str) or not raw_path.strip():
|
|
||||||
continue
|
|
||||||
path = _resolve_raw_file_edit_path(tool, workspace, raw_path)
|
|
||||||
if path is not None and path not in seen:
|
|
||||||
seen.add(path)
|
|
||||||
resolved.append(path)
|
|
||||||
return resolved
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_raw_file_edit_path(
|
|
||||||
tool: Any,
|
|
||||||
workspace: Path | None,
|
|
||||||
raw_path: str,
|
|
||||||
) -> Path | None:
|
|
||||||
resolver = getattr(tool, "_resolve", None)
|
|
||||||
if callable(resolver):
|
|
||||||
try:
|
|
||||||
resolved = resolver(raw_path)
|
|
||||||
if isinstance(resolved, Path):
|
|
||||||
return resolved
|
|
||||||
if resolved:
|
|
||||||
return Path(resolved)
|
|
||||||
except Exception:
|
|
||||||
return None
|
|
||||||
if workspace is None:
|
|
||||||
return Path(raw_path).expanduser().resolve()
|
|
||||||
return (workspace / raw_path).expanduser().resolve()
|
|
||||||
|
|
||||||
|
|
||||||
def build_file_edit_start_event(
|
def build_file_edit_start_event(
|
||||||
@@ -299,7 +212,6 @@ def build_file_edit_end_event(
|
|||||||
deleted=deleted,
|
deleted=deleted,
|
||||||
approximate=False,
|
approximate=False,
|
||||||
binary=(after.binary or after.oversized or after.unreadable) and not counted,
|
binary=(after.binary or after.oversized or after.unreadable) and not counted,
|
||||||
operation="delete" if tracker.before.exists and not after.exists else None,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -325,7 +237,6 @@ def build_file_edit_live_event(
|
|||||||
*,
|
*,
|
||||||
added: int,
|
added: int,
|
||||||
deleted: int = 0,
|
deleted: int = 0,
|
||||||
operation: str | None = None,
|
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
"""Build an approximate in-progress event while tool-call arguments stream."""
|
"""Build an approximate in-progress event while tool-call arguments stream."""
|
||||||
return _event_payload(
|
return _event_payload(
|
||||||
@@ -335,7 +246,6 @@ def build_file_edit_live_event(
|
|||||||
added=added,
|
added=added,
|
||||||
deleted=deleted,
|
deleted=deleted,
|
||||||
approximate=True,
|
approximate=True,
|
||||||
operation=operation,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -394,9 +304,6 @@ class StreamingFileEditTracker:
|
|||||||
self._states[key] = state
|
self._states[key] = state
|
||||||
|
|
||||||
state.apply_delta(payload)
|
state.apply_delta(payload)
|
||||||
if state.name == "apply_patch":
|
|
||||||
await self._update_apply_patch(state)
|
|
||||||
return
|
|
||||||
if state.name not in {"write_file", "edit_file"}:
|
if state.name not in {"write_file", "edit_file"}:
|
||||||
return
|
return
|
||||||
if state.path is None:
|
if state.path is None:
|
||||||
@@ -436,77 +343,10 @@ class StreamingFileEditTracker:
|
|||||||
deleted=deleted,
|
deleted=deleted,
|
||||||
)])
|
)])
|
||||||
|
|
||||||
async def _update_apply_patch(self, state: _StreamingFileEditState) -> None:
|
|
||||||
if _json_bool_true(state.arguments, "dry_run"):
|
|
||||||
return
|
|
||||||
tool = self._tools.get("apply_patch") if hasattr(self._tools, "get") else None
|
|
||||||
events: list[dict[str, Any]] = []
|
|
||||||
now = time.monotonic()
|
|
||||||
|
|
||||||
path_matches = list(re.finditer(r'"path"\s*:\s*"([^"]+)"', state.arguments))
|
|
||||||
if not path_matches:
|
|
||||||
return
|
|
||||||
|
|
||||||
for i, m in enumerate(path_matches):
|
|
||||||
raw_path = m.group(1)
|
|
||||||
path = _resolve_raw_file_edit_path(tool, self._workspace, raw_path)
|
|
||||||
if path is None:
|
|
||||||
continue
|
|
||||||
|
|
||||||
segment_start = m.start()
|
|
||||||
segment_end = path_matches[i + 1].start() if i + 1 < len(path_matches) else len(state.arguments)
|
|
||||||
segment = state.arguments[segment_start:segment_end]
|
|
||||||
|
|
||||||
action_match = re.search(r'"action"\s*:\s*"(replace|add)"', segment)
|
|
||||||
action = action_match.group(1) if action_match else "replace"
|
|
||||||
|
|
||||||
old_text = _extract_json_string_prefix(segment, "old_text") or ""
|
|
||||||
new_text = _extract_json_string_prefix(segment, "new_text") or ""
|
|
||||||
|
|
||||||
added = _text_line_count(new_text) if action in ("replace", "add") else 0
|
|
||||||
deleted = _text_line_count(old_text) if action == "replace" else 0
|
|
||||||
|
|
||||||
file_state = state.patch_files.get(raw_path)
|
|
||||||
if file_state is None:
|
|
||||||
tracker = FileEditTracker(
|
|
||||||
call_id=state.call_id or state.key,
|
|
||||||
tool="apply_patch",
|
|
||||||
path=path,
|
|
||||||
display_path=display_file_edit_path(path, self._workspace),
|
|
||||||
before=read_file_snapshot(path),
|
|
||||||
)
|
|
||||||
file_state = _StreamingPatchFileState(tracker=tracker)
|
|
||||||
state.patch_files[raw_path] = file_state
|
|
||||||
if not file_state.should_emit(added, deleted, now):
|
|
||||||
continue
|
|
||||||
file_state.mark_emitted(added, deleted, now)
|
|
||||||
events.append(build_file_edit_live_event(
|
|
||||||
file_state.tracker,
|
|
||||||
added=added,
|
|
||||||
deleted=deleted,
|
|
||||||
))
|
|
||||||
if events:
|
|
||||||
await self._emit(events)
|
|
||||||
|
|
||||||
async def flush(self) -> None:
|
async def flush(self) -> None:
|
||||||
events: list[dict[str, Any]] = []
|
events: list[dict[str, Any]] = []
|
||||||
now = time.monotonic()
|
now = time.monotonic()
|
||||||
for state in self._states.values():
|
for state in self._states.values():
|
||||||
for file_state in state.patch_files.values():
|
|
||||||
added, deleted = file_state.last_added, file_state.last_deleted
|
|
||||||
if not file_state.emitted_once:
|
|
||||||
continue
|
|
||||||
if (
|
|
||||||
file_state.last_emitted_added == added
|
|
||||||
and file_state.last_emitted_deleted == deleted
|
|
||||||
):
|
|
||||||
continue
|
|
||||||
file_state.mark_emitted(added, deleted, now)
|
|
||||||
events.append(build_file_edit_live_event(
|
|
||||||
file_state.tracker,
|
|
||||||
added=added,
|
|
||||||
deleted=deleted,
|
|
||||||
))
|
|
||||||
if state.tracker is None:
|
if state.tracker is None:
|
||||||
continue
|
continue
|
||||||
added, deleted = state.live_diff_counts()
|
added, deleted = state.live_diff_counts()
|
||||||
@@ -527,14 +367,12 @@ class StreamingFileEditTracker:
|
|||||||
|
|
||||||
def apply_final_call_ids(self, final_tool_calls: list[Any]) -> None:
|
def apply_final_call_ids(self, final_tool_calls: list[Any]) -> None:
|
||||||
"""Keep final start/end events keyed to any earlier streamed placeholder."""
|
"""Keep final start/end events keyed to any earlier streamed placeholder."""
|
||||||
used_canonicals: set[str] = set()
|
|
||||||
for tool_call in final_tool_calls:
|
for tool_call in final_tool_calls:
|
||||||
canonical = self.canonical_call_id_for(tool_call)
|
canonical = self.canonical_call_id_for(tool_call)
|
||||||
if canonical and canonical not in used_canonicals:
|
if canonical:
|
||||||
try:
|
try:
|
||||||
tool_call.id = canonical
|
tool_call.id = canonical
|
||||||
used_canonicals.add(canonical)
|
except Exception:
|
||||||
except (AttributeError, TypeError):
|
|
||||||
pass
|
pass
|
||||||
|
|
||||||
def canonical_call_id_for(self, tool_call: Any) -> str | None:
|
def canonical_call_id_for(self, tool_call: Any) -> str | None:
|
||||||
@@ -551,10 +389,6 @@ class StreamingFileEditTracker:
|
|||||||
"""Mark streamed edits as failed when no final tool call will run."""
|
"""Mark streamed edits as failed when no final tool call will run."""
|
||||||
events: list[dict[str, Any]] = []
|
events: list[dict[str, Any]] = []
|
||||||
for state in self._states.values():
|
for state in self._states.values():
|
||||||
for file_state in state.patch_files.values():
|
|
||||||
if any(state.matches_final_tool_call(tool_call) for tool_call in final_tool_calls):
|
|
||||||
continue
|
|
||||||
events.append(build_file_edit_error_event(file_state.tracker, error))
|
|
||||||
if state.tracker is None:
|
if state.tracker is None:
|
||||||
continue
|
continue
|
||||||
if any(state.matches_final_tool_call(tool_call) for tool_call in final_tool_calls):
|
if any(state.matches_final_tool_call(tool_call) for tool_call in final_tool_calls):
|
||||||
@@ -658,39 +492,6 @@ class _StreamingJsonStringField:
|
|||||||
self.last_char_cr = False
|
self.last_char_cr = False
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
|
||||||
class _StreamingPatchFileState:
|
|
||||||
tracker: FileEditTracker
|
|
||||||
emitted_once: bool = False
|
|
||||||
last_emitted_added: int = -1
|
|
||||||
last_emitted_deleted: int = -1
|
|
||||||
last_emit_at: float = 0.0
|
|
||||||
last_added: int = 0
|
|
||||||
last_deleted: int = 0
|
|
||||||
|
|
||||||
def should_emit(self, added: int, deleted: int, now: float) -> bool:
|
|
||||||
self.last_added = added
|
|
||||||
self.last_deleted = deleted
|
|
||||||
if not self.emitted_once:
|
|
||||||
return True
|
|
||||||
if added == self.last_emitted_added and deleted == self.last_emitted_deleted:
|
|
||||||
return False
|
|
||||||
if max(
|
|
||||||
abs(added - self.last_emitted_added),
|
|
||||||
abs(deleted - self.last_emitted_deleted),
|
|
||||||
) >= _LIVE_EMIT_LINE_STEP:
|
|
||||||
return True
|
|
||||||
return now - self.last_emit_at >= _LIVE_EMIT_INTERVAL_S
|
|
||||||
|
|
||||||
def mark_emitted(self, added: int, deleted: int, now: float) -> None:
|
|
||||||
self.emitted_once = True
|
|
||||||
self.last_added = added
|
|
||||||
self.last_deleted = deleted
|
|
||||||
self.last_emitted_added = added
|
|
||||||
self.last_emitted_deleted = deleted
|
|
||||||
self.last_emit_at = now
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
@dataclass(slots=True)
|
||||||
class _StreamingFileEditState:
|
class _StreamingFileEditState:
|
||||||
key: str
|
key: str
|
||||||
@@ -708,7 +509,6 @@ class _StreamingFileEditState:
|
|||||||
new_text: _StreamingJsonStringField = field(
|
new_text: _StreamingJsonStringField = field(
|
||||||
default_factory=lambda: _StreamingJsonStringField("new_text")
|
default_factory=lambda: _StreamingJsonStringField("new_text")
|
||||||
)
|
)
|
||||||
patch_files: dict[str, _StreamingPatchFileState] = field(default_factory=dict)
|
|
||||||
emitted_once: bool = False
|
emitted_once: bool = False
|
||||||
last_emitted_added: int = -1
|
last_emitted_added: int = -1
|
||||||
last_emitted_deleted: int = -1
|
last_emitted_deleted: int = -1
|
||||||
@@ -731,7 +531,6 @@ class _StreamingFileEditState:
|
|||||||
self.content.reset()
|
self.content.reset()
|
||||||
self.old_text.reset()
|
self.old_text.reset()
|
||||||
self.new_text.reset()
|
self.new_text.reset()
|
||||||
self.patch_files.clear()
|
|
||||||
return
|
return
|
||||||
delta = payload.get("arguments_delta")
|
delta = payload.get("arguments_delta")
|
||||||
if isinstance(delta, str) and delta:
|
if isinstance(delta, str) and delta:
|
||||||
@@ -791,14 +590,6 @@ class _StreamingFileEditState:
|
|||||||
name = getattr(tool_call, "name", None)
|
name = getattr(tool_call, "name", None)
|
||||||
if name != self.name:
|
if name != self.name:
|
||||||
return False
|
return False
|
||||||
if self.name == "apply_patch":
|
|
||||||
arguments = getattr(tool_call, "arguments", None)
|
|
||||||
if not isinstance(arguments, dict):
|
|
||||||
return False
|
|
||||||
edits = arguments.get("edits")
|
|
||||||
if not isinstance(edits, list):
|
|
||||||
return False
|
|
||||||
return '"edits"' in self.arguments
|
|
||||||
arguments = getattr(tool_call, "arguments", None)
|
arguments = getattr(tool_call, "arguments", None)
|
||||||
if not isinstance(arguments, dict):
|
if not isinstance(arguments, dict):
|
||||||
return False
|
return False
|
||||||
@@ -821,51 +612,6 @@ def _stream_key(payload: dict[str, Any]) -> str:
|
|||||||
return ""
|
return ""
|
||||||
|
|
||||||
|
|
||||||
def _json_bool_true(source: str, key: str) -> bool:
|
|
||||||
return re.search(rf'"{re.escape(key)}"\s*:\s*true\b', source) is not None
|
|
||||||
|
|
||||||
|
|
||||||
def _extract_json_string_prefix(source: str, key: str) -> str | None:
|
|
||||||
match = re.search(rf'"{re.escape(key)}"\s*:\s*"', source)
|
|
||||||
if match is None:
|
|
||||||
return None
|
|
||||||
out: list[str] = []
|
|
||||||
i = match.end()
|
|
||||||
escape = False
|
|
||||||
while i < len(source):
|
|
||||||
ch = source[i]
|
|
||||||
if escape:
|
|
||||||
escape = False
|
|
||||||
if ch == "n":
|
|
||||||
out.append("\n")
|
|
||||||
elif ch == "r":
|
|
||||||
out.append("\r")
|
|
||||||
elif ch == "t":
|
|
||||||
out.append("\t")
|
|
||||||
elif ch == "u":
|
|
||||||
digits = source[i + 1:i + 5]
|
|
||||||
if len(digits) < 4:
|
|
||||||
break
|
|
||||||
try:
|
|
||||||
out.append(chr(int(digits, 16)))
|
|
||||||
except ValueError:
|
|
||||||
break
|
|
||||||
i += 4
|
|
||||||
else:
|
|
||||||
out.append(ch)
|
|
||||||
i += 1
|
|
||||||
continue
|
|
||||||
if ch == "\\":
|
|
||||||
escape = True
|
|
||||||
i += 1
|
|
||||||
continue
|
|
||||||
if ch == '"':
|
|
||||||
return "".join(out)
|
|
||||||
out.append(ch)
|
|
||||||
i += 1
|
|
||||||
return "".join(out)
|
|
||||||
|
|
||||||
|
|
||||||
def _extract_complete_json_string(source: str, key: str) -> str | None:
|
def _extract_complete_json_string(source: str, key: str) -> str | None:
|
||||||
match = re.search(rf'"{re.escape(key)}"\s*:\s*"', source)
|
match = re.search(rf'"{re.escape(key)}"\s*:\s*"', source)
|
||||||
if match is None:
|
if match is None:
|
||||||
@@ -916,7 +662,6 @@ def _event_payload(
|
|||||||
deleted: int,
|
deleted: int,
|
||||||
approximate: bool,
|
approximate: bool,
|
||||||
binary: bool = False,
|
binary: bool = False,
|
||||||
operation: str | None = None,
|
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
payload: dict[str, Any] = {
|
payload: dict[str, Any] = {
|
||||||
"version": 1,
|
"version": 1,
|
||||||
@@ -932,8 +677,6 @@ def _event_payload(
|
|||||||
}
|
}
|
||||||
if binary:
|
if binary:
|
||||||
payload["binary"] = True
|
payload["binary"] = True
|
||||||
if operation:
|
|
||||||
payload["operation"] = operation
|
|
||||||
return payload
|
return payload
|
||||||
|
|
||||||
|
|
||||||
@@ -961,4 +704,77 @@ def _predict_after_text(
|
|||||||
return before_text.replace(old_text, new_text)
|
return before_text.replace(old_text, new_text)
|
||||||
return before_text.replace(old_text, new_text, 1)
|
return before_text.replace(old_text, new_text, 1)
|
||||||
return None
|
return None
|
||||||
|
if tool_name == "notebook_edit":
|
||||||
|
return _predict_notebook_after_text(params, before_text)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _predict_notebook_after_text(params: dict[str, Any], before_text: str) -> str | None:
|
||||||
|
try:
|
||||||
|
nb = json.loads(before_text) if before_text.strip() else _empty_notebook()
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
cells = nb.get("cells")
|
||||||
|
if not isinstance(cells, list):
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
cell_index = int(params.get("cell_index", 0))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
new_source = params.get("new_source")
|
||||||
|
source = new_source if isinstance(new_source, str) else ""
|
||||||
|
cell_type = (
|
||||||
|
params.get("cell_type") if params.get("cell_type") in ("code", "markdown") else "code"
|
||||||
|
)
|
||||||
|
mode = (
|
||||||
|
params.get("edit_mode")
|
||||||
|
if params.get("edit_mode") in ("replace", "insert", "delete")
|
||||||
|
else "replace"
|
||||||
|
)
|
||||||
|
if mode == "delete":
|
||||||
|
if 0 <= cell_index < len(cells):
|
||||||
|
cells.pop(cell_index)
|
||||||
|
else:
|
||||||
|
return None
|
||||||
|
elif mode == "insert":
|
||||||
|
insert_at = min(max(cell_index + 1, 0), len(cells))
|
||||||
|
cells.insert(insert_at, _new_notebook_cell(source, str(cell_type)))
|
||||||
|
else:
|
||||||
|
if not (0 <= cell_index < len(cells)):
|
||||||
|
return None
|
||||||
|
cell = cells[cell_index]
|
||||||
|
if not isinstance(cell, dict):
|
||||||
|
return None
|
||||||
|
cell["source"] = source
|
||||||
|
cell["cell_type"] = cell_type
|
||||||
|
if cell_type == "code":
|
||||||
|
cell.setdefault("outputs", [])
|
||||||
|
cell.setdefault("execution_count", None)
|
||||||
|
else:
|
||||||
|
cell.pop("outputs", None)
|
||||||
|
cell.pop("execution_count", None)
|
||||||
|
nb["cells"] = cells
|
||||||
|
try:
|
||||||
|
return json.dumps(nb, indent=1, ensure_ascii=False)
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_notebook() -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"nbformat": 4,
|
||||||
|
"nbformat_minor": 5,
|
||||||
|
"metadata": {
|
||||||
|
"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"},
|
||||||
|
"language_info": {"name": "python"},
|
||||||
|
},
|
||||||
|
"cells": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _new_notebook_cell(source: str, cell_type: str) -> dict[str, Any]:
|
||||||
|
cell: dict[str, Any] = {"cell_type": cell_type, "source": source, "metadata": {}}
|
||||||
|
if cell_type == "code":
|
||||||
|
cell["outputs"] = []
|
||||||
|
cell["execution_count"] = None
|
||||||
|
return cell
|
||||||
|
|||||||
@@ -576,7 +576,7 @@ def build_status_content(
|
|||||||
|
|
||||||
|
|
||||||
def sync_workspace_templates(workspace: Path, silent: bool = False) -> list[str]:
|
def sync_workspace_templates(workspace: Path, silent: bool = False) -> list[str]:
|
||||||
"""Sync bundled templates to workspace. Creates missing files without overwriting user files."""
|
"""Sync bundled templates to workspace. Only creates missing files."""
|
||||||
from importlib.resources import files as pkg_files
|
from importlib.resources import files as pkg_files
|
||||||
|
|
||||||
try:
|
try:
|
||||||
@@ -589,11 +589,10 @@ def sync_workspace_templates(workspace: Path, silent: bool = False) -> list[str]
|
|||||||
added: list[str] = []
|
added: list[str] = []
|
||||||
|
|
||||||
def _write(src, dest: Path):
|
def _write(src, dest: Path):
|
||||||
content = src.read_text(encoding="utf-8") if src else ""
|
|
||||||
if dest.exists():
|
if dest.exists():
|
||||||
return
|
return
|
||||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||||
dest.write_text(content, encoding="utf-8")
|
dest.write_text(src.read_text(encoding="utf-8") if src else "", encoding="utf-8")
|
||||||
added.append(str(dest.relative_to(workspace)))
|
added.append(str(dest.relative_to(workspace)))
|
||||||
|
|
||||||
for item in tpl.iterdir():
|
for item in tpl.iterdir():
|
||||||
@@ -626,14 +625,3 @@ def sync_workspace_templates(workspace: Path, silent: bool = False) -> list[str]
|
|||||||
logger.exception("Failed to initialize git store for {}", workspace)
|
logger.exception("Failed to initialize git store for {}", workspace)
|
||||||
|
|
||||||
return added
|
return added
|
||||||
|
|
||||||
|
|
||||||
def load_bundled_template(template_name: str) -> str | None:
|
|
||||||
"""Read a bundled template file from the nanobot package."""
|
|
||||||
from importlib.resources import files as pkg_files
|
|
||||||
|
|
||||||
with suppress(Exception):
|
|
||||||
tpl = pkg_files("nanobot") / "templates" / template_name
|
|
||||||
if tpl.is_file():
|
|
||||||
return tpl.read_text(encoding="utf-8")
|
|
||||||
return None
|
|
||||||
|
|||||||
@@ -29,11 +29,6 @@ LENGTH_RECOVERY_PROMPT = (
|
|||||||
"— no recap, no apology. Break remaining work into smaller steps if needed."
|
"— no recap, no apology. Break remaining work into smaller steps if needed."
|
||||||
)
|
)
|
||||||
|
|
||||||
SUSTAINED_GOAL_CONTINUE_PROMPT = (
|
|
||||||
"You have an active sustained goal. Please continue working toward the "
|
|
||||||
"objective using your tools, or call complete_goal if the work is truly finished."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def empty_tool_result_message(tool_name: str) -> str:
|
def empty_tool_result_message(tool_name: str) -> str:
|
||||||
"""Short prompt-safe marker for tools that completed without visible output."""
|
"""Short prompt-safe marker for tools that completed without visible output."""
|
||||||
@@ -70,11 +65,6 @@ def build_length_recovery_message() -> dict[str, str]:
|
|||||||
return {"role": "user", "content": LENGTH_RECOVERY_PROMPT}
|
return {"role": "user", "content": LENGTH_RECOVERY_PROMPT}
|
||||||
|
|
||||||
|
|
||||||
def build_goal_continue_message(custom: str | None = None) -> dict[str, str]:
|
|
||||||
"""Prompt the model to continue when a sustained goal is still active."""
|
|
||||||
return {"role": "user", "content": custom or SUSTAINED_GOAL_CONTINUE_PROMPT}
|
|
||||||
|
|
||||||
|
|
||||||
def external_lookup_signature(tool_name: str, arguments: dict[str, Any]) -> str | None:
|
def external_lookup_signature(tool_name: str, arguments: dict[str, Any]) -> str | None:
|
||||||
"""Stable signature for repeated external lookups we want to throttle."""
|
"""Stable signature for repeated external lookups we want to throttle."""
|
||||||
if tool_name == "web_fetch":
|
if tool_name == "web_fetch":
|
||||||
|
|||||||
@@ -11,10 +11,8 @@ _TOOL_FORMATS: dict[str, tuple[list[str], str, bool, bool]] = {
|
|||||||
"read_file": (["path", "file_path"], "read {}", True, False),
|
"read_file": (["path", "file_path"], "read {}", True, False),
|
||||||
"write_file": (["path", "file_path"], "write {}", True, False),
|
"write_file": (["path", "file_path"], "write {}", True, False),
|
||||||
"edit": (["file_path", "path"], "edit {}", True, False),
|
"edit": (["file_path", "path"], "edit {}", True, False),
|
||||||
"find_files": (["query", "glob", "path"], "find {}", False, False),
|
|
||||||
"grep": (["pattern"], 'grep "{}"', False, False),
|
"grep": (["pattern"], 'grep "{}"', False, False),
|
||||||
"exec": (["command"], "$ {}", False, True),
|
"exec": (["command"], "$ {}", False, True),
|
||||||
"list_exec_sessions": ([], "exec sessions", False, False),
|
|
||||||
"web_search": (["query"], 'search "{}"', False, False),
|
"web_search": (["query"], 'search "{}"', False, False),
|
||||||
"web_fetch": (["url"], "fetch {}", True, False),
|
"web_fetch": (["url"], "fetch {}", True, False),
|
||||||
"list_dir": (["path"], "ls {}", True, False),
|
"list_dir": (["path"], "ls {}", True, False),
|
||||||
@@ -83,8 +81,6 @@ def _extract_arg(tc, key_args: list[str]) -> str | None:
|
|||||||
|
|
||||||
def _fmt_known(tc, fmt: tuple, max_length: int = 40) -> str:
|
def _fmt_known(tc, fmt: tuple, max_length: int = 40) -> str:
|
||||||
"""Format a registered tool using its template."""
|
"""Format a registered tool using its template."""
|
||||||
if not fmt[0] and "{}" not in fmt[1]:
|
|
||||||
return fmt[1]
|
|
||||||
val = _extract_arg(tc, fmt[0])
|
val = _extract_arg(tc, fmt[0])
|
||||||
if val is None:
|
if val is None:
|
||||||
return tc.name
|
return tc.name
|
||||||
|
|||||||
@@ -1,93 +0,0 @@
|
|||||||
"""CLI Apps helpers for the WebUI HTTP and message surfaces."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import re
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from nanobot.apps.cli import CliAppError, CliAppManager, CliAppsRuntimeConfig
|
|
||||||
from nanobot.config.loader import load_config
|
|
||||||
|
|
||||||
QueryParams = dict[str, list[str]]
|
|
||||||
|
|
||||||
_CLI_APP_NAME_RE = re.compile(r"^[a-z0-9][a-z0-9_-]{0,63}$", re.IGNORECASE)
|
|
||||||
_CLI_APP_ATTACHMENT_KEYS = (
|
|
||||||
"name",
|
|
||||||
"display_name",
|
|
||||||
"category",
|
|
||||||
"entry_point",
|
|
||||||
"logo_url",
|
|
||||||
"brand_color",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _clip_ws_string(value: Any, limit: int = 240) -> str | None:
|
|
||||||
if not isinstance(value, str):
|
|
||||||
return None
|
|
||||||
text = value.strip()
|
|
||||||
if not text:
|
|
||||||
return None
|
|
||||||
return text[:limit]
|
|
||||||
|
|
||||||
|
|
||||||
def normalize_cli_app_mentions(raw: Any) -> list[dict[str, str]]:
|
|
||||||
"""Sanitize structured CLI app mentions sent by the WebUI."""
|
|
||||||
if not isinstance(raw, list):
|
|
||||||
return []
|
|
||||||
out: list[dict[str, str]] = []
|
|
||||||
seen: set[str] = set()
|
|
||||||
for item in raw[:8]:
|
|
||||||
if not isinstance(item, dict):
|
|
||||||
continue
|
|
||||||
name = _clip_ws_string(item.get("name"), 64)
|
|
||||||
if not name or _CLI_APP_NAME_RE.match(name) is None:
|
|
||||||
continue
|
|
||||||
key = name.lower()
|
|
||||||
if key in seen:
|
|
||||||
continue
|
|
||||||
seen.add(key)
|
|
||||||
row: dict[str, str] = {"name": key}
|
|
||||||
for field in _CLI_APP_ATTACHMENT_KEYS[1:]:
|
|
||||||
value = _clip_ws_string(item.get(field), 512 if field == "logo_url" else 160)
|
|
||||||
if value:
|
|
||||||
row[field] = value
|
|
||||||
out.append(row)
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _query_first(query: QueryParams, key: str) -> str | None:
|
|
||||||
values = query.get(key)
|
|
||||||
return values[0] if values else None
|
|
||||||
|
|
||||||
|
|
||||||
def _manager() -> CliAppManager:
|
|
||||||
config = load_config()
|
|
||||||
cli_cfg = config.tools.cli_apps
|
|
||||||
return CliAppManager(
|
|
||||||
workspace=config.workspace_path,
|
|
||||||
runtime=CliAppsRuntimeConfig(
|
|
||||||
install_timeout=cli_cfg.install_timeout,
|
|
||||||
run_timeout=cli_cfg.run_timeout,
|
|
||||||
catalog_ttl_seconds=cli_cfg.catalog_ttl_seconds,
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def cli_apps_payload() -> dict[str, Any]:
|
|
||||||
return _manager().payload()
|
|
||||||
|
|
||||||
|
|
||||||
def cli_apps_action(action: str, query: QueryParams) -> dict[str, Any]:
|
|
||||||
name = (_query_first(query, "name") or "").strip()
|
|
||||||
if not name:
|
|
||||||
raise CliAppError("missing CLI app name")
|
|
||||||
manager = _manager()
|
|
||||||
if action == "install":
|
|
||||||
return manager.install(name)
|
|
||||||
if action == "update":
|
|
||||||
return manager.update(name)
|
|
||||||
if action == "uninstall":
|
|
||||||
return manager.uninstall(name)
|
|
||||||
if action == "test":
|
|
||||||
return manager.test(name)
|
|
||||||
raise CliAppError(f"unknown CLI app action '{action}'", status=404)
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,5 +0,0 @@
|
|||||||
"""Compatibility exports for WebUI-attached MCP preset annotations."""
|
|
||||||
|
|
||||||
from nanobot.agent.tools.mcp import runtime_lines, session_extra
|
|
||||||
|
|
||||||
__all__ = ["runtime_lines", "session_extra"]
|
|
||||||
@@ -1,255 +0,0 @@
|
|||||||
"""Signed media helpers for the WebUI HTTP surface."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import base64
|
|
||||||
import binascii
|
|
||||||
import email.utils
|
|
||||||
import hashlib
|
|
||||||
import hmac
|
|
||||||
import http
|
|
||||||
import mimetypes
|
|
||||||
import re
|
|
||||||
import shutil
|
|
||||||
import uuid
|
|
||||||
from collections.abc import Callable
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from websockets.datastructures import Headers
|
|
||||||
from websockets.http11 import Request as WsRequest
|
|
||||||
from websockets.http11 import Response
|
|
||||||
|
|
||||||
from nanobot.config.paths import get_media_dir
|
|
||||||
from nanobot.utils.helpers import safe_filename
|
|
||||||
|
|
||||||
MediaDirProvider = Callable[[str | None], Path]
|
|
||||||
|
|
||||||
|
|
||||||
def b64url_encode(data: bytes) -> str:
|
|
||||||
"""URL-safe base64 without padding."""
|
|
||||||
return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
|
|
||||||
|
|
||||||
|
|
||||||
def b64url_decode(value: str) -> bytes:
|
|
||||||
"""Reverse of :func:`b64url_encode`; caller handles decode errors."""
|
|
||||||
pad = "=" * (-len(value) % 4)
|
|
||||||
return base64.urlsafe_b64decode(value + pad)
|
|
||||||
|
|
||||||
|
|
||||||
def _default_media_dir(channel: str | None = None) -> Path:
|
|
||||||
return get_media_dir(channel)
|
|
||||||
|
|
||||||
|
|
||||||
# Allowed MIME types we actually serve from the media endpoint. Anything
|
|
||||||
# outside this set is degraded to ``application/octet-stream`` so an
|
|
||||||
# attacker who somehow gets a signed URL for an unexpected file type can't
|
|
||||||
# trick the browser into sniffing executable content.
|
|
||||||
_MEDIA_ALLOWED_MIMES: frozenset[str] = frozenset({
|
|
||||||
"image/png",
|
|
||||||
"image/jpeg",
|
|
||||||
"image/webp",
|
|
||||||
"image/gif",
|
|
||||||
"image/svg+xml",
|
|
||||||
"video/mp4",
|
|
||||||
"video/webm",
|
|
||||||
"video/quicktime",
|
|
||||||
})
|
|
||||||
_SVG_MEDIA_HEADERS: tuple[tuple[str, str], ...] = (
|
|
||||||
(
|
|
||||||
"Content-Security-Policy",
|
|
||||||
"default-src 'none'; img-src 'self' data:; style-src 'unsafe-inline'; sandbox",
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
_BYTE_RANGE_RE = re.compile(r"^bytes=(\d*)-(\d*)$")
|
|
||||||
|
|
||||||
|
|
||||||
def _http_response(
|
|
||||||
body: bytes,
|
|
||||||
*,
|
|
||||||
status: int = 200,
|
|
||||||
content_type: str = "text/plain; charset=utf-8",
|
|
||||||
extra_headers: list[tuple[str, str]] | None = None,
|
|
||||||
) -> Response:
|
|
||||||
headers = [
|
|
||||||
("Date", email.utils.formatdate(usegmt=True)),
|
|
||||||
("Connection", "close"),
|
|
||||||
("Content-Length", str(len(body))),
|
|
||||||
("Content-Type", content_type),
|
|
||||||
]
|
|
||||||
if extra_headers:
|
|
||||||
headers.extend(extra_headers)
|
|
||||||
reason = http.HTTPStatus(status).phrase
|
|
||||||
return Response(status, reason, Headers(headers), body)
|
|
||||||
|
|
||||||
|
|
||||||
def _http_error(status: int, message: str | None = None) -> Response:
|
|
||||||
body = (message or http.HTTPStatus(status).phrase).encode("utf-8")
|
|
||||||
return _http_response(body, status=status)
|
|
||||||
|
|
||||||
|
|
||||||
def _case_insensitive_header(headers: Any, key: str) -> str:
|
|
||||||
try:
|
|
||||||
value = headers.get(key)
|
|
||||||
except Exception:
|
|
||||||
value = None
|
|
||||||
if value is None:
|
|
||||||
try:
|
|
||||||
value = headers.get(key.lower())
|
|
||||||
except Exception:
|
|
||||||
value = None
|
|
||||||
return str(value or "").strip()
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_single_byte_range(range_header: str, size: int) -> tuple[int, int]:
|
|
||||||
"""Parse a single HTTP byte range for signed media responses."""
|
|
||||||
if size <= 0 or "," in range_header:
|
|
||||||
raise ValueError("invalid byte range")
|
|
||||||
m = _BYTE_RANGE_RE.fullmatch(range_header.strip())
|
|
||||||
if m is None:
|
|
||||||
raise ValueError("invalid byte range")
|
|
||||||
start_text, end_text = m.groups()
|
|
||||||
if not start_text and not end_text:
|
|
||||||
raise ValueError("invalid byte range")
|
|
||||||
if not start_text:
|
|
||||||
suffix_length = int(end_text)
|
|
||||||
if suffix_length <= 0:
|
|
||||||
raise ValueError("invalid byte range")
|
|
||||||
start = max(size - suffix_length, 0)
|
|
||||||
end = size - 1
|
|
||||||
else:
|
|
||||||
start = int(start_text)
|
|
||||||
end = int(end_text) if end_text else size - 1
|
|
||||||
if start >= size or start > end:
|
|
||||||
raise ValueError("invalid byte range")
|
|
||||||
end = min(end, size - 1)
|
|
||||||
return start, end
|
|
||||||
|
|
||||||
|
|
||||||
def sign_media_path(
|
|
||||||
abs_path: Path,
|
|
||||||
*,
|
|
||||||
secret: bytes,
|
|
||||||
media_dir: MediaDirProvider = _default_media_dir,
|
|
||||||
) -> str | None:
|
|
||||||
"""Return a signed ``/api/media/<sig>/<payload>`` URL for a media-root path."""
|
|
||||||
try:
|
|
||||||
media_root = media_dir(None).resolve()
|
|
||||||
rel = abs_path.resolve().relative_to(media_root)
|
|
||||||
except (OSError, ValueError):
|
|
||||||
return None
|
|
||||||
payload = b64url_encode(rel.as_posix().encode("utf-8"))
|
|
||||||
mac = hmac.new(secret, payload.encode("ascii"), hashlib.sha256).digest()[:16]
|
|
||||||
return f"/api/media/{b64url_encode(mac)}/{payload}"
|
|
||||||
|
|
||||||
|
|
||||||
def sign_or_stage_media_path(
|
|
||||||
path: Path,
|
|
||||||
*,
|
|
||||||
secret: bytes,
|
|
||||||
media_dir: MediaDirProvider = _default_media_dir,
|
|
||||||
logger: Any | None = None,
|
|
||||||
) -> dict[str, str] | None:
|
|
||||||
"""Sign an existing media-root path, or stage an arbitrary file before signing."""
|
|
||||||
signed = sign_media_path(path, secret=secret, media_dir=media_dir)
|
|
||||||
if signed is not None:
|
|
||||||
return {"url": signed, "name": path.name}
|
|
||||||
try:
|
|
||||||
if not path.is_file():
|
|
||||||
return None
|
|
||||||
target_dir = media_dir("websocket")
|
|
||||||
safe_name = safe_filename(path.name) or "attachment"
|
|
||||||
staged = target_dir / f"{uuid.uuid4().hex[:12]}-{safe_name}"
|
|
||||||
shutil.copyfile(path, staged)
|
|
||||||
except OSError as exc:
|
|
||||||
if logger is not None:
|
|
||||||
logger.warning("failed to stage outbound media {}: {}", path, exc)
|
|
||||||
return None
|
|
||||||
signed = sign_media_path(staged, secret=secret, media_dir=media_dir)
|
|
||||||
if signed is None:
|
|
||||||
return None
|
|
||||||
return {"url": signed, "name": path.name}
|
|
||||||
|
|
||||||
|
|
||||||
def serve_signed_media(
|
|
||||||
sig: str,
|
|
||||||
payload: str,
|
|
||||||
*,
|
|
||||||
secret: bytes,
|
|
||||||
request: WsRequest | None = None,
|
|
||||||
media_dir: MediaDirProvider = _default_media_dir,
|
|
||||||
) -> Response:
|
|
||||||
"""Serve a signed media URL, including browser-friendly byte ranges."""
|
|
||||||
try:
|
|
||||||
provided_mac = b64url_decode(sig)
|
|
||||||
except (ValueError, binascii.Error):
|
|
||||||
return _http_error(401, "invalid signature")
|
|
||||||
expected_mac = hmac.new(secret, payload.encode("ascii"), hashlib.sha256).digest()[:16]
|
|
||||||
if not hmac.compare_digest(expected_mac, provided_mac):
|
|
||||||
return _http_error(401, "invalid signature")
|
|
||||||
try:
|
|
||||||
rel_bytes = b64url_decode(payload)
|
|
||||||
rel_str = rel_bytes.decode("utf-8")
|
|
||||||
except (ValueError, binascii.Error, UnicodeDecodeError):
|
|
||||||
return _http_error(400, "invalid payload")
|
|
||||||
try:
|
|
||||||
media_root = media_dir(None).resolve()
|
|
||||||
candidate = (media_root / rel_str).resolve()
|
|
||||||
candidate.relative_to(media_root)
|
|
||||||
except (OSError, ValueError):
|
|
||||||
return _http_error(404, "not found")
|
|
||||||
if not candidate.is_file():
|
|
||||||
return _http_error(404, "not found")
|
|
||||||
|
|
||||||
mime, _ = mimetypes.guess_type(candidate.name)
|
|
||||||
if mime not in _MEDIA_ALLOWED_MIMES:
|
|
||||||
mime = "application/octet-stream"
|
|
||||||
common_headers = [
|
|
||||||
("Accept-Ranges", "bytes"),
|
|
||||||
("Cache-Control", "private, max-age=31536000, immutable"),
|
|
||||||
("X-Content-Type-Options", "nosniff"),
|
|
||||||
]
|
|
||||||
if mime == "image/svg+xml":
|
|
||||||
common_headers.extend(_SVG_MEDIA_HEADERS)
|
|
||||||
try:
|
|
||||||
size = candidate.stat().st_size
|
|
||||||
except OSError:
|
|
||||||
return _http_error(500, "read error")
|
|
||||||
|
|
||||||
range_header = _case_insensitive_header(request.headers, "Range") if request else ""
|
|
||||||
if range_header:
|
|
||||||
try:
|
|
||||||
start, end = _parse_single_byte_range(range_header, size)
|
|
||||||
except ValueError:
|
|
||||||
return _http_response(
|
|
||||||
b"range not satisfiable",
|
|
||||||
status=416,
|
|
||||||
extra_headers=[
|
|
||||||
("Accept-Ranges", "bytes"),
|
|
||||||
("Content-Range", f"bytes */{size}"),
|
|
||||||
("X-Content-Type-Options", "nosniff"),
|
|
||||||
],
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
length = end - start + 1
|
|
||||||
with candidate.open("rb") as fh:
|
|
||||||
fh.seek(start)
|
|
||||||
body = fh.read(length)
|
|
||||||
except OSError:
|
|
||||||
return _http_error(500, "read error")
|
|
||||||
return _http_response(
|
|
||||||
body,
|
|
||||||
status=206,
|
|
||||||
content_type=mime,
|
|
||||||
extra_headers=[
|
|
||||||
*common_headers,
|
|
||||||
("Content-Range", f"bytes {start}-{end}/{size}"),
|
|
||||||
],
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
body = candidate.read_bytes()
|
|
||||||
except OSError:
|
|
||||||
return _http_error(500, "read error")
|
|
||||||
return _http_response(body, content_type=mime, extra_headers=common_headers)
|
|
||||||
+18
-714
@@ -6,64 +6,17 @@ settings payload shape and the allowlisted config mutations exposed to WebUI.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import os
|
from typing import Any
|
||||||
import re
|
|
||||||
import time
|
|
||||||
from contextlib import suppress
|
|
||||||
from typing import Any, Literal
|
|
||||||
from zoneinfo import ZoneInfo
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
import httpx
|
|
||||||
|
|
||||||
from nanobot.config.loader import get_config_path, load_config, save_config
|
from nanobot.config.loader import get_config_path, load_config, save_config
|
||||||
from nanobot.config.schema import ModelPresetConfig
|
|
||||||
from nanobot.providers.image_generation import (
|
from nanobot.providers.image_generation import (
|
||||||
get_image_gen_provider,
|
get_image_gen_provider,
|
||||||
image_gen_provider_names,
|
image_gen_provider_names,
|
||||||
)
|
)
|
||||||
from nanobot.providers.registry import PROVIDERS, find_by_name
|
from nanobot.providers.registry import PROVIDERS, find_by_name
|
||||||
from nanobot.security.workspace_access import workspace_sandbox_status
|
|
||||||
from nanobot.webui.workspaces import (
|
|
||||||
read_webui_default_access_mode,
|
|
||||||
write_webui_default_access_mode,
|
|
||||||
)
|
|
||||||
|
|
||||||
QueryParams = dict[str, list[str]]
|
QueryParams = dict[str, list[str]]
|
||||||
RuntimeSurface = Literal["browser", "native"]
|
|
||||||
|
|
||||||
_RUNTIME_CAPABILITIES = {
|
|
||||||
"can_restart_engine": False,
|
|
||||||
"can_pick_folder": False,
|
|
||||||
"can_open_logs": False,
|
|
||||||
"can_export_diagnostics": False,
|
|
||||||
}
|
|
||||||
|
|
||||||
_NATIVE_RUNTIME_CAPABILITIES = {
|
|
||||||
**_RUNTIME_CAPABILITIES,
|
|
||||||
"can_restart_engine": True,
|
|
||||||
"can_pick_folder": True,
|
|
||||||
"can_open_logs": True,
|
|
||||||
"can_export_diagnostics": True,
|
|
||||||
}
|
|
||||||
|
|
||||||
_BROWSER_RESTART_BEHAVIOR_BY_SECTION = {
|
|
||||||
"appearance": "none",
|
|
||||||
"models": "none",
|
|
||||||
"providers": "none",
|
|
||||||
"runtime": "engineRestart",
|
|
||||||
"browser": "engineRestart",
|
|
||||||
"image": "engineRestart",
|
|
||||||
"apps": "engineRestart",
|
|
||||||
"advanced": "appRestart",
|
|
||||||
}
|
|
||||||
|
|
||||||
_NATIVE_RESTART_BEHAVIOR_BY_SECTION = {
|
|
||||||
**_BROWSER_RESTART_BEHAVIOR_BY_SECTION,
|
|
||||||
"runtime": "engineRestart",
|
|
||||||
"browser": "engineRestart",
|
|
||||||
"image": "engineRestart",
|
|
||||||
"apps": "engineRestart",
|
|
||||||
}
|
|
||||||
|
|
||||||
_WEB_SEARCH_PROVIDER_OPTIONS: tuple[dict[str, str], ...] = (
|
_WEB_SEARCH_PROVIDER_OPTIONS: tuple[dict[str, str], ...] = (
|
||||||
{"name": "duckduckgo", "label": "DuckDuckGo", "credential": "none"},
|
{"name": "duckduckgo", "label": "DuckDuckGo", "credential": "none"},
|
||||||
@@ -88,49 +41,6 @@ _IMAGE_GENERATION_ASPECT_RATIOS = {
|
|||||||
"2:3",
|
"2:3",
|
||||||
"21:9",
|
"21:9",
|
||||||
}
|
}
|
||||||
_CONTEXT_WINDOW_TOKEN_OPTIONS = {65_536, 262_144}
|
|
||||||
_MODEL_CONFIGURATION_SLUG_RE = re.compile(r"[^a-z0-9_-]+")
|
|
||||||
_ENV_REF_RE = re.compile(r"\$\{([A-Za-z_][A-Za-z0-9_]*)\}")
|
|
||||||
|
|
||||||
_MODEL_LIST_UNSUPPORTED_BACKENDS = {
|
|
||||||
"anthropic",
|
|
||||||
"azure_openai",
|
|
||||||
"bedrock",
|
|
||||||
"github_copilot",
|
|
||||||
"openai_codex",
|
|
||||||
}
|
|
||||||
|
|
||||||
_MODEL_LIST_CATALOG_PROVIDERS = {
|
|
||||||
"aihubmix",
|
|
||||||
"byteplus",
|
|
||||||
"byteplus_coding_plan",
|
|
||||||
"huggingface",
|
|
||||||
"novita",
|
|
||||||
"openrouter",
|
|
||||||
"siliconflow",
|
|
||||||
"volcengine",
|
|
||||||
"volcengine_coding_plan",
|
|
||||||
}
|
|
||||||
|
|
||||||
_MODEL_LIST_OFFICIAL_PROVIDERS = {
|
|
||||||
"ant_ling",
|
|
||||||
"dashscope",
|
|
||||||
"deepseek",
|
|
||||||
"gemini",
|
|
||||||
"groq",
|
|
||||||
"longcat",
|
|
||||||
"minimax",
|
|
||||||
"minimax_anthropic",
|
|
||||||
"mistral",
|
|
||||||
"moonshot",
|
|
||||||
"nvidia",
|
|
||||||
"openai",
|
|
||||||
"qianfan",
|
|
||||||
"skywork",
|
|
||||||
"stepfun",
|
|
||||||
"xiaomi_mimo",
|
|
||||||
"zhipu",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class WebUISettingsError(ValueError):
|
class WebUISettingsError(ValueError):
|
||||||
@@ -142,70 +52,6 @@ class WebUISettingsError(ValueError):
|
|||||||
self.status = status
|
self.status = status
|
||||||
|
|
||||||
|
|
||||||
def _normalize_surface(surface: str | None) -> RuntimeSurface:
|
|
||||||
return "native" if surface in {"native", "desktop"} else "browser"
|
|
||||||
|
|
||||||
|
|
||||||
def runtime_capabilities(
|
|
||||||
surface: str | None = "browser",
|
|
||||||
overrides: dict[str, Any] | None = None,
|
|
||||||
) -> dict[str, bool]:
|
|
||||||
"""Return the capability flags exposed to the WebUI runtime."""
|
|
||||||
base = (
|
|
||||||
_NATIVE_RUNTIME_CAPABILITIES
|
|
||||||
if _normalize_surface(surface) == "native"
|
|
||||||
else _RUNTIME_CAPABILITIES
|
|
||||||
)
|
|
||||||
result = dict(base)
|
|
||||||
for key, value in (overrides or {}).items():
|
|
||||||
if key in result:
|
|
||||||
result[key] = bool(value)
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def restart_behavior_by_section(surface: str | None = "browser") -> dict[str, str]:
|
|
||||||
return dict(
|
|
||||||
_NATIVE_RESTART_BEHAVIOR_BY_SECTION
|
|
||||||
if _normalize_surface(surface) == "native"
|
|
||||||
else _BROWSER_RESTART_BEHAVIOR_BY_SECTION
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def decorate_settings_payload(
|
|
||||||
payload: dict[str, Any],
|
|
||||||
*,
|
|
||||||
surface: str | None = "browser",
|
|
||||||
runtime_capability_overrides: dict[str, Any] | None = None,
|
|
||||||
restart_required_sections: list[str] | None = None,
|
|
||||||
apply_state: dict[str, Any] | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Attach runtime-surface metadata without changing the core settings shape."""
|
|
||||||
surface_value = _normalize_surface(surface)
|
|
||||||
sections = restart_required_sections
|
|
||||||
if sections is None:
|
|
||||||
raw_sections = payload.get("restart_required_sections") or []
|
|
||||||
sections = [str(section) for section in raw_sections if isinstance(section, str)]
|
|
||||||
sections = sorted(dict.fromkeys(sections))
|
|
||||||
result = dict(payload)
|
|
||||||
result["surface"] = surface_value
|
|
||||||
result["runtime_surface"] = surface_value
|
|
||||||
result["runtime_capabilities"] = runtime_capabilities(
|
|
||||||
surface_value,
|
|
||||||
runtime_capability_overrides,
|
|
||||||
)
|
|
||||||
result["restart_behavior_by_section"] = restart_behavior_by_section(surface_value)
|
|
||||||
result["restart_required_sections"] = sections
|
|
||||||
if sections:
|
|
||||||
result["requires_restart"] = True
|
|
||||||
else:
|
|
||||||
result["requires_restart"] = bool(result.get("requires_restart", False))
|
|
||||||
result["apply_state"] = apply_state or {
|
|
||||||
"status": "pending" if result["requires_restart"] else "idle",
|
|
||||||
"sections": sections,
|
|
||||||
}
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def _query_first(query: QueryParams, key: str) -> str | None:
|
def _query_first(query: QueryParams, key: str) -> str | None:
|
||||||
values = query.get(key)
|
values = query.get(key)
|
||||||
return values[0] if values else None
|
return values[0] if values else None
|
||||||
@@ -224,86 +70,15 @@ def _mask_secret_hint(secret: str | None) -> str | None:
|
|||||||
return f"{secret[:4]}••••{secret[-4:]}"
|
return f"{secret[:4]}••••{secret[-4:]}"
|
||||||
|
|
||||||
|
|
||||||
def _resolve_env_placeholders(value: str | None) -> str | None:
|
|
||||||
if not value:
|
|
||||||
return None
|
|
||||||
missing = False
|
|
||||||
|
|
||||||
def replace(match: re.Match[str]) -> str:
|
|
||||||
nonlocal missing
|
|
||||||
env_value = os.environ.get(match.group(1))
|
|
||||||
if env_value is None:
|
|
||||||
missing = True
|
|
||||||
return ""
|
|
||||||
return env_value
|
|
||||||
|
|
||||||
resolved = _ENV_REF_RE.sub(replace, value).strip()
|
|
||||||
if missing and not resolved:
|
|
||||||
return None
|
|
||||||
return resolved or None
|
|
||||||
|
|
||||||
|
|
||||||
def _provider_requires_api_key(spec: Any) -> bool:
|
def _provider_requires_api_key(spec: Any) -> bool:
|
||||||
if spec.backend == "azure_openai":
|
if spec.backend == "azure_openai":
|
||||||
return True
|
return True
|
||||||
if spec.is_oauth:
|
|
||||||
return False
|
|
||||||
if spec.is_local or spec.is_direct:
|
if spec.is_local or spec.is_direct:
|
||||||
return False
|
return False
|
||||||
return True
|
return True
|
||||||
|
|
||||||
|
|
||||||
def _oauth_provider_status(spec: Any) -> dict[str, Any]:
|
|
||||||
if not getattr(spec, "is_oauth", False):
|
|
||||||
return {"configured": False, "account": None, "expires_at": None, "login_supported": False}
|
|
||||||
|
|
||||||
if spec.name == "openai_codex":
|
|
||||||
try:
|
|
||||||
from oauth_cli_kit import get_token as get_codex_token
|
|
||||||
except Exception:
|
|
||||||
return {
|
|
||||||
"configured": False,
|
|
||||||
"account": None,
|
|
||||||
"expires_at": None,
|
|
||||||
"login_supported": False,
|
|
||||||
}
|
|
||||||
token = None
|
|
||||||
with suppress(Exception):
|
|
||||||
token = get_codex_token()
|
|
||||||
expires_at = getattr(token, "expires", None) if token else None
|
|
||||||
return {
|
|
||||||
"configured": bool(token and token.access),
|
|
||||||
"account": getattr(token, "account_id", None) if token else None,
|
|
||||||
"expires_at": expires_at,
|
|
||||||
"login_supported": True,
|
|
||||||
}
|
|
||||||
|
|
||||||
if spec.name == "github_copilot":
|
|
||||||
try:
|
|
||||||
from nanobot.providers.github_copilot_provider import get_github_copilot_login_status
|
|
||||||
except Exception:
|
|
||||||
return {
|
|
||||||
"configured": False,
|
|
||||||
"account": None,
|
|
||||||
"expires_at": None,
|
|
||||||
"login_supported": False,
|
|
||||||
}
|
|
||||||
token = None
|
|
||||||
with suppress(Exception):
|
|
||||||
token = get_github_copilot_login_status()
|
|
||||||
return {
|
|
||||||
"configured": bool(token and token.access and token.expires > int(time.time() * 1000)),
|
|
||||||
"account": getattr(token, "account_id", None) if token else None,
|
|
||||||
"expires_at": getattr(token, "expires", None) if token else None,
|
|
||||||
"login_supported": True,
|
|
||||||
}
|
|
||||||
|
|
||||||
return {"configured": False, "account": None, "expires_at": None, "login_supported": False}
|
|
||||||
|
|
||||||
|
|
||||||
def _provider_configured_for_settings(spec: Any, provider_config: Any) -> bool:
|
def _provider_configured_for_settings(spec: Any, provider_config: Any) -> bool:
|
||||||
if spec.is_oauth:
|
|
||||||
return bool(_oauth_provider_status(spec)["configured"])
|
|
||||||
if _provider_requires_api_key(spec):
|
if _provider_requires_api_key(spec):
|
||||||
return bool(provider_config.api_key)
|
return bool(provider_config.api_key)
|
||||||
return bool(
|
return bool(
|
||||||
@@ -314,191 +89,6 @@ def _provider_configured_for_settings(spec: Any, provider_config: Any) -> bool:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _model_catalog_kind(spec: Any) -> str:
|
|
||||||
if spec.name in _MODEL_LIST_CATALOG_PROVIDERS:
|
|
||||||
return "catalog"
|
|
||||||
if spec.name in _MODEL_LIST_OFFICIAL_PROVIDERS:
|
|
||||||
return "official"
|
|
||||||
if spec.is_local:
|
|
||||||
return "local"
|
|
||||||
if spec.is_direct:
|
|
||||||
return "custom"
|
|
||||||
if spec.is_gateway:
|
|
||||||
return "catalog"
|
|
||||||
return "official"
|
|
||||||
|
|
||||||
|
|
||||||
def _model_id_from_row(row: Any) -> str | None:
|
|
||||||
if isinstance(row, str):
|
|
||||||
return row.strip() or None
|
|
||||||
if not isinstance(row, dict):
|
|
||||||
return None
|
|
||||||
for key in ("id", "name", "model"):
|
|
||||||
value = row.get(key)
|
|
||||||
if isinstance(value, str) and value.strip():
|
|
||||||
return value.strip()
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _model_context_window(row: Any) -> int | None:
|
|
||||||
if not isinstance(row, dict):
|
|
||||||
return None
|
|
||||||
for key in (
|
|
||||||
"context_window",
|
|
||||||
"context_length",
|
|
||||||
"max_context_length",
|
|
||||||
"max_model_len",
|
|
||||||
"max_input_tokens",
|
|
||||||
):
|
|
||||||
value = row.get(key)
|
|
||||||
if isinstance(value, int) and value > 0:
|
|
||||||
return value
|
|
||||||
if isinstance(value, float) and value > 0:
|
|
||||||
return int(value)
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _model_row_payload(row: Any) -> dict[str, Any] | None:
|
|
||||||
model_id = _model_id_from_row(row)
|
|
||||||
if not model_id:
|
|
||||||
return None
|
|
||||||
label: str | None = None
|
|
||||||
owned_by: str | None = None
|
|
||||||
if isinstance(row, dict):
|
|
||||||
raw_label = row.get("display_name") or row.get("label") or row.get("name")
|
|
||||||
if isinstance(raw_label, str) and raw_label.strip() and raw_label.strip() != model_id:
|
|
||||||
label = raw_label.strip()
|
|
||||||
raw_owner = row.get("owned_by") or row.get("owner") or row.get("organization")
|
|
||||||
if isinstance(raw_owner, str) and raw_owner.strip():
|
|
||||||
owned_by = raw_owner.strip()
|
|
||||||
return {
|
|
||||||
"id": model_id,
|
|
||||||
"label": label,
|
|
||||||
"owned_by": owned_by,
|
|
||||||
"context_window": _model_context_window(row),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _extract_model_rows(body: Any) -> list[dict[str, Any]]:
|
|
||||||
raw_rows = body.get("data") if isinstance(body, dict) else body
|
|
||||||
if not isinstance(raw_rows, list):
|
|
||||||
return []
|
|
||||||
rows: list[dict[str, Any]] = []
|
|
||||||
seen: set[str] = set()
|
|
||||||
for raw_row in raw_rows:
|
|
||||||
row = _model_row_payload(raw_row)
|
|
||||||
if row is None or row["id"] in seen:
|
|
||||||
continue
|
|
||||||
seen.add(row["id"])
|
|
||||||
rows.append(row)
|
|
||||||
return rows
|
|
||||||
|
|
||||||
|
|
||||||
def provider_models_payload(query: QueryParams) -> dict[str, Any]:
|
|
||||||
"""Fetch an OpenAI-compatible provider's model list for Settings.
|
|
||||||
|
|
||||||
The result is advisory only: users can always type a custom model id. This
|
|
||||||
helper deliberately avoids mutating config so probing model lists never
|
|
||||||
changes runtime behavior.
|
|
||||||
"""
|
|
||||||
provider_name = (_query_first(query, "provider") or "").strip()
|
|
||||||
if not provider_name:
|
|
||||||
raise WebUISettingsError("provider is required")
|
|
||||||
spec = find_by_name(provider_name)
|
|
||||||
if spec is None:
|
|
||||||
raise WebUISettingsError("unknown provider")
|
|
||||||
|
|
||||||
base_payload: dict[str, Any] = {
|
|
||||||
"provider": spec.name,
|
|
||||||
"label": spec.label,
|
|
||||||
"catalog_kind": _model_catalog_kind(spec),
|
|
||||||
"models": [],
|
|
||||||
"model_count": 0,
|
|
||||||
"message": None,
|
|
||||||
"fetched_at": time.time(),
|
|
||||||
}
|
|
||||||
if (
|
|
||||||
spec.backend in _MODEL_LIST_UNSUPPORTED_BACKENDS
|
|
||||||
and spec.name != "minimax_anthropic"
|
|
||||||
) or spec.is_oauth:
|
|
||||||
return {
|
|
||||||
**base_payload,
|
|
||||||
"status": "unsupported",
|
|
||||||
"catalog_kind": "unsupported",
|
|
||||||
"message": "Model list is not available for this provider. Type a model ID manually.",
|
|
||||||
}
|
|
||||||
|
|
||||||
config = load_config()
|
|
||||||
provider_config = getattr(config.providers, spec.name, None)
|
|
||||||
if provider_config is None:
|
|
||||||
raise WebUISettingsError("unknown provider")
|
|
||||||
|
|
||||||
api_base = _resolve_env_placeholders(provider_config.api_base) or spec.default_api_base
|
|
||||||
if spec.name == "openai" and not api_base:
|
|
||||||
api_base = "https://api.openai.com/v1"
|
|
||||||
if not api_base:
|
|
||||||
return {
|
|
||||||
**base_payload,
|
|
||||||
"status": "missing_api_base",
|
|
||||||
"message": "Configure an API base URL to load models.",
|
|
||||||
}
|
|
||||||
|
|
||||||
api_key = _resolve_env_placeholders(provider_config.api_key)
|
|
||||||
if _provider_requires_api_key(spec) and not api_key:
|
|
||||||
return {
|
|
||||||
**base_payload,
|
|
||||||
"status": "not_configured",
|
|
||||||
"message": "Configure this provider before loading models.",
|
|
||||||
}
|
|
||||||
|
|
||||||
headers = {"Accept": "application/json"}
|
|
||||||
if api_key:
|
|
||||||
if spec.name == "minimax_anthropic":
|
|
||||||
headers["X-Api-Key"] = api_key
|
|
||||||
else:
|
|
||||||
headers["Authorization"] = f"Bearer {api_key}"
|
|
||||||
|
|
||||||
models_url = f"{api_base.rstrip('/')}/models"
|
|
||||||
if spec.name == "minimax_anthropic" and not api_base.rstrip("/").endswith("/v1"):
|
|
||||||
models_url = f"{api_base.rstrip('/')}/v1/models"
|
|
||||||
|
|
||||||
try:
|
|
||||||
response = httpx.get(
|
|
||||||
models_url,
|
|
||||||
headers=headers,
|
|
||||||
timeout=10.0,
|
|
||||||
follow_redirects=False,
|
|
||||||
)
|
|
||||||
response.raise_for_status()
|
|
||||||
rows = _extract_model_rows(response.json())
|
|
||||||
except httpx.HTTPStatusError as exc:
|
|
||||||
status = exc.response.status_code
|
|
||||||
if status in {401, 403}:
|
|
||||||
return {
|
|
||||||
**base_payload,
|
|
||||||
"status": "not_configured",
|
|
||||||
"message": "The provider rejected the configured credential.",
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
**base_payload,
|
|
||||||
"status": "error",
|
|
||||||
"message": f"Model list request failed with HTTP {status}.",
|
|
||||||
}
|
|
||||||
except (httpx.HTTPError, ValueError) as exc:
|
|
||||||
return {
|
|
||||||
**base_payload,
|
|
||||||
"status": "error",
|
|
||||||
"message": f"Could not load models: {exc}",
|
|
||||||
}
|
|
||||||
|
|
||||||
return {
|
|
||||||
**base_payload,
|
|
||||||
"status": "available",
|
|
||||||
"models": rows,
|
|
||||||
"model_count": len(rows),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_bool(value: str, field: str) -> bool:
|
def _parse_bool(value: str, field: str) -> bool:
|
||||||
normalized = value.strip().lower()
|
normalized = value.strip().lower()
|
||||||
if normalized not in {"1", "0", "true", "false", "yes", "no"}:
|
if normalized not in {"1", "0", "true", "false", "yes", "no"}:
|
||||||
@@ -506,44 +96,6 @@ def _parse_bool(value: str, field: str) -> bool:
|
|||||||
return normalized in {"1", "true", "yes"}
|
return normalized in {"1", "true", "yes"}
|
||||||
|
|
||||||
|
|
||||||
def _parse_context_window_tokens(value: str | None) -> int | None:
|
|
||||||
if value is None:
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
parsed = int(value)
|
|
||||||
except ValueError:
|
|
||||||
raise WebUISettingsError("context_window_tokens must be an integer") from None
|
|
||||||
if parsed not in _CONTEXT_WINDOW_TOKEN_OPTIONS:
|
|
||||||
raise WebUISettingsError("context_window_tokens must be 65536 or 262144")
|
|
||||||
return parsed
|
|
||||||
|
|
||||||
|
|
||||||
def _model_configuration_slug(label: str) -> str:
|
|
||||||
normalized = _MODEL_CONFIGURATION_SLUG_RE.sub("-", label.strip().lower())
|
|
||||||
normalized = normalized.strip("-_")
|
|
||||||
if not normalized:
|
|
||||||
raise WebUISettingsError("configuration name is required")
|
|
||||||
if normalized == "default":
|
|
||||||
raise WebUISettingsError("configuration name is reserved")
|
|
||||||
if len(normalized) > 48:
|
|
||||||
normalized = normalized[:48].rstrip("-_")
|
|
||||||
return normalized
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_configured_provider(config: Any, provider: str) -> None:
|
|
||||||
if provider == "auto":
|
|
||||||
return
|
|
||||||
spec = find_by_name(provider)
|
|
||||||
if spec is None:
|
|
||||||
raise WebUISettingsError("unknown provider")
|
|
||||||
provider_config = getattr(config.providers, provider, None)
|
|
||||||
if (
|
|
||||||
provider_config is None
|
|
||||||
or not _provider_configured_for_settings(spec, provider_config)
|
|
||||||
):
|
|
||||||
raise WebUISettingsError("provider is not configured")
|
|
||||||
|
|
||||||
|
|
||||||
def _image_generation_provider_rows(config: Any) -> list[dict[str, Any]]:
|
def _image_generation_provider_rows(config: Any) -> list[dict[str, Any]]:
|
||||||
rows: list[dict[str, Any]] = []
|
rows: list[dict[str, Any]] = []
|
||||||
for name in image_gen_provider_names():
|
for name in image_gen_provider_names():
|
||||||
@@ -559,7 +111,6 @@ def _image_generation_provider_rows(config: Any) -> list[dict[str, Any]]:
|
|||||||
"name": name,
|
"name": name,
|
||||||
"label": spec.label if spec is not None else name,
|
"label": spec.label if spec is not None else name,
|
||||||
"configured": configured,
|
"configured": configured,
|
||||||
"auth_type": "oauth" if spec is not None and spec.is_oauth else "api_key",
|
|
||||||
"api_key_hint": _mask_secret_hint(
|
"api_key_hint": _mask_secret_hint(
|
||||||
getattr(provider_config, "api_key", None)
|
getattr(provider_config, "api_key", None)
|
||||||
),
|
),
|
||||||
@@ -572,14 +123,7 @@ def _image_generation_provider_rows(config: Any) -> list[dict[str, Any]]:
|
|||||||
return rows
|
return rows
|
||||||
|
|
||||||
|
|
||||||
def settings_payload(
|
def settings_payload(*, requires_restart: bool = False) -> dict[str, Any]:
|
||||||
*,
|
|
||||||
requires_restart: bool = False,
|
|
||||||
surface: str | None = "browser",
|
|
||||||
runtime_capability_overrides: dict[str, Any] | None = None,
|
|
||||||
restart_required_sections: list[str] | None = None,
|
|
||||||
apply_state: dict[str, Any] | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
config = load_config()
|
config = load_config()
|
||||||
defaults = config.agents.defaults
|
defaults = config.agents.defaults
|
||||||
active_preset_name = defaults.model_preset or "default"
|
active_preset_name = defaults.model_preset or "default"
|
||||||
@@ -602,30 +146,19 @@ def settings_payload(
|
|||||||
providers = []
|
providers = []
|
||||||
for spec in PROVIDERS:
|
for spec in PROVIDERS:
|
||||||
provider_config = getattr(config.providers, spec.name, None)
|
provider_config = getattr(config.providers, spec.name, None)
|
||||||
if provider_config is None:
|
if provider_config is None or spec.is_oauth:
|
||||||
continue
|
continue
|
||||||
oauth_status = _oauth_provider_status(spec) if spec.is_oauth else None
|
providers.append(
|
||||||
row = {
|
{
|
||||||
"name": spec.name,
|
"name": spec.name,
|
||||||
"label": spec.label,
|
"label": spec.label,
|
||||||
"configured": (
|
"configured": _provider_configured_for_settings(spec, provider_config),
|
||||||
bool(oauth_status["configured"])
|
|
||||||
if oauth_status is not None
|
|
||||||
else _provider_configured_for_settings(spec, provider_config)
|
|
||||||
),
|
|
||||||
"auth_type": "oauth" if spec.is_oauth else "api_key",
|
|
||||||
"api_key_required": _provider_requires_api_key(spec),
|
"api_key_required": _provider_requires_api_key(spec),
|
||||||
"api_key_hint": _mask_secret_hint(provider_config.api_key),
|
"api_key_hint": _mask_secret_hint(provider_config.api_key),
|
||||||
"api_base": provider_config.api_base,
|
"api_base": provider_config.api_base,
|
||||||
"default_api_base": spec.default_api_base or None,
|
"default_api_base": spec.default_api_base or None,
|
||||||
}
|
}
|
||||||
if oauth_status is not None:
|
)
|
||||||
row["oauth_account"] = oauth_status["account"]
|
|
||||||
row["oauth_expires_at"] = oauth_status["expires_at"]
|
|
||||||
row["oauth_login_supported"] = oauth_status["login_supported"]
|
|
||||||
if spec.name == "openai":
|
|
||||||
row["api_type"] = provider_config.api_type
|
|
||||||
providers.append(row)
|
|
||||||
|
|
||||||
search_config = config.tools.web.search
|
search_config = config.tools.web.search
|
||||||
image_config = config.tools.image_generation
|
image_config = config.tools.image_generation
|
||||||
@@ -661,7 +194,7 @@ def settings_payload(
|
|||||||
model_presets.append(
|
model_presets.append(
|
||||||
{
|
{
|
||||||
"name": name,
|
"name": name,
|
||||||
"label": preset.label or name,
|
"label": name,
|
||||||
"active": active_preset_name == name,
|
"active": active_preset_name == name,
|
||||||
"is_default": False,
|
"is_default": False,
|
||||||
"model": preset.model,
|
"model": preset.model,
|
||||||
@@ -674,11 +207,7 @@ def settings_payload(
|
|||||||
)
|
)
|
||||||
|
|
||||||
exec_config = config.tools.exec
|
exec_config = config.tools.exec
|
||||||
sandbox_status = workspace_sandbox_status(
|
return {
|
||||||
restrict_to_workspace=config.tools.restrict_to_workspace,
|
|
||||||
workspace=config.workspace_path,
|
|
||||||
)
|
|
||||||
payload = {
|
|
||||||
"agent": {
|
"agent": {
|
||||||
"model": effective_preset.model,
|
"model": effective_preset.model,
|
||||||
"provider": selected_provider,
|
"provider": selected_provider,
|
||||||
@@ -749,11 +278,6 @@ def settings_payload(
|
|||||||
},
|
},
|
||||||
"advanced": {
|
"advanced": {
|
||||||
"restrict_to_workspace": config.tools.restrict_to_workspace,
|
"restrict_to_workspace": config.tools.restrict_to_workspace,
|
||||||
"workspace_sandbox": sandbox_status.as_dict(),
|
|
||||||
"webui_allow_local_service_access": config.tools.webui_allow_local_service_access,
|
|
||||||
"allow_local_preview_access": config.tools.webui_allow_local_service_access,
|
|
||||||
"webui_default_access_mode": read_webui_default_access_mode(),
|
|
||||||
"private_service_protection_enabled": True,
|
|
||||||
"ssrf_whitelist_count": len(config.tools.ssrf_whitelist),
|
"ssrf_whitelist_count": len(config.tools.ssrf_whitelist),
|
||||||
"mcp_server_count": len(config.tools.mcp_servers),
|
"mcp_server_count": len(config.tools.mcp_servers),
|
||||||
"exec_enabled": exec_config.enable,
|
"exec_enabled": exec_config.enable,
|
||||||
@@ -762,13 +286,6 @@ def settings_payload(
|
|||||||
},
|
},
|
||||||
"requires_restart": requires_restart,
|
"requires_restart": requires_restart,
|
||||||
}
|
}
|
||||||
return decorate_settings_payload(
|
|
||||||
payload,
|
|
||||||
surface=surface,
|
|
||||||
runtime_capability_overrides=runtime_capability_overrides,
|
|
||||||
restart_required_sections=restart_required_sections,
|
|
||||||
apply_state=apply_state,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def update_agent_settings(query: QueryParams) -> dict[str, Any]:
|
def update_agent_settings(query: QueryParams) -> dict[str, Any]:
|
||||||
@@ -800,21 +317,19 @@ def update_agent_settings(query: QueryParams) -> dict[str, Any]:
|
|||||||
provider = provider.strip()
|
provider = provider.strip()
|
||||||
if not provider:
|
if not provider:
|
||||||
raise WebUISettingsError("provider is required")
|
raise WebUISettingsError("provider is required")
|
||||||
_validate_configured_provider(config, provider)
|
spec = find_by_name(provider)
|
||||||
|
if spec is None:
|
||||||
|
raise WebUISettingsError("unknown provider")
|
||||||
|
provider_config = getattr(config.providers, provider, None)
|
||||||
|
if (
|
||||||
|
provider_config is None
|
||||||
|
or not _provider_configured_for_settings(spec, provider_config)
|
||||||
|
):
|
||||||
|
raise WebUISettingsError("provider is not configured")
|
||||||
if defaults.provider != provider:
|
if defaults.provider != provider:
|
||||||
defaults.provider = provider
|
defaults.provider = provider
|
||||||
changed = True
|
changed = True
|
||||||
|
|
||||||
context_window_tokens = _parse_context_window_tokens(
|
|
||||||
_query_first_alias(query, "context_window_tokens", "contextWindowTokens")
|
|
||||||
)
|
|
||||||
if (
|
|
||||||
context_window_tokens is not None
|
|
||||||
and defaults.context_window_tokens != context_window_tokens
|
|
||||||
):
|
|
||||||
defaults.context_window_tokens = context_window_tokens
|
|
||||||
changed = True
|
|
||||||
|
|
||||||
timezone = _query_first(query, "timezone")
|
timezone = _query_first(query, "timezone")
|
||||||
if timezone is not None:
|
if timezone is not None:
|
||||||
timezone = timezone.strip()
|
timezone = timezone.strip()
|
||||||
@@ -869,98 +384,6 @@ def update_agent_settings(query: QueryParams) -> dict[str, Any]:
|
|||||||
return settings_payload(requires_restart=restart_required)
|
return settings_payload(requires_restart=restart_required)
|
||||||
|
|
||||||
|
|
||||||
def create_model_configuration(query: QueryParams) -> dict[str, Any]:
|
|
||||||
label = (_query_first_alias(query, "label", "displayName") or "").strip()
|
|
||||||
raw_name = (_query_first(query, "name") or label).strip()
|
|
||||||
model = (_query_first(query, "model") or "").strip()
|
|
||||||
provider = (_query_first(query, "provider") or "").strip()
|
|
||||||
|
|
||||||
if not label:
|
|
||||||
label = raw_name
|
|
||||||
if not model:
|
|
||||||
raise WebUISettingsError("model is required")
|
|
||||||
if not provider:
|
|
||||||
raise WebUISettingsError("provider is required")
|
|
||||||
|
|
||||||
name = _model_configuration_slug(raw_name or label)
|
|
||||||
config = load_config()
|
|
||||||
if name in config.model_presets:
|
|
||||||
raise WebUISettingsError("configuration already exists", status=409)
|
|
||||||
_validate_configured_provider(config, provider)
|
|
||||||
|
|
||||||
base = config.resolve_default_preset()
|
|
||||||
config.model_presets[name] = ModelPresetConfig(
|
|
||||||
label=label,
|
|
||||||
model=model,
|
|
||||||
provider=provider,
|
|
||||||
max_tokens=base.max_tokens,
|
|
||||||
context_window_tokens=base.context_window_tokens,
|
|
||||||
temperature=base.temperature,
|
|
||||||
reasoning_effort=base.reasoning_effort,
|
|
||||||
)
|
|
||||||
config.agents.defaults.model_preset = name
|
|
||||||
save_config(config)
|
|
||||||
return settings_payload()
|
|
||||||
|
|
||||||
|
|
||||||
def update_model_configuration(query: QueryParams) -> dict[str, Any]:
|
|
||||||
name = (_query_first(query, "name") or "").strip()
|
|
||||||
if not name or name == "default":
|
|
||||||
raise WebUISettingsError("model configuration is required")
|
|
||||||
|
|
||||||
config = load_config()
|
|
||||||
preset = config.model_presets.get(name)
|
|
||||||
if preset is None:
|
|
||||||
raise WebUISettingsError("unknown model configuration")
|
|
||||||
|
|
||||||
changed = False
|
|
||||||
label = _query_first_alias(query, "label", "displayName")
|
|
||||||
if label is not None:
|
|
||||||
label = label.strip()
|
|
||||||
if not label:
|
|
||||||
raise WebUISettingsError("label is required")
|
|
||||||
if preset.label != label:
|
|
||||||
preset.label = label
|
|
||||||
changed = True
|
|
||||||
|
|
||||||
model = _query_first(query, "model")
|
|
||||||
if model is not None:
|
|
||||||
model = model.strip()
|
|
||||||
if not model:
|
|
||||||
raise WebUISettingsError("model is required")
|
|
||||||
if preset.model != model:
|
|
||||||
preset.model = model
|
|
||||||
changed = True
|
|
||||||
|
|
||||||
provider = _query_first(query, "provider")
|
|
||||||
if provider is not None:
|
|
||||||
provider = provider.strip()
|
|
||||||
if not provider:
|
|
||||||
raise WebUISettingsError("provider is required")
|
|
||||||
_validate_configured_provider(config, provider)
|
|
||||||
if preset.provider != provider:
|
|
||||||
preset.provider = provider
|
|
||||||
changed = True
|
|
||||||
|
|
||||||
context_window_tokens = _parse_context_window_tokens(
|
|
||||||
_query_first_alias(query, "context_window_tokens", "contextWindowTokens")
|
|
||||||
)
|
|
||||||
if (
|
|
||||||
context_window_tokens is not None
|
|
||||||
and preset.context_window_tokens != context_window_tokens
|
|
||||||
):
|
|
||||||
preset.context_window_tokens = context_window_tokens
|
|
||||||
changed = True
|
|
||||||
|
|
||||||
if config.agents.defaults.model_preset != name:
|
|
||||||
config.agents.defaults.model_preset = name
|
|
||||||
changed = True
|
|
||||||
|
|
||||||
if changed:
|
|
||||||
save_config(config)
|
|
||||||
return settings_payload()
|
|
||||||
|
|
||||||
|
|
||||||
def update_provider_settings(query: QueryParams) -> dict[str, Any]:
|
def update_provider_settings(query: QueryParams) -> dict[str, Any]:
|
||||||
provider_name = (_query_first(query, "provider") or "").strip()
|
provider_name = (_query_first(query, "provider") or "").strip()
|
||||||
if not provider_name:
|
if not provider_name:
|
||||||
@@ -989,17 +412,6 @@ def update_provider_settings(query: QueryParams) -> dict[str, Any]:
|
|||||||
provider_config.api_base = api_base
|
provider_config.api_base = api_base
|
||||||
changed = True
|
changed = True
|
||||||
|
|
||||||
if "api_type" in query:
|
|
||||||
if spec.name == "openai":
|
|
||||||
api_type = (_query_first(query, "api_type") or "").strip()
|
|
||||||
try:
|
|
||||||
parsed_api_type = type(provider_config)(api_type=api_type).api_type
|
|
||||||
except Exception:
|
|
||||||
raise WebUISettingsError("api_type must be auto, chat_completions, or responses") from None
|
|
||||||
if provider_config.api_type != parsed_api_type:
|
|
||||||
provider_config.api_type = parsed_api_type
|
|
||||||
changed = True
|
|
||||||
|
|
||||||
if changed:
|
if changed:
|
||||||
save_config(config)
|
save_config(config)
|
||||||
image_config = config.tools.image_generation
|
image_config = config.tools.image_generation
|
||||||
@@ -1012,114 +424,6 @@ def update_provider_settings(query: QueryParams) -> dict[str, Any]:
|
|||||||
return settings_payload(requires_restart=restart_required)
|
return settings_payload(requires_restart=restart_required)
|
||||||
|
|
||||||
|
|
||||||
def login_oauth_provider(query: QueryParams) -> dict[str, Any]:
|
|
||||||
provider_name = (_query_first(query, "provider") or "").strip()
|
|
||||||
if not provider_name:
|
|
||||||
raise WebUISettingsError("provider is required")
|
|
||||||
spec = find_by_name(provider_name)
|
|
||||||
if spec is None or not spec.is_oauth:
|
|
||||||
raise WebUISettingsError("unknown OAuth provider")
|
|
||||||
|
|
||||||
if spec.name == "openai_codex":
|
|
||||||
try:
|
|
||||||
from oauth_cli_kit import get_token, login_oauth_interactive
|
|
||||||
except ImportError:
|
|
||||||
raise WebUISettingsError("oauth_cli_kit is not installed", status=500) from None
|
|
||||||
|
|
||||||
token = None
|
|
||||||
with suppress(Exception):
|
|
||||||
token = get_token()
|
|
||||||
if not (token and token.access):
|
|
||||||
messages: list[str] = []
|
|
||||||
token = login_oauth_interactive(
|
|
||||||
print_fn=lambda message: messages.append(str(message)),
|
|
||||||
prompt_fn=lambda _prompt: "",
|
|
||||||
)
|
|
||||||
if not (token and token.access):
|
|
||||||
raise WebUISettingsError("OAuth login failed", status=401)
|
|
||||||
return settings_payload()
|
|
||||||
|
|
||||||
if spec.name == "github_copilot":
|
|
||||||
try:
|
|
||||||
from nanobot.providers.github_copilot_provider import (
|
|
||||||
get_github_copilot_login_status,
|
|
||||||
login_github_copilot,
|
|
||||||
)
|
|
||||||
except ImportError:
|
|
||||||
raise WebUISettingsError("GitHub Copilot OAuth support is unavailable", status=500) from None
|
|
||||||
|
|
||||||
token = get_github_copilot_login_status()
|
|
||||||
if not token:
|
|
||||||
token = login_github_copilot(print_fn=lambda _message: None)
|
|
||||||
if not (token and token.access):
|
|
||||||
raise WebUISettingsError("OAuth login failed", status=401)
|
|
||||||
return settings_payload()
|
|
||||||
|
|
||||||
raise WebUISettingsError("OAuth login is not supported for this provider")
|
|
||||||
|
|
||||||
|
|
||||||
def logout_oauth_provider(query: QueryParams) -> dict[str, Any]:
|
|
||||||
provider_name = (_query_first(query, "provider") or "").strip()
|
|
||||||
if not provider_name:
|
|
||||||
raise WebUISettingsError("provider is required")
|
|
||||||
spec = find_by_name(provider_name)
|
|
||||||
if spec is None or not spec.is_oauth:
|
|
||||||
raise WebUISettingsError("unknown OAuth provider")
|
|
||||||
|
|
||||||
if spec.name == "openai_codex":
|
|
||||||
try:
|
|
||||||
from oauth_cli_kit.providers import OPENAI_CODEX_PROVIDER
|
|
||||||
from oauth_cli_kit.storage import FileTokenStorage
|
|
||||||
except ImportError:
|
|
||||||
raise WebUISettingsError("oauth_cli_kit is not installed", status=500) from None
|
|
||||||
token_path = FileTokenStorage(token_filename=OPENAI_CODEX_PROVIDER.token_filename).get_token_path()
|
|
||||||
elif spec.name == "github_copilot":
|
|
||||||
try:
|
|
||||||
from nanobot.providers.github_copilot_provider import get_storage
|
|
||||||
except ImportError:
|
|
||||||
raise WebUISettingsError("GitHub Copilot OAuth support is unavailable", status=500) from None
|
|
||||||
token_path = get_storage().get_token_path()
|
|
||||||
else:
|
|
||||||
raise WebUISettingsError("OAuth logout is not supported for this provider")
|
|
||||||
|
|
||||||
for path in (token_path, token_path.with_suffix(".lock")):
|
|
||||||
with suppress(FileNotFoundError):
|
|
||||||
path.unlink()
|
|
||||||
return settings_payload()
|
|
||||||
|
|
||||||
|
|
||||||
def update_network_safety_settings(query: QueryParams) -> dict[str, Any]:
|
|
||||||
raw_allow = (
|
|
||||||
_query_first_alias(query, "webui_allow_local_service_access", "webuiAllowLocalServiceAccess")
|
|
||||||
or _query_first_alias(query, "allow_local_preview_access", "allowLocalPreviewAccess")
|
|
||||||
)
|
|
||||||
raw_default_access_mode = _query_first_alias(query, "webui_default_access_mode", "webuiDefaultAccessMode")
|
|
||||||
if raw_allow is None and raw_default_access_mode is None:
|
|
||||||
raise WebUISettingsError("webui_allow_local_service_access or webui_default_access_mode is required")
|
|
||||||
|
|
||||||
config = load_config()
|
|
||||||
changed = False
|
|
||||||
if raw_allow is not None:
|
|
||||||
webui_allow_local_service_access = _parse_bool(raw_allow, "webui_allow_local_service_access")
|
|
||||||
if config.tools.webui_allow_local_service_access != webui_allow_local_service_access:
|
|
||||||
config.tools.webui_allow_local_service_access = webui_allow_local_service_access
|
|
||||||
changed = True
|
|
||||||
|
|
||||||
if changed:
|
|
||||||
save_config(config)
|
|
||||||
if raw_default_access_mode is not None:
|
|
||||||
default_access_mode = raw_default_access_mode.strip().lower()
|
|
||||||
if default_access_mode == "restricted":
|
|
||||||
default_access_mode = "default"
|
|
||||||
if default_access_mode not in {"default", "full"}:
|
|
||||||
raise WebUISettingsError("webui_default_access_mode must be default or full")
|
|
||||||
try:
|
|
||||||
write_webui_default_access_mode(default_access_mode)
|
|
||||||
except ValueError as exc:
|
|
||||||
raise WebUISettingsError(str(exc)) from exc
|
|
||||||
return settings_payload(requires_restart=changed)
|
|
||||||
|
|
||||||
|
|
||||||
def update_web_search_settings(query: QueryParams) -> dict[str, Any]:
|
def update_web_search_settings(query: QueryParams) -> dict[str, Any]:
|
||||||
provider_name = (_query_first(query, "provider") or "").strip().lower()
|
provider_name = (_query_first(query, "provider") or "").strip().lower()
|
||||||
provider_option = _WEB_SEARCH_PROVIDER_BY_NAME.get(provider_name)
|
provider_option = _WEB_SEARCH_PROVIDER_BY_NAME.get(provider_name)
|
||||||
|
|||||||
@@ -1,329 +0,0 @@
|
|||||||
"""HTTP route adapter for WebUI Settings APIs.
|
|
||||||
|
|
||||||
Keep WebUI Settings route handlers here, not in ``channels/websocket.py``.
|
|
||||||
The websocket channel owns transport concerns; this module owns WebUI Settings
|
|
||||||
request mapping and response shaping.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import json
|
|
||||||
from collections.abc import Callable
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from websockets.http11 import Request as WsRequest
|
|
||||||
from websockets.http11 import Response
|
|
||||||
|
|
||||||
from nanobot.agent.tools.mcp import request_mcp_reload
|
|
||||||
from nanobot.bus.queue import MessageBus
|
|
||||||
from nanobot.webui.cli_apps_api import cli_apps_action, cli_apps_payload
|
|
||||||
from nanobot.webui.mcp_presets_api import mcp_presets_settings_action
|
|
||||||
from nanobot.webui.settings_api import (
|
|
||||||
WebUISettingsError,
|
|
||||||
create_model_configuration,
|
|
||||||
decorate_settings_payload,
|
|
||||||
login_oauth_provider,
|
|
||||||
logout_oauth_provider,
|
|
||||||
provider_models_payload,
|
|
||||||
settings_payload,
|
|
||||||
update_agent_settings,
|
|
||||||
update_image_generation_settings,
|
|
||||||
update_model_configuration,
|
|
||||||
update_network_safety_settings,
|
|
||||||
update_provider_settings,
|
|
||||||
update_web_search_settings,
|
|
||||||
)
|
|
||||||
|
|
||||||
QueryParams = dict[str, list[str]]
|
|
||||||
|
|
||||||
_MCP_VALUES_HEADER = "X-Nanobot-MCP-Values"
|
|
||||||
_MCP_VALUES_HEADER_MAX_BYTES = 64 * 1024
|
|
||||||
|
|
||||||
_MCP_PRESET_ACTIONS_BY_PATH = {
|
|
||||||
"/api/settings/mcp-presets/enable": "enable",
|
|
||||||
"/api/settings/mcp-presets/remove": "remove",
|
|
||||||
"/api/settings/mcp-presets/test": "test",
|
|
||||||
"/api/settings/mcp-presets/custom": "custom",
|
|
||||||
"/api/settings/mcp-presets/import": "import",
|
|
||||||
"/api/settings/mcp-presets/import-cursor": "import-cursor",
|
|
||||||
"/api/settings/mcp-presets/tools": "tools",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class WebUISettingsRouter:
|
|
||||||
"""Route WebUI Settings HTTP requests behind a transport-neutral boundary."""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
bus: MessageBus,
|
|
||||||
logger: Any,
|
|
||||||
check_api_token: Callable[[WsRequest], bool],
|
|
||||||
parse_query: Callable[[str], QueryParams],
|
|
||||||
json_response: Callable[[dict[str, Any]], Response],
|
|
||||||
error_response: Callable[[int, str | None], Response],
|
|
||||||
runtime_surface: str,
|
|
||||||
runtime_capabilities: dict[str, Any],
|
|
||||||
) -> None:
|
|
||||||
self.bus = bus
|
|
||||||
self.logger = logger
|
|
||||||
self._check_api_token = check_api_token
|
|
||||||
self._parse_query = parse_query
|
|
||||||
self._json_response = json_response
|
|
||||||
self._error_response = error_response
|
|
||||||
self._runtime_surface = runtime_surface
|
|
||||||
self._runtime_capabilities = runtime_capabilities
|
|
||||||
self._restart_sections: set[str] = set()
|
|
||||||
|
|
||||||
async def dispatch(self, request: WsRequest, path: str) -> Response | None:
|
|
||||||
if path == "/api/settings":
|
|
||||||
return self._handle_settings(request)
|
|
||||||
if path == "/api/settings/update":
|
|
||||||
return self._handle_settings_update(request)
|
|
||||||
if path == "/api/settings/model-configurations/create":
|
|
||||||
return self._handle_settings_model_configuration_create(request)
|
|
||||||
if path == "/api/settings/model-configurations/update":
|
|
||||||
return self._handle_settings_model_configuration_update(request)
|
|
||||||
if path == "/api/settings/provider/update":
|
|
||||||
return self._handle_settings_provider_update(request)
|
|
||||||
if path == "/api/settings/provider-models":
|
|
||||||
return await self._handle_settings_provider_models(request)
|
|
||||||
if path == "/api/settings/provider/oauth-login":
|
|
||||||
return await self._handle_settings_provider_oauth(request, "login")
|
|
||||||
if path == "/api/settings/provider/oauth-logout":
|
|
||||||
return await self._handle_settings_provider_oauth(request, "logout")
|
|
||||||
if path == "/api/settings/web-search/update":
|
|
||||||
return self._handle_settings_web_search_update(request)
|
|
||||||
if path == "/api/settings/image-generation/update":
|
|
||||||
return self._handle_settings_image_generation_update(request)
|
|
||||||
if path == "/api/settings/network-safety/update":
|
|
||||||
return self._handle_settings_network_safety_update(request)
|
|
||||||
if path == "/api/settings/cli-apps":
|
|
||||||
return self._handle_settings_cli_apps(request)
|
|
||||||
if path == "/api/settings/cli-apps/install":
|
|
||||||
return await self._handle_settings_cli_apps_action(request, "install")
|
|
||||||
if path == "/api/settings/cli-apps/update":
|
|
||||||
return await self._handle_settings_cli_apps_action(request, "update")
|
|
||||||
if path == "/api/settings/cli-apps/uninstall":
|
|
||||||
return await self._handle_settings_cli_apps_action(request, "uninstall")
|
|
||||||
if path == "/api/settings/cli-apps/test":
|
|
||||||
return await self._handle_settings_cli_apps_action(request, "test")
|
|
||||||
if path == "/api/settings/mcp-presets":
|
|
||||||
return await self._handle_settings_mcp_presets(request)
|
|
||||||
mcp_action = _MCP_PRESET_ACTIONS_BY_PATH.get(path)
|
|
||||||
if mcp_action is not None:
|
|
||||||
return await self._handle_settings_mcp_presets(request, mcp_action)
|
|
||||||
return None
|
|
||||||
|
|
||||||
def _query(self, request: WsRequest) -> QueryParams:
|
|
||||||
return self._parse_query(request.path)
|
|
||||||
|
|
||||||
def _authorized(self, request: WsRequest) -> bool:
|
|
||||||
return self._check_api_token(request)
|
|
||||||
|
|
||||||
def _unauthorized(self) -> Response:
|
|
||||||
return self._error_response(401, "Unauthorized")
|
|
||||||
|
|
||||||
def _with_restart_state(
|
|
||||||
self,
|
|
||||||
payload: dict[str, Any],
|
|
||||||
*,
|
|
||||||
section: str | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Keep restart-required state alive for this gateway process."""
|
|
||||||
if section and payload.get("requires_restart"):
|
|
||||||
self._restart_sections.add(section)
|
|
||||||
sections = sorted(self._restart_sections)
|
|
||||||
payload = dict(payload)
|
|
||||||
if sections:
|
|
||||||
payload["requires_restart"] = True
|
|
||||||
return decorate_settings_payload(
|
|
||||||
payload,
|
|
||||||
surface=self._runtime_surface,
|
|
||||||
runtime_capability_overrides=self._runtime_capabilities,
|
|
||||||
restart_required_sections=sections,
|
|
||||||
)
|
|
||||||
|
|
||||||
def _parse_mcp_settings_query(self, request: WsRequest) -> QueryParams:
|
|
||||||
query = self._query(request)
|
|
||||||
raw = request.headers.get(_MCP_VALUES_HEADER)
|
|
||||||
if not raw:
|
|
||||||
return query
|
|
||||||
if len(raw.encode("utf-8")) > _MCP_VALUES_HEADER_MAX_BYTES:
|
|
||||||
raise WebUISettingsError("MCP settings payload is too large")
|
|
||||||
try:
|
|
||||||
payload = json.loads(raw)
|
|
||||||
except json.JSONDecodeError as exc:
|
|
||||||
raise WebUISettingsError("invalid MCP settings payload") from exc
|
|
||||||
if not isinstance(payload, dict):
|
|
||||||
raise WebUISettingsError("MCP settings payload must be a JSON object")
|
|
||||||
merged = {key: list(values) for key, values in query.items()}
|
|
||||||
for key, value in payload.items():
|
|
||||||
if not isinstance(key, str) or not key:
|
|
||||||
raise WebUISettingsError("MCP settings payload contains an invalid key")
|
|
||||||
if value is None:
|
|
||||||
continue
|
|
||||||
if isinstance(value, str):
|
|
||||||
text = value.strip()
|
|
||||||
else:
|
|
||||||
text = json.dumps(value, ensure_ascii=False, separators=(",", ":"))
|
|
||||||
if text:
|
|
||||||
merged[key] = [text]
|
|
||||||
return merged
|
|
||||||
|
|
||||||
def _handle_settings(self, request: WsRequest) -> Response:
|
|
||||||
if not self._authorized(request):
|
|
||||||
return self._unauthorized()
|
|
||||||
return self._json_response(
|
|
||||||
self._with_restart_state(
|
|
||||||
settings_payload(
|
|
||||||
surface=self._runtime_surface,
|
|
||||||
runtime_capability_overrides=self._runtime_capabilities,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
def _handle_settings_update(self, request: WsRequest) -> Response:
|
|
||||||
if not self._authorized(request):
|
|
||||||
return self._unauthorized()
|
|
||||||
try:
|
|
||||||
payload = update_agent_settings(self._query(request))
|
|
||||||
except WebUISettingsError as e:
|
|
||||||
return self._error_response(e.status, e.message)
|
|
||||||
return self._json_response(self._with_restart_state(payload, section="runtime"))
|
|
||||||
|
|
||||||
def _handle_settings_model_configuration_create(self, request: WsRequest) -> Response:
|
|
||||||
if not self._authorized(request):
|
|
||||||
return self._unauthorized()
|
|
||||||
try:
|
|
||||||
payload = create_model_configuration(self._query(request))
|
|
||||||
except WebUISettingsError as e:
|
|
||||||
return self._error_response(e.status, e.message)
|
|
||||||
return self._json_response(self._with_restart_state(payload))
|
|
||||||
|
|
||||||
def _handle_settings_model_configuration_update(self, request: WsRequest) -> Response:
|
|
||||||
if not self._authorized(request):
|
|
||||||
return self._unauthorized()
|
|
||||||
try:
|
|
||||||
payload = update_model_configuration(self._query(request))
|
|
||||||
except WebUISettingsError as e:
|
|
||||||
return self._error_response(e.status, e.message)
|
|
||||||
return self._json_response(self._with_restart_state(payload))
|
|
||||||
|
|
||||||
def _handle_settings_provider_update(self, request: WsRequest) -> Response:
|
|
||||||
if not self._authorized(request):
|
|
||||||
return self._unauthorized()
|
|
||||||
try:
|
|
||||||
payload = update_provider_settings(self._query(request))
|
|
||||||
except WebUISettingsError as e:
|
|
||||||
return self._error_response(e.status, e.message)
|
|
||||||
return self._json_response(self._with_restart_state(payload, section="image"))
|
|
||||||
|
|
||||||
async def _handle_settings_provider_models(self, request: WsRequest) -> Response:
|
|
||||||
if not self._authorized(request):
|
|
||||||
return self._unauthorized()
|
|
||||||
try:
|
|
||||||
payload = await asyncio.to_thread(provider_models_payload, self._query(request))
|
|
||||||
except WebUISettingsError as e:
|
|
||||||
return self._error_response(e.status, e.message)
|
|
||||||
except Exception:
|
|
||||||
self.logger.exception("failed to load provider model list")
|
|
||||||
return self._error_response(500, "failed to load provider model list")
|
|
||||||
return self._json_response(payload)
|
|
||||||
|
|
||||||
async def _handle_settings_provider_oauth(
|
|
||||||
self,
|
|
||||||
request: WsRequest,
|
|
||||||
action: str,
|
|
||||||
) -> Response:
|
|
||||||
if not self._authorized(request):
|
|
||||||
return self._unauthorized()
|
|
||||||
query = self._query(request)
|
|
||||||
try:
|
|
||||||
if action == "login":
|
|
||||||
payload = await asyncio.to_thread(login_oauth_provider, query)
|
|
||||||
else:
|
|
||||||
payload = await asyncio.to_thread(logout_oauth_provider, query)
|
|
||||||
except WebUISettingsError as e:
|
|
||||||
return self._error_response(e.status, e.message)
|
|
||||||
return self._json_response(self._with_restart_state(payload))
|
|
||||||
|
|
||||||
def _handle_settings_web_search_update(self, request: WsRequest) -> Response:
|
|
||||||
if not self._authorized(request):
|
|
||||||
return self._unauthorized()
|
|
||||||
try:
|
|
||||||
payload = update_web_search_settings(self._query(request))
|
|
||||||
except WebUISettingsError as e:
|
|
||||||
return self._error_response(e.status, e.message)
|
|
||||||
return self._json_response(self._with_restart_state(payload, section="browser"))
|
|
||||||
|
|
||||||
def _handle_settings_image_generation_update(self, request: WsRequest) -> Response:
|
|
||||||
if not self._authorized(request):
|
|
||||||
return self._unauthorized()
|
|
||||||
try:
|
|
||||||
payload = update_image_generation_settings(self._query(request))
|
|
||||||
except WebUISettingsError as e:
|
|
||||||
return self._error_response(e.status, e.message)
|
|
||||||
return self._json_response(self._with_restart_state(payload, section="image"))
|
|
||||||
|
|
||||||
def _handle_settings_network_safety_update(self, request: WsRequest) -> Response:
|
|
||||||
if not self._authorized(request):
|
|
||||||
return self._unauthorized()
|
|
||||||
try:
|
|
||||||
payload = update_network_safety_settings(self._query(request))
|
|
||||||
except WebUISettingsError as e:
|
|
||||||
return self._error_response(e.status, e.message)
|
|
||||||
return self._json_response(self._with_restart_state(payload, section="runtime"))
|
|
||||||
|
|
||||||
def _handle_settings_cli_apps(self, request: WsRequest) -> Response:
|
|
||||||
if not self._authorized(request):
|
|
||||||
return self._unauthorized()
|
|
||||||
try:
|
|
||||||
payload = cli_apps_payload()
|
|
||||||
except Exception:
|
|
||||||
self.logger.exception("failed to load CLI Apps payload")
|
|
||||||
return self._error_response(500, "failed to load CLI Apps")
|
|
||||||
return self._json_response(payload)
|
|
||||||
|
|
||||||
async def _handle_settings_cli_apps_action(
|
|
||||||
self,
|
|
||||||
request: WsRequest,
|
|
||||||
action: str,
|
|
||||||
) -> Response:
|
|
||||||
if not self._authorized(request):
|
|
||||||
return self._unauthorized()
|
|
||||||
try:
|
|
||||||
payload = await asyncio.to_thread(cli_apps_action, action, self._query(request))
|
|
||||||
except WebUISettingsError as e:
|
|
||||||
return self._error_response(e.status, e.message)
|
|
||||||
except Exception as e:
|
|
||||||
status = getattr(e, "status", 500)
|
|
||||||
message = getattr(e, "message", str(e))
|
|
||||||
if status >= 500:
|
|
||||||
self.logger.exception("CLI Apps action '{}' failed", action)
|
|
||||||
return self._error_response(status, message)
|
|
||||||
return self._json_response(payload)
|
|
||||||
|
|
||||||
async def _handle_settings_mcp_presets(
|
|
||||||
self,
|
|
||||||
request: WsRequest,
|
|
||||||
action: str | None = None,
|
|
||||||
) -> Response:
|
|
||||||
if not self._authorized(request):
|
|
||||||
return self._unauthorized()
|
|
||||||
try:
|
|
||||||
payload = await mcp_presets_settings_action(
|
|
||||||
action,
|
|
||||||
self._parse_mcp_settings_query(request),
|
|
||||||
reload_mcp=lambda: request_mcp_reload(self.bus),
|
|
||||||
)
|
|
||||||
except Exception as e:
|
|
||||||
status = getattr(e, "status", 500)
|
|
||||||
message = getattr(e, "message", str(e))
|
|
||||||
if status >= 500:
|
|
||||||
self.logger.exception("MCP preset action '{}' failed", action or "list")
|
|
||||||
return self._error_response(status, message)
|
|
||||||
if action is None:
|
|
||||||
return self._json_response(payload)
|
|
||||||
return self._json_response(self._with_restart_state(payload, section="runtime"))
|
|
||||||
@@ -38,7 +38,6 @@ def default_webui_sidebar_state() -> dict[str, Any]:
|
|||||||
"pinned_keys": [],
|
"pinned_keys": [],
|
||||||
"archived_keys": [],
|
"archived_keys": [],
|
||||||
"title_overrides": {},
|
"title_overrides": {},
|
||||||
"project_name_overrides": {},
|
|
||||||
"tags_by_key": {},
|
"tags_by_key": {},
|
||||||
"collapsed_groups": {},
|
"collapsed_groups": {},
|
||||||
"view": {
|
"view": {
|
||||||
@@ -137,9 +136,6 @@ def normalize_webui_sidebar_state(raw: Any) -> dict[str, Any]:
|
|||||||
state["pinned_keys"] = _clean_string_list(raw.get("pinned_keys"))
|
state["pinned_keys"] = _clean_string_list(raw.get("pinned_keys"))
|
||||||
state["archived_keys"] = _clean_string_list(raw.get("archived_keys"))
|
state["archived_keys"] = _clean_string_list(raw.get("archived_keys"))
|
||||||
state["title_overrides"] = _clean_title_overrides(raw.get("title_overrides"))
|
state["title_overrides"] = _clean_title_overrides(raw.get("title_overrides"))
|
||||||
state["project_name_overrides"] = _clean_title_overrides(
|
|
||||||
raw.get("project_name_overrides")
|
|
||||||
)
|
|
||||||
state["tags_by_key"] = _clean_tags_by_key(raw.get("tags_by_key"))
|
state["tags_by_key"] = _clean_tags_by_key(raw.get("tags_by_key"))
|
||||||
state["collapsed_groups"] = _clean_bool_map(raw.get("collapsed_groups"))
|
state["collapsed_groups"] = _clean_bool_map(raw.get("collapsed_groups"))
|
||||||
state["view"] = _clean_view(raw.get("view"))
|
state["view"] = _clean_view(raw.get("view"))
|
||||||
@@ -194,3 +190,4 @@ def write_webui_sidebar_state(raw: dict[str, Any]) -> dict[str, Any]:
|
|||||||
finally:
|
finally:
|
||||||
os.close(dir_fd)
|
os.close(dir_fd)
|
||||||
return state
|
return state
|
||||||
|
|
||||||
|
|||||||
+18
-332
@@ -4,12 +4,10 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import re
|
|
||||||
import time
|
import time
|
||||||
import uuid
|
import uuid
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Callable, Mapping
|
from typing import Any, Callable
|
||||||
from urllib.parse import unquote, urlparse
|
|
||||||
|
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
@@ -18,82 +16,6 @@ from nanobot.session.manager import SessionManager
|
|||||||
|
|
||||||
WEBUI_TRANSCRIPT_SCHEMA_VERSION = 3
|
WEBUI_TRANSCRIPT_SCHEMA_VERSION = 3
|
||||||
_MAX_TRANSCRIPT_FILE_BYTES = 8 * 1024 * 1024
|
_MAX_TRANSCRIPT_FILE_BYTES = 8 * 1024 * 1024
|
||||||
_MARKDOWN_LOCAL_IMAGE_RE = re.compile(
|
|
||||||
r"!\[([^\]]*)\]\((<[^>]+>|[^)\s]+)(\s+(?:\"[^\"]*\"|'[^']*'))?\)"
|
|
||||||
)
|
|
||||||
_INLINE_MARKDOWN_IMAGE_EXTS: frozenset[str] = frozenset({
|
|
||||||
".png",
|
|
||||||
".jpg",
|
|
||||||
".jpeg",
|
|
||||||
".webp",
|
|
||||||
".gif",
|
|
||||||
".svg",
|
|
||||||
})
|
|
||||||
_INLINE_MARKDOWN_VIDEO_EXTS: frozenset[str] = frozenset({
|
|
||||||
".mp4",
|
|
||||||
".mov",
|
|
||||||
".webm",
|
|
||||||
})
|
|
||||||
_INLINE_MARKDOWN_MEDIA_EXTS = _INLINE_MARKDOWN_IMAGE_EXTS | _INLINE_MARKDOWN_VIDEO_EXTS
|
|
||||||
_FILE_EDIT_TOOL_NAMES: frozenset[str] = frozenset({
|
|
||||||
"write_file",
|
|
||||||
"edit_file",
|
|
||||||
"apply_patch",
|
|
||||||
})
|
|
||||||
|
|
||||||
|
|
||||||
def rewrite_local_markdown_images(
|
|
||||||
text: str,
|
|
||||||
*,
|
|
||||||
workspace_path: Path,
|
|
||||||
sign_path: Callable[[Path], Mapping[str, Any] | None],
|
|
||||||
) -> str:
|
|
||||||
"""Rewrite markdown media paths inside the workspace to signed WebUI media URLs."""
|
|
||||||
if "![" not in text:
|
|
||||||
return text
|
|
||||||
|
|
||||||
def resolve_url(raw_url: str) -> str | None:
|
|
||||||
url = raw_url.strip()
|
|
||||||
if url.startswith("<") and url.endswith(">"):
|
|
||||||
url = url[1:-1].strip()
|
|
||||||
if not url or url.startswith(("/api/media/", "#")):
|
|
||||||
return None
|
|
||||||
parsed = urlparse(url)
|
|
||||||
if parsed.scheme or parsed.netloc or parsed.query or parsed.fragment:
|
|
||||||
return None
|
|
||||||
path_text = unquote(url)
|
|
||||||
if Path(path_text).suffix.lower() not in _INLINE_MARKDOWN_MEDIA_EXTS:
|
|
||||||
return None
|
|
||||||
candidate = Path(path_text).expanduser()
|
|
||||||
if not candidate.is_absolute():
|
|
||||||
candidate = workspace_path / candidate
|
|
||||||
try:
|
|
||||||
resolved = candidate.resolve(strict=False)
|
|
||||||
resolved.relative_to(workspace_path)
|
|
||||||
except (OSError, ValueError):
|
|
||||||
return None
|
|
||||||
if not resolved.is_file():
|
|
||||||
return None
|
|
||||||
signed = sign_path(resolved)
|
|
||||||
return str(signed.get("url")) if signed and signed.get("url") else None
|
|
||||||
|
|
||||||
def replace(match: re.Match[str]) -> str:
|
|
||||||
signed_url = resolve_url(match.group(2))
|
|
||||||
if not signed_url:
|
|
||||||
return match.group(0)
|
|
||||||
title = match.group(3) or ""
|
|
||||||
return f""
|
|
||||||
|
|
||||||
return _MARKDOWN_LOCAL_IMAGE_RE.sub(replace, text)
|
|
||||||
|
|
||||||
|
|
||||||
def _media_kind_from_name(name: str) -> str:
|
|
||||||
ext = Path(name).suffix.lower()
|
|
||||||
if ext in _INLINE_MARKDOWN_IMAGE_EXTS:
|
|
||||||
return "image"
|
|
||||||
if ext in _INLINE_MARKDOWN_VIDEO_EXTS:
|
|
||||||
return "video"
|
|
||||||
return "file"
|
|
||||||
|
|
||||||
|
|
||||||
def webui_transcript_path(session_key: str) -> Path:
|
def webui_transcript_path(session_key: str) -> Path:
|
||||||
@@ -194,149 +116,6 @@ def tool_trace_lines_from_events(events: Any) -> list[str]:
|
|||||||
return lines
|
return lines
|
||||||
|
|
||||||
|
|
||||||
_PHASE_RANK = {"start": 1, "end": 2, "error": 3}
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_tool_events(events: Any) -> list[dict[str, Any]]:
|
|
||||||
if not isinstance(events, list):
|
|
||||||
return []
|
|
||||||
out: list[dict[str, Any]] = []
|
|
||||||
for event in events:
|
|
||||||
if not event or not isinstance(event, dict):
|
|
||||||
continue
|
|
||||||
if event.get("phase") not in {"start", "end", "error"}:
|
|
||||||
continue
|
|
||||||
if not isinstance(event.get("name"), str):
|
|
||||||
fn = event.get("function")
|
|
||||||
if not (isinstance(fn, dict) and isinstance(fn.get("name"), str)):
|
|
||||||
continue
|
|
||||||
out.append(dict(event))
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _tool_event_key(event: dict[str, Any]) -> str:
|
|
||||||
call_id = event.get("call_id")
|
|
||||||
if isinstance(call_id, str) and call_id:
|
|
||||||
return f"call:{call_id}"
|
|
||||||
return _format_tool_call_trace(event) or json.dumps(event, sort_keys=True, ensure_ascii=False)
|
|
||||||
|
|
||||||
|
|
||||||
def _tool_event_file_edit_key(event: dict[str, Any]) -> str | None:
|
|
||||||
call_id = event.get("call_id")
|
|
||||||
if not isinstance(call_id, str) or not call_id:
|
|
||||||
return None
|
|
||||||
name = event.get("name")
|
|
||||||
if not isinstance(name, str) or not name:
|
|
||||||
fn = event.get("function")
|
|
||||||
name = fn.get("name") if isinstance(fn, dict) else ""
|
|
||||||
if not isinstance(name, str) or name not in _FILE_EDIT_TOOL_NAMES:
|
|
||||||
return None
|
|
||||||
return f"{call_id}|{name}"
|
|
||||||
|
|
||||||
|
|
||||||
def _merge_tool_events(previous: Any, incoming: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
|
||||||
if not isinstance(previous, list) or not previous:
|
|
||||||
return incoming
|
|
||||||
if not incoming:
|
|
||||||
return [dict(event) for event in previous if isinstance(event, dict)]
|
|
||||||
merged = [dict(event) for event in previous if isinstance(event, dict)]
|
|
||||||
index_by_key = {_tool_event_key(event): idx for idx, event in enumerate(merged)}
|
|
||||||
for event in incoming:
|
|
||||||
key = _tool_event_key(event)
|
|
||||||
existing_index = index_by_key.get(key)
|
|
||||||
if existing_index is None:
|
|
||||||
index_by_key[key] = len(merged)
|
|
||||||
merged.append(event)
|
|
||||||
continue
|
|
||||||
existing = merged[existing_index]
|
|
||||||
incoming_rank = _PHASE_RANK.get(str(event.get("phase")), 0)
|
|
||||||
existing_rank = _PHASE_RANK.get(str(existing.get("phase")), 0)
|
|
||||||
if incoming_rank >= existing_rank:
|
|
||||||
merged[existing_index] = {**existing, **event}
|
|
||||||
return merged
|
|
||||||
|
|
||||||
|
|
||||||
def _file_edit_key(edit: dict[str, Any]) -> str:
|
|
||||||
call_id = str(edit.get("call_id") or "")
|
|
||||||
tool = str(edit.get("tool") or "")
|
|
||||||
if call_id:
|
|
||||||
return f"{call_id}|{tool}"
|
|
||||||
return f"{tool}|{edit.get('path') or ''}"
|
|
||||||
|
|
||||||
|
|
||||||
def _message_has_file_edit_for_tool_event(
|
|
||||||
message: dict[str, Any],
|
|
||||||
event: dict[str, Any],
|
|
||||||
) -> bool:
|
|
||||||
key = _tool_event_file_edit_key(event)
|
|
||||||
if not key:
|
|
||||||
return False
|
|
||||||
edits = message.get("fileEdits")
|
|
||||||
if not isinstance(edits, list):
|
|
||||||
return False
|
|
||||||
return any(isinstance(edit, dict) and _file_edit_key(edit) == key for edit in edits)
|
|
||||||
|
|
||||||
|
|
||||||
def _filter_covered_file_edit_tool_events(
|
|
||||||
messages: list[dict[str, Any]],
|
|
||||||
events: list[dict[str, Any]],
|
|
||||||
) -> list[dict[str, Any]]:
|
|
||||||
if not events:
|
|
||||||
return events
|
|
||||||
return [
|
|
||||||
event
|
|
||||||
for event in events
|
|
||||||
if not any(_message_has_file_edit_for_tool_event(message, event) for message in messages)
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def _strip_covered_file_edit_tool_hints(
|
|
||||||
message: dict[str, Any],
|
|
||||||
edits: list[dict[str, Any]],
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
incoming_keys = {
|
|
||||||
_file_edit_key(edit)
|
|
||||||
for edit in edits
|
|
||||||
if isinstance(edit, dict)
|
|
||||||
}
|
|
||||||
events = message.get("toolEvents")
|
|
||||||
if not incoming_keys or not isinstance(events, list):
|
|
||||||
return message
|
|
||||||
|
|
||||||
kept_events: list[dict[str, Any]] = []
|
|
||||||
removed_trace_lines: set[str] = set()
|
|
||||||
changed = False
|
|
||||||
for event in events:
|
|
||||||
if not isinstance(event, dict):
|
|
||||||
continue
|
|
||||||
key = _tool_event_file_edit_key(event)
|
|
||||||
if key and key in incoming_keys:
|
|
||||||
changed = True
|
|
||||||
removed_trace_lines.update(tool_trace_lines_from_events([event]))
|
|
||||||
continue
|
|
||||||
kept_events.append(event)
|
|
||||||
if not changed:
|
|
||||||
return message
|
|
||||||
|
|
||||||
raw_traces = message.get("traces")
|
|
||||||
if isinstance(raw_traces, list):
|
|
||||||
previous_traces = [trace for trace in raw_traces if isinstance(trace, str)]
|
|
||||||
else:
|
|
||||||
content = message.get("content")
|
|
||||||
previous_traces = [content] if isinstance(content, str) and content else []
|
|
||||||
next_traces = [trace for trace in previous_traces if trace not in removed_trace_lines]
|
|
||||||
next_message = {
|
|
||||||
**message,
|
|
||||||
"traces": next_traces,
|
|
||||||
"content": next_traces[-1] if next_traces else "",
|
|
||||||
}
|
|
||||||
if kept_events:
|
|
||||||
next_message["toolEvents"] = kept_events
|
|
||||||
else:
|
|
||||||
next_message.pop("toolEvents", None)
|
|
||||||
return next_message
|
|
||||||
|
|
||||||
|
|
||||||
def _merge_unique_tool_trace_lines(
|
def _merge_unique_tool_trace_lines(
|
||||||
previous_traces: list[str],
|
previous_traces: list[str],
|
||||||
lines: list[str],
|
lines: list[str],
|
||||||
@@ -357,7 +136,6 @@ def replay_transcript_to_ui_messages(
|
|||||||
lines: list[dict[str, Any]],
|
lines: list[dict[str, Any]],
|
||||||
*,
|
*,
|
||||||
augment_user_media: Callable[[list[str]], list[dict[str, Any]]] | None = None,
|
augment_user_media: Callable[[list[str]], list[dict[str, Any]]] | None = None,
|
||||||
augment_assistant_text: Callable[[str], str] | None = None,
|
|
||||||
) -> list[dict[str, Any]]:
|
) -> list[dict[str, Any]]:
|
||||||
"""Fold JSONL records into ``UIMessage``-shaped dicts for the WebUI.
|
"""Fold JSONL records into ``UIMessage``-shaped dicts for the WebUI.
|
||||||
|
|
||||||
@@ -458,40 +236,6 @@ def replay_transcript_to_ui_messages(
|
|||||||
return None
|
return None
|
||||||
return str(last.get("id"))
|
return str(last.get("id"))
|
||||||
|
|
||||||
def demote_interrupted_assistant(segment: str) -> None:
|
|
||||||
nonlocal buffer_message_id, buffer_parts
|
|
||||||
for i in range(len(messages) - 1, -1, -1):
|
|
||||||
candidate = messages[i]
|
|
||||||
if candidate.get("role") == "user":
|
|
||||||
break
|
|
||||||
content = candidate.get("content")
|
|
||||||
if (
|
|
||||||
candidate.get("role") != "assistant"
|
|
||||||
or candidate.get("kind") == "trace"
|
|
||||||
or not candidate.get("isStreaming")
|
|
||||||
or not isinstance(content, str)
|
|
||||||
or not content.strip()
|
|
||||||
or candidate.get("media")
|
|
||||||
):
|
|
||||||
continue
|
|
||||||
reasoning_parts = [
|
|
||||||
part
|
|
||||||
for part in (candidate.get("reasoning"), content)
|
|
||||||
if isinstance(part, str) and part.strip()
|
|
||||||
]
|
|
||||||
messages[i] = {
|
|
||||||
**candidate,
|
|
||||||
"content": "",
|
|
||||||
"reasoning": "\n\n".join(reasoning_parts),
|
|
||||||
"reasoningStreaming": False,
|
|
||||||
"isStreaming": False,
|
|
||||||
"activitySegmentId": candidate.get("activitySegmentId") or segment,
|
|
||||||
}
|
|
||||||
if buffer_message_id == candidate.get("id"):
|
|
||||||
buffer_message_id = None
|
|
||||||
buffer_parts = []
|
|
||||||
return
|
|
||||||
|
|
||||||
def close_reasoning(prev: list[dict[str, Any]]) -> None:
|
def close_reasoning(prev: list[dict[str, Any]]) -> None:
|
||||||
for i in range(len(prev) - 1, -1, -1):
|
for i in range(len(prev) - 1, -1, -1):
|
||||||
if prev[i].get("reasoningStreaming"):
|
if prev[i].get("reasoningStreaming"):
|
||||||
@@ -553,6 +297,13 @@ def replay_transcript_to_ui_messages(
|
|||||||
active_activity_segment_id = None
|
active_activity_segment_id = None
|
||||||
active_file_edit_segment_id = None
|
active_file_edit_segment_id = None
|
||||||
|
|
||||||
|
def _file_edit_key(edit: dict[str, Any]) -> str:
|
||||||
|
call_id = str(edit.get("call_id") or "")
|
||||||
|
tool = str(edit.get("tool") or "")
|
||||||
|
if call_id:
|
||||||
|
return f"{call_id}|{tool}"
|
||||||
|
return f"{tool}|{edit.get('path') or ''}"
|
||||||
|
|
||||||
def find_file_edit_trace_index(
|
def find_file_edit_trace_index(
|
||||||
segment: str | None,
|
segment: str | None,
|
||||||
edits: list[dict[str, Any]],
|
edits: list[dict[str, Any]],
|
||||||
@@ -562,23 +313,16 @@ def replay_transcript_to_ui_messages(
|
|||||||
candidate = messages[i]
|
candidate = messages[i]
|
||||||
if candidate.get("role") == "user":
|
if candidate.get("role") == "user":
|
||||||
break
|
break
|
||||||
if candidate.get("kind") != "trace":
|
if candidate.get("kind") != "trace" or not candidate.get("fileEdits"):
|
||||||
continue
|
continue
|
||||||
if segment and candidate.get("activitySegmentId") == segment:
|
if segment and candidate.get("activitySegmentId") == segment:
|
||||||
return i
|
return i
|
||||||
existing_edits = candidate.get("fileEdits")
|
existing_edits = candidate.get("fileEdits")
|
||||||
if isinstance(existing_edits, list):
|
if not isinstance(existing_edits, list):
|
||||||
|
continue
|
||||||
for existing in existing_edits:
|
for existing in existing_edits:
|
||||||
if isinstance(existing, dict) and _file_edit_key(existing) in incoming_keys:
|
if isinstance(existing, dict) and _file_edit_key(existing) in incoming_keys:
|
||||||
return i
|
return i
|
||||||
existing_tool_events = candidate.get("toolEvents")
|
|
||||||
if isinstance(existing_tool_events, list):
|
|
||||||
for event in existing_tool_events:
|
|
||||||
if not isinstance(event, dict):
|
|
||||||
continue
|
|
||||||
key = _tool_event_file_edit_key(event)
|
|
||||||
if key and key in incoming_keys:
|
|
||||||
return i
|
|
||||||
return None
|
return None
|
||||||
|
|
||||||
def upsert_file_edits(edits: list[dict[str, Any]], idx: int) -> None:
|
def upsert_file_edits(edits: list[dict[str, Any]], idx: int) -> None:
|
||||||
@@ -586,16 +330,11 @@ def replay_transcript_to_ui_messages(
|
|||||||
if not edits:
|
if not edits:
|
||||||
return
|
return
|
||||||
segment = active_file_edit_segment_id
|
segment = active_file_edit_segment_id
|
||||||
if not segment:
|
|
||||||
segment = _new_activity_segment(activate=False)
|
|
||||||
active_file_edit_segment_id = segment
|
|
||||||
demote_interrupted_assistant(segment)
|
|
||||||
target_index = find_file_edit_trace_index(segment, edits)
|
target_index = find_file_edit_trace_index(segment, edits)
|
||||||
if target_index is not None:
|
if target_index is not None:
|
||||||
last = messages[target_index]
|
last = messages[target_index]
|
||||||
segment = str(last.get("activitySegmentId") or segment or _new_activity_segment(activate=False))
|
segment = str(last.get("activitySegmentId") or segment or _new_activity_segment(activate=False))
|
||||||
active_file_edit_segment_id = segment
|
active_file_edit_segment_id = segment
|
||||||
last = _strip_covered_file_edit_tool_hints(last, edits)
|
|
||||||
else:
|
else:
|
||||||
if not segment:
|
if not segment:
|
||||||
segment = _new_activity_segment(activate=False)
|
segment = _new_activity_segment(activate=False)
|
||||||
@@ -666,14 +405,6 @@ def replay_transcript_to_ui_messages(
|
|||||||
row["media"] = media_att
|
row["media"] = media_att
|
||||||
if all(m.get("kind") == "image" for m in media_att):
|
if all(m.get("kind") == "image" for m in media_att):
|
||||||
row["images"] = [{"url": m.get("url"), "name": m.get("name")} for m in media_att]
|
row["images"] = [{"url": m.get("url"), "name": m.get("name")} for m in media_att]
|
||||||
cli_apps = rec.get("cli_apps")
|
|
||||||
if isinstance(cli_apps, list) and cli_apps:
|
|
||||||
row["cliApps"] = [dict(app) for app in cli_apps if isinstance(app, dict)]
|
|
||||||
mcp_presets = rec.get("mcp_presets")
|
|
||||||
if isinstance(mcp_presets, list) and mcp_presets:
|
|
||||||
row["mcpPresets"] = [
|
|
||||||
dict(preset) for preset in mcp_presets if isinstance(preset, dict)
|
|
||||||
]
|
|
||||||
messages.append(row)
|
messages.append(row)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
@@ -718,24 +449,6 @@ def replay_transcript_to_ui_messages(
|
|||||||
buffer_message_id = None
|
buffer_message_id = None
|
||||||
buffer_parts = []
|
buffer_parts = []
|
||||||
continue
|
continue
|
||||||
final_text = rec.get("text")
|
|
||||||
if isinstance(final_text, str):
|
|
||||||
if buffer_message_id is None:
|
|
||||||
buffer_message_id = _new_id("buf", idx)
|
|
||||||
messages.append(
|
|
||||||
{
|
|
||||||
"id": buffer_message_id,
|
|
||||||
"role": "assistant",
|
|
||||||
"content": final_text,
|
|
||||||
"isStreaming": True,
|
|
||||||
"createdAt": _ts_base + idx,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
for i, m in enumerate(messages):
|
|
||||||
if m.get("id") == buffer_message_id:
|
|
||||||
messages[i] = {**m, "content": final_text, "isStreaming": True}
|
|
||||||
break
|
|
||||||
buffer_message_id = None
|
buffer_message_id = None
|
||||||
buffer_parts = []
|
buffer_parts = []
|
||||||
continue
|
continue
|
||||||
@@ -773,22 +486,12 @@ def replay_transcript_to_ui_messages(
|
|||||||
close_reasoning(messages)
|
close_reasoning(messages)
|
||||||
continue
|
continue
|
||||||
if kind in ("tool_hint", "progress"):
|
if kind in ("tool_hint", "progress"):
|
||||||
structured_events = _normalize_tool_events(rec.get("tool_events"))
|
structured = tool_trace_lines_from_events(rec.get("tool_events"))
|
||||||
visible_structured_events = _filter_covered_file_edit_tool_events(messages, structured_events)
|
|
||||||
structured = tool_trace_lines_from_events(visible_structured_events)
|
|
||||||
text = rec.get("text")
|
text = rec.get("text")
|
||||||
if structured:
|
trace_lines = structured if structured else ([text] if isinstance(text, str) and text else [])
|
||||||
trace_lines = structured
|
|
||||||
elif structured_events:
|
|
||||||
trace_lines = []
|
|
||||||
elif isinstance(text, str) and text:
|
|
||||||
trace_lines = [text]
|
|
||||||
else:
|
|
||||||
trace_lines = []
|
|
||||||
if not trace_lines:
|
if not trace_lines:
|
||||||
continue
|
continue
|
||||||
segment = _ensure_activity_segment()
|
segment = _ensure_activity_segment()
|
||||||
demote_interrupted_assistant(segment)
|
|
||||||
last = messages[-1] if messages else None
|
last = messages[-1] if messages else None
|
||||||
if (
|
if (
|
||||||
last
|
last
|
||||||
@@ -799,7 +502,7 @@ def replay_transcript_to_ui_messages(
|
|||||||
prev_traces = list(last.get("traces") or [last.get("content")])
|
prev_traces = list(last.get("traces") or [last.get("content")])
|
||||||
if structured:
|
if structured:
|
||||||
merged_traces, added = _merge_unique_tool_trace_lines(prev_traces, structured)
|
merged_traces, added = _merge_unique_tool_trace_lines(prev_traces, structured)
|
||||||
if not added and not visible_structured_events:
|
if not added:
|
||||||
continue
|
continue
|
||||||
else:
|
else:
|
||||||
merged_traces = prev_traces + trace_lines
|
merged_traces = prev_traces + trace_lines
|
||||||
@@ -807,9 +510,6 @@ def replay_transcript_to_ui_messages(
|
|||||||
**last,
|
**last,
|
||||||
"traces": merged_traces,
|
"traces": merged_traces,
|
||||||
"content": merged_traces[-1],
|
"content": merged_traces[-1],
|
||||||
"toolEvents": _merge_tool_events(last.get("toolEvents"), visible_structured_events)
|
|
||||||
if visible_structured_events
|
|
||||||
else last.get("toolEvents"),
|
|
||||||
"activitySegmentId": last.get("activitySegmentId") or segment,
|
"activitySegmentId": last.get("activitySegmentId") or segment,
|
||||||
}
|
}
|
||||||
messages[-1] = merged
|
messages[-1] = merged
|
||||||
@@ -821,7 +521,6 @@ def replay_transcript_to_ui_messages(
|
|||||||
"kind": "trace",
|
"kind": "trace",
|
||||||
"content": trace_lines[-1],
|
"content": trace_lines[-1],
|
||||||
"traces": trace_lines,
|
"traces": trace_lines,
|
||||||
**({"toolEvents": visible_structured_events} if visible_structured_events else {}),
|
|
||||||
"activitySegmentId": segment,
|
"activitySegmentId": segment,
|
||||||
"createdAt": _ts_base + idx,
|
"createdAt": _ts_base + idx,
|
||||||
},
|
},
|
||||||
@@ -837,12 +536,11 @@ def replay_transcript_to_ui_messages(
|
|||||||
if isinstance(media_urls, list):
|
if isinstance(media_urls, list):
|
||||||
for m in media_urls:
|
for m in media_urls:
|
||||||
if isinstance(m, dict) and m.get("url"):
|
if isinstance(m, dict) and m.get("url"):
|
||||||
name = str(m.get("name") or "")
|
|
||||||
media.append(
|
media.append(
|
||||||
{
|
{
|
||||||
"kind": _media_kind_from_name(name),
|
"kind": "image",
|
||||||
"url": str(m["url"]),
|
"url": str(m["url"]),
|
||||||
"name": name,
|
"name": str(m.get("name") or ""),
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
extra: dict[str, Any] = {"content": content_s}
|
extra: dict[str, Any] = {"content": content_s}
|
||||||
@@ -871,14 +569,7 @@ def replay_transcript_to_ui_messages(
|
|||||||
buffer_parts = []
|
buffer_parts = []
|
||||||
continue
|
continue
|
||||||
|
|
||||||
for i, m in enumerate(messages):
|
for m in messages:
|
||||||
if (
|
|
||||||
augment_assistant_text is not None
|
|
||||||
and m.get("role") == "assistant"
|
|
||||||
and m.get("kind") != "trace"
|
|
||||||
and isinstance(m.get("content"), str)
|
|
||||||
):
|
|
||||||
messages[i] = {**m, "content": augment_assistant_text(m["content"])}
|
|
||||||
m.pop("isStreaming", None)
|
m.pop("isStreaming", None)
|
||||||
m.pop("reasoningStreaming", None)
|
m.pop("reasoningStreaming", None)
|
||||||
return messages
|
return messages
|
||||||
@@ -888,17 +579,12 @@ def build_webui_thread_response(
|
|||||||
session_key: str,
|
session_key: str,
|
||||||
*,
|
*,
|
||||||
augment_user_media: Callable[[list[str]], list[dict[str, Any]]] | None = None,
|
augment_user_media: Callable[[list[str]], list[dict[str, Any]]] | None = None,
|
||||||
augment_assistant_text: Callable[[str], str] | None = None,
|
|
||||||
) -> dict[str, Any] | None:
|
) -> dict[str, Any] | None:
|
||||||
"""Return a payload compatible with ``WebuiThreadPersistedPayload``."""
|
"""Return a payload compatible with ``WebuiThreadPersistedPayload``."""
|
||||||
lines = read_transcript_lines(session_key)
|
lines = read_transcript_lines(session_key)
|
||||||
if not lines:
|
if not lines:
|
||||||
return None
|
return None
|
||||||
msgs = replay_transcript_to_ui_messages(
|
msgs = replay_transcript_to_ui_messages(lines, augment_user_media=augment_user_media)
|
||||||
lines,
|
|
||||||
augment_user_media=augment_user_media,
|
|
||||||
augment_assistant_text=augment_assistant_text,
|
|
||||||
)
|
|
||||||
return {
|
return {
|
||||||
"schemaVersion": WEBUI_TRANSCRIPT_SCHEMA_VERSION,
|
"schemaVersion": WEBUI_TRANSCRIPT_SCHEMA_VERSION,
|
||||||
"sessionKey": session_key,
|
"sessionKey": session_key,
|
||||||
|
|||||||
@@ -1,283 +0,0 @@
|
|||||||
"""Persisted WebUI project workspace state."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import time
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from loguru import logger
|
|
||||||
|
|
||||||
from nanobot.config.paths import get_webui_dir
|
|
||||||
from nanobot.security.workspace_access import (
|
|
||||||
WORKSPACE_SCOPE_METADATA_KEY,
|
|
||||||
WorkspaceScope,
|
|
||||||
WorkspaceScopeError,
|
|
||||||
build_workspace_scope,
|
|
||||||
default_workspace_scope,
|
|
||||||
validate_workspace_scope_payload,
|
|
||||||
)
|
|
||||||
|
|
||||||
WEBUI_WORKSPACE_STATE_SCHEMA_VERSION = 1
|
|
||||||
_MAX_STATE_FILE_BYTES = 128 * 1024
|
|
||||||
_DEFAULT_ACCESS_MODES = {"default", "full"}
|
|
||||||
_LEGACY_RESTRICTED_DEFAULT_ACCESS_MODE = "restricted"
|
|
||||||
_WEBUI_SCOPE_CHANNEL = "websocket"
|
|
||||||
|
|
||||||
|
|
||||||
def webui_workspace_state_path() -> Path:
|
|
||||||
return get_webui_dir() / "workspace-state.json"
|
|
||||||
|
|
||||||
|
|
||||||
def default_webui_workspace_state() -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"schema_version": WEBUI_WORKSPACE_STATE_SCHEMA_VERSION,
|
|
||||||
"default_access_mode": "default",
|
|
||||||
"updated_at": None,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def normalize_webui_workspace_state(raw: Any) -> dict[str, Any]:
|
|
||||||
if not isinstance(raw, dict):
|
|
||||||
raw = {}
|
|
||||||
state = default_webui_workspace_state()
|
|
||||||
updated_at = raw.get("updated_at")
|
|
||||||
state["updated_at"] = updated_at if isinstance(updated_at, str) else None
|
|
||||||
default_access_mode = raw.get("default_access_mode")
|
|
||||||
if default_access_mode in _DEFAULT_ACCESS_MODES:
|
|
||||||
state["default_access_mode"] = default_access_mode
|
|
||||||
return state
|
|
||||||
|
|
||||||
|
|
||||||
def read_webui_workspace_state() -> dict[str, Any]:
|
|
||||||
path = webui_workspace_state_path()
|
|
||||||
if not path.is_file():
|
|
||||||
return default_webui_workspace_state()
|
|
||||||
try:
|
|
||||||
if path.stat().st_size > _MAX_STATE_FILE_BYTES:
|
|
||||||
logger.warning("webui workspace state too large, ignoring: {}", path)
|
|
||||||
return default_webui_workspace_state()
|
|
||||||
with open(path, encoding="utf-8") as f:
|
|
||||||
raw = json.load(f)
|
|
||||||
except (OSError, json.JSONDecodeError) as e:
|
|
||||||
logger.warning("read webui workspace state failed {}: {}", path, e)
|
|
||||||
return default_webui_workspace_state()
|
|
||||||
return normalize_webui_workspace_state(raw)
|
|
||||||
|
|
||||||
|
|
||||||
def write_webui_workspace_state(raw: dict[str, Any]) -> dict[str, Any]:
|
|
||||||
state = normalize_webui_workspace_state(raw)
|
|
||||||
state["updated_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
|
||||||
encoded = json.dumps(
|
|
||||||
state,
|
|
||||||
ensure_ascii=False,
|
|
||||||
indent=2,
|
|
||||||
sort_keys=True,
|
|
||||||
).encode("utf-8")
|
|
||||||
if len(encoded) > _MAX_STATE_FILE_BYTES:
|
|
||||||
raise ValueError("workspace state is too large")
|
|
||||||
|
|
||||||
path = webui_workspace_state_path()
|
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
tmp = path.with_suffix(".json.tmp")
|
|
||||||
with open(tmp, "wb") as f:
|
|
||||||
f.write(encoded)
|
|
||||||
f.write(b"\n")
|
|
||||||
f.flush()
|
|
||||||
os.fsync(f.fileno())
|
|
||||||
os.replace(tmp, path)
|
|
||||||
try:
|
|
||||||
dir_fd = os.open(path.parent, os.O_RDONLY)
|
|
||||||
except OSError:
|
|
||||||
return state
|
|
||||||
try:
|
|
||||||
os.fsync(dir_fd)
|
|
||||||
finally:
|
|
||||||
os.close(dir_fd)
|
|
||||||
return state
|
|
||||||
|
|
||||||
|
|
||||||
def read_webui_default_access_mode() -> str:
|
|
||||||
state = read_webui_workspace_state()
|
|
||||||
mode = state.get("default_access_mode")
|
|
||||||
return mode if mode in _DEFAULT_ACCESS_MODES else "default"
|
|
||||||
|
|
||||||
|
|
||||||
def write_webui_default_access_mode(mode: str) -> bool:
|
|
||||||
if mode == _LEGACY_RESTRICTED_DEFAULT_ACCESS_MODE:
|
|
||||||
mode = "default"
|
|
||||||
if mode not in _DEFAULT_ACCESS_MODES:
|
|
||||||
raise ValueError("default access mode must be default or full")
|
|
||||||
state = read_webui_workspace_state()
|
|
||||||
changed = state.get("default_access_mode") != mode
|
|
||||||
if changed:
|
|
||||||
state["default_access_mode"] = mode
|
|
||||||
write_webui_workspace_state(state)
|
|
||||||
return changed
|
|
||||||
|
|
||||||
|
|
||||||
def default_scope_for_webui(
|
|
||||||
default_workspace: Path,
|
|
||||||
default_restrict_to_workspace: bool,
|
|
||||||
) -> WorkspaceScope:
|
|
||||||
mode = read_webui_default_access_mode()
|
|
||||||
if mode == "default":
|
|
||||||
return default_workspace_scope(
|
|
||||||
default_workspace,
|
|
||||||
default_restrict_to_workspace,
|
|
||||||
source_channel=_WEBUI_SCOPE_CHANNEL,
|
|
||||||
)
|
|
||||||
return build_workspace_scope(default_workspace, mode, source_channel=_WEBUI_SCOPE_CHANNEL)
|
|
||||||
|
|
||||||
|
|
||||||
def workspaces_payload(
|
|
||||||
*,
|
|
||||||
default_workspace: Path,
|
|
||||||
default_restrict_to_workspace: bool,
|
|
||||||
controls_available: bool,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
default_access_mode = read_webui_default_access_mode()
|
|
||||||
default_scope = (
|
|
||||||
default_workspace_scope(
|
|
||||||
default_workspace,
|
|
||||||
default_restrict_to_workspace,
|
|
||||||
source_channel=_WEBUI_SCOPE_CHANNEL,
|
|
||||||
)
|
|
||||||
if default_access_mode == "default"
|
|
||||||
else build_workspace_scope(default_workspace, default_access_mode, source_channel=_WEBUI_SCOPE_CHANNEL)
|
|
||||||
)
|
|
||||||
return {
|
|
||||||
"schema_version": WEBUI_WORKSPACE_STATE_SCHEMA_VERSION,
|
|
||||||
"default_access_mode": default_access_mode,
|
|
||||||
"default_scope": default_scope.payload(),
|
|
||||||
"controls": {
|
|
||||||
"can_change_project": controls_available,
|
|
||||||
"can_use_full_access": controls_available,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class WebUIWorkspaceController:
|
|
||||||
"""Own WebUI project scope persistence and validation."""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
session_manager: Any | None,
|
|
||||||
default_workspace: Path,
|
|
||||||
default_restrict_to_workspace: bool,
|
|
||||||
) -> None:
|
|
||||||
self._sessions = session_manager
|
|
||||||
self._default_workspace = default_workspace
|
|
||||||
self._default_restrict_to_workspace = default_restrict_to_workspace
|
|
||||||
|
|
||||||
def default_scope(self) -> WorkspaceScope:
|
|
||||||
return default_scope_for_webui(
|
|
||||||
self._default_workspace,
|
|
||||||
self._default_restrict_to_workspace,
|
|
||||||
)
|
|
||||||
|
|
||||||
def scope_for_session_key(self, session_key: str) -> WorkspaceScope:
|
|
||||||
if self._sessions is None:
|
|
||||||
return self.default_scope()
|
|
||||||
data = self._sessions.read_session_file(session_key)
|
|
||||||
metadata = data.get("metadata", {}) if isinstance(data, dict) else {}
|
|
||||||
if not isinstance(metadata, dict) or WORKSPACE_SCOPE_METADATA_KEY not in metadata:
|
|
||||||
return self.default_scope()
|
|
||||||
try:
|
|
||||||
return validate_workspace_scope_payload(
|
|
||||||
metadata.get(WORKSPACE_SCOPE_METADATA_KEY),
|
|
||||||
default_workspace=self._default_workspace,
|
|
||||||
default_restrict_to_workspace=self._default_restrict_to_workspace,
|
|
||||||
source_channel=_WEBUI_SCOPE_CHANNEL,
|
|
||||||
)
|
|
||||||
except WorkspaceScopeError:
|
|
||||||
return self.default_scope()
|
|
||||||
|
|
||||||
def payload(self, *, controls_available: bool) -> dict[str, Any]:
|
|
||||||
return workspaces_payload(
|
|
||||||
default_workspace=self._default_workspace,
|
|
||||||
default_restrict_to_workspace=self._default_restrict_to_workspace,
|
|
||||||
controls_available=controls_available,
|
|
||||||
)
|
|
||||||
|
|
||||||
def scope_from_envelope(
|
|
||||||
self,
|
|
||||||
envelope: dict[str, Any],
|
|
||||||
*,
|
|
||||||
session_key: str | None,
|
|
||||||
controls_available: bool,
|
|
||||||
) -> WorkspaceScope:
|
|
||||||
raw = envelope.get(WORKSPACE_SCOPE_METADATA_KEY)
|
|
||||||
if raw is None and session_key:
|
|
||||||
scope = self.scope_for_session_key(session_key)
|
|
||||||
elif raw is None:
|
|
||||||
scope = self.default_scope()
|
|
||||||
else:
|
|
||||||
scope = validate_workspace_scope_payload(
|
|
||||||
raw,
|
|
||||||
default_workspace=self._default_workspace,
|
|
||||||
default_restrict_to_workspace=self._default_restrict_to_workspace,
|
|
||||||
source_channel=_WEBUI_SCOPE_CHANNEL,
|
|
||||||
)
|
|
||||||
if not controls_available and scope.metadata() != self.default_scope().metadata():
|
|
||||||
raise WorkspaceScopeError("workspace controls are localhost-only", status=403)
|
|
||||||
return scope
|
|
||||||
|
|
||||||
def scope_for_new_chat(
|
|
||||||
self,
|
|
||||||
envelope: dict[str, Any],
|
|
||||||
*,
|
|
||||||
controls_available: bool,
|
|
||||||
) -> WorkspaceScope:
|
|
||||||
return self.scope_from_envelope(
|
|
||||||
envelope,
|
|
||||||
session_key=None,
|
|
||||||
controls_available=controls_available,
|
|
||||||
)
|
|
||||||
|
|
||||||
def scope_for_set_request(
|
|
||||||
self,
|
|
||||||
envelope: dict[str, Any],
|
|
||||||
*,
|
|
||||||
chat_id: str,
|
|
||||||
chat_running: bool,
|
|
||||||
controls_available: bool,
|
|
||||||
) -> WorkspaceScope:
|
|
||||||
if chat_running:
|
|
||||||
raise WorkspaceScopeError("chat_running", status=409)
|
|
||||||
return self.scope_from_envelope(
|
|
||||||
envelope,
|
|
||||||
session_key=f"websocket:{chat_id}",
|
|
||||||
controls_available=controls_available,
|
|
||||||
)
|
|
||||||
|
|
||||||
def scope_for_message(
|
|
||||||
self,
|
|
||||||
envelope: dict[str, Any],
|
|
||||||
*,
|
|
||||||
chat_id: str,
|
|
||||||
chat_running: bool,
|
|
||||||
controls_available: bool,
|
|
||||||
) -> WorkspaceScope:
|
|
||||||
scope = self.scope_from_envelope(
|
|
||||||
envelope,
|
|
||||||
session_key=f"websocket:{chat_id}",
|
|
||||||
controls_available=controls_available,
|
|
||||||
)
|
|
||||||
if (
|
|
||||||
WORKSPACE_SCOPE_METADATA_KEY in envelope
|
|
||||||
and chat_running
|
|
||||||
and scope.metadata() != self.scope_for_session_key(f"websocket:{chat_id}").metadata()
|
|
||||||
):
|
|
||||||
raise WorkspaceScopeError("chat_running", status=409)
|
|
||||||
return scope
|
|
||||||
|
|
||||||
def persist_scope(self, chat_id: str, scope: WorkspaceScope) -> None:
|
|
||||||
if self._sessions is not None:
|
|
||||||
session = self._sessions.get_or_create(f"websocket:{chat_id}")
|
|
||||||
session.metadata["webui"] = True
|
|
||||||
session.metadata[WORKSPACE_SCOPE_METADATA_KEY] = scope.metadata()
|
|
||||||
self._sessions.save(session)
|
|
||||||
+2
-2
@@ -37,7 +37,7 @@ dependencies = [
|
|||||||
"rich>=14.0.0,<15.0.0",
|
"rich>=14.0.0,<15.0.0",
|
||||||
"croniter>=6.0.0,<7.0.0",
|
"croniter>=6.0.0,<7.0.0",
|
||||||
"dingtalk-stream>=0.24.0,<1.0.0",
|
"dingtalk-stream>=0.24.0,<1.0.0",
|
||||||
"python-telegram-bot[socks,webhooks]>=22.6,<23.0",
|
"python-telegram-bot[socks]>=22.6,<23.0",
|
||||||
"lark-oapi>=1.5.0,<2.0.0",
|
"lark-oapi>=1.5.0,<2.0.0",
|
||||||
"socksio>=1.0.0,<2.0.0",
|
"socksio>=1.0.0,<2.0.0",
|
||||||
"python-socketio>=5.16.0,<6.0.0",
|
"python-socketio>=5.16.0,<6.0.0",
|
||||||
@@ -61,6 +61,7 @@ dependencies = [
|
|||||||
"openpyxl>=3.1.0,<4.0.0",
|
"openpyxl>=3.1.0,<4.0.0",
|
||||||
"python-pptx>=1.0.0,<2.0.0",
|
"python-pptx>=1.0.0,<2.0.0",
|
||||||
"filelock>=3.25.2",
|
"filelock>=3.25.2",
|
||||||
|
"keyring>=25.0.0,<26.0.0",
|
||||||
"boto3>=1.43.0",
|
"boto3>=1.43.0",
|
||||||
]
|
]
|
||||||
|
|
||||||
@@ -82,7 +83,6 @@ msteams = [
|
|||||||
|
|
||||||
matrix = [
|
matrix = [
|
||||||
"matrix-nio[e2e]>=0.25.2; sys_platform != 'win32'",
|
"matrix-nio[e2e]>=0.25.2; sys_platform != 'win32'",
|
||||||
"aiohttp>=3.9.0,<4.0.0",
|
|
||||||
"mistune>=3.0.0,<4.0.0",
|
"mistune>=3.0.0,<4.0.0",
|
||||||
"nh3>=0.2.17,<1.0.0",
|
"nh3>=0.2.17,<1.0.0",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -139,13 +139,6 @@ class TestLoadBootstrapFiles:
|
|||||||
for name in ContextBuilder.BOOTSTRAP_FILES:
|
for name in ContextBuilder.BOOTSTRAP_FILES:
|
||||||
assert f"## {name}" in result
|
assert f"## {name}" in result
|
||||||
|
|
||||||
def test_legacy_tools_md_is_not_bootstrapped(self, tmp_path):
|
|
||||||
(tmp_path / "TOOLS.md").write_text("workspace tool notes", encoding="utf-8")
|
|
||||||
builder = _builder(tmp_path)
|
|
||||||
result = builder._load_bootstrap_files()
|
|
||||||
assert "TOOLS.md" not in result
|
|
||||||
assert "workspace tool notes" not in result
|
|
||||||
|
|
||||||
def test_utf8_content(self, tmp_path):
|
def test_utf8_content(self, tmp_path):
|
||||||
(tmp_path / "AGENTS.md").write_text("用中文回复", encoding="utf-8")
|
(tmp_path / "AGENTS.md").write_text("用中文回复", encoding="utf-8")
|
||||||
builder = _builder(tmp_path)
|
builder = _builder(tmp_path)
|
||||||
@@ -178,37 +171,6 @@ class TestIsTemplateContent:
|
|||||||
assert ContextBuilder._is_template_content("totally different", "memory/MEMORY.md") is False
|
assert ContextBuilder._is_template_content("totally different", "memory/MEMORY.md") is False
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Bundled bootstrap templates
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class TestBundledToolContract:
|
|
||||||
def test_tool_contract_balances_general_and_coding_workflows(self):
|
|
||||||
from importlib.resources import files as pkg_files
|
|
||||||
|
|
||||||
tpl = pkg_files("nanobot") / "templates" / "agent" / "tool_contract.md"
|
|
||||||
content = tpl.read_text(encoding="utf-8")
|
|
||||||
|
|
||||||
assert "## General Tool Contract" in content
|
|
||||||
assert "Use the narrowest structured tool" in content
|
|
||||||
assert "Do not use `exec` as a universal workaround" in content
|
|
||||||
assert "## File and Coding Workflows" in content
|
|
||||||
assert "apply_patch" in content
|
|
||||||
assert "## Web and External Information" in content
|
|
||||||
assert "## Messaging and Media" in content
|
|
||||||
assert "## Scheduling and Background Work" in content
|
|
||||||
assert "pure coding" not in content.lower()
|
|
||||||
|
|
||||||
def test_tool_contract_is_injected_without_workspace_file(self, tmp_path):
|
|
||||||
builder = _builder(tmp_path)
|
|
||||||
prompt = builder.build_system_prompt()
|
|
||||||
|
|
||||||
assert "# Tool Usage Notes" in prompt
|
|
||||||
assert "## General Tool Contract" in prompt
|
|
||||||
assert "Do not use `exec` as a universal workaround" in prompt
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# _build_user_content
|
# _build_user_content
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -362,21 +324,6 @@ class TestBuildMessages:
|
|||||||
assert "Other chat goal." not in str(without_goal[-1]["content"])
|
assert "Other chat goal." not in str(without_goal[-1]["content"])
|
||||||
assert "Goal (active):" not in str(without_goal[-1]["content"])
|
assert "Goal (active):" not in str(without_goal[-1]["content"])
|
||||||
|
|
||||||
def test_current_runtime_lines_are_injected(self, tmp_path):
|
|
||||||
builder = _builder(tmp_path)
|
|
||||||
messages = builder.build_messages(
|
|
||||||
[],
|
|
||||||
"please use @zoom tonight",
|
|
||||||
current_runtime_lines=[
|
|
||||||
"CLI App Attachment: @zoom (installed; tool=run_cli_app; entry_point=cli-anything-zoom).",
|
|
||||||
],
|
|
||||||
)
|
|
||||||
user_msg = str(messages[-1]["content"])
|
|
||||||
|
|
||||||
assert "CLI App Attachment: @zoom" in user_msg
|
|
||||||
assert "tool=run_cli_app" in user_msg
|
|
||||||
assert "entry_point=cli-anything-zoom" in user_msg
|
|
||||||
|
|
||||||
def test_consecutive_same_role_merged(self, tmp_path):
|
def test_consecutive_same_role_merged(self, tmp_path):
|
||||||
builder = _builder(tmp_path)
|
builder = _builder(tmp_path)
|
||||||
history = [{"role": "user", "content": "previous user message"}]
|
history = [{"role": "user", "content": "previous user message"}]
|
||||||
|
|||||||
@@ -1,169 +0,0 @@
|
|||||||
import asyncio
|
|
||||||
import base64
|
|
||||||
from pathlib import Path
|
|
||||||
from unittest.mock import AsyncMock, MagicMock
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from nanobot.agent.loop import AgentLoop, TurnContext, TurnState
|
|
||||||
from nanobot.bus.events import InboundMessage
|
|
||||||
from nanobot.bus.queue import MessageBus
|
|
||||||
from nanobot.config.schema import ChannelsConfig
|
|
||||||
from nanobot.providers.base import LLMResponse
|
|
||||||
from nanobot.utils.document import reference_non_image_attachments
|
|
||||||
|
|
||||||
|
|
||||||
def _make_loop(tmp_path: Path, channels_config: ChannelsConfig | None = None) -> AgentLoop:
|
|
||||||
provider = MagicMock()
|
|
||||||
provider.get_default_model.return_value = "test-model"
|
|
||||||
provider.chat_with_retry = AsyncMock(return_value=LLMResponse(content="ok"))
|
|
||||||
return AgentLoop(
|
|
||||||
bus=MessageBus(),
|
|
||||||
provider=provider,
|
|
||||||
workspace=tmp_path,
|
|
||||||
model="test-model",
|
|
||||||
channels_config=channels_config,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_state_restore_extracts_documents_by_default(
|
|
||||||
tmp_path: Path,
|
|
||||||
monkeypatch: pytest.MonkeyPatch,
|
|
||||||
) -> None:
|
|
||||||
loop = _make_loop(tmp_path)
|
|
||||||
doc_path = tmp_path / "report.txt"
|
|
||||||
doc_path.write_text("Quarterly revenue is $5M", encoding="utf-8")
|
|
||||||
calls: list[tuple[str, list[str]]] = []
|
|
||||||
|
|
||||||
def fake_extract_documents(content: str, media: list[str]) -> tuple[str, list[str]]:
|
|
||||||
calls.append((content, media))
|
|
||||||
return f"{content}\n\n[File: report.txt]\nQuarterly revenue is $5M", []
|
|
||||||
|
|
||||||
monkeypatch.setattr("nanobot.agent.loop.extract_documents", fake_extract_documents)
|
|
||||||
|
|
||||||
ctx = TurnContext(
|
|
||||||
msg=InboundMessage(
|
|
||||||
channel="cli",
|
|
||||||
sender_id="u",
|
|
||||||
chat_id="c",
|
|
||||||
content="summarize",
|
|
||||||
media=[str(doc_path)],
|
|
||||||
),
|
|
||||||
session_key="cli:c",
|
|
||||||
state=TurnState.RESTORE,
|
|
||||||
turn_id="turn-1",
|
|
||||||
)
|
|
||||||
|
|
||||||
assert await loop._state_restore(ctx) == "ok"
|
|
||||||
|
|
||||||
assert calls == [("summarize", [str(doc_path)])]
|
|
||||||
assert "Quarterly revenue" in ctx.msg.content
|
|
||||||
assert ctx.msg.media == []
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_state_restore_references_documents_when_extraction_disabled(
|
|
||||||
tmp_path: Path,
|
|
||||||
monkeypatch: pytest.MonkeyPatch,
|
|
||||||
) -> None:
|
|
||||||
loop = _make_loop(tmp_path, ChannelsConfig(extract_document_text=False))
|
|
||||||
doc_path = tmp_path / "report.txt"
|
|
||||||
doc_path.write_text("Quarterly revenue is $5M", encoding="utf-8")
|
|
||||||
|
|
||||||
def fail_extract_documents(content: str, media: list[str]) -> tuple[str, list[str]]:
|
|
||||||
raise AssertionError("document extraction should be disabled")
|
|
||||||
|
|
||||||
monkeypatch.setattr("nanobot.agent.loop.extract_documents", fail_extract_documents)
|
|
||||||
|
|
||||||
ctx = TurnContext(
|
|
||||||
msg=InboundMessage(
|
|
||||||
channel="cli",
|
|
||||||
sender_id="u",
|
|
||||||
chat_id="c",
|
|
||||||
content="summarize",
|
|
||||||
media=[str(doc_path)],
|
|
||||||
),
|
|
||||||
session_key="cli:c",
|
|
||||||
state=TurnState.RESTORE,
|
|
||||||
turn_id="turn-1",
|
|
||||||
)
|
|
||||||
|
|
||||||
assert await loop._state_restore(ctx) == "ok"
|
|
||||||
|
|
||||||
assert "Quarterly revenue" not in ctx.msg.content
|
|
||||||
assert f"[Attachment: {doc_path}]" in ctx.msg.content
|
|
||||||
assert ctx.msg.media == []
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_pending_followup_references_documents_when_extraction_disabled(
|
|
||||||
tmp_path: Path,
|
|
||||||
monkeypatch: pytest.MonkeyPatch,
|
|
||||||
) -> None:
|
|
||||||
doc_path = tmp_path / "followup.txt"
|
|
||||||
doc_path.write_text("Do not inject this file body", encoding="utf-8")
|
|
||||||
captured_messages: list[list[dict]] = []
|
|
||||||
call_count = {"n": 0}
|
|
||||||
|
|
||||||
async def chat_with_retry(*, messages: list[dict], **kwargs: object) -> LLMResponse:
|
|
||||||
call_count["n"] += 1
|
|
||||||
captured_messages.append([dict(message) for message in messages])
|
|
||||||
return LLMResponse(content=f"answer-{call_count['n']}", tool_calls=[], usage={})
|
|
||||||
|
|
||||||
loop = _make_loop(tmp_path, ChannelsConfig(extract_document_text=False))
|
|
||||||
loop.provider.chat_with_retry = chat_with_retry
|
|
||||||
loop.tools.get_definitions = MagicMock(return_value=[])
|
|
||||||
|
|
||||||
def fail_extract_documents(content: str, media: list[str]) -> tuple[str, list[str]]:
|
|
||||||
raise AssertionError("document extraction should be disabled")
|
|
||||||
|
|
||||||
monkeypatch.setattr("nanobot.agent.loop.extract_documents", fail_extract_documents)
|
|
||||||
|
|
||||||
pending_queue: asyncio.Queue[InboundMessage] = asyncio.Queue()
|
|
||||||
await pending_queue.put(
|
|
||||||
InboundMessage(
|
|
||||||
channel="cli",
|
|
||||||
sender_id="u",
|
|
||||||
chat_id="c",
|
|
||||||
content="check this",
|
|
||||||
media=[str(doc_path)],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
final_content, _, _, _, had_injections = await loop._run_agent_loop(
|
|
||||||
[{"role": "user", "content": "hello"}],
|
|
||||||
channel="cli",
|
|
||||||
chat_id="c",
|
|
||||||
pending_queue=pending_queue,
|
|
||||||
)
|
|
||||||
|
|
||||||
assert final_content == "answer-2"
|
|
||||||
assert had_injections is True
|
|
||||||
injected_user_content = [
|
|
||||||
message["content"]
|
|
||||||
for message in captured_messages[-1]
|
|
||||||
if message.get("role") == "user" and isinstance(message.get("content"), str)
|
|
||||||
][-1]
|
|
||||||
assert "check this" in injected_user_content
|
|
||||||
assert f"[Attachment: {doc_path}]" in injected_user_content
|
|
||||||
assert "Do not inject this file body" not in injected_user_content
|
|
||||||
|
|
||||||
|
|
||||||
def test_document_extraction_disabled_still_preserves_images(tmp_path: Path) -> None:
|
|
||||||
image_path = tmp_path / "chart.png"
|
|
||||||
image_path.write_bytes(
|
|
||||||
base64.b64decode(
|
|
||||||
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+yF9kAAAAASUVORK5CYII="
|
|
||||||
)
|
|
||||||
)
|
|
||||||
doc_path = tmp_path / "report.txt"
|
|
||||||
doc_path.write_text("manual extraction target", encoding="utf-8")
|
|
||||||
|
|
||||||
content, media = reference_non_image_attachments(
|
|
||||||
"review these",
|
|
||||||
[str(image_path), str(doc_path)],
|
|
||||||
)
|
|
||||||
|
|
||||||
assert media == [str(image_path)]
|
|
||||||
assert f"[Attachment: {doc_path}]" in content
|
|
||||||
@@ -56,38 +56,8 @@ async def test_fallback_on_error() -> None:
|
|||||||
assert result is True
|
assert result is True
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_fallback_can_fail_closed() -> None:
|
|
||||||
class FailingProvider(DummyProvider):
|
|
||||||
async def chat(self, *args, **kwargs) -> LLMResponse:
|
|
||||||
raise RuntimeError("provider down")
|
|
||||||
|
|
||||||
provider = FailingProvider([])
|
|
||||||
result = await evaluate_response(
|
|
||||||
"some response",
|
|
||||||
"some task",
|
|
||||||
provider,
|
|
||||||
"m",
|
|
||||||
default_notify=False,
|
|
||||||
)
|
|
||||||
assert result is False
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_no_tool_call_fallback() -> None:
|
async def test_no_tool_call_fallback() -> None:
|
||||||
provider = DummyProvider([LLMResponse(content="I think you should notify", tool_calls=[])])
|
provider = DummyProvider([LLMResponse(content="I think you should notify", tool_calls=[])])
|
||||||
result = await evaluate_response("some response", "some task", provider, "m")
|
result = await evaluate_response("some response", "some task", provider, "m")
|
||||||
assert result is True
|
assert result is True
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_no_tool_call_can_fail_closed() -> None:
|
|
||||||
provider = DummyProvider([LLMResponse(content="I think you should notify", tool_calls=[])])
|
|
||||||
result = await evaluate_response(
|
|
||||||
"some response",
|
|
||||||
"some task",
|
|
||||||
provider,
|
|
||||||
"m",
|
|
||||||
default_notify=False,
|
|
||||||
)
|
|
||||||
assert result is False
|
|
||||||
|
|||||||
@@ -0,0 +1,336 @@
|
|||||||
|
import asyncio
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nanobot.heartbeat.service import HeartbeatService
|
||||||
|
from nanobot.providers.base import LLMProvider, LLMResponse, ToolCallRequest
|
||||||
|
from nanobot.utils.llm_runtime import LLMRuntime
|
||||||
|
|
||||||
|
|
||||||
|
class DummyProvider(LLMProvider):
|
||||||
|
def __init__(self, responses: list[LLMResponse]):
|
||||||
|
super().__init__()
|
||||||
|
self._responses = list(responses)
|
||||||
|
self.calls = 0
|
||||||
|
self.models: list[str | None] = []
|
||||||
|
|
||||||
|
async def chat(self, *args, **kwargs) -> LLMResponse:
|
||||||
|
self.calls += 1
|
||||||
|
self.models.append(kwargs.get("model"))
|
||||||
|
if self._responses:
|
||||||
|
return self._responses.pop(0)
|
||||||
|
return LLMResponse(content="", tool_calls=[])
|
||||||
|
|
||||||
|
def get_default_model(self) -> str:
|
||||||
|
return "test-model"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_start_is_idempotent(tmp_path) -> None:
|
||||||
|
provider = DummyProvider([])
|
||||||
|
|
||||||
|
service = HeartbeatService(
|
||||||
|
workspace=tmp_path,
|
||||||
|
provider=provider,
|
||||||
|
model="openai/gpt-4o-mini",
|
||||||
|
interval_s=9999,
|
||||||
|
enabled=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
await service.start()
|
||||||
|
first_task = service._task
|
||||||
|
await service.start()
|
||||||
|
|
||||||
|
assert service._task is first_task
|
||||||
|
|
||||||
|
service.stop()
|
||||||
|
await asyncio.sleep(0)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_decide_returns_skip_when_no_tool_call(tmp_path) -> None:
|
||||||
|
provider = DummyProvider([LLMResponse(content="no tool call", tool_calls=[])])
|
||||||
|
service = HeartbeatService(
|
||||||
|
workspace=tmp_path,
|
||||||
|
provider=provider,
|
||||||
|
model="openai/gpt-4o-mini",
|
||||||
|
)
|
||||||
|
|
||||||
|
action, tasks = await service._decide("heartbeat content")
|
||||||
|
assert action == "skip"
|
||||||
|
assert tasks == ""
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_trigger_now_executes_when_decision_is_run(tmp_path) -> None:
|
||||||
|
(tmp_path / "HEARTBEAT.md").write_text("- [ ] do thing", encoding="utf-8")
|
||||||
|
|
||||||
|
provider = DummyProvider([
|
||||||
|
LLMResponse(
|
||||||
|
content="",
|
||||||
|
tool_calls=[
|
||||||
|
ToolCallRequest(
|
||||||
|
id="hb_1",
|
||||||
|
name="heartbeat",
|
||||||
|
arguments={"action": "run", "tasks": "check open tasks"},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
)
|
||||||
|
])
|
||||||
|
|
||||||
|
called_with: list[str] = []
|
||||||
|
|
||||||
|
async def _on_execute(tasks: str) -> str:
|
||||||
|
called_with.append(tasks)
|
||||||
|
return "done"
|
||||||
|
|
||||||
|
service = HeartbeatService(
|
||||||
|
workspace=tmp_path,
|
||||||
|
provider=provider,
|
||||||
|
model="openai/gpt-4o-mini",
|
||||||
|
on_execute=_on_execute,
|
||||||
|
)
|
||||||
|
|
||||||
|
result = await service.trigger_now()
|
||||||
|
assert result == "done"
|
||||||
|
assert called_with == ["check open tasks"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_trigger_now_returns_none_when_decision_is_skip(tmp_path) -> None:
|
||||||
|
(tmp_path / "HEARTBEAT.md").write_text("- [ ] do thing", encoding="utf-8")
|
||||||
|
|
||||||
|
provider = DummyProvider([
|
||||||
|
LLMResponse(
|
||||||
|
content="",
|
||||||
|
tool_calls=[
|
||||||
|
ToolCallRequest(
|
||||||
|
id="hb_1",
|
||||||
|
name="heartbeat",
|
||||||
|
arguments={"action": "skip"},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
)
|
||||||
|
])
|
||||||
|
|
||||||
|
async def _on_execute(tasks: str) -> str:
|
||||||
|
return tasks
|
||||||
|
|
||||||
|
service = HeartbeatService(
|
||||||
|
workspace=tmp_path,
|
||||||
|
provider=provider,
|
||||||
|
model="openai/gpt-4o-mini",
|
||||||
|
on_execute=_on_execute,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert await service.trigger_now() is None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_tick_notifies_when_evaluator_says_yes(tmp_path, monkeypatch) -> None:
|
||||||
|
"""Phase 1 run -> Phase 2 execute -> Phase 3 evaluate=notify -> on_notify called."""
|
||||||
|
(tmp_path / "HEARTBEAT.md").write_text("- [ ] check deployments", encoding="utf-8")
|
||||||
|
|
||||||
|
provider = DummyProvider([
|
||||||
|
LLMResponse(
|
||||||
|
content="",
|
||||||
|
tool_calls=[
|
||||||
|
ToolCallRequest(
|
||||||
|
id="hb_1",
|
||||||
|
name="heartbeat",
|
||||||
|
arguments={"action": "run", "tasks": "check deployments"},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
),
|
||||||
|
])
|
||||||
|
|
||||||
|
executed: list[str] = []
|
||||||
|
notified: list[str] = []
|
||||||
|
|
||||||
|
async def _on_execute(tasks: str) -> str:
|
||||||
|
executed.append(tasks)
|
||||||
|
return "deployment failed on staging"
|
||||||
|
|
||||||
|
async def _on_notify(response: str) -> None:
|
||||||
|
notified.append(response)
|
||||||
|
|
||||||
|
service = HeartbeatService(
|
||||||
|
workspace=tmp_path,
|
||||||
|
provider=provider,
|
||||||
|
model="openai/gpt-4o-mini",
|
||||||
|
on_execute=_on_execute,
|
||||||
|
on_notify=_on_notify,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _eval_notify(*a, **kw):
|
||||||
|
return True
|
||||||
|
|
||||||
|
monkeypatch.setattr("nanobot.utils.evaluator.evaluate_response", _eval_notify)
|
||||||
|
|
||||||
|
await service._tick()
|
||||||
|
assert executed == ["check deployments"]
|
||||||
|
assert notified == ["deployment failed on staging"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_tick_suppresses_when_evaluator_says_no(tmp_path, monkeypatch) -> None:
|
||||||
|
"""Phase 1 run -> Phase 2 execute -> Phase 3 evaluate=silent -> on_notify NOT called."""
|
||||||
|
(tmp_path / "HEARTBEAT.md").write_text("- [ ] check status", encoding="utf-8")
|
||||||
|
|
||||||
|
provider = DummyProvider([
|
||||||
|
LLMResponse(
|
||||||
|
content="",
|
||||||
|
tool_calls=[
|
||||||
|
ToolCallRequest(
|
||||||
|
id="hb_1",
|
||||||
|
name="heartbeat",
|
||||||
|
arguments={"action": "run", "tasks": "check status"},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
),
|
||||||
|
])
|
||||||
|
|
||||||
|
executed: list[str] = []
|
||||||
|
notified: list[str] = []
|
||||||
|
|
||||||
|
async def _on_execute(tasks: str) -> str:
|
||||||
|
executed.append(tasks)
|
||||||
|
return "everything is fine, no issues"
|
||||||
|
|
||||||
|
async def _on_notify(response: str) -> None:
|
||||||
|
notified.append(response)
|
||||||
|
|
||||||
|
service = HeartbeatService(
|
||||||
|
workspace=tmp_path,
|
||||||
|
provider=provider,
|
||||||
|
model="openai/gpt-4o-mini",
|
||||||
|
on_execute=_on_execute,
|
||||||
|
on_notify=_on_notify,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _eval_silent(*a, **kw):
|
||||||
|
return False
|
||||||
|
|
||||||
|
monkeypatch.setattr("nanobot.utils.evaluator.evaluate_response", _eval_silent)
|
||||||
|
|
||||||
|
await service._tick()
|
||||||
|
assert executed == ["check status"]
|
||||||
|
assert notified == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_tick_uses_runtime_provider_and_model(tmp_path, monkeypatch) -> None:
|
||||||
|
"""Preset changes must apply to heartbeat decision and post-run evaluation."""
|
||||||
|
(tmp_path / "HEARTBEAT.md").write_text("- [ ] check runtime model", encoding="utf-8")
|
||||||
|
|
||||||
|
runtime_provider = DummyProvider([
|
||||||
|
LLMResponse(
|
||||||
|
content="",
|
||||||
|
tool_calls=[
|
||||||
|
ToolCallRequest(
|
||||||
|
id="hb_1",
|
||||||
|
name="heartbeat",
|
||||||
|
arguments={"action": "run", "tasks": "check runtime model"},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
),
|
||||||
|
])
|
||||||
|
runtime_model = "openai/gpt-4.1"
|
||||||
|
|
||||||
|
executed: list[str] = []
|
||||||
|
evaluated: list[tuple[LLMProvider, str]] = []
|
||||||
|
|
||||||
|
async def _on_execute(tasks: str) -> str:
|
||||||
|
executed.append(tasks)
|
||||||
|
return "runtime model produced a user-facing update"
|
||||||
|
|
||||||
|
async def _eval_capture(response, tasks, provider, model):
|
||||||
|
evaluated.append((provider, model))
|
||||||
|
return False
|
||||||
|
|
||||||
|
service = HeartbeatService(
|
||||||
|
workspace=tmp_path,
|
||||||
|
llm_runtime=lambda: LLMRuntime(runtime_provider, runtime_model),
|
||||||
|
on_execute=_on_execute,
|
||||||
|
)
|
||||||
|
|
||||||
|
monkeypatch.setattr("nanobot.utils.evaluator.evaluate_response", _eval_capture)
|
||||||
|
|
||||||
|
asyncio.run(service._tick())
|
||||||
|
|
||||||
|
assert runtime_provider.calls == 1
|
||||||
|
assert runtime_provider.models == [runtime_model]
|
||||||
|
assert executed == ["check runtime model"]
|
||||||
|
assert evaluated == [(runtime_provider, runtime_model)]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_decide_retries_transient_error_then_succeeds(tmp_path, monkeypatch) -> None:
|
||||||
|
provider = DummyProvider([
|
||||||
|
LLMResponse(content="429 rate limit", finish_reason="error"),
|
||||||
|
LLMResponse(
|
||||||
|
content="",
|
||||||
|
tool_calls=[
|
||||||
|
ToolCallRequest(
|
||||||
|
id="hb_1",
|
||||||
|
name="heartbeat",
|
||||||
|
arguments={"action": "run", "tasks": "check open tasks"},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
),
|
||||||
|
])
|
||||||
|
|
||||||
|
delays: list[int] = []
|
||||||
|
|
||||||
|
async def _fake_sleep(delay: int) -> None:
|
||||||
|
delays.append(delay)
|
||||||
|
|
||||||
|
monkeypatch.setattr(asyncio, "sleep", _fake_sleep)
|
||||||
|
|
||||||
|
service = HeartbeatService(
|
||||||
|
workspace=tmp_path,
|
||||||
|
provider=provider,
|
||||||
|
model="openai/gpt-4o-mini",
|
||||||
|
)
|
||||||
|
|
||||||
|
action, tasks = await service._decide("heartbeat content")
|
||||||
|
|
||||||
|
assert action == "run"
|
||||||
|
assert tasks == "check open tasks"
|
||||||
|
assert provider.calls == 2
|
||||||
|
assert delays == [1]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_decide_prompt_includes_current_time(tmp_path) -> None:
|
||||||
|
"""Phase 1 user prompt must contain current time so the LLM can judge task urgency."""
|
||||||
|
|
||||||
|
captured_messages: list[dict] = []
|
||||||
|
|
||||||
|
class CapturingProvider(LLMProvider):
|
||||||
|
async def chat(self, *, messages=None, **kwargs) -> LLMResponse:
|
||||||
|
if messages:
|
||||||
|
captured_messages.extend(messages)
|
||||||
|
return LLMResponse(
|
||||||
|
content="",
|
||||||
|
tool_calls=[
|
||||||
|
ToolCallRequest(
|
||||||
|
id="hb_1", name="heartbeat",
|
||||||
|
arguments={"action": "skip"},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
def get_default_model(self) -> str:
|
||||||
|
return "test-model"
|
||||||
|
|
||||||
|
service = HeartbeatService(
|
||||||
|
workspace=tmp_path,
|
||||||
|
provider=CapturingProvider(),
|
||||||
|
model="test-model",
|
||||||
|
)
|
||||||
|
|
||||||
|
await service._decide("- [ ] check servers at 10:00 UTC")
|
||||||
|
|
||||||
|
user_msg = captured_messages[1]
|
||||||
|
assert user_msg["role"] == "user"
|
||||||
|
assert "Current Time:" in user_msg["content"]
|
||||||
@@ -1,91 +0,0 @@
|
|||||||
import asyncio
|
|
||||||
from unittest.mock import AsyncMock, MagicMock
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from nanobot.agent.loop import AgentLoop
|
|
||||||
from nanobot.bus.events import OutboundMessage
|
|
||||||
from nanobot.bus.queue import MessageBus
|
|
||||||
from nanobot.providers.base import GenerationSettings, LLMResponse
|
|
||||||
|
|
||||||
|
|
||||||
def _make_loop(tmp_path):
|
|
||||||
bus = MessageBus()
|
|
||||||
provider = MagicMock()
|
|
||||||
provider.get_default_model.return_value = "test-model"
|
|
||||||
provider.generation = GenerationSettings(max_tokens=0)
|
|
||||||
provider.estimate_prompt_tokens.return_value = (0, "test-counter")
|
|
||||||
response = LLMResponse(content="done", tool_calls=[])
|
|
||||||
provider.chat_with_retry = AsyncMock(return_value=response)
|
|
||||||
provider.chat_stream_with_retry = AsyncMock(return_value=response)
|
|
||||||
|
|
||||||
loop = AgentLoop(
|
|
||||||
bus=bus,
|
|
||||||
provider=provider,
|
|
||||||
workspace=tmp_path,
|
|
||||||
model="test-model",
|
|
||||||
)
|
|
||||||
loop.tools.get_definitions = MagicMock(return_value=[])
|
|
||||||
return loop
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_process_direct_websocket_clears_run_status(tmp_path) -> None:
|
|
||||||
loop = _make_loop(tmp_path)
|
|
||||||
|
|
||||||
response = await loop.process_direct(
|
|
||||||
"deliver reminder",
|
|
||||||
session_key="cron:reminder-1",
|
|
||||||
channel="websocket",
|
|
||||||
chat_id="chat-1",
|
|
||||||
)
|
|
||||||
|
|
||||||
assert response is not None
|
|
||||||
assert response.content == "done"
|
|
||||||
|
|
||||||
events = []
|
|
||||||
while loop.bus.outbound_size:
|
|
||||||
events.append(await loop.bus.consume_outbound())
|
|
||||||
|
|
||||||
statuses = [
|
|
||||||
event.metadata
|
|
||||||
for event in events
|
|
||||||
if event.metadata.get("_goal_status") is True
|
|
||||||
]
|
|
||||||
assert [status["goal_status"] for status in statuses] == ["running", "idle"]
|
|
||||||
assert isinstance(statuses[0].get("started_at"), float)
|
|
||||||
assert "started_at" not in statuses[1]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_process_direct_reuses_existing_session_lock(tmp_path) -> None:
|
|
||||||
loop = _make_loop(tmp_path)
|
|
||||||
loop._connect_mcp = AsyncMock()
|
|
||||||
session_key = "api:fixed"
|
|
||||||
lock = loop._session_locks.setdefault(session_key, asyncio.Lock())
|
|
||||||
await lock.acquire()
|
|
||||||
entered = asyncio.Event()
|
|
||||||
|
|
||||||
async def _process_message(msg, **_kwargs):
|
|
||||||
entered.set()
|
|
||||||
return OutboundMessage(channel=msg.channel, chat_id=msg.chat_id, content=msg.content)
|
|
||||||
|
|
||||||
loop._process_message = _process_message
|
|
||||||
task = asyncio.create_task(loop.process_direct("direct", session_key=session_key))
|
|
||||||
try:
|
|
||||||
await asyncio.sleep(0)
|
|
||||||
assert not entered.is_set()
|
|
||||||
|
|
||||||
lock.release()
|
|
||||||
response = await asyncio.wait_for(task, timeout=1.0)
|
|
||||||
|
|
||||||
assert entered.is_set()
|
|
||||||
assert response is not None
|
|
||||||
assert response.content == "direct"
|
|
||||||
finally:
|
|
||||||
if lock.locked():
|
|
||||||
lock.release()
|
|
||||||
if not task.done():
|
|
||||||
task.cancel()
|
|
||||||
with pytest.raises(asyncio.CancelledError):
|
|
||||||
await task
|
|
||||||
@@ -17,7 +17,6 @@ from nanobot.session.webui_turns import (
|
|||||||
WEBUI_SESSION_METADATA_KEY,
|
WEBUI_SESSION_METADATA_KEY,
|
||||||
WEBUI_TITLE_METADATA_KEY,
|
WEBUI_TITLE_METADATA_KEY,
|
||||||
WebuiTurnCoordinator,
|
WebuiTurnCoordinator,
|
||||||
clean_generated_title,
|
|
||||||
maybe_generate_webui_title,
|
maybe_generate_webui_title,
|
||||||
)
|
)
|
||||||
from nanobot.utils.llm_runtime import LLMRuntime
|
from nanobot.utils.llm_runtime import LLMRuntime
|
||||||
@@ -54,11 +53,6 @@ def test_agent_loop_llm_runtime_reflects_current_provider_and_model(tmp_path: Pa
|
|||||||
assert runtime.model == "next-model"
|
assert runtime.model == "next-model"
|
||||||
|
|
||||||
|
|
||||||
def test_clean_generated_title_strips_reasoning_tags() -> None:
|
|
||||||
assert clean_generated_title("<think>reasoning</think> WebUI polish") == "WebUI polish"
|
|
||||||
assert clean_generated_title("Title: <think> The user said hello") == ""
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_generate_webui_title_only_for_marked_webui_sessions(tmp_path: Path) -> None:
|
async def test_generate_webui_title_only_for_marked_webui_sessions(tmp_path: Path) -> None:
|
||||||
loop = _make_full_loop(tmp_path)
|
loop = _make_full_loop(tmp_path)
|
||||||
@@ -608,17 +602,17 @@ async def test_process_message_uses_explicit_session_metadata_for_goal_context(
|
|||||||
chat_session = loop.sessions.get_or_create("websocket:chat-with-goal")
|
chat_session = loop.sessions.get_or_create("websocket:chat-with-goal")
|
||||||
chat_session.metadata[GOAL_STATE_KEY] = {
|
chat_session.metadata[GOAL_STATE_KEY] = {
|
||||||
"status": "active",
|
"status": "active",
|
||||||
"objective": "This chat goal must not leak into system.",
|
"objective": "This chat goal must not leak into heartbeat.",
|
||||||
}
|
}
|
||||||
loop.sessions.save(chat_session)
|
loop.sessions.save(chat_session)
|
||||||
system_session = loop.sessions.get_or_create("system")
|
system_session = loop.sessions.get_or_create("heartbeat")
|
||||||
system_session.metadata = {}
|
system_session.metadata = {}
|
||||||
loop.sessions.save(system_session)
|
loop.sessions.save(system_session)
|
||||||
|
|
||||||
loop.context.build_messages = MagicMock( # type: ignore[method-assign]
|
loop.context.build_messages = MagicMock( # type: ignore[method-assign]
|
||||||
return_value=[
|
return_value=[
|
||||||
{"role": "system", "content": "system"},
|
{"role": "system", "content": "system"},
|
||||||
{"role": "user", "content": "runtime + system"},
|
{"role": "user", "content": "runtime + heartbeat"},
|
||||||
]
|
]
|
||||||
)
|
)
|
||||||
loop._run_agent_loop = AsyncMock(return_value=( # type: ignore[method-assign]
|
loop._run_agent_loop = AsyncMock(return_value=( # type: ignore[method-assign]
|
||||||
@@ -626,7 +620,7 @@ async def test_process_message_uses_explicit_session_metadata_for_goal_context(
|
|||||||
[],
|
[],
|
||||||
[
|
[
|
||||||
{"role": "system", "content": "system"},
|
{"role": "system", "content": "system"},
|
||||||
{"role": "user", "content": "runtime + system"},
|
{"role": "user", "content": "runtime + heartbeat"},
|
||||||
{"role": "assistant", "content": "ok"},
|
{"role": "assistant", "content": "ok"},
|
||||||
],
|
],
|
||||||
"stop",
|
"stop",
|
||||||
@@ -636,11 +630,11 @@ async def test_process_message_uses_explicit_session_metadata_for_goal_context(
|
|||||||
result = await loop._process_message(
|
result = await loop._process_message(
|
||||||
InboundMessage(
|
InboundMessage(
|
||||||
channel="websocket",
|
channel="websocket",
|
||||||
sender_id="system",
|
sender_id="heartbeat",
|
||||||
chat_id="chat-with-goal",
|
chat_id="chat-with-goal",
|
||||||
content="system work",
|
content="heartbeat work",
|
||||||
),
|
),
|
||||||
session_key="system",
|
session_key="heartbeat",
|
||||||
)
|
)
|
||||||
|
|
||||||
assert result is not None
|
assert result is not None
|
||||||
|
|||||||
@@ -2,39 +2,12 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
|
||||||
from contextlib import AsyncExitStack
|
|
||||||
from typing import Any
|
|
||||||
from unittest.mock import MagicMock
|
from unittest.mock import MagicMock
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from nanobot.agent.loop import AgentLoop
|
from nanobot.agent.loop import AgentLoop
|
||||||
from nanobot.agent.tools import mcp as mcp_runtime
|
|
||||||
from nanobot.agent.tools.base import Tool
|
|
||||||
from nanobot.bus.queue import MessageBus
|
from nanobot.bus.queue import MessageBus
|
||||||
from nanobot.config.loader import load_config, save_config
|
|
||||||
from nanobot.config.schema import MCPServerConfig
|
|
||||||
|
|
||||||
|
|
||||||
class _FakeMcpTool(Tool):
|
|
||||||
def __init__(self, name: str) -> None:
|
|
||||||
self._name = name
|
|
||||||
|
|
||||||
@property
|
|
||||||
def name(self) -> str:
|
|
||||||
return self._name
|
|
||||||
|
|
||||||
@property
|
|
||||||
def description(self) -> str:
|
|
||||||
return "fake MCP tool"
|
|
||||||
|
|
||||||
@property
|
|
||||||
def parameters(self) -> dict[str, Any]:
|
|
||||||
return {"type": "object", "properties": {}}
|
|
||||||
|
|
||||||
async def execute(self, **_kwargs: Any) -> str:
|
|
||||||
return "ok"
|
|
||||||
|
|
||||||
|
|
||||||
def _make_loop(tmp_path, *, mcp_servers: dict | None = None) -> AgentLoop:
|
def _make_loop(tmp_path, *, mcp_servers: dict | None = None) -> AgentLoop:
|
||||||
@@ -69,152 +42,3 @@ async def test_connect_mcp_retries_when_no_servers_connect(tmp_path, monkeypatch
|
|||||||
assert attempts == 2
|
assert attempts == 2
|
||||||
assert loop._mcp_connected is False
|
assert loop._mcp_connected is False
|
||||||
assert loop._mcp_stacks == {}
|
assert loop._mcp_stacks == {}
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_reload_mcp_servers_adds_and_removes_tools_without_restart(
|
|
||||||
tmp_path,
|
|
||||||
monkeypatch: pytest.MonkeyPatch,
|
|
||||||
):
|
|
||||||
config_path = tmp_path / "config.json"
|
|
||||||
monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path)
|
|
||||||
config = load_config()
|
|
||||||
config.tools.mcp_servers["browserbase"] = MCPServerConfig(
|
|
||||||
type="stdio",
|
|
||||||
command="browserbase-mcp",
|
|
||||||
)
|
|
||||||
save_config(config)
|
|
||||||
|
|
||||||
closed: list[str] = []
|
|
||||||
|
|
||||||
async def _mark_closed(name: str) -> None:
|
|
||||||
closed.append(name)
|
|
||||||
|
|
||||||
async def _fake_connect(servers, registry):
|
|
||||||
stacks = {}
|
|
||||||
for name in servers:
|
|
||||||
registry.register(_FakeMcpTool(f"mcp_{name}_navigate"))
|
|
||||||
stack = AsyncExitStack()
|
|
||||||
await stack.__aenter__()
|
|
||||||
stack.push_async_callback(_mark_closed, name)
|
|
||||||
stacks[name] = stack
|
|
||||||
return stacks
|
|
||||||
|
|
||||||
monkeypatch.setattr("nanobot.agent.tools.mcp.connect_mcp_servers", _fake_connect)
|
|
||||||
loop = _make_loop(tmp_path, mcp_servers={})
|
|
||||||
|
|
||||||
added = await mcp_runtime.reload_servers(loop, loop.tools)
|
|
||||||
|
|
||||||
assert added["ok"] is True
|
|
||||||
assert added["added"] == ["browserbase"]
|
|
||||||
assert loop.tools.has("mcp_browserbase_navigate")
|
|
||||||
assert "browserbase" in loop._mcp_stacks
|
|
||||||
|
|
||||||
config = load_config()
|
|
||||||
del config.tools.mcp_servers["browserbase"]
|
|
||||||
save_config(config)
|
|
||||||
|
|
||||||
removed = await mcp_runtime.reload_servers(loop, loop.tools)
|
|
||||||
|
|
||||||
assert removed["ok"] is True
|
|
||||||
assert removed["removed"] == ["browserbase"]
|
|
||||||
assert not loop.tools.has("mcp_browserbase_navigate")
|
|
||||||
assert "browserbase" not in loop._mcp_stacks
|
|
||||||
assert closed == ["browserbase"]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_request_mcp_reload_reaches_runtime_control_without_restart(
|
|
||||||
tmp_path,
|
|
||||||
monkeypatch: pytest.MonkeyPatch,
|
|
||||||
):
|
|
||||||
config_path = tmp_path / "config.json"
|
|
||||||
monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path)
|
|
||||||
config = load_config()
|
|
||||||
config.tools.mcp_servers["browserbase"] = MCPServerConfig(
|
|
||||||
type="stdio",
|
|
||||||
command="browserbase-mcp",
|
|
||||||
)
|
|
||||||
save_config(config)
|
|
||||||
|
|
||||||
closed: list[str] = []
|
|
||||||
|
|
||||||
async def _mark_closed(name: str) -> None:
|
|
||||||
closed.append(name)
|
|
||||||
|
|
||||||
async def _fake_connect(servers, registry):
|
|
||||||
stacks = {}
|
|
||||||
for name in servers:
|
|
||||||
registry.register(_FakeMcpTool(f"mcp_{name}_navigate"))
|
|
||||||
stack = AsyncExitStack()
|
|
||||||
await stack.__aenter__()
|
|
||||||
stack.push_async_callback(_mark_closed, name)
|
|
||||||
stacks[name] = stack
|
|
||||||
return stacks
|
|
||||||
|
|
||||||
monkeypatch.setattr("nanobot.agent.tools.mcp.connect_mcp_servers", _fake_connect)
|
|
||||||
loop = _make_loop(tmp_path, mcp_servers={})
|
|
||||||
|
|
||||||
async def _handle_one_runtime_control() -> None:
|
|
||||||
msg = await loop.bus.consume_inbound()
|
|
||||||
handled = await mcp_runtime.handle_runtime_control(loop, msg, loop.tools)
|
|
||||||
assert handled is True
|
|
||||||
|
|
||||||
consumer = asyncio.create_task(_handle_one_runtime_control())
|
|
||||||
result = await mcp_runtime.request_mcp_reload(loop.bus, timeout=2.0)
|
|
||||||
await consumer
|
|
||||||
|
|
||||||
assert result["ok"] is True
|
|
||||||
assert result["added"] == ["browserbase"]
|
|
||||||
assert result["requires_restart"] is False
|
|
||||||
assert loop.tools.has("mcp_browserbase_navigate")
|
|
||||||
|
|
||||||
config = load_config()
|
|
||||||
del config.tools.mcp_servers["browserbase"]
|
|
||||||
save_config(config)
|
|
||||||
|
|
||||||
consumer = asyncio.create_task(_handle_one_runtime_control())
|
|
||||||
result = await mcp_runtime.request_mcp_reload(loop.bus, timeout=2.0)
|
|
||||||
await consumer
|
|
||||||
|
|
||||||
assert result["ok"] is True
|
|
||||||
assert result["removed"] == ["browserbase"]
|
|
||||||
assert result["requires_restart"] is False
|
|
||||||
assert not loop.tools.has("mcp_browserbase_navigate")
|
|
||||||
assert closed == ["browserbase"]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_reload_mcp_servers_retries_configured_server_without_live_stack(
|
|
||||||
tmp_path,
|
|
||||||
monkeypatch: pytest.MonkeyPatch,
|
|
||||||
):
|
|
||||||
config_path = tmp_path / "config.json"
|
|
||||||
monkeypatch.setattr("nanobot.config.loader._current_config_path", config_path)
|
|
||||||
config = load_config()
|
|
||||||
config.tools.mcp_servers["browserbase"] = MCPServerConfig(
|
|
||||||
type="stdio",
|
|
||||||
command="browserbase-mcp",
|
|
||||||
)
|
|
||||||
save_config(config)
|
|
||||||
|
|
||||||
async def _fake_connect(servers, registry):
|
|
||||||
stacks = {}
|
|
||||||
for name in servers:
|
|
||||||
registry.register(_FakeMcpTool(f"mcp_{name}_navigate"))
|
|
||||||
stack = AsyncExitStack()
|
|
||||||
await stack.__aenter__()
|
|
||||||
stacks[name] = stack
|
|
||||||
return stacks
|
|
||||||
|
|
||||||
monkeypatch.setattr("nanobot.agent.tools.mcp.connect_mcp_servers", _fake_connect)
|
|
||||||
loop = _make_loop(tmp_path, mcp_servers={"browserbase": config.tools.mcp_servers["browserbase"]})
|
|
||||||
|
|
||||||
result = await mcp_runtime.reload_servers(loop, loop.tools)
|
|
||||||
|
|
||||||
assert result["ok"] is True
|
|
||||||
assert result["added"] == []
|
|
||||||
assert result["changed"] == []
|
|
||||||
assert result["retried"] == ["browserbase"]
|
|
||||||
assert loop.tools.has("mcp_browserbase_navigate")
|
|
||||||
await loop.close_mcp()
|
|
||||||
|
|||||||
@@ -346,26 +346,6 @@ class TestSyncWorkspaceTemplates:
|
|||||||
content = (workspace / "AGENTS.md").read_text()
|
content = (workspace / "AGENTS.md").read_text()
|
||||||
assert content == "existing content"
|
assert content == "existing content"
|
||||||
|
|
||||||
def test_does_not_create_tools_md(self, tmp_path):
|
|
||||||
"""Tool contract is injected internally, not copied into user workspaces."""
|
|
||||||
workspace = tmp_path / "workspace"
|
|
||||||
|
|
||||||
added = sync_workspace_templates(workspace, silent=True)
|
|
||||||
|
|
||||||
assert "TOOLS.md" not in added
|
|
||||||
assert not (workspace / "TOOLS.md").exists()
|
|
||||||
|
|
||||||
def test_preserves_existing_tools_md_without_overwriting(self, tmp_path):
|
|
||||||
"""Legacy user workspaces may have TOOLS.md; sync should leave it untouched."""
|
|
||||||
workspace = tmp_path / "workspace"
|
|
||||||
workspace.mkdir(parents=True)
|
|
||||||
tools_path = workspace / "TOOLS.md"
|
|
||||||
tools_path.write_text("custom tool notes", encoding="utf-8")
|
|
||||||
|
|
||||||
sync_workspace_templates(workspace, silent=True)
|
|
||||||
|
|
||||||
assert tools_path.read_text(encoding="utf-8") == "custom tool notes"
|
|
||||||
|
|
||||||
def test_creates_memory_directory(self, tmp_path):
|
def test_creates_memory_directory(self, tmp_path):
|
||||||
"""Should create memory directory structure."""
|
"""Should create memory directory structure."""
|
||||||
workspace = tmp_path / "workspace"
|
workspace = tmp_path / "workspace"
|
||||||
|
|||||||
@@ -78,31 +78,6 @@ async def test_llm_error_not_appended_to_session_messages():
|
|||||||
assert assistant_msgs[-1]["content"] == _PERSISTED_MODEL_ERROR_PLACEHOLDER
|
assert assistant_msgs[-1]["content"] == _PERSISTED_MODEL_ERROR_PLACEHOLDER
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_llm_arrearage_error_surfaces_clear_message():
|
|
||||||
"""Arrearage errors yield a clear user-facing message, not a raw dump (#3006)."""
|
|
||||||
from nanobot.agent.runner import AgentRunSpec, AgentRunner, _ARREARAGE_ERROR_MESSAGE
|
|
||||||
|
|
||||||
provider = MagicMock(spec=LLMProvider)
|
|
||||||
provider.chat_with_retry = AsyncMock(return_value=LLMResponse(
|
|
||||||
content="HTTP 402 insufficient_quota", finish_reason="error", error_status_code=402,
|
|
||||||
))
|
|
||||||
tools = MagicMock()
|
|
||||||
tools.get_definitions.return_value = []
|
|
||||||
|
|
||||||
runner = AgentRunner(provider)
|
|
||||||
result = await runner.run(AgentRunSpec(
|
|
||||||
initial_messages=[{"role": "user", "content": "hello"}],
|
|
||||||
tools=tools,
|
|
||||||
model="test-model",
|
|
||||||
max_iterations=5,
|
|
||||||
max_tool_result_chars=_MAX_TOOL_RESULT_CHARS,
|
|
||||||
))
|
|
||||||
|
|
||||||
assert result.stop_reason == "error"
|
|
||||||
assert result.final_content == _ARREARAGE_ERROR_MESSAGE
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_runner_tool_error_sets_final_content():
|
async def test_runner_tool_error_sets_final_content():
|
||||||
from nanobot.agent.runner import AgentRunSpec, AgentRunner
|
from nanobot.agent.runner import AgentRunSpec, AgentRunner
|
||||||
|
|||||||
@@ -241,7 +241,7 @@ def test_inline_fallback_reasoning_effort_does_not_inherit_primary() -> None:
|
|||||||
signature = provider_signature(config)
|
signature = provider_signature(config)
|
||||||
fallback_signatures = signature[-1]
|
fallback_signatures = signature[-1]
|
||||||
|
|
||||||
assert fallback_signatures[0][12] is None
|
assert fallback_signatures[0][11] is None
|
||||||
|
|
||||||
|
|
||||||
# -- FallbackProvider tests --
|
# -- FallbackProvider tests --
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user