Symptom
-------
LLM requests intermittently fail with:
'utf-8' codec can't encode characters in position N-N+1: surrogates not allowed
when messages contain emoji-heavy content (e.g. HTML with mixed emoji + JSON round-trips).
This blocks the affected session until the session file is quarantined.
Root cause
----------
Surrogate sanitization was only applied at the CLI entry point
(nanobot/cli/commands.py: _sanitize_surrogates). Requests entering
the LLM provider layer through other channels (Feishu, cron, webui,
tool results, memory injection) had no defensive cleaning, so any
message that happened to carry unpaired UTF-16 surrogates (from an
upstream JSON round-trip with ensure_ascii=True on ill-formed input,
memory rehydration, or third-party content) would blow up at
json.dumps -> HTTP encode time inside the provider client.
Fix
---
1. Extract sanitize_surrogates() and sanitize_surrogates_deep() into
nanobot/utils/helpers.py as the single source of truth. Both use
utf-16-le round-tripping with errors='surrogatepass' / 'replace',
so paired surrogates reconstruct back into their real code point
and lone surrogates collapse to U+FFFD.
2. Make nanobot/cli/commands.py:_sanitize_surrogates a thin wrapper
that re-exports the shared helper (backward compatible).
3. Add defense-in-depth at the LLM provider boundary in
nanobot/providers/base.py:_sanitize_empty_content by running
sanitize_surrogates_deep over each message and its content blocks
right before requests are serialized to JSON.
Non-goals
---------
- truncate_text() is intentionally left untouched. Python str slicing
cannot split a single code point into surrogate halves, so it is
not the source of lone surrogates.
- session/manager storage layer is untouched. Archived sessions
reproduced the failure only through the request path, not through
storage.
Verification
------------
- New regression suite tests/providers/test_sanitize_surrogates.py
covers: paired surrogate reconstruction, lone surrogate replacement,
identity return on clean input (zero allocation), deep recursion on
dict/list/tuple, provider _sanitize_empty_content integration, and
full utf-8 encodability of the sanitized request body.
- 14/14 new tests pass; full existing test module also green.
- Replayed 58 archived real session messages plus adversarial
lone-surrogate injection through the provider path with no encode
errors after the fix.
Impact
------
- No behaviour change for clean inputs (sanitize_surrogates_deep is
an identity return when no surrogate is present).
- Fails-safe: unpaired surrogates degrade to U+FFFD instead of
aborting the entire request.
A global reaper can race asyncio's child watcher for just-exited
create_subprocess_exec children, causing Process returncode 255 instead
of the real command status. Keep only owned-PID reaps after wait/communicate
/kill paths in shell and exec_session.
Addresses review feedback on #4840.
The previous fix (dbcc7cb5) only added os.waitpid() to _kill_process(),
covering the timeout/cancel path of one-shot exec. Zombies continued to
accumulate because several other exit paths never reaped children:
- _ExecSession.kill(): sent SIGKILL + process.wait(5s) but had no
os.waitpid() fallback if the wait timed out
- ExecTool.execute() generic exception handler: leaked the subprocess
if communicate() raised an unexpected error
- Normal completion paths: relied entirely on asyncio's child-watcher,
which can miss exits inside Docker containers (pidfd/SIGCHLD gaps)
Changes:
- Extract _reap_pid() helper for consistent, safe os.waitpid(WNOHANG)
- Add _reap_pid() fallback to _ExecSession.kill() via try/finally
- Add _reap_pid() safety-net after normal process exit in both
ExecTool.execute() and _ExecSession.poll()
- Kill + reap subprocess in the generic except Exception handler
- Add periodic zombie reaper background task (every 30s) in the
gateway as a last line of defense
Terminals speaking the CSI-u (kitty / fixterms) keyboard protocol -- kitty,
Ghostty, WezTerm, and terminal panes that default to it -- encode Shift+Enter
as the escape sequence "\x1b[13;2u". prompt_toolkit 3.0 has no support for
that protocol and no default mapping for the sequence, so its Vt100Parser
fails to recognise it and dumps the raw bytes ("^[[13;2u") straight into the
prompt buffer. The multiline-input work kept only Alt+Enter, so on these
terminals Shift+Enter now leaks visible escape garbage into the input.
Register "\x1b[13;2u" (absent from prompt_toolkit's default ANSI_SEQUENCES,
so setdefault() overrides nothing) and bind it to insert a newline, matching
the Alt+Enter behaviour. This is best-effort: prompt_toolkit cannot negotiate
the protocol, so we only react to a CSI-u sequence a terminal already emits;
terminals that collapse Shift+Enter into plain Enter fall back to Alt+Enter,
which stays the primary shortcut.
Add a real-PromptSession regression test asserting the sequence inserts a
newline rather than leaking raw escape bytes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
maintainer edit: Align OpenAI-compatible API auth with the WebSocket channel boundary: loopback serve remains usable without a key, while wildcard binds still fail before agent initialization unless api.api_key is configured.
maintainer edit: Check api.api_key before template sync and AgentLoop construction so missing-key startup errors are not hidden by provider or workspace initialization failures.
On terminals that send a bare LF for plain Enter (WSL is the case
prompt_toolkit itself calls out), Alt+Enter arrives as ESC + LF
("\x1b\x0a" = Escape + ControlJ), not the ESC + CR the existing
"escape","enter" binding matches. The Escape was swallowed and the bare
LF hit prompt_toolkit's default submit, so the documented "universally
supported" Alt+Enter newline fallback failed on exactly the terminal path
plain Enter is preserved for.
Bind ESC + ControlJ to insert a newline too, and add a real-PromptSession
regression test covering the WSL Alt+Enter path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"\x1b[27;2;13~" (the older xterm modifyOtherKeys / rxvt encoding of
Shift+Enter) is already registered by prompt_toolkit by default -- as
Keys.ControlM, i.e. plain Enter/submit. The previous `setdefault()` call was
therefore a silent no-op against it: this Shift+Enter variant kept behaving
like a submit instead of inserting a newline, even after the ControlJ/WSL
fix, since setdefault only sets missing keys.
Assign directly to override that default for both known Shift+Enter
sequences, since inserting a newline is the whole point of the binding.
Add a regression test driving a real PromptSession/parser with this exact
sequence, since a mocked-buffer test can't observe prompt_toolkit's default
ANSI_SEQUENCES entries taking priority.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Keys.ControlJ is also the literal LF byte ("\x0a") that some terminals send
for a plain Enter keypress -- prompt_toolkit's own default bindings handle
this by re-feeding it as ControlM/submit, and calls out WSL by name as the
case that needs it. Binding our Shift+Enter handler to ControlJ shadowed
that default, so on any terminal sending LF for Enter, pressing Enter only
ever inserted a newline and the prompt could never be submitted.
Register the CSI-u Shift+Enter sequences against Keys.ControlF3 instead: an
enum member prompt_toolkit declares but never wires to a default ANSI
sequence or key binding, so it's only reachable through our own mapping.
Also add a regression test that drives a real PromptSession/Vt100Parser
with a raw LF byte -- the existing key-binding test invoked handlers
directly against a mocked buffer, which exercises the handler logic but not
prompt_toolkit's key-resolution precedence, so it couldn't have caught this.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The interactive prompt used a single-line buffer (multiline=False), so
there was no way to compose a multi-line message before submitting.
Switch to a multiline-capable buffer with custom key bindings: Enter still
submits (unchanged feel), Alt+Enter always inserts a newline, and
Shift+Enter inserts a newline on terminals that emit a distinguishable
CSI-u sequence for it (kitty, iTerm2 with the fixterms protocol) by mapping
those sequences onto the otherwise-unused ControlJ key.
fix(gateway): self-heal state file PID on server startup
After /restart on Windows, the gateway process gets a new PID (os.execv
creates a new process on Windows), but the state file at
run/gateway.<suffix>.json still contains the old PID from the initial
background spawn. Nothing rewrites it, leaving state inconsistent.
Add GatewayRuntime.refresh_state_pid() — a classmethod that reads the
existing state file, updates the PID and identity to the current
process, and writes atomically. Call it early in _run_gateway() so the
state file is always correct regardless of how the process was started
(initial spawn, os.execv, or subprocess.Popen).
On POSIX os.execv preserves the PID, so this is a no-op in normal
operation there, but still beneficial after any unusual restart path.
Fixes#4511
@
* fix(dream): ground commit messages and cursor advance in the real git diff
Dream consolidation could emit a /dream-log audit record that did not match
the actual file changes: build_dream_commit_message appended the LLM's
unverified resp.content, dream_run_completed only checked the stop reason, and
file contents were deliberately omitted from the prompt. The combination let a
single-turn self-report become the durable audit record.
- gitstore: add summarize_working_tree() — a structured, machine-derived
summary (per-file +N/-M, totals, capped unified diff) of working-tree
changes vs HEAD. Pure filesystem/git ground truth, never LLM narrative.
- memory: build_dream_commit_message now takes the diff body instead of resp;
dream_content_diff() exposes the real delta over SOUL/USER/MEMORY.md only
(excludes .dream_cursor so cursor writes aren't mistaken for edits);
build_dream_prompt embeds current file contents so the model edits reality,
not a stale mental model.
- builtin/cli: both Dream paths now compute the diff, gate cursor advance on
a non-empty delta (no-op runs no longer swallow history), and commit with
the diff-grounded message. Non-git workspaces fall back to the completion
check.
- dream.md: document that contents are embedded, and add a chain-of-
verification guardrail so the model's summary cannot claim unmade edits.
A regression test proves a lying resp.content never reaches the audit log
while the real diff does.
* fix(dream): mark non-UTF-8 memory files as binary in diff summary
Address review feedback (Q1 on PR #4673): summarize_working_tree read
working-tree files with errors="replace", which would emit U+FFFD
replacement chars into the audit record if a memory file ever held
invalid UTF-8 — misrepresenting the diff it is meant to make truthful.
Switch to errors="strict" and catch UnicodeDecodeError: a non-UTF-8
(or binary/corrupt) file is now recorded as "{path}: binary or
non-UTF-8 file changed" and omitted from the unified diff, so the
audit record stays honest. An empty diff block is also suppressed when
all changes are binary.
Adds a defensive regression test asserting no replacement char leaks.
