convert_messages() emitted reasoning items with ``content`` as a plain
string whenever preserve_reasoning was enabled (the DeepSeek spec).
DeepSeek's Responses gateway rejects that shape with a serde error
("input: invalid type: string ..., expected a sequence"), which surfaced
only after token consolidation cleared provider_state and forced the
full-history conversion path; replayed server items already carry list
content, which is why normal multi-turn requests never failed. Serialize
reasoning content as a list of output_text parts, matching the OpenAI
Responses schema and DeepSeek's accepted wire shape (verified live against
api.deepseek.com/responses).
The serde fallback classifier introduced in the previous commit remains as
a last-resort safeguard for any remaining wire incompatibility.
Tests: extend test_preserves_deepseek_reasoning_content to the array shape;
add a full-history regression with the observed failing item, a
replay/consolidation regression covering both replayed and converted
reasoning items, and provider-level request fixtures for both paths.
Full suite: 5773 passed, 22 skipped (only the known local-only
channels/sms packaging failure remains).
DeepSeek's new Responses endpoint (deepseek-v4-flash) intermittently rejects valid request bodies with serde deserialization errors such as 'input: invalid type: string ..., expected a sequence'. These were not classified as compatibility errors, so affected conversations died instead of falling back to Chat Completions.
The wire format is correct (input serializes as a list), so this is a server-side Responses compatibility issue; Chat Completions is strictly more permissive, making fallback safe. Extend the fallback classifier to recognize serde body-parsing markers. Repeated failures still trip the existing circuit breaker.
The live v1beta API rejects the legacy responseFormat.image block
(enum-based aspectRatio/imageSize fields) for gemini-3.1-flash-lite-image
with INVALID_ARGUMENT, even for documented plain-string values. Gemini
Flash image models accept plain-string hints under
generationConfig.imageConfig instead (e.g. aspectRatio 16:9, imageSize
1K), which the API accepts. Switch the flash path to imageConfig and
update the provider tests accordingly. Other providers (aihubmix,
ollama, imagen) are untouched.
Address review feedback that the capability checks were broader than the
documented per-model matrix:
- Drop the extreme aspect ratios (1:4, 4:1, 1:8, 8:1) from the Flash
allow-list. They are only documented for 3.1 Flash / Flash Lite, so the
global set could send an unsupported ratio to 2.5 Flash Image or 3.1 Pro
Image. Keep the ratios common to every Flash image model.
- Identify imageSize support positively via "gemini-3" instead of excluding
"2.5". The old predicate also matched gemini-2.0-flash-preview-image-
generation, which (with the default 1K size) altered that model's request
shape even though only Gemini 3+ image models accept a configurable size.
Add tests for the gemini-2.0 image-size drop and the extreme-ratio drop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Gemini Flash image path (`generateContent`) dropped both `aspect_ratio`
and `image_size`: `generate()` never forwarded them and
`_generate_gemini_flash` did not accept them, so every request fell back to
1:1 / input-matched output. The Imagen path was unaffected.
Forward the hints and emit them under
`generationConfig.responseFormat.image` per the current Gemini API. Aspect
ratio is validated against the accepted set; `imageSize` is validated against
{512,1K,2K,4K} and only sent to Gemini 3+ image models, since
`gemini-2.5-flash-image` supports only `aspectRatio`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add _QWEN_THINKING_MODELS to _MODEL_THINKING_STYLES with enable_thinking style. Prevents Qwen 3.5/3.6/3.7 models from exposing raw reasoning content in chat responses. Closes#4934
Symptom
-------
LLM requests intermittently fail with:
'utf-8' codec can't encode characters in position N-N+1: surrogates not allowed
when messages contain emoji-heavy content (e.g. HTML with mixed emoji + JSON round-trips).
This blocks the affected session until the session file is quarantined.
Root cause
----------
Surrogate sanitization was only applied at the CLI entry point
(nanobot/cli/commands.py: _sanitize_surrogates). Requests entering
the LLM provider layer through other channels (Feishu, cron, webui,
tool results, memory injection) had no defensive cleaning, so any
message that happened to carry unpaired UTF-16 surrogates (from an
upstream JSON round-trip with ensure_ascii=True on ill-formed input,
memory rehydration, or third-party content) would blow up at
json.dumps -> HTTP encode time inside the provider client.
Fix
---
1. Extract sanitize_surrogates() and sanitize_surrogates_deep() into
nanobot/utils/helpers.py as the single source of truth. Both use
utf-16-le round-tripping with errors='surrogatepass' / 'replace',
so paired surrogates reconstruct back into their real code point
and lone surrogates collapse to U+FFFD.
2. Make nanobot/cli/commands.py:_sanitize_surrogates a thin wrapper
that re-exports the shared helper (backward compatible).
3. Add defense-in-depth at the LLM provider boundary in
nanobot/providers/base.py:_sanitize_empty_content by running
sanitize_surrogates_deep over each message and its content blocks
right before requests are serialized to JSON.
Non-goals
---------
- truncate_text() is intentionally left untouched. Python str slicing
cannot split a single code point into surrogate halves, so it is
not the source of lone surrogates.
- session/manager storage layer is untouched. Archived sessions
reproduced the failure only through the request path, not through
storage.
Verification
------------
- New regression suite tests/providers/test_sanitize_surrogates.py
covers: paired surrogate reconstruction, lone surrogate replacement,
identity return on clean input (zero allocation), deep recursion on
dict/list/tuple, provider _sanitize_empty_content integration, and
full utf-8 encodability of the sanitized request body.
- 14/14 new tests pass; full existing test module also green.
- Replayed 58 archived real session messages plus adversarial
lone-surrogate injection through the provider path with no encode
errors after the fix.
Impact
------
- No behaviour change for clean inputs (sanitize_surrogates_deep is
an identity return when no surrogate is present).
- Fails-safe: unpaired surrogates degrade to U+FFFD instead of
aborting the entire request.
config.loader.load_config() intentionally returns the raw config with ${VAR}
references intact — env interpolation is a separate, explicit step
(resolve_config_env_vars) so that settings read/edit/save paths never
materialize secrets to disk or to the UI.
The transcription config path does not apply that step: both
channels/base.py (channel voice notes) and webui/transcription_ws.py (WebUI
recording) build their effective config via
resolve_transcription_config(load_config()). As a result a configured
api_key of "${GROQ_API_KEY}" (the documented way to reference secrets) is
passed to the provider verbatim, which fails with 401 Invalid API Key. No
amount of rotating the real key helps, because the literal placeholder
string is what gets sent.
Resolve the reference at the single choke point both callers share —
_resolve_transcription_api_key / _resolve_transcription_api_base — using a
new lenient loader.resolve_env_refs() helper (unset var -> empty string, so
a missing variable degrades to "not configured" rather than raising or
leaking). This fixes both entry points at once and cannot drift the way a
per-call-site fix does. Resolving inside load_config() was rejected: the
~20 settings-UI callers depend on it returning raw ${VAR} placeholders.
Literal keys are unaffected; the settings API only reads the derived
`configured` flag (never the key), which now reflects the resolved value.
Claude-Session: https://claude.ai/code/session_01Q3HuVaJAAQJA3kgVQVJ2Zt
_get_copilot_access_token had a check-then-act race: concurrent chat()
calls after token expiry both fetched new tokens and clobbered each other.
Add asyncio.Lock with double-checked locking so only one fetch happens
per expiry window.
Closes#4677
Maintainer edit: add mocked coverage for the enterprise endpoint and client ID override paths, and document the environment variables users must set before OAuth login.
Widen thinking_style from Literal to str | None and add a
@field_validator that produces a helpful error message listing
valid options when an invalid value is provided.
Addresses the review feedback on #4482.
ProviderConfig.thinking_style defaults to None (Optional field), but
create_dynamic_spec expects a string. Coalesce None to "" at all call
sites (factory.py, settings_api.py) and fix the test assertion to
expect None from the config default.
