Add comprehensive audio and video support across the agent pipeline:
- Generalize media placeholder system: _strip_image_content → _strip_media_content,
_media_placeholder with type-specific labels, unified across providers
- Add detect_audio_mime with magic-byte detection and filename fallback
- Add _AUDIO_FORMAT_MAP for correct MIME-to-API-format conversion
- Add InputLimitsConfig with count limits (max_input_audios/videos) and byte limits
- Support input_audio blocks in context builder with OpenAI-compatible format
- Support video_url blocks with base64 inline data
- Add audio/video passthrough in Codex provider, placeholder fallback in Anthropic provider
- Thread supports_vision/audio/video capability flags through AgentLoop
- Unify placeholder format: [audio: path]/[video: path] instead of generic [file: path]
- Optimize file I/O: single read_bytes() instead of header+full double reads
- Extract _STRIP_MEDIA_TYPES as class constant to avoid per-call allocation
Merge the three retry-after header parsers (base, OpenAI, Anthropic)
into a single _extract_retry_after_from_headers on LLMProvider that
handles retry-after-ms, case-insensitive lookup, and HTTP date.
Remove the per-provider _parse_retry_after_headers duplicates and
their now-unused email.utils / time imports. Add test for retry-after-ms.
Made-with: Cursor
Register xiaomi_mimo as an OpenAI-compatible provider with its API base URL,
add xiaomi_mimo to the provider config schema, and document it in README.
Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
Replace the flatten/unflatten approach (merging extra_content.google.*
into provider_specific_fields then reconstructing) with direct pass-through:
parse extra_content as-is, store on ToolCallRequest.extra_content, serialize
back untouched. This is lossless, requires no hardcoded field names, and
covers all three parsing branches (str, dict, SDK object) plus streaming.
Preserve the provider and agent-loop streaming primitives plus the CLI experiment scaffolding so this work can be resumed later without blocking urgent bug fixes on main.
Made-with: Cursor
Replace the static provider-level supports_vision check with a
reactive fallback: when a model returns an image-unsupported error,
strip image_url blocks from messages and retry once. This avoids
maintaining an inaccurate vision capability table and correctly
handles gateway/unknown model scenarios.
Also extract _safe_chat() to deduplicate try/except boilerplate
in chat_with_retry().
- Remove trailing whitespace and normalize blank lines
- Unify string quotes and line breaks for long lines
- Sort imports alphabetically across modules
