2 Commits

Author SHA1 Message Date
chengyongru
b9346b0d59 feat: generalize multimodal support with audio/video handling
Add comprehensive audio and video support across the agent pipeline:

- Generalize media placeholder system: _strip_image_content → _strip_media_content,
  _media_placeholder with type-specific labels, unified across providers
- Add detect_audio_mime with magic-byte detection and filename fallback
- Add _AUDIO_FORMAT_MAP for correct MIME-to-API-format conversion
- Add InputLimitsConfig with count limits (max_input_audios/videos) and byte limits
- Support input_audio blocks in context builder with OpenAI-compatible format
- Support video_url blocks with base64 inline data
- Add audio/video passthrough in Codex provider, placeholder fallback in Anthropic provider
- Thread supports_vision/audio/video capability flags through AgentLoop
- Unify placeholder format: [audio: path]/[video: path] instead of generic [file: path]
- Optimize file I/O: single read_bytes() instead of header+full double reads
- Extract _STRIP_MEDIA_TYPES as class constant to avoid per-call allocation
2026-04-08 23:14:40 +08:00
chengyongru
4fa64dc73b feat: cherry-pick InputLimitsConfig (image OOM guard) and merge with multimodal support
Cherry-pick c4c0ac8 from nightly-26-03-29 which adds InputLimitsConfig
(max_input_images, max_input_image_bytes), image size/existence checks,
and wiring through AgentLoop/CLI. Merged with existing audio/video
multimodal handling, timezone support, and supports_* capability flags.
2026-04-08 23:14:40 +08:00