chengyongru 0e754d2591 feat(multimodal): re-implement audio/video support on main ...

Re-implements PR #2908's generalized multimodal support on the
post-FSM main branch:

- Audio input: detects WAV/MP3/OGG/FLAC via magic bytes, sends
  input_audio blocks to compatible providers, falls back to
  [audio: path] placeholder when unsupported.
- Video input: sends video_url data-URI blocks to compatible
  providers, falls back to [video: path] placeholder.
- InputLimitsConfig: count limits (images/audios/videos) and byte
  limits per media type.
- AgentDefaults: pattern-matched vision_models, audio_models,
  video_models with supports_*() helpers.
- Provider retry: strips all media types (image_url, input_audio,
  video_url) on non-transient errors and retries once.
- Feishu: extracts media tags from post messages.
- Anthropic & OpenAI Responses converters handle audio/video.

Tests: 17 new multimodal tests + existing suite passes.

2026-05-20 11:47:01 +08:00

..

agent

feat(multimodal): re-implement audio/video support on main

2026-05-20 11:47:01 +08:00

channels

feat(multimodal): re-implement audio/video support on main

2026-05-20 11:47:01 +08:00

cli

Merge PR #3877: feat(webui+agent): optimize streaming, activity rendering, and runtime sync

2026-05-18 02:04:36 +08:00

command

feat(goal): /goal command & long-running tasks (long_task)

2026-05-16 01:14:11 +08:00

config

fix(config): reduce max_tokens and context_window_tokens in schema

2026-05-15 17:19:47 +00:00

cron

feat(tools): introduce plugin-based tool discovery and runtime context protocol

2026-05-12 11:28:20 +08:00

heartbeat

fix(heartbeat): prevent internal reasoning leaks and finalization fallback in delivery

2026-04-27 18:14:13 +08:00

pairing

docs(pairing): address reviewer comments — comments, error msg, __all__ test

2026-05-15 15:46:44 +08:00

providers

feat(multimodal): re-implement audio/video support on main

2026-05-20 11:47:01 +08:00

security

fix(security): add ssrfWhitelist config to unblock Tailscale/CGNAT (#2669)

2026-04-04 19:43:18 +08:00

session

fix(agent): align LLM wall timeout with sustained goals for main + subagents

2026-05-16 16:33:49 +08:00

tools

refactor(image-generation): introduce provider registry to eliminate manual wiring

2026-05-19 15:35:19 +08:00

utils

Merge remote-tracking branch 'origin/main' into codex/review-pr-3894

2026-05-19 23:19:33 +08:00

test_api_attachment.py

fix: two bugs in document extraction pipeline

2026-04-14 13:15:04 +00:00

test_api_stream.py

Fix API stream lifecycle for tool-backed requests

2026-05-01 19:42:52 +08:00

test_build_status.py

fix(status): correct context percentage calculation and sync consolidator

2026-04-16 20:30:39 +08:00

test_context_documents.py

fix: two bugs in document extraction pipeline

2026-04-14 13:15:04 +00:00

test_docker.sh

docs: add core agent line count script and update README with real-time stats

2026-02-06 07:28:39 +00:00

test_document_parsing.py

fix(utils): recurse into PPTX groups and tables when extracting text (#3250)

2026-04-18 12:30:42 +08:00

test_msteams.py

chore: remove dead code identified by vulture + coverage cross-validation

2026-05-12 20:52:48 +08:00

test_nanobot_facade.py

refactor(tools): remove GlobTool

2026-05-15 17:19:00 +08:00

test_openai_api.py

feat(api): add SSE streaming for /v1/chat/completions Wire up the existing on_stream/on_stream_end callbacks from process_direct() to emit OpenAI-compatible SSE chunks when stream=true. Non-streaming path is untouched.

2026-04-17 01:54:49 +08:00

test_package_version.py

fix(version): fall back to pyproject in source checkouts

2026-04-08 23:47:36 +08:00

test_tool_contextvars.py

feat(tools): introduce plugin-based tool discovery and runtime context protocol

2026-05-12 11:28:20 +08:00

test_truncate_text_shadowing.py

fix(agent): avoid truncate_text name shadowing

2026-04-10 17:36:31 +08:00