93 Commits

Author SHA1 Message Date
hussein1362
f8a023218d fix(telegram): improve markdown rendering for modern LLM output
Problem:
Modern LLMs (GPT-5.4, Claude, Gemini) produce markdown-heavy responses with
numbered lists, headers, and nested formatting. The Telegram channel's
_markdown_to_telegram_html() converter has gaps that leave these poorly
formatted:

1. Numbered lists (1. 2. 3.) have zero handling — sent as raw text
2. Headers (# Title) are stripped to plain text, losing visual hierarchy
3. Mid-stream edits send raw markdown (users see **bold** and ### headers
   while the response generates, before the final HTML conversion)

Root Cause:
_markdown_to_telegram_html() handles bullets (- *) but skips numbered lists
entirely. Headers are stripped of # but not given any emphasis. The streaming
path in send_delta() sends buf.text as-is during mid-stream edits (plain
text, no parse_mode) — only the final _stream_end edit converts to HTML.

Fix:
1. Headers now render as <b>bold</b> in the final HTML (using placeholder
   markers that survive HTML escaping, restored after all other processing)
2. Numbered lists are normalized (extra whitespace after the dot is cleaned)
3. New _strip_md_block() function strips markdown syntax for readable
   plain-text preview during streaming mid-edits

The final _stream_end HTML conversion is unchanged — it still produces
full HTML with parse_mode=HTML. Only the intermediate edits are improved.

Tests:
Added 10 new tests covering:
- Headers converting to bold HTML
- Numbered list preservation and whitespace normalization
- Headers with HTML special characters
- Mixed formatting (headers + bullets + numbers + bold)
- _strip_md_block for inline formatting, headers, bullets, numbers, links
- Streaming mid-edit markdown stripping (initial send + edit)
2026-04-21 21:35:34 +08:00
chengyongru
f900c5bb8e fix(telegram): address code review issues from cherry-pick merge
- Fix critical plain-text fallback that was sending raw HTML tags to
  users: keep raw markdown available for the fallback path
- Extract TELEGRAM_HTML_MAX_LEN (4096) constant to replace hardcoded
  magic number and document the difference from TELEGRAM_MAX_MESSAGE_LEN
- Add fallback to _send_text for extra HTML chunks when HTML parse fails
- Add missing @pytest.mark.asyncio decorator on
  test_send_delta_stream_end_html_expansion_does_not_overflow
2026-04-20 16:58:46 +08:00
stutiredboy
2eea82f5ee fix(telegram): split oversized stream buffer mid-flight
Cherry-picked from #3311 (stutiredboy). Streaming edits called
edit_message_text(text=buf.text) without chunking, so once accumulated
deltas crossed Telegram's 4096-char limit an ongoing stream would fail
with BadRequest.

Extracts _flush_stream_overflow helper that edits the first chunk in
place, sends any middle chunks, and re-anchors the buffer to a new
message for the tail so subsequent deltas keep streaming.

Co-Authored-By: stutiredboy <stutiredboy@users.noreply.github.com>
2026-04-20 16:58:46 +08:00
himax12
fd8f08cc83 fix(telegram): convert markdown to HTML before splitting to avoid message length overflow
Cherry-picked from #3316 (himax12). When streaming completes in send_delta(),
the code was splitting raw markdown text by 4000, then converting to HTML.
The markdown-to-HTML conversion adds 10-33% characters, which could push
the result over Telegram's 4096 character limit.

The fix converts markdown to HTML first, then splits by 4096 (actual Telegram
limit), ensuring the edited message always fits.

Fixes #3315
2026-04-20 16:58:46 +08:00
Alfredo Arenas
5d976d79ff test(discord): update tests for bot-to-bot fix (#3217)
The old test `test_on_message_ignores_bot_messages` asserted the
previous (incorrect) contract that ALL bot-authored messages are
dropped. With #3217 only self-loops are dropped, so this test was
replaced with three more precise tests:

- test_on_message_ignores_self_messages: verifies self-loop guard
  (author_id == _bot_user_id is dropped)
- test_on_message_accepts_messages_from_other_bots: new test for
  the fix itself — other bots' messages flow through
- test_on_message_stops_typing_on_handle_exception: preserves the
  typing cleanup assertion from the original test

Net result: +1 behavior tested, same behaviors retained.

Co-authored with Claude Opus 4.7
2026-04-19 23:32:40 +08:00
Xubin Ren
9ed3031a42 feat(webui): add initial webui with websocket chat flow 2026-04-18 18:51:53 +00:00
Xubin Ren
6bfb75ed03 feat(websocket): multiplex multiple chat_ids over a single connection 2026-04-18 16:49:12 +08:00
Xubin Ren
70a1279b86 test: pin retry-wait callback routing so internal heartbeats stay off channels
Add two focused regression tests for the retry-wait leak this PR fixes:

- tests/agent/test_runner.py::test_runner_binds_on_retry_wait_to_retry_callback_not_progress
  locks in that `AgentRunSpec.retry_wait_callback` (not `progress_callback`) is
  what `_build_request_kwargs` forwards to the provider as `on_retry_wait`.

- tests/channels/test_channel_manager_delta_coalescing.py::TestRetryWaitFiltering
  runs `_dispatch_outbound` end-to-end and asserts that `_retry_wait: True`
  messages never reach channel send.

Both tests fail on origin/main and pass with this PR's fix applied.

Made-with: Cursor
2026-04-18 13:50:05 +08:00
Xubin Ren
3ae4333cef test(email): cover smtp_username / imap_username / case-insensitive self-address match
The original regression only exercised a from_address match with all three
identity fields set to the same value, so it couldn't distinguish whether
_self_addresses actually picks up smtp_username and imap_username or just
collapses on from_address. Add a parametrized test covering:

- smtp_username-only match (from_address empty, imap_username different) —
  simulates SMTP relays that rewrite outbound From to the login identity.
- imap_username-only match — simulates mailbox-identity setups.
- Case-insensitive match — inbound From arriving upper-cased must still hit.

No production code changes.

Made-with: Cursor
2026-04-17 16:25:16 +08:00
yorkhellen
1011ea5ac8 fix(email): ignore self-sent mailbox messages
Skip inbound emails that come from the bot's own configured addresses so a mailbox wired to the same SMTP/IMAP account does not trigger infinite reply loops.
2026-04-17 16:25:16 +08:00
Mohamed Elkholy
ce5272c153 fix(transcription): honor api_base for OpenAI transcription provider
Complete the symmetry left by #3214: ChannelManager._resolve_transcription_base
already resolves providers.openai.api_base, but BaseChannel.transcribe_audio
instantiated OpenAITranscriptionProvider without forwarding it, and the provider
__init__ did not accept the parameter. Self-hosted OpenAI-compatible Whisper
endpoints (LiteLLM, vLLM, etc.) configured via config.json were therefore
ignored for the OpenAI backend.

- OpenAITranscriptionProvider.__init__ now accepts api_base with env fallback
  (OPENAI_TRANSCRIPTION_BASE_URL) matching the Groq pattern.
- BaseChannel.transcribe_audio forwards self.transcription_api_base to OpenAI.
- Tests mirror the existing Groq coverage: manager propagation for provider
  "openai", BaseChannel-to-provider argument passing, and provider default vs
  override for api_url.

Fully backward-compatible: when api_base is None and the env var is unset,
the default https://api.openai.com/v1/audio/transcriptions is used.

Refs #3213, follow-up to #3214.
2026-04-17 13:46:51 +08:00
Xubin Ren
d57af5c1d1 test(channels): cover groq transcription api base propagation 2026-04-17 13:46:51 +08:00
Xubin Ren
459a4d7311 test(discord): cover allow_channels filtering in _should_accept_inbound
Locks in the two key boundaries of the new channel-based filter:

1. When an incoming channel id is in allow_channels, messages are forwarded.
2. When an incoming channel id is not in allow_channels, messages are
   silently dropped.

The empty-list backward-compatible path is already covered by every
existing test that omits allow_channels (default_factory=list).

Made-with: Cursor
2026-04-17 02:14:33 +08:00
dongzeyu001
6829b8b475 unit test fix 2026-04-15 16:51:02 +08:00
dongzeyu001
cbd2315d76 unit test fix 2026-04-15 16:51:02 +08:00
dongzeyu001
cf47fa7d23 add test for wecom mixed msg parse fix 2026-04-15 16:51:02 +08:00
Xubin Ren
1f33df1ea6 fix: preserve empty dict allow_from handling
Keep dict-backed channel configs compatible with both allow_from and allowFrom without losing empty-list semantics, and add focused regression coverage for the allow-list boundary.

Made-with: Cursor
2026-04-15 01:26:51 +08:00
Xubin Ren
0a51344483 fix(slack): keep cross-target sends out of origin threads
When Slack resolves a named target to another conversation, do not reuse the origin thread timestamp on the destination send, and keep reaction cleanup anchored to the source conversation.

Made-with: Cursor
2026-04-14 20:19:48 +08:00
yeyitech
873be5180b feat(slack): resolve named message targets 2026-04-14 20:19:48 +08:00
chengyongru
0adce5405b fix(feishu): remove resuming to avoid 10-min streaming card timeout
Feishu streaming cards auto-close after 10 minutes from creation,
regardless of update activity. With resuming enabled, a single card
lives across multiple tool-call rounds and can exceed this limit,
causing the final response to be silently lost.

Remove the _resuming logic from send_delta so each tool-call round
gets its own short-lived streaming card (well under 10 min). Add a
fallback that sends a regular interactive card when the final
streaming update fails.
2026-04-14 16:53:42 +08:00
bahtya
fa98524944 fix(channels): prevent retry amplification and silent message loss across channels
Audited all channel implementations for overly broad exception handling
that causes retry amplification or silent message loss during network
errors. This is the same class of bug as #3050 (Telegram _send_text).

Fixes by channel:

Telegram (send_delta):
- _stream_end path used except Exception for HTML edit fallback
- Network errors (TimedOut, NetworkError) triggered redundant plain
  text edit, doubling connection demand during pool exhaustion
- Changed to except BadRequest, matching the _send_text fix

Discord:
- send() caught all exceptions without re-raising
- ChannelManager._send_with_retry() saw successful return, never retried
- Messages silently dropped on any send failure
- Added raise after error logging

DingTalk:
- _send_batch_message() returned False on all exceptions including
  network errors — no retry, fallback text sent unnecessarily
- _read_media_bytes() and _upload_media() swallowed transport errors,
  causing _send_media_ref() to cascade through doomed fallback attempts
- Added except httpx.TransportError handlers that re-raise immediately

WeChat:
- Media send failure triggered text fallback even for network errors
- During network issues: 3×(media + text) = 6 API calls per message
- Added specific catches: TimeoutException/TransportError re-raise,
  5xx HTTPStatusError re-raises, 4xx falls back to text

QQ:
- _send_media() returned False on all exceptions
- Network errors triggered fallback text instead of retry
- Added except (aiohttp.ClientError, OSError) that re-raises

Tests: 331 passed (283 existing + 48 new across 5 channel test files)

Fixes: #3054
Related: #3050, #3053
2026-04-13 00:30:45 +08:00
bahtya
7e91aecd7d fix(telegram): narrow exception catch in _send_text to prevent retry amplification
Previously _send_text() caught all exceptions (except Exception) when
sending HTML-formatted messages, falling back to plain text even for
network errors like TimedOut and NetworkError. This caused connection
demand to double during pool exhaustion scenarios (3 retries × 2
fallback attempts = 6 calls per message instead of 3).

Now only catches BadRequest (HTML parse errors), letting network errors
propagate immediately to the retry layer where they belong.

Fixes: HKUDS/nanobot#3050
2026-04-13 00:30:45 +08:00
Dianqi Ji
ee946d96ca feat(channels/feishu): add domain config for Lark global support
Add 'domain' field to FeishuConfig (Literal['feishu', 'lark'], default 'feishu').
Pass domain to lark.Client.builder() and lark.ws.Client to support Lark global
(open.larksuite.com) in addition to Feishu China (open.feishu.cn).
Existing configs default to 'feishu' for backward compatibility.

Also add documentation for domain field in README.md and add tests for
domain config.
2026-04-12 09:56:17 +08:00
chengyongru
9f433cab01 fix(wecom): use reply_stream for progress messages to avoid errcode=40008
The plain reply() uses cmd="reply" which does not support "text" msgtype
and causes WeCom API to return errcode=40008 (invalid message type).
Unify both progress and final text messages to use reply_stream()
(cmd="aibot_respond_msg"), differentiating via finish flag.

Fixes #2999
2026-04-11 21:47:19 +08:00
chengyongru
0d03f10fa0 test(channels): add media support tests for QQ and WeCom channels
Cover helpers (sanitize_filename, guess media type), outbound send
(exception handling, media-then-text order, fallback), inbound message
processing (attachments, dedup, empty content), _post_base64file
payload filtering, and WeCom upload/download flows.
2026-04-11 21:47:19 +08:00
chengyongru
6fd2511c8a refactor(feishu): simplify tool hint to append-only, delegate to send_delta for throttling
- Make tool_hint_prefix configurable in FeishuConfig (default: 🔧)
- Delegate tool hint card updates from send() to send_delta() so hints
  automatically benefit from _STREAM_EDIT_INTERVAL throttling
- Fix staticmethod calls to use self.__class__ instead of self
- Document all supported metadata keys in send_delta docstring
- Add test for empty/whitespace-only tool hint with active stream buffer
2026-04-10 12:29:43 +08:00
xzq.xu
049ce9baae fix(tool-hints): deduplicate by formatted string + per-line inline display
Two display fixes based on real-world Feishu testing:

1. tool_hints.py: format_tool_hints now deduplicates by comparing the
   fully formatted hint string instead of tool name alone. This fixes
   `ls /Desktop` and `ls /Downloads` being incorrectly merged as
   `ls /Desktop × 2`. Truly identical calls still fold correctly.
   (_group_consecutive and all abbreviation logic preserved unchanged.)

2. feishu.py: inline tool hints now display one tool per line with
   🔧 prefix, and use double-newline trailing to prevent Setext heading
   rendering when followed by markdown `---`.

Made-with: Cursor
2026-04-10 12:29:43 +08:00
xzq.xu
512c3b88e3 fix(feishu): preserve tool hints in final card content
Tool hints should be kept as permanent content in the streaming card
so users can see which tools were called (matching the standalone card
behavior). Previously, hints were stripped when new deltas arrived or
when the stream ended, causing tool call information to disappear.

Now:
- New delta: hint becomes permanent content, delta appends after it
- New tool hint: replaces the previous hint (unchanged)
- Resuming/stream_end: hint is preserved in the final text

Updated 3 tests to verify hint preservation semantics.

Made-with: Cursor
2026-04-10 12:29:43 +08:00
xzq.xu
589e3ac36e fix(feishu): prevent tool hint stacking and clean hints on stream_end
Three fixes for inline tool hints:

1. Consecutive tool hints now replace the previous one instead of
   stacking — the old suffix is stripped before appending the new one.

2. When _resuming flushes the buffer, any trailing tool hint suffix
   is removed so it doesn't persist into the next streaming segment.

3. When final _stream_end closes the card, tool hint suffix is
   cleaned from the text before the final card update.

Adds 3 regression tests covering all three scenarios.

Made-with: Cursor
2026-04-10 12:29:43 +08:00
xzq.xu
ac1795c158 feat(feishu): streaming resuming + inline tool hints
Two improvements to Feishu streaming card experience:

1. Handle _resuming in send_delta: when a mid-turn _stream_end arrives
   with resuming=True (tool call between segments), flush current text
   to the card but keep the buffer alive so subsequent segments append
   to the same card instead of creating a new one.

2. Inline tool hints into streaming cards: when a tool hint arrives
   while a streaming card is active, append it to the card content
   (e.g. "🔧 web_fetch(...)") instead of sending a separate card.
   The hint is automatically stripped when the next delta arrives.

Made-with: Cursor
2026-04-10 12:29:43 +08:00
Xubin Ren
69d748bf8f Merge origin/main; warn on partial proxy credentials; add only-password test
- Merged latest main (no conflicts)
- Added warning log when only one of proxy_username/proxy_password is set
- Added test_start_no_proxy_auth_when_only_password for coverage parity

Made-with: Cursor
2026-04-09 23:54:11 +08:00
Jonas
7506af7104 feat(channel): add proxy support for Discord channel
- Add proxy, proxy_username, proxy_password fields to DiscordConfig
- Pass proxy and proxy_auth to discord.Client
- Add aiohttp.BasicAuth when credentials are provided
- Add tests for proxy configuration scenarios
2026-04-09 23:54:11 +08:00
Xubin Ren
ba8bce0f45 fix(tests): add missing from typing import Any in websocket integration tests
Made-with: Cursor
2026-04-09 18:22:35 +08:00
chengyongru
56a5906db5 fix(websocket): harden security and robustness
- Use hmac.compare_digest for timing-safe static token comparison
- Add issued token capacity limit (_MAX_ISSUED_TOKENS=10000) with 429 response
- Use atomic pop in _take_issued_token_if_valid to eliminate TOCTOU window
- Enforce TLSv1.2 minimum version for SSL connections
- Extract _safe_send helper for consistent ConnectionClosed handling
- Move connection registration after ready send to prevent out-of-order delivery
- Add HTTP-level allow_from check and client_id truncation in process_request
- Make stop() idempotent with graceful shutdown error handling
- Normalize path via validator instead of leaving raw value
- Default websocket_requires_token to True for secure-by-default behavior
- Add integration tests and ws_test_client helper
- Refactor tests to use shared _ch factory and bus fixture
2026-04-09 18:22:35 +08:00
Jack Lu
ad57bcd127 feat(channels): add WebSocket server channel and tests
Port Python implementation from a1ec7b192ad97ffd58250a720891ff09bbb73888
(websocket channel module and channel tests; excludes webui debug app).
2026-04-09 18:22:35 +08:00
Rohit_Dayanand123
3cc2ebeef7 Added bug fix to Dingtalk by zipping html to prevent raw failure 2026-04-09 10:49:00 +08:00
Xubin Ren
61dd5ac13a test(discord): cover streamed reply overflow
Lock the Discord streaming path with a regression test for final chunk splitting so oversized replies stay safe to merge and ship.

Made-with: Cursor
2026-04-09 00:24:11 +08:00
SHLE1
e49b6c0c96 fix(discord): enable streaming replies 2026-04-09 00:24:11 +08:00
kronk307
e21ba5f667 feat(telegram): add location/geo support
Forward static location pins as [location: lat, lon] content so the
agent can respond to geo messages and pass coordinates to MCP tools.

Closes HKUDS/nanobot#2909
2026-04-08 02:32:19 +08:00
chengyongru
b1d3c00deb test(feishu): add compatibility tests for new tool hint format 2026-04-07 15:15:07 +08:00
Xubin Ren
0355f20919 test: add regression tests for _resolve_mentions
7 tests covering: single mention, dual IDs, no-id skip, multiple mentions,
no mentions, empty text, and key-not-in-text edge case.

Made-with: Cursor
2026-04-07 14:03:55 +08:00
Ben Lenarts
d0527a8cf4 feat(email): add attachment extraction support
Save inbound email attachments to the media directory with configurable
MIME type filtering (glob patterns like "image/*"), per-attachment size
limits, and max attachment count. Filenames are sanitized to prevent
path traversal. Controlled by allowed_attachment_types — empty (default)
means disabled, non-empty enables extraction for matching types.
2026-04-06 15:09:44 +08:00
Xubin Ren
bdec2637ae test: add regression test for oversized stream-end splitting
Made-with: Cursor
2026-04-06 06:39:23 +00:00
Xubin Ren
897d5a7e58 test: add regression tests for JID suffix classification and LID cache
Made-with: Cursor
2026-04-06 06:19:06 +00:00
Xubin Ren
35dde8a30e refactor: unify voice transcription config across all channels
- Move transcriptionProvider to global channels config (not per-channel)
- ChannelManager auto-resolves API key from matching provider config
- BaseChannel gets transcription_provider attribute, no more getattr hack
- Remove redundant transcription fields from WhatsAppConfig
- Update README: document transcriptionProvider, update provider table

Made-with: Cursor
2026-04-06 06:07:30 +00:00
Xubin Ren
7b7a3e5748 fix: media_paths NameError, import order, add error logging and tests
- Move media_paths assignment before voice message handling to prevent
  NameError at runtime
- Fix broken import layout in transcription.py (httpx/loguru after class)
- Add error logging to OpenAITranscriptionProvider matching Groq style
- Add regression tests for voice transcription and no-media fallback

Made-with: Cursor
2026-04-06 06:01:14 +00:00
Xubin Ren
c88d97c652 fix: fall back to heuristic when bot open_id fetch fails
If _fetch_bot_open_id returns None the exact-match path would silently
disable all @mention detection. Restore the old heuristic as a fallback.
Add 6 unit tests for _is_bot_mentioned covering both paths.

Made-with: Cursor
2026-04-06 13:49:38 +08:00
Xubin Ren
bb9da29eff test: add regression tests for private DM thread session key derivation
Made-with: Cursor
2026-04-06 02:44:21 +08:00
Ilya Semenov
0d6bc7fc11 fix(telegram): support threads in DMs 2026-04-06 02:44:21 +08:00
chengyongru
3003cb8465 test(feishu): add unit tests for reaction add/remove and auto-cleanup 2026-04-05 16:53:05 +08:00