nanobot/tests/tools/test_exec_allow_patterns.py
chengyongru 5853d5dfda
fix: allow_patterns take priority over deny_patterns in ExecTool (#3594)
* fix: allow_patterns take priority over deny_patterns in ExecTool

Previously deny_patterns were checked first with no bypass, meaning
allow_patterns could never exempt commands from the built-in deny list.
This made it impossible to whitelist destructive commands for specific
directories (e.g. build/cleanup tasks).

Changes:
- shell.py: check allow_patterns first; if matched, skip deny check
- shell.py: deny_patterns now appends to built-in list (not replaces)
- schema.py: add allow_patterns/deny_patterns to ExecToolConfig
- loop.py/subagent.py: pass allow_patterns/deny_patterns to ExecTool
- Add test_exec_allow_patterns.py covering priority semantics

* fix: separate deny pattern errors from workspace violation detection

The deny pattern error message "Command blocked by safety guard" was
included in _WORKSPACE_BLOCK_MARKERS, causing deny_pattern blocks to be
misclassified as fatal workspace violations. This meant LLMs had no
chance to retry with a different command — the turn was aborted
immediately.

Changes:
- shell.py: deny/allowlist error messages now use distinct phrasing
  ("blocked by deny pattern filter" / "blocked by allowlist filter")
- runner.py: remove "blocked by safety guard" from
  _WORKSPACE_BLOCK_MARKERS so deny_pattern errors are treated as normal
  tool errors (LLM can retry) instead of fatal violations
- workspace path errors still use "blocked by safety guard" and remain
  fatal as intended

* fix: update test assertions to match new deny pattern error message

* fix: indentation error in test file

* fix: restore SSRF fatal classification and tidy exec pattern plumbing

Address review feedback on the deny/allow_patterns rework:

- runner.py: re-add "internal/private url detected" to
  _WORKSPACE_BLOCK_MARKERS. The earlier marker removal also stripped
  fatal classification from SSRF / internal-URL rejections (whose
  message still says "blocked by safety guard"), turning a hard
  security boundary into something the LLM could retry.
- loop.py / subagent.py: drop `or None` between ExecToolConfig and
  ExecTool. The schema default is an empty list and ExecTool already
  normalizes None back to [], so the indirection was a no-op.
- shell.py: extract `explicitly_allowed` flag in _guard_command so
  allow_patterns are scanned once instead of twice and the control
  flow no longer relies on a no-op `pass + else` branch.
- tests/agent/test_runner.py: add a regression test asserting that
  the SSRF block message is treated as fatal, while deny/allowlist
  filter messages are deliberately non-fatal.

* fix: remove unused exec allow-pattern test import

Keep the new ExecTool allow-pattern coverage clean under ruff.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Xubin Ren <xubinrencs@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-03 00:27:17 +08:00

59 lines
2.1 KiB
Python

"""Tests for allow_patterns priority over deny_patterns."""
from __future__ import annotations
from nanobot.agent.tools.shell import ExecTool
def test_deny_patterns_block_rm_rf():
"""Baseline: rm -rf is blocked by default deny list."""
tool = ExecTool()
result = tool._guard_command("rm -rf /tmp/build", "/tmp")
assert result is not None
assert "deny pattern filter" in result.lower()
def test_allow_patterns_bypass_deny():
"""allow_patterns take priority: matching command skips deny check."""
tool = ExecTool(allow_patterns=[r"rm\s+-rf\s+/tmp/"])
result = tool._guard_command("rm -rf /tmp/build", "/tmp")
assert result is None
def test_allow_patterns_must_match_to_bypass():
"""Non-matching allow_patterns do NOT bypass deny."""
tool = ExecTool(allow_patterns=[r"rm\s+-rf\s+/opt/"])
result = tool._guard_command("rm -rf /tmp/build", "/tmp")
assert result is not None
assert "deny pattern filter" in result.lower()
def test_extra_deny_patterns_from_config():
"""User-supplied deny patterns are appended to built-in list."""
tool = ExecTool(deny_patterns=[r"\bping\b"])
# ping is blocked by extra deny
assert tool._guard_command("ping example.com", "/tmp") is not None
# rm -rf still blocked by built-in deny
assert tool._guard_command("rm -rf /tmp/x", "/tmp") is not None
def test_allow_patterns_bypass_extra_deny():
"""allow_patterns also bypasses user-supplied deny patterns."""
tool = ExecTool(
deny_patterns=[r"\bping\b"],
allow_patterns=[r"\bping\s+example\.com\b"],
)
result = tool._guard_command("ping example.com", "/tmp")
assert result is None
def test_allow_patterns_is_whitelist_only():
"""When allow_patterns is set, non-matching non-denied commands are blocked."""
tool = ExecTool(allow_patterns=[r"\becho\b"])
# echo matches allow → ok
assert tool._guard_command("echo hello", "/tmp") is None
# ls does not match allow and is not in deny → blocked by allowlist
result = tool._guard_command("ls /tmp", "/tmp")
assert result is not None
assert "allowlist" in result.lower()