nanobot/docs/memory.md
chengyongru d1a94dae8a
refactor(dream): replace two-phase Dream class with simple cron + process_direct (#3990)
* refactor(dream): replace two-phase Dream class with simple cron + process_direct

- Remove the heavyweight Dream class (AgentRunner-based two-phase system)
  from nanobot/agent/memory.py
- Delete dream_phase1.md and dream_phase2.md templates
- New dream.md template serves as the consolidation prompt
- Cron callback uses agent.process_direct(prompt, session_key=\"dream\")
  instead of agent.dream.run()
- Always performs git auto_commit after execution
- /dream command updated to use process_direct + git commit
- DreamConfig kept for backward compatibility; deprecated fields
  (model_override, max_batch_size, max_iterations, annotate_line_ages)
  are ignored but accepted in config
- interval_h remains configurable via agents.defaults.dream.interval_h
- Update tests and webui settings to match new architecture

* feat(loop): add ephemeral mode to process_direct, skip history writes for Dream

When ephemeral=True, _state_save skips enforce_file_cap (which calls
raw_archive -> append_history) and consolidator.maybe_consolidate_by_tokens.
This prevents Dream sessions from creating a positive feedback loop where
they process their own output. The session IS still saved to disk.

* fix(loop): skip extra hooks for ephemeral sessions (Dream)

* feat(dream): per-run timestamped sessions with rotation for WebUI

* test(config): restore DreamConfig schedule and alias tests

* fix(dream): include LLM response summary in git auto-commit message

The old two-phase Dream class included the Phase 1 analysis in the git
commit message body. The new single-phase version lost this. Restore it
by extracting resp.content from the process_direct return value and
appending it to the commit message in both the cron handler and the
/dream command.

* fix(test): accept ephemeral kwarg in test_openai_api fake_process

* refactor(dream): merge dream_session.py into MemoryStore

The standalone dream_session.py module only contained three small helpers
that all revolve around MemoryStore concerns (session keys, commit messages,
file pruning). Fold them into MemoryStore as @staticmethod to reduce
indirection and avoid a 35-line module with no independent reason to exist.

* fix(test): address code review — patch correct instance, use actual function

- Fix test_ephemeral_skips_raw_archive to patch loop.context.memory
  instead of the fixture's separate MemoryStore instance
- Fix TestDreamCommitMessage to call MemoryStore.build_dream_commit_message
  instead of reimplementing the logic inline
- Move Dream helpers in memory.py above the Consolidator section comment
  to avoid misleading visual boundary

* fix(dream): gate cursor advancement and restrict tools

maintainer edit: Dream now processes backlog from the oldest unprocessed entries, only advances the cursor after a completed ephemeral run, and uses a restricted file-only tool registry for background consolidation.

* fix(dream): skip idle compact for dream sessions

Dream runs use internal dream:* sessions that are pruned by Dream retention. Exclude them from AutoCompact scheduling, archive execution, and summary injection so idle-session compaction cannot truncate Dream transcripts.

* fix(dream): keep batched history isolated

* feat(dream): tag archived memory for single-phase Dream

---------

Co-authored-by: Xubin Ren <52506698+Re-bin@users.noreply.github.com>
2026-06-02 22:46:47 +08:00

6.0 KiB

Memory in nanobot

nanobot's memory is built on a simple belief: memory should feel alive, but it should not feel chaotic.

Good memory is not a pile of notes. It is a quiet system of attention. It notices what is worth keeping, lets go of what no longer needs the spotlight, and turns lived experience into something calm, durable, and useful.

That is the shape of memory in nanobot.

The Design

nanobot does not treat memory as one giant file.

It separates memory into layers, because different kinds of remembering deserve different tools:

  • session.messages holds the living short-term conversation.
  • memory/history.jsonl is the running archive of compressed past turns.
  • SOUL.md, USER.md, and memory/MEMORY.md are the durable knowledge files.
  • GitStore records how those durable files change over time.

This keeps the system light in the moment, but reflective over time.

The Flow

Memory moves through nanobot in two stages.

Stage 1: Consolidator

When a conversation grows large enough to pressure the context window, nanobot does not try to carry every old message forever.

Instead, the Consolidator summarizes the oldest safe slice of the conversation and appends that summary to memory/history.jsonl.

This file is:

  • append-only
  • cursor-based
  • optimized for machine consumption first, human inspection second

Each line is a JSON object:

{"cursor": 42, "timestamp": "2026-04-03 00:02", "content": "- User prefers dark mode\n- Decided to use PostgreSQL"}

It is not the final memory. It is the material from which final memory is shaped.

Stage 2: Dream

Dream is the slower, more thoughtful layer. It runs on a cron schedule by default and can also be triggered manually.

Dream reads:

  • new entries from memory/history.jsonl
  • the current SOUL.md
  • the current USER.md
  • the current memory/MEMORY.md

Then it edits the long-term files surgically in a single pass — not by rewriting everything, but by making the smallest honest change that keeps memory coherent.

This is why nanobot's memory is not just archival. It is interpretive.

The Files

workspace/
├── SOUL.md              # The bot's long-term voice and communication style
├── USER.md              # Stable knowledge about the user
└── memory/
    ├── MEMORY.md        # Project facts, decisions, and durable context
    ├── history.jsonl    # Append-only history summaries
    ├── .cursor          # Consolidator write cursor
    ├── .dream_cursor    # Dream consumption cursor
    └── .git/            # Version history for long-term memory files

These files play different roles:

  • SOUL.md remembers how nanobot should sound.
  • USER.md remembers who the user is and what they prefer.
  • MEMORY.md remembers what remains true about the work itself.
  • history.jsonl remembers what happened on the way there.

Why history.jsonl

The old HISTORY.md format was pleasant for casual reading, but it was too fragile as an operational substrate.

history.jsonl gives nanobot:

  • stable incremental cursors
  • safer machine parsing
  • easier batching
  • cleaner migration and compaction
  • a better boundary between raw history and curated knowledge

You can still search it with familiar tools:

# grep
grep -i "keyword" memory/history.jsonl

# jq
cat memory/history.jsonl | jq -r 'select(.content | test("keyword"; "i")) | .content' | tail -20

# Python
python -c "import json; [print(json.loads(l).get('content','')) for l in open('memory/history.jsonl','r',encoding='utf-8') if l.strip() and 'keyword' in l.lower()][-20:]"

The difference is philosophical as much as technical:

  • history.jsonl is for structure
  • SOUL.md, USER.md, and MEMORY.md are for meaning

Commands

Memory is not hidden behind the curtain. Users can inspect and guide it.

Command What it does
/dream Run Dream immediately
/dream-log Show the latest Dream memory change
/dream-log <sha> Show a specific Dream change
/dream-restore List recent Dream memory versions
/dream-restore <sha> Restore memory to the state before a specific change

These commands exist for a reason: automatic memory is powerful, but users should always retain the right to inspect, understand, and restore it.

Versioned Memory

After Dream changes long-term memory files, nanobot can record that change with GitStore.

This gives memory a history of its own:

  • you can inspect what changed
  • you can compare versions
  • you can restore a previous state

That turns memory from a silent mutation into an auditable process.

Configuration

Dream is configured under agents.defaults.dream:

{
  "agents": {
    "defaults": {
      "dream": {
        "intervalH": 2,
        "modelOverride": null,
        "maxBatchSize": 20,
        "maxIterations": 10
      }
    }
  }
}
Field Meaning
intervalH How often Dream runs, in hours
cron Cron expression override (takes precedence over intervalH)
modelOverride Optional Dream-specific model override (pending implementation)
maxBatchSize (Deprecated — not used)
maxIterations (Deprecated — not used)

In practical terms:

  • intervalH is the normal way to configure Dream frequency. Internally it runs as an every schedule.
  • cron overrides intervalH when set, allowing precise cron expressions (e.g. 0 */4 * * *).
  • modelOverride is reserved for a future release. Currently Dream uses the same model as the main agent.
  • maxBatchSize and maxIterations are preserved for config compatibility but no longer affect behavior.

In Practice

What this means in daily use is simple:

  • conversations can stay fast without carrying infinite context
  • durable facts can become clearer over time instead of noisier
  • the user can inspect and restore memory when needed

Memory should not feel like a dump. It should feel like continuity.

That is what this design is trying to protect.