A language model forgets everything the moment a turn ends. So how does your assistant remember your name, your preferences, last week's decision? It writes them down — and learns to find them again. This lesson builds the memory layer: a file-backed hierarchy, hybrid search, and a “dreaming” process that consolidates what matters.
Picture the model at the heart of your agent as a stateless brain. You give it a pile of text — the prompt — and it gives you back some text. That's the entire transaction. The model does not quietly file away “the user's name is Sam” in some hidden vault for next time. When the turn ends, that knowledge is simply gone. Ask it tomorrow and, unless you tell it again, it has never heard of you.
This is easy to miss, because a chat feels like it remembers. But the feeling is an illusion produced by re-feeding the recent conversation back into the prompt every turn. Stretch the conversation long enough, or start a fresh session next week, and the illusion breaks: the brain is exactly as blank as it was on turn one. There is no persistence inside the model. None.
Why files, and why Markdown? Because the alternatives are worse for this job. A hidden binary blob you can't inspect breeds distrust — you'd never know what your agent “believes” about you. Markdown is the opposite: human-readable, hand-editable, diff-able in git, openable in any text editor. If the agent records something wrong, you open the file and fix the line. The memory is yours, sitting in a folder, in a format you can read.
So the agent's whole relationship with the past is: write down what matters, find it again when it's relevant, inject it into the prompt. Three verbs — write, find, inject — and a background process that decides what's worth promoting from scratch notes into permanent record. Get those right and a stateless model behaves like it has known you for years. Get them wrong and it forgets your name every Monday.
If memory is files, the first design question is: which files? Dumping everything into one giant document doesn't work — it would be too big to load every turn, and it would mix “the user's birthday” (true forever) with “we're debugging the build right now” (true for an afternoon). So OpenClaw splits memory into three stores, each with a different lifespan and a different job.
The crucial design decision here is the split between short-term and long-term, and it mirrors how human memory works. You don't carry every detail of every day in permanent storage — most of it is working memory that fades. A few things prove important enough to consolidate into long-term recall. The daily notes are the agent's working memory; MEMORY.md is its long-term memory; and the dreaming process (later) is the bridge that moves the worthy few from one to the other.
Why not just auto-inject the daily notes too? Because they're verbose and they pile up. A week of working notes is thousands of words of half-relevant scratch — the build is fixed, the meeting is over. Forcing all of it into every prompt would crowd out the actual conversation and waste the context budget (Chapter 4). So daily notes earn their place differently: they sit in the search index, retrieved on demand when a query actually touches them, rather than shoved in by default.
MEMORY.md. A transient note about today (“waiting on the API key from IT”) → the daily file. A record of what the agent decided to remember and why → DREAMS.md. The rule of thumb: if it'll still matter next month, it belongs in long-term; if it's about this afternoon, it's working memory.
This three-store shape is the skeleton everything else hangs on. The next chapters are all about the two hard problems it creates: how do you find the right memory among many files (recall), and how do you decide what's worth promoting from working notes into the permanent MEMORY.md (dreaming)?
Now the hard half: recall. The agent has a question — “what did we decide about the gateway host?” — and somewhere across MEMORY.md and a month of daily notes is the answer. The tool that finds it is memory_search, and the key design choice is that it does not rely on a single way of matching. It runs two retrieval paths in parallel and merges them. This is hybrid search.
E_AUTH_4031 wants the note containing that literal string, not something “semantically near” it.The two legs cover each other's blind spots, and that's the entire point. Vector search is brilliant at paraphrase but slippery on exact tokens — an error code is just a meaningless jumble of characters to an embedding model, so it has no good vector. BM25 is the reverse: rock-solid on exact tokens but blind to synonyms — ask it for “host” and it won't find “machine.” Run only one and you have a known, gaping failure mode. Run both and merge, and each leg rescues the other.
How do you combine two score lists into one ranking? A weighted merge: for each note, take a blend of its vector score and its keyword score, merged = w·vector + (1−w)·keyword, and rank by the blend. The weight w tilts the search toward semantics or toward literal matching. And there's a robustness bonus baked in: if one path is down — the embedding service is unreachable, say — the other leg still returns useful results. Two independent paths mean memory degrades instead of failing.
One query, a handful of notes. Each note shows three bars: its vector (semantic) score, its keyword (BM25) score, and the merged blend that decides the ranking. Toggle Typo the query: a misspelling collapses the keyword leg but vector still matches by meaning. Toggle Exact-code query: now keyword nails the note with the literal error code while vector drifts. Either leg alone misses a case; the merge catches both.
Let's build memory_search's scoring core ourselves: the two scorers and the weighted merge that turns them into one ranking.
memory_search run BOTH a vector (semantic) and a BM25 (keyword) search instead of just one?Hybrid search finds relevant notes. But two more problems sit on top of raw relevance, and a good memory has to solve both. The first: a note that was burningly relevant three months ago may be stale now. The second: the top results are often near-duplicates of each other, so you get five copies of the same fact and learn nothing new. The fixes are temporal decay and MMR.
MEMORY.md is exempt — durable facts are evergreen by definition, so “your timezone is Pacific” never decays just because you set it long ago.
Decay matters because relevance and recency are different axes. A note can be a perfect lexical match for your query yet describe a problem you solved in March. Without decay, that stale-but-matching note crowds out the fresher note that's actually about your current situation. The half-life is the dial that says “all else equal, prefer the recent.” A month is a sensible default: long enough that last week's notes stay strong, short enough that last quarter's fade.
The second problem is subtler. Suppose your top five results are all variations of “the gateway host is the Mac mini.” They're all relevant — but four of them are redundant, and they've squeezed out a different useful note that ranked sixth. Pure relevance ranking is greedy and repetitive. You want results that are relevant and varied: cover the query from several angles, don't repeat.
λ·relevance(i) − (1−λ)·max-similarity(i, chosen). The first pick has nothing to be similar to, so it's just the most relevant note. Every pick after that is pulled toward novelty — a note nearly identical to one already chosen gets a big penalty and loses to a fresher angle. The dial λ trades relevance against diversity; at λ=1 the penalty vanishes and MMR collapses back to plain top-K.
Together these two reshape the raw search into a good answer set: decay keeps it current, MMR keeps it diverse. Let's implement MMR — the selection loop that turns a relevance list plus a similarity matrix into a non-redundant pick.
MEMORY.md is exempt from decay. Why the exemption?We can store memories and we can find them. The last mile is getting them into the prompt, and that mile has a hard wall: the context budget. The model can only read so many tokens per turn. Memory has to share that space with the system instructions, the conversation, and the tools — so you cannot simply paste everything you remember and hope.
The chapter-1 split already does most of the work here. MEMORY.md — small, curated, durable — goes into the bootstrap prompt at session start, so the agent always carries its long-term facts. The daily notes do not ride along every turn; they're indexed and pulled in only when a query touches them. That's a budget decision: inject the few evergreen facts always, fetch the verbose working notes on demand.
MEMORY.md itself grows past what the budget allows this turn? It is truncated in context — only the part that fits is pasted into this prompt. But the file on disk stays whole. Nothing is deleted. The dropped portion is still right there in the file, reachable on demand through memory_get when the agent needs it. The prompt sees a window; the disk keeps the book.
This distinction — the in-context view versus the on-disk truth — is the whole trick to living within a budget without amnesia. A naive system that “trims memory to fit” by editing the file would quietly destroy real knowledge every time the budget got tight. OpenClaw never touches the file; it only chooses how much of it to show this turn. Show less when space is tight, fetch more when needed, but always keep the full record safe on disk.
So the agent has two ways to reach a memory. The always-on path: MEMORY.md injected (possibly truncated) into the bootstrap prompt, free of charge every turn. And the on-demand path: memory_search to find something and memory_get to pull the full text of a specific note or the truncated tail of MEMORY.md. Cheap things are always present; expensive things are a tool call away. That is how a finite window holds an unbounded memory.
MEMORY.md exceeds this turn's context budget, so only part of it is pasted into the prompt. What happened to the rest?So far every memory has been an inert fact: your timezone, the host machine, a decision. But some memories are not just facts — they are permissions and constraints that the agent might act on, and those carry extra weight. A memory that says “you may deploy to production” isn't trivia; it's an authorization. Recall it wrongly and the agent does something it shouldn't.
The danger is sharp and specific: a stale permission must not be acted on. Suppose three weeks ago you said “go ahead and deploy whenever you're ready,” meaning that afternoon. If memory_search happily surfaces that line today as a live authorization, the agent might deploy on its own — acting on a permission that expired long ago. The fact is still recorded; what's no longer true is that it's actionable.
So recall for action-sensitive memories must do something ordinary fact-recall doesn't: it must respect expiry. When the agent considers acting on a remembered permission, the memory layer checks whether that permission is still within its valid window. An expired approval is treated as “this was once granted, but it has lapsed” — useful as context, never as a green light. The boundary between remembering a permission and acting on it is exactly where expiry lives.
This connects to a subtler class of memory: inferred commitments — promises the agent picks up implicitly (“I'll follow up tomorrow”) rather than being explicitly told. These too have timing baked in: a commitment to do something “tomorrow” is meaningful tomorrow and meaningless next month. Treating timing as a first-class property of a memory — not an afterthought — is what keeps an agent from acting on yesterday's authority today.
memory_search surfaces that line. What must action-sensitive recall do?We have the bridge to build now: how does a worthy fact climb from the noisy pile of daily notes into the permanent MEMORY.md? Doing it during a live conversation would be expensive and distracting. So OpenClaw does it the way brains do — offline, in the quiet hours. It's opt-in and runs on a schedule, by default around 3 AM. OpenClaw calls it dreaming, and it runs in three phases.
MEMORY.md. Only Deep writes durable memory. The other two phases prepare; Deep commits.The discipline that only one phase writes is what keeps dreaming safe to run unattended. Light and REM can be re-run, interrupted, or thrown away with no consequence — they touch nothing permanent. All the irreversible action is concentrated in Deep, behind explicit thresholds, so a half-finished dream never leaves a corrupted MEMORY.md.
Deep doesn't promote on a hunch. It scores each candidate with six weighted signals — relevance, frequency, query diversity, recency, consolidation, and conceptual richness — each normalized to a 0-to-1 strength, combined into one weighted score. Relevance and frequency carry the most weight; conceptual richness the least. A memory that's been recalled often, asked about in many different ways, and is recent will score high; a one-off mentioned once will score low.
minScore), a minimum recall count (minRecall — it must have been retrieved at least this many times), and a minimum number of distinct queries that found it (minUnique — proving it's broadly useful, not just hammered by one repeated query). Pass the score and all three gates, and only then does Deep promote it to MEMORY.md.
The gates exist because score alone is gameable. A note recalled fifty times by the same repeated query looks frequent, but it's narrow — the minUnique gate catches that. A note with a great relevance score but recalled only once isn't yet proven useful — the minRecall gate holds it back. The signals propose; the gates dispose. Let's build the Deep-phase decision itself.
minScore but was recalled only once (below minRecall). Is it promoted?Everything so far — the three files, hybrid search, decay, MMR, dreaming — describes the behavior of memory. But where the index and embeddings physically live is a separate, swappable choice. OpenClaw's memory store is pluggable: the same memory layer can sit on top of several different backends, and you pick the one that fits your setup.
The reason to make the store pluggable is the same reason the surfaces were adapters back in Lesson 1: the memory layer's logic shouldn't care where the bytes live. Decay, MMR, the dreaming gates — those run identically whether the index is in a SQLite file or a LanceDB store. Swap the backend and the behavior you learned in this lesson is unchanged; only the storage engine underneath differs. Start on SQLite for free, graduate to a sidecar or a service when you need cross-agent or cross-session reach.
One backend deserves special mention because it changes the shape of memory, not just its storage. The Memory-Wiki plugin structures durable knowledge as a provenance vault: instead of loose facts, it stores claims paired with their evidence — “the gateway host is the Mac mini” alongside where that was established. It's Obsidian-compatible, so the vault is a folder of linked Markdown notes you can browse and edit like a personal wiki.
Here is the whole memory life-cycle in one widget. Short-term signals accumulate across simulated days as the agent recalls notes — bars grow by how often and how diversely each candidate is retrieved. Press Run a day to advance time and let recalls pile up. Then press Dream (Deep phase): every candidate is scored by the six weighted signals, and those that clear all three gates animate up into the MEMORY.md panel — promoted to long-term. The rest stay short-term and decay. Drag the min score and min recall gates to watch the promotion bar move.
Bottom: short-term candidates, each bar's height = its current weighted score (grows with recalls, shrinks with decay each day). Top: the MEMORY.md long-term panel. Run a day accumulates recalls and decays the rest; Dream scores everyone and promotes those past both gate lines. Tighten a gate and fewer make it through.
MEMORY.md, daily working notes, and the DREAMS.md diary (Ch 0–1). Recall is hybrid: vector + BM25, weighted-merged, each leg covering the other's blind spot (Ch 2). Ranking adds temporal decay (30-day half-life, MEMORY.md exempt) and MMR diversity (Ch 3). Injection lives under a budget: MEMORY.md rides the bootstrap prompt, truncated-in-context-never-on-disk, daily notes fetched on demand (Ch 4). Action-sensitive memories carry expiry — a lapsed permission must not be acted on (Ch 5). Dreaming consolidates offline: Light → REM → Deep, with Deep scoring six signals past three gates before promoting (Ch 6). And the store is pluggable — SQLite by default, up to a provenance vault (Ch 7).
MEMORY.md rides into the model inside the bootstrap prompt this lesson built — the prompt assembly there is where injection actually happens.