AI Harness Engineering · OpenClaw 2 of 7 · Anatomy of an Agent Gateway

The Wire Protocol — One Socket, Three Frames

Lesson 1 gave us a hub with many clients. Now: what language do they speak? One WebSocket, a tiny grammar of three frame types, a strict handshake, and a few rules — idempotency, sequence numbers — that make a chatty real-time system reliable. You'll build the router that powers it.

Prerequisites: you did Lesson 1 (the gateway). You know what JSON is and that a WebSocket is a two-way pipe. A little Python for the labs. No networking background assumed.
8
Chapters
2
Simulations
3
Code Labs

Chapter 0: Why One Protocol Over One Socket

This lesson in one sentence. The gateway and its clients talk over a single WebSocket using a tiny typed grammar of just three message shapes. By the end you'll know every field on the wire, why each one exists, and you'll have written the router that decides what to do with each incoming message.

Lesson 1 left us with a hub. One daemon owns the chat-app connections; your phone, your terminal, and the desktop app are all thin clients of it. But "they connect to the gateway" begs the real question: what do they actually say to each other? Two programs sharing a pipe is not enough — they need a shared protocol, an agreed set of message shapes and rules. That protocol is this lesson.

Start by asking what the gateway has to do, and notice that ordinary web requests can't do it. A normal web call is request/response: the client asks, the server answers, the exchange ends. That works beautifully for "show me the chat history." But the gateway's whole reason to exist is that things happen on their own — a WhatsApp message lands, the agent finishes a thought, a node drops offline. The server needs to tell the client the instant it happens, with nobody having asked.

There's a clumsy way to fake this with plain request/response, called polling: the client asks "anything new?" over and over, every second or two. It works, but it's wasteful and slow. Most asks come back empty — pure overhead — and a real message that arrives just after an ask has to wait for the next one. The simulation below makes the cost visible.

Polling vs. pushing: the same message, two delivery models

Left, the polling client fires an "anything new?" request on a fixed timer; most come back empty (wasted), and when a real message arrives it sits idle until the next poll fires (that's the latency bar). Right, with a push socket the server delivers the message the instant it arrives — near-zero latency, zero wasted asks. Drag the poll interval and watch the gap grow.

poll interval (s)2.0

The fix is a WebSocket: a single, long-lived, two-way pipe. Once it's open, either side can send a message at any time without being asked. The client can still make requests, and the server can answer them — but the server can also push unsolicited messages whenever something changes. One duplex socket carrying a small typed grammar beats a pile of polling endpoints on every axis that matters: latency, wasted traffic, and complexity.

The core decision. Use one WebSocket per client, not N request/response endpoints. The socket is duplex (both directions) and persistent (stays open). That single choice is what lets the gateway push events the moment they happen — and it's why the rest of this lesson is about designing the little language those messages are written in.

So the whole lesson is: define a grammar small enough to hold in your head, strict enough that a malformed or hostile client gets dropped at the door, and reliable enough that network blips don't double-send your messages or leave your screen quietly stale. Three frame types, one handshake, a handful of rules. Let's name them.

Why isn't ordinary request/response (or polling on top of it) good enough for the gateway?

Chapter 1: Three Frame Types — req, res, event

A WebSocket carries frames. The kind OpenClaw uses are text frames, and each one holds a single chunk of JSON — a small dictionary of key/value pairs. The protocol's whole grammar is: every frame is JSON, and every frame is exactly one of three types. That's it. Master these three shapes and you've read the wire.

The three frames.
  • Request — the client asks the gateway to do something. Shape: {type:"req", id, method, params}. The method names the action (health, chat.send); params carries its arguments; id is a number the client picks so it can match the answer later.
  • Response — the gateway's answer to exactly one request. Shape: {type:"res", id, ok, payload} on success, or {type:"res", id, ok:false, error} on failure. The id is copied from the request — that's the thread that ties an answer back to its question.
  • Event — an unsolicited message the gateway pushes on its own. Shape: {type:"event", event, payload, seq}. There is no id and nobody asked for it; it just arrives because something happened (a chat event, an agent event). The seq is a counter we'll lean on hard in Chapter 6.

Pause on what the id buys you. Because the socket is duplex and persistent, a client can fire several requests without waiting for each answer — ask for the chat list, ask for settings, ask for presence, all back to back. The answers can come back in any order. The id is how the client untangles them: it remembers "I sent id:7 for the chat list," and when a response carrying id:7 shows up, it knows exactly which question that answers. Without the id, overlapping requests would be a hopeless muddle.

And notice the deep difference between a response and an event. A response is solicited and addressed: it answers one specific request, carries that request's id, and goes to the one client who asked. An event is unsolicited and broadcast: nobody requested it, it has no id, and the gateway can fan it out to every interested client at once. That broadcast is exactly the multiplexing from Lesson 1, now visible on the wire: one inbound message becomes one event, pushed to your phone and your laptop together.

Why a typed envelope at all? Every frame begins with type so the receiver can route it with a single glance — req goes to the method dispatcher, res goes to "match it to a pending id," event goes to "update the UI." A tagged union (one field that names the variant) is the simplest possible parser: read one field, branch three ways. No guessing from shape.

Let's build that router. Given a frame off the wire, the gateway has to decide: is this a request to dispatch, a response to relay, or an event to forward — and is the client even allowed to send it yet? You'll write exactly that decision, including the handshake gate we formalize in the next chapter.

A client sends three requests back-to-back without waiting; the answers arrive out of order. How does the client know which response answers which request?

Chapter 2: The Handshake — connect First, or Get Dropped

Lesson 1 named it as Invariant 2: the very first frame a client sends must be a valid connect. Now we see exactly what's in that frame and why it's so loaded. The handshake is where two strangers agree on everything before any real work happens — which version of the protocol they speak, who the client claims to be, and whether it's allowed in at all.

The flow has three beats. First the gateway, the moment the socket opens, sends a challenge: a fresh random nonce (a one-time number) and a timestamp. Second, the client replies with its connect frame — the loaded one. Third, if everything checks out, the gateway answers hello-ok, and only now is the connection live.

What the connect frame carries.
  • Version rangeminProtocol and maxProtocol: the span of protocol versions this client can speak. The gateway picks one inside that range (the current protocol is version 4).
  • Identity & role — client metadata, the role (operator or node, from Lesson 1), requested scopes, and the node's caps if it's a capability host.
  • Device auth — proof of who this device is, including a signature over the nonce the gateway just sent. Signing the fresh nonce proves the client is answering this challenge right now, not replaying an old captured handshake.

The hello-ok reply closes the negotiation. It tells the client the negotiated version (the one both sides will use), server info, the feature list (the methods and events this gateway supports — Chapter 3's topic), the policy limits (the size and timing caps — Chapter 5), and the auth result. From that frame on, the client knows precisely what it's allowed to do and how big it can speak.

Why version negotiation, not a fixed version? The gateway and the client are shipped separately and updated on different schedules — your phone app might be three releases behind the daemon. A range (minProtocol..maxProtocol) lets a new gateway still talk to an old client by dropping to a shared version, instead of refusing the connection outright. Negotiation is how a long-lived system avoids a flag-day upgrade where everything must move at once.

One guardrail wraps the whole pre-connect dance: until the handshake completes, the gateway accepts only tiny frames — pre-connect frames are capped at sixty-four kibibytes (64 KiB). An unauthenticated stranger should not be able to make the daemon allocate a giant buffer before it has even proven who it is. Strict at the door, generous only once trust is established.

The connect frame includes a signature over the nonce the gateway just sent. Why sign that specific fresh number?

Chapter 3: Discovery as Contract — Don't Hardcode the API

Look again at that hello-ok reply. Buried in it is one of the most quietly important ideas in the whole protocol: the gateway tells the client what it can do. The features block lists features.methods (every method this gateway will accept) and features.events (every event it can push). The client reads that list instead of hardcoding a fixed set of calls.

This is discovery, and it turns the API into a contract advertised at runtime rather than a fact baked into the client at build time. Why does that matter? Because the gateway and clients evolve separately (same reason as version negotiation). A newer gateway adds a method — say chat.pin. An old client that has never heard of chat.pin simply doesn't see it in its discovery list, doesn't offer it, and carries on fine. Nothing breaks.

Forward compatibility, for free. When a client reads the method list instead of assuming it, two rules fall out automatically: a client never calls a method that isn't advertised (so a downgrade to an older gateway is safe), and a client ignores any advertised method it doesn't understand (so an upgrade to a newer gateway is safe). Old and new mix freely. Hardcoding the API throws this away — the first mismatch becomes a crash.

Where do these schemas come from, and how do they stay honest? OpenClaw defines each frame's shape once, in TypeBox — a TypeScript library where you describe a type and get both a compile-time type and a runtime JSON Schema from the same source. The JSON Schema validates every incoming frame on the wire (so a malformed params is rejected before any handler runs), and the same definitions are run through code generation to produce Swift types for the iOS and macOS apps.

One source of truth, three consumers. A single TypeBox definition feeds: the TypeScript gateway (compile-time safety), the runtime JSON-Schema validator (reject bad frames at the door), and the generated Swift structs (the Apple clients can't drift from the protocol). When the protocol changes, you change one definition and every consumer follows. This is the same "define it once, normalize everywhere" instinct as the adapters in Lesson 1 — here aimed at the wire format itself.

So discovery and schema generation are two halves of one discipline: the gateway publishes a precise, machine-checkable description of what it speaks, and clients read that description rather than guessing. The payoff is a system whose pieces can be upgraded one at a time without a coordinated flag day.

A newer gateway adds a method the client has never heard of. With discovery (the client reads hello-ok.features.methods), what happens to the old client?

Chapter 4: Idempotency & Retries — Send Once, Even on Resend

Networks are unreliable in a specific, maddening way: a request can succeed on the server while the reply gets lost on the way back. The client, having heard nothing, does the sensible thing and retries. Now the server has received the same instruction twice — and if that instruction had a side effect, it just happened twice.

For a read like health, who cares — ask it ten times, same answer. But for a side-effecting method like send (post a message) or agent (run the agent), a duplicate is a real bug: your friend gets the same message twice, or the agent runs twice and bills you twice. We need a way for the server to recognize "I've already done this exact thing" and not do it again.

The idempotency key. Every side-effecting request carries an idempotency key — a unique token the client generates once for a given logical action and reuses on every retry of that action. The gateway keeps a short-lived dedup cache mapping key → the response it produced. First time it sees a key: perform the action, store the response, return it. Any later time it sees the same key: skip the action entirely and return the cached response. The action fires exactly once; the client can retry as many times as it likes.

The word idempotent means "doing it again has no further effect" — like pressing an already-lit elevator button. The key is what lets the gateway make a naturally non-idempotent action (sending a message) behave idempotently from the client's point of view. The retry is safe precisely because the second, third, tenth attempt all collapse into the cached result of the first.

Two details make it work. The key must be the same across retries of one action (the client picks it before the first attempt and holds it) but different across genuinely new actions (so two real messages aren't mistaken for a retry). And the cache is short-lived — it only needs to outlive the retry window, not remember forever. Let's build the cache and watch the difference between "send once" and "send every time."

A client sends a message, the network drops the reply, so the client retries with the same idempotency key. With a dedup cache, what does the gateway do on the retry?

Chapter 5: Backpressure & Limits — Protecting the Daemon

The gateway is the one process everything depends on. That makes it a target: a buggy or hostile client could flood the socket with enormous frames or a torrent of messages, exhaust the daemon's memory, and take down every client at once. A real socket needs limits, and the protocol advertises them up front.

Those limits ride in the hello-ok policy block, so the client knows the rules before it does anything:

The policy limits.
  • maxPayload (about twenty-six megabytes) — the largest single frame the gateway will accept. Bigger than this and the frame is refused.
  • maxBufferedBytes (about fifty-two megabytes) — the most unsent data the gateway will hold queued for one slow client before it pushes back. This is backpressure: when a consumer can't keep up, the producer must slow down rather than buffer without bound.
  • tickIntervalMs (default fifteen seconds) — the keepalive cadence (Chapter 6).

The interesting case is what happens when a frame is too big. The naive reaction would be to log the offending frame — but the offending frame might be twenty-six megabytes, and it might contain secrets. So OpenClaw does the careful thing: an oversized frame triggers a payload.large diagnostic event that records that a limit was hit and by how much, but without echoing the body and without leaking any secret it carried. You learn the limit was crossed; you don't accidentally splash the payload into a log.

Backpressure is a correctness feature, not just a performance one. Without it, one slow client — a phone on a weak connection that can't drain events fast enough — would make the gateway buffer an unbounded backlog in memory, eventually starving every other client. The maxBufferedBytes ceiling turns "one slow consumer" into a local problem (that client gets throttled or dropped) instead of a global outage. Limits protect the shared daemon from any single misbehaving connection.

Notice the pattern: every limit is a way of bounding what one client can cost the system, and every limit is announced in hello-ok rather than discovered by trial and error. A well-behaved client reads maxPayload and chunks a big upload itself; a misbehaving one hits the ceiling and gets a clean diagnostic instead of taking the daemon down. Strictness at the door (Chapter 2) and limits in the room (here) are the same instinct: be generous to good clients, cheap to refuse bad ones.

A client sends a frame larger than maxPayload. What does the gateway emit, and crucially, what does it not include?

Chapter 6: Liveness & Gaps — Keepalive, Sequence, Refresh

A WebSocket can die quietly. The Wi-Fi drops, a phone sleeps, a router silently severs an idle connection — and the socket can look open on one side while being dead on the other. The protocol needs a way to prove the link is actually alive, and a way to recover the instant it isn't.

The proof of life is a steady keepalive: the gateway emits a tick event every tickIntervalMs (default fifteen seconds), like a heartbeat. As long as ticks keep arriving, the client knows the pipe is healthy. The rule for silence is just as concrete: if the client hears nothing for twice the tick interval, it declares the link dead, closes the socket with code 4000, and reconnects — backing off from one second up to thirty seconds between tries so a flapping network doesn't become a reconnect storm.

Every event carries a per-socket sequence number. The gateway stamps each event it pushes with a monotonic seq — 1, 2, 3, … — counting up, one per socket. The client tracks the last seq it saw. If the next event arrives with a number more than one higher (it saw 6, then 9), the client knows it missed events 7 and 8. A gap is detectable purely from the numbers, with no acknowledgements and no per-event bookkeeping.

And what does the client do about a gap? It does not ask for the lost events back — recall Invariant 3 from Lesson 1: events are never replayed. The gateway keeps no rewindable log. Instead the client does the one thing that's always correct: it refreshes, refetching the current snapshot of state. Whatever it missed, the snapshot is the real present truth, so resyncing from it can never leave the client subtly wrong.

Why refresh beats replay. A replayable per-client event log needs durable storage, retention rules, and acknowledgements — expensive and easy to get subtly wrong. "Notice a gap, refetch the truth" needs nothing but a counter the client already tracks, and it is always correct because the snapshot is real current state, not a reconstruction. You trade a little extra work on the rare gap for a system that can never quietly drift out of sync.

Let's build the gap detector itself — the tiny loop every client runs over the sequence numbers to decide when to refresh.

The client's last seen event was seq 6, and the next arrives as seq 9. What does it do, and why doesn't it ask for 7 and 8?

Chapter 7: Showcase & Connections

Here is the whole protocol in motion. The client is on the left, the gateway on the right, one socket between them. Press Send req (health) to watch a request frame travel to the gateway and a response frame come back carrying the same id. Press Push event (chat) to see the gateway push an unsolicited event with a sequence number. Flip Drop next event and push again — the sequence skips, the client detects the gap, and it flashes refresh. The last frame on the wire is shown as live JSON.

The wire, live — req/res, pushed events, and a sequence gap

Blue = the client. Orange = the gateway. A req animates right and its res comes back with the matching id. An event is pushed left with its seq. With Drop next event armed, the gateway increments seq but sends nothing — the next event arrives with a skipped number and the client refreshes.

Idle. Send a request or push an event.
// last frame on the wire appears here
The protocol in one breath. One WebSocket carries three frame types: req (client asks), res (gateway answers, same id), event (gateway pushes, with a seq) (Ch 0–1). The first frame must be a connect; it negotiates the protocol version and proves identity by signing a nonce, and hello-ok returns the negotiated version, the advertised methods/events, and the policy limits (Ch 2). Clients read that discovery list instead of hardcoding the API, so old and new mix freely (Ch 3). Side-effecting calls carry an idempotency key so retries don't double-send (Ch 4). Limits in hello-ok protect the daemon from floods (Ch 5). A tick proves liveness, and a seq gap makes the client refresh rather than replay (Ch 6).

Where each thread continues

Related lessons on this site

The Feynman test. Close the lesson and explain to a friend: why a push socket beats polling; the three frame types and what the id and seq are for; the handshake sequence and why connect must come first; how discovery makes old and new clients interoperate; why send needs an idempotency key and what a retry does; and what a client does when the sequence number skips — and why it doesn't ask for the lost event back. If you can, you own the wire.
In one sentence, what IS the wire protocol?