ReAct: Synergizing Reasoning and Acting

Chapter 1: The ReAct Loop

ReAct produces a trajectory of interleaved thoughts and actions. A thought is a free-text reasoning trace — the model thinking out loud. An action is a call to an external tool (search, lookup, finish). An observation is the tool's response, fed back into the context. Then another thought, then another action, until the task is complete.

The action space in the original ReAct paper (Wikipedia-based tasks) is three actions: Search[entity] retrieves the Wikipedia intro for entity; Lookup[term] finds the next occurrence of a term in the current page; and Finish[answer] terminates with an answer. Thoughts are unconstrained free text. The model decides everything in natural language — when to think, when to act, and what to say in each.

Chapter 2: Thought → Action → Observation — Interactive

Let's watch ReAct solve a real multi-hop question step by step. The agent must answer: "Were Scott Derrickson and Ed Wood involved in the same genre of films?"

You control the pace. Each step reveals the next thought (reasoning), action (tool call), or observation (tool response). Watch how the thoughts guide the actions and how each observation updates the plan.

python (pseudocode)
def react_agent(question, tools, max_steps=10):
    context = few_shot_examples + f"Question: {question}\n"

    for step in range(max_steps):
        # Model generates next thought/action
        output = llm.generate(context, stop=["\nObservation"])

        if "Finish[" in output:
            answer = parse_finish(output)
            return answer

        # Parse and execute the action
        action_type, arg = parse_action(output)  # e.g., Search[Paris]
        obs = tools[action_type](arg)  # call Wikipedia API, etc.

        # Append thought+action+observation to context
        context += output + f"\nObservation: {obs}\n"

    return "Max steps reached"

Chapter 3: vs Chain-of-Thought

Chain-of-Thought (CoT) prompting is the most direct baseline: ask the model to "think step by step" before answering. It works well for math problems and logical deductions where the model's training contains the relevant knowledge. For factual multi-hop questions, CoT's Achilles heel is hallucination.

ReAct addresses this with two mechanisms:

• Grounding: Every factual claim can be verified by a tool call. Instead of "I recall that X was directed by Y," the model calls Search[X] and reads the actual Wikipedia answer.
• Revision: When a search contradicts the working hypothesis, the thought step explicitly acknowledges this: "Observation says Y, which contradicts my earlier assumption that Z. Let me reconsider." Pure CoT has no mechanism for this — it committed to its first answer in one shot.

Interestingly, ReAct can be combined with CoT: the CoT → ReAct strategy uses CoT when the model is confident (straightforward reasoning) and switches to ReAct when it needs to verify a fact. The paper calls this the "self-consistency + ReAct" combination and shows it achieves the best of both worlds.

Chapter 4: vs Act-Only

The other baseline: a pure tool-using agent with no reasoning traces. For each step, the model just outputs an action directly. Call it Act-only. It can call the same tools as ReAct but produces no thoughts between observations.

Act-only's failure modes are the mirror image of CoT's:

• No plan: Without articulating what it's looking for, the agent tends to call tools in a pattern-matched way ("I got partial info, so I'll search again") rather than deciding based on the task's logical structure.
• No memory of intent: After 5 tool calls, the model may have forgotten what sub-question it was trying to answer. The observation from step 5 gets processed in context but without a thought connecting it back to the original question.
• Loops: Act-only agents often get stuck in search loops — they search a term, don't find what they want, search a related term, don't find it, search the first term again. Without articulating "I already searched X and it didn't help," they repeat.

Chapter 5: Tool Integration

ReAct is tool-agnostic. The paper demonstrates it on Wikipedia search, but the pattern applies to any tool with a text input and text output: calculators, code interpreters, database queries, web browsers, calendar APIs, sensor readings. The model generates an action in natural language; a parser routes it to the right tool; the tool returns an observation in natural language; the observation goes back into context.

The Original Tool Suite (Wikipedia Tasks)

• Search[entity] → Returns first paragraph of entity's Wikipedia article (truncated to 100 words)
• Lookup[keyword] → Returns next sentence containing keyword in the current article (like browser Ctrl+F)
• Finish[answer] → Terminates the episode with the final answer string

Tool Design for ReAct

Good tools for ReAct agents have three properties:

• Text in, text out: The model generates and reads natural language. Tools that return structured JSON or binary data need a text serializer.
• Deterministic + fast: The agent may call a tool multiple times. Stochastic tools (random responses) confuse the reasoning trace. Slow tools (10+ seconds) make the trace feel interactive but break user experience at scale.
• Graceful failure messages: If Search[X] finds nothing, return "No Wikipedia article found for X" — not an empty string. The thought step needs something to reason about even when the tool fails.

Chapter 6: Results

ReAct was evaluated on four benchmarks using PaLM-540B and GPT-3 with few-shot prompting. No fine-tuning — just in-context examples of the thought-action-observation pattern.

The pattern is clear: ReAct outperforms Act-only on tasks that require multi-step reasoning (ALFWorld, FEVER). CoT + ReAct combination consistently beats both pure methods. The one exception is WebShop, where Act-only slightly outperforms ReAct — the paper attributes this to WebShop's action space being large and continuous, where generating thoughts adds noise without providing useful grounding.

Chapter 7: Connections

ReAct is one of the foundational papers in what is now called agentic AI — LLMs that act in the world, not just generate text. Understanding its lineage and descendants maps the field.