Paper2Agent: Papers as AI Agents

Chapter 1: The Key Insight

The core idea is deceptively simple: convert a research paper into a Model Context Protocol (MCP) server, then connect that server to a chat agent. The MCP server exposes the paper's methods as callable tools, its data as queryable resources, and its workflows as executable prompts.

What is MCP?

The Model Context Protocol is an open standard (created by Anthropic in late 2024) that defines how an LLM-based agent connects to external tools. Think of it as a USB port for AI — any tool that speaks MCP can be plugged into any agent that speaks MCP. No custom integration code needed.

An MCP server exposes three types of capabilities:

The Paper2Agent Pipeline

The framework has four stages, each handled by a specialized sub-agent:

The entire pipeline is orchestrated by Claude Code acting as a meta-agent — it coordinates the four sub-agents, passes context between stages, and handles failures. On a personal laptop, Paper2Agent converted the AlphaGenome paper into 22 working MCP tools in about 3 hours, and the Scanpy preprocessing pipeline into 7 tools in about 45 minutes — all without human intervention.

Chapter 2: Paper Analysis Stage

Before you can turn a paper into an agent, you need to understand what the paper does. This is the job of Stage 1: a multi-agent system that reads the paper, clones the repo, and produces a structured understanding of the codebase.

The Orchestrator and Four Sub-Agents

Paper2Agent is itself implemented as a multi-agent system inside Claude Code. An orchestrator agent coordinates four specialized sub-agents, each with a focused mandate:

What "Analysis" Actually Does

The Environment Manager does not just run pip install -r requirements.txt. It provisions an isolated workspace, analyzes which Python version the code expects, resolves version conflicts between pinned and transitive dependencies, and verifies that all imports succeed. This is the step that eliminates the two-hour dependency debugging session from Chapter 0.

The Tutorial Scanner solves a subtler problem: most repos contain many files, but only a few are genuine tutorials that demonstrate the method end-to-end. The scanner distinguishes tutorial notebooks from test files, utility scripts, and experimental code. It produces clear summaries of which resources demonstrate which capabilities.

The Six Steps in Detail

1. Locate and download the official repository linked to the paper.
2. Environment setup: provision workspace, pin dependencies, verify imports.
3. Tutorial discovery: scan for notebooks, README examples, tutorial directories.
4. Tutorial execution and audit: run selected tutorials end-to-end with their example data, capture inputs/outputs/figures, record implicit assumptions.
5. Tool extraction: convert tutorial logic into parameterized, typed functions.
6. MCP assembly: integrate tools, resources, and prompts into a deployable server.

Chapter 3: MCP Construction

The Tool Extractor sub-agent takes the executed tutorials from Stage 2 and converts each one into a clean, reusable tool. This is the most technically demanding transformation in the pipeline — turning ad-hoc notebook code into production-grade, schema-typed functions.

From Tutorial to Tool

Consider a Jupyter notebook cell that scores a genomic variant:

# Original tutorial code (AlphaGenome notebook)
from alphagenome.models import dna_client
import os
model = dna_client.create(os.getenv('ALPHAGENOME_API_KEY'))
result = model.score_variant(
    chrom='chr3',        # hardcoded!
    pos=58394738,         # hardcoded!
    ref='A', alt='T',    # hardcoded!
    modality='atac',     # hardcoded!
    tissue='CL:0000100'  # hardcoded!
)
print(result['quantile_score'])

The Tool Extractor transforms this into a parameterized function with typed inputs, default values, and a standardized return format:

# Generated MCP tool
@mcp.tool()
def score_variant_effect(
    chrom: str,
    pos: int,
    ref: str,
    alt: str,
    modality: str = "atac",
    tissue: str = "CL:0000100",
) -> dict:
    """Score functional effect of a genetic variant.

    Args:
        chrom: Chromosome (e.g., 'chr3')
        pos: Genomic position
        ref: Reference allele
        alt: Alternate allele
        modality: Assay type ('atac', 'rnaseq', 'chipseq')
        tissue: Tissue/cell ontology ID
    Returns:
        Dict with quantile_score, effect_size, metadata
    """
    model = _get_model()  # cached singleton
    result = model.score_variant(
        chrom=chrom, pos=pos, ref=ref, alt=alt,
        modality=modality, tissue=tissue
    )
    return {
        "quantile_score": result["quantile_score"],
        "effect_size": result["effect_size"],
        "source_file": "alphagenome/scoring.py:L42"
    }

Building the Three MCP Components

Tools are the extracted functions. But an MCP server is more than functions. It also contains:

Resources: The Tool Extractor identifies static assets — the manuscript text, supplementary tables, training data links, figure files — and registers them as queryable MCP resources. The AlphaGenome MCP, for instance, includes links to the training data used to train the model, accessible via a standardized resource query.

Prompts: For complex multi-step workflows, the system generates MCP prompts — templates that orchestrate tools in the correct order. A Scanpy MCP prompt might encode: "Run quality_control → normalize_data → select_features → reduce_dims → build_graph → cluster → annotate." These prompts are inferred directly from the paper's tutorials, not manually written.

Chapter 4: Test-Driven Refinement

This is the stage that separates Paper2Agent from "just ask an LLM to wrap the code." The Test Verifier sub-agent does not trust the generated tools — it validates them against the paper's own results through an iterative test-fix loop.

The Test-Fix Loop

The process works like this:

1. Generate tests from the tutorial's own examples. If the tutorial shows that score_variant('chr3', 58394738, 'A', 'T', 'atac', 'CL:0000100') returns a quantile score of -0.0203, that becomes an assertion.
2. Run all tests. Capture stdout, stderr, return values, and any generated figures.
3. Diagnose failures. The test agent reads the error messages and identifies root causes — missing imports, incorrect parameter mapping, environment issues, numerical precision mismatches.
4. Fix the code. Apply targeted edits to the tool implementation or the test itself (if the assertion tolerance was too tight).
5. Repeat until all tests pass or the agent gives up on a tool.

What Gets Tested

Tests are not just "does the function run without errors." They check:

• Numerical accuracy: Does the output match the tutorial's expected value? (With appropriate tolerance for floating-point differences.)
• Output structure: Does the return dict contain the expected keys?
• Figure generation: Does the visualization tool produce a file? Is it non-empty?
• Edge cases: Does the tool handle missing inputs gracefully?
• Reproducibility: Does running the same input twice produce the same output? (Critical for scientific tools.)

For AlphaGenome, this process validated 22 tools across 15 tutorial-based queries and 15 novel queries, achieving 100% accuracy on both sets. Every single tool produces outputs that exactly match the ground truth from the paper's own code.

Chapter 5: Chat Agent Integration

The MCP server is built and tested. Now it needs a face — a conversational agent that users actually interact with. This is the final piece: connecting the MCP server to a chat agent like Claude Code.

How the Connection Works

MCP servers can be hosted remotely — Paper2Agent deploys them to Hugging Face Spaces, eliminating local dependency issues entirely. The user's chat agent connects to the MCP server over the network. From the agent's perspective, the MCP tools appear as native functions it can call, just like file reads or shell commands.

When a user types "Score variant chr19:8134523:G>A using ATAC-seq predictions for lung tissue," the agent:

1. Parses the intent: identifies the variant, modality, and tissue from natural language.
2. Selects the tool: matches the query to score_variant_effect() from the MCP schema.
3. Constructs the call: fills in typed parameters — chrom="chr19", pos=8134523, etc.
4. Executes: the MCP server runs the locked, tested code on its own infrastructure.
5. Returns results: the agent receives the structured output and presents it in natural language.

Multi-Step Workflows

For complex queries, the agent chains multiple tool calls. When asked to "interpret why a variant associates with LDL cholesterol," the AlphaGenome agent constructs a multi-step plan:

1. Score the variant across multiple modalities (expression, chromatin, splicing).
2. Filter results for the relevant tissue (liver, for LDL).
3. Generate modality-specific visualizations.
4. Compile a report with figures and interpretation.

The agent iteratively plans, acts, observes results, and refines its approach — the classic ReAct pattern. But unlike a general agent, every action is a validated MCP tool call, not ad-hoc code generation.

Multi-Paper Agents

Because MCP servers are modular, you can connect multiple MCPs to the same agent. A researcher could have the AlphaGenome MCP, the TISSUE MCP, and the Scanpy MCP all active simultaneously. The agent seamlessly routes queries to the right server based on the task. This enables cross-paper reasoning that would be extremely difficult to set up manually.

Chapter 6: Case Study — AlphaGenome

AlphaGenome is an AI model from Google DeepMind that predicts how single-nucleotide mutations in human DNA affect gene regulation — expression, chromatin accessibility, splicing, transcription factor binding. It is powerful but complex: the codebase involves custom data loaders, GPU-accelerated inference, tissue ontology lookups, and multi-modal visualization pipelines.

What Paper2Agent Built

Paper2Agent generated 22 MCP tools in roughly 3 hours on a personal laptop, covering:

• Single-variant scoring: predict functional effects across modalities and tissues.
• Batch-variant scoring: process multiple variants in one call.
• Sequence-level prediction: given a DNA sequence, predict its regulatory landscape.
• Tissue ontology exploration: browse available tissues and cell types.
• Visualization suite: generate publication-ready figures for variant effects, TF binding, chromatin accessibility.

Each tool exposes flexible, well-annotated parameters. The visualize_variant_effects() tool, for example, lets users toggle organism (human or mouse), sequence context length, and modality (RNA-seq, ATAC-seq, ChIP-seq histone tracks) — all options discoverable through the tool's typed schema.

Benchmark Results

The SORT1 Reinterpretation

When asked to interpret a GWAS locus associated with LDL cholesterol, the agent prioritized SORT1 as the most likely causal gene — whereas the original paper emphasized CELSR2 and PSRC1. The agent's reasoning: SORT1 had a quantile score of 0.99982 for expression impact in liver, and SORT1 encodes sortilin, a protein directly involved in LDL/VLDL secretion. Independent validation in GTEx eQTL data confirmed the association (p = 1.1e-65).

This was not a bug — it was a genuine scientific reinterpretation, enabled by the agent's ability to run comprehensive multi-modal analysis with a single prompt. The original authors may have emphasized different genes for valid reasons (both CELSR2 and PSRC1 also had high scores), but the agent provided an independent, model-based perspective that users can evaluate.

Chapter 7: Case Study — Single-Cell Agents

The AlphaGenome case shows Paper2Agent on a complex deep-learning model. The single-cell case studies — TISSUE and Scanpy — show it on a different challenge: multi-step analysis pipelines where the correct sequence of operations matters as much as the individual tools.

TISSUE: Uncertainty-Aware Spatial Transcriptomics

TISSUE is a method for predicting spatial gene expression with calibrated uncertainty estimates. Paper2Agent generated 6 tools covering spatial prediction, prediction interval construction, and uncertainty-aware downstream analysis (hypothesis testing, dimensionality reduction).

The TISSUE agent serves two roles:

• Execution: Given a spatial count matrix and scRNA-seq data, it runs the full TISSUE pipeline — from data loading through imputation and uncertainty estimation — producing outputs identical to human-executed results.
• Guidance: Users can ask "What are the required inputs for TISSUE?" and receive structured, actionable instructions — transforming the paper into a live Q&A system about the method.

Scanpy: Preprocessing and Clustering

Scanpy is a widely used package with many features. Paper2Agent focused on the most common use case: the preprocessing-to-clustering pipeline. It generated 7 tools in 45 minutes:

The Role of MCP Prompts

For end-to-end workflows, the correct tool ordering is critical. You cannot cluster before normalizing, or reduce dimensions before selecting features. A general-purpose LLM might get the order wrong.

Paper2Agent solves this with MCP Prompts — workflow templates extracted from the paper's tutorials. The Scanpy MCP prompt encodes: QC → normalize → feature selection → dimensionality reduction → graph construction → clustering → annotation. The prompt also instructs the agent to inspect the data first and adjust parameters if defaults would yield incorrect results.

Chapter 8: The ADHD Discovery

This is the payoff chapter — the moment Paper2Agent stops being "a convenient way to run code" and becomes a tool for scientific discovery.

The Setup

Two papers exist independently:

• AlphaGenome (method paper): predicts variant effects on gene regulation.
• ADHD GWAS (data paper): identifies 39 genomic loci associated with Attention-Deficit/Hyperactivity Disorder through a genome-wide association study.

A human researcher wanting to combine insights from both papers would need to: understand both methods, install both codebases, convert between data formats, design an analysis strategy, execute it, and interpret results. This typically takes weeks.

The AI Co-Scientist

Paper2Agent created MCPs for both papers and connected them to the same Claude Code agent — creating an "AI co-scientist" with access to both a method and a dataset. This agent was then prompted to generate novel hypotheses and execute analyses.

The co-scientist proposed several hypotheses, including:

1. ADHD risk variants alter regulatory activity in brain-specific cell types.
2. AlphaGenome can prioritize causal variants within ADHD fine-mapping credible sets.
3. ADHD-associated variants disrupt transcription factor binding at FOXP family gene loci.

Scaling to All 39 Loci

The co-scientist did not stop at one locus. It autonomously designed and executed a workflow across all 39 ADHD-associated loci:

1. Extract credible-set variants from each locus.
2. Run AlphaGenome functional scoring in glutamatergic neurons.
3. Filter for protein-coding genes.
4. Rank by maximum quantile impact scores across modalities.
5. Compile a comprehensive report for each locus.

The entire analysis — across 39 loci — completed in approximately two hours. Manual execution by human experts would have taken weeks. The results are provided as a supplementary table in the paper, with each locus mapped to its prioritized causal variant, target gene, and molecular mechanism.

Chapter 9: Connections

What Paper2Agent Builds On

Limitations and Open Questions

• Codebase quality dependency: If the original repo is incomplete, poorly documented, or buggy, Paper2Agent cannot fix it. The framework's success is a practical measure of a paper's reproducibility.
• Scope of agentification: A paper is not always the right unit. Some ideas span multiple publications. Paper2Agent supports multi-paper MCPs but the optimal granularity remains an open question.
• Evaluation reliance on expert knowledge: Benchmarks require manually curated ground truth. Future work could use LLM-as-judge for more scalable evaluation.
• Beyond methods papers: Current focus is computational methods. Data papers, discovery papers, and theoretical papers present different agentification challenges.

The Vision: Agent Availability Sections

Just as journals now require data availability and code availability sections, the authors envision an agent availability section — specifying whether and how a paper's contribution has been embodied as an interactive agent. Well-documented, modular, transparent papers will naturally lend themselves to agentification. Papers that cannot be agentified reveal, by that failure, their reproducibility gaps.