Project

Research Report Agent

Original text · English1200 min

Level 2, Builder. About 20 hours. Rust, Python and TypeScript standard libraries. Runs offline. Requires Python 3.10+, rustc and Node 22.18+.

Turn a question and a folder of documents into a report where every sentence can be checked. You build the pipeline that deep research tools run: search, snippet extraction, planning, cited writing, claim verification, publishing and evaluation.

pj rra pipeline

Rust owns ranking, Python owns evidence and orchestration, and TypeScript owns publication. Both language boundaries use inspectable JSON. The replay model keeps tests deterministic; a live model is optional.

What you end up with

bash

python3 projects/research-report-agent/solution/run_report.py \
  "How does a secrets proxy protect API keys from prompt injection?" --out out/ --code my-report-agent

out/report.html: sections with numbered footnotes. Every footnote quotes the exact source sentence and links the original page.
out/report.json: the versioned evidence contract consumed by the TypeScript viewer.
out/trace.json: run id, each step with timings, counts, budget use and the terminal state (completed, needs_review or failed).
A scorecard on checked-in public evaluation fixtures, with component metrics you can compare and share.

Stages

#	Stage	You implement	Level
1	Search the corpus	`search/main.rs`, Python adapter (BM25)	starter
2	Extract snippets you can cite	`snippets.py` (sentence spans, offsets)	starter
3	Plan the research	`planner.py`, `model.py` (replay cassette, fallback)	core
4	Write only from evidence	`citations.py`, `writer.py`	core
5	Verify every claim	`critic.py` (support rules, budget, terminal states)	core
6	Publish the report	`publish.py`, `pipeline.py`, `viewer/render.ts`	core
7	Score the public fixtures	`evaluate.py`	stretch

Start

bash

python3 scripts/project_test.py research-report-agent --init my-report-agent
python3 scripts/project_test.py research-report-agent --stage 1 --path my-report-agent

Read stages/01-search-the-corpus/docs/en.md, fill in the stubs in my-report-agent/report_agent/, and rerun until stage 1 passes. Each later stage reruns every earlier stage, so a regression shows up immediately.

Stuck? The reference implementation is in solution/. Read it after you have tried, not before.

Folder map

text

project.json        metadata the site and the grader read
fixtures/corpus/    12 original concept notes with supporting official sources
fixtures/questions.json       development questions
fixtures/cassettes/planner.json  recorded model replies for stage 3
heldout/            public evaluation questions and poisoned drafts (legacy folder name)
stages/NN-*/docs/en.md        the lesson for each stage
stages/NN-*/starter/          stubs copied by --init
stages/NN-*/tests/            the tests the grader runs
solution/           reference implementation and run_report.py

Use it on your own documents

Point --corpus at any folder of markdown files that start with title:, source_url: and published: lines, then a blank line, then the body.

Verified reference output

The secrets-proxy example produces 4 sections, 10 sentences, no dropped claims and state completed. Its footnotes show exact evidence on hover or keyboard focus, and its expandable trace shows the five pipeline steps. The reference baseline scores 87.5 / 100 on the six checked-in public evaluation questions. The demo repeats question h4 from that file, so this is a reproducible fixture score rather than evidence from an unseen evaluation. Lexical verification does not establish real-world truth.

Inspect a source update

The original Orchard corpus is fictional operational documentation. Run both revisions with the same question and inspect orchard-after/changes.json.

bash

python3 projects/research-report-agent/solution/run_report.py "How long do Orchard guest tokens last?" --code projects/research-report-agent/solution --corpus projects/research-report-agent/examples/orchard/before --out orchard-before
python3 projects/research-report-agent/solution/run_report.py "How long do Orchard guest tokens last?" --code projects/research-report-agent/solution --corpus projects/research-report-agent/examples/orchard/after --compare orchard-before/report.json --out orchard-after

The default uses deterministic planning and extractive writing. --model replay needs --cassette. --model live uses RRA_LLM_BASE_URL, RRA_LLM_MODEL and optional RRA_LLM_API_KEY for planning only. Live service behavior requires a separate run with caller credentials. Public fixture scores do not measure unseen generalization.