Repo · Powers /research

Research synthesis

Multi-lane agentic research harness that runs parallel Claude, OpenAI, and Gemini agents, synthesises Obsidian-ready markdown, and scores each sweep with an LLM judge.

What it does

Runs up to 6 parallel research lanes (financial, frontier, academic, vc, blogs, tech) as independent Claude, OpenAI, or Gemini agent calls. Claude API lanes are instructed to use web search and receive web_search_20250305 plus a forced schema-submission tool under tool_choice: any; that forces some tool use, but does not independently guarantee a web-search invocation. OpenAI API-key lanes force web_search via tool_choice; Gemini lanes use Google Search grounding (model-decided, not forced). Lane output is schema-enforced wherever the route can carry a schema (OpenAI api-key strict JSON schema, codex exec --output-schema, Claude api-key forced tool with strict: true); a tolerant parser covers the Claude OAuth and Gemini routes. Sources are URL-normalised and deduplicated globally across lanes before depth caps. model_context is empty for shallow sweeps and optional up to about 80/120 words for standard/deep. Each run records per-lane parseModes in runs/stats.json.
Synthesises lane outputs into a single Obsidian-ready summary plus a deduplicated sources file, with optional async submission through the Anthropic, OpenAI, and Gemini Batch APIs for cost reduction.
Evaluates each sweep with an LLM-as-judge harness using claude-haiku-4-5-20251001 across coverage, source quality, synthesis, and relevance, and persists scores back to the run record.

Architecture

flowchart LR
    Brief[Research Brief] --> Orch[Orchestrator]
    Orch --> L1[Lane: financial]
    Orch --> L2[Lane: frontier]
    Orch --> L3[Lane: academic]
    Orch --> L4[Lane: vc]
    Orch --> L5[Lane: blogs]
    Orch --> L6[Lane: tech]
    L1 & L2 & L3 & L4 & L5 & L6 --> Synth[Synthesis Agent]
    Synth --> Out[Obsidian Markdown<br/>summary - sources - lanes]
    Synth --> Eval[LLM Judge<br/>claude-haiku]
    Eval --> Stats[runs/stats.json<br/>with eval score]

Built-in deep research vs this harness

Surface	Best fit
Claude.ai Research	Interactive, one-off research product in Claude.
Claude Managed Agents (beta) and Messages API web tools	Programmatic building blocks for long-running Claude work with web search, fetch and code execution.
ChatGPT Deep Research / Responses API tooling	Autonomous planning and research across web, files, MCP and code.

Built-ins can replace ad-hoc one-off research and reduce custom orchestration. They do not replace this repository where named lanes, vendor-neutral Claude/OpenAI/Gemini comparison and routing, Obsidian artefacts, a source ledger, batch/recovery, cost and parse telemetry, and evaluation are requirements. Its core Node CLI and artifact format are OS-portable; the optional secure .sh helpers require a POSIX shell. This is a controlled multi-lane harness, not a wrapper around either vendor's native Deep Research or Research product.

Checked 2026-07-28: OpenAI's deep-research guide still names o3-deep-research and o4-mini-deep-research, while the official deprecations page says those slugs shut down on 2026-07-23 and recommends GPT-5.6 Sol. Check current model availability before implementing against the guide.

Quick start (no 1Password)

cp .env.example .env
# edit .env: set ANTHROPIC_API_KEY (and OPENAI_API_KEY for the OpenAI provider,
#            GEMINI_API_KEY for the Gemini provider;
#            GOOGLE_ACCESS_TOKEN is only needed for the gemini-oauth route — GCP-billed)

npm install
npm run auth:check          # verifies the keys in .env work

npm run sweep -- \
  --brief-file "prompts/ai-transformation-financial-services.md" \
  --from 2023 \
  --lanes financial,frontier,academic,vc,blogs,tech \
  --depth standard \
  --folder "ai-transformation-fs"

.env is loaded fill-only and never overrides a real environment variable, so the secure wrappers (./batch-search.sh, etc.) also work without 1Password — they detect that op-fetch is unavailable and fall back to the .env keys.

Generated output defaults to output/<folder>/ inside the checkout, which is gitignored, so a fresh clone runs without writing anywhere else. Set RESEARCH_SWEEPER_OUTPUT_DIR (env or .env) to write into a notes vault or any other directory.

The single synthesis call defaults to claude-opus-4-8 (a reasoning-dominated call where the extra capability earns its cost once per sweep); the six lanes stay on haiku/sonnet. Override the synthesis model without a code change via SYNTHESIS_MODEL (env or .env), e.g. SYNTHESIS_MODEL=claude-fable-5, or the --synthesis-model flag. Fable 5 is access-gated: it returns not_found_error unless your Console admin has accepted 30-day data retention (misuse detection only, not training).

Tip: the run signature is long. Describe what you want in plain English to any capable LLM and ask it to emit the full --flag invocation (topic, lanes, depth, folder). Quicker and less error-prone than hand-writing the command.

Writing a research brief

Briefs are plain markdown files in prompts/. --brief-file prompts/<name>.md reads four sections and ignores everything else:

Section	Reaches
`## Topic string`	lanes and synthesis
`## Sub-questions`	lanes and synthesis
`## Lane directive` (optional)	the six lane agents only
`## Synthesis directive` (optional)	the synthesis pass only

The two directive sections are free-form: role, audience, register, output contract, whatever the brief needs. They let the lanes work at one altitude (retrieve the engineering detail) while the report lands at another (answer the strategic question from that detail). The lane directive rides in the cached system prefix, so it costs one cache write per sweep however many lanes run.

Anything under any other ## heading is reported at submit time and reaches no model, so a role block in the wrong place fails loudly rather than silently.

Three ways, easiest first:

1. The sweeper-prompt-creation skill (recommended). If you drive this repo from Claude Code or an MCP harness, invoke the sweeper-prompt-creation skill shipped in this repo at sweeper-prompt-creation/SKILL.md. Describe your research goal in plain English; it interviews you for the gaps (time window, lanes, depth), writes a well-formed prompts/<slug>.md, and offers to launch the sweep. This is the handiest path — a good brief is the single highest-leverage input to a sweep.

2. Any capable LLM. No skill? Paste prompts/sweep-template.md into any model, describe the goal, and save the returned brief into prompts/.

3. By hand.

cp prompts/sweep-template.md prompts/my-topic.md
# edit: fill in the topic string and 2–4 themes of sub-questions

The template documents the expected structure (topic string with a date range, themed sub-question blocks, and the optional directive sections). The repo ships ~24 real briefs in prompts/ as worked examples.

Quick start (with 1Password, optional)

cp op-refs.local.sh.example op-refs.local.sh   # set your real op:// vault refs
npm run auth:check:secure                       # resolves refs via op-fetch

./batch-search.sh \
  --brief-file "prompts/ai-transformation-financial-services.md" \
  --from 2023 \
  --lanes financial,frontier,academic,vc,blogs,tech \
  --depth standard \
  --folder "ai-transformation-fs"

Use ./list-batches.sh to inspect pending jobs and ./resume-batch.sh <n> to collect results.

If a completed batch comes back with some lanes errored/expired/canceled, resubmit just those lanes (Claude only — the Batches API best practice is to resend exactly the failed custom_ids, which is free since those requests were never billed):

npx ts-node research-sweep.ts --resubmit-failed <batchId>
# prints the failed lanes, submits a follow-up batch, and saves a new job manifest
npx ts-node research-sweep.ts --resume <newBatchId>

The resubmission writes to its own <folder>-resubmit-<id> output folder rather than merging into the original run automatically — combine the two by hand (see the guidance printed by --resume on a resubmitted job) or fold the resubmitted lane files into the original folder's lanes/ directory and re-run --re-synthesise <original-folder>.

Auth routes

Route	When to use	Credential source
API key (batch)	Long sweeps, async, cost-tracked	`.env`, or 1Password via `op-fetch`
Claude OAuth	Sync sweeps on Max/Pro quota	`CLAUDE_CODE_OAUTH_TOKEN` (`.env` or 1Password)
Codex auth	OpenAI sync runs, no API billing	Codex CLI auth file (`codex login`)
Gemini API key	Gemini sweeps, sync or batch	`GEMINI_API_KEY` (`.env`, or 1Password via `op-fetch`)
Gemini OAuth	Gemini sync runs, GCP-billed	`GOOGLE_ACCESS_TOKEN` (caller env; not a consumer-subscription route)

1Password is optional. When configured, all op:// references live in your gitignored op-refs.local.sh; the resolver fetches only the named refs and execs the child with a sanitised env. Without it, keys come from .env (fill-only). Batch mode requires API-key auth for all providers.

For spend-optimised re-synthesis, collect a finished API-key batch and run only the synthesis step through a sync-only auth route:

./run-secure-sweep.sh --provider openai --re-synthesise <folder> --from-batch <batchId> --openai-auth codex
./run-secure-sweep.sh --provider claude --re-synthesise <folder> --from-batch <batchId> --claude-auth claude-oauth
./run-secure-sweep.sh --provider gemini --re-synthesise <folder> --from-batch <batchId> --gemini-auth gemini-oauth

The wrapper supplies the API key needed to collect the batch results, then the provider switches the synthesis call to the explicit Codex, Claude OAuth, or Gemini OAuth route. Gemini OAuth remains GCP-billed.

Cost estimates

OpenAI estimates use list rates with one Batch discount, observed web-search fees, and reported cache read/write telemetry. Reasoning is already included in output tokens and is not billed twice; missing provider usage is called out in the run record.

Gemini provider — known limitations

Free-tier rate limits. Google AI Studio free tier enforces ~5 RPM (gemini-2.5-flash) / ~10 RPM (gemini-2.5-flash-lite). Multi-lane parallel sweeps will hit this. A GCP trial billing account does not grant paid-tier rate limits. gemini-2.5-pro requires paid tier.
Batch API requires billing. Free-tier keys return 400 FAILED_PRECONDITION or 429 RESOURCE_EXHAUSTED on batch create. Enable prepaid billing in Google AI Studio or use a billing-attached GCP project.
Grounding vs JSON mode. Google Search grounding and native JSON structured output are mutually exclusive. The provider uses the tolerant parseLaneResponse() parser instead of responseMimeType mode; some responses fall back to plain-text extraction.
Empty / dropped lanes. Upstream content-safety or grounding blocks can produce 0-source, 0-token lanes with finishReason other than STOP, and a batch item can error or fail to return. Both cases are now surfaced, not swallowed: the provider emits an explicit empty-lane placeholder and logs Warning: no batch result for this lane … rather than silently shrinking the lane set. Synthesis proceeds with the lanes that did return content and the lane count stays stable. If you see this warning, check the per-lane logs for a block reason or rerun that lane; a missing lane is expected behaviour, not a crash. The --min-lanes pre-flight still aborts if too few lanes returned real content.
Batch collection. Collect Gemini/Claude/OpenAI batches via ./run-secure-sweep.sh --resume <id> --provider <p> rather than the all-keys helper to avoid tripping the Claude both-keys auth guard.

Claude provider — transient-error retries

Sync lane and synthesis calls retry 429/500/502/503/529 with backoff (2s/6s/18s, 3 attempts) before degrading a lane; 429s honour the response's retry-after header. The SDK's own internal retry is disabled so the two layers don't multiply attempts.

Evaluation

Each sweep can be scored by an LLM judge harness using claude-haiku-4-5-20251001. The judge reads the summary, sources, and original brief, and returns four 1-5 dimension scores (coverage, source quality, synthesis, relevance), an overall mean, a 2-3 sentence verdict, and a list of unverifiable factual flags. Scores are computed from a single API call per sweep and persisted onto the matching runId in runs/stats.json, or written to runs/eval-<runId>.json if no run record exists. Run it via:

ANTHROPIC_API_KEY=... npm run eval -- \
  --summary path/to/summary-<slug>.md \
  --sources path/to/sources-<slug>.md \
  --brief path/to/brief.md \
  --run-id <optional-run-id>

The eval CLI requires ANTHROPIC_API_KEY to be injected by the caller and exits non-zero if it is absent.

Output files

Each sweep writes to <output-folder>/:

File	Purpose
`_research-sweeper-stub.md`	Placeholder written before lanes start; signals an in-flight sweep
`summary-<slug>.md`	Synthesised cross-lane narrative with section headings; deep and standard sweeps also include a Mermaid `timeline` of dated milestones (granularity scaled to span; omitted on shallow sweeps or when fewer than three events are datable)
`sources-<slug>.md`	Deduplicated, dated source list across all lanes
`lanes/lane-<name>-<slug>.md`	Per-lane markdown with narrative and citations
`lanes/lanes-<slug>.json`	Structured lane payload for re-synthesis and eval

Existing summary-*, sources-*, and lane files are not overwritten unless --overwrite is passed. Re-synthesis is allowed to rewrite generated outputs.

Generated Markdown sanitises absolute brief paths; lane JSON sanitises both brief and output paths. Re-synthesis always writes to the requested folder, never a cached output path.

A lane that hits its provider's output-token cap (Claude max_tokens, OpenAI incomplete_details, Gemini MAX_TOKENS) has its narrative prefixed with [TRUNCATED at max_tokens — findings incomplete]. If any upstream lane is marked truncated, the final summary deterministically receives a separate warning naming the affected lanes. This is distinct from a synthesis that itself truncates, which carries a > [!warning] Synthesis truncated at max_tokens — increase depth tier or reduce lane volume. callout. These are generated markers, not model prose — treat them as a signal to rerun with a lower lane volume or higher depth tier. A Claude synthesis refusal is a hard failure, so an empty response cannot overwrite a report or be recorded as a successful run.

Batch synthesis failures stop with the batch id and terminal status rather than polling indefinitely.

Security

No credentials are committed to the repository: .gitignore excludes .env* (except .env.example), op-refs.local.sh, and .claude/settings.local.json. When 1Password is configured, keys are resolved at runtime via the op-fetch wrapper and never touch disk; otherwise they are read from a local, gitignored .env. See docs/SECURITY.md for the full architecture, threat model, and rotation steps.

Development

npm run build        # compile TypeScript
npm run typecheck    # type-check without emit
npm run test         # vitest unit tests
./scripts/smoke-test.sh  # typecheck + test + build

Version history

Version	Date	Summary
v1.3.0	2026-08-04	Brief directives: `## Lane directive` and `## Synthesis directive` are read from `--brief-file` and routed to the lane agents and the synthesis pass respectively, so a brief can set role, audience, and output contract instead of only a topic and sub-questions. The lane directive rides in the cached lane system prefix (one cache write per sweep) and sits after the schema rules so it cannot displace them. Works across all three providers in both sync and batch. Also: Node 24 CI runtimes, GPT-5.6 terra/luna price-cut cost estimates, lane turn headroom beyond `searchRounds`, and an absolute-symlink-target commit guard.
v1.2.0	2026-07-28	Non-breaking cost, quality, privacy and reliability improvements: accurate OpenAI batch/cache telemetry, bounded and deduplicated synthesis context, safe generated path references, batch failure diagnostics, refusal handling, and a documented comparison with Claude and OpenAI native research.
v1.1.0	2026-07-19	Strict lane JSON schema across the API-key routes (shared `LANE_RESPONSE_SCHEMA`), `jsonrepair`-based tolerant parser with per-lane `parseMode` stats, OpenAI migration to GPT-5.6 models, forced `web_search` on OpenAI lanes, `--resubmit-failed` batch recovery, and the publish-PR workflow.
v1.0.0	2026-05-18	Initial multi-lane research harness: `claude`/`openai`/`gemini` providers, sync and batch modes, API-key and OAuth/subscription auth routes, and Obsidian-ready synthesised output.