All Benchmarks

Answers you can reproduce.

An answer you can't reproduce is an answer you can't rely on. We asked the same question five times and measured how Amdahl holds up. One question, run in May 2026.

Five runs, one question.

Same model, same budget, same prompt. The only difference is Amdahl.

Standard connectors

A different route to a different answer, every run. One never reached a usable answer.

Amdahl

The same route every run.

Route shapes are schematic. Per-run route identity is the measured result in the table below.

What held across runs.

ReproducibilityAmdahlStandard connectorsAmdahl advantage
Produced a usable answer
did the system finish the task?
5 / 5 runs4 / 5 runs+1 run
Numbers stayed the same across runs
same percentages, counts, and dollar figures every time
73%28%+45 pts
Findings repeated across runs
did the same patterns and conclusions appear every time?
71%47%+24 pts
Same accounts surfaced
did the brief cite the same companies?
62%38%+24 pts
Tools and queries used to investigate
did the AI take the same path through the data each time?
Identical every runDifferent every run—

Numbers and accounts: average overlap between runs. Findings: share seen in at least four of five runs.

Methodology.

Five identical runs per path. An independent AI judge clustered the findings, quotes, and named accounts across runs, then measured how much of each answer recurred.

  • Same model (Opus) for both paths
  • Same budget, same turn limit, same prompt
  • Independent AI judge compared outputs across runs
  • Measured: numeric consistency, finding overlap, entity overlap, tool-path identity

The full dataset, including all 10 run outputs and the per-run scoring traces, is available on request.

See Amdahl on your own data.