Evidence Graph

One coding engine builds the same application twice from the same frozen requirements: once with @ttsc/evidence in the workspace, once with neither. Same template, same instruction sequence, same model and effort. The variable is the graph.

Four subjects in ascending size. Every figure comes from benchmarks/evidence/aggregate and from nowhere else.

Coverage

Thirteen edges, folded from the leaves up. Serial hops multiply, branches average, and every edge enters once. $C_{a \to b}$ is the share of $a$ that reaches $b$ , and $Q_x$ is how whole an $x$ is below itself.

\begin{aligned} Q_{\text{test}} &= Q_{\text{journey}} = Q_{\text{property}} = 1 \\[2pt] Q_{\text{dto}} &= C_{\text{column} \to \text{property}} \\[2pt] Q_{\text{screen}} &= C_{\text{screen} \to \text{journey}} \\[2pt] Q_{\text{hook}} &= C_{\text{hook} \to \text{screen}} \cdot Q_{\text{screen}} \\[2pt] Q_{\text{op}} &= \tfrac{1}{2}\left( C_{\text{accessor} \to \text{test}} \cdot Q_{\text{test}} + C_{\text{accessor} \to \text{hook}} \cdot Q_{\text{hook}} \right) \\[2pt] Q_{\text{model}} &= \tfrac{1}{2}\left( C_{\text{model} \to \text{op}} \cdot Q_{\text{op}} + C_{\text{model} \to \text{dto}} \cdot Q_{\text{dto}} \right) \end{aligned}

A requirement carries six obligations, and each is worth its own rate times the wholeness of what it reaches.

\text{score} = \frac{1}{6} \left( \begin{aligned} & C_{\text{req} \to \text{model}} \cdot Q_{\text{model}} + C_{\text{req} \to \text{op}} \cdot Q_{\text{op}} + C_{\text{req} \to \text{dto}} \cdot Q_{\text{dto}} \\ & {} + C_{\text{req} \to \text{test}} + C_{\text{req} \to \text{screen}} \cdot Q_{\text{screen}} + C_{\text{req} \to \text{journey}} \end{aligned} \right)

Averaging the thirteen rates instead scored one subject 58.4% while 76 of its 557 published accessors had an asserting test. Multiplying them scored the same subject 0.003%. #1088 derives the operators.

Cost

Loading the measurement.

How much tokens, prices, times have consumed.

Reproduce

Clone the repository and tell Claude Code to run the benchmark following the benchmark skill. That skill is the procedure this cohort was produced under, and it owns setup, launch, supervision, recovery, and publication.


git clone https://github.com/samchon/ttsc.git
cd ttsc
corepack enable
pnpm install
claude "run the evidence benchmark following .agents/skills/benchmark/evidence"

The commands it drives, if you would rather drive them yourself:


pnpm --filter @ttsc/benchmark-evidence start codex <subject> <evidence|plain> <model> <effort>
pnpm --filter @ttsc/benchmark-evidence audit-suspensions
pnpm --filter @ttsc/benchmark-evidence report
pnpm --filter @ttsc/benchmark-evidence charts

<subject> is a directory under benchmarks/evidence/requirements. The runner refuses to launch while anything is uncommitted and records the repository HEAD as the run’s benchmark revision. report writes the aggregate under benchmarks/evidence/aggregate and its charts under website/public/benchmark/evidence; charts redraws those from the published aggregate alone, with no run records present.

benchmarks/evidence/README.md documents the runner, the arms, and what a run retains.