Evidence Graph

100% specification coverage with compile errors

https://github.com/samchon/ttsc/tree/master/packages/evidence

TL;DR

  • Evidence Graph, a compiler harness
    • No Loop Engineering required
    • @evidence <target> <reason>
    • @evidenceReview <target> <reason>
    • @evidenceExclude <target> <reason>
  • Spec Driven Development
    • Write and review only the requirements
    • AI builds everything with 100% coverage
    • Applies to programming, documents, and literature
Requirement coverage
Plain
51.6%
Evidence
100%
Token usage
Plain
5,449M
Evidence
411M
100+ tables · 150K+ LoC

Asked whether every requirement was met, a human explains while the compiler stops the build

Asked whether the rule document was read, a human reads it out again while the compiler asks at every file

Current Limitations

Why we all ended up looping

Saying yes is not doing it

  • Six frontier models: 0/60 actual process compliance under default framing
  • Verbal compliance in the same runs exceeded 90%
  • At 8 constraints: about 41% passed individually, only 5.7% passed all eight
  • The strongest model fell below 50% whole-response success at 7 constraints

Measured with tool logs and deterministic verifiers (2605.01771) (2608.12426)

Split specs and long runs both degrade

  • Split across about 60 requests, single-shot was more faithful on 16/20 papers for Claude Code and 14/20 for Codex
  • Of 15 agents on 36 iterative problems, none finished one end-to-end; best strict rate: 14.8%
  • Structural erosion rose in 77% of trajectories; verbosity in 75.5%
  • Versus 473 open-source Python repositories: 2.3× more verbose, 2× more eroded

Two 2026 coding-agent benchmarks (2603.17104) (2603.24755)

So we built Loop Engineering

  • A claim of done proves nothing → read it all again
  • Omissions show only when you look → fix every finding
  • One fix breaks another → restart from the top
  • Nothing else says done → stop after an empty round

Also called Loop Until Dry. It is the state of the art, and it works.

ERP Loop Engineering

Requirement coverage
51.6%
Time distribution
10%
90%
Initial developmentReview loops
102hwork time 5,449Mtokens
ERP · 100+ tables · 150K+ LoC

Evidence Graph

Missing specification coverage becomes a compile error

First, divide the artifacts into layers

Idea notes
Requirements
Specifications
Implementation
Test

Each arrow points to the evidence it cites.

One rule declares the relationship

type: "typescript",
files: ["src/components/**/*.tsx"], // sources
symbol: "function",
reference: {
  type: "markdown",
  files: ["docs/specifications/*.md"], // targets
  symbol: ["h2", "h3"],
},

Components implement specifications.

One grammar covers four artifact types

  • Markdown: file, H1-H4 section
  • Prisma: database model, columns, relation
  • TypeScript: type, function, property
  • Swagger: each operation under paths

Code cites the specification

/**
 * @evidence docs/specifications/discount.md#coupon-stacking
 *           Explains the stacking limit defined by this section.
 * @evidence POST:/orders/{orderId}/coupons
 *           Explains the rejection response from this endpoint.
 */
export function CouponStackingNotice(props: IProps): JSX.Element;

@evidence <target> <reason>: what this code implements, and why.

Without a citation, the build stops

$ npx ttsc
error TS16411: [evidence/graph]
  Missing acknowledgement for
  'docs/specifications/discount.md#coupon-stacking'
  (Markdown H2 'Coupon Stacking' at docs/specifications/discount.md:3)
  • One error per requirement → the error list is the task list
  • It runs alongside type errors in the same build

100% coverage can include false citations

/**
 * @evidence docs/specifications/discount.md#coupon-stacking
 *           Explains the per-issuer limit.
 */
export function CouponStackingNotice(props: IProps): JSX.Element;
  • Inexpensive models sometimes write facts that do not exist
  • A false tag removes the error, not the problem

Citations make the false claim detectable: 86-88%, no false positives (2606.30689).

Review only citation truth

/**
 * @evidence docs/specifications/discount.md#coupon-stacking
 *           Explains the per-issuer limit.
 * @evidenceReview docs/specifications/discount.md#coupon-stacking
 *                 #a1b2c3d4e5f6 Verified against policy section 3.
 */
export function CouponStackingNotice(props: IProps): JSX.Element;
  • Reviews match the same declaration and target
  • The fingerprint expires when the cited content changes

Even Luna reduced false citations to zero in one review pass.

The tag list is the review checklist

Review Plain Evidence
Target Everything Citation truth
Loop Restart every round Follow the tag list
Omissions Search manually Compiler reports them

The compiler catches omissions. Review catches falsehoods.

Benchmark

Same inputs · engine · model · Plugin only

Coverage: 51.6–85.5% → 100%

Subject Plain Evidence
todo 85.5% 100%
reddit 80.3% 100%
shopping 63.1% 100%
erp 51.6% 100%

Plain coverage falls with scope. Evidence remains at 100%.

Token usage: 4.8–13.3× lower

todo866M → 92M
reddit1,179M → 245M
shopping1,516M → 271M
erp5,449M → 411M

Plain in blue. Evidence in orange.

Original charts by phase: https://ttsc.dev/docs/benchmark/evidence

ERP: 100% coverage · $4.96 · 14h

13.3×fewer tokens
5,449M → 411M
13.9×lower cost
$68.72 → $4.96
7.5×less time
102h → 14h

Review: 90–95% → 15–41% of tokens

Subject Plain Evidence
todo Review 90% Review 28%
reddit Review 95% Review 19%
shopping Review 95% Review 41%
erp Review 90% Review 15%

Dark is development. Light is review. Each cell represents 100% of its tokens.

Spec Driven Development

Requirements are the handoff.
AI builds everything below them with 100% coverage.

Method A starts from requirements

Requirements
Specifications
Implementation
Test

Requirements are the source layer.

Method A: Humans write the requirements

  • Humans review docs/requirements directly
  • Specifications, implementation, and tests are fully delegated

The four subjects you just saw all used this method.

Method B starts from idea notes

Idea notes
Requirements
Specifications
Implementation
Test

Idea notes are the source layer.

Method B: Delegate the requirements, too

  • Hand over idea notes as-is, without organizing them
  • Delegate everything, starting with writing the requirements
  • If anything in the idea notes is omitted, the build breaks immediately

Humans provide one source layer.
The graph protects everything below it.

Even requirements cite their evidence

## Coupon stacking limit {#coupon-stacking}

<!-- @evidence docs/ideas/discount.md#discount-policy
     Carries over the per-issuer limit recorded in the idea notes. -->
  • Missing idea-note coverage breaks the requirements build
  • Idea notes, interviews, and internal documents share one layer
  • Citations are comments, so the rendered document stays clean

The backend works like this

Requirements
Specifications
DB schema
API operation
API schema
Test

Backend artifacts trace back to Requirements and Specifications.

The frontend works like this

Requirements
Specifications
Backend
Hooks
Screens
Journeys

Frontend delivery traces back to the documents and Backend.

Method D: Hand over principles only

  • The project already exists, so a full document hierarchy is hard to introduce
  • You want to develop directly instead of delegating requirements and specifications
  • You are not ready to design the whole graph yet

Start with one docs/principles.md and one claim.

Every principle must be followed

## Do not hardcode {#no-hardcoding}
Derive behavior from inputs and models. Never special-case a fixture.

## Do not monkey patch {#no-monkey-patching}
Use public extension points. Never replace prototypes or module state.

## Use the conventional solution {#conventional-solution}
Avoid unmeasured optimization. Prefer standard structures and clear algorithms.

## Fix the root cause {#fix-the-root-cause}
Do not route around one visible failure. Trace the cause and solve the whole class.

Every function answers every rule

  • Each selected function checks every H2 rule
  • Every answer records how and why the rule was followed
  • "Not applicable" can be closed as an escape hatch
  • One missing answer becomes a compile error

Add one rule, and every function immediately gains one obligation.

Every answer explains how

/**
 * @evidence docs/principles.md#no-hardcoding
 *   Builds the lookup from registered handlers, with no case-specific branch.
 * @evidence docs/principles.md#no-monkey-patching
 *   Uses the public adapter without replacing prototypes or module state.
 * @evidence docs/principles.md#conventional-solution
 *   Uses a standard Map and linear pass, with no speculative index or cache.
 * @evidence docs/principles.md#fix-the-root-cause
 *   Rejects invalid names at registration instead of retrying failed lookups.
 */
export function resolveHandler(name: string): Handler;

Both the target and a non-empty reason are required. Miss one and the build breaks.

Summary

  • Missing specification coverage becomes a compile error
  • Requirements are the handoff, and a principles list is enough to start
  • Coverage rises from 51.6–85.5% to 100%
  • Review checks the truth of the evidence

Appendix: Stories

Principles and settings become build constraints

Fluency is not authorship

  • Flattening: competent prose that no one could have signed
  • Softened conflict: the antagonist apologizes a paragraph later
  • Translationese: borrowed syntax, misplaced honorifics

Studio case: every cool character got silver hair, every genre got the same moral ending.

The failure is measured, not just felt

  • Training smooths out theme, emotion, and voice
  • Literary fiction loses the most
  • Contradictions grow steadily with length
  • Facts slip early (15-30%), contradictions late (40-60%)

Narrative Flattening (2605.27878) · ConStory-Bench (site)

A fluent scene can still be false

  • Memory: uses facts the character never learned
  • Invention: breaks history, geography, or motive
  • Contradiction: negates a number, a date, or a trait set earlier
  • Revision: keeps scenes invalidated by an earlier edit
  • Amnesia: 350 settings, and no way to tell which went unused

Long-form failure is global, not local.

Here the loop makes it worse

  • Every pass pulls the text toward the model's own average
  • Style and fluency rise, accuracy barely moves
  • Voice normalizes each time, and prompts cannot stop it

More rounds buy polish, not truth.

Two 2026 revision studies (2605.13368) (2604.22142)

Every layer cites all prior sources

Principles
Settings
Storylines
Scenarios
Manuscripts

Each edge blocks a different drift

  • Storylines, Scenarios, Manuscripts → Principles: literary purpose
  • Storylines, Scenarios, Manuscripts → Settings: facts, rules, knowledge
  • Scenarios, Manuscripts → Storylines: causes and consequences
  • Manuscripts → Scenarios: exact execution

Why it works

  • Limited context → exact obligations for this scene
  • Plausible invention → explicit lineage and review
  • Revision drift → affected reviews expire
  • Forgotten promise → 100% reverse coverage

Creative freedom inside hard continuity.

One graph, any narrative

  • Settings: history, world rules, character
  • Causality: clues, motives, consequences
  • Continuity: knowledge, arcs, revisions
  • Historical fiction, fantasy, science fiction, mystery, drama
  • Napoleon: 25 principles, 350 setting commitments, 742 scenes

References: @ttsc/evidence

References: Coding Agents

  • Faithfulness drops when the spec arrives in pieces (2603.17104)
  • Agents erode their own code over long horizons (2603.24755)
  • Process instructions agreed to, then bypassed (2605.01771)
  • Citations make hallucinated requirements detectable (2606.30689)
  • Specifications as the primary artifact (2602.00180)

References: Long-form Narrative

  • Post-training flattens theme, affect, and style (2605.27878)
  • Narrative tension measured by forecasting (2604.09854)
  • Consistency bugs scale with story length (ConStory-Bench)
  • Revision improves style, not accuracy (2605.13368)
  • Rewriting normalizes personal voice (2604.22142)
  • Korean honorifics in automatic translation (LREC 2026)

Q & A

Samchon
https://ttsc.dev