Skip to content

Retry & repair: the extraction pipeline

The design principle: repair before retry. Deterministic repair is free; retries cost tokens and latency. The pipeline always exhausts repair before reprompting, and the report records every repair that fired — so you can see how close a model is to reliable, not just whether.

The five stages

Every extraction runs a pure, deterministic state machine over these stages (one extract = one isolated session; no I/O, no clock, no randomness in core):

  1. Parse — tolerant JSON parsing. Strips code fences and prose preambles, repairs trailing commas, single quotes, unquoted keys, and comments, and completes truncated output where unambiguous (closing brackets at EOF). Every repair is recorded as a RepairNote.
  2. Align — schema-aligned mapping. Case/snake/camel-insensitive key matching, fuzzy enum resolution ("celsius" → Celsius), unambiguous type coercion ("42" → 42), and discriminator inference for unions when the tag is missing but the shape identifies exactly one variant. Every alignment is recorded as an AlignmentNote with its JSONPath.
  3. Validate. Structure (required fields, unknown-field policy), then the constraint IR (Iron refinements / Jakarta annotations / lowered keywords), then user semantic validators. Errors are JSONPath-addressed: .lineItems[2].amount: -3 violates MinNumberExclusive(0).
  4. Decide. The state machine emits Done(value, report), a retry request (within policy), or Failed(error) once attempts or budget are exhausted.
  5. Reprompt — targeted. The retry message contains only the addressed errors, schema context for the offending paths, and the instruction to preserve all other fields. The previous output is pinned as assistant context. Fixing one field is cheaper and more reliable than regenerating the whole document.

Strict decode happens after alignment: LlmSchema.decode never coerces — anything fuzzy is an explicit, recorded alignment, so a value that reaches your code was either exactly right or visibly repaired.

Policy and budget

val tc = TypeCast(transport, model = "claude-haiku-4-5")
  .withRetry(RetryPolicy(maxAttempts = 3, escalateTo = Some("claude-sonnet-4-5"), escalateAfter = 2))
  .withBudget(Budget(maxTotalTokens = Some(20000L)))
  • RetryPolicy.targeted — the default: three attempts, targeted reprompts.
  • RetryPolicy.targeted(n) — different attempt cap. RetryPolicy.none — fail fast.
  • escalateTo / escalateAfter — switch model after N failed attempts (cheap model first).
  • Budget(maxTotalTokens) — hard per-extraction ceiling; the session fails with BudgetExhausted rather than exceed it.

The report

extractWithReport always returns the ExtractionReport; on failure the same report rides on the error. Render it as a transcript with import typecast.report.*:

import typecast.report.*

val (result, report) = tc.extractWithReport[Patient](clinicalNote)
println(report.render)

A real transcript (this exact text is a golden-test fixture — a retry that succeeds after one targeted reprompt):

attempt 1  model=claude-haiku-4-5  1,204 tokens  0.9s
  repairs: extracted content of markdown code fence
  aligned: .age: string "67" -> number 67
  FAILED validation:
    .diagnosisCodes[1]: "j449" violates Pattern([A-Z][0-9]{2}\.[0-9])
attempt 2  targeted reprompt (1 error, valid fields pinned)  312 tokens  0.6s
  OK
total: 1,516 tokens, $0.0026, 1 retry

And a failure with model escalation (also a golden fixture — note attempt 3 naming the escalated model):

attempt 1  model=claude-haiku-4-5  1,000 tokens
  FAILED validation:
    .total: -3 violates MinNumber(0)
    .dueDate: required string is missing
attempt 2  targeted reprompt (2 errors, valid fields pinned)  350 tokens
  FAILED validation:
    .total: -3 violates MinNumber(0)
attempt 3  targeted reprompt (1 error, valid fields pinned)  model=claude-opus-4-8  480 tokens
  FAILED validation:
    .total: -1 violates MinNumber(0)
total: 1,830 tokens, $0.014, 2 retries

Costs come from a PricingTable (defaults provided, caller-overridable: report.render(myPricing)); unpriced models simply omit the cost. Durations appear when the runner injects them — core has no clock.

Programmatic access

The transcript is rendering; the data is plain case classes:

report.attemptCount                       // Int
report.totalUsage                         // TokenUsage(inputTokens, outputTokens)
report.attempts.head.repairs              // Vector[RepairNote]   — stage 1
report.attempts.head.alignments           // Vector[AlignmentNote] — stage 2, JSONPath-addressed
report.attempts.head.errors               // Vector[FieldError]   — stage 3

This is the seam for eval/CI tooling: assert on success rate, repair rate, or retries-per-request over a golden set without string matching. A repair note that fires on every request is a prompt bug you can see; an alignment note that disappears when you upgrade models is measurable progress.