Architecture

This page is for people evaluating TypeCast's internals or extending it — adding a transport, a runner, or a constraint front-end. It describes how an extract call actually executes: the pure five-stage pipeline, the sans-IO core with thin effect runners, the strict-decode/alignment split, capability lowering with its validation backstop, the ExtractionReport seam, and the module dependency graph. Every claim here is grounded in the contract package (typecast.contract, in typecast-core) and SPEC §3 and §5.

The five-stage extraction pipeline

One extract call is one isolated session: a pure, deterministic state machine over five stages (SPEC §5.2). No HTTP, no clock, no randomness lives in core, so identical inputs replay to identical outputs.

                    your domain type
                          │ derives LlmSchema
                          ▼
              ┌──────────────────────┐
              │  schema + lowering    │ (wire schema or compact prompt block)
              └──────────┬────────────┘
                         ▼
                 ModelRequest ──▶ transport ──▶ raw text
                         ▲                          │
                         │                          ▼
                         │      ┌────────────────────────────────┐
                         │      │ 1. PARSE   tolerant JSON         │
                         │      │            (fences, commas, EOF) │ → RepairNote
                         │      ├────────────────────────────────┤
                         │      │ 2. ALIGN   keys/enums/coercion/  │
                         │      │            discriminator infer   │ → AlignmentNote
                         │      ├────────────────────────────────┤
                         │      │ 3. VALIDATE strict decode, then  │
                         │      │            constraint IR, then   │ → FieldError
                         │      │            semantic validators   │
                         │      ├────────────────────────────────┤
                         │      │ 4. DECIDE  Done | Request | Failed│
                         │      └──────────┬─────────────┬─────────┘
                         │                 │ Done         │ Failed
                         │ 5. REPROMPT      ▼              ▼
                         └──── targeted   value A      ExtractionError
                              (only failing paths,      (carries report)
                               valid fields pinned)

1. Parse — tolerant JSON parsing. Strips code fences and prose preambles; repairs trailing commas, single quotes, unquoted keys, and comments; completes truncated output where unambiguous (closing open brackets at EOF). Every fix is recorded as a RepairNote(kind, detail).
2. Align — schema-aligned mapping. Walks the parsed document against the schema tree: case/snake/camel-insensitive key matching, fuzzy enum resolution ("celsius" → Celsius), unambiguous type coercion ("42" → 42), singleton→list promotion, and discriminator inference for unions when the tag is missing but the shape identifies exactly one variant. Every change is recorded as an AlignmentNote(path, kind, detail), JSONPath-addressed.
3. Validate. First strict structural decode (LlmSchema.decode), then the constraint IR (Iron refinements / Jakarta annotations / lowered keywords), then user SemanticValidators attached with withValidation. Problems are FieldErrors addressed by JsonPath and rendered as <path>: <problem> — e.g. .lineItems[2].amount: -3 violates MinNumber(0).
4. Decide. The state machine emits one NextStep: Done(value, report), a Request for another attempt (within policy), or Failed(error) once maxAttempts or the Budget is exhausted.
5. Reprompt — targeted. The next Request carries only the addressed errors, schema context for the offending paths, and the instruction to preserve all other fields; the previous output is pinned as assistant context. Fixing one field is cheaper and more reliable than regenerating the whole document.

The full prose walkthrough, policy/budget knobs, and real report transcripts are in retry-and-repair.md.

Sans-IO pure core, thin effect runners

The pipeline above is the ExtractionSession[A] state machine (typecast.contract.Session):

trait ExtractionSession[A]:
  def start: NextStep[A]
  def feed(response: Either[TransportError, ModelResponse]): NextStep[A]

It performs no I/O. start returns the first NextStep; whenever the session emits NextStep.Request(request: ModelRequest), the caller sends that request through a transport and hands the result back via feed exactly once. The session itself never calls a network, reads a clock, or chooses a thread (SPEC §3.1) — Attempt.durationMs is Option[Long] precisely because durations are injected by a runner; core has none.

This is the load-bearing decision behind the whole design. Driving the loop is the runner's only job, so each effect system gets a thin driver and none is privileged:

• typecast-runner-sync — blocking, returns Either[ExtractionError, A].
• typecast-runner-ce — Cats Effect F[_]; also carries the fs2 streaming surface (StreamingExtract.stream[F, A]).
• typecast-runner-zio — ZIO; ZStream streaming surface.

Because the core is pure, it is trivially property-testable (SPEC §8): a SyncTransport stub returning canned responses scripts an entire multi-attempt session with no mocking of effects, and round-trip / alignment-invariance / lowering-preservation properties can be checked deterministically. The same purity is what makes the last == strict streaming identity hold by construction — the terminal element comes out of the same pipeline as extract.

Strict decode vs. alignment

A hard internal boundary (the first of the three load-bearing facts in llms.txt): LlmSchema.decode never coerces. All fuzz — key case, enum matching, "42" → 42, discriminator inference — happens in stage 2 (Align) and is recorded as AlignmentNotes. Decode in stage 3 is a strict structural decode of an already-aligned document; a decode failure on aligned input is a validation error fed to the retry loop, not a silent fallback.

The contract states this directly on LlmSchema.decode:

STRICT structural decode of an already-aligned document. Coercion and fuzzing happen in the alignment stage, never here; decode failures on aligned input are validation errors fed to the retry loop.

The consequence: a value reaching your code was either exactly right on the wire or visibly repaired. Every coercion that got it there is itemized in the report, and all repair and alignment is exhausted before a single (token-costing) retry is spent.

Capability lowering and the post-hoc validation backstop

The second load-bearing fact. Every transport declares what it can enforce natively (typecast.contract.Transport):

final case class Capabilities(
    structuredMode: StructuredMode,  // Native | JsonMode | PromptOnly
    schemaSubset: SchemaSubset,      // which schema keywords survive on the wire
    streaming: Boolean,
    parallelTools: Boolean
)

Before a request is sent, the session lowers the full schema to that subset. Each false flag in SchemaSubset (numericBounds, stringLength, pattern, format, unions, optionalFields, additionalPropertiesFalse) is a keyword stripped from the wire schema — but the constraint stays in the IR and is enforced post-hoc by stage-3 validation, feeding the same targeted-retry loop as everything else. A constraint is never silently dropped; it moves from decode-time to validate-and-retry. User code is identical across providers.

The lowering decides how the schema travels, expressed as a SchemaPayload: WireSchema(json, strict) for Native/JsonMode, or PromptBlock(text) (the compact TypeScript-like rendering) for PromptOnly. OpenAI strict mode is the canonical lowering target — it also forces all fields into required, additionalProperties: false everywhere, and unions as anyOf.

The capability model, the recommendPromptFallback signal, and the SyncTransport SPI for writing your own provider are covered in providers-and-lowering.md; the generated, cell-by-cell on-wire-vs-validated-post-hoc table is in lowering-matrix.md.

ExtractionReport — the observability and eval seam

Every attempt is recorded, regardless of outcome, as an Attempt:

final case class Attempt(
    model: String,
    rawResponse: String,
    repairs: Vector[RepairNote],      // stage 1
    alignments: Vector[AlignmentNote],// stage 2, JSONPath-addressed
    errors: Vector[FieldError],       // stage 3, JSONPath-addressed
    usage: TokenUsage,
    durationMs: Option[Long] = None   // injected by a runner; core has no clock
)

final case class ExtractionReport(attempts: Vector[Attempt]):
  def totalUsage: TokenUsage = ...
  def attemptCount: Int = ...

extractWithReport always returns the report; on failure it rides on the ExtractionError (every ExtractionError case carries an ExtractionReport, reachable via the report extension), so a caller always sees what was attempted. This is deliberately the integration point for eval/CI tooling (SPEC §5.5): assert on success rate, repair rate, retries-per-request, or typed structural diffs over a golden set — no string matching, no judge needed for the mechanical layer. A repair that fires on every request is a visible prompt bug; an alignment that disappears on a model upgrade is measurable progress.

Module dependency graph

typecast-core carries the entire pure pipeline — JSON AST and tolerant parser, constraint IR, schema model, validation, the ExtractionSession state machine, ExtractionReport, retry policies, and prompt-mode rendering — and depends on the Scala standard library only (SPEC §3.2: transitive-dependency hygiene is a JVM-enterprise adoption gate). Everything else fans out from it:

                         ┌──────────────────┐
                         │  typecast-core    │  (scala-library only)
                         └───────┬───────────┘
        ┌──────────┬────────┬────┴─────┬──────────┬────────────┐
        ▼          ▼        ▼          ▼          ▼            ▼
   derivation    iron   anthropic   openai    runner-sync   java
        │                  │          │       runner-ce
        ▼                  └────┬─────┘       runner-zio       │
      tools                     │                              │
        │                       │            langchain4j ◀─────┘
        ▼                       │            (also dependsOn anthropic, openai)
       mcp                      ▼
                              bench (core, anthropic, openai, runner-sync)

Exact wiring (from build.sbt):

• derivation → core (the derives LlmSchema macro; LlmSchema.derived delegates to typecast.derive.Derive).
• iron → core (Iron refinements lowered into the constraint IR).
• anthropic, openai → core (each a SyncTransport; openai also covers OpenAI-compatible servers — Ollama, vLLM, Groq).
• runner-sync, runner-ce, runner-zio → core (effect drivers; runner-ce adds fs2).
• tools → derivation, then mcp → tools (derives LlmTool reuses the pipeline; mcp exports the registry).
• java → core (the records / sealed-interfaces / Jakarta-validation facade).
• langchain4j → java, anthropic, openai (a ChatModel-backed transport and Java distribution channel).
• bench → core, anthropic, openai, runner-sync (the benchmark harness, publish / skip).

The single root toward core is the design statement: the schema/pipeline engine — the piece the rest of the JVM ecosystem lacks — has no dependency on any provider, effect system, or constraint front-end. Those plug in around the contract package, never the other way around.