Skip to content

Providers & capability lowering

The same user code runs on every provider. What changes underneath is how much of your schema the provider can enforce natively — and TypeCast's job is to make that difference invisible: lower the schema to what the provider enforces, and let validation enforce the rest.

The capability model

Every transport declares its capabilities:

final case class Capabilities(
    structuredMode: StructuredMode, // Native | JsonMode | PromptOnly
    schemaSubset: SchemaSubset,     // which schema keywords survive on the wire
    streaming: Boolean,
    parallelTools: Boolean
)
  • Native — the provider enforces a JSON Schema at decode time (Anthropic structured outputs, OpenAI strict mode). The schema travels as a wire schema, lowered to the provider's subset.
  • JsonMode — the provider guarantees syntactic JSON but not schema conformance (Ollama and most OpenAI-compatible servers). The schema travels inside the prompt.
  • PromptOnly — no JSON guarantee at all; the prompt block plus repair/alignment do the heavy lifting.

In prompt modes the schema renders as a compact TypeScript-like notation rather than raw JSON Schema — materially cheaper in tokens and better followed (A | B unions, ? optionals, { [key: string]: V } maps).

SchemaSubset flags what a Native provider's enforcement actually supports: numeric bounds, string lengths, pattern, format, unions, optional fields, additionalProperties: false.

Lowering, and the post-hoc backstop

Before a request is sent, the schema is lowered: every keyword the provider can't enforce is stripped from the wire schema, but the constraint stays in the IR and is enforced by stage-3 validation after decode — feeding the same targeted-retry loop as everything else. A constraint is never silently dropped; it just moves from decode-time to validate-and-retry.

Example: OpenAI strict mode rejects minimum. Your age: Int :| Interval.Closed[0, 130] still works there — the bound leaves the wire schema, and an out-of-range value comes back as .age: 200 violates MaxNumber(130) plus a targeted reprompt.

The full, generated cell-by-cell behaviour lives in lowering-matrix.md — generated from the real renderers by a core test suite (it fails when stale), so the table is always true.

What each bundled provider declares

Transport Mode Lowered out of the wire schema (validated post-hoc instead)
AnthropicTransport Native numeric bounds, string lengths
OpenAi (strict) Native numeric bounds, string lengths, pattern, format, optional fields (all fields required + null-union on the wire)
OpenAiCompatible.jsonMode / Ollama() JsonMode nothing stripped (compact prompt block); unknown-field policy post-hoc
OpenAiCompatible.promptOnly PromptOnly as JsonMode, minus the JSON guarantee

OpenAI strict mode is the canonical lowering target: it also requires every property in required and additionalProperties: false everywhere, and accepts unions only as anyOf — the renderer handles all of that automatically.

recommendPromptFallback

Some shapes are not just unenforceable but harmful on a strict wire: a Map[String, V] under OpenAI strict mode must erase its value schema to a closed empty object — and a strict provider will then likely return an empty object for the map. Lowering reports this: LoweredSchema.recommendPromptFallback is true exactly when the lowered wire schema would likely make a Native provider produce useless output (today: map value-schema erasure). Every drop is also itemized in the DroppedFeature report, so nothing vanishes silently. Automatic downgrade from Native to the compact prompt block on this signal is a documented future enhancement; in v1, prefer a JsonMode/PromptOnly transport (or restructure the map into a list of key/value pairs) when extracting map-heavy schemas through strict providers.

Custom transports

A provider integration is one trait:

trait SyncTransport:
  def capabilities: Capabilities
  def send(request: ModelRequest): Either[TransportError, ModelResponse]

Declare honestly what the provider enforces, return the raw text, and the pipeline does the rest. Transports never retry internally — retry belongs to the policy, where it is observable and budgeted. (This is also exactly how tests script providers: a stub transport returning canned responses.)