Extract Validated JSON from LLM Responses

LLM Structured Output covers getting reliable, typed, validated data out of LLM API responses instead of parsing free text, across the three main provider mechanisms: OpenAI's response_format with json_schema (Structured Outputs, with strict: true enforcing constrained decoding so the model can only emit tokens that conform to the schema - every object in a strict-mode schema must also set additionalProperties: false, or OpenAI rejects the request with a 400 error before any response is generated), Anthropic's tool_use block, where a single tool is defined with the target schema as input_schema and tool_choice forced to it, and Google Gemini's generationConfig.responseSchema with responseMimeType: "application/json". For local models it covers GBNF grammars or a --json-schema flag for token-level constrained decoding. The workflow: define a concrete schema (Pydantic BaseModel in Python, Zod converted with zodResponseFormat() in TypeScript, or raw JSON Schema), give every field a description string the model uses as an implicit instruction, put the extraction instruction in the system prompt rather than only the user message - system prompt carries more attention weight for behavior, the user message should carry only the source data - reinforce structure, and validate the response in application code even after constrained decoding, since schema conformance alone does not catch semantic issues like empty strings, out-of-range numbers, or a model confidently mismatching the actual sentiment of an ambiguous review.

Use it when extracting structured data (JSON objects, arrays, enums) that feeds directly into code - a database write, an API call, UI rendering - or when responses come back malformed, missing fields, or wrongly typed. It does not cover free-form text generation, Zod for form or API input validation (that is a separate zod-validation-expert skill), prompt engineering for prose quality, or using tool_use for actual tool execution rather than as a structured-output extraction hack.

Input is source text plus a defined schema. Output is a validated typed object, for example a Pydantic model:

class Sentiment(str, Enum):
    positive = "positive"
    negative = "negative"
    neutral = "neutral"

A validation-failure retry loop feeds the original input, the failed output, and the validation error back to the model with an instruction to fix it, capped at 3 attempts; every call is logged with input, raw response, parsed result, and any validation errors so a production failure can be traced to a schema, prompt, or model-regression cause. Several provider-specific edge cases need explicit handling: OpenAI can return a refusal field instead of parsed data when it judges a request unsafe, so code must check response.choices[0].message.refusal before touching .parsed; array fields can come back empty even when the source has the data, usually fixed by making the field description prescriptive ("return at least one if any is referenced"); enum values can fail validation on casing alone, since OpenAI's strict mode matches case exactly while Anthropic may not; source text that exceeds the context window should be chunked and extracted independently per chunk rather than trusted to a single call; and streaming structured output arrives as partial JSON that cannot be parsed mid-stream - buffer until the stream completes, or use the SDK's built-in partial parser, though Anthropic's tool_use blocks arrive complete in one content_block_stop event with no partial assembly needed. Prefer flat schemas with a handful of fields over deeply nested ones - accuracy drops as nesting grows, and a two-pass extraction (entities first, details second) beats one deeply nested call; use Literal/enum types instead of free-form strings for categorical fields; give optional fields real defaults (Optional[str] = None / .optional()) so the model is not forced to hallucinate a value where the source has none; and keep the LLM extraction schema separate from the application's database schema, mapping between them in code rather than making the model understand foreign keys and computed fields.

OpenAI (gpt-4o family, pinned to a dated version like gpt-4o-2024-08-06 since aliases change behavior), Anthropic Claude (tool_use/tool_result), Google Gemini (responseSchema), and local inference via llama.cpp or vLLM grammars. Companion libraries: Pydantic's model_validate(), Zod's .parse(), and zodResponseFormat() from the openai npm package; instructor and marvin for Pydantic-based extraction. Avoid OpenAI's legacy json_object mode with no schema attached - it guarantees valid JSON syntax only, not conformance to any particular shape.

Engineers building a pipeline where LLM output writes to a database, calls an API, or renders UI directly, and needs guaranteed shape rather than a best-effort parse of free text - anyone hitting malformed JSON, missing fields, or type mismatches from an LLM response today.