Extract Validated JSON from LLM Responses

Extracts typed, validated data from LLM API responses instead of parsing free text, covering three provider-specific approaches: OpenAI's response_format with a JSON Schema (strict: true enables constrained decoding so the model can only emit schema-conforming tokens), Anthropic's tool_use block (defining one tool whose input_schema is the target schema, forcing it with tool_choice, then reading the structured data from the tool_use block rather than any text block), and Google Gemini's generationConfig.responseSchema with responseMimeType: "application/json". The core workflow is schema-first: define every field's type, required/optional status, and enum values before writing any code; write the schema in the caller's own language (a Pydantic BaseModel in Python, a Zod schema converted via zodResponseFormat() in TypeScript, or raw JSON Schema for direct API calls); give every field a description string, since models use it as an implicit extraction instruction; reinforce structure in the system prompt rather than the user message, since the system prompt carries more weight for behavioral instructions; validate the response in application code (model_validate() / .parse()) even after constrained decoding, since schema conformance does not guarantee correct values; and build a capped retry loop (max 3 attempts) that feeds the validation error back to the model, logging the input, raw response, parsed result, and any validation errors on every call.

Use it whenever LLM output needs to feed directly into code - database writes, API calls, UI rendering - as JSON objects, arrays, or enums, or when debugging malformed JSON, missing fields, or wrong types coming back from a model; it also covers controlled generation / constrained decoding / grammar-based sampling in local models via GBNF grammars or a --json-schema flag (llama.cpp, vLLM). Do not use it for free-form text generation (summaries, essays, chat), for Zod used as general form/API input validation, for prompt engineering aimed at text quality rather than structure, or for orchestrating real external tool calls - tool_use here is a structured-output extraction hack, not tool execution. Traps it flags explicitly: OpenAI's legacy json_object mode guarantees valid JSON syntax but not schema conformance; additionalProperties: true breaks OpenAI strict mode with a 400 error before any response is returned; and assuming a schema-conforming response is a semantically correct one is a mistake constrained decoding cannot fix.

Input is unstructured or semi-structured text plus a target schema; output is a validated, typed object. A representative OpenAI example:

from pydantic import BaseModel, Field
from openai import OpenAI
from enum import Enum

class Sentiment(str, Enum):
    positive = "positive"
    negative = "negative"
    neutral = "neutral"

class ReviewAnalysis(BaseModel):
    sentiment: Sentiment = Field(description="Overall sentiment of the review")
    key_topics: list[str] = Field(description="Main topics mentioned, max 5")
    purchase_intent: bool = Field(description="Whether the reviewer would buy again")
    confidence_score: float = Field(ge=0.0, le=1.0, description="Model confidence 0-1")

client = OpenAI()
response = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=[
        {"role": "system", "content": "Extract structured review analysis."},
        {"role": "user", "content": "This laptop is amazing. The battery lasts forever and the keyboard feels great. Definitely buying the next version."}
    ],
    response_format=ReviewAnalysis,
)
result = response.choices[0].message.parsed

Edge cases the skill covers separately: text longer than the context window should be chunked and merged rather than trusted to a single call; OpenAI can return a refusal field instead of parsed data, which must be checked before .parsed is accessed; enum values can fail validation on casing mismatches between the schema and the model's output; and OpenAI's streaming structured output arrives as partial JSON that cannot be parsed mid-stream, unlike Anthropic's tool_use blocks, which arrive complete in one content_block_stop event.

Covers OpenAI (gpt-4o family), Anthropic Claude (tool_use/tool_result), and Google Gemini (responseSchema) APIs directly, plus the Python ecosystem around them - Pydantic, instructor, marvin - and the TypeScript zod / zodResponseFormat() pairing from the openai npm package. For local inference it covers GBNF grammar-based constrained decoding in llama.cpp and vLLM.

Developers building pipelines where LLM output must be reliable enough for a database write, an API call, or direct UI rendering, and who need to choose the right provider-specific structured-output mechanism, avoid the schema-conformance traps each one has, and build the validation and retry logic production systems require around it.