Getting Reliable Structured Data From an LLM

Asking a model to "return JSON" and hoping is not a plan, models occasionally drop a closing brace, use a single quote, or wrap the JSON in a sentence of explanation. If anything downstream of the model parses that output automatically (a database write, an API call, a report), an unparseable response should never reach it silently. This module covers the practical pattern for getting structured data out reliably.

What you'll learn

• Why asking nicely for JSON fails often enough to matter in production
• The self-correcting extraction pattern that fixes most failures automatically
• What to do when the model still can't produce a valid schema

1. The problem

Every downstream step that consumes model output assumes a specific shape: a field called total_revenue that's a number, a status field that's one of three strings, not free text. When the model doesn't return that shape exactly, you get one of two bad outcomes: a hard crash on json.loads(), or worse, code that silently accepts a malformed value and produces a wrong result nobody notices until later.

"Ask nicely and hope" gets you correct JSON most of the time. For anything that touches money, a database write, or an action the agent takes on someone's behalf, "most of the time" isn't good enough.

2. Options & when to use each

Option	Good for	Costs you	When to pick it
Plain prompting for JSON	Low-stakes internal tools, prototypes	Occasional parse failures you have to handle	Getting started, or output that a human reviews anyway
Schema + self-correcting retries (Instructor)	Most production use cases	A little extra latency when a retry is needed	Default choice, validates against your schema and automatically asks the model to fix its own mistake
Grammar-constrained generation (Outlines)	High-volume, cost-sensitive, or safety-critical output where a retry isn't acceptable	Only works with models you control the internals of (local/open models), not hosted APIs like Claude or Gemini	You're running your own model and need a mathematical guarantee, not just a very reliable one

For almost every project in this course, calling Claude or Gemini through their APIs, schema validation with automatic retries (the Instructor pattern) is the right default. Grammar-constrained generation only becomes relevant if you're self-hosting a model, which is a much less common setup.

3. Build it

Define the shape you expect

Using Pydantic, define a structured schema representation of the data fields and constraints you want the model to extract and validate:

from pydantic import BaseModel, Field

class InvoiceExtraction(BaseModel):
    vendor_name: str
    total_amount: float = Field(gt=0, description="Total amount in USD")
    due_date: str = Field(description="ISO 8601 date, e.g. 2026-08-01")
    line_items: list[str]

Extract with automatic validation and retry

Initialize the Instructor-wrapped Anthropic client to handle model requests and automatically manage validation retries based on the Pydantic schema definition:

import instructor
from anthropic import Anthropic

client = instructor.from_anthropic(Anthropic())

def extract_invoice(document_text: str) -> InvoiceExtraction:
    return client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=1024,
        response_model=InvoiceExtraction,
        max_retries=2,
        messages=[
            {"role": "user", "content": f"Extract the invoice details:\n\n{document_text}"}
        ],
    )

What happens under the hood when the model's first attempt doesn't validate, say, total_amount came back negative, which your schema forbids: Instructor catches the validation error, sends the model a follow-up message describing exactly what failed ("total_amount must be greater than 0, you returned -450.00"), and asks it to correct its own output. This mimics how a person debugging a script would fix it, by reading the specific error, not guessing again from scratch. Most failures resolve on the first retry.

Handle the case where it still fails

To prevent application crashes and avoid importing invalid data when a document fails to parse after all retry attempts, wrap the extraction call with error-handling logic:

from instructor.exceptions import InstructorRetryException

def safe_extract(document_text: str) -> InvoiceExtraction | None:
    try:
        return extract_invoice(document_text)
    except InstructorRetryException:
        # The model couldn't produce a valid schema after all retries , 
        # don't guess. Flag it for a human, don't write bad data downstream.
        log.warning("invoice_extraction_failed", text_preview=document_text[:200])
        return None

Never let a failed extraction silently become a zero, an empty string, or a guess. If it can't be validated, it should stop the pipeline and get a human's attention, that's the whole point of validating in the first place.

4. What goes wrong

Mistake	How you notice it	The fix
Parsing raw response.content with json.loads() directly	Occasional JSONDecodeError in production, usually on your busiest day	Use a schema library (Instructor) instead of parsing raw text yourself
Schema too strict for what the model can realistically know	Constant retries, high latency, MaxRetriesExceeded errors	Loosen the field (e.g., make due_date optional if it's not always present in the source document)
Prompt asks the model to "explain your reasoning" and return JSON in the same response	Parsing fails because there's prose before/after the JSON	Keep the extraction call focused only on the schema, do reasoning in a separate step if you need it
Silently accepting a failed extraction as None or a default value everywhere downstream	A report shows $0.00 where an invoice actually failed to parse	Treat extraction failure as a distinct error case, not a valid zero, surface it, don't hide it
Switching models without re-testing the schema	A schema that worked with Claude produces more retries with a different model	Re-run your extraction test set whenever you change models

5. Confirm it worked

To confirm that both valid extraction inputs succeed and invalid/garbled document inputs fail cleanly under the validation schema, run these unit assertions:

# Run a known-good document through and check the shape
result = extract_invoice(sample_invoice_text)
assert isinstance(result, InvoiceExtraction)
assert result.total_amount > 0

# Confirm a malformed/edge-case document is caught, not silently accepted
result = safe_extract(garbled_document_text)
assert result is None  # should fail cleanly, not return a guess

If both cases behave as described, clean extraction on good input, a clear failure signal on bad input, the pipeline is doing its job.

Next Step: proceed to Defending Against Prompt Injection to guard against the attack vector unique to agentic systems.