Structured Outputs with JSON Schema and Pydantic

Make models return validated JSON using JSON Schema, provider structured-output modes, and Pydantic models, and recover when validation fails.

Est. 30 min Updated Suggest a fix
// Part 01 — Why structure

Text That Looks Like JSON Is Not Yet Data

A model reply is untrusted text until your application parses and validates it. A response that looks like JSON can still omit a field, return a string where an integer is needed, invent an enum value, or contain commentary around the object. If a downstream service uses this result to create a support ticket, update a catalogue, or show a UPI payment status, malformed data can become a real product bug.

Structured output narrows the format the model is asked to produce. A provider may expose JSON mode or a stricter schema-constrained response mode. JSON Schema describes the allowed shape; Pydantic validates values in Python and turns the result into a typed object. Use both the provider feature and local validation where possible: neither removes the need to handle refusal, truncation, transport errors, or domain-specific rules.

LayerResponsibilityExample check
Provider format modeEncourage or constrain response syntaxResponse conforms to supported schema mode
JSON parserConvert serialized text into a Python valueValid JSON syntax, no trailing prose
Pydantic modelValidate types, required fields and constraintsConfidence is between 0 and 1
Business logicCheck the result is appropriate for this workflowPayment state is confirmed from trusted backend data
⚠️ Important
Valid JSON is not the same as true information. A schema can prove that status is a string; it cannot prove that a payment really settled. Verify facts and authorization against trusted systems.
// Part 02 — Define contract

Design a Small Pydantic Model

Start from what the application needs, not every detail the model might mention. Use explicit field types, required values, bounds, and a controlled vocabulary where practical. The example classifies an inbound delivery-support message for triage. The order and payment data remain in the application database; the model only suggests a category and a short explanation.

bash
python -m pip install pydantic
triage_schema.py
python
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field

class TriageResult(BaseModel):
    model_config = ConfigDict(extra="forbid")

    category: Literal["delivery", "payment", "refund", "other"]
    summary: str = Field(min_length=1, max_length=240)
    needs_human: bool
    confidence: float = Field(ge=0, le=1)

print(TriageResult.model_json_schema())

extra="forbid" rejects surprise fields instead of quietly accepting an expanding contract. Bounds prevent excessively long summaries and out-of-range confidence values. Keep a schema version with your application when persisted records may outlive a deployment; changing a field can affect old data and clients.

🎯 Pro Tip
Make fields required unless there is a clear reason they may be absent. Avoid ambiguous optional strings that use an empty value to mean several different things.
// Part 03 — Generate

Ask the Provider for a Schema-Constrained Result

Provider APIs differ: they may take a JSON Schema, a Pydantic class through an SDK helper, or a response-format option. Read the documentation for your selected provider and SDK version. The following is intentionally provider- neutral pseudocode around the stable contract: supply a schema, retrieve a JSON text value, and validate locally.

parse_result.py
python
import json
from pydantic import ValidationError
from triage_schema import TriageResult


def parse_result(raw_text: str) -> TriageResult:
    """Parse the provider's JSON text and enforce our local contract."""
    value = json.loads(raw_text)
    return TriageResult.model_validate(value)

raw = '{"category":"payment","summary":"Customer reports pending status.","needs_human":true,"confidence":0.82}'
try:
    result = parse_result(raw)
except (json.JSONDecodeError, ValidationError) as error:
    print("The response needs recovery:", error)
else:
    print(result.category, result.needs_human)

Do not paste provider-specific arguments into this adapter until you have checked the official SDK documentation; names and capabilities vary. If the provider returns a parsed object, validate that object locally as well. If it returns a refusal or a truncated response, handle that provider signal explicitly rather than treating it as a schema error and blindly retrying.

// Part 04 — Validate

Separate Shape Checks from Domain Rules

Pydantic validates local shape and types. Domain rules add meaning: for example, a payment support classification might require human review if confidence is low. These checks should be deterministic and should not rely on the model to confirm its own authority. Persist provenance such as the model configuration, schema version, and whether the result passed review when the workflow requires an audit trail.

triage_policy.py
python
from triage_schema import TriageResult


def apply_policy(result: TriageResult) -> dict:
    needs_review = result.needs_human or result.confidence < 0.65
    return {
        "suggested_category": result.category,
        "summary": result.summary,
        "review_required": needs_review,
        "confidence": result.confidence,
    }

# A payment status is fetched from the trusted payment service separately.
  1. Define a narrow schema from the product contract and document its version.
  2. Request a supported provider structured-output mode, supplying the schema in the documented format.
  3. Check refusal, incomplete-output and transport status before parsing.
  4. Parse and validate locally, then apply deterministic business rules and human review policy.
  5. Record validation outcome and schema version without logging sensitive prompt content by default.
📌 Real World Example
In a Swiggy-style delivery support workflow, the model can propose "delivery" as a category. The order service, not model-generated JSON, determines the actual delivery status and whether a refund is allowed.
// Part 05 — Recover

Use a Bounded Repair or Retry Path

A malformed response can be repaired, but recovery must be explicit and bounded. First distinguish a transient provider error from invalid output. For invalid output, you may make one targeted correction request that includes the validation issues and the original task, while limiting the returned data. Avoid sending secrets or unrelated customer records into a repair prompt. If validation still fails, route to a safe fallback or human review.

bounded_recovery.py
python
from pydantic import ValidationError
from triage_schema import TriageResult

MAX_REPAIR_ATTEMPTS = 1


def validate_with_repair(initial_text: str, request_repair) -> TriageResult | None:
    text = initial_text
    for attempt in range(MAX_REPAIR_ATTEMPTS + 1):
        try:
            return TriageResult.model_validate_json(text)
        except ValidationError as error:
            if attempt == MAX_REPAIR_ATTEMPTS:
                return None  # caller routes to review or a safe fallback
            # Keep the repair request constrained; don't expose unrelated data.
            text = request_repair(
                invalid_output=text,
                issues=error.errors(include_input=False),
                schema=TriageResult.model_json_schema(),
            )
    return None

request_repair is an application adapter, not a specific provider API. It should use the configured model, a short timeout, and the provider's documented structured-output feature where available. Do not let a repair loop recurse indefinitely or treat a corrected shape as proof that the content is factually correct.

// Part 06 — Test

Test the Contract Like Any Other Interface

Keep fixtures for valid, missing-field, wrong-type, extra-field, invalid-enum, malformed JSON, refusal, and truncated cases. Test the schema itself when it changes, and test business rules independently of the model. During evaluation, compare model output with expected labels and have a human inspect failures; schema conformance alone is not a quality score.

test_triage_schema.py
python
import pytest
from pydantic import ValidationError
from triage_schema import TriageResult


def test_valid_result():
    result = TriageResult(
        category="delivery", summary="Courier has not arrived.",
        needs_human=False, confidence=0.9,
    )
    assert result.category == "delivery"


def test_rejects_unknown_category():
    with pytest.raises(ValidationError):
        TriageResult(
            category="urgent", summary="Review this",  # type: ignore[arg-type]
            needs_human=True, confidence=0.7,
        )


def test_rejects_extra_fields():
    with pytest.raises(ValidationError):
        TriageResult.model_validate({
            "category": "other", "summary": "Question", "needs_human": False,
            "confidence": 0.5, "account_number": "not-allowed",
        })
💡 Note
Treat schemas as API contracts. Review compatibility before removing a field, renaming a key, or changing the meaning of an existing value.
// Part 07 — Pitfalls

Common Structured-Output Mistakes

Assume JSON mode guarantees the right schema

Why: A syntactically valid object can still omit required fields or use invalid values.

Fix: Use a supported schema mode and validate the returned value locally.

Use model output as authoritative business state

Why: A valid schema checks shape, not truth, identity, permission or payment status.

Fix: Fetch facts and authorize actions using trusted application services.

Repair forever until something parses

Why: An unbounded loop can increase spend, delay responses and conceal a persistent failure.

Fix: Set a small attempt limit and route unresolved cases to review or fallback.

Accept any fields for future flexibility

Why: Unexpected keys can hide schema drift or expose data the consumer did not expect.

Fix: Forbid extra fields where appropriate and version intentional schema changes.

// Part 08 — Practice

Build a Structured Support Classifier

Practice

Validate the suggestion before your app uses it

  1. Extend the triage schema with a nullable language field chosen from a small documented set; decide whether it is required.
  2. Generate a JSON Schema from Pydantic and inspect required fields, bounds and additional-property behaviour.
  3. Write tests for malformed JSON, missing fields, an invalid enum, a confidence outside its range, and an extra field.
  4. Add a deterministic rule that routes low-confidence or payment-related cases to a human queue.
  5. Sketch a provider adapter using the structured-output feature documented for your chosen SDK; keep model and credentials in configuration.
  6. Decide what you will log for debugging while excluding prompts and personal data by default.

Hint: Keep the provider call replaceable. Your local validation and domain policy should be testable without making a network request.

🎯 Key Takeaways

  • ✓Model output remains untrusted text until parsing, schema validation and domain checks succeed.
  • ✓JSON Schema describes shape; Pydantic enforces a local typed contract in Python.
  • ✓Use a provider structured-output feature documented for the selected provider and SDK version.
  • ✓Handle refusal, truncation, transport errors and validation failures as distinct outcomes.
  • ✓Keep repair attempts bounded and send only the minimum context needed to correct a response.
  • ✓Use trusted services for facts and permissions; valid JSON does not make a claim true.

Next

What comes next

Structured data is useful on its own. Next, let a model request an action and build the guarded loop that executes it safely.

Lesson 04 · Function and Tool Calling →

HELP US IMPROVE

How was this lesson?

Your note is shared with the Chaduvuko team for review. Please don’t include personal or sensitive information.

Choose your feedback

Spotted a mistake or something unclear? Suggest a fix on GitHub or email us. Select the text first and it is quoted for you.

Share

Discussion

0

Have a better approach? Found something outdated? Share it — your knowledge helps everyone learning here.

Continue with GitHub
Loading...