LLM Output Validation with Pydantic and Zod: A Production Pattern

Valid JSON can still contain the wrong fields, types, or values. Pydantic and Zod let you validate model output against an application schema and business rules before it reaches a database or another system.

The Problem with Relying on JSON Parsing Alone

JSON parsing only tells you that the output is syntactically valid JSON. It does not tell you that the JSON contains the fields you need, that those fields have the right types, or that the values make sense in context.

A model that returns {"status": "sort of done", "count": "many"} can parse successfully while violating the application's expectations. Runtime type checking and semantic validation catch these failures before they propagate.

Schema-First Design: One Source of Truth

The right architecture defines the schema once and derives everything else from it:

1. The JSON Schema passed to the LLM API (for constrained generation)
2. The runtime validator applied to the model's response
3. The TypeScript or Python type used in application code

When these three are separate and maintained independently, they drift. The API call uses an old schema. The validator enforces a different schema. The application code expects a third variant. When a model output fails, you cannot tell which layer is wrong.

In Python with Pydantic

Define your model as a Pydantic class. Use model.model_json_schema() to generate the JSON Schema for the API call. Use Model.model_validate() to parse and validate the model's response.

from pydantic import BaseModel, Field
from typing import Optional
 
class ExtractedContact(BaseModel):
    name: str
    email: Optional[str] = None
    phone: Optional[str] = None
    action_items: list[str] = Field(default_factory=list)

With this model, ExtractedContact.model_json_schema() generates the JSON Schema to pass to the API. ExtractedContact.model_validate_json(response_text) validates the model's response and raises a ValidationError if it does not match. You never maintain two separate schema representations.

Pydantic's ValidationError contains field-level failure messages, which you can format and pass back to the model in a retry prompt. This makes retry logic precise rather than generic.

In TypeScript with Zod

The same pattern applies in TypeScript. Define your schema with Zod, use .toJsonSchema() (via a compatibility library) for the API call, and schema.parse() for runtime validation.

import { z } from "zod";
 
const ExtractedContact = z.object({
  name: z.string(),
  email: z.string().email().nullable(),
  phone: z.string().nullable(),
  action_items: z.array(z.string()),
});
 
type ExtractedContactType = z.infer<typeof ExtractedContact>;

ExtractedContact.parse(parsed_json) throws a ZodError with field-level failure details on validation failure. z.infer derives the TypeScript type — your application code uses the inferred type, so type safety flows from the schema.

Semantic Validation Beyond Type Checking

Type validation catches structural failures. Semantic validation catches value-level failures that are structurally valid but meaningless or incorrect.

Add validators for business rules that the schema cannot express:

• Date fields should be parseable as actual dates
• Numeric fields should be within plausible ranges
• Enum fields should map to known values
• URL fields should be parseable as valid URLs

In Pydantic, use @field_validator decorators. In Zod, use .refine(). These run after structural validation and before the value is used in application code.

Be selective about semantic validators. Over-validation creates false failures when the model's output is correct but in a slightly different format than your validator expects. Date validation that accepts only one ISO format will fail on a model that outputs a different valid ISO representation.

Retry Logic with Validation Feedback

When validation fails, you have two options: fail hard (raise an exception, return an error to the caller) or retry with feedback. Retry with feedback is appropriate when the failure mode is recoverable.

A retry prompt appends the validation failure to the conversation:

"Your previous response failed validation. The field email must be a valid email address or null. You returned "not provided". Please correct this field and return the full object again."

A retry can correct some field-level failures; measure whether it helps for your inputs. Limit retries to one or two — beyond that, the failure is likely systematic and retrying will not help.

Log every retry and every validation failure with the field-level details. These logs are your best signal for prompt improvements — recurring validation failures on a specific field usually mean the prompt's output specification is ambiguous.

Handling Constrained Decoding

Where a provider supports strict schema-constrained generation, use it to improve structural compliance. Handle refusals, incomplete responses, and API errors separately, and validate the response before using it.

In this case, Pydantic or Zod validation shifts from structural enforcement to semantic validation. Keep structural validation at the application boundary as well as checks for meaning and business rules. You are checking for date formats, range constraints, and business rule compliance.

This is the correct layering: constrained generation for structure, runtime validation for semantics. Both are necessary. Neither is sufficient alone.

What to Do When Validation Keeps Failing

Persistent validation failures on the same field or same prompt usually indicate one of three problems:

The prompt's output specification is ambiguous. The model is making a reasonable interpretation of an underspecified instruction. Clarify the instruction or add an example.

The schema is too strict for the actual data. If email is required but emails are absent in a significant fraction of inputs, make it optional. Forcing the model to populate a field when no data exists causes hallucination.

The model is not capable of this extraction reliably. Test with a larger model. If a larger model succeeds and a smaller one fails consistently, this is a capability issue, not a prompt issue.

In all three cases, the validation failures are giving you diagnostic information. Log them, analyze the patterns, and treat them as prompt feedback rather than noise.