Structured Outputs: The Unsung Hero of Reliable AI Systems
A practical guide to LLM structured outputs covering JSON Schema, strict function calling, schema design, TypeScript implementation, domain validation, versioning, evaluations, and production lessons.
Structured Outputs: The Unsung Hero of Reliable AI Systems
Structured outputs make an LLM return data that follows a defined schema instead of an unpredictable block of prose. They eliminate entire classes of formatting and parsing failures, but they do not prove that the values are true, authorized, or safe to use.
The direct answer
Structured outputs turn a model response into a typed contract that software can reliably consume. Instead of asking the model to return JSON and hoping it follows the instructions, the application supplies a schema that defines required fields, allowed values, arrays, nested objects, and failure states.
That solves the format problem. It does not solve the truth problem. Production systems still need source validation, business rules, authorization, state checks, evaluations, and human review for high-impact decisions.
A contract between generation and software
What are LLM structured outputs?
Structured outputs are model responses constrained to a predefined data shape. The shape is commonly expressed through JSON Schema or a language-native schema library such as Zod in TypeScript or Pydantic in Python.
JSON Schema is a declarative language for defining the structure and constraints of JSON data. A schema can specify that a field must be a string, integer, array, nested object, enum, or null. It can also require properties and reject unexpected keys.
“Extract the appointment details.”
Defines the task, but not a machine-enforceable response contract.
“Return valid JSON.”
May produce parseable JSON while omitting fields or changing types.
“Return this exact schema.”
Constrains keys, types, enums, arrays, and supported nested objects.
“Verify every value before use.”
Adds source checks, business rules, permissions, and workflow validation.
OpenAI’s current documentation distinguishes schema-constrained output from ordinary JSON mode: both produce JSON, but only Structured Outputs are designed to adhere to the supplied schema. The supported feature set is a subset of JSON Schema, so schemas must be tested against the exact API and model used.
Choose the right output mode
How do free text, JSON mode, and structured outputs compare?
| Capability | Free text | JSON mode | Structured outputs |
|---|---|---|---|
| Machine parseable | Not reliably | Yes, as JSON | Yes, as schema-conforming JSON |
| Required keys | Prompt-dependent | Not guaranteed | Defined by the schema |
| Field types | Uncontrolled | May vary | Constrained to supported types |
| Enum values | Can drift | Can invent alternatives | Limited to approved values |
| Unexpected keys | Common | Possible | Can be rejected by the contract |
| Integration | Parsing and repair required | Schema validation still needed | Typed parsing with fewer formatting retries |
| Factual accuracy | Not guaranteed | Not guaranteed | Still not guaranteed |
| Best use | Human-facing drafts | Simple or legacy JSON needs | Extraction, agents, tools, and workflows |
Use free text for people. Use structured outputs for software.
A user-facing explanation can remain natural language while the application receives a separate object containing status, evidence, fields, proposed actions, and review requirements.
Remove ambiguity at the system boundary
Why do structured outputs make AI systems more reliable?
They eliminate parser-class failures
The application no longer needs to strip markdown, repair commas, rename keys, or guess whether “three” is a string or number.
They make failure explicit
A contract can include supported, uncertain, refused, blocked, and requires-review states instead of forcing false success.
They improve type safety
SDK helpers can map the response into typed objects, reducing hand-written parsing and making downstream code easier to test.
They constrain workflow branches
Enums give the application a finite set of expected states rather than interpreting arbitrary prose.
They make evaluations objective
Tests can assert exact fields, statuses, citations, and actions instead of grading whether a paragraph looks right.
They simplify observability
Teams can track nulls, uncertain states, validation errors, retries, and schema versions across production traffic.
“The patient probably takes 10 mg daily.
The record ID may be PT-183.
You should update the medication list.”{
"status": "requires_review",
"patient_id": null,
"dose_mg": 10,
"frequency": "daily",
"proposed_action": "none"
}Shape is not truth
What problems do structured outputs not solve?
A schema can guarantee that a field is a string. It cannot guarantee that the string identifies a real patient. It can constrain a code to a pattern. It cannot prove that the code is supported by the source.
Fabricated values can still fit the schema
A valid-looking ID, code, date, or citation may still be invented.
The schema does not grant permission
A correct tool call can still target the wrong tenant or resource.
Types do not encode the full workflow
An ISO date may still violate scheduling, contract, or case rules.
Required fields can contain weak placeholders
The model may return generic values when evidence is missing.
Conformance is not risk assessment
A perfectly structured result can still be unsafe or outside scope.
Never execute a tool call merely because it passed schema validation. Treat model output as untrusted input until identity, authorization, resource state, business rules, duplicate risk, and approval requirements have been checked.
Good schemas encode operating decisions
How should you design an LLM output schema?
A schema should reflect what the application needs to decide next. It should not mirror the entire database or become a dumping ground for every possible explanation.
Name fields for meaning
Prefer requires_human_review over flag, and evidence_status over vague confidence.
Use enums for workflow decisions
Supported, uncertain, and blocked are easier to handle than arbitrary labels.
Represent optionality deliberately
Distinguish not found, not applicable, not authorized, and not requested.
Keep actions separate from answers
Explanations, extracted data, and proposed actions need different validation rules.
Include evidence references
Store source IDs, passage IDs, timestamps, or input spans for critical fields.
Design for repair and escalation
Allow the model to say what is missing and what is required to continue.
A practical response contract
Separate result, evidence, action, and review.
A current TypeScript pattern
How do you implement structured outputs with TypeScript?
The example below uses Zod to define the contract and the OpenAI Responses API parser to request a schema-conforming result. Select and evaluate the model for the workflow rather than copying a model name blindly.
import OpenAI from "openai";
import { zodTextFormat } from "openai/helpers/zod";
import { z } from "zod";
const openai = new OpenAI();
const IntakeResult = z.object({
status: z.enum(["supported", "uncertain", "blocked"]),
patientName: z.string().nullable(),
dateOfBirth: z.string().nullable(),
requestedService: z.string().nullable(),
missingFields: z.array(z.string()),
requiresHumanReview: z.boolean(),
});
const response = await openai.responses.parse({
model: "YOUR_EVALUATED_MODEL",
input: [
{
role: "system",
content: "Extract only stated information. Use null for missing values."
},
{ role: "user", content: transcript },
],
text: {
format: zodTextFormat(IntakeResult, "intake_result"),
},
});
const result = response.output_parsed;
if (!result) throw new Error("No parsed output was returned.");
The response is easier to consume, but the application still needs to validate it. Dates must be parsed. Patients must be matched through an approved identity workflow. Requested services must map to a valid catalog entry. Final actions must be authorized separately.
const validatedDate = result.dateOfBirth
? parseAndValidateDate(result.dateOfBirth)
: null;
const service = result.requestedService
? await serviceCatalog.findApproved(result.requestedService)
: null;
const canContinue =
result.status === "supported" &&
validatedDate !== null &&
service !== null &&
result.requiresHumanReview === false;
if (!canContinue) return routeToReviewQueue(result);
Current OpenAI documentation notes that Structured Outputs support much, but not all, of JSON Schema. In strict function-calling mode, object schemas require additionalProperties: false, and defined properties are treated as required, with nullable types used where needed.
Reliability begins after parsing
What validation should happen after structured generation?
Schema validation
Required keys, types, enums, arrays, and unexpected properties.
Syntax validation
Dates, codes, identifiers, URLs, and format-specific fields.
Existence validation
Does the patient, product, code, account, or source actually exist?
Relationship validation
Does the record belong to the tenant and match the current context?
Evidence validation
Do cited passages or input spans support the generated value?
Policy validation
Is the user allowed to view, propose, approve, or execute the result?
Structured arguments make tools safer, not automatically safe
How do structured outputs work with function calling?
Function calling uses schemas to define the arguments a model can propose for a tool. Strict mode improves adherence to that schema. The application still decides whether the call is authorized and safe.
Model proposes
The model selects an approved tool and generates schema-conforming arguments.
Application validates
The server checks identity, permission, resource state, and duplicate risk.
Human approves
High-impact, clinical, financial, or irreversible actions stop for review.
System executes
The application uses controlled credentials and records the result.
Expose only tools required for the current workflow.
Return and modify the minimum data necessary.
Retries must not create duplicate messages, tasks, or charges.
Re-check state immediately before executing a write.
Separate proposed action from authorized action.
Record input, arguments, decision, executor, result, and schema version.
Schemas become production APIs
How should structured output contracts be versioned?
Once services, dashboards, queues, analytics jobs, and agent steps consume a structured output, the schema becomes an API contract. A small rename can break downstream systems or silently change decisions.
Additive change
Add a nullable field or new branch while preserving existing meaning.
Breaking change
Rename or remove a field, change a type, or reinterpret an enum.
Dual-read or adapt
Support old and new versions through adapters and feature flags.
Trace every result
Store schema version with model, prompt, source index, validator, and release.
Measure more than parse success
How should structured outputs be evaluated and monitored?
Schema conformance is a baseline metric, not the success metric. A system can achieve perfect parse success while returning the wrong entity, unsupported values, excessive nulls, or unusable results.
Did it parse?
Track refusal, truncation, incomplete output, and parse failures.
Are values correct?
Measure precision and recall for entities, codes, statuses, and relationships.
Can values be verified?
Check whether source passages support critical generated fields.
Does it admit missing data?
Evaluate false fills, excessive nulls, and clarification quality.
Is the proposal executable?
Track tool selection, argument validity, conflicts, and approval rate.
Did it improve work?
Measure correction time, completion, escalation, latency, cost, and adoption.
Structured outputs belong inside the wider guardrail stack. See AI Guardrails: How to Stop LLMs from Hallucinating in Production for prevention, validation, monitoring, and escalation.
What 16+ production agents taught us
What have we learned about structured outputs in production?
Free-form output creates hidden engineering work
A response can look right while downstream code accumulates regex, markdown stripping, fallback key names, and silent coercion.
The best schema is often smaller than the first draft
Large contracts encourage vague fields and excess nulls. Narrow schemas produce cleaner generations and simpler validation.
Prescription and ICD workflows still require domain validation
A code can fit the expected string pattern while remaining nonexistent, unsupported, outdated, or wrong for the source.
“Unknown” must be designed as a valid outcome
Without an uncertainty state, the model is pressured to fill required fields even when evidence is missing.
Schema changes require API discipline
Dashboards, queues, analytics, and agents can all depend on one field. Versioning prevents a prompt change from becoming an incident.
The review interface matters as much as the JSON
Staff need to see evidence, correct values, approve actions, and feed decisions back into evaluation.
Structured outputs do not make the model trustworthy. They make the model testable, and testability is where production reliability begins.
Trilops production engineering principleA practical adoption sequence
How should a team introduce structured outputs?
Choose one high-friction output
Start where parsing, missing fields, or manual review creates measurable cost.
Define the downstream decision
Identify what software must display, validate, store, route, or approve.
Design the smallest useful schema
Use clear names, bounded enums, evidence, and explicit uncertainty.
Build deterministic validators
Check existence, relationships, policies, state, sources, and permissions.
Create representative fixtures
Include normal, missing, contradictory, unexpected, and adversarial cases.
Run behind human review
Compare generated objects with real decisions before adding autonomy.
Version and monitor
Track field accuracy, null behavior, failures, corrections, latency, and cost.
Expand one contract at a time
Reuse proven patterns without creating one universal, unmaintainable schema.
Turning an AI demo into a dependable workflow?
Define the output contract before adding more autonomy.
Trilops builds production agents with structured outputs, domain validation, controlled tools, evaluation, observability, and human review.
Frequently asked questions
LLM structured outputs: FAQ
Are structured outputs the same as JSON mode?+
No. JSON mode is intended to produce valid JSON, but it does not guarantee your exact field structure. Structured outputs constrain the result to a supported JSON Schema.
Do structured outputs prevent hallucinations?+
They prevent many formatting and invalid-enum failures, but not factual errors. A fabricated ID or unsupported code can still fit the schema.
Should every LLM response use a schema?+
No. Open-ended conversation and human-readable drafts can remain text. Use schemas when software must parse, validate, store, route, compare, or act on the response.
What should happen when information is missing?+
Represent it explicitly through nulls, missing-field arrays, uncertainty status, or a clarification request. Do not pressure the model to fabricate values.
Can structured outputs be used for tool calling?+
Yes. Function arguments can be constrained by schemas, and strict mode improves adherence. The application must still authorize every proposed action.
How large should an output schema be?+
As small as the downstream decision allows. Split broad workflows into multiple typed steps when one schema becomes difficult to explain or validate.
How should schema changes be deployed?+
Treat them as API changes. Version the contract, run consumer and regression tests, use shadow traffic or adapters, monitor the rollout, and keep rollback available.
Authoritative references
- OpenAI API — Structured model outputs
- OpenAI API — Function calling and strict mode
- OpenAI API — Working with evals
- OpenAI API — Safety best practices
- JSON Schema — What is JSON Schema?
- JSON Schema — Specification
API capabilities and supported schema features change over time. Verify implementation details against the current documentation for the exact model, endpoint, SDK, and provider used by your application.
Stop parsing model prose. Start validating production data.
Trilops develops AI agents with schema-constrained outputs, domain validation, evidence tracking, controlled tools, evaluation, monitoring, and review workflows.

Let's start a project together