The JSON Schema That Works on GPT and Silently Breaks on Every Other Model

You build a feature on GPT with structured output. You hand it a JSON schema, it returns clean, valid JSON every time, and you ship. Months later, someone asks for a cheaper or faster model, so you point the same code at Gemini. Same schema, same prompt. And now the output is malformed, or fields are missing, or the API rejects the request outright with something like:

[Google Generative AI] Structured Output doesn't work with advanced schema

You did not change the schema. You changed the provider, and discovered the hard way that "structured output" is not one feature. It is a different, partially-overlapping feature on every provider, and there is no portable schema that is guaranteed to work everywhere. The surface looks identical (you pass a JSON schema, you get JSON back), which is exactly why this bites so late and so hard.

This article maps what actually differs, shows a schema that passes on OpenAI and degrades elsewhere, and gives you a conservative portable subset plus a CI check so a schema change cannot silently break one of your providers again.

Why "structured output" is not one thing

Every major provider now offers a way to constrain output to a JSON schema, and on the surface the APIs look interchangeable. Underneath, each one implements a different subset of the JSON Schema specification, enforces different rules, and fails in a different way when it meets something it does not support. The dangerous part is not the features that error loudly. It is the features one provider silently ignores. A pattern or minLength constraint that one engine enforces and another drops means your validation guarantee quietly evaporates when you switch, and nothing tells you.

Schema feature	OpenAI strict mode	Gemini (responseSchema)	Portable choice
additionalProperties: false	Required on every object	Not required, handled differently	Set it explicitly everywhere
All fields in required	Required; optional not allowed	Optional fields tolerated	Make everything required; use nullable unions
Deep nesting	Capped (around 5 levels)	Deep/large schemas may be rejected	Keep it shallow, 2-3 levels
$ref / $defs	Supported with restrictions	Historically unsupported; newer paths only	Inline everything, no refs
pattern, format, minLength	Accepted but not enforced	May be ignored	Do not rely on them; validate yourself
Unrecognised keywords	May reject	Silently ignored	Include nothing non-essential

Read the vendor docs for the exact current rules before you trust any single row here, because these move: the OpenAI structured outputs guide and the Gemini structured output docs are the sources of truth. The point of the table is not the specific cells, it is that the cells differ at all.

A schema that passes on GPT and breaks elsewhere

Here is a realistic schema for extracting an invoice. It leans on features OpenAI is comfortable with and that other engines handle differently: nested objects several levels deep, a $ref reused for line items, a regex pattern on the invoice number, and an optional field that is not in required.

{
  "type": "object",
  "properties": {
    "invoice_number": {"type": "string", "pattern": "^INV-[0-9]{6}$"},
    "vendor": {
      "type": "object",
      "properties": {
        "name": {"type": "string"},
        "address": {
          "type": "object",
          "properties": {
            "country": {"type": "object",
              "properties": {"code": {"type": "string"}}}
          }
        }
      }
    },
    "line_items": {"type": "array", "items": {"$ref": "#/$defs/item"}},
    "notes": {"type": "string"}
  },
  "required": ["invoice_number", "vendor", "line_items"],
  "$defs": {
    "item": {"type": "object",
      "properties": {"sku": {"type": "string"}, "qty": {"type": "integer"}}}
  }
}

On OpenAI strict mode this needs small tweaks (every object needs additionalProperties: false and every property in required) but the shape is fine. Point it at a provider with a smaller subset and several things go wrong at once: the $ref may not resolve, so line_items comes back untyped or empty; the four-level country.code nesting may be rejected or truncated; the pattern on invoice_number is ignored, so you get INV-12 and only find out downstream; and notes being optional behaves inconsistently. None of these throws a clear "your schema is wrong" error. You get degraded output that looks plausible.

The portable subset that works on every provider

The reliable move is to write to the lowest common denominator on purpose. A schema built with these constraints behaves the same everywhere:

• Shallow nesting. Two to three levels. Flatten vendor.address.country.code to vendor_country_code. Deep trees are where providers diverge most.
• No $ref or $defs. Inline every definition even if it repeats. Reference support is the least consistent feature across engines.
• Every property required. Instead of an optional field, make it required and allow null: {"type": ["string", "null"]}. This satisfies OpenAI's strict rule and removes the "optional field" ambiguity everywhere else.
• Explicit enums instead of regex. Where a field has a fixed set of values, use "enum": ["paid", "unpaid"], which every provider respects, rather than a pattern, which many ignore.
• No pattern, format, minLength, minimum as guarantees. You may include them as hints, but never trust the model to enforce them. Enforce them yourself after the fact.
• additionalProperties: false on every object. Harmless where not required, mandatory where it is.

Rewritten to this subset, the invoice schema is flatter, fully required, uses an enum for status, and validates line items with an inlined object. It is less elegant and far more robust. If you have read structured outputs, JSON mode and constrained decoding, this is the same lesson at the schema layer: constrain aggressively and verify what comes back.

The safety net: validate output regardless of provider

No provider's structured output is a substitute for validating the result yourself, because the constraints you cannot rely on (formats, patterns, ranges) are precisely the ones a provider may ignore. Run the returned JSON through a real JSON Schema validator and, ideally, a repair pass:

from jsonschema import Draft202012Validator

def parse_or_repair(raw: str, schema: dict, ask_model_to_fix) -> dict:
    data = json.loads(raw)                       # may raise; catch upstream
    errors = list(Draft202012Validator(schema).iter_errors(data))
    if not errors:
        return data
    # one bounded repair attempt: hand the errors back to the model
    messages = "; ".join(e.message for e in errors[:5])
    return json.loads(ask_model_to_fix(raw, messages))

The key discipline: your validator holds the full schema including the patterns and formats you did not trust the provider to enforce. The provider gets the portable subset; your validator gets the strict truth. One bounded repair attempt is worth it; an unbounded retry loop is a cost and latency trap.

Verification: a CI check across every provider

The whole failure mode is "worked on the provider I tested, broke on the one I did not". So test on all of them, in CI, on every schema change. The check sends each schema to each configured provider with a fixed prompt and asserts the output validates:

import pytest
from jsonschema import Draft202012Validator

PROVIDERS = ["openai", "gemini", "anthropic"]  # whatever you actually ship on

@pytest.mark.parametrize("provider", PROVIDERS)
def test_schema_portable(provider, sample_input):
    raw = call_provider(provider, INVOICE_SCHEMA, sample_input)
    data = json.loads(raw)  # fails CI if not even valid JSON
    errs = list(Draft202012Validator(INVOICE_SCHEMA).iter_errors(data))
    assert not errs, f"{provider} produced schema-invalid output: {errs[:3]}"

Run it against a small set of representative inputs. When it goes green on every provider, the schema is portable; when a provider goes red, you found the incompatibility on your machine instead of in production. Expected output is a passing row per provider:

test_schema_portable[openai]    PASSED
test_schema_portable[gemini]    PASSED
test_schema_portable[anthropic] PASSED

What people get wrong

Assuming the API surface being identical means the behaviour is. Passing a JSON schema and getting JSON back looks the same on every provider. It is not the same. The overlap is partial, and the gaps are silent.

Trusting the schema and skipping validation. "The model has structured output, so the JSON is guaranteed valid" is only true for the constraints that provider enforces. Formats, patterns, and ranges are commonly ignored. Validate every response against your real schema, always.

Using deeply nested schemas because they model the domain nicely. Elegant nested schemas are the single biggest portability risk. A flat schema with prefixed field names is uglier and survives a provider switch. Flatten at the boundary, reshape into your nice nested object in application code after validation.

Relying on a framework to paper over it. Orchestration libraries make the call sites look uniform, but they cannot invent schema features a provider lacks. They forward your schema; the provider's subset still governs. If your structured output parser breaks on a reasoning model, the same root cause applies: the abstraction hid a real difference in the underlying engine.

When it is still broken

• Valid JSON, wrong values. The provider ignored a constraint it does not enforce. Move that constraint into your own validator and add a repair pass; do not expect the model to honour a pattern.
• Intermittent failures on one provider. You are near a nesting or size limit that is checked probabilistically. Flatten the schema and shorten property names to get well under the limit.
• The request is rejected before generation. A keyword in your schema is unsupported by that provider's parser. Strip $ref, $defs, and any non-core keyword and inline everything.
• Empty or truncated arrays. Often a $ref that did not resolve on that engine. Inline the array item definition directly under items.