Your structured-output parser broke the day you switched to a reasoning model

You did not change your prompt. You did not change your schema. You swapped the model name to a shiny new reasoning model, redeployed, and your agent started throwing JSON parse errors on responses that look, to a human, completely fine. The answer is right there in the output. Your code just cannot find it any more.

This is not your prompt's fault and it is not a bug you introduced. Reasoning models changed the shape of what a response is, and every parser written against the old assumption, one message equals one answer, breaks the moment it meets the new shape. This is going to keep happening as models evolve, so the fix is not a magic incantation, it is a parsing discipline that expects the shape to change.

The exact failure

The report that captures it: an agent with structured output fails when the response has a phased shape with commentary and final-answer channels. In your logs it shows up as a parse error on output that visibly contains the right answer:

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
# or, from a framework:
OutputParserException: Could not parse LLM output: the model returned
analysis + commentary before the JSON, parser expected a lone object

The giveaway is that the exact same code, prompt and schema worked on the previous model. Nothing in your codebase changed except the model, and the model changed the container, not the contents.

Why this happens: the output stopped being one string

The old mental model is that a completion is a single blob of text. You asked for JSON, the model returned JSON, you parsed the whole thing. Reasoning models break that premise on purpose. The Harmony format that gpt-oss uses is built on the explicit idea that the model's output is not a single continuous string, but a sequence of typed channels:

Depending on how you call the model, this reaches you in one of two forms. A good SDK gives you the channels as structured fields, so the reasoning lives in a separate reasoning attribute and the answer in the message content. A rawer integration hands you the channels serialised into one text stream with markers, and if your parser was written to json.loads(response.text), it now swallows the analysis and commentary too and dies on the first token that is not {.

This is a genuine regression-after-update problem, not a mistake on your side. The framework you use wrote its parser against last year's response shape. The vendor shipped a new shape. Until the framework special-cases it, you are the adapter.

The fix: detect the shape, select the final channel, then parse

Step 1: Stop parsing the whole blob

The one line to delete is json.loads(raw_text) applied to the entire response. Replace it with a function that first works out what shape it received. The two cases are: a structured object that already separates channels, and a flat string that concatenates them.

import json
from typing import Any

def extract_final_payload(response: Any) -> str:
    # Return the text of the final channel, whatever shape 'response' is.
    # Case 1: SDK exposes typed channels / a separate reasoning field.
    # Read the user-facing content, not the reasoning.
    content = getattr(response, "content", None)
    if content is not None:
        # Some SDKs give content as a list of typed parts.
        if isinstance(content, list):
            finals = [p for p in content
                      if getattr(p, "type", None) in ("text", "output_text")]
            if finals:
                return "".join(getattr(p, "text", "") for p in finals)
        if isinstance(content, str):
            return content

    # Case 2: a plain dict with an explicit channel field.
    if isinstance(response, dict):
        if "final" in response:
            return response["final"]
        if response.get("channel") == "final":
            return response.get("content", "")

    # Case 3: a raw concatenated string. Fall through to segment isolation.
    return _isolate_final_segment(str(response))

Step 2: Isolate the final segment when you only have raw text

If you are stuck with a concatenated string and no typed channels, do not try to regex the reasoning away token by token, because the markers differ between model families and you will be back here in a month. Instead, find the JSON object and validate it, which is robust to whatever text surrounds it. The safest general move is to locate the last balanced JSON object in the string, since the final channel comes last:

def _isolate_final_segment(text: str) -> str:
    # Find the last balanced {...} block; the final answer is emitted last.
    depth = 0
    end = None
    for i in range(len(text) - 1, -1, -1):
        c = text[i]
        if c == "}":
            if depth == 0:
                end = i
            depth += 1
        elif c == "{":
            depth -= 1
            if depth == 0 and end is not None:
                return text[i:end + 1]
    return text  # nothing JSON-shaped found; let the schema step reject it

This deliberately does not care what the reasoning or commentary looked like. It cares only about the answer's shape, which is the one thing your schema pins down.

Step 3: Validate against your schema, do not trust the parse

Parsing to a dict is not success. Validate the shape so a partial or wrong extraction fails loudly, in one place, rather than flowing bad data downstream.

from pydantic import BaseModel, ValidationError

class Result(BaseModel):
    status: str
    items: list[str]

def parse_result(response: Any) -> Result:
    payload = extract_final_payload(response)
    try:
        data = json.loads(payload)
    except json.JSONDecodeError as e:
        raise ValueError(f"final channel was not valid JSON: {payload[:200]!r}") from e
    return Result.model_validate(data)

If you can, avoid this whole class of problem at the source by asking the model for constrained JSON in the first place, which many reasoning models still support through a structured-output mode. I wrote about doing that properly in stop regex-parsing LLM output. When the API can guarantee the final channel is schema-valid JSON, your parser shrinks to selecting the channel and validating.

Verification

1. Feed your parser a captured real response from the new model, reasoning and commentary included. It must return the validated object, not raise.
2. Feed it an old-style single-string JSON response from the previous model. It must still return the same validated object. A fix that only works on the new shape and breaks the old one is a half fix.
3. Feed it a deliberately truncated response where the final JSON is cut off. It must raise your explicit ValueError, not a bare JSONDecodeError from deep in a library. Loud, located failure is the whole point.
4. Log the length of the analysis channel you discarded. If it is consistently zero, the model may not be returning reasoning where you expect it, which tells you the shape assumption is wrong before it bites in production.

What people get wrong

Blaming the prompt and adding "return only JSON, no other text". The model is not disobeying. It is returning JSON in the final channel exactly as asked, and putting its reasoning in a different channel by design. No amount of prompt scolding removes the analysis channel, because that channel is a feature of the model, not a lapse in instruction-following.

Regex-stripping the reasoning tokens. People write a regex to delete everything before the first {, and it works until a model family uses different channel markers, or the analysis itself contains a brace. You are hard-coding one vendor's current serialisation into your app. Detect the shape and extract the answer instead.

Pinning to the old model to make the error go away. Understandable under deadline, but you are declining the capability you switched for, and the format will reach you eventually as more of your providers adopt it. Fix the parser once; do not mortgage the upgrade.

Assuming your framework will handle it. It will, eventually, for the formats its maintainers have seen. There is no general adapter that unwraps a format nobody has written a special case for yet. Until then, owning the parsing boundary is not overengineering, it is the only thing standing between a vendor's format change and a production incident.

When it is still broken

The SDK hides the final channel behind a reasoning-aware type you are not reading. Print the full response object, not just its string form. Many SDKs now return a rich object where the answer is one attribute and the reasoning another; if you stringify it you get everything concatenated and reintroduce the problem you just solved.

Tool calls come back on the commentary channel and your agent ignores them. If the model is calling tools, the call may be emitted on commentary rather than as a top-level tool-call field your framework watches. That is a related shape mismatch, and it needs the same treatment: read the channel the model actually used. This intersects with agent control flow generally, which I touched on in how to evaluate an LLM feature before it touches a customer.

It works in tests but fails intermittently in production. The model sometimes emits an empty analysis channel and sometimes a long one, so a parser that got lucky on short reasoning fails on long. Test with a response that has a large analysis block, because that is the case that exposes concatenation bugs.

The lesson worth keeping is that "the response" is no longer a single thing, and it may never be a stable single thing again. Build for a response that arrives in parts, select the part you asked for, and validate it. The next new model family will change the wrapper again, and if your parsing lives in one honest function, that will be an afternoon, not an outage.