Why Tool Calls That Work on Every Other Model Return a 400 on the Newest Open-Weight Model

Your agent works. It has worked for months against a hosted model, against a couple of other open-weight models you self-host, against everything you have thrown at it. Then a shiny new open-weight reasoning model drops, you point the same agent at it because it is behind an "OpenAI-compatible" endpoint, and the first tool call comes back 400 Bad Request. No tools involved and it answers fine. Add a tool and it dies before the model even runs.

The instinct is to blame your framework, because nothing in your code changed and the 400 comes from deep inside the request builder. That instinct is wrong, and chasing it will burn a day. The request your framework sent is well-formed by the generic "OpenAI-compatible" contract. It is just not the shape this particular model's chat template and tool-call encoding will accept. "OpenAI-compatible" is a much thinner promise than the name implies, and tool calling is exactly where the thinness shows.

This is the story of the gpt-oss models and their Harmony response format, but the lesson is not about one model. It is that tool-calling wire formats are not standardised, and a new model family can reject a request that every other model accepted, for reasons that have nothing to do with your agent being wrong.

The exact error and where it comes from

Harmony: bad request on gpt-oss-120b and tool calls with create_react_agent
openai.BadRequestError: Error code: 400 - request could not be parsed

The details vary by serving stack, but the reports converge on one thing: the endpoint receives a message whose content is a list of typed blocks, one of which has type output_text (the Responses-API style), and it rejects the payload because it expects standard chat-completions content, a block of type text or a plain string. The request is syntactically valid JSON and valid under the generic OpenAI shape. It is invalid under what this model's template will parse.

Why "OpenAI-compatible" does not mean compatible

When a serving stack advertises an OpenAI-compatible API, it means the HTTP surface matches: the same URL paths, the same top-level request fields, the same response envelope. It does not, and cannot, guarantee that the model behind it interprets tool calls the same way OpenAI's hosted models do. Here is why the gap exists.

A chat model does not natively speak JSON messages. It speaks tokens. Between your structured request and the model there is a chat template that flattens your messages, system prompt, tool definitions and prior tool results into one token stream in the exact format the model was trained on. Every model family invents its own control tokens and layout for this: where the system prompt goes, how a tool definition is rendered, how the model is expected to emit a tool call, how a tool result is fed back. gpt-oss was trained on a format its authors call Harmony, with its own channels and its own encoding for reasoning, tool calls and final answers.

Your framework, meanwhile, has one generic formatter it uses for anything wearing the OpenAI-compatible badge. That formatter emits the content-block shapes OpenAI's own API accepts. When the target is a different family whose template parses a stricter or simply different set of shapes, the two disagree, and you get a 400 the instant tools enter the picture, because tool definitions and tool results are the part of the payload with the most format surface area.

How to tell a template mismatch from a genuinely broken request

Before you write any adapter, prove which failure you have, because a real malformed request needs a different fix. Three checks:

1. Does the same call work without tools? Send an identical chat request to the same model with no tools attached. If that succeeds and only the tool-enabled call 400s, the tool encoding is the suspect, not your credentials or your endpoint.
2. Where does the 400 originate? A client-side validation error (your SDK refusing to build the request) points at your code. A 400 returned by the server, in its response body, means the server parsed your request and rejected its shape. This one is server-side, which points at the template.
3. Does the same agent code work against a different model? Swap only the model, keep everything else. If the agent works elsewhere and fails here, the variable is the model family, not the agent.

When all three point the same way, you have a template mismatch, and the fix is to reshape the request for this family rather than to rewrite your agent. This is the same discipline I argue for in structured outputs and constrained decoding: do not parse or patch the model's output blindly, understand the format it actually speaks.

The fix: a per-model-family adapter

The wrong fix is to hack the one place the 400 surfaced and move on. The right fix is a thin adapter layer, keyed on the model family, that normalises the request into the shape that family's template accepts. For the gpt-oss/Harmony case the specific transform reported to work is flattening Responses-style output_text content blocks down to plain text (or a plain string) before the request goes out.

def normalize_for_family(messages, family):
    # Reshape request content blocks to what a model family's template accepts.
    if family != "gpt-oss":
        return messages  # other families take the generic shape unchanged

    fixed = []
    for m in messages:
        content = m.get("content")
        if isinstance(content, list):
            parts = []
            for block in content:
                # Harmony's chat-completions path wants type "text",
                # not the Responses-style "output_text".
                if block.get("type") == "output_text":
                    parts.append({"type": "text", "text": block.get("text", "")})
                else:
                    parts.append(block)
            # collapse a single text block to a plain string if the
            # serving stack is stricter still
            if len(parts) == 1 and parts[0].get("type") == "text":
                m = {**m, "content": parts[0]["text"]}
            else:
                m = {**m, "content": parts}
        fixed.append(m)
    return fixed

The exact transform depends on your serving stack, so treat the body of that function as the thing you tune per model. What matters is the structure: one place, selected by family, that owns the reshaping, so the rest of your agent code stays generic. When the next model arrives with its own quirk, you add a branch here instead of scattering conditionals through the agent.

Two adjacent knobs are worth knowing for the gpt-oss case specifically, because they change which template path the server uses:

• Chat Completions vs Responses API. gpt-oss was designed around the Harmony format, and different servers expose it through the chat-completions path, the responses path, or both. The output_text block is a Responses-API artefact; if your framework is set to emit responses-style content against a server expecting chat-completions, that alone triggers the mismatch. Check whether your client is configured with a responses-style output version and try the chat-completions path.
• Server-side template rendering. Many serving stacks (vLLM, Ollama, llama.cpp) render the Harmony template server-side from a plain messages array. In that mode, sending pre-rendered or Responses-style blocks fights the server's own renderer. The more generic your outgoing payload, the more likely the server's built-in template handles it.

Verifying the fix

Send the minimal tool-enabled request and confirm you get a real tool call back rather than a 400:

resp = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=normalize_for_family(messages, "gpt-oss"),
    tools=[weather_tool],
)
tc = resp.choices[0].message.tool_calls
assert tc, "model returned no tool call"
print(tc[0].function.name, tc[0].function.arguments)

Success looks like a populated tool_calls array with a valid function name and JSON arguments you can parse. Then run a multi-step case: some gpt-oss reports show simple single-tool calls passing while chains of several tool calls degrade into raw Harmony-formatted text leaking into the content. If your agent takes five or more tool hops, test that length explicitly, because the failure can hide until the conversation grows.

What people get wrong

Filing a framework bug and waiting. The 400 looks like a framework defect, so the reflex is to report it upstream and block on a fix. But the framework is emitting a valid generic request; the mismatch is with the model's template. You can adapt around it today with the layer above, and waiting for every framework to special-case every new model family is a losing bet given how fast models ship.

Assuming a shared API surface means shared behaviour. This is the root error. "OpenAI-compatible" describes the envelope, not the model. Two models behind identical endpoints can disagree completely on tool-call encoding, reasoning-channel handling and content-block shapes. Never carry an assumption of compatibility across a model swap.

Hardcoding the fix at the call site. Patching the one function where the 400 appeared works until the next model has a different quirk, and now you have model-specific hacks smeared across the codebase. Centralise the reshaping in one family-keyed adapter so the special cases live in exactly one place you can read and test.

Skipping tool-call testing on model upgrades. Teams test that a new model answers a prompt, see a fluent reply, and ship. Tool calling is a separate code path with far more format surface, and it is the part most likely to break silently on a new family. A model that chats perfectly can still 400 on every tool call.

When it is still broken

• Raw Harmony markup leaks into the answer. If the model returns control tokens or channel markers as visible text instead of a clean tool call, the server is not parsing the Harmony format on the way back out. Check that the serving stack has Harmony/tool-call parsing enabled for this model, and that you are on a server version that supports it; this is a server capability, not something you can fix from the client.
• It fails only on long tool chains. Reproduce with a fixed multi-tool scenario and check whether the server truncates or mis-renders the accumulated tool-result history. Trimming or summarising intermediate tool results can keep the request inside what the template handles reliably.
• Different serving stacks disagree. The same gpt-oss weights behind vLLM, Ollama and a hosted provider can each have slightly different template handling and different bugs. If one 400s, try another stack before concluding the model itself is broken, and pin the stack version once you find one that works.
• You need the model's exact format contract. When you are guessing at block shapes, stop guessing and read the model's own template. The chat template shipped with the weights is the authoritative source for what the model was trained to parse, and the function-calling documentation is the reference for the request shape you are adapting from.

The durable takeaway: treat every new model as a new integration, not a drop-in. A shared HTTP surface hides real differences in how the model was trained to receive tools, and tool calling is where those differences bite first. Build the adapter seam once, test tools on every model, and a 400 on the newest release becomes a five-minute branch instead of a lost day.