Why RetryError hides the real reason your LLM call failed

Your LLM feature falls over in production. You open the traceback expecting to see whether it was a rate limit, an expired key, or a slow provider, and instead you get this:

RetryError[<Future at 0x7f2a state=finished raised RateLimitError>]

Somewhere buried in that string is the word that matters, but your logging, your alerting and your except blocks all see the wrapper class RetryError, not the RateLimitError inside it. So your rate-limit alert never fires, your backoff logic never triggers, and you find out the provider was throttling you only by squinting at a log line. Then, on a different call, the opposite happens: a raw httpcore.ReadTimeout comes flying up completely unwrapped, from a library you did not know you were using, that none of your handlers were written to catch.

Both problems are the same problem wearing two coats. The framework's error surface is inconsistent: over-wrapped in one place, under-wrapped in another. You cannot build reliable retry, alerting or fallback logic on top of an error type you cannot predict. The fix is not to fight the framework's exceptions one by one; it is to put a single translation boundary in front of it.

The two failure modes, side by side

Over-wrapped. A resilience wrapper retries the call, and when the retries run out it raises its own exception type:

tenacity.RetryError: RetryError[<Future at 0x7f2a state=finished raised RateLimitError>]

Your handler does except RateLimitError. It never matches, because what propagated is RetryError. The real cause is a passenger inside it.

Under-wrapped. A different code path, often streaming or connection setup, raises the low-level transport exception with nothing translating it:

httpcore.ReadTimeout
  during handling of the above exception, another exception occurred:
httpx.ReadTimeout: The read operation timed out

Now your handler does except openai.APITimeoutError (or the equivalent for your SDK) and still misses it, because the exception came from httpcore two layers below the SDK. Same feature, same day, two completely different exception types for what a human would call "the model call failed".

Why frameworks do this

Retry libraries wrap the call so they can attempt it several times, and on the final failure they have a design choice: re-raise the original exception, or raise a wrapper that carries the retry history. Tenacity defaults to the wrapper, RetryError, because it is technically more informative: it tells you retries happened and holds the last attempt. That is the right default for a general-purpose library and the wrong surface for a caller who just wants to know "rate limit or not".

The under-wrapping is the mirror image. A framework translates provider errors into tidy SDK exceptions only along the paths its authors thought about. Streaming reads, lazy connection setup, and some helper utilities can call the HTTP stack directly, and when the socket underneath times out, the httpcore exception rises with nothing to catch and rename it. Neither behaviour is a bug exactly; together they mean you cannot assume any single exception type.

The fix, in numbered steps

Step 1: Stop losing the cause inside RetryError

If you own the tenacity decorator, the cleanest fix is one keyword. reraise=True tells tenacity to raise the underlying exception on exhaustion instead of wrapping it:

from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type

@retry(
    reraise=True,                       # raise the real exception, not RetryError
    stop=stop_after_attempt(5),
    wait=wait_exponential(min=1, max=30),
    retry=retry_if_exception_type(RateLimitError),
)
def embed(text: str):
    return client.embeddings.create(model="text-embedding-3-small", input=text)

If you do not own the decorator (it is inside a framework), unwrap at the call site instead. RetryError.last_attempt is a future holding the final failure:

from tenacity import RetryError

try:
    result = query_engine.query(prompt)
except RetryError as retry_err:
    # re-raise the real cause, with its own traceback
    retry_err.reraise()

retry_err.reraise() raises the wrapped exception directly. If you want to inspect it first, retry_err.last_attempt.exception() returns the exception object without raising.

Step 2: Define a small typed hierarchy you control

Three or four classes is enough. These are your types, decoupled from any SDK version:

# These are your types, decoupled from any SDK version.
class LLMError(Exception): ...          # base for anything the boundary raises
class RateLimited(LLMError): ...
class AuthFailed(LLMError): ...
class Timeout(LLMError): ...
class ProviderError(LLMError): ...      # 5xx or unclassified failure

Step 3: Translate at one boundary, covering both SDK and transport

Wrap the provider call once. Catch the SDK's exceptions and the raw transport ones, and always chain with raise ... from exc so the original traceback survives:

import httpx, httpcore
import openai  # or your provider's SDK

def call_model(fn, *args, **kwargs):
    try:
        return fn(*args, **kwargs)
    except openai.RateLimitError as e:
        raise RateLimited(str(e)) from e
    except openai.AuthenticationError as e:
        raise AuthFailed(str(e)) from e
    except (openai.APITimeoutError, httpx.TimeoutException, httpcore.TimeoutException) as e:
        raise Timeout(str(e)) from e
    except (httpx.TransportError, httpcore.ConnectError) as e:
        # the under-wrapped leak: translate it instead of letting it escape
        raise ProviderError(f"transport failure: {e}") from e
    except openai.APIStatusError as e:
        raise ProviderError(f"provider {e.status_code}") from e

Now every caller does the same thing regardless of what actually broke:

try:
    answer = call_model(query_engine.query, prompt)
except RateLimited:
    metrics.incr("llm.rate_limited"); return fallback()
except Timeout:
    metrics.incr("llm.timeout"); return fallback()
except AuthFailed:
    alert_oncall("LLM auth broken"); raise

Verification

Prove each real cause reaches your typed handler. Force a rate limit against the boundary and assert the type:

import pytest

def test_rate_limit_surfaces_as_rate_limited(monkeypatch):
    def boom(*a, **k):
        raise openai.RateLimitError("429", response=..., body=None)
    with pytest.raises(RateLimited):
        call_model(boom)

And confirm a raw transport error no longer escapes untranslated:

def test_httpcore_timeout_becomes_timeout():
    def boom(*a, **k):
        raise httpcore.ReadTimeout("read timed out")
    with pytest.raises(Timeout):
        call_model(boom)

If both pass, your alerting and fallback logic are now built on causes you control, not on whatever the framework happened to raise this week. This is the same instinct behind retrying a deadlock correctly: the decision depends entirely on knowing the real error class, so make sure the real error class is what you catch.

What people get wrong

The reflex fix is except Exception at the call site, log it, and move on. That makes the crash stop, but it erases the distinction between a rate limit you should back off from and an auth failure you should page someone about. A rate limit and a bad key demand opposite responses; collapsing them into one handler guarantees you do the wrong one half the time.

The second mistake is catching the provider SDK's exceptions all over your codebase. Every time the SDK renames a class or a new transport error slips through, you are editing twenty call sites. One boundary means one place to change. It also insulates you from a provider swap; the day you add a second model vendor, only the translation function learns its exception classes.

The third is trusting that the framework wraps consistently. It does not, and it is not obliged to. Assume any given call can raise the tidy SDK error, a tenacity RetryError, or a bare httpcore exception, and translate all three. The same discipline pays off when a provider changes its output format under you: the boundary is where you absorb the change.

When it is still broken

• The cause is still hidden after reraise. Some frameworks nest retries two deep. Unwrap in a loop: while the exception is a RetryError, replace it with err.last_attempt.exception() until you reach a non-wrapper type.
• Async calls swallow the type. With asyncio.gather, a raised exception can be masked by others in the group. Use return_exceptions=True and inspect each result, translating individually.
• A new transport exception appears. httpx and httpcore split their exception classes; if one slips through, add it to the boundary rather than to a caller. Check the SDK's own exceptions module for the current list.
• Retries mask a persistent outage. If before_sleep logs show every call retrying to exhaustion, the problem is upstream, not transient. Cap total retry time so a dead provider fails fast instead of holding requests open.

Consistent, typed error surfaces are not a nicety. They are the precondition for every reliability feature you want on top: backoff, circuit breaking, fallback models, alerting. Build the boundary once, and the rest of the system finally gets to reason about failures in terms it understands.