Testing Payment Integrations When the Sandbox Itself Is Flaky

Every payment integration has two halves, and teams test the wrong one. The half everyone tests is the outbound call: you push an STK prompt, you create a Stripe charge, you get back a reference. That part is a normal HTTP request and it mostly works. The half that actually breaks in production is the inbound callback, the webhook the provider sends you seconds or minutes later to tell you whether money moved. That is where the duplicates, the out-of-order deliveries, the missing signatures and the silent double-credits live.

And it is precisely the half you cannot test reliably, because the sandbox that is supposed to fire those callbacks is itself unreliable. The Daraja C2B sandbox fires its confirmation callback maybe half the time, in my experience, and I am not alone: I wrote a whole piece on why C2B sandbox callbacks only fire half the time. Stripe's test mode is better but your local tunnel still drops events. So teams do one of two things, and both are wrong. They give up on automated testing of the callback path entirely, or they wire CI to something production-adjacent "just so the tests pass reliably".

Here is the reframe that fixes it: your test does not need the sandbox to deliver a callback, it needs a callback to deliver. Those are different problems. You control the second one completely.

Why the callback is the part that breaks

The outbound request is synchronous and you see the result immediately. If it is wrong, you know in the same function call. The callback is asynchronous, arrives over a path you do not control, and carries three properties that a single happy-path sandbox test will never show you:

• It arrives more than once. Providers retry when they do not get a fast 200, so your handler must be idempotent. This is not theoretical for M-Pesa; I covered the exact duplicate-fire behaviour in your M-Pesa callback will fire twice.
• It can arrive out of order, or the callback can land before your own database has committed the record of the request that triggered it.
• It is unauthenticated until you verify it. Anyone who can reach the URL can POST a fake success, which is the whole point of verifying the webhook signature against the raw body.

None of those three happen in a clean sandbox run. So chasing sandbox reliability, even if you achieved it, would test the one scenario that is already fine and none of the three that are not.

Step 1: capture one real payload, then never depend on the network again

Do one real sandbox transaction, or in the M-Pesa case one small real deployed transaction, and log the full incoming request: method, headers (you need them for signature verification) and the raw body exactly as received. Save it as a fixture.

# conftest.py or a fixtures module
from pathlib import Path
import json

FIXTURES = Path(__file__).parent / "fixtures" / "callbacks"

def load_callback(name: str) -> dict:
    raw = (FIXTURES / f"{name}.json").read_bytes()
    meta = json.loads(raw)
    return {
        "headers": meta["headers"],
        "body": meta["body"].encode(),  # keep the raw bytes for signature checks
    }

Store the body as the exact string the provider sent, not a re-serialised dict. Signature verification runs over the raw bytes, and json.dumps(json.loads(x)) will reorder keys and change whitespace, which breaks the signature. This is the single most common reason a "correctly verified" webhook fails in test but works in production, or vice versa.

Step 2: test the handler in isolation against the fixture

Now the handler test has no provider in it at all. It POSTs the fixture at your endpoint and asserts on the outcome. Here it is against a FastAPI app with the test client.

def test_valid_callback_credits_once(client, db):
    cb = load_callback("mpesa_c2b_success")

    resp = client.post("/webhooks/mpesa", data=cb["body"], headers=cb["headers"])

    assert resp.status_code == 200
    txns = db.query(Transaction).filter_by(provider_ref="ws_CO_123456").all()
    assert len(txns) == 1
    assert txns[0].status == "completed"

That test runs in milliseconds, never flakes, and exercises the real signature verification and the real state transition. If your handler pulls the amount from the payload and credits an account, this proves the parsing and the crediting. If someone changes the payload shape upstream, you re-capture the fixture and the diff is right there in the pull request.

Step 3: test the scenarios the sandbox never shows you

This is where fixtures earn their keep, because you can now construct nasty conditions on demand.

Duplicate delivery (the one that costs real money)

def test_duplicate_callback_does_not_double_credit(client, db):
    cb = load_callback("mpesa_c2b_success")

    first = client.post("/webhooks/mpesa", data=cb["body"], headers=cb["headers"])
    second = client.post("/webhooks/mpesa", data=cb["body"], headers=cb["headers"])

    assert first.status_code == 200
    assert second.status_code == 200            # you still ack the retry
    txns = db.query(Transaction).filter_by(provider_ref="ws_CO_123456").all()
    assert len(txns) == 1                        # but you only acted once
    assert account_balance(db, "254712345678") == 1000

Your idempotency key must be the provider's own transaction identifier (M-Pesa's TransID or CheckoutRequestID, Stripe's event id), stored with a unique constraint so the second insert is rejected at the database. Do not key on a timestamp or a request counter; the retry is a different HTTP request with the same money behind it.

Forged or tampered payload

def test_tampered_body_is_rejected(client):
    cb = load_callback("stripe_payment_succeeded")
    tampered = cb["body"].replace(b'"amount":1000', b'"amount":100000')

    resp = client.post("/webhooks/stripe", data=tampered, headers=cb["headers"])
    assert resp.status_code in (400, 401)   # signature no longer matches the body

Out-of-order or early callback

Fire the success callback before the record of the originating request exists, and assert your handler either queues it or creates the record defensively rather than throwing a 500 that makes the provider retry forever.

Step 4: keep a separate, tolerant smoke test for the outbound call

You still want to know the provider has not changed the request shape under you. That is a contract concern, and it goes in its own suite that is allowed to be slow and flaky, is not required for merge, and hits only the sandbox, never production.

import pytest

@pytest.mark.smoke   # excluded from the default CI run
def test_stk_push_is_still_accepted(sandbox_credentials):
    resp = initiate_stk_push(amount=1, phone="254708374149")
    assert resp["ResponseCode"] == "0"   # accepted, we do not wait for the callback

Run these on a schedule or manually before a release, not on every push. Their job is to catch "the provider changed their API", which is real but rare, and to do it without ever gating your day-to-day work on a third party's uptime. If you have wrapped your providers behind a provider-agnostic payments port, this smoke layer is also the natural place to verify each concrete adapter.

What people get wrong

Pointing CI at live production credentials "so tests are reliable". This is the worst option on the page. It can move real money, it puts production secrets into every CI job and its logs, and it makes your build depend on a payment provider being up. Reliable and dangerous are not the same thing. Fixtures give you reliable without the danger.

Deleting the callback tests because they are flaky. The flakiness is a property of how you were testing (through the network), not of the thing being tested. Move to fixtures and the flakiness disappears while the coverage increases.

Re-serialising the payload in the fixture. Save the raw bytes. The moment you round-trip through a dict, signature verification tests become meaningless because you are no longer testing the bytes the provider actually signs.

Only ever testing the success payload. Capture the failure, the cancellation, the timeout and the "insufficient funds" callbacks too. For M-Pesa specifically, capture the undocumented result codes; I have seen ResultCode 4999 mistaken for a failure when it is not. Each is a fixture, each is a test.

When it is still broken

• Signature verifies in production but not against your fixture. Your fixture body was re-serialised, or you saved a decoded/pretty-printed version. Re-capture the raw bytes off the wire.
• Idempotency test passes locally but doubles in production. Your unique constraint is on the wrong column, or two callbacks race and both pass the "does it exist" check before either inserts. Enforce uniqueness at the database, not with a read-then-write in application code.
• The M-Pesa confirmation genuinely never arrives, even for real transactions. That is a registration or URL problem, not a test problem. Validate C2B against a small real deployed transaction and confirm your confirmation and validation URLs are registered and publicly reachable, as covered in the C2B sandbox article linked above.
• Fixtures drift from reality. Add one scheduled smoke run per provider that captures a fresh payload and diffs it against the committed fixture, so a silent shape change surfaces as a failing job rather than a production incident.

The principle underneath all of this: a payment provider's sandbox is a dependency, and you do not put an unreliable third-party dependency on the critical path of your test suite. You record what it produced when it was cooperative, and you test your own code against that recording for the rest of time.