Your M-Pesa Integration Needs Nightly Reconciliation, Not Just Callbacks

The M-Pesa integration works. You tested it fifty times, the STK push pops on the phone, the callback lands, the order flips to paid. You ship it. For three weeks it is flawless. Then your provider has a rough Tuesday night, a callback or two never arrives, and on Wednesday morning three customers are emailing to say Safaricom took their money but their order still says "pending payment".

Here is the sentence that should be printed above every payments engineer's desk: a missing callback is not a failed payment. It is an absence of information. The money may have moved perfectly. You just were not told. And a system that treats "no callback" as "no payment" will happily strand real, completed, paid-for transactions in limbo, forever, with no way out.

Callbacks are a push. Pushes get lost. The cure is not a better callback handler, it is a second, independent source of truth you pull on a schedule: the Daraja Transaction Status API. This is the safety net you build on day one, because retrofitting it after your first incident means reconciling a spreadsheet of angry customers by hand.

The failure nobody sees in testing

There is no error message here, and that is precisely why this bug ships. In a demo, nothing fails. The callback always arrives because you are watching one transaction on a good network. Production is thousands of transactions across provider maintenance windows, dropped TCP connections, a deploy that restarts your web process mid-callback, a firewall hiccup, or Safaricom simply retrying to a URL that timed out once.

Every one of those produces the same outcome: the customer completed the payment on their handset, Safaricom debited them, and your CheckoutRequestID is sitting in your database with status = 'pending' and no callback to resolve it. Nothing crashed. Nothing logged an error. The money is gone from the customer and your system does not know it arrived.

Why callbacks alone can never be enough

A callback is a network request from Safaricom's infrastructure to yours. It shares every weakness of any single network request: it can time out, hit a load balancer mid-deploy, arrive at a node that just OOM-killed your worker, or be dropped by an intermediate proxy. Safaricom does retry, but retries are finite and also to the same fragile endpoint. Layer on that callbacks can also fire twice, which is a separate problem I cover in why your M-Pesa callback will fire twice, and you have a channel that is neither guaranteed-once nor guaranteed-at-all.

Reconciliation inverts the dependency. Instead of waiting to be told, you ask. The Transaction Status query is initiated by you, on your schedule, from your infrastructure, and you can retry it until you get an answer. It does not care whether the original callback arrived. That independence is what makes it a safety net rather than just a second copy of the same fragile mechanism.

Building it, in numbered steps

Step 1: Give every payment an explicit, non-terminal state

You cannot reconcile what you cannot find. Every payment attempt must be written to your database before you fire the STK push, keyed by CheckoutRequestID, in an explicit PENDING state with a created_at timestamp. States must be a small closed set:

-- Terminal states are the only ones a callback OR reconciliation may set
-- PENDING -> PAID | FAILED | EXPIRED
CREATE TABLE mpesa_payments (
    id                BIGSERIAL PRIMARY KEY,
    checkout_request_id TEXT UNIQUE NOT NULL,
    amount            NUMERIC(12,2) NOT NULL,
    status            TEXT NOT NULL DEFAULT 'PENDING',
    result_code       INT,
    mpesa_receipt     TEXT,
    created_at        TIMESTAMPTZ NOT NULL DEFAULT now(),
    resolved_at       TIMESTAMPTZ
);

If a row is never written before the push, a lost callback leaves you with no record at all, which is unrecoverable. Write first, then push.

Step 2: Schedule a job that finds stale pending payments

Run every few minutes. Select payments still PENDING and older than your threshold. Do not reconcile a payment that is thirty seconds old, the customer may still be typing their PIN. Five to ten minutes is a sane floor.

def find_unresolved(threshold_minutes: int = 8):
    return db.query(
        '''
        SELECT checkout_request_id, amount
        FROM mpesa_payments
        WHERE status = 'PENDING'
          AND created_at < now() - (%s * interval '1 minute')
        ORDER BY created_at
        LIMIT 200
        ''',
        [threshold_minutes],
    )

The LIMIT matters. It caps how many Transaction Status calls one run can make, which is your first defence against rate limiting.

Step 3: Query Transaction Status, spaced out

The Transaction Status API is itself asynchronous: you POST a query and Daraja replies to a ResultURL you supply, exactly like the STK callback. So reconciliation resolution can arrive by callback too, which is fine, because now you have two independent chances to learn the outcome. The key operational detail is pacing. Fire two hundred queries in a tight loop and you will trip Daraja's spike-arrest and start getting 429s, which I dissect in Daraja returns 429 with a body your parser cannot read.

import time

for row in find_unresolved():
    query_transaction_status(
        transaction_id=row["checkout_request_id"],
        result_url="https://your-api/mpesa/recon-result",
    )
    time.sleep(0.5)   # spread the batch; tune to stay under the rate limit

A half-second gap turns a 200-request burst into a gentle trickle over a couple of minutes. That is invisible to your customers and invisible to Daraja's rate limiter.

Step 4: Apply the authoritative result idempotently

Whether the answer arrives via the Transaction Status result callback or you poll a result, only move a row out of PENDING if it is still PENDING. This is the same idempotency discipline the STK callback needs, so share the code path:

UPDATE mpesa_payments
   SET status = %(terminal_status)s,
       result_code = %(result_code)s,
       mpesa_receipt = %(receipt)s,
       resolved_at = now()
 WHERE checkout_request_id = %(id)s
   AND status = 'PENDING';   -- no-op if a callback already resolved it

The AND status = 'PENDING' guard is what makes a callback and a reconciliation racing to resolve the same payment safe: whichever lands first wins, the second is a harmless no-op.

Verification: prove the net catches a dropped callback

You cannot ask Safaricom to drop a callback on demand, so simulate it. In staging, complete a real sandbox payment, then deliberately discard the callback (point the STK callback URL at a dead endpoint, or drop it in your handler). The payment row stays PENDING. Now confirm:

1. After your threshold passes, the reconciliation job selects the row.
2. It issues a Transaction Status query.
3. The result callback (or poll) resolves the row to PAID with the correct receipt.
4. resolved_at is populated and the customer's order flips to paid, with no human involved.

Log the count of payments resolved by reconciliation rather than by their original callback. In healthy production that number is small but almost never zero, and watching it is how you know the net is load-bearing.

What people get wrong

• "Callbacks are reliable enough." They are reliable in a demo and probabilistic in production. Any channel that is 99.9% reliable still strands one payment in a thousand, and at scale that is a support queue.
• Treating a callback timeout as a failed payment. This is the dangerous one. Marking the order failed while the customer has been debited is worse than leaving it pending, because now you have taken money and told them it did not work.
• Reconciling in a tight loop. A batch run without pacing is the classic way to earn a 429 storm and make your recovery mechanism the thing that goes down.
• Bolting reconciliation on after an incident. Retrofitting it means back-filling states for historical transactions by hand. Design it alongside idempotency from day one, the same way you would design the double-entry ledger in M-Pesa reconciliation that scales.
• Letting unresolved transactions sit silently. If Transaction Status also cannot resolve a payment after several attempts, that needs a human. Alert on it, do not let it decay into a pending row nobody ever reads.

When it is still broken

1. Transaction Status returns nothing useful for some IDs. Confirm you are querying with the correct identifier and that the transaction is old enough to be visible. Some undocumented result codes are not failures, see ResultCode 4999 is not a failure.
2. The recon job itself is getting 429s. Lower the batch LIMIT, increase the sleep, and spread runs further apart. Reconciliation should be slow and boring.
3. Payments resolve but orders still do not update. Your status transition and your order fulfilment are decoupled. Make the terminal transition emit the same event your callback handler does, so there is one fulfilment path.
4. You are missing rows entirely. You are pushing before writing. Reverse it, per Step 1.

Reconciliation is not glamorous and it will not show up in a demo. It is the difference between an integration that looks finished and one that is actually finished. Build it before you need it, because the day you need it, you are already reading customer complaints. For the full production picture, pair this with the Daraja production guide for STK, C2B and B2C callbacks.