Exactly-once over an unreliable pipe: HMAC, nonces, and a byte-exact contract

The SMS gateway that this article is about forwards M-Pesa payment confirmations from a phone on a shop counter to a backend that turns them into accounting records. That means two properties matter more than anything else in the system.

A payment must not be recorded twice, because that is a reconciliation nightmare that gets discovered weeks later. And a payment must not be forgeable, because a record that anybody can fabricate is worse than no record at all: people will trust it, and act on it, and hand over goods.

Getting both, over a pipe that drops responses and retries blindly, needs three separate mechanisms that are frequently confused with each other. This article is about telling them apart.

Three mechanisms, three questions

It helps to name what each thing is actually for before writing any code.

Mechanism	Question it answers	Changes on retry?
HMAC signature	Did the holder of this device's secret produce this exact payload?	Yes, because the nonce and sent_at inside it change
Nonce	Have I seen this request before?	Yes, fresh UUID every attempt
message_hash	Have I seen this message before?	No, stable for the life of the message

Most systems that get this wrong get it wrong by having only one of the last two. With only a nonce, every retry looks like a new message and you get duplicate payments. With only a content hash, an attacker can capture one valid request off the wire and replay it forever, and although it will be deduplicated on arrival, you have handed them a way to keep a signed request alive indefinitely and you have no record that anything unusual happened.

The message hash, and the timestamp that makes it work

The content hash is three fields, pipe-joined, SHA-256, hex:

/// message_hash = SHA256( sender | message | received_at )
///
/// received_at here is the PDU/SMSC timestamp string (EAT ISO-8601). Using
/// the network timestamp (not arrival time) means a duplicate delivery or a
/// double receiver-fire collapses, while two genuinely distinct messages do
/// not.
static String messageHash({
  required String sender,
  required String message,
  required String receivedAt,
}) {
  final canonical = '$sender|$message|$receivedAt';
  return sha256.convert(utf8.encode(canonical)).toString();
}

The comment carries the whole argument, so it is worth expanding.

The obvious implementation hashes sender and body only. That collapses two genuinely distinct payments of the same amount from the same person on the same day, which absolutely happens at a busy till, and you have just deleted a real payment. So a timestamp has to be in there.

The tempting timestamp is the moment the app saw the message. That is wrong in the other direction. If the broadcast receiver fires twice, or the inbox backfill picks up something the live path also captured, arrival time differs by milliseconds and you get two different hashes for one payment. Your deduplication runs, finds nothing, and inserts a duplicate. Worse, it does this silently and only under the exact conditions you cannot reproduce on a desk.

The PDU timestamp is the moment the network stamped the message. It is a property of the message, not of your observation of it. It is identical across every redelivery and every rescan, and it differs between two distinct messages even when their content matches. That single choice is what makes the rest of the deduplication work, and it is the kind of decision that looks like a detail in a code review and is actually load-bearing.

The string to sign

The signature covers eight fields, newline-joined, in a frozen order, with no trailing newline, and with the signature field itself excluded for obvious reasons:

static String incomingStringToSign({
  required String deviceId,
  required String sender,
  required String message,
  required String receivedAt,
  required int simSlot,
  required String messageHash,
  required String nonce,
  required String sentAt,
}) {
  return [
    deviceId,
    sender,
    message,
    receivedAt,
    '$simSlot',
    messageHash,
    nonce,
    sentAt,
  ].join('\n');
}

And the Python side, which must produce identical bytes:

def incoming_string_to_sign(
    *,
    device_id: str,
    sender: str,
    message: str,
    received_at: str,
    sim_slot: int,
    message_hash: str,
    nonce: str,
    sent_at: str,
) -> str:
    """device_id\nsender\nmessage\nreceived_at\nsim_slot\nmessage_hash\nnonce\nsent_at"""
    return "\n".join(
        [
            device_id,
            sender,
            message,
            received_at,
            str(sim_slot),
            message_hash,
            nonce,
            sent_at,
        ]
    )

Read those two side by side and notice how deliberately boring they are. There is no JSON serialisation, no key sorting, no URL encoding, no canonical form of a canonical form. Every one of those is a place where two languages disagree about whitespace, key ordering, unicode normalisation or how to render an integer, and every disagreement shows up as a signature mismatch that reproduces on exactly one device.

Three specific choices are doing work here. The separator is a newline, which cannot appear in any of the fixed-format fields and would in the worst case only be ambiguous inside the message body, which is the last field before three fixed-format ones. The sim_slot integer is stringified explicitly on both sides, because -1 must render as -1 and not as -1.0 or "-1" with quotes. And there is no trailing newline, because "the obvious thing" differs between languages and a trailing newline is invisible in every debugger you will use to hunt for it.

The heartbeat signs a shorter string, device_id then nonce then sent_at, with the same rules.

Defending the contract

A canonical form is only a contract if something breaks when you violate it. The usual test does not qualify:

// This test can never fail. It asserts that your function agrees with itself.
expect(hash, Canonical.messageHash(
  sender: 'MPESA',
  message: 'Test message',
  receivedAt: '2026-06-19T12:30:00+03:00',
));

Reorder the fields, change the separator, add a trailing newline: it still passes. What actually defends the contract is asserting the literal bytes:

test('incoming string-to-sign uses newline join in documented order', () {
  final s = Canonical.incomingStringToSign(
    deviceId: 'PHONE_001',
    sender: 'MPESA',
    message: 'hi',
    receivedAt: '2026-06-19T12:30:00+03:00',
    simSlot: 1,
    messageHash: 'abc',
    nonce: 'n-1',
    sentAt: '2026-06-19T12:30:01+03:00',
  );
  expect(
    s,
    'PHONE_001\nMPESA\nhi\n2026-06-19T12:30:00+03:00\n1\nabc\nn-1\n2026-06-19T12:30:01+03:00',
  );
});

That literal is the specification. Anyone writing a new receiver in a new language can copy it and check their implementation against it without reading a line of Dart. Both implementations also carry a comment saying the two sides must match byte for byte and that neither may be changed alone, which is the cheapest possible mitigation for the real risk here: not that someone gets it wrong, but that someone improves one side.

The timestamp format needs the same treatment, because it is inside the signed bytes. Everything on the wire is East Africa Time as ISO-8601, produced by one helper and asserted against a literal:

// 2026-06-19T09:30:00Z == 12:30:00 +03:00
final ms = DateTime.utc(2026, 6, 19, 9, 30, 0).millisecondsSinceEpoch;
expect(TimeUtils.iso8601Eat(ms), '2026-06-19T12:30:00+03:00');

A fixed offset with no daylight saving is a real constraint and a deliberate one. It removes an entire category of signature mismatch caused by two machines disagreeing about which timezone database they are on. It is also the thing that would need rethinking first if this ever ran outside East Africa.

The seven rules, in order

Verification order is not cosmetic. Each rule is cheaper than the one after it, and each one assumes the previous rules passed.

Rule 3 has one implementation detail that is not optional. The comparison is constant-time:

def verify_signature(string_to_sign: str, secret: str, signature: str) -> bool:
    expected = sign(string_to_sign, secret)
    # Constant-time comparison.
    return hmac.compare_digest(expected, signature or "")

An ordinary string comparison returns as soon as it finds a differing byte, and the time it took leaks how many leading bytes were right. Over enough requests that is a signature forgery oracle. compare_digest costs nothing and removes the whole category.

Nonce versus hash, drawn out

The distinction between rules 5 and 6 is the part that people re-derive incorrectly every time, so here is the full grid.

On the client side this collapses into a three-value enum, and the mapping is the entire reason nothing gets stuck:

enum SendOutcome {
  /// 2xx accepted, or server reported it as a duplicate - treat as done.
  success,

  /// Transient (network/5xx/timeout) - keep in queue, back off, retry.
  transient,

  /// Permanent (4xx other than duplicate, e.g. bad signature/config) - keep but
  /// the operator must fix configuration.
  permanent,
}

And the branch that matters:

if (code == 409) {
  // Conventional duplicate response.
  return const SendResult(SendOutcome.success, 'duplicate');
}

Without that single branch, a message whose response was lost on the way back retries until it exhausts its eight attempts and then sits in the failed queue forever, while the backend has had it stored the whole time. It is the most boring three lines in the codebase and it is the difference between a queue that drains and one that does not. If that failure shape is familiar, it is the same one behind duplicate M-Pesa callbacks and behind webhooks that get processed twice on timeout.

The five-minute window

Rule 4 rejects a payload whose sent_at is more than five minutes away from the server's clock, in either direction. The nonce ledger already prevents an exact replay, so what is the window for?

It bounds how long the ledger has to remember. Without a freshness window the nonce table must grow forever, because a signed request captured today could in principle be replayed in three years and you would need the nonce still on file to catch it. With a window, anything older than the window is rejected by the timestamp check before the ledger is even consulted, so consumed nonces can be pruned. In the Frappe implementation that is a daily job:

def prune_old_nonces(hours=24):
    """Daily: drop consumed nonces well past the freshness window."""
    from frappe.utils import add_to_date

    cutoff = add_to_date(now_datetime(), hours=-cint(hours))
    frappe.db.delete(NONCE_DT, {"created_at": ["<", cutoff]})
    frappe.db.commit()

Note the safety margin: the freshness window is five minutes and the prune keeps 24 hours. The ledger only has to outlive the window, and buying a factor of nearly three hundred costs almost nothing while removing any chance of an off-by-one in the interaction between two schedules.

The window is symmetric because phone clocks drift in both directions, and a till phone that has been off for a week and just booted may be minutes out before it syncs. Five minutes is generous enough to survive that and tight enough to keep the ledger small. If you have ever chased a signature that verifies on one machine and not another, the cause is very often not the signature at all but a clock that nobody was watching.

What the secret does and does not touch

The per-device secret is generated by the operator, entered once on the phone, and registered on the backend. Two properties keep it small.

It never travels over the wire. It is only ever used to compute an HMAC, so possession is proved without disclosure. There is no bearer token to steal from a log, because there is no bearer token.

It lives in flutter_secure_storage, which on Android is Keystore-backed, alongside the database encryption key. That key is generated on first use and never leaves the device either:

Future<String> dbEncryptionKey() async {
  final existing = await _storage.read(key: _kDbKey);
  if (existing != null && existing.isNotEmpty) return existing;
  final rng = Random.secure();
  final bytes = List<int>.generate(32, (_) => rng.nextInt(256));
  final key = base64UrlEncode(bytes);
  await _storage.write(key: _kDbKey, value: key);
  return key;
}

Note Random.secure() rather than Random(). The unqualified constructor is a fast pseudo-random generator that is perfectly fine for shuffling a list and completely unsuitable for a key. It is a one-word difference and it is the sort of thing that survives review because both lines look identical at a glance.

The transport is HTTPS, and that is enforced in code rather than assumed:

void _assertSecureUrl() {
  if (!_cfg.isHttps && !_cfg.allowInsecureHttp) {
    throw StateError(
      'Refusing to send over plain HTTP. Use an https:// URL, or enable the '
      'explicit debug "allow insecure HTTP" flag.',
    );
  }
}

The manifest backs it up with android:usesCleartextTraffic="false" and android:allowBackup="false". Signing a payload and then posting it over plain HTTP protects integrity while broadcasting the message body to anyone on the network, which for M-Pesa confirmations means names, phone numbers and amounts. The signature is not a substitute for the tunnel.

What I would do differently

The contract has no version field. Every payload is implicitly version one, and if the field order ever has to change, there is no way to run old and new phones against the same backend during the rollout. A single v field inside the signed string would have cost one line at the start and would now be worth a great deal, because sideloaded phones in shops do not all update on the same afternoon. This is the change I would make first.

The message_hash is also computed by the client and merely verified by the server. That is correct as far as it goes, and rule 2 catches any mismatch, but it does mean the client is the one deciding what "the same message" means. A backend that wanted to be stricter could recompute and store its own hash and treat the client's as advisory. In practice rule 2 makes the difference invisible, and I mention it because the asymmetry is worth being conscious of rather than because it has caused a problem.

Finally, the bottom-left quadrant of that grid, a reused nonce with an unfamiliar hash, is currently indistinguishable from an ordinary duplicate in the logs. It is the only combination with no innocent explanation, and it should be an alert rather than a line in a table. The data is all there. The alerting is not.

If you want to see this contract implemented inside a real ERP, including the routing gymnastics needed to expose these paths at all, that is the Frappe article. And for why the phone carries none of the M-Pesa meaning that all of this protects, see keeping the pipe dumb.