Last spring I watched a booking platform charge a customer $1,140 three times in ninety seconds. The user tapped “Confirm” on a flaky hotel Wi-Fi connection, the request timed out client-side, the app retried, and the API happily created three orders. The fix took a week to ship. It should have taken a day, and it would have, if the team had designed idempotency in from the start instead of bolting it on after the incident report.

Idempotency keys are the single most under-built feature in small business APIs that take money or hold inventory. Payment processors like Stripe and Square support them natively, but your own booking, scheduling, and order endpoints need the same guarantee, and the design decisions matter more than most guides admit. This post covers how I store keys, what replay semantics actually look like in production, why request fingerprinting is not optional, and what the big providers do differently.

What Idempotency Actually Guarantees

An operation is idempotent if running it twice produces the same result as running it once. In HTTP terms, GET, PUT, and DELETE are idempotent by spec. POST is not, and POST is what creates charges, bookings, and orders. The idempotency key closes that gap: the client generates a unique key per logical operation, sends it with the request, and the server promises “if I have seen this key before, I will not repeat the side effects.”

The promise sounds simple. The edge cases are where teams bleed. What happens when the first request is still processing and a second one with the same key arrives? What if the response to the first attempt was lost but the write succeeded? What if a malicious client reuses another user’s key? Each of these has a correct answer, and each one is a deliberate design choice, not a default.

What Most Guides Get Wrong

Most tutorials show you a Redis SETNX key 1 example and call it done. That pattern fails in at least four production scenarios I have hit personally.

First, it conflates deduplication with execution tracking. A key is not a flag. It is a record of an in-flight or completed request, and you need the full lifecycle, not a boolean. Second, it ignores concurrency. Two requests with the same key arriving simultaneously must serialize, not both check-and-set. Third, it skips fingerprinting, which means a client bug that sends key A with request body B, then retries with key A and body C, gets silently wrong results. Fourth, it never mentions that a stored key without a stored response forces the client to retry blindly forever.

Stripe’s engineering blog documented this exact class of problem years ago: a client that retries after a network failure needs to know whether the server did the work. The answer is to store the response, not just the key.

Key Storage: What to Keep and For How Long

Every idempotency record needs five fields: the key itself, a request fingerprint, an execution state, the response payload (or an error), and a creation timestamp. My default schema in PostgreSQL looks like this:

CREATE TABLE idempotency_keys (
  key          TEXT PRIMARY KEY,
  fingerprint  TEXT NOT NULL,
  state        TEXT NOT NULL, -- in_flight | completed | failed
  response     JSONB,
  created_at   TIMESTAMPTZ DEFAULT now()
);

Retention is a business decision. For payments, keep records at least as long as your refund window. Stripe retains idempotency keys for 24 hours on standard API calls, but for your own ledger I recommend 90 days minimum. Storage is trivially cheap; a row with a 2 KB response payload is nothing next to one duplicate charge. For a broader view of building the infrastructure around patterns like this, see my notes on multi-region architecture where key storage location becomes a real constraint.

Use your primary database, not Redis alone, for the authoritative record. Redis is fine as a fast-path lock, but if Redis evicts the key under memory pressure and Postgres does not have it, you have a dedup hole. The pattern that has survived every audit I have run: INSERT ... ON CONFLICT DO NOTHING in Postgres as the source of truth, Redis SET NX PX as the concurrency gate.

Replay Semantics: The Three Rules

When a request arrives with a key that already exists, exactly one of three things must happen.

Same key, same fingerprint, in flight. Block or return a 409 Conflict with a retry hint. The first request is still running. Never run both.

Same key, same fingerprint, completed. Return the stored response with the original status code. Do not re-execute. This is the core promise, and it includes returning the same error response if the first attempt failed permanently.

Same key, different fingerprint. Return 422 Unprocessable Entity. This is a client bug or an attack, and silently proceeding is how data gets corrupted. Log it loudly.

Different fingerprint handling is the part almost nobody implements, and it is the part that catches real bugs. I have seen a mobile app reuse a UUID because of a broken retry wrapper, sending key X with a $50 payload and then key X with a $500 payload. Without fingerprinting, you charge $500 against a key the client thinks holds $50.

Request Fingerprinting

The fingerprint is a hash of everything that defines the operation’s intent: the HTTP method, the path, and a canonical serialization of the body. Canonicalize JSON before hashing: sort keys, strip insignificant whitespace, normalize number formats. Otherwise {"amount":50} and { "amount": 50 } hash differently and you get false mismatches.

Use SHA-256. Do not use MD5 or SHA-1; there is no performance argument at this scale and you do not want a collision to merge two distinct payment intents. Hash the raw bytes after canonicalization, not the parsed object re-serialized by your framework, or you inherit your serializer’s quirks.

Exclude fields that legitimately differ between retries: client timestamps, trace IDs, telemetry headers. A fingerprint that changes on every retry breaks the replay guarantee and sends you back to double-charging. The contract with your client team must be explicit about which body fields are part of the operation identity. This kind of precision in request handling pairs well with the discipline covered in the technical SEO audit checklist, where deterministic request handling prevents whole categories of silent failure.

Provider Implementation Notes

Stripe sets the standard. You send an Idempotency-Key header, they guarantee the request executes once per key for 24 hours, and concurrent requests with the same key return a 409 while the first is in flight. Keys are scoped per account, not per API key. Stripe also returns the idempotent replayed response, which is exactly the behavior you should copy.

Square behaves similarly with idempotency keys on payments, but their documentation notes keys expire and that you should generate a fresh key per logical operation, not per HTTP retry of the same operation. That distinction matters: one logical checkout gets one key, and every transport-level retry reuses it.

Twilio uses idempotency keys on its newer APIs (Verify and Frontline) and on message redaction endpoints, but classic SMS sending relies on your own dedup because the API predated the pattern. If you build on Twilio for booking confirmations, implement keys yourself.

AWS API Gateway does not give you idempotency automatically. API keys are for throttling and auth, not deduplication. You still own the logic. The same applies to API gateways: running a dedup layer alongside rate limiting protects backend databases from surge cascades, as detailed in my guide on API rate limiting and cost management, but the authoritative key storage and replay logic remain yours.

A Decision Framework for Small Teams

If you run a booking or payment flow at SMB scale, here is the order I would build in, based on having done this migration twice:

  1. Add the Idempotency-Key column and table first, before any logic. Backfill is painful.
  2. Enforce fingerprint matching on day one. Retrofitting it after clients start reusing keys is a data cleanup project.
  3. Return stored responses, including stored errors. Clients that cannot distinguish “failed” from “unknown” will retry forever.
  4. Scope keys per merchant or tenant. A global keyspace invites cross-tenant collisions when a client generates weak keys.
  5. Alert on fingerprint mismatches. They are almost always bugs in your own clients, and they surface early if you watch.

Key generation belongs to the client, but validate it server-side: require a minimum length (I use 16 bytes of random data, hex or base64 encoded), reject keys from an allowlist of obvious bad values like “test” or “1”. UUIDv4 is acceptable; UUIDv7 is better because its time-ordered component makes debugging log correlation easier.

Testing the Guarantee

Unit tests will not find your idempotency bugs. Concurrency tests will. My minimum suite:

  • Fire 20 parallel requests with the same key and assert exactly one side effect occurred and all 20 callers received the same response body.
  • Kill the worker mid-handler, retry with the same key, and assert the stored state recovers to either completed or safely re-runnable.
  • Send key A with body B, then key A with body C, and assert a 422, not a charge.

The W3C’s notes on HTTP semantics define idempotency at the spec level, but the spec is silent on concurrent retries and lost responses. That gap is yours to fill.

One more trap: clock skew breaks naive TTL logic. If your expiry check uses created_at + interval on the application server rather than the database clock, a two-second skew on a 24-hour window is harmless, but a skewed server computing expiry for a 30-second in-flight lock will double-execute under load. Compute expiry in the database.

Conclusion

Idempotency keys are not a Redis one-liner. They are a small state machine with three states, a fingerprint contract, and a storage decision that outlives your refund policy. Get the storage right in your primary database, enforce fingerprint matching from the first deploy, return stored responses including errors, and test with parallel requests, not sequential ones. The providers, Stripe above all, have already shown the correct semantics. Copy them deliberately rather than discovering the edge cases through a triple-charged customer and an incident review. The week you spend implementing this before launch is cheaper than the week you spend after.