# Digital Media Store Design: Idempotency Is Non‑Negotiable in Purchases
## Why idempotency matters for purchases
In a Digital Media Store, the purchase endpoint must be idempotent. Networks fail, clients retry, and gateways time out—so the same request can hit your backend multiple times. If you don't design for idempotency, you'll risk double‑charging users or creating duplicate PURCHASE records. That harms revenue, user trust, and data integrity.
## The rule (simple and non-negotiable)
Treat `POST /purchase` as a transaction keyed by an `Idempotency-Key`. Store the key along with a status (PENDING / SUCCESS / FAILED) and the payment `transaction_id` or error details. On retry, return the original result instead of reprocessing the payment.
This single pattern prevents duplicate charges, simplifies retry logic, and makes behaviors deterministic.
## Recommended implementation pattern
1. Client generates an Idempotency-Key (e.g., UUID v4) and sends it in a header: `Idempotency-Key: `.
2. Server receives the request and looks up the key (scoped to the user/account or global depending on your requirements).
3. If the key is new, insert a record with status = PENDING and start processing the payment.
4. If the key exists and status = PENDING, return the existing pending response or wait/stream updates.
5. If the key exists and status = SUCCESS or FAILED, return the stored result (success payload or error) without reprocessing.
## Example idempotency table schema (conceptual)
```
idempotency_keys
-----------------
id UUID PRIMARY KEY
user_id UUID -- optional, scope the key
idempotency_key TEXT UNIQUE -- or use (user_id, idempotency_key)
status TEXT -- PENDING, SUCCESS, FAILED
created_at TIMESTAMP
updated_at TIMESTAMP
payment_txn_id TEXT NULL -- payment gateway transaction identifier
response_body JSON NULL -- serialized response to return for retries
error_code TEXT NULL
expiry_at TIMESTAMP -- TTL for cleanup
```
Guidelines:
- Enforce a uniqueness constraint on `(user_id, idempotency_key)` to avoid races where two inserts try to create the same key.
- Write the initial PENDING row within a transaction or via an atomic upsert so only one worker proceeds to process the payment.
## Workflow (detailed)
- Client: POST /purchase with body and header `Idempotency-Key: abc`.
- Server: BEGIN TRANSACTION
- Try to insert idempotency row with status = PENDING. If insert fails because key exists, SELECT the row.
- If row.status == PENDING: return a 202/200 with the pending state or wait depending on your UX.
- If row.status == SUCCESS or FAILED: return the stored response_body and status.
- If this worker created the PENDING row: call the payment gateway.
- On payment success: update row to SUCCESS, set payment_txn_id and response_body, commit.
- On payment failure: update row to FAILED, set error_code and response_body, commit.
- Return the response saved in `response_body` for all retries.
## Handling concurrency and races
- Use a unique constraint and an atomic insert/upsert so only one process will see itself as the owner of the PENDING row.
- If you need to avoid blocking clients, return a consistent response for PENDING and provide a mechanism to query status (e.g., `GET /purchase/status?key=...`).
- Alternatively, use SELECT ... FOR UPDATE on the idempotency row to serialize processing for that key.
## What to store in response_body
Store the minimal canonical response that you return to the client on success or failure, including HTTP status code and body (e.g., receipt id, purchased items, errors). This lets retries receive exactly the same result.
## Edge cases and operational concerns
- Long-running payments: mark PENDING and consider a reasonable timeout before marking FAILED. Use payment gateway webhooks to update final status asynchronously.
- Partial failures / timeouts: a client may timeout but the payment completes. When the client retries with the same key, return the SUCCESS stored result.
- Reconciliation: keep logs and reconcile with your payment provider using `payment_txn_id` to detect anything missed.
- Cleanup: TTL old idempotency rows (e.g., 30–90 days) with a background job to avoid unbounded growth.
- Security: scope keys to the authenticated user to prevent cross-account replay.
## Client guidance
- Clients should generate a fresh Idempotency-Key per logical purchase attempt (UUIDs are fine).
- Retry the same key on communication failures; on a user-initiated new purchase, generate a new key.
- Do not reuse keys across different purchase intents or amounts.
## Quick pseudocode
```
if not exists (select 1 from idempotency_keys where user_id = U and key = K):
insert (K, user_id=U, status=PENDING)
process_payment()
if success:
update idempotency_keys set status=SUCCESS, payment_txn_id=..., response_body=... where key=K
else:
update idempotency_keys set status=FAILED, response_body=... where key=K
return response_body
else:
row = select * from idempotency_keys where key=K
return row.response_body
```
## Testing and observability
- Test retries by forcing client or network failures and asserting no duplicate charges.
- Log idempotency key lifecycle transitions (PENDING -> SUCCESS/FAILED) and payment_txn_id.
- Monitor metrics: number of duplicate requests, rate of retries, time spent in PENDING.
## TL;DR
Make `POST /purchase` idempotent using an `Idempotency-Key`. Store key + status + payment_txn_id + canonical response. On retries, return the saved result instead of reprocessing. This pattern protects revenue, preserves user trust, and keeps your data clean.
#SystemDesign #DistributedSystems #BackendEngineering