# Real-Time Fraud Detection: The Interview-Ready System Design Checklist
Real-Time Fraud Detection: The Interview-Ready System Design Checklist
Real-time fraud detection is primarily a system-design problem; machine learning is an important but secondary piece. In interviews you should be able to explain end-to-end trade-offs and justify choices across data, features, models, streaming architecture, and operations. Below is a concise, interview-ready checklist with practical enrichments and talking points.
## 1) Data: inputs, labeling, and quality
- Core inputs: transaction records, user behavior (clicks, page views, session durations), device/browser fingerprints, IP/geolocation, merchant metadata, and historical labeled fraud cases.
- Labeling: understand label lag and noise. Use confirmed chargebacks, fraud investigations, and human review as ground truth; be explicit about false positives/negatives in labels.
- Data quality: deduplicate, validate schemas, normalize timestamps, and enforce GDPR/PCI constraints.
- Feature signals to engineer:
- Velocity: transactions per user/card IP/time window
- Device changes: new device or fingerprint drift
- Spend deviation: deviation from historical mean/median
- Geo/IP anomalies: sudden country change, VPN/tor usage
- Merchant risk and category-specific patterns
- Behavioral patterns: mouse/typing dynamics, session flows
- Temporal features: hour-of-day, day-of-week
- Handling imbalance: keep a realistic class distribution in validation sets and track prevalence drift.
## 2) Modeling: start simple, iterate
- Baseline first: logistic regression or single decision tree to establish a clear, interpretable baseline and latency stack.
- Stronger models: random forest, gradient boosting (XGBoost/LightGBM/CatBoost). Consider ensembles only when they add measurable lift.
- Imbalance strategies: class weights, resampling, SMOTE, focal loss, and careful cross-validation that respects time ordering.
- Interpretability: use feature importance and SHAP values to explain predictions to product and compliance teams.
- Calibration & thresholding: tune thresholds for precision/recall trade-offs; consider cost-sensitive thresholds based on business impact.
- Latency-aware models: if sub-100ms scoring is required, consider model compression, pruning, or converting to lightweight models for online scoring.
## 3) Streaming architecture & real-time features
- Event transport: Kafka (or Kinesis) for high-throughput, durable event streaming.
- Stream processing: Flink or Spark Structured Streaming for stateful aggregations (velocity counts, rolling statistics) and real-time feature computation.
- Feature store: provide both online (low-latency key-value) and offline features (for training). Tools: Feast, custom Redis/Key-Value store.
- Serving: expose scoring via low-latency REST/gRPC endpoints or embed model scoring inside the stream processor for ultra-low latency.
- Idempotency & consistency: ensure exactly-once or at-least-once semantics where needed; handle duplicate events and out-of-order events.
- Backpressure & batching: design for bursts (batch scoring vs per-event); document latency-service-level objectives.
## 4) Deployment, ops & monitoring
- Metrics to monitor:
- Business: precision, recall, FPR, FNR, true/false positives over time, fraud dollars prevented
- Model: AUC, calibration, score distribution, PSI (population stability index)
- System: request latency (p95/p99), throughput, error rate
- Drift detection: monitor label distribution, feature distribution, and model performance; trigger retraining when drift exceeds thresholds.
- Feedback loop: pipeline to surface confirmed fraud labels back into training data (human-in-the-loop for verification).
- CI/CD & governance: automated model validation, data checks (Great Expectations), canary deploys, A/B testing, rollout and rollback plans.
- Logging & audit: store scores, inputs, and decisions for investigations and compliance.
- Security & privacy: encrypt PII, minimize sensitive data in logs, comply with GDPR/PCI.
## 5) Trade-offs & interview talking points
- Latency vs accuracy: justify if you choose a simpler model for sub-100ms scoring, or an ensemble if business tolerates extra latency.
- Offline vs online features: stateful streaming features are powerful but increase complexity—explain why you’d implement which features online.
- Explainability: discuss how you’ll surface reasons for blocking/flagging transactions to operations teams.
- Failure modes: describe how the system behaves on outages (fail-open vs fail-closed), and how to prevent cascading failures.
- Cost & scalability: estimate storage/compute costs for retention windows and stateful streaming; discuss partitioning and sharding strategies.
## Quick checklist to recite in interviews
- Data: transactions, behavior, device/IP, labeled outcomes
- Features: velocity, device change, spend deviation, geo anomalies
- Model: baseline (logistic/tree), then RF/GBM, handle imbalance
- Streaming: Kafka → Flink/Spark → online feature store → REST/gRPC scoring
- Ops: monitor precision/recall/F1, latency; automate retraining and feedback loop
Keep answers structured: state assumptions, trade-offs, and scalability implications. Start with a simple, clear baseline and layer complexity (feature store, ensembles, streaming state) only as needed.
#MachineLearning #MLOps #DataEngineering