# Pizza Order No‑Shows: Stop Chasing Accuracy—Pick the Right Metric

Pizza no-show prediction illustration

## Pizza order no-shows: why accuracy misleads and what to measure instead When predicting pizza order no-shows, accuracy is a trap. Real-world order data is usually imbalanced: most customers show up. A model that always predicts "show" can still report very high accuracy while being useless for business decisions. Instead of a single accuracy score, pick the metric that reflects the business cost of different errors. Below is a concise guide to choosing and validating the right metric. ### Quick refresher: confusion matrix and key metrics - True Positive (TP): predicted no-show and it was a no-show - False Positive (FP): predicted no-show but customer showed up - False Negative (FN): predicted show but customer no-showed - True Negative (TN): predicted show and customer showed up From these: - Precision = TP / (TP + FP) — of predicted no-shows, how many are actual no-shows - Recall = TP / (TP + FN) — of all actual no-shows, how many did we catch - Accuracy = (TP + TN) / (TP + FP + FN + TN) — sensitive to class balance Example (1000 orders, 100 actual no-shows): - A dumb model predicts everyone will show → Accuracy = 900/1000 = 90%, Recall = 0% (it never detects a no-show) — useless despite high accuracy. ### Which metric should you prioritize? - Prioritize precision when acting on a "no-show" prediction triggers costly operations: extra staffing, remaking pizzas, holding inventory, or idling drivers. A low-precision model will create many false alarms and drive those costs up. - Prioritize recall when missing a no-show is more expensive than a false alarm. For example, if failing to predict a no-show causes serious customer dissatisfaction, penalties, or wasted resources that are larger than the cost of occasional false alarms. Often you’ll need a balanced view: measure both, but optimize the one aligned with business impact. ### How to validate — don’t rely on a single scalar 1. Confusion matrix: always report it for your chosen threshold so stakeholders see TP/FP/FN/TN counts. 2. Precision–Recall (PR) curve: with class imbalance, PR curves show trade-offs more clearly than ROC. Use PR AUC to compare models when positive class (no-shows) is rare. 3. Threshold tuning: choose an operating threshold based on expected monetary cost (or a loss matrix) rather than default 0.5. Find the threshold that minimizes expected cost = FP_cost * FP + FN_cost * FN. 4. Calibration: ensure predicted probabilities are well-calibrated if you want to use them to make threshold decisions. 5. Monitor in production: class balance and costs can drift (promos, seasons). Re-evaluate the operating point regularly. ### Practical workflow 1. Quantify business costs of FP and FN (and any costs for TP/TN if applicable). 2. Select primary metric (precision or recall) and secondary metrics (F1, PR AUC, confusion matrix). 3. Train models with class weighting, sampling, or cost-sensitive objectives if needed. 4. Use validation PR curves and cost-based thresholding to pick the operating point. 5. Validate with confusion matrices and example cases. Deploy with monitoring and alerts for drift. ### Short checklist for interviews or reports - State the business cost matrix up front. - Explain why accuracy is misleading with imbalanced data. - Declare the primary metric (precision or recall) and why. - Show the confusion matrix at your chosen threshold and a PR curve. - Describe thresholding and calibration decisions. Choosing the right metric ties your model evaluation to real business outcomes. Don’t chase accuracy—optimize for what actually matters. #MachineLearning #DataScience #MLOps