# Stop Guessing: Pick the Right Categorical Encoding in Interviews # Stop Guessing: Pick the Right Categorical Encoding in Interviews Categorical encoding cheat-sheet Categorical features must be converted to numbers before feeding them to models. The encoding you pick can significantly affect model quality, runtime, and overfitting risk. Below is a compact guide to the three most common encodings — when to use each one, their trade-offs, and quick best-practices you can cite in interviews. ## 1) One‑Hot Encoding - What it does: Creates a binary column for each category (category_i → 0/1). - When to use: Nominal categories with low cardinality (e.g., color: red/blue/green). - Pros: No implied order; model sees categories as independent. - Cons: Explodes dimensionality with many categories → sparsity, memory blow-up, higher risk of overfitting. - Tip: Use for low-cardinality features. For medium/high cardinality consider hashing, embeddings, or target encoding instead. ## 2) Label Encoding (Ordinal Integers) - What it does: Maps categories to compact integers (A→0, B→1, C→2). - When to use: Features that are truly ordered (small/medium/large) or some tree-based models where integer labels are often acceptable. - Pros: Very compact, no extra columns. - Cons: Introduces an artificial ordinal relationship if none exists — many linear models and distance-based algorithms will be misled. - Tip: If the categories are not ordinal, avoid for linear/logistic/KNN/SVM. If using trees, label encoding is commonly used but still be cautious — consider frequency encoding or one-hot for low cardinality. ## 3) Target (Mean) Encoding - What it does: Replaces each category with the mean of the target variable for that category (e.g., average purchase rate). - When to use: High-cardinality categories where the category carries predictive signal. - Pros: Very informative and compact (one column). Works well when category prevalence correlates with target. - Cons: Major risk of target leakage and overfitting — the category mean on the training set can leak information about the target. - How to mitigate leakage: - Use out-of-fold (K-fold) encoding: compute category means using only the training folds and apply to the holdout fold. - Smooth the estimates toward a global prior to reduce variance for rare categories. Example smoothing formula: Encoded_value = (count * category_mean + alpha * global_mean) / (count + alpha) Where alpha controls how strongly rare categories are pulled toward the global mean (common choices: alpha between 5 and 50). Larger alpha → more smoothing. Quick implementation pattern for models: - During CV training: for each fold, compute smoothed means on the training portion and transform the validation portion with those means. - For final model: compute smoothed means on the full training data and apply to test/unseen data. ## Rule of Thumb - One‑Hot: use for low-cardinality nominal features. - Label: use for truly ordered features or (cautiously) with tree models. - Target: use for high-cardinality features only with strict CV and smoothing to avoid leakage. ## Extra tips to mention in interviews - Always think about the downstream model family (linear vs tree vs neural net) when choosing an encoding. - For very high cardinality, consider hashing trick or learned embeddings (deep learning) instead of one-hot. - Validate using proper CV and check for leakage — target encoding without CV is a common source of exaggerated validation scores. Keep this mental checklist in interviews: cardinality, order, model type, and leakage risk. Name the risks and the mitigations — interviewers like concise, practical answers.