Evaluating Model Performance
Accuracy isn't everything — the metrics that reveal how good a model really is.
Inspired by learning material from Microsoft
Choosing the right metric is critical. A model that looks great by one measure can be useless by another.
Classification metrics
- Accuracy: fraction of correct predictions — misleading on imbalanced data.
- Precision: of predicted positives, how many were right?
- Recall: of actual positives, how many did we catch?
- F1 score: balance of precision and recall.
The confusion matrix
A table of true/false positives and negatives — the foundation of most classification metrics.
Regression metrics
- MAE: average absolute error.
- RMSE: penalizes large errors more.
- R²: how much variance the model explains.
Why one number isn't enough
Consider the cost of different mistakes. Missing a disease (low recall) is very different from a false alarm (low precision).
Key takeaways
- Accuracy can mislead on imbalanced data.
- Precision and recall capture different errors.
- Pick metrics that match the real-world cost of mistakes.