A data science lead notices that her classifier scores 0.98 accuracy on the examples used to fit it but only 0.71 on a holdout set drawn from the same population. Which diagnosis fits best?
Show answer and explanation
Answer: C
A large gap between strong training performance and weaker held-out performance is the signature of overfitting: the model has learned details specific to the training examples that do not transfer to new data from the same population. Remedies include regularization such as L2 penalties or dropout, early stopping guided by a validation curve, data augmentation, and collecting more representative examples.
Why the other options are wrong
- A. Underfitting would produce weak scores on both the training and holdout data, so a strong training score rules out a shortage of model capacity as the explanation here.
- B. Learning rate schedules change how quickly optimization converges, but they do not explain why a model that fits training examples well fails on unseen data from the same population.
- D. The same metric applied to both splits would reveal the mismatch equally, so the gap points to poor generalization rather than to a problem with how the metric is reported.