A convolutional classifier scores 97% on training batches but only 79% on held-out images. The team wants better generalization without changing the dataset. What is the best next step?
Show answer and explanation
Answer: A
An 18-point gap between training and held-out accuracy is a classic overfitting signal, meaning high variance. Dropout randomly silences units during training so the network cannot rely on fragile co-adapted features. Weight decay penalizes large weights, which keeps the decision function smoother. Both techniques constrain the model's effective complexity without requiring new data, which is why they address the generalization gap directly.
Why the other options are wrong
- B. More layers add capacity, and with a fixed dataset that usually widens the gap between training and held-out accuracy rather than narrowing it.
- C. A higher learning rate changes optimization speed and stability, not the generalization gap. It can make training less stable without fixing the problem.
- D. Removing the held-out set hides the problem instead of fixing it, because the team loses the only measure of how well the model generalizes.