A data scientist training a convolutional classifier notices the model memorizes its training images. Which regularization method should be added so that, during each training step, a random subset of hidden units is silenced?
Show answer and explanation
Answer: C
Dropout layers zero out a chosen fraction of hidden activations on every training forward pass, so the network cannot rely on any single unit and effectively trains an ensemble of thinned sub-networks. Dropout is disabled at inference, where the full network runs. Weight decay penalizes large weights rather than silencing units, batch normalization mainly stabilizes activation distributions, and early stopping limits training duration instead of altering the forward pass.
Why the other options are wrong
- A. Weight decay shrinks parameter magnitudes through an L2 penalty on the loss. It does not randomly deactivate any units during the forward pass, so it does not match the described mechanism.
- B. Batch normalization rescales activations using batch statistics and gives a mild regularizing side effect. It does not randomly silence a fraction of neurons, so it is not the technique described.
- D. Early stopping ends training when validation performance degrades. It controls how long training runs rather than changing what happens inside each forward pass, so it does not silence hidden units.
Exam tip. Dropout is a training-only technique and is switched off at inference.