Training a deep recurrent network occasionally produces enormous gradient values that wreck an epoch. Which remedy directly addresses this?
Show answer and explanation
Answer: B
Exploding gradients arise when repeated multiplication through many steps amplifies the backward signal. Gradient clipping computes the norm of the gradient vector and, if it exceeds a threshold, rescales it to that threshold while preserving direction. The update then takes a bounded step, so a single pathological batch cannot send weights far off. It is standard in recurrent and transformer training and is usually paired with a warmup schedule; clipping by value is a cruder alternative.
Why the other options are wrong
- A. A larger learning rate amplifies the same oversized update and makes divergence more likely.
- C. Removing layers reduces capacity and changes the architecture rather than controlling the magnitude of the update.
- D. Freezing embeddings does not stop gradients in recurrent or deep layers from exploding.