A plain recurrent network fails to learn dependencies that span dozens of time steps. Which explanation and remedy are correct?
Show answer and explanation
Answer: B
Backpropagation through time multiplies the same recurrent weight matrix at every step. If its singular values are below one, the product decays geometrically and early steps receive almost no signal, so long-range dependencies are not learned. LSTM and GRU cells add input, forget and output gates plus a cell-state path whose additive updates give gradients a more direct route, easing the decay. Gradient clipping handles the opposite failure, exploding gradients.
Why the other options are wrong
- A. Unbounded growth is the exploding variant, which clipping addresses; dropout does not change the multiplicative chain.
- C. Gradients do reach the parameters but become negligibly small; convolutional layers are not a sequence-model remedy.
- D. Oscillation is an optimiser-dynamics issue, and batch normalisation does not create long-range credit assignment.