Broad intuition · Study map 18 of 85
Sequences: recurrence, attention, and state
Suggested preparation: Vision: sliding detectors and image tokens
Picture it
A recurrent model carries a changing notebook through a sequence. Attention can consult earlier entries directly. State-space models compress history into a structured evolving state, changing the memory and computation tradeoff.
What to understand
- RNNs, LSTMs, temporal dependencies, and forecasting
- State-space models, Mamba, and hybrid architectures
Follow the references
1 · Paper
Gu & Dao · Mamba ↗Read the architecture diagram and motivation for selective state updates before the equations.
You understand it when…
What might be lost when a long history must fit into a fixed-size state?
This is a study map. Follow the reference for the full explanation; use the question to test your mental model. Browse the reference shelf.