A state space model (SSM) is a way to process sequences by maintaining a hidden state that summarizes everything seen so far and updates step by step as new inputs arrive. The concept comes from control theory and signal processing, where state space equations have long described how systems evolve over time. In deep learning, the idea was adapted into a trainable layer that maps an input sequence to an output sequence through this evolving internal state.
What makes modern SSMs attractive is how they compare to the two dominant sequence architectures. Like a recurrent network, an SSM can be unrolled one step at a time with a fixed-size state, giving cheap, constant-memory inference. But unlike a traditional recurrent network, the linear structure of an SSM also allows the whole sequence to be computed in parallel during training, much like a convolution, so it scales well on modern hardware. This sidesteps the quadratic cost of attention, where every token must be compared to every other.
The breakthrough that made SSMs practical for deep learning was the S4 model in 2021, which found a stable and efficient parameterization for the state dynamics, enabling strong results on tasks with sequences thousands of steps long. The line continued with Mamba, which added input-dependent (selective) state updates and became a prominent attention alternative.
For a general reader, state space models represent one of the most credible challengers to the Transformer’s dominance. Their efficiency on very long sequences makes them appealing for long documents, audio, genomics, and any setting where context length and inference cost are limiting factors.