Mamba, a selective structured state space model introduced by Gu and Dao, makes SSM parameters functions of the input instead of fixed values, letting the model selectively propagate or forget information token by token. The design delivers 5x higher inference throughput than Transformers, roughly doubles compute and memory when sequence length doubles (versus roughly quadrupling for self-attention), and keeps improving on real data up to million-length sequences.
Why Did Transformers Need a New Sequence Architecture?
Subquadratic-time architectures — including linear attention, gated convolution and recurrent models, and structured state space models (SSMs) — were developed specifically to address Transformers' computational inefficiency on long sequencesCITE:E3. Prior to Mamba, none of these subquadratic alternatives had matched attention's performance on important modalities such as languageCITE:E3. Mamba itself is documented as a selective structured state space model designed to work around Transformers' computational inefficiency when handling long sequencesCITE:E4.
How Does Mamba's Selective State Space Parameterization Work?
Mamba's authors, Gu and Dao, let the SSM's parameters become functions of the input itself, rather than fixed valuesCITE:E1. This addresses a known weakness of earlier SSMs on discrete modalities and allows the model to selectively propagate or forget information along the sequence-length dimension depending on the current tokenCITE:E1.
How Does Mamba Compare to Transformers on Throughput, Cost, and Sequence Length?
Mamba delivers 5x higher inference throughput than Transformers while scaling linearly rather than quadratically with sequence lengthCITE:E2.
| Metric | Mamba | Self-attention (Transformer) |
|---|
| Inference throughput | 5x higher than Transformers | baseline |
| Compute/memory when sequence length doubles | roughly doubles | roughly quadruples |
| Performance trend as sequence length grows | improves on real data up to million-length sequences | not addressed in evidence |
Gu and Dao report that Mamba's performance on real data continues to improve up to sequences of million-token lengthCITE:E2. PyTorch's engineering blog quantifies the underlying cost curve: doubling the input sequence length roughly doubles Mamba's compute and memory needs, whereas self-attention would quadruple themCITE:E7.
What Does Linear Scaling Mean for Processing Million-Length Sequences?
Because its compute and memory needs scale linearly rather than quadratically, Mamba has fast inference and can scale to very long sequencesCITE:E5. PyTorch's engineering team identifies scalability to long sequences as one key advantage of Mamba-style modelsCITE:E6.
What this means: The evidence traces a single chain from problem to mechanism to payoff. Transformers' computational inefficiency on long sequences motivated the search for subquadratic architectures such as SSMsCITE:E3, and Mamba's input-dependent parameterization was the specific mechanism that made a selective SSM viableCITE:E1. That mechanism shows up directly in the cost numbers — a roughly 2x-versus-4x compute ratio when sequence length doublesCITE:E7 — which compounds into a reported 5x throughput advantage and performance that keeps improving out to million-length sequencesCITE:E2. The scalability claims from Hugging Face's documentationCITE:E5 and PyTorch's engineering blogCITE:E6 describe the same underlying property — linear rather than quadratic growth — from two different vantage points.
Author's Take・EffectStory 編輯部
The number worth isolating is the roughly 2x-versus-4x compute ratio when sequence length doubles — that arithmetic is the actual mechanical source of the reported 5x throughput gap, not a separate marketing claim layered on top of it. Because Mamba replaces full pairwise attention with input-dependent selective state updates, its real test isn't peak throughput on short benchmarks but whether that same linear cost curve keeps holding as sequence lengths keep climbing toward the million-token range Gu and Dao measured. The concrete thing to watch next is whether independent implementations reproduce that per-doubling compute ratio, since right now it rests on the original paper and one engineering blog rather than a broader base of reported measurements.