Mamba (also called the Mamba architecture) is a sequence model built on selective state space models, which process tokens through a compact recurrent state whose dynamics depend on the input. Introduced by Albert Gu and Tri Dao in late 2023, it is one of the most prominent alternatives to the Transformer among modern neural networks in artificial intelligence.
Its central promise is linear scaling with sequence length during training and constant memory per generated token during inference, while matching Transformers of similar size on many language modeling benchmarks. That trade-off makes Mamba attractive for long documents, genomics, audio and time series.
How Mamba processes a sequence
Mamba starts from the state space model formulation: a hidden state h evolves according to h’ = A h + B x, and the output is y = C h. The continuous system is discretized with a step size Δ, usually through a zero-order hold, which turns it into a recurrence that updates the state once per token.
Earlier structured state space models such as S4 used fixed, input-independent parameters, which allowed fast convolutional training but prevented the model from deciding what to remember. Mamba makes Δ, B and C functions of the current input. This selection mechanism lets the model reset its state, ignore filler tokens or retain a key fact, behavior that resembles gating in an LSTM but runs at much larger scale.
Input-dependent parameters break the convolution trick, so Mamba uses a hardware-aware parallel scan. The kernel keeps the expanded state in fast on-chip SRAM, fuses discretization, scan and output into one operation, and recomputes intermediate states in the backward pass instead of storing them.
The main parameters of the selective scan are summarized below.
| Parameter | Role | Typical setting |
|---|---|---|
| A (state matrix) | Controls how fast state decays | Diagonal, learned, input-independent |
| Δ (step size) | Decides how much of the current token enters the state | Input-dependent, per channel |
| B (input matrix) | Writes the input into the state | Input-dependent |
| C (output matrix) | Reads the state into the output | Input-dependent |
| N (state size) | Memory capacity per channel | 16 in Mamba, 64 to 256 in Mamba-2 |
Inside the Mamba block
A Mamba block replaces both the attention layer and the feedforward layer of a Transformer block with one homogeneous unit. The input is normalized, usually with RMSNorm, and projected to about twice the model width. One branch passes through a short causal convolution, a SiLU activation and the selective scan; the other branch acts as a multiplicative gate. The gated result is projected back to model width and added through a residual connection.
The following table lists the stages of the block in order.
| Stage | Operation | Purpose |
|---|---|---|
| Normalization | RMSNorm or layer normalization | Stabilizes training |
| Input projection | Linear layer with expansion factor 2 | Creates main and gate branches |
| Causal convolution | Depthwise 1D convolution, kernel about 4 | Mixes nearby tokens |
| Selective scan | Input-dependent state space recurrence | Models long-range dependencies |
| Gating | Elementwise product with SiLU-activated branch | Controls information flow |
| Output projection | Linear layer back to model width | Restores residual stream size |
Mamba compared with Transformers and recurrent networks
Mamba sits between attention-based Transformers and classic recurrent networks. It trains in parallel like a Transformer but generates like an RNN, carrying a fixed-size state instead of a growing key-value cache.
| Architecture | Training cost in sequence length | Inference memory per step | Exact recall of distant tokens |
|---|---|---|---|
| Transformer | Quadratic | Grows with context (KV cache) | Strong |
| Recurrent neural network | Linear, but sequential | Constant | Weak |
| Linear attention | Linear | Constant | Limited |
| Mamba | Linear, parallel scan | Constant | Moderate, improves with state size |
The fixed state is both the advantage and the limit. Because all history must be compressed into N values per channel, Mamba struggles more than attention on tasks that require verbatim copying or associative recall across long contexts, such as retrieving a phone number seen thousands of tokens earlier.
Mamba-2 and hybrid models
Mamba-2, published in 2024, introduced state space duality, a framework showing that a restricted selective SSM is equivalent to a form of masked linear attention. Restricting A to a scalar per head allows the computation to use matrix multiplications, which run efficiently on tensor cores, and permits much larger state sizes at similar speed.
Many production models now use hybrid stacks that interleave Mamba layers with a small share of attention layers. The attention layers restore precise recall, while the Mamba layers keep most of the stack cheap for long contexts. Published hybrids include Jamba, Zamba, Samba and Nemotron-H, typically with one attention layer for every several Mamba layers.
| Situation | Recommended design | Reason |
|---|---|---|
| Very long sequences such as DNA or audio | Pure Mamba | Linear cost dominates the budget |
| General chat and retrieval-heavy tasks | Hybrid Mamba and attention | Attention handles exact recall |
| Short contexts under a few thousand tokens | Transformer | Mature tooling, little speed gain |
| Edge inference with tight memory | Pure or mostly Mamba | No growing KV cache |
Limitations and common misconceptions
- Mamba is not an attention variant: it uses no query-key comparison, although Mamba-2 shows a formal link to linear attention.
- Linear complexity does not guarantee faster wall-clock time at short lengths, where optimized attention kernels remain competitive.
- Recall-intensive tasks such as in-context retrieval and copying remain a known weakness of pure SSMs.
- Tooling is less mature: custom CUDA kernels limit portability to some accelerators and inference servers.
- Benchmark parity is size-dependent: results reported at one scale do not automatically hold at larger or smaller scales.
A non-technical view
Imagine reading a long book in two ways. A Transformer keeps every page open on a huge table and glances back at all of them for each new word, which gets slow and crowded as the book grows. Mamba instead keeps a compact notebook of what matters, updating it page by page and deciding what to write down or erase.
The notebook stays the same size no matter how long the book is, so Mamba stays fast and light on very long texts. The cost is that some exact details may not make it into the notebook, which is why many systems mix Mamba with a few attention layers.
A concrete example
A genomics research group wants to classify regulatory regions in DNA sequences of up to one million base pairs. A Transformer at that length would need prohibitive memory for attention, so the team trains a Mamba-based model with about 100 million parameters that reads the sequence one nucleotide at a time.
Training fits on a small cluster of GPUs because cost grows linearly with sequence length, and inference runs with a constant memory footprint. Compared with a Transformer limited to 16,000-token windows, the Mamba model captures long-range interactions between distant regions, and the group reports better accuracy on enhancer prediction while keeping the experiment within its compute budget.