A Kolmogorov-Arnold network (KAN) is a neural network architecture in which every connection carries a learnable univariate function, usually a spline, while the nodes simply sum their incoming signals. Among neural networks it is a structural alternative to the multilayer perceptron, which places fixed activation functions on nodes and learnable scalar weights on edges.
The design is inspired by the Kolmogorov-Arnold representation theorem and was popularized in 2024 as a candidate for more interpretable and parameter-efficient models in scientific tasks. Its practical value is still being assessed, and results vary strongly with the problem.
The theorem behind Kolmogorov-Arnold networks
The Kolmogorov-Arnold representation theorem states that any continuous function of several variables on a bounded domain can be written as a finite sum of compositions of continuous one-variable functions and addition. In formula form, a function of n inputs equals the sum over q from 0 to 2n of an outer function applied to a sum of inner functions of each input.
The theorem implies that addition is the only truly multivariate operation needed. The catch is that the inner functions it guarantees can be highly irregular and non-smooth, which made the result look useless for learning for decades. A KAN sidesteps this by stacking more than two layers and using arbitrary widths, so smooth learnable functions can approximate the target.
| Aspect | Original theorem | Kolmogorov-Arnold network |
|---|---|---|
| Depth | Exactly two layers of functions | Any number of layers |
| Width | 2n + 1 outer terms | Chosen freely per layer |
| Inner functions | Exist, possibly fractal-like | Smooth splines learned from data |
| Purpose | Existence proof | Practical approximation and fitting |
How a KAN layer computes its output
A KAN layer maps a vector of inputs to outputs by applying a separate learnable function to each input-output pair and summing the results at every output node. A layer with 10 inputs and 5 outputs therefore holds 50 univariate functions rather than 50 scalar weights.
In the reference design each edge function is a B-spline on a fixed grid plus a residual base function such as SiLU. The spline coefficients are the trainable parameters, and gradients reach them through ordinary backpropagation, so standard optimizers work without changes.
| Parameter | Typical range | What it controls |
|---|---|---|
| Grid size | 3 to 20 intervals | Resolution of each edge function |
| Spline order | Usually 3 (cubic) | Smoothness of the learned curve |
| Network shape | Few layers, small widths | Capacity and compositional depth |
| Base function | SiLU or similar | Stable residual path for training |
| Regularization strength | Problem dependent | Sparsity used for pruning |
A useful feature is grid extension: training can start on a coarse grid and then refine it, initializing the finer splines from the coarse ones. This lets accuracy improve without restarting training, a property that plain multilayer perceptrons lack.
KAN compared with a multilayer perceptron
The essential difference is where the nonlinearity lives. A multilayer perceptron applies the same activation function, such as ReLU, at every node after a linear combination, while a KAN learns a distinct nonlinearity on every edge and keeps nodes linear.
| Property | Kolmogorov-Arnold network | Multilayer perceptron |
|---|---|---|
| Learnable element | Univariate function per edge | Scalar weight per edge |
| Activation | Learned, varies by edge | Fixed, shared across nodes |
| Parameters per edge | Grid size plus spline order | One |
| Interpretability | Edge curves can be plotted | Weights are hard to read |
| Hardware efficiency | Poor fit for dense matrix kernels | Highly optimized on GPUs |
| Theoretical basis | Kolmogorov-Arnold theorem | Universal approximation theorem |
Because each edge holds many coefficients, a KAN can reach a given accuracy with fewer edges but more parameters per edge. Reported advantages appear mainly on smooth, low-dimensional functions from physics and mathematics, while fair comparisons with equal parameter counts show smaller or no gains on many general machine learning benchmarks.
Interpretability and symbolic regression
The strongest argument for a KAN is that a trained network can be read like a diagram of formulas. Each edge function is a one-dimensional curve, so a researcher can plot it and recognize shapes such as a sine, a square or an exponential.
- Train a small KAN with sparsity regularization so that unimportant edges shrink.
- Prune inactive nodes and edges to obtain a compact graph.
- Inspect each remaining edge curve and snap it to a symbolic function when the fit is close.
- Retrain the remaining affine coefficients to obtain an explicit closed-form expression.
This workflow makes KANs attractive for scientific discovery and symbolic regression, where the goal is a compact law rather than a black box predictor. The method works best when the true relationship is compositional and fairly smooth; noisy, high-dimensional data rarely yields clean formulas.
Variants, limitations and common mistakes
Several variants replace B-splines with other basis functions to improve speed or smoothness. The table summarizes the main families and the trade-off each one makes.
| Basis | Advantage | Drawback |
|---|---|---|
| B-splines | Local control, grid extension | Slow evaluation |
| Radial basis functions | Faster, GPU friendly | Less local precision |
| Chebyshev polynomials | Smooth global fit | Unstable at high degree |
| Fourier series | Natural for periodic signals | Poor for non-periodic data |
| Wavelets | Multi-scale features | More hyperparameters |
- Slow training: spline evaluation does not map well to matrix multiplication, so KANs often train several times slower than a multilayer perceptron of similar size.
- Grid range errors: inputs outside the spline grid fall back to the base function, so input normalization and grid updates are essential.
- Overstated generality: treating KANs as a drop-in replacement for transformer or convolutional blocks rarely pays off without careful tuning.
- Overfitting on fine grids: very large grid sizes memorize noise and destroy interpretability.
In plain words
Most neural networks are built from simple switches that all behave the same way. Learning only adjusts how strongly they are connected. A Kolmogorov-Arnold network works differently. Here every connection learns its own curve, a small bendable shape that transforms the signal.
Think of a sound mixing desk. Each cable has its own adjustable equalizer instead of a plain volume knob. Each curve can be drawn and examined. So people can often see what the network has learned, sometimes even as a math formula. The price is speed. These networks are slower to train and not yet better at most everyday tasks.
A real-world example
A research group in fluid mechanics has several thousand simulated measurements of drag on small particles at different speeds, sizes and fluid viscosities. The researchers suspect a compact relationship but do not know its form, so they train a KAN with shape 3-4-1, a grid of 5 intervals and sparsity regularization.
After pruning, only five edges remain active. The plotted curves resemble a logarithm, a square and a linear term, and after the group snaps these edges to symbolic functions and retrains the coefficients, the network reduces to a short formula built from dimensionless ratios of the inputs.
The final expression matches held-out simulations within about 2 percent and can be checked against established theory by hand. A multilayer perceptron reached similar accuracy, but only the KAN produced a human-readable law that the team could publish and test.