Kolmogorov-Arnold network

Also known as: KAN

On this page 7

A Kolmogorov-Arnold network (KAN) is a neural network architecture in which every connection carries a learnable univariate function, usually a spline, while the nodes simply sum their incoming signals. Among neural networks it is a structural alternative to the multilayer perceptron, which places fixed activation functions on nodes and learnable scalar weights on edges.

The design is inspired by the Kolmogorov-Arnold representation theorem and was popularized in 2024 as a candidate for more interpretable and parameter-efficient models in scientific tasks. Its practical value is still being assessed, and results vary strongly with the problem.

The theorem behind Kolmogorov-Arnold networks

The Kolmogorov-Arnold representation theorem states that any continuous function of several variables on a bounded domain can be written as a finite sum of compositions of continuous one-variable functions and addition. In formula form, a function of n inputs equals the sum over q from 0 to 2n of an outer function applied to a sum of inner functions of each input.

The theorem implies that addition is the only truly multivariate operation needed. The catch is that the inner functions it guarantees can be highly irregular and non-smooth, which made the result look useless for learning for decades. A KAN sidesteps this by stacking more than two layers and using arbitrary widths, so smooth learnable functions can approximate the target.

AspectOriginal theoremKolmogorov-Arnold network
DepthExactly two layers of functionsAny number of layers
Width2n + 1 outer termsChosen freely per layer
Inner functionsExist, possibly fractal-likeSmooth splines learned from data
PurposeExistence proofPractical approximation and fitting
From the theorem to a trainable network

How a KAN layer computes its output

A KAN layer maps a vector of inputs to outputs by applying a separate learnable function to each input-output pair and summing the results at every output node. A layer with 10 inputs and 5 outputs therefore holds 50 univariate functions rather than 50 scalar weights.

In the reference design each edge function is a B-spline on a fixed grid plus a residual base function such as SiLU. The spline coefficients are the trainable parameters, and gradients reach them through ordinary backpropagation, so standard optimizers work without changes.

ParameterTypical rangeWhat it controls
Grid size3 to 20 intervalsResolution of each edge function
Spline orderUsually 3 (cubic)Smoothness of the learned curve
Network shapeFew layers, small widthsCapacity and compositional depth
Base functionSiLU or similarStable residual path for training
Regularization strengthProblem dependentSparsity used for pruning
Main parameters of a KAN layer

A useful feature is grid extension: training can start on a coarse grid and then refine it, initializing the finer splines from the coarse ones. This lets accuracy improve without restarting training, a property that plain multilayer perceptrons lack.

KAN compared with a multilayer perceptron

The essential difference is where the nonlinearity lives. A multilayer perceptron applies the same activation function, such as ReLU, at every node after a linear combination, while a KAN learns a distinct nonlinearity on every edge and keeps nodes linear.

PropertyKolmogorov-Arnold networkMultilayer perceptron
Learnable elementUnivariate function per edgeScalar weight per edge
ActivationLearned, varies by edgeFixed, shared across nodes
Parameters per edgeGrid size plus spline orderOne
InterpretabilityEdge curves can be plottedWeights are hard to read
Hardware efficiencyPoor fit for dense matrix kernelsHighly optimized on GPUs
Theoretical basisKolmogorov-Arnold theoremUniversal approximation theorem
Kolmogorov-Arnold network versus multilayer perceptron

Because each edge holds many coefficients, a KAN can reach a given accuracy with fewer edges but more parameters per edge. Reported advantages appear mainly on smooth, low-dimensional functions from physics and mathematics, while fair comparisons with equal parameter counts show smaller or no gains on many general machine learning benchmarks.

Interpretability and symbolic regression

The strongest argument for a KAN is that a trained network can be read like a diagram of formulas. Each edge function is a one-dimensional curve, so a researcher can plot it and recognize shapes such as a sine, a square or an exponential.

  1. Train a small KAN with sparsity regularization so that unimportant edges shrink.
  2. Prune inactive nodes and edges to obtain a compact graph.
  3. Inspect each remaining edge curve and snap it to a symbolic function when the fit is close.
  4. Retrain the remaining affine coefficients to obtain an explicit closed-form expression.

This workflow makes KANs attractive for scientific discovery and symbolic regression, where the goal is a compact law rather than a black box predictor. The method works best when the true relationship is compositional and fairly smooth; noisy, high-dimensional data rarely yields clean formulas.

Variants, limitations and common mistakes

Several variants replace B-splines with other basis functions to improve speed or smoothness. The table summarizes the main families and the trade-off each one makes.

BasisAdvantageDrawback
B-splinesLocal control, grid extensionSlow evaluation
Radial basis functionsFaster, GPU friendlyLess local precision
Chebyshev polynomialsSmooth global fitUnstable at high degree
Fourier seriesNatural for periodic signalsPoor for non-periodic data
WaveletsMulti-scale featuresMore hyperparameters
Common KAN basis function variants
  • Slow training: spline evaluation does not map well to matrix multiplication, so KANs often train several times slower than a multilayer perceptron of similar size.
  • Grid range errors: inputs outside the spline grid fall back to the base function, so input normalization and grid updates are essential.
  • Overstated generality: treating KANs as a drop-in replacement for transformer or convolutional blocks rarely pays off without careful tuning.
  • Overfitting on fine grids: very large grid sizes memorize noise and destroy interpretability.

In plain words

Most neural networks are built from simple switches that all behave the same way. Learning only adjusts how strongly they are connected. A Kolmogorov-Arnold network works differently. Here every connection learns its own curve, a small bendable shape that transforms the signal.

Think of a sound mixing desk. Each cable has its own adjustable equalizer instead of a plain volume knob. Each curve can be drawn and examined. So people can often see what the network has learned, sometimes even as a math formula. The price is speed. These networks are slower to train and not yet better at most everyday tasks.

A real-world example

A research group in fluid mechanics has several thousand simulated measurements of drag on small particles at different speeds, sizes and fluid viscosities. The researchers suspect a compact relationship but do not know its form, so they train a KAN with shape 3-4-1, a grid of 5 intervals and sparsity regularization.

After pruning, only five edges remain active. The plotted curves resemble a logarithm, a square and a linear term, and after the group snaps these edges to symbolic functions and retrains the coefficients, the network reduces to a short formula built from dimensionless ratios of the inputs.

The final expression matches held-out simulations within about 2 percent and can be checked against established theory by hand. A multilayer perceptron reached similar accuracy, but only the KAN produced a human-readable law that the team could publish and test.