A multilayer perceptron (MLP) is a fully connected feedforward neural network made of an input layer, one or more hidden layers and an output layer, in which every unit applies a nonlinear activation function to a weighted sum of the previous layer’s outputs. Within neural networks and artificial intelligence more broadly, the MLP is the canonical general-purpose architecture and the reference point against which more specialized designs are described.
The MLP extends the single-layer perceptron by stacking layers, which lets it represent nonlinear decision boundaries such as the XOR function. It is trained with backpropagation and gradient descent, and it survives today both as a standalone model for tabular data and as the feedforward block inside every transformer layer.
How a multilayer perceptron computes an output
An MLP computes its output through a sequence of affine transformations and nonlinearities. For each layer, the input vector is multiplied by a weight matrix, a bias vector is added, and an activation function is applied element by element. The output of one layer becomes the input of the next, so information flows strictly forward with no cycles or recurrence.
Formally, layer l computes h = f(W h_prev + b), where W has one row per unit in the layer. Because every unit connects to every unit in the preceding layer, these are called dense or fully connected layers. The number of parameters in one layer equals inputs times outputs plus outputs, which makes parameter count grow quickly with width.
The table below describes each layer type and what it contributes.
| Layer | Function | Typical size |
|---|---|---|
| Input layer | Holds feature values; performs no computation | Equal to the number of features |
| Hidden layer | Learns intermediate representations | Tens to thousands of units |
| Output layer | Produces predictions in the task format | One unit per class or target |
Without nonlinear activations, any stack of layers collapses into a single linear transformation, so depth would add nothing. The nonlinearity is therefore what gives the MLP its expressive power.
Activation functions and output heads
The choice of activation function strongly affects trainability and gradient flow. Early MLPs used sigmoid and tanh, which saturate for large inputs and contribute to the vanishing gradient problem in deeper stacks. Modern MLPs mostly use ReLU or smooth variants such as GELU and SiLU.
The following table compares common hidden-layer activations and their practical trade-offs.
| Activation | Output range | Strength | Weakness |
|---|---|---|---|
| Sigmoid | 0 to 1 | Smooth, probabilistic reading | Saturates, slows learning |
| Tanh | -1 to 1 | Zero-centered output | Still saturates |
| ReLU | 0 to infinity | Cheap, strong gradients | Units can die at zero |
| GELU | About -0.17 to infinity | Smooth, standard in transformers | Slightly more compute |
| SwiGLU | Unbounded | Gated, better quality per parameter | Extra weight matrix |
The output layer is matched to the task: softmax with cross-entropy loss for multiclass classification, a sigmoid for binary or multilabel problems, and a linear output with mean squared error for regression.
Training an MLP with backpropagation
An MLP learns by minimizing a loss function over training examples. A forward pass produces predictions, the loss measures their error, and backpropagation propagates that error backward, layer by layer, yielding a partial derivative for each weight and bias. Adam, SGD with momentum or a similar optimizer then nudges each parameter against its derivative.
Several settings determine whether training converges well. Weight initialization schemes such as Xavier or He keep activations at a stable scale across layers. Learning rate is usually the most sensitive hyperparameter, and regularization through dropout, weight decay or early stopping limits overfitting.
The table below lists the main design choices and their usual starting values.
| Setting | Common starting point | Effect if poorly chosen |
|---|---|---|
| Hidden layers | 2 to 4 | Too few underfit; too many slow training |
| Units per layer | 64 to 1,024 | Excess width memorizes noise |
| Learning rate | 0.001 with Adam | Divergence or very slow progress |
| Batch size | 32 to 512 | Noisy or overly smooth updates |
| Dropout rate | 0.1 to 0.5 | Underfitting if set too high |
MLP compared with other neural architectures
The MLP makes minimal assumptions about input structure, which is both its strength and its weakness. It treats every input feature as independent of position, so it cannot exploit the spatial locality of images or the order of sequences unless that structure is engineered into the features.
The following comparison shows how specialized architectures add inductive bias that the MLP lacks.
| Architecture | Connectivity | Built-in assumption | Typical data |
|---|---|---|---|
| MLP | Dense, all-to-all | None beyond smoothness | Tabular features, embeddings |
| Convolutional network | Local, shared kernels | Spatial locality | Images, audio spectrograms |
| Recurrent network | Connections through time | Sequential order | Time series, text |
| Transformer | Attention plus MLP blocks | Learned pairwise relations | Text, code, multimodal |
The universal approximation theorem states that an MLP with one sufficiently wide hidden layer can approximate any continuous function on a bounded domain. The theorem guarantees existence, not learnability: it says nothing about required width, data or training success, which is why deeper and more structured networks often perform better in practice.
Practical limitations of multilayer perceptrons
MLPs are simple to build but have predictable failure modes that practitioners check for early.
- Unscaled inputs destabilize training; standardize numeric features before fitting.
- Overfitting on small tables is common; on many tabular benchmarks, gradient-boosted trees still outperform a plain MLP.
- Parameter inefficiency on images: a dense layer on a 224 by 224 color image needs about 150,000 weights per hidden unit.
- Poor extrapolation: predictions outside the training range are often unreliable.
- Opaque reasoning: hidden representations are hard to interpret without dedicated explanation methods.
Explained without jargon
A multilayer perceptron is like an assembly line of decision makers. The first row receives raw facts, such as the size and age of a house. Each worker in the next row looks at everything the previous row passed along, weighs it in their own way, and passes a single opinion forward.
After a few rows, the last worker gives the final answer, such as a price estimate. At first the opinions are random, but after comparing answers with real outcomes thousands of times, every worker adjusts how much attention they pay to each input. The line gradually learns to give useful answers without anyone writing down the rules.
A typical use case
A regional utility wants to forecast hourly electricity demand for the next day. Its data team builds a table of features for each hour: temperature, humidity, hour of day, day of week, holiday flags and the demand recorded 24 and 168 hours earlier. After standardizing the inputs, they train an MLP with three hidden layers of 256 ReLU units, dropout of 0.2 and a single linear output.
Trained on four years of history and validated on the most recent year, the model captures interactions that a linear regression missed, such as heat that raises demand more on weekday afternoons. The team compares it against a gradient-boosted tree baseline, keeps whichever performs better each season, and uses the forecasts to schedule generation and reduce reserve costs.