A neuron, also called an artificial neuron, node or unit, is the basic computing element of a neural network. It receives several numeric inputs, multiplies each by a learned weight, adds a bias, and passes the sum through an activation function to produce a single output that other neurons or the final prediction use.
The concept is loosely inspired by biological nerve cells, which fire when incoming signals exceed a threshold. The analogy is historical rather than literal: an artificial neuron is a simple mathematical function, and the power of neural networks comes from connecting very large numbers of these units in layers and training their parameters together.
How an artificial neuron computes its output
An artificial neuron computes a weighted sum followed by a nonlinearity. For inputs x1 to xn, weights w1 to wn and bias b, it calculates z = w1x1 + w2x2 + … + wnxn + b and outputs a = f(z), where f is the activation function.
| Element | Symbol | Role |
|---|---|---|
| Inputs | x1 … xn | Values from data or from the previous layer |
| Weights | w1 … wn | Learned strength and sign of each connection |
| Bias | b | Learned offset that shifts the activation point |
| Pre-activation | z | Weighted sum plus bias |
| Activation function | f | Nonlinear transformation of z |
| Output | a = f(z) | Value passed to the next layer |
A worked example shows the arithmetic. With inputs x = (2, 1, 3), weights w = (0.5, -1.0, 0.2) and bias b = 0.1, the weighted sum is 1.0 – 1.0 + 0.6 + 0.1 = z = 0.7. A ReLU activation returns 0.7, while a sigmoid activation returns 1 / (1 + e^-0.7), or approximately 0.668.
Geometrically, the weights define a hyperplane in input space, and the bias moves it away from the origin. A single neuron with a threshold activation can therefore separate only classes that are linearly separable, which is why networks need many neurons and several layers.
Activation functions used in neurons
The activation function gives a neuron its nonlinearity, without which any stack of layers would collapse into a single linear transformation. Its choice affects training speed, gradient stability and the range of the output.
| Function | Output range | Typical use | Main drawback |
|---|---|---|---|
| Step (threshold) | 0 or 1 | Historical perceptron | No useful gradient for training |
| Sigmoid | 0 to 1 | Binary output probabilities | Vanishing gradients when saturated |
| Tanh | -1 to 1 | Recurrent networks, older hidden layers | Also saturates at extremes |
| ReLU | 0 to infinity | Default for hidden layers | Neurons can die at constant zero |
| GELU | About -0.17 to infinity | Transformer feed-forward layers | Slightly costlier to compute |
| Softmax | 0 to 1, summing to 1 | Multi-class output layer | Applies to a layer, not one neuron |
ReLU, defined as max(0, z), became the default for hidden layers because its gradient is exactly 1 for positive inputs, which keeps signals strong in deep networks. Its weakness is the dying ReLU problem: a neuron whose pre-activation stays negative for all inputs outputs zero permanently and stops learning.
How neurons learn their weights
A neuron learns by adjusting its weights and bias to reduce a loss function that measures prediction error. The network computes the loss on a batch of examples, backpropagation calculates the gradient of the loss with respect to every weight, and an optimizer moves each weight a small step against its gradient.
For one weight, the basic update is w_new = w – learning rate x gradient. With a learning rate of 0.01 and a gradient of 0.4, a weight of 0.5 becomes 0.5 – 0.004 = 0.496. Modern optimizers such as Adam adapt the step size per parameter, but the principle is the same.
Initialization matters because neurons that start with identical weights receive identical gradients and never differentiate. Schemes such as Xavier (Glorot) and He initialization draw random weights scaled to the number of inputs, so that activations keep a stable variance as signals pass through many layers.
Artificial neurons compared with biological neurons
Artificial neurons borrow vocabulary from neuroscience, but they are vastly simpler than biological neurons. The comparison clarifies what the analogy does and does not imply.
| Aspect | Artificial neuron | Biological neuron |
|---|---|---|
| Signal | Continuous number | Electrical spikes over time |
| Inputs | Tens to thousands of weighted values | Often thousands of synapses |
| Computation | Weighted sum and fixed function | Complex dendritic and chemical processing |
| Learning | Gradient-based weight updates | Synaptic plasticity, mechanisms partly unknown |
| Timing | Synchronous layer by layer | Asynchronous, timing carries information |
Spiking neural networks model the timing of discrete spikes more faithfully and are studied for energy-efficient neuromorphic hardware, but mainstream deep learning uses the simpler rate-based neuron described here.
Neurons inside modern network architectures
In practice, neurons are implemented as layers of matrix operations rather than as individual objects. A dense layer of 512 neurons receiving 1,024 inputs is one matrix multiplication with 524,288 weights plus 512 biases, which is why graphics processors that excel at matrix arithmetic accelerate neural networks so effectively.
- Fully connected layers connect every neuron to every input of the previous layer.
- Convolutional layers use neurons that share weights and see only a small local region, such as a 3 x 3 pixel patch.
- Transformer feed-forward blocks apply the same two-layer neuron stack to each token after the attention mechanism mixes information between tokens.
- Output neurons are sized to the task: one for regression, one per class for classification.
A common misconception is that individual neurons store individual concepts. Research on interpretability shows that most neurons respond to mixtures of features, a property called polysemanticity, so meaning is usually distributed across many units rather than located in one.
A non-technical view
An artificial neuron is a tiny calculator inside an AI system. It takes in a few numbers and decides how much each one matters. Then it adds them up and passes on a single result.
Think of a judge on a talent show who scores several qualities, such as voice, timing and stage presence. The judge cares about some qualities more than others. Training teaches each neuron how much weight to give each input. One neuron alone is simple, but millions working together can recognize speech or images.
A concrete example
A bank builds a small network to flag suspicious card transactions. Each transaction is described by 20 numeric features, such as amount, distance from the previous purchase and time since the last transaction. The first hidden layer has 64 neurons, each with 20 weights and one bias, which gives 1,344 learnable parameters in that layer alone.
After training, engineers inspect one neuron and find that it responds strongly when a large amount coincides with a purchase far from home within minutes of the previous one. No one programmed that rule; the neuron’s weights learned it from historical fraud cases. The final output neuron uses a sigmoid activation, and transactions scoring above 0.9 are held for review, which lets analysts focus on a small fraction of daily traffic.