A tensor core is a specialized processing unit inside a GPU that performs matrix multiply-accumulate operations on small blocks of numbers in a single step, usually at reduced precision such as FP16, BF16, FP8 or INT8. The name comes from NVIDIA, which introduced tensor cores in its Volta architecture in 2017; comparable units from other vendors are called matrix cores or matrix engines.
In graphics hardware, tensor cores sit alongside the general-purpose shader cores. They deliver many times the throughput of ordinary cores for dense linear algebra, which is the core workload of neural networks. That makes them essential for AI training and inference, and in gaming for AI-based upscaling, frame generation and denoising.
How a tensor core computes
A tensor core executes the operation D = A × B + C, where A, B, C and D are small matrices, often called tiles. In the original Volta design, each tensor core processed a 4 × 4 × 4 multiply-accumulate per clock: 64 fused multiply-add operations in one instruction, where a standard core performs one.
The key trick is mixed precision. Inputs A and B use a compact format such as FP16 or FP8, which lets more values fit in registers and memory bandwidth, while products are accumulated in higher precision, usually FP32. Keeping the sum in FP32 preserves accuracy across the thousands of additions that a large matrix multiplication requires.
Programmers rarely address tensor cores one by one. Instead, a warp of 32 threads cooperatively issues matrix instructions, and libraries such as cuBLAS and cuDNN, or frameworks built on them, tile large matrices into pieces the hardware can process. Compilers and libraries handle most of the details.
Tensor core generations and supported formats
Each GPU generation has added new numeric formats and higher throughput. Lower-precision formats roughly double peak throughput per step down, but they demand more careful scaling to avoid accuracy loss.
| Architecture | Year | Notable formats added | Key feature |
|---|---|---|---|
| Volta | 2017 | FP16 with FP32 accumulation | First tensor cores in data center GPUs |
| Turing | 2018 | INT8, INT4 | First tensor cores in consumer GeForce RTX cards |
| Ampere | 2020 | TF32, BF16, FP64 on data center parts | 2:4 structured sparsity support |
| Ada Lovelace | 2022 | FP8 in consumer GPUs | Enabled DLSS frame generation |
| Hopper | 2022 | FP8 (E4M3 and E5M2) | Transformer Engine with dynamic scaling |
| Blackwell | 2024 | FP4 and FP6 | Micro-scaling formats for large AI models |
TF32 deserves a note: it keeps FP32’s 8-bit exponent but only a 10-bit mantissa, so existing FP32 code can run on tensor cores with little or no code change and usually acceptable accuracy for training.
Ampere also introduced 2:4 structured sparsity, in which two of every four weights are zero. The tensor core skips those zeros and can double effective throughput for inference, but only when a model has been pruned to that exact pattern and retrained to recover accuracy.
Tensor cores compared with other GPU units
A modern GPU contains several kinds of execution units, each tuned for a different workload. The comparison shows where tensor cores fit among the Topic’s neighboring concepts.
| Unit or concept | Main workload | Operations per instruction | Precision focus |
|---|---|---|---|
| Tensor core | Matrix multiply-accumulate | Dozens to hundreds of FMAs | FP16, BF16, FP8, INT8 and lower |
| CUDA core | General scalar and vector math | One FMA per lane | FP32 and INT32 |
| RT core | Ray and bounding volume intersection | Fixed-function traversal | Not applicable |
| Integrated GPU | Graphics in a shared CPU package | Varies, often without matrix units | FP32 and FP16 |
| NPU | Low-power neural inference | Matrix or vector arrays | INT8 and INT4 |
Comparable hardware exists elsewhere: AMD Matrix Cores in Instinct accelerators, AI accelerators in recent Radeon GPUs and Intel XMX engines in Arc GPUs. The principle is identical, though instruction sets and supported formats differ.
Where tensor cores are used
Tensor cores are a form of hardware acceleration for one kind of math, so their value depends on whether a workload can be expressed as large matrix multiplications. The table lists the main uses.
| Workload | Typical precision | Benefit |
|---|---|---|
| Neural network training | BF16, FP16, FP8 | Large speedups over standard cores |
| AI inference | FP8, INT8, FP4 | Higher throughput and lower energy per result |
| DLSS upscaling | FP16 or FP8 | Renders at lower resolution, reconstructs a sharp image |
| Frame generation | FP8 | Inserts AI-generated frames between rendered ones |
| Ray tracing denoising | FP16 | Cleans noisy images from few rays per pixel |
| Scientific computing | FP64 on data center GPUs | Faster dense linear algebra |
Tensor core throughput is only reached when memory bandwidth and VRAM capacity keep up. Large models often become limited by how fast weights can be read, which is why data center GPUs pair tensor cores with high-bandwidth memory.
Limitations and common misconceptions
Tensor cores are powerful but narrow by design. Misunderstanding their limits leads to unrealistic performance expectations.
- Peak figures assume ideal conditions, including sparsity and lowest precision; real workloads often reach a fraction.
- Not every algorithm benefits: branching code and irregular memory access still run on standard cores.
- Low precision can harm accuracy without loss scaling, calibration or quantization-aware training.
- Tensor core counts are not comparable across generations, because each generation does more work per core.
- Games do not use tensor cores for rasterization; they help only through AI features such as upscaling.
For buyers, the practical question is whether their software actually calls tensor cores. Machine learning frameworks do so automatically when mixed precision is enabled, while a game benefits only if it implements an AI upscaler or denoiser. Otherwise the tensor cores remain idle silicon for that workload.
Put simply
Most of a graphics card is made of many small calculators that each handle one sum at a time. A tensor core is different. It is like a machine that fills out an entire multiplication table in one go.
Artificial intelligence mostly consists of exactly that kind of table math. So tensor cores make AI tasks dramatically faster. In games, they power features that sharpen lower-resolution images so the game looks good and still runs smoothly.
In practice
A gamer with a mid-range RTX graphics card plays a demanding title at 1440p with ray tracing enabled. Without help, the game runs at about 40 frames per second. Enabling DLSS in quality mode makes the GPU render internally at a lower resolution, around 960p, while tensor cores run a neural network that reconstructs a 1440p image using motion vectors and previous frames.
The frame rate rises to roughly 65 frames per second with image quality that most players find close to native. In a separate setting, a small research lab trains a language model in BF16 mixed precision on the same tensor core architecture and cuts training time several-fold compared with plain FP32 on standard cores.