FLOPS

Also known as: Floating-point operations per second · TFLOPS

On this page 7

FLOPS (floating-point operations per second) is the unit that expresses how many arithmetic operations on floating-point numbers a processor can complete each second. In graphics hardware, FLOPS is the headline figure for a GPU’s theoretical compute throughput, usually quoted in TFLOPS (trillions of operations per second) for consumer cards and in petaFLOPS or exaFLOPS for data center systems.

The number matters because shading, ray tracing, physics simulation and machine learning are all dominated by floating-point math. However, a FLOPS rating is a peak value derived from the chip design, not a measured result, so it describes what the silicon could do under ideal conditions rather than what a given game or application will achieve.

How peak FLOPS is calculated

Peak FLOPS is calculated by multiplying the number of floating-point execution lanes, the clock frequency and the operations each lane completes per cycle. For a GPU, the lanes are the shader cores, which NVIDIA calls CUDA cores and AMD calls stream processors.

Most vendors count a fused multiply-add (FMA) as two operations, because it performs a multiplication and an addition in one instruction with a single rounding step. A GPU with 10,240 FP32 lanes running at 2.5 GHz therefore reaches 10,240 x 2.5 x 10^9 x 2, or about 51 TFLOPS of FP32 throughput. This doubling is standard practice, but it means the figure assumes every lane issues an FMA on every cycle, which real workloads rarely sustain.

The table below shows the components of the formula and what each one depends on.

FactorWhat it representsTypical influence
Execution lanesShader or vector units that run floating-point mathScales with die area and architecture
Clock frequencyCycles per second, usually boost clockLimited by power and cooling
Operations per cycleTwo for FMA, more for packed or matrix mathSet by instruction set design
PrecisionBit width of each operandLower precision raises the count
Inputs to a peak FLOPS calculation

Precision levels and why FLOPS figures differ

A FLOPS figure is meaningless without its numeric precision, because the same chip delivers very different rates at different bit widths. The formats are defined by IEEE 754 for FP64, FP32 and FP16, while BF16, TF32, FP8 and FP4 are formats driven largely by machine learning.

Consumer graphics cards are optimized for FP32, the precision used by most shaders. FP64 throughput on gaming GPUs is often only 1/32 or 1/64 of the FP32 rate, while scientific accelerators keep a 1/2 ratio. Tensor cores add matrix FLOPS at reduced precision, which can be ten or more times higher than the vector FP32 rate, and some vendors also quote sparsity-accelerated figures that assume half the weights are zero.

The following table summarizes the common precision levels and where each one is used.

FormatBitsMain useRelative rate on gaming GPUs
FP6464Scientific simulation, financeVery low, often 1/64 of FP32
FP3232Shading, geometry, general computeBaseline vector rate
FP16 and BF1616Machine learning, some shaders2x vector or higher on tensor cores
FP8 and FP48 and 4AI inferenceHighest, tensor cores only
Floating-point precision levels in GPU specifications

Because of these differences, comparing a TFLOPS number from one card with a number from another is valid only when both refer to the same precision, same unit type and same sparsity assumption.

FLOPS compared with other performance metrics

FLOPS measures arithmetic capacity, but GPU performance also depends on memory bandwidth, cache behavior and fixed-function hardware. Many workloads are memory-bound: the cores wait for data from VRAM, so extra FLOPS bring no gain. The roofline model captures this by plotting achievable performance against arithmetic intensity, the number of operations performed per byte moved.

The table compares FLOPS with neighboring metrics that appear in GPU specifications.

MetricWhat it measuresWhen it is decisive
FLOPSFloating-point arithmetic rateCompute-heavy shaders, AI training
Memory bandwidthBytes per second to and from VRAMHigh resolutions, large models
Pixel fill ratePixels written per second by ROPsHeavy overdraw, high refresh rates
Texture rateTexels sampled per secondTexture-heavy scenes
Ray-triangle testsIntersection rate of RT coresRay-traced lighting
FLOPS and related throughput metrics

FLOPS is also distinct from IPS or MIPS, which count instructions of any kind, and from TOPS, which usually counts integer operations for neural processing units. Treating a TOPS rating as equivalent to TFLOPS is a frequent marketing confusion.

Theoretical peak versus sustained performance

Sustained FLOPS is the rate actually achieved by a workload, and it is almost always lower than the theoretical peak. Well-tuned dense matrix multiplication on a modern GPU can reach 70-90% of peak, while typical game shaders often stay well below half because of branching, memory stalls and instruction mix.

Several architectural details widen the gap. Some designs, such as GPUs that can dual-issue FP32 instructions, double the paper FLOPS only when the compiler finds two independent operations, so game performance rises far less than the specification suggests. Thermal and power limits also reduce clocks under sustained load, which lowers real throughput below the boost-clock figure.

Benchmarks therefore matter more than specifications. In supercomputing, the LINPACK benchmark used by the TOP500 list reports a measured FP64 figure called Rmax next to the theoretical Rpeak, and the ratio between them shows how efficiently a system converts paper FLOPS into work.

Using FLOPS correctly when comparing GPUs

FLOPS is most useful for comparing GPUs within the same architecture, where memory systems and scheduling are similar and the ratio of TFLOPS tracks performance reasonably well. Across architectures or vendors, it is a rough indicator at best.

SituationRecommended approachReason
Same architecture, different tierFP32 TFLOPS is a fair proxyShared design keeps efficiency similar
Different vendors for gamingUse game benchmarksArchitectures convert FLOPS differently
AI training hardwareCompare dense BF16 or FP8 tensor FLOPSSparsity figures inflate results
Scientific computingCheck FP64 rate and bandwidthGaming GPUs cut FP64 sharply
Laptop GPUsCheck power limit and sustained clockSame chip runs slower in thin designs
Decision guide for interpreting FLOPS ratings
  • Read the footnotes: vendors often state whether a figure includes sparsity or uses boost clocks.
  • Pair FLOPS with bandwidth to estimate whether a workload will be compute-bound or memory-bound.
  • Avoid cross-generation shortcuts: a newer card with fewer TFLOPS can still be faster in games.

The simple explanation

FLOPS tells you how much math a chip can do every second. Each operation is a calculation with decimal numbers, such as multiplying 3.75 by 0.4. A modern graphics card can do tens of trillions of these per second, which is why the number is usually written in TFLOPS.

Think of it like the top speed of a car. It shows what the engine could do on an empty road, but traffic, turns and fuel stops decide how fast you actually arrive. In the same way, a higher FLOPS number does not guarantee a faster game or application, because the chip also needs data delivered quickly enough to keep busy.

How it is used in practice

A small research lab wants to buy GPUs for training an image recognition model. The team compares two options: a gaming card advertising about 80 TFLOPS of FP32 and a data center accelerator advertising far more in tensor operations. Reading the specifications closely, they find the larger accelerator figure includes 2:4 sparsity, so they halve it and compare the dense BF16 tensor FLOPS of both cards instead.

They also check memory: the model needs 40 GB, which rules out the gaming card with 24 GB of VRAM regardless of its compute rate. After a short trial run, the accelerator sustains about 65% of its dense peak, while profiling shows the training loop is partly limited by memory bandwidth. The outcome is a purchase based on dense precision-matched FLOPS, memory capacity and measured throughput, rather than on the largest number printed on the product page.