Green AI

On this page 7

Green AI is the approach to artificial intelligence research and engineering that treats computational efficiency, energy use and carbon emissions as primary evaluation criteria, alongside accuracy. Within AI ethics, it addresses the environmental cost of training and running models and the question of who can afford to participate in research when progress depends on ever larger compute budgets.

The term was popularized in 2019 by researchers at the Allen Institute for AI, who contrasted it with Red AI: work that buys small accuracy gains with massive increases in compute. Green AI does not reject large models. It asks that their costs be measured, reported and justified, and that efficiency count as a legitimate research contribution.

How Green AI measures the cost of a model

Green AI measures cost by tracing a model from floating-point operations to electricity and then to greenhouse gas emissions. The chain matters because the same computation can have very different footprints depending on the hardware, the data center and the local electricity grid.

The basic estimate multiplies the energy drawn by accelerators and servers by the data center power usage effectiveness (PUE), which adds cooling and power distribution overhead, and then by the carbon intensity of the grid, expressed in grams of CO2 equivalent per kilowatt-hour. Grid intensity is the most variable factor: a hydro-dominated grid may sit below 50 g CO2e/kWh, while a coal-heavy grid can exceed 700 g CO2e/kWh.

The following table lists the main quantities that Green AI reporting uses and what each one captures.

QuantityUnitWhat it capturesMain limitation
Floating-point operationsFLOPsHardware-independent amount of computationIgnores hardware efficiency and idle power
Energy consumptionkWh or MWhElectricity drawn during training or inferenceHard to meter on shared clusters
Power usage effectivenessRatio, about 1.1 to 1.6Facility overhead beyond IT equipmentOften reported as a site average
Carbon intensityg CO2e/kWhEmissions per unit of grid electricityVaries by region, hour and season
Operational emissionskg or t CO2eEmissions from running the computationExcludes hardware manufacturing
Water consumptionLitersCooling water and power plant water useRarely disclosed by providers
Core quantities in Green AI reporting

Researchers often also count embodied emissions, the carbon released when chips, servers and buildings are manufactured. Embodied carbon is difficult to allocate to a single training run, so many published figures cover operational emissions only and should be read as lower bounds.

Training versus inference footprint

A model consumes energy in two phases, and Green AI treats both as part of its lifecycle footprint. Training is a large, one-time cost that is easy to report, while inference is a small cost per request that is repeated millions or billions of times once a model is deployed.

For widely used systems, cumulative inference energy can exceed training energy over the service life. Reporting only the training run therefore understates the footprint of popular products and overstates it for research models that are rarely used.

PhaseEnergy patternTypical efficiency levers
Hyperparameter searchMany repeated training runsEarly stopping, smaller search spaces, Bayesian search
PretrainingSingle large burst over weeksEfficient architectures, mixed precision, low-carbon regions
Fine-tuningShort runs on smaller dataParameter-efficient methods such as adapters
InferenceContinuous, scales with usersQuantization, distillation, batching, caching
Retraining and updatesPeriodic repeated costIncremental updates instead of full retraining
Lifecycle phases and their energy profile

Hyperparameter search deserves special attention because it is often unreported. A paper may describe one final training run while the experiments behind it consumed ten to hundreds of times more compute.

Techniques that make models more efficient

Green AI techniques reduce the compute required per unit of useful output, either by shrinking the model, by doing less work per prediction or by running the same work on cleaner and more efficient infrastructure. Most of them involve a trade-off with accuracy, latency or engineering effort.

TechniqueHow it saves energyMain trade-off
Knowledge distillationA small student model imitates a large teacherSome accuracy loss on hard cases
QuantizationWeights stored in 8-bit or 4-bit precisionPossible quality drop at very low precision
Pruning and sparsityRemoves weights that contribute littleSpeedups need hardware that exploits sparsity
Mixture of expertsActivates only part of the network per inputHigher memory use and routing complexity
Carbon-aware schedulingRuns jobs when or where the grid is cleanerDelays and data residency constraints
Model reuseFine-tunes existing models instead of training anewInherits the base model’s biases and limits
Common efficiency techniques and their trade-offs

Choosing the smallest model that meets the requirement is frequently the most effective step. A classifier, a retrieval system or a compact language model can solve many tasks with a fraction of the energy that a general-purpose large model would use.

Green AI as an ethical and equity concern

Green AI belongs to AI ethics because environmental costs are external harms borne by people who neither build nor benefit from a model. Emissions contribute to climate change, and water withdrawals for cooling can compete with local needs in drought-prone regions.

A second concern is research equity. When leaderboard progress requires budgets that only a few companies can pay, universities and researchers in lower-income countries are excluded from frontier work. Reporting efficiency alongside accuracy lets smaller groups contribute results that matter.

Green AI also connects to neighboring topics such as AI environmental impact, which studies the sector-wide footprint, and to AI transparency, because honest disclosure of compute and emissions is a precondition for accountability. Critics warn that efficiency claims can become a form of ethics washing when they are not backed by measured data.

Limitations and common mistakes

Green AI metrics are useful but easy to misread, and several recurring errors weaken published claims. The table below links each mistake to its consequence and a better practice.

MistakeConsequenceBetter practice
Reporting FLOPs onlyHides hardware and grid differencesReport energy and location as well
Ignoring inferenceUnderstates deployed footprintEstimate energy per request and volume
Annual offset accountingMasks hourly fossil generationUse location-based or hourly matching
Omitting failed runsUnderestimates research costReport total project compute
Assuming efficiency cuts useRebound effect raises total demandTrack absolute consumption over time
Frequent errors in Green AI claims

The rebound effect, also called the Jevons paradox, is the most debated limitation. Cheaper inference can increase total usage so much that aggregate energy consumption rises even as each query becomes more efficient. Green AI therefore needs absolute targets, not only per-unit improvements.

In everyday language

Training and running AI takes large amounts of electricity, and electricity often comes from burning fossil fuels. Green AI is the idea that builders of AI should count that cost and try to keep it low, just as they try to make their systems accurate.

Think of fuel economy in cars. Two cars may reach the same destination, but one uses far less fuel. Green AI asks researchers to publish the fuel economy of their models, to choose a smaller model when it does the job, and to run heavy work where power is clean. The key point is simple: a smarter model is not automatically a better one if it wastes energy for a tiny gain.

Seen in the real world

A mid-sized online retailer uses a large general-purpose language model to sort about two million customer emails per month into twenty categories. The engineering team measures the service with an energy tracking library and finds that the model runs on several high-end GPUs around the clock in a region with a coal-heavy electricity grid.

The team labels 30,000 past emails with the large model, reviews a sample by hand and distills the task into a compact classifier with about 100 million parameters. They quantize it to 8-bit precision and move the service to a data center region whose grid carbon intensity is several times lower.

Accuracy on a held-out test set drops by less than one percentage point, while measured energy per thousand emails falls by more than 90 percent. The service now runs on a single smaller GPU, and the team adds energy and emissions figures to its internal model card so that later changes are compared on both accuracy and footprint.