Ground truth is the set of verified reference answers against which a model’s outputs are trained and judged: the correct label, value, boundary or response for each example in a dataset. Artificial intelligence systems built with supervised learning cannot learn or be evaluated without it, because every loss function and every accuracy metric compares a prediction with a value that is assumed to be correct.
The phrase comes from remote sensing, where satellite images were checked against measurements taken on the ground. In modern AI practice it rarely means absolute truth. It means the best available reference, usually produced by human annotators, instruments or records, and its quality sets an upper limit on measured performance.
How ground truth enters the learning loop
Ground truth plays two distinct roles: it is the target that training minimizes error against, and it is the yardstick that evaluation reports scores against. During training, the loss function compares each prediction with its reference label and produces a gradient; during evaluation, metrics such as accuracy, F1 score or mean absolute error summarize the same comparison on held-out data.
Because the reference is treated as correct, every error in it is learned or rewarded. A mislabeled training image pushes the weights in the wrong direction, and a mislabeled test image can make a correct prediction count as a mistake. For this reason practitioners separate training labels, which can tolerate some noise, from the evaluation set, which deserves the most careful verification.
The table below shows what ground truth looks like for common task types.
| Task type | Form of ground truth | Typical metric |
|---|---|---|
| Image classification | One class label per image | Accuracy, top-5 accuracy |
| Object detection | Bounding boxes with class labels | Mean average precision |
| Semantic segmentation | Class label per pixel | Intersection over union |
| Regression | Measured numeric value | Mean absolute error, RMSE |
| Speech recognition | Verified transcript | Word error rate |
| Language generation | One or more reference answers or rubrics | Exact match, human preference |
Where ground truth comes from
Ground truth is produced by a labeling process, and the source determines both its cost and its reliability. The main options are expert annotation, crowdsourced annotation, instrument measurement, transactional records and outcomes observed later in time.
Each source carries its own characteristic weakness, summarized below.
| Source | Strength | Main weakness |
|---|---|---|
| Domain experts | High accuracy on difficult cases | Slow, expensive, small volume |
| Crowd annotators | Large volume at low cost | Inconsistent quality, needs aggregation |
| Sensors and instruments | Objective numeric values | Calibration drift and measurement error |
| Business records | Already available at scale | Reflect past decisions and their biases |
| Delayed outcomes | Close to the real target | Arrive weeks or years after prediction |
| Model-assisted labels | Fast pre-annotation | Can copy the model’s own errors |
Mature teams combine sources: a model proposes labels, annotators correct them, and adjudication by a senior reviewer settles disagreements. Written annotation guidelines with edge-case examples matter as much as the people applying them, because ambiguous instructions produce labels that differ for reasons unrelated to the data.
Measuring the quality of ground truth
The quality of ground truth is measured by inter-annotator agreement, the degree to which independent labelers assign the same answer to the same item. Common statistics are Cohen’s kappa for two annotators, Fleiss’ kappa for several and Krippendorff’s alpha for mixed data types; all correct raw agreement for the agreement expected by chance.
Agreement levels set a realistic ceiling. If two radiologists agree on only 80 percent of cases, a model reported at 95 percent accuracy against one of them is probably fitting that annotator’s habits rather than the underlying condition. Studies of widely used benchmarks have found label error rates of a few percent in test sets, which is enough to reorder closely ranked models.
| Statistic | Use case | Rough interpretation |
|---|---|---|
| Percent agreement | Quick check, any number of raters | Ignores chance agreement |
| Cohen’s kappa | Two raters, categorical labels | Above 0.8 is usually considered strong |
| Fleiss’ kappa | Three or more raters | Same scale as Cohen’s kappa |
| Krippendorff’s alpha | Missing ratings, ordinal or interval data | Above 0.8 is a common target |
Ground truth compared with related concepts
Ground truth is not the same as training data, although the two are often discussed together. Training data is the full collection of examples fed to a model; ground truth is the reference portion of that collection, the answers attached to the inputs. Unsupervised learning uses training data with no ground truth at all, while self-supervised learning derives labels from the data itself, for example by hiding a word and asking the model to predict it.
| Concept | What it refers to | Relation to ground truth |
|---|---|---|
| Label | A single annotation on one example | One unit of ground truth |
| Training data | All inputs used to fit a model | Contains ground truth in supervised learning |
| Gold standard | The best accepted reference method | Often the source of ground truth |
| Pseudo-label | A label predicted by a model | A substitute, not verified ground truth |
| Silver standard | Labels produced automatically at scale | Cheaper but noisier ground truth |
Limitations and common mistakes
The most persistent misconception is that ground truth is objective by definition. In many tasks, such as toxicity rating, sentiment or medical triage, reasonable experts disagree, and forcing a single label hides genuine ambiguity that the model will later meet in production.
- Label leakage: the reference is derived from information that will not exist at prediction time, which inflates offline scores.
- Proxy targets: an arrest record stands in for crime, or a click stands in for satisfaction, and the model learns the proxy rather than the goal.
- Stale references: definitions or real-world conditions change, but old labels remain, a form of concept drift in the reference itself.
- Single-annotator test sets: evaluation depends on one person’s judgment, with no way to estimate label noise.
- Benchmark contamination: test items and their answers appear in training data, so scores no longer measure generalization.
Remedies include multiple annotations per item, explicit uncertainty labels, periodic re-annotation and keeping a small, carefully adjudicated gold evaluation set that is never used for training. This data-centric view treats label quality as an engineering asset rather than a fixed input.
A plain-language summary
Ground truth is the answer key. When a computer system learns from examples, someone first has to say what the right answer is for each one: this photo shows a cat, this email is spam, this house sold for a certain price. The system practices against that answer key and is later graded with it.
The catch is that an answer key can contain mistakes. If the teacher’s key is wrong, a student who answers correctly gets marked down, and a student who copies the key learns the errors. That is why careful teams check their answer keys with several people before trusting the grades.
An example from practice
A hospital team builds a model that flags pneumonia on chest X-rays. Their first ground truth comes from the original radiology reports, which are extracted automatically from free text. The model scores 92 percent accuracy on a test set labeled the same way.
Before deployment, the team asks three radiologists to label 1,000 test images independently. Agreement between them reaches a Fleiss’ kappa of about 0.7, and about 6 percent of the report-derived labels turn out to be wrong. Measured against the adjudicated expert labels, the model’s accuracy falls to 86 percent, and most of its remaining errors fall on cases where the radiologists themselves disagreed.
The team rebuilds its evaluation set from adjudicated labels, adds an uncertain category and retrains. The outcome is a lower but trustworthy score and a model whose behavior on difficult images is now understood before it reaches clinicians.