Datasheets for datasets

Also known as: dataset datasheet

On this page 7

Datasheets for datasets are standardized documents that describe the motivation, composition, collection process, preprocessing, uses, distribution and maintenance of a dataset used to train or evaluate machine learning models. In AI ethics, they serve as a transparency and accountability tool: they make the origins and limits of training data visible before a model inherits them. A single document is often called a dataset datasheet.

The framework was proposed in 2018 by Timnit Gebru and colleagues and published in Communications of the ACM in 2021. It borrows its name from the electronics industry, where every component ships with a datasheet listing operating characteristics, test results and recommended conditions of use.

Why datasheets for datasets exist

Datasheets for datasets exist because a model can only be as fair and reliable as its data, and data choices are usually invisible to the people who later reuse a dataset. Many well-known fairness failures trace back to undocumented sampling, labeling or filtering decisions rather than to the learning algorithm.

The framework addresses two audiences. Dataset creators are prompted to reflect on assumptions and risks during creation, which often reveals problems early. Dataset consumers receive the information needed to judge whether the data suits their task and to anticipate representation bias, historical bias or measurement bias that could propagate into predictions.

The seven sections of a datasheet

The original proposal organizes a datasheet into seven sections that follow the dataset lifecycle, each built around open questions rather than fixed fields. The table lists the sections and representative questions.

SectionRepresentative questionsEthical relevance
MotivationWhy was the dataset created, and who funded it?Reveals intended purpose and conflicts of interest
CompositionWhat do instances represent, and are subpopulations identified?Exposes gaps in demographic coverage
Collection processHow was data acquired, and did people consent?Documents consent and sampling choices
Preprocessing and labelingWhat cleaning or annotation was done, and by whom?Surfaces annotator bias and lost information
UsesWhat tasks suit the data, and which should be avoided?Prevents harmful or invalid reuse
DistributionUnder what license and to whom is it shared?Clarifies legal and access limits
MaintenanceWho maintains it, and how are errors corrected?Supports long-term accountability
Datasheet sections and the questions they answer

The questions are deliberately open-ended. The authors argue that a checklist of yes-or-no fields encourages box-ticking, while written answers force creators to state assumptions explicitly. A complete datasheet typically runs to several pages for a large dataset.

How datasheets compare with other documentation frameworks

Datasheets for datasets belong to a family of AI documentation artifacts that differ mainly in what they describe and when they are written. They focus on data, while related frameworks describe models, systems or the labeling workforce.

FrameworkObject documentedTypical authorMain emphasis
Datasheets for datasetsA datasetDataset creatorProvenance, composition and intended use
Model cardsA trained modelModel developerPerformance across groups and use limits
Data statementsLanguage datasetsNLP researchersSpeaker demographics and language variety
Dataset nutrition labelsA datasetData publisherShort visual summary of quality indicators
System cardsA deployed AI systemProduct teamInteractions among models, data and safeguards
Datasheets for datasets and related documentation

These artifacts work best in combination. A model card can reference the datasheets of its training and evaluation data, so a reviewer conducting an AI audit can trace a disparity in model performance back to a specific gap in the data.

Writing an effective datasheet

An effective datasheet is written during dataset creation, not reconstructed afterward, because many answers depend on decisions that are hard to recover later. Teams that start the document at the design stage tend to catch consent, licensing and sampling problems while they can still be fixed.

PracticeWhat it involvesBenefit
Start at project kickoffDraft motivation and intended uses firstShapes collection decisions
Quantify compositionReport counts per subgroup and labelMakes coverage gaps measurable
Name known limitationsState what the data cannot supportDiscourages invalid reuse
Record annotator detailsDescribe training, pay and agreement ratesExposes labeling bias and ghost work
Version the documentUpdate it with each dataset releaseKeeps errata and changes traceable
Practices that make a datasheet useful

Clear out-of-scope uses deserve particular care. A face dataset collected from public figures, for example, should state plainly that it is unsuitable for surveillance or identity verification of the general public.

Limitations and common mistakes

A datasheet is a disclosure tool, not a guarantee of fairness or quality. It can document a biased dataset accurately without making the dataset any less biased, and readers sometimes mistake the presence of documentation for evidence of responsible practice.

ProblemCauseRemedy
Vague or empty answersDatasheet treated as a compliance formRequire concrete numbers and examples
Outdated documentationNo owner after initial releaseAssign maintenance responsibility
Missing provenance for scraped dataWeb-scale collection without loggingRecord sources and filters at crawl time
Datasheet ignored by usersDocument hard to find or too longLink it from every download point
Use as ethics washingPublishing documentation without acting on itTie findings to release decisions
Frequent problems with dataset datasheets

Scale is a further challenge. For web-scale corpora with billions of documents, complete answers about consent or composition may be impossible, and honest datasheets acknowledge that uncertainty explicitly instead of hiding it. Regulators increasingly expect this kind of record: the EU AI Act, for instance, requires documentation of data governance for high-risk systems.

A plain-language summary

A datasheet for a dataset is like the ingredient label on food. It tells people what went into the data and where it came from. It also says who it suits and who should avoid it.

AI systems learn from data. If the data leaves out certain people or contains old prejudices, the system can repeat those problems. The datasheet makes such weak spots visible before harm happens. It does not fix the data by itself. It helps people decide whether to use it.

An example from practice

A hospital research group assembles 40,000 chest X-ray images to train a model that flags possible pneumonia. While writing the datasheet, the team fills in the composition section and counts images by patient age and sex. The counts show that fewer than 4 percent of images come from children and that all images come from two scanner models at one urban hospital.

The team records these facts, states that the dataset is not suitable for pediatric use or for clinics with different equipment, and documents that radiologist labels showed about 85 percent agreement. A partner hospital that later considers reusing the data reads the datasheet, recognizes the mismatch with its rural patient population, and funds additional local data collection before training its own model.