Cloze prompt

Also known as: cloze-style prompt

On this page 7

A cloze prompt is a prompt written as a fill-in-the-blank text, in which the task is expressed as a sentence with a missing word or span and the language model answers by predicting what belongs in the gap. In prompting for artificial intelligence, the cloze-style prompt turns classification, fact retrieval and short extraction into the same operation the model learned during pretraining, which makes outputs short, constrained and easy to score.

The name comes from the cloze test of reading comprehension, introduced in 1953, in which readers restore deleted words. The technique matters most when a task has a small, known answer space and when an application needs a single token or phrase rather than free-form prose.

How a cloze prompt turns a task into gap filling

A cloze prompt works by rewriting the input and the question as one sentence whose completion is the answer. A sentiment task such as “Is this review positive?” becomes “The battery died after a week. Overall, the product was [blank].” The model then assigns probabilities to candidate fillers, and the most likely filler is read as the label.

Three components define the prompt. The pattern is the template that places the input next to the gap. The gap marker depends on the architecture: a [MASK] token for masked models such as BERT, a sentinel token such as <extra_id_0> for T5, or simply the end of the text for causal models. The verbalizer maps each class to a word, for example “great” for positive and “terrible” for negative.

The following table shows how the gap is expressed in the main model families.

Model familyGap positionHow the gap is markedTypical use
Masked language modelsAnywhere in the textA mask tokenKnowledge probing, classification
Span corruption modelsAnywhere, multi-token spansNumbered sentinel tokensShort span filling
Causal language modelsEnd of the textText ends where the answer startsCompletion and label prediction
Fill-in-the-middle modelsMiddle of a documentPrefix, suffix and middle markersCode infilling
Gap markers by model architecture

Pattern and verbalizer design

The quality of a cloze prompt depends mostly on pattern wording and verbalizer choice, and small changes can move accuracy by many points. This prompt sensitivity is well documented for few-shot classification: two paraphrased patterns with the same meaning can produce noticeably different label distributions.

Good verbalizer words are frequent, unambiguous and single-token in the model vocabulary. A rare word split into several subword tokens is harder to score fairly, because its probability is a product of several conditional probabilities and is systematically lower than that of a short common word.

Design choiceBetter optionWeaker optionWhy it matters
Verbalizer lengthOne common tokenRare multi-token wordAvoids length bias in scores
Gap placementNatural sentence positionAwkward or repeated gapMatches pretraining text
Label setMutually exclusive wordsOverlapping synonymsReduces ambiguous predictions
Context framingShort, factual lead-inLong instructionsKeeps attention on the gap
Number of patternsSeveral, averagedA single wordingLowers variance across phrasings
Design choices and their effect

Practitioners therefore test several patterns and verbalizers on a labeled development set and often ensemble the predictions, which connects cloze prompting to prompt ensembling and prompt evaluation.

Cloze prompts compared with prefix and instruction prompts

A cloze prompt differs from other prompt formats in where the answer goes and how much freedom the model has. A prefix prompt asks the model to continue text freely, while an instruction prompt describes the task in natural language and expects a formatted reply. The cloze format sits at the most constrained end.

FormatAnswer locationOutput freedomBest suited to
Cloze promptInside or at the end of a sentenceOne word or short spanLabels, facts, scoring
Prefix promptAfter the given textOpen-ended continuationGeneration, drafting
Instruction promptIn a separate replyFlexible, follows format rulesChat assistants, complex tasks
Few-shot promptAfter solved examplesMirrors the examplesTasks learned from demonstrations
Prompt formats side by side

Formats can be combined. A few-shot cloze prompt shows solved sentences with the gap filled and then presents a new sentence with the gap open. With instruction-tuned chat models, the cloze idea often survives as response prefilling, where the reply is started with “Answer:” so that the next token is the label.

Where cloze prompting is used in practice

Cloze prompts are most useful when the system needs a measurable answer rather than prose. Their main applications fall into four groups.

  • Knowledge probing: research benchmarks such as LAMA test what facts a model stores by asking it to complete statements like “The capital of Canada is [blank].”
  • Zero-shot and few-shot classification: comparing verbalizer probabilities gives a label without training a classifier head.
  • Semi-supervised training: Pattern-Exploiting Training (PET) uses cloze patterns to fine-tune small models on very few labeled examples and then labels unlabeled data.
  • Code and document infilling: fill-in-the-middle formats let a coding assistant complete a function body between existing lines.

The next table helps decide when a cloze prompt is the right tool.

SituationRecommended approachReason
Fixed label set, access to log-probabilitiesCloze prompt with verbalizerDirect, comparable scores
Need for a single factCloze statementShort answer, easy to check
Multi-step reasoning requiredInstruction or chain-of-thought promptA single gap leaves no room to reason
Long structured outputOutput format instructionCloze limits answer length
When to choose a cloze prompt

Limitations and common mistakes

The main weakness of a cloze prompt is that probability is not the same as truth. A model can assign high likelihood to a fluent but wrong filler, especially for facts that appear rarely in training data, so probing results show what the model finds plausible, not what it reliably knows.

A second weakness is surface-form competition: several valid words can split the probability mass of one correct answer, so “Ottawa”, “ottawa” and “the city of Ottawa” compete with each other. Calibration methods that normalize scores against a content-free input reduce this bias but do not remove it.

  • Ignoring tokenization, so that verbalizer words of different token lengths are compared unfairly.
  • Using hosted chat models without log-probabilities, where the cloze must be answered as text and parsing becomes fragile.
  • Trusting a single pattern instead of checking several phrasings on labeled data.
  • Leaking the answer through examples or wording that make one filler obvious for the wrong reason.

Explained without jargon

A cloze prompt is a fill-in-the-blank question for an AI model. Instead of asking “What is the mood of this review?”, the prompt writes the review and then adds “Overall, it was ___.” The model fills the blank with the word it finds most fitting, such as “great” or “terrible”.

This works because language models learned by guessing missing or next words in huge amounts of text. A gap is the question they understand best. The key takeaway is simple: a cloze prompt gives short, predictable answers that are easy to count and compare, but the model can still fill the blank confidently with the wrong word.

A typical use case

A support team at a mid-sized software company receives about 4,000 tickets a week and wants to route them into three queues: billing, technical fault and account access. Instead of training a classifier, the engineer writes a cloze pattern for an open-weight model: the ticket text followed by “This message is about [blank].” The verbalizer maps “payment” to billing, “error” to technical fault and “login” to account access, all single tokens.

On a development set of 300 hand-labeled tickets, the engineer tests four phrasings of the pattern and averages their scores. The ensemble reaches about 88 percent routing accuracy, several points above the best single phrasing, and tickets whose top two scores are close go to a human. The result is automatic routing within a day of work, with a clear confidence signal for the uncertain cases.