Generated knowledge prompting

On this page 7

Generated knowledge prompting is a two-stage technique in which a language model first generates relevant background statements about a question and then answers the question conditioned on those statements. In prompting, it belongs to the family of methods that make a model externalize intermediate content before committing to a final answer, but unlike chain-of-thought it produces facts rather than reasoning steps.

The method was introduced in 2022 by researchers at the University of Washington and the Allen Institute for AI for commonsense reasoning benchmarks. Its central insight is that a model often holds useful knowledge it does not apply spontaneously, and that eliciting that knowledge explicitly can improve accuracy without retrieval infrastructure or fine-tuning.

How generated knowledge prompting works

The technique separates knowledge generation from knowledge integration. In the first stage, a few-shot prompt shows several example questions, each paired with a short factual statement, and then asks the model to produce similar statements for the new question. In the second stage, each generated statement is placed before the question, and a model answers it.

In the original formulation, the system samples many knowledge statements per question, often around 20, answers the question once with each statement and once without any, and selects the answer that receives the highest model confidence across all variants. This selection rule means that a single highly relevant statement can decide the outcome, while irrelevant statements are simply outvoted.

The table summarizes the stages and the main design choices at each one.

StageInputOutputKey design choice
Knowledge generationQuestion plus few-shot demonstrationsSeveral short factual statementsNumber of statements and sampling temperature
Knowledge integrationOne statement prepended to the questionCandidate answer with confidenceSame or different answering model
Answer selectionAll candidate answersFinal answerHighest confidence or majority vote
Stages of generated knowledge prompting

Design parameters that shape results

Results depend heavily on the quality and format of the demonstrations used to elicit knowledge. Demonstrations should show concise, declarative statements that are relevant to the question type but do not state the answer directly, because statements that leak answers teach the model to guess rather than to inform.

ParameterTypical settingEffect when increased
Number of statements5 to 20Better coverage, higher token cost
Sampling temperatureModerate, about 0.5 to 0.8More diverse but less reliable facts
Statement lengthOne or two sentencesMore detail, more room for error
Demonstrations3 to 5 examples per taskBetter format control, longer prompt
Parameters and their typical effects

With current chat models, practitioners often use a single-call variant: the prompt asks the model to list relevant facts and then answer in the same response. This is cheaper and simpler, but it loses the independent sampling and voting that made the original method robust.

Generated knowledge prompting is closest to step-back prompting and to retrieval-augmented generation, but the source and purpose of the added context differ. Step-back prompting asks for a general principle, retrieval brings in external documents, and generated knowledge draws specific facts from the model’s own parameters.

TechniqueWhat the model addsSource of new information
Generated knowledge promptingRelevant factual statementsModel parameters only
Chain-of-thought promptingStep-by-step reasoningModel parameters only
Step-back promptingA higher-level principle or conceptModel parameters only
Self-ask promptingFollow-up subquestions and answersModel parameters or a search tool
Retrieval-augmented generationRetrieved passagesExternal documents
Generated knowledge prompting and its neighbors

These techniques combine well. A common pattern is to generate knowledge first and then request chain-of-thought reasoning over it, or to merge generated statements with retrieved passages so that the model fills gaps that the document index does not cover. Self-consistency can also be applied by voting across answers produced from different statements.

When to use generated knowledge prompting

The technique is most useful for questions that require implicit world knowledge that a model possesses but tends to skip, such as commonsense reasoning, numerical common sense and everyday science. It helps less when the task depends on private, recent or highly specialized information that the model never learned.

SituationRecommended approachReason
Commonsense or general science questionsGenerated knowledge promptingKnowledge exists in the model but is not applied
Company policies or private dataRetrieval-augmented generationThe model cannot know the facts
Multi-step arithmetic or logicChain-of-thought promptingThe bottleneck is reasoning, not facts
Fast-changing news or pricesRetrieval or tool useParametric knowledge is outdated
Decision guide

Limitations and failure modes

The main risk is that generated knowledge can be wrong. A model that hallucinates a plausible but false statement may then answer confidently on that basis, so the technique can amplify errors as well as correct them. Sampling several statements and voting reduces, but does not remove, this risk.

  • Higher cost and latency: the full method requires many model calls per question.
  • Answer leakage: statements that simply restate a guessed answer add no knowledge and bias the result.
  • Diminishing returns: very large models already apply much of their knowledge, so gains tend to be smaller than on earlier models.
  • No verification: unlike retrieval, the added facts have no external source that can be checked or cited.
  • Prompt sensitivity: small changes to the demonstrations can change which facts the model produces.

For high-stakes use, generated statements should be treated as hypotheses to verify, for example by pairing the method with chain-of-verification or by checking key claims against a trusted source.

The idea in brief

Before answering a tricky question, a careful person often says what they already know about the subject. Generated knowledge prompting asks an AI model to do the same. First the model writes a few short facts about the topic. Then it uses those facts to answer.

The key takeaway is that the model often knows more than it uses. Asking it to recall facts first helps it apply that knowledge. The facts still come from the model itself, so they can be wrong and deserve a check.

A practical scenario

A team building a tutoring assistant for middle school science notices that the model sometimes gives wrong answers to commonsense questions such as whether a metal spoon or a wooden spoon gets hot faster in boiling soup. The team adds a knowledge generation step with four demonstrations that each pair a question with one or two neutral facts, such as how well common materials conduct heat.

For each student question, the assistant samples eight short statements, answers once with each statement and once without, and keeps the answer with the highest average confidence. On an internal test set of 300 questions, accuracy rises by several percentage points, and the generated facts are shown to students as a brief explanation.

The outcome is more accurate and better explained answers, at the cost of roughly nine model calls per question, which the team accepts because responses are cached for repeated questions.