Multimodal hallucination

Also known as: object hallucination

On this page 7

Multimodal hallucination is a failure in which a model that reads images, audio or video produces text that is not supported by the perceptual input, such as describing objects that are absent, misreading attributes or inventing relations between things in a scene. In Multimodal AI, it is the counterpart of text-only hallucination, but the reference for truth is concrete: the pixels, the waveform or the frames the model was given.

The most studied form is object hallucination, where a vision-language model mentions an object that does not appear in the image. The problem matters because multimodal models are increasingly used for accessibility descriptions, document extraction, medical image triage and agent control, where a fluent but false statement about the input can propagate into real decisions.

Why multimodal models hallucinate

Multimodal hallucination arises because the language decoder dominates the visual evidence. In a typical architecture, a vision encoder turns an image into a few hundred visual tokens, a projector maps them into the embedding space of a large language model, and the language model generates the answer. The decoder carries far more parameters and far more training signal than the visual pathway, so strong language priors can override weak or ambiguous perceptual signals.

Several mechanisms contribute at once. Co-occurrence bias in training data teaches the model that a dining table usually comes with chairs and forks, so it reports them even when they are missing. Low-resolution encoding discards small text and fine detail, and the decoder fills the gap with plausible guesses. Attention drift during long generation causes later tokens to rely more on previously generated text than on the image, which is why hallucinated objects cluster near the end of long captions.

The following table summarizes the main causes and the stage of the pipeline where each one originates.

CausePipeline stageTypical symptom
Co-occurrence biasPretraining and instruction dataExpected objects reported when absent
Low input resolutionVision encoderMisread text, numbers and small items
Weak alignmentMultimodal projectorCorrect objects with wrong attributes
Language prior dominanceLanguage model decoderFluent answers that ignore the image
Attention driftLong-form generationErrors concentrated late in captions
Sycophantic tuningInstruction and preference tuningAgreement with false premises in prompts
Root causes of multimodal hallucination by pipeline stage

Types of multimodal hallucination

Researchers usually classify multimodal hallucination by what kind of claim is wrong, because each type needs different detection and mitigation. The categories below apply to images first, but they extend naturally to video and audio inputs.

TypeWhat is wrongExample
Object existenceAn absent entity is assertedA dog mentioned in an empty park photo
AttributeColor, count, size or state is wrongThree people described when there are two
RelationSpatial or causal link is inventedA cup said to be on the laptop
Text and OCRVisible text is misread or inventedA wrong total on a scanned receipt
TemporalEvent order or timing is wrong in videoA goal placed before the foul
Cross-modal audioSounds or speakers are inventedApplause reported in a silent clip
Common categories with examples

A related failure is premise acceptance: when a user asks “What color is the umbrella?” about an image without an umbrella, many models answer with a color instead of rejecting the premise. This behavior links hallucination to instruction-following bias and is a frequent target of adversarial benchmarks.

How multimodal hallucination is measured

Measurement relies on grounded reference annotations that state what is actually present in each input. Early metrics compared generated captions against object labels from annotated image datasets; later benchmarks use targeted yes-or-no probing, free-form answers judged by stronger models, and human review for subtle attribute and relation errors.

The table below compares the main evaluation approaches and their limits.

ApproachHow it worksMain limitation
CHAIR metricCounts caption objects absent from ground-truth labelsLimited to a fixed object vocabulary
POPE probingAsks yes-or-no questions about object presenceMeasures existence only, not attributes
Model-judged benchmarksA stronger model grades free-form answersJudge model may hallucinate itself
Human evaluationAnnotators verify each claim against the inputSlow, costly and hard to scale
Task accuracy checksCompares extracted fields with source documentsSpecific to one domain or format
Evaluation approaches for hallucination in vision-language models

Reported hallucination rates vary widely, from a few percent on simple existence probes to substantially higher rates in long, detailed captions, and they depend on prompt length, image complexity and decoding settings. Results from one benchmark therefore do not transfer directly to a production workload, which needs its own test set.

Mitigation techniques

No single technique eliminates multimodal hallucination; effective systems combine training-time and inference-time measures. Training-side methods improve how much the decoder relies on the image, while inference-side methods constrain or verify the output.

  • Higher-resolution vision encoders and image tiling preserve small text and fine detail that would otherwise be guessed.
  • Curated instruction data with negative examples teaches the model to say that an object is not present.
  • Preference optimization on grounded feedback penalizes answers that mention unsupported content.
  • Contrastive decoding compares outputs with and without a distorted image and down-weights tokens driven by language priors alone.
  • Visual grounding and citation of regions require the model to point to bounding boxes that support each claim.
  • Retrieval and tool verification, such as running a dedicated OCR engine or object detector, cross-check critical facts.

The right combination depends on the deployment. The decision table below maps common situations to a recommended strategy.

SituationRecommended measureReason
Invoice or form extractionOCR cross-check and schema validationNumeric errors are costly and testable
Accessibility alt textShort captions with low temperatureShorter outputs reduce late-stage drift
Medical or safety imageryHuman review of every outputErrors carry high consequences
GUI agentsGrounded element coordinatesActions must target real interface elements
Video summarizationFrame sampling with timestampsTimestamps expose temporal errors
Choosing a mitigation strategy

Common misconceptions and limits

A frequent misconception is that larger models stop hallucinating. Scale usually reduces object existence errors on common scenes, but attribute, counting and relation errors persist, and larger models often sound more confident when wrong. Confidence scores from the decoder are poorly calibrated for visual claims, so a high token probability is not evidence that the object exists.

Another misconception is that hallucination is purely a model defect. Ambiguous inputs, such as blurred photos, cropped charts or noisy audio, leave room for several valid readings. A well-designed system therefore treats uncertainty as a first-class output and lets the model answer “not visible” or “cannot determine” instead of forcing a definite claim.

In plain words

A multimodal model looks at a picture, listens to a recording or watches a video and then describes what it found. Sometimes it describes things that are simply not there. That mistake is called multimodal hallucination.

It happens because the model has seen millions of similar scenes and expects certain things to appear together. A kitchen photo makes it expect a fridge, so it may mention one even when the kitchen has none. It is a little like a witness who fills memory gaps with what usually happens. The key lesson is simple: a fluent description is not proof, so important details should always be checked against the original image or sound.

A real-world example

An accounts payable team at a mid-sized logistics company uses a vision-language model to read scanned supplier invoices and fill in a payment system. During a pilot of 2,000 invoices, reviewers find that most fields are correct, but on faded scans the model sometimes invents a plausible invoice total or copies a due date from a different line, because small digits were lost when the page was downscaled.

The team changes the pipeline in three ways: pages are tiled at a higher resolution, a separate OCR engine extracts every number, and any field where the model and the OCR output disagree is routed to a person. They also require the line items to sum to the stated total. After the change, mismatched totals are caught before payment instead of after, and manual review drops to a small share of invoices that genuinely need human judgment.