Artificial intelligence

Multimodal AI

Models that understand or generate several kinds of data at once, such as text, images, audio and video.

Browse A–Z

Terms

55 terms

  • Visual grounding

    Visual grounding is the multimodal AI task of locating the exact image region, box or mask, that a natural language phrase or expression refers to.

  • Visual instruction tuning

    Visual instruction tuning fine-tunes a vision-language model on image-grounded instructions and answers so it follows requests about pictures.

  • Visual question answering VQA

    Visual question answering (VQA) is the multimodal AI task in which a model reads an image and a natural-language question and returns a correct answer.

  • Visual tokens

    Visual tokens are the units into which multimodal AI models split an image or video so that a transformer can process it alongside text tokens.

  • Zero-shot image classification

    Zero-shot image classification assigns images to categories never seen in training by matching them against text descriptions in a shared embedding space.