Models that understand or generate several kinds of data at once, such as text, images, audio and video.
55 terms
Visual grounding is the multimodal AI task of locating the exact image region, box or mask, that a natural language phrase or expression refers to.
Visual instruction tuning fine-tunes a vision-language model on image-grounded instructions and answers so it follows requests about pictures.
Visual question answering (VQA) is the multimodal AI task in which a model reads an image and a natural-language question and returns a correct answer.
Visual tokens are the units into which multimodal AI models split an image or video so that a transformer can process it alongside text tokens.
Zero-shot image classification assigns images to categories never seen in training by matching them against text descriptions in a shared embedding space.