How images and audio become tokens
A model can only process tokens, so every non-text input is converted first, and each vendor publishes its own conversion. Anthropic states that Claude views images in 28×28-pixel patches, each one a visual token, so an image costs ⌈width / 28⌉ × ⌈height / 28⌉ tokens — 1,296 for a 1000×1000 picture. Google counts a Gemini image of 384 pixels or less on both sides as 258 tokens, tiles larger images into 768×768 pieces at 258 tokens each, and converts audio at 32 tokens per second.
Size is capped, which caps cost. Claude 4.7 and later models accept images up to a 2,576-pixel long edge and 4,784 visual tokens; other Claude models downscale to a 1,568-pixel edge and 1,568 tokens. Anthropic notes that the high-resolution tier can use roughly three times the visual tokens of the standard tier for the same image.
What multimodal input costs
Once converted, visual and audio tokens are billed as input at the model's rate, and they occupy the context window like any other input. They are also resent: in a multi-turn conversation, every earlier image is part of the history sent with each new message. Anthropic suggests uploading images once through its Files API and referencing them by ID to keep request payloads small, but the visual tokens are still counted on every turn they remain in the conversation.
The cheapest lever is resolution. If a task does not need fine detail — classifying a photo, reading a large chart — downscaling before upload cuts the token count directly. Anthropic recommends exactly that when the extra fidelity of high resolution is not needed.
Which models are multimodal
Image input is now standard on frontier models; audio and video input are rarer. The panel below counts the current models in our index by the input types their vendors list. Output is a separate question: most multimodal models still answer only in text, while image generation, speech and video generation are usually sold as their own models with their own units — per image, per second or per character.
In our indexInput types in our index
Counted at build from our price index: the input types each vendor lists for the model among the 167 current public models we price, from 33 vendors. Each rate comes from the vendor's own price page, read between 2026-07-11 and 2026-10-09. How we collect prices.