A multimedia prompt combines a text instruction with media, such as an image, so a generative AI system can analyze both inputs. In a web application, the feature spans an upload interface, request transformation, and a vision-capable model. section 3.4.2
A practical implementation follows this flow:
- Accept an image upload and retain its preview data in the chat component. section 3.4.2
- Attach the image URL to the request alongside the user's text. section 3.4.2
- Convert the final message into text and image parts before passing it to streamText. section 3.4.2
- Send the enriched messages to a Gemini vision model and stream its response. section 3.4.2
Listing 3.14 processIncomingMessages function in route.js
async function processRequestMessages(req) {
const { messages, data } = await req.json(); #1
if (!data?.imageUrl) return messages; #2
const initialMessages = messages.slice(0, -1);
const lastMessage = messages[messages.length - 1]; #3
return [
...initialMessages,
{
... lastMessage,
content: [ #4
{ type: "text", text: lastMessage.content },
{
type: "image",
image: data.imageUrl,
},
],
},
];
};
Use the route transformation to see how a text-only message becomes a multimedia prompt with separate text and image parts. section 3.4.2
