How deeply does Introduction to Generative AI, Second Edition cover multimodal prompting?

Defines combined-modality inputs and illustrates chart analysis, without describing implementation details.

Multimodal prompting

As explained in Introduction to Generative AI, Second Edition

Multimodal prompting is the construction of model inputs that combine multiple data modalities, such as text, images, audio, or video.

Multimodal prompting combines text with images, charts, diagrams, audio, video, or sensor data in one model input. It extends prompt engineering beyond text-only instructions as systems support broader task workflows. section 7.6

Analyzing a chart

A user submits a chart and asks the model to analyze the figure and summarize its key trends. GPT-4o and Gemini are identified as systems that support combining text with visual inputs. section 7.6

The practice is becoming standard in some systems, while video, audio, and sensor data are being incorporated more systematically into task workflows. section 7.6

How Introduction to Generative AI, Second Edition covers it

Cover of Introduction to Generative AI, Second Edition

liveBook manning.com

Read Introduction to Generative AI, Second Edition on liveBook Get the book at manning.com

Related concepts

conversational aiuses · Conversational AI is artificial intelligence that enables software systems to interact with users through natural-language dialogue.
GPT-4ouses · GPT-4o is an OpenAI multimodal generative model in the GPT family that processes and generates text and other supported modalities.
Geminiuses · Gemini is Google's family of multimodal generative AI models and the conversational AI services built around them.
text-to-image modelalternative to · A text-to-image model is a generative machine-learning model that produces images conditioned on textual descriptions or prompts.
promptkind of · An input, typically containing instructions or context, supplied to an artificial-intelligence model to guide or elicit its output.

This explainer was generated by Manning's knowledge graph from the text of Introduction to Generative AI, Second Edition. The section links point to the parts of the book it draws on.