Data modality
A data modality is a distinct form or channel in which information is represented, conveyed, or processed, such as text, images, or audio.
Effective Conversational AI
The communication mode through which a user interacts with a conversational system, such as text, voice, email, or push notification.
- How it is used
- Guides virtual-assistant flow design around how users communicate and what their situations demand.
- How deep it goes
- Defines four interaction modes and compares their strengths, limits, and design implications without giving implementation code.
- Examples
- Contrasts text and video printer guidance, then poor and improved voice interactions for flight status.
- What you can do after
- Evaluate/understand it by choosing a suitable interaction mode and spotting avoidable friction in an existing assistant flow.
Read how Effective Conversational AI explains data modality →
The Art of AI Product Development
A modality is a distinct type or form of data that an AI system can process or generate.
- How it is used
- Maps raw data to appropriate AI techniques during discovery, solution selection, and user-experience design.
- How deep it goes
- Defines modalities, explains numeric transformation and multimodal combination, but treats individual modalities only through brief task examples.
- Examples
- Uses sentiment analysis, translation, summarization, question answering, image classification, object detection, and image generation.
- What you can do after
- Evaluate a product problem’s data modality, select relevant AI techniques, and recognize when combining modalities may fit.
Read how The Art of AI Product Development explains data modality →
AI-Powered Search
A data modality is a distinct type or channel of information, such as text, images, audio, or behavioral signals.
- How it is used
- Extends search beyond text by using multiple information channels to improve matching, ranking, and relevance.
- How deep it goes
- Explains modality types, vector representation, and multimodal querying, with implementation examples for image and text search.
- Examples
- Uses DeLorean image search, superhero text queries, movie embeddings, metadata, behavior signals, and multimodal vector retrieval.
- What you can do after
- Apply it via multimodal embeddings and vector search to retrieve content from text, images, or combined queries.
Deep Learning with Python, Third Edition
A type of data such as text, images, or audio that can be represented as sequences for model processing.
- How it is used
- Frames non-text inputs and outputs as sequences so Transformer-based models can process multiple data types.
- How deep it goes
- Traces image patches through encoding, dimensional alignment, placeholder replacement, and soft-token insertion before deferring broader training details.
- Examples
- Uses image patches, Gemma image prompts, and generated descriptions of a snake wearing glasses in a chair and reading.
- What you can do after
- Apply it by framing a modality as a compatible sequence and combining that representation with text for Transformer processing.
Read how Deep Learning with Python, Third Edition explains data modality →
Related concepts
- multimodalused by · A multimodal model is a model that processes, generates, or relates information represented in two or more data modalities, such as text, images, audio, or video.
- user experiencecontrasts with · User experience is a person’s overall perceptions and responses resulting from interactions with a product, service, or system.
- voice interfaceincludes · A voice interface is an interaction interface through which users communicate with a system using spoken language, with responses that may be spoken or otherwise presented.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.