liveBook · Manning

Data modality

A data modality is a distinct form or channel in which information is represented or conveyed, such as text, images, audio, or structured data.

How 5 Manning books explain it

Effective Conversational AI

The communication mode through which a user interacts with a conversational system, such as text, voice, email, or push notification.

How it is used
Guides virtual-assistant flow design around how users communicate and what their situations demand.
How deep it goes
Defines four interaction modes and compares their strengths, limits, and design implications without giving implementation code.
Examples
Contrasts text and video printer guidance, then poor and improved voice interactions for flight status.
What you can do after
Evaluate/understand it by choosing a suitable interaction mode and spotting avoidable friction in an existing assistant flow.

Read how Effective Conversational AI explains data modality →

The Art of AI Product Development

A modality is a distinct type or form of data that an AI system can process or generate.

How it is used
Maps raw data to appropriate AI techniques during discovery, solution selection, and user-experience design.
How deep it goes
Defines modalities, explains numeric transformation and multimodal combination, but treats individual modalities only through brief task examples.
Examples
Uses sentiment analysis, translation, summarization, question answering, image classification, object detection, and image generation.
What you can do after
Evaluate a product problem’s data modality, select relevant AI techniques, and recognize when combining modalities may fit.

Read how The Art of AI Product Development explains data modality →

AI-Powered Search

A data modality is a distinct type or channel of information, such as text, images, audio, or behavioral signals.

How it is used
Extends search beyond text by using multiple information channels to improve matching, ranking, and relevance.
How deep it goes
Explains modality types, vector representation, and multimodal querying, with implementation examples for image and text search.
Examples
Uses DeLorean image search, superhero text queries, movie embeddings, metadata, behavior signals, and multimodal vector retrieval.
What you can do after
Apply it via multimodal embeddings and vector search to retrieve content from text, images, or combined queries.

Read how AI-Powered Search explains data modality →

Deep Learning with Python, Third Edition

A type of data such as text, images, or audio that can be represented as sequences for model processing.

How it is used
Frames non-text inputs and outputs as sequences so Transformer-based models can process multiple data types.
How deep it goes
Traces image patches through encoding, dimensional alignment, placeholder replacement, and soft-token insertion before deferring broader training details.
Examples
Uses image patches, Gemma image prompts, and generated descriptions of a snake wearing glasses in a chair and reading.
What you can do after
Apply it by framing a modality as a compatible sequence and combining that representation with text for Transformer processing.

Read how Deep Learning with Python, Third Edition explains data modality →

Fight Fraud with Machine Learning

Data modality is the form in which data is represented, such as structured, unstructured, or semi-structured data.

How it is used
Guides fraud analysis by identifying available evidence, suitable detection approaches, and deployment requirements.
How deep it goes
Classifies structured, unstructured, and semi-structured data, then applies the distinction to image-based document forgery.
Examples
Uses transaction fields, text and images, combined phishing evidence, and forged digital medical bills as examples.
What you can do after
Evaluate fraud data by representation type and select detection approaches aligned with its modality.

Read how Fight Fraud with Machine Learning explains data modality →

Also covered in

What to know first

Related concepts

Questions about data modality

What is data modality?

A data modality is a distinct form or channel in which information is represented or conveyed, such as text, images, audio, or structured data.

Which Manning books explain data modality?

Effective Conversational AI, The Art of AI Product Development, AI-Powered Search, Deep Learning with Python, Third Edition and Fight Fraud with Machine Learning.

What should I know before learning data modality?

Manning books that teach data modality usually introduce language model, embedding, digital image and transformer architecture first.

What is the difference between data modality and user experience?

A data modality is a distinct form or channel in which information is represented or conveyed, such as text, images, audio, or structured data. User experience is a person’s overall perceptions and responses resulting from interacting with or using a product, service, or system.

What kinds of data modality are there?

Kinds of data modality include visual modality, text interaction and voice interface.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.