Single-modal model
A single-modal model is a machine-learning model that processes or generates information in one data modality.
Hugging Face in Action
A single-modal model is a machine learning model that processes data from one modality, such as only images or only text.
- How it is used
- Uses one data modality for focused tasks such as text processing, image analysis, audio processing, or numerical prediction.
- How deep it goes
- Defines the model and demonstrates image detection through preprocessing, inference, confidence filtering, and bounding-box visualization.
- Examples
- Demonstrates GPT, BERT, ResNet, and DETR, including dog-and-cat detection with labels, scores, and bounding boxes.
- What you can do after
- Apply it via Hugging Face Transformers and PyTorch to detect objects and visualize their locations in an image.
Read how Hugging Face in Action explains single-modal model →
Build a Text-to-Image Generator (from Scratch)
A unimodal model processes data within a single modality, such as text-only or image-only data.
- How it is used
- Distinguishes models that process one data modality from multimodal systems that connect different modalities.
- How deep it goes
- Defines same-modality processing and contrasts it with multimodal processing, stopping short of detailed architecture or implementation.
- Examples
- Uses GPT-3 for text-to-text processing and image generation from random noise as unimodal examples.
- What you can do after
- Evaluate whether a model is unimodal by checking whether its inputs and outputs remain within one data modality.
Read how Build a Text-to-Image Generator (from Scratch) explains single-modal model →
Related concepts
- language modelkind of · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
- multimodalcontrasts with · A multimodal model is a model that processes, generates, or relates information represented in two or more data modalities, such as text, images, audio, or video.
- GPT modelincludes · A GPT model is a generative pretrained Transformer language model that produces text autoregressively by predicting successive tokens from preceding context.
- image generation modelincludes · An image generation model is a machine-learning model that creates, edits, or transforms images from learned patterns and conditioning inputs.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.