Transformers
Hugging Face Transformers is an open-source software library for loading, using, and training pretrained transformer models across machine-learning tasks.
Natural Language Processing in Action, Second Edition
Hugging Face Transformers is an open-source library providing pretrained transformer models and utilities for natural-language and other machine-learning tasks.
- How it is used
- Supports pretrained-model inference, text generation, semantic representations, and task-specific fine-tuning.
- How deep it goes
- Demonstrates installation, model loading, tokenization, generation, architecture inspection, and fine-tuning, but defers exhaustive compute planning.
- Examples
- Uses BERT architecture inspection, GPT-2 prompt generation, probability selection, nucleus sampling, and manuscript-line fine-tuning.
- What you can do after
- Builds inference and fine-tuning pipelines that load pretrained models, generate text, inspect outputs, and train on custom datasets.
Read how Natural Language Processing in Action, Second Edition explains transformers →
Retrieval Augmented Generation, The Foundational Ideas
Hugging Face Transformers is a library providing pretrained transformer models and tools for natural-language and other machine-learning tasks.
- How it is used
- Supports a FiD retrieval-augmented question-answering implementation using pretrained encoder–decoder models.
- How deep it goes
- Works through model loading, passage encoding, fused generation, and a fine-tuning step; FiD data flow remains manual.
- Examples
- Uses Flan-T5 code to answer a multi-passage question about a multi-GPU training decision.
- What you can do after
- Apply it via Hugging Face Transformers to load Flan-T5 and assemble a basic FiD answer-generation path.
Read how Retrieval Augmented Generation, The Foundational Ideas explains transformers →
Hugging Face in Action
Transformers is an open-source library for working with pretrained machine-learning models, especially models for natural-language processing and computer vision.
- How it is used
- Applies pretrained transformer models to practical NLP, vision, audio, and reinforcement-learning workflows.
- How deep it goes
- Explains installation, pretrained models, pipelines, and task categories while stopping short of detailed transformer internals.
- Examples
- Demonstrates sentiment classification with DistilBERT and object detection with a hosted DETR model.
- What you can do after
- Apply it via the pipeline API to run pretrained models for common language and vision tasks.
Domain-Specific Small Language Models
The Hugging Face library for working with pretrained transformer models, tokenizers, pipelines, and related training and inference utilities.
- How it is used
- Supports pretrained-model loading, fine-tuning, and Python inference, including an on-device deployment option.
- How deep it goes
- Works through tokenization, Trainer configuration, hyperparameter search, and inference steps; does not detail Transformers library internals.
- Examples
- Uses GPT-2 fine-tuning, DistilBERT question answering, GPT-Neo generation, and BERT classification on Android.
- What you can do after
- Apply it via Transformers Trainer to fine-tune a pretrained model, or operate Python inference after checking device resource limits.
Read how Domain-Specific Small Language Models explains transformers →
Learn Generative AI with PyTorch
A software library for working with pretrained transformer models and tokenizers from Hugging Face.
- How it is used
- Supports multilingual tokenization and imports pretrained transformer parameters into custom model implementations.
- How deep it goes
- Explains installation, tokenizer calls, vocabulary indexing, state-dictionary extraction, and selected weight transpositions.
- Examples
- Uses English-French phrases, XLM token sequences, vocabulary indexes, and GPT-2XL weight-shape comparisons.
- What you can do after
- Apply it via Hugging Face APIs to tokenize multilingual text or load pretrained GPT-2XL weights.
Read how Learn Generative AI with PyTorch explains transformers →
Data Analysis with LLMs
Hugging Face Transformers is an open-source library providing pretrained Transformer models and tools for natural-language, vision, and multimodal machine-learning tasks.
- How it is used
- Supports browsing, selecting, downloading, and running pretrained Transformer models for specialized machine-learning tasks.
- How deep it goes
- Demonstrates installation, pipeline creation, inference, and output inspection, while stopping short of comprehensive library features or model training.
- Examples
- Uses task filters, a Salesforce BLIP image-captioning model, and Cardiff University’s RoBERTa sentiment-classification model.
- What you can do after
- Apply it via Python to download and run a pretrained model through a task pipeline.
Transformers in Action
Transformers is an open-source software library for using pretrained transformer models across machine-learning tasks.
- How it is used
- Applies pretrained transformer models to NLP tasks and emerging computer vision and audio recognition applications.
- How deep it goes
- Explains pretrained-model use, fine-tuning, learning modes, and GPU requirements without detailing model internals or implementation code.
- Examples
- Uses translation, text generation, text classification, sentiment analysis, labeled sentence pairs, Google Colab, and GPU computation as examples.
- What you can do after
- Apply it via Hugging Face to adapt pretrained models for specific tasks while reducing training time and computational resources.
Also covered in
- Crack Any Codebase with AIIn early access (MEAP)
- Deep Learning with PyTorch, Second Edition
- Deep Learning with JAX
- Starting Data Analytics with Generative AI and Python
- Context EngineeringIn early access (MEAP)
- Time Series Forecasting Using Foundation Models
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- classifierA classifier is a predictive model that assigns an input to one or more discrete classes or categories.
- transformer architectureA transformer is a neural-network architecture that uses attention-based interactions and position-wise transformations to model relationships among elements of sequences or other structured inputs.
- foundation modelA foundation model is a broadly trained machine-learning model whose learned capabilities can be reused or adapted for a range of downstream tasks.
Related concepts
- eager evaluationuses · Eager evaluation is an evaluation strategy in which expressions are computed as they are encountered rather than deferred until their values are needed.
- T5implements · T5 is a family of encoder-decoder Transformer models that frames natural-language tasks as transformations from input text to output text.
- PyTorchuses · PyTorch is an open-source machine-learning framework that provides tensor computation, automatic differentiation, and tools for developing and training neural networks.
- Hugging Faceused by · Hugging Face is a platform and ecosystem for hosting, discovering, sharing, and using machine-learning models, datasets, applications, and related tools.
Questions about transformers
What is transformers?
Hugging Face Transformers is an open-source software library for loading, using, and training pretrained transformer models across machine-learning tasks.
Which Manning books explain transformers?
Natural Language Processing in Action, Second Edition, Retrieval Augmented Generation, The Foundational Ideas, Hugging Face in Action, Domain-Specific Small Language Models, Learn Generative AI with PyTorch, Data Analysis with LLMs and Transformers in Action.
What should I know before learning transformers?
Manning books that teach transformers usually introduce language model, classifier, transformer architecture and foundation model first.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.






