liveBook · Manning

Transformer architecture

Also called transformer model, the transformer

A transformer is a neural-network architecture that uses attention-based interactions and position-wise transformations to model relationships among elements of sequences or other structured inputs.

How 40 Manning books explain it

Learn Generative AI with PyTorch

The internal design and structural organization of a Transformer model built around decoder blocks or self-attention components.

How it is used
Uses Transformer architectures for sequence prediction, especially translation and text generation.
How deep it goes
Works through encoder and decoder layers, attention calculations, tensor shapes, and a line-by-line implementation.
Examples
Uses English-to-French translation and a scaled-down GPT-style model to demonstrate architecture choices.
What you can do after
Builds an encoder-decoder Transformer from scratch and trains it for English-to-French translation.

Read how Learn Generative AI with PyTorch explains transformer architecture →

Natural Language Processing in Action, Second Edition

A transformer is a neural network architecture that uses attention mechanisms to process sequences and model relationships among their elements.

How it is used
Uses transformers for language generation, summarization, and translation, emphasizing stackable sequence representations.
How deep it goes
Explains tokenization, attention, positional encoding, and layer shapes, then implements and evaluates an encoder–decoder translator.
Examples
Uses German–English translation data, generated sentences, attention visualizations, and a model shape sanity check.
What you can do after
Apply it via PyTorch to train and evaluate a translator and inspect decoder attention weights.

Read how Natural Language Processing in Action, Second Edition explains transformer architecture →

Sutskever's List

A Transformer is a neural network architecture based primarily on attention mechanisms for modeling relationships among sequence elements.

How it is used
Frames the architectural shift from recurrent translation models to attention-based translation and language models.
How deep it goes
Explains direct token connections and identifies major components, while pointing to an annotated implementation for line-by-line detail.
Examples
Uses a “cheese”–“mice” dependency, translation BLEU scores, GPT-2, and a GPipe-trained Transformer.
What you can do after
Evaluate/understand why direct self-attention improved translation and training, while distinguishing plausible language generation from demonstrated understanding.

Read how Sutskever's List explains transformer architecture →

Deep Learning with Python, Third Edition

A neural-network architecture based on attention mechanisms, used here as the core architecture for large language models.

How it is used
Uses attention-based sequence modeling for translation, text classification, and large language models.
How deep it goes
Builds encoder blocks and a translation model, then adds positional embeddings and compares training results.
Examples
Compares English-to-Spanish translation models with and without position embeddings and revisits IMDb sentiment classification with a pretrained Transformer.
What you can do after
Build it from scratch using Keras layers to assemble and train an encoder–decoder translation model.

Read how Deep Learning with Python, Third Edition explains transformer architecture →

Build AI Drug Discovery Pipelines

A transformer architecture is a neural-network architecture that models relationships among sequence elements using attention and position-wise transformations, potentially with encoder and decoder stacks.

How it is used
Uses transformers to represent protein sequences for structure prediction and other drug-discovery tasks.
How deep it goes
Explains encoder–decoder and encoder-only architectures, attention, and positional encoding, while focusing on one component of the wider folding pipeline.
Examples
Uses SimpleFold and ESM-2 for protein folding and antibody classification to illustrate downstream use of protein language models.
What you can do after
Evaluate and understand how sequence embeddings support protein folding predictions, including the importance of held-out targets and per-residue confidence.

Read how Build AI Drug Discovery Pipelines explains transformer architecture →

Timeless Algorithms, The Foundational Ideas

A transformer is a neural-network architecture that uses attention to model relationships among sequence elements.

How it is used
Uses relational sequence modeling to explain the architectural basis of modern language models.
How deep it goes
Explains attention calculations and stacked components, then distinguishes simplified examples from full model behavior.
Examples
Works through pronoun resolution, semantic association, and autoregressive generation using simplified attention calculations.
What you can do after
Evaluates how attention constructs context and why parallel training, sequential generation, and reliability limits matter.

Read how Timeless Algorithms, The Foundational Ideas explains transformer architecture →

Build a Text-to-Image Generator (from Scratch)

A transformer architecture is a neural-network architecture that uses attention mechanisms to process relationships among elements in a sequence or other structured input.

How it is used
Uses translation to establish an attention-based architecture before applying sequence prediction to text-to-image generation.
How deep it goes
Builds an encoder–decoder translator from embeddings through attention, masks, training, and prediction, then extends the pipeline to image tokens.
Examples
Works through German-to-English translation and image generation from a panda prompt, including intermediate image-token outputs.
What you can do after
Build it from scratch in PyTorch as a trained encoder–decoder translator, then build a transformer-based text-to-image generator.

Read how Build a Text-to-Image Generator (from Scratch) explains transformer architecture →

Deep Learning with R, Third Edition

A Transformer is a neural-network architecture designed to process sets or sequences using attention-based contextual representations.

How it is used
Uses attention-based sequence modeling for translation, text classification, language modeling, and multimodal inputs.
How deep it goes
Develops encoder and decoder components, masking, positional embeddings, and a trained translation model, with comparisons to recurrent models.
Examples
Uses English-to-Spanish translation, IMDb sentiment classification, and image-question prompts to illustrate Transformer applications.
What you can do after
Builds a translation Transformer with R and Keras, then trains it and generates translations.

Read how Deep Learning with R, Third Edition explains transformer architecture →

Applied Reinforcement Learning

A neural-network architecture built around self-attention, position-wise feedforward transformations, residual connections, and normalization.

How it is used
Uses transformers to represent relationships among cooling zones for coordinated policy decisions and to process tokenized text in language models.
How deep it goes
Explains encoder sub-layers, residual connections, normalization, and stacking, and provides a PyTorch actor-critic implementation.
Examples
Illustrates an eight-zone data center cooling controller and next-token prediction from tokenized text.
What you can do after
Apply it via PyTorch encoder layers in a transformer-based actor-critic for coordinated cooling.

Read how Applied Reinforcement Learning explains transformer architecture →

Fight Fraud with Machine Learning

A transformer is a neural-network architecture that models relationships in sequences primarily through attention mechanisms rather than recurrence.

How it is used
Applies attention-based models to phishing, document-processing, and fake-voice fraud detection.
How deep it goes
Explains self-attention coefficients, multi-head concatenation, and encoder–decoder translation, then outlines Torch training without detailing layer internals.
Examples
Uses English-to-French translation, GPT phishing detection, KYC identity documents, and real-versus-fake voice classification.
What you can do after
Build it from scratch as a Torch classifier by defining layers and forward pass, then training, validating, and testing it.

Read how Fight Fraud with Machine Learning explains transformer architecture →

How Large Language Models Work

A transformer is a neural-network architecture that uses attention mechanisms to process relationships among elements in a sequence.

How it is used
Explains how language models turn token sequences into predictions through embeddings, attention, repeated computation, and sampling.
How deep it goes
Explains layers, attention, prediction types, and scaling behavior, but stops short of building an LLM from scratch or exposing proprietary details.
Examples
Uses Python dictionaries, preceding words, next-token prediction, translation, summarization, and a component diagram.
What you can do after
Evaluate/understand transformer outputs, model variants, sampling behavior, and practical deployment trade-offs.

Read how How Large Language Models Work explains transformer architecture →

Build an Advanced RAG Application (From Scratch)

The Transformer architecture is a neural-network architecture based on attention mechanisms for processing sequences and modeling relationships among their elements.

How it is used
Supports language understanding and generation tasks such as semantic search, translation, summarization, classification, and text generation.
How deep it goes
Explains embeddings, positional encoding, self-attention, multi-head attention, and encoder-decoder roles without emphasizing complex equations.
Examples
Contrasts processing “The quick brown fox jumps over the lazy dog” with RNNs, GRUs, and a Transformer.
What you can do after
Evaluate/understand how attention-based encoder and decoder components capture context and serve different language tasks.

Read how Build an Advanced RAG Application (From Scratch) explains transformer architecture →

Machines that Think

A neural-network architecture that processes sequence elements through attention-based interactions rather than requiring strictly sequential recurrence.

How it is used
Frames attention-based sequence processing as the architectural foundation for scalable language models and multimodal integration.
How deep it goes
Explains self-attention projections, block components, and encoder–decoder masking, with a high-level account of scaling and multimodal fusion.
Examples
Illustrates attention with “The sun rises in the east” and next-token generation with a masked decoder.
What you can do after
Evaluate and understand how attention, masking, and parallel processing shape Transformer capabilities and scaling constraints.

Read how Machines that Think explains transformer architecture →

AI-Powered Search

A transformer is a neural-network architecture that models relationships among sequence elements using attention and can encode and decode representations.

How it is used
Uses transformer embeddings to support semantic search and natural-language autocomplete.
How deep it goes
Explains contextual token representations and encoder–decoder separation, then outlines an embedding-based retrieval pipeline.
Examples
Illustrates contextual meaning with capital versus capitol and cross-language embeddings with English and Armenian greetings.
What you can do after
Evaluate and understand transformers for semantic search and question answering.

Read how AI-Powered Search explains transformer architecture →

Transformers in Action

A transformer is a neural network architecture that processes sequential and other structured data using attention mechanisms.

How it is used
Motivates attention-based sequence processing as a foundation for language models and other structured-data applications.
How deep it goes
Explains encoder–decoder flow, attention, feed-forward layers, residual connections, normalization, and positional encoding at a component level.
Examples
Illustrates English-to-French translation, sentiment in “The movie was not bad,” and gradient comparisons across sequence lengths.
What you can do after
Evaluate and understand transformer variants and their computational trade-offs for language tasks.

Read how Transformers in Action explains transformer architecture →

Transfer Learning for Natural Language Processing

A transformer is a neural network architecture that models sequences using attention mechanisms rather than recurrent connections.

How it is used
Motivates attention-based sequence modeling for translation and transfer learning across NLP tasks.
How deep it goes
Explains self-attention, encoder-decoder components, and parallelization; defers a complete BERT architectural discussion in the baseline treatment.
Examples
Uses machine translation, a sentence about cells and a cell phone, and a BERT classifier to illustrate transformer applications.
What you can do after
Evaluate and understand transformer components and their parallelization advantages, and apply pretrained BERT through Keras and TensorFlow Hub.

Read how Transfer Learning for Natural Language Processing explains transformer architecture →

Deep Learning with PyTorch, Second Edition

A neural network architecture for sequence modeling and content understanding based on attention rather than recurrence or convolution.

How it is used
Introduces attention-based sequence modeling for generation, translation, and content understanding.
How deep it goes
Builds a decoder in PyTorch and explains encoder-decoder information flow, while assigning full encoder-decoder implementation as an exercise.
Examples
Uses generated names, machine translation, and image classification to illustrate transformer applications.
What you can do after
Builds a decoder from scratch using PyTorch modules and its scaled dot product attention function.

Read how Deep Learning with PyTorch, Second Edition explains transformer architecture →

LLMs in Production

A transformer architecture is a neural-network architecture that models sequence relationships primarily through attention rather than recurrence or convolution.

How it is used
Positions transformer variants as the foundation for large language models and encoder-decoder transformers as architectures for conditional generation.
How deep it goes
Explains encoder-to-decoder data flow and training trade-offs, without detailing the attention computation.
Examples
Uses translation, summarization, T5, and a Llama 2 VRAM estimate to illustrate architecture and training.
What you can do after
Evaluate transformer architecture choices against task requirements, parallelism, training data, and VRAM constraints.

Read how LLMs in Production explains transformer architecture →

Grokking AI Applications

A transformer is a neural-network architecture for sequential data that uses attention to relate elements of an input sequence.

How it is used
Explains how GPT-style systems process ordered text and generate responses through attention, deep neural networks, and token prediction.
How deep it goes
Defines attention and layered processing with an example, but stops short of mathematical or implementation-level internals.
Examples
Uses GPT-3, the question about Italy's capital, token-by-token generation, and diagrams of GPT architecture and training.
What you can do after
Understand how attention and layered neural networks support GPT-style sequence processing and text generation.

Read how Grokking AI Applications explains transformer architecture →

The AI Pocket Book

A neural network architecture that contextualizes each token using surrounding tokens rather than compressing the whole prompt into one summary vector.

How it is used
Explains how current LLMs turn token embeddings into context-sensitive representations for next-token prediction.
How deep it goes
Defines contextualization and attention-based updates, but stops short of exact calculations and implementation details.
Examples
Uses dog-versus-tree “bark” prompts, token embeddings, attention scores, and a compact GPT-2 implementation as concrete illustrations.
What you can do after
Evaluate/understand why transformers preserve contextual information better than single-summary LSTM representations.

Read how The AI Pocket Book explains transformer architecture →

Time Series Forecasting Using Foundation Models

A neural-network architecture built from stacked attention and feed-forward processing components, typically organized into encoder and decoder parts.

How it is used
Uses attention-based sequence processing to explain and support time-series foundation-model forecasting.
How deep it goes
Explains embeddings, positional encoding, encoder-decoder flow, attention variants, masking, and projection, but defers attention mathematics.
Examples
Uses electricity-demand forecasting with temperature, plus trend and seasonal patterns as attention-learning examples.
What you can do after
Evaluate transformer-based forecasters and diagnose architectural limitations involving exogenous features, multiple series, and fine-tuning.

Read how Time Series Forecasting Using Foundation Models explains transformer architecture →

Building Specialized AI Systems

A neural-network architecture based primarily on attention mechanisms for processing sequences.

How it is used
Explains the backbone used to build contextual language representations before adapting them through fine-tuning.
How deep it goes
Explains inputs, encoder-decoder components, attention computation, and architectural motivation, but stops short of implementation code.
Examples
Uses language translation, tokenized subword inputs, attention weights, and encoder-decoder architecture diagrams.
What you can do after
Evaluate how attention, masking, positional information, and encoder-decoder pathways support sequence modeling and later fine-tuning.

Read how Building Specialized AI Systems explains transformer architecture →

Machine Learning Algorithms in Depth

A transformer model is a neural network architecture that uses attention-based mechanisms to model contextual relationships in structured sequences or other data.

How it is used
Uses transformer architecture to connect attention mechanisms with an implemented text-classification task.
How deep it goes
Explains query–key–value weighting and major architectural components, then provides a classifier implementation using Keras/TensorFlow layers.
Examples
Uses a translation attention heatmap, a transformer architecture diagram, and text-classifier training and validation plots.
What you can do after
Apply it via Keras/TensorFlow to text classification and inspect training and validation loss and accuracy.

Read how Machine Learning Algorithms in Depth explains transformer architecture →

Inside Deep Learning: Math, Algorithms, Models

A sequence model built from positional encoding and multiheaded attention, used here as the standard original design for NLP and other tasks.

How it is used
Uses transformers as an alternative to RNNs for sequence tasks, especially when depth, data, and compute can scale.
How deep it goes
Explains encoder and decoder blocks, residual connections, and autoregressive masking, and includes a classifier implementation.
Examples
Uses sentiment classification, machine translation, autoregressive prediction, and a PyTorch transformer classifier as examples.
What you can do after
Evaluate when transformer models suit sequence tasks and understand their block structure, masking needs, and compute tradeoffs.

Read how Inside Deep Learning: Math, Algorithms, Models explains transformer architecture →

Grokking AI Algorithms, Second Edition

The Transformer is a neural-network architecture based primarily on attention mechanisms for processing sequences and modeling context.

How it is used
Uses attention-based sequence processing to power language models that generate text, translate languages, answer questions, and write code.
How deep it goes
Explains self-attention, block components, architecture alternatives, and hyperparameters, but stops short of equations or implementation details.
Examples
Uses a toy language model with 34 tokens, 32-token context, two layers, one attention head, and small feed-forward networks.
What you can do after
Evaluate Transformer designs by comparing vocabulary, context, width, depth, attention-head, and feed-forward-size choices.

Read how Grokking AI Algorithms, Second Edition explains transformer architecture →

Understanding AI

A Transformer is a neural-network architecture that uses attention mechanisms to model relationships among sequence elements, often in parallel.

How it is used
Frames modern AI capabilities as attention-based pattern prediction that can generalize across many tasks.
How deep it goes
Explains attention, parallel sequence processing, scalable training, and broad task generalization without detailing internal computations.
Examples
Uses distant relationships in text and multimodal actions such as image questions, screenshot debugging, and voice-text interaction.
What you can do after
Evaluate/understand why attention-based models scale beyond narrow tasks while still relying on statistical patterns rather than human understanding.

Read how Understanding AI explains transformer architecture →

Hugging Face in Action

The Transformer architecture is a neural-network architecture based on attention mechanisms for processing sequences and other structured inputs.

How it is used
Deploys Transformer models through high-level pipelines for practical NLP tasks such as classification, generation, and summarization.
How deep it goes
Explains self-attention, embeddings, positions, and encoder-decoder roles briefly, while stopping short of detailed Transformer internals.
Examples
Demonstrates IMDb sentiment analysis with a pretrained DistilBERT model and Hugging Face’s pipeline API.
What you can do after
Apply it via Hugging Face pipelines to run pretrained models for classification and other NLP workflows.

Read how Hugging Face in Action explains transformer architecture →

Domain-Specific Small Language Models

A neural network architecture for sequence modeling built around self-attention and parallel processing, underlying both large and small language models.

How it is used
Frames sequence modeling around processing whole text inputs in parallel rather than recurrent word-by-word steps.
How deep it goes
Defines self-attention and explains its parallelism advantage over recurrent networks, while leaving multimodal applications out of scope.
Examples
Contrasts sequential RNN processing with whole-sequence Transformers and uses GPT-Neo large for content-generation inference.
What you can do after
Apply it via Hugging Face Transformers and Accelerate to run generation while considering speed, accuracy, and compute cost.

Read how Domain-Specific Small Language Models explains transformer architecture →

Introduction to Generative AI, Second Edition

A transformer is a neural-network architecture for sequence modeling that uses attention, especially self-attention, rather than recurrence.

How it is used
Explains the architectural shift that enabled modern language models and generative AI applications.
How deep it goes
Defines attention and self-attention through intuition and history, but stops short of implementation details, mathematics, or training procedures.
Examples
Uses a cat-and-mouse sentence, translation benchmarks, GPT, and attention visualizations to illustrate the architecture.
What you can do after
Evaluate/understand it by relating parallel self-attention to long-range context, scalable training, and LLM applications.

Read how Introduction to Generative AI, Second Edition explains transformer architecture →

Build AI-Enhanced Web Apps

A transformer model is a neural-network architecture that uses attention mechanisms to model contextual relationships in sequence data.

How it is used
Supports generative AI and language-model applications requiring global context, contextual relationships, and large-scale sequence processing.
How deep it goes
Defines self-attention, parallel processing, and contextual relationships with an example, but stops short of implementation details.
Examples
Uses pronoun resolution in a sentence to demonstrate how attention captures relationships separated by multiple words.
What you can do after
Evaluate transformer suitability for applications involving global context, language modeling, and sequence relationships.

Read how Build AI-Enhanced Web Apps explains transformer architecture →

Retrieval Augmented Generation, The Foundational Ideas

A transformer model is a neural-network architecture based on attention mechanisms for processing sequences and modeling contextual relationships.

How it is used
Supports contextual encoding for semantic search and similarity-based routing.
How deep it goes
Defines parallel processing and self-attention with contextual examples, then stops short of implementation details.
Examples
Uses bank disambiguation and customer-support ticket matching as concrete examples.
What you can do after
Evaluate contextual representations and recognize pairwise inference costs in large-scale similarity search.

Read how Retrieval Augmented Generation, The Foundational Ideas explains transformer architecture →

Rearchitecting LLMs

A transformer is a neural-network architecture that processes representations using attention and position-wise feed-forward components.

How it is used
Maps model components to memory, compute, and re-architecture decisions for inference optimization.
How deep it goes
Inspects model structure, constructor, attention, MLP, and activations; leaves full inference-time dynamics for later.
Examples
Uses DistilGPT2's printed module tree and fictional Llama-3.2 activation patterns.
What you can do after
Understand how model components relate to optimization choices and how activations reveal input-dependent behavior.

Read how Rearchitecting LLMs explains transformer architecture →

Build a Large Language Model (From Scratch)

a neural network architecture based on self-attention mechanisms that enables processing of sequential data and underpins many large language models.

How it is used
Organizes language processing into contextual encoding and sequential generation for translation and large language models.
How deep it goes
Defines encoder, decoder, and self-attention, but defers self-attention implementation and input preprocessing to later treatment.
Examples
Uses English-to-German translation, including generating “Beispiel” after “Das ist ein.”
What you can do after
Evaluate/understand how encoder-decoder flow and self-attention support translation and GPT-like text generation.

Read how Build a Large Language Model (From Scratch) explains transformer architecture →

Build a Website with ChatGPT

A transformer is a neural-network architecture that processes sequences using attention mechanisms to model relationships among their components.

How it is used
Explains why GPT can turn plain-English website requests into relevant generated code.
How deep it goes
Defines transformer as prioritizing important request components but stops short of describing attention layers or implementation.
Examples
Uses web-page requests and the prompt-copy-save-deploy cycle as practical examples.
What you can do after
Apply it via ChatGPT by writing prompts, copying generated code, saving an HTML file, and deploying it.

Read how Build a Website with ChatGPT explains transformer architecture →

Data Analysis with LLMs

The Transformer is a neural-network architecture designed to model relationships in sequences, especially sequences with variable-length inputs or outputs.

How it is used
Supports learning tasks with variable-length inputs or outputs, especially text, and underlies generative AI.
How deep it goes
Defines its sequence-learning role and broad prevalence without describing internal components or implementation.
Examples
Illustrates variable-length sequences with text documents and names document summarization and code writing as GPT tasks.

Read how Data Analysis with LLMs explains transformer architecture →

Generative AI in Action

The Transformer architecture is a neural-network architecture based primarily on attention mechanisms for processing sequences.

How it is used
Enables foundational models to understand and generate language for summarization, translation, classification, question answering, and content generation.
How deep it goes
Explains self-attention, contextual dependencies, parallel computation, and next-item prediction, but stops short of mathematical or layer-level internals.
Examples
Uses the completion “The cat sat on the...” to illustrate attention-weighted prediction of “mat,” alongside GPT and BERT as Transformer models.
What you can do after
Evaluate Transformer use for language generation while recognizing its substantial computational demands for training and use.

Read how Generative AI in Action explains transformer architecture →

Coding with AI

A Transformer is a neural-network architecture that processes sequence elements using attention and can process them in parallel.

How it is used
Underpins large language models for coding tools, chatbots, translation, and content generation.
How deep it goes
Explains input identifiers, positions, self-attention, parallel processing, and scaling, without detailing layer internals.
Examples
Uses “Jane helps Joe,” “The dog chased the cat,” and a figure showing positions and relationships.
What you can do after
Evaluate/understand it by tracing how prompts become context-sensitive representations for language applications.

Read how Coding with AI explains transformer architecture →

Optimization Algorithms: AI techniques for design, planning, and control problems

A transformer architecture is a neural-network architecture that processes sequences primarily through attention mechanisms rather than recurrence.

How it is used
Processes sequential data such as text and translation by prioritizing relevant input parts during prediction.
How deep it goes
Defines attention intuitively as relevance scores but does not detail Transformer layers, equations, or training.
Examples
Uses word-by-word reading, long sentences, translation, and attention coefficients as explanatory examples.
What you can do after
Understand how attention can reduce context loss and confusion in long or complex sequences.

Read how Optimization Algorithms: AI techniques for design, planning, and control problems explains transformer architecture →

Essential GraphRAG

A transformer architecture is a neural-network architecture that processes sequences using attention-based representations and supports efficient language modeling.

How it is used
Explains the architecture underlying LLM text processing, generation, and instruction-following behavior.
How deep it goes
Defines the architecture’s role and training objective but stops short of explaining internal mechanisms or implementation details.
Examples
Uses next-word completion, “Never gonna” to “give you up,” and haiku generation as concrete LLM examples.
What you can do after
Evaluate/understand how transformer-based LLMs generate text and why external retrieval can improve factual responses.

Read how Essential GraphRAG explains transformer architecture →

CUDA for LLMs

A transformer is a neural-network architecture that processes sequences using attention and position-aware transformations.

How it is used
Frames transformer architecture as the target for CUDA inference-kernel optimization.
How deep it goes
Defines sequence processing through attention and position-aware transformations, and establishes terminology and a numerical baseline.
Examples
Uses a PyTorch baseline as the numerical reference for inference kernels.
What you can do after
Evaluate CUDA inference kernels against a numerically matching PyTorch baseline.

Read how CUDA for LLMs explains transformer architecture →

Also covered in

What to know first

Related concepts

Questions about transformer architecture

What is transformer architecture?

A transformer is a neural-network architecture that uses attention-based interactions and position-wise transformations to model relationships among elements of sequences or other structured inputs.

Which Manning books explain transformer architecture?

Learn Generative AI with PyTorch, Natural Language Processing in Action, Second Edition, Sutskever's List, Deep Learning with Python, Third Edition, Build AI Drug Discovery Pipelines, Timeless Algorithms, The Foundational Ideas, Build a Text-to-Image Generator (from Scratch), Deep Learning with R, Third Edition and 32 more.

What should I know before learning transformer architecture?

Manning books that teach transformer architecture usually introduce language model, recurrent neural network, natural language processing and attention first.

What kinds of transformer architecture are there?

Kinds of transformer architecture include Vision Transformer and BERT.

What is transformer architecture a kind of?

Transformer architecture is a kind of model architecture. Model architecture is the structural arrangement of a computational model’s components, connections, and operations that determines how it processes inputs and produces outputs.

What are the parts of transformer architecture?

Transformer architecture includes decoding strategy and encoder.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.