Embedding
An embedding is a numerical vector representation of an item in a feature space, encoding selected properties or relationships for computational tasks such as comparison or retrieval.
Natural Language Processing in Action, Second Edition
A vector representation is a numerical vector that encodes the properties or meaning of a text, statement, word, or document.
- How it is used
- Uses embeddings to compare meaning and support semantic search, question answering, and word analogies.
- How deep it goes
- Explains vector intuition, local-context learning, similarity, and analogy arithmetic, with practical retrieval examples.
- Examples
- Uses a nessvec occupation-neighbor lookup, an overfitting FAQ search, and a scientist-and-subject analogy.
- What you can do after
- Apply it via nessvec to inspect nearby words and evaluate semantic matches and analogy queries.
Read how Natural Language Processing in Action, Second Edition explains embedding →
Build a Text-to-Image Generator (from Scratch)
An embedding is a numerical vector representation of an input or discrete item in a learned feature space.
- How it is used
- Supplies numerical inputs for transformers and shared representations for text–image matching and conditional image generation.
- How deep it goes
- Works through lookup dimensions, positional addition, image-patch projection, and retrieval; distinguishes token-level vectors from whole-input summaries.
- Examples
- Uses a four-token German phrase, CIFAR-10 image patches, and a Flickr 8k prompt-to-image search.
- What you can do after
- Build it from scratch for transformer inputs and image patches, or apply it via CLIP to rank images against a prompt.
Read how Build a Text-to-Image Generator (from Scratch) explains embedding →
AI-Powered Search
A vector representation is a data structure or encoding used to store and work with the components of a vector.
- How it is used
- Uses embeddings to compare meaning in search and build content-based personalization profiles.
- How deep it goes
- Explains vector dimensions and similarity with worked examples, then demonstrates pretrained embedding generation while deferring model-training mechanics.
- Examples
- Uses an apple-juice query and documents, illustrative food attributes, and product-name embeddings.
- What you can do after
- Apply it via SentenceTransformer to encode product names for content-based personalization.
Math and Architectures of Deep Learning
An ordered mathematical object representing a point or direction in a coordinate space.
- How it is used
- Represents inputs, outputs, parameters, and documents as ordered numeric objects for machine-learning computation and comparison.
- How deep it goes
- Defines vectors geometrically and computationally, then develops norms, dot products, matrix transformations, sparse representations, and PyTorch operations.
- Examples
- Uses hardness–sharpness inputs, image-class probability outputs, word-frequency documents, and PyTorch tensor creation and slicing.
- What you can do after
- Apply it via PyTorch to create, index, slice, compare, and transform vector representations.
Read how Math and Architectures of Deep Learning explains embedding →
Build AI Drug Discovery Pipelines
An embedding is a learned numerical vector representation of an object that captures properties relevant to a task.
- How it is used
- Uses embeddings to represent molecules and proteins for generation, affinity prediction, and virtual screening.
- How deep it goes
- Explains token lookup, latent compression, graph pooling, and reconstruction training, while deferring longer variational derivations.
- Examples
- Illustrates 100-token molecular compression, drug–protein graph embeddings, and FADS1 virtual screening.
- What you can do after
- Build it from scratch with PyTorch embedding and encoder–decoder components for molecular reconstruction.
Read how Build AI Drug Discovery Pipelines explains embedding →
Build an Advanced RAG Application (From Scratch)
An embedding is a numerical vector representation of an item that encodes meaningful features or relationships in a continuous space.
- How it is used
- Powers semantic retrieval by converting queries and documents into comparable vectors rather than matching exact keywords.
- How deep it goes
- Explains vector meaning, sentence encoders, similarity ranking, indexing, and long-context constraints without deriving embedding-model training internals.
- Examples
- Uses hotel reviews and Eiffel Tower queries, plus chunked research-paper abstracts stored as 768-dimensional vectors.
- What you can do after
- Build it from scratch by encoding text, indexing vectors, and retrieving semantically similar content for search or RAG.
Read how Build an Advanced RAG Application (From Scratch) explains embedding →
Graph Neural Networks in Action
A node embedding is a numerical vector representation of a graph node that captures information about the node and its graph context.
- How it is used
- Uses compact node vectors for graph visualization and as features for classification or other predictive tasks.
- How deep it goes
- Works through Node2Vec generation, GCN message-passing outputs, preprocessing, and training, with intuitive rather than rigorous mathematical treatment.
- Examples
- Uses Political Books co-purchases, retrieved Node2Vec vectors, GCN outputs, and a labeled-subset classification comparison.
- What you can do after
- Build it from scratch with Node2Vec or PyTorch Geometric, then align, inspect, and use node vectors for prediction.
Read how Graph Neural Networks in Action explains embedding →
Deep Learning with Python, Third Edition
An ordered numeric representation used here to encode source tokens and target tokens for attention computations.
- How it is used
- Maps token IDs into learned vectors for sequence models, classification, and language modeling.
- How deep it goes
- Explains embedding geometry, lookup shapes, task-specific training, CBOW pretraining, and weight transfer through worked Keras models.
- Examples
- Uses movie-review sentiment classification and missing-word prediction to demonstrate learned and pretrained embeddings.
- What you can do after
- Apply embeddings via Keras, including transferring pretrained CBOW weights into an LSTM classifier.
Read how Deep Learning with Python, Third Edition explains embedding →
Advanced Algorithms and Data Structures
A vector is a mathematical quantity with magnitude and direction that can represent a displacement or directed geometric segment.
- How it is used
- Turns abstract graphs into assessable drawings and images into feature vectors for similarity search.
- How deep it goes
- Develops graph-layout validation and crossing checks with pseudocode, while treating image feature extraction as a black box.
- Examples
- Uses circuit layouts, randomly sampled graph drawings, and catalog-image feature vectors as concrete cases.
- What you can do after
- Build it from scratch by assigning graph coordinates, validating the drawing, and comparing candidates by edge crossings.
Read how Advanced Algorithms and Data Structures explains embedding →
LLMs in Production
A vector is an ordered numerical representation of values in a mathematical space.
- How it is used
- Uses embeddings to represent language for modeling, semantic comparison, and retrieval.
- How deep it goes
- Explains context-based learning and semantic limitations, then demonstrates visualization and nearest-neighbor retrieval.
- Examples
- Uses a king–man+woman analogy, a missing-word prediction, and tweet nearest-neighbor search.
- What you can do after
- Apply it via Sentence Transformers and FAISS to encode text and prepare nearest-neighbor searches.
Grokking AI Applications
An embedding is a numerical vector representation of a text unit that encodes information used for model computations.
- How it is used
- Uses embeddings for token-level model computation and meaning-based document retrieval.
- How deep it goes
- Explains vector-space intuition and retrieval stages, contrasting a three-dimensional toy vocabulary with GPT-3’s larger embeddings.
- Examples
- Illustrates clustered city names in a toy vocabulary and document chunks stored for RAG retrieval.
- What you can do after
- Evaluate how embeddings support model computation and meaning-based retrieval, including practical retrieval trade-offs.
Outlier Detection in Python
An embedding is a numerical representation of an item in a continuous vector space that preserves selected properties or relationships of the original data.
- How it is used
- Represents complex inputs as numeric features for prediction, transfer learning, and outlier detection.
- How deep it goes
- Defines vector representations and explains autoencoder and proxy-task creation, then examines why direct outlier detection on them can fail.
- Examples
- Uses mug-image vectors, rotation prediction, and feature subsets of table rows to illustrate different representations.
- What you can do after
- Evaluate/understand whether an embedding is suitable for outlier detection rather than assuming a prediction-oriented vector will work.
Learn Generative AI with PyTorch
A continuous numeric encoding that captures the meaning of an input sentence for downstream decoding.
- How it is used
- Supplies compact numeric token representations to text-generation, translation, and music-generation models.
- How deep it goes
- Explains lookup-table dimensions, random initialization, training updates, and integration into LSTM and Transformer models.
- Examples
- Compares a five-token phrase in one-hot and 128-dimensional form, and implements 256-dimensional embeddings for English-to-French translation.
- What you can do after
- Apply it via PyTorch’s nn.Embedding to map vocabulary indexes into trainable vectors for sequence models.
Read how Learn Generative AI with PyTorch explains embedding →
Knowledge Graphs and LLMs in Action
A feature vector is a numerical vector whose components represent measurable characteristics of an input for machine learning.
- How it is used
- Deploys numerical feature vectors to make graph data usable by conventional machine-learning and downstream prediction algorithms.
- How deep it goes
- Explains vector design, binary encoding, identifier alignment, and path-based scoring, while deferring automated representation learning to later treatment.
- Examples
- Uses binary movie-genre tensors, user–movie rating graphs, and metapath-based drug–disease scores as concrete vector representations.
- What you can do after
- Build it from scratch by encoding domain properties into numerical components and preparing them for graph-based prediction pipelines.
Read how Knowledge Graphs and LLMs in Action explains embedding →
Inside Deep Learning: Math, Algorithms, Models
A one-dimensional array treated as a tensor and used as a model input or feature representation.
- How it is used
- Converts token IDs into vectors for sequence models and represents labels or inputs in other neural networks.
- How deep it goes
- Works through lookup shapes and implementation choices, including padding and packed-sequence handling; defers deeper word-embedding methods.
- Examples
- Uses a five-ID PyTorch lookup, packed RNN inputs, sequence-to-sequence padding, and label-conditioned generation.
- What you can do after
- Apply it via PyTorch `nn.Embedding` to turn integer IDs into model inputs and account for padding or packed sequences.
Read how Inside Deep Learning: Math, Algorithms, Models explains embedding →
Retrieval Augmented Generation, The Foundational Ideas
A dense vector is a vector whose coordinates are generally populated with informative numerical values.
- How it is used
- Supports semantic retrieval when queries and documents express related ideas without sharing keywords.
- How deep it goes
- Defines dense vectors against sparse vectors, explains word and sentence encoding, and demonstrates indexing while exposing ranking and top-k failures.
- Examples
- Compares BM25 with averaged word vectors on automobile queries and indexes PDF chunks with sentence-transformer embeddings and FAISS.
- What you can do after
- Apply it via HuggingFaceEmbeddings and FAISS to index document chunks, retrieve passages, and check whether the returned context supports an answer.
Read how Retrieval Augmented Generation, The Foundational Ideas explains embedding →
Deep Learning with R, Third Edition
An embedding is a learned numerical vector representation of an object, token, or other data item.
- How it is used
- Uses embeddings to turn token IDs into learned vectors for sequence models and to represent documents for retrieval.
- How deep it goes
- Explains vector geometry, lookup shapes, training, and CBOW pretraining, while leaving image prompt-encoder and mask-decoder details unexplained.
- Examples
- Uses movie-review sentiment classification and missing-word prediction to demonstrate learned and pretrained word vectors.
- What you can do after
- Apply embeddings via Keras to sequence classification and initialize a classifier with pretrained CBOW weights.
Read how Deep Learning with R, Third Edition explains embedding →
Build AI-Enhanced Web Apps
A vector is an ordered collection of numerical values representing a point, direction, or feature set in a mathematical space.
- How it is used
- Uses embeddings to support semantic search, recommendations, and document-grounded responses in web applications.
- How deep it goes
- Explains vector similarity and walks through embedding generation, storage, and retrieval, using an in-memory example before vector-store pipelines.
- Examples
- Illustrates a restaurant recommendation analogy, an IT support knowledge base, and document-based chat.
- What you can do after
- Apply embeddings via the Vercel AI SDK to generate text vectors and find similar support questions.
Sutskever's List
A vector is a one-dimensional ordered collection of numerical values.
- How it is used
- Turns token IDs into learned numerical inputs for a Transformer and represents words in a space where contextual similarities can emerge.
- How deep it goes
- Defines vector lookup, dimensions, and the resulting token matrix; illustrates word-vector arithmetic without deriving embedding training.
- Examples
- Uses a token-ID lookup and the Paris − France + Italy ≈ Rome analogy to make learned coordinates concrete.
- What you can do after
- Evaluate/understand how learned vectors supply model inputs and why one fixed-length vector can become a sequence bottleneck.
MongoDB 8.0 in Action, Third Edition
A text embedding is a vector representation of text that captures semantic relationships and contextual nuances.
- How it is used
- Enables semantic retrieval of stored content for vector search and retrieval-augmented generation.
- How deep it goes
- Explains numerical representation and same-model query requirements without deriving embedding-model internals.
- Examples
- Uses an OpenAI API request and a “village life” search over embedded movie plots.
- What you can do after
- Apply it via Atlas Vector Search to index stored embeddings and retrieve documents using an embedded query.
Read how MongoDB 8.0 in Action, Third Edition explains embedding →
Transformers in Action
A fixed-length vector is an ordered numerical representation with a predetermined number of components.
- How it is used
- Uses embeddings as semantic representations for retrieval, ranking, clustering, and multimodal reasoning.
- How deep it goes
- Explains contextual representations, similarity search, and visual patch projection, with pretrained-model examples rather than training an embedding model from scratch.
- Examples
- Illustrates contextual differences using financial and river banks, and encodes sentences and queries with a pretrained model.
- What you can do after
- Apply it via SentenceTransformer to encode sentences and compare queries with documents using cosine similarity.
Build a Large Language Model (From Scratch)
A numerical representation used to encode text or other objects for neural computation.
- How it is used
- Converts categorical text and other nonnumeric data into representations usable by neural-network operations.
- How deep it goes
- Defines continuous vector representations and briefly explains Word2Vec, stopping short of implementation details.
- Examples
- Uses a three-dimensional numerical vector, Word2Vec, and a scatterplot grouping birds separately from countries and cities.
- What you can do after
- Apply it via embedding models to encode nonnumeric data for neural-network processing.
Read how Build a Large Language Model (From Scratch) explains embedding →
How Large Language Models Work
A vector is an ordered numerical quantity that represents magnitude and direction or encodes data in a multidimensional space.
- How it is used
- Represents tokens as manipulable numerical inputs for neural networks and transformer text generation.
- How deep it goes
- Explains multidimensional representations, semantic relationships, and positional additions, but stops short of mathematical derivations.
- Examples
- Uses stock, capital, rare, and debt to show why one-number representations cannot capture varied word relationships.
- What you can do after
- Evaluate how language models turn tokens into multidimensional numerical representations and why positional information is necessary.
Read how How Large Language Models Work explains embedding →
Optimization Algorithms: AI techniques for design, planning, and control problems
A function that maps input entities or features into numerical representations used by a model.
- How it is used
- Provides low-dimensional graph representations for machine-learning tasks and optimization pipelines.
- How deep it goes
- Explains encoder–decoder intuition, structural similarity, and transductive versus inductive methods, with a graph convolution implementation.
- Examples
- Uses the Cora dataset and a traveling salesman pipeline to illustrate node embeddings and route generation.
- What you can do after
- Apply it via PyTorch Geometric to explore graph convolutional node representations.
Grokking AI Algorithms, Second Edition
An input vector is an ordered one-dimensional array of values supplied to a computational layer.
- How it is used
- Uses embeddings to represent tokens for Transformers and text labels for image-generation guidance.
- How deep it goes
- Explains trainable lookup matrices, elementwise positional addition, text projection, and embedding injection with numerical examples and learning-oriented code.
- Examples
- Traces a four-value token embedding combined with position and a text-label embedding used to guide a happy-face image.
- What you can do after
- Evaluate and understand how embeddings are formed, combined with position, and used to guide model computations.
Read how Grokking AI Algorithms, Second Edition explains embedding →
Graph Algorithms for Data Science
Node embedding is a technique that represents graph nodes as numerical vectors so that selected structural or relational similarities are preserved.
- How it is used
- Transforms graph structure into features for node classification and role-oriented analysis.
- How deep it goes
- Explains node2vec through skip-gram, random walks, configuration, and similarity checks, while deferring hyperparameter optimization.
- Examples
- Uses friendship interests, coauthorship bridges, routers, and weighted shared-audience stream graphs.
- What you can do after
- Apply it via Neo4j Graph Data Science node2vec and evaluate vectors before training a stream-language classifier.
Read how Graph Algorithms for Data Science explains embedding →
AI Agents in Action, Second Edition
A dense vector is a numerical vector in which the representation generally assigns a value to every dimension.
- How it is used
- Uses embeddings to retrieve relevant document chunks and memories as context for agents.
- How deep it goes
- Explains dense vectors, approximate training intuition, similarity search, and retrieval pitfalls without detailing network internals.
- Examples
- Illustrates retrieval with a capital-of-France query and compares similarity rankings for blue and breakfast queries.
- What you can do after
- Apply it via OpenAI embeddings to compare cosine-similarity rankings and store vectors in Chroma DB.
Read how AI Agents in Action, Second Edition explains embedding →
Domain-Specific Small Language Models
A fixed-length numerical representation used for similarity search and retrieval in this section.
- How it is used
- Uses embeddings to represent text for similarity search, RAG document retrieval, and relevant conversation-memory lookup.
- How deep it goes
- Works through vector generation, indexing, and query ranking, but does not derive embedding-model training or vector-space mathematics.
- Examples
- Demonstrates labeled-text search in FAISS and cosine-similarity retrieval of PDF chunks for a generator.
- What you can do after
- Apply it via sentence-transformers and FAISS to index text and retrieve similar passages for classification or generator context.
Read how Domain-Specific Small Language Models explains embedding →
AI Agents and Applications
A vector type is a database data type that represents numerical vector values as stored fields.
- How it is used
- Uses embeddings to index document chunks and match questions to relevant context for RAG answers.
- How deep it goes
- Defines high-dimensional embeddings and explains ingestion, similarity search, and storage workflows without deriving embedding models.
- Examples
- Uses a feline-animals query, a Chroma document index, and a PostgreSQL name-embedding column.
- What you can do after
- Apply embeddings via LangChain and Chroma to index document chunks for similarity retrieval.
Enterprise RAG
An embedding process converts textual or other source content into numerical vector representations.
- How it is used
- Enables semantic retrieval of indexed content for RAG questions and search requests.
- How deep it goes
- Works through chunk ingestion, dimension compatibility, query embedding, and hybrid retrieval without embedding-level derivations.
- Examples
- Demonstrates embedded manual chunks, 1,536-dimensional record summaries, and a “Dubious Parenting Advice” hybrid query.
- What you can do after
- Apply it via Azure AI Search and an embedding API to index records and run hybrid searches.
Deep Learning with PyTorch, Second Edition
A one-dimensional numeric data structure used here as a mathematical object on which sums and dot products can be computed.
- How it is used
- Represents words, characters, images, and diffusion-model inputs as numeric vectors for downstream learning or generation.
- How deep it goes
- Explains trainable embedding lookup and visualization, but omits the sinusoidal embedding implementation.
- Examples
- Uses character lookup and a three-dimensional visualization showing vowel and consonant clusters.
- What you can do after
- Apply it via PyTorch nn.Embedding to turn character indices into trainable vectors.
Read how Deep Learning with PyTorch, Second Edition explains embedding →
The Art of AI Product Development
A numeric vector representation that encodes the semantic or structural properties of an item for computational comparison.
- How it is used
- Supports semantic document retrieval for standalone search and retrieval-augmented generation.
- How deep it goes
- Explains distributional similarity and embedding-space proximity, then discusses storage and retrieval limitations without mathematical derivations.
- Examples
- Illustrates word similarity with apple, banana, and car, and search refinement with recent cloud-computing reports.
- What you can do after
- Evaluate and understand where embeddings help retrieval and why semantic search may need additional refinements.
Read how The Art of AI Product Development explains embedding →
LLM Evaluation and Alignment, The Foundational Ideas
A vector is an ordered numerical representation of a point, direction, or feature pattern in a mathematical space.
- How it is used
- Supports semantic evaluation when word overlap misses paraphrases or synthesis.
- How deep it goes
- Explains geometric representation, cosine similarity, pooling, contextualization, and COMET’s use of sentence vectors.
- Examples
- Uses attorney–lawyer paraphrases and the financial versus river meanings of bank to illustrate similarity and context.
- What you can do after
- Evaluate and understand semantic similarity using sentence vectors while recognizing the limitations of static embeddings and averaging.
Read how LLM Evaluation and Alignment, The Foundational Ideas explains embedding →
AI Recommender Systems
A learned vector representation generated by a large language model that encodes semantic, relational, and contextual information about text or entities.
- How it is used
- Supports candidate retrieval and scoring in recommenders, with semantic content vectors complementing behavioral signals.
- How deep it goes
- Works through vector encoding, similarity ranking, and hybrid weighting; defers million-item vector indexing and approximate nearest-neighbor search.
- Examples
- Demonstrates MovieLens movie-metadata encodings and Toy Story nearest neighbors, including sequel-heavy results.
- What you can do after
- Apply it via sentence-transformers to encode movie metadata, rank similar items, and compare multi-seed aggregation strategies.
Build Python Web Apps with Streamlit
A text embedding is a numerical vector representation of text that captures features useful for semantic comparison.
- How it is used
- Enables relevant knowledge-base retrieval for a customer-service chatbot using RAG.
- How deep it goes
- Explains vectors through a two-attribute analogy and implements similarity search, without requiring embedding internals.
- Examples
- Uses movie ratings as a vector analogy and support articles as a searchable knowledge base.
- What you can do after
- Apply it via OpenAIEmbeddings and Pinecone to index document chunks and retrieve relevant passages.
Read how Build Python Web Apps with Streamlit explains embedding →
Introduction to Generative AI, Second Edition
An embedding is a numerical internal representation of data that captures features useful for computational processing.
- How it is used
- Uses embeddings to represent words and retrieve semantically similar documents for language-model responses.
- How deep it goes
- Explains embeddings through spatial similarity and a retrieval workflow without implementation details.
- Examples
- Illustrates word similarity with cucumber and pickle, and document retrieval with a travel-expense policy question.
- What you can do after
- Supports evaluating embeddings as a retrieval aid while recognizing potential bias and privacy challenges.
Read how Introduction to Generative AI, Second Edition explains embedding →
Building Reliable AI Systems
An embedding is a numerical representation of content that captures semantic meaning in a form suitable for comparison.
- How it is used
- Uses embeddings to retrieve semantically relevant document chunks for grounded responses.
- How deep it goes
- Explains vector-space similarity and indexing, then demonstrates model encoding and vector database configuration.
- Examples
- Illustrates semantic matching with adoption leave and a health-care policy assistant.
- What you can do after
- Apply embeddings via Sentence Transformers and Pinecone to encode and search policy chunks.
The AI Pocket Book
A vector representation used to encode the meaning of an image, text, or other content.
- How it is used
- Represents the meaning of tokens, images, text, and other content for comparison, prediction, and retrieval.
- How deep it goes
- Explains dimensions, meaning-space geometry, dot products, projections, and learned parameters, but not the exact semantics of individual dimensions.
- Examples
- Uses dog, elephant, anaconda, snake, beetle, and ladybug vectors, plus PyTorch embedding and projection operations.
- What you can do after
- Understand how numerical representations encode meaning and support similarity comparisons, projections, language models, image models, and retrieval.
Timeless Algorithms, The Foundational Ideas
An embedding is a numerical vector representation of an item, such as a word or token, in a learned space.
- How it is used
- Introduces embeddings as initial token representations used in transformer attention and next-token prediction.
- How deep it goes
- Defines embedding lookup and training, then traces simplified vectors through an attention calculation rather than reproducing a complete trained transformer.
- Examples
- Uses two-dimensional vectors for “trophy,” “box,” and “it” in a numerical pronoun-resolution example.
- What you can do after
- Evaluate how token embeddings contribute to a simplified attention calculation without mistaking it for a complete transformer.
Read how Timeless Algorithms, The Foundational Ideas explains embedding →
Classic Computer Science Problems in Swift: Essential techniques for practicing programmers
A vector is an ordered collection of numerical components representing a quantity or point in a mathematical space.
Designing AI Systems
An embedding is a numerical vector representation of content that preserves aspects of its meaning so semantically similar content has nearby representations.
- How it is used
- Supports semantic retrieval of organizational knowledge for grounding responses.
- How deep it goes
- Explains ingestion and search flows with generator and orchestration interfaces, while leaving model training and vector-search algorithms at a high level.
- Examples
- Uses a return-policy query and a batched embedding generator to illustrate semantic matching and implementation.
- What you can do after
- Operate/configure an embedding-based retrieval pipeline with batched generation and a consistent index model.
Generative AI in Action
A vector is an ordered sequence of numerical values representing a point or quantity in a multidimensional space.
- How it is used
- Supports semantic comparison and retrieval of proprietary data for generative AI applications.
- How deep it goes
- Explains learned vector representations and retrieval workflows, while deferring similarity-measure details.
- Examples
- Uses illustrative cat and dog vectors, an OpenAI embedding request, and a blog-question-answering application.
- What you can do after
- Apply embeddings via the OpenAI API and use them in a Redis-backed retrieval workflow.
Machines that Think
A learned vector representation of an item whose values encode relationships inferred from data.
- How it is used
- Represents language geometrically and supports similarity-based selection of external documents for retrieval-augmented generation.
- How deep it goes
- Explains dense vectors, predictive learning and geometric relationships, while noting the limitation of fixed word representations.
- Examples
- Uses nearby animal words, word-form transformations and question-to-document matching to illustrate embedding relationships.
- What you can do after
- Evaluate and understand how learned similarity supports retrieval, while recognizing limits of interpretability and grounding.
Data Pipelines with Apache Airflow, Second Edition
A vector is an ordered numerical quantity representing a point, direction, or set of measurable attributes in a multidimensional space.
- How it is used
- Enables vector similarity search over recipes so RAG can retrieve context for a user’s question.
- How deep it goes
- Explains vectors through a two-dimensional analogy and cosine similarity, but sets embedding-model training details aside.
- Examples
- Uses flavor coordinates, a poke-bowl-to-sushi comparison, a lasagna question, and Weaviate vectorizer configuration.
- What you can do after
- Operate/configure it by choosing a recipe embedding model, configuring a Weaviate collection, and embedding questions for retrieval.
Read how Data Pipelines with Apache Airflow, Second Edition explains embedding →
Data Analysis with LLMs
An embedding is a numerical vector representation of an object that is intended to preserve meaningful properties of that object.
- How it is used
- Turns text into comparable vectors for document clustering, retrieval, and outlier detection.
- How deep it goes
- Walks through endpoint calls, vector extraction, retries, and comparisons without explaining how models generate the vectors internally.
- Examples
- Uses clustered emails and poems, a Transformer-related question and document, and a Python embedding API call.
- What you can do after
- Apply it via OpenAI's embedding endpoint to compare texts, or via LlamaIndex to retrieve question-relevant data.
Build an AI Agent (From Scratch)
A vector is an ordered collection of numerical values representing an object or its properties in a mathematical space.
- How it is used
- Uses embeddings to retrieve semantically relevant knowledge for agents and store searchable memories.
- How deep it goes
- Explains vector similarity and implements embedding and retrieval flows, while leaving detailed model training mechanisms unexplored.
- Examples
- Compares cat, kitten, and dog sentences and retrieves quantum-related chunks from Nobel Prize web results.
- What you can do after
- Apply it via OpenAI’s embedding API to generate vectors and compare text similarity, or use ChromaDB for storage and retrieval.
Read how Build an AI Agent (From Scratch) explains embedding →
Fight Fraud with Machine Learning
An embedding is a learned vector representation that encodes properties or relationships of an input such as an image, text, or word.
- How it is used
- Uses embeddings to compare faces for duplicate detection, selfie-to-ID checks, language processing, and graph-based money-laundering analysis.
- How deep it goes
- Explains vector generation, similarity, pairwise training, contextual text, and graph neighborhoods, but stops short of embedding mathematics.
- Examples
- Demonstrates Einstein and Monroe face comparisons, one-hot word vectors, contextual sentences, and graph nodes whose neighborhoods reveal similarity.
- What you can do after
- Apply it via MTCNN and Inception Resnet V1 to compare selfie and ID faces with cosine similarity and flag suspicious scores.
Read how Fight Fraud with Machine Learning explains embedding →
Effective Conversational AI
A text embedding is a numerical vector representation that encodes semantic or contextual properties of text.
- How it is used
- Supports RAG search by matching embedded document chunks with semantically related user questions.
- How deep it goes
- Explains preprocessing, chunking, vector generation, storage, similarity search, and benchmarking without detailing embedding-model internals.
- Examples
- Uses document chunks, user questions, PharmaBot medical queries, vector databases, and alternative embedding-model configurations.
- What you can do after
- Evaluate embedding and indexing choices by generating vectors, running queries, and measuring retrieval and operational performance.
Building Specialized AI Systems
An embedding vector is a numerical vector representing the semantic or syntactic properties of an input token or other discrete item.
- How it is used
- Converts token IDs into dense vectors that feed Transformer blocks and can receive regularization during instruction fine-tuning.
- How deep it goes
- Explains token lookup, positional combination, and NEFTune noise scaling, but stops short of embedding-matrix training internals.
- Examples
- Uses subword-to-vector input processing and a NEFTune trainer configured with neftune_noise_alpha=5.
- What you can do after
- Apply embeddings through a Transformer input pipeline and configure NEFTune to regularize instruction fine-tuning.
Read how Building Specialized AI Systems explains embedding →
Build a DeepSeek Model (From Scratch)
An embedding is a learned vector representation of a discrete input such as a token, including its semantic information.
- How it is used
- Feeds token representations into attention projections, latent compression, and expert-routing calculations.
- How deep it goes
- Defines embeddings through shapes and matrix multiplications, then stops short of explaining how embedding vectors are learned.
- Examples
- Uses four-token matrices, the token “bright,” and explicit shapes such as (4, 8) to trace embedding transformations.
- What you can do after
- Apply via matrix multiplication to transform token embeddings into attention inputs, compressed cache entries, or expert scores.
Read how Build a DeepSeek Model (From Scratch) explains embedding →
Building LLM Applications with DSPy
A numerical vector representation of an item, such as text, that encodes useful features.
- How it is used
- Uses embeddings to select prompt demonstrations and retrieve article titles relevant to a query.
- How deep it goes
- Defines text embeddings and works through encoder and retrieval setup, without detailing model internals.
- Examples
- Compares airline utterances and retrieves discount-coupon titles for a service-pricing question.
- What you can do after
- Apply embeddings via SentenceTransformer and DSPy to select demonstrations or retrieve matching titles.
Read how Building LLM Applications with DSPy explains embedding →
LLM Customization and Fine-Tuning
A numerical vector representation of an item that encodes features useful for computational comparison.
- How it is used
- Uses embeddings for semantic retrieval, production drift detection, and domain vocabulary adaptation.
- How deep it goes
- Explains vector similarity, dimensionality tradeoffs, centroid drift detection, and frozen embedding-layer limits without deriving embedding training.
- Examples
- Uses NumPy top-k retrieval, reference and production text centroids, and medical vocabulary expansion as examples.
- What you can do after
- Evaluate embedding choices for retrieval and understand how embeddings support drift detection and vocabulary adaptation.
Read how LLM Customization and Fine-Tuning explains embedding →
Hidden Influences
A learned numerical representation that maps a user, item, or other complex entity to a vector while preserving information relevant to a task.
- How it is used
- Uses user and item representations to compare similarity and rank recommendations.
- How deep it goes
- Defines embeddings and works through one- and two-number examples without detailing how embeddings are learned.
- Examples
- Uses ranked item scores and cat-and-dog affinity coordinates to illustrate recommendations.
- What you can do after
- Understand how numerical user and item representations can guide recommendation rankings.
Deep Learning with JAX
A rank-1 multidimensional array with one axis, used here for model outputs such as class probabilities.
- How it is used
- Defines vectors for representing model outputs and partitioning data in array computations.
- How deep it goes
- Explains rank, size, batching, vmap, dot products, and device sharding without developing vector internals.
- Examples
- Uses class-probability outputs, batched predictions, wide vectors, and two-dimensional mesh partitioning.
- What you can do after
- Apply it via JAX by distinguishing vector and batch shapes, using vmap, and interpreting sharded dot-product outputs.
Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI
An embedding is a learned vector representation of an item in a numerical space.
- How it is used
- Uses learned representations as features for diversity-oriented clustering and transfer learning.
- How deep it goes
- Explains layer selection, model reuse, and an iterative cluster-classification workflow.
- Examples
- Uses a three-class model’s neurons as features for a two-class prediction task.
- What you can do after
- Evaluate embedding choices for clustering and transfer learning, including dimensionality and access to model layers.
Data Without Labels
A word embedding maps each word to a numerical vector whose geometry reflects usage-based meaning and relationships.
- How it is used
- Converts text into compact numerical features for language translation, chatbots, and generalized text models.
- How deep it goes
- Defines contextual vectors and outlines CBOW, skip-gram, and co-occurrence learning, while deferring deeper technical treatment to the original papers.
- Examples
- Uses gender, quality, and location analogies to illustrate relationships among embedded words.
- What you can do after
- Apply Word2Vec or GloVe to understand how corpus context produces word vectors and semantic neighborhoods.
Transfer Learning for Natural Language Processing
An embedding is a vector representation of an item in a space where relationships among items can be modeled geometrically.
- How it is used
- Uses text vectors to enable downstream clustering or classification with shallow statistical methods.
- How deep it goes
- Explains context-based vector representations and downstream use, then demonstrates pretrained sentence embeddings without deriving their training algorithm.
- Examples
- Uses word analogies such as man–king and woman–queen, plus sent2vec vectors for IMDB movie reviews.
- What you can do after
- Apply pretrained sent2vec vectors to IMDB reviews for classification.
Read how Transfer Learning for Natural Language Processing explains embedding →
Financial AI in Practice
An embedding is a numerical vector representation of content that preserves properties useful for similarity comparison and retrieval.
- How it is used
- Represents financial news, assets, and investor behavior for semantic search and personalization.
- How deep it goes
- Works through batch embedding and content-pipeline steps, without explaining embedding-model internals.
- Examples
- Uses cleaned financial news, ETF metadata, and investor portfolios to illustrate vector representations.
- What you can do after
- Apply embeddings via the OpenAI API to batch news text and save vectors for later use.
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
An embedding is a learned or constructed representation that maps high-dimensional data into a lower-dimensional space while preserving selected characteristics or relationships.
- How it is used
- Handles high-cardinality categorical features when deep-learning models are the preferred approach.
- How deep it goes
- Defines dimensionality reduction and contrasts embeddings with encoding, but defers their detailed implementation.
- Examples
- Uses categorical embeddings in shallow neural networks and names fastai Tabular, PyTorch Tabular, TabNet, SAINT, and DeepTables.
- What you can do after
- Choose embeddings for high-cardinality categorical data when applying deep-learning approaches to tabular problems.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains embedding →
Agent Design Patterns
A numerical vector representation of data, often designed to encode semantic features for computational comparison.
- How it is used
- Prepares agent queries for inexpensive semantic comparison with stored records before detailed candidate evaluation.
- How deep it goes
- Defines embeddings operationally as semantic search representations and follows them through retrieval, but does not specify how vectors are constructed.
- Examples
- Illustrates a burst-pipe insurance claim shortlisted against historical cases before more precise relevance evaluation.
- What you can do after
- Evaluate/understand it as a first-pass semantic search representation whose candidates still need relevance checking.
Data Storytelling with Altair and AI
An embedding is a numerical vector representation that captures semantic or structural properties of an object such as text, enabling comparison and computation in a continuous vector space.
- How it is used
- Deploys embeddings to index and retrieve textual context for retrieval-augmented generation.
- How deep it goes
- Defines numerical representations and demonstrates conversion, storage, indexing, and retrieval without explaining vector-generation internals.
- Examples
- Uses an HTML product description, overlapping text chunks, a Chroma collection, and a SmartX 2023 retrieval query.
- What you can do after
- Apply embeddings via LangChain, OpenAIEmbeddings, and Chroma to build a searchable retrieval pipeline.
Read how Data Storytelling with Altair and AI explains embedding →
Vibe Engineering
An embedding is a numerical vector representation of data whose geometric relationships encode semantic similarity.
- How it is used
- Supports retrieval-augmented generation by locating semantically relevant code and documentation for an AI prompt.
- How deep it goes
- Explains vectorization, storage, similarity search, and prompt augmentation, but does not detail embedding model internals.
- Examples
- Uses repository searches for authentication fixes and JWT-library refactoring, plus indexed functions, classes, documentation, and configuration.
- What you can do after
- Apply via a retrieval pipeline that indexes chunks, stores vectors, searches queries, and augments prompts with selected context.
Build a Multi-Agent System (from Scratch)
An embedding is a numerical vector representation of an item that encodes properties or relationships relevant to machine learning.
- How it is used
- Supports semantic recall by representing stored episodes and queries as vectors for similarity search.
- How deep it goes
- Defines shared-space vectorization and distance-based similarity, then demonstrates Qdrant indexing and retrieval without detailing embedding internals.
- Examples
- Uses Hailstone episodes and a matching query to demonstrate retrieval by similarity.
- What you can do after
- Apply embeddings through Qdrant to persist episodes and retrieve the most similar matches for a query.
Read how Build a Multi-Agent System (from Scratch) explains embedding →
Architecting for Autonomy
A numerical vector representation of content that captures features for computational similarity, retrieval, or downstream interpretation.
- How it is used
- Supports agent retrieval and interpretation by determining which documents and passages enter the agent’s governed context.
- How deep it goes
- Explains embeddings architecturally through indexing and governance, without detailing vector construction or similarity mathematics.
- Examples
- Uses an amended supplier obligation to show why indexed content, supersession metadata, retrieval priority, and memory require coordinated updates.
- What you can do after
- Operate/configure embedding-backed retrieval with versioning, access control, lineage, reuse limits, and matter isolation.
Applied Reinforcement Learning
An embedding is a dense vector representation of an item, such as a token, learned by a model.
- How it is used
- Maps token IDs to dense learned vectors before transformer processing.
- How deep it goes
- Defines learned high-dimensional token vectors and contextual similarity, but does not detail how their training works.
- Examples
- Uses “aspirin” and related medical terms to illustrate token vectors and contextual similarity.
- What you can do after
- Understand how token IDs become learned vectors that carry token and positional information into a transformer.
Read how Applied Reinforcement Learning explains embedding →
Delivering Machine Learning Projects: From design to deployment
An embedding is a numerical vector representation that encodes meaningful properties of an input for comparison or downstream computation.
- How it is used
- Supports exploratory analysis of unstructured data through visualization and similarity search.
- How deep it goes
- Defines vector generation and typical dimensionality, but stops short of making raw coordinates directly interpretable.
- Examples
- Uses EfficientNet, BERT-derived models, Shakespeare’s plays, heat maps, FAISS, and Annoy as concrete exploration artifacts.
- What you can do after
- Apply it via foundational models and indexing tools to visualize or query structure in an unstructured corpus.
Read how Delivering Machine Learning Projects: From design to deployment explains embedding →
Machine Learning System Design
An embedding is a numerical vector representation of an object that places semantically or structurally similar objects near one another in a representation space.
- How it is used
- Encodes purchasing behavior patterns for product-related data pipelines.
- How deep it goes
- Explains transaction aggregation, user-item matrices, and factorization, but does not detail embedding mathematics or model tuning.
- Examples
- Uses Supermegaretail transactions aggregated into a customer–product matrix as the concrete example.
- What you can do after
- Build it from scratch by aggregating transactions and factorizing a user-item matrix.
Read how Machine Learning System Design explains embedding →
Grokking Software Architecture
A numerical representation that maps an input such as text or an image into a vector space so semantic relationships can be measured.
- How it is used
- Enables semantic retrieval by turning text or images into vectors that a vector database compares by similarity rather than exact keywords.
- How deep it goes
- Defines embeddings and explains vector similarity with examples, while deferring vector generation to AI services rather than manual calculation.
- Examples
- Contrasts “Red Shirt” with “Crimson Top,” compares dog, puppy, and car vectors, and retrieves lasagna for “Comfort Food.”
- What you can do after
- Apply it via an AI embedding service and vector database to index text and retrieve semantically similar items.
Read how Grokking Software Architecture explains embedding →
AI Governance
An embedding is a numerical vector representation of data that captures features or relationships for computational use.
- How it is used
- Grounds model answers by enabling RAG pipelines to retrieve passages similar to a user’s query.
- How deep it goes
- Explains embedding generation, privacy attacks, linkability, and legal scope without detailing embedding internals.
- Examples
- Demonstrates reconstruction and attribute inference, including inferring a patient diagnosis from embeddings.
- What you can do after
- Evaluate embedding privacy risk by checking source sensitivity and whether vectors can be linked to people.
AI Agents in Action
A vector representation encodes an object or piece of information as a numerical vector.
- How it is used
- Uses document vectors to support semantic retrieval and compare question-answer outputs.
- How deep it goes
- Explains pretrained neural networks and vector dimensions, then demonstrates PCA visualization; it does not detail training or embedding internals.
- Examples
- Demonstrates OpenAI document vectors plotted in 3D and embeddings used to compare predicted and expected answers.
- What you can do after
- Apply embeddings with the OpenAI Python client, then use PCA to visualize document vectors or compare answer similarity.
Machine Learning Algorithms in Depth
A vector is an ordered mathematical object representing magnitude and direction or coordinates in a vector space.
- How it is used
- Uses vectors to visualize high-dimensional data, represent words, and encode sequences for downstream learning tasks.
- How deep it goes
- Explains embedding roles through t-SNE, word vectors, and recurrent encoding, but defers detailed embedding mathematics.
- Examples
- Demonstrates embeddings with MNIST digit clusters, IMDb word vectors, sequence encodings, and RBF-induced infinite-dimensional feature vectors.
- What you can do after
- Apply it via t-SNE, KD trees, recurrent networks, and trainable embedding layers to visualize or classify structured data.
Read how Machine Learning Algorithms in Depth explains embedding →
Build Applications with Local AI Models on a Mac
An embedding is a numerical vector representation of an item, such as text, positioned to capture aspects of its meaning or features.
- How it is used
- Uses embeddings to help RAG find relevant documents without exact keyword matches.
- How deep it goes
- Defines vectors and embedding models and describes similarity search, without detailing how embeddings are computed.
- Examples
- Uses “dog” and “puppy” to illustrate similarity between words with different spellings.
- What you can do after
- Understand how a question can retrieve relevant documents through vector comparison.
Read how Build Applications with Local AI Models on a Mac explains embedding →
Also covered in
- Statistics Every Programmer Needs
- Essential GraphRAG
- Just Use Postgres!
- Learn AI-Assisted Python Programming, Second Edition
- Hugging Face in Action
- Spring AI in Action
- Investing for Programmers
- Evolutionary Deep Learning: Genetic algorithms and neural networks
- Understanding AIIn early access (MEAP)
- Machine Learning Platform Engineering
- Time Series Forecasting Using Foundation Models
- Grokking Algorithms, Second Edition
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- similarity searchSimilarity search is a retrieval technique that finds or ranks items by their similarity to a query, often by comparing feature or vector representations under a similarity or distance measure.
- tokenA token is a discrete unit of text, such as a word, subword, punctuation mark, or symbol, produced by tokenization and processed by a language model.
- retrieval-augmented generationRetrieval-augmented generation is a technique that retrieves relevant information from external sources and supplies it as context to a generative model when producing an output.
Related concepts
- vectorizationproduced by · Vectorization is the transformation of data or computations into vector form so they can be processed collectively by vector-oriented operations.
- encoderproduced by · An encoder is a component or process that transforms input into an encoded or internal representation for subsequent processing, transmission, or decoding.
- semantic cachingused by · Semantic caching is a technique that stores and reuses results for requests judged sufficiently similar in meaning, rather than requiring identical text or keys.
- t-SNEproduced by · t-SNE is a nonlinear dimensionality-reduction technique that maps high-dimensional data to a lower-dimensional space while emphasizing local similarities or neighborhood relationships.
- text encoderproduced by · A text encoder is a neural-network model that transforms textual input into numerical representations for conditioning or other downstream computation.
- Retrieverused by · A retriever is a software component that searches or queries an information source and returns items relevant to a query for use by downstream processes.
Questions about embedding
What is embedding?
An embedding is a numerical vector representation of an item in a feature space, encoding selected properties or relationships for computational tasks such as comparison or retrieval.
Which Manning books explain embedding?
Natural Language Processing in Action, Second Edition, Build a Text-to-Image Generator (from Scratch), AI-Powered Search, Math and Architectures of Deep Learning, Build AI Drug Discovery Pipelines, Build an Advanced RAG Application (From Scratch), Graph Neural Networks in Action, Deep Learning with Python, Third Edition and 64 more.
What should I know before learning embedding?
Manning books that teach embedding usually introduce language model, similarity search, token and retrieval-augmented generation first.
What is embedding also called?
Embedding is also called vector representations or distributed representation.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.







































































