A vector is a list of numbers used to represent meaning. Each position can measure how strongly content matches some learned feature, and the number of positions is its dimension. section 1.13
A dog as five numbers
Imagine five positions representing Animal, Cat, Large, Scary, and Four legs. A dog gets a high Animal value, a negative Cat value, and a neutral Large value. An elephant would score highly for Large. Real embeddings use many more dimensions, so their features are not usually this easy to name. section 1.13
Treat the numbers as coordinates in a meaning space. Related items tend to cluster: anaconda and snake sit near each other, while anaconda and ladybug sit far apart. Similarity can be estimated with a dot product, while a projection compresses a long vector into selected dimensions. section 1.15 section 1.16
Machine learning discovers the vector values and organizes the dimensions from data rather than having engineers define every topic by hand. That makes embeddings useful but difficult to explain: a dimension may not correspond to one clear human concept. section 1.14
Vectors sit inside many AI pipelines. LLMs use them to represent tokens, CNNs produce them from images, and multimodal systems transform image vectors so they can be compared with text vectors. Third-party tools also generate embeddings for content retrieval. section 1.16 section 1.32 section 1.31
Figure 1.5 Each token is mapped to a vector of numbers. We can imagine that each number in the vector represents a topic. Here’s an imaginary list of topics and their respective numbers for the “dog” token.
Use the imagined dog vector to see how individual numbers can represent different aspects of meaning. section 1.13
Figure 1.7 We can think the numbers in an embedding vector as coordinates that place the token in a multidimensional “meaning space.”
Use the two-dimensional meaning space to visualize vectors as coordinates and related tokens as nearby points. section 1.15
