Machine learning builds systems for tasks that are difficult to specify with rules by deriving useful patterns from examples. A cat detector, for instance, learns from many cat and non-cat images rather than receiving a complete description of whiskers, shapes, and exceptions. section 1.4
The approach can use carefully designed algorithms, such as convolutional or recurrent neural networks, or simpler algorithms trained on large amounts of data. Large language models favor a relatively simple strategy that captures many relationships with vast data and parameters, trading deployment cost for flexibility and performance. section 1.7
From text to useful vectors
An LLM converts text into embeddings: numerical vectors that preserve aspects of meaning. Similar texts can land near each other even when they use different words, allowing clustering, outlier detection, and other machine-learning methods to operate on language. section 8.3.1
Figure 8.4 LLMs produce numeric vectors known as embeddings as an intrinsic part of their function-ing. The utility of these embeddings is dependent on the fact that these numbers only change a little bit when given similar text. The two example tests here will have similar embeddings, and thus, their plots look similar, even though they don’t share any of the same words. This is a powerful feature that was present in older machine learning techniques.
Use the embedding plot to see how similar meanings can produce nearby numerical representations, then connect those representations to clustering and outlier detection. section 8.3.1
