A vector encodes text or another discrete object as continuous numerical values that neural networks can process. Embedding maps words, images, or documents into points in a continuous vector space; GPT-like models focus on word embeddings because they generate text one word at a time. section 2.1
Vectors support several embedding workflows:
- Use distinct embedding models for different data types, such as text, audio, and video. section 2.1
- Represent words, sentences, paragraphs, or documents, with word embeddings emphasized for one-word-at-a-time generation. section 2.1
- Generate word embeddings with approaches such as Word2Vec, which predicts contextual relationships. section 2.1
Figure 2.2 Deep learning models cannot process data formats like video, audio, and text in their raw form. Thus, we use an embedding model to transform this raw data into a dense vector representation that deep learning architectures can easily understand and process. Specifically, this figure illustrates the process of converting raw data into a three-dimensional numerical vector.
The figure traces raw text, audio, or video through an embedding model into a dense three-dimensional numerical vector. section 2.1
