A foundational model is a reusable model trained on a broad domain, such as everyday images or English text. It can turn unstructured items into embeddings: ordered arrays of numbers that position items in a conceptual space. Those vectors are more useful for exploring relationships or creating features than for reading directly. section 6.1.4
Treat a pretrained model as both an exploratory tool and a governed project artifact:
- Choose a model appropriate to the data—for example, EfficientNet for photos or a BERT-derived model such as all-MiniLM-L12-v2 for text—and generate embeddings for all or a reasonable subset of items. section 6.1.4
- Use the embeddings to inspect unstructured data during exploratory data analysis, or turn them into features for a later ML model. section 6.1.4 section 7.2
- Check the license, obtain customer approval, register the model in repositories and catalogs, and keep its version consistent across development, test, and production. section 5.5.1
Finding relationships in text
The all-MiniLM-L12-v2 sentence transformer can embed passages from Shakespeare’s plays. Similarity between those representations supports an exploration of related themes and a concept graph. The same approach can supply features from text for a subsequent modeling task. section 7.2
Figure 6.4 Heat map of nearest neighbor connections in Shakespeare’s plays using all-MiniLM-L12-v2
The heat map makes nearest-neighbor connections between plays visible rather than requiring inspection of the embedding arrays. section 6.1.4
A version mismatch is an operational risk: generating embeddings with one model and matching them with another generally produces poor results. A model identifier, such as an MD5 hash checked when serving code loads the model, can enforce the intended version. section 5.5.1
