A data representation is a way of encoding information so a system can process it. The form matters: it determines what the system can observe and helps shape the patterns it can learn. In language processing, for example, a representation must contend with context and ambiguity, not just identify words. section 2.0 section 2.5
Tokenization breaks language into discrete, machine-readable units. Those units identify pieces of text, but do not by themselves express how the pieces relate. Numerical representations called embeddings address a different problem: capturing patterns of similarity, contrast, and association among tokens. section 2.0 section 2.5
Neural networks can learn representations rather than rely entirely on a predefined description of the data. Successive layers transform and refine inputs while learning to make predictions. With sufficient data and computation, networks can extract increasingly abstract features, such as edges, shapes, and objects. section 3.1.6 section 1.4.3
Useful representations capture patterns across examples, supporting responses to new data rather than merely reproducing familiar cases. This does not guarantee human-like understanding: capable behavior can arise from learned patterns without awareness. section 3.3 section 1.4.3
The strawberry test
A request to count the letter “r” in “strawberry” can expose a mismatch between a task and a model’s representation. Language models usually process tokens, which may be whole words or fragments rather than individual characters. Some newer models address the task with character-level steps or code, but the example illustrates why an apparently simple task can be brittle. section 9.4.4










