A bit vector language model represents a phrase with binary dimensions answering simple questions such as “Does it contain the word good?” or “Does it contain the word morning?” Each zero-or-one answer becomes a coordinate in a large vector. Similar phrases can receive similar values, while clustering can bring related statements closer together across selected dimensions. section 1.6
From words to bits
Imagine millions of yes/no questions about a phrase. A dimension records whether a specified word or feature appears; the resulting vector gives a machine a programmable representation of language. A related character-level representation asks whether the first character is A, B, and so on, then repeats those questions for later positions, preserving exact character order. section 1.6
This approach made large vector representations practical as computing capacity and incremental, constant-RAM, linear-algebra algorithms improved. Its central trade-off is scale: richer questions create very high-dimensional vectors. section 1.6
