A static embedding assigns each word one fixed vector, regardless of its surrounding context. Cosine similarity compares vector directions, so nearby vectors suggest semantic similarity. section 3.2.2
This representation fails when meaning depends on context: “bank” receives the same vector for financial and river-related sentences. Averaging word vectors also creates a bag-of-words representation that loses order, syntax, and context. section 3.2.2
Use static embeddings when:
- Computational resources are constrained and a lightweight representation is sufficient. section 3.2.2
- Interpretability matters and inspecting a word’s fixed neighborhood is useful. section 3.2.2
- Context-sensitive evaluation is required; prefer contextual embeddings for that need. section 3.2.2
