A response can read fluently and still miss what the words mean. Semantics concerns meaning in language: the meaning carried by expressions, not just whether they are arranged in plausible syntax. It is one part of interpretation alongside syntax, pragmatics, morphology, and phonetics. For an LLM product, that distinction matters because convincing prose is not, by itself, evidence of reliable understanding. section 2.1.1 section 2.4
In a language model, the embedding layer attempts to give otherwise meaningless token vectors useful representations based on patterns in text, including word order and words appearing in similar contexts. These representations can resemble human judgments of meaning, but an embedding does not create semantics outright. Larger models make richer approximations; they do not remove the gap between modeled usage and human interpretation. section 2.2.5 section 2.4
A relationship captured in vectors
A familiar Word2Vec example takes the vector for “king,” subtracts “man,” and adds “woman”; the nearest resulting word is “queen.” The relationship is useful precisely because it resembles a human semantic judgment. It remains a relationship learned from usage, not a guarantee that the model interprets every sentence as a person would. section 2.2.5
Keep semantics separate from pragmatics when assessing a result. The literal meaning of “I need food” does not establish that the speaker is in physical danger; people use context to interpret the statement. Likewise, supplying context in a prompt or at inference time can improve an LLM’s response without giving it a human pragmatic system. Evaluate the meaning of outputs, not only their fluency. section 2.2.5 section 2.4
Listing 2.5 Embedding visualization
words = [ #1
"King",
"Queen",
"Lord",
"Man",
"Woman",
"Prince",
"Ophelia",
"Rich",
"Happy",
]
embs = (W1.T + W2) / 2.0
idx = [word2Ind[word] for word in words]
X = embs[idx, :]
print(X.shape, idx)
result = compute_pca(X, 2)
pyplot.scatter(result[:, 0], result[:, 1])
for i, word in enumerate(words):
pyplot.annotate(word, xy=(result[i, 0], result[i, 1]))
pyplot.show()
Plotting selected word embeddings offers a practical check during pretraining: nearby representations for semantically similar words suggest progress, while the plot remains an inspection aid rather than proof of understanding. section 2.2.5
