Ambiguity is the property of a word or phrase having multiple plausible meanings or interpretations. Programming languages avoid it by enforcing one interpretation; natural-language systems must infer meaning from context, semantics, and statistical relationships. section 1.2 section 1.2.3
NLP systems approach ambiguity by:
- Representing extracted language information as numerical vectors, then applying linear algebra and machine-learning techniques. section 1.2.3
- Using context and statistical relationships between words instead of attempting to encode every possible grammatical and spelling rule. section 1.2.3
- Treating uncertain predictions as evidence of a difficult problem rather than assuming that a model’s top answer is uniquely correct. section 8.2.2
Ambiguous nationality labels
A surname can occur in multiple countries, so a name-nationality dataset may assign different labels to similar names across training and validation splits. An RNN trained on character sequences nevertheless generalized beyond random guessing: it reached 57% training accuracy and 29% validation accuracy, while random choice among 25 categories would reach 4%. For “Khalid,” the model returned several plausible alternatives, including Algerian, Moroccan, Malaysian, and Arabic. section 8.2.2
Listing 8.8 Training on a single sample
>>> def train_sample(model, category_tensor, char_seq_tens,
... criterion=nn.NLLLoss(), lr=.005):
""" Train for one epoch (one example name nationality tensor pair) """
... hidden = torch.zeros(1, model.n_hidden) #1
... model.zero_grad() #2
... for char_onehot_vector in char_seq_tens:
... category_predictions, hidden = model( #3
... x=char_onehot_vector, hidden=hidden) #4
... loss = criterion(category_predictions, category_tensor)
... loss.backward()
... for p in model.parameters():
... p.data.add_(p.grad.data, alpha=-lr)
... return model, category_predictions, loss.item()
Inspect the single-sample RNN training loop to see how a character sequence is processed with hidden state before producing a category prediction. section 8.2.2
