liveBook · Manning

Solvable machine learning problem

As explained in Natural Language Processing in Action, Second Edition

A solvable machine learning problem is a prediction task for which humans can answer the questions, most cases have a correct answer, and performance should not substantially exceed that of a trained human expert.

A solvable machine learning problem is a prediction task where humans can answer the questions, most cases have a correct answer, and a machine is not expected to substantially outperform a trained human expert. section 8.2

Test a proposed task against three conditions:

  • Can a human imagine answering the questions? section 8.2
  • Does a correct answer exist for most cases? section 8.2
  • Would expert human performance set a realistic upper benchmark? section 8.2

Predicting nationality from surnames

Surname nationality is imperfect but potentially solvable: people can make guesses, most labeled examples have a target nationality, and a well-built RNN pipeline may approach or exceed trained-expert accuracy. Ambiguous names such as Michel, Abel, and Martin reveal why certainty is impossible. section 8.2

>>> from nlpia2.string_normalizers import Asciifier
>>> asciify = Asciifier()

>>> asciify("O’Néàl")
"O'Neal"
>>> asciify("Çetin")
'Cetin'

Normalize surnames before classification by converting characters such as “O’Néàl” and “Çetin” to ASCII forms. section 8.2

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Defines three practical tests and applies them to surname classification, without detailing model training or evaluation metrics.
Examples
Uses asciified surnames, a 37-nationality dataset, oversampled names, and overlapping labels such as Michel and Martin.
What it assumes
Assumes labeled examples and a prediction pipeline while defining the solvability criteria from the task itself.
What you can do after
Evaluate whether a labeled prediction task has a defensible target and human-scaled accuracy expectation.
How it connects
Connects surname classification to text normalization, labeled data, and recurrent neural networks.
Cover of Natural Language Processing in Action, Second Edition
liveBookmanning.com
Read Natural Language Processing in Action, Second Edition on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.