liveBook · Manning

ASCII

As explained in Natural Language Processing in Action, Second Edition

ASCII is a character-encoding standard that assigns numeric codes to a limited set of Latin letters, digits, punctuation marks, and control characters.

ASCII is a character-encoding standard that assigns numeric codes to a limited set of Latin letters, digits, punctuation marks, and control characters. In multilingual text processing, converting text to pure ASCII—“asciifying” it—removes diacritics and typographic variations that could give a model irrelevant clues. section 8.2

For surname classification, asciifying O’Néàl as O'Neal and Çetin as Cetin helps a model learn spelling patterns rather than accents, cedillas, or curly punctuation. The resulting normalization supports broader generalization across Latin-script text, including transliterations. section 8.2

>>> from nlpia2.string_normalizers import Asciifier
>>> asciify = Asciifier()

>>> asciify("O’Néàl")
"O'Neal"
>>> asciify("Çetin")
'Cetin'

Run the Asciifier on names with diacritics and typographic punctuation to inspect their pure-ASCII forms. section 8.2

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Defines ASCII and demonstrates normalization, but does not detail the encoding table or character-code implementation.
Examples
Uses O’Néàl becoming O'Neal and Çetin becoming Cetin as executable normalization examples.
What you can do after
Apply it via nlpia2's Asciifier to normalize names and other Latin-script strings before modeling.
How it connects
Connects ASCII with Unicode, text normalization, natural language processing, machine learning, and Latin-script transliteration.

Also explained in

Compare how 2 books explain ASCII →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.