liveBook · Manning

Regular expression

As explained in Natural Language Processing in Action, Second Edition

A regular expression is a formal pattern notation for matching and extracting strings that satisfy specified character or sequence constraints.

A regular expression is a compact pattern for finding character sequences in text. It can recognize a greeting, identify a sentence boundary, or extract a value from an otherwise unstructured string. Think of a combination lock: a particular sequence makes it respond. A regular expression gives a program more flexible rules for which sequences count as a match. section 1.4.1 section 1.4.2

Patterns are useful when the form of the input is fairly predictable. In a chatbot, a match can trigger a scripted reply, while a captured part of the match can supply a name or other detail. Unlike a list containing every spelling of “Hi,” one pattern can accommodate repeated letters and several greeting forms. It still cannot anticipate every word, typo, or punctuation choice a person might use. section 1.4.2 section 1.5.2

Recognize a greeting

In Python, `import re; r = r'(hi|hello|hey)[ ,:.!]*([a-z]*)'; match = re.match(r, 'Hello Rosa', flags=re.IGNORECASE)` produces a match. The alternatives select a greeting, the punctuation group permits a separator, and the final group captures following letters. A chatbot can inspect captured groups before choosing its reply; a match alone does not establish what the speaker meant. section 1.5.2

Match the pattern to the job:

  • For sentence segmentation, a pattern can split on punctuation followed by whitespace. Check the resulting sentences: `re.split` can consume the terminating punctuation, while periods inside technical text and quotations make boundaries difficult. section 11.2.1 section 11.2.2
  • For information extraction, capture decimal latitude–longitude pairs in map URLs or prose. Predictable number formats are easier targets than names, places, and organizations without standard formats. section 11.4.1
Listing 11.2 A regular expression for GPS coordinates
>>> import re
>>> lat = r'([-]?[0-9]?[0-9][.][0-9]{2,10})'
>>> lon = r'([-]?1?[0-9]?[0-9][.][0-9]{2,10})'
>>> sep = r'[,/ ]{1,3}'
>>> re_gps = re.compile(lat + sep + lon)

>>> re_gps.findall('http://...maps/@34.0551066,-118.2496763...')
[(34.0551066, -118.2496763)]

>>> re_gps.findall("https://www.openstreetmap.org/#map=10/5.9666/116.0566")
[('5.9666', '116.0566')]

>>> re_gps.findall("Zig Zag Cafe is at 45.344, -121.9431 on my GPS.")
[('45.3440', '-121.9431')]

The Python listing combines latitude, separator, and longitude patterns, then uses `findall` to extract coordinate pairs from map links and a sentence. section 11.4.1

A pattern gives a definite match or nonmatch under its rules, not certainty about meaning. Adding rules for more language varieties takes manual work. Approximate matching can tolerate some errors; statistical NLP offers another approach when hardcoded patterns become brittle. Regular expressions express finite-state-style rules, although features in practical Python and POSIX implementations go beyond true regular grammars and need not share their halting guarantee. section 1.9 section 1.4.2 section 11.2.2

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Works through Python matching, capturing, splitting, and extraction; relates patterns to finite-state machines without developing their mathematics.
Examples
Tests Python greeting patterns, sentence splits, and latitude–longitude extraction from map links and prose.
What it assumes
Introduces formal matching through a lock analogy and supplies Python code without expecting readers to devise every pattern unaided.
What you can do after
Build it from scratch: write simple Python patterns for greetings, sentence boundaries, and coordinate pairs while checking where matches fail.
How it connects
Connects regular grammars and finite-state machines to intent recognition, sentence segmentation, information extraction, and statistical NLP.

Also explained in

Compare how 27 books explain regular expression →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.