Need to find records in messy text or split on several kinds of separator? A regular expression (regex) describes the character sequences to match. It becomes useful when ordinary string methods would require several passes or cannot express the pattern clearly. In Python, the standard-library re module can search with a pattern directly or compile one for repeated use. section 2.3.3 section 2.4 section 2.4.1
Build a pattern from the shape of the data:
- Use character classes such as \d for a digit and \w for a word character; [,_] matches either separator. A raw string, such as r"\d+", keeps Python’s string escaping from obscuring the regex. section 2.4.2 section 2.4.3 section 2.4
- Use ^ and $ for string boundaries, and quantifiers such as ? (zero or one), * (zero or more), + (one or more), or {3} (exactly three). Quantifiers normally match as much as possible; adding ? makes them non-greedy. section 2.4.3
- Use parentheses to group parts you want to extract. Prefer named groups when several fields would otherwise be hard to identify. section 2.5.1 section 2.5.4
Find task records among unrelated lines
A valid line has a three-digit ID, a comma and space, a one-word title, a semicolon and space, and a description. Test that shape before processing its fields: import re pattern = re.compile(r"(\d{3}), (\w+); (.+)") for line in ["101, Homework; Complete physics and math", "some random nonsense", "54, random; record"]: match = pattern.match(line) print(match.group() if match else "No match") Output: 101, Homework; Complete physics and math No match No match Here, match checks the beginning of each line. If the entire line must fit, consider an end anchor too. section 2.5.1 section 2.4.5 section 2.4.3
Table 2.3 Common regular expression methods (view table figure)
| search: Returns a Match if a match is found anywhere in the string. |
||
| match: Returns a Match only if a match is found at the string’s beginning. |
||
| findall: Returns a list of strings that match the pattern. When the pattern has multiple groups, the item is a tuple. |
||
| finditer: Returns an iterator3 that yields the Match objects. |
||
| sub: Creates a string by replacing the matched with the replacement. |
Compare search and match for one result, findall and finditer for multiple results, split for separators, and sub for replacement. section 2.4.5
Start with the required data shape, then create a pattern, inspect its matches on representative text, and revise it for edge cases. Use simpler methods such as startswith or endswith when they express the requirement clearly; a regex that merely works is not necessarily the easiest solution to maintain. section 2.4.6 section 2.4.3
