Chapter 12: Wrangling regular expressions
The chapter introduction explains why regular expressions are useful for searching and parsing strings, while cautioning that some patterns call for a full parser instead.
After this chapter you will be able to
- Concise string pattern matching. Explain how a regular expression can replace lengthy parsing logic when extracting information from strings.
- Limits of regex parsing. Explain why a developer should not use a regular expression for every string-parsing task and when a full-fledged parser may be preferable.
- Concise pattern validation. Explain why the section presents a regular expression as an alternative to a loop for validating a nine-digit ZIP code: both check for five decimal digits, a hyphen, and four decimal digits, but the regular-expression solution is much shorter.
- Regexes across languages. Explain that regular expressions are not specific to JavaScript, and that JavaScript supports Perl-style slash-delimited regular expression literals for readability.
- Regex creation choices. Distinguish a forward-slash-delimited regex literal from a RegExp constructor call that takes the pattern as a string, and explain why literals are generally preferred when the pattern need not be built dynamically.
- Dynamic patterns and user input. Explain when the RegExp constructor is needed for a pattern built at runtime, and distinguish the risk of accepting an arbitrary user regex from safely including user text as a literal part of a pattern.
- Common regex flag effects. Describe how the g, i, m, and s flags change matching, and state how flags are supplied with a literal or a RegExp constructor call.
- Literal terms and escaping. Explain how ordinary characters form sequential literal matches and how to match a metacharacter literally, including the extra escaping needed in a JavaScript string.
- Character class choices. Describe how bracketed classes match one character from a set or range, how a leading caret negates a class, and why predefined classes are useful.
- Repetition count operators. Explain the repetition counts expressed by ?, +, *, and braces, including fixed, bounded, and open-ended counts.
- Greedy and nongreedy repetition. Distinguish default greedy repetition from nongreedy repetition and explain how appending ? changes the bracketed-text example.
- Grouping alternation and backreferences. Explain how parentheses let operators act on a sequence, how | expresses alternatives, and how a numbered backreference requires the same text captured earlier.
- Compilation and execution phases. Distinguish compiling a regular expression from executing it, using the regex literal and .test(city) example.
- Regex reuse and its limits. Explain why the section suggests creating a regex once outside a loop, why the gain is usually small, and what caution applies to global or sticky regexes.
- Matching method choices. Distinguish the results of test, exec, match, and matchAll, and state the section’s rule of thumb for choosing among them.
- Value of captured matches. Explain the difference between checking whether a string matches a pattern and capturing what was matched, and why the latter can be useful.
- Capture groups and match indexes. Explain how parentheses capture parts of a regex match, such as the hours and minutes in a time value, and how those parts appear in the match array. Describe how capture indexes are assigned when groups are nested.
- Optional and non-capturing groups. Distinguish a non-capturing group from a capturing group and explain what happens when a group is made optional. Apply both ideas to the ninja-name example with an optional clan.
- Captures in global matches. Distinguish the results of match with a local regex from its results with a global regex, and explain why matchAll is useful when repeated matches need captures.
- Lazy iteration over matches. Explain how iterating over matchAll results with for…of differs from collecting every result at once.
- Backreferences for exact repetition. Explain how a numbered backreference refers to an earlier captured substring within the same regex. Describe how the example captures a word and matches that same word again after a space.
- Replacer function match information. Explain when a replacer function runs and describe the match information it receives.
- Hyphenated text transformation. Trace how the section’s global regex replacer turns each hyphen followed by a lowercase letter into an uppercase letter.
- Zero-length regex assertions. Explain how zero-length assertions affect whether a regex matches without adding characters to the resulting match, using the section’s boundary examples.
- Nonconsuming lookahead checks. Explain that a lookahead checks a pattern at the current position without consuming the characters it finds, so matching can continue from that position.
- Current-position lookahead anchoring. Explain why the names example uses `.*` inside its lookahead to find `katana` or `shuriken` later in the string. Trace why the global regex returns both Yuri and Eri.
- Three-digit comma placement. Trace how the number-formatting regex selects digits followed by one or more complete groups of three digits through the end of the string, and how replacement adds commas after those digits.
- Positive lookbehind and position. Explain how positive lookbehind checks preceding text at the current match position, using the word-character or hyphen example.
- Negative lookbehind for strike. Trace why the negative lookbehind in the strike example permits the initial Strike but rejects a strike immediately preceded by not and whitespace.
- Greedy matching and backtracking. Trace how a greedy regex term gives up characters when a later term fails to match.
- Catastrophic backtracking mechanism. Explain why the repeated group (?:\w+\s*)+ can take exponentially longer on a string ending in an unmatched !.
- Regex safety choices. Describe the safeguards and tradeoffs the section gives for risky regex use, including DFA engines, sandboxing, and choosing a parser instead.
What it builds on
- Arrays (chapter 10) · Arrays can be accessed by index.
Section by section
Introduction
The chapter introduction explains why regular expressions are useful for searching and parsing strings, while cautioning that some patterns call for a full parser instead.
- Concise string pattern matching: Explain how a regular expression can replace lengthy parsing logic when extracting information from strings.
Why it matters: This is why regular expressions are worth learning for JavaScript development. - Limits of regex parsing: Explain why a developer should not use a regular expression for every string-parsing task and when a full-fledged parser may be preferable.
Why it matters: Knowing the limit helps you avoid forcing a complex parsing task into an unsuitable regex.
Key terms: regular expression, pattern, parsing logic, string patterns, full-fledged parser
12.1 Why regular expressions rock
A nine-digit ZIP code example shows how a regular expression can replace a longer loop for checking a string’s format.
- Concise pattern validation: Explain why the section presents a regular expression as an alternative to a loop for validating a nine-digit ZIP code: both check for five decimal digits, a hyphen, and four decimal digits, but the regular-expression solution is much shorter.
Why it matters: This example shows why regular expressions are useful for checking string patterns.
Key terms: regular expression, ZIP code, decimal digit
12.2 A regular expression refresher
This introduction places regular expressions in a cross-language context and notes JavaScript’s readable syntax for regex literals.
- Regexes across languages: Explain that regular expressions are not specific to JavaScript, and that JavaScript supports Perl-style slash-delimited regular expression literals for readability.
Why it matters: It separates the broadly shared idea of regular expressions from a syntax JavaScript offers for writing them.
Key terms: regular expressions, /-delimited syntax, regular expression literals
12.2.1 Regular expressions explained
This section defines regular expressions as text-matching patterns and explains how JavaScript creates them, when to use each creation form, and what regex flags change.
- Text matching patterns: Explain that a regular expression expresses a pattern for matching strings of text, using terms and operators to define the pattern.
Why it matters: This gives you the basic purpose of the regexes used throughout the chapter. - Regex creation choices: Distinguish a forward-slash-delimited regex literal from a RegExp constructor call that takes the pattern as a string, and explain why literals are generally preferred when the pattern need not be built dynamically.
Why it matters: Choosing the right creation form keeps ordinary patterns readable while leaving room for patterns built at runtime. - Dynamic patterns and user input: Explain when the RegExp constructor is needed for a pattern built at runtime, and distinguish the risk of accepting an arbitrary user regex from safely including user text as a literal part of a pattern.
Why it matters: It separates the useful ability to build searches at runtime from the danger of treating user input as regex syntax. - Common regex flag effects: Describe how the g, i, m, and s flags change matching, and state how flags are supplied with a literal or a RegExp constructor call.
Why it matters: Flags can change a pattern's results without changing the pattern itself. - Specialized regex flag effects: Describe the roles of the d, u, v, and y flags at the level introduced here, including the recommendation to use v for regexes containing non-ASCII characters.
Why it matters: These flags identify additional matching and result behaviors without requiring their later mechanics yet.
Key terms: regular expression, regex, pattern, terms, operators, regex literal, RegExp constructor, RegExp instance, backslash, dynamically at runtime
12.2.2 Terms and operators
This section explains how common regex terms and operators combine to match characters, positions, repetitions, and captured text.
- Literal terms and escaping: Explain how ordinary characters form sequential literal matches and how to match a metacharacter literally, including the extra escaping needed in a JavaScript string.
Why it matters: This lets you tell apart a character to match from syntax that changes a pattern’s meaning. - Character class choices: Describe how bracketed classes match one character from a set or range, how a leading caret negates a class, and why predefined classes are useful.
Why it matters: Character classes express alternatives for a single position without listing every possibility separately. - Start and end anchors: Distinguish a match at the beginning or end of a string from a match required to cover the whole string.
Why it matters: Anchors prevent a substring match from being mistaken for a match of the intended whole input. - Repetition count operators: Explain the repetition counts expressed by ?, +, *, and braces, including fixed, bounded, and open-ended counts.
Why it matters: These operators let a compact pattern state exactly how many occurrences it permits. - Greedy and nongreedy repetition: Distinguish default greedy repetition from nongreedy repetition and explain how appending ? changes the bracketed-text example.
Why it matters: Greediness explains why a pattern may span farther through text than expected. - Grouping alternation and backreferences: Explain how parentheses let operators act on a sequence, how | expresses alternatives, and how a numbered backreference requires the same text captured earlier.
Why it matters: These features let a pattern repeat whole sequences, offer alternatives, and require a later match to agree with an earlier one.
Key terms: literal, metacharacters, escaping, regex literal, character class, set, range, predefined character classes, anchor, caret
12.3 Compiling regular expressions
Explains how regex compilation differs from execution, why a regex may be reused, and how to choose a matching method and read its result.
- Compilation and execution phases: Distinguish compiling a regular expression from executing it, using the regex literal and .test(city) example.
Why it matters: The distinction makes it easier to reason about when regex work happens. - Regex reuse and its limits: Explain why the section suggests creating a regex once outside a loop, why the gain is usually small, and what caution applies to global or sticky regexes.
Why it matters: It keeps a small performance habit in perspective and flags a case where reuse needs care. - Matching method choices: Distinguish the results of test, exec, match, and matchAll, and state the section’s rule of thumb for choosing among them.
Why it matters: Choosing a method determines whether you get a boolean, a match result, or a way to iterate over results. - Matched text in results: Describe how to obtain the matched text from a non-null simple match result.
Why it matters: It lets you retrieve the text found by a match rather than just checking whether one occurred.
Key terms: compilation, execution, regex literal, hoisting, compiled regular expressions, global, sticky, test, exec, match
12.4 Capturing matching segments
The section explains why capturing what a regular expression matched can be more useful than merely checking whether it matched.
- Value of captured matches: Explain the difference between checking whether a string matches a pattern and capturing what was matched, and why the latter can be useful.
Why it matters: It clarifies why you might want a regex to return a result rather than just a yes-or-no answer.
Key terms: capture, matches, pattern
12.4.1 Performing simple captures
Explains how capture groups extract parts of a regex match, including how nested, optional, and non-capturing groups affect the returned match array.
- Capture groups and match indexes: Explain how parentheses capture parts of a regex match, such as the hours and minutes in a time value, and how those parts appear in the match array. Describe how capture indexes are assigned when groups are nested.
Why it matters: Captures let you extract useful pieces without matching a whole value and then splitting it separately. - Optional and non-capturing groups: Distinguish a non-capturing group from a capturing group and explain what happens when a group is made optional. Apply both ideas to the ninja-name example with an optional clan.
Why it matters: These group forms let a pattern allow optional text while returning only the pieces you want to extract.
Key terms: capture groups, match object, full match, non-capturing group, optional, capture group, undefined
12.4.2 Repeated matching using global expressions
This section shows how to get captures from repeated regex matches with matchAll, interpret the example pattern, and consume results lazily or all at once.
- Captures in global matches: Distinguish the results of match with a local regex from its results with a global regex, and explain why matchAll is useful when repeated matches need captures.
Why it matters: Choosing matchAll lets you extract captured parts from every occurrence, rather than only getting the full matches. - Lazy iteration over matches: Explain how iterating over matchAll results with for…of differs from collecting every result at once.
Why it matters: Lazy iteration avoids computing every match up front when you can process results one by one. - Example character class pattern: Explain what [^\s,]+ matches and how the g flag affects the example regex.
Why it matters: Reading the pattern makes the repeated-match example easier to follow.
Key terms: local regular expression, global regular expression, g flag, match, matchAll, capturing groups, for…of loop, lazily, array rest syntax, character class
12.4.3 Referencing captures within a regular expression
This section shows how a regex backreference matches an exact repeated word and why word boundaries keep that match from occurring inside longer words.
- Backreferences for exact repetition: Explain how a numbered backreference refers to an earlier captured substring within the same regex. Describe how the example captures a word and matches that same word again after a space.
Why it matters: A backreference lets a regex require an exact repetition, not just two words that fit the same pattern. - Word boundaries around repetition: Explain what \b matches and why the example uses word boundaries to avoid matching repeated text embedded in longer words.
Why it matters: Word boundaries prevent an apparent repetition from being accepted when one occurrence is only part of a longer word.
Key terms: capturing group, backreference, \1, \b, word boundary, anchor
12.5 String replacement with regexes
This section shows how to use a regex and a replacer function with String.replace to transform matched text without changing the original string.
- Pattern replacement and immutability: Distinguish replacing an exact substring from using a regex to replace a pattern, and explain what replacement means for an immutable string.
Why it matters: It clarifies both what a regex adds to replace and why the original string stays unchanged. - Replacer function match information: Explain when a replacer function runs and describe the match information it receives.
Why it matters: Those arguments let the replacement depend on what the regex matched. - Hyphenated text transformation: Trace how the section’s global regex replacer turns each hyphen followed by a lowercase letter into an uppercase letter.
Why it matters: The example shows how a capture can drive a concise transformation across a string. - Replacer parameter typing: Explain why the example names an unused first parameter and annotates its captured-letter parameter in TypeScript.
Why it matters: It explains two otherwise puzzling details of the example’s callback signature.
Key terms: replace, regex, immutable, replacer function, match, captures, source string, character class, g flag, capture
12.6 Lookahead, lookbehind, and backtracking
The section explains zero-length regex assertions and introduces lookahead and lookbehind as more general ways to use them.
- Zero-length regex assertions: Explain how zero-length assertions affect whether a regex matches without adding characters to the resulting match, using the section’s boundary examples.
Why it matters: This distinction explains why a regex can check a boundary without adding that boundary to the matched text. - Generalized lookaround assertions: Describe the relationship of lookahead and lookbehind to zero-length assertions and what they let a regex check.
Why it matters: It connects simple boundary checks to the broader purpose of lookahead and lookbehind.
Key terms: zero-length assertions, ^, $, \b, word boundary, match string, lookahead, lookbehind
12.6.1 Sneaking a peek with lookahead
This section explains how lookaheads test text without consuming it, then uses them to find names before later words and to insert number-grouping commas.
- Nonconsuming lookahead checks: Explain that a lookahead checks a pattern at the current position without consuming the characters it finds, so matching can continue from that position.
Why it matters: This lets you require text to appear later without including that later text in the match. - Current-position lookahead anchoring: Explain why the names example uses `.*` inside its lookahead to find `katana` or `shuriken` later in the string. Trace why the global regex returns both Yuri and Eri.
Why it matters: It explains the difference between checking for a word immediately after a name and checking for one anywhere later. - Three-digit comma placement: Trace how the number-formatting regex selects digits followed by one or more complete groups of three digits through the end of the string, and how replacement adds commas after those digits.
Why it matters: It shows how a nonconsuming condition can choose insertion points during string replacement. - Repeated lookahead scan cost: Explain why the number-formatting lookahead can be inefficient and how the section weighs that cost for modest-sized inputs.
Why it matters: A concise regex can repeat work, but whether to optimize depends on the scale of the input. - Negative lookahead assertion: Distinguish negative lookahead from the lookahead used in the examples, including its stated syntax change.
Why it matters: It gives you the opposite assertion when a match requires a pattern not to occur.
Key terms: lookahead assertion, doesn’t consume, lookahead, anchored, current match position, global regex, .*, \d{3}, $, replace function
12.6.2 Achieving hindsight with lookbehind
This section explains how positive and negative lookbehind test text immediately before a regex match, and why extending that test across all preceding text can be costly.
- Positive lookbehind and position: Explain how positive lookbehind checks preceding text at the current match position, using the word-character or hyphen example.
Why it matters: It lets you require preceding context without treating that context as the character being matched. - Negative lookbehind for strike: Trace why the negative lookbehind in the strike example permits the initial Strike but rejects a strike immediately preceded by not and whitespace.
Why it matters: The example shows why a negative lookbehind excludes nearby context, not every occurrence of that context earlier in the string. - Broader lookbehind performance cost: Explain the trade-off described when .* is added so a lookbehind searches all preceding text.
Why it matters: Searching farther back may change which text qualifies, but it also makes the regex substantially more expensive.
Key terms: lookbehind, assertion, (?<=\w), current match position, negative lookbehind, (?<!not\s+), implicit anchor, case-insensitive flag, .*, linear search
12.6.3 Backtracking and regex performance
This section explains how regex backtracking works, why some patterns become dangerously slow, and how to reduce that risk.
- Greedy matching and backtracking: Trace how a greedy regex term gives up characters when a later term fails to match.
Why it matters: This explains why a greedy match can still leave characters for the rest of a pattern. - Catastrophic backtracking mechanism: Explain why the repeated group (?:\w+\s*)+ can take exponentially longer on a string ending in an unmatched !.
Why it matters: A short input that cannot match can demand vastly more work than one that does. - ReDoS performance risk: Describe how catastrophic backtracking can become a denial-of-service vulnerability.
Why it matters: Regex performance is a security concern, not merely a matter of making a match a little faster. - Regex safety choices: Describe the safeguards and tradeoffs the section gives for risky regex use, including DFA engines, sandboxing, and choosing a parser instead.
Why it matters: The safest choice depends on whether you need regex features, must accept user-supplied patterns, or should use a parser instead.
Key terms: state machine, greedy operator, backtracking, catastrophic backtracking, non-capturing group, zero-length sequence, $ anchor, ReDoS, denial of service, DFA
12.7 Summary
Concepts explained in this chapter
- regular expressionA pattern notation used to match and manipulate text.
- regex groupingA capturing group is a parenthesized regular-expression subpattern whose matched text is stored for later extraction or reference.
- regular-expression anchorAn anchor is a regular-expression construct that constrains a match to a position such as the start or end of a string.
- character classA character class is a regular-expression construct that matches one character from a specified set or range.
- regex captureRegular expression capture is the extraction of text matched by capturing groups for inspection or processing.
- greedy matchingGreedy repetition consumes as many consecutive characters satisfying its term as possible while still allowing the overall match.
- non-greedy quantifierA non-greedy operator is a regular-expression quantifier that initially matches as little text as possible while allowing the pattern to match.
- quantifierA repetition operator is a regular-expression quantifier that specifies how many times a preceding pattern may occur.
- backreferenceA backreference is a regular-expression construct that matches text previously captured by a capturing group.
- non-capturing groupA non-capturing group is a parenthesized regular-expression subpattern that groups terms without storing its matched text as a capture.
- backtrackingRegex backtracking is a matching strategy that revisits earlier choices to try alternative paths when later matching fails.
- regex operatorA regex operator is syntax that modifies the matching behavior of a regular-expression term or group.
- regex termA regex term is a literal, character class, group, or other pattern unit that can be matched by a regular expression.
- catastrophic backtrackingCatastrophic backtracking is regex backtracking whose possible paths grow exponentially and can exhaust execution time or block a thread.
- lookaheadA lookahead is a regular-expression assertion that checks for a pattern immediately after the current position without consuming text.
- alternationRegex alternation is syntax that allows a match to satisfy one of multiple alternative pattern branches.
- metacharacterA regex metacharacter is a character with special syntactic meaning in a regular expression.
- regex escapingRegex escaping is the use of a backslash to make a special character match literally or to represent a control character.
- lookbehind assertionA lookbehind is a regular-expression assertion that checks for a pattern immediately before the current position without consuming text.
- deterministic finite automatonA DFA is a deterministic finite automaton that processes each input through a single state transition path without backtracking.
- node-re2node-re2 is a Node.js native binding to the RE2 regular-expression engine, which uses deterministic matching without backtracking.
- redosReDoS is a denial-of-service vulnerability in which crafted input causes a regular-expression matcher to consume excessive computational resources.
- regex engineA regex engine is software that interprets regular-expression patterns and searches or matches text according to them.
- String.prototype.matchAllString.prototype.matchAll is a JavaScript String method that returns an iterator over all matches of a regular expression in a string.
- match methodString.prototype.match is a JavaScript String method that retrieves the result of matching a string against a regular expression.
- regular expression literalJavaScript literal syntax that creates a regular expression using slash delimiters and optional flags.
- RegExp constructorThe RegExp constructor is the JavaScript built-in constructor that creates regular expression objects from a pattern and optional flags.
- regular expression flagA modifier that changes the matching behavior or returned information of a regular expression.
- negative lookahead assertionA negative lookahead is a regular-expression assertion that succeeds when its specified pattern does not occur ahead of the current position.
- replace methodString.prototype.replace is a JavaScript String method that returns a new string with matched text replaced by specified text or the result of a replacement function.
- assertionAn assertion is a regular-expression condition that checks surrounding text without consuming it as part of the match.
- negative lookbehindA negative lookbehind is a regular-expression assertion that succeeds when its pattern does not match immediately before the current position.
- global regular expressionA regular expression configured to find successive matches throughout its input.
- callbackA function supplied to another computation to be invoked when a specified event occurs or operation completes.
- assertionA zero-length assertion is a regular-expression construct that tests a condition at a position without consuming any characters in the match.
- type hintsSyntax attached to a TypeScript declaration or expression that explicitly specifies expected types and can supply information for type checking and inference.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
