liveBook · Manning

Stringr

As explained in Deep Learning with R, Third Edition

stringr is an R package providing tools for consistent string manipulation and pattern matching.

stringr is an R package for consistent string manipulation and pattern matching. It helps prepare text by splitting sentences into characters, words, or other tokens before converting them into numeric representations. section 14.2

Split a sentence into words

Use boundary("word") with str_split() to identify words. Preserve punctuation with skip_word_none = FALSE, then remove whitespace-only elements with str_subset("\\S"). The result can include both words and punctuation, ready for vocabulary matching. section 14.2

Common stringr splitting choices include:

  • Use str_split(x, "") or boundary("character") for Unicode-aware character splitting. section 14.2
  • Use boundary("word") for word splitting based on ICU word-boundary rules. section 14.2

How Deep Learning with R, Third Edition covers it

How deep it goes
Explains character and word splitting, punctuation preservation, regular-expression filtering, and vocabulary indexing, but defers regex instruction.
Examples
Uses a pangram to demonstrate character splitting, word splitting, punctuation retention, and vocabulary-to-index conversion.
What it assumes
Assumes basic R string handling and introduces token splitting, while directing regular-expression beginners elsewhere for deeper study.
What you can do after
Apply it via stringr to split text into usable character or word tokens and map tokens to vocabulary indices.
How it connects
Connects string manipulation with tokenization, regular expressions, Unicode, vocabulary matching, and text classification.
Cover of Deep Learning with R, Third Edition
liveBookmanning.com
Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.