Boilerplate stripping removes repetitive material that carries little meaning for the intended domain, such as licensing declarations, HTML tags, cookie-banner code, and website navigation menus. Removing it reduces token count and helps prevent a model from learning repeated, irrelevant patterns. section 11.7
A small cleaning step
Start with an excerpt of training text containing two lines unrelated to the story. Identify those lines as boilerplate, remove them, and retain the meaningful narrative. The cleaned result contains less noise and fewer tokens. section 11.7
Figure 11.10 Excerpt of training data from Alice’s Adventures in Wonderland
Compare the original training-data excerpt with the version showing boilerplate removed. section 11.7
