How deeply does Modern Angular cover web crawler?

Defines crawling and traces the SPA indexing problem to initial HTML and JavaScript execution, without detailing crawler internals.

Web crawler

As explained in Modern Angular

A web crawler is an automated program that traverses web pages, extracts information, and discovers links for indexing or analysis.

A web crawler is an automated program that traverses web pages, extracts information, indexes keywords, and follows discovered links. Search engines use crawlers to find pages and connect them with searches. section 9.1.2

Crawlers perform an initial page load but do not execute client-side JavaScript. A single-page application can therefore expose an almost empty HTML document while JavaScript renders the useful content later, limiting discoverability. section 9.1.2

Search visibility for a legal website

A legal company in Albuquerque can gain search traffic when crawlers index its pages and associate them with terms such as “lawyer in ABQ.” section 9.1.2

Listing 9.1 index.html file of an Angular single-page application

<!doctype html>
<html lang="en">

<head>
  <meta charset="utf-8">
  <title>Hrms</title>
  <base href="/">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="icon" type="image/x-icon" href="favicon.ico">
  <link rel="stylesheet" href="styles.css">
</head>

<body>
  <app-root></app-root>
  <script src="runtime.js" type="module"></script>
  <script src="polyfills.js" type="module"></script>
  <script src="styles.js" defer></script>
  <script src="vendor.js" type="module"></script>
  <script src="main.js" type="module"></script>
</body>

</html>

Inspect the Angular SPA entry document: its app-root element contains no page content, while JavaScript bundles are responsible for rendering it. section 9.1.2

How Modern Angular covers it

liveBook manning.com

Read Modern Angular on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Grokking AI Algorithms, Second EditionA web crawler is software that automatically visits web pages and follows links to collect or index web content.

Compare how 2 books explain web crawler →

Related concepts

bloom filteruses · A Bloom filter is a space-efficient probabilistic data structure for testing set membership that permits false positives but not false negatives.
aiohttpuses · aiohttp is a Python library for asynchronous HTTP client and server applications built on asyncio.
Beautiful Soupuses · Beautiful Soup is a Python library for parsing HTML and XML documents, navigating their parsed structures, and extracting or modifying their contents.
social mediaused by · Social media is a class of online services that enables users to create, share, discover, and interact with content and with one another.
JavaScriptcontrasts with · JavaScript is a high-level, dynamically typed, multi-paradigm programming language standardized as ECMAScript and used for web and general-purpose programming.

This explainer was generated by Manning's knowledge graph from the text of Modern Angular. The section links point to the parts of the book it draws on.