HTML structures web pages and carries their content, including headings, images, links, paragraphs, sections, styling references, and script references. Its elements use tags such as <p>, <a>, and <div>, which can nest in a tree-like document structure. section 10.2.1
For data work, an HTTP request to a web page returns HTML rather than JSON. The analyst must interpret the document, select relevant tags, and extract information that can be transformed into structured data. section 10.2.1 section 10.4
Inspecting a weather page
A weather forecast page can be requested with Python, then parsed with BeautifulSoup and the html.parser parser. Printing the prettified result exposes the nested HTML structure used to locate forecast values. section 10.2.1
HTML-based extraction requires attention to:
- Select information from relevant tags, such as headers, links, paragraphs, or sections. section 10.4
- Expect brittle processing when websites change their dynamic content or structure. section 10.4
- Follow website rules, including permitted pages and rate limits, to avoid blocked requests. section 10.4
