liveBook · Manning

Invoke-WebRequest

As explained in Practical Automation with PowerShell: Effective scripting from the console to the cloud

Invoke-WebRequest is a PowerShell cmdlet that sends HTTP and HTTPS requests and processes web responses.

Invoke-WebRequest sends HTTP or HTTPS requests and returns the raw web response for PowerShell to process. Use it when an endpoint is not necessarily REST, when response handling must remain explicit, or when a web page needs request headers that another retrieval path does not supply. Unlike Invoke-RestMethod, it does not automatically convert response data into a PowerShell object, so parsing and conversion are deliberate follow-on steps. section 9.3

Start with the remote system’s interface documentation. REST describes an architectural style rather than a protocol or standard, and APIs can define their own headers, response patterns, and conventions. The same request mechanism can communicate with REST and non-REST HTTP services; choose the response-processing path based on the data returned and the automation’s needs. section 9.3

A practical web-data workflow separates request mechanics from extraction logic:

  • Build a URI from stable host and query components, such as a Microsoft Update Catalog search URL for a KB article. section 13.3.1
  • Inspect returned HTML and select the relevant nodes with XPath rather than relying on page appearance. section 13.3.1
  • Request detail pages with required headers, then pass the returned content to ConvertFrom-Html for extraction. section 13.3.1
  • Normalize extracted text with operations such as Replace, Trim, and Split before producing structured results. section 13.3.1

Request a localized update-details page

Some detail pages require an Accept-Language value before their HTML can be parsed successfully. Create a header hashtable, invoke the detail URI with that header, and send the response content to ConvertFrom-Html. Browser Developer Tools can reveal the Accept-Language value sent by a browser: open Network recording, load a KB detail link, select its request, and copy the header value that matches the desired language. section 13.3.1

$Headers = @{"accept-language"="en-US,en;q=0.9"}
$Request = Invoke-WebRequest -Uri $DetailsUri -Headers $Headers 
$Details = ConvertFrom-Html -Content $Request

This request-and-parse sequence shows the key boundary: Invoke-WebRequest obtains a header-sensitive response, and ConvertFrom-Html turns its content into a document that XPath queries can inspect. section 13.3.1

For downloads that have no no-wait option, run separate Invoke-WebRequest calls as PowerShell jobs. Start each download in a job, use Wait-Job to pause until all jobs finish, and use Receive-Job afterward when return information is needed. This keeps independent downloads concurrent while preserving an explicit synchronization point. section 6.1.2

How Practical Automation with PowerShell: Effective scripting from the console to the cloud covers it

How deep it goes
Explains raw-response handling and demonstrates headers, HTML parsing, XPath extraction, and parallel downloads; defers API-specific conventions to developer documentation.
Examples
Uses an ipify public-IP request, Microsoft Update Catalog detail pages with Accept-Language headers, and two concurrent file downloads.
What it assumes
Assumes PowerShell scripting familiarity while directing readers to API documentation and browser Developer Tools for service-specific request details.
What you can do after
Apply it via Invoke-WebRequest to retrieve raw responses, supply headers, parse returned HTML, and coordinate concurrent downloads with jobs.
How it connects
Connects HTTP, REST APIs, Invoke-RestMethod, ConvertFrom-Html, XPath, PowerShell jobs, and web scraping.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Practical Automation with PowerShell: Effective scripting from the console to the cloud. The section links point to the parts of the book it draws on.