liveBook · Manning

Robustness

As explained in Machines that Think

Robustness is the ability of a system to maintain acceptable behavior under variation, disturbance, or unfavorable conditions.

Robustness is the capacity to preserve acceptable function when parts are lost or conditions vary. In neural networks, distributed representations spread contributions across units and connections, so damage need not immediately erase function. section 5.2.4

Pruning experiments illustrate this tolerance: some networks lost up to 90% of their weights before performance degraded noticeably, then declined gradually. The Lottery Ticket Hypothesis offers a related but distinct result: some sparse subnetworks can learn a task on their own. section 5.2.4

For generative systems, narrowing operational scope, using structured prompts and examples, monitoring outputs, stress-testing, and adjusting safeguards can improve robustness. These measures cannot remove variability: reliability depends on the surrounding system as well as the probabilistic model. section 6.3.5

How Machines that Think covers it

How deep it goes
Explains pruning results and operational safeguards, while noting that safeguards cannot eliminate probabilistic variability.
Examples
Uses weight pruning, sparse subnetworks, structured prompts, output review, and stress-testing as concrete illustrations.
What you can do after
Evaluate robustness by distinguishing tolerance to structural loss from safeguards that manage, but cannot eliminate, output variability.
How it connects
Connects distributed representation and neural-network pruning with prompting, monitoring, and reliability.

Also explained in

Compare how 12 books explain robustness →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.