liveBook · Manning

Large-scale engineering

As explained in Sutskever's List

Large-scale engineering is an approach to AI development that combines substantial computation, data, systems infrastructure, and implementation effort to achieve practical performance.

Large-scale engineering treats AI progress as a practical systems problem: combine substantial computation, data, infrastructure, and implementation effort to produce measurable performance. Its landmark examples include AlexNet, Deep Speech 2 (DS2), and AlphaGo. section 4.4.5

The approach contrasts with theory-led work such as backpropagation, LSTM, and Hessian-free optimization. DS2 added little methodological novelty, according to reviewers; its contribution lay in scaling and implementation. AlphaGo’s 2016 victory over Lee Sedol signaled a field becoming more collaborative, resource-intensive, and dependent on advanced GPU infrastructure and massive datasets. section 4.4.5

The central engineering trade-off is:

  • Prefer conceptual novelty when the main goal is mathematical or theoretical insight. section 4.4.5
  • Prefer large-scale implementation when existing techniques can be combined to solve tangible problems at useful performance. section 4.4.5

How Sutskever's List covers it

How deep it goes
Contrasts methodological novelty with scaling and implementation, but stops short of detailed infrastructure procedures.
Examples
Uses AlexNet, Deep Speech 2, AlphaGo, and AlphaGo’s match against Lee Sedol as concrete cases.
What you can do after
Evaluate AI advances by separating conceptual novelty from measurable gains achieved through scale and implementation.
How it connects
Connects deep learning with datasets, GPU infrastructure, backpropagation, LSTM, Hessian-free optimization, and AlphaGo.
Cover of Sutskever's List
liveBookmanning.com
Read Sutskever's List on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Sutskever's List. The section links point to the parts of the book it draws on.