Moore's law

As explained in How Large Language Models Work

Moore's law is the empirical observation that the number of transistors on an integrated circuit historically doubled approximately every 18 to 24 months.

Moore's law is the observation that transistor counts on integrated circuits historically doubled about every 18 to 24 months. It describes a trend, not a guarantee of equal growth in overall computing power. section 7.1.1

Transistor counts can keep rising while frequency, power, single-threaded performance, and total computing grow more slowly. Costs, power, data, and infrastructure create diminishing returns, so bigger hardware cannot deliver boundless improvement. section 7.1.1

A misleading shortcut

If a chip moves from 10 billion to 20 billion transistors, Moore's law describes that increase. It does not prove that the whole system became twice as fast, because other constraints may limit performance. section 7.1.1

Figure 7.3 Moores’s law is a common example of boundless growth, but it is misleading. Transistors keep doubling, but frequency, power, single-threaded performance, and total computing do not. So the total system performance has not continued to double approximately every two years. Other similar factors will constrain LLM performance and affect capability over time. Used under CC4.0 license from https://github.com/karlrupp/microprocessor-trend-data.

Compare transistor growth with frequency, power, single-threaded performance, and total computing to see why the trend can mislead. section 7.1.1

How How Large Language Models Work covers it

How deep it goes

Defines the trend, distinguishes transistor counts from system performance, and connects both to diminishing returns and infrastructure constraints.

Examples

Uses transistor doubling and comparisons with frequency, power, single-threaded performance, and total computing.

What you can do after

Evaluate claims about computing growth without treating transistor counts as direct evidence of proportional system performance.

How it connects

Connects Moore's law with diminishing returns, S-curves, compute performance, GPUs, infrastructure, and LLM self-improvement.

liveBookmanning.com

Read How Large Language Models Work on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Fast Python: High performance techniques for large datasets
• 

  liveBookmanning.com
  Tech Startup ToolkitAn empirical observation that the number of transistors on a computer chip, along with its processing power, tends to double approximately every two years.
• 

  liveBookmanning.com
  Inside AIMoore's Law is the empirical observation that the number of transistors on an integrated circuit tends to increase exponentially, historically doubling about every two years.

Compare how 4 books explain Moore's law →

Related concepts

• computemeasures · Compute is the processing capacity and computational resources available to or consumed by a system when executing workloads.
• Edholm's lawcontrasts with · An empirical observation that telecommunications data rates tend to double approximately every 18 months.
• out-of-order executionproduces · Out-of-order execution is a processor technique that executes instructions when their operands and resources are available rather than strictly in program order.
• performancecontrasts with · Performance is the measurable effectiveness or operational behavior of a system under specified conditions, assessed through relevant qualities such as task results, speed, responsiveness, throughput, or resource use.

This explainer was generated by Manning's knowledge graph from the text of How Large Language Models Work. The section links point to the parts of the book it draws on.