Continuous smooth function

As explained in How Large Language Models Work

Smoothness is a mathematical property in which a function changes continuously or without abrupt, unstable changes over similar inputs.

A smooth function changes gradually: similar inputs produce outputs that do not fluctuate wildly. In machine learning, smoothness is one requirement for a useful loss function, alongside specificity and computability. section 4.1.1

Figure 4.2 Examples of a smooth function on the left and two nonsmooth functions on the right. The center example is mostly smooth, but one region is not smooth because the function has no value. On the right, the function is not smooth anywhere due to the hard change in value.

Compare a smooth graph with functions that contain a missing value or a hard change; these disruptions make nearby inputs behave less predictably. section 4.1.1

Investment returns

Investment returns are difficult to predict partly because they are not smooth, illustrating why abrupt variation complicates optimization. section 4.1.1

How How Large Language Models Work covers it

How deep it goes

Defines smoothness through output stability for similar inputs and contrasts it with missing values and hard changes.

Examples

Uses investment returns and graphs containing gradual, discontinuous, or abruptly changing behavior.

What you can do after

Evaluate whether a proposed loss function changes predictably enough to support optimization.

How it connects

Connects smoothness to loss functions, specificity, computability, and machine learning.

liveBookmanning.com

Read How Large Language Models Work on liveBook Get the book at manning.com

Related concepts

• linear approximationuses · Linear approximation represents a differentiable function near a point with a linear function determined by its value and first derivative there.
• continuous functionkind of · A function whose output changes arbitrarily little when its input changes arbitrarily little.

This explainer was generated by Manning's knowledge graph from the text of How Large Language Models Work. The section links point to the parts of the book it draws on.