A partial derivative measures how a multivariable function changes along one coordinate while the other coordinates remain fixed. For an n-dimensional function, each coordinate contributes one partial derivative; together they form the gradient, a vector pointing toward fastest growth. section 16.3.1
Gradient descent computes these derivatives coordinate by coordinate and updates each variable in the opposite direction of its derivative. Large derivatives produce larger steps, while small derivatives produce smaller steps. section 16.3.2
Applying partial derivatives in optimization requires:
- A cost function differentiable near the points where gradients are computed. section 16.3.3
- Exact derivative formulas when the cost function is known, or numerical estimates based on increasingly small coordinate increments. section 16.3.3
- A stopping rule, such as reaching a point where every partial derivative is zero or below a chosen precision. section 16.3.2
Linear regression
For a linear model, the loss function depends on parameters such as slope m and intercept b rather than directly on the dataset points. Partial derivatives with respect to m and b determine how gradient descent updates the model. section 16.4.1
Listing 16.5 Method forceDirectedEmbedding
function forceDirectedEmbedding(graph, alpha, maxSteps) #1
for v in graph.vertices do #2
(x[v], y[v]) ← randomVertexPosition()
for _ in {1..maxSteps} do #3
for v in graph.vertices do #4
x1[v] ← x[v] – alpha * derivative(graph, v, x) #5
y1[v] ← y[v] – alpha * derivative(graph, v, y)
if x == x1 and y == y1 then #6
break
(x,y) ← (x1, y1) #7
return (x,y)
Inspect a force-directed embedding implementation that updates each vertex's x and y coordinates with their corresponding partial derivatives. section 16.5.2
