liveBook · Manning

Cotangent

As explained in Deep Learning with JAX

A reverse-mode sensitivity value associated with a variable, representing the derivative of an output with respect to that variable.

A cotangent is a reverse-mode sensitivity: the derivative of an output with respect to a variable. It shows how output changes flow backward through a computation. section 4.3.3

Reverse-mode autodiff uses cotangents in a backward pass:

  • Runs the function forward while recording intermediate values and dependencies. section 4.3.3
  • Starts the output cotangent at 1.0, then propagates sensitivities from outputs toward inputs. section 4.3.3
  • Sums contributions when a variable affects the output along multiple paths. section 4.3.3

Cotangents with vjp()

For f(x1, x2) = x1*x2 + sin(x1*x2) at (7.0, 2.0), jax.vjp returns the primal output 14.990607 and a function. Passing output cotangent 1.0 returns input cotangents (2.2734745, 7.9571605). section 4.3.3

How Deep Learning with JAX covers it

How deep it goes
Defines cotangents, explains forward and reverse phases, and works through graph propagation without extending beyond reverse-mode mechanics.
Examples
Uses a scalar function, a hand-traced computation graph, and a two-output function evaluated with vjp().
What it assumes
Assumes familiarity with derivatives and computation graphs while grounding the idea in a concrete reverse-mode example.
What you can do after
Apply it via jax.vjp to inspect output sensitivities and recover derivatives with respect to function inputs.
How it connects
Connects cotangents with reverse-mode autodiff, adjoints, vector-Jacobian products, and Jacobian rows.
Cover of Deep Learning with JAX
liveBookmanning.com
Read Deep Learning with JAX on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with JAX. The section links point to the parts of the book it draws on.