liveBook · Manning

Optimizer.stateless_apply()

As explained in Deep Learning with R, Third Edition

stateless_apply is a Keras optimizer method that computes updated parameters and optimizer state from explicit inputs without mutating hidden state.

optimizer.stateless_apply() updates trainable variables without mutating hidden optimizer state. Pass optimizer variables, gradients, and trainable variables; receive updated trainable and optimizer variables. This explicit state flow supports JAX-style custom training loops. section 7.4.2

JAX-style Keras training step

Initialize optimizer variables with optimizer$build(), store all variable values in state, apply gradients with optimizer$stateless_apply(), then return loss and the rebuilt state.

How Deep Learning with R, Third Edition covers it

How deep it goes
Explains inputs, outputs, optimizer state, and a complete update function without deriving optimizer mathematics.
Examples
Uses an R/Keras MNIST batch and a stateful list containing trainable, non-trainable, and optimizer variables.
What it assumes
Assumes gradients and custom training steps while making explicit optimizer-state handling concrete.
What you can do after
Build it from scratch by implementing a functional training step that returns updated parameters and optimizer state.
How it connects
Connects custom training loops, gradients, optimizer state, non-trainable variables, and JAX automatic differentiation.
Cover of Deep Learning with R, Third Edition
liveBookmanning.com
Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.