A layer is a reusable data-processing module: it takes one or more tensors and returns one or more tensors. Think of it as a filter that turns data into a representation useful for the task. Chain compatible layers together, and their successive transformations form a model. section 3.6.2 section 2.1
Two layers for handwritten digits
A Sequential classifier can be assembled in R as `model <- keras_model_sequential() |> layer_dense(512, activation = "relu") |> layer_dense(10, activation = "softmax")`. The first Dense layer produces an intermediate representation; the second produces ten probability scores, one for each digit class. The layers specify the architecture. Training also requires a loss function and optimizer. section 2.1
In Keras, `Layer` is the central class for encapsulating a forward computation and, often, state in the form of weights. A custom layer typically creates its weights in `build(input_shape)` and defines the transformation in `call(inputs)`. Weight shapes can depend on the input shape, so a layer may not create its weights until it is first called or explicitly built. section 3.6.2 section 7.2.1
layer_simple_dense <- new_layer_class(
classname = "SimpleDense",
initialize = function(units, activation = NULL) {
super$initialize()
self$units <- units
self$activation <- activation
},
build = function(input_shape) { #1
.[batch_dim, input_dim] <- input_shape
self$W <- self$add_weight(shape(input_dim, self$units), #2
initializer = "random_normal")
self$b <- self$add_weight(shape(self$units), initializer = "zeros")
},
call = function(inputs) { #3
y <- op_matmul(inputs, self$W) + self$b
if (!is.null(self$activation)) {
y <- self$activation(y)
}
y
}
)
This custom Dense-layer listing puts the two responsibilities side by side: `build()` creates a weight matrix and bias; `call()` multiplies the inputs by the matrix, adds the bias, and optionally applies an activation. section 3.6.2
Choose layers to suit the data and task:
- Dense layers commonly process feature vectors; recurrent or 1D convolution layers process sequences; 2D convolution layers process images. section 3.6.2
- Check intermediate widths as well as the output layer: compressing a 46-class classifier through just four units created an information bottleneck and reduced validation accuracy. section 4.2.7
- Inspect a built Sequential model to see each layer’s output shape and parameter count; specify an input shape in advance if you want to inspect it while adding layers. section 7.2.1
A layer’s weights parameterize its transformation. During training, a loss function measures the model’s predictions against targets, and an optimizer updates the weights to reduce that loss. The learned values—not just the list of layers—hold the model’s acquired knowledge. section 1.6 section 2.5
