A Dense layer takes a vector of features and produces a new representation. Every output unit uses learned weights connected to all input features, plus a bias. In R/Keras, `layer_dense(units = 32, activation = "relu")` produces 32 output values per sample. Dense layers are also called fully connected layers. section 3.6.2 section 4.1.3
Recognize handwritten digits
Start with images reshaped from 28 × 28 pixels into vectors of 784 values, scaled to the interval 0–1. Then build `keras_model_sequential() |> layer_dense(512, activation = "relu") |> layer_dense(10, activation = "softmax")`. The first layer learns a 512-value representation; the last returns ten probability scores, one for each digit class. Compile the model with `optimizer = "adam"`, `loss = "sparse_categorical_crossentropy"`, and `metrics = "accuracy"` before fitting it. On the shown test set, the trained model reaches 0.9784 accuracy. section 2.1
Under the hood, a Dense layer multiplies its input by a weight matrix `W`, adds a bias vector `b`, and applies an activation if one is specified. With 16 units, `W` maps the input into a 16-dimensional representation. More units give the model room to learn more complex representations, but cost more computation and may encourage patterns that do not hold on new data. Without an activation, the operation is an *affine* transform: matrix multiplication plus bias. Stacking only such transforms is equivalent to one affine transform; an activation such as `relu` lets a stack learn nonlinear transformations. section 4.1.3 section 2.3.5
Choose the final Dense layer to match the prediction:
- For two-class classification, use one unit with `sigmoid` for a probability between 0 and 1, paired with `binary_crossentropy`. section 4.1.7
- For scalar regression, use `layer_dense(1)` without an activation so predictions are not restricted to a range; `mean_squared_error` is a typical loss. section 4.3.3
- For timeseries forecasting, flattening the input and trying a small Dense network provides a cheap baseline before a more complex model; compare it with a commonsense baseline. section 13.2.3
layer_simple_dense <- new_layer_class(
classname = "SimpleDense",
initialize = function(units, activation = NULL) {
super$initialize()
self$units <- units
self$activation <- activation
},
build = function(input_shape) { #1
.[batch_dim, input_dim] <- input_shape
self$W <- self$add_weight(shape(input_dim, self$units), #2
initializer = "random_normal")
self$b <- self$add_weight(shape(self$units), initializer = "zeros")
},
call = function(inputs) { #3
y <- op_matmul(inputs, self$W) + self$b
if (!is.null(self$activation)) {
y <- self$activation(y)
}
y
}
)
A small custom Dense implementation shows where the weight matrix and bias are created and where matrix multiplication, bias addition, and optional activation happen. section 3.6.2
