A feedforward network passes inputs through successive layers to produce an output, without recurrent connections. A multilayer perceptron (MLP) is one way to build one: fully connected layers transform the input, with activation functions between them. The useful question is what to feed the network and what its output should represent. section 11.6
Predicting a point on a sphere
Suppose many functions each need a low-value point on a sphere. Represent each function by its parameter vector and train an MLP to predict a three-dimensional point directly from those parameters. One implementation uses three linear layers, ReLU after the first two, and 128 hidden units. It normalizes the output vector so the predicted point lies on the sphere. Training uses functions generated with different random seeds, their values at the predicted points, and Adam with a learning rate of 5e-4. The prediction path can be plotted against function contours and optimal points. section 11.6
class AmortizedModel(nn.Module):
def __init__(self, n_params): #1
super().__init__()
self.base = nn.Sequential(
nn.Linear(n_params, n_hidden),
nn.ReLU(inplace=True),
nn.Linear(n_hidden, n_hidden),
nn.ReLU(inplace=True),
nn.Linear(n_hidden, 3)
) #2
def forward(self, p): #3
squeeze = p.ndim == 1
if squeeze:
p = p.unsqueeze(0)
assert p.ndim == 2
z = self.base(p)
z = z / z.norm(dim=-1, keepdim=True)
if squeeze:
z = z.squeeze(0)
return z
The PyTorch model makes the forward path concrete: parameter vector → linear and ReLU layers → three coordinates → normalized point. section 11.6
An MLP need not solve a whole optimization problem by itself. In a traveling-salesman pipeline, a graph neural network first produces graph embeddings; MLPs then predict probabilities that nodes or edges belong to a solution. Graph search uses those probabilities to find a feasible candidate tour. This separates prediction from the search needed to assemble a solution. Training that pipeline from scratch requires substantial code and preprocessing, so a pretrained model is used for the practical TSP example. section 11.7
The pipeline figure locates the MLP between graph embedding and search, showing why predicted probabilities are not yet a tour. section 11.7
Feedforward layers also serve as policy networks in reinforcement learning. For CartPole, a hand-built actor passes state values through linear layers and ReLU activations before outputting action probabilities; a similarly layered critic outputs a value. Alternatively, Stable Baselines3 provides an MLP-based policy through `MlpPolicy` for A2C or PPO. Choose between implementing the layers yourself and using that policy option according to whether you need to inspect the network or train and evaluate an agent quickly. section 12.3
