PyTorch is a framework for building and training neural networks. It handles gradient calculations for backpropagation while leaving the model’s forward pass and training loop visible. That makes it useful when you want to see how a model learns, rather than hide the process behind a simpler API. section 5.2.4 section 5.2.5
A name-based classifier shows the main pieces working together:
- Represent each name with character 1-, 2-, and 3-gram features, then convert the features and binary labels to torch.Tensor objects. section 5.2.4 section 5.2.5
- Subclass torch.nn.Module; define a linear layer in the constructor and apply sigmoid in forward() to produce one output. section 5.2.5
- Use binary cross-entropy for the classification loss and stochastic gradient descent (SGD) to update model parameters. section 5.2.5
- For each training epoch, clear gradients, run the model, calculate loss, call backward(), and take an optimizer step. Check loss and accuracy on held-out names. section 5.2.5
>>> import torch >>> class LogisticRegressionNN(torch.nn.Module): ... def __init__(self, num_features, num_outputs=1): ... super().__init__() ... self.linear = torch.nn.Linear(num_features, num_outputs) ... def forward(self, X): ... return torch.sigmoid(self.linear(X)) >>> model = LogisticRegressionNN(num_features=vecs.shape[1], num_outputs=1) >>> model LogisticRegressionNN( (linear): Linear(in_features=3663, out_features=1, bias=True) )
The model definition exposes the two methods to implement: a constructor for the layer and forward() for the computation. section 5.2.5
>>> from tqdm import tqdm >>> num_epochs = 200 >>> pbar_epochs = tqdm(range(num_epochs), desc='Epoch:', total=num_epochs, ascii=' =') >>> for epoch in pbar_epochs: ... optimizer.zero_grad() #1 ... outputs = model(X_train) ... loss_train = loss_func_train(outputs, y_train) #2 ... loss_train.backward() #3 ... optimizer.step() #4 Epoch:: 100%|================================| 200/200 [00:04<00:00, 42.84it/s] 96.26it/s]
The short training loop shows where PyTorch computes gradients and where SGD changes the weights. section 5.2.5
Finding a grammatical pattern
PyTorch also handles the tensor operations in a one-dimensional convolution. Convert a sentence’s four part-of-speech indicator rows to a tensor, add a batch dimension, and load a three-token kernel into torch.nn.Conv1d. A score of 3 marks the adverb–verb–noun pattern in “rightly timed pause.” This example uses hardcoded weights to inspect convolution, not a trained language model. section 7.2.6
The name classifier’s outputs are probabilities for the dataset’s binary sex labels, not reliable statements about a person’s identity. Evaluate predictions on held-out data before considering any practical use. section 5.2.4 section 5.2.5
