Language modeling trains a model to predict the next token from earlier tokens. Repeating that prediction turns a short prompt into an open-ended sequence. section 15.1.2 section 16.6
Generating Shakespeare-like text
A character-level model learns to guess one character at a time from Shakespeare. At inference, it consumes a prompt such as “KING RICHARD III:”, predicts the next character, feeds that prediction back in, and repeats. The result can resemble dramatic dialogue despite the narrow training task. section 15.1.2
Keep these practical distinctions in mind:
- Train on known sequences, but generate autoregressively: inference requires a loop that repeatedly selects and feeds back the next token. section 15.1.2
- Preserve causal direction. A bidirectional recurrent layer can exploit future tokens during training, producing high accuracy while breaking generation. section 15.1.2
- Separate prediction from selection: greedy search chooses the most likely token, while random and Top-K sampling provide alternatives. section 16.6
Listing 15.6 Modifying the language model for autoregressive inference
inputs = keras.Input(shape=(1,), dtype="int", name="token_ids") #1 input_state = keras.Input(shape=(hidden_dim,), name="state") x = layers.Embedding(vocabulary_size, embedding_dim)(inputs) x, output_state = layers.GRU(hidden_dim, return_state=True)( x, initial_state=input_state ) outputs = layers.Dense(vocabulary_size, activation="softmax")(x) generation_model = keras.Model( inputs=(inputs, input_state), outputs=(outputs, output_state), ) generation_model.set_weights(model.get_weights()) #2
Use the compact generation model to expose token input, recurrent state, vocabulary probabilities, and autoregressive inference. section 15.1.2
