A causal language model predicts the next token from tokens that came before it. It cannot use future context, so generation proceeds one token at a time—like autocomplete extended into a full sequence. section 15.4.1 section 16.3.1
Prompting a pretrained causal model
With KerasHub, a pretrained Gemma CausalLM can continue prompts such as “The following brownie recipe is easy to make in just a few steps. You can start by” with procedural text. The same next-token objective can also produce coherent but wrong continuations, such as inventing a 542nd U.S. president. section 16.3.1
The objective explains both the strength and the limitation: the model learns to produce likely text, not necessarily useful answers. Prompt wording can steer the continuation toward a recipe, dialogue, or another text pattern, but plausibility is not a guarantee of truth or task completion. section 16.3.1
This differs from masked language modeling. A causal model predicts a token from its past, whereas a masked model predicts hidden tokens from surrounding context; the latter is suited to contextual representation rather than generation in a loop. section 15.4.1
Listing 16.11 Instantiating a pretrained LLM with KerasHub
gemma_lm = keras_hub.models.CausalLM.from_preset( "gemma3_1b", dtype="float32", )
Instantiate a pretrained causal language model with KerasHub’s high-level CausalLM API. section 16.3.1
