A language model learns patterns in sequences of text. A pretrained large language model (LLM) has already learned from a large textual dataset, so it can be reused for tasks such as text generation, sentiment analysis, question answering, and named entity recognition. Using one avoids the resources needed to train a large model yourself. section 1.3.3 section 16.0
The distinction is between building a language model and applying a pretrained one. A smaller LSTM language model can be trained for word-level text generation; a GPT-style Transformer can also be assembled from decoder blocks. Pretrained LLMs instead provide capabilities that an application can call without repeating that training. Transformers use attention to capture relationships across a sequence and, unlike sequential RNNs, support parallel training on large datasets. section 8.5.2 section 1.3 section 16.0
Text must first become tokens and numerical representations. In a GPT-style model, token indexes pass through token and position embeddings, then decoder blocks with causal self-attention and feed-forward layers. A final head produces a logit for each vocabulary token; softmax turns those logits into probabilities for generating text. One small PyTorch implementation uses three decoder blocks and a 10,600-token vocabulary. It illustrates the mechanism, not the scale of a pretrained LLM. section 8.2 section 12.3.3
Answering a recent question
A GPT-4 query about the Best Actor award at the 2024 Academy Awards returned a response saying it could not provide the information. The problem is practical: training data does not give a model reliable access to recent events or live facts. A LangChain agent can instead choose a tool, using Wikipedia for recent facts or Wolfram Alpha for real-time information and computations. section 16.2.1 section 1.3.3
completion = client.chat.completions.create(
model="gpt-4",
messages=[
{"role": "system", "content":
'''You are a helpful assistant, knowledgeable about recent facts.'''},
{"role": "user", "content":
'''Who won the Best Actor Award in 2024 Academy Awards?'''}
]
)
print(completion.choices[0].message.content)
The GPT-4 completion call shows the system instruction and recent-event question used to test the model before adding external tools. section 16.2.1
Use the model for its learned language capabilities, but connect it to an appropriate external source when an answer depends on current information. section 1.3.3 section 16.2.1
