liveBook · Manning

Fine-tuning

As explained in Data Analysis with LLMs

Fine-tuning is the process of further training an existing model on task-specific examples to adapt its behavior to a particular task.

Fine-tuning takes an existing language model and trains it further on examples of a particular task. Instead of training a model from scratch, you create a specialized variant of a base model. For review sentiment classification, the examples can pair review text with the desired label, such as `pos` or `neg`. section 9.7 section 1.5.2

Try model selection, tuning parameters, and prompt design first. Fine-tuning takes upfront time and money, and using the resulting model can cost more per token. Its potential rewards are better task accuracy and shorter prompts: a model trained to classify reviews may no longer need classification instructions repeated with every review. section 9.0 section 9.7

For a review-classification workflow:

  • Start with reviews and their correct sentiment labels. The example training set contains 100 labeled reviews; more samples may help, but increase training cost. section 9.8
  • Convert each row into a successful chat interaction: the review is a user message, and its label is the assistant’s reply. Write one interaction per line in a `.jsonl` file. section 9.8
  • After fine-tuning, send review text to the resulting model ID and compare its labels with known answers. Because the task is learned, the prompt can be just the review text. section 9.10
{'messages':[
    {'role':'user', 'content':'I am willing to tolerate almost anything 
     ...'},
    {'role':'assistant', 'content':'neg'}
]}

This sample shows the input and desired output as a user–assistant message pair. section 9.8

import openai
client = openai.OpenAI()

response = client.chat.completions.create(
    model='[Fine-tuned model ID]',
    messages=[
        {'role':'user', 'content':prompt}
        ]
    )

This call uses the fine-tuned model’s ID to request a classification. section 9.10

Check whether it pays off

In a ten-review test, the fine-tuned model got seven labels right and used 2,085 tokens, versus 2,228 tokens for a generic model tuned to the same accuracy. But its stated input-token price was six times higher. Fewer tokens did not offset that price difference in this case; test quality and calculate total cost before deploying. section 9.10

How Data Analysis with LLMs covers it

How deep it goes
Works through labeled-data format, model use, and a cost comparison, without showing a fine-tuning job submission in the supplied excerpts.
Examples
Uses labeled movie reviews, a JSON-lines chat interaction, and a ten-review accuracy and token-cost test.
What it assumes
Assumes access to labeled task examples and Python-based model use; introduces the required chat-message and JSON-lines training format.
What you can do after
Apply it via OpenAI chat models: prepare labeled examples, use a fine-tuned model ID, and assess whether its results justify its costs.
How it connects
Connects model selection, tuning parameters, prompt engineering, sentiment classification, and training data.

Also explained in

Compare how 37 books explain fine-tuning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Data Analysis with LLMs. The section links point to the parts of the book it draws on.