Chapter 13: System prompts and parameter tuning

Introduction

Introduces the chatbot’s current generic behavior and previews ways the chapter will customize its personality and response style.

13.1 System prompts explained

Explains how a system prompt frames an LLM conversation and shapes responses without changing the model’s learned knowledge.

• System message role and placement: Describe the system role alongside user and assistant messages, and state when the system instruction is sent.
  Why it matters: The system role explains where instructions that frame a conversation belong.
• System prompt influence: Explain how a system prompt shapes responses and what happens when there is no system prompt.
  Why it matters: A system prompt lets you shape how the same model responds throughout a conversation.
• Prompt effects versus model knowledge: Distinguish changes to a model’s response style and priorities from changes to its learned knowledge.
  Why it matters: This distinction keeps a change in how the model answers from being mistaken for a change in what it has learned.

Key terms: user, assistant, system message, system prompt, rules, personality, boundaries, default behavior, model weights, underlying knowledge

13.2 Example system prompts: Personality presets

Introduces five system-prompt personality presets that readers can use or adapt to shape model behavior.

13.2.1 Preset 1: The professional consultant

13.2.2 Preset 2: The patient study buddy

This subsection describes the teaching style of the patient study buddy preset.

• Patient study buddy style: Describe how the patient study buddy preset responds: it acts as a warm, encouraging tutor, simplifies complex topics, uses everyday analogies, and checks in after each explanation.
  Why it matters: This gives you a concrete example of the kind of tutoring style a preset can request.

Key terms: patient study buddy, preset, everyday analogies

13.2.3 Preset 3: The pirate tutor

The pirate tutor preset changes the model’s teaching style without changing the accuracy of its explanations.

• Pirate Style Versus Accuracy: Explain that the pirate tutor adds personality and humor by teaching in pirate dialect while keeping the explanation accurate.
  Why it matters: You can give a tutor a distinctive voice without intending to change the facts it teaches.

Key terms: preset, pirate dialect, delivery style

13.2.4 Preset 4: The strict coding mentor

This subsection describes a strict coding mentor preset that guides learners through problems instead of giving them solutions.

• Strict coding mentor approach: Explain how the strict coding mentor preset helps someone learn to code: it uses the Socratic method to guide them through problems rather than handing them answers.
  Why it matters: It helps the learner practice reasoning through a coding problem rather than just receiving a solution.

Key terms: strict coding mentor, Socratic method

13.2.5 Preset 5: The concise expert

Introduces a concise-expert preset for quick, accurate answers and shows how changing system prompts can specialize the same model.

• Concise expert preset: Explain that the concise-expert preset aims for the shortest accurate answer without filler and is useful for quick fact checks.
  Why it matters: You can choose this preset when you want a fast answer rather than a lengthy response.
• Specialized assistants through prompts: Explain that changing a system prompt can make the same model serve many specialized assistant roles, such as an assistant given law-firm-specific instructions.
  Why it matters: A specialized assistant can be shaped by its instructions rather than by switching models.

Key terms: preset, concise expert, system prompts, specialized AI assistants

13.3 LLM temperature and creative parameters

13.3.1 What temperature does

This section explains how temperature changes an LLM’s next-token choices and why a middle setting suits most tasks.

• Temperature and token selection: Explain how an LLM chooses each next token and how temperature affects that choice, including what happens at temperature 0.
  Why it matters: It helps you understand what the temperature setting actually changes—and what it cannot guarantee.
• Temperature response tradeoffs: Describe how low, middle, and high temperatures change the style and reliability of the example responses, and justify why a middle range suits most tasks.
  Why it matters: The tradeoff helps you choose a setting that adds variety without making responses needlessly erratic.

Key terms: token, probabilities, temperature, sampling, creative, unpredictable, incoherent

13.3.2 Other important parameters

This section explains how top_k, top_p, and repeat_penalty control candidate selection and repetition beyond temperature.

• Fixed and flexible shortlists: Distinguish how top_k and top_p choose candidate next words: top_k keeps a fixed number of the most likely words, while top_p keeps a variable number whose combined probability reaches a threshold. Explain why lowering top_k is more conservative and why top_p can adapt to the completion.
  Why it matters: The distinction helps you describe what each setting changes without treating them as interchangeable.
• Repetition penalty tradeoff: Explain that repeat_penalty discourages repeated words or phrases and that raising it can reduce repetition at the cost of text that feels forced.
  Why it matters: It shows why stronger repetition control is not automatically better.

Key terms: top_k, top_p, nucleus sampling, candidate words, probability mass, repeat_penalty

13.4 Creating custom models with Ollama Modelfiles

Introduces a Modelfile as a named recipe that combines a base model with system prompts and parameter settings.

• Modelfile customization recipe: Explain that an Ollama Modelfile defines a custom model by starting with a base model and bundling customizations such as a system prompt, temperature, and other parameters; the resulting model can be used by name.
  Why it matters: It gives you one named way to reuse a base model with your chosen settings.

Key terms: Modelfile, base model, system prompts, temperature, custom model definition

13.4.1 The Modelfile format

This section defines an Ollama Modelfile and distinguishes its required base-model instruction from optional customization settings.

• Modelfile Structure and Purpose: Explain that a Modelfile is a plain-text configuration file Ollama uses to create a customized model, with a required FROM instruction naming the base model and optional customization settings.
  Why it matters: This distinction helps you recognize what a Modelfile needs before adding custom behavior.

Key terms: Modelfile, FROM, SYSTEM, temperature, top_p

13.4.2 Key Modelfile instructions

This section identifies the main Modelfile instructions and explains how the context-window setting affects conversation length and RAM use.

• Core Modelfile instructions: Describe what FROM, PARAMETER, and SYSTEM do in a Modelfile, including which instruction is required.
  Why it matters: These instructions let you tell apart the model you start with from the settings and prompt you customize.
• Context window tradeoff: Explain what num_ctx controls and why increasing it does not guarantee that the model receives more conversation history.
  Why it matters: A larger context setting helps only when the application actually sends enough history to use it.

Key terms: FROM, PARAMETER, SYSTEM, base model, system prompt, num_ctx, context window, history trimming, tokens, RAM

13.4.3 Building a custom model step by step

Shows how to create and run a named Ollama model from a Modelfile, and explains what that custom model adds to the base model.

• Custom model creation workflow: Describe creating Modelfile.studybuddy, using it to build a local model named studybuddy, and then running that model. Explain that the result is a configuration layer on the base model, not a duplicate of it.
  Why it matters: This lets you make a reusable custom model without duplicating its base model.
• Reusable model personality: Explain that the named model has a built-in personality and can be used wherever a regular model is used. State that Modelfiles can package different system prompts and parameter settings.
  Why it matters: A saved personality can be reused across the terminal, API, and chatbot.

Key terms: Modelfile.studybuddy, studybuddy, base model, configuration layer, built-in personality, system prompts, parameter settings, Modelfile

13.4.4 Managing custom models

This subsection identifies how to manage a growing collection of custom models by listing, inspecting, and removing them.

• Custom model management: Describe the three management actions for custom models: list them, inspect one with the show command, and remove one when it is no longer needed.
  Why it matters: These actions help you keep track of custom models and discard ones you no longer need.

Key terms: custom models, show command

13.5 Adding customization controls to the Streamlit UI

This section introduces Streamlit controls that let users customize chatbot behavior without editing code.

• No-code chatbot customization: Explain that adding system prompt and temperature controls to the Streamlit chatbot lets users customize its personality and behavior without changing the code.
  Why it matters: Users can adjust the chatbot themselves rather than asking someone to edit its code.

Key terms: system prompt, temperature controls, Streamlit chatbot

13.5.1 The enhanced chatbot

This section describes an enhanced Streamlit chatbot’s customization controls and how it sends those settings with each chat request.

• Chatbot sidebar controls: Describe the sidebar controls for choosing an installed model, setting temperature, and selecting a personality preset or writing a custom system prompt.
  Why it matters: These controls let a reader identify what can be customized from the app’s interface.
• Per-message Ollama requests: Trace what the application sends to Ollama whenever a user submits a chat message, including the system message, retained history, new message, and generation settings.
  Why it matters: This explains how the chatbot’s instructions, conversation context, and settings reach Ollama on each turn.

Key terms: model selection, temperature slider, personality presets, Custom, system message, retained conversation history, Ollama Python library, local API, temperature

13.5.2 How the system prompt affects the pipeline

This subsection explains that the system prompt enters the chatbot’s message flow early and shapes the model’s response throughout generation.

• System prompt influence: Explain when the system prompt enters the chatbot pipeline and how it affects the response.
  Why it matters: The prompt’s early influence helps explain why it can shape the whole response.

Key terms: system prompt, pipeline, token

13.6 Exercises

13.7 Summary