Chapter 12: Comparing and selecting LLM models

Introduction

Introduces the challenge of choosing among Ollama models and gives a way to check current model availability before selecting one.

• Current model checking workflow: Explain why model comparison tables should be treated as snapshots, and describe how to check installed models, find current tags, and download a chosen model.
  Why it matters: Checking current tags helps you avoid choosing a model based on outdated names or rankings.

Key terms: ollama list, ollama pull <model>, Ollama model library, tags

12.1 Model families and their strengths

Introduces open-weight LLMs as downloadable model families with differing approaches and strengths.

• Open-weight model families: Explain what open-weight means and describe why Llama, Gemma, Qwen, Mistral, and DeepSeek are treated as distinct model families, without claiming specific strengths for any one family.
  Why it matters: This distinction gives you a starting point for comparing local models without assuming they all serve the same needs.

Key terms: open-weight, model weights, model families, Llama, Gemma, Qwen

12.1.1 Llama (Meta)

Introduces Meta’s open-weight Llama family, its development over time, and why it is a practical starting model.

• Open weights and community: Explain what Llama being open-weight means and how Meta’s release helped create a community of people adapting it for specific tasks.
  Why it matters: This explains why Llama has a large community and many task-specific adaptations.
• Llama release progression: Describe the broad progression of Llama releases, from early improvements in performance and licensing to smaller variants and Llama 4’s new capabilities.
  Why it matters: The timeline shows why a Llama version matters when comparing models.
• Llama as a starting choice: Explain which tasks the section identifies as Llama’s strengths and why it recommends Llama when a model choice is uncertain.
  Why it matters: These strengths give you a practical default when choosing a model to try.

Key terms: Llama, open-weight, fine-tuning, Llama 2, Llama 3, Llama 3.2, Llama 4, MoE, multimodal, general-purpose tasks

12.1.2 Gemma (Google)

This section outlines Gemma’s major releases and explains why its efficiency makes a small Gemma-family model suitable for the book’s main path.

• Gemma efficiency and model choice: Explain how Gemma’s performance relative to its size motivates the book’s choice of a small Gemma-family model.
  Why it matters: Model size alone does not tell you how useful a model will be on your Mac.
• Gemma release progression: Describe the broad progression from Gemma 1 and 2’s parameter-size options to the additional capabilities listed for Gemma 3 and 4.
  Why it matters: The release list shows that choosing a Gemma model involves more than picking a parameter size.

Key terms: Gemma, 9B Gemma 2, efficiency, modest hardware, Gemma 1, Gemma 2, Gemma 3, Gemma 4, multimodal, context

12.1.3 Qwen (Alibaba Cloud)

This section identifies Qwen’s multilingual strength and explains how to read the active-parameter counts in its MoE model names.

• Qwen multilingual strengths: Explain why Qwen is often a good choice for work with Asian languages, and name its other stated strengths.
  Why it matters: These strengths help you decide whether Qwen fits the tasks and languages you use.
• Qwen MoE active parameters: Explain what A3B and A22B indicate in the Qwen 3 MoE model names.
  Why it matters: The notation lets you distinguish a model’s active parameter count from the larger number in its name.

Key terms: Qwen, multilingual strength, Asian languages, coding, mathematical reasoning, MoE, A3B, A22B, active for each token

12.1.4 Mistral (Mistral AI)

Introduces Mistral’s notable Mixtral architecture and the strengths that distinguish the model family.

• Mistral language and reasoning strengths: Describe Mistral’s strengths in European languages, reasoning, and structured thinking.
  Why it matters: These strengths help you recognize when Mistral may be worth trying for a task.
• Mixtral selective parameter activation: Explain the mixture-of-experts idea stated for Mixtral 8x7B: it activates only a subset of its parameters for each token.
  Why it matters: It gives you the defining architectural detail this section provides about Mixtral.

Key terms: Mistral, European languages, reasoning, structured thinking, Mixtral 8x7B, mixture-of-experts (MoE), parameters, token

12.1.5 DeepSeek (DeepSeek AI)

Introduces DeepSeek through two models with different defining characteristics.

• DeepSeek model distinctions: Distinguish DeepSeek-V3 from DeepSeek-R1 using the characteristics given for each model.
  Why it matters: The two DeepSeek models are presented as notable for different reasons.

Key terms: DeepSeek-V3, DeepSeek-R1, MoE, active parameters, reasoning model, MIT license

12.1.6 The model ecosystem

This subsection shows that five open-weight model families reach the reader’s laptop through Ollama, making it easy to switch among them.

• Shared Ollama model access: Explain that the five model families connect to a local laptop through Ollama and that switching among them is easy.
  Why it matters: You can consider models from different families without needing a separate local setup for each one.

Key terms: model families, Ollama, local machine

12.2 Model size vs. quality tradeoffs

Explains what model size means and the general tradeoff between quality, speed, and memory use.

• Parameter count and tradeoffs: Explain what the “B” in a model name means and how parameter count generally affects quality, inference speed, and memory requirements.
  Why it matters: This tradeoff helps you choose a model that fits both your needs and your Mac.

Key terms: B, parameters, inference, memory requirements

12.2.1 Model comparison quick reference

This section uses a model-size comparison to show why available RAM limits practical local LLM choices.

• Model size and RAM tradeoff: Explain why a larger local model may be a poor choice even if it is smarter, using the section’s 8 GB laptop and 14B model example.
  Why it matters: A model’s potential quality is useful only if your computer can run it at a practical speed.

Key terms: RAM, disk swap, 14B model

12.2.2 Understanding tokens per second

This section explains what tokens are, how token-processing speed feels to a user, and which model sizes suit Macs with 8 GB or 16 GB of RAM.

• Tokens and response speed: Explain what a token is and describe how different token-per-second rates affect the experience of reading a model’s response.
  Why it matters: Tokens per second helps you judge how responsive a model will feel in use.
• RAM-based model size choices: Justify the section’s recommended model sizes for machines with 8 GB and 16 GB of RAM.
  Why it matters: Choosing a model that fits your RAM balances responsiveness with answer quality.
• Mixture of experts activation: Describe what the section says about which parts of a mixture of experts model activate for a token.
  Why it matters: This gives you a basic meaning for MoE when it appears in later model comparisons.

Key terms: token, tokenization, tokens/sec, RAM, 2B to 3B models, 7B or 8B models, mixture of experts, MoE

12.2.3 Benchmarks: What they tell you (and what they don’t)

This subsection explains what LLM benchmarks measure and why testing a model on your own use case matters more than relying on scores alone.

• Benchmark Scores and Limits: Explain what LLM benchmarks measure, how the section compares score differences across model sizes, and why benchmark results should not decide model choice on their own.
  Why it matters: A benchmark score can narrow your choices, but your own tasks reveal whether a model works well for you.

Key terms: benchmarks, standardized tests, use case

12.2.4 The RAM reality check

This section shows how to check whether a local LLM will fit in available RAM and why choosing one that is too large can make it impractically slow.

• Available RAM for models: Explain why a computer’s total RAM is not all available to a model, and describe how to check available memory before downloading one.
  Why it matters: Checking free memory first helps you avoid downloading a model your Mac cannot run well.
• Model file size estimate: Describe how a model’s file size can help estimate its RAM needs, using the section’s examples.
  Why it matters: The file size gives you a practical first estimate when deciding whether a model will fit.
• Oversized model swap slowdown: Explain what happens when a model is too large for available RAM and why a smaller model may be the better choice.
  Why it matters: A model that technically runs may still be too slow to use comfortably.

Key terms: RAM, operating system’s monitoring tool, model’s file size on disk, RAM needs, rule of thumb, disk swap space, response generation

12.3 Multilingual capabilities

This section explains why LLMs differ in how well they handle different languages.

• Training Data and Language Strength: Explain that models learn languages from their training text, so their multilingual abilities vary; a model trained mostly on English text will usually be stronger in English than in Japanese.
  Why it matters: The languages you plan to use are worth considering when choosing a model.

Key terms: multilingual capabilities, training data

12.3.1 Language strengths by model family

Compares model families’ language strengths and explains why multilingual performance varies.

• Language performance and training data: Explain why model families can perform similarly in English but differ more in other languages.
  Why it matters: It explains why a model that works well in English may not be the best choice for another language.
• Model family language strengths: Distinguish the listed model families by their Japanese and European-language strengths, including how Llama’s Japanese changes with size.
  Why it matters: Matching a model family to the intended language can produce more natural responses.

Key terms: training data, model size, task, Qwen 2.5, Llama 3.1/3.2, Gemma 2, Mistral, Japanese, European languages

12.3.2 A side-by-side experiment

This subsection shows how to compare models’ responses to the same language prompt.

• Side-by-side language comparison: Describe how to test models on a language by asking three models the same question and comparing their responses for differences in phrasing, structure, and detail.
  Why it matters: A shared question gives you a direct way to see how models differ in a language you care about.

Key terms: same question, three models, phrasing, structure, level of detail, training data

12.3.3 Multilingual recommendation summary

The section gives starting model-family recommendations based on the language a reader needs.

• Language-based model choices: Apply the section’s recommendations: any major model works well for English, Qwen is a starting point for Japanese, and Mistral is worth trying for European languages.
  Why it matters: These recommendations give you a practical starting point when choosing a model for a language.

Key terms: Qwen, Mistral, multilingual capability

12.4 Switching models in your chatbot

12.4.1 Step 1: Download the new model

Explains that pulling a model downloads it locally, where it remains until deleted, and that you can check which models are installed.

• Local model download and storage: Explain that pulling a model downloads it to the local machine, where it stays until deleted, and that you can check which models are installed.
  Why it matters: You can tell whether a model is already on your Mac and account for the storage it uses.

Key terms: pull, local machine, installed models

12.4.2 Step 2: Change one line of chat application code

This subsection shows how to switch the chatbot to a different AI model by editing one line in app.py.

• Chatbot model name change: Describe how to switch the chatbot’s model by finding the model name in app.py, replacing it with the desired model name, and saving the file.
  Why it matters: You can try a different model in your chatbot without rewriting the application.

Key terms: app.py, model name

12.4.3 Step 3: Add a Model Selector in the Streamlit UI

This subsection adds a sidebar dropdown to the existing Streamlit chatbot so users can compare installed local models without editing code.

• Sidebar model selector: Describe where to add the model dropdown in the chapter 11 Streamlit chatbot and what it lets users do.
  Why it matters: The selector makes comparing your installed models convenient without changing the chatbot’s code for each test.

Key terms: dropdown, sidebar, installed models

12.4.4 Managing your model collection

This subsection explains how to inspect and prune downloaded models so they do not occupy unnecessary disk space.

• Model collection management options: Describe how to inspect installed models and their sizes, get details about a particular model, check their disk usage, and remove a model that is no longer needed.
  Why it matters: These options let you see what is installed and clear out models you no longer use.
• On demand model retention: Explain why you need not keep every model installed and describe the suggested download-and-removal approach.
  Why it matters: Keeping only models you use helps avoid unnecessary disk-space use.

Key terms: installed models, sizes, disk space, download models, remove

12.4.5 A quick model comparison script

This section shows how to compare local models by asking them the same question and examining their responses side by side.

• Side-by-side model comparison: Describe how to use test_models.py to ask multiple models the same question, then compare their responses for differences in phrasing, structure, and detail.
  Why it matters: A side-by-side test helps you choose a model whose style suits your chatbot.

Key terms: test_models.py, model’s personality

12.5 The 2026 revolution: Frontier open models

Introduces three February 2026 model releases as a turning point for locally run AI, without explaining the models or their impact.

12.5.1 The mixture of experts revolution

Explains how mixture-of-experts models activate only selected subnetworks, giving them a different computation tradeoff from dense models.

• Dense and MoE activation: Explain how a dense model differs from a mixture-of-experts (MoE) model when producing a response, including how the router and experts acquire their roles.
  Why it matters: This distinction explains why an MoE model does not use all its parameters for every query.
• Total versus active parameters: Distinguish total parameters from active parameters and explain why the gap can make an MoE model less computationally expensive per token than its total size suggests.
  Why it matters: The two counts help you interpret a model’s stated size without assuming it uses that much computation for every token.

Key terms: dense, mixture of experts (MoE), experts, router, cotraining, total parameters, active parameters, computation per token

12.5.2 Model 1: Zhipu GLM-5

This section describes GLM-5’s specifications and training approach, and explains how to interpret its reported benchmark results.

• GLM-5 architecture and scale: Describe GLM-5 as an open-weight mixture-of-experts model with 744 billion total parameters and 40 billion active parameters, and explain the stated purpose of its DeepSeek Sparse Attention.
  Why it matters: These specifications help you compare GLM-5 with other models without treating its total parameter count as its active parameter count.
• Benchmark results as evidence: Explain why GLM-5’s reported reasoning benchmarks, including Humanity’s Last Exam, are useful for comparison but do not fully measure real-world performance.
  Why it matters: This keeps a reported score from standing in for a test of how well the model serves your needs.
• Asynchronous reinforcement learning training: Describe how Zhipu’s slime framework coordinates data generation and model training, and what benefit the section attributes to that arrangement.
  Why it matters: It explains one way Zhipu made GLM-5’s training pipeline more productive.

Key terms: open-weight MoE model, total parameters, active parameters, DeepSeek Sparse Attention, Humanity’s Last Exam, reasoning evaluations, benchmark results, slime, asynchronous data generation, training workers

12.5.3 Model 2: Alibaba Qwen3-Coder-Next

This section explains Qwen3-Coder-Next’s ultra-sparse design, its local memory requirements, and its reported coding-agent performance.

• Ultra-sparse MoE design: Explain how Qwen3-Coder-Next activates only a small fraction of its parameters per token and how the section relates that design to generation speed.
  Why it matters: The active parameter count helps explain the model’s speed, but not its storage or memory needs.
• Local storage and memory limits: Explain why activating only 3 billion parameters does not mean Qwen3-Coder-Next will fit on a 16 GB or 32 GB Mac.
  Why it matters: Checking the build and available memory can prevent an impractical local download.
• Coding-agent performance context: Describe the model’s intended coding-agent use and the section’s comparison of its reported benchmark performance with proprietary models.
  Why it matters: Its reported benchmark result is relevant to the coding-agent work the model was built for.

Key terms: Qwen3-Coder-Next, ultra-sparse MoE, parameters per token, dense model, q4_K_M, context, runtime overhead, available memory, coding agents, SWE-bench Verified

12.5.4 Model 3: MiniMax M2.5

This section presents MiniMax M2.5 as an open-source model with a coding benchmark score close to a leading proprietary model’s, at a substantially lower stated API price.

• M2.5 coding benchmark comparison: Explain how MiniMax M2.5’s SWE-bench Verified score compared with Claude Opus 4.6’s at the time, while noting that newer Claude Opus versions have since improved.
  Why it matters: It puts M2.5’s reported coding result in context without treating a historical comparison as a current ranking.
• M2.5 access cost options: Describe the section’s two cost options for using MiniMax M2.5: its own API pricing compared with Claude Opus 4.6, and running the open-source model on your own infrastructure.
  Why it matters: The cost comparison shows why a model with a close benchmark result may be attractive to use.

Key terms: MiniMax M2.5, SWE-bench Verified, Claude Opus 4.6, MiniMax’s own API, open source, own infrastructure

12.5.5 The February 2026 benchmark landscape

This subsection highlights how open models compared with proprietary models on demanding benchmarks in mid-February 2026.

• Open model benchmark standing: Explain that, as of mid-February 2026, open models genuinely competed with the best proprietary models on difficult benchmarks, including real-world software engineering and research-level reasoning.
  Why it matters: This changes how seriously you might consider open models for demanding work.

Key terms: open models, proprietary models, benchmarks, real-world software engineering, research-level reasoning

12.5.6 The new model comparison table

This section explains why a frontier model can use far fewer parameters per token than its total size suggests and how to check whether a build will fit on a Mac.

• Total versus active parameters: Explain why Qwen3-Coder-Next’s 80 billion total parameters do not mean all 80 billion are used for each token.
  Why it matters: The total parameter count alone does not tell you how much compute the model uses per token.
• Frontier model memory check: Describe what to check before downloading a frontier model for a Mac and why its RAM requirement is not a single fixed number.
  Why it matters: Checking the particular build against your Mac’s memory helps you avoid downloading one that is too large.

Key terms: total parameters, active, compute per token, router, experts, RAM requirements, quantized build, context length, Ollama tag, unified memory

12.5.7 What this means for you

Explains why mixture-of-experts models can use computing power selectively and notes the emerging availability of quantized versions on Ollama.

• Selective knowledge activation: Explain how MoE addresses the tradeoff between knowledge and computing power by activating the knowledge relevant to a question rather than all of it.
  Why it matters: This is the section’s reason that a model can draw on broad knowledge without using all of it for every answer.
• Emerging Ollama model options: Describe what the section says about community-quantized models on Ollama and why Qwen3-Coder-Next may appeal to early adopters.
  Why it matters: It connects the computing-power discussion to models a reader may be able to try through Ollama.

Key terms: MoE architecture, knowledge, computing power, community-quantized, Ollama, Qwen3-Coder-Next, active-parameter count

12.6 Exercises

12.7 Summary