How deeply does Build Applications with Local AI Models on a Mac cover Unsloth?

Outlines data preparation, notebook training, GGUF export, and Ollama deployment, without detailing Unsloth’s optimization internals.

Unsloth

As explained in Build Applications with Local AI Models on a Mac

A software library that accelerates and streamlines fine-tuning of language models while aiming to reduce memory use.

Unsloth is a Python library for fine-tuning a base language model on your own examples. Its optimized workflow aims to reduce training time and GPU memory use, though gains depend on the model, hardware, and settings. It handles training and export; Ollama handles local serving. section 16.6.4

To turn domain-specific examples into a local custom model:

Choose a supported base model and prepare examples, such as conversations, documents, or input/output pairs. Check support for its architecture, training method, and GGUF export. section 16.6.4 section 17.7.1
Run an Unsloth notebook cell by cell to fine-tune the model. Google Colab can provide a hosted GPU, but free access and completion time are not guaranteed. section 16.6.4
Export the trained result as a quantized GGUF file. Copy it and a Modelfile to your local machine, make the Modelfile’s FROM path match the file, and preserve the training chat template. section 16.6.4
Use ollama create to register the model, then ollama run to try it locally. section 16.6.4 section 17.7.1

model.save_pretrained_gguf("my_model", tokenizer, quantization_method="q4_k_m")

This notebook call exports the fine-tuned model to GGUF using q4_k_m quantization; the exported file is the handoff to Ollama. section 16.6.4

How Build Applications with Local AI Models on a Mac covers it

Cover of Build Applications with Local AI Models on a Mac

liveBook manning.com

Read Build Applications with Local AI Models on a Mac on liveBook Get the book at manning.com

Related concepts

low-rank adaptationuses · Low-rank adaptation is a parameter-efficient model fine-tuning method that learns low-rank updates to model weights, typically while keeping most original parameters fixed.
Ollamacontrasts with · Ollama is software for downloading, running, and serving language models locally.
Google Colabuses · Google Colab is a hosted Jupyter notebook service that provides browser-based interactive coding environments with access to cloud computing resources.
GGUFproduces · GGUF is a binary file format for storing machine-learning model tensors and associated metadata, designed to support efficient model loading and inference.
fine-tuninguses · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.

This explainer was generated by Manning's knowledge graph from the text of Build Applications with Local AI Models on a Mac. The section links point to the parts of the book it draws on.