liveBook · Manning

Unsloth

As explained in Building Specialized AI Systems

An open-source library for accelerating and simplifying large-language-model fine-tuning.

Unsloth accelerates and simplifies large-language-model fine-tuning by combining performance optimization, parameter-efficient training, and GPU memory management. It supports adapting a quantized LLaMA-3 8B model with LoRA on limited hardware. section 2.1.2

A practical Unsloth workflow includes:

  • Install Unsloth with supporting packages such as xformers, TRL, PEFT, Accelerate, and bitsandbytes. section 2.1.2
  • Load a 4-bit LLaMA-3 8B model and tokenizer, then attach lightweight LoRA adapters. section 2.1.2
  • Format domain data into instruction, input, and response fields before supervised fine-tuning. section 2.1.2
  • Train with SFTTrainer, gradient accumulation, sequence packing, and an 8-bit AdamW optimizer. section 2.1.2

Credit-risk adaptation

A fine-tuned model evaluates an applicant earning $1,137.38, carrying $766.31 in monthly debt, holding a 600 credit score, and having one late payment, classifying the applicant as high risk. section 2.1.2

Listing 2.2 Installing Unsloth and other supporting packages
!pip install "unsloth[colab-new]@git+https://github.com/unslothai
➥/unsloth.git" #A
!pip install --no-deps xformers trl peft accelerate bitsandbytes
 

Use the installation listing to reproduce the Unsloth environment and its supporting fine-tuning packages. section 2.1.2

How Building Specialized AI Systems covers it

How deep it goes
Demonstrates installation, model loading, LoRA configuration, dataset formatting, training, testing, and saving, while deferring detailed parameter explanations.
Examples
Uses a credit-risk dataset, chat prompt template, applicant test case, and generated high-risk assessment.
What it assumes
Assumes a prepared dataset and initially defers detailed understanding of training steps and parameters.
What you can do after
Operate a LoRA fine-tuning workflow for a quantized language model and save the resulting model and tokenizer.
How it connects
Connects LoRA, quantization, LLaMA-3, supervised fine-tuning, GPU memory management, and parameter-efficient fine-tuning.

Also explained in

Compare how 2 books explain Unsloth →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Building Specialized AI Systems. The section links point to the parts of the book it draws on.