QLoRA — Module 12.4

How This Lesson Fits the Module & Volume

LoRA shrinks trainable params; the frozen base can still be too large for a single consumer GPU. QLoRA loads that base in 4-bit quantization (NF4 + double quant tricks) and trains LoRA adapters in higher precision—the practical recipe behind many open SFT runs (links to Vol. 11 PEFT).

Learning Objectives

By the end of this lesson, students should be able to:

Define QLoRA as quantized base + LoRA adapters.
Explain why adapters stay in FP16/BF16 while the base is 4-bit.
Configure BitsAndBytes + PEFT for a QLoRA SFT run.
Compare VRAM of full FT, LoRA, and QLoRA qualitatively.
List pitfalls: slow backprop, kernel support, eval dtype.
Decide when QLoRA is enough versus multi-GPU full FT.

Definition

QLoRA fine-tunes a large model by keeping the pretrained weights in a compressed 4-bit storage format (commonly NormalFloat4) while learning standard LoRA adapters in 16-bit. Gradients flow to adapters (and optionally a few other light params); the quantized base remains frozen.

Memory Ladder

Method	Base weights	Trainable	Typical VRAM story
Full FT	FP16/BF16	All + Adam	Highest
LoRA	FP16/BF16 frozen	Adapters + Adam	Medium
QLoRA	4-bit frozen	Adapters + Adam	Lowest among three

Load 4-bit

BitsAndBytes NF4 base

Attach LoRA

FP16/BF16 adapters

SFT

Backprop into adapters

Export

Save adapter (merge optional)

Hugging Face + BitsAndBytes

import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig, AutoTokenizer
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training

model_id = "meta-llama/Meta-Llama-3-8B"
bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
)
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb, device_map="auto")
model = prepare_model_for_kbit_training(model)

lora = LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"], lora_dropout=0.05)
model = get_peft_model(model, lora)
model.print_trainable_parameters()
# Proceed with TRL SFTTrainer / Transformers Trainer as in the SFT lecture

Design Details That Matter

NF4

Quantiles for normal weights.
Better than naive INT4 often.

Double quant

Quantize quantization constants.
Extra memory shave.

Paged optim

CPU offload under spikes.
Helps long-sequence SFT.

Strengths and Tradeoffs

Strengths

Fine-tune 7B–70B-class models on fewer GPUs.
Quality often near 16-bit LoRA.
Dominant open-source SFT recipe.

Tradeoffs

Slower than non-quant LoRA sometimes.
Depends on CUDA/bitsandbytes stack.
Not a substitute for careful data.

Common Misconception

“QLoRA quantizes the LoRA matrices to 4-bit.” The base is quantized; adapters are trained in higher precision. Confusing the two leads to broken setups and odd quality drops.

Knowledge Check

Short Answer: What two pieces make QLoRA? Answer: 4-bit quantized frozen base + LoRA adapters.
True/False: QLoRA typically trains adapters in FP16/BF16. Answer: True.
Multiple Choice: NF4 refers to: (a) a 4-bit weight format, (b) a tokenizer, (c) a batcher. Answer: (a).
Short Answer: Why use QLoRA instead of LoRA? Answer: Much lower VRAM for the frozen base.
True/False: Full FT usually uses less memory than QLoRA. Answer: False.
Multiple Choice: prepare_model_for_kbit_training helps: (a) k-bit training stability/setup, (b) CSS, (c) DNS. Answer: (a).
Short Answer: Name the HF quantization helper config class. Answer: BitsAndBytesConfig.
True/False: Double quantization compresses quantization constants further. Answer: True.
Multiple Choice: QLoRA is primarily about: (a) serving continuous batching, (b) memory-efficient fine-tuning, (c) BPE merges. Answer: (b).
Short Answer: Which PEFT method scales activations with learned vectors next? Answer: IA3.

Key Takeaways

QLoRA = 4-bit base + 16-bit LoRA for consumer-scale SFT.
Adapters—not the LoRA weights—are the usual trainable FP tensors.
BitsAndBytes + PEFT is the standard toolchain.
Trade some speed for large VRAM savings versus full FT/LoRA.
Next: IA3.

Trainer’s Guide

Hands-on idea: Compare nvidia-smi while loading 7B in FP16 LoRA vs 4-bit QLoRA; note trainable param equality.

Discussion prompt: For production quality, when do you graduate from QLoRA to multi-GPU LoRA/full FT?

Recap: QLoRA makes LoRA practical on quantized bases so large SFT fits in limited VRAM. Continue with IA3.