CloudSignal AI

Blog

Fine-tune without a GPU farm: LoRA and QLoRA, explained with code

By Ashwin Rajendraprasad Tutorials

Fine-tuning sounds like it needs a data center. Parameter-efficient methods make it a single-GPU job. You’ll learn LoRA and QLoRA from first principles and fine-tune a small model with runnable code. Just as importantly, you’ll learn when fine-tuning is the wrong tool entirely.

1. First, the question you should ask before fine-tuning

Most “we need to fine-tune” instincts are really prompt or retrieval problems wearing a fine-tuning costume, so start with the decision, not the technique:

  • Prompt engineering: try this first. If a better instruction or a few examples fix it, you’re done, at zero training cost.
  • Retrieval (RAG): reach for this when the model needs facts it doesn’t have: your documents, your data, anything that changes. Fetch the knowledge at query time (see the RAG-from-scratch tutorial).
  • Fine-tuning: reach for this when you need to change behavior, format, or style: a consistent output shape, a domain tone, a task the base model performs unreliably.

Here’s the caveat to tape to your monitor: fine-tuning teaches behavior, not knowledge. It is not a reliable way to “add facts”: a fine-tuned model will happily produce your format around a hallucinated fact. If the problem is what the model knows, that’s retrieval. If it’s how the model behaves, read on.

2. Why full fine-tuning is expensive

Fine-tuning “the normal way” means updating every weight in the model. That’s costly for a reason that has little to do with the weights themselves: training also has to hold optimizer state and gradients for each of those billions of parameters. A standard optimizer keeps running statistics that multiply the memory footprint several times over, on top of the activations from the forward pass. So the working memory to fully fine-tune a model is a large multiple of just loading it, which is precisely why it reads as a GPU-farm activity. Parameter-efficient fine-tuning attacks exactly this cost: train far fewer parameters, and the optimizer-state bill collapses with them.

3. LoRA: freeze the model, train the adapters

LoRA (Low-Rank Adaptation), from Hu et al., 2021 (arXiv:2106.09685), starts from an observation: the update a model needs during fine-tuning can be approximated by a low-rank matrix, far less information than a full weight update. So LoRA freezes the pretrained weights entirely and injects small, trainable low-rank matrices alongside them (typically into the attention projections). Instead of nudging a giant weight matrix directly, you learn the product of two skinny matrices of rank r. The frozen model does the heavy lifting; the adapters, a tiny fraction of the total parameters, learn the new behavior. Because you’re only training that fraction, the optimizer-state cost from section 2 shrinks dramatically, and the resulting adapter file is small enough to email.

4. QLoRA: quantize the base, adapt on top

LoRA shrinks the trainable parameters, but the frozen base still has to sit in memory. QLoRA, from Dettmers et al., 2023 (arXiv:2305.14314), shrinks that too: it quantizes the frozen base model to 4-bit (a data type the paper calls NF4, with a further “double quantization” pass to squeeze the quantization constants), then trains ordinary LoRA adapters in higher precision on top. The base is compressed and never updated; the adapters stay precise and do the learning. The result is the headline the technique earned: you can fine-tune large models on a single GPU. How large depends on the model and the card, so measure it for your setup rather than trusting a round number.

5. Runnable: fine-tune a small model with PEFT

The standard, well-documented toolchain is Hugging Face PEFT, transformers, TRL, and bitsandbytes (Tim Dettmers) for the 4-bit base. We’ll fine-tune a deliberately tiny, permissively licensed base, Qwen2.5-0.5B (Alibaba), so the code runs on modest hardware; QLoRA’s real payoff shows up on 7B-and-up models, but the mechanics are identical.

pip install "transformers==4.46.2" "peft==0.13.2" "bitsandbytes==0.44.1" \
            "trl==0.12.1" "datasets==3.1.0" "accelerate==1.1.1"
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training

BASE = "Qwen/Qwen2.5-0.5B"   # small + permissive so this runs anywhere

# 4-bit NF4 base with double quantization: the QLoRA recipe (Dettmers et al., 2023)
bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
)

tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
model = prepare_model_for_kbit_training(model)

# LoRA: freeze the base, train low-rank adapters on the attention projections
lora = LoraConfig(
    r=8, lora_alpha=16, lora_dropout=0.05,
    target_modules=["q_proj", "v_proj"],
    task_type="CAUSAL_LM",
)
model = get_peft_model(model, lora)
model.print_trainable_parameters()   # prints the (tiny) trainable fraction, your real number

That last line makes LoRA’s point concrete: it prints how few parameters you’re actually training. Now teach it a format with a small dataset. Note the examples encode a style, not facts:

from datasets import Dataset
from trl import SFTConfig, SFTTrainer   # trainer APIs move fast; the pinned versions matter

rows = [
    {"text": "### Instruction:\nSummarize: the meeting moved to 3pm.\n"
             "### Response:\n- Meeting moved to 3:00 PM"},
    {"text": "### Instruction:\nSummarize: sales rose last quarter.\n"
             "### Response:\n- Sales rose last quarter"},
    # ...a few dozen more in practice; quality and consistency matter more than volume
]
data = Dataset.from_list(rows)

trainer = SFTTrainer(
    model=model,
    train_dataset=data,
    args=SFTConfig(
        output_dir="adapter-out",
        num_train_epochs=3,
        per_device_train_batch_size=1,
        learning_rate=2e-4,
        dataset_text_field="text",
        max_seq_length=256,
    ),
)
trainer.train()
model.save_pretrained("adapter-out")   # saves ONLY the adapter, a few megabytes

6. Evaluate honestly, then serve

Training loss going down is not success. Hold out a set of examples the model never trained on and check behavior there, because two failure modes hide behind a falling loss curve. The first is catastrophic forgetting: the model gets better at your task and worse at things it used to do, and only a held-out check across both catches it. The second is upstream of all of it: data quality dominates outcomes. A few dozen clean, consistent examples beat thousands of sloppy ones, and no hyperparameter rescues a muddled dataset.

To serve the result, you have two options. Keep the adapter separate and load it on top of the base at inference (PEFT does this in a line), which lets you hot-swap adapters for different tasks. Or merge it into the weights for a single standalone model, with one catch worth knowing: you merge into a full-precision copy of the base, not the 4-bit one you trained against.

from peft import PeftModel

base_fp16 = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16)
merged = PeftModel.from_pretrained(base_fp16, "adapter-out").merge_and_unload()
merged.save_pretrained("merged-model")

You now have the whole loop: quantize a base, adapt it on a single GPU, evaluate for regressions, and ship either the adapter or a merged model. The base models to run this on are the same open-weight ones from the local-LLM tutorial, and the retrieval approach is the alternative to reach for when the real problem is knowledge, not behavior. Not sure whether your problem is a fine-tune or a retrieval problem? That’s the conversation worth having first. It’s exactly where a CloudSignal advisory session earns its place, before a training run, not after.

Sources / further reading

Written by Ashwin Rajendraprasad for CloudSignal AI. The code above is free to reuse.