Fine-Tuning an LLM Project

Learn how to efficiently adapt a large language model to specific tasks using Low-Rank Adaptation (LoRA) without retraining the entire network.

What it is

Fine-tuning an LLM with LoRA involves freezing the original pretrained weights and injecting small, trainable low-rank matrices into specific layers (usually attention projections). Instead of updating millions or billions of parameters, you train only a tiny fraction (often less than 1%). This creates a lightweight "adapter" that can be swapped in and out. Related terms include PEFT (Parameter-Efficient Fine-Tuning), Adapter Layers, and Instruction Tuning.

Why it matters

• Memory Efficiency: Requires significantly less GPU VRAM compared to full fine-tuning, allowing larger models to run on consumer hardware.
• Speed: Training converges faster because there are fewer parameters to update.
• Storage: The resulting adapter files are small (megabytes vs gigabytes), making version control and deployment easier.
• Modularity: You can load different adapters for different tasks onto the same base model at runtime.

Syntax or steps

The core workflow involves three main components: configuring the LoRA setup, preparing the dataset, and running the training loop. 1. Define LoraConfig specifying rank (r) and target modules. 2. Wrap the base model using get_peft_model. 3. Tokenize instruction-response pairs. 4. Initialize Trainer with standard arguments. 5. Save only the adapter weights.

Example

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer
from peft import LoraConfig, get_peft_model
import torch

# 1. Load base model and tokenizer
model_name = "mistralai/Mistral-7B-v0.1" # Example small open model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

# 2. Configure LoRA
lora_config = LoraConfig(
    r=8,                                 # Rank of low-rank matrices
    lora_alpha=16,                       # Scaling factor
    target_modules=["q_proj", "v_proj"], # Attention layers to modify
    lora_dropout=0.05,
    task_type="CAUSAL_LM"
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()       # Verify efficiency

# 3. Prepare Dataset (Assume raw_dataset is a HuggingFace Dataset object)
def format_example(example):
    return f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['response']}"

def tokenize_function(examples):
    texts = [format_example(ex) for ex in examples]
    return tokenizer(texts, truncation=True, padding="max_length", max_length=256)

tokenized_dataset = raw_dataset.map(tokenize_function, batched=True)

# 4. Train
training_args = TrainingArguments(
    output_dir="./lora-finetuned",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    learning_rate=2e-4,                  # Higher LR often needed for LoRA
    logging_steps=10,
    save_strategy="epoch"
)

trainer = Trainer(model=model, args=training_args, train_dataset=tokenized_dataset)
trainer.train()

# 5. Save Adapter Only
model.save_pretrained("./lora-adapter")

Explanation: The code loads a base model but immediately wraps it with get_peft_model. This freezes the original weights and adds trainable lora_A and lora_B matrices. The target_modules list specifies which linear layers receive these adapters. During training, gradients flow only through these new matrices. Finally, save_pretrained writes only the adapter weights, not the full model.

Common mistakes

• Wrong Target Modules: Using incorrect layer names (e.g., query instead of q_proj) results in no parameters being trained. Check the model architecture carefully.
• Learning Rate Too Low: LoRA often requires a higher learning rate (e.g., 2e-4 or 1e-3) than full fine-tuning because the parameter space is smaller.
• Ignoring Padding: Failing to pad sequences to the same length within batches causes shape mismatch errors during training.
• Overfitting Small Datasets: With very few examples, high ranks (r) can lead to overfitting. Start with r=8 or lower.

When to use it

Compare LoRA with Full Fine-Tuning based on resources and goals.

Feature	LoRA	Full Fine-Tuning
Hardware Requirement	Low (Consumer GPUs)	High (Enterprise/A100s)
Training Speed	Fast	Slow
Performance Ceiling	Near-full (95%+)	Highest possible
Best For	Domain adaptation, chatbots	Specialized scientific tasks

Practice

Guided Exercise: Modify the example above to target all linear layers by setting target_modules="all-linear" (if supported by your PEFT version) or explicitly listing k_proj and o_proj. Observe how print_trainable_parameters() changes.
Challenge: Implement inference loading. Write a function that takes the base model name and the path to your saved ./lora-adapter, merges the weights using PeftModel.from_pretrained and merge_and_unload(), and generates text.
Hint: Use from peft import PeftModel.

Quick check

Question: Why does LoRA require a higher learning rate than full fine-tuning? Answer: Because the number of trainable parameters is drastically reduced, the gradient updates need to be more aggressive to converge effectively within the limited parameter space.

Summary

LoRA enables efficient customization of LLMs by training small adapter matrices instead of the full network. It balances performance and resource usage, making advanced AI accessible on limited hardware. Always verify target module names and adjust learning rates accordingly for optimal results.