What is Parameter-Efficient Fine-Tuning (PEFT)?

Parameter-Efficient Fine-Tuning (PEFT) is a method used to fine-tune Large Language Models (LLMs) by updating a small subset of the model's parameter while keeping the majority of the pre-trained weights frozen.

This makes fine-tuning much more efficient in terms of:

• Computational cost: Less computing power is required.
• Storage: Task-specific modules take up minimal storage.
• Training time: It’s faster because fewer parameters are being updated.

Problem with Traditional Fine-Tuning

Fine-Tuning takes the pre-trained model and adapt the model for specific task. For example, BERT or GPT can be fine-tuned to perform sentiment analysis or text summarization. Traditionally, fine-tuning involves updating all the parameters (weights) of the model based on the new data. This works well but the problem is that modern Large Language Models (LLMs) are huge and have billions of parameters.

Imagine we have a large language model with 100 billion parameters. If we fine-tune all those parameters for every new task, we'll need a lot of computing power and storage. To solve this problem, researchers developed Parameter-Efficient Fine-Tuning (PEFT). With PEFT, we can achieve similar performance by tweaking only a small fraction of the model making it faster, cheaper and easier to manage while still giving strong performance.

Working of Parameter-Efficient Fine-Tuning

Let's see how it works step by step:

1. Start with a Pre-trained Model: We begin with a large language model (LLM) such as GPT, BERT or T5 that is already trained on massive amounts of text. This model contains general knowledge about language.

2. Freeze the Original Weights: Instead of updating all the billions of weights, we keep them fixed. This saves computing power and avoids the need to store multiple full versions of the model.

3. Add Lightweight Components: Since the main model is frozen, we introduce tiny, task-specific modules or allow updates only to certain parameters. These are the only parts that will be trained. Some common techniques are:

• Adapters: Small layers added between existing layers to capture task-specific adjustments.
• LoRA (Low-Rank Adaptation): Adds compact matrices that approximate updates without touching the main weights.
• BitFit: Updates only bias terms in the model which are very few in number.
• Prompt / Prefix Tuning: Attaches short, learnable prompt vectors that guide the model’s behavior for each task.

4. Train Only the New Parts: During fine-tuning, we update just the added or selected parameters and the frozen weights stays untouched.

5. Deploy Efficiently: For each new task, we only need to save the small trained modules not the full model which means:

• One big pre-trained model can serve many tasks.
• Switching tasks is as simple as loading the right lightweight module.

PEFT Techniques for LLMs

PEFT is not a single method but a family of techniques. The choice of method depends on the task, resources and flexibility needed. Let's see some popular techniques used with large language models:

1. Adapter Modules

• Adapters Modules are small, trainable modules inserted between the layers of a pre-trained model. During fine-tuning, only the adapter modules are updated while the original model weights remain fixed. Once fine-tuned, it can be easily added or removed allowing for modular customization of the model.
• It allow for efficient multi-task learning where different adapters can be used for different tasks while sharing the same base model.
• For example: The Hugging Face AdapterHub provides an extensive library of pre-trained adapters for various NLP tasks.

2. LoRA (Low-Rank Adaptation)

• LoRA (Low-Rank Adaptation) reduces the number of trainable parameters by decomposing weight updates into low-rank matrices. Instead of updating the entire weight matrix, it modifies only a small, low-rank component which approximates the changes needed for fine-tuning.
• It achieves results close to full fine-tuning but with far fewer parameters.
• It has been successfully applied to LLMs like GPT-3 and T5 making it a popular choice for parameter-efficient fine-tuning at scale.

3. DoRA (Weight-Decomposed Low-Rank Adaptation)

• DoRA builds upon the concept of LoRA but introduces a novel weight-decomposed approach to further enhance efficiency. In DoRA, the weight matrix is decomposed into two components i.e a low-rank update and a scaling factor.
• It also maintains the low computational cost of LoRA while potentially improving performance.
• It is useful in scenarios where fine-tuning must be both efficient and robust such as in cross-domain applications or when adapting models to new languages.

4. Prefix Tuning

• Prefix tuning works by adding a small set of learnable "prefix" tokens to the model’s input at every layer. These prefix tokens act as task-specific prompts that helps the model's behavior without changing its original parameters.
• It allows the model to retain its general knowledge while adapting to specific tasks through the learned prefixes.
• It is used for tasks like text generation where controlling the output style or content is important.

5. Prompt Tuning

• Prompt tuning involves adding a set of learnable soft prompts to the input sequence. However, instead of modifying internal model layers, it operates completely at the input level making it even simpler to implement.
• It is lightweight and works well for tasks that require minimal changes to the model architecture.
• It works well in few-shot learning where we only have a small amount of labeled data and for tasks where quick adaptation is needed.

6. BitFit (Bias-Term Fine-Tuning)

• BitFit focuses on fine-tuning only the bias terms of a neural network while keeping all other parameters frozen. Despite its simplicity, it has showed competitive results on various NLP benchmarks.
• It requires minimal changes to the model and is efficient in terms of both computation and memory.
• Its effectiveness may vary depending on the complexity of the task and the architecture of the model.

7. (IA)³ (Infused Adapter by Inhibiting and Amplifying Inner Activations)

• (IA)³ takes a different approach compared to other PEFT methods. Instead of adding new modules or training extra parameters, it controls how the model’s internal activations behave during the forward pass.
• It offers fine-grained control over how the model processes information making it suitable for tasks that require subtle adjustments to the model's behavior.
• It has been shown to be effective in tasks such as text classification where slight modifications to the model's internal representations can lead to significant performance improvements.

Implementation of PEFT (LoRA) with BERT on IMDb Sentiment Analysis

Here we will see a practical implementation of Parameter-Efficient Fine-Tuning (PEFT) using LoRA on the IMDb movie reviews dataset. Instead of updating the entire BERT model, we will fine-tune only small LoRA modules, saving time and resources while still achieving strong performance.

1. Installing Required Libraries

We will be using Transformers, Datasets, Peft, Accelerate and Scikit Learn libraries for this implementation.

!pip install -q transformers datasets peft accelerate evaluate scikit-learn

2. Loading Model & Dataset

We use BERT-base-uncased as our pre-trained model which already has strong language understanding. The IMDb dataset contains 50k movie reviews labeled as positive or negative. This combination makes it a great benchmark to show PEFT for sentiment analysis.

from transformers import AutoModelForSequenceClassification, AutoTokenizer, TrainingArguments, Trainer
from datasets import load_dataset
from peft import LoraConfig, get_peft_model

model_name = "bert-base-uncased"
dataset = load_dataset("imdb")  
tokenizer = AutoTokenizer.from_pretrained(model_name)

3. Preprocessing Data

Before training, we tokenize the reviews so that BERT can process them. Each review is truncated or padded to a maximum length of 128 tokens for consistency. Finally we rename label to labels and set the dataset format to PyTorch tensors.

def preprocess(example):
    return tokenizer(example["text"], truncation=True, padding="max_length", max_length=128)

encoded_dataset = dataset.map(preprocess, batched=True)
encoded_dataset = encoded_dataset.rename_column("label", "labels")
encoded_dataset.set_format("torch")

4. Configuring LoRA

Instead of updating all BERT weights, we configure LoRA (Low-Rank Adaptation) to inject small trainable matrices inside the attention layers (query, value). This reduces the number of trainable parameters while still adapting the model effectively. The dropout helps avoid overfitting during fine-tuning.

lora_config = LoraConfig(
    r=8,                          
    lora_alpha=16,                
    target_modules=["query", "value"],  
    lora_dropout=0.05,
    bias="none",
    task_type="SEQ_CLS"           
)

model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)
model = get_peft_model(model, lora_config)

5. Training

We define training arguments using Hugging Face’s Trainer.

• Batch size is set to 16 for both training and evaluation.
• Learning rate is slightly higher (2e-4) since we are only training small LoRA layers.
• For demonstration, we train on a subset (2000 training + 1000 test samples) to reduce runtime.

training_args = TrainingArguments(
    output_dir="./lora-imdb",
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    learning_rate=2e-4,
    num_train_epochs=2,
    logging_dir="./logs",
    report_to="none",
    eval_strategy="epoch",
    save_strategy="epoch"
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=encoded_dataset["train"].shuffle(seed=42).select(range(2000)), 
    eval_dataset=encoded_dataset["test"].shuffle(seed=42).select(range(1000)),
)

trainer.train()
model.save_pretrained("./lora-imdb-adapter")

Output:

6. Making Predictions

Once trained, we can test our model on new sentences. We load the fine-tuned LoRA adapter and run a few sample reviews through a sentiment pipeline. The output gives us the predicted label (POSITIVE or NEGATIVE) along with confidence scores between 0 and 1.

from transformers import pipeline

sentiment = pipeline(
    "text-classification",
    model="./lora-imdb-adapter",
    tokenizer="bert-base-uncased"
)
label_map = {"LABEL_0": "NEGATIVE", "LABEL_1": "POSITIVE"}

examples = [
    "I absolutely loved this film—best sci-fi I’ve seen in years!",
    "It was okay, not great, but worth a watch.",
    "Terrible plot, terrible acting, total waste of time."
]

for text in examples:
    result = sentiment(text)[0]
    print(f"{text[:40]}... → {label_map[result['label']]} ({result['score']:.2f})")

Output:

The results are decent but not highly accurate yet, mainly because we trained for only 2 epochs on a small subset of the IMDb dataset (2000 samples). With longer training, more data, hyperparameter tuning or larger LoRA ranks, the model’s performance would improve significantly.

You can download source code from here.

Full Fine-Tuning vs PEFT

When we compare full fine-tuning with parameter-efficient fine-tuning (PEFT), the differences become clear:

Applications of PEFT

Some key applications of PEFT include:

1. Edge Deployment: Fine-tuning models to run on devices with limited memory or computing power such as mobile phones, IoT devices or embedded systems.
2. Multi-Task Learning: Using one shared base model with separate lightweight adapters for different tasks. This avoids storing a full copy of the model for each task.
3. Few-Shot and Zero-Shot Learning: Applying methods like prompt or prefix tuning to get strong results with little or no labeled data.
4. Personalized AI Models: Creating task-specific or user-specific adapters so organizations and individuals can have custom models without retraining everything.
5. Domain Adaptation: Specializing a general-purpose model for specific areas such as healthcare, law, finance or scientific research without the cost of full fine-tuning.

Challenges with PEFT

While Parameter-Efficient Fine-Tuning solves many problems of traditional fine-tuning, it also comes with its own set of challenges:

1. Performance Gaps: In some cases, full fine-tuning still delivers slightly higher accuracy, especially for very complex tasks that require deep adaptation.
2. Picking the Right Method: It includes techniques like LoRA, Adapters, Prefix-Tuning and BitFit. Each has its strengths but also limitations. Choosing the right one for a specific task is not always straightforward.
3. Generalization Issues: Some PEFT models may perform well on the training dataset but struggle to generalize to different domains or unseen data.
4. Managing Multiple Adapters: When we use one model for many tasks, we often end up with many small adapters. Keeping track of them and integrating them efficiently can become complex.