16:47 29/5/24 Fine-Tuning AI Models: A Guide.

Fine-tuning is a technique for adapting… | by Prabhu Srivastava | Medium

Get unlimited access to the best of Medium for less than $1/week. Become a member

Fine-Tuning AI Models: A Guide

Prabhu Srivastava · Follow
4 min read · Jul 22, 2023

Listen Share More

Fine-tuning is a technique for adapting a pre-trained machine learning model to

new data or tasks. Rather than training a model from scratch, fine-tuning allows you
to start with an existing model like GPT-3 and specialize it for your needs. This
approach can be much more efficient and accurate than training a model from
scratch.

In this article, we’ll learn the fine-tuning process and how to implement it yourself
using tools like the OpenAI API.

What is Fine-Tuning?
Pre-trained models like GPT-3 have been trained on massive datasets to learn
general linguistic skills and knowledge. This gives them strong capabilities out of
the box.

However, their knowledge is still general. To adapt them to specialized domains and
tasks, we can fine-tune the models on smaller datasets specific to our needs.

For example, GPT-3 has not been specifically trained to generate Python code. But
we can fine-tune it on Python data to specialize it for that task.

Fine-tuning adjusts a model’s internal weights to bias it towards new data, without
overwriting everything it has learned. This allows the model to retain its general
skills while gaining new specialized skills.

When to Fine-Tune a Model

https://medium.com/@prabhuss73/fine-tuning-ai-models-a-guide-c515bcd4b580 1/12
16:47 29/5/24 Fine-Tuning AI Models: A Guide. Fine-tuning is a technique for adapting… | by Prabhu Srivastava | Medium

Here are some examples of when fine-tuning can be beneficial:

Adapting to a new domain or genre: Fine-tune a general model on technical

documents to specialize in that field.

Improving performance on a specific task: Fine-tune a model to generate better

poetry or translate between two languages.

Customizing output characteristics: Fine-tune a model to adjust its tone,

personality or level of detail.

Adapting to new data: If your data distribution changes over time, fine-tune the
model to keep up.

In general, fine-tuning is helpful when you want to specialize a general model for
your specific needs.

How to Fine-Tune a Model

Here is an overview of the fine-tuning process:

1. Start with a pre-trained model like GPT-3.

2. Gather a dataset specific to your task. This is known as the “fine-tuning set.”

3. Pass examples from the dataset to the model and collect its outputs.

4. Calculate the loss between the model’s outputs and the expected outputs.

5. Update the model parameters to reduce the loss using gradient descent and
backpropagation.

6. Repeat steps 3–5 for multiple epochs until the model converges.

7. The fine-tuned model can now be deployed for inference on new data.

The training data should be high-quality and representative of the end task. You
typically need hundreds to thousands of examples to effectively fine-tune a large
model.

Fine-Tuning with the OpenAI API

The OpenAI API makes it easy to fine-tune their models like GPT-3.

To fine-tune a model:

https://medium.com/@prabhuss73/fine-tuning-ai-models-a-guide-c515bcd4b580 2/12
16:47 29/5/24 Fine-Tuning AI Models: A Guide. Fine-tuning is a technique for adapting… | by Prabhu Srivastava | Medium

1. Upload your fine-tuning data to OpenAI’s dataset tool.

2. Define a fine-tuning job using their API. Specify the model, dataset, and
hyperparameters.

3. Monitor training progress as the model fine-tunes on your data.

4. When complete, deploy the fine-tuned model or download it for local use.

The API manages everything from scratch, allowing you to focus on curating a
quality dataset.

Use Cases for Fine-Tuned Models

Some examples of real-world use cases for fine-tuned models:

Customer support: Fine-tune a model on customer support tickets to generate

automated responses.

Content generation: Fine-tune on a company’s documentation to automatically

generate content adhering to their voice and style.

Translation: Fine-tune translation models on industry-specific data like medical

or legal documents.

Information retrieval: Fine-tune a QA model to answer domain-specific

questions.

Sentiment analysis: Fine-tune a classifier to identify sentiment in social media

related to your products.

The possibilities are endless! Fine-tuning unlocks highly specific AI applications

using relatively small datasets.

When Not to Fine-Tune

While fine-tuning is powerful, it isn’t always the best approach. Here are some cases
where it may not be beneficial:

Your dataset is very small. Fine-tuning requires hundreds to thousands of

quality examples.

Your task is extremely dissimilar from the original model’s training data. The
model may struggle to connect its existing knowledge to this new domain.

https://medium.com/@prabhuss73/fine-tuning-ai-models-a-guide-c515bcd4b580 3/12
16:47 29/5/24 Fine-Tuning AI Models: A Guide. Fine-tuning is a technique for adapting… | by Prabhu Srivastava | Medium

You need to frequently update or modify the model. Retraining from scratch
allows for more flexibility.

Your problem can be solved with simpler methods. Fine-tuning large models
can be overkill.

Understanding the strengths and limitations of fine-tuning will help guide you to
the best approach.

Putting Fine-Tuning to Work

We’ve covered the fundamentals of fine-tuning, but mastering it still requires
diligence and experimentation. Here are some tips:

Take time to curate a high-quality dataset that represents your end task. Garbage
in, garbage out!

Preprocess your data to clean and normalize it before fine-tuning.

Start with reasonable hyperparameters like a small learning rate, then refine
from there.

Monitor loss during fine-tuning to check if the model is learning properly.

Use a holdout dev set to evaluate the fine-tuned model before finalizing.

Finally, don’t be afraid to iterate! Fine-tuning models takes trial and error. But the
payoff can be state-of-the-art results.

Hopefully this article gives you a solid starting point for leveraging fine-tuning in
your own projects. The techniques give us an efficient path to highly customized AI
applications.

NLP OpenAI Deep Learning

https://medium.com/@prabhuss73/fine-tuning-ai-models-a-guide-c515bcd4b580 4/12
16:47 29/5/24 Fine-Tuning AI Models: A Guide. Fine-tuning is a technique for adapting… | by Prabhu Srivastava | Medium

Follow

Written by Prabhu Srivastava

18 Followers Open in app

Search

More from Prabhu Srivastava

Prabhu Srivastava

SpaCy vs. NLTK: A Comprehensive Comparison of Two Popular NLP

Libraries in Python”
When it comes to Natural Language Processing (NLP) in Python, two popular libraries that are
often compared are spaCy and NLTK. Both…

4 min read · Apr 29, 2023

https://medium.com/@prabhuss73/fine-tuning-ai-models-a-guide-c515bcd4b580 5/12
16:47 29/5/24 Fine-Tuning AI Models: A Guide. Fine-tuning is a technique for adapting… | by Prabhu Srivastava | Medium

Prabhu Srivastava

OpenAI GPT API Parameters: Practical Examples

Introduction

3 min read · Apr 30, 2023

Prabhu Srivastava

The Role of Morphemes and Lexemes in Natural Language Processing

Introduction:
https://medium.com/@prabhuss73/fine-tuning-ai-models-a-guide-c515bcd4b580 6/12
16:47 29/5/24 Fine-Tuning AI Models: A Guide. Fine-tuning is a technique for adapting… | by Prabhu Srivastava | Medium

3 min read · Apr 27, 2023

Prabhu Srivastava

Vector Databases: The Next Generation of Data Storage

As businesses collect more and more data, they need efficient and effective ways to store,
search, and retrieve that data. One solution…

7 min read · Apr 22, 2023

See all from Prabhu Srivastava

Recommended from Medium

https://medium.com/@prabhuss73/fine-tuning-ai-models-a-guide-c515bcd4b580 7/12
16:47 29/5/24 Fine-Tuning AI Models: A Guide. Fine-tuning is a technique for adapting… | by Prabhu Srivastava | Medium

Suman Das

Fine Tune Large Language Model (LLM) on a Custom Dataset with QLoRA
The field of natural language processing has been revolutionized by large language models
(LLMs), which showcase advanced capabilities and…

15 min read · Jan 25, 2024

1.2K 15

Practicing DatScy

Fine-tuning with OpenAI

https://medium.com/@prabhuss73/fine-tuning-ai-models-a-guide-c515bcd4b580 8/12
16:47 29/5/24 Fine-Tuning AI Models: A Guide. Fine-tuning is a technique for adapting… | by Prabhu Srivastava | Medium

I finally was able to test fine-tuning using OpenAI!! Fine-tuning using OpenAI’s gpt models (gpt-
3.5-turbo-1106, gpt-3.5-turbo-0613…

9 min read · Dec 4, 2023

12

Lists

Natural Language Processing

1477 stories · 989 saves

data science and AI

40 stories · 169 saves

The New Chatbots: ChatGPT, Bard, and Beyond

12 stories · 385 saves

AI Regulation
6 stories · 466 saves

Heiko Hotz in Towards Data Science

RAG vs Finetuning — Which Is the Best Tool to Boost Your LLM

Application?
The definitive guide for choosing the right method for your use case

https://medium.com/@prabhuss73/fine-tuning-ai-models-a-guide-c515bcd4b580 9/12
16:47 29/5/24 Fine-Tuning AI Models: A Guide. Fine-tuning is a technique for adapting… | by Prabhu Srivastava | Medium

· 19 min read · Aug 25, 2023

3K 22

Tuan Tran

Fine-Tuning Large Language Model with Hugging Face & PyTorch

Using GPT-2 to generate Cooking recipes

15 min read · Mar 10, 2024

https://medium.com/@prabhuss73/fine-tuning-ai-models-a-guide-c515bcd4b580 10/12
16:47 29/5/24 Fine-Tuning AI Models: A Guide. Fine-tuning is a technique for adapting… | by Prabhu Srivastava | Medium

Harsha Srivatsa

Fine-tuning versus RAG in Generative AI Applications Architecture

This article aims to simplify the choice between Fine-tuning and Retrieval-Augmented
Generation (RAG) and comprehensive insights to make an…

7 min read · Feb 25, 2024

234 1

Kailash Thiyagarajan

https://medium.com/@prabhuss73/fine-tuning-ai-models-a-guide-c515bcd4b580 11/12
16:47 29/5/24 Fine-Tuning AI Models: A Guide. Fine-tuning is a technique for adapting… | by Prabhu Srivastava | Medium

Fine-Tuning Large Language Models with LORA: Demystifying Efficient

Adaptation
Introduction

5 min read · Jan 27, 2024

34 1

See more recommendations

https://medium.com/@prabhuss73/fine-tuning-ai-models-a-guide-c515bcd4b580 12/12