Scribd
Upload
143 views3 pages

Vicuna - Open-Source Chatbot - Alternative For GPT-4

GPT-4 is a powerful natural language processing system that can generate coherent and diverse texts on various topics and domains. However, it is not publicly available and requires a lot of computational resources to run. Therefore, there is a need for an alternative model that can offer similar capabilities but is accessible and free for anyone to use. There are few models that can understand and provide informative and engaging responses.

Uploaded by

My Social
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
143 views3 pages

Vicuna - Open-Source Chatbot - Alternative For GPT-4

GPT-4 is a powerful natural language processing system that can generate coherent and diverse texts on various topics and domains. However, it is not publicly available and requires a lot of computational resources to run. Therefore, there is a need for an alternative model that can offer similar capabilities but is accessible and free for anyone to use. There are few models that can understand and provide informative and engaging responses.

Uploaded by

My Social
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

Vicuna-13B: An Open-Source Chatbot Trained with LLaMA and ShareGPT

GPT-4 is a powerful natural language processing system that can


generate coherent and diverse texts on various topics and domains.
However, it is not publicly available and requires a lot of computational
resources to run. Therefore, there is a need for an alternative model that
can offer similar capabilities but is accessible and free for anyone to use.
There are few models that can understand and provide informative and
engaging responses.

General overview of each model:


1. Vicuna: Vicuna is a chat assistant that has been fine-tuned from
LLaMA, a language model, on user-shared conversations. It is
expected to perform well and is similar in performance to Koala,
which is also a chat assistant fine-tuned from LLaMA on
user-shared conversations.

2. Koala: Koala is a chatbot that has been fine-tuned from LLaMA on


user-shared conversations and open-source datasets. It performs
similarly to Vicuna, which is also fine-tuned from LLaMA on
user-shared conversations. Github code repository

3. ChatGLM: ChatGLM is an open bilingual dialogue language model


that is capable of understanding and responding to text in both
English and Spanish. It is fine-tuned from LLaMA and is an
open-source model that can be used for a variety of natural
language processing tasks. Github code repository

4. Alpaca: Alpaca is a model that has been fine-tuned from LLaMA


on 52K instruction-following demonstrations. It is not a chatbot or
chat assistant, but rather a language model that has been trained
on a specific set of data. Github code repository

5. LLaMA: LLaMA is an open and efficient foundation language


model that can be used for a variety of natural language
processing tasks. It is capable of understanding and generating
text in multiple languages and has a wide range of potential
applications. It is the foundation on which several other models,
such as Vicuna, Koala, and ChatGLM, have been fine-tuned.
Github code repository

How Vicuna is Leading the Race?


The collaborative project that involves partners from several leading
institutions, such as UC Berkeley, CMU, Stanford, UC San Diego, and
MBZUAI has developed a chatbot called "Vicuna-13B". Team introduce
Vicuna-13B, an open-source chatbot trained by fine-tuning LLaMA on
user-shared conversations collected from ShareGPT.

Team claims that Preliminary evaluation using GPT-4 as a judge shows


Vicuna-13B achieves more than 90%* quality of OpenAI ChatGPT and
Google Bard while outperforming other models like LLaMA and Stanford
Alpaca in more than 90%* of cases The cost of training Vicuna-13B is
around $300. The training and serving code, along with an online demo,
are publicly available for non-commercial use.

Team presents examples of Alpaca and Vicuna responses to their


benchmark questions. After fine-tuning Vicuna with 70K user-shared
ChatGPT conversations, team discover that Vicuna becomes capable of
generating more detailed and well-structured answers compared to
Alpaca (see examples below), with the quality on par with ChatGPT.
With recent advancements in GPT-4, team is curious whether its
capabilities have reached a human-like level that could enable an
automated evaluation framework for benchmark generation and
performance assessments. Their initial finding indicates that GPT-4 can
produce highly consistent ranks and detailed assessment when
comparing chatbots’ answers. Preliminary evaluations based on GPT-4,
show that Vicuna achieves 90%* capability of Bard/ChatGPT.

source - https://vicuna.lmsys.org/

While this proposed framework shows a potential to automate chatbot


assessment, it is not yet a rigorous approach. Building an evaluation
system for chatbots remains an open question requiring further research.
More details are provided in the below section.

Vicuna Model Training:

Team created Vicuna by fine-tuning a LLaMA base model using around


70,000 conversations gathered from ShareGPT.com with public APIs.
They ensured data quality by filtering out inappropriate or low-quality
samples, converting the HTML back to markdown, and dividing lengthy
conversations into smaller segments that fit the model’s maximum
context length. Team made several improvements to the training recipe,
such as expanding the maximum context length to 2048, enabling
understanding of long context, and reducing the cost of training by
employing SkyPilot managed spot. They built a serving system that can
work with cheaper spot instances to reduce serving costs…read more

You might also like