Tools to Fine-Tune LLMs Locally

LLaMA-Factory is a local model training platform for developers and researchers who want to adapt pretrained models to their own tasks. Its web UI lets you fine-tune models without writing code.

RD Agent is a self-hosted AI research agent for data scientists, researchers and quantitative developers. It runs experiments on a Linux machine or server, while chat and embedding requests use providers through LiteLLM, including OpenAI and Azure OpenAI. Those remote configurations require API credentials and internet access; hosting the agent yourself doesn't move their inference onto your hardware.

npcpy is a Python library for developers and researchers building AI agents and applications with language models. It supports local LLM backends as well as cloud providers, so projects can choose where model requests run.

Ludwig is a framework for developers and researchers training custom AI models without writing their own training loops. It handles preprocessing, training and evaluation from a model definition, covering LLMs as well as tabular and multimodal tasks.

Oumi helps ML researchers and teams build specialized AI models for their own tasks. It brings data preparation, training, evaluation, and deployment into one platform, with reusable recipes and components you can customize.

DB-GPT is a self-hosted AI data assistant for teams analyzing business data and developers building data applications. It connects plain-language questions to database queries and code-based analysis.

RWKV is a language model architecture for developers and researchers building LLM applications or training models. Its recurrent design keeps inference memory constant as text grows, making it relevant when a growing context cache would strain available memory.

PEFT is a library for developers adapting pretrained AI models on their own hardware. It trains a small portion of a model's parameters to reduce the compute and storage costs of full fine-tuning.

XTuner is an LLM training engine for researchers working with very large mixture-of-experts (MoE) models. Its focus is reducing memory pressure and improving distributed training efficiency when model experts receive uneven workloads.

H2O LLM Studio helps teams adapt language models to their own datasets through a browser interface, without writing training code. It's aimed at organizations building models for specific tasks.

LazyLLM is a Python framework for developers building applications with multiple AI agents. It brings application workflows, model training and evaluation together so teams can refine a prototype using task-specific data.

MaxText is an LLM training library for researchers and engineers building models from scratch or adapting existing ones. It provides a reference implementation teams can modify for their own model designs, with training that scales from a single host to large clusters.

StarCoder2 is a family of code generation models for developers who want code completion on their own hardware. These models predict continuations of code; they aren't instruction models, so conversational requests aren't their intended use.

AutoTrain supports model training on your own hardware through AutoTrain Advanced, a no-code application for people fine-tuning LLMs or building models from their data. Advanced can run locally or on cloud infrastructure. The project is no longer maintained, and it won't receive bug fixes or new features.

Colossal-AI is a Python framework for developers and researchers training or serving large AI models on their own GPUs. It provides tools to spread workloads across GPUs and work within available memory.

text-generation-webui, also called TextGen, runs language models on your own hardware for chat, writing and application integrations. You can use it as a desktop app or a self-hosted browser interface.

PaddleNLP is a Python toolkit for developers training and deploying language models on their own infrastructure. Built on PaddlePaddle, it covers pretraining, fine-tuning, model compression and inference across several accelerator families.

Soup is a local LLM fine-tuning tool for developers and teams adapting models to their own datasets. It brings training, evaluation and export into one workflow that runs on your own hardware without requiring a cloud service.

OpenRLHF is a self-hosted Python framework for researchers and teams training language models with human feedback or custom rewards. It supports fine-tuning and reinforcement learning on your own GPU hardware, including training agents that interact with an environment.

Augmentoolkit turns documents into datasets for training an LLM on a particular subject. It's for people building custom models around material such as research papers or fictional lore.

verl is a Python library for researchers and engineers training large language models on their own hardware. It implements HybridFlow for reinforcement learning after initial training, with support for supervised fine-tuning too.

ms-swift is a Python framework for developers and researchers training and deploying language and multimodal models on their own hardware. It covers model adaptation through evaluation and serving, with support for Qwen3, DeepSeek-R1, Llama4, Mistral, Qwen3-VL and InternVL3.5.

Axolotl is a free, open-source LLM fine-tuning framework for developers and researchers adapting models to their own data. It supports training on local hardware or cloud infrastructure you control.

DataDreamer is a Python library for researchers and developers who need to create training datasets and adapt LLMs to specific tasks. It connects data preparation and model training in reproducible workflows.

MLX VLM is a Python package for developers and researchers who want to run multimodal models on their own Mac. It supports local inference and fine-tuning through MLX.

Kaito manages LLM inference, fine-tuning and document retrieval services in Kubernetes. It's for teams that want to run models in their own cluster with less manual work sizing GPU capacity and managing deployments.

HarnessX is a Python framework for developers building AI agents whose behavior they can reuse and adapt. It separates model choice from tool use, memory and context handling, so changing an agent's role doesn't require rewriting it.

torchtitan is a training platform for researchers and developers experimenting with generative AI models on their own GPU servers or clusters. Its replaceable components and extension points support work on model architectures and training infrastructure.

AI Toolkit (ostris) is a training suite for people fine-tuning image and video generation models on their own hardware or rented servers. It provides a browser interface and command-line access to training jobs.

SkyRL is a Python library for researchers and developers who want to train LLMs with reinforcement learning on their own hardware. Its Tinker API support lets you run existing Tinker training scripts against a server you control without changing the scripts. It's open source under Apache 2.0.

LLaVA is a vision-language model project for researchers and developers who want to ask questions about images on their own hardware.

FastChat is a self-hosted LLM platform for developers and researchers who want to serve chat models on their own hardware and compare their responses. It powers Chatbot Arena and includes a locally hosted side-by-side comparison interface. The Python code is open source under Apache 2.0; model licenses apply separately, including Llama's license for Vicuna.

HQQ is a Python library for developers compressing language and vision models to fit the memory available on their own hardware. Its Half-Quadratic Quantization method doesn't require a calibration dataset.

Kiln is a desktop workbench for teams building and testing AI applications. Engineers, QA staff and subject matter experts can work with the same tasks and datasets, including contributing ratings and examples without writing code.

Kubeflow is a self-hosted AI platform for teams developing and running machine learning workloads on Kubernetes. Its modular projects cover interactive development, training and the work of moving models into production.

DSPy is a Python framework for developers building language model applications with local backends such as Ollama on a laptop or SGLang on a GPU server. It connects application code to those model servers, which handle inference on your hardware. It also supports cloud providers, so where model requests go depends on the backend you choose.