MiniMind
A free, open-source local LLM training toolkit with dense and mixture-of-experts models, Apache 2.0 licensing, and Ollama compatibility.
MiniMind is a toolkit for learners and developers who want to understand language models by training small ones on their own hardware. Its core training algorithms use native PyTorch code that you can inspect and change.
- Model and tokenizer training: Build dense or mixture-of-experts models aligned with Qwen3 and Qwen3-MoE, with tokenizer training code and datasets for different training stages.
- Fine-tuning and learning: Experiment with supervised fine-tuning, LoRA, preference optimization, reinforcement learning and distillation after pretraining.
- Agent capabilities: Train for tool use and multi-turn agent tasks, with a Streamlit chat interface that displays model thinking and supports tool selection and calls.
- Inference integrations: Use compatible runtimes including Transformers, llama.cpp, vLLM and Ollama, or connect FastGPT and Open-WebUI through a self-hosted OpenAI-compatible API.
- Evaluation: Assess models with benchmark suites such as C-Eval, C-MMLU and OpenBookQA.
MiniMind is free and open source under Apache 2.0. Training supports a single GPU or multiple GPUs on one machine, with pause and resume support. Its small models are intended to make training feasible on personal GPUs.
CUDA, CPU and Apple MPS backends are available, though training speed and compatibility differ substantially. Optional Weights & Biases and SwanLab integrations provide training visualizations.
