LazyLLM

LazyLLM is a Python framework for developers building applications with multiple AI agents. It brings application workflows, model training and evaluation together so teams can refine a prototype using task-specific data.

• Agent workflows: Combine model and functional modules into workflows with branching, loops and parallel tasks, with MCP integration for task planning and tool use.
• Document question answering: Build retrieval-augmented generation (RAG) applications with document parsing, retrieval and reranking, plus browser interfaces for chat and document management.
• Local and cloud models: Use a common interface for local inference through vLLM, LMDeploy or LightLLM and hosted providers such as OpenAI, Kimi and Qwen; Infinity supports local embeddings.
• Fine-tuning and evaluation: Assess application results and adapt models with local training frameworks including LLaMA-Factory, Alpaca-LoRA and Collie, or supported online fine-tuning services.

LazyLLM is open source under Apache 2.0. It runs on development machines, bare-metal servers and Slurm clusters, and provides a Docker image. Applications can also be packaged for Kubernetes deployment.

Local model services run on your own hardware. Hosted model connections require provider accounts and API keys, and requests go to the selected provider.