Skip to main content
Use llama.cpp when you want direct control over the runtime build, GGUF artifact, device offload, and local server. Use a Flint-compatible llama.cpp revision and a matching GGUF. Keep the chat template and tool parser aligned with the model.

​
Choose your hardware

Apple Silicon / Metal

Build the supplied runtime with Metal and start a local server.

NVIDIA / CUDA

Build the supplied runtime with CUDA on Linux or WSL2.
Both guides use a supplied artifact and runtime revision, bind the server to localhost, and connect through an OpenAI-compatible endpoint. Keep the model revision, quantization, context length, and tool-parser settings together when reporting results. Prefer rec if you want Recursive to manage the matching model and runtime. For GPU serving with other engines, see SGLang and vLLM.

​
Connect a coding harness

Start llama-server using the Mac or NVIDIA recipe. Keep it running; llama-cli alone does not provide an HTTP endpoint. Our recipes use --alias flint-local, --host 127.0.0.1, and --port 8080. Use API base URL http://127.0.0.1:8080/v1, model flint-local, and placeholder key local for this unauthenticated localhost setup. If you enable server authentication, use its actual API key. Verify discovery:
Match the client context budget to --ctx-size and preserve the supplied Flint chat template and tool parser. Choose your harness: OpenCode, Pi, OpenHands, VS Code, or Codex. For automatic setup with Recursive’s packaged runtime, use Pi with rec, our recommended local harness. OpenCode is also supported.