> ## Documentation Index
> Fetch the complete documentation index at: https://docs.recursive.com/llms.txt
> Use this file to discover all available pages before exploring further.

# llama.cpp

> Build a compatible runtime and serve a Flint GGUF locally.

Use llama.cpp when you want direct control over the runtime build, GGUF artifact, device offload, and local server.

Use a Flint-compatible llama.cpp revision and a matching GGUF. Keep the chat template and tool parser aligned with the model.

## Choose your hardware

> [card] [Apple Silicon / Metal](https://docs.recursive.com/local/mac)
>
> Build the supplied runtime with Metal and start a local server.

> [card] [NVIDIA / CUDA](https://docs.recursive.com/local/nvidia)
>
> Build the supplied runtime with CUDA on Linux or WSL2.

Both guides use a supplied artifact and runtime revision, bind the server to localhost, and connect through an OpenAI-compatible endpoint. Keep the model revision, quantization, context length, and tool-parser settings together when reporting results.

Prefer [rec](https://docs.recursive.com/edge/rec.md) if you want Recursive to manage the matching model and runtime. For GPU serving with other engines, see [SGLang](https://docs.recursive.com/gpu/sglang.md) and [vLLM](https://docs.recursive.com/gpu/vllm.md).

## Connect a coding harness

Start `llama-server` using the [Mac](https://docs.recursive.com/local/mac) or [NVIDIA](https://docs.recursive.com/local/nvidia) recipe. Keep it running; `llama-cli` alone does not provide an HTTP endpoint. Our recipes use `--alias flint-local`, `--host 127.0.0.1`, and `--port 8080`.

Use API base URL `http://127.0.0.1:8080/v1`, model `flint-local`, and placeholder key `local` for this unauthenticated localhost setup. If you enable server authentication, use its actual API key. Verify discovery:

```bash theme={null}
curl --fail-with-body --silent --show-error http://127.0.0.1:8080/v1/models
```

Match the client context budget to `--ctx-size` and preserve the supplied Flint chat template and tool parser.

Choose your harness: [OpenCode](https://docs.recursive.com/guides/coding-agents.md#connect-local-runtimes), [Pi](https://docs.recursive.com/guides/pi.md#connect-local-runtimes), [OpenHands](https://docs.recursive.com/guides/openhands.md#connect-local-runtimes), [VS Code](https://docs.recursive.com/guides/vscode#llama-cpp-custom-configuration), or [Codex](https://docs.recursive.com/guides/codex#connect-local-runtimes).

For automatic setup with Recursive’s packaged runtime, use [Pi with rec, our recommended local harness](https://docs.recursive.com/edge/rec.md#recommended-harness). OpenCode is also supported.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.