> ## Documentation Index
> Fetch the complete documentation index at: https://docs.recursive.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Hugging Face Transformers

> Load the public Flint checkpoint from Hugging Face in Python.

Flint is available on Hugging Face at [recursivesi/flint](https://huggingface.co/recursivesi/flint). This guide shows how to load the public BF16 checkpoint in Python with a Flint-compatible Transformers installation.

> [!NOTE]
>
> The checkpoint is public, but its [model card](https://huggingface.co/recursivesi/flint#getting-started) states that Transformers modeling code is not included yet. Direct Transformers loading still requires access to the integration fork below; the [Flint PR](https://github.com/huggingface/new-model-addition-rsi/pull/5) remains open. For the public serving workflow, use [SGLang](https://docs.recursive.com/gpu/sglang.md).

## Install the Flint fork

Use a separate Python environment with PyTorch supported by your hardware. The fork requires PyTorch 2.6 or newer; for CUDA, install the appropriate PyTorch build for your system. See [Transformers installation](https://huggingface.co/docs/transformers/installation).

With Git authenticated to an account that can read the fork, run:

```bash theme={null}
python3 -m venv .venv-flint
source .venv-flint/bin/activate
python -m pip install --upgrade pip

git clone --branch feat/flint-model --single-branch \
  https://github.com/huggingface/new-model-addition-rsi.git transformers-flint && \
  git -C transformers-flint checkout a7d66d92ce34cd44593b1cd1945f5290d725644e && \
  python -m pip install './transformers-flint[torch]'
```

Confirm the installed package includes Flint:

```bash theme={null}
python -c "from transformers import FlintConfig, FlintForCausalLM; print('Flint support is installed')"
```

This checks the Python implementation without downloading model weights.

## Load Flint in Python

Download the public [Flint checkpoint](https://huggingface.co/recursivesi/flint) through `from_pretrained` below. No private-model access is required.

The example below adapts the fork’s [AutoModel example](https://github.com/huggingface/new-model-addition-rsi/blob/a7d66d92ce34cd44593b1cd1945f5290d725644e/docs/source/en/model_doc/flint.md#usage) to a pinned BF16 checkpoint. It has not been run end to end as part of this documentation update.

The weights alone occupy about **79.6 GB**; allow additional memory for activations, attention, and the KV cache. `device_map="auto"` places the model across available devices and may offload to CPU. This is a separate memory requirement from the quantized Mac model managed by [rec](https://docs.recursive.com/edge/rec.md).

```python theme={null}
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "recursivesi/flint"
revision = "2a247be0c0aec895ccb0ac7ca92229ec6df37288"

tokenizer = AutoTokenizer.from_pretrained(model_id, revision=revision)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    revision=revision,
    dtype="auto",
    device_map="auto",
    attn_implementation="eager",
    trust_remote_code=False,
)

messages = [{"role": "user", "content": "Say hello in one sentence."}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=4096,
    do_sample=True,
    temperature=1.0,
    top_p=0.95,
    top_k=20,
)
print(tokenizer.decode(
    outputs[0, inputs["input_ids"].shape[-1]:],
    skip_special_tokens=True,
))
```

The installed fork supplies the model implementation; this example does not load remote model code. Keep the matching tokenizer and chat template unchanged. The public checkpoint is released under the [Apache 2.0 license](https://huggingface.co/recursivesi/flint/blob/main/LICENSE).

## Attention and generation

Flint uses attention sinks. The fork defaults to **eager attention**, which computes attention scores in float32; SDPA does not support the required sinks. Start with a short prompt before attempting long contexts.

For longer inputs, the fork documents sink-aware FlashAttention backends: `kernels-community/vllm-flash-attn3` on Hopper and `flash_attention_4` on Blackwell. Those paths require matching hardware and kernel dependencies; the basic example above uses eager attention.

Generation returns model text, which can include reasoning or tool markup. Loading the model does not provide SGLang’s OpenAI-compatible API or its tool/reasoning parsers. Before connecting a coding harness through your own serving layer, verify the [complete tool cycle](https://docs.recursive.com/guides/coding-tools#verify-a-complete-tool-cycle). For a packaged server with these parsers, use [SGLang](https://docs.recursive.com/gpu/sglang.md).

## Other runtimes

For Recursive’s optimized Mac engine and automatic harness setup, use [rec](https://docs.recursive.com/edge/rec.md). For GPU serving, Flint is supported in both [vLLM](https://docs.recursive.com/gpu/vllm.md) and [SGLang](https://docs.recursive.com/gpu/sglang.md). Follow the setup guide for your preferred engine.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.