Use the API

Direct Flint API access is limited to select partners. These examples also apply to a compatible local server. Flint endpoints use an OpenAI-compatible chat-completions interface. Configure the base URL, key, and model ID using the partner API quickstart. The code below reads the hosted key from NEBIUS_API_KEY. To adapt it for a local server, set the local base URL and model, then replace the key lookup with os.environ["LOCAL_API_KEY"]. Use local for the unauthenticated llama.cpp examples, or the token returned by rec auth token for rec.

Stream a response

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["FLINT_BASE_URL"],
    api_key=os.environ["NEBIUS_API_KEY"],
)

stream = client.chat.completions.create(
    model=os.environ["FLINT_MODEL"],
    messages=[{"role": "user", "content": "Explain Python generators with a short example."}],
    temperature=1.0,
    top_p=0.95,
    max_tokens=4096,
    stream=True,
)

finish_reason = None
for chunk in stream:
    if not chunk.choices:
        continue
    choice = chunk.choices[0]
    if choice.delta.content:
        print(choice.delta.content, end="", flush=True)
    if choice.finish_reason:
        finish_reason = choice.finish_reason
print("\nFinish reason:", finish_reason)

This example displays answer text. Some endpoints expose reasoning in a separate field, so a delay before answer text does not necessarily mean the connection has stalled. Reasoning-field names and streaming behavior depend on the serving stack.

Request configuration

Field	How to use it
`model`	Exact identifier supplied by your endpoint
`messages`	Conversation history with roles and content
`temperature`, `top_p`	Start with recommended defaults
`max_tokens`	Output budget within the provider’s limit
`stream`	Set to `true` for incremental output
`tools`	Function definitions; see tool use

Do not assume every parameter accepted by another compatible API is supported here. In particular, reasoning-effort controls and schema-constrained output are not established hosted features.

Handle errors

Symptom	Check
`401` or `403`	Correct key, account access, and provider
`404`	Base URL, `/v1` convention, and model ID
`400`	Unsupported parameters, malformed messages, or context overflow
`429`	Provider quota or rate limit; respect retry guidance
`5xx` or timeout	Endpoint availability; retry with bounded backoff
Completion ends with `length`	Output budget, reasoning usage, and conversation size

After an interrupted agent request, inspect tool execution state before retrying so a write or external action is not accidentally repeated.

Hosted data handling

Hosted requests leave your device. Use your provider’s applicable terms for retention, logging, and training use; local execution does not establish the policy of a hosted endpoint. Pricing, rate limits, and regions are also provider-specific.

Get started

Local inference

GPU inference

Harness

Resources

Stream a response

Request configuration

Handle errors

Hosted data handling

​Stream a response

​Request configuration

​Handle errors

​Hosted data handling

Stream a response

Request configuration

Handle errors

Hosted data handling