Skip to content

feat: add Unsloth Studio provider (local OpenAI-compatible runtime) - #11692

Open
yannickloth wants to merge 3 commits into
anomalyco:devfrom
yannickloth:add-unsloth-studio
Open

yannickloth wants to merge 3 commits into
anomalyco:devfrom
yannickloth:add-unsloth-studio

Conversation

@yannickloth

Copy link
Copy Markdown

What

Adds unsloth-studio as a provider: Unsloth Studio is a local model runtime (Desktop app / web UI, llama-server-backed) that exposes an OpenAI-compatible API on localhost (default port 8888, per their opencode integration docs; the API docs say "typically 8000 or 8888").

Entry mirrors the existing local-runtime precedent (providers/lmstudio/):

  • provider.toml: @ai-sdk/openai-compatible + @opencode/ai/providers/openai-compatible, api = "http://127.0.0.1:8888/v1", env = ["UNSLOTH_API_KEY"] (documented in their unsloth-start reference: "--api-key Supply a Unsloth API key. You can also set UNSLOTH_API_KEY."), sources commented in-file.
  • 3 example models (gpt-oss-20b, qwen3-coder-30b, qwen3-30b-a3b-2507), zero-cost like LM Studio's local entries. A provider must have at least one model for the v1 generator, but the live inventory of a local runtime is per-machine — the header comment says so explicitly, and opencode discovers it at runtime.
  • No logo (none sourced; several providers ship without one).

Validation

  • bun validate: passes.
  • bun run v1:check: exits 1 with a purely additive diff, as expected for any new provider:
    Added to v1 (3): unsloth-studio/qwen/qwen3-30b-a3b-2507, unsloth-studio/qwen/qwen3-coder-30b, unsloth-studio/openai/gpt-oss-20b
    Changed v1 fields: api {id,env,npm,api,name,doc}: added [unsloth-studio (provider)]
    
    Nothing existing changes; no v1 fields are removed or retyped.

Context

Filed anomalyco/opencode#59557 requesting runtime model discovery for Unsloth Studio in opencode (the same mechanism Ollama/LM Studio/vLLM get via createLocalProviderPlugin). This catalog entry gives the static /connect surface; discovery covers the dynamic per-machine inventory.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/unsloth-studio/provider.toml:8 - Check: New provider has compliant logo.svg. Why: This PR adds providers/unsloth-studio/ with provider.toml plus models but no providers/unsloth-studio/logo.svg, and the PR body states no logo was sourced. AGENTS.md (Adding a provider, Logo blocker, Review checklist) and README.md (Add a Logo, required for new providers) require a currentColor, square-viewBox, plain-markup SVG for every new non-alias provider; providers/lmstudio/, providers/lynkr/, and providers/atomic-chat/ local precedents all ship one. Action: Add providers/unsloth-studio/logo.svg meeting the currentColor/no-fixed-size/plain-markup rules.
  • [medium] [violation] providers/unsloth-studio/models/qwen/qwen3-30b-a3b-2507.toml:5 - Check: Provider base_model files are override-only; lab facts stay in models/. Why: The lab file models/alibaba/qwen3-30b-a3b-instruct-2507.toml has no knowledge, but this provider file adds knowledge = "2025-04" with no leading source comment. knowledge is explicitly non-provider-authored in AGENTS.md; the mirrored providers/lmstudio/models/qwen/qwen3-30b-a3b-2507.toml marks the same value as an unsourced v1-preservation, which this copy drops. Action: Remove knowledge from this provider file to inherit the lab (none), or if 2025-04 is correct, add it to the lab file with a direct source and drop the per-host override.
  • [medium] [possible mistake] providers/unsloth-studio/models/openai/gpt-oss-20b.toml:6 - Check: Host-specific capabilities.reasoning controls and evidence standard. Why: The file asserts effort = ["low", "medium", "high"] with no leading wire-path comment and no mapped citation: the PR body and provider.toml header citations map only to base URL (8888) and UNSLOTH_API_KEY, not to a reasoning-effort field on Unsloth's llama-server-backed /v1 surface. The mirrored LMStudio entry documents that triple on /v1/responses, not its Chat Completions surface, so direct copying risks inventing caller control for this host. Action: Cite Unsloth/llama-server docs or a live probe showing the exact request field that produces low/medium/high on this host and add a leading wire comment (e.g. # Effort: ...); if the host exposes no caller control, reduce to reasoning = { supported = true }.

…t reasoning support only

- logo.svg: official Unsloth mark from unslothai/unsloth (AGPL-3.0),
  embedded as data URI per the atomic-chat precedent (image is on the
  logo allowlist); provenance commented in-file.
- qwen3-30b-a3b-2507: remove knowledge (lab fact, not a host override).
- gpt-oss-20b: reasoning = { supported = true } only — Unsloth's
  llama-server /v1 surface documents no per-request effort control.
@yannickloth

Copy link
Copy Markdown
Author

All three action items addressed in e0f52ff:

  • [high] logo: added providers/unsloth-studio/logo.svg — Unsloth's official mark, extracted from their own repo (unslothai/unsloth, docker/jupyter/unsloth_labext/src/logo.ts, AGPL-3.0, provenance commented in-file). No official SVG exists upstream (their branding ships as PNG only — website and repo both), so it's embedded as a data URI, following the atomic-chat precedent; image is on the packages/core/src/logo.ts allowlist and bun validate passes.
  • [medium] knowledge: removed knowledge = "2025-04" from qwen3-30b-a3b-2507.toml — lab facts don't belong in a host-override file; the value was copied from lmstudio's flagged v1-preservation.
  • [medium] reasoning effort: reduced to reasoning = { supported = true } with a leading citation. Unsloth's llama-server-backed /v1 surface documents thinking control via server launch flags, not a per-request effort field, so the LM Studio /v1/responses triple was indeed not transferable.

bun validate: passing. bun run v1:check: unchanged (additive-only, as disclosed in the PR body).

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/unsloth-studio/logo.svg:2 - Check: Compliant logo for new provider. Why: SVG is an embedded raster <image> (data:image/png;base64,...) with viewBox="0 0 100 97" and fixed width="100" height="97", no currentColor; it violates AGENTS.md Logo rules (vector currentColor, no fixed size/color) and logo.ts allowlist passing does not equal compliance. Action: Replace with a compliant vector redraw using currentColor, no width/height, preferably square viewBox.
  • [high] [possible mistake] providers/unsloth-studio/models/openai/gpt-oss-20b.toml:10 - Check: Reasoning-controls baseline for this host's API. Why: Same model on same-surface peers (providers/groq/.../gpt-oss-20b.toml, providers/openrouter/.../gpt-oss-20b.toml, providers/lmstudio/.../gpt-oss-20b.toml) all author effort = ["low", "medium", "high"]; asserting reasoning = { supported = true } with no control means affirmative no caller control per .opencode/skills/audit-reasoning-options/SKILL.md, but evidence is only a generic unsloth.ai/docs/basics/api link plus launch-flag assertion with no explanation why sibling llama-server-backed lmstudio can expose effort. Action: Verify live Unsloth /v1 surface (reasoning_effort on chat-completions, existence of /v1/responses) and either add the forwarded effort set or cite the host doc section proving per-request control is impossible.
  • [high] [violation] providers/unsloth-studio/models/openai/gpt-oss-20b.toml:7 - Check: Override-only provider file. Why: description = "Open-weight GPT model..." verbatim copies lmstudio's stale string instead of inheriting lab models/openai/gpt-oss-20b.toml ("Open GPT reasoning model...") with no host evidence of a different display description; AGENTS.md allows only provider-specific fields plus real deltas after base_model. Action: Drop description to inherit unless the host documents that display string.
  • [high] [violation] providers/unsloth-studio/models/qwen/qwen3-30b-a3b-2507.toml:3 - Check: Override-only provider file. Why: name/description duplicate lmstudio unsourced strings ("Qwen3 30B A3B 2507" vs lab "Qwen3 30B-A3B Instruct 2507", "Qwen instruction model for multilingual chat, reasoning, and tool use" vs lab "Updated compact Qwen3 MoE..." which is explicitly non-thinking reasoning = { supported = false }); same applies to providers/unsloth-studio/models/qwen/qwen3-coder-30b.toml:3 ("Qwen3 Coder 30B" vs lab "Qwen3-Coder 30B-A3B Instruct"); no host display-name evidence. Action: Drop name/description in both Qwen files to inherit from base_model unless Unsloth documents those display names.

…ed reasoning effort

- logo.svg: compliant currentColor vector redraw of the official badge
  (simplified monochrome: badge ring, sloth-head cutout, eye-patch
  islands, stroked smile); no fixed size, square viewBox.
- gpt-oss-20b: reasoning effort per live probe of /v1/chat/completions
  (server validates reasoning_effort with none/minimal/low/medium/high/
  max/xhigh); drop copied description.
- qwen files: drop copied name/description, inherit from base_model.
@yannickloth

Copy link
Copy Markdown
Author

All four items addressed in d691ce1:

  • [high] logo: replaced the raster embed with a compliant vector redraw — currentColor, no width/height, square viewBox="0 0 100 100", plain markup only (one path with fill-rule="evenodd", two ellipse, one stroked path). It's a simplified monochrome redraw of the official badge: ring + sloth-head cutout, eye patches as filled islands, stroked smile. Upstream has no vector source to copy from (their logo ships as PNG only), so this is hand-traced; provenance commented in-file.
  • [high] gpt-oss-20b reasoning — verified live against a running Studio (v0.1.910-beta): POST /v1/chat/completions with reasoning_effort: "bogus_value" is rejected with "reasoning_effort: Input should be 'none', 'minimal', 'low', 'medium', 'high', 'max' or 'xhigh'" — i.e. a validated per-request control on chat-completions (not /v1/responses; that probe did not answer — the opposite surface split from lmstudio, which documents effort only on /v1/responses). The file now asserts the full probed set ["none", "minimal", "low", "medium", "high", "max", "xhigh"] (all in ReasoningEffortV2Values) with the probe quoted in a leading comment. The probe was validation-only — no model was loaded (invalid value fails before resolution).
  • [high] gpt-oss-20b description: dropped — inherits the lab string.
  • [high] qwen name/description: dropped in both Qwen files — inherit from base_model.

bun validate: passing. The two Qwen files are now pure base_model + zero-cost overrides; gpt-oss-20b adds only the live-probed reasoning controls.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [possible mistake] providers/unsloth-studio/models/openai/gpt-oss-20b.toml:9 - Check: Reasoning effort baseline must be copied from the lab/same-surface peers, not the generic schema enum; extra levels beyond peers need host docs or live meaningful effect. Why: File advertises ["none", "minimal", "low", "medium", "high", "max", "xhigh"] from a single reasoning_effort validation error, while the sibling llama-server host (providers/lmstudio/models/openai/gpt-oss-20b.toml:9), Bedrock (low|medium|high), Databricks (low|medium|high with minimal→low, xhigh/max→high, none as unset), and OpenReason (low|medium|high safe intersection, explicitly excluding none|minimal|xhigh|max as Baseten-only) all publish low|medium|high for gpt-oss; acceptance in a 400 message does not prove distinct effort levels or that none disables reasoning on Unsloth. Action: Restrict to ["low", "medium", "high"] like the LMStudio peer, or provide per-level evidence that each extra value has a distinct reasoning effect on Unsloth chat-completions and that none actually turns reasoning off.
  • [medium] [violation] providers/unsloth-studio/models/openai/gpt-oss-20b.toml:9 - Check: effort must be sorted low to high (none, minimal, low, medium, high, xhigh, max). Why: File orders ... "high", "max", "xhigh" with max before xhigh, contradicting the canonical order enforced in aimlapi.ts, prime.ts, digitalocean.ts and other sync modules where v2 compares effort lists in order. Action: Reorder to ["none", "minimal", "low", "medium", "high", "xhigh", "max"] if the full set is kept, or to the corrected reduced set.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant