Documentation
¶
Overview ¶
Package config loads versioned, secret-reference-only inference configuration and materializes immutable inference provider definitions.
Provider-specific packages own the interpretation of ProviderConfig.Spec and ProfileConfig.Spec. This package deliberately does not define model capability flags or provider wire rules.
Package config loads versioned inference deployment configuration and assembles inference.Runtime instances from it.
The document is JSON at the protocol level; YAML is accepted as authoring sugar through sdk/config/utils, which detects JSON by the Kubernetes rule (first non-whitespace byte is an open brace) and converts YAML before strict decoding. Provider specs stay as opaque JSON subtrees and are decoded by the owning provider factory through DecodeSpec.
The package enforces a SecretRef-only boundary: documents never contain plaintext credentials, and every credential is resolved through an explicit SecretResolver catalog at build time. Provider packages own their Spec decoding and model rules through the Factory contract; this package owns envelope validation, secret resolution, and assembly.
Index ¶
- Constants
- func DecodeSpec[T any](raw json.RawMessage) (T, error)
- func NewDeployFactory(factories map[string]Factory, resolvers map[string]SecretResolver) config.Factory
- type Assembly
- type Builder
- func (b *Builder) Build(ctx context.Context, document Document) ([]inference.ProviderDefinition, error)
- func (b *Builder) NewAssembly(ctx context.Context, document Document, options ...inference.RuntimeOption) (Assembly, error)
- func (b *Builder) NewRuntime(ctx context.Context, document Document, options ...inference.RuntimeOption) (*inference.Runtime, error)
- type Document
- type Factory
- type FactoryFunc
- type ProfileConfig
- type ProviderConfig
- type ProviderInput
- type ResolvedProfile
- type Secret
- type SecretRef
- type SecretResolver
- type SecretResolverFunc
Constants ¶
const ResourceKind = "inference.Assembly"
ResourceKind is the deployment resource category this package builds. The value bound is a *Assembly, holding the Runtime and the optional Router: graph's inference node needs both, and a route policy is meaningless once separated from the runtime it validated against.
const VersionV1 = "v1"
Variables ¶
This section is empty.
Functions ¶
func DecodeSpec ¶
func DecodeSpec[T any](raw json.RawMessage) (T, error)
DecodeSpec lets a provider factory strictly decode its own typed Spec. Provider configuration types should deliberately omit credentials: resolved secrets arrive separately through ResolvedProfile.Secrets.
func NewDeployFactory ¶
func NewDeployFactory( factories map[string]Factory, resolvers map[string]SecretResolver, ) config.Factory
NewDeployFactory returns the deployment factory for inference assemblies.
Provider factories and secret resolvers are Go values that the document cannot name. A deployment declares WHICH providers it wants and where their credentials live; the host decides which driver code and which secret backends exist in the binary. That split is what keeps credentials out of the document — profiles carry SecretRef, never values.
Types ¶
type Assembly ¶
type Assembly struct {
Runtime *inference.Runtime
// Router is nil when the document has no route section; callers then
// address exact models through Runtime directly.
Router *route.Router
// Policy is the route policy the Router was built from, or nil when the
// document has no route section. It is an owned copy, so consumers can
// inspect targets (for example to derive policy digests or validate
// stable embedding identities) without mutating the document.
Policy *route.Policy
}
Assembly is the fully built output of one Document: the runtime plus, when the document declares a route section, a Router composed above it.
type Builder ¶
type Builder struct {
// contains filtered or unexported fields
}
Builder is immutable after construction and safe for concurrent use when its factories and resolvers are safe for concurrent use. Catalogs are instance-owned; package-global provider registration is intentionally absent.
func NewBuilder ¶
func (*Builder) NewAssembly ¶
func (b *Builder) NewAssembly( ctx context.Context, document Document, options ...inference.RuntimeOption, ) (Assembly, error)
NewAssembly builds the runtime and, when document.Route is set, composes the deployment Router above it. The Router is driven by the route policy in declared order (see Policy.Selectors); callers that need score-aware or request-aware selection build route.New themselves with custom Selectors. Route targets were already validated against the runtime, so Router construction cannot fail on target resolution.
type Document ¶
type Document struct {
Version string `json:"version"`
Providers []ProviderConfig `json:"providers"`
// Route declares deployment routing tiers and target pools. It is
// optional: nil means callers select exact models themselves. Builder
// validates every target against the assembled runtime at build time,
// so a snapshot referencing unknown or retired models fails fast.
Route *route.Policy `json:"route,omitempty"`
}
Document is one complete, immutable deployment snapshot.
type Factory ¶
type Factory interface {
Build(context.Context, ProviderInput) (inference.ProviderDefinition, error)
}
Factory is implemented by a provider package. It owns strict decoding of provider/profile Spec, model rules, compiler construction, and validation of provider-specific secret names.
type FactoryFunc ¶
type FactoryFunc func( context.Context, ProviderInput, ) (inference.ProviderDefinition, error)
func (FactoryFunc) Build ¶
func (f FactoryFunc) Build( ctx context.Context, input ProviderInput, ) (inference.ProviderDefinition, error)
type ProfileConfig ¶
type ProfileConfig struct {
ID string `json:"id,omitempty"`
Operations []inference.Operation `json:"operations,omitempty"`
Secrets map[string]SecretRef `json:"secrets,omitempty"`
Spec json.RawMessage `json:"spec,omitempty"`
}
ProfileConfig declares one credential profile. Secrets contains references, never resolved values. Spec contains provider-owned, non-secret settings such as endpoints, organizations, or projects.
func (ProfileConfig) Clone ¶
func (p ProfileConfig) Clone() ProfileConfig
type ProviderConfig ¶
type ProviderConfig struct {
ID string `json:"id"`
Driver string `json:"driver"`
Profiles []ProfileConfig `json:"profiles,omitempty"`
Spec json.RawMessage `json:"spec,omitempty"`
}
ProviderConfig is the provider-neutral envelope consumed by a provider Factory. Spec contains provider-owned, non-secret configuration.
func (ProviderConfig) Clone ¶
func (p ProviderConfig) Clone() ProviderConfig
type ProviderInput ¶
type ProviderInput struct {
ID string
Profiles []ResolvedProfile
Spec json.RawMessage
}
func (ProviderInput) Clone ¶
func (i ProviderInput) Clone() ProviderInput
type ResolvedProfile ¶
type ResolvedProfile struct {
ID string
Operations []inference.Operation
Secrets map[string]Secret
Spec json.RawMessage
}
func (ResolvedProfile) Clone ¶
func (p ResolvedProfile) Clone() ResolvedProfile
type Secret ¶
type Secret struct {
// contains filtered or unexported fields
}
Secret is an owned, non-serializable resolved credential value. Bytes returns a copy so callers cannot mutate the value retained by a provider factory.
func (Secret) MarshalJSON ¶
func (Secret) MarshalText ¶
type SecretResolver ¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package env resolves secret references from process environment variables.
|
Package env resolves secret references from process environment variables. |
|
Package file resolves secret references from mounted secret files, the layout used by Kubernetes secret volumes, Docker secrets, and systemd credentials.
|
Package file resolves secret references from mounted secret files, the layout used by Kubernetes secret volumes, Docker secrets, and systemd credentials. |