Documentation
¶
Overview ¶
Package agenttest provides reusable contract-test machinery for the interfaces declared in core/agent — agent.Engine, agent.Host, agent.Observer, agent.Referee, agent.CheckpointStore, agent.Resumer, agent.StreamSink, agent.Preparer, agent.Committer, agent.CommitViewProvider, agent.CheckpointSuggester and agent.ScriptRuntime.
Modeled on net/http/httptest, testing/iotest, and gocloud.dev/blob/drivertest: a single sibling package next to its subject covers every contract-checking surface the parent package exposes. Importable from any concrete implementation's *_test files; contains exclusively test-support code; nothing in here should be referenced from non-test production paths.
What lives here ¶
EngineSuite — the standard contract every agent.Engine implementation should pass. New engines add a one-liner:
func TestEngineContract(t *testing.T) { agenttest.EngineSuite(t, func() (agent.Engine, agenttest.Capabilities) { return newMyEngine(), agenttest.Capabilities{} }) }
HostSuite — the standard contract every agent.Host implementation should pass:
func TestMyHost_Contract(t *testing.T) { agenttest.HostSuite(t, func() agent.Host { return newMyHost() }) }
ObserverSuite — the standard contract every agent.Observer implementation should pass (no input mutation, prompt return, concurrent safety).
RefereeSuite — the standard contract every agent.Referee implementation should pass.
CheckpointStoreSuite — the shared conformance suite every agent.CheckpointStore implementation should pass, including the optional agent.CheckpointLister / agent.CheckpointDeleter extensions.
ResumerSuite — the contract every agent.Resumer implementation should pass (cheap, classified, mutation-free, concurrent-safe).
StreamSinkSuite — the contract every agent.StreamSink implementation should pass (panic-free, ctx-observing, concurrent-safe).
PreparerSuite — the contract every agent.Preparer implementation should pass (non-nil fresh boards, no input mutation, ctx-observing).
CommitterSuite — the contract every agent.Committer implementation should pass (no input mutation, ctx-observing).
CommitViewProviderSuite — the contract every agent.CommitViewProvider implementation should pass (non-nil projected board, no input mutation, ctx-observing).
CheckpointSuggesterSuite — the contract every agent.CheckpointSuggester implementation should pass (panic-free, prompt return).
ScriptRuntimeSuite — the contract every agent.ScriptRuntime implementation should pass (panic-free, executes a caller-supplied minimal fixture, concurrent-safe).
MockHost — a minimal Host implementation that records every interaction, lets tests inject interrupts / user replies / publish-checkpoint-usage errors / steered messages, and exposes the captured envelopes / usage / checkpoints / steer queue for assertion. It implements the optional agent.SteerSource capability, so engine tests can assert where a script's host.drainSteer() call lands relative to the rest of the round. Engines may use it directly in their own tests instead of re-implementing the full Host surface.
What does NOT live here ¶
This package does not enumerate engine- or host-specific behaviours (graph-edge ordering, script-language semantics, application-specific sandbox bus routing, …). It only asserts what every implementation of the targeted interface is contractually obliged to do. Implementation-specific tests live next to the implementation.
Index ¶
- func CheckpointStoreSuite(t *testing.T, newStore func() agent.CheckpointStore)
- func CheckpointSuggesterSuite(t *testing.T, f SuggesterFactory)
- func CommitViewProviderSuite(t *testing.T, f CommitViewFactory)
- func CommitterSuite(t *testing.T, f CommitterFactory)
- func EngineSuite(t *testing.T, f Factory)
- func HostSuite(t *testing.T, f HostFactory, caps ...HostCapabilities)
- func ObserverSuite(t *testing.T, f ObserverFactory, caps ...ObserverCapabilities)
- func PreparerSuite(t *testing.T, f PreparerFactory)
- func RefereeSuite(t *testing.T, f RefereeFactory, caps ...RefereeCapabilities)
- func ResumerSuite(t *testing.T, f ResumerFactory)
- func ScriptRuntimeSuite(t *testing.T, f ScriptRuntimeFactory, fixture ScriptFixture)
- func StreamSinkSuite(t *testing.T, f SinkFactory)
- type Capabilities
- type CommitViewFactory
- type CommitterFactory
- type Factory
- type HostCapabilities
- type HostFactory
- type MockHost
- func (h *MockHost) AskUser(_ context.Context, p agent.UserPrompt) (agent.UserReply, error)
- func (h *MockHost) Checkpoint(_ context.Context, cp agent.Checkpoint) error
- func (h *MockHost) Checkpoints() []agent.Checkpoint
- func (h *MockHost) DrainSteer() []message.Message
- func (h *MockHost) Envelopes() []event.Envelope
- func (h *MockHost) Interrupt(cause agent.Cause, detail string)
- func (h *MockHost) Interrupts() <-chan agent.Interrupt
- func (h *MockHost) Prompts() []agent.UserPrompt
- func (h *MockHost) Publish(_ context.Context, env event.Envelope) error
- func (h *MockHost) ReportUsage(_ context.Context, usage inference.Usage) error
- func (h *MockHost) SetCheckpointError(err error)
- func (h *MockHost) SetPublishError(err error)
- func (h *MockHost) SetUsageError(err error)
- func (h *MockHost) SetUserReply(reply *agent.UserReply)
- func (h *MockHost) Steer(msg message.Message)
- func (h *MockHost) SteeredMessages() []message.Message
- func (h *MockHost) TotalUsage() inference.Usage
- func (h *MockHost) Usages() []inference.Usage
- type ObserverCapabilities
- type ObserverFactory
- type PreparerFactory
- type RefereeCapabilities
- type RefereeFactory
- type ResumerFactory
- type ScriptFixture
- type ScriptRuntimeFactory
- type SinkFactory
- type SuggesterFactory
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CheckpointStoreSuite ¶
func CheckpointStoreSuite(t *testing.T, newStore func() agent.CheckpointStore)
CheckpointStoreSuite exercises the agent.CheckpointStore contract: absent-load semantics, Save/Load round trips, last-write-wins ordering, caller ownership on both Save and Load, and concurrent use. newStore must return a fresh, ready-to-use store for every call.
The optional CheckpointLister / CheckpointDeleter interfaces are exercised when the store implements them; otherwise those subtests skip.
func CheckpointSuggesterSuite ¶ added in v0.1.2
func CheckpointSuggesterSuite(t *testing.T, f SuggesterFactory)
CheckpointSuggesterSuite runs every contract probe that applies to the agent.CheckpointSuggester produced by f:
- a suggestion on a fresh engine must not panic and may return any error (the host logs/retries; there is no classification contract);
- SuggestCheckpoint is advisory and MUST return promptly — it is not obligated to have written anything by the time it returns.
func CommitViewProviderSuite ¶ added in v0.1.2
func CommitViewProviderSuite(t *testing.T, f CommitViewFactory)
CommitViewProviderSuite runs every contract probe that applies to the agent.CommitViewProvider produced by f:
- zero inputs must not panic;
- a successful CommitView MUST carry a non-nil LastBoard (a nil board with a nil error silently fails finalization);
- CommitView MUST NOT mutate the Request or Result it receives;
- CommitView MUST honour ctx cancellation and return promptly.
func CommitterSuite ¶ added in v0.1.2
func CommitterSuite(t *testing.T, f CommitterFactory)
CommitterSuite runs every contract probe that applies to the agent.Committer produced by f:
- zero inputs must not panic;
- Commit MUST NOT mutate the Request or Result it receives;
- Commit MUST honour ctx cancellation and return promptly.
func EngineSuite ¶
EngineSuite runs every contract test that applies to the engine produced by f. Engines should call this from their own *_test.go:
func TestEngineContract(t *testing.T) {
agenttest.EngineSuite(t, func() (agent.Engine, agenttest.Capabilities) {
return graph.NewEngine(), agenttest.Capabilities{SupportsResume: true}
})
}
Each subtest constructs a fresh engine, so failures isolate cleanly. The whole suite must pass for an implementation to be considered agent.Engine-compliant.
func HostSuite ¶
func HostSuite(t *testing.T, f HostFactory, caps ...HostCapabilities)
HostSuite runs every applicable contract subtest against hosts produced by f. Each subtest builds a fresh host so failures isolate cleanly.
func ObserverSuite ¶
func ObserverSuite(t *testing.T, f ObserverFactory, caps ...ObserverCapabilities)
ObserverSuite runs every applicable contract subtest against observers produced by f.
func PreparerSuite ¶ added in v0.1.2
func PreparerSuite(t *testing.T, f PreparerFactory)
PreparerSuite runs every contract probe that applies to the agent.Preparer produced by f:
- zero inputs must not panic;
- a successful Before MUST return a non-nil, fresh Board (the engine mutates the returned board in place, so returning the input aliases the previous stage's state);
- the previous board must not be mutated on the success path;
- Before MUST honour ctx cancellation and return promptly.
func RefereeSuite ¶
func RefereeSuite(t *testing.T, f RefereeFactory, caps ...RefereeCapabilities)
RefereeSuite runs every applicable contract subtest against deciders produced by f.
func ResumerSuite ¶ added in v0.1.2
func ResumerSuite(t *testing.T, f ResumerFactory)
ResumerSuite runs every contract probe that applies to the agent.Resumer produced by f. Resumer is the optional cheap, local-only admission probe an engine exposes ahead of resume:
- zero / malformed checkpoints must not panic, and any returned error must be classified as Validation (wrong shape) or NotAvailable (recognised but not resumable);
- an empty SpecVersion must never be rejected as a drift failure (the checkpoint contract says empty means "skip drift check");
- the probe must not mutate the caller's checkpoint;
- it must be cheap (no I/O, no LLM calls) and safe for concurrent use.
func ScriptRuntimeSuite ¶ added in v0.1.2
func ScriptRuntimeSuite(t *testing.T, f ScriptRuntimeFactory, fixture ScriptFixture)
ScriptRuntimeSuite runs every contract probe that applies to the agent.ScriptRuntime produced by f:
- zero inputs must not panic (an error is acceptable);
- the declared fixture must execute without an error;
- Exec MUST be safe for concurrent use.
func StreamSinkSuite ¶ added in v0.1.2
func StreamSinkSuite(t *testing.T, f SinkFactory)
StreamSinkSuite runs every contract probe that applies to the agent.StreamSink produced by f:
- zero-value envelopes and payloads must not panic;
- OnDelta MUST observe ctx cancellation and return promptly;
- implementations MUST be safe for concurrent OnDelta calls.
Types ¶
type Capabilities ¶
type Capabilities struct {
// SupportsResume is true when the engine implements Run.ResumeFrom
// (i.e. given a non-nil ResumeFrom whose ExecID matches Run.ID it
// resumes the run rather than returning errdefs.NotAvailable).
//
// Engines that implement agent.Resumer with stricter CanResume
// admission rules (e.g. graph runner requires cp.Steps to name
// real node ids) MAY surface errdefs.Validation when the suite's
// minimal test checkpoint fails admission; the suite accepts
// that as a valid SupportsResume=true outcome since the engine
// honestly rejected an unworkable resume request.
SupportsResume bool
}
Capabilities lets an engine declare which optional behaviours it implements. The contract suite uses this to skip resume-specific subtests when the engine has SupportsResume == false (those subtests instead assert the engine returns NotAvailable).
type CommitViewFactory ¶ added in v0.1.2
type CommitViewFactory func() agent.CommitViewProvider
CommitViewFactory builds a fresh agent.CommitViewProvider for each subtest.
type CommitterFactory ¶ added in v0.1.2
CommitterFactory builds a fresh agent.Committer for each subtest.
type Factory ¶
type Factory func() (agent.Engine, Capabilities)
Factory builds a fresh engine and reports its capabilities. The suite calls Factory once per subtest so subtests do not share engine state.
Engines that take no construction arguments can wrap a constructor:
agenttest.EngineSuite(t, agenttest.NewFactory(graph.NewEngine))
or implement Factory directly when the construction needs more setup.
type HostCapabilities ¶
type HostCapabilities struct {
// SkipAskUserNotAvailable is true when the host is expected
// to SUCCEED on AskUser (a real UI host with a configured
// UserPrompter). The suite then asserts the call returns a
// non-zero reply rather than allowing NotAvailable.
SkipAskUserNotAvailable bool
// SkipPublishConcurrency is true when the host's Publish
// implementation is intentionally serial (e.g. a writer-locked
// log shipper that must preserve order). The suite then keeps
// Publish calls out of the concurrent phase and issues them
// sequentially instead, so the test reflects the real
// serial-only contract.
SkipPublishConcurrency bool
// AcceptsBudgetExceeded, when true, indicates the host's
// ReportUsage MAY return an errdefs.BudgetExceeded error for
// the suite's zero-usage probe (= unusual but legal for hosts
// pre-configured with a zero budget). When false (default),
// the suite asserts ReportUsage(0-usage) returns nil.
AcceptsBudgetExceeded bool
}
HostCapabilities lets a host opt out of subtests that don't apply. Most hosts pass the zero value (= every subtest runs).
type HostFactory ¶
HostFactory builds a fresh agent.Host for each subtest. The suite invokes it once per case so subtests do not share host state.
type MockHost ¶
type MockHost struct {
// contains filtered or unexported fields
}
MockHost is a fully-featured agent.Host for tests. It records every interaction and lets the test inject cooperative interrupts and user replies.
It also implements the optional agent.SteerSource capability, so engine suites can drive host.drainSteer() the way a deployment's steer node would: pre-load messages with MockHost.Steer, then assert what the engine did with them.
All methods are safe for concurrent use; any number of goroutines inside the engine may call them while the test inspects state.
Zero value is NOT ready to use. Call NewMockHost which pre-allocates the channels.
func NewMockHost ¶
func NewMockHost() *MockHost
NewMockHost returns a ready-to-use MockHost. The interrupt channel is buffered so MockHost.Interrupt never blocks the test goroutine when the engine has not yet reached its select on Interrupts().
func (*MockHost) AskUser ¶
AskUser records the prompt and returns the configured reply, or a NotAvailable error when no reply has been set.
func (*MockHost) Checkpoint ¶
Checkpoint records cp and returns the configured checkpointErr.
func (*MockHost) Checkpoints ¶
func (h *MockHost) Checkpoints() []agent.Checkpoint
Checkpoints returns a copy of every checkpoint the engine submitted.
func (*MockHost) DrainSteer ¶ added in v0.4.6
DrainSteer implements agent.SteerSource: it returns clones of everything queued and empties the queue, the way the real turn-owned queue does — the mock never hands out a message it still holds.
func (*MockHost) Interrupt ¶
Interrupt queues an interrupt for the agent. Non-blocking: if the buffer is full the call drops the new signal silently — tests that care should call only once or assert via MockHost.Envelopes that the engine has reached its select.
func (*MockHost) Interrupts ¶
Interrupts returns the cooperative-interrupt channel.
func (*MockHost) Prompts ¶
func (h *MockHost) Prompts() []agent.UserPrompt
Prompts returns a copy of every UserPrompt the engine submitted.
func (*MockHost) Publish ¶
Publish records the envelope and returns the configured publishErr (default nil).
func (*MockHost) ReportUsage ¶
ReportUsage records the usage delta. Multiple calls are kept in order so tests can assert per-call totals; sum them with MockHost.TotalUsage when only the total matters.
func (*MockHost) SetCheckpointError ¶
SetCheckpointError configures all subsequent Checkpoint calls to return err. Pass nil to clear.
func (*MockHost) SetPublishError ¶
SetPublishError configures all subsequent Publish calls to return err. Pass nil to clear.
func (*MockHost) SetUsageError ¶ added in v0.1.2
SetUsageError configures all subsequent ReportUsage calls to return err (typically errdefs.BudgetExceeded). Pass nil to clear.
func (*MockHost) SetUserReply ¶
SetUserReply configures what AskUser returns. Pass nil to revert to the NotAvailable default.
func (*MockHost) Steer ¶ added in v0.4.6
Steer queues msg the way a running turn receives it from Turn.Steer. It never rejects: MockHost models delivery to the engine — admission control (bounds, terminal state, size limits) belongs to the real turn-owned queue and is tested there.
func (*MockHost) SteeredMessages ¶ added in v0.4.6
SteeredMessages returns clones of the messages still queued, without draining them, so a test can assert delivery position and let the engine drain afterwards. Cloning keeps a caller that mutates the returned messages from changing what the next drain delivers.
func (*MockHost) TotalUsage ¶
TotalUsage sums every recorded usage report.
type ObserverCapabilities ¶ added in v0.1.2
type ObserverCapabilities struct {
// SkipMutationCheck is true for observers that intentionally
// mutate the *Request / *Result they receive. The interface
// godoc says "MUST treat as read-only"; this knob exists only
// for the rare migration shim that documents a deliberate
// breach.
SkipMutationCheck bool
// SkipPromptReturnCheck is true for observers that legitimately
// block (e.g. a synchronous flush observer the caller knows
// about). The interface godoc says "blocking blocks the run",
// so the suite enforces a 2s upper bound by default.
SkipPromptReturnCheck bool
}
ObserverCapabilities lets observers opt out of subtests that don't apply. Most observers pass the zero value.
type ObserverFactory ¶
ObserverFactory builds a fresh agent.Observer for each subtest.
type PreparerFactory ¶ added in v0.1.2
PreparerFactory builds a fresh agent.Preparer for each subtest so subtests do not share implementation state.
type RefereeCapabilities ¶
type RefereeCapabilities struct {
// SkipMutationCheck is true when the decider intentionally
// mutates *Result (extremely rare — the interface comment
// says "MUST NOT mutate res"; this knob exists only for
// deciders that document a deliberate breach for migration
// purposes).
SkipMutationCheck bool
// SkipAllStatusProbe is true for deciders that legitimately
// only handle a subset of statuses and panic on others. The
// suite then skips the cross-status probe; the decider is
// expected to ship its own focused unit tests.
SkipAllStatusProbe bool
}
RefereeCapabilities lets a decider opt out of subtests that don't apply. Most deciders pass the zero value (= every subtest runs).
type RefereeFactory ¶
RefereeFactory builds a fresh agent.Referee for each subtest. The suite invokes it once per case so subtests do not share decider state.
type ResumerFactory ¶ added in v0.1.2
ResumerFactory builds a fresh agent.Resumer for each subtest so subtests do not share implementation state.
type ScriptFixture ¶ added in v0.1.2
type ScriptFixture struct {
// Source is a minimal runnable script. The zero value (empty
// source) is a valid trivial script for both bundled runtimes.
Source string
}
ScriptFixture describes a minimal script the runtime under test must execute without any host bindings. The source is passed verbatim to agent.ScriptRuntime.Exec; the suite only requires it to complete without an error (a nil *ScriptSignal is a valid plain completion).
type ScriptRuntimeFactory ¶ added in v0.1.2
type ScriptRuntimeFactory func() agent.ScriptRuntime
ScriptRuntimeFactory builds a fresh agent.ScriptRuntime for each subtest.
type SinkFactory ¶ added in v0.1.2
type SinkFactory func() agent.StreamSink
SinkFactory builds a fresh agent.StreamSink for each subtest so subtests do not share sink state.
type SuggesterFactory ¶ added in v0.1.2
type SuggesterFactory func() agent.CheckpointSuggester
SuggesterFactory builds a fresh agent.CheckpointSuggester for each subtest.
Source Files