detection

package
v0.2.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 28, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package detection is the pure domain for the agent-side blue-team detection engine (issue #422): the typed event classes an eBPF sensor observes, the clean-room rules that match over them, the detection a match emits, and the coverage honesty that says a class the agent could not observe is a GAP, never a clean host.

It has no I/O, no clock, and no eBPF: the kernel programs, their loaders and the agent-side engine live in internal/infrastructure/ebpf and internal/usecase/fleet/detect, which depend on this domain. A rule Matcher is TYPED DATA evaluated by Go — never a shell expression or an eval'd string (golden rule 1): the engine observes and matches, it never executes anything.

Milestone one ships DETECTIONS, not raw events. This package models the detection and the bounded evidence window that travels with it; the raw event stream stays on the host (issue #424).

Index

Constants

View Source
const MaxEvidence = 64

MaxEvidence bounds the context window that travels with a detection. Milestone one ships the detection plus a bounded window of surrounding events, never the raw stream — this cap is what keeps a detection small enough for Postgres to remain the system of record (the columnar tier for full telemetry is #424). Evidence beyond the cap is dropped from the shipped detection, not from the on-host ring buffer.

View Source
const MaxSequenceSteps = 8

MaxSequenceSteps bounds a sequence rule's length. Per-group state holds at most this many events, so a rule cannot make the evaluator allocate an unbounded partial match.

View Source
const MaxWindowCount = 10000

MaxWindowCount caps a window's count; a detection keeps the last MaxEvidence events of a larger burst and marks the evidence truncated.

View Source
const MaxWindowGroups = 1024

MaxWindowGroups bounds the distinct groups an Evaluator tracks per rule; beyond it the stalest group is dropped. A sensor on a busy host must not let an attacker grow the evaluator without bound by varying the grouped field. Memory per rule is bounded by MaxWindowGroups groups, each holding at most MaxWindowCount timestamps and MaxEvidence events.

Variables

This section is empty.

Functions

This section is empty.

Types

type Class

type Class string

Class is a category of host activity the engine observes. Each class is backed by its own eBPF program with its own bounded map and can be loaded, disabled and fail INDEPENDENTLY — a missing kernel feature takes out one class with a coverage report, never the whole engine (#422 requirement 6).

The set mirrors the telemetry classes the emulation catalogue (#421) expects a detection for, so the purple ledger (#426) can reconcile "what we executed" against "what we detected" by detection id.

const (
	ClassProcess   Class = "process"   // process execution (exec/fork)
	ClassNetwork   Class = "network"   // outbound/inbound connect
	ClassFile      Class = "file"      // file access to sensitive paths
	ClassPrivilege Class = "privilege" // privilege / capability change
)

func Classes

func Classes() []Class

Classes returns every event class in a stable order, so a per-class coverage report and its trend are comparable across runs and across hosts.

func (Class) Valid

func (c Class) Valid() bool

Valid reports whether c is a known event class.

type ClassCoverage

type ClassCoverage struct {
	Class   Class
	HostID  shared.ID
	AgentID shared.ID
	State   ClassState
	Reason  string // required when the state is not active/disabled: WHY the class is not observing
	Since   time.Time
}

ClassCoverage is the observation status of one event class on one host, over a window. It is the input to the coverage report the purple ledger (#426) and the detections-as-evidence issue (#423) consume.

func (ClassCoverage) IsObservationGap

func (c ClassCoverage) IsObservationGap() bool

IsObservationGap reports whether this class is an observation gap — a window during which the host was NOT observing this class. StateDisabled is a gap too: a class switched off by config still means the host is not covered for it, and hiding that would let a config change quietly erase coverage.

func (ClassCoverage) Observing

func (c ClassCoverage) Observing() bool

Observing reports whether this class is actually watching. Only StateActive observes; a caller must never treat any other state as "no activity".

func (ClassCoverage) Validate

func (c ClassCoverage) Validate() error

Validate ensures a coverage record is meaningful. A non-active, non-disabled state MUST carry a reason, so a gap can always be explained to an operator rather than appearing as an unexplained hole.

type ClassState

type ClassState string

ClassState is whether a host is actually observing a given event class right now.

The whole point of this type is honesty about the negative case. A host that reports NO detections for a class is ambiguous: either nothing happened, or the sensor was not watching. StateActive is the only state under which "no detections" means "nothing detected". Every other state is an OBSERVATION GAP — the host is not clean for that class, it is unobserved, and a coverage number must say so (#422 requirements 4, 5 and 6).

const (
	StateActive   ClassState = "active"   // program loaded, map bound, events flowing
	StateDegraded ClassState = "degraded" // a required kernel feature is missing; this class is disabled
	StateFailed   ClassState = "failed"   // the program failed to load or attach on a supported kernel
	StateDisabled ClassState = "disabled" // switched off by configuration (SYNAPSE_DETECT_CLASSES)
)

type Detection

type Detection struct {
	RuleID      string
	RuleVersion int
	Class       Class
	Severity    shared.Severity
	HostID      shared.ID
	AgentID     shared.ID
	Evidence    []Event // bounded (MaxEvidence); the triggering event plus surrounding context
	// Truncated records that the evidence window was capped: the shipped Evidence is not the complete
	// sequence. It is a field, not just a constructor return, so the incompleteness travels with the
	// stored/hash-chained record (#423) and a reconstructed detection cannot present a bounded window as
	// the whole story. Observed count is the number of events seen before truncation (>= len(Evidence)).
	Truncated     bool
	ObservedCount int
	Observed      time.Time
}

Detection is what a matched rule emits. It carries full attribution — the host, the agent identity, the rule id and version, the matched evidence and the time — because the next issue (#423) turns these into hash-chained, attributable evidence, and evidence with a hole in its provenance is not evidence.

func NewBurstDetection added in v0.2.0

func NewBurstDetection(r Rule, host, agent shared.ID, evidence []Event, observed int, at time.Time) (Detection, error)

NewBurstDetection is NewDetection for a windowed rule whose burst was longer than the evidence the evaluator kept: observed is the number of matching events in the burst, evidence the most recent of them. When observed exceeds the evidence, the detection is marked truncated with that count, so a 120-packet burst that ships 64 packets never presents as a 64-packet one.

func NewDetection

func NewDetection(r Rule, host, agent shared.ID, evidence []Event, at time.Time) (Detection, error)

NewDetection builds a detection from a rule and the observed evidence, deep-copying and bounding the evidence defensively so a caller mutating its slice OR the payloads it points at afterwards cannot alter the sealed record. The rule must validate and there must be at least one evidence event — a detection with no evidence is a claim, not a detection.

If more than MaxEvidence events are supplied, the MOST RECENT MaxEvidence are kept (the tail), because the triggering event and its immediate lead-up are the useful context; the drop is recorded in the detection's Truncated/ObservedCount fields so a caller never presents a bounded window as complete.

func (Detection) Validate

func (d Detection) Validate() error

Validate re-checks a detection's attribution, so a detection reconstructed from storage (rather than built through NewDetection) cannot present without a full provenance.

type Evaluator added in v0.2.0

type Evaluator struct {
	// contains filtered or unexported fields
}

Evaluator applies a rule set to a stream of events, keeping the per-group state windowed rules need. Plain rules pass straight through Rule.Match. It is not safe for concurrent use; the caller serialises events, which a sensor stream already does.

func NewEvaluator added in v0.2.0

func NewEvaluator(rules []Rule) (*Evaluator, error)

NewEvaluator validates the rules and prepares state for the windowed and sequence ones.

func (*Evaluator) Evaluate added in v0.2.0

func (ev *Evaluator) Evaluate(e Event) []Fired

Evaluate feeds one event and returns the rules that fire on it, in rule order.

type Event

type Event struct {
	Class Class
	At    time.Time
	Host  shared.ID

	Process   *ProcessEvent
	Network   *NetworkEvent
	File      *FileEvent
	Privilege *PrivilegeEvent
}

Event is one observed host activity, typed by Class. Exactly one of the per-class payloads is set — the one matching Class. The payloads are deliberately small and bounded: milestone one ships a detection plus a bounded window of these, never the raw stream.

Fields are TYPED, not a free-form map, so a rule matches over a closed, known set (see Field). That is what keeps a rule from being an arbitrary expression over untrusted keys.

func (Event) Validate

func (e Event) Validate() error

Validate enforces that an event is well-formed: a known class, a non-zero timestamp, and exactly the one payload its class names.

type Field

type Field string

Field is a closed enumeration of the event fields a rule may match over. It is deliberately NOT a free-form string key: a rule matches over this known set only, which is what stops a rule from being an arbitrary expression over untrusted input. Each field belongs to exactly one Class.

const (
	// process
	FieldProcComm Field = "proc.comm"
	FieldProcPath Field = "proc.path"
	FieldProcArg  Field = "proc.arg" // repeated: matches if ANY argv element satisfies the predicate
	FieldProcUID  Field = "proc.uid" // numeric

	// network
	FieldNetProto      Field = "net.proto"
	FieldNetRemoteAddr Field = "net.remote_addr"
	FieldNetRemotePort Field = "net.remote_port" // numeric
	FieldNetDirection  Field = "net.direction"

	// file
	FieldFilePath Field = "file.path"
	FieldFileOp   Field = "file.op"
	FieldFileComm Field = "file.comm"

	// privilege
	FieldPrivComm  Field = "priv.comm"
	FieldPrivCap   Field = "priv.cap"
	FieldPrivKind  Field = "priv.kind"
	FieldPrivToUID Field = "priv.to_uid" // numeric
)

func (Field) Numeric added in v0.2.0

func (f Field) Numeric() bool

Numeric reports whether f is compared through the matcher's integer semantics.

func (Field) Valid added in v0.2.0

func (f Field) Valid() bool

Valid reports whether f is one of the closed matcher fields understood by this build.

type FileEvent

type FileEvent struct {
	Path string
	Op   string // read | write | open | unlink
	PID  int
	Comm string
}

FileEvent is a sensitive-path access observation.

type Fired added in v0.2.0

type Fired struct {
	Rule     Rule
	Evidence []Event
	// Observed is the number of matching events in the burst that fired (>= len(Evidence)); zero for a
	// plain rule. Pass it to NewBurstDetection so a burst longer than the kept evidence is marked truncated.
	Observed int
}

Fired is one rule that produced a detection for the evaluated event. For a windowed rule Evidence is the burst that crossed the threshold, oldest first; for a plain rule it is nil and the caller supplies whatever context it keeps.

type HostCoverage

type HostCoverage struct {
	HostID  shared.ID
	Classes []ClassCoverage // exactly one per Class(), in Classes() order
}

HostCoverage is the per-host roll-up across every event class. Building it from ClassCoverage records makes the honest default structural: a class with no record at all is reported as an unknown gap, not silently omitted — you cannot accidentally drop a class and make a host look more covered than it is.

func NewHostCoverage

func NewHostCoverage(host shared.ID, reported []ClassCoverage) (HostCoverage, error)

NewHostCoverage assembles a host roll-up from whatever class records were reported. Any class in Classes() with no reported record is filled in as StateFailed with an explicit "no report" reason — the absence of a report is itself an observation gap, never treated as clean.

func (HostCoverage) FullyObserved

func (h HostCoverage) FullyObserved() bool

FullyObserved reports whether every event class is actively observed on this host.

func (HostCoverage) Gaps

func (h HostCoverage) Gaps() []ClassCoverage

Gaps returns the classes this host is NOT observing. An empty result means every class is actively observed; anything else is the honest list of where "no detections" cannot be trusted.

type Incident

type Incident struct {
	Key          string // stable dedup key: rule + asset
	RuleID       string
	AssetID      shared.ID
	Severity     shared.Severity // the most severe among the folded detections
	Count        int
	First        time.Time
	Last         time.Time
	DetectionIDs []shared.ID // the underlying records, preserved
}

Incident is an incident-level rollup of repeated detections of the same rule on the same asset. It is a VIEW: it never replaces the underlying records, which remain the ledger. DetectionIDs lists every record folded in, so an auditor can always descend from the incident to the individual, attributable, hash-chained detections beneath it.

func Rollup

func Rollup(records []Record) []Incident

Rollup folds records into incidents by (rule, asset), preserving every underlying record id. The result is deterministically ordered (by key) so an incident view and its trend compare across runs. It is a pure projection — it neither drops nor mutates the records it summarises.

type Matcher

type Matcher struct {
	Class Class
	All   []Predicate
}

Matcher is the AND of its predicates over one event class. A rule needing OR is expressed as two rules — milestone one keeps the matcher language small and total on purpose. An empty predicate list is refused: a matcher that matches every event of a class is not a detection rule, it is a firehose.

func (Matcher) Match

func (m Matcher) Match(e Event) bool

Match reports whether the event satisfies every predicate. A cross-class event, or one missing the field a predicate names, does not match — a rule never matches something it cannot see.

type NetworkEvent

type NetworkEvent struct {
	Proto      string // tcp | udp
	RemoteAddr string
	RemotePort int
	Direction  string // egress | ingress
	PID        int
	Comm       string
}

NetworkEvent is a connect observation.

type Op

type Op string

Op is a comparison operator. The set is small and total: every op has a defined meaning for the field type it is used with, checked by Matcher.validate.

const (
	OpEquals   Op = "eq"       // string or numeric equality
	OpPrefix   Op = "prefix"   // string has prefix
	OpContains Op = "contains" // string contains
	OpIn       Op = "in"       // string is one of Values
	OpGTE      Op = "gte"      // numeric >=
)

type Predicate

type Predicate struct {
	Field  Field
	Op     Op
	Value  string
	Values []string
}

Predicate is one field comparison. Value is used by every op except OpIn, which uses Values.

type PrivilegeEvent

type PrivilegeEvent struct {
	PID     int
	Comm    string
	FromUID int
	ToUID   int
	Cap     string // capability gained, if any
	Kind    string // setuid | setgid | capset
}

PrivilegeEvent is a privilege/capability change observation.

type ProcessEvent

type ProcessEvent struct {
	Kind                 string // exec | fork | exit; empty means legacy exec
	PID                  int
	PPID                 int
	StartTimeNanos       uint64
	ParentStartTimeNanos uint64
	Comm                 string   // short command name (kernel comm)
	Path                 string   // resolved executable path
	Args                 []string // bounded argv
	UID                  int
}

ProcessEvent is an exec/fork observation.

type Record

type Record struct {
	ID           shared.ID
	TenantID     shared.ID
	EngagementID shared.ID
	AssetID      shared.ID // the asset the detection was observed on, so it joins the asset risk story
	AgentID      shared.ID
	Detection    Detection
	EvidenceID   shared.ID // the evidence-chain link that sealed this detection
	BatchSeq     uint64    // the agent batch sequence it arrived in
	RecordedAt   time.Time
	ExpiresAt    time.Time // zero = never expires; a set value is enforced by audited retention
}

Record is a detection as it lives in the control-plane ledger (#423): the detection itself, bound to the evidence-chain link that sealed it, the asset it was observed on, the agent that produced it, and the agent batch sequence it arrived in. The evidence-chain link is the tamper-evident ledger; this Record is the queryable projection over it (and the one subject to retention — the chain link is permanent, this row is not).

func (Record) Expired

func (r Record) Expired(now time.Time) bool

Expired reports whether the record's retention window has elapsed at now. A zero ExpiresAt never expires. Expiry removes the projection row (an audited action); the sealed chain link remains.

func (Record) Validate

func (r Record) Validate() error

Validate enforces the binding invariants: a record with no evidence link, asset, tenant, or a malformed detection is not a chained, attributable record and must not be stored.

type Rule

type Rule struct {
	ID       string // stable, catalogued; matches an emulation ExpectedObservable.DetectionID
	Version  int
	Class    Class
	Title    string
	Severity shared.Severity
	Matcher  Matcher
	// Window, when set, makes the rule fire on a rate of matching events rather than on each one. Nil is
	// the plain per-event rule.
	Window *Window
	// Sequence, when set, makes the rule fire on an ordered series of matching events rather than on one.
	// It is mutually exclusive with Window, and it replaces the top-level Matcher (each step carries its
	// own), so a sequence rule leaves Matcher empty.
	Sequence *Sequence
}

Rule is a typed, versioned, clean-room detection definition. It matches over one event class and, on a match, the engine emits a Detection carrying the matched evidence. Rules are catalogued exactly like the SAST catalogue, with a drift test that fails the build if the shipped catalogue does not validate.

ID is stable and public-facing: it is the detection id the emulation catalogue (#421) names as the expected observable, so the purple ledger (#426) can reconcile executed techniques against detections by this id. Changing an ID breaks that linkage, so IDs are append-only in practice.

func Catalogue

func Catalogue() ([]Rule, error)

Catalogue returns a validated copy of the built-in rule set, deterministically ordered by id so a coverage report and its trend compare like with like. It validates on every call rather than trusting the literal above, so a malformed rule fails the build via the catalogue drift test rather than shipping a rule that cannot produce a trustworthy detection.

func CatalogueByClass

func CatalogueByClass(c Class) ([]Rule, error)

CatalogueByClass returns the catalogued rules for one event class, deterministically ordered. The agent-side engine (issue #422, later phase) evaluates only the rules for the classes it can actually observe on a given host, so a disabled or degraded class runs no rules and reports a coverage gap.

func Lookup

func Lookup(id string) (Rule, bool)

Lookup returns a catalogued rule by id. The second result is false for an unknown id, which the caller MUST treat as "not a known detection" rather than fabricating one.

func (Rule) Match

func (r Rule) Match(e Event) bool

Match reports whether an event satisfies this rule's predicates. It observes and matches only — it never executes anything (golden rule 1); the Matcher is typed data, not a shell expression. For a windowed rule a match is one event that counts toward the burst, not yet a detection; Evaluator decides when the count is reached.

func (Rule) Sequenced added in v0.2.0

func (r Rule) Sequenced() bool

Sequenced reports whether the rule fires on an ordered series of events rather than one.

func (Rule) Validate

func (r Rule) Validate() error

Validate enforces the invariants a catalogued rule must hold. A rule that fails these cannot produce a trustworthy detection and must not ship.

func (Rule) Windowed added in v0.2.0

func (r Rule) Windowed() bool

Windowed reports whether the rule fires on a rate rather than on every matching event.

type Sequence added in v0.2.0

type Sequence struct {
	// Steps are the ordered matchers. An event advances the sequence only when it matches the next
	// unsatisfied step. At least two steps: a one-step sequence is a plain rule.
	Steps []Matcher
	// Within is the span from the FIRST matched step's event to the last, exclusive. A partial match that
	// lapses is reset, so a slow, unrelated recurrence is never stitched into a sequence.
	Within time.Duration
	// GroupBy partitions progress by these fields so two hosts' (or two entities') unrelated events do
	// not interleave into one sequence. Empty groups by host only. Fields must belong to the rule's class.
	GroupBy []Field
}

Sequence turns a rule from "one matching event is a detection" into "these matchers, satisfied by events of the rule's class IN THIS ORDER, for the same group, all within this span, is a detection". It expresses an ordered behaviour that no single event and no rate shows: stage a tool, then use it; probe identity, then attempt escalation. The steps match over the rule's own class; a cross-class sequence is out of scope for the per-class engine, which sees one class at a time.

type Window added in v0.2.0

type Window struct {
	// Count is the number of matching events that must fall inside the span (>= 2; a count of one is a
	// plain rule).
	Count int
	// Within is the sliding span measured on event timestamps, exclusive: events exactly Within apart
	// are not in one burst.
	Within time.Duration
	// GroupBy partitions the count by the values of these fields, so ten queries to ten resolvers do not
	// add up to a burst against one. Empty groups by host only. Fields must belong to the rule's class.
	GroupBy []Field
}

Window turns a rule from "every matching event is a detection" into "this many matching events within this span, for the same group, is a detection". It is what separates one DNS packet from a burst of them to one destination. The predicates still decide which events count; the window decides when the count is a finding.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL