Documentation
¶
Overview ¶
Package detection is the pure domain for the agent-side blue-team detection engine (issue #422): the typed event classes an eBPF sensor observes, the clean-room rules that match over them, the detection a match emits, and the coverage honesty that says a class the agent could not observe is a GAP, never a clean host.
It has no I/O, no clock, and no eBPF: the kernel programs, their loaders and the agent-side engine live in internal/infrastructure/ebpf and internal/usecase/fleet/detect, which depend on this domain. A rule Matcher is TYPED DATA evaluated by Go — never a shell expression or an eval'd string (golden rule 1): the engine observes and matches, it never executes anything.
Milestone one ships DETECTIONS, not raw events. This package models the detection and the bounded evidence window that travels with it; the raw event stream stays on the host (issue #424).
Index ¶
- Constants
- type Class
- type ClassCoverage
- type ClassState
- type Detection
- type Evaluator
- type Event
- type Field
- type FileEvent
- type Fired
- type HostCoverage
- type Incident
- type Matcher
- type NetworkEvent
- type Op
- type Predicate
- type PrivilegeEvent
- type ProcessEvent
- type Record
- type Rule
- type Sequence
- type Window
Constants ¶
const MaxEvidence = 64
MaxEvidence bounds the context window that travels with a detection. Milestone one ships the detection plus a bounded window of surrounding events, never the raw stream — this cap is what keeps a detection small enough for Postgres to remain the system of record (the columnar tier for full telemetry is #424). Evidence beyond the cap is dropped from the shipped detection, not from the on-host ring buffer.
const MaxSequenceSteps = 8
MaxSequenceSteps bounds a sequence rule's length. Per-group state holds at most this many events, so a rule cannot make the evaluator allocate an unbounded partial match.
const MaxWindowCount = 10000
MaxWindowCount caps a window's count; a detection keeps the last MaxEvidence events of a larger burst and marks the evidence truncated.
const MaxWindowGroups = 1024
MaxWindowGroups bounds the distinct groups an Evaluator tracks per rule; beyond it the stalest group is dropped. A sensor on a busy host must not let an attacker grow the evaluator without bound by varying the grouped field. Memory per rule is bounded by MaxWindowGroups groups, each holding at most MaxWindowCount timestamps and MaxEvidence events.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Class ¶
type Class string
Class is a category of host activity the engine observes. Each class is backed by its own eBPF program with its own bounded map and can be loaded, disabled and fail INDEPENDENTLY — a missing kernel feature takes out one class with a coverage report, never the whole engine (#422 requirement 6).
The set mirrors the telemetry classes the emulation catalogue (#421) expects a detection for, so the purple ledger (#426) can reconcile "what we executed" against "what we detected" by detection id.
type ClassCoverage ¶
type ClassCoverage struct {
Class Class
HostID shared.ID
AgentID shared.ID
State ClassState
Reason string // required when the state is not active/disabled: WHY the class is not observing
Since time.Time
}
ClassCoverage is the observation status of one event class on one host, over a window. It is the input to the coverage report the purple ledger (#426) and the detections-as-evidence issue (#423) consume.
func (ClassCoverage) IsObservationGap ¶
func (c ClassCoverage) IsObservationGap() bool
IsObservationGap reports whether this class is an observation gap — a window during which the host was NOT observing this class. StateDisabled is a gap too: a class switched off by config still means the host is not covered for it, and hiding that would let a config change quietly erase coverage.
func (ClassCoverage) Observing ¶
func (c ClassCoverage) Observing() bool
Observing reports whether this class is actually watching. Only StateActive observes; a caller must never treat any other state as "no activity".
func (ClassCoverage) Validate ¶
func (c ClassCoverage) Validate() error
Validate ensures a coverage record is meaningful. A non-active, non-disabled state MUST carry a reason, so a gap can always be explained to an operator rather than appearing as an unexplained hole.
type ClassState ¶
type ClassState string
ClassState is whether a host is actually observing a given event class right now.
The whole point of this type is honesty about the negative case. A host that reports NO detections for a class is ambiguous: either nothing happened, or the sensor was not watching. StateActive is the only state under which "no detections" means "nothing detected". Every other state is an OBSERVATION GAP — the host is not clean for that class, it is unobserved, and a coverage number must say so (#422 requirements 4, 5 and 6).
const ( StateActive ClassState = "active" // program loaded, map bound, events flowing StateDegraded ClassState = "degraded" // a required kernel feature is missing; this class is disabled StateFailed ClassState = "failed" // the program failed to load or attach on a supported kernel StateDisabled ClassState = "disabled" // switched off by configuration (SYNAPSE_DETECT_CLASSES) )
type Detection ¶
type Detection struct {
RuleID string
RuleVersion int
Class Class
Severity shared.Severity
HostID shared.ID
AgentID shared.ID
Evidence []Event // bounded (MaxEvidence); the triggering event plus surrounding context
// Truncated records that the evidence window was capped: the shipped Evidence is not the complete
// sequence. It is a field, not just a constructor return, so the incompleteness travels with the
// stored/hash-chained record (#423) and a reconstructed detection cannot present a bounded window as
// the whole story. Observed count is the number of events seen before truncation (>= len(Evidence)).
Truncated bool
ObservedCount int
Observed time.Time
}
Detection is what a matched rule emits. It carries full attribution — the host, the agent identity, the rule id and version, the matched evidence and the time — because the next issue (#423) turns these into hash-chained, attributable evidence, and evidence with a hole in its provenance is not evidence.
func NewBurstDetection ¶ added in v0.2.0
func NewBurstDetection(r Rule, host, agent shared.ID, evidence []Event, observed int, at time.Time) (Detection, error)
NewBurstDetection is NewDetection for a windowed rule whose burst was longer than the evidence the evaluator kept: observed is the number of matching events in the burst, evidence the most recent of them. When observed exceeds the evidence, the detection is marked truncated with that count, so a 120-packet burst that ships 64 packets never presents as a 64-packet one.
func NewDetection ¶
NewDetection builds a detection from a rule and the observed evidence, deep-copying and bounding the evidence defensively so a caller mutating its slice OR the payloads it points at afterwards cannot alter the sealed record. The rule must validate and there must be at least one evidence event — a detection with no evidence is a claim, not a detection.
If more than MaxEvidence events are supplied, the MOST RECENT MaxEvidence are kept (the tail), because the triggering event and its immediate lead-up are the useful context; the drop is recorded in the detection's Truncated/ObservedCount fields so a caller never presents a bounded window as complete.
type Evaluator ¶ added in v0.2.0
type Evaluator struct {
// contains filtered or unexported fields
}
Evaluator applies a rule set to a stream of events, keeping the per-group state windowed rules need. Plain rules pass straight through Rule.Match. It is not safe for concurrent use; the caller serialises events, which a sensor stream already does.
func NewEvaluator ¶ added in v0.2.0
NewEvaluator validates the rules and prepares state for the windowed and sequence ones.
type Event ¶
type Event struct {
Class Class
At time.Time
Host shared.ID
Process *ProcessEvent
Network *NetworkEvent
File *FileEvent
Privilege *PrivilegeEvent
}
Event is one observed host activity, typed by Class. Exactly one of the per-class payloads is set — the one matching Class. The payloads are deliberately small and bounded: milestone one ships a detection plus a bounded window of these, never the raw stream.
Fields are TYPED, not a free-form map, so a rule matches over a closed, known set (see Field). That is what keeps a rule from being an arbitrary expression over untrusted keys.
type Field ¶
type Field string
Field is a closed enumeration of the event fields a rule may match over. It is deliberately NOT a free-form string key: a rule matches over this known set only, which is what stops a rule from being an arbitrary expression over untrusted input. Each field belongs to exactly one Class.
const ( // process FieldProcComm Field = "proc.comm" FieldProcPath Field = "proc.path" FieldProcArg Field = "proc.arg" // repeated: matches if ANY argv element satisfies the predicate FieldProcUID Field = "proc.uid" // numeric // network FieldNetProto Field = "net.proto" FieldNetRemoteAddr Field = "net.remote_addr" FieldNetRemotePort Field = "net.remote_port" // numeric FieldNetDirection Field = "net.direction" // file FieldFilePath Field = "file.path" FieldFileOp Field = "file.op" FieldFileComm Field = "file.comm" // privilege FieldPrivComm Field = "priv.comm" FieldPrivCap Field = "priv.cap" FieldPrivKind Field = "priv.kind" FieldPrivToUID Field = "priv.to_uid" // numeric )
type Fired ¶ added in v0.2.0
type Fired struct {
Rule Rule
Evidence []Event
// Observed is the number of matching events in the burst that fired (>= len(Evidence)); zero for a
// plain rule. Pass it to NewBurstDetection so a burst longer than the kept evidence is marked truncated.
Observed int
}
Fired is one rule that produced a detection for the evaluated event. For a windowed rule Evidence is the burst that crossed the threshold, oldest first; for a plain rule it is nil and the caller supplies whatever context it keeps.
type HostCoverage ¶
type HostCoverage struct {
HostID shared.ID
Classes []ClassCoverage // exactly one per Class(), in Classes() order
}
HostCoverage is the per-host roll-up across every event class. Building it from ClassCoverage records makes the honest default structural: a class with no record at all is reported as an unknown gap, not silently omitted — you cannot accidentally drop a class and make a host look more covered than it is.
func NewHostCoverage ¶
func NewHostCoverage(host shared.ID, reported []ClassCoverage) (HostCoverage, error)
NewHostCoverage assembles a host roll-up from whatever class records were reported. Any class in Classes() with no reported record is filled in as StateFailed with an explicit "no report" reason — the absence of a report is itself an observation gap, never treated as clean.
func (HostCoverage) FullyObserved ¶
func (h HostCoverage) FullyObserved() bool
FullyObserved reports whether every event class is actively observed on this host.
func (HostCoverage) Gaps ¶
func (h HostCoverage) Gaps() []ClassCoverage
Gaps returns the classes this host is NOT observing. An empty result means every class is actively observed; anything else is the honest list of where "no detections" cannot be trusted.
type Incident ¶
type Incident struct {
Key string // stable dedup key: rule + asset
RuleID string
AssetID shared.ID
Severity shared.Severity // the most severe among the folded detections
Count int
First time.Time
Last time.Time
DetectionIDs []shared.ID // the underlying records, preserved
}
Incident is an incident-level rollup of repeated detections of the same rule on the same asset. It is a VIEW: it never replaces the underlying records, which remain the ledger. DetectionIDs lists every record folded in, so an auditor can always descend from the incident to the individual, attributable, hash-chained detections beneath it.
type Matcher ¶
Matcher is the AND of its predicates over one event class. A rule needing OR is expressed as two rules — milestone one keeps the matcher language small and total on purpose. An empty predicate list is refused: a matcher that matches every event of a class is not a detection rule, it is a firehose.
type NetworkEvent ¶
type NetworkEvent struct {
Proto string // tcp | udp
RemoteAddr string
RemotePort int
Direction string // egress | ingress
PID int
Comm string
}
NetworkEvent is a connect observation.
type Op ¶
type Op string
Op is a comparison operator. The set is small and total: every op has a defined meaning for the field type it is used with, checked by Matcher.validate.
type Predicate ¶
Predicate is one field comparison. Value is used by every op except OpIn, which uses Values.
type PrivilegeEvent ¶
type PrivilegeEvent struct {
PID int
Comm string
FromUID int
ToUID int
Cap string // capability gained, if any
Kind string // setuid | setgid | capset
}
PrivilegeEvent is a privilege/capability change observation.
type ProcessEvent ¶
type ProcessEvent struct {
Kind string // exec | fork | exit; empty means legacy exec
PID int
PPID int
StartTimeNanos uint64
ParentStartTimeNanos uint64
Comm string // short command name (kernel comm)
Path string // resolved executable path
Args []string // bounded argv
UID int
}
ProcessEvent is an exec/fork observation.
type Record ¶
type Record struct {
ID shared.ID
TenantID shared.ID
EngagementID shared.ID
AssetID shared.ID // the asset the detection was observed on, so it joins the asset risk story
AgentID shared.ID
Detection Detection
EvidenceID shared.ID // the evidence-chain link that sealed this detection
BatchSeq uint64 // the agent batch sequence it arrived in
RecordedAt time.Time
ExpiresAt time.Time // zero = never expires; a set value is enforced by audited retention
}
Record is a detection as it lives in the control-plane ledger (#423): the detection itself, bound to the evidence-chain link that sealed it, the asset it was observed on, the agent that produced it, and the agent batch sequence it arrived in. The evidence-chain link is the tamper-evident ledger; this Record is the queryable projection over it (and the one subject to retention — the chain link is permanent, this row is not).
type Rule ¶
type Rule struct {
ID string // stable, catalogued; matches an emulation ExpectedObservable.DetectionID
Version int
Class Class
Title string
Severity shared.Severity
Matcher Matcher
// Window, when set, makes the rule fire on a rate of matching events rather than on each one. Nil is
// the plain per-event rule.
Window *Window
// Sequence, when set, makes the rule fire on an ordered series of matching events rather than on one.
// It is mutually exclusive with Window, and it replaces the top-level Matcher (each step carries its
// own), so a sequence rule leaves Matcher empty.
Sequence *Sequence
}
Rule is a typed, versioned, clean-room detection definition. It matches over one event class and, on a match, the engine emits a Detection carrying the matched evidence. Rules are catalogued exactly like the SAST catalogue, with a drift test that fails the build if the shipped catalogue does not validate.
ID is stable and public-facing: it is the detection id the emulation catalogue (#421) names as the expected observable, so the purple ledger (#426) can reconcile executed techniques against detections by this id. Changing an ID breaks that linkage, so IDs are append-only in practice.
func Catalogue ¶
Catalogue returns a validated copy of the built-in rule set, deterministically ordered by id so a coverage report and its trend compare like with like. It validates on every call rather than trusting the literal above, so a malformed rule fails the build via the catalogue drift test rather than shipping a rule that cannot produce a trustworthy detection.
func CatalogueByClass ¶
CatalogueByClass returns the catalogued rules for one event class, deterministically ordered. The agent-side engine (issue #422, later phase) evaluates only the rules for the classes it can actually observe on a given host, so a disabled or degraded class runs no rules and reports a coverage gap.
func Lookup ¶
Lookup returns a catalogued rule by id. The second result is false for an unknown id, which the caller MUST treat as "not a known detection" rather than fabricating one.
func (Rule) Match ¶
Match reports whether an event satisfies this rule's predicates. It observes and matches only — it never executes anything (golden rule 1); the Matcher is typed data, not a shell expression. For a windowed rule a match is one event that counts toward the burst, not yet a detection; Evaluator decides when the count is reached.
func (Rule) Sequenced ¶ added in v0.2.0
Sequenced reports whether the rule fires on an ordered series of events rather than one.
type Sequence ¶ added in v0.2.0
type Sequence struct {
// Steps are the ordered matchers. An event advances the sequence only when it matches the next
// unsatisfied step. At least two steps: a one-step sequence is a plain rule.
Steps []Matcher
// Within is the span from the FIRST matched step's event to the last, exclusive. A partial match that
// lapses is reset, so a slow, unrelated recurrence is never stitched into a sequence.
Within time.Duration
// GroupBy partitions progress by these fields so two hosts' (or two entities') unrelated events do
// not interleave into one sequence. Empty groups by host only. Fields must belong to the rule's class.
GroupBy []Field
}
Sequence turns a rule from "one matching event is a detection" into "these matchers, satisfied by events of the rule's class IN THIS ORDER, for the same group, all within this span, is a detection". It expresses an ordered behaviour that no single event and no rate shows: stage a tool, then use it; probe identity, then attempt escalation. The steps match over the rule's own class; a cross-class sequence is out of scope for the per-class engine, which sees one class at a time.
type Window ¶ added in v0.2.0
type Window struct {
// Count is the number of matching events that must fall inside the span (>= 2; a count of one is a
// plain rule).
Count int
// Within is the sliding span measured on event timestamps, exclusive: events exactly Within apart
// are not in one burst.
Within time.Duration
// GroupBy partitions the count by the values of these fields, so ten queries to ten resolvers do not
// add up to a burst against one. Empty groups by host only. Fields must belong to the rule's class.
GroupBy []Field
}
Window turns a rule from "every matching event is a detection" into "this many matching events within this span, for the same group, is a detection". It is what separates one DNS packet from a burst of them to one destination. The predicates still decide which events count; the window decides when the count is a finding.