Documentation
¶
Overview ¶
Package routex is an embeddable library of Enterprise Integration Patterns in the style of Apache Camel, for in-process use in Go programs.
A route reads exchanges from a consumer endpoint, runs them through steps and usually sends them to a producer endpoint:
e := routex.New()
_ = e.AddRoute(routex.From("timer:tick?period=1s").Then(
routex.Filter(func(t time.Time) bool { return t.Second()%2 == 0 }),
routex.To("log:even"),
))
err := e.Run(ctx) // blocks until ctx is cancelled, then drains gracefully
Model ¶
An Exchange carries an ID, headers and a body of any type. Typed steps such as Transform, Filter and Enrich use generics to read the body as a concrete type and report a wrapped ErrBodyType on mismatch.
Patterns ¶
Choice (content-based router), Filter, Transform (translator), Enrich, Split and SplitSeq, Aggregate, Multicast, RecipientList, WireTap, Idempotent (idempotent consumer), Retry, Throttle and route-level dead-letter channels via WithDeadLetter.
Components ¶
Endpoints are URIs. Built in: direct: (synchronous call), seda: (bounded asynchronous queue), timer:, log:, file:, http:/https: (producer) and httpd: (server consumer). Add your own with Engine.Register; message brokers belong in separate modules or examples implementing Component.
Concurrency ¶
Engine, Route (after AddRoute), all Steps built by this package, and MemoryRepository are safe for concurrent use. An Exchange is not; patterns that fan out hand each branch its own copy. Cancelling the context passed to Engine.Run shuts everything down gracefully. The package has no third-party dependencies.
Index ¶
- Constants
- Variables
- func BodyAs[T any](ex *Exchange) (T, error)
- func Permanent(err error) error
- type AggregateOption
- type AggregateStrategy
- type Branch
- type Component
- type Consumer
- type Engine
- func (e *Engine) AddRoute(r *Route) error
- func (e *Engine) Register(scheme string, c Component)
- func (e *Engine) Request(ctx context.Context, uri string, body any) (*Exchange, error)
- func (e *Engine) Run(ctx context.Context) error
- func (e *Engine) Send(ctx context.Context, uri string, ex *Exchange) error
- func (e *Engine) Start(ctx context.Context) (wait func() error, err error)
- type Exchange
- func (e *Exchange) Copy() *Exchange
- func (e *Exchange) DeleteHeader(key string)
- func (e *Exchange) Err() error
- func (e *Exchange) Header(key string) (any, bool)
- func (e *Exchange) Headers() iter.Seq2[string, any]
- func (e *Exchange) SetHeader(key string, value any)
- func (e *Exchange) Stop()
- func (e *Exchange) Stopped() bool
- type ForkOption
- type HTTPStatusError
- type Handler
- type IdempotentRepository
- type MemoryRepository
- type Option
- type PanicError
- type Producer
- type RetryPolicy
- type Route
- type RouteOption
- type Step
- func Aggregate(key func(*Exchange) (string, error), strategy AggregateStrategy, ...) Step
- func Choice(branches ...Branch) Step
- func Enrich[In, Extra, Out any](fetch func(ctx context.Context, in In) (Extra, error), ...) Step
- func Filter[T any](keep func(T) bool) Step
- func Idempotent(key func(*Exchange) (string, error), repo IdempotentRepository, sub Step) Step
- func Multicast(branches []Step, opts ...ForkOption) Step
- func Pipeline(steps ...Step) Step
- func Process(fn func(ctx context.Context, ex *Exchange) error) Step
- func RecipientList(resolve func(*Exchange) ([]string, error), opts ...ForkOption) Step
- func Retry(policy RetryPolicy, sub Step) Step
- func SetHeader(key string, value any) Step
- func Split[T, E any](fn func(T) ([]E, error), sub Step, opts ...ForkOption) Step
- func SplitSeq[T, E any](fn func(T) iter.Seq[E], sub Step, opts ...ForkOption) Step
- func Stop() Step
- func Throttle(n int, per time.Duration) Step
- func To(uri string) Step
- func Transform[In, Out any](fn func(ctx context.Context, in In) (Out, error)) Step
- func WireTap(sub Step) Step
- type StepFunc
Examples ¶
Constants ¶
const ( // HeaderFileName is the base name of the consumed file, or the name to write when producing. HeaderFileName = "file.name" // HeaderFilePath is the full path of the consumed or written file. HeaderFilePath = "file.path" )
Headers set by the file component.
const ( // HeaderHTTPStatus is the response status code (http producer) or the status to answer with (httpd consumer, default 200). HeaderHTTPStatus = "http.status" // HeaderHTTPMethod is the request method (httpd consumer) or the method to use (http producer, default POST). HeaderHTTPMethod = "http.method" // HeaderHTTPPath is the request path (httpd consumer). HeaderHTTPPath = "http.path" // HeaderContentType is the Content-Type of the request or response body. HeaderContentType = "Content-Type" )
Headers set by the http components.
const ( // HeaderError holds the error text of an exchange delivered to a dead-letter step. HeaderError = "routex.error" // HeaderSplitIndex is the zero-based fragment index set by Split. HeaderSplitIndex = "routex.split.index" // HeaderSplitSize is the number of fragments produced by Split. HeaderSplitSize = "routex.split.size" // HeaderParentID is the ID of the exchange a fragment or branch was copied from. HeaderParentID = "routex.parent" // HeaderBranchIndex is the zero-based branch index set by Multicast and RecipientList. HeaderBranchIndex = "routex.branch.index" // HeaderAggregateKey is the correlation key of an aggregated exchange. HeaderAggregateKey = "routex.aggregate.key" // HeaderAggregateSize is the number of exchanges combined into an aggregate. HeaderAggregateSize = "routex.aggregate.size" )
Well-known header names set by routex patterns.
Variables ¶
var ( // ErrBodyType is returned when an exchange body does not have the type a step expects. ErrBodyType = errors.New("routex: unexpected body type") // ErrInvalidURI is returned for an endpoint URI that cannot be parsed or lacks a scheme. ErrInvalidURI = errors.New("routex: invalid endpoint uri") // ErrUnknownScheme is returned when no component is registered for a URI scheme. ErrUnknownScheme = errors.New("routex: no component registered for scheme") // ErrNoConsumer is returned when a synchronous producer (direct:) has no consumer listening. ErrNoConsumer = errors.New("routex: no consumer for endpoint") // ErrDuplicateRoute is returned when a route id is added twice. ErrDuplicateRoute = errors.New("routex: duplicate route id") )
Sentinel errors returned (wrapped) by routex. Test them with errors.Is.
Functions ¶
Types ¶
type AggregateOption ¶
type AggregateOption func(*aggregator)
AggregateOption configures Aggregate.
func CompleteOnStop ¶
func CompleteOnStop() AggregateOption
CompleteOnStop emits incomplete groups at shutdown instead of dropping them.
func CompleteSize ¶
func CompleteSize(n int) AggregateOption
CompleteSize completes a group when it has n exchanges.
func CompleteTimeout ¶
func CompleteTimeout(d time.Duration) AggregateOption
CompleteTimeout completes a group after d without a new exchange. The completed aggregate continues through the rest of the route from a background goroutine, so this only works under Engine.Run.
func CompleteWhen ¶
func CompleteWhen(pred func(acc *Exchange) bool) AggregateOption
CompleteWhen completes a group when pred(accumulator) returns true.
type AggregateStrategy ¶
AggregateStrategy folds an incoming exchange into the group accumulator. acc is nil for the first exchange of a group. It returns the new accumulator. The strategy runs under the aggregator's lock and must not block.
func AppendBodies ¶
func AppendBodies[T any]() AggregateStrategy
AppendBodies is a strategy collecting the bodies (each a T) of a group into a []T on the accumulator, in arrival order.
type Branch ¶
type Branch struct {
// contains filtered or unexported fields
}
Branch is one arm of a Choice.
type Component ¶
type Component interface {
Consumer(u *url.URL) (Consumer, error)
Producer(u *url.URL) (Producer, error)
}
Component creates consumers and producers for endpoint URIs of one scheme. A component that supports only one direction returns errors.ErrUnsupported for the other.
type Consumer ¶
Consumer receives messages from an endpoint and passes each to h. Consume blocks until ctx is cancelled (then returns nil after finishing in-flight work) or until the endpoint is exhausted; a non-nil error aborts the engine.
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine owns components and routes and runs them. The zero value is ready to use and has the built-in components registered: direct, seda, timer, log, file, http, https and httpd. All methods are safe for concurrent use.
func (*Engine) Request ¶
Request sends body to uri in a new exchange and returns the exchange after processing, so a direct: route's reply is in its Body.
Example ¶
package main
import (
"context"
"fmt"
"github.com/JiaBao-do/routex"
)
func main() {
e := routex.New()
_ = e.AddRoute(routex.From("direct:square").Then(
routex.Transform(func(_ context.Context, n int) (int, error) { return n * n, nil })))
ctx, cancel := context.WithCancel(context.Background())
wait, _ := e.Start(ctx)
reply, _ := e.Request(ctx, "direct:square", 7)
fmt.Println(reply.Body)
cancel()
_ = wait()
}
Output: 49
func (*Engine) Run ¶
Run starts every route and blocks until all consumers return, which for most endpoints means until ctx is cancelled. Shutdown is graceful: consumers finish the exchange in flight, seda queues are drained, wire taps complete and aggregates configured with CompleteOnStop are flushed. The first consumer error cancels the rest and is returned (joined with any others).
An exchange a consumer has already taken runs with a context that is NOT cancelled by shutdown, so in-flight work is never failed half way; put timeouts in your steps if a step can block forever.
func (*Engine) Send ¶
Send delivers ex to the endpoint uri. Synchronous endpoints (direct:, http:) may modify ex with a reply.
func (*Engine) Start ¶
Start is Run split in two: it creates every consumer, registers synchronous endpoints and launches the routes, then returns. Sends to direct: endpoints are safe as soon as Start returns. The returned wait function blocks until shutdown completes (as Run would) and returns Run's error; call it exactly once. Cancel ctx to stop.
type Exchange ¶
type Exchange struct {
// ID identifies the exchange. NewExchange sets a random one.
ID string
// Body is the payload. Use BodyAs for a typed, checked read.
Body any
// contains filtered or unexported fields
}
Exchange is the unit of work travelling through a route: an ID, a body of any type and string-keyed headers. Steps read and mutate the exchange in place.
An Exchange is NOT safe for concurrent use. Patterns that fan out (Split, Multicast, WireTap, ...) hand each branch its own Copy; the body value itself is shared, so mutable bodies (maps, pointers) must be treated as read-only or cloned by the caller.
func NewExchange ¶
NewExchange returns an exchange with a random ID and the given body.
func (*Exchange) Copy ¶
Copy returns a new exchange with the same ID, the same body value and a cloned header map. Error and stopped state are not copied.
func (*Exchange) DeleteHeader ¶
DeleteHeader removes a header.
func (*Exchange) Err ¶
Err returns the error that caused the exchange to be sent to a dead-letter step, if any.
type ForkOption ¶
type ForkOption func(*forkConfig)
ForkOption configures Split, Multicast and RecipientList.
func CollectBodies ¶
func CollectBodies[E any]() ForkOption
CollectBodies is a Join that replaces the original body with a []E holding each part's body in order. A part whose body is not an E is an error.
func ContinueOnError ¶
func ContinueOnError() ForkOption
ContinueOnError keeps processing the remaining fragments after one fails. Failed parts are excluded from the join and all errors are returned together at the end. The default is to stop at the first error.
func Join ¶
func Join(fn func(orig *Exchange, parts []*Exchange) error) ForkOption
Join combines the finished parts back into the original exchange. Parts are in original order; stopped and failed parts are omitted. Without Join the original exchange continues unchanged.
func Parallel ¶
func Parallel(n int) ForkOption
Parallel runs up to n fragments or branches concurrently (n <= 1 is sequential, the default). Parts share the body value of the original: with Parallel the sub-step must not mutate shared state.
type HTTPStatusError ¶
type HTTPStatusError struct {
// Status is the response status code.
Status int
// Body is the response body.
Body []byte
}
HTTPStatusError is returned by the http producer for a non-2xx response.
type Handler ¶
Handler processes one exchange. Consumers call a Handler for each message they receive.
type IdempotentRepository ¶
type IdempotentRepository interface {
// Add records key and reports whether it was new (true) or already present (false).
Add(ctx context.Context, key string) (bool, error)
// Remove forgets key so that a redelivery is processed again.
Remove(ctx context.Context, key string) error
}
IdempotentRepository remembers which message keys were already processed. Implementations must be safe for concurrent use.
type MemoryRepository ¶
type MemoryRepository struct {
// contains filtered or unexported fields
}
MemoryRepository is an in-memory IdempotentRepository that remembers at most the capacity most recently added keys (LRU eviction). It is safe for concurrent use. Keys are lost on restart; use a persistent repository if duplicates across restarts matter.
func NewMemoryRepository ¶
func NewMemoryRepository(capacity int) *MemoryRepository
NewMemoryRepository returns a repository remembering up to capacity keys (a capacity below 1 is treated as 1).
func (*MemoryRepository) Add ¶
Add implements IdempotentRepository. A repeated key is refreshed as most recent.
func (*MemoryRepository) Len ¶
func (m *MemoryRepository) Len() int
Len returns the number of remembered keys.
type Option ¶
type Option func(*Engine)
Option configures an Engine.
func WithLogger ¶
WithLogger sets the logger used for errors nobody can receive (default slog.Default()).
type PanicError ¶
type PanicError struct {
// Value is the recovered panic value.
Value any
}
PanicError wraps a value recovered from a panic inside a route. It is returned by the route handler instead of crashing the process.
type Producer ¶
Producer sends an exchange to an endpoint. A synchronous producer (direct:, http:) may replace ex.Body with a reply. Implementations must be safe for concurrent use.
type RetryPolicy ¶
type RetryPolicy struct {
// MaxAttempts is the total number of tries including the first (default 3).
MaxAttempts int
// Delay is the wait before the second attempt.
Delay time.Duration
// Multiplier scales the delay after every failed attempt (values below 1 mean 1).
Multiplier float64
// MaxDelay caps the delay when non-zero.
MaxDelay time.Duration
// Retryable decides whether err deserves another attempt. Nil retries every
// error except those marked with Permanent and context cancellation.
Retryable func(err error) bool
}
RetryPolicy configures Retry. The zero value means 3 attempts with no delay.
type Route ¶
type Route struct {
// contains filtered or unexported fields
}
Route is a pipeline fed by a consumer endpoint. Build one with From and Then. A Route must not be modified after AddRoute.
func From ¶
func From(uri string, opts ...RouteOption) *Route
From starts a route consuming the endpoint uri, for example "seda:orders".
type RouteOption ¶
type RouteOption func(*Route)
RouteOption configures a Route.
func WithDeadLetter ¶
func WithDeadLetter(step Step) RouteOption
WithDeadLetter sets the dead-letter channel: when the route fails for an exchange, the exchange (with HeaderError set and Err populated) is handed to step. If step succeeds the failure counts as handled and the consumer sees no error. The step runs even during shutdown.
Example ¶
package main
import (
"context"
"fmt"
"github.com/JiaBao-do/routex"
)
func main() {
e := routex.New()
dlq := routex.Process(func(_ context.Context, ex *routex.Exchange) error {
fmt.Println("parked:", ex.Body, "-", ex.Err())
return nil
})
_ = e.AddRoute(routex.From("direct:in", routex.WithDeadLetter(dlq)).Then(
routex.Transform(func(_ context.Context, s string) (int, error) { return 0, fmt.Errorf("cannot parse %q", s) })))
ctx, cancel := context.WithCancel(context.Background())
wait, _ := e.Start(ctx)
_, err := e.Request(ctx, "direct:in", "abc")
fmt.Println("caller sees:", err)
cancel()
_ = wait()
}
Output: parked: abc - transform: cannot parse "abc" caller sees: <nil>
func WithID ¶
func WithID(id string) RouteOption
WithID names the route (default: the from URI). Ids must be unique per engine.
type Step ¶
Step is one stage of a route. Process may mutate ex. Returning an error aborts the route for this exchange and triggers the route's dead-letter step. Calling ex.Stop ends processing without error.
Steps built by this package are safe for concurrent use by multiple goroutines (routes with several concurrent consumers do that); custom steps used in such routes must be too.
func Aggregate ¶
func Aggregate(key func(*Exchange) (string, error), strategy AggregateStrategy, opts ...AggregateOption) Step
Aggregate is the aggregator. Exchanges are grouped by key; strategy folds each into its group; when a group completes (CompleteSize, CompleteWhen or CompleteTimeout) the aggregate continues down the route and the individual exchanges are stopped. With no completion option a group never completes.
Groups live in memory until they complete: always set a size or timeout and bound the key space, otherwise memory grows without limit. Aggregate must be a direct step of a route or of a nested Pipeline.
Example ¶
package main
import (
"context"
"fmt"
"github.com/JiaBao-do/routex"
)
func main() {
step := routex.Aggregate(
func(ex *routex.Exchange) (string, error) { return "all", nil },
routex.AppendBodies[int](),
routex.CompleteSize(3),
)
for i := 1; i <= 3; i++ {
ex := routex.NewExchange(i)
_ = step.Process(context.Background(), ex)
if !ex.Stopped() {
fmt.Println(ex.Body)
}
}
}
Output: [1 2 3]
func Choice ¶
Choice is the content-based router: the first branch whose condition matches processes the exchange; an Otherwise branch matches always. If nothing matches the exchange passes through unchanged.
Example ¶
package main
import (
"context"
"fmt"
"github.com/JiaBao-do/routex"
)
func main() {
step := routex.Choice(
routex.WhenBody(func(n int) bool { return n < 0 }, routex.SetHeader("kind", "negative")),
routex.Otherwise(routex.SetHeader("kind", "other")),
)
ex := routex.NewExchange(-5)
_ = step.Process(context.Background(), ex)
kind, _ := ex.Header("kind")
fmt.Println(kind)
}
Output: negative
func Enrich ¶
func Enrich[In, Extra, Out any](fetch func(ctx context.Context, in In) (Extra, error), merge func(in In, extra Extra) Out) Step
Enrich is the content enricher: fetch obtains extra data for the body (a lookup, an HTTP call, ...) and merge combines both into the new body.
func Filter ¶
Filter is the message filter: exchanges whose body (as T) fails keep is dropped, that is stopped without error. A body that is not a T is an error.
Example ¶
package main
import (
"context"
"fmt"
"github.com/JiaBao-do/routex"
)
func main() {
step := routex.Filter(func(n int) bool { return n > 0 })
for _, n := range []int{-1, 1} {
ex := routex.NewExchange(n)
_ = step.Process(context.Background(), ex)
fmt.Println(n, "dropped:", ex.Stopped())
}
}
Output: -1 dropped: true 1 dropped: false
func Idempotent ¶
Idempotent is the idempotent consumer. key derives a message key; if repo has seen it the exchange is stopped as a duplicate, otherwise sub processes it. When sub fails the key is removed again so a redelivery is not lost.
Example ¶
package main
import (
"context"
"fmt"
"github.com/JiaBao-do/routex"
)
func main() {
repo := routex.NewMemoryRepository(100)
step := routex.Idempotent(
func(ex *routex.Exchange) (string, error) { return ex.Body.(string), nil },
repo, routex.Process(func(_ context.Context, ex *routex.Exchange) error {
fmt.Println("processing", ex.Body)
return nil
}))
for _, id := range []string{"a", "a", "b"} {
_ = step.Process(context.Background(), routex.NewExchange(id))
}
}
Output: processing a processing b
func Multicast ¶
func Multicast(branches []Step, opts ...ForkOption) Step
Multicast sends a copy of the exchange to every branch. Combine the branch results with Join; without it the original exchange continues unchanged.
func Pipeline ¶
Pipeline returns a Step that runs steps in order and stops at the first error or when the exchange is stopped. Only steps placed directly in a route or in a nested Pipeline can own background work such as Aggregate timeouts.
func RecipientList ¶
func RecipientList(resolve func(*Exchange) ([]string, error), opts ...ForkOption) Step
RecipientList sends a copy of the exchange to each endpoint URI returned by resolve, computed per exchange. Options are those of Multicast.
func Retry ¶
func Retry(policy RetryPolicy, sub Step) Step
Retry runs sub and retries it on error according to policy. Before every retry the exchange is restored to its state on entry (ID, body value, headers); mutations made in place to the body value itself are not undone. After the last attempt the final error is returned wrapped with the attempt count. The wait between attempts honours context cancellation.
Example ¶
package main
import (
"context"
"fmt"
"github.com/JiaBao-do/routex"
)
func main() {
calls := 0
flaky := routex.Process(func(context.Context, *routex.Exchange) error {
calls++
if calls < 3 {
return fmt.Errorf("attempt %d failed", calls)
}
return nil
})
err := routex.Retry(routex.RetryPolicy{MaxAttempts: 5}, flaky).Process(context.Background(), routex.NewExchange(nil))
fmt.Println(calls, err)
}
Output: 3 <nil>
func Split ¶
func Split[T, E any](fn func(T) ([]E, error), sub Step, opts ...ForkOption) Step
Split is the splitter: fn breaks the body (as T) into fragments and every fragment is processed by sub in its own exchange, with HeaderSplitIndex, HeaderSplitSize and HeaderParentID set. Use Join or CollectBodies to re-assemble the results into the original exchange.
Example ¶
package main
import (
"context"
"fmt"
"strings"
"github.com/JiaBao-do/routex"
)
func main() {
step := routex.Split(
func(s string) ([]string, error) { return strings.Fields(s), nil },
routex.Transform(func(_ context.Context, w string) (string, error) { return strings.ToUpper(w), nil }),
routex.CollectBodies[string](),
)
ex := routex.NewExchange("a quick fox")
_ = step.Process(context.Background(), ex)
fmt.Println(ex.Body)
}
Output: [A QUICK FOX]
func SplitSeq ¶
SplitSeq is Split for a splitter function returning an iterator. The iterator is drained before any fragment is processed.
func Throttle ¶
Throttle is the throttler: it lets at most n exchanges per period pass, spacing them evenly (period/n apart) and blocking callers until their slot arrives. A blocked caller returns ctx.Err() if the context ends first. The returned step is safe for concurrent use and its state is per step value. Waiting callers are served in arrival order of their slot reservation.
func To ¶
To sends the exchange to the endpoint uri and continues with the (possibly replaced) exchange. The producer is resolved on first use through the running Engine, so an unknown scheme fails at runtime, not at build time.
func Transform ¶
Transform is the message translator: it reads the body as In, applies fn and stores the result as the new body. A body that is not an In yields an error wrapping ErrBodyType.
Example ¶
package main
import (
"context"
"fmt"
"github.com/JiaBao-do/routex"
)
func main() {
step := routex.Transform(func(_ context.Context, s string) (int, error) { return len(s), nil })
ex := routex.NewExchange("hello")
_ = step.Process(context.Background(), ex)
fmt.Println(ex.Body)
}
Output: 5
func WireTap ¶
WireTap sends a copy of the exchange to sub in a separate goroutine and continues immediately. Errors from sub are logged, never propagated. Under Engine.Run the tap goroutines are awaited at shutdown and run to completion even after the context is cancelled. The body value is shared with the main flow: sub must not mutate it.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
examples
|
|
|
deadletter
command
Dead-letter channel with retry: a flaky downstream is retried with backoff; exchanges that still fail are parked in the dead-letter step with the error.
|
Dead-letter channel with retry: a flaky downstream is retried with backoff; exchanges that still fail are parked in the dead-letter step with the error. |
|
idempotent
command
Idempotent consumer: redelivered messages (same key) are processed once; a failed message releases its key so the redelivery is processed.
|
Idempotent consumer: redelivered messages (same key) are processed once; a failed message releases its key so the redelivery is processed. |
|
orders
command
Realistic order processing.
|
Realistic order processing. |
|
quickstart
command
Quickstart: a timer emits six ticks, only even ones pass the filter, and each is printed.
|
Quickstart: a timer emits six ticks, only even ones pass the filter, and each is printed. |
|
router
command
Content-based router: orders go to different handlers by amount and country.
|
Content-based router: orders go to different handlers by amount and country. |
|
splitagg
command
Splitter and aggregator: a batch of line items is split, each item priced in parallel, and the results are re-assembled; a second route aggregates a stream of readings per sensor in groups of three.
|
Splitter and aggregator: a batch of line items is split, each item priced in parallel, and the results are re-assembled; a second route aggregates a stream of readings per sensor in groups of three. |