fdbgo

Pure Go FoundationDB Client

A native Go FDB client that speaks the FDB wire protocol directly — no cgo, no libfdb_c. Connects to FDB clusters over TCP, handles the full transaction lifecycle, and is wire-compatible with FDB 7.3.x.

Why

The official Go binding (github.com/apple/foundationdb/bindings/go) wraps libfdb_c via cgo. This creates deployment friction (must ship the C library), cross-compilation pain, and CGo overhead. This package eliminates all of that.

Status

Feature-complete. Full Apple C binding API parity — zero stubs remaining (except RebootWorker, admin-only). Beats CGo on reads.

Working and tested against real FDB 7.3.77:

• Get, GetKey, GetRange (multi-shard, all streaming modes), GetEstimatedRangeSizeBytes, GetRangeSplitPoints
• Set, Clear, ClearRange, all 14 atomic mutation types
• Transact with automatic retry (all retryable FDB error codes including tag_throttled, cluster_version_changed)
• MVCC conflict detection, OnError, exponential backoff with configurable MaxRetryDelay
• Coordinator bootstrap, GRV batching + caching (100ms TTL), storage server location discovery
• Snapshot reads, Watch (long-poll), Versionstamp, Tenants (CRUD via system keys)
• Transaction options: RYW disable, snapshot RYW disable, size limit, timeout, retry limit, lock-aware
• GetPipelined for true request pipelining (no goroutine per Get)
• TLS support (mutual auth + CA cert), QueueModel load balancing (C++ Smoother + server penalty), connection keep-warm
• Read-your-writes cache with full atomic op merging (all 14 types mirror C++ Atomic.h)
• LocalityGetAddressesForKey, LocalityGetBoundaryKeys, OpenWithConnectionString, GetClientStatus

Retries and timeouts are UNBOUNDED by default — read this before migrating

A transaction created with no options has no timeout and no retry limit. A Transact against a down or unreachable cluster retries until the cluster returns or the caller stops it. Nothing internal stops it.

This is not a divergence from libfdb_c — it is libfdb_c's behaviour, matched deliberately. The C++ client's per-transaction defaults (ReadYourWrites.actor.cpp:2078-2082) are:

void ReadYourWritesTransactionOptions::reset(Transaction const& tr) {
	memset(this, 0, sizeof(*this));
	timeoutInSeconds = 0.0;
	maxRetries = -1;

and resetTimeout() (:1576-1578) arms the timebomb actor only when timeoutInSeconds is non-zero, so by default no timer exists at all. fdb.options says the same of the timeout option: "If set to 0, will disable all timeouts." A default-configured libfdb_c transaction hangs against a dead cluster exactly as long as this one does. Do not read "no internal timeout" as a Go weakness to work around — both clients hand the bound to the caller. That is the FoundationDB contract.

Where this client is stricter than libfdb_c

Bootstrap — the initial coordinator connection — is internally capped at 60s (defaultBootstrapTimeout, fdb/database.go:139-153) whenever the caller supplies no deadline of its own. libfdb_c has no equivalent bound and waits indefinitely. So fdb.OpenDatabase against an unreachable cluster fails fast where the C client would not. Only the live database — post-open reconnection and the transaction retry loop — is unbounded.

What you must do

Pick at least one bound for every transaction. Any of these terminates it:

The no-context Database.Transact runs on context.Background() to stay drop-in compatible with the Apple Go binding, so a ctx deadline is not available to it — bound it with a timeout or retry limit, or use TransactCtx.

Migrating from libfdb_c: if you set a transaction timeout there, keep setting it here — it works the same way. If you set nothing there, you were already unbounded there, and you are equally unbounded here.

Pinned by client/unbounded_default_pin_test.go (default is unbounded; each of the three bounds terminates) and fdb/unbounded_default_pin_test.go + fdb/database_bootstrap_test.go (no internal deadline on the bare Transact path; the 60s bootstrap cap). Full detail: go doc fdb.dev/pkg/fdbgo.

Architecture

pkg/fdbgo/
├── client/          Transaction lifecycle, retry logic, read/commit paths
├── transport/       TCP framing, ConnectPacket handshake, connection multiplexing
├── wire/            FDB FlatBuffers framework: Writer, Reader, VTable computation
│   └── types/       One file per FDB message type + vtables_generated.go
└── cmd/             (at repo root: cmd/fdb-schema-extract/)

wire/ — Serialization framework

FDB uses a custom FlatBuffers-like binary format (NOT Google FlatBuffers). Each serialized message has:

• A vtable — array of uint16 field offsets, determines where each field lives in the object body
• An object body — fixed-size struct with fields at vtable-specified offsets
• Out-of-line data — variable-length fields (strings, vectors, nested structs) referenced via RelativeOffsets

The wire package implements this format:

• Writer / ObjectWriter — serialize messages. Handles vtable emission, soffset computation, RelativeOffset patching, nested struct allocation, OOL data.
• Reader — deserialize messages. Navigates FakeRoot wrapper, reads fields by vtable slot index. Self-describing: reads the vtable from the wire data, so forward/backward compatible.
• VTable — the type alias []uint16. Generated constants in vtables_generated.go.
• ReadErrorOr — unwraps FDB's ErrorOr<T> union responses.
• MarshalStructBlob / PackVectorOfStructBlobs — serialize vector elements for VectorRef<serialize_member>.
• WriteRootObject — for union_like_traits types (ErrorOr) where the root object IS the union (no FakeRoot wrapper).

wire/types/ — Message types

Each FDB message type (request or reply) has a Go file with:

• A struct with typed fields
• MarshalFDB() for requests — constructs the full wire message using vtable-derived offsets
• UnmarshalFDB() / UnmarshalFrom() for replies — reads fields via wire.Reader

All field offsets are derived from vtable constants in vtables_generated.go. No hardcoded byte offsets. If FDB changes a struct layout, regenerating the vtables is sufficient.

Shared write helpers (WriteReplyPromise, WriteTenantInfo, writeKeySelectorRef) encapsulate common nested struct patterns.

vtables_generated.go — The bridge between C++ and Go

This is the single source of truth for wire layout. Generated by cmd/fdb-schema-extract/, a C++ binary that compiles against real FDB headers and extracts:

• VTable constants — field byte offsets, computed by C++ template metaprogramming (get_vtable<Fields...>())
• VTable closures — all vtables transitively reachable from a message type
• File identifiers — per-message type hashes
• Slot constants — Reader slot indices with field names (e.g., ClientDBInfoSlotGrvProxies = 0)

28 types extracted. Slot indices computed mechanically from C++ traits (union_like = 2 slots, everything else = 1). Field names parsed from C++ source via name_capture.cpp.

transport/ — TCP layer

• Conn — multiplexed FDB connection. Multiple concurrent requests share one TCP connection, matched by endpoint tokens (128-bit UIDs).
• ConnectPacket — FDB handshake (protocol version negotiation, IPv4/IPv6).
• Frame format: [packetLen(4)][xxh3Checksum(8)][destToken(16)][body].
• PING keepalive: responds to server PINGs, sends outbound PINGs via connectionMonitor (matches C++ FlowTransport). Detects dead connections in ~3.5s.

client/ — Transaction lifecycle

• Database — connection manager. Transact() with automatic retry.
• Transaction — buffered mutations, read conflict tracking, GRV, commit.
• GRVBatcher — batches concurrent GetReadVersion calls into single RPC.
• LocationCache — maps keys to storage server addresses via GetKeyServerLocationsRequest.
• Connection pool with port-match reuse (handles Docker address translation).

FDB version compatibility

This client targets FDB 7.3.x. The wire format is defined by C++ code, not an IDL. Compatibility with other versions requires:

1. Matching the protocol version in the ConnectPacket handshake
2. Matching the vtable layouts for all message types we send
3. Handling any new fields in responses (the Reader is forward-compatible — unknown vtable entries are ignored)

Upgrading to a new FDB version

# 1. Checkout the FDB source at the target version
cd /path/to/foundationdb
git checkout 7.4.0  # or whatever

# 2. Rebuild the C++ vtable extractor
cd /path/to/this-repo
bash cmd/fdb-schema-extract/build.sh /path/to/foundationdb pkg/fdbgo/wire/types/vtables_generated.go

# 3. Diff the generated output
git diff pkg/fdbgo/wire/types/vtables_generated.go

# 4. For each changed type:
#    - If only vtable VALUES changed (field offsets shifted): no Go code changes needed,
#      the MarshalFDB methods use int(vt[N]) which adapts automatically.
#    - If SLOT COUNT changed (field added/removed): update the MarshalFDB method to
#      write/skip the new field. Update the slot constant references.
#    - If a NEW message type was added: create a new Go file in wire/types/.

# 5. Update the protocol version constant in transport/
#    (ProtocolVersion73 → ProtocolVersion74)

# 6. Run tests against a testcontainer at the new version

The key insight: vtable constants are the only C++ build-time artifact. Everything else is Go code that references these constants. The C++ extractor bridges C++'s compile-time template metaprogramming (which Go can't do) into static Go constants.

What changes between FDB versions

What does NOT change

• The FlatBuffers wire format itself (vtable + soffset + RelativeOffset)
• The FakeRoot wrapping pattern
• The ErrorOr union flattening
• UID layout (always 16 bytes)
• ReplyPromise structure
• TCP framing and checksum format

Testing

# All tests (uses Bazel, includes testcontainer tests against real FDB)
just test

# Specific test
bazelisk test //pkg/fdbgo/client:client_test --test_arg="-test.run=TestSetGet" \
  --test_arg="-test.v" --test_output=streamed --strategy=TestRunner=local

Client tests run against real FDB 7.3.77 via testcontainers-go (Docker required). 78 C binding port tests (96% of C test suite) + 30 correctness tests + 6 fault injection tests + 15 interop tests (Go↔CGo) + benchmarks. Line coverage: 72.4% (client), 78.8% (record layer). Binding stress: 200+ seeds × 1000 ops validated (0 failures). just coverage generates HTML report with per-package coverage.

Benchmarks

# Pure Go vs CGo (libfdb_c) — full Get path comparison
bazelisk run //pkg/fdbgo/client:client_test -- \
  -test.run='^$' \
  -test.bench='BenchmarkGetValue' \
  -test.benchtime=10s \
  -test.benchmem \
  -test.count=3

Both benchmarks read the same 100-byte key from FDB testcontainers. Measures the full path: GRV + locate + read + parse.

Baseline (Ryzen 9 3900X, FDB 7.3.77 testcontainer, 2026-04-12):

Reads: Go beats CGo by 3.5x. Writes: parity (1.00x). The 18 allocs/op on Get is the structural floor — top allocators: ReadFrame payload (9%), PrepareReply channel (13% with pool miss), conflict range buffer (6%), transaction struct (5%). All already pooled or minimal. Frame buffer pooling investigated (dayshift-6c) — no improvement because body shares backing array with payload, pooling requires an extra copy.

Fault injection

The client supports a custom DialFunc for testing failure scenarios against real FDB. Same pattern as http.Transport.DialContext — no mocks, no artificial interfaces.

// faultConn wraps net.Conn to inject failures at the TCP level.
type faultConn struct {
    net.Conn
    killReads atomic.Bool
}

func (f *faultConn) Read(b []byte) (int, error) {
    if f.killReads.Load() {
        return 0, io.EOF  // simulate network failure
    }
    return f.Conn.Read(b)
}

// Inject the custom dialer before connecting.
cluster := client.NewClusterFromConfig(cf)
cluster.SetDialFunc(func(ctx context.Context, network, addr string) (net.Conn, error) {
    conn, err := net.DialTimeout(network, addr, 5*time.Second)
    if err != nil {
        return nil, err
    }
    fc := &faultConn{Conn: conn}
    // Store fc somewhere so the test can arm/disarm it later
    return fc, nil
})
cluster.Connect(ctx)

// Later, arm the fault:
fc.killReads.Store(true)   // next Read → EOF → connection dies
// ... commit happens, server processes it, but reply is lost ...

// Disarm and reconnect:
fc.killReads.Store(false)

What you can test

How self-conflicting works (commit_unknown_result)

When OnError receives error 1021, the transaction MAY have committed on the server. To prevent double-apply on retry:

1. OnError(1021) copies all write conflict ranges into the read conflict set
2. Transaction is reset, Transact() retries the user function
3. On the retry's commit, the commit proxy checks: "did any read conflict ranges change since read version?"
4. Since the ORIGINAL commit wrote to those ranges, the check fails → not_committed (1020)
5. The retry does NOT apply mutations — no double-apply

This achieves the same safety as C++ NativeAPI::makeSelfConflicting(). Additionally, commitDummyTransaction runs a synchronization barrier before returning commit_unknown_result — a separate transaction that conflicts with the original, confirming it's no longer in-flight at the commit proxy. Both mechanisms combined match C++ exactly. Verified by TestCommitUnknownResult_NoDoubleApply: atomic ADD 5 to a counter, kill the reply, verify counter=15 (not 20).

Known divergences from C++ (audited 2026-04-12)

Systematic audit against foundationdb/fdbclient/NativeAPI.actor.cpp, ReadYourWritesTransaction.actor.cpp, and Atomic.h. All correctness bugs were fixed; these are intentional or architectural differences:

Adding a new request/response type

1. Add the C++ type to cmd/fdb-schema-extract/main.cpp (include header, add extractType<T>() call)
2. If the type has field names in its serialize() method, add the source file + type name to name_capture.cpp
3. Rebuild: bash cmd/fdb-schema-extract/build.sh /path/to/fdb pkg/fdbgo/wire/types/vtables_generated.go
4. Create wire/types/my_type.go with struct + MarshalFDB() / UnmarshalFrom() using the generated vtable and slot constants
5. Wire it into the client code
6. Test against real FDB