README
¶
XRPL Client Implementation Specification
Overview
This document outlines the architectural design of a blockchain client implementation for XRPL in GoLang. The design emphasizes concurrent processing, data consistency, and efficient storage management through careful separation of concerns.
Core Architecture
Concurrent Processing Components
The system operates through multiple concurrent goroutines, each responsible for a specific aspect of the blockchain client:
The Mempool Handler operates as an independent goroutine managing pending transactions. It continuously processes incoming transactions, validates them according to network rules, and maintains them until they are included in blocks or expire.
The Consensus Engine runs in its own goroutine and serves as the heart of the blockchain client. It implements the consensus protocol, coordinates block creation, and ensures network-wide agreement on the blockchain state.
The JSON-RPC Server provides the primary interface for external interaction. Running in a dedicated goroutine, it handles API requests for blockchain data, transaction submission, and queries about network state.
The WebSocket Server maintains persistent connections with clients, running separately from the JSON-RPC server. It enables real-time updates about network events, new blocks, and transaction status changes.
Storage Architecture
The system employs multiple specialized databases, each optimized for specific data access patterns:
The Mempool Database serves as temporary storage for pending transactions. It requires high-throughput write operations and quick read access for transaction validation and block creation.
The State Database maintains the current blockchain state. It stores account balances, smart contract states, and other mutable blockchain data. This database must support atomic updates during block processing while allowing concurrent reads for query operations.
The Block Database stores the immutable history of all blocks. It maintains block headers, block bodies, and the relationships between blocks, supporting chain reorganizations when necessary.
The Transaction Database maintains a comprehensive record of all processed transactions. It enables efficient querying of transaction history, status, and receipts without needing to scan through blocks.
Data Flow and Interaction
All components interact through Go channels, ensuring thread-safe communication. The system maintains data consistency through strategic use of read-write mutexes (sync.RWMutex), allowing concurrent reads while protecting write operations.
Future Considerations
The architecture allows for future implementation of additional indexes to optimize specific query patterns. These indexes would enhance performance for common operations while maintaining the system's core functionality.
Implementation Notes
This specification intentionally omits implementation details of the XRPL consensus protocol, focusing instead on the architectural framework that will support it. The actual consensus implementation will need to adhere to the XRPL protocol specifications while fitting within this architectural design.
The use of separate databases should not be confused with separate physical storage - these may be implemented as separate namespaces within a single database engine, provided the chosen engine can maintain the required performance characteristics for each data type.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cmdexit defines the sentinel error a CLI command returns to request a non-zero process exit without an additional error line.
|
Package cmdexit defines the sentinel error a CLI command returns to request a non-zero process exit without an additional error line. |
|
Package consensus defines the interface and types for XRPL consensus algorithms.
|
Package consensus defines the interface and types for XRPL consensus algorithms. |
|
adaptor
Package adaptor provides the concrete implementation of the consensus.Adaptor interface, bridging the consensus engine to the ledger service, P2P overlay, and transaction queue.
|
Package adaptor provides the concrete implementation of the consensus.Adaptor interface, bridging the consensus engine to the ledger service, P2P overlay, and transaction queue. |
|
amendmentvote
Package amendmentvote decides whether to inject EnableAmendment pseudo-txs into the consensus tx set at a flag-ledger boundary.
|
Package amendmentvote decides whether to inject EnableAmendment pseudo-txs into the consensus tx set at a flag-ledger boundary. |
|
archive
Package archive persists validations evicted from the in-memory tracker.
|
Package archive persists validations evicted from the in-memory tracker. |
|
common
Package common holds helpers shared by the consensus vote producers (feevote, amendmentvote, negativeunlvote) that assemble pseudo-txs for injection into the flag-ledger tx set.
|
Package common holds helpers shared by the consensus vote producers (feevote, amendmentvote, negativeunlvote) that assemble pseudo-txs for injection into the flag-ledger tx set. |
|
csf
Package csf provides a Consensus Simulation Framework: a deterministic discrete-event scheduler, a simulated peer-to-peer network, a trust graph and a ledger oracle for exercising the production consensus engine without real time or sockets.
|
Package csf provides a Consensus Simulation Framework: a deterministic discrete-event scheduler, a simulated peer-to-peer network, a trust graph and a ledger oracle for exercising the production consensus engine without real time or sockets. |
|
feevote
Package feevote decides whether to inject a SetFee pseudo-tx into the consensus tx set at a flag-ledger boundary, tallying trusted validators' fee votes from the prior voting ledger.
|
Package feevote decides whether to inject a SetFee pseudo-tx into the consensus tx set at a flag-ledger boundary, tallying trusted validators' fee votes from the prior voting ledger. |
|
ledgertrie
Package ledgertrie implements branchSupport-based preferred-ledger selection over a compressed ancestry trie, porting rippled's LedgerTrie (LedgerTrie.h).
|
Package ledgertrie implements branchSupport-based preferred-ledger selection over a compressed ancestry trie, porting rippled's LedgerTrie (LedgerTrie.h). |
|
negativeunlvote
Package negativeunlvote produces UNLModify pseudo-transactions for flag ledgers.
|
Package negativeunlvote produces UNLModify pseudo-transactions for flag ledgers. |
|
rcl
Package rcl implements the Ripple Consensus Ledger algorithm.
|
Package rcl implements the Ripple Consensus Ledger algorithm. |
|
Package feetrack implements the local-node load-fee tracker, mirroring the behaviour of rippled's LoadFeeTrack.
|
Package feetrack implements the local-node load-fee tracker, mirroring the behaviour of rippled's LoadFeeTrack. |
|
Package grpc implements the XRPLedgerAPIService gRPC surface mirroring rippled's binary-only ledger RPCs (the API surface consumed by Clio): GetLedger, GetLedgerEntry, GetLedgerData and GetLedgerDiff.
|
Package grpc implements the XRPLedgerAPIService gRPC surface mirroring rippled's binary-only ledger RPCs (the API surface consumed by Clio): GetLedger, GetLedgerEntry, GetLedgerData and GetLedgerDiff. |
|
cleaner
Package cleaner implements a background ledger-integrity verifier, the go-xrpl analog of rippled's LedgerCleaner.
|
Package cleaner implements a background ledger-integrity verifier, the go-xrpl analog of rippled's LedgerCleaner. |
|
inbound
Package inbound provides lightweight ledger acquisition from peers.
|
Package inbound provides lightweight ledger acquisition from peers. |
|
inbound/inboundtest
Package inboundtest provides shared test utilities for the internal/ledger/inbound package and its sibling-package consumers.
|
Package inboundtest provides shared test utilities for the internal/ledger/inbound package and its sibling-package consumers. |
|
localtxs
Package localtxs is goxrpl's port of rippled's app/ledger/LocalTxs.
|
Package localtxs is goxrpl's port of rippled's app/ledger/LocalTxs. |
|
negativeunl
Package negativeunl applies flag-ledger NegativeUNL transitions (pending ValidatorToDisable / ValidatorToReEnable → DisabledValidators) on a state map.
|
Package negativeunl applies flag-ledger NegativeUNL transitions (pending ValidatorToDisable / ValidatorToReEnable → DisabledValidators) on a state map. |
|
openledger
Package openledger implements rippled's OpenLedger semantics for go-xrpl.
|
Package openledger implements rippled's OpenLedger semantics for go-xrpl. |
|
replayfault
Package replayfault records replay failures that must stop signing and proposal publication until the affected state has been independently verified.
|
Package replayfault records replay failures that must stop signing and proposal publication until the affected state has been independently verified. |
|
service/svcerr
Package svcerr defines the typed sentinel errors returned by the ledger service.
|
Package svcerr defines the typed sentinel errors returned by the ledger service. |
|
shamapstore
Package shamapstore implements go-xrpl's online-delete subsystem — the go-xrpl equivalent of rippled's SHAMapStore (src/xrpld/app/misc/SHAMapStore.h).
|
Package shamapstore implements go-xrpl's online-delete subsystem — the go-xrpl equivalent of rippled's SHAMapStore (src/xrpld/app/misc/SHAMapStore.h). |
|
skiplist
Package skiplist manages the LedgerHashes skip-list SLEs that let a single ledger resolve the hashes of its ancestors.
|
Package skiplist manages the LedgerHashes skip-list SLEs that let a single ledger resolve the hashes of its ancestors. |
|
Package manifest implements validator manifest parsing, verification, caching, and persistence.
|
Package manifest implements validator manifest parsing, verification, caching, and persistence. |
|
Package observability hosts process-level metrics surfaced to RPC.
|
Package observability hosts process-level metrics surfaced to RPC. |
|
Package peermanagement implements XRPL peer-to-peer networking.
|
Package peermanagement implements XRPL peer-to-peer networking. |
|
cluster
Package cluster maintains the registry of cluster-trusted node identities — operators run a small set of nodes that they configure to know about each other via [cluster_nodes].
|
Package cluster maintains the registry of cluster-trusted node identities — operators run a small set of nodes that they configure to know about each other via [cluster_nodes]. |
|
message
Package message implements XRPL peer protocol message types and serialization.
|
Package message implements XRPL peer protocol message types and serialization. |
|
peertls
Package peertls provides the TLS 1.2-or-newer transport used by XRPL peers.
|
Package peertls provides the TLS 1.2-or-newer transport used by XRPL peers. |
|
peertls/shim
Package shim is the cgo binding for the OpenSSL TLS engine used by peertls.
|
Package shim is the cgo binding for the OpenSSL TLS engine used by peertls. |
|
proto/internal/protogen
command
|
|
|
resource
Package resource implements bounded per-endpoint peer load tracking.
|
Package resource implements bounded per-endpoint peer load tracking. |
|
Package replaytool implements the offline mainnet-replay and fixture-replay developer commands (`replay`, `replay-range`).
|
Package replaytool implements the offline mainnet-replay and fixture-replay developer commands (`replay`, `replay-range`). |
|
txprojection
Package txprojection contains the API-version-specific transaction JSON shaping shared by RPC handlers and WebSocket publishers.
|
Package txprojection contains the API-version-specific transaction JSON shaping shared by RPC handlers and WebSocket publishers. |
|
Package statecompare reads replay manifests and immutable pack objects from the xrpl-state-compare data plane.
|
Package statecompare reads replay manifests and immutable pack objects from the xrpl-state-compare data plane. |
|
Package jtx provides test infrastructure for XRPL transaction testing.
|
Package jtx provides test infrastructure for XRPL transaction testing. |
|
amm
Package amm provides test builders for AMM transactions.
|
Package amm provides test builders for AMM transactions. |
|
batch
Package batch provides test builder helpers for Batch transactions.
|
Package batch provides test builder helpers for Batch transactions. |
|
conformance
Package conformance provides a test runner for xrpl-fixtures test vectors.
|
Package conformance provides a test runner for xrpl-fixtures test vectors. |
|
consensus
Package consensus provides integration test utilities for multi-node consensus testing.
|
Package consensus provides integration test utilities for multi-node consensus testing. |
|
depositpreauth
Package depositpreauth provides fluent transaction builder helpers for DepositPreauth testing, plus integration tests matching rippled's DepositAuth_test.cpp and DepositPreauth_test sections.
|
Package depositpreauth provides fluent transaction builder helpers for DepositPreauth testing, plus integration tests matching rippled's DepositAuth_test.cpp and DepositPreauth_test sections. |
|
enginefuzz
Package enginefuzz hosts a stateful, property-based fuzz harness over the goXRPL transaction engine.
|
Package enginefuzz hosts a stateful, property-based fuzz harness over the goXRPL transaction engine. |
|
marketfixtures
Package marketfixtures supplies shared MPT book and amount vectors for flow and RPC tests.
|
Package marketfixtures supplies shared MPT book and amount vectors for flow and RPC tests. |
|
metadata
Package metadata provides test helpers for validating transaction metadata.
|
Package metadata provides test helpers for validating transaction metadata. |
|
mpt
Package mpt provides test helpers for MPT (Multi-Purpose Token) transaction testing.
|
Package mpt provides test helpers for MPT (Multi-Purpose Token) transaction testing. |
|
pathfuzz
Package pathfuzz hosts generative, property-based fuzz harnesses over the goXRPL payment engine: cross-currency flow (RippleCalc) and path discovery (the Pathfinder).
|
Package pathfuzz hosts generative, property-based fuzz harnesses over the goXRPL payment engine: cross-currency flow (RippleCalc) and path discovery (the Pathfinder). |
|
payment
Package builders provides fluent transaction builder helpers for testing.
|
Package builders provides fluent transaction builder helpers for testing. |
|
permissioneddex
Package permissioneddex provides test helpers for PermissionedDEX tests.
|
Package permissioneddex provides test helpers for PermissionedDEX tests. |
|
rpcenv
Package rpcenv wires the in-memory test ledger built by internal/testing into the same RPC handler registry the production server uses, so handlers can be exercised end-to-end against real ledger state.
|
Package rpcenv wires the in-memory test ledger built by internal/testing into the same RPC handler registry the production server uses, so handlers can be exercised end-to-end against real ledger state. |
|
ticket
Package ticket provides test helpers for Ticket transaction testing.
|
Package ticket provides test helpers for Ticket transaction testing. |
|
testutil
|
|
|
rippled
Package rippled locates pinned protocol source files for inventory tests.
|
Package rippled locates pinned protocol source files for inventory tests. |
|
all
Package all aggregates all transaction sub-packages and exposes a single RegisterAll() entry point that registers every transaction type with the tx registry.
|
Package all aggregates all transaction sub-packages and exposes a single RegisterAll() entry point that registers every transaction type with the tx registry. |
|
escrow
Package escrow implements EscrowCreate, EscrowFinish, and EscrowCancel transactions.
|
Package escrow implements EscrowCreate, EscrowFinish, and EscrowCancel transactions. |
|
lending
Package lending implements the XLS-66 LendingProtocol transaction types (LoanBroker* and Loan*), porting rippled 3.1.0's transactor semantics onto the XRPLNumber amortization math in the lmath subpackage.
|
Package lending implements the XLS-66 LendingProtocol transaction types (LoanBroker* and Loan*), porting rippled 3.1.0's transactor semantics onto the XRPLNumber amortization math in the lmath subpackage. |
|
lending/lmath
Package lmath ports the XLS-66 lending amortization math from rippled 3.1.0 (src/xrpld/app/misc/detail/LendingHelpers.cpp) onto the XRPLNumber foundation.
|
Package lmath ports the XLS-66 lending amortization math from rippled 3.1.0 (src/xrpld/app/misc/detail/LendingHelpers.cpp) onto the XRPLNumber foundation. |
|
offer
Reference: rippled CreateOffer.cpp, CancelOffer.cpp
|
Reference: rippled CreateOffer.cpp, CancelOffer.cpp |
|
sigcache
Package sigcache caches positive signature verdicts by transaction ID and signing namespace.
|
Package sigcache caches positive signature verdicts by transaction ID and signing namespace. |
|
validator
|
|
|
list
Package list implements the publisher-trust validator-list subsystem — the rippled-faithful counterpart of rippled/src/xrpld/app/misc/ ValidatorList.cpp + ValidatorSite.cpp.
|
Package list implements the publisher-trust validator-list subsystem — the rippled-faithful counterpart of rippled/src/xrpld/app/misc/ ValidatorList.cpp + ValidatorSite.cpp. |
|
Package watchdog detects stalled node event loops and guarantees terminal recovery when a stall reaches the abort threshold.
|
Package watchdog detects stalled node event loops and guarantees terminal recovery when a stall reaches the abort threshold. |
Click to show internal directories.
Click to hide internal directories.