After 3 years of productive and in-depth development, our Exonum blockchain team has released its 1.0.0 version of the framework! Exonum now boasts a rich set of features, supports multiple execution environments, and benefits from increased modularity and maintainability — making it an ideal blockchain framework for any organization or use case.

We will continue advancing the Exonum framework, but we have made its 1.0.0 release official because of its feature completeness and backward compatibility:

• Feature completeness: Exonum provides all necessary tools to develop full-scale blockchain applications. Some of the Exonum features positively differentiate it from other frameworks (for example, service lifecycles, which we’ve covered previously).
• Backward compatibility: Exonum developers can rely on the core framework interfaces knowing that they will be maintained in the successive releases. Exonum still has some experimental APIs, but they are relatively few and are clearly marked in the documentation.

Below, we cover the main changes between version 1.0.0 and its first release candidate (that is, 1.0.0-rc.1). See our previous article for an overview of functionalities present in the release candidate and, of course, visit our the project changelog to see the complete list of changes.

More Complete Service Lifecycle

The 1.0.0 release adds two new operations for service lifecycle: artifact unloading and service freezing.

Artifact unloading allows you to free up node resources by commanding the nodes in the blockchain network to unload a service artifact which is not used by any service. For example, such a situation may occur if a service that previously used the artifact was migrated to a newer artifact revision. In this case, the old artifact will most probably consume node resources, such as RAM and disk space, without necessity — it is unlikely (although not impossible) that a new service would be instantiated from the old artifact once the newer one is readily available.

In spirit of separating mechanism and policy and blockchain transparency, the Exonum framework and/or separate nodes do not decide on their own when to unload unused artifacts. Instead, this operation is implemented through the supervisor service similarly to other lifecycle events. In this way, unloading is fully deterministic, ensuring it is properly authorized and agreed among all nodes in the network.

Service freezing allows you to “freeze” the service state without switching off its read-only interfaces (for example, HTTP API provided by both Rust and Java runtimes). This enables serving these interfaces while the service is being migrated to a newer artifact; the service downtime in this case may become close to zero, even if the data migration takes a very long time to complete. Another use case for service freezing is temporarily disabling transaction processing if a bug / vulnerability is discovered in service code. In this case, service remains mostly functional, and is guaranteed to stay immutable until a newer artifact with the patched bug is available.

Please note: Unloading artifacts and freezing services are not yet supported by the Java runtime.

Customized Block Proposals and Block Skips

Previously, Exonum did not offer an avenue for you to customize the logic regarding how transactions are included into blocks. With 1.0.0, Exonum allows you to customize which transactions a node should include into a block using a flexible interface. This could be used in a few different ways, including:

• Whitelist / blacklist transactions by the public key that has authorized the transaction and/or by the service processing the transaction
• Prioritizing transactions by the processing service or other criteria
• Filtering transactions by proof-of-work or fee attached to the transaction
• Decoupling “similar” transactions (for example, timestamps generated by the time service) with user-defined similarity criteria

This block proposal customization is made even more powerful by the ability of nodes to skip block creation at a specific iteration of the consensus algorithm. Such a block skip is considered an outcome of the consensus algorithm alterative to accepting a normal block with transactions. Correspondingly, introducing block skips has no influence on the security properties of the consensus algorithm.

Unlike normal blocks, block skips are saved in a separate section of the blockchain storage. No more than one block skip is saved by the node at any given time. (since skipping a block does not change the blockchain state in any way, it is easy to see that such “forgetful” storage is enough to keep node syncing functional.) Thus, block skips allow you to immensely save on blockchain storage if transactions appear in the blockchain network seldomly — instead of permanently saving a new empty block each second or so, the nodes can now generate block skips with the same frequency (this providing “heartbeat” for the network), but only save the latest skip.

Please note: The block proposal customization is an experimental feature as of now. As per Rust traditions, you don’t pay for this feature (in terms of safety and performance) if you don’t use it.

Async / Await Throughout Codebase

Exonum now extensively uses async / await syntax, which was recently shipped to stable Rust. This affects both public-facing interfaces (for example, the way HTTP endpoints are defined for Rust services) and internals (such as the network module within exonum-node crate).

Using async / await allows to make code more concise and readable and thus reduce the amount of inefficiencies and logic errors. In some cases, the refactoring associated with async / await has opened qualitatively new capabilities. As an example, sending transactions to the node (say, from an HTTP API handler of a service) was previously a synchronous operation which blocked internally if the capacity of the node to process transactions was exceeded. This could potentially lead to API hang-up. After refactoring, sending transactions is explicitly asynchronous so that there is no hidden cost to it.

A final note: Since our launch of Exonum in 2017, we’ve worked with several developers around the world who have made contributions, asked questions and advised us on our Exonum open-source blockchain framework. Thanks to all of you for your help — we would not be here today without you!

—

Original source: https://exonum.com/blog/blockchain-framework-exonum-1–0–0

Supply chains in the retail oil and gas industry are internationally distributed and often complicated. The timely flow of accurate paperwork is absolutely essential for operational success, but mistakes are often made. This is in part due to the manual reconciliation system that is time-consuming and subject to human error. There’s also the issue of tracking and processing expenses, as paperwork constantly changes hands. Using blockchain to improve the exchange of electronic data can improve efficiency and seriously streamline operational timelines and costs. In this article, we highlight the work of IT company IK Sibintek, who used Exonum to build a better supply chain for gas stations and their retail partners.

About Blockchain for Supply Chains

Blockchain can help simplify the complexity (and accompanying costs) of a supply chain by maximizing efficiencies and reducing the likelihood of error when sharing information and documentation. It can also enhance security, as the technology eliminates the need to trust any one person in the network. Data entered into the blockchain cannot be changed or tampered with, thanks to its built-in cryptography. The blockchain can then be easily checked by any member of the network. This means that there is no longer any need for intermediaries, and online audits can be conducted by any member of the network. The result is a system that reduces risk and lowers transaction processing costs. Up-to-date information on goods accepted by the network, as well as the reasons behind any “non-acceptance,” make it easy to increase efficiency. Blockchain can also provide added flexibility. A decision to place an additional order can be processed smoothly, without any hiccups.

The Use Case:

Sibintek’s goal was to create a unified trustworthy environment with its counterparties providing guaranteed data integrity, so that the data used and maintained by the system are complete, correct, and free of contradictions. There are large amounts of paperwork and emails exchanged by gas stations and suppliers when making orders. Each entity (gas stations and suppliers) uses its own accounting systems for managing this paper trail, often leading to supply order delays, inconsistencies and high-cost payment reconciliations. Classic integration methods (to help the systems communication with one another) weren’t a suitable solution. Blockchain, however, could provide a trusted shared-information environment which could be accessed and audited by any member of the supply chain network. Sibintek has chosen Exonum, Bitfury’s permissioned blockchain framework, as the most suitable platform for such a solution.

In Sibintek’s solution all documents (product nomenclatures, orders, invoices, and payment reconciliations) and actions (order confirmation, shipment notification, etc.) used in this supply chain are subject to blockchain storing and timestamping. Only nomenclatures that are confirmed and marked as relevant can be used to form a supply order (preventing confusion later on if different vocabulary is used by one of the companies). After the order is confirmed by the supplier, smart contracts are automatically generated from the order documentation and are checked to ensure they contain no errors or discrepancies. All documents and confirmations within the system are then signed by each entity, using a digital signature protected by their corresponding private key. After the order has been delivered and accepted, the payment can be immediately executed by the respective accounting departments, without any additional effort of the counterparties involved. Besides the other benefits of vastly accelerated processing times for each entity, a decreased number of order returns/complaints, and a significant reduction in operational costs for documentation processes, their Exonum solution’s main advantage is the absence of reconciliations of mutual settlements. Payment reconciliation reports are generated automatically from the trusted environment with the data that all participants are agree with. Overall, they found that using blockchain helped increase overall supply chain reliability and transparency for all parties.

A common question we receive from potential partners is whether using blockchain for your supply chain operations is safe and reliable. Our recommendation is that you consider using extra security precautions when designing your private blockchain platform. Exonum, for example, has the option for you to “anchor” your blockchain to the Bitcoin Blockchain or another public blockchain. It’s designed to give users the best of both worlds: access to the benefits of using the public blockchain with added security and control over their data.

In summary: Supply chains that use blockchain can benefit from streamlined operations, increased trust and more cost-efficient operations, which can lead to substantial savings across the network. Gas stations and their supply chain networks stand to benefit substantially by making the switch. To learn more about using blockchain in your business, visit www.exonum.com.

Original source: https://exonum.com/blog/exonum-partner-highlight-oil-and-gas-industry/

This week, we have introduced “service lifecycles” into Exonum. This is one of the largest updates to Exonum to date and reflects on the growing conversation around business logic for blockchains.

We believe that software should be constantly evolving to adapt to the needs of users and its environment. Lehman and Belady pioneered this idea in their laws of software evolution — if a software does not adapt and improve, it will progressively become less satisfactory. In short: new functionalities are always needed to satisfy the growing needs of the customer, bugs are found and need to be fixed, suboptimal design/coding decisions should be periodically reviewed and updated — among other changes.

In light of this ethos, we have taken the time to review specifically how business logic is being used in Exonum and how it can be evolved securely with a service lifecycle without sacrificing performance, transparency or auditability.

A note: Business logic on blockchain is packaged into components often called “smart contracts.” We prefer to use the term “services,” which you will see throughout this article.

About Smart Contracts / Services

Smart contracts in many blockchains (for example, Ethereum) are immutable — once a smart contract is deployed on the blockchain, it cannot be modified. There are good arguments for smart contract immutability for public, permissionless blockchains. Immutability makes it easier to interact in low-trust environments. For example, a malfeasant smart contract author cannot replace smart contract code to scam its users. In the immutable smart contract model, ad-hoc upgrades can be implemented on the individual contract level, which is often the case for more the complex and modern Ethereum DApps.

However, the immutability requirement of these contracts means they must be flawless upon production. In contrast, in permissioned blockchains, flexibility, maintainability and keeping a competitive edge are all higher priorities. The trust brought by immutability can be achieved through other means such as auditability and transparency controls — while still allowing for the participants to evolve the system.

The following table illustrates differences between permissionless and permissioned chains in terms of features described above.

The unique requirements of permissioned blockchains form the foundation of the new service lifecycle in Exonum. We wanted to allow for the evolution of Exonum services using the same conceptual framework that would be applied to an ordinary web app (for example, data migrations). At the same time, Exonum keeps the service lifecycle and interaction with services safe and secure, and all lifecycle events fully transparent and auditable.

Services and Artifacts

To understand the service lifecycle in Exonum, one first needs to understand services. Exonum allows you to develop business logic in several programming languages, which is encapsulated in the concept of runtimes. How services are defined is fully determined by the runtime; the Exonum core logic uses runtimes as a proxy for all interactions with the services. Consider two primary runtimes supported by Exonum:

• In the Rust runtime, services are placed in crates (usually a single service per crate). A service needs to be linked with the runtime at the compile time in order to be deployable. In other words, Rust services are difficult to add on the fly; for a new service to be added after a blockchain is started, node binaries should be recompiled and swapped for each blockchain node.
• In the Java runtime, services are packaged into JARs, which can be dynamically added during a blockchain’s lifetime. The consequence is that Java services are slower.

Because of the runtime abstraction, it is possible to support new programming environments, and to define new runtime-specific abilities without changing a single line in the core framework! For example, both Rust and Java services have full-fledged REST APIs (Rust services may use the same tools to implement other HTTP-based interfaces, such as WebSockets), despite the core framework not having a single line of code dedicated towards it.

To allow for service representations that may vastly differ among runtimes, Exonum splits services and artifacts. Services represent instantiated business logic, and artifacts are sources which services may be instantiated from. Using an analogy from object-oriented programming, artifacts are like classes and services are instances of these classes. It is allowed (and desirable) to reuse artifacts among blockchains and on the same blockchain. Services instantiated from the same artifact have independent data and may have different configuration influencing service behavior, but their logic is shared by design.

Artifacts follow semantic versioning — a version, together with the artifact name, are built into the artifact ID. This allows your blockchain to reason about artifact compatibility much more productively. In fact, services may depend on other services and can use familiar caret or tilde requirements to check dependency compatibility, allowing for statements like “My service depends on exonum.Token@¹”. (Dependencies largely rely on ad-hoc mechanisms as of now; please see the discussion of future improvements at the end of the article.)

Basic Artifact Workflow

Exonum assumes that deploying an artifact could take a long time, for several reasons including network I/O (for example, downloading the corresponding package from Web or IPFS) and/or non-trivial computations (for example, compiling an artifact from sources). In the worst case, deployment may even result in a local failure (such as a compilation error).

Controlling deployment under these circumstances is a non-trivial task, and it likely doesn’t have a one-size-fits-all solution. Fortunately, the Exonum core is not responsible for deployment — instead, according to the separation of mechanism and policy design pattern, deployment and other lifecycle events are controlled by a fully customizable supervisor service[AO1] . While the core implements artifact deployment, it does not decide when to initiate deployment or when to consider it (un)successful; both these policies are defined in the supervisor. Since the supervisor is just an ordinary Exonum service, its commands to the core are fully auditable and are replicated onto all nodes in the blockchain network; the consensus algorithm itself guarantees that the lifecycle events are invoked in the same order on all nodes and have the agreed outcome. Additionally, since supervisor commands are usually given as a result of transaction processing, they are guaranteed to be properly authorized.

A typical supervisor implementation (such as the reference supervisor from the Exonum monorepo) will require the artifact deployment to be authorized by the blockchain administrators. The reference supervisor has two settings in this regard: in the simple setup, the deployment may be initiated by a single admin, while in the decentralized setup, a supermajority of more than two thirds of admins is required. Once the deployment is initiated, it will be performed in background by each node in the network. Nodes can then report the local deployment results via transactions to the supervisor service; based on these reports (and possibly the deployment deadline), the supervisor makes the final decision whether to consider the artifact deployed globally.

Once the decision to deploy an artifact is relayed to the core (that is, the deployment is committed), the node will block deployment if it has not finished locally; the node will not process any further blocks until it has the artifact deployed. The supervisor should be configured so that such “lags” occur only seldomly, since they hurt network liveness. For example, the supervisor may wait for confirmations from all validators (and possibly even auditors) and/or set a waiting window after receiving the last necessary confirmation.

The deployment workflow guarantees that the artifact is deployed for all nodes in the network by a definite blockchain height; commitment acts like a “thread join” for the deployment task. Thus, operations regarding the artifact after commitment will be treated in the same way by all nodes. One of such operations is service instantiations, which we cover below; without an artifact deployment being committed, it is impossible to instantiate services from it.

If the decision to deploy the artifact is not made, the nodes continue to operate as usual. Presently, a node will not unload the artifact (we plan to introduce this functionality in the nearest future) but will “forget” about it if restarted. In contrast, the node will re-deploy all committed artifacts on restart; this is expected to not take much time, since runtimes are encouraged to cache deployments.

The diagram below illustrates successful deployment workflow in a network with 3 validators and one auditing node. The deployment is initiated by a validator and then voted for by the other validators; in this example, the supervisor is configured so that the approval of all three validators is necessary to start deployment. After the last vote, all nodes (including the auditor) start deployment in the background. Once all validators have reported successful deployment result via a transaction to the supervisor, the supervisor commits to the deployment. Since the auditor has not finished deployment by this time, the auditor node brings deployment into foreground and blocks until it is completed.

Basic Service Workflow

Once a service artifact is deployed, it can be used to instantiate services. Like deployment, the supervisor controls instantiation (in particular, its authorization) and Exonum core executes it. In contrast to deployment, instantiation is fully synchronous — all long-running operations have been performed during previous deployment. The service artifact receives an instantiation request, which may contain service-specific configuration (such as the ticker and the decimal precision for a token service, or injection of service dependencies). If the artifact processes it normally, the service is considered instantiated. The artifact may return an error (for example, if the service is misconfigured, or if the required dependencies are not instantiated on the blockchain). In this case, the service is not considered instantiated.

A similar division of responsibilities between the core and the supervisor is used to stop an existing service instance. Once stopped, the service does not process transactions and in general does not interact with the outside world. Stopping services is not that useful on its own (except for marginal cases, for example, if a vulnerability was discovered in the service logic), but it serves as a stepping stone for the next lifecycle event — service migration.

Generally, migration in the Exonum context means updating the version of the artifact associated with an existing service instance. In the simplest case, migration just updates service logic (but not its data). Some scenarios include:

• patching a bug in transaction processing logic
• adding a new transaction to the service
• removing the transaction from the service (naturally, this is a breaking change; it could be necessary, e.g., if the transaction logic is irredeemably flawed)
• changing runtime-specific service APIs, such as HTTP endpoints for Rust or Java services

In this case, the workflow is simple:

1. Deploy a newer service artifact.
2. Stop the service.
3. Assign the service to the newer artifact.
4. Resume the service.

Since steps 2–4 do not involve long-running or asynchronous tasks, they can be performed very quickly (in consecutive blocks — order of seconds in terms of real time). A supervisor implementation can even batch them as a single operation, needing only a single authorization.

Data Migrations

A more complex — and more interesting — case for the service lifecycle involves data migration. Data migration is ubiquitous in real-world applications, and we think that blockchain business logic should be no different. While the use of Protobuf as the recommended serialization format may allow Exonum services to avoid data migrations in some cases, sooner or later migration becomes unavoidable — and Exonum provides a well-defined, safe interface for it.

The diagram below shows the possible states of Exonum services and transitions between them:

Like artifact deployment, data migrations are performed in background by special pieces of business logic — migration scripts. Since the Exonum storage engine is non-relational, migration scripts are written using the same programming languages the main service logic is written in. A script essentially creates data collections (lists, maps, sets) for the new service version and fills them based on the existing service data. The migration is non-destructive; the new data is placed in a separate migration namespace and cannot be accessed by active services until the migration is complete. Retaining data from the old service version does not require any actions in the script. If an old collection needs to be removed, it is marked with a tombstone (the term is taken from RocksDB and other log-structured storages).

Unlike the simple case, it is instrumental that the service instance is stopped during data migration. If a service remains active, its data may change during migration, which would likely result in logically inconsistent migrated data. In fact, stopping the service in the simple case is done for the uniformity with the data migration case (as we’ve discussed, the service downtime in this case may be minimal).

Exonum provides tools to ensure that migration scripts are fault-tolerant (that is, can resume if a node is stopped while the script is executing) and at the same time make progress. A migration script can dump changes to the database; to store information about script progress, scripts may use a special temporary storage namespace (a scratchpad). Scratchpads are especially powerful when used within persistent iterators — iterators over a collection in the storage, which automatically remember their current position.

To ensure that migrated data is the same on all nodes, Exonum utilizes automatic state aggregation. The contents of all top-level hashable collections within the migration namespace is aggregated into a single hash value, which thus commits to the entire migrated data (or its important part, anyway; like with the ordinary data, Exonum does not forcemerklization of the entire blockchain state, recognizing that there may be reasons against it, such as performance or legal). If the resulting migration hashes are the same on all nodes in the network, we can decisively infer that the migration data is the same, too.

In summary, the data migration workflow is organized as follows:

1. Precondition: Ensure that the service is stopped.
2. Fetch a migration script from the newer service artifact. The artifact may return a special no-op script (this is how simple service migrations are organized). The artifact may also signal an error (for example, the initial service version is too old, so the artifact doesn’t know how to migrate from it).
3. Start executing the migration script in the background.
4. Once the script is finished locally, remember the aggregated migration hash.
5. Using the supervisor, report local migration hash and determine if there is consensus among nodes as to the migration outcome. (The exact policy depends on the supervisor.)
6. If there is consensus among nodes, commit the migration outcome. At this point, nodes that did not finish the script locally will bring it to the foreground and block until it is completed. If the script is completed with a different outcome, the node stops. (The node administrator can then retry remove roll back the migration locally, so it is retried again.)
7. Replace old service data with new one. At this point, the service returns to the “stopped” status. The service may be resumed, or more migration scripts may be applied to it to further update the data layout.

If nodes do not achieve consensus on step 6, or the supervisor finds another reason to consider the migration failed, the migrated data is removed, and the service returns to the “stopped” status.

The diagram below illustrates workflow for one successful and one unsuccessful data migration. The supervisor in this example is configured to start a migration when authorized by a single validator (that is, no voting is required). The first migration is finished successfully, and the service data is replaced with the newer version as a result. The second migration fails because two validators report different migration outcomes, and the supervisor is configured to command migration rollback in this case. The second migration script is still being executed on validator #3 when this happens. This script is aborted since its outcome will not be used.

The described data migration workflow is modular and flexible:

• It is possible to perform several data migrations in a row without resuming a service. This mirrors a common workflow with migrations in relational databases / ORMs, in which migration scripts are linearly ordered and each script is applied to any service instance exactly once.
• It is possible to perform one or more data migrations and then decide to resume a service with an intermediate version (for example, because the latest data migration has failed).

Conclusion

The Exonum service lifecycle allows you to safely evolve the business logic deployed on your Exonum blockchain. And because the lifecycle policy is implemented outside the Exonum core, the policy itself can be flexibly adjusted during blockchain evolution. And since interactions with Exonum services are transparent and auditable, the lifecycle is guaranteed to satisfy these qualities by design. Finally, the Exonum storage engine allows to fearlessly migrate service data, ensuring its agreement among all nodes in the network. (The design of the storage engine is a good subject for another article.)

We are continuing to streamline and improve the service lifecycle as well — here are a few improvements in our Exonum roadmap:

• Support a more complete artifact lifecycle, such as unloading artifacts
• Support more service states, such as frozen states (a state in which a service cannot change its data, but it can still interact with the external world via read-only interfaces, such as HTTP API)
• Support notion of service dependencies
• Provide an interface description language for services.

Our team is working hard to integrate these Exonum features into Exonum Enterprise, our Blockchain-as-a-Service platform that allows any business to launch a blockchain in just 5 minutes. You can request a trial of Exonum Enterprise today on our website: https://exonum.com/enterprise

We are happy to announce that we have finalized our Exonum 1.0 release candidate (in preparation for the full release of Exonum 1.0). In this blog, we’ll talk through the features of the Exonum 1.0 release candidate as well as the changes from Exonum version 0.12.

Introducing the Service Lifecycle

The core feature introduced in this release is a logically complete service lifecycle. This means it is now possible to add new services to the blockchain during blockchain operation, stop or upgrade existing service instances, and even perform data migration in background in order to change your data layout to the one required by the new service version.

Some examples of the benefits of using the service lifecycle include allowing you to define new APIs to interact with the service (in form of either transactions or HTTP endpoints); add new parameters to existing APIs; or remove your existing APIs altogether. (We also now offer semantic versioning to help determine the compatibility of this change). This upgradeability brings Exonum services closer to traditional applications, enabling you to build a more durable and agile blockchain infrastructure.

All service lifecycle events are managed by the Exonum supervisor service, which brings the following benefits:

• Service state is agreed among all nodes in the network by the virtue of the consensus algorithm.
• Service administration is fully auditable and properly authorized.
• Service administration can be fine-tuned according to the use case. For example, in a small blockchain, lifecycle events could be authorized by a single administrator, while in a consortium network their authorization could require an agreement of a (super)majority of administrators.
• Supervisor logic itself is fully upgradeable.

The reference supervisor implementation can be found in the exonum-supervisor crate.

We will be sharing more details on our blog about the service lifecycle soon.

Data Migrations

As a part of this service upgrade workflow, Exonum now supports background (aka asynchronous) data migrations. This allows you to migrate data from one schema to the other without needing to stop a node or your blockchain network; instead, the whole process is performed in the background while the node continues processing transactions and other requests.

Data migrations are performed with the help of migration scripts, which can be written in the same languages that Exonum supports (currently Rust — Java migration support is in progress). The migration workflow defined in the Exonum core ensures that the outcome of a data migration is the same on all blockchain nodes, and that migration finishes by a certain blockchain height. A node will bring the migration script into the foreground if it hasn’t managed to finish it in time, and block consensus logic until it is completed.

Relevant migration support was also added to the storage backend (MerkleDB) and can be used independently of Exonum.

Splitting up Crates

The crate structure was significantly refactored in this version of Exonum. The node logic, Rust runtime, HTTP API wrapper, explorer endpoints and system endpoints (all of which were previously parts of the main crate) have now been moved to separate crates. This provides greater flexibility for core and service developers and allows you to evolve downstream crates independently of the core.

An additional benefit of this change is that both the development and testing of Rust services can now be performed without relying on node-specific dependencies, such as tokio providing the event loop or snow implementing the Noise protocol for encrypting communication channels among nodes. As a result, compilation times are sped up, and the build artifacts become slimmer.

MerkleDB Updates

MerkleDB now automatically aggregates the storage state, which was previously performed manually by the core. State aggregation produces a single Merkelized map (the state aggregator), the keys of which are the names of the Merkelized indexes in the database, and values are the corresponding index hashes. The hash of the aggregator thus commits to the Merkelized part of the database state and can be used to prove the existence or absence of elements in any Merkelized index.

With automated aggregation, all aggregation logic previously present in the core and runtimes was removed. The new approach requires zero effort from the service developers — it is enough to declare the data schema, and all Merkelized indexes will be added to aggregation automatically!

Speaking of data schemas, the Rust services can now utilize a more intuitive declarative format to declare them. For example:

Indexes or index groups can then be simply accessed via the corresponding properties of the Schema.

Compared to 0.12 release, MerkleDB provides new tools to enforce database access restrictions. First, the access can be restricted to a namespace (this is used by Exonum to provide read/write restrictions for services during their execution — a service should not be able to write to the schema of another service). Besides the general namespaces, MerkleDB provides two types of namespaces for migrations — Migration for storing new data and Scratchpad for temporary data which is discarded after a migration is flushed to the general storage or rolled back.

Besides namespaces, MerkleDB now provides separation between read-only and read-write access, which is enforced on the level of the Rust type system. This helps you to write safer code and eliminates potential unexpected behavior of services.

To assist in developing bindings in other programming languages, MerkleDB now supports generic database accesses and shares iterator types among all indexes. Any kind of access can be converted to a generic access, allowing to decouple and/or reuse much code in the “native” part of a binding.

Service Interfaces in Rust

To make service development easier (and more aesthetically pleasing), service interfaces were reworked in the Rust runtime. Now they are represented by the ordinary Rust traits, like this one from the cryptocurrency tutorial service:

Transactions are now plain data types, and business logic of a service is defined by implementing the service interface trait, which is idiomatic from the Rust perspective and easier for general comprehension.

Besides the service implementation, any interface trait is automatically implemented for an assortment of stub types. For example, the KeyPair stub allows to generate signed transactions to the service — it’s enough to import the interface trait to the scope:

Moreover, new stub types can be defined in the user code! For example, the middleware service implements stubs in order to create checked calls, that is transactions, which are only routed if the target service has a specified name and satisfies a version requirement.

We will publish an article about how stubs work soon. In the meantime, you can read a literary code explainer in the stubs module of the Rust runtime crate.

Other Changes

Besides functional changes, this new release improves user experience and prepares Exonum APIs for future compatibility in the following ways:

• Protobuf declarations in the core and upstream crates have been reviewed, documented and structured according to their packages.
• Data types in the core and upstream crates have been made non-exhaustive.
• Experimental or unstable APIs have been marked as such in the documentation or hidden from the documentation.

You can find a more detailed list of changes in CHANGELOG.md.

Coming Soon: Exonum 1.0

We are working to release Exonum 1.0 by the end of February. No substantial functional changes are expected before then. At the same time, there may be breaking changes to the APIs documented as unstable (for example, data migration). Another kind of potential breaking changes is related to updating core dependencies. Right now, Exonum core targets compatibility with Rust 1.36 due to the problems of compiling the Java runtime with the more recent versions. If this restriction is lifted, some core dependencies will be updated, which could result in breaking changes (at least technically).

Along with release preparations, we have updated our technical roadmap. It can be found on the Exonum website.

Our team is working hard to integrate these Exonum features into Exonum Enterprise, our Blockchain-as-a-Service platform that allows any business to launch a blockchain in just 5 minutes. You can request a trial of Exonum Enterprise today on our website: https://exonum.com/enterprise

In the past three years, our dedicated developer community (now numbering more than 200 on GitHub!) has seen us launch 13 different versions of Exonum — beginning with Exonum core 0.1 and ending with today’s current version, Exonum core 0.13.

Recently, one of our developers asked us why we chose owls for our version names. We knew when we launched Exonum that we would be working hard to frequently issue new updates and create new features to make it an even more robust blockchain platform. We wanted to find a creative way to share our story while differentiating between these different versions.

We decided to use owls in our version names because our logo was initially inspired by the image of owl. We chose owls because they symbolize intelligence and wisdom and, in the ecosystem, they keep the forest clear of pests. We believe that our blockchain platform does something similar. Blockchain makes business operations more streamlined and intelligent, and our proprietary design protects businesses from malicious actors while maintaining secure operations.

This is how our Exonum “parliament of owls” was born! In July 2017, our very first version of Exonum, Exonum core 0.1 “Spectacled owl” was released, designed to help enterprises and governments run their own blockchain-based applications secured by our custom Byzantine Fault Tolerant algorithm and anchoring to the Bitcoin Blockchain.

Since then, we’ve launched:

Exonum core 0.2 “Akun Eagle-owl”

Exonum core 0.3 “Barn Owl”

Exonum core 0.4 “Chestnut Owlet”

Exonum core 0.5 “Dusky Eagle-Owl”

Exonum core 0.6 “Elf Owl”

Exonum core 0.7 “Fulvous Owl”

Exonum core 0.8 “Great Grey Owl”

Exonum core 0.9 “Halmahera Boobook”

Exonum core 0.10 “Indian Scops-Owl”

Exonum core 0.11 “Jungle Boobook”

Exonum core 0.12 “Koepcke’s Screech-Owl”

Exonum core 0.13 “Little Owl”

Coming up soon is Exonum core 1.0 “Mottled Owl” — we’ll have more to share on this soon!

In June 2018, we also made Exonum available to Java developers through Exonum Java Binding. Our team was very familiar with the Java dev communities around the world and wanted to pay homage to their incredible work — so we named each Java binding update after a type of “java,” or coffee! Our first release was Exonum Java 0.1 “Affogato”; in December 2019 we launched Exonum Java 0.9 “Galao coffee.”

We’re looking forward to our upcoming Exonum core and Java releases — if you want to use Exonum Core or Java binding in your operation, you can download them both from GitHub or contact us at contact@exonum.com. Be sure to follow us on Twitter to learn more about the upcoming release of Exonum 1.0!

We recently released Exonum v0.13, an update dedicated to introducing dynamic new services into the framework. The inclusion of dynamic services further signals Exonum’s suitability for large-scale blockchain projects.

Here are some of the biggest changes. The full list is available in our Changelog.

Dynamic Services:

The inclusion of “dynamic services” means that new services can be added into an Exonum network after the network has already been launched, without requiring the restart and recompilation of the network nodes. You can also start any number of instances of the same service.

Exonum v0.13 now enables users to include different environments of code execution as well. These environments are called runtimes, and they can be written in different programming languages. Our team provides runtimes in Rust and Java. This means that one Exonum-based project can now simultaneously contain both Rust and Java services. Still, service developers may introduce custom runtimes into their Exonum-based projects.

It is important to note that the dynamism of Rust services remains restricted in the current implementation (as compared to Java). Java offers the possibility to introduce new service types called “artifacts” into the system at any point in the system’s operation. In Rust, all service artifacts that the project may require should be included in the binary file from the system start, but they can be activated later as needed. We expect to soon roll out Exonum Java Binding v0.9, which will support full-fledged dynamic services.

We introduce new `exonum-supervisor` service for blockchain developers. It is now responsible for starting and stopping services in Exonum. It also replaces the outdated Configuration service and is used for configuring dynamic services.

The “Migration Guide” section of the Changelog contains a list of backward-incompatible changes that Exonum service developers should take into account when upgrading their projects. For example, we had to pay additional attention to security before introducing dynamic services in Exonum. To protect active services from harm caused by new business logic of later-added services, we isolated access to the database for services. Exonum services do not have write access to each other or the blockchain schema.

Finally, we have optimized the structure of our services to improve the usability of the Exonum framework, including making the schema of Exonum MerkleDB declarative. Service developers do not need to describe the behavior of the schema and implement trivial methods for interacting with the indices (tables) of the schema anymore. This is covered “under the hood” already.

Additionally, both the service and consensus keys of validators are now derived from a single master key. There is only one file with a key now secured by a single password. The consensus and service keys derive from the master key as the leaves of a tree. To run the node now, it is sufficient to do the following:

`cargo run -- run -d 0/db/ -c 0/node.toml --master-key-pass pass:123`

We also managed to replace the nested (or recursive) proofs of MerkleDB `ListProof` and `MapProof` indices for flat proofs. These proofs are quicker and easier to verify, which makes the process of proof validation easier for the client.

The present release still contains a number of important improvements and optimizations that altogether facilitate release of Exonum v1.0, expected in the first quarter of 2020.

As usual, we’d be happy to answer any questions you have. And we will be happy to assist you in upgrading your projects to a new version of the Exonum framework. Please contact us anytime through our Gitter channels.

All the latest news on Exonum is available on our Medium page — please subscribe to stay updated!

Last week we released version 0.12 of our Exonum blockchain framework. This version incorporates several usability improvements, functionality normalizations, and groundwork for a unique independent storage for your blockchain projects. More details are included below, and the complete list of updates is available in our Changelog.

Simplified Bootstrapping Procedure

In the previous version of the Exonum platform (v.0.11), we made a system security improvement that resulted in the time consensus and service keys of the nodes being stored in separate files. Each file can be secured by a password at the administrator’s discretion. Such security measures prevent thefts of the secret keys of the nodes by third parties. At the same time, we expanded the “generate-config” command of the bootstrapping procedure. In version 0.11, the routes to both files had to be mentioned in the options of the command. An example of such a command is here:

To optimize the flow of the project set-up, we placed all the configuration files (namely, the files with the public and secret node parameters and the two files with keys in question) into a separate folder. So only the name of the folder is used now when addressing the files within the “generate-config” command:

The mentioned principle also reduced the number of arguments for the “finalize” stage of the procedure. The routes to the public configuration files of each node are now organized into one as follows:

Exonum MerkleDB Storage

In this release, we have completed the first step in our development of an authentic storage for blockchain projects. Exonum MerkleDB is a persistent storage solution based on RocksDB. It provides APIs to work with Merkelized data structures characteristic of blockchain projects.

MerkleDB is an object database. The main MerkleDB objects internally are wrappers around key-value stores; they perform the same role as tables in relational databases (RDBMSs).These stores also meet specific requirements that blockchains have for data management (for example, building cryptographic proofs for checking presence/absence of particular information in the blockchain).

In the low level storage, all data and its keys are stored as a sequence of bytes in a single table called the “column family” in terms of RocksDB engine. It is also possible to apply different database engines at the developer’s discretion.

You can find the detailed description of the MerkleDB module in our documentation.

With regards to Exonum architecture, the storage module is now extracted into a separate crate. See the complete set of object stores we currently ship with Exonum.

We also advanced our data security while upgrading our storage. The hashing rules for the leaf and branch nodes of our “ProofList” objects now support the requirements of the Transparency Certificatepromoted by Google. Adoption of such a highly rated standard, in our opinion, should add to attractiveness of the framework as a safe and secure instrument.

Exonum API Extension

Beginning in Exonum v.0.10, you could connect to nodes through the REST API as well as through WebSockets. The idea here was that light clients could subscribe to network events and obtain information on the blockchain from the nodes. Previously, the light client could only be notified if there was a new block event. In other words, if using the socket, the client was notified with every new block accepted in the network.

In the present release, we extended the WebSocket API to also support subscription to transaction events. This means that you (through the light client) can be notified every time a transaction of a specified service is committed to the blockchain.

We also enabled multiple subscriptions with filters. You can subscribe simultaneously to various types of events (for example, to new transactions and blocks, and/or to transactions from various services and of different types).

We also extended the API to enable you to send transactions to the blockchain via WebSocket. This functionality is similar to the one underpinning the Exonum Blockchain Explorer. We expect this will be more suitable in some cases compared to the old workflow.

See our documentation for the full list of available sockets.

If you are developing on Exonum, you should also pay attention to the changes in the Exonum REST API. A couple of new endpoints were added to make node management more flexible and some old endpoints were updated to deliver the requested information more efficiently.

For any questions, or if you need assistance with our new features, feel free to contact us through our Gitter channels.

Follow our page to stay up-to-date with our further developments and news.

We recently released Java Binding 0.7, which includes several new features, in response to community requests and our own improvements. The full list of changes and updates can be found in our Changelog. In this article, we highlight some of the outstanding new features in versions 0.6 and 0.7.

New Features of Java Binding 0.6

The first new feature, and a great step towards improving usability, is packaging and shipping Exonum Java application as a single archive. The application contains all the necessary dependencies to run and develop Java services on Exonum. It no longer requires the installation of the Rust compiler and build of Exonum Java. By eliminating these steps, it will be even easier to launch your custom blockchain solutions in Java! Our Installation Guide demonstrates how to use this new feature.

The second important usability improvement is the ability to launch multiple Java services within one Exonum project (previously, only one Java service could be launched in a project). The latest version of Exonum Java allows you to upload as many Java service packages into the project as you need at the time of network configuration. The nodes will catch those service files and will be configured at the start with all the provided Java services.

Learn how to install several Java services at once in our step-by-step guide.

Reminder: The Exonum Java application also features three built-in Exonum core services — configuration service, time service and Bitcoin anchoring service — that are written in Rust.

New Features of Java Binding 0.7

We are proud to announce that Java services have become considerably faster with our upgraded transaction processing mechanism. In earlier versions, Java services were 2–3 times slower compared to equivalent Rust services. We’ve now reduced this rate to 15–20 percent, and we expect more performance improvements in upcoming versions.

We also have implemented the Exonum Testkit module specifically for Java services. This Testkit provides the same testing possibilities as the one for Rust services. It allows for Java developers to execute integration tests on Java service operations — namely transaction processing and read requests.

To make this possible, the Testkit emulates a blockchain network so developers can test transaction business logic in Java. Also, as its Rust analogue, the Testkit recreates the operation of a single node only (either a validator or an auditor). This means the emulated network operates in a synchronous environment, without consensus and network operations involved. For your convenience, we have prepared some detailed testing examples in our documentation.

The Testkit is shipped together with the Exonum Java application package. If you are new to Exonum and plan to install our Java instance for the first time, you can apply the new Testkit right out of the box!

Exonum Java Light Client v0.3 Is Available

Exonum Java releases 0.6 and 0.7 included upgrades for our Exonum Java Light Client. First, we improved the mechanism for retrieving blocks from the blockchain according to user’s requests. The full list of method updates is available in our Changelog.

We have also implemented the possibility to add custom prefixes to the endpoint addresses for routing API requests. This provides some flexibility for the cases when a middleware, e.g. proxy, is used in the network setup.

Thanks for reading! As always, we are available to answer your questions and requests related to Exonum and/or Java Binding in our Gitter channels.

A note: The new features of Exonum Java Binding tool listed here are based on Exonum Core 0.11. Exonum Java Light client 0.3 is compatible with Exonum service based on Exonum Core 0.11 and Exonum Java Binding 0.6–0.7.

Last month, Facebook announced it had launched the Libra Association alongside several corporate partners such as VISA, Mastercard and Uber. The association’s goal is to launch the Libra Blockchain, which would “serve as a solid foundation for financial services, including a new global currency, which could meet the daily financial needs of billions of people” (source). Here we take a closer look at the technical documentation of Libra to evaluate whether its objectives are realistic, as well as to determine if some of the widespread criticism of the project is warranted.

Overall, we found many reasons to be enthusiastic about Libra, including:

- Libra Core is written in Rust, which is one of the most safe and efficient programming languages. Rust provides comprehensive execution safety and predictable resource usage.

- Libra uses several well-tested blockchain attributes, such as Merkle trees, secure cryptographic libraries and robust consensus protocols.

- Libra has taken upon itself to create a new smart-contract language, Move, which is an ambitious undertaking but also avoids the well-documented issues of other smart contract languages like Solidity.

There is a clear difference in the ideal use cases for Libra and the Exonum™ platform, which is due to the different approaches in defining business-logic. From our review we find that Exonum is better suited for private systems, whereas Libra is designed as a (initially more centralized) public blockchain. Exonum has more customizable options (like cryptographic module replacement), access to time-tested and well-known libraries in Rust, C++ and Java, and more instruments for storing and processing data. For these reasons, it seems Exonum is a better choice for those interested in designing a private application.

Below we get into more detail about these topics and share some comparisons to Exonum’s design structure to help illustrate the similarities and differences between the two blockchain platforms.

Programming Language — Rust

The Exonum team has been building blockchain solutions in Rust since 2016, so it was interesting for us to review Libra Core, which is written entirely in Rust. It consists of 85,000 lines of Rust code split into 607 files within a tidy modules structure. We can say that the source code is well-documented and readable, which will help developers build products for Libra more easily. The installation and usage instructions are clear and concise. It uses the most recent Rust features such as async syntax, but it is apparent that it is primarily focused on utilizing a stable Rust channel — therefore, the code leaves little opportunity for experiments. Libra uses common Rust libraries such as lazy_static, tokio, clap, failure, and has also implemented necessary features, including wrapping the existing code with helpers or using repository forks (in case of the rocksdb library). The Libra team has also implemented their own async actors (instead of using the ones from the actix library).

Processing of Transactions

Any operation involving data on a blockchain is done through a transaction. These transactions are special messages, which contain data and instructions on how to process this data. For example, to add a new user on an Exonum blockchain, we send a transaction called “CreateUser” with the user’s name and his/her public key. After processing this transaction on all the blockchain nodes, our distributed database now contains information about this user. Everyone will be able to review the list of all committed transactions to find when and how this data was added into the blockchain.

Libra transactions also follow this model and work in a similar way to Ethereum transactions. All transactions on Libra are “Protobuf” messages and contain some Move bytecode (we talk about Move later on), arguments for this bytecode program, a public key and an address of the sender. The transactions are also signed with the sender’s private key. To restrict the maximum complexity of transaction business-logic, the Libra Blockchain requires transactions to use gas. Gas is an abstract virtual currency that allows the sender to estimate the “cost” of the bytecode program. Each bytecode operation costs certain amount of gas, and every transaction contains the maximum available gas amount. So, if the program spends all the gas (i.e. by entering the infinite loop), its execution will be stopped. This does mean that anyone can add their own business logic to their transactions, but they will have to use gas for it. For this reason, Libra is efficient for building services for end users. However, the need to pay gas for every transaction can be an issue for large-scale infrastructure projects.

Exonum transactions are Protobuf messages as well, but Exonum transactions contain only parameters and some internal metainformation. All the business-logic is stored in the source code of the blockchain nodes, not inside the transaction. This means Exonum transactions are smaller (in terms of data) but it’s more difficult to update the business-logic.

To prevent a transactions replay attack, Libra uses a simple feature called “Sequence number.” The sequence number is the number of transactions any account sends, and each account keeps its own sequence record. This number is included in every transaction message. The Libra blockchain will not execute the transaction if the Sequence number of the transaction is less than the Sequence number of the sender’s account. In this way, the attacker will be unable to submit the same transaction twice.

Exonum solves the issue of replay attacks by discarding transactions already included into the Exonum blockchain. The framework determines if there are duplicate transactions by the hash of the transaction at the verification step (before broadcasting to other blockchain nodes). To do this, it is necessary to have an additional field in our design to distinguish among transactions with the same set of parameters. We call this field “seed,” and we often use either the current time or a random number as its value.

Another difference is the use of public keys. In Libra, each transaction includes both the public key of the sender as well as his/her address (which is just the hash of the public key). We believe the reason for this is due to the message format they chose. Essentially, they have built a “layer cake” for their transactions, with the payload wrapped in an integrity guarantee — the public key and the cryptographic signature. In Exonum, we use a similar message format, but only the public key is stored in the transaction.

When it comes to transaction design, Exonum and Libra’s concepts look very similar, but there is a fundamental difference in the approaches. On the basis of our review, Exonum’s design is more suitable for private systems where the business logic can be determined only by authorized participants. The typical examples are various state registries, e-voting systems, infrastructure parts of the Internet like ICANN. One of the major advantages of this design is that you don’t need to pay for each transaction. In contrast, Libra is designed as a more centralized public blockchain, with each user writing their own business logic and paying accordingly.

Database Design

The Libra blockchain stores all its data inside a distributed persistent key-value database built on top of the well-known RocksDB library, called LibraDB. The blockchain is represented as a Merkle tree of transactions. Each “leaf” of such a tree is an executed transaction. Merkle trees enable the blockchain to easily handle hashing and comparing and synchronizing data, but it also offers a powerful feature called cryptographic proofs. Merkle trees allow users to acquire a cryptographic proof for every piece of data inside the blockchain without downloading the whole database. LibraDB is also smart with its storage — it effectively

utilizes disk storage by compressing the tree with different methods. For example, each node has 16 children (instead of two) and long branches are represented with a single node.

Exonum has a similar database structure called MerkleDB, built on top of RocksDB as well. Like Libra, Exonum has cryptographic proofs thanks to its Merkle tree design, but in addition MerkleDB has a more diverse set of data structures and a more complex hierarchical structure of the storage itself. It allows for the developer to choose the right tool for every possible task and effectively solve various programming problems while developing large applications with complicated business-logic.

Cryptographic Libraries

Both Libra and Exonum use the same cryptographic algorithms and libraries, namely SHA-3 for hashes, elliptic curves Ed25519 for signatures, and all communications between the nodes are encrypted with the Noise Protocol. It is important to note that the current cryptographic module of Libra is subject to change. The new cryptographic module, available in the Libra source code, utilizes traits of Rust for a more flexible API. Exonum’s cryptographic model is secure, but it is more easily replaceable depending on client requirements (which do vary across regions).

In comparison, Libra has more freedom to determine its cryptographic module as it is not dependent on the design needs of clients. The Libra team chose the most standard and well-known algorithms, which seems to indicate they aim for an effective and safe implementation (rather than the “right” one, which again can vary greatly).

Consensus Mechanism

Both Libra and Exonum use variations of a well-known practical Byzantine fault tolerant consensus algorithm based on the Tendermint and HotStuff protocols. These algorithms offer strong performance, including the ability to handle thousands of transactions per second; and they are robust and do not require any excessive computational power. However, these algorithms work best with a limited number of blockchain nodes. Libra’s goal is to have 100 nodes processing about 1,000 transactions per second; the baseline goal of an Exonum blockchain is to offer 5,000 transactions per second between four and 16 nodes. This is possible due to Exonum’s light client and use of cryptographic proofs, which means users do not have to run their own node to trust the blockchain.

Internally, the consensus modules of both Exonum and Libra are implemented in an async manner with the tokio library. Libra, as a much younger project, uses the most recent and powerful Rust features including async syntax. It makes the consensus implementation code more readable and maintainable.

Smart Contracts

Move is a program language developed by the Libra team and is used for smart-contracts on the Libra blockchain. The language is currently noted as a work-in-progress by the Libra Association. The syntax and IR-bytecode are not stabilized yet and may be altered soon. There is no information yet about availability of Move for third party programmers. For now, Move is used for the internal business-logic of the blockchain, including the use of the Libra coin and recording user accounts.

From our review, Move is a simple language and translates to the IR-bytecode almost line-by-line. This bytecode is executed on the Move virtual machine. The virtual machine isolates the smart-contracts and uses the concept of gas (as mentioned above). Move will probably evolve in a more high-level language in the future, as we expect more instruments for developing, debugging and testing of the smart-contracts will be built.

The syntax and semantics of the Move language are definitely inspired by Rust. Like Rust, Move also controls the mutability of references, and uses the same rules for ownership and borrowing as Rust does. Unlike in Rust, however, these rules are controlled on the bytecode level, not at the compiler level. Move also has additional requirements that Rust does not have. For example, all the variables must be declared before usage at the beginning of the function scope.

The current (initial) version of Move supports functions, modules and structures with some pattern-matching abilities.

Internally, Move bytecode is similar to LLVM IR-bytecode, which definitely opens up wide performance optimization opportunities.

A note: We will continue to review the developments of the Libra blockchain and Move programming language and will issue any updates here.

The Exonum™ platform is the leading open-source framework for building blockchain solutions. Critical to its efficiency and security is our custom consensus algorithm that is Byzantine fault tolerant and works without requiring your blockchain to ‘mine’ blocks.

Here’s what sets our consensus algorithm apart.

Solving for a Consensus Algorithm

A blockchain is a digital distributed ledger of data, organized as a chain of blocks. Adding a new block to the chain is possible only if: a) the member nodes of the network reach an agreement as to inclusion of the same suggested block; and b) the suggested block is accurate.

To make this decision, member nodes need rules to follow. The rules would also allow network administrators to assess the activity of the member nodes.

In short: A consensus algorithm is needed to ensure that peer-to-peer network members can reach a common opinion on the new ledger state (In other words, reach a joint decision on the contents of the new block in the blockchain).

The consensus algorithm that powers the Bitcoin Blockchain is the Proof-of-Work (PoW) algorithm. Here, the complex computational work to determine a new block that the participant (miner) does, and the reward he receives for it, guarantees the correctness of the current state of the network. The PoW algorithm gives economic assurance that the system works as expected and predictably. The next main alternative to PoW, Proof-of-Stake, and their hybrid forms (Proof-of-Activity, Proof-of-Burn, Proof-of-Capacity, and others) also provide an opportunity to reach a consensus by investing and / or spending the participants’ own material resources. In such conditions, it is believed that unfair activity becomes at least extremely expensive, if not absolutely unprofitable.

However, despite the fact that Bitcoin is considered the most reliable distributed systems existing today, neither it nor its alternatives essentially solve the problem of the Byzantine node behavior. The Byzantine generals’ story is well known to everyone, so we won’t explain it here. For the purpose of this article, we mean that we tried to find a solution for any “Byzantine node behavior,” i.e. any activity that is intentionally malicious and aimed at disrupting or completely halting the operation of the consensus algorithm.

When addressing this issue in private blockchain networks, you must formulate the properties of consensus mathematically. This is done by creating strict rules of behavior in the network (in comparison to PoW and other economically grounded consensus algorithms, which operate with very few almost spontaneously. Some of these mathematically grounded consensuses (in particular, the Byzantine fault tolerant consensus (BFT)) have a clear algorithm that selects a leader node that makes a new block proposal.

A consensus algorithm must also take into account common problems that can occur in the blockchain, namely: nodes losing connection with the network (communication failure between the network participants), nodes not operating at an arbitrary point in time (disconnect), and when nodes work arbitrarily and / or incorrectly. All of these issues describe “Byzantine behavior” of the node. In addition, the maximum allowable number of anomalies that do not affect the adoption of a new block in the blockchain depends on the properties of the network and the its processors / nodes. These properties include, for example, the synchronization of communication between the nodes in the network.

Classical studies of the 1980s showed that, to achieve stable operation of the consensus in the asynchronous network that contains nodes with Byzantine behavior, is impossible. Honest members of the network will not be able to tell a Byzantine node from a node that is simply inactive (the so-called `FLP Impossibility`). To ensure the stability of the system to the above-mentioned failures, it must be at least partially synchronous. At the same time, to work correctly, the consensus algorithm should have at least the following properties:

- Liveness — the ability at any time to accept a new block into the chain within a finite period of time. In other words, each node will ultimately make a decision about the next block;

- Consistency or absence-of-fork — identity of the base state on all the nodes in the network. In other words, in the end, the decision of all the honest nodes about the next block will be identical.

Another property gains importance if we speak of the consensus algorithms as applied specifically to blockchains. This is censorship resistance, which means absence of the censorship of transactions, which is a specific characteristic of consensus in blockchains. It prevents the nodes from intentionally selecting only certain transactions for the blocks, leaving other transactions in the pool of unconfirmed transactions.

Based on the results of the same research, for a network with a known number of nodes, the most optimal is the model of an algorithm that is resistant to the Byzantine node behavior. In the network with partially synchronous nodes this model tolerates up to 1/3 of the malicious (Byzantine) members in the network.

Given our goals and objectives for the Exonum framework, we elected this model to form our consensus algorithm.

In summary — we designed our consensus algorithm to be able to determine the behavior of each honest member of the network under conditions in which the majority of the members behave honestly, and some may behave in an arbitrary manner.

We needed to design a reaction scheme for an honest node to incoming events (messages from other nodes). Such reactions include sending an appropriate response message or “realizing” that a certain decision has already been made by the network. This is what determines the day to day operation of the blockchain.

A Critical Threshold for Achieving a Consensus: 2/3 vs. 1/3 Votes

The basic operation of a consensus algorithm has 2 steps. First, some validator node (a node that participates in consensus) makes a new block proposal and broadcasts it among all the nodes in the network. Second, the remaining validator nodes vote (“for” / “against”) for this proposal.

The moment one of the nodes first receives a certain number of identical reactions from the other validators, the decision becomes valid for the entire network.

The closest analogy to this process in the real world is the situation with presidential elections. The choice is made between two candidates. There is some finite number of voters, of which some may vote for one candidate, and some for another.

Among the non-Byzantine systems (for example, Cassandra), the decision is considered to be made when a proposal gains more than 50% of the votes. Since each member votes only once, the remaining votes are not counted. It is worth mentioning that since the number of votes coincides with the number of participants in the network, in this case there is no need to identify the voters.

However, in the Byzantine systems the situation is different. Here, voting is open, i.e. members’ identities are known to everyone. However, as soon as we assume that some nodes may behave in an arbitrary manner, we face a number of problems. Byzantine nodes can either sabotage the voting process by ignoring it, or send different data about their decision to different members. In fact, this situation is similar to vote falsification in an election. The decision based on a 50%-plus majority vote violates the system consistency property. This is because now the number of ballots (or the number of votes) is not equal to the number of voters (number of nodes in the network).

In this case, the nodes do not have a unified view of the system state at some point in time. And the system itself does not have a single center that can gather and communicate the final decision. Therefore, the problem of the Byzantine consensus is not in the correctness / incorrectness of the choice, but in the following:

- The whole system must come to a unanimous decision on the final result.

- The end result must be achievable, provided that at least some of the network members are interested in doing so.

Let’s determine the critical mass of the honest validators in the network that would allow the system to reach consensus and accept a new block into the blockchain.

Suppose we have `h` honest validator nodes and `f` (faulty) Byzantine ones, then the total number of validators is `N = h + f`. As mentioned above, all validators vote for one of the two nominated candidates. Each collects votes from other members of the voting process. And each comes to its own decision on the winner based on the threshold rule. The rule says: the number of votes for the winning candidate must be `≥ α * N`, where `α` is in the range from `0` to `1`. For example, an absolute majority of votes is reached when `α> 1/2`.

At the end of the voting, each validator independently decides which of the two candidates has won. It is worth noting that validators may fail to make the decision. This happens, for example, if too few validators have broadcasted their votes. This can occur, in particular, because Byzantine validators can broadcast different votes to different honest validators or skip voting at all.

To avoid this, in our reasoning, we must observe two conditions:

1. Honest validators should be able to make a choice even without the participation of Byzantine nodes. This means that the liveness property of the consensus algorithm is preserved even if the Byzantine nodes skip voting intentionally. This condition can be expressed by the following inequality: `h ≥ α * N`;

2. The alternative, for which voted the minority of the honest validators, cannot overcome the threshold of `α * N`. That is, the consistency property of the consensus algorithm is preserved even if the Byzantine nodes intentionally and consistently broadcast votes for one candidate to some honest validators, and votes for the second candidate — to the remaining honest validators. Let’s express this condition as follows: `[h / 2] + f <α * N`, where `[h / 2]` is the integer part of the number `h / 2`.

Solving inequalities from 1 and 2, together we get the following:

h> 2f,

α> 2/3 and

N ≥ 3f + 1.

Thus, the maximum number of Byzantine nodes in the network, which allows correct consensus operation, is `<1/3` of the total number of validators.

This result is easy to verify. Imagine that all the Byzantine nodes decided to vote twice. They broadcasted votes for one candidate to some honest validators, and votes for the second candidate to the remaining honest validators. If the number of Byzantine nodes is 1/3 of the total number of validators, then the total number of votes according to the voting results will be 4/3. Consequently, the number of honest nodes that voted only once (2/3) will be exactly 50% of the total number of votes. Thus, to gain the majority of honest votes and reach a consensus, the number of honest nodes should be at least `2/3 +1` of the total number of validators.

Stages of the Consensus Algorithm

In modern blockchain operations, user’s expectations for system speed and performance are very high. Classic algorithm solutions do not provide these qualities. We reviewed several other upgraded alternatives, but they also failed to fully satisfy our research team, as these algorithms did not meet all the requirements that experts place on the framework. As a result, we developed our own algorithm.

Exonum’s consensus algorithm relies on some of the ideas presented in Tendermint. However, it contains a number of characteristics that set it apart from Tendermint and from other consensus algorithms used for blockchain-based solutions.

Below we will discuss the mechanism of the Exonum consensus algorithm. In general, its scheme is as follows:

The process of reaching a consensus begins with a leading node (a leader) forming a list of transactions that must be added to the blockchain (making up a proposal). The leader is selected by a separate algorithm. According to our algorithm, the leader alters every round. The proposal is then broadcast throughout the network to the validator nodes (validators).

The validators check the received message for correspondence to the serialization format. In case of errors, the node completely ignores the received message. For example, the validators ignore a suggestion to add a block to the middle of the blockchain or to duplicate an already existing transaction. If everything is in order, then the voting stage begins. The validators vote to add the block to the blockchain, broadcasting prevote messages.

The node whose proposal gains more than 2/3 of the approvals from the validators, loses the opportunity to vote for proposals of other validators and cannot change its own proposal. This state is called proof-of-lock.

After the leader collects the required number of votes from the validators, it puts the approved transactions into a block and broadcasts a special message — precommit. The precommit contains a hash of the updated blockchain state and indicates that the node is ready to add the proposed block to the blockchain. The moment most validators respond with similar precommit messages (with the same hash), the block is added to the blockchain. The consensus is reached, and the procedure repeats for each subsequent block.

To increase the stability of the system, the validators periodically exchange Request and Block messages with each other. The former is required if the nodes lack any transaction data. The latter serves to transfer information about a block of transactions to a lagging node (for example, the node turned off for some time). This allows to synchronize the work of the entire network.

The proposed consensus algorithm is correct for each height, i.e. it satisfies the properties of liveness, consistency and censorship resistance. The formal proof of the consensus properties is available in our whitepaper.

Performance Tests

To assess the efficiency of the Exonum consensus algorithm, we checked how the blockchain operates with it using two configurations. The configurations include one data center and several geographically distributed data centers accordingly. During the tests, the TPS parameter — the number of transactions per second — was estimated for different numbers of validators. Below we present the charts that show network performance changes in blockchains with cryptocurrencies (the black graph) and in blockchains with timestamps (the blue graph).

On average, Exonum and its new Byzantine fault tolerant consensus algorithm for blockchains can process from 2,000 to 13,000 transactions per second, depending on the network configuration. In terms of in-production solutions, the algorithm can process 4,000 transactions per second with 0.5 seconds block acceptance time over the globally distributed network. Exonum’s performance is stable in case of fail-stop validators, but it can significantly decrease with the total number of validators growth.