We have two big pieces of news to share.

First, we’re spinning-out a new app-focused company, called Pogo, targeting interoperability between mobile wallets around the world. Pogo has been in stealth development here for some time, and, as a natural extension of that process, Adam Ludwin will lead this spin-out. As you’ll see when the service goes live, Pogo is the kind of application that Stellar is uniquely positioned to enable, thanks to the network’s ability to create low-friction, high-liquidity, real-time corridors for FX settlement between financial entities.

Second, and most exciting, Mike Kennedy, founder of Zelle, is joining Interstellar as our new CEO. Mike’s success with Zelle speaks for itself — it’s a payment system used by Chase, Citi, and Bank of America, among many others. His history of innovation in mobile payments, FX and banking, coupled with his track record as a high-impact advisor to Stellar make him the perfect fit to lead Interstellar.

We expect that Mike and the team he’s bringing in will build powerful new services for Stellar, just as he scaled and built Zelle. Below are a few words from Mike about why he is joining Interstellar and the future we’ll be working toward:

“I am excited to be taking on the role of CEO at Interstellar. I have always been interested in working on making things easier for individuals and companies, and I believe we have the opportunity to help make international payments more efficient, lower cost, and faster.

This was my mindset when founding and running Zelle. We realized that no one wanted to carry or pay with paper — cash and checks — and rather wanted to pay and move money digitally and securely. We therefore created a digital network of banks in the US to allow for this. We were able to scale this to over $50 billion in transaction volumes from over 25 million users. Our success was because we were solving a real-world problem.

With Interstellar I see another opportunity to solve a real-world problem. Today, international payments are inefficient, expensive, and slow. Millions of individuals and businesses would benefit from solving these problems. The Stellar Development Foundation (SDF) has developed what I believe is the best blockchain based payments network, enabling fast and inexpensive international payments. However there are still parts of the ecosystem that are not optimally solved for, and ones that a non-profit like SDF is not best suited to solve. This is where Interstellar comes in. We will deliver these missing pieces of the puzzle to allow the Stellar network to achieve its potential. We will solve the remaining problems, turbocharging adoption and growth — making international payments better for millions and millions of companies and end users.

It is solving these real-world problems — making payments easier, faster and cheaper for end users — that gets me excited! I look forward to the growth ahead, leading Interstellar.”

The Interstellar Update was originally published in Interstellar on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Stellar Consensus Protocol was first described in a whitepaper by David Mazières in 2015. It is a “federated Byzantine agreement system” that allows decentralized, leaderless computing networks efficiently to reach a consensus outcome on some decision. The Stellar payment network uses SCP to provide a consistent view of the network’s transaction history to all participants.

Consensus protocols have a reputation for being difficult to understand. SCP is simpler than most but still shares that reputation — due in part to the mistaken idea that “federated voting,” which the whitepaper spends its first half describing, is SCP. But it’s not! Instead, it’s an essential building block used by the second half of the whitepaper to construct the actual Stellar Consensus Protocol.

In this article, we’ll give some brief background about what an “agreement system” even is, what can make one “Byzantine,” and why you’d want to make a Byzantine one “federated.” We’ll then explain the federated voting procedure described by the SCP whitepaper, and finally explain SCP itself.

Agreement systems

An agreement system allows a group of participants to reach the same decision about something — for example, what to order for lunch.

At the offices of Interstellar, we have implemented our own lunch-agreement system: we order whatever our operations manager, John, says. It’s a simple and effective agreement system. We all trust John to order something interesting and nutritious each day.

But what if John were to abuse that trust? He could unilaterally decide we must all become vegans. After a week or two of that we’d probably depose him and give his authority to Elizabeth, but maybe she’s on an avocado-and-anchovy-sandwich kick and thinks we all should be too. Power corrupts, we might decide, and so we would seek some more democratic method: some way to make sure different preferences are heard while still reaching a timely, unambiguous outcome, so that we don’t end up with no one ordering lunch, or five of us placing competing lunch orders, or no decision about what to order until 4pm.

It might seem that the solution is simple: just conduct a vote! But this is deceptive. Who gets to collect ballots and report the results? And why should the rest of us trust what they say? Perhaps we could first vote on a leader whom we all trust to run the vote — but then who gets to run that vote? What if we can’t agree on a single leader? Or, what if we can agree, but then that leader gets stuck in a meeting, or goes home sick?

Similar problems are common in distributed computing networks. All the participants, or nodes, must agree on some decision, such as whose turn it is to update a shared file or pull a task from a processing queue. In a cryptocurrency network, nodes repeatedly must decide what the complete history of the shared ledger looks like, from among multiple possible versions that occasionally conflict. This network-wide agreement allows the recipient of a crypto coin to have faith that the coin is both (a) valid (not counterfeit) and (b) not already spent elsewhere. It also assures them that they’ll be able to spend it in the future, because the new recipient will have the same faith in it, for the same reasons.

Any agreement system in a distributed computing network needs to be fault-tolerant: it must produce consistent results despite errors like slow communication links, unresponsive nodes, and misordered messages. A Byzantine agreement system is additionally tolerant of “Byzantine” faults: nodes that give false information, whether due to error or in a deliberate attempt to subvert the system or gain some advantage.¹ Consider the owner, Alice, of a crypto coin, who has to choose between buying a delicious gelato with it from Bob, or paying it to Carol to settle a debt. Alice might like to have it both ways by fraudulently paying the same coin to both Bob and Carol. To do so she must convince Bob’s computer that the coin was never paid to Carol, and she must convince Carol’s computer that the coin was never paid to Bob. A Byzantine agreement system can make this effectively impossible using a form of majority rule called a quorum. A node in such a network refuses to commit to a particular version of history until it sees that enough of its peers — a quorum — are also prepared to commit. Once that happens, they’ve formed a voting bloc large enough to force the remaining nodes in the network to agree with their decision. Alice might be able to cause some nodes to lie on her behalf, but if the network’s large enough, her attempt will be overwhelmed by the votes of honest nodes.

How many nodes does it take to form a quorum? At least a majority and more typically a supermajority to combat errors and fraud. But knowing when you have a majority means knowing how many total participants you have. In the Interstellar office, or in a county election, those numbers are easy to know. But if your collection of participants is a loosely defined network that members can join and leave at will, without needing to coordinate with any central authority, then you need a federated Byzantine agreement system: one that can determine quorums not from some predetermined roster of nodes, but dynamically, from an ever-changing and inevitably incomplete snapshot of membership at a point in time.

It might not seem possible to construct a quorum from only the limited perspective of a single node in a sprawling network, but it is. This quorum can even create confidence in the outcome of a decentralized vote. The SCP whitepaper shows how to do this using a procedure called federated voting.

For the impatient

The rest of this article describes federated voting and the Stellar Consensus Protocol in some detail. To serve as a guide to what follows — or if you don’t care about the detail and just want the tl;dr — here’s an overview of the process.

1. Nodes conduct rounds of federated voting on “nominees.” A round of federated voting means:
   • A node casts a vote for some statement, such as “I nominate value V”;
   • The node listens to votes from its peers until it finds one it can “accept”;
   • The node seeks a “quorum” that also accepts the statement. This “confirms” the statement.
2. As soon as a node can confirm one or more nominees, it starts trying to “prepare” a “ballot” via more rounds of federated voting.
3. As soon as a node can verify that a ballot is prepared, it starts trying to “commit” the ballot via still more rounds of federated voting.
4. Once a node can confirm that a ballot is committed, it can “externalize” the value in that ballot, using it as the outcome of consensus.

These steps involve multiple rounds of federated voting that collectively form a single round of SCP. To understand what each step means, why so many are needed, how it all works, and what can go wrong, read on!

Federated voting

Federated voting is a procedure for discovering whether a network of participants can agree on a proposal. In a round of federated voting, each node must choose one of potentially many possible values as the outcome of that round. It cannot do so until it is sure that the other nodes in the network won’t choose any different outcome. To be sure of that, the nodes exchange a flurry of messages back and forth allowing each of them to confirm that a quorum of nodes accepts the same vote. The rest of this section explains the terms in that sentence and how such a confirmation can be achieved.

Quorums and quorum slices

Let’s start with identifying a quorum. As we discussed above, in a decentralized network with dynamic membership, it’s impossible to know ahead of time how many nodes there are, and therefore how many make up a majority. Federated voting solves this by introducing the novel idea of a quorum slice: a small collection of peers that a node trusts to convey information about the state of voting in the rest of the network. Every node defines its own quorum slice (of which it is also ipso facto a member).

To form a quorum, start with a quorum slice. For each member, add the members of its quorum slice. Then add the members of those members’ slices, and so on. As you continue you’ll encounter more and more nodes that you can’t add because they’re already included. When there are no more new nodes to add, stop: you have formed a quorum from a “transitive closure” of the starting node’s quorum slice.

In fact each node may have more than one quorum slice. To form a quorum, choose just one of the slices and add the members; then choose any one slice for each of the members and add those members, and so on. This means that each node is a member of many possible quorums.

How does a node know the membership of another node’s quorum slices? The same way it knows anything else about other nodes: from the broadcasts that each node sends to the network whenever its voting state changes. Each broadcast includes the details of the sending node’s slices.²

Recall that in a non-federated Byzantine agreement system, a quorum is defined as a majority of all nodes.³ Once a proposal passes the quorum threshold, the rest of the network members are convinced that any competing proposals will fail. This is how the network converges on an outcome.

But in a federated Byzantine agreement system, not only can there be no majority (because no one knows the total size of the network), but the concept of majority is not even useful! If membership in the system is open, then someone could gain a majority simply by conducting a so-called Sybil attack: joining the network many times using multiple nodes. So what is it about the transitive closure of a node’s quorum slice that makes it into a quorum, and what makes that able to overwhelm competing proposals?

Technically, nothing! Imagine a network containing Alice, Bob, Carol, Dave, Elsie, and Frank. Alice has Bob and Carol in her quorum slice. Bob has Alice and Carol, Carol has Alice and Bob. Meanwhile, Dave, Elsie, and Frank all have one another in their respective quorum slices. The Alice-Bob-Carol subgroup can reach a decision that the Dave-Elsie-Frank group will never hear about, and vice versa. There is no way for this network to achieve consensus (except by accident).

So SCP requires that, in order for federated voting to work (and for the paper’s important theorems to apply), the network must enjoy a property called quorum intersection. In a network with this property, any two quorums you can construct always overlap in at least one node. For determining the prevailing sentiment of the network, this is as good as having a majority. Intuitively, it means that if any quorum agrees to statement X, no other quorum can ever agree to not-X, because it will necessarily include some node from the first quorum that has already voted for X.

(Of course it could be that the overlapping nodes are all Byzantine — lying or otherwise misbehaving. In that case, having quorum intersection doesn’t help the network agree at all. For that reason, many of the results in the SCP whitepaper rely on explicitly stated assumptions, such as that the network enjoys quorum intersection even if misbehaving nodes are removed from the network. For the sake of clarity we’ll leave those assumptions implicit for the remainder of this article.)

It might seem unreasonable to expect a collection of independent nodes to organize their slices in such a way that the network will reliably enjoy quorum intersection. But there are two reasons why this isn’t so far-fetched.

The first reason is the existence of the Internet itself. The Internet is the perfect example of a network of independent nodes with quorum intersection. Most nodes on the Internet connect to just a few other local nodes, but those small sets overlap enough that every node is reachable from every other node by one route or another.

The second reason is specific to the Stellar payment network (the most widespread application of SCP). Each asset type in the Stellar network has an issuer, and Stellar best practices require each issuer to designate one or more nodes in the network for handling redemption requests. It’s in your interest to include those nodes in your quorum slices, directly or indirectly, for each asset you care about. The quorums for all nodes interested in a given asset will then overlap in at least those redemption nodes. Nodes interested in multiple assets will include in its quorum slices all the relevant issuers’ redemption nodes, and these will tend to bridge all assets together. Further, any assets that are not connected in this way to others on the network don’t need to be — it’s OK for the network to lack quorum intersection there. (Think of the way that banks operating in dollars sometimes want to trade with banks operating in euros and banks operating in pesos, so they are on a network together, but none of them care about the separate network of kids trading baseball cards.)

Of course, expecting that the network should enjoy quorum intersection is not the same as a guarantee. Other Byzantine agreement systems owe much of their complexity to making guarantees about quorums. An important innovation of SCP is that it removes the responsibility for making quorums from the consensus algorithm itself and pushes it into the application layer. This suggests that, although federated voting is general enough to work with any kind of value being voted on, in fact its robustness depends critically on the broader meaning of those values. Some hypothetical uses might not lend themselves as readily to producing well-connected networks as others.

Voting, accepting, and confirming

In a round of federated voting, a node optionally begins by casting a vote for some value V. This means broadcasting a message to the network saying, “I am node N, my quorum slices are Q, and I vote V.” When a node votes in this way, it promises that it has never voted against V and never will.

A node can see how its peers are voting from their broadcast messages. Once the node collects enough such messages, it can traverse the quorum slices in them to find quorums. If it can see a quorum of peers that all vote for V also, then it can move to accepting V, and it broadcasts this new message to the network. (“I am node N, my quorum slices are Q, and I accept V.”) Accepting provides a stronger guarantee than mere voting. When a node votes for V, it can never vote for not-V. But when a node accepts V, no node in the network will ever accept not-V. (Theorem 8 in the SCP whitepaper proves this.)

Of course, there’s a good chance that N won’t see a quorum of nodes agreeing with its V vote right off the bat. Other nodes may vote for other values. But there is another way for a node to advance from mere voting to accepting. N can accept a different value, W, even if N didn’t vote for it, and even if it doesn’t see a quorum voting for it, as long as it sees a blocking set accepting it. A blocking set is just one node chosen from each of N’s quorum slices. As its name suggests, it is capable of blocking any other value. If all nodes in such a set accept W, then (by Theorem 8) it will never be possible to form a quorum accepting not-W, and so it’s safe for N to accept W too.

But a blocking set is not a quorum. It would be too easy for someone to fool node N into accepting a value when it shouldn’t, if they can just subvert one node in each of N’s slices. So accepting a value is not the end of voting. Instead, N must confirm the value, meaning it sees a quorum of nodes all accepting it. If it gets this far, then as the SCP whitepaper proves (in Theorem 11), the rest of the network will also eventually confirm the same value, and so N has reached the end of federated voting with the value as its outcome.

The process of voting, accepting, and confirming makes up one complete round of federated voting. The Stellar Consensus Protocol combines many such rounds to create a complete consensus system.

The Stellar Consensus Protocol

The two most important properties of a consensus system are safety and liveness. A consensus algorithm is “safe” if it can never produce different results for different participants. (Bob’s copy of history will never disagree with Carol’s.) Meanwhile, “liveness” means the algorithm will never fail to produce a result — i.e., that it won’t get stuck.

The federated voting procedure just described is safe in the sense that if a node confirms value V, no other node will confirm a different value. But “won’t confirm different things” is not the same as “will confirm something.” There may be so many different values being voted on that nothing even reaches the “accept” threshold. This means that federated voting lacks liveness.

The Stellar Consensus Protocol uses federated voting in a way that guarantees safety and liveness.⁴ The idea, in a nutshell, is to conduct multiple federated votes on multiple values until one makes it all the way through SCP’s various voting phases, described below.

The values on which SCP seeks consensus may be Stellar ledgers or lunch orders or anything else, but it’s important to note that these are not the values that SCP’s federated-voting rounds vote on or accept or confirm. Instead, federated voting happens on statements about those values.

The first rounds of federated voting happen in the nomination phase, on a family of statements of the form, “I nominate V,” for possibly many different values of V. The goal of nomination is to find one or more such statements that can make it all the way through acceptance and confirmation.

After finding confirmable nominees, SCP moves to the balloting phase, where the goal is to find some ballot (which is a container for a nominated value) and a quorum that can commit to it. If a quorum commits to this ballot, its value is the outcome of this round of consensus. But before a node can vote to commit to a ballot, it must first confirm that all lesser ballots are aborted. These steps — aborting ballots in order to find one that can be confirmed committed — involve multiple rounds of federated voting on multiple statements about ballots.

The following sections describe nomination and balloting in more detail.

Nomination

At the start of the nomination phase, each node may spontaneously choose a value V and vote for the statement “I nominate V.” The goal at this stage is to confirm the nomination of some value, via federated voting.

It’s possible that enough nodes vote for enough different “I nominate” statements that it’s a free-for-all — no nomination can ever reach the “accept” threshold. So in addition to casting their own nomination votes, nodes “echo” the nominations of their peers. Echoing means that a node voting “I nominate V,” upon seeing a message from a peer nominating W, now votes both “I nominate V” and “I nominate W.”⁵ Conceptually these are separate federated votes happening in parallel, each separately able to reach acceptance or confirmation. In practice SCP protocol messages batch these separate votes together.

Although a vote to nominate V is a promise never to vote against nominating V, the application layer above federated voting — SCP in this case — gets to define what “against” means. SCP defines no statement that contradicts an “I nominate X” vote — there is no “I reject the nomination of X” — so a node can vote to nominate as many values as it likes. Many of those nominations may go nowhere, but eventually there will be one or more that a node can accept or confirm. Once a nominee is confirmed, it is called a candidate.

Nomination may produce multiple confirmable candidates, so SCP requires the application layer to supply some method for combining candidates into a single composite. The combining method can be anything the application layer calls for, as long as that method is deterministic, meaning that every node combining the same two candidates produces the same composite. In a lunch-ordering system, “combining” might simply mean discarding one of the two candidates. (Deterministically, though: every node must choose the same value to discard. The alphabetically earlier choice, for example.) In the Stellar payment network, where nominees are ledgers, combining two proposed nominees involves taking the union of the transactions they contain, and the later of their two timestamps.

The SCP whitepaper proves (Theorem 12) that nomination causes the network eventually to converge on a single composite candidate by the time nomination ends. But there’s a problem: federated voting is an asynchronous protocol. (So is SCP.) In other words, nodes are not coordinated by time, only by the messages they send. From a node’s point of view it’s not clear when nomination has ended. And although all nodes will eventually arrive at the same composite candidate, they may take different routes to get there, producing different, valid composite candidates along the way, never able to tell which is the final one.

This is OK though. Nomination is only about producing — and limiting the number of — candidates for consensus. The rest of consensus is handled by a process called balloting.

Balloting

A ballot is a pair: <counter,value>, where counter is an integer starting at 1 and value is a candidate from the nomination phase. It could be the node’s own candidate, or some peer’s candidate that the node comes to accept. Roughly speaking, balloting makes repeated attempts to get the network to reach consensus on some candidate in some ballot by conducting potentially many federated votes on statements about ballots. The counters in ballots track the attempts made, and ballots with higher counters take precedence over ballots with lower counters. If ballot <counter,value> appears to be getting stuck, a new vote begins, now on ballot <counter+1,value>.

It’s important to distinguish between values (e.g., what the lunch order should be, such as pizza or salads), ballots (a counter-value pair), and statements about ballots. A round of SCP includes multiple rounds of federated voting on such statements, specifically these:

• “I am prepared to commit to ballot B,” and
• “I commit to ballot B.”

From the point of view of a given node, consensus is reached when it finds a ballot B for which it is able to confirm (find a quorum accepting) the statement “I commit to ballot B.” At this point it is safe to act on the value in B— e.g., to place that lunch order. This is called externalizing the value. Once a ballot is confirmed committed, a node can be sure that every other node has externalized, or inevitably will externalize, the same value.

Although conceptually many federated votes take place on statements about many different ballots, far fewer actual protocol messages are exchanged, because each one encapsulates a range of ballots. A single message thus advances the state of many federated votes at once, e.g.: “I accept that ballots in the range <min,V> to <max,V> are committed.”

So what do “prepared” and “committed” mean?

A node votes to commit to a ballot when it is convinced that other nodes won’t commit to ballots with different values. Becoming convinced of this is the purpose of the prepare statement. A vote that says “I am prepared to commit to ballot B” is a promise never to commit to any ballot less than B — i.e., one with a smaller counter.⁶ Those lesser ballots are said to be “aborted” by the prepare vote, while B is “prepared.”

Why does “I am prepared to commit to ballot B” mean “I promise never to commit to ballots less than B”? It’s because SCP defines “abort” to be the opposite of “commit.” A vote to prepare a ballot is implicitly also a vote to abort some other ballots, and as we discussed earlier, voting for something is a promise never to vote against it.

Before ever casting a “commit” vote, a node must first find a ballot that it can confirm is prepared. In other words, it conducts federated voting on “I am prepared to commit to ballot B” for possibly many different ballots until it finds one that a quorum accepts.

Where do ballots come from for the prepare votes? The first prepare vote that a node casts is for <1,C>, where C is the composite candidate produced by the nomination phase. However, even after prepare votes begin, nomination may produce additional candidates, so these become new ballots. Meanwhile, peers may have different candidates, and they could form a blocking set that accepts “I am prepared to commit to ballot B2,” which would convince the node to accept it too. Finally, there’s a timeout mechanism that causes new rounds of federated voting to begin on new ballots with higher counters if the current ballots seem to be stuck.

Once a node finds a ballot B that it can confirm is prepared, it casts a new vote for “I commit to ballot B.” This vote tells peers that the node will never abort B. In fact, if B is the ballot <N,C>, then “I commit to ballot <N,C>” is implicitly also a vote to prepare every ballot from <N,C> through <∞,C>. This extra meaning helps other nodes receiving this message to catch up, if they’re still in earlier phases of the protocol.

It’s worth re-emphasizing at this point that these are asynchronous protocols. Just because one node is sending commit votes doesn’t mean that its peers are, too. Some might still be voting on prepare statements, others might already have externalized a value. SCP explains how a node should handle each type of peer message no matter what phase it’s in.

If “I commit to <N,C>” can’t be accepted or confirmed, perhaps <N+1,C> can, or <N+2,C> — or, in any case, some ballot with C in it and not any other value, since the node has now promised never to abort <N,C>. By the time a node is casting commit votes, it’s C or bust as far as consensus goes. However, this isn’t enough for the node to externalize C yet. Some Byzantine peers (amounting to less than a quorum, based on our safety assumptions) could be lying to the node. Accepting and then confirming some ballot (or range of ballots) is what gives the node the confidence finally to externalize C.

And that’s it! Once the network has come to consensus, it’s ready to do it all over again. In the Stellar payment network, this happens roughly once every 5 seconds: a feat that requires both the safety and liveness guaranteed by SCP.

SCP is able to accomplish this by relying on multiple rounds of federated voting. Federated voting is made possible by the concept of quorum slices: sets of peers that each node has chosen to trust as part of its own (subjective) quorum. This configuration means that it’s possible to come to consensus, even in a network with open membership and Byzantine failures.

Further reading

• The original SCP whitepaper can be found here. A draft specification for implementers is here.
• SCP’s original author, David Mazières, has a simplified (but still technical) explanation of it here.
• You may have been surprised not to find the terms “mining” or “proof of work” in this article. SCP doesn’t use those techniques, but some other consensus algorithms do. Zane Witherspoon has written an accessible survey of consensus algorithms.
• A step-by-step description of a simple network coming to consensus via one complete round of SCP can be seen at A round of lunch.
• For readers interested in implementations of SCP, see the C++ code used by the Stellar payment network or the Go code that this author wrote in order to better understand SCP.

Endnotes

1. “Byzantine” fault tolerance — the ability to trust a group decision even when some members of the group might be lying or otherwise not following the decision-making rules — is so named because of a parable about generals of the Byzantine Empire trying to coordinate an attack. Anthony Stevens has written a good description of it.
2. The SCP whitepaper does not specify a communication mechanism. Implementations will typically use a gossip protocol to ensure that messages propagate throughout the network.
3. A Byzantine agreement system is designed in terms of the question: How many misbehaving nodes should the system survive? In a system of N nodes intended to survive f failures, a node must be able to make progress after hearing from N−f peers, since f of them may have crashed. However, after hearing from N−f peers, it’s possible that all the remaining f peers (that the node hasn’t heard from) are honest, and up to f out of the N−f peers (that the node has heard from) are malicious. To ensure nodes all reach the same conclusion, then, a majority of the N−f nodes one hears from must be honest, meaning we need N−f > 2f, or N > 3f. So typically a system designed to survive f failures will have a total of N=3f+1 nodes and a quorum size of 2f+1.
4. The safety and liveness guarantees of SCP are subject to theoretical limits. The design of SCP trades a very strong safety guarantee for a slightly weaker liveness guarantee: given a sufficient time bound, consensus will be reached with high probability.
5. Not all votes from peers get echoed by a node during nomination, because this could lead to an explosion of different nominees. SCP includes a mechanism for throttling these echoed votes. In short, there is a formula for determining the “priority” of a peer from a node’s point of view, and only high-priority nodes get their votes echoed. The longer nomination takes, the lower the threshold gets, so a node expands the set of peers whose votes it will echo. The priority formula includes the slot number as one of its inputs, so a high-priority peer for one slot may be low-priority for the next, and vice versa.
6. SCP requires that the values in ballots have some order defined on them. So ballot <N1,V1> is less than <N2,V2> if N1<N2, but also if N1=N2 and V1<V2.

Acknowledgments

Many thanks to those who contributed feedback about, suggestions for, and content to this document: David Mazières; Tess Rinearson; Oleg Andreev; Vicki Niu; Cathie Yun; Adam Diehl; Lindsay Lin; Suzanne Glickstein.

Understanding the Stellar Consensus Protocol was originally published in Interstellar on Medium, where people are continuing the conversation by highlighting and responding to this story.

In our previous article, we announced a pre-release of Bulletproofs 1.0, the Rust implementation of range proofs. We (Cathie Yun, Henry de Valence and Oleg Andreev) have been working on extending it to support proving arbitrary statements in zero knowledge using a constraint system, which we would like to share with you today. We are also excited to publish our work on the next generation of Chain’s Confidential Assets protocol called Cloak, which builds directly on top of these Bulletproofs constraint system proofs.

Constraint System

Bulletproofs are an optimization to an earlier paper by Bootle, Cerulli, Chaidos, Groth, and Petit, which showed that it was possible to prove arithmetic circuit satisfiability in the discrete log setting. Their paper introduced the logarithmic-sized inner-product argument, as well as a technique to convert an arithmetic circuit into a rank-1 constraint system (R1CS). The Bulletproofs paper optimizes the inner-product argument by a constant factor, and shows how to use Pedersen commitments as inputs to the constraint system.

R1CS expresses linear combinations of various variables and their multiplications. The Bulletproofs paper uses the following format for the constraint system: an array of n multiplications that gives 3*n low-level variables: left, right and output “wires” of each multiplication “gate”, and an array of q linear constraints between these variables and additional m high-level variables that represent external facts.

We realized that it is more efficient to bypass arithmetic circuits altogether and provide the API for building a constraint system directly. This way we do not have to construct the graph or arithmetic expressions and then transform them into constraints: higher-level protocols can often keep that graph represented statically by their source code and build constraints on the fly.

Moreover, since the matrices representing the constraints are usually very sparse, we only store the non-zero terms. The only operation that needs to be done on the constraints is flattening with a challenge variable, and the result is defined solely by the non-zero terms.

Variables

The constraint system has three kinds of variables: high-level witness variables, low-level witness variables, and instance variables.

Instance variables are public parameters, known to both the prover and the verifier. In other proving systems that require a setup for each constraint system, this allows public inputs to the circuit. In Bulletproofs, no setup is required, so there’s no difference between a constraint system with public inputs and a family of constraint systems parameterized by those inputs. This means we can internally fold all instance variables into a single constant parameter.

High-level witness variables are known only to the prover, and represent external inputs to the constraint system. In Bulletproofs, these are represented as individual Pedersen commitments to the external variables, for which we construct and prove some relation. For instance, in the Cloak protocol (described below), the transacted amounts are represented as commitments to high-level variables.

Low-level witness variables are known only to the prover, and are internal to the constraint system, representing the inputs and outputs of the multiplication gates. For instance, in Cloak these represent bits inside a range proof or intermediate amounts as they are passed through shuffle, merge and split stages. Low-level variables are computed per-proof and committed using a single vector Pedersen commitment.

Challenges

As noted above, instance variables can select the constraint system out of a family for each proof. This enables a novel approach: by generating some of them as a challenge from a verifier to a prover, the constraint system itself can be thought of as a challenge from a verifier to a prover, where some constraints are generated randomly in response to the prover’s commitments.

The use of challenges to parametrize constraint systems can enable significantly fewer multipliers, which makes the resulting proof smaller. For instance, a verifiable shuffle may require O(n²) multiplications in a static constraint system, but by using challenges we reduce the overhead to O(n) multiplications — inputs and outputs can be used as permuted roots of the polynomials that are randomly sampled at a challenge point and constrained to be equal:

{a,b} = {c,d} ⟺ (a−x)⋅(b−x)=(c−x)⋅(d−x)

Our implementation performs the Fiat-Shamir transform using Merlin transcripts to generate such challenges. The challenges are bound to the high-level variable commitments which are added to the transcript before any of the constraints are created. Then, the prover and verifier can compute weights for some constraints with the use of the challenges.

The use of challenges comes with an important caveat: the challenges are bound only to the high-level witness variables (the external inputs to the constraint system), not the low-level ones, so it requires careful analysis to ensure that the construction is safe. That said, we are working on an improvement to the protocol that would allow challenges to be bound to a subset of the low-level witness variables, and have a safer API using features of Rust’s type system.

Gadgets

The resulting API is largely inspired by the Bellman API used in Zcash: there is a single code path used by both the prover and verifier to allocate variables and define constraints, organized into a hierarchy of task-specific “gadgets”. This way the allocation, assignment and constraints on the variables all happen in one place, providing assurance that variables are not accidentally left unconstrained. In other words, each gadget can be verified independently, without having to analyze correctness of the entire constraint system as a whole (see also the caveat about challenges in the previous section).

Each gadget is a function that interacts with a mutable “constraint system” object represented by a trait. The prover and verifier use their specific implementations of this trait respectively. The gadget receives secret variables and public parameters, allocates and assigns variables, generates challenges and adds constraints to the constraint system object.

Gadgets can be composed. For instance, in the Cloak protocol, a “merge” gadget takes M inputs and M outputs and calls a sequence of M-1 “mix” gadgets that operate on pairs of values. Then, the whole collection of gadgets in Cloak is abstracted away with a single “transaction” gadget that takes all input and output values, which could be used either directly or as a part of a higher-level protocol.

The Bulletproofs library does not provide any standard gadgets, but only an API for the constraint system. It is the job of each protocol on top of it to create a collection of gadgets for its use and build a complete constraint system out of them. For an example of a protocol built using a collection of gadgets, see our Cloak implementation (described below).

Cloak

Cloak is the redesign of the Chain’s Confidential Assets scheme using Bulletproofs. We are making its implementation open source and calling it Spacesuit as in interstellar cloak.

Cloak is focused on one thing: proving that some number of values with different asset types is correctly transferred from inputs to outputs. Cloak ensures that values are balanced per asset type (so that one type is not transmuted to any other), that quantities do not overflow the group order (in other words, negative quantities are forbidden) and that both quantities and asset types are kept secret. Cloak does not specify how the transfers are authenticated or what kind of ledger represents those transfers: these are left to be defined in a protocol built around Cloak.

Cloak builds a constraint system using a collection of gadgets like “shuffle”, “merge”, “split” and “range proof” all combined together under a single gadget called a “cloaked transaction”. The layout of all the gadgets is determined only by the number of inputs and outputs and not affected by actual values of the assets. This way, all transactions of the same size are indistinguishable.

Bulletproofs constraint system turns out surprisingly efficient: compared to a single-asset transaction (such as the Confidential Transactions proposal for Bitcoin) where only range proofs are necessary, the additional constraints and gadgets needed to support the issued assets add less than 20% of the multipliers. And thanks to the inner product proof, the impact on the proof size is close to zero.

The work on Cloak and Spacesuit is far from complete, and now will continue in the open in our repository. To learn more about Cloak check out its specification.

Next steps

The work on constraint systems and Cloak is still ongoing. For the R1CS implementation, there are two major improvements that we will focus on:

First, we would like to extend the Bulletproofs protocol slightly, to enable committing to a portion of low-level variables using a single vector Pedersen commitment without an overhead of additional individual high-level Pedersen commitments. This will also come with a change in the API that would ensure that challenge-based variables cannot be inspected, preventing the user from accidentally breaking soundness of their gadgets.

Second, we want to enable multi-party proving of a single constraint system. This is different from the existing aggregation MPC that we’ve done for the range proofs: instead of aggregating proofs of distinct statements purely to make proofs smaller, we want to enable building a joint proof of a single constraint system by multiple parties, without sharing their secret inputs with each other. This not only makes the proofs smaller, but could also improve privacy in Cloak by enabling a zero-knowledge version of a CoinJoin protocol.

Further reading on constraint system proofs

• How to build a constraint system proof
• Prover and verifier algorithms for a constraint system proof
• Example: a proof-of-shuffle constraint system
• Example: Cloak constraint system specs

Thanks

We’d like to thank Sean Bowe, Daira Hopwood, and Jack Grigg from the Zcash team, as well as Mary Maller from University College of London, for their helpful suggestions and discussion.

Programmable Constraint Systems for Bulletproofs was originally published in Interstellar on Medium, where people are continuing the conversation by highlighting and responding to this story.

As we described a few months ago, our implementation is implemented in pure Rust using the Ristretto group; our previous post has more detail and background information. Since then, we improved performance slightly, pushing verification performance down to 1040 µs for a 64-bit range proof. For comparison, this is:

• 1.83x faster than the libsecp implementation (with endomorphisms);
• 2.00x faster than the libsecp implementation (without endomorphisms);
• 4.63x faster than the Monero implementation.

In addition to better performance, we also provide a clean, safe, and extensible API for both single-party proving as well as multi-party computation of aggregated proofs. The single-party proving is actually implemented by self-MPC internally, to avoid code duplication.

This post shares some details of our design and says a few words about the next version of Bulletproofs, that will provide a constraint system API to allow proofs of arbitrary statements.

Composable proof transcripts

Our implementation performs the Fiat-Shamir transform using Merlin transcripts. This provides per-application domain separation, since constructing a transcript requires passing a domain separation label. It also means that it’s possible for users of our API to bind the Bulletproofs to arbitrary structured data (by committing it to the transcript prior to proving), or to compose the Bulletproofs with other proof statements (by using a common transcript for both proofs), all without any changes to our implementation.

But this also has benefits in the implementation internals. To create a range proof, Bulletproofs first performs a protocol which reduces the statement about ranges to a statement about inner-products, and then performs an inner-product proof. Instead of implementing these logically distinct components together, using Merlin transcripts allows us to implement them separately internally, and compose the two parts into a single proof returned to the user. This makes the implementation significantly cleaner and easier to understand, because the software components of the implementation correspond to the conceptual components of the protocol. You can learn more about Merlin in this overview, or in its docs.

Strongly-typed multiparty computation

As described in our previous post, we take advantage of Rust’s type system and ownership semantics to implement the multiparty computation (MPC) protocol, by encoding each state of the MPC protocol using a different Rust type, and using ownership semantics to enforce that state transitions are not reversible.

For instance, in the MPC protocol for aggregated range proof creation, each party constructs a vector polynomial and reveals the result of its evaluation at a single challenge point to construct its proof share. However, if the party could be tricked into revealing multiple evaluations of the same polynomial at different points, it would be possible to recover the original polynomial and reveal the party’s secrets.

But this is impossible in our implementation: evaluation is performed by the apply_challenge method of the PartyAwaitingPolyChallenge state, which consumes the state and returns a ProofShare. Because Rust’s ownership model prevents the use of moved values, it’s not possible to call apply_challenge twice, because the original PartyAwaitingPolyChallenge is already consumed. Similarly, because each state transition is performed by a method on a particular type, it’s not possible to perform the steps in the wrong order.

In other words, the type-checking and borrow-checking performed by the Rust compiler provide a compile-time proof that any instantiation of the MPC protocol components is one that performs the protocol steps correctly.

Extensible generator chains

The Bulletproofs paper describes the aggregated range proof protocol for m parties proving n-bit ranges as operating on two vectors of generators, G and H, both of size m*n. But implementing the proofs in this way would mean that the generators would be tied to both the aggregation size as well as the bitsize. And this approach isn’t extensible to the constraint system case, where the number of generators required depends on the number of constraints, which might not be known in advance.

Instead, we construct generators for each aggregation party separately, feeding a per-party domain separation label into the SHAKE extendable-output function (XOF), and passing the SHAKE output into the ristretto255 hash-to-group function. This provides an arbitrarily-long chain of orthogonal generators unique to each party. Instead of viewing G and H as vectors of size m*n, we view them as concatenations of m slices of the first n generators for each party. In other words, we construct the m*n generators by taking rectangular sections of a 2D array of generators, with the vertical axis corresponding to the number of parties and the horizontal axis corresponding to the size of each party’s proof share.

This means that a single cache of pre-computed Bulletproofs generators can be used with multiple choices of aggregation size and bitsize, and it’s also future-compatible with more flexible aggregation — for instance, there’s no need for the parties to be assigned contiguous rows in the grid.

For constraint system proofs, the number of generators required depends on the number of constraints. If the constraints are programmatically defined, it may not be convenient to specify the number of constraints up-front, but this construction allows the constraint system to just pull an appropriately-sized slice of generators from a pre-computed cache. And, because the generator chain is domain-separated by a per-party label, it also means that single-party constraint system proofs can be forward-compatible with a multiparty protocol for aggregated constraint system proofs (and even compatible with aggregation of proofs of different statements).

Finally, we also took care to ensure that the generators used for the inner-product argument in Bulletproofs are handled separately from the generators used for Pedersen commitments, so that the choice of Pedersen generators can be made per-application, in order to maximize reusability.

Next steps: arbitrary proofs via constraint systems

The building blocks we described above now allow us to extend the proving system from range proofs to arbitrary user-defined statements.

Bulletproofs are an optimization to an earlier paper by Bootle, Cerulli, Chaidos, Groth, and Petit, which showed that it was possible to prove arithmetic circuit satisfiability in the discrete log setting. Their paper introduced the logarithmic-sized inner-product argument, as well as a technique to convert an arithmetic circuit into a constraint system. The Bulletproofs paper optimizes the inner-product argument by a constant factor, and shows how to use Pedersen commitments as inputs to the circuit.

We’ve been working on an API for building Bulletproof constraint systems directly, inspired by the Bellman API used in Zcash, which allows both the prover and verifier to programmatically define a constraint system and then create a proof (or verification). We’ll have more to share in a future blog post, but we’d like to thank Sean Bowe, Daira Hopwood, and Jack Grigg from the Zcash team, as well as Mary Maller from University College of London, for their helpful suggestions and discussion.

Bulletproofs pre-release was originally published in Interstellar on Medium, where people are continuing the conversation by highlighting and responding to this story.

Mentality

Imagine we are software engineers at an organization. We expect to write software to support the organization, and for that software to take the shape of multiple programs.

We plan to work incrementally, to choose technologies suited to their tasks that we also enjoy, and to be unafraid to write bespoke software that improves our happiness and productivity.

New repo

We initialize a monorepo on GitHub: org/repo:

.
└── README.md

We write a README.md:

Add to your shell profile:

  # Set environment variable to monorepo path
  export ORG="$GOPATH/src/org"

  # Prepend monorepo scripts
  export PATH="$ORG/bin:$PATH"

Clone:

  git clone https://github.com/org/repo.git $ORG
  cd $ORG

The $ORG environment variable will be used throughout the codebase.

Initial design

We design an initial architecture like this:

• a “Dashboard” web interface for customers, written in React (TypeScript)
• mobile apps for customers, written in React Native (TypeScript)
• SDKs for customers, written for Go, Node (TypeScript), and Ruby
• an internal HTTP API used by the web interface and SDKs, written in Go
• a Postgres database backing the HTTP API

We add new files and directories (old files omitted with ... for brevity):

.
├── ...
├── cmd
│   └── serverd
│       └── main.go
├── dashboard
├── mobile
│   └── app
│       ├── android
│       └── ios
├── sdk
│   ├── go
│   ├── node
│   └── ruby
└── server
    └── http_handler.go

Commit messages

We use community conventions for commit messages with the addition that the first line (“subject line”) should be prefixed:

$ git log
dashboard: redirect to new project form after sign up
sdk/node: enable Keep-Alive in HTTP agent
server: document access to prod db

The prefix refers to the module that the patch is primarily concerned with. It often, but not always, matches a filesystem directory,

If a module’s interface changes, the patch will likely touch other files to conform to the new interface, but the prefix should be where the action is.

Although we don’t worry about it much, we try to keep the subject line under 50 characters. It can be nice to err on the side of brevity for module names.

Dependencies

To aid local development and help onboard teammates, we might provide a setup.sh script for each project. In turn, each setup.sh script might invoke more general$ORG/bin/setup-* scripts.

Since all engineers have $ORG/bin on our $PATH, all scripts in bin/ are available everywhere at our shell without a directory prefix.

We can also use ASDF’s .tool-versions file to pin versions for dependencies:

.
├── ...
├── .tool-versions
├── bin
│   ├── setup-go
│   ├── setup-localhost-tls
│   ├── setup-node
│   ├── setup-postgres
│   └── setup-ruby
├── dashboard
│   └── setup.sh
├── mobile
│   └── setup.sh
├── sdk
│  ├── go
│  │   └── setup.sh
│  ├── node
│  │   └── setup.sh
│  └── ruby
│      └── setup.sh
└── server
    └── setup.sh

Formatting

Our Go code is already formatted with gofmt but we could be more consistent in our TypeScript and Ruby code. We add config files for Prettier and Rubocop at the top of the file hierarchy:

.
├── ...
├── .rubocop.yml
└── .prettierrc.yml

Infrastructure

Our software can only be fully realized once it’s in our customers’ hands.

We choose Amazon Web Services (AWS) as a host for our core software, such as EC2 for the HTTP API.

We add an infra directory for Terraform, systemd, bash files, etc.:

.
├── ...
└── infra

CI

To move quickly without breaking things, we write automated tests and run those tests continuously when we integrate our changes into the codebase.

We also want our Continuous Integration (CI) service to:

• begin running the tests in < 5s after opening or editing a pull request
• run the tests in an environment with parity to the production environment
• run only the tests relevant to the change

We observe common causes of slow-starting tests on hosted CI services are multi-tenant queues and containerization. In those environments, noisy neighbors backlog the queues and containers have cache misses.

To meet our design goals, we write our own CI service, testbot (design credit Keith Rarick):

.
├── ...
├── cmd
│   └── testbot
│       └── main.go
├── dashboard
│   └── Testfile
├── mobile
│   └── Testfile
├── sdk
│   ├── go
│   │   └── Testfile
│   ├── node
│   │   └── Testfile
│   └── ruby
│       └── Testfile
├── server
│   └── Testfile
└── testbot
    ├── Testfile
    ├── farmer
    │   ├── Procfile
    │   └── main.go
    └── worker
        └── main.go

We open a GitHub pull request to add a new feature to the SDKs:

sdk/go/account.go             | 10 +++++-----
sdk/go/account_test.go        | 10 +++++-----
sdk/node/src/account.ts       | 10 +++++-----
sdk/node/test/account.ts      | 10 +++++-----
sdk/ruby/lib/account.rb       | 10 +++++-----
sdk/ruby/spec/account_spec.rb | 10 +++++-----
6 files changed, 60 insertions(+), 60 deletions(-)

A testbot farmer process on a server responds to the GitHub webhook by:

• identifying the directories containing files that have changed
• walking up the file hierarchy to find Testfiles for changed directories
• saving test jobs to its backing Postgres database

Each Testfile defines test jobs for its directory. Ours might be:

$ cat $ORG/sdk/go/Testfile
tests: cd $ORG/sdk && go test -cover ./...
$ cat $ORG/sdk/node/Testfile
tests: cd $ORG/sdk/node && npm install && npm test
$ cat $ORG/sdk/ruby/Testfile
tests: $ORG/sdk/ruby/test.sh

A single Testfile can define multiple test jobs. As test scripts become lengthier, it is convenient to extract them to their own script.

Each line contains a name and command separated by a colon. The name appears in GitHub checks. The command is run by a testbot worker, which is continuously long polling testbot farmer via HTTP over TLS, asking for jobs.

Each testbot worker runs on an EC2 instance created from the same Amazon Machine Image (AMI) used by our production server. We run more instances to increase test parallelism.

In this example, the tests for the Go, Node, and Ruby SDKs will begin to run almost simultaneously as different testbot worker processes pick them up.

Tests for dashboard and server will not run in this pull request because no files in their directories were changed.

Testing across services

To make it convenient to test across service boundaries, we write a with-serverd script that:

• installs the serverd binary
• migrates the database
• creates a team and credential
• runs serverd serve without blocking programs passed as arguments to with-serverd

We place this script in $ORG/bin to make it available on our $PATH.

This script can be used in Testfiles:

$ cat $ORG/dashboard/Testfile
tests: with-serverd $ORG/dashboard/test.sh
$ cat $ORG/sdk/go/Testfile
tests: cd $ORG/sdk && with-serverd go test -cover ./...

Tests that depend on with-serverd can avoid mocking on HTTP boundaries, making actual requests to the backing service and generating observable logs.

To cover product’s surface area, we move the Go SDK’s Testfile to the top of the file hierarchy so its test jobs will run on all pull requests:

.
├── ...
├── Testfile
├── bin
│   └── with-serverd
└── sdk
    └── go

Open source

We want our SDKs to be open source on GitHub and available on registries such as NPM and Rubygems but we like the monorepo as the canonical place for our development workflow.

So, we decide to mirror the SDK code to open source repos.

This provides community access. Although the community patch workflow is a bit manual, these are SDKs tightly coupled to our product and we expect community patches to be rare.

Other kinds of projects such as general-purpose libraries might be better suited to live as standalone open source repos outside the monorepo.

We write a mirrorbot program (design credit Bob Glickstein):

.
├── ...
├── .mirrorbot.yml
└── cmd
    └── mirrorbot
        ├── Procfile
        ├── Testfile
        └── main.go

mirrorbot runs as a command-line program that copies relevant changes from the $ORG monorepo to one or more target repos. It can be run on a laptop or a server.

On startup, it clones the monorepo and each target repo. Each copied commit includes an upstream:[SHA] line (example). mirrorbot reads that SHA from the latest commit on the destination branch of each target repo to get a cursor.

Mirror then fast-forwards the local monorepo, looking for new commits.

For each target repo, a patch file is produced for each new commit. The patch file is then filtered to remove anything not applicable to that repo. If anything remains, the patch is applied and committed. Any new commits are then pushed to GitHub.

A .mirrorbot.yml file configures the program. It maps monorepo branches to target repo branches and monorepo directories and files to target repo directories and files. For example:

github.com/org/org-sdk-node:
  - branch:
    - master: master
  - mirror:
    - sdk/node: /
github.com/org/org-sdk-ruby:
  - branch:
    - master: master
  - mirror:
    - sdk/ruby: /

Backwards compatibility

As our customers use the software, we better understand their needs and begin to design v2.

To do this well, we add new interfaces and deprecate old interfaces in the v1.x series of the SDKs. The server continues to support the v1.x series now and for a well-communicated time period past the release of v2.0 (such as 90 days).

As we release minor versions v1.1 and v1.2, patch versions v1.2.1 and v1.2.2, and eventually v2, we want to carefully ensure backwards compatibility.

We write with-go-sdk, with-node-sdk, and with-ruby-sdk scripts (design credit Keith Rarick):

.
├── ...
└── bin
    ├── with-go-sdk
    ├── with-node-sdk
    └── with-ruby-sdk

We use these scripts to run processes using a given version of our SDKs:

with-go-sdk [version] [command]
with-node-sdk [version] [command]
with-ruby-sdk [version] [command]

The version can be a released version to a registry such as NPM or Rubygems or a special value monorepo to use the current $ORG source code.

For example:

with-ruby-sdk 1.0 ruby -e "require 'org-sdk'; puts OrgSDK::VERSION"

We update the Testfile at the top of the file hierarchy to test supported releases and the current source code (pre-release) on every pull request:

$ cat $ORG/Testfile
gohead: with-serverd with-go-sdk monorepo go test -cover ./...
gov1: with-serverd with-go-sdk 1.5 go test -cover ./...
gov2: with-serverd with-go-sdk 2 go test -cover ./...
# ... etc.
rubyv2: with-serverd with-ruby-sdk 2 $ORG/sdk/ruby/test.sh

The SDK scripts are composable with the with-serverd script.

In addition to running on CI, they are useful for quickly testing bug reports from customers on a particular version, and hopefully providing a great customer experience for them.

Repo integrations

We often want to integrate tools such as Netlify or Heroku with the repo.

Usually, integrating with third-party tools involves providing a manifest like:

cd subdir && command

For Heroku specifically, we found it helpful to write a Monorepo buildpack, which involves adding a heroku.sh file to the subdirectory for Heroku-deployable projects:

.
├── ...
├── cmd
│   └── mirrorbot
│       ├── Procfile
│       └── heroku.sh
└── testbot
    └── farmer
        ├── Procfile
        └── heroku.sh

Conclusion

Working in a monorepo can encourage a feeling of “tight integration”, where service boundaries are well-defined and less likely to be mocked out. Some tasks are a particular pleasure, such as searching across projects for callsites of a function or RPC.

For other tasks, it can help to write custom tools. Writing those custom tools offers an opportunity to design an ideal experience for the engineering team.

In this example, the final directory structure looks something like this:

.
├── .mirrorbot.yml
├── .prettierrc.yml
├── .rubocop.yml
├── .tool-versions
├── README.md
├── Testfile
├── bin
│   ├── setup-go
│   ├── setup-localhost-tls
│   ├── setup-node
│   ├── setup-postgres
│   ├── setup-ruby
│   ├── with-go-sdk
│   ├── with-node-sdk
│   ├── with-ruby-sdk
│   └── with-serverd
├── cmd
│   ├── mirrorbot
│   │   ├── Procfile
│   │   ├── Testfile
│   │   ├── heroku.sh
│   │   ├── main.go
│   │   └── setup.sh
│   ├── serverd
│   │   └── main.go
│   └── testbot
│       └── main.go
├── dashboard
│   ├── Testfile
│   └── setup.sh
├── mobile
│   ├── Testfile
│   ├── app
│   │   ├── android
│   │   └── ios
│   └── setup.sh
├── infra
│   └── setup.sh
├── sdk
│   ├── go
│   │   └── setup.sh
│   ├── node
│   │   ├── Testfile
│   │   └── setup.sh
│   └── ruby
│       ├── Testfile
│       └── setup.sh
├── server
│   ├── Testfile
│   ├── http_handler.go
│   └── setup.sh
└── testbot
    ├── Testfile
    ├── farmer
    │   ├── Procfile
    │   ├── heroku.sh
    │   └── main.go
    ├── setup.sh
    └── worker
        └── main.go

Monorepo was originally published in Interstellar on Medium, where people are continuing the conversation by highlighting and responding to this story.

Interstellar is pleased to announce a preview release of Starlight, an implementation of bidirectional payment channels on Stellar. Payment channels allow parties to transact privately, instantly, and securely, while paying zero fees.

Normally, making a payment on a network like Stellar involves publishing details of the payment to all observers, paying a small fee, and waiting for the network to reach consensus and confirm the payment.

A payment channel, on the other hand, uses the network to “lock up” some funds for use by the channel’s participants. They may then pay each other these funds without further involving the network. Such payments are thus private, instant, and free.

The security of the channel is ensured by special information exchanged between the participants with each payment. This information includes signed Stellar transactions that are not published to the network until one party chooses to close the channel. When the channel is closed, these “settlement transactions” send to each party’s Stellar account their respective share of the balance, taking into account the channel’s history of payments back and forth.

Only the final settlement balances are revealed to the network when the channel closes. All other activity in the channel remains private and known only to its participants.

Starlight payment channels are analogous to the Bitcoin payment channels used in the Lightning Network. As we extend the Starlight project to support not only payment channels but also multi-hop payments across those channels, we plan to build in compatibility with other payment-channel networks, such as Lightning and Interledger.

This preview release supports bilateral channels (channels with exactly two participants). It includes the Starlight payment channel software as well as a built-in wallet application. It allows transacting in lumens only (the native asset of the Stellar network). Future versions will support any asset issued on the Stellar network.

This is an early demo release, and you should expect bugs as well as frequent incompatible upgrades. Until we work out the kinks, Starlight connects only to the Stellar testnet, not the mainnet, to eliminate the risk of losing real funds to a software bug. Please do not attempt to use this application with the Stellar mainnet.

You can download the Starlight application, read the source code, and try the demo here. You can learn more about the Starlight payment channel protocol by reading the specification.

Thank you to Jeremy Rubin, Jed McCaleb, Nicolas Barry, David Mazières, Akash Khosla, and others for inventing and developing the designs for payment channels on the Stellar protocol.

Starlight: payment channels on Stellar was originally published in Interstellar on Medium, where people are continuing the conversation by highlighting and responding to this story.

FOR IMMEDIATE RELEASE

Chain Merges with Lightyear, Goes All-In on Stellar

Combined company, to be called Interstellar, will help organizations build on the Stellar Network; Stellar Development Foundation to remain independent

SAN FRANCISCO, September 10, 2018 — Chain, Inc. (“Chain”) today announced it has been acquired by Lightyear Corporation (“Lightyear”), a Stellar-focused commercial entity formed last year with the support of the Stellar Development Foundation. The Chain and Lightyear brands will be retired and the combined company will be re-named to Interstellar.

The merger brings Chain’s enterprise products and customer base to Stellar’s global public ledger, creating an end-to-end solution that will enable organizations to issue, exchange, and manage assets on a highly-scalable public network. Chain’s cloud ledger service, Sequence, will allow organizations to easily track assets as they move between private ledgers and the Stellar network.

Key facts:

• Adam Ludwin, who is Chain’s CEO, will be Interstellar’s CEO
• Jed McCaleb, who co-founded the Stellar Development Foundation and Lightyear, will be CTO of Interstellar
• The Stellar Development Foundation, which develops the Stellar protocol and supports the open source community, remains independent

Stellar is an open network that allows any currency or asset to be digitally issued, transferred, and exchanged over the internet. Interstellar will make it easy for developers and enterprises to leverage Stellar as a platform to build novel financial products and services.

“Chain’s team has led the market for enterprise adoption of blockchain technology, which is a critical component of building a future where money and digital assets move over open protocols,” said Jed McCaleb. “We are thrilled to be joining forces to help organizations build on Stellar.”

“Chain has admired Stellar for years, and from day one we’ve shared their goal of enabling financial assets to move seamlessly over the internet,” said Adam Ludwin. “Chain has worked from inside the enterprise while Stellar has focused on the network between organizations. As a single team we will have a complete view and set of capabilities to make value-over-IP a reality.”

Interstellar’s product portfolio will also include StellarX, a recently announced marketplace for trading assets on Stellar. StellarX is currently in beta and will be launching to the public soon.

At close, Interstellar will employ 60 people, with its headquarters in San Francisco and offices in New York City and Singapore.

More information is available at www.interstellar.com.

About the Stellar Network

Stellar is an open network that allows any currency or asset to be digitally issued, transferred, and exchanged over the internet. The Stellar public ledger has a built-in order book, pathfinding algorithm, and network token that together allow assets to move seamlessly between participants. The Stellar network is open source and validated by decentralized nodes that anyone can run, yet it is highly scalable and energy efficient thanks to a novel consensus system called SCP. SCP was developed by professor David Mazières, who leads the secure computer systems group at Stanford and is the Chief Scientist at the Stellar Development Foundation.

About the Stellar Development Foundation and Lightyear Corporation

The Stellar Development Foundation (SDF) is a non-profit founded in 2014. It helps build and maintain the Stellar network and supports the Stellar community through education, grant programs, and technical leadership of Stellar’s open source projects. The Foundation is obligated by its charter to distribute 95% of the network’s token, lumen (XLM), to ensure the operational and economic decentralization of the network. SDF received its initial funding from payments innovator Stripe.

Lightyear, a for-profit entity focused on adoption of the Stellar network by enterprise, was founded by Brit Yonge and Jed McCaleb in 2017 with support from SDF. Lightyear will be re-named to Interstellar concurrent with the merger.

About Chain

Chain was founded in 2014 by Adam Ludwin, Devon Gundry and Ryan Smith. The company builds enterprise-grade blockchain products leveraged by leading financial organizations including Visa, Nasdaq, and Citigroup. Chain’s cloud product, Sequence, will become an important component of Interstellar’s product portfolio and enable enterprises to issue, manage, and track assets as they move between private ledgers and the Stellar network. The company previously raised $44M from Khosla Ventures, RRE Ventures, Thrive Capital, and strategic partners including Visa, Nasdaq, Citi, Capital One, Fiserv, and Orange.

Announcing Interstellar was originally published in Interstellar on Medium, where people are continuing the conversation by highlighting and responding to this story.