In Search of a Scalable Geo-Distributed Consensus Protocol

Abstract

Byzantine fault tolerant consensus protocols are a crucial component in blockchain systems. Traditional BFT consensus protocols have poor scalability, and their performance is sensitive to the latency between their participants, which leads to low performance in a geo-distributed deployment. RCanopus is a consensus protocol that aims to provide high throughput and good scalability in a geo-distributed environment. It organizes participants into a hierarchical structure that is topology-aware. We implemented an ordering service for HyperLedger Fabric with RCanopus, and evaluated its performance on AWS. Our implementation uses SBFT internally to provide BFT consensus. Comparing to running SBFT across all datacenters, RCanopus is able to achieve a 10.7x increase in peak throughput in a deployment across 4 AWS regions. During our evaluation, we identi ed several design limitations that may a ect its performance or safety property. Therefore, we proposed five protocol extensions to further improve RCanopus in various aspects. This includes handling stragglers, handling failures of entire Byzantine Groups, reducing the bandwidth usage and removing duplicated transactions. We implemented a prototype in order to evaluate the extensions that are not included in our ordering service. Our evaluation results show that the new extensions can improve the peak throughput by 12.7% to 114.3%, depending on the available bandwidth on wide area links.