Skip to Main content Skip to Navigation
Conference papers

Testing consensus implementations using communication closure

Abstract : Large scale production distributed systems are difficult to design and test. Correctness must be ensured when processes run asynchronously, at arbitrary rates relative to each other, and in the presence of failures, e.g., process crashes or message losses. These conditions create a huge space of executions that is difficult to explore in a principled way. Current testing techniques focus on systematic or randomized exploration of all executions of an implementation while treating the implemented algorithms as black boxes. On the other hand, proofs of correctness of many of the underlying algorithms often exploit semantic properties that reduce reasoning about correctness to a subset of behaviors. For example, the communication-closure property, used in many proofs of distributed consensus algorithms, shows that every asynchronous execution of the algorithm is equivalent to a lossy synchronous execution, thus reducing the burden of proof to only that subset. In a lossy synchronous execution, processes execute in lock-step rounds, and messages are either received in the same round or lost forever-such executions form a small subset of all asynchronous ones. We formulate the communication-closure hypothesis, which states that bugs in implementations of distributed consensus algorithms will already manifest in lossy synchronous executions and present a testing algorithm based on this hypothesis. We prioritize the search space based on a bound on the number of failures in the execution and the rate at which these failures are recovered. We show that a random testing algorithm based on sampling lossy synchronous executions can empirically find a number of bugs-including previously unknown ones-in production distributed systems such as Zookeeper, Cassandra, and Ratis, and also produce more understandable bug traces.
Document type :
Conference papers
Complete list of metadata

https://hal.inria.fr/hal-03134294
Contributor : Cezara Dragoi Connect in order to contact the contributor
Submitted on : Monday, February 8, 2021 - 11:18:57 AM
Last modification on : Friday, January 21, 2022 - 3:19:49 AM
Long-term archiving on: : Sunday, May 9, 2021 - 6:41:16 PM

File

main.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Cezara Dragoi, Constantin Enea, Burcu Kulahcioglu Ozkan, Rupak Majumdar, Filip Niksic. Testing consensus implementations using communication closure. SPLASH 2020 : ACM SIGPLAN conference on Systems, Programming, Languages, and Applications: Software for Humanity, Oct 2021, Chiccago / Virtual, United States. ⟨10.1145/3428278⟩. ⟨hal-03134294⟩

Share

Metrics

Les métriques sont temporairement indisponibles