consul/website/pages/docs/architecture/jepsen.mdx

123 lines
4.9 KiB
Plaintext
Raw Normal View History

2014-04-17 00:00:37 +00:00
---
2020-04-07 18:55:19 +00:00
layout: docs
page_title: Jepsen Testing
sidebar_title: Jepsen Testing
2020-04-07 18:55:19 +00:00
description: >-
Jepsen is a tool, written by Kyle Kingsbury, designed to test the partition
tolerance of distributed systems. It creates network partitions while fuzzing
the system with random operations. The results are analyzed to see if the
system violates any of the consistency properties it claims to have.
2014-04-17 00:00:37 +00:00
---
# Jepsen Testing
[Jepsen](http://aphyr.com/posts/281-call-me-maybe-carly-rae-jepsen-and-the-perils-of-network-partitions)
is a tool, written by Kyle Kingsbury, designed to test the partition
2014-04-17 00:29:37 +00:00
tolerance of distributed systems. It creates network partitions while fuzzing
the system with random operations. The results are analyzed to see if the system
violates any of the consistency properties it claims to have.
2014-04-17 00:00:37 +00:00
As part of our Consul testing, we ran a Jepsen test to determine if
any consistency issues could be uncovered. In our testing, Consul
gracefully recovered from partitions without introducing any consistency
issues.
2014-04-17 00:00:37 +00:00
2014-04-17 00:03:33 +00:00
## Running the tests
At the moment, testing with Jepsen is rather complex as it requires
2014-04-17 00:29:37 +00:00
setting up multiple virtual machines, SSH keys, DNS configuration,
and a working Clojure environment. We hope to contribute our Consul
testing code upstream and to provide a Vagrant environment for Jepsen
testing soon.
2014-04-17 00:03:33 +00:00
2014-04-17 00:00:37 +00:00
## Output
Below is the output captured from Jepsen. We ran Jepsen multiple times,
and it passed each time. This output is only representative of a single
2020-10-14 20:13:38 +00:00
run and has been edited for length. Please reach out on [Consul's Discuss](https://discuss.hashicorp.com/c/consul)
if you would like to reproduce the Jepsen results.
2014-04-17 00:00:37 +00:00
<!--googleoff: all-->
2020-05-19 18:32:38 +00:00
```shell-session
2014-04-17 00:00:37 +00:00
$ lein test :only jepsen.system.consul-test
lein test jepsen.system.consul-test
INFO jepsen.os.debian - :n5 setting up debian
INFO jepsen.os.debian - :n3 setting up debian
INFO jepsen.os.debian - :n4 setting up debian
INFO jepsen.os.debian - :n1 setting up debian
INFO jepsen.os.debian - :n2 setting up debian
INFO jepsen.os.debian - :n4 debian set up
INFO jepsen.os.debian - :n5 debian set up
INFO jepsen.os.debian - :n3 debian set up
INFO jepsen.os.debian - :n1 debian set up
INFO jepsen.os.debian - :n2 debian set up
INFO jepsen.system.consul - :n1 consul nuked
INFO jepsen.system.consul - :n4 consul nuked
INFO jepsen.system.consul - :n5 consul nuked
INFO jepsen.system.consul - :n3 consul nuked
INFO jepsen.system.consul - :n2 consul nuked
INFO jepsen.system.consul - Running nodes: {:n1 false, :n2 false, :n3 false, :n4 false, :n5 false}
INFO jepsen.system.consul - :n2 consul nuked
INFO jepsen.system.consul - :n3 consul nuked
INFO jepsen.system.consul - :n4 consul nuked
INFO jepsen.system.consul - :n5 consul nuked
INFO jepsen.system.consul - :n1 consul nuked
INFO jepsen.system.consul - :n1 starting consul
INFO jepsen.system.consul - :n2 starting consul
INFO jepsen.system.consul - :n4 starting consul
INFO jepsen.system.consul - :n5 starting consul
INFO jepsen.system.consul - :n3 starting consul
INFO jepsen.system.consul - :n3 consul ready
INFO jepsen.system.consul - :n2 consul ready
INFO jepsen.system.consul - Running nodes: {:n1 true, :n2 true, :n3 true, :n4 true, :n5 true}
INFO jepsen.system.consul - :n5 consul ready
INFO jepsen.system.consul - :n1 consul ready
INFO jepsen.system.consul - :n4 consul ready
INFO jepsen.core - Worker 0 starting
INFO jepsen.core - Worker 2 starting
INFO jepsen.core - Worker 1 starting
INFO jepsen.core - Worker 3 starting
INFO jepsen.core - Worker 4 starting
INFO jepsen.util - 2 :invoke :read nil
INFO jepsen.util - 3 :invoke :cas [4 4]
INFO jepsen.util - 0 :invoke :write 4
INFO jepsen.util - 1 :invoke :write 1
INFO jepsen.util - 4 :invoke :cas [4 0]
INFO jepsen.util - 2 :ok :read nil
INFO jepsen.util - 4 :fail :cas [4 0]
2020-10-14 20:13:38 +00:00
(Log Truncated...)
2014-04-17 00:00:37 +00:00
INFO jepsen.util - 4 :invoke :cas [3 3]
INFO jepsen.util - 4 :fail :cas [3 3]
INFO jepsen.util - :nemesis :info :stop nil
INFO jepsen.util - :nemesis :info :stop "fully connected"
INFO jepsen.util - 0 :fail :read nil
INFO jepsen.util - 1 :fail :write 0
INFO jepsen.util - :nemesis :info :stop nil
INFO jepsen.util - :nemesis :info :stop "fully connected"
INFO jepsen.core - nemesis done
INFO jepsen.core - Worker 3 done
INFO jepsen.util - 1 :invoke :read nil
INFO jepsen.core - Worker 2 done
INFO jepsen.core - Worker 4 done
INFO jepsen.core - Worker 0 done
INFO jepsen.util - 1 :ok :read 3
INFO jepsen.core - Worker 1 done
INFO jepsen.core - Run complete, writing
INFO jepsen.core - Analyzing
2020-10-14 20:13:38 +00:00
(Log Truncated...)
2014-04-17 00:00:37 +00:00
INFO jepsen.core - Analysis complete
INFO jepsen.system.consul - :n3 consul nuked
INFO jepsen.system.consul - :n2 consul nuked
INFO jepsen.system.consul - :n4 consul nuked
INFO jepsen.system.consul - :n1 consul nuked
INFO jepsen.system.consul - :n5 consul nuked
1964 element history linearizable. :D
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
```
<!--googleon: all-->