9ce10769ce
This includes fixes that improve gossip scalability on very large (> 10k node) clusters. The Serf changes: - take snapshot disk IO out of the critical path for handling messages hashicorp/serf#524 - make snapshot compaction much less aggressive - the old fixed threshold caused snapshots to be constantly compacted (synchronously with request handling) on clusters larger than about 2000 nodes! hashicorp/serf#525 Memberlist changes: - prioritize handling alive messages over suspect/dead to improve stability, and handle queue in LIFO order to avoid acting on info that 's already stale in the queue by the time we handle it. hashicorp/memberlist#159 - limit the number of concurrent pushPull requests being handled at once to 128. In one test scenario with 10s of thousands of servers we saw channel and lock blocking cause over 3000 pushPulls at once which ballooned the memory of the server because each push pull contained a de-serialised list of all known 10k+ nodes and their tags for a total of about 60 million objects and 7GB of memory stuck. While the rest of the fixes here should prevent the same root cause from blocking in the same way, this prevents any other bug or source of contention from allowing pushPull messages to stack up and eat resources. hashicorp/memberlist#158
Consul 
- Website: https://www.consul.io
- Chat: Gitter
- Mailing list: Google Groups
Consul is a tool for service discovery and configuration. Consul is distributed, highly available, and extremely scalable.
Consul provides several key features:
-
Service Discovery - Consul makes it simple for services to register themselves and to discover other services via a DNS or HTTP interface. External services such as SaaS providers can be registered as well.
-
Health Checking - Health Checking enables Consul to quickly alert operators about any issues in a cluster. The integration with service discovery prevents routing traffic to unhealthy hosts and enables service level circuit breakers.
-
Key/Value Storage - A flexible key/value store enables storing dynamic configuration, feature flagging, coordination, leader election and more. The simple HTTP API makes it easy to use anywhere.
-
Multi-Datacenter - Consul is built to be datacenter aware, and can support any number of regions without complex configuration.
-
Service Segmentation - Consul Connect enables secure service-to-service communication with automatic TLS encryption and identity-based authorization.
Consul runs on Linux, Mac OS X, FreeBSD, Solaris, and Windows. A commercial version called Consul Enterprise is also available.
Please note: We take Consul's security and our users' trust very seriously. If you believe you have found a security issue in Consul, please responsibly disclose by contacting us at security@hashicorp.com.
Quick Start
An extensive quick start is viewable on the Consul website:
https://www.consul.io/intro/getting-started/install.html
Documentation
Full, comprehensive documentation is viewable on the Consul website:
Developing Consul
If you wish to work on Consul itself, you'll first need Go installed (version 1.9+ is required). Make sure you have Go properly installed, including setting up your GOPATH.
Next, clone this repository into $GOPATH/src/github.com/hashicorp/consul and
then just type make. In a few moments, you'll have a working consul executable:
$ make
...
$ bin/consul
...
Note: make will build all os/architecture combinations. Set the environment variable CONSUL_DEV=1 to build it just for your local machine's os/architecture, or use make dev.
Note: make will also place a copy of the binary in the first part of your $GOPATH.
You can run tests by typing make test. The test suite may fail if
over-parallelized, so if you are seeing stochastic failures try
GOTEST_FLAGS="-p 2 -parallel 2" make test.
If you make any changes to the code, run make format in order to automatically
format the code according to Go standards.
Vendoring
Consul currently uses govendor for
vendoring and vendorfmt for formatting
vendor.json to a more merge-friendly "one line per package" format.