Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure. https://www.consul.io
Go to file
James Phillips c01a3871c9 Adds support for snapshots and restores. (#2396)
* Updates Raft library to get new snapshot/restore API.

* Basic backup and restore working, but need some cleanup.

* Breaks out a snapshot module and adds a SHA256 integrity check.

* Adds snapshot ACL and fills in some missing comments.

* Require a consistent read for snapshots.

* Make sure snapshot works if ACLs aren't enabled.

* Adds a bit of package documentation.

* Returns an empty response from restore to avoid EOF errors.

* Adds API client support for snapshots.

* Makes internal file names match on-disk file snapshots.

* Adds DC and token coverage for snapshot API test.

* Adds missing documentation.

* Adds a unit test for the snapshot client endpoint.

* Moves the connection pool out of the client for easier testing.

* Fixes an incidental issue in the prepared query unit test.

I realized I had two servers in bootstrap mode so this wasn't a good setup.

* Adds a half close to the TCP stream and fixes panic on error.

* Adds client and endpoint tests for snapshots.

* Moves the pool back into the snapshot RPC client.

* Adds a TLS test and fixes half-closes for TLS connections.

* Tweaks some comments.

* Adds a low-level snapshot test.

This is independent of Consul so we can pull this out into a library
later if we want to.

* Cleans up snapshot and archive and completes archive tests.

* Sends a clear error for snapshot operations in dev mode.

Snapshots require the Raft snapshots to be readable, which isn't supported
in dev mode. Send a clear error instead of a deep-down Raft one.

* Adds docs for the snapshot endpoint.

* Adds a stale mode and index feedback for snapshot saves.

This gives folks a way to extract data even if the cluster has no
leader.

* Changes the internal format of a snapshot from zip to tgz.

* Pulls in Raft fix to cancel inflight before a restore.

* Pulls in new Raft restore interface.

* Adds metadata to snapshot saves and a verify function.

* Adds basic save and restore snapshot CLI commands.

* Gets rid of tarball extensions and adds restore message.

* Fixes an incidental bad link in the KV docs.

* Adds documentation for the snapshot CLI commands.

* Scuttle any request body when a snapshot is saved.

* Fixes archive unit test error message check.

* Allows for nil output writers in snapshot RPC handlers.

* Renames hash list Decode to DecodeAndVerify.

* Closes the client connection for snapshot ops.

* Lowers timeout for restore ops.

* Updates Raft vendor to get new Restore signature and integrates with Consul.

* Bounces the leader's internal state when we do a restore.
2016-10-25 19:20:24 -07:00
acl Adds support for snapshots and restores. (#2396) 2016-10-25 19:20:24 -07:00
api Adds support for snapshots and restores. (#2396) 2016-10-25 19:20:24 -07:00
bench Fix Consul download link in benchmark scripts 2016-02-10 14:18:19 -02:00
command Adds support for snapshots and restores. (#2396) 2016-10-25 19:20:24 -07:00
consul Adds support for snapshots and restores. (#2396) 2016-10-25 19:20:24 -07:00
contrib Merge pull request #1863 from mssola/bash-completion 2016-07-05 12:52:58 -07:00
demo/vagrant-cluster update vagrant demo 2016-04-27 17:52:23 -04:00
lib Use a cryptographically secure seed 2016-05-02 23:52:37 -07:00
scripts Use the `gox` binary found in a user's $PATH, which we assume includes 2016-08-10 23:55:40 -07:00
terraform key_file is deprecated. Switching to private_key using the file() interpolation function. 2016-09-24 16:36:04 -05:00
test Adds support for snapshots and restores. (#2396) 2016-10-25 19:20:24 -07:00
testutil Adds performance tuning capability for Raft, detuned defaults, and supplemental docs. 2016-08-24 21:58:37 -07:00
tlsutil tlsutil: Testing hostname verification 2015-05-11 16:05:39 -07:00
types Revert "Move `structs.CheckID` to a new top-level package, `types`." 2016-06-07 16:59:02 -04:00
ui Updates built-in UI static assets to latest. 2016-08-12 18:01:59 -07:00
vendor Adds support for snapshots and restores. (#2396) 2016-10-25 19:20:24 -07:00
watch Moves stale code to same spot and adds it everywhere that supports it. 2016-08-10 15:49:51 -07:00
website Adds support for snapshots and restores. (#2396) 2016-10-25 19:20:24 -07:00
.gitignore Use gox for building 2015-10-22 14:16:01 -04:00
.travis.yml Only builds the binary stuff under master branch CI. 2016-10-25 13:49:57 -07:00
CHANGELOG.md Update CHANGELOG.md 2016-10-06 22:11:48 -07:00
GNUmakefile Only builds the binary stuff under master branch CI. 2016-10-25 13:49:57 -07:00
ISSUE_TEMPLATE.md Revise issue template to include a hint to use a gist 2016-04-28 22:27:25 -07:00
LICENSE Initial commit 2013-11-04 14:15:27 -08:00
README.md Update to the higher quality Travis build icon 2016-10-25 18:11:35 -07:00
commands.go Adds support for snapshots and restores. (#2396) 2016-10-25 19:20:24 -07:00
main.go Always seed math/rand on consul startup 2016-01-29 17:00:08 -08:00
main_test.go Adding basic CLI infrastructure 2013-12-19 11:22:08 -08:00
make.bat Removes the integration test runner, there weren't any tests using it. 2015-10-26 11:34:01 -07:00
version.go Puts tree back into development mode. 2016-09-14 09:20:31 -07:00

README.md

Consul Build Status Join the chat at https://gitter.im/hashicorp-consul/Lobby

Consul is a tool for service discovery and configuration. Consul is distributed, highly available, and extremely scalable.

Consul provides several key features:

  • Service Discovery - Consul makes it simple for services to register themselves and to discover other services via a DNS or HTTP interface. External services such as SaaS providers can be registered as well.

  • Health Checking - Health Checking enables Consul to quickly alert operators about any issues in a cluster. The integration with service discovery prevents routing traffic to unhealthy hosts and enables service level circuit breakers.

  • Key/Value Storage - A flexible key/value store enables storing dynamic configuration, feature flagging, coordination, leader election and more. The simple HTTP API makes it easy to use anywhere.

  • Multi-Datacenter - Consul is built to be datacenter aware, and can support any number of regions without complex configuration.

Consul runs on Linux, Mac OS X, FreeBSD, Solaris, and Windows.

Quick Start

An extensive quick start is viewable on the Consul website:

https://www.consul.io/intro/getting-started/install.html

Documentation

Full, comprehensive documentation is viewable on the Consul website:

https://www.consul.io/docs

Developing Consul

If you wish to work on Consul itself, you'll first need Go installed (version 1.6+ is required). Make sure you have Go properly installed, including setting up your GOPATH.

Next, clone this repository into $GOPATH/src/github.com/hashicorp/consul and then just type make. In a few moments, you'll have a working consul executable:

$ make
...
$ bin/consul
...

Note: make will also place a copy of the binary in the first part of your $GOPATH.

You can run tests by typing make test.

If you make any changes to the code, run make format in order to automatically format the code according to Go standards.

Building Consul on Windows

Make sure Go 1.6+ is installed on your system and that the Go command is in your %PATH%.

For building Consul on Windows, you also need to have MinGW installed. TDM-GCC is a simple bundle installer which has all the required tools for building Consul with MinGW.

Install TDM-GCC and make sure it has been added to your %PATH%.

If all goes well, you should be able to build Consul by running make.bat from a command prompt.

See also golang/winstrap and golang/wiki/WindowsBuild for more information of how to set up a general Go build environment on Windows with MinGW.

Vendoring

Consul currently uses govendor for vendoring.