Commit Graph

181 Commits

Author SHA1 Message Date
James Phillips c01a3871c9 Adds support for snapshots and restores. (#2396)
* Updates Raft library to get new snapshot/restore API.

* Basic backup and restore working, but need some cleanup.

* Breaks out a snapshot module and adds a SHA256 integrity check.

* Adds snapshot ACL and fills in some missing comments.

* Require a consistent read for snapshots.

* Make sure snapshot works if ACLs aren't enabled.

* Adds a bit of package documentation.

* Returns an empty response from restore to avoid EOF errors.

* Adds API client support for snapshots.

* Makes internal file names match on-disk file snapshots.

* Adds DC and token coverage for snapshot API test.

* Adds missing documentation.

* Adds a unit test for the snapshot client endpoint.

* Moves the connection pool out of the client for easier testing.

* Fixes an incidental issue in the prepared query unit test.

I realized I had two servers in bootstrap mode so this wasn't a good setup.

* Adds a half close to the TCP stream and fixes panic on error.

* Adds client and endpoint tests for snapshots.

* Moves the pool back into the snapshot RPC client.

* Adds a TLS test and fixes half-closes for TLS connections.

* Tweaks some comments.

* Adds a low-level snapshot test.

This is independent of Consul so we can pull this out into a library
later if we want to.

* Cleans up snapshot and archive and completes archive tests.

* Sends a clear error for snapshot operations in dev mode.

Snapshots require the Raft snapshots to be readable, which isn't supported
in dev mode. Send a clear error instead of a deep-down Raft one.

* Adds docs for the snapshot endpoint.

* Adds a stale mode and index feedback for snapshot saves.

This gives folks a way to extract data even if the cluster has no
leader.

* Changes the internal format of a snapshot from zip to tgz.

* Pulls in Raft fix to cancel inflight before a restore.

* Pulls in new Raft restore interface.

* Adds metadata to snapshot saves and a verify function.

* Adds basic save and restore snapshot CLI commands.

* Gets rid of tarball extensions and adds restore message.

* Fixes an incidental bad link in the KV docs.

* Adds documentation for the snapshot CLI commands.

* Scuttle any request body when a snapshot is saved.

* Fixes archive unit test error message check.

* Allows for nil output writers in snapshot RPC handlers.

* Renames hash list Decode to DecodeAndVerify.

* Closes the client connection for snapshot ops.

* Lowers timeout for restore ops.

* Updates Raft vendor to get new Restore signature and integrates with Consul.

* Bounces the leader's internal state when we do a restore.
2016-10-25 19:20:24 -07:00
Adam Wolfe Gordon de4fcf9a4e vendor: Remove go-reap
go-reap is no longer used since we no longer reap child processes.
2016-10-04 09:41:29 -06:00
R.B. Boyer 1c4b4ac54f
Validate gossip encryption key before made persistent in local.keyring 2016-08-12 11:42:45 -07:00
James Phillips f1fbfb9423
Updates hashicorp/yamux. 2016-08-09 17:24:41 -07:00
James Phillips a07938dc45
Updates hashicorp/scada-client. 2016-08-09 17:24:40 -07:00
James Phillips 13f89e68c5
Updates hashicorp/memberlist. 2016-08-09 17:24:40 -07:00
James Phillips 3b9ab553b7
Updates hashicorp/logutils. 2016-08-09 17:24:40 -07:00
James Phillips ccb86d20b0
Updates hashicorp/go-uuid. 2016-08-09 17:24:40 -07:00
James Phillips 7aedab2407
Updates hashicorp/go-syslog. 2016-08-09 17:24:40 -07:00
James Phillips a4ad479ce1
Updates hashicorp/go-retryablehttp. 2016-08-09 17:24:39 -07:00
James Phillips 952cabfad4
Updates hashicorp/go-reap. 2016-08-09 17:24:39 -07:00
James Phillips df69b76972
Updates hashicorp/go-cleanhttp. 2016-08-09 17:24:39 -07:00
James Phillips 7b97b570f2
Updates hashicorp/go-immutable-radix and hashicorp/go-memdb. 2016-08-09 17:24:39 -07:00
James Phillips ce141896df
Updates hashicorp/hcl and hashicorp/hil.
This required a small mod to core Consul code to cope with an interface
change.
2016-08-09 17:24:13 -07:00
James Phillips 5dc9d92fe0
Removes some unused vendored dependencies. 2016-08-09 16:50:34 -07:00
James Phillips 1eca282bee
Updates hashicorp/golang-lru. 2016-08-09 16:26:59 -07:00
James Phillips cc1f709333
Vendors first stage branch of the v2 Raft library. 2016-08-08 19:19:17 -07:00
James Phillips 8ec559a235
Updates Serf to pick up intent queue fix.
This fixes #1062 by storing intents per-node instead of in a small, fixed-
size circular buffer.
2016-08-08 18:58:44 -07:00
James Phillips 2f3e346dc1 Updates armon/go-metrics and adds Circonus vendored libraries. 2016-07-19 16:54:21 -07:00
James Phillips a46ed7fa32 Updates go-immutable-radix to pull in caching fix. 2016-06-08 19:08:31 -07:00
James Phillips b5f1c41aef Updates serf to get health score in stats. 2016-06-08 17:20:12 -07:00
James Phillips 785cb70cd3 Updates to latest Serf/memberlist to get lifeguard and TCP joins over DNS. 2016-06-07 14:30:47 -07:00
James Phillips a8bafda8b6 Pulls in latest Serf to get flap metric. 2016-06-07 09:22:41 -07:00
Jeff Mitchell ebf7ea1d75 Use upstream high-level SCADA provider 2016-06-01 18:25:39 -04:00
Sean Chittenden bf8c860663 Update Serf to include `serf.NumNodes()` 2016-03-23 22:10:50 -07:00
James Phillips 328d138466 Adds in basic query template lookups and vendors newly-updated memdb as well as improved iradix tree. 2016-03-07 10:45:39 -08:00
James Phillips d4b01a6c53 Updates vendored dependencies.
This removes some unused dependencies that I know we plan to re-add later,
but this makes for a clean `godep save` for now.
2016-03-07 10:45:39 -08:00
James Phillips 9139cee5e5 Updates memberlist to get goroutine leak fix for TCP pings. 2016-03-07 10:41:54 -08:00
Mat Schaffer 3142a51688 Metric name typo 2016-02-25 11:20:53 +09:00
Jeff Mitchell 14a3a14543 Update go-cleanhttp 2016-02-17 17:03:57 -05:00
Sean Chittenden 25b09713b9 Manage dependencies via Godep
Embrace the future and use Go 1.6's vendor support via Godep.

Go 1.5 users should `export GO15VENDOREXPERIMENT=1`
2016-02-12 16:50:37 -08:00