status-im
/
consul
mirror of https://github.com/status-im/consul.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
10,223 Commits 445 Branches 179 Tags 489 MiB
Commit Graph

601 Commits

Author SHA1 Message Date
Alex Dadgar 43d0f96c42 do not bootstrap with non voters 2018-09-19 17:41:36 -07:00
Kyle Havlovitz d515d25856
Merge pull request #4644 from hashicorp/ca-refactor 
connect/ca: rework initialization/root generation in providers
 2018-09-13 13:08:34 -07:00
Paul Banks 74f2a80a42
Fix CA pruning when CA config uses string durations. (#4669) 
* Fix CA pruning when CA config uses string durations.

The tl;dr here is:

 - Configuring LeafCertTTL with a string like "72h" is how we do it by default and should be supported
 - Most of our tests managed to escape this by defining them as time.Duration directly
 - Out actual default value is a string
 - Since this is stored in a map[string]interface{} config, when it is written to Raft it goes through a msgpack encode/decode cycle (even though it's written from server not over RPC).
 - msgpack decode leaves the string as a `[]uint8`
 - Some of our parsers required string and failed
 - So after 1 hour, a default configured server would throw an error about pruning old CAs
 - If a new CA was configured that set LeafCertTTL as a time.Duration, things might be OK after that, but if a new CA was just configured from config file, intialization would cause same issue but always fail still so would never prune the old CA.
 - Mostly this is just a janky error that got passed tests due to many levels of complicated encoding/decoding.

tl;dr of the tl;dr: Yay for type safety. Map[string]interface{} combined with msgpack always goes wrong but we somehow get bitten every time in a new way :D

We already fixed this once! The main CA config had the same problem so @kyhavlov already wrote the mapstructure DecodeHook that fixes it. It wasn't used in several places it needed to be and one of those is notw in `structs` which caused a dependency cycle so I've moved them.

This adds a whole new test thta explicitly tests the case that broke here. It also adds tests that would have failed in other places before (Consul and Vaul provider parsing functions). I'm not sure if they would ever be affected as it is now as we've not seen things broken with them but it seems better to explicitly test that and support it to not be bitten a third time!

* Typo fix

* Fix bad Uint8 usage
 2018-09-13 15:43:00 +01:00
Pierre Souchay 1a906ef34e Fix more unstable tests in agent and command 2018-09-12 14:49:27 +01:00
Kyle Havlovitz c112a72880
connect/ca: some cleanup and reorganizing of the new methods 2018-09-11 16:43:04 -07:00
Pierre Souchay 22500f242e Fix unstable tests in agent, api, and command/watch 2018-09-10 16:58:53 +01:00
Pierre Souchay eddcf228ea Implementation of Weights Data structures (#4468) 
* Implementation of Weights Data structures

Adding this datastructure will allow us to resolve the
issues #1088 and #4198

This new structure defaults to values:
```
   { Passing: 1, Warning: 0 }
```

Which means, use weight of 0 for a Service in Warning State
while use Weight 1 for a Healthy Service.
Thus it remains compatible with previous Consul versions.

* Implemented weights for DNS SRV Records

* DNS properly support agents with weight support while server does not (backwards compatibility)

* Use Warning value of Weights of 1 by default

When using DNS interface with only_passing = false, all nodes
with non-Critical healthcheck used to have a weight value of 1.
While having weight.Warning = 0 as default value, this is probably
a bad idea as it breaks ascending compatibility.

Thus, we put a default value of 1 to be consistent with existing behaviour.

* Added documentation for new weight field in service description

* Better documentation about weights as suggested by @banks

* Return weight = 1 for unknown Check states as suggested by @banks

* Fixed typo (of -> or) in error message as requested by @mkeeler

* Fixed unstable unit test TestRetryJoin

* Fixed unstable tests

* Fixed wrong Fatalf format in `testrpc/wait.go`

* Added notes regarding DNS SRV lookup limitations regarding number of instances

* Documentation fixes and clarification regarding SRV records with weights as requested by @banks

* Rephrase docs
 2018-09-07 15:30:47 +01:00
Kyle Havlovitz 546bdf8663
connect/ca: add Configure/GenerateRoot to provider interface 2018-09-06 19:18:59 -07:00
Pierre Souchay 9a2ae6e8eb Fixed more flaky tests in ./agent/consul (#4617) 2018-09-04 14:02:47 +01:00
Freddy d7a404f2ee
Bugfix: Use "%#v" when formatting structs (#4600) 2018-08-28 12:37:34 -04:00
Pierre Souchay b898131723 [BUGFIX] Avoid returning empty data on startup of a non-leader server (#4554) 
Ensure that DB is properly initialized when performing stale queries

Addresses:
- https://github.com/hashicorp/consul-replicate/issues/82
- https://github.com/hashicorp/consul/issues/3975
- https://github.com/hashicorp/consul-template/issues/1131
 2018-08-23 12:06:39 -04:00
Kyle Havlovitz e5e1f867e5
Merge branch 'master' into ca-snapshot-fix 2018-08-16 13:00:54 -07:00
Kyle Havlovitz f186edc42c
fsm: add connect service config to snapshot/restore test 2018-08-16 12:58:54 -07:00
nickmy9729 beddf03b26 Added code to allow snapshot inclusion of NodeMeta (#4527) 2018-08-16 15:33:35 -04:00
Kyle Havlovitz b51d76f469
fsm: add missing CA config to snapshot/restore logic 2018-08-16 11:58:50 -07:00
Kyle Havlovitz 4b35d877ca
autopilot: don't follow the normal server removal rules for nonvoters 2018-08-14 14:24:51 -07:00
Kyle Havlovitz ea14482376
Fix stats fetcher healthcheck RPCs not being independent 2018-08-14 14:23:52 -07:00
Pierre Souchay 0d6de257a2 Display more information about check being not properly added when it fails (#4405) 
* Display more information about check being not properly added when it fails

It follows an incident where we add lots of error messages:

  [WARN] consul.fsm: EnsureRegistration failed: failed inserting check: Missing service registration

That seems related to Consul failing to restart on respective agents.

Having Node information as well as service information would help diagnose the issue.

* Renamed ensureCheckIfNodeMatches() as requested by @banks
 2018-08-14 17:45:33 +01:00
Pierre Souchay ef3b81ab13 Allow to rename nodes with IDs, will fix #3974 and #4413 (#4415) 
* Allow to rename nodes with IDs, will fix #3974 and #4413

This change allow to rename any well behaving recent agent with an
ID to be renamed safely, ie: without taking the name of another one
with case insensitive comparison.

Deprecated behaviour warning
----------------------------

Due to asceding compatibility, it is still possible however to
"take" the name of another name by not providing any ID.

Note that when not providing any ID, it is possible to have 2 nodes
having similar names with case differences, ie: myNode and mynode
which might lead to DB corruption on Consul server side and
lead to server not properly restarting.

See #3983 and #4399 for Context about this change.

Disabling registration of nodes without IDs as specified in #4414
should probably be the way to go eventually.

* Removed the case-insensitive search when adding a node within the else
block since it breaks the test TestAgentAntiEntropy_Services

While the else case is probably legit, it will be fixed with #4414 in
a later release.

* Added again the test in the else to avoid duplicated names, but
enforce this test only for nodes having IDs.

Thus most tests without any ID will work, and allows us fixing

* Added more tests regarding request with/without IDs.

`TestStateStore_EnsureNode` now test registration and renaming with IDs

`TestStateStore_EnsureNodeDeprecated` tests registration without IDs
and tests removing an ID from a node as well as updated a node
without its ID (deprecated behaviour kept for backwards compatibility)

* Do not allow renaming in case of conflict, including when other node has no ID

* Fixed function GetNodeID that was not working due to wrong type when searching node from its ID

Thus, all tests about renaming were not working properly.

Added the full test cas that allowed me to detect it.

* Better error messages, more tests when nodeID is not a valid UUID in GetNodeID()

* Added separate TestStateStore_GetNodeID to test GetNodeID.

More complete test coverage for GetNodeID

* Added new unit test `TestStateStore_ensureNoNodeWithSimilarNameTxn`

Also fixed comments to be clearer after remarks from @banks

* Fixed error message in unit test to match test case

* Use uuid.ParseUUID to parse Node.ID as requested by @mkeeler
 2018-08-10 11:30:45 -04:00
Siva Prasad c88900aaa9
PR to fix TestAgent_IndexChurn and TestPreparedQuery_Wrapper. (#4512) 
* Fixes TestAgent_IndexChurn

* Fixes TestPreparedQuery_Wrapper

* Increased sleep in agent_test for IndexChurn to 500ms

* Made the comment about joinWAN operation much less of a cliffhanger
 2018-08-09 12:40:07 -04:00
Armon Dadgar 4f1fd34e9e consul: Update buffer sizes 2018-08-08 10:26:58 -07:00
Siva Prasad 288d350a73
Revert "CA initialization while boostrapping and TestLeader_ChangeServerID fix." (#4497) 
* Revert "BUGFIX: Unit test relying on WaitForLeader() did not work due to wrong test (#4472)"

This reverts commit cec5d72396.

* Revert "CA initialization while boostrapping and TestLeader_ChangeServerID fix. (#4493)"

This reverts commit 589b589b53.
 2018-08-07 08:29:48 -04:00
Pierre Souchay cec5d72396 BUGFIX: Unit test relying on WaitForLeader() did not work due to wrong test (#4472) 
- Improve resilience of testrpc.WaitForLeader()

- Add additionall retry to CI

- Increase "go test" timeout to 8m

- Add wait for cluster leader to several tests in the agent package

- Add retry to some tests in the api and command packages
 2018-08-06 19:46:09 -04:00
Siva Prasad 589b589b53
CA initialization while boostrapping and TestLeader_ChangeServerID fix. (#4493) 
* connect: fix an issue with Consul CA bootstrapping being interrupted

* streamline change server id test
 2018-08-06 16:15:24 -04:00
Kyle Havlovitz fa0d8aff33
fix inconsistency in TestConnectCAConfig_GetSet 2018-07-26 07:46:47 -07:00
Kyle Havlovitz ed87949385
Merge pull request #4400 from hashicorp/leaf-cert-ttl 
Add configurable leaf cert TTL to Connect CA
 2018-07-25 17:53:25 -07:00
Paul Banks 8cbeb29e73
Fixes #4421: General solution to stop blocking queries with index 0 (#4437) 
* Fix theoretical cache collision bug if/when we use more cache types with same result type

* Generalized fix for blocking query handling when state store methods return zero index

* Refactor test retry to only affect CI

* Undo make file merge

* Add hint to error message returned to end-user requests if Connect is not enabled when they try to request cert

* Explicit error for Roots endpoint if connect is disabled

* Fix tests that were asserting old behaviour
 2018-07-25 20:26:27 +01:00
Kyle Havlovitz ce10de036e
connect/ca: check LeafCertTTL when rotating expired roots 2018-07-20 16:04:04 -07:00
Kyle Havlovitz d6ca015a42
connect/ca: add configurable leaf cert TTL 2018-07-16 13:33:37 -07:00
Matt Keeler 63d5c069fc
Merge pull request #4379 from hashicorp/persist-intermediates 
connect: persist intermediate CAs on leader change
 2018-07-12 12:09:13 -04:00
Matt Keeler 0e83059d1f
Revert "Allow changing Node names since Node now have IDs" 2018-07-12 11:19:21 -04:00
Matt Keeler 91150cca59 Fixup formatting 2018-07-12 10:14:26 -04:00
Matt Keeler 3807e04de9 Revert PR 4294 - Catalog Register: Generate UUID for services registered without one 
UUID auto-generation here causes trouble in a few cases. The biggest being older
nodes reregistering will fail when the UUIDs are different and the names match

This reverts commit 0f70034082.
This reverts commit d1a8f9cb3f.
This reverts commit cf69ec42a4.
 2018-07-12 10:06:50 -04:00
Kyle Havlovitz f95c6807e7
connect: use reflect.DeepEqual instead for test 2018-07-11 13:10:58 -07:00
Matt Keeler 98ead2a8f8
Merge pull request #3983 from pierresouchay/node_renaming 
Allow changing Node names since Node now have IDs
 2018-07-11 16:03:02 -04:00
Kyle Havlovitz 4e5fb6bc19
connect: add provider state to snapshots 2018-07-11 11:34:49 -07:00
Kyle Havlovitz 462ace4867
connect: update leader initializeCA comment 2018-07-11 10:00:42 -07:00
Kyle Havlovitz 1d3f4b5099
connect: persist intermediate CAs on leader change 2018-07-11 09:44:30 -07:00
Pierre Souchay fecae3de21 When renaming a node, ensure the name is not taken by another node. 
Since DNS is case insensitive and DB as issues when similar names with different
cases are added, check for unicity based on case insensitivity.

Following another big incident we had in our cluster, we also validate
that adding/renaming a not does not conflicts with case insensitive
matches.

We had the following error once:

 - one node called: mymachine.MYDC.mydomain was shut off
 - another node (different ID) was added with name: mymachine.mydc.mydomain before
   72 hours

When restarting the consul server of domain, the consul server restarted failed
to start since it detected an issue in RAFT database because
mymachine.MYDC.mydomain and mymachine.mydc.mydomain had the same names.

Checking at registration time with case insensitivity should definitly fix
those issues and avoid Consul DB corruption.
 2018-07-11 14:42:54 +02:00
Matt Keeler d19c7d8882
Merge pull request #4303 from pierresouchay/non_blocking_acl 
Only send one single ACL cache refresh across network when TTL is over
 2018-07-10 08:57:33 -04:00
MagnumOpus21 300330e24b Agent/Proxy: Formatting and test cases fix 2018-07-09 12:46:10 -04:00
Kyle Havlovitz 401b206a2e
Store the time CARoot is rotated out instead of when to prune 2018-07-06 16:05:25 -07:00
Kyle Havlovitz 1492243e0a
connect/ca: add logic for pruning old stale RootCA entries 2018-07-02 10:35:05 -07:00
Pierre Souchay bd023f352e Updated swith case to use same branch for async-cache and extend-cache 2018-07-02 17:39:34 +02:00
Pierre Souchay 1e7665c0d5 Updated documentation and adding more test case for async-cache 2018-07-01 23:50:30 +02:00
Pierre Souchay abde81a3e7 Added async-cache with similar behaviour as extend-cache but asynchronously 2018-07-01 23:50:30 +02:00
Pierre Souchay 9406ca1c95 Only send one single ACL cache refresh across network when TTL is over 
It will allow the following:

 * when connectivity is limited (saturated linnks between DCs), only one
   single request to refresh an ACL will be sent to ACL master DC instead
   of statcking ACL refresh queries
 * when extend-cache is used for ACL, do not wait for result, but refresh
   the ACL asynchronously, so no delay is not impacting slave DC
 * When extend-cache is not used, keep the existing blocking mechanism,
   but only send a single refresh request.

This will fix https://github.com/hashicorp/consul/issues/3524
 2018-07-01 23:50:30 +02:00
Matt Keeler 22b7b688a3
Move starting enterprise functionality 2018-06-29 17:38:29 -04:00
Matt Keeler 0f70034082 Move default uuid test into the consul package 2018-06-27 09:21:58 -04:00
Matt Keeler d1a8f9cb3f go fmt changes 2018-06-27 09:07:22 -04:00
Matt Keeler cf69ec42a4 Make sure to generate UUIDs when services are registered without one 
This makes the behavior line up with the docs and expected behavior
 2018-06-26 17:04:08 -04:00
mkeeler 6813a99081 Merge remote-tracking branch 'connect/f-connect' 2018-06-25 19:42:51 +00:00
Kyle Havlovitz 3baa67cdef connect/ca: pull the cluster ID from config during a rotation 2018-06-25 12:25:42 -07:00
Kyle Havlovitz b4ef7bb64d connect/ca: leave blank root key/cert out of the default config (unnecessary) 2018-06-25 12:25:42 -07:00
Kyle Havlovitz 050da22473 connect/ca: undo the interface changes and use sign-self-issued in Vault 2018-06-25 12:25:42 -07:00
Kyle Havlovitz bc997688e3 connect/ca: update Consul provider to use new cross-sign CSR method 2018-06-25 12:25:41 -07:00
Kyle Havlovitz 226a59215d connect/ca: fix vault provider URI SANs and test 2018-06-25 12:25:41 -07:00
Kyle Havlovitz 1a8ac686b2 connect/ca: add the Vault CA provider 2018-06-25 12:25:41 -07:00
Paul Banks e33bfe249e Note leadership issues in comments 2018-06-25 12:25:41 -07:00
Paul Banks e514570dfa Actually return Intermediate certificates bundled with a leaf! 2018-06-25 12:25:40 -07:00
Paul Banks 2e223ea2b7 Fix hot loop in cache for RPC returning zero index. 2018-06-25 12:25:37 -07:00
Paul Banks 05a8097c5d Fix misc test failures (some from other PRs) 2018-06-25 12:25:13 -07:00
Paul Banks 382ce8f98a Only set precedence on write path 2018-06-25 12:25:13 -07:00
Paul Banks 4a54f8f7e3 Fix some tests failures caused by the sorting change and some cuased by previous UpdatePrecedence() change 2018-06-25 12:25:13 -07:00
Paul Banks bf7a62e0e0 Sort intention list by precedence 2018-06-25 12:25:13 -07:00
Kyle Havlovitz edbeeeb23c agent: update accepted CA config fields and defaults 2018-06-25 12:25:09 -07:00
Mitchell Hashimoto 028aa78e83 agent/consul: set precedence value on struct itself 2018-06-25 12:24:16 -07:00
Mitchell Hashimoto daf46c9cfa agent/consul: support a Connect option on prepared query request 2018-06-25 12:24:12 -07:00
Mitchell Hashimoto 440b1b2d97 agent/consul: prepared query supports "Connect" field 2018-06-25 12:24:11 -07:00
Mitchell Hashimoto 1830c6b308 agent: switch ConnectNative to an embedded struct 2018-06-25 12:24:10 -07:00
Mitchell Hashimoto eb3fcb39b3 agent/consul/state: support querying by Connect native 2018-06-25 12:24:08 -07:00
Mitchell Hashimoto d6a823ad0d agent/consul: support catalog registration with Connect native 2018-06-25 12:24:07 -07:00
Matt Keeler af910bda39
Merge pull request #4216 from hashicorp/rpc-limiting 
Make RPC limits reloadable
 2018-06-20 09:05:28 -04:00
Mitchell Hashimoto 1906fe1c0d
agent: address feedback 2018-06-14 09:42:20 -07:00
Mitchell Hashimoto 0accfc1628
agent: rename test to check 2018-06-14 09:42:18 -07:00
Mitchell Hashimoto 2a29679e9d
agent/consul: forward request if necessary 2018-06-14 09:42:17 -07:00
Mitchell Hashimoto 54ac5adb08
agent: comments to point to differing logic 2018-06-14 09:42:17 -07:00
Mitchell Hashimoto d68462fca6
agent/consul: implement Intention.Test endpoint 2018-06-14 09:42:17 -07:00
Paul Banks f4b8e8c96d
Add default CA config back - I didn't add it and causes nil panics 2018-06-14 09:42:17 -07:00
Paul Banks 1228a5839a
Ooops remove the CA stuff from actual server defaults and make it test server only 2018-06-14 09:42:16 -07:00
Paul Banks 4aeab3897c
Fixed many tests after rebase. Some still failing and seem unrelated to any connect changes. 2018-06-14 09:42:16 -07:00
Paul Banks b4803eca59
Generate CSR using real trust-domain 2018-06-14 09:42:16 -07:00
Paul Banks 622a475eb1
Add CSR signing verification of service ACL, trust domain and datacenter. 2018-06-14 09:42:16 -07:00
Paul Banks c1f2025d96
Return TrustDomain from CARoots RPC 2018-06-14 09:42:15 -07:00
Kyle Havlovitz e00088e8ee
Rename some of the CA structs/files 2018-06-14 09:42:15 -07:00
Kyle Havlovitz 6e9f1f8acb
Add more metadata to structs.CARoot 2018-06-14 09:42:15 -07:00
Kyle Havlovitz 627aa80d5a
Use provider state table for a global serial index 2018-06-14 09:42:15 -07:00
Kyle Havlovitz de72834b8c
Move connect CA provider to separate package 2018-06-14 09:42:15 -07:00
Mitchell Hashimoto bc605a1576
agent/consul: change provider wait from goto to a loop 2018-06-14 09:42:14 -07:00
Mitchell Hashimoto c8b65217c3
agent/consul: check nil on getCAProvider result 2018-06-14 09:42:14 -07:00
Mitchell Hashimoto 9b3495dddb
agent/consul: retry reading provider a few times 2018-06-14 09:42:14 -07:00
Paul Banks 90c574ebaa
Wire up agent leaf endpoint to cache framework to support blocking. 2018-06-14 09:42:07 -07:00
Kyle Havlovitz a4d18f0eaa
Fill out connect CA rpc endpoint tests 2018-06-14 09:42:06 -07:00
Kyle Havlovitz cce7f1cca1
Add tests for the built in CA's state store table 2018-06-14 09:42:06 -07:00
Kyle Havlovitz 15fbc2fd97
Add more tests for built-in provider 2018-06-14 09:42:06 -07:00
Kyle Havlovitz edcfdb37af
Fix some inconsistencies around the CA provider code 2018-06-14 09:42:06 -07:00
Kyle Havlovitz daa8dd1779
Add CA config to connect section of agent config 2018-06-14 09:42:05 -07:00
Kyle Havlovitz 32d1eae28b
Move ConsulCAProviderConfig into structs package 2018-06-14 09:42:04 -07:00
Kyle Havlovitz 315b8bf594
Simplify the CAProvider.Sign method 2018-06-14 09:42:04 -07:00
Kyle Havlovitz c6e1b72ccb
Simplify the CA provider interface by moving some logic out 2018-06-14 09:42:04 -07:00
Kyle Havlovitz a325388939
Clarify some comments and names around CA bootstrapping 2018-06-14 09:42:04 -07:00
Kyle Havlovitz 33418afd3c
Add cross-signing mechanism to root rotation 2018-06-14 09:42:00 -07:00
Kyle Havlovitz d83fbfc766
Add the root rotation mechanism to the CA config endpoint 2018-06-14 09:41:59 -07:00
Kyle Havlovitz f9d92d795e
Have the built in CA store its state in raft 2018-06-14 09:41:59 -07:00
Kyle Havlovitz 30c1973e8b
Fix the testing endpoint's root set op 2018-06-14 09:41:59 -07:00
Kyle Havlovitz ab737ef0f8
Hook the CA RPC endpoint into the provider interface 2018-06-14 09:41:59 -07:00
Kyle Havlovitz 1f6501895f
Add CA bootstrapping on establishing leadership 2018-06-14 09:41:59 -07:00
Kyle Havlovitz 682f105c7c
Add the bootstrap config for the CA 2018-06-14 09:41:59 -07:00
Kyle Havlovitz 1787f88618
Add CA config set to fsm operations 2018-06-14 09:41:58 -07:00
Kyle Havlovitz 6b3416e480
Add the Connect CA config to the state store 2018-06-14 09:41:58 -07:00
Paul Banks 730da74369
Fix various test failures and vet warnings. 
Intention de-duplication in previously merged PR actualy failed some tests that were not caught be me or CI. I ran the test files for state changes but they happened not to trigger this case so I made sure they did first and then fixed. That fixed some upstream intention endpoint tests that I'd not run as part of testing the previous fix.
 2018-06-14 09:41:58 -07:00
Paul Banks 88541bba17
Add tests all the way up through the endpoints to ensure duplicate src/destination is supported and so ultimately deny/allow nesting works. 
Also adds a sanity check test for `api.Agent().ConnectAuthorize()` and a fix for a trivial bug in it.
 2018-06-14 09:41:57 -07:00
Paul Banks ed9f07c361
Allow duplicate source or destination, but enforce uniqueness across all four. 2018-06-14 09:41:57 -07:00
Mitchell Hashimoto 845f7cd8ad
agent/consul/state: ensure exactly one active CA exists when setting 2018-06-14 09:41:54 -07:00
Mitchell Hashimoto 17ca8ad083
agent/connect: rename SpiffeID to CertURI 2018-06-14 09:41:53 -07:00
Mitchell Hashimoto 0cbcb07d61
agent/connect: use proper keyusage fields for CA and leaf 2018-06-14 09:41:53 -07:00
Mitchell Hashimoto a54d1af421
agent/consul: encode issued cert serial number as hex encoded 2018-06-14 09:41:53 -07:00
Mitchell Hashimoto 63d674d07d
agent: /v1/connect/ca/configuration PUT for setting configuration 2018-06-14 09:41:52 -07:00
Mitchell Hashimoto 1c3dbc83ff
agent/consul/fsm,state: snapshot/restore for CA roots 2018-06-14 09:41:52 -07:00
Mitchell Hashimoto 90f423fd02
agent/consul/fsm,state: tests for CA root related changes 2018-06-14 09:41:52 -07:00
Mitchell Hashimoto 1c72639d60
agent/consul: set more fields on the issued cert 2018-06-14 09:41:52 -07:00
Mitchell Hashimoto c2588262b7
agent: /v1/connect/ca/leaf/:service_id 2018-06-14 09:41:52 -07:00
Mitchell Hashimoto e40afd6a73
agent/consul: CAS operations for setting the CA root 2018-06-14 09:41:51 -07:00
Mitchell Hashimoto 578db06600
agent/consul: tests for CA endpoints 2018-06-14 09:41:51 -07:00
Mitchell Hashimoto 891cd22ad9
agent/consul: key the public key of the CSR, verify in test 2018-06-14 09:41:51 -07:00
Mitchell Hashimoto d768d5e9a7
agent/consul: test for ConnectCA.Sign 2018-06-14 09:41:51 -07:00
Mitchell Hashimoto f4ec28bfe3
agent/consul: basic sign endpoint not tested yet 2018-06-14 09:41:51 -07:00
Mitchell Hashimoto 5a950190f3
agent/consul: RPC endpoints to list roots 2018-06-14 09:41:50 -07:00
Mitchell Hashimoto 130098b7b5
agent/consul/state: CARoot structs and initial state store 2018-06-14 09:41:49 -07:00
Mitchell Hashimoto 4d852e62a3
agent: address PR feedback 2018-06-14 09:41:49 -07:00
Mitchell Hashimoto 6313bc5615
agent: clarified a number of comments per PR feedback 2018-06-14 09:41:49 -07:00
Mitchell Hashimoto 353953fcd2
agent/consul: Health.ServiceNodes ACL check for Connect 2018-06-14 09:41:49 -07:00
Mitchell Hashimoto b6c0cb7115
agent/consul: Catalog endpoint ACL requirements for Connect proxies 2018-06-14 09:41:49 -07:00
Mitchell Hashimoto 2feef5f7a3
agent/consul: require name for proxies 2018-06-14 09:41:48 -07:00
Mitchell Hashimoto 44ec8d94d2
agent: clean up connect/non-connect duplication by using shared methods 2018-06-14 09:41:48 -07:00
Mitchell Hashimoto 7d79f9c46f
agent/consul: implement Health.ServiceNodes for Connect, DNS works 2018-06-14 09:41:47 -07:00
Mitchell Hashimoto e01914a025
agent/consul: Catalog.ServiceNodes supports Connect filtering 2018-06-14 09:41:47 -07:00
Mitchell Hashimoto 2062e37270
agent/consul/state: ConnectServiceNodes 2018-06-14 09:41:47 -07:00
Mitchell Hashimoto 7ed26e2c64
agent/consul: enforce ACL on ProxyDestination 2018-06-14 09:41:47 -07:00
Mitchell Hashimoto 0c0c0a58e7
agent/consul: proxy registration and tests 2018-06-14 09:41:46 -07:00
Mitchell Hashimoto 4d4a8443e8
agent: test /v1/catalog/node/:node to list connect proxies 2018-06-14 09:41:46 -07:00
Mitchell Hashimoto 6e257ea51c
agent: /v1/catalog/service/:service works with proxies 2018-06-14 09:41:46 -07:00
Mitchell Hashimoto 63e4a35827
agent/consul/state: convert proxy test to testify/assert 2018-06-14 09:41:46 -07:00
Mitchell Hashimoto 21c6fc623a
agent/consul/state: service registration with proxy works 2018-06-14 09:41:46 -07:00
Mitchell Hashimoto a621afe72c
agent/consul: convert intention ACLs to testify/assert 2018-06-14 09:41:46 -07:00
Mitchell Hashimoto 9dc8aa0fb3
agent/consul,structs: add tests for ACL filter and prefix for intentions 2018-06-14 09:41:45 -07:00
Mitchell Hashimoto 5ac649af7f
agent/consul: Intention.Match ACLs 2018-06-14 09:41:45 -07:00
Mitchell Hashimoto 4d87601bf4
agent/consul: Intention.Get ACLs 2018-06-14 09:41:45 -07:00
Mitchell Hashimoto 9bbbb73734
agent/consul: Intention.Apply ACL on rename 2018-06-14 09:41:45 -07:00
Mitchell Hashimoto 01b644e213
agent/consul: tests for ACLs on Intention.Apply update/delete 2018-06-14 09:41:45 -07:00
Mitchell Hashimoto a67ff1c0dc
agent/consul: Basic ACL on Intention.Apply 2018-06-14 09:41:44 -07:00
Mitchell Hashimoto 0719ff6905
agent: convert all intention tests to testify/assert 2018-06-14 09:41:44 -07:00
Mitchell Hashimoto 454ef7d106
agent/consul/fsm,state: snapshot/restore for intentions 2018-06-14 09:41:44 -07:00
Mitchell Hashimoto 80d068aaa4
agent: use UTC time for intention times, move empty list check to 
agent/consul
 2018-06-14 09:41:43 -07:00
Mitchell Hashimoto 370b2599a1
agent/consul/fsm: switch tests to use structs.TestIntention 2018-06-14 09:41:43 -07:00
Mitchell Hashimoto 97e2a73145
agent/consul/state: need to set Meta for intentions for tests 2018-06-14 09:41:43 -07:00
Mitchell Hashimoto ad42f42a17
agent/consul/state: remove TODO 2018-06-14 09:41:43 -07:00
Mitchell Hashimoto 70858598e4
agent: use testing intention to get valid intentions 2018-06-14 09:41:43 -07:00
Mitchell Hashimoto ab4ea3efb4
agent/consul: set default intention SourceType, validate it 2018-06-14 09:41:43 -07:00
Mitchell Hashimoto d92993f75b
agent/structs: Intention validation 2018-06-14 09:41:42 -07:00
Mitchell Hashimoto 82a50245e0
agent/consul: support intention description, meta is non-nil 2018-06-14 09:41:42 -07:00
Mitchell Hashimoto c12690b837
agent/consul/fsm: add tests for intention requests 2018-06-14 09:41:42 -07:00
Mitchell Hashimoto a9743f4f15
agent,agent/consul: set default namespaces 2018-06-14 09:41:42 -07:00
Mitchell Hashimoto 10c370c0fb
agent/consul: set CreatedAt, UpdatedAt on intentions 2018-06-14 09:41:42 -07:00
Mitchell Hashimoto 93de03fe8b
agent/consul: RPC endpoint for Intention.Match 2018-06-14 09:41:42 -07:00
Mitchell Hashimoto f93edadbbe
agent/consul/state: IntentionMatch for performing match resolution 2018-06-14 09:41:41 -07:00
Mitchell Hashimoto fb02e53536
agent/consul: test that Apply works to delete an intention 2018-06-14 09:41:41 -07:00
Mitchell Hashimoto 4417f37ede
agent/consul/state,fsm: support for deleting intentions 2018-06-14 09:41:41 -07:00
Mitchell Hashimoto 1b44c1befa
agent/consul: creating intention must not have ID set 2018-06-14 09:41:40 -07:00
Mitchell Hashimoto 771b1737e3
agent/consul: support updating intentions 2018-06-14 09:41:40 -07:00
Mitchell Hashimoto 0d96cdc0a5
agent: GET /v1/connect/intentions/:id 2018-06-14 09:41:40 -07:00
Mitchell Hashimoto e8c4156f07
agent/consul: Intention.Get endpoint 2018-06-14 09:41:40 -07:00
Mitchell Hashimoto 9e307e178e
agent/consul: Intention.Apply, FSM methods, very little validation 2018-06-14 09:41:39 -07:00
Mitchell Hashimoto 212a272989
agent/consul: start Intention RPC endpoints, starting with List 2018-06-14 09:41:39 -07:00
Mitchell Hashimoto 9639bfb1be
agent/consul/state: list intentions 2018-06-14 09:41:39 -07:00
Mitchell Hashimoto cc8a6f7f15
agent/consul/state: initial work on intentions memdb table 2018-06-14 09:41:39 -07:00
Guido Iaquinti f7fe6c2a87 Attach server.Name label to client.rpc.failed 2018-06-13 14:56:14 +01:00
Guido Iaquinti 3d230dee80 Attach server.ID label to client.rpc.failed 2018-06-13 14:53:44 +01:00
Guido Iaquinti e85e63c18c Client: add metric for failed RPC calls to server 2018-06-13 12:35:45 +01:00
Matt Keeler 0df7cd22aa Add a Client ReloadConfig test 2018-06-11 16:23:51 -04:00
Matt Keeler 08e26d10b8 Merge branch 'master' of github.com:hashicorp/consul into rpc-limiting 
# Conflicts:
#	agent/agent.go
#	agent/consul/client.go
 2018-06-11 16:11:36 -04:00
Matt Keeler 65746b2f8f Apply the limits to the clients rpcLimiter 2018-06-11 15:51:17 -04:00
Matt Keeler b6e9abe926 Allow for easy enterprise/oss coexistence 
Uses struct/interface embedding with the embedded structs/interfaces being empty for oss. Also methods on the server/client types are defaulted to do nothing for OSS
 2018-05-24 10:36:42 -04:00
Wim 5c04864b28 Add support for reverse lookup of services 2018-05-19 19:39:02 +02:00
Preetha Appan ca67094619
Change default raft threshold config values and add a section to upgrade notes 2018-05-11 10:45:41 -05:00
Preetha Appan d721da7b67
Also make snapshot interval configurable 2018-05-11 10:43:24 -05:00
Preetha Appan 66f31cd25a
Make raft snapshot commit threshold configurable 2018-05-11 10:43:24 -05:00
Jack Pearkes 291e8b83ae
Merge pull request #4097 from hashicorp/remove-deprecated 
Remove deprecated check/service fields and metric names
 2018-05-10 15:45:49 -07:00
Kyle Havlovitz ba3971d2c1
Remove deprecated metric names 2018-05-08 16:23:15 -07:00
Paul Banks b7fa3358d1
Merge pull request #3970 from pierresouchay/node_health_should_change_service_index 
[BUGFIX] When a node level check is removed, ensure all services of node are notified
 2018-05-08 16:44:50 +01:00
Pierre Souchay c152cb7bdf Added Missing Service Meta synchronization and field 2018-04-21 17:34:29 +02:00
Pierre Souchay 89ab642928 Allow renaming nodes when ID is unchanged 2018-04-18 15:39:38 +02:00
Kyle Havlovitz af4be34a2a
Update make static-assets goal and run format 2018-04-13 09:57:25 -07:00
Matt Keeler d926679278
Merge pull request #4023 from hashicorp/f-near-ip 
Add near=_ip support for prepared queries
 2018-04-12 12:10:48 -04:00
Matt Keeler 136efeb3be GH-3798: A couple more PR updates 
Test HTTP/DNS source IP without header/extra EDNS data.
Add WARN log for when prepared query with near=_ip is executed without specifying the source ip
 2018-04-12 10:10:37 -04:00
Matt Keeler cec8d5145b GH-3798: A few more PR updates 2018-04-11 20:32:35 -04:00
Matt Keeler d065d3a6db GH-3798: Updates for PR 
Allow DNS peer IP as the source IP.
Break early when the right node was found for executing the preapred query.
Update docs
 2018-04-11 17:02:04 -04:00
Matt Keeler 45a537def9 GH-3798: Add near=_ip support for prepared queries 2018-04-10 14:50:50 -04:00
Paul Banks 0d8993e338
Allow ignoring checks by ID when defining a PreparedQuery. Fixes #3727. 2018-04-10 14:04:16 +01:00
Preetha Appan c7581d68c6
Renames agent API layer for service metadata to "meta" for consistency 2018-03-28 09:04:50 -05:00
Preetha daa61c5803
Merge pull request #3881 from pierresouchay/service_metadata 
Feature Request: Support key-value attributes for services
 2018-03-27 16:33:57 -05:00
Pierre Souchay 980189a33f Added validation of ServiceMeta in Catalog 
Fixed Error Message when ServiceMeta is not valid

Added Unit test for adding a Service with badly formatted ServiceMeta
 2018-03-27 22:22:42 +02:00
Preetha Appan 226cb2e95c
fix typo and remove comment 2018-03-27 14:28:05 -05:00
Preetha Appan 010a459365
Remove unnecessary nil checks 2018-03-27 10:59:42 -05:00
Preetha Appan 6c0bb5a810
Fix test and remove unused method 2018-03-27 09:44:41 -05:00
Preetha Appan d77ab91123
Allows disabling WAN federation by setting serf WAN port to -1 2018-03-26 14:21:06 -05:00
Pierre Souchay a9868ae956 Added support for renaming nodes when their IP does not change 2018-03-26 16:44:13 +02:00
Pierre Souchay 18baff80ae Merge remote-tracking branch 'origin/master' into node_health_should_change_service_index 2018-03-22 13:07:11 +01:00
Pierre Souchay 5fb1b18073 More test cases 2018-03-22 12:41:06 +01:00
Pierre Souchay 39a7b5c20d Added new test regarding checks index 2018-03-22 12:20:25 +01:00
Pierre Souchay dd9efb755a Fixed minor typo in comments 
Might fix unstable travis build
 2018-03-22 10:30:10 +01:00
Josh Soref 94835a2715 Spelling (#3958) 
* spelling: another

* spelling: autopilot

* spelling: beginning

* spelling: circonus

* spelling: default

* spelling: definition

* spelling: distance

* spelling: encountered

* spelling: enterprise

* spelling: expands

* spelling: exits

* spelling: formatting

* spelling: health

* spelling: hierarchy

* spelling: imposed

* spelling: independence

* spelling: inspect

* spelling: last

* spelling: latest

* spelling: client

* spelling: message

* spelling: minimum

* spelling: notify

* spelling: nonexistent

* spelling: operator

* spelling: payload

* spelling: preceded

* spelling: prepared

* spelling: programmatically

* spelling: required

* spelling: reconcile

* spelling: responses

* spelling: request

* spelling: response

* spelling: results

* spelling: retrieve

* spelling: service

* spelling: significantly

* spelling: specifies

* spelling: supported

* spelling: synchronization

* spelling: synchronous

* spelling: themselves

* spelling: unexpected

* spelling: validations

* spelling: value
 2018-03-19 16:56:00 +00:00
Pierre Souchay b6914617d9 Fixed typo in comments 2018-03-19 17:12:08 +01:00
Pierre Souchay 5e974843f1 Refactoring to have clearer code without weird bool 2018-03-19 16:12:54 +01:00
Pierre Souchay a44b9e84b1 [BUGFIX] When a node level check is removed, ensure all services of node are notified 
Bugfix for https://github.com/hashicorp/consul/pull/3899

When a node level check is removed (example: maintenance),
some watchers on services might have to recompute their state.

If those nodes are performing blocking queries, they have to be notified.
While their state was updated when node-level state did change or was added
this was not the case when the check was removed. This fixes it.
 2018-03-19 14:14:03 +01:00
Devin Canterberry a61abcd931
🐛 Formatting changes only; add missing trailing commas 2018-03-15 10:19:46 -07:00
Mitchell Hashimoto 8217564c48
agent/consul/fsm: begin using testify/assert 2018-03-06 09:48:15 -08:00
Paul Banks 9a47449c6d
Merge pull request #3899 from pierresouchay/fix_blocking_queries_index 
Services Indexes modified per service instead of using a global Index
 2018-03-02 16:24:43 +00:00
Pierre Souchay 360dc1dd8d Simplified error handling for maxIndexForService 
* added unit tests to ensure service index is properly garbage collected
* added Upgrade from Version 1.0.6 to higher section in documentation
 2018-03-01 14:09:36 +01:00
Preetha Appan 80791d5b21
Remove extra newline 2018-02-21 13:21:47 -06:00
Preetha Appan 907b97b7f2
Unit test that calls revokeLeadership twice to make sure its idempotent 2018-02-21 12:48:53 -06:00
Preetha Appan f59abcc394
Make sure revokeLeadership is called if establishLeadership errors 2018-02-21 12:33:22 -06:00
Alex Dadgar 18bf9647d5 Test autopilots start/stop idempotency 2018-02-21 10:19:30 -08:00
Alex Dadgar 33c5afdb31 Improve autopilot shutdown to be idempotent 2018-02-20 15:51:59 -08:00
Pierre Souchay a8d3745104 Fixed comments for function maxIndexForService 2018-02-20 23:57:28 +01:00
Pierre Souchay 09351ba9a6 [Revert] Only update services if tags are different 
This patch did give some better results, but break watches on
the services of a node.

It is possible to apply the same optimization for nodes than
to services (one index per instance), but it would complicate
further the patch.

Let's do it in another PR.
 2018-02-20 23:34:42 +01:00
Pierre Souchay 60454b570a Only update services if tags are different 2018-02-20 23:08:04 +01:00
Pierre Souchay a05d38737c Enable Raft index optimization per service name on health endpoint 
Had to fix unit test in order to check properly indexes.
 2018-02-20 01:35:50 +01:00
Pierre Souchay 4f10fae3c3 Get only first service to test whether we have to cleanup index of a service 2018-02-19 22:44:49 +01:00
Pierre Souchay bac8fb046f Fixed comment about raftIndex + use test.Helper() 2018-02-19 19:30:25 +01:00
Pierre Souchay 73127ef407 Services Indexes modified per service instead of using a global Index 
This patch improves the watches for services on large cluster:
each service has now its own index, such watches on a specific service
are not modified by changes in the global catalog.

It should improve a lot the performance of tools such as consul-template
or libraries performing watches on very large clusters with many
services/watches.
 2018-02-19 18:29:22 +01:00
Veselkov Konstantin 7de57ba4de remove golint warnings 2018-01-28 22:40:13 +04:00
Kyle Havlovitz bfeb09983b
Reset clusterHealth when autopilot starts 2018-01-23 12:52:28 -08:00
Kyle Havlovitz 17805e4634
Move autopilot health loop into leader operations 2018-01-23 11:17:41 -08:00
James Phillips da6a4635b0
Fixes a `go fmt` cleanup. 2017-12-20 13:43:38 -08:00
Kyle Havlovitz 11a0c9cc58
Fix vet error 2017-12-18 18:04:42 -08:00
Kyle Havlovitz 77dc52f430
Move autopilot initializing to oss file 2017-12-18 18:02:44 -08:00
Kyle Havlovitz 039e7f1880
Move autopilot setup to a separate file 2017-12-18 16:55:51 -08:00
Kyle Havlovitz d08ab9fd19
Make some final tweaks to autopilot package 2017-12-18 12:26:47 -08:00
Kyle Havlovitz a86d11ec0a
Merge pull request #3737 from hashicorp/autopilot-refactor 
Move autopilot to a standalone package
 2017-12-15 14:09:40 -08:00
James Phillips 06f980061e
Merge pull request #3728 from weiwei04/fix_globalRPC_goroutine_leak 
fix globalRPC goroutine leak
 2017-12-14 17:54:19 -08:00
Kyle Havlovitz 324c2ecb53
Expose IsPotentialVoter for advanced autopilot logic 2017-12-13 17:53:51 -08:00
Kyle Havlovitz 12bf61c851
Merge branch 'master' into autopilot-refactor 2017-12-13 11:54:32 -08:00
Kyle Havlovitz d6b266c045
A few last autopilot adjustments 2017-12-13 11:19:17 -08:00
Kyle Havlovitz 2310687c1d
More autopilot reorganizing 2017-12-13 10:57:37 -08:00
James Phillips 46742a5041
Adds TODOs referencing #3744. 2017-12-13 10:52:06 -08:00
Kyle Havlovitz b92f895c23
More refactoring to make autopilot consul-agnostic 2017-12-12 17:46:28 -08:00
Kyle Havlovitz de28555671
Move autopilot to a standalone package 2017-12-11 16:45:33 -08:00
James Phillips d12e81860f
Moves Serf helper into lib to fix import cycle in consul-enterprise. 2017-12-07 16:57:58 -08:00
James Phillips 5065f3d82e
Turns of intent queue warnings and enables dynamic queue sizing. 2017-12-07 16:27:06 -08:00