144 Commits

Author SHA1 Message Date
Paul Banks
1e7eace066 Add SidecarService Syntax sugar to Service Definition (#4686)
* Added new Config for SidecarService in ServiceDefinitions.

* WIP: all the code needed for SidecarService is written... none of it is tested other than config :). Need API updates too.

* Test coverage for the new sidecarServiceFromNodeService method.

* Test API registratrion with SidecarService

* Recursive Key Translation 🤦

* Add tests for nested sidecar defintion arrays to ensure they are translated correctly

* Use dedicated internal state rather than Service Meta for tracking sidecars for deregistration.

Add tests for deregistration.

* API struct for agent register. No other endpoint should be affected yet.

* Additional test cases to cover updates to API registrations
2018-10-10 16:55:34 +01:00
Paul Banks
b83bbf248c Add Proxy Upstreams to Service Definition (#4639)
* Refactor Service Definition ProxyDestination.

This includes:
 - Refactoring all internal structs used
 - Updated tests for both deprecated and new input for:
   - Agent Services endpoint response
   - Agent Service endpoint response
   - Agent Register endpoint
     - Unmanaged deprecated field
     - Unmanaged new fields
     - Managed deprecated upstreams
     - Managed new
   - Catalog Register
     - Unmanaged deprecated field
     - Unmanaged new fields
     - Managed deprecated upstreams
     - Managed new
   - Catalog Services endpoint response
   - Catalog Node endpoint response
   - Catalog Service endpoint response
 - Updated API tests for all of the above too (both deprecated and new forms of register)

TODO:
 - config package changes for on-disk service definitions
 - proxy config endpoint
 - built-in proxy support for new fields

* Agent proxy config endpoint updated with upstreams

* Config file changes for upstreams.

* Add upstream opaque config and update all tests to ensure it works everywhere.

* Built in proxy working with new Upstreams config

* Command fixes and deprecations

* Fix key translation, upstream type defaults and a spate of other subtele bugs found with ned to end test scripts...

TODO: tests still failing on one case that needs a fix. I think it's key translation for upstreams nested in Managed proxy struct.

* Fix translated keys in API registration.
≈

* Fixes from docs
 - omit some empty undocumented fields in API
 - Bring back ServiceProxyDestination in Catalog responses to not break backwards compat - this was removed assuming it was only used internally.

* Documentation updates for Upstreams in service definition

* Fixes for tests broken by many refactors.

* Enable travis on f-connect branch in this branch too.

* Add consistent Deprecation comments to ProxyDestination uses

* Update version number on deprecation notices, and correct upstream datacenter field with explanation in docs
2018-10-10 16:55:34 +01:00
Pierre Souchay
1a906ef34e Fix more unstable tests in agent and command 2018-09-12 14:49:27 +01:00
Pierre Souchay
eddcf228ea Implementation of Weights Data structures (#4468)
* Implementation of Weights Data structures

Adding this datastructure will allow us to resolve the
issues #1088 and #4198

This new structure defaults to values:
```
   { Passing: 1, Warning: 0 }
```

Which means, use weight of 0 for a Service in Warning State
while use Weight 1 for a Healthy Service.
Thus it remains compatible with previous Consul versions.

* Implemented weights for DNS SRV Records

* DNS properly support agents with weight support while server does not (backwards compatibility)

* Use Warning value of Weights of 1 by default

When using DNS interface with only_passing = false, all nodes
with non-Critical healthcheck used to have a weight value of 1.
While having weight.Warning = 0 as default value, this is probably
a bad idea as it breaks ascending compatibility.

Thus, we put a default value of 1 to be consistent with existing behaviour.

* Added documentation for new weight field in service description

* Better documentation about weights as suggested by @banks

* Return weight = 1 for unknown Check states as suggested by @banks

* Fixed typo (of -> or) in error message as requested by @mkeeler

* Fixed unstable unit test TestRetryJoin

* Fixed unstable tests

* Fixed wrong Fatalf format in `testrpc/wait.go`

* Added notes regarding DNS SRV lookup limitations regarding number of instances

* Documentation fixes and clarification regarding SRV records with weights as requested by @banks

* Rephrase docs
2018-09-07 15:30:47 +01:00
Pierre Souchay
cec5d72396 BUGFIX: Unit test relying on WaitForLeader() did not work due to wrong test (#4472)
- Improve resilience of testrpc.WaitForLeader()

- Add additionall retry to CI

- Increase "go test" timeout to 8m

- Add wait for cluster leader to several tests in the agent package

- Add retry to some tests in the api and command packages
2018-08-06 19:46:09 -04:00
Paul Banks
8cbeb29e73
Fixes #4421: General solution to stop blocking queries with index 0 (#4437)
* Fix theoretical cache collision bug if/when we use more cache types with same result type

* Generalized fix for blocking query handling when state store methods return zero index

* Refactor test retry to only affect CI

* Undo make file merge

* Add hint to error message returned to end-user requests if Connect is not enabled when they try to request cert

* Explicit error for Roots endpoint if connect is disabled

* Fix tests that were asserting old behaviour
2018-07-25 20:26:27 +01:00
Abhishek Chanda
36306c0076 Change bind_port to an int 2018-06-30 14:18:13 +01:00
mkeeler
6813a99081 Merge remote-tracking branch 'connect/f-connect' 2018-06-25 19:42:51 +00:00
Paul Banks
b5f24a21cb Fix test broken by final telemetry PR change! 2018-06-25 12:25:40 -07:00
Paul Banks
280f14d64c Make proxy only listen after initial certs are fetched 2018-06-25 12:25:40 -07:00
Paul Banks
420ae3df69 Limit proxy telemetry config to only be visible with authenticated with a proxy token 2018-06-25 12:25:39 -07:00
Paul Banks
597e55e8e2 Misc test fixes 2018-06-25 12:25:39 -07:00
Paul Banks
c6ef6a61c9 Refactor to use embedded struct. 2018-06-25 12:25:39 -07:00
Paul Banks
38405bd4a9 Allow user override of proxy telemetry config 2018-06-25 12:25:38 -07:00
Paul Banks
7649d630c6 Basic proxy telemetry working; not sure if it's too ugly; need to instrument things we care about 2018-06-25 12:25:38 -07:00
Paul Banks
d83f2e8e21 Expose telemetry config from RuntimeConfig to proxy config endpoint 2018-06-25 12:25:38 -07:00
Paul Banks
43b48bc06b Get agent cache tests passing without global hit count (which is racy).
Few other fixes in here just to get a clean run locally - they are all also fixed in other PRs but shouldn't conflict.

This should be robust to timing between goroutines now.
2018-06-25 12:25:37 -07:00
Mitchell Hashimoto
a08faf5a11 agent: add additional assertion to test 2018-06-25 12:25:13 -07:00
Paul Banks
2c21ead80e More test tweaks 2018-06-25 12:25:13 -07:00
Mitchell Hashimoto
e9e6514c9b agent: disallow deregistering a managed proxy directly 2018-06-25 12:25:12 -07:00
Mitchell Hashimoto
66a573e496 agent: deregister service deregisters the proxy along with it 2018-06-25 12:25:12 -07:00
Mitchell Hashimoto
e2653bec02 Fix broken tests from PR merge related to proxy secure defaults 2018-06-25 12:25:12 -07:00
Mitchell Hashimoto
0d6dcbd2f1 agent: disallow API registration with managed proxy if not enabled 2018-06-25 12:25:11 -07:00
Mitchell Hashimoto
927b45bf91 agent/config: move ports to ports structure, update docs 2018-06-25 12:24:15 -07:00
Mitchell Hashimoto
1830c6b308 agent: switch ConnectNative to an embedded struct 2018-06-25 12:24:10 -07:00
Paul Banks
e21723a891 Persist proxy state through agent restart 2018-06-25 12:24:08 -07:00
Mitchell Hashimoto
424272361d agent: agent service registration supports Connect native services 2018-06-25 12:24:08 -07:00
Mitchell Hashimoto
cd39f09693
agent: leaf endpoint accepts name, not service ID
This change is important so that requests can made representing a
service that may not be registered with the same local agent.
2018-06-14 09:42:20 -07:00
Paul Banks
c58d47ba59
Fix broken api test for service Meta (logical conflict rom OSS). Add test that would make this much easier to catch in future. 2018-06-14 09:42:17 -07:00
Paul Banks
4aeab3897c
Fixed many tests after rebase. Some still failing and seem unrelated to any connect changes. 2018-06-14 09:42:16 -07:00
Paul Banks
1722734313
Verify trust domain on /authorize calls 2018-06-14 09:42:16 -07:00
Paul Banks
b4803eca59
Generate CSR using real trust-domain 2018-06-14 09:42:16 -07:00
Mitchell Hashimoto
e54e69d11f
agent: verify local proxy tokens for CA leaf + tests 2018-06-14 09:42:14 -07:00
Mitchell Hashimoto
a099c27b07
agent: verify proxy token for ProxyConfig endpoint + tests 2018-06-14 09:42:14 -07:00
Mitchell Hashimoto
171bf8d599
agent: clean up defaulting of proxy configuration
This cleans up and unifies how proxy settings defaults are applied.
2018-06-14 09:42:10 -07:00
Mitchell Hashimoto
3d3eee2f6e
agent: resolve some conflicts and fix tests 2018-06-14 09:42:10 -07:00
Mitchell Hashimoto
1a2b28602c
agent: start proxy manager 2018-06-14 09:42:09 -07:00
Mitchell Hashimoto
476ea7b04a
agent: start/stop proxies 2018-06-14 09:42:08 -07:00
Mitchell Hashimoto
aaa2431350
agent: change connect command paths to be slices, not strings
This matches other executable configuration and allows us to cleanly
separate executable from arguments without trying to emulate shell
parsing.
2018-06-14 09:42:08 -07:00
Paul Banks
e0e12e165b
TLS watching integrated into Service with some basic tests.
There are also a lot of small bug fixes found when testing lots of things end-to-end for the first time and some cleanup now it's integrated with real CA code.
2018-06-14 09:42:07 -07:00
Paul Banks
90c574ebaa
Wire up agent leaf endpoint to cache framework to support blocking. 2018-06-14 09:42:07 -07:00
Paul Banks
cd88b2a351
Basic watch support for connect proxy config and certificate endpoints.
- Includes some bug fixes for previous `api` work and `agent` that weren't tested
 - Needed somewhat pervasive changes to support hash based blocking - some TODOs left in our watch toolchain that will explicitly fail on hash-based watches.
 - Integration into `connect` is partially done here but still WIP
2018-06-14 09:42:05 -07:00
Mitchell Hashimoto
8c1d5a2cdc
agent: resolve flaky test by checking cache hits increase, rather than
exact
2018-06-14 09:42:04 -07:00
Mitchell Hashimoto
051f004683
agent: use helper/retry instead of timing related tests 2018-06-14 09:42:04 -07:00
Mitchell Hashimoto
6c01e402e0
agent: augment /v1/connect/authorize to cache intentions 2018-06-14 09:42:02 -07:00
Mitchell Hashimoto
917a9e63d5
agent: check cache hit count to verify CA root caching, background update 2018-06-14 09:42:00 -07:00
Paul Banks
36dbd878c9
Adds api client code and tests for new Proxy Config endpoint, registering with proxy and seeing proxy config in /agent/services list. 2018-06-14 09:41:58 -07:00
Paul Banks
d73f079d0f
Add X-Consul-ContentHash header; implement removing all proxies; add load/unload test. 2018-06-14 09:41:57 -07:00
Paul Banks
2a69663448
Agent Connect Proxy config endpoint with hash-based blocking 2018-06-14 09:41:57 -07:00
Paul Banks
3e3f0e1f31
HTTP agent registration allows proxy to be defined. 2018-06-14 09:41:57 -07:00