Commit Graph

103 Commits

Author SHA1 Message Date
Paul Banks e812f5516a Add -sidecar-for and new /agent/service/:service_id endpoint (#4691)
- A new endpoint `/v1/agent/service/:service_id` which is a generic way to look up the service for a single instance. The primary value here is that it:
   - **supports hash-based blocking** and so;
   - **replaces `/agent/connect/proxy/:proxy_id`** as the mechanism the built-in proxy uses to read its config.
   - It's not proxy specific and so works for any service.
   - It has a temporary shim to call through to the existing endpoint to preserve current managed proxy config defaulting behaviour until that is removed entirely (tested).
 - The built-in proxy now uses the new endpoint exclusively for it's config
 - The built-in proxy now has a `-sidecar-for` flag that allows the service ID of the _target_ service to be specified, on the condition that there is exactly one "sidecar" proxy (that is one that has `Proxy.DestinationServiceID` set) for the service registered.
 - Several fixes for edge cases for SidecarService
 - A fix for `Alias` checks - when running locally they didn't update their state until some external thing updated the target. If the target service has no checks registered as below, then the alias never made it past critical.
2018-10-10 16:55:34 +01:00
Paul Banks b83bbf248c Add Proxy Upstreams to Service Definition (#4639)
* Refactor Service Definition ProxyDestination.

This includes:
 - Refactoring all internal structs used
 - Updated tests for both deprecated and new input for:
   - Agent Services endpoint response
   - Agent Service endpoint response
   - Agent Register endpoint
     - Unmanaged deprecated field
     - Unmanaged new fields
     - Managed deprecated upstreams
     - Managed new
   - Catalog Register
     - Unmanaged deprecated field
     - Unmanaged new fields
     - Managed deprecated upstreams
     - Managed new
   - Catalog Services endpoint response
   - Catalog Node endpoint response
   - Catalog Service endpoint response
 - Updated API tests for all of the above too (both deprecated and new forms of register)

TODO:
 - config package changes for on-disk service definitions
 - proxy config endpoint
 - built-in proxy support for new fields

* Agent proxy config endpoint updated with upstreams

* Config file changes for upstreams.

* Add upstream opaque config and update all tests to ensure it works everywhere.

* Built in proxy working with new Upstreams config

* Command fixes and deprecations

* Fix key translation, upstream type defaults and a spate of other subtele bugs found with ned to end test scripts...

TODO: tests still failing on one case that needs a fix. I think it's key translation for upstreams nested in Managed proxy struct.

* Fix translated keys in API registration.
≈

* Fixes from docs
 - omit some empty undocumented fields in API
 - Bring back ServiceProxyDestination in Catalog responses to not break backwards compat - this was removed assuming it was only used internally.

* Documentation updates for Upstreams in service definition

* Fixes for tests broken by many refactors.

* Enable travis on f-connect branch in this branch too.

* Add consistent Deprecation comments to ProxyDestination uses

* Update version number on deprecation notices, and correct upstream datacenter field with explanation in docs
2018-10-10 16:55:34 +01:00
Matt Keeler 89ba649252
Connect: Verify the leaf cert to determine its readiness. (#4540)
This improves the checking so that if a certificate were to expire or the roots changed then we will go into a non-ready state.

This parses the x509 certificates from the TLS certificate when the leaf is set. The readyCh will be closed whenever a parseable certificate is set and the ca roots are set. This does not mean that the certificate is valid but that it has been setup and is generally valid. The Ready function will now do x509 certificate verification which will in addition to verifying the signatures with the installed CA roots will also verify the certificate isn't expired or not set to become valid in the future. 

The correct way to use these functions is to wait for the ReadyWait chan to be closed and then periodically check the readiness to determine if the certificate is currently useable.
2018-09-07 10:58:06 -04:00
Pierre Souchay 92acdaa94c Fixed flaky tests (#4626) 2018-09-04 12:31:51 +01:00
Pierre Souchay 3101086a27 Fixed unstable test TestProxy_public (#4587) 2018-08-27 11:45:07 -04:00
Freddy e21f554923
Improve flaky connect/proxy Listener tests (#4498)
Improve flaky connect/proxy Listener tests

- Add sleep to TestEchoConn to allow for Read/Write to finish before fetching data in reportStats

- Account for flakiness around interval for Gauge

- Improve debug output when dumping metrics
2018-08-08 14:56:03 -04:00
Paul Banks af2901130d
Implement missing HTTP host to ConsulResolver func for Connect SDK. 2018-07-13 22:39:18 +01:00
Paul Banks b5f24a21cb Fix test broken by final telemetry PR change! 2018-06-25 12:25:40 -07:00
Paul Banks 3a6024e1b0 Fix merge error 2018-06-25 12:25:40 -07:00
Paul Banks 280f14d64c Make proxy only listen after initial certs are fetched 2018-06-25 12:25:40 -07:00
Paul Banks c6ef6a61c9 Refactor to use embedded struct. 2018-06-25 12:25:39 -07:00
Paul Banks 32f362bad9 StartupTelemetry => InitTelemetry 2018-06-25 12:25:39 -07:00
Paul Banks 96c416012e Misc rebase and test fixes 2018-06-25 12:25:38 -07:00
Paul Banks dc260f42fa Basic proxy active conns and bandwidth telemetry 2018-06-25 12:25:38 -07:00
Paul Banks c08b6f6fec Add accessor and helpers to SDK for fetching self-name and client service ID 2018-06-25 12:25:38 -07:00
Paul Banks 7649d630c6 Basic proxy telemetry working; not sure if it's too ugly; need to instrument things we care about 2018-06-25 12:25:38 -07:00
Paul Banks d83f2e8e21 Expose telemetry config from RuntimeConfig to proxy config endpoint 2018-06-25 12:25:38 -07:00
Paul Banks 01fefd3d92 Return defensive error if API response is jank 2018-06-25 12:25:10 -07:00
Paul Banks e7a345cb9a Refactor resolver logic to be clearer 2018-06-25 12:25:10 -07:00
Paul Banks 41a29a469e Fix roots race with CA setup hammering bug and defensive nil check hit during obscure upgrade scenario 2018-06-25 12:25:10 -07:00
Paul Banks 541cbae5f5 More misc comment cleanup 2018-06-25 12:24:17 -07:00
Paul Banks 0824d1df5f Misc comment cleanups 2018-06-25 12:24:16 -07:00
Mitchell Hashimoto 77a8003475 api: change Connect to a query option 2018-06-25 12:24:14 -07:00
Mitchell Hashimoto 8d66d1045a connect: remove old unused code 2018-06-25 12:24:14 -07:00
Mitchell Hashimoto e587b7c161 connect: support prepared query resolution 2018-06-25 12:24:13 -07:00
Mitchell Hashimoto 7a4463013d connect: resolver works with native services 2018-06-25 12:24:12 -07:00
Mitchell Hashimoto 11f57ed4f4
connect/proxy: remove dev CA settings 2018-06-14 09:42:22 -07:00
Mitchell Hashimoto ec4e600aeb
connect/proxy: add a full proxy test, parallel 2018-06-14 09:42:21 -07:00
Mitchell Hashimoto baa551355e
connect/proxy: don't start public listener if 0 port 2018-06-14 09:42:21 -07:00
Mitchell Hashimoto 0487cacd10
connect/proxy: use the right variable for loading the new service 2018-06-14 09:42:20 -07:00
Mitchell Hashimoto 8c713e6104
connect/proxy: don't require proxy ID 2018-06-14 09:42:20 -07:00
Paul Banks f6673ce164
Make Service logger log to right place again 2018-06-14 09:42:17 -07:00
Paul Banks 0bfffc92f2
Make connect client resolver resolve trust domain properly 2018-06-14 09:42:17 -07:00
Paul Banks 4aeab3897c
Fixed many tests after rebase. Some still failing and seem unrelated to any connect changes. 2018-06-14 09:42:16 -07:00
Mitchell Hashimoto 867db89303
command/connect/proxy: set proxy ID from env var if set 2018-06-14 09:42:14 -07:00
Paul Banks e0e12e165b
TLS watching integrated into Service with some basic tests.
There are also a lot of small bug fixes found when testing lots of things end-to-end for the first time and some cleanup now it's integrated with real CA code.
2018-06-14 09:42:07 -07:00
Paul Banks d1f4ad3d8a
Fix build error introduced in bad merge of TLS stuff 2018-06-14 09:42:07 -07:00
Paul Banks 67af5c740b
Add TODO for false-sharing 2018-06-14 09:42:07 -07:00
Paul Banks e112386426
Add support for measuring tx/rx packets through proxied connections. 2018-06-14 09:42:06 -07:00
Paul Banks 946e872f2f
Fix tests and listeners to work with Config changes (splitting host and port fields) 2018-06-14 09:42:05 -07:00
Paul Banks e8c510332c
Support legacy watch.HandlerFunc type for backward compat reduces impact of change 2018-06-14 09:42:05 -07:00
Paul Banks ab3df3d4a6
Working proxy config reload tests 2018-06-14 09:42:05 -07:00
Paul Banks cd88b2a351
Basic `watch` support for connect proxy config and certificate endpoints.
- Includes some bug fixes for previous `api` work and `agent` that weren't tested
 - Needed somewhat pervasive changes to support hash based blocking - some TODOs left in our watch toolchain that will explicitly fail on hash-based watches.
 - Integration into `connect` is partially done here but still WIP
2018-06-14 09:42:05 -07:00
Paul Banks 5310561c11
Refactor reloadableTLSConfig and verifyier shenanigans into simpler dynamicTLSConfig 2018-06-14 09:42:05 -07:00
Paul Banks e00ca9a7b7
Connect verification and AuthZ 2018-06-14 09:42:05 -07:00
Paul Banks 18a34c6836
Fix racy connect network tests that always fail in Docker due to listen races 2018-06-14 09:42:04 -07:00
Paul Banks 730da74369
Fix various test failures and vet warnings.
Intention de-duplication in previously merged PR actualy failed some tests that were not caught be me or CI. I ran the test files for state changes but they happened not to trigger this case so I made sure they did first and then fixed. That fixed some upstream intention endpoint tests that I'd not run as part of testing the previous fix.
2018-06-14 09:41:58 -07:00
Paul Banks 10db79c8ae
Rework connect/proxy and command/connect/proxy. End to end demo working again 2018-06-14 09:41:57 -07:00
Paul Banks aa19be4651
Remove old connect client and proxy implementation 2018-06-14 09:41:56 -07:00
Paul Banks 26e65f6bfd
connect.Service based implementation after review feedback. 2018-06-14 09:41:56 -07:00
Paul Banks 69d5efdbbd
Original proxy and connect.Client implementation. Working end to end. 2018-06-14 09:41:56 -07:00
Mitchell Hashimoto f4ec28bfe3
agent/consul: basic sign endpoint not tested yet 2018-06-14 09:41:51 -07:00
Mitchell Hashimoto 7349c94c23
connect: create connect package for helpers 2018-06-14 09:41:50 -07:00