Commit Graph

463 Commits

Author SHA1 Message Date
Matt Keeler b069d6777b
OSS KV Modifications to Support Namespaces 2019-11-25 12:57:35 -05:00
Paul Banks cd1b613352
connect: Add AWS PCA provider (#6795)
* Update AWS SDK to use PCA features.

* Add AWS PCA provider

* Add plumbing for config, config validation tests, add test for inheriting existing CA resources created by user

* Unparallel the tests so we don't exhaust PCA limits

* Merge updates

* More aggressive polling; rate limit pass through on sign; Timeout on Sign and CA create

* Add AWS PCA docs

* Fix Vault doc typo too

* Doc typo

* Apply suggestions from code review

Co-Authored-By: R.B. Boyer <rb@hashicorp.com>
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>

* Doc fixes; tests for erroring if State is modified via API

* More review cleanup

* Uncomment tests!

* Minor suggested clean ups
2019-11-21 17:40:29 +00:00
Matt Keeler f6af75ae74
Refactor the Read Namespace requests to not use the query funct… (#6720)
The query function doesn’t handle non-200 return codes properly so this ended up still trying to decode the body which resulted in weird error messages.
2019-11-11 10:57:32 -05:00
Blake Covarrubias 0aa025df1c
docs: Miscellaneous docs cleanup (#6742)
Fix spelling errors, API doc inconsistencies, and formatting issues.

* Fix several spelling errors.
* Prepend / to v1/event/list path in Watches.
* Rename script handlers to match Watch type.
* Remove /v1 path prefix on service health API endpoints.

Makes request path consistent with the rest of the HTTP API
documentation which does not include the /v1 prefix.

* Fix bracket formatting issue on Telemetry page.

The HTML codes used for brackets inside of the code block are not
interpolated, and are shown as literal strings.

Replace the numeric HTML codes with the intended character value to
fix display formatting.

Also placed variable reference on agent/options.html inside code block
for consistency with the presentation of other options on the page.

* Add missing word to Coordinate.Node docstring.

Resolves #6014
2019-11-05 20:34:46 -08:00
Sarah Adams 78ad8203a4
Use encoding/json as JSON decoder instead of mapstructure (#6680)
Fixes #6147
2019-10-29 11:13:36 -07:00
Sarah Christoff 5e1c6e907b
Set MinQuorum variable in Autopilot (#6654)
* Add MinQuorum to Autopilot
2019-10-29 09:04:41 -05:00
Freddy 60f6ec0c2f
Store check type in catalog (#6561) 2019-10-17 20:33:11 +02:00
Matt Keeler 4a5ebb57b0
Add support for the Namespace HTTP API in the API Client (#6581) 2019-10-07 15:19:38 -04:00
Sarah Christoff 5e26971864
Prune Unhealthy Agents (#6571)
* Add -prune flag to ForceLeave
2019-10-04 16:10:02 -05:00
Freddy 2274f64cab
Add Namespace as an api query/write option (#6551) 2019-09-26 10:05:13 -06:00
Freddy fdd10dd8b8
Expose HTTP-based paths through Connect proxy (#6446)
Fixes: #5396

This PR adds a proxy configuration stanza called expose. These flags register
listeners in Connect sidecar proxies to allow requests to specific HTTP paths from outside of the node. This allows services to protect themselves by only
listening on the loopback interface, while still accepting traffic from non
Connect-enabled services.

Under expose there is a boolean checks flag that would automatically expose all
registered HTTP and gRPC check paths.

This stanza also accepts a paths list to expose individual paths. The primary
use case for this functionality would be to expose paths for third parties like
Prometheus or the kubelet.

Listeners for requests to exposed paths are be configured dynamically at run
time. Any time a proxy, or check can be registered, a listener can also be
created.

In this initial implementation requests to these paths are not
authenticated/encrypted.
2019-09-25 20:55:52 -06:00
R.B. Boyer 92fd596d32
api/watch: try to avoid more flakes in this package (#6538) 2019-09-24 11:24:48 -05:00
R.B. Boyer af01d397a5
connect: don't colon-hex-encode the AuthorityKeyId and SubjectKeyId fields in connect certs (#6492)
The fields in the certs are meant to hold the original binary
representation of this data, not some ascii-encoded version.

The only time we should be colon-hex-encoding fields is for display
purposes or marshaling through non-TLS mediums (like RPC).
2019-09-23 12:52:35 -05:00
R.B. Boyer f74244dde8 api/watch: reduce timing dependence on tests of watch behavior
Also for debugging purposes send the stdout/stderr streams from consul
processes spawned for API tests to testing.T.Logf
2019-09-19 09:20:53 -05:00
R.B. Boyer ea6231ce35 api: bump to sdk v0.2.0 2019-08-23 16:41:19 -05:00
R.B. Boyer fd1c62ee8b
connect: ensure time.Duration fields retain their human readable forms in the API (#6348)
This applies for both config entries and the compiled discovery chain.

Also omit some other config entries fields when empty.
2019-08-19 15:31:05 -05:00
R.B. Boyer 561b2fe606
connect: generate the full SNI names for discovery targets in the compiler rather than in the xds package (#6340) 2019-08-19 13:03:03 -05:00
R.B. Boyer ae79cdab1b
connect: introduce ExternalSNI field on service-defaults (#6324)
Compiling this will set an optional SNI field on each DiscoveryTarget.
When set this value should be used for TLS connections to the instances
of the target. If not set the default should be used.

Setting ExternalSNI will disable mesh gateway use for that target. It also 
disables several service-resolver features that do not make sense for an 
external service.
2019-08-19 12:19:44 -05:00
hashicorp-ci 5919c7c184 Merge Consul OSS branch 'master' at commit 8f7586b339 2019-08-13 02:00:43 +00:00
Matt Keeler 69c7d64701
Add missing LicenseReset API function (#6311) 2019-08-12 15:24:02 -04:00
Sarah Adams 8ff1f481fe
add flag to allow /operator/keyring requests to only hit local servers (#6279)
Add parameter local-only to operator keyring list requests to force queries to only hit local servers (no WAN traffic).

HTTP API: GET /operator/keyring?local-only=true
CLI: consul keyring -list --local-only

Sending the local-only flag with any non-GET/list request will result in an error.
2019-08-12 11:11:11 -07:00
Matt Keeler 6a1e0dfed8
Update the v1/agent/service/:service endpoint to output tagged… (#6304) 2019-08-10 09:15:19 -04:00
Mike Morris 65be58703c
connect: remove managed proxies (#6220)
* connect: remove managed proxies implementation and all supporting config options and structs

* connect: remove deprecated ProxyDestination

* command: remove CONNECT_PROXY_TOKEN env var

* agent: remove entire proxyprocess proxy manager

* test: remove all managed proxy tests

* test: remove irrelevant managed proxy note from TestService_ServerTLSConfig

* test: update ContentHash to reflect managed proxy removal

* test: remove deprecated ProxyDestination test

* telemetry: remove managed proxy note

* http: remove /v1/agent/connect/proxy endpoint

* ci: remove deprecated test exclusion

* website: update managed proxies deprecation page to note removal

* website: remove managed proxy configuration API docs

* website: remove managed proxy note from built-in proxy config

* website: add note on removing proxy subdirectory of data_dir
2019-08-09 15:19:30 -04:00
R.B. Boyer 9753c77a79
api: un-deprecate api.DecodeConfigEntry (#6278)
Add clarifying commentary about when it is not safe to use it. Also add
tests.
2019-08-05 17:15:22 -05:00
R.B. Boyer c395affc93
connect: expose an API endpoint to compile the discovery chain (#6248)
In addition to exposing compilation over the API cleaned up the structures that would be exchanged to be cleaner and easier to support and understand.

Also removed ability to configure the envoy OverprovisioningFactor.
2019-08-02 15:34:54 -05:00
Matt Keeler 5ce6bfa292
Add license management functions to API client (#6268)
* Add license management functions to API client

* Get rid of jsonapi struct tags
2019-08-02 16:20:38 -04:00
R.B. Boyer ad9e7b6ae9
connect: allow L7 routers to match on http methods (#6164)
Fixes #6158
2019-07-23 20:56:39 -05:00
R.B. Boyer 85cf2706e6
connect: change router syntax for matching query parameters to resemble the syntax for matching paths and headers for consistency. (#6163)
This is a breaking change, but only in the context of the beta series.
2019-07-23 20:55:26 -05:00
Alvin Huang 6f1953d96d Merge branch 'master' into release/1-6 2019-07-17 15:43:30 -04:00
Freddy 59dbd070d7
More flaky test fixes (#6151)
* Add retry to TestAPI_ClientTxn

* Add retry to TestLeader_RegisterMember

* Account for empty watch result in ConnectRootsWatch
2019-07-17 09:33:38 -06:00
hashicorp-ci fa20c7db97 Merge Consul OSS branch 'master' at commit 95dbb7f2f1 2019-07-17 02:00:21 +00:00
Freddy d219e31db8
Update retries that weren't using retry.R (#6146) 2019-07-16 14:47:45 -06:00
Jack Pearkes 338aed32af Merge branch 'master' into release/1-6 2019-07-12 14:51:25 -07:00
Freddy 97ecc0517c
Fix some retries in api pkg (#6124) 2019-07-12 12:57:41 -06:00
Freddy 5873c56a03
Flaky test overhaul (#6100) 2019-07-12 09:52:26 -06:00
Freddy 0b9111714b
Update TestServer creation in sdk/testutil (#6084)
* Retry the creation of the test server three times.
* Reduce the retry timeout for the API wait to 2 seconds, opting to fail faster and start over.
* Remove wait for leader from server creation. This wait can be added on a test by test basis now that the function is being exported.
* Remove wait for anti-entropy sync. This is built into the existing WaitForSerfCheck func, so that can be used if the anti-entropy wait is needed
2019-07-12 09:37:29 -06:00
Matt Keeler bdebe62fd0
Fix some tests that I broke when refactoring the ConfigSnapshot (#6051)
* Fix some tests that I broke when refactoring the ConfigSnapshot

* Make sure the MeshGateway config is added to all the right api structs

* Fix some more tests
2019-07-01 19:47:58 -04:00
Matt Keeler 8d953f5840 Implement Mesh Gateways
This includes both ingress and egress functionality.
2019-07-01 16:28:30 -04:00
R.B. Boyer 2ad516aeaf
do some initial config entry graph validation during writes (#6047) 2019-07-01 15:23:36 -05:00
R.B. Boyer 38d76c624e
Allow for both snake_case and CamelCase for config entries written with 'consul config write'. (#6044)
This also has the added benefit of fixing an issue with passing
time.Duration fields through config entries.
2019-06-28 11:35:35 -05:00
R.B. Boyer ceef44bbc9
adding new config entries for L7 discovery chain (unused) (#5987) 2019-06-27 12:37:43 -05:00
hashicorp-ci 4d185baf55 Merge Consul OSS branch 'master' at commit 88b15d84f9
skip-checks: true
2019-06-25 02:00:26 +00:00
Matt Keeler 6cc1451895
Update some tests to fix ContentHash broken by the tagged service addresses (#5996) 2019-06-20 11:50:18 -04:00
Aestek b839f52195 kv: do not trigger watches when setting the same value (#5885)
If a KVSet is performed but does not update the entry, do not trigger
watches for this key.
This avoids releasing blocking queries for KV values that did not
actually changed.
2019-06-18 15:06:29 +02:00
Matt Keeler f3d9b999ee
Add tagged addresses for services (#5965)
This allows addresses to be tagged at the service level similar to what we allow for nodes already. The address translation that can be enabled with the `translate_wan_addrs` config was updated to take these new addresses into account as well.
2019-06-17 10:51:50 -04:00
Pavel Drankov b3a0b273ed api: update link to agent caching in comments (#5935) 2019-06-13 23:56:19 +02:00
R.B. Boyer b2478036d8 api: go mod tidy 2019-05-08 13:26:07 -05:00
R.B. Boyer cc9bcccc73 api: bump sdk version to v0.1.1 2019-05-08 13:18:12 -05:00
Paul Banks 0cfb6051ea Add integration test for central config; fix central config WIP (#5752)
* Add integration test for central config; fix central config WIP

* Add integration test for central config; fix central config WIP

* Set proxy protocol correctly and begin adding upstream support

* Add upstreams to service config cache key and start new notify watcher if they change.

This doesn't update the tests to pass though.

* Fix some merging logic get things working manually with a hack (TODO fix properly)

* Simplification to not allow enabling sidecars centrally - it makes no sense without upstreams anyway

* Test compile again and obvious ones pass. Lots of failures locally not debugged yet but may be flakes. Pushing up to see what CI does

* Fix up service manageer and API test failures

* Remove the enable command since it no longer makes much sense without being able to turn on sidecar proxies centrally

* Remove version.go hack - will make integration test fail until release

* Remove unused code from commands and upstream merge

* Re-bump version to 1.5.0
2019-05-01 16:39:31 -07:00
Matt Keeler 3145bf5230 Centralized Config CLI (#5731)
* Add HTTP endpoints for config entry management

* Finish implementing decoding in the HTTP Config entry apply endpoint

* Add CAS operation to the config entry apply endpoint

Also use this for the bootstrapping and move the config entry decoding function into the structs package.

* First pass at the API client for the config entries

* Fixup some of the ConfigEntry APIs

Return a singular response object instead of a list for the ConfigEntry.Get RPC. This gets plumbed through the HTTP API as well.

Dont return QueryMeta in the JSON response for the config entry listing HTTP API. Instead just return a list of config entries.

* Minor API client fixes

* Attempt at some ConfigEntry api client tests

These don’t currently work due to weak typing in JSON

* Get some of the api client tests passing

* Implement reflectwalk magic to correct JSON encoding a ProxyConfigEntry

Also added a test for the HTTP endpoint that exposes the problem. However, since the test doesn’t actually do the JSON encode/decode its still failing.

* Move MapWalk magic into a binary marshaller instead of JSON.

* Add a MapWalk test

* Get rid of unused func

* Get rid of unused imports

* Fixup some tests now that the decoding from msgpack coerces things into json compat types

* Stub out most of the central config cli

Fully implement the config read command.

* Basic config delete command implementation

* Implement config write command

* Implement config list subcommand

Not entirely sure about the output here. Its basically the read output indented with a line specifying the kind/name of each type which is also duplicated in the indented output.

* Update command usage

* Update some help usage formatting

* Add the connect enable helper cli command

* Update list command output

* Rename the config entry API client methods.

* Use renamed apis

* Implement config write tests

Stub the others with the noTabs tests.

* Change list output format

Now just simply output 1 line per named config

* Add config read tests

* Add invalid args write test.

* Add config delete tests

* Add config list tests

* Add connect enable tests

* Update some CLI commands to use CAS ops

This also modifies the HTTP API for a write op to return a boolean indicating whether the value was written or not.

* Fix up the HTTP API CAS tests as I realized they weren’t testing what they should.

* Update config entry rpc tests to properly test CAS

* Fix up a few more tests

* Fix some tests that using ConfigEntries.Apply

* Update config_write_test.go

* Get rid of unused import
2019-04-30 16:27:16 -07:00