Commit Graph

802 Commits

Author SHA1 Message Date
R.B. Boyer 80dfcb1bcd
raft: update to v1.3.5 (#12325)
This includes closing some leadership transfer gaps and adding snapshot
restore progress logging.
2022-02-14 13:48:52 -06:00
Chris S. Kim f5d816dace
Remove incorrect usage of url.PathEscape (#12184)
When r.toHTTP is called, http.Request is built with the path
already escaped. This removes all calls to url.PathEscape that
would have led to double-escaped URLs.
2022-01-25 12:15:06 -05:00
Dao Thanh Tung 6765c0d965
Add api changes to agent endpoint for part 2 (#12150) 2022-01-24 10:23:08 -05:00
R.B. Boyer b60d89e7ef bulk rewrite using this script
set -euo pipefail

    unset CDPATH

    cd "$(dirname "$0")"

    for f in $(git grep '\brequire := require\.New(' | cut -d':' -f1 | sort -u); do
        echo "=== require: $f ==="
        sed -i '/require := require.New(t)/d' $f
        # require.XXX(blah) but not require.XXX(tblah) or require.XXX(rblah)
        sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\([^tr]\)/require.\1(t,\2/g' $f
        # require.XXX(tblah) but not require.XXX(t, blah)
        sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/require.\1(t,\2/g' $f
        # require.XXX(rblah) but not require.XXX(r, blah)
        sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/require.\1(t,\2/g' $f
        gofmt -s -w $f
    done

    for f in $(git grep '\bassert := assert\.New(' | cut -d':' -f1 | sort -u); do
        echo "=== assert: $f ==="
        sed -i '/assert := assert.New(t)/d' $f
        # assert.XXX(blah) but not assert.XXX(tblah) or assert.XXX(rblah)
        sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\([^tr]\)/assert.\1(t,\2/g' $f
        # assert.XXX(tblah) but not assert.XXX(t, blah)
        sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/assert.\1(t,\2/g' $f
        # assert.XXX(rblah) but not assert.XXX(r, blah)
        sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/assert.\1(t,\2/g' $f
        gofmt -s -w $f
    done
2022-01-20 10:46:23 -06:00
Dan Upton ca3aca92c4
[OSS] Remove remaining references to master (#11827) 2022-01-20 12:47:50 +00:00
VictorBac 31a39c9528
Add GRPC and GRPCUseTLS to api.HealthCheckDefinition (#12108)
* Add GRPC to HealthCheckDefinition

* add GRPC and GRPCUseTLS
2022-01-19 16:09:15 -05:00
Mike Morris 1b1a97e8f9
ingress: allow setting TLS min version and cipher suites in ingress gateway config entries (#11576)
* xds: refactor ingress listener SDS configuration

* xds: update resolveListenerSDS call args in listeners_test

* ingress: add TLS min, max and cipher suites to GatewayTLSConfig

* xds: implement envoyTLSVersions and envoyTLSCipherSuites

* xds: merge TLS config

* xds: configure TLS parameters with ingress TLS context from leaf

* xds: nil check in resolveListenerTLSConfig validation

* xds: nil check in makeTLSParameters* functions

* changelog: add entry for TLS params on ingress config entries

* xds: remove indirection for TLS params in TLSConfig structs

* xds: return tlsContext, nil instead of ambiguous err

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>

* xds: switch zero checks to types.TLSVersionUnspecified

* ingress: add validation for ingress config entry TLS params

* ingress: validate listener TLS config

* xds: add basic ingress with TLS params tests

* xds: add ingress listeners mixed TLS min version defaults precedence test

* xds: add more explicit tests for ingress listeners inheriting gateway defaults

* xds: add test for single TLS listener on gateway without TLS defaults

* xds: regen golden files for TLSVersionInvalid zero value, add TLSVersionAuto listener test

* types/tls: change TLSVersion to string

* types/tls: update TLSCipherSuite to string type

* types/tls: implement validation functions for TLSVersion and TLSCipherSuites, make some maps private

* api: add TLS params to GatewayTLSConfig, add tests

* api: add TLSMinVersion to ingress gateway config entry test JSON

* xds: switch to Envoy TLS cipher suite encoding from types package

* xds: fixup validation for TLSv1_3 min version with cipher suites

* add some kitchen sink tests and add a missing struct tag

* xds: check if mergedCfg.TLSVersion is in TLSVersionsWithConfigurableCipherSuites

* xds: update connectTLSEnabled comment

* xds: remove unsued resolveGatewayServiceTLSConfig function

 * xds: add makeCommonTLSContextFromLeafWithoutParams

* types/tls: add LessThan comparator function for concrete values

* types/tls: change tlsVersions validation map from string to TLSVersion keys

* types/tls: remove unused envoyTLSCipherSuites

* types/tls: enable chacha20 cipher suites for Consul agent

* types/tls: remove insecure cipher suites from allowed config

TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 and TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 are both explicitly listed as insecure and disabled in the Go source.

Refs https://cs.opensource.google/go/go/+/refs/tags/go1.17.3:src/crypto/tls/cipher_suites.go;l=329-330

* types/tls: add ValidateConsulAgentCipherSuites function, make direct lookup map private

* types/tls: return all unmatched cipher suites in validation errors

* xds: check that Envoy API value matching TLS version is found when building TlsParameters

* types/tls: check that value is found in map before appending to slice in MarshalEnvoyTLSCipherSuiteStrings

* types/tls: cast to string rather than fmt.Printf in TLSCihperSuite.String()

* xds: add TLSVersionUnspecified to list of configurable cipher suites

* structs: update note about config entry warning

* xds: remove TLS min version cipher suite unconfigurable test placeholder

* types/tls: update tests to remove assumption about private map values

Co-authored-by: R.B. Boyer <rb@hashicorp.com>
2022-01-11 11:46:42 -05:00
littlestar642 634c72d22f add path escape and unescape to path params 2022-01-03 08:18:32 -08:00
Chris S. Kim 71bad67a4d
Add partitions to prettyformatters (#11789) 2021-12-09 15:58:45 -05:00
R.B. Boyer aa9a22a8fe fix test failures 2021-12-06 14:45:44 -06:00
freddygv 9b44861ce4 Update api module and decoding tests 2021-12-06 12:32:29 -07:00
Freddy a725f06c83
Merge pull request #11739 from hashicorp/ap/exports-rename 2021-12-06 08:20:50 -07:00
freddygv ed6076db26 Rename partition-exports to exported-services
Using a name less tied to partitions gives us more flexibility to use
this config entry in OSS for exports between datacenters/meshes.
2021-12-03 17:47:31 -07:00
freddygv 3e5400c4c5 Clarify feature name in partition docstring 2021-12-03 17:05:17 -07:00
freddygv 7811edd055 Rename internal AdminPartition references
This commit finishes replacing references to "AdminPartition" with
"Partition". This now matches other uses in the codebase such as the CLI
command, HTTP API, and the query parameter.
2021-12-03 17:05:05 -07:00
Dan Upton 1e47e3c82b
Groundwork for exposing when queries are filtered by ACLs (#11569) 2021-12-03 17:11:26 +00:00
R.B. Boyer c46f9f9f31
agent: add variation of force-leave that exclusively works on the WAN (#11722)
Fixes #6548
2021-12-02 17:15:10 -06:00
Dan Upton bf56a2c495
Rename `agent_master` ACL token in the API and CLI (#11669) 2021-12-02 17:05:27 +00:00
R.B. Boyer e6956893fb
api: ensure new partition fields are omit empty for back compat (#11585) 2021-11-16 12:28:34 -06:00
R.B. Boyer 1e02460bd1
re-run gofmt on 1.17 (#11579)
This should let freshly recompiled golangci-lint binaries using Go 1.17
pass 'make lint'
2021-11-16 12:04:01 -06:00
R.B. Boyer eb21649f82
partitions: various refactors to support partitioning the serf LAN pool (#11568) 2021-11-15 09:51:14 -06:00
freddygv 4c9c1b52ce Support partitions in connect expose cmd 2021-11-12 14:45:32 -07:00
freddygv a6d985040f Fixup shared oss/ent tests 2021-11-12 14:34:31 -07:00
freddygv e5b7c4713f Accept partition for ingress services 2021-11-12 14:33:14 -07:00
Dhia Ayachi 98735a6d12
KV refactoring, part 2 (#11512)
* add partition to the kv get pretty print

* fix failing test

* add test for kvs RPC endpoint
2021-11-08 11:43:21 -05:00
FFMMM 4ddf973a31
add root_cert_ttl option for consul connect, vault ca providers (#11428)
* add root_cert_ttl option for consul connect, vault ca providers

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>

* add changelog, pr feedback

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Update .changelog/11428.txt, more docs

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* Update website/content/docs/agent/options.mdx

Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>
2021-11-02 11:02:10 -07:00
Daniel Nephin 51d8417545
Merge pull request #10690 from tarat44/h2c-support-in-ping-checks
add support for h2c in h2 ping health checks
2021-11-02 13:53:06 -04:00
Daniel Upton d47b7311b8
Support Check-And-Set deletion of config entries (#11419)
Implements #11372
2021-11-01 16:42:01 +00:00
Dhia Ayachi 2801785710
regenerate expired certs (#11462)
* regenerate expired certs

* add documentation to generate tests certificates
2021-11-01 11:40:16 -04:00
R.B. Boyer 61361c2e5d
cli: update consul members output to display partitions and sort the results usefully (#11446) 2021-10-28 17:27:31 -05:00
Mark Anderson 977be77493
Fix back compat issues with UDS config (#11318)
SocketPath needs to be omitted when empty to avoid confusing older versions of Consul

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-10-28 13:31:10 -07:00
sidzi 29f192a130
Refactor requireHttpCodes for segregated error handling (#11287) 2021-10-28 12:24:23 -04:00
freddygv 4c45cafce2 Update filename to match entry kind - mesh 2021-10-27 15:01:26 -06:00
freddygv 5c24ed61a8 Rename service-exports to partition-exports
Existing config entries prefixed by service- are specific to individual
services. Since this config entry applies to partitions it is being
renamed.

Additionally, the Partition label was changed to Name because using
Partition at the top-level and in the enterprise meta was leading to the
enterprise meta partition being dropped by msgpack.
2021-10-25 17:58:48 -06:00
Daniel Nephin 4ae2c8de9d
Merge pull request #11232 from hashicorp/dnephin/acl-legacy-remove-docs
acl: add docs and changelog for the removal of the legacy ACL system
2021-10-25 18:38:00 -04:00
Kyle Havlovitz 04cd2c983e Add new service-exports config entry 2021-10-20 12:24:18 -07:00
Mike Morris 55659f3c9d
deps: update consul-api and consul-sdk (#11361)
* deps: update consul/sdk in api/go.mod

* deps: update consul/api to v1.11.0
2021-10-20 14:58:53 -04:00
Daniel Nephin dc58a8c339 api: remove the test for TestAPI_RulesTranslate_Raw
The API endpoint was removed in a previous commit, so we can no longer test this deprecated method.
2021-10-05 18:56:27 -04:00
tarat44 c1ed3a9a94 change config option to H2PingUseTLS 2021-10-05 00:12:21 -04:00
tarat44 3c9f5a73d9 add support for h2c in h2 ping health checks 2021-10-04 22:51:08 -04:00
FFMMM 8bb6d8571c
wrap few doRequest calls for error handling (#11158) 2021-09-29 13:12:15 -07:00
Paul Banks 7b4cbe3143 Final readability tweaks from review 2021-09-23 10:17:12 +01:00
Paul Banks b5345ea878 Add changelog; Add API package support for new fields. 2021-09-23 10:12:37 +01:00
FFMMM 377c3a9b0b
add StatusError to api package (#11054)
* add require http codes in api and use in operator_autopilot health check

* add StatusError type in api package

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-09-20 14:04:13 -07:00
R.B. Boyer 2af8e16ef9
api: add partition field to acl structs (#11080) 2021-09-17 11:53:03 -05:00
Evan Culver c4c833fd0e
sso/oidc: add support for acr_values request parameter (#11026)
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2021-09-17 09:10:05 -07:00
freddygv e0a7900f52 Fixup api config and Envoy test 2021-09-15 17:25:21 -06:00
freddygv 886f873fa4 Trim newlines before parenthesis end 2021-09-15 17:23:25 -06:00
Daniel Nephin 1f9479603c
Add failures_before_warning to checks (#10969)
Signed-off-by: Jakub Sokołowski <jakub@status.im>

* agent: add failures_before_warning setting

The new setting allows users to specify the number of check failures
that have to happen before a service status us updated to be `warning`.
This allows for more visibility for detected issues without creating
alerts and pinging administrators. Unlike the previous behavior, which
caused the service status to not update until it reached the configured
`failures_before_critical` setting, now Consul updates the Web UI view
with the `warning` state and the output of the service check when
`failures_before_warning` is breached.

The default value of `FailuresBeforeWarning` is the same as the value of
`FailuresBeforeCritical`, which allows for retaining the previous default
behavior of not triggering a warning.

When `FailuresBeforeWarning` is set to a value higher than that of
`FailuresBeforeCritical it has no effect as `FailuresBeforeCritical`
takes precedence.

Resolves: https://github.com/hashicorp/consul/issues/10680

Signed-off-by: Jakub Sokołowski <jakub@status.im>

Co-authored-by: Jakub Sokołowski <jakub@status.im>
2021-09-14 12:47:52 -04:00
Chris S. Kim 5b658d2f39
Sync partition fields from enterprise (#11021) 2021-09-13 17:53:52 -04:00
Dhia Ayachi ad914e16c2
acl-tokens table partitioning (#11020)
* convert `Policies` index to use `indexerMulti`

* remove non used indexer

* fix oss policy Get

* add oss tests

* remove reference to partition in oss test

* convert `Roles` index to use `indexerMulti`

* add role test in oss

* fix oss to use the right index func

* convert `Roles` index to use `indexerSingle`

* split authmethod write indexer to oss and ent

* add auth method unit tests

* add index locality

* move intFromBool to be available for oss

* add expiry indexes

* add api tests

* fix rebase

* use Bool func

* preallocate slice

* rename variable
2021-09-13 16:53:09 -04:00
Paul Banks a9bf4c9b82 Add Changelog entry and api package support for HTTP Header manip 2021-09-10 21:09:24 +01:00
Dhia Ayachi bc0e4f2f46
partition dicovery chains (#10983)
* partition dicovery chains

* fix default partition for OSS
2021-09-07 16:29:32 -04:00
Paul Ewing 1a45e96307
Revert early out errors in license API (#10952)
Licensing recently changed in Consul v1.10 and along with those changes
the client API was updated such that PutLicense and ResetLicense both
immediately return an error to avoid an unecessary round trip that will
inevitably fail.

For reference, see: 08eb600ee5

Unfortunately, this change broke forward compatibility such that a v1.10
client can no longer make these requests to a v1.9 server which is a
valid use case.

This commit reintroduces these requests to fix this compatibility
breakage but leaves the deprecation notices in tact.
2021-08-31 10:36:35 -06:00
Freddy d2e50fd130
Sync partitions support in api module (#10938) 2021-08-26 18:40:34 -06:00
Dhia Ayachi 1950ebbe1f
oss portion of ent #1069 (#10883) 2021-08-20 12:57:45 -04:00
R.B. Boyer 097e1645e3
agent: ensure that most agent behavior correctly respects partition configuration (#10880) 2021-08-19 15:09:42 -05:00
Daniel Nephin 4359e38114 debug: restore cancel on SigInt
Some previous changes broke interrupting the debug on SigInterupt. This change restores
the original behaviour by passing a context to requests.

Since a new API client function was required to pass the context, I had
it also return an io.ReadCloser, so that output can be streamed to files
instead of fully buffering in process memory.
2021-08-18 12:29:34 -04:00
R.B. Boyer 310e775a8a
state: partition nodes and coordinates in the state store (#10859)
Additionally:

- partitioned the catalog indexes appropriately for partitioning
- removed a stray reference to a non-existent index named "node.checks"
2021-08-17 13:29:39 -05:00
Daniel Nephin de3439b2cc api: remove client library tests for old API endpoints
These methods are being kept around in the api module for now to allow the api module
to work with older versions of the HTTP API, but we are no longer able to test them.

Since the endpoints are removed there is no way for them to change, so there does not
appear to be a need to keep running the tests.
2021-08-17 13:09:30 -04:00
Kyle Havlovitz 073b6c8411 oss: Rename default partition 2021-08-12 14:31:37 -07:00
Mark Anderson d3cebbd32c
Fixup to support unix domain socket via command line (#10758)
Missed the need to add support for unix domain socket config via
api/command line. This is a variant of the problems described in
it is easy to drop one.

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-08-12 10:05:22 -07:00
Daniel Nephin d3325b0253
Merge pull request #10612 from bigmikes/acl-replication-fix
acl: acl replication routine to report the last error message
2021-08-06 18:29:51 -04:00
Giulio Micheloni d4a3fe33e8 String type instead of error type and changelog. 2021-08-06 22:35:27 +01:00
Daniel Nephin 97fed47708
Merge pull request #10632 from hashicorp/pairing/acl-authorizer-when-acl-disabled
acls: Update ACL authorizer to return meaningful permission when ACLs are disabled
2021-07-30 13:22:55 -04:00
Blake Covarrubias c919f2d9ea
api: Support QueryOptions on additional agent endpoints (#10691)
Add support for setting QueryOptions on the following agent API endpoints:

- /agent/health/service/name/:name
- /agent/health/service/id/:id
- /agent/service/maintenance/:id

This follows the same pattern used in #9903 to support query options
for other agent API endpoints.

Resolves #9710
2021-07-30 10:07:13 -07:00
Evan Culver 4c4bbfb7c0 Fix api/agent test 2021-07-30 12:58:46 -04:00
Evan Culver f2d30bbc88 Fix api/agent test 2021-07-30 12:58:46 -04:00
Daniel Nephin c3149ec0fd debug: use the new metrics stream in debug command 2021-07-26 17:53:32 -04:00
R.B. Boyer 96b97d6554
replumbing a bunch of api and agent structs for partitions (#10681) 2021-07-22 14:33:22 -05:00
R.B. Boyer fc9b1a277d
sync changes to oss files made in enterprise (#10670) 2021-07-22 13:58:08 -05:00
Evan Culver 0527dcff57
acls: Show `AuthMethodNamespace` when reading/listing ACL token meta (#10598) 2021-07-15 10:38:52 -07:00
Kyle Havlovitz 77a2f38677 http: add partition query param parsing 2021-07-14 12:07:38 -07:00
Evan Culver 8fce7083c1
tests: skip cases that depend on test server when -short flag set (#10576) 2021-07-09 16:42:30 -07:00
Evan Culver 13bd86527b
Add support for returning ACL secret IDs for accessors with acl:write (#10546) 2021-07-08 15:13:08 -07:00
Daniel Nephin ec6da0859d
Merge pull request #10570 from hashicorp/copy-of-master
Changes that were accidentally merged into the old master branch
2021-07-08 16:28:56 -04:00
Jared Kirschner e517e744af
Merge pull request #10559 from jkirschner-hashicorp/fix-autopilot-config-post-default-values
Fix defaults for autopilot config update
2021-07-06 19:19:52 -04:00
Jared Kirschner 14059c2653 Fix defaults for autopilot config update
Previously, for a POST request to the /v1/operator/autopilot/configuration
endpoint, any fields not included in the payload were set to a zero-initialized
value rather than the documented default value.

Now, if an optional field is not included in the payload, it will be set to its
documented default value:
- CleanupDeadServers:      true
- LastContactThreshold:    "200ms"
- MaxTrailingLogs:         250
- MinQuorum:               0
- ServerStabilizationTime: "10s"
- RedundancyZoneTag:       ""
- DisableUpgradeMigration: false
- UpgradeVersionTag:       ""
2021-07-06 18:39:40 -04:00
Daniel Nephin 3a045cca8d ca: remove unused RotationPeriod field
This field was never used. Since it is persisted as part of a map[string]interface{} it
is pretty easy to remove it.
2021-07-05 19:15:44 -04:00
Kyle Nusbaum e72a703041 api: change io.Discard to ioutil.Discard 2021-06-23 11:45:20 -05:00
Daniel Nephin 6ac9149c3f api: properly close the response body
reads resp.Body until EOF, so that the http client is able to re-use the TCP connection.
2021-06-14 18:52:59 -04:00
Freddy ffb13f35f1
Rename CatalogDestinationsOnly (#10397)
CatalogDestinationsOnly is a passthrough that would enable dialing
addresses outside of Consul's catalog. However, when this flag is set to
true only _connect_ endpoints for services can be dialed.

This flag is being renamed to signal that non-Connect endpoints can't be
dialed by transparent proxies when the value is set to true.
2021-06-14 14:15:09 -06:00
Freddy 429f9d8bb8
Add flag for transparent proxies to dial individual instances (#10329) 2021-06-09 14:34:17 -06:00
Dhia Ayachi 4c7f5f31c7
debug: remove the CLI check for debug_enabled (#10273)
* debug: remove the CLI check for debug_enabled

The API allows collecting profiles even debug_enabled=false as long as
ACLs are enabled. Remove this check from the CLI so that users do not
need to set debug_enabled=true for no reason.

Also:
- fix the API client to return errors on non-200 status codes for debug
  endpoints
- improve the failure messages when pprof data can not be collected

Co-Authored-By: Dhia Ayachi <dhia@hashicorp.com>

* remove parallel test runs

parallel runs create a race condition that fail the debug tests

* Add changelog

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-05-27 09:41:53 -04:00
Rémi Lapeyre 213524a657
Always set the Content-Type header when a body is present (#10204)
* Always set the Content-Type header when a body is present

Closes https://github.com/hashicorp/consul/issues/10011

* Add Changelog entry

* Add more Content-Type exceptions

* Fix tests
2021-05-25 16:03:48 +01:00
Matt Keeler 08eb600ee5 Deprecate API driven licensing.
The two methods in the API client to Put or Reset a license will now always return an error.
2021-05-21 11:08:50 -04:00
Iryna Shustava d7d44f6ae7
Save exposed ports in agent's store and expose them via API (#10173)
* Save exposed HTTP or GRPC ports to the agent's store
* Add those the health checks API so we can retrieve them from the API
* Change redirect-traffic command to also exclude those ports from inbound traffic redirection when expose.checks is set to true.
2021-05-12 13:51:39 -07:00
Daniel Nephin 0f53d6e119 api: fix mesh config entry tests for default namespace 2021-05-06 13:25:32 -04:00
Mark Anderson f808a25a73 Fixup checksums for tests
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-05-05 10:13:41 -07:00
Mark Anderson 4d0346bc46 Fixup filtering tests
Missed annotations for SocketPath, LocalSocketPath, LocalServiceSocketPath

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-05-05 09:28:17 -07:00
Daniel Nephin 347f3d2128
Merge pull request #10155 from hashicorp/dnephin/config-entry-remove-fields
config-entry: remove Kind and Name field from Mesh config entry
2021-05-04 17:27:56 -04:00
Daniel Nephin c8c85523e1 config-entries: add a test for the API client
Also fixes a bug with listing kind=mesh config entries. ValidateConfigEntryKind was only being used by
the List endpoint, and was yet another place where we have to enumerate all the kinds.

This commit removes ValidateConfigEntryKind and uses MakeConfigEntry instead. This change removes
the need to maintain two separate functions at the cost of creating an instance of the config entry which will be thrown away immediately.
2021-05-04 17:14:21 -04:00
Mark Anderson 8040f91a43 Add support for downstreams
Enhance config by adding SocketPath and LocalSocketPath config values

Supports syntax of the form:
```
services {
  name = "sock_forwarder"
  id = "sock_forwarder.1"
  socket_path = "/tmp/downstream_3.sock"
  connect {
    sidecar_service {
      proxy {
	local_service_socket_path = "/tmp/downstream.sock"
      }
    }
  }
}
```

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-05-04 12:41:43 -07:00
Mark Anderson 583ae65d5b Convert mode to string representation
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-05-04 12:41:43 -07:00
Mark Anderson 8b1217417a First changes for unix domain sockets upstreams
Start making structure changes to support unix domain socket address for upstreams

	upstreams = [
	  {
	    destination_name = "echo-service"
	    local_bind_socket_path = "/tmp/upstream.sock"
	    config {
	      passive_health_check {
		interval = "10s"
		max_failures = 42
	      }
	    }
	  }

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-05-04 12:41:43 -07:00
Daniel Nephin a07a58a873 config-entry: use custom MarshalJSON for mesh type
So that the Kind field is added to the JSON object.
2021-04-29 17:49:43 -04:00
Daniel Nephin 62efaaab21 config-entry: remove Kind and Name field from Mesh config entry
No config entry needs a Kind field. It is only used to determine the Go type to
target. As we introduce new config entries (like this one) we can remove the kind field
and have the GetKind method return the single supported value.

In this case (similar to proxy-defaults) the Name field is also unnecessary. We always
use the same value. So we can omit the name field entirely.
2021-04-29 17:11:21 -04:00
Freddy 078c40425f
Rename "cluster" config entry to "mesh" (#10127)
This config entry is being renamed primarily because in k8s the name
cluster could be confusing given that the config entry applies across
federated datacenters.

Additionally, this config entry will only apply to Consul as a service
mesh, so the more generic "cluster" name is not needed.
2021-04-28 16:13:29 -06:00
R.B. Boyer 4db8b78854
connect: update centralized upstreams representation in service-defaults (#10015) 2021-04-15 14:21:44 -05:00
freddygv 7cb3f32672 Convert new tproxy structs in api module into ptrs
This way we avoid serializing these when empty. Otherwise users of the
latest version of the api submodule cannot interact with older versions
of Consul, because a new api client would send keys that the older Consul
doesn't recognize yet.
2021-04-13 12:44:25 -06:00
freddygv 7bd51ff536 Replace TransparentProxy bool with ProxyMode
This PR replaces the original boolean used to configure transparent
proxy mode. It was replaced with a string mode that can be set to:

- "": Empty string is the default for when the setting should be
defaulted from other configuration like config entries.
- "direct": Direct mode is how applications originally opted into the
mesh. Proxy listeners need to be dialed directly.
- "transparent": Transparent mode enables configuring Envoy as a
transparent proxy. Traffic must be captured and redirected to the
inbound and outbound listeners.

This PR also adds a struct for transparent proxy specific configuration.
Initially this is not stored as a pointer. Will revisit that decision
before GA.
2021-04-12 09:35:14 -06:00
freddygv 98ba582797 Fixup mesh gateway docs 2021-04-11 15:48:04 -06:00
Freddy a02245b75a
Merge pull request #9976 from hashicorp/centralized-upstream-fixups 2021-04-08 12:26:56 -06:00
freddygv d6db67ef86 Fixup test 2021-04-08 11:53:07 -06:00
Daniel Nephin 5d21d4d77f
Merge pull request #9925 from hashicorp/dnephin/update-memberlist
Update memberlist to v0.2.3
2021-04-08 12:17:29 -04:00
freddygv ab752c1c86 Avoid sending zero-value upstream defaults from api 2021-04-07 15:03:42 -06:00
R.B. Boyer 499fee73b3
connect: add toggle to globally disable wildcard outbound network access when transparent proxy is enabled (#9973)
This adds a new config entry kind "cluster" with a single special name "cluster" where this can be controlled.
2021-04-06 13:19:59 -05:00
Daniel Nephin 46279547ec Update memberlist to v0.2.3
To pickup data race fixes
2021-03-24 18:20:19 -04:00
Seth Hoenig cd1cd4febd api: enable query options on agent endpoints
This PR adds support for setting QueryOptions on a few agent API
endpoints. Nomad needs to be able to set the Namespace field on
these endpoints to:
 - query for services / checks in a namespace
 - deregister services / checks in a namespace
 - update TTL status on checks in a namespace
2021-03-19 13:08:26 -05:00
Freddy 8207b832df
Add TransparentProxy option to proxy definitions 2021-03-17 17:01:45 -06:00
Freddy c664938bae
Add per-upstream configuration to service-defaults 2021-03-17 16:59:51 -06:00
Christopher Broglie f0307c73e5 Add support for configuring TLS ServerName for health checks
Some TLS servers require SNI, but the Golang HTTP client doesn't
include it in the ClientHello when connecting to an IP address. This
change adds a new TLSServerName field to health check definitions to
optionally set it. This fixes #9473.
2021-03-16 18:16:44 -04:00
freddygv 6090cfcf68 PR comments 2021-03-15 16:02:03 -06:00
freddygv 8b46d8dcbb Restore old Envoy prefix on escape hatches
This is done because after removing ID and NodeName from
ServiceConfigRequest we will no longer know whether a request coming in
is for a Consul client earlier than v1.10.
2021-03-15 14:12:57 -06:00
freddygv 41b2ba1e58 Add omitempty across the board for UpstreamConfig 2021-03-15 13:23:18 -06:00
Matt Keeler 30903db442
AutopilotServerHealth now handles the 429 status code (#8599)
AutopilotServerHealthy now handles the 429 status code

Previously we would error out and not parse the response. Now either a 200 or 429 status code are considered expected statuses and will result in the method returning the reply allowing API consumers to not only see if the system is healthy or not but which server is unhealthy.
2021-03-12 09:40:49 -05:00
freddygv 23ffa3d3f3 And another test fix 2021-03-11 18:39:53 -07:00
Kyle Havlovitz 1e87c7183a
Merge pull request #9672 from hashicorp/ca-force-skip-xc
connect/ca: Allow ForceWithoutCrossSigning for all providers
2021-03-11 11:49:15 -08:00
freddygv 6fd30d0384 Add TransparentProxy opt to proxy definition 2021-03-11 11:37:21 -07:00
freddygv e3dc2a49df Turn Limits and PassiveHealthChecks into pointers 2021-03-11 11:04:40 -07:00
freddygv 87cde19b4c Create new types for service-defaults upstream cfg 2021-03-08 22:10:27 -07:00
Mark Anderson b9d22f48cd
Add fields to the /acl/auth-methods endpoint. (#9741)
* A GET of the /acl/auth-method/:name endpoint returns the fields
MaxTokenTTL and TokenLocality, while a LIST (/acl/auth-methods) does
not.

The list command returns a filtered subset of the full set. This is
somewhat deliberate, so that secrets aren't shown, but the TTL and
Locality fields aren't (IMO) security critical, and it is useful for
the front end to be able to show them.

For consistency these changes mirror the 'omit empty' and string
representation choices made for the GET call.

This includes changes to the gRPC and API code in the client.

The new output looks similar to this
curl 'http://localhost:8500/v1/acl/auth-methods' | jq '.'

  {
    "MaxTokenTTL": "8m20s",
    "Name": "minikube-ttl-local2",
    "Type": "kubernetes",
    "Description": "minikube auth method",
    "TokenLocality": "local",
    "CreateIndex": 530,
    "ModifyIndex": 530,
    "Namespace": "default"
  }
]

Signed-off-by: Mark Anderson <manderson@hashicorp.com>

* Add changelog

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-02-17 08:16:57 -08:00
Kyle Havlovitz 7dac583863 connect/ca: Allow ForceWithoutCrossSigning for all providers
This allows setting ForceWithoutCrossSigning when reconfiguring the CA
for any provider, in order to forcibly move to a new root in cases where
the old provider isn't reachable or able to cross-sign for whatever
reason.
2021-01-29 13:38:11 -08:00
Conor Mongey 7a368bd2b6
Move header methods from config to client 2021-01-20 01:30:54 +00:00
Conor Mongey eb65e59741
Only override headers if they're set 2021-01-20 01:12:19 +00:00
Conor Mongey f647569b84
Prefer http.Header over map[string]string to allow for multi-valued headers 2021-01-20 01:12:19 +00:00
Conor Mongey cdc8cd7b0e
Allow setting arbitrary headers in API client 2021-01-20 01:12:19 +00:00
Daniel Nephin 5a7f4c0dea
Merge pull request #8609 from hashicorp/dnephin/add-query-options-to-ServiceRegister
api: Add a context to ServiceRegisterOpts
2021-01-06 18:52:49 -05:00
Mike Morris ab927d5480 Merge pull request #9270 from hashicorp/release/1.9.0
merge: release/1.9.0 back into 1.9.x
2020-11-24 17:36:47 -05:00
R.B. Boyer 7c7a3e5165
command: when generating envoy bootstrap configs use the datacenter returned from the agent services endpoint (#9229)
Fixes #9215
2020-11-19 15:27:31 -06:00
Matt Keeler 66fd23d67f
Refactor to call non-voting servers read replicas (#9191)
Co-authored-by: Kit Patella <kit@jepsen.io>
2020-11-17 10:53:57 -05:00
Freddy fe728855ed
Add DC and NS support for Envoy metrics (#9207)
This PR updates the tags that we generate for Envoy stats.

Several of these come with breaking changes, since we can't keep two stats prefixes for a filter.
2020-11-16 16:37:19 -07:00
Matt Keeler 7f87adbcf4
Remove this constant as it is soon to be changing and we want to prevent backwards compat issues (#9193) 2020-11-13 17:09:51 -05:00
R.B. Boyer 61eac21f1a
agent: return the default ACL policy to callers as a header (#9101)
Header is: X-Consul-Default-ACL-Policy=<allow|deny>

This is of particular utility when fetching matching intentions, as the
fallthrough for a request that doesn't match any intentions is to
enforce using the default acl policy.
2020-11-12 10:38:32 -06:00
Matt Keeler 7ef9b04f90
Add a CLI command for retrieving the autopilot configuration. (#9142) 2020-11-11 13:19:02 -05:00
Matt Keeler c048e86bb2
Switch to using the external autopilot module 2020-11-09 09:22:11 -05:00
R.B. Boyer 8baf158ea8
Revert "Add namespace support for metrics (OSS) (#9117)" (#9124)
This reverts commit 06b3b017d3.
2020-11-06 10:24:32 -06:00
Freddy 06b3b017d3
Add namespace support for metrics (OSS) (#9117) 2020-11-05 18:24:29 -07:00
R.B. Boyer e113dc0fe2
upstream some differences from enterprise (#8902) 2020-10-09 09:42:53 -05:00
Matt Keeler 38f5ddce2a
Add per-agent reconnect timeouts (#8781)
This allows for client agent to be run in a more stateless manner where they may be abruptly terminated and not expected to come back. If advertising a per-agent reconnect timeout using the advertise_reconnect_timeout configuration when that agent leaves, other agents will wait only that amount of time for the agent to come back before reaping it.

This has the advantageous side effect of causing servers to deregister the node/services/checks for that agent sooner than if the global reconnect_timeout was used.
2020-10-08 15:02:19 -04:00
R.B. Boyer 0c9177f6a5
api: unflake some intention-related api tests (#8857) 2020-10-07 13:32:53 -05:00
R.B. Boyer 1b413b0444
connect: support defining intentions using layer 7 criteria (#8839)
Extend Consul’s intentions model to allow for request-based access control enforcement for HTTP-like protocols in addition to the existing connection-based enforcement for unspecified protocols (e.g. tcp).
2020-10-06 17:09:13 -05:00
R.B. Boyer a2a8e9c783
connect: intentions are now managed as a new config entry kind "service-intentions" (#8834)
- Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older
copies of consul will not see new config entries it doesn't understand
replicate down.

- Add shim conversion code so that the old API/CLI method of interacting
with intentions will continue to work so long as none of these are
edited via config entry endpoints. Almost all of the read-only APIs will
continue to function indefinitely.

- Add new APIs that operate on individual intentions without IDs so that
the UI doesn't need to implement CAS operations.

- Add a new serf feature flag indicating support for
intentions-as-config-entries.

- The old line-item intentions way of interacting with the state store
will transparently flip between the legacy memdb table and the config
entry representations so that readers will never see a hiccup during
migration where the results are incomplete. It uses a piece of system
metadata to control the flip.

- The primary datacenter will begin migrating intentions into config
entries on startup once all servers in the datacenter are on a version
of Consul with the intentions-as-config-entries feature flag. When it is
complete the old state store representations will be cleared. We also
record a piece of system metadata indicating this has occurred. We use
this metadata to skip ALL of this code the next time the leader starts
up.

- The secondary datacenters continue to run the old intentions
replicator until all servers in the secondary DC and primary DC support
intentions-as-config-entries (via serf flag). Once this condition it met
the old intentions replicator ceases.

- The secondary datacenters replicate the new config entries as they are
migrated in the primary. When they detect that the primary has zeroed
it's old state store table it waits until all config entries up to that
point are replicated and then zeroes its own copy of the old state store
table. We also record a piece of system metadata indicating this has
occurred. We use this metadata to skip ALL of this code the next time
the leader starts up.
2020-10-06 13:24:05 -05:00
R.B. Boyer d2eb27e0a3
api: support GetMeta() and GetNamespace() on all config entry kinds (#8764)
Fixes #8755

Since I was updating the interface, i also added the missing `GetNamespace()`.

Depending upon how you look at it, this is a breaking change since it adds methods to the exported interface `api.ConfigEntry`. Given that you cannot define your own config entry kinds, and all of the machinery of the `api.Client` acts like a factory to construct the canned ones from the rest of the module, this feels like it's not a problematic change as it would only break someone who had reimplemented the `ConfigEntry` interface themselves for no apparent utility?
2020-09-29 09:11:57 -05:00
freddygv 7b9d1b41d5 Resolve conflicts against master 2020-09-11 18:41:58 -06:00
freddygv 768dbaa68d Add session flag to cookie config 2020-09-11 18:34:03 -06:00
freddygv eab90ea9fa Revert EnvoyConfig nesting 2020-09-11 09:21:43 -06:00
Seth Hoenig 9fab3fe990
api: create fresh http client for unix sockets (#8602)
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
2020-09-06 12:27:39 -04:00
Freddy c9c9e4face
Make LockDelay configurable in api locks (#8621) 2020-09-04 13:38:26 -06:00
Daniel Nephin d083d4717b api: Add a context to ServiceRegisterOpts
So that operations can timeout instead of hanging forever
2020-09-03 12:25:06 -04:00