Commit Graph

1116 Commits

Author SHA1 Message Date
John Landa eded58b62a
Remove artificial ACLTokenMaxTTL limit for configuring acl token expiry (#17066)
* Remove artificial ACLTokenMaxTTL limit for configuring acl token expiry

* Add changelog

* Remove test on default MaxTokenTTL

* Change to imperitive tense for changelog entry
2023-04-28 10:57:30 -05:00
Freddy e02ef16f02
Update HCP bootstrapping to support existing clusters (#16916)
* Persist HCP management token from server config

We want to move away from injecting an initial management token into
Consul clusters linked to HCP. The reasoning is that by using a separate
class of token we can have more flexibility in terms of allowing HCP's
token to co-exist with the user's management token.

Down the line we can also more easily adjust the permissions attached to
HCP's token to limit it's scope.

With these changes, the cloud management token is like the initial
management token in that iit has the same global management policy and
if it is created it effectively bootstraps the ACL system.

* Update SDK and mock HCP server

The HCP management token will now be sent in a special field rather than
as Consul's "initial management" token configuration.

This commit also updates the mock HCP server to more accurately reflect
the behavior of the CCM backend.

* Refactor HCP bootstrapping logic and add tests

We want to allow users to link Consul clusters that already exist to
HCP. Existing clusters need care when bootstrapped by HCP, since we do
not want to do things like change ACL/TLS settings for a running
cluster.

Additional changes:

* Deconstruct MaybeBootstrap so that it can be tested. The HCP Go SDK
  requires HTTPS to fetch a token from the Auth URL, even if the backend
  server is mocked. By pulling the hcp.Client creation out we can modify
  its TLS configuration in tests while keeping the secure behavior in
  production code.

* Add light validation for data received/loaded.

* Sanitize initial_management token from received config, since HCP will
  only ever use the CloudConfig.MangementToken.

* Add changelog entry
2023-04-27 22:27:39 +02:00
John Maguire 391ed069c4
APIGW: Update how status conditions for certificates are handled (#17115)
* Move status condition for invalid certifcate to reference the listener
that is using the certificate

* Fix where we set the condition status for listeners and certificate
refs, added tests

* Add changelog
2023-04-27 15:54:44 +00:00
Semir Patel 5eaeb7b8e5
Support Envoy's MaxEjectionPercent and BaseEjectionTime config entries for passive health checks (#15979)
* Add MaxEjectionPercent to config entry

* Add BaseEjectionTime to config entry

* Add MaxEjectionPercent and BaseEjectionTime to protobufs

* Add MaxEjectionPercent and BaseEjectionTime to api

* Fix integration test breakage

* Verify MaxEjectionPercent and BaseEjectionTime in integration test upstream confings

* Website docs for MaxEjectionPercent and BaseEjection time

* Add `make docs` to browse docs at http://localhost:3000

* Changelog entry

* so that is the difference between consul-docker and dev-docker

* blah

* update proto funcs

* update proto

---------

Co-authored-by: Maliz <maliheh.monshizadeh@hashicorp.com>
2023-04-26 15:59:48 -07:00
Anita Akaeze d4cacc7232
Merge pull request #5200 from hashicorp/NET-3758 (#17102)
* Merge pull request #5200 from hashicorp/NET-3758

NET-3758: connect: update supported envoy versions to 1.26.0

* lint
2023-04-24 18:23:24 +00:00
Paul Banks a011d8c944
Bump raft to 1.5.0 (#17081)
* Bump raft to 1.5.0

* Add CHANGELOG entry

* Add CHANGELOG entry with right extension (thanks VSCode)

* Add CHANGELOG entry with right extension (thanks VSCode)

* Go mod tidy
2023-04-21 20:13:55 +01:00
Paul Glass 77ecff3209
Permissive mTLS (#17035)
This implements permissive mTLS , which allows toggling services into "permissive" mTLS mode.
Permissive mTLS mode allows incoming "non Consul-mTLS" traffic to be forward unmodified to the application.

* Update service-defaults and proxy-defaults config entries with a MutualTLSMode field
* Update the mesh config entry with an AllowEnablingPermissiveMutualTLS field and implement the necessary validation. AllowEnablingPermissiveMutualTLS must be true to allow changing to MutualTLSMode=permissive, but this does not require that all proxy-defaults and service-defaults are currently in strict mode.
* Update xDS listener config to add a "permissive filter chain" when MutualTLSMode=permissive for a particular service. The permissive filter chain matches incoming traffic by the destination port. If the destination port matches the service port from the catalog, then no mTLS is required and the traffic sent is forwarded unmodified to the application.
2023-04-19 14:45:00 -05:00
R.B. Boyer d07aac8d7e
Revert "cache: refactor agent cache fetching to prevent unnecessary f… (#16818) (#17046)
Revert "cache: refactor agent cache fetching to prevent unnecessary fetches on error (#14956)"

Co-authored-by: Derek Menteer <105233703+hashi-derek@users.noreply.github.com>
2023-04-19 13:17:21 -05:00
Kyle Havlovitz bdc3dd14c2
Avoid decoding nil pointer in map walker (#17048) 2023-04-19 10:23:38 -07:00
Kevin Wang 268f93e6f4
Bump the golang.org/x/net to 0.7.0 to address CVE-2022-41723 (#16754)
* Bump the golang.org/x/net to 0.7.0 to address CVE-2022-41723

https://nvd.nist.gov/vuln/detail/CVE-2022-41723

* Add changelog entry

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2023-04-18 17:31:08 +00:00
Andrei Komarov eb9f671eaf
api: enable query options on agent force-leave endpoint (#15987) 2023-04-18 11:31:48 -05:00
Nathan Coleman 5410139575
Update list of Envoy versions (#16889)
* Update list of Envoy versions

* Update docs + CI + tests

* Add changelog entry

* Add newly-released Envoy versions 1.23.8 and 1.24.6

* Add newly-released Envoy version 1.22.11
2023-04-12 17:43:15 -04:00
Dhia Ayachi b85a149eaf
Memdb Txn Commit race condition fix (#16871)
* Add a test to reproduce the race condition

* Fix race condition by publishing the event after the commit and adding a lock to prevent out of order events.

* split publish to generate the list of events before committing the transaction.

* add changelog

* remove extra func

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* add comment to explain test

---------

Co-authored-by: Dan Upton <daniel@floppy.co>
2023-04-12 13:18:01 -04:00
Derek Menteer 1bcaeabfc3
Remove deprecated service-defaults upstream behavior. (#16957)
Prior to this change, peer services would be targeted by service-default
overrides as long as the new `peer` field was not found in the config entry.
This commit removes that deprecated backwards-compatibility behavior. Now
it is necessary to specify the `peer` field in order for upstream overrides
to apply to a peer upstream.
2023-04-11 10:20:33 -05:00
Chris Thain 175bb1a303
Wasm Envoy HTTP extension (#16877) 2023-04-06 14:12:07 -07:00
Freddy f6de5ff635
Allow dialer to re-establish terminated peering (#16776)
Currently, if an acceptor peer deletes a peering the dialer's peering
will eventually get to a "terminated" state. If the two clusters need to
be re-peered the acceptor will re-generate the token but the dialer will
encounter this error on the call to establish:

"failed to get addresses to dial peer: failed to refresh peer server
addresses, will continue to use initial addresses: there is no active
peering for "<<<ID>>>""

This is because in `exchangeSecret().GetDialAddresses()` we will get an
error if fetching addresses for an inactive peering. The peering shows
up as inactive at this point because of the existing terminated state.

Rather than checking whether a peering is active we can instead check
whether it was deleted. This way users do not need to delete terminated
peerings in the dialing cluster before re-establishing them.
2023-04-03 12:07:45 -06:00
Eric Haberkorn 0d1d2fc4c9
add order by locality failover to Consul enterprise (#16791) 2023-03-30 10:08:38 -04:00
John Maguire c833464daf
Update normalization of route refs (#16789)
* Use merge of enterprise meta's rather than new custom method

* Add merge logic for tcp routes

* Add changelog

* Normalize certificate refs on gateways

* Fix infinite call loop

* Explicitly call enterprise meta
2023-03-28 11:23:49 -04:00
Michael Wilkerson e5d58c59c9
changes to support new PQ enterprise fields (#16793) 2023-03-27 15:40:49 -07:00
John Maguire 351bdc3c0d
Fix struct tags for TCPService enterprise meta (#16781)
* Fix struct tags for TCPService enterprise meta

* Add changelog
2023-03-27 16:17:04 +00:00
Derek Menteer 2236975011
Change partition for peers in discovery chain targets (#16769)
This commit swaps the partition field to the local partition for
discovery chains targeting peers. Prior to this change, peer upstreams
would always use a value of default regardless of which partition they
exist in. This caused several issues in xds / proxycfg because of id
mismatches.

Some prior fixes were made to deal with one-off id mismatches that this
PR also cleans up, since they are no longer needed.
2023-03-24 15:40:19 -05:00
Luke Kysow 4845816c60
Changelog for audit logging fix. (#16700)
* Changelog for audit logging fix.
2023-03-22 13:06:53 -07:00
Eric Haberkorn 3c5c53aa80
fix bug where pqs that failover to a cluster peer dont un-fail over (#16729) 2023-03-22 09:24:13 -04:00
Nitya Dhanushkodi b9bd2c3780
peering: peering partition failover fixes (#16673)
add local source partition for peered upstreams
2023-03-20 10:00:29 -07:00
John Maguire 1ef9f4dade
Fix route subscription when using namespaces (#16677)
* Fix route subscription when using namespaces

* Update changelog

* Fix changelog entry to reference that the bug was enterprise only
2023-03-20 12:42:30 -04:00
Melisa Griffin 606f8fbbab
Adds check to verify that the API Gateway is being created with at least one listener 2023-03-20 12:37:30 -04:00
Dhia Ayachi b9d8552e25
Snapshot restore tests (#16647)
* add snapshot restore test

* add logstore as test parameter

* Use the correct image version

* make sure we read the logs from a followers to test the follower snapshot install path.

* update to raf-wal v0.3.0

* add changelog.

* updating changelog for bug description and removed integration test.

* setting up test container builder to only set logStore for 1.15 and higher

---------

Co-authored-by: Paul Banks <pbanks@hashicorp.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-03-18 14:43:22 -06:00
Andrew Stucki 501b87fd31
[API Gateway] Fix invalid cluster causing gateway programming delay (#16661)
* Add test for http routes

* Add fix

* Fix tests

* Add changelog entry

* Refactor and fix flaky tests
2023-03-17 13:31:04 -04:00
Valeriia Ruban b473151994
fix: add AccessorID property to PUT token request (#16660) 2023-03-16 18:57:59 -07:00
Valeriia Ruban ad25ba3068
feat: update typography to consume hds styles (#16577) 2023-03-14 19:49:14 -07:00
Derek Menteer 8f75d99299
Fix issue with trust bundle read ACL check. (#16630)
This commit fixes an issue where trust bundles could not be read
by services in a non-default namespace, unless they had excessive
ACL permissions given to them.

Prior to this change, `service:write` was required in the default
namespace in order to read the trust bundle. Now, `service:write`
to a service in any namespace is sufficient.
2023-03-14 12:24:33 -05:00
Chris S. Kim d5677e5680
Preserve CARoots when updating Vault CA configuration (#16592)
If a CA config update did not cause a root change, the codepath would return early and skip some steps which preserve its intermediate certificates and signing key ID. This commit re-orders some code and prevents updates from generating new intermediate certificates.
2023-03-13 17:32:59 -04:00
Ashvitha f95ffe0355
Allow HCP metrics collection for Envoy proxies
Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>

Add a new envoy flag: "envoy_hcp_metrics_bind_socket_dir", a directory
where a unix socket will be created with the name
`<namespace>_<proxy_id>.sock` to forward Envoy metrics.

If set, this will configure:
- In bootstrap configuration a local stats_sink and static cluster.
  These will forward metrics to a loopback listener sent over xDS.

- A dynamic listener listening at the socket path that the previously
  defined static cluster is sending metrics to.

- A dynamic cluster that will forward traffic received at this listener
  to the hcp-metrics-collector service.


Reasons for having a static cluster pointing at a dynamic listener:
- We want to secure the metrics stream using TLS, but the stats sink can
  only be defined in bootstrap config. With dynamic listeners/clusters
  we can use the proxy's leaf certificate issued by the Connect CA,
  which isn't available at bootstrap time.

- We want to intelligently route to the HCP collector. Configuring its
  addreess at bootstrap time limits our flexibility routing-wise. More
  on this below.

Reasons for defining the collector as an upstream in `proxycfg`:
- The HCP collector will be deployed as a mesh service.

- Certificate management is taken care of, as mentioned above.

- Service discovery and routing logic is automatically taken care of,
  meaning that no code changes are required in the xds package.

- Custom routing rules can be added for the collector using discovery
  chain config entries. Initially the collector is expected to be
  deployed to each admin partition, but in the future could be deployed
  centrally in the default partition. These config entries could even be
  managed by HCP itself.
2023-03-10 13:52:54 -07:00
Tyler Wendlandt e6aeb31a26
UI: Fix htmlsafe errors throughout the app (#16574)
* Upgrade ember-intl

* Add changelog

* Add yarn lock
2023-03-09 12:43:35 -07:00
Eric Haberkorn 89de91b263
fix bug that can lead to peering service deletes impacting the state of local services (#16570) 2023-03-08 11:24:03 -05:00
John Eikenberry f5641ffccc
support vault auth config for alicloud ca provider
Add support for using existing vault auto-auth configurations as the
provider configuration when using Vault's CA provider with AliCloud.

AliCloud requires 2 extra fields to enable it to use STS (it's preferred
auth setup). Our vault-plugin-auth-alicloud package contained a method
to help generate them as they require you to make an http call to
a faked endpoint proxy to get them (url and headers base64 encoded).
2023-03-07 03:02:05 +00:00
Valeriia Ruban 63204b5183
feat: update notification to use hds toast component (#16519) 2023-03-06 14:10:09 -08:00
Chris S. Kim 8daddff08d
Follow-up fixes to consul connect envoy command (#16530) 2023-03-06 10:32:06 -05:00
Ronald bf501a337b
Improve ux around ACL token to help users avoid overwriting node/service identities (#16506)
* Deprecate merge-node-identities and merge-service-identities flags

* added tests for node identities changes

* added changelog file and docs
2023-03-06 15:00:39 +00:00
Melisa Griffin fc232326a0
NET-2904 Fixes API Gateway Route Service Weight Division Error 2023-03-06 08:41:57 -05:00
Andrew Stucki 897e5ef2d3
Add some basic UI improvements for api-gateway services (#16508)
* Add some basic ui improvements for api-gateway services

* Add changelog entry

* Use ternary for null check

* Update gateway doc links

* rename changelog entry for new PR

* Fix test
2023-03-03 16:59:04 -05:00
Melisa Griffin 129eca8fdb
NET-2903 Normalize weight for http routes (#16512)
* NET-2903 Normalize weight for http routes

* Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-03-03 16:39:59 -05:00
R.B. Boyer 9a485cdb49
proxycfg: ensure that an irrecoverable error in proxycfg closes the xds session and triggers a replacement proxycfg watcher (#16497)
Receiving an "acl not found" error from an RPC in the agent cache and the
streaming/event components will cause any request loops to cease under the
assumption that they will never work again if the token was destroyed. This
prevents log spam (#14144, #9738).

Unfortunately due to things like:

- authz requests going to stale servers that may not have witnessed the token
  creation yet

- authz requests in a secondary datacenter happening before the tokens get
  replicated to that datacenter

- authz requests from a primary TO a secondary datacenter happening before the
  tokens get replicated to that datacenter

The caller will get an "acl not found" *before* the token exists, rather than
just after. The machinery added above in the linked PRs will kick in and
prevent the request loop from looping around again once the tokens actually
exist.

For `consul-dataplane` usages, where xDS is served by the Consul servers
rather than the clients ultimately this is not a problem because in that
scenario the `agent/proxycfg` machinery is on-demand and launched by a new xDS
stream needing data for a specific service in the catalog. If the watching
goroutines are terminated it ripples down and terminates the xDS stream, which
CDP will eventually re-establish and restart everything.

For Consul client usages, the `agent/proxycfg` machinery is ahead-of-time
launched at service registration time (called "local" in some of the proxycfg
machinery) so when the xDS stream comes in the data is already ready to go. If
the watching goroutines terminate it should terminate the xDS stream, but
there's no mechanism to re-spawn the watching goroutines. If the xDS stream
reconnects it will see no `ConfigSnapshot` and will not get one again until
the client agent is restarted, or the service is re-registered with something
changed in it.

This PR fixes a few things in the machinery:

- there was an inadvertent deadlock in fetching snapshot from the proxycfg
  machinery by xDS, such that when the watching goroutine terminated the
  snapshots would never be fetched. This caused some of the xDS machinery to
  get indefinitely paused and not finish the teardown properly.

- Every 30s we now attempt to re-insert all locally registered services into
  the proxycfg machinery.

- When services are re-inserted into the proxycfg machinery we special case
  "dead" ones such that we unilaterally replace them rather that doing that
  conditionally.
2023-03-03 14:27:53 -06:00
John Eikenberry 56ffee6d42
add provider ca support for approle auth-method
Adds support for the approle auth-method. Only handles using the approle
role/secret to auth and it doesn't support the agent's extra management
configuration options (wrap and delete after read) as they are not
required as part of the auth (ie. they are vault agent things).
2023-03-03 19:29:53 +00:00
Andrew Stucki cc0765b87d
Fix resolution of service resolvers with subsets for external upstreams (#16499)
* Fix resolution of service resolvers with subsets for external upstreams

* Add tests

* Add changelog entry

* Update view filter logic
2023-03-03 14:17:11 -05:00
Andrew Stucki 5deffbd95b
Fix issue where terminating gateway service resolvers weren't properly cleaned up (#16498)
* Fix issue where terminating gateway service resolvers weren't properly cleaned up

* Add integration test for cleaning up resolvers

* Add changelog entry

* Use state test and drop integration test
2023-03-03 09:56:57 -05:00
Andrew Stucki 4b661d1e0c
Add ServiceResolver RequestTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable (#16495)
* Leverage ServiceResolver ConnectTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable

* Regenerate golden files

* Add RequestTimeout field

* Add changelog entry
2023-03-03 09:37:12 -05:00
John Eikenberry e8eec1fa80
add provider ca auth support for kubernetes
Adds support for Kubernetes jwt/token file based auth. Only needs to
read the file and save the contents as the jwt/token.
2023-03-02 22:05:40 +00:00
John Eikenberry 4211069080
add provider ca support for jwt file base auth
Adds support for a jwt token in a file. Simply reads the file and sends
the read in jwt along to the vault login.

It also supports a legacy mode with the jwt string being passed
directly. In which case the path is made optional.
2023-03-02 20:33:06 +00:00
Ronald 4f8594b28f
Improve ux to help users avoid overwriting fields of ACL tokens, roles and policies (#16288)
* Deprecate merge-policies and add options add-policy-name/add-policy-id to improve CLI token update command

* deprecate merge-roles fields

* Fix potential flakey tests and update ux to remove 'completely' + typo fixes
2023-03-01 15:00:37 -05:00
cskh 3970115753
fix (cli): return error msg if acl policy not found (#16485)
* fix: return error msg if acl policy not found

* changelog

* add test
2023-03-01 19:50:03 +00:00
John Eikenberry 4f2d9a91e5
add provider ca auth-method support for azure
Does the required dance with the local HTTP endpoint to get the required
data for the jwt based auth setup in Azure. Keeps support for 'legacy'
mode where all login data is passed on via the auth methods parameters.
Refactored check for hardcoded /login fields.
2023-03-01 00:07:33 +00:00
R.B. Boyer 26820219cd
cli: ensure acl token read -self works (#16445)
Fixes a regression in #16044

The consul acl token read -self cli command should not require an -accessor-id because typically the persona invoking this would not already know the accessor id of their own token.
2023-02-28 10:58:29 -06:00
Tyler Wendlandt a0862e64a5
UI: Fix rendering issue in search and lists (#16444)
* Upgrade ember-cli-string-helpers

* add extra lock change
2023-02-27 16:31:47 -07:00
Valeriia Ruban 859abf8a34
fix: ui tests run is fixed (applying class attribute twice to the hbs element caused the issue (#16428) 2023-02-24 23:46:45 -08:00
Valeriia Ruban d9e6748738
feat: update alerts to Hds::Alert component (CC-4035) (#16412) 2023-02-24 20:07:12 -08:00
Valeriia Ruban 0c66bbf2b4
[UI] CC-4031: change from Action, a and button to hds::Button (#16251) 2023-02-22 13:05:15 -08:00
Nathan Coleman c0384c2e30
Add changelog entry for API Gateway (Beta) (#16369)
* Placeholder commit for changelog entry

* Add changelog entry announcing support for API Gateway on VMs

* Adjust casing
2023-02-22 13:10:05 -06:00
Derek Menteer 5309f68bc0
Upgrade Alpine image to 3.17 (#16358) 2023-02-22 10:09:41 -06:00
Derek Menteer ad865f549b
Fix issue with peer services incorrectly appearing as connect-enabled. (#16339)
Prior to this commit, all peer services were transmitted as connect-enabled
as long as a one or more mesh-gateways were healthy. With this change, there
is now a difference between typical services and connect services transmitted
via peering.

A service will be reported as "connect-enabled" as long as any of these
conditions are met:

1. a connect-proxy sidecar is registered for the service name.
2. a connect-native instance of the service is registered.
3. a service resolver / splitter / router is registered for the service name.
4. a terminating gateway has registered the service.
2023-02-21 13:59:36 -06:00
cskh 8e5942f5ca
fix: add tls config to unix socket when https is used (#16301)
* fix: add tls config to unix socket when https is used

* unit test and changelog
2023-02-21 08:28:13 -05:00
malizz c9c49ea3a2
new docs for consul and consul-k8s troubleshoot command (#16284)
* new docs for consul and consul-k8s troubleshoot command

* add changelog

* add troubleshoot command

* address comments, and update cli output to match

* revert changes to troubleshoot upstreams, changes will happen in separate pr

* Update .changelog/16284.txt

Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>

* address comments

* update trouble proxy output

* add missing s, add required fields in usage

---------

Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
2023-02-17 13:25:49 -08:00
Dan Stough f1436109ea
[OSS] security: update go to 1.20.1 (#16263)
* security: update go to 1.20.1
2023-02-17 15:04:12 -05:00
Dhia Ayachi 388876c206
add server side rate-limiter changelog entry (#16292) 2023-02-16 19:21:50 +00:00
Derek Menteer 30112288c8
Fix mesh gateways incorrectly matching peer locality. (#16257)
Fix mesh gateways incorrectly matching peer locality.

This fixes an issue where local mesh gateways use an
incorrect address when attempting to forward traffic to a
peered datacenter. Prior to this change it would use the
lan address instead of the wan if the locality matched. This
should never be done for peering, since we must route all
traffic through the remote mesh gateway.
2023-02-16 09:22:41 -06:00
Curt Bushko 1d9ee50681
[OSS] connect: Bump Envoy 1.22.5 to 1.22.7, 1.23.2 to 1.23.4, 1.24.0 to 1.24.2, add 1.25.1, remove 1.21.5 (#16274)
* Bump Envoy 1.22.5 to 1.22.7, 1.23.2 to 1.23.4, 1.24.0 to 1.24.2, add 1.25.1, remove 1.21.5
2023-02-15 11:45:43 -05:00
Tyler Wendlandt 3f22879106
UI: CC-4032 - Update sidebar width (#16204)
* Update chrome-width var to be 280px

* Formatting & Changelog
2023-02-13 11:48:31 -07:00
Valeriia Ruban 663a5642c2
[UI]: update Ember to 3.27 (#16227)
* Upgrade to 3.25 via ember-cli-update

* v3.25.3...v3.26.1

* v3.26.1...v3.27.0


Co-authored-by: Michael Klein <michael@firstiwaslike.com>
2023-02-10 13:32:19 -08:00
Derek Menteer 4f2ce60654
Fix peering acceptors in secondary datacenters. (#16230)
Prior to this commit, secondary datacenters could not be initialized
as peering acceptors if ACLs were enabled. This is due to the fact that
internal server-to-server API calls would fail because the management
token was not generated. This PR makes it so that both primary and
secondary datacenters generate their own management token whenever
a leader is elected in their respective clusters.
2023-02-10 09:47:17 -06:00
skpratt 6f0b226b0d
ACL error improvements: incomplete bootstrapping and non-existent token (#16105)
* add bootstrapping detail for acl errors

* error detail improvements

* update acl bootstrapping test coverage

* update namespace errors

* update test coverage

* add changelog

* update message for unbootstrapped error

* consolidate error message code and update changelog

* logout message change
2023-02-08 23:49:44 +00:00
Kyle Havlovitz 898e59b13c
Add the `operator usage instances` command and api endpoint (#16205)
This endpoint shows total services, connect service instances and
billable service instances in the local datacenter or globally. Billable
instances = total service instances - connect services - consul server instances.
2023-02-08 12:07:21 -08:00
Paul Banks 5397e9ee7f
Adding experimental support for a more efficient LogStore implementation (#16176)
* Adding experimental support for a more efficient LogStore implementation

* Adding changelog entry

* Fix go mod tidy issues
2023-02-08 16:50:22 +00:00
skpratt 9199e99e21
Update token language to distinguish Accessor and Secret ID usage (#16044)
* remove legacy tokens

* remove lingering legacy token references from docs

* update language and naming for token secrets and accessor IDs

* updates all tokenID references to clarify accessorID

* remove token type references and lookup tokens by accessorID index

* remove unnecessary constants

* replace additional tokenID param names

* Add warning info for deprecated -id parameter

Co-authored-by: Paul Glass <pglass@hashicorp.com>

* Update field comment

Co-authored-by: Paul Glass <pglass@hashicorp.com>

---------

Co-authored-by: Paul Glass <pglass@hashicorp.com>
2023-02-07 12:26:30 -06:00
Dhia Ayachi e42ab7e429
Remove empty tags 2 (#16113)
* Add support for RemoveEmptyTags in API client

* Add changelog

---------

Co-authored-by: Rémi Lapeyre <remi.lapeyre@lenstra.fr>
2023-02-06 11:12:43 -08:00
skpratt a010902978
Remove legacy acl policies (#15922)
* remove legacy tokens

* remove legacy acl policies

* flatten test policies to *_prefix

* address oss feedback re: phrasing and tests
2023-02-06 15:35:52 +00:00
Derek Menteer 2f149d60cc
[OSS] Add Peer field to service-defaults upstream overrides (#15956)
* Add Peer field to service-defaults upstream overrides.

* add api changes, compat mode for service default overrides

* Fixes based on testing

---------

Co-authored-by: DanStough <dan.stough@hashicorp.com>
2023-02-03 10:51:53 -05:00
Paul Glass a884d0d7c7
Use agent token for service/check deregistration during anti-entropy (#16097)
Use only the agent token for deregistration during anti-entropy

The previous behavior had the agent attempt to use the "service" token
(i.e. from the `token` field in a service definition file), and if that
was not set then it would use the agent token.

The previous behavior was problematic because, if the service token had
been deleted, the deregistration request would fail. The agent would
retry the deregistration during each anti-entropy sync, and the
situation would never resolve.

The new behavior is to only/always use the agent token for service and
check deregistration during anti-entropy. This approach is:

* Simpler: No fallback logic to try different tokens
* Faster (slightly): No time spent attempting the service token
* Correct: The agent token is able to deregister services on that
  agent's node, because:
  * node:write permissions allow deregistration of services/checks on
    that node.
  * The agent token must have node:write permission, or else the agent
    is not be able to (de)register itself into the catalog

Co-authored-by: Vesa Hagström <weeezes@gmail.com>
2023-02-03 08:45:11 -06:00
Kyle Havlovitz d53c331a37
Add a flag for enabling debug logs to the `connect envoy` command (#15988)
* Add a flag for enabling debug logs to the `connect envoy` command

* Update website/content/commands/connect/envoy.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Add changelog note

* Add debug log note to envoy proxy doc page

* Update website/content/docs/connect/proxies/envoy.mdx

Co-authored-by: Kendall Strautman <36613477+kendallstrautman@users.noreply.github.com>

* Wording tweak in envoy bootstrap section

---------

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Kendall Strautman <36613477+kendallstrautman@users.noreply.github.com>
2023-01-31 13:30:20 -08:00
skpratt ad43846755
Remove legacy acl tokens (#15947)
* remove legacy tokens

* Update test comment

Co-authored-by: Paul Glass <pglass@hashicorp.com>

* fix imports

* update docs for additional CLI changes

* add test case for anonymous token

* set deprecated api fields to json ignore and fix patch errors

* update changelog to breaking-change

* fix import

* update api docs to remove legacy reference

* fix docs nav data

---------

Co-authored-by: Paul Glass <pglass@hashicorp.com>
2023-01-27 09:17:07 -06:00
Ronald 6167aef641
Warn when the token query param is used for auth (#16009) 2023-01-24 16:21:41 +00:00
cskh 25396d81c9
Apply agent partition to load services and agent api (#16024)
* Apply agent partition to load services and agent api

changelog
2023-01-20 12:59:26 -05:00
Ashwin Venkatesh a1e2a4f8d6
Add support for envoy readiness flags (#16015)
* Add support for envoy readiness flags
- add flags 'envoy-ready-bind-port` and `envoy-ready-bind-addr` on consul connect envoy to create a ready listener on that address.
2023-01-19 16:54:11 -05:00
Chris Thain 2f4c8e50f2
Support Vault agent auth config for AWS/GCP CA provider auth (#15970) 2023-01-18 11:53:04 -08:00
Derek Menteer 2facf50923
Fix configuration merging for implicit tproxy upstreams. (#16000)
Fix configuration merging for implicit tproxy upstreams.

Change the merging logic so that the wildcard upstream has correct proxy-defaults
and service-defaults values combined into it. It did not previously merge all fields,
and the wildcard upstream did not exist unless service-defaults existed (it ignored
proxy-defaults, essentially).

Change the way we fetch upstream configuration in the xDS layer so that it falls back
to the wildcard when no matching upstream is found. This is what allows implicit peer
upstreams to have the correct "merged" config.

Change proxycfg to always watch local mesh gateway endpoints whenever a peer upstream
is found. This simplifies the logic so that we do not have to inspect the "merged"
configuration on peer upstreams to extract the mesh gateway mode.
2023-01-18 13:43:53 -06:00
Dan Upton 7a55de375c
xds: don't attempt to load-balance sessions for local proxies (#15789)
Previously, we'd begin a session with the xDS concurrency limiter
regardless of whether the proxy was registered in the catalog or in
the server's local agent state.

This caused problems for users who run `consul connect envoy` directly
against a server rather than a client agent, as the server's locally
registered proxies wouldn't be included in the limiter's capacity.

Now, the `ConfigSource` is responsible for beginning the session and we
only do so for services in the catalog.

Fixes: https://github.com/hashicorp/consul/issues/15753
2023-01-18 12:33:21 -06:00
Chris S. Kim e4a268e33e
Warn if ACL is enabled but no token is provided to Envoy (#15967) 2023-01-16 12:31:56 -05:00
Derek Menteer 19a46d6ca4
Enforce lowercase peer names. (#15697)
Enforce lowercase peer names.

Prior to this change peer names could be mixed case.
This can cause issues, as peer names are used as DNS labels
in various locations. It also caused issues with envoy
configuration.
2023-01-13 14:20:28 -06:00
Frank DiRocco 59a3a0749c
Update go-discover to support ECS discovery (#13782)
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-01-12 12:06:29 -06:00
Dan Stough 5d3643f4f0
docs(access logs): new docs for access logging (#15948)
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2023-01-11 16:41:02 +00:00
Paul Glass f5231b9157
Add new config_file_service_registration token (#15828) 2023-01-10 10:24:02 -06:00
Chris S. Kim a7b34d50fc
Output user-friendly name for anonymous token (#15884) 2023-01-09 12:28:53 -06:00
Derek Menteer 7b4f45e2d5
Fix issue where TLS configuration was ignored for unix sockets in consul connect envoy. (#15913)
Fix issue where TLS configuration was ignored for unix sockets in consul connect envoy.
Disable xds check on bootstrap mode and change check to warn only.
2023-01-06 12:34:49 -06:00
Eric Haberkorn 8d923c1789
Add the Lua Envoy extension (#15906) 2023-01-06 12:13:40 -05:00
Dan Upton d53ce39c32
grpc: switch servers and retry on error (#15892)
This is the OSS portion of enterprise PR 3822.

Adds a custom gRPC balancer that replicates the router's server cycling
behavior. Also enables automatic retries for RESOURCE_EXHAUSTED errors,
which we now get for free.
2023-01-05 10:21:27 +00:00
Nick Irvine 6fb628c07d
fix: return error when config file with unknown extension is passed (#15107) 2023-01-04 16:57:00 -08:00
Florian Apolloner 077b0a48a3
Allow Operator Generated bootstrap token (#14437)
Add support to provide an initial token via the bootstrap HTTP API, similar to hashicorp/nomad#12520
2023-01-04 20:19:33 +00:00
Dan Upton 7c7503c849
grpc/acl: relax permissions required for "core" endpoints (#15346)
Previously, these endpoints required `service:write` permission on _any_
service as a sort of proxy for "is the caller allowed to participate in
the mesh?".

Now, they're called as part of the process of establishing a server
connection by any consumer of the consul-server-connection-manager
library, which will include non-mesh workloads (e.g. Consul KV as a
storage backend for Vault) as well as ancillary components such as
consul-k8s' acl-init process, which likely won't have `service:write`
permission.

So this commit relaxes those requirements to accept *any* valid ACL token
on the following gRPC endpoints:

- `hashicorp.consul.dataplane.DataplaneService/GetSupportedDataplaneFeatures`
- `hashicorp.consul.serverdiscovery.ServerDiscoveryService/WatchServers`
- `hashicorp.consul.connectca.ConnectCAService/WatchRoots`
2023-01-04 12:40:34 +00:00
Derek Menteer 1f7e7abeac
Fix issue with incorrect proxycfg watch on upstream peer-targets. (#15865)
This fixes an issue where the incorrect partition was given to the
upstream target watch, which meant that failover logic would not
work correctly.
2023-01-03 10:44:08 -06:00
Derek Menteer f3776894bf
Fix agent cache incorrectly notifying unchanged protobufs. (#15866)
Fix agent cache incorrectly notifying unchanged protobufs.

This change fixes a situation where the protobuf private fields
would be read by reflect.DeepEqual() and indicate data was modified.
This resulted in change notifications being fired every time, which
could cause performance problems in proxycfg.
2023-01-03 10:11:56 -06:00
Dan Stough b3bd3a6586
[OSS] feat: access logs for listeners and listener filters (#15864)
* feat: access logs for listeners and listener filters

* changelog

* fix integration test
2022-12-22 15:18:15 -05:00