This change adds ACL hooks to the remaining catalog and mesh resources, excluding any computed ones. Those will for now continue using the default operator:x permissions.
It refactors a lot of the common testing functions so that they can be re-used between resources.
There are also some types that we don't yet support (e.g. virtual IPs) that this change adds ACL hooks to for future-proofing.
This implements the Filter field on pbcatalog.WorkloadSelector to be
a post-fetch in-memory filter using the https://github.com/hashicorp/go-bexpr
expression language to filter resources based on their envelope metadata fields.
All existing usages of WorkloadSelector should be able to make use of the filter.
* xdsv2: support l7 by adding xfcc policy/headers, tweaking routes, and make a bunch of listeners l7 tests pass
* sidecarproxycontroller: add l7 local app support
* trafficpermissions: make l4 traffic permissions work on l7 workloads
* rename route name field for consistency with l4 cluster name field
* resolve conflicts and rebase
* fix: ensure route name is used in l7 destination route name as well. previously it was only in the route names themselves, now the route name and l7 destination route name line up
Sometimes workloads could come with unspecified protocols such as when running on Kubernetes. Currently, if this is the case, we will just default to tcp protocol.
However, to make sidecar-proxy controller work with l7 protocols we should instead inherit the protocol from service. This change adds tracking for services that a workload is part of and attempts to inherit the protocol whenever services a workload is part of doesn't have conflicting protocols.
This change builds on #19043 and #19067 and updates the sidecar controller to use those computed resources. This achieves several benefits:
* The cache is now simplified which helps us solve for previous bugs (such as multiple Upstreams/Destinations targeting the same service would overwrite each other)
* We no longer need proxy config cache
* We no longer need to do merging of proxy configs as part of the controller logic
* Controller watches are simplified because we no longer need to have complex mapping using cache and can instead use the simple ReplaceType mapper.
It also makes several other improvements/refactors:
* Unifies all caches into one. This is because originally the caches were more independent, however, now that they need to interact with each other it made sense to unify them where sidecar proxy controller uses one cache with 3 bimappers
* Unifies cache and mappers. Mapper already needed all caches anyway and so it made sense to make the cache do the mapping also now that the cache is unified.
* Gets rid of service endpoints watches. This was needed to get updates in a case when service's identities have changed and we need to update proxy state template's spiffe IDs for those destinations. This will however generate a lot of reconcile requests for this controller as service endpoints objects can change a lot because they contain workload's health status. This is solved by adding a status to the service object tracking "bound identities" and have service endpoints controller update it. Having service's status updated allows us to get updates in the sidecar proxy controller because it's already watching service objects
* Add a watch for workloads. We need it so that we get updates if workload's ports change. This also ensures that we update cached identities in case workload's identity changes.
This commit adds a new type ComputedDestinations that will contain all destinations from any Destinations resources and will be name-aligned with a workload. This also adds an explicit-destinations controller that computes these resources.
This is needed to simplify the tracking we need to do currently in the sidecar-proxy controller and makes it easier to query all explicit destinations that apply to a workload.
We serially attempt to decode resources in the consul resource apply command
using HCL and then falling back on JSON. This causes the HCL errors to be
dropped completely in the case where the HCL decode failed due to a typo
instead of it actually being JSON instead.
This PR proposes sniffing to see if the first non-whitespace character in the
input is { and if so treat it as JSON, otherwise as HCL and not
double-decode on error.
When the v2 catalog experiment is enabled the old v1 catalog apis will be
forcibly disabled at both the API (json) layer and the RPC (msgpack) layer.
This will also disable anti-entropy as it uses the v1 api.
This includes all of /v1/catalog/*, /v1/health/*, most of /v1/agent/*,
/v1/config/*, and most of /v1/internal/*.
* Introduce a new type `ComputedProxyConfiguration` and add a controller for it. This is needed for two reasons. The first one is that external integrations like kubernetes may need to read the fully computed and sorted proxy configuration per workload. The second reasons is that it makes sidecar-proxy controller logic quite a bit simpler as it no longer needs to do this.
* Generalize workload selection mapper and fix a bug where it would delete IDs from the tree if only one is left after a removal is done.
* update main apigw overview
* moved the tech specs to main gw folder
* merged tech specs into single topic
* restructure nav part 1
* fix typo in nav json file
* moved k8s install up one level
* restructure nav part 2
* moved and created all listeners and routes content
* moved errors ref and upgrades
* fix error in upgrade-k8s link
* moved conf refs to appropriate spots
* updated conf overview
* fixed some links and bad formatting
* fixed link
* added JWT on VMs usage page
* added JWT conf to APIGW conf entry
* added JWTs to HTTP route conf entry
* added new gatwaypolicy k8s conf reference
* added metadesc for gatewaypolicy conf ref
* added http route auth filter k8s conf ref
* added http route auth filter k8s conf ref to nav
* updates to k8s route conf ref to include extensionRef
* added JWTs usage page for k8s
* fixed link in gwpolicy conf ref
* added openshift installation info to installation pages
* fixed bad link on tech specs
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* fixed VerityClaims param
* best guess at verifyclaims params
* tweaks to gateway policy dconf ref
* Docs/ce 475 retries timeouts for apigw (#19086)
* added timeout and retry conf ref for k8s
* added retry and TO filters to HTTP routes conf ref for VMs
* Apply suggestions from code review
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
* fix copy/paste error in http route conf entry
---------
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
* update links across site and add redirects
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Thomas Eckert <teckert@hashicorp.com>
* Applied feedback from review
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* Update CRD configuration for responseHeaderModifiers
* Update Config Entry for http-route
* Add ResponseFilter example to service
* Update website/redirects.js
errant curly brace breaking the preview
* fix links and bad MD
* fixed md formatting issues
* fix formatting errors
* fix formatting errors
* Update website/content/docs/connect/gateways/api-gateway/secure-traffic/verify-jwts-k8s.mdx
* Apply suggestions from code review
* fixed typo
* Fix headers in http-route
* Apply suggestions from code review
Co-authored-by: John Maguire <john.maguire@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
---------
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
Co-authored-by: Thomas Eckert <teckert@hashicorp.com>
Co-authored-by: John Maguire <john.maguire@hashicorp.com>
* updated nav; renamed L7 traffic folder
* Added locality-aware routing to traffic mgmt overview
* Added route to local upstreams topic
* Updated agent configuration reference
* Added locality param to services conf ref
* Added locality param to conf entries
* mentioned traffic management in proxies overview
* added locality-aware to failover overview
* added docs for service rate limiting
* updated service defaults conf entry
* Apply suggestions from code review
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
* updated links and added redirects
---------
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
This PR fixes an issue where upstreams did not correctly inherit the proper
namespace / partition from the parent service when attempting to fetch the
upstream protocol due to inconsistent normalization.
Some of the merge-service-configuration logic would normalize to default, while
some of the proxycfg logic would normalize to match the parent service. Due to
this mismatch in logic, an incorrect service-defaults configuration entry would
be fetched and have its protocol applied to the upstream.
Iryna Shustava
Michael Zalimeni
Ashwin Venkatesh
R.B. Boyer
Dhia Ayachi
Thomas Eckert
Poonam Jadhav
Nitya Dhanushkodi
John Murret
Derek Menteer
trujillo-adam
Chris S. Kim
Semir Patel
John Maguire
Tu Nguyen
Jeff Boruszak
Chris Thain
Ashesh Vidyut
Matt Keeler