This change builds on #19043 and #19067 and updates the sidecar controller to use those computed resources. This achieves several benefits:
* The cache is now simplified which helps us solve for previous bugs (such as multiple Upstreams/Destinations targeting the same service would overwrite each other)
* We no longer need proxy config cache
* We no longer need to do merging of proxy configs as part of the controller logic
* Controller watches are simplified because we no longer need to have complex mapping using cache and can instead use the simple ReplaceType mapper.
It also makes several other improvements/refactors:
* Unifies all caches into one. This is because originally the caches were more independent, however, now that they need to interact with each other it made sense to unify them where sidecar proxy controller uses one cache with 3 bimappers
* Unifies cache and mappers. Mapper already needed all caches anyway and so it made sense to make the cache do the mapping also now that the cache is unified.
* Gets rid of service endpoints watches. This was needed to get updates in a case when service's identities have changed and we need to update proxy state template's spiffe IDs for those destinations. This will however generate a lot of reconcile requests for this controller as service endpoints objects can change a lot because they contain workload's health status. This is solved by adding a status to the service object tracking "bound identities" and have service endpoints controller update it. Having service's status updated allows us to get updates in the sidecar proxy controller because it's already watching service objects
* Add a watch for workloads. We need it so that we get updates if workload's ports change. This also ensures that we update cached identities in case workload's identity changes.
This commit adds a new type ComputedDestinations that will contain all destinations from any Destinations resources and will be name-aligned with a workload. This also adds an explicit-destinations controller that computes these resources.
This is needed to simplify the tracking we need to do currently in the sidecar-proxy controller and makes it easier to query all explicit destinations that apply to a workload.
We serially attempt to decode resources in the consul resource apply command
using HCL and then falling back on JSON. This causes the HCL errors to be
dropped completely in the case where the HCL decode failed due to a typo
instead of it actually being JSON instead.
This PR proposes sniffing to see if the first non-whitespace character in the
input is { and if so treat it as JSON, otherwise as HCL and not
double-decode on error.
When the v2 catalog experiment is enabled the old v1 catalog apis will be
forcibly disabled at both the API (json) layer and the RPC (msgpack) layer.
This will also disable anti-entropy as it uses the v1 api.
This includes all of /v1/catalog/*, /v1/health/*, most of /v1/agent/*,
/v1/config/*, and most of /v1/internal/*.
* Introduce a new type `ComputedProxyConfiguration` and add a controller for it. This is needed for two reasons. The first one is that external integrations like kubernetes may need to read the fully computed and sorted proxy configuration per workload. The second reasons is that it makes sidecar-proxy controller logic quite a bit simpler as it no longer needs to do this.
* Generalize workload selection mapper and fix a bug where it would delete IDs from the tree if only one is left after a removal is done.
* update main apigw overview
* moved the tech specs to main gw folder
* merged tech specs into single topic
* restructure nav part 1
* fix typo in nav json file
* moved k8s install up one level
* restructure nav part 2
* moved and created all listeners and routes content
* moved errors ref and upgrades
* fix error in upgrade-k8s link
* moved conf refs to appropriate spots
* updated conf overview
* fixed some links and bad formatting
* fixed link
* added JWT on VMs usage page
* added JWT conf to APIGW conf entry
* added JWTs to HTTP route conf entry
* added new gatwaypolicy k8s conf reference
* added metadesc for gatewaypolicy conf ref
* added http route auth filter k8s conf ref
* added http route auth filter k8s conf ref to nav
* updates to k8s route conf ref to include extensionRef
* added JWTs usage page for k8s
* fixed link in gwpolicy conf ref
* added openshift installation info to installation pages
* fixed bad link on tech specs
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* fixed VerityClaims param
* best guess at verifyclaims params
* tweaks to gateway policy dconf ref
* Docs/ce 475 retries timeouts for apigw (#19086)
* added timeout and retry conf ref for k8s
* added retry and TO filters to HTTP routes conf ref for VMs
* Apply suggestions from code review
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
* fix copy/paste error in http route conf entry
---------
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
* update links across site and add redirects
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Thomas Eckert <teckert@hashicorp.com>
* Applied feedback from review
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* Update CRD configuration for responseHeaderModifiers
* Update Config Entry for http-route
* Add ResponseFilter example to service
* Update website/redirects.js
errant curly brace breaking the preview
* fix links and bad MD
* fixed md formatting issues
* fix formatting errors
* fix formatting errors
* Update website/content/docs/connect/gateways/api-gateway/secure-traffic/verify-jwts-k8s.mdx
* Apply suggestions from code review
* fixed typo
* Fix headers in http-route
* Apply suggestions from code review
Co-authored-by: John Maguire <john.maguire@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
---------
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
Co-authored-by: Thomas Eckert <teckert@hashicorp.com>
Co-authored-by: John Maguire <john.maguire@hashicorp.com>
* updated nav; renamed L7 traffic folder
* Added locality-aware routing to traffic mgmt overview
* Added route to local upstreams topic
* Updated agent configuration reference
* Added locality param to services conf ref
* Added locality param to conf entries
* mentioned traffic management in proxies overview
* added locality-aware to failover overview
* added docs for service rate limiting
* updated service defaults conf entry
* Apply suggestions from code review
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
* updated links and added redirects
---------
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
This PR fixes an issue where upstreams did not correctly inherit the proper
namespace / partition from the parent service when attempting to fetch the
upstream protocol due to inconsistent normalization.
Some of the merge-service-configuration logic would normalize to default, while
some of the proxycfg logic would normalize to match the parent service. Due to
this mismatch in logic, an incorrect service-defaults configuration entry would
be fetched and have its protocol applied to the upstream.
* Add InboundPeerTrustBundle maps to Terminating Gateway
* Add notify and cancelation of watch for inbound peer trust bundles
* Pass peer trust bundles to the RBAC creation function
* Regenerate Golden Files
* add changelog, also adds another spot that needed peeredTrustBundles
* Add basic test for terminating gateway with peer trust bundle
* Add intention to cluster peered golden test
* rerun codegen
* update changelog
* really update the changelog
---------
Co-authored-by: Melisa Griffin <melisa.griffin@hashicorp.com>
* updated architecture topic
* fixed type in arch diagram filenames
* fixed path to img file
* updated index page - still need to add links
* moved arch and tech specs to reference folder
* moved other ref topics to ref folder
* set up the Deploy folder and TF install topics
* merged secure conf into TF deploy instructions
* moved bind addr and route conf to their own topics
* moved arch and tech specs back to main folder
* update migrate-existing-tasks content
* merged manual deploy content; added serv conf ref
* fixed links
* added procedure for upgrading to dataplanes
* fixed linked reported by checker
* added updates to dataplanes overview page
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>
* updated links and added redirects
* removed old architecture content
---------
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>
Derek Menteer
trujillo-adam
Iryna Shustava
Chris S. Kim
R.B. Boyer
John Murret
Semir Patel
John Maguire
Dhia Ayachi
Tu Nguyen
Jeff Boruszak
Chris Thain
Ashesh Vidyut
Matt Keeler
Eric Haberkorn
David Yu
Thomas Eckert
cskh
Valeriia Ruban