* Upgrade hcp-sdk-go to latest version v0.73
Changes:
- go get github.com/hashicorp/hcp-sdk-go
- go mod tidy
* From upgrade: regenerate protobufs for upgrade from 1.30 to 1.31
Ran: `make proto`
Slack: https://hashicorp.slack.com/archives/C0253EQ5B40/p1701105418579429
* From upgrade: fix mock interface implementation
After upgrading, there is the following compile error:
cannot use &mockHCPCfg{} (value of type *mockHCPCfg) as "github.com/hashicorp/hcp-sdk-go/config".HCPConfig value in return statement: *mockHCPCfg does not implement "github.com/hashicorp/hcp-sdk-go/config".HCPConfig (missing method Logout)
Solution: update the mock to have the missing Logout method
* From upgrade: Lint: remove usage of deprecated req.ServerState.TLS
Due to upgrade, linting is erroring due to usage of a newly deprecated field
22:47:56 [consul]: make lint
--> Running golangci-lint (.)
agent/hcp/testing.go:157:24: SA1019: req.ServerState.TLS is deprecated: use server_tls.internal_rpc instead. (staticcheck)
time.Until(time.Time(req.ServerState.TLS.CertExpiry)).Hours()/24,
^
* From upgrade: adjust oidc error message
From the upgrade, this test started failing:
=== FAIL: internal/go-sso/oidcauth TestOIDC_ClaimsFromAuthCode/failed_code_exchange (re-run 2) (0.01s)
oidc_test.go:393: unexpected error: Provider login failed: Error exchanging oidc code: oauth2: "invalid_grant" "unexpected auth code"
Prior to the upgrade, the error returned was:
```
Provider login failed: Error exchanging oidc code: oauth2: cannot fetch token: 401 Unauthorized\nResponse: {\"error\":\"invalid_grant\",\"error_description\":\"unexpected auth code\"}\n
```
Now the error returned is as below and does not contain "cannot fetch token"
```
Provider login failed: Error exchanging oidc code: oauth2: "invalid_grant" "unexpected auth code"
```
* Update AgentPushServerState structs with new fields
HCP-side changes for the new fields are in:
https://github.com/hashicorp/cloud-global-network-manager-service/pull/1195/files
* Minor refactor for hcpServerStatus to abstract tlsInfo into struct
This will make it easier to set the same tls-info information to both
- status.TLS (deprecated field)
- status.ServerTLSMetadata (new field to use instead)
* Update hcpServerStatus to parse out information for new fields
Changes:
- Improve error message and handling (encountered some issues and was confused)
- Set new field TLSInfo.CertIssuer
- Collect certificate authority metadata and set on TLSInfo.CertificateAuthorities
- Set TLSInfo on both server.TLS and server.ServerTLSMetadata.InternalRPC
* Update serverStatusToHCP to convert new fields to GNM rpc
* Add changelog
* Feedback: connect.ParseCert, caCerts
* Feedback: refactor and unit test server status
* Feedback: test to use expected struct
* Feedback: certificate with intermediate
* Feedback: catch no leaf, remove expectedErr
* Feedback: update todos with jira ticket
* Feedback: mock tlsConfigurator
* add build tags/import k8s specific proto packages
* fix generated import paths
* fix gomod linting issue
* mod tidy every go mod file
* revert protobuff version, take care of in different pr
* cleaned up new lines
* added newline to end of file
* Generate resource_types for MeshGateway by specifying spec option
* Register MeshGateway type w/ TODOs for hooks
* Initialize controller for MeshGateway resources
* Add meshgateway to list of v2 resource dependencies for golden test
* Scope MeshGateway resource to partition
* xds: Ensure v2 route match is populated for gRPC
Similar to HTTP, ensure that route match config (which is required by
Envoy) is populated when default values are used.
Because the default matches generated for gRPC contain a single empty
`GRPCRouteMatch`, and that proto does not directly support prefix-based
config, an interpretation of the empty struct is needed to generate the
same output that the `HTTPRouteMatch` is explicitly configured to
provide in internal/mesh/internal/controllers/routes/generate.go.
* xds: Ensure protocol set for gRPC resources
Add explicit protocol in `ProxyStateTemplate` builders and validate it
is always set on clusters. This ensures that HTTP filters and
`http2_protocol_options` are populated in all the necessary places for
gRPC traffic and prevents future unintended omissions of non-TCP
protocols.
Co-authored-by: John Murret <john.murret@hashicorp.com>
---------
Co-authored-by: John Murret <john.murret@hashicorp.com>
* xdsv2: support l7 by adding xfcc policy/headers, tweaking routes, and make a bunch of listeners l7 tests pass
* sidecarproxycontroller: add l7 local app support
* trafficpermissions: make l4 traffic permissions work on l7 workloads
* rename route name field for consistency with l4 cluster name field
* resolve conflicts and rebase
* fix: ensure route name is used in l7 destination route name as well. previously it was only in the route names themselves, now the route name and l7 destination route name line up
This change builds on #19043 and #19067 and updates the sidecar controller to use those computed resources. This achieves several benefits:
* The cache is now simplified which helps us solve for previous bugs (such as multiple Upstreams/Destinations targeting the same service would overwrite each other)
* We no longer need proxy config cache
* We no longer need to do merging of proxy configs as part of the controller logic
* Controller watches are simplified because we no longer need to have complex mapping using cache and can instead use the simple ReplaceType mapper.
It also makes several other improvements/refactors:
* Unifies all caches into one. This is because originally the caches were more independent, however, now that they need to interact with each other it made sense to unify them where sidecar proxy controller uses one cache with 3 bimappers
* Unifies cache and mappers. Mapper already needed all caches anyway and so it made sense to make the cache do the mapping also now that the cache is unified.
* Gets rid of service endpoints watches. This was needed to get updates in a case when service's identities have changed and we need to update proxy state template's spiffe IDs for those destinations. This will however generate a lot of reconcile requests for this controller as service endpoints objects can change a lot because they contain workload's health status. This is solved by adding a status to the service object tracking "bound identities" and have service endpoints controller update it. Having service's status updated allows us to get updates in the sidecar proxy controller because it's already watching service objects
* Add a watch for workloads. We need it so that we get updates if workload's ports change. This also ensures that we update cached identities in case workload's identity changes.
This commit adds a new type ComputedDestinations that will contain all destinations from any Destinations resources and will be name-aligned with a workload. This also adds an explicit-destinations controller that computes these resources.
This is needed to simplify the tracking we need to do currently in the sidecar-proxy controller and makes it easier to query all explicit destinations that apply to a workload.
* Introduce a new type `ComputedProxyConfiguration` and add a controller for it. This is needed for two reasons. The first one is that external integrations like kubernetes may need to read the fully computed and sorted proxy configuration per workload. The second reasons is that it makes sidecar-proxy controller logic quite a bit simpler as it no longer needs to do this.
* Generalize workload selection mapper and fix a bug where it would delete IDs from the tree if only one is left after a removal is done.
Convert more of the xRoutes features that were skipped in an earlier PR into ComputedRoutes and make them work:
- DestinationPolicy defaults
- more timeouts
- load balancer policy
- request/response header mutations
- urlrewrite
- GRPCRoute matches
xRoute resource types contain a slice of parentRefs to services that they
manipulate traffic for. All xRoutes that have a parentRef to given Service
will be merged together to generate a ComputedRoutes resource
name-aligned with that Service.
This means that a write of an xRoute with 2 parent ref pointers will cause
at most 2 reconciles for ComputedRoutes.
If that xRoute's list of parentRefs were ever to be reduced, or otherwise
lose an item, that subsequent map event will only emit events for the current
set of refs. The removed ref will not cause the generated ComputedRoutes
related to that service to be re-reconciled to omit the influence of that xRoute.
To combat this, we will store on the ComputedRoutes resource a
BoundResources []*pbresource.Reference field with references to all
resources that were used to influence the generated output.
When the routes controller reconciles, it will use a bimapper to index this
influence, and the dependency mappers for the xRoutes will look
themselves up in that index to discover additional (former) ComputedRoutes
that need to be notified as well.
Ensure that configuring a FailoverPolicy for a service that is reachable via a xRoute or a direct upstream causes an envoy aggregate cluster to be created for the original cluster name, but with separate clusters for each one of the possible destinations.
Reworks the sidecar controller to accept ComputedRoutes as an input and use it to generate appropriate ProxyStateTemplate resources containing L4/L7 mesh configuration.
This new controller produces an intermediate output (ComputedRoutes) that is meant to summarize all relevant xRoutes and related mesh configuration in an easier-to-use format for downstream use to construct the ProxyStateTemplate.
It also applies status updates to the xRoute resource types to indicate that they are themselves semantically valid inputs.
* mesh-controller: handle L4 protocols for a proxy without upstreams
* sidecar-controller: Support explicit destinations for L4 protocols and single ports.
* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.
* endpoints-controller: add workload identity to the service endpoints resource
* small fixes
* review comments
* Address PR comments
* sidecar-proxy controller: Add support for transparent proxy
This currently does not support inferring destinations from intentions.
* PR review comments
* mesh-controller: handle L4 protocols for a proxy without upstreams
* sidecar-controller: Support explicit destinations for L4 protocols and single ports.
* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.
* endpoints-controller: add workload identity to the service endpoints resource
* small fixes
* review comments
* Make sure endpoint refs route to mesh port instead of an app port
* Address PR comments
* fixing copyright
* tidy imports
* sidecar-proxy controller: Add support for transparent proxy
This currently does not support inferring destinations from intentions.
* tidy imports
* add copyright headers
* Prefix sidecar proxy test files with source and destination.
* Update controller_test.go
* NET-5132 - Configure multiport routing for connect proxies in TProxy mode
* formatting golden files
* reverting golden files and adding changes in manually. build implicit destinations still has some issues.
* fixing files that were incorrectly repeating the outbound listener
* PR comments
* extract AlpnProtocol naming convention to getAlpnProtocolFromPortName(portName)
* removing address level filtering.
* adding license to resources_test.go
---------
Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
This commit adds support for transparent proxy to the sidecar proxy controller. As we do not yet support inferring destinations from intentions, this assumes that all services in the cluster are destinations.
* Adding explicit MPL license for sub-package
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Adding explicit MPL license for sub-package
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Updating the license from MPL to Business Source License
Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.
* add missing license headers
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Also, change the ProxyState.id to identity. This is because we already have the id of this proxy
from the resource, and this id should be name-aligned with the workload it represents. It should
also have the owner ref set to the workload ID if we need that. And so the id field seems unnecessary.
We do, however, need a reference to workload identity so that we can authorize the proxy when it initially
connects to the xDS server.
Configuration that previously was inlined into the Upstreams resource
applies to both explicit and implicit upstreams and so it makes sense to
split it out into its own resource.
It also has other minor changes:
- Renames `proxy.proto` proxy_configuration.proto`
- Changes the type of `Upstream.destination_ref` from `pbresource.ID` to
`pbresource.Reference`
- Adds comments to fields that didn't have them
lornasong
John Maguire
sarahalsmiller
Melisa Griffin
Thomas Eckert
Nathan Coleman
Michael Zalimeni
John Murret
Iryna Shustava
Ashwin Venkatesh
Nitya Dhanushkodi
Eric Haberkorn
R.B. Boyer
Matt Keeler
Dan Stough
Semir Patel
hashicorp-copywrite[bot]
Derek Menteer