Commit Graph

48 Commits

Author SHA1 Message Date
Michael Zalimeni d4761c0ccd
security: upgrade google.golang.org/protobuf to 1.33.0 (#20801)
Resolves CVE-2024-24786.
2024-03-06 23:04:42 +00:00
John Maguire 02e735f40f
[NET-7953] Added computed gateway routes protos and generated code (#20595)
Added computed gateway routes protos and generated code
2024-02-14 11:36:55 -05:00
R.B. Boyer 671c436415
mesh: use ComputedImplicitDestinations resource in the sidecar controller (#20553)
Wire the ComputedImplicitDestinations resource into the sidecar controller, replacing the inline version already present.

Also:

- Rewrite the controller to use the controller cache
- Rewrite it to no longer depend on ServiceEndpoints
- Remove the fetcher and (local) cache abstraction
2024-02-12 14:10:33 -06:00
R.B. Boyer 6742340878
mesh: add ComputedImplicitDestinations resource for future use (#20547)
Creates a new controller to create ComputedImplicitDestinations resources by 
composing ComputedRoutes, Services, and ComputedTrafficPermissions to 
infer all ParentRef services that could possibly send some portion of traffic to a 
Service that has at least one accessible Workload Identity. A followup PR will 
rewire the sidecar controller to make use of this new resource.

As this is a performance optimization, rather than a security feature the following 
aspects of traffic permissions have been ignored:

- DENY rules
- port rules (all ports are allowed)

Also:

- Add some v2 TestController machinery to help test complex dependency mappers.
2024-02-09 15:42:10 -06:00
Nathan Coleman 2456fe5148
[NET-7657] Remove proto definitions for GatewayClass + GatewayClassConfig (#20523) 2024-02-07 16:53:39 -05:00
skpratt 57bad0df85
add traffic permissions excludes and tests (#20453)
* add traffic permissions tests

* review fixes

* Update internal/mesh/internal/controllers/sidecarproxy/builder/local_app.go

Co-authored-by: John Landa <jonathanlanda@gmail.com>

---------

Co-authored-by: John Landa <jonathanlanda@gmail.com>
2024-02-07 20:21:44 +00:00
Nathan Coleman 21b3c18d5d
Use a full EndpointRef on ComputedRoutes targets instead of just the ID (#20400)
* Use a full EndpointRef on ComputedRoutes targets instead of just the ID

Today, the `ComputedRoutes` targets have the appropriate ID set for their `ServiceEndpoints` reference; however, the `MeshPort` and `RoutePort` are assumed to be that of the target when adding the endpoints reference in the sidecar's `ProxyStateTemplate`.

This is problematic when the target lives behind a `MeshGateway` and the `Mesh/RoutePort` used in the sidecar's `ProxyStateTemplate` should be that of the `MeshGateway` instead of the target.

Instead of assuming the `MeshPort` and `RoutePort` when building the `ProxyStateTemplate` for the sidecar, let's just add the full `EndpointRef` -- including the ID and the ports -- when hydrating the computed destinations.

* Make sure the UID from the existing ServiceEndpoints makes it onto ComputedRoutes

* Update test assertions

* Undo confusing whitespace change

* Remove one-line function wrapper

* Use plural name for endpoints ref

* Add constants for gateway name, kind and port names
2024-01-30 16:25:44 -05:00
Matt Keeler 34a32d4ce5
Remove V2 PeerName field from pbresource.Tenancy (#19865)
The peer name will eventually show up elsewhere in the resource. For now though this rips it out of where we don’t want it to be.
2024-01-29 15:08:31 -05:00
Nitya Dhanushkodi 92aab7ea31
[NET-5586][rebased] v2: Support virtual port references in config (#20371)
[OG Author: michael.zalimeni@hashicorp.com, rebase needed a separate PR]

* v2: support virtual port in Service port references

In addition to Service target port references, allow users to specify a
port by stringified virtual port value. This is useful in environments
such as Kubernetes where typical configuration is written in terms of
Service virtual ports rather than workload (pod) target port names.

Retaining the option of referencing target ports by name supports VMs,
Nomad, and other use cases where virtual ports are not used by default.

To support both uses cases at once, we will strictly interpret port
references based on whether the value is numeric. See updated
`ServicePort` docs for more details.

* v2: update service ref docs for virtual port support

Update proto and generated .go files with docs reflecting virtual port
reference support.

* v2: add virtual port references to L7 topo test

Add coverage for mixed virtual and target port references to existing
test.

* update failover policy controller tests to work with computed failover policy and assert error conditions against FailoverPolicy and ComputedFailoverPolicy resources

* accumulate services; don't overwrite them in enterprise

---------

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
2024-01-29 10:43:41 -08:00
Nitya Dhanushkodi 0ec7bddb9a
[Net-5594][Net-7466] v2: Only route to endpoints that implement the port being routed to, and make xdscontroller and xdsv2 golden tests use tenancy (#20356)
* If a workload does not implement a port, it should not be included in the list of endpoints for the Envoy cluster for that port.

* Adds tenancy tests for xds controller and xdsv2 resource generation, and adds all those files.

* The original change in this PR was for filtering the list of endpoints by the port being routed to (bullet 1). Since I made changes to sidecarproxycontroller golden files, I realized some of the golden files were unused because of the tenancy changes, so when I deleted those, that broke xds controller tests which weren't correctly using tenancy. So when I fixed that, then the xdsv2 tests broke, so I added tenancy support there too. So now, from sidecarproxy controller -> xds controller -> xdsv2 we now have tenancy support and all the golden files are lined up.
2024-01-26 10:07:21 -08:00
sarahalsmiller 1eca44aef9
NET-7153 Proto messages for API Gateway (#20250)
* API Gateway proto

* fix lint issue

* new line

* run make proto format

* regened with comment

* lint

* utilizie existing TLS struct

* Update proto-public/pbmesh/v2beta1/api_gateway.proto

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* generated file

* Update proto-public/pbmesh/v2beta1/api_gateway.proto

* regen with comment

* format the comment

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2024-01-24 12:10:20 -06:00
skpratt 0abf8f8426
Net 5092/internal l7 traffic permissions (#20276)
* wire up L7 Traffic Permissions

* testing

* update comment
2024-01-23 20:07:58 -06:00
Anita Akaeze 6a85543175
NET-7178: add workload selector to MeshGateway protobuf definition (#20159)
NET-7178: Add workload selector to MeshGateway proto
2024-01-16 21:48:18 +00:00
sarahalsmiller 1e351e286e
NET-7189 Add Protocol to MeshGatewayListeners (#20171)
add protocol to listeners struct
2024-01-11 13:12:32 -06:00
sarahalsmiller 7d92a5dfd6
Add min/max kubetags to meshgatewaylistener (#20149)
* add kubetags

* generate proto
2024-01-10 19:12:29 +00:00
Semir Patel 25b37d75b0
license: MPL'ify proto-public (#20143) 2024-01-10 11:05:12 -06:00
sarahalsmiller 96790a814c
NET-6822 Add kubebuilder tag to struct (#20120)
add kubebuilder tag
2024-01-09 20:13:32 +00:00
sarahalsmiller a820e9af26
Add listeners to meshgateway spec (#20082)
add listeners to meshgateway
2024-01-04 18:42:32 +00:00
John Murret 7a410d7c5b
NET-6945 - Replace usage of deprecated Envoy field envoy.config.core.v3.HeaderValueOption.append (#20078)
* NET-6945 - Replace usage of deprecated Envoy field envoy.config.core.v3.HeaderValueOption.append

* update proto for v2 and then update xds v2 logic

* add changelog

* Update 20078.txt to be consistent with existing changelog entries

* swap enum values tomatch envoy.
2024-01-04 00:36:25 +00:00
Matt Keeler 123bc95e1a
Add Common Controller Caching Infrastructure (#19767)
* Add Common Controller Caching Infrastructure
2023-12-13 10:06:39 -05:00
lornasong edf4610ed9
[Cloud][CC-6925] Updates to pushing server state (#19682)
* Upgrade hcp-sdk-go to latest version v0.73

Changes:
- go get github.com/hashicorp/hcp-sdk-go
- go mod tidy

* From upgrade: regenerate protobufs for upgrade from 1.30 to 1.31

Ran: `make proto`

Slack: https://hashicorp.slack.com/archives/C0253EQ5B40/p1701105418579429

* From upgrade: fix mock interface implementation

After upgrading, there is the following compile error:

cannot use &mockHCPCfg{} (value of type *mockHCPCfg) as "github.com/hashicorp/hcp-sdk-go/config".HCPConfig value in return statement: *mockHCPCfg does not implement "github.com/hashicorp/hcp-sdk-go/config".HCPConfig (missing method Logout)

Solution: update the mock to have the missing Logout method

* From upgrade: Lint: remove usage of deprecated req.ServerState.TLS

Due to upgrade, linting is erroring due to usage of a newly deprecated field

22:47:56 [consul]: make lint
--> Running golangci-lint (.)
agent/hcp/testing.go:157:24: SA1019: req.ServerState.TLS is deprecated: use server_tls.internal_rpc instead. (staticcheck)
                time.Until(time.Time(req.ServerState.TLS.CertExpiry)).Hours()/24,
                                     ^

* From upgrade: adjust oidc error message

From the upgrade, this test started failing:

=== FAIL: internal/go-sso/oidcauth TestOIDC_ClaimsFromAuthCode/failed_code_exchange (re-run 2) (0.01s)
    oidc_test.go:393: unexpected error: Provider login failed: Error exchanging oidc code: oauth2: "invalid_grant" "unexpected auth code"

Prior to the upgrade, the error returned was:
```
Provider login failed: Error exchanging oidc code: oauth2: cannot fetch token: 401 Unauthorized\nResponse: {\"error\":\"invalid_grant\",\"error_description\":\"unexpected auth code\"}\n
```

Now the error returned is as below and does not contain "cannot fetch token"
```
Provider login failed: Error exchanging oidc code: oauth2: "invalid_grant" "unexpected auth code"

```

* Update AgentPushServerState structs with new fields

HCP-side changes for the new fields are in:
https://github.com/hashicorp/cloud-global-network-manager-service/pull/1195/files

* Minor refactor for hcpServerStatus to abstract tlsInfo into struct

This will make it easier to set the same tls-info information to both
 - status.TLS (deprecated field)
 - status.ServerTLSMetadata (new field to use instead)

* Update hcpServerStatus to parse out information for new fields

Changes:
 - Improve error message and handling (encountered some issues and was confused)
 - Set new field TLSInfo.CertIssuer
 - Collect certificate authority metadata and set on TLSInfo.CertificateAuthorities
 - Set TLSInfo on both server.TLS and server.ServerTLSMetadata.InternalRPC

* Update serverStatusToHCP to convert new fields to GNM rpc

* Add changelog

* Feedback: connect.ParseCert, caCerts

* Feedback: refactor and unit test server status

* Feedback: test to use expected struct

* Feedback: certificate with intermediate

* Feedback: catch no leaf, remove expectedErr

* Feedback: update todos with jira ticket

* Feedback: mock tlsConfigurator
2023-12-04 10:25:18 -05:00
John Maguire 69b1d2072b
[V2] Move resource field on gateway class config from repeated map to single map (#19773)
Move resource field on gateway class config from repeated map to single
map
2023-11-29 18:12:42 +00:00
sarahalsmiller fd1d97c334
Add Kubebuilder tags to Gatewayclassconfig proto messages (#19725)
* add build tags/import k8s specific proto packages

* fix generated import paths

* fix gomod linting issue

* mod tidy every go mod file

* revert protobuff version, take care of in different pr

* cleaned up new lines

* added newline to end of file
2023-11-28 14:46:11 -06:00
Melisa Griffin 3f0a75228f
Adds GatewayClassName field to MeshGateway Proto (#19738) 2023-11-27 14:49:54 -05:00
Thomas Eckert d9432f9032
Add stub for MeshConfiguration proto (#19642)
* Add mesh_configuration.proto

* Run make proto

* Add cluster scope to MeshConfiguration

* Run make proto
2023-11-16 13:03:14 -05:00
sarahalsmiller 0c675433e0
Added Gatewayclassconfig resource type to proto package (#19664)
resource type + regen
2023-11-16 17:18:44 +00:00
sarahalsmiller 29042b2938
NET-6550 generate stubs for GatewayClassConfig (#19602)
* NET-6550 generate stubs

* gatewayclassconfig

* fix minor spacing issue

* fix minor spacing issue

* convert to snake case

* add ports

* snakecase
2023-11-15 17:07:01 +00:00
Melisa Griffin 7628fed0a5
Updates GatewayClass protobuf to set optional fields to optional (#19648) 2023-11-15 15:44:53 +00:00
Melisa Griffin 9ca62aaf33
Adds proto for the GatewayClass based on the GAMMA Kubernetes Sig (#19615) 2023-11-14 16:18:20 -05:00
Nathan Coleman 40c57f10a0
NET-6391 Initialize controller for MeshGateway resource (#19552)
* Generate resource_types for MeshGateway by specifying spec option

* Register MeshGateway type w/ TODOs for hooks

* Initialize controller for MeshGateway resources

* Add meshgateway to list of v2 resource dependencies for golden test

* Scope MeshGateway resource to partition
2023-11-09 16:33:14 -05:00
Nathan Coleman 028f1d8c50
NET-6390 Initialize MeshGateway proto (#19548) 2023-11-07 12:24:54 -05:00
Michael Zalimeni a7803bd829
[NET-6305] xds: Ensure v2 route match and protocol are populated for gRPC (#19343)
* xds: Ensure v2 route match is populated for gRPC

Similar to HTTP, ensure that route match config (which is required by
Envoy) is populated when default values are used.

Because the default matches generated for gRPC contain a single empty
`GRPCRouteMatch`, and that proto does not directly support prefix-based
config, an interpretation of the empty struct is needed to generate the
same output that the `HTTPRouteMatch` is explicitly configured to
provide in internal/mesh/internal/controllers/routes/generate.go.

* xds: Ensure protocol set for gRPC resources

Add explicit protocol in `ProxyStateTemplate` builders and validate it
is always set on clusters. This ensures that HTTP filters and
`http2_protocol_options` are populated in all the necessary places for
gRPC traffic and prevents future unintended omissions of non-TCP
protocols.

Co-authored-by: John Murret <john.murret@hashicorp.com>

---------

Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-10-25 17:43:58 +00:00
John Murret a7fbd00865
NET-5073 - ProxyConfiguration: implement various connection options (#19187)
* NET-5073 - ProxyConfiguration: implement various connection options

* PR feedback - LocalConnection and InboundConnection do not affect exposed routes. configure L7 route destinations. fix connection proto sequence numbers.

* add timeout to L7 Route Destinations
2023-10-14 13:54:08 +00:00
Iryna Shustava e94d6ceca6
mesh: add validation hook to proxy configuration (#19186) 2023-10-13 15:21:39 -06:00
Ashwin Venkatesh 3d1a606c3b
Clone proto into deepcopy correctly (#19204) 2023-10-13 16:41:22 -04:00
Ashwin Venkatesh c2a0d4f9ca
Create DeepCopy() and Json Marshal/Unmarshal for proto-public (#19015)
* Override Marshal/UnmarshalJSON for proto-public types
* Generate Deepcopy() for proto-public types for Kubernetes CRDs.
2023-10-13 14:55:58 +00:00
Nitya Dhanushkodi 95d9b2c7e4
[NET-4931] xdsv2, sidecarproxycontroller, l4 trafficpermissions: support L7 (#19185)
* xdsv2: support l7 by adding xfcc policy/headers, tweaking routes, and make a bunch of listeners l7 tests pass

* sidecarproxycontroller: add l7 local app support 

* trafficpermissions: make l4 traffic permissions work on l7 workloads

* rename route name field for consistency with l4 cluster name field

* resolve conflicts and rebase

* fix: ensure route name is used in l7 destination route name as well. previously it was only in the route names themselves, now the route name and l7 destination route name line up
2023-10-12 23:45:45 +00:00
Iryna Shustava 54a12ab3c9
mesh: sidecar proxy controller improvements (#19083)
This change builds on #19043 and #19067 and updates the sidecar controller to use those computed resources. This achieves several benefits:

   * The cache is now simplified which helps us solve for previous bugs (such as multiple Upstreams/Destinations targeting the same service would overwrite each other)
   * We no longer need proxy config cache
   * We no longer need to do merging of proxy configs as part of the controller logic
   * Controller watches are simplified because we no longer need to have complex mapping using cache and can instead use the simple ReplaceType mapper.

It also makes several other improvements/refactors:

  * Unifies all caches into one. This is because originally the caches were more independent, however, now that they need to interact with each other it made sense to unify them where sidecar proxy controller uses one cache with 3 bimappers
   * Unifies cache and mappers. Mapper already needed all caches anyway and so it made sense to make the cache do the mapping also now that the cache is unified.
   * Gets rid of service endpoints watches. This was needed to get updates in a case when service's identities have changed and we need to update proxy state template's spiffe IDs for those destinations. This will however generate a lot of reconcile requests for this controller as service endpoints objects can change a lot because they contain workload's health status. This is solved by adding a status to the service object tracking "bound identities" and have service endpoints controller update it. Having service's status updated allows us to get updates in the sidecar proxy controller because it's already watching service objects
   * Add a watch for workloads. We need it so that we get updates if workload's ports change. This also ensures that we update cached identities in case workload's identity changes.
2023-10-12 13:20:13 -06:00
Iryna Shustava ad06c96456
mesh: add computed destinations with a controller that computes them (#19067)
This commit adds a new type ComputedDestinations that will contain all destinations from any Destinations resources and will be name-aligned with a workload. This also adds an explicit-destinations controller that computes these resources.

This is needed to simplify the tracking we need to do currently in the sidecar-proxy controller and makes it easier to query all explicit destinations that apply to a workload.
2023-10-12 12:04:12 -06:00
Iryna Shustava 60b75a55f7
mesh: implement exposed paths (#19044)
Implement exposed paths listeners in the sidecar proxy controller.
2023-10-11 19:23:16 -06:00
Iryna Shustava c35df12c95
mesh: Add ComputedProxyConfiguration and a controller that computes it. (#19043)
* Introduce a new type `ComputedProxyConfiguration` and add a controller for it. This is needed for two reasons. The first one is that external integrations like kubernetes may need to read the fully computed and sorted proxy configuration per workload. The second reasons is that it makes sidecar-proxy controller logic quite a bit simpler as it no longer needs to do this.
* Generalize workload selection mapper and fix a bug where it would delete IDs from the tree if only one is left after a removal is done.
2023-10-10 17:34:53 -06:00
Eric Haberkorn f2b7b4591a
Fix Traffic Permissions Default Deny (#19028)
Whenver a traffic permission exists for a given workload identity, turn on default deny.

Previously, this was only working at the port level.
2023-10-04 09:58:28 -04:00
Iryna Shustava 3ea6afb4d4
mesh: rename Upstreams and UpstreamsConfiguration to Destinations* (#18995) 2023-09-25 12:03:45 -06:00
R.B. Boyer ca7533850c
mesh: update various protobuf comments for mesh types (#18993) 2023-09-22 18:41:49 -05:00
Iryna Shustava e6b724d062
catalog,mesh,auth: Move resource types to the proto-public module (#18935) 2023-09-22 15:50:56 -06:00
R.B. Boyer 9e48607893
mesh: compute more of the xRoute features into ComputedRoutes (#18980)
Convert more of the xRoutes features that were skipped in an earlier PR into ComputedRoutes and make them work:

- DestinationPolicy defaults
- more timeouts
- load balancer policy
- request/response header mutations
- urlrewrite
- GRPCRoute matches
2023-09-22 16:13:24 -05:00
R.B. Boyer 11d6b0df45
mesh: store bound reference pointers on a ComputedRoutes resource and use during reconcile (#18965)
xRoute resource types contain a slice of parentRefs to services that they 
manipulate traffic for. All xRoutes that have a parentRef to given Service 
will be merged together to generate a ComputedRoutes resource 
name-aligned with that Service.

This means that a write of an xRoute with 2 parent ref pointers will cause 
at most 2 reconciles for ComputedRoutes.

If that xRoute's list of parentRefs were ever to be reduced, or otherwise
 lose an item, that subsequent map event will only emit events for the current 
set of refs. The removed ref will not cause the generated ComputedRoutes 
related to that service to be re-reconciled to omit the influence of that xRoute.

To combat this, we will store on the ComputedRoutes resource a 
BoundResources []*pbresource.Reference field with references to all 
resources that were used to influence the generated output.

When the routes controller reconciles, it will use a bimapper to index this
 influence, and the dependency mappers for the xRoutes will look 
themselves up in that index to discover additional (former) ComputedRoutes
 that need to be notified as well.
2023-09-22 15:46:14 -05:00
Iryna Shustava d88888ee8b
catalog,mesh,auth: Bump versions to v2beta1 (#18930) 2023-09-22 10:51:15 -06:00