* [NET-6438] Add tenancy to xDS Tests
* [NET-6438] Add tenancy to xDS Tests
- Fixing imports
* [NET-6438] Add tenancy to xDS Tests
- Added cleanup post test run
* [NET-6356] Add tenancy to xDS Tests
- Added cleanup post test run
* [NET-6438] Add tenancy to xDS Tests
- using t.Cleanup instead of defer delete
* [NET-6438] Add tenancy to xDS Tests
- rebased
* [NET-6438] Add tenancy to xDS Tests
- rebased
* Generate resource_types for MeshGateway by specifying spec option
* Register MeshGateway type w/ TODOs for hooks
* Initialize controller for MeshGateway resources
* Add meshgateway to list of v2 resource dependencies for golden test
* Scope MeshGateway resource to partition
* [NET-6356] Add tenancy to Failover Tests
* [NET-6438] Add tenancy to xDS Tests
- Added cleanup post test run
* [NET-6356] Add tenancy to failover Tests
- using t.Cleanup instead of defer delete
* node health controller tenancy
* some prog
* some fixes
* revert
* pr comment resolved
* removed name
* Add namespace and tenancy in sidecar proxy controller test
* revert node health controller
* clean up data
* fix local
* copy from ENT
* removed dup code
* removed tenancy
* add test tenancies
As the V2 architecture hinges on eventual consistency and controllers reconciling the existing state in response to writes, there are potential issues we could run into regarding ordering and timing of operations. We want to be able to guarantee that given a set of resources the system will always eventually get to the desired correct state. The order of resource writes and delays in performing those writes should not alter the final outcome of reaching the desired state.
To that end, this commit introduces arbitrary randomized delays before performing resources writes into the `resourcetest.Client`. Its `PublishResources` method was already randomizing the order of resource writes. By default, no delay is added to normal writes and deletes but tests can opt-in via either passing hard coded options when creating the `resourcetest.Client` or using the `resourcetest.ConfigureTestCLIFlags` function to allow processing of CLI parameters.
In addition to allowing configurability of the request delay min and max, the client also has a configurable random number generator seed. When Using the CLI parameter helpers, a test log will be written noting the currently used settings. If the test fails then you can reproduce the same delays and order randomizations by providing the seed during the previous test failure.
* fix: update watch endpoint to default based on scope
* test: additional test
* refactor: rename list validate function
* refactor: rename validate<Op>Request() -> ensure<Op>RequestValid() for consistency
* cover all protocols in local_app golden tests
* fix xds tests
* updating latest
* fix broken test
* add sorting of routers to TestBuildLocalApp to get rid of the flaking
Add some generic type hook wrappers to first decode the data
There seems to be a pattern for Validation, Mutation and Write Authorization hooks where they first need to decode the Any data before doing the domain specific work.
This PR introduces 3 new functions to generate wrappers around the other hooks to pre-decode the data into a DecodedResource and pass that in instead of the original pbresource.Resource.
This PR also updates the various catalog data types to use the new hook generators.
* init
* computed exported service
* make proto
* exported services resource
* exported services test
* added some tests and namespace exported service
* partition exported services
* computed service
* computed services tests
* register types
* fix comment
* make proto lint
* fix proto format make proto
* make codegen
* Update proto-public/pbmulticluster/v1alpha1/computed_exported_services.proto
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* Update internal/multicluster/internal/types/computed_exported_services.go
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* using different way of resource creation in tests
* make proto
* fix computed exported services test
* fix tests
* differnet validation for computed services for ent and ce
* Acls for exported services
* added validations for enterprise features in ce
* fix error
* fix acls test
* Update internal/multicluster/internal/types/validation_exported_services_ee.go
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* removed the create method
* update proto
* removed namespace
* created seperate function for ce and ent
* test files updated and validations fixed
* added nil checks
* fix tests
* added comments
* removed tenancy check
* added mutation function
* fix mutation method
* fix list permissions in test
* fix pr comments
* fix tests
* lisence
* busl license
* Update internal/multicluster/internal/types/helpers_ce.go
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* Update internal/multicluster/internal/types/helpers_ce.go
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* Update internal/multicluster/internal/types/helpers_ce.go
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* make proto
* some pr comments addressed
* some pr comments addressed
* acls helper
* some comment changes
* removed unused files
* fixes
* fix function in file
* caps
* some positioing
* added test for validation error
* fix names
* made valid a function
* remvoed patch
* removed mutations
* v2 beta1
* v2beta1
* rmeoved v1alpha1
* validate error
* merge ent
* some nits
* removed dup func
* removed nil check
---------
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* test: add missing tests for read resource namespace
* refactor: remove redundant test
* refactor: rename import aliases
* fix: typo var name
* refctor: remove another redundant test
* xds: Ensure v2 route match is populated for gRPC
Similar to HTTP, ensure that route match config (which is required by
Envoy) is populated when default values are used.
Because the default matches generated for gRPC contain a single empty
`GRPCRouteMatch`, and that proto does not directly support prefix-based
config, an interpretation of the empty struct is needed to generate the
same output that the `HTTPRouteMatch` is explicitly configured to
provide in internal/mesh/internal/controllers/routes/generate.go.
* xds: Ensure protocol set for gRPC resources
Add explicit protocol in `ProxyStateTemplate` builders and validate it
is always set on clusters. This ensures that HTTP filters and
`http2_protocol_options` are populated in all the necessary places for
gRPC traffic and prevents future unintended omissions of non-TCP
protocols.
Co-authored-by: John Murret <john.murret@hashicorp.com>
---------
Co-authored-by: John Murret <john.murret@hashicorp.com>
* NET-5397 - wire up golden tests from sidecar-proxy controller for xds controller and xdsv2
* WIP
* WIP
* everything matching except leafCerts. need to mock those
* single port destinations working except mixed destinations
* golden test input to xds controller tests for destinations
* proposed fix for failover group naming errors
* clean up test to use helper.
* clean up test to use helper.
* fix test file
* add docstring for test function.
* add docstring for test function.
* fix linting error
* fixing test after route fix merged into main
* first source test works
* WIP
* modify all source files
* source tests pass
* fixing tests after bug fix in main
* NET-5397 - wire up golden tests from sidecar-proxy controller for xds controller and xdsv2
* WIP
* WIP
* everything matching except leafCerts. need to mock those
* single port destinations working except mixed destinations
* golden test input to xds controller tests for destinations
* proposed fix for failover group naming errors
* clean up test to use helper.
* clean up test to use helper.
* fix test file
* add docstring for test function.
* add docstring for test function.
* fix linting error
* fixing test after route fix merged into main
To properly enforce writes on resources that have workload selectors with prefixes, we need another service authorization rule that allows us to check whether read is allowed within a given prefix. Specifically we need to only allow writes if the policy prefix allows for a wider set of names than the prefix selector on the resource. We should also not allow policies with exact names for prefix matches.
Part of [NET-3993]
When testing adding http probes to apps, I ran into some issues which I fixed here:
- The listener should be listening on the exposed listener port, updated that.
- The listener and route names were pointing to the path of the exposed path. In my test, the path was "/" resulting in an empty string path. Also, the path may not be unique across exposed path listeners, so I decided to use the path+exposed port as the unique identifier.
This change adds ACL hooks to the remaining catalog and mesh resources, excluding any computed ones. Those will for now continue using the default operator:x permissions.
It refactors a lot of the common testing functions so that they can be re-used between resources.
There are also some types that we don't yet support (e.g. virtual IPs) that this change adds ACL hooks to for future-proofing.
This implements the Filter field on pbcatalog.WorkloadSelector to be
a post-fetch in-memory filter using the https://github.com/hashicorp/go-bexpr
expression language to filter resources based on their envelope metadata fields.
All existing usages of WorkloadSelector should be able to make use of the filter.
* xdsv2: support l7 by adding xfcc policy/headers, tweaking routes, and make a bunch of listeners l7 tests pass
* sidecarproxycontroller: add l7 local app support
* trafficpermissions: make l4 traffic permissions work on l7 workloads
* rename route name field for consistency with l4 cluster name field
* resolve conflicts and rebase
* fix: ensure route name is used in l7 destination route name as well. previously it was only in the route names themselves, now the route name and l7 destination route name line up
Sometimes workloads could come with unspecified protocols such as when running on Kubernetes. Currently, if this is the case, we will just default to tcp protocol.
However, to make sidecar-proxy controller work with l7 protocols we should instead inherit the protocol from service. This change adds tracking for services that a workload is part of and attempts to inherit the protocol whenever services a workload is part of doesn't have conflicting protocols.
This change builds on #19043 and #19067 and updates the sidecar controller to use those computed resources. This achieves several benefits:
* The cache is now simplified which helps us solve for previous bugs (such as multiple Upstreams/Destinations targeting the same service would overwrite each other)
* We no longer need proxy config cache
* We no longer need to do merging of proxy configs as part of the controller logic
* Controller watches are simplified because we no longer need to have complex mapping using cache and can instead use the simple ReplaceType mapper.
It also makes several other improvements/refactors:
* Unifies all caches into one. This is because originally the caches were more independent, however, now that they need to interact with each other it made sense to unify them where sidecar proxy controller uses one cache with 3 bimappers
* Unifies cache and mappers. Mapper already needed all caches anyway and so it made sense to make the cache do the mapping also now that the cache is unified.
* Gets rid of service endpoints watches. This was needed to get updates in a case when service's identities have changed and we need to update proxy state template's spiffe IDs for those destinations. This will however generate a lot of reconcile requests for this controller as service endpoints objects can change a lot because they contain workload's health status. This is solved by adding a status to the service object tracking "bound identities" and have service endpoints controller update it. Having service's status updated allows us to get updates in the sidecar proxy controller because it's already watching service objects
* Add a watch for workloads. We need it so that we get updates if workload's ports change. This also ensures that we update cached identities in case workload's identity changes.
This commit adds a new type ComputedDestinations that will contain all destinations from any Destinations resources and will be name-aligned with a workload. This also adds an explicit-destinations controller that computes these resources.
This is needed to simplify the tracking we need to do currently in the sidecar-proxy controller and makes it easier to query all explicit destinations that apply to a workload.
* Introduce a new type `ComputedProxyConfiguration` and add a controller for it. This is needed for two reasons. The first one is that external integrations like kubernetes may need to read the fully computed and sorted proxy configuration per workload. The second reasons is that it makes sidecar-proxy controller logic quite a bit simpler as it no longer needs to do this.
* Generalize workload selection mapper and fix a bug where it would delete IDs from the tree if only one is left after a removal is done.
Fix issues with empty sources
* Validate that each permission on traffic permissions resources has at least one source.
* Don't construct RBAC policies when there aren't any principals. This resulted in Envoy rejecting xDS updates with a validation error.
```
error=
| rpc error: code = Internal desc = Error adding/updating listener(s) public_listener: Proto constraint validation failed (RBACValidationError.Rules: embedded message failed validation | caused by RBACValidationError.Policies[consul-intentions-layer4-1]: embedded message failed validation | caused by PolicyValidationError.Principals: value must contain at least 1 item(s)): rules {
```
Convert more of the xRoutes features that were skipped in an earlier PR into ComputedRoutes and make them work:
- DestinationPolicy defaults
- more timeouts
- load balancer policy
- request/response header mutations
- urlrewrite
- GRPCRoute matches
xRoute resource types contain a slice of parentRefs to services that they
manipulate traffic for. All xRoutes that have a parentRef to given Service
will be merged together to generate a ComputedRoutes resource
name-aligned with that Service.
This means that a write of an xRoute with 2 parent ref pointers will cause
at most 2 reconciles for ComputedRoutes.
If that xRoute's list of parentRefs were ever to be reduced, or otherwise
lose an item, that subsequent map event will only emit events for the current
set of refs. The removed ref will not cause the generated ComputedRoutes
related to that service to be re-reconciled to omit the influence of that xRoute.
To combat this, we will store on the ComputedRoutes resource a
BoundResources []*pbresource.Reference field with references to all
resources that were used to influence the generated output.
When the routes controller reconciles, it will use a bimapper to index this
influence, and the dependency mappers for the xRoutes will look
themselves up in that index to discover additional (former) ComputedRoutes
that need to be notified as well.
xRoute resources are not name-aligned with the Services they control. They
have a list of "parent ref" services that they alter traffic flow for, and they
contain a list of "backend ref" services that they direct that traffic to.
The ACLs should be:
- list: (default)
- read:
- ALL service:<parent_ref_service>:read
- write:
- ALL service:<parent_ref_service>:write
- ALL service:<backend_ref_service>:read
DestinationPolicy resources are name-aligned with the Service they control.
The ACLs should be:
- list: (default)
- read: service:<resource_name>:read
- write: service:<resource_name>:write
FailoverPolicy resources are name-aligned with the Service they control.
They also contain a list of possible failover destinations that are References
to other Services.
The ACLs should be:
- list: (default)
- read: service:<resource_name>:read
- write: service:<resource_name>:write + service:<destination_name>:read (for any destination)
The ACLs.Read hook for a resource only allows for the identity of a
resource to be passed in for use in authz consideration. For some
resources we wish to allow for the current stored value to dictate how
to enforce the ACLs (such as reading a list of applicable services from
the payload and allowing service:read on any of them to control reading the enclosing resource).
This change update the interface to usually accept a *pbresource.ID,
but if the hook decides it needs more data it returns a sentinel error
and the resource service knows to defer the authz check until after
fetching the data from storage.
* add multiple upstream ports to golden file test for destination builder
* NET-5131 - add unit tests for multiple ported upstreams
* fix merge conflicts
* add namespace proto and registration
* fix proto generation
* add missing copywrite headers
* fix proto linter errors
* fix exports and Type export
* add mutate hook and more validation
* add more validation rules and tests
* Apply suggestions from code review
Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
* fix owner error and add test
* remove ACL for now
* add tests around space suffix prefix.
* only fait when ns and ap are default, add test for it
---------
Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
Ensure that configuring a FailoverPolicy for a service that is reachable via a xRoute or a direct upstream causes an envoy aggregate cluster to be created for the original cluster name, but with separate clusters for each one of the possible destinations.
Adding coauthors who mobbed/paired at various points throughout last week.
Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: Michael Wilkerson <mwilkerson@hashicorp.com>
Previously, when using implicit upstreams, we'd build outbound listener per destination instead of one for all destinations. This will result in port conflicts when trying to send this config to envoy.
This PR also makes sure that leaf and root references are always added (before we would only add it if there are inbound non-mesh ports).
Also, black-hole traffic when there are no inbound ports other than mesh
HTTPRoute, GRPCRoute, TCPRoute, and Upstreams resources contain inner
Reference fields. We want to ensure that components of those reference Tenancy
fields left unspecified are defaulted using the tenancy of the enclosing resource.
As the underlying helper being used to do the normalization calls the function
modified in #18822, it also means that the PeerName field will be set to "local" for
now automatically to avoid "local" != "" issues downstream.
FailoverPolicy resources contain inner Reference fields. We want to ensure
that components of those reference Tenancy fields left unspecified are defaulted
using the tenancy of the enclosing FailoverPolicy resource.
As the underlying helper being used to do the normalization calls the function
modified in #18822, it also means that the PeerName field will be set to "local" for
now automatically to avoid "local" != "" issues downstream.
Reworks the sidecar controller to accept ComputedRoutes as an input and use it to generate appropriate ProxyStateTemplate resources containing L4/L7 mesh configuration.
When one resource contains an inner field that is of type *pbresource.Reference we want the
Tenancy to be reasonably defaulted by the following rules:
1. The final values will be limited by the scope of the referenced type.
2. Values will be inferred from the parent's tenancy, and if that is insufficient then using
the default tenancy for the type's scope.
3. Namespace will only be used from a parent if the reference and the parent share a
partition, otherwise the default namespace will be used.
Until we tackle peering, this hard codes an assumption of peer name being local. The
logic for defaulting may need adjustment when that is addressed.
* Refactors the leafcert package to not have a dependency on agent/consul and agent/cache to avoid import cycles. This way the xds controller can just import the leafcert package to use the leafcert manager.
The leaf cert logic in the controller:
* Sets up watches for leaf certs that are referenced in the ProxyStateTemplate (which generates the leaf certs too).
* Gets the leaf cert from the leaf cert cache
* Stores the leaf cert in the ProxyState that's pushed to xds
* For the cert watches, this PR also uses a bimapper + a thin wrapper to map leaf cert events to related ProxyStateTemplates
Since bimapper uses a resource.Reference or resource.ID to map between two resource types, I've created an internal type for a leaf certificate to use for the resource.Reference, since it's not a v2 resource.
The wrapper allows mapping events to resources (as opposed to mapping resources to resources)
The controller tests:
Unit: Ensure that we resolve leaf cert references
Lifecycle: Ensure that when the CA is updated, the leaf cert is as well
Also adds a new spiffe id type, and adds workload identity and workload identity URI to leaf certs. This is so certs are generated with the new workload identity based SPIFFE id.
* Pulls out some leaf cert test helpers into a helpers file so it
can be used in the xds controller tests.
* Wires up leaf cert manager dependency
* Support getting token from proxytracker
* Add workload identity spiffe id type to the authorize and sign functions
---------
Co-authored-by: John Murret <john.murret@hashicorp.com>
This new controller produces an intermediate output (ComputedRoutes) that is meant to summarize all relevant xRoutes and related mesh configuration in an easier-to-use format for downstream use to construct the ProxyStateTemplate.
It also applies status updates to the xRoute resource types to indicate that they are themselves semantically valid inputs.
* mesh-controller: handle L4 protocols for a proxy without upstreams
* sidecar-controller: Support explicit destinations for L4 protocols and single ports.
* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.
* endpoints-controller: add workload identity to the service endpoints resource
* small fixes
* review comments
* Address PR comments
* sidecar-proxy controller: Add support for transparent proxy
This currently does not support inferring destinations from intentions.
* PR review comments
* mesh-controller: handle L4 protocols for a proxy without upstreams
* sidecar-controller: Support explicit destinations for L4 protocols and single ports.
* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.
* endpoints-controller: add workload identity to the service endpoints resource
* small fixes
* review comments
* Make sure endpoint refs route to mesh port instead of an app port
* Address PR comments
* fixing copyright
* tidy imports
* sidecar-proxy controller: Add support for transparent proxy
This currently does not support inferring destinations from intentions.
* tidy imports
* add copyright headers
* Prefix sidecar proxy test files with source and destination.
* Update controller_test.go
* NET-5132 - Configure multiport routing for connect proxies in TProxy mode
* formatting golden files
* reverting golden files and adding changes in manually. build implicit destinations still has some issues.
* fixing files that were incorrectly repeating the outbound listener
* PR comments
* extract AlpnProtocol naming convention to getAlpnProtocolFromPortName(portName)
* removing address level filtering.
* adding license to resources_test.go
---------
Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
* mesh-controller: handle L4 protocols for a proxy without upstreams
* sidecar-controller: Support explicit destinations for L4 protocols and single ports.
* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.
* endpoints-controller: add workload identity to the service endpoints resource
* small fixes
* review comments
* Address PR comments
* sidecar-proxy controller: Add support for transparent proxy
This currently does not support inferring destinations from intentions.
* PR review comments
* mesh-controller: handle L4 protocols for a proxy without upstreams
* sidecar-controller: Support explicit destinations for L4 protocols and single ports.
* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.
* endpoints-controller: add workload identity to the service endpoints resource
* small fixes
* review comments
* Make sure endpoint refs route to mesh port instead of an app port
* Address PR comments
* fixing copyright
* tidy imports
* sidecar-proxy controller: Add support for transparent proxy
This currently does not support inferring destinations from intentions.
* tidy imports
* add copyright headers
* Prefix sidecar proxy test files with source and destination.
* Update controller_test.go
---------
Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
This commit adds support for transparent proxy to the sidecar proxy controller. As we do not yet support inferring destinations from intentions, this assumes that all services in the cluster are destinations.
* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.
* This commit also changes service endpoints to include workload identity. This made the implementation a bit easier as we don't need to look up as many workloads and instead rely on endpoints data.
We explicitly enumerate the allowed protocols in validation, so this
change is necessary to use the new enum value.
Also add tests for enum validators to ensure they stay aligned to
protos unless we explicitly want them to diverge.
Chris S. Kim
Semir Patel
Ganesh S
aahel
Michael Zalimeni
John Murret
Ashesh Vidyut
R.B. Boyer
Kumar Kavish
Nathan Coleman
Matt Keeler
Poonam Jadhav
skpratt
Iryna Shustava
Dhia Ayachi
Nitya Dhanushkodi
Eric Haberkorn
Ashwin Venkatesh
Derek Menteer
wangxinyi7