Commit Graph

125 Commits

Author SHA1 Message Date
Nitya Dhanushkodi 78b170ad50
xds controller: setup watches for and compute leaf cert references in ProxyStateTemplate, and wire up leaf cert manager dependency (#18756)
* Refactors the leafcert package to not have a dependency on agent/consul and agent/cache to avoid import cycles. This way the xds controller can just import the leafcert package to use the leafcert manager.

The leaf cert logic in the controller:
* Sets up watches for leaf certs that are referenced in the ProxyStateTemplate (which generates the leaf certs too).
* Gets the leaf cert from the leaf cert cache
* Stores the leaf cert in the ProxyState that's pushed to xds
* For the cert watches, this PR also uses a bimapper + a thin wrapper to map leaf cert events to related ProxyStateTemplates

Since bimapper uses a resource.Reference or resource.ID to map between two resource types, I've created an internal type for a leaf certificate to use for the resource.Reference, since it's not a v2 resource.
The wrapper allows mapping events to resources (as opposed to mapping resources to resources)

The controller tests:
Unit: Ensure that we resolve leaf cert references
Lifecycle: Ensure that when the CA is updated, the leaf cert is as well

Also adds a new spiffe id type, and adds workload identity and workload identity URI to leaf certs. This is so certs are generated with the new workload identity based SPIFFE id.

* Pulls out some leaf cert test helpers into a helpers file so it
can be used in the xds controller tests.
* Wires up leaf cert manager dependency
* Support getting token from proxytracker
* Add workload identity spiffe id type to the authorize and sign functions



---------

Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-09-12 12:56:43 -07:00
R.B. Boyer 89e6725eee
mesh: create new routes-controller to reconcile xRoute types into a ComputedRoutes resource (#18460)
This new controller produces an intermediate output (ComputedRoutes) that is meant to summarize all relevant xRoutes and related mesh configuration in an easier-to-use format for downstream use to construct the ProxyStateTemplate.

It also applies status updates to the xRoute resource types to indicate that they are themselves semantically valid inputs.
2023-09-12 12:08:51 -05:00
Poonam Jadhav 264166fcc0
fix: write endpoint errors out gracefully (#18743) 2023-09-12 09:22:15 -04:00
John Murret 62062fd4fd
NET-5132 - Configure multiport routing for connect proxies in TProxy mode (#18606)
* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Address PR comments

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* PR review comments

* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Make sure endpoint refs route to mesh port instead of an app port

* Address PR comments

* fixing copyright

* tidy imports

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* tidy imports

* add copyright headers

* Prefix sidecar proxy test files with source and destination.

* Update controller_test.go

* NET-5132 - Configure multiport routing for connect proxies in TProxy mode

* formatting golden files

* reverting golden files and adding changes in manually.  build implicit destinations still has some issues.

* fixing files that were incorrectly repeating the outbound listener

* PR comments

* extract AlpnProtocol naming convention to getAlpnProtocolFromPortName(portName)

* removing address level filtering.

* adding license to resources_test.go

---------

Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
2023-09-12 01:17:56 +00:00
Dhia Ayachi b1688ad856
Run copyright after running deep-copy as part of the Makefile/CI (#18741)
* execute copyright headers after performing deep-copy generation.

* fix copyright install

* Apply suggestions from code review

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* Apply suggestions from code review

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* rename steps to match codegen naming

* remove copywrite install category

---------

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
2023-09-11 13:50:52 -04:00
R.B. Boyer a69e901660
xds: update golden tests to be deterministic (#18707) 2023-09-11 11:40:19 -05:00
John Murret 3e78b4cf34
Prefix sidecar proxy test files with source and destination. (#18620)
* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Address PR comments

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* PR review comments

* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Make sure endpoint refs route to mesh port instead of an app port

* Address PR comments

* fixing copyright

* tidy imports

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* tidy imports

* add copyright headers

* Prefix sidecar proxy test files with source and destination.

* Update controller_test.go

---------

Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
2023-09-08 17:28:55 -06:00
Iryna Shustava 1557e1d6a3
sidecar-proxy controller: Add support for transparent proxy (NET-5069) (#18458)
This commit adds support for transparent proxy to the sidecar proxy controller. As we do not yet support inferring destinations from intentions, this assumes that all services in the cluster are destinations.
2023-09-08 16:18:01 -06:00
R.B. Boyer acd9b3d1c4
test: update sidecarproxy/builder golden tests to use determinstic golden data (#18703) 2023-09-07 13:04:17 -05:00
Iryna Shustava 3c70e14713
sidecar-proxy controller: L4 controller with explicit upstreams (NET-3988) (#18352)
* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.
* This commit also changes service endpoints to include workload identity. This made the implementation a bit easier as we don't need to look up as many workloads and instead rely on endpoints data.
2023-09-07 09:37:15 -06:00
wangxinyi7 df9d12a56a
Net 2714/xw cli read command (#18462)
enable `consul resource read` command in cli
2023-09-05 09:17:19 -07:00
Semir Patel b96cff7436
resource: Require scope for resource registration (#18635) 2023-09-01 09:44:53 -05:00
Michael Zalimeni 699aa47416
fix: make UNSPECIFIED protocol pass validation (#18634)
We explicitly enumerate the allowed protocols in validation, so this
change is necessary to use the new enum value.

Also add tests for enum validators to ensure they stay aligned to
protos unless we explicitly want them to diverge.
2023-08-31 19:59:29 +00:00
Dhia Ayachi f8d77f027a
delete all v2 resources type when deleting a namespace (CE) (#18621)
* add namespace scope to ServiceV1Alpha1Type

* add CE portion of namespace deletion
2023-08-31 10:18:25 -04:00
Ashwin Venkatesh 797e42dc24
Watch the ProxyTracker from xDS controller (#18611) 2023-08-29 14:39:29 -07:00
John Murret 0e606504bc
NET-4944 - wire up controllers with proxy tracker (#18603)
Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
2023-08-29 09:15:34 -06:00
John Murret 051f250edb
NET-5338 - NET-5338 - Run a v2 mode xds server (#18579)
* NET-5338 - NET-5338 - Run a v2 mode xds server

* fix linting
2023-08-24 16:44:14 -06:00
Semir Patel 067a0112e2
resource: Make resource listbyowner tenancy aware (#18566) 2023-08-24 10:49:46 -05:00
R.B. Boyer 8a931241f2
chore: fix missing/incorrect license headers (#18555) 2023-08-22 17:23:54 -05:00
R.B. Boyer 5b88aae3b4
catalog: validating Protocol and Health enums on Service, Workload, and ServiceEndpoints (#18554) 2023-08-22 15:16:55 -05:00
Ashwin Venkatesh 4f9955d91e
Update trust bundle into proxy-state-template (#18550) 2023-08-22 19:38:31 +00:00
R.B. Boyer 17667a1c75
mesh: adding type aliases for mesh resource usage (#18448)
Introduces some simple type aliases for DecodedResource[*X] wrappers for each type which cut down on the verbosity
2023-08-22 12:31:06 -05:00
Ashwin Venkatesh 0d60380214
xds controller: resolve ServiceEndpoints references in ProxyStateTemp… (#18544)
xds controller: resolve ServiceEndpoints references in ProxyStateTemplate
2023-08-22 10:01:45 -07:00
R.B. Boyer 55723c541e
mesh: add validation for the new pbmesh resources (#18410)
Adds validation for HTTPRoute, GRPCRoute, TCPRoute, DestinationPolicy, and ComputedRoutes.
2023-08-22 11:27:09 -05:00
R.B. Boyer 570c84d032
catalog: add failover mode enum validation (#18545) 2023-08-22 11:09:37 -05:00
Semir Patel 53e28a4963
OSS -> CE (community edition) changes (#18517) 2023-08-22 09:46:03 -05:00
Matt Keeler 547f4f8395
Reduce required type arguments for DecodedResource (#18540) 2023-08-21 20:20:19 -04:00
Iryna Shustava 0b580ffd22
bimapper: fix data race (#18519) 2023-08-18 07:55:12 -06:00
Iryna Shustava cc596ce772
bimapper: allow to untrack links and support reference or id (#18451) 2023-08-17 18:03:05 -06:00
Ashwin Venkatesh 97b41d946f
Support custom watches on controller (#18439)
* Support custom watches on controller
* refactor mapper methods
2023-08-17 20:34:18 +00:00
Semir Patel 217107f627
resource: Make resource list tenancy aware (#18475) 2023-08-15 16:57:59 -05:00
Poonam Jadhav f88d4fe28f
Net-2707/list resource endpoint (#18444)
feat: list resources endpoint
2023-08-15 09:11:50 -04:00
wangxinyi7 cda884ac81
read endpoint (#18268)
implement http read endpoint to expose resource grpc service read method
2023-08-11 14:11:11 -07:00
Poonam Jadhav 559c61e6b6
Net-2712/resource hcl parsing (#18250)
* Initial protohcl implementation

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
Co-authored-by: Daniel Upton <daniel@floppy.co>

* resourcehcl: implement resource decoding on top of protohcl

Co-authored-by: Daniel Upton <daniel@floppy.co>

* fix: resolve ci failures

* test: add additional unmarshalling tests

* refactor: update function test to clean protohcl package imports

---------

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
Co-authored-by: Daniel Upton <daniel@floppy.co>
2023-08-11 15:52:51 -04:00
Poonam Jadhav 5717cbd466
Net-2708/delete resource endpoint (#18420)
* feat: add http delete endpoint for resource service

* refactor: clean up
2023-08-11 13:22:30 +00:00
hashicorp-copywrite[bot] 5fb9df1640
[COMPLIANCE] License changes (#18443)
* Adding explicit MPL license for sub-package

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Adding explicit MPL license for sub-package

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Updating the license from MPL to Business Source License

Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.

* add missing license headers

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

---------

Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
2023-08-11 09:12:13 -04:00
Semir Patel bee12c6b1f
resource: Make resource write tenancy aware (#18423) 2023-08-10 09:53:38 -05:00
R.B. Boyer 42efc11b4e
catalog: adding a controller to reconcile FailoverPolicy resources (#18399)
Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller.
2023-08-09 11:02:17 -05:00
R.B. Boyer bfc519f293
catalog: add FailoverPolicy mutation and validation hooks (#18390)
Add most of the validation and mutation hooks for the FailoverPolicy resource.
2023-08-08 14:42:43 -05:00
Matt Keeler 91d331bbaa
Add ServiceEndpoints Mutation hook tests (#18404)
* Add ServiceEndpoints Mutation hook tests

* Move endpoint owner validation into the validation hook

Also there were some minor changes to error validation to account for go-cmp not liking to peer through an errors.errorstring type that get created by errors.New
2023-08-08 15:22:14 -04:00
Semir Patel 63cc037110
resource: Make resource read tenancy aware (#18397) 2023-08-07 16:37:03 -05:00
R.B. Boyer 1ebd001a07
bimapper: fix a bug and add some more test coverage (#18387) 2023-08-04 16:45:10 -05:00
wangxinyi7 1f28ac2664
expose grpc as http endpoint (#18221)
expose resource grpc endpoints as http endpoints
2023-08-04 11:27:48 -07:00
R.B. Boyer 9c227e2c36
mesh: adding the protobuf types and resources backing mesh config v2 (#18351) 2023-08-03 13:42:04 -05:00
Iryna Shustava a33001f4d4
Register ProxyStateTemplate Resource (#18316)
Also, change the ProxyState.id to identity. This is because we already have the id of this proxy
  from the resource, and this id should be name-aligned with the workload it represents. It should
  also have the owner ref set to the workload ID if we need that. And so the id field seems unnecessary.
  We do, however, need a reference to workload identity so that we can authorize the proxy when it initially
  connects to the xDS server.
2023-08-02 08:15:13 -06:00
R.B. Boyer 13ce787a3f
resource: adding various helpers for working with resources (#18342)
This is a bit of a grab bag of helpers that I found useful for working with them when authoring substantial Controllers. Subsequent PRs will make use of them.
2023-08-01 13:39:15 -05:00
Semir Patel efb45fe851
resource: Add scope to resource type registration [NET-4976] (#18214)
Enables querying a resource type's registration to determine if a resource is cluster, partition, or partition and namespace scoped.
2023-07-24 11:34:30 -05:00
Semir Patel ada767fc9f
resource: Pass resource to Write ACL hook instead of just resource Id [NET-4908] (#18192) 2023-07-20 12:06:29 -05:00
Semir Patel 003370ded0
Call resource mutate hook before validate hook (NET-4907) (#18178) 2023-07-19 13:10:57 -05:00
Iryna Shustava c328ba85bd
Split pbmesh.UpstreamsConfiguration as a resource out of pbmesh.Upstreams (#17991)
Configuration that previously was inlined into the Upstreams resource
applies to both explicit and implicit upstreams and so it makes sense to
split it out into its own resource.

It also has other minor changes:
- Renames `proxy.proto` proxy_configuration.proto`
- Changes the type of `Upstream.destination_ref` from `pbresource.ID` to
`pbresource.Reference`
- Adds comments to fields that didn't have them
2023-07-13 13:06:56 -06:00