Commit Graph

20849 Commits

Author SHA1 Message Date
skpratt 1fda2965e8
Allow empty data writes for resources (#18819)
* allow nil data writes for resources

* update demo to test valid type with no data
2023-09-15 14:00:23 -05:00
James Hartig b2e21c103f
consul operator raft transfer-leader should send the id (#17107)
Fixes #16955

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2023-09-15 14:38:59 -04:00
Ronald aff13cd4c2
Use embedded strings for templated policies (#18829) 2023-09-15 13:49:22 -04:00
sarahalsmiller 753c8f1774
Retry and timeout test acceptance test (#18791)
* retry and timeout test

* add docker mirrior

* checkpoint

* add in error

* add in delay

* up error rate

* fix status code
2023-09-15 11:54:51 -05:00
Iryna Shustava 6838441c54
Default to tcp protocol when workload protocol is unspecified (#18824) 2023-09-15 10:39:53 -06:00
trujillo-adam 850fbda2e9
added consul and envoy version constraints (#18726)
* added consul and envoy version constraints

* fixed Destination configuraiton and added tproxy requirement

* Apply suggestions from code review

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>

---------

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
2023-09-15 09:37:11 -07:00
Eric Haberkorn 21fdbbabbc
Wire up traffic permissions (#18812)
Wire up traffic permissions
2023-09-15 12:31:22 -04:00
Semir Patel d3dad14030
resource: default peername to "local" for now (#18822) 2023-09-15 09:34:18 -05:00
Ronald e24b3d9b3f
fix templated policy cli output (#18821) 2023-09-15 14:20:44 +00:00
R.B. Boyer 9c1a1ffcde
mesh: call the right mapper (#18818) 2023-09-14 22:43:05 +00:00
R.B. Boyer 66e1cdf40c
mesh: Wire ComputedRoutes into the ProxyStateTemplate via the sidecar controller (#18752)
Reworks the sidecar controller to accept ComputedRoutes as an input and use it to generate appropriate ProxyStateTemplate resources containing L4/L7 mesh configuration.
2023-09-14 17:19:04 -05:00
Ronald 1afeb6e040
[NET-5334] Added CLI commands for templated policies (#18816) 2023-09-14 20:14:55 +00:00
Ronald 802122640b
[NET-5329] use acl templated policy under the hood for node/service identities (#18813) 2023-09-14 14:36:34 -04:00
skpratt e5808d85f7
register traffic permission and workload identity types (#18704)
* add workload identity and traffic permission protos

* register new types

* add generated pb code

* fix exports.go path

* add proto newlines

* fix type name

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>

* address review

* fix protos and add tests

* fix validation constraints

* add tests

---------

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
2023-09-14 12:40:54 -05:00
Chris S. Kim d667cc3809
Fix flaky test (#18805) 2023-09-14 16:46:47 +00:00
Melisa Griffin e6b3c26d5f
Removes errant word in docs that was breaking the build (#18804) 2023-09-14 16:37:30 +00:00
Melisa Griffin 8997d1a4f8
Updates gateway-class-config docs to include fields for privileged container port-mapping and Openshift Security Context Constraints
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-09-14 12:08:26 -04:00
Chris S. Kim 6748fac43d
Clean up resources in test (#18799) 2023-09-14 15:33:19 +00:00
Dhia Ayachi 658c27a684
add fuzz tests to resourcehcl package and fix some panics (#18798)
* add a simple fuzz test for the resourcehcl package and fix some panics uncovered by the test

* fix default for null values
2023-09-14 10:59:33 -04:00
Iryna Shustava 7f2a1d9812
catalog: service endpoints inherits protocol from service when workload doesn't have one (#18792) 2023-09-14 08:52:12 -06:00
Derek Menteer 02259ef964
Fix snapshot creation issue. (#18783)
The renaming of files from oss -> ce caused incorrect snapshots
to be created due to ce writes now happening prior to ent writes.
When this happens various entities will attempt to be restored
from the snapshot prior to a partition existing and will cause a
panic to occur.
2023-09-13 15:35:31 -05:00
Chris S. Kim 4dfca64ded
Vault CA provider clean up previous default issuers (#18773) 2023-09-13 19:33:02 +00:00
wangxinyi7 426c906ee1
delete command ready (#18679)
* delete command for resource management
2023-09-13 11:48:40 -07:00
Michael Zalimeni 09fffd4b1e
Disable docs change CI skip detection (#18775)
There's currently a bug that causes CI to be skipped on all non-PR
changes. Until that's fixed and we can be certain the check will fail CI
or default to running tests in the case of errors, disabling this check.
2023-09-13 17:27:02 +00:00
R.B. Boyer 07f54fe3b8
resource: add helper to normalize inner Reference tenancy during mutate (#18765)
When one resource contains an inner field that is of type *pbresource.Reference we want the
Tenancy to be reasonably defaulted by the following rules:

1. The final values will be limited by the scope of the referenced type.
2. Values will be inferred from the parent's tenancy, and if that is insufficient then using
   the default tenancy for the type's scope.
3. Namespace will only be used from a parent if the reference and the parent share a
   partition, otherwise the default namespace will be used.

Until we tackle peering, this hard codes an assumption of peer name being local. The
logic for defaulting may need adjustment when that is addressed.
2023-09-13 12:08:12 -05:00
Eric Haberkorn 12be06f8e5
Add V2 TCP traffic permissions (#18771)
Add support for TCP traffic permissions
2023-09-13 09:03:42 -04:00
Ashesh Vidyut 0b6efbb9c1
Windows Integration Test Changes (#18758)
* some changes to debug

* revert machines

* increased timeout

* added sleep 10 seconds before test start

* chagne envoy version

* removed sleep

* revert timeout

* replace position

* removed date

* Revert "[NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437)"

This reverts commit 05604eeec1.

* fix build

* Revert "replace position"

This reverts commit 48e6af46a8daae186c283f30d316b1104906993e.

* Revert "Revert "[NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437)""

This reverts commit d7c568e2be727b72e6827225782e0e17ac06b74a.

* comment out api gateway http hostnames test

* fix import

* revert integ test run on PR
2023-09-13 15:40:08 +05:30
Chris S. Kim d090668c37
Add workload identity ACL rules (#18769) 2023-09-12 17:22:51 -04:00
Sophie Gairo a55c4a1c25
Test: NET-5592 - update Nomad integration testing (#18768)
NET-5592 - update Nomad integration testing
2023-09-12 14:59:27 -05:00
Nitya Dhanushkodi 78b170ad50
xds controller: setup watches for and compute leaf cert references in ProxyStateTemplate, and wire up leaf cert manager dependency (#18756)
* Refactors the leafcert package to not have a dependency on agent/consul and agent/cache to avoid import cycles. This way the xds controller can just import the leafcert package to use the leafcert manager.

The leaf cert logic in the controller:
* Sets up watches for leaf certs that are referenced in the ProxyStateTemplate (which generates the leaf certs too).
* Gets the leaf cert from the leaf cert cache
* Stores the leaf cert in the ProxyState that's pushed to xds
* For the cert watches, this PR also uses a bimapper + a thin wrapper to map leaf cert events to related ProxyStateTemplates

Since bimapper uses a resource.Reference or resource.ID to map between two resource types, I've created an internal type for a leaf certificate to use for the resource.Reference, since it's not a v2 resource.
The wrapper allows mapping events to resources (as opposed to mapping resources to resources)

The controller tests:
Unit: Ensure that we resolve leaf cert references
Lifecycle: Ensure that when the CA is updated, the leaf cert is as well

Also adds a new spiffe id type, and adds workload identity and workload identity URI to leaf certs. This is so certs are generated with the new workload identity based SPIFFE id.

* Pulls out some leaf cert test helpers into a helpers file so it
can be used in the xds controller tests.
* Wires up leaf cert manager dependency
* Support getting token from proxytracker
* Add workload identity spiffe id type to the authorize and sign functions



---------

Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-09-12 12:56:43 -07:00
R.B. Boyer 89e6725eee
mesh: create new routes-controller to reconcile xRoute types into a ComputedRoutes resource (#18460)
This new controller produces an intermediate output (ComputedRoutes) that is meant to summarize all relevant xRoutes and related mesh configuration in an easier-to-use format for downstream use to construct the ProxyStateTemplate.

It also applies status updates to the xRoute resource types to indicate that they are themselves semantically valid inputs.
2023-09-12 12:08:51 -05:00
Chris Thain 4724a4e169
Add Envoy golden test for OTEL access logging extension (#18760) 2023-09-12 09:58:53 -07:00
Michael Zalimeni 5e7afdf9a1
[NET-5574] Update Go version to 1.20.8 (#18742)
Update Go version to 1.20.8

This resolves several CVEs (see changelog entry).
2023-09-12 11:40:51 -04:00
Poonam Jadhav 264166fcc0
fix: write endpoint errors out gracefully (#18743) 2023-09-12 09:22:15 -04:00
Ashesh Vidyut 697836b19a
added time stamps (#18757) 2023-09-12 09:22:31 +00:00
John Murret 62062fd4fd
NET-5132 - Configure multiport routing for connect proxies in TProxy mode (#18606)
* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Address PR comments

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* PR review comments

* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Make sure endpoint refs route to mesh port instead of an app port

* Address PR comments

* fixing copyright

* tidy imports

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* tidy imports

* add copyright headers

* Prefix sidecar proxy test files with source and destination.

* Update controller_test.go

* NET-5132 - Configure multiport routing for connect proxies in TProxy mode

* formatting golden files

* reverting golden files and adding changes in manually.  build implicit destinations still has some issues.

* fixing files that were incorrectly repeating the outbound listener

* PR comments

* extract AlpnProtocol naming convention to getAlpnProtocolFromPortName(portName)

* removing address level filtering.

* adding license to resources_test.go

---------

Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
2023-09-12 01:17:56 +00:00
Ronald 9776c10efb
[NET-5333] Add api to read/list and preview templated policies (#18748) 2023-09-11 18:11:37 +00:00
Dhia Ayachi b1688ad856
Run copyright after running deep-copy as part of the Makefile/CI (#18741)
* execute copyright headers after performing deep-copy generation.

* fix copyright install

* Apply suggestions from code review

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* Apply suggestions from code review

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* rename steps to match codegen naming

* remove copywrite install category

---------

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
2023-09-11 13:50:52 -04:00
R.B. Boyer a69e901660
xds: update golden tests to be deterministic (#18707) 2023-09-11 11:40:19 -05:00
Ashesh Vidyut 172e915074
Added logs for debugging (#18740)
* added logs

* added echo

* removed pull request
2023-09-11 14:24:50 +00:00
Poonam Jadhav 48ef8b679c
Net-4292/list command (#18649)
* feat: implement list command

* refactor: apply command file parsing
2023-09-11 10:06:00 -04:00
Ashesh Vidyut 7429aba3c9
Increase timeout for integration test windows. (#18737)
increase timeout
2023-09-11 08:24:49 +00:00
Ashesh Vidyut bf291f5b8d
Fix windows integrations tests machine size (#18736)
fix windows integrations tests machine size
2023-09-11 06:36:56 +00:00
David Yu 24d031d1ed
k8s: Update audit-logging docs to include K8s examples (#18721)
Update audit-logging.mdx
2023-09-10 11:10:28 -07:00
John Murret 3e78b4cf34
Prefix sidecar proxy test files with source and destination. (#18620)
* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Address PR comments

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* PR review comments

* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Make sure endpoint refs route to mesh port instead of an app port

* Address PR comments

* fixing copyright

* tidy imports

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* tidy imports

* add copyright headers

* Prefix sidecar proxy test files with source and destination.

* Update controller_test.go

---------

Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
2023-09-08 17:28:55 -06:00
Iryna Shustava 1557e1d6a3
sidecar-proxy controller: Add support for transparent proxy (NET-5069) (#18458)
This commit adds support for transparent proxy to the sidecar proxy controller. As we do not yet support inferring destinations from intentions, this assumes that all services in the cluster are destinations.
2023-09-08 16:18:01 -06:00
Nathan Coleman ed79c60e78
NET-5530 Generate deep-copy code (#18730)
* Generate deep-copy code

* Undo license header removal
2023-09-08 20:09:44 +00:00
Ronald 40d7ebc318
[NET-5330] Support templated policies in Binding rules (#18719)
* [NET-5330] Support templated policies in Binding rules

* changelog for templated policy support in binding rules
2023-09-08 14:39:09 -04:00
Semir Patel 576ffdf705
fix: emit consul version metric on a regular interval (#18724) 2023-09-08 13:09:07 -05:00
Nathan Coleman e5d26a13cd
NET-5530 Support response header modifiers on http-route config entry (#18646)
* Add response header filters to http-route config entry definitions

* Map response header filters from config entry when constructing route destination

* Support response header modifiers at the service level as well

* Update protobuf definitions

* Update existing unit tests

* Add response filters to route consolidation logic

* Make existing unit tests more robust

* Add missing docstring

* Add changelog entry

* Add response filter modifiers to existing integration test

* Add more robust testing for response header modifiers in the discovery chain

* Add more robust testing for request header modifiers in the discovery chain

* Modify test to verify that service filter modifiers take precedence over rule filter modifiers
2023-09-08 14:04:56 -04:00