Commit Graph

5523 Commits

Author SHA1 Message Date
John Murret 21ea5c92fd
NET-6944 - Replace usage of deprecated Envoy field envoy.extensions.filters.http.lua.v3.Lua.inline_code (#20012) 2023-12-22 17:20:41 +00:00
Nathan Coleman ab60fec15a
[NET-6426] Add gateway proxy controller that generates empty proxy state template (#19901)
* NET-6426 Create ProxyStateTemplate when reconciling MeshGateway resource

* Add TODO for switching fetch method based on gateway type

* Use gateway-kind in workload metadata instead of owner reference

* Create ProxyStateTemplate builder for gatewayproxy controller

* Update to use new controller interface

* Add copyright headers

* Set correct name for ProxyStateTemplate identity reference

* Generate empty ProxyStateTemplate by fetching MeshGateway

This cheats and looks up the MeshGateway directly. In the future, we will need a Workload => xGateway mapper

* Specify owner reference when writing ProxyStateTemplate

* Update dependency mapper to account for multiple controllers per resource type

* Regenerate v2 resource dependencies map

* Add helpful trace logs, tag TODOs with ticket identifiers
2023-12-21 16:37:47 -05:00
Nitya Dhanushkodi 9975b8bd73
[NET-5455] Allow disabling request and idle timeouts with negative values in service router and service resolver (#19992)
* add coverage for testing these timeouts
2023-12-19 15:36:07 -08:00
cskh cff872749d
agent: prevent empty server_metadata.json (#19935) 2023-12-19 10:01:56 -05:00
aahel ae998a698a
added computed failover policy resource (#19975) 2023-12-18 05:52:24 +00:00
Derek Menteer bbdbf3e4f8
Fix bug with prepared queries using sameness-groups. (#19970)
This commit fixes an issue where the partition was not properly set
on the peering query failover target created from sameness-groups.
Before this change, it was always empty, meaning that the data
would be queried with respect to the default partition always. This
resulted in a situation where a PQ that was attempting to use a
sameness-group for failover would select peers from the default
partition, rather than the partition of the sameness-group itself.
2023-12-15 11:42:13 -06:00
aahel a6496898de
added tenancy to TestBuildL4TrafficPermissions (#19932) 2023-12-14 10:41:24 +05:30
Matt Keeler 123bc95e1a
Add Common Controller Caching Infrastructure (#19767)
* Add Common Controller Caching Infrastructure
2023-12-13 10:06:39 -05:00
Dhia Ayachi f2b26ac194
Hash based config entry replication (#19795)
* add a hash to config entries when normalizing

* add GetHash and implement comparing hashes

* only update if the Hash is different

* only update if the Hash is different and not 0

* fix proto to include the Hash

* fix proto gen

* buf format

* add SetHash and fix tests

* fix config load tests

* fix state test and config test

* recalculate hash when restoring config entries

* fix snapshot restore test

* add changelog

* fix missing normalize, fix proto indexes and add normalize test
2023-12-12 08:29:13 -05:00
Ronald e13fbc743e
Remove warning for consul 1.17 deprecation (#19897) 2023-12-11 23:28:04 +00:00
Derek Menteer dfab5ade50
Fix ClusterLoadAssignment timeouts dropping endpoints. (#19871)
When a large number of upstreams are configured on a single envoy
proxy, there was a chance that it would timeout when waiting for
ClusterLoadAssignments. While this doesn't always immediately cause
issues, consul-dataplane instances appear to consistently drop
endpoints from their configurations after an xDS connection is
re-established (the server dies, random disconnect, etc).

This commit adds an `xds_fetch_timeout_ms` config to service registrations
so that users can set the value higher for large instances that have
many upstreams. The timeout can be disabled by setting a value of `0`.

This configuration was introduced to reduce the risk of causing a
breaking change for users if there is ever a scenario where endpoints
would never be received. Rather than just always blocking indefinitely
or for a significantly longer period of time, this config will affect
only the service instance associated with it.
2023-12-11 09:25:11 -06:00
Derek Menteer 0ac958f27b
Fix xDS missing endpoint race condition. (#19866)
This fixes the following race condition:
- Send update endpoints
- Send update cluster
- Recv ACK endpoints
- Recv ACK cluster

Prior to this fix, it would have resulted in the endpoints NOT existing in
Envoy. This occurred because the cluster update implicitly clears the endpoints
in Envoy, but we would never re-send the endpoint data to compensate for the
loss, because we would incorrectly ACK the invalid old endpoint hash. Since the
endpoint's hash did not actually change, they would not be resent.

The fix for this is to effectively clear out the invalid pending ACKs for child
resources whenever the parent changes. This ensures that we do not store the
child's hash as accepted when the race occurs.

An escape-hatch environment variable `XDS_PROTOCOL_LEGACY_CHILD_RESEND` was
added so that users can revert back to the old legacy behavior in the event
that this produces unknown side-effects. Visit the following thread for some
extra context on why certainty around these race conditions is difficult:
https://github.com/envoyproxy/envoy/issues/13009

This bug report and fix was mostly implemented by @ksmiley with some minor
tweaks.

Co-authored-by: Keith Smiley <ksmiley@salesforce.com>
2023-12-08 11:37:12 -06:00
Thomas Eckert 8125a32a4e
Add CE version of Gateway Upstream Disambiguation (#19860)
* Add CE version of gateway-upstream-disambiguation

* Use NamespaceOrDefault and PartitionOrDefault

* Add Changelog entry

* Remove the unneeded reassignment

* Use c.ID()
2023-12-07 17:56:14 -05:00
Dhia Ayachi d93f7f730d
parse config protocol on write to optimize disco-chain compilation (#19829)
* parse config protocol on write to optimize disco-chain compilation

* add changelog
2023-12-07 13:46:46 -05:00
Jared Kirschner d3e658b0e7
improve client RPC metrics consistency (#19721)
The client.rpc metric now excludes internal retries for consistency
with client.rpc.exceeded and client.rpc.failed. All of these metrics
now increment at most once per RPC method call, allowing for
accurate calculation of failure / rate limit application occurrence.

Additionally, if an RPC fails because no servers are present,
client.rpc.failed is now incremented.
2023-12-06 13:21:08 -05:00
Matt Keeler efe279f802
Retry lint fixes (#19151)
* Add a make target to run lint-consul-retry on all the modules
* Cleanup sdk/testutil/retry
* Fix a bunch of retry.Run* usage to not use the outer testing.T
* Fix some more recent retry lint issues and pin to v1.4.0 of lint-consul-retry
* Fix codegen copywrite lint issues
* Don’t perform cleanup after each retry attempt by default.
* Use the common testutil.TestingTB interface in test-integ/tenancy
* Fix retry tests
* Update otel access logging extension test to perform requests within the retry block
2023-12-06 12:11:32 -05:00
Ronald dc02fa695f
[NET-6251] Nomad client templated policy (#19827) 2023-12-06 10:32:12 -05:00
Semir Patel c1bbda8128
resource: block default namespace deletion + test refactorings (#19822) 2023-12-05 14:00:06 -05:00
lornasong edf4610ed9
[Cloud][CC-6925] Updates to pushing server state (#19682)
* Upgrade hcp-sdk-go to latest version v0.73

Changes:
- go get github.com/hashicorp/hcp-sdk-go
- go mod tidy

* From upgrade: regenerate protobufs for upgrade from 1.30 to 1.31

Ran: `make proto`

Slack: https://hashicorp.slack.com/archives/C0253EQ5B40/p1701105418579429

* From upgrade: fix mock interface implementation

After upgrading, there is the following compile error:

cannot use &mockHCPCfg{} (value of type *mockHCPCfg) as "github.com/hashicorp/hcp-sdk-go/config".HCPConfig value in return statement: *mockHCPCfg does not implement "github.com/hashicorp/hcp-sdk-go/config".HCPConfig (missing method Logout)

Solution: update the mock to have the missing Logout method

* From upgrade: Lint: remove usage of deprecated req.ServerState.TLS

Due to upgrade, linting is erroring due to usage of a newly deprecated field

22:47:56 [consul]: make lint
--> Running golangci-lint (.)
agent/hcp/testing.go:157:24: SA1019: req.ServerState.TLS is deprecated: use server_tls.internal_rpc instead. (staticcheck)
                time.Until(time.Time(req.ServerState.TLS.CertExpiry)).Hours()/24,
                                     ^

* From upgrade: adjust oidc error message

From the upgrade, this test started failing:

=== FAIL: internal/go-sso/oidcauth TestOIDC_ClaimsFromAuthCode/failed_code_exchange (re-run 2) (0.01s)
    oidc_test.go:393: unexpected error: Provider login failed: Error exchanging oidc code: oauth2: "invalid_grant" "unexpected auth code"

Prior to the upgrade, the error returned was:
```
Provider login failed: Error exchanging oidc code: oauth2: cannot fetch token: 401 Unauthorized\nResponse: {\"error\":\"invalid_grant\",\"error_description\":\"unexpected auth code\"}\n
```

Now the error returned is as below and does not contain "cannot fetch token"
```
Provider login failed: Error exchanging oidc code: oauth2: "invalid_grant" "unexpected auth code"

```

* Update AgentPushServerState structs with new fields

HCP-side changes for the new fields are in:
https://github.com/hashicorp/cloud-global-network-manager-service/pull/1195/files

* Minor refactor for hcpServerStatus to abstract tlsInfo into struct

This will make it easier to set the same tls-info information to both
 - status.TLS (deprecated field)
 - status.ServerTLSMetadata (new field to use instead)

* Update hcpServerStatus to parse out information for new fields

Changes:
 - Improve error message and handling (encountered some issues and was confused)
 - Set new field TLSInfo.CertIssuer
 - Collect certificate authority metadata and set on TLSInfo.CertificateAuthorities
 - Set TLSInfo on both server.TLS and server.ServerTLSMetadata.InternalRPC

* Update serverStatusToHCP to convert new fields to GNM rpc

* Add changelog

* Feedback: connect.ParseCert, caCerts

* Feedback: refactor and unit test server status

* Feedback: test to use expected struct

* Feedback: certificate with intermediate

* Feedback: catch no leaf, remove expectedErr

* Feedback: update todos with jira ticket

* Feedback: mock tlsConfigurator
2023-12-04 10:25:18 -05:00
aahel 7936e55807
added node health resource (#19803) 2023-12-02 11:14:03 +05:30
John Maguire a0240e3794
[NET-5688] APIGateway UI Topology Fixes (#19657)
* Update catalog and ui endpoints to show APIGateway in gateway service
topology view

* Added initial implementation for service view

* updated ui

* Fix topology view for gateways

* Adding tests for gw controller

* remove unused args

* Undo formatting changes

* Fix call sites for upstream/downstream gw changes

* Add config entry tests

* Fix function calls again

* Move from ServiceKey to ServiceName, cleanup from PR review

* Add additional check for length of services in bound apigateway for
IsSame comparison

* fix formatting for proto

* gofmt

* Add DeepCopy for retrieved BoundAPIGateway

* gofmt

* gofmt

* Rename function to be more consistent
2023-11-28 21:27:14 +00:00
Thomas Eckert 419677cc9e
[NET-6420] Add MeshConfiguration Controller stub (#19745)
* Add meshconfiguration/controller

* Add MeshConfiguration Registration function

* Fix the TODOs on the RegisterMeshGateway function

* Call RegisterMeshConfiguration

* Add comment to MeshConfigurationRegistration

* Add a test for Reconcile and some comments
2023-11-28 18:56:07 +00:00
Semir Patel 5930748cb0
resource: ListByOwner returns empty list on non-existent tenancy (#19742) 2023-11-27 14:56:08 -06:00
Ronald eded2ff347
[NET-6249] Add templated policies description (#19735) 2023-11-27 10:34:22 -05:00
Ronald c1dbf00a85
NET-6251 API gateway templated policy (#19728) 2023-11-24 17:55:05 +00:00
Poonam Jadhav 78f918a103
feat: create a default namespace (#19681)
* feat: create a default namespace on leader

* refactor: add comment and move inittenancy to leader file

* refactor: rephrase comment
2023-11-22 14:32:57 -05:00
Mike Nomitch 302f994410
[NET-6640] Adds "Policy" BindType to BindingRule (#19499)
feat: add bind type of policy

Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
2023-11-20 13:11:08 +00:00
Semir Patel 75c2def1ca
resource: preserve deferred deletion metadata on non-CAS writes (#19674) 2023-11-17 10:51:25 -06:00
Ronald ea0caa3e0f
[NET-6103] Enable query tokens by service name using templated policy (#19666) 2023-11-16 14:32:06 -05:00
John Murret 2591318c82
Skip tests with p95 greater than 30 seconds outside of main and release branches. (#19628)
Skip tests with p95 greater than 30 seconds.
2023-11-15 13:43:33 -07:00
Semir Patel 1eed205286
resource: freeze resources after marked for deletion (4 of 5) (#19603) 2023-11-15 10:58:27 -06:00
Kumar Kavish 68e7f27fd2
[NET-6438] Add tenancy to xDS Tests (#19551)
* [NET-6438] Add tenancy to xDS Tests

* [NET-6438] Add tenancy to xDS Tests
- Fixing imports

* [NET-6438] Add tenancy to xDS Tests
- Added cleanup post test run

* [NET-6356] Add tenancy to xDS Tests
- Added cleanup post test run

* [NET-6438] Add tenancy to xDS Tests
- using t.Cleanup instead of defer delete

* [NET-6438] Add tenancy to xDS Tests
- rebased

* [NET-6438] Add tenancy to xDS Tests
- rebased
2023-11-10 15:32:36 +05:30
aahel 005e1b9926
added exported svc controller (#19589)
* added exported svc controller

* added license headers
2023-11-10 07:27:53 +05:30
Nathan Coleman 40c57f10a0
NET-6391 Initialize controller for MeshGateway resource (#19552)
* Generate resource_types for MeshGateway by specifying spec option

* Register MeshGateway type w/ TODOs for hooks

* Initialize controller for MeshGateway resources

* Add meshgateway to list of v2 resource dependencies for golden test

* Scope MeshGateway resource to partition
2023-11-09 16:33:14 -05:00
John Murret 780e91688d
Migrate remaining individual resource tests for service mesh to TestAllResourcesFromSnapshot (#19583)
* migrate expose checks and paths  tests to resources_test.go

* fix failing expose paths tests

* fix the way endpoint resources get created to make expose tests pass.

* remove endpoint resources that are already inlined on local_app clusters

* renaiming and comments

* migrate remaining service mesh tests to resources_test.go

* cleanup

* update proxystateconverter to skip ading alpn to clusters and listener filterto match v1 behavior
2023-11-09 20:08:37 +00:00
Kumar Kavish f09dbb99e9
[NET-6356] Add tenancy to Failover Tests (#19547)
* [NET-6356] Add tenancy to Failover Tests

* [NET-6438] Add tenancy to xDS Tests
- Added cleanup post test run

* [NET-6356] Add tenancy to failover Tests
- using t.Cleanup instead of defer delete
2023-11-10 01:14:09 +05:30
John Murret f5bf256425
Migrate individual resource tests for API Gateway to TestAllResourcesFromSnapshot (#19584)
migrate individual api gateway tests to resources_test.go
2023-11-09 17:01:54 +00:00
John Murret a94fa4c3ed
Migrate individual resource tests for Mesh Gateway to TestAllResourcesFromSnapshot (#19502)
migrate mesh-gateway tests to resources_test.go
2023-11-09 16:39:16 +00:00
John Murret 4aa95f3d1f
Migrate individual resource tests for Ingress Gateway to TestAllResourcesFromSnapshot (#19506)
migrate ingress-gateway tests to resources_test.go
2023-11-09 16:08:07 +00:00
John Murret 2553d6e8b9
Migrate individual resource tests for Terminating Gateway to TestAllResourcesFromSnapshot (#19505)
migrate terminating-gateway tests to resources_test.go
2023-11-09 08:38:33 -07:00
John Murret 7de0b45ba4
Fix xds v2 from creating envoy endpoint resources when already inlined in the cluster (#19580)
* migrate expose checks and paths  tests to resources_test.go

* fix failing expose paths tests

* fix the way endpoint resources get created to make expose tests pass.

* wip

* remove endpoint resources that are already inlined on local_app clusters

* renaiming and comments
2023-11-08 22:18:51 +00:00
John Murret 5aff19f9bc
Migrate individual resource tests for JWT Provider to TestAllResourcesFromSnapshot (#19511)
migrate jwt provider tests to resources_test.go
2023-11-08 14:34:40 -07:00
John Murret 903ff7fccb
Migrate individual resource tests for custom configuration to TestAllResourcesFromSnapshot (#19512)
* Configure TestAllResourcesFromSnapshot to run V2 tests

* migrate custom configuration tests to resources_test.go
2023-11-08 10:34:23 -07:00
John Murret 09f73d1abf
Migrate individual resource tests for expose paths and checks to TestAllResourcesFromSnapshot (#19513)
* migrate expose checks and paths  tests to resources_test.go

* fix failing expose paths tests
2023-11-08 14:24:27 +00:00
John Murret 7bc2581c81
Migrate individual resource tests for Discovery Chains to TestAllResourcesFromSnapshot (#19508)
migrate disco chain tests to resources_test.go
2023-11-08 01:34:42 +00:00
John Murret caaff73337
add DeliverLatest as common function for use by Manager and ProxyTracker Open (#19564)
Open
add DeliverLatest as common function for use by Manager and ProxyTracker
2023-11-07 23:03:37 +00:00
Derek Menteer 393f7a429b
Fix more test flakes (#19533)
Fix flaky snapshot and metrics tests.
2023-11-07 10:15:50 -06:00
John Murret f115cdb1d5
NET-6385 - Static routes that are inlined in listener filters are also created as a resource. (#19459)
* cover all protocols in local_app golden tests

* fix xds tests

* updating latest

* fix broken test

* add sorting of routers to TestBuildLocalApp to get rid of the flaking

* cover all protocols in local_app golden tests

* cover all protocols in local_app golden tests

* cover all protocols in local_app golden tests

* process envoy resource by walking the map.  use a map rather than array for envoy resource to prevent duplication.

* cleanup.  doc strings.

* update to latest

* fix broken test

* update tests after adding sorting of routers in local_app builder tests

* do not make endpoints for local_app

* fix catalog destinations only by creating clusters for any cluster not already created by walking the graph.

* Configure TestAllResourcesFromSnapshot to run V2 tests

* wip

* fix processing of failover groups

* add endpoints and clusters for any clusters that were not created from walking the listener -> path

* fix xds v2 golden files for clusters to include failover group clusters
2023-11-07 08:00:08 -07:00
Semir Patel 2da7dd077a
v2tenancy: register tenancy controller deps (#19531) 2023-11-07 08:06:10 -06:00
Derek Menteer 6baf695cd9
[NET-6459] Fix issue with wanfed lan ip conflicts. (#19503)
Fix issue with wanfed lan ip conflicts.

Prior to this commit, the connection pools were unaware which datacenter the
connection was associated with. This meant that any time servers with
overlapping LAN IP addresses and node shortnames existed, they would be
incorrectly co-located in the same pool. Whenever this occurred, the servers
would get stuck in an infinite loop of forwarding RPCs to themselves (rather
than the intended remote DC) until they eventually run out of memory.

Most notably, this issue can occur whenever wan federation through mesh
gateways is enabled.

This fix adds extra metadata to specify which DC the connection is associated
with in the pool.
2023-11-06 08:47:12 -06:00
John Murret 74daaa5043
XDS V1 should not make runs for TCP Disco Chains. (#19496)
* XDS V1 should not make runs for TCP Disco Chains.

* update TestEnvoyExtenderWithSnapshot
2023-11-03 14:53:17 -06:00
Poonam Jadhav c3c836edae
Net-6291/fix/watch resources (#19467)
* fix: update watch endpoint to default based on scope

* test: additional test

* refactor: rename list validate function

* refactor: rename validate<Op>Request() -> ensure<Op>RequestValid() for consistency
2023-11-03 16:03:07 -04:00
Semir Patel ef35525cf1
resource: finalizer aware delete endpoint (2 of 5) (#19493)
resource: make delete endpoint finalizer aware
2023-11-03 10:10:58 -04:00
John Murret d94d316204
NET-6319 - L7 routes have statePrefix of upstream. and should have a full path (#19473) 2023-11-02 19:58:54 -06:00
Derek Menteer 8f4c43727d
[NET-5916] Fix locality-aware routing config and tests (CE) (#19483)
Fix locality-aware routing config and tests
2023-11-02 14:05:06 -05:00
John Murret 77e9a50f8b
Source / local_app golden tests to include all protocols. (#19436)
* cover all protocols in local_app golden tests

* fix xds tests

* updating latest

* fix broken test

* add sorting of routers to TestBuildLocalApp to get rid of the flaking
2023-11-02 18:31:06 +00:00
John Murret f0cf8f2f40
NET-6294 - v1 Agentless proxycfg datasource errors after v2 changes (#19365) 2023-10-27 14:06:38 -06:00
Semir Patel 0abd96c0d9
resource: resource service now checks for `v2tenancy` feature flag (#19400) 2023-10-27 08:55:02 -05:00
Poonam Jadhav 1806bcb38c
test: add missing tests for list endpoint (#19364) 2023-10-26 11:24:33 -04:00
Ashesh Vidyut 0295b959c9
Net 5875 - Create the Exported Services Resources (#19117)
* init

* computed exported service

* make proto

* exported services resource

* exported services test

* added some tests and namespace exported service

* partition exported services

* computed service

* computed services tests

* register types

* fix comment

* make proto lint

* fix proto format make proto

* make codegen

* Update proto-public/pbmulticluster/v1alpha1/computed_exported_services.proto

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>

* Update internal/multicluster/internal/types/computed_exported_services.go

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>

* using different way of resource creation in tests

* make proto

* fix computed exported services test

* fix tests

* differnet validation for computed services for ent and ce

* Acls for exported services

* added validations for enterprise features in ce

* fix error

* fix acls test

* Update internal/multicluster/internal/types/validation_exported_services_ee.go

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>

* removed the create method

* update proto

* removed namespace

* created seperate function for ce and ent

* test files updated and validations fixed

* added nil checks

* fix tests

* added comments

* removed tenancy check

* added mutation function

* fix mutation method

* fix list permissions in test

* fix pr comments

* fix tests

* lisence

* busl license

* Update internal/multicluster/internal/types/helpers_ce.go

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>

* Update internal/multicluster/internal/types/helpers_ce.go

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>

* Update internal/multicluster/internal/types/helpers_ce.go

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>

* make proto

* some pr comments addressed

* some pr comments addressed

* acls helper

* some comment changes

* removed unused files

* fixes

* fix function in file

* caps

* some positioing

* added test for validation error

* fix names

* made valid a function

* remvoed patch

* removed mutations

* v2 beta1

* v2beta1

* rmeoved v1alpha1

* validate error

* merge ent

* some nits

* removed dup func

* removed nil check

---------

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
2023-10-26 19:34:15 +05:30
Chris S. Kim 6360c745b5
Add clarification for route match behavior (#19363)
* Add clarification for route match behavior

* Update website/content/docs/connect/config-entries/service-defaults.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-10-25 20:01:46 +00:00
Michael Zalimeni a7803bd829
[NET-6305] xds: Ensure v2 route match and protocol are populated for gRPC (#19343)
* xds: Ensure v2 route match is populated for gRPC

Similar to HTTP, ensure that route match config (which is required by
Envoy) is populated when default values are used.

Because the default matches generated for gRPC contain a single empty
`GRPCRouteMatch`, and that proto does not directly support prefix-based
config, an interpretation of the empty struct is needed to generate the
same output that the `HTTPRouteMatch` is explicitly configured to
provide in internal/mesh/internal/controllers/routes/generate.go.

* xds: Ensure protocol set for gRPC resources

Add explicit protocol in `ProxyStateTemplate` builders and validate it
is always set on clusters. This ensures that HTTP filters and
`http2_protocol_options` are populated in all the necessary places for
gRPC traffic and prevents future unintended omissions of non-TCP
protocols.

Co-authored-by: John Murret <john.murret@hashicorp.com>

---------

Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-10-25 17:43:58 +00:00
Andrew Stucki e414cbee4a
Use strict DNS for mesh gateways with hostnames (#19268)
* Use strict DNS for mesh gateways with hostnames

* Add changelog
2023-10-24 15:04:14 -04:00
John Murret dc00be0ea8
NET-6081 - xds controller golden file inputs into xds resources - sources (#19250)
* NET-5397 - wire up golden tests from sidecar-proxy controller for xds controller and xdsv2

* WIP

* WIP

* everything matching except leafCerts.  need to mock those

* single port destinations working except mixed destinations

* golden test input to xds controller tests for destinations

* proposed fix for failover group naming errors

* clean up test to use helper.

* clean up test to use helper.

* fix test file

* add docstring for test function.

* add docstring for test function.

* fix linting error

* fixing test after route fix merged into main

* first source test works

* WIP

* modify all source files

* source tests pass

* fixing tests after bug fix in main

* got first destination working.

* adding destinations

* fix docstring for test

* fixing tests after bug fix in main

* adding source proxies

* fixing tests after bug fix in main

* got first destination working.

* adding destinations

* fix docstring for test

* fixing tests after bug fix in main

* got first destination working.

* adding destinations

* fix docstring for test

* fixing tests after bug fix in main
2023-10-24 17:00:30 +00:00
John Murret abbf85831f
NET-6080 - xds controller golden file inputs into xds resources - destinations (#19244)
* NET-5397 - wire up golden tests from sidecar-proxy controller for xds controller and xdsv2

* WIP

* WIP

* everything matching except leafCerts.  need to mock those

* single port destinations working except mixed destinations

* golden test input to xds controller tests for destinations

* proposed fix for failover group naming errors

* clean up test to use helper.

* clean up test to use helper.

* fix test file

* add docstring for test function.

* add docstring for test function.

* fix linting error

* fixing test after route fix merged into main

* first source test works

* WIP

* modify all source files

* source tests pass

* fixing tests after bug fix in main

* got first destination working.

* adding destinations

* fix docstring for test

* fixing tests after bug fix in main
2023-10-24 10:23:27 -06:00
Derek Menteer 48c4a5b736
Add grpc keepalive configuration. (#19339)
Prior to the introduction of this configuration, grpc keepalive messages were
sent after 2 hours of inactivity on the stream. This posed issues in various
scenarios where the server-side xds connection balancing was unaware that envoy
instances were uncleanly killed / force-closed, since the connections would
only be cleaned up after ~5 minutes of TCP timeouts occurred. Setting this
config to a 30 second interval with a 20 second timeout ensures that at most,
it should take up to 50 seconds for a dead xds connection to be closed.
2023-10-24 08:05:31 -05:00
Semir Patel 96606d114c
resource: default peername to local in list endpoints (#19340) 2023-10-23 16:30:47 -05:00
Dhia Ayachi d5c9f11b59
Tenancy Bridge v2 (#19220)
* tenancy bridge v2 for v2 resources

* add missing copywrite headers
2023-10-20 14:49:54 -04:00
aahel 1280f45485
added ent to ce downgrade changes (#19311)
* added ent to ce downgrade changes

* added changelog

* added busl headers
2023-10-20 22:34:25 +05:30
Chris S. Kim 9d00b13140
Vault CA bugfixes (#19285)
* Re-add retry logic to Vault token renewal

* Fix goroutine leak

* Add test for detecting goroutine leak

* Add changelog

* Rename tests

* Add comment
2023-10-20 15:03:27 +00:00
Michael Zalimeni 5e517c5980
[NET-6221] Ensure LB policy set for locality-aware routing (CE) (#19283)
Ensure LB policy set for locality-aware routing (CE)

`overprovisioningFactor` should be overridden with the expected value
(100,000) when there are multiple endpoint groups. Update code and
tests to enforce this.

This is an Enterprise feature. This commit represents the CE portions of
the change; tests are added in the corresponding `consul-enterprise`
change.
2023-10-19 10:13:27 -04:00
Eric Haberkorn f45be222bb
Prevent circular dependencies between v2 resources and generate a mermaid diagram with their dependencies (#19230) 2023-10-18 10:55:32 -04:00
John Maguire b78465b491
[NET-5810] CE changes for multiple virtual hosts (#19246)
CE changes for multiple virtual hosts
2023-10-17 15:08:04 +00:00
Semir Patel ad177698f7
resource: enforce lowercase v2 resource names (#19218) 2023-10-16 12:55:30 -05:00
Iryna Shustava 105ebfdd00
catalog, mesh: implement missing ACL hooks (#19143)
This change adds ACL hooks to the remaining catalog and mesh resources, excluding any computed ones. Those will for now continue using the default operator:x permissions.

It refactors a lot of the common testing functions so that they can be re-used between resources.

There are also some types that we don't yet support (e.g. virtual IPs) that this change adds ACL hooks to for future-proofing.
2023-10-13 23:16:26 +00:00
R.B. Boyer 20d1fb8c78
server: run the api checks against the path without params (#19205) 2023-10-13 15:32:06 -05:00
Dhia Ayachi 5fbf0c00d3
Add namespace read write tests (#19173) 2023-10-13 12:03:06 -04:00
Thomas Eckert 76c60fdfac
Golden File Tests for TermGW w/ Cluster Peering (#19096)
Add intention to create golden file for terminating gateway peered trust bundle
2023-10-13 11:56:58 -04:00
Nitya Dhanushkodi 95d9b2c7e4
[NET-4931] xdsv2, sidecarproxycontroller, l4 trafficpermissions: support L7 (#19185)
* xdsv2: support l7 by adding xfcc policy/headers, tweaking routes, and make a bunch of listeners l7 tests pass

* sidecarproxycontroller: add l7 local app support 

* trafficpermissions: make l4 traffic permissions work on l7 workloads

* rename route name field for consistency with l4 cluster name field

* resolve conflicts and rebase

* fix: ensure route name is used in l7 destination route name as well. previously it was only in the route names themselves, now the route name and l7 destination route name line up
2023-10-12 23:45:45 +00:00
Iryna Shustava 25283f0ec2
get-envoy-bootstrap-params: when v2 is enabled, use computed proxy configuration (#19175) 2023-10-12 14:01:36 -06:00
Chris S. Kim 197bcd4164
Refactor connect_auth.go into agent_endpoint.go (#19166) 2023-10-12 12:54:32 -04:00
John Maguire 7a323c492b
[NET-5457] Golden Files for Multiple Virtual Hosts (#19131)
* Add new golden file tests

* Update with latest deterministic code
2023-10-11 18:11:29 +00:00
John Murret 6cbd417f29
NET-5822 - Add default outbound router in TProxy (#19087)
* NET-5822 - Add default outbound router in TProxy

* fixing connection timeout to be 5 s instead of 10 seconds
2023-10-11 10:31:45 -06:00
R.B. Boyer b9ab63c55d
server: when the v2 catalog experiment is enabled reject api and rpc requests that are for the v1 catalog (#19129)
When the v2 catalog experiment is enabled the old v1 catalog apis will be
forcibly disabled at both the API (json) layer and the RPC (msgpack) layer.
This will also disable anti-entropy as it uses the v1 api.

This includes all of /v1/catalog/*, /v1/health/*, most of /v1/agent/*,
/v1/config/*, and most of /v1/internal/*.
2023-10-11 10:44:03 -05:00
John Maguire 8bebfc147d
[NET-5457] Fix CE code for jwt multiple virtual hosts bug (#19123)
* Fix CE code for jwt multiple virtual hosts bug

* Fix struct definition

* fix bug with always appending route to jwt config

* Update comment to be correct

* Update comment
2023-10-10 16:25:36 -04:00
Semir Patel 830c4ea81c
v2tenancy: cluster scoped reads (#19082) 2023-10-10 13:30:23 -05:00
Dhia Ayachi 226590541c
Activate verifier when running WAL with experimental features (#19102)
* activate verifier when running WAL with experimental features

* only change verifier parameters if it's disabled (default value)
2023-10-10 14:14:20 -04:00
Chris S. Kim 92ce814693
Remove old build tags (#19128) 2023-10-10 10:58:06 -04:00
Chris Thain dcdf2fc6ba
Update Vault CA provider namespace configuration (#19095) 2023-10-10 13:53:00 +00:00
Ashesh Vidyut a30ccdf5dc
NET-4135 - Fix NodeMeta filtering Catalog List Services API (#18322)
* logs for debugging

* Init

* white spaces fix

* added change log

* Fix tests

* fix typo

* using queryoptionfilter to populate args.filter

* tests

* fix test

* fix tests

* fix tests

* fix tests

* fix tests

* fix variable name

* fix tests

* fix tests

* fix tests

* Update .changelog/18322.txt

Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>

* fix change log

* address nits

* removed unused line

* doing join only when filter has nodemeta

* fix tests

* fix tests

* Update agent/consul/catalog_endpoint.go

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* fix tests

* removed unwanted code

---------

Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2023-10-08 12:48:31 +00:00
Derek Menteer af3439b53d
Ensure that upstream configuration is properly normalized. (#19076)
This PR fixes an issue where upstreams did not correctly inherit the proper
namespace / partition from the parent service when attempting to fetch the
upstream protocol due to inconsistent normalization.

Some of the merge-service-configuration logic would normalize to default, while
some of the proxycfg logic would normalize to match the parent service. Due to
this mismatch in logic, an incorrect service-defaults configuration entry would
be fetched and have its protocol applied to the upstream.
2023-10-06 13:59:47 -05:00
Dhia Ayachi ed882e2522
Make raft-wal default when `resource-apis` is active (#19090)
Make raft-wal default when v2 catalog experiment is on
2023-10-06 10:24:21 -04:00
Thomas Eckert 342306c312
Allow connections through Terminating Gateways from peered clusters NET-3463 (#18959)
* Add InboundPeerTrustBundle maps to Terminating Gateway

* Add notify and cancelation of watch for inbound peer trust bundles

* Pass peer trust bundles to the RBAC creation function

* Regenerate Golden Files

* add changelog, also adds another spot that needed peeredTrustBundles

* Add basic test for terminating gateway with peer trust bundle

* Add intention to cluster peered golden test

* rerun codegen

* update changelog

* really update the changelog

---------

Co-authored-by: Melisa Griffin <melisa.griffin@hashicorp.com>
2023-10-05 21:54:23 +00:00
Chris S. Kim aa526db225
Retry flaky tests (#19088) 2023-10-05 21:27:28 +00:00
Chris S. Kim ad26494016
[CE] Add workload bind type and templated policy (#19077) 2023-10-05 19:45:41 +00:00
Eric Haberkorn f2b7b4591a
Fix Traffic Permissions Default Deny (#19028)
Whenver a traffic permission exists for a given workload identity, turn on default deny.

Previously, this was only working at the port level.
2023-10-04 09:58:28 -04:00
John Murret d67e5c6e35
NET-5590 - authorization: check for identity:write in CA certs, xds server, and getting envoy bootstrap params (#19049)
* NET-5590 - authorization: check for identity:write in CA certs, xds server, and getting envoy bootstrap params

* gofmt file
2023-10-03 22:02:23 +00:00
Chris S. Kim 41e6f6cd8b
Reduce number of ports that consul test agents take (#19047) 2023-10-03 18:57:53 +00:00
sarahalsmiller 9addd9ed7c
[NET-5788] Fix needed for JWTAuth in Consul Enterprise (#19038)
change needed for fix in consul-enterprise
2023-10-03 09:48:50 -05:00
Nitya Dhanushkodi 9a48266712
remove log (#19029) 2023-09-29 16:11:50 -07:00
Chris Thain 5e45db18b7
Include RequestTimeout in marshal/unmarshal of ServiceResolverConfigE… (#19031) 2023-09-29 10:39:46 -07:00
Eric Haberkorn 7ce6ebaeb3
Handle Traffic Permissions With Empty Sources Properly (#19024)
Fix issues with empty sources

* Validate that each permission on traffic permissions resources has at least one source.
* Don't construct RBAC policies when there aren't any principals. This resulted in Envoy rejecting xDS updates with a validation error.

```
error=
  | rpc error: code = Internal desc = Error adding/updating listener(s) public_listener: Proto constraint validation failed (RBACValidationError.Rules: embedded message failed validation | caused by RBACValidationError.Policies[consul-intentions-layer4-1]: embedded message failed validation | caused by PolicyValidationError.Principals: value must contain at least 1 item(s)): rules {
```
2023-09-28 15:11:59 -04:00
Iryna Shustava 06c15d0656
auth: register auth controllers with the server (#19000) 2023-09-25 22:54:03 +00:00
Iryna Shustava e6b724d062
catalog,mesh,auth: Move resource types to the proto-public module (#18935) 2023-09-22 15:50:56 -06:00
R.B. Boyer 7688178ad2
peerstream: fix flaky test related to autopilot integration (#18979) 2023-09-22 13:12:00 -05:00
Iryna Shustava d88888ee8b
catalog,mesh,auth: Bump versions to v2beta1 (#18930) 2023-09-22 10:51:15 -06:00
R.B. Boyer ef6f2494c7
resource: allow for the ACLs.Read hook to request the entire data payload to perform the authz check (#18925)
The ACLs.Read hook for a resource only allows for the identity of a 
resource to be passed in for use in authz consideration. For some 
resources we wish to allow for the current stored value to dictate how 
to enforce the ACLs (such as reading a list of applicable services from 
the payload and allowing service:read on any of them to control reading the enclosing resource).

This change update the interface to usually accept a *pbresource.ID, 
but if the hook decides it needs more data it returns a sentinel error 
and the resource service knows to defer the authz check until after
 fetching the data from storage.
2023-09-22 09:53:55 -05:00
Nitya Dhanushkodi 0a11499588
net-5689 fix disabling panic threshold logic (#18958) 2023-09-21 15:52:30 -07:00
Chris S. Kim 565e79344f
Dump response body on fail (#18962) 2023-09-21 21:10:53 +00:00
Ronald 276c60a947
skip flaky test (#18949) 2023-09-21 14:25:12 -04:00
John Landa 9eaa8eb026
dns token (#17936)
* dns token

fix whitespace for docs and comments

fix test cases

fix test cases

remove tabs in help text

Add changelog

Peering dns test

Peering dns test

Partial implementation of Peered DNS test

Swap to new topology lib

expose dns port for integration tests on client

remove partial test implementation

remove extra port exposure

remove changelog from the ent pr

Add dns token to set-agent-token switch

Add enterprise golden file

Use builtin/dns template in tests

Update ent dns policy

Update ent dns template test

remove local gen certs

fix templated policy specs

* add changelog

* go mod tidy
2023-09-20 15:50:06 -06:00
Dhia Ayachi 341dc28ff9
Add namespace proto and registration (#18848)
* add namespace proto and registration

* fix proto generation

* add missing copywrite headers

* fix proto linter errors

* fix exports and Type export

* add mutate hook and more validation

* add more validation rules and tests

* Apply suggestions from code review

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* fix owner error and add test

* remove ACL for now

* add tests around space suffix prefix.

* only fait when ns and ap are default, add test for it

---------

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
2023-09-20 15:20:20 -04:00
R.B. Boyer d574473fd1
mesh: make FailoverPolicy work in xdsv2 and ProxyStateTemplate (#18900)
Ensure that configuring a FailoverPolicy for a service that is reachable via a xRoute or a direct upstream causes an envoy aggregate cluster to be created for the original cluster name, but with separate clusters for each one of the possible destinations.
2023-09-20 11:59:01 -05:00
Ronald c8299522b5
[NET-5332] Add nomad server templated policy (#18888)
* [NET-5332] Add nomad server templated policy

* slksfd
2023-09-20 12:10:55 -04:00
Nitya Dhanushkodi 3a2e62053a
v2: various fixes to make K8s tproxy multiport acceptance tests and manual explicit upstreams (single port) tests pass (#18874)
Adding coauthors who mobbed/paired at various points throughout last week.
Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: Michael Wilkerson <mwilkerson@hashicorp.com>
2023-09-20 00:02:01 +00:00
Nick Ethier 1a3081ab32
agent/config: prevent startup if resource-apis experiment and cloud are enabled (#18876) 2023-09-19 19:50:45 -04:00
Blake Covarrubias 019c62e1ba
xds: Use downstream protocol when connecting to local app (#18573)
Configure Envoy to use the same HTTP protocol version used by the
downstream caller when forwarding requests to a local application that
is configured with the protocol set to either `http2` or `grpc`.

This allows upstream applications that support both HTTP/1.1 and
HTTP/2 on a single port to receive requests using either protocol. This
is beneficial when the application primarily communicates using HTTP/2,
but also needs to support HTTP/1.1, such as to respond to Kubernetes
HTTP readiness/liveness probes.

Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>
2023-09-19 14:32:28 -07:00
Eric Haberkorn 170417ac97
Honor Default Traffic Permissions in V2 (#18886)
wire up v2 default traffic permissions
2023-09-19 10:42:32 -04:00
Iryna Shustava 212793a4ee
mesh: only build tproxy outbound listener once per destination (#18836)
Previously, when using implicit upstreams, we'd build outbound listener per destination instead of one for all destinations. This will result in port conflicts when trying to send this config to envoy.

This PR also makes sure that leaf and root references are always added (before we would only add it if there are inbound non-mesh ports).

Also, black-hole traffic when there are no inbound ports other than mesh
2023-09-18 18:26:13 -06:00
Chris S. Kim 91e6c3a82f
Remove flaky test assertions (#18870) 2023-09-18 15:56:23 -07:00
Semir Patel 62796a1454
resource: mutate and validate before acls on write (#18868) 2023-09-18 17:04:29 -05:00
Ronald 49cb84297f
Move ACL templated policies to hcl files (#18853) 2023-09-18 17:10:35 -04:00
Andrew Stucki 087539fc7b
Fix gateway services cleanup where proxy deregistration happens after service deregistration (#18831)
* Fix gateway services cleanup where proxy deregistration happens after service deregistration

* Add test

* Add changelog

* Fix comment
2023-09-18 16:19:17 -04:00
Dhia Ayachi 4435e4a420
add v2 tenancy bridge Flag and v2 Tenancy Bridge initial implementation (#18830)
* add v2 tenancy bridge and a feature flag for v2 tenancy

* move tenancy bridge v2 under resource package
2023-09-18 12:25:05 -04:00
Chris S. Kim 461549e304
Adjust metrics test (#18837) 2023-09-15 23:15:42 +00:00
skpratt 1fda2965e8
Allow empty data writes for resources (#18819)
* allow nil data writes for resources

* update demo to test valid type with no data
2023-09-15 14:00:23 -05:00
Ronald aff13cd4c2
Use embedded strings for templated policies (#18829) 2023-09-15 13:49:22 -04:00
Eric Haberkorn 21fdbbabbc
Wire up traffic permissions (#18812)
Wire up traffic permissions
2023-09-15 12:31:22 -04:00
Semir Patel d3dad14030
resource: default peername to "local" for now (#18822) 2023-09-15 09:34:18 -05:00
R.B. Boyer 66e1cdf40c
mesh: Wire ComputedRoutes into the ProxyStateTemplate via the sidecar controller (#18752)
Reworks the sidecar controller to accept ComputedRoutes as an input and use it to generate appropriate ProxyStateTemplate resources containing L4/L7 mesh configuration.
2023-09-14 17:19:04 -05:00
Ronald 1afeb6e040
[NET-5334] Added CLI commands for templated policies (#18816) 2023-09-14 20:14:55 +00:00
Ronald 802122640b
[NET-5329] use acl templated policy under the hood for node/service identities (#18813) 2023-09-14 14:36:34 -04:00
skpratt e5808d85f7
register traffic permission and workload identity types (#18704)
* add workload identity and traffic permission protos

* register new types

* add generated pb code

* fix exports.go path

* add proto newlines

* fix type name

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>

* address review

* fix protos and add tests

* fix validation constraints

* add tests

---------

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
2023-09-14 12:40:54 -05:00
Chris S. Kim d667cc3809
Fix flaky test (#18805) 2023-09-14 16:46:47 +00:00
Chris S. Kim 6748fac43d
Clean up resources in test (#18799) 2023-09-14 15:33:19 +00:00
Derek Menteer 02259ef964
Fix snapshot creation issue. (#18783)
The renaming of files from oss -> ce caused incorrect snapshots
to be created due to ce writes now happening prior to ent writes.
When this happens various entities will attempt to be restored
from the snapshot prior to a partition existing and will cause a
panic to occur.
2023-09-13 15:35:31 -05:00
Chris S. Kim 4dfca64ded
Vault CA provider clean up previous default issuers (#18773) 2023-09-13 19:33:02 +00:00
Eric Haberkorn 12be06f8e5
Add V2 TCP traffic permissions (#18771)
Add support for TCP traffic permissions
2023-09-13 09:03:42 -04:00
Chris S. Kim d090668c37
Add workload identity ACL rules (#18769) 2023-09-12 17:22:51 -04:00
Nitya Dhanushkodi 78b170ad50
xds controller: setup watches for and compute leaf cert references in ProxyStateTemplate, and wire up leaf cert manager dependency (#18756)
* Refactors the leafcert package to not have a dependency on agent/consul and agent/cache to avoid import cycles. This way the xds controller can just import the leafcert package to use the leafcert manager.

The leaf cert logic in the controller:
* Sets up watches for leaf certs that are referenced in the ProxyStateTemplate (which generates the leaf certs too).
* Gets the leaf cert from the leaf cert cache
* Stores the leaf cert in the ProxyState that's pushed to xds
* For the cert watches, this PR also uses a bimapper + a thin wrapper to map leaf cert events to related ProxyStateTemplates

Since bimapper uses a resource.Reference or resource.ID to map between two resource types, I've created an internal type for a leaf certificate to use for the resource.Reference, since it's not a v2 resource.
The wrapper allows mapping events to resources (as opposed to mapping resources to resources)

The controller tests:
Unit: Ensure that we resolve leaf cert references
Lifecycle: Ensure that when the CA is updated, the leaf cert is as well

Also adds a new spiffe id type, and adds workload identity and workload identity URI to leaf certs. This is so certs are generated with the new workload identity based SPIFFE id.

* Pulls out some leaf cert test helpers into a helpers file so it
can be used in the xds controller tests.
* Wires up leaf cert manager dependency
* Support getting token from proxytracker
* Add workload identity spiffe id type to the authorize and sign functions



---------

Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-09-12 12:56:43 -07:00
Chris Thain 4724a4e169
Add Envoy golden test for OTEL access logging extension (#18760) 2023-09-12 09:58:53 -07:00
John Murret 62062fd4fd
NET-5132 - Configure multiport routing for connect proxies in TProxy mode (#18606)
* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Address PR comments

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* PR review comments

* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Make sure endpoint refs route to mesh port instead of an app port

* Address PR comments

* fixing copyright

* tidy imports

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* tidy imports

* add copyright headers

* Prefix sidecar proxy test files with source and destination.

* Update controller_test.go

* NET-5132 - Configure multiport routing for connect proxies in TProxy mode

* formatting golden files

* reverting golden files and adding changes in manually.  build implicit destinations still has some issues.

* fixing files that were incorrectly repeating the outbound listener

* PR comments

* extract AlpnProtocol naming convention to getAlpnProtocolFromPortName(portName)

* removing address level filtering.

* adding license to resources_test.go

---------

Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
2023-09-12 01:17:56 +00:00
Ronald 9776c10efb
[NET-5333] Add api to read/list and preview templated policies (#18748) 2023-09-11 18:11:37 +00:00
Dhia Ayachi b1688ad856
Run copyright after running deep-copy as part of the Makefile/CI (#18741)
* execute copyright headers after performing deep-copy generation.

* fix copyright install

* Apply suggestions from code review

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* Apply suggestions from code review

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* rename steps to match codegen naming

* remove copywrite install category

---------

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
2023-09-11 13:50:52 -04:00
R.B. Boyer a69e901660
xds: update golden tests to be deterministic (#18707) 2023-09-11 11:40:19 -05:00
Iryna Shustava 1557e1d6a3
sidecar-proxy controller: Add support for transparent proxy (NET-5069) (#18458)
This commit adds support for transparent proxy to the sidecar proxy controller. As we do not yet support inferring destinations from intentions, this assumes that all services in the cluster are destinations.
2023-09-08 16:18:01 -06:00
Nathan Coleman ed79c60e78
NET-5530 Generate deep-copy code (#18730)
* Generate deep-copy code

* Undo license header removal
2023-09-08 20:09:44 +00:00
Ronald 40d7ebc318
[NET-5330] Support templated policies in Binding rules (#18719)
* [NET-5330] Support templated policies in Binding rules

* changelog for templated policy support in binding rules
2023-09-08 14:39:09 -04:00
Semir Patel 576ffdf705
fix: emit consul version metric on a regular interval (#18724) 2023-09-08 13:09:07 -05:00
Nathan Coleman e5d26a13cd
NET-5530 Support response header modifiers on http-route config entry (#18646)
* Add response header filters to http-route config entry definitions

* Map response header filters from config entry when constructing route destination

* Support response header modifiers at the service level as well

* Update protobuf definitions

* Update existing unit tests

* Add response filters to route consolidation logic

* Make existing unit tests more robust

* Add missing docstring

* Add changelog entry

* Add response filter modifiers to existing integration test

* Add more robust testing for response header modifiers in the discovery chain

* Add more robust testing for request header modifiers in the discovery chain

* Modify test to verify that service filter modifiers take precedence over rule filter modifiers
2023-09-08 14:04:56 -04:00
Jeremy Jacobson 876c662e36
[CC-6039] Update builtin policy descriptions (#18705) 2023-09-08 09:16:22 -07:00
Iryna Shustava bbc2763b9f
Instantiate secure resource service client after the grpc server (#18712) 2023-09-08 09:48:49 -06:00
Ronald bbef879f85
[NET-5325] ACL templated policies support in tokens and roles (#18708)
* [NET-5325] ACL templated policies support in tokens and roles
- Add API support for creating tokens/roles with templated-policies
- Add CLI support for creating tokens/roles with templated-policies

* adding changelog
2023-09-08 12:45:24 +00:00
Gerard Nguyen 56d6e54ac7
fix: NET-1521 show latest config in /v1/agent/self (#18681)
* fix: NET-1521 show latest config in /v1/agent/self
2023-09-08 09:47:31 +10:00
John Maguire 2c244b6f42
[APIGW] NET-5017 JWT Cleanup/Status Conditions (#18700)
* Fixes issues in setting status

* Update golden files for changes to xds generation to not use deprecated
methods

* Fixed default for validation of JWT for route
2023-09-07 19:03:09 +00:00
Iryna Shustava 3c70e14713
sidecar-proxy controller: L4 controller with explicit upstreams (NET-3988) (#18352)
* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.
* This commit also changes service endpoints to include workload identity. This made the implementation a bit easier as we don't need to look up as many workloads and instead rely on endpoints data.
2023-09-07 09:37:15 -06:00
Iryna Shustava 4eb2197e82
dataplane: Allow getting bootstrap parameters when using V2 APIs (#18504)
This PR enables the GetEnvoyBootstrapParams endpoint to construct envoy bootstrap parameters from v2 catalog and mesh resources.

   * Make bootstrap request and response parameters less specific to services so that we can re-use them for workloads or service instances.
   * Remove ServiceKind from bootstrap params response. This value was unused previously and is not needed for V2.
   * Make access logs generation generic so that we can generate them using v1 or v2 resources.
2023-09-06 16:46:25 -06:00
Derek Menteer 56917eb4c9
Add support for querying tokens by service name. (#18667)
Add support for querying tokens by service name

The consul-k8s endpoints controller has a workflow where it fetches all tokens.
This is not performant for large clusters, where there may be a sizable number
of tokens. This commit attempts to alleviate that problem and introduces a new
way to query by the token's service name.
2023-09-06 10:52:45 -05:00
Phil Porada 7ea986783d
Add TCP+TLS Healthchecks (#18381)
* Begin adding TCPUseTLS

* More TCP with TLS plumbing

* Making forward progress

* Keep on adding TCP+TLS support for healthchecks

* Removed too many lines

* Unit tests for TCP+TLS

* Update tlsutil/config.go

Co-authored-by: Samantha <hello@entropy.cat>

* Working on the tcp+tls unit test

* Updated the runtime integration tests

* Progress

* Revert this file back to HEAD

* Remove debugging lines

* Implement TLS enabled TCP socket server and make a successful TCP+TLS healthcheck on it

* Update docs

* Update agent/agent_test.go

Co-authored-by: Samantha <hello@entropy.cat>

* Update website/content/docs/ecs/configuration-reference.mdx

Co-authored-by: Samantha <hello@entropy.cat>

* Update website/content/docs/ecs/configuration-reference.mdx

Co-authored-by: Samantha <hello@entropy.cat>

* Update agent/checks/check.go

Co-authored-by: Samantha <hello@entropy.cat>

* Address comments

* Remove extraneous bracket

* Update agent/agent_test.go

Co-authored-by: Samantha <hello@entropy.cat>

* Update agent/agent_test.go

Co-authored-by: Samantha <hello@entropy.cat>

* Update website/content/docs/ecs/configuration-reference.mdx

Co-authored-by: Samantha <hello@entropy.cat>

* Update the mockTLSServer

* Remove trailing newline

* Address comments

* Fix merge problem

* Add changelog entry

---------

Co-authored-by: Samantha <hello@entropy.cat>
2023-09-05 13:34:44 -07:00
Derek Menteer a698142325
Add extra logging for mesh health endpoints. (#18647) 2023-09-01 12:29:09 -05:00
Derek Menteer b56fbc7a62
[NET-4958] Fix issue where envoy endpoints would fail to populate after snapshot restore (#18636)
Fix issue where agentless endpoints would fail to populate after snapshot restore.

Fixes an issue that was introduced in #17775. This issue happens because
a long-lived pointer to the state store is held, which is unsafe to do.
Snapshot restorations will swap out this state store, meaning that the
proxycfg watches would break for agentless.
2023-09-01 10:18:10 -05:00
Semir Patel b96cff7436
resource: Require scope for resource registration (#18635) 2023-09-01 09:44:53 -05:00
John Maguire 9876923e23
Add the plumbing for APIGW JWT work (#18609)
* Add the plumbing for APIGW JWT work

* Remove unneeded import

* Add deep equal function for HTTPMatch

* Added plumbing for status conditions

* Remove unneeded comment

* Fix comments

* Add calls in xds listener for apigateway to setup listener jwt auth
2023-08-31 12:23:59 -04:00
Semir Patel 7b9e243297
resource: Allow nil tenancy (#18618) 2023-08-31 09:24:09 -05:00
Dhia Ayachi f8d77f027a
delete all v2 resources type when deleting a namespace (CE) (#18621)
* add namespace scope to ServiceV1Alpha1Type

* add CE portion of namespace deletion
2023-08-31 10:18:25 -04:00
Ashvitha 0f48b7af5e
[HCP Telemetry] Move first TelemetryConfig Fetch into the TelemetryConfigProvider (#18318)
* Add Enabler interface to turn sink on/off

* Use h for hcpProviderImpl vars, fix PR feeback and fix errors

* Keep nil check in exporter and fix tests

* Clarify comment and fix function name

* Use disable instead of enable

* Fix errors nit in otlp_transform

* Add test for refreshInterval of updateConfig

* Add disabled field in MetricsConfig struct

* Fix PR feedback: improve comment and remove double colons

* Fix deps test which requires a maybe

* Update hcp-sdk-go to v0.61.0

* use disabled flag in telemetry_config.go

* Handle 4XX errors in telemetry_provider

* Fix deps test

* Check 4XX instead

* Run make go-mod-tidy
2023-08-30 13:25:26 -04:00
Hardik Shingala 58e5658810
Added OpenTelemetry Access Logging Envoy extension (#18336) 2023-08-30 07:51:58 -07:00
Ashwin Venkatesh 797e42dc24
Watch the ProxyTracker from xDS controller (#18611) 2023-08-29 14:39:29 -07:00
John Murret 0e606504bc
NET-4944 - wire up controllers with proxy tracker (#18603)
Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
2023-08-29 09:15:34 -06:00
Joshua Timmons 48c8a834f5
Reduce the frequency of metric exports to minutely (#18584) 2023-08-28 17:49:34 +00:00
Chris S. Kim ecdcde4309
CE commit (#18583) 2023-08-25 12:47:20 -04:00
John Murret 051f250edb
NET-5338 - NET-5338 - Run a v2 mode xds server (#18579)
* NET-5338 - NET-5338 - Run a v2 mode xds server

* fix linting
2023-08-24 16:44:14 -06:00
Semir Patel 2225bf0550
resource: Make resource writestatus tenancy aware (#18577) 2023-08-24 19:18:47 +00:00
John Maguire 59ab57f350
NET-5147: Added placeholder structs for JWT functionality (#18575)
* Added placeholder structs for JWT functionality

* Added watches for CE vs ENT

* Add license header

* Undo plumbing work

* Add context arg
2023-08-24 15:07:14 -04:00
Semir Patel 067a0112e2
resource: Make resource listbyowner tenancy aware (#18566) 2023-08-24 10:49:46 -05:00
Chris S. Kim 82993fcc4f
CE port of enterprise extension (#18572)
CE commit
2023-08-24 15:43:26 +00:00
cskh b37587bb2c
bug: prevent go routine leakage due to existing DeferCheck (#18558)
* bug: prevent go routine leakage due to existing DeferCheck

* add changelog
2023-08-23 10:33:07 -04:00
R.B. Boyer 8a931241f2
chore: fix missing/incorrect license headers (#18555) 2023-08-22 17:23:54 -05:00
Ashwin Venkatesh 4f9955d91e
Update trust bundle into proxy-state-template (#18550) 2023-08-22 19:38:31 +00:00
Semir Patel 53e28a4963
OSS -> CE (community edition) changes (#18517) 2023-08-22 09:46:03 -05:00
Semir Patel 6d22179625
resource: Make resource watchlist tenancy aware (#18539) 2023-08-21 15:02:23 -05:00
John Murret 217d305b38
NET-4943 - Implement ProxyTracker (#18535) 2023-08-21 14:08:13 -04:00
John Murret 9ea182f6ad
NET-4858 - xds v2 - implement base connect proxy functionality for routes (#18501)
* NET-4853 - xds v2 - implement base connect proxy functionality for clusters

* NET-4853 - xds v2 - implement base connect proxy functionality for clusters

* NET-4932 - xds v2 - implement base connect proxy functionality for endpoints

* Update endpoints_test.go

* gofmt

* NET-4858 - Make connect proxy route tests pass using xds v2

* Update endpoints_test.go

* Update naming.go

* use alsoRunTestForV2

* remove unused makeAddress

* gofmt

* fixing clusters
2023-08-17 21:04:53 +00:00
John Murret 92cfb4a07e
NET-4932 - xds v2 - implement base connect proxy functionality for endpoints (#18500)
* NET-4853 - xds v2 - implement base connect proxy functionality for clusters

* NET-4853 - xds v2 - implement base connect proxy functionality for clusters

* NET-4932 - xds v2 - implement base connect proxy functionality for endpoints

* Update endpoints_test.go

* gofmt

* Update naming.go
2023-08-17 19:55:54 +00:00
John Murret b80c5258fa
NET-4853 - xds v2 - implement base connect proxy functionality for clusters (#18499) 2023-08-17 14:43:21 -04:00
Semir Patel e6c1c479b7
resource: Make resource delete tenancy aware (#18476)
resource: Make resource delete tenancy awarae
2023-08-16 11:44:10 -05:00
Semir Patel 217107f627
resource: Make resource list tenancy aware (#18475) 2023-08-15 16:57:59 -05:00
Nitya Dhanushkodi 6b7ccd06cf
[NET-4799] [OSS] xdsv2: listeners L4 support for connect proxies (#18436)
* refactor to avoid future import cycles
2023-08-15 11:57:07 -07:00
hashicorp-copywrite[bot] 5fb9df1640
[COMPLIANCE] License changes (#18443)
* Adding explicit MPL license for sub-package

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Adding explicit MPL license for sub-package

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Updating the license from MPL to Business Source License

Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.

* add missing license headers

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

---------

Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
2023-08-11 09:12:13 -04:00
John Maguire df11e4e7b4
APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422)
* Updated httproute config entry for JWT Filters

* Added manual deepcopy method for httproute jwt filter

* Fix test

* Update JWTFilter to be in oss file

* Add changelog

* Add build tags for deepcopy oss file
2023-08-10 21:23:42 +00:00
John Maguire 6c8ca0f89d
NET-4984: Update APIGW Config Entries for JWT Auth (#18366)
* Added oss config entries for Policy and JWT on APIGW

* Updated structs for config entry

* Updated comments, ran deep-copy

* Move JWT configuration into OSS file

* Add in the config entry OSS file for jwts

* Added changelog

* fixing proto spacing

* Moved to using manually written deep copy method

* Use pointers for override/default fields in apigw config entries

* Run gen scripts for changed types
2023-08-10 19:49:51 +00:00
Michael Zalimeni 05604eeec1
[NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437)
* Add logging to locality policy application

In OSS, this is currently a no-op.

* Inherit locality when registering sidecars

When sidecar locality is not explicitly configured, inherit locality
from the proxied service.
2023-08-10 14:00:44 -04:00
Semir Patel bee12c6b1f
resource: Make resource write tenancy aware (#18423) 2023-08-10 09:53:38 -05:00
wangxinyi7 facd5b0ec1
fix the error in ent repo (#18421)
fix the error in ent repo
2023-08-09 09:36:58 -07:00
sarahalsmiller e235c8be3c
NET-5115 Add retry + timeout filters for api-gateway (#18324)
* squash, implement retry/timeout in consul core

* update tests
2023-08-08 16:39:46 -05:00
cskh 43d8898e08
bump testcontainers-go from 0.22.0 and remove pinned go version in in… (#18395)
* bump testcontainers-go from 0.22.0 and remove pinned go version in integ test

* go mod tidy

* Replace deprecated target.Authority with target.URL.Host
2023-08-08 18:08:14 +00:00
Semir Patel 63cc037110
resource: Make resource read tenancy aware (#18397) 2023-08-07 16:37:03 -05:00
Ashesh Vidyut 417ae9fc39
Fix #17730 - Dev mode has new line (#18367)
* adding new line only in case of pretty in url not in dev mode

* change log added
2023-08-05 08:15:24 +05:30
wangxinyi7 1f28ac2664
expose grpc as http endpoint (#18221)
expose resource grpc endpoints as http endpoints
2023-08-04 11:27:48 -07:00
Jeremy Jacobson 8e5e16de60
Fix policy lookup to allow for slashes (#18347)
* Fix policy lookup to allow for slashes

* Fix suggestions

* Fix other test

* Revert some lines
2023-08-03 13:21:43 -07:00