Commit Graph

21500 Commits

Author SHA1 Message Date
Mark Campbell-Vincent 27f6fef999
Update cache section for certain API calls (#20464)
* Update cache section for certain API calls

Providing more detail to the cache section to address behavior of API calls using streaming backend. This will help users understand that when '?index' is used and '?cached' is not, caching to servers will be bypassed, causing entry_fetch_max_burst and entry_fetch_rate to not be used in this scenario.

* Update website/content/docs/agent/config/config-files.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

---------

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2024-02-07 21:09:26 -05:00
Semir Patel 3c1ee84d0a
fix typos in docs for `consul snapshot agent` (#20522) 2024-02-07 17:09:34 -06:00
Melissa Kam c95380e3da
[CC-7434] Skip collecting data directory metrics in dev mode (#20521)
* Skip collecting data directory stats in dev mode

In dev mode, the data directory is not set, so this metrics collection would
always fail and logs errors.

* Log collection errors at DEBUG level

There isn't much action a user can take to fix these errors, so
logging them as DEBUG rather than ERROR.
2024-02-07 16:59:06 -06:00
John Maguire 2ee32b1980
Add tests for gw proxy controller (#20510)
* Added basic tests for gatewayproxy controller

* add copyright header

* Clean up tests
2024-02-07 17:01:10 -05:00
Nathan Coleman 2456fe5148
[NET-7657] Remove proto definitions for GatewayClass + GatewayClassConfig (#20523) 2024-02-07 16:53:39 -05:00
Nathan Coleman 45d645471b
[NET-7414] Reconcile PST for mesh gateway workloads on change to ComputedExportedServices (#20271)
* Reconcile ProxyStateTemplate on change to ComputedExportedServices

* gofmt changeset

---------

Co-authored-by: NiniOak <anita.akaeze@hashicorp.com>
2024-02-07 21:27:13 +00:00
skpratt 57bad0df85
add traffic permissions excludes and tests (#20453)
* add traffic permissions tests

* review fixes

* Update internal/mesh/internal/controllers/sidecarproxy/builder/local_app.go

Co-authored-by: John Landa <jonathanlanda@gmail.com>

---------

Co-authored-by: John Landa <jonathanlanda@gmail.com>
2024-02-07 20:21:44 +00:00
Eric Haberkorn 1bd253021b
V1 Compat Exported Services Controller Optimizations (#20517)
V1 compat exported services controller optimizations

* Don't start the v2 exported services controller in v1 mode.
* Use the controller cache.
2024-02-07 14:05:42 -05:00
Matt Keeler 3ca4f39fa1
Register the multicluster types for the catalogtest integration tests (#20516)
In particular the failover controller needs these in Consul Enterprise
2024-02-07 13:35:02 -05:00
wangxinyi7 ab8f23478a
add more integration tests (#20479)
* add more integration tests
2024-02-06 11:00:59 -08:00
John Maguire 24e9603d9b
Fix Gatewayproxy Controller and Re-Enable APIGW v2 Controller (#20508)
re-enable apigw controller, fix typo in key name for metadata for
gatewayproxy
2024-02-06 18:55:55 +00:00
Matt Keeler 49e6c0232d
Panic for unregistered types (#20476)
* Panic when controllers attempt to make invalid requests to the resource service

This will help to catch bugs in tests that could cause infinite errors to be emitted.

* Disable the API GW v2 controller

With the previous commit, this would cause a server to panic due to watching a type which has not yet been created/registered.

* Ensure that a test server gets the full type registry instead of constructing its own

* Skip TestServer_ControllerDependencies

* Fix peering tests so that they use the full resource registry.
2024-02-06 11:23:06 -05:00
Dan Stough fcc43a9a36
feat(v2dns): catalog v2 SOA and NS support (#20480) 2024-02-06 11:12:04 -05:00
John Maguire 54c974748e
[NET-7280] Add APIGW support to the gatewayproxy controller (#20484)
* Add APIGW support to the gatewayproxy controller

* update copywrite headers
2024-02-06 11:03:37 -05:00
John Murret 3bf999e46b
NET-7631 - Fix Node records that point to external/ non-IP addresses (#20491)
* NET-7630 - Fix TXT record creation on node queries

* NET-7631 - Fix Node records that point to external/ non-IP addresses

* NET-7630 - Fix TXT record creation on node queries
2024-02-06 15:16:02 +00:00
John Murret 7d4deda640
NET-7630 - Fix TXT record creation on node queries (#20483) 2024-02-06 09:53:39 -05:00
Ashesh Vidyut cffb5d7c6e
Fix audit-log encoding issue (CC-7337) (#20345)
* add changes

* added changelog

* change update

* CE chnages

* Removed gzip size fix

* fix changelog

* Update .changelog/20345.txt

Co-authored-by: Hans Hasselberg <hans@hashicorp.com>

* Adding comments

---------

Co-authored-by: Abhishek Sahu <abhishek.sahu@hashicorp.com>
Co-authored-by: Hans Hasselberg <hans@hashicorp.com>
Co-authored-by: srahul3 <rahulsharma@hashicorp.com>
2024-02-06 16:40:07 +05:30
Tauhid Anjum 88b8a1cc36
NET-6776 - Update Routes controller to use ComputedFailoverPolicy CE (#20496)
Update Routes controller to use ComputedFailoverPolicy
2024-02-06 13:28:18 +05:30
Tauhid Anjum 0c509a60a4
Exported services CLI and docs (#20331)
* Exported services CLI and docs

* Changelog added

* Added format option for pretty print

* Update command/exportedservices/exported_services.go

Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com>

* Addressing PR comments, moving the command under services category

* Add consumer peer and partition filter

* Adding bexpr filter, change format of data

---------

Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com>
2024-02-06 09:01:20 +05:30
Derek Menteer 922844b8e0
Fix issue with persisting proxy-defaults (#20481)
Fix issue with persisting proxy-defaults

This resolves an issue introduced in hashicorp/consul#19829
where the proxy-defaults configuration entry with an HTTP protocol
cannot be updated after it has been persisted once and a router
exists. This occurs because the protocol field is not properly
pre-computed before being passed into validation functions.
2024-02-05 16:00:19 -06:00
John Murret 0d434dafac
Do not parallelize DNS tests because they consume too many ports (#20482) 2024-02-05 14:54:05 -07:00
John Murret 602e3c4fd5
DNS V2 - Revise discovery result to have service and node name and address fields. (#20468)
* DNS V2 - Revise discovery result to have service and node name and address fields.

* NET-7488 - dns v2 add support for prepared queries in catalog v1 data model (#20470)

NET-7488 - dns v2 add support for prepared queries in catalog v1 data model.
2024-02-03 03:23:52 +00:00
Dan Stough 9602b43183
feat(v2dns): catalog v2 workload query support (#20466) 2024-02-02 18:29:38 -05:00
R.B. Boyer deca6a49bd
catalog: improve the bound workload identity encoding on services (#20458)
The endpoints controller currently encodes the list of unique workload identities 
referenced by all workload matched by a Service into a special data-bearing 
status condition on that Service. This allows a downstream controller to avoid an 
expensive watch on the ServiceEndpoints type just to get this data.

The current encoding does not lend itself well to machine parsing, which is what 
the field is meant for, so this PR simplifies the encoding from:

    "blah blah: " + strings.Join(ids, ",") + "."

to

    strings.Join(ids, ",")

It also provides an exported utility function to easily extract this data.
2024-02-02 16:28:39 -06:00
Nick Ethier 9d4ad74a63
internal/hcp: prevent write loop on telemetrystate resource updates (#20435)
* internal/hcp: prevent write loop on telemetrystate resource updates

* Update controller.go

Co-authored-by: Nick Cellino <nick.cellino@hashicorp.com>

* internal/hcp: add assertion for looping controller

---------

Co-authored-by: Nick Cellino <nick.cellino@hashicorp.com>
2024-02-02 16:28:20 -05:00
R.B. Boyer c029b20615
v2: ensure the controller caches are fully populated before first use (#20421)
The new controller caches are initialized before the DependencyMappers or the 
Reconciler run, but importantly they are not populated. The expectation is that 
when the WatchList call is made to the resource service it will send an initial 
snapshot of all resources matching a single type, and then perpetually send 
UPSERT/DELETE events afterward. This initial snapshot will cycle through the 
caching layer and will catch it up to reflect the stored data.

Critically the dependency mappers and reconcilers will race against the restoration 
of the caches on server startup or leader election. During this time it is possible a
 mapper or reconciler will use the cache to lookup a specific relationship and 
not find it. That very same reconciler may choose to then recompute some 
persisted resource and in effect rewind it to a prior computed state.

Change

- Since we are updating the behavior of the WatchList RPC, it was aligned to 
  match that of pbsubscribe and pbpeerstream using a protobuf oneof instead of the enum+fields option.

- The WatchList rpc now has 3 alternating response events: Upsert, Delete, 
  EndOfSnapshot. When set the initial batch of "snapshot" Upserts sent on a new 
  watch, those operations will be followed by an EndOfSnapshot event before beginning 
  the never-ending sequence of Upsert/Delete events.

- Within the Controller startup code we will launch N+1 goroutines to execute WatchList 
  queries for the watched types. The UPSERTs will be applied to the nascent cache
   only (no mappers will execute).

- Upon witnessing the END operation, those goroutines will terminate.

- When all cache priming routines complete, then the normal set of N+1 long lived 
watch routines will launch to officially witness all events in the system using the 
primed cached.
2024-02-02 15:11:05 -06:00
Derek Menteer 266f6548f9
Debug failing CICD tests (#20455)
Fix CICD test flakes by locking container socket.
2024-02-02 15:05:10 -06:00
wangxinyi7 fb2b696c0e
missing prefix / (#20447)
* missing prefix / and fix typos
2024-02-02 12:48:45 -08:00
Eric Haberkorn 543c6a30af
Trigger the V1 Compat exported-services Controller when V1 Config Entries are Updated (#20456)
* Trigger the v1 compat exported-services controller when the v1 config entry is modified.

* Hook up exported-services config entries to the event publisher.
* Add tests to the v2 exported services shim.
* Use the local materializer trigger updates on the v1 compat exported services controller when exported-services config entries are modified.

* stop sleeping when context is cancelled
2024-02-02 15:30:04 -05:00
Derek Menteer 1fe0a87546
Fix SDK iptables.Config marshalling (#20451)
This fixes behavior introduced by hashicorp/consul#20232 where
a function was added to the iptables configuration struct. Since this
struct is actually marshalled into json by consul-k8s, we should not be
placing functions inside of it.
2024-02-02 12:25:00 -06:00
Chris Hut 22e6ce0df1
Add nav bar item to show HCP link status and encourage folks to link (#20370)
* Convert consul-hcp to a simpler component

* update existing test to use envStub helper

* An hcp link item for the navbar

* A method of linking to HCP

* Hook up fetching linking status to the nav-item

* Hooking up fetching link status to the hcp link friend

* Adding some tests

* remove a comment - but also fix padding justify-content

* Fix the banner tests

* Adding permission tests as well

* some more sane formatting

* Rename function with its now multipurpose use

* Feature change: No more NEW Badge since it breaks padding - instead a linked badge

* Removing unused class
2024-02-01 15:04:01 -08:00
Luke Kysow 49025105f0
Fix typo (#20441)
Update establish-cluster-peering.mdx
2024-02-01 14:35:52 -08:00
natemollica-dev 2b07b326c4
Resolve Consul DNS in OpenShift (#20439) 2024-02-01 14:00:27 -08:00
Derek Menteer 70575760c7
Add known issue for GH-20360. (#20420) 2024-02-01 15:29:46 -06:00
Matt Keeler 24a7b17a6f
Controller testing docs (#20398)
* Create testing.md

* Update guide.md
2024-02-01 15:16:22 -05:00
Andrew Stucki af0c4573bc
Rev VERSION for 1.19.0-dev (#20437) 2024-02-01 13:08:53 -06:00
Heat Hamilton 18a6804824
website: fix husky command (#20405)
Update husky pre-commit to work in website, a sub-directory of the directory containing the .git dir
2024-02-01 13:13:27 -05:00
Eric Haberkorn d0243b618d
Change the multicluster group to v2 (#20430) 2024-02-01 12:08:26 -05:00
Melisa Griffin 7c00d396cf
[NET-6417] Add validation of MeshGateway name + listeners (#20425)
* Add validation of MeshGateway name + listeners

* Adds test for ValidateMeshGateway

* Fixes data fetcher test for gatewayproxy

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2024-01-31 18:47:57 -05:00
Chris S. Kim b6f10bc58f
Skip filter chain created by permissive mtls (#20406) 2024-01-31 16:39:12 -05:00
wangxinyi7 3b44be530d
only forwarding the resource service traffic in client agent to server agent (#20347)
* only forwarding the resource service traffic in client agent to server agent
2024-01-31 12:05:47 -08:00
Nick Ethier 383d92e9ab
hcp.v2.TelemetryState resource and controller implementation (#20257)
* pbhcp: add TelemetryState resource

* agent/hcp: add GetObservabilitySecrets to client

* internal/hcp: add TelemetryState controller logic

* hcp/telemetry-state: added config options for hcp sdk and debug key to skip deletion during reconcile

* pbhcp: update proto documentation

* hcp: address PR feedback, additional validations and code cleanup

* internal/hcp: fix type sig change in test

* update testdata/v2-resource-dependencies
2024-01-31 14:47:05 -05:00
Derek Menteer 3e8ec8d18e
Fix SAN matching on terminating gateways (#20417)
Fixes issue: hashicorp/consul#20360

A regression was introduced in hashicorp/consul#19954 where the SAN validation
matching was reduced from 4 potential types down to just the URI.

Terminating gateways will need to match on many fields depending on user
configuration, since they make egress calls outside of the cluster. Having more
than one matcher behaves like an OR operation, where any match is sufficient to
pass the certificate validation. To maintain backwards compatibility with the
old untyped `match_subject_alt_names` Envoy behavior, we should match on all 4
enum types.

https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/transport_sockets/tls/v3/common.proto#enum-extensions-transport-sockets-tls-v3-subjectaltnamematcher-santype
2024-01-31 12:17:45 -06:00
cskh 890332cacb
docs: Fix indention of wal object in agent config (#20399) 2024-01-31 11:24:00 -05:00
Nathan Coleman 74e4200d07
[NET-6429] Program ProxyStateTemplate to route cross-partition traffi… (#20410)
[NET-6429] Program ProxyStateTemplate to route cross-partition traffic to the correct destination mesh gateway

* Program mesh port to route wildcarded gateway SNI to the appropriate remote partition's mesh gateway

* Update target + route ports in service endpoint refs when building PST

* Use proper name of local datacenter when constructing SNI for gateway target

* Use destination identities for TLS when routing L4 traffic through the mesh gateway

* Use new constants, move comment to correct location

* Use new constants for port names

* Update test assertions

* Undo debug logging change
2024-01-31 10:46:04 -05:00
John Murret c82b78b088
NET-7165 - fix address and target setting (#20403) 2024-01-30 15:34:35 -07:00
Ronald 8799c36410
[NET-6231] Handle Partition traffic permissions when reconciling traffic permissions (#20408)
[NET-6231] Partition traffic permissions

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
2024-01-30 22:14:32 +00:00
NicoletaPopoviciu b7b9bb0e83
Update Vault/Nomad versions. (#20322)
Update Vault/Nomad versions to ensure we're testing all the latest versions.

Update Vault/Nomad versions to ensure we're testing all the latest versions .
2024-01-30 16:36:08 -05:00
Chris S. Kim 7cc88a1577
Handle NamespaceTrafficPermissions when reconciling TrafficPermissions (#20407) 2024-01-30 21:31:25 +00:00
Nathan Coleman 21b3c18d5d
Use a full EndpointRef on ComputedRoutes targets instead of just the ID (#20400)
* Use a full EndpointRef on ComputedRoutes targets instead of just the ID

Today, the `ComputedRoutes` targets have the appropriate ID set for their `ServiceEndpoints` reference; however, the `MeshPort` and `RoutePort` are assumed to be that of the target when adding the endpoints reference in the sidecar's `ProxyStateTemplate`.

This is problematic when the target lives behind a `MeshGateway` and the `Mesh/RoutePort` used in the sidecar's `ProxyStateTemplate` should be that of the `MeshGateway` instead of the target.

Instead of assuming the `MeshPort` and `RoutePort` when building the `ProxyStateTemplate` for the sidecar, let's just add the full `EndpointRef` -- including the ID and the ports -- when hydrating the computed destinations.

* Make sure the UID from the existing ServiceEndpoints makes it onto ComputedRoutes

* Update test assertions

* Undo confusing whitespace change

* Remove one-line function wrapper

* Use plural name for endpoints ref

* Add constants for gateway name, kind and port names
2024-01-30 16:25:44 -05:00