Commit Graph

21246 Commits

Author SHA1 Message Date
natemollica-dev 3f3477cdd3
Consul Kubernetes Datadog Integration Docs Update (#20661) 2024-02-20 06:14:07 -08:00
John Maguire 2ed67ba9ae
[NET-7375] Add Skeleton for APIGW ProxyStateTemplateBuilder (#20628)
* WIP

* got empty state working and cleaned up additional code

* Moved api gateway mapper to it's own file, removed mesh port usage for
api gateway, allow gateway to reconcile without routes

* Update internal/mesh/internal/controllers/gatewayproxy/controller.go

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Feedback from PR Review:
  - rename referencedTCPRoutes variable to be more descriptive
  - fetchers are in alphabetical order
  - move log statements to be more indicative of internal state

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2024-02-16 10:38:08 -05:00
Chris S. Kim f5c2b408f7
Add enterprise docs for deny action (#20654) 2024-02-16 10:17:21 -05:00
Semir Patel 2baf5e78a4
docs: developer docs for resource finalizers (#20631) 2024-02-15 16:41:00 +00:00
Valeriia Ruban 8425cd0f90
chor: change cluster name param to have datacenter.name as default value (#20644)
* chor: change cluster name param to have datacenter.name as default value
2024-02-15 01:26:34 -08:00
Dan Stough 14efb28086
fix(v2dns): add node ttl to workloads, comment cleanup, and changelog (#20643)
* fix(v2dns): add node ttl to workloads, plus comment cleanup

* docs(v2dns): changelog
2024-02-14 17:38:11 -05:00
Derek Menteer 9f7626d501
Ensure all topics are refreshed on FSM restore and add supervisor loop to v1 controller subscriptions (#20642)
Ensure all topics are refreshed on FSM restore and add supervisor loop to v1 controller subscriptions

This PR fixes two issues:

1. Not all streams were force closed whenever a snapshot restore happened. This means that anything consuming data from the stream (controllers, queries, etc) were unaware that the data they have is potentially stale / invalid. This first part ensures that all topics are purged.

2. The v1 controllers did not properly handle stream errors (which are likely to appear much more often due to 1 above) and so it introduces a supervisor thread to restart the watches when these errors occur.
2024-02-14 14:17:55 -06:00
Dan Stough 137c9c0973
[CE] Misc cleanup for V2 DNS (#20640)
* chore: gitignore zed editor

* chore(v2dns): remove ent/ce split from router

* fix(v2dns): v2 workloads now have tenancy in output

* feat(v2dns): support 'cluster' label

* chore(v2dns): less chatty debug logs
2024-02-14 12:40:38 -05:00
wangxinyi7 daebd7a717
update entry sequence (#20637)
* update entry sequence
2024-02-14 09:30:00 -08:00
John Maguire 02e735f40f
[NET-7953] Added computed gateway routes protos and generated code (#20595)
Added computed gateway routes protos and generated code
2024-02-14 11:36:55 -05:00
wangxinyi7 aca26f51d0
update releasing changelog (#20633)
released 1.17.3 1.16.6 and 1.15.10
2024-02-13 17:12:11 -08:00
Valeriia Ruban 9d712ccfc7
chor: adds informative error message when acls disabled and read-only… (#20600)
* adds informative error message when acls disabled and read-only selected

* adds alert to the modal when there is no acls enabled
2024-02-13 14:00:04 -08:00
Chris S. Kim 248969c2a7
Update ComputedTrafficPermissions ACL hooks (#20622) 2024-02-13 15:16:54 -05:00
Melissa Kam 64cd172f30
[CC-7411] Fix environment variable precedence when linking to HCP (#20527)
Fix so that link API values are used over env vars

When a link is created via the API, those values should take precedence over
the values set by environment variables. This change loads all the env vars
initially as part of the config builder rather than on demand.
2024-02-13 14:06:18 -06:00
Chris S. Kim 0c2c7ca9c7
Add samenessgroup registration for enterprise (#20624) 2024-02-13 18:23:19 +00:00
Michael Zalimeni 2c1addfd64
[NET-7015] DNS v2 + Catalog v2 int test (#20607)
test(v2dns): Add Catalog v2 integration test

Add a basic integration test covering major functionality tested against
Catalog v2 resources. This complements existing tests that ensure
compatibility between v1 and v2 DNS when testing against Catalog v1
resources.
2024-02-13 17:40:08 +00:00
Chris S. Kim ab3c6cf1e5
Add BoundReferences to ComputedTrafficPermissions (#20593) 2024-02-13 17:27:24 +00:00
Dan Stough 0f0b080514
[CE] feat(v2dns): add v2 style query metrics (#20608)
feat(v2dns): add v2 style query metrics
2024-02-13 12:08:01 -05:00
Chris S. Kim 7575b53737
Refactor xTP tests (#20597) 2024-02-13 12:02:42 -05:00
Luke Kysow 191561e44f
docs: document behaviour of tls.https.verify_outgoing (#20555)
* docs: document behaviour of tls.https.verify_outgoing

At first it's not clear what verify_outgoing would do for the https
listener as it seems like Consul agent's don't make https requests. Upon
further investigation, it's clear that Consul agents do make https
requests in the following scenarios:
- to implement watches
- to perform checks

In the first scenario, this setting is used here:
a1c8d4dd19/agent/config/runtime.go (L1725)

In the second scenario, it's actually the internal_rpc setting that is
used:
a1c8d4dd19/tlsutil/config.go (L903)

* Update website/content/docs/agent/config/config-files.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

---------

Co-authored-by: David Yu <dyu@hashicorp.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2024-02-13 09:01:03 -08:00
Semir Patel b716a9ef6b
resource: reconcile managed types every ~8hrs (#20606) 2024-02-13 10:51:54 -06:00
Tu Nguyen 35f1173689
Updated docs for Consul ECS 0.8.x, architecture, tproxy support (#20473)
* Updated docs for Consul ECS 0.8.x, architecture, tproxy support

* Apply suggestions from code review

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>

* add apigw as feature, update images

---------

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>
2024-02-12 22:59:10 -08:00
Michael Zalimeni 5862c52642
[NET-7948] Bump Envoy version to address multiple CVEs (#20589)
security: Bump Envoy versions to address CVEs
2024-02-12 22:29:50 +00:00
R.B. Boyer 671c436415
mesh: use ComputedImplicitDestinations resource in the sidecar controller (#20553)
Wire the ComputedImplicitDestinations resource into the sidecar controller, replacing the inline version already present.

Also:

- Rewrite the controller to use the controller cache
- Rewrite it to no longer depend on ServiceEndpoints
- Remove the fetcher and (local) cache abstraction
2024-02-12 14:10:33 -06:00
John Murret 7e8f2e5f08
NET-7644/NET-7634 - Implement query lookup for tagged addresses on nodes and services including WAN translation. (#20583)
NET-7644 - Implement tagged addresses and wan translation
2024-02-12 14:27:25 -05:00
Dan Stough 5802080db1
feat(v2dns): enable peering queries (#20581) 2024-02-12 14:25:45 -05:00
John Maguire ec76090be9
[NET-7450] Fix listenerToProtocol function for input (#20536)
* make listenerProtocolToCatalogProtocol function more forgiving for
different cased input

* update tests
2024-02-12 11:24:56 -05:00
Nick Cellino 5fb6ab6a3a
Move HCP Manager lifecycle management out of Link controller (#20401)
* Add function to get update channel for watching HCP Link

* Add MonitorHCPLink function

This function can be called in a goroutine to manage the lifecycle
of the HCP manager.

* Update HCP Manager config in link monitor before starting

This updates HCPMonitorLink so it updates the HCP manager
with an HCP client and management token when a Link is upserted.

* Let MonitorHCPManager handle lifecycle instead of link controller

* Remove cleanup from Link controller and move it to MonitorHCPLink

Previously, the Link Controller was responsible for cleaning up the
HCP-related files on the file system. This change makes it so
MonitorHCPLink handles this cleanup. As a result, we are able to remove
the PlacementEachServer placement strategy for the Link controller
because it no longer needs to do this per-node cleanup.

* Remove HCP Manager dependency from Link Controller

The Link controller does not need to have HCP Manager
as a dependency anymore, so this removes that dependency
in order to simplify the design.

* Add Linked prefix to Linked status variables

This is in preparation for adding a new status type to the
Link resource.

* Add new "validated" status type to link resource

The link resource controller will now set a "validated" status
in addition to the "linked" status. This is needed so that other
components (eg the HCP manager) know when the Link is ready to link
with HCP.

* Fix tests

* Handle new 'EndOfSnapshot' WatchList event

* Fix watch test

* Remove unnecessary config from TestAgent_scadaProvider

Since the Scada provider is now started on agent startup
regardless of whether a cloud config is provided, this removes
the cloud config override from the relevant test.

This change is not exactly related to the changes from this PR,
but rather is something small and sort of related that was noticed
while working on this PR.

* Simplify link watch test and remove sleep from link watch

This updates the link watch test so that it uses more mocks
and does not require setting up the infrastructure for the HCP Link
controller.

This also removes the time.Sleep delay in the link watcher loop in favor
of an error counter. When we receive 10 consecutive errors, we shut down
the link watcher loop.

* Add better logging for link validation. Remove EndOfSnapshot test.

* Refactor link monitor test into a table test

* Add some clarifying comments to link monitor

* Simplify link watch test

* Test a bunch more errors cases in link monitor test

* Use exponential backoff instead of errorCounter in LinkWatch

* Move link watch and link monitor into a single goroutine called from server.go

* Refactor HCP link watcher to use single go-routine.

Previously, if the WatchClient errored, we would've never recovered
because we never retry to create the stream. With this change,
we have a single goroutine that runs for the life of the server agent
and if the WatchClient stream ever errors, we retry the creation
of the stream with an exponential backoff.
2024-02-12 10:48:23 -05:00
Tauhid Anjum 9d8f9a5470
NET-7783: Fix sameness group expansion to 0 sources error CE (#20584) 2024-02-12 17:04:18 +05:30
John Murret c8e4cea69c
set up ent and CE specific DNS tests to be able to run v1 and v2 (#20571) 2024-02-09 15:53:56 -07:00
Dan Stough 01001f630e
feat(v2dns): catalog v2 service query support (#20564) 2024-02-09 17:41:40 -05:00
David Yu e24b73a6dd
docs: Add legacy api gateway deprecation to Consul release notes (#20569)
* Update v1_16_x.mdx

* Update website/content/docs/release-notes/consul/v1_16_x.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update v1_2_x.mdx

---------

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2024-02-09 21:57:57 +00:00
R.B. Boyer 6742340878
mesh: add ComputedImplicitDestinations resource for future use (#20547)
Creates a new controller to create ComputedImplicitDestinations resources by 
composing ComputedRoutes, Services, and ComputedTrafficPermissions to 
infer all ParentRef services that could possibly send some portion of traffic to a 
Service that has at least one accessible Workload Identity. A followup PR will 
rewire the sidecar controller to make use of this new resource.

As this is a performance optimization, rather than a security feature the following 
aspects of traffic permissions have been ignored:

- DENY rules
- port rules (all ports are allowed)

Also:

- Add some v2 TestController machinery to help test complex dependency mappers.
2024-02-09 15:42:10 -06:00
Valeriia Ruban 8c05e57ac1
Cc 7147 link to hcp modal (#20474)
* add link hcp modal component

* integrate modal with SideNav and link to hcp banner
---------

Co-authored-by: Chris Hut <tophernuts@gmail.com>
2024-02-09 18:23:13 +00:00
Matt Keeler 6c4b83c119
Allow reuse of cache indexes (#20562)
Previously calling `index.New` would return an object with the index information such as the Indexer, whether it was required, and the name of the index as well as a radix tree to store indexed data.

Now the main `Index` type doesn’t contain the radix tree for indexed data. Instead the `IndexedData` method can be used to combine the main `Index` with a radix tree in the `IndexedData` structure.

The cache still only allows configuring the `Index` type and will invoke the `IndexedData` method on the provided indexes to get the structure that the cache can use for actual data management.

All of this makes it now safe to reuse the `index.Index` types.
2024-02-09 13:00:21 -05:00
Valeriia Ruban 6708e88ec9
chor: remove temporary CONSUL_HCP_LINK_ENABLED env flag (#20552) 2024-02-09 09:42:25 -08:00
Dan Stough 24e15cc24e
feat(v2dns): prepared query ttls (#20563) 2024-02-09 11:26:02 -05:00
John Murret 7cac918811
NET-7637 / NET-7659/NET-7636/NET-7647/NET-7648/NET-7646/NET-7649/NET-7645 - Multiple DNS v2 fixes (#20556) 2024-02-08 19:56:04 -07:00
skpratt 738dc8c89d
use go 1.21.7 (#20545)
* 1.21.7

* changelog
2024-02-08 23:39:11 +00:00
Derek Menteer a1c8d4dd19
Decouple xds capacity controller and raft-autopilot (#20511)
Decouple xds capacity controller and autopilot

This prevents a potential bug where autopilot deadlocks while attempting
to execute `AutopilotDelegate.NotifyState()` on an xdscapacity controller
that stopped consuming messages.
2024-02-08 15:31:44 -06:00
Chris S. Kim 26661a1c3b
Add default intention policy (#20544) 2024-02-08 20:25:42 +00:00
John Maguire 7c3a379e48
Fixes gatewayproxy controller tests for ent (#20543)
fix tests for ent
2024-02-08 18:34:44 +00:00
Eric Haberkorn b26282568f
Move sameness groups to v2beta1 version (#20531) 2024-02-08 11:05:06 -05:00
Joshua Timmons 242b777547
Fix logging when we fail to export metrics to hcp (#20514) 2024-02-08 11:00:47 -05:00
Joshua Timmons c790740cc6
Fix: avoid redundant logs on failures to export metrics (#20519) 2024-02-08 11:00:20 -05:00
John Murret 8ac54707d6
DNS v2 Multiple fixes. (#20525)
* DNS v2 Multiple fixes.

* add license header

* get rid of DefaultIntentionPolicy change that was not supposed to be there.
2024-02-07 21:24:00 -07:00
Mark Campbell-Vincent 27f6fef999
Update cache section for certain API calls (#20464)
* Update cache section for certain API calls

Providing more detail to the cache section to address behavior of API calls using streaming backend. This will help users understand that when '?index' is used and '?cached' is not, caching to servers will be bypassed, causing entry_fetch_max_burst and entry_fetch_rate to not be used in this scenario.

* Update website/content/docs/agent/config/config-files.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

---------

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2024-02-07 21:09:26 -05:00
Semir Patel 3c1ee84d0a
fix typos in docs for `consul snapshot agent` (#20522) 2024-02-07 17:09:34 -06:00
Melissa Kam c95380e3da
[CC-7434] Skip collecting data directory metrics in dev mode (#20521)
* Skip collecting data directory stats in dev mode

In dev mode, the data directory is not set, so this metrics collection would
always fail and logs errors.

* Log collection errors at DEBUG level

There isn't much action a user can take to fix these errors, so
logging them as DEBUG rather than ERROR.
2024-02-07 16:59:06 -06:00
John Maguire 2ee32b1980
Add tests for gw proxy controller (#20510)
* Added basic tests for gatewayproxy controller

* add copyright header

* Clean up tests
2024-02-07 17:01:10 -05:00