Commit Graph

1195 Commits

Author SHA1 Message Date
Chris S. Kim ad26494016
[CE] Add workload bind type and templated policy (#19077) 2023-10-05 19:45:41 +00:00
Chris Thain 5e45db18b7
Include RequestTimeout in marshal/unmarshal of ServiceResolverConfigE… (#19031) 2023-09-29 10:39:46 -07:00
Tim Gross e5f5fc9301
api: add `CheckRegisterOpts` method to Agent API (#18943)
Ongoing work to support Nomad Workload Identity for authenticating with Consul
will mean that Nomad's service registration sync with Consul will want to use
Consul tokens scoped to individual workloads for registering services and
checks. The `CheckRegister` method in the API doesn't have an option to pass the
token in, which prevent us from sharing the same Consul connection for all
workloads. Add a `CheckRegisterOpts` to match the behavior of
`ServiceRegisterOpts`.
2023-09-25 08:25:02 -07:00
Tim Gross aedc03b7ae
api: add Token field to ServiceRegisterOpts (#18983)
Ongoing work to support Nomad Workload Identity for authenticating with Consul
will mean that Nomad's service registration sync with Consul will want to use
Consul tokens scoped to individual workloads for registering services and
checks. The `ServiceRegisterOpts` type in the API doesn't have an option to pass
the token in, which prevent us from sharing the same Consul connection for all
workloads. Add a `Token` field to match the behavior of `ServiceDeregisterOpts`.
2023-09-25 08:24:30 -07:00
Nitya Dhanushkodi 58d06175ab
docs: add changelog (#18994) 2023-09-25 10:46:51 -04:00
John Landa 9eaa8eb026
dns token (#17936)
* dns token

fix whitespace for docs and comments

fix test cases

fix test cases

remove tabs in help text

Add changelog

Peering dns test

Peering dns test

Partial implementation of Peered DNS test

Swap to new topology lib

expose dns port for integration tests on client

remove partial test implementation

remove extra port exposure

remove changelog from the ent pr

Add dns token to set-agent-token switch

Add enterprise golden file

Use builtin/dns template in tests

Update ent dns policy

Update ent dns template test

remove local gen certs

fix templated policy specs

* add changelog

* go mod tidy
2023-09-20 15:50:06 -06:00
Blake Covarrubias 019c62e1ba
xds: Use downstream protocol when connecting to local app (#18573)
Configure Envoy to use the same HTTP protocol version used by the
downstream caller when forwarding requests to a local application that
is configured with the protocol set to either `http2` or `grpc`.

This allows upstream applications that support both HTTP/1.1 and
HTTP/2 on a single port to receive requests using either protocol. This
is beneficial when the application primarily communicates using HTTP/2,
but also needs to support HTTP/1.1, such as to respond to Kubernetes
HTTP readiness/liveness probes.

Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>
2023-09-19 14:32:28 -07:00
Ashesh Vidyut 6fd33ba30d
NET-4519 Collecting journald logs in "consul debug" bundle (#18797)
* debug since

* fix docs

* chagelog added

* fix go mod

* debug test fix

* fix test

* tabs test fix

* Update .changelog/18797.txt

Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>

---------

Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>
2023-09-19 08:46:50 +05:30
Andrew Stucki 087539fc7b
Fix gateway services cleanup where proxy deregistration happens after service deregistration (#18831)
* Fix gateway services cleanup where proxy deregistration happens after service deregistration

* Add test

* Add changelog

* Fix comment
2023-09-18 16:19:17 -04:00
James Hartig b2e21c103f
consul operator raft transfer-leader should send the id (#17107)
Fixes #16955

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2023-09-15 14:38:59 -04:00
Ronald 1afeb6e040
[NET-5334] Added CLI commands for templated policies (#18816) 2023-09-14 20:14:55 +00:00
Ronald 802122640b
[NET-5329] use acl templated policy under the hood for node/service identities (#18813) 2023-09-14 14:36:34 -04:00
Chris S. Kim 4dfca64ded
Vault CA provider clean up previous default issuers (#18773) 2023-09-13 19:33:02 +00:00
Chris S. Kim d090668c37
Add workload identity ACL rules (#18769) 2023-09-12 17:22:51 -04:00
Michael Zalimeni 5e7afdf9a1
[NET-5574] Update Go version to 1.20.8 (#18742)
Update Go version to 1.20.8

This resolves several CVEs (see changelog entry).
2023-09-12 11:40:51 -04:00
Ronald 40d7ebc318
[NET-5330] Support templated policies in Binding rules (#18719)
* [NET-5330] Support templated policies in Binding rules

* changelog for templated policy support in binding rules
2023-09-08 14:39:09 -04:00
Semir Patel 576ffdf705
fix: emit consul version metric on a regular interval (#18724) 2023-09-08 13:09:07 -05:00
Nathan Coleman e5d26a13cd
NET-5530 Support response header modifiers on http-route config entry (#18646)
* Add response header filters to http-route config entry definitions

* Map response header filters from config entry when constructing route destination

* Support response header modifiers at the service level as well

* Update protobuf definitions

* Update existing unit tests

* Add response filters to route consolidation logic

* Make existing unit tests more robust

* Add missing docstring

* Add changelog entry

* Add response filter modifiers to existing integration test

* Add more robust testing for response header modifiers in the discovery chain

* Add more robust testing for request header modifiers in the discovery chain

* Modify test to verify that service filter modifiers take precedence over rule filter modifiers
2023-09-08 14:04:56 -04:00
Ronald bbef879f85
[NET-5325] ACL templated policies support in tokens and roles (#18708)
* [NET-5325] ACL templated policies support in tokens and roles
- Add API support for creating tokens/roles with templated-policies
- Add CLI support for creating tokens/roles with templated-policies

* adding changelog
2023-09-08 12:45:24 +00:00
Gerard Nguyen 56d6e54ac7
fix: NET-1521 show latest config in /v1/agent/self (#18681)
* fix: NET-1521 show latest config in /v1/agent/self
2023-09-08 09:47:31 +10:00
John Maguire 2e7d951086
Added changelog for jwt features (#18709) 2023-09-07 16:30:49 -04:00
Iryna Shustava 4eb2197e82
dataplane: Allow getting bootstrap parameters when using V2 APIs (#18504)
This PR enables the GetEnvoyBootstrapParams endpoint to construct envoy bootstrap parameters from v2 catalog and mesh resources.

   * Make bootstrap request and response parameters less specific to services so that we can re-use them for workloads or service instances.
   * Remove ServiceKind from bootstrap params response. This value was unused previously and is not needed for V2.
   * Make access logs generation generic so that we can generate them using v1 or v2 resources.
2023-09-06 16:46:25 -06:00
Derek Menteer 56917eb4c9
Add support for querying tokens by service name. (#18667)
Add support for querying tokens by service name

The consul-k8s endpoints controller has a workflow where it fetches all tokens.
This is not performant for large clusters, where there may be a sizable number
of tokens. This commit attempts to alleviate that problem and introduces a new
way to query by the token's service name.
2023-09-06 10:52:45 -05:00
Phil Porada 7ea986783d
Add TCP+TLS Healthchecks (#18381)
* Begin adding TCPUseTLS

* More TCP with TLS plumbing

* Making forward progress

* Keep on adding TCP+TLS support for healthchecks

* Removed too many lines

* Unit tests for TCP+TLS

* Update tlsutil/config.go

Co-authored-by: Samantha <hello@entropy.cat>

* Working on the tcp+tls unit test

* Updated the runtime integration tests

* Progress

* Revert this file back to HEAD

* Remove debugging lines

* Implement TLS enabled TCP socket server and make a successful TCP+TLS healthcheck on it

* Update docs

* Update agent/agent_test.go

Co-authored-by: Samantha <hello@entropy.cat>

* Update website/content/docs/ecs/configuration-reference.mdx

Co-authored-by: Samantha <hello@entropy.cat>

* Update website/content/docs/ecs/configuration-reference.mdx

Co-authored-by: Samantha <hello@entropy.cat>

* Update agent/checks/check.go

Co-authored-by: Samantha <hello@entropy.cat>

* Address comments

* Remove extraneous bracket

* Update agent/agent_test.go

Co-authored-by: Samantha <hello@entropy.cat>

* Update agent/agent_test.go

Co-authored-by: Samantha <hello@entropy.cat>

* Update website/content/docs/ecs/configuration-reference.mdx

Co-authored-by: Samantha <hello@entropy.cat>

* Update the mockTLSServer

* Remove trailing newline

* Address comments

* Fix merge problem

* Add changelog entry

---------

Co-authored-by: Samantha <hello@entropy.cat>
2023-09-05 13:34:44 -07:00
Ashesh Vidyut 0c184042c5
change log for audit log (#18668)
* change log for audit log

* fix file name

* breaking change
2023-09-05 19:39:06 +00:00
Derek Menteer b56fbc7a62
[NET-4958] Fix issue where envoy endpoints would fail to populate after snapshot restore (#18636)
Fix issue where agentless endpoints would fail to populate after snapshot restore.

Fixes an issue that was introduced in #17775. This issue happens because
a long-lived pointer to the state store is held, which is unsafe to do.
Snapshot restorations will swap out this state store, meaning that the
proxycfg watches would break for agentless.
2023-09-01 10:18:10 -05:00
Ashesh Vidyut 78e3cbe156
NET 1594 - Snapshot Agent Filename Should Include Consul Version / Datacenter (#18625)
* init

* tests added and few fixes

* revert arg message

* changelog added

* removed var declaration

* fix CI

* fix test

* added node name and status

* updated save.mdx

* added example

* fix tense

* fix description
2023-09-01 10:23:44 +05:30
Ashesh Vidyut d45c3c2755
NET-3181 - Allow log file naming like Nomad (#18617)
* fixes file name for consul

* added log file

* added tests for rename method
2023-08-31 21:51:09 +05:30
Hardik Shingala 58e5658810
Added OpenTelemetry Access Logging Envoy extension (#18336) 2023-08-30 07:51:58 -07:00
Joshua Timmons 48c8a834f5
Reduce the frequency of metric exports to minutely (#18584) 2023-08-28 17:49:34 +00:00
Chris S. Kim ecdcde4309
CE commit (#18583) 2023-08-25 12:47:20 -04:00
Natalie Smith 34eb70037b
UI: community verbiage (#18560)
* chore: update community verbiage

* chore: add changelog entry
2023-08-23 12:59:30 -07:00
cskh b37587bb2c
bug: prevent go routine leakage due to existing DeferCheck (#18558)
* bug: prevent go routine leakage due to existing DeferCheck

* add changelog
2023-08-23 10:33:07 -04:00
Vijay a1755d158c
fix for , non presence of consul-version meta (#18464)
* fix for #18406 , non presence of consul-version meta

* removed redundant checks

* updated mock-api to mimic api response for synthetic nodes

* added test to test getDistinctConsulVersions method with synthetic-node case

* updated typo in comments

* added change log
2023-08-22 16:26:05 -06:00
Ashwin Venkatesh 97b41d946f
Support custom watches on controller (#18439)
* Support custom watches on controller
* refactor mapper methods
2023-08-17 20:34:18 +00:00
John Maguire df11e4e7b4
APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422)
* Updated httproute config entry for JWT Filters

* Added manual deepcopy method for httproute jwt filter

* Fix test

* Update JWTFilter to be in oss file

* Add changelog

* Add build tags for deepcopy oss file
2023-08-10 21:23:42 +00:00
John Maguire 6c8ca0f89d
NET-4984: Update APIGW Config Entries for JWT Auth (#18366)
* Added oss config entries for Policy and JWT on APIGW

* Updated structs for config entry

* Updated comments, ran deep-copy

* Move JWT configuration into OSS file

* Add in the config entry OSS file for jwts

* Added changelog

* fixing proto spacing

* Moved to using manually written deep copy method

* Use pointers for override/default fields in apigw config entries

* Run gen scripts for changed types
2023-08-10 19:49:51 +00:00
Michael Zalimeni 05604eeec1
[NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437)
* Add logging to locality policy application

In OSS, this is currently a no-op.

* Inherit locality when registering sidecars

When sidecar locality is not explicitly configured, inherit locality
from the proxied service.
2023-08-10 14:00:44 -04:00
sarahalsmiller e235c8be3c
NET-5115 Add retry + timeout filters for api-gateway (#18324)
* squash, implement retry/timeout in consul core

* update tests
2023-08-08 16:39:46 -05:00
Ashesh Vidyut 417ae9fc39
Fix #17730 - Dev mode has new line (#18367)
* adding new line only in case of pretty in url not in dev mode

* change log added
2023-08-05 08:15:24 +05:30
Michael Zalimeni 905e371607
[NET-5146] security: Update Go version to 1.20.7 and `x/net` to 0.13.0 (#18358)
* Update Go version to 1.20.7

This resolves [CVE-2023-29409]
(https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`).

* Bump golang.org/x/net to 0.13.0

Addresses [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978)
for security scans (non-impacting).
2023-08-02 13:10:29 -04:00
Ashesh Vidyut 67fc93e26d
NET-4240 - Snapshots are failing on Windows (#18302)
* fix go mod

* fix go sum

* added change log

* ran make go mod tidy
2023-08-02 08:14:35 +05:30
Ashvitha 828567c62e
[HCP Telemetry] Periodic Refresh for Dynamic Telemetry Configuration (#18168)
* OTElExporter now uses an EndpointProvider to discover the endpoint

* OTELSink uses a ConfigProvider to obtain filters and labels configuration

* improve tests for otel_sink

* Regex logic is moved into client for a method on the TelemetryConfig object

* Create a telemetry_config_provider and update deps to use it

* Fix conversion

* fix import newline

* Add logger to hcp client and move telemetry_config out of the client.go file

* Add a telemetry_config.go to refactor client.go

* Update deps

* update hcp deps test

* Modify telemetry_config_providers

* Check for nil filters

* PR review updates

* Fix comments and move around pieces

* Fix comments

* Remove context from client struct

* Moved ctx out of sink struct and fixed filters, added a test

* Remove named imports, use errors.New if not fformatting

* Remove HCP dependencies in telemetry package

* Add success metric and move lock only to grab the t.cfgHahs

* Update hash

* fix nits

* Create an equals method and add tests

* Improve telemetry_config_provider.go tests

* Add race test

* Add missing godoc

* Remove mock for MetricsClient

* Avoid goroutine test panics

* trying to kick CI lint issues by upgrading mod

* imprve test code and add hasher for testing

* Use structure logging for filters, fix error constants, and default to allow all regex

* removed hashin and modify logic to simplify

* Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test

* Ran make go-mod-tidy

* Use errtypes in the test

* Add changelog

* add safety check for exporter endpoint

* remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter

* Fixed race test to have changing config values

* Send success metric before modifying config

* Avoid the defer and move the success metric under
2023-08-01 17:20:18 -04:00
Jeremy Jacobson 6424ef6a56
[CC-5719] Add support for builtin global-read-only policy (#18319)
* [CC-5719] Add support for builtin global-read-only policy

* Add changelog

* Add read-only to docs

* Fix some minor issues.

* Change from ReplaceAll to Sprintf

* Change IsValidPolicy name to return an error instead of bool

* Fix PolicyList test

* Fix other tests

* Apply suggestions from code review

Co-authored-by: Paul Glass <pglass@hashicorp.com>

* Fix state store test for policy list.

* Fix naming issues

* Update acl/validation.go

Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com>

* Update agent/consul/acl_endpoint.go

---------

Co-authored-by: Paul Glass <pglass@hashicorp.com>
Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com>
2023-08-01 17:12:14 +00:00
Michael Zalimeni b1b05f0bac
[NET-4703] Prevent partial application of Envoy extensions (#18068)
Prevent partial application of Envoy extensions

Ensure that non-required extensions do not change xDS resources before
exiting on failure by cloning proto messages prior to applying each
extension.

To support this change, also move `CanApply` checks up a layer and make
them prior to attempting extension application, s.t. we avoid
unnecessary copies where extensions can't be applied.

Last, ensure that we do not allow panics from `CanApply` or `Extend`
checks to escape the attempted extension application.
2023-07-31 15:24:33 -04:00
Ronald 356b29bf35
Stop JWT provider from being written in non default namespace (#18325) 2023-07-31 09:13:16 -04:00
Florian Apolloner 6ada2e05ff
Fix topology view when displaying mixed connect-native/normal services. (#13023)
* Fix topoloy intention with mixed connect-native/normal services.

If a service is registered twice, once with connect-native and once
without, the topology views would prune the existing intentions. This
change brings the code more in line with the transparent proxy behavior.

* Dedupe nodes in the ServiceTopology ui endpoint (like done with tags).

* Consider a service connect-native as soon as one instance is.
2023-07-31 08:10:55 -04:00
Michael Zalimeni cf4deeb7ea
Update list of Envoy versions (#18300)
Update supported envoy versions to 1.24.10, 1.25.9, 1.26.4, 1.27.0.
2023-07-26 21:48:29 +00:00
Nathan Coleman 5caa0ae3f5
api-gateway: subscribe to bound-api-gateway only after receiving api-gateway (#18291)
* api-gateway: subscribe to bound-api-gateway only after receiving api-gateway

This fixes a race condition due to our dependency on having the listener(s) from the api-gateway config entry in order to fully and properly process the resources on the bound-api-gateway config entry.

* Apply suggestions from code review

* Add changelog entry
2023-07-26 16:02:04 -04:00
cskh 31d2813714
member cli: add -filter expression to flags (#18223)
* member cli: add -filter expression to flags

* changelog

* update doc

* Add test cases

* use quote
2023-07-25 13:54:52 -04:00