Commit Graph

1225 Commits

Author SHA1 Message Date
lornasong edf4610ed9
[Cloud][CC-6925] Updates to pushing server state (#19682)
* Upgrade hcp-sdk-go to latest version v0.73

Changes:
- go get github.com/hashicorp/hcp-sdk-go
- go mod tidy

* From upgrade: regenerate protobufs for upgrade from 1.30 to 1.31

Ran: `make proto`

Slack: https://hashicorp.slack.com/archives/C0253EQ5B40/p1701105418579429

* From upgrade: fix mock interface implementation

After upgrading, there is the following compile error:

cannot use &mockHCPCfg{} (value of type *mockHCPCfg) as "github.com/hashicorp/hcp-sdk-go/config".HCPConfig value in return statement: *mockHCPCfg does not implement "github.com/hashicorp/hcp-sdk-go/config".HCPConfig (missing method Logout)

Solution: update the mock to have the missing Logout method

* From upgrade: Lint: remove usage of deprecated req.ServerState.TLS

Due to upgrade, linting is erroring due to usage of a newly deprecated field

22:47:56 [consul]: make lint
--> Running golangci-lint (.)
agent/hcp/testing.go:157:24: SA1019: req.ServerState.TLS is deprecated: use server_tls.internal_rpc instead. (staticcheck)
                time.Until(time.Time(req.ServerState.TLS.CertExpiry)).Hours()/24,
                                     ^

* From upgrade: adjust oidc error message

From the upgrade, this test started failing:

=== FAIL: internal/go-sso/oidcauth TestOIDC_ClaimsFromAuthCode/failed_code_exchange (re-run 2) (0.01s)
    oidc_test.go:393: unexpected error: Provider login failed: Error exchanging oidc code: oauth2: "invalid_grant" "unexpected auth code"

Prior to the upgrade, the error returned was:
```
Provider login failed: Error exchanging oidc code: oauth2: cannot fetch token: 401 Unauthorized\nResponse: {\"error\":\"invalid_grant\",\"error_description\":\"unexpected auth code\"}\n
```

Now the error returned is as below and does not contain "cannot fetch token"
```
Provider login failed: Error exchanging oidc code: oauth2: "invalid_grant" "unexpected auth code"

```

* Update AgentPushServerState structs with new fields

HCP-side changes for the new fields are in:
https://github.com/hashicorp/cloud-global-network-manager-service/pull/1195/files

* Minor refactor for hcpServerStatus to abstract tlsInfo into struct

This will make it easier to set the same tls-info information to both
 - status.TLS (deprecated field)
 - status.ServerTLSMetadata (new field to use instead)

* Update hcpServerStatus to parse out information for new fields

Changes:
 - Improve error message and handling (encountered some issues and was confused)
 - Set new field TLSInfo.CertIssuer
 - Collect certificate authority metadata and set on TLSInfo.CertificateAuthorities
 - Set TLSInfo on both server.TLS and server.ServerTLSMetadata.InternalRPC

* Update serverStatusToHCP to convert new fields to GNM rpc

* Add changelog

* Feedback: connect.ParseCert, caCerts

* Feedback: refactor and unit test server status

* Feedback: test to use expected struct

* Feedback: certificate with intermediate

* Feedback: catch no leaf, remove expectedErr

* Feedback: update todos with jira ticket

* Feedback: mock tlsConfigurator
2023-12-04 10:25:18 -05:00
Michael Zalimeni cc14ccf34a
[NET-6617] security: Bump github.com/golang-jwt/jwt/v4 to 4.5.0 (#19705)
security: Bump github.com/golang-jwt/jwt/v4 to 4.5.0

This version is accepted by Prisma/Twistlock, resolving scan results for
issue PRISMA-2022-0270. Chosen over later versions to avoid a major
version with breaking changes that is otherwise unnecessary.

Note that in practice this is a false positive (see
https://github.com/golang-jwt/jwt/issues/258), but we should update the
version to aid customers relying on scanners that flag it.
2023-11-27 11:03:26 -05:00
Ronald eded2ff347
[NET-6249] Add templated policies description (#19735) 2023-11-27 10:34:22 -05:00
Ronald c1dbf00a85
NET-6251 API gateway templated policy (#19728) 2023-11-24 17:55:05 +00:00
Ashvitha bfb3a43648
Default "stats_flush_interval" to 1 minute for Consul Telemetry Collector (#19663)
* Set default of 1m for StatsFlushInterval when the collector is setup

* Add documentation on the stats_flush_interval value

* Do not default in two conditions 1) preconfigured sinks exist 2) preconfigured flush interval exists

* Fix wording of docs

* Add changelog

* Fix docs
2023-11-20 16:18:30 -05:00
Dhia Ayachi f027d61014
fix a panic in the CLI when deleting an acl policy with an unknown name (#19679)
* fix a panic in the CLI when deleting an acl policy with an unknown name

* add changelog
2023-11-20 09:47:44 -05:00
Mike Nomitch 302f994410
[NET-6640] Adds "Policy" BindType to BindingRule (#19499)
feat: add bind type of policy

Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
2023-11-20 13:11:08 +00:00
Ronald ea0caa3e0f
[NET-6103] Enable query tokens by service name using templated policy (#19666) 2023-11-16 14:32:06 -05:00
Tyler Wendlandt 4d64ef0961
ui: move queries for selectors within the dropdowns (#19594)
Move queries for selectors within the dropdowns
2023-11-10 00:59:21 +00:00
Tyler Wendlandt 7699fb12eb
NET-5414: sameness group service show (#19586)
Fix viewing peered services on different namespaces
2023-11-09 15:25:01 -07:00
Tyler Wendlandt 1f5aa83a9e
ui: clear peer on home link (#19549)
Clear peer on home link
2023-11-07 10:27:20 -07:00
Tyler Wendlandt e5948e8eb4
CC-5545: Side Nav (#19342)
* Initial work for sidenav

* Use HDS::Text

* Add resolution for ember-element-helper

* WIP dc selector

* Update HCP Home link

* DC selector

* Hook up remaining selectors

* Fix settings and tutorial links

* Remove comments

* Remove skip-links

* Replace auth with new dropdown

* Use href-to helper for sidenav links

* Changelog

* Add description to NavSelector

* Wrap version in footer and role

* Fix login tests

* Add data-test selectors for namespaces

* Fix datacenter disclosure menu test

* Stop rendering auth dialog if acls are disabled

* Update disabled selector state and token selector

* Fix logic in ACL selector

* Fix HCP Home integration test

* Remove toggling the sidenav in tests

* Add sidenav to eng docs

* Re-add debug navigation for eng docs

* Remove ember-in-viewport

* Remove unused styles

* Upgrade @hashicorp/design-system-componentseee

* Add translations for side-nav

* Only show back to hcp link if url is present

* Disable responsive due to a11y-dialog issue
2023-11-06 08:18:48 -07:00
Derek Menteer 6baf695cd9
[NET-6459] Fix issue with wanfed lan ip conflicts. (#19503)
Fix issue with wanfed lan ip conflicts.

Prior to this commit, the connection pools were unaware which datacenter the
connection was associated with. This meant that any time servers with
overlapping LAN IP addresses and node shortnames existed, they would be
incorrectly co-located in the same pool. Whenever this occurred, the servers
would get stuck in an infinite loop of forwarding RPCs to themselves (rather
than the intended remote DC) until they eventually run out of memory.

Most notably, this issue can occur whenever wan federation through mesh
gateways is enabled.

This fix adds extra metadata to specify which DC the connection is associated
with in the pool.
2023-11-06 08:47:12 -06:00
Nitya Dhanushkodi 2bc0bc30b9
update v2 changelog (#19446) 2023-11-02 14:59:55 -07:00
Michael Zalimeni 42647de35d
[NET-6138] security: Bump `google.golang.org/grpc` to 1.56.3 (CVE-2023-44487) (#19414)
Bump google.golang.org/grpc to 1.56.3

This resolves [CVE-2023-44487](https://nvd.nist.gov/vuln/detail/CVE-2023-44487).

Co-authored-by: Chris Thain <chris.m.thain@gmail.com>
2023-10-30 08:44:22 -04:00
Ronald ea91e58045
Stop use of templated-policy and templated-policy-file simultaneously (#19389) 2023-10-26 18:15:12 +00:00
Andrew Stucki e414cbee4a
Use strict DNS for mesh gateways with hostnames (#19268)
* Use strict DNS for mesh gateways with hostnames

* Add changelog
2023-10-24 15:04:14 -04:00
Dhia Ayachi 12ef115b61
bump raft-wal version to 0.4.1 (#19314)
* bump raft-wal version to 0.4.1

* changelog

* go mod tidy integration tests

* go mod tidy test-integ
2023-10-24 10:47:46 -04:00
Derek Menteer 48c4a5b736
Add grpc keepalive configuration. (#19339)
Prior to the introduction of this configuration, grpc keepalive messages were
sent after 2 hours of inactivity on the stream. This posed issues in various
scenarios where the server-side xds connection balancing was unaware that envoy
instances were uncleanly killed / force-closed, since the connections would
only be cleaned up after ~5 minutes of TCP timeouts occurred. Setting this
config to a 30 second interval with a 20 second timeout ensures that at most,
it should take up to 50 seconds for a dead xds connection to be closed.
2023-10-24 08:05:31 -05:00
aahel 1280f45485
added ent to ce downgrade changes (#19311)
* added ent to ce downgrade changes

* added changelog

* added busl headers
2023-10-20 22:34:25 +05:30
Chris Thain b1871fd08c
Backout Envoy 1.28.0 (#19306) 2023-10-20 17:03:54 +00:00
Chris S. Kim 9d00b13140
Vault CA bugfixes (#19285)
* Re-add retry logic to Vault token renewal

* Fix goroutine leak

* Add test for detecting goroutine leak

* Add changelog

* Rename tests

* Add comment
2023-10-20 15:03:27 +00:00
Chris Thain 681aef31e9
Update supported Envoy versions (#19276) 2023-10-19 21:08:20 +00:00
Poonam Jadhav 2bd38d8806
fix: allow snake case keys for ip based rate limit config entry (#19277)
* fix: allow snake case keys for ip based rate limit config entry

* chore: add changelog
2023-10-19 10:54:00 -04:00
Michael Zalimeni 8eb074e7c1
[NET-5944] security: Update Go version to 1.20.10 and `x/net` to 0.17.0 (#19225)
* Bump golang.org/x/net to 0.17.0

This resolves [CVE-2023-39325](https://nvd.nist.gov/vuln/detail/CVE-2023-39325)
/ [CVE-2023-44487](https://nvd.nist.gov/vuln/detail/CVE-2023-44487).

* Update Go version to 1.20.10

This resolves [CVE-2023-39325](https://nvd.nist.gov/vuln/detail/CVE-2023-39325)
/ [CVE-2023-44487](https://nvd.nist.gov/vuln/detail/CVE-2023-44487)
(`net/http`).
2023-10-16 17:49:04 -04:00
Semir Patel ad177698f7
resource: enforce lowercase v2 resource names (#19218) 2023-10-16 12:55:30 -05:00
Chris Thain dcdf2fc6ba
Update Vault CA provider namespace configuration (#19095) 2023-10-10 13:53:00 +00:00
Ashesh Vidyut a30ccdf5dc
NET-4135 - Fix NodeMeta filtering Catalog List Services API (#18322)
* logs for debugging

* Init

* white spaces fix

* added change log

* Fix tests

* fix typo

* using queryoptionfilter to populate args.filter

* tests

* fix test

* fix tests

* fix tests

* fix tests

* fix tests

* fix variable name

* fix tests

* fix tests

* fix tests

* Update .changelog/18322.txt

Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>

* fix change log

* address nits

* removed unused line

* doing join only when filter has nodemeta

* fix tests

* fix tests

* Update agent/consul/catalog_endpoint.go

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* fix tests

* removed unwanted code

---------

Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2023-10-08 12:48:31 +00:00
Derek Menteer af3439b53d
Ensure that upstream configuration is properly normalized. (#19076)
This PR fixes an issue where upstreams did not correctly inherit the proper
namespace / partition from the parent service when attempting to fetch the
upstream protocol due to inconsistent normalization.

Some of the merge-service-configuration logic would normalize to default, while
some of the proxycfg logic would normalize to match the parent service. Due to
this mismatch in logic, an incorrect service-defaults configuration entry would
be fetched and have its protocol applied to the upstream.
2023-10-06 13:59:47 -05:00
Thomas Eckert 342306c312
Allow connections through Terminating Gateways from peered clusters NET-3463 (#18959)
* Add InboundPeerTrustBundle maps to Terminating Gateway

* Add notify and cancelation of watch for inbound peer trust bundles

* Pass peer trust bundles to the RBAC creation function

* Regenerate Golden Files

* add changelog, also adds another spot that needed peeredTrustBundles

* Add basic test for terminating gateway with peer trust bundle

* Add intention to cluster peered golden test

* rerun codegen

* update changelog

* really update the changelog

---------

Co-authored-by: Melisa Griffin <melisa.griffin@hashicorp.com>
2023-10-05 21:54:23 +00:00
Chris S. Kim ad26494016
[CE] Add workload bind type and templated policy (#19077) 2023-10-05 19:45:41 +00:00
Chris Thain 5e45db18b7
Include RequestTimeout in marshal/unmarshal of ServiceResolverConfigE… (#19031) 2023-09-29 10:39:46 -07:00
Tim Gross e5f5fc9301
api: add `CheckRegisterOpts` method to Agent API (#18943)
Ongoing work to support Nomad Workload Identity for authenticating with Consul
will mean that Nomad's service registration sync with Consul will want to use
Consul tokens scoped to individual workloads for registering services and
checks. The `CheckRegister` method in the API doesn't have an option to pass the
token in, which prevent us from sharing the same Consul connection for all
workloads. Add a `CheckRegisterOpts` to match the behavior of
`ServiceRegisterOpts`.
2023-09-25 08:25:02 -07:00
Tim Gross aedc03b7ae
api: add Token field to ServiceRegisterOpts (#18983)
Ongoing work to support Nomad Workload Identity for authenticating with Consul
will mean that Nomad's service registration sync with Consul will want to use
Consul tokens scoped to individual workloads for registering services and
checks. The `ServiceRegisterOpts` type in the API doesn't have an option to pass
the token in, which prevent us from sharing the same Consul connection for all
workloads. Add a `Token` field to match the behavior of `ServiceDeregisterOpts`.
2023-09-25 08:24:30 -07:00
Nitya Dhanushkodi 58d06175ab
docs: add changelog (#18994) 2023-09-25 10:46:51 -04:00
John Landa 9eaa8eb026
dns token (#17936)
* dns token

fix whitespace for docs and comments

fix test cases

fix test cases

remove tabs in help text

Add changelog

Peering dns test

Peering dns test

Partial implementation of Peered DNS test

Swap to new topology lib

expose dns port for integration tests on client

remove partial test implementation

remove extra port exposure

remove changelog from the ent pr

Add dns token to set-agent-token switch

Add enterprise golden file

Use builtin/dns template in tests

Update ent dns policy

Update ent dns template test

remove local gen certs

fix templated policy specs

* add changelog

* go mod tidy
2023-09-20 15:50:06 -06:00
Blake Covarrubias 019c62e1ba
xds: Use downstream protocol when connecting to local app (#18573)
Configure Envoy to use the same HTTP protocol version used by the
downstream caller when forwarding requests to a local application that
is configured with the protocol set to either `http2` or `grpc`.

This allows upstream applications that support both HTTP/1.1 and
HTTP/2 on a single port to receive requests using either protocol. This
is beneficial when the application primarily communicates using HTTP/2,
but also needs to support HTTP/1.1, such as to respond to Kubernetes
HTTP readiness/liveness probes.

Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>
2023-09-19 14:32:28 -07:00
Ashesh Vidyut 6fd33ba30d
NET-4519 Collecting journald logs in "consul debug" bundle (#18797)
* debug since

* fix docs

* chagelog added

* fix go mod

* debug test fix

* fix test

* tabs test fix

* Update .changelog/18797.txt

Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>

---------

Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>
2023-09-19 08:46:50 +05:30
Andrew Stucki 087539fc7b
Fix gateway services cleanup where proxy deregistration happens after service deregistration (#18831)
* Fix gateway services cleanup where proxy deregistration happens after service deregistration

* Add test

* Add changelog

* Fix comment
2023-09-18 16:19:17 -04:00
James Hartig b2e21c103f
consul operator raft transfer-leader should send the id (#17107)
Fixes #16955

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2023-09-15 14:38:59 -04:00
Ronald 1afeb6e040
[NET-5334] Added CLI commands for templated policies (#18816) 2023-09-14 20:14:55 +00:00
Ronald 802122640b
[NET-5329] use acl templated policy under the hood for node/service identities (#18813) 2023-09-14 14:36:34 -04:00
Chris S. Kim 4dfca64ded
Vault CA provider clean up previous default issuers (#18773) 2023-09-13 19:33:02 +00:00
Chris S. Kim d090668c37
Add workload identity ACL rules (#18769) 2023-09-12 17:22:51 -04:00
Michael Zalimeni 5e7afdf9a1
[NET-5574] Update Go version to 1.20.8 (#18742)
Update Go version to 1.20.8

This resolves several CVEs (see changelog entry).
2023-09-12 11:40:51 -04:00
Ronald 40d7ebc318
[NET-5330] Support templated policies in Binding rules (#18719)
* [NET-5330] Support templated policies in Binding rules

* changelog for templated policy support in binding rules
2023-09-08 14:39:09 -04:00
Semir Patel 576ffdf705
fix: emit consul version metric on a regular interval (#18724) 2023-09-08 13:09:07 -05:00
Nathan Coleman e5d26a13cd
NET-5530 Support response header modifiers on http-route config entry (#18646)
* Add response header filters to http-route config entry definitions

* Map response header filters from config entry when constructing route destination

* Support response header modifiers at the service level as well

* Update protobuf definitions

* Update existing unit tests

* Add response filters to route consolidation logic

* Make existing unit tests more robust

* Add missing docstring

* Add changelog entry

* Add response filter modifiers to existing integration test

* Add more robust testing for response header modifiers in the discovery chain

* Add more robust testing for request header modifiers in the discovery chain

* Modify test to verify that service filter modifiers take precedence over rule filter modifiers
2023-09-08 14:04:56 -04:00
Ronald bbef879f85
[NET-5325] ACL templated policies support in tokens and roles (#18708)
* [NET-5325] ACL templated policies support in tokens and roles
- Add API support for creating tokens/roles with templated-policies
- Add CLI support for creating tokens/roles with templated-policies

* adding changelog
2023-09-08 12:45:24 +00:00
Gerard Nguyen 56d6e54ac7
fix: NET-1521 show latest config in /v1/agent/self (#18681)
* fix: NET-1521 show latest config in /v1/agent/self
2023-09-08 09:47:31 +10:00