Commit Graph

21089 Commits

Author SHA1 Message Date
Michael Zalimeni 79e02f8a89
ci: upload test results to DataDog on test failure (#19956)
Due to the unintuitive behavior of GHA w.r.t. implicit status check
`success()`, test results were only being uploaded on success (failures
presumably came from retried tests that passed).
2023-12-14 23:13:04 +00:00
Nathan Coleman 02d4520235
Fix typo in service-defaults documentation (#19957) 2023-12-14 22:12:28 +00:00
John Murret 83cbe15b44
cli: Deprecate the `-admin-access-log-path` flag from `consul connect envoy` command in favor of: `-admin-access-log-config`. (#19943)
* cli: Deprecate the `-admin-access-log-path` flag from `consul connect envoy` command in favor of: `-admin-access-log-config`.

* fix changelog

* add in documentation change.
2023-12-14 20:36:47 +00:00
John Murret a995505976
NET-6317 - update usage of deprecated fields: http2_protocol_options and access_log_path (#19940)
* updating usage of http2_protocol_options and access_log_path

* add changelog

* update template for AdminAccessLogConfig

* remove mucking with AdminAccessLogConfig
2023-12-14 13:08:53 -07:00
natemollica-dev afc6fe8308
Update telemetry.mdx RPC Metrics (#19593)
* Update telemetry.mdx RPC Metrics

Update Server Workload telemetry section to demonstrate explicitly enabling metric emission as they're [default disabled](f5bf256425/agent/config/builder.go (L2763C1-L2763C1)).

* Update telemetry.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update telemetry.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-12-14 10:52:45 -08:00
Tauhid Anjum 0250e230a3
NET-6785: updating peering docs to include stream status and remote data (#19929)
Updating peering docs to include stream status and remote data
2023-12-14 12:07:35 +05:30
aahel a6496898de
added tenancy to TestBuildL4TrafficPermissions (#19932) 2023-12-14 10:41:24 +05:30
cskh 33a90edfab
Upgrade test(LTS): use network area to federate cluster (#19934)
- Join areas
- wait for members alive and validate cross area service discovery
2023-12-13 20:15:55 -05:00
Ashesh Vidyut 3443db7885
NET 6762 (#19931)
NET-6762
2023-12-14 06:37:01 +05:30
Matt Keeler 123bc95e1a
Add Common Controller Caching Infrastructure (#19767)
* Add Common Controller Caching Infrastructure
2023-12-13 10:06:39 -05:00
Jeff Boruszak c870c00e70
docs: service rate limiting examples (#19925)
* Include examples on usage page.

* Description/example alignment
2023-12-12 15:58:14 -08:00
Semir Patel 69e3f93ee8
resource: add partition resource to proto-public to keep ENT and CE in sync (#19920) 2023-12-12 14:50:19 -05:00
John Murret a5d5fd348b
fix actions to no longer use envoy 1.24.x to match supported versions. (#19918) 2023-12-12 12:37:07 -05:00
Valeriia Ruban d7e0fca28b
fix: token list in Role details page is updated with tokens linked to… (#19912) 2023-12-12 09:36:50 -08:00
Tyler Wendlandt e8164c7c04
NET-6900: stop reconciling services when peering is enabled (#19907)
stop reconciling services when peering is enabled
2023-12-12 07:36:35 -07:00
Dhia Ayachi f2b26ac194
Hash based config entry replication (#19795)
* add a hash to config entries when normalizing

* add GetHash and implement comparing hashes

* only update if the Hash is different

* only update if the Hash is different and not 0

* fix proto to include the Hash

* fix proto gen

* buf format

* add SetHash and fix tests

* fix config load tests

* fix state test and config test

* recalculate hash when restoring config entries

* fix snapshot restore test

* add changelog

* fix missing normalize, fix proto indexes and add normalize test
2023-12-12 08:29:13 -05:00
Ganesh S 90010587f0
Move enterprise multicluster types to Register function (#19913)
* Move enterprise types to Register function

* Fix function name

* Address comments
2023-12-12 17:05:10 +05:30
Ganesh S 173fe11c2b
Refactor exported services controller tests (#19906) 2023-12-12 10:57:27 +05:30
Tauhid Anjum 1484c6db47
NET-6771 - Adding sameness group protobuff in consul CE (#19883)
Adding sameness group protobuff in consul CE
2023-12-12 10:43:20 +05:30
Ashesh Vidyut c5cce63777
NET 6761 (#19837)
NET-6761 explicit destinations tests updated
2023-12-12 10:38:00 +05:30
Valeriia Ruban a6d6164ba0
fix: remove test to unblock CI (#19908) 2023-12-11 20:11:36 -08:00
Ronald e13fbc743e
Remove warning for consul 1.17 deprecation (#19897) 2023-12-11 23:28:04 +00:00
Jeff Boruszak 659868ee73
docs: Updates to required ports (#19755)
* improvements

* Anchor link fixes

* Apply suggestions from code review

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Explicit list of six ports

* Apply suggestions from code review

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-12-11 14:42:57 -08:00
Derek Menteer ccb2bf6170
Add documentation for proxy-config-map and xds_fetch_timeout_ms. (#19893)
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2023-12-11 15:53:35 -06:00
Ronald 195e3aab8c
[NET-6842] splitting go version on different lines (#19887) 2023-12-11 11:15:32 -05:00
Derek Menteer dfab5ade50
Fix ClusterLoadAssignment timeouts dropping endpoints. (#19871)
When a large number of upstreams are configured on a single envoy
proxy, there was a chance that it would timeout when waiting for
ClusterLoadAssignments. While this doesn't always immediately cause
issues, consul-dataplane instances appear to consistently drop
endpoints from their configurations after an xDS connection is
re-established (the server dies, random disconnect, etc).

This commit adds an `xds_fetch_timeout_ms` config to service registrations
so that users can set the value higher for large instances that have
many upstreams. The timeout can be disabled by setting a value of `0`.

This configuration was introduced to reduce the risk of causing a
breaking change for users if there is ever a scenario where endpoints
would never be received. Rather than just always blocking indefinitely
or for a significantly longer period of time, this config will affect
only the service instance associated with it.
2023-12-11 09:25:11 -06:00
John Murret 5ec84dbfd8
security: update supported envoy version 1.28.0 in addition to 1.25.11, 1.26.6, 1.27.2, 1.28.0 to address CVE-2023-44487 (#19879)
* update too support envoy 1.28.0

* add changelog

* update docs
2023-12-08 14:42:04 -07:00
Michael Zalimeni 1d9234a87a
ci: sanitize commit message for Slack failure alerts (#19876)
To ensure that shell code cannot be injected, capture the commit message
in an env var, then format it as needed.

Also fix several other issues with formatting and JSON escaping by
wrapping the entire message in a `toJSON` expression.
2023-12-08 16:04:45 -05:00
Derek Menteer 0ac958f27b
Fix xDS missing endpoint race condition. (#19866)
This fixes the following race condition:
- Send update endpoints
- Send update cluster
- Recv ACK endpoints
- Recv ACK cluster

Prior to this fix, it would have resulted in the endpoints NOT existing in
Envoy. This occurred because the cluster update implicitly clears the endpoints
in Envoy, but we would never re-send the endpoint data to compensate for the
loss, because we would incorrectly ACK the invalid old endpoint hash. Since the
endpoint's hash did not actually change, they would not be resent.

The fix for this is to effectively clear out the invalid pending ACKs for child
resources whenever the parent changes. This ensures that we do not store the
child's hash as accepted when the race occurs.

An escape-hatch environment variable `XDS_PROTOCOL_LEGACY_CHILD_RESEND` was
added so that users can revert back to the old legacy behavior in the event
that this produces unknown side-effects. Visit the following thread for some
extra context on why certainty around these race conditions is difficult:
https://github.com/envoyproxy/envoy/issues/13009

This bug report and fix was mostly implemented by @ksmiley with some minor
tweaks.

Co-authored-by: Keith Smiley <ksmiley@salesforce.com>
2023-12-08 11:37:12 -06:00
cskh 0ca070b301
upgrade test(LTS): add segments to version 1.10 (#19861) 2023-12-08 12:22:16 -05:00
Matt Keeler d4fda945bb
Fix a test flake where a retry timer was being reused causing tests after the first to exit early (#19864)
Fix a test flake where a retry timer was being reused causing tests after the first to exit too early.
2023-12-08 11:31:59 -05:00
Thomas Eckert 8125a32a4e
Add CE version of Gateway Upstream Disambiguation (#19860)
* Add CE version of gateway-upstream-disambiguation

* Use NamespaceOrDefault and PartitionOrDefault

* Add Changelog entry

* Remove the unneeded reassignment

* Use c.ID()
2023-12-07 17:56:14 -05:00
Dhia Ayachi d93f7f730d
parse config protocol on write to optimize disco-chain compilation (#19829)
* parse config protocol on write to optimize disco-chain compilation

* add changelog
2023-12-07 13:46:46 -05:00
Matt Keeler bfad6a4e07
Ensure that the default namespace always exists even prior to resource creation (#19852) 2023-12-07 13:23:06 -05:00
Poonam Jadhav 06b3038643
Net-6730/namespace intg test (#19798)
test: add intg test for namespace lifecycle
2023-12-07 13:12:45 -05:00
Michael Zalimeni 645cbf9098
chore: update changelog for patch releases (#19855)
* 1.16.3
* 1.15.7
* 1.14.11
2023-12-07 12:43:33 -05:00
Tauhid Anjum ab68ddff91
NET-6784: Adding cli command to list exported services to a peer (#19821)
* Adding cli command to list exported services to a peer

* Changelog added

* Addressing docs comments

* Adding test case for no exported services scenario
2023-12-07 12:55:15 +05:30
Michael Zalimeni 3a78446114
ci: fix escaping for Slack failure notifications (#19838)
Allow '()', '#', and other bash-interpretable special characters by
properly quoting the commit message when shortening.
2023-12-06 21:00:30 +00:00
cskh 04d4412afd
NET-6643: upgrade test from 1.10 to 1.15 (lts) of a single cluster (#19847)
* NET-6643: upgrade test from 1.10 to 1.15 (lts) of a single cluster

* license header
2023-12-06 19:45:37 +00:00
Ronald 053367a3b2
[NET-6650] Bump go version to 1.20.12 (#19840) 2023-12-06 13:22:00 -05:00
Jared Kirschner d3e658b0e7
improve client RPC metrics consistency (#19721)
The client.rpc metric now excludes internal retries for consistency
with client.rpc.exceeded and client.rpc.failed. All of these metrics
now increment at most once per RPC method call, allowing for
accurate calculation of failure / rate limit application occurrence.

Additionally, if an RPC fails because no servers are present,
client.rpc.failed is now incremented.
2023-12-06 13:21:08 -05:00
Matt Keeler efe279f802
Retry lint fixes (#19151)
* Add a make target to run lint-consul-retry on all the modules
* Cleanup sdk/testutil/retry
* Fix a bunch of retry.Run* usage to not use the outer testing.T
* Fix some more recent retry lint issues and pin to v1.4.0 of lint-consul-retry
* Fix codegen copywrite lint issues
* Don’t perform cleanup after each retry attempt by default.
* Use the common testutil.TestingTB interface in test-integ/tenancy
* Fix retry tests
* Update otel access logging extension test to perform requests within the retry block
2023-12-06 12:11:32 -05:00
Ronald dc02fa695f
[NET-6251] Nomad client templated policy (#19827) 2023-12-06 10:32:12 -05:00
aahel 334de1460c
update l7expplicit dest test to test cross tenancy (#19834) 2023-12-06 06:42:19 +00:00
Ashesh Vidyut 6c88122fdb
NET-3860 - [Supportability] consul troubleshoot CLI for verifying ports (#18329)
* init

* udp

* added support for custom port

* removed grpc

* rename constants

* removed udp

* added change log

* fix synopsis

* pr comment chagnes

* make private

* added tests

* added one more test case

* defer close results channel

* removed unwanted comment

* licence update

* updated docs

* fix indent

* fix path

* example update

* Update website/content/commands/troubleshoot/ports.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/commands/troubleshoot/ports.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update command/troubleshoot/ports/troubleshoot_ports.go

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/commands/troubleshoot/ports.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/commands/troubleshoot/index.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update command/troubleshoot/ports/troubleshoot_ports.go

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update command/troubleshoot/ports/troubleshoot_ports.go

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/commands/troubleshoot/ports.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/commands/troubleshoot/ports.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/commands/troubleshoot/ports.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* pr comment resolved

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-12-06 11:12:15 +05:30
cskh b5edf5cd10
doc: clarify the portNames used in trafficpermission V2 (#19807)
* doc: clarify the portNames used in trafficpermission V2 and fix broken links and examples
2023-12-05 19:21:52 +00:00
Semir Patel c1bbda8128
resource: block default namespace deletion + test refactorings (#19822) 2023-12-05 14:00:06 -05:00
Michael Zalimeni aca8a185ca
ci: fix test failure Slack notifications (#19766)
- Skip notifications for cancelled workflows. Cancellation can be
manual or caused by branch concurrency limits.
- Fix multi-line JSON parsing error by only printing the summary line
of the commit message. We do not need more than this in Slack.
- Update Slack webhook name to match purpose.
2023-12-05 10:24:04 -05:00
aahel 649aa5655f
skip TestCatalogUpgrade for consul versions < 1.18.0 (#19811)
skip TestCatalogUpgrade for conul versions < 1.18.0
2023-12-04 18:27:36 +00:00
lornasong edf4610ed9
[Cloud][CC-6925] Updates to pushing server state (#19682)
* Upgrade hcp-sdk-go to latest version v0.73

Changes:
- go get github.com/hashicorp/hcp-sdk-go
- go mod tidy

* From upgrade: regenerate protobufs for upgrade from 1.30 to 1.31

Ran: `make proto`

Slack: https://hashicorp.slack.com/archives/C0253EQ5B40/p1701105418579429

* From upgrade: fix mock interface implementation

After upgrading, there is the following compile error:

cannot use &mockHCPCfg{} (value of type *mockHCPCfg) as "github.com/hashicorp/hcp-sdk-go/config".HCPConfig value in return statement: *mockHCPCfg does not implement "github.com/hashicorp/hcp-sdk-go/config".HCPConfig (missing method Logout)

Solution: update the mock to have the missing Logout method

* From upgrade: Lint: remove usage of deprecated req.ServerState.TLS

Due to upgrade, linting is erroring due to usage of a newly deprecated field

22:47:56 [consul]: make lint
--> Running golangci-lint (.)
agent/hcp/testing.go:157:24: SA1019: req.ServerState.TLS is deprecated: use server_tls.internal_rpc instead. (staticcheck)
                time.Until(time.Time(req.ServerState.TLS.CertExpiry)).Hours()/24,
                                     ^

* From upgrade: adjust oidc error message

From the upgrade, this test started failing:

=== FAIL: internal/go-sso/oidcauth TestOIDC_ClaimsFromAuthCode/failed_code_exchange (re-run 2) (0.01s)
    oidc_test.go:393: unexpected error: Provider login failed: Error exchanging oidc code: oauth2: "invalid_grant" "unexpected auth code"

Prior to the upgrade, the error returned was:
```
Provider login failed: Error exchanging oidc code: oauth2: cannot fetch token: 401 Unauthorized\nResponse: {\"error\":\"invalid_grant\",\"error_description\":\"unexpected auth code\"}\n
```

Now the error returned is as below and does not contain "cannot fetch token"
```
Provider login failed: Error exchanging oidc code: oauth2: "invalid_grant" "unexpected auth code"

```

* Update AgentPushServerState structs with new fields

HCP-side changes for the new fields are in:
https://github.com/hashicorp/cloud-global-network-manager-service/pull/1195/files

* Minor refactor for hcpServerStatus to abstract tlsInfo into struct

This will make it easier to set the same tls-info information to both
 - status.TLS (deprecated field)
 - status.ServerTLSMetadata (new field to use instead)

* Update hcpServerStatus to parse out information for new fields

Changes:
 - Improve error message and handling (encountered some issues and was confused)
 - Set new field TLSInfo.CertIssuer
 - Collect certificate authority metadata and set on TLSInfo.CertificateAuthorities
 - Set TLSInfo on both server.TLS and server.ServerTLSMetadata.InternalRPC

* Update serverStatusToHCP to convert new fields to GNM rpc

* Add changelog

* Feedback: connect.ParseCert, caCerts

* Feedback: refactor and unit test server status

* Feedback: test to use expected struct

* Feedback: certificate with intermediate

* Feedback: catch no leaf, remove expectedErr

* Feedback: update todos with jira ticket

* Feedback: mock tlsConfigurator
2023-12-04 10:25:18 -05:00