21441 Commits

Author SHA1 Message Date
Jeff Boruszak
e341fa04ec
docs: DNS caching tutorial becomes doc (#21010)
* DNS cache page

* Add page to nav

* Replace old link text

* Page edits

* fix content check error

* formatting fixes

* Heading adjustment

* nav

* It was an H1 error all along
2024-04-24 15:24:19 -07:00
Jeff Boruszak
dbc0889c6f
docs: Enterprise upgrade instruction (#20985)
* Upgrade general process updates

* Add alert + adjust structure

* typo
2024-04-24 14:17:54 +03:00
Jeff Boruszak
4a3c3c0b4a
docs: Redirect fix (#21008)
Redirect fix
2024-04-23 08:38:07 -07:00
Dan Stough
03ab7367a6
feat(dataplane): allow token and tenancy information for proxied DNS (#20899)
* feat(dataplane): allow token and tenancy information for proxied DNS

* changelog
2024-04-22 14:30:43 -04:00
Jeff Boruszak
057ad7e952
docs: Initial HCP Rebrand (#21000)
* Initial rebrand for HCP Terraform

* Apply suggestions from code review

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* path fix and redirect

* reintroduce nav from #20873 and #20994

---------

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>
Co-authored-by: Krastin Krastev <krastin@hashicorp.com>
2024-04-22 16:07:14 +03:00
Jeff Boruszak
d106c7f665
docs: KV tutorial becomes usage doc (#20994)
* Add KV store usage page

* nav typo
2024-04-18 10:50:51 -07:00
Michael Zalimeni
5eea0b6c76
test: force IPv4 on Docker 26+ to fix Envoy int tests (#20986)
As of Docker Engine 26.0.0 (https://github.com/moby/moby/pull/47062),
IPv6 is enabled by default where supported. This causes issues for our
tests attempting to resolve requests to other containers over
localhost, since on Linux IPv6 will be preferred over IPv4 when
available when applying the default behavior defined in RFC3484.

As a workaround, force IPv4 with a flag passed to `docker run`.
2024-04-17 19:49:33 +00:00
sarahalsmiller
08761f16c8
Net 6820 customize mesh gateway limits (#20945)
* add upstream limits to mesh gateway cluster generation

* changelog

* go mod tidy

* readd changelog data

* undo reversion from rebase

* run codegen

* Update .changelog/20945.txt

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* address notes

* gofmt

* clean up

* gofmt

* Update agent/proxycfg/mesh_gateway.go

* gofmt

* nil check

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2024-04-16 10:59:41 -05:00
Nathan Coleman
5e9f02d4be
[NET-8091] Add file-system-certificate config entry for API gateway (#20873)
* Define file-system-certificate config entry

* Collect file-system-certificate(s) referenced by api-gateway onto snapshot

* Add file-system-certificate to config entry kind allow lists

* Remove inapplicable validation

This validation makes sense for inline certificates since Consul server is holding the certificate; however, for file system certificates, Consul server never actually sees the certificate.

* Support file-system-certificate as source for listener TLS certificate

* Add more required mappings for the new config entry type

* Construct proper TLS context based on certificate kind

* Add support or SDS in xdscommon

* Remove unused param

* Adds back verification of certs for inline-certificates

* Undo tangential changes to TLS config consumption

* Remove stray curly braces

* Undo some more tangential changes

* Improve function name for generating API gateway secrets

* Add changelog entry

* Update .changelog/20873.txt

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

* Add some nil-checking, remove outdated TODO

* Update test assertions to include file-system-certificate

* Add documentation for file-system-certificate config entry

Add new doc to nav

* Fix grammar mistake

* Rename watchmaps, remove outdated TODO

---------

Co-authored-by: Melisa Griffin <melisa.griffin@hashicorp.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
2024-04-15 16:45:05 -04:00
Di Sheng
e52b1702e9
FIX: wrong indentation of to block in Example yaml (#20974)
docs: Fix wrong indentation of `to` block in cross-namespace `backendRef` example YAML file
2024-04-11 15:23:15 -07:00
Michael Zalimeni
a8d08e759f
fix: consume ignored entries in CE downgrade via Ent snapshot (#20977)
This operation would previously fail due to unconsumed bytes in the
decoder buffer when reading the Ent snapshot (the first byte of the
record would be misinterpreted as a type indicator, and the remaining
bytes would fail to be deserialized or read as invalid data).

Ensure restore succeeds by decoding the ignored record as an
interface{}, which will consume the record bytes without requiring a
concrete target struct, then moving on to the next record.
2024-04-11 21:08:44 +00:00
Eric Haberkorn
e231f0ee9b
Add an agent config option to diable per tenancy usage metrics. (#20976) 2024-04-11 15:20:09 -04:00
John Murret
d261a987f1
update go-control-plane envoy dependency to 0.12.0 (#20973)
* update go-control-plane envoy dependency to 0.12.0

* add changelog

* go mod tidy

* fix linting issues

* add agent/grpc-internal to the list of SA1019 ignores
2024-04-10 01:23:04 +00:00
Michael Zalimeni
159fcfb2fa
security: ignore test and internal tool modules (#20963) 2024-04-08 17:30:04 -04:00
Michael Zalimeni
ad23e96a32
ci: fix Envoy int test versions (#20964)
Follow-up to #20956
2024-04-08 21:27:38 +00:00
Deniz Onur Duzgun
3152ac3702
security: bump go, x/net and envoy versions (#20956)
* Bump go version

* Bump x/net

* Bump envoy version

* Add changelog

---------

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
2024-04-08 19:18:40 +00:00
Freddy
8659c06a73
Add diagrams for write flow through Raft (#20948)
Add diagrams about write flow through Raft
2024-04-04 09:28:53 -06:00
Jared Kirschner
174f92aa24
docs: fix apply DNS ACL token via CLI (#20951) 2024-04-03 15:28:35 -04:00
sarahalsmiller
be8d572eb2
NET-8524 Remove registation of api gateway controller (#20950)
remove registation of api gateway controller
2024-04-03 19:13:57 +00:00
Nathan Coleman
9af713ff17
[NET-5772] Make tcp external service registered on terminating gw reachable from peered cluster (#19881)
* Include SNI + root PEMs from peered cluster on terminating gw filter chain

This allows an external service registered on a terminating gateway to be exported to and reachable from a peered cluster

* Abstract existing logic into re-usable function

* Regenerate golden files w/ new listener logic

* Add changelog entry

* Use peering bundles that are stable across test runs
2024-04-03 12:38:09 -04:00
Manu Nicolas
3dc27518d2
Documentation: update python SDKs list (#20935)
Update python SDKs

The original python-consul is unmaintained with no activity for 6 years.
The python-consul2 fork has had no activity for 3 years, whether it's commits or responding to PRs and issues.
2024-04-02 04:07:25 +00:00
John Murret
a6d9ad5225
remove self-referencing link on network segments page (#20937) 2024-04-01 08:59:32 -06:00
George Ma
44facc2ea3
chore: remove repetitive words (#20890)
Signed-off-by: availhang <mayangang@outlook.com>
2024-03-28 16:31:55 -07:00
John Murret
39112c7a98
GH-20889 - put conditionals are hcp initialization for consul server (#20926)
* put conditionals are hcp initialization for consul server

* put more things behind configuration flags

* add changelog

* TestServer_hcpManager

* fix TestAgent_scadaProvider
2024-03-28 14:47:11 -06:00
David Yu
2a2e773908
Update Dockerfile: Base image for dev bump (#20919)
Update Dockerfile
2024-03-27 18:40:11 -07:00
David Yu
4259b7b33c
Update Dockerfile: bump alpine (#20897)
* Update Dockerfile: bump alpine

* Create 20897

* Rename 20897 to 20897.txt
2024-03-27 14:43:14 -07:00
wangxinyi7
6212e7ecc1
update changelog for 1.18.1 (#20912)
* update changelog for 1.18.1

* update changelog
2024-03-27 13:10:05 -07:00
Michael Zalimeni
cc959dcdf4
security: triage false positive for go-jose/v3 (#20901)
Per https://osv.dev/vulnerability/GO-2024-2631 this vulnerability is not
present in the version currently used (go-jose/v3@3.0.3).
2024-03-26 21:27:50 +00:00
Melisa Griffin
d7f25631ce
Adds docs to upgrade-specific page to include the removal of the depr… (#20844)
* Adds docs to upgrade-specific page to include the removal of the deprecated API Gateway stanza for 1.19

* Apply suggestions from code review

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

* Remove legacy api-gateway from helm docs

* change .Values.apiGateway to .apiGateway

---------

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
2024-03-26 11:49:55 -04:00
John Murret
20210a8d86
fix broken link on sameness groups page (#20894) 2024-03-25 17:22:20 -06:00
Dan Stough
6026ada0c9
[CE] feat(v2dns): enable v2 dns as default (#20715)
* feat(v2dns): enable v2 dns as default

* changelog
2024-03-25 16:09:01 -04:00
Iryna Shustava
d747b51dab
Handle ACL errors consistently when blocking query timeout is reached. (#20876)
Currently, when a client starts a blocking query and an ACL token expires within
that time, Consul will return ACL not found error with a 403 status code. However,
sometimes if an ACL token is invalidated at the same time as the query's deadline is reached,
Consul will instead return an empty response with a 200 status code.

This is because of the events being executed.
1. Client issues a blocking query request with timeout `t`.
2. ACL is deleted.
3. Server detects a change in ACLs and force closes the gRPC stream.
4. Client resubscribes with the same token and resets its state (view).
5. Client sees "ACL not found" error.

If ACL is deleted before step 4, the client is unaware that the stream was closed due to
an ACL error and will return an empty view (from the reset state) with the 200 status code.

To fix this problem, we introduce another state to the subsciption to indicate when a change
to ACLs has occured. If the server sees that there was an error due to ACL change, it will
re-authenticate the request and return an error if the token is no longer valid.

Fixes #20790
2024-03-22 14:59:54 -06:00
Chris S. Kim
12fd9db45d
Add docs for default_intention_policy (#20886) 2024-03-22 15:33:17 -04:00
Chris S. Kim
f3f2175edd
Update go-jose library (#20888) 2024-03-22 10:54:58 -04:00
Nathan Coleman
fea6926de3
Fix typo in example yaml for MeshService (#20879) 2024-03-19 13:04:53 -04:00
Jared Kirschner
e2b966c896
docs: clarify LTS language (#20875)
* docs: clarify LTS language
2024-03-18 23:06:39 +00:00
Derek Menteer
ac83ac1343
Fix streaming RPCs for agentless. (#20868)
* Fix streaming RPCs for agentless.

This PR fixes an issue where cross-dc RPCs were unable to utilize
the streaming backend due to having the node name set. The result
of this was the agent-cache being utilized, which would cause high
cpu utilization and memory consumption due to the fact that it
keeps queries alive for 72 hours before purging inactive entries.

This resource consumption is compounded by the fact that each pod
in consul-k8s gets a unique token. Since the agent-cache uses the
token as a component of the key, the same query is duplicated for
each pod that is deployed.

* Add changelog.
2024-03-15 14:44:51 -05:00
Derek Menteer
0ac8ae6c3b
Fix xDS deadlock due to syncLoop termination. (#20867)
* Fix xDS deadlock due to syncLoop termination.

This fixes an issue where agentless xDS streams can deadlock permanently until
a server is restarted. When this issue occurs, no new proxies are able to
successfully connect to the server.

Effectively, the trigger for this deadlock stems from the following return
statement:
https://github.com/hashicorp/consul/blob/v1.18.0/agent/proxycfg-sources/catalog/config_source.go#L199-L202

When this happens, the entire `syncLoop()` terminates and stops consuming from
the following channel:
https://github.com/hashicorp/consul/blob/v1.18.0/agent/proxycfg-sources/catalog/config_source.go#L182-L192

Which results in the `ConfigSource.cleanup()` function never receiving a
response and holding a mutex indefinitely:
https://github.com/hashicorp/consul/blob/v1.18.0/agent/proxycfg-sources/catalog/config_source.go#L241-L247

Because this mutex is shared, it effectively deadlocks the server's ability to
process new xDS streams.

----

The fix to this issue involves removing the `chan chan struct{}` used like an
RPC-over-channels pattern and replacing it with two distinct channels:

+ `stopSyncLoopCh` - indicates that the `syncLoop()` should terminate soon.  +
`syncLoopDoneCh` - indicates that the `syncLoop()` has terminated.

Splitting these two concepts out and deferring a `close(syncLoopDoneCh)` in the
`syncLoop()` function ensures that the deadlock above should no longer occur.

We also now evict xDS connections of all proxies for the corresponding
`syncLoop()` whenever it encounters an irrecoverable error. This is done by
hoisting the new `syncLoopDoneCh` upwards so that it's visible to the xDS delta
processing. Prior to this fix, the behavior was to simply orphan them so they
would never receive catalog-registration or service-defaults updates.

* Add changelog.
2024-03-15 13:57:11 -05:00
Derek Menteer
eabff257d7
Various bug-fixes and improvements (#20866)
* Shuffle the list of servers returned by `pbserverdiscovery.WatchServers`.

This randomizes the list of servers to help reduce the chance of clients
all connecting to the same server simultaneously. Consul-dataplane is one
such client that does not randomize its own list of servers.

* Fix potential goroutine leak in xDS recv loop.

This commit ensures that the goroutine which receives xDS messages from
proxies will not block forever if the stream's context is cancelled but
the `processDelta()` function never consumes the message (due to being
terminated).

* Add changelog.
2024-03-15 13:10:48 -05:00
Austin Workman
94c0d783b8
K8s v1 Multiport documentation indentation updates (#20858)
Fixing indentation in service account token secrets and addingn clarity about how they are used.

Co-authored-by: David Yu <dyu@hashicorp.com>
2024-03-14 22:11:47 +00:00
David Yu
2d053c3ac6
docs: Update release notes for 1.17.x for legacy api gateway removal (#20857)
* Update v1_17_x.mdx

* Update v1_3_x.mdx

* Update website/content/docs/release-notes/consul-k8s/v1_3_x.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

---------

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2024-03-14 14:51:15 -07:00
Matt Keeler
8fcafb139c
Add consul snapshot decode command (#20824)
Add snapshot decoding command
2024-03-14 12:59:06 -04:00
Deniz Onur Duzgun
e9029ccd7a
[NET-8368] security: bump Go version to 1.21.8 (#20812)
* [NET-8368] Bump Go version
2024-03-14 09:46:15 -04:00
Chris Hut
bfbc0ee4fd
Revert link existing but better 🪦 (#20830)
* Revert "feat: add alert to link to hcp modal to ask a user refresh a page; up… (#20682)"

This reverts commit dd833d9a3649402e23ced070121e0d0c131f610e.

* Revert "chor: change cluster name param to have datacenter.name as default value (#20644)"

This reverts commit 8425cd0f9017f640cce711dc32e0fa0d136899d8.

* Revert "chor: adds informative error message when acls disabled and read-only… (#20600)"

This reverts commit 9d712ccfc7a67193423f1a102ac2b9d3c6dc3733.

* Revert "Cc 7147 link to hcp modal (#20474)"

This reverts commit 8c05e57ac1fdc27ea74040e2dfc35192ac6d067b.

* Revert "Add nav bar item to show HCP link status and encourage folks to link (#20370)"

This reverts commit 22e6ce0df10091bc66ee7fbf8e5d1c0f158ab5a9.

* Revert "Cc 7145 hcp link status api (#20330)"

This reverts commit 049ca102c41fbf646b07e34f5f69f652de9fbc6c.

* Revert "💜 Cc 7187/purple banner for linking existing clusters (#20275)"

This reverts commit 5119667cd16c527af111c339594a08354b7a5cb0.
2024-03-13 13:59:00 -07:00
Nathan Coleman
cff9161bb5
Add API gateway to index of configuration entries (#20849) 2024-03-13 16:57:12 -04:00
Semir Patel
217e5e1d78
snapshot agent: docs to deprecate top-level snapshot destinations (#20848) 2024-03-13 15:37:27 -05:00
Semir Patel
223714bdea
docs: document support for multiple snapshot destinations (#20829) 2024-03-12 16:37:01 -05:00
Nathan Coleman
eccb144bbf
Fix typo in ingress-gateway docs (#20835) 2024-03-12 21:08:33 +00:00
sarahalsmiller
262f435800
NET-6821 Disable Terminating Gateway Auto Host Header Rewrite (#20802)
* disable terminating gateway auto host rewrite

* add changelog

* clean up unneeded additional snapshot fields

* add new field to docs

* squash

* fix test
2024-03-12 15:37:20 -05:00
David Yu
08bfca0193
docs: K8s docs cleanup (#20820)
* Update install.mdx
* Update install-k8s.mdx
2024-03-11 10:56:37 -07:00