Commit Graph

20764 Commits

Author SHA1 Message Date
Dan Stough 2793761702
[OSS] Improve xDS Code Coverage - Clusters (#18165)
test: improve xDS cluster code coverage
2023-07-20 18:02:21 -04:00
cskh 7e6ce76fcd
NET-4804: Add dashboard for monitoring consul-k8s (#18208) 2023-07-20 16:35:23 -04:00
cskh 5cd287660a
docs: fix the description of client rpc (#18206) 2023-07-20 16:34:36 -04:00
Blake Covarrubias 2c5a09bb0a
Explicitly enable WebSocket upgrades (#18150)
This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all
proxy types. (API Gateway, Ingress, and Sidecar.)

Fixes #8283
2023-07-20 13:24:43 -07:00
Semir Patel ada767fc9f
resource: Pass resource to Write ACL hook instead of just resource Id [NET-4908] (#18192) 2023-07-20 12:06:29 -05:00
Michael Zalimeni 1c7fcdf188
Fix Backport Assistant PR commenting (#18200)
* Fix Backport Assistant failure PR commenting

For general comments on a PR, it looks like you have to use the `/issue`
endpoint rather than `/pulls`, which requires commit/other
review-specific target details.

This matches the endpoint used in `backport-reminder.yml`.

* Remove Backport Reminder workflow

This is noisy (even when adding multiple labels, individual comments per
label are generated), and likely no longer needed: we haven't had this
work in a long time due to an expired GH token, and we now have better
automation for backport PR assignment.
2023-07-20 11:39:01 -04:00
hashicorp-tsccr[bot] 1ef5dfcfc6
SEC-090: Automated trusted workflow pinning (2023-07-18) (#18174)
Result of tsccr-helper -log-level=info -pin-all-workflows .

Co-authored-by: hashicorp-tsccr[bot] <hashicorp-tsccr[bot]@users.noreply.github.com>
2023-07-19 16:56:50 -07:00
Michael Zalimeni 72999bb735
Update submodules to latest following 1.16.0 (#18197)
Align all our internal use of submodules on the latest versions.
2023-07-19 18:57:34 -04:00
Anita Akaeze 271e5af155
update readme.md (#18191)
u[date readme.md
2023-07-19 18:24:55 -04:00
Ronald 18bc04165c
Improve XDS test coverage: JWT auth edition (#18183)
* Improve XDS test coverage: JWT auth edition

more tests

* test: xds coverage for jwt listeners

---------

Co-authored-by: DanStough <dan.stough@hashicorp.com>
2023-07-19 17:19:00 -04:00
Michael Zalimeni e8dd04d662
[NET-4865] security: Update Go version to 1.20.6 (#18190)
Update Go version to 1.20.6

This resolves [CVE-2023-29406]
(https://nvd.nist.gov/vuln/detail/CVE-2023-29406) for uses of the
`net/http` standard library.

Note that until the follow-up to #18124 is done, the version of Go used
in those impacted tests will need to remain on 1.20.5.
2023-07-19 17:02:18 -04:00
Semir Patel 003370ded0
Call resource mutate hook before validate hook (NET-4907) (#18178) 2023-07-19 13:10:57 -05:00
Michael Zalimeni 29cdb75d05
[NET-4865] Bump golang.org/x/net to 0.12.0 (#18186)
Bump golang.org/x/net to 0.12.0

While not necessary to directly address CVE-2023-29406 (which should be
handled by using a patched version of Go when building), an
accompanying change to HTTP/2 error handling does impact agent code.

See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2
change.

Bump this dependency across our submodules as well for the sake of
potential indirect consumers of `x/net/http`.
2023-07-19 11:37:58 -04:00
David Yu 2e326e2e5f
docs - remove Sentinel from enterprise features list (#18176)
* Update index.mdx
* Update kv.mdx
* Update docs-nav-data.json
* delete sentinel.mdx
* Update redirects.js

---------

Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
2023-07-19 00:30:01 +00:00
Nick Irvine cd3fc9e1d0
add peering_commontopo tests [NET-3700] (#17951)
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
Co-authored-by: NiniOak <anita.akaeze@hashicorp.com>
2023-07-18 16:41:30 -07:00
Tu Nguyen 548a5ca385
Add FIPS reference to consul enterprise docs (#18028)
* Add FIPS reference to consul enterprise docs

* Update website/content/docs/enterprise/index.mdx

Co-authored-by: David Yu <dyu@hashicorp.com>

* remove support for ecs client (fips)

---------

Co-authored-by: David Yu <dyu@hashicorp.com>
2023-07-18 20:35:27 +00:00
Ronald 921445712e
[NET-4792] Add integrations tests for jwt-auth (#18169) 2023-07-18 14:59:01 -04:00
Nick Irvine 62005369b5
Add `testing/deployer` (neé `consul-topology`) [NET-4610] (#17823)
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
2023-07-17 15:15:22 -07:00
wangxinyi7 07fce869af
group and document make file (#17943)
* group and document make file
2023-07-17 12:27:24 -07:00
Ronald 03cf37e7b8
Re-order expected/actual for assertContainerState in consul container tests (#18157)
Re-order expected/actual, consul container tests
2023-07-17 18:11:19 +00:00
Dan Stough 33d898b857
[OSS] test: improve xDS listener code coverage (#18138)
test: improve xDS listener code coverage
2023-07-17 13:49:40 -04:00
David Yu e52ea0ee7a
Change docs to say 168h instead of 7d for server_rejoin_age_max (#18154)
### Description

Addresses
https://github.com/hashicorp/consul/pull/17171#issuecomment-1636930705
2023-07-17 16:53:36 +00:00
Ashesh Vidyut f7c5ba5f90
Support Consul Connect Envoy Command on Windows (#17694)
### Description

Add support for consul connect envoy command on windows. This PR fixes
the comments of PR - https://github.com/hashicorp/consul/pull/15114

### Testing
* Built consul.exe from this branch on windows and hosted here - [AWS
S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip)
* Updated the
[tutorial](https://developer.hashicorp.com/consul/tutorials/developer-mesh/consul-windows-workloads)
and changed the `consul_url.default` value to [AWS
S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip)
* Followed the steps in the tutorial and verified that everything is
working as described.

### PR Checklist

* [x] updated test coverage
* [ ] external facing docs updated
* [x] appropriate backport labels added
* [x] not a security concern

---------

Co-authored-by: Franco Bruno Lavayen <cocolavayen@gmail.com>
Co-authored-by: Jose Ignacio Lorenzo <74208929+joselo85@users.noreply.github.com>
Co-authored-by: Jose Ignacio Lorenzo <joseignaciolorenzo85@gmail.com>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2023-07-17 21:40:07 +05:30
Ronald bcc6a9d752
Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter (#18062)
### Description

<!-- Please describe why you're making this change, in plain English.
-->

- Currently the jwt-auth filter doesn't take into account the service
identity when validating jwt-auth, it only takes into account the path
and jwt provider during validation. This causes issues when multiple
source intentions restrict access to an endpoint with different JWT
providers.
- To fix these issues, rather than use the JWT auth filter for
validation, we use it in metadata mode and allow it to forward the
successful validated JWT token payload to the RBAC filter which will
make the decisions.

This PR ensures requests with and without JWT tokens successfully go
through the jwt-authn filter. The filter however only forwards the data
for successful/valid tokens. On the RBAC filter level, we check the
payload for claims and token issuer + existing rbac rules.

### Testing & Reproduction steps

<!--

* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding

-->

- This test covers a multi level jwt requirements (requirements at top
level and permissions level). It also assumes you have envoy running,
you have a redis and a sidecar proxy service registered, and have a way
to generate jwks with jwt. I mostly use:
https://www.scottbrady91.com/tools/jwt for this.

- first write your proxy defaults
```
Kind = "proxy-defaults"
name = "global"
config {
  protocol = "http"
}
```
- Create two providers 
```
Kind = "jwt-provider"
Name = "auth0"
Issuer = "https://ronald.local"

JSONWebKeySet = {
    Local = {
     JWKS = "eyJrZXlzIjog....."
    }
}
```

```
Kind = "jwt-provider"
Name = "okta"
Issuer = "https://ronald.local"

JSONWebKeySet = {
   Local = {
     JWKS = "eyJrZXlzIjogW3...."
    }
}
```

- add a service intention
```
Kind = "service-intentions"
Name = "redis"

JWT = {
  Providers = [
    {
      Name = "okta"
    },
  ]
}

Sources = [
  {
    Name = "*"
    Permissions = [{
      Action = "allow"
      HTTP = {
        PathPrefix = "/workspace"
      }
      JWT = {
        Providers = [
          {
            Name = "okta"
            VerifyClaims = [
              {
                  Path = ["aud"]
                  Value = "my_client_app"
              },
              {
                Path = ["sub"]
                Value = "5be86359073c434bad2da3932222dabe"
              }
            ]
          },
        ]
      }

    },
    {
      Action = "allow"
      HTTP = {
        PathPrefix = "/"
      }
      JWT = {
        Providers = [
          {
            Name = "auth0"
          },
        ]
      }

    }]
  }
]
```
- generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with
different claims than `/workspace` expects and 1 with correct claims
- connect to your envoy (change service and address as needed) to view
logs and potential errors. You can add: `-- --log-level debug` to see
what data is being forwarded
```
consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502
```
- Make the following requests: 
```
curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v

RBAC filter denied

curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v

RBAC filter denied

curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v

Successful request
```


### TODO

* [x] Update test coverage
* [ ] update integration tests (follow-up PR)
* [x] appropriate backport labels added
2023-07-17 11:32:49 -04:00
Poonam Jadhav 5930518489
fix: update delegateMock used in ENT (#18149)
### Description

<!-- Please describe why you're making this change, in plain English.
-->
The mock is used in `http_ent_test` file which caused lint failures. For
OSS->ENT parity adding the same change here.

### Links

<!--

Include any links here that might be helpful for people reviewing your
PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc).
If there are none, feel free to delete this section.

Please be mindful not to leak any customer or confidential information.
HashiCorp employees may want to use our internal URL shortener to
obfuscate links.

-->

Identified in OSS->ENT [merge
PR](https://github.com/hashicorp/consul-enterprise/pull/6328)

### PR Checklist

* [ ] ~updated test coverage~
* [ ] ~external facing docs updated~
* [x] appropriate backport labels added
* [ ] ~not a security concern~
2023-07-17 09:44:49 -04:00
wangxinyi7 e7194787a7
re org resource type registry (#18133) 2023-07-14 18:00:17 -07:00
Nick Irvine 05b665e856
chore: bump upgrade integrations tests to 1.15, 116 [NET-4743] (#18130) 2023-07-14 17:13:41 -04:00
John Murret 691bc9673a
add a conditional around setting LANFilter.AllSegments to make sure it is valid (#18139)
### Description

This is to correct a code problem because this assumes all segments, but
when you get to Enterprise, you can be in partition that is not the
default partition, in which case specifying all segments does not
validate and fails. This is to correct the setting of this filter with
`AllSegments` to `true` to only occur when in the the `default`
partition.

### Testing & Reproduction steps

<!--

* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding

-->

### Links

<!--

Include any links here that might be helpful for people reviewing your
PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc).
If there are none, feel free to delete this section.

Please be mindful not to leak any customer or confidential information.
HashiCorp employees may want to use our internal URL shortener to
obfuscate links.

-->

### PR Checklist

* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern
2023-07-14 14:53:44 -06:00
John Murret 5af73901a2
[NET-4897] net/http host header is now verified and request.host that contains socked now error (#18129)
### Description

This is related to https://github.com/hashicorp/consul/pull/18124 where
we pinned the go versions in CI to 1.20.5 and 1.19.10.

go 1.20.6 and 1.19.11 now validate request host headers for validity,
including the hostname cannot be prefixed with slashes.

For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname. Prior versions go Go would
clean the host header, and strip slashes in the process, but go1.20.6
and go1.19.11 no longer do, and reject the host header. Around the
community we are seeing that others are intercepting the req.host and if
it starts with a slash or ends with .sock, they changing the host to
localhost or another dummy value.

[client: define a "dummy" hostname to use for local connections by
thaJeztah · Pull Request #45942 ·
moby/moby](https://github.com/moby/moby/pull/45942)

### Testing & Reproduction steps

Check CI tests.

### Links
* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern
2023-07-14 14:53:27 -06:00
Chris S. Kim 747a4c73c1
Fix bug with Vault CA provider (#18112)
Updating RootPKIPath but not IntermediatePKIPath would not update 
leaf signing certs with the new root. Unsure if this happens in practice 
but manual testing showed it is a bug that would break mesh and agent 
connections once the old root is pruned.
2023-07-14 15:58:33 -04:00
Poonam Jadhav 5208ea90e4
NET-4657/add resource service client (#18053)
### Description

<!-- Please describe why you're making this change, in plain English.
-->
Dan had already started on this
[task](https://github.com/hashicorp/consul/pull/17849) which is needed
to start building the HTTP APIs. This just needed some cleanup to get it
ready for review.

Overview:

- Rename `internalResourceServiceClient` to
`insecureResourceServiceClient` for name consistency
- Configure a `secureResourceServiceClient` with auth enabled

### PR Checklist

* [ ] ~updated test coverage~
* [ ] ~external facing docs updated~
* [x] appropriate backport labels added
* [ ] ~not a security concern~
2023-07-14 14:09:02 -04:00
cskh ad6364af9e
Docs: fix unmatched bracket for health checks page (#18134) 2023-07-14 09:44:21 -04:00
Ronald 2229206bbe
Add docs for jwt cluster configuration (#18004)
### Description

<!-- Please describe why you're making this change, in plain English.
-->

- Add jwt-provider docs for jwks cluster configuration. The
configuration was added here:
https://github.com/hashicorp/consul/pull/17978
2023-07-14 11:10:42 +00:00
Jeff Apple 68863b42f8
Add ingress gateway deprecation notices to docs (#18102)
### Description

This adds notices, that ingress gateway is deprecated, to several places
in the product docs where ingress gateway is the topic.

### Testing & Reproduction steps

Tested with a local copy of the website.

### Links

Deprecation of ingress gateway was announced in the Release Notes for
Consul 1.16 and Consul-K8s 1.2. See:

[https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated
)

[https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated)

### PR Checklist

* [N/A] updated test coverage
* [X] external facing docs updated
* [X] appropriate backport labels added
* [X] not a security concern

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-07-13 15:17:32 -07:00
John Murret a2c6953d0d
[NET-4895] ci - api tests and consul container tests error because of dependency bugs with go 1.20.6. Pin go to 1.20.5. (#18124)
### Description
The following jobs started failing when go 1.20.6 was released:
- `go-test-api-1-19`
- `go-test-api-1-20`
- `compatibility-integration-tests`
- `upgrade-integration-tests`

`compatibility-integration-tests` and `compatibility-integration-tests`
to this testcontainers issue:
https://github.com/testcontainers/testcontainers-go/issues/1359. This
issue calls for testcontainers to release a new version when one of
their dependencies is fixed. When that is done, we will unpin the go
versions in `compatibility-integration-tests` and
`compatibility-integration-tests`.

### Testing & Reproduction steps

See these jobs broken in CI and then see them work with this PR.

---------

Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com>
2023-07-13 19:26:35 +00:00
Iryna Shustava c328ba85bd
Split pbmesh.UpstreamsConfiguration as a resource out of pbmesh.Upstreams (#17991)
Configuration that previously was inlined into the Upstreams resource
applies to both explicit and implicit upstreams and so it makes sense to
split it out into its own resource.

It also has other minor changes:
- Renames `proxy.proto` proxy_configuration.proto`
- Changes the type of `Upstream.destination_ref` from `pbresource.ID` to
`pbresource.Reference`
- Adds comments to fields that didn't have them
2023-07-13 13:06:56 -06:00
nv-hashi efe981637b
:ermahgerd "Sevice Mesh" -> "Service Mesh" (#18116)
Just a typo in the docs.
2023-07-12 18:46:16 -07:00
Dan Bond 3b3aa1f260
[NET-4103] ci: build s390x (#18067)
* ci: build s390x

* ci: test s390x

* ci: dev build s390x

* no GOOS

* ent only

* build: publish s390x

* fix syntax error

* fix syntax error again

* fix syntax error again x2

* test branch

* Move s390x conditionals to step level

* remove test branch

---------

Co-authored-by: emilymianeil <eneil@hashicorp.com>
2023-07-12 16:10:34 -07:00
Eddie Rowe d1f5d9b905
api gw 1.16 updates (#18081)
* api gw 1.16 updates

* Apply suggestions from code review

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* update CodeBlockConfig filename

* Apply suggestions from code review

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* remove non-standard intentions page

* Update website/content/docs/api-gateway/configuration/index.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

---------

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-07-12 21:43:22 +00:00
Vijay 2f20c77e4d
Displays Consul version of each nodes in UI nodes section (#17754)
* update UINodes and UINodeInfo response with consul-version info added as NodeMeta, fetched from serf members

* update test cases TestUINodes, TestUINodeInfo

* added nil check for map

* add consul-version in local agent node metadata

* get consul version from serf member and add this as node meta in catalog register request

* updated ui mock response to include consul versions as node meta

* updated ui trans and added version as query param to node list route

* updates in ui templates to display consul version with filter and sorts

* updates in ui - model class, serializers,comparators,predicates for consul version feature

* added change log for Consul Version Feature

* updated to get version from consul service, if for some reason not available from serf

* updated changelog text

* updated dependent testcases

* multiselection version filter

* Update agent/consul/state/catalog.go

comments updated

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

---------

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
2023-07-12 13:34:39 -06:00
John Murret f51a9d29ae
docs - update upgrade index page to not recommend consul leave. (#18100) 2023-07-12 16:56:38 +00:00
Luke Kysow ebfed566b2
Docs for dataplane upgrade on k8s (#18051)
* Docs for dataplane upgrade on k8s

---------

Co-authored-by: David Yu <dyu@hashicorp.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-07-12 16:54:35 +00:00
Tom Davies f472164f05
Pass configured role name to Vault for AWS auth in Connect CA (#17885) 2023-07-12 08:24:12 -07:00
Curt Bushko 51d8eb8e07
Docs: Update proxy lifecycle annotations and consul-dataplane flags (#18075)
* Update proxy lifecycle annotations and consul-dataplane flags
2023-07-11 23:11:38 -04:00
Nick Irvine 3dc6f8fc06
ci: use gotestsum v1.10.1 [NET-4042] (#18088) 2023-07-11 17:13:54 -07:00
Curt Bushko bd5af7fe7d
Update helm docs on main (#18085) 2023-07-11 19:59:44 -04:00
david3a 0e58c89978
Update service-mesh-compare.mdx (#17279)
grammar change
2023-07-11 23:05:13 +00:00
David Yu bfb921229d
docs updates - cluster peering and virtual services (#18069)
* Update route-to-virtual-services.mdx
* Update establish-peering.mdx
2023-07-11 22:37:53 +00:00
Joshua Timmons a30ba335b6
Fix a couple typos in Agent Telemetry Metrics docs (#18080)
* Fix metrics docs

* Add changelog

Signed-off-by: josh <josh.timmons@hashicorp.com>

---------

Signed-off-by: josh <josh.timmons@hashicorp.com>
2023-07-11 20:13:30 +00:00
Dan Stough da79997f3d
test: fix FIPS inline cert test message (#18076) 2023-07-11 11:28:27 -04:00