Commit Graph

4970 Commits

Author SHA1 Message Date
Dan Upton 671d5825ca
Raft storage backend (#16619) 2023-04-04 17:30:06 +01:00
cskh a319953576
docs: add envoy to the proxycfg diagram (#16834)
* docs: add envoy to the proxycfg diagram
2023-04-04 09:42:42 -04:00
Freddy f6de5ff635
Allow dialer to re-establish terminated peering (#16776)
Currently, if an acceptor peer deletes a peering the dialer's peering
will eventually get to a "terminated" state. If the two clusters need to
be re-peered the acceptor will re-generate the token but the dialer will
encounter this error on the call to establish:

"failed to get addresses to dial peer: failed to refresh peer server
addresses, will continue to use initial addresses: there is no active
peering for "<<<ID>>>""

This is because in `exchangeSecret().GetDialAddresses()` we will get an
error if fetching addresses for an inactive peering. The peering shows
up as inactive at this point because of the existing terminated state.

Rather than checking whether a peering is active we can instead check
whether it was deleted. This way users do not need to delete terminated
peerings in the dialing cluster before re-establishing them.
2023-04-03 12:07:45 -06:00
Chris S. Kim a5397b1f23
Connect CA Primary Provider refactor (#16749)
* Rename Intermediate cert references to LeafSigningCert

Within the Consul CA subsystem, the term "Intermediate"
is confusing because the meaning changes depending on
provider and datacenter (primary vs secondary). For
example, when using the Consul CA the "ActiveIntermediate"
may return the root certificate in a primary datacenter.

At a high level, we are interested in knowing which
CA is responsible for signing leaf certs, regardless of
its position in a certificate chain. This rename makes
the intent clearer.

* Move provider state check earlier

* Remove calls to GenerateLeafSigningCert

GenerateLeafSigningCert (formerly known
as GenerateIntermediate) is vestigial in
non-Vault providers, as it simply returns
the root certificate in primary
datacenters.

By folding Vault's intermediate cert logic
into `GenerateRoot` we can encapsulate
the intermediate cert handling within
`newCARoot`.

* Move GenerateLeafSigningCert out of PrimaryProvidder

Now that the Vault Provider calls
GenerateLeafSigningCert within
GenerateRoot, we can remove the method
from all other providers that never
used it in a meaningful way.

* Add test for IntermediatePEM

* Rename GenerateRoot to GenerateCAChain

"Root" was being overloaded in the Consul CA
context, as different providers and configs
resulted in a single root certificate or
a chain originating from an external trusted
CA. Since the Vault provider also generates
intermediates, it seems more accurate to
call this a CAChain.
2023-04-03 11:40:33 -04:00
Eric Haberkorn a6d69adcf5
Add default resolvers to disco chains based on the default sameness group (#16837) 2023-03-31 14:35:56 -04:00
Derek Menteer 8d40cf9858
Add sameness-group to exported-services config entries (#16836)
This PR adds the sameness-group field to exported-service
config entries, which allows for services to be exported
to multiple destination partitions / peers easily.
2023-03-31 12:36:44 -05:00
Dan Upton 651549c97d
storage: fix resource leak in Watch (#16817) 2023-03-31 13:24:19 +01:00
Eric Haberkorn 0d1d2fc4c9
add order by locality failover to Consul enterprise (#16791) 2023-03-30 10:08:38 -04:00
Ronald b64674623e
Copyright headers for missing files/folders (#16708)
* copyright headers for agent folder
2023-03-28 18:48:58 -04:00
Ronald 94ec4eb2f4
copyright headers for agent folder (#16704)
* copyright headers for agent folder

* Ignore test data files

* fix proto files and remove headers in agent/uiserver folder

* ignore deep-copy files
2023-03-28 14:39:22 -04:00
John Maguire c833464daf
Update normalization of route refs (#16789)
* Use merge of enterprise meta's rather than new custom method

* Add merge logic for tcp routes

* Add changelog

* Normalize certificate refs on gateways

* Fix infinite call loop

* Explicitly call enterprise meta
2023-03-28 11:23:49 -04:00
Michael Wilkerson e5d58c59c9
changes to support new PQ enterprise fields (#16793) 2023-03-27 15:40:49 -07:00
Semir Patel 440f11203f
Resource service List(..) endpoint (#16753) 2023-03-27 16:25:27 -05:00
Dhia Ayachi 10df4d83aa
add ip rate limiter controller OSS parts (#16790) 2023-03-27 17:00:25 -04:00
Kyle Havlovitz 42c5b29713
Allocate virtual ip for resolver/router/splitter config entries (#16760) 2023-03-27 13:04:24 -07:00
Semir Patel 032aba3175
WatchList(..) endpoint for the resource service (#16726) 2023-03-27 14:37:54 -05:00
John Maguire 351bdc3c0d
Fix struct tags for TCPService enterprise meta (#16781)
* Fix struct tags for TCPService enterprise meta

* Add changelog
2023-03-27 16:17:04 +00:00
Semir Patel 3415689eb6
Read(...) endpoint for the resource service (#16655) 2023-03-27 10:35:39 -05:00
Derek Menteer 2236975011
Change partition for peers in discovery chain targets (#16769)
This commit swaps the partition field to the local partition for
discovery chains targeting peers. Prior to this change, peer upstreams
would always use a value of default regardless of which partition they
exist in. This caused several issues in xds / proxycfg because of id
mismatches.

Some prior fixes were made to deal with one-off id mismatches that this
PR also cleans up, since they are no longer needed.
2023-03-24 15:40:19 -05:00
John Eikenberry 0b1dc4ec36
tests instantiating clients w/o shutting down (#16755)
noticed via their port still in use messages.
2023-03-24 16:54:11 +00:00
Poonam Jadhav 3df271959c
fix: remove unused tenancy category from rate limit spec (#16740) 2023-03-23 12:14:59 -04:00
Dhia Ayachi 3ba0eb5074
delete config when nil (#16690)
* delete config when nil

* fix mock interface implementation

* fix handler test to use the right assertion

* extract DeleteConfig as a separate API.

* fix mock limiter implementation to satisfy the new interface

* fix failing tests

* add test comments
2023-03-22 15:19:54 -04:00
Eric Haberkorn 495ad4c7ef
add enterprise xds tests (#16738) 2023-03-22 14:56:18 -04:00
Eric Haberkorn 3c5c53aa80
fix bug where pqs that failover to a cluster peer dont un-fail over (#16729) 2023-03-22 09:24:13 -04:00
cskh 7f6f6891f7
fix: gracefully fail on invalid port number (#16721) 2023-03-21 22:29:21 -04:00
John Maguire 8dd1d73874
Remove unused are hosts set check (#16691)
* Remove unused are hosts set check

* Remove all traces of unused 'AreHostsSet' parameter

* Remove unused Hosts attribute

* Remove commented out use of snap.APIGateway.Hosts
2023-03-21 16:23:23 +00:00
Nitya Dhanushkodi b9bd2c3780
peering: peering partition failover fixes (#16673)
add local source partition for peered upstreams
2023-03-20 10:00:29 -07:00
John Maguire 1ef9f4dade
Fix route subscription when using namespaces (#16677)
* Fix route subscription when using namespaces

* Update changelog

* Fix changelog entry to reference that the bug was enterprise only
2023-03-20 12:42:30 -04:00
Melisa Griffin 606f8fbbab
Adds check to verify that the API Gateway is being created with at least one listener 2023-03-20 12:37:30 -04:00
Poonam Jadhav 9c64731a56
feat: add category annotation to RPC and gRPC methods (#16646) 2023-03-20 11:24:29 -04:00
Eric Haberkorn 7477f52a16
add sameness groups to discovery chains (#16671) 2023-03-20 09:12:37 -04:00
Andrew Stucki 501b87fd31
[API Gateway] Fix invalid cluster causing gateway programming delay (#16661)
* Add test for http routes

* Add fix

* Fix tests

* Add changelog entry

* Refactor and fix flaky tests
2023-03-17 13:31:04 -04:00
Eric Haberkorn eaa39f4ef5
add sameness group support to service resolver failover and redirects (#16664) 2023-03-17 10:48:06 -04:00
Eric Haberkorn 57e034b746
fix confusing spiffe ids in golden tests (#16643) 2023-03-15 14:30:36 -04:00
wangxinyi7 152c75349e
net 2731 ip config entry OSS version (#16642)
* ip config entry

* name changing

* move to ent

* ent version

* renaming

* change format

* renaming

* refactor

* add default values
2023-03-15 11:21:24 -07:00
John Maguire ff5887a99e
Update e2e tests for namespaces (#16627)
* Refactored "NewGatewayService" to handle namespaces, fixed
TestHTTPRouteFlattening test

* Fixed existing http_route tests for namespacing

* Squash aclEnterpriseMeta for ResourceRefs and HTTPServices, accept
namespace for creating connect services and regular services

* Use require instead of assert after creating namespaces in
http_route_tests

* Refactor NewConnectService and NewGatewayService functions to use cfg
objects to reduce number of method args

* Rename field on SidecarConfig in tests from `SidecarServiceName` to
`Name` to avoid stutter
2023-03-15 17:51:36 +00:00
Freddy 724b752ca7
Backport ENT-4704 (#16612) 2023-03-14 14:55:11 -06:00
Derek Menteer 8f75d99299
Fix issue with trust bundle read ACL check. (#16630)
This commit fixes an issue where trust bundles could not be read
by services in a non-default namespace, unless they had excessive
ACL permissions given to them.

Prior to this change, `service:write` was required in the default
namespace in order to read the trust bundle. Now, `service:write`
to a service in any namespace is sufficient.
2023-03-14 12:24:33 -05:00
Chris S. Kim d5677e5680
Preserve CARoots when updating Vault CA configuration (#16592)
If a CA config update did not cause a root change, the codepath would return early and skip some steps which preserve its intermediate certificates and signing key ID. This commit re-orders some code and prevents updates from generating new intermediate certificates.
2023-03-13 17:32:59 -04:00
Derek Menteer f2902e6608
Add sameness-group configuration entry. (#16608)
This commit adds a sameness-group config entry to the API and structs packages. It includes some validation logic and a new memdb index that tracks the default sameness-group for each partition. Sameness groups will simplify the effort of managing failovers / intentions / exports for peers and partitions.

Note that this change purely to introduce the configuration entry and does not include the full functionality of sameness-groups.
2023-03-13 16:19:11 -05:00
Ashvitha f95ffe0355
Allow HCP metrics collection for Envoy proxies
Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>

Add a new envoy flag: "envoy_hcp_metrics_bind_socket_dir", a directory
where a unix socket will be created with the name
`<namespace>_<proxy_id>.sock` to forward Envoy metrics.

If set, this will configure:
- In bootstrap configuration a local stats_sink and static cluster.
  These will forward metrics to a loopback listener sent over xDS.

- A dynamic listener listening at the socket path that the previously
  defined static cluster is sending metrics to.

- A dynamic cluster that will forward traffic received at this listener
  to the hcp-metrics-collector service.


Reasons for having a static cluster pointing at a dynamic listener:
- We want to secure the metrics stream using TLS, but the stats sink can
  only be defined in bootstrap config. With dynamic listeners/clusters
  we can use the proxy's leaf certificate issued by the Connect CA,
  which isn't available at bootstrap time.

- We want to intelligently route to the HCP collector. Configuring its
  addreess at bootstrap time limits our flexibility routing-wise. More
  on this below.

Reasons for defining the collector as an upstream in `proxycfg`:
- The HCP collector will be deployed as a mesh service.

- Certificate management is taken care of, as mentioned above.

- Service discovery and routing logic is automatically taken care of,
  meaning that no code changes are required in the xds package.

- Custom routing rules can be added for the collector using discovery
  chain config entries. Initially the collector is expected to be
  deployed to each admin partition, but in the future could be deployed
  centrally in the default partition. These config entries could even be
  managed by HCP itself.
2023-03-10 13:52:54 -07:00
Eric Haberkorn e298f506a5
Add Peer Locality to Discovery Chains (#16588)
Add peer locality to discovery chains
2023-03-10 12:59:47 -05:00
Eric Haberkorn 57e2493415
allow setting locality on services and nodes (#16581) 2023-03-10 09:36:15 -05:00
Semir Patel 176945aa86
GRPC stub for the ResourceService (#16528) 2023-03-09 13:40:23 -06:00
Andrew Stucki 040647e0ba
auto-updated agent/uiserver/dist/ from commit 63204b518 (#16587)
Co-authored-by: hc-github-team-consul-core <github-team-consul-core@hashicorp.com>
2023-03-09 13:56:53 -05:00
Eric Haberkorn 89de91b263
fix bug that can lead to peering service deletes impacting the state of local services (#16570) 2023-03-08 11:24:03 -05:00
Eric Haberkorn dbaf8bf49c
add agent locality and replicate it across peer streams (#16522) 2023-03-07 14:05:23 -05:00
John Eikenberry f5641ffccc
support vault auth config for alicloud ca provider
Add support for using existing vault auto-auth configurations as the
provider configuration when using Vault's CA provider with AliCloud.

AliCloud requires 2 extra fields to enable it to use STS (it's preferred
auth setup). Our vault-plugin-auth-alicloud package contained a method
to help generate them as they require you to make an http call to
a faked endpoint proxy to get them (url and headers base64 encoded).
2023-03-07 03:02:05 +00:00
Melisa Griffin fc232326a0
NET-2904 Fixes API Gateway Route Service Weight Division Error 2023-03-06 08:41:57 -05:00
Melisa Griffin 129eca8fdb
NET-2903 Normalize weight for http routes (#16512)
* NET-2903 Normalize weight for http routes

* Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-03-03 16:39:59 -05:00