Commit Graph

858 Commits

Author SHA1 Message Date
John Maguire e47f3216e5
APIGW Normalize Status Conditions (#16994)
* normalize status conditions for gateways and routes

* Added tests for checking condition status and panic conditions for
validating combinations, added dummy code for fsm store

* get rid of unneeded gateway condition generator struct

* Remove unused file

* run go mod tidy

* Update tests, add conflicted gateway status

* put back removed status for test

* Fix linting violation, remove custom conflicted status

* Update fsm commands oss

* Fix incorrect combination of type/condition/status

* cleaning up from PR review

* Change "invalidCertificate" to be of accepted status

* Move status condition enums into api package

* Update gateways controller and generated code

* Update conditions in fsm oss tests

* run go mod tidy on consul-container module to fix linting

* Fix type for gateway endpoint test

* go mod tidy from changes to api

* go mod tidy on troubleshoot

* Fix route conflicted reason

* fix route conflict reason rename

* Fix text for gateway conflicted status

* Add valid certificate ref condition setting

* Revert change to resolved refs to be handled in future PR
2023-04-24 16:22:55 -04:00
Michael Wilkerson 001d540afc
Add sameness group field to prepared queries (#17089)
* added method for converting SamenessGroupConfigEntry
- added new method `ToQueryFailoverTargets` for converting a SamenessGroupConfigEntry's members to a list of QueryFailoverTargets
- renamed `ToFailoverTargets` ToServiceResolverFailoverTargets to distinguish it from `ToQueryFailoverTargets`

* Added SamenessGroup to PreparedQuery
- exposed Service.Partition to API when defining a prepared query
- added a method for determining if a QueryFailoverOptions is empty
- This will be useful for validation
- added unit tests

* added method for retrieving a SamenessGroup to state store

* added logic for using PQ with SamenessGroup
- added branching path for SamenessGroup handling in execute. It will be handled separate from the normal PQ case
- added a new interface so that the `GetSamenessGroupFailoverTargets` can be properly tested
- separated the execute logic into a `targetSelector` function so that it can be used for both failover and sameness group PQs
- split OSS only methods into new PQ OSS files
- added validation that `samenessGroup` is an enterprise only feature

* added documentation for PQ SamenessGroup
2023-04-24 13:21:28 -07:00
Kyle Havlovitz d5277af70d
Add manual virtual IP support to state store (#16815) 2023-04-21 09:19:02 -07:00
Eric Haberkorn 53cdda8d17
Fix a bug with disco chain config entry fetching (#17078)
Before this change, we were not fetching service resolvers (and therefore
service defaults) configuration entries for services on members of sameness
groups.
2023-04-21 09:18:32 -04:00
Eric Haberkorn b1fae05983
Add sameness groups to service intentions. (#17064) 2023-04-20 12:16:04 -04:00
hashicorp-copywrite[bot] 9f81fc01e9
[COMPLIANCE] Add Copyright and License Headers (#16854)
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>
2023-04-20 12:40:22 +00:00
Paul Glass f4406e69b9
[NET-3091] Update service intentions to support jwt provider references (#17037)
* [NET-3090] Add new JWT provider config entry

* Add initial test cases

* update validations for jwt-provider config entry fields

* more validation

* start improving tests

* more tests

* Normalize

* Improve tests and move validate fns

* usage test update

* Add split between ent and oss for partitions

* fix lint issues

* Added retry backoff, fixed tests, removed unused defaults

* take into account default partitions

* use countTrue and add aliases

* omit audiences if empty

* fix failing tests

* add omit-entry

* Add JWT intentions

* generate proto

* fix deep copy issues

* remove extra field

* added some tests

* more tests

* add validation for creating existing jwt

* fix nil issue

* More tests, fix conflicts and improve memdb call

* fix namespace

* add aliases

* consolidate errors, skip duplicate memdb calls

* reworked iteration over config entries

* logic improvements from review

---------

Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
2023-04-19 18:16:39 -04:00
Paul Glass ac200cfec8
[NET-3090] Add new JWT provider config entry (#17036)
* [NET-3090] Add new JWT provider config entry

* Add initial test cases

* update validations for jwt-provider config entry fields

* more validation

* start improving tests

* more tests

* Normalize

* Improve tests and move validate fns

* usage test update

* Add split between ent and oss for partitions

* fix lint issues

* Added retry backoff, fixed tests, removed unused defaults

* take into account default partitions

* use countTrue and add aliases

* omit audiences if empty

* fix failing tests

* add omit-entry

* update copyright headers ids

---------

Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>
2023-04-19 17:54:14 -04:00
Paul Glass 77ecff3209
Permissive mTLS (#17035)
This implements permissive mTLS , which allows toggling services into "permissive" mTLS mode.
Permissive mTLS mode allows incoming "non Consul-mTLS" traffic to be forward unmodified to the application.

* Update service-defaults and proxy-defaults config entries with a MutualTLSMode field
* Update the mesh config entry with an AllowEnablingPermissiveMutualTLS field and implement the necessary validation. AllowEnablingPermissiveMutualTLS must be true to allow changing to MutualTLSMode=permissive, but this does not require that all proxy-defaults and service-defaults are currently in strict mode.
* Update xDS listener config to add a "permissive filter chain" when MutualTLSMode=permissive for a particular service. The permissive filter chain matches incoming traffic by the destination port. If the destination port matches the service port from the catalog, then no mTLS is required and the traffic sent is forwarded unmodified to the application.
2023-04-19 14:45:00 -05:00
Derek Menteer 87324c9ec8
Add PrioritizeByLocality to config entries. (#17007)
This commit adds the PrioritizeByLocality field to both proxy-config
and service-resolver config entries for locality-aware routing. The
field is currently intended for enterprise only, and will be used to
enable prioritization of service-mesh connections to services based
on geographical region / zone.
2023-04-14 15:42:54 -05:00
Derek Menteer 1bcaeabfc3
Remove deprecated service-defaults upstream behavior. (#16957)
Prior to this change, peer services would be targeted by service-default
overrides as long as the new `peer` field was not found in the config entry.
This commit removes that deprecated backwards-compatibility behavior. Now
it is necessary to specify the `peer` field in order for upstream overrides
to apply to a peer upstream.
2023-04-11 10:20:33 -05:00
John Maguire 92be8bd762
APIGW: Routes with duplicate parents should be invalid (#16926)
* ensure route parents are unique when creating an http route

* Ensure tcp route parents are unique

* Added unit tests
2023-04-10 13:20:32 -04:00
Dan Upton 671d5825ca
Raft storage backend (#16619) 2023-04-04 17:30:06 +01:00
Eric Haberkorn a6d69adcf5
Add default resolvers to disco chains based on the default sameness group (#16837) 2023-03-31 14:35:56 -04:00
Derek Menteer 8d40cf9858
Add sameness-group to exported-services config entries (#16836)
This PR adds the sameness-group field to exported-service
config entries, which allows for services to be exported
to multiple destination partitions / peers easily.
2023-03-31 12:36:44 -05:00
Eric Haberkorn 0d1d2fc4c9
add order by locality failover to Consul enterprise (#16791) 2023-03-30 10:08:38 -04:00
Ronald 94ec4eb2f4
copyright headers for agent folder (#16704)
* copyright headers for agent folder

* Ignore test data files

* fix proto files and remove headers in agent/uiserver folder

* ignore deep-copy files
2023-03-28 14:39:22 -04:00
John Maguire c833464daf
Update normalization of route refs (#16789)
* Use merge of enterprise meta's rather than new custom method

* Add merge logic for tcp routes

* Add changelog

* Normalize certificate refs on gateways

* Fix infinite call loop

* Explicitly call enterprise meta
2023-03-28 11:23:49 -04:00
Michael Wilkerson e5d58c59c9
changes to support new PQ enterprise fields (#16793) 2023-03-27 15:40:49 -07:00
Dhia Ayachi 10df4d83aa
add ip rate limiter controller OSS parts (#16790) 2023-03-27 17:00:25 -04:00
John Maguire 351bdc3c0d
Fix struct tags for TCPService enterprise meta (#16781)
* Fix struct tags for TCPService enterprise meta

* Add changelog
2023-03-27 16:17:04 +00:00
Derek Menteer 2236975011
Change partition for peers in discovery chain targets (#16769)
This commit swaps the partition field to the local partition for
discovery chains targeting peers. Prior to this change, peer upstreams
would always use a value of default regardless of which partition they
exist in. This caused several issues in xds / proxycfg because of id
mismatches.

Some prior fixes were made to deal with one-off id mismatches that this
PR also cleans up, since they are no longer needed.
2023-03-24 15:40:19 -05:00
Eric Haberkorn 495ad4c7ef
add enterprise xds tests (#16738) 2023-03-22 14:56:18 -04:00
Melisa Griffin 606f8fbbab
Adds check to verify that the API Gateway is being created with at least one listener 2023-03-20 12:37:30 -04:00
Eric Haberkorn 7477f52a16
add sameness groups to discovery chains (#16671) 2023-03-20 09:12:37 -04:00
Eric Haberkorn eaa39f4ef5
add sameness group support to service resolver failover and redirects (#16664) 2023-03-17 10:48:06 -04:00
Eric Haberkorn 57e034b746
fix confusing spiffe ids in golden tests (#16643) 2023-03-15 14:30:36 -04:00
wangxinyi7 152c75349e
net 2731 ip config entry OSS version (#16642)
* ip config entry

* name changing

* move to ent

* ent version

* renaming

* change format

* renaming

* refactor

* add default values
2023-03-15 11:21:24 -07:00
John Maguire ff5887a99e
Update e2e tests for namespaces (#16627)
* Refactored "NewGatewayService" to handle namespaces, fixed
TestHTTPRouteFlattening test

* Fixed existing http_route tests for namespacing

* Squash aclEnterpriseMeta for ResourceRefs and HTTPServices, accept
namespace for creating connect services and regular services

* Use require instead of assert after creating namespaces in
http_route_tests

* Refactor NewConnectService and NewGatewayService functions to use cfg
objects to reduce number of method args

* Rename field on SidecarConfig in tests from `SidecarServiceName` to
`Name` to avoid stutter
2023-03-15 17:51:36 +00:00
Freddy 724b752ca7
Backport ENT-4704 (#16612) 2023-03-14 14:55:11 -06:00
Derek Menteer f2902e6608
Add sameness-group configuration entry. (#16608)
This commit adds a sameness-group config entry to the API and structs packages. It includes some validation logic and a new memdb index that tracks the default sameness-group for each partition. Sameness groups will simplify the effort of managing failovers / intentions / exports for peers and partitions.

Note that this change purely to introduce the configuration entry and does not include the full functionality of sameness-groups.
2023-03-13 16:19:11 -05:00
Eric Haberkorn e298f506a5
Add Peer Locality to Discovery Chains (#16588)
Add peer locality to discovery chains
2023-03-10 12:59:47 -05:00
Eric Haberkorn 57e2493415
allow setting locality on services and nodes (#16581) 2023-03-10 09:36:15 -05:00
Eric Haberkorn 89de91b263
fix bug that can lead to peering service deletes impacting the state of local services (#16570) 2023-03-08 11:24:03 -05:00
Eric Haberkorn dbaf8bf49c
add agent locality and replicate it across peer streams (#16522) 2023-03-07 14:05:23 -05:00
Melisa Griffin 129eca8fdb
NET-2903 Normalize weight for http routes (#16512)
* NET-2903 Normalize weight for http routes

* Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-03-03 16:39:59 -05:00
Eric Haberkorn 5f81662066
Add support for failover policies (#16505) 2023-03-03 11:12:38 -05:00
Andrew Stucki 4b661d1e0c
Add ServiceResolver RequestTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable (#16495)
* Leverage ServiceResolver ConnectTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable

* Regenerate golden files

* Add RequestTimeout field

* Add changelog entry
2023-03-03 09:37:12 -05:00
Andrew Stucki 641737f32b
[API Gateway] Fix infinite loop in controller and binding non-accepted routes and gateways (#16377) 2023-02-22 14:55:40 -05:00
Andrew Stucki 0972697661
[API Gateway] Various fixes for Config Entry fields (#16347)
* [API Gateway] Various fixes for Config Entry fields

* simplify logic per PR review
2023-02-22 04:02:04 +00:00
Andrew Stucki 18e2ee77ca
[API Gateway] Fix targeting service splitters in HTTPRoutes (#16350)
* [API Gateway] Fix targeting service splitters in HTTPRoutes

* Fix test description
2023-02-22 03:48:26 +00:00
Derek Menteer ad865f549b
Fix issue with peer services incorrectly appearing as connect-enabled. (#16339)
Prior to this commit, all peer services were transmitted as connect-enabled
as long as a one or more mesh-gateways were healthy. With this change, there
is now a difference between typical services and connect services transmitted
via peering.

A service will be reported as "connect-enabled" as long as any of these
conditions are met:

1. a connect-proxy sidecar is registered for the service name.
2. a connect-native instance of the service is registered.
3. a service resolver / splitter / router is registered for the service name.
4. a terminating gateway has registered the service.
2023-02-21 13:59:36 -06:00
Andrew Stucki 7f9ec78932
[API Gateway] Validate listener name is not empty (#16340)
* [API Gateway] Validate listener name is not empty

* Update docstrings and test
2023-02-21 14:12:19 -05:00
Andrew Stucki 4607b535be
Fix HTTPRoute and TCPRoute expectation for enterprise metadata (#16322) 2023-02-17 17:28:49 -05:00
Andrew Stucki 15d2684ecc
Normalize all API Gateway references (#16316) 2023-02-17 21:37:34 +00:00
Matt Keeler 085c0addc0
Protobuf Refactoring for Multi-Module Cleanliness (#16302)
Protobuf Refactoring for Multi-Module Cleanliness

This commit includes the following:

Moves all packages that were within proto/ to proto/private
Rewrites imports to account for the packages being moved
Adds in buf.work.yaml to enable buf workspaces
Names the proto-public buf module so that we can override the Go package imports within proto/buf.yaml
Bumps the buf version dependency to 1.14.0 (I was trying out the version to see if it would get around an issue - it didn't but it also doesn't break things and it seemed best to keep up with the toolchain changes)

Why:

In the future we will need to consume other protobuf dependencies such as the Google HTTP annotations for openapi generation or grpc-gateway usage.
There were some recent changes to have our own ratelimiting annotations.
The two combined were not working when I was trying to use them together (attempting to rebase another branch)
Buf workspaces should be the solution to the problem
Buf workspaces means that each module will have generated Go code that embeds proto file names relative to the proto dir and not the top level repo root.
This resulted in proto file name conflicts in the Go global protobuf type registry.
The solution to that was to add in a private/ directory into the path within the proto/ directory.
That then required rewriting all the imports.

Is this safe?

AFAICT yes
The gRPC wire protocol doesn't seem to care about the proto file names (although the Go grpc code does tack on the proto file name as Metadata in the ServiceDesc)
Other than imports, there were no changes to any generated code as a result of this.
2023-02-17 16:14:46 -05:00
Andrew Stucki 58801cc8aa
Add stricter validation and some normalization code for API Gateway ConfigEntries (#16304)
* Add stricter validation and some normalization code for API Gateway ConfigEntries
2023-02-17 19:22:01 +00:00
Andrew Stucki e4a992c581
Fix hostname alignment checks for HTTPRoutes (#16300)
* Fix hostname alignment checks for HTTPRoutes
2023-02-17 18:18:11 +00:00
Andrew Stucki b3ddd4d24e
Inline API Gateway TLS cert code (#16295)
* Include secret type when building resources from config snapshot

* First pass at generating envoy secrets from api-gateway snapshot

* Update comments for xDS update order

* Add secret type + corresponding golden files to existing tests

* Initialize test helpers for testing api-gateway resource generation

* Generate golden files for new api-gateway xDS resource test

* Support ADS for TLS certificates on api-gateway

* Configure TLS on api-gateway listeners

* Inline TLS cert code

* update tests

* Add SNI support so we can have multiple certificates

* Remove commented out section from helper

* regen deep-copy

* Add tcp tls test

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2023-02-17 12:46:03 -05:00
Derek Menteer 30112288c8
Fix mesh gateways incorrectly matching peer locality. (#16257)
Fix mesh gateways incorrectly matching peer locality.

This fixes an issue where local mesh gateways use an
incorrect address when attempting to forward traffic to a
peered datacenter. Prior to this change it would use the
lan address instead of the wan if the locality matched. This
should never be done for peering, since we must route all
traffic through the remote mesh gateway.
2023-02-16 09:22:41 -06:00