Commit Graph

3961 Commits

Author SHA1 Message Date
Daniel Nephin 3390f85ab4 acl: remove ACLTokenTypeManagement 2021-10-25 17:25:14 -04:00
Daniel Nephin 32b4ad42ac acl: remove ACLTokenTypeClient,
along with the last test referencing it.
2021-10-25 17:25:14 -04:00
Daniel Nephin aea4cc5a6d acl: remove legacy arg to store.ACLTokenSet
And remove the tests for legacy=true
2021-10-25 17:25:14 -04:00
Daniel Nephin c77e5747b1 acl: remove EmbeddedPolicy
This method is no longer. It only existed for legacy tokens, which are no longer supported.
2021-10-25 17:25:14 -04:00
Daniel Nephin 121431bf17 acl: remove tests for resolving legacy tokens
The code for this was already removed, which suggests this is not actually testing what it claims.

I'm guessing these are still resolving because the tokens are converted to non-legacy tokens?
2021-10-25 17:25:14 -04:00
Daniel Nephin 0d0761927a acl: stop replication on leadership lost
It seems like this was missing. Previously this was only called by init of ACLs during an upgrade.
Now that legacy ACLs are  removed, nothing was calling stop.

Also remove an unused method from client.
2021-10-25 17:24:12 -04:00
Daniel Nephin 98823e573f Remove incorrect TODO 2021-10-25 17:20:06 -04:00
Daniel Nephin 1344137ce2 acl: move the legacy ACL struct to the one package where it is used
It is now only used for restoring snapshots. We can remove it in phase 2.
2021-10-25 17:20:06 -04:00
Daniel Nephin 531f2f8a3f acl: remove most of the rest of structs/acl_legacy.go 2021-10-25 17:20:06 -04:00
Paul Banks 954b283fec
Merge pull request #11163 from hashicorp/feature/ingress-tls-mixed
Add support for enabling connect-based ingress TLS per listener.
2021-10-25 21:36:01 +01:00
FFMMM fea6f08bf9
fix autopilot_failure_tolerance, add autopilot metrics test case (#11399)
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-10-25 10:55:59 -07:00
FFMMM 0954d261ae
use *telemetry.MetricsPrefix as prometheus.PrometheusOpts.Name (#11290)
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-10-21 13:33:01 -07:00
Dhia Ayachi 58f5686c08
fix leadership transfer on leave suggestions (#11387)
* add suggestions

* set isLeader to false when leadership transfer succeed
2021-10-21 14:02:26 -04:00
Dhia Ayachi f424faffdd
try to perform a leadership transfer when leaving (#11376)
* try to perform a leadership transfer when leaving

* add a changelog
2021-10-21 12:44:31 -04:00
Kyle Havlovitz 04cd2c983e Add new service-exports config entry 2021-10-20 12:24:18 -07:00
Jared Kirschner 14af8cb7a9
Merge pull request #11293 from bisakhmondal/service_filter
expression validation of service-resolver subset filter
2021-10-20 08:57:37 -04:00
Paul Banks c891f30c24 Rebase and rebuild golden files for Envoy version bump 2021-10-19 21:37:58 +01:00
Paul Banks 6faf85bccd Refactor `resolveListenerSDSConfig` to pass in whole config 2021-10-19 20:58:29 +01:00
Paul Banks 78a00f2e1c Add support for enabling connect-based ingress TLS per listener. 2021-10-19 20:58:28 +01:00
Giulio Micheloni a3fb665b88 Restored comment. 2021-10-16 18:05:32 +01:00
Giulio Micheloni fecce25658 Separete test file and no stack trace in ret error 2021-10-16 18:02:03 +01:00
Giulio Micheloni 0c78ddacde Merge branch 'main' of https://github.com/hashicorp/consul into hashicorp-main 2021-10-16 16:59:32 +01:00
R.B. Boyer cc2abb79ba
acl: small OSS refactors to help ensure that auth methods with namespace rules work with partitions (#11323) 2021-10-14 15:38:05 -05:00
freddygv e22f0cc033 Use stored entmeta to fill authzContext 2021-10-14 08:57:40 -06:00
freddygv 53ea1f634a Ensure partition is handled by auto-encrypt 2021-10-14 08:32:45 -06:00
FFMMM 62980ffaa2
fix: only add prom autopilot gauges to servers (#11241)
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-10-13 09:25:30 -07:00
Chris S. Kim c6906b4d37
Update Intentions.List with partitions (#11299) 2021-10-13 10:47:12 -04:00
R.B. Boyer 0c94095dfd
acl: fix bug in 'consul members' filtering with partitions (#11263) 2021-10-13 09:18:16 -05:00
Bisakh Mondal a350a383d3
add service resolver subset filter validation 2021-10-13 02:56:04 +05:30
Connor 257d00c908
Merge pull request #11222 from hashicorp/clly/service-mesh-metrics
Start tracking connect service mesh usage metrics
2021-10-11 14:35:03 -05:00
Connor Kelly 786d2896ff
Replace fmt.Sprintf with function 2021-10-11 12:43:38 -05:00
tarat44 166269f93b preload json values in structs to determine defaults 2021-10-10 17:52:26 -04:00
Daniel Nephin b2f49279e2 ca: split Primary/Secondary Provider
To make it more clear which methods are necessary for each scenario. This can
also prevent problems which force all DCs to use the same Vault instance, which
is currently a problem.
2021-10-10 15:48:02 -04:00
Daniel Nephin 1d14889eca ca: extract primaryUpdateRootCA
This function is only run when the CAManager is a primary. Extracting this function
makes it clear which parts of UpdateConfiguration are run only in the primary and
also makes the cleanup logic simpler. Instead of both a defer and a local var we
can call the cleanup function in two places.
2021-10-10 15:26:55 -04:00
Daniel Nephin 0bc812a8e5 ca: rename functions to use a primary or secondary prefix
This commit renames functions to use a consistent pattern for identifying the functions that
can only be called when the Manager is run as the primary or secondary.

This is a step toward eventually creating separate types and moving these methods off of CAManager.
2021-10-10 15:26:55 -04:00
Daniel Nephin eaea56c7b2 ca: make receiver variable name consistent
Every other method uses c not ca
2021-10-10 15:26:55 -04:00
tarat44 3fe637156c add test cases for h2ping_use_tls default behavior 2021-10-09 17:12:52 -04:00
FFMMM a0bba9171d
fix consul_autopilot_healthy metric emission (#11231)
https://github.com/hashicorp/consul/issues/10730
2021-10-08 10:31:50 -07:00
Connor Kelly a5cf4a9b57
Rename ConfigUsageEnterprise to EnterpriseConfigEntryUsage 2021-10-08 10:53:34 -05:00
Connor Kelly 8c519d5458
Rename and prefix ConfigEntry in Usage table
Rename ConfigUsage functions to ConfigEntry

prefix ConfigEntry kinds with the ConfigEntry table name to prevent
potential conflicts
2021-10-07 16:19:55 -05:00
Connor Kelly 533e7dbe85
Add connect specific prefix to Usage table
Ensure that connect Kind's are separate from ConfigEntry Kind's to
prevent miscounting
2021-10-07 16:16:23 -05:00
tarat44 ecdcfd6360 only set default on H2PingUseTLS if H2PING is set 2021-10-06 22:13:01 -04:00
Daniel Nephin b4e3367e63 docs: add notice that legacy ACLs have been removed.
Add changelog

Also remove a metric that is no longer emitted that was missed in a
previous step.
2021-10-05 18:30:22 -04:00
Daniel Nephin 18b3ac33e8 acl: remove unused translate rules endpoint
The CLI command does not use this endpoint, so we can remove it. It was missed in an
earlier pass.
2021-10-05 18:26:05 -04:00
Connor Kelly 024715eb11
Add changelog, website and metric docs
Add changelog to document what changed.
Add entry to telemetry section of the website to document what changed
Add docs to the usagemetric endpoint to help document the metrics in code
2021-10-05 13:34:24 -05:00
Joshua Montgomery 8eb5915f7d
Fixing SOA record to use alt domain when alt domain in use (#10431) 2021-10-05 10:47:27 -04:00
tarat44 c5479cefe6 fix test 2021-10-05 00:48:09 -04:00
tarat44 ca2e7c2039 fix formatting 2021-10-05 00:15:04 -04:00
tarat44 1e8e44d442 fix formatting 2021-10-05 00:12:23 -04:00
tarat44 c1ed3a9a94 change config option to H2PingUseTLS 2021-10-05 00:12:21 -04:00
tarat44 3c9f5a73d9 add support for h2c in h2 ping health checks 2021-10-04 22:51:08 -04:00
Daniel Nephin ab587f5221
Merge pull request #11182 from hashicorp/dnephin/acl-legacy-remove-upgrade
acl: remove upgrade from legacy, start in non-legacy mode
2021-10-04 17:25:39 -04:00
Evan Culver e808620463
Merge pull request #11118 from hashicorp/eculver/remove-envoy-1.15
Remove support for Envoy 1.15
2021-10-04 23:14:24 +02:00
Evan Culver c7747212c3
Merge pull request #11115 from hashicorp/eculver/envoy-1.19.1
Add support for Envoy 1.19.1
2021-10-04 23:13:26 +02:00
Daniel Nephin c7f74deb17 acl: remove updateEnterpriseSerfTags
The only remaining caller is a test helper, and the tests don't use the enterprise gossip
pools.
2021-10-04 17:01:51 -04:00
Daniel Nephin 9b1d2685bf
Merge pull request #11126 from hashicorp/dnephin/acl-legacy-remove-resolve-and-get-policy
acl: remove ACL.GetPolicy RPC endpoint and ACLResolver.resolveTokenLegacy
2021-10-04 16:29:51 -04:00
Connor Kelly c2583a1b7f
Add metrics to count the number of service-mesh config entries 2021-10-04 14:50:17 -05:00
Connor Kelly 536838b004
Add metrics to count connect native service mesh instances
This will add the counts of the service mesh instances tagged by
whether or not it is connect native
2021-10-04 14:37:05 -05:00
Connor Kelly 46bf882620
Add metrics to count service mesh Kind instance counts
This will add the counts of service mesh instances tagged by the
different ServiceKind's.
2021-10-04 14:36:59 -05:00
Daniel Nephin a1e3fa818c acl: fix test failures caused by remocving legacy ACLs
This commit two test failures:

1. Remove check for "in legacy ACL mode", the actual upgrade will be removed in a following commit.
2. Remove the early WaitForLeader in dc2, because with it the test was
   failing with ACL not found.
2021-10-01 18:03:10 -04:00
Evan Culver db397d62c5
Add 1.15 versions to too old list 2021-10-01 11:28:26 -07:00
Chris S. Kim 1c9b58a8af
agent: Reject partitions in legacy intention endpoints (#11181) 2021-10-01 13:18:57 -04:00
Chris S. Kim 53a35181e5
Support partitions in parseIntentionStringComponent (#11202) 2021-10-01 12:36:12 -04:00
Dhia Ayachi a5b09493ab
fix token list by auth method (#11196)
* add tests to OIDC authmethod and fix entMeta when retrieving auth-methods

* fix oss compilation error
2021-10-01 12:00:43 -04:00
Evan Culver e41830af8a
Merge branch 'eculver/envoy-1.19.1' into eculver/remove-envoy-1.15 2021-09-30 11:32:28 -07:00
Evan Culver fdbb742ffd
regenerate more envoy golden files 2021-09-30 10:57:47 -07:00
Daniel Nephin 4faf805716 acl: call stop for the upgrade goroutine when done
TestAgentLeaks_Server was reporting a goroutine leak without this. Not sure if it would actually
be a leak in production or if this is due to the test setup, but seems easy enough to call it
this way until we remove legacyACLTokenUpgrade.
2021-09-29 17:36:43 -04:00
Daniel Nephin 02da08ce77 acl: only run startACLUpgrade once
Since legacy ACL tokens can no longer be created we only need to run this upgrade a single
time when leadership is estalbished.
2021-09-29 16:22:01 -04:00
Daniel Nephin 3ac910606c acl: remove reading of serf acl tags
We no long need to read the acl serf tag, because servers are always either ACL enabled or
ACL disabled.

We continue to write the tag so that during an upgarde older servers will see the tag.
2021-09-29 15:45:11 -04:00
Daniel Nephin 3d7c07e1e4 acl: fix test failure
For some reason removing legacy ACL upgrade requires using an ACL token now
for this WaitForLeader.
2021-09-29 15:21:30 -04:00
Daniel Nephin 5c721832dc acl: remove legacy ACL upgrades from Server
As part of removing the legacy ACL system
2021-09-29 15:19:23 -04:00
Daniel Nephin 94be1835b2 acl: fix test failures caused by remocving legacy ACLs
This commit two test failures:

1. Remove check for "in legacy ACL mode", the actual upgrade will be removed in a following commit.
2. Use the root token in WaitForLeader, because without it the test was
   failing with ACL not found.
2021-09-29 15:15:50 -04:00
Daniel Nephin 8e9773e20b acl: remove ACL.GetPolicy endpoint and resolve legacy acls
And all code that was no longer used once those two were removed.
2021-09-29 14:33:19 -04:00
Daniel Nephin d12dd48c61 acl: remove ACL upgrading from Clients
As part of removing the legacy ACL system ACL upgrading and the flag for
legacy ACLs is removed from Clients.

This commit also removes the 'acls' serf tag from client nodes. The tag is only ever read
from server nodes.

This commit also introduces a constant for the acl serf tag, to make it easier to track where
it is used.
2021-09-29 14:02:38 -04:00
Daniel Nephin 19040586ce
Merge pull request #11136 from hashicorp/dnephin/acl-resolver-fix-default-authz
acl: fix default Authorizer for down_policy extend-cache/async-cache
2021-09-29 13:45:12 -04:00
Daniel Nephin a53fdd68c8
Merge pull request #11110 from hashicorp/dnephin/acl-legacy-remove-initialize
acl: remove initializeLegacyACL and the rest of the legacy FSM commands
2021-09-29 13:44:30 -04:00
Daniel Nephin 8f754aba14
Merge pull request #10999 from hashicorp/dnephin/revert-config-xds-port
Revert config xds_port
2021-09-29 13:39:15 -04:00
Daniel Nephin cc310224aa command/envoy: stop using the DebugConfig from Self endpoint
The DebugConfig in the self endpoint can change at any time. It's not a stable API.

This commit adds the XDSPort to a stable part of the XDS api, and changes the envoy command to read
this new field.

It includes support for the old API as well, in case a newer CLI is used with an older API, and
adds a test for both cases.
2021-09-29 13:21:28 -04:00
Daniel Nephin 6e1ebd3df7 acl: remove the last of the legacy FSM
Replace it with an implementation that returns an error, and rename some symbols
to use a Deprecated suffix to make it clear.

Also remove the ACLRequest struct, which is no longer referenced.
2021-09-29 12:42:23 -04:00
Daniel Nephin ed928511ca acl: remove bootstrap-init FSM operation 2021-09-29 12:42:23 -04:00
Daniel Nephin dab5d1bdc8 acl: remove initializeLegacyACL from leader init 2021-09-29 12:42:23 -04:00
Daniel Nephin 05f0cc3993 acl: remove ACLDelete FSM command, and state store function
These are no longer used now that ACL.Apply has been removed.
2021-09-29 12:42:23 -04:00
Daniel Nephin 966e50e00e acl: remove legacy field to ACLBoostrap 2021-09-29 12:42:23 -04:00
Daniel Nephin 1502547e38 Revert "Merge pull request #10588 from hashicorp/dnephin/config-fix-ports-grpc"
This reverts commit 74fb650b6b, reversing
changes made to 58bd817336.
2021-09-29 12:28:41 -04:00
Daniel Nephin 0330966315
Merge pull request #11101 from hashicorp/dnephin/acl-legacy-remove-rpc-2
acl: remove legacy ACL.Apply RPC
2021-09-29 12:23:55 -04:00
Daniel Nephin ea4a8343cd
Merge pull request #11177 from hashicorp/dnephin/remove-entmeta-methods
structs: remove EnterpriseMeta helper methods
2021-09-29 12:08:07 -04:00
Daniel Nephin 4c579a49ed
Merge pull request #10986 from hashicorp/dnephin/acl-legacy-remove-rpc
acl: remove legacy ACL RPC - part 1
2021-09-29 12:04:09 -04:00
Daniel Nephin eb632c53a2 structs: rename the last helper method.
This one gets used a bunch, but we can rename it to make the behaviour more obvious.
2021-09-29 11:48:38 -04:00
Daniel Nephin 8d8c1f9d5e structs: remove another helper
We already have a helper funtion.
2021-09-29 11:48:03 -04:00
Daniel Nephin 6d72517682 structs: remove two methods that were only used once each.
These methods only called a single function. Wrappers like this end up making code harder to read
because it adds extra ways of doing things.

We already have many helper functions for constructing these types, we don't need additional methods.
2021-09-29 11:47:03 -04:00
Daniel Nephin 8d1378cc1d
Merge pull request #10988 from hashicorp/dnephin/acl-legacy-remove-config
acl: isolate deprecated config and warn when they are used
2021-09-29 11:40:14 -04:00
Daniel Nephin 6b33e3bfd7
Merge pull request #9456 from hashicorp/dnephin/config-deprecation
config: Use DeprecatedConfig struct for deprecated config fields
2021-09-29 11:37:40 -04:00
Evan Culver 60170dfbe7
Merge remote-tracking branch 'origin/eculver/remove-envoy-1.15' into eculver/remove-envoy-1.15 2021-09-28 16:06:36 -07:00
Evan Culver 4f1a8d4ea6
Fix typo
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
2021-09-29 01:05:45 +02:00
Evan Culver 03e44da9f7
Merge branch 'eculver/envoy-1.19.1' into eculver/remove-envoy-1.15 2021-09-28 15:59:43 -07:00
Evan Culver 9b73e7319d
Merge branch 'main' into eculver/envoy-1.19.1 2021-09-28 15:58:20 -07:00
Chris S. Kim 5c37819d09
Cleanup unnecessary normalizing method (#11169) 2021-09-28 15:31:12 -04:00
Daniel Nephin bf2a3f6e79
Merge pull request #11084 from krastin/krastin-autopilot-loggingtypo
Fix a tiny typo in logging in autopilot.go
2021-09-28 15:11:11 -04:00
Evan Culver 585d9363ed
Merge branch 'main' into eculver/envoy-1.19.1 2021-09-28 11:54:33 -07:00
Chris S. Kim e3248c20c9
agent: Clean up unused built-in proxy config (#11165) 2021-09-28 11:29:10 -04:00
Daniel Nephin cd4e70b34c acl: fix default authorizer for down_policy
This was causing a nil panic because a nil authorizer is no longer valid after the cleanup done
in https://github.com/hashicorp/consul/pull/10632.
2021-09-23 18:12:22 -04:00
Daniel Nephin 6bb7aef15c Remove t.Parallel from TestACLResolver_DownPolicy
These tests run in under 10ms, t.Parallel does nothing but slow them down and
make failures harder to debug when one panics.
2021-09-23 18:12:22 -04:00
Dhia Ayachi e664dbc352
Refactor table index acl phase 2 (#11133)
* extract common methods from oss and ent

* remove unreachable code

* add missing normalize for binding rules

* fix oss to use Query
2021-09-23 15:26:09 -04:00
Daniel Nephin e8ac5fd90b config: Move ACLEnableKeyListPolicy to DeprecatedConfig 2021-09-23 15:15:00 -04:00
Daniel Nephin 5c40b717ed config: move acl_ttl to DeprecatedConfig 2021-09-23 15:14:59 -04:00
Daniel Nephin 977f6d8888 config: move acl_{default,down}_policy to DeprecatedConfig 2021-09-23 15:14:59 -04:00
Daniel Nephin 5eafcea4d4 config: Deprecate EnableACLReplication
replaced by ACL.TokenReplication
2021-09-23 15:14:59 -04:00
Daniel Nephin 5dc16180ad config: move ACL master token and replication to DeprecatedConfig 2021-09-23 15:14:59 -04:00
Paul Banks 1ecec84fd7
Merge pull request #10903 from hashicorp/feature/ingress-sds
Add Support to for providing TLS certificates for Ingress listeners from an SDS source
2021-09-23 16:19:05 +01:00
Dhia Ayachi 5904e4ac79
Refactor table index (#11131)
* convert tableIndex to use the new pattern

* make `indexFromString` available for oss as well

* refactor `indexUpdateMaxTxn`
2021-09-23 11:06:23 -04:00
Paul Banks 7b4cbe3143 Final readability tweaks from review 2021-09-23 10:17:12 +01:00
Paul Banks 70bc89b7f4 Fix subtle loop bug and add test 2021-09-23 10:13:41 +01:00
Paul Banks 07f81991df Refactor SDS validation to make it more contained and readable 2021-09-23 10:13:19 +01:00
Paul Banks 5cfd030d03 Refactor Ingress-specific lister code to separate file 2021-09-23 10:13:19 +01:00
Paul Banks 136928a90f Minor PR typo and cleanup fixes 2021-09-23 10:13:19 +01:00
Paul Banks 20d0bf81f7 Revert abandonned changes to proxycfg for Ent test consistency 2021-09-23 10:13:19 +01:00
Paul Banks a9119e36a5 Fix merge conflict in xds tests 2021-09-23 10:12:37 +01:00
Paul Banks 2281d883b9 Fix some more Enterprise Normalization issues affecting tests 2021-09-23 10:12:37 +01:00
Paul Banks 9fa60c7472 Remove unused argument to fix lint error 2021-09-23 10:09:11 +01:00
Paul Banks 659321d008 Handle namespaces in route names correctly; add tests for enterprise 2021-09-23 10:09:11 +01:00
Paul Banks 2a3d3d3c23 Update xDS routes to support ingress services with different TLS config 2021-09-23 10:08:02 +01:00
Paul Banks 16b3b1c737 Update xDS Listeners with SDS support 2021-09-23 10:08:02 +01:00
Paul Banks ccbda0c285 Update proxycfg to hold more ingress config state 2021-09-23 10:08:02 +01:00
Paul Banks 4e39f03d5b Add ingress-gateway config for SDS 2021-09-23 10:08:02 +01:00
Daniel Nephin c84867feda acl: remove ACL.Apply
As part of removing the legacy ACL system.
2021-09-22 18:28:08 -04:00
Daniel Nephin ae05419aea acl: made acl rules in tests slightly more specific
When converting these tests from the legacy ACL system to the new RPC endpoints I
initially changed most things to use _prefix rules, because that was equivalent to
the old legacy rules.

This commit modifies a few of those rules to be a bit more specific by replacing the _prefix
rule with a non-prefix one where possible.
2021-09-22 18:24:56 -04:00
Mark Anderson d88d9e71c2
partitions/authmethod-index work from enterprise (#11056)
* partitions/authmethod-index work from enterprise

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-09-22 13:19:20 -07:00
Chris S. Kim f972048ebc
connect: Allow upstream listener escape hatch for prepared queries (#11109) 2021-09-22 15:27:10 -04:00
Evan Culver 7e20a5e4f9
connect: remove support for Envoy 1.15 2021-09-22 11:48:50 -07:00
R.B. Boyer 706fc8bcd0
grpc: strip local ACL tokens from RPCs during forwarding if crossing datacenters (#11099)
Fixes #11086
2021-09-22 13:14:26 -05:00
Daniel Nephin 54256fb751 config: Move two more fields to DeprecatedConfig
And add a test for deprecated config fields.
2021-09-22 13:23:03 -04:00
Daniel Nephin 8ed14296ea config: Introduce DeprecatedConfig
This struct allows us to move all the deprecated config options off of
the main config struct, and keeps all the deprecation logic in a single
place, instead of spread across 3+ places.
2021-09-22 13:22:16 -04:00
Evan Culver 2d23f92b35
add 1.19.x versions to test config 2021-09-22 09:30:45 -07:00
Connor 1e3ba26223
Merge pull request #11090 from hashicorp/clly/kv-usage-metrics
Add KVUsage to consul state usage metrics
2021-09-22 11:26:56 -05:00
Connor Kelly ba706501e1
Strip out go 1.17 bits 2021-09-22 11:04:48 -05:00
Matt Keeler a6a359cc80 Add a mock Agent delegate to ease/improve some types of testing 2021-09-22 10:23:01 -04:00
hc-github-team-consul-core 47b99d0b78 auto-updated agent/uiserver/bindata_assetfs.go from commit 9c0233cf5 2021-09-22 13:05:38 +00:00
hc-github-team-consul-core 7efb015ca9 auto-updated agent/uiserver/bindata_assetfs.go from commit cfbd1bb84 2021-09-22 09:26:14 +00:00
Daniel Nephin 72f2199ea1 acl: remove remaining tests that use ACL.Apply
In preparation for removing ACL.Apply.

Tests for ACL.Apply, ACL.GetPolicy, and ACL upgrades were removed
because all 3 of those will be removed shortly.

The forth test appears to be for the ACLResolver cache, so the test was moved to the correct
test file, and the name was updated to make it obvious what is being tested.
2021-09-21 19:35:26 -04:00
Evan Culver 2798383dbc
regenerate envoy golden files 2021-09-21 16:21:00 -07:00
Evan Culver 7605dff46e
add envoy 1.19.1 2021-09-21 15:39:36 -07:00
Daniel Nephin eb991c18c2 fsm: restore the legacy commands
and emit a helpful error message.
2021-09-21 18:35:12 -04:00
Daniel Nephin fa4b33ab8f Convert tests to the new ACL system
In preparation for removing ACL.Apply
2021-09-21 18:35:12 -04:00
Daniel Nephin 5779af1f62 config: use the new ACL system in tests
In preparation for removing ACL.Apply
2021-09-21 17:57:29 -04:00
Daniel Nephin d64409f66f catalog: use the new ACL system in tests
In preparation for removing ACL.Apply
2021-09-21 17:57:29 -04:00
Daniel Nephin 3b9578d7eb Update 4 non-acl tests that used the legacy ACL.Apply
These tests don't really care about the endpoint, they just need some way to create an ACL token.
2021-09-21 17:57:29 -04:00
Daniel Nephin 746f67b3a1 acl: remove two commented out tests for legacy ACL replication
They were commented out in 2018.
2021-09-21 17:57:29 -04:00
Daniel Nephin abd9cd0e15 acl: replace legacy Get and List RPCs with an error impl
These endpoints are being removed as part of the legacy ACL system.
2021-09-21 17:57:29 -04:00
Daniel Nephin e7c63004a8 acl: remove a couple legacy ACL operation constants
structs.ACLForceSet was deprecated 4 years ago, it should be safe to remove now.
ACLBootstrapNow was removed in a recent commit. While it is technically possible that a cluster with mixed version
could still attempt a legacy boostrap, we documented that the legacy system was deprecated in 1.4, so no
clusters that are being upgraded should be attempting a legacy boostrap.
2021-09-21 17:57:29 -04:00
Daniel Nephin 868bfc7a0a acl: Remove unused ACLPolicyIDType 2021-09-21 17:57:29 -04:00
Daniel Nephin aee8a9511d
Merge pull request #10985 from hashicorp/dnephin/acl-legacy-remove-replication
acl: remove legacy ACL replication
2021-09-21 17:56:54 -04:00
Connor 1ddee0680c
Apply suggestions from code review
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
2021-09-21 10:52:46 -05:00
R.B. Boyer b2d17ac448
xds: fix representation of incremental xDS subscriptions (#10987)
Fixes #10563

The `resourceVersion` map was doing two jobs prior to this PR. The first job was
to track what version of every resource we know envoy currently has. The
second was to track subscriptions to those resources (by way of the empty
string for a version). This mostly works out fine, but occasionally leads to
consul removing a resource and accidentally (effectively) unsubscribing at the
same time.

The fix separates these two jobs. When all of the resources for a subscription
are removed we continue to track the subscription until envoy explicitly
unsubscribes
2021-09-21 09:58:56 -05:00
Connor Kelly ae3457b96a
Fix test 2021-09-20 13:44:43 -05:00
Connor Kelly 052f224ee5
Add KVUsage to consul state usage metrics
This change will add the number of entries in the consul KV store to the
already existing usage metrics.
2021-09-20 12:41:54 -05:00
R.B. Boyer 5fe613dd05
xds: ensure the active streams counters are 64 bit aligned on 32 bit systems (#11085) 2021-09-20 11:07:11 -05:00
Krastin Krastev f7b0938633
Update autopilot.go
Fixing a minuscule typo in logging
2021-09-20 14:40:58 +02:00
Freddy 8591620b5d
Merge pull request #11071 from hashicorp/partitions/ixn-decisions 2021-09-16 15:18:23 -06:00
freddygv 49248a0802 Fixup proxycfg tproxy case 2021-09-16 15:05:28 -06:00
freddygv fc8fc060a7 Remove ent checks from oss test 2021-09-16 14:53:28 -06:00
R.B. Boyer faa6fd0919
acl: ensure the global management policy grants all necessary partition privileges (#11072) 2021-09-16 15:53:10 -05:00
freddygv bf7a1358d6 Ensure partition is defaulted in authz 2021-09-16 14:39:01 -06:00
freddygv 47109e0c0c Default the partition in ixn check 2021-09-16 14:39:01 -06:00
freddygv 82d2caa288 Fixup test 2021-09-16 14:39:01 -06:00
freddygv 95a6db9cfa Account for partitions in ixn match/decision 2021-09-16 14:39:01 -06:00
Jeff Widman 2dc62aa0c4
Bump `go-discover` to fix broken dep tree (#10898) 2021-09-16 15:31:22 -04:00
hc-github-team-consul-core 42b7fd3e60 auto-updated agent/uiserver/bindata_assetfs.go from commit 1d9d3349c 2021-09-16 17:31:08 +00:00
R.B. Boyer ca73abdea1
acl: fix intention:*:write checks (#11061)
This is a partial revert of #10793
2021-09-16 11:08:45 -05:00
Freddy cd08a36ce0
Merge pull request #11051 from hashicorp/partitions/fixes 2021-09-16 09:29:00 -06:00
Freddy fcef19f94b
acl: small resolver changes to account for partitions (#11052)
Also refactoring the enterprise side of a test to make it easier to reason about.
2021-09-16 09:17:02 -05:00
freddygv 3f3a61c6e1 Fixup manager tests 2021-09-15 17:24:05 -06:00
freddygv 99c6e4fe41 Default partition in match endpoint 2021-09-15 17:23:52 -06:00
freddygv 77681b9f6c Pass partition to intention match query 2021-09-15 17:23:52 -06:00
freddygv 9cd30e8650 Ensure partition is used for SAN validation 2021-09-15 17:23:48 -06:00
Mark Anderson 9f12fbd3cc
ACL Binding Rules table partitioning (#11044)
* ACL Binding Rules table partitioning

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-09-15 13:26:08 -07:00
hc-github-team-consul-core 02051c141e auto-updated agent/uiserver/bindata_assetfs.go from commit fc14a412f 2021-09-15 18:55:29 +00:00
hc-github-team-consul-core 0eb4a98fab auto-updated agent/uiserver/bindata_assetfs.go from commit b16a6fa03 2021-09-15 17:14:42 +00:00
Dhia Ayachi af21578039
use const instead of literals for `tableIndex` (#11039) 2021-09-15 10:24:04 -04:00
Mark Anderson 6be54052f7
Refactor `indexAuthMethod` in `tableACLBindingRules` (#11029)
* Port consul-enterprise #1123 to OSS

Signed-off-by: Mark Anderson <manderson@hashicorp.com>

* Fixup missing query field

Signed-off-by: Mark Anderson <manderson@hashicorp.com>

* change to re-trigger ci system

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-09-15 09:34:19 -04:00
Freddy ce04ce13dd
Merge pull request #11024 from hashicorp/partitions/rbac 2021-09-14 11:18:19 -06:00
Freddy e18f3c1f6d
Update error texts (#11022)
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-09-14 11:08:06 -06:00
freddygv d90e30f009 Update spiffe ID patterns used for RBAC 2021-09-14 11:00:03 -06:00
freddygv 5e54f253d7 Expand testing of simplifyNotSourceSlice for partitions 2021-09-14 10:55:15 -06:00
freddygv 19da23be28 Expand testing of removeSameSourceIntentions for partitions 2021-09-14 10:55:09 -06:00
freddygv beab0cd962 Account for partition when matching src intentions 2021-09-14 10:55:02 -06:00
Daniel Nephin 1f9479603c
Add failures_before_warning to checks (#10969)
Signed-off-by: Jakub Sokołowski <jakub@status.im>

* agent: add failures_before_warning setting

The new setting allows users to specify the number of check failures
that have to happen before a service status us updated to be `warning`.
This allows for more visibility for detected issues without creating
alerts and pinging administrators. Unlike the previous behavior, which
caused the service status to not update until it reached the configured
`failures_before_critical` setting, now Consul updates the Web UI view
with the `warning` state and the output of the service check when
`failures_before_warning` is breached.

The default value of `FailuresBeforeWarning` is the same as the value of
`FailuresBeforeCritical`, which allows for retaining the previous default
behavior of not triggering a warning.

When `FailuresBeforeWarning` is set to a value higher than that of
`FailuresBeforeCritical it has no effect as `FailuresBeforeCritical`
takes precedence.

Resolves: https://github.com/hashicorp/consul/issues/10680

Signed-off-by: Jakub Sokołowski <jakub@status.im>

Co-authored-by: Jakub Sokołowski <jakub@status.im>
2021-09-14 12:47:52 -04:00
Dhia Ayachi b4d5860197
convert expiration indexed in ACLToken table to use `indexerSingle` (#11018)
* move intFromBool to be available for oss

* add expiry indexes

* remove dead code: `TokenExpirationIndex`

* fix remove indexer `TokenExpirationIndex`

* fix rebase issue
2021-09-13 14:37:16 -04:00
Dhia Ayachi 11f44dfcf8
add locality indexer partitioning (#11016)
* convert `Roles` index to use `indexerSingle`

* split authmethod write indexer to oss and ent

* add index locality

* add locality unit tests

* move intFromBool to be available for oss

* use Bool func

* refactor `aclTokenList` to merge func
2021-09-13 11:53:00 -04:00
Dhia Ayachi ba4ee6e67c
convert `indexAuthMethod` index to use `indexerSingle` (#11014)
* convert `Roles` index to use `indexerSingle`

* fix oss build

* split authmethod write indexer to oss and ent

* add auth method unit tests
2021-09-10 16:56:56 -04:00
Paul Banks b38e84df63 Include namespace and partition in error messages when validating ingress header manip 2021-09-10 21:11:00 +01:00
Paul Banks 1079089f20 Refactor HTTPHeaderModifiers.MergeDefaults based on feedback 2021-09-10 21:11:00 +01:00
Paul Banks 9e4e204e96 Fix enterprise test failures caused by differences in normalizing EnterpriseMeta 2021-09-10 21:11:00 +01:00
Paul Banks 3004eadd08 Fix enterprise discovery chain tests; Fix multi-level split merging 2021-09-10 21:11:00 +01:00
Paul Banks b5ae00d753 Remove unnecessary check 2021-09-10 21:09:24 +01:00
Paul Banks f1c0876b4c Fix discovery chain test fixtures 2021-09-10 21:09:24 +01:00
Paul Banks 1b9632531a Integration tests for all new header manip features 2021-09-10 21:09:24 +01:00
Paul Banks e22cc9c53a Header manip for split legs plumbing 2021-09-10 21:09:24 +01:00
Paul Banks 83fc8723a3 Header manip for service-router plumbed through 2021-09-10 21:09:24 +01:00
Paul Banks f439dfc04f Ingress gateway header manip plumbing 2021-09-10 21:09:24 +01:00
Paul Banks d776a2d236 Add HTTP header manip for router and splitter entries 2021-09-10 21:09:24 +01:00
Paul Banks 46e4041283 Header manip and validation added for ingress-gateway entries 2021-09-10 21:09:24 +01:00
Dhia Ayachi 6cac30aa22
convert `Roles` index to use `indexerMulti` (#11013)
* convert `Roles` index to use `indexerMulti`

* add role test in oss

* fix oss to use the right index func

* preallocate slice
2021-09-10 16:04:33 -04:00
Dhia Ayachi f3f0654038
convert indexPolicies in ACLTokens table to the new index (#11011) 2021-09-10 14:57:37 -04:00
Dhia Ayachi 584faec6e3
convert indexSecret to the new index (#11007) 2021-09-10 09:10:11 -04:00
Dhia Ayachi 6e6cf1c043
convert indexAccessor to the new index (#11002) 2021-09-09 16:28:04 -04:00
Hans Hasselberg 13238dbab6
tls: consider presented intermediates during server connection tls handshake. (#10964)
* use intermediates when verifying

* extract connection state

* remove useless import

* add changelog entry

* golint

* better error

* wording

* collect errors

* use SAN.DNSName instead of CommonName

* Add test for unknown intermediate

* improve changelog entry
2021-09-09 21:48:54 +02:00
Chris S. Kim 9bbfa048a2
Sync enterprise changes to oss (#10994)
This commit updates OSS with files for enterprise-specific admin partitions feature work
2021-09-08 11:59:30 -04:00
Kyle Havlovitz a14950025a
Merge pull request #10984 from hashicorp/mesh-resource
acl: adding a new mesh resource
2021-09-07 15:06:20 -07:00
Dhia Ayachi bc0e4f2f46
partition dicovery chains (#10983)
* partition dicovery chains

* fix default partition for OSS
2021-09-07 16:29:32 -04:00
Daniel Nephin f063402b29 acl: remove ACL.IsSame
The only caller of this method was removed in a recent commit along with replication.
2021-09-03 12:59:12 -04:00
Daniel Nephin d63cef1219 acl: remove legacy ACL replication 2021-09-03 12:42:06 -04:00
R.B. Boyer ee372a854a acl: adding a new mesh resource 2021-09-03 09:12:03 -04:00
Dhia Ayachi ced8329d80
try to infer command partition from node partition (#10981) 2021-09-03 08:37:23 -04:00
Dhia Ayachi 09197c989c
add partition to SNI when partition is non default (#10917) 2021-09-01 10:35:39 -04:00
Freddy 8d83d27674
connect: update envoy supported versions to latest patch release
(#10961)

Relevant advisory: 
https://github.com/envoyproxy/envoy/security/advisories/GHSA-6g4j-5vrw-2m8h
2021-08-31 10:39:18 -06:00
Evan Culver 79c7e73618
rpc: authorize raft requests (#10925) 2021-08-26 15:04:32 -07:00
hc-github-team-consul-core cd3333ad6a auto-updated agent/uiserver/bindata_assetfs.go from commit eeeb91bea 2021-08-26 18:13:08 +00:00
Chris S. Kim 1a9b2f09dd
ent->oss test fix (#10926) 2021-08-26 14:06:49 -04:00
hc-github-team-consul-core 2d66c4ea13 auto-updated agent/uiserver/bindata_assetfs.go from commit a907e1d87 2021-08-26 18:02:18 +00:00
hc-github-team-consul-core a163051dbb auto-updated agent/uiserver/bindata_assetfs.go from commit a0b0ed2bc 2021-08-26 16:06:09 +00:00
Chris S. Kim 45dcc8b553
api: expose upstream routing configurations in topology view (#10811)
Some users are defining routing configurations that do not have associated services. This commit surfaces these configs in the topology visualization. Also fixes a minor internal bug with non-transparent proxy upstream/downstream references.
2021-08-25 15:20:32 -04:00
R.B. Boyer a6d22efb49
acl: some acl authz refactors for nodes (#10909) 2021-08-25 13:43:11 -05:00
hc-github-team-consul-core 11b1dc1f97 auto-updated agent/uiserver/bindata_assetfs.go from commit a777b0a9b 2021-08-25 13:46:51 +00:00
hc-github-team-consul-core 5e31421602 auto-updated agent/uiserver/bindata_assetfs.go from commit 8192dde48 2021-08-25 11:39:14 +00:00
R.B. Boyer 5b6d96d27d
grpc: ensure that streaming gRPC requests work over mesh gateway based wan federation (#10838)
Fixes #10796
2021-08-24 16:28:44 -05:00
hc-github-team-consul-core 4993d877d9 auto-updated agent/uiserver/bindata_assetfs.go from commit 05a28c311 2021-08-24 16:04:24 +00:00
Giulio Micheloni 7fa01105cc Fix merge conflicts 2021-08-22 19:35:08 +01:00
Giulio Micheloni 655da1fc42
Merge branch 'main' into serve-panic-recovery 2021-08-22 20:31:11 +02:00
Giulio Micheloni 4b0eaa4bff grpc, xds: recovery middleware to return and log error in case of panic
1) xds and grpc servers:
   1.1) to use recovery middleware with callback that prints stack trace to log
   1.2) callback turn the panic into a core.Internal error
2) added unit test for grpc server
2021-08-22 19:06:26 +01:00
freddygv 01936ddb70 Avoid passing zero value into variadic 2021-08-20 17:40:33 -06:00
freddygv f52bd80f6d Update comment for test function 2021-08-20 17:40:33 -06:00
freddygv af52d21884 Update prepared query cluster SAN validation
Previously SAN validation for prepared queries was broken because we
validated against the name, namespace, and datacenter for prepared
queries.

However, prepared queries can target:

- Services with a name that isn't their own
- Services in multiple datacenters

This means that the SpiffeID to validate needs to be based on the
prepared query endpoints, and not the prepared query's upstream
definition.

This commit updates prepared query clusters to account for that.
2021-08-20 17:40:33 -06:00
freddygv 85878685b7 Fixup proxy config test fixtures
- The TestNodeService helper created services with the fixed name "web",
and now that name is overridable.

- The discovery chain snapshot didn't have prepared query endpoints so
the endpoints tests were missing data for prepared queries
2021-08-20 17:38:57 -06:00
R.B. Boyer fb27c1b24f
agent: add partition labels to catalog API metrics where appropriate (#10890) 2021-08-20 15:09:39 -05:00
R.B. Boyer d66a43f5f2
fixing various bits of enterprise meta plumbing to be more correct (#10889) 2021-08-20 14:34:23 -05:00
Dhia Ayachi 1950ebbe1f
oss portion of ent #1069 (#10883) 2021-08-20 12:57:45 -04:00
R.B. Boyer ac41e30614
state: partition the nodes.uuid and nodes.meta indexes as well (#10882) 2021-08-19 16:17:59 -05:00
R.B. Boyer 097e1645e3
agent: ensure that most agent behavior correctly respects partition configuration (#10880) 2021-08-19 15:09:42 -05:00
Daniel Nephin 271352dbb7
Merge pull request #10849 from hashicorp/dnephin/contrib-doc-xds-auth
xds: document how authorization works
2021-08-18 13:25:16 -04:00
R.B. Boyer e44bce3c4f
state: partition the usage metrics subsystem (#10867) 2021-08-18 09:27:15 -05:00
Daniel Nephin 8252a2691c xds: document how authorization works 2021-08-17 19:26:34 -04:00
R.B. Boyer 613dd7d053
state: adjust streaming event generation to account for partitioned nodes (#10860)
Also re-enabled some tests that had to be disabled in the prior PR.
2021-08-17 16:49:26 -05:00
R.B. Boyer 310e775a8a
state: partition nodes and coordinates in the state store (#10859)
Additionally:

- partitioned the catalog indexes appropriately for partitioning
- removed a stray reference to a non-existent index named "node.checks"
2021-08-17 13:29:39 -05:00
Daniel Nephin 01bf115c2b acl: small improvements to ACLResolver disable due to RPC error
Remove the error return, so that not handling is not reported as an
error by errcheck. It was returning the error passed as an arg
unmodified so there is no reason to return the same value that was
passed in.

Remove the term upstreams to remove any confusion with the term used in
service mesh.

Remove the AutoDisable field, and replace it with the TTL value, using 0
to indicate the setting is turned off.

Replace "not Before" with "After".

Add some test coverage to show the behaviour is still correct.
2021-08-17 13:34:18 -04:00
Daniel Nephin d5498770fa acl: make ACLDisabledTTL a constant
This field was never user-configurable. We always overwrote the value with 120s from
NonUserSource. However, we also never copied the value from RuntimeConfig to consul.Config,
So the value in NonUserSource was always ignored, and we used the default value of 30s
set by consul.DefaultConfig.

All of this code is an unnecessary distraction because a user can not actually configure
this value.

This commit removes the fields and uses a constant value instad. Someone attempting to set
acl.disabled_ttl in their config will now get an error about an unknown field, but previously
the value was completely ignored, so the new behaviour seems more correct.

We have to keep this field in the AutoConfig response for backwards compatibility, but the value
will be ignored by the client, so it doesn't really matter what value we set.
2021-08-17 13:34:18 -04:00
Daniel Nephin abd2e160f9 Fix test failures
Tests only specified one of the fields, but in production we copy the
value from a single place, so we can do the same in tests.

The AutoConfig test broke because of the problem noticed in a previous
commit. The DisabledTTL is not wired up properly so it reports 0s here.
Changed the test to use an explicit value.
2021-08-17 13:32:52 -04:00
Daniel Nephin 17841248dd config: remove ACLResolver settings from RuntimeConfig 2021-08-17 13:32:52 -04:00
Daniel Nephin 31e034215f acl: remove ACLResolver config fields from consul.Config 2021-08-17 13:32:52 -04:00
Daniel Nephin d4701903f6 acl: replace ACLResolver.Config with its own struct
This is step toward decoupling ACLResolver from the agent/consul
package.
2021-08-17 13:32:52 -04:00
Daniel Nephin c2b24adb5f acl: remove ACLRulesTranslateLegacyToken API endpoint 2021-08-17 13:10:02 -04:00