Commit Graph

4516 Commits

Author SHA1 Message Date
Daniel Nephin f31e0b8b1a
Merge pull request #11661 from hashicorp/dnephin/ca-remove-one-call-to-active-root
ca: remove one call to Provider.ActiveRoot
2022-01-13 16:48:12 -05:00
Kyle Havlovitz 0db874c38b Add virtual IP generation for term gateway backed services 2022-01-12 12:08:49 -08:00
Chris S. Kim 98ea6d1cf1
Fix race with tags (#12041) 2022-01-12 11:24:51 -05:00
Chris S. Kim a0acf9978f
Fix races in anti-entropy tests (#12028) 2022-01-11 14:28:51 -05:00
Mike Morris 1b1a97e8f9
ingress: allow setting TLS min version and cipher suites in ingress gateway config entries (#11576)
* xds: refactor ingress listener SDS configuration

* xds: update resolveListenerSDS call args in listeners_test

* ingress: add TLS min, max and cipher suites to GatewayTLSConfig

* xds: implement envoyTLSVersions and envoyTLSCipherSuites

* xds: merge TLS config

* xds: configure TLS parameters with ingress TLS context from leaf

* xds: nil check in resolveListenerTLSConfig validation

* xds: nil check in makeTLSParameters* functions

* changelog: add entry for TLS params on ingress config entries

* xds: remove indirection for TLS params in TLSConfig structs

* xds: return tlsContext, nil instead of ambiguous err

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>

* xds: switch zero checks to types.TLSVersionUnspecified

* ingress: add validation for ingress config entry TLS params

* ingress: validate listener TLS config

* xds: add basic ingress with TLS params tests

* xds: add ingress listeners mixed TLS min version defaults precedence test

* xds: add more explicit tests for ingress listeners inheriting gateway defaults

* xds: add test for single TLS listener on gateway without TLS defaults

* xds: regen golden files for TLSVersionInvalid zero value, add TLSVersionAuto listener test

* types/tls: change TLSVersion to string

* types/tls: update TLSCipherSuite to string type

* types/tls: implement validation functions for TLSVersion and TLSCipherSuites, make some maps private

* api: add TLS params to GatewayTLSConfig, add tests

* api: add TLSMinVersion to ingress gateway config entry test JSON

* xds: switch to Envoy TLS cipher suite encoding from types package

* xds: fixup validation for TLSv1_3 min version with cipher suites

* add some kitchen sink tests and add a missing struct tag

* xds: check if mergedCfg.TLSVersion is in TLSVersionsWithConfigurableCipherSuites

* xds: update connectTLSEnabled comment

* xds: remove unsued resolveGatewayServiceTLSConfig function

 * xds: add makeCommonTLSContextFromLeafWithoutParams

* types/tls: add LessThan comparator function for concrete values

* types/tls: change tlsVersions validation map from string to TLSVersion keys

* types/tls: remove unused envoyTLSCipherSuites

* types/tls: enable chacha20 cipher suites for Consul agent

* types/tls: remove insecure cipher suites from allowed config

TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 and TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 are both explicitly listed as insecure and disabled in the Go source.

Refs https://cs.opensource.google/go/go/+/refs/tags/go1.17.3:src/crypto/tls/cipher_suites.go;l=329-330

* types/tls: add ValidateConsulAgentCipherSuites function, make direct lookup map private

* types/tls: return all unmatched cipher suites in validation errors

* xds: check that Envoy API value matching TLS version is found when building TlsParameters

* types/tls: check that value is found in map before appending to slice in MarshalEnvoyTLSCipherSuiteStrings

* types/tls: cast to string rather than fmt.Printf in TLSCihperSuite.String()

* xds: add TLSVersionUnspecified to list of configurable cipher suites

* structs: update note about config entry warning

* xds: remove TLS min version cipher suite unconfigurable test placeholder

* types/tls: update tests to remove assumption about private map values

Co-authored-by: R.B. Boyer <rb@hashicorp.com>
2022-01-11 11:46:42 -05:00
Dao Thanh Tung 88c7cfa578
URL-encode/decode resource names for HTTP API part 2 (#11957) 2022-01-11 08:52:45 -05:00
Daniel Nephin d57dec5878 ca: remove unnecessary var, and slightly reduce cyclo complexity
`newIntermediate` is always equal to `needsNewIntermediate`, so we can
remove the extra variable and use the original directly.

Also remove the `activeRoot.ID != newActiveRoot.ID` case from an if,
because that case is already checked above, and `needsNewIntermediate` will
already be true in that case.

This condition now reads a lot better:

> Persist a new root if we did not have one before, or if generated a new intermediate.
2022-01-06 16:56:49 -05:00
Daniel Nephin 0de7efb316 ca: remove unused provider.ActiveRoot call
In the previous commit the single use of this storedRoot was removed.

In this commit the original objective is completed. The
Provider.ActiveRoot is being removed because

1. the secondary should get the active root from the Consul primary DC,
   not the provider, so that secondary DCs do not need to communicate
   with a provider instance in a different DC.
2. so that the Provider.ActiveRoot interface can be changed without
   impacting other code paths.
2022-01-06 16:56:48 -05:00
Daniel Nephin d0578c6dfc ca: extract the lookup of the active primary CA
This method had only one caller, which always looked for the active
root. This commit moves the lookup into the method to reduce the logic
in the one caller.

This is being done in preparation for a larger change. Keeping this
separate so it is easier to see.

The `storedRootID != primaryRoots.ActiveRootID` is being removed because
these can never be different.

The `storedRootID` comes from `provider.ActiveRoot`, the
`primaryRoots.ActiveRootID` comes from the store `CARoot` from the
primary. In both cases the source of the data is the primary DC.

Technically they could be different if someone modified the provider
outside of Consul, but that would break many things, so is not a
supported flow.

If these were out of sync because of ordering of events then the
secondary will soon receive an update to `primaryRoots` and everything
will be sorted out again.
2022-01-06 16:56:48 -05:00
Daniel Nephin 7121c78d34 ca: update godoc
To clarify what to expect from the data stored in this field, and the
behaviour of this function.
2022-01-06 16:56:48 -05:00
Daniel Nephin abac8baa5d ca: remove one call to provider.ActiveRoot
ActiveRoot should not be called from the secondary DC, because there
should not be a requirement to run the same Vault instance in a
secondary DC. SignIntermediate is called in a secondary DC, so it should
not call ActiveRoot

We would also like to change the interface of ActiveRoot so that we can
support using an intermediate cert as the primary CA in Consul. In
preparation for making that change I am reducing the number of calls to
ActiveRoot, so that there are fewer code paths to modify when the
interface changes.

This change required a change to the mockCAServerDelegate we use in
tests. It was returning the RootCert for SignIntermediate, but that is
not an accurate fake of production. In production this would also be a
separate cert.
2022-01-06 16:55:50 -05:00
Daniel Nephin eaa084fd41 ca: remove redundant append of an intermediate cert
Immediately above this line we are already appending the full list of
intermediates. The `provider.ActiveIntermediate` MUST be in this list of
intermediates because it must be available to all the other non-leader
Servers.  If it was not in this list of intermediates then any proxy
that received data from a non-leader would have the wrong certs.

This is being removed now because we are planning on changing the
`Provider.ActiveIntermediate` interface, and removing these extra calls ahead of
time helps make that change easier.
2022-01-06 16:55:50 -05:00
Daniel Nephin 11f4cdaa49 ca: only generate a single private key for the whole test case
Using tracing and cpu profiling I found that the majority of the time in
these test cases is spent generating a private key. We really don't need
separate private keys, so we can generate only one and use it for all
cases.

With this change the test runs much faster.
2022-01-06 16:55:50 -05:00
Daniel Nephin b3ffe7ac72 ca: cleanup a test
Fix the name to match the function it is testing

Remove unused code

Fix the signature, instead of returning (error, string) which should be (string, error)
accept a testing.T to emit errors.

Handle the error from encode.
2022-01-06 16:55:49 -05:00
Daniel Nephin 1fd6b16399 ca: use the new leaf signing lookup func in leader metrics 2022-01-06 16:55:49 -05:00
Blake Covarrubias 4bd92921f4
api: Return 404 when deregistering a non-existent check (#11950)
Update the `/agent/check/deregister/` API endpoint to return a 404
HTTP response code when an attempt is made to de-register a check ID
that does not exist on the agent.

This brings the behavior of /agent/check/deregister/ in line with the
behavior of /agent/service/deregister/ which was changed in #10632 to
similarly return a 404 when de-registering non-existent services.

Fixes #5821
2022-01-06 12:38:37 -08:00
Dhia Ayachi 1eac39ae9c
clone the service under lock to avoid a data race (#11940)
* clone the service under lock to avoid a data race

* add change log

* create a struct and copy the pointer to mutate it to avoid a data race

* fix failing test

* revert added space

* add comments, to clarify the data race.
2022-01-06 14:33:06 -05:00
Daniel Nephin 065f6f89fb
Merge pull request #11918 from hashicorp/dnephin/tob-followup
Fix a few small bugs
2022-01-05 18:50:48 -05:00
Daniel Nephin abfc1e4840 snapshot: return the error from replyFn
The only function passed to SnapshotRPC today always returns a nil error, so there's no
way to exercise this bug in practice. This change is being made for correctness so that
it doesn't become a problem in the future, if we ever pass a different function to
SnapshotRPC.
2022-01-05 17:51:03 -05:00
Daniel Nephin 0166b0839c config: correctly capture all errors.
Some calls to multierror.Append were not using the existing b.err, which meant we
were losing all previous errors.
2022-01-05 17:51:03 -05:00
Chris S. Kim 4cd2542a3e
Fix test for ENT (#11946) 2022-01-05 15:18:08 -05:00
Chris S. Kim e4bcaac08c
Fix test for ENT (#11941) 2022-01-05 12:24:44 -05:00
Dhia Ayachi e653f81919
reset `coalesceTimer` to nil as soon as the event is consumed (#11924)
* reset `coalesceTimer` to nil as soon as the event is consumed

* add change log

* refactor to add relevant test.

* fix linter

* Apply suggestions from code review

Co-authored-by: Freddy <freddygv@users.noreply.github.com>

* remove non needed check

Co-authored-by: Freddy <freddygv@users.noreply.github.com>
2022-01-05 12:17:47 -05:00
Mathew Estafanous 0fdd1318e9
Ensure consistency with error-handling across all handlers. (#11599) 2022-01-05 12:11:03 -05:00
Jared Kirschner b393c90ce7 Clarify service and check error messages (use ID)
Error messages related to service and check operations previously included
the following substrings:
- service %q
- check %q

From this error message, it isn't clear that the expected field is the ID for
the entity, not the name. For example, if the user has a service named test,
the error message would read 'Unknown service "test"'. This is misleading -
a service with that *name* does exist, but not with that *ID*.

The substrings above have been modified to make it clear that ID is needed,
not name:
- service with ID %q
- check with ID %q
2022-01-04 11:42:37 -08:00
Jared Kirschner a36ddc31c7
Merge pull request #11335 from littlestar642/url-encoded-args
URL-encode/decode resource names for HTTP API
2022-01-04 14:00:14 -05:00
Chris S. Kim 30550f2c63
testing: Revert assertion for virtual IP flag (#11932) 2022-01-04 11:24:56 -05:00
Jared Kirschner e0ddb9e4c5
Merge pull request #11820 from hashicorp/improve-ui-disabled-api-response
http: improve UI not enabled response message
2022-01-03 12:00:01 -05:00
littlestar642 634c72d22f add path escape and unescape to path params 2022-01-03 08:18:32 -08:00
Daniel Nephin 1683da66b0
Merge pull request #11796 from hashicorp/dnephin/cleanup-test-server
testing: stop using an old version in testServer
2021-12-22 16:04:04 -05:00
freddygv 21f2c2e68d Purge chain if it shouldn't be there 2021-12-13 18:56:44 -07:00
freddygv fe85138453 additional test fixes 2021-12-13 18:56:44 -07:00
freddygv d26b4860fd Account for new upstreams constraint in tests 2021-12-13 18:56:28 -07:00
freddygv 2fe27b748d Check ingress upstreams when gating chain watches 2021-12-13 18:56:28 -07:00
freddygv 6814e84459 Use ptr receiver in all Upstream methods 2021-12-13 18:56:14 -07:00
freddygv 6af9a0d8cf Avoid storing chain without an upstream 2021-12-13 18:56:14 -07:00
freddygv ba12dc215b Clean up chains separately from their watches 2021-12-13 18:56:14 -07:00
freddygv c5c290c503 Validate chains are associated with upstreams
Previously we could get into a state where discovery chain entries were
not cleaned up after the associated watch was cancelled. These changes
add handling for that case where stray chain references are encountered.
2021-12-13 18:56:13 -07:00
freddygv 70d6358426 Store intention upstreams in snapshot 2021-12-13 18:56:13 -07:00
R.B. Boyer 81ea8129d7
proxycfg: ensure all of the watches are canceled if they are cancelable (#11824) 2021-12-13 15:56:17 -06:00
Jared Kirschner f81dd817ff
Merge pull request #11818 from hashicorp/improve-url-not-found-response
http: improve 404 Not Found response message
2021-12-13 16:08:50 -05:00
R.B. Boyer 4aabbe529c
proxycfg: use external addresses in tproxy when crossing partition boundaries (#11823) 2021-12-13 14:34:49 -06:00
Jared Kirschner 2de79abc00 http: improve 404 Not Found response message
When a URL path is not found, return a non-empty message with the 404 status
code to help the user understand what went wrong. If the URL path was not
prefixed with '/v1/', suggest that may be the cause of the problem (which is a
common mistake).
2021-12-13 11:03:25 -08:00
Freddy 85fe875d07
Use anonymousToken when querying by secret ID (#11813)
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Dan Upton <daniel@floppy.co>

This query has been incorrectly querying by accessor ID since New ACLs
were added. However, the legacy token compat allowed this to continue to
work, since it made a fallback query for the anonymousToken ID.

PR #11184 removed this legacy token query, which means that the query by
accessor ID is now the only check for the anonymous token's existence.

This PR updates the GetBySecret call to use the secret ID of the token.
2021-12-13 10:56:09 -07:00
R.B. Boyer 631c649291
various partition related todos (#11822) 2021-12-13 11:43:33 -06:00
Jared Kirschner 34ea9ae8c9 http: improve UI not enabled response message
Response now clearly indicates:
- the UI is disabled
- how to enable the UI
2021-12-13 08:48:33 -08:00
Kyle Havlovitz b50ef696c6
Merge pull request #11812 from hashicorp/metrics-ui-acls
oss: use wildcard partition in metrics proxy ui endpoint
2021-12-10 16:24:47 -08:00
Kyle Havlovitz 9dcaf0539c
Merge pull request #11798 from hashicorp/vip-goroutine-check
leader: move the virtual IP version check into a goroutine
2021-12-10 15:59:35 -08:00
Kyle Havlovitz 018693b6ee acl: use wildcard partition in metrics proxy ui endpoint 2021-12-10 15:58:17 -08:00
Kyle Havlovitz 80a4489844 state: fix freed VIP table id index 2021-12-10 14:41:45 -08:00
Kyle Havlovitz ecbd3eb2a6 Exit before starting the vip check routine if possible 2021-12-10 14:30:50 -08:00
Daniel Nephin 0a9cb62859 testing: Deprecate functions for creating a server.
These helper functions actually end up hiding important setup details
that should be visible from the test case. We already have a convenient
way of setting this config when calling newTestServerWithConfig.
2021-12-09 20:09:29 -05:00
Daniel Nephin c9a992f5e8 testing: remove old config.Build version
DefaultConfig already sets the version to version.Version, so by removing this
our tests will run with the version that matches the code.
2021-12-09 20:09:29 -05:00
Kyle Havlovitz 04ef1c3fa0 leader: move the virtual IP version check into a goroutine 2021-12-09 17:00:33 -08:00
FFMMM 74eb257b1c
[sync ent] increase segment max limit to 4*64, make configurable (#1424) (#11795)
* commit b6eb27563e747a78b7647d2b5da405e46364cc46
Author: FFMMM <FFMMM@users.noreply.github.com>
Date:   Thu Dec 9 13:53:44 2021 -0800

    increase segment max limit to 4*64, make configurable (#1424)

    Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* fix: rename ent changelog file

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-12-09 15:36:11 -08:00
Daniel Nephin f9647ece05
Merge pull request #11780 from hashicorp/dnephin/ca-test-vault-in-secondary
ca: improve test coverage for RenewIntermediate
2021-12-09 12:29:43 -05:00
R.B. Boyer bb75e63eb4
agent: ensure service maintenance checks for matching partitions ahead of other errors (#11788)
This matches behavior in most other agent api endpoints.
2021-12-09 10:05:02 -06:00
Daniel Nephin 4116a143e0 fix misleading errors on vault shutdown 2021-12-08 18:42:52 -05:00
Daniel Nephin 968aeff1bb ca: prune some unnecessary lookups in the tests 2021-12-08 18:42:52 -05:00
Daniel Nephin 305655a8b1 ca: remove duplicate WaitFor function 2021-12-08 18:42:52 -05:00
Daniel Nephin 1dec6bb815 ca: fix flakes in RenewIntermediate tests
I suspect one problem was that we set structs.IntermediateCertRenewInterval to 1ms, which meant
that in some cases the intermediate could renew before we stored the original value.

Another problem was that the 'wait for intermediate' loop was calling the provider.ActiveIntermediate,
but the comparison needs to use the RPC endpoint to accurately represent a user request. So
changing the 'wait for' to use the state store ensures we don't race.

Also moves the patching into a separate function.

Removes the addition of ca.CertificateTimeDriftBuffer as part of calculating halfTime. This was added
in a previous commit to attempt to fix the flake, but it did not appear to fix the problem. Adding the
time here was making the tests fail when using the shared patch
function. It's not clear to me why, but there's no reason we should be
including this time in the halfTime calculation.
2021-12-08 18:42:52 -05:00
Daniel Nephin 2e4e8bd791 ca: improve RenewIntermediate tests
Use the new verifyLearfCert to show the cert verifies with intermediates
from both sources. This required using the RPC interface so that the
leaf pem was constructed correctly.

Add IndexedCARoots.Active since that is a common operation we see in a
few places.
2021-12-08 18:42:52 -05:00
Daniel Nephin a4ba1f348d ca: add a test for Vault in secondary DC 2021-12-08 18:42:51 -05:00
Daniel Nephin a5d9b1d322 ca: Add CARoots.Active method
Which will be used in the next commit.
2021-12-08 18:41:51 -05:00
R.B. Boyer 5f5720837b
acl: ensure that the agent recovery token is properly partitioned (#11782) 2021-12-08 17:11:55 -06:00
Daniel Nephin f72e285fe8
Merge pull request #11721 from hashicorp/dnephin/ca-export-fsm-operation
ca: use the real FSM operation in tests
2021-12-08 17:49:00 -05:00
Daniel Nephin 214dcf8d0d ca: use the real FSM operation in tests
Previously we had a couple copies that reproduced the FSM operation.
These copies introduce risk that the test does not accurately match
production.

This PR removes the test versions of the FSM operation, and exports the
real production FSM operation so that it can be used in tests.

The consul provider tests did need to change because of this. Previously
we would return a hardcoded value of 2, but in production this value is
always incremented.
2021-12-08 17:29:44 -05:00
R.B. Boyer 592ac8f96a
test: test server should auto cleanup (#11779) 2021-12-08 13:26:06 -06:00
Evan Culver 7a365fa0da
rpc: Unset partition before forwarding to remote datacenter (#11758) 2021-12-08 11:02:14 -08:00
Daniel Nephin dccd3f5806 Merge remote-tracking branch 'origin/main' into serve-panic-recovery 2021-12-07 16:30:41 -05:00
Dan Upton 7efab269c0
Rename `Master` and `AgentMaster` fields in config protobuf (#11764) 2021-12-07 19:59:38 +00:00
Chris S. Kim f8f8580ab2
Godocs updates for catalog endpoints (#11716) 2021-12-07 10:18:28 -05:00
Mathew Estafanous 0a9621ec7a
Transition all endpoint tests in agent_endpoint_test.go to go through ServeHTTP (#11499) 2021-12-07 09:44:03 -05:00
Dan Upton 205ce9a69d
Remove references to "master" ACL tokens in tests (#11751) 2021-12-07 12:48:50 +00:00
Dan Upton 7fe81171d9
Rename `ACLMasterToken` => `ACLInitialManagementToken` (#11746) 2021-12-07 12:39:28 +00:00
Dan Upton 3a91815169
agent/token: rename `agent_master` to `agent_recovery` (internally) (#11744) 2021-12-07 12:12:47 +00:00
R.B. Boyer 9315a9812f return the max 2021-12-06 15:36:52 -06:00
freddygv 60fe5f75bb Remove support for failover to partition
Failing over to a partition is more siimilar to failing over to another
datacenter than it is to failing over to a namespace. In a future
release we should update how localities for failover are specified. We
should be able to accept a list of localities which can include both
partition and datacenter.
2021-12-06 12:32:24 -07:00
freddygv 5c1f7aa372 Allow cross-partition references in disco chain
* Add partition fields to targets like service route destinations
* Update validation to prevent cross-DC + cross-partition references
* Handle partitions when reading config entries for disco chain
* Encode partition in compiled targets
2021-12-06 12:32:19 -07:00
R.B. Boyer b1605639fc
light refactors to support making partitions and serf-based wan federation are mutually exclusive (#11755) 2021-12-06 13:18:02 -06:00
R.B. Boyer e20e6348dd
areas: make the gRPC server tracker network area aware (#11748)
Fixes a bug whereby servers present in multiple network areas would be
properly segmented in the Router, but not in the gRPC mirror. This would
lead servers in the current datacenter leaving from a network area
(possibly during the network area's removal) from deleting their own
records that still exist in the standard WAN area.

The gRPC client stack uses the gRPC server tracker to execute all RPCs,
even those targeting members of the current datacenter (which is unlike
the net/rpc stack which has a bypass mechanism).

This would manifest as a gRPC method call never opening a socket because
it would block forever waiting for the current datacenter's pool of
servers to be non-empty.
2021-12-06 09:55:54 -06:00
Freddy a725f06c83
Merge pull request #11739 from hashicorp/ap/exports-rename 2021-12-06 08:20:50 -07:00
freddygv e91509383f Clean up additional refs to partition exports 2021-12-04 15:16:40 -07:00
freddygv ed6076db26 Rename partition-exports to exported-services
Using a name less tied to partitions gives us more flexibility to use
this config entry in OSS for exports between datacenters/meshes.
2021-12-03 17:47:31 -07:00
freddygv f5b25401b3 Update intention topology to use new table 2021-12-03 17:28:31 -07:00
freddygv 55970c6ccd Avoid updating default decision from wildcard ixn
Given that we do not allow wildcard partitions in intentions, no one ixn
can override the DefaultAllow setting. Only the default ACL policy
applies across all partitions.
2021-12-03 17:28:12 -07:00
freddygv 497aab669f Add a new table to query service names by kind
This table purposefully does not index by partition/namespace. It's a
global view into all service names.

This table is intended to replace the current serviceListTxn watch in
intentionTopologyTxn. For cross-partition transparent proxying we need
to be able to calculate upstreams from intentions in any partition. This
means that the existing serviceListTxn function is insufficient since
it's scoped to a partition.

Moving away from that function is also beneficial because it watches the
main "services" table, so watchers will wake up when any instance is
registered or deregistered.
2021-12-03 17:28:12 -07:00
freddygv e7a7042c69 Update listener generation to account for consul VIP 2021-12-03 17:27:56 -07:00
Freddy f032d6ef05
Merge pull request #11680 from hashicorp/ap/partition-exports-oss 2021-12-03 16:57:50 -07:00
Dan Upton 3b9dfca88d
internal: support `ResultsFilteredByACLs` flag/header (#11643) 2021-12-03 23:04:24 +00:00
Dan Upton c8204330ed
query: support `ResultsFilteredByACLs` in query list endpoint (#11620) 2021-12-03 23:04:09 +00:00
Dhia Ayachi ce326b6074
port oss changes (#11736) 2021-12-03 17:23:55 -05:00
Freddy e246defb6c
Merge pull request #11720 from hashicorp/bbolt 2021-12-03 14:44:36 -07:00
Dan Upton 047aa2ffb0
fedstate: support `ResultsFilteredByACLs` in `ListMeshGateways` endpoint (#11644) 2021-12-03 20:56:55 +00:00
Dan Upton 361d9c2862
catalog: support `ResultsFilteredByACLs` flag/header (#11594) 2021-12-03 20:56:14 +00:00
Dan Upton 4c0956c03a
coordinate: support `ResultsFilteredByACLs` flag/header (#11617) 2021-12-03 20:51:02 +00:00
Dan Upton bf1e2ca551
sessions: support `ResultsFilteredByACLs` flag/header (#11606) 2021-12-03 20:43:43 +00:00
Dan Upton d92f0d84c6
txn: support `ResultsFilteredByACLs` flag in `Read` endpoint (#11632) 2021-12-03 20:41:03 +00:00
Dan Upton 547aa219ea
agent: support `X-Consul-Results-Filtered-By-ACLs` header in agent-local endpoints (#11610) 2021-12-03 20:36:28 +00:00
Dhia Ayachi 86159c6ed8
sessions partitioning tests (#11734)
* state: port KV and Tombstone tables to new pattern

* go fmt'ed

* handle wildcards for tombstones

* Fix graveyard ent vs oss

* fix oss compilation error

* add partition to tombstones and kv state store indexes

* refactor to use `indexWithEnterpriseIndexable`

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add `singleValueID` implementation assertions

* partition `tableSessions` table

* fix sessions to use UUID and fix prefix index

* fix oss build

* clean up unused functions

* fix oss compilation

* add a partition indexer for sessions

* Fix oss to not have partition index

* fix oss tests

* remove unused operations_ent.go and operations_oss.go func

* remove unused const

* convert `IndexID` of `session_checks` table

* convert `indexSession` of `session_checks` table

* convert `indexNodeCheck` of `session_checks` table

* partition `indexID` and `indexSession` of `tableSessionChecks`

* fix oss linter

* fix review comments

* remove partition for Checks as it's always use the session partition

* fix tests

* fix tests

* do not namespace nodeChecks index

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2021-12-03 15:36:07 -05:00
Dan Upton c314be2ff9
intention: support `ResultsFilteredByACLs` flag/header (#11612) 2021-12-03 20:35:54 +00:00
Mark Anderson a89ffba2d4
Cross port of ent #1383 (#11726)
Cross port of ent #1383 "Reject non-default datacenter when making partitioned ACLs"

On the OSS side this is a minor refactor to add some more checks that are only applicable to enterprise code.

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-12-03 10:20:25 -08:00
Dan Upton 599a4d6619
config: support `ResultsFilteredByACLs` in list/list all endpoints (#11621) 2021-12-03 17:39:47 +00:00
Dan Upton c4c68915c9
event: support `X-Consul-Results-Filtered-By-ACLs` header in list (#11616) 2021-12-03 17:38:59 +00:00
Dan Upton 474ef7cc1f
kv: support `ResultsFilteredByACLs` in list/list keys (#11593) 2021-12-03 17:31:48 +00:00
Dan Upton cf1bd585f6
health: support `ResultsFilteredByACLs` flag/header (#11602) 2021-12-03 17:31:32 +00:00
Dan Upton 1e47e3c82b
Groundwork for exposing when queries are filtered by ACLs (#11569) 2021-12-03 17:11:26 +00:00
Kyle Havlovitz 0546bbe08a dns: add endpoint for querying service virtual IPs 2021-12-02 16:40:28 -08:00
Kyle Havlovitz 6f34a4f777
Merge pull request #11724 from hashicorp/service-virtual-ips
oss: add virtual IP generation for connect services
2021-12-02 16:16:57 -08:00
Kyle Havlovitz 4f2cfee4b0 consul: add virtual IP generation for connect services 2021-12-02 15:42:47 -08:00
R.B. Boyer c46f9f9f31
agent: add variation of force-leave that exclusively works on the WAN (#11722)
Fixes #6548
2021-12-02 17:15:10 -06:00
Matt Keeler c7a94843ee Emit raft-boltdb metrics 2021-12-02 16:56:15 -05:00
Daniel Nephin e47cecc653 config: add NoFreelistSync option
# Conflicts:
#	agent/config/testdata/TestRuntimeConfig_Sanitize-enterprise.golden
#	agent/consul/server.go
2021-12-02 16:56:15 -05:00
Matt Keeler 42a5635bc3 Use raft-boltdb/v2 2021-12-02 16:56:15 -05:00
Daniel Nephin 17a2d14d49 ca: set the correct SigningKeyID after config update with Vault provider
The test added in this commit shows the problem. Previously the
SigningKeyID was set to the RootCert not the local leaf signing cert.

This same bug was fixed in two other places back in 2019, but this last one was
missed.

While fixing this bug I noticed I had the same few lines of code in 3
places, so I extracted a new function for them.

There would be 4 places, but currently the InitializeCA flow sets this
SigningKeyID in a different way, so I've left that alone for now.
2021-12-02 16:07:11 -05:00
Daniel Nephin 96f95889db
Merge pull request #11713 from hashicorp/dnephin/ca-test-names
ca: make test naming consistent
2021-12-02 16:05:42 -05:00
Daniel Nephin ff4581092e
Merge pull request #11671 from hashicorp/dnephin/ca-fix-storing-vault-intermediate
ca: fix storing the leaf signing cert with Vault provider
2021-12-02 16:02:24 -05:00
Daniel Nephin 81afb208ac
Merge pull request #11677 from hashicorp/dnephin/freeport-interface
sdk: use t.Cleanup in freeport and remove unnecessary calls
2021-12-02 15:58:41 -05:00
Daniel Nephin 447097b166 ca: make test naming consistent
While working on the CA system it is important to be able to run all the
tests related to the system, without having to wait for unrelated tests.
There are many slow and unrelated tests in agent/consul, so we need some
way to filter to only the relevant tests.

This PR renames all the CA system related tests to start with either
`TestCAMananger` for tests of internal operations that don't have RPC
endpoint, or `TestConnectCA` for tests of RPC endpoints. This allows us
to run all the test with:

    go test -run 'TestCAMananger|TestConnectCA' ./agent/consul

The test naming follows an undocumented convention of naming tests as
follows:

    Test[<struct name>_]<function name>[_<test case description>]

I tried to always keep Primary/Secondary at the end of the description,
and _Vault_ has to be in the middle because of our regex to run those
tests as a separate CI job.

You may notice some of the test names changed quite a bit. I did my best
to identify the underlying method being tested, but I may have been
slightly off in some cases.
2021-12-02 14:57:09 -05:00
FFMMM 384d497f26
add MustRevalidate flag to connect_ca_leaf cache type; always use on non-blocking queries (#11693)
* always use MustRevalidate on non-blocking queries for connect ca leaf

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Update agent/agent_endpoint_test.go

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* pr feedback

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-12-02 11:32:15 -08:00
Daniel Nephin 28a8a64019 ca: make getLeafSigningCertFromRoot safer
As a method on the struct type this would not be safe to call without first checking
c.isIntermediateUsedToSignLeaf.

So for now, move this logic to the CAMananger, so that it is always correct.
2021-12-02 12:42:49 -05:00
Daniel Nephin b29faa3e50 ca: fix stored CARoot representation with Vault provider
We were not adding the local signing cert to the CARoot. This commit
fixes that bug, and also adds support for fixing existing CARoot on
upgrade.

Also update the tests for both primary and secondary to be more strict.
Check the SigningKeyID is correct after initialization and rotation.
2021-12-02 12:42:49 -05:00
Dan Upton bf56a2c495
Rename `agent_master` ACL token in the API and CLI (#11669) 2021-12-02 17:05:27 +00:00
Dan Upton d8afd2f6c8
Rename `master` and `agent_master` ACL tokens in the config file format (#11665) 2021-12-01 21:08:14 +00:00
Chris S. Kim 54e4d1b7b2
ENT to OSS sync (#11703) 2021-12-01 14:56:10 -05:00
R.B. Boyer db91cbf484
auto-config: ensure the feature works properly with partitions (#11699) 2021-12-01 13:32:34 -06:00
Daniel Nephin 32ef9c5d5c ca: add some godoc and func for finding leaf signing cert
This will be used in a follow up commit.
2021-11-30 18:36:41 -05:00
Daniel Nephin 4185045a7f sdk/freeport: rename Port to GetOne
For better consistency with GetN
2021-11-30 17:32:41 -05:00
Chris S. Kim 56fab21582
Refactor test helper (#11689)
Allow custom ACL root tokens to be passed
2021-11-30 13:22:07 -05:00
Chris S. Kim 36246c5791
acl: Fill authzContext from token in Coordinate endpoints (#11688) 2021-11-30 13:17:41 -05:00
freddygv dd662d7058 Move ent config test to ent file 2021-11-29 12:15:17 -07:00
freddygv 5e1f7b7c36 Prevent partition-exports entry from OSS usage
Validation was added on the config entry kind since that is called when
validating config entries to bootstrap via agent configuration and when
applying entries via the config RPC endpoint.
2021-11-29 11:24:16 -07:00
Daniel Nephin e8312d6b5a testing: remove unnecessary calls to freeport
Previously we believe it was necessary for all code that required ports
to use freeport to prevent conflicts.

https://github.com/dnephin/freeport-test shows that it is actually save
to use port 0 (`127.0.0.1:0`) as long as it is passed directly to
`net.Listen`, and the listener holds the port for as long as it is
needed.

This works because freeport explicitly avoids the ephemeral port range,
and port 0 always uses that range. As you can see from the test output
of https://github.com/dnephin/freeport-test, the two systems never use
overlapping ports.

This commit converts all uses of freeport that were being passed
directly to a net.Listen to use port 0 instead. This allows us to remove
a bit of wrapping we had around httptest, in a couple places.
2021-11-29 12:19:43 -05:00
Daniel Nephin d795a73f78 testing: use the new freeport interfaces 2021-11-27 15:39:46 -05:00
Daniel Nephin 56f9238d15 go-sso: remove returnFunc now that freeport handles return 2021-11-27 15:29:38 -05:00
Daniel Nephin 8c7475d95e sdk: add freeport functions that use t.Cleanup 2021-11-27 15:04:43 -05:00
Daniel Nephin 59204598c8 ca: clean up unnecessary raft.Apply response checking
In d2ab767fef raftApply was changed to handle this check in
a single place, instad of having every caller check it. It looks like these few places
were missed when I did that clean up.

This commit removes the remaining resp.(error) checks, since they are all no-ops now.
2021-11-26 17:57:55 -05:00
Daniel Nephin 52f0853ff9
Merge pull request #11339 from hashicorp/dnephin/ca-manager-isolate-secondary-2
ca: reduce use of state in the secondary
2021-11-26 14:41:45 -05:00
Daniel Nephin 91a0c25932 ca: remove state check in secondarySetPrimaryRoots
This function is only ever called from operations that have already acquired the state lock, so checking
the value of state can never fail.

This change is being made in preparation for splitting out a separate type for the secondary logic. The
state can't easily be shared, so really only the expored top-level functions should acquire the 'state lock'.
2021-11-26 14:14:47 -05:00
Daniel Nephin f1944458e4 ca: remove actingSecondaryCA
This commit removes the actingSecondaryCA field, and removes the stateLock around it. This field
was acting as a proxy for providerRoot != nil, so replace it with that check instead.

The two methods which called secondarySetCAConfigured already set the state, so checking the
state again at this point will not catch runtime errors (only programming errors, which we can catch with tests).
In general, handling state transitions should be done on the "entrypoint" methods where execution starts, not
in every internal method.

This is being done to remove some unnecessary references to c.state, in preparations for extracting
types for primary/secondary.
2021-11-26 14:14:47 -05:00
Daniel Nephin b92084b8e8 ca: reduce consul provider backend interface a bit
This makes it easier to fake, which will allow me to use the ConsulProvider as
an 'external PKI' to test a customer setup where the actual root CA is not
the root we use for the Consul CA.

Replaces a call to the state store to fetch the clusterID with the
clusterID field already available on the built-in provider.
2021-11-25 11:46:06 -05:00
Dhia Ayachi 3820e09a47
Partition/kv indexid sessions (#11639)
* state: port KV and Tombstone tables to new pattern

* go fmt'ed

* handle wildcards for tombstones

* Fix graveyard ent vs oss

* fix oss compilation error

* add partition to tombstones and kv state store indexes

* refactor to use `indexWithEnterpriseIndexable`

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* partition `tableSessions` table

* fix sessions to use UUID and fix prefix index

* fix oss build

* clean up unused functions

* fix oss compilation

* add a partition indexer for sessions

* Fix oss to not have partition index

* fix oss tests

* remove unused operations_ent.go and operations_oss.go func

* convert `indexNodeCheck` of `session_checks` table

* partition `indexID` and `indexSession` of `tableSessionChecks`

* remove partition for Checks as it's always use the session partition

* partition sessions index id table

* fix rebase issues

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2021-11-24 11:34:36 -05:00
Dhia Ayachi bb83624950
Partition session checks store (#11638)
* state: port KV and Tombstone tables to new pattern

* go fmt'ed

* handle wildcards for tombstones

* Fix graveyard ent vs oss

* fix oss compilation error

* add partition to tombstones and kv state store indexes

* refactor to use `indexWithEnterpriseIndexable`

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add `singleValueID` implementation assertions

* partition `tableSessions` table

* fix sessions to use UUID and fix prefix index

* fix oss build

* clean up unused functions

* fix oss compilation

* add a partition indexer for sessions

* Fix oss to not have partition index

* fix oss tests

* remove unused operations_ent.go and operations_oss.go func

* remove unused const

* convert `IndexID` of `session_checks` table

* convert `indexSession` of `session_checks` table

* convert `indexNodeCheck` of `session_checks` table

* partition `indexID` and `indexSession` of `tableSessionChecks`

* fix oss linter

* fix review comments

* remove partition for Checks as it's always use the session partition

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2021-11-24 09:10:38 -05:00
Chris S. Kim 2350e7e56a
cleanup: Clarify deprecated legacy intention endpoints (#11635) 2021-11-23 19:32:18 -05:00
Chris S. Kim db5ee0e4d2
Merge from ent (#11506) 2021-11-19 11:50:44 -05:00
R.B. Boyer dd4a59db8e
agent: purge service/check registration files for incorrect partitions on reload (#11607) 2021-11-18 14:44:20 -06:00
Iryna Shustava 0ee456649f
connect: Support auth methods for the vault connect CA provider (#11573)
* Support vault auth methods for the Vault connect CA provider
* Rotate the token (re-authenticate to vault using auth method) when the token can no longer be renewed
2021-11-18 13:15:28 -07:00
Daniel Nephin b4080bc0dc ca: use the cluster ID passed to the primary
instead of fetching it from the state store.
2021-11-16 16:57:22 -05:00
Daniel Nephin b9ab9bae12 ca: accept only the cluster ID to SpiffeIDSigningForCluster
To make it more obivous where ClusterID is used, and remove the need to create a struct
when only one field is used.
2021-11-16 16:57:21 -05:00
Will Jordan 68efecafed
Update node info sync comment (#11465) 2021-11-16 11:16:11 -08:00
R.B. Boyer 1e02460bd1
re-run gofmt on 1.17 (#11579)
This should let freshly recompiled golangci-lint binaries using Go 1.17
pass 'make lint'
2021-11-16 12:04:01 -06:00
R.B. Boyer eb21649f82
partitions: various refactors to support partitioning the serf LAN pool (#11568) 2021-11-15 09:51:14 -06:00
freddygv 0e507492d0 Update proxycfg for ingress service partitions 2021-11-12 14:33:31 -07:00
freddygv e5b7c4713f Accept partition for ingress services 2021-11-12 14:33:14 -07:00
freddygv 400697507b Move assertion to after config fetch 2021-11-10 10:50:08 -07:00
freddygv da5bcc574e Use ClusterID to check for readiness
The TrustDomain is populated from the Host() method which includes the
hard-coded "consul" domain. This means that despite having an empty
cluster ID, the TrustDomain won't be empty.
2021-11-10 10:45:22 -07:00
freddygv 6976044bc4 Prevent replicating partition-exports 2021-11-09 16:42:42 -07:00
freddygv 5c121d7a48 handle error scenario of empty local DC 2021-11-09 16:42:42 -07:00
freddygv af29cda415 Restrict DC for partition-exports writes
There are two restrictions:
- Writes from the primary DC which explicitly target a secondary DC.
- Writes to a secondary DC that do not explicitly target the primary DC.

The first restriction is because the config entry is not supported in
secondary datacenters.

The second restriction is to prevent the scenario where a user writes
the config entry to a secondary DC, the write gets forwarded to the
primary, but then the config entry does not apply in the secondary.
This makes the scope more explicit.
2021-11-09 16:42:42 -07:00
Freddy 00b5b0a0a2
Update filter chain creation for sidecar/ingress listeners (#11245)
The duo of `makeUpstreamFilterChainForDiscoveryChain` and `makeListenerForDiscoveryChain` were really hard to reason about, and led to concealing a bug in their branching logic. There were several issues here:

- They tried to accomplish too much: determining filter name, cluster name, and whether RDS should be used. 
- They embedded logic to handle significantly different kinds of upstream listeners (passthrough, prepared query, typical services, and catch-all)
- They needed to coalesce different data sources (Upstream and CompiledDiscoveryChain)

Rather than handling all of those tasks inside of these functions, this PR pulls out the RDS/clusterName/filterName logic.

This refactor also fixed a bug with the handling of [UpstreamDefaults](https://www.consul.io/docs/connect/config-entries/service-defaults#defaults). These defaults get stored as UpstreamConfig in the proxy snapshot with a DestinationName of "*", since they apply to all upstreams. However, this wildcard destination name must not be used when creating the name of the associated upstream cluster. The coalescing logic in the original functions here was in some situations creating clusters with a `*.` prefix, which is not a valid destination.
2021-11-09 14:43:51 -07:00
Kyle Havlovitz 6c0bd0550f
Merge pull request #11461 from deblasis/feature/empty_client_addr_warning
config: warn the user if client_addr is empty
2021-11-09 09:37:38 -08:00
Daniel Upton 50a1f20ff9
xds: prefer fed state gateway definitions if they're fresher (#11522)
Fixes an issue described in #10132, where if two DCs are WAN federated
over mesh gateways, and the gateway in the non-primary DC is terminated
and receives a new IP address (as is commonly the case when running them
on ephemeral compute instances) the primary DC is unable to re-establish
its connection until the agent running on its own gateway is restarted.

This was happening because we always preferred gateways discovered by
the `Internal.ServiceDump` RPC (which would fail because there's no way
to dial the remote DC) over those discovered in the federation state,
which is replicated as long as the primary DC's gateway is reachable.
2021-11-09 16:45:36 +00:00
Freddy 7d95d90fce
Merge pull request #11514 from hashicorp/dnephin/ca-fix-secondary-init
ca: properly handle the case where the secondary initializes after the primary
2021-11-08 17:16:16 -07:00
freddygv cc5a7ed36c Avoid returning empty roots with uninitialized CA
Currently getCARoots could return an empty object with an empty trust
domain before the CA is initialized. This commit returns an error while
there is no CA config or no trust domain.

There could be a CA config and no trust domain because the CA config can
be created in InitializeCA before initialization succeeds.
2021-11-08 16:51:49 -07:00
Dhia Ayachi 7916268c40
refactor session state store tables to use the new index pattern (#11525)
* state: port KV and Tombstone tables to new pattern

* go fmt'ed

* handle wildcards for tombstones

* Fix graveyard ent vs oss

* fix oss compilation error

* add partition to tombstones and kv state store indexes

* refactor to use `indexWithEnterpriseIndexable`

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add `singleValueID` implementation assertions

* partition `tableSessions` table

* fix sessions to use UUID and fix prefix index

* fix oss build

* clean up unused functions

* fix oss compilation

* add a partition indexer for sessions

* Fix oss to not have partition index

* fix oss tests

* remove unused func `prefixIndexFromServiceNameAsString`

* fix test error check

* remove unused operations_ent.go and operations_oss.go func

* remove unused const

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2021-11-08 16:20:50 -05:00
Dhia Ayachi 98735a6d12
KV refactoring, part 2 (#11512)
* add partition to the kv get pretty print

* fix failing test

* add test for kvs RPC endpoint
2021-11-08 11:43:21 -05:00
Dhia Ayachi 520cb5858c
KV state store refactoring and partitioning (#11510)
* state: port KV and Tombstone tables to new pattern

* go fmt'ed

* handle wildcards for tombstones

* Fix graveyard ent vs oss

* fix oss compilation error

* add partition to tombstones and kv state store indexes

* refactor to use `indexWithEnterpriseIndexable`

* partition kvs indexID table

* add `partitionedIndexEntryName` in oss for test purpose

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add `singleValueID` implementation assertions

* remove entmeta reference from oss

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2021-11-08 09:35:56 -05:00
Giulio Micheloni af7b7b5693
Merge branch 'main' into serve-panic-recovery 2021-11-06 16:12:06 +01:00
Daniel Nephin d9110136f2 ca: Only initialize clusterID in the primary
The secondary must get the clusterID from the primary
2021-11-05 18:08:44 -04:00
Daniel Nephin 01bd3d118d ca: return an error when secondary fails to initialize
Previously secondaryInitialize would return nil in this case, which prevented the
deferred initialize from happening, and left the CA in an uninitialized state until a config
update or root rotation.

To fix this I extracted the common parts into the delegate implementation. However looking at this
again, it seems like the handling in secondaryUpdateRoots is impossible, because that function
should never be called before the secondary is initialzied. I beleive we can remove some of that
logic in a follow up.
2021-11-05 18:02:51 -04:00
Daniel Nephin 8ba760a2fc acl: remove id and revision from Policy constructors
The fields were removed in a previous commit.

Also remove an unused constructor for PolicyMerger
2021-11-05 15:45:08 -04:00
Daniel Nephin 7c679c11e6 acl: remove Policy.ID and Policy.Revision
These two fields do not appear to be used anywhere. We use the structs.ACLPolicy ID in the
ACLResolver cache, but the acl.Policy ID and revision are not used.
2021-11-05 15:43:52 -04:00
R.B. Boyer c7c5013edd
rename helper method to reflect the non-deprecated terminology (#11509) 2021-11-05 13:51:50 -05:00
Connor efe4b21287
Support Vault Namespaces explicitly in CA config (#11477)
* Support Vault Namespaces explicitly in CA config

If there is a Namespace entry included in the Vault CA configuration,
set it as the Vault Namespace on the Vault client

Currently the only way to support Vault namespaces in the Consul CA
config is by doing one of the following:
1) Set the VAULT_NAMESPACE environment variable which will be picked up
by the Vault API client
2) Prefix all Vault paths with the namespace

Neither of these are super pleasant. The first requires direct access
and modification to the Consul runtime environment. It's possible and
expected, not super pleasant.

The second requires more indepth knowledge of Vault and how it uses
Namespaces and could be confusing for anyone without that context. It
also infers that it is not supported

* Add changelog

* Remove fmt.Fprint calls

* Make comment clearer

* Add next consul version to website docs

* Add new test for default configuration

* go mod tidy

* Add skip if vault not present

* Tweak changelog text
2021-11-05 11:42:28 -05:00
R.B. Boyer 44c023a302
segments: ensure that the serf_lan_allowed_cidrs applies to network segments (#11495) 2021-11-04 17:17:19 -05:00
Mark Anderson 7e8228a20b
Remove some usage of md5 from the system (#11491)
* Remove some usage of md5 from the system

OSS side of https://github.com/hashicorp/consul-enterprise/pull/1253

This is a potential security issue because an attacker could conceivably manipulate inputs to cause persistence files to collide, effectively deleting the persistence file for one of the colliding elements.

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-11-04 13:07:54 -07:00
FFMMM 61bd417a82
plumb thru root cert tll to the aws ca provider (#11449)
* plumb thru root cert ttl to the aws ca provider

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Update .changelog/11449.txt

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2021-11-04 12:19:08 -07:00
FFMMM 6004a21f35
fix aws pca certs (#11470)
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-11-03 12:21:24 -07:00
Mathew Estafanous 8fb90aacef
Convert (some) test endpoints to use ServeHTTP instead of direct calls to handlers. (#11445) 2021-11-03 11:12:36 -04:00
FFMMM 4ddf973a31
add root_cert_ttl option for consul connect, vault ca providers (#11428)
* add root_cert_ttl option for consul connect, vault ca providers

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>

* add changelog, pr feedback

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Update .changelog/11428.txt, more docs

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* Update website/content/docs/agent/options.mdx

Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>
2021-11-02 11:02:10 -07:00
Daniel Nephin 51d8417545
Merge pull request #10690 from tarat44/h2c-support-in-ping-checks
add support for h2c in h2 ping health checks
2021-11-02 13:53:06 -04:00
Alessandro De Blasis 2f970555d9 config: warn the user if client_addr is empty
if the provided value is empty string then the client services
(DNS, HTTP, HTTPS, GRPC) are not listening and the user is not notified
in any way about what's happening.
Also, since a not provided client_addr defaults to 127.0.0.1, we make sure
we are not getting unwanted warnings

Signed-off-by: Alessandro De Blasis <alex@deblasis.net>
2021-11-01 22:47:20 +00:00
Daniel Nephin b57cae94de
Merge pull request #10771 from hashicorp/dnephin/emit-telemetry-metrics-immediately
telemetry: improve cert expiry metrics
2021-11-01 18:31:03 -04:00
freddygv 60066e5154 Exclude default partition from GatewayKey string
This will behave the way we handle SNI and SPIFFE IDs, where the default
partition is excluded.

Excluding the default ensures that don't attempt to compare default.dc2
to dc2 in OSS.
2021-11-01 14:45:52 -06:00
freddygv e3666b0bc4 Update GatewayKeys deduplication
Federation states data is only keyed on datacenter, so it cannot be
directly compared against keys for gateway groups.
2021-11-01 13:58:53 -06:00
freddygv 90ce897456 Store GatewayKey in proxycfg snapshot for re-use 2021-11-01 13:58:53 -06:00
freddygv bbe46e9522 Update locality check in xds 2021-11-01 13:58:53 -06:00
freddygv 4d4ccedb3a Update locality check in proxycfg 2021-11-01 13:58:53 -06:00
Daniel Nephin 7337cfd6dc
Merge pull request #11340 from hashicorp/dnephin/ca-manager-provider
ca: split the Provider interface into Primary/Secondary
2021-11-01 14:11:15 -04:00
Daniel Nephin eee598e91c
Merge pull request #11338 from hashicorp/dnephin/ca-manager-isolate-secondary
ca: clearly identify methods that are primary-only or secondary-only
2021-11-01 14:10:31 -04:00
Daniel Upton d47b7311b8
Support Check-And-Set deletion of config entries (#11419)
Implements #11372
2021-11-01 16:42:01 +00:00
Dhia Ayachi 2801785710
regenerate expired certs (#11462)
* regenerate expired certs

* add documentation to generate tests certificates
2021-11-01 11:40:16 -04:00
Jared Kirschner 0854e1d684
Merge pull request #11348 from kbabuadze/fix-answers-alt-domain
Fix answers for alt domain
2021-10-29 17:09:20 -04:00
R.B. Boyer c8cafb7654
agent: for various /v1/agent endpoints parse the partition parameter on the request (#11444)
Also update the corresponding CLI commands to send the parameter
appropriately.

NOTE: Behavioral changes are not happening in this PR.
2021-10-28 16:44:38 -05:00
R.B. Boyer af9ffc214d
agent: add a clone function for duplicating the serf lan configuration (#11443) 2021-10-28 16:11:26 -05:00
Daniel Nephin 367b664318 Add tests for cert expiry metrics 2021-10-28 14:38:57 -04:00
Daniel Nephin 210d37e4ab
Merge pull request #10671 from hashicorp/dnephin/fix-subscribe-test-flake
subscribe: improve TestSubscribeBackend_IntegrationWithServer_DeliversAllMessages
2021-10-28 12:57:09 -04:00
Evan Culver 61be9371f5
connect: Remove support for Envoy 1.16 (#11354) 2021-10-27 18:51:35 -07:00
Evan Culver bec08f4ec3
connect: Add support for Envoy 1.20 (#11277) 2021-10-27 18:38:10 -07:00
freddygv ac96ce6552 Ensure partition-exports kind gets marshalled
The api module has decoding functions that rely on 'kind' being present
of payloads. This is so that we can decode into the appropriate api type
for the config entry.

This commit ensures that a static kind is marshalled in responses from
Consul's api endpoints so that the api module can decode them.
2021-10-27 15:01:26 -06:00
Daniel Nephin a8e2e1c365 agent: move agent tls metric monitor to a more appropriate place
And add a test for it
2021-10-27 16:26:09 -04:00
Daniel Nephin c92513ec16 telemetry: set cert expiry metrics to NaN on start
So that followers do not report 0, which would make alerting difficult.
2021-10-27 15:19:25 -04:00
Daniel Nephin 9264ce89d2 telemetry: fix cert expiry metrics by removing labels
These labels should be set by whatever process scrapes Consul (for
prometheus), or by the agent that receives them (for datadog/statsd).

We need to remove them here because the labels are part of the "metric
key", so we'd have to pre-declare the metrics with the labels. We could
do that, but that is extra work for labels that should be added from
elsewhere.

Also renames the closure to be more descriptive.
2021-10-27 15:19:25 -04:00
Daniel Nephin 7948720bbb telemetry: only emit leader cert expiry metrics on the servers 2021-10-27 15:19:25 -04:00
Daniel Nephin 7fe60e5989 telemetry: prevent stale values from cert monitors
Prometheus scrapes metrics from each process, so when leadership transfers to a different node
the previous leader would still be reporting the old cached value.

By setting NaN, I believe we should zero-out the value, so that prometheus should only consider the
value from the new leader.
2021-10-27 15:19:25 -04:00
Daniel Nephin 0cc58f54de telemetry: improve cert expiry metrics
Emit the metric immediately so that after restarting an agent, the new expiry time will be
emitted. This is particularly important when this metric is being monitored, because we want
the alert to resovle itself immediately.

Also fixed a bug that was exposed in one of these metrics. The CARoot can be nil, so we have
to handle that case.
2021-10-27 15:19:25 -04:00
Daniel Nephin a3c781682d subscribe: attempt to fix a flaky test
TestSubscribeBackend_IntegrationWithServer_DeliversAllMessages has been
flaking a few times. This commit cleans up the test a bit, and improves
the failure output.

I don't believe this actually fixes the flake, but I'm not able to
reproduce it reliably.

The failure appears to be that the event with Port=0 is being sent in
both the snapshot and as the first event after the EndOfSnapshot event.

Hopefully the improved logging will show us if these are really
duplicate events, or actually different events with different indexes.
2021-10-27 15:09:09 -04:00
Freddy fbcf9f3f6c
Merge pull request #11435 from hashicorp/ent-authorizer-refactor
[OSS] Export ACLs refactor
2021-10-27 13:04:40 -06:00
Freddy 303532825f
Merge pull request #11432 from hashicorp/ap/exports-mgw
[OSS] Update mesh gateways to handle partitions
2021-10-27 12:54:53 -06:00
freddygv 43360eb216 Rework acl exports interface 2021-10-27 12:50:39 -06:00
Freddy ec7e94d129
Merge pull request #11433 from hashicorp/exported-service-acls
[OSS] acl: Expand ServiceRead and NodeRead to account for partition exports
2021-10-27 12:48:08 -06:00
freddygv e93c144d2f Update comments 2021-10-27 12:36:44 -06:00
Freddy a8762be529
Merge pull request #11431 from hashicorp/ap/exports-proxycfg
[OSS] Update partitioned mesh gw handling for connect proxies
2021-10-27 11:27:43 -06:00
Freddy b1b6f682e1
Merge pull request #11416 from hashicorp/ap/exports-update
Rename service-exports to partition-exports
2021-10-27 11:27:31 -06:00
freddygv 3a2061544d Fixup partitions assertion 2021-10-27 11:15:25 -06:00
freddygv 9480670b72 Fixup imports 2021-10-27 11:15:25 -06:00
freddygv c72bbb6e8d Split up locality check from hostname check 2021-10-27 11:15:25 -06:00
freddygv d28b9052b2 Move the exportingpartitions constant to enterprise 2021-10-27 11:15:25 -06:00
freddygv 448701dbd8 Replace default partition check 2021-10-27 11:15:25 -06:00
freddygv 12923f5ebc PR comments 2021-10-27 11:15:25 -06:00
freddygv 327e6bff25 Leave todo about default name 2021-10-27 11:15:25 -06:00
freddygv 5bf2497f71 Add oss impl of registerEntCache 2021-10-27 11:15:25 -06:00
freddygv 954d21c6ba Register the ExportingPartitions cache type 2021-10-27 11:15:25 -06:00
freddygv a33b6923e0 Account for partitions in xds gen for mesh gw
This commit avoids skipping gateways in remote partitions of the local
DC when generating listeners/clusters/endpoints.
2021-10-27 11:15:25 -06:00
freddygv 935112a47a Account for partition in SNI for gateways 2021-10-27 11:15:25 -06:00
freddygv 110fae820a Update xds pkg to account for GatewayKey 2021-10-27 09:03:56 -06:00
freddygv 7e65678c52 Update mesh gateway proxy watches for partitions
This commit updates mesh gateway watches for cross-partitions
communication.

* Mesh gateways are keyed by partition and datacenter.

* Mesh gateways will now watch gateways in partitions that export
services to their partition.

* Mesh gateways in non-default partitions will not have cross-datacenter
watches. They are not involved in traditional WAN federation.
2021-10-27 09:03:56 -06:00
freddygv aa931682ea Avoid mixing named and unnamed params 2021-10-26 23:42:25 -06:00
freddygv bf350224a0 Avoid passing nil config pointer 2021-10-26 23:42:25 -06:00
freddygv df7b5af6f0 Avoid panic on nil partitionAuthorizer config
partitionAuthorizer.config can be nil if it wasn't provided on calls to
newPartitionAuthorizer outside of the ACLResolver. This usage happens
often in tests.

This commit: adds a nil check when the config is going to be used,
updates non-test usage of NewPolicyAuthorizerWithDefaults to pass a
non-nil config, and dettaches setEnterpriseConf from the ACLResolver.
2021-10-26 23:42:25 -06:00
freddygv 22bdf279d1 Update NodeRead for partition-exports
When issuing cross-partition service discovery requests, ACL filtering
often checks for NodeRead privileges. This is because the common return
type is a CheckServiceNode, which contains node data.
2021-10-26 23:42:11 -06:00
Kyle Havlovitz 65c9109396 acl: pass PartitionInfo through ent ACLConfig 2021-10-26 23:41:52 -06:00
Kyle Havlovitz d03f849e49 acl: Expand ServiceRead logic to look at service-exports for cross-partition 2021-10-26 23:41:32 -06:00
freddygv 8006c6df73 Swap in structs.EqualPartitions for cmp 2021-10-26 23:36:01 -06:00
freddygv 37a16e9487 Replace Split with SplitN 2021-10-26 23:36:01 -06:00
freddygv b9b6447977 Finish removing useInDatacenter 2021-10-26 23:36:01 -06:00
freddygv e1691d1627 Update XDS for sidecars dialing through gateways 2021-10-26 23:35:48 -06:00
freddygv 62e0fc62c1 Configure sidecars to watch gateways in partitions
Previously the datacenter of the gateway was the key identifier, now it
is the datacenter and partition.

When dialing services in other partitions or datacenters we now watch
the appropriate partition.
2021-10-26 23:35:37 -06:00
freddygv eacb73cb78 Remove useInDatacenter from disco chain requests
useInDatacenter was used to determine whether the mesh gateway mode of
the upstream should be returned in the discovery chain target. This
commit makes it so that the mesh gateway mode is returned every time,
and it is up to the caller to decide whether mesh gateways should be
watched or used.
2021-10-26 23:35:21 -06:00
R.B. Boyer ef559dfdd4
agent: refactor the agent delegate interface to be partition friendly (#11429) 2021-10-26 15:08:55 -05:00
Chris S. Kim fa293362be
agent: Ensure partition is considered in agent endpoints (#11427) 2021-10-26 15:20:57 -04:00
Konstantine 55599d0b41 remove spaces 2021-10-26 12:38:13 -04:00
Konstantine ce85d2eada fix altDomain responses for services where address is IP, added tests 2021-10-26 12:38:13 -04:00
Konstantine a7e8c51f80 fix encodeIPAsFqdn to return alt-domain when requested, added test case 2021-10-26 12:38:12 -04:00
Konstantine ffb00f01b5 fixed altDomain response for NS type queries, and added test 2021-10-26 12:38:12 -04:00
Konstantine a828c45a62 edited TestDNS_AltDomains_Service to test responses for altDomains, and added TXT additional section check 2021-10-26 12:38:12 -04:00
Konstantine 0864bfdb71 fixed alt-domain answer for SRV records, and TXT records in additional section 2021-10-26 12:38:12 -04:00
Chris S. Kim 76bbeb3baf
ui: Pass primary dc through to uiserver (#11317)
Co-authored-by: John Cowen <johncowen@users.noreply.github.com>
2021-10-26 10:30:17 -04:00
freddygv 8aefdc31da Remove outdated partition label from test 2021-10-25 18:47:02 -06:00
freddygv 5c24ed61a8 Rename service-exports to partition-exports
Existing config entries prefixed by service- are specific to individual
services. Since this config entry applies to partitions it is being
renamed.

Additionally, the Partition label was changed to Name because using
Partition at the top-level and in the enterprise meta was leading to the
enterprise meta partition being dropped by msgpack.
2021-10-25 17:58:48 -06:00
Daniel Nephin 4ae2c8de9d
Merge pull request #11232 from hashicorp/dnephin/acl-legacy-remove-docs
acl: add docs and changelog for the removal of the legacy ACL system
2021-10-25 18:38:00 -04:00
Daniel Nephin 5d41b4d2f4 Update agent/consul/acl_client.go
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
2021-10-25 17:25:14 -04:00
Daniel Nephin 65d48e5042 state: remove support for updating legacy ACL tokens 2021-10-25 17:25:14 -04:00
Daniel Nephin 0784a31e85 acl: remove init check for legacy anon token
This token should always already be migrated from a previous version.
2021-10-25 17:25:14 -04:00
Daniel Nephin daba3c2309 acl: remove legacy parameter to ACLDatacenter
It is no longer used now that legacy ACLs have been removed.
2021-10-25 17:25:14 -04:00
Daniel Nephin 3390f85ab4 acl: remove ACLTokenTypeManagement 2021-10-25 17:25:14 -04:00
Daniel Nephin 32b4ad42ac acl: remove ACLTokenTypeClient,
along with the last test referencing it.
2021-10-25 17:25:14 -04:00
Daniel Nephin aea4cc5a6d acl: remove legacy arg to store.ACLTokenSet
And remove the tests for legacy=true
2021-10-25 17:25:14 -04:00
Daniel Nephin c77e5747b1 acl: remove EmbeddedPolicy
This method is no longer. It only existed for legacy tokens, which are no longer supported.
2021-10-25 17:25:14 -04:00
Daniel Nephin 121431bf17 acl: remove tests for resolving legacy tokens
The code for this was already removed, which suggests this is not actually testing what it claims.

I'm guessing these are still resolving because the tokens are converted to non-legacy tokens?
2021-10-25 17:25:14 -04:00
Daniel Nephin 0d0761927a acl: stop replication on leadership lost
It seems like this was missing. Previously this was only called by init of ACLs during an upgrade.
Now that legacy ACLs are  removed, nothing was calling stop.

Also remove an unused method from client.
2021-10-25 17:24:12 -04:00
Daniel Nephin 98823e573f Remove incorrect TODO 2021-10-25 17:20:06 -04:00
Daniel Nephin 1344137ce2 acl: move the legacy ACL struct to the one package where it is used
It is now only used for restoring snapshots. We can remove it in phase 2.
2021-10-25 17:20:06 -04:00
Daniel Nephin 531f2f8a3f acl: remove most of the rest of structs/acl_legacy.go 2021-10-25 17:20:06 -04:00
Paul Banks 954b283fec
Merge pull request #11163 from hashicorp/feature/ingress-tls-mixed
Add support for enabling connect-based ingress TLS per listener.
2021-10-25 21:36:01 +01:00
FFMMM fea6f08bf9
fix autopilot_failure_tolerance, add autopilot metrics test case (#11399)
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-10-25 10:55:59 -07:00
FFMMM 0954d261ae
use *telemetry.MetricsPrefix as prometheus.PrometheusOpts.Name (#11290)
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-10-21 13:33:01 -07:00
Dhia Ayachi 58f5686c08
fix leadership transfer on leave suggestions (#11387)
* add suggestions

* set isLeader to false when leadership transfer succeed
2021-10-21 14:02:26 -04:00
Dhia Ayachi f424faffdd
try to perform a leadership transfer when leaving (#11376)
* try to perform a leadership transfer when leaving

* add a changelog
2021-10-21 12:44:31 -04:00
Kyle Havlovitz 04cd2c983e Add new service-exports config entry 2021-10-20 12:24:18 -07:00
Jared Kirschner 14af8cb7a9
Merge pull request #11293 from bisakhmondal/service_filter
expression validation of service-resolver subset filter
2021-10-20 08:57:37 -04:00
Paul Banks c891f30c24 Rebase and rebuild golden files for Envoy version bump 2021-10-19 21:37:58 +01:00
Paul Banks 6faf85bccd Refactor `resolveListenerSDSConfig` to pass in whole config 2021-10-19 20:58:29 +01:00
Paul Banks 78a00f2e1c Add support for enabling connect-based ingress TLS per listener. 2021-10-19 20:58:28 +01:00
Giulio Micheloni a3fb665b88 Restored comment. 2021-10-16 18:05:32 +01:00
Giulio Micheloni fecce25658 Separete test file and no stack trace in ret error 2021-10-16 18:02:03 +01:00
Giulio Micheloni 0c78ddacde Merge branch 'main' of https://github.com/hashicorp/consul into hashicorp-main 2021-10-16 16:59:32 +01:00
R.B. Boyer cc2abb79ba
acl: small OSS refactors to help ensure that auth methods with namespace rules work with partitions (#11323) 2021-10-14 15:38:05 -05:00
freddygv e22f0cc033 Use stored entmeta to fill authzContext 2021-10-14 08:57:40 -06:00
freddygv 53ea1f634a Ensure partition is handled by auto-encrypt 2021-10-14 08:32:45 -06:00
FFMMM 62980ffaa2
fix: only add prom autopilot gauges to servers (#11241)
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-10-13 09:25:30 -07:00
Chris S. Kim c6906b4d37
Update Intentions.List with partitions (#11299) 2021-10-13 10:47:12 -04:00
R.B. Boyer 0c94095dfd
acl: fix bug in 'consul members' filtering with partitions (#11263) 2021-10-13 09:18:16 -05:00
Bisakh Mondal a350a383d3
add service resolver subset filter validation 2021-10-13 02:56:04 +05:30
Connor 257d00c908
Merge pull request #11222 from hashicorp/clly/service-mesh-metrics
Start tracking connect service mesh usage metrics
2021-10-11 14:35:03 -05:00
Connor Kelly 786d2896ff
Replace fmt.Sprintf with function 2021-10-11 12:43:38 -05:00
tarat44 166269f93b preload json values in structs to determine defaults 2021-10-10 17:52:26 -04:00
Daniel Nephin b2f49279e2 ca: split Primary/Secondary Provider
To make it more clear which methods are necessary for each scenario. This can
also prevent problems which force all DCs to use the same Vault instance, which
is currently a problem.
2021-10-10 15:48:02 -04:00
Daniel Nephin 1d14889eca ca: extract primaryUpdateRootCA
This function is only run when the CAManager is a primary. Extracting this function
makes it clear which parts of UpdateConfiguration are run only in the primary and
also makes the cleanup logic simpler. Instead of both a defer and a local var we
can call the cleanup function in two places.
2021-10-10 15:26:55 -04:00
Daniel Nephin 0bc812a8e5 ca: rename functions to use a primary or secondary prefix
This commit renames functions to use a consistent pattern for identifying the functions that
can only be called when the Manager is run as the primary or secondary.

This is a step toward eventually creating separate types and moving these methods off of CAManager.
2021-10-10 15:26:55 -04:00
Daniel Nephin eaea56c7b2 ca: make receiver variable name consistent
Every other method uses c not ca
2021-10-10 15:26:55 -04:00
tarat44 3fe637156c add test cases for h2ping_use_tls default behavior 2021-10-09 17:12:52 -04:00
FFMMM a0bba9171d
fix consul_autopilot_healthy metric emission (#11231)
https://github.com/hashicorp/consul/issues/10730
2021-10-08 10:31:50 -07:00
Connor Kelly a5cf4a9b57
Rename ConfigUsageEnterprise to EnterpriseConfigEntryUsage 2021-10-08 10:53:34 -05:00
Connor Kelly 8c519d5458
Rename and prefix ConfigEntry in Usage table
Rename ConfigUsage functions to ConfigEntry

prefix ConfigEntry kinds with the ConfigEntry table name to prevent
potential conflicts
2021-10-07 16:19:55 -05:00
Connor Kelly 533e7dbe85
Add connect specific prefix to Usage table
Ensure that connect Kind's are separate from ConfigEntry Kind's to
prevent miscounting
2021-10-07 16:16:23 -05:00
tarat44 ecdcfd6360 only set default on H2PingUseTLS if H2PING is set 2021-10-06 22:13:01 -04:00
Daniel Nephin b4e3367e63 docs: add notice that legacy ACLs have been removed.
Add changelog

Also remove a metric that is no longer emitted that was missed in a
previous step.
2021-10-05 18:30:22 -04:00
Daniel Nephin 18b3ac33e8 acl: remove unused translate rules endpoint
The CLI command does not use this endpoint, so we can remove it. It was missed in an
earlier pass.
2021-10-05 18:26:05 -04:00
Connor Kelly 024715eb11
Add changelog, website and metric docs
Add changelog to document what changed.
Add entry to telemetry section of the website to document what changed
Add docs to the usagemetric endpoint to help document the metrics in code
2021-10-05 13:34:24 -05:00
Joshua Montgomery 8eb5915f7d
Fixing SOA record to use alt domain when alt domain in use (#10431) 2021-10-05 10:47:27 -04:00
tarat44 c5479cefe6 fix test 2021-10-05 00:48:09 -04:00
tarat44 ca2e7c2039 fix formatting 2021-10-05 00:15:04 -04:00
tarat44 1e8e44d442 fix formatting 2021-10-05 00:12:23 -04:00
tarat44 c1ed3a9a94 change config option to H2PingUseTLS 2021-10-05 00:12:21 -04:00
tarat44 3c9f5a73d9 add support for h2c in h2 ping health checks 2021-10-04 22:51:08 -04:00
Daniel Nephin ab587f5221
Merge pull request #11182 from hashicorp/dnephin/acl-legacy-remove-upgrade
acl: remove upgrade from legacy, start in non-legacy mode
2021-10-04 17:25:39 -04:00
Evan Culver e808620463
Merge pull request #11118 from hashicorp/eculver/remove-envoy-1.15
Remove support for Envoy 1.15
2021-10-04 23:14:24 +02:00
Evan Culver c7747212c3
Merge pull request #11115 from hashicorp/eculver/envoy-1.19.1
Add support for Envoy 1.19.1
2021-10-04 23:13:26 +02:00
Daniel Nephin c7f74deb17 acl: remove updateEnterpriseSerfTags
The only remaining caller is a test helper, and the tests don't use the enterprise gossip
pools.
2021-10-04 17:01:51 -04:00
Daniel Nephin 9b1d2685bf
Merge pull request #11126 from hashicorp/dnephin/acl-legacy-remove-resolve-and-get-policy
acl: remove ACL.GetPolicy RPC endpoint and ACLResolver.resolveTokenLegacy
2021-10-04 16:29:51 -04:00
Connor Kelly c2583a1b7f
Add metrics to count the number of service-mesh config entries 2021-10-04 14:50:17 -05:00
Connor Kelly 536838b004
Add metrics to count connect native service mesh instances
This will add the counts of the service mesh instances tagged by
whether or not it is connect native
2021-10-04 14:37:05 -05:00
Connor Kelly 46bf882620
Add metrics to count service mesh Kind instance counts
This will add the counts of service mesh instances tagged by the
different ServiceKind's.
2021-10-04 14:36:59 -05:00
Daniel Nephin a1e3fa818c acl: fix test failures caused by remocving legacy ACLs
This commit two test failures:

1. Remove check for "in legacy ACL mode", the actual upgrade will be removed in a following commit.
2. Remove the early WaitForLeader in dc2, because with it the test was
   failing with ACL not found.
2021-10-01 18:03:10 -04:00
Evan Culver db397d62c5
Add 1.15 versions to too old list 2021-10-01 11:28:26 -07:00
Chris S. Kim 1c9b58a8af
agent: Reject partitions in legacy intention endpoints (#11181) 2021-10-01 13:18:57 -04:00
Chris S. Kim 53a35181e5
Support partitions in parseIntentionStringComponent (#11202) 2021-10-01 12:36:12 -04:00
Dhia Ayachi a5b09493ab
fix token list by auth method (#11196)
* add tests to OIDC authmethod and fix entMeta when retrieving auth-methods

* fix oss compilation error
2021-10-01 12:00:43 -04:00
Evan Culver e41830af8a
Merge branch 'eculver/envoy-1.19.1' into eculver/remove-envoy-1.15 2021-09-30 11:32:28 -07:00
Evan Culver fdbb742ffd
regenerate more envoy golden files 2021-09-30 10:57:47 -07:00
Daniel Nephin 4faf805716 acl: call stop for the upgrade goroutine when done
TestAgentLeaks_Server was reporting a goroutine leak without this. Not sure if it would actually
be a leak in production or if this is due to the test setup, but seems easy enough to call it
this way until we remove legacyACLTokenUpgrade.
2021-09-29 17:36:43 -04:00
Daniel Nephin 02da08ce77 acl: only run startACLUpgrade once
Since legacy ACL tokens can no longer be created we only need to run this upgrade a single
time when leadership is estalbished.
2021-09-29 16:22:01 -04:00
Daniel Nephin 3ac910606c acl: remove reading of serf acl tags
We no long need to read the acl serf tag, because servers are always either ACL enabled or
ACL disabled.

We continue to write the tag so that during an upgarde older servers will see the tag.
2021-09-29 15:45:11 -04:00
Daniel Nephin 3d7c07e1e4 acl: fix test failure
For some reason removing legacy ACL upgrade requires using an ACL token now
for this WaitForLeader.
2021-09-29 15:21:30 -04:00
Daniel Nephin 5c721832dc acl: remove legacy ACL upgrades from Server
As part of removing the legacy ACL system
2021-09-29 15:19:23 -04:00
Daniel Nephin 94be1835b2 acl: fix test failures caused by remocving legacy ACLs
This commit two test failures:

1. Remove check for "in legacy ACL mode", the actual upgrade will be removed in a following commit.
2. Use the root token in WaitForLeader, because without it the test was
   failing with ACL not found.
2021-09-29 15:15:50 -04:00
Daniel Nephin 8e9773e20b acl: remove ACL.GetPolicy endpoint and resolve legacy acls
And all code that was no longer used once those two were removed.
2021-09-29 14:33:19 -04:00
Daniel Nephin d12dd48c61 acl: remove ACL upgrading from Clients
As part of removing the legacy ACL system ACL upgrading and the flag for
legacy ACLs is removed from Clients.

This commit also removes the 'acls' serf tag from client nodes. The tag is only ever read
from server nodes.

This commit also introduces a constant for the acl serf tag, to make it easier to track where
it is used.
2021-09-29 14:02:38 -04:00
Daniel Nephin 19040586ce
Merge pull request #11136 from hashicorp/dnephin/acl-resolver-fix-default-authz
acl: fix default Authorizer for down_policy extend-cache/async-cache
2021-09-29 13:45:12 -04:00
Daniel Nephin a53fdd68c8
Merge pull request #11110 from hashicorp/dnephin/acl-legacy-remove-initialize
acl: remove initializeLegacyACL and the rest of the legacy FSM commands
2021-09-29 13:44:30 -04:00
Daniel Nephin 8f754aba14
Merge pull request #10999 from hashicorp/dnephin/revert-config-xds-port
Revert config xds_port
2021-09-29 13:39:15 -04:00
Daniel Nephin cc310224aa command/envoy: stop using the DebugConfig from Self endpoint
The DebugConfig in the self endpoint can change at any time. It's not a stable API.

This commit adds the XDSPort to a stable part of the XDS api, and changes the envoy command to read
this new field.

It includes support for the old API as well, in case a newer CLI is used with an older API, and
adds a test for both cases.
2021-09-29 13:21:28 -04:00
Daniel Nephin 6e1ebd3df7 acl: remove the last of the legacy FSM
Replace it with an implementation that returns an error, and rename some symbols
to use a Deprecated suffix to make it clear.

Also remove the ACLRequest struct, which is no longer referenced.
2021-09-29 12:42:23 -04:00
Daniel Nephin ed928511ca acl: remove bootstrap-init FSM operation 2021-09-29 12:42:23 -04:00
Daniel Nephin dab5d1bdc8 acl: remove initializeLegacyACL from leader init 2021-09-29 12:42:23 -04:00
Daniel Nephin 05f0cc3993 acl: remove ACLDelete FSM command, and state store function
These are no longer used now that ACL.Apply has been removed.
2021-09-29 12:42:23 -04:00
Daniel Nephin 966e50e00e acl: remove legacy field to ACLBoostrap 2021-09-29 12:42:23 -04:00
Daniel Nephin 1502547e38 Revert "Merge pull request #10588 from hashicorp/dnephin/config-fix-ports-grpc"
This reverts commit 74fb650b6b, reversing
changes made to 58bd817336.
2021-09-29 12:28:41 -04:00
Daniel Nephin 0330966315
Merge pull request #11101 from hashicorp/dnephin/acl-legacy-remove-rpc-2
acl: remove legacy ACL.Apply RPC
2021-09-29 12:23:55 -04:00
Daniel Nephin ea4a8343cd
Merge pull request #11177 from hashicorp/dnephin/remove-entmeta-methods
structs: remove EnterpriseMeta helper methods
2021-09-29 12:08:07 -04:00
Daniel Nephin 4c579a49ed
Merge pull request #10986 from hashicorp/dnephin/acl-legacy-remove-rpc
acl: remove legacy ACL RPC - part 1
2021-09-29 12:04:09 -04:00
Daniel Nephin eb632c53a2 structs: rename the last helper method.
This one gets used a bunch, but we can rename it to make the behaviour more obvious.
2021-09-29 11:48:38 -04:00
Daniel Nephin 8d8c1f9d5e structs: remove another helper
We already have a helper funtion.
2021-09-29 11:48:03 -04:00
Daniel Nephin 6d72517682 structs: remove two methods that were only used once each.
These methods only called a single function. Wrappers like this end up making code harder to read
because it adds extra ways of doing things.

We already have many helper functions for constructing these types, we don't need additional methods.
2021-09-29 11:47:03 -04:00
Daniel Nephin 8d1378cc1d
Merge pull request #10988 from hashicorp/dnephin/acl-legacy-remove-config
acl: isolate deprecated config and warn when they are used
2021-09-29 11:40:14 -04:00
Daniel Nephin 6b33e3bfd7
Merge pull request #9456 from hashicorp/dnephin/config-deprecation
config: Use DeprecatedConfig struct for deprecated config fields
2021-09-29 11:37:40 -04:00
Evan Culver 60170dfbe7
Merge remote-tracking branch 'origin/eculver/remove-envoy-1.15' into eculver/remove-envoy-1.15 2021-09-28 16:06:36 -07:00
Evan Culver 4f1a8d4ea6
Fix typo
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
2021-09-29 01:05:45 +02:00
Evan Culver 03e44da9f7
Merge branch 'eculver/envoy-1.19.1' into eculver/remove-envoy-1.15 2021-09-28 15:59:43 -07:00
Evan Culver 9b73e7319d
Merge branch 'main' into eculver/envoy-1.19.1 2021-09-28 15:58:20 -07:00
Chris S. Kim 5c37819d09
Cleanup unnecessary normalizing method (#11169) 2021-09-28 15:31:12 -04:00
Daniel Nephin bf2a3f6e79
Merge pull request #11084 from krastin/krastin-autopilot-loggingtypo
Fix a tiny typo in logging in autopilot.go
2021-09-28 15:11:11 -04:00
Evan Culver 585d9363ed
Merge branch 'main' into eculver/envoy-1.19.1 2021-09-28 11:54:33 -07:00
Chris S. Kim e3248c20c9
agent: Clean up unused built-in proxy config (#11165) 2021-09-28 11:29:10 -04:00
Daniel Nephin cd4e70b34c acl: fix default authorizer for down_policy
This was causing a nil panic because a nil authorizer is no longer valid after the cleanup done
in https://github.com/hashicorp/consul/pull/10632.
2021-09-23 18:12:22 -04:00
Daniel Nephin 6bb7aef15c Remove t.Parallel from TestACLResolver_DownPolicy
These tests run in under 10ms, t.Parallel does nothing but slow them down and
make failures harder to debug when one panics.
2021-09-23 18:12:22 -04:00
Dhia Ayachi e664dbc352
Refactor table index acl phase 2 (#11133)
* extract common methods from oss and ent

* remove unreachable code

* add missing normalize for binding rules

* fix oss to use Query
2021-09-23 15:26:09 -04:00
Daniel Nephin e8ac5fd90b config: Move ACLEnableKeyListPolicy to DeprecatedConfig 2021-09-23 15:15:00 -04:00
Daniel Nephin 5c40b717ed config: move acl_ttl to DeprecatedConfig 2021-09-23 15:14:59 -04:00
Daniel Nephin 977f6d8888 config: move acl_{default,down}_policy to DeprecatedConfig 2021-09-23 15:14:59 -04:00
Daniel Nephin 5eafcea4d4 config: Deprecate EnableACLReplication
replaced by ACL.TokenReplication
2021-09-23 15:14:59 -04:00
Daniel Nephin 5dc16180ad config: move ACL master token and replication to DeprecatedConfig 2021-09-23 15:14:59 -04:00
Paul Banks 1ecec84fd7
Merge pull request #10903 from hashicorp/feature/ingress-sds
Add Support to for providing TLS certificates for Ingress listeners from an SDS source
2021-09-23 16:19:05 +01:00
Dhia Ayachi 5904e4ac79
Refactor table index (#11131)
* convert tableIndex to use the new pattern

* make `indexFromString` available for oss as well

* refactor `indexUpdateMaxTxn`
2021-09-23 11:06:23 -04:00
Paul Banks 7b4cbe3143 Final readability tweaks from review 2021-09-23 10:17:12 +01:00
Paul Banks 70bc89b7f4 Fix subtle loop bug and add test 2021-09-23 10:13:41 +01:00
Paul Banks 07f81991df Refactor SDS validation to make it more contained and readable 2021-09-23 10:13:19 +01:00
Paul Banks 5cfd030d03 Refactor Ingress-specific lister code to separate file 2021-09-23 10:13:19 +01:00
Paul Banks 136928a90f Minor PR typo and cleanup fixes 2021-09-23 10:13:19 +01:00
Paul Banks 20d0bf81f7 Revert abandonned changes to proxycfg for Ent test consistency 2021-09-23 10:13:19 +01:00
Paul Banks a9119e36a5 Fix merge conflict in xds tests 2021-09-23 10:12:37 +01:00
Paul Banks 2281d883b9 Fix some more Enterprise Normalization issues affecting tests 2021-09-23 10:12:37 +01:00
Paul Banks 9fa60c7472 Remove unused argument to fix lint error 2021-09-23 10:09:11 +01:00
Paul Banks 659321d008 Handle namespaces in route names correctly; add tests for enterprise 2021-09-23 10:09:11 +01:00
Paul Banks 2a3d3d3c23 Update xDS routes to support ingress services with different TLS config 2021-09-23 10:08:02 +01:00
Paul Banks 16b3b1c737 Update xDS Listeners with SDS support 2021-09-23 10:08:02 +01:00
Paul Banks ccbda0c285 Update proxycfg to hold more ingress config state 2021-09-23 10:08:02 +01:00
Paul Banks 4e39f03d5b Add ingress-gateway config for SDS 2021-09-23 10:08:02 +01:00
Daniel Nephin c84867feda acl: remove ACL.Apply
As part of removing the legacy ACL system.
2021-09-22 18:28:08 -04:00
Daniel Nephin ae05419aea acl: made acl rules in tests slightly more specific
When converting these tests from the legacy ACL system to the new RPC endpoints I
initially changed most things to use _prefix rules, because that was equivalent to
the old legacy rules.

This commit modifies a few of those rules to be a bit more specific by replacing the _prefix
rule with a non-prefix one where possible.
2021-09-22 18:24:56 -04:00
Mark Anderson d88d9e71c2
partitions/authmethod-index work from enterprise (#11056)
* partitions/authmethod-index work from enterprise

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-09-22 13:19:20 -07:00
Chris S. Kim f972048ebc
connect: Allow upstream listener escape hatch for prepared queries (#11109) 2021-09-22 15:27:10 -04:00
Evan Culver 7e20a5e4f9
connect: remove support for Envoy 1.15 2021-09-22 11:48:50 -07:00
R.B. Boyer 706fc8bcd0
grpc: strip local ACL tokens from RPCs during forwarding if crossing datacenters (#11099)
Fixes #11086
2021-09-22 13:14:26 -05:00
Daniel Nephin 54256fb751 config: Move two more fields to DeprecatedConfig
And add a test for deprecated config fields.
2021-09-22 13:23:03 -04:00
Daniel Nephin 8ed14296ea config: Introduce DeprecatedConfig
This struct allows us to move all the deprecated config options off of
the main config struct, and keeps all the deprecation logic in a single
place, instead of spread across 3+ places.
2021-09-22 13:22:16 -04:00
Evan Culver 2d23f92b35
add 1.19.x versions to test config 2021-09-22 09:30:45 -07:00
Connor 1e3ba26223
Merge pull request #11090 from hashicorp/clly/kv-usage-metrics
Add KVUsage to consul state usage metrics
2021-09-22 11:26:56 -05:00
Connor Kelly ba706501e1
Strip out go 1.17 bits 2021-09-22 11:04:48 -05:00
Matt Keeler a6a359cc80 Add a mock Agent delegate to ease/improve some types of testing 2021-09-22 10:23:01 -04:00
hc-github-team-consul-core 47b99d0b78 auto-updated agent/uiserver/bindata_assetfs.go from commit 9c0233cf5 2021-09-22 13:05:38 +00:00
hc-github-team-consul-core 7efb015ca9 auto-updated agent/uiserver/bindata_assetfs.go from commit cfbd1bb84 2021-09-22 09:26:14 +00:00
Daniel Nephin 72f2199ea1 acl: remove remaining tests that use ACL.Apply
In preparation for removing ACL.Apply.

Tests for ACL.Apply, ACL.GetPolicy, and ACL upgrades were removed
because all 3 of those will be removed shortly.

The forth test appears to be for the ACLResolver cache, so the test was moved to the correct
test file, and the name was updated to make it obvious what is being tested.
2021-09-21 19:35:26 -04:00
Evan Culver 2798383dbc
regenerate envoy golden files 2021-09-21 16:21:00 -07:00
Evan Culver 7605dff46e
add envoy 1.19.1 2021-09-21 15:39:36 -07:00
Daniel Nephin eb991c18c2 fsm: restore the legacy commands
and emit a helpful error message.
2021-09-21 18:35:12 -04:00
Daniel Nephin fa4b33ab8f Convert tests to the new ACL system
In preparation for removing ACL.Apply
2021-09-21 18:35:12 -04:00
Daniel Nephin 5779af1f62 config: use the new ACL system in tests
In preparation for removing ACL.Apply
2021-09-21 17:57:29 -04:00
Daniel Nephin d64409f66f catalog: use the new ACL system in tests
In preparation for removing ACL.Apply
2021-09-21 17:57:29 -04:00
Daniel Nephin 3b9578d7eb Update 4 non-acl tests that used the legacy ACL.Apply
These tests don't really care about the endpoint, they just need some way to create an ACL token.
2021-09-21 17:57:29 -04:00
Daniel Nephin 746f67b3a1 acl: remove two commented out tests for legacy ACL replication
They were commented out in 2018.
2021-09-21 17:57:29 -04:00
Daniel Nephin abd9cd0e15 acl: replace legacy Get and List RPCs with an error impl
These endpoints are being removed as part of the legacy ACL system.
2021-09-21 17:57:29 -04:00
Daniel Nephin e7c63004a8 acl: remove a couple legacy ACL operation constants
structs.ACLForceSet was deprecated 4 years ago, it should be safe to remove now.
ACLBootstrapNow was removed in a recent commit. While it is technically possible that a cluster with mixed version
could still attempt a legacy boostrap, we documented that the legacy system was deprecated in 1.4, so no
clusters that are being upgraded should be attempting a legacy boostrap.
2021-09-21 17:57:29 -04:00
Daniel Nephin 868bfc7a0a acl: Remove unused ACLPolicyIDType 2021-09-21 17:57:29 -04:00
Daniel Nephin aee8a9511d
Merge pull request #10985 from hashicorp/dnephin/acl-legacy-remove-replication
acl: remove legacy ACL replication
2021-09-21 17:56:54 -04:00
Connor 1ddee0680c
Apply suggestions from code review
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
2021-09-21 10:52:46 -05:00
R.B. Boyer b2d17ac448
xds: fix representation of incremental xDS subscriptions (#10987)
Fixes #10563

The `resourceVersion` map was doing two jobs prior to this PR. The first job was
to track what version of every resource we know envoy currently has. The
second was to track subscriptions to those resources (by way of the empty
string for a version). This mostly works out fine, but occasionally leads to
consul removing a resource and accidentally (effectively) unsubscribing at the
same time.

The fix separates these two jobs. When all of the resources for a subscription
are removed we continue to track the subscription until envoy explicitly
unsubscribes
2021-09-21 09:58:56 -05:00
Connor Kelly ae3457b96a
Fix test 2021-09-20 13:44:43 -05:00
Connor Kelly 052f224ee5
Add KVUsage to consul state usage metrics
This change will add the number of entries in the consul KV store to the
already existing usage metrics.
2021-09-20 12:41:54 -05:00
R.B. Boyer 5fe613dd05
xds: ensure the active streams counters are 64 bit aligned on 32 bit systems (#11085) 2021-09-20 11:07:11 -05:00
Krastin Krastev f7b0938633
Update autopilot.go
Fixing a minuscule typo in logging
2021-09-20 14:40:58 +02:00
Freddy 8591620b5d
Merge pull request #11071 from hashicorp/partitions/ixn-decisions 2021-09-16 15:18:23 -06:00
freddygv 49248a0802 Fixup proxycfg tproxy case 2021-09-16 15:05:28 -06:00
freddygv fc8fc060a7 Remove ent checks from oss test 2021-09-16 14:53:28 -06:00
R.B. Boyer faa6fd0919
acl: ensure the global management policy grants all necessary partition privileges (#11072) 2021-09-16 15:53:10 -05:00
freddygv bf7a1358d6 Ensure partition is defaulted in authz 2021-09-16 14:39:01 -06:00
freddygv 47109e0c0c Default the partition in ixn check 2021-09-16 14:39:01 -06:00
freddygv 82d2caa288 Fixup test 2021-09-16 14:39:01 -06:00
freddygv 95a6db9cfa Account for partitions in ixn match/decision 2021-09-16 14:39:01 -06:00
Jeff Widman 2dc62aa0c4
Bump `go-discover` to fix broken dep tree (#10898) 2021-09-16 15:31:22 -04:00
hc-github-team-consul-core 42b7fd3e60 auto-updated agent/uiserver/bindata_assetfs.go from commit 1d9d3349c 2021-09-16 17:31:08 +00:00
R.B. Boyer ca73abdea1
acl: fix intention:*:write checks (#11061)
This is a partial revert of #10793
2021-09-16 11:08:45 -05:00
Freddy cd08a36ce0
Merge pull request #11051 from hashicorp/partitions/fixes 2021-09-16 09:29:00 -06:00
Freddy fcef19f94b
acl: small resolver changes to account for partitions (#11052)
Also refactoring the enterprise side of a test to make it easier to reason about.
2021-09-16 09:17:02 -05:00
freddygv 3f3a61c6e1 Fixup manager tests 2021-09-15 17:24:05 -06:00
freddygv 99c6e4fe41 Default partition in match endpoint 2021-09-15 17:23:52 -06:00
freddygv 77681b9f6c Pass partition to intention match query 2021-09-15 17:23:52 -06:00
freddygv 9cd30e8650 Ensure partition is used for SAN validation 2021-09-15 17:23:48 -06:00
Mark Anderson 9f12fbd3cc
ACL Binding Rules table partitioning (#11044)
* ACL Binding Rules table partitioning

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-09-15 13:26:08 -07:00
hc-github-team-consul-core 02051c141e auto-updated agent/uiserver/bindata_assetfs.go from commit fc14a412f 2021-09-15 18:55:29 +00:00
hc-github-team-consul-core 0eb4a98fab auto-updated agent/uiserver/bindata_assetfs.go from commit b16a6fa03 2021-09-15 17:14:42 +00:00
Dhia Ayachi af21578039
use const instead of literals for `tableIndex` (#11039) 2021-09-15 10:24:04 -04:00
Mark Anderson 6be54052f7
Refactor `indexAuthMethod` in `tableACLBindingRules` (#11029)
* Port consul-enterprise #1123 to OSS

Signed-off-by: Mark Anderson <manderson@hashicorp.com>

* Fixup missing query field

Signed-off-by: Mark Anderson <manderson@hashicorp.com>

* change to re-trigger ci system

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-09-15 09:34:19 -04:00
Freddy ce04ce13dd
Merge pull request #11024 from hashicorp/partitions/rbac 2021-09-14 11:18:19 -06:00
Freddy e18f3c1f6d
Update error texts (#11022)
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-09-14 11:08:06 -06:00
freddygv d90e30f009 Update spiffe ID patterns used for RBAC 2021-09-14 11:00:03 -06:00
freddygv 5e54f253d7 Expand testing of simplifyNotSourceSlice for partitions 2021-09-14 10:55:15 -06:00
freddygv 19da23be28 Expand testing of removeSameSourceIntentions for partitions 2021-09-14 10:55:09 -06:00
freddygv beab0cd962 Account for partition when matching src intentions 2021-09-14 10:55:02 -06:00
Daniel Nephin 1f9479603c
Add failures_before_warning to checks (#10969)
Signed-off-by: Jakub Sokołowski <jakub@status.im>

* agent: add failures_before_warning setting

The new setting allows users to specify the number of check failures
that have to happen before a service status us updated to be `warning`.
This allows for more visibility for detected issues without creating
alerts and pinging administrators. Unlike the previous behavior, which
caused the service status to not update until it reached the configured
`failures_before_critical` setting, now Consul updates the Web UI view
with the `warning` state and the output of the service check when
`failures_before_warning` is breached.

The default value of `FailuresBeforeWarning` is the same as the value of
`FailuresBeforeCritical`, which allows for retaining the previous default
behavior of not triggering a warning.

When `FailuresBeforeWarning` is set to a value higher than that of
`FailuresBeforeCritical it has no effect as `FailuresBeforeCritical`
takes precedence.

Resolves: https://github.com/hashicorp/consul/issues/10680

Signed-off-by: Jakub Sokołowski <jakub@status.im>

Co-authored-by: Jakub Sokołowski <jakub@status.im>
2021-09-14 12:47:52 -04:00
Dhia Ayachi b4d5860197
convert expiration indexed in ACLToken table to use `indexerSingle` (#11018)
* move intFromBool to be available for oss

* add expiry indexes

* remove dead code: `TokenExpirationIndex`

* fix remove indexer `TokenExpirationIndex`

* fix rebase issue
2021-09-13 14:37:16 -04:00
Dhia Ayachi 11f44dfcf8
add locality indexer partitioning (#11016)
* convert `Roles` index to use `indexerSingle`

* split authmethod write indexer to oss and ent

* add index locality

* add locality unit tests

* move intFromBool to be available for oss

* use Bool func

* refactor `aclTokenList` to merge func
2021-09-13 11:53:00 -04:00
Dhia Ayachi ba4ee6e67c
convert `indexAuthMethod` index to use `indexerSingle` (#11014)
* convert `Roles` index to use `indexerSingle`

* fix oss build

* split authmethod write indexer to oss and ent

* add auth method unit tests
2021-09-10 16:56:56 -04:00
Paul Banks b38e84df63 Include namespace and partition in error messages when validating ingress header manip 2021-09-10 21:11:00 +01:00
Paul Banks 1079089f20 Refactor HTTPHeaderModifiers.MergeDefaults based on feedback 2021-09-10 21:11:00 +01:00
Paul Banks 9e4e204e96 Fix enterprise test failures caused by differences in normalizing EnterpriseMeta 2021-09-10 21:11:00 +01:00
Paul Banks 3004eadd08 Fix enterprise discovery chain tests; Fix multi-level split merging 2021-09-10 21:11:00 +01:00
Paul Banks b5ae00d753 Remove unnecessary check 2021-09-10 21:09:24 +01:00
Paul Banks f1c0876b4c Fix discovery chain test fixtures 2021-09-10 21:09:24 +01:00
Paul Banks 1b9632531a Integration tests for all new header manip features 2021-09-10 21:09:24 +01:00
Paul Banks e22cc9c53a Header manip for split legs plumbing 2021-09-10 21:09:24 +01:00
Paul Banks 83fc8723a3 Header manip for service-router plumbed through 2021-09-10 21:09:24 +01:00
Paul Banks f439dfc04f Ingress gateway header manip plumbing 2021-09-10 21:09:24 +01:00
Paul Banks d776a2d236 Add HTTP header manip for router and splitter entries 2021-09-10 21:09:24 +01:00
Paul Banks 46e4041283 Header manip and validation added for ingress-gateway entries 2021-09-10 21:09:24 +01:00
Dhia Ayachi 6cac30aa22
convert `Roles` index to use `indexerMulti` (#11013)
* convert `Roles` index to use `indexerMulti`

* add role test in oss

* fix oss to use the right index func

* preallocate slice
2021-09-10 16:04:33 -04:00
Dhia Ayachi f3f0654038
convert indexPolicies in ACLTokens table to the new index (#11011) 2021-09-10 14:57:37 -04:00
Dhia Ayachi 584faec6e3
convert indexSecret to the new index (#11007) 2021-09-10 09:10:11 -04:00
Dhia Ayachi 6e6cf1c043
convert indexAccessor to the new index (#11002) 2021-09-09 16:28:04 -04:00
Hans Hasselberg 13238dbab6
tls: consider presented intermediates during server connection tls handshake. (#10964)
* use intermediates when verifying

* extract connection state

* remove useless import

* add changelog entry

* golint

* better error

* wording

* collect errors

* use SAN.DNSName instead of CommonName

* Add test for unknown intermediate

* improve changelog entry
2021-09-09 21:48:54 +02:00
Chris S. Kim 9bbfa048a2
Sync enterprise changes to oss (#10994)
This commit updates OSS with files for enterprise-specific admin partitions feature work
2021-09-08 11:59:30 -04:00
Kyle Havlovitz a14950025a
Merge pull request #10984 from hashicorp/mesh-resource
acl: adding a new mesh resource
2021-09-07 15:06:20 -07:00
Dhia Ayachi bc0e4f2f46
partition dicovery chains (#10983)
* partition dicovery chains

* fix default partition for OSS
2021-09-07 16:29:32 -04:00
Daniel Nephin f063402b29 acl: remove ACL.IsSame
The only caller of this method was removed in a recent commit along with replication.
2021-09-03 12:59:12 -04:00
Daniel Nephin d63cef1219 acl: remove legacy ACL replication 2021-09-03 12:42:06 -04:00
R.B. Boyer ee372a854a acl: adding a new mesh resource 2021-09-03 09:12:03 -04:00
Dhia Ayachi ced8329d80
try to infer command partition from node partition (#10981) 2021-09-03 08:37:23 -04:00
Dhia Ayachi 09197c989c
add partition to SNI when partition is non default (#10917) 2021-09-01 10:35:39 -04:00
Freddy 8d83d27674
connect: update envoy supported versions to latest patch release
(#10961)

Relevant advisory: 
https://github.com/envoyproxy/envoy/security/advisories/GHSA-6g4j-5vrw-2m8h
2021-08-31 10:39:18 -06:00
Evan Culver 79c7e73618
rpc: authorize raft requests (#10925) 2021-08-26 15:04:32 -07:00
hc-github-team-consul-core cd3333ad6a auto-updated agent/uiserver/bindata_assetfs.go from commit eeeb91bea 2021-08-26 18:13:08 +00:00
Chris S. Kim 1a9b2f09dd
ent->oss test fix (#10926) 2021-08-26 14:06:49 -04:00
hc-github-team-consul-core 2d66c4ea13 auto-updated agent/uiserver/bindata_assetfs.go from commit a907e1d87 2021-08-26 18:02:18 +00:00
hc-github-team-consul-core a163051dbb auto-updated agent/uiserver/bindata_assetfs.go from commit a0b0ed2bc 2021-08-26 16:06:09 +00:00
Chris S. Kim 45dcc8b553
api: expose upstream routing configurations in topology view (#10811)
Some users are defining routing configurations that do not have associated services. This commit surfaces these configs in the topology visualization. Also fixes a minor internal bug with non-transparent proxy upstream/downstream references.
2021-08-25 15:20:32 -04:00
R.B. Boyer a6d22efb49
acl: some acl authz refactors for nodes (#10909) 2021-08-25 13:43:11 -05:00
hc-github-team-consul-core 11b1dc1f97 auto-updated agent/uiserver/bindata_assetfs.go from commit a777b0a9b 2021-08-25 13:46:51 +00:00
hc-github-team-consul-core 5e31421602 auto-updated agent/uiserver/bindata_assetfs.go from commit 8192dde48 2021-08-25 11:39:14 +00:00
R.B. Boyer 5b6d96d27d
grpc: ensure that streaming gRPC requests work over mesh gateway based wan federation (#10838)
Fixes #10796
2021-08-24 16:28:44 -05:00
hc-github-team-consul-core 4993d877d9 auto-updated agent/uiserver/bindata_assetfs.go from commit 05a28c311 2021-08-24 16:04:24 +00:00
Giulio Micheloni 7fa01105cc Fix merge conflicts 2021-08-22 19:35:08 +01:00
Giulio Micheloni 655da1fc42
Merge branch 'main' into serve-panic-recovery 2021-08-22 20:31:11 +02:00
Giulio Micheloni 4b0eaa4bff grpc, xds: recovery middleware to return and log error in case of panic
1) xds and grpc servers:
   1.1) to use recovery middleware with callback that prints stack trace to log
   1.2) callback turn the panic into a core.Internal error
2) added unit test for grpc server
2021-08-22 19:06:26 +01:00
freddygv 01936ddb70 Avoid passing zero value into variadic 2021-08-20 17:40:33 -06:00
freddygv f52bd80f6d Update comment for test function 2021-08-20 17:40:33 -06:00
freddygv af52d21884 Update prepared query cluster SAN validation
Previously SAN validation for prepared queries was broken because we
validated against the name, namespace, and datacenter for prepared
queries.

However, prepared queries can target:

- Services with a name that isn't their own
- Services in multiple datacenters

This means that the SpiffeID to validate needs to be based on the
prepared query endpoints, and not the prepared query's upstream
definition.

This commit updates prepared query clusters to account for that.
2021-08-20 17:40:33 -06:00
freddygv 85878685b7 Fixup proxy config test fixtures
- The TestNodeService helper created services with the fixed name "web",
and now that name is overridable.

- The discovery chain snapshot didn't have prepared query endpoints so
the endpoints tests were missing data for prepared queries
2021-08-20 17:38:57 -06:00
R.B. Boyer fb27c1b24f
agent: add partition labels to catalog API metrics where appropriate (#10890) 2021-08-20 15:09:39 -05:00
R.B. Boyer d66a43f5f2
fixing various bits of enterprise meta plumbing to be more correct (#10889) 2021-08-20 14:34:23 -05:00
Dhia Ayachi 1950ebbe1f
oss portion of ent #1069 (#10883) 2021-08-20 12:57:45 -04:00
R.B. Boyer ac41e30614
state: partition the nodes.uuid and nodes.meta indexes as well (#10882) 2021-08-19 16:17:59 -05:00
R.B. Boyer 097e1645e3
agent: ensure that most agent behavior correctly respects partition configuration (#10880) 2021-08-19 15:09:42 -05:00
Daniel Nephin 271352dbb7
Merge pull request #10849 from hashicorp/dnephin/contrib-doc-xds-auth
xds: document how authorization works
2021-08-18 13:25:16 -04:00
R.B. Boyer e44bce3c4f
state: partition the usage metrics subsystem (#10867) 2021-08-18 09:27:15 -05:00
Daniel Nephin 8252a2691c xds: document how authorization works 2021-08-17 19:26:34 -04:00
R.B. Boyer 613dd7d053
state: adjust streaming event generation to account for partitioned nodes (#10860)
Also re-enabled some tests that had to be disabled in the prior PR.
2021-08-17 16:49:26 -05:00
R.B. Boyer 310e775a8a
state: partition nodes and coordinates in the state store (#10859)
Additionally:

- partitioned the catalog indexes appropriately for partitioning
- removed a stray reference to a non-existent index named "node.checks"
2021-08-17 13:29:39 -05:00
Daniel Nephin 01bf115c2b acl: small improvements to ACLResolver disable due to RPC error
Remove the error return, so that not handling is not reported as an
error by errcheck. It was returning the error passed as an arg
unmodified so there is no reason to return the same value that was
passed in.

Remove the term upstreams to remove any confusion with the term used in
service mesh.

Remove the AutoDisable field, and replace it with the TTL value, using 0
to indicate the setting is turned off.

Replace "not Before" with "After".

Add some test coverage to show the behaviour is still correct.
2021-08-17 13:34:18 -04:00
Daniel Nephin d5498770fa acl: make ACLDisabledTTL a constant
This field was never user-configurable. We always overwrote the value with 120s from
NonUserSource. However, we also never copied the value from RuntimeConfig to consul.Config,
So the value in NonUserSource was always ignored, and we used the default value of 30s
set by consul.DefaultConfig.

All of this code is an unnecessary distraction because a user can not actually configure
this value.

This commit removes the fields and uses a constant value instad. Someone attempting to set
acl.disabled_ttl in their config will now get an error about an unknown field, but previously
the value was completely ignored, so the new behaviour seems more correct.

We have to keep this field in the AutoConfig response for backwards compatibility, but the value
will be ignored by the client, so it doesn't really matter what value we set.
2021-08-17 13:34:18 -04:00
Daniel Nephin abd2e160f9 Fix test failures
Tests only specified one of the fields, but in production we copy the
value from a single place, so we can do the same in tests.

The AutoConfig test broke because of the problem noticed in a previous
commit. The DisabledTTL is not wired up properly so it reports 0s here.
Changed the test to use an explicit value.
2021-08-17 13:32:52 -04:00
Daniel Nephin 17841248dd config: remove ACLResolver settings from RuntimeConfig 2021-08-17 13:32:52 -04:00
Daniel Nephin 31e034215f acl: remove ACLResolver config fields from consul.Config 2021-08-17 13:32:52 -04:00
Daniel Nephin d4701903f6 acl: replace ACLResolver.Config with its own struct
This is step toward decoupling ACLResolver from the agent/consul
package.
2021-08-17 13:32:52 -04:00
Daniel Nephin c2b24adb5f acl: remove ACLRulesTranslateLegacyToken API endpoint 2021-08-17 13:10:02 -04:00
Daniel Nephin b7bced9bcf acl: remove legacy bootstrap
Return an explicit error from the RPC, and remove the flag from the HTTP API.
2021-08-17 13:10:00 -04:00
Daniel Nephin 858071d55a agent: update some tests that were using legacy ACL endpoints
The tests were updated to use the new ACL endpoints now that the legacy ones have been removed.
2021-08-17 13:09:30 -04:00
Daniel Nephin 7ecd2e5466 http: update legacy ACL endpoints to return an error
Also move a test for the ACLReplicationStatus endpoint into the correct file.
2021-08-17 13:09:29 -04:00
Daniel Nephin 9671dd6b97 acl: add some notes about removing legacy ACL system 2021-08-17 13:08:29 -04:00
Daniel Nephin 887d11923b
Merge pull request #10792 from hashicorp/dnephin/rename-authz-vars
acl: use authz consistently as the variable name for an acl.Authorizer
2021-08-17 13:07:17 -04:00
Daniel Nephin c85c62dffb
Merge pull request #10807 from hashicorp/dnephin/remove-acl-datacenter
config: remove ACLDatacenter
2021-08-17 13:07:09 -04:00
Daniel Nephin e637cd71f3 acl: use authz consistently as the variable name for an acl.Authorizer
Follow up to https://github.com/hashicorp/consul/pull/10737#discussion_r682147950

Renames all variables for acl.Authorizer to use `authz`. Previously some
places used `rule` which I believe was an old name carried over from the
legacy ACL system.

A couple places also used authorizer.

This commit also removes another couple of authorizer nil checks that
are no longer necessary.
2021-08-17 12:14:10 -04:00
hc-github-team-consul-core ed85684d96 auto-updated agent/uiserver/bindata_assetfs.go from commit ae9c31338 2021-08-16 16:10:17 +00:00
Kyle Havlovitz fa5df0349d
Merge pull request #10843 from hashicorp/partitions/rename-default
oss: Rename default partition
2021-08-12 14:45:53 -07:00
Kyle Havlovitz 073b6c8411 oss: Rename default partition 2021-08-12 14:31:37 -07:00
Daniel Nephin 0575498d0d proxycfg: Lookup the agent token as a default
When no ACL token is provided with the service registration.
2021-08-12 15:51:34 -04:00
Daniel Nephin b313f495b8 proxycfg: Add a test to show the bug
When a token is not provided at registration, the agent token is not being used.
2021-08-12 15:47:59 -04:00
Mike Morris 3bae53a989
deps: upgrade gogo-protobuf to v1.3.2 (#10813)
* deps: upgrade gogo-protobuf to v1.3.2

* go mod tidy using go 1.16

* proto: regen protobufs after upgrading gogo/protobuf

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-08-12 14:05:46 -04:00
Mark Anderson d3cebbd32c
Fixup to support unix domain socket via command line (#10758)
Missed the need to add support for unix domain socket config via
api/command line. This is a variant of the problems described in
it is easy to drop one.

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-08-12 10:05:22 -07:00
hc-github-team-consul-core 199e850d16 auto-updated agent/uiserver/bindata_assetfs.go from commit ab6a67520 2021-08-11 17:05:51 +00:00
Giulio Micheloni 2b14a9b59a grpc Server: turn panic into error through middleware 2021-08-07 13:21:12 +01:00
Daniel Nephin 67fc97522f server: remove defaulting of PrimaryDatacenter
The constructor for Server is not at all the appropriate place to be setting default
values for a config struct that was passed in.

In production this value is always set from agent/config. In tests we should set the
default in a test helper.
2021-08-06 18:45:24 -04:00
Daniel Nephin d3325b0253
Merge pull request #10612 from bigmikes/acl-replication-fix
acl: acl replication routine to report the last error message
2021-08-06 18:29:51 -04:00
Daniel Nephin 7160f7a614 acl: remove ACLDatacenter
This field has been unnecessary for a while now. It was always set to the same value
as PrimaryDatacenter. So we can remove the duplicate field and use PrimaryDatacenter
directly.

This change was made by GoLand refactor, which did most of the work for me.
2021-08-06 18:27:00 -04:00
Giulio Micheloni d4a3fe33e8 String type instead of error type and changelog. 2021-08-06 22:35:27 +01:00
Daniel Nephin b837ba35a0 acl: remove Server.ResolveTokenIdentityAndDefaultMeta
This method suffered from similar naming to a couple other methods on Server, and had not great
re-use (2 callers). By copying a few of the lines into one of the callers we can move the
implementation into the second caller.

Once moved, we can see that ResolveTokenAndDefaultMeta is identical in both Client and Server, and
likely should be further refactored, possibly into ACLResolver.

This change is being made to make ACL resolution easier to trace.
2021-08-05 15:20:13 -04:00
Daniel Nephin 6cf6e7c5fe acl: remove Server.ResolveTokenToIdentityAndAuthorizer
This method was an alias for ACLResolver.ResolveTokenToIdentityAndAuthorizer. By removing the
method that does nothing the code becomes easier to trace.
2021-08-05 15:20:13 -04:00
Daniel Nephin cc4f155801 acl: recouple acl filtering from ACLResolver
ACL filtering only needs an authorizer and a logger. We can decouple filtering from
the ACLResolver by passing in the necessary logger.

This change is being made in preparation for moving the ACLResolver into an acl package
2021-08-05 15:20:13 -04:00
Daniel Nephin 111f3620a8 acl: remove unused error return
filterACLWithAuthorizer could never return an error. This change moves us a little bit
closer to being able to enable errcheck and catch problems caused by unhandled error
return values.
2021-08-05 15:20:13 -04:00
Daniel Nephin c0100543d0 acl: rename acl.Authorizer vars to authz
For consistency
2021-08-05 15:19:47 -04:00
Daniel Nephin 4f5477ccfa acl: move vet functions
These functions are moved to the one place they are called to improve code locality.

They are being moved out of agent/consul/acl.go in preparation for moving
ACLResolver to an acl package.
2021-08-05 15:19:24 -04:00
Daniel Nephin c4eadb6b96 acl: move vetRegisterWithACL and vetDeregisterWithACL
These functions are used in only one place. Move the functions next to their one caller
to improve code locality.

This change is being made in preparation for moving the ACLResolver into an
acl package. The moved functions were previously in the same file as the ACLResolver.
By moving them out of that file we may be able to move the entire file
with fewer modifications.
2021-08-05 15:17:54 -04:00
Daniel Nephin 1deba421c7
Merge pull request #10770 from hashicorp/dnephin/log-cert-expiration
telemetry: add log message when certs are about to expire
2021-08-05 15:17:20 -04:00
Daniel Nephin 0c42b38c92
Merge pull request #10793 from hashicorp/dnephin/acl-intentions
acl: small cleanup of a couple Authorization flows
2021-08-05 15:16:49 -04:00
Dhia Ayachi b495036823
defer setting the state before returning to avoid stuck in `INITIALIZING` state (#10630)
* defer setting the state before returning to avoid being stuck in `INITIALIZING` state

* add changelog

* move comment with the right if statement

* ca: report state transition error from setSTate

* update comment to reflect state transition

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-08-05 14:51:19 -04:00
Daniel Nephin e94016872a
Merge pull request #10768 from hashicorp/dnephin/agent-tls-cert-expiration-metric
telemetry: add Agent TLS Certificate expiration metric
2021-08-04 18:42:02 -04:00
Daniel Nephin c718de730b acl: remove special handling of services in txn_endpoint
Follow up to: https://github.com/hashicorp/consul/pull/10738#discussion_r680190210

Previously we were passing an Authorizer that would always allow the
operation, then later checking the authorization using vetServiceTxnOp.

On the surface this seemed strange, but I think it was actually masking
a bug as well. Over time `servicePreApply` was changed to add additional
authorization for `service.Proxy.DestinationServiceName`, but because
we were passing a nil Authorizer, that authorization was not handled on
the txn_endpoint.

`TxnServiceOp.FillAuthzContext` has some special handling in enterprise,
so we need to make sure to continue to use that from the Txn endpoint.

This commit removes the `vetServiceTxnOp` function, and passes in the
`FillAuthzContext` function so that `servicePreApply` can be used by
both the catalog and txn endpoints. This should be much less error prone
and prevent bugs like this in the future.
2021-08-04 18:32:20 -04:00
hc-github-team-consul-core af9acf4943 auto-updated agent/uiserver/bindata_assetfs.go from commit bcd53e73a 2021-08-04 22:27:44 +00:00
Daniel Nephin 5b2e5882b4 acl: move check for Intention.DestinationName into Authorizer
Follow up to https://github.com/hashicorp/consul/pull/10737#discussion_r680134445

Move the check for the Intention.DestinationName into the Authorizer to remove the
need to check what kind of Authorizer is being used.

It sounds like this check is only for legacy ACLs, so is probably just a safeguard
.
2021-08-04 18:06:44 -04:00
Daniel Nephin bbce192b4d
Merge pull request #10738 from hashicorp/dnephin/remove-authorizer-nil-checks-2
acl: remove the last of the authz == nil checks
2021-08-04 17:41:40 -04:00
Daniel Nephin 9cdd823ffc
Merge pull request #10737 from hashicorp/dnephin/remove-authorizer-nil-checks
acl: remove authz == nil checks
2021-08-04 17:39:34 -04:00
Daniel Nephin 0e58e1ac4b telemetry: add log message when certs are about to expire 2021-08-04 14:18:59 -04:00
Daniel Nephin 9420506fae telemetry: fix a couple bugs in cert expiry metrics
1. do not emit the metric if Query fails
2. properly check for PrimaryUsersIntermediate, the logic was inverted

Also improve the logging by including the metric name in the log message
2021-08-04 13:51:44 -04:00
Daniel Nephin 8c575445da telemetry: add a metric for agent TLS cert expiry 2021-08-04 13:51:44 -04:00
Dhia Ayachi cfa9cf6d84
fix state index for `CAOpSetRootsAndConfig` op (#10675)
* fix state index for `CAOpSetRootsAndConfig` op

* add changelog

* Update changelog

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* remove the change log as it's not needed

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-08-04 13:07:49 -04:00
hc-github-team-consul-core 2f6c95011b auto-updated agent/uiserver/bindata_assetfs.go from commit 8ad1ab9c0 2021-08-04 16:47:13 +00:00
Evan Culver 710bd90ef7
checks: Add Interval and Timeout to API response (#10717) 2021-08-03 15:26:49 -07:00
Daniel Nephin 8cf1aa1bda acl: Remove the remaining authz == nil checks
These checks were a bit more involved. They were previously skipping some code paths
when the authorizer was nil. After looking through these it seems correct to remove the
authz == nil check, since it will never evaluate to true.
2021-07-30 14:55:35 -04:00
Daniel Nephin dc50b36b0f acl: remove acl == nil checks 2021-07-30 14:28:19 -04:00
Daniel Nephin 4f1a36629a acl: remove authz == nil checks
These case are already impossible conditions, because most of these functions already start
with a check for ACLs being disabled. So the code path being removed could never be reached.

The one other case (ConnectAuthorized) was already changed in a previous commit. This commit
removes an impossible branch because authz == nil can never be true.
2021-07-30 13:58:35 -04:00