Commit Graph

3938 Commits

Author SHA1 Message Date
Dan Upton 7efab269c0
Rename `Master` and `AgentMaster` fields in config protobuf (#11764) 2021-12-07 19:59:38 +00:00
Chris S. Kim f8f8580ab2
Godocs updates for catalog endpoints (#11716) 2021-12-07 10:18:28 -05:00
Mathew Estafanous 0a9621ec7a
Transition all endpoint tests in agent_endpoint_test.go to go through ServeHTTP (#11499) 2021-12-07 09:44:03 -05:00
Dan Upton 205ce9a69d
Remove references to "master" ACL tokens in tests (#11751) 2021-12-07 12:48:50 +00:00
Dan Upton 7fe81171d9
Rename `ACLMasterToken` => `ACLInitialManagementToken` (#11746) 2021-12-07 12:39:28 +00:00
Dan Upton 3a91815169
agent/token: rename `agent_master` to `agent_recovery` (internally) (#11744) 2021-12-07 12:12:47 +00:00
R.B. Boyer 9315a9812f return the max 2021-12-06 15:36:52 -06:00
freddygv 60fe5f75bb Remove support for failover to partition
Failing over to a partition is more siimilar to failing over to another
datacenter than it is to failing over to a namespace. In a future
release we should update how localities for failover are specified. We
should be able to accept a list of localities which can include both
partition and datacenter.
2021-12-06 12:32:24 -07:00
freddygv 5c1f7aa372 Allow cross-partition references in disco chain
* Add partition fields to targets like service route destinations
* Update validation to prevent cross-DC + cross-partition references
* Handle partitions when reading config entries for disco chain
* Encode partition in compiled targets
2021-12-06 12:32:19 -07:00
R.B. Boyer b1605639fc
light refactors to support making partitions and serf-based wan federation are mutually exclusive (#11755) 2021-12-06 13:18:02 -06:00
R.B. Boyer e20e6348dd
areas: make the gRPC server tracker network area aware (#11748)
Fixes a bug whereby servers present in multiple network areas would be
properly segmented in the Router, but not in the gRPC mirror. This would
lead servers in the current datacenter leaving from a network area
(possibly during the network area's removal) from deleting their own
records that still exist in the standard WAN area.

The gRPC client stack uses the gRPC server tracker to execute all RPCs,
even those targeting members of the current datacenter (which is unlike
the net/rpc stack which has a bypass mechanism).

This would manifest as a gRPC method call never opening a socket because
it would block forever waiting for the current datacenter's pool of
servers to be non-empty.
2021-12-06 09:55:54 -06:00
Freddy a725f06c83
Merge pull request #11739 from hashicorp/ap/exports-rename 2021-12-06 08:20:50 -07:00
freddygv e91509383f Clean up additional refs to partition exports 2021-12-04 15:16:40 -07:00
freddygv ed6076db26 Rename partition-exports to exported-services
Using a name less tied to partitions gives us more flexibility to use
this config entry in OSS for exports between datacenters/meshes.
2021-12-03 17:47:31 -07:00
freddygv f5b25401b3 Update intention topology to use new table 2021-12-03 17:28:31 -07:00
freddygv 55970c6ccd Avoid updating default decision from wildcard ixn
Given that we do not allow wildcard partitions in intentions, no one ixn
can override the DefaultAllow setting. Only the default ACL policy
applies across all partitions.
2021-12-03 17:28:12 -07:00
freddygv 497aab669f Add a new table to query service names by kind
This table purposefully does not index by partition/namespace. It's a
global view into all service names.

This table is intended to replace the current serviceListTxn watch in
intentionTopologyTxn. For cross-partition transparent proxying we need
to be able to calculate upstreams from intentions in any partition. This
means that the existing serviceListTxn function is insufficient since
it's scoped to a partition.

Moving away from that function is also beneficial because it watches the
main "services" table, so watchers will wake up when any instance is
registered or deregistered.
2021-12-03 17:28:12 -07:00
freddygv e7a7042c69 Update listener generation to account for consul VIP 2021-12-03 17:27:56 -07:00
Freddy f032d6ef05
Merge pull request #11680 from hashicorp/ap/partition-exports-oss 2021-12-03 16:57:50 -07:00
Dan Upton 3b9dfca88d
internal: support `ResultsFilteredByACLs` flag/header (#11643) 2021-12-03 23:04:24 +00:00
Dan Upton c8204330ed
query: support `ResultsFilteredByACLs` in query list endpoint (#11620) 2021-12-03 23:04:09 +00:00
Dhia Ayachi ce326b6074
port oss changes (#11736) 2021-12-03 17:23:55 -05:00
Freddy e246defb6c
Merge pull request #11720 from hashicorp/bbolt 2021-12-03 14:44:36 -07:00
Dan Upton 047aa2ffb0
fedstate: support `ResultsFilteredByACLs` in `ListMeshGateways` endpoint (#11644) 2021-12-03 20:56:55 +00:00
Dan Upton 361d9c2862
catalog: support `ResultsFilteredByACLs` flag/header (#11594) 2021-12-03 20:56:14 +00:00
Dan Upton 4c0956c03a
coordinate: support `ResultsFilteredByACLs` flag/header (#11617) 2021-12-03 20:51:02 +00:00
Dan Upton bf1e2ca551
sessions: support `ResultsFilteredByACLs` flag/header (#11606) 2021-12-03 20:43:43 +00:00
Dan Upton d92f0d84c6
txn: support `ResultsFilteredByACLs` flag in `Read` endpoint (#11632) 2021-12-03 20:41:03 +00:00
Dan Upton 547aa219ea
agent: support `X-Consul-Results-Filtered-By-ACLs` header in agent-local endpoints (#11610) 2021-12-03 20:36:28 +00:00
Dhia Ayachi 86159c6ed8
sessions partitioning tests (#11734)
* state: port KV and Tombstone tables to new pattern

* go fmt'ed

* handle wildcards for tombstones

* Fix graveyard ent vs oss

* fix oss compilation error

* add partition to tombstones and kv state store indexes

* refactor to use `indexWithEnterpriseIndexable`

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add `singleValueID` implementation assertions

* partition `tableSessions` table

* fix sessions to use UUID and fix prefix index

* fix oss build

* clean up unused functions

* fix oss compilation

* add a partition indexer for sessions

* Fix oss to not have partition index

* fix oss tests

* remove unused operations_ent.go and operations_oss.go func

* remove unused const

* convert `IndexID` of `session_checks` table

* convert `indexSession` of `session_checks` table

* convert `indexNodeCheck` of `session_checks` table

* partition `indexID` and `indexSession` of `tableSessionChecks`

* fix oss linter

* fix review comments

* remove partition for Checks as it's always use the session partition

* fix tests

* fix tests

* do not namespace nodeChecks index

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2021-12-03 15:36:07 -05:00
Dan Upton c314be2ff9
intention: support `ResultsFilteredByACLs` flag/header (#11612) 2021-12-03 20:35:54 +00:00
Mark Anderson a89ffba2d4
Cross port of ent #1383 (#11726)
Cross port of ent #1383 "Reject non-default datacenter when making partitioned ACLs"

On the OSS side this is a minor refactor to add some more checks that are only applicable to enterprise code.

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-12-03 10:20:25 -08:00
Dan Upton 599a4d6619
config: support `ResultsFilteredByACLs` in list/list all endpoints (#11621) 2021-12-03 17:39:47 +00:00
Dan Upton c4c68915c9
event: support `X-Consul-Results-Filtered-By-ACLs` header in list (#11616) 2021-12-03 17:38:59 +00:00
Dan Upton 474ef7cc1f
kv: support `ResultsFilteredByACLs` in list/list keys (#11593) 2021-12-03 17:31:48 +00:00
Dan Upton cf1bd585f6
health: support `ResultsFilteredByACLs` flag/header (#11602) 2021-12-03 17:31:32 +00:00
Dan Upton 1e47e3c82b
Groundwork for exposing when queries are filtered by ACLs (#11569) 2021-12-03 17:11:26 +00:00
Kyle Havlovitz 0546bbe08a dns: add endpoint for querying service virtual IPs 2021-12-02 16:40:28 -08:00
Kyle Havlovitz 6f34a4f777
Merge pull request #11724 from hashicorp/service-virtual-ips
oss: add virtual IP generation for connect services
2021-12-02 16:16:57 -08:00
Kyle Havlovitz 4f2cfee4b0 consul: add virtual IP generation for connect services 2021-12-02 15:42:47 -08:00
R.B. Boyer c46f9f9f31
agent: add variation of force-leave that exclusively works on the WAN (#11722)
Fixes #6548
2021-12-02 17:15:10 -06:00
Matt Keeler c7a94843ee Emit raft-boltdb metrics 2021-12-02 16:56:15 -05:00
Daniel Nephin e47cecc653 config: add NoFreelistSync option
# Conflicts:
#	agent/config/testdata/TestRuntimeConfig_Sanitize-enterprise.golden
#	agent/consul/server.go
2021-12-02 16:56:15 -05:00
Matt Keeler 42a5635bc3 Use raft-boltdb/v2 2021-12-02 16:56:15 -05:00
Daniel Nephin 17a2d14d49 ca: set the correct SigningKeyID after config update with Vault provider
The test added in this commit shows the problem. Previously the
SigningKeyID was set to the RootCert not the local leaf signing cert.

This same bug was fixed in two other places back in 2019, but this last one was
missed.

While fixing this bug I noticed I had the same few lines of code in 3
places, so I extracted a new function for them.

There would be 4 places, but currently the InitializeCA flow sets this
SigningKeyID in a different way, so I've left that alone for now.
2021-12-02 16:07:11 -05:00
Daniel Nephin 96f95889db
Merge pull request #11713 from hashicorp/dnephin/ca-test-names
ca: make test naming consistent
2021-12-02 16:05:42 -05:00
Daniel Nephin ff4581092e
Merge pull request #11671 from hashicorp/dnephin/ca-fix-storing-vault-intermediate
ca: fix storing the leaf signing cert with Vault provider
2021-12-02 16:02:24 -05:00
Daniel Nephin 81afb208ac
Merge pull request #11677 from hashicorp/dnephin/freeport-interface
sdk: use t.Cleanup in freeport and remove unnecessary calls
2021-12-02 15:58:41 -05:00
Daniel Nephin 447097b166 ca: make test naming consistent
While working on the CA system it is important to be able to run all the
tests related to the system, without having to wait for unrelated tests.
There are many slow and unrelated tests in agent/consul, so we need some
way to filter to only the relevant tests.

This PR renames all the CA system related tests to start with either
`TestCAMananger` for tests of internal operations that don't have RPC
endpoint, or `TestConnectCA` for tests of RPC endpoints. This allows us
to run all the test with:

    go test -run 'TestCAMananger|TestConnectCA' ./agent/consul

The test naming follows an undocumented convention of naming tests as
follows:

    Test[<struct name>_]<function name>[_<test case description>]

I tried to always keep Primary/Secondary at the end of the description,
and _Vault_ has to be in the middle because of our regex to run those
tests as a separate CI job.

You may notice some of the test names changed quite a bit. I did my best
to identify the underlying method being tested, but I may have been
slightly off in some cases.
2021-12-02 14:57:09 -05:00
FFMMM 384d497f26
add MustRevalidate flag to connect_ca_leaf cache type; always use on non-blocking queries (#11693)
* always use MustRevalidate on non-blocking queries for connect ca leaf

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Update agent/agent_endpoint_test.go

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* pr feedback

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-12-02 11:32:15 -08:00