Commit Graph

3206 Commits

Author SHA1 Message Date
Daniel Nephin a07a58a873 config-entry: use custom MarshalJSON for mesh type
So that the Kind field is added to the JSON object.
2021-04-29 17:49:43 -04:00
Daniel Nephin 62efaaab21 config-entry: remove Kind and Name field from Mesh config entry
No config entry needs a Kind field. It is only used to determine the Go type to
target. As we introduce new config entries (like this one) we can remove the kind field
and have the GetKind method return the single supported value.

In this case (similar to proxy-defaults) the Name field is also unnecessary. We always
use the same value. So we can omit the name field entirely.
2021-04-29 17:11:21 -04:00
R.B. Boyer 71d45a3460
Support Incremental xDS mode (#9855)
This adds support for the Incremental xDS protocol when using xDS v3. This is best reviewed commit-by-commit and will not be squashed when merged.

Union of all commit messages follows to give an overarching summary:

xds: exclusively support incremental xDS when using xDS v3

Attempts to use SoTW via v3 will fail, much like attempts to use incremental via v2 will fail.
Work around a strange older envoy behavior involving empty CDS responses over incremental xDS.
xds: various cleanups and refactors that don't strictly concern the addition of incremental xDS support

Dissolve the connectionInfo struct in favor of per-connection ResourceGenerators instead.
Do a better job of ensuring the xds code uses a well configured logger that accurately describes the connected client.
xds: pull out checkStreamACLs method in advance of a later commit

xds: rewrite SoTW xDS protocol tests to use protobufs rather than hand-rolled json strings

In the test we very lightly reuse some of the more boring protobuf construction helper code that is also technically under test. The important thing of the protocol tests is testing the protocol. The actual inputs and outputs are largely already handled by the xds golden output tests now so these protocol tests don't have to do double-duty.

This also updates the SoTW protocol test to exclusively use xDS v2 which is the only variant of SoTW that will be supported in Consul 1.10.

xds: default xds.Server.AuthCheckFrequency at use-time instead of construction-time
2021-04-29 13:54:05 -05:00
hc-github-team-consul-core cbabf82587 auto-updated agent/uiserver/bindata_assetfs.go from commit 5d89ba031 2021-04-29 09:46:32 +00:00
Freddy 701b49f772
Rename cluster config files to mesh as well (#10148) 2021-04-28 18:15:26 -06:00
Freddy 078c40425f
Rename "cluster" config entry to "mesh" (#10127)
This config entry is being renamed primarily because in k8s the name
cluster could be confusing given that the config entry applies across
federated datacenters.

Additionally, this config entry will only apply to Consul as a service
mesh, so the more generic "cluster" name is not needed.
2021-04-28 16:13:29 -06:00
Daniel Nephin 9b344b3300
Merge pull request #10112 from hashicorp/dnephin/remove-streaming-from-cache
streaming: replace agent/cache with submatview.Store
2021-04-28 17:31:42 -04:00
Daniel Nephin 3a27fce2ad submatview: fix godoc and comment typos
Co-authored-by: Paul Banks <banks@banksco.de>
2021-04-28 12:27:38 -04:00
hc-github-team-consul-core 4ad37c326e auto-updated agent/uiserver/bindata_assetfs.go from commit 345765f76 2021-04-28 11:21:09 +00:00
Daniel Nephin 11398892b3 submatview: avoid sorting results unnecessarily
Previous getFromView would call view.Result when the result may not have been returned
(because the index is updated past the minIndex. This would allocate a slice and sort it
for no reason, because the values would never be returned.

Fix this by re-ordering the operations in getFromView.

The test changes in this commit were an attempt to cover the case where
an update is received but the index does not exceed the minIndex.
2021-04-27 19:03:17 -04:00
Daniel Nephin 23e1cd6f93 submatview: only return materializer from getEntry
Also rename it to readEntry now that it doesn't return the entire entry. Based on feedback
in PR review, the full entry is not used by the caller, and accessing the fields wouldn't be
safe outside the lock, so it is safer to return only the Materializer
2021-04-27 19:03:17 -04:00
Daniel Nephin 46b7d0b573 submatview: godoc 2021-04-27 19:03:17 -04:00
Daniel Nephin d9537411b8 rpcclient/health: convert tests to the new submatview.Store interface
Also fixes a minor data race in Materializer. Capture the error before
releasing the lock.
2021-04-27 19:03:17 -04:00
Daniel Nephin 79a54f208e agent: fix data race in tests caused by grpc log init
grpclog.SetLoggerV2 is meant to be called only once before any gRPC requests are received, but
each test that uses TestAgent will call NewBaseDeps again. Use a sync.Once to prevent the grpc
logging from being re-initialized by each test.

This will mean that a test can't use a fake logger to capture logs from the gRPC server.
2021-04-27 19:03:17 -04:00
Daniel Nephin dacf50038d submatview: fix two flaky tests
These tests can flake when we get a notification for an earlier event.
Retry the read from update channel a few times to make sure we get the
event we expect.
2021-04-27 19:03:17 -04:00
Daniel Nephin c932833acb rpcclient:health: fix a data race and flake in tests
Split the TestStreamingClient into the two logical components the real
client uses. This allows us to test multiple clients properly.

Previously writing of ctx from multiple Subscribe calls was showing a
data race.

Once this was fixed a test started to fail because the request had to be
made with a greater index, so that the store.Get call did not return
immediately.
2021-04-27 19:03:17 -04:00
Daniel Nephin 5fa0dea63a rpcclient/health: fix data race in a test
The idleTTL was being written and read concurrently. Instead move the idleTTL to a struct
field so that when one test patches the TTL it does not impact others.

The background goroutines for the store can outlive a test because context cancellation
is async.
2021-04-27 19:03:17 -04:00
Daniel Nephin 10ec9c2be3 rpcclient: close the grpc.ClientConn on shutdown 2021-04-27 19:03:16 -04:00
Daniel Nephin e229b877d8 health: create health.Client in Agent.New 2021-04-27 19:03:16 -04:00
Daniel Nephin a16c377ca7 rpcclient/health: move all backend routing logic to client 2021-04-27 19:03:16 -04:00
Daniel Nephin 0558586dbd health: use blocking queries for near query parameter 2021-04-27 19:03:16 -04:00
Daniel Nephin 2a26085b2c connect: do not set QuerySource.Node
Setting this field to a value is equivalent to using the 'near' query paramter.
The intent is to sort the results by proximity to the node requesting
them. However with connect we send the results to envoy, which doesn't
care about the order, so setting this field is increasing the work
performed for no gain.

It is necessary to unset this field now because we would like connect
to use streaming, but streaming does not support sorting by proximity.
2021-04-27 19:03:16 -04:00
Daniel Nephin 4fb2ba9de7 submatview: move error return to NewMaterializer
So that we don't have to create views ahead of time, when we will never use that view.
2021-04-27 19:03:16 -04:00
Daniel Nephin 55a677b7d1 rpcclient/health: integrate submatview.Store into rpcclient/health 2021-04-27 19:03:16 -04:00
Daniel Nephin 6f29fa0de8 rpcclient: move streaming cache tests 2021-04-27 19:03:16 -04:00
Daniel Nephin 6ebb0f8b74 submatview: test Store.Run 2021-04-27 19:03:16 -04:00
Daniel Nephin 9854c424eb submatview: test store with Get and Notify calls together 2021-04-27 19:03:16 -04:00
Daniel Nephin c1063999fc submatview: more test cases for Store.Get
And simplify the return value by using a Result type.
2021-04-27 19:03:16 -04:00
Daniel Nephin 832f1a2847 submatview: reduce the getFromView implementation
Remove View.Result error return value, it was always nil, and seems like it will likely always remain nill
since it is simply reading a stored value.

Also replace some cache types with local types.
2021-04-27 19:03:16 -04:00
Daniel Nephin 1d9d7d0aa5 submatview: track requests instead of notifiers
And only start expiration time when the last request ends. This makes tracking expiry simpler, and
ensures that no entry can be expired while there are active requests.
2021-04-27 19:03:16 -04:00
Daniel Nephin 7d13465c7b submatview: set up expiry of materializers 2021-04-27 19:03:16 -04:00
Daniel Nephin c574108354 submatview: setup testing structure 2021-04-27 19:03:16 -04:00
Daniel Nephin ddddbdb990 submatview: rough outline of the Get and Notify methods. 2021-04-27 19:03:16 -04:00
Daniel Nephin 3d52abb5e2 submatview: setup the interface for the store 2021-04-27 19:03:15 -04:00
Matt Keeler 65d73771a5
Add prometheus guage definitions for replication metrics. (#10109) 2021-04-23 17:05:33 -04:00
Matt Keeler ecbccdc261
Add replication metrics (#10073) 2021-04-22 11:20:53 -04:00
hashicorp-ci 8f3223a984 auto-updated agent/uiserver/bindata_assetfs.go from commit 7a095e7d2 2021-04-22 11:28:27 +00:00
hashicorp-ci 78886e8f4f auto-updated agent/uiserver/bindata_assetfs.go from commit f6f282ac7 2021-04-22 11:21:21 +00:00
Daniel Nephin 2e4dc7b705
Merge pull request #10045 from hashicorp/dnephin/state-proxy-defaults
state: remove config-entries kind index
2021-04-20 17:12:50 -04:00
Daniel Nephin 87cd3fc2e8 Handle ErrChunkingResubmit.Error properly
Previously canRetry was attempting to retrieve this error from args, however there was never
any callers that would pass an error to args.

With the change to raftApply to move this error to the error return value, it is now possible
to receive this error from the err argument.

This commit updates canRetry to check for ErrChunkingResubmit in err.
2021-04-20 13:29:31 -04:00
Daniel Nephin d2ab767fef Handle FSM.Apply errors in raftApply
Previously we were inconsistently checking the response for errors. This
PR moves the response-is-error check into raftApply, so that all callers
can look at only the error response, instead of having to know that
errors could come from two places.

This should expose a few more errors that were previously hidden because
in some calls to raftApply we were ignoring the response return value.

Also handle errors more consistently. In some cases we would log the
error before returning it. This can be very confusing because it can
result in the same error being logged multiple times. Instead return
a wrapped error.
2021-04-20 13:29:29 -04:00
Freddy f265dd635f
Disallow * as service-defaults name (#10069) 2021-04-19 14:23:01 -06:00
R.B. Boyer 70f1ba3a2b
fix test when run in enterprise (#10048) 2021-04-15 17:25:25 -05:00
freddygv a0f3591aee Don't panic on nil UpstreamConfiguration.Clone() 2021-04-15 15:46:21 -06:00
Daniel Nephin b57b3726d2 state: remove unnecessary kind index
The query can be performed using a prefix query on the ID index.

Also backport some enterprise changes to prevent conflicts.
2021-04-15 17:37:28 -04:00
Daniel Nephin 653f41ce41 state: use index constants for ConfigEntry indexes 2021-04-15 17:30:07 -04:00
hashicorp-ci 131599bfab auto-updated agent/uiserver/bindata_assetfs.go from commit daf897f1a 2021-04-15 20:19:13 +00:00
Freddy 3be304be16
Merge pull request #10016 from hashicorp/topology-update 2021-04-15 14:11:23 -06:00
Freddy 439a7fce2d
Split Upstream.Identifier() so non-empty namespace is always prepended in ent (#10031) 2021-04-15 13:54:40 -06:00
R.B. Boyer 4db8b78854
connect: update centralized upstreams representation in service-defaults (#10015) 2021-04-15 14:21:44 -05:00