Commit Graph

3071 Commits

Author SHA1 Message Date
Daniel Nephin 69ce10602f state: add tests for config-entry indexers 2021-03-17 14:41:46 -04:00
Daniel Nephin a414649543 state: convert config-entries kind index to new pattern 2021-03-17 14:40:57 -04:00
Daniel Nephin aadf187094 state: remove config-entries namespace index
Use a prefix of the ID index instead.
2021-03-17 14:40:57 -04:00
Daniel Nephin d70bbf671a state: remove unnecessary method receiver 2021-03-17 14:40:57 -04:00
Daniel Nephin 650ac62098 state: convert config-entries table to new indexer pattern
Using functional indexes to isolate enterprise differentiation and
remove reflection.
2021-03-17 14:40:57 -04:00
Daniel Nephin 9f03e23e44
Merge pull request #9881 from hashicorp/dnephin/state-index-service-check-nodes
state: convert services.node and checks.node indexes
2021-03-17 14:12:02 -04:00
Daniel Nephin bd6332ae25
Merge pull request #9863 from hashicorp/dnephin/config-entry-kind-name
state: move ConfigEntryKindName
2021-03-17 14:09:39 -04:00
hashicorp-ci 583743424f auto-updated agent/uiserver/bindata_assetfs.go from commit 9e715842d 2021-03-17 16:03:18 +00:00
hashicorp-ci bac1afbb89 auto-updated agent/uiserver/bindata_assetfs.go from commit f9e8b26af 2021-03-17 14:45:58 +00:00
hashicorp-ci 6872c33881 auto-updated agent/uiserver/bindata_assetfs.go from commit aca797658 2021-03-17 11:27:44 +00:00
hashicorp-ci f4a96768d1 auto-updated agent/uiserver/bindata_assetfs.go from commit 41471719e 2021-03-17 10:50:59 +00:00
freddygv 3f2489c31d Refactor makePublicListener
By accepting a name the function can be used for other inbound listeners,
like the one for TransparentProxy.
2021-03-16 19:22:26 -06:00
Christopher Broglie f0307c73e5 Add support for configuring TLS ServerName for health checks
Some TLS servers require SNI, but the Golang HTTP client doesn't
include it in the ClientHello when connecting to an IP address. This
change adds a new TLSServerName field to health check definitions to
optionally set it. This fixes #9473.
2021-03-16 18:16:44 -04:00
freddygv 7892964a0c Add cache-type for Internal.IntentionUpstreams 2021-03-16 11:06:47 -06:00
Daniel Nephin 34eb6c01ff state: convert services.node and checks.node indexes
Using NodeIdentity to share the indexes with both.
2021-03-16 13:00:31 -04:00
freddygv 942334b208 Prefix match type vars to match use 2021-03-16 09:49:24 -06:00
freddygv 4cb9fdc27f Pass txn into service list queries 2021-03-16 09:33:08 -06:00
freddygv 86ff9065c1 Pass txn into intention match queries 2021-03-16 08:03:52 -06:00
freddygv 31e757de2a Replace CertURI.Authorize() calls.
AuthorizeIntentionTarget is a generalized version of the old function,
and can be evaluated against sources or destinations.
2021-03-15 18:06:04 -06:00
freddygv f5ed751c91 Fixup typo, comments, and regression 2021-03-15 17:50:47 -06:00
freddygv 4bdbcff9c0 Fixup upstream test 2021-03-15 17:20:30 -06:00
freddygv 3492f9e0d6 Finish cleanup from ServiceConfigRequest changes 2021-03-15 16:38:01 -06:00
freddygv 770c5552d6 Update service manager to pass MeshGateway with config req 2021-03-15 16:08:03 -06:00
freddygv 6090cfcf68 PR comments 2021-03-15 16:02:03 -06:00
freddygv 7df846aa24 Pass MeshGateway config in service config request
ResolveServiceConfig is called by service manager before the proxy
registration is in the catalog. Therefore we should pass proxy
registration flags in the request rather than trying to fetch
them from the state store (where they may not exist yet).
2021-03-15 14:32:13 -06:00
freddygv 8b46d8dcbb Restore old Envoy prefix on escape hatches
This is done because after removing ID and NodeName from
ServiceConfigRequest we will no longer know whether a request coming in
is for a Consul client earlier than v1.10.
2021-03-15 14:12:57 -06:00
freddygv 93c3c1780d Only lowercase the protocol when normalizing 2021-03-15 14:12:15 -06:00
freddygv 41b2ba1e58 Add omitempty across the board for UpstreamConfig 2021-03-15 13:23:18 -06:00
freddygv 08759e46ed Add RPC endpoint for intention upstreams 2021-03-15 08:50:35 -06:00
freddygv 08737fa606 Add state store function for intention upstreams 2021-03-15 08:50:35 -06:00
freddygv 3722ce2fff Refactor IntentionDecision
This enables it to be called for many upstreams or downstreams of a
service while only querying intentions once.

Additionally, decisions are now optionally denied due to L7 permissions
being present. This enables the function to be used to filter for
potential upstreams/downstreams of a service.
2021-03-15 08:50:35 -06:00
Daniel Nephin f40b76af2d proxycfg: use rpcclient/health.Client instead of passing around cache name
This should allow us to swap out the implementation with something other
than `agent/cache` without making further code changes.
2021-03-12 11:46:04 -05:00
Daniel Nephin 566741a143 catalog_events: set the right key for connect snapshots 2021-03-12 11:35:43 -05:00
Daniel Nephin 906834ce8e proxycfg: Use streaming in connect state 2021-03-12 11:35:42 -05:00
Daniel Nephin 1a764553c0 rpcclient: use streaming for connect health 2021-03-12 11:35:42 -05:00
freddygv d80e4b27b1 Update content hash due to new field 2021-03-11 19:59:19 -07:00
freddygv 682f357185 Fixup more tests 2021-03-11 16:26:55 -07:00
freddygv 756ab4c546 Fixup protobufs and tests 2021-03-11 14:58:59 -07:00
Kyle Havlovitz 1e87c7183a
Merge pull request #9672 from hashicorp/ca-force-skip-xc
connect/ca: Allow ForceWithoutCrossSigning for all providers
2021-03-11 11:49:15 -08:00
freddygv df1f3995f8 Update service manager to store centrally configured upstreams 2021-03-11 11:37:21 -07:00
freddygv 6fd30d0384 Add TransparentProxy opt to proxy definition 2021-03-11 11:37:21 -07:00
freddygv 306ef7d252 Restore old escape hatch alias 2021-03-11 11:36:35 -07:00
freddygv e3dc2a49df Turn Limits and PassiveHealthChecks into pointers 2021-03-11 11:04:40 -07:00
hashicorp-ci f3556f6bba auto-updated agent/uiserver/bindata_assetfs.go from commit fa6687b7f 2021-03-11 09:34:21 +00:00
freddygv acec711a6a Update server-side config resolution and client-side merging 2021-03-10 21:05:11 -07:00
freddygv 1710ec87d2 finish moving UpstreamConfig and related fields to structs pkg 2021-03-10 21:04:13 -07:00
Daniel Nephin 9d924a81a9
Merge pull request #9797 from hashicorp/dnephin/state-index-node-id
state: convert nodes.ID to the new pattern of functional indexers
2021-03-10 17:34:23 -05:00
Daniel Nephin b06b3dd8f8 state: move ConfigEntryKindName
Previously this type was defined in structs, but unlike the other types in structs this type
is not used by RPC requests. By moving it to state we can better indicate that this is not
an API type, but part of the state implementation.
2021-03-10 12:27:22 -05:00
Daniel Nephin 948d1a317d
Merge pull request #9796 from hashicorp/dnephin/state-cleanup-catalog-index-oss
state: remove duplicate tableCheck indexes
2021-03-10 12:20:09 -05:00
Daniel Nephin 3a3007298f
Merge pull request #9851 from panascais-forks/fix-wan-ipv6-key
Fix advertise_addr_wan_ipv6 configuration key
2021-03-10 11:56:07 -05:00
Daniel Nephin 71b0f0a7a6 structs: remove EnterpriseMeta.GetNamespace
I added this recently without realizing that the method already existed and was named
NamespaceOrEmpty. Replace all calls to GetNamespace with NamespaceOrEmpty or NamespaceOrDefault
as appropriate.
2021-03-09 15:17:26 -05:00
Daniel Nephin 2b612a8e92
Merge pull request #9671 from hashicorp/streaming/terminating-gateway-events
state: Add terminating gateway events for streaming
2021-03-09 14:20:21 -05:00
Daniel Nephin 23421e190c state: adjust compare for catalog events
Document that this comparison should roughly match MatchesKey

Only sort by overrideKey or service name, but not both
Add namespace to the sort.

The client side also builds a map of these based on the namespace/node/service key, so the only order
that really matters is the ordering of register/dereigster events.
2021-03-09 14:00:36 -05:00
Daniel Nephin 68ec20f66a state: handle terminating gateway events properly in snapshot
Refactored out a function that can be used for both the snapshot and stream of events to translate
an event into an appropriate connect event.

Previously terminating gateway events would have used the wrong key in the snapshot, which would have
caused them to be filtered out later on.

Also removed an unused function, and some commented out code.
2021-03-09 14:00:35 -05:00
Kyle Havlovitz db572aca59 Add remaining terminating gateway tests for namespaces
Co-Authored-By: Daniel Nephin <dnephin@hashicorp.com>
2021-03-09 14:00:35 -05:00
Daniel Nephin 701285e470 Start to setup enterprise tests for terminating gateway streaming events.
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:35 -05:00
Daniel Nephin ae368768e5 state: Add support for override of namespace
in MatchesKey
also tests for MatchesKey

Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:35 -05:00
Daniel Nephin 4756ff059d state: update calls to ensureConfigEntryTxn
The EnterpriseMeta paramter was removed after this code was written, but before it merged.

Also the table name constant has changed.
2021-03-09 14:00:35 -05:00
Daniel Nephin 30a575dd33 state: add 2 more test cases for terminate gateway streaming events
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:34 -05:00
Kyle Havlovitz a21be5efa8 Added 6 new test cases for terminating gateway events
Co-Authored-By: Daniel Nephin <dnephin@hashicorp.com>
2021-03-09 14:00:34 -05:00
Daniel Nephin 06b1c32e25 state: Add two more tests for connect events with terminating gateways
And expand one test case to cover more.

Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:34 -05:00
Daniel Nephin eb58a39738 state: Include the override key in the sorting of events
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:34 -05:00
Kyle Havlovitz c2481ca10f state: Add terminating gateway events on updating a config entry
Co-Authored-By: Daniel Nephin <dnephin@hashicorp.com>
2021-03-09 14:00:34 -05:00
Daniel Nephin 28de159c14 state: add first terminating catalog catalog event
Health of a terminating gateway instance changes
- Generate an event for creating/destroying this instance of the terminating gateway,
  duplicate it for each affected service

Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:33 -05:00
Silas Rech ab9c484137
Fix advertise_addr_wan_ipv6 configuration key 2021-03-09 14:56:44 +01:00
hashicorp-ci 0e9250df80 auto-updated agent/uiserver/bindata_assetfs.go from commit 33d038377 2021-03-09 09:35:32 +00:00
freddygv 87cde19b4c Create new types for service-defaults upstream cfg 2021-03-08 22:10:27 -07:00
hashicorp-ci 35daee45bc auto-updated agent/uiserver/bindata_assetfs.go from commit 308e5a480 2021-03-08 12:28:15 +00:00
Daniel Nephin a4e68e32d6 state: convert nodes.ID to new functional pattern
In preparation for adding other identifiers to the index.
2021-03-05 12:30:40 -05:00
R.B. Boyer 398b766532
xds: default to speaking xDS v3, but allow for v2 to be spoken upon request (#9658)
- Also add support for envoy 1.17.0
2021-02-26 16:23:15 -06:00
Daniel Nephin 6b95e8dfe2
Merge pull request #9188 from hashicorp/dnephin/more-streaming-tests
Add more streaming tests
2021-02-26 12:36:55 -05:00
Daniel Nephin 566efad2e7
Merge pull request #9759 from hashicorp/dnephin/streaming-default-rpc-enabled
streaming: default rpc.enable_streaming to true
2021-02-26 12:08:00 -05:00
Daniel Nephin 5c8a6311b6
Merge pull request #9703 from pierresouchay/streaming_tags_and_case_insensitive
Streaming filter tags + case insensitive lookups for Service Names
2021-02-26 12:06:26 -05:00
Daniel Nephin 55add28725 catalog_events: set the right key for connect snapshots
Add a test for catalog_event snapshot on connect topic
2021-02-25 14:30:39 -05:00
Daniel Nephin 432dd2d204 consul: Add integration tests of streaming.
Restored from streaming-rpc-final branch.

Co-authored-by: Paul Banks <banks@banksco.de>
2021-02-25 14:30:39 -05:00
Daniel Nephin b7f8e3bad2 state: Add a test for ServiceHealthSnapshot 2021-02-25 14:08:10 -05:00
Daniel Nephin 1d2d15b1e1 agent: add a test for streaming in the service health endpoint
Co-authored-by: Paul Banks <banks@banksco.de>
2021-02-25 14:08:10 -05:00
Daniel Nephin 3aec942b17 streaming: default rpc.enable_streaming to true
So that all servers will start the grpc server used by streaming
2021-02-25 14:06:04 -05:00
hashicorp-ci b76dfa1441 auto-updated agent/uiserver/bindata_assetfs.go from commit 779f7f7b6 2021-02-25 09:41:02 +00:00
John Cowen 5892e75452
ui: Remove any trailing fullstop/period DNS characters from Gateways UI API (#9752)
Previous to this commit, the API response would include Gateway
Addresses in the form `domain.name.:8080`, which due to the addition of
the port is probably not the expected response.

This commit rightTrims any `.` characters from the end of the domain
before formatting the address to include the port resulting in
`domain.name:8080`
2021-02-25 09:34:47 +00:00
R.B. Boyer be89557fb4
test: omit envoy golden test files that differ from the latest version (#9807)
Since we currently do no version switching this removes 75% of the PR
noise.

To generate all *.golden files were removed and then I ran:

    go test ./agent/xds -update
2021-02-24 14:04:31 -06:00
hashicorp-ci f0872a4ac1 auto-updated agent/uiserver/bindata_assetfs.go from commit f9b0e50ca 2021-02-24 09:08:42 +00:00
hashicorp-ci 1413e9ddca auto-updated agent/uiserver/bindata_assetfs.go from commit 8b12d0d09 2021-02-23 09:02:41 +00:00
R.B. Boyer 3b6ffc447b
xds: remove deprecated usages of xDS (#9602)
Note that this does NOT upgrade to xDS v3. That will come in a future PR.

Additionally:

- Ignored staticcheck warnings about how github.com/golang/protobuf is deprecated.
- Shuffled some agent/xds imports in advance of a later xDS v3 upgrade.
- Remove support for envoy 1.13.x but don't add in 1.17.x yet. We have to wait until the xDS v3 support is added in a follow-up PR.

Fixes #8425
2021-02-22 15:00:15 -06:00
R.B. Boyer a0d26430cc
connect: if the token given to the vault provider returns no data avoid a panic (#9806)
Improves #9800
2021-02-22 14:08:49 -06:00
hashicorp-ci 02fe20ec25 auto-updated agent/uiserver/bindata_assetfs.go from commit 2c2e9d1ce 2021-02-22 14:58:06 +00:00
Daniel Nephin b1d3c8002b
Merge pull request #9683 from hashicorp/dnephin/fix-zombie-service-dereg
local: use agent token to deregister services
2021-02-19 18:56:40 -05:00
Daniel Nephin 8a5163b184 local: default to the agent token instead of the user token
When de-registering in anti-entropy sync, when there is no service or
check token.

The agent token will fall back to the default (aka user) token if no agent
token is set, so the existing behaviour still works, but it will prefer
the agent token over the user token if both are set.

ref: https://www.consul.io/docs/agent/options#acl_tokens

The agent token seems more approrpiate in this case, since this is an
"internal operation", not something initiated by the user.
2021-02-19 18:35:08 -05:00
Daniel Nephin 1f9247677b
Merge pull request #9763 from hashicorp/dnephin/cache-warn-on-error-in-notify
cache: log a warning when Cache.Notify handles an error
2021-02-19 18:30:36 -05:00
Daniel Nephin 340d714fd6 state: add a test case for memdb indexers 2021-02-19 17:14:46 -05:00
Daniel Nephin cc5c88213a state: support for functional indexers
These new functional indexers provide a few advantages:

1. enterprise differences can be isolated to a single function (the
   indexer function), making code easier to change
2. as a consequence of (1) we no longer need to wrap all the calls to
   Txn operations, making code easier to read.
3. by removing reflection we should increase the performance of all
   operations.

One important change is in making all the function signatures the same.

https://blog.golang.org/errors-are-values

An extra boolean return value for SingleIndexer.FromObject is superfluous.
The error value can indicate when the index value could not be created.
By removing this extra return value we can use the same signature for both
indexer functions.

This has the nice properly of a function being usable for both indexing operations.
2021-02-19 17:14:46 -05:00
Daniel Nephin b861642910 state: remove duplicate index on the checks table
By using a new pattern for more specific indexes. This allows us to use
the same index for both service checks and node checks. It removes the
abstraction around memdb.Txn operations, and isolates all of the
enterprise differences in a single place (the indexer).
2021-02-19 17:14:46 -05:00
Daniel Nephin 519bb82a00 state: remove duplicate function
catalogChecksForNodeService was a duplicate of catalogListServiceChecks
2021-02-19 17:14:46 -05:00
R.B. Boyer 39effd620c
xds: only try to create an ipv6 expose checks listener if ipv6 is supported by the kernel (#9765)
Fixes #9311

This only fails if the kernel has ipv6 hard-disabled. It is not sufficient to merely not provide an ipv6 address for a network interface.
2021-02-19 14:38:43 -06:00
hashicorp-ci 21713f1b73 auto-updated agent/uiserver/bindata_assetfs.go from commit 8263879e6 2021-02-19 16:48:03 +00:00
hashicorp-ci 087247967c auto-updated agent/uiserver/bindata_assetfs.go from commit 7be21a902 2021-02-18 18:37:38 +00:00
hashicorp-ci c309d2b6cb auto-updated agent/uiserver/bindata_assetfs.go from commit 1507dd8ab 2021-02-17 19:01:18 +00:00
hashicorp-ci 61fd873f24 auto-updated agent/uiserver/bindata_assetfs.go from commit 1b5087f6e 2021-02-17 18:56:34 +00:00
Mark Anderson b9d22f48cd
Add fields to the /acl/auth-methods endpoint. (#9741)
* A GET of the /acl/auth-method/:name endpoint returns the fields
MaxTokenTTL and TokenLocality, while a LIST (/acl/auth-methods) does
not.

The list command returns a filtered subset of the full set. This is
somewhat deliberate, so that secrets aren't shown, but the TTL and
Locality fields aren't (IMO) security critical, and it is useful for
the front end to be able to show them.

For consistency these changes mirror the 'omit empty' and string
representation choices made for the GET call.

This includes changes to the gRPC and API code in the client.

The new output looks similar to this
curl 'http://localhost:8500/v1/acl/auth-methods' | jq '.'

  {
    "MaxTokenTTL": "8m20s",
    "Name": "minikube-ttl-local2",
    "Type": "kubernetes",
    "Description": "minikube auth method",
    "TokenLocality": "local",
    "CreateIndex": 530,
    "ModifyIndex": 530,
    "Namespace": "default"
  }
]

Signed-off-by: Mark Anderson <manderson@hashicorp.com>

* Add changelog

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-02-17 08:16:57 -08:00
Daniel Nephin d5448c004d
Merge pull request #9452 from hashicorp/dnephin/config-tests-flags-and-edgecases
config: make TestBuilder_BuildAndValidate_ConfigFlagsAndEdgecases easier to work with
2021-02-16 16:43:49 -05:00