Commit Graph

3329 Commits

Author SHA1 Message Date
Daniel Nephin d81f527be8
Merge pull request #9924 from hashicorp/dnephin/cert-expiration-metric
connect: emit a metric for the seconds until root CA expiry
2021-06-18 14:18:55 -04:00
Daniel Nephin 19d3eeff3c
Merge pull request #9489 from hashicorp/dnephin/proxycfg-state-2
proxycfg: split state into a handler for each kind
2021-06-18 13:57:28 -04:00
Daniel Nephin 345e979b4c
Merge pull request #10425 from hashicorp/dnephin/tls-cert-exploration
tlsutil: fix a possible panic, and make the package safer
2021-06-18 13:54:07 -04:00
Daniel Nephin 0a14a3e17c inline assignment 2021-06-17 15:43:04 -04:00
Nitya Dhanushkodi 52043830b4 proxycfg: reference to entry in map should not panic 2021-06-17 11:49:04 -07:00
Daniel Nephin e738fa3b80 Replace type conversion with embedded structs 2021-06-17 13:23:35 -04:00
Daniel Nephin 32c15d9a88 proxycfg: split state into kind-specific types
This commit extracts all the kind-specific logic into handler types, and
keeps the generic parts on the state struct. This change should make it
easier to add new kinds, and see the implementation of each kind more
clearly.
2021-06-16 14:04:01 -04:00
Daniel Nephin cd05df7157 proxycfg: unmethod hostnameEndpoints
the method receiver can be replaced by the first argument.

This will allow us to extract more from the state struct in the future.
2021-06-16 14:03:30 -04:00
Daniel Nephin 97c6ee00d7 Remove duplicate import
because two PRs crossed paths.
2021-06-16 13:19:54 -04:00
Daniel Nephin 0547d0c046
Merge pull request #9466 from hashicorp/dnephin/proxycfg-state
proxycfg: prepare state for split by kind
2021-06-16 13:14:26 -04:00
R.B. Boyer 5b495ae8e0
xds: fix flaky protocol tests (#10410) 2021-06-16 11:57:43 -05:00
Freddy ae886136f1
Merge pull request #10404 from hashicorp/ingress-stats 2021-06-15 14:28:07 -06:00
R.B. Boyer 80c39f1083
xds: adding more delta protocol tests (#10398)
Fixes #10125
2021-06-15 15:21:07 -05:00
freddygv 924a5ba642 Regen golden files 2021-06-15 14:18:25 -06:00
Freddy 0a38c8fe10
Update agent/xds/listeners.go
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2021-06-15 14:09:26 -06:00
Freddy 3ee66b2e9a
Omit empty tproxy config in JSON responses (#10402) 2021-06-15 13:53:35 -06:00
Nitya Dhanushkodi b8b44419a0
proxycfg: Ensure that endpoints for explicit upstreams in other datacenters are watched in transparent mode (#10391)
Co-authored-by: Freddy Vallenilla <freddy@hashicorp.com>
2021-06-15 11:00:26 -07:00
freddygv f3e4705923 Remove unused param 2021-06-15 11:19:45 -06:00
Dhia Ayachi c8ba2d40fd
improve monitor performance (#10368)
* remove flush for each write to http response in the agent monitor endpoint

* fix race condition when we stop and start monitor multiple times, the doneCh is closed and never recover.

* start log reading goroutine before adding the sink to avoid filling the log channel before getting a chance of reading from it

* flush every 500ms to optimize log writing in the http server side.

* add changelog file

* add issue url to changelog

* fix changelog url

* Update changelog

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* use ticker to flush and avoid race condition when flushing in a different goroutine

* stop the ticker when done

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* Revert "fix race condition when we stop and start monitor multiple times, the doneCh is closed and never recover."

This reverts commit 1eeddf7a

* wait for log consumer loop to start before registering the sink

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-06-15 12:05:52 -04:00
freddygv 0aec6761dc Update ingress gateway stats labeling
In the absence of stats_tags to handle this pattern, when we pass
"ingress_upstream.$port" as the stat_prefix, Envoy splits up that prefix
and makes the port a part of the metric name.

For example:
- stat_prefix: ingress_upstream.8080

This leads to metric names like envoy_http_8080_no_route. Changing the
stat_prefix to ingress_upstream_80880 yields the expected metric names
such as envoy_http_no_route.

Note that we don't encode the destination's name/ns/dc in this
stat_prefix because for HTTP services ingress gateways use a single
filter chain. Only cluster metrics are available on a per-upstream
basis.
2021-06-15 08:52:18 -06:00
freddygv 6f8c6043b6 Update terminating gateway stats labeling
This change makes it so that the stat prefix for terminating gateways
matches that of connect proxies. By using the structure of
"upstream.svc.ns.dc" we can extract labels for the destination service,
namespace, and datacenter.
2021-06-15 08:52:18 -06:00
R.B. Boyer 848ad8535b
xds: ensure that dependent xDS resources are reconfigured during primary type warming (#10381)
Updates to a cluster will clear the associated endpoints, and updates to
a listener will clear the associated routes. Update the incremental xDS
logic to account for this implicit cleanup so that we can finish warming
the clusters and listeners.

Fixes #10379
2021-06-14 17:20:27 -05:00
Daniel Nephin aec7e798b0 Update metric name
and handle the case where there is no active root CA.
2021-06-14 17:01:16 -04:00
Daniel Nephin 1c980e4700 connect: emit a metric for the number of seconds until root CA expiration 2021-06-14 16:57:01 -04:00
Freddy ffb13f35f1
Rename CatalogDestinationsOnly (#10397)
CatalogDestinationsOnly is a passthrough that would enable dialing
addresses outside of Consul's catalog. However, when this flag is set to
true only _connect_ endpoints for services can be dialed.

This flag is being renamed to signal that non-Connect endpoints can't be
dialed by transparent proxies when the value is set to true.
2021-06-14 14:15:09 -06:00
Freddy 33bd9b5be8
Relax validation for expose.paths config (#10394)
Previously we would return an error if duplicate paths were specified.
This could lead to problems in cases where a user has the same path,
say /healthz, on two different ports.

This validation was added to signal a potential misconfiguration.
Instead we will only check for duplicate listener ports, since that is
what would lead to ambiguity issues when generating xDS config.

In the future we could look into using a single listener and creating
distinct filter chains for each path/port.
2021-06-14 14:04:11 -06:00
Daniel Nephin 016c5611d1 proxycfg: extract two types from state struct
These two new struct types will allow us to make polymorphic handler for each kind, instad of
having all the logic for each proxy kind on the state struct.
2021-06-10 17:42:17 -04:00
Daniel Nephin 9c40aa729f proxycfg: pass context around where it is needed
context.Context should never be stored on a struct (as it says in the godoc) because it is easy to
to end up with the wrong context when it is stored.

Also see https://blog.golang.org/context-and-structs

This change is also in preparation for splitting state into kind-specific handlers so that the
implementation of each kind is grouped together.
2021-06-10 17:34:50 -04:00
Daniel Nephin 3726cb52c7 http: add PrimaryDatacenter to the /v1/agent/self response
This field is available in DebugConfig, but that field is not stable and could change at any time.

The consul-k8s needs to be able to detect the primary DC for tests, so adding this field to the
stable part of the API response.
2021-06-10 17:19:16 -04:00
Freddy 429f9d8bb8
Add flag for transparent proxies to dial individual instances (#10329) 2021-06-09 14:34:17 -06:00
Daniel Nephin 3d5ff8b5db submatview: add test cases for store.Get with timeout and no index
Also set a more unique name for the serviceRequest.Type to prevent potential name conflicts
in the future.
2021-06-08 18:04:38 -04:00
Daniel Nephin cf5cdf07a0
Merge pull request #10364 from hashicorp/dnephin/streaming-e2e-test
submatview: and Store integration test with stream backend
2021-06-08 16:13:45 -04:00
Freddy 7577f0e991
Revert "Avoid adding original_dst filter when not needed" (#10365) 2021-06-08 13:18:41 -06:00
Daniel Nephin c5f0cf9456 submatview: and Store integration test with stream backend 2021-06-08 12:15:35 -04:00
Daniel Nephin eb0c0d7740 stream: remove bufferItem.NextLink
Both NextLink and NextNoBlock had the same logic, with slightly
different return values. By adding a bool return value (similar to map
lookups) we can remove the duplicate method.
2021-06-07 17:04:46 -04:00
Daniel Nephin 5ef8a045f3 stream: fix a bug with creating a snapshot
The head of the topic buffer was being ignored when creating a snapshot. This commit fixes
the bug by ensuring that the head of the topic buffer is included in the snapshot
before handing it off to the subscription.
2021-06-04 18:33:04 -04:00
Daniel Nephin 59d201e148 submatview: fix a bug with Store.Get
When info.Timeout is 0, it should have no timeout. Previously it was using a 0 duration timeout
which caused it to return without waiting.

This bug was masked by using a timeout in the tests. Removing the timeout caused the tests to fail.
2021-06-03 17:48:44 -04:00
Paul Ewing 42a51b1a2c
usagemetrics: add cluster members to metrics API (#10340)
This PR adds cluster members to the metrics API. The number of members per
segment are reported as well as the total number of members.

Tested by running a multi-node cluster locally and ensuring the numbers were
correct. Also added unit test coverage to add the new expected gauges to
existing test cases.
2021-06-03 08:25:53 -07:00
Daniel Nephin 29e93f6338 grpc: fix a data race by using a static resolver
We have seen test flakes caused by 'concurrent map read and map write', and the race detector
reports the problem as well (prevent us from running some tests with -race).

The root of the problem is the grpc expects resolvers to be registered at init time
before any requests are made, but we were using a separate resolver for each test.

This commit introduces a resolver registry. The registry is registered as the single
resolver for the consul scheme. Each test uses the Authority section of the target
(instead of the scheme) to identify the resolver that should be used for the test.
The scheme is used for lookup, which is why it can no longer be used as the unique
key.

This allows us to use a lock around the map of resolvers, preventing the data race.
2021-06-02 11:35:38 -04:00
Daniel Nephin c94eaa4957 submatview: improve a couple comments 2021-06-01 17:49:31 -04:00
Dhia Ayachi 15dddc9edb
make tests use a dummy node_name to avoid environment related failures (#10262)
* fix tests to use a dummy nodeName and not fail when hostname is not a valid nodeName

* remove conditional testing

* add test when node name is invalid
2021-06-01 11:58:03 -04:00
Daniel Nephin ba15f92a8a structs: fix cache keys
So that requests are cached properly, and the cache does not return the wrong data for a
request.
2021-05-31 17:22:16 -04:00
Daniel Nephin 920ae31598 structs: add two cache completeness tests types that implement cache.Request 2021-05-31 16:54:41 -04:00
Daniel Nephin 46dfdb611f structs: improve the interface of assertCacheInfoKeyIsComplete 2021-05-31 16:54:41 -04:00
Daniel Nephin 7c2957e24d structs: Add more cache key tests 2021-05-31 16:54:40 -04:00
Dhia Ayachi f785c5b332
RPC Timeout/Retries account for blocking requests (#8978) 2021-05-27 17:29:43 -04:00
hc-github-team-consul-core 8ab1013ed9 auto-updated agent/uiserver/bindata_assetfs.go from commit 18190fb07 2021-05-27 15:00:34 +00:00
Dhia Ayachi 4c7f5f31c7
debug: remove the CLI check for debug_enabled (#10273)
* debug: remove the CLI check for debug_enabled

The API allows collecting profiles even debug_enabled=false as long as
ACLs are enabled. Remove this check from the CLI so that users do not
need to set debug_enabled=true for no reason.

Also:
- fix the API client to return errors on non-200 status codes for debug
  endpoints
- improve the failure messages when pprof data can not be collected

Co-Authored-By: Dhia Ayachi <dhia@hashicorp.com>

* remove parallel test runs

parallel runs create a race condition that fail the debug tests

* Add changelog

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-05-27 09:41:53 -04:00
hc-github-team-consul-core c68a931e0b auto-updated agent/uiserver/bindata_assetfs.go from commit ddee7afbb 2021-05-27 12:29:12 +00:00
Freddy 353280660f
Ensure passthrough clusters can be created (#10301) 2021-05-26 15:05:14 -06:00