Commit Graph

1649 Commits

Author SHA1 Message Date
R.B. Boyer 20eb0d3e94 cache: remove data race in agent cache
In normal operations there is a read/write race related to request
QueryOptions fields. An example race:

    WARNING: DATA RACE
    Read at 0x00c000836950 by goroutine 30:
      github.com/hashicorp/consul/agent/structs.(*ServiceConfigRequest).CacheInfo()
          /go/src/github.com/hashicorp/consul/agent/structs/config_entry.go:506 +0x109
      github.com/hashicorp/consul/agent/cache.(*Cache).getWithIndex()
          /go/src/github.com/hashicorp/consul/agent/cache/cache.go:262 +0x5c
      github.com/hashicorp/consul/agent/cache.(*Cache).notifyBlockingQuery()
          /go/src/github.com/hashicorp/consul/agent/cache/watch.go:89 +0xd7

    Previous write at 0x00c000836950 by goroutine 147:
      github.com/hashicorp/consul/agent/cache-types.(*ResolvedServiceConfig).Fetch()
          /go/src/github.com/hashicorp/consul/agent/cache-types/resolved_service_config.go:31 +0x219
      github.com/hashicorp/consul/agent/cache.(*Cache).fetch.func1()
          /go/src/github.com/hashicorp/consul/agent/cache/cache.go:495 +0x112

This patch does a lightweight copy of the request struct so that the
embedded QueryOptions fields that are mutated during Fetch() are scoped
to just that one RPC.
2019-09-12 16:18:01 -05:00
hashicorp-ci 4f499a8044
update bindata_assetfs.go 2019-09-12 19:39:58 +00:00
Hans Hasselberg 4a20efda9b
agent: handleEnterpriseLeave (#6453) 2019-09-11 11:01:37 +02:00
R.B. Boyer a86e63f81e
test: actually wait for the TestAgent to be fully shutdown (#6441) 2019-09-05 13:36:26 -05:00
Sarah Adams 001137e5e5
test: ensure all TestAgent constructions use a constructor (#6443)
ensure all TestAgent constructions use a constructor to get start retries + test logs going to the right place

Fixes #6435
2019-09-05 10:24:36 -07:00
Sarah Adams 74461406e0
remove funky panic/recover in agent tests (#6442) 2019-09-04 13:59:11 -07:00
Sarah Adams 4ed5515fca
refactor & add better retry logic to NewTestAgent (#6363)
Fixes #6361
2019-09-03 15:05:51 -07:00
Pierre Souchay be50400c62 Distinguish between DC not existing and not being available (#6399) 2019-09-03 09:46:24 -06:00
Aestek 7c7b7f24fd Add option to register services and their checks idempotently (#4905) 2019-09-02 09:38:29 -06:00
Sarah Adams 9f4b329b6d
txn: don't try to decode request bodies > raft.SuggestedMaxDataSize (#6422)
txn: don't try to decode request bodies > raft.SuggestedMaxDataSize
2019-08-30 10:41:25 -07:00
Matt Keeler 42d608587f
Store primaries root in secondary after intermediate signature (#6333)
* Store primaries root in secondary after intermediate signature

This ensures that the intermediate exists within the CA root stored in raft and not just in the CA provider state. This has the very nice benefit of actually outputting the intermediate cert within the ca roots HTTP/RPC endpoints.

This change means that if signing the intermediate fails it will not set the root within raft. So far I have not come up with a reason why that is bad. The secondary CA roots watch will pull the root again and go through all the motions. So as soon as getting an intermediate CA works the root will get set.

* Make TestAgentAntiEntropy_Check_DeferSync less flaky

I am not sure this is the full fix but it seems to help for me.
2019-08-30 11:38:46 -04:00
R.B. Boyer 7deaba63e1
test: ensure the node name is a valid dns name (#6424)
The space in the node name was making every test emit a useless warning.
2019-08-29 16:52:13 -05:00
R.B. Boyer 3be88df207
test: explicitly run the pprof tests for 1s instead of the 30s default (#6421) 2019-08-29 12:06:50 -05:00
R.B. Boyer d979d8c239
test: add additional http status code assertions in coordinate HTTP API tests (#6410)
When this test flakes sometimes this happens:

    --- FAIL: TestCoordinate_Node (1.69s)
    panic: interface conversion: interface {} is nil, not structs.Coordinates [recovered]
    FAIL    github.com/hashicorp/consul/agent       19.999s
    Exit code: 1
    panic: interface conversion: interface {} is nil, not structs.Coordinates [recovered]
            panic: interface conversion: interface {} is nil, not structs.Coordinates

There is definitely a bug lurking, but the code seems to imply this can
only return nil on 404. The tests previously were not checking the
status code.

The underlying cause of the flake is unknown, but this should turn the
failure into a more normal test failure.
2019-08-29 09:55:05 -05:00
Pierre Souchay 58f04815d5 Display IPs of machines when node names conflict to ease troubleshooting
When there is an node name conflicts, such messages are displayed within Consul:

`consul.fsm: EnsureRegistration failed: failed inserting node: Error while renaming Node ID: "e1d456bc-f72d-98e5-ebb3-26ae80d785cf": Node name node001 is reserved by node 05f10209-1b9c-b90c-e3e2-059e64556d4a with name node001`

While it is easy to find the node that has reserved the name, it is hard to find
the node trying to aquire the name since it is not registered, because it
is not part of `consul members` output

This PR will display the IP of the offender and solve far more easily those issues.
2019-08-28 15:57:05 -04:00
Alvin Huang c516fabfac
revert commits on master (#6413) 2019-08-27 17:45:58 -04:00
tradel 5a22b77340 update tests to match new method signatures 2019-08-27 14:16:39 -07:00
tradel 1ff46f3f0a confi\gure providers with DC and domain 2019-08-27 14:16:25 -07:00
tradel 5ba28a6a7b create a common name for autoTLS agent certs 2019-08-27 14:15:53 -07:00
tradel 9b1ac4e7ef add subject names to issued certs 2019-08-27 14:15:10 -07:00
tradel 7f36a5b676 construct a common name for each CSR 2019-08-27 14:12:56 -07:00
tradel 672e181399 add serviceID to leaf cert request 2019-08-27 14:12:22 -07:00
tradel a4312d2e6e add domain and nodeName to agent cert request 2019-08-27 14:11:40 -07:00
tradel 82ae7caf3e Added DC and domain args to Configure method 2019-08-27 14:09:01 -07:00
R.B. Boyer b962fe38cd
test: send testagent logs through testing.Logf (#6411) 2019-08-27 12:21:30 -05:00
R.B. Boyer 91da908d2f
test: fix TestAgent.Start() to not segfault if the DNSServer cannot ListenAndServe (#6409)
The embedded `Server` field on a `DNSServer` is only set inside of the
`ListenAndServe` method. If that method fails for reasons like the
address being in use and is not bindable, then the `Server` field will
not be set and the overall `Agent.Start()` will fail.

This will trigger the inner loop of `TestAgent.Start()` to invoke
`ShutdownEndpoints` which will attempt to pretty print the DNS servers
using fields on that inner `Server` field. Because it was never set,
this causes a nil pointer dereference and crashes the test.
2019-08-27 10:45:05 -05:00
Alvin Huang 0be1531d80
add nil pointer check for pointer to ACLToken struct (#6407) 2019-08-27 11:23:28 -04:00
Hans Hasselberg d051342902
make sure auto_encrypt has private key type and bits (#6392) 2019-08-27 14:37:56 +02:00
Hans Hasselberg faa54ab989
auto_encrypt: verify_incoming_rpc is good enough for auto_encrypt.allow_tls (#6376)
Previously `verify_incoming` was required when turning on `auto_encrypt.allow_tls`, but that doesn't work together with HTTPS UI in some scenarios. Adding `verify_incoming_rpc` to the allowed configurations.
2019-08-27 14:36:36 +02:00
R.B. Boyer 7bc941575c
test: don't leak agent goroutines in TestAgent_sidecarServiceFromNodeService (#6396)
A goroutine dump using runtime.Stack() before/after shows a drop from 121 => 4.
2019-08-26 15:19:59 -05:00
Hans Hasselberg f3def8c0d0 make sure auto_encrypt has private key type and bits 2019-08-26 13:09:50 +02:00
hashicorp-ci 3757cb6a96
update bindata_assetfs.go 2019-08-23 22:10:50 +00:00
R.B. Boyer cc9a6f7993
Merge pull request #6388 from hashicorp/release/1-6
merging release/1-6 into master
2019-08-23 13:44:46 -05:00
Matt Keeler cbd1857186
Secondary CA `establishLeadership` fix (#6383)
This prevents ACL issues (or other issues) during intermediate CA cert signing from failing leader establishment.
2019-08-23 11:32:37 -04:00
Hans Hasselberg 3e46352ccb
auto_encrypt: use server-port (#6287)
AutoEncrypt needs the server-port because it wants to talk via RPC. Information from gossip might not be available at that point and thats why the server-port is being used.
2019-08-23 10:18:46 +02:00
R.B. Boyer dfcdc41ef8
connect: allow 'envoy_cluster_json' escape hatch to continue to function (#6378) 2019-08-22 15:11:56 -05:00
R.B. Boyer 7a6faccf2f
docs: document how envoy escape hatches work with the discovery chain (#6350)
- Bootstrap escape hatches are OK.
- Public listener/cluster escape hatches are OK.
- Upstream listener/cluster escape hatches are not supported.

If an unsupported escape hatch is configured and the discovery chain is
activated log a warning and act like it was not configured.

Fixes #6160
2019-08-21 15:10:12 -05:00
Matt Keeler 0c3cf47266
Ensure that config entry writes are forwarded to the primary DC (#6339) 2019-08-20 12:01:13 -04:00
R.B. Boyer fd1c62ee8b
connect: ensure time.Duration fields retain their human readable forms in the API (#6348)
This applies for both config entries and the compiled discovery chain.

Also omit some other config entries fields when empty.
2019-08-19 15:31:05 -05:00
R.B. Boyer 561b2fe606
connect: generate the full SNI names for discovery targets in the compiler rather than in the xds package (#6340) 2019-08-19 13:03:03 -05:00
R.B. Boyer ae79cdab1b
connect: introduce ExternalSNI field on service-defaults (#6324)
Compiling this will set an optional SNI field on each DiscoveryTarget.
When set this value should be used for TLS connections to the instances
of the target. If not set the default should be used.

Setting ExternalSNI will disable mesh gateway use for that target. It also 
disables several service-resolver features that do not make sense for an 
external service.
2019-08-19 12:19:44 -05:00
R.B. Boyer 1a485011d0
connect: updating a service-defaults config entry should leave an unset protocol alone (#6342)
If the entry is updated for reasons other than protocol it is surprising
that the value is explicitly persisted as 'tcp' rather than leaving it
empty and letting it fall back dynamically on the proxy-defaults value.
2019-08-19 10:44:06 -05:00
Matt Keeler 318b9ebbe3
Filter out left/leaving serf members when determining if new AC… (#6332) 2019-08-16 10:34:18 -04:00
R.B. Boyer 72207256b9
xds: improve how envoy metrics are emitted (#6312)
Since generated envoy clusters all are named using (mostly) SNI syntax
we can have envoy read the various fields out of that structure and emit
it as stats labels to the various telemetry backends.

I changed the delimiter for the 'customization hash' from ':' to '~'
because ':' is always reencoded by envoy as '_' when generating metrics
keys.
2019-08-16 09:30:17 -05:00
R.B. Boyer 3975cb89bf
agent: blocking central config RPCs iterations should not interfere with each other (#6316) 2019-08-14 09:08:46 -05:00
hashicorp-ci d7bb2bbee4
update bindata_assetfs.go 2019-08-13 15:28:06 +00:00
hashicorp-ci 5919c7c184 Merge Consul OSS branch 'master' at commit 8f7586b339 2019-08-13 02:00:43 +00:00
Sarah Adams 8ff1f481fe
add flag to allow /operator/keyring requests to only hit local servers (#6279)
Add parameter local-only to operator keyring list requests to force queries to only hit local servers (no WAN traffic).

HTTP API: GET /operator/keyring?local-only=true
CLI: consul keyring -list --local-only

Sending the local-only flag with any non-GET/list request will result in an error.
2019-08-12 11:11:11 -07:00
Mike Morris 61206fdf42
snapshot: add TLS support to HalfCloser interface (#6216)
Calls net.TCPConn.CloseWrite or mtls.Conn.CloseWrite, which was added in https://go-review.googlesource.com/c/go/+/31318/
2019-08-12 12:47:02 -04:00
Matt Keeler 6a1e0dfed8
Update the v1/agent/service/:service endpoint to output tagged… (#6304) 2019-08-10 09:15:19 -04:00