Commit Graph

8980 Commits

Author SHA1 Message Date
Paul Banks 74f2a80a42
Fix CA pruning when CA config uses string durations. (#4669)
* Fix CA pruning when CA config uses string durations.

The tl;dr here is:

 - Configuring LeafCertTTL with a string like "72h" is how we do it by default and should be supported
 - Most of our tests managed to escape this by defining them as time.Duration directly
 - Out actual default value is a string
 - Since this is stored in a map[string]interface{} config, when it is written to Raft it goes through a msgpack encode/decode cycle (even though it's written from server not over RPC).
 - msgpack decode leaves the string as a `[]uint8`
 - Some of our parsers required string and failed
 - So after 1 hour, a default configured server would throw an error about pruning old CAs
 - If a new CA was configured that set LeafCertTTL as a time.Duration, things might be OK after that, but if a new CA was just configured from config file, intialization would cause same issue but always fail still so would never prune the old CA.
 - Mostly this is just a janky error that got passed tests due to many levels of complicated encoding/decoding.

tl;dr of the tl;dr: Yay for type safety. Map[string]interface{} combined with msgpack always goes wrong but we somehow get bitten every time in a new way :D

We already fixed this once! The main CA config had the same problem so @kyhavlov already wrote the mapstructure DecodeHook that fixes it. It wasn't used in several places it needed to be and one of those is notw in `structs` which caused a dependency cycle so I've moved them.

This adds a whole new test thta explicitly tests the case that broke here. It also adds tests that would have failed in other places before (Consul and Vaul provider parsing functions). I'm not sure if they would ever be affected as it is now as we've not seen things broken with them but it seems better to explicitly test that and support it to not be bitten a third time!

* Typo fix

* Fix bad Uint8 usage
2018-09-13 15:43:00 +01:00
Mitchell Hashimoto 113f568509 website: document k8s go-discover (#4666)
This adds documentation for the `k8s` go-discover provider that will be part of 1.2.3.
2018-09-13 10:12:27 -04:00
Hans Hasselberg 220b17131f
Update CHANGELOG.md 2018-09-13 16:09:08 +02:00
Hans Hasselberg 8e235a72b4
Allow disabling the HTTP API again. (#4655)
If you provide an invalid HTTP configuration consul will still start again instead of failing. But if you do so the build-in proxy won't be able to start which you might need for connect.
2018-09-13 16:06:04 +02:00
John Cowen 805310e2df
Update CHANGELOG.md 2018-09-13 09:45:41 +01:00
John Cowen eb218ab93f
ui: Test bugfix. Specifically set deny for intention creation (#4663)
Make sure we speficially set and test for deny on testing intention
creation
2018-09-13 09:10:18 +01:00
John Cowen 5bb38f6ab5
ui: Tests Bugfix. Reflect extra json property `ExternalSources` in mocks (#4662)
The mocks where using randomly generated `ExternalSources` this change
makes sure they are fixed so we can reliably test the values. No change
to actual UI code
2018-09-13 09:09:59 +01:00
John Cowen 028875a732
UI: Set the CODE view as the default view for editing KV's (#4651)
Sets the code toggle on the KV edit/create page to be on by default, we figured most people probably prefer this view.

Also, previously we forced the KV toggle back to a default setting for every
time you visited a KV form page. We've now changed this so that the KV code
toggle button acts as a 'global' toggle. So whatever you set it as will
be the same for every KV for the lifetime of your 'ember session'

If we are to keep this, then consider saving this into localStorage
settings or similar, added some thoughts in comments re: this as it's very likely
to happen.
2018-09-13 09:09:30 +01:00
Rebecca Zanzig b8736e2b5c
Merge pull request #4665 from hashicorp/docs/gh-4616
Update required golang version in Readme
2018-09-12 15:19:58 -07:00
Rebecca Zanzig 183a847c7f Update required golang version in Readme
Fixes #4616.
2018-09-12 14:44:07 -07:00
Kyle Havlovitz 5c7fbc284d connect/ca: hash the consul provider ID and include isRoot 2018-09-12 13:44:15 -07:00
John Cowen f54fd21946
ui: [BUGFIX] Intentions were showing the wrong notification on creation (#4658)
The error notification was being shown on creation of an intention. This
was as a result of #4572 and/or #4572 and has not been included in a
release.

This includes a fix, plus tests to try to prevent any further regression.
2018-09-12 20:41:43 +01:00
John Cowen 0757a08684
ui: Adds a `default` view helper for providing a default value (#4650)
If the first value passed to the helper is an empty string or undefined
then return the second value
2018-09-12 20:38:57 +01:00
John Cowen 5ea748005c
UI: External Source markers (#4640)
1. Addition of external source icons for services marked as such.
2. New %with-tooltip css component (wip)
3. New 'no healthcheck' icon as external sources might not have
healthchecks, also minus icon on node cards in the service detail view
4. If a service doesn't have healthchecks, we use the [Services] tabs as the
default instead of the [Health Checks] tab in the Service detail page. 
5. `css-var` helper. The idea here is that it will eventually be
replaced with pure css custom properties instead of having to use JS. It
would be nice to be able to build the css variables into the JS at build
time (you'd probably still want to specify in config which variables you
wanted available in JS), but that's possible future work.

Lastly there is probably a tiny bit more testing edits here than usual,
I noticed that there was an area where the dynamic mocking wasn't
happening, it was just using the mocks from consul-api-double, the mocks
I was 'dynamically' setting happened to be the same as the ones in
consul-api-double. I've fixed this here also but it wasn't effecting
anything until actually made certain values dynamic.
2018-09-12 20:23:39 +01:00
John Cowen 981882d5ff
UI: Bugfix. Remove split view code editor (#4615)
When adding an auto resizing (heightwise) code editor, the
ivy-codemirror plugin seems to do this using more nested divs. This div
had a horizontal scroller but couldn't be seen on some platforms (with
hidden scrollbars). This commit makes the code editor slightly more
usable and more visually correct by removing the scroll bar in this div
to stop producing the 'split view look', yet keeping the horizontal
scroller at the bottom of the code editor for when you enter code that
is wider than the area. A max-width has also been added here to prevent
the text area from growing off the side of the page.

Another improvement to the code editor here is the addition of a nicer
color for hightlighting text selection so its at least visible.

Lastly, there was a way you could get the bottom horizontal scrollbar to overlay
the code in the editor. This makes sure there is always some space at
the bottom of the editor to make sure the code won't be obscured
2018-09-12 20:18:12 +01:00
John Cowen e5f300dd21
UI: Bugfix. Move to a different TextEncoder/Decoder (#4613)
1. The previously used TextEncoder/Decoder (used as a polyfill for
browsers that don't have a native version) didn't expose an encoder via
CommonJS. Use a different polyfill that exposes both a decoder and an
encoder.
2. The feature detection itself was flawed. This does a less error prone
detection that ensures native encoding/decoding where available and polyfilled
encoding/decoding where not available.
2018-09-12 20:15:58 +01:00
Jack Pearkes f916962afe
Update CHANGELOG.md 2018-09-12 09:57:03 -07:00
Benjamin Sago 20645c8459 Exit with error code 1 when failing to list DCs (#4583)
Fixes #4582.
2018-09-12 09:55:02 -07:00
Mitchell Hashimoto 5943c79ed4
Initial Helm Chart/K8S Docs (#4653)
* website: initial Kubernetes section with Helm information

* website: extraConfig for clients

* website: add more helm fields

* website: document extraVolumes

* website: document Consul DNS

* website: fix typos and show example of downward API
2018-09-12 08:44:30 -07:00
Pierre Souchay 1a906ef34e Fix more unstable tests in agent and command 2018-09-12 14:49:27 +01:00
Kyle Havlovitz c112a72880
connect/ca: some cleanup and reorganizing of the new methods 2018-09-11 16:43:04 -07:00
Paul Banks ea5acd3dd3
Update CHANGELOG.md 2018-09-11 17:35:59 +01:00
Pierre Souchay 2fe728c7bd Ensure that Proxies ARE always cleaned up, event with DeregisterCriticalServiceAfter (#4649)
This fixes https://github.com/hashicorp/consul/issues/4648
2018-09-11 17:34:09 +01:00
Freddy b66763eb33
Update snapshot agent docs to include s3-endpoint (#4652) 2018-09-11 16:32:31 +01:00
Matt Keeler b3ca084e82
Update CHANGELOG.md 2018-09-11 10:48:12 -04:00
Matt Keeler 3005011efd
Update CHANGELOG.md 2018-09-11 10:42:55 -04:00
Paul Banks 97b9cdbfe8
Update CHANGELOG.md 2018-09-11 15:34:24 +01:00
Matt Keeler 60f5fb1419
Update CHANGELOG.md 2018-09-11 09:40:17 -04:00
Matt Keeler d3ee66eed4
Add ECS option to EDNS responses where appropriate (#4647)
This implements parts of RFC 7871 where Consul is acting as an authoritative name server (or forwarding resolver when recursors are configured)

If ECS opt is present in the request we will mirror it back and return a response with a scope of 0 (global) or with the same prefix length as the request (indicating its valid specifically for that subnet).

We only mirror the prefix-length (non-global) for prepared queries as those could potentially use nearness checks that could be affected by the subnet. In the future we could get more sophisticated with determining the scope bits and allow for better caching of prepared queries that don’t rely on nearness checks.

The other thing this does not do is implement the part of the ECS RFC related to originating ECS headers when acting as a intermediate DNS server (forwarding resolver). That would take a quite a bit more effort and in general provide very little value. Consul will currently forward the ECS headers between recursors and the clients transparently, we just don't originate them for non-ECS clients to get potentially more accurate "location aware" results.
2018-09-11 09:37:46 -04:00
Mitchell Hashimoto e3cabb3a26
Merge pull request #4646 from hashicorp/b-fix-build
Detect correct GOOS/ARCH for copying binary with build_consul_local
2018-09-10 13:42:20 -07:00
Mitchell Hashimoto e7dfb8a0d2
Detect correct GOOS/ARCH for copying binary with build_consul_local
If GOOS/ARCH is set to something custom, we need to unset it before
testing the value so we can compare to the _original value_.
2018-09-10 10:25:15 -07:00
Pierre Souchay 22500f242e Fix unstable tests in agent, api, and command/watch 2018-09-10 16:58:53 +01:00
Freddy b2032b05be
Add script and makefile goal to help debug flaky tests 2018-09-10 16:44:07 +01:00
Mitchell Hashimoto 49b165965d
Merge pull request #4642 from hashicorp/f-ui-meta
agent: aggregate service instance meta for UI purposes
2018-09-07 17:36:23 -07:00
Mitchell Hashimoto b95348c4b1
agent: ExternalSources instead of Meta 2018-09-07 10:06:55 -07:00
Matt Keeler f07406c6fd
Update CHANGELOG.md 2018-09-07 10:59:15 -04:00
Matt Keeler 89ba649252
Connect: Verify the leaf cert to determine its readiness. (#4540)
This improves the checking so that if a certificate were to expire or the roots changed then we will go into a non-ready state.

This parses the x509 certificates from the TLS certificate when the leaf is set. The readyCh will be closed whenever a parseable certificate is set and the ca roots are set. This does not mean that the certificate is valid but that it has been setup and is generally valid. The Ready function will now do x509 certificate verification which will in addition to verifying the signatures with the installed CA roots will also verify the certificate isn't expired or not set to become valid in the future. 

The correct way to use these functions is to wait for the ReadyWait chan to be closed and then periodically check the readiness to determine if the certificate is currently useable.
2018-09-07 10:58:06 -04:00
Matt Keeler 647abbe4e5
Update CHANGELOG.md 2018-09-07 10:49:54 -04:00
Matt Keeler cc8327ed9a
Ensure that errors setting up the DNS servers get propagated back to the shell (#4598)
Fixes: #4578 

Prior to this fix if there was an error binding to ports for the DNS servers the error would be swallowed by the gated log writer and never output. This fix propagates the DNS server errors back to the shell with a multierror.
2018-09-07 10:48:29 -04:00
Paul Banks 6adaf57a3d
Update CHANGELOG.md 2018-09-07 15:32:42 +01:00
Pierre Souchay eddcf228ea Implementation of Weights Data structures (#4468)
* Implementation of Weights Data structures

Adding this datastructure will allow us to resolve the
issues #1088 and #4198

This new structure defaults to values:
```
   { Passing: 1, Warning: 0 }
```

Which means, use weight of 0 for a Service in Warning State
while use Weight 1 for a Healthy Service.
Thus it remains compatible with previous Consul versions.

* Implemented weights for DNS SRV Records

* DNS properly support agents with weight support while server does not (backwards compatibility)

* Use Warning value of Weights of 1 by default

When using DNS interface with only_passing = false, all nodes
with non-Critical healthcheck used to have a weight value of 1.
While having weight.Warning = 0 as default value, this is probably
a bad idea as it breaks ascending compatibility.

Thus, we put a default value of 1 to be consistent with existing behaviour.

* Added documentation for new weight field in service description

* Better documentation about weights as suggested by @banks

* Return weight = 1 for unknown Check states as suggested by @banks

* Fixed typo (of -> or) in error message as requested by @mkeeler

* Fixed unstable unit test TestRetryJoin

* Fixed unstable tests

* Fixed wrong Fatalf format in `testrpc/wait.go`

* Added notes regarding DNS SRV lookup limitations regarding number of instances

* Documentation fixes and clarification regarding SRV records with weights as requested by @banks

* Rephrase docs
2018-09-07 15:30:47 +01:00
Kyle Havlovitz 546bdf8663
connect/ca: add Configure/GenerateRoot to provider interface 2018-09-06 19:18:59 -07:00
Mitchell Hashimoto e9ea190df0
agent: aggregate service instance meta for UI purposes 2018-09-06 12:19:05 -07:00
Matt Keeler da931445d3
Update CHANGELOG.md 2018-09-06 15:08:51 -04:00
Matt Keeler d1e52e5292
Update Raft Vendoring (#4539)
Pulls in a fix for a potential memory leak regarding consistent reads that invoke VerifyLeader.
2018-09-06 15:07:42 -04:00
Mitchell Hashimoto 9b96b4baea
Update CHANGELOG 2018-09-06 12:02:38 -07:00
Mitchell Hashimoto 12e58ddbf8
Merge pull request #4635 from hashicorp/f-discover-k8s
Update go-discover, add k8s auto-join
2018-09-06 12:01:14 -07:00
Paul Banks af7c6cdfbb
Enable CI on f-envoy feature branch 2018-09-06 15:41:29 +01:00
Mitchell Hashimoto bbb13598bf
vendor k8s client lib 2018-09-05 14:59:02 -07:00
Mitchell Hashimoto 99eb154f6f
agent: configure k8s go-discover 2018-09-05 13:38:13 -07:00