Commit Graph

389 Commits

Author SHA1 Message Date
Frank Schröder 1073de49db Revert "config: return error on extra command line arguments (#3397)"
This reverts commit ce935cef55.
2017-10-23 09:58:37 +02:00
Frank Schroeder 078e50b635
fix go vet issue 2017-10-23 08:13:52 +02:00
Frank Schroeder ce935cef55 config: return error on extra command line arguments (#3397)
The `consul agent` command was ignoring extra command line arguments
which can lead to confusion when the user has for example forgotten to
add a dash in front of an argument or is not using an `=` when setting
boolean flags to `true`. `-bootstrap true` is not the same as
`-bootstrap=true`, for example.

Since all command line flags are known and we don't expect unparsed
arguments we can return an error. However, this may make it slightly
more difficult in the future if we ever wanted to have these kinds of
arguments.

Fixes #3397
2017-10-23 08:08:09 +02:00
Frank Schroeder f2a291a470 config: rename test struct field to args 2017-10-23 08:08:09 +02:00
Frank Schroeder f6e9ad99ec config: address review comments 2017-10-23 08:06:26 +02:00
Frank Schroeder 0ed4561f12 config: document remaining config options 2017-10-23 08:06:26 +02:00
Frank Schroeder 9864609201 config: document more config options 2017-10-23 08:06:26 +02:00
Frank Schroeder bf81cdea6e config: document more config options 2017-10-23 08:06:26 +02:00
Frank Schroeder 2682ce0b82 config: document more acl options 2017-10-23 08:06:26 +02:00
Frank Schroeder a38c69d784 config: document config options 2017-10-23 08:06:26 +02:00
Frank Schroeder 7396bd31fd config: document acl options 2017-10-23 08:06:26 +02:00
Frank Schroeder 7685ef409c config: document autopilot options 2017-10-23 08:06:26 +02:00
Frank Schroeder 5e57e9273e config: document dns options 2017-10-23 08:06:26 +02:00
Frank Schroeder 5f59857448 config: document http options 2017-10-23 08:06:26 +02:00
Frank Schroeder 7fede4472d config: document telemetry options 2017-10-23 08:06:26 +02:00
Frank Schroeder 35f4acdddc local state: remove stale comment 2017-10-23 08:03:18 +02:00
Frank Schroeder f9267380db local state: make test more robust 2017-10-23 08:03:18 +02:00
Frank Schroeder af1243c725 local state: clone check to avoid side effect 2017-10-23 08:03:18 +02:00
Frank Schroeder 39a2d8d25e local state: use synchronized access to internal maps 2017-10-23 08:03:18 +02:00
Frank Schroeder bd00814301 ae: do not trigger on Resume while holding the lock 2017-10-23 08:03:18 +02:00
Frank Schroeder c32915bb4f ae: add remaining test cases 2017-10-23 08:03:18 +02:00
Frank Schroeder 8a45365f68 ae: refactor StateSyncer to state machine for better testing 2017-10-23 08:03:18 +02:00
Frank Schroeder 8158cec829 ae: add test that we run a full before a partial sync 2017-10-23 08:03:18 +02:00
Frank Schroeder b9a8b53d52 ae: make control flow more explicit 2017-10-23 08:03:18 +02:00
Frank Schroeder e88f49e2cc ae: fix typo in constructor name 2017-10-23 08:03:18 +02:00
Frank Schroeder cd0262744d ae: add test for resume triggering SyncChanges 2017-10-23 08:03:18 +02:00
Frank Schroeder f5177ef332 ae: add test for ifNotPausedRun 2017-10-23 08:03:18 +02:00
Frank Schroeder 066ad01c38 ae: make stagger function pluggable for testing 2017-10-23 08:03:18 +02:00
Frank Schroeder 126046be23 ae: restore previous pause/resume behavior 2017-10-23 08:03:18 +02:00
Frank Schroeder ffb265dd93 ae: ensure that syncs are blocked when paused 2017-10-23 08:03:18 +02:00
Frank Schroeder 9280841a80 local state: rename Add{Check,Service}State to Set{Check,Service}State 2017-10-23 08:03:18 +02:00
Frank Schroeder 9bc8127728 local state: move Metadata methods together 2017-10-23 08:03:18 +02:00
Frank Schroeder e86521e637 local state: update documentation of updateSyncState 2017-10-23 08:03:18 +02:00
Frank Schroeder 42188164f8 local state: update comments 2017-10-23 08:03:18 +02:00
Frank Schroeder 1d315075b1 local state: address review comments
* move non-blocking notification mechanism into ae.Trigger
* move Pause/Resume into separate type
2017-10-23 08:03:18 +02:00
Frank Schroeder c28e23eac8 local state: refactor TestAgentAntiEntropy_EnableTagOverride
Make intent clearer by being more explicit and adding some comments.
Use verify.Values to compare service entries.
2017-10-23 08:03:18 +02:00
Frank Schroeder 86f7ea6013 local state: fix TestAgentAntiEntropy_EnableTagOverride
The test had a race condition where it relied on the first service to be
synced to the remote catalog which sometimes failed.
2017-10-23 08:03:18 +02:00
Frank Schroeder ff62eaf063 local state: rename tests 2017-10-23 08:03:18 +02:00
Frank Schroeder 2bdba8ab06 local state: drop retry loops from tests
Since the tests are now using synchronous calls for state syncing
we no longer need to use retry loops to wait for the changes to
propagate.
2017-10-23 08:03:18 +02:00
Frank Schroeder ce1f35373e agent: skip non-sensical TestCatalogRegister
It is not clear what this test is supposed to verify.
2017-10-23 08:03:18 +02:00
Frank Schroeder f8e20cd996 local state: fix anti-entropy state tests
The anti-entropy tests relied on the side-effect of the StartSync()
method to perform a full sync instead of a partial sync. This lead to
multiple anti-entropy go routines being started unnecessary retry loops.

This change changes the behavior to perform synchronous full syncs when
necessary removing the need for all of the time.Sleep and most of the
retry loops.
2017-10-23 08:03:18 +02:00
Frank Schroeder e9149f64d9 local state: fix test with updated error message 2017-10-23 08:03:18 +02:00
Frank Schroeder 76682da4a0 local state: fix failing tests 2017-10-23 08:03:18 +02:00
Frank Schroeder 1af52bf7be local state: tests compile 2017-10-23 08:03:18 +02:00
Frank Schroeder ccbae7da5b local state: replace multi-map state with structs
The state of the service and health check records was spread out over
multiple maps guarded by a single lock. Access to the maps has to happen
in a coordinated effort and the tests often violated this which made
them brittle and racy.

This patch replaces the multiple maps with a single one for both checks
and services to make the code less fragile.

This is also necessary since moving the local state into its own package
creates circular dependencies for the tests. To avoid this the tests can
no longer access internal data structures which they should not be doing
in the first place.

The tests still don't compile but this is a ncessary step in that
direction.
2017-10-23 08:03:18 +02:00
Frank Schroeder d447e823c6 local state: move to separate package
This patch moves the local state to a separate package to further
decouple it from the agent code.

The code compiles but the tests do not yet.
2017-10-23 08:03:18 +02:00
Frank Schroeder b5dbad910c agent: simplify some loops 2017-10-23 08:03:18 +02:00
Frank Schroeder 7a2af206ea agent: refactor sync loop to linear flow of control 2017-10-23 08:03:18 +02:00
Frank Schroeder b7136e100b agent: cleanup StateSyncer
This patch cleans up the state syncer code by renaming fields, adding
helpers and documentation.
2017-10-23 08:03:18 +02:00
Frank Schroeder a842dc9c2b agent: decouple anti-entropy from local state
The anti-entropy code manages background synchronizations of the local
state on a regular basis or on demand when either the state has changed
or a new consul server has been added.

This patch moves the anti-entropy code into its own package and
decouples it from the local state code since they are performing
two different functions.

To simplify code-review this revision does not make any optimizations,
renames or refactorings. This will happen in subsequent commits.
2017-10-23 08:03:18 +02:00
Frank Schroeder 1db8d3cb00 config: do not allow an ANY address as DNS recursor 2017-10-23 08:01:25 +02:00
Frank Schroeder 72bee6284d config: add support for go-sockaddr templates for DNS recursors
DNS recursors can be added through go-sockaddr templates. Entries
are deduplicated while the order is maintained.

Originally proposed by @taylorchu

See #2932
2017-10-23 08:01:25 +02:00
James Phillips 23d4b24792
Cleans up import sorting. 2017-10-21 20:08:11 -07:00
Hadar Greinsmark 7e1a860978 Implement HTTP Watch handler (#3413)
Implement HTTP Watch handler
2017-10-21 20:39:09 -05:00
Frank Schroeder c94751ad43 test: replace porter tool with freeport lib
This patch removes the porter tool which hands out free ports from a
given range with a library which does the same thing. The challenge for
acquiring free ports in concurrent go test runs is that go packages are
tested concurrently and run in separate processes. There has to be some
inter-process synchronization in preventing processes allocating the
same ports.

freeport allocates blocks of ports from a range expected to be not in
heavy use and implements a system-wide mutex by binding to the first
port of that block for the lifetime of the application. Ports are then
provided sequentially from that block and are tested on localhost before
being returned as available.
2017-10-21 22:01:09 +02:00
Frank Schröder 350932161d dns: return NXDOMAIN if datacenter is invalid (#3200) (#3596)
Queries to the DNS server can contain an optional datacenter
name in the query name. You can query for 'foo.service.consul'
or 'foo.service.dc.consul' to get a response for either the
default or a specific datacenter.

Datacenter names cannot have dots, therefore the datacenter
name can refer to only one element in the DNS query name.

The DNS server allowed extra labels between the optional
datacenter name and the domain and returned a valid response
instead of returning NXDOMAIN. For example, if the domain
is set to '.consul' then 'foo.service.dc1.extra.consul'
should return NXDOMAIN because of 'extra' being between
the datacenter name 'dc1' and the domain '.consul'.

Fixes #3200
2017-10-20 16:49:17 -07:00
James Phillips 53f67c3993 Fixes API client for ScriptArgs and updates documentation. (#3589)
* Updates the API client to support the current `ScriptArgs` parameter
for checks.

* Updates docs for checks to explain the `ScriptArgs` parameter issue.

* Adds mappings for "args" and "script-args" to give th API parity
with config.

* Adds checks on return codes.

* Removes debug logging that shows empty when args are used.
2017-10-18 11:28:39 -07:00
Ryan Slade 85e4aea9d1 Replace time.Now().Sub(x) with time.Since(x) 2017-10-17 20:38:24 +02:00
James Phillips f25c66dfd9 Fixes an XSS issue with unescaped node names. (#3578)
* Fixes an XSS issue with node names in the tomography graph.

* Updates built-in static web assets.

* Updates the change log.
2017-10-16 09:12:36 -07:00
James Phillips c943b1b151 Adds a brief wait and poll period to update check status after a timeout. (#3573)
* Adds a brief wait and poll period to update the check status
if we get stucking waiting for the processes to terminate.

Fixes #3570

* Jumps out of timeout case and includes script output.
2017-10-12 13:49:46 -07:00
James Phillips 575d70aaa7
Cleans up some drift between the OSS and Enterprise trees. 2017-10-11 15:53:07 -07:00
Kyle Havlovitz 106b8b0b33 Kill check processes after the timeout is reached (#3567)
* Kill check processes after the timeout is reached

Kill the subprocess spawned by a script check once the timeout is reached. Previously Consul just marked the check critical and left the subprocess around.

Fixes #3565.

* Set err to non-nil when timeout occurs

* Fix check timeout test

* Kill entire process subtree on check timeout

* Add a docs note about windows subprocess termination
2017-10-11 11:57:39 -07:00
Frank Schroeder 6780964eb8
config: remove redundant code 2017-10-11 10:16:21 +02:00
Frank Schroeder 0faff32c73
config: fix check for segment.port <= 0 and add test 2017-10-11 10:15:55 +02:00
James Phillips 16eb2ef014
Adds check to make sure port is given so we avoid a nil bind address. 2017-10-10 18:11:21 -07:00
James Phillips e6c50f4a71
Removes obsolete segment stub. 2017-10-10 17:21:32 -07:00
Frank Schröder 94f58199b1 agent: add option to discard health output (#3562)
* agent: add option to discard health output

In high volatile environments consul will have checks with "noisy"
output which changes every time even though the status does not change.
Since the output is stored in the raft log every health check update
unblocks a blocking call on health checks since the raft index has
changed even though the status of the health checks may not have changed
at all. By discarding the output of the health checks the users can
choose a different tradeoff. Less visibility on why a check failed in
exchange for a reduced change rate on the raft log.

* agent: discard output also when adding a check

* agent: add test for discard check output

* agent: update docs

* go vet

* Adds discard_check_output to reloadable config table.

* Updates the change log.
2017-10-10 17:04:52 -07:00
preetapan 77c972f594 Fixes agent error handling when check definition is invalid. Distingu… (#3560)
* Fixes agent error handling when check definition is invalid. Distinguishes between empty checks vs invalid checks

* Made CheckTypes return Checks from service definition struct rather than a new copy, and other changes from code review. This also errors when json payload contains empty structs

* Simplify and improve validate method, and make sure that CheckTypes always returns a new copy of validated check definitions

* Tweaks some small style things and error messages.

* Updates the change log.
2017-10-10 16:54:06 -07:00
Frank Schröder 759ef8a1d4 config: add generic method to translate between CamelCase and snake_case (#3557)
* doc: document discrepancy between id and CheckID

* doc: document enable_tag_override change

* config: add TranslateKeys helper

TranslateKeys makes it easier to map between different representations
of internal structures. It allows to recursively map alias keys to
canonical keys in structured maps.

* config: use TranslateKeys for config file

This also adds support for 'enabletagoverride' and removes
the need for a separate CheckID alias field.

* config: remove dead code

* agent: use TranslateKeys for FixupCheckType

* agent: translate enable_tag_override during service registration

* doc: add '.hcl' as valid extension

* config: map ScriptArgs to args

* config: add comment for TranslateKeys
2017-10-10 16:40:59 -07:00
James Phillips bb12368eac Makes RPC handling more robust when rolling servers. (#3561)
* Adds client-side retry for no leader errors.

This paves over the case where the client was connected to the leader
when it loses leadership.

* Adds a configurable server RPC drain time and a fail-fast path for RPCs.

When a server leaves it gets removed from the Raft configuration, so it will
never know who the new leader server ends up being. Without this we'd be
doomed to wait out the RPC hold timeout and then fail. This makes things fail
a little quicker while a sever is draining, and since we added a client retry
AND since the server doing this has already shut down and left the Serf LAN,
clients should retry against some other server.

* Makes the RPC hold timeout configurable.

* Reorders struct members.

* Sets the RPC hold timeout default for test servers.

* Bumps the leave drain time up to 5 seconds.

* Robustifies retries with a simpler client-side RPC hold.

* Reverts untended delete.
2017-10-10 15:19:50 -07:00
Preetha Appan e7dc345cfa Fix unit test after dns library upgrade to account for correct data length 2017-10-06 17:40:17 -05:00
James Phillips 4dab70cb93 Fixes handling of stop channel and failed barrier attempts. (#3546)
* Fixes handling of stop channel and failed barrier attempts.

There were two issues here. First, we needed to not exit when there
was a timeout trying to write the barrier, because Raft might not
step down, so we'd be left as the leader but having run all the step
down actions.

Second, we didn't close over the stopCh correctly, so it was possible
to nil that out and have the leaderLoop never exit. We close over it
properly AND sequence the nil-ing of it AFTER the leaderLoop exits for
good measure, so the code is more robust.

Fixes #3545

* Cleans up based on code review feedback.

* Tweaks comments.

* Renames variables and removes comments.
2017-10-06 07:54:49 -07:00
Victor Boivie 8e361beb7a Minor typo (boostrap) 2017-10-05 16:28:48 +02:00
James Phillips 3bc6df5f0e
Adds script warning and fixes Docker args recognition. 2017-10-04 21:41:27 -07:00
Kyle Havlovitz adf29675f3 Merge pull request #3535 from hashicorp/metric-docs
Update metric names and add a legacy config flag
2017-10-04 17:39:16 -07:00
Kyle Havlovitz a3e9ac5840
Add a test for legacy metrics with a whitelist filter 2017-10-04 17:27:57 -07:00
Kyle Havlovitz 198ed6076d Clean up subprocess handling and make shell use optional (#3509)
* Clean up handling of subprocesses and make using a shell optional

* Update docs for subprocess changes

* Fix tests for new subprocess behavior

* More cleanup of subprocesses

* Minor adjustments and cleanup for subprocess logic

* Makes the watch handler reload test use the new path.

* Adds check tests for new args path, and updates existing tests to use new path.

* Adds support for script args in Docker checks.

* Fixes the sanitize unit test.

* Adds panic for unknown watch type, and reverts back to Run().

* Adds shell option back to consul lock command.

* Adds shell option back to consul exec command.

* Adds shell back into consul watch command.

* Refactors signal forwarding and makes Windows-friendly.

* Adds a clarifying comment.

* Changes error wording to a warning.

* Scopes signals to interrupt and kill.

This avoids us trying to send SIGCHILD to the dead process.

* Adds an error for shell=false for consul exec.

* Adds notes about the deprecated script and handler fields.

* De-nests an if statement.
2017-10-04 16:48:00 -07:00
Kyle Havlovitz c728564994
Update metric names and add a legacy config flag 2017-10-04 16:43:27 -07:00
Frank Schröder ce887a0c45 Provide stable config for agent/self (#3532)
* config: provide stable config for /v1/agent/self (#3530)

This patch adds a stable subset of the previous Config struct to the
agent/self response. The actual runtime configuration is moved into
DebugConfig and will be documented to change.

Fixes #3530

* config: fix tests

* doc: update api documentation for /v1/agent/self
2017-10-04 10:43:17 -07:00
James Phillips 4f2dccc2a9 Merge pull request #3531 from hashicorp/pr-3521-slackpad
ui: Use monospace font for textarea controls.
2017-10-04 09:53:41 -07:00
James Phillips b34d576193
Updates checked in web assets to pick up CSS change.
Closes #3521
2017-10-04 09:52:15 -07:00
Preetha Appan 8dcd7e700c Remove extra newline 2017-10-03 15:19:31 -05:00
Preetha Appan 26accb3b8a Only allow 'list' policies within 'key' policy definitions. Consolidated two similar tests into one and fixed alignment. 2017-10-03 15:15:56 -05:00
Preetha Appan 51a04ec87d Introduces new 'list' permission that applies to KV store recursive reads, and enforced only when opted in. 2017-10-02 17:10:21 -05:00
Frank Schroeder 1944218492 use ports from derived addresses 2017-09-29 20:26:43 +02:00
Frank Schroeder 42f8ff7b3c config: drop advertise_addrs
Fixes #3516
2017-09-29 20:26:43 +02:00
Frank Schroeder abe41d231c
Fix tests after config refactor 2017-09-28 12:32:46 +02:00
Patrick Sodré 7501331d13
Implement encodeKVasRFC1464 function 2017-09-28 12:32:46 +02:00
Patrick Sodré 2cc6ac542c
Add RFC1464 tests 2017-09-28 12:32:45 +02:00
Patrick Sodré 865f087ec9
Turn encodeKVasRFC1464 into a plain function 2017-09-28 12:32:45 +02:00
Patrick Sodré 12fb0bfd5b
Use verify for NodeLookup CNAME, and TXT tests 2017-09-28 12:32:45 +02:00
Patrick Sodré d5e3b9d843
Refactor formatTxtRecords as encodeKVasRFC1464
- Move the logic of rfc1035 out of the encoding function
  - Left basic version of encodingKV as 'k=v'
2017-09-28 12:32:45 +02:00
Patrick Sodré 655c89ee10
Fix editorial suggestions 2017-09-28 12:32:45 +02:00
Patrick Sodré afb0c92334
Remove redundant check of Node.Meta size 2017-09-28 12:32:45 +02:00
Patrick Sodré 53e812e759
Return Node.Meta info using the DNS interface 2017-09-28 12:32:45 +02:00
Patrick Sodré ab90865865
Add test for NoteLookup ANY request 2017-09-28 12:32:45 +02:00
Patrick Sodré 4c6b8022c2
Add test for querying Node.Meta with DNS TXT
- Lookup TXT records using recursive lookups
  - Expect TXT record equal to value if key starts with rfc1035-
  - Expect TXT record in rfc1464 otherwise, i.e. (k=v)

ref #2709
2017-09-28 12:32:45 +02:00
Frank Schröder 07dea89f31 fail early when advertise addr is set to ANY (#3507) 2017-09-27 13:57:55 -07:00
Frank Schröder 9a67556bb5 only detect advertise address if derived value is any (#3506)
* only detect advertise address if derived value is any

* determine detect function only when advertise addr is any
2017-09-27 12:59:47 -07:00
James Phillips 98850322c0
Adds a comment about Datacenter and NodeName being stable interfaces
in the runtime config strucutre.
2017-09-27 11:59:22 -07:00