Commit Graph

53 Commits

Author SHA1 Message Date
Daniel Nephin 51efba2c7d testutil: NewLogBuffer - buffer logs until a test fails
Replaces #7559

Running tests in parallel, with background goroutines, results in test output not being associated with the correct test. `go test` does not make any guarantees about output from goroutines being attributed to the correct test case.

Attaching log output from background goroutines also cause data races.  If the goroutine outlives the test, it will race with the test being marked done. Previously this was noticed as a panic when logging, but with the race detector enabled it is shown as a data race.

The previous solution did not address the problem of correct test attribution because test output could still be hidden when it was associated with a test that did not fail. You would have to look at all of the log output to find the relevant lines. It also made debugging test failures more difficult because each log line was very long.

This commit attempts a new approach. Instead of printing all the logs, only print when a test fails. This should work well when there are a small number of failures, but may not work well when there are many test failures at the same time. In those cases the failures are unlikely a result of a specific test, and the log output is likely less useful.

All of the logs are printed from the test goroutine, so they should be associated with the correct test.

Also removes some test helpers that were not used, or only had a single caller. Packages which expose many functions with similar names can be difficult to use correctly.

Related:
https://github.com/golang/go/issues/38458 (may be fixed in go1.15)
https://github.com/golang/go/issues/38382#issuecomment-612940030
2020-07-21 12:50:40 -04:00
Hans Hasselberg 6a18f01b42
agent: ensure node info sync and full sync. (#7189)
This fixes #7020.

There are two problems this PR solves:
  * if the node info changes it is highly likely to get service and check registration permission errors unless those service tokens have node:write. Hopefully services you register don’t have this permission.
  * the timer for a full sync gets reset for every partial sync which means that many partial syncs are preventing a full sync from happening

Instead of syncing node info last, after services and checks, and possibly saving one RPC because it is included in every service sync, I am syncing node info first. It is only ever going to be a single RPC that we are only doing when node info has changed. This way we are guaranteed to sync node info even when something goes wrong with services or checks which is more likely because there are more syncs happening for them.
2020-02-06 15:30:58 +01:00
Chris Piraino 401221de58
Allow users to configure either unstructured or JSON logging (#7130)
* hclog Allow users to choose between unstructured and JSON logging
2020-01-28 17:50:41 -06:00
Paul Banks e812f5516a Add -sidecar-for and new /agent/service/:service_id endpoint (#4691)
- A new endpoint `/v1/agent/service/:service_id` which is a generic way to look up the service for a single instance. The primary value here is that it:
   - **supports hash-based blocking** and so;
   - **replaces `/agent/connect/proxy/:proxy_id`** as the mechanism the built-in proxy uses to read its config.
   - It's not proxy specific and so works for any service.
   - It has a temporary shim to call through to the existing endpoint to preserve current managed proxy config defaulting behaviour until that is removed entirely (tested).
 - The built-in proxy now uses the new endpoint exclusively for it's config
 - The built-in proxy now has a `-sidecar-for` flag that allows the service ID of the _target_ service to be specified, on the condition that there is exactly one "sidecar" proxy (that is one that has `Proxy.DestinationServiceID` set) for the service registered.
 - Several fixes for edge cases for SidecarService
 - A fix for `Alias` checks - when running locally they didn't update their state until some external thing updated the target. If the target service has no checks registered as below, then the alias never made it past critical.
2018-10-10 16:55:34 +01:00
Josh Soref 94835a2715 Spelling (#3958)
* spelling: another

* spelling: autopilot

* spelling: beginning

* spelling: circonus

* spelling: default

* spelling: definition

* spelling: distance

* spelling: encountered

* spelling: enterprise

* spelling: expands

* spelling: exits

* spelling: formatting

* spelling: health

* spelling: hierarchy

* spelling: imposed

* spelling: independence

* spelling: inspect

* spelling: last

* spelling: latest

* spelling: client

* spelling: message

* spelling: minimum

* spelling: notify

* spelling: nonexistent

* spelling: operator

* spelling: payload

* spelling: preceded

* spelling: prepared

* spelling: programmatically

* spelling: required

* spelling: reconcile

* spelling: responses

* spelling: request

* spelling: response

* spelling: results

* spelling: retrieve

* spelling: service

* spelling: significantly

* spelling: specifies

* spelling: supported

* spelling: synchronization

* spelling: synchronous

* spelling: themselves

* spelling: unexpected

* spelling: validations

* spelling: value
2018-03-19 16:56:00 +00:00
Frank Schroeder cc0499da3d
ae: do not trigger on Resume while holding the lock 2017-10-23 10:56:05 +02:00
Frank Schroeder 92f136de12
ae: add remaining test cases 2017-10-23 10:56:05 +02:00
Frank Schroeder 622ace2829
ae: refactor StateSyncer to state machine for better testing 2017-10-23 10:56:05 +02:00
Frank Schroeder 1212598ae2
ae: add test that we run a full before a partial sync 2017-10-23 10:56:05 +02:00
Frank Schroeder 4431e222fe
ae: make control flow more explicit 2017-10-23 10:56:05 +02:00
Frank Schroeder 3231385089
ae: fix typo in constructor name 2017-10-23 10:56:05 +02:00
Frank Schroeder 51daa96dfe
ae: add test for resume triggering SyncChanges 2017-10-23 10:56:05 +02:00
Frank Schroeder 92088d21e8
ae: add test for ifNotPausedRun 2017-10-23 10:56:05 +02:00
Frank Schroeder 11e172d1e9
ae: make stagger function pluggable for testing 2017-10-23 10:56:05 +02:00
Frank Schroeder e2452efed8
ae: restore previous pause/resume behavior 2017-10-23 10:56:04 +02:00
Frank Schroeder aba072bd1d
ae: ensure that syncs are blocked when paused 2017-10-23 10:56:04 +02:00
Frank Schroeder de57b16d99
local state: address review comments
* move non-blocking notification mechanism into ae.Trigger
* move Pause/Resume into separate type
2017-10-23 10:56:04 +02:00
Frank Schroeder 71c74e62c7
local state: fix anti-entropy state tests
The anti-entropy tests relied on the side-effect of the StartSync()
method to perform a full sync instead of a partial sync. This lead to
multiple anti-entropy go routines being started unnecessary retry loops.

This change changes the behavior to perform synchronous full syncs when
necessary removing the need for all of the time.Sleep and most of the
retry loops.
2017-10-23 10:56:04 +02:00
Frank Schroeder cbaf97bced
agent: refactor sync loop to linear flow of control 2017-10-23 10:56:03 +02:00
Frank Schroeder 94ef1041a1
agent: cleanup StateSyncer
This patch cleans up the state syncer code by renaming fields, adding
helpers and documentation.
2017-10-23 10:56:03 +02:00
Frank Schroeder 29e18c7494
agent: decouple anti-entropy from local state
The anti-entropy code manages background synchronizations of the local
state on a regular basis or on demand when either the state has changed
or a new consul server has been added.

This patch moves the anti-entropy code into its own package and
decouples it from the local state code since they are performing
two different functions.

To simplify code-review this revision does not make any optimizations,
renames or refactorings. This will happen in subsequent commits.
2017-10-23 10:56:03 +02:00
Frank Schroeder 58b0e153f9
Revert "agent: decouple anti-entropy from local state"
This reverts commit a842dc9c2b.
2017-10-23 10:08:35 +02:00
Frank Schroeder b4e7d0b974
Revert "agent: cleanup StateSyncer"
This reverts commit b7136e100b.
2017-10-23 10:08:35 +02:00
Frank Schroeder 26a155eb41
Revert "agent: refactor sync loop to linear flow of control"
This reverts commit 7a2af206ea.
2017-10-23 10:08:35 +02:00
Frank Schroeder 67cdfc038e
Revert "local state: fix anti-entropy state tests"
This reverts commit f8e20cd996.
2017-10-23 10:08:34 +02:00
Frank Schroeder 46641e44d9
Revert "local state: address review comments"
This reverts commit 1d315075b1.
2017-10-23 10:08:33 +02:00
Frank Schroeder e95d22b9a8
Revert "ae: ensure that syncs are blocked when paused"
This reverts commit ffb265dd93.
2017-10-23 10:08:33 +02:00
Frank Schroeder 23a9ac9d56
Revert "ae: restore previous pause/resume behavior"
This reverts commit 126046be23.
2017-10-23 10:08:33 +02:00
Frank Schroeder 93d03595d1
Revert "ae: make stagger function pluggable for testing"
This reverts commit 066ad01c38.
2017-10-23 10:08:33 +02:00
Frank Schroeder 3d202b59bc
Revert "ae: add test for ifNotPausedRun"
This reverts commit f5177ef332.
2017-10-23 10:08:33 +02:00
Frank Schroeder e0e96496f1
Revert "ae: add test for resume triggering SyncChanges"
This reverts commit cd0262744d.
2017-10-23 10:08:32 +02:00
Frank Schroeder cab3b17292
Revert "ae: fix typo in constructor name"
This reverts commit e88f49e2cc.
2017-10-23 10:08:32 +02:00
Frank Schroeder 25ed78c8f7
Revert "ae: make control flow more explicit"
This reverts commit b9a8b53d52.
2017-10-23 10:08:32 +02:00
Frank Schroeder 65166cce8e
Revert "ae: add test that we run a full before a partial sync"
This reverts commit 8158cec829.
2017-10-23 10:08:32 +02:00
Frank Schroeder e78520c3f9
Revert "ae: refactor StateSyncer to state machine for better testing"
This reverts commit 8a45365f68.
2017-10-23 10:08:32 +02:00
Frank Schroeder 4121cafed7
Revert "ae: add remaining test cases"
This reverts commit c32915bb4f.
2017-10-23 10:08:32 +02:00
Frank Schroeder f8202f300f
Revert "ae: do not trigger on Resume while holding the lock"
This reverts commit bd00814301.
2017-10-23 10:08:32 +02:00
Frank Schroeder bd00814301 ae: do not trigger on Resume while holding the lock 2017-10-23 08:03:18 +02:00
Frank Schroeder c32915bb4f ae: add remaining test cases 2017-10-23 08:03:18 +02:00
Frank Schroeder 8a45365f68 ae: refactor StateSyncer to state machine for better testing 2017-10-23 08:03:18 +02:00
Frank Schroeder 8158cec829 ae: add test that we run a full before a partial sync 2017-10-23 08:03:18 +02:00
Frank Schroeder b9a8b53d52 ae: make control flow more explicit 2017-10-23 08:03:18 +02:00
Frank Schroeder e88f49e2cc ae: fix typo in constructor name 2017-10-23 08:03:18 +02:00
Frank Schroeder cd0262744d ae: add test for resume triggering SyncChanges 2017-10-23 08:03:18 +02:00
Frank Schroeder f5177ef332 ae: add test for ifNotPausedRun 2017-10-23 08:03:18 +02:00
Frank Schroeder 066ad01c38 ae: make stagger function pluggable for testing 2017-10-23 08:03:18 +02:00
Frank Schroeder 126046be23 ae: restore previous pause/resume behavior 2017-10-23 08:03:18 +02:00
Frank Schroeder ffb265dd93 ae: ensure that syncs are blocked when paused 2017-10-23 08:03:18 +02:00
Frank Schroeder 1d315075b1 local state: address review comments
* move non-blocking notification mechanism into ae.Trigger
* move Pause/Resume into separate type
2017-10-23 08:03:18 +02:00
Frank Schroeder f8e20cd996 local state: fix anti-entropy state tests
The anti-entropy tests relied on the side-effect of the StartSync()
method to perform a full sync instead of a partial sync. This lead to
multiple anti-entropy go routines being started unnecessary retry loops.

This change changes the behavior to perform synchronous full syncs when
necessary removing the need for all of the time.Sleep and most of the
retry loops.
2017-10-23 08:03:18 +02:00