0f4e24f72c
Consolidate server lookup into one place and replace usages of localConsuls.
2017-08-30 09:30:33 -05:00
e639154abd
Remove stray commented line
2017-08-30 09:30:33 -05:00
00836a6aab
Remove server address tracking logic from manager/router and maintain it as part of lan event listener instead. Used sync.Map to track this, and added unit tests
2017-08-30 09:30:33 -05:00
830aca958a
ServerAddressProvider interface also returns an error now
2017-08-30 09:30:33 -05:00
c68fce89b5
Use config struct to create NetworkTransport layer when setting up raft
2017-08-30 09:30:33 -05:00
393ce1581b
Implement AddressProvider and wire that up to raft transport layer to support server nodes changing their IP addresses in containerized environments
2017-08-30 09:30:33 -05:00
831d84c940
build: make tests independent of build tags
...
When the metadata server is scanning the agents for potential servers
it is parsing the version number which the agent provided when it
joined. This version number has to conform to a certain format, i.e.
'n.n.n'. Without this version number properly set some tests fail with
error messages that disguise the root cause.
The default version number is currently set to 'unknown' in
version/version.go which does not parse and triggers the tests to fail.
The work around is to use a build tag 'consul' which will use the
version number set in version_base.go instead which has the correct
format and is set to the current release version.
In addition, some parts of the code also require the version number to
be of a certain value. Setting it to '0.0.0' for example makes some
tests pass and others fail since they don't pass the semantic check.
When using go build/install/test one has to remember to use '-tags
consul' or tests will fail with non-obvious error messages.
Using build tags makes the build process more complex and error prone
since it prevents the use of the plain go toolchain and - at least in
its current form - introduces subtle build and test issues. We should
try to eliminate build tags for anything else but platform specific
code.
This patch removes all references to specific version numbers in the
code and tests and sets the default version to '9.9.9' which is
syntactically correct and passes the semantic check. This solves the
issue of running go build/install/test without tags for the OSS build.
2017-08-30 13:40:18 +02:00
a3934c263c
acl: consolidate error handling ( #3401 )
...
The error handling of the ACL code relies on the presence of certain
magic error messages. Since the error values are sent via RPC between
older and newer consul agents we cannot just replace the magic values
with typed errors and switch to type checks since this would break
compatibility with older clients.
Therefore, this patch moves all magic ACL error messages into the acl
package and provides default error values and helper functions which
determine the type of error.
2017-08-23 16:52:48 +02:00
16c58da27d
agent: drop unused code
...
This code from http://github.com/hashicorp/consul/pull/3353 is no longer
required.
2017-08-22 00:02:46 +02:00
e8a83bb463
Revert "Return 403 rather than a 404 when acls cause all results to be filter…"
2017-08-09 15:06:57 -07:00
02a87df044
Revert "Ensure that we return a permission denied only if the list of keys/en…"
2017-08-09 15:06:20 -07:00
42fb49c00b
Added unit test case to kvs_endpointtest
2017-08-09 15:50:22 -05:00
3276891142
Ensure that we return a permission denied only if the list of keys/entries prior to filtering by ACL is non empty
2017-08-09 15:32:18 -05:00
7cff50a4df
agent: move agent/consul/agent to agent/metadata
2017-08-09 14:36:52 +02:00
c395599cea
agent: move agent/consul/servers to agent/router
2017-08-09 14:36:37 +02:00
1acff3533e
agent: move agent/consul/structs to agent/structs
2017-08-09 14:32:12 +02:00
cf02e3bc22
Merge pull request #3369 from hashicorp/metrics-enhancements
...
Add support for labels/filters from go-metrics
2017-08-08 13:55:30 -07:00
d5634fe2a8
Add support for labels/filters from go-metrics
2017-08-08 01:45:10 -07:00
37f75a393e
Use sanitized version of node name of server in NS record, and start with "server" rather than "ns"
2017-08-07 11:11:55 +02:00
794d1afe44
Removed a copy pasted irrelevant comment, and other code review feedback
2017-08-07 11:11:54 +02:00
f9db387097
Add NS records and A records for each server. Constructs ns host names using the advertise address of the server.
2017-08-07 11:11:54 +02:00
4bee2e49f5
Adds secure introduction for the ACL replication token. ( #3357 )
...
Adds secure introduction for the ACL replication token, as well as a separate enable config for ACL replication.
2017-08-03 15:39:31 -07:00
c0a5ad7903
Adds a new /v1/acl/bootstrap API ( #3349 )
2017-08-02 17:05:18 -07:00
4076c0d741
Return nil instead of empty list when returning a PermissionDenied error, updated unit test
2017-07-31 17:23:20 -05:00
6336014a86
Return 403 rather than a 404 when acls cause all results to be filtered out. This fixes #2637
2017-07-31 13:50:29 -05:00
10b660d77a
Adds missing autopilot snapshot test and avoids snapshotting nil. ( #3333 )
2017-07-28 15:48:42 -07:00
6250cd70f5
Adds option to prepared queries to remove empty tags. ( #3330 )
2017-07-26 22:46:43 -07:00
496b0bcf07
Adds support for agent-side ACL token management via API instead of config files. ( #3324 )
...
* Adds token store and removes all runtime use of config for ACL tokens.
* Adds a new API for changing agent tokens on the fly.
2017-07-26 11:03:43 -07:00
b94617b281
Add extra test case for deleting entire tree with empty prefix
2017-07-26 09:42:07 -05:00
4498814843
Don't insert tombstone for empty prefix delete. Other minor unit test fixes
2017-07-25 21:54:11 -05:00
fee418d378
Removed redundant comments and unit test
2017-07-25 20:39:33 -05:00
b772c477c2
Removed redundant call to reap tombstone from unit test
2017-07-25 19:39:05 -05:00
ae443e21d6
Improved unit test per code review
2017-07-25 19:17:40 -05:00
36acf8d6a4
Use new DeletePrefixMethod for implementing KVSDeleteTree operation. This makes deletes on sub trees larger than one million nodes about 100 times faster. Added unit tests.
2017-07-25 17:21:18 -05:00
0047b7d3f0
fix spelling in filenames
...
Fixes #3301
2017-07-19 13:16:38 +02:00
19eae3d14b
Add UpgradeVersionTag to autopilot config
2017-07-18 13:35:41 -07:00
1791d99a10
Adds new config to make script checks opt-in, updates documentation. ( #3284 )
2017-07-17 11:20:35 -07:00
78c3a86405
Add TLS setting to router areas
2017-07-14 17:38:08 -07:00
0881e46111
Cleans up version 8 ACLs in the agent and the docs. ( #3248 )
...
* Moves magic check and service constants into shared structs package.
* Removes the "consul" service from local state.
Since this service is added by the leader, it doesn't really make sense to
also keep it in local state (which requires special ACLs to configure), and
requires a bunch of special cases in the local state logic. This requires
fewer special cases and makes ACL bootstrapping cleaner.
* Makes coordinate update ACL log message a warning, similar to other AE warnings.
* Adds much more detailed examples for bootstrapping ACLs.
This can hopefully replace https://gist.github.com/slackpad/d89ce0e1cc0802c3c4f2d84932fa3234 .
2017-07-13 22:33:47 -07:00
1781fd311f
address review comments
2017-07-07 09:22:34 +02:00
e4b40acc7e
agent: remove unused code
2017-07-07 09:22:34 +02:00
8c792ad57d
agent: make TestClient_RPC_ConsulServerPing more robust
2017-07-07 09:22:34 +02:00
a855d31f84
Adds a comment about flood joining.
2017-07-07 09:22:34 +02:00
5b5217528a
Simplifies Serf dynamic port selection code.
...
This isn't racy, it's just a little dirty. The listen will happen and a port
will be selected and injected into the config once the Serf instance is
created, so we don't need the retry loop here.
2017-07-07 09:22:34 +02:00
d8db4bc086
test: Changes WAN/LAN join confirmer to use port number vs. address.
...
This fixes TestServer_JoinSeparateLanAndWanAddresses which sets bogus
advertise addresses as part of the test. Port numbers uniquely identify
members since everything is running on localhost.
2017-07-07 09:22:34 +02:00
d92f70f313
test: make joinLAN/WAN reliable
...
only return if the members can see each other
2017-07-07 09:22:34 +02:00
112bc19cd5
rpc: make TestServer_JoinSeparateLanAndWanAddresses more robust
2017-07-07 09:22:34 +02:00
ffd45f5da5
rpc: make TestClient_SnapshotRPC_TLS more robust
2017-07-07 09:22:34 +02:00
2159d499e3
rpc: try shutting down leader first to avoid hang in TestLeader_LeftServer
2017-07-07 09:22:34 +02:00
f12fac278e
rpc: fix logging and try quicker timing of TestServer_JoinSeparateLanAndWanAddresses
2017-07-07 09:22:34 +02:00
bae4b1d045
rpc: less agressive raft timeouts
...
Allowing more time for raft to consolidate should
drop the number of leader elections.
2017-07-07 09:22:34 +02:00
457b98a099
rpc: run agent/consul tests in parallel
2017-07-07 09:22:34 +02:00
13eeeb720d
rpc: refactor sessionTimers and fix racy tests
...
The sessionTimers map was secured by a lock which wasn't used
properly in the tests. This lead to data races and failing tests
when accessing the length or the members of the map.
This patch adds a separate SessionTimers struct which is safe
for concurrent use and which ecapsulates the behavior of the
sessionTimers map.
2017-07-07 09:22:34 +02:00
05f756853e
rpc: fix TestServer_Leave
...
wait for the leader election.
2017-07-07 09:22:34 +02:00
583959392b
rpc: fix TestSession_Renew
...
make the timing less tight
2017-07-07 09:22:34 +02:00
ff2c29c0be
rpc: fix TestReadyForConsistentRead
...
timing was too tight. Standardized name.
2017-07-07 09:22:34 +02:00
fcab525053
rpc: fix for 'no leader' in TLS tests
...
Ensure both servers know about each other before looking
for a leader.
2017-07-07 09:22:34 +02:00
b2a71fd8b0
rpc: fix TestServer_JoinWAN_Flood
...
The second server in the first data center should not be
in bootstrap mode.
2017-07-07 09:22:34 +02:00
8369b6cb9d
rpc: provide unique node names for server and client
2017-07-07 09:22:34 +02:00
534977239b
rpc: prefix log output with test name
2017-07-07 09:22:34 +02:00
c8ef588d8d
rpc: discover serf wan port before starting serf lan
...
When using dynamic ports for the serf clusters then
the actual bind port of the serf WAN cluster needs to
be discovered before the serf LAN cluster is started
since the serf LAN cluster announces the port of the WAN
cluster.
2017-07-07 09:22:34 +02:00
53eab7e970
rpc: bind rpc test server to port 0
2017-07-07 09:22:34 +02:00
e9e2c599db
rpc: refactor: unify test server setup
2017-07-07 09:22:34 +02:00
c803146550
rpc: fix typos
2017-07-07 09:22:34 +02:00
a0368e3827
agent: refactor: log to stderr during tests
2017-07-07 09:22:34 +02:00
f549c06764
Rename to raftNotifyCh, fix typo
2017-07-06 09:10:36 -05:00
f2171a6720
Fixes deadlock between barrier write and leader notify channel read . Fixes #3230
2017-07-05 17:09:18 -05:00
e4b11682bc
Fixes broken HTTP header and method for health checks. ( #3178 )
...
* Fixes broken HTTP header and method for health checks.
* Adds a fuzz utility and test to make sure copy is complete.
2017-06-23 01:15:48 -07:00
2b41f2e3a3
agent: make the RPC endpoint overwrite mechanism more transparent
...
This patch hides the RPC handler overwrite mechanism from the
rest of the code so that it works in all cases and that there
is no cooperation required from the tested code, i.e. we can
drop a.getEndpoint().
2017-06-21 05:42:39 +02:00
c49a15d0f3
agent: move structs into consul/structs pkg
...
* CheckDefinition
* ServiceDefinition
* CheckType
2017-06-21 05:42:39 +02:00
4273fb8444
agent: move NotifyGroup into the agent pkg
2017-06-21 05:42:39 +02:00
82a132da60
agent: move conn pool for muxed connections into separate pkg
2017-06-21 05:42:39 +02:00
80971c8a85
agent: move the SnapshotReplyFn out of the way
...
When splitting up the consul package into server and client
the SnapshotReplyFn needs to be in a separate package to avoid
a circular dependency.
2017-06-21 05:42:39 +02:00
04b9392b00
agent: use the delegate interface for local state
2017-06-21 05:42:39 +02:00
f658231ab9
Minor fixes per code review
2017-06-20 19:43:07 -05:00
b3b2e9dcb4
Added unit test to verify consistentRead method behavior
2017-06-16 11:58:12 -05:00
44f5086873
Code review feedback, fixed major logic bug
2017-06-16 10:49:54 -05:00
72af7b9bc4
Redo bug fix for stale reads on server startup, leveraging RPCHOldtimeout instead of maxQueryTime, plus tests
2017-06-15 22:41:30 -05:00
1c75cf1af5
pkg refactor
...
command/agent/* -> agent/*
command/consul/* -> agent/consul/*
command/agent/command{,_test}.go -> command/agent{,_test}.go
command/base/command.go -> command/base.go
command/base/* -> command/*
commands.go -> command/commands.go
The script which did the refactor is:
(
cd $GOPATH/src/github.com/hashicorp/consul
git mv command/agent/command.go command/agent.go
git mv command/agent/command_test.go command/agent_test.go
git mv command/agent/flag_slice_value{,_test}.go command/
git mv command/agent .
git mv command/base/command.go command/base.go
git mv command/base/config_util{,_test}.go command/
git mv commands.go command/
git mv consul agent
rmdir command/base/
gsed -i -e 's|package agent|package command|' command/agent{,_test}.go
gsed -i -e 's|package agent|package command|' command/flag_slice_value{,_test}.go
gsed -i -e 's|package base|package command|' command/base.go command/config_util{,_test}.go
gsed -i -e 's|package main|package command|' command/commands.go
gsed -i -e 's|base.Command|BaseCommand|' command/commands.go
gsed -i -e 's|agent.Command|AgentCommand|' command/commands.go
gsed -i -e 's|\tCommand:|\tBaseCommand:|' command/commands.go
gsed -i -e 's|base\.||' command/commands.go
gsed -i -e 's|command\.||' command/commands.go
gsed -i -e 's|command|c|' main.go
gsed -i -e 's|range Commands|range command.Commands|' main.go
gsed -i -e 's|Commands: Commands|Commands: command.Commands|' main.go
gsed -i -e 's|base\.BoolValue|BoolValue|' command/operator_autopilot_set.go
gsed -i -e 's|base\.DurationValue|DurationValue|' command/operator_autopilot_set.go
gsed -i -e 's|base\.StringValue|StringValue|' command/operator_autopilot_set.go
gsed -i -e 's|base\.UintValue|UintValue|' command/operator_autopilot_set.go
gsed -i -e 's|\bCommand\b|BaseCommand|' command/base.go
gsed -i -e 's|BaseCommand Options|Command Options|' command/base.go
gsed -i -e 's|base.Command|BaseCommand|' command/*.go
gsed -i -e 's|c\.Command|c.BaseCommand|g' command/*.go
gsed -i -e 's|\tCommand:|\tBaseCommand:|' command/*_test.go
gsed -i -e 's|base\.||' command/*_test.go
gsed -i -e 's|\bCommand\b|AgentCommand|' command/agent{,_test}.go
gsed -i -e 's|cmd.AgentCommand|cmd.BaseCommand|' command/agent.go
gsed -i -e 's|cli.AgentCommand = new(Command)|cli.Command = new(AgentCommand)|' command/agent_test.go
gsed -i -e 's|exec.AgentCommand|exec.Command|' command/agent_test.go
gsed -i -e 's|exec.BaseCommand|exec.Command|' command/agent_test.go
gsed -i -e 's|NewTestAgent|agent.NewTestAgent|' command/agent_test.go
gsed -i -e 's|= TestConfig|= agent.TestConfig|' command/agent_test.go
gsed -i -e 's|: RetryJoin|: agent.RetryJoin|' command/agent_test.go
gsed -i -e 's|\.\./\.\./|../|' command/config_util_test.go
gsed -i -e 's|\bverifyUniqueListeners|VerifyUniqueListeners|' agent/config{,_test}.go command/agent.go
gsed -i -e 's|\bserfLANKeyring\b|SerfLANKeyring|g' agent/{agent,keyring,testagent}.go command/agent.go
gsed -i -e 's|\bserfWANKeyring\b|SerfWANKeyring|g' agent/{agent,keyring,testagent}.go command/agent.go
gsed -i -e 's|\bNewAgent\b|agent.New|g' command/agent{,_test}.go
gsed -i -e 's|\bNewAgent|New|' agent/{acl_test,agent,testagent}.go
gsed -i -e 's|\bAgent\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bBool\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bConfig\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bDefaultConfig\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bDevConfig\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bMergeConfig\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bReadConfigPaths\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bParseMetaPair\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bSerfLANKeyring\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bSerfWANKeyring\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|circonus\.agent|circonus|g' command/agent{,_test}.go
gsed -i -e 's|logger\.agent|logger|g' command/agent{,_test}.go
gsed -i -e 's|metrics\.agent|metrics|g' command/agent{,_test}.go
gsed -i -e 's|// agent.Agent|// agent|' command/agent{,_test}.go
gsed -i -e 's|a\.agent\.Config|a.Config|' command/agent{,_test}.go
gsed -i -e 's|agent\.AppendSliceValue|AppendSliceValue|' command/{configtest,validate}.go
gsed -i -e 's|consul/consul|agent/consul|' GNUmakefile
gsed -i -e 's|\.\./test|../../test|' agent/consul/server_test.go
# fix imports
f=$(grep -rl 'github.com/hashicorp/consul/command/agent' * | grep '\.go')
gsed -i -e 's|github.com/hashicorp/consul/command/agent|github.com/hashicorp/consul/agent|' $f
goimports -w $f
f=$(grep -rl 'github.com/hashicorp/consul/consul' * | grep '\.go')
gsed -i -e 's|github.com/hashicorp/consul/consul|github.com/hashicorp/consul/agent/consul|' $f
goimports -w $f
goimports -w command/*.go main.go
)
2017-06-10 18:52:45 +02:00
Preetha Appan
Frank Schroeder
James Phillips
Kyle Havlovitz