Commit Graph

752 Commits

Author SHA1 Message Date
Hans Hasselberg ac45b17482
fix remaining CI failures after Go 1.12.1 Upgrade (#5576) 2019-03-29 16:29:27 +01:00
Jeff Mitchell f1fe406aa8
Update SDK version (#5572) 2019-03-27 08:58:50 -04:00
Jeff Mitchell 4243c3ae42
Move internal/ to sdk/ (#5568)
* Move internal/ to sdk/

* Add a readme to the SDK folder
2019-03-27 08:54:56 -04:00
Jeff Mitchell 6327d3fd6f
Update internal dep (#5564) 2019-03-26 17:30:23 -04:00
Jeff Mitchell 47c390025b
Convert to Go Modules (#5517)
* First conversion

* Use serf 0.8.2 tag and associated updated deps

* * Move freeport and testutil into internal/

* Make internal/ its own module

* Update imports

* Add replace statements so API and normal Consul code are
self-referencing for ease of development

* Adapt to newer goe/values

* Bump to new cleanhttp

* Fix ban nonprintable chars test

* Update lock bad args test

The error message when the duration cannot be parsed changed in Go 1.12
(ae0c435877d3aacb9af5e706c40f9dddde5d3e67). This updates that test.

* Update another test as well

* Bump travis

* Bump circleci

* Bump go-discover and godo to get rid of launchpad dep

* Bump dockerfile go version

* fix tar command

* Bump go-cleanhttp
2019-03-26 17:04:58 -04:00
R.B. Boyer f8bcfcfa46
api: fix panic in 'consul acl set-agent-token' (#5533)
api: fix panic in 'consul acl set-agent-token'

Fixes #5531
2019-03-21 15:04:40 -05:00
R.B. Boyer f4a3b9d518
fix typos reported by golangci-lint:misspell (#5434) 2019-03-06 11:13:28 -06:00
Matt Keeler 118adbb123
ACL Token Persistence and Reloading (#5328)
This PR adds two features which will be useful for operators when ACLs are in use.

1. Tokens set in configuration files are now reloadable.
2. If `acl.enable_token_persistence` is set to `true` in the configuration, tokens set via the `v1/agent/token` endpoint are now persisted to disk and loaded when the agent starts (or during configuration reload)

Note that token persistence is opt-in so our users who do not want tokens on the local disk will see no change.

Some other secondary changes:

* Refactored a bunch of places where the replication token is retrieved from the token store. This token isn't just for replicating ACLs and now it is named accordingly.
* Allowed better paths in the `v1/agent/token/` API. Instead of paths like: `v1/agent/token/acl_replication_token` the path can now be just `v1/agent/token/replication`. The old paths remain to be valid. 
* Added a couple new API functions to set tokens via the new paths. Deprecated the old ones and pointed to the new names. The names are also generally better and don't imply that what you are setting is for ACLs but rather are setting ACL tokens. There is a minor semantic difference there especially for the replication token as again, its no longer used only for ACL token/policy replication. The new functions will detect 404s and fallback to using the older token paths when talking to pre-1.4.3 agents.
* Docs updated to reflect the API additions and to show using the new endpoints.
* Updated the ACL CLI set-agent-tokens command to use the non-deprecated APIs.
2019-02-27 14:28:31 -05:00
Alvin Huang e999e404da add serf check to TestAPI_ACLToken_List 2019-02-22 17:34:45 -05:00
Alvin Huang aed9d123fa add serf check to various api/agent tests 2019-02-22 17:34:45 -05:00
Alvin Huang 8124f5889c add serf check to TestAPI_CatalogNodes 2019-02-22 17:34:45 -05:00
Alvin Huang 9780892ebe add serf check to TestAPI_ConnectCARoots_empty, TestAPI_ConnectCAConfig_get_set 2019-02-22 17:34:45 -05:00
Alvin Huang 7b4fb32545 add serf check to TestAPI_CoordinateUpdate 2019-02-22 17:34:45 -05:00
Alvin Huang a2eb3c204d add serf check to TestAPI_HealthChecks_NodeMetaFilter, TestAPI_HealthConnect 2019-02-22 17:34:45 -05:00
Alvin Huang 7252679232 add serf check to TestAPI_ClientPutGetDelete, TestAPI_ClientWatchGet 2019-02-22 17:34:45 -05:00
Alvin Huang d93c82e966 add serf check to TestAPI_SemaphoreConflict 2019-02-22 17:34:45 -05:00
Alvin Huang f98512f1a3 add serf check to TestAPI_Snapshot 2019-02-22 17:34:45 -05:00
Alvin Huang a5bd6d7c00 add serf check to TestAPI_AgentServices_ManagedConnectProxyDeprecatedUpstreams 2019-02-22 17:34:45 -05:00
Alvin Huang 5f2e709c3f remove serf check 2019-02-22 17:34:45 -05:00
Alvin Huang fe7b841e4a add serf check to TestAPI_SemaphoreConflict 2019-02-22 17:34:45 -05:00
Matt Keeler acfd87c673
Improve Connect with Prepared Queries (#5291)
Given a query like:

```
{
   "Name": "tagged-connect-query",
   "Service": {
      "Service": "foo",
      "Tags": ["tag"],
      "Connect": true
   }
}
```

And a Consul configuration like:

```
{
   "services": [
      "name": "foo",
      "port": 8080,
      "connect": { "sidecar_service": {} },
      "tags": ["tag"]
   ]
}
```

If you executed the query it would always turn up with 0 results. This was because the sidecar service was being created without any tags. You could instead make your config look like:

```
{
   "services": [
      "name": "foo",
      "port": 8080,
      "connect": { "sidecar_service": {
         "tags": ["tag"]
      } },
      "tags": ["tag"]
   ]
}
```

However that is a bit redundant for most cases. This PR ensures that the tags and service meta of the parent service get copied to the sidecar service. If there are any tags or service meta set in the sidecar service definition then this copying does not take place. After the changes, the query will now return the expected results.

A second change was made to prepared queries in this PR which is to allow filtering on ServiceMeta just like we allow for filtering on NodeMeta.
2019-02-04 09:36:51 -05:00
Matt Keeler 1885cf78f6
Fall back to using old fields when marshaling
Co-Authored-By: kyhavlov <kylehav@gmail.com>
2019-01-28 08:47:26 -08:00
Kyle Havlovitz 1a4978fb94
Re-add ReadableDuration types to health check definition
This is to fix the backwards-incompatible change made in 1.4.1 by
changing these fields to time.Duration.
2019-01-25 14:47:35 -08:00
Hans Hasselberg 174099593a
agent: display messages from serf in cli (#5236)
* display messages from serf in cli
2019-01-22 21:08:50 +01:00
Kyle Havlovitz 5bdf130767
Merge pull request #4869 from hashicorp/txn-checks
Add node/service/check operations to transaction api
2019-01-22 11:16:09 -08:00
Paul Banks ef9f27cbc8
connect: tame thundering herd of CSRs on CA rotation (#5228)
* Support rate limiting and concurrency limiting CSR requests on servers; handle CA rotations gracefully with jitter and backoff-on-rate-limit in client

* Add CSR rate limiting docs

* Fix config naming and add tests for new CA configs
2019-01-22 17:19:36 +00:00
Kyle Havlovitz 21380021af txn: update existing txn api docs with new operations 2019-01-15 16:54:07 -08:00
Paul Banks bb7145f27d
agent: add default weights to service in local state to prevent AE churn (#5126)
* Add default weights when adding a service with no weights to local state to prevent constant AE re-sync.

This fix was contributed by @42wim in https://github.com/hashicorp/consul/pull/5096 but was merged against the wrong base. This adds it to master and adds a test to cover the behaviour.

* Fix tests that broke due to comparing internal state which now has default weights
2019-01-08 10:13:49 +00:00
Grégoire Seux 4f62a3b528 Implement /v1/agent/health/service/<service name> endpoint (#3551)
This endpoint aggregates all checks related to <service id> on the agent
and return an appropriate http code + the string describing the worst
check.

This allows to cleanly expose service status to other component, hiding
complexity of multiple checks.
This is especially useful to use consul to feed a load balancer which
would delegate health checking to consul agent.

Exposing this endpoint on the agent is necessary to avoid a hit on
consul servers and avoid decreasing resiliency (this endpoint will work
even if there is no consul leader in the cluster).
2019-01-07 09:39:23 -05:00
Pierre Souchay f4dc8b42e0 [Travis][UnstableTests] Fixed unstable tests in travis (#5013)
* [Travis][UnstableTests] Fixed unstable tests in travis as seen in https://travis-ci.org/hashicorp/consul/jobs/460824602

* Fixed unstable tests in https://travis-ci.org/hashicorp/consul/jobs/460857687
2018-12-12 12:09:42 -08:00
Kyle Havlovitz 67bac7a815 api: add support for new txn operations 2018-12-12 10:54:09 -08:00
Kyle Havlovitz ab58986ac3 txn: add node operations 2018-12-12 10:04:10 -08:00
Kyle Havlovitz b371ea8783 Add check operations to transaction api 2018-12-12 10:04:10 -08:00
Philipp Gillé 2de7fefedd Remove wrong space character (#4910)
There should be no space before a comma
2018-11-06 22:34:36 +01:00
Matt Keeler db2cf01406 Adds documentation for the new ACL APIs (#4851)
* Update the ACL API docs

* Add a CreateTime to the anon token

Also require acl:read permissions at least to perform rule translation. Don’t want someone DoSing the system with an open endpoint that actually does a bit of work.

* Fix one place where I was referring to id instead of AccessorID

* Add godocs for the API package additions.

* Minor updates: removed some extra commas and updated the acl intro paragraph

* minor tweaks

* Updated the language to be clearer

* Updated the language to be clearer for policy page

* I was also confused by that! Your updates are much clearer.

Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>

* Sounds much better.

Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>

* Updated sidebar layout and deprecated warning
2018-10-31 15:11:51 -07:00
Rémi Lapeyre f8e56d5365 Add support for multiple checks in Consul Client (#4849) 2018-10-29 09:39:25 -07:00
Matt Keeler d238cb181c
New ACL API Tests (#4848)
* A few API mods and unit tests.

* Update the unit tests to verify query/write metadata and to fix the rules endpoint tests.

* Make sure the full information for the replication status is in the api packge
2018-10-25 11:09:46 -04:00
Matt Keeler a02a6be6b9
Implement CLI token cloning & special ID handling (#4827)
* Implement CLI token cloning & special ID handling

* Update a couple CLI commands to take some alternative options.

* Document the CLI.

* Update the policy list and set-agent-token synopsis
2018-10-24 10:24:29 -04:00
Matt Keeler 18b29c45c4
New ACLs (#4791)
This PR is almost a complete rewrite of the ACL system within Consul. It brings the features more in line with other HashiCorp products. Obviously there is quite a bit left to do here but most of it is related docs, testing and finishing the last few commands in the CLI. I will update the PR description and check off the todos as I finish them over the next few days/week.
Description

At a high level this PR is mainly to split ACL tokens from Policies and to split the concepts of Authorization from Identities. A lot of this PR is mostly just to support CRUD operations on ACLTokens and ACLPolicies. These in and of themselves are not particularly interesting. The bigger conceptual changes are in how tokens get resolved, how backwards compatibility is handled and the separation of policy from identity which could lead the way to allowing for alternative identity providers.

On the surface and with a new cluster the ACL system will look very similar to that of Nomads. Both have tokens and policies. Both have local tokens. The ACL management APIs for both are very similar. I even ripped off Nomad's ACL bootstrap resetting procedure. There are a few key differences though.

    Nomad requires token and policy replication where Consul only requires policy replication with token replication being opt-in. In Consul local tokens only work with token replication being enabled though.
    All policies in Nomad are globally applicable. In Consul all policies are stored and replicated globally but can be scoped to a subset of the datacenters. This allows for more granular access management.
    Unlike Nomad, Consul has legacy baggage in the form of the original ACL system. The ramifications of this are:
        A server running the new system must still support other clients using the legacy system.
        A client running the new system must be able to use the legacy RPCs when the servers in its datacenter are running the legacy system.
        The primary ACL DC's servers running in legacy mode needs to be a gate that keeps everything else in the entire multi-DC cluster running in legacy mode.

So not only does this PR implement the new ACL system but has a legacy mode built in for when the cluster isn't ready for new ACLs. Also detecting that new ACLs can be used is automatic and requires no configuration on the part of administrators. This process is detailed more in the "Transitioning from Legacy to New ACL Mode" section below.
2018-10-19 12:04:07 -04:00
Jack Pearkes 8c684db488 New command: consul debug (#4754)
* agent/debug: add package for debugging, host info

* api: add v1/agent/host endpoint

* agent: add v1/agent/host endpoint

* command/debug: implementation of static capture

* command/debug: tests and only configured targets

* agent/debug: add basic test for host metrics

* command/debug: add methods for dynamic data capture

* api: add debug/pprof endpoints

* command/debug: add pprof

* command/debug: timing, wg, logs to disk

* vendor: add gopsutil/disk

* command/debug: add a usage section

* website: add docs for consul debug

* agent/host: require operator:read

* api/host: improve docs and no retry timing

* command/debug: fail on extra arguments

* command/debug: fixup file permissions to 0644

* command/debug: remove server flags

* command/debug: improve clarity of usage section

* api/debug: add Trace for profiling, fix profile

* command/debug: capture profile and trace at the same time

* command/debug: add index document

* command/debug: use "clusters" in place of members

* command/debug: remove address in output

* command/debug: improve comment on metrics sleep

* command/debug: clarify usage

* agent: always register pprof handlers and protect

This will allow us to avoid a restart of a target agent
for profiling by always registering the pprof handlers.

Given this is a potentially sensitive path, it is protected
with an operator:read ACL and enable debug being
set to true on the target agent. enable_debug still requires
a restart.

If ACLs are disabled, enable_debug is sufficient.

* command/debug: use trace.out instead of .prof

More in line with golang docs.

* agent: fix comment wording

* agent: wrap table driven tests in t.run()
2018-10-19 08:41:03 -07:00
Rebecca Zanzig 34e5516834 Support multiple tags for health and catalog http api endpoints (#4717)
* Support multiple tags for health and catalog api endpoints

Fixes #1781.

Adds a `ServiceTags` field to the ServiceSpecificRequest to support
multiple tags, updates the filter logic in the catalog store, and
propagates these change through to the health and catalog endpoints.

Note: Leaves `ServiceTag` in the struct, since it is being used as
part of the DNS lookup, which in turn uses the health check.

* Update the api package to support multiple tags

Includes additional tests.

* Update new tests to use the `require` library

* Update HealthConnect check after a bad merge
2018-10-11 12:50:05 +01:00
Pierre Souchay 251156eb68 Added SOA configuration for DNS settings. (#4714)
This will allow to fine TUNE SOA settings sent by Consul in DNS responses,
for instance to be able to control negative ttl.

Will fix: https://github.com/hashicorp/consul/issues/4713

# Example

Override all settings:

* min_ttl: 0 => 60s
* retry: 600 (10m) => 300s (5 minutes),
* expire: 86400 (24h) => 43200 (12h)
* refresh: 3600 (1h) => 1800 (30 minutes)

```
consul agent -dev -hcl 'dns_config={soa={min_ttl=60,retry=300,expire=43200,refresh=1800}}'
```

Result:
```
dig +multiline @localhost -p 8600 service.consul

; <<>> DiG 9.12.1 <<>> +multiline @localhost -p 8600 service.consul
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 36557
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;service.consul.		IN A

;; AUTHORITY SECTION:
consul.			0 IN SOA ns.consul. hostmaster.consul. (
				1537959133 ; serial
				1800       ; refresh (30 minutes)
				300        ; retry (5 minutes)
				43200      ; expire (12 hours)
				60         ; minimum (1 minute)
				)

;; Query time: 4 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Wed Sep 26 12:52:13 CEST 2018
;; MSG SIZE  rcvd: 93
```
2018-10-10 15:50:56 -04:00
R.B. Boyer c310451b2b cli: avoid passing envoy bootstrap configuration as arguments (#4747)
Play a trick with CLOEXEC to pass the envoy bootstrap configuration as
an open file descriptor to the exec'd envoy process. The file only
briefly touches disk before being unlinked.

We convince envoy to read from this open file descriptor by using the
/dev/fd/$FDNUMBER mechanism to read the open file descriptor as a file.

Because the filename no longer has an extension envoy's sniffing logic
falls back on JSON instead of YAML, so the bootstrap configuration must
be generated as JSON instead.
2018-10-10 16:55:34 +01:00
Paul Banks 161482d2cd Fix up tests broken by master merge; add proxy tests to services command (and fix it!); actually run the proxycfg.Manager 2018-10-10 16:55:34 +01:00
Paul Banks dca1303d05 Connect Envoy Command (#4735)
* Plumb xDS server and proxyxfg into the agent startup

* Add `consul connect envoy` command to allow running Envoy as a connect sidecar.

* Add test for help tabs; typos and style fixups from review
2018-10-10 16:55:34 +01:00
Paul Banks e812f5516a Add -sidecar-for and new /agent/service/:service_id endpoint (#4691)
- A new endpoint `/v1/agent/service/:service_id` which is a generic way to look up the service for a single instance. The primary value here is that it:
   - **supports hash-based blocking** and so;
   - **replaces `/agent/connect/proxy/:proxy_id`** as the mechanism the built-in proxy uses to read its config.
   - It's not proxy specific and so works for any service.
   - It has a temporary shim to call through to the existing endpoint to preserve current managed proxy config defaulting behaviour until that is removed entirely (tested).
 - The built-in proxy now uses the new endpoint exclusively for it's config
 - The built-in proxy now has a `-sidecar-for` flag that allows the service ID of the _target_ service to be specified, on the condition that there is exactly one "sidecar" proxy (that is one that has `Proxy.DestinationServiceID` set) for the service registered.
 - Several fixes for edge cases for SidecarService
 - A fix for `Alias` checks - when running locally they didn't update their state until some external thing updated the target. If the target service has no checks registered as below, then the alias never made it past critical.
2018-10-10 16:55:34 +01:00
Paul Banks 1e7eace066 Add SidecarService Syntax sugar to Service Definition (#4686)
* Added new Config for SidecarService in ServiceDefinitions.

* WIP: all the code needed for SidecarService is written... none of it is tested other than config :). Need API updates too.

* Test coverage for the new sidecarServiceFromNodeService method.

* Test API registratrion with SidecarService

* Recursive Key Translation 🤦

* Add tests for nested sidecar defintion arrays to ensure they are translated correctly

* Use dedicated internal state rather than Service Meta for tracking sidecars for deregistration.

Add tests for deregistration.

* API struct for agent register. No other endpoint should be affected yet.

* Additional test cases to cover updates to API registrations
2018-10-10 16:55:34 +01:00
Paul Banks b83bbf248c Add Proxy Upstreams to Service Definition (#4639)
* Refactor Service Definition ProxyDestination.

This includes:
 - Refactoring all internal structs used
 - Updated tests for both deprecated and new input for:
   - Agent Services endpoint response
   - Agent Service endpoint response
   - Agent Register endpoint
     - Unmanaged deprecated field
     - Unmanaged new fields
     - Managed deprecated upstreams
     - Managed new
   - Catalog Register
     - Unmanaged deprecated field
     - Unmanaged new fields
     - Managed deprecated upstreams
     - Managed new
   - Catalog Services endpoint response
   - Catalog Node endpoint response
   - Catalog Service endpoint response
 - Updated API tests for all of the above too (both deprecated and new forms of register)

TODO:
 - config package changes for on-disk service definitions
 - proxy config endpoint
 - built-in proxy support for new fields

* Agent proxy config endpoint updated with upstreams

* Config file changes for upstreams.

* Add upstream opaque config and update all tests to ensure it works everywhere.

* Built in proxy working with new Upstreams config

* Command fixes and deprecations

* Fix key translation, upstream type defaults and a spate of other subtele bugs found with ned to end test scripts...

TODO: tests still failing on one case that needs a fix. I think it's key translation for upstreams nested in Managed proxy struct.

* Fix translated keys in API registration.
≈

* Fixes from docs
 - omit some empty undocumented fields in API
 - Bring back ServiceProxyDestination in Catalog responses to not break backwards compat - this was removed assuming it was only used internally.

* Documentation updates for Upstreams in service definition

* Fixes for tests broken by many refactors.

* Enable travis on f-connect branch in this branch too.

* Add consistent Deprecation comments to ProxyDestination uses

* Update version number on deprecation notices, and correct upstream datacenter field with explanation in docs
2018-10-10 16:55:34 +01:00
Paul Banks 88388d760d Support Agent Caching for Service Discovery Results (#4541)
* Add cache types for catalog/services and health/services and basic test that caching works

* Support non-blocking cache types with Cache-Control semantics.

* Update API docs to include caching info for every endpoint.

* Comment updates per PR feedback.

* Add note on caching to the 10,000 foot view on the architecture page to make the new data path more clear.

* Document prepared query staleness quirk and force all background requests to AllowStale so we can spread service discovery load across servers.
2018-10-10 16:55:34 +01:00
Freddy 524192c919 Improve resilience of api pkg tests (#4676)
* Add function to wait for serfHealth in api tests

* Disable connect when creating semaphore test clients

* Wait for serfHealth when creating sessions in their tests

* Add helper functions to create lock/semaphore sessions without checks

* Log passing tests to prevent timeout in Travis due to lack of output
2018-09-18 17:47:01 +01:00
Pierre Souchay 22500f242e Fix unstable tests in agent, api, and command/watch 2018-09-10 16:58:53 +01:00
Pierre Souchay eddcf228ea Implementation of Weights Data structures (#4468)
* Implementation of Weights Data structures

Adding this datastructure will allow us to resolve the
issues #1088 and #4198

This new structure defaults to values:
```
   { Passing: 1, Warning: 0 }
```

Which means, use weight of 0 for a Service in Warning State
while use Weight 1 for a Healthy Service.
Thus it remains compatible with previous Consul versions.

* Implemented weights for DNS SRV Records

* DNS properly support agents with weight support while server does not (backwards compatibility)

* Use Warning value of Weights of 1 by default

When using DNS interface with only_passing = false, all nodes
with non-Critical healthcheck used to have a weight value of 1.
While having weight.Warning = 0 as default value, this is probably
a bad idea as it breaks ascending compatibility.

Thus, we put a default value of 1 to be consistent with existing behaviour.

* Added documentation for new weight field in service description

* Better documentation about weights as suggested by @banks

* Return weight = 1 for unknown Check states as suggested by @banks

* Fixed typo (of -> or) in error message as requested by @mkeeler

* Fixed unstable unit test TestRetryJoin

* Fixed unstable tests

* Fixed wrong Fatalf format in `testrpc/wait.go`

* Added notes regarding DNS SRV lookup limitations regarding number of instances

* Documentation fixes and clarification regarding SRV records with weights as requested by @banks

* Rephrase docs
2018-09-07 15:30:47 +01:00
Freddy 4c5758d891
Remove operator_area note from godoc overview (#4603) 2018-08-28 16:02:24 -04:00
Siva Prasad 771e0bafd2
Addresses the flakiness of CatalogNodes (#4530) 2018-08-15 11:16:05 -04:00
Pierre Souchay cec5d72396 BUGFIX: Unit test relying on WaitForLeader() did not work due to wrong test (#4472)
- Improve resilience of testrpc.WaitForLeader()

- Add additionall retry to CI

- Increase "go test" timeout to 8m

- Add wait for cluster leader to several tests in the agent package

- Add retry to some tests in the api and command packages
2018-08-06 19:46:09 -04:00
Geoffrey Grosenbach 827fbac3ab Copy-and-paste Go client example (#4448)
* Copy-and-paste Go client example

Includes Go source that runs without modification, as well as simple
instructions for compiling, running, and viewing the output in the
Consul UI.

* Remove unnecessary flags from development server example

This is a bare minimum Go example needed to store keys and values in
Consul. The `-ui` and `-server` flags aren't needed when running with
`-dev`.
2018-07-30 12:48:19 +01:00
Kyle Havlovitz ed87949385
Merge pull request #4400 from hashicorp/leaf-cert-ttl
Add configurable leaf cert TTL to Connect CA
2018-07-25 17:53:25 -07:00
Paul Banks 8cbeb29e73
Fixes #4421: General solution to stop blocking queries with index 0 (#4437)
* Fix theoretical cache collision bug if/when we use more cache types with same result type

* Generalized fix for blocking query handling when state store methods return zero index

* Refactor test retry to only affect CI

* Undo make file merge

* Add hint to error message returned to end-user requests if Connect is not enabled when they try to request cert

* Explicit error for Roots endpoint if connect is disabled

* Fix tests that were asserting old behaviour
2018-07-25 20:26:27 +01:00
Kyle Havlovitz d6ca015a42
connect/ca: add configurable leaf cert TTL 2018-07-16 13:33:37 -07:00
Matt Keeler 7572ca0f37
Merge pull request #4374 from hashicorp/feature/proxy-env-vars
Setup managed proxy environment with API client env vars
2018-07-12 09:13:54 -04:00
Matt Keeler 7dfd2ab316 Add some tests for GenerateEnv 2018-07-12 07:43:51 -04:00
Matt Keeler c54b43bef3 PR Updates
Proxy now doesn’t need to know anything about the api as we pass env vars to it instead of the api config.
2018-07-11 09:44:54 -04:00
Matt Keeler 3b6eef8ec6 Pass around an API Config object and convert to env vars for the managed proxy 2018-07-10 12:13:51 -04:00
Hamish 9043966efd Fix lock and semaphore timeouts 2018-07-06 10:55:25 +01:00
Mitchell Hashimoto 1c3e9af316
agent: 400 error on invalid UUID format, api handles errors properly 2018-06-27 07:40:06 +02:00
Kyle Havlovitz 162daca4d7 revert go changes to hide rotation config 2018-06-25 12:26:18 -07:00
Kyle Havlovitz c20bbf8760 connect/ca: hide the RotationPeriod config field since it isn't used yet 2018-06-25 12:26:18 -07:00
Kyle Havlovitz bc997688e3 connect/ca: update Consul provider to use new cross-sign CSR method 2018-06-25 12:25:41 -07:00
Paul Banks b5f24a21cb Fix test broken by final telemetry PR change! 2018-06-25 12:25:40 -07:00
Paul Banks 597e55e8e2 Misc test fixes 2018-06-25 12:25:39 -07:00
Paul Banks 96c416012e Misc rebase and test fixes 2018-06-25 12:25:38 -07:00
Paul Banks 43b48bc06b Get agent cache tests passing without global hit count (which is racy).
Few other fixes in here just to get a clean run locally - they are all also fixed in other PRs but shouldn't conflict.

This should be robust to timing between goroutines now.
2018-06-25 12:25:37 -07:00
Paul Banks 05a8097c5d Fix misc test failures (some from other PRs) 2018-06-25 12:25:13 -07:00
Kyle Havlovitz 82a4b3c13f connect: fix two CA tests that were broken in a previous PR (#60) 2018-06-25 12:25:10 -07:00
Paul Banks 0824d1df5f Misc comment cleanups 2018-06-25 12:24:16 -07:00
Mitchell Hashimoto 71216631d7 api: update intention struct for precedence 2018-06-25 12:24:16 -07:00
Mitchell Hashimoto 77a8003475 api: change Connect to a query option 2018-06-25 12:24:14 -07:00
Mitchell Hashimoto b55f0641e6 api: support ExecuteConnect 2018-06-25 12:24:12 -07:00
Mitchell Hashimoto 297e4f272e api: support native connect 2018-06-25 12:24:10 -07:00
Paul Banks df2cb30b01 Make tests pass and clean proxy persistence. No detached child changes yet.
This is a good state for persistence stuff to re-start the detached child work that got mixed up last time.
2018-06-25 12:24:10 -07:00
Paul Banks cdc7cfaa36 Abandon daemonize for simpler solution (preserving history):
Reverts:
  - bdb274852ae469c89092d6050697c0ff97178465
  - 2c689179c4f61c11f0016214c0fc127a0b813bfe
  - d62e25c4a7ab753914b6baccd66f88ffd10949a3
  - c727ffbcc98e3e0bf41e1a7bdd40169bd2d22191
  - 31b4d18933fd0acbe157e28d03ad59c2abf9a1fb
  - 85c3f8df3eabc00f490cd392213c3b928a85aa44
2018-06-25 12:24:10 -07:00
Paul Banks ba0fb58a72 Make daemoinze an option on test binary without hacks. Misc fixes for racey or broken tests. Still failing on several though. 2018-06-25 12:24:09 -07:00
Kyle Havlovitz 74f225fcd2
Add client api support for CA config endpoints 2018-06-14 09:42:22 -07:00
Mitchell Hashimoto 3a6a750972
api: IntentionUpdate API 2018-06-14 09:42:20 -07:00
Mitchell Hashimoto f27711a319
api: IntentionDelete + tests 2018-06-14 09:42:18 -07:00
Mitchell Hashimoto 272211e171
command/intention/get: the get command without tests 2018-06-14 09:42:18 -07:00
Mitchell Hashimoto 988d7d984a
command/intention/finder: package for finding based on src/dst 2018-06-14 09:42:18 -07:00
Mitchell Hashimoto 961e9c1eaf
command/intention/create 2018-06-14 09:42:18 -07:00
Mitchell Hashimoto a5ecdc5798
api: IntentionCheck 2018-06-14 09:42:18 -07:00
Paul Banks 7c7c858a9a
More test cleanup 2018-06-14 09:42:16 -07:00
Paul Banks 4aeab3897c
Fixed many tests after rebase. Some still failing and seem unrelated to any connect changes. 2018-06-14 09:42:16 -07:00
Mitchell Hashimoto aaa2431350
agent: change connect command paths to be slices, not strings
This matches other executable configuration and allows us to cleanly
separate executable from arguments without trying to emulate shell
parsing.
2018-06-14 09:42:08 -07:00
Paul Banks e0e12e165b
TLS watching integrated into Service with some basic tests.
There are also a lot of small bug fixes found when testing lots of things end-to-end for the first time and some cleanup now it's integrated with real CA code.
2018-06-14 09:42:07 -07:00
Paul Banks cd88b2a351
Basic `watch` support for connect proxy config and certificate endpoints.
- Includes some bug fixes for previous `api` work and `agent` that weren't tested
 - Needed somewhat pervasive changes to support hash based blocking - some TODOs left in our watch toolchain that will explicitly fail on hash-based watches.
 - Integration into `connect` is partially done here but still WIP
2018-06-14 09:42:05 -07:00
Paul Banks 36dbd878c9
Adds `api` client code and tests for new Proxy Config endpoint, registering with proxy and seeing proxy config in /agent/services list. 2018-06-14 09:41:58 -07:00
Paul Banks 88541bba17
Add tests all the way up through the endpoints to ensure duplicate src/destination is supported and so ultimately deny/allow nesting works.
Also adds a sanity check test for `api.Agent().ConnectAuthorize()` and a fix for a trivial bug in it.
2018-06-14 09:41:57 -07:00
Mitchell Hashimoto 97f5414d94
api: rename Authorize field to ClientCertURI 2018-06-14 09:41:56 -07:00
Mitchell Hashimoto 9638466b88
api: fix up some comments and rename IssuedCert to LeafCert 2018-06-14 09:41:56 -07:00
Mitchell Hashimoto 4689d8373a
api: IntentionMatch 2018-06-14 09:41:56 -07:00
Mitchell Hashimoto 663a12d96b
api: starting intention endpoints, reorganize files slightly 2018-06-14 09:41:55 -07:00
Mitchell Hashimoto 263e2c7cf7
api: endpoints for working with CA roots, agent authorize, etc. 2018-06-14 09:41:55 -07:00
Paul Banks 125555e1aa
require -> assert until rebase 2018-06-14 09:41:54 -07:00
Paul Banks 9309422fd9
Add Connect agent, catalog and health endpoints to api Client 2018-06-14 09:41:54 -07:00
Kyle Havlovitz b73323aa42
Remove the script field from checks in favor of args 2018-05-08 15:31:53 -07:00
Pierre Souchay c152cb7bdf Added Missing Service Meta synchronization and field 2018-04-21 17:34:29 +02:00
Paul Banks 0d8993e338
Allow ignoring checks by ID when defining a PreparedQuery. Fixes #3727. 2018-04-10 14:04:16 +01:00
Preetha Appan c7581d68c6
Renames agent API layer for service metadata to "meta" for consistency 2018-03-28 09:04:50 -05:00
Preetha daa61c5803
Merge pull request #3881 from pierresouchay/service_metadata
Feature Request: Support key-value attributes for services
2018-03-27 16:33:57 -05:00
Josh Soref 94835a2715 Spelling (#3958)
* spelling: another

* spelling: autopilot

* spelling: beginning

* spelling: circonus

* spelling: default

* spelling: definition

* spelling: distance

* spelling: encountered

* spelling: enterprise

* spelling: expands

* spelling: exits

* spelling: formatting

* spelling: health

* spelling: hierarchy

* spelling: imposed

* spelling: independence

* spelling: inspect

* spelling: last

* spelling: latest

* spelling: client

* spelling: message

* spelling: minimum

* spelling: notify

* spelling: nonexistent

* spelling: operator

* spelling: payload

* spelling: preceded

* spelling: prepared

* spelling: programmatically

* spelling: required

* spelling: reconcile

* spelling: responses

* spelling: request

* spelling: response

* spelling: results

* spelling: retrieve

* spelling: service

* spelling: significantly

* spelling: specifies

* spelling: supported

* spelling: synchronization

* spelling: synchronous

* spelling: themselves

* spelling: unexpected

* spelling: validations

* spelling: value
2018-03-19 16:56:00 +00:00
Anubhav Mishra 0813175340
fix typo 2018-02-18 23:13:57 -08:00
Pierre Souchay b259b1609c Merge remote-tracking branch 'origin/master' into service_metadata 2018-02-11 13:20:49 +01:00
Kyle Havlovitz c3e94970a0
Add gRPC fields to client api and agent check endpoint docs 2018-02-09 16:02:27 -08:00
Pierre Souchay 80dde5465b Added support for Service Metadata 2018-02-07 01:54:42 +01:00
Veselkov Konstantin 7de57ba4de remove golint warnings 2018-01-28 22:40:13 +04:00
James Hartig ac225b2e7d Added CheckID and Name to AgentServiceCheck struct 2018-01-08 15:59:55 -05:00
James Phillips 53a881becc
Tightens up the API test verification. 2017-12-14 09:07:57 -08:00
James Phillips ca3f9024ac
Works around mapstructure behavior to enable sessions with no checks.
Fixes #3732
2017-12-14 09:07:56 -08:00
James Phillips 3177c2b450
Opens up the timing on the metrics test so it's less flaky. 2017-11-10 13:03:31 -08:00
James Phillips 93f68555d0
Adds enable_agent_tls_for_checks configuration option which allows (#3661)
HTTP health checks for services requiring 2-way TLS to be checked
using the agent's credentials.
2017-11-07 18:22:09 -08:00
James Phillips 4a2cafe525
Adds HTTP/2 support to Consul's HTTPS server. (#3657)
* Refactors the HTTP listen path to create servers in the same spot.

* Adds HTTP/2 support to Consul's HTTPS server.

* Vendors Go HTTP/2 library and associated deps.
2017-11-07 15:06:59 -08:00
James Phillips 172d5c0c40
Fetches metrics in the retry loop so there's some hope.
Fixes #3656
2017-11-06 21:43:39 -08:00
Kyle Havlovitz c9c083d843
Update AgentCheck struct in api 2017-11-01 15:30:29 -07:00
Kyle Havlovitz d3dd2b1402
Move check definition to a sub-struct 2017-11-01 14:54:46 -07:00
Kyle Havlovitz dbab3cd5f6
Merge branch 'master' into esm-changes 2017-11-01 11:37:48 -07:00
Kyle Havlovitz 021b2849c5
Remove redundant lines from coordinate test 2017-11-01 11:25:33 -07:00
Kyle Havlovitz b0536a96cc
Fill out the tests around coordinate/node functionality 2017-10-31 15:36:44 -07:00
Kyle Havlovitz a1d14019dd
Add tests around coordinate update endpoint 2017-10-26 20:12:54 -07:00
Kyle Havlovitz 2392545adc
Merge branch 'coordinate-node-endpoint' of github.com:hashicorp/consul into esm-changes 2017-10-26 19:20:24 -07:00
Kyle Havlovitz 5589eadcf5
Added Coordinate.Node rpc endpoint and client api method 2017-10-26 19:16:40 -07:00
Kyle Havlovitz ce4e8c46fa
Add deregister critical service field and refactor duration parsing 2017-10-25 19:17:41 -07:00
James Phillips 21f38bff6f
Adds retry to API metrics test (flaky test). 2017-10-25 10:46:02 -07:00
Kyle Havlovitz 291455f475
Added coordinate update http endpoint 2017-10-25 19:37:30 +02:00
Kyle Havlovitz d56936e27a
Added remaining HTTP health check fields to structs 2017-10-25 19:37:30 +02:00
Kyle Havlovitz a7c42a6c2a
Expose SkipNodeUpdate field and some health check info in the http api 2017-10-25 19:37:30 +02:00
Ryan Slade dcb392df46 Simplify string(buf.Bytes()) to buf.String() (#3590) 2017-10-18 13:26:09 -07:00
James Phillips 53f67c3993 Fixes API client for ScriptArgs and updates documentation. (#3589)
* Updates the API client to support the current `ScriptArgs` parameter
for checks.

* Updates docs for checks to explain the `ScriptArgs` parameter issue.

* Adds mappings for "args" and "script-args" to give th API parity
with config.

* Adds checks on return codes.

* Removes debug logging that shows empty when args are used.
2017-10-18 11:28:39 -07:00
Ryan Slade 85e4aea9d1 Replace time.Now().Sub(x) with time.Since(x) 2017-10-17 20:38:24 +02:00
James Phillips 575d70aaa7
Cleans up some drift between the OSS and Enterprise trees. 2017-10-11 15:53:07 -07:00
James Bardin fea32fb7d7 retry locks on network errors (#3553)
* retry locks on network errors

When communicating with a local agent and watching a lock, a dropped
connection between the agent and server will show up as a server error
and immediately be retried. However if the client is connected to a
remote server, a dropped connection immediately aborts the lock.

* Updates comment about it being unsafe for writes.
2017-10-11 07:42:10 -07:00
Artiom Diomin 2e229617ce Fix example code formatting in godoc 2017-10-11 14:52:22 +02:00
Frank Schröder ce887a0c45 Provide stable config for agent/self (#3532)
* config: provide stable config for /v1/agent/self (#3530)

This patch adds a stable subset of the previous Config struct to the
agent/self response. The actual runtime configuration is moved into
DebugConfig and will be documented to change.

Fixes #3530

* config: fix tests

* doc: update api documentation for /v1/agent/self
2017-10-04 10:43:17 -07:00
Frank Schröder 21118cafeb Recursive sanitize (#3505)
* vendor: add github.com/sergi/go-diff/diffmatchpatch for diff'ing test output

* config: refactor Sanitize to recursively clean runtime config and format complex fields

* Removes an extra int cast.

* Adds a top-level check test case for sanitization.
2017-09-27 11:47:40 -07:00
Preetha Appan 3c4a108769 Move Raft protocol version for list peers end point to server side, fix unit tests. This fixes #3449 2017-09-26 09:35:39 -05:00
Preetha Appan 00840929ea Fix import order 2017-09-25 18:06:49 -05:00
preetapan 73951d8319 Merge pull request #3494 from hashicorp/enforce_json_extension
Enforce json or hcl extension to Consul config files, updated unit tests
2017-09-25 17:30:33 -05:00
James Phillips 45646ac3f4 Bumps default Raft protocol to version 3. (#3477)
* Changes default Raft protocol to 3.

* Changes numPeers() to report only voters.

This should have been there before, but it's more obvious that this
is incorrect now that we default the Raft protocol to 3, which puts
new servers in a read-only state while Autopilot waits for them to
become healthy.

* Fixes TestLeader_RollRaftServer.

* Fixes TestOperator_RaftRemovePeerByAddress.

* Fixes TestServer_*.

Relaxed the check for a given number of voter peers and instead do
a thorough check that all servers see each other in their Raft
configurations.

* Fixes TestACL_*.

These now just check for Raft replication to be set up, and don't
care about the number of voter peers.

* Fixes TestOperator_Raft_ListPeers.

* Fixes TestAutopilot_CleanupDeadServerPeriodic.

* Fixes TestCatalog_ListNodes_ConsistentRead_Fail.

* Fixes TestLeader_ChangeServerID and adjusts the conn pool to throw away
sockets when it sees io.EOF.

* Changes version to 1.0.0 in the options doc.

* Makes metrics test more deterministic with autopilot metrics possible.
2017-09-25 15:27:04 -07:00
Preetha Appan a4331a480c Fix one more unit test that created a config file 2017-09-25 17:26:49 -05:00
Frank Schröder 12216583a1 New config parser, HCL support, multiple bind addrs (#3480)
* new config parser for agent

This patch implements a new config parser for the consul agent which
makes the following changes to the previous implementation:

 * add HCL support
 * all configuration fragments in tests and for default config are
   expressed as HCL fragments
 * HCL fragments can be provided on the command line so that they
   can eventually replace the command line flags.
 * HCL/JSON fragments are parsed into a temporary Config structure
   which can be merged using reflection (all values are pointers).
   The existing merge logic of overwrite for values and append
   for slices has been preserved.
 * A single builder process generates a typed runtime configuration
   for the agent.

The new implementation is more strict and fails in the builder process
if no valid runtime configuration can be generated. Therefore,
additional validations in other parts of the code should be removed.

The builder also pre-computes all required network addresses so that no
address/port magic should be required where the configuration is used
and should therefore be removed.

* Upgrade github.com/hashicorp/hcl to support int64

* improve error messages

* fix directory permission test

* Fix rtt test

* Fix ForceLeave test

* Skip performance test for now until we know what to do

* Update github.com/hashicorp/memberlist to update log prefix

* Make memberlist use the default logger

* improve config error handling

* do not fail on non-existing data-dir

* experiment with non-uniform timeouts to get a handle on stalled leader elections

* Run tests for packages separately to eliminate the spurious port conflicts

* refactor private address detection and unify approach for ipv4 and ipv6.

Fixes #2825

* do not allow unix sockets for DNS

* improve bind and advertise addr error handling

* go through builder using test coverage

* minimal update to the docs

* more coverage tests fixed

* more tests

* fix makefile

* cleanup

* fix port conflicts with external port server 'porter'

* stop test server on error

* do not run api test that change global ENV concurrently with the other tests

* Run remaining api tests concurrently

* no need for retry with the port number service

* monkey patch race condition in go-sockaddr until we understand why that fails

* monkey patch hcl decoder race condidtion until we understand why that fails

* monkey patch spurious errors in strings.EqualFold from here

* add test for hcl decoder race condition. Run with go test -parallel 128

* Increase timeout again

* cleanup

* don't log port allocations by default

* use base command arg parsing to format help output properly

* handle -dc deprecation case in Build

* switch autopilot.max_trailing_logs to int

* remove duplicate test case

* remove unused methods

* remove comments about flag/config value inconsistencies

* switch got and want around since the error message was misleading.

* Removes a stray debug log.

* Removes a stray newline in imports.

* Fixes TestACL_Version8.

* Runs go fmt.

* Adds a default case for unknown address types.

* Reoders and reformats some imports.

* Adds some comments and fixes typos.

* Reorders imports.

* add unix socket support for dns later

* drop all deprecated flags and arguments

* fix wrong field name

* remove stray node-id file

* drop unnecessary patch section in test

* drop duplicate test

* add test for LeaveOnTerm and SkipLeaveOnInt in client mode

* drop "bla" and add clarifying comment for the test

* split up tests to support enterprise/non-enterprise tests

* drop raft multiplier and derive values during build phase

* sanitize runtime config reflectively and add test

* detect invalid config fields

* fix tests with invalid config fields

* use different values for wan sanitiziation test

* drop recursor in favor of recursors

* allow dns_config.udp_answer_limit to be zero

* make sure tests run on machines with multiple ips

* Fix failing tests in a few more places by providing a bind address in the test

* Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder.

* Add porter to server_test.go to make tests there less flaky

* go fmt
2017-09-25 11:40:42 -07:00
James Phillips 00605c0214
Shows the segment name in the keyring API and command output. 2017-09-07 12:17:39 -07:00
James Phillips 1a117ba0a8
Makes the all segments query explict, and the default for `consul members`. 2017-09-05 12:22:20 -07:00
Kyle Havlovitz 20296d4a89
Change segment list endpoint in docs/client api 2017-09-01 12:40:07 -07:00
Kyle Havlovitz 62102a537e
Organize segments for a cleaner split between enterprise and OSS 2017-08-31 17:39:46 -07:00
Kyle Havlovitz 21513b0393
Update coord display in ui to account for segments 2017-08-30 11:58:29 -07:00
Kyle Havlovitz 2ada0439d4
Add rpc_listener option to segment config 2017-08-30 11:58:29 -07:00
James Phillips b1a15e0c3d
Adds open source side of network segments (feature is Enterprise-only). 2017-08-30 11:58:29 -07:00
Illirgway e0cc1ce679 Fix bug with unused (replaced with "") CONSUL_HTTP_AUTH in some places
example: https://github.com/hashicorp/consul/blob/master/watch/plan.go#L26

	conf := consulapi.DefaultConfig()
	conf.Address = address
	conf.Datacenter = p.Datacenter
	conf.Token = p.Token                             # <-- replace Token from DefaultConfig/CONSUL_HTTP_AUTH with ""
	client, err := consulapi.NewClient(conf)

how to reproduce bug:
0. consul -> localhost:8500 with more than 0 service checks
1. deny all for anonymous token
2. create appropriate acl <token> for watch checks (agent:read + node:read,service:read)
3. bash:
CONSUL_HTTP_AUTH=<token> consul watch -http-addr=localhost:8500 -type=checks # --> return []
consul watch -http-addr=localhost:8500 -type=checks -token=<token> # -> return { .... right json result .... }
2017-08-16 01:51:18 +03:00
James Phillips 6ccf724c0c
Adds a guard to make sure that empty log lines don't close consul monitor.
Fixes #3253.
2017-08-08 16:05:29 -07:00
Kyle Havlovitz 5b998cacb1
Update api structs for metrics endpoint 2017-08-08 12:33:59 -07:00
Kyle Havlovitz d5634fe2a8
Add support for labels/filters from go-metrics 2017-08-08 01:45:10 -07:00
James Phillips 4bee2e49f5 Adds secure introduction for the ACL replication token. (#3357)
Adds secure introduction for the ACL replication token, as well as a separate enable config for ACL replication.
2017-08-03 15:39:31 -07:00
James Phillips c0a5ad7903 Adds a new /v1/acl/bootstrap API (#3349) 2017-08-02 17:05:18 -07:00
James Phillips 496b0bcf07 Adds support for agent-side ACL token management via API instead of config files. (#3324)
* Adds token store and removes all runtime use of config for ACL tokens.
* Adds a new API for changing agent tokens on the fly.
2017-07-26 11:03:43 -07:00
Kyle Havlovitz 19eae3d14b
Add UpgradeVersionTag to autopilot config 2017-07-18 13:35:41 -07:00
James Phillips 1791d99a10 Adds new config to make script checks opt-in, updates documentation. (#3284) 2017-07-17 11:20:35 -07:00
James Phillips 8d0a048408 Expands and rework context support in the API client. (#3273) 2017-07-14 17:30:08 -07:00
James Bardin deb43fea6b accept recv-only channels for cancellations (#3271)
Cancellation channels are often derived from a Context, which
returns a directional `<-chan struct{}` from Done(). In order to use
this with parts of of the consul API, one is required to create a new
channel and dispatch a separate goroutine to watch for context
cancellation and close the new channel.

Changing the signature for the methods that take cancellation channels
will allow easier integration with existing uses of Context. Since the
cancellation pattern only reads from these channels, there should be no
backwards incompatibility with existing codebases, and most of the
methods already accept only the correct type.
2017-07-14 16:31:44 -07:00
Frank Schroeder 9c54f30c94 api: fix TestAPI_CatalogNodes test 2017-07-07 09:22:34 +02:00
Frank Schroeder 0e931a713c agent: fix failing test from localState decoupling
Would be nice to know what changed though.
2017-07-07 09:22:34 +02:00
Frank Schroeder 7bbfef7000 api: refactor: unify naming of API tests 2017-07-07 09:22:34 +02:00
Frank Schroeder 8edeabce9a api: refactor: prefix all API tests with API_ 2017-07-07 09:22:34 +02:00
Preetha Appan 9eaf56bfe3 Fix socket file handle leaks from old blocking queries upon consul reload. This fixes issue #3018 2017-06-26 15:52:03 -05:00
Frank Schröder 825f72f5ef agent: support custom header and method for http checks (#3106)
This patch adds support for custom headers and
method for HTTP checks.

Fixes #2474
Fixes #2657
Fixes #3106
2017-06-07 01:11:56 +02:00
James Phillips 3baa703d02 Merge pull request #3077 from jwasinger/issue-2850
Prevent Session.Destroy from leaving open TCP connections
2017-06-02 21:43:55 -07:00
Jared Wasinger d2ef61d079 Prevent Session.Destroy from leaving open TCP connections 2017-05-25 20:51:27 -07:00
Kyle Havlovitz 4e44db87e3
Don't overwrite Transport's TLS config if it's been set 2017-05-24 13:45:19 -07:00
Kyle Havlovitz 04f1ff78f8 Remove superfluous default HttpClient check in NewClient 2017-05-24 11:40:21 -07:00
Frank Schroeder 96c677fdba test: make test less flaky 2017-05-12 22:12:47 +02:00
Frank Schroeder 65b5c51ec7 test: add helper for ioutil.TempDir/TempFile
This creates a simplified helper for temporary directories and files.
All path names are prefixed with the name of the current test.
All files and directories are stored either in /tmp/consul-test
or /tmp if the former could not be created.

Using the system temp dir breaks some tests on macOS where the unix
socket path becomes too long.
2017-05-12 22:12:47 +02:00
Frank Schroeder 3522a41bad test: include test name in temp file/dir
This helps identifying hanging tests by looking
at the process list.
2017-05-12 22:12:47 +02:00
James Phillips c1d196152f
Fixes vet errors. 2017-05-09 06:48:52 -07:00
James Phillips ddfa57765c
retry: Removes the description parameter. 2017-05-05 17:07:03 +02:00
Frank Schroeder 21a82a0a16
test: Refactor WaitForResult tests with retry
Refactor tests that use testutil.WaitForResult to use retry.

Since this requires refactoring the test functions in general this patch
also shows the use of the github.com/pascaldekloe/goe/verify library
which provides a good mechanism for comparing nested data structures.
Instead of just converting the tests from testutil.WaitForResult to
retry the tests that performing a nested comparison of data structures
are converted to the verify library at the same time.
2017-05-05 17:07:02 +02:00
Kyle Havlovitz cd56a5ebdd Add separate option for verifying incoming HTTPS traffic (#2974)
* Add separate option for verifying incoming HTTPS traffic
2017-04-28 16:15:55 -07:00
Frank Schroeder 9685bdcd0b api: Add ServiceTags to Health state endpoint (#153)
This patch adds the ServiceTags to the /v1/health/state/<state>
endpoint.

Fixes #153
2017-04-28 15:00:08 -07:00
James Phillips 05b1cabba9 Merge pull request #2947 from bogdanov1609/added_ACLReplication
API: Add ACLReplication
2017-04-25 16:50:52 -07:00
James Phillips 36725cef11 Tweaks ACL replication struct name. 2017-04-25 16:32:09 -07:00
James Phillips 138abeb5f7 Tweaks ACL replication status struct name. 2017-04-25 16:31:20 -07:00
Frank Schroeder 3e00e36f41 golint: Untangle if blocks with return in else 2017-04-25 09:26:13 -07:00
Ivan Bogdanov 1ebd6402fd API: Add ACLReplication 2017-04-25 00:39:50 +03:00
Frank Schröder 6dd2804d09 Issue #2905: Add check-not-exists to TXN endpoint
This patch adds support for asserting that a given
key does not exist in the KV store.

Fixes #2905
2017-04-20 17:50:52 -07:00
Kyle Havlovitz c6d0b4cf90 Make the API client's httpClient more pluggable (#2926) 2017-04-18 16:39:23 -07:00
Frank Schröder c40621ae10 Add datacenter to catalog node API (#2917)
This patch stores the datacenter of a node in memdb
and exposes it via the API in all places where a Node
structure is returned.

 * /catalog/nodes
 * /catalog/node/:node
 * /catalog/service/:service
 * /health/service/:service
 * /query/:uuid/execute
2017-04-18 05:02:24 -07:00
Kyle Havlovitz ae6bf56ee1
Add tls client options to api/cli 2017-04-14 13:37:29 -07:00
Brian Shumate f499584c38
API: Add struct fields, fixes #2882
- Add CreateIndex and ModifyIndex to AgentService
- Add CreateIndex and ModifyIndex to Node
2017-04-13 11:36:19 -04:00
James Phillips 08f81ebddb
Moves operator sub-functions into their own files. 2017-03-30 12:35:50 -07:00
Kyle Havlovitz da9c825592
Add CLI/API endpoints for removing peer by ID 2017-03-30 10:13:32 -07:00
Seth Vargo 0170a28ec9
Use new APIs 2017-03-23 18:48:13 -04:00
Kyle Havlovitz 390f41d8d5
Add advanced autopilot features 2017-03-22 15:25:16 -07:00
James Phillips 3f3889022e
Tweaks the agent leave test to patch an occasional local failure. 2017-03-22 09:56:53 -07:00
James Phillips 7d011d7d77
Adds missing area API. 2017-03-21 23:11:15 -07:00
James Phillips e91377fa26
Cleans up based on review feedback. 2017-03-20 16:23:40 -07:00
James Phillips 3a0739cdfc
Tweaks the join response to be more useful. 2017-03-16 21:01:03 -07:00
James Phillips 79e192fd07
Adds public API for the network area endpoints. 2017-03-16 16:42:19 -07:00
James Phillips 472f1bd63e
Adds router into RPC paths with work in progress on coordinates. 2017-03-16 16:42:18 -07:00
Kyle Havlovitz 5353221666
Reorganized cluster health check loop and logic 2017-03-15 18:27:17 -07:00
Kyle Havlovitz 51b11cd344
Fix an issue with changing server IDs and add a few UX enhancements around autopilot features 2017-03-15 16:09:55 -07:00
Kyle Havlovitz 9b4497de09
Cleaned up and reorganized some autopilot-related code 2017-03-09 18:21:40 -08:00
Kyle Havlovitz 2eefe3ca5b
Add autopilot server health tracking
This adds two goroutines to perform autopilot tasks on the leader - one
to monitor the health of servers and another to periodically clean up
dead servers with a limit on removal count. Also adds a new http endpoint,
`/v1/operator/autopilot/health`, for querying this information through an
operator RPC endpoint.
2017-03-06 16:00:10 -08:00
Kyle Havlovitz 92c8b9c3a0
Rename DeadServerCleanup and make wording adjustments 2017-02-28 14:45:21 -08:00
Kyle Havlovitz c1f776c78b
Added operator autopilot subcommands 2017-02-24 15:54:49 -08:00
Kyle Havlovitz c2e7f45002
Add CAS capability to autopilot config endpoint 2017-02-24 13:08:49 -08:00
Kyle Havlovitz 1c24c5d8d6
Add docs and api client methods for autopilot config 2017-02-23 21:00:15 -08:00
Jeff Mitchell 44733caf5c Update unix dial functions to use DialContext with new go-cleanhttp 2017-02-10 21:11:21 -05:00
Kyle Havlovitz c9abafab92
Update docs and give better error for unknown client scheme 2017-02-10 19:55:54 -05:00
Kyle Havlovitz a7a6ee9f4c
Allow prefixing -http-addr with http/https schemes 2017-02-10 18:25:46 -05:00
Kyle Havlovitz bad037fb58
Cleanup and formatting adjustments 2017-02-09 20:49:17 -05:00
Kyle Havlovitz abdf1fbab3
Convert keyring command to use base.Command 2017-02-08 18:25:47 -05:00
James Phillips 585958746a
Lets the leave unit test proceed even after an EOF. 2017-01-26 22:22:18 -08:00
Kyle Havlovitz a55968f009
Merge branch 'master' into f-prepared-query-nodemeta 2017-01-23 20:17:48 -05:00
Kyle Havlovitz 3f3d7f9891
Add tests for node meta in prepared queries and update docs 2017-01-23 19:17:30 -05:00
James Phillips 6ca0173907
Adds catalog support for node IDs. 2017-01-18 14:26:42 -08:00
James Phillips bd605e330c
Adds basic support for node IDs. 2017-01-17 22:47:59 -08:00
Kyle Havlovitz 5acd69b4fc
Add node metadata filtering to remaining health/catalog endpoints 2017-01-13 20:08:43 -05:00
Kyle Havlovitz 023b6533e0
Fix formatting 2017-01-11 20:44:22 -05:00
Kyle Havlovitz 87c0283bb1
Update client api and docs for node metadata 2017-01-11 19:28:58 -05:00
James Phillips 0ed6b1bb18
Bans anonymous queries that aren't tied to a session.
This gets us coverage of PQ creation under the existing service
policy or the soon-to-be-added session policy.
2016-12-12 14:52:27 -08:00
James Phillips 0282dd908a Merge pull request #2531 from alicebob/txntypes
fix KVOp types
2016-12-01 07:41:10 -08:00
Kyle Havlovitz bd69c6d871 Add reload/leave http endpoints (#2516) 2016-11-30 13:29:42 -05:00
Seth Vargo f7958983e4
Return the correct type 2016-11-29 21:17:00 -05:00
Seth Vargo 7520ce2d49 Merge pull request #2544 from hashicorp/sethvargo/best_status
Add an API method for determining the best status
2016-11-29 19:07:52 -05:00
James Phillips d079e62d57 Merge pull request #2530 from coffeehc/master
adapt to server's checker field TLSSkipVerify type
2016-11-29 16:00:21 -08:00
Seth Vargo 190249424b
Do not log 2016-11-29 18:55:34 -05:00
Seth Vargo 4179aacf11
Add an API method for determining the best status
Given a list of HealthChecks, this determines the "best" status for the
collective group. This is useful for nodes and services, which may have
multiple checks associated with them.
2016-11-29 18:41:46 -05:00
Kyle Havlovitz 8079c49c0a Add QueryOptions to api package's monitor 2016-11-28 18:36:26 -05:00
Kyle Havlovitz 124f907063 Add monitor http endpoint 2016-11-28 18:36:26 -05:00
Harmen 92b2369f82 fix KVOp types 2016-11-24 20:43:41 +01:00
James Phillips a5baa091c8
Runs `go fmt`. 2016-11-22 17:49:48 -08:00
Kyle Havlovitz dcdadd012f Fix keyring doc method wording 2016-11-22 20:10:43 -05:00
Kyle Havlovitz 6bd65c668b Combine keyring endpoints into one 2016-11-22 20:10:43 -05:00
Kyle Havlovitz dd3368c19e Add keyring http endpoints 2016-11-22 20:10:43 -05:00
coffee a5e08845c2 adapt to server's checker field TLSSkipVerify type 2016-11-21 18:02:38 +08:00
James Phillips 6ce295e0de Merge pull request #2366 from aksentyev/master
api client: CatalogService added fields CreateIndex, ModifyIndex
2016-11-17 16:38:39 -08:00
James Phillips ad25acaf6a Moves index fields to the bottom of the structure. 2016-11-17 16:38:30 -08:00
James Phillips 9a81d402b7
Adds notes field to API.
Closes #2336.
2016-11-17 16:33:50 -08:00
James Phillips c744792fc4 Merge pull request #2238 from hasyimibhar/master
Remove duplicated environment variables
2016-11-17 16:18:02 -08:00
Kyle Havlovitz 8c3db1c5c6 Trim leading slash on key to avoid redirect (golang/go#4800) (#2476) 2016-11-04 21:55:10 -07:00
Kyle McCullough 73b281a27c Add setting to skip ssl certificate verification for HTTP checks (#1984)
* http check: add setting to skip ssl certificate verification

* update http check documentation

* fix typo in documentation

* Add TLSSkipVerify to agent api
2016-11-03 13:17:30 -07:00
Mike Cowgill cbf2874464 Delete prepared query using WriteOptions (#2417) 2016-11-03 15:54:07 -04:00
Kyle Havlovitz dce6702268 More flaky unit test fixes (#2449)
* More flaky unit test fixes
* Raise some test timeouts that were too low
2016-10-31 09:59:20 -07:00
James Phillips c01a3871c9 Adds support for snapshots and restores. (#2396)
* Updates Raft library to get new snapshot/restore API.

* Basic backup and restore working, but need some cleanup.

* Breaks out a snapshot module and adds a SHA256 integrity check.

* Adds snapshot ACL and fills in some missing comments.

* Require a consistent read for snapshots.

* Make sure snapshot works if ACLs aren't enabled.

* Adds a bit of package documentation.

* Returns an empty response from restore to avoid EOF errors.

* Adds API client support for snapshots.

* Makes internal file names match on-disk file snapshots.

* Adds DC and token coverage for snapshot API test.

* Adds missing documentation.

* Adds a unit test for the snapshot client endpoint.

* Moves the connection pool out of the client for easier testing.

* Fixes an incidental issue in the prepared query unit test.

I realized I had two servers in bootstrap mode so this wasn't a good setup.

* Adds a half close to the TCP stream and fixes panic on error.

* Adds client and endpoint tests for snapshots.

* Moves the pool back into the snapshot RPC client.

* Adds a TLS test and fixes half-closes for TLS connections.

* Tweaks some comments.

* Adds a low-level snapshot test.

This is independent of Consul so we can pull this out into a library
later if we want to.

* Cleans up snapshot and archive and completes archive tests.

* Sends a clear error for snapshot operations in dev mode.

Snapshots require the Raft snapshots to be readable, which isn't supported
in dev mode. Send a clear error instead of a deep-down Raft one.

* Adds docs for the snapshot endpoint.

* Adds a stale mode and index feedback for snapshot saves.

This gives folks a way to extract data even if the cluster has no
leader.

* Changes the internal format of a snapshot from zip to tgz.

* Pulls in Raft fix to cancel inflight before a restore.

* Pulls in new Raft restore interface.

* Adds metadata to snapshot saves and a verify function.

* Adds basic save and restore snapshot CLI commands.

* Gets rid of tarball extensions and adds restore message.

* Fixes an incidental bad link in the KV docs.

* Adds documentation for the snapshot CLI commands.

* Scuttle any request body when a snapshot is saved.

* Fixes archive unit test error message check.

* Allows for nil output writers in snapshot RPC handlers.

* Renames hash list Decode to DecodeAndVerify.

* Closes the client connection for snapshot ops.

* Lowers timeout for restore ops.

* Updates Raft vendor to get new Restore signature and integrates with Consul.

* Bounces the leader's internal state when we do a restore.
2016-10-25 19:20:24 -07:00
Kyle Havlovitz f6c971f46c Fix race condition in TestClient_WatchList and TestClient_WatchGet 2016-10-25 17:48:11 -07:00
James Phillips 554c898338 Merge pull request #2382 from zaunerc/master
Add info about return values for function Get().
2016-10-20 09:40:58 -07:00
James Phillips 556ffc588b Adds comment about SessionOpts. 2016-10-05 17:53:27 -07:00
James Phillips 25455c97c3 Merge pull request #2372 from HotelsDotCom/flexible-lock-sessions
More flexible Session configuration when using api.Lock
2016-10-05 17:52:12 -07:00
Christoph Zauner faf85f602b Add info about return values for function Get(). 2016-10-03 08:24:04 +00:00
Maxim Ivanov 2ed58a2747 More flexible Session configuration when using api.Lock 2016-09-29 14:07:54 +01:00
aksentyev 4f96fa0aea CreateIndex, ModifyIndex added for CatalogService 2016-09-28 00:40:25 +03:00
Seth Vargo 40de5858d3
Fix typo 2016-09-26 16:06:56 -07:00
Seth Vargo 55f1b4ac44
Rename session name to session ID 2016-09-26 16:06:55 -07:00
Seth Vargo 12e46503f7
Update ModifyIndex comment 2016-09-26 16:06:55 -07:00
Seth Vargo 227019ec59
Add documentation to KVPair
Based on discussions with @slackpad, some of the values in the KVPair are
ReadOnly. This commit updates the docs to reflect that.
2016-09-26 16:06:52 -07:00
James Phillips 84e8fc5848
Removes Raft types from public API interface.
This will cause a lot of breakage because we've currently vendored a
branch of the Raft library.
2016-08-30 14:59:16 -07:00
James Phillips 3c9188c38b
Makes the Raft configuration API easier to consume. 2016-08-30 11:30:56 -07:00
James Phillips 0558b9c44b
Fixes a stale comment. 2016-08-30 00:10:46 -07:00
James Phillips e5850d8a26
Adds new consul operator endpoint, CLI, and ACL and some basic Raft commands. 2016-08-30 00:02:50 -07:00
James Phillips defa2a6180 Merge pull request #2226 from abhinavdahiya/rm-health-unknown
Fixes #1775; Removes 'unknown' state
2016-08-17 17:51:04 -07:00
James Phillips 4c7a0ed3b0
Merge branch 'master' into f-deregister-critical 2016-08-16 12:53:21 -07:00
James Phillips 9f7a973ace
Adds an `X-Consul-Translate-Addresses` to signal translation is enabled. 2016-08-16 11:31:41 -07:00
James Phillips 1d5eb36689
Adds missing TaggedAddress structures to API client. 2016-08-16 10:30:30 -07:00
James Phillips 231f5a957f
Fixes a typo and adds an admonition about only being in Consul 0.7+. 2016-08-16 09:27:20 -07:00
James Phillips 4a3d7db24f
Adds ability to deregister a service based on critical check state longer than a timeout. 2016-08-16 01:00:26 -07:00
Hasyimi Bahrudin 105cea9bae Add api environment variables as constants for consistency 2016-08-03 14:40:31 +08:00
Hasyimi Bahrudin bb6bb19aad Remove redundant hardcoded environment variables
The following hardcoded environment variables are removed:

* CONSUL_RPC_ADDR
* CONSUL_HTTP_ADDR
2016-08-03 14:40:26 +08:00
Jeff Mitchell f1a67836d8 Use header to send Consul token rather than query param. 2016-08-02 16:54:59 -04:00
Abhinav Dahiya 7c6c3b38bc Fixes #1775; Removes 'unknown' state
Signed-off-by: Abhinav Dahiya <abhinavdtu2012@gmail.com>
2016-07-30 19:33:14 +05:30
Ryan Uber 499537addf api: add query templates 2016-07-02 16:05:41 -07:00
Ryan Uber 72b43eba26 api: add Near parameter to PQ's 2016-07-01 16:41:46 -07:00
James Phillips 778b975e7a Adds a get-tree verb to KV transaction operations. 2016-05-13 16:57:39 -07:00
James Phillips a37bf9de56 Adds a read-only optimized path for transactions. 2016-05-13 00:34:05 -07:00
James Phillips 04d99cd702 Makes get fail a transaction if the key doesn't exist. 2016-05-11 14:18:31 -07:00
James Phillips 4882a9fe43 De-nests the KV output structure (removes DirEnt member). 2016-05-11 13:48:03 -07:00
James Phillips 960b9d6fb6 Switches to "KV" instead of "KV" for the KV operations. 2016-05-11 10:58:27 -07:00
James Phillips 38d0f6676f Refactors TxnRequest/TxnResponse into a form that will allow non-KV ops.
This isn't needed/used yet, but it's a good hook to get in there so we
can add more atomic operations in the future. The Go API hides this detail
so that feels like a KV-specific API. The implications on the REST API are
pretty minimal.
2016-05-11 01:39:10 -07:00
James Phillips 69f58ad04a Moves txn code into a new endpoint, not specific to KV. 2016-05-10 21:58:02 -07:00
James Phillips 44ab1aa418 Adds type for API ops and an example transaction. 2016-05-10 13:36:48 -07:00
James Phillips e491245062 Performs basic plumbing of KVS transactions through all the layers. 2016-05-09 22:15:49 -07:00
Sean Chittenden 4255a0826d Correct a small typo 2016-04-23 20:18:19 -07:00
Sean Chittenden f1873c21d7 consul/ uses structs.Health*, the api uses api.Health* 2016-04-23 16:06:58 -07:00
Sean Chittenden e63d3a1275 Update Check API to use constants
Use constants where appropriate to advocate their use.  Also add a deprecation notice re: `updateTTL`.
2016-04-23 16:01:59 -07:00
James Phillips 3340d7ccd7 Merge pull request #1876 from hashicorp/f-tls-helper
Adds TLS config helper to API client.
2016-03-24 11:34:24 -07:00
James Phillips 0eb6279c5e Improves the comment for the Address field. 2016-03-24 11:33:44 -07:00
Diptanu Choudhury 4811a72d80 Added some constants in the api for check health statuses 2016-03-24 11:26:07 -07:00
James Phillips 1cfed981b3 Adds TLS config helper to API client. 2016-03-24 11:24:18 -07:00
James Phillips abacccd6e5 Switches default for API client to pooled connections. 2016-03-10 12:29:50 -08:00
James Phillips 23247b3699 Adds support for new PUT API for checks. 2016-03-04 15:18:25 -08:00
James Phillips afdeb2f1fc Adds support for EnableTagOverride to the API client. 2016-02-16 11:45:29 -08:00
Seth Vargo cae0ff354a Use HTTPS + www. where appropriate 2016-01-13 17:44:01 -05:00
James Phillips 58fb27e70f Factors server error checking into a new function. 2016-01-06 11:35:16 -08:00
James Phillips c048c5aca6 Makes the API behave better with small wait values. 2016-01-06 11:05:11 -08:00
James Phillips 49342dc973 Makes the timeout behavior more intuitive.
Previously, it would try once "up to" the timeout, but in practice it would
just fall through. This modifies the behavior to block until the timeout has
been reached.
2016-01-06 09:40:20 -08:00
James Phillips ca08ba3aee Adds the ability for semaphore monitors to ride out brief periods of 500 errors. 2016-01-05 18:22:36 -08:00
James Phillips 8caa9e4c7e Adds "try" support to locks and semaphores. 2016-01-05 18:22:36 -08:00
James Phillips ebff3fe0d0 Builds the insecure SSL transport off the clean default transport. 2015-12-17 12:15:18 -08:00
Jeff Mitchell 4604af6aa5 Revert "Adds client and transport pooling in the API so we don't leak connections." 2015-12-17 10:56:50 -05:00
James Phillips f6546a0e46 Makes the insecure transport work like the default one. 2015-12-17 06:48:18 -08:00
James Phillips d57e60a123 Adds client and transport pooling in the API so we don't leak connections. 2015-12-16 22:27:07 -08:00
James Phillips 380658f6f1 Adds a unit test for lock monitor retries. 2015-11-30 20:16:40 -08:00
James Phillips c2a8fee76c Adds custom retry time for lock monitors. 2015-11-30 20:16:22 -08:00
James Phillips 86e6230de6 Adds a retry capability to lock monitors in the API client. 2015-11-30 17:09:41 -08:00
James Phillips 87ccd1eb34 Updates API docs for version 0.6.0. 2015-11-28 20:04:29 -08:00
Craig Wickesser 7890a3dda8 Fix version support
Updated the version that the library supports.
2015-11-23 13:03:38 -05:00
James Phillips 95c708f65e Adds Docker checks support to client API.
Also changed `DockerContainerId` to `DockerContainerID`, and updated the agent
API docs to reflect their support for Docker checks.
2015-11-18 07:40:02 -08:00
James Phillips 1059a8b3a3 Removes a useless empty import and fixes some stale comments. 2015-11-17 08:29:20 -08:00
James Phillips 8fc6a6a986 Switches to helpers for prepared query API wrappers. 2015-11-16 23:12:44 -08:00
James Phillips 712a3dba2f Adds API client support for prepared queries. 2015-11-15 17:06:00 -08:00
James Phillips 787f946f8d Adds support for coordinates to client API. 2015-10-23 15:23:01 -07:00
Mikhail Dyakov 01aba571a1 Handle error in api README example 2015-10-23 21:20:01 +02:00
Jeff Mitchell 1e3840b044 Update cleanhttp repo location 2015-10-22 14:14:22 -04:00
Jeff Mitchell 9a5fd5424a Use cleanhttp to get rid of DefaultTransport 2015-10-22 10:47:50 -04:00
James Phillips 2c36c4f969 Fixes a typo. 2015-10-15 14:59:09 -07:00
James Phillips 181c216f53 Fixes index management for KVS. 2015-10-15 14:59:09 -07:00
Jeff Mitchell f49fc095ef Don't use http.DefaultClient
Two of the changes are in tests; the one of consequence is in the API.
As explained in #1308 this can cause conflicts with downstream programs.

Fixes #1308.
2015-10-15 17:49:35 -04:00
James Phillips c74355fec0 Merge pull request #1041 from rboyer/api-fix-session-renew
Fix Session.Renew to care about 404s properly
2015-09-25 13:15:54 -07:00
Dale Wijnand 5a28ebcaa3 Fix a bunch of typos. 2015-09-15 13:22:08 +01:00
R.B. Boyer 8b072467eb Correct the Session.Renew{,Periodic} to handle session expiration better 2015-09-14 08:52:32 -05:00
Anthony Scalisi 10e028d599 remove various typos 2015-09-11 12:29:54 -07:00
Peter Fern b023904298 Add TCP check type
Adds the ability to simply check whether a TCP socket accepts
connections to determine if it is healthy.  This is a light-weight -
though less comprehensive than scripting - method of checking network
service health.

The check parameter `tcp` should be set to the `address:port`
combination for the service to be tested.  Supports both IPv6 and IPv4,
in the case of a hostname that resolves to both, connections will be
attempted via both protocol versions, with the first successful
connection returning a successful check result.

Example check:

```json
{
  "check": {
    "id": "ssh",
    "name": "SSH (TCP)",
    "tcp": "example.com:22",
    "interval": "10s"
  }
}
```
2015-07-24 14:06:05 +10:00
Robert Gogolok 2b8d0c4785 api: run ACL tests by default 2015-06-13 23:51:30 +02:00
James Phillips de29cb4968 Fixes a broken unit test introduced by #834. 2015-06-09 12:01:20 -07:00
Armon Dadgar c2f8f954e2 Merge pull request #1008 from highlyunavailable/master
Check to see why a lock acquisition failed
2015-06-09 07:59:28 -04:00
Ryan Uber 20a2d19198 Merge pull request #834 from apoydence/master
Returns an error for a key with a '/'
2015-06-05 15:23:57 -07:00
Tiru Srikantha 3d97ea0bae Check to see why a lock acquisition failed 2015-06-05 13:23:23 -07:00
highlyunavailable 16de0a909b Remove unused DefaultSemaphoreRetryTime
Fixes #932

DefaultSemaphoreRetryTime is actually unused, unlike DefaultLockRetryTime.
2015-05-15 08:25:02 -07:00
Armon Dadgar dc6795a55a api: improve test reliability 2015-05-12 12:10:25 -07:00
Ryan Uber 72ee584df3 Fix tests after merge 2015-05-11 18:53:09 -07:00
Armon Dadgar 8d86290ebf Fixing merge conflict 2015-05-11 16:48:10 -07:00
Armon Dadgar db134f6190 Merge pull request #912 from rojojo23/renew-periodic
modify RenewPeriodic to retry failed Renew until TTL elapses
2015-05-08 11:38:51 -07:00
Ryan Uber 565cac0340 api: run tests in parallel 2015-05-08 10:27:24 -07:00
Robert Johnson 379a69d7a4 modify RenewPeriodic to retry failed Renews until TTL elapses 2015-05-06 16:40:45 -07:00
Armon Dadgar 5921fcb29b api: Fixing test compilation issue 2015-05-04 15:27:22 -07:00
Armon Dadgar 0450854f12 Merge pull request #861 from fraenkel/reclaim_lock
Reclaim locks
2015-05-04 15:23:29 -07:00
Michael Fraenkel 0e25381494 Reclaim locks
- When attempting to lock with the same session that already owns the lock,
  Lock() will re-acquire the lock.
2015-04-29 10:10:24 -07:00
Ryan Mills 275af975e8 Allow specifying a status field in the agent/service/register and agent/check/register endpoints.
This status must be one of the valid check statuses: 'passing', 'warning', 'critical', 'unknown'.
If the status field is not present or the empty string, the default of 'critical' is used.
2015-04-12 02:00:31 +00:00
Andrew Poydence b5f6451e74 Returns an error for a key with a '/' 2015-03-31 21:08:06 -06:00
Ryan Uber fa7414e276 testutil: Use HTTPAddr 2015-03-19 17:03:07 -07:00
Ryan Uber 78f9f53bf1 testutil: exposing the API address from the test server 2015-03-19 17:03:06 -07:00
Ryan Uber 37f6301856 testutil: initial pass at moving test server harness into testutil 2015-03-19 17:03:06 -07:00
Seth Vargo b4f8c98fbb Read select environment variables when generating the default configuration 2015-03-18 11:12:10 -04:00
Chris Grieger f70ccdb4d2 Add 'ServiceAddress' field to 'CatalogService's truct 2015-03-04 10:29:57 +01:00
Ryan Uber 33390779eb api: remove unneeded static config 2015-02-24 09:50:35 -08:00
Armon Dadgar 5f627fca1b api: Refactoring into shared write logic 2015-02-18 15:15:02 -08:00
Armon Dadgar c0c9fb58ce api: Refactoring into shared query logic 2015-02-18 15:15:02 -08:00
Ryan Uber 5c2a764cbe api: allow timeout field to be specified for checks 2015-02-05 23:34:30 -08:00
Nicholas Capo 8cbca78b29 api/agent: Support HTTP service checks in client api 2015-02-03 20:13:55 +00:00
Armon Dadgar 17d1c392e4 CHANGELOG update, gofmt 2015-01-22 13:42:22 -08:00
Armon Dadgar e1a5d537d4 Merge pull request #625 from hashicorp/f-maintcmd
New "maint" command
2015-01-22 11:56:49 -08:00
Dr. Stefan Schimanski acd9efbc40 Add test case for service Address field 2015-01-22 11:50:20 +01:00
Dr. Stefan Schimanski a85575dcaa Add agent service Address field to the api
The Address field was introduced in #570. This patch extends this to the api.
2015-01-22 10:26:27 +01:00
Ryan Uber 089c4396c4 api: support reason flag for maintenance mode 2015-01-21 13:02:47 -08:00
Ryan Uber e088240c95 api: finish up maintenance tests 2015-01-21 11:08:57 -08:00
Ryan Uber e47b64197e api: add node/service maintenance mode functions 2015-01-21 10:51:43 -08:00
Ryan Uber 197a5a9a9a api: support multiple checks during service/check registration 2015-01-20 21:48:45 -08:00
Armon Dadgar d998bb3d67 Fixing merge conflict 2015-01-20 13:01:13 -08:00
Armon Dadgar 2e281dc51f api: Fixing some comments 2015-01-20 12:58:09 -08:00
Armon Dadgar 7195de51a2 api: Refactor to share session renew code 2015-01-19 16:53:47 -10:00
Armon Dadgar 14691f7e29 api: Detect conflicting existing values for lock/semaphore 2015-01-19 15:32:19 -10:00
Armon Dadgar 4fef14163d api: Make channels receive only 2015-01-19 14:37:36 -10:00
Ryan Uber b30af9536d Merge pull request #608 from imkira/fixBodyLeak
fixed: body not closed for non HTTP 200 responses
2015-01-19 09:33:06 -08:00
Mário Freitas f45105383c fixed: requireOK should close Body on error 2015-01-19 11:53:21 +09:00
Mário Freitas f30237a92e Revert "fixed: body not closed for non HTTP 200 responses"
This reverts commit dafa61ad10.
2015-01-19 11:51:51 +09:00
Ryan Uber 4b635738bf api: simple test for unix socket connectivity 2015-01-16 10:56:47 -08:00
Ryan Uber 4c3ec248a5 agent: fixing up tests 2015-01-16 09:58:37 -08:00
Mário Freitas dafa61ad10 fixed: body not closed for non HTTP 200 responses 2015-01-16 18:54:55 +09:00
Ryan Uber 4675cdf01c agent: beginning refactor 2015-01-16 00:45:03 -08:00
Jeff Mitchell 9fcea08dbb Ensure a socket is created for permissions adjustment tests and fix some
items pointed out in the code review

This code is copyright 2014 Akamai Technologies, Inc. <opensource@akamai.com>
2015-01-15 14:33:44 +00:00
Jeff Mitchell 70dd5a1e81 Unix socket-based HTTP API test functionality. As a consequence this
also required making some hardcoded values into more generic
functionality, which is generally a good thing. I verified that each
test function that I modified still passed.:

This code is copyright 2014 Akamai Technologies, Inc. <opensource@akamai.com>
2015-01-14 19:49:37 +00:00
Jeff Mitchell 11a3ce0bdd RPC and HTTP interfaces fully generically-sockified so Unix is supported.
Client works for RPC; will honor CONSUL_RPC_ADDR. HTTP works via consul/api;
honors CONSUL_HTTP_ADDR.

The format of a Unix socket in configuration data is:
"unix://[/path/to/socket];[username or uid];[gid];[mode]"

Obviously, the user must have appropriate permissions to create the socket
file in the given path and assign the requested uid/gid. Also note that Go does
not support gid lookups from group name, so gid must be numeric. See
https://codereview.appspot.com/101310044

When connecting from the client, the format is just the first part of the
above line:
"unix://[/path/to/socket]"

This code is copyright 2014 Akamai Technologies, Inc. <opensource@akamai.com>
2015-01-14 19:31:21 +00:00
Armon Dadgar 7ed1449b6c api: Adding Destroy to cleanup a semaphore 2015-01-13 14:18:28 -08:00
Armon Dadgar 9608108e64 api: Adding Destroy to cleanup a lock 2015-01-13 14:01:50 -08:00
Armon Dadgar 94d7022a88 api: Add support for DeleteCAS 2015-01-13 13:57:48 -08:00
Armon Dadgar f2e1594814 Merge pull request #600 from hashicorp/f-api-semaphore
Adding Semaphore support to API
2015-01-13 11:59:22 -08:00
Armon Dadgar ed6abe05fb api: Changing default semaphore key 2015-01-13 11:51:12 -08:00
Armon Dadgar 4a038927ee api: More reliable session check 2015-01-13 11:50:57 -08:00
Armon Dadgar c19b0e2ab2 api: Enable debug output from Consul for tests 2015-01-13 11:50:09 -08:00
Ryan Uber 78328ec149 api: make node health test more reliable 2015-01-13 11:25:32 -08:00
Armon Dadgar 5107f5d1f9 api: More semaphore tests 2015-01-12 18:18:32 -08:00
Armon Dadgar 8ad16ca390 api: Adding semaphore tests and fixes 2015-01-12 18:13:52 -08:00
Armon Dadgar b39374acae api: First pass at semaphore 2015-01-12 17:43:54 -08:00
Armon Dadgar c291acd96e api: Minor cleanups in lock 2015-01-12 17:43:13 -08:00
Armon Dadgar 3cba4df127 Merge pull request #594 from hashicorp/f-api-lock
api: Adding Lock helpers for leader election
2015-01-12 15:47:35 -08:00
Armon Dadgar 7225088c7d api: Fixing typo 2015-01-12 15:46:40 -08:00
Armon Dadgar e05ab983ab api: Fixing typos 2015-01-09 17:42:24 -08:00
Armon Dadgar 854aef82b0 api: Adding Lock helpers for leader election 2015-01-09 17:35:17 -08:00
Ryan Uber fab30406be api: test for a known leader as well as index > 0 2015-01-09 16:39:35 -08:00
Ryan Uber cb1b722813 api: improve reliability of catalog endpoint tests 2015-01-07 18:29:42 -08:00
Armon Dadgar bb3829c2d9 api: Adding constants for session behaviors 2015-01-07 14:01:27 -08:00
Ryan Uber 60ebd63760 Import HTTP basic auth patch from armon/consul-api#16 2015-01-07 13:01:44 -08:00
Ryan Uber 6c4ff58d2a api: improve test reliability 2015-01-06 16:51:28 -08:00
Ryan Uber 0c6c06cb0c api: use WaitForResult() to improve health tests 2015-01-06 16:18:50 -08:00
Ryan Uber f49189252e api: run consul instance on testing ports 2015-01-06 16:18:29 -08:00