Commit Graph

11177 Commits

Author SHA1 Message Date
Daniel Nephin 0888c6575b Step 3: fix a bug in api.NewClient and fix the tests
The api client should never rever to HTTP if the user explicitly
requested TLS. This change broke some tests because the tests always use
an non-TLS http server, but some tests explicitly enable TLS.
2020-04-07 18:02:56 -04:00
Daniel Nephin 1a8ffec6a7 Step 2: extract the grpc address logic and a new type
The new grpcAddress function contains all of the logic to translate the
command line options into the values used in the template.

The new type has two advantages.

1. It introduces a logical grouping of values in the BootstrapTplArgs
   struct which is exceptionally large. This grouping makes the struct
   easier to understand because each set of nested values can be seen
   as a single entity.
2. It gives us a reasonable return value for this new function.
2020-04-07 16:36:51 -04:00
Daniel Nephin 830b4a15f6 Step 1: move all the grpcAddr logic into the same spot
There is no reason a reader should have to jump around to find this value. It is only
used in 1 place
2020-04-07 15:53:12 -04:00
Alvin Huang 1bb4e1ccd2
filter out non go branches from the 'go-tests' workflow (#7606) 2020-04-07 15:39:23 -04:00
Matt Keeler 32daa2b27c
Update CHANGELOG.md 2020-04-07 11:49:36 -04:00
Matt Keeler 0e7d3d93b3
Enable filtering language support for the v1/connect/intentions… (#7593)
* Enable filtering language support for the v1/connect/intentions listing API

* Update website for filtering of Intentions

* Update website/source/api/connect/intentions.html.md
2020-04-07 11:48:44 -04:00
Daniel Nephin 8549cc2d99
Merge pull request #7598 from pierresouchay/preallocation_of_dns_meta
Pre-allocations of DNS meta to avoid several allocations
2020-04-06 14:00:32 -04:00
Pierre Souchay d1d016d61d
[LINT] Close resp.Body to avoid linter complaining (#7600) 2020-04-06 09:11:04 -04:00
Pierre Souchay c9e01ed0a3 Pre-allocations of DNS meta to avoid several allocations 2020-04-05 11:12:41 +02:00
Jono Sosulska 93509690be
Change style to match "join" singular (#7569)
* Change style to match "join" singular

- Replaced "(Consul) cluster" with  "Consul Datacenter"
- Removed "ing" so the feature fits "Consul Auto-join", and that the tense is correct.

Co-authored-by: danielehc <40759828+danielehc@users.noreply.github.com>
2020-04-03 16:04:07 -04:00
David Yu b51ad875c3
[docs] Built-in Proxies not meant for production (#7579)
* [docs] Built-in Proxies not meant for production

* Adding link to Envoy for Connect

* Update website/source/docs/connect/proxies/built-in.md

Co-Authored-By: Blake Covarrubias <blake@covarrubi.as>

* Revising note

* Update website/source/docs/connect/proxies/built-in.md

period

Co-Authored-By: Hans Hasselberg <me@hans.io>

Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
Co-authored-by: Hans Hasselberg <me@hans.io>
2020-04-03 11:52:05 -07:00
Daniel Nephin b9cb1e93c3
Merge pull request #7589 from pierresouchay/fix_unstable_test_TestForwardSignals
Fixed unstable test TestForwardSignals()
2020-04-03 12:43:28 -04:00
Pierre Souchay 73056fecf8 Fixed unstable test TestForwardSignals()
Sometimes, in the CI, it could receive a SIGURG, producing this line:

  FAIL: TestForwardSignals/signal-interrupt (0.06s)
        util_test.go:286: expected to read line "signal: interrupt" but got "signal: urgent I/O condition"

Only forward the signals we test to avoid this kind of false positive

Example of such unstable errors in CI:
https://circleci.com/gh/hashicorp/consul/153571
2020-04-03 14:23:03 +02:00
Pierre Souchay 09e638a9c6
tests: more tolerance to latency for unstable test `TestCacheNotifyPolling()`. (#7574) 2020-04-03 10:29:38 +02:00
Freddy b61214ef24
Fix regression with gateway registration and update docs (#7582) 2020-04-02 12:52:11 -06:00
Matt Keeler 8aec09aa8f
Ensure that token clone copies the roles (#7577) 2020-04-02 12:09:35 -04:00
Chris Piraino bcb7a89fda
Update CHANGELOG.md 2020-04-02 10:14:17 -05:00
Chris Piraino 584f90bbeb
Fix flapping of mesh gateway connect-service watches (#7575) 2020-04-02 10:12:13 -05:00
Pierre Souchay 2a8bf45e38
agent: show warning when enable_script_checks is enabled without safty net (#7437)
In order to enforce a bit security on Consul agents, add a new method in agent
to highlight possible security issues.

This does not return an error for now, but might in the future.

For now, it detects issues such as:

https://www.hashicorp.com/blog/protecting-consul-from-rce-risk-in-specific-configurations/

This would display this kind of messages:

```
2020-03-11T18:27:49.873+0100 [ERROR] agent: [SECURITY] issue: error="using enable-script-checks without ACLs and without allow_write_http_from is DANGEROUS, use enable-local-script-checks instead see https://www.hashicorp.com/blog/protecting-consul-from-rce-risk-in-specific-configurations/"
```
2020-04-02 09:59:23 +02:00
Andy Lindeman fb0a990e4d
agent: rewrite checks with proxy address, not local service address (#7518)
Exposing checks is supposed to allow a Consul agent bound to a different
IP address (e.g., in a different Kubernetes pod) to access healthchecks
through the proxy while the underlying service binds to localhost. This
is an important security feature that makes sure no external traffic
reaches the service except through the proxy.

However, as far as I can tell, this is subtly broken in the case where
the Consul agent cannot reach the proxy over localhost.

If a proxy is configured with: `{ LocalServiceAddress: "127.0.0.1",
Checks: true }`, as is typical with a sidecar proxy, the Consul checks
are currently rewritten to `127.0.0.1:<random port>`. A Consul agent
that does not share the loopback address cannot reach this address. Just
to make sure I was not misunderstanding, I tried configuring the proxy
with `{ LocalServiceAddress: "<pod ip>", Checks: true }`. In this case,
while the checks are rewritten as expected and the agent can reach the
dynamic port, the proxy can no longer reach its backend because the
traffic is no longer on the loopback interface.

I think rewriting the checks to use `proxy.Address`, the proxy's own
address, is more correct in this case. That is the IP where the proxy
can be reached, both by other proxies and by a Consul agent running on
a different IP. The local service address should continue to use
`127.0.0.1` in most cases.
2020-04-02 09:35:43 +02:00
Andy Lindeman c1cb18c648
proxycfg: support path exposed with non-HTTP2 protocol (#7510)
If a proxied service is a gRPC or HTTP2 service, but a path is exposed
using the HTTP1 or TCP protocol, Envoy should not be configured with
`http2ProtocolOptions` for the cluster backing the path.

A situation where this comes up is a gRPC service whose healthcheck or
metrics route (e.g. for Prometheus) is an HTTP1 service running on
a different port. Previously, if these were exposed either using
`Expose: { Checks: true }` or `Expose: { Paths: ... }`, Envoy would
still be configured to communicate with the path over HTTP2, which would
not work properly.
2020-04-02 09:35:04 +02:00
Pierre Souchay be1c5c4b48
config: validate system limits against limits.http_max_conns_per_client (#7434)
I spent some time today on my local Mac to figure out why Consul 1.6.3+
was not accepting limits.http_max_conns_per_client.

This adds an explicit check on number of file descriptors to be sure
it might work (this is no guarantee as if many clients are reaching
the agent, it might consume even more file descriptors)

Anyway, many users are fighting with RLIMIT_NOFILE, having a clear
message would allow them to figure out what to fix.

Example of message (reload or start):

```
2020-03-11T16:38:37.062+0100 [ERROR] agent: Error starting agent: error="system allows a max of 512 file descriptors, but limits.http_max_conns_per_client: 8192 needs at least 8212"
```
2020-04-02 09:22:17 +02:00
Daniel Nephin a6cede401a
Merge pull request #7572 from hashicorp/dnephin/ci-fix-go-mod-download
ci: Fix working_directory for go mod download
2020-04-01 18:57:21 -04:00
Daniel Nephin 625ce9baf2 ci: Fix working_directory for go mod download
The previous PR which added these was accidentally performing the download
in the root directory. For the api, and sdk directories it should be in done
in the same directory that will be used to run tests. Otherwise the
wrong dependencies will be downloaded which may add unnecessary time to
the CI run.
2020-04-01 17:02:23 -04:00
Shaker Islam ac309d55f4
docs: document exported functions in agent.go (closes #7101) (#7366)
and fix one linter error
2020-04-01 22:52:23 +02:00
R.B. Boyer c2f4063103 update changelog 2020-04-01 13:16:01 -05:00
Pierre Souchay 5703aadb59
[BUGFIX] Fix race condition in freeport (#7567)
This removes a race condition in reset since pendingPorts can be set to nil in reset()

If ticker is hit at wrong time, it would crash the unit test.

We ensure in reset to avoid this race condition by cancelling the goroutine using
killTicker chan.

We also properly clean up eveything, so garbage collector can work as expected.

To reproduce existing bug:
`while go test -timeout 30s github.com/hashicorp/consul/sdk/freeport -run '^(Test.*)$'; do go clean -testcache; done`

Will crash after a few 10s runs on my machine.

Error could be seen in unit tests sometimes:

[INFO] freeport: resetting the freeport package state
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x1125536]

goroutine 25 [running]:
container/list.(*List).Len(...)
	/usr/local/Cellar/go/1.14/libexec/src/container/list/list.go:66
github.com/hashicorp/consul/sdk/freeport.checkFreedPortsOnce()
	/Users/p.souchay/go/src/github.com/hashicorp/consul/sdk/freeport/freeport.go:157 +0x86
github.com/hashicorp/consul/sdk/freeport.checkFreedPorts()
	/Users/p.souchay/go/src/github.com/hashicorp/consul/sdk/freeport/freeport.go:147 +0x71
created by github.com/hashicorp/consul/sdk/freeport.initialize
	/Users/p.souchay/go/src/github.com/hashicorp/consul/sdk/freeport/freeport.go:113 +0x2cf
FAIL	github.com/hashicorp/consul/sdk/freeport	1.607s
2020-04-01 13:14:33 -05:00
Daniel Nephin fce56e4cb6
Merge pull request #7570 from pierresouchay/fix_lint_upstream_due_to_change_args_in_NewTestAgent
[FIX BUILD] fix build due to merge of #7562
2020-04-01 12:39:34 -04:00
Pierre Souchay 08acd6a03e [FIX BUILD] fix build due to merge of #7562
Due to merge #7562, upstream does not compile anymore.

Error is:

ERRO Running error: gofmt: analysis skipped: errors in package: [/Users/p.souchay/go/src/github.com/hashicorp/consul/agent/config_endpoint_test.go:188:33: too many arguments]
2020-04-01 18:29:45 +02:00
Daniel Nephin 0d8edc3e27
Merge pull request #7562 from hashicorp/dnephin/remove-tname-from-name
testing: Remove old default value from NewTestAgent() calls
2020-04-01 11:48:45 -04:00
Daniel Nephin 76f03982eb
Merge pull request #7427 from hashicorp/dnephin/website-fix-errrors-in-upgrade-docs
docs: fix some errors in upgrade-specific
2020-04-01 11:36:53 -04:00
Daniel Nephin 2c631a7a78
Merge pull request #7465 from hashicorp/dnephin/correct-docs-log-file
docs: Update config reference for log-file
2020-04-01 11:29:20 -04:00
Daniel Nephin 5f44bbcac2
Merge pull request #7519 from hashicorp/dnephin/help-to-stdout
cli: send requested help text to stdout
2020-04-01 11:26:12 -04:00
Daniel Nephin 190fc3c732
Merge pull request #7533 from hashicorp/dnephin/xds-server-1
agent/xds: small cleanup
2020-04-01 11:24:50 -04:00
Hans Hasselberg efaf0f9a18
docs: revert "docs: fix param for register checks id->checkid (#7393)" (#7565)
This reverts commit ed5202943e.
2020-04-01 11:49:20 +02:00
Rasmus Mikkelsen ed5202943e
docs: fix param for register checks id->checkid (#7393) 2020-04-01 10:59:28 +02:00
John Cowen 8d4631526c
ui: Fix token duplication bug (#7552)
We need to detect whether an object is an ember-data snapshot or just a
plain object, and we where restricted from using `instanceof` due to
ember-data's `Snapshot` class being private.

We'd chosen to go with `constructor.name` instead, which seemed to work,
but when the javascript gets minifed the name of the contructor is also
minified and therefore is not what you are expecting.

This commit reverts to our original idea of checking for the existence
and type of the `.attributes` function, which is the function we require
within the conditional, and therefore is more reliable (if the function
doesn't exist it will error out during development aswell as production)
2020-04-01 09:55:20 +01:00
Emre Savcı 2083b7b04d
agent: add len, cap while initializing arrays 2020-04-01 10:54:51 +02:00
Pierre Souchay 4d35816bee
tests: fixed unstable test TestAPI_AgentMonitor (#7561)
Similar fix as https://github.com/hashicorp/consul/pull/7541
2020-04-01 09:47:57 +02:00
Daniel Nephin e759daafdd Rename NewTestAgentWithFields to StartTestAgent
This function now only starts the agent.

Using:

git grep -l 'StartTestAgent(t, true,' | \
        xargs sed -i -e 's/StartTestAgent(t, true,/StartTestAgent(t,/g'
2020-03-31 17:14:55 -04:00
Daniel Nephin f9f6b14533 Convert the remaining calls to NewTestAgentWithFields
After removing the t.Name() parameter with sed, convert the last few tests which
use a custom name to call NewTestAgentWithFields instead.
2020-03-31 17:14:55 -04:00
Daniel Nephin 9d83a45663
Merge pull request #7557 from hashicorp/dnephin/ci-small-improvements
ci: Upload coverage from each job
2020-03-31 17:07:01 -04:00
Daniel Nephin 0c409d460f
Merge pull request #7470 from hashicorp/dnephin/dns-unused-params
dns: Remove a few unused function parameters
2020-03-31 16:56:19 -04:00
Andreas Gruhler 3cb9aa8010
docs: make example code for RedundancyZoneTag match description (#7464) 2020-03-31 22:54:04 +02:00
Pierre Souchay 54b22c638d
config: allow running `consul agent -dev -ui-dir=some_path` (#7525)
When run in with `-dev` in DevMode, it is not possible to replace
the embeded UI with another one because `-dev` implies `-ui`.

This commit allows this an slightly change the error message
about Consul 0.7.0 which is very old and does not apply to
current version anyway.
2020-03-31 22:36:20 +02:00
Geoffrey Grosenbach ab6c410064
docs: add link to Learn from downloads page (#7534)
Adds a link to Learn from the downloads page as well as a link to the Consul
Template guide.
2020-03-31 22:30:29 +02:00
Pierre Souchay eafe9a895a
tests: fixed bats warning (#7544)
This fixes this bats warning:

  duplicate test name(s) in /workdir/primary/bats/verify.bats: test_s1_upstream_made_1_connection

Test was already defined at line 42, rename it to avoid test name duplication
2020-03-31 22:29:27 +02:00
Daniel Nephin 475659a132 Remove name from NewTestAgent
Using:

git grep -l 'NewTestAgent(t, t.Name(),' | \
    xargs sed -i -e 's/NewTestAgent(t, t.Name(),/NewTestAgent(t,/g'
2020-03-31 16:13:44 -04:00
Freddy 90576060bc
Add config entry for terminating gateways (#7545)
This config entry will be used to configure terminating gateways.

It accepts the name of the gateway and a list of services the gateway will represent.

For each service users will be able to specify: its name, namespace, and additional options for TLS origination.

Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>
Co-authored-by: Chris Piraino <cpiraino@hashicorp.com>
2020-03-31 13:27:32 -06:00
Daniel Nephin 66a51c9701 Upload coverage from each job
Allows us to remove the merge coverage report step
2020-03-31 14:43:13 -04:00