Exposing checks is supposed to allow a Consul agent bound to a different
IP address (e.g., in a different Kubernetes pod) to access healthchecks
through the proxy while the underlying service binds to localhost. This
is an important security feature that makes sure no external traffic
reaches the service except through the proxy.
However, as far as I can tell, this is subtly broken in the case where
the Consul agent cannot reach the proxy over localhost.
If a proxy is configured with: `{ LocalServiceAddress: "127.0.0.1",
Checks: true }`, as is typical with a sidecar proxy, the Consul checks
are currently rewritten to `127.0.0.1:<random port>`. A Consul agent
that does not share the loopback address cannot reach this address. Just
to make sure I was not misunderstanding, I tried configuring the proxy
with `{ LocalServiceAddress: "<pod ip>", Checks: true }`. In this case,
while the checks are rewritten as expected and the agent can reach the
dynamic port, the proxy can no longer reach its backend because the
traffic is no longer on the loopback interface.
I think rewriting the checks to use `proxy.Address`, the proxy's own
address, is more correct in this case. That is the IP where the proxy
can be reached, both by other proxies and by a Consul agent running on
a different IP. The local service address should continue to use
`127.0.0.1` in most cases.
If a proxied service is a gRPC or HTTP2 service, but a path is exposed
using the HTTP1 or TCP protocol, Envoy should not be configured with
`http2ProtocolOptions` for the cluster backing the path.
A situation where this comes up is a gRPC service whose healthcheck or
metrics route (e.g. for Prometheus) is an HTTP1 service running on
a different port. Previously, if these were exposed either using
`Expose: { Checks: true }` or `Expose: { Paths: ... }`, Envoy would
still be configured to communicate with the path over HTTP2, which would
not work properly.
I spent some time today on my local Mac to figure out why Consul 1.6.3+
was not accepting limits.http_max_conns_per_client.
This adds an explicit check on number of file descriptors to be sure
it might work (this is no guarantee as if many clients are reaching
the agent, it might consume even more file descriptors)
Anyway, many users are fighting with RLIMIT_NOFILE, having a clear
message would allow them to figure out what to fix.
Example of message (reload or start):
```
2020-03-11T16:38:37.062+0100 [ERROR] agent: Error starting agent: error="system allows a max of 512 file descriptors, but limits.http_max_conns_per_client: 8192 needs at least 8212"
```
The previous PR which added these was accidentally performing the download
in the root directory. For the api, and sdk directories it should be in done
in the same directory that will be used to run tests. Otherwise the
wrong dependencies will be downloaded which may add unnecessary time to
the CI run.
This removes a race condition in reset since pendingPorts can be set to nil in reset()
If ticker is hit at wrong time, it would crash the unit test.
We ensure in reset to avoid this race condition by cancelling the goroutine using
killTicker chan.
We also properly clean up eveything, so garbage collector can work as expected.
To reproduce existing bug:
`while go test -timeout 30s github.com/hashicorp/consul/sdk/freeport -run '^(Test.*)$'; do go clean -testcache; done`
Will crash after a few 10s runs on my machine.
Error could be seen in unit tests sometimes:
[INFO] freeport: resetting the freeport package state
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x1125536]
goroutine 25 [running]:
container/list.(*List).Len(...)
/usr/local/Cellar/go/1.14/libexec/src/container/list/list.go:66
github.com/hashicorp/consul/sdk/freeport.checkFreedPortsOnce()
/Users/p.souchay/go/src/github.com/hashicorp/consul/sdk/freeport/freeport.go:157 +0x86
github.com/hashicorp/consul/sdk/freeport.checkFreedPorts()
/Users/p.souchay/go/src/github.com/hashicorp/consul/sdk/freeport/freeport.go:147 +0x71
created by github.com/hashicorp/consul/sdk/freeport.initialize
/Users/p.souchay/go/src/github.com/hashicorp/consul/sdk/freeport/freeport.go:113 +0x2cf
FAIL github.com/hashicorp/consul/sdk/freeport 1.607s
Due to merge #7562, upstream does not compile anymore.
Error is:
ERRO Running error: gofmt: analysis skipped: errors in package: [/Users/p.souchay/go/src/github.com/hashicorp/consul/agent/config_endpoint_test.go:188:33: too many arguments]
We need to detect whether an object is an ember-data snapshot or just a
plain object, and we where restricted from using `instanceof` due to
ember-data's `Snapshot` class being private.
We'd chosen to go with `constructor.name` instead, which seemed to work,
but when the javascript gets minifed the name of the contructor is also
minified and therefore is not what you are expecting.
This commit reverts to our original idea of checking for the existence
and type of the `.attributes` function, which is the function we require
within the conditional, and therefore is more reliable (if the function
doesn't exist it will error out during development aswell as production)
This function now only starts the agent.
Using:
git grep -l 'StartTestAgent(t, true,' | \
xargs sed -i -e 's/StartTestAgent(t, true,/StartTestAgent(t,/g'
When run in with `-dev` in DevMode, it is not possible to replace
the embeded UI with another one because `-dev` implies `-ui`.
This commit allows this an slightly change the error message
about Consul 0.7.0 which is very old and does not apply to
current version anyway.
This fixes this bats warning:
duplicate test name(s) in /workdir/primary/bats/verify.bats: test_s1_upstream_made_1_connection
Test was already defined at line 42, rename it to avoid test name duplication
This config entry will be used to configure terminating gateways.
It accepts the name of the gateway and a list of services the gateway will represent.
For each service users will be able to specify: its name, namespace, and additional options for TLS origination.
Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>
Co-authored-by: Chris Piraino <cpiraino@hashicorp.com>
* Add Ingress gateway config entry and other relevant structs
* Add api package tests for ingress gateways
* Embed EnterpriseMeta into ingress service struct
* Add namespace fields to api module and test consul config write decoding
* Don't require a port for ingress gateways
* Add snakeJSON and camelJSON cases in command test
* Run Normalize on service's ent metadata
Sadly cannot think of a way to test this in OSS.
* Every protocol requires at least 1 service
* Validate ingress protocols
* Update agent/structs/config_entry_gateways.go
Co-authored-by: Chris Piraino <cpiraino@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
Run 'go mod download' first so that dependencies are pulled in a
separate step. This makes the test splitting and test output easier
to scroll through.
Remove test splitting from the api/sdk test runs. These jobs do not
use parallelism, so there is no test timing data injected into the
job. They can't be split.
Remove the dependencies from the go-test job, so that we can start
the slowest test job earlier in the workflow.
Ember tries to reuse DOM elements when it can but as ember looks for
changes to objects rather than the DOM itself sometimes. This and the
fact that an objects identity may change even though its value hasn't,
results in ember occasionally re-mutating DOM when it doesn't need to.
The `each` helper includes a `key` attribute to hint to ember what it
should look for when deciding whether something has changed, rather than
the objects identity.
https://api.emberjs.com/ember/release/classes/Ember.Templates.helpers/methods/each#specifying-keys
We use this here to fix an issue where DOM was being redrawn after the
user had scrolled the page and was therefore resetting the scroll back
to 0 (the top of the page)
Previously the log output included the test name twice and a long date
format. The test output is already grouped by test, so adding the test
name did not add any new information. The date and time are only useful
to understand elapsed time, so using a short format should provide
succident detail.
Also fixed a bug in NewTestAgentWithFields where nil was returned
instead of the test agent.
* ui: Ensure configuration is passed through to findInstanceBySlug
Due to the addition of namespace support, this arguments passed to this
method have been increased. Whilst the nspace support continues ot work
here, the configuration for blocking queries is never passed through.
This results in a 2 second poll rather than a blocking query.
This commit fixes that
* ui: Add a basic test to check the number of arguments passed through
Use systemctl to properly detect ephemeral ports on Mac OS (aka darwin) by fetching
systemctl values:
* net.inet.ip.portrange.first
* net.inet.ip.portrange.last
This will avoid the message:
`[INFO] freeport: ephemeral port range detection not configured for GOOS="darwin"`
and properly detect the correct port range
Matt Keeler
Chris Piraino
Pierre Souchay
Andy Lindeman
Daniel Nephin
Daniel Nephin
Shaker Islam
R.B. Boyer
Pierre Souchay
Hans Hasselberg
Rasmus Mikkelsen
John Cowen
Emre Savcı
Andreas Gruhler
Geoffrey Grosenbach
Freddy
Kyle Havlovitz
Jono Sosulska