Commit Graph

11888 Commits

Author SHA1 Message Date
Freddy c9dbb6c51a Only pass one hostname via EDS and prefer healthy ones (#8084)
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>

Currently when passing hostname clusters to Envoy, we set each service instance registered with Consul as an LbEndpoint for the cluster.

However, Envoy can only handle one per cluster:
[2020-06-04 18:32:34.094][1][warning][config] [source/common/config/grpc_subscription_impl.cc:87] gRPC config for type.googleapis.com/envoy.api.v2.Cluster rejected: Error adding/updating cluster(s) dc2.internal.ddd90499-9b47-91c5-4616-c0cbf0fc358a.consul: LOGICAL_DNS clusters must have a single locality_lb_endpoint and a single lb_endpoint, server.dc2.consul: LOGICAL_DNS clusters must have a single locality_lb_endpoint and a single lb_endpoint

Envoy is currently handling this gracefully by only picking one of the endpoints. However, we should avoid passing multiple to avoid these warning logs.

This PR:

* Ensures we only pass one endpoint, which is tied to one service instance.
* We prefer sending an endpoint which is marked as Healthy by Consul.
* If no endpoints are healthy we emit a warning and skip the cluster.
* If multiple unique hostnames are spread across service instances we emit a warning and let the user know which will be resolved.
2020-06-12 19:46:51 +00:00
John Cowen a0393c6ecf ui: Colocate pageobject components with their components (#8098) 2020-06-12 15:50:34 +00:00
Kenia fb1fd730a4 ui: Add health check icon to Ingress Upstreams and add Tooltips in additional areas (#8091)
* Add Health Checks and update Tooltips in Ingress Upstreams

* Update Tooltip in Proxy Info tab Upstreams

* Add Tooltips to Proxy Info tab Exposed Paths

* Add Health Checks with Tooltips to Service List page
2020-06-12 13:36:34 +00:00
John Cowen 51660ffb25 ui: Exclude any component/pageobject.js files from being included (#7978)
This excludes any /components/**/pageobject.js files from our production
builds which means we can co-locate all of our component page objects
(and selectors) along with the components themselves.
2020-06-12 12:08:00 +00:00
Chris Piraino 7f89ab990e Allow users to set hosts to the wildcard specifier when TLS is disabled (#8083)
This allows easier demoing/testing of ingress gateways, while still
preserving the validation we have for DNSSANs
2020-06-11 15:03:46 +00:00
Hans Hasselberg a8830aca88 Support envoy 1.14.2, 1.13.2, 1.12.4 (#8057) 2020-06-10 21:20:42 +00:00
Chris Piraino 42c8f34788 Move ingress param to a new endpoint (#8081)
In discussion with team, it was pointed out that query parameters tend
to be filter mechanism, and that semantically the "/v1/health/connect"
endpoint should return "all healthy connect-enabled endpoints (e.g.
could be side car proxies or native instances) for this service so I can
connect with mTLS".

That does not fit an ingress gateway, so we remove the query parameter
and add a new endpoint "/v1/health/ingress" that semantically means
"all the healthy ingress gateway instances that I can connect to
to access this connect-enabled service without mTLS"
2020-06-10 18:07:41 +00:00
John Cowen 2738deaa40
Update CHANGELOG.md 2020-06-10 17:44:30 +01:00
John Cowen daf283d315 Merge pull request #8080 from hashicorp/ui-staging
ui: UI Release Merge (1.8-beta-3: ui-staging merge)
2020-06-10 16:41:25 +00:00
Kyle Havlovitz d7d37bace4 Merge pull request #8075 from hashicorp/expose-cli-testfix
Fix a CLI test failure with namespaces in enterprise
2020-06-09 22:26:23 +00:00
Chris Piraino ea1b54a826 Merge pull request #8064 from hashicorp/ingress/health-query-param
Add API query parameter ?ingress to allow users to find ingress gateways associated to a service
2020-06-09 21:09:09 +00:00
Hans Hasselberg a678b47c73 acl: do not resolve local tokens from remote dcs (#8068) 2020-06-09 19:14:19 +00:00
Kyle Havlovitz 9e6718ad0f Merge pull request #8040 from hashicorp/ingress/expose-cli
Ingress expose CLI command
2020-06-09 19:11:51 +00:00
Daniel Nephin 1bfb7f3b07 Merge pull request #7964 from hashicorp/dnephin/remove-patch-slice-of-maps-forward-compat
config: Use HookWeakDecodeFromSlice in place of PatchSliceOfMaps
2020-06-08 23:53:31 +00:00
Hans Hasselberg cfc95732f3
Tokens converted from legacy ACLs get their Hash computed (#8047) (#8054)
This allows new style token replication to work for legacy tokens as well when they change.
Fixes #5606
2020-06-08 23:36:55 +02:00
Hans Hasselberg b4f33b52a2 agent: add option to disable agent cache for HTTP endpoints (#8023)
This allows the operator to disable agent caching for the http endpoint.
It is on by default for backwards compatibility and if disabled will
ignore the url parameter `cached`.
2020-06-08 22:49:33 +02:00
Chris Piraino 165a9af053 Always require Host header values for http services (#7990)
Previously, we did not require the 'service-name.*' host header value
when on a single http service was exposed. However, this allows a user
to get into a situation where, if they add another service to the
listener, suddenly the previous service's traffic might not be routed
correctly. Thus, we always require the Host header, even if there is
only 1 service.

Also, we add the make the default domain matching more restrictive by
matching "service-name.ingress.*" by default. This lines up better with
the namespace case and more accurately matches the Consul DNS value we
expect people to use in this case.
2020-06-08 18:16:48 +00:00
Hans Hasselberg c675166e1b Setup intermediate_pki_path on secondary when using vault (#8001)
Make sure to mount vault backend for intermediate_pki_path on secondary
dc.
2020-06-05 19:37:21 +00:00
Hans Hasselberg de3e68c577 Merge pull request #7966 from hashicorp/pool_improvements
Agent connection pool cleanup
2020-06-05 19:03:24 +00:00
R.B. Boyer 476a50769e Use linkmode=external for the arm builds (#8029)
This gets around some issues regarding too large plt offsets when compiling with CGO

Co-authored-by: Matt Keeler <mjkeeler7@gmail.com>
2020-06-05 18:22:20 +00:00
John Cowen f622542ce7 ui: Blocking with filtering intentions amends (#8027)
* ui: Now intentions use blocking queries invalidation isn't needed

* ui: Ensure reconciliation doesn't happen when filtering for intentions
2020-06-05 13:45:42 +00:00
R.B. Boyer 89fc98322e tests: ensure that the ServiceExists helper function normalizes entmeta (#8025)
This fixes a unit test failure over in enterprise due to https://github.com/hashicorp/consul/pull/7384
2020-06-05 08:42:35 +00:00
Hans Hasselberg 0491a9301b tests: use constructor instead init (#8024) 2020-06-04 23:12:44 +02:00
R.B. Boyer ebc5fc039f server: don't activate federation state replication or anti-entropy until all servers are running 1.8.0+ (#8014) 2020-06-04 21:05:49 +00:00
John Cowen a58d52e58a Merge pull request #8022 from hashicorp/ui/bugfix/switch-help-icons
ui: Switch help menu icons to use the correct icons
2020-06-04 15:04:43 +00:00
Pierre Souchay 621862606e checks: when a service does not exists in an alias, consider it failing (#7384)
In current implementation of Consul, check alias cannot determine
if a service exists or not. Because a service without any check
is semantically considered as passing, so when no healthchecks
are found for an agent, the check was considered as passing.

But this make little sense as the current implementation does not
make any difference between:
 * a non-existing service (passing)
 * a service without any check (passing as well)

In order to make it work, we have to ensure that when a check did
not find any healthcheck, the service does indeed exists. If it
does not, lets consider the check as failing.
2020-06-04 12:51:23 +00:00
freddygv f8b910aa10 Update CHANGELOG.md 2020-06-03 18:53:43 -06:00
Freddy 5d2475232a Enable gateways to resolve hostnames to IPv4 addresses (#7999)
The DNS resolution will be handled by Envoy and defaults to LOGICAL_DNS. This discovery type can be overridden on a per-gateway basis with the envoy_dns_discovery_type Gateway Option.

If a service contains an instance with a hostname as an address we set the Envoy cluster to use DNS as the discovery type rather than EDS. Since both mesh gateways and terminating gateways route to clusters using SNI, whenever there is a mix of hostnames and IP addresses associated with a service we use the hostname + CDS rather than the IPs + EDS.

Note that we detect hostnames by attempting to parse the service instance's address as an IP. If it is not a valid IP we assume it is a hostname.
2020-06-03 18:51:33 -06:00
R.B. Boyer 7a46c3908e fix some flaky snapshot tests (#8015) 2020-06-03 19:19:36 +00:00
John Cowen a93e9fdd9d
Update CHANGELOG.md 2020-06-03 18:50:38 +01:00
John Cowen 300a52f5ae Merge pull request #8013 from hashicorp/ui-staging
ui: UI Release Merge (1.8-beta-3: ui-staging merge)
2020-06-03 17:46:46 +00:00
Matt Keeler 1e2754d59c Fix legacy management tokens in unupgraded secondary dcs (#7908)
The ACL.GetPolicy RPC endpoint was supposed to return the “parent” policy and not always the default policy. In the case of legacy management tokens the parent policy was supposed to be “manage”. The result of us not sending this properly was that operations that required specifically a management token such as saving a snapshot would not work in secondary DCs until they were upgraded.
2020-06-03 15:42:57 +00:00
Matt Keeler a539c5de88 Fix segfault due to race condition for checking server versions (#7957)
The ACL monitoring routine uses c.routers to check for server version updates. Therefore it needs to be started after initializing the routers.
2020-06-03 14:37:10 +00:00
R.B. Boyer 5404155d36 acl: allow auth methods created in the primary datacenter to optionally create global tokens (#7899) 2020-06-01 16:45:22 +00:00
R.B. Boyer c4b875cae4 acl: remove the deprecated `acl_enforce_version_8` option (#7991)
Fixes #7292
2020-06-01 10:40:22 -05:00
Jono Sosulska cedcbf3299 Replace whitelist/blacklist terminology with allowlist/denylist (#7971)
* Replace whitelist/blacklist terminology with allowlist/denylist
2020-06-01 10:40:14 -05:00
Daniel Nephin 1664067943 ci: Add staticcheck and fix most errors
Three of the checks are temporarily disabled to limit the size of the
diff, and allow us to enable all the other checks in CI.

In a follow up we can fix the issues reported by the other checks one
at a time, and enable them.
2020-06-01 10:40:04 -05:00
Daniel Nephin 1aeede5eb7 config: use the new HookTranslateKeys instead of lib.TranslateKeys
With the exception of CA provider config, which will be migrated at some
later time.
2020-06-01 10:39:58 -05:00
Daniel Nephin b11a615f0c Add alias struct tags for new decode hook 2020-06-01 10:39:51 -05:00
Daniel Nephin 225059294e config: add HookTranslteKeys
This hook replaces lib.TranslateKeys and has a number of advantages:

1. Primarily, aliases for fields are defined on the field itself, making
   the aliases much easier to maintain, and more obvious to the reader.
2. TranslateKeys translation rules are not aware of structure. It could
   very easily incorrectly translate a key on one struct that was intended
   to be a translation rule for a completely different struct, leading
   to very hard to debug errors. The hook removes the need for the
   unexpected "translation rule is an empty string to indicate stop
   traversal" special case.
3. TranslateKeys attempts to duplicate a bunch of tree traversal logic
   that already exists in mapstructure. Using mapstructure for traversal
   removes the need to traverse the entire structure multiple times, and
   makes the behaviour more obvious to the reader.

This change is being made to enable a future change of replacing
PatchSliceOfMaps. TranslateKeys sits in between PatchSliceOfMaps and
mapstructure.Decode, so it must be converted to a hook first, before
PatchSliceOfMaps can be replaced by a decode hook.
2020-06-01 10:38:27 -05:00
Raphaël Rondeau b29c954480 connect: fix endpoints clusterName when using cluster escape hatch (#7319)
```changelog
* fix(connect): fix endpoints clusterName when using cluster escape hatch
```
2020-06-01 10:35:31 -05:00
Pierre Souchay 0d86e802be Stop all watches before shuting down anything dring shutdown. (#7526)
This will prevent watches from being triggered.

```changelog
* fix(agent):  stop all watches before shuting down
```
2020-06-01 10:35:14 -05:00
Pierre Souchay 66612e5dc6 tests: added unit test to ensure watches are not re-triggered on consul reload (#7449)
This ensures no regression about https://github.com/hashicorp/consul/issues/7318
And ensure that https://github.com/hashicorp/consul/issues/7446 cannot happen anymore
2020-06-01 10:33:31 -05:00
Pierre Souchay 876ee89d4a Allow to restrict servers that can join a given Serf Consul cluster. (#7628)
Based on work done in https://github.com/hashicorp/memberlist/pull/196
this allows to restrict the IP ranges that can join a given Serf cluster
and be a member of the cluster.

Restrictions on IPs can be done separatly using 2 new differents flags
and config options to restrict IPs for LAN and WAN Serf.
2020-06-01 10:31:32 -05:00
R.B. Boyer c2b903b597 create lib/stringslice package (#7934) 2020-05-27 16:48:01 +00:00
R.B. Boyer b527e77850 agent: handle re-bootstrapping in a secondary datacenter when WAN federation via mesh gateways is configured (#7931)
The main fix here is to always union the `primary-gateways` list with
the list of mesh gateways in the primary returned from the replicated
federation states list. This will allow any replicated (incorrect) state
to be supplemented with user-configured (correct) state in the config
file. Eventually the game of random selection whack-a-mole will pick a
winning entry and re-replicate the latest federation states from the
primary. If the user-configured state is actually the incorrect one,
then the same eventual correct selection process will work in that case,
too.

The secondary fix is actually to finish making wanfed-via-mgws actually
work as originally designed. Once a secondary datacenter has replicated
federation states for the primary AND managed to stand up its own local
mesh gateways then all of the RPCs from a secondary to the primary
SHOULD go through two sets of mesh gateways to arrive in the consul
servers in the primary (one hop for the secondary datacenter's mesh
gateway, and one hop through the primary datacenter's mesh gateway).
This was neglected in the initial implementation. While everything
works, ideally we should treat communications that go around the mesh
gateways as just provided for bootstrapping purposes.

Now we heuristically use the success/failure history of the federation
state replicator goroutine loop to determine if our current mesh gateway
route is working as intended. If it is, we try using the local gateways,
and if those don't work we fall back on trying the primary via the union
of the replicated state and the go-discover configuration flags.

This can be improved slightly in the future by possibly initializing the
gateway choice to local on startup if we already have replicated state.
This PR does not address that improvement.

Fixes #7339
2020-05-27 16:32:22 +00:00
R.B. Boyer 1765fa854e connect: ensure proxy-defaults protocol is used for upstreams (#7938) 2020-05-21 21:09:51 +00:00
hashicorp-ci a0f27e6597 Putting source back into Dev Mode 2020-05-21 20:27:47 +00:00
hashicorp-ci a774d9b12f
Release v1.8.0-beta2 2020-05-21 19:33:59 +00:00
hashicorp-ci 7dd0a87286
update bindata_assetfs.go 2020-05-21 19:33:58 +00:00