Commit Graph

159 Commits

Author SHA1 Message Date
Mike Morris 29e4485074 changelog: add entries for secondary datacenter CA fixes (#9322) 2020-12-03 18:34:04 +00:00
John Cowen af5e6e6d12 ui: Add copyable IDs to the Role and Policy views (#9296) 2020-11-30 17:29:06 +00:00
Daniel Nephin 60d7f30169 Merge pull request #9284 from hashicorp/dnephin/agent-service-register
local: mark service as InSync when added to local agent state
2020-11-27 20:50:53 +00:00
Daniel Nephin d230cea541 Merge pull request #9247 from pierresouchay/streaming_predictible_order_for_health
[Streaming] Predictable order for results of /health/service/:serviceName to mimic memdb
2020-11-25 20:55:00 +00:00
Mike Morris 3ee6d1c14f
Merge branch 'release/1.9.x' into release/1.9.0 2020-11-24 14:50:39 -05:00
R.B. Boyer 7467ffbff3 server: fix panic when deleting a non existent intention (#9254)
* server: fix panic when deleting a non existent intention

* add changelog

* Always return an error when deleting non-existent ixn

Co-authored-by: freddygv <gh@freddygv.xyz>
2020-11-24 18:44:58 +00:00
R.B. Boyer 3c7cf0216d server: fix panic when deleting a non existent intention (#9254)
* server: fix panic when deleting a non existent intention

* add changelog

* Always return an error when deleting non-existent ixn

Co-authored-by: freddygv <gh@freddygv.xyz>
2020-11-24 13:44:45 -05:00
Mike Morris 52e5a2fb32
changelog: 1.9.0 (#9265)
* changelog: add post-rc1 entries

* changelog: regenerate entries from LAST_RELEASE_GIT_TAG=v1.8.4, remove beta releases

* changelog: tweak categories for a few entries and add Go 1.15 note

* changelog: apply category changes to CHANGELOG.md

manually remove Go 1.14 upgrade note and two intermediate UI bug fix
entries for the new topology feature
2020-11-24 12:21:43 -05:00
Freddy ff5215d882 Require operator:write to get Connect CA config (#9240)
A vulnerability was identified in Consul and Consul Enterprise (“Consul”) such that operators with `operator:read` ACL permissions are able to read the Consul Connect CA configuration when explicitly configured with the `/v1/connect/ca/configuration` endpoint, including the private key. This allows the user to effectively privilege escalate by enabling the ability to mint certificates for any Consul Connect services. This would potentially allow them to masquerade (receive/send traffic) as any service in the mesh.

--

This PR increases the permissions required to read the Connect CA's private key when it was configured via the `/connect/ca/configuration` endpoint. They are now `operator:write`.
2020-11-23 06:27:20 -07:00
R.B. Boyer 140c220131
[1.9.0] command: when generating envoy bootstrap configs use the datacenter returned from the agent services endpoint (#9230)
Manual backport of #9229 into 1.9.0 branch

Fixes #9215
2020-11-19 15:33:41 -06:00
R.B. Boyer 32f6d17e5d command: when generating envoy bootstrap configs use the datacenter returned from the agent services endpoint (#9229)
Fixes #9215
2020-11-19 21:28:09 +00:00
Freddy 5137e4501d Require operator:write to get Connect CA config (#9240)
A vulnerability was identified in Consul and Consul Enterprise (“Consul”) such that operators with `operator:read` ACL permissions are able to read the Consul Connect CA configuration when explicitly configured with the `/v1/connect/ca/configuration` endpoint, including the private key. This allows the user to effectively privilege escalate by enabling the ability to mint certificates for any Consul Connect services. This would potentially allow them to masquerade (receive/send traffic) as any service in the mesh.

--

This PR increases the permissions required to read the Connect CA's private key when it was configured via the `/connect/ca/configuration` endpoint. They are now `operator:write`.
2020-11-19 17:15:17 +00:00
Mike Morris f3108c4901 changelog: fixup changelog.tmpl formatting 2020-11-17 11:37:52 -05:00
Kenia 64bf6d9ca7 ui: Changelog changes (#9209) 2020-11-17 11:15:35 -05:00
Matt Keeler dfaaa0b73a Refactor to call non-voting servers read replicas (#9191)
Co-authored-by: Kit Patella <kit@jepsen.io>
2020-11-17 15:54:38 +00:00
Kenia d3e379b712 ui: Changelog changes (#9209) 2020-11-17 15:39:32 +00:00
Freddy ef7ee6840a Add DC and NS support for Envoy metrics (#9207)
This PR updates the tags that we generate for Envoy stats.

Several of these come with breaking changes, since we can't keep two stats prefixes for a filter.
2020-11-16 19:55:18 -07:00
Kit Patella 88b013be99 Merge pull request #9198 from hashicorp/mkcp/telemetry/add-all-metric-definitions
Add metric definitions for all metrics known at Consul start
2020-11-16 16:26:16 -08:00
Kit Patella 82e7363b90 Merge pull request #9198 from hashicorp/mkcp/telemetry/add-all-metric-definitions
Add metric definitions for all metrics known at Consul start
2020-11-17 00:13:51 +00:00
Freddy 4d39305442 Add DC and NS support for Envoy metrics (#9207)
This PR updates the tags that we generate for Envoy stats.

Several of these come with breaking changes, since we can't keep two stats prefixes for a filter.
2020-11-16 23:37:50 +00:00
Matt Keeler dd857bfa37
Prevent panic if autopilot health is requested prior to leader establishment finishing. (#9204) 2020-11-16 17:14:56 -05:00
Matt Keeler acb44bb3b5
Add changelog entry for namespace licensing fix (#9203) 2020-11-16 17:14:45 -05:00
Matt Keeler e421da3b59 Prevent panic if autopilot health is requested prior to leader establishment finishing. (#9204) 2020-11-16 22:08:44 +00:00
Matt Keeler c88ada194f Add changelog entry for namespace licensing fix (#9203) 2020-11-16 20:46:29 +00:00
Kit Patella 07c0179bf8 Merge pull request #9195 from hashicorp/mkcp/changelog/add-1dot9-metrics-flag-note
add note about future metric fixes and deprecations under disable_com…
2020-11-13 22:46:14 +00:00
R.B. Boyer fee0c44ab2 server: remove config entry CAS in legacy intention API bridge code (#9151)
Change so line-item intention edits via the API are handled via the state store instead of via CAS operations.

Fixes #9143
2020-11-13 20:42:57 +00:00
R.B. Boyer a955705e5e server: skip deleted and deleting namespaces when migrating intentions to config entries (#9186) 2020-11-13 19:57:12 +00:00
R.B. Boyer d69640a6e9 server: break up Intention.Apply monolithic method (#9007)
The Intention.Apply RPC is quite large, so this PR attempts to break it down into smaller functions and dissolves the pre-config-entry approach to the breakdown as it only confused things.
2020-11-13 15:16:34 +00:00
R.B. Boyer f815014432 agent: return the default ACL policy to callers as a header (#9101)
Header is: X-Consul-Default-ACL-Policy=<allow|deny>

This is of particular utility when fetching matching intentions, as the
fallthrough for a request that doesn't match any intentions is to
enforce using the default acl policy.
2020-11-12 16:39:16 +00:00
Matt Keeler cbf788b649 Add changelog entry for autopilot state CLI (#9161) 2020-11-11 19:55:45 +00:00
Mike Morris 9c989fef4d
Merge pull request #9155 from hashicorp/release/1.9.0-beta3
merge: 1.9.0-beta3
2020-11-11 12:55:23 -05:00
Matt Keeler e669899abf Add a paramter in state store methods to indicate whether a resource insertion is from a snapshot restoration (#9156)
The Catalog, Config Entry, KV and Session resources potentially re-validate the input as its coming in. We need to prevent snapshot restoration failures due to missing namespaces or namespaces that are being deleted in enterprise.
2020-11-11 16:22:11 +00:00
Matt Keeler be33212046 Fixup the autopilot changelog (#9145) 2020-11-09 17:30:22 -05:00
Matt Keeler 0b7e14552e Fixup the autopilot changelog (#9145) 2020-11-09 22:29:31 +00:00
Mike Morris 508e15b7bf changelog: add entry for Go 1.14.11 update 2020-11-09 16:07:38 -05:00
Matt Keeler f2dee21aca Add some autopilot docs and update the changelog (#9139) 2020-11-09 19:15:12 +00:00
Mike Morris 1bf84a768f connect: switch the default gateway port from 443 to 8443 (#9116)
* test: update ingress gateway golden file to port 8443

* test: update Envoy flags_test to port 8443

Co-authored-by: R.B. Boyer <rb@hashicorp.com>
2020-11-07 01:48:02 +00:00
Mike Morris e66362d7a0 changelog: add entries for 1.9.0-beta2 (#9129) 2020-11-06 22:19:14 +00:00
R.B. Boyer fff77349ed Revert "Add namespace support for metrics (OSS) (#9117)" (#9124)
This reverts commit 06b3b017d3.
2020-11-06 16:57:56 +00:00
Freddy e86f58b163 Add namespace support for metrics (OSS) (#9117) 2020-11-05 18:30:37 -07:00
Mike Morris 5cf9ddedf9
changelog: update to hashicorp/sentinel@v0.16.0 in Enterprise (#8984) 2020-10-26 12:32:44 -04:00
Mike Morris 4fdc69842d Merge pull request #9027 from hashicorp/release/1.8.5
Merge back release/1.8.5
2020-10-26 10:59:48 -04:00
R.B. Boyer 58387fef0a
server: config entry replication now correctly uses namespaces in comparisons (#9024)
Previously config entries sharing a kind & name but in different
namespaces could occasionally cause "stuck states" in replication
because the namespace fields were ignored during the differential
comparison phase.

Example:

Two config entries written to the primary:

    kind=A,name=web,namespace=bar
    kind=A,name=web,namespace=foo

Under the covers these both get saved to memdb, so they are sorted by
all 3 components (kind,name,namespace) during natural iteration. This
means that before the replication code does it's own incomplete sort,
the underlying data IS sorted by namespace ascending (bar comes before
foo).

After one pass of replication the primary and secondary datacenters have
the same set of config entries present. If
"kind=A,name=web,namespace=bar" were to be deleted, then things get
weird. Before replication the two sides look like:

primary: [
    kind=A,name=web,namespace=foo
]
secondary: [
    kind=A,name=web,namespace=bar
    kind=A,name=web,namespace=foo
]

The differential comparison phase walks these two lists in sorted order
and first compares "kind=A,name=web,namespace=foo" vs
"kind=A,name=web,namespace=bar" and falsely determines they are the SAME
and are thus cause an update of "kind=A,name=web,namespace=foo". Then it
compares "<nothing>" with "kind=A,name=web,namespace=foo" and falsely
determines that the latter should be DELETED.

During reconciliation the deletes are processed before updates, and so
for a brief moment in the secondary "kind=A,name=web,namespace=foo" is
erroneously deleted and then immediately restored.

Unfortunately after this replication phase the final state is identical
to the initial state, so when it loops around again (rate limited) it
repeats the same set of operations indefinitely.
2020-10-23 13:41:54 -05:00
Daniel Nephin 3a55c30a05
Merge pull request #8924 from ShimmerGlass/fix-sidecar-deregister-after-restart
Fix: service LocallyRegisteredAsSidecar property is not persisted
2020-10-22 13:26:55 -04:00
Daniel Nephin 8b601fdcac
Merge pull request #8771 from amenzhinsky/fix-grpc-use-tls-mapping
Fix GRPCUseTLS flag HTTP API mapping
2020-10-21 18:37:11 -04:00
Daniel Nephin 97db15edb4 Add changelog entry 2020-10-20 16:42:06 -04:00
Preetha ccb3f4e67e
Merge pull request #8947 from hashicorp/dnephin/changelog-for-streaming
Add streaming changelog file
2020-10-14 09:42:10 -05:00
R.B. Boyer 3b71b5e415
fix 1.9.0-beta1 changelog formatting (#8941) 2020-10-14 09:35:59 -05:00
Daniel Nephin e9d50433ba Add streaming changelog file 2020-10-13 18:16:33 -04:00
Mathilde Gilles 1c8369b3c3 Fix: service LocallyRegisteredAsSidecar property is not persisted
When a service is deregistered, we check whever matching services were
registered as sidecar along with it and deregister them as well.
To determine if a service is indeed a sidecar we check the
structs.ServiceNode.LocallyRegisteredAsSidecar property. However, to
avoid interal API leakage, it is excluded from JSON serialization,
meaning it is not persisted to disk either.
When the agent is restarted, this property lost and sidecars are no
longer deregistered along with their parent service.
To fix this, we now specifically save this property in the persisted
service file.
2020-10-13 19:38:58 +02:00
Mike Morris aacf1fbeab changelog: fixup note.tmpl syntax 2020-10-09 22:44:51 -04:00
Paul Banks 87278a739b
changelog: add entries for ui_config and service metrics config (#8919)
* Create 8694.txt

* Apply suggestions from code review

Co-authored-by: Freddy <freddygv@users.noreply.github.com>

Co-authored-by: Freddy <freddygv@users.noreply.github.com>
2020-10-09 17:31:00 -04:00
Paul Banks 04509a4004
changelog: add entries for UI topology viz (#8918)
* Create 8858.txt

* add separate changelog entries for original topology impl and intentions

Co-authored-by: Mike Morris <mikemorris@users.noreply.github.com>
2020-10-09 17:29:14 -04:00
Pierre Souchay 9b7ed75552
Consul Service meta wrongly computes and exposes non_voter meta (#8731)
* Consul Service meta wrongly computes and exposes non_voter meta

In Serf Tags, entreprise members being non-voters use the tag
`nonvoter=1`, not `non_voter = false`, so non-voters in members
were wrongly displayed as voter.

Demonstration:

```
consul members -detailed|grep voter
consul20-hk5 10.200.100.110:8301   alive   acls=1,build=1.8.4+ent,dc=hk5,expect=3,ft_fs=1,ft_ns=1,id=xxxxxxxx-5629-08f2-3a79-10a1ab3849d5,nonvoter=1,port=8300,raft_vsn=3,role=consul,segment=<all>,use_tls=1,vsn=2,vsn_max=3,vsn_min=2,wan_join_port=8302
```

* Added changelog

* Added changelog entry
2020-10-09 17:18:24 -04:00
s-christoff 9bb348c6c7
Enhance the output of consul snapshot inspect (#8787) 2020-10-09 14:57:29 -05:00
Kit Patella 5b2833d1a6
Merge pull request #8914 from hashicorp/mkcp/changelog/8877
add changelog entries for 8877
2020-10-09 12:51:23 -07:00
Kit Patella 122e036659 add template generation for entries tagged deprecation 2020-10-09 12:40:41 -07:00
Kit Patella 71f95ab7dd add changelog entries for 8877 2020-10-09 12:38:57 -07:00
Kyle Havlovitz e5ab1b45bc
Merge pull request #8784 from hashicorp/renew-intermediate-primary
connect: Enable renewing the intermediate cert in the primary DC
2020-10-09 12:18:59 -07:00
Hans Hasselberg 3d85793b57
note template with ent support (#8910) 2020-10-09 21:06:49 +02:00
Mike Morris a705695add
changelog: update raft to v1.2.0 (#8901) 2020-10-09 11:28:13 -04:00
Kyle Havlovitz cc901dfd47 Add changelog note 2020-10-09 08:01:55 -07:00
Matt Keeler 219515e519
Create _619.txt 2020-10-09 10:51:37 -04:00
Matt Keeler 8f890bc027
Add capability for the v1/connect/ca/roots endpoint to return a PEM encoded certificate chain (#8774)
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
2020-10-09 10:43:33 -04:00
R.B. Boyer 46a4ef7248
agent: allow the /v1/connect/intentions/match endpoint to use the agent cache (#8875)
This is the recommended proxy integration API for listing intentions
which should not require an active connection to the servers to resolve
after the initial cache filling.
2020-10-08 14:51:53 -05:00
Matt Keeler 38f5ddce2a
Add per-agent reconnect timeouts (#8781)
This allows for client agent to be run in a more stateless manner where they may be abruptly terminated and not expected to come back. If advertising a per-agent reconnect timeout using the advertise_reconnect_timeout configuration when that agent leaves, other agents will wait only that amount of time for the agent to come back before reaping it.

This has the advantageous side effect of causing servers to deregister the node/services/checks for that agent sooner than if the global reconnect_timeout was used.
2020-10-08 15:02:19 -04:00
R.B. Boyer 9fbcb2e68d
command: remove conditional envoy bootstrap generation for versions <=1.10.0 since those are not supported (#8855) 2020-10-07 10:53:23 -05:00
R.B. Boyer 3e6cbc649e add missing changelog entry for #8839 2020-10-07 10:22:40 -05:00
Pierre Souchay eabba09b66 Added changelog for merged PR #8221 2020-10-06 17:15:33 -04:00
R.B. Boyer a2a8e9c783
connect: intentions are now managed as a new config entry kind "service-intentions" (#8834)
- Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older
copies of consul will not see new config entries it doesn't understand
replicate down.

- Add shim conversion code so that the old API/CLI method of interacting
with intentions will continue to work so long as none of these are
edited via config entry endpoints. Almost all of the read-only APIs will
continue to function indefinitely.

- Add new APIs that operate on individual intentions without IDs so that
the UI doesn't need to implement CAS operations.

- Add a new serf feature flag indicating support for
intentions-as-config-entries.

- The old line-item intentions way of interacting with the state store
will transparently flip between the legacy memdb table and the config
entry representations so that readers will never see a hiccup during
migration where the results are incomplete. It uses a piece of system
metadata to control the flip.

- The primary datacenter will begin migrating intentions into config
entries on startup once all servers in the datacenter are on a version
of Consul with the intentions-as-config-entries feature flag. When it is
complete the old state store representations will be cleared. We also
record a piece of system metadata indicating this has occurred. We use
this metadata to skip ALL of this code the next time the leader starts
up.

- The secondary datacenters continue to run the old intentions
replicator until all servers in the secondary DC and primary DC support
intentions-as-config-entries (via serf flag). Once this condition it met
the old intentions replicator ceases.

- The secondary datacenters replicate the new config entries as they are
migrated in the primary. When they detect that the primary has zeroed
it's old state store table it waits until all config entries up to that
point are replicated and then zeroes its own copy of the old state store
table. We also record a piece of system metadata indicating this has
occurred. We use this metadata to skip ALL of this code the next time
the leader starts up.
2020-10-06 13:24:05 -05:00
R.B. Boyer 4998a08c56
server: create new memdb table for storing system metadata (#8703)
This adds a new very tiny memdb table and corresponding raft operation
for updating a very small effective map[string]string collection of
"system metadata". This can persistently record a fact about the Consul
state machine itself.

The first use of this feature will come in a later PR.
2020-10-06 10:08:37 -05:00
R.B. Boyer 9801ef8eb1
agent: enable enable_central_service_config by default (#8746) 2020-10-01 09:19:14 -05:00
R.B. Boyer 237a7a0da0
server: ensure that we also shutdown network segment serf instances on server shutdown (#8786)
This really only matters for unit tests, since typically if an agent shuts down its server, it follows that up by exiting the process, which would also clean up all of the networking anyway.
2020-09-30 16:23:43 -05:00
R.B. Boyer d2eb27e0a3
api: support GetMeta() and GetNamespace() on all config entry kinds (#8764)
Fixes #8755

Since I was updating the interface, i also added the missing `GetNamespace()`.

Depending upon how you look at it, this is a breaking change since it adds methods to the exported interface `api.ConfigEntry`. Given that you cannot define your own config entry kinds, and all of the machinery of the `api.Client` acts like a factory to construct the canned ones from the rest of the module, this feels like it's not a problematic change as it would only break someone who had reimplemented the `ConfigEntry` interface themselves for no apparent utility?
2020-09-29 09:11:57 -05:00
Daniel Nephin 6200325e3b
Merge pull request #8726 from amenzhinsky/grpc-hc-error
Return grpc serving status in health check errors
2020-09-25 13:24:32 -04:00
Daniel Nephin edac9f943f Add changelog file 2020-09-25 12:03:49 -04:00
R.B. Boyer 7eef25daf5
agent: when enable_central_service_config is enabled ensure agent reload doesn't revert check state to critical (#8747)
Likely introduced when #7345 landed.
2020-09-24 16:24:04 -05:00
R.B. Boyer 0064f1936e
server: make sure that the various replication loggers use consistent logging (#8745) 2020-09-24 15:49:38 -05:00
R.B. Boyer 0fb088aac3
agent: make the json/hcl decoding of ConnectProxyConfig fully work with CamelCase and snake_case (#8741)
Fixes #7418
2020-09-24 13:58:52 -05:00
Hans Hasselberg a89ee1a7ca
use service datacenter for dns name (#8704)
* Use args.Datacenter instead of configured datacenter
2020-09-22 20:34:09 +02:00
Kyle Havlovitz 1d22a0bc51
Merge pull request #8560 from hashicorp/vault-ca-renew-token
Automatically renew the token used by the Vault CA provider
2020-09-16 07:30:30 -07:00
Daniel Nephin 3995cc3408
Merge pull request #8685 from pierresouchay/do_not_flood_logs_with_Non-server_in_server-only_area
[BUGFIX] Avoid GetDatacenter* methods to flood Consul servers logs
2020-09-15 17:57:05 -04:00
Daniel Nephin 898d845257
Update .changelog/8685.txt 2020-09-15 17:56:06 -04:00
Kyle Havlovitz b1b21139ca Merge branch 'master' into vault-ca-renew-token 2020-09-15 14:39:04 -07:00
Kyle Havlovitz 1cd7c43544 Update vault CA for latest api client 2020-09-15 13:33:55 -07:00
Kyle Havlovitz 35bb09f85c
Merge pull request #8646 from hashicorp/common-intermediate-ttl
Move IntermediateCertTTL to common CA config
2020-09-15 12:03:29 -07:00
Kyle Havlovitz 04bf7373ec
Create 8646.txt 2020-09-15 10:05:23 -07:00
Pierre Souchay 4797a9c516 Added Changelog for PR #8685. 2020-09-15 12:30:12 +02:00
Freddy a24a60ec75
Create 8585.txt 2020-09-14 14:16:47 -06:00
Hans Hasselberg 5f5aa801db
add entry for 8588 (#8650) 2020-09-10 18:53:36 +02:00
Tim Arenz a1fe711390
Add support for -ca-path option in the connect envoy command (#8606)
* Add support for -ca-path option in the connect envoy command
* Adding changelog entry
2020-09-08 12:16:16 +02:00
Seth Hoenig 9fab3fe990
api: create fresh http client for unix sockets (#8602)
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
2020-09-06 12:27:39 -04:00
Freddy cd9398aedc
Adds changelog entry for snapshot agent improvement (#8622) 2020-09-04 14:07:57 -06:00
R.B. Boyer 119e945c3e
connect: all config entries pick up a meta field (#8596)
Fixes #8595
2020-09-02 14:10:25 -05:00
Chris Piraino 28f163c2d2
Merge pull request #8603 from hashicorp/feature/usage-metrics
Track node and service counts in the state store and emit them periodically as metrics
2020-09-02 13:23:39 -05:00
Chris Piraino 40cbd5a8f3 Changelog entry for usage metrics 2020-09-02 10:48:11 -05:00
R.B. Boyer d0f74cd1e8
connect: fix bug in preventing some namespaced config entry modifications (#8601)
Whenever an upsert/deletion of a config entry happens, within the open
state store transaction we speculatively test compile all discovery
chains that may be affected by the pending modification to verify that
the write would not create an erroneous scenario (such as splitting
traffic to a subset that did not exist).

If a single discovery chain evaluation references two config entries
with the same kind and name in different namespaces then sometimes the
upsert/deletion would be falsely rejected. It does not appear as though
this bug would've let invalid writes through to the state store so the
correction does not require a cleanup phase.
2020-09-02 10:47:19 -05:00
Daniel Nephin a0d7615a7f
Merge pull request #8577 from hashicorp/dnephin/changelog-for-8537
Add missing changelog for PR 8537
2020-08-31 11:56:20 -04:00
Pierre Souchay 242c8dac6f Added changelog for #8552 2020-08-28 23:01:04 +02:00
Jack 9e1c6727f9
Add http2 and grpc support to ingress gateways (#8458) 2020-08-27 15:34:08 -06:00