Commit Graph

10545 Commits

Author SHA1 Message Date
R.B. Boyer 8c1b3f6a6e update changelog 2019-10-17 16:45:47 -05:00
R.B. Boyer 8dcba472a2
xds: tcp services using the discovery chain should not assume RDS during LDS (#6623)
Previously the logic for configuring RDS during LDS for L7 upstreams was
overapplied to TCP proxies resulting in a cluster name of <emptystring>
being used incorrectly.

Fixes #6621
2019-10-17 16:44:59 -05:00
Freddy 895a82ed78
Update CHANGELOG.md 2019-10-17 20:38:51 +02:00
Freddy 60f6ec0c2f
Store check type in catalog (#6561) 2019-10-17 20:33:11 +02:00
R.B. Boyer 7a13850db2 update changelog 2019-10-17 11:02:05 -05:00
R.B. Boyer 274b5a46b3
sdk: ignore panics due to stray goroutines logging after a test completes (#6632)
If there is imperfect goroutine lifespan tracking if we pipe our logs
through testing.T.Logf there is a chance of a stray goroutine attempting
to log after the test that spawned it completes.

This results in a panic of:

    panic: Log in goroutine after TestLeader_SecondaryCA_Initialize has completed...

This isn't great and should be fixed, but quickly runs into situations
around externally cancelling blocking queries which isn't terribly
possible at the moment. The concession here is to ignore these specific
panics for now.

This can be triggered easily when running some tests with a high
`-count=YYY` value.
2019-10-17 11:01:11 -05:00
R.B. Boyer 3c58d29fad update changelog 2019-10-17 10:58:22 -05:00
R.B. Boyer de6ce5b1d9
server: ensure the primary dc and ACL dc match (#6634)
This is mostly a sanity check for server tests that skip the normal
config builder equivalent fixup.
2019-10-17 10:57:17 -05:00
R.B. Boyer 3aeb740430
unflake TestLeader_SecondaryCA_Initialize (#6631) 2019-10-16 16:49:01 -05:00
kaitlincarter-hc 4f9e639d49
updating broken link (#6633) 2019-10-16 16:18:39 -05:00
R.B. Boyer e6bfcb0ca8
fix flaky multidc acl tests that failed to wait for token replication (#6628)
If acls have not yet replicated to the secondary then authz requests
will be remotely resolved by the primary. Now these tests explicitly
wait until replication has caught up first.
2019-10-16 12:24:29 -05:00
R.B. Boyer 040f47c46e
appease the retry linter (#6629) 2019-10-16 11:39:22 -05:00
Matt Keeler 7f00fc60f1
Dependency: Update github.com/hashicorp/go-memdb to v1.0.3 (#6626) 2019-10-16 12:10:12 -04:00
Paul Banks d7aa425339
Allow time for secondary CA to initialize (#6627) 2019-10-16 17:03:31 +01:00
Matt Keeler 973341a592
ACL Authorizer overhaul (#6620)
* ACL Authorizer overhaul

To account for upcoming features every Authorization function can now take an extra *acl.EnterpriseAuthorizerContext. These are unused in OSS and will always be nil.

Additionally the acl package has received some thorough refactoring to enable all of the extra Consul Enterprise specific authorizations including moving sentinel enforcement into the stubbed structs. The Authorizer funcs now return an acl.EnforcementDecision instead of a boolean. This improves the overall interface as it makes multiple Authorizers easily chainable as they now indicate whether they had an authoritative decision or should use some other defaults. A ChainedAuthorizer was added to handle this Authorizer enforcement chain and will never itself return a non-authoritative decision.

* Include stub for extra enterprise rules in the global management policy

* Allow for an upgrade of the global-management policy
2019-10-15 16:58:50 -04:00
Paul Banks 6d645fe53c
Update CHANGELOG.md 2019-10-14 21:54:15 +01:00
PHBourquin 039615641e Checks to passing/critical only after reaching a consecutive success/failure threshold (#5739)
A check may be set to become passing/critical only if a specified number of successive
checks return passing/critical in a row. Status will stay identical as before until
the threshold is reached.
This feature is available for HTTP, TCP, gRPC, Docker & Monitor checks.
2019-10-14 21:49:49 +01:00
kaitlincarter-hc b0310364c6
[docs] Adding Links to Learn (#6611)
* adding links to Learn

* fixing a couple typos

* adding utm paramaters

* Update website/source/docs/connect/registration/sidecar-service.md

* Update website/source/docs/connect/registration/sidecar-service.md

* Update website/source/docs/acl/acl-system.html.md

* Update website/source/docs/acl/acl-system.html.md

* Update website/source/docs/agent/encryption.html.md

Co-Authored-By: Judith Malnick <judith.patudith@gmail.com>

* Update website/source/docs/connect/proxies/built-in.md

Co-Authored-By: Judith Malnick <judith.patudith@gmail.com>

* Update website/source/docs/connect/registration/sidecar-service.md

Co-Authored-By: Judith Malnick <judith.patudith@gmail.com>

* Update website/source/docs/install/index.html.md

Co-Authored-By: Judith Malnick <judith.patudith@gmail.com>

* Update website/source/docs/agent/kv.html.md

* Update website/source/docs/connect/security.html.md

* Update website/source/docs/connect/security.html.md

* Update website/source/docs/internals/architecture.html.md
2019-10-14 10:40:35 -05:00
Blake Covarrubias 97953454c4 Add Consul's L7 features to Istio comparison
Add text listing Consul's L7 features (via Envoy). Re-organize text to
flow similarly to Istio section.

Co-Authored-By: Judith Malnick <judith.patudith@gmail.com>
2019-10-10 11:33:34 -07:00
Sarah Christoff 194f5740ce
ui_content_path config option fix (#6601)
* fix ui-content-path config option
2019-10-09 09:14:48 -05:00
Hans Hasselberg b6499fe6b8 Do not surface left servers (#6420)
* do not surface left servers in catalog
2019-10-08 22:16:00 -05:00
Alvin Huang 8d4b8eae38
add go cache to distro builds (#6600) 2019-10-08 15:55:36 -04:00
Alvin Huang 97af2b0929
add circleci and discuss badge (#6599) 2019-10-08 15:55:12 -04:00
Matt Keeler a31e7a5d58
Remove `gox` usage (#6596) 2019-10-08 13:42:29 -04:00
R.B. Boyer 6439af86eb
agent: clients should only attempt to remove pruned nodes once per call (#6591) 2019-10-07 16:15:23 -05:00
Matt Keeler 7713d4a988
Update CHANGELOG.md 2019-10-07 16:43:09 -04:00
Matt Keeler 4a5ebb57b0
Add support for the Namespace HTTP API in the API Client (#6581) 2019-10-07 15:19:38 -04:00
Luke Kysow 17990aabf3
Document new annotations for Connect injections 2019-10-04 15:31:24 -07:00
Sarah Christoff 53f3962cab
Update CHANGELOG.md 2019-10-04 16:12:32 -05:00
Sarah Christoff 5e26971864
Prune Unhealthy Agents (#6571)
* Add -prune flag to ForceLeave
2019-10-04 16:10:02 -05:00
R.B. Boyer 4d8e5a18bc update changelog 2019-10-04 13:39:44 -05:00
R.B. Boyer 06ca8cd2d7
agent: updates to the agent token trigger anti-entropy full syncs (#6577) 2019-10-04 13:37:34 -05:00
Matt Keeler d65bbbfd4e
Implement Leader Routine Management (#6580)
* Implement leader routine manager

Switch over the following to use it for go routine management:

• Config entry Replication
• ACL replication - tokens, policies, roles and legacy tokens
• ACL legacy token upgrade
• ACL token reaping
• Intention Replication
• Secondary CA Roots Watching
• CA Root Pruning

Also added the StopAll call into the Server Shutdown method to ensure all leader routines get killed off when shutting down.

This should be mostly unnecessary as `revokeLeadership` should manually stop each one but just in case we really want these to go away (eventually).
2019-10-04 13:08:45 -04:00
Matt Keeler 28221f66f2
Use encoding/json instead of jsonpb even for protobuf types (#6572)
This only works so long as we use simplistic protobuf types. Constructs such as oneof or Any types that require type annotations for decoding properly will fail hard but that is by design. If/when we want to use any of that we will probably need to consider a v2 API.
2019-10-02 15:32:15 -04:00
Alvin Huang b52cd5c8cd
check for go binary in ci builds (#6570) 2019-10-02 14:32:44 -04:00
Matt Keeler fc4bcfd81f
Add EnterpriseConfig stubs (#6566) 2019-10-01 14:34:55 -04:00
Matt Keeler abed91d069
Generate JSON and Binary Marshalers for Protobuf Types (#6564)
* Add JSON and Binary Marshaler Generators for Protobuf Types

* Generate files with the correct version of gogo/protobuf

I have pinned the version in the makefile so when you run make tools you get the right version. This pulls the version out of go.mod so it should remain up to date.

The version at the time of this commit we are using is v1.2.1

* Fixup some shell output

* Update how we determine the version of gogo
This just greps the go.mod file instead of expecting the go mod cache to already be present

* Fixup vendoring and remove no longer needed json encoder functions
2019-09-30 15:39:20 -04:00
John Cowen b3b32dc0f6
ui: UI Release Merge (ui-staging merge) (#6527)
## HTTPAdapter (#5637)

## Ember upgrade 2.18 > 3.12 (#6448)

### Proxies can no longer get away with not calling _super

This means that we can't use create anymore to define dynamic methods.
Therefore we dynamically make 2 extended Proxies on demand, and then
create from those. Therefore we can call _super in the init method of
the extended Proxies.

### We aren't allowed to reset a service anymore

We never actually need to now anyway, this is a remnant of the refactor
from browser based confirmations. We fix it as simply as possible here
but will revisit and remove the old browser confirm functionality at a
later date

### Revert classes to use ES5 style to workaround babel transp. probs

Using a mixture of ES6 classes (and hence super) and arrow functions
means that when babel transpiles the arrow functions down to ES5, a
reference to this is moved before the call to super, hence causing a js
error.

Furthermore, we the testing environment no longer lets use use
apply/call on the constructor.

These errors only manifests during testing (only in the testing
environment), the application itself runs fine with no problems without
this change.

Using ES5 style class definitions give us freedom to do all of the above
without causing any errors, so we reverted these classes back to ES5
class definitions

### Skip test that seems to have changed due to a change in RSVP timing

This test tests a usecase/area of the API that will probably never ever
be used, it was more testing out the API. We've skipped the test for now
as this doesn't affect the application itself, but left a note to come
back here later to investigate further

### Remove enumerableContentDidChange

Initial testing looks like we don't need to call this function anymore,
the function no longer exists

### Rework Changeset.isSaving to take into account new ember APIs

Setting/hanging a computedProperty of an instantiated object no longer
works. Move to setting it on the prototype/class definition instead

### Change how we detect whether something requires listening

New ember API's have changed how you can detect whether something is a
computedProperty or not. It's not immediately clear if its even possible
now. Therefore we change how we detect whether something should be
listened to or not by just looking for presence of `addEventListener`

### Potentially temporary change of ci test scripts to ensure deps exist

All our tooling scripts run through a Makefile (for people familiar with
only using those), which then call yarn scripts which can be called
independently (for people familar with only using yarn).

The Makefile targets always check to make sure all the dependencies are
installed before running anything that requires them (building, testing
etc).

The CI scripts/targets didn't follow this same route and called the yarn
scripts directly (usually CI builds a cache of the dependencies first).

For some reason this cache isn't doing what it usually does, and it
looks as though, in CI, ember isn't installed.

This commit makes the CI scripts consistently use the same method as all
of the other tooling scripts (Makefile target > Install Deps if
required > call yarn script). This should install the dependencies if
for some reason the CI cache building doesn't complete/isn't successful.

Potentially this commit may be reverted if, the root of the problem is
elsewhere, although consistency is always good, so it might be a good
idea to leave this commit as is even if we need to debug and fix things
elsewhere.

### Make test-parallel consistent with the rest of the tooling scripts

As we are here making changes for CI purposes (making test-ci
consistent), we spotted that test-parallel is also inconsistent and also
the README manual instructions won't work without `ember` installed
globally.

This commit makes everything consistent and changes the manual
instructions to use the local ember instance that gets installed via
yarn

### Re-wrangle catchable to fit with new ember 3.12 APIs

In the upgrade from ember 3.8 > 3.12 the public interfaces for
ComputedProperties have changed slightly. `meta` is no longer a public
property of ComputedProperty but of a ComputedDecoratorImpl mixin
instead.

7e4ba1096e/packages/%40ember/-internals/metal/lib/computed.ts (L725)

There seems to be no way, by just using publically available
methods, to replicate this behaviour so that we can create our own
'ComputedProperty` factory via injecting the ComputedProperty class as
we did previously.

3f333bada1/ui-v2/app/utils/computed/factory.js (L1-L18)

Instead we dynamically hang our `Catchable` `catch` method off the
instantiated ComputedProperty. In doing it like this `ComputedProperty`
has already has its `meta` method mixed in so we don't have to manually
mix it in ourselves (which doesn't seem possible)

This functionality is only used during our work in trying to ensure
our EventSource/BlockingQuery work was as 'ember-like' as possible (i.e.
using the traditional Route.model hooks and ember-like Controller
properties). Our ongoing/upcoming work on a componentized approach to
data a.k.a `<DataSource />` means we will be able to remove the majority
of the code involved here now that it seems to be under an amount of
flux in ember.

### Build bindata_assetfs.go with new UI changes
2019-09-30 14:47:49 +01:00
Freddy 2eaece10a7
Update Force Leave docs (#6550)
Fixes #2742

Previously the docs didn't clarify that if a server restarts as a client then force-leave won't lead to removing the node from the raft config. This is because the node, which is alive after a restart, will refute messages about it having left . These messages about members leaving are in turn what trigger Consul's leader to remove a server from raft.
2019-09-27 17:49:28 -06:00
Matt Keeler 923d8671a4
Add support for parameterizing the ACL config used with a TestA… (#6559)
* Add support for parameterizing the ACL config used with a TestAgent

Using tokens that are UUIDs will get rid of some warnings

* Refactor to allow setting all tokens and change the template to ignore unset values.
2019-09-27 17:06:43 -04:00
Mike Morris 6bf75d201d
docs: add managed proxy removal note to specific version upgrade notes (#6557) 2019-09-27 10:52:47 -04:00
Judith Malnick fd41003138
Specify that mesh gateways require Envoy (#6506)
* Specify that mesh gateways must operate on L7

* Add feedback from Matt

* clarify gateway requirements
2019-09-26 20:06:58 -07:00
Freddy 82a8051b0d
Update CHANGELOG.md 2019-09-26 17:10:21 -06:00
R.B. Boyer e096b575ad update changelog 2019-09-26 11:55:15 -05:00
R.B. Boyer c4b92d5534
connect: connect CA Roots in secondary datacenters should use a SigningKeyID derived from their local intermediate (#6513)
This fixes an issue where leaf certificates issued in secondary
datacenters would be reissued very frequently (every ~20 seconds)
because the logic meant to detect root rotation was errantly triggering
because a hash of the ultimate root (in the primary) was being compared
against a hash of the local intermediate root (in the secondary) and
always failing.
2019-09-26 11:54:14 -05:00
Freddy 2274f64cab
Add Namespace as an api query/write option (#6551) 2019-09-26 10:05:13 -06:00
R.B. Boyer 78366c5830 update changelog 2019-09-26 10:45:32 -05:00
R.B. Boyer 9566df524e
agent: cache notifications work after error if the underlying RPC returns index=1 (#6547)
Fixes #6521

Ensure that initial failures to fetch an agent cache entry using the
notify API where the underlying RPC returns a synthetic index of 1
correctly recovers when those RPCs resume working.

The bug in the Cache.notifyBlockingQuery used to incorrectly "fix" the
index for the next query from 0 to 1 for all queries, when it should
have not done so for queries that errored.

Also fixed some things that made debugging difficult:

- config entry read/list endpoints send back QueryMeta headers
- xds event loops don't swallow the cache notification errors
2019-09-26 10:42:17 -05:00
Matt Keeler 76cf54068b
Expand the QueryOptions and QueryMeta interfaces (#6545)
In a previous PR I made it so that we had interfaces that would work enough to allow blockingQueries to work. However to complete this we need all fields to be settable and gettable.

Notes:
   • If Go ever gets contracts/generics then we could get rid of all the Getters/Setters
   • protoc / protoc-gen-gogo are going to generate all the getters for us.
   • I copied all the getters/setters from the protobuf funcs into agent/structs/protobuf_compat.go
   • Also added JSON marshaling funcs that use jsonpb for protobuf types.
2019-09-26 09:55:02 -04:00
Freddy fdd10dd8b8
Expose HTTP-based paths through Connect proxy (#6446)
Fixes: #5396

This PR adds a proxy configuration stanza called expose. These flags register
listeners in Connect sidecar proxies to allow requests to specific HTTP paths from outside of the node. This allows services to protect themselves by only
listening on the loopback interface, while still accepting traffic from non
Connect-enabled services.

Under expose there is a boolean checks flag that would automatically expose all
registered HTTP and gRPC check paths.

This stanza also accepts a paths list to expose individual paths. The primary
use case for this functionality would be to expose paths for third parties like
Prometheus or the kubelet.

Listeners for requests to exposed paths are be configured dynamically at run
time. Any time a proxy, or check can be registered, a listener can also be
created.

In this initial implementation requests to these paths are not
authenticated/encrypted.
2019-09-25 20:55:52 -06:00