Commit Graph

17430 Commits

Author SHA1 Message Date
Karl Cardenas 1db1905cb5
docs: fixes broken url in acl overview page 2022-04-04 09:47:15 -07:00
Dhia Ayachi 319fe48561
documentation for config auto reload feature (#12548)
* add config watcher to the config package

* add logging to watcher

* add test and refactor to add WatcherEvent.

* add all API calls and fix a bug with recreated files

* add tests for watcher

* remove the unnecessary use of context

* Add debug log and a test for file rename

* use inode to detect if the file is recreated/replaced and only listen to create events.

* tidy ups (#1535)

* tidy ups

* Add tests for inode reconcile

* fix linux vs windows syscall

* fix linux vs windows syscall

* fix windows compile error

* increase timeout

* use ctime ID

* remove remove/creation test as it's a use case that fail in linux

* fix linux/windows to use Ino/CreationTime

* fix the watcher to only overwrite current file id

* fix linter error

* fix remove/create test

* set reconcile loop to 200 Milliseconds

* fix watcher to not trigger event on remove, add more tests

* on a remove event try to add the file back to the watcher and trigger the handler if success

* fix race condition

* fix flaky test

* fix race conditions

* set level to info

* fix when file is removed and get an event for it after

* fix to trigger handler when we get a remove but re-add fail

* fix error message

* add tests for directory watch and fixes

* detect if a file is a symlink and return an error on Add

* rename Watcher to FileWatcher and remove symlink deref

* add fsnotify@v1.5.1

* fix go mod

* do not reset timer on errors, rename OS specific files

* rename New func

* events trigger on write and rename

* add missing test

* fix flaking tests

* fix flaky test

* check reconcile when removed

* delete invalid file

* fix test to create files with different mod time.

* back date file instead of sleeping

* add watching file in agent command.

* fix watcher call to use new API

* add configuration and stop watcher when server stop

* add certs as watched files

* move FileWatcher to the agent start instead of the command code

* stop watcher before replacing it

* save watched files in agent

* add add and remove interfaces to the file watcher

* fix remove to not return an error

* use `Add` and `Remove` to update certs files

* fix tests

* close events channel on the file watcher even when the context is done

* extract `NotAutoReloadableRuntimeConfig` is a separate struct

* fix linter errors

* add Ca configs and outgoing verify to the not auto reloadable config

* add some logs and fix to use background context

* add tests to auto-config reload

* remove stale test

* add tests to changes to config files

* add check to see if old cert files still trigger updates

* rename `NotAutoReloadableRuntimeConfig` to `StaticRuntimeConfig`

* fix to re add both key and cert file. Add test to cover this case.

* review suggestion

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add check to static runtime config changes

* fix test

* add changelog file

* fix review comments

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* update flag description

Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>

* fix compilation error

* add static runtime config support

* fix test

* fix review comments

* fix log test

* Update .changelog/12329.txt

Co-authored-by: Dan Upton <daniel@floppy.co>

* transfer tests to runtime_test.go

* fix filewatcher Replace to not deadlock.

* avoid having lingering locks

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* split ReloadConfig func

* fix warning message

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* convert `FileWatcher` into an interface

* fix compilation errors

* fix tests

* extract func for adding and removing files

* add a coalesceTimer with a very small timer

* extract coaelsce Timer and add a shim for testing

* add tests to coalesceTimer fix to send remaining events

* set `coalesceTimer` to 1 Second

* support symlink, fix a nil deref.

* fix compile error

* fix compile error

* refactor file watcher rate limiting to be a Watcher implementation

* fix linter issue

* fix runtime config

* fix runtime test

* fix flaky tests

* fix compile error

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* fix agent New to return an error if File watcher New return an error

* add a coalesceTimer with a very small timer

* extract coaelsce Timer and add a shim for testing

* set `coalesceTimer` to 1 Second

* add flag description to agent command docs

* fix link

* add Static runtime config docs

* fix links and alignment

* fix typo

* Revert "add a coalesceTimer with a very small timer"

This reverts commit d9db2fcb8213a81ac761f04b458091409c5fb1ee.

* Revert "extract coaelsce Timer and add a shim for testing"

This reverts commit 0ab86012a415ffeb452acf58e52c9f37c9f49254.

* Apply suggestions from code review

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>
Co-authored-by: Daniel Upton <daniel@floppy.co>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2022-04-04 12:01:38 -04:00
Dhia Ayachi 83720e5737
add a rate limiter to config auto-reload (#12490)
* add config watcher to the config package

* add logging to watcher

* add test and refactor to add WatcherEvent.

* add all API calls and fix a bug with recreated files

* add tests for watcher

* remove the unnecessary use of context

* Add debug log and a test for file rename

* use inode to detect if the file is recreated/replaced and only listen to create events.

* tidy ups (#1535)

* tidy ups

* Add tests for inode reconcile

* fix linux vs windows syscall

* fix linux vs windows syscall

* fix windows compile error

* increase timeout

* use ctime ID

* remove remove/creation test as it's a use case that fail in linux

* fix linux/windows to use Ino/CreationTime

* fix the watcher to only overwrite current file id

* fix linter error

* fix remove/create test

* set reconcile loop to 200 Milliseconds

* fix watcher to not trigger event on remove, add more tests

* on a remove event try to add the file back to the watcher and trigger the handler if success

* fix race condition

* fix flaky test

* fix race conditions

* set level to info

* fix when file is removed and get an event for it after

* fix to trigger handler when we get a remove but re-add fail

* fix error message

* add tests for directory watch and fixes

* detect if a file is a symlink and return an error on Add

* rename Watcher to FileWatcher and remove symlink deref

* add fsnotify@v1.5.1

* fix go mod

* do not reset timer on errors, rename OS specific files

* rename New func

* events trigger on write and rename

* add missing test

* fix flaking tests

* fix flaky test

* check reconcile when removed

* delete invalid file

* fix test to create files with different mod time.

* back date file instead of sleeping

* add watching file in agent command.

* fix watcher call to use new API

* add configuration and stop watcher when server stop

* add certs as watched files

* move FileWatcher to the agent start instead of the command code

* stop watcher before replacing it

* save watched files in agent

* add add and remove interfaces to the file watcher

* fix remove to not return an error

* use `Add` and `Remove` to update certs files

* fix tests

* close events channel on the file watcher even when the context is done

* extract `NotAutoReloadableRuntimeConfig` is a separate struct

* fix linter errors

* add Ca configs and outgoing verify to the not auto reloadable config

* add some logs and fix to use background context

* add tests to auto-config reload

* remove stale test

* add tests to changes to config files

* add check to see if old cert files still trigger updates

* rename `NotAutoReloadableRuntimeConfig` to `StaticRuntimeConfig`

* fix to re add both key and cert file. Add test to cover this case.

* review suggestion

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add check to static runtime config changes

* fix test

* add changelog file

* fix review comments

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* update flag description

Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>

* fix compilation error

* add static runtime config support

* fix test

* fix review comments

* fix log test

* Update .changelog/12329.txt

Co-authored-by: Dan Upton <daniel@floppy.co>

* transfer tests to runtime_test.go

* fix filewatcher Replace to not deadlock.

* avoid having lingering locks

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* split ReloadConfig func

* fix warning message

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* convert `FileWatcher` into an interface

* fix compilation errors

* fix tests

* extract func for adding and removing files

* add a coalesceTimer with a very small timer

* extract coaelsce Timer and add a shim for testing

* add tests to coalesceTimer fix to send remaining events

* set `coalesceTimer` to 1 Second

* support symlink, fix a nil deref.

* fix compile error

* fix compile error

* refactor file watcher rate limiting to be a Watcher implementation

* fix linter issue

* fix runtime config

* fix runtime test

* fix flaky tests

* fix compile error

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* fix agent New to return an error if File watcher New return an error

* quit timer loop if ctx is canceled

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>

Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>
Co-authored-by: Daniel Upton <daniel@floppy.co>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
2022-04-04 11:31:39 -04:00
John Cowen 18f55be3c4
ui: Initial Server Status Overview Page (#12599) 2022-04-04 09:45:03 +01:00
Eric Haberkorn 61af7947f9
Merge pull request #12681 from hashicorp/lambda-patching-tweaks
Tweak the Lambda Envoy configuration generated by the serverless patcher
2022-04-01 19:59:30 -04:00
FFMMM 973d2d0f9a
mark disable_compat_1.9 to deprecate in 1.13, change default to true (#12675)
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2022-04-01 10:35:56 -07:00
R.B. Boyer cb82949ac6
xds: errors from the xds serverless plugin are fatal (#12682) 2022-04-01 10:30:26 -05:00
Eric 5682f3ce1f Tweak the Lambda Envoy configuration generated by the serverless patcher
- Move from `strip_matching_host_port` to `strip_any_host_port`
- Remove `auto_host_rewrite` since it conflicts with `strip_any_host_port`
2022-04-01 11:13:44 -04:00
Eric Haberkorn a7d5aa94a3
Merge pull request #12680 from hashicorp/fix-k8s-dns-service-name
Fix the Kubernetes service name for DNS
2022-04-01 11:03:24 -04:00
Eric d7b4ed1597 Fix the Kubernetes service name for DNS 2022-04-01 10:38:56 -04:00
Eric Haberkorn 26cfbc70b0
Merge pull request #12676 from hashicorp/implement-lambda-patching
Implement Lambda Patching in the Serverless Plugin
2022-04-01 09:58:56 -04:00
Mark Anderson 018edc222e
Avoid using sys/mounts to enable namespaces (#12655)
* Avoid doing list of /sys/mounts

From an internal ticket "Support standard "Vault namespace in the path" semantics for Connect Vault CA Provider"

Vault allows the namespace to be specified as a prefix in the path of
a PKI definition, but this doesn't currently work for
```IntermediatePKIPath``` specifications, because we attempt to list
all of the paths to check if ours is already defined. This doesn't
really work in a namespaced world.

This changes the IntermediatePKIPath code to follow the same pattern
as the root key, where we directly get the key rather than listing.

This code is difficult to write automated tests for because it relies
on features of Vault Enterprise, which isn't currently part of our
test framework, so it was tested manually.

Signed-off-by: Mark Anderson <manderson@hashicorp.com>

* add changelog

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2022-03-31 23:35:38 -07:00
Kyle Havlovitz d86bcd0583
Merge pull request #12677 from hashicorp/expanded-token-read-docs
docs: Add examples for expanded token read CLI and API
2022-03-31 15:37:11 -07:00
Kyle Havlovitz 763f728df4 Add doc examples for expanded token read CLI and API 2022-03-31 15:03:41 -07:00
Kyle Havlovitz 3dc4f50125 Add changelog note 2022-03-31 14:05:02 -07:00
Kyle Havlovitz 1a3b885027 Use the GatewayService SNI field for upstream SAN validation 2022-03-31 13:54:25 -07:00
Eric e0a15690ae Implement Lambda Patching in the Serverless Plugin 2022-03-31 16:45:32 -04:00
FFMMM 64e35777e0
docs: new rpc metric (#12608) 2022-03-31 13:04:33 -07:00
Kyle Havlovitz 059bd0a92e
Merge pull request #12670 from hashicorp/token-read-expanded
oss: Add expanded token read flag and endpoint option
2022-03-31 12:24:11 -07:00
Kyle Havlovitz 51527907ab Recommend SNI with TLS in the terminating gateway docs 2022-03-31 12:19:16 -07:00
Kyle Havlovitz f8efe9a208 Log a warning when a terminating gateway service has TLS but not SNI configured 2022-03-31 12:18:40 -07:00
Dhia Ayachi 16b19dd82d
auto-reload configuration when config files change (#12329)
* add config watcher to the config package

* add logging to watcher

* add test and refactor to add WatcherEvent.

* add all API calls and fix a bug with recreated files

* add tests for watcher

* remove the unnecessary use of context

* Add debug log and a test for file rename

* use inode to detect if the file is recreated/replaced and only listen to create events.

* tidy ups (#1535)

* tidy ups

* Add tests for inode reconcile

* fix linux vs windows syscall

* fix linux vs windows syscall

* fix windows compile error

* increase timeout

* use ctime ID

* remove remove/creation test as it's a use case that fail in linux

* fix linux/windows to use Ino/CreationTime

* fix the watcher to only overwrite current file id

* fix linter error

* fix remove/create test

* set reconcile loop to 200 Milliseconds

* fix watcher to not trigger event on remove, add more tests

* on a remove event try to add the file back to the watcher and trigger the handler if success

* fix race condition

* fix flaky test

* fix race conditions

* set level to info

* fix when file is removed and get an event for it after

* fix to trigger handler when we get a remove but re-add fail

* fix error message

* add tests for directory watch and fixes

* detect if a file is a symlink and return an error on Add

* rename Watcher to FileWatcher and remove symlink deref

* add fsnotify@v1.5.1

* fix go mod

* do not reset timer on errors, rename OS specific files

* rename New func

* events trigger on write and rename

* add missing test

* fix flaking tests

* fix flaky test

* check reconcile when removed

* delete invalid file

* fix test to create files with different mod time.

* back date file instead of sleeping

* add watching file in agent command.

* fix watcher call to use new API

* add configuration and stop watcher when server stop

* add certs as watched files

* move FileWatcher to the agent start instead of the command code

* stop watcher before replacing it

* save watched files in agent

* add add and remove interfaces to the file watcher

* fix remove to not return an error

* use `Add` and `Remove` to update certs files

* fix tests

* close events channel on the file watcher even when the context is done

* extract `NotAutoReloadableRuntimeConfig` is a separate struct

* fix linter errors

* add Ca configs and outgoing verify to the not auto reloadable config

* add some logs and fix to use background context

* add tests to auto-config reload

* remove stale test

* add tests to changes to config files

* add check to see if old cert files still trigger updates

* rename `NotAutoReloadableRuntimeConfig` to `StaticRuntimeConfig`

* fix to re add both key and cert file. Add test to cover this case.

* review suggestion

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add check to static runtime config changes

* fix test

* add changelog file

* fix review comments

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* update flag description

Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>

* fix compilation error

* add static runtime config support

* fix test

* fix review comments

* fix log test

* Update .changelog/12329.txt

Co-authored-by: Dan Upton <daniel@floppy.co>

* transfer tests to runtime_test.go

* fix filewatcher Replace to not deadlock.

* avoid having lingering locks

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* split ReloadConfig func

* fix warning message

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* convert `FileWatcher` into an interface

* fix compilation errors

* fix tests

* extract func for adding and removing files

Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>
Co-authored-by: Daniel Upton <daniel@floppy.co>
2022-03-31 15:11:49 -04:00
R.B. Boyer dc62293978
build: run mog separately after the protobufs are generated (#12665)
Also ensure that we run mog serially on each package in dependency
order.
2022-03-31 13:49:42 -05:00
Kyle Havlovitz b21b4346b4 Add expanded token read flag and endpoint option 2022-03-31 10:49:49 -07:00
FFMMM 1adfd7b94c
polish rpc.service.call metric behavior (#12624) 2022-03-31 10:49:37 -07:00
Paul Glass 706c844423
Add IAM Auth Method (#12583)
This adds an aws-iam auth method type which supports authenticating to Consul using AWS IAM identities.

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2022-03-31 10:18:48 -05:00
Bryce Kalow 6bf67b7ef4
website: redirect /api to /api-docs (#12660) 2022-03-30 16:16:26 -05:00
Mike Morris 7cb9bfdfa9
website(api-gateway): add common errors page (#12643)
* Adding common errors page for API Gateway

* website(api-gateway): add missing CRDs common error message

* Update website/content/docs/api-gateway/common-errors.mdx

Co-authored-by: Nathan Coleman <nathandanielcoleman@gmail.com>

* Update website/content/docs/api-gateway/common-errors.mdx

Co-authored-by: Nathan Coleman <nathandanielcoleman@gmail.com>

* Update website/content/docs/api-gateway/common-errors.mdx

Co-authored-by: Nathan Coleman <nathandanielcoleman@gmail.com>

* Update website/content/docs/api-gateway/common-errors.mdx

* Additional page editing instructions and formating

* Update website/content/docs/api-gateway/common-errors.mdx

* Update website/content/docs/api-gateway/common-errors.mdx

* Update website/content/docs/api-gateway/common-errors.mdx

* Update website/content/docs/api-gateway/common-errors.mdx

* Update website/content/docs/api-gateway/common-errors.mdx

Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>

* Apply suggestions from code review

Co-authored-by: Jeff-Apple <79924108+Jeff-Apple@users.noreply.github.com>
Co-authored-by: Nathan Coleman <nathandanielcoleman@gmail.com>
Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>
2022-03-30 16:05:00 -04:00
Eric Haberkorn 458b1838db
Merge pull request #12659 from hashicorp/bump-go-control-plane
Bump Go Control Plane
2022-03-30 15:07:47 -04:00
R.B. Boyer e79ce8ab03
xds: adding control of the mesh-wide min/max TLS versions and cipher suites from the mesh config entry (#12601)
- `tls.incoming`: applies to the inbound mTLS targeting the public
  listener on `connect-proxy` and `terminating-gateway` envoy instances

- `tls.outgoing`: applies to the outbound mTLS dialing upstreams from
  `connect-proxy` and `ingress-gateway` envoy instances

Fixes #11966
2022-03-30 13:43:59 -05:00
R.B. Boyer c98f2acd75 similar bump 2022-03-30 13:28:00 -05:00
R.B. Boyer 33fcc83d00 fail on error and use ptypes.MarshalAny for now instead of anypb.New 2022-03-30 13:27:49 -05:00
Eric e4b4f175ed Bump go-control-plane
* `go get cloud.google.com/go@v0.59.0`
* `go get github.com/envoyproxy/go-control-plane@v0.9.9`
* `make envoy-library`
* Bumpprotoc to 3.15.8
2022-03-30 13:11:27 -04:00
Matt Keeler 15ddbbc686
Update raft-boltdb to pull in new writeCapacity metric (#12646) 2022-03-30 11:38:44 -04:00
R.B. Boyer e1e610d0ae
need two dollar signs for a shell variable in makefiles (#12657) 2022-03-30 10:37:44 -05:00
R.B. Boyer 82bfc04f87
build: auto install correct version of protoc locally (#12651) 2022-03-30 10:08:17 -05:00
R.B. Boyer ac5bea862a
server: ensure that service-defaults meta is incorporated into the discovery chain response (#12511)
Also add a new "Default" field to the discovery chain response to clients
2022-03-30 10:04:18 -05:00
FFMMM bbab030beb
introduce EmptyReadRequest for status_endpoint (#12653)
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2022-03-29 18:05:45 -07:00
Mike Morris 885edde8c1
website(api-gateway): add consul namespace to helm install (#12644)
* website: api-gateway helm install consul namespace

To mirror instructions at https://learn.hashicorp.com/tutorials/consul/kubernetes-api-gateway

* website(api-gateway): add notes on where to find available versions

* website(api-gateway): fixup link to more clearly indicate Consul Helm chart releases

* Update website/content/docs/api-gateway/api-gateway-usage.mdx
2022-03-29 17:36:21 -04:00
R.B. Boyer cef336a0a3
build: install mog and execute it during protobuf compilation (#12647)
- also import replace isn't needed anymore
2022-03-29 15:18:05 -05:00
Fulvio b94bbf7f43
remove DualStack field from check TCP #12629 (#12630) 2022-03-29 14:56:01 -04:00
R.B. Boyer 64e1501bda
add missing line from prototest.AssertDeepEqual (#12645) 2022-03-29 13:17:41 -05:00
R.B. Boyer 8aebca5794
build: enforce protoc binary is the expected version (#12641) 2022-03-29 12:02:43 -05:00
Eric Haberkorn e5ed4fe493
Merge pull request #12639 from hashicorp/gogo-away
Remove the Rest of Gogo
2022-03-29 10:14:44 -04:00
Eric 76c65e873c code review changes 2022-03-29 09:36:34 -04:00
Eric 5f050614e5 remove the rest of gogo 2022-03-28 17:34:41 -04:00
Eric Haberkorn 9bc15e2f1a
Merge pull request #12638 from hashicorp/remove-gogo-from-acls
Remove gogo from ACL Protobufs
2022-03-28 16:52:13 -04:00
Michele Degges 535edfa247
[RelAPI Onboarding] Add release API metadata file (#12591) 2022-03-28 13:45:53 -07:00
Eric 02d8a11ff0 remove gogo from acl protobufs 2022-03-28 16:20:56 -04:00
R.B. Boyer 4676960b93
proto-gen-rpc-glue: support QueryMeta and QueryOptions (#12637) 2022-03-28 13:12:51 -05:00