19558 Commits

Author SHA1 Message Date
lbausch
4795daa809
docs: Fix link (#14890)
Co-authored-by: David Yu <dyu@hashicorp.com>
2023-01-06 05:21:56 +00:00
claire labry
590ae5fa5c
Onboard consul to use new version/VERSION file and reproducible actions (#15631)
* Onboard consul to use new .release/VERSION file and reproducible actions-go-build

* Onboard consul to use new .release/VERSION file and reproducible actions

* Onboard consul to use new .release/VERSION file and reproducible actions

* fix to consul

* Onboard consul to use new .release/VERSION file and reproducible actions

* Onboard consul to use new .release/VERSION file and reproducible actions

* Onboard consul to use new .release/VERSION file and reproducible actions

* test out ent changes

* just or testing

* Added setup go for build ui

* try removing VERSION file out of .release dir

* add checkout action for build ui and update checkout version

* try no -dev marker

* try removing extra ldflags

* test version

* add back in setup-go step?

* Update utils.js

read from static VERSION file

* remove actions-setup go

* add 1.15.0-dev

* Using prepare workflow for pre-stable channel workflow

* Test prepare workflow

* Remove set-product-version branch from release pipeline

* Use METADATA in environment

* Correct env vars

* Remove current branch from build trigger list

Co-authored-by: emilymianeil <emilymianeil@gmail.com>
Co-authored-by: Sarah <sthompson@hashicorp.com>
Co-authored-by: hc-github-team-nomad-core <github-team-nomad-core@hashicorp.com>
Co-authored-by: emily neil <63985869+emilymianeil@users.noreply.github.com>
2023-01-05 12:16:47 -08:00
Paul Glass
666c2b2e2b
Fix TLS_BadVerify test assertions on macOS (#15903) 2023-01-05 11:47:45 -06:00
Dan Upton
b78de5a7a2
grpc/acl: fix bug where ACL token was required even if disabled (#15904)
Fixes a bug introduced by #15346 where we'd always require an ACL
token even if ACLs were disabled because we were erroneously
treating `nil` identity as anonymous.
2023-01-05 16:31:18 +00:00
Dan Upton
ee2d47da83
Fix missing comment in generated enterprise rate-limit file (#15895)
Fixes bug in #15564 where gofmt would strip out the generated code
warning comment because it was on the same line as the build tag.
2023-01-05 10:31:14 +00:00
Dan Upton
d53ce39c32
grpc: switch servers and retry on error (#15892)
This is the OSS portion of enterprise PR 3822.

Adds a custom gRPC balancer that replicates the router's server cycling
behavior. Also enables automatic retries for RESOURCE_EXHAUSTED errors,
which we now get for free.
2023-01-05 10:21:27 +00:00
Nick Irvine
6fb628c07d
fix: return error when config file with unknown extension is passed (#15107) 2023-01-04 16:57:00 -08:00
Tu Nguyen
01a4510d6c
Update api gateway version to latest (#15197)
* Update api gateway version to latest

* Update website/content/docs/api-gateway/install.mdx

* update to latest apigw version 0.5.1

* update consul and helm version
2023-01-04 12:44:02 -08:00
cskh
1113314641
Refactoring the peering integ test to accommodate coming changes of o… (#15885)
* Refactoring the peering integ test to accommodate coming changes of other upgrade scenarios.

- Add a utils package under test that contains methods to set up various test scenarios.
- Deduplication: have a single CreatingPeeringClusterAndSetup replace
  CreatingAcceptingClusterAndSetup and CreateDialingClusterAndSetup.
- Separate peering cluster creation and server registration.

* Apply suggestions from code review

Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
2023-01-04 15:28:15 -05:00
Florian Apolloner
077b0a48a3
Allow Operator Generated bootstrap token (#14437)
Add support to provide an initial token via the bootstrap HTTP API, similar to hashicorp/nomad#12520
2023-01-04 20:19:33 +00:00
Semir Patel
a6482341a5
Wire up the rate limiter to net/rpc calls (#15879) 2023-01-04 13:38:44 -06:00
Tu Nguyen
4bbfad8dca
Update links to new get started tutorials (#15154) 2023-01-04 09:58:08 -08:00
Dan Upton
d4c435856b
grpc: protoc plugin for generating gRPC rate limit specifications (#15564)
Adds automation for generating the map of `gRPC Method Name → Rate Limit Type`
used by the middleware introduced in #15550, and will ensure we don't forget
to add new endpoints.

Engineers must annotate their RPCs in the proto file like so:

```
rpc Foo(FooRequest) returns (FooResponse) {
  option (consul.internal.ratelimit.spec) = {
    operation_type: READ,
  };
}
```

When they run `make proto` a protoc plugin `protoc-gen-consul-rate-limit` will
be installed that writes rate-limit specs as a JSON array to a file called
`.ratelimit.tmp` (one per protobuf package/directory).

After running Buf, `make proto` will execute a post-process script that will
ingest all of the `.ratelimit.tmp` files and generate a Go file containing the
mappings in the `agent/grpc-middleware` package. In the enterprise repository,
it will write an additional file with the enterprise-only endpoints.

If an engineer forgets to add the annotation to a new RPC, the plugin will
return an error like so:

```
RPC Foo is missing rate-limit specification, fix it with:

	import "proto-public/annotations/ratelimit/ratelimit.proto";

	service Bar {
	  rpc Foo(...) returns (...) {
	    option (hashicorp.consul.internal.ratelimit.spec) = {
	      operation_type: OPERATION_READ | OPERATION_WRITE | OPERATION_EXEMPT,
	    };
	  }
	}
```

In the future, this annotation can be extended to support rate-limit
category (e.g. KV vs Catalog) and to determine the retry policy.
2023-01-04 16:07:02 +00:00
Dan Upton
7c7503c849
grpc/acl: relax permissions required for "core" endpoints (#15346)
Previously, these endpoints required `service:write` permission on _any_
service as a sort of proxy for "is the caller allowed to participate in
the mesh?".

Now, they're called as part of the process of establishing a server
connection by any consumer of the consul-server-connection-manager
library, which will include non-mesh workloads (e.g. Consul KV as a
storage backend for Vault) as well as ancillary components such as
consul-k8s' acl-init process, which likely won't have `service:write`
permission.

So this commit relaxes those requirements to accept *any* valid ACL token
on the following gRPC endpoints:

- `hashicorp.consul.dataplane.DataplaneService/GetSupportedDataplaneFeatures`
- `hashicorp.consul.serverdiscovery.ServerDiscoveryService/WatchServers`
- `hashicorp.consul.connectca.ConnectCAService/WatchRoots`
2023-01-04 12:40:34 +00:00
Hans Hasselberg
275a0b8e7f
fix cli string for id flag (#15695) 2023-01-03 20:36:26 +00:00
Derek Menteer
1f7e7abeac
Fix issue with incorrect proxycfg watch on upstream peer-targets. (#15865)
This fixes an issue where the incorrect partition was given to the
upstream target watch, which meant that failover logic would not
work correctly.
2023-01-03 10:44:08 -06:00
Derek Menteer
f3776894bf
Fix agent cache incorrectly notifying unchanged protobufs. (#15866)
Fix agent cache incorrectly notifying unchanged protobufs.

This change fixes a situation where the protobuf private fields
would be read by reflect.DeepEqual() and indicate data was modified.
This resulted in change notifications being fired every time, which
could cause performance problems in proxycfg.
2023-01-03 10:11:56 -06:00
Dan Upton
7747384f1f
Wire in rate limiter to handle internal and external gRPC calls (#15857) 2022-12-23 13:42:16 -06:00
Dan Stough
8607eb2f82
chore: fix remote docker make target (#15870) 2022-12-22 15:44:18 -05:00
Dan Stough
b3bd3a6586
[OSS] feat: access logs for listeners and listener filters (#15864)
* feat: access logs for listeners and listener filters

* changelog

* fix integration test
2022-12-22 15:18:15 -05:00
Gerard Nguyen
53250b19f4
docs: update intentions config (#15868) 2022-12-22 20:07:23 +00:00
Freddy
5f2ae23a1b
Expand service mesh dev docs (#15867) 2022-12-22 12:18:38 -07:00
Jeff Boruszak
a38a3d0b5e
Removed technical preview callout (#15872) 2022-12-22 12:52:34 -06:00
Nitya Dhanushkodi
24f01f96b1
add extensions for local service to GetExtensionConfigurations (#15871)
This gets the extensions information for the local service into the snapshot and ExtensionConfigurations for a proxy. It grabs the extensions from config entries and puts them in structs.NodeService.Proxy field, which already is copied into the config snapshot.

Also:
* add EnvoyExtensions to api.AgentService so that it matches structs.NodeService
2022-12-22 10:03:33 -08:00
Nitya Dhanushkodi
c7ef04c597
[OSS] extensions: refactor PluginConfiguration into a more generic type ExtensionConfiguration (#15846)
* extensions: refactor PluginConfiguration into a more generic type
ExtensionConfiguration

Also:
* adds endpoints configuration to lambda golden tests
* uses string constant for builtin/aws/lambda
Co-authored-by: Eric <eric@haberkorn.co>
2022-12-20 22:26:20 -08:00
John Murret
f5e01f8c6b
Rate Limit Handler - ensure rate limiting is not in the code path when not configured (#15819)
* Rate limiting handler - ensure configuration has changed before modifying limiters

* Updating test to validate arguments to UpdateConfig

* Removing duplicate test.  Updating mock.

* Renaming NullRateLimiter to NullRequestLimitsHandler

* Rate Limit Handler - ensure rate limiting is not in the code path when not configured

* Update agent/consul/rate/handler.go

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>

* formatting handler.go

* Rate limiting handler - ensure configuration has changed before modifying limiters

* Updating test to validate arguments to UpdateConfig

* Removing duplicate test.  Updating mock.

* adding logging for when UpdateConfig is called but the config has not changed.

* Update agent/consul/rate/handler.go

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>

* Update agent/consul/rate/handler_test.go

Co-authored-by: Dan Upton <daniel@floppy.co>

* modifying existing variable name based on pr feedback

* updating a broken merge conflict;

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
Co-authored-by: Dan Upton <daniel@floppy.co>
2022-12-20 15:00:22 -07:00
John Murret
aba43d85d9
Rate limiting handler - ensure configuration has changed before modifying limiters (#15805)
* Rate limiting handler - ensure configuration has changed before modifying limiters

* Updating test to validate arguments to UpdateConfig

* Removing duplicate test.  Updating mock.

* adding logging for when UpdateConfig is called but the config has not changed.

* Update agent/consul/rate/handler.go

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2022-12-20 14:12:03 -07:00
Vladislav Sharapov
629878a687
Update service-discovery.mdx (#15832)
* Update service-discovery.mdx

* Update website/content/docs/concepts/service-discovery.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2022-12-20 13:54:33 -06:00
Michael Wilkerson
1b28b89439
Enhancement: Consul Compatibility Checking (#15818)
* add functions for returning the max and min Envoy major versions
- added an UnsupportedEnvoyVersions list
- removed an unused error from TestDetermineSupportedProxyFeaturesFromString
- modified minSupportedVersion to use the function for getting the Min Envoy major version. Using just the major version without the patch is equivalent to using `.0`

* added a function for executing the envoy --version command
- added a new exec.go file to not be locked to unix system

* added envoy version check when using consul connect envoy

* added changelog entry

* added docs change
2022-12-20 09:58:19 -08:00
Derek Menteer
74b11c416c
Fix incorrect protocol check on discovery chains with peer targets. (#15833) 2022-12-20 10:15:03 -06:00
trujillo-adam
1f89de051c
fixed bad markdown in network segments usage (#15852) 2022-12-19 16:23:57 -08:00
trujillo-adam
f3f9e54b0d
Docs/network segments tutorial docs conversion (#15829)
* added a NS folder and refactored main page into the overview page

* added NS usage page to NS folder

* updated links to NS docs

* updated nav

* addressed feedback from review
2022-12-19 15:35:05 -08:00
Semir Patel
799b34f1a9
Map net/rpc endpoints to a read/write/exempt op for rate-limiting (#15825)
Also fixed TestRequestRecorder flaky tests due to loss of precision in elapsed time in the test.
2022-12-19 16:04:52 -06:00
Nitya Dhanushkodi
d382ca0aec
extensions: refactor serverless plugin to use extensions from config entry fields (#15817)
docs: update config entry docs and the Lambda manual registration docs

Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
Co-authored-by: Eric <eric@haberkorn.co>
2022-12-19 12:19:37 -08:00
Chris S. Kim
f7b7f5d4b6
Error out consul connect envoy if agent explicitly disabled grpc (#15794)
Co-authored-by: Paul Glass <pglass@hashicorp.com>
2022-12-19 14:37:27 -05:00
David Yu
0300bba51f
Add Service Mesh work stream to JIRA API call (#15845) 2022-12-19 19:10:30 +00:00
Chris S. Kim
d44b23cb31
Break instead (#15844) 2022-12-19 11:53:05 -07:00
Tu Nguyen
3df2b7d2ab
Update docs to reflect vault and consul compatibility (#15826)
* update docs to reflect vault and consul compatibility

* Update website/content/docs/connect/ca/vault.mdx

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
2022-12-19 10:32:25 -08:00
Chris S. Kim
831680d2c5
Add custom balancer to always remove subConns (#15701)
The new balancer is a patched version of gRPC's default pick_first balancer
which removes the behavior of preserving the active subconnection if
a list of new addresses contains the currently active address.
2022-12-19 17:39:31 +00:00
cskh
252a08e903
Change field to pointer so it will be parsed as nil (#15831) 2022-12-19 10:25:36 -05:00
Andrew Stucki
ab199a11b0
Add async reconciliation controller subpackage (#15534)
* Add async reconciliation controller subpackage

* Address initial feedback

* Add tests for panic assertions

* Fix comment
2022-12-16 16:49:26 -05:00
Dhia Ayachi
f04f88e4b9
add missing code and fix enterprise specific code (#15375)
* add missing code and fix enterprise specific code

* fix retry

* fix flaky tests

* fix linter error in test
2022-12-16 16:31:05 -05:00
David Yu
1406d428dd
JIRA: Small fix to search plugin for JIRA issues (#15816) 2022-12-15 21:32:46 +00:00
cskh
692a6edd7d
Upgrade test: test peering upgrade from an old version of consul (#15768)
* upgrade test: test peering upgrade from an old version of consul

NET-1809
2022-12-15 16:31:12 -05:00
Semir Patel
9fcfe8d7c5
Force installation of shadow-utils for access to groupadd executable in ubi image (#15812)
Removing curl inadvertently dropped the transitive dependency on shadow-utils
which is required for the groupadd executable. Since curl-minimal does not
have this dependency, make it explicit as part of `microdnf install`.
2022-12-15 12:42:22 -06:00
Dhia Ayachi
2d902b26ac
add log-drop package (#15670)
* add log-drop package

* refactor to extract level

* extract metrics

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* fix compile errors

* change to implement a log sink

* fix tests to remove sleep

* rename and add go docs

* fix expending variadic

Co-authored-by: Dan Upton <daniel@floppy.co>
2022-12-15 12:52:48 -05:00
Semir Patel
2884090570
Fix curl vs curl-minimail install conflict in ubi9 image (#15808)
ubi9 already has curl-minimal installed. Attempting to install curl
results in a microdnf conflict and errors out.
2022-12-15 11:00:39 -06:00
Dan Stough
20f9e606b2
docs: update changelog from 1.14.3, 1.13.5, 1.12.8 (#15804) 2022-12-14 18:47:35 -05:00
David Yu
f7d8477295
add .yaml extension to enable Github Actions workflow (#15799) 2022-12-14 21:46:07 +00:00
Pier-Luc Caron St-Pierre
76fc2f6562
connect: Add support for ConsulResolver to specifies a filter expression (#15659)
* connect: Add support for ConsulResolver to specifies a filter expression
2022-12-14 12:41:07 -08:00