diff --git a/website/content/docs/agent/config/config-files.mdx b/website/content/docs/agent/config/config-files.mdx index 6e50cbb44a..4183a5a7d2 100644 --- a/website/content/docs/agent/config/config-files.mdx +++ b/website/content/docs/agent/config/config-files.mdx @@ -711,6 +711,10 @@ Refer to the [formatting specification](https://golang.org/pkg/time/#ParseDurati servers in all federated datacenters must have this enabled before any client can use [`use_streaming_backend`](#use_streaming_backend). +- `reporting` - This option allows options for HashiCorp reporting. + - `license` - The license object allows users to control automatic reporting of license utilization metrics to HashiCorp. + - `enabled`: (Defaults to `true`) Enables automatic license utilization reporting. + - `segment` - Equivalent to the [`-segment` command-line flag](/consul/docs/agent/config/cli-flags#_segment). ~> **Warning:** The `segment` option cannot be used with the [`partition`](#partition-1) option. diff --git a/website/content/docs/agent/config/index.mdx b/website/content/docs/agent/config/index.mdx index 0ea4a030fb..c620ef72e0 100644 --- a/website/content/docs/agent/config/index.mdx +++ b/website/content/docs/agent/config/index.mdx @@ -73,6 +73,7 @@ The following agent configuration options are reloadable at runtime: them without a restart provides a recovery path that doesn't involve downtime. They generally shouldn't be changed otherwise. - [RPC rate limits](/consul/docs/agent/config/config-files#limits) +- [Reporting](/consul/docs/agent/config/config-files#reporting) - [HTTP Maximum Connections per Client](/consul/docs/agent/config/config-files#http_max_conns_per_client) - Services - TLS Configuration diff --git a/website/content/docs/agent/limits/index.mdx b/website/content/docs/agent/limits/index.mdx index ffa259cfd6..0d12f1127a 100644 --- a/website/content/docs/agent/limits/index.mdx +++ b/website/content/docs/agent/limits/index.mdx @@ -2,32 +2,51 @@ layout: docs page_title: Limit Traffic Rates Overview description: Rate limiting is a set of Consul server agent configurations that you can use to mitigate the risks to Consul servers when clients send excessive requests to Consul resources. - --- # Traffic rate limiting overview -This topic provides an overview of the rates limits you can configure for Consul servers. + +This topic provides overview information about the traffic rates limits you can configure for Consul datacenters. ## Introduction -You can configure global RPC rate limits to mitigate the risks to Consul servers when clients send excessive read or write requests to Consul resources. A _read request_ is defined as any request that does not modify Consul internal state. A _write request_ is defined as any request that modifies Consul internal state. Rate limits for read and write requests are configured separately. -## Rate limit modes +Configuring rate limits on RPC and gRPC traffic mitigates the risks to Consul servers when client agents or services send excessive read or write requests to Consul resources. A _read_ request is defined as any request that does not modify Consul internal state. A _write_ request is defined as any request that modifies Consul internal state. Configure read and write request limits independently. -You can set one of the following modes to determine how Consul servers react when exceeding request limits. - -- **Enforcing mode**: The rate limiter denies requests to a server once they exceed the configured rate. In this mode, Consul generates metrics and logs to help you understand your network's load and configure limits accordingly. -- **Permissive mode**: The rate limiter allows requests to a server once they exceed the configured rate. In this mode, Consul generates metrics and logs to help you understand your Consul load and configure limits accordingly. Use this mode to help you debug specific issues as you configure limits. -- **Disabled mode**: Disables the rate limiter. This mode allows all requests Consul does not generate logs or metrics. This is the default mode. +## Workflow -Refer to [`rate_limits`](/consul/docs/agent/config/config-files#request_limits) for additional configuration information. +You can set global limits on the rate of read and write requests that affect individual servers in the datacenter. You can set limits for all source IP addresses, which enables you to specify a budget for read and write requests to prevent any single source IP from overwhelming the Consul server and negatively affecting the network. The following steps describe the general process for setting global read and write rate limits: -## Request denials +1. Set arbitrary limits to begin understanding the upper boundary of RPC and gRPC loads in your network. Refer to [Initialize rate limit settings](/consul/docs/agent/limits/usage/init-rate-limits) for additional information. -When an HTTP request is denied for rate limiting reason, Consul returns one of the following errors: +1. Monitor the metrics and logs and readjust the initial configurations as necessary. Refer to [Monitor rate limit data](/consul/docs/agent/limits/usage/monitor-rate-limit-data) -- **429 Resource Exhausted**: Indicates that a server is not able to perform the request but that another server could potentially fulfill it. This error is most common on stale reads because any server may fulfill stale read requests. To resolve this type of error, we recommend immediately retrying the request to another server. If the request came from a Consul client agent, the agent automatically retries the request up to the limit set in the [`rpc_hold_timeout`](/consul/docs/agent/config/config-files#rpc_hold_timeout) configuration . +1. Define your final operational limits based on your observations. If you are defining global rate limits, refer to [Set global traffic rate limits](/consul/docs/agent/limits/usage/set-global-rate-limits) for additional information. For information about setting limits based on source IP, refer to [Limit traffic rates for a source IP](/consul/docs/agent/limits/usage/set-source-ip-rate-limits). -- **503 Service Unavailable**: Indicates that server is unable to perform the request and that no other server can fulfill the request, either. This usually occurs on consistent reads or for writes. In this case we recommend retrying according to an exponential backoff schedule. If the request came from a Consul client agent, the agent automatically retries the request according to the [`rpc_hold_timeout`](/consul/docs/agent/config/config-files#rpc_hold_timeout) configuration. +### Order of operations -Refer to [Rate limit reached on the server](/consul/docs/troubleshoot/common-errors#rate-limit-reached-on-the-server) for additional information. \ No newline at end of file +You can define request rate limits in the agent configuration and in the control plane request limit configuration entry. The configuration entry also supports rate limit configurations for Consul resources. Consul perfroms the following order of operations when determing request rate limits: + +1. Parse request. +1. Does the request reach a global server limit? + - No: Proceed to the next stage. + - Yes: Return an error that the requested resource has been exhausted. +1. Does the request reach a limit associated with its source IP address? + - No: Proceed to the next stage. + - Yes: Return an error that the requested resource has been exhausted. +1. Resolve the Consul Enterprise metadata. +1. Does the request reach a limit associated with the source partition? + - No: Proceed to the next stage. + - Yes: Return an error that the requested resource has been exhausted. +1. Does the request reach a limit associated with the source namespace? + - No: Proceed to the next stage. + - Yes: Return an error that the requested resource has been exhausted. +1. Resolve the ACL identity associated with the request. +1. Does the request reach a limit associated with its identity? + - No: Proceed to the next stage. + - Yes: Return an error that the requested resource has been exhausted. +1. Handle the request. + +## Kubernetes + +To define global rate limits, configure the `request_limits` settings in the Consul Helm chart. Refer to the [Helm chart reference](/consul/docs/k8s/helm) for additional information. Refer to the [control plane request limit configuration entry reference](/consul/docs/connect/config-entries/control-plane-request-limit) for information about applying a CRD for limiting traffic rates from source IPs. \ No newline at end of file diff --git a/website/content/docs/agent/limits/set-global-traffic-rate-limits.mdx b/website/content/docs/agent/limits/set-global-traffic-rate-limits.mdx deleted file mode 100644 index 5d4fc43df0..0000000000 --- a/website/content/docs/agent/limits/set-global-traffic-rate-limits.mdx +++ /dev/null @@ -1,114 +0,0 @@ ---- -layout: docs -page_title: Set a Global Limit on Traffic Rates -description: Use global rate limits to prevent excessive rates of requests to Consul servers. ---- - -# Set a global limit on traffic rates - -This topic describes how to configure rate limits for RPC and gRPC traffic to the Consul server. - -## Introduction - -Rate limits apply to each Consul server separately and limit the number of read requests or write requests to the server on the RPC and internal gRPC endpoints. - -Because all requests coming to a Consul server eventually perform an RPC or an internal gRPC request, global rate limits apply to Consul's user interfaces, such as the HTTP API interface, the CLI, and the external gRPC endpoint for services in the service mesh. - -Refer to [Initialize Rate Limit Settings](/consul/docs/agent/limits/init-rate-limits) for additional information about right-sizing your gRPC request configurations. - -## Set a global rate limit for a Consul server - -Configure the following settings in your Consul server configuration to limit the RPC and gRPC traffic rates. - -- Set the rate limiter [`mode`](/consul/docs/agent/config/config-files#mode-1) -- Set the [`read_rate`](/consul/docs/agent/config/config-files#read_rate) -- Set the [`write_rate`](/consul/docs/agent/config/config-files#write_rate) - -In the following example, the Consul server is configured to prevent more than `500` read and `200` write RPC calls: - - - -```hcl -limits = { - rate_limit = { - mode = "enforcing" - read_rate = 500 - write_rate = 200 - } -} -``` - -```json -{ - "limits" : { - "rate_limit" : { - "mode" : "enforcing", - "read_rate" : 500, - "write_rate" : 200 - } - } -} - -``` - - - -## Access rate limit logs - -Consul prints a log line for each rate limit request. The log includes information to identify the source of the request and the server's configured limit. Consul prints to `DEBUG` log level, and can be configured to drop log lines to avoid affecting the server health. Dropping a log line increments the `rpc.rate_limit.log_dropped` metric. - -The following example log shows that RPC request from `127.0.0.1:53562` to `KVS.Apply` exceeded the rate limit: - - - -```shell-session -2023-02-17T10:01:15.565-0500 [DEBUG] agent.server.rpc-rate-limit: RPC -exceeded allowed rate limit: rpc=KVS.Apply source_addr=127.0.0.1:53562 -limit_type=global/write limit_enforced=false -``` - - -## Review rate limit metrics - -Consul captures the following metrics associated with rate limits: - -- Type of limit -- Operation -- Rate limit mode - -Call the `agent/metrics` API endpoint to view the metrics associated with rate limits. Refer to [View Metrics](/consul/api-docs/agent#view-metrics) for API usage information. - -In the following example, Consul dropped a call to the `consul` service because it exceeded the limit by one call: - -```shell-session -$ curl http://127.0.0.1:8500/v1/agent/metrics -{ - . . . - "Counters": [ - { - "Name": "consul.rpc.rate_limit.exceeded", - "Count": 1, - "Sum": 1, - "Min": 1, - "Max": 1, - "Mean": 1, - "Stddev": 0, - "Labels": { - "service": "consul" - } - }, - { - "Name": "consul.rpc.rate_limit.log_dropped", - "Count": 1, - "Sum": 1, - "Min": 1, - "Max": 1, - "Mean": 1, - "Stddev": 0, - "Labels": {} - } - ], - . . . -``` - -Refer to [Telemetry](/consul/docs/agent/telemetry) for additional information. diff --git a/website/content/docs/agent/limits/init-rate-limits.mdx b/website/content/docs/agent/limits/usage/init-rate-limits.mdx similarity index 66% rename from website/content/docs/agent/limits/init-rate-limits.mdx rename to website/content/docs/agent/limits/usage/init-rate-limits.mdx index 7dbabbe6c7..e90aaf77af 100644 --- a/website/content/docs/agent/limits/init-rate-limits.mdx +++ b/website/content/docs/agent/limits/usage/init-rate-limits.mdx @@ -1,6 +1,6 @@ --- layout: docs -page_title: Initialize Rate Limit Settings +page_title: Initialize rate limit settings description: Learn how to determine regular and peak loads in your network so that you can set the initial global rate limit configurations. --- @@ -14,7 +14,7 @@ Because each network has different needs and application, you need to find out w - Number of servers and the projected load - Existing metrics expressing requests per second -1. Set the `mode` to `permissive`. In the following example, the configuration allows up to 1000 reads and 500 writes per second for each Consul agent: +1. Set the [`limits.request_limits.mode`](/consul/docs/agent/config/config-files#mode-1) parameter in the agent configuration to `permissive`. In the following example, the configuration allows up to 1000 reads and 500 writes per second for each Consul agent: ```hcl request_limits { @@ -22,9 +22,8 @@ Because each network has different needs and application, you need to find out w read_rate = 1000.0 write_rate = 500.0 } - ``` - -1. Observe the logs and metrics for your application's typical cycle, such as a 24 hour period. Refer to [`log_file`](/consul/docs/agent/config/config-files#log_file) for information about where to retrieve logs. Call the [`/agent/metrics`](/consul/api-docs/agent#view-metrics) HTTP API endpoint and check the data for the following metrics: + ``` +1. Observe the logs and metrics for your application's typical cycle, such as a 24 hour period. Refer to [Monitor traffic rate limit data](/consul/docs/agent/limits/usage/monitor-rate-limit) for additional information. Call the [`/agent/metrics`](/consul/api-docs/agent#view-metrics) HTTP API endpoint and check the data for the following metrics: - `rpc.rate_limit.exceeded` with value `global/read` for label `limit_type` - `rpc.rate_limit.exceeded` with value `global/write` for label `limit_type` diff --git a/website/content/docs/agent/limits/usage/limit-request-rates-from-ips.mdx b/website/content/docs/agent/limits/usage/limit-request-rates-from-ips.mdx new file mode 100644 index 0000000000..c074d3007a --- /dev/null +++ b/website/content/docs/agent/limits/usage/limit-request-rates-from-ips.mdx @@ -0,0 +1,66 @@ +--- +layout: docs +page_title: Limit traffic rates for a source IP address +description: Learn how to set read and request rate limits on RPC and gRPC traffic from all source IP addresses to a Consul resource. +--- + +# Limit traffic rates from source IP addresses + +This topic describes how to configure RPC and gRPC traffic rate limits for source IP addresses. This enables you to specify a budget for read and write requests to prevent any single source IP from overwhelming the Consul server and negatively affecting the network. For information about setting global traffic rate limits, refer to [Set a global limit on traffic rates](/consul/docs/agent/limits/usage/set-glogal-traffic-rate-limits). For an overview of Consul's server rate limiting capabilities, refer to [Limit traffic rates overview](/consul/docs/agent/limits/overview). + +## Overview + +You can set limits on the rate of read and write requests from source IP addresses to specific resources, which mitigates the risks to Consul servers when consul clients send excessive requests to a specific resource type. Before configuring traffic rate limits, you should complete the initialization process to understand normal traffic loads in your network. Refer to [Initialize rate limit settings](/consul/docs/agent/limits/init-rate-limits) for additional information. + +Complete the following steps to configure traffic rate limits from a source IP address: + +1. Define rate limits in a control plan request limit configuration entry. You can set limits for different types of resources calls. + +1. Apply the configuration entry to enact the limits. + +You should also monitor read and write rate activity and make any necessary adjustments. Refer to [Monitor rate limit data](/consul/docs/agent/limits/usage/monitor-rate-limits) for additional information. + +## Define rate limits + +Create a control plane request limit configuration entry in the `default` partition. The configuration entry applies to all client requests targeting any partition. Refer to the [control plane request limit configuration entry](/consul/docs/connect/config-entries/control-plan-request-limit) reference documentation for details about the available configuration parameters. + +Specify the following parameters: + +- `kind`: This must be set to `control-plane-request-limit`. +- `name`: Specify the name of the service that you want to limit read and write operations to. +- `read_rate`: Specify overall number of read operations per second allowed from the service. +- `write_rate`: Specify overall number of write operations per second allowed from the service. + +You can also configure limits on calls to the key-value store, ACL system, and Consul catalog. + +## Apply the configuration entry + +If your network is deployed to virtual machines, use the `consul config write` command and specify the control plane request limit configuration entry to apply the configuration. For Kubernetes-orchestrated networks, use the `kubectl apply` command. + + + + +```shell-session +$ consul config write control-plane-request-limit.hcl +``` + + + + +```shell-session +$ consul config write control-plane-request-limit.json +``` + + + + +```shell-session +$ kubectl apply control-plane-request-limit.yaml +``` + + + + +## Disable request rate limits + +Set the [limits.request_limits.mode](/consul/docs/agent/config/config-files#mode-1) in the agent configuration to `disabled` to allow services to exceed the specified read and write requests limits. The `disabled` mode applies to all request rate limits, even limits specifed in the [control plane request limits configuration entry](/consul/docs/connect/config-entries/control-plane-request-limits). Note that any other mode specified in the agent configuration only applies to global traffic rate limits. \ No newline at end of file diff --git a/website/content/docs/agent/limits/usage/monitor-rate-limits.mdx b/website/content/docs/agent/limits/usage/monitor-rate-limits.mdx new file mode 100644 index 0000000000..23906041b3 --- /dev/null +++ b/website/content/docs/agent/limits/usage/monitor-rate-limits.mdx @@ -0,0 +1,77 @@ +--- +layout: docs +page_title: Monitor traffic rate limit data +description: Learn about the metrics and logs you can use to monitor server rate limiting activity, include rate limits for read operations and writer operations +--- + +# Monitor traffic rate limit data + +This topic describes Consul functionality that enables you to monitor read and write request operations taking place in your network. Use the functionality to help you understand normal workloads and set safe limits on the number of requests Consul client agents and services can make to Consul servers. + +## Access rate limit logs + +Consul prints a log line for each rate limit request. The log provides the necessary information for identifying the source of the request and the configured limit. The log provides the information necessary for identifying the source of the request and the configured limit. Consul prints the log `DEBUG` log level and can drop the log to avoid affecting the server health. Dropping a log line increments the `rpc.rate_limit.log_dropped` metric. + +The following example log shows that RPC request from `127.0.0.1:53562` to `KVS.Apply` exceeded the limit: + +```text +2023-02-17T10:01:15.565-0500 [DEBUG] agent.server.rpc-rate-limit: RPC +exceeded allowed rate limit: rpc=KVS.Apply source_addr=127.0.0.1:53562 +limit_type=global/write limit_enforced=false +``` + +Refer to [`log_file`](/consul/docs/agent/config/config-files#log_file) for information about where to retrieve log files. + +## Review rate limit metrics + +Consul captures the following metrics associated with rate limits: + +- Type of limit +- Operation +- Rate limit mode + +Call the `/agent/metrics` API endpoint to view the metrics associated with rate limits. Refer to [View Metrics](/consul/api-docs/agent#view-metrics) for API usage information. In the following example, Consul dropped a call to the consul service because it exceeded the limit by one call: + +```shell-session +$ curl http://127.0.0.1:8500/v1/agent/metrics +{ + . . . + "Counters": [ + { + "Name": "consul.rpc.rate_limit.exceeded", + "Count": 1, + "Sum": 1, + "Min": 1, + "Max": 1, + "Mean": 1, + "Stddev": 0, + "Labels": { + "service": "consul" + } + }, + { + "Name": "consul.rpc.rate_limit.log_dropped", + "Count": 1, + "Sum": 1, + "Min": 1, + "Max": 1, + "Mean": 1, + "Stddev": 0, + "Labels": {} + } + ], + . . . +} +``` + +Refer to [Telemetry](/consul/docs/telemetry) for additional information. + +## Request denials + +When an HTTP request is denied for rate limiting reason, Consul returns one of the following errors: + +- **429 Resource Exhausted**: Indicates that a server is not able to perform the request but that another server could potentially fulfill it. This error is most common on stale reads because any server may fulfill stale read requests. To resolve this type of error, we recommend immediately retrying the request to another server. If the request came from a Consul client agent, the agent automatically retries the request up to the limit set in the [`rpc_hold_timeout`](/consul/docs/agent/config/config-files#rpc_hold_timeout) configuration . + +- **503 Service Unavailable**: Indicates that server is unable to perform the request and that no other server can fulfill the request, either. This usually occurs on consistent reads or for writes. In this case we recommend retrying according to an exponential backoff schedule. If the request came from a Consul client agent, the agent automatically retries the request according to the [`rpc_hold_timeout`](/consul/docs/agent/config/config-files#rpc_hold_timeout) configuration. + +Refer to [Rate limit reached on the server](/consul/docs/troubleshoot/common-errors#rate-limit-reached-on-the-server) for additional information. \ No newline at end of file diff --git a/website/content/docs/agent/limits/usage/set-global-traffic-rate-limits.mdx b/website/content/docs/agent/limits/usage/set-global-traffic-rate-limits.mdx new file mode 100644 index 0000000000..61a4066ec9 --- /dev/null +++ b/website/content/docs/agent/limits/usage/set-global-traffic-rate-limits.mdx @@ -0,0 +1,62 @@ +--- +layout: docs +page_title: Set a global limit on traffic rates +description: Use global rate limits to prevent excessive rates of requests to Consul servers. +--- + +# Set a global limit on traffic rates + +This topic describes how to configure rate limits for RPC and gRPC traffic to the Consul server. + +## Introduction + +Rate limits apply to each Consul server separately and limit the number of read requests or write requests to the server on the RPC and internal gRPC endpoints. + +Because all requests coming to a Consul server eventually perform an RPC or an internal gRPC request, global rate limits apply to Consul's user interfaces, such as the HTTP API interface, the CLI, and the external gRPC endpoint for services in the service mesh. + +Refer to [Initialize Rate Limit Settings](/consul/docs/agent/limits/init-rate-limits) for additional information about right-sizing your gRPC request configurations. + +## Set a global rate limit for a Consul server + +Configure the following settings in your Consul server configuration to limit the RPC and gRPC traffic rates. + +- Set the rate limiter [`mode`](/consul/docs/agent/config/config-files#mode-1) +- Set the [`read_rate`](/consul/docs/agent/config/config-files#read_rate) +- Set the [`write_rate`](/consul/docs/agent/config/config-files#write_rate) + +In the following example, the Consul server is configured to prevent more than `500` read and `200` write RPC calls: + + + +```hcl +limits = { + rate_limit = { + mode = "enforcing" + read_rate = 500 + write_rate = 200 + } +} +``` + +```json +{ + "limits" : { + "rate_limit" : { + "mode" : "enforcing", + "read_rate" : 500, + "write_rate" : 200 + } + } +} + +``` + + + +## Monitor request rate traffic + +You should continue to mmonitor request traffic to ensure that request rates remain within the threshold you defined. Refer to [Monitor traffic rate limit data](/consul/docs/agent/limits/usage/monitor-rate-limits) for instructions about checking metrics and log entries, as well as troubleshooting informaiton. + +## Disable request rate limits + +Set the [limits.request_limits.mode](/consul/docs/agent/config/config-files#mode-1) to `disabled` to allow services to exceed the specified read and write requests limits, even limits specifed in the [control plane request limits configuration entry](/consul/docs/connect/config-entries/control-plane-request-limits). Note that any other mode specified in the agent configuration only applies to global traffic rate limits. diff --git a/website/content/docs/connect/config-entries/control-plane-request-limit.mdx b/website/content/docs/connect/config-entries/control-plane-request-limit.mdx new file mode 100644 index 0000000000..c6b44436ac --- /dev/null +++ b/website/content/docs/connect/config-entries/control-plane-request-limit.mdx @@ -0,0 +1,224 @@ +--- +layout: docs +page_title: Control Plane Request Limit Configuration Entry Configuration Reference +description: Learn how to configure the control-plane-request-limit configuration entry, which defines how Consul agents limit read and reqeust traffic rate limits. +--- + +# Control Plane Request Limit Configuration Entry Configuration Reference + +This topic describes the configuration options for the `control-plane-request-limit` configuration entry. You can only write the `control-plane-request-limit` configuration entry to the `default` partition, but the configuration entry applies to all client requests that target any partition. + +## Configuration model + +The following list outlines field hierarchy, language-specific data types, and requirements in a control plane request limit configuration entry. Click on a property name to view additional details, including default values. + +- [`kind`](#kind): string | required | must be set to `control-plane-request-limit` +- [`mode`](#mode): string | required | default is `permissive` +- [`name`](#name): string | required +- [`read_rate`](#read-rate): number | `100` +- [`write_rate`](#write-rate): number | `100` +- [`kv`](#kv): map | no default + - [`read_rate`](#kv-read-rate): number | `100` + - [`write_rate`](#kv-write-rate): number | `100` +- [`acl`](#acl): map | no default + - [`read_rate`](#acl-read-rate): number | `100` + - [`write_rate`](#acl-write-rate): number | `100` +- [`catalog`](#catalog): map + - [`read_rate`](#catalog-read-rate): number | default is `100` + - [`write_rate`](#catalog-write-rate): number | default is `100` + +## Complete configuration + +When every field is defined, a control plane request limit configuration entry has the following form: + + + +```hcl +kind = "control-plane-request-limit" +mode = "permissive" +name = "" +read_rate = 100 +write_rate = 100 +kv = { + read_rate = 100 + write_rate = 100 + } +acl = { + read_rate = 100 + write_rate = 100 +mode = "permissive" + } +catalog = { + read_rate = 100 + write_rate = 100 + } +``` + +```json +{ + "kind": "control-plane-request-limit", + "mode": "permissive", + "name": "", + "read_rate": 100, + "write_rate": 100, + "kv": { + "read_rate": 100, + "write_rate": 100 + }, + "acl": { + "read_rate": 100, + "write_rate": 100 + }, + "catalog: { + "read_rate": 100, + "write_rate": 100 + } +} +``` + +```yaml +kind: control-plane-request-limit +mode: permissive +name: +read_rate: 100 +write_rate: 100 +kv: + read_rate: 100 + write_rate: 100 +acl: + read_rate: 100 + write_rate: 100 +catalog: + read_rate: 100 + write_rate: 100 +``` + + + +## Specification + +This section provides details about the fields you can configure in the control plane request limit configuration entry. + +### `kind` + +Specifies the type of configuration entry to implement. Must be set to`control-plane-request-limit` + +#### Values + +- Default: none +- This field is required. +- Data type: String value that must be set to`control-plane-request-limit`. + +### `mode` + +Specifies an action to take if the rate of requests exceeds the limit. + +#### Values + +- Default: None +- This field is required. +- One of the following string values: + - `permissive`: The server continues to allow requests and records an error in the logs. This is the default value for `mode`. + - `enforcing`: The server stops accepting requests and records an error in the logs. + - `disabled`: Limits are not enforced or tracked. + +### `name` + +Specifies the name of the configuration entry. + +#### Values + +- Default: none +- This field is required. +- Data type: string + +### `read_rate` + +Specifies the maximum number of read requests per second that the agent allows. + +#### Values + +- Default: No limit. +- Data type: number + +### `write_rate` + +Specifies the maximum number of write requests per second that the agent allows. + +#### Values + +- Default: No limit. +- Data type: number + +### `kv` + +Specifies the maximum number of read and write requests to the Consul key-value store. + +#### Values + +- Default: none +- Data type is a map containing the following parameters: + - `read_rate` + - `write_rate` + +### `kv.read_rate` + +Specifies the maximum number of read requests per second allowed to the Consul key-value store. + +#### Values + +- Default: No limit. +- Data type: number + +### `kv.write_rate` + +Specifies the maximum number of write requests per second allowed to the Consul key-value store. + +#### Values + +- Default: No limit. +- Data type: number + +### `acl` + +Specifies the maximum number of read and write ACL requests to the Consul server. + +### `acl.read_rate` +S +pecifies the maximum number of ACL read requests per second allowed to the Consul server. + +#### Values + +- Default: No limit. +- Data type: number + +### `acl.write_rate` + +Specifies the maximum number of ACL write requests per second allowed to the Consul server. + +#### Values + +- Default: No limit. +- Data type: number + +### catalog + +Specifies the maximum number of read and write requests to the Consul catalog. + +### `catalog.read_rate` + +Specifies the maximum number of read requests per second allowed to the Consul catalog. + +#### Values + +- Default: No limit. +- Data type: number + +### `catalog.write_rate` + +Specifies the maximum number of write requests per second allowed to the Consul catalog. + +#### Values + +- Default: No limit. +- Data type: number \ No newline at end of file diff --git a/website/content/docs/connect/proxies/envoy-extensions/configuration/ext-authz.mdx b/website/content/docs/connect/proxies/envoy-extensions/configuration/ext-authz.mdx new file mode 100644 index 0000000000..6b5d8cc272 --- /dev/null +++ b/website/content/docs/connect/proxies/envoy-extensions/configuration/ext-authz.mdx @@ -0,0 +1,726 @@ +--- +layout: docs +page_title: External authorization extension configuration reference +description: Learn how to configure the ext-authz Envoy extension, which is a builtin Consul extension that configures Envoy proxies to request authorization from an external service. +--- + +# External authorization extension configuration reference + +This topic describes how to configure the external authorization Envoy extension, which configures Envoy proxies to request authorization from an external service. Refer to [Delegate authorization to an external service](/consul/docs/connect/proxies/envoy-extensions/usage/ext-authz) for usage information. + +## Configuration model + +The following list outlines the field hierarchy, data types, and requirements for the external authorization configuration. Place the configuration inside the `EnvoyExtension.Arguments` field in the proxy defaults or service defaults configuration entry. Refer to the following documentation for additional information: + +- [`EnvoyExtensions` in proxy defaults](/consul/docs/connect/config-entries/proxy-defaults#envoyextensions) +- [`EnvoyExtensions` in service defaults](/consul/docs/connect/config-entries/service-defaults#envoyextensions) + - [Envoy External Authorization documentation](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/ext_authz/v3/ext_authz.proto) + +Click on a property name to view additional details, including default values. + +- [`Name`](#name): string | required | must be set to `builtin/ext-authz` +- [`Arguments`](#arguments): map | required + - [`ProxyType`](#arguments-proxytype): string | required | `connect-proxy` + - [`InsertOptions`](#arguments-insertoptions): map + - [`Location`](#arguments-insertoptions-location): string + - [FilterName](#arguments-insertoptions-filtername): string + - [`Config`](#arguments-config): map | required + - [`BootstrapMetadataLabelsKey`](#arguments-config-bootstrapmetadatalabelskey): string + - [`ClearRouteCache`](#arguments-config-grpcservice): boolean | `false` | HTTP only + - [`GrpcService`](#arguments-config-grpcservice): map + - [`Target`](#arguments-config-grpcservice-target): map | required + - [`Service`](#arguments-config-grpcservice-target-service): map + - [`Name`](#arguments-config-grpcservice-target-service): string + - [`Namespace`](#arguments-config-grpcservice-target-service): string | + - [`Partition`](#arguments-config-grpcservice-target-service): string | + - [`URI`](#arguments-config-grpcservice-target-uri): string + - [`Timeout`](#arguments-config-grpcservice-target-uri): string | `1s` + - [`Authority`](#arguments-config-grpcservice-authority): string + - [`InitialMetadata`](#arguments-config-grpcservice-initialmetadata): list + - [`Key`](#arguments-config-grpcservice-initialmetadata): string + - [`Value`](#arguments-config-grpcservice-initialmetadata): string + - [`HttpService`](#arguments-config-httpservice): map + - [`Target`](#arguments-config-httpservice-target): map | required + - [`Service`](#arguments-config-httpservice): string + - [`Name`](#arguments-config-httpservice-target-service): string + - [`Namespace`](#arguments-config-httpservice-target-service): string | + - [`Partition`](#arguments-config-httpservice-target-service): string | + - [`URI`](#arguments-config-httpservice): string + - [`Timeout`](#arguments-config-httpservice): string | `1s` + - [`PathPrefix`](#arguments-config-httpservice-pathprefix): string + - [`AuthorizationRequest`](#arguments-config-httpservice-authorizationrequest): map + - [`AllowedHeaders`](#arguments-config-httpservice-authorizationrequest-allowedheaders): list + - [`Contains`](#arguments-config-httpservice-authorizationrequest-allowedheaders): string + - [`Exact`](#arguments-config-httpservice-authorizationrequest-allowedheaders): string + - [`IgnoreCase`](#arguments-config-httpservice-authorizationrequest-allowedheaders): boolean + - [`Prefix`](#arguments-config-httpservice-authorizationrequest-allowedheaders): string + - [`SafeRegex`](#arguments-config-httpservice-authorizationrequest-allowedheaders): string + - [`HeadersToAdd`](#arguments-config-httpservice-authorizationrequest-headerstoadd): list + - [`Key`](#arguments-config-httpservice-authorizationrequest-headerstoadd): string + - [`Value`](#arguments-config-httpservice-authorizationrequest-headerstoadd): string + - [`AuthorizationResponse`](#arguments-config-httpservice-authorizationresponse): map + - [`AllowedUpstreamHeaders`](#arguments-config-httpservice-authorizationresponse-allowedupstreamheaders): list + - [`Contains`](#arguments-config-httpservice-authorizationresponse-allowedupstreamheaders): string + - [`Exact`](#arguments-config-httpservice-authorizationresponse-allowedheaders): string + - [`IgnoreCase`](#arguments-config-httpservice-authorizationresponse-allowedheaders): boolean + - [`Prefix`](#arguments-config-httpservice-authorizationresponse-allowedheaders): string + - [`SafeRegex`](#arguments-config-httpservice-authorizationresponse-allowedheaders): string + - [`Suffix`](#arguments-config-httpservice-authorizationresponse-allowedheaders): string + - [`AllowedUpstreamHeadersToAppend`](#arguments-config-httpservice-authorizationresponse-allowedupstreamheaderstoappend): list + - [`Contains`](#arguments-config-httpservice-authorizationresponse-allowedupstreamheaderstoappend): string + - [`Exact`](#arguments-config-httpservice-authorizationresponse-allowedupstreamheaderstoappend): string + - [`IgnoreCase`](#arguments-config-httpservice-authorizationresponse-allowedupstreamheaderstoappend): boolean + - [`Prefix`](#arguments-config-httpservice-authorizationresponse-allowedupstreamheaderstoappend): string + - [`SafeRegex`](#arguments-config-httpservice-authorizationresponse-allowedupstreamheaderstoappend): string + - [`Suffix`](#arguments-config-httpservice-authorizationresponse-allowedupstreamheaderstoappend): string + - [`AllowedClientHeaders`](#arguments-config-httpservice-authorizationresponse-allowedclientheaders): list + - [`Contains`](#arguments-config-httpservice-authorizationresponse-allowedclientheaders): string + - [`Exact`](#arguments-config-httpservice-authorizationresponse-allowedclientheaders): string + - [`IgnoreCase`](#arguments-config-httpservice-authorizationresponse-allowedclientheaders): boolean + - [`Prefix`](#arguments-config-httpservice-authorizationresponse-allowedclientheaders): string + - [`SafeRegex`](#arguments-config-httpservice-authorizationresponse-allowedclientheaders): string + - [`Suffix`](#arguments-config-httpservice-authorizationresponse-allowedclientheaders): string + - [`AllowedClientHeadersOnSuccess`](#arguments-config-httpservice-authorizationresponse-allowedclientheadersonsuccess): list + - [`Contains`](#arguments-config-httpservice-authorizationresponse-allowedclientheadersonsuccess): string + - [`Exact`](#arguments-config-httpservice-authorizationresponse-allowedclientheadersonsuccess): string + - [`IgnoreCase`](#arguments-config-httpservice-authorizationresponse-allowedclientheadersonsuccess): boolean + - [`Prefix`](#arguments-config-httpservice-authorizationresponse-allowedclientheadersonsuccess): string + - [`SafeRegex`](#arguments-config-httpservice-authorizationresponse-allowedclientheadersonsuccess): string + - [`Suffix`](#arguments-config-httpservice-authorizationresponse-allowedclientheadersonsuccess): string + - [`DynamicMetadataFromHeaders`](#arguments-config-httpservice-authorizationresponse-dynamicmetadatafromheaders): list + - [`Contains`](#arguments-config-httpservice-authorizationresponse-dynamicmetadatafromheaders): string + - [`Exact`](#arguments-config-httpservice-authorizationresponse-dynamicmetadatafromheaders): string + - [`IgnoreCase`](#arguments-config-httpservice-authorizationresponse-dynamicmetadatafromheaders): boolean + - [`Prefix`](#arguments-config-httpservice-authorizationresponse-dynamicmetadatafromheaders): string + - [`SafeRegex`](#arguments-config-httpservice-authorizationresponse-dynamicmetadatafromheaders): string + - [`Suffix`](#arguments-config-httpservice-authorizationresponse-dynamicmetadatafromheaders): string + - [`IncludePeerCertificate`](#arguments-config-includepeercertificate): boolean | `false` + - [`MetadataContextNamespaces`](#arguments-config-metadatacontextnamespaces): list of strings | HTTP only + - [`StatusOnError`](#arguments-config-statusonerror): number | `403` | HTTP only + - [`StatPrefix`](#arguments-config-statprefix): string | `response` + - [`WithRequestBody`](#arguments-config-withrequestbody): map | HTTP only + - [`MaxRequestBytes`](#arguments-config-withrequestbody-maxrequestbytes): number + - [`AllowPartialMessage`](#arguments-config-withrequestbody-allowpartialmessage): boolean | `false` + - [`PackAsBytes`](#arguments-config-withrequestbody-packasbytes): boolean | `false` + +## Complete configuration + +When each field is defined, an `ext-authz` configuration has the following form: + +```hcl +Name = "builtin/ext-authz" +Arguments = { + ProxyType = "connect-proxy" + InsertOptions = { + Location = "" + FilterName = "" + } + Config = { + BootstrapMetadataLabelsKey = "" + ClearRouteCache = false // HTTP only + GrpcService = { + Target = { + Service = { + Name = "" + Namespace = "" + Partition = "" + URI = "" + Timeout = "1s" + Authority = "" + InitialMetadata = [ + "" : "" + HttpService = { + Target = { + Service = { + Name = "" + Namespace = "" + Partition = "" + URI = "" + Timeout = "1s" + } + } + PathPrefix = "//" + AuthorizationRequest = { + AllowedHeaders = [ + Contains = "", + Exact = "", + IgnoreCase = false, + Prefix = "", + SafeRegex = "" + ] + HeadersToAdd = [ + "
" = "
" + ] + } + AuthorizationResponse = { + AllowedUpstreamHeaders = [ + Contains = "", + Exact = "", + IgnoreCase = false, + Prefix = "", + SafeRegex = "" + Suffix = "" + ] + AllowedUpstreamHeadersToAppend = [ + Contains = "", + Exact = "", + IgnoreCase = false, + Prefix = "", + SafeRegex = "" + Suffix = "" + ] + AllowedClientHeaders = [ + Contains = "", + Exact = "", + IgnoreCase = false, + Prefix = "", + SafeRegex = "" + Suffix = "" + ] + AllowedClientHeadersOnSuccess = [ + Contains = "", + Exact = "", + IgnoreCase = false, + Prefix = "", + SafeRegex = "" + Suffix = "" + DynamicMetadataFromHeaders = [ + Contains = "", + Exact = "", + IgnoreCase = false, + Prefix = "", + SafeRegex = "" + Suffix = "" + ] + IncludePeerCertificate = false + MetadataContextNamespaces = [ + "" + ] + StatusOnError = 403 // HTTP only + StatPrefix = "response" + WithRequestBody = { //HTTP only + MaxRequestBytes = + AllowPartialMessage = false + PackAsBytes = false +``` + +## Specification + +This section provides details about the fields you can configure for the external authorization extension. +### `Name` + +Specifies the name of the extension. Must be set to `builtin/ext-authz`. + +#### Values + +- Default: None +- This field is required. +- Data type: String value set to `builtin/ext-authz`. + +### `Arguments` + +Contains the global configuration for the extension. + +#### Values + +- Default: None +- This field is required. +- Data type: Map + +### `Arguments.ProxyType` + +Specifies the type of Envoy proxy that this extension applies to. The extension only applies to proxies that match this type and is ignored for all other proxy types. The only supported value is `connect-proxy`. + +#### Values + +- Default: `connect-proxy` +- This field is required. +- Data type: String + +### `Arguments.InsertOptions` + +Specifies options for defining the insertion point for the external authorization filter in the Envoy filter chain. By default, the external authorization filter is inserted as the first filter in the filter chain per the default setting for the [`Location`](#arguments-insertoptions-location) field. + +#### Values + +- Default: None +- Data type: Map + +### `Arguments.InsertOptions.Location` + +Specifies the insertion point for the external authorization filter in the Envoy filter chain. You can specify one of the following string values: + +- `First`: Inserts the filter as the first filter in the filter chain, regardless of the filter specified in the `FilterName` field. +- `BeforeLast`: Inserts the filter before the last filter in the chain, regardless of the filter specified in the `FilterName` field. This allows the filter to be inserted after all other filters and immediately before the terminal filter. +- `AfterFirstMatch`: Inserts the filter after the first filter in the chain that has a name matching the value of the `FilterName` field. +- `AfterLastMatch`: Inserts the filter after the last filter in the chain that has a name matching the value of the `FilterName` field. +- `BeforeFirstMatch`: Inserts the filter before the first filter in the chain that has a name matching the value of the `FilterName` field. +- `BeforeLastMatch`: Inserts the filter before the last filter in the chain that has a name matching the value of the `FilterName` field. + +#### Values + +- Default: `BeforeFirstMatch` +- Data type: String + +### `Arguments.InsertOptions.FilterName` + +Specifies the name of an existing filter in the chain to match when inserting the external authorization filter. Specifying a filter name enables you to configure an insertion point relative to the position of another filter in the chain. + +#### Values + +- Default: `envoy.filters.network.tcp_proxy` for TCP services. `envoy.filters.http.router` for HTTP services. +- Data type: String + +### `Arguments.Config` + +Contains the configuration settings for the extension. + +#### Values + +- Default: None +- This field is required. +- Data type: Map + +### `Arguments.Config.BootstrapMetadataLabelsKey` + +Specifies a key from the Envoy bootstrap metadata. Envoy adds labels associated with the key to the authorization request context. + +#### Values + +- Default: None +- Data type: String + +### `Arguments.Config.ClearRouteCache` + +Directs Envoy to clear the route cache so that the external authorization service correctly affects routing decisions. If set to `true`, the filter clears all cached routes. + +Envoy also clears cached routes if the status returned from the authorization service is `200` for HTTP responses or `0` for gRPC responses. Envoy also clears cached routes if at least one authorization response header is added to the client request or is used for altering another client request header. + +#### Values + +- Default: `false` +- Data type: Boolean + + +### `Arguments.Config.GrpcService` + +Specifies the external authorization configuration for gRPC requests. Configure the `GrpcService` or the [`HttpService`](#arguments-config-httpservice) settings, but not both. + +#### Values + +- Default: None +- Either the `GrpcService` or the `HttpService` configuration is required. +- Data type: Map + +### `Arguments.Config.GrpcService.Target` + +Configuration for specifying the service to send gRPC authorization requests to. The `Target` field may contain the following fields: + +- [`Service`](#arguments-config-grpcservice-target-service) or [`Uri`](#arguments-config-grpcservice-target-uri) +- [`Timeout`](#arguments-config-grpcservice-target-timeout) + + +#### Values + +- Default: None +- This field is required. +- Data type: Map + +### `Arguments{}.Config{}.GrpcService{}.Target{}.Service{}` + +Specifies the upstream external authorization service. Configure this field when authorization requests are sent to an upstream service within the service mesh. The service must be configured as an upstream of the service that the filter is applied to. + +Configure either the `Service` field or the [`Uri`](#arguments-config-grpcservice-target-uri) field, but not both. + +#### Values + +- Default: None +- This field or [`Uri`](#arguments-config-grpcservice-target-uri) is required. +- Data type: Map + +The following table describes how to configure parameters for the `Service` field: + +| Parameter | Description | Data type | Default | +| --- | --- | --- | --- | +| `Name` | Specifies the name of the upstream service. | String | None | +| `Namespace` | Specifies the Consul namespace that the upstream service belongs to. | String | `default` | +| `Partition` | Specifies the Consul admin partition that the upstream service belongs to. | String | `default` | + +### `Arguments.Config.GrpcService.Target.Uri` + +Specifies the URI of the external authorization service. Configure this field when you must provide an explicit URI to the external authorization service, such as cases in which the authorization service is running on the same host or pod. If set, the value of this field must be either `localhost:` or `127.0.0.1:` + +Configure either the `Uri` field or the [`Service`](#arguments-config-grpcservice-target-service) field, but not both. + +#### Values + +- Default: None +- This field or [`Service`](#arguments-config-grpcservice-target-service) is required. +- Data type: String + +### `Arguments.Config.GrpcService.Target.Timeout` + +Specifies the maximum duration that a response can take to arrive upon request. + +#### Values + +- Default: `1s` +- Data type: String + +### `Arguments.Config.GrpcService.Authority` + +Specifies the authority header to send in the gRPC request. If this field is not set, the authority field is set to the cluster name. This field does not override the SNI that Envoy sends to the external authorization service. + +#### Values + +- Default: Cluster name +- Data type: String + +### `Arguments.Config.GrpcService.InitialMetadata[]` + +Specifies additional metadata to include in streams initiated to the `GrpcService`. You can specify metadata for injecting additional ad-hoc authorization headers, for example, `x-foo-bar: baz-key`. For more information, including details on header value syntax, refer to the [Envoy documentation on custom request headers](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_conn_man/headers#config-http-conn-man-headers-custom-request-headers). + +#### Values + +- Default: None +- Data type: List of one or more key-value pairs: + + - KEY: String + - VALUE: String + +### `Arguments{}.Config{}.HttpService{}` + +Contains the configuration for raw HTTP communication between the filter and the external authorization service. Configure the `HttpService` or the [`GrpcService`](#arguments-config-grpcservice) settings, but not both. + +#### Values + +- Default: None +- Either the `HttpService` or the `GrpcService` configuration is required. +- Data type: Map + +### `Arguments{}.Config{}.HttpService{}.Target{}` + +Configuration for specifying the service to send HTTP authorization requests to. The `Target` field may contain the following fields: + +- [`Service`](#arguments-config-httpservice-target-service) or [`Uri`](#arguments-config-httpservice-target-uri) +- [`Timeout`](#arguments-config-httpservice-target-timeout) + + +#### Values + +- Default: None +- This field is required. +- Data type: Map + +### `Arguments{}.Config{}.HttpService{}.Target{}.Service{}` + +Specifies the upstream external authorization service. Configure this field when HTTP authorization requests are sent to an upstream service within the service mesh. The service must be configured as an upstream of the service that the filter is applied to. + +Configure either the `Service` field or the [`Uri`](#arguments-config-httpservice-target-uri) field, but not both. + +#### Values + +- Default: None +- This field or [`Uri`](#arguments-config-httpservice-target-uri) is required. +- Data type: Map + +The following table describes how to configure parameters for the `Service` field: + +| Parameter | Description | Data type | Default | +| --- | --- | --- | --- | +| `Name` | Specifies the name of the upstream service. | String | None | +| `Namespace` | Specifies the Consul namespace that the upstream service belongs to. | String | `default` | +| `Partition` | Specifies the Consul admin partition that the upstream service belongs to. | String | `default` | + +### `Arguments{}.Config{}.HttpService{}.Target{}.Uri` + +Specifies the URI of the external authorization service. Configure this field when you must provide an explicit URI to the external authorization service, such as cases in which the authorization service is running on the same host or pod. + +Configure either the `Uri` field or the [`Service`](#arguments-config-httpservice-target-service) field, but not both. + +#### Values + +- Default: None +- This field or [`Service`](#arguments-config-httpservice-target-service) is required. +- Data type: String + +### `Arguments{}.Config{}.HttpService{}.Target{}.Timeout` + +Specifies the maximum duration that a response can take to arrive upon request. + +#### Values + +- Default: `1s` +- Data type: String + +### `Arguments{}.Config{}.HttpService{}.PathPrefix` + +Specifies a prefix for the value of the authorization request header `Path`. You must include the preceding forward slash (`/`). + +#### Values + +- Default: None +- Data type: String + +### `Arguments{}.Config{}.HttpService{}.AuthorizationRequest{}` + +HTTP-only configuration that controls the HTTP authorization request metadata. The `AuthorizationRequest` field may contain the following parameters: + +- [`AllowHeaders`](#arguments-config-httpservice-authorizationrequest-allowheaders) +- [`HeadersToAdd`](#arguments-config-httpservice-authorizationrequest-headerstoadd) + +#### Values + +- Default: None +- Data type: Map + +### `Arguments{}.Config{}.HttpService{}.AuthorizationRequest{}.AllowHeaders[]` + +Specifies a set of rules for matching client request headers. The request to the external authorization service includes any client request headers that satisfy any of the rules. Refer to the [Envoy documentation](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/ext_authz/v3/ext_authz.proto#extensions-filters-http-ext-authz-v3-extauthz) for a detailed explanation. + +#### Values + +- Default: None +- Data type: List of key-value pairs + +The following table describes the matching rules you can configure in the `AllowHeaders` field: + +@include 'envoy_ext_rule_matcher.mdx' + +### `Arguments{}.Config{}.HttpService{}.AuthorizationRequest{}.HeadersToAdd[]` + +Specifies a list of headers to include in the request to the authorization service. Note that Envoy overwrites client request headers with the same key. + +#### Values + +- Default: None +- Data type: List of one or more key-value pairs: + + - KEY: String + - VALUE: String + +### `Arguments{}.Config{}.HttpService{}.AuthorizationResponse{}` + +HTTP-only configuration that controls HTTP authorization response metadata. The `AuthorizationResponse` field may contain the following parameters: + +- [`AllowedUpstreamHeaders`](#arguments-config-httpservice-authorizationresponse-allowedupstreamheaders) +- [`AllowedUpstreamHeadersToAppend`](#arguments-config-httpservice-authorizationresponse-allowedupstreamheaderstoappend) +- [`AllowedClientHeaders`](#arguments-config-httpservice-authorizationresponse-allowedclientheaders) +- [`AllowedClientHeadersOnSuccess`](#arguments-config-httpservice-authorizationresponse-allowedclientheadersonsuccess) +- [`DynamicMetadataFromHeaders`](#arguments-config-httpservice-authorizationresponse-dynamicmetadatafromheaders) + +#### Values + +- Default: None +- Data type: Map + +### `Arguments{}.Config{}.HttpService{}.AuthorizationResponse{}.AllowedUpstreamHeaders[]` + +Specifies a set of rules for matching authorization response headers. Envoy adds any headers from the external authorization service to the client response that satisfy the rules. Envoy overwrites existing headers. + +#### Values + +- Default: None +- Data type: Map + +The following table describes the matching rules you can configure in the `AllowedUpstreamHeaders` field: + +@include 'envoy_ext_rule_matcher.mdx' + +### `Arguments{}.Config{}.HttpService{}.AuthorizationResponse{}.AllowedUpstreamHeadersToAppend[]` + +Specifies a set of rules for matching authorization response headers. Envoy appends any headers from the external authorization service to the client response that satisfy the rules. Envoy appends existing headers. + +#### Values + +- Default: None +- Data type: Map + +The following table describes the matching rules you can configure in the `AllowedUpstreamHeadersToAppend` field: + +@include 'envoy_ext_rule_matcher.mdx' + +### `Arguments{}.Config{}.HttpService{}.AuthorizationResponse{}.AllowedClientHeaders[]` + +Specifies a set of rules for matching client response headers. Envoy adds any headers from the external authorization service to the client response that satisfy the rules. When the list is not set, Envoy includes all authorization response headers except `Authority (Host)`. When a header is included in this list, Envoy automatically adds the following headers: + +- `Path` +- `Status` +- `Content-Length` +- `WWWAuthenticate` +- `Location` + +#### Values + +- Default: None +- Data type: Map + +The following table describes the matching rules you can configure in the `AllowedClientHeaders` field: + +@include 'envoy_ext_rule_matcher.mdx' + +### `Arguments{}.Config{}.HttpService{}.AuthorizationResponse{}.AllowedClientHeadersOnSuccess[]` + +Specifies a set of rules for matching client response headers. Envoy adds headers from the external authorization service to the client response when the headers satisfy the rules and the authorization is successful. If the headers match the rules but the authorization fails or is denied, the headers are not added. If this field is not set, Envoy does not add any additional headers to the client's response on success. + +#### Values + +- Default: None +- Data type: Map + +The following table describes the matching rules you can configure in the `AllowedClientHeadersOnSuccess` field: + +@include 'envoy_ext_rule_matcher.mdx' + +### `Arguments{}.Config{}.HttpService{}.AuthorizationResponse{}.DynamicMetadataFromHeaders[]` + +Specifies a set of rules for matching authorization response headers. Envoy emits headers from the external authorization service as dynamic metadata that the next filter in the chain can consume. + +#### Values + +- Default: None +- Data type: Map + +The following table describes the matching rules you can configure in the `DynamicMetadataFromHeaders` field: + +@include 'envoy_ext_rule_matcher.mdx' + +### `Arguments{}.Config{}.IncludePeerCertificate` + +If set to `true`, Envoy includes the peer X.509 certificate in the authorization request if the certificate is available. + +#### Values + +- Default: `false` +- Data type: Boolean + +### `Arguments{}.Config{}.MetadataContextNamespace[]` + +HTTP only field that specifies a list of metadata namespaces. The values of the namespaces are included in the authorization request context. The `consul` namespace is always included in addition to the namespaces you configure. + +#### Values + +- Default: `["consul"]` +- Data type: List of string values + +### `Arguments{}.Config{}.StatusOnError` + +HTTP only field that specifies a return code status to respond with on error. Refer to the [Envoy documentation](https://www.envoyproxy.io/docs/envoy/latest/api-v3/type/v3/http_status.proto#enum-type-v3-statuscode) for additional information. + +#### Values + +- Default: `403` +- Data type: Integer + +### `Arguments{}.Config{}.StatPrefix` + +Specifies a prefix to add when writing statistics. + +#### Values + +- Default: `response` +- Data type: String + +### `Arguments{}.Config{}.WithRequestBody{}` + +HTTP only field that configures Envoy to buffer the client request body and send it with the authorization request. If unset, the request body is not sent with the authorization request. + +#### Values + +- Default: None +- Data type: Map + +The following table describes the parameters that you can include in the `WithRequestBody` field: + +| Parameter | Description | Data type | Default | +| --- | --- | --- | --- | +| `MaxRequestBytes` | Specifies the maximum size of the message body that the filter holds in memory. Envoy returns HTTP `403` and does not initiate the authorization process when the buffer reaches the number set in this field unless `AllowPartialMessage` is set to `true`. | uint32 | None | +| `AllowPartialMessage` | If set to `true`, Envoy buffers the request body until the value of `MaxRequestBytes` is reached. The authorization request is dispatched with a partial body and no `413` HTTP error returns by the filter. | Boolean | `false` | +| `PackAsBytes` | If set to `true`, Envoy sends the request body to the external authorization as raw bytes. Otherwise, Envoy sends the request body as a UTF-8 encoded string. | Boolean | `false` | + +## Examples + +The following examples demonstrate common configuration patterns for specific use cases. + +### Authorize gRPC requests to a URI + +In the following example, a service defaults configuration entry contains an `ext-authz` configuration. The configuration allows the `api` service to make gRPC authorization requests to a service at `localhost:9191`: + +```hcl +Kind = "service-defaults" +Name = "api" +EnvoyExtensions = [ + { + Name = "builtin/ext-authz" + Arguments = { + ProxyType = "connect-proxy" + Config = { + GrpcService = { + Target = { + URI = "127.0.0.1:9191" + } + } + } + } + } +] +``` + +### Upstream authorization + +In the following example, a service defaults configuration entry contains an `ext-authz` configuration. The configuration allows the `api` service to make gRPC authorization requests to a service named `authz`: + +```hcl +Kind = "service-defaults" +Name = "api" +EnvoyExtensions = [ + { + Name = "builtin/ext-authz" + Arguments = { + ProxyType = "connect-proxy" + Config = { + GrpcService = { + Target = { + Service = { + Name = "authz" + } + } + } + } + } + } +] +``` + +### Authorization requests after service intentions for Consul Enterprise + +In the following example for Consul Enterprise, the `api` service is configured to make an HTTP authorization requests to a service named `authz` in the `foo` namespace and `bar` partition. Envoy also inserts the external authorization filter after the `envoy.filters.http.rbac` filter: + +```hcl +Kind = "service-defaults" +Name = "api" +Protocol = "http" +EnvoyExtensions = [ + { + Name = "builtin/ext-authz" + Arguments = { + ProxyType = "connect-proxy" + InsertOptions = { + Location = "AfterLastMatch" + FilterName = "envoy.filters.http.rbac" + } + Config = { + HttpService = { + Target = { + Service = { + Name = "authz" + Namespace = "foo" + Partition = "bar" + } + } + } + } + } + } +] +``` diff --git a/website/content/docs/connect/proxies/envoy-extensions/configuration/property-override.mdx b/website/content/docs/connect/proxies/envoy-extensions/configuration/property-override.mdx new file mode 100644 index 0000000000..9b13cb940b --- /dev/null +++ b/website/content/docs/connect/proxies/envoy-extensions/configuration/property-override.mdx @@ -0,0 +1,273 @@ +--- +layout: docs +page_title: Property override configuration reference +description: Learn how to configure the property-override plugin, which is a builtin Consul plugin that allows you to set and remove Envoy proxy properties. +--- + +# Property override configuration reference + +This topic describes how to configure the `property-override` extension so that you can set and remove individual properties on the Envoy resources Consul generates. Refer to [Configure Envoy proxy properties](/consul/docs/connect/proxies/envoy-extensions/usage/property-override) for usage information. + +## Configuration model + +The following list outlines the field hierarchy, data types, and requirements for the `property-override` configuration. Place the configuration inside the `EnvoyExtension.Arguments` field in the proxy defaults or service defaults configuration entry. Refer the following documentation for additional information: + +- [`EnvoyExtensions` in proxy defaults](/consul/docs/connect/config-entries/proxy-defaults#envoyextensions) +- [`EnvoyExtensions` in service defaults](/consul/docs/connect/config-entries/service-defaults#envoyextensions) + +Click on a property name to view additional details, including default values. + +- [`ProxyType`](#proxytype): string | required +- [`Debug`](#debug): bool | `false` +- [`Patches`](#patches): list | required + - [`ResourceFilter`](#patches-resourcefilter): map + - [`ResourceType`](#patches-resourcefilter-resourcetype): string | required + - [`TrafficDirection`](#patches-resourcefilter-trafficdirection): string | required + - [`Services`](#patches-resourcefilter-services): list + - [`Name`](#patches-resourcefilter-services-name): string + - [`Namespace`](#patches-resourcefilter-services-namespace): string | `default` | + - [`Partition`](#patches-resourcefilter-services-partition): string | `default` | + - [`Op`](#patches-op): string | required + - [`Path`](#patches-path): string | required + - [`Value`](#patches-value): map, number, boolean, or string + +## Complete configuration + +When each field is defined, a `property-override` configuration has the following form: + + +```hcl +ProxyType = "connect-proxy" +Debug = false +Patches = [ + { + ResourceFilter = { + ResourceType = "" + TrafficDirection = "" + Services = [ + { + Name = "", + Namespace = "" + Partition = "" + } + ] + Op = "", + Path = "", + Value = "" + } +] +``` + +## Specification + +This section provides details about the fields you can configure for the `property-override` extension. + +### `ProxyType` + +Specifies the type of Envoy proxy that the extension applies to. The only supported value is `connect-proxy`. + +#### Values + +- Default: `connect-proxy` +- This field is required. +- Data type: String + +### `Debug` + +Enables full debug mode. When `Debug` is set to `true`, all possible fields for the given `ResourceType` and first unmatched segment of `Path` are returned on error. When set to `false`, the error message only includes the first ten possible fields. + +#### Values + +- Default: `false` +- Data type: Boolean + +### `Patches[]` + +Specifies a list of one or more JSON Patches that map to the Envoy proxy configurations you want to modify. Refer to [IETF RFC 6902](https://datatracker.ietf.org/doc/html/rfc6902/) for information about the JSON Patch specification. + +#### Values + +- Default: None +- The `Patches` parameter is a list of configurations in JSON Patch format. Each patch can contain the following fields: + - [`ResourceFilter`](#patches-resourcefilter) + - [`Op`](#patches-op) + - [`Path`](#patches-path) + - [`Value`](#patches-value) + + +### `Patches[].ResourceFilter{}` + +Specifies the filter for targeting specific Envoy resources. The `ResourceFilter` configuration is not part of the JSON Patch specification. + +#### Values + +- Default: None +- This field is required. +- Data type: Map + +The following table describes how to configure a `ResourceFilter`: + +| Parameter | Description | Type | +| --- | --- | --- | +| `ProxyType` | Specifies the proxy type that the extension applies to. The only supported value is `connect-proxy`. | String | +| `ResourceType` | Specifies the Envoy resource type that the extension applies to. You can specify one of the following values for each `ResourceFilter`:
  • `cluster`
  • `cluster-load-assignment`
  • `route`
  • `listener`
| String | +| `TrafficDirection` | Specifies the type of traffic that the extension applies to relative to the current proxy. You can specify one of the following values for each `ResourceFilter`:
  • `inbound`: Targets resources for the proxy's inbound traffic.
  • `outbound`: Targets resources for the proxy's upstream services.
| String | +| `Services` | Specifies a list of services to target. Each member of the list has the following fields:
  • `Name`: Specifies the service associated with the traffic.
  • `Namespace`: Specifies the Consul Enterprise namespace the service is in.
  • `Partition`: Specifies the Consul Enterprise admin partition the service is in.
If `TrafficDirection` is set to `outbound`, upstream services in this field correspond to local Envoy resources that Consul patches at runtime.

Do not configure the `Services` field if `TrafficDirection` is set to `inbound`.

If this field is not set, Envoy targets all applicable resources. When patching outbound listeners, the patch includes the outbound transparent proxy listener only if `Services` is unset and if the local service is in transparent proxy mode. | List of maps | + +### `Patches[].Op` + +Specifies the JSON Patch operation to perform when the `ResourceFilter` matches a local Envoy proxy configuration. You can specify one of the following values for each patch: + +- `add`: Replaces a property or message specified by [`Path`](#patches-path) with the given value. The JSON patch format does not merge objects. To emulate merges, you must configure discrete `add` operations for each changed field. Consul returns an error if the target field does not exist in the corresponding schema. +- `remove`: Unsets the value of the field specified by [`Path`](#patches-path). If the field is not set, no changes are made. Consul returns an error if the target field does not exist in the corresponding schema. + +#### Values + +- Default: None +- This field is required. +- Data type is one of the following string values: + - `add` + - `remove` + +### `Patches[].Path` + +Specifies where the extension performs the associated operation on the specified resource type. Refer to [`ResourceType`](#patches-resourcefilter) for information about specifying a resource type to target. Refer to [`Op`](#patches-op) for information about setting an operation to perform on the resources. + +The `Path` field does not support addressing array elements or protobuf map field entries. Refer to [Constructing paths](/consul/docs/connect/proxies/envoy-extensions/usage/property-override#constructing-paths) for information about how to construct paths. + +When setting fields, the extension sets any unset intermediate fields to their default values. A a single operation on a nested field can set multiple intermediate fields. Because Consul sets the intermediate fields to their default values, you may need to configure subsequent patches to satisfy Envoy or Consul validation. + +#### Values + +- Default: None +- This field is required. +- Data type: String + +### `Patches[].Value{}` + +Defines a value to set at the specified [path](#patches-path) if the [operation](#patches-op) is set to `add`. You can specify either a scalar or enum value or define a map that contains string keys and values corresponding to scalar or enum child fields. Refer to the [example configurations](#examples) for additional guidance and to the [Envoy API documentation](https://www.envoyproxy.io/docs/envoy/latest/api-v3/api) for additional information about Envoy proxy interfaces. + +If Envoy specifies a wrapper as the target field type, the extension automatically coerces simple values to the wrapped type when patching. For example, the value `32768` is allowed when targeting a cluster's `per_connection_buffer_limit_bytes`, which is a `UInt32Value` field. Refer to the [protobuf documentation](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/wrappers.proto) for additional information about wrappers. +#### Values + +- Default: None +- This field is required if [`Op`](#patches-op) is set to `add`, otherwise you must omit the field. +- This field takes one of the following data types: + - scalar + - enum + - map + +## Examples + +The following examples demonstrate patterns that you may be able to model your configurations on. + +### Enable `enforcing_consecutive_5xx` outlier detection + +In the following example, the `add` operation patches an outlier detection property into outbound cluster traffic. The `Path` specifies the `enforcing_consecutive_5xx` interface and sets a value of `1234`: + +```hcl +Kind = "service-defaults" +Name = "my-svc" +Protocol = "http" +EnvoyExtensions = [ + { + Name = "builtin/property-override", + Arguments = { + ProxyType = "connect-proxy", + Patches = [ + { + "ResourceFilter" = { + "ResourceType" = "cluster", + "TrafficDirection" = "outbound", + "Service" = { + "Name" = "other-svc" + }, + }, + "Op" = "add", + "Path" = "/outlier_detection/enforcing_consecutive_5xx", + "Value" = 1234, + } + ] + } + } +] +``` + +### Update multiple values in the default map + +In the following example, two `ResourceFilter` blocks target outbound traffic to the `db` service and add `/outlier_detection/enforcing_consecutive_5xx` and `/outlier_detection/failure_percentage_request_volume` properties: + +```hcl +Kind = "service-defaults" +Name = "my-svc" +Protocol = "http" +EnvoyExtensions = [ + { + Name = "builtin/property-override", + Arguments = { + ProxyType = "connect-proxy", + Patches = [ + { + ResourceFilter = { + ResourceType = "cluster", + TrafficDirection = "outbound", + Services = [{ + Name = "other-svc" + }], + }, + Op = "add", + Path = "/outlier_detection/enforcing_consecutive_5xx", + Value = 1234, + }, + { + ResourceFilter = { + ResourceType = "cluster", + TrafficDirection = "outbound", + Services = [{ + Name = "other-svc" + }], + }, + Op = "add", + Path = "/outlier_detection/failure_percentage_request_volume", + Value = 2345, + } + ] + } + } +] +``` + +### Set multiple values that replace the map + +In the following example, a `ResourceFilter` targets outbound traffic to the `db` service and replaces the map of properties located at `/outlier_detection` with `enforcing_consecutive_5xx` and `failure_percentage_request_volume` and properties: + +```hcl +Kind = "service-defaults" +Name = "my-svc" +Protocol = "http" +EnvoyExtensions = [ + { + Name = "builtin/property-override", + Arguments = { + ProxyType = "connect-proxy", + Patches = [ + { + ResourceFilter = { + ResourceType = "cluster", + TrafficDirection = "outbound", + Services = [{ + Name = "other-svc" + }], + }, + Op = "add", + Path = "/outlier_detection", + Value = { + "enforcing_consecutive_5xx" = 1234, + "failure_percentage_request_volume" = 2345, + }, + } + ] + } + } +] +``` diff --git a/website/content/docs/connect/proxies/envoy-extensions/configuration/wasm.mdx b/website/content/docs/connect/proxies/envoy-extensions/configuration/wasm.mdx new file mode 100644 index 0000000000..ed1e2061a5 --- /dev/null +++ b/website/content/docs/connect/proxies/envoy-extensions/configuration/wasm.mdx @@ -0,0 +1,484 @@ +--- +layout: docs +page_title: WebAssembly extension configuration reference +description: Learn how to configure the wasm Envoy extension, which is a builtin Consul extension that allows you to run WebAssembly plugins in Envoy proxies. +--- + +# WebAssembly extension configuration reference + +This topic describes how to configure the `wasm` extension, which directs Consul to run WebAssembly (Wasm) plugins in Envoy proxies. Refer to [Run WebAssembly plug-ins in Envoy proxy](/consul/docs/connect/proxies/envoy-extensions/usage/wasm) for usage information. + +## Configuration model + +The following list outlines the field hierarchy, data types, and requirements for the `wasm` configuration. Place the configuration inside the `EnvoyExtension.Arguments` field in the proxy defaults or service defaults configuration entry. Refer the following documentation for additional information: + +- [`EnvoyExtensions` in proxy defaults](/consul/docs/connect/config-entries/proxy-defaults#envoyextensions) +- [`EnvoyExtensions` in service defaults](/consul/docs/connect/config-entries/service-defaults#envoyextensions) + +Click on a property name to view additional details, including default values. + +- [`Protocol`](#protocol): string +- [`ListenerType`](#listenertype): string | required +- [`ProxyType`](#proxytype): string | `connect-proxy` +- [`PluginConfig`](#pluginconfig): map | required + - [`Name`](#pluginconfig-name): string + - [`RootID`](#pluginconfig-rootid): string | required + - [`VmConfig`](#pluginconfig-vmconfig): map + - [`VmID`](#pluginconfig-vmconfig-vmid): string + - [`Runtime`](#pluginconfig-vmconfig): string | `v8` + - [`Code`](#pluginconfig-vmconfig-code): map + - [`Local`](#pluginconfig-vmconfig-code-local): map + - [`Filename`](#pluginconfig-vmconfig-code-local): string + - [`Remote`](#pluginconfig-vmconfig-code-remote): map + - [`HttpURI`](#pluginconfig-vmconfig-code-remote-httpuri): map + - [`Service`](#pluginconfig-vmconfig-code-remote-httpuri-service): map + - [`Name`](#pluginconfig-vmconfig-code-remote-httpuri-service): string + - [`Namespace`](#pluginconfig-vmconfig-code-remote-httpuri-service): string + - [`Partition`](#pluginconfig-vmconfig-code-remote-httpuri-service): string + - [`URI`](#pluginconfig-vmconfig-code-remote-httpuri-uri): string + - [`Timeout`](#pluginconfig-vmconfig-code-remote-httpuri-timeout): string + - [`SHA256`](#pluginconfig-vmconfig-code-remote-sha256): string + - [`RetryPolicy`](#pluginconfig-vmconfig-code-remote-retrypolicy): map + - [`RetryBackOff`](#pluginconfig-vmconfig-code-remote-retrypolicy-retrybackoff): map + - [`BaseInterval`](#pluginconfig-vmconfig-code-remote-retrypolicy-retrybackoff): string + - [`MaxInterval`](#pluginconfig-vmconfig-code-remote-retrypolicy-retrybackoff): string + - [`NumRetries`](#pluginconfig-vmconfig-code-remote-retrypolicy-numretries): number | `-1` + - [`Configuration`](#pluginconfig-vmconfig-configuration): string + - [`EnvironmentVariables`](#pluginconfig-vmconfig-environmentvariables): map + - [`HostEnvKeys`](#pluginconfig-vmconfig-environmentvariables-hostenvkeys): list of strings + - [`KeyValues`](#pluginconfig-vmconfig-environmentvariables-keyvalues): map + - [`Configuration`](#pluginconfig-configuration): string + - [`CapabilityRestrictionConfiguration`](#pluginconfig-vmconfig-capabilityrestrictionconfiguration): map + - [`AllowedCapabilities`](#pluginconfig-vmconfig-capabilityrestrictionconfiguration): map of strings + +## Complete configuration + +When all parameters are set for the extension, the configuration has the following form: + +```hcl +Protocol = "" +ListenerType = "" +ProxyType = "connect-proxy" +PluginConfig = { + Name = "" + RootID = "" + VmConfig = { + VmID = "" + Runtime = "v8" + Code = { + Local = { # Set either `Local` or `Remote', not both + Filename = "" + } + Remote = { # Set either `Local` or `Remote', not both + HttpURI = { + Service = { + Name = "" + Namespace = "" + Partition = "" + } + URI = "" + Timeout = "1s" + SHA256 = "" + RetryPolicy = { + RetryBackOff = { + BaseInterval = "1s" + MaxInterval = "10s" + } + NumRetries = -1 + } + } + Configuration = "" + EnvironmentVariables = { + HostEnvKeys = [ + <"keys"> + ] + KeyValues = { + [ + <"key = value"> + ] + } + } + Configuration = "" + CapabilityRestrictionConfiguration = { + AllowedCapabilities = { + "fd_read" = {} + "fd_seek" = {} + "environ_get" = {} + "clock_get_time" = {} + } + } +} +``` + +## Specification + +This section provides details about the fields you can configure for the `wasm` extension. + +### `Protocol` + +Specifies the type of Wasm filter to apply. You can set either `tcp` or `http`. Set the `Protocol` to the protocol that the Wasm plugin implements when loaded by the filter. For Consul to apply the filter, the protocol must match the service’s protocol. + +#### Values + +- Default: None +- This field is required. +- Data type is one of the following string values: + - `tcp` + - `http` + +### `ListenerType` + +Specifies the type of listener the extension applies to. The listener type is either `inbound` or `outbound`. If the listener type is set to `inbound`, Consul applies the extension so the Wasm plugin is run when other services in the mesh send messages to the service attached to the proxy. If the listener type is set to `outbound`, Consul applies the extension so the Wasm plugin is run when the attached proxy sends messages to other services in the mesh. + +#### Values + +- Default: None +- This field is required. +- Data type is one of the following string values: + - `inbound` + - `outbound` + +### `ProxyType` + +Specifies the type of Envoy proxy that the extension applies to. The only supported value is `connect-proxy`. + +#### Values + +- Default: `connect-proxy` +- This field is required. +- Data type: String + +### `PluginConfig{}` + +Map containing the following configuration parameters for your Wasm plugin: + +- [`Name`](#pluginconfig-name) +- [`RootID`](#pluginconfig-rootid) +- [`VmConfig`](#pluginconfig-vmconfig) +- [`Configuration`](#pluginconfig-configuration) +- [`CapabilitiesRestrictionConfiguration`](#pluginconfig-capabilitiesrestrictionconfiguration) + +#### Values + +- Default: None +- This field is required. +- Data type: Map + +### `PluginConfig{}.Name` + +Specifies a unique name for a filter in a VM. Envoy uses the name to identify specific filters if multiple filters are processed on a VM with the same `VmID` and `RootID`. The name also appears in logs for debugging purposes. + +#### Values + +- Default: None +- Data type: String + +### `PluginConfig{}.RootID` + +Specifies a unique ID for a set of filters in a VM that share a `RootContext` and `Contexts`, such as a Wasm `HttpFilter` and a Wasm `AccessLog`, if applicable. All filters with the same `RootID` and `VmID` share `Context`s. + +#### Values + +- Default: None +- Data type: String + +### `PluginConfig{}.VmConfig{}` + +Map containing the following configuration parameters for the VM that runs your Wasm plugin: + +- [`VmID`](#pluginconfig-vmconfig-vmid) +- [`Runtime`](#pluginconfig-vmconfig-runtime) +- [`Code`](#pluginconfig-vmconfig-code) +- [`Configuration`](#pluginconfig-vmconfig-configuration) +- [`EnvironmentVariables`](#pluginconfig-vmconfig-environmentvariables) + +#### Values + +- Default: None +- Data type: Map + +### `PluginConfig{}.VmConfig{}.VmID` + +Specifies an ID that Envoy uses with a hash of the Wasm code to determine which VM runs the plugin. All plugins with the same `VmID` and `Code` use the same VM. If unspecified, all plugins with the same code run in the same VM. Sharing a VM between plugins may have security implications, but can reduce memory utilization and can make data sharing easier. + +#### Values + +- Default: None +- Data type: String + +### `PluginConfig{}.VmConfig{}.Runtime` + +Specifies the type of Wasm runtime. +#### Values + +- Default: `v8` +- Data type is one of the following string values: + - `v8` + - `wastime` + - `wamr` + - `wavm` + +### `PluginConfig{}.VmConfig{}.Code{}` + +Map containing one of the following configuration parameters: + +- [`Local`](#pluginconfig-vmconfig-code-local) +- [`Remote`](#pluginconfig-vmconfig-code-local) + +You can configure either `Local` or `Remote`, but not both. The `Code` block instructs Consul how to find the Wasm plugin code for Envoy to execute. + +#### Values + +- Default: None +- This field is required. +- Data type is a map containing one of the following configurations: + - [`Local`](#pluginconfig-vmconfig-code-local) + - [`Remote`](#pluginconfig-vmconfig-code-local) + +### `PluginConfig{}.VmConfig{}.Code{}.Local{}` + +Instructs Envoy to load the plugin code from a local volume. Do not configure the `Local` parameter if the plugin code is on a remote server. + +The `Local` field is a map that contains a `Filename` parameter. The `Filename` parameter takes a string value that specifies the path to the plugin on the local file system. + +Local plug-ins are not supported in Kubernetes-orchestrated environments. + +#### Values + +- Default: None +- Data type is a map containing the `Filename` parameter. The `Filename` parameter takes a string value that specifies the path to the plugin on the local file system. + +### `PluginConfig{}.VmConfig{}.Code{}.Remote{}` + +Instructs Envoy to load the plugin code from a remote server. Do not configure the `Remote` parameter if the plugin code is on the local VM. + +The `Remote` field is a map containing the following parameters: + +- [`HttpURI`](#pluginconfig-vmconfig-code-remote-httpuri) +- [`SHA256`](#pluginconfig-vmconfig-code-remote-sha256) +- [`RetryPolicy`](#pluginconfig-vmconfig-code-remote-retrypolicy) + +#### Values + +- Default: None +- Data type: Map + +### `PluginConfig{}.VmConfig{}.Code{}.Remote{}.HttpURI{}` + +Specifies the configuration for fetching the remote data. The `HttpURI` field is a map containing the following parameters: + +- [`Service`](#pluginconfig-vmconfig-code-remote-httpuri-service) +- [`URI`](#pluginconfig-vmconfig-code-remote-httpuri-uri) +- [`Timeout`](#pluginconfig-vmconfig-code-remote-httpuri-uri) + +#### Values + +- Default: None +- Data type: Map + +### `PluginConfig{}.VmConfig{}.Code{}.Remote{}.HttpURI{}.Service` + +Specifies the upstream service to fetch the remote plugin from. + +#### Values + +- Default: None +- Data type: Map + +The following table describes the fields you can specify in the `Service` map: + +| Parameter | Description | Data type | Default | +| --- | --- | --- | --- | +| `Name` | Specifies the name of the upstream service. | String | None | +| `Namespace` | Specifies the Consul namespace that the upstream service belongs to. | String | `default` | +| `Partition` | Specifies the Consul admin partition that the upstream service belongs to. | String | `default` | + +### `PluginConfig{}.VmConfig{}.Code{}.Remote{}.HttpURI{}.URI` + +Specifies the URI Envoy uses to fetch the plugin file from the upstream. This field is required for Envoy to retrieve plugin code from a remote location. You must specify the fully-qualified domain name (FDQN) of the remote URI, which includes the protocol, host, and path. + +#### Values + +- Default: None +- This field is required. +- Data type: String value that specifies a FDQN + +### `PluginConfig{}.VmConfig{}.Code{}.Remote{}.HttpURI{}.Timeout` + +Specifies the maximum duration that a response can take to complete the request for the plugin data. + +#### Values + +- Default: `1s` +- Data type: String + +### `PluginConfig{}.VmConfig{}.Code{}.Remote{}.SHA256` + +Specifies the required SHA256 string for verifying the remote data. + +#### Values + +- Default: None +- This field is required. +- Data type: String + +### `PluginConfig{}.VmConfig{}.Code{}.Remote{}.RetryPolicy{}` + +Defines a policy for retrying requests to the upstream service when fetching the plugin data. The `RetryPolicy` field is a map containing the following parameters: + +- [`RetryBackoff`](#pluginconfig-vmconfig-code-remote-retrypolicy) +- [`NumRetries`](#pluginconfig-vmconfig-code-remote-numretries) + +#### Values + +- Default: None +- Data type: Map +### `PluginConfig{}.VmConfig{}.Code{}.Remote{}.RetryPolicy{}.RetryBackOff{}` + +Specifies parameters that control retry backoff strategy. + +#### Values + +- Default: None +- Data type: Map + +The following table describes the fields you can specify in the `RetryBackOff` map: + +| Parameter | Description | Data type | Default | +| --- | --- | --- | --- | +| `BaseInterval` | Specifies the base interval for determining the next backoff computation. Set a value greater than `0` and less than or equal to the `MaxInterval` value. | String | `1s` | +| `MaxInterval` | Specifies the maximum interval between retries. Set the value greater than or equal to the `BaseInterval` value. | String | `10s` | + +### `PluginConfig{}.VmConfig{}.Code{}.Remote{}.RetryPolicy{}.NumRetries` + +Specifies the number of times Envoy retries to fetch plugin data if the initial attempt is unsuccessful. + +#### Values + +- Default: `1` +- Data type: Integer + +### `PluginConfig{}.VmConfig{}.Configuration` + +Specifies the configuration Envoy encodes as bytes and passes to the plugin during VM startup. Refer to [`proxy_on_vm_start` in the Proxy Wasm ABI documentation](https://github.com/proxy-wasm/spec/tree/master/abi-versions/vNEXT#proxy_on_vm_start) for additional information. + +#### Values + +- Default: None +- This field is required. +- Data type: String + +### `PluginConfig{}.VmConfig{}.EnvironmentVariables{}` + +Specifies environment variables for Enovy to inject into this VM so that they are available through WASI’s `environ_get` and `environ_get_sizes` system calls. + +In most cases, WASI calls the functions implicitly in your language’s standard library. As a result, you do not need to call them directly. You can also access environment variables as you would on native platforms. + +Envoy rejects the configuration if there’s conflict of key space. + +The `EnvironmentVariables` field is a map containing parameters for setting the keys and values. + +#### Values + +- Default: None +- Data type: Map + +The following table describes the parameters contained in the `EnvironmentVariables` map: + +| Parameter | Description | Data type | Default | +| --- | --- | --- | --- | +| `HostEnvKeys` | Specifies a list of Envoy environment variable keys to expose to the VM. If a key exists in Envoy’s environment variables, then the key-value pair is injected. Envoy ignores `HostEnvKeys` that do not exist in its environment variables. | List | None | +| `KeyValues` | Specifies a map of explicit key-value pairs to inject into the VM. | Map of string keys and values | None | + +### `PluginConfig{}.Configuration` + +Specifies the configuration Consul encodes as bytes and passes to the plugin during plugin startup. Refer to [`proxy_on_configure` in the Envoy documentation](https://github.com/proxy-wasm/spec/tree/master/abi-versions/vNEXT#proxy_on_configure) for additional information. + +#### Values + +- Default: None +- Data type: String + +### `PluginConfig{}.CapabilityRestrictionConfiguration{}` + +Specifies a configuration for restricting the proxy-Wasm capabilities that are available to the module. + +The `CapabilityRestrictionConfiguration` field is a map that contains a `AllowedCapabilities` parameter. The `AllowedCapabilities` parameter takes a map of string values that correspond to Envoy capability names. Refer to the [Envoy documentation](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/wasm/v3/wasm.proto#extensions-wasm-v3-capabilityrestrictionconfig) for additional information. + +!> **Security warning**: Consul ignores the value that each capability maps to. You can leave the `AllowedCapabilities` empty to allow all capabilities, but doing so gives the configured plugin full unrestricted access to the runtime API provided by the Wasm VM. You must set this to a non-empty map if you want to restrict access to specific capabilities provided by the Wasm runtime API. + +#### Values + +- Default: `""` +- Data type is a map containing the `AllowedCapabilities` parameter. The `AllowedCapabilities` parameter takes a map of string values that correspond to Envoy capability names. Refer to the [Envoy documentation](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/wasm/v3/wasm.proto#extensions-wasm-v3-capabilityrestrictionconfig) for additional information. + +## Examples + +The following examples demonstrate patterns that you may be able to model your configurations on. + +### Run a Wasm plugin from a local file + +In the following example, Consul figures the Envoy proxy for the `db` service with an inbound TCP Wasm filter that uses the plugin code from the local `/consul/extensions/sqli.wasm` file. + +```hcl + +Kind = "service-defaults" +Name = "db" +Protocol = "tcp" +EnvoyExtensions = [ + { + Name = "builtin/wasm" + Required = true + Arguments = { + Protocol = "tcp" + ListenerType = “inbound” + PluginConfig = { + VmConfig = { + Code = { + Local = { + Filename = "file:///consul/extensions/sqli.wasm" + } + } + } + Configuration = < + + + +```hcl +Kind = "service-defaults" +Name = "api" +EnvoyExtensions = [ + { + Name = "builtin/ext-authz" + Arguments = { + ProxyType = "connect-proxy" + Config = { + GrpcService = { + Target = { + Service = { + Name = "authz" + } + } + } + } + } + } +] +``` + + + + + +```json +"Kind": "service-defaults", +"Name": "api", +"EnvoyExtensions": [{ + "Name": "builtin/ext-authz", + "Arguments": { + "ProxyType": "connect-proxy", + "Config": { + "GrpcService": { + "Target": { + "Service": { + "Name": "authz" + } + } + } + } + } + } +] +``` + + + + + + +```yaml +kind: ServiceDefaults +name: api +envoyExtensions: + - name: builtin/ext-authz + arguments: + proxyType: connect-proxy + config: + grpcService: + target: + service: + name: authz +``` + + + + +Refer to the [external authorization extension configuration reference](/consul/docs/connect/proxies/envoy-extensions/configuration/ext-authz) for details on how to configure the extension. + +Refer to the [proxy defaults configuration entry reference](/consul/docs/connect/config-entries/proxy-defaults) and [service defaults configuration entry reference](/consul/docs/connect/config-entries/service-defaults) for details on how to define the configuration entries. + +!> **Warning:** Adding Envoy extensions default proxy configurations may have unintended consequences. We recommend configuring `EnvoyExtensions` in service defaults configuration entries in most cases. + +### Unsupported Envoy configuration fields + +The following Envoy configurations are not supported: + +| Configuration | Workaround | +| --- | --- | +| `deny_at_disable` | Disable filter by removing it from the service’s configuration in the configuration entry. | +| `failure_mode_allow` | Set the `EnvoyExtension.Required` field to `true` in the [service defaults configuration entry](/consul/docs/connect/config-entries/service-defaults#envoyextensions) or [proxy defaults configuration entry](/consul/docs/connect/config-entries/proxy-defaults#envoyextensions). | +| `filter_enabled` | Set the `EnvoyExtension.Required` field to `true` in the [service defaults configuration entry](/consul/docs/connect/config-entries/service-defaults#envoyextensions) or [proxy defaults configuration entry](/consul/docs/connect/config-entries/proxy-defaults#envoyextensions). | +| `filter_enabled_metadata` | Set the `EnvoyExtension.Required` field to `true` in the [service defaults configuration entry](/consul/docs/connect/config-entries/service-defaults#envoyextensions) or [proxy defaults configuration entry](/consul/docs/connect/config-entries/proxy-defaults#envoyextensions). | +| `transport_api_version` | Consul only supports v3 of the transport API. As a result, there is no workaround for implement the behavior of this field. | + +## Apply the configuration entry + +If your network is deployed to virtual machines, use the `consul config write` command and specify the proxy defaults or service defaults configuration entry to apply the configuration. For Kubernetes-orchestrated networks, use the `kubectl apply` command. The following example applies the extension in a proxy defaults configuration entry. + + + + +```shell-session +$ consul config write api-auth-service-defaults.hcl +``` + + + + +```shell-session +$ consul config write api-auth-service-defaults.json + +``` + + + + +```shell-session +$ kubectl apply api-auth-service-defaults.yaml +``` + + + diff --git a/website/content/docs/connect/proxies/envoy-extensions/usage/property-override.mdx b/website/content/docs/connect/proxies/envoy-extensions/usage/property-override.mdx new file mode 100644 index 0000000000..0166f84ec1 --- /dev/null +++ b/website/content/docs/connect/proxies/envoy-extensions/usage/property-override.mdx @@ -0,0 +1,203 @@ +--- +layout: docs +page_title: Configure Envoy proxy properties +description: Learn how to use the property-override extension for Envoy proxies to set and remove individual properties for the Envoy resources Consul generates. +--- + +# Configure Envoy proxy properties + +This topic describes how to use the `property-override` extension to set and remove individual properties for the Envoy resources Consul generates. The extension uses the [protoreflect](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect), which enables Consul to dynamically manipulate messages. + +## Workflow + +- Complete the following steps to use the `property-override` extension: +- Configure an `EnvoyExtensions` block in a service defaults or proxy defaults configuration entry. +- Apply the configuration entry. + +!> **Security warning**: The property override extension is an advanced feature capable of introducing unintended consequences or reducing cluster security if used incorrectly. Consul does not enforce TLS retention, intentions, or other security-critical components of the Envoy configuration. Additionally, Consul does not verify that the configuration does not contain errors that affect service traffic. + +## Add the `EnvoyExtensions` + +Add Envoy extension configurations to a proxy defaults or service defaults configuration entry. Place the extension configuration in an `EnvoyExtensions` block in the configuration entry. + +- When you configure Envoy extensions on proxy defaults, they apply to every service. +- When you configure Envoy extensions on service defaults, they apply to a specific service. + +Consul applies Envoy extensions configured in proxy defaults before it applies extensions in service defaults. As a result, the Envoy extension configuration in service defaults may override configurations in proxy defaults. + +In the following service defaults configuration entry example, Consul adds a new `/upstream_connection_options/tcp_keepalive/keepalive_probes-5` field to each of the proxy's cluster configuration for the outbound `db`service upstream. The configuration applies to all `connect-proxy` proxies with services configured to communicate over HTTP: + + + + + +```hcl +Kind = "service-defaults" +Name = "global" +Protocol = "http" +EnvoyExtensions = [ + { + Name = "builtin/property-override" + Arguments = { + ProxyType = "connect-proxy", + Patches = [ + { + ResourceFilter = { + ResourceType = "cluster", + TrafficDirection = "outbound" + Services = [{ + Name = "other-svc" + }], + Op = "add" + Path = "/upstream_connection_options/tcp_keepalive/keepalive_probes", + Value = 5, + } + ] + } +] +``` + + + + + + +```json +"kind": "service-defaults", +"name": "global", +"protocol": "http", +"envoy_extensions": [{ + "name": "builtin/property-override", + "arguments": { + "proxyType": "connect-proxy", + "patches": [{ + "resourceFilter": { + "resourceType": "cluster", + "trafficDirection": "outbound", + "services": [{ "name": "other-svc" }], + "op": "add", + "path": "/upstream_connection_options/tcp_keepalive/keepalive_probes", + "value": 5 + } + }] + } +}] +``` + + + + + +```yaml +apiversion: consul.hashicorp.com/v1alpha1 +kind: ServiceDefaults +metadata: + name: global +spec: + protocol: http + envoyExtensions: + name = "builtin/property-override" + arguments: + proxyType: "connect-proxy", + patches: + - resourceFilter: + resourceType: "cluster" + trafficDirection: "outbound" + services: + - name: "other-svc" + op: "add" + path: "/upstream_connection_options/tcp_keepalive/keepalive_probes", + value: 5 +``` + + + + + + +Refer to the [property override configuration reference](/consul/docs/connect/proxies/envoy-extensions/configuration/property-override) for details on how to configure the extension. + +Refer to the [proxy defaults configuration entry reference](/consul/docs/connect/config-entries/proxy-defaults) and [service defaults configuration entry reference](/consul/docs/connect/config-entries/service-defaults) for details on how to define the configuration entries. + +!> **Warning:** Adding Envoy extensions default proxy configurations may have unintended consequences. We recommend configuring `EnvoyExtensions` in service defaults configuration entries in most cases. + +### Constructing paths + +To target the properties for an Envoy resource type, you must specify the path where the properties exist in the [`Path` field](/consul/docs/connect/proxies/envoy-extensions/configuration/property-override#patches-path) of the property override extension configuration. Set the `Path` field to an empty or partially invalid string when saving the configuration entry and Consul returns an error with a list of supported fields for the first unrecognized segment of the path. By default, Consul only returns the first ten fields, but you can set the [`Debug` field](/consul/docs/connect/proxies/envoy-extensions/configuration/property-override#debug) to `true` to direct Consul to output all possible fields. + +In the following example, Consul outputs the top-level fields available for the Envoy cluster resource: + +```hcl +Kind = "service-defaults" +Name = "api" +EnvoyExtensions = [ + { + Name = "builtin/property-override" + Arguments = { + ProxyType = "connect-proxy" + Patches = [ + { + ResourceFilter = { + ResourceType = "cluster" + TrafficDirection = "outbound" + } + Op = "add" + Path = "" + Value = 5 + } + ] + } + } +] +``` + +After applying the configuration entry, Consul prints a message that includes the possible fields for the resource: + +```shell-session +$ consul config write api.hcl +non-empty, non-root Path is required. available cluster fields: +/outlier_detection +/outlier_detection/enforcing_consecutive_5xx +/outlier_detection/failure_percentage_request_volume +/round_robin_lb_config +/round_robin_lb_config/slow_start_config +``` + +You can use the output to help you construct the appropriate value for the `Path` field. For example: + +```shell-session +$ consul config write api.hcl | grep round_robin +/round_robin_lb_config +``` + + + + +## Apply the configuration entry + +If your network is deployed to virtual machines, use the `consul config write` command and specify the proxy defaults or service defaults configuration entry to apply the configuration. For Kubernetes-orchestrated networks, use the `kubectl apply` command. The following example applies the extension in a proxy defaults configuration entry. + + + + +```shell-session +$ consul config write property-override-extension-service-defaults.hcl +``` + + + + +```shell-session +$ consul config write property-override-extension-service-defaults.json + +``` + + + + +```shell-session +$ kubectl apply property-override-extension-service-defaults.yaml +``` + + + diff --git a/website/content/docs/connect/proxies/envoy-extensions/usage/wasm.mdx b/website/content/docs/connect/proxies/envoy-extensions/usage/wasm.mdx new file mode 100644 index 0000000000..de899efe48 --- /dev/null +++ b/website/content/docs/connect/proxies/envoy-extensions/usage/wasm.mdx @@ -0,0 +1,191 @@ +--- +layout: docs +page_title: Run WebAssembly plug-ins in Envoy proxy +description: Learn how to use the Consul wasm extension for Envoy, which directs Consul to run your WebAssembly (Wasm) plugins for Envoy proxies in your service mesh. +--- + + +# Run WebAssembly plug-ins in Envoy proxy + +This topic describes how to use the `wasm` extension, which directs Consul to run your WebAssembly (Wasm) plug-ins for Envoy proxies. + +## Workflow + +You can create Wasm plugins for Envoy and integrate them using the `wasm` extension. Wasm is a binary instruction format for stack-based virtual machines that has the potential to run anywhere after it has been compiled. Wasm plug-ins run as filters in a service mesh application's sidecar proxy. + +The following steps describe the process of integrating Wasm plugins: + +- Create your Wasm plugin. You must ensure that your plugin functions as expected. Refer to the [WebAssembly website](https://webassembly.org/) for information and links to documentation. +- Configure an `EnvoyExtensions` block in a service defaults or proxy defaults configuration entry. +- Apply the configuration entry. + +## Add the `EnvoyExtensions` + +Add Envoy extension configuration to a proxy defaults or service defaults configuration entry. Place the extension configuration in an `EnvoyExtensions` block in the configuration entry. + +- When you configure Envoy extensions on proxy defaults, they apply to every service. +- When you configure Envoy extensions on service defaults, they apply to a specific service. + +Consul applies Envoy extensions configured in proxy defaults before it applies extensions in service defaults. As a result, the Envoy extension configuration in service defaults may override configurations in proxy defaults. + +In the following example, the extension uses an upstream service named `file-server` to serve a Wasm-based web application firewall (WAF). + + + + + +```hcl +Kind = "service-defaults" +Name = "api" +Protocol = "http" +EnvoyExtensions = [ + { + Name = "builtin/wasm" + Arguments = { + Protocol = "http" + ListenerType = "inbound" + PluginConfig = { + VmConfig = { + Code = { + Remote = { + HttpURI = { + Service = { + Name = "file-server" + } + URI = "https://file-server/waf.wasm" + } + SHA256 = "c9ef17f48dcf0738b912111646de6d30575718ce16c0cbde3e38b21bb1771807" + } + } + } + Configuration = < + + + + +```json +{ + "kind": "service-defaults", + "name": "api", + "protocol": "http", + "envoyExtensions": [{ + "name": "builtin/wasm", + "arguments": { + "protocol": "http", + "listenerType": "inbound", + "pluginConfig": { + "VmConfig": { + "Code": { + "Remote": { + "HttpURI": { + "Service": { + "Name": "file-server" + }, + "URI": "https://file-server/waf.wasm" + } + } + } + }, + "Configuration": { + "rules": [ + "Include @demo-conf", + "Include @crs-setup-demo-conf", + "SecDebugLogLevel 9", + "SecRuleEngine On", + "Include @owasp_crs/*.conf" + ] + } + + } + } + }] +} +``` + + + + + + + +```yaml +kind: service-defaults +name: api +protocol: http +envoyExtensions: + - name: builtin/wasm + required: true + arguments: + protocol: http + listenerType: inbound + pluginConfig: + VmConfig: + Code: + Remote: + HttpURI: + Service: + Name: file-server + URI: https://file-server/waf.wasm + Configuration: + rules: + - Include @demo-conf + - Include @crs-setup-demo-conf + - SecDebugLogLevel 9 + - SecRuleEngine On + - Include @owasp_crs/*.conf +``` + + + + + + +Refer to the [Wasm extension configuration reference](/consul/docs/connect/proxies/envoy-extensions/configuration/wasm) for details on how to configure the extension. + +Refer to the [proxy defaults configuration entry reference](/consul/docs/connect/config-entries/proxy-defaults) and [service defaults configuration entry reference](/consul/docs/connect/config-entries/service-defaults) for details on how to define the configuration entries. + +!> **Warning:** Adding Envoy extensions default proxy configurations may have unintended consequences. We recommend configuring `EnvoyExtensions` in service defaults configuration entries in most cases. + +## Apply the configuration entry + +If your network is deployed to virtual machines, use the `consul config write` command and specify the proxy defaults or service defaults configuration entry to apply the configuration. For Kubernetes-orchestrated networks, use the `kubectl apply` command. The following example applies the extension in a proxy defaults configuration entry. + + + + +```shell-session +$ consul config write wasm-extension-serve-waf.hcl +``` + + + + +```shell-session +$ consul config write wasm-extension-serve-waf.json +``` + + + + +```shell-session +$ kubectl apply wasm-extension-serve-waf.yaml +``` + + + diff --git a/website/content/docs/enterprise/license/utilization-reporting.mdx b/website/content/docs/enterprise/license/utilization-reporting.mdx new file mode 100644 index 0000000000..50db538609 --- /dev/null +++ b/website/content/docs/enterprise/license/utilization-reporting.mdx @@ -0,0 +1,168 @@ +--- +page_title: Automated license utilization reporting +description: >- + Learn what data HashiCorp collects to meter Enterprise license utilization. Enable or disable reporting. Review sample payloads and logs. +--- + +# Automated license utilization reporting + +Automated license utilization reporting sends license utilization data to HashiCorp without requiring you to manually collect and report them. It also enables you to review your license usage with the monitoring solution you already use, such as Splunk and Datadog, as you optimize and manage your deployments. You can use these reports to understand how much more you can deploy under your current contract, which can help you protect against overutilization and budget for predicted consumption. + +Automated reporting shares the minimum data required to validate license utilization as defined in our contracts. This data mostly consists of computed metrics, and it will never contain Personal Identifiable Information (PII) or other sensitive information. Automated reporting shares the data with HashiCorp using a secure unidirectional HTTPS API and makes an auditable record in the product logs each time it submits a report. + +## Enable automated reporting + +Before you enable automated reporting, make sure that outbound network traffic is configured correctly and upgrade your enterprise product to a version that supports it. If your installation is air-gapped or network settings are not in place, automated reporting will not work. + +To enable automated reporting, complete the following steps: + +1. [Allow outbound HTTPS traffic on port 443](#allow-outbound-https-traffic) +1. Upgrade to Consul Enterprise v1.16.0 or newer (#upgrade-to-consul-enterprise) +1. Check product logs(#check-product-logs) + +### Allow outbound HTTPS traffic on port 443 + +Make sure that your network allows HTTPS egress on port 443 from `https://reporting.hashicorp.services` by allow-listing the following IP addresses: + +- `100.20.70.12` +- `35.166.5.222` +- `23.95.85.111` +- `44.215.244.1` + +### Upgrade to Consul Enterprise v1.16.0 or newer + +Upgrade to a release that supports license utilization reporting. These [releases](https://releases.hashicorp.com/consul/) include: + +- Consul Enterprise 1.16.0 and newer. +- Consul Enterprise 1.15.4 and newer. +- Consul Enterprise 1.14.8 and newer. +- Consul Enterprise 1.13.9 and newer. + +### Check product logs + +Automatic license utilization reporting starts sending data within 24 hours. Check the product logs for records that the data sent successfully. + + + +``` +[DEBUG] beginning snapshot export +[DEBUG] creating payload +[DEBUG] marshalling payload to json +[DEBUG] generating authentication headers +[DEBUG] creating request +[DEBUG] sending request +[DEBUG] performing request: method=POST url=https://census.license.hashicorp.services +[DEBUG] recording audit record +[INFO] reporting: Report sent: auditRecord={"payload":{"payload_version":"1","license_id":"d2cdd857-4202-5a45-70a6-e4b531050c34","product":"consul","product_version":"1.16.0-dev+ent","export_timestamp":"2023-05-26T20:09:13.753921087Z","snapshots":[{"snapshot_version":1,"snapshot_id":"0001J724F90F4XWQDSAA76ZQWA","process_id":"01H1CTJPC1S8H7Q45MKTJ689ZW","timestamp":"2023-05-26T20:09:13.753513962Z","schema_version":"1.0.0","service":"consul","metrics":{"consul.billable.nodes":{"key":"consul.billable.nodes","kind":"counter","mode":"write","value":2},"consul.billable.service_instances":{"key":"consul.billable.service_instances","kind":"counter","mode":"write","value":2}}}],"metadata":{}}} +[DEBUG] completed recording audit record +[DEBUG] export finished successfully" +``` + + + +If your installation is air-gapped or your network does not allow the correct egress, the logs show an error. + + + +``` +[DEBUG] reporting: beginning snapshot export +[DEBUG] reporting: creating payload +[DEBUG] reporting: marshalling payload to json +[DEBUG] reporting: generating authentication headers +[DEBUG] reporting: creating request +[DEBUG] reporting: sending request +[DEBUG] reporting: performing request: method=POST url=https://census.license.hashicorp.services +[DEBUG] reporting: error status code received: statusCode=403 +``` + + + +In this case, reconfigure your network to allow egress and check back in 24 hours. + +## Opt out + +If your installation is air-gapped or you want to manually collect and report on the same license utilization metrics, you can opt out of automated reporting. + +Manually reporting these metrics can be time consuming. Opting out of automated reporting does not mean that you also opt out from sending license utilization metrics. Customers who opt out of automated reporting are still required to manually collect and send license utilization metrics to HashiCorp. + +If you are considering opting out because you are worried about the data, we strongly recommend that you review the [example payloads](#example-payloads) before opting out. If you have concerns with any of the automatically reported data, raise these concerns with your account manager. + +There are two methods for opting out of automated reporting: + +- HCL configuration (recommended) +- Environment variable (requires restart) + +We recommend opting out in your product's configuration file because it does not require a system restart. Add the following block to your `configuration.hcl` or `configuration.json` file. + +```hcl +reporting { + license { + enabled = false + } +} +``` + +When you opt out using an environment variable, once you restart your system it will provide a startup message confirming that you have disabled automated reporting. Set the following environment variable to disable automated reporting: + + + +```shell-session +$ export OPTOUT_LICENSE_REPORTING=true +``` + + + +After you set the environment variable, restart your system to complete the process for opting out. + +```shell-session +$ consul reload +``` + + +Check your product logs 24 hours after opting out to make sure that the system is not trying to send reports. Keep in mind that if your configuration file and environment variable differ, the environment variable setting takes precedence. + +## Example payloads + +HashiCorp collects the following utilization data as JSON payloads: +`exporter_version` - The version of the licensing exporter + + + +```json +{ + "payload": { + "payload_version": "1", + "license_id": "d2cdd857-4202-5a45-70a6-e4b531050c34", + "product": "consul", + "product_version": "1.16.0-dev+ent", + "export_timestamp": "2023-05-26T20:09:13.753921087Z", + "snapshots": [ + { + "snapshot_version": 1, + "snapshot_id": "0001J724F90F4XWQDSAA76ZQWA", + "process_id": "01H1CTJPC1S8H7Q45MKTJ689ZW", + "timestamp": "2023-05-26T20:09:13.753513962Z", + "schema_version": "1.0.0", + "service": "consul", + "metrics": { + "consul.billable.nodes": { + "key": "consul.billable.nodes", + "kind": "counter", + "mode": "write", + "value": 2 + }, + "consul.billable.service_instances": { + "key": "consul.billable.service_instances", + "kind": "counter", + "mode": "write", + "value": 2 + } + } + } + ], + "metadata": {} + } +} +``` + + \ No newline at end of file diff --git a/website/content/partials/envoy_ext_rule_matcher.mdx b/website/content/partials/envoy_ext_rule_matcher.mdx new file mode 100644 index 0000000000..1d97c34e4b --- /dev/null +++ b/website/content/partials/envoy_ext_rule_matcher.mdx @@ -0,0 +1,9 @@ + +| Rule | Description | Data type | Default | +| --- | --- | --- | --- | +| `Contains` | Specifies a string that the input string must contain. | String | N/A | +| `Exact` | Specifies a string that the input string must match exactly. | String | N/A | +| `IgnoreCase` | Directs Envoy to ignore capitalization. If set to `true`, the other matching rules in the configuration are not case sensitive. This rule does not affect `SafeRegex`. | Boolean | `false` | +| `Prefix` | Specifies a string that the input string must begin with. | String | N/A | +| `SafeRegex` | Specifies a regular expression for matching the input string programmatically. Envoy supports [Google's RE2 regex engine](https://github.com/google/re2/wiki/Syntax). | String | N/A | +| `Suffix` | Specifies a string that the input string must end with. | String | N/A | \ No newline at end of file diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 20335f715b..edd16a4eb8 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -394,14 +394,14 @@ "path": "connect/configuration" }, { - "title": "Configuration Entries", + "title": "Configuration entries", "routes": [ { "title": "Overview", "path": "connect/config-entries" }, { - "title": "API Gateway", + "title": "API gateway", "href": "/consul/docs/connect/gateways/api-gateway/configuration/api-gateway", "badge": { "text": "BETA", @@ -410,7 +410,7 @@ } }, { - "title": "HTTP Route", + "title": "HTTP route", "href": "/consul/docs/connect/gateways/api-gateway/configuration/http-route", "badge": { "text": "BETA", @@ -419,7 +419,7 @@ } }, { - "title": "TCP Route", + "title": "TCP route", "href": "/consul/docs/connect/gateways/api-gateway/configuration/tcp-route", "badge": { "text": "BETA", @@ -428,7 +428,7 @@ } }, { - "title": "Inline Certificate", + "title": "Inline certificate", "href": "/consul/docs/connect/gateways/api-gateway/configuration/inline-certificate", "badge": { "text": "BETA", @@ -437,7 +437,7 @@ } }, { - "title": "Ingress Gateway", + "title": "Ingress gateway", "path": "connect/config-entries/ingress-gateway" }, { @@ -445,36 +445,40 @@ "path": "connect/config-entries/mesh" }, { - "title": "Exported Services", + "title": "Exported services", "path": "connect/config-entries/exported-services" }, { - "title": "Proxy Defaults", + "title": "Proxy defaults", "path": "connect/config-entries/proxy-defaults" }, { - "title": "Service Defaults", + "title": "Service defaults", "path": "connect/config-entries/service-defaults" }, { - "title": "Service Intentions", + "title": "Service intentions", "path": "connect/config-entries/service-intentions" }, { - "title": "Service Resolver", + "title": "Service resolver", "path": "connect/config-entries/service-resolver" }, { - "title": "Service Router", + "title": "Service router", "path": "connect/config-entries/service-router" }, { - "title": "Service Splitter", + "title": "Service splitter", "path": "connect/config-entries/service-splitter" }, { - "title": "Terminating Gateway", + "title": "Terminating gateway", "path": "connect/config-entries/terminating-gateway" + }, + { + "title": "Control plane request limit", + "path": "connect/config-entries/control-plane-request-limit" } ] }, @@ -499,6 +503,10 @@ { "title": "Usage", "routes": [ + { + "title": "Delegate authorization to external services", + "path": "connect/proxies/envoy-extensions/usage/ext-authz" + }, { "title": "Run Lua scripts in Envoy proxies", "path": "connect/proxies/envoy-extensions/usage/lua" @@ -506,8 +514,32 @@ { "title": "Invoke Lambda functions in Envoy proxies", "path": "connect/proxies/envoy-extensions/usage/lambda" + }, + { + "title": "Configure Envoy proxy properties", + "path": "connect/proxies/envoy-extensions/usage/property-override" + }, + { + "title": "Run WebAssembly plug-ins in Envoy proxies", + "path": "connect/proxies/envoy-extensions/usage/wasm" + } ] + }, + { + "title": "Configuration", + "routes": [ + { + "title": "External authorization", + "path": "connect/proxies/envoy-extensions/configuration/ext-authz" + }, + { + "title": "Property override", + "path": "connect/proxies/envoy-extensions/configuration/property-override" + }, + { + "title": "WebAssembly", + "path": "connect/proxies/envoy-extensions/configuration/wasm" } - ] + ] } ] }, @@ -960,19 +992,36 @@ ] }, { - "title": "Limit Traffic Rates", + "title": "Limit traffic rates", "routes": [ { "title": "Overview", "path": "agent/limits" }, { - "title": "Initialize Rate Limit Settings", - "path": "agent/limits/init-rate-limits" + "title": "Usage", + "routes": [ + { + "title": "Initialize rate limit settings", + "path": "agent/limits/usage/init-rate-limits" + }, + { + "title": "Monitor traffic rate limits", + "path": "agent/limits/usage/monitor-rate-limits" + }, + { + "title": "Set global traffic rate limits", + "path": "agent/limits/usage/set-global-traffic-rate-limits" + }, + { + "title": "Limit traffic rates from source IP addresses", + "path": "agent/limits/usage/limit-request-rates-from-ips" + } + ] }, { - "title": "Set Global Traffic Rate Limits", - "path": "agent/limits/set-global-traffic-rate-limits" + "title": "Configuration", + "href": "/docs/connect/config-entries/control-plane-request-limit" } ] }, @@ -1607,6 +1656,10 @@ "title": "Overview", "path": "enterprise/license/overview" }, + { + "title": "Automated entitlement utilization reporting", + "path": "enterprise/license/utilization-reporting" + }, { "title": "FAQ", "path": "enterprise/license/faq"