[WIP] Initial draft of Sidecar Service and Managed Proxy deprecation docs (#4752)

* Initial draft of Sidecar Service and Managed Proxy deprecation docs

* Service definition deprecation notices and sidecar service

* gRPC and sidecar service config options; Deprecate managed proxy options

* Envoy Docs: Basic envoy command; envoy getting started/intro

* Remove change that snuck in

* Envoy custom config example

* Add agent/service API docs; deprecate proxy config endpoint

* Misc grep cleanup for managed proxies; capitalize Envoy

* Updates to getting started guide

* Add missing link

* Refactor Envoy guide into a separate guide and add bootstrap reference notes.

* Add limitations to Envoy docs; Highlight no fixes for known managed proxy issues on deprecation page; clarify snake cae stuff; Sidecar Service lifecycle
This commit is contained in:
Paul Banks 2018-10-11 10:44:42 +01:00 committed by GitHub
parent 852b930739
commit 51c0001aad
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
22 changed files with 1742 additions and 457 deletions

View File

@ -2386,7 +2386,7 @@ func (a *Agent) addProxyLocked(proxy *structs.ConnectManagedProxy, persist, From
return nil
}
// addProxyLocked adds a new local Connect Proxy instance to be managed by the agent.
// AddProxy adds a new local Connect Proxy instance to be managed by the agent.
//
// It REQUIRES that the service that is being proxied is already present in the
// local state. Note that this is only used for agent-managed proxies so we can

View File

@ -218,12 +218,14 @@ $ curl \
- `ValidBefore` `(string)` - The time before which the certificate is valid.
Used with `ValidAfter` this can determine the validity period of the certificate.
## Managed Proxy Configuration
## Managed Proxy Configuration ([Deprecated](/docs/connect/proxies/managed-deprecated.html))
This endpoint returns the configuration for a
[managed proxy](/docs/connect/proxies.html).
Ths endpoint is only useful for _managed proxies_ and not relevant
for unmanaged proxies.
This endpoint returns the configuration for a [managed
proxy](/docs/connect/proxies.html). Ths endpoint is only useful for _managed
proxies_ and not relevant for unmanaged proxies. This endpoint will be removed
in a future major release as part of [managed proxy
deprecation].(/docs/connect/proxies/managed-deprecated.html). The equivalent API
for use will all future proxies is the more generic `
Managed proxy configuration is set in the service definition. When Consul
starts the managed proxy, it provides the service ID and ACL token. The proxy
@ -242,7 +244,10 @@ The table below shows this endpoint's support for
| Blocking Queries | Consistency Modes | Agent Caching | ACL Required |
| ---------------- | ----------------- | ------------- | ---------------------------- |
| `YES` | `all` | `none` | `service:write, proxy token` |
| `YES`<sup>1</sup>| `all` | `none` | `service:write, proxy token` |
<sup>1</sup> Supports [hash-based
blocking](/api/index.html#hash-based-blocking-queries) only.
### Parameters

View File

@ -63,6 +63,90 @@ $ curl \
}
```
## Get Service Configuration
This endpoint was added in Consul 1.3.0 and returns the full service definition
for a single service instance registered on the local agent. It is used by
[Connect proxies](/docs/connect/proxies.html) to discover the embedded proxy
configuration that was registered with the instance.
It is important to note that the services known by the agent may be different
from those reported by the catalog. This is usually due to changes being made
while there is no leader elected. The agent performs active
[anti-entropy](/docs/internals/anti-entropy.html), so in most situations
everything will be in sync within a few seconds.
| Method | Path | Produces |
| ------ | ---------------------------- | -------------------------- |
| `GET` | `/agent/service/:service_id` | `application/json` |
The table below shows this endpoint's support for
[blocking queries](/api/index.html#blocking-queries),
[consistency modes](/api/index.html#consistency-modes),
[agent caching](/api/index.html#agent-caching), and
[required ACLs](/api/index.html#acls).
| Blocking Queries | Consistency Modes | Agent Caching | ACL Required |
| ---------------- | ----------------- | ------------- | -------------- |
| `YES`<sup>1</sup>| `none` | `none` | `service:read` |
<sup>1</sup> Supports [hash-based
blocking](/api/index.html#hash-based-blocking-queries) only.
### Parameters
- `service_id` `(string: <required>)` - Specifies the ID of the service to
fetch. This is specified as part of the URL.
### Sample Request
```text
$ curl \
http://127.0.0.1:8500/v1/agent/service/web-sidecar-proxy
```
### Sample Response
```json
{
"Kind": "connect-proxy",
"ID": "web-sidecar-proxy",
"Service": "web-sidecar-proxy",
"Tags": null,
"Meta": null,
"Port": 18080,
"Address": "",
"Weights": {
"Passing": 1,
"Warning": 1
},
"EnableTagOverride": false,
"ContentHash": "4ecd29c7bc647ca8",
"Proxy": {
"DestinationServiceName": "web",
"DestinationServiceID": "web",
"LocalServiceAddress": "127.0.0.1",
"LocalServicePort": 8080,
"Config": {
"foo": "bar"
},
"Upstreams": [
{
"DestinationType": "service",
"DestinationName": "db",
"LocalBindPort": 9191
}
]
}
}
```
The response has the same structure as the [service
definition](/docs/agent/services.html) with one extra field `ContentHash` which
contains the [hash-based blocking
query](/api/index.html#hash-based-blocking-queries) hash for the result. The
same hash is also present in `X-Consul-ContentHash`.
## Register Service
This endpoint adds a new service, with an optional health check, to the local
@ -126,8 +210,7 @@ service definition keys for compatibility with the config file format.
- `Proxy` `(Proxy: nil)` - From 1.2.3 on, specifies the configuration for a
Connect proxy instance. This is only valid if `Kind == "connect-proxy"`. See
the [Unmanaged Proxy](/docs/connect/proxies.html#unmanaged-proxies)
documentation for full details.
the [Proxy documentation](/docs/connect/proxies.html) for full details.
- `Connect` `(Connect: nil)` - Specifies the
[configuration for Connect](/docs/connect/configuration.html). See the
@ -178,10 +261,14 @@ For the `Connect` field, the parameters are:
the [Connect](/docs/connect/index.html) protocol [natively](/docs/connect/native.html).
If this is true, then Connect proxies, DNS queries, etc. will be able to
service discover this service.
- `Proxy` `(Proxy: nil)` - Specifies that a managed Connect proxy should be
started for this service instance, and optionally provides configuration for
the proxy. The format is as documented in
[Managed Proxies](/docs/connect/proxies.html#managed-proxies) .
- `Proxy` `(Proxy: nil)` -
[**Deprecated**](/docs/connect/proxies/managed-deprecated.html) Specifies that
a managed Connect proxy should be started for this service instance, and
optionally provides configuration for the proxy. The format is as documented
in [Managed Proxy Deprecation](/docs/connect/proxies/managed-deprecated.html).
- `SidecarService` `(ServiceDefinition: nil)` - Specifies an optional nested
service definition to register. For more information see
[Sidecar Service Registration](/docs/connect/proxies/sidecar-service.html).
### Sample Payload

View File

@ -506,7 +506,7 @@ $ curl \
removed in a future major version release.
- `ServiceProxy` is the proxy config as specified in
[Unmanaged Proxies](/docs/connect/proxies.html#unmanaged-proxies).
[Connect Proxies](/docs/connect/proxies.html).
- `ServiceConnect` are the [Connect](/docs/connect/index.html) settings. The
value of this struct is equivalent to the `Connect` field for service

View File

@ -77,6 +77,29 @@ to the supplied maximum `wait` time to spread out the wake up time of any
concurrent requests. This adds up to `wait / 16` additional time to the maximum
duration.
### Hash-based Blocking Queries
A limited number of agent endpoints also support blocking however because the
state is local to the agent and not managed with a consistent raft index, their
blocking mechanism is different.
Since there is no monotonically increasing index, each response instead contains
a header `X-Consul-ContentHash` which is an opaque hash digest generated by
hashing over all fields in the response that are relevant.
Subsequent requests may be sent with a query parameter `hash=<value>` where
`value` is the last hash header value seen, and this will block until the `wait`
timeout is passed or until the local agent's state changes in such a way that
the hash would be different.
Other than the different header and query parameter names, the biggest
difference is that hash values are opaque and can't be compared to see if one
result is older or newer than another. In general hash-based blocking will not
return too early due to an idempotent update since the hash will remain the same
unless the result actually changes, however as with index-based blocking there
is no strict guarantee that clients will never observe the same result delivered
before the full timeout has elapsed.
## Consistency Modes
Most of the read query endpoints support multiple levels of consistency. Since

View File

@ -172,10 +172,11 @@ will exit with an error at startup.
* <a name="_dev"></a><a href="#_dev">`-dev`</a> - Enable development server
mode. This is useful for quickly starting a Consul agent with all persistence
options turned off, enabling an in-memory server which can be used for rapid
prototyping or developing against the API. In this mode,
[Connect is enabled](/docs/connect/configuration.html) and will by default
create a new root CA certificate on startup. This mode is **not** intended for
production use as it does not write any data to disk.
prototyping or developing against the API. In this mode, [Connect is
enabled](/docs/connect/configuration.html) and will by default create a new
root CA certificate on startup. This mode is **not** intended for production
use as it does not write any data to disk. The gRPC port is also defaulted to
`8502` in this mode.
* <a name="_disable_host_node_id"></a><a href="#_disable_host_node_id">`-disable-host-node-id`</a> - Setting
this to true will prevent Consul from using information from the host to generate a deterministic node ID,
@ -216,6 +217,10 @@ will exit with an error at startup.
initialized with an encryption key, then the provided key is ignored and
a warning will be displayed.
* <a name="_grpc_port"></a><a href="#_grpc_port">`-grpc-port`</a> - the gRPC API
port to listen on. Default -1 (gRPC disabled). See [ports](#ports)
documentation for more detail.
* <a name="_hcl"></a><a href="#_hcl">`-hcl`</a> - A HCL configuration fragment.
This HCL configuration fragment is appended to the configuration and allows
to specify the full range of options of a config file on the command line.
@ -225,6 +230,7 @@ will exit with an error at startup.
This overrides the default port 8500. This option is very useful when deploying Consul
to an environment which communicates the HTTP port through the environment e.g. PaaS like CloudFoundry, allowing
you to set the port directly via a Procfile.
* <a name="_log_file"></a><a href="#_log_file">`-log-file`</a> - to redirect all the Consul agent log messages to a file. This can be specified with the complete path along with the name of the log. In case the path doesn't have the filename, the filename defaults to Consul-timestamp.log . Can be combined with <a href="#_log_rotate_bytes"> -log-rotate-bytes</a> and <a href="#_log_rotate_duration"> -log-rotate-duration </a> for a fine-grained log rotation experience.
* <a name="_log_rotate_bytes"></a><a href="#_log_rotate_bytes">`-log-rotate-bytes`</a> - to specify the number of bytes that should be written to a log before it needs to be rotated. Unless specified, there is no limit to the number of bytes that can be written to a log file.
@ -477,7 +483,7 @@ definitions support being updated during a reload.
"https": "0.0.0.0"
},
"ports": {
"https": 8080
"https": 8501
},
"key_file": "/etc/pki/tls/private/my.key",
"cert_file": "/etc/pki/tls/certs/my.crt",
@ -489,11 +495,13 @@ See, especially, the use of the `ports` setting:
```javascript
"ports": {
"https": 8080
"https": 8501
}
```
Consul will not enable TLS for the HTTP API unless the `https` port has been assigned a port number `> 0`.
Consul will not enable TLS for the HTTP API unless the `https` port has been
assigned a port number `> 0`. We recommend using `8501` for `https` as this
default will automatically work with some tooling.
#### Configuration Key Reference
@ -586,24 +594,27 @@ Consul will not enable TLS for the HTTP API unless the `https` port has been ass
addresses to bind to, or a [go-sockaddr](https://godoc.org/github.com/hashicorp/go-sockaddr/template)
template that can potentially resolve to multiple addresses.
`http` supports binding to a Unix domain socket. A socket can be
specified in the form `unix:///path/to/socket`. A new domain socket will be
created at the given path. If the specified file path already exists, Consul
will attempt to clear the file and create the domain socket in its place. The
permissions of the socket file are tunable via the [`unix_sockets` config construct](#unix_sockets).
`http`, `https` and `grpc` all support binding to a Unix domain socket. A
socket can be specified in the form `unix:///path/to/socket`. A new domain
socket will be created at the given path. If the specified file path already
exists, Consul will attempt to clear the file and create the domain socket
in its place. The permissions of the socket file are tunable via the
[`unix_sockets` config construct](#unix_sockets).
When running Consul agent commands against Unix socket interfaces, use the
`-http-addr` argument to specify the path to the socket. You can also place
the desired values in the `CONSUL_HTTP_ADDR` environment variable.
For TCP addresses, the variable values should be an IP address with the port. For
example: `10.0.0.1:8500` and not `10.0.0.1`. However, ports are set separately in the
<a href="#ports">`ports`</a> structure when defining them in a configuration file.
For TCP addresses, the environment variable value should be an IP address
_with the port_. For example: `10.0.0.1:8500` and not `10.0.0.1`. However,
ports are set separately in the <a href="#ports">`ports`</a> structure when
defining them in a configuration file.
The following keys are valid:
- `dns` - The DNS server. Defaults to `client_addr`
- `http` - The HTTP API. Defaults to `client_addr`
- `https` - The HTTPS API. Defaults to `client_addr`
- `grpc` - The gRPC API. Defaults to `client_addr`
* <a name="advertise_addr"></a><a href="#advertise_addr">`advertise_addr`</a> Equivalent to
the [`-advertise` command-line flag](#_advertise).
@ -748,13 +759,13 @@ Consul will not enable TLS for the HTTP API unless the `https` port has been ass
has been inactive (rotated out) for more than twice the *current* `leaf_cert_ttl`, it will be removed from
the trusted list.
* <a name="connect_proxy"></a><a href="#connect_proxy">`proxy`</a> This object allows setting options for the Connect proxies. The following sub-keys are available:
* <a name="connect_proxy"></a><a href="#connect_proxy">`proxy`</a> [**Deprecated**](/docs/connect/proxies/managed-deprecated.html) This object allows setting options for the Connect proxies. The following sub-keys are available:
* <a name="connect_proxy_allow_managed_registration"></a><a href="#connect_proxy_allow_managed_registration">`allow_managed_api_registration`</a> Allows managed proxies to be configured with services that are registered via the Agent HTTP API. Enabling this would allow anyone with permission to register a service to define a command to execute for the proxy. By default, this is false to protect against arbitrary process execution.
* <a name="connect_proxy_allow_managed_registration"></a><a href="#connect_proxy_allow_managed_registration">`allow_managed_api_registration`</a> [**Deprecated**](/docs/connect/proxies/managed-deprecated.html) Allows managed proxies to be configured with services that are registered via the Agent HTTP API. Enabling this would allow anyone with permission to register a service to define a command to execute for the proxy. By default, this is false to protect against arbitrary process execution.
* <a name="connect_proxy_allow_managed_root"></a><a href="#connect_proxy_allow_managed_root">`allow_managed_root`</a> Allows Consul to start managed proxies if Consul is running as root (EUID of the process is zero). We recommend running Consul as a non-root user. By default, this is false to protect inadvertently running external processes as root.
* <a name="connect_proxy_allow_managed_root"></a><a href="#connect_proxy_allow_managed_root">`allow_managed_root`</a> [**Deprecated**](/docs/connect/proxies/managed-deprecated.html) Allows Consul to start managed proxies if Consul is running as root (EUID of the process is zero). We recommend running Consul as a non-root user. By default, this is false to protect inadvertently running external processes as root.
* <a name="connect_proxy_defaults"></a><a href="#connect_proxy_defaults">`proxy_defaults`</a> This object configures the default proxy settings for [service definitions with managed proxies](/docs/agent/services.html). It accepts the fields `exec_mode`, `daemon_command`, and `config`. These are used as default values for the respective fields in the service definition.
* <a name="connect_proxy_defaults"></a><a href="#connect_proxy_defaults">`proxy_defaults`</a> [**Deprecated**](/docs/connect/proxies/managed-deprecated.html) This object configures the default proxy settings for service definitions with [managed proxies](/docs/connect/proxies/managed-deprecated.html) (now deprecated). It accepts the fields `exec_mode`, `daemon_command`, and `config`. These are used as default values for the respective fields in the service definition.
* <a name="replication_token"></a><a href="#replication_token">`replication_token`</a> When provided, this will enable Connect replication using this token to retrieve and replicate the Intentions to the non-authoritative local datacenter.
@ -1134,14 +1145,31 @@ Consul will not enable TLS for the HTTP API unless the `https` port has been ass
the bind ports for the following keys:
* <a name="dns_port"></a><a href="#dns_port">`dns`</a> - The DNS server, -1 to disable. Default 8600.
* <a name="http_port"></a><a href="#http_port">`http`</a> - The HTTP API, -1 to disable. Default 8500.
* <a name="https_port"></a><a href="#https_port">`https`</a> - The HTTPS API, -1 to disable. Default -1 (disabled).
* <a name="https_port"></a><a href="#https_port">`https`</a> - The HTTPS
API, -1 to disable. Default -1 (disabled). **We recommend using `8501`** for
`https` by convention as some tooling will work automatically with this.
* <a name="grpc_port"></a><a href="#grpc_port">`grpc`</a> - The gRPC API, -1
to disable. Default -1 (disabled). **We recommend using `8502`** for
`grpc` by convention as some tooling will work automatically with this.
This is set to `8502` by default when the agent runs in `-dev` mode.
Currently gRPC is only used to expose Envoy xDS API to Envoy proxies.
* <a name="serf_lan_port"></a><a href="#serf_lan_port">`serf_lan`</a> - The Serf LAN port. Default 8301.
* <a name="serf_wan_port"></a><a href="#serf_wan_port">`serf_wan`</a> - The Serf WAN port. Default 8302. Set to -1
to disable. **Note**: this will disable WAN federation which is not recommended. Various catalog and WAN related
endpoints will return errors or empty results.
* <a name="server_rpc_port"></a><a href="#server_rpc_port">`server`</a> - Server RPC address. Default 8300.
* <a name="proxy_min_port"></a><a href="#proxy_min_port">`proxy_min_port`</a> - Minimum port number to use for automatically assigned [managed Connect proxies](/docs/connect/proxies.html). If Connect is disabled, managed proxies are unused, or ports are always specified, then this value is unused. Defaults to 20000.
* <a name="proxy_max_port"></a><a href="#proxy_max_port">`proxy_max_port`</a> - Maximum port number to use for automatically assigned [managed Connect proxies](/docs/connect/proxies.html). See [`proxy_min_port`](#proxy_min_port) for more information. Defaults to 20255.
* <a name="proxy_min_port"></a><a href="#proxy_min_port">`proxy_min_port`</a> [**Deprecated**](/docs/connect/proxies/managed-deprecated.html) - Minimum port number to use for automatically assigned [managed proxies](/docs/connect/proxies/managed-deprecated.html). Default 20000.
* <a name="proxy_max_port"></a><a href="#proxy_max_port">`proxy_max_port`</a> [**Deprecated**](/docs/connect/proxies/managed-deprecated.html) - Maximum port number to use for automatically assigned [managed proxies](/docs/connect/proxies/managed-deprecated.html). Default 20255.
* <a name="sidecar_min_port"></a><a
href="#sidecar_min_port">`sidecar_min_port`</a> - Inclusive minimum port
number to use for automatically assigned [sidecar service
registrations](/docs/connect/proxies/sidecar-service.html). Default 21000.
Set to `0` to disable automatic port assignment.
* <a name="sidecar_max_port"></a><a
href="#sidecar_max_port">`sidecar_max_port`</a> - Inclusive maximum port
number to use for automatically assigned [sidecar service
registrations](/docs/connect/proxies/sidecar-service.html). Default 21255.
Set to `0` to disable automatic port assignment.
* <a name="protocol"></a><a href="#protocol">`protocol`</a> Equivalent to the
[`-protocol` command-line flag](#_protocol).

View File

@ -45,7 +45,7 @@ example shows all possible fields, but note that only a few are required.
}
],
"kind": "connect-proxy",
"proxy_destination": "redis",
"proxy_destination": "redis", // Deprecated
"proxy": {
"destination_service_name": "redis",
"destination_service_id": "redis1",
@ -56,7 +56,8 @@ example shows all possible fields, but note that only a few are required.
}
"connect": {
"native": false,
"proxy": {
"sidecar_service": {}
"proxy": { // Deprecated
"command": [],
"config": {}
}
@ -93,39 +94,65 @@ meta object in node definition.
All those meta data can be retrieved individually per instance of the service
and all the instances of a given service have their own copy of it.
The `kind` field is used to optionally identify the service as an [unmanaged
Connect proxy](/docs/connect/proxies.html#unmanaged-proxies) instance with the
value `connect-proxy`. For typical non-proxy instances the `kind` field must be
omitted. The `proxy` field is also required for unmanaged proxy registrations
and is only valid if `kind` is `connect-proxy`. The only required `proxy` field
is `destination_service_name`. From version 1.2.0 to 1.3.0 this was specified
using `proxy_destination` which still works but is now deprecated. See the
[unmanaged proxy
configuration](/docs/connect/proxies.html#complete-configuration-example)
documentation for full details.
Services may also contain a `token` field to provide an ACL token. This token is
used for any interaction with the catalog for the service, including
[anti-entropy syncs](/docs/internals/anti-entropy.html) and deregistration.
The `enable_tag_override` can optionally be specified to disable the
anti-entropy feature for this service. If `enable_tag_override` is set to
`TRUE` then external agents can update this service in the
[catalog](/api/catalog.html) and modify the tags. Subsequent
local sync operations by this agent will ignore the updated tags. For
example, if an external agent modified both the tags and the port for
this service and `enable_tag_override` was set to `TRUE` then after the next
sync cycle the service's port would revert to the original value but the
tags would maintain the updated value. As a counter example: If an
external agent modified both the tags and port for this service and
`enable_tag_override` was set to `FALSE` then after the next sync cycle the
service's port *and* the tags would revert to the original value and all
modifications would be lost.
It's important to note that this applies only to the locally registered
service. If you have multiple nodes all registering the same service
their `enable_tag_override` configuration and all other service
configuration items are independent of one another. Updating the tags
for the service registered on one node is independent of the same
service (by name) registered on another node. If `enable_tag_override` is
not specified the default value is false. See [anti-entropy
syncs](/docs/internals/anti-entropy.html) for more info.
For Consul 0.9.3 and earlier you need to use `enableTagOverride`. Consul 1.0
supports both `enable_tag_override` and `enableTagOverride` but the latter is
deprecated and has been removed as of Consul 1.1.
### Connect
The `kind` field is used to optionally identify the service as a [Connect
proxy](/docs/connect/proxies.html) instance with the value `connect-proxy`. For
typical non-proxy instances the `kind` field must be omitted. The `proxy` field
is also required for Connect proxy registrations and is only valid if `kind` is
`connect-proxy`. The only required `proxy` field is `destination_service_name`.
For more detail please see [complete proxy configuration
example](/docs/connect/proxies.html#complete-configuration-example)
-> **Deprecation Notice:** From version 1.2.0 to 1.3.0, proxy destination was
specified using `proxy_destination` at the top level. This will continue to work
until at least 1.5.0 but it's highly recommended to switch to using
`proxy.destination_service_name`.
The `connect` field can be specified to configure
[Connect](/docs/connect/index.html) for a service. This field is available in
Consul 1.2 and later. The `native` value can be set to true to advertise the
service as [Connect-native](/docs/connect/native.html). If the `proxy` field is
set (even to an empty object), then this will enable a [managed
proxy](/docs/connect/proxies.html) for the service. The fields within `proxy`
are used to configure the proxy and are specified in the [proxy
docs](/docs/connect/proxies.html). If `native` is true, it is an error to also
specifiy a managed proxy instance.
Consul 1.2.0 and later. The `native` value can be set to true to advertise the
service as [Connect-native](/docs/connect/native.html). The `sidecar_service`
field is an optional nested service definition its behavior and defaults are
described in [Sidecar Service
Registration](/docs/connect/proxies/sidecar-service.html). If `native` is true,
it is an error to also specify a sidecar service registration.
The `weights` field is an optional field to specify the weight of a service in
DNS SRV responses. If this field is not specified, its default value is:
`"weights": {"passing": 1, "warning": 1}`.
When a service is `critical`, it is excluded from DNS responses.
Services with warning checks are in included in responses by default,
but excluded if the optional param `only_passing = true` is present in
agent DNS configuration or `?passing` is used via the API.
When DNS SRV requests are made, the response will include the weights
specified given the state of the service.
This allows some instances to be given higher weight if they have more capacity,
and optionally allows reducing load on services with checks in `warning` status
by giving passing instances a higher weight.
-> **Deprecation Notice:** From version 1.2.0 to 1.3.0 during beta, Connect
supported "Managed" proxies which are specified with the `connect.proxy` field.
[Managed Proxies are deprecated](/docs/connect/proxies/managed-deprecated.html)
and the `connect.proxy` field will be removed in a future major release.
### Checks
@ -136,17 +163,29 @@ the DNS interface as well. If a service is failing its health check or a
node has any failing system-level check, the DNS interface will omit that
node from any service query.
The check must be of the script, HTTP, TCP or TTL type. If it is a script type,
`args` and `interval` must be provided. If it is a HTTP type, `http` and
`interval` must be provided. If it is a TCP type, `tcp` and `interval` must be
provided. If it is a TTL type, then only `ttl` must be provided. The check name
is automatically generated as `service:<service-id>`. If there are multiple
service checks registered, the ID will be generated as
`service:<service-id>:<num>` where `<num>` is an incrementing number starting
from `1`.
There are several check types that have differing required options as
[documented here](/docs/agent/checks.html). The check name is automatically
generated as `service:<service-id>`. If there are multiple service checks
registered, the ID will be generated as `service:<service-id>:<num>` where
`<num>` is an incrementing number starting from `1`.
-> **Note:** There is more information about [checks here](/docs/agent/checks.html).
### DNS SRV Weights
The `weights` field is an optional field to specify the weight of a service in
DNS SRV responses. If this field is not specified, its default value is:
`"weights": {"passing": 1, "warning": 1}`. When a service is `critical`, it is
excluded from DNS responses. Services with warning checks are included in
responses by default, but excluded if the optional param `only_passing = true`
is present in agent DNS configuration or `?passing` is used via the API.
When DNS SRV requests are made, the response will include the weights specified
given the state of the service. This allows some instances to be given higher
weight if they have more capacity, and optionally allows reducing load on
services with checks in `warning` status by giving passing instances a higher
weight.
### Enable Tag Override and Anti-Entropy
Services may also contain a `token` field to provide an ACL token. This token is
@ -236,7 +275,12 @@ delimit service tags.
## Service Definition Parameter Case
For historical reasons Consul's API uses `CamelCased` parameter names in
responses, however it's configuration file syntax borrowed from HCL uses
`snake_case`. For this reason the registration APIs accept both cases for
service definition parameters although APIs will return the listings using
`CamelCase`.
responses, however it's configuration file uses `snake_case` for both HCL and
JSON representations. For this reason the registration _HTTP APIs_ accept both
name styles for service definition parameters although APIs will return the
listings using `CamelCase`.
Note though that **all config file formats require
`snake_case` fields**. We always document service definition examples using
`snake_case` and JSON since this format works in both config files and API
calls.

View File

@ -16,9 +16,11 @@
* `-http-addr=<addr>` - Address of the Consul agent with the port. This can be
an IP address or DNS address, but it must include the port. This can also be
specified via the `CONSUL_HTTP_ADDR` environment variable. In Consul 0.8 and
later, the default value is http://127.0.0.1:8500, and https can optionally
be used instead. The scheme can also be set to HTTPS by setting the
environment variable `CONSUL_HTTP_SSL=true`.
later, the default value is http://127.0.0.1:8500, and https can optionally be
used instead. The scheme can also be set to HTTPS by setting the environment
variable `CONSUL_HTTP_SSL=true`. This may be a unix domain socket using
`unix:///path/to/socket` if the [agent is configured to
listen](/docs/agent/options.html#addresses) that way.
* `-tls-server-name=<value>` - The server name to use as the SNI host when
connecting via TLS. This can also be specified via the `CONSUL_TLS_SERVER_NAME`

View File

@ -0,0 +1,178 @@
---
layout: "docs"
page_title: "Commands: Connect Proxy"
sidebar_current: "docs-commands-connect-envoy"
description: >
The connect proxy subcommand is used to run the built-in mTLS proxy for Connect.
---
# Consul Connect Envoy
Command: `consul connect envoy`
The connect Envoy command is used to generate a bootstrap configuration for
[Envoy proxy](https://envoyproxy.io) for use with [Consul
Connect](/docs/connect/).
The default behaviour is to generate the necessary bootstrap configuration for
Envoy based on the environment variables and options provided and by taking to
the local Consul agent. It `exec`s an external Envoy binary with that
configuration leaving the Envoy process running in the foreground. An error is
returned on operating systems other than linux or macOS since Envoy does not
build for other platforms currently.
If the `-bootstrap` option is specified, the bootstrap config is generated in
the same way and then printed to stdout. This allows it to be redirected to a
file and used with `envoy -c bootstrap.json`. This works on all operating
systems allowing configuration to be generated on a host that Envoy doesn't
build on but then used in a virtualized environment that can run Envoy.
## Usage
Usage: `consul connect envoy [options] [-- pass-through options]`
#### API Options
The standard API options are used to connect to the local agent to discover the
proxy configuration needed.
- `-grpc-addr=<addr>` - Address of the Consul agent with `grpc` port. This can
be an IP address or DNS address, but it must include the port. This can also
be specified via the CONSUL_GRPC_ADDR environment variable. In Consul 1.3 and
later, the default value is http://127.0.0.1:8502, and https can optionally
be used instead. The scheme can also be set to HTTPS by setting the
environment variable CONSUL_HTTP_SSL=true. This may be a unix domain socket
using `unix:///path/to/socket` if the [agent is configured to
listen](/docs/agent/options.html#addresses) that way.
-> **Note:** gRPC uses the same TLS
settings as the HTTPS API. If HTTPS is enabled then gRPC will require HTTPS
as well.
<%= partial "docs/commands/http_api_options_client" %>
#### Envoy Options
* `-sidecar-for` - The _ID_ (not name if they differ) of the service instance
this proxy will represent. The target service doesn't need to exist on the
local agent yet but a [sidecar proxy
registration](/docs/connect/proxies.html#sidecar-proxy-fields) with
`proxy.destination_service_id` equal to the passed value must be present. If
multiple proxy registrations targeting the same local service instance are
present the command will error and `-proxy-id` should be used instead.
* `-proxy-id` - The [proxy
service](/docs/connect/proxies.html#proxy-service-definitions) ID on the
local agent. This must already be present on the local agent.
-> **Note:** If ACLs are enabled, a token granting `service:write` for the
_target_ service (configured in `proxy.destination_service_name`) must be
passed using the `-token` option or `CONSUL_HTTP_TOKEN` environment variable.
This token authorizes the proxy to obtain TLS certificates representing the
target service.
* `-envoy-binary` - The full path to a specific Envoy binary to exec. By
default the current `$PATH` is searched for `envoy`.
* `-admin-bind` - The `host:port` to bind Envoy's admin HTTP API. Default is
`localhost:19000`. Envoy requires that this be enabled. The host part must be
resolvable DNS name or IP address.
* `-bootstrap` - If present, the command will simply output the generated
bootstrap config to stdout in JSON protobuf form. This can be directed to a
file and used to start Envoy with `envoy -c bootstrap.json`.
~> **Security Note:** If ACLs are enabled the bootstrap JSON will contain the
ACL token from `-token` or the environment and so should be handled as a secret.
This token grants the identity of any service it has `service:write` permission
for and so can be used to access any upstream service that that service is
allowed to access by [Connect intentions](/docs/connect/intentions.html).
* `-- [pass-through options]` - Any options given after a double dash are passed
directly through to the `envoy` invocation. See [Envoy's
documentation](https://www.envoyproxy.io/docs) for more details. The command
always specifies `--config-file` and `--v2-config-only` and by default passes
`--disable-hot-restart` see [hot restart](#hot-restart).
## Examples
Assume a local service instance is registratered on the local agent with a
sidecar proxy (using the [sidecar service
registration](/docs/connect/proxies/sidecar-service.html) helper) as below.
```hcl
service {
name = "web"
port = 8080
connect { sidecar_service {} }
}
```
The sidecar Envoy process can be started with.
```text
$ consul connect envoy -sidecar-for web
```
This example assumes that the correct [environment variables](#api-options) are
used to set the local agent connection information and ACL token, or that the
agent is using all-default configuration.
To pass additional arguments directly to Envoy, for example output logging
level, you can use:
```text
$ consul connect envoy -sidecar-for web -- -l debug
```
To run multiple different proxy instances on the same host, you will
need to use `-admin-bind` on all but one to ensure they don't attempt to bind to
the same port as in the following example.
```text
$ consul connect envoy -sidecar-for db -admin-bind localhost:19001
```
## Exec Security Details
The command needs to pass the bootstrap config through to Envoy. Envoy currently
only supports passing this as a file path or passing a whole string on the
command line with `--config-yaml`. Since the bootstrap needs to contain the ACL
token to authorize the proxy, this secret needs careful handling.
Passing a secret via command option is unacceptable as on many unix systems
these are readable to any user on the host for example via `/proc` or via a
setuid process like `ps`.
Creating a temporary file is more secure in that it can only be read by the
current user but risks leaving secret material on disk for an unbounded length
of time and in a location that is opaque to the operator.
To work around these issues, the command currently creates a temporary file and
immediately unlinks it so it can't be read by any other process that doesn't
already have the file descriptor. It then writes the bootstrap JSON, and unsets
the CLOEXEC bit on the file handle so that it remains available to the Envoy
process after exec. Finally it `exec`s Envoy with `--config-file /dev/fd/X`
where `X` is the the file descriptor number of the temp file.
This ensures that Envoy can read the file without any other normal user process
being able to (assuming they don't have privileged access to /proc). Once the
Envoy process stops, there is no longer any reference to the file to clean up.
## Envoy Hot Restart
Envoy supports hot restart which requires simple external coordination. By
default, this command will add `--disable-hot-restart` when it runs Envoy.
The reason for this default behavior is to make it easy to test and run local
demonstrations with multiple Envoy instances outside of cgroups or network
namespaces.
To use hot restart, Envoy needs to be started with either the `--restart-epoch`
option. If this command detects that option in the pass-through flags it will
_not_ add `--disable-hot-restart` allowing hot restart to work normally.
The only difference to note over running Envoy directly is that
`--restart-epoch` must be explicitly set to `0` for the initial launch of the
Envoy instance to avoid disabling hot restart entirely. The official
`hot-restarter.py` always sets this option so should work as recommended.

View File

@ -13,7 +13,7 @@ Command: `consul connect proxy`
The connect proxy command is used to run Consul's built-in mTLS proxy for
use with Connect. This can be used in production to enable a Connect-unaware
application to accept and establish Connect-based connections. This proxy
can also be used in development to establish Connect-based connections.
can also be used in development to connect to Connect-enabled services.
## Usage
@ -27,9 +27,28 @@ Usage: `consul connect proxy [options]`
#### Proxy Options
* `-sidecar-for` - The _ID_ (not name if they differ) of the service instance
this proxy will represent. The target service doesn't need to exist on the
local agent yet but a [sidecar proxy
registration](/docs/connect/proxies.html#sidecar-proxy-fields) with
`proxy.destination_service_id` equal to the passed value must be present. If
multiple proxy registrations targeting the same local service instance are
present the command will error and `-proxy-id` should be used instead.
* `-proxy-id` - The [proxy
service](/docs/connect/proxies.html#proxy-service-definitions) ID on the
local agent. This must already be present on the local agent.
* `-log-level` - Specifies the log level.
* `-pprof-addr` - Enable debugging via pprof. Providing a host:port (or just ':port')
enables profiling HTTP endpoints on that address.
* `-service` - Name of the service this proxy is representing. This service
doesn't need to actually exist in the Consul catalog, but proper ACL
permissions (`service:write`) are required.
permissions (`service:write`) are required. This and the remaining options can
be used to setup a proxy that is not registered already with local config
[useful for development](/docs/connect/dev.html).
* `-upstream` - Upstream service to support connecting to. The format should be
'name:addr', such as 'db:8181'. This will make 'db' available on port 8181.
@ -48,17 +67,12 @@ Usage: `consul connect proxy [options]`
proxy available as Connect-capable service in the catalog. This is only
useful with `-listen`.
* `-register-id` - Optional ID suffix for the service when `-register` is set
to disambiguate the service ID. By default the service ID is "<service>-proxy"
where `<service>` is the `-service` value.
* `-log-level` - Specifies the log level.
* `-pprof-addr` - Enable debugging via pprof. Providing a host:port (or just ':port')
enables profiling HTTP endpoints on that address.
* `-proxy-id` - The proxy's ID on the local agent. This is only useful to
test the managed proxy mode.
* `-register-id` - Optional ID suffix for the service when `-register` is set to
disambiguate the service ID. By default the service ID is "<service>-proxy"
where `<service>` is the `-service` value. In most cases it is now preferable
to use [`consul services register`](/docs/commands/services/register.html) to
register a fully configured proxy instance rather than specify config and
registration via this command.
## Examples

View File

@ -115,7 +115,7 @@ These environment variables and their purpose are described below:
## `CONSUL_HTTP_ADDR`
This is the HTTP API address to the *local* Consul agent
(not the remote server) specified as a URI:
(not the remote server) specified as a URI with optional scheme:
```
CONSUL_HTTP_ADDR=127.0.0.1:8500
@ -127,6 +127,8 @@ or as a Unix socket path:
CONSUL_HTTP_ADDR=unix://var/run/consul_http.sock
```
If the `https://` scheme is used, `CONSUL_HTTP_SSL` is implied to be true.
### `CONSUL_HTTP_TOKEN`
This is the API access token required when access control lists (ACLs)
@ -155,8 +157,9 @@ CONSUL_HTTP_SSL=true
### `CONSUL_HTTP_SSL_VERIFY`
This is a boolean value (default true) to specify SSL certificate verification; setting this value to `false` is not recommended for production use. Example
for development purposes:
This is a boolean value (default true) to specify SSL certificate verification;
setting this value to `false` is not recommended for production use. Example for
development purposes:
```
CONSUL_HTTP_SSL_VERIFY=false
@ -201,3 +204,26 @@ The server name to use as the SNI host when connecting via TLS.
```
CONSUL_TLS_SERVER_NAME=consulserver.domain
```
### `CONSUL_GRPC_ADDR`
Like [`CONSUL_HTTP_ADDR`](#consul_http_addr) but configures the address the
local agent is listening for gRPC requests. Currently gRPC is only used for
integrating [Envoy proxy](/docs/connect/proxies/envoy.html) and must be [enabled
explicitly](/docs/agent/options.html#grpc_port) in agent configuration.
```
CONSUL_GRPC_ADDR=127.0.0.1:8502
```
or as a Unix socket path:
```
CONSUL_GRPC_ADDR=unix://var/run/consul_grpc.sock
```
If the agent is [configured with TLS
certificates](/docs/agent/encryption.html#rpc-encryption-with-tls), then the
gRPC listener will require TLS and present the same certificate as the https
listener. As with `CONSUL_HTTP_ADDR`, if TLS is enabled either the `https://`
scheme should be used, or `CONSUL_HTTP_SSL` set.

View File

@ -69,6 +69,23 @@ local caching, background updating, and support blocking queries. As a result,
most API calls operate on purely local in-memory data and can respond
in microseconds.
## Getting Started With Connect
There are several ways to try Connect in different environments.
* The [Connect introduction](/intro/getting-started/connect.html) in the
Getting Started guide provides a simple walk through of getting two services
to communicate via Connect using only Consul directly on your local machine.
* The [Envoy guide](/docs/guides/connect-envoy.html) walks through getting
started with Envoy as a proxy, and uses Docker to run components locally
without installing anything else.
* The [Kubernetes documentation](/docs/platform/k8s/run.html) shows how to get
from an empty Kubernetes cluster to having Consul installed and Envoy
configured to proxy application traffic automatically using the official helm
chart.
## Agent Caching and Performance
To enable microsecond-speed responses on

View File

@ -17,181 +17,131 @@ connections on a loopback address. This requires all external connections
to be established via the Connect protocol to provide authentication and
authorization.
Consul supports both _managed_ and _unmanaged_ proxies. A managed proxy
is started, configured, and stopped by Consul. An unmanaged proxy is the
responsibility of the user, like any other Consul service.
-> **Deprecation Note:** Managed Proxies are deprecated as of Consul 1.3. See
[managed proxy deprecation](/docs/connect/proxies/managed-deprecated.html) for
more information. It's strongly recommended to switch to one of the approaches
listed on this page as soon as possible.
## Managed Proxies
## Proxy Service Definitions
Managed proxies are started, configured, and stopped by Consul. They are
enabled via basic configurations within the
[service definition](/docs/agent/services.html).
This is the easiest way to start a proxy and allows Consul users to begin
using Connect with only a small configuration change.
Connect proxies are registered using regular [service
definitions](/docs/agent/services.html). They can be registered both in config
files or via the API just like any other service.
Managed proxies also offer the best security. Managed proxies are given
a unique proxy-specific ACL token that allows read-only access to Connect
information for the specific service the proxy is representing. This ACL
token is more restrictive than can be currently expressed manually in
an ACL policy.
Additionally, to reduce the amount of boilerplate needed for a sidecar proxy,
application service definitions may define inline [sidecar service
registrations](/docs/connect/proxies/sidecar-service.html) which are an
opinionated shorthand for a separate full proxy registration as described here.
The default managed proxy is a basic proxy built-in to Consul and written
in Go. Having a basic built-in proxy allows Consul to have a sane default
with performance that is good enough for most workloads. In some basic
benchmarks, the service-to-service communication over the built-in proxy
could sustain 5 Gbps with sub-millisecond latency. Therefore,
the performance impact of even the basic built-in proxy is minimal.
To function as a Connect proxy, they must be declared as a proxy type and
provide information about the service they represent.
Consul will be integrating with advanced proxies in the near future to support
more complex configurations and higher performance. The configuration below is
all for the built-in proxy.
To declare a service as a proxy, the service definition must contain
the following fields:
-> **Security note:** 1.) Managed proxies can only be configured
via agent configuration files. They _cannot_ be registered via the HTTP API.
And 2.) Managed proxies are not started at all if Consul is running as root.
Both of these default configurations help prevent arbitrary process
execution or privilege escalation. This behavior can be configured
[per-agent](/docs/agent/options.html#connect_proxy).
* `kind` (string) must be set to `connect-proxy`. This declares that the
service is a proxy type.
### Lifecycle
* `proxy.destination_service_name` (string) must be set to the service that
this proxy is representing. Note that this replaces `proxy_destination` in
versions 1.2.0 to 1.3.0.
The Consul agent starts managed proxies on demand and supervises them,
restarting them if they crash. The lifecycle of the proxy process is decoupled
from the agent so if the agent crashes or is restarted for an upgrade, the
managed proxy instances will _not_ be stopped.
* `port` must be set so that other Connect services can discover the exact
address for connections. `address` is optional if the service is being
registered against an agent, since it'll inherit the node address.
Note that this behaviour while desirable in production might leave proxy
processes running indefinitely if you manually stop the agent and clear it's
data dir during testing.
To terminate a managed proxy cleanly you need to deregister the service that
requested it. If the agent is already stopped and will not be restarted again,
you may choose to locate the proxy processes and kill them manually.
While in `-dev` mode, unless a `-data-dir` is explicitly set, managed proxies
switch to being killed when the agent exits since it can't store state in order
to re-adopt them on restart.
### Minimal Configuration
Managed proxies are configured within a
[service definition](/docs/agent/services.html). The simplest possible
managed proxy configuration is an empty configuration. This enables the
default managed proxy and starts a listener for that service:
Minimal Example:
```json
{
"service": {
"name": "redis",
"port": 6379,
"connect": { "proxy": {} }
}
"name": "redis-proxy",
"kind": "connect-proxy",
"proxy": {
"destination_service_name": "redis"
},
"port": 8181
}
```
The listener is started on random port within the configured Connect
port range. It can be discovered using the
[DNS interface](/docs/agent/dns.html#connect-capable-service-lookups)
or
[Catalog API](#).
In most cases, service-to-service communication is established by
a proxy configured with upstreams (described below), which handle the
discovery transparently.
With this service registered, any Connect clients searching for a
Connect-capable endpoint for "redis" will find this proxy.
### Upstream Configuration
### Sidecar Proxy Fields
To transparently discover and establish Connect-based connections to
dependencies, they must be configured with a static port on the managed
proxy configuration:
Most Connect proxies are deployed as "sidecars" which means they are co-located
with a single service instance which they represent and proxy all inbound
traffic to. In this case the following fields must may also be set:
* `proxy.destination_service_id` (string) is set to the _id_ (and not the
_name_ if they are different) of the specific service instance that is being
proxied. The proxied service is assumed to be registered on the same agent
although it's not strictly validated to allow for un-coordinated
registrations.
* `proxy.local_service_port` (string) must specify the port the proxy should use
to connect to the _local_ service instance.
* `proxy.local_service_address` (string) can be set to override the IP or
hostname the proxy should use to connect to the _local_ service. Defaults to
`127.0.0.1`.
### Complete Configuration Example
The following is a complete example showing all the options available when
registering a proxy instance.
```json
{
"service": {
"name": "web",
"port": 8080,
"connect": {
"proxy": {
"upstreams": [{
"destination_name": "redis",
"local_bind_port": 1234
}]
}
}
}
"name": "redis-proxy",
"kind": "connect-proxy",
"proxy": {
"destination_service_name": "redis",
"destination_service_id": "redis1",
"local_service_address": "127.0.0.1",
"local_service_port": 9090,
"config": {},
"upstreams": []
},
"port": 8181
}
```
In the example above,
"redis" is configured as an upstream with static port 1234 for service "web".
When a TCP connection is established on port 1234, the proxy
will find Connect-compatible "redis" services via Consul service discovery
and establish a TLS connection identifying as "web".
-> **Deprecation Notice:** From version 1.2.0 to 1.3.0, proxy destination was
specified using `proxy_destination` at the top level. This will continue to work
until at least 1.5.0 but it's highly recommended to switch to using
`proxy.destination_service_name`.
~> **Security:** Any application that can communicate to the configured
static port will be able to masquerade as the source service ("web" in the
example above). You must either trust any loopback access on that port or
use namespacing techniques provided by your operating system.
#### Proxy Parameters
-> **Deprecation Note:** versions 1.2.0 to 1.3.0 required specifying `upstreams`
as part of the opaque `config` that is passed to the proxy. However, since
1.3.0, the `upstreams` configuration is now specified directily under the
`proxy` key. Old service definitions using the nested config will continue to
work and have the values copied into the new location. This allows the upstreams
to be registered centrally rather than being part of the local-only config
passed to the proxy instance.
- `destination_service_name` `string: <required>` - Specifies the _name_ of the
service this instance is proxying. Both side-car and centralized
load-balancing proxies must specify this. It is used during service
discovery to find the correct proxy instances to route to for a given service
name.
For full details of the additional configurable options available when using the
built-in proxy see the [built-in proxy configuration
reference](/docs/connect/configuration.html#built-in-proxy-options).
- `destination_service_id` `string: <optional>` - Specifies the _ID_ of a single
specific service instance that this proxy is representing. This is only valid
for side-car style proxies that run on the same node. It is assumed that the
service instance is registered via the same Consul agent so the ID is unique
and has no node qualifier. This is useful to show in tooling which proxy
instance is a side-car for which application instance and will enable
fine-grained analysis of the metrics coming from the proxy.
### Prepared Query Upstreams
- `local_service_address` `string: <optional>` - Specifies the address a side-car
proxy should attempt to connect to the local application instance on.
Defaults to 127.0.0.1.
The upstream destination may also be a
[prepared query](/api/query.html).
This allows complex service discovery behavior such as connecting to
the nearest neighbor or filtering by tags.
- `local_service_port` `int: <optional>` - Specifies the port a side-car
proxy should attempt to connect to the local application instance on.
Defaults to the port advertised by the service instance identified by
`destination_service_id` if it exists otherwise it may be empty in responses.
For example, given a prepared query named "nearest-redis" that is
configured to route to the nearest Redis instance, an upstream can be
configured to route to this query. In the example below, any TCP connection
to port 1234 will attempt a Connect-based connection to the nearest Redis
service.
- `config` `object: <optional>` - Specifies opaque config JSON that will be
stored and returned along with the service instance from future API calls.
```json
{
"service": {
"name": "web",
"port": 8080,
"connect": {
"proxy": {
"upstreams": [{
"destination_name": "redis",
"destination_type": "prepared_query",
"local_bind_port": 1234
}]
}
}
}
}
```
-> **Note:** Connect does not currently support cross-datacenter
service communication. Therefore, prepared queries with Connect should
only be used to discover services within a single datacenter. See
[Multi-Datacenter Connect](/docs/connect/index.html#multi-datacenter) for
more information.
For full details of the additional configurable options available when using the
built-in proxy see the [built-in proxy configuration
reference](/docs/connect/configuration.html#built-in-proxy-options).
### Dynamic Upstreams
If an application requires dynamic dependencies that are only available
at runtime, it must currently [natively integrate](/docs/connect/native.html)
with Connect. After natively integrating, the HTTP API or
[DNS interface](/docs/agent/dns.html#connect-capable-service-lookups)
can be used.
- `upstreams` `array<Upstream>: <optional>` - Specifies the upstream services
this proxy should create listeners for. The format is defined in
[Upstream Configuration Reference](#upstream-configuration-reference).
### Upstream Configuration Reference
@ -237,164 +187,18 @@ registrations](/docs/agent/services.html#service-definition-parameter-case).
* `config` `object: <optional>` - Specifies opaque configuration options that
will be provided to the proxy instance for this specific upstream. Can contain
any valid JSON object. This might be used to configure proxy-specific features
like timeouts or retries for the given upstream. See the
[built-in proxy configuration reference](/docs/connect/configuration.html#built-in-proxy-options)
for options available when using the built-in proxy.
like timeouts or retries for the given upstream. See the [built-in proxy
configuration
reference](/docs/connect/configuration.html#built-in-proxy-options) for
options available when using the built-in proxy. If using Envoy as a proxy,
see [Envoy configuration
reference](/docs/connect/configuration.html#envoy-options)
### Custom Managed Proxy
[Custom proxies](/docs/connect/proxies/integrate.html) can also be
configured to run as a managed proxy. To configure custom proxies, specify
an alternate command to execute for the proxy:
### Dynamic Upstreams
```json
{
"service": {
"name": "web",
"port": 8080,
"connect": {
"proxy": {
"exec_mode": "daemon",
"command": ["/usr/bin/my-proxy", "-flag-example"],
"config": {
"foo": "bar"
}
}
}
}
}
```
The `exec_mode` value specifies how the proxy is executed. The only
supported value at this time is "daemon". The command is the binary and
any arguments to execute.
The "daemon" mode expects a proxy to run as a long-running, blocking
process. It should not double-fork into the background. The custom
proxy should retrieve its configuration (such as the port to run on)
via the [custom proxy integration APIs](/docs/connect/proxies/integrate.html).
The default proxy command can be changed at an agent-global level
in the agent configuration. An example in HCL format is shown below.
```
connect {
proxy_defaults {
command = ["/usr/bin/my-proxy"]
}
}
```
With this configuration, all services registered without an explicit
proxy command will use `my-proxy` instead of the default built-in proxy.
The `config` key is an optional opaque JSON object which will be passed through
to the proxy via the proxy configuration endpoint to allow any configuration
options the proxy needs to be specified. See the [built-in proxy
configuration reference](/docs/connect/configuration.html#built-in-proxy-options)
for details of config options that can be passed when using the built-in proxy.
### Managed Proxy Logs
Managed proxies have both stdout and stderr captured in log files in the agent's
`data_dir`. They can be found in
`<data_dir>/proxy/logs/<proxy_service_id>-std{err,out}.log`.
The built-in proxy will inherit it's log level from the agent so if the agent is
configured with `log_level = DEBUG`, a proxy it starts will also output `DEBUG`
level logs showing service discovery, certificate and authorization information.
~> **Note:** In `-dev` mode there is no `data_dir` unless one is explicitly
configured so logging is disabled. You can access logs by providing the
[`-data-dir`](/docs/agent/options.html#_data_dir) CLI option. If a data dir is
configured, this will also cause proxy processes to stay running when the agent
terminates as described in [Lifecycle](#lifecycle).
## Unmanaged Proxies
Unmanaged proxies are regular Consul services that are registered as a
proxy type and declare the service they represent. The proxy process must
be started, configured, and stopped manually by some external process such
as an operator or scheduler.
To declare a service as a proxy, the service definition must contain
the following fields:
* `kind` (string) must be set to `connect-proxy`. This declares that the
service is a proxy type.
* `proxy.destination_service_name` (string) must be set to the service that this
proxy is representing. Note that this replaces `proxy_destination` in
versions 1.2.0 to 1.3.0.
* `port` must be set so that other Connect services can discover the exact
address for connections. `address` is optional if the service is being
registered against an agent, since it'll inherit the node address.
Minimal Example:
```json
{
"name": "redis-proxy",
"kind": "connect-proxy",
"proxy": {
"destination_service_name": "redis"
},
"port": 8181
}
```
With this service registered, any Connect proxies searching for a
Connect-capable endpoint for "redis" will find this proxy.
### Complete Configuration Example
The following is a complete example showing all the options available when
registering an unmanaged proxy instance.
```json
{
"name": "redis-proxy",
"kind": "connect-proxy",
"proxy": {
"destination_service_name": "redis",
"destination_service_id": "redis1",
"local_service_address": "127.0.0.1",
"local_service_port": 9090,
"config": {},
"upstreams": []
},
"port": 8181
}
```
#### Proxy Parameters
- `destination_service_name` `string: <required>` - Specifies the _name_ of the
service this instance is proxying. Both side-car and centralized
load-balancing proxies must specify this. It is used during service
discovery to find the correct proxy instances to route to for a given service
name.
- `destination_service_id` `string: <optional>` - Specifies the _ID_ of a single
specific service instance that this proxy is representing. This is only valid
for side-car style proxies that run on the same node. It is assumed that the
service instance is registered via the same Consul agent so the ID is unique
and has no node qualifier. This is useful to show in tooling which proxy
instance is a side-car for which application instance and will enable
fine-grained analysis of the metrics coming from the proxy.
- `local_service_address` `string: <optional>` - Specifies the address a side-car
proxy should attempt to connect to the local application instance on.
Defaults to 127.0.0.1.
- `local_service_port` `int: <optional>` - Specifies the port a side-car
proxy should attempt to connect to the local application instance on.
Defaults to the port advertised by the service instance identified by
`destination_service_id` if it exists otherwise it may be empty in responses.
- `config` `object: <optional>` - Specifies opaque config JSON that will be
stored and returned along with the service instance from future API calls.
- `upstreams` `array<Upstream>: <optional>` - Specifies the upstream services
this proxy should create listeners for. The format is defined in
[Upstream Configuration Reference](#upstream-configuration-reference).
If an application requires dynamic dependencies that are only available
at runtime, it must currently [natively integrate](/docs/connect/native.html)
with Connect. After natively integrating, the HTTP API or
[DNS interface](/docs/agent/dns.html#connect-capable-service-lookups)
can be used.

View File

@ -0,0 +1,270 @@
---
layout: "docs"
page_title: "Connect - Envoy Integration"
sidebar_current: "docs-connect-proxies-envoy"
description: |-
Consul Connect has first-class support for configuring Envoy proxy.
---
# Envoy Integration
Consul Connect has first class support for using
[Envoy](https://www.envoyproxy.io) as a proxy. Consul configures Envoy by
optionally exposing a gRPC service on the local agent that serves [Envoy's xDS
configuration
API](https://github.com/envoyproxy/data-plane-api/blob/master/XDS_PROTOCOL.md).
Currently Consul only supports TCP proxying between services, however HTTP and
gRPC features are planned for the near future along with first class ways to
configure them in Consul.
As an interim solution, [custom Envoy configuration](#custom-configuration) can
be specified in [proxy service definition](/docs/connect/proxies.html) allowing
more powerful features of Envoy to be used.
## Supported Versions
Consul's Envoy support was added in version 1.3.0. It has been tested against
Envoy 1.7.1 and 1.8.0.
## Getting Started
To get started with Envoy and see a working example you can follow the [Using
Envoy with Connect](/docs/guides/connect-envoy.html) guide.
## Limitations
The following list limitations of the Envoy integration as released in 1.3.0.
All of these are planned to be lifted in the near future.
* Default Envoy configuration only supports Layer 4 (TCP) proxying. More
[advanced listener configuration](#advanced-listener-configuration) is
possible but experimental and requires deep Envoy knowledge. First class
workflows for configuring Layer 7 features across the cluster are planned for
the near future.
* There is currently no way to override the configuration of upstream clusters
which makes it impossible to configure Envoy features like circuit breakers,
load balancing policy, custom protocol settings etc. This will be fixed in a
near-future release first with an "escape hatch" similar to the one for
listeners below, then later with first-class support.
* The configuration delivered to Envoy is suitable for a sidecar proxy
currently. Later we plan to support more flexibility to be able to configure
Envoy as an edge router or gateway and similar.
* There is currently no way to disable the public listener and have a "client
only" sidecar for services that don't expose Connect-enabled service but want
to consume others. This will be fixed in a near-future release.
* Once authorized, a persistent TCP connection will not be closed if the
intentions change to deny access. This is currently a limitation of how TCP
proxy and network authz filter work in Envoy. All new connections will be
denied though and destination services can limit exposure by closing inbound
connections periodically or by a rolling restart of the destination service
as an emergency measure.
## Bootstrap Configuration
Envoy requires an initial bootstrap configuration that directs it to the local
agent for further configuration discovery.
To assist in generating this, Consul 1.3.0 adds a [`consul connect envoy`
command](/docs/commands/connect/envoy.html). The command can either output the
bootstrap configuration directly or can generate it and then `exec` the Envoy
binary as a convenience wrapper.
Some Envoy configuration options like metrics and tracing sinks can only be
specified via the bootstrap config currently and so a custom bootstrap must be
used. In order to work with Connect it's necessary to start with the following
basic template and add additional configuration as needed.
```yaml
admin:
# access_log_path and address are required by Envoy, Consul doesn't care what
# they are set to though and never accesses the admin API.
node:
# cluter is required by Envoy but Consul doesn't use it
cluster: "<cluster_name"
# id must be the ID (not name if they differ) of the proxy service
# registration in Consul
id: "<proxy_service_id>"
static_resources:
clusters:
# local_agent is the "cluster" used to make further discovery requests for
# config and should point to the gRPC port of the local Consul agent instance.
- name: local_agent
connect_timeout: 1s
type: STATIC
# tls_context is needed if and only if Consul agent TLS is configured
tls_context:
common_tls_context:
validation_context:
trusted_ca:
filename: "<path to CA cert file Consul is using>"
http2_protocol_options: {}
hosts:
- socket_address:
address: "<agent's local IP address, usually 127.0.0.1>"
port_value: "<agent's grpc port, default 8502>"
dynamic_resources:
lds_config:
ads: {}
cds_config:
ads: {}
ads_config:
api_type: GRPC
grpc_services:
initial_metadata:
- key: "x-consul-token"
token: "<Consul ACL token with service:write on the target service>"
envoy_grpc:
cluster_name: local_agent
```
This configures a "cluster" pointing to the local Consul agent and sets that as
the target for discovering all types of dynamic resources.
~> **Security Note**: The bootstrap configuration must contain the Consul ACL
token authorizing the proxy to identify as the target service. As such it should
be treated as a secret value and handled with care - an attacker with access to
one is able to obtain Connect TLS certificates for the target service and so
access anything that service is authorized to connect to.
## Advanced Listener Configuration
Consul 1.3.0 includes initial Envoy support which includes automatic Layer 4
(TCP) proxying over mTLS, and authorization. Near future versions of Consul will
bring Layer 7 features like HTTP-path-based routing, retries, tracing and more.
-> **Advanced Topic!** This section covers an optional way of taking almost
complete control of Envoy's listener configuration which is provided as a way to
experiment with advanced integrations ahead of full layer 7 feature support.
While we don't plan to remove the ability to do this in the future, it should be
considered experimental and requires in-depth knowledge of Envoy's configuration
format.
For advanced users there is an "escape hatch" available in 1.3.0. The
`proxy.config` map in the [proxy service
definition](/docs/connect/proxies.html#proxy-service-definitions) may contain a
special key called `envoy_public_listener_json`. If this is set, it's value must
be a string containing the serialized proto3 JSON encoding of a complete [envoy
listener
config](https://www.envoyproxy.io/docs/envoy/v1.8.0/api-v2/api/v2/lds.proto).
Each upstream listener may also be customized in the same way by adding a
`envoy_listener_json` key to the `config` map of [the upstream
definition](/docs/connect/proxies.html#upstream-configuration-reference).
The JSON supplied may describe a protobuf `types.Any` message with `@type` set
to `type.googleapis.com/envoy.api.v2.Listener`, or it may be the direct encoding
of the listener with no `@type` field.
Once parsed, it is passed to Envoy in place of the listener config that Consul
would typically configure. The only modifications Consul will make to the config
provided are noted below.
#### Public Listener Configuration
For the `proxy.config.envoy_public_listener_json`, every `FilterChain` added to
the listener will have it's `TlsContext` overwritten with the Connect TLS
certificates. This means there is no way to override Connect TLS settings or the
requirement for all inbound clients to present valid Connect certificates.
Also, every `FilterChain` will have the `envoy.ext_authz` filter prepended to
the filters array to ensure that all incoming connections must be authorized
explicitly by the Consul agent based on their presented client certificate.
To work properly with Consul Connect, the public listener should bind to the
same address in the service definition so it is discoverable. It may also use
the special cluster name `local_app` to forward requests to a single local
instance if the proxy was configured [as a
sidecar](/docs/connect/proxies.html#sidecar-proxy-fields).
#### Example
The following example shows a public listener being configured with an http
connection manager. As specified this behaves exactly like the default TCP proxy
filter however it provides metrics on HTTP request volume and response codes.
If additional config outside of the listener is needed (for example the
top-level `tracing` configuration to send traces to a collecting service), those
currently need to be added to a custom bootstrap. You may generate the default
connect bootstrap with the [`consul connect envoy -bootstrap`
command](/docs/commands/connect/envoy.html) and then add the required additional
resources.
```text
service {
kind = "connect-proxy"
name = "web-http-aware-proxy"
port = 8080
proxy {
destination_service_name web
destination_service_id web
config {
envoy_public_listener_json = <<EOL
{
"@type": "type.googleapis.com/envoy.api.v2.Listener",
"name": "public_listener:0.0.0.0:18080",
"address": {
"socketAddress": {
"address": "0.0.0.0",
"portValue": 8080
}
},
"filterChains": [
{
"filters": [
{
"name": "envoy.http_connection_manager",
"config": {
"stat_prefix": "public_listener",
"route_config": {
"name": "local_route",
"virtual_hosts": [
{
"name": "backend",
"domains": ["*"],
"routes": [
{
"match": {
"prefix": "/"
},
"route": {
"cluster": "local_app"
}
}
]
}
]
},
"http_filters": [
{
"name": "envoy.router",
"config": {}
}
]
}
}
]
}
]
}
EOL
}
}
}
```
#### Upstream Listener Configuration
For the upstream listeners `proxy.upstreams[].config.envoy_listener_json`, no
modification is performed. The `Clusters` served via the xDS API all have the
correct client certificates and verification contexts configured so outbound
traffic should be authenticated.
Each upstream may separately choose to define a custom listener config. If
multiple upstreams define them care must be taken to ensure they all listen on
separate ports.
Currently there is no way to disable a listener for an upstream, or modify how
upstream service discovery clusters are delivered. Richer support for features
like this is planned for the near future.

View File

@ -50,15 +50,9 @@ The certificate served by the remote endpoint can be verified against the
root certificates from the
[`/v1/agent/connect/ca/roots`](/api/agent/connect.html) endpoint.
## Managed Mode Support
## Configuration Discovery
Any custom proxy can also run as a [custom managed proxy](/docs/connect/proxies.html#custom-managed-proxy).
If you want the proxy you're integrating to support this mode, then it should accept
two environment variables that Consul populates on process startup. These
are both required to make the necessary API requests for configuration.
Any proxy can discover proxy configuration registered with a local service
instance using the [agent/service/:service_id
endpoint](/api/agent/service.html#get-service-configuration).
* `CONSUL_PROXY_TOKEN` - The ACL token to use for all requests to proxy-related
API endpoints.
* `CONSUL_PROXY_ID` - The service ID for requesting configuration for the
proxy from [`/v1/agent/connect/proxy/`](/api/agent/connect.html).

View File

@ -0,0 +1,284 @@
---
layout: "docs"
page_title: "Connect - Deprecated Managed Proxies"
sidebar_current: "docs-connect-proxies"
description: |-
Consul 1.2 launched it's Connect Beta period with a feature named Managed
Proxies which are now deprecated. This page describes how they worked and why
they are no longer supported.
---
# Managed Proxy Deprecation
Consul Connect was first released as a beta feature in Consul 1.2.0. The initial
release included a feature called "Managed Proxies". Managed proxies were
Connect proxies where the proxy process was started, configured, and stopped by
Consul. They were enabled via basic configurations within the service
definition.
-> **Consul 1.3.0 deprecates Managed Proxies completely.** It's _strongly_
recommended you do not build anything using Managed proxies and consider using
[sidecar service
registrations](/docs/connect/proxies/sidecar-service.html) instead.
Even though this was a beta feature, managed proxies will continue to work at
least until Consul 1.5 to prevent disruption to demonstration and
proof-of-concept deployments of Consul Connect. Anyone using managed proxies
though should aim to change their workflow as soon as possible to avoid issues
with a later upgrade.
While the current functionality will remain present for a few major releases,
**new and known issues will not be fixed**.
## Deprecation Rationale
Originally managed proxies traded higher implementation complexity for an easier
"getting started" user experience. After seeing how Connect was investigated and
adopted during beta it because obvious that they were not the best trade off.
Managed proxies only really helped in local testing or VM-per-service based
models whereas a lot of users jumped straight to containers where they are not
helpful. They also add only targeted fairly basic supervisor features which
meant most people would want to use something else in production for consistency
with other workloads. So the high implementation cost of building robust process
supervision didn't actually benefit most real use-cases.
Instead of this Connect 1.3.0 introduces the concept of [sidecar service
registrations](/docs/connect/proxies/sidecar-service.html) which
have almost all of the benefits of simpler configuration but without any of the
additional process management complexity. As a result they can be used to
simplify configuration in both container-based and realistic production
supervisor settings.
## Managed Proxy Documentation
As the managed proxy features continue to be supported for now, the rest of this
page will document how they work in the interim.
-> **Deprecation Note:** It's _strongly_ recommended you do not build anything
using Managed proxies and consider using [sidecar service
registrations](/docs/connect/proxies.html/sidecar-service.html) instead.
Managed proxies are given
a unique proxy-specific ACL token that allows read-only access to Connect
information for the specific service the proxy is representing. This ACL
token is more restrictive than can be currently expressed manually in
an ACL policy.
The default managed proxy is a basic proxy built-in to Consul and written
in Go. Having a basic built-in proxy allows Consul to have a sane default
with performance that is good enough for most workloads. In some basic
benchmarks, the service-to-service communication over the built-in proxy
could sustain 5 Gbps with sub-millisecond latency. Therefore,
the performance impact of even the basic built-in proxy is minimal.
Consul will be integrating with advanced proxies in the near future to support
more complex configurations and higher performance. The configuration below is
all for the built-in proxy.
-> **Security Note:** 1.) Managed proxies can only be configured
via agent configuration files. They _cannot_ be registered via the HTTP API.
And 2.) Managed proxies are not started at all if Consul is running as root.
Both of these default configurations help prevent arbitrary process
execution or privilege escalation. This behavior can be configured
[per-agent](/docs/agent/options.html#connect_proxy).
### Lifecycle
The Consul agent starts managed proxies on demand and supervises them,
restarting them if they crash. The lifecycle of the proxy process is decoupled
from the agent so if the agent crashes or is restarted for an upgrade, the
managed proxy instances will _not_ be stopped.
Note that this behaviour while desirable in production might leave proxy
processes running indefinitely if you manually stop the agent and clear it's
data dir during testing.
To terminate a managed proxy cleanly you need to deregister the service that
requested it. If the agent is already stopped and will not be restarted again,
you may choose to locate the proxy processes and kill them manually.
While in `-dev` mode, unless a `-data-dir` is explicitly set, managed proxies
switch to being killed when the agent exits since it can't store state in order
to re-adopt them on restart.
### Minimal Configuration
Managed proxies are configured within a
[service definition](/docs/agent/services.html). The simplest possible
managed proxy configuration is an empty configuration. This enables the
default managed proxy and starts a listener for that service:
```json
{
"service": {
"name": "redis",
"port": 6379,
"connect": { "proxy": {} }
}
}
```
The listener is started on random port within the configured Connect
port range. It can be discovered using the
[DNS interface](/docs/agent/dns.html#connect-capable-service-lookups)
or
[Catalog API](#).
In most cases, service-to-service communication is established by
a proxy configured with upstreams (described below), which handle the
discovery transparently.
### Upstream Configuration
To transparently discover and establish Connect-based connections to
dependencies, they must be configured with a static port on the managed
proxy configuration:
```json
{
"service": {
"name": "web",
"port": 8080,
"connect": {
"proxy": {
"upstreams": [{
"destination_name": "redis",
"local_bind_port": 1234
}]
}
}
}
}
```
In the example above,
"redis" is configured as an upstream with static port 1234 for service "web".
When a TCP connection is established on port 1234, the proxy
will find Connect-compatible "redis" services via Consul service discovery
and establish a TLS connection identifying as "web".
~> **Security:** Any application that can communicate to the configured
static port will be able to masquerade as the source service ("web" in the
example above). You must either trust any loopback access on that port or
use namespacing techniques provided by your operating system.
-> **Deprecation Note:** versions 1.2.0 to 1.3.0 required specifying `upstreams`
as part of the opaque `config` that is passed to the proxy. However, since
1.3.0, the `upstreams` configuration is now specified directily under the
`proxy` key. Old service definitions using the nested config will continue to
work and have the values copied into the new location. This allows the upstreams
to be registered centrally rather than being part of the local-only config
passed to the proxy instance.
For full details of the additional configurable options available when using the
built-in proxy see the [built-in proxy configuration
reference](/docs/connect/configuration.html#built-in-proxy-options).
### Prepared Query Upstreams
The upstream destination may also be a
[prepared query](/api/query.html).
This allows complex service discovery behavior such as connecting to
the nearest neighbor or filtering by tags.
For example, given a prepared query named "nearest-redis" that is
configured to route to the nearest Redis instance, an upstream can be
configured to route to this query. In the example below, any TCP connection
to port 1234 will attempt a Connect-based connection to the nearest Redis
service.
```json
{
"service": {
"name": "web",
"port": 8080,
"connect": {
"proxy": {
"upstreams": [{
"destination_name": "redis",
"destination_type": "prepared_query",
"local_bind_port": 1234
}]
}
}
}
}
```
-> **Note:** Connect does not currently support cross-datacenter
service communication. Therefore, prepared queries with Connect should
only be used to discover services within a single datacenter. See
[Multi-Datacenter Connect](/docs/connect/index.html#multi-datacenter) for
more information.
For full details of the additional configurable options available when using the
built-in proxy see the [built-in proxy configuration
reference](/docs/connect/configuration.html#built-in-proxy-options).
### Custom Managed Proxy
[Custom proxies](/docs/connect/proxies/integrate.html) can also be
configured to run as a managed proxy. To configure custom proxies, specify
an alternate command to execute for the proxy:
```json
{
"service": {
"name": "web",
"port": 8080,
"connect": {
"proxy": {
"exec_mode": "daemon",
"command": ["/usr/bin/my-proxy", "-flag-example"],
"config": {
"foo": "bar"
}
}
}
}
}
```
The `exec_mode` value specifies how the proxy is executed. The only
supported value at this time is "daemon". The command is the binary and
any arguments to execute.
The "daemon" mode expects a proxy to run as a long-running, blocking
process. It should not double-fork into the background. The custom
proxy should retrieve its configuration (such as the port to run on)
via the [custom proxy integration APIs](/docs/connect/proxies/integrate.html).
The default proxy command can be changed at an agent-global level
in the agent configuration. An example in HCL format is shown below.
```
connect {
proxy_defaults {
command = ["/usr/bin/my-proxy"]
}
}
```
With this configuration, all services registered without an explicit
proxy command will use `my-proxy` instead of the default built-in proxy.
The `config` key is an optional opaque JSON object which will be passed through
to the proxy via the proxy configuration endpoint to allow any configuration
options the proxy needs to be specified. See the [built-in proxy
configuration reference](/docs/connect/configuration.html#built-in-proxy-options)
for details of config options that can be passed when using the built-in proxy.
### Managed Proxy Logs
Managed proxies have both stdout and stderr captured in log files in the agent's
`data_dir`. They can be found in
`<data_dir>/proxy/logs/<proxy_service_id>-std{err,out}.log`.
The built-in proxy will inherit it's log level from the agent so if the agent is
configured with `log_level = DEBUG`, a proxy it starts will also output `DEBUG`
level logs showing service discovery, certificate and authorization information.
~> **Note:** In `-dev` mode there is no `data_dir` unless one is explicitly
configured so logging is disabled. You can access logs by providing the
[`-data-dir`](/docs/agent/options.html#_data_dir) CLI option. If a data dir is
configured, this will also cause proxy processes to stay running when the agent
terminates as described in [Lifecycle](#lifecycle).

View File

@ -0,0 +1,178 @@
---
layout: "docs"
page_title: "Connect - Sidecar Service Registration"
sidebar_current: "docs-connect-proxies-sidecar-service"
description: |-
Sidecar service registrations provide a convenient shorthand for registering a
sidecar proxy inline with a regular service definition.
---
# Sidecar Service Registration
Connect proxies are typically deployed as "sidecars" that run on the same node
as the single service instance that they handle traffic for. They might be on
the same VM or running as a separate container in the same network namespace.
To simplify the configuration experience when deploying a sidecar for a service
instance, Consul 1.3 introduced a new field in the Connect block of the [service
definition](/docs/agent/services.html).
The `connect.sidecar_service` field is a complete nested service definition on
which almost any regular service definition field can be set. The exceptions are
[noted below](#limitations). If used, the service definition is treated
identically to another top-level service definition. The value of the nested
definition is that _all fields are optional_ with some opinionated defaults
applied that make setting up a sidecar proxy much simpler.
## Minimal Example
To register a service instance with a sidecar, all that's needed is:
```json
{
"name": "web",
"port": 8080,
"connect": { "sidecar_service": {} }
}
```
This will register the `web` service as normal, but will also register another
[proxy service](/docs/connect/proxies.html) with defaults values used.
The above expands out to be equivalent to the following explicit service
definitions:
```json
{
"name": "web",
"port": 8080,
}
{
"name": "web-sidecar-proxy",
"port": 20000,
"kind": "connect-proxy",
"checks": [
{
"Name": "Connect Sidecar Listening",
"TCP": "127.0.0.1:20000",
"Interval": "10s"
},
{
"name": "Connect Sidecar Aliasing web",
"alias_service": "web"
}
],
"proxy": {
"destination_service_name": "db",
"destination_service_id": "db",
"local_service_address": "127.0.0.1",
"local_service_port": 9090,
}
}
```
Details on how the defaults are determined are [documented
below](#sidecar-service-defaults).
-> **Note:** Sidecar service registrations are only a shorthand for registering
multiple services. Consul will not start up or manage the actual proxy processes
for you.
## Overridden Example
The following example shows a service definition where some fields are
overridden to customize the proxy configuration.
```json
{
"name": "web",
"port": 8080,
"connect": {
"sidecar_service": {
"proxy": {
"upstreams": [
{
"destination_name": "db",
"local_bind_port": 9191
}
],
"config": {
"handshake_timeout_ms": 1000
}
}
}
}
}
```
This example customizes the [proxy
upstreams](/docs/connect/proxies.html#upstream-configuration-reference)
and some [built-in proxy
configuration](/docs/connect/configuration.html#built-in-proxy-options).
## Sidecar Service Defaults
The following fields are set by default on a sidecar service registration. With
[the exceptions noted](#limitations) any field may be overridden explicitly in
the `connect.sidecar_service` definition to customize the proxy registration.
The "parent" service refers to the service definition that embeds the sidecar
proxy.
- `id` - ID defaults to being `<parent-service-id>-sidecar-proxy`. This can't
be overridden as it is used to [manage the lifecycle](#lifecycle) of the
registration.
- `name` - Defaults to being `<parent-service-name>-sidecar-proxy`.
- `port` - Defaults to being auto-assigned from a [configurable
range](/docs/agent/options.html#sidecar_min_port) that is
by default `[21000, 21255]`.
- `kind` - Defaults to `connect-proxy`. This can't be overridden currently.
- `check`, `checks` - By default we add a TCP check on the local address and
port for the proxy, and a [service alias
check](/docs/agent/checks.html#alias) for the parent service. If either
`check` or `checks` fields are set, only the provided checks are registered.
- `proxy.destination_service_name` - Defaults to the parent service name.
- `proxy.destination_service_id` - Defaults to the parent service ID.
- `proxy.local_service_address` - Defaults to `127.0.0.1`.
- `proxy.local_service_port` - Defaults to the parent service port.
## Limitations
Almost all fields in a [service definition](/docs/agent/services.html) may be
set on the `connect.sidecar_service` except for the following:
- `id` - Sidecar services get an ID assigned and it is an error to override
this. This ensures the agent can correctly de-register the sidecar service
later when the parent service is removed.
- `kind` - Kind defaults to `connect-proxy` and there is currently no way to
unset this to make the registration be for a regular non-connect-proxy
service.
- `connect.sidecar_service` - Service definitions can't be nested recursively.
- `connect.proxy` - (Deprecated) [Managed
proxies](/docs/connect/proxies/managed-deprecated.html) can't be defined on a
sidecar.
- `connect.native` - Currently the `kind` is fixed to `connect-proxy` and it's
an error to register a `connect-proxy` that is also Connect-native.
## Lifecycle
Sidecar service registration is mostly a configuration syntax helper to avoid
adding lots of boiler plate for basic sidecar options, however the agent does
have some specific behavior around their lifecycle that makes them easier to
work with.
The agent fixes the ID of the sidecar service to be based on the parent
service's ID. This enables the following behavior.
- A service instance can _only ever have one_ sidecar service registered.
- When re-registering via API or reloading from configuration file:
- If something changes in the nested sidecar service definition, the change
will _update_ the current sidecar registration instead of creating a new
one.
- If a service registration removes the nested `sidecar_service` then the
previously registered sidecar for that service will be deregistered
automatically.
- When reloading the configuration files, if a service definition changes it's
ID, then a new service instance _and_ a new sidecar instance will be
registered. The old ones will be removed since they are no longer found in
the config files.

View File

@ -0,0 +1,258 @@
---
layout: "docs"
page_title: "Using Envoy with Connect"
sidebar_current: "docs-guides-connect-envoy"
description: |-
This guide walks though getting started running Envoy as a Connect Proxy.
---
# Using Connect with Envoy Proxy
Consul Connect has first class support for using
[Envoy](https://www.envoyproxy.io) as a proxy. This guide will walk through a
working example on a local development machine that shows the moving parts.
For reference documentation on how the integration works and is configured,
please see [Envoy](/docs/connect/proxies/envoy.html).
## Setup Overview
This guide will describe how to setup a development-mode Consul server and two
Envoy proxies on a single machine using [Docker](https://www.docker.com/). The
aim is to demonstrate a minimal working setup and the moving parts involved.
We choose to run in Docker since Envoy is only distributed as a Docker image so
it's the quickest way to get a demo running. The same commands used here will
work in just the same way outside of Docker if you build an Envoy binary
yourself.
We'll start all containers using Docker's `host` network mode which is not a
realistic simulation of a production setup, but makes the following steps much
simpler.
We should end up with five containers running:
1. The Consul agent
2. An example TCP `echo` service as a destination
3. An Envoy sidecar proxy for the `echo` service
4. An Envoy sidecar proxy for the `client` service
5. An example `client` service (netcat)
## Building an Envoy Image
Starting Envoy requires a bootstrap configuration file that points Envoy to the
local agent for discovering the rest of it's configuration. The Consul binary
includes the [`consul connect envoy` command](/docs/commands/connect/envoy.html)
which can generate the bootstrap configuration for Envoy and optionally run it
directly.
Envoy's official Docker image can be used with Connect directly however it
requires some additional steps to generate bootstrap configuration and inject it
into the container.
Instead, we'll use Docker multi-stage builds (added in version 17.05) to make a
local image that has both `envoy` and `consul` binaries.
We'll create a local Docker image to use that contains both binaries. First
create a `Dockerfile` containing the following:
```sh
FROM consul:latest
FROM envoyproxy/envoy:v1.8.0
COPY --from=0 /bin/consul /bin/consul
ENTRYPOINT ["dumb-init", "consul", "connect", "envoy"]
```
This takes the Consul binary from the latest release image and copies it into a
new image based on the official Envoy image.
This can be built locally with:
```sh
docker build -t consul-envoy .
```
We will use the `consul-envoy` image we just made to configure and run Envoy
processes later.
## Consul Agent Setup
Next we need a Consul agent. We'll work with a single Consul agent in `-dev`
mode for simplicity.
-> **Note:** `-dev` mode enables the gRPC server on port 8502 by default. For a
production agent you'll need to [explicitly configure the gRPC
port](/docs/agent/options.html#grpc_port).
In order to start a proxy instance, a [proxy service
definition](/docs/connect/proxies.html) must exist on the local agent. We'll
create one using the [sidecar service
registration](/docs/connect/proxies/sidecar-service.html) syntax.
Create a config file called `envoy_demo.hcl` containing the following service
definitions.
```hcl
services {
name = "client"
port = 8080
connect {
sidecar_service {
proxy {
upstreams {
destination_name = "echo"
local_bind_port = 9191
}
}
}
}
}
services {
name = "echo"
port = 9090
connect {
sidecar_service {}
}
}
```
The Consul agent container can now be started with that configuration.
```sh
$ docker run --rm -d -v$(pwd)/envoy_demo.hcl:/etc/consul/envoy_demo.hcl \
--network host --name consul-agent consul:latest \
agent -dev -config-file /etc/consul/envoy_demo.hcl
1c90f7fcc83f5390332d7a4fdda2f1bf74cf62762de9ea2f67cd5a09c0573641
```
Running with `-d` like this puts the container into the background so we can
continue in the same terminal. Log output can be seen using the name we gave.
```sh
docker logs -f consul-agent
```
Note that the Consul agent has registered two services `client` and `echo`, but
also registered two proxies `client-sidecar-proxy` and `echo-sidecar-proxy`.
Next we'll need to run those services and proxies.
## Running the Echo Service
Next we'll run the `echo` service. We can use an existing tcp echo utility image
for this.
Start the echo service on port 9090 as registered before.
```sh
$ docker run -d --network host abrarov/tcp-echo --port 9090
1a0b0c569016d00aadc4fc2b2954209b32b510966083f2a9e17d3afc6d185d87
```
## Running the Proxies
We can now run "sidecar" proxy instances.
```sh
$ docker run --rm -d --network host --name echo-proxy \
consul-envoy -sidecar-for echo
3f213a3cf9b7583a194dd0507a31e0188a03fc1b6e165b7f9336b0b1bb2baccb
$ docker run --rm -d --network host --name client-proxy \
consul-envoy -sidecar-for client -admin-bind localhost:19001
d8399b54ee0c1f67d729bc4c8b6e624e86d63d2d9225935971bcb4534233012b
```
The `-admin-bind` flag on the second proxy command is needed because both
proxies are running on the host network and so can't bind to the same port for
their admin API (which cannot be disabled).
Again we can see the output using docker logs. To see more verbose information
from Envoy you can add `-- -l debug` to the end of the commands above. This
passes the `-l` (log level) option directly through to Envoy. With debug level
logs you should see the config being delivered to the proxy in the output.
The [`consul connect envoy` command](/docs/commands/connect/envoy.html) here is
connecting to the local agent, getting the proxy configuration from the proxy
service registration and generating the required Envoy bootstrap configuration
before `exec`ing the envoy binary directly to run it with the generated
configuration.
Envoy uses the bootstrap configuration to connect to the local agent directly
via gRPC and use it's xDS protocol to retrieve the actual configuration for
listeners, TLS certificates, upstream service instances and so on. The xDS API
allows the Envoy instance to watch for any changes so certificate rotations or
changes to the upstream service instances are immediately sent to the proxy.
## Running the Client
Finally, we can see the connectivity by running a dummy "client" service. Rather
than run a full service that itself can listen, we'll simulate the service with
a simple netcat process that will only talk to the `client-sidecar-proxy` Envoy
instance.
Recall that we configured the `client` sidecar with one declared "upstream"
dependency (the `echo` service). In that declaration we also requested that the
`echo` service should be exposed to the client on local port 9191.
This configuration causes the `client-sidecar-proxy` to start a TCP proxy
listening on `localhost:9191` and proxying to the `echo` service. Importantly,
the listener will use the correct `client` service mTLS certificate to authorize
the connection. It discovers the IP addresses of instances of the echo service
via Consul service discovery.
We can now see this working if we run netcat.
```sh
$ docker run -ti --rm --network host gophernet/netcat localhost 9191
Hello World!
Hello World!
^C
```
## Testing Authorization
To demonstrate that Connect is controlling authorization for the echo service,
we can add an explicit deny rule.
```sh
$ docker run -ti --rm --network host consul:latest intention create -deny client echo
Created: client => echo (deny)
```
Now, new connections will be denied. Depending on a few factors, netcat may not
see the connection being closed but will not get a response from the service.
```sh
$ docker run -ti --rm --network host gophernet/netcat localhost 9191
Hello?
Anyone there?
^C
```
-> **Note:** Envoy will not currently re-authenticate already established TCP
connections so if you still have the netcat terminal open from before, that will
still be able to communicate with "echo". _New_ connections should be denied
though.
Removing the intention restores connectivity.
```
$ docker run -ti --rm --network host consul:latest intention delete client echo
Intention deleted.
$ docker run -ti --rm --network host gophernet/netcat localhost 9191
Hello?
Hello?
^C
```
## Summary
In this guide we walked through getting a minimal working example of two plain
TCP processes communicating over mTLS using Envoy sidecars configured by
Connect.
For more details on how the Envoy integration works, please see the [Envoy
reference documentation](/docs/connect/proxies/envoy.html).
To see how to get Consul Connect working in different environments like
Kubernetes see the [Connect Getting
Started](/docs/connect/index.html#getting-started-with-connect) overview.

View File

@ -6,7 +6,7 @@ description: |-
This guide describes best practices for running Consul Connect in production.
---
## Running Connect in Production
# Running Connect in Production
Consul Connect can secure all inter-service communication via mutual TLS. It's
designed to work with [minimal configuration out of the
@ -159,22 +159,21 @@ ports.
In addition to Consul agent's [communication
ports](/docs/agent/options.html#ports) any
[managed proxies](/docs/connect/proxies.html#managed-proxies) will need to have
[proxies](/docs/connect/proxies.html) will need to have
ports open to accept incoming connections.
Consul will by default assign them ports from [a configurable
range](/docs/agent/options.html#ports) the default
range is 20000 - 20255. If this feature is used, the agent assumes all ports in
that range are both free to use (no other processes listening on them) and are
exposed in the firewall to accept connections from other service hosts.
If using [sidecar service
registration](/docs/connect/proxies/sidecar-service.html) Consul will by default
assign ports from [a configurable
range](/docs/agent/options.html#sidecar_min_port) the default range is 21000 -
21255. If this feature is used, the agent assumes all ports in that range are
both free to use (no other processes listening on them) and are exposed in the
firewall to accept connections from other service hosts.
Alternatively, managed proxies can have their public ports specified as part of
the [proxy
configuration](/docs/connect/configuration.html#local_bind_port) in the
service definition. It is possible to use this exclusively and prevent
automated port selection by [configuring `proxy_min_port` and
`proxy_max_port`](/docs/agent/options.html#ports) to both be `0`, forcing any
managed proxies to have an explicit port configured.
It is possible to prevent automated port selection by [configuring
`sidecar_min_port` and
`sidecar_max_port`](/docs/agent/options.html#sidecar_min_port) to both be `0`,
forcing any sidecar service registrations to need an explicit port configured.
It then becomes the same problem as opening ports necessary for any other
application and might be managed by configuration management or a scheduler.
@ -190,13 +189,9 @@ be set.
For registration via the API [the token is passed in the request
header](/api/index.html#acls) or by using the [Go
client configuration](https://godoc.org/github.com/hashicorp/consul/api#Config).
Note that by default API registration will not allow managed proxies to be
configured since it potentially opens a remote execution vulnerability if the
agent API endpoints are publicly accessible. This can be [configured
per-agent](/docs/agent/options.html#connect_proxy).
For examples of service definitions with managed or unmanaged proxies see
[proxies documentation](/docs/connect/proxies.html#managed-proxies).
For examples of proxy service definitions see the [proxy
documentation](/docs/connect/proxies.html).
To avoid the overhead of a proxy, applications may [natively
integrate](/docs/connect/native.html) with connect.

View File

@ -6,7 +6,7 @@ description: |-
This guide describes how to run Consul on containers, with Docker as the primary focus. It also describes best practices when running a Consul cluster in production on Docker.
---
## Consul with Containers
# Consul with Containers
This guide describes critical aspects of operating a Consul cluster that's run inside containers. It primarily focuses on the Docker container runtime, but the principles largely apply to rkt, oci, and other container runtimes as well.
## Consul Official Docker Image

View File

@ -28,11 +28,10 @@ guide](/docs/guides/connect-production.html) to understand the tradeoffs.
## Starting a Connect-unaware Service
Let's begin by starting a service that is unaware of Connect all. To
keep it simple, let's just use `socat` to start a basic echo service. This
service will accept TCP connections and echo back any data sent to it. If
`socat` isn't installed on your machine, it should be easily available via
a package manager.
Let's begin by starting a service that is unaware of Connect. To keep it simple,
let's just use `socat` to start a basic echo service. This service will accept
TCP connections and echo back any data sent to it. If `socat` isn't installed on
your machine, it should be easily available via a package manager.
```sh
$ socat -v tcp-l:8181,fork exec:"/bin/cat"
@ -67,7 +66,7 @@ $ cat <<EOF | sudo tee /etc/consul.d/socat.json
"service": {
"name": "socat",
"port": 8181,
"connect": { "proxy": {} }
"connect": { "sidecar_service": {} }
}
}
EOF
@ -76,20 +75,34 @@ EOF
After saving this, run `consul reload` or send a `SIGHUP` signal to Consul
so it reads the new configuration.
Notice the only difference is the line starting with `"connect"`. The
existence of this empty configuration notifies Consul to manage a
proxy process for this process.
The proxy process represents that specific service. It accepts inbound
Notice the only difference is the line starting with `"connect"`. The existence
of this empty configuration notifies Consul to register a sidecar proxy for this
process. The proxy process represents that specific service. It accepts inbound
connections on a dynamically allocated port, verifies and authorizes the TLS
connection, and proxies back a standard TCP connection to the process.
The sidecar service registration here is just telling Consul that a proxy should
be running, Consul won't actually run a proxy process for you.
We need to start the proxy process in another terminal:
```sh
$ consul connect proxy -sidecar-for socat
==> Consul Connect proxy starting...
Configuration mode: Agent API
Sidecar for ID: socat
Proxy ID: socat-sidecar-proxy
...
```
## Connecting to the Service
Next, let's connect to the service. We'll first do this by using the
`consul connect proxy` command directly. This command manually runs a local
proxy that can represent a service. This is a useful tool for development
since it'll let you masquerade as any service (that you have permissions for)
and establish connections to other services via Connect.
Next, let's connect to the service. We'll first do this by using the `consul
connect proxy` command again directly. This time we use the command to configure
and run a local proxy that can represent a service. This is a useful tool for
development since it'll let you masquerade as any service (that you have
permissions for) and establish connections to other services via Connect.
The command below starts a proxy representing a service "web". We request
an upstream dependency of "socat" (the service we previously registered)
@ -124,10 +137,10 @@ machine is always encrypted.
## Registering a Dependent Service
We previously established a connection by directly running
`consul connect proxy`. Realistically, services need to establish connections
We previously established a connection by directly running `consul connect
proxy` in developer mode. Realistically, services need to establish connections
to dependencies over Connect. Let's register a service "web" that registers
"socat" as an upstream dependency:
"socat" as an upstream dependency in it's sidecar registration:
```sh
$ cat <<EOF | sudo tee /etc/consul.d/web.json
@ -136,8 +149,8 @@ $ cat <<EOF | sudo tee /etc/consul.d/web.json
"name": "web",
"port": 8080,
"connect": {
"proxy": {
"config": {
"sidecar_service": {
"proxy": {
"upstreams": [{
"destination_name": "socat",
"local_bind_port": 9191
@ -150,15 +163,51 @@ $ cat <<EOF | sudo tee /etc/consul.d/web.json
EOF
```
This configures a Consul-managed proxy for the service "web" that
is listening on port 9191 to establish connections to "socat" as "web". The
This registers a sidecar proxy for the service "web" that
should listen on port 9191 to establish connections to "socat" as "web". The
"web" service should then use that local port to talk to socat rather than
directly attempting to connect.
With that file in place, use `consul reload` or SIGHUP to reload Consul. If the
proxy command from the previous section (with the inline upstream listener) is
still running, stop it with `Ctrl-C`. Now we can start the web proxy using the
configuration from the sidecar registration as we did for socat.
```sh
$ consul connect proxy -sidecar-for web
==> Consul Connect proxy starting...
Configuration mode: Agent API
Sidecar for ID: web
Proxy ID: web-sidecar-proxy
==> Log data will now stream in as it occurs:
2018/10/09 12:34:20 [INFO] 127.0.0.1:9191->service:default/socat starting on 127.0.0.1:9191
2018/10/09 12:34:20 [INFO] Proxy loaded config and ready to serve
2018/10/09 12:34:20 [INFO] TLS Identity: spiffe://df34ef6b-5971-ee61-0790-ca8622c3c287.consul/ns/default/dc/dc1/svc/web
2018/10/09 12:34:20 [INFO] TLS Roots : [Consul CA 7]
```
Note in the first log line that the proxy discovered its configuration from the
local agent and setup a local listener on port 9191 that will proxy to the socat
service just as we configured in the sidecar registration.
You can also see the identity URL from the certificate it loaded from the agent
identifying it as the "web" service and the set of trusted root CAs it knows
about.
-> **Security note:** The Connect security model requires trusting
loopback connections when proxies are in use. To further secure this,
tools like network namespacing may be used.
We can verify it works by establishing a new connection:
```
$ nc 127.0.0.1 9191
hello
hello
```
## Controlling Access with Intentions
Intentions are used to define which services may communicate. Our connections
@ -180,9 +229,18 @@ $ nc 127.0.0.1 9191
$
```
Try deleting the intention (or updating it to allow) and attempting the
connection again. Intentions allow services to be segmented via a centralized
control plane (Consul). To learn more, read the reference documentation on
Try deleting the intention and attempt the connection again.
```sh
$ consul intention delete web socat
Intention deleted.
$ nc 127.0.0.1 9191
hello
hello
```
Intentions allow services to be segmented via a centralized control plane
(Consul). To learn more, read the reference documentation on
[intentions](/docs/connect/intentions.html).
Note that in the current release of Consul, changing intentions will not
@ -190,6 +248,13 @@ affect existing connections. Therefore, you must establish a new connection
to see the effects of a changed intention. This will be addressed in the near
term in a future version of Consul.
## Discover More Connect
This quick guide has given a taste of what Connect can do but there is much
more. Take a look at [getting started with
Connect](/docs/connect/index.html#getting-started-with-connect) for more guides
on setting up Connect with Envoy proxy, with Docker and in Kubernetes.
## Next Steps
We've now configured a service on a single agent and used Connect for

View File

@ -79,6 +79,9 @@
<li<%= sidebar_current("docs-commands-connect-proxy") %>>
<a href="/docs/commands/connect/proxy.html">proxy</a>
</li>
<li<%= sidebar_current("docs-commands-connect-envoy") %>>
<a href="/docs/commands/connect/envoy.html">envoy</a>
</li>
</ul>
</li>
<li<%= sidebar_current("docs-commands-event") %>>
@ -269,6 +272,13 @@
<li<%= sidebar_current("docs-connect-proxies") %>>
<a href="/docs/connect/proxies.html">Proxies</a>
<ul class="nav">
<li<%= sidebar_current("docs-connect-proxies-sidecar-service") %>>
<a href="/docs/connect/proxies/sidecar-service.html">Sidecar Service Registration</a>
</li>
<li<%= sidebar_current("docs-connect-proxies-envoy") %>>
<a href="/docs/connect/proxies/envoy.html">Envoy</a>
</li>
</li>
<li<%= sidebar_current("docs-connect-proxies-integrate") %>>
<a href="/docs/connect/proxies/integrate.html">Proxy Integration</a>
</li>
@ -355,6 +365,9 @@
<li<%= sidebar_current("docs-guides-connect-production") %>>
<a href="/docs/guides/connect-production.html">Connect in Production</a>
</li>
<li<%= sidebar_current("docs-guides-connect-envoy") %>>
<a href="/docs/guides/connect-envoy.html">Connect with Envoy</a>
</li>
<li<%= sidebar_current("docs-guides-consul-containers") %>>
<a href="/docs/guides/consul-containers.html">Consul with Containers</a>
</li>