mirror of https://github.com/status-im/consul.git
248 lines
14 KiB
Plaintext
248 lines
14 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: Annotations and Labels
|
|
description: >-
|
|
The list of available labels and annotations for running Consul on Kubernetes.
|
|
---
|
|
|
|
# Annotations and Labels
|
|
|
|
## Overview
|
|
|
|
Consul on Kubernetes provides a few options for customizing how connect-inject behavior should be configured.
|
|
This allows the user to configure natively configure Consul on select Kubernetes resources (i.e. pods, services).
|
|
|
|
- [Annotations](#annotations)
|
|
- [Labels](#labels)
|
|
|
|
## Annotations
|
|
|
|
The following Kubernetes resource annotations could be used on a pod to control connect-inject behavior:
|
|
|
|
- `consul.hashicorp.com/connect-inject` - If this is "true" then injection
|
|
is enabled. If this is "false" then injection is explicitly disabled.
|
|
The default injector behavior requires pods to opt-in to injection by
|
|
specifying this value as "true". This default can be changed in the
|
|
injector's configuration if desired.
|
|
|
|
- `consul.hashicorp.com/transparent-proxy` - If this is "true", this Pod
|
|
will run with transparent proxy enabled. This means you can use Kubernetes
|
|
DNS to access upstream services and all inbound and outbound traffic within
|
|
the pod is redirected to go through the proxy.
|
|
|
|
- `consul.hashicorp.com/transparent-proxy-overwrite-probes` - If this is "true"
|
|
and transparent proxy is enabled, the Connect injector will overwrite Kubernetes
|
|
HTTP probes to point to the Envoy proxy.
|
|
|
|
- `consul.hashicorp.com/transparent-proxy-exclude-inbound-ports` - A comma-separated
|
|
list of inbound ports to exclude from traffic redirection when running in transparent proxy
|
|
mode.
|
|
|
|
- `consul.hashicorp.com/transparent-proxy-exclude-outbound-cidrs` - A comma-separated
|
|
list of outbound CIDRs to exclude from traffic redirection when running in transparent proxy
|
|
mode.
|
|
|
|
- `consul.hashicorp.com/transparent-proxy-exclude-outbound-ports` - A comma-separated
|
|
list of outbound ports to exclude from traffic redirection when running in transparent proxy
|
|
mode.
|
|
|
|
- `consul.hashicorp.com/transparent-proxy-exclude-uids` - A comma-separated
|
|
list of additional user IDs to exclude from traffic redirection when running in transparent proxy
|
|
mode.
|
|
|
|
- `consul.hashicorp.com/connect-service` - For pods that accept inbound
|
|
connections, this specifies the name of the service that is being
|
|
served. This defaults to the name of the Kubernetes service associated with the pod.
|
|
|
|
If using ACLs, this must be the same name as the Pod's `ServiceAccount`.
|
|
|
|
- `consul.hashicorp.com/connect-service-port` - For pods that accept inbound
|
|
connections, this specifies the port to route inbound connections to. This
|
|
is the port that the service is listening on. The service port defaults to
|
|
the first exposed port on any container in the pod. If specified, the value
|
|
can be the _name_ of a configured port, such as "http" or it can be a direct
|
|
port value such as "8080". This is the port of the _service_, the proxy
|
|
public listener will listen on a dynamic port.
|
|
|
|
- `consul.hashicorp.com/connect-service-upstreams` - The list of upstream
|
|
services that this pod needs to connect to via Connect along with a static
|
|
local port to listen for those connections. When transparent proxy is enabled,
|
|
this annotation is optional. There are a few formats this annotation can take:
|
|
|
|
- Unlabeled:
|
|
Use the unlabeled annotation format to specify a service name, Consul Enterprise namespaces and partitions, and
|
|
datacenters. To use [cluster peering](/docs/connect/cluster-peering/k8s) with upstreams, use the following
|
|
labeled format.
|
|
- Service name: Place the service name at the beginning of the annotation to specify the upstream service. You can
|
|
also append the datacenter where the service is deployed (optional).
|
|
```yaml
|
|
annotations:
|
|
"consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter]"
|
|
```
|
|
- Namespace (requires Consul Enterprise 1.7+): Upstream services may be running in different a namespace. Place
|
|
the upstream namespace after the service name. For additional details about configuring the injector, refer to
|
|
[Consul Enterprise Namespaces](#consul-enterprise-namespaces) .
|
|
```yaml
|
|
annotations:
|
|
"consul.hashicorp.com/connect-service-upstreams":"[service-name].[service-namespace]:[port]:[optional datacenter]"
|
|
```
|
|
If the namespace is not specified, the annotation defaults to the namespace of the source service.
|
|
If you are not using Consul Enterprise 1.7+, Consul interprets the value placed in the namespace position as part of the service name.
|
|
|
|
- Admin partitions (requires Consul Enterprise 1.11+): Upstream services may be running in a different
|
|
partition. You must specify the namespace when specifying a partition. Place the partition name after the namespace. If you specify the name of the datacenter (optional), it must be the local datacenter. Communicating across partitions using this method is only supported within a
|
|
datacenter. For cross partition communication across datacenters, refer to [cluster
|
|
peering](/docs/connect/cluster-peering/k8s).
|
|
```yaml
|
|
annotations:
|
|
"consul.hashicorp.com/connect-service-upstreams":"[service-name].[service-namespace].[service-partition]:[port]:[optional datacenter]"
|
|
```
|
|
- [Prepared queries](/docs/connect/proxies#dynamic-upstreams-require-native-integration): Prepend the annotation
|
|
with `prepared_query` and place the name of the query at the beginning of the string.
|
|
```yaml
|
|
annotations:
|
|
'consul.hashicorp.com/connect-service-upstreams': 'prepared_query:[query name]:[port]'
|
|
```
|
|
|
|
- Labeled (requires Consul for Kubernetes v0.45.0+):
|
|
The labeled format is required when using the cluster peering feature and specifying an upstream in another
|
|
peer. You can specify a Consul Enterprise namespace, partition, or datacenter. The format supports only one peer, datacenter, or partition.
|
|
- Service name: Place the service name at the beginning of the annotation followed by `.svc` to specify the upstream service.
|
|
```yaml
|
|
annotations:
|
|
"consul.hashicorp.com/connect-service-upstreams":"[service-name].svc:[port]"
|
|
```
|
|
- Peer or datacenter: Place the peer or datacenter after `svc.` followed by either `peer` or `dc` and the port number.
|
|
```yaml
|
|
annotations:
|
|
"consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-peer].peer:[port]"
|
|
```
|
|
```yaml
|
|
annotations:
|
|
"consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-dc].dc:[port]"
|
|
```
|
|
- Namespace (required Consul Enterprise): Place the namespace after `svc.` followed by `ns` and the port number.
|
|
```yaml
|
|
annotations:
|
|
"consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns:[port]"
|
|
```
|
|
When specifying a peer, datacenter, or admin partition when namespaces are enabled, you must
|
|
provide the namespace .
|
|
```yaml
|
|
annotations:
|
|
"consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-peer].peer:[port]"
|
|
```
|
|
```yaml
|
|
annotations:
|
|
"consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-partition].ap:[port]"
|
|
```
|
|
```yaml
|
|
annotations:
|
|
"consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-dc].dc:[port]"
|
|
```
|
|
|
|
- Multiple upstreams: Delimit multiple services or upstreams with commas. You can specify any of the unlabeled, labeled, or prepared query formats when using the supported versions for the formats.
|
|
|
|
```yaml
|
|
annotations:
|
|
"consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter],[service-name]:[port]:[optional datacenter]"
|
|
```
|
|
|
|
```yaml
|
|
annotations:
|
|
"consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter],prepared_query:[query name]:[port],[service-name].svc:[port]"
|
|
```
|
|
|
|
- `consul.hashicorp.com/envoy-extra-args` - A space-separated list of [arguments](https://www.envoyproxy.io/docs/envoy/latest/operations/cli)
|
|
to be passed to the injected envoy binary.
|
|
|
|
```yaml
|
|
annotations:
|
|
consul.hashicorp.com/envoy-extra-args: '--log-level debug --disable-hot-restart'
|
|
```
|
|
|
|
- `consul.hashicorp.com/kubernetes-service` - Specifies the name of the Kubernetes service used for Consul service registration.
|
|
This is useful when multiple Kubernetes services reference the same deployment. Any service that does not match the name
|
|
specified in this annotation is ignored. When not specified no service is ignored.
|
|
|
|
```yaml
|
|
annotations:
|
|
consul.hashicorp.com/kubernetes-service: 'service-name-to-use'
|
|
```
|
|
|
|
- `consul.hashicorp.com/service-tags` - A comma separated list of tags that will
|
|
be applied to the Consul service and its sidecar.
|
|
|
|
```yaml
|
|
annotations:
|
|
consul.hashicorp.com/service-tags: foo,bar,baz
|
|
```
|
|
|
|
If you need your tag to have a comma in it you can escape the comma with `\,`. For example,
|
|
`consul.hashicorp.com/service-tags: foo\,bar\,baz` will become the single tag `foo,bar,baz`.
|
|
|
|
- `consul.hashicorp.com/service-meta-<YOUR_KEY>` - Set Consul meta key/value
|
|
pairs that will be applied to the Consul service and its sidecar.
|
|
The key will be what comes after `consul.hashicorp.com/service-meta-`, e.g.
|
|
`consul.hashicorp.com/service-meta-foo: bar` will result in `foo: bar`.
|
|
|
|
```yaml
|
|
annotations:
|
|
consul.hashicorp.com/service-meta-foo: baz
|
|
consul.hashicorp.com/service-meta-bar: baz
|
|
```
|
|
|
|
- `consul.hashicorp.com/sidecar-proxy-` - Override default resource settings for
|
|
the sidecar proxy container.
|
|
The defaults are set in Helm config via the [`connectInject.sidecarProxy.resources`](/docs/k8s/helm#v-connectinject-sidecarproxy-resources) key.
|
|
|
|
- `consul.hashicorp.com/sidecar-proxy-cpu-limit` - Override the default CPU limit.
|
|
- `consul.hashicorp.com/sidecar-proxy-cpu-request` - Override the default CPU request.
|
|
- `consul.hashicorp.com/sidecar-proxy-memory-limit` - Override the default memory limit.
|
|
- `consul.hashicorp.com/sidecar-proxy-memory-request` - Override the default memory request.
|
|
|
|
- `consul.hashicorp.com/consul-envoy-proxy-concurrency` - Override the default envoy worker thread count. This should be set low for sidecar
|
|
usecases and can be raised for edge proxies like gateways.
|
|
|
|
- `consul.hashicorp.com/consul-sidecar-` - Override default resource settings for
|
|
the `consul-sidecar` container.
|
|
The defaults are set in Helm config via the [`global.consulSidecarContainer.resources`](/docs/k8s/helm#v-global-consulsidecarcontainer) key.
|
|
|
|
- `consul.hashicorp.com/consul-sidecar-cpu-limit` - Override the default CPU limit.
|
|
- `consul.hashicorp.com/consul-sidecar-cpu-request` - Override the default CPU request.
|
|
- `consul.hashicorp.com/consul-sidecar-memory-limit` - Override the default memory limit.
|
|
- `consul.hashicorp.com/consul-sidecar-memory-request` - Override the default memory request.
|
|
|
|
- `consul.hashicorp.com/enable-metrics` - Override the default Helm value [`connectInject.metrics.defaultEnabled`](/docs/k8s/helm#v-connectinject-metrics-defaultenabled).
|
|
- `consul.hashicorp.com/enable-metrics-merging` - Override the default Helm value [`connectInject.metrics.defaultEnableMerging`](/docs/k8s/helm#v-connectinject-metrics-defaultenablemerging).
|
|
- `consul.hashicorp.com/merged-metrics-port` - Override the default Helm value [`connectInject.metrics.defaultMergedMetricsPort`](/docs/k8s/helm#v-connectinject-metrics-defaultmergedmetricsport).
|
|
- `consul.hashicorp.com/prometheus-scrape-port` - Override the default Helm value [`connectInject.metrics.defaultPrometheusScrapePort`](/docs/k8s/helm#v-connectinject-metrics-defaultprometheusscrapeport).
|
|
- `consul.hashicorp.com/prometheus-scrape-path` - Override the default Helm value [`connectInject.metrics.defaultPrometheusScrapePath`](/docs/k8s/helm#v-connectinject-metrics-defaultprometheusscrapepath).
|
|
- `consul.hashicorp.com/prometheus-ca-file` - Local filesystem path to a CA file for Envoy to use
|
|
when serving TLS on the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr`
|
|
is set in proxy config.
|
|
- `consul.hashicorp.com/prometheus-ca-path` - Local filesystem path to a directory of CA certificates
|
|
for Envoy to use when serving TLS on the Prometheus metrics endpoint. Only applicable when
|
|
`envoy_prometheus_bind_addr` is set in proxy config.
|
|
- `consul.hashicorp.com/prometheus-cert-file` - Local filesystem path to a certificate file for Envoy to use
|
|
when serving TLS on the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr`
|
|
is set in proxy config.
|
|
- `consul.hashicorp.com/prometheus-key-file` - Local filesystem path to a private key file for Envoy to use
|
|
when serving TLS on the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr`
|
|
is set in proxy config.
|
|
- `consul.hashicorp.com/service-metrics-port` - Set the port where the Connect service exposes metrics.
|
|
- `consul.hashicorp.com/service-metrics-path` - Set the path where the Connect service exposes metrics.
|
|
- `consul.hashicorp.com/connect-inject-mount-volume` - Comma separated list of container names to mount the connect-inject volume into. The volume will be mounted at `/consul/connect-inject`. The connect-inject volume contains Consul internals data needed by the other sidecar containers, for example the `consul` binary, and the Pod's Consul ACL token. This data can be valuable for advanced use-cases, such as making requests to the Consul API from within application containers.
|
|
|
|
## Labels
|
|
|
|
Resource labels could be used on a Kubernetes service to control connect-inject behavior.
|
|
|
|
- `consul.hashicorp.com/service-ignore` - This label can be set on a Kubernetes Service.
|
|
If set to "true", the service will not be used to register a Consul endpoint. This can be
|
|
useful in cases where 2 or more services point to the same deployment. Consul cannot register
|
|
multiple endpoints to the same deployment. This label can be used to tell the endpoint
|
|
registration to ignore all services except for the one which should be used for routing requests
|
|
using Consul.
|
|
|