Merge pull request #7207 from hashicorp/namespace-k8s-docs

Docs for consul-k8s namespaces support
This commit is contained in:
Luke Kysow 2020-02-21 14:05:38 -07:00 committed by GitHub
commit ca6ba769ff
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 346 additions and 22 deletions

View File

@ -18,7 +18,7 @@ your cluster, making configuration for Kubernetes automatic.
This functionality is provided by the
[consul-k8s project](https://github.com/hashicorp/consul-k8s) and can be
automatically installed and configured using the
[Consul Helm chart](/docs/platform/k8s/helm.html).
[Consul Helm chart](/docs/platform/k8s/run.html).
## Usage
@ -132,8 +132,8 @@ Pods must specify upstream dependencies with the
This annotation declares the names of any upstream dependencies and a
local port for the proxy to listen on. When a connection is established to that local
port, the proxy establishes a connection to the target service
("static-server" in this example) using
mutual TLS and identifying as the source service ("static-client" in this
(`static-server` in this example) using
mutual TLS and identifying as the source service (`static-client` in this
example).
The injector will also set environment variables `<NAME>_CONNECT_SERVICE_HOST`
@ -206,6 +206,23 @@ Annotations can be used to configure the injection behavior.
annotations:
"consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter]"
```
* Consul Enterprise Namespaces
If running Consul Enterprise 1.7+, your upstream services may be running in different
namespaces. The upstream namespace can be specified after the service name
as `[service-name].[namespace]`. See [Consul Enterprise Namespaces](#consul-enterprise-namespaces)
below for more details on configuring the injector.
```yaml
annotations:
"consul.hashicorp.com/connect-service-upstreams":"[service-name].[service-namespace]:[port]:[optional datacenter]"
```
-> **NOTE:** If the namespace is not specified it will default to the namespace
of the source service.
~> **WARNING:** Setting a namespace when not using Consul Enterprise or using a version < 1.7
is not supported. It will be treated as part of the service name.
* [Prepared Query](https://www.consul.io/docs/connect/proxies.html#upstreams)
@ -303,7 +320,7 @@ provided by the
[consul-k8s project](https://github.com/hashicorp/consul-k8s).
This enables the automatic pod mutation shown in the usage section above.
Installation of the mutating admission webhook is automated using the
[Helm chart](/docs/platform/k8s/helm.html).
[Helm chart](/docs/platform/k8s/run.html).
To install the Connect injector, enable the Connect injection feature using
[Helm values](/docs/platform/k8s/helm.html#configuration-values-) and
@ -319,23 +336,167 @@ connectInject:
client:
enabled: true
grpc: true
```
This will configure the injector to inject when the
[injection annotation](#)
is present. Other values in the Helm chart can be used to limit the namespaces
[injection annotation](#consul-hashicorp-com-connect-inject)
is set to `true`. Other values in the Helm chart can be used to limit the namespaces
the injector runs in, enable injection by default, and more.
As noted above, the Connect auto-injection requires that local client agents
are configured. These client agents must be successfully joined to a Consul
cluster.
The Consul server cluster can run either in or out of a Kubernetes cluster.
~> NOTE: If setting `global.bootstrapACLs: true`, it's important that your Pod's `ServiceAccount`
~> NOTE: If setting `global.bootstrapACLs: true`, it's important that your pod's `ServiceAccount`
has the **same name** as the Consul service that's being registered. If not, the init
container will log: `Error logging in: Unexpected response code: 403 (rpc error making call: rpc error making call: Permission denied)`.
### Controlling Injection Via Annotation
By default, the injector will inject only when the
[injection annotation](#consul-hashicorp-com-connect-inject)
on the pod (not the deployment) is set to `true`:
```yaml
annotations:
"consul.hashicorp.com/connect-inject": "true"
```
### Injection Defaults
If you wish for the injector to always inject, you can set the default to `true`
in the Helm chart:
```yaml
connectInject:
enabled: true
default: true
```
You can then exclude specific pods via annotation:
```yaml
annotations:
"consul.hashicorp.com/connect-inject": "false"
```
### Controlling Injection Via Namespace
You can control which Kubernetes namespaces are allowed to be injected via
the `k8sAllowNamespaces` and `k8sDenyNamespaces` keys:
```yaml
connectInject:
enabled: true
k8sAllowNamespaces: ["*"]
k8sDenyNamespaces: []
```
In the default configuration (shown above), services from all namespaces are allowed
to be injected. Whether or not they're injected depends on the value of `connectInject.default`
and the `consul.hashicorp.com/connect-inject` annotation.
If you wish to only enable injection in specific namespaces, you can list only those
namespaces in the `k8sAllowNamespaces` key. In the configuration below
only the `my-ns-1` and `my-ns-2` namespaces will be enabled for injection.
All other namespaces will be ignored, even if the connect inject [annotation](#consul-hashicorp-com-connect-inject)
is set.
```yaml
connectInject:
enabled: true
k8sAllowNamespaces: ["my-ns-1", "my-ns-2"]
k8sDenyNamespaces: []
```
If you wish to enable injection in every namespace *except* specific namespaces, you can
use `*` in the allow list to allow all namespaces and then specify the namespaces to exclude in the deny list:
```yaml
syncCatalog:
enabled: true
k8sAllowNamespaces: ["*"]
k8sDenyNamespaces: ["no-sync-ns-1", "no-sync-ns-2"]
```
-> **NOTE:** The deny list takes precedence over the allow list. If a namespace
is listed in both lists, it will **not** be synced.
~> **NOTE:** The `kube-system` and `kube-public` namespaces will never be injected.
### Consul Clients Required
Connect injection requires that local client agents
are running on each Kubernetes node. These client agents must be joined to a Consul
server cluster.
The Consul server cluster can run either in or out of a Kubernetes cluster.
### Consul Enterprise Namespaces
Consul Enterprise 1.7+ supports Consul namespaces. When Kubernetes pods are registered
into Consul, you can control which Consul namespace they are registered into.
There are three options available:
1. **Single Destination Namespace** Register all Kubernetes pods, regardless of namespace,
into the same Consul namespace.
This can be configured with:
```yaml
global:
enableConsulNamespaces: true
connectInject:
enabled: true
consulNamespaces:
consulDestinationNamespace: "my-consul-ns"
```
-> **NOTE:** If the destination namespace does not exist we will create it.
1. **Mirror Namespaces** - Register each Kubernetes pod into a Consul namespace with the same name as its Kubernetes namespace.
For example, pod `foo` in Kubernetes namespace `ns-1` will be synced to the Consul namespace `ns-1`.
If a mirrored namespace does not exist in Consul, it will be created.
This can be configured with:
```yaml
global:
enableConsulNamespaces: true
connectInject:
enabled: true
consulNamespaces:
mirroringK8S: true
```
1. **Mirror Namespaces With Prefix** - Register each Kubernetes pod into a Consul namespace with the same name as its Kubernetes
namespace **with a prefix**.
For example, given a prefix `k8s-`, pod `foo` in Kubernetes namespace `ns-1` will be synced to the Consul namespace `k8s-ns-1`.
This can be configured with:
```yaml
global:
enableConsulNamespaces: true
connectInject:
enabled: true
consulNamespaces:
mirroringK8S: true
mirroringK8SPrefix: "k8s-"
```
### Consul Enterprise Namespace Upstreams
To specify the namespace of your upstream services in the upstream annotation,
use the format `[service-name].[namespace]:[port]:[optional datacenter]`:
```yaml
annotations:
"consul.hashicorp.com/connect-inject": "true"
"consul.hashicorp.com/connect-service-upstreams": "[service-name].[namespace]:[port]:[optional datacenter]"
```
See [consul.hashicorp.com/connect-service-upstreams](#consul-hashicorp-com-connect-service-upstreams) for more details.
### Verifying the Installation
To verify the installation, run the

View File

@ -60,6 +60,8 @@ and consider if they're appropriate for your deployment.
* <a name="v-global-gossip-encryption-secret-key" href="#v-global-gossip-encryption-secret-key">`secretKey`</a> (`string: ""`) - The key within the Kubernetes secret that holds the gossip encryption key.
* <a name="v-global-enableconsulnamespaces" href="#v-global-enableconsulnamespaces">`enableConsulNamespaces`</a> (`boolean: false`) - [Enterprise Only] `enableConsulNamespaces` indicates that you are running Consul Enterprise v1.7+ with a valid Consul Enterprise license and would like to make use of configuration beyond registering everything into the `default` Consul namespace. Requires consul-k8s v0.12+. Additional configuration options are found in the `consulNamespaces` section of both the catalog sync and connect injector.
* <a name="v-global-bootstrap-acls" href="#v-global-bootstrap-acls">`bootstrapACLs`</a> (`boolean: false`) - Automatically create and assign ACL tokens within the Consul cluster. This requires servers to be running inside Kubernetes. Additionally requires Consul >= 1.4 and consul-k8s >= 0.8.0.
* <a name="v-server" href="#v-server">`server`</a> - Values that configure running a Consul server within Kubernetes.
@ -254,7 +256,27 @@ to run the sync program.
* <a name="v-synccatalog-tok8s" href="#v-synccatalog-tok8s">`toK8S`</a> (`boolean: true`) - If true, will sync Consul services to Kubernetes. This can be disabled to have a one-way sync.
* <a name="v-synccatalog-k8sprefix" href="#v-synccatalog-k8sprefix">`k8sPrefix`</a> (`string: ""`) - A prefix to prepend to all services registered in Kubernetes from Consul. This defaults to `""` where no prefix is prepended; Consul services are synced with the same name to Kubernetes. (Consul -> Kubernetes sync only)
* <a name="v-synccatalog-k8sallownamespaces" href="#v-synccatalog-k8sallownamespaces">`k8sAllowNamespaces`</a> (`[]string: ["*"]`) - list of k8s namespaces to sync the k8s services from. If a k8s namespace is not included in this list or is listed in `k8sDenyNamespaces`, services in that k8s namespace will not be synced even if they are explicitly annotated. Use `["*"]` to automatically allow all k8s namespaces.
For example, `["namespace1", "namespace2"]` will only allow services in the k8s namespaces `namespace1` and `namespace2` to be synced and registered with Consul. All other k8s namespaces will be ignored.
Note: `k8sDenyNamespaces` takes precedence over values defined here. Requires consul-k8s v0.12+
* <a name="v-synccatalog-k8sdenynamespaces" href="#v-synccatalog-k8sdenynamespaces">`k8sDenyNamespaces`</a> (`[]string: ["kube-system", "kube-public"]` - list of k8s namespaces that should not have their services synced. This list takes precedence over `k8sAllowNamespaces`. `*` is not supported because then nothing would be allowed to sync. Requires consul-k8s v0.12+.
For example, if `k8sAllowNamespaces` is `["*"]` and `k8sDenyNamespaces` is `["namespace1", "namespace2"]`, then all k8s namespaces besides `namespace1` and `namespace2` will be synced.
* <a name="v-synccatalog-k8ssourcenamespace" href="#v-synccatalog-k8ssourcenamespace">`k8sSourceNamespace`</a> (`string: ""`) - **[DEPRECATED] Use `k8sAllowNamespaces` and `k8sDenyNamespaces` instead.** `k8sSourceNamespace` is the Kubernetes namespace to watch for service changes and sync to Consul. If this is not set then it will default to all namespaces.
* <a name="v-synccatalog-consulnamespaces" href="#v-synccatalog-consulnamespaces">`consulNamespaces`</a> - [Enterprise Only] These settings manage the catalog sync's interaction with Consul namespaces (requires consul-ent v1.7+ and consul-k8s v0.12+). Also, `global.enableConsulNamespaces` must be true.
* <a name="v-synccatalog-consulnamespaces-consuldestinationnamespace" href="#v-synccatalog-consulnamespaces-consuldestinationnamespace">`consulDestinationNamespace`</a> (`string: "default"`) - Name of the Consul namespace to register all k8s services into. If the Consul namespace does not already exist, it will be created. This will be ignored if `mirroringK8S` is true.
* <a name="v-synccatalog-consulnamespaces-mirroringk8s" href="#v-synccatalog-consulnamespaces-mirroringk8s">`mirroringK8S`</a> (`bool: false`) - causes k8s services to be registered into a Consul namespace of the same name as their k8s namespace, optionally prefixed if `mirroringK8SPrefix` is set below. If the Consul namespace does not already exist, it will be created. Turning this on overrides the `consulDestinationNamespace` setting. `addK8SNamespaceSuffix` may no longer be needed if enabling this option.
* <a name="v-synccatalog-consulnamespaces-mirroringk8sprefix" href="#v-synccatalog-consulnamespaces-mirroringk8sprefix">`mirroringK8SPrefix`</a> (`string: ""`) - If `mirroringK8S` is set to true, `mirroringK8SPrefix` allows each Consul namespace to be given a prefix. For example, if `mirroringK8SPrefix` is set to `"k8s-"`, a service in the k8s `staging` namespace will be registered into the `k8s-staging` Consul namespace.
* <a name="v-synccatalog-consulPrefix" href="#v-synccatalog-consulPrefix">`consulPrefix`</a> (`string: ""`) - A prefix to prepend to all services registered in Consul from Kubernetes. This defaults to `""` where no prefix is prepended. Service names within Kubernetes remain unchanged. (Kubernetes -> Consul sync only)
* <a name="v-synccatalog-k8stag" href="#v-synccatalog-k8stag">`k8sTag`</a> (`string: null`) - An optional tag that is applied to all of the Kubernetes services that are synced into Consul. If nothing is set, this defaults to "k8s". (Kubernetes -> Consul sync only)
@ -305,6 +327,26 @@ to run the sync program.
namespace-label: label-value
```
* <a name="v-connectinject-k8sallownamespaces" href="#v-connectinject-k8sallownamespaces">`k8sAllowNamespaces`</a> - list of k8s namespaces to allow Connect sidecar injection in. If a k8s namespace is not included or is listed in `k8sDenyNamespaces`, pods in that k8s namespace will not be injected even if they are explicitly annotated. Use `["*"]` to automatically allow all k8s namespaces.
For example, `["namespace1", "namespace2"]` will only allow pods in the k8s namespaces `namespace1` and `namespace2` to have Connect sidecars injected and registered with Consul. All other k8s namespaces will be ignored.
Note: `k8sDenyNamespaces` takes precedence over values defined here and `namespaceSelector` takes precedence over both since it is applied first. `kube-system` and `kube-public` are never injected, even if included here. Requires consul-k8s v0.12+
* <a name="v-connectinject-k8sdenynamespaces" href="#v-connectinject-k8sdenynamespaces">`k8sDenyNamespaces`</a> - list of k8s namespaces that should not allow Connect sidecar injection. This list takes precedence over `k8sAllowNamespaces`. `*` is not supported because then nothing would be allowed to be injected.
For example, if `k8sAllowNamespaces` is `["*"]` and `k8sDenyNamespaces` is `["namespace1", "namespace2"]`, then all k8s namespaces besides `namespace1` and `namespace2` will be injected.
Note: `namespaceSelector` takes precedence over this since it is applied first. `kube-system` and `kube-public` are never injected. Requires consul-k8s v0.12+.
* <a name="v-connectinject-consulnamespaces" href="#v-connectinject-consulnamespaces">`consulNamespaces`</a> - [Enterprise Only] These settings manage the connect injector's interaction with Consul namespaces (requires consul-ent v1.7+ and consul-k8s v0.12+). Also, `global.enableConsulNamespaces` must be true.
* <a name="v-connectinject-consulnamespaces-consuldestinationnamespace" href="#v-connectinject-consulnamespaces-consuldestinationnamespace">`consulDestinationNamespace`</a> (`string: "default"`) - Name of the Consul namespace to register all k8s services into. If the Consul namespace does not already exist, it will be created. This will be ignored if `mirroringK8S` is true.
* <a name="v-connectinject-consulnamespaces-mirroringk8s" href="#v-connectinject-consulnamespaces-mirroringk8s">`mirroringK8S`</a> (`bool: false`) - causes k8s services to be registered into a Consul namespace of the same name as their k8s namespace, optionally prefixed if `mirroringK8SPrefix` is set below. If the Consul namespace does not already exist, it will be created. Turning this on overrides the `consulDestinationNamespace` setting.
* <a name="v-connectinject-consulnamespaces-mirroringk8sprefix" href="#v-connectinject-consulnamespaces-mirroringk8sprefix">`mirroringK8SPrefix`</a> (`string: ""`) - If `mirroringK8S` is set to true, `mirroringK8SPrefix` allows each Consul namespace to be given a prefix. For example, if `mirroringK8SPrefix` is set to `"k8s-"`, a service in the k8s `staging` namespace will be registered into the `k8s-staging` Consul namespace.
* <a name="v-connectinject-certs" href="#v-connectinject-certs">`certs`</a> - The certs section configures how the webhook TLS certs are configured. These are the TLS certs for the Kube apiserver communicating to the webhook. By default, the injector will generate and manage its own certs, but this requires the ability for the injector to update its own `MutatingWebhookConfiguration`. In a production environment, custom certs should probably be used. Configure the values below to enable this.
- <a name="v-connectinject-certs-secretname" href="#v-connectinject-certs-secretname">`secretName`</a> (`string: null`) -
@ -329,11 +371,18 @@ to run the sync program.
* <a name="v-connectinject-acl-bindingrule-selector" href="#v-connectinject-acl-bindingrule-selector">`aclBindingRuleSelector`</a> (`string: "serviceaccount.name!=default"`) -
A [selector](/docs/acl/acl-auth-methods.html#binding-rules) for restricting automatic injection to only matching services based on
their associated service account. By default, services using the `default` Kubernetes service account will not have a proxy injected.
* <a name="v-connectinject-aclinjecttoken" href="#v-connectinject-aclinjecttoken">`aclInjectToken`</a> - Refers to a Kubernetes secret that you have created that contains an ACL token for your Consul cluster which allows the Connect injector the correct permissions. This is only needed if Consul namespaces and ACLs are enabled on the Consul cluster and you are not setting `global.bootstrapACLs` to `true`. This token needs to have `operator = "write"` privileges so that it can create namespaces.
- <a name="v-connectinject-aclinjecttoken-secretname" href="#v-synccatalog-aclinjecttoken-secretname">secretName </a>`(string: null)` - The name of the Kubernetes secret.
- <a name="v-connectinject-aclinjecttoken-secretkey" href="#v-synccatalog-aclinjecttoken-secretkey">secretKey </a>`(string: null)` - The key within the Kubernetes secret that holds the acl token.
* <a name="v-connectinject-centralconfig" href="#v-connectinject-centralconfig">`centralConfig`</a> - Values that configure
Consul's [central configuration](/docs/agent/config_entries.html) feature (requires Consul v1.5+ and consul-k8s v0.8.1+).
- <a name="v-connectinject-centralconfig-enabled" href="#v-connectinject-centralconfig-enabled">`enabled`</a> (`boolean: false`) -
- <a name="v-connectinject-centralconfig-enabled" href="#v-connectinject-centralconfig-enabled">`enabled`</a> (`boolean: true`) -
Turns on the central configuration feature. Pods that have a Connect proxy injected will have their service
automatically registered in this central configuration.

View File

@ -13,7 +13,7 @@ services are available to Consul agents and services in Consul can be available
as first-class Kubernetes services. This functionality is provided by the
[consul-k8s project](https://github.com/hashicorp/consul-k8s) and can be
automatically installed and configured using the
[Consul Helm chart](/docs/platform/k8s/helm.html).
[Consul Helm chart](/docs/platform/k8s/run.html).
**Why sync Kubernetes services to Consul?** Kubernetes services synced to the
Consul catalog enable Kubernetes services to be accessed by any node that
@ -38,10 +38,10 @@ the Kubernetes cluster is generally easier since it is automated using the
The Consul server cluster can run either in or out of a Kubernetes cluster.
The Consul server cluster does not need to be running on the same machine
or same platform as the sync process. The sync process needs to be configured
with the address to the Consul cluster as well as any additional access
with the address to a Consul agent as well as any additional access
information such as ACL tokens.
To install the sync, enable the catalog sync feature using
To install the sync process, enable the catalog sync feature using
[Helm values](/docs/platform/k8s/helm.html#configuration-values-) and
upgrade the installation using `helm upgrade` for existing installs or
`helm install` for a fresh install.
@ -159,10 +159,17 @@ to `false` in the Helm chart values file.
### Sync Enable/Disable
By default, all valid services (as explained above) are synced. This default can
be changed using the [configuration](/docs/platform/k8s/helm.html#v-synccatalog-default).
Syncing can also be explicitly enabled or disabled using an
annotation:
By default, all valid service types (as explained above) are synced from every Kubernetes
namespace (except for `kube-system` and `kube-public`).
If you wish to only sync specific services via annotation, set the default to `false`:
```yaml
syncCatalog:
enabled: true
default: false
```
And explicitly enable syncing specific services via the `consul.hashicorp.com/service-sync` annotation:
```yaml
kind: Service
@ -170,9 +177,50 @@ apiVersion: v1
metadata:
name: my-service
annotations:
"consul.hashicorp.com/service-sync": "false"
"consul.hashicorp.com/service-sync": "true"
```
-> **NOTE:** If the annotation is set to `false` when the default sync is `true`, the service will **not** be synced.
You can allow or deny syncing from specific Kubernetes namespaces by setting the
`k8sAllowNamespaces` and `k8sDenyNamespaces` keys:
```yaml
syncCatalog:
enabled: true
default: true
k8sAllowNamespaces: ["*"]
k8sDenyNamespaces: ["kube-system", "kube-public"]
```
In the default configuration (shown above), services from all namespaces except for
`kube-system` and `kube-public` will be synced.
If you wish to only sync from specific namespaces, you can list only those
namespaces in the `k8sAllowNamespaces` key:
```yaml
syncCatalog:
enabled: true
default: true
k8sAllowNamespaces: ["my-ns-1", "my-ns-2"]
k8sDenyNamespaces: []
```
If you wish to sync from every namespace *except* specific namespaces, you can
use `*` in the allow list and then specify the non-syncing namespaces in the deny list:
```yaml
syncCatalog:
enabled: true
default: true
k8sAllowNamespaces: ["*"]
k8sDenyNamespaces: ["no-sync-ns-1", "no-sync-ns-2"]
```
-> **NOTE:** The deny list takes precedence over the allow list. If a namespace
is listed in both lists, it will **not** be synced.
### Service Name
When a Kubernetes service is synced to Consul, the name of the service in Consul
@ -252,6 +300,72 @@ metadata:
"consul.hashicorp.com/service-meta-KEY": "value"
```
### Consul Enterprise Namespaces
Consul Enterprise supports Consul namespaces. These can be used when syncing
from Kubernetes to Consul (although not vice-versa).
There are three options available:
1. **Single Destination Namespace** Sync all Kubernetes services, regardless of namespace,
into the same Consul namespace.
This can be configured with:
```yaml
global:
enableConsulNamespaces: true
syncCatalog:
enabled: true
consulNamespaces:
consulDestinationNamespace: "my-consul-ns"
```
1. **Mirror Namespaces** - Each Kubernetes service will be synced to a Consul namespace with the same name as its Kubernetes namespace.
For example, service `foo` in Kubernetes namespace `ns-1` will be synced to the Consul namespace `ns-1`.
If a mirrored namespace does not exist in Consul, it will be created.
This can be configured with:
```yaml
global:
enableConsulNamespaces: true
syncCatalog:
enabled: true
consulNamespaces:
mirroringK8S: true
addK8SNamespaceSuffix: false
```
1. **Mirror Namespaces With Prefix** - Each Kubernetes service will be synced to a Consul namespace with the same name as its Kubernetes
namespace **with a prefix**.
For example, given a prefix `k8s-`, service `foo` in Kubernetes namespace `ns-1` will be synced to the Consul namespace `k8s-ns-1`.
This can be configured with:
```yaml
global:
enableConsulNamespaces: true
syncCatalog:
enabled: true
consulNamespaces:
mirroringK8S: true
mirroringK8SPrefix: "k8s-"
addK8SNamespaceSuffix: false
```
-> Note that in both mirroring examples we're setting `addK8SNamespaceSuffix: false`. If set to `true`
(the default), the Kubernetes namespace will be added as a suffix to each
Consul service name. For example Kubernetes service `foo` in namespace `k8s-ns`
would be registered into Consul with the name `foo-k8s-ns`.
This is useful when syncing from multiple Kubernetes namespaces to
a single consul namespace but is likely something you'll want turned off
when mirroring namespaces since services won't overlap with services from
other namespaces.
## Consul to Kubernetes
This syncs Consul services into first-class Kubernetes services.