mirror of https://github.com/status-im/consul.git
383 lines
12 KiB
Plaintext
383 lines
12 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: Consul Custom Resource Definitions
|
|
description: >-
|
|
Consul supports managing configuration entries via Kubernetes Custom Resources.
|
|
These custom resource can be used to manage the configuration for workloads
|
|
deployed within the cluster.
|
|
---
|
|
|
|
# Custom Resource Definitions
|
|
|
|
-> This feature requires consul-helm >= 0.28.0, consul-k8s >= 0.22.0 and consul >= 1.8.4.
|
|
|
|
We support managing Consul [configuration entries](/docs/agent/config-entries)
|
|
via Kubernetes Custom Resources. Configuration entries are used to provide
|
|
cluster-wide defaults for the service mesh.
|
|
|
|
We currently support the follow configuration entry kinds:
|
|
|
|
- [`ProxyDefaults`](/docs/connect/config-entries/proxy-defaults)
|
|
- [`ServiceDefaults`](/docs/connect/config-entries/service-defaults)
|
|
- [`ServiceSplitter`](/docs/connect/config-entries/service-splitter)
|
|
- [`ServiceRouter`](/docs/connect/config-entries/service-router)
|
|
- [`ServiceResolver`](/docs/connect/config-entries/service-resolver)
|
|
- [`ServiceIntentions`](/docs/connect/config-entries/service-intentions) (requires Consul >= 1.9.0)
|
|
- [`IngressGateway`](/docs/connect/config-entries/ingress-gateway)
|
|
- [`TerminatingGateway`](/docs/connect/config-entries/terminating-gateway)
|
|
|
|
## Installation
|
|
|
|
Ensure you have at least version `0.28.0` of the helm chart:
|
|
|
|
```shell-session
|
|
$ helm search repo hashicorp/consul
|
|
NAME CHART VERSION APP VERSION DESCRIPTION
|
|
hashicorp/consul 0.28.0 1.9.1 Official HashiCorp Consul Chart
|
|
```
|
|
|
|
If you don't have `0.28.0`, you will need to update your helm repository cache:
|
|
|
|
```shell-session
|
|
$ helm repo update
|
|
Hang tight while we grab the latest from your chart repositories...
|
|
...Successfully got an update from the "hashicorp" chart repository
|
|
Update Complete. ⎈Happy Helming!⎈
|
|
```
|
|
|
|
Next, you must configure consul-helm via your `values.yaml` to install the custom resource definitions
|
|
and enable the controller that acts on them:
|
|
|
|
```yaml
|
|
global:
|
|
name: consul
|
|
|
|
controller:
|
|
enabled: true
|
|
|
|
connectInject:
|
|
enabled: true
|
|
```
|
|
|
|
Note that:
|
|
|
|
1. `controller.enabled: true` installs the CRDs and enables the controller.
|
|
1. Configuration entries are used to configure Consul service mesh so it's also
|
|
expected that `connectInject` will be enabled.
|
|
|
|
See [Install with Helm Chart](/docs/k8s/installation/install) for further installation
|
|
instructions.
|
|
|
|
## Upgrading An Existing Cluster to CRDs
|
|
|
|
If you have an existing Consul cluster running on Kubernetes you may need to perform
|
|
extra steps to migrate to CRDs. See [Upgrade An Existing Cluster to CRDs](/docs/k8s/crds/upgrade-to-crds) for full instructions.
|
|
|
|
## Usage
|
|
|
|
Once installed, you can use `kubectl` to create and manage Consul's configuration entries.
|
|
|
|
### Create
|
|
|
|
You can create configuration entries via `kubectl apply`.
|
|
|
|
```shell-session
|
|
$ cat <<EOF | kubectl apply -f -
|
|
apiVersion: consul.hashicorp.com/v1alpha1
|
|
kind: ServiceDefaults
|
|
metadata:
|
|
name: foo
|
|
spec:
|
|
protocol: "http"
|
|
EOF
|
|
|
|
servicedefaults.consul.hashicorp.com/foo created
|
|
```
|
|
|
|
See [Configuration Entries](/docs/agent/config-entries) for detailed schema documentation.
|
|
|
|
### Get
|
|
|
|
You can use `kubectl get [kind] [name]` to get the status of the configuration entry:
|
|
|
|
```shell-session
|
|
$ kubectl get servicedefaults foo
|
|
NAME SYNCED
|
|
foo True
|
|
```
|
|
|
|
The `SYNCED` status shows whether the configuration entry was successfully created
|
|
in Consul.
|
|
|
|
### Describe
|
|
|
|
You can use `kubectl describe [kind] [name]` to investigate the status of the
|
|
configuration entry. If `SYNCED` is false, the status will contain the reason
|
|
why.
|
|
|
|
```shell-session
|
|
$ kubectl describe servicedefaults foo
|
|
Status:
|
|
Conditions:
|
|
Last Transition Time: 2020-10-09T21:15:50Z
|
|
Status: True
|
|
Type: Synced
|
|
```
|
|
|
|
### Edit
|
|
|
|
You can use `kubectl edit [kind] [name]` to edit the configuration entry:
|
|
|
|
```shell
|
|
$ kubectl edit servicedefaults foo
|
|
# change protocol: http => protocol: tcp
|
|
servicedefaults.consul.hashicorp.com/foo edited
|
|
```
|
|
|
|
You can then use `kubectl get` to ensure the change was synced to Consul:
|
|
|
|
```shell-session
|
|
$ kubectl get servicedefaults foo
|
|
NAME SYNCED
|
|
foo True
|
|
```
|
|
|
|
### Delete
|
|
|
|
You can use `kubectl delete [kind] [name]` to delete the configuration entry:
|
|
|
|
```shell-session
|
|
$ kubectl delete servicedefaults foo
|
|
servicedefaults.consul.hashicorp.com "foo" deleted
|
|
```
|
|
|
|
You can then use `kubectl get` to ensure the configuration entry was deleted:
|
|
|
|
```shell-session
|
|
$ kubectl get servicedefaults foo
|
|
Error from server (NotFound): servicedefaults.consul.hashicorp.com "foo" not found
|
|
```
|
|
|
|
#### Delete Hanging
|
|
|
|
If running `kubectl delete` hangs without exiting, there may be
|
|
a dependent configuration entry registered with Consul that prevents the target configuration entry from being
|
|
deleted. For example, if you set the protocol of your service to `http` via `ServiceDefaults` and then
|
|
create a `ServiceSplitter`, you won't be able to delete the `ServiceDefaults`.
|
|
|
|
This is because by deleting the `ServiceDefaults` config, you are setting the
|
|
protocol back to the default which is `tcp`. Since `ServiceSplitter` requires
|
|
that the service has an `http` protocol, Consul will not allow the `ServiceDefaults`
|
|
to be deleted since that would put Consul into a broken state.
|
|
|
|
In order to delete the `ServiceDefaults` config, you would need to first delete
|
|
the `ServiceSplitter`.
|
|
|
|
## Kubernetes Namespaces
|
|
|
|
### Consul OSS
|
|
|
|
Consul Open Source (Consul OSS) ignores Kubernetes namespaces and registers all services into the same
|
|
global Consul registry based on their names. For example, service `web` in Kubernetes namespace
|
|
`web-ns` and service `admin` in Kubernetes namespace `admin-ns` will be registered into
|
|
Consul as `web` and `admin` with the Kubernetes source namespace ignored.
|
|
|
|
When creating custom resources to configure these services, the namespace of the
|
|
custom resource is also ignored. For example, you can create a `ServiceDefaults`
|
|
custom resource for service `web` in the Kubernetes namespace `admin-ns` even though
|
|
the `web` service is actually running in the `web-ns` namespace (although this is not recommended):
|
|
|
|
```yaml
|
|
apiVersion: consul.hashicorp.com/v1alpha1
|
|
kind: ServiceDefaults
|
|
metadata:
|
|
name: web
|
|
namespace: admin-ns
|
|
spec:
|
|
protocol: http
|
|
---
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: web
|
|
namespace: web-ns
|
|
spec: ...
|
|
```
|
|
|
|
~> **NOTE:** If two custom resources of the same kind **and** the same name are attempted to
|
|
be created in different Kubernetes namespaces, the last one created will not be synced.
|
|
|
|
#### ServiceIntentions Special Case
|
|
|
|
`ServiceIntentions` are different from the other custom resources because the
|
|
name of the resource doesn't matter. For other resources, the name of the resource
|
|
determines which service it configures. For example, this resource configures
|
|
the service `web`:
|
|
|
|
```yaml
|
|
apiVersion: consul.hashicorp.com/v1alpha1
|
|
kind: ServiceDefaults
|
|
metadata:
|
|
name: web
|
|
spec:
|
|
protocol: http
|
|
```
|
|
|
|
For `ServiceIntentions`, because we need to support the ability to create
|
|
wildcard intentions (e.g. `foo => * (allow)` meaning that `foo` can talk to **any** service),
|
|
and because `*` is not a valid Kubernetes resource name, we instead use the field `spec.destination.name`
|
|
to configure the destination service for the intention:
|
|
|
|
```yaml
|
|
# foo => * (allow)
|
|
apiVersion: consul.hashicorp.com/v1alpha1
|
|
kind: ServiceIntentions
|
|
metadata:
|
|
name: name-does-not-matter
|
|
spec:
|
|
destination:
|
|
name: '*'
|
|
sources:
|
|
- name: foo
|
|
action: allow
|
|
---
|
|
# foo => web (allow)
|
|
apiVersion: consul.hashicorp.com/v1alpha1
|
|
kind: ServiceIntentions
|
|
metadata:
|
|
name: name-does-not-matter
|
|
spec:
|
|
destination:
|
|
name: web
|
|
sources:
|
|
- name: foo
|
|
action: allow
|
|
```
|
|
|
|
~> **NOTE:** If two `ServiceIntentions` resources set the same `spec.destination.name`, the
|
|
last one created will not be synced.
|
|
|
|
### Consul Enterprise <EnterpriseAlert inline />
|
|
|
|
Consul Enterprise supports multiple configurations for how Kubernetes namespaces are mapped
|
|
to Consul namespaces. The Consul namespace that the custom resource is registered
|
|
into depends on the configuration being used but in general, you should create your
|
|
custom resources in the same Kubernetes namespace as the service they're configuring and
|
|
everything will work as expected.
|
|
|
|
The details on each configuration are:
|
|
|
|
1. **Mirroring** - The Kubernetes namespace will be "mirrored" into Consul, i.e.
|
|
service `web` in Kubernetes namespace `web-ns` will be registered as service `web`
|
|
in the Consul namespace `web-ns`. In the same vein, a `ServiceDefaults` custom resource with
|
|
name `web` in Kubernetes namespace `web-ns` will configure that same service.
|
|
|
|
This is configured via [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces):
|
|
|
|
```yaml
|
|
global:
|
|
name: consul
|
|
enableConsulNamespaces: true
|
|
image: hashicorp/consul-enterprise:<tag>-ent
|
|
connectInject:
|
|
consulNamespaces:
|
|
mirroringK8S: true
|
|
```
|
|
|
|
1. **Mirroring with prefix** - The Kubernetes namespace will be "mirrored" into Consul
|
|
with a prefix added to the Consul namespace, i.e.
|
|
if the prefix is `k8s-` then service `web` in Kubernetes namespace `web-ns` will be registered as service `web`
|
|
in the Consul namespace `k8s-web-ns`. In the same vein, a `ServiceDefaults` custom resource with
|
|
name `web` in Kubernetes namespace `web-ns` will configure that same service.
|
|
|
|
This is configured via [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces):
|
|
|
|
```yaml
|
|
global:
|
|
name: consul
|
|
enableConsulNamespaces: true
|
|
image: hashicorp/consul-enterprise:<tag>-ent
|
|
connectInject:
|
|
consulNamespaces:
|
|
mirroringK8S: true
|
|
mirroringK8SPrefix: k8s-
|
|
```
|
|
|
|
1. **Single destination namespace** - The Kubernetes namespace is ignored and all services
|
|
will be registered into the same Consul namespace, i.e. if the destination Consul
|
|
namespace is `my-ns` then service `web` in Kubernetes namespace `web-ns` will
|
|
be registered as service `web` in Consul namespace `my-ns`.
|
|
|
|
In this configuration, the Kubernetes namespace of the custom resource is ignored.
|
|
For example, a `ServiceDefaults` custom resource with the name `web` in Kubernetes
|
|
namespace `admin-ns` will configure the service with name `web` even though that
|
|
service is running in Kubernetes namespace `web-ns` because the `ServiceDefaults`
|
|
resource ends up registered into the same Consul namespace `my-ns`.
|
|
|
|
This is configured via [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces):
|
|
|
|
```yaml
|
|
global:
|
|
name: consul
|
|
enableConsulNamespaces: true
|
|
image: hashicorp/consul-enterprise:<tag>-ent
|
|
connectInject:
|
|
consulNamespaces:
|
|
consulDestinationNamespace: 'my-ns'
|
|
```
|
|
|
|
~> **NOTE:** In this configuration, if two custom resources of the same kind **and** the same name are attempted to
|
|
be created in two Kubernetes namespaces, the last one created will not be synced.
|
|
|
|
#### ServiceIntentions Special Case (Enterprise)
|
|
|
|
`ServiceIntentions` are different from the other custom resources because the
|
|
name of the resource doesn't matter. For other resources, the name of the resource
|
|
determines which service it configures. For example, this resource configures
|
|
the service `web`:
|
|
|
|
```yaml
|
|
apiVersion: consul.hashicorp.com/v1alpha1
|
|
kind: ServiceDefaults
|
|
metadata:
|
|
name: web
|
|
spec:
|
|
protocol: http
|
|
```
|
|
|
|
For `ServiceIntentions`, because we need to support the ability to create
|
|
wildcard intentions (e.g. `foo => * (allow)` meaning that `foo` can talk to **any** service),
|
|
and because `*` is not a valid Kubernetes resource name, we instead use the field `spec.destination.name`
|
|
to configure the destination service for the intention:
|
|
|
|
```yaml
|
|
# foo => * (allow)
|
|
apiVersion: consul.hashicorp.com/v1alpha1
|
|
kind: ServiceIntentions
|
|
metadata:
|
|
name: name-does-not-matter
|
|
spec:
|
|
destination:
|
|
name: '*'
|
|
sources:
|
|
- name: foo
|
|
action: allow
|
|
---
|
|
# foo => web (allow)
|
|
apiVersion: consul.hashicorp.com/v1alpha1
|
|
kind: ServiceIntentions
|
|
metadata:
|
|
name: name-does-not-matter
|
|
spec:
|
|
destination:
|
|
name: web
|
|
sources:
|
|
- name: foo
|
|
action: allow
|
|
```
|
|
|
|
In addition, we support the field `spec.destination.namespace` to configure
|
|
the destination service's Consul namespace. If `spec.destination.namespace`
|
|
is empty, then the Consul namespace used will be the same as the other
|
|
config entries as outlined above.
|