--- layout: docs page_title: Consul Custom Resource Definitions description: >- Consul supports managing configuration entries via Kubernetes Custom Resources. These custom resource can be used to manage the configuration for workloads deployed within the cluster. --- # Custom Resource Definitions -> This feature requires consul-helm >= 0.28.0, consul-k8s >= 0.22.0 and consul >= 1.8.4. We support managing Consul [configuration entries](/docs/agent/config-entries) via Kubernetes Custom Resources. Configuration entries are used to provide cluster-wide defaults for the service mesh. We currently support the follow configuration entry kinds: - [`ProxyDefaults`](/docs/connect/config-entries/proxy-defaults) - [`ServiceDefaults`](/docs/connect/config-entries/service-defaults) - [`ServiceSplitter`](/docs/connect/config-entries/service-splitter) - [`ServiceRouter`](/docs/connect/config-entries/service-router) - [`ServiceResolver`](/docs/connect/config-entries/service-resolver) - [`ServiceIntentions`](/docs/connect/config-entries/service-intentions) (requires Consul >= 1.9.0) - [`IngressGateway`](/docs/connect/config-entries/ingress-gateway) - [`TerminatingGateway`](/docs/connect/config-entries/terminating-gateway) ## Installation Ensure you have at least version `0.28.0` of the helm chart: ```shell-session $ helm search repo hashicorp/consul NAME CHART VERSION APP VERSION DESCRIPTION hashicorp/consul 0.28.0 1.9.1 Official HashiCorp Consul Chart ``` If you don't have `0.28.0`, you will need to update your helm repository cache: ```shell-session $ helm repo update Hang tight while we grab the latest from your chart repositories... ...Successfully got an update from the "hashicorp" chart repository Update Complete. ⎈Happy Helming!⎈ ``` Next, you must configure consul-helm via your `values.yaml` to install the custom resource definitions and enable the controller that acts on them: ```yaml global: name: consul controller: enabled: true connectInject: enabled: true ``` Note that: 1. `controller.enabled: true` installs the CRDs and enables the controller. 1. Configuration entries are used to configure Consul service mesh so it's also expected that `connectInject` will be enabled. See [Install with Helm Chart](/docs/k8s/installation/install) for further installation instructions. ## Upgrading An Existing Cluster to CRDs If you have an existing Consul cluster running on Kubernetes you may need to perform extra steps to migrate to CRDs. See [Upgrade An Existing Cluster to CRDs](/docs/k8s/crds/upgrade-to-crds) for full instructions. ## Usage Once installed, you can use `kubectl` to create and manage Consul's configuration entries. ### Create You can create configuration entries via `kubectl apply`. ```shell-session $ cat < protocol: tcp servicedefaults.consul.hashicorp.com/foo edited ``` You can then use `kubectl get` to ensure the change was synced to Consul: ```shell-session $ kubectl get servicedefaults foo NAME SYNCED foo True ``` ### Delete You can use `kubectl delete [kind] [name]` to delete the configuration entry: ```shell-session $ kubectl delete servicedefaults foo servicedefaults.consul.hashicorp.com "foo" deleted ``` You can then use `kubectl get` to ensure the configuration entry was deleted: ```shell-session $ kubectl get servicedefaults foo Error from server (NotFound): servicedefaults.consul.hashicorp.com "foo" not found ``` #### Delete Hanging If running `kubectl delete` hangs without exiting, there may be a dependent configuration entry registered with Consul that prevents the target configuration entry from being deleted. For example, if you set the protocol of your service to `http` via `ServiceDefaults` and then create a `ServiceSplitter`, you won't be able to delete the `ServiceDefaults`. This is because by deleting the `ServiceDefaults` config, you are setting the protocol back to the default which is `tcp`. Since `ServiceSplitter` requires that the service has an `http` protocol, Consul will not allow the `ServiceDefaults` to be deleted since that would put Consul into a broken state. In order to delete the `ServiceDefaults` config, you would need to first delete the `ServiceSplitter`. ## Kubernetes Namespaces ### Consul OSS Consul Open Source (Consul OSS) ignores Kubernetes namespaces and registers all services into the same global Consul registry based on their names. For example, service `web` in Kubernetes namespace `web-ns` and service `admin` in Kubernetes namespace `admin-ns` will be registered into Consul as `web` and `admin` with the Kubernetes source namespace ignored. When creating custom resources to configure these services, the namespace of the custom resource is also ignored. For example, you can create a `ServiceDefaults` custom resource for service `web` in the Kubernetes namespace `admin-ns` even though the `web` service is actually running in the `web-ns` namespace (although this is not recommended): ```yaml apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceDefaults metadata: name: web namespace: admin-ns spec: protocol: http --- apiVersion: apps/v1 kind: Deployment metadata: name: web namespace: web-ns spec: ... ``` ~> **NOTE:** If two custom resources of the same kind **and** the same name are attempted to be created in different Kubernetes namespaces, the last one created will not be synced. #### ServiceIntentions Special Case `ServiceIntentions` are different from the other custom resources because the name of the resource doesn't matter. For other resources, the name of the resource determines which service it configures. For example, this resource configures the service `web`: ```yaml apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceDefaults metadata: name: web spec: protocol: http ``` For `ServiceIntentions`, because we need to support the ability to create wildcard intentions (e.g. `foo => * (allow)` meaning that `foo` can talk to **any** service), and because `*` is not a valid Kubernetes resource name, we instead use the field `spec.destination.name` to configure the destination service for the intention: ```yaml # foo => * (allow) apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceIntentions metadata: name: name-does-not-matter spec: destination: name: '*' sources: - name: foo action: allow --- # foo => web (allow) apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceIntentions metadata: name: name-does-not-matter spec: destination: name: web sources: - name: foo action: allow ``` ~> **NOTE:** If two `ServiceIntentions` resources set the same `spec.destination.name`, the last one created will not be synced. ### Consul Enterprise Consul Enterprise supports multiple configurations for how Kubernetes namespaces are mapped to Consul namespaces. The Consul namespace that the custom resource is registered into depends on the configuration being used but in general, you should create your custom resources in the same Kubernetes namespace as the service they're configuring and everything will work as expected. The details on each configuration are: 1. **Mirroring** - The Kubernetes namespace will be "mirrored" into Consul, i.e. service `web` in Kubernetes namespace `web-ns` will be registered as service `web` in the Consul namespace `web-ns`. In the same vein, a `ServiceDefaults` custom resource with name `web` in Kubernetes namespace `web-ns` will configure that same service. This is configured via [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces): ```yaml global: name: consul enableConsulNamespaces: true image: hashicorp/consul-enterprise:-ent connectInject: consulNamespaces: mirroringK8S: true ``` 1. **Mirroring with prefix** - The Kubernetes namespace will be "mirrored" into Consul with a prefix added to the Consul namespace, i.e. if the prefix is `k8s-` then service `web` in Kubernetes namespace `web-ns` will be registered as service `web` in the Consul namespace `k8s-web-ns`. In the same vein, a `ServiceDefaults` custom resource with name `web` in Kubernetes namespace `web-ns` will configure that same service. This is configured via [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces): ```yaml global: name: consul enableConsulNamespaces: true image: hashicorp/consul-enterprise:-ent connectInject: consulNamespaces: mirroringK8S: true mirroringK8SPrefix: k8s- ``` 1. **Single destination namespace** - The Kubernetes namespace is ignored and all services will be registered into the same Consul namespace, i.e. if the destination Consul namespace is `my-ns` then service `web` in Kubernetes namespace `web-ns` will be registered as service `web` in Consul namespace `my-ns`. In this configuration, the Kubernetes namespace of the custom resource is ignored. For example, a `ServiceDefaults` custom resource with the name `web` in Kubernetes namespace `admin-ns` will configure the service with name `web` even though that service is running in Kubernetes namespace `web-ns` because the `ServiceDefaults` resource ends up registered into the same Consul namespace `my-ns`. This is configured via [`connectInject.consulNamespaces`](/docs/k8s/helm#v-connectinject-consulnamespaces): ```yaml global: name: consul enableConsulNamespaces: true image: hashicorp/consul-enterprise:-ent connectInject: consulNamespaces: consulDestinationNamespace: 'my-ns' ``` ~> **NOTE:** In this configuration, if two custom resources of the same kind **and** the same name are attempted to be created in two Kubernetes namespaces, the last one created will not be synced. #### ServiceIntentions Special Case (Enterprise) `ServiceIntentions` are different from the other custom resources because the name of the resource doesn't matter. For other resources, the name of the resource determines which service it configures. For example, this resource configures the service `web`: ```yaml apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceDefaults metadata: name: web spec: protocol: http ``` For `ServiceIntentions`, because we need to support the ability to create wildcard intentions (e.g. `foo => * (allow)` meaning that `foo` can talk to **any** service), and because `*` is not a valid Kubernetes resource name, we instead use the field `spec.destination.name` to configure the destination service for the intention: ```yaml # foo => * (allow) apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceIntentions metadata: name: name-does-not-matter spec: destination: name: '*' sources: - name: foo action: allow --- # foo => web (allow) apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceIntentions metadata: name: name-does-not-matter spec: destination: name: web sources: - name: foo action: allow ``` In addition, we support the field `spec.destination.namespace` to configure the destination service's Consul namespace. If `spec.destination.namespace` is empty, then the Consul namespace used will be the same as the other config entries as outlined above.