diff --git a/website/content/docs/k8s/connect/connect-ca-provider.mdx b/website/content/docs/k8s/connect/connect-ca-provider.mdx index 9eaad0bd04..baddd34c3a 100644 --- a/website/content/docs/k8s/connect/connect-ca-provider.mdx +++ b/website/content/docs/k8s/connect/connect-ca-provider.mdx @@ -6,7 +6,7 @@ description: Configuring a Connect CA Provider # Configuring a Connect CA Provider -~> NOTE: Instructions below should only be used for initially bootstrapping a cluster. +~> **NOTE:** The instructions below should only be used for initially bootstrapping a cluster with **Consul K8s 0.38.0+.** To update the Connect CA provider on an existing cluster or to update any properties, such as tokens, of the CA provider, please use the [Update CA Configuration Endpoint](/api/connect/ca#update-ca-configuration). @@ -14,17 +14,22 @@ Consul has support for different certificate authority (CA) providers to be used Please see [Connect Certificate Management](/docs/connect/ca) for the information on the providers we currently support. -Generally, to configure a provider via the Consul Helm chart, you need to follow three steps: +To configure a provider via the Consul Helm chart, you need to follow three steps: 1. Create a configuration file containing your provider information. 1. Create a Kubernetes secret containing the configuration file. 1. Reference the Kubernetes secret in the [`server.extraVolumes`](/docs/k8s/helm#v-server-extravolumes) value in the Helm chart. -Below we will go over this process for configuring Vault as the Connect CA. +To configure the Vault Connect Provider please see [Vault as the Service Mesh Certificate Provider on Kubernetes](/docs/k8s/installation/vault/connect-ca). + + +~> **NOTE:** The following instructions are only valid for Consul-k8s 0.37.0 and prior. + +Below we will go over the process for configuring Vault as the Connect CA. However, other providers can be configured similarly by providing the appropriate `ca_config` and `ca_provider` values for the provider you're using. -## Configuring Vault as a Connect CA +## Configuring Vault as a Connect CA (Consul K8s 0.37.0 and earlier) -> **NOTE:** If using Vault as your Connect CA, it's highly recommended to run a Consul version >= 1.8.5 that supports token auto-renewal. With this feature, if the Vault token is [renewable](https://www.vaultproject.io/api-docs/auth/token#renewable) @@ -89,27 +94,23 @@ $ kubectl create secret generic vault-config --from-file=config=vault-config.jso We will provide this secret and the Vault CA secret, to the Consul server via the `server.extraVolumes` Helm value. - - -```yaml -global: - name: consul -server: - extraVolumes: - - type: secret - name: vault-config - load: true - items: - - key: config - path: vault-config.json - - type: secret - name: vault-ca - load: false -connectInject: - enabled: true -``` - - + ```yaml + global: + name: consul + server: + extraVolumes: + - type: secret + name: vault-config + load: true + items: + - key: config + path: vault-config.json + - type: secret + name: vault-ca + load: false + connectInject: + enabled: true + ``` Finally, [install](/docs/k8s/installation/install#installing-consul) the Helm chart using the above config file: diff --git a/website/content/docs/k8s/installation/vault/connect-ca.mdx b/website/content/docs/k8s/installation/vault/connect-ca.mdx new file mode 100644 index 0000000000..693c0b1953 --- /dev/null +++ b/website/content/docs/k8s/installation/vault/connect-ca.mdx @@ -0,0 +1,72 @@ +--- +layout: docs +page_title: Vault as the Service Mesh Certificate Provider on Kubernetes +description: >- + Using Vault as the provider for the Service Mesh certificates on Kubernetes. +--- + +# Vault as the Service Mesh Certificate Provider on Kubernetes + +-> **Note:** This feature requires Consul 1.11 or higher. As of v1.11, +Consul allows using Kubernetes auth methods to configure Connect CA. +This allows for automatic token rotation once the renewal is no longer possible. + +To configure [Vault as the provider](/docs/connect/ca/vault) for the Consul service certificates, +you will first need to decide on the type of policy that is suitable for you. +To see the permissions that Consul would need in Vault, please see [Vault ACL policies](/docs/connect/ca/vault#vault-acl-policies) +documentation. + +Once you have a policy, you will need to link that policy to the Consul server service account. + +```shell-session +vault write auth/kubernetes/role/consul-server \ + bound_service_account_names= \ + bound_service_account_namespaces= \ + policies= \ + ttl=1h +``` + +To find out the service account name of the Consul server, +you can run: + +```shell-session +helm template --release-name -s templates/server-serviceaccount.yaml hashicorp/consul +``` + +Now we can configure the Consul Helm chart to use Vault as the Connect CA provider: + +```yaml +global: + secretsBackend: + vault: + enabled: true + consulServerRole: consul-server + consulClientRole: consul-client + consulCARole: consul-ca + connectCA: + address: + rootPKIPath: + intermediatePKIPath: + ca: + secretName: +``` + +The `address` you provide to the `connectCA` configuration can be a Kubernetes DNS +address if the Vault cluster is running the same Kubernetes cluster. +The `rootPKIPath` and `intermediatePKIPath` should be the same as the ones +defined in your Connect CA policy. Behind the scenes, Consul will authenticate to Vault using a Kubernetes +service account using the [Kubernetes auth method](https://www.vaultproject.io/docs/auth/kubernetes) and will use the Vault token for any API calls to Vault. If the Vault token can not be renewed, Consul will re-authenticate to +generate a new Vault token. + +The `vaultCASecret` is the Kubernetes secret that stores the CA Certificate that is used for Vault communication. To provide a CA, you first need to create a Kubernetes secret containing the CA. For example, you may create a secret with the Vault CA like so: + +``` +kubectl create secret generic vault-ca --from-file vault.ca=/path/to/your/vault/ca +``` + +### Secondary Datacenters + +To configure Vault as the Connect CA in secondary datacenters, you need to make sure that the Root CA path is the same, +but the intermediate is different for each datacenter. In the `connectCA` Helm configuration for a secondary datacenter, +you can specify a `intermediatePKIPath` that is, for example, prefixed with the datacenter +for which this configuration is intended (e.g. `dc2/connect-intermediate`). diff --git a/website/content/docs/k8s/installation/vault/gossip.mdx b/website/content/docs/k8s/installation/vault/gossip.mdx new file mode 100644 index 0000000000..b5933cb296 --- /dev/null +++ b/website/content/docs/k8s/installation/vault/gossip.mdx @@ -0,0 +1,92 @@ +--- +layout: docs +page_title: Storing Gossip Encryption Key in Vault +description: >- + Configuring the Consul Helm chart to use gossip encryption key stored in Vault. +--- + +# Storing Gossip Encryption Key in Vault + +To use a gossip encryption key stored in Vault we need the following: + +1. Generate and store an encryption key in Vault. +1. Create policies that will allow Consul client and server to access that key. +1. Create a Kubernetes auth roles that link policies from step 2 to Kubernetes service accounts of the Consul servers and clients. + +## Configuring Vault + +First, generate and store the gossip key in Vault: + +```shell-session +vault kv put secret/consul/gossip key="$(consul keygen)" +``` + +Next, we will need to create a policy that allows read access to this secret: + +```shell-session +# gossip-policy.hcl +path "secret/data/consul/gossip" { + capabilities = ["read"] +} +``` + +```shell-session +vault policy write gossip-policy gossip-policy.hcl +``` + +Prior to creating auth roles for the Consul server and client, ensure that the Vault Kubernetes auth method is enabled: + +```shell-session +vault auth enable kubernetes +``` + +Next, we will create Kubernetes auth roles for the Consul server and client: + +```shell-session +vault write auth/kubernetes/role/consul-server \ + bound_service_account_names= \ + bound_service_account_namespaces= \ + policies=gossip-policy \ + ttl=1h +``` + +```shell-session +vault write auth/kubernetes/role/consul-client \ + bound_service_account_names= \ + bound_service_account_namespaces= \ + policies=gossip-policy \ + ttl=1h +``` + +To find out the service account names of the Consul server and client, +you can run the following `helm template` commands with your Consul on Kubernetes values file: + +``` +# Generate Consul server service account name +helm template --release-name -s templates/server-serviceaccount.yaml hashicorp/consul +# Generate Consul client service account name +helm template --release-name -s templates/client-serviceaccount.yaml hashicorp/consul +``` + +## Deploying the Consul Helm chart + +Now that we've configured Vault, you can configure the Consul Helm chart to +use the gossip key in Vault: + +```yaml +global: + secretsBackend: + vault: + enabled: true + consulServerRole: consul-server + consulClientRole: consul-client + gossipEncryption: + secretName: secret/data/consul/gossip + secretKey: key +``` + +Note that `global.gossipEncryption.secretName` is the path of the secret in Vault. +This should be the same path as the one you'd include in your Vault policy. +`global.gossipEncryption.secretKey` is the key inside the secret data. This should be the same +as the key we passed when we created the gossip secret in Vault. + diff --git a/website/content/docs/k8s/installation/vault/index.mdx b/website/content/docs/k8s/installation/vault/index.mdx new file mode 100644 index 0000000000..0943c3c92c --- /dev/null +++ b/website/content/docs/k8s/installation/vault/index.mdx @@ -0,0 +1,77 @@ +--- +layout: docs +page_title: Vault as Secrets Backend Overview +description: >- + Using Vault as secrets backend for Consul on Kubernetes. +--- + +# Vault as Secrets Backend Overview + +By default, Consul Helm chart will expect that any credentials it needs are stored as Kubernetes secrets. +As of Consul 1.11 and Consul Helm chart v0.38.0, we integrate more natively with Vault making it easier +to use Consul Helm chart with Vault as the secrets storage backend. + +At a high level, there are two points of integration with Vault: +- **Gossip encryption** - The encryption key for gossip communication is stored in Vault. +- **TLS certificates and keys**: + - **Consul Server TLS credentials** - TLS certificate and key for the Consul server is stored in Vault and issued from Vault. + - **Service Mesh and Consul client TLS credentials** - Consul uses Vault as the provider for mTLS certificates and keys for the service mesh services + and TLS certificates and keys for the Consul clients. + +## Requirements + +1. Vault 1.9+ and Vault-k8s 0.14+ is required. +1. Vault must be installed and accessible to the Consul on Kubernetes installation. +1. `global.tls.enableAutoencrypt=true` is required if TLS is enabled for the Consul installation when using the Vault secrets backend. +1. The Vault installation must have been initialized, unsealed and the KV2 and PKI secrets engines enabled and the Kubernetes Auth Method enabled. + +A minimal valid installation of Vault must include the Agent Injector: +```yaml +injector: + enabled: "true" +``` + +## Known Limitations + +- TLS + - `global.tls.serverAdditionalDNSSans` is not currently configurable and must be manually added to the server certificate in Vault. + - `global.tls.serverAdditionalIPSans` is not currently configurable and must be manually added to the server certificate in Vault. + - Mesh gateway is not currently supported. +- Multi-DC Federation is not currently supported. +- Certificate rotation is not currently supported, ensure the TTL for your certificates is sufficiently long. Should your certificates +expire it will be necessary to issue a `consul reload` on each server. +- CA rotation is not currently supported. + +## Next Steps + +To utilize Vault as a secrets backend with Consul it is necessary to add several configuration fields to the Vault installation +which bootstrap Vault Auth roles and Policies for Consul to use. For the supported Vault secrets please see the individual secret +guides and ensure to, when combining the secrets, append the Vault Policies to your Vault Kube Auth Roles via a comma separated value (i.e. `policies=gossip-policy,consul-ca,consul-server,custom-policy`). +Ex: +```shell-session +vault write auth/kubernetes/role/consul-server \ + bound_service_account_names= \ + bound_service_account_namespaces= \ + policies=gossip-policy,consul-ca,consul-server \ + ttl=1h +``` + + +## Troubleshooting + +The Vault integration with Consul on Kubernetes makes use of the Vault Agent Injectors. Kubernetes annotations are added to the +deployments of the Consul components which cause the Vault Agent Injector to be added as an init-container that will then attach +Vault secrets to Consul's pods at startup. Additionally the Vault Agent sidecar is added to the Consul component pods which +is responsible for synchronizing and reissuing secrets at runtime. +As a result of these additional sidecar containers the typical location for logging is expanded in the Consul components. + +As a general rule the best way to troubleshoot startup issues for your Consul installation when using the Vault integration +is to establish if the `vault-agent-init` container has completed or not via `kubectl logs -f -c vault-agent-int` +and checking to see if the secrets have completed rendering. +* If the secrets are not properly rendered the underlying problem will be logged in `vault-agent-init` init-container + and generally is related to the Vault Kube Auth Role not having the correct policies for the specific secret + e.g. `global.secretsBackend.vault.consulServerRole` not having the correct policies for TLS. +* If the secrets are rendered and the `vault-agent-init` container has completed AND the Consul component has not become `Ready`, + this generally points to an issue with Consul being unable to utilize the Vault secret. This can occur if, for example, the Vault Role + created for the PKI engine does not have the correct `alt_names` or otherwise is not properly configured. The best logs for this + circumstance are the Consul container logs: `kubectl logs -f -c consul`. diff --git a/website/content/docs/k8s/installation/vault/server-tls.mdx b/website/content/docs/k8s/installation/vault/server-tls.mdx new file mode 100644 index 0000000000..40d221d1f1 --- /dev/null +++ b/website/content/docs/k8s/installation/vault/server-tls.mdx @@ -0,0 +1,170 @@ +--- +layout: docs +page_title: Storing Server TLS certificates in Vault +description: >- + Configuring the Consul Helm chart to use TLS certificates issued by Vault for the Consul server. +--- + +# Storing Server TLS certificates in Vault + +To use Vault to issue Server TLS certificates the following will be needed: + +1. Bootstrap the Vault PKI engine and boostrap it with any configuration required for your infrastructure. +1. Create Vault Policies that will allow the Consul server to access the certificate issuing url. +1. Create Vault Policies that will allow the Consul components, e.g. ingress gateways, controller, to access the CA url. +1. Create Kubernetes auth roles that link these policies to the Kubernetes service accounts of the Consul components. + + +### Bootstrapping the PKI Engine +First, we need to bootstrap the Vault cluster by enabling and configuring the PKI Secrets Engine to be able to serve +TLS certificates to Consul. The process can be as simple as the following, or more complicated such as in this [example](https://learn.hashicorp.com/tutorials/consul/vault-pki-consul-secure-tls) +which also uses an intermediate signing authority. + +* Enable the PKI Secrets Engine: +```shell-session +vault secrets enable pki +``` +* Tune the engine to enable longer TTL: +```shell-session +vault secrets tune -max-lease-ttl=87600h pki +``` +* Generate the root CA +```shell-session +vault write -field=certificate pki/root/generate/internal \ + common_name="dc1.consul" \ + ttl=87600h +``` +-> **Note:** Where `common_name` is comprised of combining `global.datacenter` dot `global.domain`. + +### Create Vault Policies for the Server TLS Certificates + +Next we will create a policy that allows `["create", "update"]` access to the +[certificate issuing URL](https://www.vaultproject.io/api/secret/pki#generate-certificate) so the Consul servers can +fetch a new certificate/key pair. + +```shell-session +# consul-server-policy.hcl +path "pki/issue/consul-server" { + capabilities = ["create", "update"] +} +``` + +```shell-session +vault policy write consul-server consul-server-policy.hcl +``` +-> **Note:** The PKI secret path referenced by the above Policy will be your `server.serverCert.secretName` Helm value. + +### Create Vault Policies for the CA URL + +Next, we will create a policy that allows `["read"]` access to the [CA URL](https://www.vaultproject.io/api/secret/pki#read-certificate), +this is required for the Consul components to communicate with the Consul servers in order to fetch their auto-encryption certificates. + +```shell-session +# ca-policy.hcl +path "pki/cert/ca" { + capabilities = ["read"] +} +``` + +```shell-session +vault policy write ca-policy ca-policy.hcl +``` +-> **Note:** The PKI secret path referenced by the above Policy will be your `global.tls.caCert.secretName` Helm value. + +### Create Vault Roles for the PKI engine, Consul servers and components + +Next, a Vault role for the PKI engine will set the default certificate issuance parameters: + +```shell-session +vault write pki/roles/consul-server \ + allowed_domains="" \ + allow_subdomains=true \ + allow_bare_domains=true \ + allow_localhost=true \ + generate_lease=true \ + max_ttl="720h" +``` + +To generate the `` use the following script as a template: + +```shell-session +#!/bin/sh + +# NAME is set to either the value from `global.name` from your Consul K8s value file, or your $HELM_RELEASE_NAME-consul +export NAME=consulk8s +# NAMESPACE is where the Consul on Kubernetes is installed +export NAMESPACE=consul +# DATACENTER is the value of `global.datacenter` from your Helm values config file +export DATACENTER=dc1 + +echo allowed_domains=\"$DATACENTER.consul, $NAME-server, $NAME-server.$NAMESPACE, $NAME-server.$NAMESPACE.svc\" +``` + +Prior to creating the Kubernetes auth roles required for Consul to securely access Vault, ensure that the Vault Kubernetes Auth method is enabled: + +```shell-session +vault auth enable kubernetes +``` + +Finally, two Kubernetes auth roles need to be created, one for the Consul servers and one for the Consul components: + +```shell-session +vault write auth/kubernetes/role/consul-server \ + bound_service_account_names= \ + bound_service_account_namespaces= \ + policies=consul-server \ + ttl=1h +``` + +To find out the service account name of the Consul server, +you can run: +```shell-session + helm template --release-name -s templates/server-serviceaccount.yaml hashicorp/consul +``` + +-> **Note:** Should you enable other supported features such as gossip-encryption be sure to append additional policies to +the Kube auth role in a comma separated value e.g. `policies=consul-server,consul-gossip` + +```shell-session +vault write auth/kubernetes/role/consul-ca \ + bound_service_account_names="*" \ + bound_service_account_namespaces= \ + policies=consul-ca \ + ttl=1h +``` + +The above Vault Roles will now be your Helm values for `global.secretsBackend.vault.consulServerRole` and +`global.secretsBAckend.vault.consulCARole` respectively. + + +## Deploying the Consul Helm chart + +Now that we've configured Vault, you can configure the Consul Helm chart to +use the Server TLS certificates from Vault: + +```yaml +global: + secretsBackend: + vault: + enabled: true + consulServerRole: consul-server + consulClientRole: consul-client + consulCARole: consul-ca + tls: + enabled: true + caCert: + secretName: "pki/cert/ca" +server: + serverCert: + secretName: "pki/issue/consul-server" + extraVolumes: + - type: "secret" + name: + load: "false" +``` + +The `vaultCASecret` is the Kubernetes secret that stores the CA Certificate that is used for Vault communication. To provide a CA, you first need to create a Kubernetes secret containing the CA. For example, you may create a secret with the Vault CA like so: + +``` +kubectl create secret generic vault-ca --from-file vault.ca=/path/to/your/vault/ +``` diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 47f87740b9..9de0a9dcd5 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -436,6 +436,27 @@ "path": "k8s/installation/multi-cluster/vms-and-kubernetes" } ] + }, + { + "title": "Vault as Secrets Backend", + "routes": [ + { + "title": "Overview", + "path": "k8s/installation/vault" + }, + { + "title": "Gossip Encryption Key", + "path": "k8s/installation/vault/gossip" + }, + { + "title": "Server TLS", + "path": "k8s/installation/vault/server-tls" + }, + { + "title": "Service Mesh Certificates", + "path": "k8s/installation/vault/connect-ca" + } + ] } ] },