mirror of https://github.com/status-im/consul.git
368 lines
14 KiB
Plaintext
368 lines
14 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: Install with Helm Chart - Kubernetes
|
|
description: >-
|
|
Consul can run directly on Kubernetes, both in server or client mode. For
|
|
pure-Kubernetes workloads, this enables Consul to also exist purely within
|
|
Kubernetes. For heterogeneous workloads, Consul agents can join a server
|
|
running inside or outside of Kubernetes.
|
|
---
|
|
|
|
# Installing Consul on Kubernetes
|
|
|
|
Consul can run directly on Kubernetes, both in server or client mode.
|
|
For pure-Kubernetes workloads, this enables Consul to also exist purely
|
|
within Kubernetes. For heterogeneous workloads, Consul agents can join
|
|
a server running inside or outside of Kubernetes.
|
|
|
|
You can install Consul on Kubernetes using the Helm chart or by using the Consul K8s CLI tool <sup>ALPHA</sup>.
|
|
Refer to the [architecture](/docs/k8s/installation/install#architecture) section to learn more about the general architecture of Consul on Kubernetes.
|
|
For a hands-on experience with Consul as a service mesh
|
|
for Kubernetes, follow the [Getting Started with Consul service
|
|
mesh](https://learn.hashicorp.com/tutorials/consul/service-mesh-deploy?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS) tutorial.
|
|
|
|
## Helm Chart Installation
|
|
|
|
The recommended way to run Consul on Kubernetes is via the
|
|
[Helm chart](/docs/k8s/helm). This will install and configure
|
|
all the necessary components to run Consul. The configuration enables you
|
|
to run a server cluster, a client cluster, or both.
|
|
|
|
Step-by-step tutorials for how to deploy Consul to Kubernetes, please see
|
|
our [Deploy to Kubernetes](https://learn.hashicorp.com/collections/consul/kubernetes-deploy)
|
|
collection. This collection includes configuration caveats for single-node deployments.
|
|
|
|
The Helm chart exposes several useful configurations and automatically
|
|
sets up complex resources, but it **does not automatically operate Consul.**
|
|
You must still become familiar with how to monitor, backup,
|
|
upgrade, etc. the Consul cluster.
|
|
|
|
The Helm chart has no required configuration and will install a Consul
|
|
cluster with default configurations. We strongly recommend [learning about the configuration options](/docs/k8s/helm#configuration-values) prior to going to production.
|
|
|
|
~> **Security Warning:** By default, the chart will install an insecure configuration
|
|
of Consul. This provides a less complicated out-of-box experience for new users,
|
|
but is not appropriate for a production setup. We strongly recommend using
|
|
a properly-secured Kubernetes cluster or making sure that you understand and enable
|
|
the [recommended security features](/docs/internals/security). Currently,
|
|
some of these features are not supported in the Helm chart and require additional
|
|
manual configuration.
|
|
|
|
### Prerequisites
|
|
|
|
The Consul Helm only supports Helm 3. Install the latest version of the Helm CLI here:
|
|
[Installing Helm](https://helm.sh/docs/intro/install/).
|
|
|
|
### Installing Consul
|
|
|
|
1. Add the HashiCorp Helm Repository:
|
|
|
|
```shell-session
|
|
$ helm repo add hashicorp https://helm.releases.hashicorp.com
|
|
"hashicorp" has been added to your repositories
|
|
```
|
|
|
|
1. Verify that you have access to the consul chart:
|
|
|
|
```shell-session
|
|
$ helm search repo hashicorp/consul
|
|
NAME CHART VERSION APP VERSION DESCRIPTION
|
|
hashicorp/consul 0.32.0 1.10.0 Official HashiCorp Consul Chart
|
|
```
|
|
|
|
Now you're ready to install Consul! To install Consul with the default
|
|
configuration using Helm 3 run:
|
|
|
|
```shell-session
|
|
$ helm install consul hashicorp/consul --set global.name=consul
|
|
NAME: consul
|
|
...
|
|
```
|
|
|
|
The Helm chart does everything to set up a recommended Consul-on-Kubernetes deployment.
|
|
After installation, a Consul cluster will be formed, a leader will be elected, and every node will have a running Consul agent.
|
|
|
|
### Customizing Your Installation
|
|
|
|
If you want to customize your installation,
|
|
create a `config.yaml` file to override the default settings.
|
|
You can learn what settings are available by running `helm inspect values hashicorp/consul`
|
|
or by reading the [Helm Chart Reference](/docs/k8s/helm).
|
|
|
|
For example, if you want to enable the [Consul Connect](/docs/k8s/connect) feature,
|
|
use the following config file:
|
|
|
|
<CodeBlockConfig filename="config.yaml">
|
|
|
|
```yaml
|
|
global:
|
|
name: consul
|
|
connectInject:
|
|
enabled: true
|
|
controller:
|
|
enabled: true
|
|
```
|
|
|
|
</CodeBlockConfig>
|
|
|
|
Once you've created your `config.yaml` file, run `helm install` with the `-f` flag:
|
|
|
|
```shell-session
|
|
$ helm install consul hashicorp/consul -f config.yaml
|
|
NAME: consul
|
|
...
|
|
```
|
|
|
|
If you've already installed Consul and want to make changes, you'll need to run
|
|
`helm upgrade`. See [Upgrading](/docs/k8s/operations/upgrading) for more details.
|
|
|
|
## Consul K8s CLI Installation
|
|
|
|
You can install Consul on Kubernetes using the Consul K8s CLI tool.
|
|
The tool is currently availabe as an alpha release and is not recommended for production environments.
|
|
|
|
1. Download and build the CLI as described in the [Consul K8s CLI reference](/docs/k8s/k8s-cli).
|
|
1. Issue the `install` subcommand to install Consul on Kubernetes:
|
|
|
|
```shell-session
|
|
consul-k8s install <OPTIONS>
|
|
```
|
|
|
|
Refer to the [Consul K8s CLI reference](/docs/k8s/k8s-cli) for details about all commands and available options.
|
|
|
|
If you did not set the `-auto-approve` option to `true`, you will be prompted to proceed with the installation if the pre-install checks pass.
|
|
|
|
```shell-session
|
|
==> Pre-Install Checks
|
|
✓ No existing installations found
|
|
✓ No previous persistent volume claims found
|
|
✓ No previous secrets found
|
|
|
|
==> Consul Installation Summary
|
|
Installation name: consul
|
|
Namespace: myns
|
|
Overrides:
|
|
connectInject:
|
|
enabled: true
|
|
global:
|
|
name: consul
|
|
server:
|
|
bootstrapExpect: 1
|
|
replicas: 1
|
|
|
|
Proceed with installation? (y/n)
|
|
```
|
|
|
|
1. Enter `y` to proceed. The pre-install checks may fail if existing `PersistentVolumeClaims` (PVC) are detected. Refer to the [uninstall instructions](/docs/k8s/operations/uninstall#uninstall-consul) for information about removing PVCs.
|
|
|
|
## Viewing the Consul UI
|
|
|
|
The Consul UI is enabled by default when using the Helm chart.
|
|
For security reasons, it isn't exposed via a `LoadBalancer` Service by default so you must
|
|
use `kubectl port-forward` to visit the UI.
|
|
|
|
#### TLS Disabled
|
|
|
|
If running with TLS disabled, the Consul UI will be accessible via http on port 8500:
|
|
|
|
```shell-session
|
|
$ kubectl port-forward service/consul-server 8500:8500
|
|
...
|
|
```
|
|
|
|
Once the port is forwarded navigate to [http://localhost:8500](http://localhost:8500).
|
|
|
|
#### TLS Enabled
|
|
|
|
If running with TLS enabled, the Consul UI will be accessible via https on port 8501:
|
|
|
|
```shell-session
|
|
$ kubectl port-forward service/consul-server 8501:8501
|
|
...
|
|
```
|
|
|
|
Once the port is forwarded navigate to [https://localhost:8501](https://localhost:8501).
|
|
|
|
~> You'll need to click through an SSL warning from your browser because the
|
|
Consul certificate authority is self-signed and not in the browser's trust store.
|
|
|
|
#### ACLs Enabled
|
|
|
|
If ACLs are enabled, you will need to input an ACL token into the UI in order
|
|
to see all resources and make modifications.
|
|
|
|
To retrieve the bootstrap token that has full permissions, run:
|
|
|
|
```shell-session
|
|
$ kubectl get secrets/consul-bootstrap-acl-token --template={{.data.token}} | base64 --decode
|
|
e7924dd1-dc3f-f644-da54-81a73ba0a178%
|
|
```
|
|
|
|
Then paste the token into the UI under the ACLs tab (without the `%`).
|
|
|
|
~> NOTE: If using multi-cluster federation, your kubectl context must be in the primary datacenter
|
|
to retrieve the bootstrap token since secondary datacenters use a separate token
|
|
with less permissions.
|
|
|
|
### Exposing the UI via a service
|
|
|
|
If you want to expose the UI via a Kubernetes Service, configure
|
|
the [`ui.service` chart values](/docs/k8s/helm#v-ui-service).
|
|
This service will allow requests to the Consul servers so it should
|
|
not be open to the world.
|
|
|
|
## Accessing the Consul HTTP API
|
|
|
|
The Consul HTTP API should be accessed by communicating to the local agent
|
|
running on the same node. While technically any listening agent (client or
|
|
server) can respond to the HTTP API, communicating with the local agent
|
|
has important caching behavior, and allows you to use the simpler
|
|
[`/agent` endpoints for services and checks](/api/agent).
|
|
|
|
For Consul installed via the Helm chart, a client agent is installed on
|
|
each Kubernetes node. This is explained in the [architecture](/docs/k8s/installation/install#client-agents)
|
|
section. To access the agent, you may use the
|
|
[downward API](https://kubernetes.io/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/).
|
|
|
|
An example pod specification is shown below. In addition to pods, anything
|
|
with a pod template can also access the downward API and can therefore also
|
|
access Consul: StatefulSets, Deployments, Jobs, etc.
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata:
|
|
name: consul-example
|
|
spec:
|
|
containers:
|
|
- name: example
|
|
image: 'consul:latest'
|
|
env:
|
|
- name: HOST_IP
|
|
valueFrom:
|
|
fieldRef:
|
|
fieldPath: status.hostIP
|
|
command:
|
|
- '/bin/sh'
|
|
- '-ec'
|
|
- |
|
|
export CONSUL_HTTP_ADDR="${HOST_IP}:8500"
|
|
consul kv put hello world
|
|
restartPolicy: Never
|
|
```
|
|
|
|
An example `Deployment` is also shown below to show how the host IP can
|
|
be accessed from nested pod specifications:
|
|
|
|
<CodeBlockConfig highlight="18-28">
|
|
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: consul-example-deployment
|
|
spec:
|
|
replicas: 1
|
|
selector:
|
|
matchLabels:
|
|
app: consul-example
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: consul-example
|
|
spec:
|
|
containers:
|
|
- name: example
|
|
image: 'consul:latest'
|
|
env:
|
|
- name: HOST_IP
|
|
valueFrom:
|
|
fieldRef:
|
|
fieldPath: status.hostIP
|
|
command:
|
|
- '/bin/sh'
|
|
- '-ec'
|
|
- |
|
|
export CONSUL_HTTP_ADDR="${HOST_IP}:8500"
|
|
consul kv put hello world
|
|
```
|
|
|
|
</CodeBlockConfig>
|
|
|
|
## Architecture
|
|
|
|
Consul runs on Kubernetes with the same
|
|
[architecture](/docs/internals/architecture)
|
|
as other platforms. There are some benefits Kubernetes can provide
|
|
that eases operating a Consul cluster and we document those below. The standard
|
|
[production deployment guide](https://learn.hashicorp.com/consul/datacenter-deploy/deployment-guide) is still an
|
|
important read even if running Consul within Kubernetes.
|
|
|
|
Each section below will outline the different components of running Consul
|
|
on Kubernetes and an overview of the resources that are used within the
|
|
Kubernetes cluster.
|
|
|
|
### Server Agents
|
|
|
|
The server agents are run as a **StatefulSet**, using persistent volume
|
|
claims to store the server state. This also ensures that the
|
|
[node ID](/docs/agent/options#_node_id) is persisted so that servers
|
|
can be rescheduled onto new IP addresses without causing issues. The server agents
|
|
are configured with
|
|
[anti-affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity)
|
|
rules so that they are placed on different nodes. A readiness probe is
|
|
configured that marks the pod as ready only when it has established a leader.
|
|
|
|
A **Service** is registered to represent the servers and expose the various
|
|
ports. The DNS address of this service is used to join the servers to each
|
|
other without requiring any other access to the Kubernetes cluster. The
|
|
service is configured to publish non-ready endpoints so that it can be used
|
|
for joining during bootstrap and upgrades.
|
|
|
|
Additionally, a **PodDisruptionBudget** is configured so the Consul server
|
|
cluster maintains quorum during voluntary operational events. The maximum
|
|
unavailable is `(n/2)-1` where `n` is the number of server agents.
|
|
|
|
-> **Note:** Kubernetes and Helm do not delete Persistent Volumes or Persistent
|
|
Volume Claims when a
|
|
[StatefulSet is deleted](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#stable-storage),
|
|
so this must done manually when removing servers.
|
|
|
|
### Client Agents
|
|
|
|
The client agents are run as a **DaemonSet**. This places one agent
|
|
(within its own pod) on each Kubernetes node.
|
|
The clients expose the Consul HTTP API via a static port (8500)
|
|
bound to the host port. This enables all other pods on the node to connect
|
|
to the node-local agent using the host IP that can be retrieved via the
|
|
Kubernetes downward API. See
|
|
[accessing the Consul HTTP API](/docs/k8s/installation/install#accessing-the-consul-http-api)
|
|
for an example.
|
|
|
|
We do not use a **NodePort** Kubernetes service because requests to node ports get randomly routed
|
|
to any pod in the service and we need to be able to route directly to the Consul
|
|
client running on our node.
|
|
|
|
-> **Note:** There is no way to bind to a local-only
|
|
host port. Therefore, any other node can connect to the agent. This should be
|
|
considered for security. For a properly production-secured agent with TLS
|
|
and ACLs, this is safe.
|
|
|
|
We run Consul clients as a **DaemonSet** instead of running a client in each
|
|
application pod as a sidecar because this would turn
|
|
a pod into a "node" in Consul and also causes an explosion of resource usage
|
|
since every pod needs a Consul agent. Service registration should be handled via the
|
|
catalog syncing feature with Services rather than pods.
|
|
|
|
-> **Note:** Due to a limitation of anti-affinity rules with DaemonSets,
|
|
a client-mode agent runs alongside server-mode agents in Kubernetes. This
|
|
duplication wastes some resources, but otherwise functions perfectly fine.
|
|
|
|
## Next Steps
|
|
|
|
If you are still considering a move to Kubernetes, or to Consul on Kubernetes specifically, our [Migrate to Microservices with Consul Service Mesh on Kubernetes](https://learn.hashicorp.com/collections/consul/microservices?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS)
|
|
collection uses an example application written by a fictional company to illustrate why and how organizations can
|
|
migrate from monolith to microservices using Consul service mesh on Kubernetes. The case study in this collection
|
|
should provide information valuable for understanding how to develop services that leverage Consul during any stage
|
|
of your microservices journey.
|