consul/website/content/docs/k8s/connect/terminating-gateways.mdx

---
layout: docs
page_title: Terminating Gateways - Kubernetes
description: Configuring Terminating Gateways on Kubernetes
---

# Terminating Gateways on Kubernetes

-> 1.9.0+: This feature is available in Consul versions 1.9.0 and higher

~> This topic requires familiarity with [Terminating Gateways](/docs/connect/gateways/terminating-gateway).

Adding a terminating gateway is a multi-step process:

- Update the Helm chart with terminating gateway config options
- Deploy the Helm chart
- Access the Consul agent
- Register external services with Consul

## Update the helm chart with terminating gateway config options

Minimum required Helm options:

<CodeBlockConfig filename="config.yaml">

```yaml
global:
  name: consul
connectInject:
  enabled: true
controller:
  enabled: true
terminatingGateways:
  enabled: true
```

</CodeBlockConfig>

## Deploying the Helm chart

Ensure you have the latest consul-helm chart and install Consul via helm using the following
[guide](/docs/k8s/installation/install#installing-consul) while being sure to provide the yaml configuration
as previously discussed.

## Accessing the Consul agent

You can access the Consul server directly from your host via `kubectl port-forward`. This is helpful for interacting with your Consul UI locally as well as to validate connectivity of the application.

```shell-session
$ kubectl port-forward consul-server-0 8500 &
```

If TLS is enabled use port 8501:

```shell-session
$ kubectl port-forward consul-server-0 8501 &
```

-> Be sure the latest consul binary is installed locally on your host.
[https://releases.hashicorp.com/consul/](https://releases.hashicorp.com/consul/)

```shell-session
$ export CONSUL_HTTP_ADDR=http://localhost:8500
```

If TLS is enabled set:

```shell-session
$ export CONSUL_HTTP_ADDR=https://localhost:8501
$ export CONSUL_HTTP_SSL_VERIFY=false
```

If ACLs are enabled also set:

```shell-session
$ export CONSUL_HTTP_TOKEN=$(kubectl get secret consul-bootstrap-acl-token --template='{{.data.token | base64decode }}')
```

## Register external services with Consul

Registering the external services with Consul is a multi-step process:

- Register external services with Consul
- Update the terminating gateway ACL token if ACLs are enabled
- Create a [`TerminatingGateway`](/docs/connect/config-entries/terminating-gateway) resource to configure the terminating gateway
- Create a [`ServiceIntentions`](/docs/connect/config-entries/service-intentions) resource to allow access from services in the mesh to external service
- Define upstream annotations for any services that need to talk to the external services

### Register external services with Consul

There are two ways to register an external service with Consul:
1. If [`TransparentProxy`](/docs/k8s/helm#v-connectinject-transparentproxy) is enabled, you can declare external endpoints in the [`Destination`](/docs/connect/config-entries/service-defaults#terminating-gateway-destination) field of `service-defaults`.
1. You can add the service as a node in the Consul catalog.

#### Register an external service as a Destination

`Destination` fields allow clients to dial the external service directly and are valid only in [`TransparentProxy`](/docs/k8s/helm#v-connectinject-transparentproxy) mode.
The following table describes traffic behaviors when using `Destination`s to route traffic through a terminating gateway:

| External Services Layer | Client dials | Client uses TLS | Allowed | Notes |
|---|---|---|---|---|
| L4 | Hostname | Yes | Allowed | `CAFiles` are not allowed because traffic is already end-to-end encrypted by the client. |
| L4 | IP | Yes | Allowed | `CAFiles` are not allowed because traffic is already end-to-end encrypted by the client. |
| L4 | Hostname | No | Not allowed | The sidecar is not protocol aware and can not identify traffic going to the external service. |
| L4 | IP | No | Allowed | There are no limitations on dialing IPs without TLS. |
| L7 | Hostname | Yes | Not allowed | Because traffic is already encrypted before the sidecar, it cannot route as L7 traffic. |
| L7 | IP | Yes | Not allowed | Because traffic is already encrypted before the sidecar, it cannot route as L7 traffic. |
| L7 | Hostname | No | Allowed | A `Host` or `:authority` header is required. |
| L7 | IP | No | Allowed | There are no limitations on dialing IPs without TLS.  |

You can provide a `caFile` to secure traffic between unencrypted clients that connect to external services through the terminating gateway.
Refer to [Create the configuration entry for the terminating gateway](/docs/k8s/connect/terminating-gateways#create-the-configuration-entry-for-the-terminating-gateway) for details.

Create a `service-defaults` custom resource for the external service:

<CodeBlockConfig filename="service-defaults.yaml">

```yaml
  apiVersion: consul.hashicorp.com/v1alpha1
  kind: ServiceDefaults
  metadata:
    name: example-https
  spec:
    protocol: tcp
    destination:
      addresses:
        - "example.com"
      port: 443
```

</CodeBlockConfig>

Apply the `ServiceDefaults` resource with `kubectl apply`:

```shell-session
$ kubectl apply --filename service-defaults.yaml
```

All other terminating gateway operations can use the name of the `service-defaults` in place of a typical Consul service name.

#### Register an external service as a Catalog Node

-> **Note:** Normal Consul services are registered with the Consul client on the node that
they're running on. Since this is an external service, there is no Consul node
to register it onto. Instead, we will make up a node name and register the
service to that node.

Create a sample external service and register it with Consul.

<CodeBlockConfig filename="external.json">

```json
{
  "Node": "example_com",
  "Address": "example.com",
  "NodeMeta": {
    "external-node": "true",
    "external-probe": "true"
  },
  "Service": {
    "Address": "example.com",
    "ID": "example-https",
    "Service": "example-https",
    "Port": 443
  }
}
```

</CodeBlockConfig>

- `"Node": "example_com"` is our made up node name.
- `"Address": "example.com"` is the address of our node. Services registered to that node will use this address if
  their own address isn't specified. If you're registering multiple external services, ensure you
  use different node names with different addresses or set the `Service.Address` key.
- `"Service": { "Address": "example.com" ... }` is the address of our service. In this example this doesn't need to be set
  since the address of the node is the same, but if there were two services registered to that same node
  then this should be set.

Register the external service with Consul:

```shell-session
$ curl --request PUT --data @external.json --insecure $CONSUL_HTTP_ADDR/v1/catalog/register
true
```

If ACLs and TLS are enabled :

```shell-session
$ curl --request PUT --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" --data @external.json --insecure $CONSUL_HTTP_ADDR/v1/catalog/register
true
```

### Update terminating gateway ACL role if ACLs are enabled

If ACLs are enabled, update the terminating gateway acl role to have `service: write` permissions on all of the services
being represented by the gateway:

- Create a new policy that includes these permissions
- Update the existing role to include the new policy

<CodeBlockConfig filename="write-policy.hcl">

```hcl
service "example-https" {
  policy = "write"
}
```

</CodeBlockConfig>

```shell-session
$ consul acl policy create -name "example-https-write-policy" -rules @write-policy.hcl
ID:           xxxxxxxxxxxxxxx
Name:         example-https-write-policy
Description:
Datacenters:
Rules:
service "example-https" {
  policy = "write"
}
```

Now fetch the ID of the terminating gateway token

```shell-session
consul acl role list | grep -B 6 -- "- RELEASE_NAME-terminating-gateway-policy" | grep ID

ID:       <role id>
```

Update the terminating gateway acl token with the new policy

```shell-session
$ consul acl role update -id <role id> -policy-name example-https-write-policy
AccessorID:       <role id>
SecretID:         <secret id>
Description:      RELEASE_NAME-terminating-gateway-acl-role
Local:            true
Create Time:      2021-01-08 21:18:47.957450486 +0000 UTC
Policies:
   63bf1d9b-a87d-8672-ddcb-d25e2d88adb8 - RELEASE_NAME-terminating-gateway-policy
   f63d1ae6-ffe7-44bd-bf7a-704a86939a63 - example-https-write-policy
```

### Create the configuration entry for the terminating gateway

Once the roles have been updated, create the [TerminatingGateway](/docs/connect/config-entries/terminating-gateway)
resource to configure the terminating gateway:

<CodeBlockConfig filename="terminating-gateway.yaml">

```yaml
apiVersion: consul.hashicorp.com/v1alpha1
kind: TerminatingGateway
metadata:
  name: terminating-gateway
spec:
  services:
    - name: example-https
```

</CodeBlockConfig>

-> **NOTE**: If TLS is enabled for external services registered through the Consul catalog, you must include the `caFile` parameter that points to the system trust store of the terminating gateway container.
By default, the trust store is located in the `/etc/ssl/certs/ca-certificates.crt` directory.
Configure the `caFile` parameter to point to the `/etc/ssl/cert.pem` directory if TLS is enabled and you are using one of the following components:
 * Consul Helm chart 0.43 or older
 * Or an Envoy image with an alpine base image

Apply the `TerminatingGateway` resource with `kubectl apply`:

```shell-session
$ kubectl apply --filename terminating-gateway.yaml
```

If using ACLs and TLS, create a [`ServiceIntentions`](/docs/connect/config-entries/service-intentions) resource to allow access from services in the mesh to the external service

<CodeBlockConfig filename="service-intentions.yaml">

```yaml
apiVersion: consul.hashicorp.com/v1alpha1
kind: ServiceIntentions
metadata:
  name: example-https
spec:
  destination:
    name: example-https
  sources:
    - name: static-client
      action: allow
```

</CodeBlockConfig>

Apply the `ServiceIntentions` resource with `kubectl apply`:

```shell-session
$ kubectl apply --filename service-intentions.yaml
```

### Define the external services as upstreams for services in the mesh

Finally define and deploy the external services as upstreams for the internal mesh services that wish to talk to them.
An example deployment is provided which will serve as a static client for the terminating gateway service.

<CodeBlockConfig filename="static-client.yaml">

```yaml
apiVersion: v1
kind: Service
metadata:
  name: static-client
spec:
  selector:
    app: static-client
  ports:
    - port: 80
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: static-client
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: static-client
spec:
  replicas: 1
  selector:
    matchLabels:
      app: static-client
  template:
    metadata:
      name: static-client
      labels:
        app: static-client
      annotations:
        'consul.hashicorp.com/connect-inject': 'true'
        'consul.hashicorp.com/connect-service-upstreams': 'example-https:1234'
    spec:
      containers:
        - name: static-client
          image: curlimages/curl:latest
          command: ['/bin/sh', '-c', '--']
          args: ['while true; do sleep 30; done;']
      serviceAccountName: static-client
```

</CodeBlockConfig>

Run the service via `kubectl apply`:

```shell-session
$ kubectl apply --filename static-client.yaml
```

Wait for the service to be ready:

```shell-session
$ kubectl rollout status deploy static-client --watch
deployment "static-client" successfully rolled out
```

You can verify connectivity of the static-client and terminating gateway via a curl command:

<CodeBlockConfig heading="External services registered with the Consul catalog">

```shell-session
$ kubectl exec deploy/static-client -- curl -vvvs --header "Host: example-https.com" http://localhost:1234/
```

</CodeBlockConfig>

<CodeBlockConfig heading="External services registered with `service-defaults` destinations">

```shell-session
$ kubectl exec deploy/static-client -- curl -vvvs https://example.com/
```

</CodeBlockConfig>