status-im
/
consul
mirror of https://github.com/status-im/consul.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
consul/website/content/docs/ecs/install-no-terraform.mdx

493 lines
21 KiB
Plaintext
Raw Blame History

---
layout: docs
page_title: Manual Installation - AWS ECS
description: >-
Manually Install Consul Service Mesh on AWS ECS (Elastic Container Service).
---
# Manual Installation
While the [Consul ECS Terraform module](/docs/ecs/install) is the easiest way to use Consul on ECS,
this page will describe how to directly create the ECS task definition using the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs)
for use without Terraform.
## Pre-requisites
* This page assumes you are familiar with AWS ECS. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for more details.
* This page does not show how to create all necessary AWS resources, such as a VPC or the ECS Cluster.
For complete runnable examples, see the links in the [Getting Started](/docs/ecs#getting-started) section.
# Task Definition
You must define a Task Definition which includes the following containers:
* Your application container
* An Envoy sidecar-proxy container
* A Consul Agent container
* The `consul-ecs-mesh-init` container for service mesh setup
* Optionally, a `consul-ecs-health-sync` container to sync ECS health checks into Consul
## Top-level fields
In your task definition, you'll need to define these important top-level fields:
```json
{
"family": "my-example-client-app",
"networkMode": "awsvpc",
"volumes": [
{
"name": "consul_data",
},
{
"name": "consul_binary",
}
],
"containerDefinitions": [...]
}
```
| Field name | Type | Description |
| ------------- | ------ | --------------------------------------------------------------------------------------------------- |
| `family` | string | The task family name. This is used as the Consul service name, by default. |
| `networkMode` | string | Must be `awsvpc`, which is the only network mode supported by Consul on ECS. |
| `volumes` | list | These are volumes used to share information between containers, which is important for setup steps. |
| `containerDefinitions` | list | The list of containers to run in this task (see below). |
## Application container
First, include your application container in the `containerDefinitions` list
in the task definition:
```json
{
"containerDefinitions": [
{
"name": "example-client-app",
"image": "docker.io/org/my_task:v0.0.1",
"essential": true,
"dependsOn": [
{
"containerName": "consul-ecs-mesh-init",
"condition": "SUCCESS"
},
{
"containerName": "sidecar-proxy",
"condition": "HEALTHY"
}
]
}
]
}
```
| Field name | Type | Description |
| ----------- | ------- | ------------------------------------------------------------------------------------------------------------------------ |
| `name` | string | The name of your application container. |
| `image` | string | The container image used to run your application. |
| `essential` | boolean | Set this to `true` ensures your application container ties into the health of the task. All tasks must have at least one essential container. |
| `dependsOn` | list | Container dependendencies are used to ensure the service mesh is ready before your application starts. |
See the [ECS Task Definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html) documentation for a complete reference.
## `sidecar-proxy` container
The sidecar proxy container runs [Envoy proxy](/docs/connect/proxies/envoy) for Consul Connect.
```json
{
"containerDefinitions": [
{
"name": "example-client-app",
"image": "docker.io/org/my_task:v0.0.1",
...
},
{
"name": "sidecar-proxy",
"image": "envoyproxy/envoy-alpine:<VERSION>",
"essential": false,
"dependsOn": [
{
"containerName": "consul-ecs-mesh-init",
"condition": "SUCCESS"
}
],
"healthCheck": {
"retries": 3,
"command": ["nc", "-z", "127.0.0.1", "20000"],
"timeout": 5,
"interval": 30
},
"mountPoints": [
{
"readOnly": true,
"containerPath": "/consul",
"sourceVolume": "consul_data"
}
],
"ulimits": [
{
"name": "nofile",
"softLimit": 1048576,
"hardLimit": 1048576
}
],
"command": ["envoy", "--config-path", "/consul/envoy-bootstrap.json"],
"entryPoint": ["/consul/consul-ecs", "envoy-entrypoint"],
}
]
}
```
| Field name | Type | Description |
| ----------- | ------- | ------------------------------------------------------------------------------------------------------------------ |
| `name` | string | The name of the container, which should always be `sidecar-proxy`. |
| `image` | string | The container image for Envoy. We recommend using `envoyproxy/envoy-alpine`. |
| `dependsOn` | list | Envoy must start after `consul-ecs-mesh-init`, which creates the bootstrap configuration file for Envoy. |
| `healthCheck` | list | A health check should be definied for Envoy's primary listener port. |
| `mountPoints` | list | The mounts the `/consul` data volume which contains the Envoy configuration file. |
| `ulimits` | list | Set the file descriptor limit, `nofile`, to a sufficiently high value so that Envoy does not fail to open sockets. |
| `entrypoint` | list | A custom entrypoint binary is used to help facilitate graceful shutdown. |
| `command` | list | The startup command. This passes the bootstrap configuration to Envoy. |
-> **NOTE**: Envoy and Consul must be compatible versions. See the [supported versions of Envoy](https://www.consul.io/docs/connect/proxies/envoy#supported-versions) for each Consul release.
## `consul-client` container
Each task must include a Consul agent container in order for the task to join your Consul cluster.
```json
{
"containerDefinitions": [
{
"name": "example-client-app",
"image": "docker.io/org/my_task:v0.0.1",
...
},
{
"name": "sidecar-proxy",
"image": "envoyproxy/envoy-alpine:<ENVOY_VERSION>",
...
}
{
"name": "consul-client"
"image": "public.ecr.aws/hashicorp/consul:<CONSUL_VERSION>",
"mountPoints": [
{
"readOnly": false,
"containerPath": "/consul",
"sourceVolume": "consul_data"
},
{
"containerPath": "/bin/consul-inject",
"sourceVolume": "consul_binary"
}
],
"entryPoint": ["/bin/sh", "-ec"],
"command": [
"cp /bin/consul /bin/consul-inject/consul\n\nECS_IPV4=$(curl -s $ECS_CONTAINER_METADATA_URI_V4 | jq -r '.Networks[0].IPv4Addresses[0]')\n\n\ncat << EOF > /consul/agent-defaults.hcl\naddresses = {\n dns = \"127.0.0.1\"\n grpc = \"127.0.0.1\"\n http = \"127.0.0.1\"\n}\nadvertise_addr = \"$ECS_IPV4\"\nadvertise_reconnect_timeout = \"15m\"\nclient_addr = \"0.0.0.0\"\ndatacenter = \"dc1\"\nenable_central_service_config = true\nleave_on_terminate = true\nports {\n grpc = 8502\n}\nretry_join = [\n \"<Consul server location>",\n]\ntelemetry {\n disable_compat_1.9 = true\n}\n\nEOF\n\ncat << EOF > /consul/agent-extra.hcl\naddresses = {\n dns = \"0.0.0.0\"\n}\nlog_level = \"debug\"\n\nEOF\n\nexec consul agent \\\n -data-dir /consul/data \\\n -config-file /consul/agent-defaults.hcl \\\n -config-file /consul/agent-extra.hcl\n"
]
}
]
}
```
| Field name | Type | Description |
| ----------- | ------- | ------------------------------------------------------------------------------------------------------------------------ |
| `name` | string | The name of the Consul agent container, which should always be `consul-client`. |
| `image` | string | The container image for Consul. Use our public AWS registry, `public.ecr.aws/hashicorp/consul`, to avoid rate limits. |
| `mountPoints` | list | The mounts the `/consul` data volume which contains the Envoy configuration file. |
| `ulimits` | list | Set the file descriptor limit, `nofile`, to a sufficiently high value so that Envoy does not fail to open sockets. |
| `entrypoint` | list | A custom entrypoint binary is used to run Envoy, in order to facilitate graceful shutdown. |
| `command` | list | The startup command. See below for details.
The following is the recommended `command` script for the Consul agent.
This is the same as the above `command` field, but is unescaped and has comments added.
```shell
# Copy the consul binary to a shared volume for mesh-init to use to generate Envoy configuration.
cp /bin/consul /bin/consul-inject/consul
# At runtime, determine the IP address assigned to this ECS Task.
ECS_IPV4=$(curl -s $ECS_CONTAINER_METADATA_URI_V4 | jq -r '.Networks[0].IPv4Addresses[0]')
# Define the Consul agent configuration file.
cat << EOF > /consul/agent-defaults.hcl
addresses = {
dns = "127.0.0.1"
grpc = "127.0.0.1"
http = "127.0.0.1"
}
advertise_addr = "$ECS_IPV4"
advertise_reconnect_timeout = "15m"
client_addr = "0.0.0.0"
datacenter = "dc1"
enable_central_service_config = true
leave_on_terminate = true
ports {
grpc = 8502
}
retry_join = ["<consul server location>"]
telemetry {
disable_compat_1.9 = true
}
EOF
# Start the consul agent.
exec consul agent \
-data-dir /consul/data \
-config-file /consul/agent-defaults.hcl
```
You can tailor the configuration above for your use case, but it's important to set the following fields as show:
| Field name | Type | Description |
| -------------------- | ------- | ------------------------------------------------------------------------------------------------------------ |
| `addresses.*` | strings | Set the DNS, GRPC, and HTTP addresses to `127.0.0.1` to ensure these are not accessible outside of the task. |
| `advertise_addr` | string | This must be set to the task IP address so that other Consul agents know how to reach this agent. |
| `client_addr` | string | This must bind to an interface reacable by other Consul agents. |
| `leave_on_terminate` | boolean | This ensures this Consul agent will leave the cluster gracefully before exiting. |
| `retry_join` | string | This must be set to your Consul server location(s) so this agent can join the Consul cluster. |
-> **NOTE**: It is important to use `exec` to start the Consul agent, so that the Consul agent runs as PID 1. This ensures
the Consul agent directly receives signals from ECS, which is important for graceful shutdown of the Consul agent.
## `mesh-init` container
The mesh-init container runs at task startup to setup this instance for Consul service mesh.
It will register this service and proxy with Consul and write Envoy bootstrap configuration
to a shared volume.
```json
{
"containerDefinitions": [
{
"name": "example-client-app",
"image": "docker.io/org/my_task:v0.0.1",
...
},
{
"name": "sidecar-proxy",
"image": "envoyproxy/envoy-alpine:<ENVOY_VERSION>",
...
},
{
"name": "consul-client"
"image": "public.ecr.aws/hashicorp/consul:<CONSUL_VERSION>",
...
},
{
"name": "consul-ecs-mesh-init",
"image": "public.ecr.aws/hashicorp/consul-ecs:<CONSUL_ECS_VERSION>",
"command": ["mesh-init"],
"essential": false,
"environment": [
{
"name": "CONSUL_ECS_CONFIG_JSON",
"value": "{\"bootstrapDir\":\"/consul\",\"healthSyncContainers\":[],\"proxy\":{\"upstreams\":[{\"destinationName\":\"example-server-app\",\"localBindPort\":1234}]},\"service\":{\"checks\":[],\"meta\":{},\"name\":\"example-client-app\",\"port\":9090,\"tags\":[]}}"
}
],
"mountPoints": [
{
"readOnly": false,
"containerPath": "/consul",
"sourceVolume": "consul_data"
},
{
"readOnly": true,
"containerPath": "/bin/consul-inject",
"sourceVolume": "consul_binary"
}
]
}
]
}
```
| Field name | Type | Description |
| ----------- | ------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `name` | string | The container name should be `consul-ecs-mesh-init`. |
| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. |
| `mountPoints` | list | The mounts two volumes so that mesh-init can invoke the `consul` binary to write bootstrap Envoy configuration. |
| `command` | list | Set the `["mesh-init"]` so that the container runs the `consul-ecs mesh-init` command. |
| `environment` | list | This must include the `CONSUL_ECS_CONFIG_JSON` variable. See below for details. |
Additional configuration is passed to the `consul-ecs mesh-init` command in JSON format using the `CONSUL_ECS_CONFIG_JSON` environment variable.
Here is the sample config from above, expanded to be readable:
```json
{
"bootstrapDir": "/consul",
"healthSyncContainers": [],
"proxy": {
"upstreams": [
{
"destinationName": "example-server-app",
"localBindPort": 1234
}
]
},
"service": {
"checks": [],
"meta": {},
"name": "example-client-app",
"port": 9090,
"tags": []
}
}
```
| Field name | Type | Description |
| ----------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------ |
| `bootstrapDir` | string | This is the path of a shared volume the is mounted to other containers, where `mesh-init` will write out Envoy configuration. |
| `proxy.upstreams` | list | The upstream services that your application calls over the service mesh, if any. |
| `service.name` | string | The name used to register this service into the Consul service catalog. |
| `service.port` | number | The port your application listens on. Set to `0` if your application does not listen on any port. |
| `service.checks` | list | Consul [checks](/docs/discovery/checks) to include, to have Consul run health checks against your application. |
See the [`consul-ecs JSON Schema`](https://github.com/hashicorp/consul-ecs/blob/main/config/schema.json) for a complete reference of fields.
## `consul-ecs-health-sync` container
Optionally, Consul ECS can sync health checks for this task into Consul checks.
This allows you to configure a health check for your application in one place, and
see a consistent health status in both ECS and Consul.
First, you'll need to add an ECS health check command to the container definition for your application.
This configures a health check command that runs `curl localhost:9090/health` to check the application health:
```json
{
"containerDefinitions": [
{
"name": "example-client-app",
"image": "docker.io/org/my_task:v0.0.1",
"healthCheck": {
"retries": 3,
"command": ["CMD-SHELL", "curl localhost:9090/health"],
"timeout": 5,
"interval": 30
},
...
},
...
]
}
```
See the [ECS health check documentation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definition_healthcheck) for details on defining ECS health checks.
Next, you must tell Consul ECS which containers will have their health status synced into Consul. To do this,
set the `healthSyncContainers` field of the `CONSUL_ECS_CONFIG_JSON` variable to include your application container name.
Here is the expanded configuration:
```json
{
"bootstrapDir": "/consul",
"healthSyncContainers": ["example-client-app"],
"proxy": {
"upstreams": [
{
"destinationName": "example-server-app",
"localBindPort": 1234
}
]
},
"service": {
"checks": [],
"meta": {},
"name": "example-client-app",
"port": 9090,
"tags": []
}
}
```
Then, update the task definition so that the `consul-ecs-mesh-init` container uses new configuration.
You should compact and escape the JSON configuration above, and copy the result into the `CONSUL_ECS_CONFIG_JSON`
environment variable:
```json
{
"containerDefinitions": [
{
"name": "consul-ecs-mesh-init",
"image": "public.ecr.aws/hashicorp/consul-ecs:<VERSION>",
"environment": [
{
"name": "CONSUL_ECS_CONFIG_JSON",
"value": "{\"bootstrapDir\":\"/consul\",\"healthSyncContainers\":[\"example-client-app\"],\"proxy\":{\"upstreams\":[{\"destinationName\":\"example-server-app\",\"localBindPort\":1234}]},\"service\":{\"checks\":[],\"meta\":{},\"name\":\"example-client-app\",\"port\":9090,\"tags\":[]}}"
}
],
...
},
...
]
}
```
Finally, include the `consul-ecs-health-sync` container in the `containerDefinitions` list.
Be sure to pass the same value as above for the `CONSUL_ECS_CONFIG_JSON` environment
variable:
```json
{
"containerDefinitions": [
{
"name": "example-client-app",
"image": "docker.io/org/my_task:v0.0.1",
...
},
{
"name": "sidecar-proxy",
"image": "envoyproxy/envoy-alpine:<ENVOY_VERSION>",
...
},
{
"name": "consul-client"
"image": "public.ecr.aws/hashicorp/consul:<CONSUL_VERSION>",
...
},
{
"name": "consul-ecs-mesh-init",
"image": "public.ecr.aws/hashicorp/consul-ecs:<CONSUL_ECS_VERSION>",
...
},
{
"name": "consul-ecs-health-sync",
"image": "public.ecr.aws/hashicorp/consul-ecs:<CONSUL_ECS_VERSION>",
"command": ["health-sync"],
"essential": false,
"dependsOn": [
{
"containerName": "consul-ecs-mesh-init",
"condition": "SUCCESS"
}
],
"environment": [
{
"name": "CONSUL_ECS_CONFIG_JSON",
"value": "{\"bootstrapDir\":\"/consul\",\"healthSyncContainers\":[\"example-client-app\"],\"proxy\":{\"upstreams\":[{\"destinationName\":\"example-server-app\",\"localBindPort\":1234}]},\"service\":{\"checks\":[],\"meta\":{},\"name\":\"example-client-app\",\"port\":9090,\"tags\":[]}}"
}
]
}
]
}
```
| Field name | Type | Description |
| ------------- | ------ | ----------------------------------------------------------------------------------------------------------------- |
| `name` | string | The container name which should be `consul-ecs-health-sync`. |
| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. |
| `command` | list | Set to `["health-sync"]` to run the `consul-ecs health-sync` command. |
| `dependsOn` | list | The `health-sync` container should not start until `mesh-init` has finished service and proxy registration. |
| `environment` | list | Set the `CONSUL_ECS_CONFIG_JSON` variable to pass configuration to the `consul-ecs health-sync command. |
# Next Steps
* Create the task definition using the [AWS Console](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html) or the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/ecs/register-task-definition.html), or another method of your choice.
* Create an [ECS Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) to start tasks using the task definition.
Reference in New Issue View Git Blame Copy Permalink