status-im/consul
2
0
Fork 0
mirror of https://github.com/status-im/consul.git synced 2025-01-28 14:34:59 +00:00
Code Issues Projects Releases Wiki Activity

Merge pull request #11048 from hashicorp/pglass/consul-ecs-migrate-task-doc

Browse Source 
docs: Migrate Existing Tasks page for ECS
This commit is contained in:
Paul Glass 2021-09-16 12:34:02 -05:00 committed by GitHub
commit b861ef6fbc
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 159 additions and 136 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

155
website/content/docs/ecs/get-started/install.mdx
View File

@ -1,82 +1,28 @@
--- ---
layout: docs layout: docs
page_title: Install - AWS ECS page_title: Installation - AWS ECS
description: >- description: >-
Install Consul Service Mesh on AWS ECS (Elastic Container Service). Install Consul Service Mesh on AWS ECS (Elastic Container Service).
--- ---
# Install # Installation
Installing Consul on ECS is a multi-part process: Installing Consul on ECS is a multi-part process:
1. [**Terraform:**](#terraform) Your tasks must be specified in Terraform using [`ecs_task_definition`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition) 1. [**Task Module:**](#task-module) Define the [`mesh-task` Terraform module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task)
and [`ecs_service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) resources. to create a task definition with the necessary sidecar containers for your application to join the service mesh.
1. [**Task Module:**](#task-module) You can then take your `ecs_task_definition` resources and copy their configuration into a new [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task)
resource that will add the necessary containers to the task definition.
1. [**Routing:**](#routing) With your tasks as part of the mesh, you must specify their upstream 1. [**Routing:**](#routing) With your tasks as part of the mesh, you must specify their upstream
services and change the URLs the tasks are using so that they're making requests services and change the URLs the tasks are using so that they're making requests through the service mesh.
through the service mesh.
1. [**Bind Address:**](#bind-address) Now that all communication is flowing through the service mesh, 1. [**Bind Address:**](#bind-address) Now that all communication is flowing through the service mesh,
you should change the address your application is listening on to `127.0.0.1` you should change the address your application is listening on to `127.0.0.1`
so that it only receives requests through the sidecar proxy. so that it only receives requests through the sidecar proxy.
-> **NOTE:** This page assumes you're familiar with ECS. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for more details. -> **NOTE:** This page assumes you're familiar with ECS. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for more details.
## Terraform
Your tasks must first be specified in Terraform using [`ecs_task_definition`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition)
and [`ecs_service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) resources so that
they can later be converted to use the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task).
For example, your tasks should be defined with Terraform similar to the following:
```hcl
resource "aws_ecs_task_definition" "my_task" {
family = "my_task"
requires_compatibilities = ["FARGATE"]
network_mode = "awsvpc"
cpu = 256
memory = 512
container_definitions = jsonencode(
[{
name = "example-client-app"
image = "docker.io/org/my_task:v0.0.1"
essential = true
portMappings = [
{
containerPort = 9090
hostPort = 9090
protocol = "tcp"
}
]
cpu = 0
mountPoints = []
volumesFrom = []
}]
)
}
resource "aws_ecs_service" "my_task" {
name = "my_task"
cluster = "arn:aws:ecs:us-east-1:111111111111:cluster/my-cluster"
task_definition = aws_ecs_task_definition.my_task.arn
desired_count = 1
network_configuration {
subnets = ["subnet-abc123"]
}
launch_type = "FARGATE"
}
```
## Task Module ## Task Module
In order to add the necessary sidecar containers for your task to join the mesh, In order to add the necessary sidecar containers for your task to join the mesh,
you must use the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task). you must use the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task):
The module will reference the same inputs as your old ECS task definition but it will
create a new version of the task definition with additional containers.
The `mesh-task` module is used as follows:
```hcl ```hcl
module "my_task" { module "my_task" {
@ -107,66 +53,25 @@ module "my_task" {
} }
``` ```
All possible inputs are documented on the [module reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task?tab=inputs) All possible inputs are documented on the [module reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task?tab=inputs),
however there are some important inputs worth highlighting: however there are some important inputs worth highlighting:
- `family` is used as the [task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family) - `family` is used as the [task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family)
but it's also used as the name of the service that gets registered in Consul. but it's also used as the name of the service that gets registered in Consul.
- `container_definitions` accepts an array of [container definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definitions). - `container_definitions` accepts an array of [container definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definitions).
These are your application containers and this should be set to the same value as what you This is where you include application containers.
were passing into the `container_definitions` key in the `aws_ecs_task_definition` resource
without the `jsonencode() function`.
For example, if your original task definition looked like:
```hcl
resource "aws_ecs_task_definition" "my_task" {
...
container_definitions = jsonencode(
[
{
name = "example-client-app"
image = "docker.io/org/my_task:v0.0.1"
essential = true
...
}
]
)
}
```
Then you would remove the `jsonencode()` function and use the rest of the value
as the input for the `mesh-task` module:
```hcl
module "my_task" {
source = "hashicorp/consul/aws-ecs//modules/mesh-task"
version = "<latest version>"
...
container_definitions = [
{
name = "example-client-app"
image = "docker.io/org/my_task:v0.0.1"
essential = true
...
}
]
}
```
- `port` is the port that your application listens on. This should be set to a - `port` is the port that your application listens on. This should be set to a
string, not an integer, i.e. `port = "9090"`, not `port = 9090`. string, not an integer, i.e. `port = "9090"`, not `port = 9090`.
- `consul_server_service_name` should be set to the name of the ECS service for - `retry_join` is passed to the [`-retry-join`](/docs/agent/options#_retry_join) option for the Consul agent. This tells
the Consul dev server. This is an output of the `dev-server` module so it the agent the location of your Consul server so that it can join the Consul cluster.
can be referenced, e.g. `consul_server_service_name = module.dev_consul_server.ecs_service_name`.
The `mesh-task` module will create a new version of your task definition with the -> **NOTE:** If your tasks run in a public subnet, they must have `assign_public_ip = true`
necessary sidecar containers added so you can delete your existing `aws_ecs_task_definition` in their [`network_configuration`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service#network_configuration) block so that ECS can pull the Docker images.
resource.
Your `aws_ecs_service` resource can remain unchanged except for the `task_definition` ## ECS Service
input which should reference the new module's output of the task definition's ARN:
To define an ECS Service, reference the mesh-task module's `task_definition_arn` output value
in your `aws_ecs_service` resource:
```hcl ```hcl
resource "aws_ecs_service" "my_task" { resource "aws_ecs_service" "my_task" {
@ -175,9 +80,6 @@ resource "aws_ecs_service" "my_task" {
} }
``` ```
-> **NOTE:** If your tasks run in a public subnet, they must have `assign_public_ip = true`
in their [`network_configuration`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service#network_configuration) block so that ECS can pull the Docker images.
After running `terraform apply`, you should see your tasks registered in After running `terraform apply`, you should see your tasks registered in
the Consul UI. the Consul UI.
@ -191,11 +93,9 @@ proxy to listen on a different port for each upstream service your application
needs to call. You then must modify your application to make requests to the sidecar needs to call. You then must modify your application to make requests to the sidecar
proxy on that port. proxy on that port.
For example, say my application `web` wants to make calls to my other application For example, if your application `web` makes calls to another application called `backend`, then you would first configure the `mesh-task` module's upstream(s):
`backend`. `backend`.
First, I must configure the `mesh-task` module's upstreams:
```hcl ```hcl
module "web" { module "web" {
family = "web" family = "web"
@ -208,19 +108,19 @@ module "web" {
} }
``` ```
I set the `destination_name` to the name of the upstream service (in this case `backend`), - Set the `destination_name` to the name of the upstream service (in this case `backend`)
and I set `local_bind_port` to an unused port. This is the port that the sidecar proxy - Set `local_bind_port` to an unused port. This is the port that the sidecar proxy
will listen on and any requests to this port will be forwarded over to the `destination_name`. will listen on. Any requests to this port will be forwarded over to the `destination_name`.
This does not have to be the port that `backend` is listening on because the service mesh This does not have to be the port that `backend` is listening on because the service mesh
will handle routing the request to the right port. will handle routing the request to the right port.
If you have multiple upstream services they'll each need to be listed here. If you have multiple upstream services they each need to be listed here.
Next, I must configure my application to make requests to `localhost:8080` when Next, configure your application to make requests to `localhost:8080` when
it wants to call the `backend` service. it wants to call the `backend` service.
For example, if my service allows configuring the URL for `backend` via the For example, if your service allows configuring the URL for `backend` via the
`BACKEND_URL` environment variable, I would set: `BACKEND_URL` environment variable, you would set:
```hcl ```hcl
module "web" { module "web" {
@ -251,7 +151,7 @@ module "web" {
To ensure that your application only receives traffic through the service mesh, To ensure that your application only receives traffic through the service mesh,
you must change the address that your application is listening on to only the loopback address you must change the address that your application is listening on to only the loopback address
(also known as `localhost`, `lo` and `127.0.0.1`) (also known as `localhost`, `lo`, and `127.0.0.1`)
so that only the sidecar proxy running in the same task can make requests to it. so that only the sidecar proxy running in the same task can make requests to it.
If your application is listening on all interfaces, e.g. `0.0.0.0`, then other If your application is listening on all interfaces, e.g. `0.0.0.0`, then other
@ -279,6 +179,7 @@ python manage.py runserver "127.0.0.1:8080"
## Next Steps ## Next Steps
- Configure a secure [Production Installation](/docs/ecs/get-started/production-installation).
- Now that your applications are running in the service mesh, read about - Now that your applications are running in the service mesh, read about
other [Service Mesh features](/docs/connect). other [Service Mesh features](/docs/connect).
- View the [Architecture](/docs/ecs/architecture) documentation to understand - View the [Architecture](/docs/ecs/architecture) documentation to understand

115
website/content/docs/ecs/get-started/migrate-existing-tasks.mdx Normal file
View File

@ -0,0 +1,115 @@
---
layout: docs
page_title: Migrate Existing Tasks - AWS ECS
description: >-
Migrate Existing Tasks
---
# Migrate Existing Tasks
This topic describes how to migrate your existing ECS Tasks to use our [`mesh-task` Terraform module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task).
## Define Tasks in Terraform
Your tasks must first be specified in Terraform using the [`ecs_task_definition`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition)
resource so that they can then be converted to use the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task).
For example, your tasks should be defined with Terraform similar to the following:
```hcl
resource "aws_ecs_task_definition" "my_task" {
family = "my_task"
requires_compatibilities = ["FARGATE"]
network_mode = "awsvpc"
cpu = 256
memory = 512
execution_role_arn = "arn:aws:iam::111111111111:role/execution-role"
task_role_arn = "arn:aws:iam::111111111111:role/task-role"
container_definitions = jsonencode(
[{
name = "example-client-app"
image = "docker.io/org/my_task:v0.0.1"
essential = true
portMappings = [
{
containerPort = 9090
hostPort = 9090
protocol = "tcp"
}
]
cpu = 0
mountPoints = []
volumesFrom = []
}]
)
}
resource "aws_ecs_service" "my_task" {
name = "my_task"
cluster = "arn:aws:ecs:us-east-1:111111111111:cluster/my-cluster"
task_definition = aws_ecs_task_definition.my_task.arn
desired_count = 1
network_configuration {
subnets = ["subnet-abc123"]
}
launch_type = "FARGATE"
}
```
## Convert to the `mesh-task` Module
In order to add the necessary sidecar containers for your task to join the mesh,
you must use the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task).
The `mesh-task` module uses inputs similar to your old ECS task definition but
creates a new version of the task definition with additional containers.
The `mesh-task` module is used as follows:
```hcl
module "my_task" {
source = "hashicorp/consul/aws-ecs//modules/mesh-task"
version = "<latest version>"
family = "my_task"
container_definitions = [
{
name = "example-client-app"
image = "docker.io/org/my_task:v0.0.1"
essential = true
portMappings = [
{
containerPort = 9090
hostPort = 9090
protocol = "tcp"
}
]
cpu = 0
mountPoints = []
volumesFrom = []
}
]
port = "9090"
retry_join = "<address of the Consul server>"
}
```
The main differences are:
- You must remove the `execution_role_arn` and `task_role_arn` fields. The `mesh-task` module will create the task and execution roles.
- You must set the `port` field to the port that your application listens on.
If your application has no listening port, set `outbound_only = true` and remove the `port` field.
- You must add the `retry_join` field. This specifies the location of your Consul servers so that your task can join the mesh.
- You must remove the `jsonencode()` function from the `container_definitions` field.
The `mesh-task` module will create a new version of your task definition with the
necessary sidecar containers added so you can delete your existing `aws_ecs_task_definition`
resource.
## Next Steps
Now that your task(s) are migrated to the `mesh-task` module,
- Start at the [ECS Service section](/docs/ecs/get-started/install#ecs-service) of the Installation Guide to continue installing Consul on ECS.
- Refer to the [`mesh-task` reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task?tab=inputs) for all available inputs to your mesh tasks.

1
website/content/docs/ecs/get-started/production-installation.mdx
View File

@ -106,4 +106,3 @@ module "my_task" {
Now you can deploy your services! Follow the rest of the steps in the [Installation instructions](/docs/ecs/get-started/install#task-module) Now you can deploy your services! Follow the rest of the steps in the [Installation instructions](/docs/ecs/get-started/install#task-module)
to deploy and connect your services. to deploy and connect your services.

12
website/data/docs-nav-data.json
View File

@ -566,20 +566,28 @@
"title": "Get Started", "title": "Get Started",
"routes": [ "routes": [
{ {
"title": "Example Installation", "title": "Example Installation on ECS Fargate",
"href": "https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/examples/dev-server-fargate" "href": "https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/examples/dev-server-fargate"
}, },
{
"title": "Example Installation on ECS EC2",
"href": "https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/examples/dev-server-ec2"
},
{ {
"title": "Requirements", "title": "Requirements",
"path": "ecs/get-started/requirements" "path": "ecs/get-started/requirements"
}, },
{ {
"title": "Install", "title": "Installation",
"path": "ecs/get-started/install" "path": "ecs/get-started/install"
}, },
{ {
"title": "Production Installation", "title": "Production Installation",
"path": "ecs/get-started/production-installation" "path": "ecs/get-started/production-installation"
},
{
"title": "Migrate Existing Tasks",
"path": "ecs/get-started/migrate-existing-tasks"
} }
] ]
}, },