mirror of https://github.com/status-im/consul.git
191 lines
12 KiB
Plaintext
191 lines
12 KiB
Plaintext
|
---
|
||
|
|
layout: docs
|
||
|
|
page_title: Automate Lambda Function Registeration
|
||
|
|
description: >-
|
||
|
|
Register AWS Lambda functions with Consul service mesh using the Consul Lambda registrator. The Consul Lambda registrator automates Lambda function registration.
|
||
|
|
---
|
||
|
|
|
||
|
|
# Automate Lambda Function Registeration
|
||
|
|
|
||
|
|
This topic describes how to automate Lambda function registration using the Consul Lambda registrator module for Terraform.
|
||
|
|
|
||
|
|
## Introduction
|
||
|
|
|
||
|
|
You can deploy the Lambda registrator to your environment to automatically register and deregister Lambda functions with Consul based on the function's tags. Refer to the [AWS Lambda tags documentation](https://docs.aws.amazon.com/lambda/latest/dg/configuration-tags.html) to learn about tags.
|
||
|
|
|
||
|
|
The registrator also stores and periodically updates information required to make mTLS requests to upstream services in the AWS parameter store. This enables Lambda functions to invoke mesh services. Refer to [Invoke Services from Lambda Functions](TODO) for additional information.
|
||
|
|
|
||
|
|
The registrator runs as a Lambda function that is invoked by AWS EventBridge. Refer to the [AWS EventBridge documentation](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-what-is.html) for additional information.
|
||
|
|
|
||
|
|
EventBridge invokes the registrator using either [AWS CloudTrail](https://docs.aws.amazon.com/lambda/latest/dg/logging-using-cloudtrail.html) to synchronize with Consul in real-time or in scheduled intervals.
|
||
|
|
|
||
|
|
CloudTrail events typically synchronize updates, registration, and deregistration within one minute, but events may occasionally be delayed.
|
||
|
|
|
||
|
|
Scheduled events fully synchronize functions between Lambda and Consul to prevent entropy. By default, EventBridge triggers a full sync every five minutes.
|
||
|
|
|
||
|
|
The following diagram shows the flow of events from EventBridge into Consul:
|
||
|
|
|
||
|
|
|
||
|
|
<ImageConfig width={500}>
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
</ImageConfig>
|
||
|
|
|
||
|
|
1. EventBridge invokes the Lambda registrator based on CloudTrail Lambda events or a schedule.
|
||
|
|
1. Lambda registrator determines how to reconcile Lambda's control plane state
|
||
|
|
with Consul state and ensures they are in sync by registering, updating, and
|
||
|
|
deregistering Lambda services.
|
||
|
|
|
||
|
|
## Requirements
|
||
|
|
|
||
|
|
Verify that your environment meets the requirements specified in [Lambda Function Registration Requirements](/docs/lambda/registration/index).
|
||
|
|
|
||
|
|
## Configuration
|
||
|
|
|
||
|
|
The Lambda registrator module stores data in the AWS parameter store. You can configure the type of data stored and how to store it.
|
||
|
|
|
||
|
|
### Optional: Store the CA certificate in Parameter Store
|
||
|
|
|
||
|
|
When Lambda registrator makes a request to Consul's [HTTP API](/api-docs) over HTTPS and the Consul API is signed by a custom CA, Lambda registrator uses the CA certificate stored in AWS Parameter Store (refer to the [Parameter Store documentation](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html) for additional information) to verify the authenticity of the Consul API. You can apply the following Terraform configuration to store Consul's server CA in Parameter Store:
|
||
|
|
|
||
|
|
```hcl
|
||
|
|
resource "aws_ssm_parameter" "ca-cert" {
|
||
|
|
name = "/lambda-registrator/ca-cert"
|
||
|
|
type = "SecureString"
|
||
|
|
value = <VALUE>
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
### Optional: Store the ACL token in Parameter Store
|
||
|
|
|
||
|
|
If [Consul access control lists (ACLs)](/docs/security/acl) are enabled, Lambda registrator must present an ACL token stored in Parameter Store to access resources. You can use the Consul CLI, API, or the Terraform provider to facilitate the ACL workflow. The following procedure describes how to create and store a token from the command line:
|
||
|
|
|
||
|
|
1. Create an ACL policy that includes the following rule:
|
||
|
|
|
||
|
|
<CodeBlockConfig filename="rules.hcl">
|
||
|
|
|
||
|
|
```hcl
|
||
|
|
service_prefix "" {
|
||
|
|
policy = "write"
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
</CodeBlockConfig>
|
||
|
|
|
||
|
|
1. Issue `consul acl policy create` command to create the policy. The following example creates a policy called `lambda-registrator-policy` containing permissions specified in `rules.hcl`:
|
||
|
|
```shell-session
|
||
|
|
$ consul acl policy create -name "lambda-registrator-policy" -rules @rules.hcl
|
||
|
|
```
|
||
|
|
|
||
|
|
1. Issue the `consul acl token create` command to create the token. The following example creates a token linked to the `lambda-registrator-policy` policy:
|
||
|
|
```shell-session
|
||
|
|
$ consul acl token create -policy-name "lambda-registrator-policy"
|
||
|
|
```
|
||
|
|
|
||
|
|
1. Store the token in Parameter Store by applying the following Terraform:
|
||
|
|
```hcl
|
||
|
|
resource "aws_ssm_parameter" "acl-token" {
|
||
|
|
name = "/lambda-registrator/acl-token"
|
||
|
|
type = "SecureString"
|
||
|
|
value = <VALUE>
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
### Optional: Store extension data in Parameter Store
|
||
|
|
|
||
|
|
If you want to enable Lambda functions to invoke services in the mesh, then you must specify a non-empty string in the `consul_extension_data_prefix` configuration. The string represents a path in the AWS Parameter Store so that the registrator can store data necessary for making mTLS requests to upstream services and update the data periodically. If the path does not exist it will be created.
|
||
|
|
|
||
|
|
Lambda registrator encrypts and stores all data for Lambda functions in the AWS Parameter Store according to the [Lambda registrator configuration options](#lambda-registrator-configuration-options). The data is stored in the following directory as a `SecureString` type:
|
||
|
|
|
||
|
|
`${var.consul_extension_data_prefix}/${<partition>}/${<namespace>}/${<service_name>}`
|
||
|
|
|
||
|
|
The registrator also requires the following IAM permissions to access the parameter store:
|
||
|
|
|
||
|
|
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"Version": "2012-10-17",
|
||
|
|
"Statement": [
|
||
|
|
{
|
||
|
|
"Effect": "Allow",
|
||
|
|
"Action": ["ssm:PutParameter","ssm:DeleteParameter"],
|
||
|
|
"Resource": "arn:aws:ssm:us-east-2:123456789012:parameter/${var.consul_extension_data_prefix}/*"
|
||
|
|
},
|
||
|
|
]
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
### Lambda Registrator configuration options
|
||
|
|
|
||
|
|
| Name | Description |
|
||
|
|
| - | - |
|
||
|
|
| `name` | Specifies the name name of the Lambda function associated with the Lambda registrator. The name is also used to construct the Identity and Access Management (IAM) role and policy names used by the Lambda function. |
|
||
|
|
| `schedule_frequency_in_minutes` | Specifies the interval in minutes that EventBridge uses to trigger a full synchronization. Default is `10`. |
|
||
|
|
| `timeout` | The maximum number of seconds Lambda registrator can run per invocation before timing out. |
|
||
|
|
| `consul_http_addr` | Specifies the address of the Consul API client. |
|
||
|
|
| `consul_datacenter` | Specifies the Consul datacenter to synchronize with AWS Lambda state data. By default, the Lambda registrator manages Lambda services for all Consul datacenters. When configured for a specific datacenter, Lambda registrator only manages Lambda services with a matching datacenter tag. Refer to [Supported tags](#supported-tags) for additional information. |
|
||
|
|
| `consul_extension_data_prefix` | Specifies the path prefix in the AWS Parameter Store under which the registrator manages mTLS data. If Lambda functions call mesh services, the value must be set to a non-empty string starting with `/`. |
|
||
|
|
| `consul_ca_cert_path` | Specifies the path to the CA certificate stored in the AWS Parameter Store. When Lambda registrator makes an HTTPS request to Consul's API and the Consul API is signed by a custom CA, Lambda registrator uses this CA certificate to verify the authenticity of the Consul API. At startup, Lambda registrator pulls the CA certificate at this path from Parameter Store, writes the certificate to the filesystem and stores the path of that file in `CONSUL_CACERT`. Also see [Optional: Store the CA Certificate in Parameter Store](#optional-store-the-ca-certificate-in-parameter-store)|
|
||
|
|
| `consul_http_token_path` | Specifies the path to the ACL token stored in AWS Parameter Store that Lambda registrator presents to access resources. This parameter only required when ACLs are enabled for the Consul server. It is used to fetch an ACL token from Parameter Store and is stored in the `CONSUL_HTTP_TOKEN` environment variable. Also see [Optional: Store the ACL Token in Parameter Store](#optional-store-the-acl-token-in-parameter-store)|
|
||
|
|
| `node_name` | The Consul node name that Lambdas will be registered to. This defaults to `lambdas`. |
|
||
|
|
| `enterprise` | <EnterpriseAlert inline />Determines if the Consul server at `consul_http_addr` is running open source or enterprise. |
|
||
|
|
| `partitions` | <EnterpriseAlert inline />The partitions that Lambda registrator manages. |
|
||
|
|
|
||
|
|
## Deploy the Lambda registrator
|
||
|
|
|
||
|
|
1. Create a Terraform configuration and specify the `lambda-registrator` module. In the following example, the Lambda registrator is deployed to `https://consul.example.com:8501`. Refer to [the Lambda registrator module documentation](https://registry.terraform.io/modules/hashicorp/consul-lambda-registrator/aws/0.1.0-beta1/submodules/lambda-registrator) for additional usage information:
|
||
|
|
```hcl
|
||
|
|
module "lambda-registrator" {
|
||
|
|
source = "hashicorp/consul-lambda/consul-lambda-registrator"
|
||
|
|
version = "x.y.z"
|
||
|
|
name = "consul-lambda-registrator"
|
||
|
|
consul_http_addr = “https://aecfe39d629774e348a9844439f5e3c1-1471365273.us-east-1.elb.amazonaws.com:8501”
|
||
|
|
ca_cert_path = aws_ssm_parameter.ca-cert.name
|
||
|
|
http_token_path = aws_ssm_parameter.acl-token.name
|
||
|
|
consul_extension_data_prefix = “/lambda_extension_data”
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
1. Deploy Lambda registrator with `terraform apply`.
|
||
|
|
|
||
|
|
|
||
|
|
## Register Lambda functions
|
||
|
|
|
||
|
|
Lambda registrator registers Lambda functions into Consul, regardless of how the functions are
|
||
|
|
deployed. The following procedure describes how to register Lambda functions with the Lambda registrator using Terraform, but you can also deploy a Lambda function with CloudFormation, the AWS user
|
||
|
|
interface, or Cloud Development Kit (CDK):
|
||
|
|
|
||
|
|
1. Add the `aws_lambda_function` resource to your Terraform configuration and specify the name of the Lambda function.
|
||
|
|
1. Add a `tags` block to the resource and specify the tags you want to use to register the function (refer to [Supported tags](#supported-tags)).
|
||
|
|
|
||
|
|
In the following example, the `example` Lambda function is registered using the `enabled`, `payload-passthrough`, and `invocation-mode` tags:
|
||
|
|
|
||
|
|
```hcl
|
||
|
|
resource "aws_lambda_function" "example" {
|
||
|
|
…
|
||
|
|
function_name = "lambda"
|
||
|
|
tags = {
|
||
|
|
"serverless.consul.hashicorp.com/v1alpha1/lambda/enabled" = "true"
|
||
|
|
"serverless.consul.hashicorp.com/alpha/lambda/payload-passthrough" = "true"
|
||
|
|
"serverless.consul.hashicorp.com/alpha/lambda/invocation-mode" = "ASYNCHRONOUS"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
### Supported tags
|
||
|
|
|
||
|
|
The following tags are supported. The path prefix for all tags is `serverless.consul.hashicorp.com/v1alpha1/lambda`. For example, specify the following tag to enable the Lambda registrator to sync the Lambda with Consul:
|
||
|
|
|
||
|
|
`serverless.consul.hashicorp.com/v1alpha1/lambda/enabled`.
|
||
|
|
|
||
|
|
| Tag | Description |
|
||
|
|
| - | - |
|
||
|
|
| `<prefix-path>/enabled` | Enables the Lambda registrator to sync the Lambda with Consul. |
|
||
|
|
| <nobr>`<prefix-path>/payload-passthrough`</nobr> | Determines if the body Envoy receives is converted to JSON or directly passed to Lambda. This attribute is optional and defaults to `false`. |
|
||
|
|
| `<prefix-path>/invocation-mode` | Specifies the [Lambda invocation mode](https://docs.aws.amazon.com/lambda/latest/operatorguide/invocation-modes.html) Consul uses to invoke the Lambda. The default is `SYNCHRONOUS`, but `ASYNCHRONOUS` invocations are also supported. |
|
||
|
|
| `<prefix-path>/datacenter` | Specifies the Consul datacenter in which to register the service. The default is the datacenter configured for Lambda registrator. |
|
||
|
|
| `<prefix-path>/namespace` | <EnterpriseAlert inline />Specifies the Consul namespace the service will be registered in. Default is `default` if `enterprise` is enabled. |
|
||
|
|
| `<prefix-path>/partition` | <EnterpriseAlert inline />Specifies the Consul partition the service will be registered in. Defaults is `default` if `enterprise` is enabled. |
|
||
|
|
| `<prefix-path>/aliases` | Specifies a `+`-separated string of Lambda aliases that will be registered into Consul. For example, if set to `dev+staging+prod`, the `dev`, `staging`, and `prod` aliases of the Lambda function will be registered into Consul. |
|