mirror of https://github.com/status-im/consul.git
302 lines
12 KiB
Plaintext
302 lines
12 KiB
Plaintext
---
|
||
layout: docs
|
||
page_title: Invoke Services from Lambda Functions
|
||
description: >-
|
||
This topic describes how to invoke services in the mesh from Lambda functions registered with Consul.
|
||
---
|
||
|
||
# Invoke Services from Lambda Functions
|
||
|
||
This topic describes how to invoke services in the mesh from Lambda functions registered with Consul.
|
||
|
||
## Introduction
|
||
|
||
The following steps describe the process:
|
||
|
||
1. Deploy the services you want to allow Lambda to invoke.
|
||
1. (Optional) Enable L7 traffic management in the local datacenter.
|
||
1. Deploy the mesh gateway
|
||
1. Deploy the Lambda registrator
|
||
1. Invoke the Lambda function
|
||
|
||
You must add the `consul-lambda-extension` extension as a Lambda layer to enable Lambda functions to send requests to mesh services. Refer to the [AWS Lambdas documentation](https://docs.aws.amazon.com/lambda/latest/dg/invocation-layers.html) for instructions on how to add layers to your Lambda functions.
|
||
|
||
The layer runs an external Lambda extension that starts a sidecar proxy. The proxy listens on one port for each upstream service and upgrades the outgoing connections to mTLS. It then proxies the requests through to [mesh gateways](/docs/connect/gateways#mesh-gateways).
|
||
|
||
## Prerequisites
|
||
|
||
You must deploy the destination services and mesh gateway prior to deploying your Lambda service with the `consul-lambda-extension` layer. It’s not required, but you can also enable L7 traffic management in the local datacenter prior to implementing the `consul-lambda-extension` layer.
|
||
|
||
### Deploy the destination service
|
||
|
||
There are several methods for deploying services to Consul service mesh. The following example configuration deploys a service named `static-server` with Consul on Kubernetes.
|
||
|
||
```yaml
|
||
kind: Service
|
||
apiVersion: v1
|
||
metadata:
|
||
# Specifies the service name in Consul.
|
||
name: static-server
|
||
spec:
|
||
selector:
|
||
app: static-server
|
||
ports:
|
||
- protocol: TCP
|
||
port: 80
|
||
targetPort: 8080
|
||
---
|
||
apiVersion: v1
|
||
kind: ServiceAccount
|
||
metadata:
|
||
name: static-server
|
||
---
|
||
apiVersion: apps/v1
|
||
kind: Deployment
|
||
metadata:
|
||
name: static-server
|
||
spec:
|
||
replicas: 1
|
||
selector:
|
||
matchLabels:
|
||
app: static-server
|
||
template:
|
||
metadata:
|
||
name: static-server
|
||
labels:
|
||
app: static-server
|
||
annotations:
|
||
'consul.hashicorp.com/connect-inject': 'true'
|
||
spec:
|
||
containers:
|
||
- name: static-server
|
||
image: hashicorp/http-echo:latest
|
||
args:
|
||
- -text="hello world"
|
||
- -listen=:8080
|
||
ports:
|
||
- containerPort: 8080
|
||
name: http
|
||
serviceAccountName: static-server
|
||
```
|
||
|
||
### Enable L7 traffic management (optional)
|
||
|
||
Mesh gateways do not implement L7 traffic management features, but you can enable L7 in the local data center so that your service can use service resolvers, splitters, and routers.
|
||
|
||
1. Define an `exported-services` configuration entry. Refer to [Exported Services](/docs/connect/config-entries/exported-services) for additional information. The following example exports `static-server` service instances to a peered cluster specified in the `PeerName` field.
|
||
|
||
<CodeBlock filename="static-server-configuration-entry.hcl">
|
||
|
||
```hcl
|
||
Kind = "exported-services"
|
||
Name = "default"
|
||
Services = [
|
||
{
|
||
Name = "static-server"
|
||
Consumers = [
|
||
{
|
||
PeerName = "<name of the peered cluster that dials the exported service>"
|
||
}
|
||
]
|
||
}
|
||
]
|
||
```
|
||
|
||
</CodeBlock>
|
||
|
||
1. Apply the configuration using the Consul CLI or by using a custom resource definition (CRD) if Consul is running on Kubernetes. The following example shows the command line usage:
|
||
|
||
```shell-session
|
||
$ consul config write static-server-configuration-entry.hcl
|
||
```
|
||
### Deploy the mesh gateway
|
||
|
||
The mesh gateway must be running and registered to the Lambda function’s Consul datacenter. Refer to the following documentation and tutorials for instructions:
|
||
|
||
- (Mesh Gateways between Datacenters)(/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters)
|
||
- [Mesh Gateways between Admin Partitions](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-partitions)
|
||
- [Mesh Gateways between Peered Clusters](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-peers)
|
||
- [Connect Services Across Datacenters with Mesh Gateways](https://developer.hashicorp.com/consul/tutorials/developer-mesh/service-mesh-gateways)
|
||
|
||
## Deploy the Lambda extension layer
|
||
|
||
The `consul-lambda-extension` extension runs during the `init` phase of the Lambda function execution. The extension retrieves the data that the Lambda registrator has been configured to store from AWS Parameter Store and creates a lightweight TCP proxy. The proxy creates a local listener for each upstream defined in the `CONSUL_SERVICE_UPSTREAMS` environment variable.
|
||
|
||
When the Lambda function is invoked, the extension retrieves the data from the AWS Parameter Store so that the function can process requests. When the Lambda function receives a shutdown event, the extension also stops.
|
||
|
||
1. Download the `consul-lambda-extension` extension from releases.hashicorp.com:
|
||
|
||
```shell-session
|
||
curl -o consul-lambda-extension_<version>_linux_amd64.zip https://releases.hashicorp.com/consul-lambda/<version>/consul-lambda-extension_<version>_linux_amd64.zip
|
||
```
|
||
1. Create the AWS Lambda layer. You can create the layer manually using the AWS CLI or AWS Console, but we recommend using Terraform:
|
||
|
||
<CodeBlockConfig filename="consul-lambda-extension.tf">
|
||
|
||
```
|
||
resource "aws_lambda_layer_version" "consul_lambda_extension" {
|
||
layer_name = "consul-lambda-extension"
|
||
filename = "consul-lambda-extension_<version>_linux_amd64.zip"
|
||
source_code_hash = filebase64sha256("consul-lambda-extension_<version>_linux_amd64.zip")
|
||
description = "Consul service mesh extension for AWS Lambda"
|
||
}
|
||
```
|
||
|
||
</CodeBlockConfig>
|
||
|
||
## Deploy the Lambda registrator
|
||
|
||
Configure and deploy the Lambda registrator. Refer to the [registrator configuration documentation](/docs/lambda/registration/automate#configuration) and the [registrator deployment documentation](/docs/lambda/registration/automate#deploy-the-lambda-registrator) for instructions.
|
||
|
||
## Write the Lambda function code
|
||
|
||
Refer to the [AWS Lambda documentation](https://docs.aws.amazon.com/lambda/latest/dg/getting-started.html) for instructions on how to write a Lambda function. In the following example, the function calls an upstream service on port `2345`:
|
||
|
||
|
||
```go
|
||
package main
|
||
|
||
import (
|
||
"context"
|
||
"io"
|
||
"fmt"
|
||
"net/http"
|
||
"github.com/aws/aws-lambda-go/lambda"
|
||
)
|
||
|
||
type Response struct {
|
||
StatusCode int `json:"statusCode"`
|
||
Body string `json:"body"`
|
||
}
|
||
|
||
func HandleRequest(ctx context.Context, _ interface{}) (Response, error) {
|
||
resp, err := http.Get("http://localhost:2345")
|
||
fmt.Println("Got response", resp)
|
||
if err != nil {
|
||
return Response{StatusCode: 500, Body: "Something bad happened"}, err
|
||
}
|
||
|
||
if resp.StatusCode != 200 {
|
||
return Response{StatusCode: resp.StatusCode, Body: resp.Status}, err
|
||
}
|
||
|
||
defer resp.Body.Close()
|
||
|
||
b, err := io.ReadAll(resp.Body)
|
||
if err != nil {
|
||
return Response{StatusCode: 500, Body: "Error decoding body"}, err
|
||
}
|
||
|
||
return Response{StatusCode: 200, Body: string(b)}, nil
|
||
}
|
||
|
||
func main() {
|
||
lambda.Start(HandleRequest)
|
||
}
|
||
```
|
||
|
||
## Deploy the Lambda function
|
||
|
||
1. Create and apply an IAM policy that allows the Lambda function’s role to fetch the Lambda extension’s data from the Parameter Store. The following example, creates an IAM role for the Lambda function, creates an IAM policy with the necessary permissions and attaches the policy to the role:
|
||
|
||
<CodeBlockConfig filename="lambda-iam-policy.tf">
|
||
|
||
```hcl
|
||
resource "aws_iam_role" "lambda" {
|
||
name = "lambda-role"
|
||
|
||
assume_role_policy = <<EOF
|
||
{
|
||
"Version": "2012-10-17",
|
||
"Statement": [
|
||
{
|
||
"Action": "sts:AssumeRole",
|
||
"Principal": {
|
||
"Service": "lambda.amazonaws.com"
|
||
},
|
||
"Effect": "Allow",
|
||
"Sid": ""
|
||
}
|
||
]
|
||
}
|
||
EOF
|
||
}
|
||
|
||
resource "aws_iam_policy" "lambda" {
|
||
name = "lambda-policy"
|
||
path = "/"
|
||
description = "IAM policy lambda"
|
||
|
||
policy = <<EOF
|
||
{
|
||
"Version": "2012-10-17",
|
||
"Statement": [
|
||
{
|
||
"Effect": "Allow",
|
||
"Action": [
|
||
"ssm:GetParameter"
|
||
],
|
||
"Resource": "arn:aws:ssm:*:*:parameter${local.this_lambdas_extension_data_path}"
|
||
}
|
||
]
|
||
}
|
||
EOF
|
||
}
|
||
|
||
resource "aws_iam_role_policy_attachment" "lambda" {
|
||
role = aws_iam_role.lambda.name
|
||
policy_arn = aws_iam_policy.lambda.arn
|
||
}
|
||
```
|
||
</CodeBlockConfig>
|
||
|
||
1. Configure and deploy the Lambda function. Refer to the [Lambda function configuration](#lambda-function-configuration) reference for information about all available options. There are several methods for deploying Lambda functions. The following example uses Terraform to deploy a function that can invoke the `static-server` upstream service using mTLS data stored under the `/lambda_extension_data` prefix:
|
||
|
||
<CodeBlockConfig filename="lambda-function.tf">
|
||
|
||
```hcl
|
||
resource "aws_lambda_function" "example" {
|
||
…
|
||
function_name = "lambda"
|
||
role = aws_iam_role.lambda.arn
|
||
tags = {
|
||
"serverless.consul.hashicorp.com/v1alpha1/lambda/enabled" = "true"
|
||
}
|
||
variables = {
|
||
environment = {
|
||
CONSUL_MESH_GATEWAY_URI = var.mesh_gateway_http_addr
|
||
CONSUL_SERVICE_UPSTREAMS = "static-server:2345:dc1"
|
||
CONSUL_EXTENSION_DATA_PREFIX = “/lambda_extension_data”
|
||
}
|
||
}
|
||
layers = [aws_lambda_layer_version.consul_lambda_extension.arn]
|
||
```
|
||
|
||
</CodeBlockConfig>
|
||
|
||
1. Issue the `terraform apply` command and Consul automatically configures a service for the Lambda function.
|
||
|
||
|
||
### Lambda function configuration
|
||
|
||
Define the following environment variables to configure each Lambda function. The configurations apply to each Lambda function in your environment:
|
||
|
||
| Variable | Description | Default |
|
||
| --- | --- | --- |
|
||
| `CONSUL_MESH_GATEWAY_URI` | Specifies the URI where the mesh gateways that the plugin makes requests are running. The mesh gateway should be registered in the same Consul datacenter and partition that the service is running in. For optimal performance, this mesh gateway should run in the same AWS region. | none |
|
||
| <nobr>`CONSUL_EXTENSION_DATA_PREFIX`</nobr> | Specifies the prefix that the plugin pulls configuration data from. The data must be located in the following directory: <br/>`“${CONSUL_EXTENSION_DATA_PREFIX}/${CONSUL_SERVICE_PARTITION}/${CONSUL_SERVICE_NAMESPACE}/<lambda-function-name>”` | none |
|
||
| `CONSUL_SERVICE_NAMESPACE` | Specifies the Consul namespace the service is registered into. | `default` |
|
||
| `CONSUL_SERVICE_PARTITION` | Specifies the Consul partition the service is registered into. | `default` |
|
||
| `CONSUL_REFRESH_FREQUENCY` | Specifies the amount of time the extension waits before re-pulling data from the Parameter Store. Use [Go `time.Duration`](https://pkg.go.dev/time@go1.19.1#ParseDuration) string values, for example, `”30s”`. <br/>The time is added to the duration configured in the Lambda registrator `sync_frequency_in_minutes` configuration. Refer to [Lambda registrator configuration options](/docs/lambda/registration/automate#lambda-registrator-configuration-options). The combined configurations determine how stale the data may become. Lambda functions can run for up to 14 hours. We recommend configuring an acceptable value to preview stale certificates. | `“5m”` |
|
||
| `CONSUL_SERVICE_UPSTREAMS` | Specifies the upstream services that the Lambda function can call. Specify the value as an unlabelled annotation according to the [`consul.hashicorp.com/connect-service-upstreams` annotation format](/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service-upstreams) in Consul on Kubernetes. For example, `"[service-name]:[port]:[optional-datacenter]"` | none |
|
||
|
||
## Invoke the Lambda function
|
||
|
||
You can create an _intention_ in Consul prior to invoking the Lambda function. Intentions define access control for services in the mesh. Refer to [Service Mesh Intentions](/docs/connect/intentions) for additional information.
|
||
|
||
There are several ways to invoke Lambda functions. In the following example, the `aws lambda invoke` CLI command invokes the function.:
|
||
|
||
```shell-session
|
||
$ aws lambda invoke --function-name lambda-registrator-2345 /dev/stdout | cat
|
||
```
|