diff --git a/website/content/docs/nia/configuration.mdx b/website/content/docs/nia/configuration.mdx index f76c34fb1c..c9b191f876 100644 --- a/website/content/docs/nia/configuration.mdx +++ b/website/content/docs/nia/configuration.mdx @@ -68,7 +68,7 @@ consul { - `ca_path` - (string) The path to a directory of CA certs to use for talking to Consul over TLS. Can also be provided through the `CONSUL_CAPATH` environment variable. - `cert` - (string) The client cert file to use for talking to Consul over TLS. Can also be provided through the `CONSUL_CLIENT_CERT` environment variable. - `server_name` - (string) The server name to use as the SNI host when connecting via TLS. Can also be provided through the `CONSUL_TLS_SERVER_NAME` environment variable. -- `token` - (string) The ACL token to use for client communication with the local Consul agent. The token can also be provided through the `CONSUL_TOKEN` or `CONSUL_HTTP_TOKEN` environment variables. More information on the required privileges required by Consul-Terraform-Sync are available in the [Secure Consul-Terraform-Sync for Production](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-secure?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS#configure-acl-privileges-for-consul-terraform-sync) tutorial +- `token` - (string) The ACL token to use for client communication with the local Consul agent. The token can also be provided through the `CONSUL_TOKEN` or `CONSUL_HTTP_TOKEN` environment variables. More information on the required privileges required by Consul-Terraform-Sync are available in the [Secure Consul-Terraform-Sync for Production](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-secure?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS#configure-acl-privileges-for-consul-terraform-sync) tutorial - `transport` - Transport configures the low-level network connection details. - `dial_keep_alive` - (string: "30s") The amount of time for keep-alives. - `dial_timeout` - (string: "30s") The amount of time to wait to establish a connection. @@ -97,13 +97,14 @@ service { - `id` - (string) ID identifies the service for Consul-Terraform-Sync. This is used to explicitly identify the service config for a task to use. If no ID is provided, the service is identified by the service name within a [task definition](#task). - `name` - (string: required) The Consul logical name of the service (required). - `namespace` - (string: "default") The namespace of the service. If not provided, the namespace will be inferred from the Consul-Terraform-Sync ACL token, or default to the `default` namespace. -- `tag` - (string) Tag is used to filter nodes based on the tag for the service. +- `tag` - (string) **This field is deprecated in Consul-Terraform-Sync 0.2.0. Use `filter` with the `Service.Tags` selector instead.** Tag is used to filter nodes based on the tag for the service. +- `filter` - (string) Specifies the expression used to filter nodes for the service. For more details on supported filters, see the Consul documentation on [filtering service nodes](/api-docs/health#filtering-2). - `cts_user_defined_meta` - (map[string]) User-defined metadata is a map of strings that will be appended to the [service input variable](/docs/nia/installation/requirements#module-specifications) for compatible Terraform modules. Not all modules may use this value. To determine if your task uses metadata or what the expected keys and format are, reference documentation for the module(s) configured for your tasks. - If multiple tasks depend on the same service but require different metadata, you can declare different sets of metadata for the same service. Define multiple service blocks for the service with unique IDs (and identical names) for those blocks. The metadata can then be separated per task based on the service IDs. ## Task -A `task` block configures which task to run in automation for the selected services. The list of services can include services explicitly defined by a `service` block or implicitly declared by the service name. The `task` block may be specified multiple times to configure multiple tasks. +A `task` block configures which task to execute in automation. When the task should execute can be determined by the `service` block and `condition` block. The `task` block may be specified multiple times to configure multiple tasks. ```hcl task { @@ -115,6 +116,9 @@ task { source = "org/example/module" version = "1.0.0" variable_files = [] + condition "catalog-services" { + regexp = ".*" + } } ``` @@ -122,7 +126,7 @@ task { - `name` - (string: required) Name is the unique name of the task (required). A task name must start with a letter or underscore and may contain only letters, digits, underscores, and dashes. - `enabled` - (bool: true) Enable or disable a task from running and managing resources. - `providers` - (list[string]) Providers is the list of provider names the task is dependent on. This is used to map [Terraform provider configuration](#terraform-provider) to the task. -- `services` - (list[string]: required) Services is the list of logical service names or service IDs the task executes on. Consul-Terraform-Sync monitors the Consul Catalog for changes to these services and triggers the task to run. Any service value not explicitly defined by a `service` block with a matching ID is assumed to be a logical service name in the default namespace. +- `services` - (list[string]) Required depending on [`condition`](#condition) configuration. Services is the list of logical service names or service IDs the task executes on. Consul-Terraform-Sync monitors the Consul Catalog for changes to these services and triggers the task to run. Any service value not explicitly defined by a `service` block with a matching ID is assumed to be a logical service name in the default namespace. Alternative to configuring `services`, a `condition` can be configured so that the task does not trigger on changes to services (default behavior) but instead trigger on a different condition. See [Task Condition](#task-condition) configuration for more details. - `source` - (string: required) Source is the location the driver uses to fetch task dependencies. The source format is dependent on the driver. For the [Terraform driver](#terraform-driver), the source is the module path (local or remote). Read more on [Terraform module source here](https://www.terraform.io/docs/modules/sources.html). - `variable_files` - (list[string]) A list of paths to files containing variables for the task. For the [Terraform driver](#terraform-driver), these are used as Terraform [variable defintion (`.tfvars`) files](https://www.terraform.io/docs/configuration/variables.html#variable-definitions-tfvars-files) and consists of only variable name assignments. The variable assignments must match the corresponding variable declarations available by the Terraform module for the task. Consul-Terraform-Sync will generate the intermediate variable declarations to pass as arguments from the auto-generated root module to the task's module. Variables are loaded in the same order as they appear in the order of the files. Duplicate variables are overwritten with the later value. _Note: unless specified by the module, configure arguments for Terraform providers using [`terraform_provider` blocks](#terraform-provider)._ ```hcl @@ -137,6 +141,67 @@ task { - `enabled` - (bool) Enable or disable buffer periods for this task. Specifying `min` will also enable it. - `min` - (string: "5s") The minimum period of time to wait after changes are detected before triggering related tasks. - `max` - (string: "20s") The maximum period of time to wait after changes are detected before triggering related tasks. If `min` is set, the default period for `max` is 4 times the value of `min`. +- `condition` - (obj) The requirement that, when met, triggers Consul-Terraform-Sync to execute the task. When unconfigured, the default condition is to trigger the task on changes in the services configured in [`services`](#services). Only one `condition` may be configured per task. Consul-Terraform-Sync supports different types of conditions, which each have their own configuration options. See [Task Condition](#task-condition) configuration for full details on configuration options for each condition type. + +### Task Condition + +A `task` block can be optionally configured with a `condition` block to set the conditions that should be met in order to execute that particular task. Below are the different types of conditions that Consul-Terraform-Sync supports. + +#### Services Condition + +This is the default type of condition when no `condition` block is configured for a task. This condition will trigger the task on changes in services configured in [`task.services`](#services). The task can optionally be configured with a services condition block. However, this is not necessary since this is the default type of condition and there is currently no additional configuration options available for services condition. + +See [Task Execution: Services Condition](/docs/nia/tasks#services-condition) for more details on how tasks are triggered with a services condition. + +```hcl +task { + name = "services_condition_task" + description = "execute on changes to api, db, and web services" + services = ["api", "db", "web"] + providers = ["my-provider"] + source = "path/to/services-condition-module" + + // optional: declare empty services condition block + condition "services" { + // no additional configuration options currently + } +} +``` + +#### Catalog-Services Condition + +A catalog-services condition block configures a task to only execute on service registration and deregistration, more specifically on first service instance registration and last service instance deregistration respectively. The catalog-services condition has additional configuration options to specify the services that can trigger the task on registration and deregistration. + +See [Task Execution: Catalog Services Condition](/docs/nia/tasks#catalog-services-condition) for more information on how tasks are triggered with a catalog-services condition. + +```hcl +task { + name = "catalog_service_condition_task" + description = "execute on service de/registrations with name matching 'web.*'" + source = "path/to/catalog-service-module" + providers = ["my-provider"] + + // configure depending on module. provides detailed information for these + // services but does not execute task. refer to module docs on how to configure. + services = ["web-api"] + + condition "catalog-service" { + datacenter = "dc1" + namespace = "default" + regexp = "web.*" + source_includes_var = false + node_meta { + key = "value" + } + } +} +``` + +- `datacenter` - (string) The datacenter of the services to query for the task. If not provided, the datacenter will default to the datacenter of the agent that Consul-Terraform-Sync queries. +- `namespace` - (string) The namespace of the services to query for the task. If not provided, the namespace will be inferred from the Consul-Terraform-Sync ACL token or default to the `default` namespace. +- `node_meta` - (map[string]) The node metadata key/value pairs used to filter services. Only services registered at a node with the specified key/value pairs are used by the task. +- `regexp` - (string) Optional if [`task.services`](/docs/nia/configuration#services) is configured. Either `regexp` or `task.services` or both must be configured. Only services that have a name which matches the regular expression are used by the task. If not provided, `regexp` will default to an exact match on `task.services` list. For example, if `task.services = ["api", "web"]`, then `regexp` will default to `^api$|^web$`. See [Task Execution: Catalog Services Condition](/docs/nia/tasks#catalog-services-condition) for more details on the relationship between `task.services` and `regexp`. Some resources for more information on regular expressions: [regular expression syntax](https://github.com/google/re2/wiki/Syntax), [try out regular expression string matching](https://golang.org/pkg/regexp/#Regexp.MatchString). +- `source_includes_var` - (bool: false) Whether or not the module configured at [`task.source`](#source) includes the [`catalog_services` variable](/docs/nia/terraform-modules#catalog-services-variable). Please refer to the documentation of the selected module for guidance on how to configure this field. If configured inconsistently with the module, Consul-Terraform-Sync will error and exit. ## Terraform Driver diff --git a/website/content/docs/nia/index.mdx b/website/content/docs/nia/index.mdx index acac39c7ba..a27a06ce04 100644 --- a/website/content/docs/nia/index.mdx +++ b/website/content/docs/nia/index.mdx @@ -19,11 +19,13 @@ Consul-Terraform-Sync executes one or more automation tasks with the most recent ## Glossary +**Condition** - A task-level defined environmental requirement that, when met, triggers the Consul-Terraform-Sync binary to execute the related task to update network infrastructure. + **Consul-Terraform-Sync** - [GitHub repo](https://github.com/hashicorp/consul-terraform-sync) and binary/CLI name for the project that is used to perform Network Infrastructure Automation. **Network Drivers** - Network drivers are used by Consul-Terraform-Sync to execute and update network infrastructure. Drivers transform Consul service level information into downstream changes by understanding and abstracting API and resource details tied to specific network infrastructure. -**Network Infrastructure Automation (NIA)** - Enables dynamic updates to network infrastructure devices triggered by service changes. +**Network Infrastructure Automation (NIA)** - Enables dynamic updates to network infrastructure devices triggered when specific conditions, such as service changes and registration, are met. **Tasks** - A task is the translation of dynamic service information from the Consul Catalog into network infrastructure changes downstream. diff --git a/website/content/docs/nia/installation/configure.mdx b/website/content/docs/nia/installation/configure.mdx index 8edef8bf96..89f2df02a2 100644 --- a/website/content/docs/nia/installation/configure.mdx +++ b/website/content/docs/nia/installation/configure.mdx @@ -11,9 +11,9 @@ The page will cover the main components for configuring your Network Infrastruct ## Tasks -A task captures a network automation process by defining which network resources to update for a set of services as those services change over time. Configure Consul-Terraform-Sync with one or more tasks that contain a list of Consul services, a Terraform module, and various Terraform providers. +A task captures a network automation process by defining which network resources to update on a given condition. Configure Consul-Terraform-Sync with one or more tasks that contain a list of Consul services, a Terraform module, and various Terraform providers. -Within the [`task` block](/docs/nia/configuration#task), the list of services for a task represents the service layer that drives network automation. The `source` is the discovery location of the Terraform module that defines the network automation process for the task. +Within the [`task` block](/docs/nia/configuration#task), the list of services for a task represents the service layer that drives network automation. The `source` is the discovery location of the Terraform module that defines the network automation process for the task. The `condition`, not shown below, defaults to the services condition when unconfigured such that network resources are updated on changes to the list of services over time. Review the Terraform module to be used for network automation and identify the Terraform providers required by the module. If the module depends on a set of providers, include the list of provider names in the `providers` field to associate the corresponding provider configuration with the task. These providers will need to be configured later in a separate block. @@ -57,7 +57,7 @@ terraform_provider "myprovider" { Piecing it all together, the configuration file for Consul-Terraform-Sync will have several HCL blocks in addition to other options for configuring the Consul-Terraform-Sync daemon: `task`, `driver.terraform`, and `terraform_provider` blocks. -An example HCL configuration file is shown below to automate one task to execute a Terraform module for 2 services. +An example HCL configuration file is shown below to automate one task to execute a Terraform module on the condition when there are changes to two services. ```hcl log_level = "info" diff --git a/website/content/docs/nia/installation/requirements.mdx b/website/content/docs/nia/installation/requirements.mdx index 3f53b1aad8..45f990870c 100644 --- a/website/content/docs/nia/installation/requirements.mdx +++ b/website/content/docs/nia/installation/requirements.mdx @@ -72,7 +72,7 @@ If there is no existing Terraform provider, a new Terraform provider can be [cre ## Network Integration (using a Terraform Module) -The Terraform module for a task in Consul-Terraform-Sync is the core component of the integration. It declares which resources and how your infrastructure is dynamically updated. +The Terraform module for a task in Consul-Terraform-Sync is the core component of the integration. It declares which resources and how your infrastructure is dynamically updated. The module along with how it is configured within a task determines the condition under which your infrastructure is updated. Working with a Terraform provider, you can write an integration task for Consul-Terraform-Sync by [creating a Terraform module](/docs/nia/terraform-modules) that is compatible with the Terraform driver or use a module built by partners below. diff --git a/website/content/docs/nia/network-drivers.mdx b/website/content/docs/nia/network-drivers.mdx index 6282e30536..cbabd026a8 100644 --- a/website/content/docs/nia/network-drivers.mdx +++ b/website/content/docs/nia/network-drivers.mdx @@ -19,11 +19,11 @@ On startup, Consul-Terraform-Sync downloads and installs [Terraform](https://www ~> **Note:** Although Terraform state files for task workspaces are independent, this does not guarantee the infrastructure changes from concurrent task executions are independent. Ensure that modules across all tasks are not modifying the same resource objects or have overlapping changes that may result in race conditions during automation. -Within the Consul-Terraform-Sync configuration for a task, practitioners can select the desired module to run for the task as well the set of services for the module to execute on. Each task executed by the Terraform driver corresponds to an automated root module that calls the selected module in an isolated Terraform environment. Consul-Terraform-Sync will manage concurrent execution of these tasks. +Within the Consul-Terraform-Sync configuration for a task, practitioners can select the desired module to run for the task as well as set the condition to execute the task. Each task executed by the Terraform driver corresponds to an automated root module that calls the selected module in an isolated Terraform environment. Consul-Terraform-Sync will manage concurrent execution of these tasks. ### Root Module -The root module is simple in structure and proxies Consul service information, configuration, and other variables to the Terraform module for the task. The content of the files that make up the root module are sourced from Consul-Terraform-Sync configuration, information for task's module to use as the automation playbook, and the Consul catalog for discovering service information. +The root module is simple in structure and proxies Consul information, configuration, and other variables to the Terraform module for the task. The content of the files that make up the root module are sourced from Consul-Terraform-Sync configuration, information for task's module to use as the automation playbook, and information from Consul such as service information. Autogenerated root modules for tasks are stored in local subdirectories of the Terraform working directory. By default, the working directory `sync-tasks` is created in the current directory. To configure where Terraform configuration files are stored, set [`working_dir`](/docs/nia/configuration#working_dir) for the Terraform driver to the desired path. @@ -44,9 +44,17 @@ The following files of the root module are generated for each task. An [example - `terraform` block - The corresponding provider source and versions for the task from the configuration files are placed into this block for the root module. The Terraform backend from the configuration is also templated here. - `provider` blocks - The provider blocks generated in the root module resemble the `terraform_provider` blocks in the configuration. They have identical arguments present and are set from the intermediate variable created per provider. - `module` block - The module block is where the task's module is called as a [child module](https://www.terraform.io/docs/configuration/modules.html#calling-a-child-module). The child module contains the core logic for automation. Required and optional input variables are passed as arguments to the module. -- `variables.tf` - This file contains 2 types of variable declarations. The required `services` input variable which determines module compatibility with Consul-Terraform Sync (read more on [compatible Terraform modules](/docs/nia/terraform-modules) for more details) and various intermediate variables used to dynamically configure providers. These intermediate provider variables are interpolated from the provider blocks and arguments configured in the Consul-Terraform-Sync configuration. +- `variables.tf` - This file contains 3 types of variable declarations. The first type is the required `services` input variable which determines module compatibility with Consul-Terraform Sync (read more on [compatible Terraform modules](/docs/nia/terraform-modules) for more details). The second type is any additional Consul-Terraform-Sync provided [optional input variables](/docs/nia/terraform-modules#optional-input-variables) that the module may include. The final type is various intermediate variables used to dynamically configure providers. These intermediate provider variables are interpolated from the provider blocks and arguments configured in the Consul-Terraform-Sync configuration. - `variables.module.tf` - This file is conditionally created if there are [variables configured for the task](/docs/nia/configuration#variable_files) and contains the interpolated variable declarations that match the variables from configuration. These are then used to proxy the configured variables to the module through explicit assignment in the module block. -- `terraform.tfvars` - The variable definitions file is where the services input variable is assigned values from the Consul catalog. It is periodically updated to reflect the current state of the configured set of services for the task. -- `terraform.tfvars.tmpl` - The template file is used by Consul-Terraform-Sync to template service information from the Consul catalog by using the HashiCorp configuration and templating library ([hashicorp/hcat](https://github.com/hashicorp/hcat)). +- `terraform.tfvars` - The variable definitions file is where the services input variable and any optional Consul-Terraform-Sync input variables are assigned values from Consul. It is periodically updated, typically when the task condition is met, to reflect the current state of Consul. +- `terraform.tfvars.tmpl` - The template file is used by Consul-Terraform-Sync to template information from Consul by using the HashiCorp configuration and templating library ([hashicorp/hcat](https://github.com/hashicorp/hcat)). -> **Note:** Generated template and Terraform configuration files are crucial to the automation of tasks. Any manual changes to the files may not be preserved and could be overwritten by a subsequent update. + +### Upgrading Terraform + +Upgrading the Terraform version used by Consul-Terraform-Sync may introduce breaking changes that can impact the Terraform modules. Refer to the Terraform [upgrade guides](https://www.terraform.io/upgrade-guides/index.html) for details before upgrading. + +The following versions were identified as containing changes that may impact Terraform modules. + +- [Terraform v0.15](https://www.terraform.io/upgrade-guides/0-15.html) diff --git a/website/content/docs/nia/tasks.mdx b/website/content/docs/nia/tasks.mdx index 1bce19c07c..2b39afa712 100644 --- a/website/content/docs/nia/tasks.mdx +++ b/website/content/docs/nia/tasks.mdx @@ -30,7 +30,19 @@ A task can be either enabled or disabled using the [task cli](/docs/nia/cli/task ## Task Execution -An enabled task is executed when any change of information for services the task is configured for is detected from the Consul catalog. Execution could include one or more changes to service values, like IP address, added or removed service instance, or tags. A complete list of values that would cause a task to run are expanded below: +An enabled task can be configured to monitor and execute on different types of conditions. For example, on changes to services ([services condition](/docs/nia/tasks#services-condition)) or on service registration and deregistration ([catalog-services condition](/docs/nia/tasks#catalog-services-condition)). + +A task can also monitor but not execute on other variables that are simply included to provide additional information to the task's module. For example, a task with a catalog-services condition may execute on registration changes but additionally monitor service instances for IP information. The details of what values are monitored and what values can execute the task are determined by the module. + +Below are details on the types of execution conditions that Consul-Terraform-Sync supports. + +### Services Condition + +The services condition is the default condition to execute a task. Tasks with the services condition monitor and execute on changes to a list of configured services. + +In order to configure a task with the services condition, the list of services to trigger the task must be configured in the task's [`services`](/docs/nia/configuration#services). The task's `condition` block can be left unconfigured since it is the default. See the [Services Condition](/docs/nia/configuration#services-condition) configuration section for more details. + +The services condition operates by monitoring the [Health List Nodes For Service API](/api-docs/health#list-nodes-for-service) and executing the task on any change of information for services configured. These changes include one or more changes to service values, like IP address, added or removed service instance, or tags. A complete list of values that would cause a task to run are expanded below: | Attribute | Description | | ----------------------- | ------------------------------------------------------------------------------------------------- | @@ -49,7 +61,39 @@ An enabled task is executed when any change of information for services the task | `node_tagged_addresses` | List of explicit LAN and WAN IP addresses for the agent | | `node_meta` | List of user-defined metadata key/value pairs for the node | -Consul-Terraform-Sync automatically generates any files needed to execute the network driver for each task. See [network drivers](/docs/nia/network-drivers) for more details on the files generated for the Terraform driver. + +### Catalog-Services Condition + +Tasks with a catalog-services condition monitor and execute on service registration changes for services that satisfy the condition configuration. 'Service registration changes' specifically refers to service registration and deregistration where service registration occurs on the first service instance registration, and service deregistration occurs on the last service instance registration. Tasks with a catalog-services condition may, depending on the module, additionally monitor but not execute on service instance information. + +The catalog-services condition operates by monitoring the [Catalog List Services API](/api-docs/catalog#list-services) and executing the task when services are added or removed in the list of registered services. Note, the task does not execute on changes to the tags of the list of services. This is similar to how changes to service instance information, mentioned above, also does not execute a task. + +Below is an example configuration for a task that will execute when a service with a name that matches the "web.*" regular expression in datacenter "dc1" has a registration change. It additionally monitors but does not execute on service instance changes to "web-api" in datacenter "dc2". +```hcl +task { + name = "catalog_service_condition_task" + source = "path/to/catalog-service-module" + providers = ["my-provider"] + services = ["web-api"] + + condition "catalog-service" { + datacenter = "dc1" + regexp = "web.*" + source_includes_var = false + } +} + +service { + name = "web-api" + datacenter = "dc2" +} +``` + +The module selected for a task will likely determine whether the task should be configured with a catalog-services condition. Module authors have the option to include and use the [`catalog_services` input variable](/docs/nia/terraform-modules#catalog-services-variable) in addition to the required `services` variable. When this variable is included in the module, the user of this module will need to configure the task condition's [`source_includes_var`](/docs/nia/configuration#source_includes_var) value to be true. Catalog-Services condition modules should have the condition type and `source_includes_var` configuration value documented for users to reference when configuring a task. + +In addition to `source_includes_var`, a task's catalog-services condition has other configuration fields that can be used to specify which services' registration changes should execute the task. For example, registration changes can be filtered for services in a specific datacenter or namespace. See the [Catalog-Services Condition](/docs/nia/configuration#catalog-services-condition) configuration section for more details. + +One particular condition configuration, [`regexp`](/docs/nia/configuration#regexp), filters service registration changes by services that have a name which matches the configured regular expression. When unconfigured, the `regexp` value will default to match on any service that has a name which matches the services listed in [`task.services`](/docs/nia/configuration#services). As such, either `regexp` or `task.services` must be configured. When both are configured, see example above, the task will monitor and execute on registration changes to services matching the `regexp`, and the task will additionally monitor but not execute on service instances listed in `task.services` to optionally provide additional service information to the module. ## Task Automation diff --git a/website/content/docs/nia/terraform-modules.mdx b/website/content/docs/nia/terraform-modules.mdx index e13d622447..a97b16c308 100644 --- a/website/content/docs/nia/terraform-modules.mdx +++ b/website/content/docs/nia/terraform-modules.mdx @@ -11,11 +11,21 @@ Consul-Terraform-Sync automates execution of Terraform modules through tasks. A ## Module Specifications -Compatible modules for Consul-Terraform-Sync follow the [standard module](https://www.terraform.io/docs/modules/index.html#standard-module-structure) structure. Modules can use syntax supported by Terraform version 0.13 and newer. There are **2 required elements for compatibility:** +Compatible modules for Consul-Terraform-Sync follow the [standard module](https://www.terraform.io/docs/modules/index.html#standard-module-structure) structure. Modules can use syntax supported by Terraform version 0.13 and newer. + +### Compatibility Requirements + +Below are the two required elements for module compatibility with Consul-Terraform-Sync 1. **Root module** - Terraform has one requirement for files in the root directory of the repository to function as the primary entrypoint for the module. It should encapsulate the core logic to be used by Consul-Terraform-Sync for task automation. `main.tf` is the recommended filename for the main file where resources are created. 2. [**`services` input variable**](#services-variable) - Consul-Terraform-Sync requires all modules to have the following input variable declared within the root module. The declaration of the `services` variable can be included at the top of the suggested `variables.tf` file where other input variables are commonly declared. This variable functions as the response object from the Consul catalog API and surfaces network information to be consumed by the module. It is structured as a map of objects. +### Optional Input Variables + +Below are additional input variables provided by Consul-Terraform-Sync that may be included in a module alongside the `services` input variable. Details for each input variable contain guidance on when to include them in a module. + + - [**`catalog_services` input variable**](#catalog-services-variable) - This variable can optionally be included in modules for the [catalog-services condition](/docs/nia/tasks#catalog-services-condition). The declaration of the `catalog_services` input variable can be included at the top of the suggested `variables.tf` file alongside the required `services` input variable and other input variables. This variable functions as the response object from the [Consul catalog list services API](/api-docs/catalog#list-services) and surfaces registered services information. It is structured as a map of lists. + ## How to Create a Compatible Terraform Module You can read more on how to [create a module](https://www.terraform.io/docs/modules/index.html) or work through a [tutorial to build a module](https://learn.hashicorp.com/tutorials/terraform/module-create). Consul-Terraform-Sync is designed to integrate with any module that satisfies the specifications in the following section. @@ -36,9 +46,11 @@ resource "local_file" "consul_services" { } ``` -### Services variable +Something important to consider before authoring your module is deciding the [condition under which it will execute](/docs/nia/tasks#task-execution). This will allow you to potentially include other types of Consul-Terraform-Sync provided input variables in your module. It will also help inform your documentation and how users should configure their task for your module. -To satisfy the specification requirements for a compatible module, copy the `services` variable declaration to the `variables.tf` file. Your module can optionally have other [variable declarations](#module-input-variables) in addition to `var.services`. +### Services Variable + +To satisfy the specification requirements for a compatible module, copy the `services` variable declaration to the `variables.tf` file. Your module can optionally have other [variable declarations](#module-input-variables) and [Consul-Terraform-Sync provided input variables](/docs/nia/terraform-modules#optional-input-variables) in addition to `var.services`. ```hcl variable "services" { @@ -68,7 +80,7 @@ variable "services" { } ``` -Keys of the `services` map are unique identifiers of the service across Consul agents and data centers. Keys follow the format `service-id.node.datacenter` (or `service-id.node.namespace.datacenter` for Consul Enterprise). A complete list of attributes available for the `services` variable is included in the [documentation for Consul-Terraform-Sync tasks](/docs/nia/tasks#task-execution). +Keys of the `services` map are unique identifiers of the service across Consul agents and data centers. Keys follow the format `service-id.node.datacenter` (or `service-id.node.namespace.datacenter` for Consul Enterprise). A complete list of attributes available for the `services` variable is included in the [documentation for Consul-Terraform-Sync tasks](/docs/nia/tasks#services-condition). Terraform variables when passed as module arguments can be [lossy for object types](https://www.terraform.io/docs/configuration/types.html#conversion-of-complex-types). This allows Consul-Terraform-Sync to declare the full variable with every object attribute in the generated root module, and pass the variable to a child module that contains a subset of these attributes for its variable declaration. Modules compatible with Consul-Terraform-Sync may simplify the `var.services` declaration within the module by omitting unused attributes. For example, the following services variable has 4 attributes with the rest omitted. @@ -87,6 +99,23 @@ variable "services" { } ``` +### Catalog Services Variable + +If you are creating a module for a [catalog-service condition](/docs/nia/tasks#catalog-services-condition), then you have the option to add the `catalog_services` variable, which contains service registration and tag information. If your module would benefit from consuming this information, you can copy the `catalog_services` variable declaration to your `variables.tf` file in addition to the other variables. + +```hcl +variable "catalog_services" { + description = "Consul catalog service names and tags monitored by Consul Terraform Sync" + type = map(list(string)) +} +``` + +The keys of the `catalog_services` map are the names of the services that are registered with Consul at the given datacenter. The value for each service name is a list of all known tags for that service. + +We recommend that if you make a module with with a catalog-service condition, that you document this in the README. This way, users that want to configure a task with your module will know to configure a catalog-services [condition](/docs/nia/configuration#condition) block. + +Similarly, if you include the `catalog_services` variable in your module, we recommend that you also document this usage in the README. Users of your module will then know to set the catalog-services condition [`source_includes_var`](/docs/nia/configuration#source_includes_var) configuration to be true. When this field is set to true, Consul-Terraform-Sync will declare the `catalog_services` variable in the generated root module, and pass the variable to a child module. Therefore, if this field is configured inconsistently, Consul-Terraform-Sync will error and exit. + ### Module Input Variables Network infrastructure differs vastly across teams and organizations, and the automation needs of practitioners are unique based on their existing setup. [Input variables](https://www.terraform.io/docs/configuration/variables.html) can be used to serve as customization parameters to the module for practitioners.