mirror of https://github.com/status-im/consul.git
Update network drives to dedicated pages
removes the tabbed sections of the network drivers since the content was really long and hard to compare side-by-side. The tabbed anchors do not work as I had hoped for.
This commit is contained in:
findkim
parent
ff220eaf11
commit
cdaf23e289
|
|
@ -1,188 +0,0 @@
|
|||
---
|
||||
layout: docs
|
||||
page_title: Network Drivers
|
||||
description: >-
|
||||
Consul-Terraform-Sync Network Drivers with Terraform and Terraform Cloud
|
||||
---
|
||||
|
||||
# Network Drivers
|
||||
|
||||
Network drivers are used by Consul-Terraform-Sync to execute and update network infrastructure. Drivers transform Consul service level information into downstream changes by understanding and abstracting API and resource details tied to specific network infrastructure.
|
||||
|
||||
## Terraform
|
||||
|
||||
Consul-Terraform-Sync is a HashiCorp solution to Network Infrastructure Automation by bridging the networking features of Consul and infrastructure management with Terraform. The solution seamlessly embeds Terraform as a network driver to manage automation of Terraform modules. This expands the Consul ecosystem and taps into the rich features and community of Terraform and Terraform providers.
|
||||
|
||||
In addition to the core functionality Terraform provides Consul-Terraform-Sync for network automation, the table below highlights product features Terraform and Terraform Enterprise offers when used as network drivers.
|
||||
|
||||
| Network Driver | Description | Features |
|
||||
| -------------- | ----------- | -------- |
|
||||
| [Terraform driver](/docs/nia/configuration#terraform-driver) | Consul-Terraform-Sync automates a local installation of the [Terraform CLI](https://www.terraform.io/) | - Local Terraform execution <br /> - Local workspace directories <br /> - [Backend options](/docs/nia/configuration#backend) available for state storage <br /> |
|
||||
| [Terraform Cloud driver](/docs/nia/configuration#terraform-cloud-driver) | Consul-Terraform-Sync automates remote workspaces on [Terraform Enterprise](https://www.terraform.io/docs/cloud/index.html)<sup>1</sup> | - [Remote Terraform execution](https://www.terraform.io/docs/cloud/run/index.html) <br/> - [Secured variables](https://www.terraform.io/docs/cloud/workspaces/variables.html) <br /> - [State versions](https://www.terraform.io/docs/cloud/workspaces/state.html) <br /> - [Audit logs](https://www.terraform.io/docs/enterprise/admin/logging.html) <br /> - Run [history](https://www.terraform.io/docs/cloud/run/manage.html), [triggers](https://www.terraform.io/docs/cloud/workspaces/run-triggers.html) and [notifications](https://www.terraform.io/docs/cloud/workspaces/notifications.html) |
|
||||
|
||||
_The table does not provide a comprehensive list of features. For more information, visit the [Terraform product page](https://www.hashicorp.com/products/terraform) or [contact our sales team](https://www.hashicorp.com/contact-sales)._
|
||||
|
||||
-> <sup>1</sup> **Upcoming**: The Terraform Cloud driver is introduced in Consul-Terraform-Sync v0.3.0 and only supports [Terraform Enterprise](https://www.hashicorp.com/products/terraform/editions/enterprise), the self-hosted distribution. The upcoming v0.4.0 release will support integration with [Terraform Cloud](https://www.terraform.io/cloud), the HashiCorp managed service.
|
||||
|
||||
### Understanding Terraform Automation
|
||||
|
||||
Consul-Terraform-Sync automates Terraform execution with templated configuration to carry out infrastructure changes. The auto-generated configuration leverages input variables sourced from Consul and builds on top of reusable Terraform modules published and maintained by HashiCorp partners and the community. Consul-Terraform-Sync can also run your custom built modules that suit your team's specific network automation needs.
|
||||
|
||||
The network driver for Consul-Terraform-Sync determines how the Terraform automation operates.
|
||||
|
||||
<Tabs>
|
||||
<Tab heading="Terraform Driver">
|
||||
|
||||
On startup, Consul-Terraform-Sync:
|
||||
1. Downloads and installs [Terraform](https://www.terraform.io/downloads.html)
|
||||
2. Prepares local workspace directories. Terraform configuration and execution for each task is organized as separate [Terraform workspaces](https://www.terraform.io/docs/state/workspaces.html). The state files for tasks are independent of each other.
|
||||
3. Generates Terraform configuration files that make up the root module for each task.
|
||||
|
||||
Once all workspaces are set up, Consul-Terraform-Sync monitors the Consul catalog for service changes. When relevant changes are detected, the Terraform driver dynamically updates input variables for that task using a template to render them to a file named [`terraform.tfvars`](/docs/nia/network-drivers#terraform-tfvars). This file is passed as a parameter to the Terraform CLI when executing `terraform plan` and `terraform apply` to update your network infrastructure with the latest Consul service details.
|
||||
|
||||
#### Local Workspaces
|
||||
|
||||
Within the Consul-Terraform-Sync configuration for a task, practitioners can select the desired module to run for the task as well as set the condition to execute the task. Each task executed by the Terraform driver corresponds to an automated root module that calls the selected module in an isolated Terraform environment. Consul-Terraform-Sync will manage concurrent execution of these tasks.
|
||||
|
||||
Autogenerated root modules for tasks are mantained in local subdirectories of the Consul-Terraform-Sync working directory. Each subdirectory represents the local workspace for a task. By default, the working directory `sync-tasks` is created in the current directory. To configure where Terraform configuration files are stored, set [`working_dir`](/docs/nia/configuration#working_dir) to the desired path or configure the [`task.working_dir`](/docs/nia/configuration#working_dir-1) individually.
|
||||
|
||||
~> **Note:** Although Terraform state files for task workspaces are independent, this does not guarantee the infrastructure changes from concurrent task executions are independent. Ensure that modules across all tasks are not modifying the same resource objects or have overlapping changes that may result in race conditions during automation.
|
||||
|
||||
#### Root Module
|
||||
|
||||
The root module is simple in structure and proxies Consul information, configuration, and other variables to the Terraform module for the task. The content of the files that make up the root module are sourced from Consul-Terraform-Sync configuration, information for task's module to use as the automation playbook, and information from Consul such as service information.
|
||||
|
||||
A working directory with one task named "cts-example" would have the folder structure below when running with the Terraform driver.
|
||||
|
||||
```shell-session
|
||||
$ tree sync-tasks/
|
||||
|
||||
sync-tasks/
|
||||
└── cts-example/
|
||||
├── main.tf
|
||||
├── variables.tf
|
||||
├── terraform.tfvars
|
||||
└── terraform.tfvars.tmpl
|
||||
```
|
||||
|
||||
The following files of the root module are generated for each task. An [example of a root module created by Consul-Terraform-Sync](https://github.com/hashicorp/consul-terraform-sync/tree/master/examples) can be found in the project repository.
|
||||
|
||||
- `main.tf` - The main file contains the terraform block, provider blocks, and a module block calling the module configured for the task.
|
||||
- `terraform` block - The corresponding provider source and versions for the task from the configuration files are placed into this block for the root module. The Terraform backend from the configuration is also templated here.
|
||||
- `provider` blocks - The provider blocks generated in the root module resemble the `terraform_provider` blocks from the configuration for Consul-Terraform-Sync. They have identical arguments present and are set from the intermediate variable created per provider.
|
||||
- `module` block - The module block is where the task's module is called as a [child module](https://www.terraform.io/docs/configuration/modules.html#calling-a-child-module). The child module contains the core logic for automation. Required and optional input variables are passed as arguments to the module.
|
||||
- `variables.tf` - This file contains 3 types of variable declarations. The first type is the required `services` input variable which determines module compatibility with Consul-Terraform Sync (read more on [compatible Terraform modules](/docs/nia/terraform-modules) for more details). The second type is any additional Consul-Terraform-Sync provided [optional input variables](/docs/nia/terraform-modules#optional-input-variables) that the module may include. The final type is various intermediate variables used to configure providers. These intermediate provider variables are interpolated from the provider blocks and arguments configured in the Consul-Terraform-Sync configuration.
|
||||
- `variables.module.tf` - This file is created if there are [variables configured for the task](/docs/nia/configuration#variable_files) and contains the interpolated variable declarations that match the variables from configuration. These are then used to proxy the configured variables to the module through explicit assignment in the module block.
|
||||
- `providers.tfvars` - This file is created if there are [providers configured for the task](/docs/nia/configuration#providers) and defined [`terraform_provider` blocks](/docs/nia/configuration#terraform-provider). This file may contain sensitive information. To omit sensitive information from this file, you can [securely configure Terraform providers for Consul-Terraform-Sync](/docs/nia/configuration#securely-configure-terraform-providers) using environment variables or templating.
|
||||
- `terraform.tfvars` - The variable definitions file is where the services input variable and any optional Consul-Terraform-Sync input variables are assigned values from Consul. It is periodically updated, typically when the task condition is met, to reflect the current state of Consul.
|
||||
- `terraform.tfvars.tmpl` - The template file is used by Consul-Terraform-Sync to template information from Consul by using the HashiCorp configuration and templating library ([hashicorp/hcat](https://github.com/hashicorp/hcat)).
|
||||
|
||||
~> **Note:** Generated template and Terraform configuration files are crucial for the automation of tasks. Any manual changes to the files may not be preserved and could be overwritten by a subsequent update. Unexpected manual changes to the format of the files may cause automation to error.
|
||||
|
||||
</Tab>
|
||||
<Tab heading="Terraform Cloud Driver">
|
||||
|
||||
<EnterpriseAlert>
|
||||
This feature requires{' '}
|
||||
<a href="https://www.hashicorp.com/products/consul/features">Consul-Terraform-Sync Enterprise</a>{' '}
|
||||
which is available with <strong>Consul Enterprise</strong>.
|
||||
</EnterpriseAlert>
|
||||
|
||||
Consul-Terraform-Sync manages Terraform runs following the [API-driven run workflow](https://www.terraform.io/docs/cloud/run/api.html) for workspaces in Terraform Enterprise.
|
||||
|
||||
On startup, Consul-Terraform-Sync:
|
||||
1. Creates or discovers Terraform Enterprise workspaces corresponding to the configured tasks.
|
||||
2. Prepares the local environment and generates Terraform configuration files that make up the root module for each task.
|
||||
3. Packages the generated files and uploads them as a configuration version for the task's workspace on Terraform Enterprise.
|
||||
|
||||
Once all workspaces are set up, Consul-Terraform-Sync monitors the Consul catalog for service changes. When relevant changes are detected, the Terraform Cloud driver dynamically updates input variables for that task directly as [workspace variables](https://www.terraform.io/docs/cloud/workspaces/variables.html) using the Terraform Cloud API. The driver then queues a run on the workspace, with auto-apply enabled, to update your network infrastructure.
|
||||
|
||||
~> **Note:** Although workspaces for tasks are executed in isolated environments, this does not guarantee the infrastructure changes from concurrent task executions are independent. Ensure that modules across all tasks are not modifying the same resource objects or have overlapping changes that may result in race conditions during automation.
|
||||
|
||||
#### Remote Workspaces
|
||||
|
||||
Consul-Terraform-Sync will discover or create a new workspaces based on your configured tasks. The task `name`, `description`, and `terraform_version` are used to set the workspace name, description, and Terraform version.
|
||||
|
||||
[](/img/nia/cts-tfc-workspace.png)
|
||||
|
||||
Workspace automation requirements for Consul-Terraform-Sync:
|
||||
* Must be set to remote execution mode
|
||||
* Cannot be connected to a VCS
|
||||
* Cannot have an existing configuration version uploaded by another application
|
||||
|
||||
Other workspace settings can be pre-configured or updated, like adding a [run notification](https://www.terraform.io/docs/cloud/workspaces/notifications.html) to send messages to a Slack channel when Consul-Terraform-Sync updates your network infrastructure.
|
||||
|
||||
[](/img/nia/cts-tfc-run-notifications.png)
|
||||
|
||||
#### Configuration Version
|
||||
|
||||
An example configuration version for a task named "cts-example" would have the folder structure below when running with the Terraform Cloud driver and using the default working directory.
|
||||
|
||||
```shell-session
|
||||
$ tree sync-tasks/
|
||||
|
||||
sync-tasks/
|
||||
└── cts-example/
|
||||
├── main.tf
|
||||
└── variables.tf
|
||||
```
|
||||
|
||||
- `main.tf` - The main file contains the terraform block, provider blocks, and a module block calling the module configured for the task.
|
||||
- `terraform` block - The corresponding provider source and versions for the task from the configuration files are placed into this block for the root module.
|
||||
- `provider` blocks - The provider blocks generated in the root module resemble the `terraform_provider` blocks from the configuration for Consul-Terraform-Sync. They have identical arguments present and are set from the intermediate variable created per provider.
|
||||
- `module` block - The module block is where the task's module is called as a [child module](https://www.terraform.io/docs/configuration/modules.html#calling-a-child-module). The child module contains the core logic for automation. Required and optional input variables are passed as arguments to the module.
|
||||
- `variables.tf` - This file contains 3 types of variable declarations. The first type is the required `services` input variable which determines module compatibility with Consul-Terraform Sync (read more on [compatible Terraform modules](/docs/nia/terraform-modules) for more details). The second type is any additional Consul-Terraform-Sync provided [optional input variables](/docs/nia/terraform-modules#optional-input-variables) that the module may include. The final type is various intermediate variables used to configure providers. These intermediate provider variables are interpolated from the provider blocks and arguments configured in the Consul-Terraform-Sync configuration.
|
||||
- `variables.module.tf` - This file is created if there are [variables configured for the task](/docs/nia/configuration#variable_files) and contains the interpolated variable declarations that match the variables from configuration. These are then used to proxy the configured variables to the module through explicit assignment in the module block.
|
||||
|
||||
#### Variables
|
||||
|
||||
Consul-Terraform-Sync uses Terraform input variables to reflect the latest Consul service information and are used as parameters for your Terraform module. These input variables are dynamic and are updated by the driver throughout the runtime of Consul-Terraform-Sync.
|
||||
|
||||
You can view the latest service information in the Terraform UI by navigating to that workspace and clicking the "Variables" tab in the workspace navigation.
|
||||
|
||||
[](/img/nia/cts-tfc-workspace-variables.png)
|
||||
|
||||
~> **Caution:** Dynamic variables maintained by Consul-Terraform-Sync are formatted for automation. Unexpected manual changes to these variables may result in automation errors.
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
### Setting Up Terraform Cloud Driver
|
||||
|
||||
<EnterpriseAlert>
|
||||
This feature requires{' '}
|
||||
<a href="https://www.hashicorp.com/products/consul/features">Consul-Terraform-Sync Enterprise</a>{' '}
|
||||
which is available with <strong>Consul Enterprise</strong>.
|
||||
</EnterpriseAlert>
|
||||
|
||||
#### Deployment
|
||||
|
||||
Because a Consul-Terraform-Sync instance can only be configured with one driver, an instance can only be associated with either a Terraform driver or a Terraform Cloud driver. If there is a need to run both types of drivers, users will need to deploy a separate Consul-Terraform-Sync instance for each type of driver. Relatedly, if there is a need to run Consul-Terraform-Sync across multiple Terraform Cloud organizations, users will need to deploy a separate instance for each organization.
|
||||
|
||||
#### Required Setup
|
||||
|
||||
Prior to running Consul-Terraform-Sync with a Terraform Cloud driver, you will need to set up an account, organization, and token. We recommend that the token is a team-level token restricted to [Manage Workspaces](https://www.terraform.io/docs/cloud/users-teams-organizations/teams.html#managing-workspace-access) level permissions. Below are the steps for the recommended setup.
|
||||
|
||||
The first step is to create an account with your Terraform Cloud service. After creating an account, create an [organization](https://www.terraform.io/docs/cloud/users-teams-organizations/organizations.html#creating-organizations). The address of your Terraform Cloud service will be used to configure the [`hostname`](/docs/nia/configuration#hostname), and the organization name will be used to configure the [`organization`](/docs/nia/configuration#organization) on the Terraform Cloud driver.
|
||||
|
||||
Once you have an account and organization, the next step is to [create a team](https://www.terraform.io/docs/cloud/users-teams-organizations/teams.html). We recommend using a dedicated team and team token to run and authenticate Consul-Terraform-Sync. Using a team token rather than a user token has the benefits of allowing cross-organization permissions as well as associating Consul-Terraform-Sync automated actions with the team rather than an individual.
|
||||
|
||||
After creating a dedicated team, please configure the team's permissions with only the "Manage Workspaces" organization access-level. Consul-Terraform-Sync's main work revolves around creating and managing workspaces. Therefore restricting the dedicated team's permission to Manage Workspaces level is sufficient and reduces security risk.
|
||||
|
||||
[](/img/nia/cts-tfc-team-setup.png)
|
||||
|
||||
After setting the team's permissions, the final setup step is to [generate the associated team token](https://www.terraform.io/docs/cloud/users-teams-organizations/api-tokens.html#team-api-tokens), which can be done on the same team management page. This token will be used by Consul-Terraform-Sync for API authentication and will be used to configure the [`token`](/docs/nia/configuration#token-1) on the Terraform Cloud driver.
|
||||
|
||||
#### Additional Recommendations
|
||||
|
||||
We recommend users to configure Consul-Terraform-Sync workspaces with [run notifications](https://www.terraform.io/docs/cloud/workspaces/notifications.html) through the Terraform Cloud web application. Run notifications notify external systems about the progress of runs and could help notify users of Consul-Terraform-Sync events, particularly errored runs.
|
||||
|
||||
In order to configure a run notification, users can [manually create a notification configuration](https://www.terraform.io/docs/cloud/workspaces/notifications.html#creating-a-notification-configuration) on an existing Consul-Terraform-Sync workspace. A workspace may already exist for a task if the workspace name is identical to the configured task's [`name`](/docs/nia/configuration#name-2). This may occur if Consul-Terraform-Sync has already already run and created the workspace for the task. This may also occur if the workspace is manually created for the task prior to Consul-Terraform-Sync running.
|
||||
|
||||
### Upgrading Terraform
|
||||
|
||||
Upgrading the Terraform version used by Consul-Terraform-Sync may introduce breaking changes that can impact the Terraform modules. Refer to the Terraform [upgrade guides](https://www.terraform.io/upgrade-guides/index.html) for details before upgrading.
|
||||
|
||||
The following versions were identified as containing changes that may impact Terraform modules.
|
||||
|
||||
- [Terraform v0.15](https://www.terraform.io/upgrade-guides/0-15.html)
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
---
|
||||
layout: docs
|
||||
page_title: Network Drivers
|
||||
description: >-
|
||||
Consul-Terraform-Sync Network Drivers with Terraform and Terraform Cloud
|
||||
---
|
||||
|
||||
# Network Drivers
|
||||
|
||||
Network drivers are used by Consul-Terraform-Sync to execute and update network infrastructure. Drivers transform Consul service level information into downstream changes by understanding and abstracting API and resource details tied to specific network infrastructure.
|
||||
|
||||
## Terraform
|
||||
|
||||
Consul-Terraform-Sync is a HashiCorp solution to Network Infrastructure Automation by bridging the networking features of Consul and infrastructure management with Terraform. The solution seamlessly embeds Terraform as network drivers to manage automation of Terraform modules. This expands the Consul ecosystem and taps into the rich features and community of Terraform and Terraform providers.
|
||||
|
||||
In addition to the core functionality Terraform provides Consul-Terraform-Sync for network automation, the table below highlights product features Terraform and Terraform Enterprise offers when used as a network driver.
|
||||
|
||||
| Network Driver | Description | Features |
|
||||
| -------------- | ----------- | -------- |
|
||||
| [Terraform driver](/docs/nia/network-drivers/terraform) | Consul-Terraform-Sync automates a local installation of the [Terraform CLI](https://www.terraform.io/) | - Local Terraform execution <br /> - Local workspace directories <br /> - [Backend options](/docs/nia/configuration#backend) available for state storage <br /> |
|
||||
| [Terraform Cloud driver](/docs/nia/network-drivers/terraform-cloud) | Consul-Terraform-Sync automates remote workspaces on [Terraform Enterprise](https://www.terraform.io/docs/cloud/index.html)<sup>1</sup> | - [Remote Terraform execution](https://www.terraform.io/docs/cloud/run/index.html) <br/> - [Secured variables](https://www.terraform.io/docs/cloud/workspaces/variables.html) <br /> - [State versions](https://www.terraform.io/docs/cloud/workspaces/state.html) <br /> - [Audit logs](https://www.terraform.io/docs/enterprise/admin/logging.html) <br /> - Run [history](https://www.terraform.io/docs/cloud/run/manage.html), [triggers](https://www.terraform.io/docs/cloud/workspaces/run-triggers.html) and [notifications](https://www.terraform.io/docs/cloud/workspaces/notifications.html) |
|
||||
|
||||
_The table does not provide a comprehensive list of features. For more information, visit the [Terraform product page](https://www.hashicorp.com/products/terraform) or [contact our sales team](https://www.hashicorp.com/contact-sales)._
|
||||
|
||||
-> <sup>1</sup> **Upcoming**: The Terraform Cloud driver is introduced in Consul-Terraform-Sync v0.3.0 and only supports [Terraform Enterprise](https://www.hashicorp.com/products/terraform/editions/enterprise), the self-hosted distribution. The upcoming v0.4.0 release will support integration with [Terraform Cloud](https://www.terraform.io/cloud), the HashiCorp managed service.
|
||||
|
||||
## Understanding Terraform Automation
|
||||
|
||||
Consul-Terraform-Sync automates Terraform execution with templated configuration to carry out infrastructure changes. The auto-generated configuration leverages input variables sourced from Consul and builds on top of reusable Terraform modules published and maintained by HashiCorp partners and the community. Consul-Terraform-Sync can also run your custom built modules that suit your team's specific network automation needs.
|
||||
|
||||
The network driver for Consul-Terraform-Sync determines how the Terraform automation operates. Visit the driver pages to read more about the [Terraform driver](/docs/nia/network-drivers/terraform) and the [Terraform Cloud driver](/docs/nia/network-drivers/terraform-cloud).
|
||||
|
||||
### Upgrading Terraform
|
||||
|
||||
Upgrading the Terraform version used by Consul-Terraform-Sync may introduce breaking changes that can impact the Terraform modules. Refer to the Terraform [upgrade guides](https://www.terraform.io/upgrade-guides/index.html) for details before upgrading.
|
||||
|
||||
The following versions were identified as containing changes that may impact Terraform modules.
|
||||
|
||||
- [Terraform v0.15](https://www.terraform.io/upgrade-guides/0-15.html)
|
||||
|
|
@ -0,0 +1,109 @@
|
|||
---
|
||||
layout: docs
|
||||
page_title: Terraform Cloud Driver
|
||||
description: >-
|
||||
Consul-Terraform-Sync Network Drivers with Terraform Enterprise
|
||||
---
|
||||
|
||||
# Terraform Cloud Driver
|
||||
<EnterpriseAlert>
|
||||
This feature requires{' '}
|
||||
<a href="https://www.hashicorp.com/products/consul/features">Consul-Terraform-Sync Enterprise</a>{' '}
|
||||
which is available with <strong>Consul Enterprise</strong>.
|
||||
</EnterpriseAlert>
|
||||
|
||||
Consul-Terraform-Sync is more powerful when you integrate it with [Terraform Enterprise](https://www.hashicorp.com/products/terraform/editions/enterprise)<sup>1</sup>. Integrating with Terraform Enterprise provides features like enhanced workspaces and insight into Terraform operations as Consul-Terraform-Sync dynamically updates your network infrastructure.
|
||||
|
||||
This page describes how the Terraform Cloud driver operates within Consul-Terraform-Sync.
|
||||
|
||||
-> <sup>1</sup> **Upcoming**: The Terraform Cloud driver is introduced in Consul-Terraform-Sync v0.3.0 and only supports [Terraform Enterprise](https://www.hashicorp.com/products/terraform/editions/enterprise), the self-hosted distribution. The upcoming v0.4.0 release will support integration with [Terraform Cloud](https://www.terraform.io/cloud), the HashiCorp managed service.
|
||||
|
||||
## Terraform Workspace Automation
|
||||
|
||||
Consul-Terraform-Sync manages Terraform runs following the [API-driven run workflow](https://www.terraform.io/docs/cloud/run/api.html) for workspaces in Terraform Enterprise.
|
||||
|
||||
On startup, Consul-Terraform-Sync:
|
||||
1. Creates or discovers Terraform Enterprise workspaces corresponding to the configured tasks.
|
||||
2. Prepares the local environment and generates Terraform configuration files that make up the root module for each task.
|
||||
3. Packages the generated files and uploads them as a configuration version for the task's workspace on Terraform Enterprise.
|
||||
|
||||
Once all workspaces are set up, Consul-Terraform-Sync monitors the Consul catalog for service changes. When relevant changes are detected, the Terraform Cloud driver dynamically updates input variables for that task directly as [workspace variables](https://www.terraform.io/docs/cloud/workspaces/variables.html) using the Terraform Cloud API. The driver then queues a run on the workspace, with auto-apply enabled, to update your network infrastructure.
|
||||
|
||||
~> **Note:** Although workspaces for tasks are executed in isolated environments, this does not guarantee the infrastructure changes from concurrent task executions are independent. Ensure that modules across all tasks are not modifying the same resource objects or have overlapping changes that may result in race conditions during automation.
|
||||
|
||||
## Remote Workspaces
|
||||
|
||||
Consul-Terraform-Sync will discover or create a new workspaces based on your configured tasks. The task configuration options for `name`, `description`, and `terraform_version` are used to set the workspace name, description, and Terraform version.
|
||||
|
||||
[](/img/nia/cts-tfc-workspace.png)
|
||||
|
||||
Workspace automation requirements for Consul-Terraform-Sync:
|
||||
* Must be set to remote execution mode
|
||||
* Cannot be connected to a VCS
|
||||
* Cannot have an existing configuration version uploaded by another application
|
||||
|
||||
Other workspace settings can be pre-configured or updated, like adding a [run notification](https://www.terraform.io/docs/cloud/workspaces/notifications.html) to send messages to a Slack channel when Consul-Terraform-Sync updates your network infrastructure.
|
||||
|
||||
## Configuration Version
|
||||
|
||||
An example configuration version for a task named "cts-example" would have the folder structure below when running with the Terraform Cloud driver and using the default working directory.
|
||||
|
||||
```shell-session
|
||||
$ tree sync-tasks/
|
||||
|
||||
sync-tasks/
|
||||
└── cts-example/
|
||||
├── main.tf
|
||||
└── variables.tf
|
||||
```
|
||||
|
||||
- `main.tf` - The main file contains the terraform block, provider blocks, and a module block calling the module configured for the task.
|
||||
- `terraform` block - The corresponding provider source and versions for the task from the configuration files are placed into this block for the root module.
|
||||
- `provider` blocks - The provider blocks generated in the root module resemble the `terraform_provider` blocks from the configuration for Consul-Terraform-Sync. They have identical arguments present and are set from the intermediate variable created per provider.
|
||||
- `module` block - The module block is where the task's module is called as a [child module](https://www.terraform.io/docs/configuration/modules.html#calling-a-child-module). The child module contains the core logic for automation. Required and optional input variables are passed as arguments to the module.
|
||||
- `variables.tf` - This file contains 3 types of variable declarations. The first type is the required `services` input variable which determines module compatibility with Consul-Terraform Sync (read more on [compatible Terraform modules](/docs/nia/terraform-modules) for more details). The second type is any additional Consul-Terraform-Sync provided [optional input variables](/docs/nia/terraform-modules#optional-input-variables) that the module may include. The final type is various intermediate variables used to configure providers. These intermediate provider variables are interpolated from the provider blocks and arguments configured in the Consul-Terraform-Sync configuration.
|
||||
- `variables.module.tf` - This file is created if there are [variables configured for the task](/docs/nia/configuration#variable_files) and contains the interpolated variable declarations that match the variables from configuration. These are then used to proxy the configured variables to the module through explicit assignment in the module block.
|
||||
|
||||
## Variables
|
||||
|
||||
Consul-Terraform-Sync uses Terraform input variables to reflect the latest Consul service information and are used as parameters for your Terraform module. These input variables are dynamic and are updated by the driver throughout the runtime of Consul-Terraform-Sync.
|
||||
|
||||
You can view the latest service information in the Terraform UI by navigating to that workspace and clicking the "Variables" tab in the workspace navigation.
|
||||
|
||||
[](/img/nia/cts-tfc-workspace-variables.png)
|
||||
|
||||
~> **Caution:** Dynamic variables maintained by Consul-Terraform-Sync are formatted for automation. Unexpected manual changes to these variables may result in automation errors.
|
||||
|
||||
## Setting Up Terraform Cloud Driver
|
||||
|
||||
### Deployment
|
||||
|
||||
Because a Consul-Terraform-Sync instance can only be configured with one driver, an instance can only be associated with either a Terraform driver or a Terraform Cloud driver. If there is a need to run both types of drivers, users will need to deploy a separate Consul-Terraform-Sync instance for each type of driver. Relatedly, if there is a need to run Consul-Terraform-Sync across multiple Terraform Enterprise organizations, users will need to deploy a separate instance for each organization.
|
||||
|
||||
### Required Setup
|
||||
|
||||
This section captures requirements for setting up Consul-Terraform-Sync to integrate with your [Terraform Enterprise](https://www.hashicorp.com/products/terraform/editions/enterprise) solution.
|
||||
|
||||
1. Hostname of your Terraform Enterprise, self-hosted distribution
|
||||
1. Name of your organization
|
||||
1. [Team API token](https://www.terraform.io/docs/cloud/users-teams-organizations/api-tokens.html#team-api-tokens) used for authentication with Terraform Enterprise
|
||||
|
||||
Prior to running Consul-Terraform-Sync with a Terraform Cloud driver, you will need to have an account and organization set up, and a dedicated token. We recommend that the token is a team-level token restricted to [Manage Workspaces](https://www.terraform.io/docs/cloud/users-teams-organizations/teams.html#managing-workspace-access) level permissions. Below are the steps for the recommended setup.
|
||||
|
||||
The first step is to create an account with your Terraform Cloud service. After creating an account, create a new [organization](https://www.terraform.io/docs/cloud/users-teams-organizations/organizations.html#creating-organizations) or select an existing organization. The address of your Terraform Cloud service will be used to configure the [`hostname`](/docs/nia/configuration#hostname), and the organization name will be used to configure the [`organization`](/docs/nia/configuration#organization) on the Terraform Cloud driver.
|
||||
|
||||
Once you have an account and organization, the next step is to [create a team](https://www.terraform.io/docs/cloud/users-teams-organizations/teams.html). We recommend using a dedicated team and team token to run and authenticate Consul-Terraform-Sync. Using a team token has the benefits of restricting organization permissions as well as associating Consul-Terraform-Sync automated actions with the team rather than an individual.
|
||||
|
||||
After creating a dedicated team, update the team's permissions with "Manage Workspaces" organization access-level. Consul-Terraform-Sync's main work revolves around creating and managing workspaces. Therefore restricting the dedicated team's permission to Manage Workspaces level is sufficient and reduces security risk.
|
||||
|
||||
[](/img/nia/cts-tfc-team-setup.png)
|
||||
|
||||
After setting the team's permissions, the final setup step is to [generate the associated team token](https://www.terraform.io/docs/cloud/users-teams-organizations/api-tokens.html#team-api-tokens), which can be done on the same team management page. This token will be used by Consul-Terraform-Sync for API authentication and will be used to configure the [`token`](/docs/nia/configuration#token-1) on the Terraform Cloud driver.
|
||||
|
||||
### Recommendations
|
||||
|
||||
We recommend users to configure workspaces managed by Consul-Terraform-Sync with [run notifications](https://www.terraform.io/docs/cloud/workspaces/notifications.html) through the Terraform web application. Run notifications notify external systems about the progress of runs and could help notify users of Consul-Terraform-Sync events, particularly errored runs.
|
||||
|
||||
[](/img/nia/cts-tfc-run-notifications.png)
|
||||
|
||||
In order to configure a run notification, users can [manually create a notification configuration](https://www.terraform.io/docs/cloud/workspaces/notifications.html#creating-a-notification-configuration) for workspaces automated by Consul-Terraform-Sync. A workspace may already exist for a task if the workspace name is identical to the configured task's [`name`](/docs/nia/configuration#name-2). This may occur if Consul-Terraform-Sync has already already run and created the workspace for the task. This may also occur if the workspace is manually created for the task prior to Consul-Terraform-Sync running.
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
---
|
||||
layout: docs
|
||||
page_title: Terraform Driver
|
||||
description: >-
|
||||
Consul-Terraform-Sync Network Drivers with Terraform
|
||||
---
|
||||
|
||||
# Terraform Driver
|
||||
|
||||
Consul-Terraform-Sync extends the Consul ecosystem to include Terraform as an officially supported tooling project. With the Terraform driver, Consul-Terraform-Sync installs the [Terraform CLI](https://www.terraform.io/downloads.html) locally and runs Terraform commands based on monitored Consul changes. This page details how the Terraform driver operates using local workspaces and templated files.
|
||||
|
||||
## Terraform CLI Automation
|
||||
|
||||
On startup, Consul-Terraform-Sync:
|
||||
1. Downloads and installs Terraform
|
||||
2. Prepares local workspace directories. Terraform configuration and execution for each task is organized as separate [Terraform workspaces](https://www.terraform.io/docs/state/workspaces.html). The state files for tasks are independent of each other.
|
||||
3. Generates Terraform configuration files that make up the root module for each task.
|
||||
|
||||
Once all workspaces are set up, Consul-Terraform-Sync monitors the Consul catalog for service changes. When relevant changes are detected, the Terraform driver dynamically updates input variables for that task using a template to render them to a file named [`terraform.tfvars`](/docs/nia/network-drivers#terraform-tfvars). This file is passed as a parameter to the Terraform CLI when executing `terraform plan` and `terraform apply` to update your network infrastructure with the latest Consul service details.
|
||||
|
||||
### Local Workspaces
|
||||
|
||||
Within the Consul-Terraform-Sync configuration for a task, practitioners can select the desired module to run for the task as well as set the condition to execute the task. Each task executed by the Terraform driver corresponds to an automated root module that calls the selected module in an isolated Terraform environment. Consul-Terraform-Sync will manage concurrent execution of these tasks.
|
||||
|
||||
Autogenerated root modules for tasks are mantained in local subdirectories of the Consul-Terraform-Sync working directory. Each subdirectory represents the local workspace for a task. By default, the working directory `sync-tasks` is created in the current directory. To configure where Terraform configuration files are stored, set [`working_dir`](/docs/nia/configuration#working_dir) to the desired path or configure the [`task.working_dir`](/docs/nia/configuration#working_dir-1) individually.
|
||||
|
||||
~> **Note:** Although Terraform state files for task workspaces are independent, this does not guarantee the infrastructure changes from concurrent task executions are independent. Ensure that modules across all tasks are not modifying the same resource objects or have overlapping changes that may result in race conditions during automation.
|
||||
|
||||
### Root Module
|
||||
|
||||
The root module is simple in structure and proxies Consul information, configuration, and other variables to the Terraform module for the task. The content of the files that make up the root module are sourced from Consul-Terraform-Sync configuration, information for task's module to use as the automation playbook, and information from Consul such as service information.
|
||||
|
||||
A working directory with one task named "cts-example" would have the folder structure below when running with the Terraform driver.
|
||||
|
||||
```shell-session
|
||||
$ tree sync-tasks/
|
||||
|
||||
sync-tasks/
|
||||
└── cts-example/
|
||||
├── main.tf
|
||||
├── variables.tf
|
||||
├── terraform.tfvars
|
||||
└── terraform.tfvars.tmpl
|
||||
```
|
||||
|
||||
The following files of the root module are generated for each task. An [example of a root module created by Consul-Terraform-Sync](https://github.com/hashicorp/consul-terraform-sync/tree/master/examples) can be found in the project repository.
|
||||
|
||||
- `main.tf` - The main file contains the terraform block, provider blocks, and a module block calling the module configured for the task.
|
||||
- `terraform` block - The corresponding provider source and versions for the task from the configuration files are placed into this block for the root module. The Terraform backend from the configuration is also templated here.
|
||||
- `provider` blocks - The provider blocks generated in the root module resemble the `terraform_provider` blocks from the configuration for Consul-Terraform-Sync. They have identical arguments present and are set from the intermediate variable created per provider.
|
||||
- `module` block - The module block is where the task's module is called as a [child module](https://www.terraform.io/docs/configuration/modules.html#calling-a-child-module). The child module contains the core logic for automation. Required and optional input variables are passed as arguments to the module.
|
||||
- `variables.tf` - This file contains 3 types of variable declarations. The first type is the required `services` input variable which determines module compatibility with Consul-Terraform Sync (read more on [compatible Terraform modules](/docs/nia/terraform-modules) for more details). The second type is any additional Consul-Terraform-Sync provided [optional input variables](/docs/nia/terraform-modules#optional-input-variables) that the module may include. The final type is various intermediate variables used to configure providers. These intermediate provider variables are interpolated from the provider blocks and arguments configured in the Consul-Terraform-Sync configuration.
|
||||
- `variables.module.tf` - This file is created if there are [variables configured for the task](/docs/nia/configuration#variable_files) and contains the interpolated variable declarations that match the variables from configuration. These are then used to proxy the configured variables to the module through explicit assignment in the module block.
|
||||
- `providers.tfvars` - This file is created if there are [providers configured for the task](/docs/nia/configuration#providers) and defined [`terraform_provider` blocks](/docs/nia/configuration#terraform-provider). This file may contain sensitive information. To omit sensitive information from this file, you can [securely configure Terraform providers for Consul-Terraform-Sync](/docs/nia/configuration#securely-configure-terraform-providers) using environment variables or templating.
|
||||
- `terraform.tfvars` - The variable definitions file is where the services input variable and any optional Consul-Terraform-Sync input variables are assigned values from Consul. It is periodically updated, typically when the task condition is met, to reflect the current state of Consul.
|
||||
- `terraform.tfvars.tmpl` - The template file is used by Consul-Terraform-Sync to template information from Consul by using the HashiCorp configuration and templating library ([hashicorp/hcat](https://github.com/hashicorp/hcat)).
|
||||
|
||||
~> **Note:** Generated template and Terraform configuration files are crucial for the automation of tasks. Any manual changes to the files may not be preserved and could be overwritten by a subsequent update. Unexpected manual changes to the format of the files may cause automation to error.
|
||||
|
|
@ -661,7 +661,20 @@
|
|||
},
|
||||
{
|
||||
"title": "Network Drivers",
|
||||
"path": "nia/network-drivers"
|
||||
"routes": [
|
||||
{
|
||||
"title": "Overview",
|
||||
"path": "nia/network-drivers"
|
||||
},
|
||||
{
|
||||
"title": "Terraform",
|
||||
"path": "nia/network-drivers/terraform"
|
||||
},
|
||||
{
|
||||
"title": "Terraform Cloud",
|
||||
"path": "nia/network-drivers/terraform-cloud"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "Compatibility",
|
||||
|
|
|
|||
Loading…
Reference in New Issue