diff --git a/website/content/docs/nia/api/health.mdx b/website/content/docs/nia/api/health.mdx new file mode 100644 index 0000000000..6a04aba5ea --- /dev/null +++ b/website/content/docs/nia/api/health.mdx @@ -0,0 +1,35 @@ +--- +layout: docs +page_title: Consul-Terraform-Sync Health API +description: >- + Consul-Terraform-Sync Health API +--- + +# Health + +The `/health` endpoint returns a successful response when Consul-Terraform-Sync (CTS) is available and running. Requests to this endpoint are not logged, which makes it suitable for health checks that constantly poll CTS. + +| Method | Path | Produces | +| ------ | ------------------- | ------------------ | +| `GET` | `/health` | `application/json` | + +### Response Statuses + +| Status | Reason | +| ------ | ---------------------------------------------------- | +| 200 | CTS is healthy | + +### Example + +The following request makes a `GET` call to the `health` endpoint: + +```shell-session +$ curl --request GET \ + localhost:8558/v1/health +``` + +Response: + +```json +{} +``` diff --git a/website/content/docs/nia/api/status.mdx b/website/content/docs/nia/api/status.mdx index 0e41027883..edfd512072 100644 --- a/website/content/docs/nia/api/status.mdx +++ b/website/content/docs/nia/api/status.mdx @@ -18,16 +18,16 @@ This endpoint currently returns the overall status information for all tasks. | ------ | ------------------- | ------------------ | | `GET` | `/status` | `application/json` | -#### Request Parameters +### Request Parameters Currently no request parameters are offered for the overall status API. -#### Response Statuses +### Response Statuses | Status | Reason | | ------ | ---------------- | | 200 | Successfully retrieved the status | -#### Response Fields +### Response Fields | Name | Type | Description | | ----- | ------ | ------------------ | @@ -40,7 +40,7 @@ Currently no request parameters are offered for the overall status API. |`enabled.true` | integer | The number of tasks that are enabled. |`enabled.false` | integer | The number of tasks that are disabled. -#### Example +### Example Request: ```shell-session @@ -79,7 +79,7 @@ Task health status value is determined by the success or failure of all stored [ | ------ | ------------------- | ------------------ | | `GET` | `/status/tasks/:task_name` | `application/json` | -#### Request Parameters +### Request Parameters | Name | Required | Type | Description | Default | | -------- | -------- | ------ | ------------------ | ------- | @@ -87,14 +87,14 @@ Task health status value is determined by the success or failure of all stored [ |`include` | Optional | string | Only accepts the value `"events"`. Use to include stored event information in response. | none |`status` | Optional | string | Only accepts health status values `"successful"`, `"errored"`, `"critical"`, or `"unknown"`. Use to filter response by tasks that have the specified health status value. Recommend setting this parameter when requesting all tasks i.e. no `task` parameter is set. | none -#### Response Statuses +### Response Statuses | Status | Reason | | ------ | ---------------- | | 200 | Successfully retrieved the task status | | 404 | Task with the given name not found | -#### Response Fields +### Response Fields The response is a JSON map of task name to a status information structure with the following fields. @@ -108,7 +108,7 @@ The response is a JSON map of task name to a status information structure with t |`events_url` | string | Relative URL to retrieve the event data stored for the task. |`events` | list[[Event](/docs/nia/api/status#event)] | List of stored events that inform the task's status. See [section below](/docs/nia/api/status#event) for information on event data. This field is only included in the response upon request by setting the `?include=events` parameter. The relative URL for the request to include events can be retrieved from the `events_url` field. -##### Event +#### Event Event represents the process of updating network infrastructure of a task. The data is captured in a JSON structure. For more details on the scope of an event, see [Event](/docs/nia/tasks#event). @@ -126,7 +126,7 @@ Event represents the process of updating network infrastructure of a task. The d |`config.source` | string | **Deprecated in CTS 0.5.0 and will be removed in v0.8.0.** Module configured for the task. |`config.providers` | list[string] | **Deprecated in CTS 0.5.0 and will be removed in v0.8.0.** List of the providers configured for the task. -#### Example: All Task Statuses +### Example: All Task Statuses Request: ```shell-session @@ -163,7 +163,7 @@ Response: } ``` -#### Example: Individual Task Status with Events +### Example: Individual Task Status with Events Request: ```shell-session diff --git a/website/content/docs/nia/api/tasks.mdx b/website/content/docs/nia/api/tasks.mdx index 14fb75dc12..b870d26718 100644 --- a/website/content/docs/nia/api/tasks.mdx +++ b/website/content/docs/nia/api/tasks.mdx @@ -9,6 +9,73 @@ description: >- The `/tasks` endpoints interact with the tasks that Consul-Terraform-Sync (CTS) is responsible for running. +## Get Tasks + +This endpoint returns information about all existing tasks. + +| Method | Path | Produces | +| ------ | ------------------- | ------------------ | +| `GET` | `/tasks` | `application/json` | + +### Response Statuses + +| Status | Reason | +| ------ | ---------------------------------------------------- | +| 200 | Successfully retrieved and returned a list of tasks | + +### Response Fields + +| Name | Type | Description | +| ------------ | ------------- | ---------------------------------------------------------------------------------- | +| `request_id` | string | The ID of the request. Used for auditing and debugging purposes. | +| `tasks` | list[[Task](/docs/nia/api/tasks#task-object)] | A list of Tasks | + +### Example: Retrieve all tasks + +In this example, CTS contains a single task + +Request: + +```shell-session +$ curl --request GET \ + localhost:8558/v1/tasks +``` + +Response: + +```json +{ + "request_id": "0e0290f9-94df-3a4a-fd17-72467a16083c", + "tasks": [ + { + "buffer_period": { + "enabled": true, + "max": "25s", + "min": "5s" + }, + "condition": { + "services": { + "cts_user_defined_meta": {}, + "datacenter": "", + "filter": "", + "names": ["api", "web"], + "namespace": "", + "use_as_module_input": true + } + }, + "description": "Writes the service name, id, and IP address to a file", + "enabled": true, + "module": "path/to/module", + "module_input": {}, + "name": "task_a", + "providers": ["my-provider"], + "variables": {}, + "version": "" + } + ] +} +``` + ## Get Task This endpoint returns information about an existing task. @@ -17,27 +84,27 @@ This endpoint returns information about an existing task. | ------ | ------------------- | ------------------ | | `GET` | `/tasks/:task_name` | `application/json` | -#### Request Parameters +### Request Parameters | Name | Required | Type | Description | Default | | ------------ | -------- | ------ | ------------------------------------------ | ------- | | `:task_name` | Required | string | Specifies the name of the task to retrieve | none | -#### Response Statuses +### Response Statuses | Status | Reason | | ------ | ---------------------------------------------------- | | 200 | Successfully retrieved and returned task information | | 404 | Task with the given name not found | -#### Response Fields +### Response Fields | Name | Type | Description | | ------------ | ------ | ---------------------------------------------------------------------------------- | | `request_id` | string | The ID of the request. Used for auditing and debugging purposes. | | `task` | object | The task's configuration information. See [task object](#task-object) for details. | -#### Example: Retrieve a task +### Example: Retrieve a task Request: @@ -94,32 +161,32 @@ This endpoint allows for creation of a task. It only supports creation of a new | ------ | -------- | ------------------ | | `POST` | `/tasks` | `application/json` | -#### Request Parameters +### Request Parameters | Name | Required | Type | Description | Default | | ----- | -------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | | `run` | Optional | string | Values can only be `"inspect"` and `"now"`.
| none | -#### Request Body +### Request Body | Name | Required | Type | Description | | ------ | -------- | ------ | ---------------------------------------------------------------------------------- | | `task` | Required | object | The task's configuration information. See [task object](#task-object) for details. | -#### Response Statuses +### Response Statuses | Status | Reason | | ------ | ----------------------------- | | 201 | Successfully created the task | -#### Response Fields +### Response Fields | Name | Type | Description | | ------------ | ------ | ------------------------------------------------------------------------------------------------- | | `request_id` | string | The ID of the request. Used for auditing and debugging purposes. | | `task` | object | The task's configuration information after creation. See [task object](#task-object) for details. | -#### Example: Create a task +### Example: Create a task Payload: @@ -184,33 +251,33 @@ Response: ## Update Task -This endpoint allows patch updates to specifically supported fields for existing tasks. Currently only supports updating a task's [`enabled` value](/docs/nia/configuration#enabled-4). +This endpoint allows patch updates to specifically supported fields for existing tasks. Currently only supports updating a task's [`enabled` value](/docs/nia/configuration#enabled-7). | Method | Path | Produces | | ------- | ------------------- | ------------------ | | `PATCH` | `/tasks/:task_name` | `application/json` | -#### Request Parameters +### Request Parameters | Name | Required | Type | Description | Default | | ------------ | -------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | | `:task_name` | Required | string | Specifies the name of the task to update | none | | `run` | Optional | string | Values can only be `"inspect"` and `"now"`.
| none | -#### Request Body +### Request Body | Name | Required | Type | Description | | --------- | -------- | ------- | -------------------------------------------------------- | | `enabled` | Required | boolean | Whether the task is enabled to run and manage resources. | -#### Response Statuses +### Response Statuses | Status | Reason | | ------ | ---------------------------------- | | 200 | Successfully updated the task | | 404 | Task with the given name not found | -#### Response Fields +### Response Fields | Name | Type | Description | | ------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -218,7 +285,7 @@ This endpoint allows patch updates to specifically supported fields for existing | `inspect.changes_present` | boolean | Whether or not changes were detected for running the proposed task update. | | `inspect.plan` | string | The Terraform plan generated for the proposed task update. Note: a non-empty string does not necessarily indicate changes were detected. | -#### Example: Disable a task +### Example: Disable a task Request: @@ -235,7 +302,7 @@ Response: {} ``` -#### Example: Inspect enabling a task +### Example: Inspect enabling a task Request: @@ -278,33 +345,33 @@ This endpoint allows for deletion of existing tasks. It marks a task for deletio | -------- | ------------------- | ------------------ | | `DELETE` | `/tasks/:task_name` | `application/json` | -#### Request Parameters +### Request Parameters | Name | Required | Type | Description | Default | | ------------ | -------- | ------ | ------------------------------------------ | ------- | | `:task_name` | Required | string | Specifies the name of the task to retrieve | none | -#### Response Statuses +### Response Statuses | Status | Reason | | ------ | ----------------------------------------- | | 202 | Successfully marked the task for deletion | | 404 | Task with the given name not found | -#### Response Fields +### Response Fields | Name | Type | Description | | ------------ | ------ | ---------------------------------------------------------------- | | `request_id` | string | The ID of the request. Used for auditing and debugging purposes. | -#### Sample Request +### Sample Request ```shell-session $ curl --request DELETE \ localhost:8558/v1/tasks/task_a ``` -#### Sample Response +### Sample Response ```json { @@ -331,4 +398,5 @@ The task object is used by the Task APIs as part of a request or response. It re | `providers` | list[string] | Optional | The list of provider names that the task's module uses. | none | | `variables` | map[string] | Optional | The map of variables that are provided to the task's module. | none | | `version` | string | Optional | The version of the configured module that the task uses. | The latest version. | -| `terraform_version` | string | Optional | The version of Terraform to use for the Terraform Cloud workspace associated with the task. This is only available when used with the [Terraform Cloud driver](/docs/nia/configuration#terraform-driver). | The latest compatible version supported by the organization. | +| `terraform_version` | string | Optional | **Deprecated in CTS 0.6.0 and will be removed in 0.8.0. Review `terraform_version` in `terraform_cloud_workspace` instead.** The version of Terraform to use for the Terraform Cloud workspace associated with the task. This is only available when used with the [Terraform Cloud driver](/docs/nia/configuration#terraform-cloud-driver). | The latest compatible version supported by the organization. | +| `terraform_cloud_workspace` | object | Optional | The [configurable attributes of the Terraform Cloud workspace](/docs/nia/configuration#terraform_cloud_workspace) associated with the task. This option is only available when used with the [Terraform Cloud driver](/docs/nia/configuration#terraform-cloud-driver).| none | diff --git a/website/content/docs/nia/cli/index.mdx b/website/content/docs/nia/cli/index.mdx index de64b28d05..2187f617d4 100644 --- a/website/content/docs/nia/cli/index.mdx +++ b/website/content/docs/nia/cli/index.mdx @@ -11,53 +11,19 @@ Consul-Terraform-Sync (CTS) is controlled via an easy to use command-line interf ## Daemon -When running as a daemon, there is no default configuration to run CTS in a meaningful way. Setting a configuration flag `-config-file` or `-config-dir` is required. For example: +~> Running CTS as a daemon without using a command is deprecated in CTS 0.6.0 and will be removed at a much later date in a major release. For information on the preferred way to run CTS as a daemon review the [`start` command docs](/docs/nia/cli/start) + +When CTS runs as a daemon, there is no default configuration to start CTS. You must set a configuration flag -config-file or -config-dir. For example: ```shell-session $ consul-terraform-sync -config-file=config.hcl ``` -To view a list of available flags, use the `-help` or `-h` flag. - -### Modes - -CTS can be run as a daemon in different modes. - -#### Long-running Mode - -Flag: none - -Behavior: This is the default mode in which CTS passes through a once-mode phase and then turns into a long running process. During the once-mode phase, the daemon will exit with a non-zero status if it encounters an error. After successfully passing through once-mode phase, it will begin a long-running process in which errors are logged and exiting is not expected behavior. Once beginning the long-running process, the daemon serves any API and command requests. - -Usage: Intended to be run as a long running process after running once-mode successfully for given configuration and tasks. - -#### Inspect Mode - -Flag: `-inspect` - -Behavior: CTS will display the proposed state changes for all tasks once and exit. No changes are applied in this mode. On encountering an error before completing, CTS will exit with a non-zero status. - -Usage: Intended to be run before daemon-mode in order to confirm configuration is accurate and tasks would update network infrastructure as expected. - ---- - -Flag: `-inspect-task [task-name]` - -Behavior: This has similar behavior as `-inspect` mode for the selected task. The flag can be specified multiple times to inspect multiple tasks. No changes are applied in this mode. - -Usage: Useful to debug one or more tasks to confirm configuration is accurate and the selected tasks would update network infrastructure as expected. - -#### Once Mode - -Flag: `-once` - -Behavior: CTS will run all tasks once with buffer periods disabled and exit. On encountering an error before completing, CTS will exit with a non-zero status. - -Usage: Intended to be run before daemon-mode in order to confirm configuration is accurate and tasks update network infrastructure as expected. +To review a list of available flags, use the `-help` or `-h` flag. ## Commands -In addition to running the daemon, CTS has a set of commands that act as a client to the daemon server. The commands provide a user-friendly experience interacting with the daemon. The commands use the CTS APIs but does not correspond one-to-one with it. Please see the individual commands in the left navigation for more details. +In addition to running the daemon, CTS has a set of commands that act as a client to the daemon server. The commands provide a user-friendly experience interacting with the daemon. The commands use the CTS APIs but does not correspond one-to-one with it. Please review the individual commands in the left navigation for more details. To get help for a command, run: `consul-terraform-sync -h` @@ -78,6 +44,40 @@ Example: consul-terraform-sync task disable -http-addr=http://localhost:2000 task_a ``` +### Autocompletion + +The `consul-terraform-sync` command features opt-in autocompletion for flags, subcommands, and +arguments (where supported). + +To enable autocompletion, run: +```shell-session +$ consul-terraform-sync -autocomplete-install +``` + +~> After you install autocomplete, you must restart your shell for the change to take effect. + +When you start typing a CTS command, press the `` key to show a +list of available completions. To show available flag completes, type `-`. + +Autocompletion will query the running CTS server to return helpful argument suggestions. For example, for the `task disable` command, autocompletion will return the names of all enabled tasks that can be disabled. + +When autocompletion makes the query to the running CTS server, it will also use any `CTS_*` environment variables (for example `CTS_ADDRESS`) set on the CTS server. + +#### Example: Use autocomplete to discover how to disable a task + +Assume a tab is typed at the end of each prompt line: + +```shell-session +$ consul-terraform-sync +start task + +$ consul-terraform-sync task +create delete disable enable + +$ consul task disable +task_name_a task_name_b task_name_c +``` + ### General Options Below are options that can be used across all commands: diff --git a/website/content/docs/nia/cli/start.mdx b/website/content/docs/nia/cli/start.mdx new file mode 100644 index 0000000000..04534d4a38 --- /dev/null +++ b/website/content/docs/nia/cli/start.mdx @@ -0,0 +1,62 @@ +--- +layout: docs +page_title: Start Command +description: >- + Consul-Terraform-Sync supports start command for starting CTS as a daemon +--- + +# start + +The `start` command starts Consul-Terraform-Sync (CTS) as a daemon. When CTS runs as a daemon, there is no default configuration to start CTS. You must set a configuration flag `-config-file` or `-config-dir`. For example: + +```shell-session +$ consul-terraform-sync start -config-file=config.hcl +``` + +To review a list of available flags, use the `-help` or `-h` flag. + +## Modes + +CTS can be run as a daemon in different modes. + +### Long-running Mode + +Flag: none + +Behavior: This is the default mode in which CTS passes through a once-mode phase and then turns into a long-running process. During the once-mode phase, the daemon will exit with a non-zero status if it encounters an error. After successfully passing through once-mode phase, it will begin a long-running process in which errors are logged and exiting is not expected behavior. When the long-running process begins, CTS daemon starts serving API and command requests. + +### Inspect Mode + +Flag: `-inspect` + +Behavior: CTS will display the proposed state changes for all tasks once and exit. No changes are applied in this mode. On encountering an error before completing, CTS will exit with a non-zero status. + +Usage: Intended to be run before long-running mode in order to confirm configuration is accurate and tasks would update network infrastructure as expected. + +--- + +Flag: `-inspect-task [task-name]` + +Behavior: This has similar behavior as `-inspect` mode for the selected task. The flag can be specified multiple times to inspect multiple tasks. No changes are applied in this mode. + +Usage: Useful to debug one or more tasks to confirm configuration is accurate and the selected tasks would update network infrastructure as expected. + +### Once Mode + +Flag: `-once` + +Behavior: CTS will run all tasks once with buffer periods disabled and exit. On encountering an error before completing, CTS will exit with a non-zero status. + +Usage: Intended to be run before long-running mode in order to confirm configuration is accurate and tasks update network infrastructure as expected. + +## Options + +The `start` command supports the following options: + +| Name | Required | Type | Description |Default | +| --------------- | -------- | ------- | ------------------------------- | ----------------------- | +| `-config-dir `      | Required when`-config-file` is not set | string | A directory to load files for configuring CTS. Configuration files require an .hcl or .json file extension in order to specify their format. This option can be specified multiple times to load different directories. | none | +| `-config-file`                        | Required when `-config-dir` is not set | string | A file to load for configuring CTS. Configuration file requires an .hcl or .json extension in order to specify their format. This option can be specified multiple times to load different configuration files. | none | +| `-inspect` | Optional | boolean | Run CTS in Inspect mode to print the proposed state changes for all tasks, and then exit. No changes are applied in this mode. | false | +| `-inspect-task`                             | Optional | string | Run CTS in Inspect mode to print the proposed state changes for the task, and then exit. The flag can be specified multiple times to inspect multiple tasks. No changes are applied in this mode. | none | +| `-once` | Optional | boolean | Render templates and run tasks once. Does not start the process in long-running mode and disables buffer periods. | false | diff --git a/website/content/docs/nia/compatibility.mdx b/website/content/docs/nia/compatibility.mdx index bdba0a4fbd..c4cc632858 100644 --- a/website/content/docs/nia/compatibility.mdx +++ b/website/content/docs/nia/compatibility.mdx @@ -13,10 +13,12 @@ The following tables list the Consul-Terraform-Sync (CTS) version compatibility Below are CTS versions with supported Consul versions. The latest CTS binary supports the three most recent Consul minor versions, along with their latest patch versions. -| CTS Version | Consul OSS & Enterprise Version | -| :------------------ | :------------------------------ | -| CTS Enterprise 0.3+ | 1.8+ | -| CTS OSS 0.1+ | 1.8+ | +| CTS Version | Consul OSS & Enterprise Version | HCP Consul Version | +| :--------------------- | :------------------------------ | :----------------- | +| CTS Enterprise 0.6+ | 1.8+ | 1.9+ | +| CTS Enterprise 0.3-0.5 | 1.8+ | N/A | +| CTS OSS 0.6+ | 1.8+ | 1.9+ | +| CTS OSS 0.1-0.5 | 1.8+ | N/A | ## Terraform @@ -25,7 +27,7 @@ CTS integration with Terraform is supported for the following: | CTS Version | Terraform CLI Version | Terraform Cloud Version | Terraform Enterprise Version | | :------------------ | :-------------------- | :---------------------- | :--------------------------- | | CTS Enterprise 0.4+ | 0.13 - 1.1 | Latest | v202010-2 - Latest | -| CTS Enterprise 0.3+ | 0.13 - 1.1 | N/A | v202010-2 - Latest | +| CTS Enterprise 0.3 | 0.13 - 1.1 | N/A | v202010-2 - Latest | | CTS OSS 0.3+ | 0.13 - 1.1 | N/A | N/A | | CTS OSS 0.2 | 0.13 - 1.0 | N/A | N/A | | CTS OSS 0.1 | 0.13 - 0.14 | N/A | N/A | diff --git a/website/content/docs/nia/configuration.mdx b/website/content/docs/nia/configuration.mdx index 49b3a13a43..2576a9fa0f 100644 --- a/website/content/docs/nia/configuration.mdx +++ b/website/content/docs/nia/configuration.mdx @@ -17,6 +17,7 @@ Top level options are reserved for configuring CTS. log_level = "INFO" working_dir = "sync-tasks" port = 8558 +id = "cts-01" syslog { facility = "local2" @@ -43,6 +44,7 @@ tls { - `max` - (string: "20s") The maximum period of time to wait after changes are detected before triggering related tasks. If `min` is set, the default period for `max` is 4 times the value of `min`. - `log_level` - (string: "INFO") The log level to use for CTS logging. This defaults to "INFO". The available log levels are "TRACE", "DEBUG", "INFO", "WARN", and "ERR". - `port` - (int: 8558) The port for CTS to use to serve API requests. +- `id` (string: Generated ID with the format `cts-`) The ID of the CTS instance. Used as the service ID for CTS if service registration is enabled. - `syslog` - Specifies the syslog server for logging. - `enabled` - (bool) Enable syslog logging. Specifying other option also enables syslog logging. - `facility` - (string: "local0") Name of the syslog facility to log to. @@ -55,11 +57,44 @@ tls { - `verify_incoming` - (bool: false) Enable mutual TLS. Requires all incoming connections to the CTS API to use a TLS connection and provide a certificate signed by a Certificate Authority specified by the `ca_cert` or `ca_path`. - `ca_cert` - (string) The path to a PEM-encoded certificate authority file used to verify the authenticity of the incoming client connections to the CTS API when `verify_incoming` is set to true. Takes precedence over `ca_path` if both are configured. - `ca_path` - (string) The path to a directory of PEM-encoded certificate authority files used to verify the authenticity of the incoming client connections to the CTS API when `verify_incoming` is set to true. -- `license_path` - (string) Configures the path to the file containing the license. A license must be set to use enterprise features. You can also set the license by defining the `CONSUL_LICENSE` and `CONSUL_LICENSE_PATH` environment variables. See [Setting the License](/docs/nia/enterprise/license#setting-the-license) for details. +- `license_path` - (string) **Deprecated in CTS 0.6.0 and will be removed in a future release. Use [license block](/docs/nia/configuration#license) instead.** Configures the path to the file that contains the license. You must specify a license in order to use enterprise features. You can also set the license by defining the `CONSUL_LICENSE` and `CONSUL_LICENSE_PATH` environment variables. For more information, refer to [Setting the License](/docs/nia/enterprise/license#setting-the-license). + +## License + +The `license` block configures how CTS loads its license with options to: +- Configure CTS to automatically retrieve a license from Consul. +- Provide a path to a license file. + +When a license block is not configured, CTS uses automatic license retrieval. + +```hcl +license { + path = "path/to/license.lic" + auto_retrieval { + enabled = true + } +} +``` + +| Parameter | Required | Type | Description | Default | +| --------- | -------- | ---- | ----------- | ------- | +| `path` | Optional | string | Configures the path to the file containing a license. If a path to a license is configured, this license is used until you enable automatic license retrieval. You can also set the license by defining the `CONSUL_LICENSE` and `CONSUL_LICENSE_PATH` environment variables. To learn more, review [Setting the License](/docs/nia/enterprise/license#setting-the-license). | none | +| `auto_retrieval` | Optional | object | Configures the license auto-retrieval used by CTS. To learn more, review [Auto-Retrieval](/docs/nia/configuration#auto-retrieval) for details | Review [Auto-Retrieval](/docs/nia/configuration#auto-retrieval) for defaults. | + + +#### Auto-Retrieval + +You can use the `auto_retrieval` block to configure the the automatic license retrieval in CTS. When enabled, CTS attempts to retrieve a new license from its configured Consul Enterprise backend once a day. If CTS cannot retrieve a license and the current license is reaching its expiration date, CTS attempts to retrieve a license with increased frequency, as defined by the [License Expiration Date Handling](/docs/nia/enterprise/license#license-expiration-handling). + +~> Enabling `auto_retrieval` is recommended when using HCP Consul, as HCP Consul licenses expire more frequently than Consul Enterprise licenses. Without auto-retrieval enabled, you have to restart CTS every time you load a new license. + +| Parameter | Required | Type | Description | Default | +| --------- | -------- | ---- | ----------- | ------- | +| `enabled` | Optional | string | If set to true, enables license auto-retrieval | true | ## Consul -The `consul` block is used to configure CTS connection with a Consul agent to perform queries to the Consul Catalog and Consul KV pertaining to task execution. +The `consul` block configures the CTS connection with a Consul agent so that CTS can perform queries for task execution. It also configures the automatic registration of CTS as a service with Consul. -> **Note:** Use HTTP/2 to improve Consul-Terraform-Sync performance when communicating with the local Consul process. [TLS/HTTPS](/docs/agent/config/config-files) must be configured for the local Consul with the [cert_file](/docs/agent/config/config-files#cert_file) and [key_file](/docs/agent/config/config-files#key_file) parameters set. For the Consul-Terraform-Sync configuration, set `tls.enabled = true` and set the `address` parameter to the HTTPS URL, e.g., `address = example.consul.com:8501`. If using self-signed certificates for Consul, you will also need to set `tls.verify = false` or add the certificate to `ca_cert` or `ca_path`. @@ -72,34 +107,111 @@ consul { tls {} token = null transport {} + service_registration { + service_name = "cts" + address = "172.22.0.2" + default_check { + address = "http://172.22.0.2:8558" + } + } } ``` -- `address` - (string: "localhost:8500") Address is the address of the Consul agent. It may be an IP or FQDN. -- `auth` - Auth is the HTTP basic authentication for communicating with Consul. - - `enabled` - (bool) - - `username` - (string) - - `password` - (string) -- `tls` - Configure TLS to use a secure client connection with Consul. Using HTTP/2 can solve issues related to hitting Consul's maximum connection limits, as well as improve efficiency when processing many blocking queries. This option is required for Consul-Terraform-Sync when connecting to a [Consul agent with TLS verification enabled for HTTPS connections](/docs/agent/config/config-files#verify_incoming). - - `enabled` - (bool) Enable TLS. Providing a value for any of the TLS options will enable this parameter implicitly. - - `verify` - (bool: true) Enables TLS peer verification. The default is enabled, which will check the global certificate authority (CA) chain to make sure the certificates returned by Consul are valid. - - If Consul is using a self-signed certificate that you have not added to the global CA chain, you can set this certificate with `ca_cert` or `ca_path`. Alternatively, you can disable SSL verification by setting `verify` to false. However, disabling verification is a potential security vulnerability. - - `ca_cert` - (string) The path to a PEM-encoded certificate authority file used to verify the authenticity of the connection to Consul over TLS. Can also be provided through the `CONSUL_CACERT` environment variable. - - `ca_path` - (string) The path to a directory of PEM-encoded certificate authority files used to verify the authenticity of the connection to Consul over TLS. Can also be provided through the `CONSUL_CAPATH` environment variable. - - `cert` - (string) The path to a PEM-encoded client certificate file provided to Consul over TLS in order for Consul to verify the authenticity of the connection from CTS. Required if Consul has `verify_incoming` set to true. Can also be provided through the `CONSUL_CLIENT_CERT` environment variable. - - `key` - (string) The path to the PEM-encoded private key file used with the client certificate configured by `cert`. Required if Consul has `verify_incoming` set to true. Can also be provided through the `CONSUL_CLIENT_KEY` environment variable. - - `server_name` - (string) The server name to use as the Server Name Indication (SNI) for Consul when connecting via TLS. Can also be provided through the `CONSUL_TLS_SERVER_NAME` environment variable. -- `token` - (string) The ACL token to use for client communication with the local Consul agent. The token can also be provided through the `CONSUL_TOKEN` or `CONSUL_HTTP_TOKEN` environment variables. More information on the required privileges required by CTS are available in the [Secure Consul-Terraform-Sync for Production](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-secure?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS#configure-acl-privileges-for-consul-terraform-sync) tutorial -- `transport` - Transport configures the low-level network connection details. - - `dial_keep_alive` - (string: "30s") The amount of time for keep-alives. - - `dial_timeout` - (string: "30s") The amount of time to wait to establish a connection. - - `disable_keep_alives` - (bool) Determines if keep-alives should be used. Disabling this significantly decreases performance. - - `idle_conn_timeout` - (string: "5s") The timeout for idle connections. - - `max_idle_conns` - (int: 0) The maximum number of total idle connections across all hosts. The limit is disabled by default. - - `max_idle_conns_per_host` - (int: 100) The maximum number of idle connections per remote host. The majority of connections are established with one host, the Consul agent. - - To achieve the shortest latency between a Consul service update to a task execution, configure `max_idle_conns_per_host` equal to or greater than the number of services in automation across all tasks. - - This value should be lower than the configured [`http_max_conns_per_client`](/docs/agent/config/config-files#http_max_conns_per_client) for the Consul agent. If `max_idle_conns_per_host` and the number of services in automation is greater than the Consul agent limit, Consul-Terraform-Sync may error due to connection limits (status code 429). You may increase the agent limit with caution. _Note: requests to the Consul agent made by Terraform subprocesses or any other process on the same host as Consul-Terraform-Sync will contribute to the Consul agent connection limit._ - - `tls_handshake_timeout` - (string: "10s") amount of time to wait to complete the TLS handshake. + +| Parameter | Required | Type | Description | Default | +| --------- | -------- | ---- | ----------- | ------- | +| `address` | Optional | string | The address of the Consul agent. It may be an IP or FQDN. | `localhost:8500` | +| `token` | Optional | string | The ACL token to use for client communication with the local Consul agent. See [ACL Requirements](/docs/nia/configuration#acl-requirements) for required privileges.

The token can also be provided through the `CONSUL_TOKEN` or `CONSUL_HTTP_TOKEN` environment variables. | none | +| `auth` | Optional | [auth](/docs/nia/configuration#auth) | HTTP basic authentication for communicating with Consul || +| `tls` | Optional | [tls](/docs/nia/configuration#tls-1) | Secure client connection with Consul || +| `transport` | Optional | [transport](/docs/nia/configuration#transport) | Low-level network connection details || +| `service_registration` | Optional| [service_registration](/docs/nia/configuration#service-registration) | Options for how CTS will register itself as a service with a health check to Consul. || + +##### ACL Requirements +The following table describes the ACL policies required by CTS. For more information, refer to the [Secure Consul-Terraform-Sync for Production](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-secure?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS#configure-acl-privileges-for-consul-terraform-sync) tutorial. + +| Policy | Resources | +| ------ | --------- | +| `service:read` | Any services monitored by tasks | +| `node:read` | Any nodes hosting services monitored by tasks | +| `keys:read` | Any Consul KV pairs monitored by tasks | +| `namespace:read` | Any namespaces for resources monitored by tasks | +| `service:write` | The CTS service when [service registration](/docs/nia/configuration#service-registration) is enabled | +| `keys:write` | `consul-terraform-sync/` Only required when using Consul as the [Terraform backend](/docs/nia/configuration#backend). | + + +#### Auth +Configures HTTP basic authentication for communicating with Consul. + +| Parameter | Required | Type | Description | Default | +| --------- | -------- | ---- | ----------- | ------- | +| `enabled` | Optional | boolean | Enables using HTTP basic authentication | `false` | +| `username` | Optional | string | Username for authentication| none | +| `password` | Optional | string | Password for authentication| none | + +#### TLS +Configure TLS to use a secure client connection with Consul. Using HTTP/2 can solve issues related to hitting Consul's maximum connection limits, as well as improve efficiency when processing many blocking queries. This option is required for Consul-Terraform-Sync when connecting to a [Consul agent with TLS verification enabled for HTTPS connections](/docs/agent/config/config-files#verify_incoming). + +If Consul is using a self-signed certificate that you have not added to the global CA chain, you can set this certificate with `ca_cert` or `ca_path`. Alternatively, you can disable SSL verification by setting `verify` to false. However, disabling verification is a potential security vulnerability. + +| Parameter | Required | Type | Description | Default | +| --------- | -------- | ---- | ----------- | ------- | +| `enabled` | Optional | boolean | Enable TLS. Providing a value for any of the TLS options enables this parameter implicitly. | `false` | +| `verify` | Optional | boolean | Enables TLS peer verification, which checks the global certificate authority (CA) chain to make sure the certificates returned by Consul are valid. | `true` | +| `ca_cert` | Optional | string | The path to a PEM-encoded certificate authority file used to verify the authenticity of the connection to Consul over TLS.

Can also be provided through the `CONSUL_CACERT` environment variable. | none | +| `ca_path` | Optional | string | The path to a directory of PEM-encoded certificate authority files used to verify the authenticity of the connection to Consul over TLS.

Can also be provided through the `CONSUL_CAPATH` environment variable. | none | +| `cert` | Optional | string | The path to a PEM-encoded client certificate file provided to Consul over TLS in order for Consul to verify the authenticity of the connection from CTS. Required if Consul has `verify_incoming` set to true.

Can also be provided through the `CONSUL_CLIENT_CERT` environment variable. | none | +| `key` | Optional | string | The path to the PEM-encoded private key file used with the client certificate configured by `cert`. Required if Consul has `verify_incoming` set to true.

Can also be provided through the `CONSUL_CLIENT_KEY` environment variable. | none | +| `server_name` | Optional | string | The server name to use as the Server Name Indication (SNI) for Consul when connecting via TLS.

Can also be provided through the `CONSUL_TLS_SERVER_NAME` environment variable. | none | + +#### Transport +Configures the low-level network connection details to Consul. + +To achieve the shortest latency between a Consul service update to a task execution, configure `max_idle_conns_per_host` equal to or greater than the number of services in automation across all tasks. This value should be lower than the configured [`http_max_conns_per_client`](/docs/agent/config/config-files#http_max_conns_per_client) for the Consul agent. + +If `max_idle_conns_per_host` and the number of services in automation is greater than the Consul agent limit, CTS may error due to connection limits (status code 429). You may increase the agent limit with caution. _Note: requests to the Consul agent made by Terraform subprocesses or any other process on the same host as CTS will contribute to the Consul agent connection limit._ + +| Parameter | Required | Type | Description | Default | +| --------- | -------- | ---- | ----------- | ------- | +| `dial_keep_alive` | Optional | string | The amount of time for keep-alives. | `30s` | +| `dial_timeout` | Optional | string | The amount of time to wait to establish a connection. | `30s` +| `disable_keep_alives` | Optional | boolean | Determines if keep-alives should be used. Disabling this significantly decreases performance. | `false`| +| `idle_conn_timeout` | Optional | string | The timeout for idle connections. | `5s` | +| `max_idle_conns` | Optional | integer | The maximum number of total idle connections across all hosts. The limit is disabled by default. | `0` | +| `max_idle_conns_per_host` | Optional | integer | The maximum number of idle connections per remote host. The majority of connections are established with one host, the Consul agent. | `100`| +| `tls_handshake_timeout` | Optional | string | The amount of time to wait to complete the TLS handshake. | `10s`| + + +#### Service Registration +CTS automatically registers itself with Consul as a service with a health check, using the [`id`](/docs/nia/configuration#id) configuration as the service ID. CTS deregisters itself with Consul when CTS stops gracefully. If CTS is unable to register with Consul, then it will log the error and continue without exiting. + +Service registration requires that the [Consul token](/docs/nia/configuration#consul) has an ACL policy of `service:write` for the CTS service. + +| Parameter | Required | Type | Description | Default | +| --------- | -------- | ---- | ----------- | ------- | +| `enabled` | Optional | boolean | Enables CTS to register itself as a service with Consul.| `true` | +| `service_name` | Optional | string | The service name for CTS. | `Consul-Terraform-Sync` | +| `address` | Optional | string | The IP address or hostname for CTS. | IP address of the Consul agent node | +| `namespace` | Optional | string | The namespace to register CTS in. | In order of precedence:
1. Inferred from the CTS ACL token
2. The `default` namespace. | +| `default_check.enabled` | Optional | boolean | Enables CTS to create the default health check. | `true` | +| `default_check.address` | Optional | string | The address to use for the default HTTP health check. Needs to include the scheme (`http`/`https`) and the port, if applicable. | `http://localhost:` or `https://localhost:`. Determined from the [port configuration](/docs/nia/configuration#port) and whether [TLS is enabled](/docs/nia/configuration#enabled-2) on the CTS API. | + +The default health check is an [HTTP check](/docs/discovery/checks#http-interval) that calls the [Health API](/docs/nia/api/health). The following table describes the values CTS sets for this default check, corresponding to the [Consul register check API](/api-docs/agent/check#register-check). If an option is not listed in this table, then CTS is using the default value. + +| Parameter | Value | +| --------- | ----- | +| Name | `CTS Health Status`| +| ID | `-health` | +| Namespace | `service_registration.namespace` | +| Notes | `Check created by Consul-Terraform-Sync`| +| DeregisterCriticalServiceAfter | `30m` | +| ServiceID | [`id`](/docs/nia/configuration#id) | +| Status | `critical` | +| HTTP | `/v1/health` | +| Method | `GET` | +| Interval | `10s` | +| Timeout | `2s` | +| TLSSkipVerify | `false` | ## Service @@ -197,7 +309,12 @@ task { - `condition` - (obj: required) The requirement that, when met, triggers CTS to execute the task. Only one `condition` may be configured per task. CTS supports different types of conditions, which each have their own configuration options. See [Task Condition](#task-condition) configuration for full details on configuration options for each condition type. - `source_input` - (obj) **Deprecated in CTS 0.5.0 and will be removed in 0.8.0. See the `module_input` block instead.** - `module_input` - (obj) Specifies a Consul object containing values or metadata to be provided to the Terraform Module. The `module_input` block defines any extra module inputs needed for task execution. This is in addition to any module input provided by the `condition` block or `services` field (deprecated). Multiple `module_input` blocks can be configured per task. [Task Module Input](#task-module-input) configuration for full details on usage and restrictions. -- `terraform_version` - (string) The version of Terraform to use for the Terraform Cloud workspace associated with the task. Defaults to the latest compatible version supported by the organization. This option is only available when used with the [Terraform Cloud driver](#terraform-cloud-driver); otherwise, set the version within the [Terraform driver](#terraform-driver). +- `terraform_version` - (string) **Deprecated in CTS 0.6.0 and will be removed in 0.8.0. Review `terraform_cloud_workspace.terraform_version` instead.** The version of Terraform to use for the Terraform Cloud workspace associated with the task. Defaults to the latest compatible version supported by the organization. This option is only available when used with the [Terraform Cloud driver](#terraform-cloud-driver); otherwise, set the version within the [Terraform driver](#terraform-driver). +- `terraform_cloud_workspace` - (obj) Configures attributes of the Terraform Cloud workspace associated with the task. This option is only available when used with the [Terraform Cloud driver](#terraform-cloud-driver). For global configurations of all workspaces, review [`driver.workspaces`](#workspaces). + - `execution_mode` - (string: "remote") The execution mode that determines whether to use Terraform Cloud as the Terraform execution platform. Only supports "remote" or "agent". + - `agent_pool_id` - (string) Only supported if `execution_mode` is set to "agent". The ID of the agent pool that should run the Terraform workloads. Either `agent_pool_id` or `agent_pool_name` are required if `execution_mode` is set to "agent". `agent_pool_id` takes precedence over `agent_pool_name` if both are provided. + - `agent_pool_name` - (string) Only supported if `execution_mode` is set to "agent". The name of the agent pool that should run the Terraform workloads. Only supported if `execution_mode` is set to "agent". Either `agent_pool_id` or `agent_pool_name` are required. `agent_pool_id` takes precedence over `agent_pool_name` if both are provided. + - `terraform_version` - (string) The version of Terraform to use for the Terraform Cloud workspace associated with the task. Defaults to the latest compatible version supported by the organization. ### Task Condition @@ -505,7 +622,7 @@ driver "terraform" { - `backend` - (obj) The backend stores [Terraform state files](https://www.terraform.io/docs/state/index.html) for each task. This option is similar to the [Terraform backend configuration](https://www.terraform.io/docs/configuration/backend.html). CTS supports Terraform backends used as a state store. - Supported backend options: [azurerm](https://www.terraform.io/docs/backends/types/azurerm.html), [consul](https://www.terraform.io/docs/backends/types/consul.html), [cos](https://www.terraform.io/docs/backends/types/cos.html), [gcs](https://www.terraform.io/docs/backends/types/gcs.html), [kubernetes](https://www.terraform.io/docs/backends/types/kubernetes.html), [local](https://www.terraform.io/docs/backends/types/local.html), [manta](https://www.terraform.io/docs/backends/types/manta.html), [pg](https://www.terraform.io/docs/backends/types/pg.html) (Terraform v0.14+), [s3](https://www.terraform.io/docs/backends/types/s3.html). Visit the Terraform documentation links for details on backend configuration options. - - If omitted, CTS will generate default values and use configurations from the [`consul` block](#consul) to configure [Consul as the backend](https://www.terraform.io/docs/backends/types/consul.html), which stores Terraform statefiles in the Consul KV. The [ACL token provided for Consul authentication](#token) is used to read and write to the KV store and requires [Consul KV privileges](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-secure?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS#configure-acl-privileges-for-consul-terraform-sync). The Consul KV path is the base path to store state files for tasks. The full path of each state file will have the task identifier appended to the end of the path, e.g. `consul-terraform-sync/terraform-env:task-name`. + - If omitted, CTS will generate default values and use configurations from the [`consul` block](#consul) to configure [Consul as the backend](https://www.terraform.io/docs/backends/types/consul.html), which stores Terraform statefiles in the Consul KV. The [ACL token provided for Consul authentication](#consul) is used to read and write to the KV store and requires [Consul KV privileges](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-secure?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS#configure-acl-privileges-for-consul-terraform-sync). The Consul KV path is the base path to store state files for tasks. The full path of each state file will have the task identifier appended to the end of the path, e.g. `consul-terraform-sync/terraform-env:task-name`. - The remote enhanced backend is not supported with the Terraform driver to run operations in Terraform Cloud. Use the [Terraform Cloud driver](#terraform-cloud-driver) to integrate CTS with Terraform Cloud for remote workspaces and remote operations. - `log` - (bool) Enable all Terraform output (stderr and stdout) to be included in the CTS log. This is useful for debugging and development purposes. It may be difficult to work with log aggregators that expect uniform log format. - `path` - (string) The file path to install Terraform or discover an existing Terraform binary. If omitted, Terraform will be installed in the same directory as the CTS daemon. To resolve an incompatible Terraform version or to change versions will require removing the existing binary or change to a different path. @@ -693,7 +810,7 @@ terraform_provider "example" { - `enabled` - (bool) Enabled controls whether the Vault integration is active. - `namespace` - (string) Namespace is the Vault namespace to use for reading secrets. This can also be set via the `VAULT_NAMESPACE` environment variable. - `renew_token` - (bool) Renews the Vault token. This can also be set via the `VAULT_RENEW_TOKEN` environment variable. -- `tls` - [(tls block)](#tls) TLS indicates the client should use a secure connection while talking to Vault. Supports the environment variables: +- `tls` - [(tls block)](#tls-1) TLS indicates the client should use a secure connection while talking to Vault. Supports the environment variables: - `VAULT_CACERT` - `VAULT_CAPATH` - `VAULT_CLIENT_CERT` diff --git a/website/content/docs/nia/enterprise/license.mdx b/website/content/docs/nia/enterprise/license.mdx index 15b2b96c52..740d52fbc7 100644 --- a/website/content/docs/nia/enterprise/license.mdx +++ b/website/content/docs/nia/enterprise/license.mdx @@ -17,9 +17,14 @@ All CTS Enterprise features are available with a valid Consul Enterprise license To get a trial license for CTS, you can sign-up for the [trial license for Consul Enterprise](/docs/enterprise/license/faq#q-where-can-users-get-a-trial-license-for-consul-enterprise). -## Setting the License +## Automatic License Retrieval +CTS automatically retrieves a license from Consul on startup and then attempts to retrieve a new license once a day. If the current license is reaching its expiration date, CTS attempts to retrieve a license with increased frequency, as defined by the [License Expiration Date Handling](/docs/nia/enterprise/license#license-expiration-handling). -Choose one of the following methods (in order of precedence) to set the license: +~> Enabling automatic license retrieval is recommended when using HCP Consul, as HCP Consul licenses expire more frequently than Consul Enterprise licenses. Without auto-retrieval enabled, you have to restart CTS every time you load a new license. + +## Setting the License Manually + +If a license needs to be manually set, choose one of the following methods (in order of precedence) to set the license: 1. Set the `CONSUL_LICENSE` environment variable to the license string. @@ -33,7 +38,15 @@ Choose one of the following methods (in order of precedence) to set the license: export CONSUL_LICENSE_PATH=// ``` -1. Configure the [`license_path`](/docs/nia/configuration#license_path) option in the configuration file to point to the file containing the license. +1. To point to the file containing the license, in the configuration file, configure the [`license`](/docs/nia/configuration#license) path option. + + ```hcl + license { + path = "//" + } + ``` + +1. To point to the file containing the license, in the configuration file, configure the [`license_path`](/docs/nia/configuration#license_path) option i. **Deprecated in CTS 0.6.0 and will be removed in a future release. Use [license block](/docs/nia/configuration#license) instead.** ```hcl license_path = "//" @@ -42,30 +55,29 @@ Choose one of the following methods (in order of precedence) to set the license: ~> **Note**: the [options to set the license and the order of precedence](/docs/enterprise/license/overview#binaries-without-built-in-licenses) are the same as Consul Enterprise server agents. Visit the [Enterprise License Tutorial](https://learn.hashicorp.com/tutorials/nomad/hashicorp-enterprise-license?in=consul/enterprise) for detailed steps on how to install the license key. -### Updating the License - -The previous section describes options for [setting the license](#setting-the-license) needed to start running CTS Enterprise. Use the following procedure to update the license when it expires or is near the expiration date: +### Updating the License Manually +To update the license when it expires or is near the expiration date and automatic license retrieval is disabled: 1. Update the license environment variable or configuration with the new license value or path to the new license file 1. Stop and restart CTS Enterprise Once CTS Enterprise starts again, it will pick up the new license and run the tasks with any changes that may have occurred between the stop and restart period. -## Notification and Termination Behavior +## License Expiration Handling Licenses have an expiration date and a termination date. The termination date is a time at or after the license expires. CTS Enterprise will cease to function once the termination date has passed. The time between the expiration and termination dates is a grace period. Grace periods are generally 24-hours, but you should refer to your license agreement for complete terms of your grace period. -When approaching expiration and termination, CTS Enterprise will provide notifications in the system logs: +When approaching expiration and termination, by default, CTS Enterprise will attempt to retrieve a new license. If auto-retrieval is disabled, CTS Enterprise will provide notifications in the system logs: -| Time period | Behavior | -| ------------------------------------------- | --------------------------------- | -| 30 days before expiration | Warning-level log every 24-hours | -| 7 days before expiration | Warning-level log every 1 hour | -| 1 hour before expiration | Warning-level log every 1 minute | -| 1 day before expiration | Warning-level log every 5 minutes | -| At or after expiration (before termination) | Error-level log every 1 minute | -| At or after termination | Error-level log and exit | +| Time period | Behavior - auto-retrieval enabled (default) |Behavior - auto-retrieval disabled | +| ------------------------------------------- |-------------------------------------------- |---------------------------------- | +| 30 days before expiration | License retrieval attempt every 24-hours | Warning-level log every 24-hours | +| 7 days before expiration | License retrieval attempt every 1 hour | Warning-level log every 1 hour | +| 1 day before expiration | License retrieval attempt every 5 minutes | Warning-level log every 5 minutes | +| 1 hour before expiration | License retrieval attempt every 1 minute | Warning-level log every 1 minute | +| At or after expiration (before termination) | License retrieval attempt every 1 minute | Error-level log every 1 minute | +| At or after termination | Error-level log and exit | Error-level log and exit | ~> **Note**: Notification frequency and [grace period](/docs/enterprise/license/faq#q-is-there-a-grace-period-when-licenses-expire) behavior is the same as Consul Enterprise. diff --git a/website/content/docs/nia/lorna.txt b/website/content/docs/nia/lorna.txt deleted file mode 100644 index e69de29bb2..0000000000 diff --git a/website/content/docs/nia/network-drivers/terraform-cloud.mdx b/website/content/docs/nia/network-drivers/terraform-cloud.mdx index 80894ad87a..829bc189e3 100644 --- a/website/content/docs/nia/network-drivers/terraform-cloud.mdx +++ b/website/content/docs/nia/network-drivers/terraform-cloud.mdx @@ -31,12 +31,12 @@ Once all workspaces are set up, CTS monitors the Consul catalog for service chan ## Remote Workspaces -CTS will discover or create a new workspaces based on your configured tasks. The task configuration options for `name`, `description`, and `terraform_version` are used to set the workspace name, description, and Terraform version. +CTS will discover or create a new workspaces based on your configured tasks. The task configuration [`name`](/docs/nia/configuration#name-1) and [`description`](/docs/nia/configuration#description) are used to set the workspace name and description. The task configuration [`terraform_cloud_workspace`](/docs/nia/configuration#terraform_cloud_workspace) is used to set options like Terraform version, execution mode, and agent pool if relevant. CTS will also use any globally set workspace configurations, specified in the driver configuration [`workspaces`](/docs/nia/configuration#workspaces). [![CTS Workspace Overview](/img/nia/cts-tfc-workspace.png)](/img/nia/cts-tfc-workspace.png) Workspace automation requirements for CTS are in place to avoid overriding other workspaces unintentionally. -* Must be set to remote execution mode +* Must be set to remote or agent execution mode * Cannot be connected to a VCS * Cannot have an existing configuration version uploaded by another application * Must satisfy workspace [tag requirements](/docs/nia/configuration#tags_allowlist) and [tag restrictions](/docs/nia/configuration#tags_denylist) set by the CTS operator @@ -47,9 +47,9 @@ Workspaces created by CTS will be configured with the following settings: | ------- | ----- | | Workspace name | CTS task name | | Description | CTS task description | -| Execution mode | Remote | +| Execution mode | [`task.terraform_cloud_workspace.execution_mode`](/docs/nia/configuration#execution_mode) or remote by default | | Apply method | Auto apply | -| Terraform Version | [`task.terraform_version`](/docs/nia/configuration#terraform_version) or the latest [Terraform version compatible with CTS](/docs/nia/compatibility#terraform) available for the organization. | +| Terraform Version | [`task.terraform_cloud_workspace.terraform_version`](/docs/nia/configuration#terraform_version-1), [`task.terraform_version`](/docs/nia/configuration#terraform_version) (deprecated), or the latest [Terraform version compatible with CTS](/docs/nia/compatibility#terraform) available for the organization. | | Tags | `source:cts` and [additional tags](/docs/nia/configuration#tags) set by the CTS operator | Other workspace settings can be pre-configured or updated, such as setting the workspace to [manual apply](#manual-apply) or adding a [run notification](https://www.terraform.io/docs/cloud/workspaces/notifications.html) to send messages to a Slack channel when CTS updates your network infrastructure. @@ -127,7 +127,7 @@ After creating a dedicated team, update the team's permissions with "Manage Work [![CTS Terraform Team Setup](/img/nia/cts-tfc-team-setup.png)](/img/nia/cts-tfc-team-setup.png) -After setting the team's permissions, the final setup step is to [generate the associated team token](https://www.terraform.io/docs/cloud/users-teams-organizations/api-tokens.html#team-api-tokens), which can be done on the same team management page. This token will be used by CTS for API authentication and will be used to configure the [`token`](/docs/nia/configuration#token-1) on the Terraform Cloud driver. +After setting the team's permissions, the final setup step is to [generate the associated team token](https://www.terraform.io/docs/cloud/users-teams-organizations/api-tokens.html#team-api-tokens), which can be done on the same team management page. This token will be used by CTS for API authentication and will be used to configure the [`token`](/docs/nia/configuration#token) on the Terraform Cloud driver. ### Recommendations diff --git a/website/content/docs/release-notes/consul-terraform-sync/v0_6_0_beta.mdx b/website/content/docs/release-notes/consul-terraform-sync/v0_6_0_beta.mdx deleted file mode 100644 index ba9ea61394..0000000000 --- a/website/content/docs/release-notes/consul-terraform-sync/v0_6_0_beta.mdx +++ /dev/null @@ -1,325 +0,0 @@ ---- -layout: docs -page_title: CTS v0.6.0 Beta -description: >- - Consul-Terraform-Sync beta release notes for v0.6.0 ---- - --> The beta release notes for v0.6.0 focus on changes and deprecations in the upcoming Consul-Terraform-Sync (CTS) v0.6.0 release. Details of these features are subject to change as the beta period commences. - -# New and Deprecated Features in v0.6.0 - -**Feature Summary:** - -- New CLI Start Command -- New CLI Autocompletion -- Addition of Features to Support HCP Consul -- Addition of Features to Support Terraform Cloud Agents -- Get All Tasks API Endpoint -- Health API Endpoint -- CTS Service Registration with Consul - -For additional information, see the [changelog in the CTS repository](https://github.com/hashicorp/consul-terraform-sync/blob/release/0.6.0-beta/CHANGELOG.md) - -## Introduction of CLI Start Command -~> Running CTS as a daemon without using a command is deprecated in CTS 0.6.0. The feature will be removed at a much later date in a major release. The preferred way to run CTS is to use the `start` command. - -### start - -In v0.6.0, the `start` command is the new and preferred way to start CTS as a daemon. This command shares the same behavior as running CTS without a command. - -When CTS runs as a daemon, there is no default configuration to start CTS. You must set a configuration flag `-config-file` or `-config-dir`. For example: - -```shell-session -$ consul-terraform-sync start -config-file=config.hcl -``` - -To review a list of available flags, use the `-help` or `-h` flag. - -##### Options - -The `start` command supports the following options: - -| Name | Required | Type | Description |Default | -| --------------- | -------- | ------- | ------------------------------- | ----------------------- | -| `-config-dir `      | Required if `-config-file` not set | string | A directory to load files for configuring CTS. Configuration files require an .hcl or .json file extension in order to specify their format. This option can be specified multiple times to load different directories. | none | -| `-config-file`                        | Required if `-config-dir` not set | string | A file to load for configuring CTS. Configuration file requires an .hcl or .json extension in order to specify their format. This option can be specified multiple times to load different configuration files. | none | -| `-inspect` | Optional | boolean | Run CTS in Inspect mode to print the proposed state changes for all tasks, and then exit. No changes are applied in this mode. | false | -| `-inspect-task`                             | Optional | string | Run CTS in Inspect mode to print the proposed state changes for the task, and then exit. The flag can be specified multiple times to inspect multiple tasks. No changes are applied in this mode. | none | -| `-once` | Optional | boolean | Render templates and run tasks once. Does not start the process in long-running mode and disables buffer periods. | false | - -## New CLI Autocompletion - -In v0.6.0, the `consul-terraform-sync` command features opt-in autocompletion for flags, subcommands, and -arguments (where supported). - -To enable autocompletion, run: - -```shell-session -$ consul-terraform-sync -autocomplete-install -``` - -~> After you install autocomplete, you must restart your shell for the change to take effect. - -When you start typing a CTS command, press the `` key to show a -list of available completions. To show available flag completes, type `-`. - -If the `CTS_*` environment variables are set, the autocompletion -automatically queries the running CTS server and returns helpful argument suggestions. - -## Addition of Features to Support HCP Consul - -In v0.6.0, CTS officially supports HCP Consul. To help enable this support, v0.6.0 introduces license auto-retrieval (**Enterprise only**). The auto-retrieval feature works for both on-premise Consul Enterprise and HCP Consul, but it is especially useful for HCP Consul because the licenses for this service expire much sooner. - -### License - -#### Setting the License Manually -~> In v0.6.0, CTS is configured by default to automatically retrieve a license from its configured Consul Enterprise backend when it starts. It also retrieves new licenses periodically. - -If a license needs to be manually set, use one of the [existing methods](/docs/nia/enterprise/license#setting-the-license), or use the new license block: - - ```hcl - license { - path = "//" - } - ``` - -#### Updating the License Manually -~> CTS automatically retrieves a license from its configured Consul Enterprise backend by default. If this default settings is disabled, follow the instructions under [Updating the License](/docs/nia/enterprise/license#updating-the-license). - -#### License Expiration Handling -As the license expiration date approaches, CTS Enterprise automatically retrieves a new license by default. If the automatic license retrieval fails, CTS increases the number of retrieval attempts. If auto-retrieval is disabled, CTS Enterprise provides notifications in the system logs: - -| Time period | Behavior - auto-retrieval enabled (default) |Behavior - auto-retrieval disabled | -| ------------------------------------------- |-------------------------------------------- |---------------------------------- | -| 30 days before expiration | License retrieval attempt every 24-hours | Warning-level log every 24-hours | -| 7 days before expiration | License retrieval attempt every 1 hour | Warning-level log every 1 hour | -| 1 day before expiration | License retrieval attempt every 5 minutes | Warning-level log every 5 minutes | -| 1 hour before expiration | License retrieval attempt every 1 minute | Warning-level log every 1 minute | -| At or after expiration (before termination) | License retrieval attempt every 1 minute | Error-level log every 1 minute | -| At or after termination | Error-level log and exit | Error-level log and exit | - -#### License Configuration - -~> [`license_path`](/docs/nia/configuration#license_path) is deprecated as of v0.6.0. The preferred way to set the license configuration is to use the 'license' block. - -To configure how CTS attains a license, use the 'license' block. You can: -- Provide a path to a license file. -- Configure CTS to automatically retrieve a license from Consul. -If a license block is not configured, 'license.auto_retrieval.enabled' defaults to true. - -```hcl -license { - path = "path/to/license.lic" - auto_retrieval { - enabled = true - } -} -``` - -##### License Config -| Parameter | Required | Type | Description | Default | -| --------- | -------- | ---- | ----------- | ------- | -| `path` | Optional | string | Configures the path to the file containing a license. If a path to a license is configured, this license is used until you enable automatic license retrieval. You can also set the license by defining the `CONSUL_LICENSE` and `CONSUL_LICENSE_PATH` environment variables. To learn more, review [Setting the License](/docs/nia/enterprise/license#setting-the-license). | none | -| `auto_retrieval` | Optional | object | Configures the license auto-retrieval used by CTS. To learn more, review [Auto-Retrieval Config](/docs/release-notes/consul-terraform-sync/v0_6_0_beta#auto-retrieval-config) for details | Review [Auto-Retrieval Config](/docs/release-notes/consul-terraform-sync/v0_6_0_beta#auto-retrieval-config) for defaults. | - - -##### Auto-Retrieval Config - -You can use the `auto_retrieval` block to configure the the automatic license retrieval in CTS. When enabled, CTS attempts to retrieve a new license from its configured Consul Enterprise backend once a day. If CTS cannot retrieve a license and the current license is reaching its expiration date, CTS attempts to retrieve a license with increased frequency, as defined by the [License Expiration Date Handling](/docs/release-notes/consul-terraform-sync/v0_6_0_beta#license-expiration-handling). - -~> `auto_retrieval` is an important feature to enable when you connect to HCP Consul. To ensure that CTS can work without restarting the service, it is important for CTS to have the ability to dynamically obtain fresh licenses from HCP Consul. - -| Parameter | Required | Type | Description | Default | -| --------- | -------- | ---- | ----------- | ------- | -| `enabled` | Optional | string | If set to true, enables license auto-retrieval | true | - -## Additional Features to Support Terraform Cloud (TFC) Agents - -In v0.6.0, CTS officially supports [Terraform Cloud agents](https://www.terraform.io/cloud-docs/agents). Previously, CTS could use only existing workspaces that were manually configured with Terraform Cloud agents. CTS now supports creating new workspaces that use agents and updating existing workspaces to use agents. - -### Terraform Cloud Remote Workspaces - -Use the [`terraform_cloud_workspace`](/docs/release-notes/consul-terraform-sync/v0_6_0_beta#terraform-cloud-workspace-configuration) configuration in the [`task`](/docs/nia/configuration#task) configuration block to enable Terraform Cloud agents for a task's workspace. The `terraform_cloud_workspace` controls options for Terraform version, execution mode, and agent pools. Note that these options are different from the global workspace configurations set on the [`driver.workspaces`](/docs/nia/configuration#workspaces) configuration. - -For a task to use Terraform Cloud agents, `terraform_cloud_workspace.execution_mode` must be set to "agent" and either `terraform_cloud_workspace.agent_pool_id` or `terraform_cloud_workspace.agent_pool_name` must be specified. - -#### Terraform Cloud Workspace Configuration -~> Configuring [`terraform_version`](/docs/nia/configuration#terraform_version) is deprecated as of v0.6.0. Use `terraform_cloud_workspace.terraform_version` instead. - -You can use the new `terraform_cloud_workspace` `task` configuration object to configure attributes of the Terraform Cloud workspace associated with the task. This option is only available when used with the [Terraform Cloud driver](/docs/nia/configuration#terraform-cloud-driver). For global configurations of all workspaces, review [`driver.workspaces`](/docs/nia/configuration#workspaces). - -```hcl -task { - name = "terraform_cloud_agent_task" - description = "execute on changes to services with names starting with web using cloud agents" - providers = ["my-provider"] - module = "path/to/services-condition-module" - - condition "services" { - regexp = "^web.*" - datacenter = "dc1" - } - - terraform_cloud_workspace { - execution_mode = "agent" - agent_pool_id = "apool-v9ZZW1qACRjTyzWh" - terraform_version = "1.1.5" - } -} -``` - -The configuration options are: - -| Name | Required | Type | Description |Default | -| --------------- | -------- | ------- | ------------------------------- | ----------------------- | -| `execution_mode `      | Optional | string | The execution mode that determines whether to use Terraform Cloud as the Terraform execution platform. Only supports "remote" or "agent".| "remote" | -| `agent_pool_id`                        | Optional | string | The id of the agent pool that should run the Terraform workloads. Only supported if `execution_mode` is set to "agent". Either `agent_pool_id` or `agent_pool_name` are required. `agent_pool_id` takes precedence over `agent_pool_name` if both are provided. | none | -| `agent_pool_name` | Optional | string | The name of the agent pool that should run the Terraform workloads. Only supported if `execution_mode` is set to "agent". Either `agent_pool_id` or `agent_pool_name` are required. `agent_pool_id` takes precedence over `agent_pool_name` if both are provided. | none | -| `terraform_version`                             | Optional | string | The version of Terraform to use for the Terraform Cloud workspace associated with the task. Defaults to the latest compatible version supported by the organization. | none | - -## Get All Tasks Endpoint - -The Get `/tasks` endpoint returns information about all existing tasks in CTS. - -| Method | Path | Produces | -| ------ | ------------------- | ------------------ | -| `GET` | `/tasks` | `application/json` | - -### Response Statuses - -| Status | Reason | -| ------ | ---------------------------------------------------- | -| 200 | Successfully retrieved and returned a list of tasks | - -### Response Fields - -| Name | Type | Description | -| ------------ | ------------- | ---------------------------------------------------------------------------------- | -| `request_id` | string | The ID of the request. Used for auditing and debugging purposes. | -| `tasks` | list[[Task](/docs/nia/api/tasks#task-object)] | A list of Tasks | - -### Example: Retrieve all tasks - -In this example, CTS contains a single task - -Request: - -```shell-session -$ curl --request GET \ - localhost:8558/v1/tasks -``` - -Response: - -```json -{ - "request_id": "0e0290f9-94df-3a4a-fd17-72467a16083c", - "tasks": [ - { - "buffer_period": { - "enabled": true, - "max": "25s", - "min": "5s" - }, - "condition": { - "services": { - "cts_user_defined_meta": {}, - "datacenter": "", - "filter": "", - "names": ["api", "web"], - "namespace": "", - "use_as_module_input": true - } - }, - "description": "Writes the service name, id, and IP address to a file", - "enabled": true, - "module": "path/to/module", - "module_input": {}, - "name": "task_a", - "providers": ["my-provider"], - "variables": {}, - "version": "" - } - ] -} -``` - -## Health Endpoint -The `/health` endpoint returns a successful response if CTS is available and running. Requests to this endpoint are not logged, which makes it suitable for health checks that constantly poll CTS. - -| Method | Path | Produces | -| ------ | ------------------- | ------------------ | -| `GET` | `/health` | `application/json` | - -### Response Statuses - -| Status | Reason | -| ------ | ---------------------------------------------------- | -| 200 | CTS is healthy | - -### Response Fields - -| Name | Type | Description | -| ------------ | ------ | ---------------------------------------------------------------------------------- | -| `request_id` | string | The ID of the request. Used for auditing and debugging purposes. | - -### Example - -The following request makes a `GET` call to the `health` endpoint: - -```shell-session -$ curl --request GET \ - localhost:8558/v1/health -``` - -Response: - -```json -{ - "request_id": "b7559ab0-5111-381b-367a-0dfb7e216d41" -} -``` - -## CTS Service Registration With Consul -By default, CTS automatically registers itself as a service in Consul with a health check. The default health check is an HTTP check to the [health API endpoint](#health-endpoint). CTS deregisters itself with Consul when CTS stops gracefully. - -If ACLs are enabled in Consul, then the [Consul token](/docs/nia/configuration#token) must have the policy `service:write` for service registration to succeed. Previously, CTS only required read access for services, so tokens used prior to 0.6.0 may not work with service registration. - -If CTS is unable to register with Consul, then it will log the error and continue without exiting. - -### Configuration -Specify the global configuration `id` to assign instances of CTS an ID, which is used as the service ID for CTS if service registration is enabled. Configure the `service_registration` block in the `consul` configuration to configure the registration details. All configurations are optional. - -```hcl -id = "cts-01" - -consul { - service_registration { - enabled = true - service_name = "cts" - namespace = "ns-1" - default_check { - enabled = true - address = "http://172.22.0.2:8558" - } - } -} -``` - -The following table describes the options for the `id` global configuration: - -| Name | Required | Type | Description |Default | -| --------------- | -------- | ------- | ------------------------------- | ----------------------- | -| `id ` | Optional | string | The ID of the CTS instance. Used as the service ID for CTS if service registration is enabled. | Generated ID with the format `cts-` | - - -The following table describes configuration options for `consul.service_registration`: - -| Name | Required | Type | Description |Default | -| --------------- | -------- | ------- | ------------------------------- | ----------------------- | -| `enabled ` | Optional | boolean | Enables CTS to register itself as a service with Consul. | `true` | -| `service_name` | Optional | string | The service name for CTS. | `Consul-Terraform-Sync` | -| `namespace` | Optional | string | The namespace to register CTS in. | In order of precedence:
1. Inferred from the CTS ACL token
2. The `default` namespace. | -| `default_check.enabled` | Optional | boolean | Enables CTS to create the default health check. | `true` | -| `default_check.address` | Optional | string | The address to use for the default HTTP health check. Needs to include the scheme (`http`/`https`) and the port (if necessary). | `http://localhost:` or `https://localhost:`. Determined from the [port configuration](/docs/nia/configuration#port) and whether [TLS is enabled](/docs/nia/configuration#enabled-2) on the CTS API. | diff --git a/website/content/docs/release-notes/consul-terraform-sync/v0_6_x.mdx b/website/content/docs/release-notes/consul-terraform-sync/v0_6_x.mdx new file mode 100644 index 0000000000..31fb9a8983 --- /dev/null +++ b/website/content/docs/release-notes/consul-terraform-sync/v0_6_x.mdx @@ -0,0 +1,59 @@ +--- +layout: docs +page_title: 0.6.x +description: >- + Consul-Terraform-Sync release notes for version 0.6.x +--- + +# Consul-Terraform-Sync 0.6.0 + +## Release Highlights + + +We have implemented the following features in this release: + +### Per-task Execution Mode with Support for Terraform Cloud Agent + +You can now execute Terraform tasks in `remote` or `cloud agent` mode. For more information, refer to the [per-task execution mode documentation](/docs/nia/network-drivers/terraform-cloud#remote-workspaces). + +### HCP Consul Support + +CTS now supports interoperability with HCP Consul. CTS retrieves licenses from HCP Consul so that users can keep their HCP Consul license or Consul enterprise deployment license in sync. + +### Auto Service-registration with Consul and Health API Endpoint + +CTS includes a new configuration block that automatically registers CTS as a service within Consul as soon as CTS is instantiated. This eliminates the manual step of registering CTS as a service within the Consul catalog. This service registration also enables health checking on CTS using the newly added [health API endpoint](/docs/nia/api/health#health). + +### CLI Discoverability + +We improved CTS's CLI discoverability functionality. The `consul-terraform-sync` command features opt-in [autocompletion](/docs/nia/cli#autocompletion) for flags, subcommands, and arguments + +### Health Check API + +CTS includes a new [API endpoint](/docs/nia/api/health#health) that provides operators with the health status of their CTS instance. + + +## Supported Software Versions + +- Consul: v1.9+ +- HCP Consul: Latest +- Terraform CLI: v0.13 - v1.1 +- Terraform Cloud: Latest +- Terraform Enterprise: v202010-2 - Latest + + +## What's Changed + +**Deprecations** + +* [[GH-790](https://github.com/hashicorp/consul-terraform-sync/issues/790)] Deprecate `terraform_version` task configuration in favor of `terraform_cloud_workspace.terraform_version` task configuration +* [[GH-810](https://github.com/hashicorp/consul-terraform-sync/issues/810)] Deprecate `license_path` configuration in favor of `license.path` configuration +* [[GH-866](https://github.com/hashicorp/consul-terraform-sync/issues/866)] Deprecate starting CTS as a daemon using no CLI command in favor of starting CTS as a daemon using the new `start` command + +## Changelogs + +The changelogs for this major release version and any maintenance versions are listed below. + +-> **Note**: These links will redirect you to the changelogs on the GitHub website. + +- [0.6.0](https://github.com/hashicorp/consul-terraform-sync/releases/tag/v0.6.0) diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 3199d40fae..f009f56112 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -789,6 +789,10 @@ { "title": "Tasks", "path": "nia/api/tasks" + }, + { + "title": "Health", + "path": "nia/api/health" } ] }, @@ -802,6 +806,10 @@ { "title": "task", "path": "nia/cli/task" + }, + { + "title": "start", + "path": "nia/cli/start" } ] }, @@ -1254,12 +1262,12 @@ "title": "Consul Terraform Sync", "routes": [ { - "title": "v0.5.x", - "path": "release-notes/consul-terraform-sync/v0_5_x" + "title": "v0.6.x", + "path": "release-notes/consul-terraform-sync/v0_6_x" }, { - "title": "v0.6.0-beta", - "path": "release-notes/consul-terraform-sync/v0_6_0_beta" + "title": "v0.5.x", + "path": "release-notes/consul-terraform-sync/v0_5_x" } ] }