From 1b2df5388cdd34bb3579824f77b89dd52af90de2 Mon Sep 17 00:00:00 2001 From: Freddy Date: Fri, 17 Jun 2022 18:40:38 -0600 Subject: [PATCH] Additional service mesh docs updates for peering (#13464) This PR covers two sets of changes: - Documenting the new `destination_peer` for proxy upstream definitions. - Updating the exported-services config entry documentation. Updates to the `exported-services` config entry include: - As of 1.13.0 it is no longer only for Consul Enterprise - A `PeerName` is now a possible consumer for an exported service. - Added examples for OSS and Enterprise - Linked to peering docs --- .../config-entries/exported-services.mdx | 591 ++++++++++++++++-- .../registration/service-registration.mdx | 21 +- 2 files changed, 543 insertions(+), 69 deletions(-) diff --git a/website/content/docs/connect/config-entries/exported-services.mdx b/website/content/docs/connect/config-entries/exported-services.mdx index b822a039c6..60a247e438 100644 --- a/website/content/docs/connect/config-entries/exported-services.mdx +++ b/website/content/docs/connect/config-entries/exported-services.mdx @@ -8,22 +8,20 @@ description: >- # Exported Services - +This topic describes the `exported-services` configuration entry type. The `exported-services` configuration entry enables Consul to export service instances to other clusters from a single file and connect services across clusters. For additional information, refer to [Cluster Peering](/docs/connect/cluster-peering) and [Admin Partitions](/docs/enterprise/admin-partitions). -This topic describes the `exported-services` configuration entry type. The `exported-services` configuration entry enables Consul to export service instances to other admin partitions from a single file. This enables your services to be networked across admin partitions. See [Admin Partitions](/docs/enterprise/admin-partitions) for additional information. - --> **v1.11.0+:** This config entry is supported in Consul Enterprise versions 1.11.0+. +-> **v1.11.0+:** This config entry is supported in Consul versions 1.11.0+. ## Introduction -You can configure Consul to export services contained in an admin partition to one or more additional partitions by declaring the `exported-services` configuration entry in the `kind` field. This enables you to route traffic between services in different clusters that share a single set of Consul servers. +To configure Consul to export services contained in a Consul Enterprise admin partition or Consul OSS datacenter to one or more additional clusters, create a new configuration entry and declare `exported-services` in the `kind` field. This configuration entry enables you to route traffic between services in different clusters. -You can configure the settings defined in the `exported-services` configuration entry to apply to all namespaces and federated datacenters. +You can configure the settings defined in the `exported-services` configuration entry to apply to all namespaces in a Consul Enterprise admin partition. ## Requirements -- A Consul Enterprise binary -- A corresponding partition that the configuration entry can export to. For example, the `exported-services` configuration entry for a partition named `frontend` requires an existing `frontend` partition. +- A 1.11.0+ Consul Enteprise binary or a 1.13.0+ Consul OSS binary. +- **Enterprise Only**: A corresponding partition that the configuration entry can export from. For example, the `exported-services` configuration entry for a partition named `frontend` requires an existing `frontend` partition. ## Usage @@ -37,6 +35,56 @@ You can configure the settings defined in the `exported-services` configuration Configure the following parameters to define a `exported-services` configuration entry: + + + + +```hcl +Kind = "exported-services" +Name = "default" +Services = [ + { + Name = "" + Consumers = [ + { + PeerName = "" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +kind: ExportedServices +metadata: + name: default +spec: + services: + - name: + consumers: + - peerName: +``` + +```json +"Kind": "exported-services", +"Name": "default", +"Services": [ + { + "Name": "", + "Consumers": [ + { + "PeerName": "" + } + ] + } +] +``` + + + + + ```hcl @@ -49,8 +97,8 @@ Services = [ Namespace = "" Consumers = [ { - Partition = "" - }, + PeerName = "" + } ] } ] @@ -66,7 +114,7 @@ spec: - name: namespace: consumers: - - partition: + - peerName: ``` ```json @@ -75,18 +123,73 @@ spec: "Name": "", "Services": [ { - "Consumers": [ - { - "Partition": "" - } - ], "Name": "", "Namespace": "" + "Consumers": [ + { + "PeerName": "" + } + ] } ] ``` + + + + + +```hcl +Kind = "exported-services" +Partition = "" +Name = "" +Services = [ + { + Name = "" + Namespace = "" + Consumers = [ + { + Partition = "" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +kind: ExportedServices +metadata: + name: +spec: + services: + - name: + namespace: + consumers: + - partition: +``` + +```json +"Kind": "exported-services", +"Partition": "", +"Name": "", +"Services": [ + { + "Name": "", + "Namespace": "" + "Consumers": [ + { + "Partition": "" + } + ] + } +] +``` + + + + ### Configuration Parameters @@ -94,25 +197,113 @@ The following table describes the parameters associated with the `exported-servi | Parameter | Description | Required | Default | | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------- | -| `Kind` | String value that enables the configuration entry. The value should always be `exported-services` (HCL and JSON) or `ExportedServices` (YAML) | Required | None | -| `Partition` | String value that specifies the name of the partition that contains the services you want to export. | Required | None | -| `Name` | String value that specifies the name of the partition that contains the services you want to export. | Required | None | -| `Services` | List of objects that specify which services to export. See [`Services`](#services) for details. | Required | None | -| `Meta` | Object that defines a map of the max 64 key/value pairs. | Optional | None | +| `Kind` | String value that enables the configuration entry. The value should always be `exported-services` (HCL and JSON) or `ExportedServices` (YAML) | Required | None | +| `Partition` | String value that specifies the name of the partition that contains the services you want to export. | Required | None | +| `Name` | String value that specifies the name of the partition that contains the services you want to export. Must be `default` in Consul OSS. | Required | None | +| `Services` | List of objects that specify which services to export. For details, refer to [`Services`](#services). | Required | None | +| `Meta` | Object that defines a map of the max 64 key/value pairs. | Optional | None | ### Services -The `Services` parameter contains one or more lists of parameters that specify which services to export, which namespaces the services reside, and the destination partition for the exported services. Each list in the `Services` block must contain the following parameters: +The `Services` parameter contains a list of one or more parameters that specify which services to export, which namespaces the services reside, and the destination cluster for the exported services. Each item in the `Services` list must contain the following parameters: -- `Name`: Specifies the name of the service to export. You can use a asterisk wildcard (`*`) to include all services in the namespace. -- `Namespace`: Specifies the namespace containing the services to export. You can use a asterisk wildcard (`*`) to include all namespaces in the partition. -- `Consumers`: Specifies one ore more objects that identify a destination partition for the exported services. +- `Name`: Specifies the name of the service to export. You can use an asterisk wildcard (`*`) to include all services in the namespace. +- `Namespace`: Specifies the namespace containing the services to export. You can use an asterisk wildcard (`*`) to include all namespaces in the partition. +- `Consumers`: Specifies one or more objects that identify a destination cluster for the exported services. -## Example +### Consumers -The following example configures the agent to export the `billing` service from the `default` namespace of the `finance` admin partition to the `frontend` and `backend` partitions. Additionally, all services in all namespaces within the `finance` partition will be exported to the `monitoring` partition. +The `Consumers` parameter contains a list of one or more parameters that specify the destination cluster for +an exported service. Each item in the `Consumers` list must contain exactly one of the following parameters: - +- `PeerName`: Specifies the name of the peered cluster to export the service to. +A asterisk wildcard (`*`) cannot be specified as the `PeerName`. Added in Consul 1.13.0. +- `Partition`: Specifies an admin partition in the datacenter to export the service to. +A asterisk wildcard (`*`) cannot be specified as the `Partition`. + +## Examples + + +### Exporting services to peered clusters + + + + + +The following example configures Consul to export the `payments` and `refunds` services to the peered `web-shop` cluster. + + + +```hcl +Kind = "exported-services" +Name = "default" + +Services = [ + { + Name = "payments" + Consumers = [ + { + PeerName = "web-shop" + }, + ] + }, + { + Name = "refunds" + Consumers = [ + { + PeerName = "web-shop" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +Kind: ExportedServices +metadata: + name: default +spec: + services: + - name: payments + consumers: + - peerName: web-shop + - name: refunds + consumers: + - peerName: web-shop +``` + +```json +"Kind": "exported-services", + "Name": "default", + "Services": [ + { + "Name": "payments", + "Consumers": [ + { + "PeerName": "web-shop" + }, + ], + }, + { + "Name": "refunds", + "Consumers": [ + { + "PeerName": "web-shop" + } + ] + } + ] +``` + + + + + + +The following example configures Consul to export the `payments` and `refunds` services from the `billing` namespace of the `finance` admin partition to the `web-shop` peer. + + ```hcl Kind = "exported-services" @@ -121,23 +312,20 @@ Name = "finance" Services = [ { - Name = "billing" - Namespace = "default" + Name = "payments" + Namespace = "billing" Consumers = [ { - Partition = "frontend" + PeerName = "web-shop" }, - { - Partition = "backend" - } ] }, { - Name = "*" - Namespace = "*" + Name = "refunds" + Namespace = "billing" Consumers = [ { - Partition = "monitoring" + PeerName = "web-shop" } ] } @@ -151,15 +339,14 @@ metadata: name: finance spec: services: - - name: mesh-gateway - namespace: default + - name: payments + namespace: billing consumers: - - partition: default - - name: billing - namespace: default + - peerName: web-shop + - name: refunds + namespace: billing consumers: - - partition: frontend - - partition: backend + - peerName: web-shop ``` ```json @@ -168,41 +355,327 @@ spec: "Name": "finance", "Services": [ { + "Name": "payments", + "Namespace": "billing" "Consumers": [ { - "Partition": "frontend" + "PeerName": "web-shop" }, - { - "Partition": "backend" - } ], - "Name": "billing", - "Namespace": "default" }, { + "Name": "refunds", + "Namespace": "billing", "Consumers": [ { - "Partition": "monitoring" + "PeerName": "web-shop" } - ], - "Name": "*", - "Namespace": "*" + ] } ] ``` + + + + +The following example configures Consul to export the `payments` and `refunds` services from the `billing` namespace of the `finance` admin partition to the `web-shop` partition. + + + +```hcl +Kind = "exported-services" +Partition = "finance" +Name = "finance" + +Services = [ + { + Name = "payments" + Namespace = "billing" + Consumers = [ + { + Partition = "web-shop" + }, + ] + }, + { + Name = "refunds" + Namespace = "billing" + Consumers = [ + { + Partition = "web-shop" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +Kind: ExportedServices +metadata: + name: finance +spec: + services: + - name: payments + namespace: billing + consumers: + - partition: web-shop + - name: refunds + namespace: billing + consumers: + - partition: web-shop +``` + +```json +"Kind": "exported-services", + "Partition": "finance", + "Name": "finance", + "Services": [ + { + "Name": "payments", + "Namespace": "billing" + "Consumers": [ + { + "Partition": "web-shop" + }, + ], + }, + { + "Name": "refunds", + "Namespace": "billing", + "Consumers": [ + { + "Partition": "web-shop" + } + ] + } + ] +``` + + + + + +### Exporting all services + + + + +The following example configures Consul to export all services in the datacenter to the peered `monitoring` and `platform` clusters. + + + +```hcl +Kind = "exported-services" +Name = "default" + +Services = [ + { + Name = "*" + Consumers = [ + { + PeerName = "monitoring" + }, + { + PeerName = "platform" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +Kind: ExportedServices +metadata: + name: default +spec: + services: + - name: * + consumers: + - peerName: monitoring + - peerName: platform +``` + +```json +"Kind": "exported-services", + "Name": "default", + "Services": [ + { + "Name": "*", + "Namespace": "*" + "Consumers": [ + { + "PeerName": "monitoring" + }, + { + "PeerName": "platform" + } + ] + } + ] +``` + + + + + + +The following example configures Consul to export all services in all namespaces of the `finance` partition to the peered `monitoring` and `platform` clusters. + + + +```hcl +Kind = "exported-services" +Partition = "finance" +Name = "finance" + +Services = [ + { + Name = "*" + Namespace = "*" + Consumers = [ + { + PeerName = "monitoring" + }, + { + PeerName = "platform" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +Kind: ExportedServices +metadata: + name: finance +spec: + services: + - name: * + namespace: * + consumers: + - peerName: monitoring + - peerName: platform +``` + +```json +"Kind": "exported-services", + "Partition": "finance", + "Name": "finance", + "Services": [ + { + "Name": "*", + "Namespace": "*" + "Consumers": [ + { + "PeerName": "monitoring" + }, + { + "PeerName": "platform" + } + ] + } + ] +``` + + + + + + +The following example configures Consul to export all services in all namespaces of the `finance` partition to the `monitoring` and `platform` partitions. + + + +```hcl +Kind = "exported-services" +Partition = "finance" +Name = "finance" + +Services = [ + { + Name = "*" + Namespace = "*" + Consumers = [ + { + Partition = "monitoring" + }, + { + Partition = "platform" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +Kind: ExportedServices +metadata: + name: finance +spec: + services: + - name: * + namespace: * + consumers: + - partition: monitoring + - partition: platform +``` + +```json +"Kind": "exported-services", + "Partition": "finance", + "Name": "finance", + "Services": [ + { + "Name": "*", + "Namespace": "*" + "Consumers": [ + { + "Partition": "monitoring" + }, + { + "Partition": "platform" + } + ] + } + ] +``` + + + + ## Reading Services -When an exported service has been imported to another partition, you can use the `health` REST API endpoint to query the service on the consumer partition. The following example queries the `finance` partition for the imported `billing` service: +When an exported service has been imported to another cluster, you can use the `health` REST API endpoint to query the service on the consumer cluster. + + + + +The following example queries the `finance` peer for the imported `payments` service: ```shell-session -$ curl 'localhost:8500/v1/health/connect/billing?partition=finance' +$ curl 'localhost:8500/v1/health/service/payments?peer=finance' ``` + -An ACL token with `service:write` permissions is required for the partition from which the query is made. If the call in the previous example is made from a service named `web` in a partition named `frontend`, then the request will require a token with `write` permissions to `web` in the `frontend` partition. + -Exports are available to all services in the consumer partition. In the previous example, any service with `write` permissions for the `frontend` partition will be able to read exports. +The following example queries the `finance` partition for the imported `payments` service: -See [Health HTTP Endpoint](/api-docs/health) for additional information. +```shell-session +$ curl 'localhost:8500/v1/health/service/payments?partition=finance' +``` + + + +An ACL token with `service:write` permissions is required for the cluster the query is made from. If the call in the previous example is made from a service named `web` in a partition named `frontend`, then the request requires a token with `write` permissions to `web` in the `frontend` partition. + +Exports are available to all services in the consumer cluster. In the previous example, any service with `write` permissions for the `frontend` partition can read exports. + +For additional information, refer to [Health HTTP Endpoint](/api-docs/health). diff --git a/website/content/docs/connect/registration/service-registration.mdx b/website/content/docs/connect/registration/service-registration.mdx index 3deac820e5..66b1aa96f7 100644 --- a/website/content/docs/connect/registration/service-registration.mdx +++ b/website/content/docs/connect/registration/service-registration.mdx @@ -35,13 +35,13 @@ port = "destination_service_name": "", "" : "" }, - "port": + "port": } ``` -The following table describes the parameters that must be added to the service definition to declare the service as a proxy. +The following table describes the parameters that must be added to the service definition to declare the service as a proxy. | Parameter | Description | Required | Default | | --- | --- | --- | --- | @@ -82,8 +82,8 @@ proxy = { ### Sidecar Proxy Configuration -Many service mesh proxies are deployed as sidecars. -Sidecar proxies are co-located with the single service instance they represent and proxy all inbound traffic to. +Many service mesh proxies are deployed as sidecars. +Sidecar proxies are co-located with the single service instance they represent and proxy all inbound traffic to. Specify the following parameters in the `proxy` code block to configure a sidecar proxy in its own service registration: @@ -99,7 +99,7 @@ The following example includes values for all available options when registering -```hcl +```hcl kind = "connect-proxy" name = "redis-proxy" port = 8181 @@ -166,6 +166,7 @@ You can configure the service mesh proxy to create listeners for upstream servic | --- | --- | --- | --- | |`destination_name` | String value that specifies the name of the service or prepared query to route the service mesh to. The prepared query should be the name or the ID of the prepared query. | Required | None | | `destination_namespace` | String value that specifies the namespace containing the upstream service. | Optional | `default` | +| `destination_peer` | String value that specifies the name of the peer cluster containing the upstream service. | Optional | None | | `destination_partition` | String value that specifies the name of the admin partition containing the upstream service. | Optional | `default` | | `local_bind_port` | Integer value that specifies the port to bind a local listener to. The application will make outbound connections to the upstream from the local port. | Required | None | | `local_bind_address` | String value that specifies the address to bind a local listener to. The application will make outbound connections to the upstream service from the local bind address. | Optional | `127.0.0.1` | @@ -265,7 +266,7 @@ You can configure which mode a proxy operates in by specifying `"direct"` or `"t * `transparent`: In this mode, inbound and outbound application traffic is captured and redirected through the proxy. This mode does not enable the traffic redirection. It directs Consul to configure Envoy as if traffic is already being redirected. * `direct`: In this mode, the proxy's listeners must be dialed directly by the local application and other proxies. -You can also specify an empty string (`""`), which configures the proxy to operate in the default mode. The default mode is inherited from parent parameters in the following order of precedence: +You can also specify an empty string (`""`), which configures the proxy to operate in the default mode. The default mode is inherited from parent parameters in the following order of precedence: 1. Proxy service's `Proxy` configuration 1. The `service-defaults` configuration for the service. @@ -294,12 +295,12 @@ registrations](/docs/discovery/services#service-definition-parameter-case). - `outbound_listener_port` `(int: 15001)` - The port the proxy should listen on for outbound traffic. This must be the port where outbound application traffic is captured and redirected to. - `dialed_directly` `(bool: false)` - Determines whether this proxy instance's IP address can be dialed - directly by transparent proxies. Typically transparent proxies dial upstreams using the "virtual" - tagged address, which load balances across instances. Dialing individual instances can be helpful - in cases like stateful services such as a database cluster with a leader. + directly by transparent proxies. Transparent proxies typically dial upstreams using the "virtual" + tagged address, which load balances across instances. A database cluster with a leader is an example + where dialing individual instances can be helpful. ~> **Note:** Dynamic routing rules such as failovers and redirects do not apply to services dialed directly. - Additionally, the connection is proxied using a TCP proxy with a connection timeout of 5 seconds. + Additionally, the connection is proxied using a TCP proxy with a connection timeout of 5 seconds. ### Mesh Gateway Configuration Reference