Merge pull request #14770 from hashicorp/docs/agentless-beta

docs: Consul Dataplane (Agentless) Beta

website/content/docs/connect/dataplane/consul-dataplane.mdx

@@ -0,0 +1,130 @@
+---
+layout: docs
+page_title: Consul Dataplane CLI Reference
+description: >-
+   Consul Dataplane runs as a separate binary controlled with the `consul-dataplane` CLI command. Learn how to use this command to configure your dataplane on Kubernetes with this reference guide and example code.
+---
+
+# Consul Dataplane CLI Reference
+
+The `consul-dataplane` command interacts with the binary for [simplified service mesh with Consul Dataplane](/consul/docs/k8s/dataplane/index). Use this command to install Consul Dataplane, configure its Envoy proxies, and secure Dataplane deployments.
+
+## Usage
+
+Usage: `consul-dataplane [options]`
+
+### Requirements
+
+Consul Dataplane requires servers running Consul version `v1.14-beta+`. To find a specific version of Consul, refer to [Hashicorp's Official Release Channels](https://www.hashicorp.com/official-release-channels).
+
+### Startup
+
+The following options are required when starting `consul-dataplane` with the CLI:
+
+- `-addresses`
+- `-service-node-name`
+- `-proxy-service-id`
+
+### Command Options
+
+- `-addresses` - Consul server gRPC addresses. Can be a DNS name or an executable command. Refer to [go-netaddrs](https://github.com/hashicorp/go-netaddrs#summary) for details and examples.
+- `-ca-certs` - The path to a file or directory containing CA certificates used to verify the server's certificate.
+- `-credential-type` - The type of credentials used to authenticate with Consul servers, either `"static"` or `"login"`.
+- `-envoy-admin-bind-address` - The address the Envoy admin server is available on. Default is `"127.0.0.1"`.
+- `-envoy-admin-bind-port` - The port the Envoy admin server is available on. Default is `19000`.
+- `-envoy-concurrency` - The number of worker threads that Envoy uses. Default is `2`.
+- `-envoy-ready-bind-address` - The address Envoy's readiness probe is available on.
+- `-grpc-port` - The Consul server gRPC port to which consul-dataplane connects. Default is `8502`.
+- `-log-json` - Enables log messages in JSON format. Default is `false`.
+- `-log-level` - Log level of the messages to print. Available log levels are `"trace"`, `"debug"`, `"info"`, `"warn"`, and `"error"`. Default is `"info"`.
+- `-login-auth-method`  - The auth method used to log in.
+- `-login-bearer-token` - The bearer token presented to the auth method.
+- `-login-bearer-token-path` - The path to a file containing the bearer token presented to the auth method.
+- `-login-datacenter` - The datacenter containing the auth method.
+- `-login-meta` - A set of key/value pairs to attach to the ACL token. Each pair is formatted as `<key>=<value>`. This flag may be passed multiple times.
+- `-login-namespace` <EnterpriseAlert inline /> - The Consul Enterprise namespace containing the auth method.
+- `-login-partition` <EnterpriseAlert inline /> - The Consul Enterprise partition containing the auth method.
+- `-proxy-service-id` - The proxy service instance's ID.
+- `-server-watch-disabled` - Prevent `consul-dataplane` from consuming the server update stream. Use this flag when Consul servers are behind a load balancer. Default is `false`.
+- `-service-namespace`  <EnterpriseAlert inline /> - The Consul Enterprise namespace in which the proxy service instance is registered.
+- `-service-node-id` - The ID of the Consul node to which the proxy service instance is registered.
+- `-service-node-name` - The name of the Consul node to which the proxy service instance is registered.
+- `-service-partition` <EnterpriseAlert inline /> - The Consul Enterprise partition in which the proxy service instance is registered.
+- `-static-token` - The ACL token used to authenticate requests to Consul servers when `-credential-type` is set to `"static"`.
+- `-telemetry-use-central-config` - Controls whether the proxy applies the central telemetry configuration. Default is `true`.
+- `-tls-cert` - The path to a client certificate file. This flag is required if `tls.grpc.verify_incoming` is enabled on the server.
+- `-tls-disabled` - Communicate with Consul servers over a plaintext connection. Useful for testing, but not recommended for production. Default is `false`.
+- `-tls-insecure-skip-verify` - Do not verify the server's certificate. Useful for testing, but not recommended for production. Default is `false`.
+- `-tls-key` - The path to a client private key file. This flag is required if `tls.grpc.verify_incoming` is enabled on the server.
+- `-tls-server-name` - The hostname to expect in the server certificate's subject. This flag is required if `-addresses` is not a DNS name.
+- `-version` - Print the current version of `consul-dataplane`.
+- `-xds-bind-addr` - The address the Envoy xDS server is available on. Default is `"127.0.0.1"`.
+
+## Examples
+
+### DNS
+
+Consul Dataplane resolves a domain name to discover Consul server IP addresses.
+
+  ```shell-session
+  $ consul-dataplane -addresses my.consul.example.com
+  ```
+
+### Executable Command
+
+Consul Dataplane runs a script that, on success, returns one or more IP addresses separated by whitespace.
+
+  ```shell-session
+  $ ./my-script.sh
+  172.20.0.1
+  172.20.0.2
+  172.20.0.3
+
+  $ consul-dataplane -addresses "exec=./my-script.sh"
+  ```
+
+### Go Discover Nodes for Cloud Providers
+
+The [`go-discover`](https://github.com/hashicorp/go-discover) binary is included in the `hashicorp/consul-dataplane` image for use with this mode of server discovery, which functions in
+  a way similar to [Cloud Auto-join](/consul/docs/install/cloud-auto-join). The
+  following example demonstrates how to use the `go-discover` binary with Consul Dataplane.
+
+  ```shell-session
+  $ consul-dataplane -addresses "exec=discover -q addrs provider=aws region=us-west-2 tag_key=consul-server tag_value=true"
+  ```
+
+### Static token
+
+A static ACL token is passed to Consul Dataplane.
+
+  ```shell-session
+  $ consul-dataplane -credential-type "static"` -static-token "12345678-90ab-cdef-0000-12345678abcd"
+  ```
+
+### Auth method login
+
+Consul Dataplane logs in to one of Consul's supported [auth methods](/consul/docs/security/acl/auth-methods).
+
+  ```shell-session
+  $ consul-dataplane -credential-type "login"
+    -login-auth-method <method> \
+    -login-bearer-token <token> \  ## Or -login-bearer-token-path
+    -login-datacenter <datacenter> \
+    -login-meta key1=val1 -login-meta key2=val2 \
+    -login-namespace <namespace> \
+    -login-partition <partition>
+  ```
+
+### Consul Servers Behind a Load Balancer
+
+When Consul servers are behind a load balancer, you must pass `-server-watch-disabled` to Consul
+Dataplane.
+
+```shell-session
+$ consul-dataplane -server-watch-disabled
+```
+
+By default, Consul Dataplane opens a server watch stream to a Consul server, which enables the server
+to inform Consul Dataplane of new or different Consul server addresses. However, if Consul Dataplane
+is connecting through a load balancer, then it must ignore the Consul server addresses that are
+returned from the server watch stream.

website/content/docs/connect/dataplane/index.mdx

@@ -0,0 +1,78 @@
+---
+layout: docs
+page_title: Simplified Service Mesh with Consul Dataplane
+description: >-
+   Consul Dataplane removes the need to a run client agent for service discovery and service mesh by leveraging orchestrator functions. Learn about Consul Dataplane, how it can lower latency for Consul on Kubernetes, and how it enables Consul support for AWS Fargate and GKE Autopilot.
+---
+
+# Simplified Service Mesh with Consul Dataplane
+
+~> **Consul Dataplane is currently in beta:** Functionality associated with Consul Dataplane is subject to change. You should never use the beta release in secure environments or production scenarios. Features in beta may have performance issues, scaling issues, and limited support.
+
+This topic provides an overview of Consul Dataplane, a lightweight process for managing Envoy proxies introduced in Consul v1.14.0. Consul Dataplane removes the need to run client agents on every node in a cluster for service discovery and service mesh. Instead, Consul deploys sidecar proxies that provide lower latency, support additional runtimes, and integrate with cloud infrastructure providers.
+
+Consul Dataplane requires servers running Consul v1.14.0-beta1+.
+
+## What is Consul Dataplane?
+
+In standard deployments, Consul uses a control plane that contains both *server agents* and *client agents*. Server agents maintain the service catalog and service mesh, including its security and consistency, while client agents manage communications between service instances, their sidecar proxies, and the servers. While this model is optimal for applications deployed on virtual machines or bare metal servers, orchestrators such as Kubernetes already include components called *kubelets* that support health checking and service location functions typically provided by the client agent.
+
+Consul Dataplane manages Envoy proxies and leaves responsibility for other functions to the orchestrator. As a result, it removes the need to run client agents on every pod. In addition, services no longer need to be reregistered to a local client agent after restarting a service instance, as a client agent’s lack of access to persistent data storage in Kubernetes deployments is no longer an issue.
+
+## Benefits
+
+**Fewer networking requirements**: Without client agents, Consul does not require bidirectional network connectivity across multiple protocols to enable gossip communication. Instead, it requires a single gRPC connection to the Consul servers, which significantly simplifies requirements for the operator.
+
+**Simplified set up**: Because there are no client agents to engage in gossip, you do not have to generate and distribute a gossip encryption key to agents during the initial bootstrapping process. Securing agent communication also becomes simpler, with fewer tokens to track, distribute, and rotate.
+
+**Additional environment and runtime support**: Current Consul on Kubernetes deployments require using `hostPorts` and `DaemonSets` for client agents, which limits Consul’s ability to be deployed in environments where those features are not supported. As a result, Consul Dataplane supports AWS Fargate and GKE Autopilot.
+
+**Easier upgrades**: With Consul Dataplane, updating Consul to a new version no longer requires upgrading client agents. Consul Dataplane also has better compatibility across Consul server versions, so the process to upgrade Consul servers becomes easier.
+
+## Get started
+
+To get started with Consul Dataplane, use the following reference resources:
+
+- For `consul-dataplane` commands and usage examples, including required flags for startup, refer to the [`consul-dataplane` CLI reference](/consul/docs/dataplane/consul-dataplane).
+- For Helm chart information, refer to the [Helm Chart reference](/consul/docs/k8s/helm).
+- For Envoy, Consul, and Consul Dataplane version compatibility, refer to the [Envoy compatibility matrix](/docs/connect/proxies/envoy).
+
+### Installation
+
+To install the beta release of Consul Dataplane, set `VERSION` to `1.0.0-beta` and then follow the instructions to install a specific version of Consul [with the Helm Chart](/docs/k8s/installation/install#install-consul) or [with the Consul-k8s CLI](/docs/k8s/installation/install-cli#install-a-previous-version).
+
+#### Helm
+
+```shell-session
+$ export VERSION=1.0.0-beta
+$ helm install consul hashicorp/consul --set global.name=consul --version ${VERSION} --create-namespace --namespace consul
+```
+
+#### Consul-k8s CLI
+
+```shell-session
+$ export VERSION=1.0.0-beta && \
+    curl --location "https://releases.hashicorp.com/consul-k8s/${VERSION}/consul-k8s_${VERSION}_darwin_amd64.zip" --output consul-k8s-cli.zip
+```
+
+## Beta release features
+
+The beta release of Consul Dataplane supports the following features:
+
+- Single and multi-cluster installations, including those with WAN federation, cluster peering, and admin partitions are supported.
+- Ingress, terminating, and mesh gateways are supported.
+- Running Consul service mesh in AWS Fargate and GKE Autopilot is supported.
+- xDS load balancing is supported.
+- Servers running in Kubernetes and servers external to Kubernetes are both supported.
+
+Integration with HCP Consul is being tested in an invitation-only closed beta. HCP Consul support for Dataplane will be available for all users in a future release.
+
+### Technical Constraints
+
+Be aware of the following limitations and recommendations for Consul Dataplane:
+
+- Metrics and telemetry are not currently available for services deployed with Dataplane.
+- Consul API Gateway is not currently supported.
+- Transparent proxies are not supported.
+- Using `proxy-defaults` and `service-defaults` to configure default proxy behavior is not supported.
+- Consul Dataplane is not supported on Windows.

website/content/docs/k8s/installation/install-cli.mdx

@@ -11,6 +11,7 @@ description: >-
 
 This topic describes how to install Consul on Kubernetes using the Consul K8s CLI tool. The Consul K8s CLI tool enables you to quickly install and interact with Consul on Kubernetes. Use the Consul K8s CLI tool to install Consul on Kubernetes if you are deploying a single cluster. We recommend using the [Helm chart installation method](/docs/k8s/installation/install) if you are installing Consul on Kubernetes for multi-cluster deployments that involve cross-partition or cross datacenter communication.
 
+
 ## Introduction
 
 If it is your first time installing Consul on Kubernetes, then you must first install the Consul K8s CLI tool. You can install Consul on Kubernetes using the Consul K8s tool after installing the CLI.

website/content/docs/k8s/installation/install.mdx

@@ -9,6 +9,7 @@ description: >-
 
 This topic describes how to install Consul on Kubernetes using the official Consul Helm chart. For instruction on how to install Consul on Kubernetes using the Consul K8s CLI, refer to [Installing the Consul K8s CLI](/docs/k8s/installation/install-cli).
 
+
 ## Introduction
 
 We recommend using the Consul Helm chart to install Consul on Kubernetes for multi-cluster installations that involve cross-partition or cross datacenter communication. The Helm chart installs and configures all necessary components to run Consul. The configuration enables you to run a server cluster, a client cluster, or both.

website/data/docs-nav-data.json

@@ -561,6 +561,19 @@
         "title": "Security",
         "path": "connect/security",
         "hidden": true
+      },
+      {
+        "title": "Consul Dataplane",
+        "routes": [
+          {
+            "title": "Overview",
+            "path": "connect/dataplane"
+          },
+          {
+            "title": "CLI Reference",
+            "path": "connect/dataplane/consul-dataplane"
+          }
+        ]
       }
     ]
   },
@@ -995,6 +1008,19 @@
           }
         ]
       },
+      {
+        "title": "Consul Dataplane",
+        "routes": [
+          {
+            "title": "Overview",
+            "href": "/docs/connect/dataplane"
+          },
+          {
+            "title": "CLI Reference",
+            "href": "/docs/connect/dataplane/consul-dataplane"
+          }
+        ]
+      },
       {
         "title": "Operations",
         "routes": [