mirror of https://github.com/status-im/consul.git
new docs for consul and consul-k8s troubleshoot command (#16284)
* new docs for consul and consul-k8s troubleshoot command * add changelog * add troubleshoot command * address comments, and update cli output to match * revert changes to troubleshoot upstreams, changes will happen in separate pr * Update .changelog/16284.txt Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * address comments * update trouble proxy output * add missing s, add required fields in usage --------- Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
This commit is contained in:
malizz
GitHub
parent
085c0addc0
commit
c9c49ea3a2
|
|
@ -0,0 +1,3 @@
|
|||
```release-note:feature
|
||||
cli: adds new CLI commands `consul troubleshoot upstreams` and `consul troubleshoot proxy` to troubleshoot Consul's service mesh configuration and network issues.
|
||||
```
|
||||
|
|
@ -56,6 +56,7 @@ Available commands are:
|
|||
services Interact with services
|
||||
snapshot Saves, restores and inspects snapshots of Consul server state
|
||||
tls Builtin helpers for creating CAs and certificates
|
||||
troubleshoot Provides tools to troubleshoot Consul's service mesh configuration
|
||||
validate Validate config files/directories
|
||||
version Prints the Consul version
|
||||
watch Watch for changes in Consul
|
||||
|
|
|
|||
|
|
@ -0,0 +1,31 @@
|
|||
---
|
||||
layout: commands
|
||||
page_title: 'Commands: Troubleshoot'
|
||||
description: >-
|
||||
The `consul troubleshoot` command provides tools to troubleshoot Consul's service mesh configuration.
|
||||
---
|
||||
|
||||
# Consul Troubleshooting
|
||||
|
||||
Command: `consul troubleshoot`
|
||||
|
||||
Use the `troubleshoot` command to diagnose Consul service mesh configuration or network issues.
|
||||
|
||||
## Usage
|
||||
|
||||
```text
|
||||
Usage: consul troubleshoot <subcommand> [options]
|
||||
|
||||
# ...
|
||||
|
||||
Subcommands:
|
||||
|
||||
proxy Troubleshoots service mesh issues from the current Envoy instance
|
||||
upstreams Gets upstream Envoy identifiers and IPs configured for the proxy
|
||||
```
|
||||
|
||||
For more information, examples, and usage about a subcommand, click on the name
|
||||
of the subcommand in the sidebar or one of the links below:
|
||||
|
||||
- [proxy](/consul/commands/troubleshoot/proxy)
|
||||
- [upstreams](/consul/commands/troubleshoot/upstreams)
|
||||
|
|
@ -0,0 +1,68 @@
|
|||
---
|
||||
layout: commands
|
||||
page_title: 'Commands: Troubleshoot Proxy'
|
||||
description: >-
|
||||
The `consul troubleshoot proxy` diagnoses Consul service mesh configuration and network issues to an upstream.
|
||||
---
|
||||
|
||||
# Consul Troubleshoot Proxy
|
||||
|
||||
Command: `consul troubleshoot proxy`
|
||||
|
||||
The `troubleshoot proxy` command diagnoses Consul service mesh configuration and network issues to an upstream.
|
||||
|
||||
## Usage
|
||||
|
||||
Usage: `consul troubleshoot proxy (-upstream-ip <ip>|-upstream-envoy-id <envoy-id>) [options]`
|
||||
This command requires `-envoy-admin-endpoint` or `-upstream-ip` to specify the upstream.
|
||||
|
||||
#### Command Options
|
||||
|
||||
- `-envoy-admin-endpoint=<value>` - Envoy admin endpoint address for the local Envoy instance.
|
||||
Defaults to `127.0.0.1:19000`.
|
||||
|
||||
- `-upstream-ip=<value>` - The IP address of the upstream service; Use when the upstream is reached via a transparent proxy DNS address (KubeDNS or Consul DNS)
|
||||
|
||||
- `-upstream-envoy-id=<value>` - The Envoy identifier of the upstream service; Use when the upstream is explicitly configured on the downstream service.
|
||||
|
||||
## Examples
|
||||
|
||||
The following example illustrates how to troubleshoot Consul service mesh configuration and network issues to an upstream from a source proxy.
|
||||
|
||||
```shell-session
|
||||
$ consul troubleshoot proxy -upstream-ip 10.4.6.160
|
||||
|
||||
==> Validation
|
||||
✓ Certificates are valid
|
||||
✓ Envoy has 0 rejected configurations
|
||||
✓ Envoy has detected 0 connection failure(s)
|
||||
✓ Listener for upstream "backend" found
|
||||
✓ Route for upstream "backend" found
|
||||
✓ Cluster "backend.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "backend" found
|
||||
✓ Healthy endpoints for cluster "backend.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "backend" found
|
||||
✓ Cluster "backend2.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "backend" found
|
||||
! No healthy endpoints for cluster "backend2.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "backend" found
|
||||
-> Check that your upstream service is healthy and running
|
||||
-> Check that your upstream service is registered with Consul
|
||||
-> Check that the upstream proxy is healthy and running
|
||||
-> If you are explicitly configuring upstreams, ensure the name of the upstream is correct
|
||||
```
|
||||
|
||||
The following example troubleshoots Consul service mesh configuration and network issues from the local Envoy instance to the given upstream.
|
||||
|
||||
```shell-session
|
||||
$ consul-k8s troubleshoot proxy -upstream-envoy-id db
|
||||
|
||||
==> Validation
|
||||
✓ Certificates are valid
|
||||
✓ Envoy has 0 rejected configurations
|
||||
✓ Envoy has detected 0 connection failure(s)
|
||||
✓ Listener for upstream "db" found
|
||||
✓ Route for upstream "db" found
|
||||
✓ Cluster "db.default.dc1.internal.e08fa6d6-e91e-dfe0.consul" for upstream "db" found
|
||||
✓ Healthy endpoints for cluster "db.default.dc1.internal.e08fa6d6-e91e-dfe0.consul" for upstream "db" found
|
||||
|
||||
If you are still experiencing issues, you can:
|
||||
-> Check intentions to ensure the upstream allows traffic from this source
|
||||
-> If using transparent proxy, ensure DNS resolution is to the same IP you have verified here
|
||||
```
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
---
|
||||
layout: commands
|
||||
page_title: 'Commands: Troubleshoot Upstreams'
|
||||
description: >-
|
||||
The `consul troubleshoot upstreams` lists the available upstreams in the Consul service mesh from the current service.
|
||||
---
|
||||
|
||||
# Consul Troubleshoot Upstreams
|
||||
|
||||
Command: `consul troubleshoot upstreams`
|
||||
|
||||
The `troubleshoot upstreams` lists the available upstreams in the Consul service mesh from the current service.
|
||||
|
||||
## Usage
|
||||
|
||||
Usage: `consul troubleshoot upstreams [options]`
|
||||
|
||||
#### Command Options
|
||||
|
||||
- `-envoy-admin-endpoint=<value>` - Envoy admin endpoint address for the local Envoy instance.
|
||||
Defaults to `127.0.0.1:19000`.
|
||||
|
||||
## Examples
|
||||
|
||||
Display all transparent proxy upstreams in Consul service mesh from the current Envoy instance.
|
||||
|
||||
```shell-session
|
||||
$ consul troubleshoot upstreams
|
||||
==> Upstreams (explicit upstreams only) (0)
|
||||
|
||||
==> Upstreams IPs (transparent proxy only) (1)
|
||||
[10.4.6.160 240.0.0.3] true map[backend.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul backend2.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul]
|
||||
|
||||
If you cannot find the upstream address or cluster for a transparent proxy upstream:
|
||||
- Check intentions: Tproxy upstreams are configured based on intentions. Make sure you have configured intentions to allow traffic to your upstream.
|
||||
- To check that the right cluster is being dialed, run a DNS lookup for the upstream you are dialing. For example, run `dig backend.svc.consul` to return the IP address for the `backend` service. If the address you get from that is missing from the upstream IPs, it means that your proxy may be misconfigured.
|
||||
```
|
||||
|
|
@ -33,6 +33,7 @@ You can use the following commands with `consul-k8s`.
|
|||
- [`proxy list`](#proxy-list): List all Pods running proxies managed by Consul.
|
||||
- [`proxy read`](#proxy-read): Inspect the Envoy configuration for a given Pod.
|
||||
- [`status`](#status): Check the status of a Consul installation on Kubernetes.
|
||||
- [`troubleshoot`](#troubleshoot): Troubleshoot Consul service mesh and networking issues from a given pod.
|
||||
- [`uninstall`](#uninstall): Uninstall Consul deployment.
|
||||
- [`upgrade`](#upgrade): Upgrade Consul on Kubernetes from an existing installation.
|
||||
- [`version`](#version): Print the version of the Consul on Kubernetes CLI.
|
||||
|
|
@ -490,6 +491,106 @@ $ consul-k8s status
|
|||
✓ Consul clients healthy (3/3)
|
||||
```
|
||||
|
||||
### `troubleshoot`
|
||||
|
||||
The `troubleshoot` command exposes two subcommands for troubleshooting Consul
|
||||
service mesh and network issues from a given pod.
|
||||
|
||||
- [`troubleshoot upstreams`](#troubleshoot-upstreams): List all Envoy upstreams in Consul service mesh from the given pod.
|
||||
- [`troubleshoot proxy`](#troubleshoot-proxy): Troubleshoot Consul service mesh configuration and network issues between the given pod and the given upstream.
|
||||
|
||||
### `troubleshoot upstreams`
|
||||
|
||||
```shell-session
|
||||
$ consul-k8s troubleshoot upstreams -pod <pod> <OPTIONS>
|
||||
```
|
||||
|
||||
| Flag | Description | Default |
|
||||
| ------------------------------------ | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
| <nobr>`-namespace`, `-n`</nobr> | `String` The Kubernetes namespace to list proxies in. | Current [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) namespace. |
|
||||
| <nobr>`-envoy-admin-endpoint`</nobr> | `String` Envoy sidecar address and port | `127.0.0.1:19000` |
|
||||
|
||||
#### Example Commands
|
||||
|
||||
The following example displays all transparent proxy upstreams in Consul service mesh from the given pod.
|
||||
|
||||
```shell-session
|
||||
$ consul-k8s troubleshoot upstreams -pod frontend-767ccfc8f9-6f6gx
|
||||
|
||||
==> Upstreams (explicit upstreams only) (0)
|
||||
|
||||
==> Upstreams IPs (transparent proxy only) (1)
|
||||
[10.4.6.160 240.0.0.3] true map[backend.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul backend2.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul]
|
||||
|
||||
If you cannot find the upstream address or cluster for a transparent proxy upstream:
|
||||
- Check intentions: Tproxy upstreams are configured based on intentions. Make sure you have configured intentions to allow traffic to your upstream.
|
||||
- To check that the right cluster is being dialed, run a DNS lookup for the upstream you are dialing. For example, run `dig backend.svc.consul` to return the IP address for the `backend` service. If the address you get from that is missing from the upstream IPs, it means that your proxy may be misconfigured.
|
||||
```
|
||||
|
||||
The following example displays all explicit upstreams from the given pod in the Consul service mesh.
|
||||
|
||||
```shell-session
|
||||
$ consul-k8s troubleshoot upstreams -pod client-767ccfc8f9-6f6gx
|
||||
|
||||
==> Upstreams (explicit upstreams only) (1)
|
||||
server
|
||||
counting
|
||||
|
||||
==> Upstreams IPs (transparent proxy only) (0)
|
||||
|
||||
If you cannot find the upstream address or cluster for a transparent proxy upstream:
|
||||
- Check intentions: Tproxy upstreams are configured based on intentions. Make sure you have configured intentions to allow traffic to your upstream.
|
||||
- To check that the right cluster is being dialed, run a DNS lookup for the upstream you are dialing. For example, run `dig backend.svc.consul` to return the IP address for the `backend` service. If the address you get from that is missing from the upstream IPs, it means that your proxy may be misconfigured.
|
||||
```
|
||||
|
||||
### `troubleshoot proxy`
|
||||
|
||||
```shell-session
|
||||
$ consul-k8s troubleshoot proxy -pod <pod> -upstream-ip <ip> <OPTIONS>
|
||||
$ consul-k8s troubleshoot proxy -pod <pod> -upstream-envoy-id <envoy-id> <OPTIONS>
|
||||
```
|
||||
|
||||
| Flag | Description | Default |
|
||||
| ------------------------------------ | ----------------------------------------------------------| ---------------------------------------------------------------------------------------------------------------------- |
|
||||
| <nobr>`-namespace`, `-n`</nobr> | `String` The Kubernetes namespace to list proxies in. | Current [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) namespace. |
|
||||
| <nobr>`-envoy-admin-endpoint`</nobr> | `String` Envoy sidecar address and port | `127.0.0.1:19000` |
|
||||
| <nobr>`-upstream-ip`</nobr> | `String` The IP address of the upstream transparent proxy | |
|
||||
| <nobr>`-upstream-envoy-id`</nobr> | `String` The Envoy identifier of the upstream | |
|
||||
|
||||
#### Example Commands
|
||||
|
||||
The following example troubleshoots the Consul service mesh configuration and network issues between the given pod and the given upstream IP.
|
||||
|
||||
```shell-session
|
||||
$ consul-k8s troubleshoot proxy -pod frontend-767ccfc8f9-6f6gx -upstream-ip 10.4.6.160
|
||||
|
||||
==> Validation
|
||||
✓ certificates are valid
|
||||
✓ Envoy has 0 rejected configurations
|
||||
✓ Envoy has detected 0 connection failure(s)
|
||||
✓ listener for upstream "backend" found
|
||||
✓ route for upstream "backend" found
|
||||
✓ cluster "backend.default.dc1.internal..consul" for upstream "backend" found
|
||||
✓ healthy endpoints for cluster "backend.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "backend" found
|
||||
✓ cluster "backend2.default.dc1.internal..consul" for upstream "backend" found
|
||||
! no healthy endpoints for cluster "backend2.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "backend" found
|
||||
```
|
||||
|
||||
The following example troubleshoots the Consul service mesh configuration and network issues between the given pod and the given upstream.
|
||||
|
||||
```shell-session
|
||||
$ consul-k8s troubleshoot proxy -pod frontend-767ccfc8f9-6f6gx -upstream-envoy-id db
|
||||
|
||||
==> Validation
|
||||
✓ certificates are valid
|
||||
✓ Envoy has 0 rejected configurations
|
||||
✓ Envoy has detected 0 connection failure(s)
|
||||
! no listener for upstream "db" found
|
||||
! no route for upstream "backend" found
|
||||
! no cluster "db.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "db" found
|
||||
! no healthy endpoints for cluster "db.default.dc1.internal.e08fa6d6-e91e-dfe0-f6e1-ba097a828e31.consul" for upstream "db" found
|
||||
```
|
||||
|
||||
### `uninstall`
|
||||
|
||||
The `uninstall` command removes Consul from Kubernetes.
|
||||
|
|
|
|||
|
|
@ -528,6 +528,23 @@
|
|||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "troubleshoot",
|
||||
"routes": [
|
||||
{
|
||||
"title": "Overview",
|
||||
"path": "troubleshoot"
|
||||
},
|
||||
{
|
||||
"title": "upstreams",
|
||||
"path": "troubleshoot/upstreams"
|
||||
},
|
||||
{
|
||||
"title": "proxy",
|
||||
"path": "troubleshoot/proxy"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "validate",
|
||||
"path": "validate"
|
||||
|
|
|
|||
Loading…
Reference in New Issue