status-im
/
consul
mirror of https://github.com/status-im/consul.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

Add doc sections for network segments

This commit is contained in:
Kyle Havlovitz 2017-08-31 11:19:08 -07:00
parent baa501e0c5
commit 00d03f1141
No known key found for this signature in database
GPG Key ID: 8A5E6B173056AD6C
8 changed files with 324 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
website/source/api/agent.html.md
View File

@ -44,6 +44,10 @@ The table below shows this endpoint's support for
members (which is the default). This is only eligible for agents running in
**server mode**. This is specified as part of the URL as a query parameter.
- `segment` `(string: "")` - (Enterprise-only) Specifies the segment to list members in. If left blank,
this will query for the default segment when connecting to a server and the agent's
own segment when connecting to a client (clients can only be part of one network segment).
### Sample Request
```text

59
website/source/api/operator/segment.html.md Normal file
View File

@ -0,0 +1,59 @@
---
layout: api
page_title: Network Segments - Operator - HTTP API
sidebar_current: api-operator-segment
description: |-
The /operator/segments endpoints expose the network segment information via
Consul's HTTP API.
---
# Network Areas - Operator HTTP API
The `/operator/segment` endpoints provide tools to manage network segments via
Consul's HTTP API.
~> **Enterprise Only!** This API endpoint and functionality only exists in
Consul Enterprise. This is not present in the open source version of Consul.
The network area functionality described here is available only in
[Consul Enterprise](https://www.hashicorp.com/products/consul/) version 0.9.3 and
later. Network segments are operator-defined sections of agents on the LAN, typically
isolated from other segments by network configuration.
Please see the [Network Segments Guide](/docs/guides/segments.html) for more details.
## List Network Segments
This endpoint lists all network areas.
| Method | Path | Produces |
| ------ | ---------------------------- | -------------------------- |
| `GET` | `/operator/segment/list` | `application/json` |
The table below shows this endpoint's support for
[blocking queries](/api/index.html#blocking-queries),
[consistency modes](/api/index.html#consistency-modes), and
[required ACLs](/api/index.html#acls).
| Blocking Queries | Consistency Modes | ACL Required |
| ---------------- | ----------------- | --------------- |
| `NO` | `none` | `operator:read` |
### Parameters
- `dc` `(string: "")` - Specifies the datacenter to query. This will default to
the datacenter of the agent being queried. This is specified as a URL query
parameter.
### Sample Request
```text
$ curl \
https://consul.rocks/v1/operator/segment/list
```
### Sample Response
```json
["","alpha","beta"]
```

21
website/source/docs/agent/options.html.md
View File

@ -462,6 +462,11 @@ will exit with an error at startup.
as a permanent intent and does not attempt to join the cluster again when starting. This flag
allows the previous state to be used to rejoin the cluster.
* <a name="_segment"></a><a href="#_segment">`-segment`</a> - (Enterprise-only) This flag is used to set
the name of the network segment the agent belongs to. An agent can only join and communicate with other agents
within its network segment. See the [Network Segments Guide](/docs/guides/segments.html) for more details.
By default, this is an empty string, which is the default network segment.
* <a name="_server"></a><a href="#_server">`-server`</a> - This flag is used to control if an
agent is in server or client mode. When provided,
an agent will act as a Consul server. Each Consul cluster must have at least one server and ideally
@ -1052,6 +1057,22 @@ Consul will not enable TLS for the HTTP API unless the `https` port has been ass
* <a name="retry_interval_wan"></a><a href="#retry_interval_wan">`retry_interval_wan`</a> Equivalent to the
[`-retry-interval-wan` command-line flag](#_retry_interval_wan).
* <a name="segment"></a><a href="#segment">`segment`</a> (Enterprise-only) Equivalent to the
[`-segment` command-line flag](#_segment).
* <a name="segments"></a><a href="#segments">`segments`</a> (Enterprise-only) This is a list of nested objects that allows setting
the bind/advertise information for network segments. This can only be set on servers.
* <a name="segment_name"></a><a href="#segment_name">`name`</a> - The name of the segment. Must be a string between
1 and 64 characters in length.
* <a name="segment_bind"></a><a href="#segment_bind">`bind`</a> - The bind address to use for the segment's gossip layer.
Defaults to the [`-bind`](#_bind) value if not provided.
* <a name="segment_port"></a><a href="#segment_port">`port`</a> - The port to use for the segment's gossip layer.
* <a name="segment_advertise"></a><a href="#segment_advertise">`advertise`</a> - The advertise address to use for the
segment's gossip layer. Defaults to the [`-advertise`](#_advertise) value if not provided.
* <a name="segment_rpc_listener"></a><a href="#segment_rpc_listener">`rpc_listener`</a> - If true, a separate RPC listener will
be started on this segment's [`-bind`](#_bind) address on the rpc port. Only valid if the segment's bind address differs from the
[`-bind`](#_bind) address. Defaults to false.
* <a name="server"></a><a href="#server">`server`</a> Equivalent to the
[`-server` command-line flag](#_server).

3
website/source/docs/commands/members.html.markdown.erb
View File

@ -31,6 +31,9 @@ Usage: `consul members [options]`
* `-detailed` - If provided, output shows more detailed information
about each node.
* `-segment` - (Enterprise-only) The segment to show members in. If not provided, members
in all segments visible to the agent will be listed.
* `-status` - If provided, output is filtered to only nodes matching
the regular expression for status

2
website/source/docs/guides/index.html.md
View File

@ -36,6 +36,8 @@ The following guides are available:
* [Leader Election](/docs/guides/leader-election.html) - The goal of this guide is to cover how to build client-side leader election using Consul.
* [Network Segments](/docs/guides/segments.html) - Configuring Consul to support partial LAN connectivity using Network Segments.
* [Outage Recovery](/docs/guides/outage.html) - This guide covers recovering a cluster that has become unavailable due to server failures.
* [Semaphore](/docs/guides/semaphore.html) - This guide covers using the KV store to implement a semaphore.

229
website/source/docs/guides/segments.html.markdown.erb Normal file
View File

@ -0,0 +1,229 @@
---
layout: "docs"
page_title: "Partial LAN Connectivity - Configuring Network Segments"
sidebar_current: "docs-guides-areas"
description: |-
Many advanced Consul users have the need to run clusters with segmented networks, meaning that
not all agents can be in a full mesh. This is usually the result of business policies enforced
via network rules or firewalls. Prior to Consul 0.9.3 this was only possible through federation,
which for some users is too heavyweight or expensive as it requires running multiple servers per
segment.
---
# Partial LAN Connectivity
## Configuring Network Segments
[//]: # ( ~> The network area functionality described here is available only in )
[//]: # ( [Consul Enterprise](https://www.hashicorp.com/products/consul/) version 0.8.0 and later. )
<%= enterprise_alert :consul %>
Many advanced Consul users have the need to run clusters with segmented networks, meaning that
not all agents can be in a full mesh. This is usually the result of business policies enforced
via network rules or firewalls. Prior to Consul 0.9.3 this was only possible through federation,
which for some users is too heavyweight or expensive as it requires running multiple servers per
segment.
By default, all Consul agents in one datacenter are part of a shared gossip pool over the LAN;
this means that the partial connectivity caused by segmented networks would cause health flapping
as nodes failed to communicate. In this guide we will cover the Network Segments feature, added
in [Consul Enterprise](https://www.hashicorp.com/products/consul/) version 0.9.3, which allows users
to configure Consul to support this kind of segmented network topology.
This guide will cover the basic configuration for setting up multiple segments, as well as
how to configure a prepared query to limit service discovery to the services in the local agent's
network segment.
## Network Segments
All Consul agents are part of the default network segment, `""`, unless a segment is specified in
their configuration. In a standard cluster setup all agents will normally be part of this default
segment and as a result, part of one shared LAN gossip pool. Network segments can be used to break
up the LAN gossip pool into multiple isolated smaller pools by specifying the configuration for segments
on the servers. Each desired segment must be given a name and port, as well as optionally a custom
bind and advertise address for that segment's gossip listener to bind to on the server.
A few things to note:
1. Servers will be a part of all segments they have been configured with. They are the common point
linking the different segments together.
2. Client agents can only be part of one segment at a given time, specified by the [`-segment`]
(/docs/agent/options.html#_segment) option.
3. Clients can only join agents in the same segment as them. If they attempt to join a client in
another segment, or the listening port of another segment on a server, they will get a segment mismatch error.
Once the servers have been configured with the correct segment info, the clients only need to specify
their own segment in the [Agent Config](/docs/agent/options.html) and join by connecting to another
agent within the same segment.
## Getting Started
To get started, follow the [bootstrapping guide](/docs/guides/bootstrapping.html) to
start a server or group of servers, with the following section added to the configuration (you may need to
adjust the bind/advertise addresses for your setup):
```json
{
...
"segments": [
{"name": "alpha", "bind": "{{GetPrivateIP}}", "advertise": "{{GetPrivateIP}}", "port": 8303},
{"name": "beta", "bind": "{{GetPrivateIP}}", "advertise": "{{GetPrivateIP}}", "port": 8304}
]
}
```
You should see a log message on the servers for each segment's listener as the agent starts up:
```text
2017/08/30 19:05:13 [INFO] serf: EventMemberJoin: server1.dc1 192.168.0.4
2017/08/30 19:05:13 [INFO] serf: EventMemberJoin: server1 192.168.0.4
2017/08/30 19:05:13 [INFO] consul: Started listener for LAN segment "alpha" on 192.168.0.4:8303
2017/08/30 19:05:13 [INFO] serf: EventMemberJoin: server1 192.168.0.4
2017/08/30 19:05:13 [INFO] consul: Started listener for LAN segment "beta" on 192.168.0.4:8304
2017/08/30 19:05:13 [INFO] serf: EventMemberJoin: server1 192.168.0.4
```
Running `consul members` should show the server as being part of all segments:
```text
(server1) $ consul members
Node Address Status Type Build Protocol DC Segment
server1 192.168.0.4:8301 alive server 0.9.3+ent 2 dc1 <all>
```
Next, start a client agent in the 'alpha' segment, with `-join` set to the server's segment
address/port for that segment:
```text
(client1) $ consul agent ... -join 192.168.0.4:8303 -node client1 -segment alpha
```
After the join is successful, we should see the client show up by running the `consul members` command
on the server again:
```text
(server1) $ consul members
Node Address Status Type Build Protocol DC Segment
server1 192.168.0.4:8301 alive server 0.9.3+ent 2 dc1 <all>
client1 192.168.0.5:8301 alive client 0.9.3+ent 2 dc1 alpha
```
Now join another client in segment 'beta' and run the `consul members` command another time:
```text
(client2) $ consul agent ... -join 192.168.0.4:8304 -node client2 -segment beta
```
```text
(server1) $ consul members
Node Address Status Type Build Protocol DC Segment
server1 192.168.0.4:8301 alive server 0.9.3+ent 2 dc1 <all>
client1 192.168.0.5:8301 alive client 0.9.3+ent 2 dc1 alpha
client2 192.168.0.6:8301 alive client 0.9.3+ent 2 dc1 beta
```
If we pass the `-segment` flag when running `consul members`, we can limit the view to agents
in a specific segment:
```text
(server1) $ consul members -segment alpha
Node Address Status Type Build Protocol DC Segment
client1 192.168.0.5:8301 alive client 0.9.3+ent 2 dc1 alpha
server1 192.168.0.4:8303 alive server 0.9.3+ent 2 dc1 alpha
```
Using the `consul catalog nodes` command, we can filter on an internal metadata key,
`consul-network-segment`, which stores the network segment of the node:
```text
(server1) $ consul catalog nodes -node-meta consul-network-segment=alpha
Node ID Address DC
client1 4c29819c 192.168.0.5 dc1
```
With this metadata key, we can construct a prepared query that can be used for DNS to return
only services within the same network segment as the local agent.
First, register a service on each of the client nodes:
```text
(client1) $ curl \
--request PUT \
--data '{"Name": "redis", "Port": 8000}' \
localhost:8500/v1/agent/service/register
```
```text
(client2) $ curl \
--request PUT \
--data '{"Name": "redis", "Port": 9000}' \
localhost:8500/v1/agent/service/register
```
Next, write the following to `query.json` and create the query using the HTTP endpoint:
```text
(server1) $ curl \
--request POST \
--data \
'{
"Name": "",
"Template": {
"Type": "name_prefix_match"
},
"Service": {
"Service": "${name.full}",
"NodeMeta": {"consul-network-segment": "${agent.segment}"}
}
}' localhost:8500/v1/query
{"ID":"6f49dd24-de9b-0b6c-fd29-525eca069419"}
```
Now, we can replace any dns lookups of the form `<service>.service.consul` with
`<service>.query.consul` to look up only services within the same network segment:
**Client 1:**
```text
(client1) $ dig @127.0.0.1 -p 8600 redis.query.consul SRV
; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p 8600 redis.query.consul SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3149
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;redis.query.consul. IN SRV
;; ANSWER SECTION:
redis.query.consul. 0 IN SRV 1 1 8000 client1.node.dc1.consul.
;; ADDITIONAL SECTION:
client1.node.dc1.consul. 0 IN A 192.168.0.5
```
**Client 2:**
```text
(client2) $ dig @127.0.0.1 -p 8600 redis.query.consul SRV
; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p 8600 redis.query.consul SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3149
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;redis.query.consul. IN SRV
;; ANSWER SECTION:
redis.query.consul. 0 IN SRV 1 1 9000 client2.node.dc1.consul.
;; ADDITIONAL SECTION:
client2.node.dc1.consul. 0 IN A 192.168.0.6
```

3
website/source/layouts/api.erb
View File

@ -54,6 +54,9 @@
<li<%= sidebar_current("api-operator-raft") %>>
<a href="/api/operator/raft.html">Raft</a>
</li>
<li<%= sidebar_current("api-operator-segment") %>>
<a href="/api/operator/segment.html">Segment</a>
</li>
</ul>
</li>
<li<%= sidebar_current("api-query") %>>

3
website/source/layouts/docs.erb
View File

@ -251,6 +251,9 @@
<li<%= sidebar_current("docs-guides-leader") %>>
<a href="/docs/guides/leader-election.html">Leader Election</a>
</li>
<li<%= sidebar_current("docs-guides-segments") %>>
<a href="/docs/guides/segments.html">Network Segments</a>
</li>
<li<%= sidebar_current("docs-guides-outage") %>>
<a href="/docs/guides/outage.html">Outage Recovery</a>
</li>