Add doc sections for network segments

website/source/api/agent.html.md

@@ -44,6 +44,10 @@ The table below shows this endpoint's support for
   members (which is the default). This is only eligible for agents running in
   **server mode**. This is specified as part of the URL as a query parameter.
 
+- `segment` `(string: "")` - (Enterprise-only) Specifies the segment to list members in. If left blank,
+this will query for the default segment when connecting to a server and the agent's
+own segment when connecting to a client (clients can only be part of one network segment).
+
 ### Sample Request
 
 ```text

website/source/api/operator/segment.html.md

@@ -0,0 +1,59 @@
+---
+layout: api
+page_title: Network Segments - Operator - HTTP API
+sidebar_current: api-operator-segment
+description: |-
+  The /operator/segments endpoints expose the network segment information via
+  Consul's HTTP API.
+---
+
+# Network Areas - Operator HTTP API
+
+The `/operator/segment` endpoints provide tools to manage network segments via
+Consul's HTTP API.
+
+~> **Enterprise Only!** This API endpoint and functionality only exists in
+Consul Enterprise. This is not present in the open source version of Consul.
+
+The network area functionality described here is available only in
+[Consul Enterprise](https://www.hashicorp.com/products/consul/) version 0.9.3 and
+later. Network segments are operator-defined sections of agents on the LAN, typically
+isolated from other segments by network configuration.
+
+Please see the [Network Segments Guide](/docs/guides/segments.html) for more details.
+
+## List Network Segments
+
+This endpoint lists all network areas.
+
+| Method | Path                         | Produces                   |
+| ------ | ---------------------------- | -------------------------- |
+| `GET`  | `/operator/segment/list`     | `application/json`         |
+
+The table below shows this endpoint's support for
+[blocking queries](/api/index.html#blocking-queries),
+[consistency modes](/api/index.html#consistency-modes), and
+[required ACLs](/api/index.html#acls).
+
+| Blocking Queries | Consistency Modes | ACL Required    |
+| ---------------- | ----------------- | --------------- |
+| `NO`             | `none`            | `operator:read` |
+
+### Parameters
+
+- `dc` `(string: "")` - Specifies the datacenter to query. This will default to
+  the datacenter of the agent being queried. This is specified as a URL query
+  parameter.
+
+### Sample Request
+
+```text
+$ curl \
+    https://consul.rocks/v1/operator/segment/list
+```
+
+### Sample Response
+
+```json
+["","alpha","beta"]
+```

website/source/docs/agent/options.html.md

@@ -462,6 +462,11 @@ will exit with an error at startup.
   as a permanent intent and does not attempt to join the cluster again when starting. This flag
   allows the previous state to be used to rejoin the cluster.
 
+* <a name="_segment"></a><a href="#_segment">`-segment`</a> - (Enterprise-only) This flag is used to set
+  the name of the network segment the agent belongs to. An agent can only join and communicate with other agents
+  within its network segment. See the [Network Segments Guide](/docs/guides/segments.html) for more details.
+  By default, this is an empty string, which is the default network segment.
+
 * <a name="_server"></a><a href="#_server">`-server`</a> - This flag is used to control if an
   agent is in server or client mode. When provided,
   an agent will act as a Consul server. Each Consul cluster must have at least one server and ideally
@@ -1052,6 +1057,22 @@ Consul will not enable TLS for the HTTP API unless the `https` port has been ass
 * <a name="retry_interval_wan"></a><a href="#retry_interval_wan">`retry_interval_wan`</a> Equivalent to the
   [`-retry-interval-wan` command-line flag](#_retry_interval_wan).
 
+* <a name="segment"></a><a href="#segment">`segment`</a> (Enterprise-only) Equivalent to the
+  [`-segment` command-line flag](#_segment).
+
+* <a name="segments"></a><a href="#segments">`segments`</a> (Enterprise-only) This is a list of nested objects that allows setting
+  the bind/advertise information for network segments. This can only be set on servers.
+    * <a name="segment_name"></a><a href="#segment_name">`name`</a> - The name of the segment. Must be a string between
+    1 and 64 characters in length.
+    * <a name="segment_bind"></a><a href="#segment_bind">`bind`</a> - The bind address to use for the segment's gossip layer.
+    Defaults to the [`-bind`](#_bind) value if not provided.
+    * <a name="segment_port"></a><a href="#segment_port">`port`</a> - The port to use for the segment's gossip layer.
+    * <a name="segment_advertise"></a><a href="#segment_advertise">`advertise`</a> - The advertise address to use for the
+    segment's gossip layer. Defaults to the [`-advertise`](#_advertise) value if not provided.
+    * <a name="segment_rpc_listener"></a><a href="#segment_rpc_listener">`rpc_listener`</a> - If true, a separate RPC listener will
+    be started on this segment's [`-bind`](#_bind) address on the rpc port. Only valid if the segment's bind address differs from the
+    [`-bind`](#_bind) address. Defaults to false.
+
 * <a name="server"></a><a href="#server">`server`</a> Equivalent to the
   [`-server` command-line flag](#_server).
 

website/source/docs/commands/members.html.markdown.erb

@@ -31,6 +31,9 @@ Usage: `consul members [options]`
 * `-detailed` - If provided, output shows more detailed information
   about each node.
 
+* `-segment` - (Enterprise-only) The segment to show members in. If not provided, members
+  in all segments visible to the agent will be listed.
+
 * `-status` - If provided, output is filtered to only nodes matching
   the regular expression for status
 

website/source/docs/guides/index.html.md

@@ -36,6 +36,8 @@ The following guides are available:
 
 * [Leader Election](/docs/guides/leader-election.html) - The goal of this guide is to cover how to build client-side leader election using Consul.
 
+* [Network Segments](/docs/guides/segments.html) - Configuring Consul to support partial LAN connectivity using Network Segments.
+
 * [Outage Recovery](/docs/guides/outage.html) - This guide covers recovering a cluster that has become unavailable due to server failures.
 
 * [Semaphore](/docs/guides/semaphore.html) - This guide covers using the KV store to implement a semaphore.

website/source/docs/guides/segments.html.markdown.erb

@@ -0,0 +1,229 @@
+---
+layout: "docs"
+page_title: "Partial LAN Connectivity - Configuring Network Segments"
+sidebar_current: "docs-guides-areas"
+description: |-
+  Many advanced Consul users have the need to run clusters with segmented networks, meaning that
+  not all agents can be in a full mesh. This is usually the result of business policies enforced
+  via network rules or firewalls. Prior to Consul 0.9.3 this was only possible through federation,
+  which for some users is too heavyweight or expensive as it requires running multiple servers per
+  segment.
+---
+
+# Partial LAN Connectivity
+## Configuring Network Segments
+
+[//]: # ( ~> The network area functionality described here is available only in )
+[//]: # (   [Consul Enterprise](https://www.hashicorp.com/products/consul/) version 0.8.0 and later. )
+
+<%= enterprise_alert :consul %>
+
+Many advanced Consul users have the need to run clusters with segmented networks, meaning that
+not all agents can be in a full mesh. This is usually the result of business policies enforced
+via network rules or firewalls. Prior to Consul 0.9.3 this was only possible through federation,
+which for some users is too heavyweight or expensive as it requires running multiple servers per
+segment.
+
+By default, all Consul agents in one datacenter are part of a shared gossip pool over the LAN;
+this means that the partial connectivity caused by segmented networks would cause health flapping
+as nodes failed to communicate. In this guide we will cover the Network Segments feature, added
+in [Consul Enterprise](https://www.hashicorp.com/products/consul/) version 0.9.3, which allows users
+to configure Consul to support this kind of segmented network topology.
+
+This guide will cover the basic configuration for setting up multiple segments, as well as
+how to configure a prepared query to limit service discovery to the services in the local agent's
+network segment.
+
+## Network Segments
+
+All Consul agents are part of the default network segment, `""`, unless a segment is specified in
+their configuration. In a standard cluster setup all agents will normally be part of this default
+segment and as a result, part of one shared LAN gossip pool. Network segments can be used to break
+up the LAN gossip pool into multiple isolated smaller pools by specifying the configuration for segments
+on the servers. Each desired segment must be given a name and port, as well as optionally a custom
+bind and advertise address for that segment's gossip listener to bind to on the server.
+
+A few things to note:
+1. Servers will be a part of all segments they have been configured with. They are the common point
+linking the different segments together.
+2. Client agents can only be part of one segment at a given time, specified by the [`-segment`]
+(/docs/agent/options.html#_segment) option.
+3. Clients can only join agents in the same segment as them. If they attempt to join a client in
+another segment, or the listening port of another segment on a server, they will get a segment mismatch error.
+
+Once the servers have been configured with the correct segment info, the clients only need to specify
+their own segment in the [Agent Config](/docs/agent/options.html) and join by connecting to another
+agent within the same segment.
+
+## Getting Started
+
+To get started, follow the [bootstrapping guide](/docs/guides/bootstrapping.html) to
+start a server or group of servers, with the following section added to the configuration (you may need to
+adjust the bind/advertise addresses for your setup):
+
+```json
+{
+...
+  "segments": [
+    {"name": "alpha", "bind": "{{GetPrivateIP}}", "advertise": "{{GetPrivateIP}}", "port": 8303},
+    {"name": "beta", "bind": "{{GetPrivateIP}}", "advertise": "{{GetPrivateIP}}", "port": 8304}
+  ]
+}
+```
+
+You should see a log message on the servers for each segment's listener as the agent starts up:
+
+```text
+2017/08/30 19:05:13 [INFO] serf: EventMemberJoin: server1.dc1 192.168.0.4
+2017/08/30 19:05:13 [INFO] serf: EventMemberJoin: server1 192.168.0.4
+2017/08/30 19:05:13 [INFO] consul: Started listener for LAN segment "alpha" on 192.168.0.4:8303
+2017/08/30 19:05:13 [INFO] serf: EventMemberJoin: server1 192.168.0.4
+2017/08/30 19:05:13 [INFO] consul: Started listener for LAN segment "beta" on 192.168.0.4:8304
+2017/08/30 19:05:13 [INFO] serf: EventMemberJoin: server1 192.168.0.4
+```
+
+Running `consul members` should show the server as being part of all segments:
+
+```text
+(server1) $ consul members
+Node     Address           Status  Type    Build      Protocol  DC   Segment
+server1  192.168.0.4:8301  alive   server  0.9.3+ent  2         dc1  <all>
+```
+
+Next, start a client agent in the 'alpha' segment, with `-join` set to the server's segment
+address/port for that segment:
+
+```text
+(client1) $ consul agent ... -join 192.168.0.4:8303 -node client1 -segment alpha
+```
+
+After the join is successful, we should see the client show up by running the `consul members` command
+on the server again:
+
+```text
+(server1) $ consul members
+Node     Address           Status  Type    Build      Protocol  DC   Segment
+server1  192.168.0.4:8301  alive   server  0.9.3+ent  2         dc1  <all>
+client1  192.168.0.5:8301  alive   client  0.9.3+ent  2         dc1  alpha
+```
+
+Now join another client in segment 'beta' and run the `consul members` command another time:
+
+```text
+(client2) $ consul agent ... -join 192.168.0.4:8304 -node client2 -segment beta
+```
+
+```text
+(server1) $ consul members
+Node     Address           Status  Type    Build      Protocol  DC   Segment
+server1  192.168.0.4:8301  alive   server  0.9.3+ent  2         dc1  <all>
+client1  192.168.0.5:8301  alive   client  0.9.3+ent  2         dc1  alpha
+client2  192.168.0.6:8301  alive   client  0.9.3+ent  2         dc1  beta
+```
+
+If we pass the `-segment` flag when running `consul members`, we can limit the view to agents
+in a specific segment:
+
+```text
+(server1) $ consul members -segment alpha
+Node     Address           Status  Type    Build      Protocol  DC   Segment
+client1  192.168.0.5:8301  alive   client  0.9.3+ent  2         dc1  alpha
+server1  192.168.0.4:8303  alive   server  0.9.3+ent  2         dc1  alpha
+```
+
+Using the `consul catalog nodes` command, we can filter on an internal metadata key,
+`consul-network-segment`, which stores the network segment of the node:
+
+```text
+(server1) $ consul catalog nodes -node-meta consul-network-segment=alpha
+Node     ID        Address      DC
+client1  4c29819c  192.168.0.5  dc1
+```
+
+With this metadata key, we can construct a prepared query that can be used for DNS to return
+only services within the same network segment as the local agent.
+
+First, register a service on each of the client nodes:
+
+```text
+(client1) $ curl \
+    --request PUT \
+    --data '{"Name": "redis", "Port": 8000}' \
+    localhost:8500/v1/agent/service/register
+```
+
+```text
+(client2) $ curl \
+    --request PUT \
+    --data '{"Name": "redis", "Port": 9000}' \
+    localhost:8500/v1/agent/service/register
+```
+
+Next, write the following to `query.json` and create the query using the HTTP endpoint:
+
+```text
+(server1) $ curl \
+    --request POST \
+    --data \
+'{
+   "Name": "",
+   "Template": {
+     "Type": "name_prefix_match"
+   },
+   "Service": {
+     "Service": "${name.full}",
+     "NodeMeta": {"consul-network-segment": "${agent.segment}"}
+   }
+}' localhost:8500/v1/query
+
+{"ID":"6f49dd24-de9b-0b6c-fd29-525eca069419"}
+```
+
+Now, we can replace any dns lookups of the form `<service>.service.consul` with
+`<service>.query.consul` to look up only services within the same network segment:
+
+**Client 1:**
+
+```text
+(client1) $ dig @127.0.0.1 -p 8600 redis.query.consul SRV
+
+; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p 8600 redis.query.consul SRV
+; (1 server found)
+;; global options: +cmd
+;; Got answer:
+;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3149
+;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
+;; WARNING: recursion requested but not available
+
+;; QUESTION SECTION:
+;redis.query.consul.		IN	SRV
+
+;; ANSWER SECTION:
+redis.query.consul.	0	IN	SRV	1 1 8000 client1.node.dc1.consul.
+
+;; ADDITIONAL SECTION:
+client1.node.dc1.consul. 0	IN	A	192.168.0.5
+```
+
+**Client 2:**
+
+```text
+(client2) $ dig @127.0.0.1 -p 8600 redis.query.consul SRV
+
+; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p 8600 redis.query.consul SRV
+; (1 server found)
+;; global options: +cmd
+;; Got answer:
+;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3149
+;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
+;; WARNING: recursion requested but not available
+
+;; QUESTION SECTION:
+;redis.query.consul.		IN	SRV
+
+;; ANSWER SECTION:
+redis.query.consul.	0	IN	SRV	1 1 9000 client2.node.dc1.consul.
+
+;; ADDITIONAL SECTION:
+client2.node.dc1.consul. 0	IN	A	192.168.0.6
+```

website/source/layouts/api.erb

@@ -54,6 +54,9 @@
           <li<%= sidebar_current("api-operator-raft") %>>
             <a href="/api/operator/raft.html">Raft</a>
           </li>
+          <li<%= sidebar_current("api-operator-segment") %>>
+            <a href="/api/operator/segment.html">Segment</a>
+          </li>
         </ul>
       </li>
       <li<%= sidebar_current("api-query") %>>

website/source/layouts/docs.erb

@@ -251,6 +251,9 @@
           <li<%= sidebar_current("docs-guides-leader") %>>
             <a href="/docs/guides/leader-election.html">Leader Election</a>
           </li>
+          <li<%= sidebar_current("docs-guides-segments") %>>
+            <a href="/docs/guides/segments.html">Network Segments</a>
+          </li>
           <li<%= sidebar_current("docs-guides-outage") %>>
             <a href="/docs/guides/outage.html">Outage Recovery</a>
           </li>