diff --git a/website/source/api/agent.html.md b/website/source/api/agent.html.md index ea09e231ca..7cf0ad3aae 100644 --- a/website/source/api/agent.html.md +++ b/website/source/api/agent.html.md @@ -44,6 +44,10 @@ The table below shows this endpoint's support for members (which is the default). This is only eligible for agents running in **server mode**. This is specified as part of the URL as a query parameter. +- `segment` `(string: "")` - (Enterprise-only) Specifies the segment to list members in. If left blank, +this will query for the default segment when connecting to a server and the agent's +own segment when connecting to a client (clients can only be part of one network segment). + ### Sample Request ```text diff --git a/website/source/api/operator/segment.html.md b/website/source/api/operator/segment.html.md new file mode 100644 index 0000000000..f27c9a3d56 --- /dev/null +++ b/website/source/api/operator/segment.html.md @@ -0,0 +1,59 @@ +--- +layout: api +page_title: Network Segments - Operator - HTTP API +sidebar_current: api-operator-segment +description: |- + The /operator/segments endpoints expose the network segment information via + Consul's HTTP API. +--- + +# Network Areas - Operator HTTP API + +The `/operator/segment` endpoints provide tools to manage network segments via +Consul's HTTP API. + +~> **Enterprise Only!** This API endpoint and functionality only exists in +Consul Enterprise. This is not present in the open source version of Consul. + +The network area functionality described here is available only in +[Consul Enterprise](https://www.hashicorp.com/products/consul/) version 0.9.3 and +later. Network segments are operator-defined sections of agents on the LAN, typically +isolated from other segments by network configuration. + +Please see the [Network Segments Guide](/docs/guides/segments.html) for more details. + +## List Network Segments + +This endpoint lists all network areas. + +| Method | Path | Produces | +| ------ | ---------------------------- | -------------------------- | +| `GET` | `/operator/segment/list` | `application/json` | + +The table below shows this endpoint's support for +[blocking queries](/api/index.html#blocking-queries), +[consistency modes](/api/index.html#consistency-modes), and +[required ACLs](/api/index.html#acls). + +| Blocking Queries | Consistency Modes | ACL Required | +| ---------------- | ----------------- | --------------- | +| `NO` | `none` | `operator:read` | + +### Parameters + +- `dc` `(string: "")` - Specifies the datacenter to query. This will default to + the datacenter of the agent being queried. This is specified as a URL query + parameter. + +### Sample Request + +```text +$ curl \ + https://consul.rocks/v1/operator/segment/list +``` + +### Sample Response + +```json +["","alpha","beta"] +``` \ No newline at end of file diff --git a/website/source/docs/agent/options.html.md b/website/source/docs/agent/options.html.md index 22d55ca702..3a156bc10e 100644 --- a/website/source/docs/agent/options.html.md +++ b/website/source/docs/agent/options.html.md @@ -462,6 +462,11 @@ will exit with an error at startup. as a permanent intent and does not attempt to join the cluster again when starting. This flag allows the previous state to be used to rejoin the cluster. +* `-segment` - (Enterprise-only) This flag is used to set + the name of the network segment the agent belongs to. An agent can only join and communicate with other agents + within its network segment. See the [Network Segments Guide](/docs/guides/segments.html) for more details. + By default, this is an empty string, which is the default network segment. + * `-server` - This flag is used to control if an agent is in server or client mode. When provided, an agent will act as a Consul server. Each Consul cluster must have at least one server and ideally @@ -1052,6 +1057,22 @@ Consul will not enable TLS for the HTTP API unless the `https` port has been ass * `retry_interval_wan` Equivalent to the [`-retry-interval-wan` command-line flag](#_retry_interval_wan). +* `segment` (Enterprise-only) Equivalent to the + [`-segment` command-line flag](#_segment). + +* `segments` (Enterprise-only) This is a list of nested objects that allows setting + the bind/advertise information for network segments. This can only be set on servers. + * `name` - The name of the segment. Must be a string between + 1 and 64 characters in length. + * `bind` - The bind address to use for the segment's gossip layer. + Defaults to the [`-bind`](#_bind) value if not provided. + * `port` - The port to use for the segment's gossip layer. + * `advertise` - The advertise address to use for the + segment's gossip layer. Defaults to the [`-advertise`](#_advertise) value if not provided. + * `rpc_listener` - If true, a separate RPC listener will + be started on this segment's [`-bind`](#_bind) address on the rpc port. Only valid if the segment's bind address differs from the + [`-bind`](#_bind) address. Defaults to false. + * `server` Equivalent to the [`-server` command-line flag](#_server). diff --git a/website/source/docs/commands/members.html.markdown.erb b/website/source/docs/commands/members.html.markdown.erb index 8eb3fc3570..41c6e2412f 100644 --- a/website/source/docs/commands/members.html.markdown.erb +++ b/website/source/docs/commands/members.html.markdown.erb @@ -31,6 +31,9 @@ Usage: `consul members [options]` * `-detailed` - If provided, output shows more detailed information about each node. +* `-segment` - (Enterprise-only) The segment to show members in. If not provided, members + in all segments visible to the agent will be listed. + * `-status` - If provided, output is filtered to only nodes matching the regular expression for status diff --git a/website/source/docs/guides/index.html.md b/website/source/docs/guides/index.html.md index cc66cb3be5..2c88a20424 100644 --- a/website/source/docs/guides/index.html.md +++ b/website/source/docs/guides/index.html.md @@ -36,6 +36,8 @@ The following guides are available: * [Leader Election](/docs/guides/leader-election.html) - The goal of this guide is to cover how to build client-side leader election using Consul. +* [Network Segments](/docs/guides/segments.html) - Configuring Consul to support partial LAN connectivity using Network Segments. + * [Outage Recovery](/docs/guides/outage.html) - This guide covers recovering a cluster that has become unavailable due to server failures. * [Semaphore](/docs/guides/semaphore.html) - This guide covers using the KV store to implement a semaphore. diff --git a/website/source/docs/guides/segments.html.markdown.erb b/website/source/docs/guides/segments.html.markdown.erb new file mode 100644 index 0000000000..ef7fd18813 --- /dev/null +++ b/website/source/docs/guides/segments.html.markdown.erb @@ -0,0 +1,229 @@ +--- +layout: "docs" +page_title: "Partial LAN Connectivity - Configuring Network Segments" +sidebar_current: "docs-guides-areas" +description: |- + Many advanced Consul users have the need to run clusters with segmented networks, meaning that + not all agents can be in a full mesh. This is usually the result of business policies enforced + via network rules or firewalls. Prior to Consul 0.9.3 this was only possible through federation, + which for some users is too heavyweight or expensive as it requires running multiple servers per + segment. +--- + +# Partial LAN Connectivity +## Configuring Network Segments + +[//]: # ( ~> The network area functionality described here is available only in ) +[//]: # ( [Consul Enterprise](https://www.hashicorp.com/products/consul/) version 0.8.0 and later. ) + +<%= enterprise_alert :consul %> + +Many advanced Consul users have the need to run clusters with segmented networks, meaning that +not all agents can be in a full mesh. This is usually the result of business policies enforced +via network rules or firewalls. Prior to Consul 0.9.3 this was only possible through federation, +which for some users is too heavyweight or expensive as it requires running multiple servers per +segment. + +By default, all Consul agents in one datacenter are part of a shared gossip pool over the LAN; +this means that the partial connectivity caused by segmented networks would cause health flapping +as nodes failed to communicate. In this guide we will cover the Network Segments feature, added +in [Consul Enterprise](https://www.hashicorp.com/products/consul/) version 0.9.3, which allows users +to configure Consul to support this kind of segmented network topology. + +This guide will cover the basic configuration for setting up multiple segments, as well as +how to configure a prepared query to limit service discovery to the services in the local agent's +network segment. + +## Network Segments + +All Consul agents are part of the default network segment, `""`, unless a segment is specified in +their configuration. In a standard cluster setup all agents will normally be part of this default +segment and as a result, part of one shared LAN gossip pool. Network segments can be used to break +up the LAN gossip pool into multiple isolated smaller pools by specifying the configuration for segments +on the servers. Each desired segment must be given a name and port, as well as optionally a custom +bind and advertise address for that segment's gossip listener to bind to on the server. + +A few things to note: +1. Servers will be a part of all segments they have been configured with. They are the common point +linking the different segments together. +2. Client agents can only be part of one segment at a given time, specified by the [`-segment`] +(/docs/agent/options.html#_segment) option. +3. Clients can only join agents in the same segment as them. If they attempt to join a client in +another segment, or the listening port of another segment on a server, they will get a segment mismatch error. + +Once the servers have been configured with the correct segment info, the clients only need to specify +their own segment in the [Agent Config](/docs/agent/options.html) and join by connecting to another +agent within the same segment. + +## Getting Started + +To get started, follow the [bootstrapping guide](/docs/guides/bootstrapping.html) to +start a server or group of servers, with the following section added to the configuration (you may need to +adjust the bind/advertise addresses for your setup): + +```json +{ +... + "segments": [ + {"name": "alpha", "bind": "{{GetPrivateIP}}", "advertise": "{{GetPrivateIP}}", "port": 8303}, + {"name": "beta", "bind": "{{GetPrivateIP}}", "advertise": "{{GetPrivateIP}}", "port": 8304} + ] +} +``` + +You should see a log message on the servers for each segment's listener as the agent starts up: + +```text +2017/08/30 19:05:13 [INFO] serf: EventMemberJoin: server1.dc1 192.168.0.4 +2017/08/30 19:05:13 [INFO] serf: EventMemberJoin: server1 192.168.0.4 +2017/08/30 19:05:13 [INFO] consul: Started listener for LAN segment "alpha" on 192.168.0.4:8303 +2017/08/30 19:05:13 [INFO] serf: EventMemberJoin: server1 192.168.0.4 +2017/08/30 19:05:13 [INFO] consul: Started listener for LAN segment "beta" on 192.168.0.4:8304 +2017/08/30 19:05:13 [INFO] serf: EventMemberJoin: server1 192.168.0.4 +``` + +Running `consul members` should show the server as being part of all segments: + +```text +(server1) $ consul members +Node Address Status Type Build Protocol DC Segment +server1 192.168.0.4:8301 alive server 0.9.3+ent 2 dc1 +``` + +Next, start a client agent in the 'alpha' segment, with `-join` set to the server's segment +address/port for that segment: + +```text +(client1) $ consul agent ... -join 192.168.0.4:8303 -node client1 -segment alpha +``` + +After the join is successful, we should see the client show up by running the `consul members` command +on the server again: + +```text +(server1) $ consul members +Node Address Status Type Build Protocol DC Segment +server1 192.168.0.4:8301 alive server 0.9.3+ent 2 dc1 +client1 192.168.0.5:8301 alive client 0.9.3+ent 2 dc1 alpha +``` + +Now join another client in segment 'beta' and run the `consul members` command another time: + +```text +(client2) $ consul agent ... -join 192.168.0.4:8304 -node client2 -segment beta +``` + +```text +(server1) $ consul members +Node Address Status Type Build Protocol DC Segment +server1 192.168.0.4:8301 alive server 0.9.3+ent 2 dc1 +client1 192.168.0.5:8301 alive client 0.9.3+ent 2 dc1 alpha +client2 192.168.0.6:8301 alive client 0.9.3+ent 2 dc1 beta +``` + +If we pass the `-segment` flag when running `consul members`, we can limit the view to agents +in a specific segment: + +```text +(server1) $ consul members -segment alpha +Node Address Status Type Build Protocol DC Segment +client1 192.168.0.5:8301 alive client 0.9.3+ent 2 dc1 alpha +server1 192.168.0.4:8303 alive server 0.9.3+ent 2 dc1 alpha +``` + +Using the `consul catalog nodes` command, we can filter on an internal metadata key, +`consul-network-segment`, which stores the network segment of the node: + +```text +(server1) $ consul catalog nodes -node-meta consul-network-segment=alpha +Node ID Address DC +client1 4c29819c 192.168.0.5 dc1 +``` + +With this metadata key, we can construct a prepared query that can be used for DNS to return +only services within the same network segment as the local agent. + +First, register a service on each of the client nodes: + +```text +(client1) $ curl \ + --request PUT \ + --data '{"Name": "redis", "Port": 8000}' \ + localhost:8500/v1/agent/service/register +``` + +```text +(client2) $ curl \ + --request PUT \ + --data '{"Name": "redis", "Port": 9000}' \ + localhost:8500/v1/agent/service/register +``` + +Next, write the following to `query.json` and create the query using the HTTP endpoint: + +```text +(server1) $ curl \ + --request POST \ + --data \ +'{ + "Name": "", + "Template": { + "Type": "name_prefix_match" + }, + "Service": { + "Service": "${name.full}", + "NodeMeta": {"consul-network-segment": "${agent.segment}"} + } +}' localhost:8500/v1/query + +{"ID":"6f49dd24-de9b-0b6c-fd29-525eca069419"} +``` + +Now, we can replace any dns lookups of the form `.service.consul` with +`.query.consul` to look up only services within the same network segment: + +**Client 1:** + +```text +(client1) $ dig @127.0.0.1 -p 8600 redis.query.consul SRV + +; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p 8600 redis.query.consul SRV +; (1 server found) +;; global options: +cmd +;; Got answer: +;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3149 +;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 +;; WARNING: recursion requested but not available + +;; QUESTION SECTION: +;redis.query.consul. IN SRV + +;; ANSWER SECTION: +redis.query.consul. 0 IN SRV 1 1 8000 client1.node.dc1.consul. + +;; ADDITIONAL SECTION: +client1.node.dc1.consul. 0 IN A 192.168.0.5 +``` + +**Client 2:** + +```text +(client2) $ dig @127.0.0.1 -p 8600 redis.query.consul SRV + +; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p 8600 redis.query.consul SRV +; (1 server found) +;; global options: +cmd +;; Got answer: +;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3149 +;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 +;; WARNING: recursion requested but not available + +;; QUESTION SECTION: +;redis.query.consul. IN SRV + +;; ANSWER SECTION: +redis.query.consul. 0 IN SRV 1 1 9000 client2.node.dc1.consul. + +;; ADDITIONAL SECTION: +client2.node.dc1.consul. 0 IN A 192.168.0.6 +``` \ No newline at end of file diff --git a/website/source/layouts/api.erb b/website/source/layouts/api.erb index 1cfb7c7482..98604b12bc 100644 --- a/website/source/layouts/api.erb +++ b/website/source/layouts/api.erb @@ -54,6 +54,9 @@ > Raft + > + Segment + > diff --git a/website/source/layouts/docs.erb b/website/source/layouts/docs.erb index 7002520ca1..5cd527bbe9 100644 --- a/website/source/layouts/docs.erb +++ b/website/source/layouts/docs.erb @@ -251,6 +251,9 @@ > Leader Election + > + Network Segments + > Outage Recovery