From d4470839fd3527b442e217b5b813e08e7ca49c84 Mon Sep 17 00:00:00 2001 From: Armon Dadgar Date: Thu, 10 Apr 2014 17:41:49 -0700 Subject: [PATCH] website: Working on getting started guide --- .../intro/getting-started/agent.html.markdown | 128 ++++++++++++------ .../getting-started/checks.html.markdown | 8 ++ .../getting-started/install.html.markdown | 48 +++---- .../intro/getting-started/join.html.markdown | 57 ++++---- .../intro/getting-started/kv.html.markdown | 8 ++ .../getting-started/services.html.markdown | 8 ++ website/source/layouts/intro.erb | 4 + 7 files changed, 168 insertions(+), 93 deletions(-) create mode 100644 website/source/intro/getting-started/checks.html.markdown create mode 100644 website/source/intro/getting-started/kv.html.markdown create mode 100644 website/source/intro/getting-started/services.html.markdown diff --git a/website/source/intro/getting-started/agent.html.markdown b/website/source/intro/getting-started/agent.html.markdown index c18d26b700..c32398f405 100644 --- a/website/source/intro/getting-started/agent.html.markdown +++ b/website/source/intro/getting-started/agent.html.markdown @@ -4,79 +4,125 @@ page_title: "Run the Agent" sidebar_current: "gettingstarted-agent" --- -# Run the Serf Agent +# Run the Consul Agent -After Serf is installed, the agent must be run. The agent is a lightweight -process that runs until told to quit and maintains cluster membership -and communication. The agent must be run for every node that will be part of -the cluster. - -It is possible to run multiple agents, and thus participate in multiple Serf -clusters. For example, you may want to run a separate Serf cluster to -maintain web server membership info for a load balancer from another Serf -cluster that manages membership of Memcached nodes, but perhaps the web -servers need to be part of the Memcached cluster too so they can be notified -when Memcached nodes come online or go offline. Other examples include a Serf -cluster within a datacenter, and a seperate cluster used for cross WAN gossip -which has more relaxed timing. +After Consul is installed, the agent must be run. The agent can either run +in a server or client mode. Each datacenter must at least one server, and +a recommended 3 or 5. [This guide](/docs/guides/bootstrapping.html) covers +bootstrapping a new datacenter. All other agents run in client mode, which +is a very lightweight process that registers services, runs health checks, +and forwards queries to servers. The agent must be run for every node that +will be part of the cluster. ## Starting the Agent -For simplicity, we'll run a single Serf agent right now: +For simplicity, we'll run a single Consul agent in server mode right now: ``` -$ serf agent -==> Starting Serf agent... -==> Serf agent running! - Node name: 'foobar' - Bind addr: '0.0.0.0:7946' - RPC addr: '127.0.0.1:7373' +$ ./bin/consul agent -server -bootstrap -data-dir /tmp/consul +==> WARNING: Bootstrap mode enabled! Do not enable unless necessary +==> WARNING: It is highly recommended to set GOMAXPROCS higher than 1 +==> Starting Consul agent... +==> Starting Consul agent RPC... +==> Consul agent running! + Node name: 'Armons-MacBook-Air' + Datacenter: 'dc1' + Advertise addr: '10.1.10.38' + RPC addr: '127.0.0.1:8400' + HTTP addr: '127.0.0.1:8500' + DNS addr: '127.0.0.1:8600' + Encrypted: false + Server: true (bootstrap: true) ==> Log data will now stream in as it occurs: -2013/10/21 18:57:15 [INFO] Serf agent starting -2013/10/21 18:57:15 [INFO] serf: EventMemberJoin: mitchellh.local 10.0.1.60 -2013/10/21 18:57:15 [INFO] Serf agent started -2013/10/21 18:57:15 [INFO] agent: Received event: member-join +[INFO] serf: EventMemberJoin: Armons-MacBook-Air 10.1.10.38 +[INFO] serf: EventMemberJoin: Armons-MacBook-Air 10.1.10.38 +[INFO] raft: Node at 10.1.10.38:8300 [Follower] entering Follower state +[INFO] consul: adding server for datacenter: dc1, addr: 10.1.10.38:8300 +[ERR] agent: failed to sync remote state: rpc error: No cluster leader +[WARN] raft: Heartbeat timeout reached, starting election +[INFO] raft: Node at 10.1.10.38:8300 [Candidate] entering Candidate state +[INFO] raft: Election won. Tally: 1 +[INFO] raft: Node at 10.1.10.38:8300 [Leader] entering Leader state +[INFO] consul: cluster leadership acquired +[INFO] consul: New leader elected: Armons-MacBook-Air +[INFO] consul: member 'Armons-MacBook-Air' joined, marking health alive ``` -As you can see, the Serf agent has started and has output some log -data. From the log data, you can see that a member has joined the cluster. -This member is yourself. +As you can see, the Consul agent has started and has output some log +data. From the log data, you can see that our agent is running in server mode, +and has claimed leadership of the cluster. Additionally, the local member has +been marked as a healthy member of the cluster. ## Cluster Members -If you run `serf members` in another terminal, you can see the members of -the Serf cluster. You should only see one member (yourself). We'll cover +If you run `consul members` in another terminal, you can see the members of +the Consul cluster. You should only see one member (yourself). We'll cover joining clusters in the next section. ``` -$ serf members -mitchellh.local 10.0.1.60 alive +$ consul members +Armons-MacBook-Air 10.1.10.38:8301 alive role=consul,dc=dc1,vsn=1,vsn_min=1,vsn_max=1,port=8300,bootstrap=1 ``` -This command, along with many others, communicates with a running Serf -agent via an internal RPC protocol. When starting the Serf agent, you +This command, along with many others, communicates with a running Consul +agent via an internal RPC protocol. When starting the Consul agent, you may have noticed that it tells you the "RPC addr". This is the address -that commands such as `serf members` use to communicate with the agent. +that commands such as `consul members` use to communicate with the agent. By default, RPC listens only on loopback, so it is inaccessible outside of your machine for security reasons. -If you're running multiple Serf agents, you'll have to specify +If you're changed the default RPC address, you'll have to specify an `-rpc-addr` to both the agent and any commands so that it doesn't collide with other agents. +It is important to note that the output of the `members` command is +generated by from the [gossip protocol](/docs/internals/gossip.html), +and is eventually consistent. Consul uses this to maintain a strongly +consistent catalog of nodes that can be queried using the [HTTP API](/docs/agent/http.html): + +``` +$ curl localhost:8500/v1/catalog/nodes +[{"Node":"Armons-MacBook-Air","Address":"10.1.10.38"}] +``` + +Alternatively, the [DNS interface](/docs/agent/dns.html) could be used: + +``` +$ dig @127.0.0.1 -p 8600 Armons-MacBook-Air.node.consul + +; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p 8600 Armons-MacBook-Air.node.consul +; (1 server found) +;; global options: +cmd +;; Got answer: +;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63911 +;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 +;; WARNING: recursion requested but not available + +;; QUESTION SECTION: +;Armons-MacBook-Air.node.consul. IN A + +;; ANSWER SECTION: +Armons-MacBook-Air.node.consul. 0 IN A 10.1.10.38 +``` + ## Stopping the Agent You can use `Ctrl-C` (the interrupt signal) to gracefully halt the agent. After interrupting the agent, you should see it leave the cluster gracefully and shut down. -By gracefully leaving, Serf notifies other cluster members that the +By gracefully leaving, Consul notifies other cluster members that the node _left_. If you had forcibly killed the agent process, other members -of the cluster would have detected that the node _failed_. This can be a -crucial difference depending on what your use case of Serf is. Serf will -automatically try to reconnect to _failed_ nodes, which allows it to recover -from certain network conditions, while _left_ nodes are no longer contacted. +of the cluster would have detected that the node _failed_. When a member leaves, +it's services and checks are removed from the catalog. When a member fails, +it's health is simply marked as critical, but is not removed from the catalog. +Consul will automatically try to reconnect to _failed_ nodes, which allows it +to recover from certain network conditions, while _left_ nodes are no longer contacted. + +Additionally, if an agent is operating as a server, a graceful leave is important +to avoid causing a potential availability outage affecting the [consensus protocol](/docs/internals/consensus.html). +See the [guides section](/docs/guides/index.html) to safely add and remove servers. diff --git a/website/source/intro/getting-started/checks.html.markdown b/website/source/intro/getting-started/checks.html.markdown new file mode 100644 index 0000000000..36c3f01830 --- /dev/null +++ b/website/source/intro/getting-started/checks.html.markdown @@ -0,0 +1,8 @@ +--- +layout: "intro" +page_title: "Registering Health Checks" +sidebar_current: "gettingstarted-checks" +--- + +# Registering Health Checks + diff --git a/website/source/intro/getting-started/install.html.markdown b/website/source/intro/getting-started/install.html.markdown index 5ab73a4f9d..1b3215ccb2 100644 --- a/website/source/intro/getting-started/install.html.markdown +++ b/website/source/intro/getting-started/install.html.markdown @@ -1,23 +1,23 @@ --- layout: "intro" -page_title: "Installing Serf" +page_title: "Installing Consul" sidebar_current: "gettingstarted-install" --- -# Install Serf +# Install Consul -Serf must first be installed on every node that will be a member of a -Serf cluster. To make installation easy, Serf is distributed as a +Consul must first be installed on every node that will be a member of a +Consul cluster. To make installation easy, Consul is distributed as a [binary package](/downloads.html) for all supported platforms and -architectures. This page will not cover how to compile Serf from +architectures. This page will not cover how to compile Consul from source. -## Installing Serf +## Installing Consul -To install Serf, find the [appropriate package](/downloads.html) for -your system and download it. Serf is packaged as a "zip" archive. +To install Consul, find the [appropriate package](/downloads.html) for +your system and download it. Consul is packaged as a "zip" archive. -After downloading Serf, unzip the package. Copy the `serf` binary to +After downloading Consul, unzip the package. Copy the `consul` binary to somewhere on the PATH so that it can be executed. On Unix systems, `~/bin` and `/usr/local/bin` are common installation directories, depending on if you want to restrict the install to a single user or @@ -26,28 +26,28 @@ you would like. ## Verifying the Installation -After installing Serf, verify the installation worked by opening a new -terminal session and checking that `serf` is available. By executing -`serf` you should see help output similar to that below: +After installing Consul, verify the installation worked by opening a new +terminal session and checking that `consul` is available. By executing +`consul` you should see help output similar to that below: ``` -$ serf -usage: serf [--version] [--help] [] +$ consul +usage: consul [--version] [--help] [] Available commands are: - agent Runs a Serf agent - event Send a custom event through the Serf cluster + agent Runs a Consul agent force-leave Forces a member of the cluster to enter the "left" state - join Tell Serf agent to join cluster + info Provides debugging information for operators + join Tell Consul agent to join cluster keygen Generates a new encryption key - leave Gracefully leaves the Serf cluster and shuts down - members Lists the members of a Serf cluster - monitor Stream logs from a Serf agent - version Prints the Serf version + leave Gracefully leaves the Consul cluster and shuts down + members Lists the members of a Consul cluster + monitor Stream logs from a Consul agent + version Prints the Consul version ``` -If you get an error that `serf` could not be found, then your PATH +If you get an error that `consul` could not be found, then your PATH environmental variable was not setup properly. Please go back and ensure -that your PATH variable contains the directory where Serf was installed. +that your PATH variable contains the directory where Consul was installed. -Otherwise, Serf is installed and ready to go! +Otherwise, Consul is installed and ready to go! diff --git a/website/source/intro/getting-started/join.html.markdown b/website/source/intro/getting-started/join.html.markdown index a016b9b2a5..8d73f8112c 100644 --- a/website/source/intro/getting-started/join.html.markdown +++ b/website/source/intro/getting-started/join.html.markdown @@ -7,46 +7,46 @@ sidebar_current: "gettingstarted-join" # Join a Cluster In the previous page, we started our first agent. While it showed how easy -it is to run Serf, it wasn't very exciting since we simply made a cluster of +it is to run Consul, it wasn't very exciting since we simply made a cluster of one member. In this page, we'll create a real cluster with multiple members. -When starting a Serf agent, it begins without knowledge of any other node, and is +When starting a Consul agent, it begins without knowledge of any other node, and is an isolated cluster of one. To learn about other cluster members, the agent must -_join_ an existing cluster. To join an existing cluster, Serf only needs to know +_join_ an existing cluster. To join an existing cluster, only needs to know about a _single_ existing member. After it joins, the agent will gossip with this member and quickly discover the other members in the cluster. ## Starting the Agents -First, let's start two agents. Serf agents must all listen on a unique -IP and port pair, so we must bind each agent to a different ports. +To simulate a more realistic cluster, we are using a two node cluster in +Vagrant. The Vagrantfile can be found in the demo section of the repo +[here](https://github.com/hashicorp/consul/tree/master/demo/vagrant-cluster). -The first agent we'll start will listen on `127.0.0.1:7946`. We also will -specify a node name. The node name must be unique and is how a machine -is uniquely identified. By default it is the hostname of the machine, but -since we'll be running multiple agents on a single machine, we'll manually -override it. +We start the first agent on our first node and also specify a node name. +The node name must be unique and is how a machine is uniquely identified. +By default it is the hostname of the machine, but we'll manually override it. +We are also providing a bind address. This is the address that Consul listens on, +and it *must* be accessible by all other nodes in the cluster. The first node +will act as our server in this cluster. ``` -$ serf agent -node=agent-one -bind=127.0.0.1:7946 +$ consul agent -node=agent-one -bind=172.20.20.10 ... ``` -Then, in another terminal, start a second agent. We'll bind this agent -to `127.0.0.1:7947`. In addition to overriding the node name, we're also going -to override the RPC address. The RPC address is the address that Serf binds -to for RPC operations. The other `serf` commands communicate with a running -Serf agent over RPC. We left the first agent with the default RPC address -so lets select another for this agent. +Then, in another terminal, start the second agent on the new node. +This time, we set the bind address to match the IP of the second node +as specified in the Vagrantfile. In production, you will generally want +to provide a bind address or interface as well. ``` -$ serf agent -node=agent-two -bind=127.0.0.1:7947 -rpc-addr=127.0.0.1:7374 +$ consul agent -node=agent-two -bind=172.20.20.11 ... ``` -At this point, you have two Serf agents running. The two Serf agents -still don't know anything about each other, and are each part of their own -clusters (of one member). You can verify this by running `serf members` +At this point, you have two Consul agents running, one server and one client. +The two Consul agents still don't know anything about each other, and are each part of their own +clusters (of one member). You can verify this by running `consul members` against each agent and noting that only one member is a part of each. ## Joining a Cluster @@ -55,34 +55,34 @@ Now, let's tell the first agent to join the second agent by running the following command in a new terminal: ``` -$ serf join 127.0.0.1:7947 +$ consul join 127.0.0.1:7947 Successfully joined cluster by contacting 1 nodes. ``` You should see some log output in each of the agent logs. If you read carefully, you'll see that they received join information. If you -run `serf members` against each agent, you'll see that both agents now +run `consul members` against each agent, you'll see that both agents now know about each other: ``` -$ serf members +$ consul members agent-one 127.0.0.1:7946 alive agent-two 127.0.0.1:7947 alive -$ serf members -rpc-addr=127.0.0.1:7374 +$ consul members -rpc-addr=127.0.0.1:7374 agent-two 127.0.0.1:7947 alive agent-one 127.0.0.1:4946 alive ```
-

Remember: To join a cluster, a Serf agent needs to only +

Remember: To join a cluster, a Consul agent needs to only learn about one existing member. After joining the cluster, the agents gossip with each other to propagate full membership information.

-In addition to using `serf join` you can use the `-join` flag on -`serf agent` to join a cluster as part of starting up the agent. +In addition to using `consul join` you can use the `-join` flag on +`consul agent` to join a cluster as part of starting up the agent. ## Leaving a Cluster @@ -91,3 +91,4 @@ To leave the cluster, you can either gracefully quit an agent (using the node to transition into the _left_ state, otherwise other nodes will detect it as having _failed_. The difference is covered in more detail [here](/intro/getting-started/agent.html#toc_3). + diff --git a/website/source/intro/getting-started/kv.html.markdown b/website/source/intro/getting-started/kv.html.markdown new file mode 100644 index 0000000000..cae74e6d7b --- /dev/null +++ b/website/source/intro/getting-started/kv.html.markdown @@ -0,0 +1,8 @@ +--- +layout: "intro" +page_title: "Key/Value Data" +sidebar_current: "gettingstarted-kv" +--- + +# Key/Value Data + diff --git a/website/source/intro/getting-started/services.html.markdown b/website/source/intro/getting-started/services.html.markdown new file mode 100644 index 0000000000..9e7568605a --- /dev/null +++ b/website/source/intro/getting-started/services.html.markdown @@ -0,0 +1,8 @@ +--- +layout: "intro" +page_title: "Registering Services" +sidebar_current: "gettingstarted-services" +--- + +# Registering Services + diff --git a/website/source/layouts/intro.erb b/website/source/layouts/intro.erb index 92e5102dd5..b5ddeb29cc 100644 --- a/website/source/layouts/intro.erb +++ b/website/source/layouts/intro.erb @@ -60,6 +60,10 @@ > Health Checks + + + > + Key/Value Data >