status-im
/
consul
mirror of https://github.com/status-im/consul.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

Website: tweaks to docs/internals/architecture.html.

This commit is contained in:
Ryan Breen 2015-03-30 18:07:58 -04:00
parent efb374e080
commit 5e6a4d4f7d
1 changed files with 33 additions and 27 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

60
website/source/docs/internals/architecture.html.markdown
View File

@ -36,29 +36,34 @@ bandwidth.
* Server - A server is an agent with an expanded set of responsibilities including
participating in the Raft quorum, maintaining cluster state, responding to RPC queries,
WAN gossip to other datacenters, and forwarding queries to leaders or remote datacenters.
exchanging WAN gossip with other datacenters, and forwarding queries to leaders or
remote datacenters.
* Datacenter - A datacenter seems obvious, but there are subtle details such as multiple
availability zones in EC2. We define a datacenter to be a networking environment that is
* Datacenter - While the definition of a datacenter seems obvious, there are subtle details
that must be considered. For example, in EC2, are multiple availability zones considered
to comprise a single datacenter? We define a datacenter to be a networking environment that is
private, low latency, and high bandwidth. This excludes communication that would traverse
the public internet.
the public internet, but for our purposes multiple availability zones within a single EC2
region would be considered part of a single datacenter.
* Consensus - When used in our documentation we use consensus to mean agreement upon
the elected leader as well as agreement on the ordering of transactions. Since these
transactions are applied to a FSM, we implicitly include the consistency of a replicated
state machine. Consensus is described in more detail on [Wikipedia](http://en.wikipedia.org/wiki/Consensus_(computer_science)),
transactions are applied to a
[finite-state machine](http://en.wikipedia.org/wiki/Finite-state_machine), our definition
of consensus implies the consistency of a replicated state machine. Consensus is described
in more detail on [Wikipedia](http://en.wikipedia.org/wiki/Consensus_(computer_science)),
and our implementation is described [here](/docs/internals/consensus.html).
* Gossip - Consul is built on top of [Serf](https://www.serfdom.io/), which provides a full
* Gossip - Consul is built on top of [Serf](https://www.serfdom.io/) which provides a full
[gossip protocol](http://en.wikipedia.org/wiki/Gossip_protocol) that is used for multiple purposes.
Serf provides membership, failure detection, and event broadcast mechanisms. Our use of these
Serf provides membership, failure detection, and event broadcast. Our use of these
is described more in the [gossip documentation](/docs/internals/gossip.html). It is enough to know
that gossip involves random node-to-node communication, primarily over UDP.
* LAN Gossip - Refers to the LAN gossip pool, which contains nodes that are all
* LAN Gossip - Refers to the LAN gossip pool which contains nodes that are all
located on the same local area network or datacenter.
* WAN Gossip - Refers to the WAN gossip pool, which contains only servers. These
* WAN Gossip - Refers to the WAN gossip pool which contains only servers. These
servers are primarily located in different datacenters and typically communicate
over the internet or wide area network.
@ -73,11 +78,12 @@ From a 10,000 foot altitude the architecture of Consul looks like this:
![Consul Architecture](consul-arch.png)
</div>
Let's break down this image and describe each piece. First of all we can see
that there are two datacenters, one and two respectively. Consul has first
class support for multiple datacenters and expects this to be the common case.
Let's break down this image and describe each piece. First of all, we can see
that there are two datacenters, labeled "one" and "two". Consul has first
class support for [multiple datacenters](/docs/guides/datacenters.html) and
expects this to be the common case.
Within each datacenter we have a mixture of clients and servers. It is expected
Within each datacenter, we have a mixture of clients and servers. It is expected
that there be between three to five servers. This strikes a balance between
availability in the case of failure and performance, as consensus gets progressively
slower as more machines are added. However, there is no limit to the number of clients,
@ -92,18 +98,18 @@ scalable than naive heartbeating schemes. Thirdly, it is used as a messaging lay
when important events such as leader election take place.
The servers in each datacenter are all part of a single Raft peer set. This means that
they work together to elect a leader, which has extra duties. The leader is responsible for
processing all queries and transactions. Transactions must also be replicated to all peers
as part of the [consensus protocol](/docs/internals/consensus.html). Because of this requirement,
when a non-leader server receives an RPC request it forwards it to the cluster leader.
they work together to elect a single leader, a selected server which has extra duties. The leader
is responsible for processing all queries and transactions. Transactions must also be replicated to
all peers as part of the [consensus protocol](/docs/internals/consensus.html). Because of this
requirement, when a non-leader server receives an RPC request, it forwards it to the cluster leader.
The server nodes also operate as part of a WAN gossip pool. This pool is different from the LAN pool,
as it is optimized for the higher latency of the internet, and is expected to contain only
The server nodes also operate as part of a WAN gossip pool. This pool is different from the LAN pool
as it is optimized for the higher latency of the internet and is expected to contain only
other Consul server nodes. The purpose of this pool is to allow datacenters to discover each
other in a low touch manner. Bringing a new datacenter online is as easy as joining the existing
WAN gossip. Because the servers are all operating in this pool, it also enables cross-datacenter requests.
When a server receives a request for a different datacenter, it forwards it to a random server
in the correct datacenter. That server may then forward to the local leader.
other in a low-touch manner. Bringing a new datacenter online is as easy as joining the existing
WAN gossip. Because the servers are all operating in this pool, it also enables cross-datacenter
requests. When a server receives a request for a different datacenter, it forwards it to a random
server in the correct datacenter. That server may then forward to the local leader.
This results in a very low coupling between datacenters, but because of failure detection,
connection caching and multiplexing, cross-datacenter requests are relatively fast and reliable.
@ -111,8 +117,8 @@ connection caching and multiplexing, cross-datacenter requests are relatively fa
## Getting in depth
At this point we've covered the high level architecture of Consul, but there are many
more details for each of the sub-systems. The [consensus protocol](/docs/internals/consensus.html) is
documented in detail, as is the [gossip protocol](/docs/internals/gossip.html). The [documentation](/docs/internals/security.html)
more details for each of the subsystems. The [consensus protocol](/docs/internals/consensus.html) is
documented in detail as is the [gossip protocol](/docs/internals/gossip.html). The [documentation](/docs/internals/security.html)
for the security model and protocols used are also available.
For other details, either consult the code, ask in IRC or reach out to the mailing list.
For other details, either consult the code, ask in IRC, or reach out to the mailing list.