consul/website/pages/docs/internals/coordinates.mdx

104 lines
4.1 KiB
Plaintext
Raw Normal View History

2015-10-22 00:29:43 +00:00
---
2020-04-07 18:55:19 +00:00
layout: docs
page_title: Network Coordinates
sidebar_title: Network Coordinates
2020-04-07 18:55:19 +00:00
description: ''
Serf uses a network tomography system to compute network coordinates for nodes in the cluster. These coordinates are useful for easily calculating the estimated network round trip time between any two nodes in the cluster. This page documents the details of this system. The core of the network tomography system us based on Vivaldi: >-
2020-04-07 18:55:19 +00:00
A Decentralized Network Coordinate System, with several improvements based on
several follow-on papers.
2015-10-22 00:29:43 +00:00
---
# Network Coordinates
Consul uses a [network tomography](https://en.wikipedia.org/wiki/Network_tomography)
system to compute network coordinates for nodes in the cluster. These coordinates
allow the network round trip time to be estimated between any two nodes using a
2015-10-31 06:50:11 +00:00
very simple calculation. This allows for many useful applications, such as finding
2015-10-22 00:29:43 +00:00
the service node nearest a requesting node, or failing over to services in the next
closest datacenter.
2016-08-08 16:44:27 +00:00
All of this is provided through the use of the [Serf library](https://www.serf.io/).
2015-10-22 00:29:43 +00:00
Serf's network tomography is based on ["Vivaldi: A Decentralized Network Coordinate System"](http://www.cs.ucsb.edu/~ravenben/classes/276/papers/vivaldi-sigcomm04.pdf),
with some enhancements based on other research. There are more details about
2016-08-08 16:44:27 +00:00
[Serf's network coordinates here](https://www.serf.io/docs/internals/coordinates.html).
2015-10-22 00:29:43 +00:00
## Network Coordinates in Consul
Network coordinates manifest in several ways inside Consul:
2020-04-09 23:46:54 +00:00
- The [`consul rtt`](/docs/commands/rtt) command can be used to query for the
2015-10-22 00:29:43 +00:00
network round trip time between any two nodes.
2020-04-09 23:46:54 +00:00
- The [Catalog endpoints](/api/catalog) and
[Health endpoints](/api/health) can sort the results of queries based
2015-10-22 00:29:43 +00:00
on the network round trip time from a given node using a "?near=" parameter.
2020-04-09 23:46:54 +00:00
- [Prepared queries](/api/query) can automatically fail over services
2017-11-01 04:23:51 +00:00
to other Consul datacenters based on network round trip times. See the
[Geo Failover](https://learn.hashicorp.com/consul/developer-discovery/geo-failover) for some examples.
2020-04-09 23:46:54 +00:00
- The [Coordinate endpoint](/api/coordinate) exposes raw network
2015-10-22 00:29:43 +00:00
coordinates for use in other applications.
Consul uses Serf to manage two different gossip pools, one for the LAN with members
of a given datacenter, and one for the WAN which is made up of just the Consul servers
in all datacenters. It's important to note that **network coordinates are not compatible
between these two pools**. LAN coordinates only make sense in calculations with other
LAN coordinates, and WAN coordinates only make sense with other WAN coordinates.
## Working with Coordinates
Computing the estimated network round trip time between any two nodes is simple
once you have their coordinates. Here's a sample coordinate, as returned from the
2020-04-09 23:46:54 +00:00
[Coordinate endpoint](/api/coordinate).
2015-10-22 00:29:43 +00:00
```
"Coord": {
"Adjustment": 0.1,
"Error": 1.5,
"Height": 0.02,
"Vec": [0.34,0.68,0.003,0.01,0.05,0.1,0.34,0.06]
}
```
All values are floating point numbers in units of seconds, except for the error
term which isn't used for distance calculations.
Here's a complete example in Go showing how to compute the distance between two
coordinates:
```
import (
"math"
"time"
2017-05-05 10:36:47 +00:00
"github.com/hashicorp/serf/coordinate"
2015-10-22 00:29:43 +00:00
)
func dist(a *coordinate.Coordinate, b *coordinate.Coordinate) time.Duration {
// Coordinates will always have the same dimensionality, so this is
// just a sanity check.
if len(a.Vec) != len(b.Vec) {
panic("dimensions aren't compatible")
}
// Calculate the Euclidean distance plus the heights.
sumsq := 0.0
for i := 0; i < len(a.Vec); i++ {
diff := a.Vec[i] - b.Vec[i]
sumsq += diff * diff
}
rtt := math.Sqrt(sumsq) + a.Height + b.Height
// Apply the adjustment components, guarding against negatives.
adjusted := rtt + a.Adjustment + b.Adjustment
if adjusted > 0.0 {
rtt = adjusted
}
// Go's times are natively nanoseconds, so we convert from seconds.
const secondsToNanoseconds = 1.0e9
return time.Duration(rtt * secondsToNanoseconds)
}
```