Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure. https://www.consul.io
Go to file
Mitchell Hashimoto f76022fa63 CA Provider Plugins (#4751)
This adds the `agent/connect/ca/plugin` library for consuming/serving Connect CA providers as [go-plugin](https://github.com/hashicorp/go-plugin) plugins. This **does not** wire this up in any way to Consul itself, so this will not enable using these plugins yet. 

## Why?

We want to enable CA providers to be pluggable without modifying Consul so that any CA or PKI system can potentially back the Connect certificates. This CA system may also be used in the future for easier bootstrapping and internal cluster security.

### go-plugin

The benefit of `go-plugin` is that for the plugin consumer, the fact that the interface implementation is communicating over multi-process RPC is invisible. Internals of Consul will continue to just use `ca.Provider` interface implementations as if they're local. For plugin _authors_, they simply have to implement the interface. The network/transport/process management issues are handled by go-plugin itself.

The CA provider plugins support both `net/rpc` and gRPC transports. This enables easy authoring in any language. go-plugin handles the actual protocol handshake and connection. This is just a feature of go-plugin. 

`go-plugin` is already in production use for years by Packer, Terraform, Nomad, Vault, and Sentinel. We've shown stability for both desktop and server-side software. It is very mature.

## Implementation Details

### `map[string]interface{}`

The `Configure` method passes a `map[string]interface{}`. This map contains only Go primitives and containers of primitives (no funcs, chans, etc.). For `net/rpc` we encode as-is using Gob. For gRPC we marshal to JSON and transmit as a `bytes` type. This is the same approach we take with Vault and other software.

Note that this is just the transport protocol, the end software views it fully decoded.

### `x509.Certificate` and `CertificateRequest`

We transmit the raw ASN.1  bytes and decode on the other side. Unit tests are verifying we get the same cert/csrs across the wire.

### Testing

`go-plugin` exposes test helpers that enable testing the full plugin RPC over real loopback network connections. We test all endpoints for success and error for both `net/rpc` and gRPC.

### Vendoring

This PR doesn't introduce vendoring for two reasons:

  1. @banks's `f-envoy` branch introduces a lot of these and I didn't want conflict.
  2. The library isn't actually used yet so it doesn't introduce compile-time errors (it does introduce test errors).

## Next Steps

With this in place, we need to figure out the proper way to actually hook these up to Consul, load them, etc. This discussion can happen elsewhere, since regardless of approach this plugin library implementation is the exact same.
2019-01-07 12:48:44 -05:00
.circleci refresh cache key and fix comments 2019-01-03 22:57:40 -05:00
.github add codeowners for consul docs 2019-01-03 15:01:16 -05:00
acl New ACLs (#4791) 2018-10-19 12:04:07 -04:00
agent CA Provider Plugins (#4751) 2019-01-07 12:48:44 -05:00
api Implement /v1/agent/health/service/<service name> endpoint (#3551) 2019-01-07 09:39:23 -05:00
bench Gets benchmarks running again and does a rough pass for 0.7.1. 2016-11-29 13:02:26 -08:00
build-support Dont expect the freebsd/arm build 2018-10-19 14:50:51 -04:00
command Implement /v1/agent/health/service/<service name> endpoint (#3551) 2019-01-07 09:39:23 -05:00
connect connect: remove additional trust-domain validation (#4934) 2018-11-12 20:20:12 +00:00
demo demo: Added udp port forwarding 2018-05-30 13:56:56 +09:00
ipaddr New config parser, HCL support, multiple bind addrs (#3480) 2017-09-25 11:40:42 -07:00
lib New ACLs (#4791) 2018-10-19 12:04:07 -04:00
logger Connect Envoy Command (#4735) 2018-10-10 16:55:34 +01:00
sentinel Renames stubs to be more consistent. 2017-11-29 18:36:52 -08:00
service_os Changes made : 2018-06-28 21:18:14 -04:00
snapshot Removes timeout when restoring snapshots. 2017-12-13 14:10:54 -08:00
terraform terraform: fix formatting of consul.tf (#4580) 2018-10-24 08:02:38 -07:00
test Update test certificates that expire this year to be way in the future 2018-05-12 10:15:45 +01:00
testrpc Bugfix: Use "%#v" when formatting structs (#4600) 2018-08-28 12:37:34 -04:00
testutil testutil: If testing.T is nil panic with error (#4725) 2018-10-26 12:13:40 -07:00
tlsutil agent: honor when ca is set but verify_outgoing is disabled (#4826) 2018-12-17 09:56:18 -08:00
types Removes remoteConsuls in favor of the new router. 2017-03-16 16:42:19 -07:00
ui update ffi to dodge CVE-2018-1000201 (#4670) 2018-09-14 11:22:48 +02:00
ui-v2 add documentation on how to use ember-exam 2019-01-03 23:50:02 -05:00
vendor vendor: upgrade to latest version of gopsutil 2018-10-19 11:33:23 -07:00
version Put the tree in v1.4.1 dev mode 2018-11-14 16:56:48 -08:00
watch Add -sidecar-for and new /agent/service/:service_id endpoint (#4691) 2018-10-10 16:55:34 +01:00
website website: fix stray sentinel references using the old syntax (#5191) 2019-01-07 09:59:17 -06:00
.dockerignore Update the scripting 2018-06-14 21:42:47 -04:00
.gitignore merge feedback: fix typos; actually use deliverLatest added previously but not plumbed in 2018-10-10 16:55:34 +01:00
.travis.yml Bump Go version for CI and build to 1.11.1 (#4782) 2018-10-11 12:21:53 +01:00
CHANGELOG.md Update CHANGELOG.md 2019-01-07 09:56:31 -05:00
GNUmakefile CA Provider Plugins (#4751) 2019-01-07 12:48:44 -05:00
INTERNALS.md docs: correct link to top level agent package (#4750) 2018-10-04 09:15:55 -05:00
LICENSE Initial commit 2013-11-04 14:15:27 -08:00
NOTICE.md add copyright notice file 2018-07-09 10:58:26 -07:00
README.md Contribution guide (#4704) 2018-10-05 09:06:40 -07:00
Vagrantfile Adds a basic Linux Vagrant setup, stolen from Nomad. 2017-10-06 08:10:12 -07:00
main.go Added Side Effect import for Windows Service 2018-06-18 14:55:11 -04:00
main_test.go Adding basic CLI infrastructure 2013-12-19 11:22:08 -08:00

README.md

Consul Build Status Join the chat at https://gitter.im/hashicorp-consul/Lobby

Consul is a tool for service discovery and configuration. Consul is distributed, highly available, and extremely scalable.

Consul provides several key features:

  • Service Discovery - Consul makes it simple for services to register themselves and to discover other services via a DNS or HTTP interface. External services such as SaaS providers can be registered as well.

  • Health Checking - Health Checking enables Consul to quickly alert operators about any issues in a cluster. The integration with service discovery prevents routing traffic to unhealthy hosts and enables service level circuit breakers.

  • Key/Value Storage - A flexible key/value store enables storing dynamic configuration, feature flagging, coordination, leader election and more. The simple HTTP API makes it easy to use anywhere.

  • Multi-Datacenter - Consul is built to be datacenter aware, and can support any number of regions without complex configuration.

  • Service Segmentation - Consul Connect enables secure service-to-service communication with automatic TLS encryption and identity-based authorization.

Consul runs on Linux, Mac OS X, FreeBSD, Solaris, and Windows. A commercial version called Consul Enterprise is also available.

Please note: We take Consul's security and our users' trust very seriously. If you believe you have found a security issue in Consul, please responsibly disclose by contacting us at security@hashicorp.com.

Quick Start

An extensive quick start is viewable on the Consul website:

https://www.consul.io/intro/getting-started/install.html

Documentation

Full, comprehensive documentation is viewable on the Consul website:

https://www.consul.io/docs

Contributing

Thank you for your interest in contributing! Please refer to CONTRIBUTING.md for guidance.