CA Provider Plugins (#4751)
This adds the `agent/connect/ca/plugin` library for consuming/serving Connect CA providers as [go-plugin](https://github.com/hashicorp/go-plugin) plugins. This **does not** wire this up in any way to Consul itself, so this will not enable using these plugins yet.
## Why?
We want to enable CA providers to be pluggable without modifying Consul so that any CA or PKI system can potentially back the Connect certificates. This CA system may also be used in the future for easier bootstrapping and internal cluster security.
### go-plugin
The benefit of `go-plugin` is that for the plugin consumer, the fact that the interface implementation is communicating over multi-process RPC is invisible. Internals of Consul will continue to just use `ca.Provider` interface implementations as if they're local. For plugin _authors_, they simply have to implement the interface. The network/transport/process management issues are handled by go-plugin itself.
The CA provider plugins support both `net/rpc` and gRPC transports. This enables easy authoring in any language. go-plugin handles the actual protocol handshake and connection. This is just a feature of go-plugin.
`go-plugin` is already in production use for years by Packer, Terraform, Nomad, Vault, and Sentinel. We've shown stability for both desktop and server-side software. It is very mature.
## Implementation Details
### `map[string]interface{}`
The `Configure` method passes a `map[string]interface{}`. This map contains only Go primitives and containers of primitives (no funcs, chans, etc.). For `net/rpc` we encode as-is using Gob. For gRPC we marshal to JSON and transmit as a `bytes` type. This is the same approach we take with Vault and other software.
Note that this is just the transport protocol, the end software views it fully decoded.
### `x509.Certificate` and `CertificateRequest`
We transmit the raw ASN.1 bytes and decode on the other side. Unit tests are verifying we get the same cert/csrs across the wire.
### Testing
`go-plugin` exposes test helpers that enable testing the full plugin RPC over real loopback network connections. We test all endpoints for success and error for both `net/rpc` and gRPC.
### Vendoring
This PR doesn't introduce vendoring for two reasons:
1. @banks's `f-envoy` branch introduces a lot of these and I didn't want conflict.
2. The library isn't actually used yet so it doesn't introduce compile-time errors (it does introduce test errors).
## Next Steps
With this in place, we need to figure out the proper way to actually hook these up to Consul, load them, etc. This discussion can happen elsewhere, since regardless of approach this plugin library implementation is the exact same.
2019-01-07 17:48:44 +00:00
|
|
|
package plugin
|
|
|
|
|
|
|
|
import (
|
|
|
|
"crypto/x509"
|
|
|
|
"net/rpc"
|
|
|
|
|
|
|
|
"github.com/hashicorp/consul/agent/connect/ca"
|
|
|
|
)
|
|
|
|
|
|
|
|
// providerPluginRPCServer implements a net/rpc backed transport for
|
|
|
|
// an underlying implementation of a ca.Provider. The server side is the
|
|
|
|
// plugin binary itself.
|
|
|
|
type providerPluginRPCServer struct {
|
|
|
|
impl ca.Provider
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCServer) Configure(args *ConfigureRPCRequest, _ *struct{}) error {
|
2019-08-27 21:09:01 +00:00
|
|
|
return p.impl.Configure(args.ClusterId, args.DatacenterName, args.DNSDomain, args.IsRoot, args.RawConfig)
|
CA Provider Plugins (#4751)
This adds the `agent/connect/ca/plugin` library for consuming/serving Connect CA providers as [go-plugin](https://github.com/hashicorp/go-plugin) plugins. This **does not** wire this up in any way to Consul itself, so this will not enable using these plugins yet.
## Why?
We want to enable CA providers to be pluggable without modifying Consul so that any CA or PKI system can potentially back the Connect certificates. This CA system may also be used in the future for easier bootstrapping and internal cluster security.
### go-plugin
The benefit of `go-plugin` is that for the plugin consumer, the fact that the interface implementation is communicating over multi-process RPC is invisible. Internals of Consul will continue to just use `ca.Provider` interface implementations as if they're local. For plugin _authors_, they simply have to implement the interface. The network/transport/process management issues are handled by go-plugin itself.
The CA provider plugins support both `net/rpc` and gRPC transports. This enables easy authoring in any language. go-plugin handles the actual protocol handshake and connection. This is just a feature of go-plugin.
`go-plugin` is already in production use for years by Packer, Terraform, Nomad, Vault, and Sentinel. We've shown stability for both desktop and server-side software. It is very mature.
## Implementation Details
### `map[string]interface{}`
The `Configure` method passes a `map[string]interface{}`. This map contains only Go primitives and containers of primitives (no funcs, chans, etc.). For `net/rpc` we encode as-is using Gob. For gRPC we marshal to JSON and transmit as a `bytes` type. This is the same approach we take with Vault and other software.
Note that this is just the transport protocol, the end software views it fully decoded.
### `x509.Certificate` and `CertificateRequest`
We transmit the raw ASN.1 bytes and decode on the other side. Unit tests are verifying we get the same cert/csrs across the wire.
### Testing
`go-plugin` exposes test helpers that enable testing the full plugin RPC over real loopback network connections. We test all endpoints for success and error for both `net/rpc` and gRPC.
### Vendoring
This PR doesn't introduce vendoring for two reasons:
1. @banks's `f-envoy` branch introduces a lot of these and I didn't want conflict.
2. The library isn't actually used yet so it doesn't introduce compile-time errors (it does introduce test errors).
## Next Steps
With this in place, we need to figure out the proper way to actually hook these up to Consul, load them, etc. This discussion can happen elsewhere, since regardless of approach this plugin library implementation is the exact same.
2019-01-07 17:48:44 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCServer) GenerateRoot(struct{}, *struct{}) error {
|
|
|
|
return p.impl.GenerateRoot()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCServer) ActiveRoot(_ struct{}, resp *ActiveRootResponse) error {
|
|
|
|
var err error
|
|
|
|
resp.CrtPem, err = p.impl.ActiveRoot()
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCServer) GenerateIntermediateCSR(_ struct{}, resp *GenerateIntermediateCSRResponse) error {
|
|
|
|
var err error
|
|
|
|
resp.CsrPem, err = p.impl.GenerateIntermediateCSR()
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCServer) SetIntermediate(args *SetIntermediateRPCRequest, _ *struct{}) error {
|
|
|
|
return p.impl.SetIntermediate(args.IntermediatePEM, args.RootPEM)
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCServer) ActiveIntermediate(_ struct{}, resp *ActiveIntermediateResponse) error {
|
|
|
|
var err error
|
|
|
|
resp.CrtPem, err = p.impl.ActiveIntermediate()
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCServer) GenerateIntermediate(_ struct{}, resp *GenerateIntermediateResponse) error {
|
|
|
|
var err error
|
|
|
|
resp.CrtPem, err = p.impl.GenerateIntermediate()
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCServer) Sign(args *SignRequest, resp *SignResponse) error {
|
|
|
|
csr, err := x509.ParseCertificateRequest(args.Csr)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
resp.CrtPem, err = p.impl.Sign(csr)
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCServer) SignIntermediate(args *SignIntermediateRequest, resp *SignIntermediateResponse) error {
|
|
|
|
csr, err := x509.ParseCertificateRequest(args.Csr)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
resp.CrtPem, err = p.impl.SignIntermediate(csr)
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCServer) CrossSignCA(args *CrossSignCARequest, resp *CrossSignCAResponse) error {
|
|
|
|
crt, err := x509.ParseCertificate(args.Crt)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
resp.CrtPem, err = p.impl.CrossSignCA(crt)
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCServer) Cleanup(struct{}, *struct{}) error {
|
|
|
|
return p.impl.Cleanup()
|
|
|
|
}
|
|
|
|
|
|
|
|
// providerPluginRPCClient implements a net/rpc backed transport for
|
|
|
|
// an underlying implementation of a ca.Provider. The client side is the
|
|
|
|
// software calling into the plugin binary over rpc.
|
|
|
|
//
|
|
|
|
// This implements ca.Provider.
|
|
|
|
type providerPluginRPCClient struct {
|
|
|
|
client *rpc.Client
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCClient) Configure(
|
|
|
|
clusterId string,
|
2019-08-27 21:09:01 +00:00
|
|
|
datacenterName string,
|
|
|
|
dnsDomain string,
|
CA Provider Plugins (#4751)
This adds the `agent/connect/ca/plugin` library for consuming/serving Connect CA providers as [go-plugin](https://github.com/hashicorp/go-plugin) plugins. This **does not** wire this up in any way to Consul itself, so this will not enable using these plugins yet.
## Why?
We want to enable CA providers to be pluggable without modifying Consul so that any CA or PKI system can potentially back the Connect certificates. This CA system may also be used in the future for easier bootstrapping and internal cluster security.
### go-plugin
The benefit of `go-plugin` is that for the plugin consumer, the fact that the interface implementation is communicating over multi-process RPC is invisible. Internals of Consul will continue to just use `ca.Provider` interface implementations as if they're local. For plugin _authors_, they simply have to implement the interface. The network/transport/process management issues are handled by go-plugin itself.
The CA provider plugins support both `net/rpc` and gRPC transports. This enables easy authoring in any language. go-plugin handles the actual protocol handshake and connection. This is just a feature of go-plugin.
`go-plugin` is already in production use for years by Packer, Terraform, Nomad, Vault, and Sentinel. We've shown stability for both desktop and server-side software. It is very mature.
## Implementation Details
### `map[string]interface{}`
The `Configure` method passes a `map[string]interface{}`. This map contains only Go primitives and containers of primitives (no funcs, chans, etc.). For `net/rpc` we encode as-is using Gob. For gRPC we marshal to JSON and transmit as a `bytes` type. This is the same approach we take with Vault and other software.
Note that this is just the transport protocol, the end software views it fully decoded.
### `x509.Certificate` and `CertificateRequest`
We transmit the raw ASN.1 bytes and decode on the other side. Unit tests are verifying we get the same cert/csrs across the wire.
### Testing
`go-plugin` exposes test helpers that enable testing the full plugin RPC over real loopback network connections. We test all endpoints for success and error for both `net/rpc` and gRPC.
### Vendoring
This PR doesn't introduce vendoring for two reasons:
1. @banks's `f-envoy` branch introduces a lot of these and I didn't want conflict.
2. The library isn't actually used yet so it doesn't introduce compile-time errors (it does introduce test errors).
## Next Steps
With this in place, we need to figure out the proper way to actually hook these up to Consul, load them, etc. This discussion can happen elsewhere, since regardless of approach this plugin library implementation is the exact same.
2019-01-07 17:48:44 +00:00
|
|
|
isRoot bool,
|
|
|
|
rawConfig map[string]interface{}) error {
|
|
|
|
return p.client.Call("Plugin.Configure", &ConfigureRPCRequest{
|
2019-08-27 21:09:01 +00:00
|
|
|
ClusterId: clusterId,
|
|
|
|
DatacenterName: datacenterName,
|
|
|
|
DNSDomain: dnsDomain,
|
|
|
|
IsRoot: isRoot,
|
|
|
|
RawConfig: rawConfig,
|
CA Provider Plugins (#4751)
This adds the `agent/connect/ca/plugin` library for consuming/serving Connect CA providers as [go-plugin](https://github.com/hashicorp/go-plugin) plugins. This **does not** wire this up in any way to Consul itself, so this will not enable using these plugins yet.
## Why?
We want to enable CA providers to be pluggable without modifying Consul so that any CA or PKI system can potentially back the Connect certificates. This CA system may also be used in the future for easier bootstrapping and internal cluster security.
### go-plugin
The benefit of `go-plugin` is that for the plugin consumer, the fact that the interface implementation is communicating over multi-process RPC is invisible. Internals of Consul will continue to just use `ca.Provider` interface implementations as if they're local. For plugin _authors_, they simply have to implement the interface. The network/transport/process management issues are handled by go-plugin itself.
The CA provider plugins support both `net/rpc` and gRPC transports. This enables easy authoring in any language. go-plugin handles the actual protocol handshake and connection. This is just a feature of go-plugin.
`go-plugin` is already in production use for years by Packer, Terraform, Nomad, Vault, and Sentinel. We've shown stability for both desktop and server-side software. It is very mature.
## Implementation Details
### `map[string]interface{}`
The `Configure` method passes a `map[string]interface{}`. This map contains only Go primitives and containers of primitives (no funcs, chans, etc.). For `net/rpc` we encode as-is using Gob. For gRPC we marshal to JSON and transmit as a `bytes` type. This is the same approach we take with Vault and other software.
Note that this is just the transport protocol, the end software views it fully decoded.
### `x509.Certificate` and `CertificateRequest`
We transmit the raw ASN.1 bytes and decode on the other side. Unit tests are verifying we get the same cert/csrs across the wire.
### Testing
`go-plugin` exposes test helpers that enable testing the full plugin RPC over real loopback network connections. We test all endpoints for success and error for both `net/rpc` and gRPC.
### Vendoring
This PR doesn't introduce vendoring for two reasons:
1. @banks's `f-envoy` branch introduces a lot of these and I didn't want conflict.
2. The library isn't actually used yet so it doesn't introduce compile-time errors (it does introduce test errors).
## Next Steps
With this in place, we need to figure out the proper way to actually hook these up to Consul, load them, etc. This discussion can happen elsewhere, since regardless of approach this plugin library implementation is the exact same.
2019-01-07 17:48:44 +00:00
|
|
|
}, &struct{}{})
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCClient) GenerateRoot() error {
|
|
|
|
return p.client.Call("Plugin.GenerateRoot", struct{}{}, &struct{}{})
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCClient) ActiveRoot() (string, error) {
|
|
|
|
var resp ActiveRootResponse
|
|
|
|
err := p.client.Call("Plugin.ActiveRoot", struct{}{}, &resp)
|
|
|
|
return resp.CrtPem, err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCClient) GenerateIntermediateCSR() (string, error) {
|
|
|
|
var resp GenerateIntermediateCSRResponse
|
|
|
|
err := p.client.Call("Plugin.GenerateIntermediateCSR", struct{}{}, &resp)
|
|
|
|
return resp.CsrPem, err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCClient) SetIntermediate(intermediatePEM, rootPEM string) error {
|
|
|
|
return p.client.Call("Plugin.SetIntermediate", &SetIntermediateRPCRequest{
|
|
|
|
IntermediatePEM: intermediatePEM,
|
|
|
|
RootPEM: rootPEM,
|
|
|
|
}, &struct{}{})
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCClient) ActiveIntermediate() (string, error) {
|
|
|
|
var resp ActiveIntermediateResponse
|
|
|
|
err := p.client.Call("Plugin.ActiveIntermediate", struct{}{}, &resp)
|
|
|
|
return resp.CrtPem, err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCClient) GenerateIntermediate() (string, error) {
|
|
|
|
var resp GenerateIntermediateResponse
|
|
|
|
err := p.client.Call("Plugin.GenerateIntermediate", struct{}{}, &resp)
|
|
|
|
return resp.CrtPem, err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCClient) Sign(csr *x509.CertificateRequest) (string, error) {
|
|
|
|
var resp SignResponse
|
|
|
|
err := p.client.Call("Plugin.Sign", &SignRequest{
|
|
|
|
Csr: csr.Raw,
|
|
|
|
}, &resp)
|
|
|
|
return resp.CrtPem, err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCClient) SignIntermediate(csr *x509.CertificateRequest) (string, error) {
|
|
|
|
var resp SignIntermediateResponse
|
|
|
|
err := p.client.Call("Plugin.SignIntermediate", &SignIntermediateRequest{
|
|
|
|
Csr: csr.Raw,
|
|
|
|
}, &resp)
|
|
|
|
return resp.CrtPem, err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCClient) CrossSignCA(crt *x509.Certificate) (string, error) {
|
|
|
|
var resp CrossSignCAResponse
|
|
|
|
err := p.client.Call("Plugin.CrossSignCA", &CrossSignCARequest{
|
|
|
|
Crt: crt.Raw,
|
|
|
|
}, &resp)
|
|
|
|
return resp.CrtPem, err
|
|
|
|
}
|
|
|
|
|
|
|
|
func (p *providerPluginRPCClient) Cleanup() error {
|
|
|
|
return p.client.Call("Plugin.Cleanup", struct{}{}, &struct{}{})
|
|
|
|
}
|
|
|
|
|
|
|
|
// Verification
|
|
|
|
var _ ca.Provider = &providerPluginRPCClient{}
|
|
|
|
|
|
|
|
//-------------------------------------------------------------------
|
|
|
|
// Structs for net/rpc request and response
|
|
|
|
|
|
|
|
type ConfigureRPCRequest struct {
|
2019-08-27 21:09:01 +00:00
|
|
|
ClusterId string
|
|
|
|
DatacenterName string
|
|
|
|
DNSDomain string
|
|
|
|
IsRoot bool
|
|
|
|
RawConfig map[string]interface{}
|
CA Provider Plugins (#4751)
This adds the `agent/connect/ca/plugin` library for consuming/serving Connect CA providers as [go-plugin](https://github.com/hashicorp/go-plugin) plugins. This **does not** wire this up in any way to Consul itself, so this will not enable using these plugins yet.
## Why?
We want to enable CA providers to be pluggable without modifying Consul so that any CA or PKI system can potentially back the Connect certificates. This CA system may also be used in the future for easier bootstrapping and internal cluster security.
### go-plugin
The benefit of `go-plugin` is that for the plugin consumer, the fact that the interface implementation is communicating over multi-process RPC is invisible. Internals of Consul will continue to just use `ca.Provider` interface implementations as if they're local. For plugin _authors_, they simply have to implement the interface. The network/transport/process management issues are handled by go-plugin itself.
The CA provider plugins support both `net/rpc` and gRPC transports. This enables easy authoring in any language. go-plugin handles the actual protocol handshake and connection. This is just a feature of go-plugin.
`go-plugin` is already in production use for years by Packer, Terraform, Nomad, Vault, and Sentinel. We've shown stability for both desktop and server-side software. It is very mature.
## Implementation Details
### `map[string]interface{}`
The `Configure` method passes a `map[string]interface{}`. This map contains only Go primitives and containers of primitives (no funcs, chans, etc.). For `net/rpc` we encode as-is using Gob. For gRPC we marshal to JSON and transmit as a `bytes` type. This is the same approach we take with Vault and other software.
Note that this is just the transport protocol, the end software views it fully decoded.
### `x509.Certificate` and `CertificateRequest`
We transmit the raw ASN.1 bytes and decode on the other side. Unit tests are verifying we get the same cert/csrs across the wire.
### Testing
`go-plugin` exposes test helpers that enable testing the full plugin RPC over real loopback network connections. We test all endpoints for success and error for both `net/rpc` and gRPC.
### Vendoring
This PR doesn't introduce vendoring for two reasons:
1. @banks's `f-envoy` branch introduces a lot of these and I didn't want conflict.
2. The library isn't actually used yet so it doesn't introduce compile-time errors (it does introduce test errors).
## Next Steps
With this in place, we need to figure out the proper way to actually hook these up to Consul, load them, etc. This discussion can happen elsewhere, since regardless of approach this plugin library implementation is the exact same.
2019-01-07 17:48:44 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
type SetIntermediateRPCRequest struct {
|
|
|
|
IntermediatePEM string
|
|
|
|
RootPEM string
|
|
|
|
}
|