status-im/consul
2
0
Fork 0
mirror of https://github.com/status-im/consul.git synced 2025-02-02 08:56:43 +00:00
Code Issues Projects Releases Wiki Activity

New ACLs (#4791)

Browse Source 
This PR is almost a complete rewrite of the ACL system within Consul. It brings the features more in line with other HashiCorp products. Obviously there is quite a bit left to do here but most of it is related docs, testing and finishing the last few commands in the CLI. I will update the PR description and check off the todos as I finish them over the next few days/week.
Description

At a high level this PR is mainly to split ACL tokens from Policies and to split the concepts of Authorization from Identities. A lot of this PR is mostly just to support CRUD operations on ACLTokens and ACLPolicies. These in and of themselves are not particularly interesting. The bigger conceptual changes are in how tokens get resolved, how backwards compatibility is handled and the separation of policy from identity which could lead the way to allowing for alternative identity providers.

On the surface and with a new cluster the ACL system will look very similar to that of Nomads. Both have tokens and policies. Both have local tokens. The ACL management APIs for both are very similar. I even ripped off Nomad's ACL bootstrap resetting procedure. There are a few key differences though.

    Nomad requires token and policy replication where Consul only requires policy replication with token replication being opt-in. In Consul local tokens only work with token replication being enabled though.
    All policies in Nomad are globally applicable. In Consul all policies are stored and replicated globally but can be scoped to a subset of the datacenters. This allows for more granular access management.
    Unlike Nomad, Consul has legacy baggage in the form of the original ACL system. The ramifications of this are:
        A server running the new system must still support other clients using the legacy system.
        A client running the new system must be able to use the legacy RPCs when the servers in its datacenter are running the legacy system.
        The primary ACL DC's servers running in legacy mode needs to be a gate that keeps everything else in the entire multi-DC cluster running in legacy mode.

So not only does this PR implement the new ACL system but has a legacy mode built in for when the cluster isn't ready for new ACLs. Also detecting that new ACLs can be used is automatic and requires no configuration on the part of administrators. This process is detailed more in the "Transitioning from Legacy to New ACL Mode" section below.
This commit is contained in:
Matt Keeler 2018-10-19 12:04:07 -04:00 committed by GitHub
parent 34d01e6254
commit 18b29c45c4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
165 changed files with 20505 additions and 6182 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
GNUmakefile
View File

@ -117,8 +117,8 @@ dev: changelogfmt vendorfmt dev-build
dev-build:
@$(SHELL) $(CURDIR)/build-support/scripts/build-local.sh -o $(GOOS) -a $(GOARCH)
dev-docker:
@docker build -t '$(CONSUL_DEV_IMAGE)' --build-arg 'GIT_COMMIT=$(GIT_COMMIT)' --build-arg 'GIT_DIRTY=$(GIT_DIRTY)' --build-arg 'GIT_DESCRIBE=$(GIT_DESCRIBE)' -f $(CURDIR)/build-support/docker/Consul-Dev.dockerfile $(CURDIR)
dev-docker: go-build-image
@docker build -t '$(CONSUL_DEV_IMAGE)' --build-arg 'GIT_COMMIT=$(GIT_COMMIT)' --build-arg 'GIT_DIRTY=$(GIT_DIRTY)' --build-arg 'GIT_DESCRIBE=$(GIT_DESCRIBE)' --build-arg 'CONSUL_BUILD_IMAGE=$(GO_BUILD_TAG)' -f $(CURDIR)/build-support/docker/Consul-Dev.dockerfile $(CURDIR)
vendorfmt:
@echo "--> Formatting vendor/vendor.json"
@ -138,7 +138,7 @@ dist:
@$(SHELL) $(CURDIR)/build-support/scripts/release.sh -t '$(DIST_TAG)' -b '$(DIST_BUILD)' -S '$(DIST_SIGN)' $(DIST_VERSION_ARG) $(DIST_DATE_ARG) $(DIST_REL_ARG)
verify:
@$(SHELL) $(CURDIR)/build-support/scripts/verify.sh
@$(SHELL) $(CURDIR)/build-support/scripts/verify.sh
publish:
@$(SHELL) $(CURDIR)/build-support/scripts/publish.sh $(PUB_GIT_ARG) $(PUB_WEBSITE_ARG)
@ -243,34 +243,34 @@ version:
@$(SHELL) $(CURDIR)/build-support/scripts/version.sh -g
@echo -n "Version + release + git: "
@$(SHELL) $(CURDIR)/build-support/scripts/version.sh -r -g
docker-images: go-build-image ui-build-image ui-legacy-build-image
go-build-image:
@echo "Building Golang build container"
@docker build $(NOCACHE) $(QUIET) --build-arg 'GOTOOLS=$(GOTOOLS)' -t $(GO_BUILD_TAG) - < build-support/docker/Build-Go.dockerfile
ui-build-image:
@echo "Building UI build container"
@docker build $(NOCACHE) $(QUIET) -t $(UI_BUILD_TAG) - < build-support/docker/Build-UI.dockerfile
ui-legacy-build-image:
@echo "Building Legacy UI build container"
@docker build $(NOCACHE) $(QUIET) -t $(UI_LEGACY_BUILD_TAG) - < build-support/docker/Build-UI-Legacy.dockerfile
static-assets-docker: go-build-image
@$(SHELL) $(CURDIR)/build-support/scripts/build-docker.sh static-assets
consul-docker: go-build-image
@$(SHELL) $(CURDIR)/build-support/scripts/build-docker.sh consul
ui-docker: ui-build-image
@$(SHELL) $(CURDIR)/build-support/scripts/build-docker.sh ui
ui-legacy-docker: ui-legacy-build-image
@$(SHELL) $(CURDIR)/build-support/scripts/build-docker.sh ui-legacy
.PHONY: all ci bin dev dist cov test test-ci test-internal test-install-deps cover format vet ui static-assets tools vendorfmt
.PHONY: docker-images go-build-image ui-build-image ui-legacy-build-image static-assets-docker consul-docker ui-docker ui-legacy-docker version

466
acl/acl.go
View File

@ -8,14 +8,14 @@ import (
var (
// allowAll is a singleton policy which allows all
// non-management actions
allowAll ACL
allowAll Authorizer
// denyAll is a singleton policy which denies all actions
denyAll ACL
denyAll Authorizer
// manageAll is a singleton policy which allows all
// actions, including management
manageAll ACL
manageAll Authorizer
)
// DefaultPolicyEnforcementLevel will be used if the user leaves the level
@ -24,27 +24,27 @@ const DefaultPolicyEnforcementLevel = "hard-mandatory"
func init() {
// Setup the singletons
allowAll = &StaticACL{
allowAll = &StaticAuthorizer{
allowManage: false,
defaultAllow: true,
}
denyAll = &StaticACL{
denyAll = &StaticAuthorizer{
allowManage: false,
defaultAllow: false,
}
manageAll = &StaticACL{
manageAll = &StaticAuthorizer{
allowManage: true,
defaultAllow: true,
}
}
// ACL is the interface for policy enforcement.
type ACL interface {
// ACLList checks for permission to list all the ACLs
ACLList() bool
// Authorizer is the interface for policy enforcement.
type Authorizer interface {
// ACLRead checks for permission to list all the ACLs
ACLRead() bool
// ACLModify checks for permission to manipulate ACLs
ACLModify() bool
// ACLWrite checks for permission to manipulate ACLs
ACLWrite() bool
// AgentRead checks for permission to read from agent endpoints for a
// given node.
@ -133,135 +133,135 @@ type ACL interface {
Snapshot() bool
}
// StaticACL is used to implement a base ACL policy. It either
// StaticAuthorizer is used to implement a base ACL policy. It either
// allows or denies all requests. This can be used as a parent
// ACL to act in a blacklist or whitelist mode.
type StaticACL struct {
type StaticAuthorizer struct {
allowManage bool
defaultAllow bool
}
func (s *StaticACL) ACLList() bool {
func (s *StaticAuthorizer) ACLRead() bool {
return s.allowManage
}
func (s *StaticACL) ACLModify() bool {
func (s *StaticAuthorizer) ACLWrite() bool {
return s.allowManage
}
func (s *StaticACL) AgentRead(string) bool {
func (s *StaticAuthorizer) AgentRead(string) bool {
return s.defaultAllow
}
func (s *StaticACL) AgentWrite(string) bool {
func (s *StaticAuthorizer) AgentWrite(string) bool {
return s.defaultAllow
}
func (s *StaticACL) EventRead(string) bool {
func (s *StaticAuthorizer) EventRead(string) bool {
return s.defaultAllow
}
func (s *StaticACL) EventWrite(string) bool {
func (s *StaticAuthorizer) EventWrite(string) bool {
return s.defaultAllow
}
func (s *StaticACL) IntentionDefaultAllow() bool {
func (s *StaticAuthorizer) IntentionDefaultAllow() bool {
return s.defaultAllow
}
func (s *StaticACL) IntentionRead(string) bool {
func (s *StaticAuthorizer) IntentionRead(string) bool {
return s.defaultAllow
}
func (s *StaticACL) IntentionWrite(string) bool {
func (s *StaticAuthorizer) IntentionWrite(string) bool {
return s.defaultAllow
}
func (s *StaticACL) KeyRead(string) bool {
func (s *StaticAuthorizer) KeyRead(string) bool {
return s.defaultAllow
}
func (s *StaticACL) KeyList(string) bool {
func (s *StaticAuthorizer) KeyList(string) bool {
return s.defaultAllow
}
func (s *StaticACL) KeyWrite(string, sentinel.ScopeFn) bool {
func (s *StaticAuthorizer) KeyWrite(string, sentinel.ScopeFn) bool {
return s.defaultAllow
}
func (s *StaticACL) KeyWritePrefix(string) bool {
func (s *StaticAuthorizer) KeyWritePrefix(string) bool {
return s.defaultAllow
}
func (s *StaticACL) KeyringRead() bool {
func (s *StaticAuthorizer) KeyringRead() bool {
return s.defaultAllow
}
func (s *StaticACL) KeyringWrite() bool {
func (s *StaticAuthorizer) KeyringWrite() bool {
return s.defaultAllow
}
func (s *StaticACL) NodeRead(string) bool {
func (s *StaticAuthorizer) NodeRead(string) bool {
return s.defaultAllow
}
func (s *StaticACL) NodeWrite(string, sentinel.ScopeFn) bool {
func (s *StaticAuthorizer) NodeWrite(string, sentinel.ScopeFn) bool {
return s.defaultAllow
}
func (s *StaticACL) OperatorRead() bool {
func (s *StaticAuthorizer) OperatorRead() bool {
return s.defaultAllow
}
func (s *StaticACL) OperatorWrite() bool {
func (s *StaticAuthorizer) OperatorWrite() bool {
return s.defaultAllow
}
func (s *StaticACL) PreparedQueryRead(string) bool {
func (s *StaticAuthorizer) PreparedQueryRead(string) bool {
return s.defaultAllow
}
func (s *StaticACL) PreparedQueryWrite(string) bool {
func (s *StaticAuthorizer) PreparedQueryWrite(string) bool {
return s.defaultAllow
}
func (s *StaticACL) ServiceRead(string) bool {
func (s *StaticAuthorizer) ServiceRead(string) bool {
return s.defaultAllow
}
func (s *StaticACL) ServiceWrite(string, sentinel.ScopeFn) bool {
func (s *StaticAuthorizer) ServiceWrite(string, sentinel.ScopeFn) bool {
return s.defaultAllow
}
func (s *StaticACL) SessionRead(string) bool {
func (s *StaticAuthorizer) SessionRead(string) bool {
return s.defaultAllow
}
func (s *StaticACL) SessionWrite(string) bool {
func (s *StaticAuthorizer) SessionWrite(string) bool {
return s.defaultAllow
}
func (s *StaticACL) Snapshot() bool {
func (s *StaticAuthorizer) Snapshot() bool {
return s.allowManage
}
// AllowAll returns an ACL rule that allows all operations
func AllowAll() ACL {
// AllowAll returns an Authorizer that allows all operations
func AllowAll() Authorizer {
return allowAll
}
// DenyAll returns an ACL rule that denies all operations
func DenyAll() ACL {
// DenyAll returns an Authorizer that denies all operations
func DenyAll() Authorizer {
return denyAll
}
// ManageAll returns an ACL rule that can manage all resources
func ManageAll() ACL {
// ManageAll returns an Authorizer that can manage all resources
func ManageAll() Authorizer {
return manageAll
}
// RootACL returns a possible ACL if the ID matches a root policy
func RootACL(id string) ACL {
// RootAuthorizer returns a possible Authorizer if the ID matches a root policy
func RootAuthorizer(id string) Authorizer {
switch id {
case "allow":
return allowAll
@ -274,9 +274,9 @@ func RootACL(id string) ACL {
}
}
// PolicyRule binds a regular ACL policy along with an optional piece of
// RulePolicy binds a regular ACL policy along with an optional piece of
// code to execute.
type PolicyRule struct {
type RulePolicy struct {
// aclPolicy is used for simple acl rules(allow/deny/manage)
aclPolicy string
@ -284,39 +284,43 @@ type PolicyRule struct {
sentinelPolicy Sentinel
}
// PolicyACL is used to wrap a set of ACL policies to provide
// the ACL interface.
type PolicyACL struct {
// PolicyAuthorizer is used to wrap a set of ACL policies to provide
// the Authorizer interface.
//
type PolicyAuthorizer struct {
// parent is used to resolve policy if we have
// no matching rule.
parent ACL
parent Authorizer
// sentinel is an interface for validating and executing sentinel code
// policies.
sentinel sentinel.Evaluator
// agentRules contains the agent policies
// aclRule contains the acl management policy.
aclRule string
// agentRules contain the exact-match agent policies
agentRules *radix.Tree
// intentionRules contains the service intention policies
// intentionRules contains the service intention exact-match policies
intentionRules *radix.Tree
// keyRules contains the key policies
// keyRules contains the key exact-match policies
keyRules *radix.Tree
// nodeRules contains the node policies
// nodeRules contains the node exact-match policies
nodeRules *radix.Tree
// serviceRules contains the service policies
// serviceRules contains the service exact-match policies
serviceRules *radix.Tree
// sessionRules contains the session policies
// sessionRules contains the session exact-match policies
sessionRules *radix.Tree
// eventRules contains the user event policies
// eventRules contains the user event exact-match policies
eventRules *radix.Tree
// preparedQueryRules contains the prepared query policies
// preparedQueryRules contains the prepared query exact-match policies
preparedQueryRules *radix.Tree
// keyringRule contains the keyring policies. The keyring has
@ -328,6 +332,52 @@ type PolicyACL struct {
operatorRule string
}
// policyAuthorizerRadixLeaf is used as the main
// structure for storing in the radix.Tree's within the
// PolicyAuthorizer
type policyAuthorizerRadixLeaf struct {
exact interface{}
prefix interface{}
}
// getPolicy first attempts to get an exact match for the segment from the "exact" tree and then falls
// back to getting the policy for the longest prefix from the "prefix" tree
func getPolicy(segment string, tree *radix.Tree) (policy interface{}, found bool) {
found = false
tree.WalkPath(segment, func(path string, leaf interface{}) bool {
policies := leaf.(*policyAuthorizerRadixLeaf)
if policies.exact != nil && path == segment {
found = true
policy = policies.exact
return true
}
if policies.prefix != nil {
found = true
policy = policies.prefix
}
return false
})
return
}
func insertPolicyIntoRadix(segment string, tree *radix.Tree, exactPolicy interface{}, prefixPolicy interface{}) {
leaf, found := tree.Get(segment)
if found {
policy := leaf.(*policyAuthorizerRadixLeaf)
if exactPolicy != nil {
policy.exact = exactPolicy
}
if prefixPolicy != nil {
policy.prefix = prefixPolicy
}
} else {
policy := &policyAuthorizerRadixLeaf{exact: exactPolicy, prefix: prefixPolicy}
tree.Insert(segment, policy)
}
}
func enforce(rule string, requiredPermission string) (allow, recurse bool) {
switch rule {
case PolicyWrite:
@ -356,10 +406,10 @@ func enforce(rule string, requiredPermission string) (allow, recurse bool) {
}
}
// New is used to construct a policy based ACL from a set of policies
// NewPolicyAuthorizer is used to construct a policy based ACL from a set of policies
// and a parent policy to resolve missing cases.
func New(parent ACL, policy *Policy, sentinel sentinel.Evaluator) (*PolicyACL, error) {
p := &PolicyACL{
func NewPolicyAuthorizer(parent Authorizer, policies []*Policy, sentinel sentinel.Evaluator) (*PolicyAuthorizer, error) {
p := &PolicyAuthorizer{
parent: parent,
agentRules: radix.New(),
intentionRules: radix.New(),
@ -372,40 +422,62 @@ func New(parent ACL, policy *Policy, sentinel sentinel.Evaluator) (*PolicyACL, e
sentinel: sentinel,
}
// Load the agent policy
policy := MergePolicies(policies)
// Load the agent policy (exact matches)
for _, ap := range policy.Agents {
p.agentRules.Insert(ap.Node, ap.Policy)
insertPolicyIntoRadix(ap.Node, p.agentRules, ap.Policy, nil)
}
// Load the key policy
// Load the agent policy (prefix matches)
for _, ap := range policy.AgentPrefixes {
insertPolicyIntoRadix(ap.Node, p.agentRules, nil, ap.Policy)
}
// Load the key policy (exact matches)
for _, kp := range policy.Keys {
policyRule := PolicyRule{
policyRule := RulePolicy{
aclPolicy: kp.Policy,
sentinelPolicy: kp.Sentinel,
}
p.keyRules.Insert(kp.Prefix, policyRule)
insertPolicyIntoRadix(kp.Prefix, p.keyRules, policyRule, nil)
}
// Load the node policy
// Load the key policy (prefix matches)
for _, kp := range policy.KeyPrefixes {
policyRule := RulePolicy{
aclPolicy: kp.Policy,
sentinelPolicy: kp.Sentinel,
}
insertPolicyIntoRadix(kp.Prefix, p.keyRules, nil, policyRule)
}
// Load the node policy (exact matches)
for _, np := range policy.Nodes {
policyRule := PolicyRule{
policyRule := RulePolicy{
aclPolicy: np.Policy,
sentinelPolicy: np.Sentinel,
}
p.nodeRules.Insert(np.Name, policyRule)
insertPolicyIntoRadix(np.Name, p.nodeRules, policyRule, nil)
}
// Load the service policy
// Load the node policy (prefix matches)
for _, np := range policy.NodePrefixes {
policyRule := RulePolicy{
aclPolicy: np.Policy,
sentinelPolicy: np.Sentinel,
}
insertPolicyIntoRadix(np.Name, p.nodeRules, nil, policyRule)
}
// Load the service policy (exact matches)
for _, sp := range policy.Services {
policyRule := PolicyRule{
policyRule := RulePolicy{
aclPolicy: sp.Policy,
sentinelPolicy: sp.Sentinel,
}
p.serviceRules.Insert(sp.Name, policyRule)
insertPolicyIntoRadix(sp.Name, p.serviceRules, policyRule, nil)
// Determine the intention. The intention could be blank (not set).
// If the intention is not set, the value depends on the value of
// the service policy.
intention := sp.Intentions
if intention == "" {
switch sp.Policy {
@ -416,28 +488,71 @@ func New(parent ACL, policy *Policy, sentinel sentinel.Evaluator) (*PolicyACL, e
}
}
policyRule = PolicyRule{
policyRule = RulePolicy{
aclPolicy: intention,
sentinelPolicy: sp.Sentinel,
}
p.intentionRules.Insert(sp.Name, policyRule)
insertPolicyIntoRadix(sp.Name, p.intentionRules, policyRule, nil)
}
// Load the session policy
// Load the service policy (prefix matches)
for _, sp := range policy.ServicePrefixes {
policyRule := RulePolicy{
aclPolicy: sp.Policy,
sentinelPolicy: sp.Sentinel,
}
insertPolicyIntoRadix(sp.Name, p.serviceRules, nil, policyRule)
intention := sp.Intentions
if intention == "" {
switch sp.Policy {
case PolicyRead, PolicyWrite:
intention = PolicyRead
default:
intention = PolicyDeny
}
}
policyRule = RulePolicy{
aclPolicy: intention,
sentinelPolicy: sp.Sentinel,
}
insertPolicyIntoRadix(sp.Name, p.intentionRules, nil, policyRule)
}
// Load the session policy (exact matches)
for _, sp := range policy.Sessions {
p.sessionRules.Insert(sp.Node, sp.Policy)
insertPolicyIntoRadix(sp.Node, p.sessionRules, sp.Policy, nil)
}
// Load the event policy
// Load the session policy (prefix matches)
for _, sp := range policy.SessionPrefixes {
insertPolicyIntoRadix(sp.Node, p.sessionRules, nil, sp.Policy)
}
// Load the event policy (exact matches)
for _, ep := range policy.Events {
p.eventRules.Insert(ep.Event, ep.Policy)
insertPolicyIntoRadix(ep.Event, p.eventRules, ep.Policy, nil)
}
// Load the prepared query policy
for _, pq := range policy.PreparedQueries {
p.preparedQueryRules.Insert(pq.Prefix, pq.Policy)
// Load the event policy (prefix matches)
for _, ep := range policy.EventPrefixes {
insertPolicyIntoRadix(ep.Event, p.eventRules, nil, ep.Policy)
}
// Load the prepared query policy (exact matches)
for _, qp := range policy.PreparedQueries {
insertPolicyIntoRadix(qp.Prefix, p.preparedQueryRules, qp.Policy, nil)
}
// Load the prepared query policy (prefix matches)
for _, qp := range policy.PreparedQueryPrefixes {
insertPolicyIntoRadix(qp.Prefix, p.preparedQueryRules, nil, qp.Policy)
}
// Load the acl policy
p.aclRule = policy.ACL
// Load the keyring policy
p.keyringRule = policy.Keyring
@ -447,21 +562,29 @@ func New(parent ACL, policy *Policy, sentinel sentinel.Evaluator) (*PolicyACL, e
return p, nil
}
// ACLList checks if listing of ACLs is allowed
func (p *PolicyACL) ACLList() bool {
return p.parent.ACLList()
// ACLRead checks if listing of ACLs is allowed
func (p *PolicyAuthorizer) ACLRead() bool {
if allow, recurse := enforce(p.aclRule, PolicyRead); !recurse {
return allow
}
return p.parent.ACLRead()
}
// ACLModify checks if modification of ACLs is allowed
func (p *PolicyACL) ACLModify() bool {
return p.parent.ACLModify()
// ACLWrite checks if modification of ACLs is allowed
func (p *PolicyAuthorizer) ACLWrite() bool {
if allow, recurse := enforce(p.aclRule, PolicyWrite); !recurse {
return allow
}
return p.parent.ACLWrite()
}
// AgentRead checks for permission to read from agent endpoints for a given
// node.
func (p *PolicyACL) AgentRead(node string) bool {
func (p *PolicyAuthorizer) AgentRead(node string) bool {
// Check for an exact rule or catch-all
if _, rule, ok := p.agentRules.LongestPrefix(node); ok {
if rule, ok := getPolicy(node, p.agentRules); ok {
if allow, recurse := enforce(rule.(string), PolicyRead); !recurse {
return allow
}
@ -473,11 +596,9 @@ func (p *PolicyACL) AgentRead(node string) bool {
// AgentWrite checks for permission to make changes via agent endpoints for a
// given node.
func (p *PolicyACL) AgentWrite(node string) bool {
func (p *PolicyAuthorizer) AgentWrite(node string) bool {
// Check for an exact rule or catch-all
_, rule, ok := p.agentRules.LongestPrefix(node)
if ok {
if rule, ok := getPolicy(node, p.agentRules); ok {
if allow, recurse := enforce(rule.(string), PolicyWrite); !recurse {
return allow
}
@ -488,15 +609,18 @@ func (p *PolicyACL) AgentWrite(node string) bool {
}
// Snapshot checks if taking and restoring snapshots is allowed.
func (p *PolicyACL) Snapshot() bool {
func (p *PolicyAuthorizer) Snapshot() bool {
if allow, recurse := enforce(p.aclRule, PolicyWrite); !recurse {
return allow
}
return p.parent.Snapshot()
}
// EventRead is used to determine if the policy allows for a
// specific user event to be read.
func (p *PolicyACL) EventRead(name string) bool {
func (p *PolicyAuthorizer) EventRead(name string) bool {
// Longest-prefix match on event names
if _, rule, ok := p.eventRules.LongestPrefix(name); ok {
if rule, ok := getPolicy(name, p.eventRules); ok {
if allow, recurse := enforce(rule.(string), PolicyRead); !recurse {
return allow
}
@ -508,9 +632,9 @@ func (p *PolicyACL) EventRead(name string) bool {
// EventWrite is used to determine if new events can be created
// (fired) by the policy.
func (p *PolicyACL) EventWrite(name string) bool {
func (p *PolicyAuthorizer) EventWrite(name string) bool {
// Longest-prefix match event names
if _, rule, ok := p.eventRules.LongestPrefix(name); ok {
if rule, ok := getPolicy(name, p.eventRules); ok {
if allow, recurse := enforce(rule.(string), PolicyWrite); !recurse {
return allow
}
@ -522,17 +646,17 @@ func (p *PolicyACL) EventWrite(name string) bool {
// IntentionDefaultAllow returns whether the default behavior when there are
// no matching intentions is to allow or deny.
func (p *PolicyACL) IntentionDefaultAllow() bool {
func (p *PolicyAuthorizer) IntentionDefaultAllow() bool {
// We always go up, this can't be determined by a policy.
return p.parent.IntentionDefaultAllow()
}
// IntentionRead checks if writing (creating, updating, or deleting) of an
// intention is allowed.
func (p *PolicyACL) IntentionRead(prefix string) bool {
func (p *PolicyAuthorizer) IntentionRead(prefix string) bool {
// Check for an exact rule or catch-all
if _, rule, ok := p.intentionRules.LongestPrefix(prefix); ok {
pr := rule.(PolicyRule)
if rule, ok := getPolicy(prefix, p.intentionRules); ok {
pr := rule.(RulePolicy)
if allow, recurse := enforce(pr.aclPolicy, PolicyRead); !recurse {
return allow
}
@ -544,11 +668,12 @@ func (p *PolicyACL) IntentionRead(prefix string) bool {
// IntentionWrite checks if writing (creating, updating, or deleting) of an
// intention is allowed.
func (p *PolicyACL) IntentionWrite(prefix string) bool {
func (p *PolicyAuthorizer) IntentionWrite(prefix string) bool {
// Check for an exact rule or catch-all
if _, rule, ok := p.intentionRules.LongestPrefix(prefix); ok {
pr := rule.(PolicyRule)
if rule, ok := getPolicy(prefix, p.intentionRules); ok {
pr := rule.(RulePolicy)
if allow, recurse := enforce(pr.aclPolicy, PolicyWrite); !recurse {
// TODO (ACL-V2) - should we do sentinel enforcement here
return allow
}
}
@ -558,10 +683,10 @@ func (p *PolicyACL) IntentionWrite(prefix string) bool {
}
// KeyRead returns if a key is allowed to be read
func (p *PolicyACL) KeyRead(key string) bool {
func (p *PolicyAuthorizer) KeyRead(key string) bool {
// Look for a matching rule
if _, rule, ok := p.keyRules.LongestPrefix(key); ok {
pr := rule.(PolicyRule)
if rule, ok := getPolicy(key, p.keyRules); ok {
pr := rule.(RulePolicy)
if allow, recurse := enforce(pr.aclPolicy, PolicyRead); !recurse {
return allow
}
@ -572,10 +697,10 @@ func (p *PolicyACL) KeyRead(key string) bool {
}
// KeyList returns if a key is allowed to be listed
func (p *PolicyACL) KeyList(key string) bool {
func (p *PolicyAuthorizer) KeyList(key string) bool {
// Look for a matching rule
if _, rule, ok := p.keyRules.LongestPrefix(key); ok {
pr := rule.(PolicyRule)
if rule, ok := getPolicy(key, p.keyRules); ok {
pr := rule.(RulePolicy)
if allow, recurse := enforce(pr.aclPolicy, PolicyList); !recurse {
return allow
}
@ -586,10 +711,10 @@ func (p *PolicyACL) KeyList(key string) bool {
}
// KeyWrite returns if a key is allowed to be written
func (p *PolicyACL) KeyWrite(key string, scope sentinel.ScopeFn) bool {
func (p *PolicyAuthorizer) KeyWrite(key string, scope sentinel.ScopeFn) bool {
// Look for a matching rule
if _, rule, ok := p.keyRules.LongestPrefix(key); ok {
pr := rule.(PolicyRule)
if rule, ok := getPolicy(key, p.keyRules); ok {
pr := rule.(RulePolicy)
if allow, recurse := enforce(pr.aclPolicy, PolicyWrite); !recurse {
if allow {
return p.executeCodePolicy(&pr.sentinelPolicy, scope)
@ -603,21 +728,53 @@ func (p *PolicyACL) KeyWrite(key string, scope sentinel.ScopeFn) bool {
}
// KeyWritePrefix returns if a prefix is allowed to be written
func (p *PolicyACL) KeyWritePrefix(prefix string) bool {
//
// This is mainly used to detect whether a whole tree within
// the KV can be removed. For that reason we must be able to
// delete everything under the prefix. First we must have "write"
// on the prefix itself
func (p *PolicyAuthorizer) KeyWritePrefix(prefix string) bool {
// Look for a matching rule that denies
_, rule, ok := p.keyRules.LongestPrefix(prefix)
if ok && rule.(PolicyRule).aclPolicy != PolicyWrite {
prefixAllowed := true
found := false
// Look for a prefix rule that would apply to the prefix we are checking
// WalkPath starts at the root and walks down to the given prefix.
// Therefore the last prefix rule we see is the one that matters
p.keyRules.WalkPath(prefix, func(path string, leaf interface{}) bool {
rule := leaf.(*policyAuthorizerRadixLeaf)
if rule.prefix != nil {
found = true
if rule.prefix.(RulePolicy).aclPolicy != PolicyWrite {
prefixAllowed = false
} else {
prefixAllowed = true
}
}
return false
})
if !prefixAllowed {
return false
}
// Look if any of our children have a deny policy
// Look if any of our children do not allow write access. This loop takes
// into account both prefix and exact match rules.
deny := false
p.keyRules.WalkPrefix(prefix, func(path string, rule interface{}) bool {
// We have a rule to prevent a write in a sub-directory!
if rule.(PolicyRule).aclPolicy != PolicyWrite {
p.keyRules.WalkPrefix(prefix, func(path string, leaf interface{}) bool {
found = true
rule := leaf.(*policyAuthorizerRadixLeaf)
if rule.prefix != nil && rule.prefix.(RulePolicy).aclPolicy != PolicyWrite {
deny = true
return true
}
if rule.exact != nil && rule.exact.(RulePolicy).aclPolicy != PolicyWrite {
deny = true
return true
}
return false
})
@ -627,7 +784,7 @@ func (p *PolicyACL) KeyWritePrefix(prefix string) bool {
}
// If we had a matching rule, done
if ok {
if found {
return true
}
@ -637,7 +794,7 @@ func (p *PolicyACL) KeyWritePrefix(prefix string) bool {
// KeyringRead is used to determine if the keyring can be
// read by the current ACL token.
func (p *PolicyACL) KeyringRead() bool {
func (p *PolicyAuthorizer) KeyringRead() bool {
if allow, recurse := enforce(p.keyringRule, PolicyRead); !recurse {
return allow
}
@ -646,7 +803,7 @@ func (p *PolicyACL) KeyringRead() bool {
}
// KeyringWrite determines if the keyring can be manipulated.
func (p *PolicyACL) KeyringWrite() bool {
func (p *PolicyAuthorizer) KeyringWrite() bool {
if allow, recurse := enforce(p.keyringRule, PolicyWrite); !recurse {
return allow
}
@ -655,7 +812,7 @@ func (p *PolicyACL) KeyringWrite() bool {
}
// OperatorRead determines if the read-only operator functions are allowed.
func (p *PolicyACL) OperatorRead() bool {
func (p *PolicyAuthorizer) OperatorRead() bool {
if allow, recurse := enforce(p.operatorRule, PolicyRead); !recurse {
return allow
}
@ -665,7 +822,7 @@ func (p *PolicyACL) OperatorRead() bool {
// OperatorWrite determines if the state-changing operator functions are
// allowed.
func (p *PolicyACL) OperatorWrite() bool {
func (p *PolicyAuthorizer) OperatorWrite() bool {
if allow, recurse := enforce(p.operatorRule, PolicyWrite); !recurse {
return allow
}
@ -674,11 +831,12 @@ func (p *PolicyACL) OperatorWrite() bool {
}
// NodeRead checks if reading (discovery) of a node is allowed
func (p *PolicyACL) NodeRead(name string) bool {
func (p *PolicyAuthorizer) NodeRead(name string) bool {
// Check for an exact rule or catch-all
if _, rule, ok := p.nodeRules.LongestPrefix(name); ok {
pr := rule.(PolicyRule)
if rule, ok := getPolicy(name, p.nodeRules); ok {
pr := rule.(RulePolicy)
if allow, recurse := enforce(pr.aclPolicy, PolicyRead); !recurse {
// TODO (ACL-V2) - Should we do sentinel enforcement here
return allow
}
}
@ -688,10 +846,10 @@ func (p *PolicyACL) NodeRead(name string) bool {
}
// NodeWrite checks if writing (registering) a node is allowed
func (p *PolicyACL) NodeWrite(name string, scope sentinel.ScopeFn) bool {
func (p *PolicyAuthorizer) NodeWrite(name string, scope sentinel.ScopeFn) bool {
// Check for an exact rule or catch-all
if _, rule, ok := p.nodeRules.LongestPrefix(name); ok {
pr := rule.(PolicyRule)
if rule, ok := getPolicy(name, p.nodeRules); ok {
pr := rule.(RulePolicy)
if allow, recurse := enforce(pr.aclPolicy, PolicyWrite); !recurse {
return allow
}
@ -703,9 +861,9 @@ func (p *PolicyACL) NodeWrite(name string, scope sentinel.ScopeFn) bool {
// PreparedQueryRead checks if reading (listing) of a prepared query is
// allowed - this isn't execution, just listing its contents.
func (p *PolicyACL) PreparedQueryRead(prefix string) bool {
func (p *PolicyAuthorizer) PreparedQueryRead(prefix string) bool {
// Check for an exact rule or catch-all
if _, rule, ok := p.preparedQueryRules.LongestPrefix(prefix); ok {
if rule, ok := getPolicy(prefix, p.preparedQueryRules); ok {
if allow, recurse := enforce(rule.(string), PolicyRead); !recurse {
return allow
}
@ -717,9 +875,9 @@ func (p *PolicyACL) PreparedQueryRead(prefix string) bool {
// PreparedQueryWrite checks if writing (creating, updating, or deleting) of a
// prepared query is allowed.
func (p *PolicyACL) PreparedQueryWrite(prefix string) bool {
func (p *PolicyAuthorizer) PreparedQueryWrite(prefix string) bool {
// Check for an exact rule or catch-all
if _, rule, ok := p.preparedQueryRules.LongestPrefix(prefix); ok {
if rule, ok := getPolicy(prefix, p.preparedQueryRules); ok {
if allow, recurse := enforce(rule.(string), PolicyWrite); !recurse {
return allow
}
@ -730,10 +888,10 @@ func (p *PolicyACL) PreparedQueryWrite(prefix string) bool {
}
// ServiceRead checks if reading (discovery) of a service is allowed
func (p *PolicyACL) ServiceRead(name string) bool {
func (p *PolicyAuthorizer) ServiceRead(name string) bool {
// Check for an exact rule or catch-all
if _, rule, ok := p.serviceRules.LongestPrefix(name); ok {
pr := rule.(PolicyRule)
if rule, ok := getPolicy(name, p.serviceRules); ok {
pr := rule.(RulePolicy)
if allow, recurse := enforce(pr.aclPolicy, PolicyRead); !recurse {
return allow
}
@ -744,10 +902,10 @@ func (p *PolicyACL) ServiceRead(name string) bool {
}
// ServiceWrite checks if writing (registering) a service is allowed
func (p *PolicyACL) ServiceWrite(name string, scope sentinel.ScopeFn) bool {
func (p *PolicyAuthorizer) ServiceWrite(name string, scope sentinel.ScopeFn) bool {
// Check for an exact rule or catch-all
if _, rule, ok := p.serviceRules.LongestPrefix(name); ok {
pr := rule.(PolicyRule)
if rule, ok := getPolicy(name, p.serviceRules); ok {
pr := rule.(RulePolicy)
if allow, recurse := enforce(pr.aclPolicy, PolicyWrite); !recurse {
return allow
}
@ -758,9 +916,9 @@ func (p *PolicyACL) ServiceWrite(name string, scope sentinel.ScopeFn) bool {
}
// SessionRead checks for permission to read sessions for a given node.
func (p *PolicyACL) SessionRead(node string) bool {
func (p *PolicyAuthorizer) SessionRead(node string) bool {
// Check for an exact rule or catch-all
if _, rule, ok := p.sessionRules.LongestPrefix(node); ok {
if rule, ok := getPolicy(node, p.sessionRules); ok {
if allow, recurse := enforce(rule.(string), PolicyRead); !recurse {
return allow
}
@ -771,9 +929,9 @@ func (p *PolicyACL) SessionRead(node string) bool {
}
// SessionWrite checks for permission to create sessions for a given node.
func (p *PolicyACL) SessionWrite(node string) bool {
func (p *PolicyAuthorizer) SessionWrite(node string) bool {
// Check for an exact rule or catch-all
if _, rule, ok := p.sessionRules.LongestPrefix(node); ok {
if rule, ok := getPolicy(node, p.sessionRules); ok {
if allow, recurse := enforce(rule.(string), PolicyWrite); !recurse {
return allow
}
@ -785,7 +943,7 @@ func (p *PolicyACL) SessionWrite(node string) bool {
// executeCodePolicy will run the associated code policy if code policies are
// enabled.
func (p *PolicyACL) executeCodePolicy(policy *Sentinel, scope sentinel.ScopeFn) bool {
func (p *PolicyAuthorizer) executeCodePolicy(policy *Sentinel, scope sentinel.ScopeFn) bool {
if p.sentinel == nil {
return true
}

648
acl/acl_test.go
View File

@ -7,230 +7,251 @@ import (
"github.com/stretchr/testify/require"
)
func legacyPolicy(policy *Policy) *Policy {
return &Policy{
Agents: policy.Agents,
AgentPrefixes: policy.Agents,
Nodes: policy.Nodes,
NodePrefixes: policy.Nodes,
Keys: policy.Keys,
KeyPrefixes: policy.Keys,
Services: policy.Services,
ServicePrefixes: policy.Services,
Sessions: policy.Sessions,
SessionPrefixes: policy.Sessions,
Events: policy.Events,
EventPrefixes: policy.Events,
PreparedQueries: policy.PreparedQueries,
PreparedQueryPrefixes: policy.PreparedQueries,
Keyring: policy.Keyring,
Operator: policy.Operator,
}
}
//
// The following 1 line functions are created to all conform to what
// can be stored in the aclCheck type to make defining ACL tests
// nicer in the embedded struct within TestACL
//
func checkAllowACLList(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.ACLList())
func checkAllowACLRead(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.ACLRead())
}
func checkAllowACLModify(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.ACLModify())
func checkAllowACLWrite(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.ACLWrite())
}
func checkAllowAgentRead(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.AgentRead(prefix))
func checkAllowAgentRead(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.AgentRead(prefix))
}
func checkAllowAgentWrite(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.AgentWrite(prefix))
func checkAllowAgentWrite(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.AgentWrite(prefix))
}
func checkAllowEventRead(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.EventRead(prefix))
func checkAllowEventRead(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.EventRead(prefix))
}
func checkAllowEventWrite(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.EventWrite(prefix))
func checkAllowEventWrite(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.EventWrite(prefix))
}
func checkAllowIntentionDefaultAllow(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.IntentionDefaultAllow())
func checkAllowIntentionDefaultAllow(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.IntentionDefaultAllow())
}
func checkAllowIntentionRead(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.IntentionRead(prefix))
func checkAllowIntentionRead(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.IntentionRead(prefix))
}
func checkAllowIntentionWrite(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.IntentionWrite(prefix))
func checkAllowIntentionWrite(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.IntentionWrite(prefix))
}
func checkAllowKeyRead(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.KeyRead(prefix))
func checkAllowKeyRead(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.KeyRead(prefix))
}
func checkAllowKeyList(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.KeyList(prefix))
func checkAllowKeyList(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.KeyList(prefix))
}
func checkAllowKeyringRead(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.KeyringRead())
func checkAllowKeyringRead(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.KeyringRead())
}
func checkAllowKeyringWrite(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.KeyringWrite())
func checkAllowKeyringWrite(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.KeyringWrite())
}
func checkAllowKeyWrite(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.KeyWrite(prefix, nil))
func checkAllowKeyWrite(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.KeyWrite(prefix, nil))
}
func checkAllowKeyWritePrefix(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.KeyWritePrefix(prefix))
func checkAllowKeyWritePrefix(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.KeyWritePrefix(prefix))
}
func checkAllowNodeRead(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.NodeRead(prefix))
func checkAllowNodeRead(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.NodeRead(prefix))
}
func checkAllowNodeWrite(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.NodeWrite(prefix, nil))
func checkAllowNodeWrite(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.NodeWrite(prefix, nil))
}
func checkAllowOperatorRead(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.OperatorRead())
func checkAllowOperatorRead(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.OperatorRead())
}
func checkAllowOperatorWrite(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.OperatorWrite())
func checkAllowOperatorWrite(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.OperatorWrite())
}
func checkAllowPreparedQueryRead(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.PreparedQueryRead(prefix))
func checkAllowPreparedQueryRead(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.PreparedQueryRead(prefix))
}
func checkAllowPreparedQueryWrite(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.PreparedQueryWrite(prefix))
func checkAllowPreparedQueryWrite(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.PreparedQueryWrite(prefix))
}
func checkAllowServiceRead(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.ServiceRead(prefix))
func checkAllowServiceRead(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.ServiceRead(prefix))
}
func checkAllowServiceWrite(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.ServiceWrite(prefix, nil))
func checkAllowServiceWrite(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.ServiceWrite(prefix, nil))
}
func checkAllowSessionRead(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.SessionRead(prefix))
func checkAllowSessionRead(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.SessionRead(prefix))
}
func checkAllowSessionWrite(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.SessionWrite(prefix))
func checkAllowSessionWrite(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.SessionWrite(prefix))
}
func checkAllowSnapshot(t *testing.T, acl ACL, prefix string) {
require.True(t, acl.Snapshot())
func checkAllowSnapshot(t *testing.T, authz Authorizer, prefix string) {
require.True(t, authz.Snapshot())
}
func checkDenyACLList(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.ACLList())
func checkDenyACLRead(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.ACLRead())
}
func checkDenyACLModify(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.ACLModify())
func checkDenyACLWrite(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.ACLWrite())
}
func checkDenyAgentRead(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.AgentRead(prefix))
func checkDenyAgentRead(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.AgentRead(prefix))
}
func checkDenyAgentWrite(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.AgentWrite(prefix))
func checkDenyAgentWrite(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.AgentWrite(prefix))
}
func checkDenyEventRead(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.EventRead(prefix))
func checkDenyEventRead(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.EventRead(prefix))
}
func checkDenyEventWrite(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.EventWrite(prefix))
func checkDenyEventWrite(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.EventWrite(prefix))
}
func checkDenyIntentionDefaultAllow(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.IntentionDefaultAllow())
func checkDenyIntentionDefaultAllow(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.IntentionDefaultAllow())
}
func checkDenyIntentionRead(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.IntentionRead(prefix))
func checkDenyIntentionRead(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.IntentionRead(prefix))
}
func checkDenyIntentionWrite(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.IntentionWrite(prefix))
func checkDenyIntentionWrite(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.IntentionWrite(prefix))
}
func checkDenyKeyRead(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.KeyRead(prefix))
func checkDenyKeyRead(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.KeyRead(prefix))
}
func checkDenyKeyList(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.KeyList(prefix))
func checkDenyKeyList(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.KeyList(prefix))
}
func checkDenyKeyringRead(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.KeyringRead())
func checkDenyKeyringRead(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.KeyringRead())
}
func checkDenyKeyringWrite(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.KeyringWrite())
func checkDenyKeyringWrite(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.KeyringWrite())
}
func checkDenyKeyWrite(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.KeyWrite(prefix, nil))
func checkDenyKeyWrite(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.KeyWrite(prefix, nil))
}
func checkDenyKeyWritePrefix(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.KeyWritePrefix(prefix))
func checkDenyKeyWritePrefix(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.KeyWritePrefix(prefix))
}
func checkDenyNodeRead(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.NodeRead(prefix))
func checkDenyNodeRead(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.NodeRead(prefix))
}
func checkDenyNodeWrite(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.NodeWrite(prefix, nil))
func checkDenyNodeWrite(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.NodeWrite(prefix, nil))
}
func checkDenyOperatorRead(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.OperatorRead())
func checkDenyOperatorRead(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.OperatorRead())
}
func checkDenyOperatorWrite(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.OperatorWrite())
func checkDenyOperatorWrite(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.OperatorWrite())
}
func checkDenyPreparedQueryRead(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.PreparedQueryRead(prefix))
func checkDenyPreparedQueryRead(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.PreparedQueryRead(prefix))
}
func checkDenyPreparedQueryWrite(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.PreparedQueryWrite(prefix))
func checkDenyPreparedQueryWrite(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.PreparedQueryWrite(prefix))
}
func checkDenyServiceRead(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.ServiceRead(prefix))
func checkDenyServiceRead(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.ServiceRead(prefix))
}
func checkDenyServiceWrite(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.ServiceWrite(prefix, nil))
func checkDenyServiceWrite(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.ServiceWrite(prefix, nil))
}
func checkDenySessionRead(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.SessionRead(prefix))
func checkDenySessionRead(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.SessionRead(prefix))
}
func checkDenySessionWrite(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.SessionWrite(prefix))
func checkDenySessionWrite(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.SessionWrite(prefix))
}
func checkDenySnapshot(t *testing.T, acl ACL, prefix string) {
require.False(t, acl.Snapshot())
func checkDenySnapshot(t *testing.T, authz Authorizer, prefix string) {
require.False(t, authz.Snapshot())
}
func TestACL(t *testing.T) {
type aclCheck struct {
name string
prefix string
check func(t *testing.T, acl ACL, prefix string)
check func(t *testing.T, authz Authorizer, prefix string)
}
type aclTest struct {
name string
defaultPolicy ACL
defaultPolicy Authorizer
policyStack []*Policy
checks []aclCheck
}
@ -240,8 +261,8 @@ func TestACL(t *testing.T) {
name: "DenyAll",
defaultPolicy: DenyAll(),
checks: []aclCheck{
{name: "DenyACLList", check: checkDenyACLList},
{name: "DenyACLModify", check: checkDenyACLModify},
{name: "DenyACLRead", check: checkDenyACLRead},
{name: "DenyACLWrite", check: checkDenyACLWrite},
{name: "DenyAgentRead", check: checkDenyAgentRead},
{name: "DenyAgentWrite", check: checkDenyAgentWrite},
{name: "DenyEventRead", check: checkDenyEventRead},
@ -270,8 +291,8 @@ func TestACL(t *testing.T) {
name: "AllowAll",
defaultPolicy: AllowAll(),
checks: []aclCheck{
{name: "DenyACLList", check: checkDenyACLList},
{name: "DenyACLModify", check: checkDenyACLModify},
{name: "DenyACLRead", check: checkDenyACLRead},
{name: "DenyACLWrite", check: checkDenyACLWrite},
{name: "AllowAgentRead", check: checkAllowAgentRead},
{name: "AllowAgentWrite", check: checkAllowAgentWrite},
{name: "AllowEventRead", check: checkAllowEventRead},
@ -300,8 +321,8 @@ func TestACL(t *testing.T) {
name: "ManageAll",
defaultPolicy: ManageAll(),
checks: []aclCheck{
{name: "AllowACLList", check: checkAllowACLList},
{name: "AllowACLModify", check: checkAllowACLModify},
{name: "AllowACLRead", check: checkAllowACLRead},
{name: "AllowACLWrite", check: checkAllowACLWrite},
{name: "AllowAgentRead", check: checkAllowAgentRead},
{name: "AllowAgentWrite", check: checkAllowAgentWrite},
{name: "AllowEventRead", check: checkAllowEventRead},
@ -330,7 +351,7 @@ func TestACL(t *testing.T) {
name: "AgentBasicDefaultDeny",
defaultPolicy: DenyAll(),
policyStack: []*Policy{
&Policy{
legacyPolicy(&Policy{
Agents: []*AgentPolicy{
&AgentPolicy{
Node: "root",
@ -345,7 +366,7 @@ func TestACL(t *testing.T) {
Policy: PolicyWrite,
},
},
},
}),
},
checks: []aclCheck{
{name: "DefaultReadDenied", prefix: "ro", check: checkDenyAgentRead},
@ -368,7 +389,7 @@ func TestACL(t *testing.T) {
name: "AgentBasicDefaultAllow",
defaultPolicy: AllowAll(),
policyStack: []*Policy{
&Policy{
legacyPolicy(&Policy{
Agents: []*AgentPolicy{
&AgentPolicy{
Node: "root",
@ -383,7 +404,7 @@ func TestACL(t *testing.T) {
Policy: PolicyWrite,
},
},
},
}),
},
checks: []aclCheck{
{name: "DefaultReadDenied", prefix: "ro", check: checkAllowAgentRead},
@ -406,14 +427,14 @@ func TestACL(t *testing.T) {
name: "PreparedQueryDefaultAllow",
defaultPolicy: AllowAll(),
policyStack: []*Policy{
&Policy{
legacyPolicy(&Policy{
PreparedQueries: []*PreparedQueryPolicy{
&PreparedQueryPolicy{
Prefix: "other",
Policy: PolicyDeny,
},
},
},
}),
},
checks: []aclCheck{
// in version 1.2.1 and below this would have failed
@ -428,7 +449,7 @@ func TestACL(t *testing.T) {
name: "AgentNestedDefaultDeny",
defaultPolicy: DenyAll(),
policyStack: []*Policy{
&Policy{
legacyPolicy(&Policy{
Agents: []*AgentPolicy{
&AgentPolicy{
Node: "root-nope",
@ -447,8 +468,8 @@ func TestACL(t *testing.T) {
Policy: PolicyDeny,
},
},
},
&Policy{
}),
legacyPolicy(&Policy{
Agents: []*AgentPolicy{
&AgentPolicy{
Node: "child-nope",
@ -467,7 +488,7 @@ func TestACL(t *testing.T) {
Policy: PolicyWrite,
},
},
},
}),
},
checks: []aclCheck{
{name: "DefaultReadDenied", prefix: "nope", check: checkDenyAgentRead},
@ -504,7 +525,7 @@ func TestACL(t *testing.T) {
name: "AgentNestedDefaultAllow",
defaultPolicy: AllowAll(),
policyStack: []*Policy{
&Policy{
legacyPolicy(&Policy{
Agents: []*AgentPolicy{
&AgentPolicy{
Node: "root-nope",
@ -523,8 +544,8 @@ func TestACL(t *testing.T) {
Policy: PolicyDeny,
},
},
},
&Policy{
}),
legacyPolicy(&Policy{
Agents: []*AgentPolicy{
&AgentPolicy{
Node: "child-nope",
@ -543,7 +564,7 @@ func TestACL(t *testing.T) {
Policy: PolicyWrite,
},
},
},
}),
},
checks: []aclCheck{
{name: "DefaultReadAllowed", prefix: "nope", check: checkAllowAgentRead},
@ -784,7 +805,7 @@ func TestACL(t *testing.T) {
name: "NodeDefaultDeny",
defaultPolicy: DenyAll(),
policyStack: []*Policy{
&Policy{
legacyPolicy(&Policy{
Nodes: []*NodePolicy{
&NodePolicy{
Name: "root-nope",
@ -803,8 +824,8 @@ func TestACL(t *testing.T) {
Policy: PolicyDeny,
},
},
},
&Policy{
}),
legacyPolicy(&Policy{
Nodes: []*NodePolicy{
&NodePolicy{
Name: "child-nope",
@ -823,7 +844,7 @@ func TestACL(t *testing.T) {
Policy: PolicyWrite,
},
},
},
}),
},
checks: []aclCheck{
{name: "DefaultReadDenied", prefix: "nope", check: checkDenyNodeRead},
@ -860,7 +881,7 @@ func TestACL(t *testing.T) {
name: "NodeDefaultAllow",
defaultPolicy: AllowAll(),
policyStack: []*Policy{
&Policy{
legacyPolicy(&Policy{
Nodes: []*NodePolicy{
&NodePolicy{
Name: "root-nope",
@ -879,8 +900,8 @@ func TestACL(t *testing.T) {
Policy: PolicyDeny,
},
},
},
&Policy{
}),
legacyPolicy(&Policy{
Nodes: []*NodePolicy{
&NodePolicy{
Name: "child-nope",
@ -899,7 +920,7 @@ func TestACL(t *testing.T) {
Policy: PolicyWrite,
},
},
},
}),
},
checks: []aclCheck{
{name: "DefaultReadAllowed", prefix: "nope", check: checkAllowNodeRead},
@ -936,7 +957,7 @@ func TestACL(t *testing.T) {
name: "SessionDefaultDeny",
defaultPolicy: DenyAll(),
policyStack: []*Policy{
&Policy{
legacyPolicy(&Policy{
Sessions: []*SessionPolicy{
&SessionPolicy{
Node: "root-nope",
@ -955,8 +976,8 @@ func TestACL(t *testing.T) {
Policy: PolicyDeny,
},
},
},
&Policy{
}),
legacyPolicy(&Policy{
Sessions: []*SessionPolicy{
&SessionPolicy{
Node: "child-nope",
@ -975,7 +996,7 @@ func TestACL(t *testing.T) {
Policy: PolicyWrite,
},
},
},
}),
},
checks: []aclCheck{
{name: "DefaultReadDenied", prefix: "nope", check: checkDenySessionRead},
@ -1012,7 +1033,7 @@ func TestACL(t *testing.T) {
name: "SessionDefaultAllow",
defaultPolicy: AllowAll(),
policyStack: []*Policy{
&Policy{
legacyPolicy(&Policy{
Sessions: []*SessionPolicy{
&SessionPolicy{
Node: "root-nope",
@ -1031,8 +1052,8 @@ func TestACL(t *testing.T) {
Policy: PolicyDeny,
},
},
},
&Policy{
}),
legacyPolicy(&Policy{
Sessions: []*SessionPolicy{
&SessionPolicy{
Node: "child-nope",
@ -1051,7 +1072,7 @@ func TestACL(t *testing.T) {
Policy: PolicyWrite,
},
},
},
}),
},
checks: []aclCheck{
{name: "DefaultReadAllowed", prefix: "nope", check: checkAllowSessionRead},
@ -1088,7 +1109,7 @@ func TestACL(t *testing.T) {
name: "Parent",
defaultPolicy: DenyAll(),
policyStack: []*Policy{
&Policy{
legacyPolicy(&Policy{
Keys: []*KeyPolicy{
&KeyPolicy{
Prefix: "foo/",
@ -1119,8 +1140,8 @@ func TestACL(t *testing.T) {
Policy: PolicyRead,
},
},
},
&Policy{
}),
legacyPolicy(&Policy{
Keys: []*KeyPolicy{
&KeyPolicy{
Prefix: "foo/priv/",
@ -1147,7 +1168,7 @@ func TestACL(t *testing.T) {
Policy: PolicyDeny,
},
},
},
}),
},
checks: []aclCheck{
{name: "KeyReadDenied", prefix: "other", check: checkDenyKeyRead},
@ -1185,8 +1206,8 @@ func TestACL(t *testing.T) {
{name: "PreparedQueryWriteDenied", prefix: "baz", check: checkDenyPreparedQueryWrite},
{name: "PreparedQueryReadDenied", prefix: "nope", check: checkDenyPreparedQueryRead},
{name: "PreparedQueryWriteDenied", prefix: "nope", check: checkDenyPreparedQueryWrite},
{name: "ACLListDenied", check: checkDenyACLList},
{name: "ACLModifyDenied", check: checkDenyACLModify},
{name: "ACLReadDenied", check: checkDenyACLRead},
{name: "ACLWriteDenied", check: checkDenyACLWrite},
{name: "SnapshotDenied", check: checkDenySnapshot},
{name: "IntentionDefaultAllowDenied", check: checkDenyIntentionDefaultAllow},
},
@ -1195,7 +1216,7 @@ func TestACL(t *testing.T) {
name: "ComplexDefaultAllow",
defaultPolicy: AllowAll(),
policyStack: []*Policy{
&Policy{
legacyPolicy(&Policy{
Events: []*EventPolicy{
&EventPolicy{
Event: "",
@ -1274,7 +1295,7 @@ func TestACL(t *testing.T) {
Intentions: PolicyDeny,
},
},
},
}),
},
checks: []aclCheck{
{name: "KeyReadAllowed", prefix: "other", check: checkAllowKeyRead},
@ -1368,13 +1389,328 @@ func TestACL(t *testing.T) {
{name: "PreparedQueryWriteAllowed", prefix: "zookeeper", check: checkAllowPreparedQueryWrite},
},
},
{
name: "ExactMatchPrecedence",
defaultPolicy: DenyAll(),
policyStack: []*Policy{
&Policy{
Agents: []*AgentPolicy{
&AgentPolicy{
Node: "foo",
Policy: PolicyWrite,
},
&AgentPolicy{
Node: "football",
Policy: PolicyDeny,
},
},
AgentPrefixes: []*AgentPolicy{
&AgentPolicy{
Node: "foot",
Policy: PolicyRead,
},
&AgentPolicy{
Node: "fo",
Policy: PolicyRead,
},
},
Keys: []*KeyPolicy{
&KeyPolicy{
Prefix: "foo",
Policy: PolicyWrite,
},
&KeyPolicy{
Prefix: "football",
Policy: PolicyDeny,
},
},
KeyPrefixes: []*KeyPolicy{
&KeyPolicy{
Prefix: "foot",
Policy: PolicyRead,
},
&KeyPolicy{
Prefix: "fo",
Policy: PolicyRead,
},
},
Nodes: []*NodePolicy{
&NodePolicy{
Name: "foo",
Policy: PolicyWrite,
},
&NodePolicy{
Name: "football",
Policy: PolicyDeny,
},
},
NodePrefixes: []*NodePolicy{
&NodePolicy{
Name: "foot",
Policy: PolicyRead,
},
&NodePolicy{
Name: "fo",
Policy: PolicyRead,
},
},
Services: []*ServicePolicy{
&ServicePolicy{
Name: "foo",
Policy: PolicyWrite,
Intentions: PolicyWrite,
},
&ServicePolicy{
Name: "football",
Policy: PolicyDeny,
},
},
ServicePrefixes: []*ServicePolicy{
&ServicePolicy{
Name: "foot",
Policy: PolicyRead,
Intentions: PolicyRead,
},
&ServicePolicy{
Name: "fo",
Policy: PolicyRead,
Intentions: PolicyRead,
},
},
Sessions: []*SessionPolicy{
&SessionPolicy{
Node: "foo",
Policy: PolicyWrite,
},
&SessionPolicy{
Node: "football",
Policy: PolicyDeny,
},
},
SessionPrefixes: []*SessionPolicy{
&SessionPolicy{
Node: "foot",
Policy: PolicyRead,
},
&SessionPolicy{
Node: "fo",
Policy: PolicyRead,
},
},
Events: []*EventPolicy{
&EventPolicy{
Event: "foo",
Policy: PolicyWrite,
},
&EventPolicy{
Event: "football",
Policy: PolicyDeny,
},
},
EventPrefixes: []*EventPolicy{
&EventPolicy{
Event: "foot",
Policy: PolicyRead,
},
&EventPolicy{
Event: "fo",
Policy: PolicyRead,
},
},
PreparedQueries: []*PreparedQueryPolicy{
&PreparedQueryPolicy{
Prefix: "foo",
Policy: PolicyWrite,
},
&PreparedQueryPolicy{
Prefix: "football",
Policy: PolicyDeny,
},
},
PreparedQueryPrefixes: []*PreparedQueryPolicy{
&PreparedQueryPolicy{
Prefix: "foot",
Policy: PolicyRead,
},
&PreparedQueryPolicy{
Prefix: "fo",
Policy: PolicyRead,
},
},
},
},
checks: []aclCheck{
{name: "AgentReadPrefixAllowed", prefix: "fo", check: checkAllowAgentRead},
{name: "AgentWritePrefixDenied", prefix: "fo", check: checkDenyAgentWrite},
{name: "AgentReadPrefixAllowed", prefix: "for", check: checkAllowAgentRead},
{name: "AgentWritePrefixDenied", prefix: "for", check: checkDenyAgentWrite},
{name: "AgentReadAllowed", prefix: "foo", check: checkAllowAgentRead},
{name: "AgentWriteAllowed", prefix: "foo", check: checkAllowAgentWrite},
{name: "AgentReadPrefixAllowed", prefix: "foot", check: checkAllowAgentRead},
{name: "AgentWritePrefixDenied", prefix: "foot", check: checkDenyAgentWrite},
{name: "AgentReadPrefixAllowed", prefix: "foot2", check: checkAllowAgentRead},
{name: "AgentWritePrefixDenied", prefix: "foot2", check: checkDenyAgentWrite},
{name: "AgentReadPrefixAllowed", prefix: "food", check: checkAllowAgentRead},
{name: "AgentWritePrefixDenied", prefix: "food", check: checkDenyAgentWrite},
{name: "AgentReadDenied", prefix: "football", check: checkDenyAgentRead},
{name: "AgentWriteDenied", prefix: "football", check: checkDenyAgentWrite},
{name: "KeyReadPrefixAllowed", prefix: "fo", check: checkAllowKeyRead},
{name: "KeyWritePrefixDenied", prefix: "fo", check: checkDenyKeyWrite},
{name: "KeyReadPrefixAllowed", prefix: "for", check: checkAllowKeyRead},
{name: "KeyWritePrefixDenied", prefix: "for", check: checkDenyKeyWrite},
{name: "KeyReadAllowed", prefix: "foo", check: checkAllowKeyRead},
{name: "KeyWriteAllowed", prefix: "foo", check: checkAllowKeyWrite},
{name: "KeyReadPrefixAllowed", prefix: "foot", check: checkAllowKeyRead},
{name: "KeyWritePrefixDenied", prefix: "foot", check: checkDenyKeyWrite},
{name: "KeyReadPrefixAllowed", prefix: "foot2", check: checkAllowKeyRead},
{name: "KeyWritePrefixDenied", prefix: "foot2", check: checkDenyKeyWrite},
{name: "KeyReadPrefixAllowed", prefix: "food", check: checkAllowKeyRead},
{name: "KeyWritePrefixDenied", prefix: "food", check: checkDenyKeyWrite},
{name: "KeyReadDenied", prefix: "football", check: checkDenyKeyRead},
{name: "KeyWriteDenied", prefix: "football", check: checkDenyKeyWrite},
{name: "NodeReadPrefixAllowed", prefix: "fo", check: checkAllowNodeRead},
{name: "NodeWritePrefixDenied", prefix: "fo", check: checkDenyNodeWrite},
{name: "NodeReadPrefixAllowed", prefix: "for", check: checkAllowNodeRead},
{name: "NodeWritePrefixDenied", prefix: "for", check: checkDenyNodeWrite},
{name: "NodeReadAllowed", prefix: "foo", check: checkAllowNodeRead},
{name: "NodeWriteAllowed", prefix: "foo", check: checkAllowNodeWrite},
{name: "NodeReadPrefixAllowed", prefix: "foot", check: checkAllowNodeRead},
{name: "NodeWritePrefixDenied", prefix: "foot", check: checkDenyNodeWrite},
{name: "NodeReadPrefixAllowed", prefix: "foot2", check: checkAllowNodeRead},
{name: "NodeWritePrefixDenied", prefix: "foot2", check: checkDenyNodeWrite},
{name: "NodeReadPrefixAllowed", prefix: "food", check: checkAllowNodeRead},
{name: "NodeWritePrefixDenied", prefix: "food", check: checkDenyNodeWrite},
{name: "NodeReadDenied", prefix: "football", check: checkDenyNodeRead},
{name: "NodeWriteDenied", prefix: "football", check: checkDenyNodeWrite},
{name: "ServiceReadPrefixAllowed", prefix: "fo", check: checkAllowServiceRead},
{name: "ServiceWritePrefixDenied", prefix: "fo", check: checkDenyServiceWrite},
{name: "ServiceReadPrefixAllowed", prefix: "for", check: checkAllowServiceRead},
{name: "ServiceWritePrefixDenied", prefix: "for", check: checkDenyServiceWrite},
{name: "ServiceReadAllowed", prefix: "foo", check: checkAllowServiceRead},
{name: "ServiceWriteAllowed", prefix: "foo", check: checkAllowServiceWrite},
{name: "ServiceReadPrefixAllowed", prefix: "foot", check: checkAllowServiceRead},
{name: "ServiceWritePrefixDenied", prefix: "foot", check: checkDenyServiceWrite},
{name: "ServiceReadPrefixAllowed", prefix: "foot2", check: checkAllowServiceRead},
{name: "ServiceWritePrefixDenied", prefix: "foot2", check: checkDenyServiceWrite},
{name: "ServiceReadPrefixAllowed", prefix: "food", check: checkAllowServiceRead},
{name: "ServiceWritePrefixDenied", prefix: "food", check: checkDenyServiceWrite},
{name: "ServiceReadDenied", prefix: "football", check: checkDenyServiceRead},
{name: "ServiceWriteDenied", prefix: "football", check: checkDenyServiceWrite},
{name: "NodeReadPrefixAllowed", prefix: "fo", check: checkAllowNodeRead},
{name: "NodeWritePrefixDenied", prefix: "fo", check: checkDenyNodeWrite},
{name: "NodeReadPrefixAllowed", prefix: "for", check: checkAllowNodeRead},
{name: "NodeWritePrefixDenied", prefix: "for", check: checkDenyNodeWrite},
{name: "NodeReadAllowed", prefix: "foo", check: checkAllowNodeRead},
{name: "NodeWriteAllowed", prefix: "foo", check: checkAllowNodeWrite},
{name: "NodeReadPrefixAllowed", prefix: "foot", check: checkAllowNodeRead},
{name: "NodeWritePrefixDenied", prefix: "foot", check: checkDenyNodeWrite},
{name: "NodeReadPrefixAllowed", prefix: "foot2", check: checkAllowNodeRead},
{name: "NodeWritePrefixDenied", prefix: "foot2", check: checkDenyNodeWrite},
{name: "NodeReadPrefixAllowed", prefix: "food", check: checkAllowNodeRead},
{name: "NodeWritePrefixDenied", prefix: "food", check: checkDenyNodeWrite},
{name: "NodeReadDenied", prefix: "football", check: checkDenyNodeRead},
{name: "NodeWriteDenied", prefix: "football", check: checkDenyNodeWrite},
{name: "IntentionReadPrefixAllowed", prefix: "fo", check: checkAllowIntentionRead},
{name: "IntentionWritePrefixDenied", prefix: "fo", check: checkDenyIntentionWrite},
{name: "IntentionReadPrefixAllowed", prefix: "for", check: checkAllowIntentionRead},
{name: "IntentionWritePrefixDenied", prefix: "for", check: checkDenyIntentionWrite},
{name: "IntentionReadAllowed", prefix: "foo", check: checkAllowIntentionRead},
{name: "IntentionWriteAllowed", prefix: "foo", check: checkAllowIntentionWrite},
{name: "IntentionReadPrefixAllowed", prefix: "foot", check: checkAllowIntentionRead},
{name: "IntentionWritePrefixDenied", prefix: "foot", check: checkDenyIntentionWrite},
{name: "IntentionReadPrefixAllowed", prefix: "foot2", check: checkAllowIntentionRead},
{name: "IntentionWritePrefixDenied", prefix: "foot2", check: checkDenyIntentionWrite},
{name: "IntentionReadPrefixAllowed", prefix: "food", check: checkAllowIntentionRead},
{name: "IntentionWritePrefixDenied", prefix: "food", check: checkDenyIntentionWrite},
{name: "IntentionReadDenied", prefix: "football", check: checkDenyIntentionRead},
{name: "IntentionWriteDenied", prefix: "football", check: checkDenyIntentionWrite},
{name: "SessionReadPrefixAllowed", prefix: "fo", check: checkAllowSessionRead},
{name: "SessionWritePrefixDenied", prefix: "fo", check: checkDenySessionWrite},
{name: "SessionReadPrefixAllowed", prefix: "for", check: checkAllowSessionRead},
{name: "SessionWritePrefixDenied", prefix: "for", check: checkDenySessionWrite},
{name: "SessionReadAllowed", prefix: "foo", check: checkAllowSessionRead},
{name: "SessionWriteAllowed", prefix: "foo", check: checkAllowSessionWrite},
{name: "SessionReadPrefixAllowed", prefix: "foot", check: checkAllowSessionRead},
{name: "SessionWritePrefixDenied", prefix: "foot", check: checkDenySessionWrite},
{name: "SessionReadPrefixAllowed", prefix: "foot2", check: checkAllowSessionRead},
{name: "SessionWritePrefixDenied", prefix: "foot2", check: checkDenySessionWrite},
{name: "SessionReadPrefixAllowed", prefix: "food", check: checkAllowSessionRead},
{name: "SessionWritePrefixDenied", prefix: "food", check: checkDenySessionWrite},
{name: "SessionReadDenied", prefix: "football", check: checkDenySessionRead},
{name: "SessionWriteDenied", prefix: "football", check: checkDenySessionWrite},
{name: "EventReadPrefixAllowed", prefix: "fo", check: checkAllowEventRead},
{name: "EventWritePrefixDenied", prefix: "fo", check: checkDenyEventWrite},
{name: "EventReadPrefixAllowed", prefix: "for", check: checkAllowEventRead},
{name: "EventWritePrefixDenied", prefix: "for", check: checkDenyEventWrite},
{name: "EventReadAllowed", prefix: "foo", check: checkAllowEventRead},
{name: "EventWriteAllowed", prefix: "foo", check: checkAllowEventWrite},
{name: "EventReadPrefixAllowed", prefix: "foot", check: checkAllowEventRead},
{name: "EventWritePrefixDenied", prefix: "foot", check: checkDenyEventWrite},
{name: "EventReadPrefixAllowed", prefix: "foot2", check: checkAllowEventRead},
{name: "EventWritePrefixDenied", prefix: "foot2", check: checkDenyEventWrite},
{name: "EventReadPrefixAllowed", prefix: "food", check: checkAllowEventRead},
{name: "EventWritePrefixDenied", prefix: "food", check: checkDenyEventWrite},
{name: "EventReadDenied", prefix: "football", check: checkDenyEventRead},
{name: "EventWriteDenied", prefix: "football", check: checkDenyEventWrite},
{name: "PreparedQueryReadPrefixAllowed", prefix: "fo", check: checkAllowPreparedQueryRead},
{name: "PreparedQueryWritePrefixDenied", prefix: "fo", check: checkDenyPreparedQueryWrite},
{name: "PreparedQueryReadPrefixAllowed", prefix: "for", check: checkAllowPreparedQueryRead},
{name: "PreparedQueryWritePrefixDenied", prefix: "for", check: checkDenyPreparedQueryWrite},
{name: "PreparedQueryReadAllowed", prefix: "foo", check: checkAllowPreparedQueryRead},
{name: "PreparedQueryWriteAllowed", prefix: "foo", check: checkAllowPreparedQueryWrite},
{name: "PreparedQueryReadPrefixAllowed", prefix: "foot", check: checkAllowPreparedQueryRead},
{name: "PreparedQueryWritePrefixDenied", prefix: "foot", check: checkDenyPreparedQueryWrite},
{name: "PreparedQueryReadPrefixAllowed", prefix: "foot2", check: checkAllowPreparedQueryRead},
{name: "PreparedQueryWritePrefixDenied", prefix: "foot2", check: checkDenyPreparedQueryWrite},
{name: "PreparedQueryReadPrefixAllowed", prefix: "food", check: checkAllowPreparedQueryRead},
{name: "PreparedQueryWritePrefixDenied", prefix: "food", check: checkDenyPreparedQueryWrite},
{name: "PreparedQueryReadDenied", prefix: "football", check: checkDenyPreparedQueryRead},
{name: "PreparedQueryWriteDenied", prefix: "football", check: checkDenyPreparedQueryWrite},
},
},
{
name: "ACLRead",
defaultPolicy: DenyAll(),
policyStack: []*Policy{
&Policy{
ACL: PolicyRead,
},
},
checks: []aclCheck{
{name: "ReadAllowed", check: checkAllowACLRead},
// in version 1.2.1 and below this would have failed
{name: "WriteDenied", check: checkDenyACLWrite},
},
},
{
name: "ACLRead",
defaultPolicy: DenyAll(),
policyStack: []*Policy{
&Policy{
ACL: PolicyWrite,
},
},
checks: []aclCheck{
{name: "ReadAllowed", check: checkAllowACLRead},
// in version 1.2.1 and below this would have failed
{name: "WriteAllowed", check: checkAllowACLWrite},
},
},
}
for _, tcase := range tests {
t.Run(tcase.name, func(t *testing.T) {
acl := tcase.defaultPolicy
for _, policy := range tcase.policyStack {
newACL, err := New(acl, policy, nil)
newACL, err := NewPolicyAuthorizer(acl, []*Policy{policy}, nil)
require.NoError(t, err)
acl = newACL
}
@ -1392,11 +1728,11 @@ func TestACL(t *testing.T) {
}
}
func TestRootACL(t *testing.T) {
require.Equal(t, AllowAll(), RootACL("allow"))
require.Equal(t, DenyAll(), RootACL("deny"))
require.Equal(t, ManageAll(), RootACL("manage"))
require.Nil(t, RootACL("foo"))
func TestRootAuthorizer(t *testing.T) {
require.Equal(t, AllowAll(), RootAuthorizer("allow"))
require.Equal(t, DenyAll(), RootAuthorizer("deny"))
require.Equal(t, ManageAll(), RootAuthorizer("manage"))
require.Nil(t, RootAuthorizer("foo"))
}
func TestACLEnforce(t *testing.T) {

180
acl/cache.go
View File

@ -1,180 +0,0 @@
package acl
import (
"crypto/md5"
"fmt"
"github.com/hashicorp/consul/sentinel"
"github.com/hashicorp/golang-lru"
)
// FaultFunc is a function used to fault in the parent,
// rules for an ACL given its ID
type FaultFunc func(id string) (string, string, error)
// aclEntry allows us to store the ACL with it's policy ID
type aclEntry struct {
ACL ACL
Parent string
RuleID string
}
// Cache is used to implement policy and ACL caching
type Cache struct {
faultfn FaultFunc
aclCache *lru.TwoQueueCache // Cache id -> acl
policyCache *lru.TwoQueueCache // Cache policy -> acl
ruleCache *lru.TwoQueueCache // Cache rules -> policy
sentinel sentinel.Evaluator
}
// NewCache constructs a new policy and ACL cache of a given size
func NewCache(size int, faultfn FaultFunc, sentinel sentinel.Evaluator) (*Cache, error) {
if size <= 0 {
return nil, fmt.Errorf("Must provide positive cache size")
}
rc, err := lru.New2Q(size)
if err != nil {
return nil, err
}
pc, err := lru.New2Q(size)
if err != nil {
return nil, err
}
ac, err := lru.New2Q(size)
if err != nil {
return nil, err
}
c := &Cache{
faultfn: faultfn,
aclCache: ac,
policyCache: pc,
ruleCache: rc,
sentinel: sentinel,
}
return c, nil
}
// GetPolicy is used to get a potentially cached policy set.
// If not cached, it will be parsed, and then cached.
func (c *Cache) GetPolicy(rules string) (*Policy, error) {
return c.getPolicy(RuleID(rules), rules)
}
// getPolicy is an internal method to get a cached policy,
// but it assumes a pre-computed ID
func (c *Cache) getPolicy(id, rules string) (*Policy, error) {
raw, ok := c.ruleCache.Get(id)
if ok {
return raw.(*Policy), nil
}
policy, err := Parse(rules, c.sentinel)
if err != nil {
return nil, err
}
policy.ID = id
c.ruleCache.Add(id, policy)
return policy, nil
}
// RuleID is used to generate an ID for a rule
func RuleID(rules string) string {
return fmt.Sprintf("%x", md5.Sum([]byte(rules)))
}
// policyID returns the cache ID for a policy
func (c *Cache) policyID(parent, ruleID string) string {
return parent + ":" + ruleID
}
// GetACLPolicy is used to get the potentially cached ACL
// policy. If not cached, it will be generated and then cached.
func (c *Cache) GetACLPolicy(id string) (string, *Policy, error) {
// Check for a cached acl
if raw, ok := c.aclCache.Get(id); ok {
cached := raw.(aclEntry)
if raw, ok := c.ruleCache.Get(cached.RuleID); ok {
return cached.Parent, raw.(*Policy), nil
}
}
// Fault in the rules
parent, rules, err := c.faultfn(id)
if err != nil {
return "", nil, err
}
// Get cached
policy, err := c.GetPolicy(rules)
return parent, policy, err
}
// GetACL is used to get a potentially cached ACL policy.
// If not cached, it will be generated and then cached.
func (c *Cache) GetACL(id string) (ACL, error) {
// Look for the ACL directly
raw, ok := c.aclCache.Get(id)
if ok {
return raw.(aclEntry).ACL, nil
}
// Get the rules
parentID, rules, err := c.faultfn(id)
if err != nil {
return nil, err
}
ruleID := RuleID(rules)
// Check for a compiled ACL
policyID := c.policyID(parentID, ruleID)
var compiled ACL
if raw, ok := c.policyCache.Get(policyID); ok {
compiled = raw.(ACL)
} else {
// Get the policy
policy, err := c.getPolicy(ruleID, rules)
if err != nil {
return nil, err
}
// Get the parent ACL
parent := RootACL(parentID)
if parent == nil {
parent, err = c.GetACL(parentID)
if err != nil {
return nil, err
}
}
// Compile the ACL
acl, err := New(parent, policy, c.sentinel)
if err != nil {
return nil, err
}
// Cache the compiled ACL
c.policyCache.Add(policyID, acl)
compiled = acl
}
// Cache and return the ACL
c.aclCache.Add(id, aclEntry{compiled, parentID, ruleID})
return compiled, nil
}
// ClearACL is used to clear the ACL cache if any
func (c *Cache) ClearACL(id string) {
c.aclCache.Remove(id)
}
// Purge is used to clear all the ACL caches. The
// rule and policy caches are not purged, since they
// are content-hashed anyways.
func (c *Cache) Purge() {
c.aclCache.Purge()
}

328
acl/cache_test.go
View File

@ -1,328 +0,0 @@
package acl
import (
"testing"
)
func TestCache_GetPolicy(t *testing.T) {
c, err := NewCache(2, nil, nil)
if err != nil {
t.Fatalf("err: %v", err)
}
p, err := c.GetPolicy("")
if err != nil {
t.Fatalf("err: %v", err)
}
// Should get the same policy
p1, err := c.GetPolicy("")
if err != nil {
t.Fatalf("err: %v", err)
}
if p != p1 {
t.Fatalf("should be cached")
}
// Work with some new policies to evict the original one
_, err = c.GetPolicy(testSimplePolicy)
if err != nil {
t.Fatalf("err: %v", err)
}
_, err = c.GetPolicy(testSimplePolicy)
if err != nil {
t.Fatalf("err: %v", err)
}
_, err = c.GetPolicy(testSimplePolicy2)
if err != nil {
t.Fatalf("err: %v", err)
}
_, err = c.GetPolicy(testSimplePolicy2)
if err != nil {
t.Fatalf("err: %v", err)
}
// Test invalidation of p
p3, err := c.GetPolicy("")
if err != nil {
t.Fatalf("err: %v", err)
}
if p == p3 {
t.Fatalf("should be not cached")
}
}
func TestCache_GetACL(t *testing.T) {
policies := map[string]string{
"foo": testSimplePolicy,
"bar": testSimplePolicy2,
"baz": testSimplePolicy3,
}
faultfn := func(id string) (string, string, error) {
return "deny", policies[id], nil
}
c, err := NewCache(2, faultfn, nil)
if err != nil {
t.Fatalf("err: %v", err)
}
acl, err := c.GetACL("foo")
if err != nil {
t.Fatalf("err: %v", err)
}
if acl.KeyRead("bar/test") {
t.Fatalf("should deny")
}
if !acl.KeyRead("foo/test") {
t.Fatalf("should allow")
}
acl2, err := c.GetACL("foo")
if err != nil {
t.Fatalf("err: %v", err)
}
if acl != acl2 {
t.Fatalf("should be cached")
}
// Invalidate cache
_, err = c.GetACL("bar")
if err != nil {
t.Fatalf("err: %v", err)
}
_, err = c.GetACL("bar")
if err != nil {
t.Fatalf("err: %v", err)
}
_, err = c.GetACL("baz")
if err != nil {
t.Fatalf("err: %v", err)
}
_, err = c.GetACL("baz")
if err != nil {
t.Fatalf("err: %v", err)
}
acl3, err := c.GetACL("foo")
if err != nil {
t.Fatalf("err: %v", err)
}
if acl == acl3 {
t.Fatalf("should not be cached")
}
}
func TestCache_ClearACL(t *testing.T) {
policies := map[string]string{
"foo": testSimplePolicy,
"bar": testSimplePolicy,
}
faultfn := func(id string) (string, string, error) {
return "deny", policies[id], nil
}
c, err := NewCache(16, faultfn, nil)
if err != nil {
t.Fatalf("err: %v", err)
}
acl, err := c.GetACL("foo")
if err != nil {
t.Fatalf("err: %v", err)
}
// Nuke the cache
c.ClearACL("foo")
// Clear the policy cache
c.policyCache.Purge()
acl2, err := c.GetACL("foo")
if err != nil {
t.Fatalf("err: %v", err)
}
if acl == acl2 {
t.Fatalf("should not be cached")
}
}
func TestCache_Purge(t *testing.T) {
policies := map[string]string{
"foo": testSimplePolicy,
"bar": testSimplePolicy,
}
faultfn := func(id string) (string, string, error) {
return "deny", policies[id], nil
}
c, err := NewCache(16, faultfn, nil)
if err != nil {
t.Fatalf("err: %v", err)
}
acl, err := c.GetACL("foo")
if err != nil {
t.Fatalf("err: %v", err)
}
// Nuke the cache
c.Purge()
c.policyCache.Purge()
acl2, err := c.GetACL("foo")
if err != nil {
t.Fatalf("err: %v", err)
}
if acl == acl2 {
t.Fatalf("should not be cached")
}
}
func TestCache_GetACLPolicy(t *testing.T) {
policies := map[string]string{
"foo": testSimplePolicy,
"bar": testSimplePolicy,
}
faultfn := func(id string) (string, string, error) {
return "deny", policies[id], nil
}
c, err := NewCache(16, faultfn, nil)
if err != nil {
t.Fatalf("err: %v", err)
}
p, err := c.GetPolicy(testSimplePolicy)
if err != nil {
t.Fatalf("err: %v", err)
}
_, err = c.GetACL("foo")
if err != nil {
t.Fatalf("err: %v", err)
}
parent, p2, err := c.GetACLPolicy("foo")
if err != nil {
t.Fatalf("err: %v", err)
}
if parent != "deny" {
t.Fatalf("bad: %v", parent)
}
if p2 != p {
t.Fatalf("expected cached policy")
}
parent, p3, err := c.GetACLPolicy("bar")
if err != nil {
t.Fatalf("err: %v", err)
}
if parent != "deny" {
t.Fatalf("bad: %v", parent)
}
if p3 != p {
t.Fatalf("expected cached policy")
}
}
func TestCache_GetACL_Parent(t *testing.T) {
faultfn := func(id string) (string, string, error) {
switch id {
case "foo":
// Foo inherits from bar
return "bar", testSimplePolicy, nil
case "bar":
return "deny", testSimplePolicy2, nil
}
t.Fatalf("bad case")
return "", "", nil
}
c, err := NewCache(16, faultfn, nil)
if err != nil {
t.Fatalf("err: %v", err)
}
acl, err := c.GetACL("foo")
if err != nil {
t.Fatalf("err: %v", err)
}
if !acl.KeyRead("bar/test") {
t.Fatalf("should allow")
}
if !acl.KeyRead("foo/test") {
t.Fatalf("should allow")
}
}
func TestCache_GetACL_ParentCache(t *testing.T) {
// Same rules, different parent
faultfn := func(id string) (string, string, error) {
switch id {
case "foo":
return "allow", testSimplePolicy, nil
case "bar":
return "deny", testSimplePolicy, nil
}
t.Fatalf("bad case")
return "", "", nil
}
c, err := NewCache(16, faultfn, nil)
if err != nil {
t.Fatalf("err: %v", err)
}
acl, err := c.GetACL("foo")
if err != nil {
t.Fatalf("err: %v", err)
}
if !acl.KeyRead("bar/test") {
t.Fatalf("should allow")
}
if !acl.KeyRead("foo/test") {
t.Fatalf("should allow")
}
acl2, err := c.GetACL("bar")
if err != nil {
t.Fatalf("err: %v", err)
}
if acl == acl2 {
t.Fatalf("should not match")
}
if acl2.KeyRead("bar/test") {
t.Fatalf("should not allow")
}
if !acl2.KeyRead("foo/test") {
t.Fatalf("should allow")
}
}
var testSimplePolicy = `
key "foo/" {
policy = "read"
}
`
var testSimplePolicy2 = `
key "bar/" {
policy = "read"
}
`
var testSimplePolicy3 = `
key "baz/" {
policy = "read"
}
`

5
acl/errors.go
View File

@ -13,6 +13,7 @@ const (
errRootDenied = "Cannot resolve root ACL"
errDisabled = "ACL support disabled"
errPermissionDenied = "Permission denied"
errInvalidParent = "Invalid Parent"
)
var (
@ -29,6 +30,10 @@ var (
// ErrPermissionDenied is returned when an ACL based rejection
// happens.
ErrPermissionDenied = PermissionDeniedError{}
// ErrInvalidParent is returned when a remotely resolve ACL
// token claims to have a non-root parent
ErrInvalidParent = errors.New(errInvalidParent)
)
// IsErrNotFound checks if the given error message is comparable to

643
acl/policy.go
View File

@ -1,10 +1,22 @@
package acl
import (
"bytes"
"encoding/binary"
"fmt"
"github.com/hashicorp/consul/sentinel"
"github.com/hashicorp/hcl"
"github.com/hashicorp/hcl/hcl/ast"
hclprinter "github.com/hashicorp/hcl/hcl/printer"
"golang.org/x/crypto/blake2b"
)
type SyntaxVersion int
const (
SyntaxCurrent SyntaxVersion = iota
SyntaxLegacy
)
const (
@ -17,16 +29,25 @@ const (
// Policy is used to represent the policy specified by
// an ACL configuration.
type Policy struct {
ID string `hcl:"-"`
Agents []*AgentPolicy `hcl:"agent,expand"`
Keys []*KeyPolicy `hcl:"key,expand"`
Nodes []*NodePolicy `hcl:"node,expand"`
Services []*ServicePolicy `hcl:"service,expand"`
Sessions []*SessionPolicy `hcl:"session,expand"`
Events []*EventPolicy `hcl:"event,expand"`
PreparedQueries []*PreparedQueryPolicy `hcl:"query,expand"`
Keyring string `hcl:"keyring"`
Operator string `hcl:"operator"`
ID string `hcl:"id"`
Revision uint64 `hcl:"revision"`
ACL string `hcl:"acl,expand"`
Agents []*AgentPolicy `hcl:"agent,expand"`
AgentPrefixes []*AgentPolicy `hcl:"agent_prefix,expand"`
Keys []*KeyPolicy `hcl:"key,expand"`
KeyPrefixes []*KeyPolicy `hcl:"key_prefix,expand"`
Nodes []*NodePolicy `hcl:"node,expand"`
NodePrefixes []*NodePolicy `hcl:"node_prefix,expand"`
Services []*ServicePolicy `hcl:"service,expand"`
ServicePrefixes []*ServicePolicy `hcl:"service_prefix,expand"`
Sessions []*SessionPolicy `hcl:"session,expand"`
SessionPrefixes []*SessionPolicy `hcl:"session_prefix,expand"`
Events []*EventPolicy `hcl:"event,expand"`
EventPrefixes []*EventPolicy `hcl:"event_prefix,expand"`
PreparedQueries []*PreparedQueryPolicy `hcl:"query,expand"`
PreparedQueryPrefixes []*PreparedQueryPolicy `hcl:"query_prefix,expand"`
Keyring string `hcl:"keyring"`
Operator string `hcl:"operator"`
}
// Sentinel defines a snippet of Sentinel code that can be attached to a policy.
@ -155,27 +176,29 @@ func isSentinelValid(sentinel sentinel.Evaluator, basicPolicy string, sp Sentine
return sentinel.Compile(sp.Code)
}
// Parse is used to parse the specified ACL rules into an
// intermediary set of policies, before being compiled into
// the ACL
func Parse(rules string, sentinel sentinel.Evaluator) (*Policy, error) {
// Decode the rules
func parseCurrent(rules string, sentinel sentinel.Evaluator) (*Policy, error) {
p := &Policy{}
if rules == "" {
// Hot path for empty rules
return p, nil
}
if err := hcl.Decode(p, rules); err != nil {
return nil, fmt.Errorf("Failed to parse ACL rules: %v", err)
}
// Validate the acl policy
if p.ACL != "" && !isPolicyValid(p.ACL) {
return nil, fmt.Errorf("Invalid acl policy: %#v", p.ACL)
}
// Validate the agent policy
for _, ap := range p.Agents {
if !isPolicyValid(ap.Policy) {
return nil, fmt.Errorf("Invalid agent policy: %#v", ap)
}
}
for _, ap := range p.AgentPrefixes {
if !isPolicyValid(ap.Policy) {
return nil, fmt.Errorf("Invalid agent_prefix policy: %#v", ap)
}
}
// Validate the key policy
for _, kp := range p.Keys {
@ -186,6 +209,14 @@ func Parse(rules string, sentinel sentinel.Evaluator) (*Policy, error) {
return nil, fmt.Errorf("Invalid key Sentinel policy: %#v, got error:%v", kp, err)
}
}
for _, kp := range p.KeyPrefixes {
if kp.Policy != PolicyList && !isPolicyValid(kp.Policy) {
return nil, fmt.Errorf("Invalid key_prefix policy: %#v", kp)
}
if err := isSentinelValid(sentinel, kp.Policy, kp.Sentinel); err != nil {
return nil, fmt.Errorf("Invalid key_prefix Sentinel policy: %#v, got error:%v", kp, err)
}
}
// Validate the node policies
for _, np := range p.Nodes {
@ -196,6 +227,14 @@ func Parse(rules string, sentinel sentinel.Evaluator) (*Policy, error) {
return nil, fmt.Errorf("Invalid node Sentinel policy: %#v, got error:%v", np, err)
}
}
for _, np := range p.NodePrefixes {
if !isPolicyValid(np.Policy) {
return nil, fmt.Errorf("Invalid node_prefix policy: %#v", np)
}
if err := isSentinelValid(sentinel, np.Policy, np.Sentinel); err != nil {
return nil, fmt.Errorf("Invalid node_prefix Sentinel policy: %#v, got error:%v", np, err)
}
}
// Validate the service policies
for _, sp := range p.Services {
@ -209,6 +248,17 @@ func Parse(rules string, sentinel sentinel.Evaluator) (*Policy, error) {
return nil, fmt.Errorf("Invalid service Sentinel policy: %#v, got error:%v", sp, err)
}
}
for _, sp := range p.ServicePrefixes {
if !isPolicyValid(sp.Policy) {
return nil, fmt.Errorf("Invalid service_prefix policy: %#v", sp)
}
if sp.Intentions != "" && !isPolicyValid(sp.Intentions) {
return nil, fmt.Errorf("Invalid service_prefix intentions policy: %#v", sp)
}
if err := isSentinelValid(sentinel, sp.Policy, sp.Sentinel); err != nil {
return nil, fmt.Errorf("Invalid service_prefix Sentinel policy: %#v, got error:%v", sp, err)
}
}
// Validate the session policies
for _, sp := range p.Sessions {
@ -216,6 +266,11 @@ func Parse(rules string, sentinel sentinel.Evaluator) (*Policy, error) {
return nil, fmt.Errorf("Invalid session policy: %#v", sp)
}
}
for _, sp := range p.SessionPrefixes {
if !isPolicyValid(sp.Policy) {
return nil, fmt.Errorf("Invalid session_prefix policy: %#v", sp)
}
}
// Validate the user event policies
for _, ep := range p.Events {
@ -223,6 +278,11 @@ func Parse(rules string, sentinel sentinel.Evaluator) (*Policy, error) {
return nil, fmt.Errorf("Invalid event policy: %#v", ep)
}
}
for _, ep := range p.EventPrefixes {
if !isPolicyValid(ep.Policy) {
return nil, fmt.Errorf("Invalid event_prefix policy: %#v", ep)
}
}
// Validate the prepared query policies
for _, pq := range p.PreparedQueries {
@ -230,6 +290,11 @@ func Parse(rules string, sentinel sentinel.Evaluator) (*Policy, error) {
return nil, fmt.Errorf("Invalid query policy: %#v", pq)
}
}
for _, pq := range p.PreparedQueryPrefixes {
if !isPolicyValid(pq.Policy) {
return nil, fmt.Errorf("Invalid query_prefix policy: %#v", pq)
}
}
// Validate the keyring policy - this one is allowed to be empty
if p.Keyring != "" && !isPolicyValid(p.Keyring) {
@ -243,3 +308,543 @@ func Parse(rules string, sentinel sentinel.Evaluator) (*Policy, error) {
return p, nil
}
func parseLegacy(rules string, sentinel sentinel.Evaluator) (*Policy, error) {
p := &Policy{}
type LegacyPolicy struct {
Agents []*AgentPolicy `hcl:"agent,expand"`
Keys []*KeyPolicy `hcl:"key,expand"`
Nodes []*NodePolicy `hcl:"node,expand"`
Services []*ServicePolicy `hcl:"service,expand"`
Sessions []*SessionPolicy `hcl:"session,expand"`
Events []*EventPolicy `hcl:"event,expand"`
PreparedQueries []*PreparedQueryPolicy `hcl:"query,expand"`
Keyring string `hcl:"keyring"`
Operator string `hcl:"operator"`
}
lp := &LegacyPolicy{}
if err := hcl.Decode(lp, rules); err != nil {
return nil, fmt.Errorf("Failed to parse ACL rules: %v", err)
}
// Validate the agent policy
for _, ap := range lp.Agents {
if !isPolicyValid(ap.Policy) {
return nil, fmt.Errorf("Invalid agent policy: %#v", ap)
}
p.AgentPrefixes = append(p.AgentPrefixes, ap)
}
// Validate the key policy
for _, kp := range lp.Keys {
if kp.Policy != PolicyList && !isPolicyValid(kp.Policy) {
return nil, fmt.Errorf("Invalid key policy: %#v", kp)
}
if err := isSentinelValid(sentinel, kp.Policy, kp.Sentinel); err != nil {
return nil, fmt.Errorf("Invalid key Sentinel policy: %#v, got error:%v", kp, err)
}
p.KeyPrefixes = append(p.KeyPrefixes, kp)
}
// Validate the node policies
for _, np := range lp.Nodes {
if !isPolicyValid(np.Policy) {
return nil, fmt.Errorf("Invalid node policy: %#v", np)
}
if err := isSentinelValid(sentinel, np.Policy, np.Sentinel); err != nil {
return nil, fmt.Errorf("Invalid node Sentinel policy: %#v, got error:%v", np, err)
}
p.NodePrefixes = append(p.NodePrefixes, np)
}
// Validate the service policies
for _, sp := range lp.Services {
if !isPolicyValid(sp.Policy) {
return nil, fmt.Errorf("Invalid service policy: %#v", sp)
}
if sp.Intentions != "" && !isPolicyValid(sp.Intentions) {
return nil, fmt.Errorf("Invalid service intentions policy: %#v", sp)
}
if err := isSentinelValid(sentinel, sp.Policy, sp.Sentinel); err != nil {
return nil, fmt.Errorf("Invalid service Sentinel policy: %#v, got error:%v", sp, err)
}
p.ServicePrefixes = append(p.ServicePrefixes, sp)
}
// Validate the session policies
for _, sp := range lp.Sessions {
if !isPolicyValid(sp.Policy) {
return nil, fmt.Errorf("Invalid session policy: %#v", sp)
}
p.SessionPrefixes = append(p.SessionPrefixes, sp)
}
// Validate the user event policies
for _, ep := range lp.Events {
if !isPolicyValid(ep.Policy) {
return nil, fmt.Errorf("Invalid event policy: %#v", ep)
}
p.EventPrefixes = append(p.EventPrefixes, ep)
}
// Validate the prepared query policies
for _, pq := range lp.PreparedQueries {
if !isPolicyValid(pq.Policy) {
return nil, fmt.Errorf("Invalid query policy: %#v", pq)
}
p.PreparedQueryPrefixes = append(p.PreparedQueryPrefixes, pq)
}
// Validate the keyring policy - this one is allowed to be empty
if lp.Keyring != "" && !isPolicyValid(lp.Keyring) {
return nil, fmt.Errorf("Invalid keyring policy: %#v", lp.Keyring)
} else {
p.Keyring = lp.Keyring
}
// Validate the operator policy - this one is allowed to be empty
if lp.Operator != "" && !isPolicyValid(lp.Operator) {
return nil, fmt.Errorf("Invalid operator policy: %#v", lp.Operator)
} else {
p.Operator = lp.Operator
}
return p, nil
}
// NewPolicyFromSource is used to parse the specified ACL rules into an
// intermediary set of policies, before being compiled into
// the ACL
func NewPolicyFromSource(id string, revision uint64, rules string, syntax SyntaxVersion, sentinel sentinel.Evaluator) (*Policy, error) {
if rules == "" {
// Hot path for empty source
return &Policy{ID: id, Revision: revision}, nil
}
var policy *Policy
var err error
switch syntax {
case SyntaxLegacy:
policy, err = parseLegacy(rules, sentinel)
case SyntaxCurrent:
policy, err = parseCurrent(rules, sentinel)
default:
return nil, fmt.Errorf("Invalid rules version: %d", syntax)
}
if err == nil {
policy.ID = id
policy.Revision = revision
}
return policy, err
}
func (policy *Policy) ConvertToLegacy() *Policy {
converted := &Policy{
ID: policy.ID,
Revision: policy.Revision,
ACL: policy.ACL,
Keyring: policy.Keyring,
Operator: policy.Operator,
}
converted.Agents = append(converted.Agents, policy.Agents...)
converted.Agents = append(converted.Agents, policy.AgentPrefixes...)
converted.Keys = append(converted.Keys, policy.Keys...)
converted.Keys = append(converted.Keys, policy.KeyPrefixes...)
converted.Nodes = append(converted.Nodes, policy.Nodes...)
converted.Nodes = append(converted.Nodes, policy.NodePrefixes...)
converted.Services = append(converted.Services, policy.Services...)
converted.Services = append(converted.Services, policy.ServicePrefixes...)
converted.Sessions = append(converted.Sessions, policy.Sessions...)
converted.Sessions = append(converted.Sessions, policy.SessionPrefixes...)
converted.Events = append(converted.Events, policy.Events...)
converted.Events = append(converted.Events, policy.EventPrefixes...)
converted.PreparedQueries = append(converted.PreparedQueries, policy.PreparedQueries...)
converted.PreparedQueries = append(converted.PreparedQueries, policy.PreparedQueryPrefixes...)
return converted
}
func (policy *Policy) ConvertFromLegacy() *Policy {
return &Policy{
ID: policy.ID,
Revision: policy.Revision,
AgentPrefixes: policy.Agents,
KeyPrefixes: policy.Keys,
NodePrefixes: policy.Nodes,
ServicePrefixes: policy.Services,
SessionPrefixes: policy.Sessions,
EventPrefixes: policy.Events,
PreparedQueryPrefixes: policy.PreparedQueries,
Keyring: policy.Keyring,
Operator: policy.Operator,
}
}
// takesPrecedenceOver returns true when permission a
// should take precedence over permission b
func takesPrecedenceOver(a, b string) bool {
if a == PolicyDeny {
return true
} else if b == PolicyDeny {
return false
}
if a == PolicyWrite {
return true
} else if b == PolicyWrite {
return false
}
if a == PolicyList {
return true
} else if b == PolicyList {
return false
}
if a == PolicyRead {
return true
} else if b == PolicyRead {
return false
}
return false
}
func multiPolicyID(policies []*Policy) []byte {
cacheKeyHash, err := blake2b.New256(nil)
if err != nil {
panic(err)
}
for _, policy := range policies {
cacheKeyHash.Write([]byte(policy.ID))
binary.Write(cacheKeyHash, binary.BigEndian, policy.Revision)
}
return cacheKeyHash.Sum(nil)
}
// MergePolicies merges multiple ACL policies into one policy
// This function will not set either the ID or the Scope fields
// of the resulting policy as its up to the caller to determine
// what the merged value is.
func MergePolicies(policies []*Policy) *Policy {
// maps are used here so that we can lookup each policy by
// the segment that the rule applies to during the policy
// merge. Otherwise we could do a linear search through a slice
// and replace it inline
aclPolicy := ""
agentPolicies := make(map[string]*AgentPolicy)
agentPrefixPolicies := make(map[string]*AgentPolicy)
eventPolicies := make(map[string]*EventPolicy)
eventPrefixPolicies := make(map[string]*EventPolicy)
keyringPolicy := ""
keyPolicies := make(map[string]*KeyPolicy)
keyPrefixPolicies := make(map[string]*KeyPolicy)
nodePolicies := make(map[string]*NodePolicy)
nodePrefixPolicies := make(map[string]*NodePolicy)
operatorPolicy := ""
preparedQueryPolicies := make(map[string]*PreparedQueryPolicy)
preparedQueryPrefixPolicies := make(map[string]*PreparedQueryPolicy)
servicePolicies := make(map[string]*ServicePolicy)
servicePrefixPolicies := make(map[string]*ServicePolicy)
sessionPolicies := make(map[string]*SessionPolicy)
sessionPrefixPolicies := make(map[string]*SessionPolicy)
// Parse all the individual rule sets
for _, policy := range policies {
if takesPrecedenceOver(policy.ACL, aclPolicy) {
aclPolicy = policy.ACL
}
for _, ap := range policy.Agents {
update := true
if permission, found := agentPolicies[ap.Node]; found {
update = takesPrecedenceOver(ap.Policy, permission.Policy)
}
if update {
agentPolicies[ap.Node] = ap
}
}
for _, ap := range policy.AgentPrefixes {
update := true
if permission, found := agentPrefixPolicies[ap.Node]; found {
update = takesPrecedenceOver(ap.Policy, permission.Policy)
}
if update {
agentPrefixPolicies[ap.Node] = ap
}
}
for _, ep := range policy.Events {
update := true
if permission, found := eventPolicies[ep.Event]; found {
update = takesPrecedenceOver(ep.Policy, permission.Policy)
}
if update {
eventPolicies[ep.Event] = ep
}
}
for _, ep := range policy.EventPrefixes {
update := true
if permission, found := eventPrefixPolicies[ep.Event]; found {
update = takesPrecedenceOver(ep.Policy, permission.Policy)
}
if update {
eventPrefixPolicies[ep.Event] = ep
}
}
if takesPrecedenceOver(policy.Keyring, keyringPolicy) {
keyringPolicy = policy.Keyring
}
for _, kp := range policy.Keys {
update := true
if permission, found := keyPolicies[kp.Prefix]; found {
update = takesPrecedenceOver(kp.Policy, permission.Policy)
}
if update {
keyPolicies[kp.Prefix] = kp
}
}
for _, kp := range policy.KeyPrefixes {
update := true
if permission, found := keyPrefixPolicies[kp.Prefix]; found {
update = takesPrecedenceOver(kp.Policy, permission.Policy)
}
if update {
keyPrefixPolicies[kp.Prefix] = kp
}
}
for _, np := range policy.Nodes {
update := true
if permission, found := nodePolicies[np.Name]; found {
update = takesPrecedenceOver(np.Policy, permission.Policy)
}
if update {
nodePolicies[np.Name] = np
}
}
for _, np := range policy.NodePrefixes {
update := true
if permission, found := nodePrefixPolicies[np.Name]; found {
update = takesPrecedenceOver(np.Policy, permission.Policy)
}
if update {
nodePrefixPolicies[np.Name] = np
}
}
if takesPrecedenceOver(policy.Operator, operatorPolicy) {
operatorPolicy = policy.Operator
}
for _, qp := range policy.PreparedQueries {
update := true
if permission, found := preparedQueryPolicies[qp.Prefix]; found {
update = takesPrecedenceOver(qp.Policy, permission.Policy)
}
if update {
preparedQueryPolicies[qp.Prefix] = qp
}
}
for _, qp := range policy.PreparedQueryPrefixes {
update := true
if permission, found := preparedQueryPrefixPolicies[qp.Prefix]; found {
update = takesPrecedenceOver(qp.Policy, permission.Policy)
}
if update {
preparedQueryPrefixPolicies[qp.Prefix] = qp
}
}
for _, sp := range policy.Services {
existing, found := servicePolicies[sp.Name]
if !found {
servicePolicies[sp.Name] = sp
continue
}
if takesPrecedenceOver(sp.Policy, existing.Policy) {
existing.Policy = sp.Policy
existing.Sentinel = sp.Sentinel
}
if takesPrecedenceOver(sp.Intentions, existing.Intentions) {
existing.Intentions = sp.Intentions
}
}
for _, sp := range policy.ServicePrefixes {
existing, found := servicePrefixPolicies[sp.Name]
if !found {
servicePrefixPolicies[sp.Name] = sp
continue
}
if takesPrecedenceOver(sp.Policy, existing.Policy) {
existing.Policy = sp.Policy
existing.Sentinel = sp.Sentinel
}
if takesPrecedenceOver(sp.Intentions, existing.Intentions) {
existing.Intentions = sp.Intentions
}
}
for _, sp := range policy.Sessions {
update := true
if permission, found := sessionPolicies[sp.Node]; found {
update = takesPrecedenceOver(sp.Policy, permission.Policy)
}
if update {
sessionPolicies[sp.Node] = sp
}
}
for _, sp := range policy.SessionPrefixes {
update := true
if permission, found := sessionPrefixPolicies[sp.Node]; found {
update = takesPrecedenceOver(sp.Policy, permission.Policy)
}
if update {
sessionPrefixPolicies[sp.Node] = sp
}
}
}
merged := &Policy{ACL: aclPolicy, Keyring: keyringPolicy, Operator: operatorPolicy}
// All the for loop appends are ugly but Go doesn't have a way to get
// a slice of all values within a map so this is necessary
for _, policy := range agentPolicies {
merged.Agents = append(merged.Agents, policy)
}
for _, policy := range agentPrefixPolicies {
merged.AgentPrefixes = append(merged.AgentPrefixes, policy)
}
for _, policy := range eventPolicies {
merged.Events = append(merged.Events, policy)
}
for _, policy := range eventPrefixPolicies {
merged.EventPrefixes = append(merged.EventPrefixes, policy)
}
for _, policy := range keyPolicies {
merged.Keys = append(merged.Keys, policy)
}
for _, policy := range keyPrefixPolicies {
merged.KeyPrefixes = append(merged.KeyPrefixes, policy)
}
for _, policy := range nodePolicies {
merged.Nodes = append(merged.Nodes, policy)
}
for _, policy := range nodePrefixPolicies {
merged.NodePrefixes = append(merged.NodePrefixes, policy)
}
for _, policy := range preparedQueryPolicies {
merged.PreparedQueries = append(merged.PreparedQueries, policy)
}
for _, policy := range preparedQueryPrefixPolicies {
merged.PreparedQueryPrefixes = append(merged.PreparedQueryPrefixes, policy)
}
for _, policy := range servicePolicies {
merged.Services = append(merged.Services, policy)
}
for _, policy := range servicePrefixPolicies {
merged.ServicePrefixes = append(merged.ServicePrefixes, policy)
}
for _, policy := range sessionPolicies {
merged.Sessions = append(merged.Sessions, policy)
}
for _, policy := range sessionPrefixPolicies {
merged.SessionPrefixes = append(merged.SessionPrefixes, policy)
}
merged.ID = fmt.Sprintf("%x", multiPolicyID(policies))
return merged
}
func TranslateLegacyRules(policyBytes []byte) ([]byte, error) {
parsed, err := hcl.ParseBytes(policyBytes)
if err != nil {
return nil, fmt.Errorf("Failed to parse rules: %v", err)
}
rewritten := ast.Walk(parsed, func(node ast.Node) (ast.Node, bool) {
switch n := node.(type) {
case *ast.ObjectKey:
switch n.Token.Text {
case "agent":
n.Token.Text = "agent_prefix"
case "key":
n.Token.Text = "key_prefix"
case "node":
n.Token.Text = "node_prefix"
case "query":
n.Token.Text = "query_prefix"
case "service":
n.Token.Text = "service_prefix"
case "session":
n.Token.Text = "session_prefix"
case "event":
n.Token.Text = "event_prefix"
}
}
return node, true
})
buffer := new(bytes.Buffer)
if err := hclprinter.Fprint(buffer, rewritten); err != nil {
return nil, fmt.Errorf("Failed to output new rules: %v", err)
}
return buffer.Bytes(), nil
}

1894
acl/policy_test.go
View File

File diff suppressed because it is too large Load Diff

245
agent/acl.go
View File

@ -2,242 +2,65 @@ package agent
import (
"fmt"
"sync"
"time"
"github.com/armon/go-metrics"
"github.com/hashicorp/consul/acl"
"github.com/hashicorp/consul/agent/config"
"github.com/hashicorp/consul/agent/local"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/consul/types"
"github.com/hashicorp/golang-lru"
"github.com/hashicorp/serf/serf"
)
// There's enough behavior difference with client-side ACLs that we've
// intentionally kept this code separate from the server-side ACL code in
// consul/acl.go. We may refactor some of the caching logic in the future,
// but for now we are developing this separately to see how things shake out.
const (
// anonymousToken is the token ID we re-write to if there is no token ID
// provided.
anonymousToken = "anonymous"
// Maximum number of cached ACL entries.
aclCacheSize = 10 * 1024
)
// aclCacheEntry is used to cache ACL tokens.
type aclCacheEntry struct {
// ACL is the cached ACL.
ACL acl.ACL
// Expires is set based on the TTL for the ACL.
Expires time.Time
// ETag is used as an optimization when fetching ACLs from servers to
// avoid transmitting data back when the agent has a good copy, which is
// usually the case when refreshing a TTL.
ETag string
}
// aclManager is used by the agent to keep track of state related to ACLs,
// including caching tokens from the servers. This has some internal state that
// we don't want to dump into the agent itself.
type aclManager struct {
// acls is a cache mapping ACL tokens to compiled policies.
acls *lru.TwoQueueCache
// master is the ACL to use when the agent master token is supplied.
master acl.ACL
// down is the ACL to use when the servers are down. This may be nil
// which means to try and use the cached policy if there is one (or
// deny if there isn't a policy in the cache).
down acl.ACL
// disabled is used to keep track of feedback from the servers that ACLs
// are disabled. If the manager discovers that ACLs are disabled, this
// will be set to the next time we should check to see if they have been
// enabled. This helps cut useless traffic, but allows us to turn on ACL
// support at the servers without having to restart the whole cluster.
disabled time.Time
disabledLock sync.RWMutex
}
// newACLManager returns an ACL manager based on the given config.
func newACLManager(config *config.RuntimeConfig) (*aclManager, error) {
// Set up the cache from ID to ACL (we don't cache policies like the
// servers; only one level).
acls, err := lru.New2Q(aclCacheSize)
if err != nil {
return nil, err
// resolveToken is the primary interface used by ACL-checkers in the agent
// endpoints, which is the one place where we do some ACL enforcement on
// clients. Some of the enforcement is normative (e.g. self and monitor)
// and some is informative (e.g. catalog and health).
func (a *Agent) resolveToken(id string) (acl.Authorizer, error) {
// ACLs are disabled
if !a.delegate.ACLsEnabled() {
return nil, nil
}
// Disable ACLs if version 8 enforcement isn't enabled.
if !a.config.ACLEnforceVersion8 {
return nil, nil
}
if acl.RootAuthorizer(id) != nil {
return nil, acl.ErrRootDenied
}
if a.tokens.IsAgentMasterToken(id) {
return a.aclMasterAuthorizer, nil
}
return a.delegate.ResolveToken(id)
}
func (a *Agent) initializeACLs() error {
// Build a policy for the agent master token.
// The builtin agent master policy allows reading any node information
// and allows writes to the agent with the node name of the running agent
// only. This used to allow a prefix match on agent names but that seems
// entirely unnecessary so it is now using an exact match.
policy := &acl.Policy{
Agents: []*acl.AgentPolicy{
&acl.AgentPolicy{
Node: config.NodeName,
Node: a.config.NodeName,
Policy: acl.PolicyWrite,
},
},
Nodes: []*acl.NodePolicy{
NodePrefixes: []*acl.NodePolicy{
&acl.NodePolicy{
Name: "",
Policy: acl.PolicyRead,
},
},
}
master, err := acl.New(acl.DenyAll(), policy, nil)
master, err := acl.NewPolicyAuthorizer(acl.DenyAll(), []*acl.Policy{policy}, nil)
if err != nil {
return nil, err
return err
}
var down acl.ACL
switch config.ACLDownPolicy {
case "allow":
down = acl.AllowAll()
case "deny":
down = acl.DenyAll()
case "async-cache", "extend-cache":
// Leave the down policy as nil to signal this.
default:
return nil, fmt.Errorf("invalid ACL down policy %q", config.ACLDownPolicy)
}
// Give back a manager.
return &aclManager{
acls: acls,
master: master,
down: down,
}, nil
}
// isDisabled returns true if the manager has discovered that ACLs are disabled
// on the servers.
func (m *aclManager) isDisabled() bool {
m.disabledLock.RLock()
defer m.disabledLock.RUnlock()
return time.Now().Before(m.disabled)
}
// lookupACL attempts to locate the compiled policy associated with the given
// token. The agent may be used to perform RPC calls to the servers to fetch
// policies that aren't in the cache.
func (m *aclManager) lookupACL(a *Agent, id string) (acl.ACL, error) {
// Handle some special cases for the ID.
if len(id) == 0 {
id = anonymousToken
} else if acl.RootACL(id) != nil {
return nil, acl.ErrRootDenied
} else if a.tokens.IsAgentMasterToken(id) {
return m.master, nil
}
// Try the cache first.
var cached *aclCacheEntry
if raw, ok := m.acls.Get(id); ok {
cached = raw.(*aclCacheEntry)
}
if cached != nil && time.Now().Before(cached.Expires) {
metrics.IncrCounter([]string{"acl", "cache_hit"}, 1)
return cached.ACL, nil
}
metrics.IncrCounter([]string{"acl", "cache_miss"}, 1)
// At this point we might have a stale cached ACL, or none at all, so
// try to contact the servers.
args := structs.ACLPolicyRequest{
Datacenter: a.config.ACLDatacenter,
ACL: id,
}
if cached != nil {
args.ETag = cached.ETag
}
var reply structs.ACLPolicy
err := a.RPC("ACL.GetPolicy", &args, &reply)
if err != nil {
if acl.IsErrDisabled(err) {
a.logger.Printf("[DEBUG] agent: ACLs disabled on servers, will check again after %s", a.config.ACLDisabledTTL)
m.disabledLock.Lock()
m.disabled = time.Now().Add(a.config.ACLDisabledTTL)
m.disabledLock.Unlock()
return nil, nil
} else if acl.IsErrNotFound(err) {
return nil, acl.ErrNotFound
} else {
a.logger.Printf("[DEBUG] agent: Failed to get policy for ACL from servers: %v", err)
if m.down != nil {
return m.down, nil
} else if cached != nil {
return cached.ACL, nil
} else {
return acl.DenyAll(), nil
}
}
}
// Use the old cached compiled ACL if we can, otherwise compile it and
// resolve any parents.
var compiled acl.ACL
if cached != nil && cached.ETag == reply.ETag {
compiled = cached.ACL
} else {
parent := acl.RootACL(reply.Parent)
if parent == nil {
parent, err = m.lookupACL(a, reply.Parent)
if err != nil {
return nil, err
}
}
acl, err := acl.New(parent, reply.Policy, nil)
if err != nil {
return nil, err
}
compiled = acl
}
// Update the cache.
cached = &aclCacheEntry{
ACL: compiled,
ETag: reply.ETag,
}
if reply.TTL > 0 {
cached.Expires = time.Now().Add(reply.TTL)
}
m.acls.Add(id, cached)
return compiled, nil
}
// resolveToken is the primary interface used by ACL-checkers in the agent
// endpoints, which is the one place where we do some ACL enforcement on
// clients. Some of the enforcement is normative (e.g. self and monitor)
// and some is informative (e.g. catalog and health).
func (a *Agent) resolveToken(id string) (acl.ACL, error) {
// Disable ACLs if version 8 enforcement isn't enabled.
if !a.config.ACLEnforceVersion8 {
return nil, nil
}
// Bail if there's no ACL datacenter configured. This means that agent
// enforcement isn't on.
if a.config.ACLDatacenter == "" {
return nil, nil
}
// Bail if the ACL manager is disabled. This happens if it gets feedback
// from the servers that ACLs are disabled.
if a.acls.isDisabled() {
return nil, nil
}
// This will look in the cache and fetch from the servers if necessary.
return a.acls.lookupACL(a, id)
a.aclMasterAuthorizer = master
return nil
}
// resolveProxyToken attempts to resolve an ACL ID to a local proxy token.

659
agent/acl_endpoint.go
View File

@ -2,22 +2,26 @@ package agent
import (
"fmt"
"io/ioutil"
"net/http"
"strconv"
"strings"
"time"
"github.com/hashicorp/consul/acl"
"github.com/hashicorp/consul/agent/structs"
)
// aclCreateResponse is used to wrap the ACL ID
type aclCreateResponse struct {
type aclBootstrapResponse struct {
ID string
structs.ACLToken
}
// checkACLDisabled will return a standard response if ACLs are disabled. This
// returns true if they are disabled and we should not continue.
func (s *HTTPServer) checkACLDisabled(resp http.ResponseWriter, req *http.Request) bool {
if s.agent.config.ACLDatacenter != "" {
if s.agent.delegate.ACLsEnabled() {
return false
}
@ -34,211 +38,42 @@ func (s *HTTPServer) ACLBootstrap(resp http.ResponseWriter, req *http.Request) (
}
args := structs.DCSpecificRequest{
Datacenter: s.agent.config.ACLDatacenter,
Datacenter: s.agent.config.Datacenter,
}
var out structs.ACL
err := s.agent.RPC("ACL.Bootstrap", &args, &out)
if err != nil {
if strings.Contains(err.Error(), structs.ACLBootstrapNotAllowedErr.Error()) {
resp.WriteHeader(http.StatusForbidden)
fmt.Fprint(resp, acl.PermissionDeniedError{Cause: err.Error()}.Error())
return nil, nil
} else {
return nil, err
legacy := false
legacyStr := req.URL.Query().Get("legacy")
if legacyStr != "" {
legacy, _ = strconv.ParseBool(legacyStr)
}
if legacy && s.agent.delegate.UseLegacyACLs() {
var out structs.ACL
err := s.agent.RPC("ACL.Bootstrap", &args, &out)
if err != nil {
if strings.Contains(err.Error(), structs.ACLBootstrapNotAllowedErr.Error()) {
resp.WriteHeader(http.StatusForbidden)
fmt.Fprint(resp, acl.PermissionDeniedError{Cause: err.Error()}.Error())
return nil, nil
} else {
return nil, err
}
}
}
return aclCreateResponse{out.ID}, nil
}
func (s *HTTPServer) ACLDestroy(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
args := structs.ACLRequest{
Datacenter: s.agent.config.ACLDatacenter,
Op: structs.ACLDelete,
}
s.parseToken(req, &args.Token)
// Pull out the acl id
args.ACL.ID = strings.TrimPrefix(req.URL.Path, "/v1/acl/destroy/")
if args.ACL.ID == "" {
resp.WriteHeader(http.StatusBadRequest)
fmt.Fprint(resp, "Missing ACL")
return nil, nil
}
var out string
if err := s.agent.RPC("ACL.Apply", &args, &out); err != nil {
return nil, err
}
return true, nil
}
func (s *HTTPServer) ACLCreate(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
return s.aclSet(resp, req, false)
}
func (s *HTTPServer) ACLUpdate(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
return s.aclSet(resp, req, true)
}
func (s *HTTPServer) aclSet(resp http.ResponseWriter, req *http.Request, update bool) (interface{}, error) {
args := structs.ACLRequest{
Datacenter: s.agent.config.ACLDatacenter,
Op: structs.ACLSet,
ACL: structs.ACL{
Type: structs.ACLTypeClient,
},
}
s.parseToken(req, &args.Token)
// Handle optional request body
if req.ContentLength > 0 {
if err := decodeBody(req, &args.ACL, nil); err != nil {
resp.WriteHeader(http.StatusBadRequest)
fmt.Fprintf(resp, "Request decode failed: %v", err)
return nil, nil
return &aclBootstrapResponse{ID: out.ID}, nil
} else {
var out structs.ACLToken
err := s.agent.RPC("ACL.BootstrapTokens", &args, &out)
if err != nil {
if strings.Contains(err.Error(), structs.ACLBootstrapNotAllowedErr.Error()) {
resp.WriteHeader(http.StatusForbidden)
fmt.Fprint(resp, acl.PermissionDeniedError{Cause: err.Error()}.Error())
return nil, nil
} else {
return nil, err
}
}
return &aclBootstrapResponse{ID: out.SecretID, ACLToken: out}, nil
}
// Ensure there is an ID set for update. ID is optional for
// create, as one will be generated if not provided.
if update && args.ACL.ID == "" {
resp.WriteHeader(http.StatusBadRequest)
fmt.Fprint(resp, "ACL ID must be set")
return nil, nil
}
// Create the acl, get the ID
var out string
if err := s.agent.RPC("ACL.Apply", &args, &out); err != nil {
return nil, err
}
// Format the response as a JSON object
return aclCreateResponse{out}, nil
}
func (s *HTTPServer) ACLClone(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
args := structs.ACLSpecificRequest{
Datacenter: s.agent.config.ACLDatacenter,
}
var dc string
if done := s.parse(resp, req, &dc, &args.QueryOptions); done {
return nil, nil
}
// Pull out the acl id
args.ACL = strings.TrimPrefix(req.URL.Path, "/v1/acl/clone/")
if args.ACL == "" {
resp.WriteHeader(http.StatusBadRequest)
fmt.Fprint(resp, "Missing ACL")
return nil, nil
}
var out structs.IndexedACLs
defer setMeta(resp, &out.QueryMeta)
if err := s.agent.RPC("ACL.Get", &args, &out); err != nil {
return nil, err
}
// Bail if the ACL is not found, this could be a 404 or a 403, so
// always just return a 403.
if len(out.ACLs) == 0 {
return nil, acl.ErrPermissionDenied
}
// Create a new ACL
createArgs := structs.ACLRequest{
Datacenter: args.Datacenter,
Op: structs.ACLSet,
ACL: *out.ACLs[0],
}
createArgs.ACL.ID = ""
createArgs.Token = args.Token
// Create the acl, get the ID
var outID string
if err := s.agent.RPC("ACL.Apply", &createArgs, &outID); err != nil {
return nil, err
}
// Format the response as a JSON object
return aclCreateResponse{outID}, nil
}
func (s *HTTPServer) ACLGet(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
args := structs.ACLSpecificRequest{
Datacenter: s.agent.config.ACLDatacenter,
}
var dc string
if done := s.parse(resp, req, &dc, &args.QueryOptions); done {
return nil, nil
}
// Pull out the acl id
args.ACL = strings.TrimPrefix(req.URL.Path, "/v1/acl/info/")
if args.ACL == "" {
resp.WriteHeader(http.StatusBadRequest)
fmt.Fprint(resp, "Missing ACL")
return nil, nil
}
var out structs.IndexedACLs
defer setMeta(resp, &out.QueryMeta)
if err := s.agent.RPC("ACL.Get", &args, &out); err != nil {
return nil, err
}
// Use empty list instead of nil
if out.ACLs == nil {
out.ACLs = make(structs.ACLs, 0)
}
return out.ACLs, nil
}
func (s *HTTPServer) ACLList(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
args := structs.DCSpecificRequest{
Datacenter: s.agent.config.ACLDatacenter,
}
var dc string
if done := s.parse(resp, req, &dc, &args.QueryOptions); done {
return nil, nil
}
var out structs.IndexedACLs
defer setMeta(resp, &out.QueryMeta)
if err := s.agent.RPC("ACL.List", &args, &out); err != nil {
return nil, err
}
// Use empty list instead of nil
if out.ACLs == nil {
out.ACLs = make(structs.ACLs, 0)
}
return out.ACLs, nil
}
func (s *HTTPServer) ACLReplicationStatus(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
@ -261,3 +96,423 @@ func (s *HTTPServer) ACLReplicationStatus(resp http.ResponseWriter, req *http.Re
}
return out, nil
}
func (s *HTTPServer) ACLRulesTranslate(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
policyBytes, err := ioutil.ReadAll(req.Body)
if err != nil {
return nil, BadRequestError{Reason: fmt.Sprintf("Failed to read body: %v", err)}
}
translated, err := acl.TranslateLegacyRules(policyBytes)
if err != nil {
return nil, BadRequestError{Reason: err.Error()}
}
resp.Write(translated)
return nil, nil
}
func (s *HTTPServer) ACLRulesTranslateLegacyToken(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
tokenID := strings.TrimPrefix(req.URL.Path, "/v1/acl/rules/translate/")
if tokenID == "" {
return nil, BadRequestError{Reason: "Missing token ID"}
}
args := structs.ACLTokenReadRequest{
Datacenter: s.agent.config.Datacenter,
TokenID: tokenID,
TokenIDType: structs.ACLTokenAccessor,
}
if done := s.parse(resp, req, &args.Datacenter, &args.QueryOptions); done {
return nil, nil
}
if args.Datacenter == "" {
args.Datacenter = s.agent.config.Datacenter
}
// Do not allow blocking
args.QueryOptions.MinQueryIndex = 0
var out structs.ACLTokenResponse
defer setMeta(resp, &out.QueryMeta)
if err := s.agent.RPC("ACL.TokenRead", &args, &out); err != nil {
return nil, err
}
if out.Token == nil {
return nil, acl.ErrNotFound
}
if out.Token.Rules == "" {
return nil, fmt.Errorf("The specified token does not have any rules set")
}
translated, err := acl.TranslateLegacyRules([]byte(out.Token.Rules))
if err != nil {
return nil, fmt.Errorf("Failed to parse legacy rules: %v", err)
}
resp.Write(translated)
return nil, nil
}
func (s *HTTPServer) ACLPolicyList(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
var args structs.ACLPolicyListRequest
if done := s.parse(resp, req, &args.Datacenter, &args.QueryOptions); done {
return nil, nil
}
if args.Datacenter == "" {
args.Datacenter = s.agent.config.Datacenter
}
var out structs.ACLPolicyListResponse
defer setMeta(resp, &out.QueryMeta)
if err := s.agent.RPC("ACL.PolicyList", &args, &out); err != nil {
return nil, err
}
// make sure we return an array and not nil
if out.Policies == nil {
out.Policies = make(structs.ACLPolicyListStubs, 0)
}
return out.Policies, nil
}
func (s *HTTPServer) ACLPolicyCRUD(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
var fn func(resp http.ResponseWriter, req *http.Request, policyID string) (interface{}, error)
switch req.Method {
case "GET":
fn = s.ACLPolicyRead
case "PUT":
fn = s.ACLPolicyWrite
case "DELETE":
fn = s.ACLPolicyDelete
default:
return nil, MethodNotAllowedError{req.Method, []string{"GET", "PUT", "DELETE"}}
}
policyID := strings.TrimPrefix(req.URL.Path, "/v1/acl/policy/")
if policyID == "" && req.Method != "PUT" {
return nil, BadRequestError{Reason: "Missing policy ID"}
}
return fn(resp, req, policyID)
}
func (s *HTTPServer) ACLPolicyRead(resp http.ResponseWriter, req *http.Request, policyID string) (interface{}, error) {
args := structs.ACLPolicyReadRequest{
Datacenter: s.agent.config.Datacenter,
PolicyID: policyID,
}
if done := s.parse(resp, req, &args.Datacenter, &args.QueryOptions); done {
return nil, nil
}
if args.Datacenter == "" {
args.Datacenter = s.agent.config.Datacenter
}
var out structs.ACLPolicyResponse
defer setMeta(resp, &out.QueryMeta)
if err := s.agent.RPC("ACL.PolicyRead", &args, &out); err != nil {
return nil, err
}
if out.Policy == nil {
return nil, acl.ErrNotFound
}
return out.Policy, nil
}
func (s *HTTPServer) ACLPolicyCreate(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
return s.ACLPolicyWrite(resp, req, "")
}
// fixCreateTimeAndHash is used to help in decoding the CreateTime and Hash
// attributes from the ACL Token create/update requests. It is needed
// to help mapstructure decode things properly when decodeBody is used.
func fixCreateTimeAndHash(raw interface{}) error {
rawMap, ok := raw.(map[string]interface{})
if !ok {
return nil
}
if val, ok := rawMap["CreateTime"]; ok {
if sval, ok := val.(string); ok {
t, err := time.Parse(time.RFC3339, sval)
if err != nil {
return err
}
rawMap["CreateTime"] = t
}
}
if val, ok := rawMap["Hash"]; ok {
if sval, ok := val.(string); ok {
rawMap["Hash"] = []byte(sval)
}
}
return nil
}
func (s *HTTPServer) ACLPolicyWrite(resp http.ResponseWriter, req *http.Request, policyID string) (interface{}, error) {
args := structs.ACLPolicyUpsertRequest{
Datacenter: s.agent.config.Datacenter,
}
s.parseToken(req, &args.Token)
if err := decodeBody(req, &args.Policy, nil); err != nil {
return nil, BadRequestError{Reason: fmt.Sprintf("Policy decoding failed: %v", err)}
}
args.Policy.Syntax = acl.SyntaxCurrent
if args.Policy.ID != "" && args.Policy.ID != policyID {
return nil, BadRequestError{Reason: "Policy ID in URL and payload do not match"}
} else if args.Policy.ID == "" {
args.Policy.ID = policyID
}
var out structs.ACLPolicy
if err := s.agent.RPC("ACL.PolicyUpsert", args, &out); err != nil {
return nil, err
}
return &out, nil
}
func (s *HTTPServer) ACLPolicyDelete(resp http.ResponseWriter, req *http.Request, policyID string) (interface{}, error) {
args := structs.ACLPolicyDeleteRequest{
Datacenter: s.agent.config.Datacenter,
PolicyID: policyID,
}
s.parseToken(req, &args.Token)
var out string
if err := s.agent.RPC("ACL.PolicyDelete", args, &out); err != nil {
return nil, err
}
return true, nil
}
func (s *HTTPServer) ACLTokenList(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
args := &structs.ACLTokenListRequest{
IncludeLocal: true,
IncludeGlobal: true,
}
if done := s.parse(resp, req, &args.Datacenter, &args.QueryOptions); done {
return nil, nil
}
if args.Datacenter == "" {
args.Datacenter = s.agent.config.Datacenter
}
args.Policy = req.URL.Query().Get("policy")
var out structs.ACLTokenListResponse
defer setMeta(resp, &out.QueryMeta)
if err := s.agent.RPC("ACL.TokenList", &args, &out); err != nil {
return nil, err
}
return out.Tokens, nil
}
func (s *HTTPServer) ACLTokenCRUD(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
var fn func(resp http.ResponseWriter, req *http.Request, tokenID string) (interface{}, error)
switch req.Method {
case "GET":
fn = s.ACLTokenRead
case "PUT":
fn = s.ACLTokenWrite
case "DELETE":
fn = s.ACLTokenDelete
default:
return nil, MethodNotAllowedError{req.Method, []string{"GET", "PUT", "DELETE"}}
}
tokenID := strings.TrimPrefix(req.URL.Path, "/v1/acl/token/")
if strings.HasSuffix(tokenID, "/clone") && req.Method == "PUT" {
tokenID = tokenID[:len(tokenID)-6]
fn = s.ACLTokenClone
}
if tokenID == "" && req.Method != "PUT" {
return nil, BadRequestError{Reason: "Missing token ID"}
}
return fn(resp, req, tokenID)
}
func (s *HTTPServer) ACLTokenSelf(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
args := structs.ACLTokenReadRequest{
TokenIDType: structs.ACLTokenSecret,
}
if done := s.parse(resp, req, &args.Datacenter, &args.QueryOptions); done {
return nil, nil
}
// copy the token parameter to the ID
args.TokenID = args.Token
if args.Datacenter == "" {
args.Datacenter = s.agent.config.Datacenter
}
var out structs.ACLTokenResponse
defer setMeta(resp, &out.QueryMeta)
if err := s.agent.RPC("ACL.TokenRead", &args, &out); err != nil {
return nil, err
}
if out.Token == nil {
return nil, acl.ErrNotFound
}
return out.Token, nil
}
func (s *HTTPServer) ACLTokenCreate(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
return s.ACLTokenWrite(resp, req, "")
}
func (s *HTTPServer) ACLTokenRead(resp http.ResponseWriter, req *http.Request, tokenID string) (interface{}, error) {
args := structs.ACLTokenReadRequest{
Datacenter: s.agent.config.Datacenter,
TokenID: tokenID,
TokenIDType: structs.ACLTokenAccessor,
}
if done := s.parse(resp, req, &args.Datacenter, &args.QueryOptions); done {
return nil, nil
}
if args.Datacenter == "" {
args.Datacenter = s.agent.config.Datacenter
}
var out structs.ACLTokenResponse
defer setMeta(resp, &out.QueryMeta)
if err := s.agent.RPC("ACL.TokenRead", &args, &out); err != nil {
return nil, err
}
if out.Token == nil {
return nil, acl.ErrNotFound
}
return out.Token, nil
}
func (s *HTTPServer) ACLTokenWrite(resp http.ResponseWriter, req *http.Request, tokenID string) (interface{}, error) {
args := structs.ACLTokenUpsertRequest{
Datacenter: s.agent.config.Datacenter,
}
s.parseToken(req, &args.Token)
if err := decodeBody(req, &args.ACLToken, fixCreateTimeAndHash); err != nil {
return nil, BadRequestError{Reason: fmt.Sprintf("Token decoding failed: %v", err)}
}
if args.ACLToken.AccessorID != "" && args.ACLToken.AccessorID != tokenID {
return nil, BadRequestError{Reason: "Token Accessor ID in URL and payload do not match"}
} else if args.ACLToken.AccessorID == "" {
args.ACLToken.AccessorID = tokenID
}
var out structs.ACLToken
if err := s.agent.RPC("ACL.TokenUpsert", args, &out); err != nil {
return nil, err
}
return &out, nil
}
func (s *HTTPServer) ACLTokenDelete(resp http.ResponseWriter, req *http.Request, tokenID string) (interface{}, error) {
args := structs.ACLTokenDeleteRequest{
Datacenter: s.agent.config.Datacenter,
TokenID: tokenID,
}
s.parseToken(req, &args.Token)
var out string
if err := s.agent.RPC("ACL.TokenDelete", args, &out); err != nil {
return nil, err
}
return true, nil
}
func (s *HTTPServer) ACLTokenClone(resp http.ResponseWriter, req *http.Request, tokenID string) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
args := structs.ACLTokenUpsertRequest{
Datacenter: s.agent.config.Datacenter,
}
if err := decodeBody(req, &args.ACLToken, fixCreateTimeAndHash); err != nil && err.Error() != "EOF" {
return nil, BadRequestError{Reason: fmt.Sprintf("Token decoding failed: %v", err)}
}
s.parseToken(req, &args.Token)
// Set this for the ID to clone
args.ACLToken.AccessorID = tokenID
var out structs.ACLToken
if err := s.agent.RPC("ACL.TokenClone", args, &out); err != nil {
return nil, err
}
return &out, nil
}

203
agent/acl_endpoint_legacy.go Normal file
View File

@ -0,0 +1,203 @@
package agent
import (
"fmt"
"net/http"
"strings"
"github.com/hashicorp/consul/acl"
"github.com/hashicorp/consul/agent/structs"
)
type aclCreateResponse struct {
ID string
}
func (s *HTTPServer) ACLDestroy(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
args := structs.ACLRequest{
Datacenter: s.agent.config.ACLDatacenter,
Op: structs.ACLDelete,
}
s.parseToken(req, &args.Token)
// Pull out the acl id
args.ACL.ID = strings.TrimPrefix(req.URL.Path, "/v1/acl/destroy/")
if args.ACL.ID == "" {
resp.WriteHeader(http.StatusBadRequest)
fmt.Fprint(resp, "Missing ACL")
return nil, nil
}
var out string
if err := s.agent.RPC("ACL.Apply", &args, &out); err != nil {
return nil, err
}
return true, nil
}
func (s *HTTPServer) ACLCreate(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
return s.aclSet(resp, req, false)
}
func (s *HTTPServer) ACLUpdate(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
return s.aclSet(resp, req, true)
}
func (s *HTTPServer) aclSet(resp http.ResponseWriter, req *http.Request, update bool) (interface{}, error) {
args := structs.ACLRequest{
Datacenter: s.agent.config.ACLDatacenter,
Op: structs.ACLSet,
ACL: structs.ACL{
Type: structs.ACLTokenTypeClient,
},
}
s.parseToken(req, &args.Token)
// Handle optional request body
if req.ContentLength > 0 {
if err := decodeBody(req, &args.ACL, nil); err != nil {
resp.WriteHeader(http.StatusBadRequest)
fmt.Fprintf(resp, "Request decode failed: %v", err)
return nil, nil
}
}
// Ensure there is an ID set for update. ID is optional for
// create, as one will be generated if not provided.
if update && args.ACL.ID == "" {
resp.WriteHeader(http.StatusBadRequest)
fmt.Fprint(resp, "ACL ID must be set")
return nil, nil
}
// Create the acl, get the ID
var out string
if err := s.agent.RPC("ACL.Apply", &args, &out); err != nil {
return nil, err
}
// Format the response as a JSON object
return aclCreateResponse{out}, nil
}
func (s *HTTPServer) ACLClone(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
args := structs.ACLSpecificRequest{
Datacenter: s.agent.config.ACLDatacenter,
}
var dc string
if done := s.parse(resp, req, &dc, &args.QueryOptions); done {
return nil, nil
}
// Pull out the acl id
args.ACL = strings.TrimPrefix(req.URL.Path, "/v1/acl/clone/")
if args.ACL == "" {
resp.WriteHeader(http.StatusBadRequest)
fmt.Fprint(resp, "Missing ACL")
return nil, nil
}
var out structs.IndexedACLs
defer setMeta(resp, &out.QueryMeta)
if err := s.agent.RPC("ACL.Get", &args, &out); err != nil {
return nil, err
}
// Bail if the ACL is not found, this could be a 404 or a 403, so
// always just return a 403.
if len(out.ACLs) == 0 {
return nil, acl.ErrPermissionDenied
}
// Create a new ACL
createArgs := structs.ACLRequest{
Datacenter: args.Datacenter,
Op: structs.ACLSet,
ACL: *out.ACLs[0],
}
createArgs.ACL.ID = ""
createArgs.Token = args.Token
// Create the acl, get the ID
var outID string
if err := s.agent.RPC("ACL.Apply", &createArgs, &outID); err != nil {
return nil, err
}
// Format the response as a JSON object
return aclCreateResponse{outID}, nil
}
func (s *HTTPServer) ACLGet(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
args := structs.ACLSpecificRequest{
Datacenter: s.agent.config.ACLDatacenter,
}
var dc string
if done := s.parse(resp, req, &dc, &args.QueryOptions); done {
return nil, nil
}
// Pull out the acl id
args.ACL = strings.TrimPrefix(req.URL.Path, "/v1/acl/info/")
if args.ACL == "" {
resp.WriteHeader(http.StatusBadRequest)
fmt.Fprint(resp, "Missing ACL")
return nil, nil
}
var out structs.IndexedACLs
defer setMeta(resp, &out.QueryMeta)
if err := s.agent.RPC("ACL.Get", &args, &out); err != nil {
return nil, err
}
// Use empty list instead of nil
if out.ACLs == nil {
out.ACLs = make(structs.ACLs, 0)
}
return out.ACLs, nil
}
func (s *HTTPServer) ACLList(resp http.ResponseWriter, req *http.Request) (interface{}, error) {
if s.checkACLDisabled(resp, req) {
return nil, nil
}
args := structs.DCSpecificRequest{
Datacenter: s.agent.config.ACLDatacenter,
}
var dc string
if done := s.parse(resp, req, &dc, &args.QueryOptions); done {
return nil, nil
}
var out structs.IndexedACLs
defer setMeta(resp, &out.QueryMeta)
if err := s.agent.RPC("ACL.List", &args, &out); err != nil {
return nil, err
}
// Use empty list instead of nil
if out.ACLs == nil {
out.ACLs = make(structs.ACLs, 0)
}
return out.ACLs, nil
}

297
agent/acl_endpoint_legacy_test.go Normal file
View File

@ -0,0 +1,297 @@
package agent
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/hashicorp/consul/acl"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/consul/testrpc"
)
func TestACL_Legacy_Disabled_Response(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), "")
defer a.Shutdown()
tests := []func(resp http.ResponseWriter, req *http.Request) (interface{}, error){
a.srv.ACLDestroy,
a.srv.ACLCreate,
a.srv.ACLUpdate,
a.srv.ACLClone,
a.srv.ACLGet,
a.srv.ACLList,
}
testrpc.WaitForLeader(t, a.RPC, "dc1")
for i, tt := range tests {
t.Run(fmt.Sprintf("%d", i), func(t *testing.T) {
req, _ := http.NewRequest("PUT", "/should/not/care", nil)
resp := httptest.NewRecorder()
obj, err := tt(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
if obj != nil {
t.Fatalf("bad: %#v", obj)
}
if got, want := resp.Code, http.StatusUnauthorized; got != want {
t.Fatalf("got %d want %d", got, want)
}
if !strings.Contains(resp.Body.String(), "ACL support disabled") {
t.Fatalf("bad: %#v", resp)
}
})
}
}
func makeTestACL(t *testing.T, srv *HTTPServer) string {
body := bytes.NewBuffer(nil)
enc := json.NewEncoder(body)
raw := map[string]interface{}{
"Name": "User Token",
"Type": "client",
"Rules": "",
}
enc.Encode(raw)
req, _ := http.NewRequest("PUT", "/v1/acl/create?token=root", body)
resp := httptest.NewRecorder()
obj, err := srv.ACLCreate(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
aclResp := obj.(aclCreateResponse)
return aclResp.ID
}
func TestACL_Legacy_Update(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
testrpc.WaitForLeader(t, a.RPC, "dc1")
id := makeTestACL(t, a.srv)
body := bytes.NewBuffer(nil)
enc := json.NewEncoder(body)
raw := map[string]interface{}{
"ID": id,
"Name": "User Token 2",
"Type": "client",
"Rules": "",
}
enc.Encode(raw)
req, _ := http.NewRequest("PUT", "/v1/acl/update?token=root", body)
resp := httptest.NewRecorder()
obj, err := a.srv.ACLUpdate(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
aclResp := obj.(aclCreateResponse)
if aclResp.ID != id {
t.Fatalf("bad: %v", aclResp)
}
}
func TestACL_Legacy_UpdateUpsert(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
body := bytes.NewBuffer(nil)
enc := json.NewEncoder(body)
raw := map[string]interface{}{
"ID": "my-old-id",
"Name": "User Token 2",
"Type": "client",
"Rules": "",
}
enc.Encode(raw)
req, _ := http.NewRequest("PUT", "/v1/acl/update?token=root", body)
resp := httptest.NewRecorder()
testrpc.WaitForLeader(t, a.RPC, "dc1")
obj, err := a.srv.ACLUpdate(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
aclResp := obj.(aclCreateResponse)
if aclResp.ID != "my-old-id" {
t.Fatalf("bad: %v", aclResp)
}
}
func TestACL_Legacy_Destroy(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
testrpc.WaitForLeader(t, a.RPC, "dc1")
id := makeTestACL(t, a.srv)
req, _ := http.NewRequest("PUT", "/v1/acl/destroy/"+id+"?token=root", nil)
resp := httptest.NewRecorder()
obj, err := a.srv.ACLDestroy(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
if resp, ok := obj.(bool); !ok || !resp {
t.Fatalf("should work")
}
req, _ = http.NewRequest("GET", "/v1/acl/info/"+id, nil)
resp = httptest.NewRecorder()
obj, err = a.srv.ACLGet(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
respObj, ok := obj.(structs.ACLs)
if !ok {
t.Fatalf("should work")
}
if len(respObj) != 0 {
t.Fatalf("bad: %v", respObj)
}
}
func TestACL_Legacy_Clone(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
testrpc.WaitForLeader(t, a.RPC, "dc1")
id := makeTestACL(t, a.srv)
req, _ := http.NewRequest("PUT", "/v1/acl/clone/"+id, nil)
resp := httptest.NewRecorder()
_, err := a.srv.ACLClone(resp, req)
if !acl.IsErrPermissionDenied(err) {
t.Fatalf("err: %v", err)
}
req, _ = http.NewRequest("PUT", "/v1/acl/clone/"+id+"?token=root", nil)
resp = httptest.NewRecorder()
obj, err := a.srv.ACLClone(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
aclResp, ok := obj.(aclCreateResponse)
if !ok {
t.Fatalf("should work: %#v %#v", obj, resp)
}
if aclResp.ID == id {
t.Fatalf("bad id")
}
req, _ = http.NewRequest("GET", "/v1/acl/info/"+aclResp.ID, nil)
resp = httptest.NewRecorder()
obj, err = a.srv.ACLGet(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
respObj, ok := obj.(structs.ACLs)
if !ok {
t.Fatalf("should work")
}
if len(respObj) != 1 {
t.Fatalf("bad: %v", respObj)
}
}
func TestACL_Legacy_Get(t *testing.T) {
t.Parallel()
t.Run("wrong id", func(t *testing.T) {
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
req, _ := http.NewRequest("GET", "/v1/acl/info/nope", nil)
resp := httptest.NewRecorder()
testrpc.WaitForLeader(t, a.RPC, "dc1")
obj, err := a.srv.ACLGet(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
respObj, ok := obj.(structs.ACLs)
if !ok {
t.Fatalf("should work")
}
if respObj == nil || len(respObj) != 0 {
t.Fatalf("bad: %v", respObj)
}
})
t.Run("right id", func(t *testing.T) {
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
testrpc.WaitForLeader(t, a.RPC, "dc1")
id := makeTestACL(t, a.srv)
req, _ := http.NewRequest("GET", "/v1/acl/info/"+id, nil)
resp := httptest.NewRecorder()
obj, err := a.srv.ACLGet(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
respObj, ok := obj.(structs.ACLs)
if !ok {
t.Fatalf("should work")
}
if len(respObj) != 1 {
t.Fatalf("bad: %v", respObj)
}
})
}
func TestACL_Legacy_List(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
testrpc.WaitForLeader(t, a.RPC, "dc1")
var ids []string
for i := 0; i < 10; i++ {
ids = append(ids, makeTestACL(t, a.srv))
}
req, _ := http.NewRequest("GET", "/v1/acl/list?token=root", nil)
resp := httptest.NewRecorder()
obj, err := a.srv.ACLList(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
respObj, ok := obj.(structs.ACLs)
if !ok {
t.Fatalf("should work")
}
// 10 + master
// anonymous token is a new token and wont show up in this list
if len(respObj) != 11 {
t.Fatalf("bad: %v", respObj)
}
}
func TestACLReplicationStatus(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
req, _ := http.NewRequest("GET", "/v1/acl/replication", nil)
resp := httptest.NewRecorder()
testrpc.WaitForLeader(t, a.RPC, "dc1")
obj, err := a.srv.ACLReplicationStatus(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
_, ok := obj.(structs.ACLReplicationStatus)
if !ok {
t.Fatalf("should work")
}
}

744
agent/acl_endpoint_test.go
View File

@ -3,80 +3,70 @@ package agent
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/hashicorp/consul/acl"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/consul/testrpc"
"github.com/stretchr/testify/require"
)
// NOTE: The tests contained herein are designed to test the HTTP API
// They are not intented to thoroughly test the backing RPC
// functionality as that will be done with other tests.
func TestACL_Disabled_Response(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), "")
defer a.Shutdown()
tests := []func(resp http.ResponseWriter, req *http.Request) (interface{}, error){
a.srv.ACLBootstrap,
a.srv.ACLDestroy,
a.srv.ACLCreate,
a.srv.ACLUpdate,
a.srv.ACLClone,
a.srv.ACLGet,
a.srv.ACLList,
a.srv.ACLReplicationStatus,
a.srv.AgentToken, // See TestAgent_Token.
type testCase struct {
name string
fn func(resp http.ResponseWriter, req *http.Request) (interface{}, error)
}
tests := []testCase{
{"ACLBootstrap", a.srv.ACLBootstrap},
{"ACLReplicationStatus", a.srv.ACLReplicationStatus},
{"AgentToken", a.srv.AgentToken}, // See TestAgent_Token
{"ACLRulesTranslate", a.srv.ACLRulesTranslate},
{"ACLRulesTranslateLegacyToken", a.srv.ACLRulesTranslateLegacyToken},
{"ACLPolicyList", a.srv.ACLPolicyList},
{"ACLPolicyCRUD", a.srv.ACLPolicyCRUD},
{"ACLPolicyCreate", a.srv.ACLPolicyCreate},
{"ACLTokenList", a.srv.ACLTokenList},
{"ACLTokenCreate", a.srv.ACLTokenCreate},
{"ACLTokenSelf", a.srv.ACLTokenSelf},
{"ACLTokenCRUD", a.srv.ACLTokenCRUD},
}
testrpc.WaitForLeader(t, a.RPC, "dc1")
for i, tt := range tests {
t.Run(fmt.Sprintf("%d", i), func(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req, _ := http.NewRequest("PUT", "/should/not/care", nil)
resp := httptest.NewRecorder()
obj, err := tt(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
if obj != nil {
t.Fatalf("bad: %#v", obj)
}
if got, want := resp.Code, http.StatusUnauthorized; got != want {
t.Fatalf("got %d want %d", got, want)
}
if !strings.Contains(resp.Body.String(), "ACL support disabled") {
t.Fatalf("bad: %#v", resp)
}
obj, err := tt.fn(resp, req)
require.NoError(t, err)
require.Nil(t, obj)
require.Equal(t, http.StatusUnauthorized, resp.Code)
require.Contains(t, resp.Body.String(), "ACL support disabled")
})
}
}
func makeTestACL(t *testing.T, srv *HTTPServer) string {
func jsonBody(v interface{}) io.Reader {
body := bytes.NewBuffer(nil)
enc := json.NewEncoder(body)
raw := map[string]interface{}{
"Name": "User Token",
"Type": "client",
"Rules": "",
}
enc.Encode(raw)
req, _ := http.NewRequest("PUT", "/v1/acl/create?token=root", body)
resp := httptest.NewRecorder()
obj, err := srv.ACLCreate(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
aclResp := obj.(aclCreateResponse)
return aclResp.ID
enc.Encode(v)
return body
}
func TestACL_Bootstrap(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), TestACLConfig()+`
acl_master_token = ""
`)
acl_master_token = ""
`)
defer a.Shutdown()
tests := []struct {
@ -94,20 +84,23 @@ func TestACL_Bootstrap(t *testing.T) {
resp := httptest.NewRecorder()
req, _ := http.NewRequest(tt.method, "/v1/acl/bootstrap", nil)
out, err := a.srv.ACLBootstrap(resp, req)
if err != nil {
if tt.token && err != nil {
t.Fatalf("err: %v", err)
}
if got, want := resp.Code, tt.code; got != want {
t.Fatalf("got %d want %d", got, want)
}
if tt.token {
wrap, ok := out.(aclCreateResponse)
wrap, ok := out.(*aclBootstrapResponse)
if !ok {
t.Fatalf("bad: %T", out)
}
if len(wrap.ID) != len("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx") {
t.Fatalf("bad: %v", wrap)
}
if wrap.ID != wrap.SecretID {
t.Fatalf("bad: %v", wrap)
}
} else {
if out != nil {
t.Fatalf("bad: %T", out)
@ -117,228 +110,487 @@ func TestACL_Bootstrap(t *testing.T) {
}
}
func TestACL_Update(t *testing.T) {
func TestACL_HTTP(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
testrpc.WaitForLeader(t, a.RPC, "dc1")
id := makeTestACL(t, a.srv)
body := bytes.NewBuffer(nil)
enc := json.NewEncoder(body)
raw := map[string]interface{}{
"ID": id,
"Name": "User Token 2",
"Type": "client",
"Rules": "",
}
enc.Encode(raw)
idMap := make(map[string]string)
policyMap := make(map[string]*structs.ACLPolicy)
tokenMap := make(map[string]*structs.ACLToken)
req, _ := http.NewRequest("PUT", "/v1/acl/update?token=root", body)
resp := httptest.NewRecorder()
obj, err := a.srv.ACLUpdate(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
aclResp := obj.(aclCreateResponse)
if aclResp.ID != id {
t.Fatalf("bad: %v", aclResp)
}
}
// This is all done as a subtest for a couple reasons
// 1. It uses only 1 test agent and these are
// somewhat expensive to bring up and tear down often
// 2. Instead of having to bring up a new agent and prime
// the ACL system with some data before running the test
// we can intelligently order these tests so we can still
// test everything with less actual operations and do
// so in a manner that is less prone to being flaky
// 3. While this test will be large it should
t.Run("Policy", func(t *testing.T) {
t.Run("Create", func(t *testing.T) {
policyInput := &structs.ACLPolicy{
Name: "test",
Description: "test",
Rules: `acl = "read"`,
Datacenters: []string{"dc1"},
}
func TestACL_UpdateUpsert(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
req, _ := http.NewRequest("PUT", "/v1/acl/policy?token=root", jsonBody(policyInput))
resp := httptest.NewRecorder()
obj, err := a.srv.ACLPolicyCreate(resp, req)
require.NoError(t, err)
body := bytes.NewBuffer(nil)
enc := json.NewEncoder(body)
raw := map[string]interface{}{
"ID": "my-old-id",
"Name": "User Token 2",
"Type": "client",
"Rules": "",
}
enc.Encode(raw)
policy, ok := obj.(*structs.ACLPolicy)
require.True(t, ok)
req, _ := http.NewRequest("PUT", "/v1/acl/update?token=root", body)
resp := httptest.NewRecorder()
// 36 = length of the string form of uuids
require.Len(t, policy.ID, 36)
require.Equal(t, policyInput.Name, policy.Name)
require.Equal(t, policyInput.Description, policy.Description)
require.Equal(t, policyInput.Rules, policy.Rules)
require.Equal(t, policyInput.Datacenters, policy.Datacenters)
require.True(t, policy.CreateIndex > 0)
require.Equal(t, policy.CreateIndex, policy.ModifyIndex)
require.NotNil(t, policy.Hash)
require.NotEqual(t, policy.Hash, []byte{})
testrpc.WaitForLeader(t, a.RPC, "dc1")
obj, err := a.srv.ACLUpdate(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
aclResp := obj.(aclCreateResponse)
if aclResp.ID != "my-old-id" {
t.Fatalf("bad: %v", aclResp)
}
}
idMap["policy-test"] = policy.ID
policyMap[policy.ID] = policy
})
func TestACL_Destroy(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
t.Run("Minimal", func(t *testing.T) {
policyInput := &structs.ACLPolicy{
Name: "minimal",
Rules: `key_prefix "" { policy = "read" }`,
}
testrpc.WaitForLeader(t, a.RPC, "dc1")
id := makeTestACL(t, a.srv)
req, _ := http.NewRequest("PUT", "/v1/acl/destroy/"+id+"?token=root", nil)
resp := httptest.NewRecorder()
obj, err := a.srv.ACLDestroy(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
if resp, ok := obj.(bool); !ok || !resp {
t.Fatalf("should work")
}
req, _ := http.NewRequest("PUT", "/v1/acl/policy?token=root", jsonBody(policyInput))
resp := httptest.NewRecorder()
obj, err := a.srv.ACLPolicyCreate(resp, req)
require.NoError(t, err)
req, _ = http.NewRequest("GET", "/v1/acl/info/"+id, nil)
resp = httptest.NewRecorder()
obj, err = a.srv.ACLGet(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
respObj, ok := obj.(structs.ACLs)
if !ok {
t.Fatalf("should work")
}
if len(respObj) != 0 {
t.Fatalf("bad: %v", respObj)
}
}
policy, ok := obj.(*structs.ACLPolicy)
require.True(t, ok)
func TestACL_Clone(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
// 36 = length of the string form of uuids
require.Len(t, policy.ID, 36)
require.Equal(t, policyInput.Name, policy.Name)
require.Equal(t, policyInput.Description, policy.Description)
require.Equal(t, policyInput.Rules, policy.Rules)
require.Equal(t, policyInput.Datacenters, policy.Datacenters)
require.True(t, policy.CreateIndex > 0)
require.Equal(t, policy.CreateIndex, policy.ModifyIndex)
require.NotNil(t, policy.Hash)
require.NotEqual(t, policy.Hash, []byte{})
testrpc.WaitForLeader(t, a.RPC, "dc1")
id := makeTestACL(t, a.srv)
idMap["policy-minimal"] = policy.ID
policyMap[policy.ID] = policy
})
req, _ := http.NewRequest("PUT", "/v1/acl/clone/"+id, nil)
resp := httptest.NewRecorder()
_, err := a.srv.ACLClone(resp, req)
if !acl.IsErrPermissionDenied(err) {
t.Fatalf("err: %v", err)
}
t.Run("Name Chars", func(t *testing.T) {
policyInput := &structs.ACLPolicy{
Name: "read-all_nodes-012",
Rules: `node_prefix "" { policy = "read" }`,
}
req, _ = http.NewRequest("PUT", "/v1/acl/clone/"+id+"?token=root", nil)
resp = httptest.NewRecorder()
obj, err := a.srv.ACLClone(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
aclResp, ok := obj.(aclCreateResponse)
if !ok {
t.Fatalf("should work: %#v %#v", obj, resp)
}
if aclResp.ID == id {
t.Fatalf("bad id")
}
req, _ := http.NewRequest("PUT", "/v1/acl/policy?token=root", jsonBody(policyInput))
resp := httptest.NewRecorder()
obj, err := a.srv.ACLPolicyCreate(resp, req)
require.NoError(t, err)
req, _ = http.NewRequest("GET", "/v1/acl/info/"+aclResp.ID, nil)
resp = httptest.NewRecorder()
obj, err = a.srv.ACLGet(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
respObj, ok := obj.(structs.ACLs)
if !ok {
t.Fatalf("should work")
}
if len(respObj) != 1 {
t.Fatalf("bad: %v", respObj)
}
}
policy, ok := obj.(*structs.ACLPolicy)
require.True(t, ok)
func TestACL_Get(t *testing.T) {
t.Parallel()
t.Run("wrong id", func(t *testing.T) {
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
// 36 = length of the string form of uuids
require.Len(t, policy.ID, 36)
require.Equal(t, policyInput.Name, policy.Name)
require.Equal(t, policyInput.Description, policy.Description)
require.Equal(t, policyInput.Rules, policy.Rules)
require.Equal(t, policyInput.Datacenters, policy.Datacenters)
require.True(t, policy.CreateIndex > 0)
require.Equal(t, policy.CreateIndex, policy.ModifyIndex)
require.NotNil(t, policy.Hash)
require.NotEqual(t, policy.Hash, []byte{})
req, _ := http.NewRequest("GET", "/v1/acl/info/nope", nil)
resp := httptest.NewRecorder()
testrpc.WaitForLeader(t, a.RPC, "dc1")
obj, err := a.srv.ACLGet(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
respObj, ok := obj.(structs.ACLs)
if !ok {
t.Fatalf("should work")
}
if respObj == nil || len(respObj) != 0 {
t.Fatalf("bad: %v", respObj)
}
idMap["policy-read-all-nodes"] = policy.ID
policyMap[policy.ID] = policy
})
t.Run("Update Name ID Mistmatch", func(t *testing.T) {
policyInput := &structs.ACLPolicy{
ID: "ac7560be-7f11-4d6d-bfcf-15633c2090fd",
Name: "read-all-nodes",
Description: "Can read all node information",
Rules: `node_prefix "" { policy = "read" }`,
Datacenters: []string{"dc1"},
}
req, _ := http.NewRequest("PUT", "/v1/acl/policy/"+idMap["policy-read-all-nodes"]+"?token=root", jsonBody(policyInput))
resp := httptest.NewRecorder()
_, err := a.srv.ACLPolicyCRUD(resp, req)
require.Error(t, err)
_, ok := err.(BadRequestError)
require.True(t, ok)
})
t.Run("Policy CRUD Missing ID in URL", func(t *testing.T) {
req, _ := http.NewRequest("GET", "/v1/acl/policy/?token=root", nil)
resp := httptest.NewRecorder()
_, err := a.srv.ACLPolicyCRUD(resp, req)
require.Error(t, err)
_, ok := err.(BadRequestError)
require.True(t, ok)
})
t.Run("Update", func(t *testing.T) {
policyInput := &structs.ACLPolicy{
Name: "read-all-nodes",
Description: "Can read all node information",
Rules: `node_prefix "" { policy = "read" }`,
Datacenters: []string{"dc1"},
}
req, _ := http.NewRequest("PUT", "/v1/acl/policy/"+idMap["policy-read-all-nodes"]+"?token=root", jsonBody(policyInput))
resp := httptest.NewRecorder()
obj, err := a.srv.ACLPolicyCRUD(resp, req)
require.NoError(t, err)
policy, ok := obj.(*structs.ACLPolicy)
require.True(t, ok)
// 36 = length of the string form of uuids
require.Len(t, policy.ID, 36)
require.Equal(t, policyInput.Name, policy.Name)
require.Equal(t, policyInput.Description, policy.Description)
require.Equal(t, policyInput.Rules, policy.Rules)
require.Equal(t, policyInput.Datacenters, policy.Datacenters)
require.True(t, policy.CreateIndex > 0)
require.True(t, policy.CreateIndex < policy.ModifyIndex)
require.NotNil(t, policy.Hash)
require.NotEqual(t, policy.Hash, []byte{})
idMap["policy-read-all-nodes"] = policy.ID
policyMap[policy.ID] = policy
})
t.Run("ID Supplied", func(t *testing.T) {
policyInput := &structs.ACLPolicy{
ID: "12123d01-37f1-47e6-b55b-32328652bd38",
Name: "with-id",
Description: "test",
Rules: `acl = "read"`,
Datacenters: []string{"dc1"},
}
req, _ := http.NewRequest("PUT", "/v1/acl/policy?token=root", jsonBody(policyInput))
resp := httptest.NewRecorder()
_, err := a.srv.ACLPolicyCreate(resp, req)
require.Error(t, err)
_, ok := err.(BadRequestError)
require.True(t, ok)
})
t.Run("Invalid payload", func(t *testing.T) {
body := bytes.NewBuffer(nil)
body.Write([]byte{0, 1, 2, 3, 4, 5, 6, 7, 8, 9})
req, _ := http.NewRequest("PUT", "/v1/acl/policy?token=root", body)
resp := httptest.NewRecorder()
_, err := a.srv.ACLPolicyCreate(resp, req)
require.Error(t, err)
_, ok := err.(BadRequestError)
require.True(t, ok)
})
t.Run("Delete", func(t *testing.T) {
req, _ := http.NewRequest("DELETE", "/v1/acl/policy/"+idMap["policy-minimal"]+"?token=root", nil)
resp := httptest.NewRecorder()
_, err := a.srv.ACLPolicyCRUD(resp, req)
require.NoError(t, err)
delete(policyMap, idMap["policy-minimal"])
delete(idMap, "policy-minimal")
})
t.Run("List", func(t *testing.T) {
req, _ := http.NewRequest("GET", "/v1/acl/policies?token=root", nil)
resp := httptest.NewRecorder()
raw, err := a.srv.ACLPolicyList(resp, req)
require.NoError(t, err)
policies, ok := raw.(structs.ACLPolicyListStubs)
require.True(t, ok)
// 2 we just created + global management
require.Len(t, policies, 3)
for policyID, expected := range policyMap {
found := false
for _, actual := range policies {
if actual.ID == policyID {
require.Equal(t, expected.Name, actual.Name)
require.Equal(t, expected.Datacenters, actual.Datacenters)
require.Equal(t, expected.Hash, actual.Hash)
require.Equal(t, expected.CreateIndex, actual.CreateIndex)
require.Equal(t, expected.ModifyIndex, actual.ModifyIndex)
found = true
break
}
}
require.True(t, found)
}
})
t.Run("Read", func(t *testing.T) {
req, _ := http.NewRequest("GET", "/v1/acl/policy/"+idMap["policy-read-all-nodes"]+"?token=root", nil)
resp := httptest.NewRecorder()
raw, err := a.srv.ACLPolicyCRUD(resp, req)
require.NoError(t, err)
policy, ok := raw.(*structs.ACLPolicy)
require.True(t, ok)
require.Equal(t, policyMap[idMap["policy-read-all-nodes"]], policy)
})
})
t.Run("right id", func(t *testing.T) {
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
t.Run("Token", func(t *testing.T) {
t.Run("Create", func(t *testing.T) {
tokenInput := &structs.ACLToken{
Description: "test",
Policies: []structs.ACLTokenPolicyLink{
structs.ACLTokenPolicyLink{
ID: idMap["policy-test"],
Name: policyMap[idMap["policy-test"]].Name,
},
structs.ACLTokenPolicyLink{
ID: idMap["policy-read-all-nodes"],
Name: policyMap[idMap["policy-read-all-nodes"]].Name,
},
},
}
testrpc.WaitForLeader(t, a.RPC, "dc1")
id := makeTestACL(t, a.srv)
req, _ := http.NewRequest("PUT", "/v1/acl/token?token=root", jsonBody(tokenInput))
resp := httptest.NewRecorder()
obj, err := a.srv.ACLTokenCreate(resp, req)
require.NoError(t, err)
req, _ := http.NewRequest("GET", "/v1/acl/info/"+id, nil)
resp := httptest.NewRecorder()
obj, err := a.srv.ACLGet(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
respObj, ok := obj.(structs.ACLs)
if !ok {
t.Fatalf("should work")
}
if len(respObj) != 1 {
t.Fatalf("bad: %v", respObj)
}
token, ok := obj.(*structs.ACLToken)
require.True(t, ok)
// 36 = length of the string form of uuids
require.Len(t, token.AccessorID, 36)
require.Len(t, token.SecretID, 36)
require.Equal(t, tokenInput.Description, token.Description)
require.Equal(t, tokenInput.Policies, token.Policies)
require.True(t, token.CreateIndex > 0)
require.Equal(t, token.CreateIndex, token.ModifyIndex)
require.NotNil(t, token.Hash)
require.NotEqual(t, token.Hash, []byte{})
idMap["token-test"] = token.AccessorID
tokenMap[token.AccessorID] = token
})
t.Run("Create Local", func(t *testing.T) {
tokenInput := &structs.ACLToken{
Description: "local",
Policies: []structs.ACLTokenPolicyLink{
structs.ACLTokenPolicyLink{
ID: idMap["policy-test"],
Name: policyMap[idMap["policy-test"]].Name,
},
structs.ACLTokenPolicyLink{
ID: idMap["policy-read-all-nodes"],
Name: policyMap[idMap["policy-read-all-nodes"]].Name,
},
},
Local: true,
}
req, _ := http.NewRequest("PUT", "/v1/acl/token?token=root", jsonBody(tokenInput))
resp := httptest.NewRecorder()
obj, err := a.srv.ACLTokenCreate(resp, req)
require.NoError(t, err)
token, ok := obj.(*structs.ACLToken)
require.True(t, ok)
// 36 = length of the string form of uuids
require.Len(t, token.AccessorID, 36)
require.Len(t, token.SecretID, 36)
require.Equal(t, tokenInput.Description, token.Description)
require.Equal(t, tokenInput.Policies, token.Policies)
require.True(t, token.CreateIndex > 0)
require.Equal(t, token.CreateIndex, token.ModifyIndex)
require.NotNil(t, token.Hash)
require.NotEqual(t, token.Hash, []byte{})
idMap["token-local"] = token.AccessorID
tokenMap[token.AccessorID] = token
})
t.Run("Read", func(t *testing.T) {
expected := tokenMap[idMap["token-test"]]
req, _ := http.NewRequest("GET", "/v1/acl/token/"+expected.AccessorID+"?token=root", nil)
resp := httptest.NewRecorder()
obj, err := a.srv.ACLTokenCRUD(resp, req)
require.NoError(t, err)
token, ok := obj.(*structs.ACLToken)
require.True(t, ok)
require.Equal(t, expected, token)
})
t.Run("Self", func(t *testing.T) {
expected := tokenMap[idMap["token-test"]]
req, _ := http.NewRequest("GET", "/v1/acl/token/self?token="+expected.SecretID, nil)
resp := httptest.NewRecorder()
obj, err := a.srv.ACLTokenSelf(resp, req)
require.NoError(t, err)
token, ok := obj.(*structs.ACLToken)
require.True(t, ok)
require.Equal(t, expected, token)
})
t.Run("Clone", func(t *testing.T) {
tokenInput := &structs.ACLToken{
Description: "cloned token",
}
baseToken := tokenMap[idMap["token-test"]]
req, _ := http.NewRequest("PUT", "/v1/acl/token/"+baseToken.AccessorID+"/clone?token=root", jsonBody(tokenInput))
resp := httptest.NewRecorder()
obj, err := a.srv.ACLTokenCRUD(resp, req)
require.NoError(t, err)
token, ok := obj.(*structs.ACLToken)
require.True(t, ok)
require.NotEqual(t, baseToken.AccessorID, token.AccessorID)
require.NotEqual(t, baseToken.SecretID, token.SecretID)
require.Equal(t, tokenInput.Description, token.Description)
require.Equal(t, baseToken.Policies, token.Policies)
require.True(t, token.CreateIndex > 0)
require.Equal(t, token.CreateIndex, token.ModifyIndex)
require.NotNil(t, token.Hash)
require.NotEqual(t, token.Hash, []byte{})
idMap["token-cloned"] = token.AccessorID
tokenMap[token.AccessorID] = token
})
t.Run("Update", func(t *testing.T) {
originalToken := tokenMap[idMap["token-cloned"]]
// Accessor and Secret will be filled in
tokenInput := &structs.ACLToken{
Description: "Better description for this cloned token",
Policies: []structs.ACLTokenPolicyLink{
structs.ACLTokenPolicyLink{
ID: idMap["policy-read-all-nodes"],
Name: policyMap[idMap["policy-read-all-nodes"]].Name,
},
},
}
req, _ := http.NewRequest("PUT", "/v1/acl/token/"+originalToken.AccessorID+"?token=root", jsonBody(tokenInput))
resp := httptest.NewRecorder()
obj, err := a.srv.ACLTokenCRUD(resp, req)
require.NoError(t, err)
token, ok := obj.(*structs.ACLToken)
require.True(t, ok)
require.Equal(t, originalToken.AccessorID, token.AccessorID)
require.Equal(t, originalToken.SecretID, token.SecretID)
require.Equal(t, tokenInput.Description, token.Description)
require.Equal(t, tokenInput.Policies, token.Policies)
require.True(t, token.CreateIndex > 0)
require.True(t, token.CreateIndex < token.ModifyIndex)
require.NotNil(t, token.Hash)
require.NotEqual(t, token.Hash, []byte{})
require.NotEqual(t, token.Hash, originalToken.Hash)
tokenMap[token.AccessorID] = token
})
t.Run("CRUD Missing Token Accessor ID", func(t *testing.T) {
req, _ := http.NewRequest("GET", "/v1/acl/token/?token=root", nil)
resp := httptest.NewRecorder()
obj, err := a.srv.ACLTokenCRUD(resp, req)
require.Error(t, err)
require.Nil(t, obj)
_, ok := err.(BadRequestError)
require.True(t, ok)
})
t.Run("Update Accessor Mismatch", func(t *testing.T) {
originalToken := tokenMap[idMap["token-cloned"]]
// Accessor and Secret will be filled in
tokenInput := &structs.ACLToken{
AccessorID: "e8aeb69a-0ace-42b9-b95f-d1d9eafe1561",
Description: "Better description for this cloned token",
Policies: []structs.ACLTokenPolicyLink{
structs.ACLTokenPolicyLink{
ID: idMap["policy-read-all-nodes"],
Name: policyMap[idMap["policy-read-all-nodes"]].Name,
},
},
}
req, _ := http.NewRequest("PUT", "/v1/acl/token/"+originalToken.AccessorID+"?token=root", jsonBody(tokenInput))
resp := httptest.NewRecorder()
obj, err := a.srv.ACLTokenCRUD(resp, req)
require.Error(t, err)
require.Nil(t, obj)
_, ok := err.(BadRequestError)
require.True(t, ok)
})
t.Run("Delete", func(t *testing.T) {
req, _ := http.NewRequest("DELETE", "/v1/acl/token/"+idMap["token-cloned"]+"?token=root", nil)
resp := httptest.NewRecorder()
_, err := a.srv.ACLTokenCRUD(resp, req)
require.NoError(t, err)
delete(tokenMap, idMap["token-cloned"])
delete(idMap, "token-cloned")
})
t.Run("List", func(t *testing.T) {
req, _ := http.NewRequest("GET", "/v1/acl/tokens?token=root", nil)
resp := httptest.NewRecorder()
raw, err := a.srv.ACLTokenList(resp, req)
require.NoError(t, err)
tokens, ok := raw.(structs.ACLTokenListStubs)
require.True(t, ok)
// 3 tokens created but 1 was deleted + master token + anon token
require.Len(t, tokens, 4)
// this loop doesn't verify anything about the master token
for tokenID, expected := range tokenMap {
found := false
for _, actual := range tokens {
if actual.AccessorID == tokenID {
require.Equal(t, expected.Description, actual.Description)
require.Equal(t, expected.Policies, actual.Policies)
require.Equal(t, expected.Local, actual.Local)
require.Equal(t, expected.CreateTime, actual.CreateTime)
require.Equal(t, expected.Hash, actual.Hash)
require.Equal(t, expected.CreateIndex, actual.CreateIndex)
require.Equal(t, expected.ModifyIndex, actual.ModifyIndex)
found = true
break
}
}
require.True(t, found)
}
})
t.Run("List by Policy", func(t *testing.T) {
req, _ := http.NewRequest("GET", "/v1/acl/tokens?token=root&policy="+structs.ACLPolicyGlobalManagementID, nil)
resp := httptest.NewRecorder()
raw, err := a.srv.ACLTokenList(resp, req)
require.NoError(t, err)
tokens, ok := raw.(structs.ACLTokenListStubs)
require.True(t, ok)
require.Len(t, tokens, 1)
token := tokens[0]
require.Equal(t, "Master Token", token.Description)
require.Len(t, token.Policies, 1)
require.Equal(t, structs.ACLPolicyGlobalManagementID, token.Policies[0].ID)
})
})
}
func TestACL_List(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
testrpc.WaitForLeader(t, a.RPC, "dc1")
var ids []string
for i := 0; i < 10; i++ {
ids = append(ids, makeTestACL(t, a.srv))
}
req, _ := http.NewRequest("GET", "/v1/acl/list?token=root", nil)
resp := httptest.NewRecorder()
obj, err := a.srv.ACLList(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
respObj, ok := obj.(structs.ACLs)
if !ok {
t.Fatalf("should work")
}
// 10 + anonymous + master
if len(respObj) != 12 {
t.Fatalf("bad: %v", respObj)
}
}
func TestACLReplicationStatus(t *testing.T) {
t.Parallel()
a := NewTestAgent(t.Name(), TestACLConfig())
defer a.Shutdown()
req, _ := http.NewRequest("GET", "/v1/acl/replication", nil)
resp := httptest.NewRecorder()
testrpc.WaitForLeader(t, a.RPC, "dc1")
obj, err := a.srv.ACLReplicationStatus(resp, req)
if err != nil {
t.Fatalf("err: %v", err)
}
_, ok := obj.(structs.ACLReplicationStatus)
if !ok {
t.Fatalf("should work")
}
}

918
agent/acl_test.go
View File

File diff suppressed because it is too large Load Diff

40
agent/agent.go
View File

@ -88,7 +88,10 @@ type delegate interface {
LocalMember() serf.Member
JoinLAN(addrs []string) (n int, err error)
RemoveFailedNode(node string) error
ResolveToken(secretID string) (acl.Authorizer, error)
RPC(method string, args interface{}, reply interface{}) error
ACLsEnabled() bool
UseLegacyACLs() bool
SnapshotRPC(args *structs.SnapshotRequest, in io.Reader, out io.Writer, replyFn structs.SnapshotReplyFn) error
Shutdown() error
Stats() map[string]map[string]string
@ -127,8 +130,8 @@ type Agent struct {
// depending on the configuration
delegate delegate
// acls is an object that helps manage local ACL enforcement.
acls *aclManager
// aclMasterAuthorizer is an object that helps manage local ACL enforcement.
aclMasterAuthorizer acl.Authorizer
// state stores a local representation of the node,
// services and checks. Used for anti-entropy.
@ -255,14 +258,9 @@ func New(c *config.RuntimeConfig) (*Agent, error) {
if c.DataDir == "" && !c.DevMode {
return nil, fmt.Errorf("Must configure a DataDir")
}
acls, err := newACLManager(c)
if err != nil {
return nil, err
}
a := &Agent{
config: c,
acls: acls,
checkReapAfter: make(map[types.CheckID]time.Duration),
checkMonitors: make(map[types.CheckID]*checks.CheckMonitor),
checkTTLs: make(map[types.CheckID]*checks.CheckTTL),
@ -281,6 +279,10 @@ func New(c *config.RuntimeConfig) (*Agent, error) {
tokens: new(token.Store),
}
if err := a.initializeACLs(); err != nil {
return nil, err
}
// Set up the initial state of the token store based on the config.
a.tokens.UpdateUserToken(a.config.ACLToken)
a.tokens.UpdateAgentToken(a.config.ACLAgentToken)
@ -515,12 +517,10 @@ func (a *Agent) listenAndServeGRPC() error {
}
a.xdsServer = &xds.Server{
Logger: a.logger,
CfgMgr: a.proxyConfig,
Authz: a,
ResolveToken: func(id string) (acl.ACL, error) {
return a.resolveToken(id)
},
Logger: a.logger,
CfgMgr: a.proxyConfig,
Authz: a,
ResolveToken: a.resolveToken,
}
var err error
a.grpcServer, err = a.xdsServer.GRPCServer(a.config.CertFile, a.config.KeyFile)
@ -968,8 +968,11 @@ func (a *Agent) consulConfig() (*consul.Config, error) {
if a.config.ACLDatacenter != "" {
base.ACLDatacenter = a.config.ACLDatacenter
}
if a.config.ACLTTL != 0 {
base.ACLTTL = a.config.ACLTTL
if a.config.ACLTokenTTL != 0 {
base.ACLTokenTTL = a.config.ACLTokenTTL
}
if a.config.ACLPolicyTTL != 0 {
base.ACLPolicyTTL = a.config.ACLPolicyTTL
}
if a.config.ACLDefaultPolicy != "" {
base.ACLDefaultPolicy = a.config.ACLDefaultPolicy
@ -977,10 +980,9 @@ func (a *Agent) consulConfig() (*consul.Config, error) {
if a.config.ACLDownPolicy != "" {
base.ACLDownPolicy = a.config.ACLDownPolicy
}
base.EnableACLReplication = a.config.EnableACLReplication
if a.config.ACLEnforceVersion8 {
base.ACLEnforceVersion8 = a.config.ACLEnforceVersion8
}
base.ACLEnforceVersion8 = a.config.ACLEnforceVersion8
base.ACLTokenReplication = a.config.ACLTokenReplication
base.ACLsEnabled = a.config.ACLsEnabled
if a.config.ACLEnableKeyListPolicy {
base.ACLEnableKeyListPolicy = a.config.ACLEnableKeyListPolicy
}

852
agent/bindata_assetfs.go
View File

File diff suppressed because one or more lines are too long

76
agent/config/builder.go
View File

@ -574,6 +574,9 @@ func (b *Builder) Build() (rt RuntimeConfig, err error) {
})
}
datacenter := strings.ToLower(b.stringVal(c.Datacenter))
aclsEnabled := false
primaryDatacenter := strings.ToLower(b.stringVal(c.PrimaryDatacenter))
if c.ACLDatacenter != nil {
b.warn("The 'acl_datacenter' field is deprecated. Use the 'primary_datacenter' field instead.")
@ -581,8 +584,27 @@ func (b *Builder) Build() (rt RuntimeConfig, err error) {
if primaryDatacenter == "" {
primaryDatacenter = strings.ToLower(b.stringVal(c.ACLDatacenter))
}
// when the acl_datacenter config is used it implicitly enables acls
aclsEnabled = true
}
if c.ACL.Enabled != nil {
aclsEnabled = b.boolVal(c.ACL.Enabled)
}
aclDC := primaryDatacenter
if aclsEnabled && aclDC == "" {
aclDC = datacenter
}
enableTokenReplication := false
if c.ACLReplicationToken != nil {
enableTokenReplication = true
}
b.boolValWithDefault(c.ACL.TokenReplication, b.boolValWithDefault(c.EnableACLReplication, enableTokenReplication))
proxyDefaultExecMode := b.stringVal(c.Connect.ProxyDefaults.ExecMode)
proxyDefaultDaemonCommand := c.Connect.ProxyDefaults.DaemonCommand
proxyDefaultScriptCommand := c.Connect.ProxyDefaults.ScriptCommand
@ -596,7 +618,7 @@ func (b *Builder) Build() (rt RuntimeConfig, err error) {
//
rt = RuntimeConfig{
// non-user configurable values
ACLDisabledTTL: b.durationVal("acl_disabled_ttl", c.ACLDisabledTTL),
ACLDisabledTTL: b.durationVal("acl.disabled_ttl", c.ACL.DisabledTTL),
AEInterval: b.durationVal("ae_interval", c.AEInterval),
CheckDeregisterIntervalMin: b.durationVal("check_deregister_interval_min", c.CheckDeregisterIntervalMin),
CheckReapInterval: b.durationVal("check_reap_interval", c.CheckReapInterval),
@ -632,18 +654,20 @@ func (b *Builder) Build() (rt RuntimeConfig, err error) {
GossipWANRetransmitMult: b.intVal(c.GossipWAN.RetransmitMult),
// ACL
ACLAgentMasterToken: b.stringVal(c.ACLAgentMasterToken),
ACLAgentToken: b.stringVal(c.ACLAgentToken),
ACLDatacenter: strings.ToLower(b.stringVal(c.ACLDatacenter)),
ACLDefaultPolicy: b.stringVal(c.ACLDefaultPolicy),
ACLDownPolicy: b.stringVal(c.ACLDownPolicy),
ACLEnforceVersion8: b.boolVal(c.ACLEnforceVersion8),
ACLEnableKeyListPolicy: b.boolVal(c.ACLEnableKeyListPolicy),
ACLMasterToken: b.stringVal(c.ACLMasterToken),
ACLReplicationToken: b.stringVal(c.ACLReplicationToken),
ACLTTL: b.durationVal("acl_ttl", c.ACLTTL),
ACLToken: b.stringVal(c.ACLToken),
EnableACLReplication: b.boolVal(c.EnableACLReplication),
ACLEnforceVersion8: b.boolValWithDefault(c.ACLEnforceVersion8, true),
ACLsEnabled: aclsEnabled,
ACLAgentMasterToken: b.stringValWithDefault(c.ACL.Tokens.AgentMaster, b.stringVal(c.ACLAgentMasterToken)),
ACLAgentToken: b.stringValWithDefault(c.ACL.Tokens.Agent, b.stringVal(c.ACLAgentToken)),
ACLDatacenter: aclDC,
ACLDefaultPolicy: b.stringValWithDefault(c.ACL.DefaultPolicy, b.stringVal(c.ACLDefaultPolicy)),
ACLDownPolicy: b.stringValWithDefault(c.ACL.DownPolicy, b.stringVal(c.ACLDownPolicy)),
ACLEnableKeyListPolicy: b.boolValWithDefault(c.ACL.EnableKeyListPolicy, b.boolVal(c.ACLEnableKeyListPolicy)),
ACLMasterToken: b.stringValWithDefault(c.ACL.Tokens.Master, b.stringVal(c.ACLMasterToken)),
ACLReplicationToken: b.stringValWithDefault(c.ACL.Tokens.Replication, b.stringVal(c.ACLReplicationToken)),
ACLTokenTTL: b.durationValWithDefault("acl.token_ttl", c.ACL.TokenTTL, b.durationVal("acl_ttl", c.ACLTTL)),
ACLPolicyTTL: b.durationVal("acl.policy_ttl", c.ACL.PolicyTTL),
ACLToken: b.stringValWithDefault(c.ACL.Tokens.Default, b.stringVal(c.ACLToken)),
ACLTokenReplication: b.boolValWithDefault(c.ACL.TokenReplication, b.boolValWithDefault(c.EnableACLReplication, enableTokenReplication)),
// Autopilot
AutopilotCleanupDeadServers: b.boolVal(c.Autopilot.CleanupDeadServers),
@ -733,7 +757,7 @@ func (b *Builder) Build() (rt RuntimeConfig, err error) {
ConnectProxyDefaultScriptCommand: proxyDefaultScriptCommand,
ConnectProxyDefaultConfig: proxyDefaultConfig,
DataDir: b.stringVal(c.DataDir),
Datacenter: strings.ToLower(b.stringVal(c.Datacenter)),
Datacenter: datacenter,
DevMode: b.boolVal(b.Flags.DevMode),
DisableAnonymousSignature: b.boolVal(c.DisableAnonymousSignature),
DisableCoordinates: b.boolVal(c.DisableCoordinates),
@ -826,10 +850,6 @@ func (b *Builder) Build() (rt RuntimeConfig, err error) {
b.warn(`BootstrapExpect is set to 1; this is the same as Bootstrap mode.`)
}
if rt.ACLReplicationToken != "" {
rt.EnableACLReplication = true
}
return rt, nil
}
@ -1253,9 +1273,9 @@ func (b *Builder) serviceConnectVal(v *ServiceConnect) *structs.ServiceConnect {
}
}
func (b *Builder) boolValWithDefault(v *bool, default_val bool) bool {
func (b *Builder) boolValWithDefault(v *bool, defaultVal bool) bool {
if v == nil {
return default_val
return defaultVal
}
return *v
@ -1265,9 +1285,9 @@ func (b *Builder) boolVal(v *bool) bool {
return b.boolValWithDefault(v, false)
}
func (b *Builder) durationVal(name string, v *string) (d time.Duration) {
func (b *Builder) durationValWithDefault(name string, v *string, defaultVal time.Duration) (d time.Duration) {
if v == nil {
return 0
return defaultVal
}
d, err := time.ParseDuration(*v)
if err != nil {
@ -1276,6 +1296,10 @@ func (b *Builder) durationVal(name string, v *string) (d time.Duration) {
return d
}
func (b *Builder) durationVal(name string, v *string) (d time.Duration) {
return b.durationValWithDefault(name, v, 0)
}
func (b *Builder) intVal(v *int) int {
if v == nil {
return 0
@ -1293,13 +1317,17 @@ func (b *Builder) portVal(name string, v *int) int {
return *v
}
func (b *Builder) stringVal(v *string) string {
func (b *Builder) stringValWithDefault(v *string, defaultVal string) string {
if v == nil {
return ""
return defaultVal
}
return *v
}
func (b *Builder) stringVal(v *string) string {
return b.stringValWithDefault(v, "")
}
func (b *Builder) float64Val(v *float64) float64 {
if v == nil {
return 0

53
agent/config/config.go
View File

@ -147,17 +147,29 @@ func Parse(data string, format string) (c Config, err error) {
// configuration it should be treated as an external API which cannot be
// changed and refactored at will since this will break existing setups.
type Config struct {
ACLAgentMasterToken *string `json:"acl_agent_master_token,omitempty" hcl:"acl_agent_master_token" mapstructure:"acl_agent_master_token"`
ACLAgentToken *string `json:"acl_agent_token,omitempty" hcl:"acl_agent_token" mapstructure:"acl_agent_token"`
ACLDatacenter *string `json:"acl_datacenter,omitempty" hcl:"acl_datacenter" mapstructure:"acl_datacenter"`
ACLDefaultPolicy *string `json:"acl_default_policy,omitempty" hcl:"acl_default_policy" mapstructure:"acl_default_policy"`
ACLDownPolicy *string `json:"acl_down_policy,omitempty" hcl:"acl_down_policy" mapstructure:"acl_down_policy"`
ACLEnableKeyListPolicy *bool `json:"acl_enable_key_list_policy,omitempty" hcl:"acl_enable_key_list_policy" mapstructure:"acl_enable_key_list_policy"`
ACLEnforceVersion8 *bool `json:"acl_enforce_version_8,omitempty" hcl:"acl_enforce_version_8" mapstructure:"acl_enforce_version_8"`
ACLMasterToken *string `json:"acl_master_token,omitempty" hcl:"acl_master_token" mapstructure:"acl_master_token"`
ACLReplicationToken *string `json:"acl_replication_token,omitempty" hcl:"acl_replication_token" mapstructure:"acl_replication_token"`
ACLTTL *string `json:"acl_ttl,omitempty" hcl:"acl_ttl" mapstructure:"acl_ttl"`
// DEPRECATED (ACL-Legacy-Compat) - moved into the "acl.tokens" stanza
ACLAgentMasterToken *string `json:"acl_agent_master_token,omitempty" hcl:"acl_agent_master_token" mapstructure:"acl_agent_master_token"`
// DEPRECATED (ACL-Legacy-Compat) - moved into the "acl.tokens" stanza
ACLAgentToken *string `json:"acl_agent_token,omitempty" hcl:"acl_agent_token" mapstructure:"acl_agent_token"`
// DEPRECATED (ACL-Legacy-Compat) - moved to "primary_datacenter"
ACLDatacenter *string `json:"acl_datacenter,omitempty" hcl:"acl_datacenter" mapstructure:"acl_datacenter"`
// DEPRECATED (ACL-Legacy-Compat) - moved into the "acl" stanza
ACLDefaultPolicy *string `json:"acl_default_policy,omitempty" hcl:"acl_default_policy" mapstructure:"acl_default_policy"`
// DEPRECATED (ACL-Legacy-Compat) - moved into the "acl" stanza
ACLDownPolicy *string `json:"acl_down_policy,omitempty" hcl:"acl_down_policy" mapstructure:"acl_down_policy"`
// DEPRECATED (ACL-Legacy-Compat) - moved into the "acl" stanza
ACLEnableKeyListPolicy *bool `json:"acl_enable_key_list_policy,omitempty" hcl:"acl_enable_key_list_policy" mapstructure:"acl_enable_key_list_policy"`
// DEPRECATED (ACL-Legacy-Compat) - pre-version8 enforcement is deprecated.
ACLEnforceVersion8 *bool `json:"acl_enforce_version_8,omitempty" hcl:"acl_enforce_version_8" mapstructure:"acl_enforce_version_8"`
// DEPRECATED (ACL-Legacy-Compat) - moved into the "acl" stanza
ACLMasterToken *string `json:"acl_master_token,omitempty" hcl:"acl_master_token" mapstructure:"acl_master_token"`
// DEPRECATED (ACL-Legacy-Compat) - moved into the "acl.tokens" stanza
ACLReplicationToken *string `json:"acl_replication_token,omitempty" hcl:"acl_replication_token" mapstructure:"acl_replication_token"`
// DEPRECATED (ACL-Legacy-Compat) - moved into the "acl.tokens" stanza
ACLTTL *string `json:"acl_ttl,omitempty" hcl:"acl_ttl" mapstructure:"acl_ttl"`
// DEPRECATED (ACL-Legacy-Compat) - moved into the "acl.tokens" stanza
ACLToken *string `json:"acl_token,omitempty" hcl:"acl_token" mapstructure:"acl_token"`
ACL ACL `json:"acl,omitempty" hcl:"acl" mapstructure:"acl"`
Addresses Addresses `json:"addresses,omitempty" hcl:"addresses" mapstructure:"addresses"`
AdvertiseAddrLAN *string `json:"advertise_addr,omitempty" hcl:"advertise_addr" mapstructure:"advertise_addr"`
AdvertiseAddrWAN *string `json:"advertise_addr_wan,omitempty" hcl:"advertise_addr_wan" mapstructure:"advertise_addr_wan"`
@ -263,6 +275,7 @@ type Config struct {
SnapshotAgent map[string]interface{} `json:"snapshot_agent,omitempty" hcl:"snapshot_agent" mapstructure:"snapshot_agent"`
// non-user configurable values
// DEPRECATED (ACL-Legacy-Compat) - moved into the "acl" stanza
ACLDisabledTTL *string `json:"acl_disabled_ttl,omitempty" hcl:"acl_disabled_ttl" mapstructure:"acl_disabled_ttl"`
AEInterval *string `json:"ae_interval,omitempty" hcl:"ae_interval" mapstructure:"ae_interval"`
CheckDeregisterIntervalMin *string `json:"check_deregister_interval_min,omitempty" hcl:"check_deregister_interval_min" mapstructure:"check_deregister_interval_min"`
@ -613,3 +626,23 @@ type Segment struct {
Port *int `json:"port,omitempty" hcl:"port" mapstructure:"port"`
RPCListener *bool `json:"rpc_listener,omitempty" hcl:"rpc_listener" mapstructure:"rpc_listener"`
}
type ACL struct {
Enabled *bool `json:"enabled,omitempty" hcl:"enabled" mapstructure:"enabled"`
TokenReplication *bool `json:"enable_token_replication,omitempty" hcl:"enable_token_replication" mapstructure:"enable_token_replication"`
PolicyTTL *string `json:"policy_ttl,omitempty" hcl:"policy_ttl" mapstructure:"policy_ttl"`
TokenTTL *string `json:"token_ttl,omitempty" hcl:"token_ttl" mapstructure:"token_ttl"`
DownPolicy *string `json:"down_policy,omitempty" hcl:"down_policy" mapstructure:"down_policy"`
DefaultPolicy *string `json:"default_policy,omitempty" hcl:"default_policy" mapstructure:"default_policy"`
EnableKeyListPolicy *bool `json:"enable_key_list_policy,omitempty" hcl:"enable_key_list_policy" mapstructure:"enable_key_list_policy"`
Tokens Tokens `json:"tokens,omitempty" hcl:"tokens" mapstructure:"tokens"`
DisabledTTL *string `json:"disabled_ttl,omitempty" hcl:"disabled_ttl" mapstructure:"disabled_ttl"`
}
type Tokens struct {
Master *string `json:"master,omitempty" hcl:"master" mapstructure:"master"`
Replication *string `json:"replication,omitempty" hcl:"replication" mapstructure:"replication"`
AgentMaster *string `json:"agent_master,omitempty" hcl:"agent_master" mapstructure:"agent_master"`
Default *string `json:"default,omitempty" hcl:"default" mapstructure:"default"`
Agent *string `json:"agent,omitempty" hcl:"agent" mapstructure:"agent"`
}

14
agent/config/default.go
View File

@ -30,6 +30,11 @@ func DefaultSource() Source {
serfLAN := cfg.SerfLANConfig.MemberlistConfig
serfWAN := cfg.SerfWANConfig.MemberlistConfig
// DEPRECATED (ACL-Legacy-Compat) - when legacy ACL support is removed these defaults
// the acl_* config entries here should be transitioned to their counterparts in the
// acl stanza for now we need to be able to detect the new entries not being set (not
// just set to the defaults here) so that we can use the old entries. So the true
// default still needs to reside in the original config values
return Source{
Name: "default",
Format: "hcl",
@ -38,6 +43,9 @@ func DefaultSource() Source {
acl_down_policy = "extend-cache"
acl_enforce_version_8 = true
acl_ttl = "30s"
acl = {
policy_ttl = "30s"
}
bind_addr = "0.0.0.0"
bootstrap = false
bootstrap_expect = 0
@ -115,7 +123,7 @@ func DefaultSource() Source {
metrics_prefix = "consul"
filter_default = true
}
`,
}
}
@ -167,7 +175,9 @@ func NonUserSource() Source {
Name: "non-user",
Format: "hcl",
Data: `
acl_disabled_ttl = "120s"
acl = {
disabled_ttl = "120s"
}
check_deregister_interval_min = "1m"
check_reap_interval = "30s"
ae_interval = "1m"

63
agent/config/runtime.go
View File

@ -30,13 +30,6 @@ type RuntimeConfig struct {
// non-user configurable values
AEInterval time.Duration
// ACLDisabledTTL is used by clients to determine how long they will
// wait to check again with the servers if they discover ACLs are not
// enabled. (not user configurable)
//
// hcl: acl_disabled_ttl = "duration"
ACLDisabledTTL time.Duration
CheckDeregisterIntervalMin time.Duration
CheckReapInterval time.Duration
SegmentLimit int
@ -56,18 +49,30 @@ type RuntimeConfig struct {
ConsulRaftLeaderLeaseTimeout time.Duration
ConsulServerHealthInterval time.Duration
// ACLDisabledTTL is used by agents to determine how long they will
// wait to check again with the servers if they discover ACLs are not
// enabled. (not user configurable)
//
// hcl: acl.disabled_ttl = "duration"
ACLDisabledTTL time.Duration
// ACLsEnabled is used to determine whether ACLs should be enabled
//
// hcl: acl.enabled = boolean
ACLsEnabled bool
// ACLAgentMasterToken is a special token that has full read and write
// privileges for this agent, and can be used to call agent endpoints
// when no servers are available.
//
// hcl: acl_agent_master_token = string
// hcl: acl.tokens.agent_master = string
ACLAgentMasterToken string
// ACLAgentToken is the default token used to make requests for the agent
// itself, such as for registering itself with the catalog. If not
// configured, the 'acl_token' will be used.
//
// hcl: acl_agent_token = string
// hcl: acl.tokens.agent = string
ACLAgentToken string
// ACLDatacenter is the central datacenter that holds authoritative
@ -82,7 +87,7 @@ type RuntimeConfig struct {
// ACLs are used to black-list, or "deny" which means ACLs are
// white-lists.
//
// hcl: acl_default_policy = ("allow"|"deny")
// hcl: acl.default_policy = ("allow"|"deny")
ACLDefaultPolicy string
// ACLDownPolicy is used to control the ACL interaction when we cannot
@ -97,9 +102,10 @@ type RuntimeConfig struct {
// * async-cache - Same behaviour as extend-cache, but perform ACL
// Lookups asynchronously when cache TTL is expired.
//
// hcl: acl_down_policy = ("allow"|"deny"|"extend-cache"|"async-cache")
// hcl: acl.down_policy = ("allow"|"deny"|"extend-cache"|"async-cache")
ACLDownPolicy string
// DEPRECATED (ACL-Legacy-Compat)
// ACLEnforceVersion8 is used to gate a set of ACL policy features that
// are opt-in prior to Consul 0.8 and opt-out in Consul 0.8 and later.
//
@ -112,14 +118,14 @@ type RuntimeConfig struct {
// See https://www.consul.io/docs/guides/acl.html#list-policy-for-keys for
// more details.
//
// hcl: acl_enable_key_list_policy = (true|false)
// hcl: acl.enable_key_list_policy = (true|false)
ACLEnableKeyListPolicy bool
// ACLMasterToken is used to bootstrap the ACL system. It should be specified
// on the servers in the ACLDatacenter. When the leader comes online, it ensures
// that the Master token is available. This provides the initial token.
//
// hcl: acl_master_token = string
// hcl: acl.tokens.master = string
ACLMasterToken string
// ACLReplicationToken is used to fetch ACLs from the ACLDatacenter in
@ -127,19 +133,31 @@ type RuntimeConfig struct {
// also enables replication. Replication is only available in datacenters
// other than the ACLDatacenter.
//
// hcl: acl_replication_token = string
// hcl: acl.tokens.replication = string
ACLReplicationToken string
// ACLTTL is used to control the time-to-live of cached ACLs . This has
// ACLtokenReplication is used to indicate that both tokens and policies
// should be replicated instead of just policies
//
// hcl: acl.token_replication = boolean
ACLTokenReplication bool
// ACLTokenTTL is used to control the time-to-live of cached ACL tokens. This has
// a major impact on performance. By default, it is set to 30 seconds.
//
// hcl: acl_ttl = "duration"
ACLTTL time.Duration
// hcl: acl.policy_ttl = "duration"
ACLTokenTTL time.Duration
// ACLPolicyTTL is used to control the time-to-live of cached ACL policies. This has
// a major impact on performance. By default, it is set to 30 seconds.
//
// hcl: acl.token_ttl = "duration"
ACLPolicyTTL time.Duration
// ACLToken is the default token used to make requests if a per-request
// token is not provided. If not configured the 'anonymous' token is used.
//
// hcl: acl_token = string
// hcl: acl.tokens.default = string
ACLToken string
// AutopilotCleanupDeadServers enables the automatic cleanup of dead servers when new ones
@ -617,15 +635,6 @@ type RuntimeConfig struct {
// hcl: discard_check_output = (true|false)
DiscardCheckOutput bool
// EnableACLReplication is used to turn on ACL replication when using
// /v1/agent/token/acl_replication_token to introduce the token, instead
// of setting acl_replication_token in the config. Setting the token via
// config will also set this to true for backward compatibility.
//
// hcl: enable_acl_replication = (true|false)
// todo(fs): rename to ACLEnableReplication
EnableACLReplication bool
// EnableAgentTLSForChecks is used to apply the agent's TLS settings in
// order to configure the HTTP client used for health checks. Enabling
// this allows HTTP checks to present a client certificate and verify

73
agent/config/runtime_test.go
View File

@ -1378,6 +1378,7 @@ func TestConfigFlagsAndEdgecases(t *testing.T) {
json: []string{`{ "acl_datacenter": "A" }`},
hcl: []string{`acl_datacenter = "A"`},
patch: func(rt *RuntimeConfig) {
rt.ACLsEnabled = true
rt.ACLDatacenter = "a"
rt.DataDir = dataDir
rt.PrimaryDatacenter = "a"
@ -1391,7 +1392,7 @@ func TestConfigFlagsAndEdgecases(t *testing.T) {
hcl: []string{`acl_replication_token = "a"`},
patch: func(rt *RuntimeConfig) {
rt.ACLReplicationToken = "a"
rt.EnableACLReplication = true
rt.ACLTokenReplication = true
rt.DataDir = dataDir
},
},
@ -2809,11 +2810,27 @@ func TestFullConfig(t *testing.T) {
"acl_default_policy": "ArK3WIfE",
"acl_down_policy": "vZXMfMP0",
"acl_enforce_version_8": true,
"acl_enable_key_list_policy": true,
"acl_enable_key_list_policy": true,
"acl_master_token": "C1Q1oIwh",
"acl_replication_token": "LMmgy5dO",
"acl_token": "O1El0wan",
"acl_ttl": "18060s",
"acl" : {
"enabled" : true,
"down_policy" : "03eb2aee",
"default_policy" : "72c2e7a0",
"enable_key_list_policy": false,
"policy_ttl": "1123s",
"token_ttl": "3321s",
"enable_token_replication" : true,
"tokens" : {
"master" : "8a19ac27",
"agent_master" : "64fd0e08",
"replication" : "5795983a",
"agent" : "bed2377c",
"default" : "418fdff1"
}
},
"addresses": {
"dns": "93.95.95.81",
"http": "83.39.91.39",
@ -3344,6 +3361,22 @@ func TestFullConfig(t *testing.T) {
acl_replication_token = "LMmgy5dO"
acl_token = "O1El0wan"
acl_ttl = "18060s"
acl = {
enabled = true
down_policy = "03eb2aee"
default_policy = "72c2e7a0"
enable_key_list_policy = false
policy_ttl = "1123s"
token_ttl = "3321s"
enable_token_replication = true
tokens = {
master = "8a19ac27",
agent_master = "64fd0e08",
replication = "5795983a",
agent = "bed2377c",
default = "418fdff1"
}
}
addresses = {
dns = "93.95.95.81"
http = "83.39.91.39"
@ -3871,6 +3904,9 @@ func TestFullConfig(t *testing.T) {
Data: `
{
"acl_disabled_ttl": "957s",
"acl" : {
"disabled_ttl" : "957s"
},
"ae_interval": "10003s",
"check_deregister_interval_min": "27870s",
"check_reap_interval": "10662s",
@ -3910,6 +3946,9 @@ func TestFullConfig(t *testing.T) {
Format: "hcl",
Data: `
acl_disabled_ttl = "957s"
acl = {
disabled_ttl = "957s"
}
ae_interval = "10003s"
check_deregister_interval_min = "27870s"
check_reap_interval = "10662s"
@ -3982,17 +4021,20 @@ func TestFullConfig(t *testing.T) {
// user configurable values
ACLAgentMasterToken: "furuQD0b",
ACLAgentToken: "cOshLOQ2",
ACLDatacenter: "m3urck3z",
ACLDefaultPolicy: "ArK3WIfE",
ACLDownPolicy: "vZXMfMP0",
ACLAgentMasterToken: "64fd0e08",
ACLAgentToken: "bed2377c",
ACLsEnabled: true,
ACLDatacenter: "ejtmd43d",
ACLDefaultPolicy: "72c2e7a0",
ACLDownPolicy: "03eb2aee",
ACLEnforceVersion8: true,
ACLEnableKeyListPolicy: true,
ACLMasterToken: "C1Q1oIwh",
ACLReplicationToken: "LMmgy5dO",
ACLTTL: 18060 * time.Second,
ACLToken: "O1El0wan",
ACLEnableKeyListPolicy: false,
ACLMasterToken: "8a19ac27",
ACLReplicationToken: "5795983a",
ACLTokenTTL: 3321 * time.Second,
ACLPolicyTTL: 1123 * time.Second,
ACLToken: "418fdff1",
ACLTokenReplication: true,
AdvertiseAddrLAN: ipAddr("17.99.29.16"),
AdvertiseAddrWAN: ipAddr("78.63.37.19"),
AutopilotCleanupDeadServers: true,
@ -4129,7 +4171,6 @@ func TestFullConfig(t *testing.T) {
DisableUpdateCheck: true,
DiscardCheckOutput: true,
DiscoveryMaxStale: 5 * time.Second,
EnableACLReplication: true,
EnableAgentTLSForChecks: true,
EnableDebug: true,
EnableRemoteScriptChecks: true,
@ -4802,9 +4843,12 @@ func TestSanitize(t *testing.T) {
"ACLEnableKeyListPolicy": false,
"ACLEnforceVersion8": false,
"ACLMasterToken": "hidden",
"ACLPolicyTTL": "0s",
"ACLReplicationToken": "hidden",
"ACLTTL": "0s",
"ACLTokenReplication": false,
"ACLTokenTTL": "0s",
"ACLToken": "hidden",
"ACLsEnabled": false,
"AEInterval": "0s",
"AdvertiseAddrLAN": "",
"AdvertiseAddrWAN": "",
@ -4919,7 +4963,6 @@ func TestSanitize(t *testing.T) {
"DisableUpdateCheck": false,
"DiscardCheckOutput": false,
"DiscoveryMaxStale": "0s",
"EnableACLReplication": false,
"EnableAgentTLSForChecks": false,
"EnableDebug": false,
"EnableLocalScriptChecks": false,

1040
agent/consul/acl.go
View File

File diff suppressed because it is too large Load Diff

101
agent/consul/acl_client.go Normal file
View File

@ -0,0 +1,101 @@
package consul
import (
"sync/atomic"
"time"
"github.com/hashicorp/consul/acl"
"github.com/hashicorp/consul/agent/metadata"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/consul/lib"
"github.com/hashicorp/serf/serf"
)
var clientACLCacheConfig *structs.ACLCachesConfig = &structs.ACLCachesConfig{
// The ACL cache configuration on client agents is more conservative than
// on the servers. It is assumed that individual client agents will have
// fewer distinct identities accessing the client than a server would
// and thus can put smaller limits on the amount of ACL caching done.
//
// Identities - number of identities/acl tokens that can be cached
Identities: 1024,
// Policies - number of unparsed ACL policies that can be cached
Policies: 128,
// ParsedPolicies - number of parsed ACL policies that can be cached
ParsedPolicies: 128,
// Authorizers - number of compiled multi-policy effective policies that can be cached
Authorizers: 256,
}
func (c *Client) UseLegacyACLs() bool {
return atomic.LoadInt32(&c.useNewACLs) == 0
}
func (c *Client) monitorACLMode() {
waitTime := aclModeCheckMinInterval
for {
canUpgrade := false
for _, member := range c.LANMembers() {
if valid, parts := metadata.IsConsulServer(member); valid && parts.Status == serf.StatusAlive {
if parts.ACLs != structs.ACLModeEnabled {
canUpgrade = false
break
} else {
canUpgrade = true
}
}
}
if canUpgrade {
c.logger.Printf("[DEBUG] acl: transition out of legacy ACL mode")
atomic.StoreInt32(&c.useNewACLs, 1)
lib.UpdateSerfTag(c.serf, "acls", string(structs.ACLModeEnabled))
return
}
select {
case <-c.shutdownCh:
return
case <-time.After(waitTime):
// do nothing
}
// calculate the amount of time to wait for the next round
waitTime = waitTime * 2
if waitTime > aclModeCheckMaxInterval {
waitTime = aclModeCheckMaxInterval
}
}
}
func (c *Client) ACLDatacenter(legacy bool) string {
// For resolution running on clients, when not in
// legacy mode the servers within the current datacenter
// must be queried first to pick up local tokens. When
// in legacy mode the clients should directly query the
// ACL Datacenter. When no ACL datacenter has been set
// then we assume that the local DC is the ACL DC
if legacy && c.config.ACLDatacenter != "" {
return c.config.ACLDatacenter
}
return c.config.Datacenter
}
func (c *Client) ACLsEnabled() bool {
return c.config.ACLsEnabled
}
func (c *Client) ResolveIdentityFromToken(token string) (bool, structs.ACLIdentity, error) {
// clients do no local identity resolution at the moment
return false, nil, nil
}
func (c *Client) ResolvePolicyFromID(policyID string) (bool, *structs.ACLPolicy, error) {
// clients do no local policy resolution at the moment
return false, nil, nil
}
func (c *Client) ResolveToken(token string) (acl.Authorizer, error) {
return c.acls.ResolveToken(token)
}

917
agent/consul/acl_endpoint.go
View File

File diff suppressed because it is too large Load Diff

258
agent/consul/acl_endpoint_legacy.go Normal file
View File

@ -0,0 +1,258 @@
package consul
import (
"fmt"
"time"
"github.com/armon/go-metrics"
"github.com/hashicorp/consul/acl"
"github.com/hashicorp/consul/agent/consul/state"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/consul/lib"
"github.com/hashicorp/go-memdb"
)
// Bootstrap is used to perform a one-time ACL bootstrap operation on
// a cluster to get the first management token.
func (a *ACL) Bootstrap(args *structs.DCSpecificRequest, reply *structs.ACL) error {
if done, err := a.srv.forward("ACL.Bootstrap", args, args, reply); done {
return err
}
// Verify we are allowed to serve this request
if !a.srv.InACLDatacenter() {
return acl.ErrDisabled
}
// By doing some pre-checks we can head off later bootstrap attempts
// without having to run them through Raft, which should curb abuse.
state := a.srv.fsm.State()
allowed, _, err := state.CanBootstrapACLToken()
if err != nil {
return err
}
if !allowed {
return structs.ACLBootstrapNotAllowedErr
}
// Propose a new token.
token, err := lib.GenerateUUID(a.srv.checkTokenUUID)
if err != nil {
return fmt.Errorf("failed to make random token: %v", err)
}
// Attempt a bootstrap.
req := structs.ACLRequest{
Datacenter: a.srv.config.ACLDatacenter,
Op: structs.ACLBootstrapNow,
ACL: structs.ACL{
ID: token,
Name: "Bootstrap Token",
Type: structs.ACLTokenTypeManagement,
},
}
resp, err := a.srv.raftApply(structs.ACLRequestType, &req)
if err != nil {
return err
}
switch v := resp.(type) {
case error:
return v
case *structs.ACL:
*reply = *v
default:
// Just log this, since it looks like the bootstrap may have
// completed.
a.srv.logger.Printf("[ERR] consul.acl: Unexpected response during bootstrap: %T", v)
}
a.srv.logger.Printf("[INFO] consul.acl: ACL bootstrap completed")
return nil
}
// aclApplyInternal is used to apply an ACL request after it has been vetted that
// this is a valid operation. It is used when users are updating ACLs, in which
// case we check their token to make sure they have management privileges. It is
// also used for ACL replication. We want to run the replicated ACLs through the
// same checks on the change itself.
func aclApplyInternal(srv *Server, args *structs.ACLRequest, reply *string) error {
// All ACLs must have an ID by this point.
if args.ACL.ID == "" {
return fmt.Errorf("Missing ACL ID")
}
switch args.Op {
case structs.ACLSet:
// Verify the ACL type
switch args.ACL.Type {
case structs.ACLTokenTypeClient:
case structs.ACLTokenTypeManagement:
default:
return fmt.Errorf("Invalid ACL Type")
}
_, existing, _ := srv.fsm.State().ACLTokenGetBySecret(nil, args.ACL.ID)
if existing != nil && len(existing.Policies) > 0 {
return fmt.Errorf("Cannot use legacy endpoint to modify a non-legacy token")
}
// Verify this is not a root ACL
if acl.RootAuthorizer(args.ACL.ID) != nil {
return acl.PermissionDeniedError{Cause: "Cannot modify root ACL"}
}
// Validate the rules compile
_, err := acl.NewPolicyFromSource("", 0, args.ACL.Rules, acl.SyntaxLegacy, srv.sentinel)
if err != nil {
return fmt.Errorf("ACL rule compilation failed: %v", err)
}
case structs.ACLDelete:
if args.ACL.ID == anonymousToken {
return acl.PermissionDeniedError{Cause: "Cannot delete anonymous token"}
}
default:
return fmt.Errorf("Invalid ACL Operation")
}
// Apply the update
resp, err := srv.raftApply(structs.ACLRequestType, args)
if err != nil {
srv.logger.Printf("[ERR] consul.acl: Apply failed: %v", err)
return err
}
if respErr, ok := resp.(error); ok {
return respErr
}
// Check if the return type is a string
if respString, ok := resp.(string); ok {
*reply = respString
}
return nil
}
// Apply is used to apply a modifying request to the data store. This should
// only be used for operations that modify the data
func (a *ACL) Apply(args *structs.ACLRequest, reply *string) error {
if done, err := a.srv.forward("ACL.Apply", args, args, reply); done {
return err
}
defer metrics.MeasureSince([]string{"acl", "apply"}, time.Now())
// Verify we are allowed to serve this request
if a.srv.config.ACLDatacenter != a.srv.config.Datacenter {
return acl.ErrDisabled
}
// Verify token is permitted to modify ACLs
if rule, err := a.srv.ResolveToken(args.Token); err != nil {
return err
} else if rule == nil || !rule.ACLWrite() {
return acl.ErrPermissionDenied
}
// If no ID is provided, generate a new ID. This must be done prior to
// appending to the Raft log, because the ID is not deterministic. Once
// the entry is in the log, the state update MUST be deterministic or
// the followers will not converge.
if args.Op == structs.ACLSet && args.ACL.ID == "" {
var err error
args.ACL.ID, err = lib.GenerateUUID(a.srv.checkTokenUUID)
if err != nil {
return err
}
}
// Do the apply now that this update is vetted.
if err := aclApplyInternal(a.srv, args, reply); err != nil {
return err
}
// Clear the cache if applicable
if args.ACL.ID != "" {
a.srv.acls.cache.RemoveIdentity(args.ACL.ID)
}
return nil
}
// Get is used to retrieve a single ACL
func (a *ACL) Get(args *structs.ACLSpecificRequest,
reply *structs.IndexedACLs) error {
if done, err := a.srv.forward("ACL.Get", args, args, reply); done {
return err
}
// Verify we are allowed to serve this request
if a.srv.config.ACLDatacenter != a.srv.config.Datacenter {
return acl.ErrDisabled
}
return a.srv.blockingQuery(&args.QueryOptions,
&reply.QueryMeta,
func(ws memdb.WatchSet, state *state.Store) error {
index, token, err := state.ACLTokenGetBySecret(ws, args.ACL)
if err != nil {
return err
}
// converting an ACLToken to an ACL will return nil and an error
// (which we ignore) when it is unconvertible.
var acl *structs.ACL
if token != nil {
acl, _ = token.Convert()
}
reply.Index = index
if acl != nil {
reply.ACLs = structs.ACLs{acl}
} else {
reply.ACLs = nil
}
return nil
})
}
// List is used to list all the ACLs
func (a *ACL) List(args *structs.DCSpecificRequest,
reply *structs.IndexedACLs) error {
if done, err := a.srv.forward("ACL.List", args, args, reply); done {
return err
}
// Verify we are allowed to serve this request
if a.srv.config.ACLDatacenter != a.srv.config.Datacenter {
return acl.ErrDisabled
}
// Verify token is permitted to list ACLs
if rule, err := a.srv.ResolveToken(args.Token); err != nil {
return err
} else if rule == nil || !rule.ACLWrite() {
return acl.ErrPermissionDenied
}
return a.srv.blockingQuery(&args.QueryOptions,
&reply.QueryMeta,
func(ws memdb.WatchSet, state *state.Store) error {
index, tokens, err := state.ACLTokenList(ws, false, true, "")
if err != nil {
return err
}
var acls structs.ACLs
for _, token := range tokens {
if acl, err := token.Convert(); err == nil && acl != nil {
acls = append(acls, acl)
}
}
reply.Index, reply.ACLs = index, acls
return nil
})
}

1082
agent/consul/acl_endpoint_test.go
View File

File diff suppressed because it is too large Load Diff

753
agent/consul/acl_replication.go
View File

@ -1,347 +1,552 @@
package consul
import (
"bytes"
"context"
"fmt"
"sort"
"time"
"github.com/armon/go-metrics"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/consul/lib"
)
// aclIterator simplifies the algorithm below by providing a basic iterator that
// moves through a list of ACLs and returns nil when it's exhausted. It also has
// methods for pre-sorting the ACLs being iterated over by ID, which should
// already be true, but since this is crucial for correctness and we are taking
// input from other servers, we sort to make sure.
type aclIterator struct {
acls structs.ACLs
const (
// aclReplicationMaxRetryBackoff is the max number of seconds to sleep between ACL replication RPC errors
aclReplicationMaxRetryBackoff = 64
)
// index is the current position of the iterator.
index int
}
func diffACLPolicies(local structs.ACLPolicies, remote structs.ACLPolicyListStubs, lastRemoteIndex uint64) ([]string, []string) {
local.Sort()
remote.Sort()
// newACLIterator returns a new ACL iterator.
func newACLIterator(acls structs.ACLs) *aclIterator {
return &aclIterator{acls: acls}
}
var deletions []string
var updates []string
var localIdx int
var remoteIdx int
for localIdx, remoteIdx = 0, 0; localIdx < len(local) && remoteIdx < len(remote); {
if local[localIdx].ID == remote[remoteIdx].ID {
// policy is in both the local and remote state - need to check raft indices and the Hash
if remote[remoteIdx].ModifyIndex > lastRemoteIndex && !bytes.Equal(remote[remoteIdx].Hash, local[localIdx].Hash) {
updates = append(updates, remote[remoteIdx].ID)
}
// increment both indices when equal
localIdx += 1
remoteIdx += 1
} else if local[localIdx].ID < remote[remoteIdx].ID {
// policy no longer in remoted state - needs deleting
deletions = append(deletions, local[localIdx].ID)
// See sort.Interface.
func (a *aclIterator) Len() int {
return len(a.acls)
}
// increment just the local index
localIdx += 1
} else {
// local state doesn't have this policy - needs updating
updates = append(updates, remote[remoteIdx].ID)
// See sort.Interface.
func (a *aclIterator) Swap(i, j int) {
a.acls[i], a.acls[j] = a.acls[j], a.acls[i]
}
// See sort.Interface.
func (a *aclIterator) Less(i, j int) bool {
return a.acls[i].ID < a.acls[j].ID
}
// Front returns the item at index position, or nil if the list is exhausted.
func (a *aclIterator) Front() *structs.ACL {
if a.index < len(a.acls) {
return a.acls[a.index]
// increment just the remote index
remoteIdx += 1
}
}
return nil
}
// Next advances the iterator to the next index.
func (a *aclIterator) Next() {
a.index++
}
// reconcileACLs takes the local and remote ACL state, and produces a list of
// changes required in order to bring the local ACLs into sync with the remote
// ACLs. You can supply lastRemoteIndex as a hint that replication has succeeded
// up to that remote index and it will make this process more efficient by only
// comparing ACL entries modified after that index. Setting this to 0 will force
// a full compare of all existing ACLs.
func reconcileACLs(local, remote structs.ACLs, lastRemoteIndex uint64) structs.ACLRequests {
// Since sorting the lists is crucial for correctness, we are depending
// on data coming from other servers potentially running a different,
// version of Consul, and sorted-ness is kind of a subtle property of
// the state store indexing, it's prudent to make sure things are sorted
// before we begin.
localIter, remoteIter := newACLIterator(local), newACLIterator(remote)
sort.Sort(localIter)
sort.Sort(remoteIter)
// Run through both lists and reconcile them.
var changes structs.ACLRequests
for localIter.Front() != nil || remoteIter.Front() != nil {
// If the local list is exhausted, then process this as a remote
// add. We know from the loop condition that there's something
// in the remote list.
if localIter.Front() == nil {
changes = append(changes, &structs.ACLRequest{
Op: structs.ACLSet,
ACL: *(remoteIter.Front()),
})
remoteIter.Next()
continue
}
// If the remote list is exhausted, then process this as a local
// delete. We know from the loop condition that there's something
// in the local list.
if remoteIter.Front() == nil {
changes = append(changes, &structs.ACLRequest{
Op: structs.ACLDelete,
ACL: *(localIter.Front()),
})
localIter.Next()
continue
}
// At this point we know there's something at the front of each
// list we need to resolve.
// If the remote list has something local doesn't, we add it.
if localIter.Front().ID > remoteIter.Front().ID {
changes = append(changes, &structs.ACLRequest{
Op: structs.ACLSet,
ACL: *(remoteIter.Front()),
})
remoteIter.Next()
continue
}
// If local has something remote doesn't, we delete it.
if localIter.Front().ID < remoteIter.Front().ID {
changes = append(changes, &structs.ACLRequest{
Op: structs.ACLDelete,
ACL: *(localIter.Front()),
})
localIter.Next()
continue
}
// Local and remote have an ACL with the same ID, so we might
// need to compare them.
l, r := localIter.Front(), remoteIter.Front()
if r.RaftIndex.ModifyIndex > lastRemoteIndex && !r.IsSame(l) {
changes = append(changes, &structs.ACLRequest{
Op: structs.ACLSet,
ACL: *r,
})
}
localIter.Next()
remoteIter.Next()
for ; localIdx < len(local); localIdx += 1 {
deletions = append(deletions, local[localIdx].ID)
}
return changes
}
// FetchLocalACLs returns the ACLs in the local state store.
func (s *Server) fetchLocalACLs() (structs.ACLs, error) {
_, local, err := s.fsm.State().ACLList(nil)
if err != nil {
return nil, err
for ; remoteIdx < len(remote); remoteIdx += 1 {
updates = append(updates, remote[remoteIdx].ID)
}
return local, nil
return deletions, updates
}
// FetchRemoteACLs is used to get the remote set of ACLs from the ACL
// datacenter. The lastIndex parameter is a hint about which remote index we
// have replicated to, so this is expected to block until something changes.
func (s *Server) fetchRemoteACLs(lastRemoteIndex uint64) (*structs.IndexedACLs, error) {
defer metrics.MeasureSince([]string{"leader", "fetchRemoteACLs"}, time.Now())
func (s *Server) deleteLocalACLPolicies(deletions []string, ctx context.Context) (bool, error) {
ticker := time.NewTicker(time.Second / time.Duration(s.config.ACLReplicationApplyLimit))
defer ticker.Stop()
args := structs.DCSpecificRequest{
for i := 0; i < len(deletions); i += aclBatchDeleteSize {
req := structs.ACLPolicyBatchDeleteRequest{}
if i+aclBatchDeleteSize > len(deletions) {
req.PolicyIDs = deletions[i:]
} else {
req.PolicyIDs = deletions[i : i+aclBatchDeleteSize]
}
resp, err := s.raftApply(structs.ACLPolicyDeleteRequestType, &req)
if err != nil {
return false, fmt.Errorf("Failed to apply policy deletions: %v", err)
}
if respErr, ok := resp.(error); ok && err != nil {
return false, fmt.Errorf("Failed to apply policy deletions: %v", respErr)
}
if i+aclBatchDeleteSize < len(deletions) {
select {
case <-ctx.Done():
return true, nil
case <-ticker.C:
// do nothing - ready for the next batch
}
}
}
return false, nil
}
func (s *Server) updateLocalACLPolicies(policies structs.ACLPolicies, ctx context.Context) (bool, error) {
ticker := time.NewTicker(time.Second / time.Duration(s.config.ACLReplicationApplyLimit))
defer ticker.Stop()
// outer loop handles submitting a batch
for batchStart := 0; batchStart < len(policies); {
// inner loop finds the last element to include in this batch.
batchSize := 0
batchEnd := batchStart
for ; batchEnd < len(policies) && batchSize < aclBatchUpsertSize; batchEnd += 1 {
batchSize += policies[batchEnd].EstimateSize()
}
req := structs.ACLPolicyBatchUpsertRequest{
Policies: policies[batchStart:batchEnd],
}
resp, err := s.raftApply(structs.ACLPolicyUpsertRequestType, &req)
if err != nil {
return false, fmt.Errorf("Failed to apply policy upserts: %v", err)
}
if respErr, ok := resp.(error); ok && err != nil {
return false, fmt.Errorf("Failed to apply policy upsert: %v", respErr)
}
s.logger.Printf("[DEBUG] acl: policy replication - upserted 1 batch with %d policies of size %d", batchEnd-batchStart, batchSize)
// policies[batchEnd] wasn't include as the slicing doesn't include the element at the stop index
batchStart = batchEnd
// prevent waiting if we are done
if batchEnd < len(policies) {
select {
case <-ctx.Done():
return true, nil
case <-ticker.C:
// nothing to do - just rate limiting
}
}
}
return false, nil
}
func (s *Server) fetchACLPoliciesBatch(policyIDs []string) (*structs.ACLPoliciesResponse, error) {
req := structs.ACLPolicyBatchReadRequest{
Datacenter: s.config.ACLDatacenter,
PolicyIDs: policyIDs,
QueryOptions: structs.QueryOptions{
Token: s.tokens.ACLReplicationToken(),
MinQueryIndex: lastRemoteIndex,
AllowStale: true,
AllowStale: true,
Token: s.tokens.ACLReplicationToken(),
},
}
var remote structs.IndexedACLs
if err := s.RPC("ACL.List", &args, &remote); err != nil {
var response structs.ACLPoliciesResponse
if err := s.RPC("ACL.PolicyBatchRead", &req, &response); err != nil {
return nil, err
}
return &remote, nil
return &response, nil
}
// UpdateLocalACLs is given a list of changes to apply in order to bring the
// local ACLs in-line with the remote ACLs from the ACL datacenter.
func (s *Server) updateLocalACLs(changes structs.ACLRequests) error {
defer metrics.MeasureSince([]string{"leader", "updateLocalACLs"}, time.Now())
func (s *Server) fetchACLPolicies(lastRemoteIndex uint64) (*structs.ACLPolicyListResponse, error) {
defer metrics.MeasureSince([]string{"leader", "replication", "acl", "policy", "fetch"}, time.Now())
minTimePerOp := time.Second / time.Duration(s.config.ACLReplicationApplyLimit)
for _, change := range changes {
// Note that we are using the single ACL interface here and not
// performing all this inside a single transaction. This is OK
// for two reasons. First, there's nothing else other than this
// replication routine that alters the local ACLs, so there's
// nothing to contend with locally. Second, if an apply fails
// in the middle (most likely due to losing leadership), the
// next replication pass will clean up and check everything
// again.
var reply string
start := time.Now()
if err := aclApplyInternal(s, change, &reply); err != nil {
return err
req := structs.ACLPolicyListRequest{
Datacenter: s.config.ACLDatacenter,
QueryOptions: structs.QueryOptions{
AllowStale: true,
MinQueryIndex: lastRemoteIndex,
Token: s.tokens.ACLReplicationToken(),
},
}
var response structs.ACLPolicyListResponse
if err := s.RPC("ACL.PolicyList", &req, &response); err != nil {
return nil, err
}
return &response, nil
}
func diffACLTokens(local structs.ACLTokens, remote structs.ACLTokenListStubs, lastRemoteIndex uint64) ([]string, []string) {
local.Sort()
remote.Sort()
var deletions []string
var updates []string
var localIdx int
var remoteIdx int
for localIdx, remoteIdx = 0, 0; localIdx < len(local) && remoteIdx < len(remote); {
if local[localIdx].AccessorID == remote[remoteIdx].AccessorID {
// policy is in both the local and remote state - need to check raft indices and Hash
if remote[remoteIdx].ModifyIndex > lastRemoteIndex && !bytes.Equal(remote[remoteIdx].Hash, local[localIdx].Hash) {
updates = append(updates, remote[remoteIdx].AccessorID)
}
// increment both indices when equal
localIdx += 1
remoteIdx += 1
} else if local[localIdx].AccessorID < remote[remoteIdx].AccessorID {
// policy no longer in remoted state - needs deleting
deletions = append(deletions, local[localIdx].AccessorID)
// increment just the local index
localIdx += 1
} else {
// local state doesn't have this policy - needs updating
updates = append(updates, remote[remoteIdx].AccessorID)
// increment just the remote index
remoteIdx += 1
}
}
for ; localIdx < len(local); localIdx += 1 {
deletions = append(deletions, local[localIdx].AccessorID)
}
for ; remoteIdx < len(remote); remoteIdx += 1 {
updates = append(updates, remote[remoteIdx].AccessorID)
}
return deletions, updates
}
func (s *Server) deleteLocalACLTokens(deletions []string, ctx context.Context) (bool, error) {
ticker := time.NewTicker(time.Second / time.Duration(s.config.ACLReplicationApplyLimit))
defer ticker.Stop()
for i := 0; i < len(deletions); i += aclBatchDeleteSize {
req := structs.ACLTokenBatchDeleteRequest{}
if i+aclBatchDeleteSize > len(deletions) {
req.TokenIDs = deletions[i:]
} else {
req.TokenIDs = deletions[i : i+aclBatchDeleteSize]
}
// Do a smooth rate limit to wait out the min time allowed for
// each op. If this op took longer than the min, then the sleep
// time will be negative and we will just move on.
elapsed := time.Since(start)
time.Sleep(minTimePerOp - elapsed)
resp, err := s.raftApply(structs.ACLTokenDeleteRequestType, &req)
if err != nil {
return false, fmt.Errorf("Failed to apply token deletions: %v", err)
}
if respErr, ok := resp.(error); ok && err != nil {
return false, fmt.Errorf("Failed to apply token deletions: %v", respErr)
}
if i+aclBatchDeleteSize < len(deletions) {
select {
case <-ctx.Done():
return true, nil
case <-ticker.C:
// do nothing - ready for the next batch
}
}
}
return nil
return false, nil
}
// replicateACLs is a runs one pass of the algorithm for replicating ACLs from
// a remote ACL datacenter to local state. If there's any error, this will return
// 0 for the lastRemoteIndex, which will cause us to immediately do a full sync
// next time.
func (s *Server) replicateACLs(lastRemoteIndex uint64) (uint64, error) {
remote, err := s.fetchRemoteACLs(lastRemoteIndex)
if err != nil {
return 0, fmt.Errorf("failed to retrieve remote ACLs: %v", err)
func (s *Server) updateLocalACLTokens(tokens structs.ACLTokens, ctx context.Context) (bool, error) {
ticker := time.NewTicker(time.Second / time.Duration(s.config.ACLReplicationApplyLimit))
defer ticker.Stop()
// outer loop handles submitting a batch
for batchStart := 0; batchStart < len(tokens); {
// inner loop finds the last element to include in this batch.
batchSize := 0
batchEnd := batchStart
for ; batchEnd < len(tokens) && batchSize < aclBatchUpsertSize; batchEnd += 1 {
batchSize += tokens[batchEnd].EstimateSize()
}
req := structs.ACLTokenBatchUpsertRequest{
Tokens: tokens[batchStart:batchEnd],
AllowCreate: true,
}
resp, err := s.raftApply(structs.ACLTokenUpsertRequestType, &req)
if err != nil {
return false, fmt.Errorf("Failed to apply token upserts: %v", err)
}
if respErr, ok := resp.(error); ok && err != nil {
return false, fmt.Errorf("Failed to apply token upserts: %v", respErr)
}
s.logger.Printf("[DEBUG] acl: token replication - upserted 1 batch with %d tokens of size %d", batchEnd-batchStart, batchSize)
// tokens[batchEnd] wasn't include as the slicing doesn't include the element at the stop index
batchStart = batchEnd
// prevent waiting if we are done
if batchEnd < len(tokens) {
select {
case <-ctx.Done():
return true, nil
case <-ticker.C:
// nothing to do - just rate limiting here
}
}
}
return false, nil
}
func (s *Server) fetchACLTokensBatch(tokenIDs []string) (*structs.ACLTokensResponse, error) {
req := structs.ACLTokenBatchReadRequest{
Datacenter: s.config.ACLDatacenter,
AccessorIDs: tokenIDs,
QueryOptions: structs.QueryOptions{
AllowStale: true,
Token: s.tokens.ACLReplicationToken(),
},
}
// This will be pretty common because we will be blocking for a long time
// and may have lost leadership, so lets control the message here instead
// of returning deeper error messages from from Raft.
if !s.IsLeader() {
return 0, fmt.Errorf("no longer cluster leader")
var response structs.ACLTokensResponse
if err := s.RPC("ACL.TokenBatchRead", &req, &response); err != nil {
return nil, err
}
return &response, nil
}
func (s *Server) fetchACLTokens(lastRemoteIndex uint64) (*structs.ACLTokenListResponse, error) {
defer metrics.MeasureSince([]string{"leader", "replication", "acl", "token", "fetch"}, time.Now())
req := structs.ACLTokenListRequest{
Datacenter: s.config.ACLDatacenter,
QueryOptions: structs.QueryOptions{
AllowStale: true,
MinQueryIndex: lastRemoteIndex,
Token: s.tokens.ACLReplicationToken(),
},
IncludeLocal: false,
IncludeGlobal: true,
}
var response structs.ACLTokenListResponse
if err := s.RPC("ACL.TokenList", &req, &response); err != nil {
return nil, err
}
return &response, nil
}
func (s *Server) replicateACLPolicies(lastRemoteIndex uint64, ctx context.Context) (uint64, bool, error) {
remote, err := s.fetchACLPolicies(lastRemoteIndex)
if err != nil {
return 0, false, fmt.Errorf("failed to retrieve remote ACL policies: %v", err)
}
s.logger.Printf("[DEBUG] acl: finished fetching policies tokens: %d", len(remote.Policies))
// Need to check if we should be stopping. This will be common as the fetching process is a blocking
// RPC which could have been hanging around for a long time and during that time leadership could
// have been lost.
select {
case <-ctx.Done():
return 0, true, nil
default:
// do nothing
}
// Measure everything after the remote query, which can block for long
// periods of time. This metric is a good measure of how expensive the
// replication process is.
defer metrics.MeasureSince([]string{"leader", "replicateACLs"}, time.Now())
defer metrics.MeasureSince([]string{"leader", "replication", "acl", "policy", "apply"}, time.Now())
local, err := s.fetchLocalACLs()
_, local, err := s.fsm.State().ACLPolicyList(nil, "")
if err != nil {
return 0, fmt.Errorf("failed to retrieve local ACLs: %v", err)
return 0, false, fmt.Errorf("failed to retrieve local ACL policies: %v", err)
}
// If the remote index ever goes backwards, it's a good indication that
// the remote side was rebuilt and we should do a full sync since we
// can't make any assumptions about what's going on.
if remote.QueryMeta.Index < lastRemoteIndex {
s.logger.Printf("[WARN] consul: ACL replication remote index moved backwards (%d to %d), forcing a full ACL sync", lastRemoteIndex, remote.QueryMeta.Index)
s.logger.Printf("[WARN] consul: ACL policy replication remote index moved backwards (%d to %d), forcing a full ACL policy sync", lastRemoteIndex, remote.QueryMeta.Index)
lastRemoteIndex = 0
}
s.logger.Printf("[DEBUG] acl: policy replication - local: %d, remote: %d", len(local), len(remote.Policies))
// Calculate the changes required to bring the state into sync and then
// apply them.
changes := reconcileACLs(local, remote.ACLs, lastRemoteIndex)
if err := s.updateLocalACLs(changes); err != nil {
return 0, fmt.Errorf("failed to sync ACL changes: %v", err)
deletions, updates := diffACLPolicies(local, remote.Policies, lastRemoteIndex)
s.logger.Printf("[DEBUG] acl: policy replication - deletions: %d, updates: %d", len(deletions), len(updates))
var policies *structs.ACLPoliciesResponse
if len(updates) > 0 {
policies, err = s.fetchACLPoliciesBatch(updates)
if err != nil {
return 0, false, fmt.Errorf("failed to retrieve ACL policy updates: %v", err)
}
s.logger.Printf("[DEBUG] acl: policy replication - downloaded %d policies", len(policies.Policies))
}
if len(deletions) > 0 {
s.logger.Printf("[DEBUG] acl: policy replication - performing deletions")
exit, err := s.deleteLocalACLPolicies(deletions, ctx)
if exit {
return 0, true, nil
}
if err != nil {
return 0, false, fmt.Errorf("failed to delete local ACL policies: %v", err)
}
s.logger.Printf("[DEBUG] acl: policy replication - finished deletions")
}
if len(updates) > 0 {
s.logger.Printf("[DEBUG] acl: policy replication - performing updates")
exit, err := s.updateLocalACLPolicies(policies.Policies, ctx)
if exit {
return 0, true, nil
}
if err != nil {
return 0, false, fmt.Errorf("failed to update local ACL policies: %v", err)
}
s.logger.Printf("[DEBUG] acl: policy replication - finished updates")
}
// Return the index we got back from the remote side, since we've synced
// up with the remote state as of that index.
return remote.QueryMeta.Index, nil
return remote.QueryMeta.Index, false, nil
}
func (s *Server) replicateACLTokens(lastRemoteIndex uint64, ctx context.Context) (uint64, bool, error) {
remote, err := s.fetchACLTokens(lastRemoteIndex)
if err != nil {
return 0, false, fmt.Errorf("failed to retrieve remote ACL tokens: %v", err)
}
s.logger.Printf("[DEBUG] acl: finished fetching remote tokens: %d", len(remote.Tokens))
// Need to check if we should be stopping. This will be common as the fetching process is a blocking
// RPC which could have been hanging around for a long time and during that time leadership could
// have been lost.
select {
case <-ctx.Done():
return 0, true, nil
default:
// do nothing
}
// Measure everything after the remote query, which can block for long
// periods of time. This metric is a good measure of how expensive the
// replication process is.
defer metrics.MeasureSince([]string{"leader", "replication", "acl", "token", "apply"}, time.Now())
_, local, err := s.fsm.State().ACLTokenList(nil, false, true, "")
if err != nil {
return 0, false, fmt.Errorf("failed to retrieve local ACL tokens: %v", err)
}
// If the remote index ever goes backwards, it's a good indication that
// the remote side was rebuilt and we should do a full sync since we
// can't make any assumptions about what's going on.
if remote.QueryMeta.Index < lastRemoteIndex {
s.logger.Printf("[WARN] consul: ACL token replication remote index moved backwards (%d to %d), forcing a full ACL token sync", lastRemoteIndex, remote.QueryMeta.Index)
lastRemoteIndex = 0
}
s.logger.Printf("[DEBUG] acl: token replication - local: %d, remote: %d", len(local), len(remote.Tokens))
// Calculate the changes required to bring the state into sync and then
// apply them.
deletions, updates := diffACLTokens(local, remote.Tokens, lastRemoteIndex)
s.logger.Printf("[DEBUG] acl: token replication - deletions: %d, updates: %d", len(deletions), len(updates))
var tokens *structs.ACLTokensResponse
if len(updates) > 0 {
tokens, err = s.fetchACLTokensBatch(updates)
if err != nil {
return 0, false, fmt.Errorf("failed to retrieve ACL token updates: %v", err)
}
s.logger.Printf("[DEBUG] acl: token replication - downloaded %d tokens", len(tokens.Tokens))
}
if len(deletions) > 0 {
s.logger.Printf("[DEBUG] acl: token replication - performing deletions")
exit, err := s.deleteLocalACLTokens(deletions, ctx)
if exit {
return 0, true, nil
}
if err != nil {
return 0, false, fmt.Errorf("failed to delete local ACL tokens: %v", err)
}
s.logger.Printf("[DEBUG] acl: token replication - finished deletions")
}
if len(updates) > 0 {
s.logger.Printf("[DEBUG] acl: token replication - performing updates")
exit, err := s.updateLocalACLTokens(tokens.Tokens, ctx)
if exit {
return 0, true, nil
}
if err != nil {
return 0, false, fmt.Errorf("failed to update local ACL tokens: %v", err)
}
s.logger.Printf("[DEBUG] acl: token replication - finished updates")
}
// Return the index we got back from the remote side, since we've synced
// up with the remote state as of that index.
return remote.QueryMeta.Index, false, nil
}
// IsACLReplicationEnabled returns true if ACL replication is enabled.
// DEPRECATED (ACL-Legacy-Compat) - with new ACLs at least policy replication is required
func (s *Server) IsACLReplicationEnabled() bool {
authDC := s.config.ACLDatacenter
return len(authDC) > 0 && (authDC != s.config.Datacenter) &&
s.config.EnableACLReplication
s.config.ACLTokenReplication
}
// updateACLReplicationStatus safely updates the ACL replication status.
func (s *Server) updateACLReplicationStatus(status structs.ACLReplicationStatus) {
// Fixup the times to shed some useless precision to ease formatting,
// and always report UTC.
status.LastError = status.LastError.Round(time.Second).UTC()
status.LastSuccess = status.LastSuccess.Round(time.Second).UTC()
// Set the shared state.
func (s *Server) updateACLReplicationStatusError() {
s.aclReplicationStatusLock.Lock()
s.aclReplicationStatus = status
s.aclReplicationStatusLock.Unlock()
defer s.aclReplicationStatusLock.Unlock()
s.aclReplicationStatus.LastError = time.Now().Round(time.Second).UTC()
}
// runACLReplication is a long-running goroutine that will attempt to replicate
// ACLs while the server is the leader, until the shutdown channel closes.
func (s *Server) runACLReplication() {
var status structs.ACLReplicationStatus
status.Enabled = true
status.SourceDatacenter = s.config.ACLDatacenter
s.updateACLReplicationStatus(status)
func (s *Server) updateACLReplicationStatusIndex(index uint64) {
s.aclReplicationStatusLock.Lock()
defer s.aclReplicationStatusLock.Unlock()
// Show that it's not running on the way out.
defer func() {
status.Running = false
s.updateACLReplicationStatus(status)
}()
// Give each server's replicator a random initial phase for good
// measure.
select {
case <-s.shutdownCh:
return
case <-time.After(lib.RandomStagger(s.config.ACLReplicationInterval)):
}
// We are fairly conservative with the lastRemoteIndex so that after a
// leadership change or an error we re-sync everything (we also don't
// want to block the first time after one of these events so we can
// show a successful sync in the status endpoint).
var lastRemoteIndex uint64
replicate := func() {
if !status.Running {
lastRemoteIndex = 0 // Re-sync everything.
status.Running = true
s.updateACLReplicationStatus(status)
s.logger.Printf("[INFO] consul: ACL replication started")
}
index, err := s.replicateACLs(lastRemoteIndex)
if err != nil {
lastRemoteIndex = 0 // Re-sync everything.
status.LastError = time.Now()
s.updateACLReplicationStatus(status)
s.logger.Printf("[WARN] consul: ACL replication error (will retry if still leader): %v", err)
} else {
lastRemoteIndex = index
status.ReplicatedIndex = index
status.LastSuccess = time.Now()
s.updateACLReplicationStatus(status)
s.logger.Printf("[DEBUG] consul: ACL replication completed through remote index %d", index)
}
}
pause := func() {
if status.Running {
lastRemoteIndex = 0 // Re-sync everything.
status.Running = false
s.updateACLReplicationStatus(status)
s.logger.Printf("[INFO] consul: ACL replication stopped (no longer leader)")
}
}
// This will slowly poll to see if replication should be active. Once it
// is and we've caught up, the replicate() call will begin to block and
// only wake up when the query timer expires or there are new ACLs to
// replicate. We've chosen this design so that the ACLReplicationInterval
// is the lower bound for how quickly we will replicate, no matter how
// much ACL churn is happening on the remote side.
//
// The blocking query inside replicate() respects the shutdown channel,
// so we won't get stuck in here as things are torn down.
for {
select {
case <-s.shutdownCh:
return
case <-time.After(s.config.ACLReplicationInterval):
if s.IsLeader() {
replicate()
} else {
pause()
}
}
}
s.aclReplicationStatus.LastSuccess = time.Now().Round(time.Second).UTC()
s.aclReplicationStatus.ReplicatedIndex = index
}
func (s *Server) updateACLReplicationStatusTokenIndex(index uint64) {
s.aclReplicationStatusLock.Lock()
defer s.aclReplicationStatusLock.Unlock()
s.aclReplicationStatus.LastSuccess = time.Now().Round(time.Second).UTC()
s.aclReplicationStatus.ReplicatedTokenIndex = index
}
func (s *Server) initReplicationStatus() {
s.aclReplicationStatusLock.Lock()
defer s.aclReplicationStatusLock.Unlock()
s.aclReplicationStatus.Enabled = true
s.aclReplicationStatus.Running = true
s.aclReplicationStatus.SourceDatacenter = s.config.ACLDatacenter
}
func (s *Server) updateACLReplicationStatusStopped() {
s.aclReplicationStatusLock.Lock()
defer s.aclReplicationStatusLock.Unlock()
s.aclReplicationStatus.Running = false
}
func (s *Server) updateACLReplicationStatusRunning(replicationType structs.ACLReplicationType) {
s.aclReplicationStatusLock.Lock()
defer s.aclReplicationStatusLock.Unlock()
s.aclReplicationStatus.Running = true
s.aclReplicationStatus.ReplicationType = replicationType
}

265
agent/consul/acl_replication_legacy.go Normal file
View File

@ -0,0 +1,265 @@
package consul
import (
"context"
"fmt"
"sort"
"time"
"github.com/armon/go-metrics"
"github.com/hashicorp/consul/agent/structs"
)
// aclIterator simplifies the algorithm below by providing a basic iterator that
// moves through a list of ACLs and returns nil when it's exhausted. It also has
// methods for pre-sorting the ACLs being iterated over by ID, which should
// already be true, but since this is crucial for correctness and we are taking
// input from other servers, we sort to make sure.
type aclIterator struct {
acls structs.ACLs
// index is the current position of the iterator.
index int
}
// newACLIterator returns a new ACL iterator.
func newACLIterator(acls structs.ACLs) *aclIterator {
return &aclIterator{acls: acls}
}
// See sort.Interface.
func (a *aclIterator) Len() int {
return len(a.acls)
}
// See sort.Interface.
func (a *aclIterator) Swap(i, j int) {
a.acls[i], a.acls[j] = a.acls[j], a.acls[i]
}
// See sort.Interface.
func (a *aclIterator) Less(i, j int) bool {
return a.acls[i].ID < a.acls[j].ID
}
// Front returns the item at index position, or nil if the list is exhausted.
func (a *aclIterator) Front() *structs.ACL {
if a.index < len(a.acls) {
return a.acls[a.index]
}
return nil
}
// Next advances the iterator to the next index.
func (a *aclIterator) Next() {
a.index++
}
// reconcileACLs takes the local and remote ACL state, and produces a list of
// changes required in order to bring the local ACLs into sync with the remote
// ACLs. You can supply lastRemoteIndex as a hint that replication has succeeded
// up to that remote index and it will make this process more efficient by only
// comparing ACL entries modified after that index. Setting this to 0 will force
// a full compare of all existing ACLs.
func reconcileLegacyACLs(local, remote structs.ACLs, lastRemoteIndex uint64) structs.ACLRequests {
// Since sorting the lists is crucial for correctness, we are depending
// on data coming from other servers potentially running a different,
// version of Consul, and sorted-ness is kind of a subtle property of
// the state store indexing, it's prudent to make sure things are sorted
// before we begin.
localIter, remoteIter := newACLIterator(local), newACLIterator(remote)
sort.Sort(localIter)
sort.Sort(remoteIter)
// Run through both lists and reconcile them.
var changes structs.ACLRequests
for localIter.Front() != nil || remoteIter.Front() != nil {
// If the local list is exhausted, then process this as a remote
// add. We know from the loop condition that there's something
// in the remote list.
if localIter.Front() == nil {
changes = append(changes, &structs.ACLRequest{
Op: structs.ACLSet,
ACL: *(remoteIter.Front()),
})
remoteIter.Next()
continue
}
// If the remote list is exhausted, then process this as a local
// delete. We know from the loop condition that there's something
// in the local list.
if remoteIter.Front() == nil {
changes = append(changes, &structs.ACLRequest{
Op: structs.ACLDelete,
ACL: *(localIter.Front()),
})
localIter.Next()
continue
}
// At this point we know there's something at the front of each
// list we need to resolve.
// If the remote list has something local doesn't, we add it.
if localIter.Front().ID > remoteIter.Front().ID {
changes = append(changes, &structs.ACLRequest{
Op: structs.ACLSet,
ACL: *(remoteIter.Front()),
})
remoteIter.Next()
continue
}
// If local has something remote doesn't, we delete it.
if localIter.Front().ID < remoteIter.Front().ID {
changes = append(changes, &structs.ACLRequest{
Op: structs.ACLDelete,
ACL: *(localIter.Front()),
})
localIter.Next()
continue
}
// Local and remote have an ACL with the same ID, so we might
// need to compare them.
l, r := localIter.Front(), remoteIter.Front()
if r.RaftIndex.ModifyIndex > lastRemoteIndex && !r.IsSame(l) {
changes = append(changes, &structs.ACLRequest{
Op: structs.ACLSet,
ACL: *r,
})
}
localIter.Next()
remoteIter.Next()
}
return changes
}
// FetchLocalACLs returns the ACLs in the local state store.
func (s *Server) fetchLocalLegacyACLs() (structs.ACLs, error) {
_, local, err := s.fsm.State().ACLTokenList(nil, false, true, "")
if err != nil {
return nil, err
}
var acls structs.ACLs
for _, token := range local {
if acl, err := token.Convert(); err == nil && acl != nil {
acls = append(acls, acl)
}
}
return acls, nil
}
// FetchRemoteACLs is used to get the remote set of ACLs from the ACL
// datacenter. The lastIndex parameter is a hint about which remote index we
// have replicated to, so this is expected to block until something changes.
func (s *Server) fetchRemoteLegacyACLs(lastRemoteIndex uint64) (*structs.IndexedACLs, error) {
defer metrics.MeasureSince([]string{"leader", "fetchRemoteACLs"}, time.Now())
args := structs.DCSpecificRequest{
Datacenter: s.config.ACLDatacenter,
QueryOptions: structs.QueryOptions{
Token: s.tokens.ACLReplicationToken(),
MinQueryIndex: lastRemoteIndex,
AllowStale: true,
},
}
var remote structs.IndexedACLs
if err := s.RPC("ACL.List", &args, &remote); err != nil {
return nil, err
}
return &remote, nil
}
// UpdateLocalACLs is given a list of changes to apply in order to bring the
// local ACLs in-line with the remote ACLs from the ACL datacenter.
func (s *Server) updateLocalLegacyACLs(changes structs.ACLRequests, ctx context.Context) (bool, error) {
defer metrics.MeasureSince([]string{"leader", "updateLocalACLs"}, time.Now())
minTimePerOp := time.Second / time.Duration(s.config.ACLReplicationApplyLimit)
for _, change := range changes {
// Note that we are using the single ACL interface here and not
// performing all this inside a single transaction. This is OK
// for two reasons. First, there's nothing else other than this
// replication routine that alters the local ACLs, so there's
// nothing to contend with locally. Second, if an apply fails
// in the middle (most likely due to losing leadership), the
// next replication pass will clean up and check everything
// again.
var reply string
start := time.Now()
if err := aclApplyInternal(s, change, &reply); err != nil {
return false, err
}
// Do a smooth rate limit to wait out the min time allowed for
// each op. If this op took longer than the min, then the sleep
// time will be negative and we will just move on.
elapsed := time.Since(start)
select {
case <-ctx.Done():
return true, nil
case <-time.After(minTimePerOp - elapsed):
// do nothing
}
}
return false, nil
}
// replicateACLs is a runs one pass of the algorithm for replicating ACLs from
// a remote ACL datacenter to local state. If there's any error, this will return
// 0 for the lastRemoteIndex, which will cause us to immediately do a full sync
// next time.
func (s *Server) replicateLegacyACLs(lastRemoteIndex uint64, ctx context.Context) (uint64, bool, error) {
remote, err := s.fetchRemoteLegacyACLs(lastRemoteIndex)
if err != nil {
return 0, false, fmt.Errorf("failed to retrieve remote ACLs: %v", err)
}
// Need to check if we should be stopping. This will be common as the fetching process is a blocking
// RPC which could have been hanging around for a long time and during that time leadership could
// have been lost.
select {
case <-ctx.Done():
return 0, true, nil
default:
// do nothing
}
// Measure everything after the remote query, which can block for long
// periods of time. This metric is a good measure of how expensive the
// replication process is.
defer metrics.MeasureSince([]string{"leader", "replicateACLs"}, time.Now())
local, err := s.fetchLocalLegacyACLs()
if err != nil {
return 0, false, fmt.Errorf("failed to retrieve local ACLs: %v", err)
}
// If the remote index ever goes backwards, it's a good indication that
// the remote side was rebuilt and we should do a full sync since we
// can't make any assumptions about what's going on.
if remote.QueryMeta.Index < lastRemoteIndex {
s.logger.Printf("[WARN] consul: Legacy ACL replication remote index moved backwards (%d to %d), forcing a full ACL sync", lastRemoteIndex, remote.QueryMeta.Index)
lastRemoteIndex = 0
}
// Calculate the changes required to bring the state into sync and then
// apply them.
changes := reconcileLegacyACLs(local, remote.ACLs, lastRemoteIndex)
exit, err := s.updateLocalLegacyACLs(changes, ctx)
if exit {
return 0, true, nil
}
if err != nil {
return 0, false, fmt.Errorf("failed to sync ACL changes: %v", err)
}
// Return the index we got back from the remote side, since we've synced
// up with the remote state as of that index.
return remote.QueryMeta.Index, false, nil
}

246
agent/consul/acl_replication_test.go
View File

@ -1,6 +1,8 @@
package consul
import (
"bytes"
"context"
"fmt"
"os"
"reflect"
@ -10,9 +12,11 @@ import (
"testing"
"time"
"github.com/hashicorp/consul/acl"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/consul/testrpc"
"github.com/hashicorp/consul/testutil/retry"
"github.com/stretchr/testify/require"
)
func TestACLReplication_Sorter(t *testing.T) {
@ -216,7 +220,7 @@ func TestACLReplication_reconcileACLs(t *testing.T) {
}
for i, test := range tests {
local, remote := parseACLs(test.local), parseACLs(test.remote)
changes := reconcileACLs(local, remote, test.lastRemoteIndex)
changes := reconcileLegacyACLs(local, remote, test.lastRemoteIndex)
if actual := parseChanges(changes); actual != test.expected {
t.Errorf("test case %d failed: %s", i, actual)
}
@ -228,6 +232,7 @@ func TestACLReplication_updateLocalACLs_RateLimit(t *testing.T) {
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.Datacenter = "dc2"
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLReplicationApplyLimit = 1
})
s1.tokens.UpdateACLReplicationToken("secret")
@ -247,7 +252,7 @@ func TestACLReplication_updateLocalACLs_RateLimit(t *testing.T) {
// Should be throttled to 1 Hz.
start := time.Now()
if err := s1.updateLocalACLs(changes); err != nil {
if _, err := s1.updateLocalLegacyACLs(changes, context.Background()); err != nil {
t.Fatalf("err: %v", err)
}
if dur := time.Since(start); dur < time.Second {
@ -265,7 +270,7 @@ func TestACLReplication_updateLocalACLs_RateLimit(t *testing.T) {
// Should be throttled to 1 Hz.
start = time.Now()
if err := s1.updateLocalACLs(changes); err != nil {
if _, err := s1.updateLocalLegacyACLs(changes, context.Background()); err != nil {
t.Fatalf("err: %v", err)
}
if dur := time.Since(start); dur < 2*time.Second {
@ -278,6 +283,7 @@ func TestACLReplication_IsACLReplicationEnabled(t *testing.T) {
// ACLs not enabled.
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = ""
c.ACLsEnabled = false
})
defer os.RemoveAll(dir1)
defer s1.Shutdown()
@ -289,6 +295,7 @@ func TestACLReplication_IsACLReplicationEnabled(t *testing.T) {
dir2, s2 := testServerWithConfig(t, func(c *Config) {
c.Datacenter = "dc2"
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
})
defer os.RemoveAll(dir2)
defer s2.Shutdown()
@ -303,7 +310,8 @@ func TestACLReplication_IsACLReplicationEnabled(t *testing.T) {
dir3, s3 := testServerWithConfig(t, func(c *Config) {
c.Datacenter = "dc2"
c.ACLDatacenter = "dc1"
c.EnableACLReplication = true
c.ACLsEnabled = true
c.ACLTokenReplication = true
})
defer os.RemoveAll(dir3)
defer s3.Shutdown()
@ -317,7 +325,8 @@ func TestACLReplication_IsACLReplicationEnabled(t *testing.T) {
dir4, s4 := testServerWithConfig(t, func(c *Config) {
c.Datacenter = "dc1"
c.ACLDatacenter = "dc1"
c.EnableACLReplication = true
c.ACLsEnabled = true
c.ACLTokenReplication = true
})
defer os.RemoveAll(dir4)
defer s4.Shutdown()
@ -331,6 +340,7 @@ func TestACLReplication(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
})
defer os.RemoveAll(dir1)
@ -342,12 +352,14 @@ func TestACLReplication(t *testing.T) {
dir2, s2 := testServerWithConfig(t, func(c *Config) {
c.Datacenter = "dc2"
c.ACLDatacenter = "dc1"
c.EnableACLReplication = true
c.ACLReplicationInterval = 10 * time.Millisecond
c.ACLsEnabled = true
c.ACLTokenReplication = true
c.ACLReplicationRate = 100
c.ACLReplicationBurst = 100
c.ACLReplicationApplyLimit = 1000000
})
testrpc.WaitForLeader(t, s2.RPC, "dc2")
s2.tokens.UpdateACLReplicationToken("root")
testrpc.WaitForLeader(t, s2.RPC, "dc2")
defer os.RemoveAll(dir2)
defer s2.Shutdown()
@ -364,7 +376,7 @@ func TestACLReplication(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: testACLPolicy,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -375,19 +387,19 @@ func TestACLReplication(t *testing.T) {
}
checkSame := func() error {
index, remote, err := s1.fsm.State().ACLList(nil)
index, remote, err := s1.fsm.State().ACLTokenList(nil, true, true, "")
if err != nil {
return err
}
_, local, err := s2.fsm.State().ACLList(nil)
_, local, err := s2.fsm.State().ACLTokenList(nil, true, true, "")
if err != nil {
return err
}
if got, want := len(remote), len(local); got != want {
return fmt.Errorf("got %d remote ACLs want %d", got, want)
}
for i, acl := range remote {
if !acl.IsSame(local[i]) {
for i, token := range remote {
if !bytes.Equal(token.Hash, local[i].Hash) {
return fmt.Errorf("ACLs differ")
}
}
@ -397,7 +409,7 @@ func TestACLReplication(t *testing.T) {
status = s2.aclReplicationStatus
s2.aclReplicationStatusLock.RUnlock()
if !status.Enabled || !status.Running ||
status.ReplicatedIndex != index ||
status.ReplicatedTokenIndex != index ||
status.SourceDatacenter != "dc1" {
return fmt.Errorf("ACL replication status differs")
}
@ -418,7 +430,7 @@ func TestACLReplication(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: testACLPolicy,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -455,3 +467,207 @@ func TestACLReplication(t *testing.T) {
}
})
}
func TestACLReplication_diffACLPolicies(t *testing.T) {
local := structs.ACLPolicies{
&structs.ACLPolicy{
ID: "44ef9aec-7654-4401-901b-4d4a8b3c80fc",
Name: "policy1",
Description: "policy1 - already in sync",
Rules: `acl = "read"`,
Syntax: acl.SyntaxCurrent,
Datacenters: nil,
Hash: []byte{1, 2, 3, 4},
RaftIndex: structs.RaftIndex{CreateIndex: 1, ModifyIndex: 2},
},
&structs.ACLPolicy{
ID: "8ea41efb-8519-4091-bc91-c42da0cda9ae",
Name: "policy2",
Description: "policy2 - updated but not changed",
Rules: `acl = "read"`,
Syntax: acl.SyntaxCurrent,
Datacenters: nil,
Hash: []byte{1, 2, 3, 4},
RaftIndex: structs.RaftIndex{CreateIndex: 1, ModifyIndex: 25},
},
&structs.ACLPolicy{
ID: "539f1cb6-40aa-464f-ae66-a900d26bc1b2",
Name: "policy3",
Description: "policy3 - updated and changed",
Rules: `acl = "read"`,
Syntax: acl.SyntaxCurrent,
Datacenters: nil,
Hash: []byte{1, 2, 3, 4},
RaftIndex: structs.RaftIndex{CreateIndex: 1, ModifyIndex: 25},
},
&structs.ACLPolicy{
ID: "e9d33298-6490-4466-99cb-ba93af64fa76",
Name: "policy4",
Description: "policy4 - needs deleting",
Rules: `acl = "read"`,
Syntax: acl.SyntaxCurrent,
Datacenters: nil,
Hash: []byte{1, 2, 3, 4},
RaftIndex: structs.RaftIndex{CreateIndex: 1, ModifyIndex: 25},
},
}
remote := structs.ACLPolicyListStubs{
&structs.ACLPolicyListStub{
ID: "44ef9aec-7654-4401-901b-4d4a8b3c80fc",
Name: "policy1",
Description: "policy1 - already in sync",
Datacenters: nil,
Hash: []byte{1, 2, 3, 4},
CreateIndex: 1,
ModifyIndex: 2,
},
&structs.ACLPolicyListStub{
ID: "8ea41efb-8519-4091-bc91-c42da0cda9ae",
Name: "policy2",
Description: "policy2 - updated but not changed",
Datacenters: nil,
Hash: []byte{1, 2, 3, 4},
CreateIndex: 1,
ModifyIndex: 50,
},
&structs.ACLPolicyListStub{
ID: "539f1cb6-40aa-464f-ae66-a900d26bc1b2",
Name: "policy3",
Description: "policy3 - updated and changed",
Datacenters: nil,
Hash: []byte{5, 6, 7, 8},
CreateIndex: 1,
ModifyIndex: 50,
},
&structs.ACLPolicyListStub{
ID: "c6e8fffd-cbd9-4ecd-99fe-ab2f200c7926",
Name: "policy5",
Description: "policy5 - needs adding",
Datacenters: nil,
Hash: []byte{1, 2, 3, 4},
CreateIndex: 1,
ModifyIndex: 50,
},
}
// Do the full diff. This full exercises the main body of the loop
deletions, updates := diffACLPolicies(local, remote, 28)
require.Len(t, updates, 2)
require.ElementsMatch(t, updates, []string{
"c6e8fffd-cbd9-4ecd-99fe-ab2f200c7926",
"539f1cb6-40aa-464f-ae66-a900d26bc1b2"})
require.Len(t, deletions, 1)
require.Equal(t, "e9d33298-6490-4466-99cb-ba93af64fa76", deletions[0])
deletions, updates = diffACLPolicies(local, nil, 28)
require.Len(t, updates, 0)
require.Len(t, deletions, 4)
require.ElementsMatch(t, deletions, []string{
"44ef9aec-7654-4401-901b-4d4a8b3c80fc",
"8ea41efb-8519-4091-bc91-c42da0cda9ae",
"539f1cb6-40aa-464f-ae66-a900d26bc1b2",
"e9d33298-6490-4466-99cb-ba93af64fa76"})
deletions, updates = diffACLPolicies(nil, remote, 28)
require.Len(t, deletions, 0)
require.Len(t, updates, 4)
require.ElementsMatch(t, updates, []string{
"44ef9aec-7654-4401-901b-4d4a8b3c80fc",
"8ea41efb-8519-4091-bc91-c42da0cda9ae",
"539f1cb6-40aa-464f-ae66-a900d26bc1b2",
"c6e8fffd-cbd9-4ecd-99fe-ab2f200c7926"})
}
func TestACLReplication_diffACLTokens(t *testing.T) {
local := structs.ACLTokens{
&structs.ACLToken{
AccessorID: "44ef9aec-7654-4401-901b-4d4a8b3c80fc",
SecretID: "44ef9aec-7654-4401-901b-4d4a8b3c80fc",
Description: "token1 - already in sync",
Hash: []byte{1, 2, 3, 4},
RaftIndex: structs.RaftIndex{CreateIndex: 1, ModifyIndex: 2},
},
&structs.ACLToken{
AccessorID: "8ea41efb-8519-4091-bc91-c42da0cda9ae",
SecretID: "8ea41efb-8519-4091-bc91-c42da0cda9ae",
Description: "token2 - updated but not changed",
Hash: []byte{1, 2, 3, 4},
RaftIndex: structs.RaftIndex{CreateIndex: 1, ModifyIndex: 25},
},
&structs.ACLToken{
AccessorID: "539f1cb6-40aa-464f-ae66-a900d26bc1b2",
SecretID: "539f1cb6-40aa-464f-ae66-a900d26bc1b2",
Description: "token3 - updated and changed",
Hash: []byte{1, 2, 3, 4},
RaftIndex: structs.RaftIndex{CreateIndex: 1, ModifyIndex: 25},
},
&structs.ACLToken{
AccessorID: "e9d33298-6490-4466-99cb-ba93af64fa76",
SecretID: "e9d33298-6490-4466-99cb-ba93af64fa76",
Description: "token4 - needs deleting",
Hash: []byte{1, 2, 3, 4},
RaftIndex: structs.RaftIndex{CreateIndex: 1, ModifyIndex: 25},
},
}
remote := structs.ACLTokenListStubs{
&structs.ACLTokenListStub{
AccessorID: "44ef9aec-7654-4401-901b-4d4a8b3c80fc",
Description: "token1 - already in sync",
Hash: []byte{1, 2, 3, 4},
CreateIndex: 1,
ModifyIndex: 2,
},
&structs.ACLTokenListStub{
AccessorID: "8ea41efb-8519-4091-bc91-c42da0cda9ae",
Description: "token2 - updated but not changed",
Hash: []byte{1, 2, 3, 4},
CreateIndex: 1,
ModifyIndex: 50,
},
&structs.ACLTokenListStub{
AccessorID: "539f1cb6-40aa-464f-ae66-a900d26bc1b2",
Description: "token3 - updated and changed",
Hash: []byte{5, 6, 7, 8},
CreateIndex: 1,
ModifyIndex: 50,
},
&structs.ACLTokenListStub{
AccessorID: "c6e8fffd-cbd9-4ecd-99fe-ab2f200c7926",
Description: "token5 - needs adding",
Hash: []byte{1, 2, 3, 4},
CreateIndex: 1,
ModifyIndex: 50,
},
}
// Do the full diff. This full exercises the main body of the loop
deletions, updates := diffACLTokens(local, remote, 28)
require.Len(t, updates, 2)
require.ElementsMatch(t, updates, []string{
"c6e8fffd-cbd9-4ecd-99fe-ab2f200c7926",
"539f1cb6-40aa-464f-ae66-a900d26bc1b2"})
require.Len(t, deletions, 1)
require.Equal(t, "e9d33298-6490-4466-99cb-ba93af64fa76", deletions[0])
deletions, updates = diffACLTokens(local, nil, 28)
require.Len(t, updates, 0)
require.Len(t, deletions, 4)
require.ElementsMatch(t, deletions, []string{
"44ef9aec-7654-4401-901b-4d4a8b3c80fc",
"8ea41efb-8519-4091-bc91-c42da0cda9ae",
"539f1cb6-40aa-464f-ae66-a900d26bc1b2",
"e9d33298-6490-4466-99cb-ba93af64fa76"})
deletions, updates = diffACLTokens(nil, remote, 28)
require.Len(t, deletions, 0)
require.Len(t, updates, 4)
require.ElementsMatch(t, updates, []string{
"44ef9aec-7654-4401-901b-4d4a8b3c80fc",
"8ea41efb-8519-4091-bc91-c42da0cda9ae",
"539f1cb6-40aa-464f-ae66-a900d26bc1b2",
"c6e8fffd-cbd9-4ecd-99fe-ab2f200c7926"})
}

179
agent/consul/acl_server.go Normal file
View File

@ -0,0 +1,179 @@
package consul
import (
"sync/atomic"
"github.com/hashicorp/consul/acl"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/consul/lib"
)
var serverACLCacheConfig *structs.ACLCachesConfig = &structs.ACLCachesConfig{
// The servers ACL caching has a few underlying assumptions:
//
// 1 - All policies can be resolved locally. Hence we do not cache any
// unparsed policies as we have memdb for that.
// 2 - While there could be many identities being used within a DC the
// number of distinct policies and combined multi-policy authorizers
// will be much less.
// 3 - If you need more than 10k tokens cached then you should probably
// enabled token replication or be using DC local tokens. In both
// cases resolving the tokens from memdb will avoid the cache
// entirely
//
Identities: 10 * 1024,
Policies: 0,
ParsedPolicies: 512,
Authorizers: 1024,
}
func (s *Server) checkTokenUUID(id string) (bool, error) {
state := s.fsm.State()
if _, token, err := state.ACLTokenGetByAccessor(nil, id); err != nil {
return false, err
} else if token != nil {
return false, nil
}
if _, token, err := state.ACLTokenGetBySecret(nil, id); err != nil {
return false, err
} else if token != nil {
return false, nil
}
return !structs.ACLIDReserved(id), nil
}
func (s *Server) checkPolicyUUID(id string) (bool, error) {
state := s.fsm.State()
if _, policy, err := state.ACLPolicyGetByID(nil, id); err != nil {
return false, err
} else if policy != nil {
return false, nil
}
return !structs.ACLIDReserved(id), nil
}
func (s *Server) updateACLAdvertisement() {
// One thing to note is that once in new ACL mode the server will
// never transition to legacy ACL mode. This is not currently a
// supported use case.
// always advertise to all the LAN Members
lib.UpdateSerfTag(s.serfLAN, "acls", string(structs.ACLModeEnabled))
if s.serfWAN != nil {
// advertise on the WAN only when we are inside the ACL datacenter
lib.UpdateSerfTag(s.serfWAN, "acls", string(structs.ACLModeEnabled))
}
}
func (s *Server) canUpgradeToNewACLs(isLeader bool) bool {
if atomic.LoadInt32(&s.useNewACLs) != 0 {
// can't upgrade because we are already upgraded
return false
}
if !s.InACLDatacenter() {
mode, _ := ServersGetACLMode(s.WANMembers(), "", s.config.ACLDatacenter)
if mode != structs.ACLModeEnabled {
return false
}
}
if isLeader {
if mode, _ := ServersGetACLMode(s.LANMembers(), "", ""); mode == structs.ACLModeLegacy {
return true
}
} else {
leader := string(s.raft.Leader())
if _, leaderMode := ServersGetACLMode(s.LANMembers(), leader, ""); leaderMode == structs.ACLModeEnabled {
return true
}
}
return false
}
func (s *Server) InACLDatacenter() bool {
return s.config.Datacenter == s.config.ACLDatacenter
}
func (s *Server) UseLegacyACLs() bool {
return atomic.LoadInt32(&s.useNewACLs) == 0
}
func (s *Server) LocalTokensEnabled() bool {
// in ACL datacenter so local tokens are always enabled
if s.InACLDatacenter() {
return true
}
if !s.config.ACLTokenReplication || s.tokens.ACLReplicationToken() == "" {
return false
}
// token replication is off so local tokens are disabled
return true
}
func (s *Server) ACLDatacenter(legacy bool) string {
// For resolution running on servers the only option
// is to contact the configured ACL Datacenter
if s.config.ACLDatacenter != "" {
return s.config.ACLDatacenter
}
// This function only gets called if ACLs are enabled.
// When no ACL DC is set then it is assumed that this DC
// is the primary DC
return s.config.Datacenter
}
func (s *Server) ACLsEnabled() bool {
return s.config.ACLsEnabled
}
func (s *Server) ResolveIdentityFromToken(token string) (bool, structs.ACLIdentity, error) {
// only allow remote RPC resolution when token replication is off and
// when not in the ACL datacenter
if !s.InACLDatacenter() && !s.config.ACLTokenReplication {
return false, nil, nil
}
index, aclToken, err := s.fsm.State().ACLTokenGetBySecret(nil, token)
if err != nil {
return true, nil, err
} else if aclToken != nil {
return true, aclToken, nil
}
return s.InACLDatacenter() || index > 0, nil, acl.ErrNotFound
}
func (s *Server) ResolvePolicyFromID(policyID string) (bool, *structs.ACLPolicy, error) {
index, policy, err := s.fsm.State().ACLPolicyGetByID(nil, policyID)
if err != nil {
return true, nil, err
} else if policy != nil {
return true, policy, nil
}
// If the max index of the policies table is non-zero then we have acls, until then
// we may need to allow remote resolution. This is particularly useful to allow updating
// the replication token via the API in a non-primary dc.
return s.InACLDatacenter() || index > 0, policy, acl.ErrNotFound
}
func (s *Server) ResolveToken(token string) (acl.Authorizer, error) {
return s.acls.ResolveToken(token)
}
func (s *Server) filterACL(token string, subj interface{}) error {
return s.acls.filterACL(token, subj)
}
func (s *Server) filterACLWithAuthorizer(authorizer acl.Authorizer, subj interface{}) error {
return s.acls.filterACLWithAuthorizer(authorizer, subj)
}

1767
agent/consul/acl_test.go
View File

File diff suppressed because it is too large Load Diff

11
agent/consul/catalog_endpoint.go
View File

@ -2,6 +2,7 @@ package consul
import (
"fmt"
"sort"
"time"
"github.com/armon/go-metrics"
@ -40,7 +41,7 @@ func (c *Catalog) Register(args *structs.RegisterRequest, reply *struct{}) error
}
// Fetch the ACL token, if any.
rule, err := c.srv.resolveToken(args.Token)
rule, err := c.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -138,7 +139,7 @@ func (c *Catalog) Deregister(args *structs.DeregisterRequest, reply *struct{}) e
}
// Fetch the ACL token, if any.
rule, err := c.srv.resolveToken(args.Token)
rule, err := c.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -284,7 +285,7 @@ func (c *Catalog) ServiceNodes(args *structs.ServiceSpecificRequest, reply *stru
// we're trying to find proxies for, so check that.
if args.Connect {
// Fetch the ACL token, if any.
rule, err := c.srv.resolveToken(args.Token)
rule, err := c.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -335,6 +336,10 @@ func (c *Catalog) ServiceNodes(args *structs.ServiceSpecificRequest, reply *stru
[]metrics.Label{{Name: "service", Value: args.ServiceName}, {Name: "tag", Value: args.ServiceTag}})
}
if len(args.ServiceTags) > 0 {
// Sort tags so that the metric is the same even if the request
// tags are in a different order
sort.Strings(args.ServiceTags)
// Build metric labels
labels := []metrics.Label{{Name: "service", Value: args.ServiceName}}
for _, tag := range args.ServiceTags {

25
agent/consul/catalog_endpoint_test.go
View File

@ -159,6 +159,7 @@ func TestCatalog_Register_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = false
@ -175,7 +176,7 @@ func TestCatalog_Register_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: `
service "foo" {
policy = "write"
@ -424,6 +425,7 @@ func TestCatalog_Register_ConnectProxy_ACLProxyDestination(t *testing.T) {
assert := assert.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -440,7 +442,7 @@ func TestCatalog_Register_ConnectProxy_ACLProxyDestination(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: `
service "foo" {
policy = "write"
@ -537,6 +539,7 @@ func TestCatalog_Deregister_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = false
@ -554,7 +557,7 @@ func TestCatalog_Deregister_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: `
node "node" {
policy = "write"
@ -1184,6 +1187,7 @@ func TestCatalog_ListNodes_ACLFilter(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = false
@ -1231,7 +1235,7 @@ func TestCatalog_ListNodes_ACLFilter(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: fmt.Sprintf(`
node "%s" {
policy = "read"
@ -1501,6 +1505,7 @@ func TestCatalog_ListServices_Stale(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
})
defer os.RemoveAll(dir1)
defer s1.Shutdown()
@ -1508,7 +1513,8 @@ func TestCatalog_ListServices_Stale(t *testing.T) {
testrpc.WaitForTestAgent(t, s1.RPC, "dc1")
dir2, s2 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1" // Enable ACLs!
c.Bootstrap = false // Disable bootstrap
c.ACLsEnabled = true
c.Bootstrap = false // Disable bootstrap
})
defer os.RemoveAll(dir2)
defer s2.Shutdown()
@ -1958,6 +1964,7 @@ func TestCatalog_ListServiceNodes_ConnectProxy_ACL(t *testing.T) {
assert := assert.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -1974,7 +1981,7 @@ func TestCatalog_ListServiceNodes_ConnectProxy_ACL(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: `
service "foo" {
policy = "write"
@ -2229,6 +2236,7 @@ func TestCatalog_Register_FailedCase1(t *testing.T) {
func testACLFilterServer(t *testing.T) (dir, token string, srv *Server, codec rpc.ClientCodec) {
dir, srv = testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = false
@ -2243,7 +2251,7 @@ func testACLFilterServer(t *testing.T) (dir, token string, srv *Server, codec rp
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: `
service "foo" {
policy = "write"
@ -2376,6 +2384,7 @@ func TestCatalog_NodeServices_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = false
@ -2415,7 +2424,7 @@ func TestCatalog_NodeServices_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: fmt.Sprintf(`
node "%s" {
policy = "read"

25
agent/consul/client.go
View File

@ -48,6 +48,13 @@ const (
type Client struct {
config *Config
// acls is used to resolve tokens to effective policies
acls *ACLResolver
// DEPRECATED (ACL-Legacy-Compat) - Only needed while we support both
// useNewACLs is a flag to indicate whether we are using the new ACL system
useNewACLs int32
// Connection pool to consul servers
connPool *pool.ConnPool
@ -141,6 +148,20 @@ func NewClientLogger(config *Config, logger *log.Logger) (*Client, error) {
return nil, err
}
c.useNewACLs = 0
aclConfig := ACLResolverConfig{
Config: config,
Delegate: c,
Logger: logger,
AutoDisable: true,
CacheConfig: clientACLCacheConfig,
Sentinel: nil,
}
if c.acls, err = NewACLResolver(&aclConfig); err != nil {
c.Shutdown()
return nil, fmt.Errorf("Failed to create ACL resolver: %v", err)
}
// Initialize the LAN Serf
c.serf, err = c.setupSerf(config.SerfLANConfig,
c.eventCh, serfLANSnapshot)
@ -149,6 +170,10 @@ func NewClientLogger(config *Config, logger *log.Logger) (*Client, error) {
return nil, fmt.Errorf("Failed to start lan serf: %v", err)
}
if c.acls.ACLsEnabled() {
go c.monitorACLMode()
}
// Start maintenance task for servers
c.routers = router.New(c.logger, c.shutdownCh, c.serf, c.connPool)
go c.routers.Start()

9
agent/consul/client_serf.go
View File

@ -6,6 +6,7 @@ import (
"strings"
"github.com/hashicorp/consul/agent/metadata"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/consul/lib"
"github.com/hashicorp/serf/serf"
)
@ -23,6 +24,14 @@ func (c *Client) setupSerf(conf *serf.Config, ch chan serf.Event, path string) (
conf.Tags["vsn_min"] = fmt.Sprintf("%d", ProtocolVersionMin)
conf.Tags["vsn_max"] = fmt.Sprintf("%d", ProtocolVersionMax)
conf.Tags["build"] = c.config.Build
if c.acls.ACLsEnabled() {
// we start in legacy mode and then transition to normal
// mode once we know the cluster can handle it.
conf.Tags["acls"] = string(structs.ACLModeLegacy)
} else {
conf.Tags["acls"] = string(structs.ACLModeDisabled)
}
if c.logger == nil {
conf.MemberlistConfig.LogOutput = c.config.LogOutput
conf.LogOutput = c.config.LogOutput

52
agent/consul/config.go
View File

@ -217,6 +217,13 @@ type Config struct {
// operators track which versions are actively deployed
Build string
// ACLEnabled is used to enable ACLs
ACLsEnabled bool
// ACLEnforceVersion8 is used to gate a set of ACL policy features that
// are opt-in prior to Consul 0.8 and opt-out in Consul 0.8 and later.
ACLEnforceVersion8 bool
// ACLMasterToken is used to bootstrap the ACL system. It should be specified
// on the servers in the ACLDatacenter. When the leader comes online, it ensures
// that the Master token is available. This provides the initial token.
@ -226,10 +233,26 @@ type Config struct {
// tokens. If not provided, ACL verification is disabled.
ACLDatacenter string
// ACLTTL controls the time-to-live of cached ACL policies.
// ACLTokenTTL controls the time-to-live of cached ACL tokens.
// It can be set to zero to disable caching, but this adds
// a substantial cost.
ACLTTL time.Duration
ACLTokenTTL time.Duration
// ACLPolicyTTL controls the time-to-live of cached ACL policies.
// It can be set to zero to disable caching, but this adds
// a substantial cost.
ACLPolicyTTL time.Duration
// ACLDisabledTTL is the time between checking if ACLs should be
// enabled. This
ACLDisabledTTL time.Duration
// ACLTokenReplication is used to enabled token replication.
//
// By default policy-only replication is enabled. When token
// replication is off and the primary datacenter is not
// yet upgraded to the new ACLs no replication will be performed
ACLTokenReplication bool
// ACLDefaultPolicy is used to control the ACL interaction when
// there is no defined policy. This can be "allow" which means
@ -245,25 +268,20 @@ type Config struct {
// "allow" can be used to allow all requests. This is not recommended.
ACLDownPolicy string
// EnableACLReplication is used to control ACL replication.
EnableACLReplication bool
// ACLReplicationRate is the max number of replication rounds that can
// be run per second. Note that either 1 or 2 RPCs are used during each replication
// round
ACLReplicationRate int
// ACLReplicationInterval is the interval at which replication passes
// will occur. Queries to the ACLDatacenter may block, so replication
// can happen less often than this, but the interval forms the upper
// limit to how fast we will go if there was constant ACL churn on the
// remote end.
ACLReplicationInterval time.Duration
// ACLReplicationBurst is how many replication RPCs can be bursted after a
// period of idleness
ACLReplicationBurst int
// ACLReplicationApplyLimit is the max number of replication-related
// apply operations that we allow during a one second period. This is
// used to limit the amount of Raft bandwidth used for replication.
ACLReplicationApplyLimit int
// ACLEnforceVersion8 is used to gate a set of ACL policy features that
// are opt-in prior to Consul 0.8 and opt-out in Consul 0.8 and later.
ACLEnforceVersion8 bool
// ACLEnableKeyListPolicy is used to gate enforcement of the new "list" policy that
// protects listing keys by prefix. This behavior is opt-in
// by default in Consul 1.0 and later.
@ -411,10 +429,12 @@ func DefaultConfig() *Config {
SerfFloodInterval: 60 * time.Second,
ReconcileInterval: 60 * time.Second,
ProtocolVersion: ProtocolVersion2Compatible,
ACLTTL: 30 * time.Second,
ACLPolicyTTL: 30 * time.Second,
ACLTokenTTL: 30 * time.Second,
ACLDefaultPolicy: "allow",
ACLDownPolicy: "extend-cache",
ACLReplicationInterval: 30 * time.Second,
ACLReplicationRate: 1,
ACLReplicationBurst: 5,
ACLReplicationApplyLimit: 100, // ops / sec
TombstoneTTL: 15 * time.Minute,
TombstoneTTLGranularity: 30 * time.Second,

6
agent/consul/connect_ca_endpoint.go
View File

@ -36,7 +36,7 @@ func (s *ConnectCA) ConfigurationGet(
}
// This action requires operator read access.
rule, err := s.srv.resolveToken(args.Token)
rule, err := s.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -68,7 +68,7 @@ func (s *ConnectCA) ConfigurationSet(
}
// This action requires operator write access.
rule, err := s.srv.resolveToken(args.Token)
rule, err := s.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -349,7 +349,7 @@ func (s *ConnectCA) Sign(
}
// Verify that the ACL token provided has permission to act as this service
rule, err := s.srv.resolveToken(args.Token)
rule, err := s.srv.ResolveToken(args.Token)
if err != nil {
return err
}

3
agent/consul/connect_ca_endpoint_test.go
View File

@ -341,6 +341,7 @@ func TestConnectCASignValidation(t *testing.T) {
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -359,7 +360,7 @@ func TestConnectCASignValidation(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: `
service "web" {
policy = "write"

4
agent/consul/coordinate_endpoint.go
View File

@ -134,7 +134,7 @@ func (c *Coordinate) Update(args *structs.CoordinateUpdateRequest, reply *struct
}
// Fetch the ACL token, if any, and enforce the node policy if enabled.
rule, err := c.srv.resolveToken(args.Token)
rule, err := c.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -205,7 +205,7 @@ func (c *Coordinate) Node(args *structs.NodeSpecificRequest, reply *structs.Inde
}
// Fetch the ACL token, if any, and enforce the node policy if enabled.
rule, err := c.srv.resolveToken(args.Token)
rule, err := c.srv.ResolveToken(args.Token)
if err != nil {
return err
}

9
agent/consul/coordinate_endpoint_test.go
View File

@ -181,6 +181,7 @@ func TestCoordinate_Update_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = false
@ -223,7 +224,7 @@ func TestCoordinate_Update_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: `
node "node1" {
policy = "write"
@ -351,6 +352,7 @@ func TestCoordinate_ListNodes_ACLFilter(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = false
@ -456,7 +458,7 @@ func TestCoordinate_ListNodes_ACLFilter(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: `
node "foo" {
policy = "read"
@ -538,6 +540,7 @@ func TestCoordinate_Node_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = false
@ -597,7 +600,7 @@ func TestCoordinate_Node_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: `
node "node1" {
policy = "read"

30
agent/consul/filter.go
View File

@ -6,15 +6,15 @@ import (
)
type dirEntFilter struct {
acl acl.ACL
ent structs.DirEntries
authorizer acl.Authorizer
ent structs.DirEntries
}
func (d *dirEntFilter) Len() int {
return len(d.ent)
}
func (d *dirEntFilter) Filter(i int) bool {
return !d.acl.KeyRead(d.ent[i].Key)
return !d.authorizer.KeyRead(d.ent[i].Key)
}
func (d *dirEntFilter) Move(dst, src, span int) {
copy(d.ent[dst:dst+span], d.ent[src:src+span])
@ -22,21 +22,21 @@ func (d *dirEntFilter) Move(dst, src, span int) {
// FilterDirEnt is used to filter a list of directory entries
// by applying an ACL policy
func FilterDirEnt(acl acl.ACL, ent structs.DirEntries) structs.DirEntries {
df := dirEntFilter{acl: acl, ent: ent}
func FilterDirEnt(authorizer acl.Authorizer, ent structs.DirEntries) structs.DirEntries {
df := dirEntFilter{authorizer: authorizer, ent: ent}
return ent[:FilterEntries(&df)]
}
type keyFilter struct {
acl acl.ACL
keys []string
authorizer acl.Authorizer
keys []string
}
func (k *keyFilter) Len() int {
return len(k.keys)
}
func (k *keyFilter) Filter(i int) bool {
return !k.acl.KeyRead(k.keys[i])
return !k.authorizer.KeyRead(k.keys[i])
}
func (k *keyFilter) Move(dst, src, span int) {
@ -45,14 +45,14 @@ func (k *keyFilter) Move(dst, src, span int) {
// FilterKeys is used to filter a list of keys by
// applying an ACL policy
func FilterKeys(acl acl.ACL, keys []string) []string {
kf := keyFilter{acl: acl, keys: keys}
func FilterKeys(authorizer acl.Authorizer, keys []string) []string {
kf := keyFilter{authorizer: authorizer, keys: keys}
return keys[:FilterEntries(&kf)]
}
type txnResultsFilter struct {
acl acl.ACL
results structs.TxnResults
authorizer acl.Authorizer
results structs.TxnResults
}
func (t *txnResultsFilter) Len() int {
@ -62,7 +62,7 @@ func (t *txnResultsFilter) Len() int {
func (t *txnResultsFilter) Filter(i int) bool {
result := t.results[i]
if result.KV != nil {
return !t.acl.KeyRead(result.KV.Key)
return !t.authorizer.KeyRead(result.KV.Key)
}
return false
}
@ -73,8 +73,8 @@ func (t *txnResultsFilter) Move(dst, src, span int) {
// FilterTxnResults is used to filter a list of transaction results by
// applying an ACL policy.
func FilterTxnResults(acl acl.ACL, results structs.TxnResults) structs.TxnResults {
rf := txnResultsFilter{acl: acl, results: results}
func FilterTxnResults(authorizer acl.Authorizer, results structs.TxnResults) structs.TxnResults {
rf := txnResultsFilter{authorizer: authorizer, results: results}
return results[:FilterEntries(&rf)]
}

12
agent/consul/filter_test.go
View File

@ -10,8 +10,8 @@ import (
func TestFilter_DirEnt(t *testing.T) {
t.Parallel()
policy, _ := acl.Parse(testFilterRules, nil)
aclR, _ := acl.New(acl.DenyAll(), policy, nil)
policy, _ := acl.NewPolicyFromSource("", 0, testFilterRules, acl.SyntaxLegacy, nil)
aclR, _ := acl.NewPolicyAuthorizer(acl.DenyAll(), []*acl.Policy{policy}, nil)
type tcase struct {
in []string
@ -52,8 +52,8 @@ func TestFilter_DirEnt(t *testing.T) {
func TestFilter_Keys(t *testing.T) {
t.Parallel()
policy, _ := acl.Parse(testFilterRules, nil)
aclR, _ := acl.New(acl.DenyAll(), policy, nil)
policy, _ := acl.NewPolicyFromSource("", 0, testFilterRules, acl.SyntaxLegacy, nil)
aclR, _ := acl.NewPolicyAuthorizer(acl.DenyAll(), []*acl.Policy{policy}, nil)
type tcase struct {
in []string
@ -84,8 +84,8 @@ func TestFilter_Keys(t *testing.T) {
func TestFilter_TxnResults(t *testing.T) {
t.Parallel()
policy, _ := acl.Parse(testFilterRules, nil)
aclR, _ := acl.New(acl.DenyAll(), policy, nil)
policy, _ := acl.NewPolicyFromSource("", 0, testFilterRules, acl.SyntaxLegacy, nil)
aclR, _ := acl.NewPolicyAuthorizer(acl.DenyAll(), []*acl.Policy{policy}, nil)
type tcase struct {
in []string

84
agent/consul/fsm/commands_oss.go
View File

@ -14,6 +14,7 @@ func init() {
registerCommand(structs.DeregisterRequestType, (*FSM).applyDeregister)
registerCommand(structs.KVSRequestType, (*FSM).applyKVSOperation)
registerCommand(structs.SessionRequestType, (*FSM).applySessionOperation)
// DEPRECATED (ACL-Legacy-Compat) - Only needed for v1 ACL compat
registerCommand(structs.ACLRequestType, (*FSM).applyACLOperation)
registerCommand(structs.TombstoneRequestType, (*FSM).applyTombstoneOperation)
registerCommand(structs.CoordinateBatchUpdateType, (*FSM).applyCoordinateBatchUpdate)
@ -22,6 +23,11 @@ func init() {
registerCommand(structs.AutopilotRequestType, (*FSM).applyAutopilotUpdate)
registerCommand(structs.IntentionRequestType, (*FSM).applyIntentionOperation)
registerCommand(structs.ConnectCARequestType, (*FSM).applyConnectCAOperation)
registerCommand(structs.ACLTokenUpsertRequestType, (*FSM).applyACLTokenUpsertOperation)
registerCommand(structs.ACLTokenDeleteRequestType, (*FSM).applyACLTokenDeleteOperation)
registerCommand(structs.ACLBootstrapRequestType, (*FSM).applyACLTokenBootstrap)
registerCommand(structs.ACLPolicyUpsertRequestType, (*FSM).applyACLPolicyUpsertOperation)
registerCommand(structs.ACLPolicyDeleteRequestType, (*FSM).applyACLPolicyDeleteOperation)
}
func (c *FSM) applyRegister(buf []byte, index uint64) interface{} {
@ -134,7 +140,10 @@ func (c *FSM) applySessionOperation(buf []byte, index uint64) interface{} {
}
}
// DEPRECATED (ACL-Legacy-Compat) - Only needed for legacy compat
func (c *FSM) applyACLOperation(buf []byte, index uint64) interface{} {
// TODO (ACL-Legacy-Compat) - Should we warn here somehow about using deprecated features
// maybe emit a second metric?
var req structs.ACLRequest
if err := structs.Decode(buf, &req); err != nil {
panic(fmt.Errorf("failed to decode request: %v", err))
@ -143,23 +152,34 @@ func (c *FSM) applyACLOperation(buf []byte, index uint64) interface{} {
[]metrics.Label{{Name: "op", Value: string(req.Op)}})
switch req.Op {
case structs.ACLBootstrapInit:
enabled, err := c.state.ACLBootstrapInit(index)
enabled, _, err := c.state.CanBootstrapACLToken()
if err != nil {
return err
}
return enabled
case structs.ACLBootstrapNow:
if err := c.state.ACLBootstrap(index, &req.ACL); err != nil {
// This a bootstrap request from a non-upgraded node
if err := c.state.ACLBootstrap(index, 0, req.ACL.Convert(), true); err != nil {
return err
}
return &req.ACL
if _, token, err := c.state.ACLTokenGetBySecret(nil, req.ACL.ID); err != nil {
return err
} else {
acl, err := token.Convert()
if err != nil {
return err
}
return acl
}
case structs.ACLForceSet, structs.ACLSet:
if err := c.state.ACLSet(index, &req.ACL); err != nil {
if err := c.state.ACLTokenSet(index, req.ACL.Convert(), true); err != nil {
return err
}
return req.ACL.ID
case structs.ACLDelete:
return c.state.ACLDelete(index, req.ACL.ID)
return c.state.ACLTokenDeleteSecret(index, req.ACL.ID)
default:
c.logger.Printf("[WARN] consul.fsm: Invalid ACL operation '%s'", req.Op)
return fmt.Errorf("Invalid ACL operation '%s'", req.Op)
@ -330,3 +350,57 @@ func (c *FSM) applyConnectCAOperation(buf []byte, index uint64) interface{} {
return fmt.Errorf("Invalid CA operation '%s'", req.Op)
}
}
func (c *FSM) applyACLTokenUpsertOperation(buf []byte, index uint64) interface{} {
var req structs.ACLTokenBatchUpsertRequest
if err := structs.Decode(buf, &req); err != nil {
panic(fmt.Errorf("failed to decode request: %v", err))
}
defer metrics.MeasureSinceWithLabels([]string{"fsm", "acl", "token"}, time.Now(),
[]metrics.Label{{Name: "op", Value: "upsert"}})
return c.state.ACLTokensUpsert(index, req.Tokens, req.AllowCreate)
}
func (c *FSM) applyACLTokenDeleteOperation(buf []byte, index uint64) interface{} {
var req structs.ACLTokenBatchDeleteRequest
if err := structs.Decode(buf, &req); err != nil {
panic(fmt.Errorf("failed to decode request: %v", err))
}
defer metrics.MeasureSinceWithLabels([]string{"fsm", "acl", "token"}, time.Now(),
[]metrics.Label{{Name: "op", Value: "delete"}})
return c.state.ACLTokensDelete(index, req.TokenIDs)
}
func (c *FSM) applyACLTokenBootstrap(buf []byte, index uint64) interface{} {
var req structs.ACLTokenBootstrapRequest
if err := structs.Decode(buf, &req); err != nil {
panic(fmt.Errorf("failed to decode request: %v", err))
}
defer metrics.MeasureSinceWithLabels([]string{"fsm", "acl", "token"}, time.Now(),
[]metrics.Label{{Name: "op", Value: "bootstrap"}})
return c.state.ACLBootstrap(index, req.ResetIndex, &req.Token, false)
}
func (c *FSM) applyACLPolicyUpsertOperation(buf []byte, index uint64) interface{} {
var req structs.ACLPolicyBatchUpsertRequest
if err := structs.Decode(buf, &req); err != nil {
panic(fmt.Errorf("failed to decode request: %v", err))
}
defer metrics.MeasureSinceWithLabels([]string{"fsm", "acl", "policy"}, time.Now(),
[]metrics.Label{{Name: "op", Value: "upsert"}})
return c.state.ACLPoliciesUpsert(index, req.Policies)
}
func (c *FSM) applyACLPolicyDeleteOperation(buf []byte, index uint64) interface{} {
var req structs.ACLPolicyBatchDeleteRequest
if err := structs.Decode(buf, &req); err != nil {
panic(fmt.Errorf("failed to decode request: %v", err))
}
defer metrics.MeasureSinceWithLabels([]string{"fsm", "acl", "policy"}, time.Now(),
[]metrics.Label{{Name: "op", Value: "delete"}})
return c.state.ACLPoliciesDelete(index, req.PolicyIDs)
}

22
agent/consul/fsm/commands_oss_test.go
View File

@ -796,7 +796,7 @@ func TestFSM_ACL_CRUD(t *testing.T) {
ACL: structs.ACL{
ID: generateUUID(),
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
},
}
buf, err := structs.Encode(structs.ACLRequestType, req)
@ -810,7 +810,7 @@ func TestFSM_ACL_CRUD(t *testing.T) {
// Get the ACL.
id := resp.(string)
_, acl, err := fsm.state.ACLGet(nil, id)
_, acl, err := fsm.state.ACLTokenGetBySecret(nil, id)
if err != nil {
t.Fatalf("err: %v", err)
}
@ -819,13 +819,13 @@ func TestFSM_ACL_CRUD(t *testing.T) {
}
// Verify the ACL.
if acl.ID != id {
if acl.SecretID != id {
t.Fatalf("bad: %v", *acl)
}
if acl.Name != "User token" {
if acl.Description != "User token" {
t.Fatalf("bad: %v", *acl)
}
if acl.Type != structs.ACLTypeClient {
if acl.Type != structs.ACLTokenTypeClient {
t.Fatalf("bad: %v", *acl)
}
@ -846,7 +846,7 @@ func TestFSM_ACL_CRUD(t *testing.T) {
t.Fatalf("resp: %v", resp)
}
_, acl, err = fsm.state.ACLGet(nil, id)
_, acl, err = fsm.state.ACLTokenGetBySecret(nil, id)
if err != nil {
t.Fatalf("err: %v", err)
}
@ -868,15 +868,13 @@ func TestFSM_ACL_CRUD(t *testing.T) {
if enabled, ok := resp.(bool); !ok || !enabled {
t.Fatalf("resp: %v", resp)
}
gotB, err := fsm.state.ACLGetBootstrap()
canBootstrap, _, err := fsm.state.CanBootstrapACLToken()
if err != nil {
t.Fatalf("err: %v", err)
}
wantB := &structs.ACLBootstrap{
AllowBootstrap: true,
RaftIndex: gotB.RaftIndex,
if !canBootstrap {
t.Fatalf("bad: shouldn't be able to bootstrap")
}
verify.Values(t, "", gotB, wantB)
// Do a bootstrap.
bootstrap := structs.ACLRequest{
@ -885,7 +883,7 @@ func TestFSM_ACL_CRUD(t *testing.T) {
ACL: structs.ACL{
ID: generateUUID(),
Name: "Bootstrap Token",
Type: structs.ACLTypeManagement,
Type: structs.ACLTokenTypeManagement,
},
}
buf, err = structs.Encode(structs.ACLRequestType, bootstrap)

79
agent/consul/fsm/snapshot_oss.go
View File

@ -24,6 +24,9 @@ func init() {
registerRestorer(structs.ConnectCARequestType, restoreConnectCA)
registerRestorer(structs.ConnectCAProviderStateType, restoreConnectCAProviderState)
registerRestorer(structs.ConnectCAConfigType, restoreConnectCAConfig)
registerRestorer(structs.IndexRequestType, restoreIndex)
registerRestorer(structs.ACLTokenUpsertRequestType, restoreToken)
registerRestorer(structs.ACLPolicyUpsertRequestType, restorePolicy)
}
func persistOSS(s *snapshot, sink raft.SnapshotSink, encoder *codec.Encoder) error {
@ -60,6 +63,9 @@ func persistOSS(s *snapshot, sink raft.SnapshotSink, encoder *codec.Encoder) err
if err := s.persistConnectCAConfig(sink, encoder); err != nil {
return err
}
if err := s.persistIndex(sink, encoder); err != nil {
return err
}
return nil
}
@ -161,29 +167,30 @@ func (s *snapshot) persistSessions(sink raft.SnapshotSink,
func (s *snapshot) persistACLs(sink raft.SnapshotSink,
encoder *codec.Encoder) error {
acls, err := s.state.ACLs()
tokens, err := s.state.ACLTokens()
if err != nil {
return err
}
for acl := acls.Next(); acl != nil; acl = acls.Next() {
if _, err := sink.Write([]byte{byte(structs.ACLRequestType)}); err != nil {
for token := tokens.Next(); token != nil; token = tokens.Next() {
if _, err := sink.Write([]byte{byte(structs.ACLTokenUpsertRequestType)}); err != nil {
return err
}
if err := encoder.Encode(acl.(*structs.ACL)); err != nil {
if err := encoder.Encode(token.(*structs.ACLToken)); err != nil {
return err
}
}
bs, err := s.state.ACLBootstrap()
policies, err := s.state.ACLPolicies()
if err != nil {
return err
}
if bs != nil {
if _, err := sink.Write([]byte{byte(structs.ACLBootstrapRequestType)}); err != nil {
for policy := policies.Next(); policy != nil; policy = policies.Next() {
if _, err := sink.Write([]byte{byte(structs.ACLPolicyUpsertRequestType)}); err != nil {
return err
}
if err := encoder.Encode(bs); err != nil {
if err := encoder.Encode(policy.(*structs.ACLPolicy)); err != nil {
return err
}
}
@ -346,6 +353,26 @@ func (s *snapshot) persistIntentions(sink raft.SnapshotSink,
return nil
}
func (s *snapshot) persistIndex(sink raft.SnapshotSink, encoder *codec.Encoder) error {
// Get all the indexes
iter, err := s.state.Indexes()
if err != nil {
return err
}
for raw := iter.Next(); raw != nil; raw = iter.Next() {
// Prepare the request struct
idx := raw.(*state.IndexEntry)
// Write out a node registration
sink.Write([]byte{byte(structs.IndexRequestType)})
if err := encoder.Encode(idx); err != nil {
return err
}
}
return nil
}
func restoreRegistration(header *snapshotHeader, restore *state.Restore, decoder *codec.Decoder) error {
var req structs.RegisterRequest
if err := decoder.Decode(&req); err != nil {
@ -403,21 +430,23 @@ func restoreACL(header *snapshotHeader, restore *state.Restore, decoder *codec.D
if err := decoder.Decode(&req); err != nil {
return err
}
if err := restore.ACL(&req); err != nil {
if err := restore.ACLToken(req.Convert()); err != nil {
return err
}
return nil
}
// DEPRECATED (ACL-Legacy-Compat) - remove once v1 acl compat is removed
func restoreACLBootstrap(header *snapshotHeader, restore *state.Restore, decoder *codec.Decoder) error {
var req structs.ACLBootstrap
if err := decoder.Decode(&req); err != nil {
return err
}
if err := restore.ACLBootstrap(&req); err != nil {
return err
}
return nil
// With V2 ACLs whether bootstrapping has been performed is stored in the index table like nomad
// so this "restores" into that index table.
return restore.IndexRestore(&state.IndexEntry{"acl-token-bootstrap", req.ModifyIndex})
}
func restoreCoordinates(header *snapshotHeader, restore *state.Restore, decoder *codec.Decoder) error {
@ -496,3 +525,27 @@ func restoreConnectCAConfig(header *snapshotHeader, restore *state.Restore, deco
}
return nil
}
func restoreIndex(header *snapshotHeader, restore *state.Restore, decoder *codec.Decoder) error {
var req state.IndexEntry
if err := decoder.Decode(&req); err != nil {
return err
}
return restore.IndexRestore(&req)
}
func restoreToken(header *snapshotHeader, restore *state.Restore, decoder *codec.Decoder) error {
var req structs.ACLToken
if err := decoder.Decode(&req); err != nil {
return err
}
return restore.ACLToken(&req)
}
func restorePolicy(header *snapshotHeader, restore *state.Restore, decoder *codec.Decoder) error {
var req structs.ACLPolicy
if err := decoder.Decode(&req); err != nil {
return err
}
return restore.ACLPolicy(&req)
}

70
agent/consul/fsm/snapshot_oss_test.go
View File

@ -7,14 +7,16 @@ import (
"testing"
"time"
"github.com/hashicorp/consul/acl"
"github.com/hashicorp/consul/agent/connect"
"github.com/hashicorp/consul/agent/consul/autopilot"
"github.com/hashicorp/consul/agent/consul/state"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/consul/api"
"github.com/hashicorp/consul/lib"
"github.com/pascaldekloe/goe/verify"
// "github.com/pascaldekloe/goe/verify"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestFSM_SnapshotRestore_OSS(t *testing.T) {
@ -66,11 +68,31 @@ func TestFSM_SnapshotRestore_OSS(t *testing.T) {
})
session := &structs.Session{ID: generateUUID(), Node: "foo"}
fsm.state.SessionCreate(9, session)
acl := &structs.ACL{ID: generateUUID(), Name: "User Token"}
fsm.state.ACLSet(10, acl)
if _, err := fsm.state.ACLBootstrapInit(10); err != nil {
t.Fatalf("err: %v", err)
policy := structs.ACLPolicy{
ID: structs.ACLPolicyGlobalManagementID,
Name: "global-management",
Description: "Builtin Policy that grants unlimited access",
Rules: structs.ACLPolicyGlobalManagement,
Syntax: acl.SyntaxCurrent,
}
policy.SetHash(true)
require.NoError(t, fsm.state.ACLPolicySet(1, &policy))
token := &structs.ACLToken{
AccessorID: "30fca056-9fbb-4455-b94a-bf0e2bc575d6",
SecretID: "cbe1c6fd-d865-4034-9d6d-64fef7fb46a9",
Description: "Bootstrap Token (Global Management)",
Policies: []structs.ACLTokenPolicyLink{
{
ID: structs.ACLPolicyGlobalManagementID,
},
},
CreateTime: time.Now(),
Local: false,
// DEPRECATED (ACL-Legacy-Compat) - This is used so that the bootstrap token is still visible via the v1 acl APIs
Type: structs.ACLTokenTypeManagement,
}
require.NoError(t, fsm.state.ACLBootstrap(10, 0, token, false))
fsm.state.KVSSet(11, &structs.DirEntry{
Key: "/remove",
@ -257,29 +279,21 @@ func TestFSM_SnapshotRestore_OSS(t *testing.T) {
t.Fatalf("bad index: %d", idx)
}
// Verify ACL is restored
_, a, err := fsm2.state.ACLGet(nil, acl.ID)
if err != nil {
t.Fatalf("err: %v", err)
}
if a.Name != "User Token" {
t.Fatalf("bad: %v", a)
}
if a.ModifyIndex <= 1 {
t.Fatalf("bad index: %d", idx)
}
gotB, err := fsm2.state.ACLGetBootstrap()
if err != nil {
t.Fatalf("err: %v", err)
}
wantB := &structs.ACLBootstrap{
AllowBootstrap: true,
RaftIndex: structs.RaftIndex{
CreateIndex: 10,
ModifyIndex: 10,
},
}
verify.Values(t, "", gotB, wantB)
// Verify ACL Token is restored
_, a, err := fsm2.state.ACLTokenGetByAccessor(nil, token.AccessorID)
require.NoError(t, err)
require.Equal(t, token.AccessorID, a.AccessorID)
require.Equal(t, token.ModifyIndex, a.ModifyIndex)
// Verify the acl-token-bootstrap index was restored
canBootstrap, index, err := fsm2.state.CanBootstrapACLToken()
require.False(t, canBootstrap)
require.True(t, index > 0)
// Verify ACL Policy is restored
_, policy2, err := fsm2.state.ACLPolicyGetByID(nil, structs.ACLPolicyGlobalManagementID)
require.NoError(t, err)
require.Equal(t, policy.Name, policy2.Name)
// Verify tombstones are restored
func() {

7
agent/consul/health_endpoint.go
View File

@ -2,6 +2,7 @@ package consul
import (
"fmt"
"sort"
"github.com/armon/go-metrics"
"github.com/hashicorp/consul/agent/consul/state"
@ -126,7 +127,7 @@ func (h *Health) ServiceNodes(args *structs.ServiceSpecificRequest, reply *struc
// we're trying to find proxies for, so check that.
if args.Connect {
// Fetch the ACL token, if any.
rule, err := h.srv.resolveToken(args.Token)
rule, err := h.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -171,6 +172,10 @@ func (h *Health) ServiceNodes(args *structs.ServiceSpecificRequest, reply *struc
[]metrics.Label{{Name: "service", Value: args.ServiceName}, {Name: "tag", Value: args.ServiceTag}})
}
if len(args.ServiceTags) > 0 {
// Sort tags so that the metric is the same even if the request
// tags are in a different order
sort.Strings(args.ServiceTags)
labels := []metrics.Label{{Name: "service", Value: args.ServiceName}}
for _, tag := range args.ServiceTags {
labels = append(labels, metrics.Label{Name: "tag", Value: tag})

3
agent/consul/health_endpoint_test.go
View File

@ -891,6 +891,7 @@ func TestHealth_ServiceNodes_ConnectProxy_ACL(t *testing.T) {
assert := assert.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = false
@ -908,7 +909,7 @@ func TestHealth_ServiceNodes_ConnectProxy_ACL(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: `
service "foo" {
policy = "write"

8
agent/consul/intention_endpoint.go
View File

@ -74,7 +74,7 @@ func (s *Intention) Apply(
*reply = args.Intention.ID
// Get the ACL token for the request for the checks below.
rule, err := s.srv.resolveToken(args.Token)
rule, err := s.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -225,7 +225,7 @@ func (s *Intention) Match(
}
// Get the ACL token for the request for the checks below.
rule, err := s.srv.resolveToken(args.Token)
rule, err := s.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -291,7 +291,7 @@ func (s *Intention) Check(
}
// Get the ACL token for the request for the checks below.
rule, err := s.srv.resolveToken(args.Token)
rule, err := s.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -344,7 +344,7 @@ func (s *Intention) Check(
// NOTE(mitchellh): This is the same behavior as the agent authorize
// endpoint. If this behavior is incorrect, we should also change it there
// which is much more important.
rule, err = s.srv.resolveToken("")
rule, err = s.srv.ResolveToken("")
if err != nil {
return err
}

30
agent/consul/intention_endpoint_test.go
View File

@ -314,6 +314,7 @@ func TestIntentionApply_aclDeny(t *testing.T) {
assert := assert.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -338,7 +339,7 @@ service "foo" {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -392,6 +393,7 @@ func TestIntentionApply_aclDelete(t *testing.T) {
assert := assert.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -416,7 +418,7 @@ service "foo" {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -468,6 +470,7 @@ func TestIntentionApply_aclUpdate(t *testing.T) {
assert := assert.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -492,7 +495,7 @@ service "foo" {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -532,6 +535,7 @@ func TestIntentionApply_aclManagement(t *testing.T) {
assert := assert.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -572,6 +576,7 @@ func TestIntentionApply_aclUpdateChange(t *testing.T) {
assert := assert.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -596,7 +601,7 @@ service "foo" {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -633,6 +638,7 @@ func TestIntentionGet_acl(t *testing.T) {
assert := assert.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -657,7 +663,7 @@ service "foo" {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -737,6 +743,7 @@ func TestIntentionList_acl(t *testing.T) {
assert := assert.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -761,7 +768,7 @@ service "foo" {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -904,6 +911,7 @@ func TestIntentionMatch_acl(t *testing.T) {
assert := assert.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -928,7 +936,7 @@ service "bar" {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -1048,6 +1056,7 @@ func TestIntentionCheck_defaultACLDeny(t *testing.T) {
require := require.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -1082,6 +1091,7 @@ func TestIntentionCheck_defaultACLAllow(t *testing.T) {
require := require.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "allow"
})
@ -1116,6 +1126,7 @@ func TestIntentionCheck_aclDeny(t *testing.T) {
require := require.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -1139,7 +1150,7 @@ service "bar" {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -1171,6 +1182,7 @@ func TestIntentionCheck_match(t *testing.T) {
require := require.New(t)
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -1194,7 +1206,7 @@ service "bar" {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},

4
agent/consul/internal_endpoint.go
View File

@ -70,7 +70,7 @@ func (m *Internal) EventFire(args *structs.EventFireRequest,
}
// Check ACLs
rule, err := m.srv.resolveToken(args.Token)
rule, err := m.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -105,7 +105,7 @@ func (m *Internal) KeyringOperation(
reply *structs.KeyringResponses) error {
// Check ACLs
rule, err := m.srv.resolveToken(args.Token)
rule, err := m.srv.ResolveToken(args.Token)
if err != nil {
return err
}

1
agent/consul/internal_endpoint_test.go
View File

@ -347,6 +347,7 @@ func TestInternal_EventFire_Token(t *testing.T) {
t.Parallel()
dir, srv := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDownPolicy = "deny"
c.ACLDefaultPolicy = "deny"

10
agent/consul/kvs_endpoint.go
View File

@ -21,7 +21,7 @@ type KVS struct {
// preApply does all the verification of a KVS update that is performed BEFORE
// we submit as a Raft log entry. This includes enforcing the lock delay which
// must only be done on the leader.
func kvsPreApply(srv *Server, rule acl.ACL, op api.KVOp, dirEnt *structs.DirEntry) (bool, error) {
func kvsPreApply(srv *Server, rule acl.Authorizer, op api.KVOp, dirEnt *structs.DirEntry) (bool, error) {
// Verify the entry.
if dirEnt.Key == "" && op != api.KVDeleteTree {
@ -84,7 +84,7 @@ func (k *KVS) Apply(args *structs.KVSRequest, reply *bool) error {
defer metrics.MeasureSince([]string{"kvs", "apply"}, time.Now())
// Perform the pre-apply checks.
acl, err := k.srv.resolveToken(args.Token)
acl, err := k.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -120,7 +120,7 @@ func (k *KVS) Get(args *structs.KeyRequest, reply *structs.IndexedDirEntries) er
return err
}
aclRule, err := k.srv.resolveToken(args.Token)
aclRule, err := k.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -159,7 +159,7 @@ func (k *KVS) List(args *structs.KeyRequest, reply *structs.IndexedDirEntries) e
return err
}
aclToken, err := k.srv.resolveToken(args.Token)
aclToken, err := k.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -203,7 +203,7 @@ func (k *KVS) ListKeys(args *structs.KeyListRequest, reply *structs.IndexedKeyLi
return err
}
aclToken, err := k.srv.resolveToken(args.Token)
aclToken, err := k.srv.ResolveToken(args.Token)
if err != nil {
return err
}

13
agent/consul/kvs_endpoint_test.go
View File

@ -74,6 +74,7 @@ func TestKVS_Apply_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -90,7 +91,7 @@ func TestKVS_Apply_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: testListRules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -185,6 +186,7 @@ func TestKVS_Get_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -393,6 +395,7 @@ func TestKVSEndpoint_List_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -432,7 +435,7 @@ func TestKVSEndpoint_List_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: testListRules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -478,6 +481,7 @@ func TestKVSEndpoint_List_ACLEnableKeyListPolicy(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnableKeyListPolicy = true
@ -530,7 +534,7 @@ key "zip" {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: testListRules1,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -676,6 +680,7 @@ func TestKVSEndpoint_ListKeys_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -715,7 +720,7 @@ func TestKVSEndpoint_ListKeys_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: testListRules,
},
WriteRequest: structs.WriteRequest{Token: "root"},

506
agent/consul/leader.go
View File

@ -1,11 +1,13 @@
package consul
import (
"context"
"fmt"
"net"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/armon/go-metrics"
@ -16,11 +18,14 @@ import (
"github.com/hashicorp/consul/agent/metadata"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/consul/api"
"github.com/hashicorp/consul/lib"
"github.com/hashicorp/consul/types"
memdb "github.com/hashicorp/go-memdb"
uuid "github.com/hashicorp/go-uuid"
"github.com/hashicorp/go-version"
"github.com/hashicorp/raft"
"github.com/hashicorp/serf/serf"
"golang.org/x/time/rate"
)
const (
@ -47,6 +52,11 @@ func (s *Server) monitorLeadership() {
// cleanup and to ensure we never run multiple leader loops.
raftNotifyCh := s.raftNotifyCh
aclModeCheckWait := aclModeCheckMinInterval
var aclUpgradeCh <-chan time.Time
if s.ACLsEnabled() {
aclUpgradeCh = time.After(aclModeCheckWait)
}
var weAreLeaderCh chan struct{}
var leaderLoop sync.WaitGroup
for {
@ -79,7 +89,33 @@ func (s *Server) monitorLeadership() {
weAreLeaderCh = nil
s.logger.Printf("[INFO] consul: cluster leadership lost")
}
case <-aclUpgradeCh:
if atomic.LoadInt32(&s.useNewACLs) == 0 {
aclModeCheckWait = aclModeCheckWait * 2
if aclModeCheckWait > aclModeCheckMaxInterval {
aclModeCheckWait = aclModeCheckMaxInterval
}
aclUpgradeCh = time.After(aclModeCheckWait)
if canUpgrade := s.canUpgradeToNewACLs(weAreLeaderCh != nil); canUpgrade {
if weAreLeaderCh != nil {
if err := s.initializeACLs(true); err != nil {
s.logger.Printf("[ERR] consul: error transitioning to using new ACLs: %v", err)
continue
}
}
s.logger.Printf("[DEBUG] acl: transitioning out of legacy ACL mode")
atomic.StoreInt32(&s.useNewACLs, 1)
s.updateACLAdvertisement()
// setting this to nil ensures that we will never hit this case again
aclUpgradeCh = nil
}
} else {
// establishLeadership probably transitioned us
aclUpgradeCh = nil
}
case <-s.shutdownCh:
return
}
@ -193,9 +229,15 @@ WAIT:
// previously inflight transactions have been committed and that our
// state is up-to-date.
func (s *Server) establishLeadership() error {
// This will create the anonymous token and master token (if that is
// configured).
if err := s.initializeACL(); err != nil {
// check for the upgrade here - this helps us transition to new ACLs much
// quicker if this is a new cluster or this is a test agent
if canUpgrade := s.canUpgradeToNewACLs(true); canUpgrade {
if err := s.initializeACLs(true); err != nil {
return err
}
atomic.StoreInt32(&s.useNewACLs, 1)
s.updateACLAdvertisement()
} else if err := s.initializeACLs(false); err != nil {
return err
}
@ -253,60 +295,58 @@ func (s *Server) revokeLeadership() error {
s.setCAProvider(nil, nil)
s.stopACLUpgrade()
s.resetConsistentReadReady()
s.autopilot.Stop()
return nil
}
// initializeACL is used to setup the ACLs if we are the leader
// and need to do this.
func (s *Server) initializeACL() error {
// Bail if not configured or we are not authoritative.
authDC := s.config.ACLDatacenter
if len(authDC) == 0 || authDC != s.config.Datacenter {
// DEPRECATED (ACL-Legacy-Compat) - Remove once old ACL compatibility is removed
func (s *Server) initializeLegacyACL() error {
if !s.ACLsEnabled() {
return nil
}
// Purge the cache, since it could've changed while we were not the
// leader.
s.aclAuthCache.Purge()
authDC := s.config.ACLDatacenter
// Create anonymous token if missing.
state := s.fsm.State()
_, acl, err := state.ACLGet(nil, anonymousToken)
_, token, err := state.ACLTokenGetBySecret(nil, anonymousToken)
if err != nil {
return fmt.Errorf("failed to get anonymous token: %v", err)
}
if acl == nil {
if token == nil {
req := structs.ACLRequest{
Datacenter: authDC,
Op: structs.ACLSet,
ACL: structs.ACL{
ID: anonymousToken,
Name: "Anonymous Token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
},
}
_, err := s.raftApply(structs.ACLRequestType, &req)
if err != nil {
return fmt.Errorf("failed to create anonymous token: %v", err)
}
s.logger.Printf("[INFO] acl: Created the anonymous token")
}
// Check for configured master token.
if master := s.config.ACLMasterToken; len(master) > 0 {
_, acl, err = state.ACLGet(nil, master)
_, token, err = state.ACLTokenGetBySecret(nil, master)
if err != nil {
return fmt.Errorf("failed to get master token: %v", err)
}
if acl == nil {
if token == nil {
req := structs.ACLRequest{
Datacenter: authDC,
Op: structs.ACLSet,
ACL: structs.ACL{
ID: master,
Name: "Master Token",
Type: structs.ACLTypeManagement,
Type: structs.ACLTokenTypeManagement,
},
}
_, err := s.raftApply(structs.ACLRequestType, &req)
@ -324,11 +364,11 @@ func (s *Server) initializeACL() error {
// servers consuming snapshots, so we have to wait to create it.
var minVersion = version.Must(version.NewVersion("0.9.1"))
if ServersMeetMinimumVersion(s.LANMembers(), minVersion) {
bs, err := state.ACLGetBootstrap()
canBootstrap, _, err := state.CanBootstrapACLToken()
if err != nil {
return fmt.Errorf("failed looking for ACL bootstrap info: %v", err)
}
if bs == nil {
if canBootstrap {
req := structs.ACLRequest{
Datacenter: authDC,
Op: structs.ACLBootstrapInit,
@ -359,6 +399,432 @@ func (s *Server) initializeACL() error {
return nil
}
// initializeACLs is used to setup the ACLs if we are the leader
// and need to do this.
func (s *Server) initializeACLs(upgrade bool) error {
if !s.ACLsEnabled() {
return nil
}
// Purge the cache, since it could've changed while we were not the
// leader.
s.acls.cache.Purge()
if s.InACLDatacenter() {
if s.UseLegacyACLs() && !upgrade {
s.logger.Printf("[INFO] acl: initializing legacy acls")
return s.initializeLegacyACL()
}
s.logger.Printf("[INFO] acl: initializing acls")
// Create the builtin global-management policy
_, policy, err := s.fsm.State().ACLPolicyGetByID(nil, structs.ACLPolicyGlobalManagementID)
if err != nil {
return fmt.Errorf("failed to get the builtin global-management policy")
}
if policy == nil {
policy := structs.ACLPolicy{
ID: structs.ACLPolicyGlobalManagementID,
Name: "global-management",
Description: "Builtin Policy that grants unlimited access",
Rules: structs.ACLPolicyGlobalManagement,
Syntax: acl.SyntaxCurrent,
}
policy.SetHash(true)
req := structs.ACLPolicyBatchUpsertRequest{
Policies: structs.ACLPolicies{&policy},
}
_, err := s.raftApply(structs.ACLPolicyUpsertRequestType, &req)
if err != nil {
return fmt.Errorf("failed to create global-management policy: %v", err)
}
s.logger.Printf("[INFO] consul: Created ACL 'global-management' policy")
}
// Check for configured master token.
if master := s.config.ACLMasterToken; len(master) > 0 {
state := s.fsm.State()
if _, err := uuid.ParseUUID(master); err != nil {
s.logger.Printf("[WARN] consul: Configuring a non-UUID master token is deprecated")
}
_, token, err := state.ACLTokenGetBySecret(nil, master)
if err != nil {
return fmt.Errorf("failed to get master token: %v", err)
}
if token == nil {
accessor, err := lib.GenerateUUID(s.checkTokenUUID)
if err != nil {
return fmt.Errorf("failed to generate the accessor ID for the master token: %v", err)
}
token := structs.ACLToken{
AccessorID: accessor,
SecretID: master,
Description: "Master Token",
Policies: []structs.ACLTokenPolicyLink{
{
ID: structs.ACLPolicyGlobalManagementID,
},
},
CreateTime: time.Now(),
Local: false,
// DEPRECATED (ACL-Legacy-Compat) - only needed for compatibility
Type: structs.ACLTokenTypeManagement,
}
token.SetHash(true)
done := false
if canBootstrap, _, err := state.CanBootstrapACLToken(); err == nil && canBootstrap {
req := structs.ACLTokenBootstrapRequest{
Token: token,
ResetIndex: 0,
}
if _, err := s.raftApply(structs.ACLBootstrapRequestType, &req); err == nil {
s.logger.Printf("[INFO] consul: Bootstrapped ACL master token from configuration")
done = true
} else {
if err.Error() != structs.ACLBootstrapNotAllowedErr.Error() &&
err.Error() != structs.ACLBootstrapInvalidResetIndexErr.Error() {
return fmt.Errorf("failed to bootstrap master token: %v", err)
}
}
}
if !done {
// either we didn't attempt to or setting the token with a bootstrap request failed.
req := structs.ACLTokenBatchUpsertRequest{
Tokens: structs.ACLTokens{&token},
}
if _, err := s.raftApply(structs.ACLTokenUpsertRequestType, &req); err != nil {
return fmt.Errorf("failed to create master token: %v", err)
}
s.logger.Printf("[INFO] consul: Created ACL master token from configuration")
}
}
}
state := s.fsm.State()
_, token, err := state.ACLTokenGetBySecret(nil, structs.ACLTokenAnonymousID)
if err != nil {
return fmt.Errorf("failed to get anonymous token: %v", err)
}
if token == nil {
// DEPRECATED (ACL-Legacy-Compat) - Don't need to query for previous "anonymous" token
// check for legacy token that needs an upgrade
_, legacyToken, err := state.ACLTokenGetBySecret(nil, anonymousToken)
if err != nil {
return fmt.Errorf("failed to get anonymous token: %v", err)
}
// the token upgrade routine will take care of upgrading the token if a legacy version exists
if legacyToken == nil {
token = &structs.ACLToken{
AccessorID: structs.ACLTokenAnonymousID,
SecretID: anonymousToken,
Description: "Anonymous Token",
}
token.SetHash(true)
req := structs.ACLTokenBatchUpsertRequest{
Tokens: structs.ACLTokens{token},
AllowCreate: true,
}
_, err := s.raftApply(structs.ACLTokenUpsertRequestType, &req)
if err != nil {
return fmt.Errorf("failed to create anonymous token: %v", err)
}
s.logger.Printf("[INFO] consul: Created ACL anonymous token from configuration")
}
}
s.startACLUpgrade()
} else {
if s.UseLegacyACLs() && !upgrade {
if s.IsACLReplicationEnabled() {
s.startLegacyACLReplication()
}
}
if upgrade {
s.stopACLReplication()
}
// ACL replication is now mandatory
s.startACLReplication()
}
// launch the upgrade go routine to generate accessors for everything
return nil
}
func (s *Server) startACLUpgrade() {
s.aclUpgradeLock.Lock()
defer s.aclUpgradeLock.Unlock()
if s.aclUpgradeEnabled {
return
}
ctx, cancel := context.WithCancel(context.Background())
s.aclUpgradeCancel = cancel
go func() {
limiter := rate.NewLimiter(aclUpgradeRateLimit, int(aclUpgradeRateLimit))
for {
if err := limiter.Wait(ctx); err != nil {
return
}
// actually run the upgrade here
state := s.fsm.State()
tokens, waitCh, err := state.ACLTokenListUpgradeable(aclUpgradeBatchSize)
if err != nil {
s.logger.Printf("[WARN] acl: encountered an error while searching for tokens without accessor ids: %v", err)
}
if len(tokens) == 0 {
ws := memdb.NewWatchSet()
ws.Add(state.AbandonCh())
ws.Add(waitCh)
ws.Add(ctx.Done())
// wait for more tokens to need upgrading or the aclUpgradeCh to be closed
ws.Watch(nil)
continue
}
var newTokens structs.ACLTokens
for _, token := range tokens {
// This should be entirely unnessary but is just a small safeguard against changing accessor IDs
if token.AccessorID != "" {
continue
}
newToken := *token
if token.SecretID == anonymousToken {
newToken.AccessorID = structs.ACLTokenAnonymousID
} else {
accessor, err := lib.GenerateUUID(s.checkTokenUUID)
if err != nil {
s.logger.Printf("[WARN] acl: failed to generate accessor during token auto-upgrade: %v", err)
continue
}
newToken.AccessorID = accessor
}
// Assign the global-management policy to legacy management tokens
if len(newToken.Policies) == 0 && newToken.Type == structs.ACLTokenTypeManagement {
newToken.Policies = append(newToken.Policies, structs.ACLTokenPolicyLink{ID: structs.ACLPolicyGlobalManagementID})
}
newTokens = append(newTokens, &newToken)
}
req := &structs.ACLTokenBatchUpsertRequest{Tokens: newTokens, AllowCreate: false}
resp, err := s.raftApply(structs.ACLTokenUpsertRequestType, req)
if err != nil {
s.logger.Printf("[ERR] acl: failed to apply acl token upgrade batch: %v", err)
}
if err, ok := resp.(error); ok {
s.logger.Printf("[ERR] acl: failed to apply acl token upgrade batch: %v", err)
}
}
}()
s.aclUpgradeEnabled = true
}
func (s *Server) stopACLUpgrade() {
s.aclUpgradeLock.Lock()
defer s.aclUpgradeLock.Unlock()
if !s.aclUpgradeEnabled {
return
}
s.aclUpgradeCancel()
s.aclUpgradeCancel = nil
s.aclUpgradeEnabled = false
}
func (s *Server) startLegacyACLReplication() {
s.aclReplicationLock.Lock()
defer s.aclReplicationLock.Unlock()
if s.aclReplicationEnabled {
return
}
s.initReplicationStatus()
ctx, cancel := context.WithCancel(context.Background())
s.aclReplicationCancel = cancel
go func() {
var lastRemoteIndex uint64
limiter := rate.NewLimiter(rate.Limit(s.config.ACLReplicationRate), s.config.ACLReplicationBurst)
for {
if err := limiter.Wait(ctx); err != nil {
return
}
if s.tokens.ACLReplicationToken() == "" {
continue
}
index, exit, err := s.replicateLegacyACLs(lastRemoteIndex, ctx)
if exit {
return
}
if err != nil {
lastRemoteIndex = 0
s.updateACLReplicationStatusError()
s.logger.Printf("[WARN] consul: Legacy ACL replication error (will retry if still leader): %v", err)
} else {
lastRemoteIndex = index
s.updateACLReplicationStatusIndex(index)
s.logger.Printf("[DEBUG] consul: Legacy ACL replication completed through remote index %d", index)
}
}
}()
s.updateACLReplicationStatusRunning(structs.ACLReplicateLegacy)
s.aclReplicationEnabled = true
}
func (s *Server) startACLReplication() {
s.aclReplicationLock.Lock()
defer s.aclReplicationLock.Unlock()
if s.aclReplicationEnabled {
return
}
s.initReplicationStatus()
ctx, cancel := context.WithCancel(context.Background())
s.aclReplicationCancel = cancel
replicationType := structs.ACLReplicatePolicies
go func() {
var failedAttempts uint
limiter := rate.NewLimiter(rate.Limit(s.config.ACLReplicationRate), s.config.ACLReplicationBurst)
var lastRemoteIndex uint64
for {
if err := limiter.Wait(ctx); err != nil {
return
}
if s.tokens.ACLReplicationToken() == "" {
continue
}
index, exit, err := s.replicateACLPolicies(lastRemoteIndex, ctx)
if exit {
return
}
if err != nil {
lastRemoteIndex = 0
s.updateACLReplicationStatusError()
s.logger.Printf("[WARN] consul: ACL policy replication error (will retry if still leader): %v", err)
if (1 << failedAttempts) < aclReplicationMaxRetryBackoff {
failedAttempts++
}
select {
case <-ctx.Done():
return
case <-time.After((1 << failedAttempts) * time.Second):
// do nothing
}
} else {
lastRemoteIndex = index
s.updateACLReplicationStatusIndex(index)
s.logger.Printf("[DEBUG] consul: ACL policy replication completed through remote index %d", index)
failedAttempts = 0
}
}
}()
s.logger.Printf("[INFO] acl: started ACL Policy replication")
if s.config.ACLTokenReplication {
replicationType = structs.ACLReplicateTokens
go func() {
var failedAttempts uint
limiter := rate.NewLimiter(rate.Limit(s.config.ACLReplicationRate), s.config.ACLReplicationBurst)
var lastRemoteIndex uint64
for {
if err := limiter.Wait(ctx); err != nil {
return
}
if s.tokens.ACLReplicationToken() == "" {
continue
}
index, exit, err := s.replicateACLTokens(lastRemoteIndex, ctx)
if exit {
return
}
if err != nil {
lastRemoteIndex = 0
s.updateACLReplicationStatusError()
s.logger.Printf("[WARN] consul: ACL token replication error (will retry if still leader): %v", err)
if (1 << failedAttempts) < aclReplicationMaxRetryBackoff {
failedAttempts++
}
select {
case <-ctx.Done():
return
case <-time.After((1 << failedAttempts) * time.Second):
// do nothing
}
} else {
lastRemoteIndex = index
s.updateACLReplicationStatusTokenIndex(index)
s.logger.Printf("[DEBUG] consul: ACL token replication completed through remote index %d", index)
failedAttempts = 0
}
}
}()
s.logger.Printf("[INFO] acl: started ACL Token replication")
}
s.updateACLReplicationStatusRunning(replicationType)
s.aclReplicationEnabled = true
}
func (s *Server) stopACLReplication() {
s.aclReplicationLock.Lock()
defer s.aclReplicationLock.Unlock()
if !s.aclReplicationEnabled {
return
}
s.aclReplicationCancel()
s.aclReplicationCancel = nil
s.updateACLReplicationStatusStopped()
s.aclReplicationEnabled = false
}
// getOrCreateAutopilotConfig is used to get the autopilot config, initializing it if necessary
func (s *Server) getOrCreateAutopilotConfig() *autopilot.Config {
state := s.fsm.State()

41
agent/consul/leader_test.go
View File

@ -20,6 +20,7 @@ func TestLeader_RegisterMember(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = true
@ -89,6 +90,7 @@ func TestLeader_FailedMember(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = true
@ -150,6 +152,7 @@ func TestLeader_LeftMember(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = true
@ -196,6 +199,7 @@ func TestLeader_ReapMember(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = true
@ -257,6 +261,7 @@ func TestLeader_ReapServer(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "allow"
c.ACLEnforceVersion8 = true
@ -267,6 +272,7 @@ func TestLeader_ReapServer(t *testing.T) {
dir2, s2 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "allow"
c.ACLEnforceVersion8 = true
@ -277,6 +283,7 @@ func TestLeader_ReapServer(t *testing.T) {
dir3, s3 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "allow"
c.ACLEnforceVersion8 = true
@ -332,6 +339,7 @@ func TestLeader_Reconcile_ReapMember(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = true
@ -381,6 +389,7 @@ func TestLeader_Reconcile(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = true
@ -710,6 +719,7 @@ func TestLeader_ReapTombstones(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.TombstoneTTL = 50 * time.Millisecond
@ -950,13 +960,12 @@ func TestLeader_ACL_Initialization(t *testing.T) {
name string
build string
master string
init bool
bootstrap bool
}{
{"old version, no master", "0.8.0", "", false, false},
{"old version, master", "0.8.0", "root", false, false},
{"new version, no master", "0.9.1", "", true, true},
{"new version, master", "0.9.1", "root", true, false},
{"old version, no master", "0.8.0", "", true},
{"old version, master", "0.8.0", "root", false},
{"new version, no master", "0.9.1", "", true},
{"new version, master", "0.9.1", "root", false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
@ -965,6 +974,7 @@ func TestLeader_ACL_Initialization(t *testing.T) {
c.Bootstrap = true
c.Datacenter = "dc1"
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = tt.master
}
dir1, s1 := testServerWithConfig(t, conf)
@ -973,7 +983,7 @@ func TestLeader_ACL_Initialization(t *testing.T) {
testrpc.WaitForLeader(t, s1.RPC, "dc1")
if tt.master != "" {
_, master, err := s1.fsm.State().ACLGet(nil, tt.master)
_, master, err := s1.fsm.State().ACLTokenGetBySecret(nil, tt.master)
if err != nil {
t.Fatalf("err: %v", err)
}
@ -982,7 +992,7 @@ func TestLeader_ACL_Initialization(t *testing.T) {
}
}
_, anon, err := s1.fsm.State().ACLGet(nil, anonymousToken)
_, anon, err := s1.fsm.State().ACLTokenGetBySecret(nil, anonymousToken)
if err != nil {
t.Fatalf("err: %v", err)
}
@ -990,21 +1000,16 @@ func TestLeader_ACL_Initialization(t *testing.T) {
t.Fatalf("anonymous token wasn't created")
}
bs, err := s1.fsm.State().ACLGetBootstrap()
canBootstrap, _, err := s1.fsm.State().CanBootstrapACLToken()
if err != nil {
t.Fatalf("err: %v", err)
}
if !tt.init {
if bs != nil {
t.Fatalf("bootstrap should not be initialized")
}
} else {
if bs == nil {
t.Fatalf("bootstrap should be initialized")
}
if got, want := bs.AllowBootstrap, tt.bootstrap; got != want {
t.Fatalf("got %v want %v", got, want)
if tt.bootstrap {
if !canBootstrap {
t.Fatalf("bootstrap should be allowed")
}
} else if canBootstrap {
t.Fatalf("bootstrap should not be allowed")
}
})
}

6
agent/consul/operator_autopilot_endpoint.go
View File

@ -15,7 +15,7 @@ func (op *Operator) AutopilotGetConfiguration(args *structs.DCSpecificRequest, r
}
// This action requires operator read access.
rule, err := op.srv.resolveToken(args.Token)
rule, err := op.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -44,7 +44,7 @@ func (op *Operator) AutopilotSetConfiguration(args *structs.AutopilotSetConfigRe
}
// This action requires operator write access.
rule, err := op.srv.resolveToken(args.Token)
rule, err := op.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -80,7 +80,7 @@ func (op *Operator) ServerHealth(args *structs.DCSpecificRequest, reply *autopil
}
// This action requires operator read access.
rule, err := op.srv.resolveToken(args.Token)
rule, err := op.srv.ResolveToken(args.Token)
if err != nil {
return err
}

6
agent/consul/operator_autopilot_endpoint_test.go
View File

@ -44,6 +44,7 @@ func TestOperator_Autopilot_GetConfiguration_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.AutopilotConfig.CleanupDeadServers = false
@ -77,7 +78,7 @@ func TestOperator_Autopilot_GetConfiguration_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -138,6 +139,7 @@ func TestOperator_Autopilot_SetConfiguration_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.AutopilotConfig.CleanupDeadServers = false
@ -174,7 +176,7 @@ func TestOperator_Autopilot_SetConfiguration_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},

6
agent/consul/operator_raft_endpoint.go
View File

@ -18,7 +18,7 @@ func (op *Operator) RaftGetConfiguration(args *structs.DCSpecificRequest, reply
}
// This action requires operator read access.
rule, err := op.srv.resolveToken(args.Token)
rule, err := op.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -80,7 +80,7 @@ func (op *Operator) RaftRemovePeerByAddress(args *structs.RaftRemovePeerRequest,
// This is a super dangerous operation that requires operator write
// access.
rule, err := op.srv.resolveToken(args.Token)
rule, err := op.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -147,7 +147,7 @@ func (op *Operator) RaftRemovePeerByID(args *structs.RaftRemovePeerRequest, repl
// This is a super dangerous operation that requires operator write
// access.
rule, err := op.srv.resolveToken(args.Token)
rule, err := op.srv.ResolveToken(args.Token)
if err != nil {
return err
}

9
agent/consul/operator_raft_endpoint_test.go
View File

@ -62,6 +62,7 @@ func TestOperator_RaftGetConfiguration_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -94,7 +95,7 @@ func TestOperator_RaftGetConfiguration_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -198,6 +199,7 @@ func TestOperator_RaftRemovePeerByAddress_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -231,7 +233,7 @@ func TestOperator_RaftRemovePeerByAddress_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -315,6 +317,7 @@ func TestOperator_RaftRemovePeerByID_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.RaftConfig.ProtocolVersion = 3
@ -349,7 +352,7 @@ func TestOperator_RaftRemovePeerByID_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},

2
agent/consul/prepared_query_endpoint.go
View File

@ -60,7 +60,7 @@ func (p *PreparedQuery) Apply(args *structs.PreparedQueryRequest, reply *string)
*reply = args.Query.ID
// Get the ACL token for the request for the checks below.
rule, err := p.srv.resolveToken(args.Token)
rule, err := p.srv.ResolveToken(args.Token)
if err != nil {
return err
}

23
agent/consul/prepared_query_endpoint_test.go
View File

@ -188,6 +188,7 @@ func TestPreparedQuery_Apply_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -212,7 +213,7 @@ func TestPreparedQuery_Apply_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -627,6 +628,7 @@ func TestPreparedQuery_ACLDeny_Catchall_Template(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -651,7 +653,7 @@ func TestPreparedQuery_ACLDeny_Catchall_Template(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -841,6 +843,7 @@ func TestPreparedQuery_Get(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -865,7 +868,7 @@ func TestPreparedQuery_Get(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -1093,6 +1096,7 @@ func TestPreparedQuery_List(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -1117,7 +1121,7 @@ func TestPreparedQuery_List(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -1300,6 +1304,7 @@ func TestPreparedQuery_Explain(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -1324,7 +1329,7 @@ func TestPreparedQuery_Explain(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -1436,6 +1441,7 @@ func TestPreparedQuery_Execute(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = false
@ -1448,6 +1454,7 @@ func TestPreparedQuery_Execute(t *testing.T) {
dir2, s2 := testServerWithConfig(t, func(c *Config) {
c.Datacenter = "dc2"
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
})
defer os.RemoveAll(dir2)
defer s2.Shutdown()
@ -1479,7 +1486,7 @@ func TestPreparedQuery_Execute(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -2163,7 +2170,7 @@ func TestPreparedQuery_Execute(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: rules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -2867,6 +2874,7 @@ func TestPreparedQuery_Wrapper(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -2876,6 +2884,7 @@ func TestPreparedQuery_Wrapper(t *testing.T) {
dir2, s2 := testServerWithConfig(t, func(c *Config) {
c.Datacenter = "dc2"
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})

53
agent/consul/server.go
View File

@ -1,6 +1,7 @@
package consul
import (
"context"
"crypto/tls"
"errors"
"fmt"
@ -17,7 +18,6 @@ import (
"sync/atomic"
"time"
"github.com/hashicorp/consul/acl"
ca "github.com/hashicorp/consul/agent/connect/ca"
"github.com/hashicorp/consul/agent/consul/autopilot"
"github.com/hashicorp/consul/agent/consul/fsm"
@ -94,11 +94,24 @@ type Server struct {
// sentinel is the Sentinel code engine (can be nil).
sentinel sentinel.Evaluator
// aclAuthCache is the authoritative ACL cache.
aclAuthCache *acl.Cache
// acls is used to resolve tokens to effective policies
acls *ACLResolver
// aclCache is the non-authoritative ACL cache.
aclCache *aclCache
// aclUpgradeCancel is used to cancel the ACL upgrade goroutine when we
// lose leadership
aclUpgradeCancel context.CancelFunc
aclUpgradeLock sync.RWMutex
aclUpgradeEnabled bool
// aclReplicationCancel is used to shut down the ACL replication goroutine
// when we lose leadership
aclReplicationCancel context.CancelFunc
aclReplicationLock sync.RWMutex
aclReplicationEnabled bool
// DEPRECATED (ACL-Legacy-Compat) - only needed while we support both
// useNewACLs is used to determine whether we can use new ACLs or not
useNewACLs int32
// autopilot is the Autopilot instance for this server.
autopilot *autopilot.Autopilot
@ -344,23 +357,20 @@ func NewServerLogger(config *Config, logger *log.Logger, tokens *token.Store) (*
// Initialize the stats fetcher that autopilot will use.
s.statsFetcher = NewStatsFetcher(logger, s.connPool, s.config.Datacenter)
// Initialize the authoritative ACL cache.
s.sentinel = sentinel.New(logger)
s.aclAuthCache, err = acl.NewCache(aclCacheSize, s.aclLocalFault, s.sentinel)
if err != nil {
s.Shutdown()
return nil, fmt.Errorf("Failed to create authoritative ACL cache: %v", err)
s.useNewACLs = 0
aclConfig := ACLResolverConfig{
Config: config,
Delegate: s,
CacheConfig: serverACLCacheConfig,
AutoDisable: false,
Logger: logger,
Sentinel: s.sentinel,
}
// Set up the non-authoritative ACL cache. A nil local function is given
// if ACL replication isn't enabled.
var local acl.FaultFunc
if s.IsACLReplicationEnabled() {
local = s.aclLocalFault
}
if s.aclCache, err = newACLCache(config, logger, s.RPC, local, s.sentinel); err != nil {
// Initialize the ACL resolver.
if s.acls, err = NewACLResolver(&aclConfig); err != nil {
s.Shutdown()
return nil, fmt.Errorf("Failed to create non-authoritative ACL cache: %v", err)
return nil, fmt.Errorf("Failed to create ACL resolver: %v", err)
}
// Initialize the RPC layer.
@ -456,11 +466,6 @@ func NewServerLogger(config *Config, logger *log.Logger, tokens *token.Store) (*
// since it can fire events when leadership is obtained.
go s.monitorLeadership()
// Start ACL replication.
if s.IsACLReplicationEnabled() {
go s.runACLReplication()
}
// Start listening for RPC requests.
go s.listen(s.Listener)

8
agent/consul/server_serf.go
View File

@ -8,6 +8,7 @@ import (
"time"
"github.com/hashicorp/consul/agent/metadata"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/consul/lib"
"github.com/hashicorp/raft"
"github.com/hashicorp/serf/serf"
@ -69,6 +70,13 @@ func (s *Server) setupSerf(conf *serf.Config, ch chan serf.Event, path string, w
if s.config.UseTLS {
conf.Tags["use_tls"] = "1"
}
if s.config.ACLDatacenter != "" {
// we start in legacy mode and allow upgrading later
conf.Tags["acls"] = string(structs.ACLModeLegacy)
} else {
conf.Tags["acls"] = string(structs.ACLModeDisabled)
}
if s.logger == nil {
conf.MemberlistConfig.LogOutput = s.config.LogOutput
conf.LogOutput = s.config.LogOutput

4
agent/consul/session_endpoint.go
View File

@ -34,7 +34,7 @@ func (s *Session) Apply(args *structs.SessionRequest, reply *string) error {
}
// Fetch the ACL token, if any, and apply the policy.
rule, err := s.srv.resolveToken(args.Token)
rule, err := s.srv.ResolveToken(args.Token)
if err != nil {
return err
}
@ -236,7 +236,7 @@ func (s *Session) Renew(args *structs.SessionSpecificRequest,
}
// Fetch the ACL token, if any, and apply the policy.
rule, err := s.srv.resolveToken(args.Token)
rule, err := s.srv.ResolveToken(args.Token)
if err != nil {
return err
}

9
agent/consul/session_endpoint_test.go
View File

@ -140,6 +140,7 @@ func TestSession_Apply_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = false
@ -157,7 +158,7 @@ func TestSession_Apply_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: `
session "foo" {
policy = "write"
@ -331,6 +332,7 @@ func TestSession_Get_List_NodeSessions_ACLFilter(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = false
@ -348,7 +350,7 @@ func TestSession_Get_List_NodeSessions_ACLFilter(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: `
session "foo" {
policy = "read"
@ -705,6 +707,7 @@ func TestSession_Renew_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
c.ACLEnforceVersion8 = false
@ -721,7 +724,7 @@ func TestSession_Renew_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: `
session "foo" {
policy = "write"

2
agent/consul/snapshot_endpoint.go
View File

@ -59,7 +59,7 @@ func (s *Server) dispatchSnapshotRequest(args *structs.SnapshotRequest, in io.Re
// Verify token is allowed to operate on snapshots. There's only a
// single ACL sense here (not read and write) since reading gets you
// all the ACLs and you could escalate from there.
if rule, err := s.resolveToken(args.Token); err != nil {
if rule, err := s.ResolveToken(args.Token); err != nil {
return nil, err
} else if rule != nil && !rule.Snapshot() {
return nil, acl.ErrPermissionDenied

1
agent/consul/snapshot_endpoint_test.go
View File

@ -250,6 +250,7 @@ func TestSnapshot_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})

903
agent/consul/state/acl.go
View File

File diff suppressed because it is too large Load Diff

533
agent/consul/state/acl_test.go
View File

@ -1,299 +1,110 @@
package state
import (
"reflect"
// "reflect"
"testing"
"time"
"github.com/hashicorp/consul/acl"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/go-memdb"
"github.com/pascaldekloe/goe/verify"
// "github.com/hashicorp/go-memdb"
// "github.com/pascaldekloe/goe/verify"
"github.com/stretchr/testify/require"
)
func TestStateStore_ACLBootstrap(t *testing.T) {
acl1 := &structs.ACL{
ID: "03f43a07-7e78-1f72-6c72-5a4e3b1ac3df",
Type: structs.ACLTypeManagement,
func setupGlobalManagement(t *testing.T, s *Store) {
policy := structs.ACLPolicy{
ID: structs.ACLPolicyGlobalManagementID,
Name: "global-management",
Description: "Builtin Policy that grants unlimited access",
Rules: structs.ACLPolicyGlobalManagement,
Syntax: acl.SyntaxCurrent,
}
acl2 := &structs.ACL{
ID: "0546a993-aa7a-741e-fb7f-09159ae56ec1",
Type: structs.ACLTypeManagement,
}
setup := func() *Store {
s := testStateStore(t)
// The clean state store should initially have no bootstrap record.
bs, err := s.ACLGetBootstrap()
if err != nil {
t.Fatalf("err: %v", err)
}
if bs != nil {
t.Fatalf("bad: %#v", bs)
}
// Make sure that a bootstrap attempt fails in this state.
if err := s.ACLBootstrap(1, acl1); err != structs.ACLBootstrapNotInitializedErr {
t.Fatalf("err: %v", err)
}
_, gotA, err := s.ACLList(nil)
if err != nil {
t.Fatalf("err: %v", err)
}
verify.Values(t, "", gotA, structs.ACLs{})
// Initialize bootstrapping.
enabled, err := s.ACLBootstrapInit(2)
if err != nil {
t.Fatalf("err: %v", err)
}
if !enabled {
t.Fatalf("bad")
}
// Read it back.
gotB, err := s.ACLGetBootstrap()
if err != nil {
t.Fatalf("err: %v", err)
}
wantB := &structs.ACLBootstrap{
AllowBootstrap: true,
RaftIndex: structs.RaftIndex{
CreateIndex: 2,
ModifyIndex: 2,
},
}
verify.Values(t, "", gotB, wantB)
return s
}
// This is the bootstrap happy path.
t.Run("bootstrap", func(t *testing.T) {
s := setup()
// Perform a regular bootstrap.
if err := s.ACLBootstrap(3, acl1); err != nil {
t.Fatalf("err: %v", err)
}
// Read it back.
gotB, err := s.ACLGetBootstrap()
if err != nil {
t.Fatalf("err: %v", err)
}
wantB := &structs.ACLBootstrap{
AllowBootstrap: false,
RaftIndex: structs.RaftIndex{
CreateIndex: 2,
ModifyIndex: 3,
},
}
verify.Values(t, "", gotB, wantB)
// Make sure another attempt fails.
if err := s.ACLBootstrap(4, acl2); err != structs.ACLBootstrapNotAllowedErr {
t.Fatalf("err: %v", err)
}
// Check that the bootstrap state remains the same.
gotB, err = s.ACLGetBootstrap()
if err != nil {
t.Fatalf("err: %v", err)
}
verify.Values(t, "", gotB, wantB)
// Make sure the ACLs are in an expected state.
_, gotA, err := s.ACLList(nil)
if err != nil {
t.Fatalf("err: %v", err)
}
wantA := structs.ACLs{
&structs.ACL{
ID: acl1.ID,
Type: acl1.Type,
RaftIndex: structs.RaftIndex{
CreateIndex: 3,
ModifyIndex: 3,
},
},
}
verify.Values(t, "", gotA, wantA)
})
// This case initialized bootstrap but it gets canceled because a
// management token gets created manually.
t.Run("bootstrap canceled", func(t *testing.T) {
s := setup()
// Make a management token manually.
if err := s.ACLSet(3, acl1); err != nil {
t.Fatalf("err: %v", err)
}
// Bootstrapping should have gotten disabled.
gotB, err := s.ACLGetBootstrap()
if err != nil {
t.Fatalf("err: %v", err)
}
wantB := &structs.ACLBootstrap{
AllowBootstrap: false,
RaftIndex: structs.RaftIndex{
CreateIndex: 2,
ModifyIndex: 3,
},
}
verify.Values(t, "", gotB, wantB)
// Make sure another attempt fails.
if err := s.ACLBootstrap(4, acl2); err != structs.ACLBootstrapNotAllowedErr {
t.Fatalf("err: %v", err)
}
// Check that the bootstrap state remains the same.
gotB, err = s.ACLGetBootstrap()
if err != nil {
t.Fatalf("err: %v", err)
}
verify.Values(t, "", gotB, wantB)
// Make sure the ACLs are in an expected state.
_, gotA, err := s.ACLList(nil)
if err != nil {
t.Fatalf("err: %v", err)
}
wantA := structs.ACLs{
&structs.ACL{
ID: acl1.ID,
Type: acl1.Type,
RaftIndex: structs.RaftIndex{
CreateIndex: 3,
ModifyIndex: 3,
},
},
}
verify.Values(t, "", gotA, wantA)
})
policy.SetHash(true)
require.NoError(t, s.ACLPolicySet(1, &policy))
}
func TestStateStore_ACLBootstrap_InitialTokens(t *testing.T) {
acl1 := &structs.ACL{
ID: "03f43a07-7e78-1f72-6c72-5a4e3b1ac3df",
Type: structs.ACLTypeManagement,
func TestStateStore_ACLBootstrap(t *testing.T) {
token1 := &structs.ACLToken{
AccessorID: "30fca056-9fbb-4455-b94a-bf0e2bc575d6",
SecretID: "cbe1c6fd-d865-4034-9d6d-64fef7fb46a9",
Description: "Bootstrap Token (Global Management)",
Policies: []structs.ACLTokenPolicyLink{
{
ID: structs.ACLPolicyGlobalManagementID,
},
},
CreateTime: time.Now(),
Local: false,
// DEPRECATED (ACL-Legacy-Compat) - This is used so that the bootstrap token is still visible via the v1 acl APIs
Type: structs.ACLTokenTypeManagement,
}
acl2 := &structs.ACL{
ID: "0546a993-aa7a-741e-fb7f-09159ae56ec1",
Type: structs.ACLTypeManagement,
token2 := &structs.ACLToken{
AccessorID: "fd5c17fa-1503-4422-a424-dd44cdf35919",
SecretID: "7fd776b1-ded1-4d15-931b-db4770fc2317",
Description: "Bootstrap Token (Global Management)",
Policies: []structs.ACLTokenPolicyLink{
{
ID: structs.ACLPolicyGlobalManagementID,
},
},
CreateTime: time.Now(),
Local: false,
// DEPRECATED (ACL-Legacy-Compat) - This is used so that the bootstrap token is still visible via the v1 acl APIs
Type: structs.ACLTokenTypeManagement,
}
s := testStateStore(t)
setupGlobalManagement(t, s)
// Make a management token manually. This also makes sure that it's ok
// to set a token if bootstrap has not been initialized.
if err := s.ACLSet(1, acl1); err != nil {
t.Fatalf("err: %v", err)
}
canBootstrap, index, err := s.CanBootstrapACLToken()
require.NoError(t, err)
require.True(t, canBootstrap)
require.Equal(t, uint64(0), index)
// Initialize bootstrapping, which should not be enabled since an
// existing token is present.
enabled, err := s.ACLBootstrapInit(2)
if err != nil {
t.Fatalf("err: %v", err)
}
if enabled {
t.Fatalf("bad")
}
// Perform a regular bootstrap.
require.NoError(t, s.ACLBootstrap(3, 0, token1, false))
// Read it back.
gotB, err := s.ACLGetBootstrap()
if err != nil {
t.Fatalf("err: %v", err)
}
wantB := &structs.ACLBootstrap{
AllowBootstrap: false,
RaftIndex: structs.RaftIndex{
CreateIndex: 2,
ModifyIndex: 2,
},
}
verify.Values(t, "", gotB, wantB)
// Make sure we can't bootstrap again
canBootstrap, index, err = s.CanBootstrapACLToken()
require.NoError(t, err)
require.False(t, canBootstrap)
require.Equal(t, uint64(3), index)
// Make sure an attempt fails.
if err := s.ACLBootstrap(3, acl2); err != structs.ACLBootstrapNotAllowedErr {
t.Fatalf("err: %v", err)
}
// Make sure another attempt fails.
err = s.ACLBootstrap(4, 0, token2, false)
require.Error(t, err)
require.Equal(t, structs.ACLBootstrapNotAllowedErr, err)
// Check that the bootstrap state remains the same.
gotB, err = s.ACLGetBootstrap()
if err != nil {
t.Fatalf("err: %v", err)
}
verify.Values(t, "", gotB, wantB)
canBootstrap, index, err = s.CanBootstrapACLToken()
require.NoError(t, err)
require.False(t, canBootstrap)
require.Equal(t, uint64(3), index)
// Make sure the ACLs are in an expected state.
_, gotA, err := s.ACLList(nil)
if err != nil {
t.Fatalf("err: %v", err)
}
wantA := structs.ACLs{
&structs.ACL{
ID: acl1.ID,
Type: acl1.Type,
RaftIndex: structs.RaftIndex{
CreateIndex: 1,
ModifyIndex: 1,
},
},
}
verify.Values(t, "", gotA, wantA)
_, tokens, err := s.ACLTokenList(nil, true, true, "")
require.NoError(t, err)
require.Len(t, tokens, 1)
require.Equal(t, token1, tokens[0])
// bootstrap reset
err = s.ACLBootstrap(32, index-1, token2, false)
require.Error(t, err)
require.Equal(t, structs.ACLBootstrapInvalidResetIndexErr, err)
// bootstrap reset
err = s.ACLBootstrap(32, index, token2, false)
require.NoError(t, err)
_, tokens, err = s.ACLTokenList(nil, true, true, "")
require.NoError(t, err)
require.Len(t, tokens, 2)
}
func TestStateStore_ACLBootstrap_Snapshot_Restore(t *testing.T) {
s := testStateStore(t)
enabled, err := s.ACLBootstrapInit(1)
if err != nil {
t.Fatalf("err: %v", err)
}
if !enabled {
t.Fatalf("bad")
}
gotB, err := s.ACLGetBootstrap()
if err != nil {
t.Fatalf("err: %v", err)
}
wantB := &structs.ACLBootstrap{
AllowBootstrap: true,
RaftIndex: structs.RaftIndex{
CreateIndex: 1,
ModifyIndex: 1,
},
}
verify.Values(t, "", gotB, wantB)
snap := s.Snapshot()
defer snap.Close()
bs, err := snap.ACLBootstrap()
if err != nil {
t.Fatalf("err: %v", err)
}
verify.Values(t, "", bs, wantB)
r := testStateStore(t)
restore := r.Restore()
if err := restore.ACLBootstrap(bs); err != nil {
t.Fatalf("err: %v", err)
}
restore.Commit()
gotB, err = r.ACLGetBootstrap()
if err != nil {
t.Fatalf("err: %v", err)
}
verify.Values(t, "", gotB, wantB)
}
/*
func TestStateStore_ACLSet_ACLGet(t *testing.T) {
s := testStateStore(t)
@ -322,7 +133,7 @@ func TestStateStore_ACLSet_ACLGet(t *testing.T) {
acl := &structs.ACL{
ID: "acl1",
Name: "First ACL",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: "rules1",
}
if err := s.ACLSet(1, acl); err != nil {
@ -351,7 +162,7 @@ func TestStateStore_ACLSet_ACLGet(t *testing.T) {
expect := &structs.ACL{
ID: "acl1",
Name: "First ACL",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: "rules1",
RaftIndex: structs.RaftIndex{
CreateIndex: 1,
@ -366,7 +177,7 @@ func TestStateStore_ACLSet_ACLGet(t *testing.T) {
acl = &structs.ACL{
ID: "acl1",
Name: "First ACL",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: "rules2",
}
if err := s.ACLSet(2, acl); err != nil {
@ -385,7 +196,7 @@ func TestStateStore_ACLSet_ACLGet(t *testing.T) {
expect = &structs.ACL{
ID: "acl1",
Name: "First ACL",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: "rules2",
RaftIndex: structs.RaftIndex{
CreateIndex: 1,
@ -411,7 +222,7 @@ func TestStateStore_ACLList(t *testing.T) {
acls := structs.ACLs{
&structs.ACL{
ID: "acl1",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: "rules1",
RaftIndex: structs.RaftIndex{
CreateIndex: 1,
@ -420,7 +231,7 @@ func TestStateStore_ACLList(t *testing.T) {
},
&structs.ACL{
ID: "acl2",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: "rules2",
RaftIndex: structs.RaftIndex{
CreateIndex: 2,
@ -490,88 +301,144 @@ func TestStateStore_ACLDelete(t *testing.T) {
t.Fatalf("expected nil, got: %#v", result)
}
}
*/
func TestStateStore_ACL_Snapshot_Restore(t *testing.T) {
func TestStateStore_ACLTokens_Snapshot_Restore(t *testing.T) {
s := testStateStore(t)
// Insert some ACLs.
acls := structs.ACLs{
&structs.ACL{
ID: "acl1",
Type: structs.ACLTypeClient,
Rules: "rules1",
RaftIndex: structs.RaftIndex{
CreateIndex: 1,
ModifyIndex: 1,
tokens := structs.ACLTokens{
&structs.ACLToken{
AccessorID: "68016c3d-835b-450c-a6f9-75db9ba740be",
SecretID: "838f72b5-5c15-4a9e-aa6d-31734c3a0286",
Description: "token1",
Policies: []structs.ACLTokenPolicyLink{
structs.ACLTokenPolicyLink{
ID: "ca1fc52c-3676-4050-82ed-ca223e38b2c9",
Name: "policy1",
},
structs.ACLTokenPolicyLink{
ID: "7b70fa0f-58cd-412d-93c3-a0f17bb19a3e",
Name: "policy2",
},
},
Hash: []byte{1, 2, 3, 4},
RaftIndex: structs.RaftIndex{CreateIndex: 1, ModifyIndex: 2},
},
&structs.ACL{
ID: "acl2",
Type: structs.ACLTypeClient,
Rules: "rules2",
RaftIndex: structs.RaftIndex{
CreateIndex: 2,
ModifyIndex: 2,
&structs.ACLToken{
AccessorID: "b2125a1b-2a52-41d4-88f3-c58761998a46",
SecretID: "ba5d9239-a4ab-49b9-ae09-1f19eed92204",
Description: "token2",
Policies: []structs.ACLTokenPolicyLink{
structs.ACLTokenPolicyLink{
ID: "ca1fc52c-3676-4050-82ed-ca223e38b2c9",
Name: "policy1",
},
structs.ACLTokenPolicyLink{
ID: "7b70fa0f-58cd-412d-93c3-a0f17bb19a3e",
Name: "policy2",
},
},
Hash: []byte{1, 2, 3, 4},
RaftIndex: structs.RaftIndex{CreateIndex: 1, ModifyIndex: 2},
},
}
for _, acl := range acls {
if err := s.ACLSet(acl.ModifyIndex, acl); err != nil {
t.Fatalf("err: %s", err)
}
}
require.NoError(t, s.ACLTokensUpsert(2, tokens, true))
// Snapshot the ACLs.
snap := s.Snapshot()
defer snap.Close()
// Alter the real state store.
if err := s.ACLDelete(3, "acl1"); err != nil {
t.Fatalf("err: %s", err)
}
require.NoError(t, s.ACLTokenDeleteAccessor(3, tokens[0].AccessorID))
// Verify the snapshot.
if idx := snap.LastIndex(); idx != 2 {
t.Fatalf("bad index: %d", idx)
}
iter, err := snap.ACLs()
if err != nil {
t.Fatalf("err: %s", err)
}
var dump structs.ACLs
for acl := iter.Next(); acl != nil; acl = iter.Next() {
dump = append(dump, acl.(*structs.ACL))
}
if !reflect.DeepEqual(dump, acls) {
t.Fatalf("bad: %#v", dump)
require.Equal(t, uint64(2), snap.LastIndex())
iter, err := snap.ACLTokens()
require.NoError(t, err)
var dump structs.ACLTokens
for token := iter.Next(); token != nil; token = iter.Next() {
dump = append(dump, token.(*structs.ACLToken))
}
require.ElementsMatch(t, dump, tokens)
// Restore the values into a new state store.
func() {
s := testStateStore(t)
restore := s.Restore()
for _, acl := range dump {
if err := restore.ACL(acl); err != nil {
t.Fatalf("err: %s", err)
}
for _, token := range dump {
require.NoError(t, restore.ACLToken(token))
}
restore.Commit()
// Read the restored ACLs back out and verify that they match.
idx, res, err := s.ACLList(nil)
if err != nil {
t.Fatalf("err: %s", err)
}
if idx != 2 {
t.Fatalf("bad index: %d", idx)
}
if !reflect.DeepEqual(res, acls) {
t.Fatalf("bad: %#v", res)
}
// Check that the index was updated.
if idx := s.maxIndex("acls"); idx != 2 {
t.Fatalf("bad index: %d", idx)
}
idx, res, err := s.ACLTokenList(nil, true, true, "")
require.NoError(t, err)
require.Equal(t, uint64(2), idx)
require.ElementsMatch(t, tokens, res)
require.Equal(t, uint64(2), s.maxIndex("acl-tokens"))
}()
}
func TestStateStore_ACLPolicies_Snapshot_Restore(t *testing.T) {
s := testStateStore(t)
policies := structs.ACLPolicies{
&structs.ACLPolicy{
ID: "68016c3d-835b-450c-a6f9-75db9ba740be",
Name: "838f72b5-5c15-4a9e-aa6d-31734c3a0286",
Description: "policy1",
Rules: `acl = "read"`,
Hash: []byte{1, 2, 3, 4},
RaftIndex: structs.RaftIndex{CreateIndex: 1, ModifyIndex: 2},
},
&structs.ACLPolicy{
ID: "b2125a1b-2a52-41d4-88f3-c58761998a46",
Name: "ba5d9239-a4ab-49b9-ae09-1f19eed92204",
Description: "policy2",
Rules: `operator = "read"`,
Hash: []byte{1, 2, 3, 4},
RaftIndex: structs.RaftIndex{CreateIndex: 1, ModifyIndex: 2},
},
}
require.NoError(t, s.ACLPoliciesUpsert(2, policies))
// Snapshot the ACLs.
snap := s.Snapshot()
defer snap.Close()
// Alter the real state store.
require.NoError(t, s.ACLPolicyDeleteByID(3, policies[0].ID))
// Verify the snapshot.
require.Equal(t, uint64(2), snap.LastIndex())
iter, err := snap.ACLPolicies()
require.NoError(t, err)
var dump structs.ACLPolicies
for policy := iter.Next(); policy != nil; policy = iter.Next() {
dump = append(dump, policy.(*structs.ACLPolicy))
}
require.ElementsMatch(t, dump, policies)
// Restore the values into a new state store.
func() {
s := testStateStore(t)
restore := s.Restore()
for _, policy := range dump {
require.NoError(t, restore.ACLPolicy(policy))
}
restore.Commit()
// Read the restored ACLs back out and verify that they match.
idx, res, err := s.ACLPolicyList(nil, "")
require.NoError(t, err)
require.Equal(t, uint64(2), idx)
require.ElementsMatch(t, policies, res)
require.Equal(t, uint64(2), s.maxIndex("acl-policies"))
}()
}

34
agent/consul/state/state_store.go
View File

@ -21,9 +21,21 @@ var (
// is attempted with an empty session ID.
ErrMissingSessionID = errors.New("Missing session ID")
// ErrMissingACLID is returned when an ACL set is called on
// an ACL with an empty ID.
ErrMissingACLID = errors.New("Missing ACL ID")
// ErrMissingACLTokenSecret is returned when an token set is called on
// an token with an empty SecretID.
ErrMissingACLTokenSecret = errors.New("Missing ACL Token SecretID")
// ErrMissingACLTokenAccessor is returned when an token set is called on
// an token with an empty AccessorID.
ErrMissingACLTokenAccessor = errors.New("Missing ACL Token AccessorID")
// ErrMissingACLPolicyID is returned when an policy set is called on
// an policy with an empty ID.
ErrMissingACLPolicyID = errors.New("Missing ACL Policy ID")
// ErrMissingACLPolicyName is returned when an policy set is called on
// an policy with an empty Name.
ErrMissingACLPolicyName = errors.New("Missing ACL Policy Name")
// ErrMissingQueryID is returned when a Query set is called on
// a Query with an empty ID.
@ -138,6 +150,22 @@ func (s *Snapshot) LastIndex() uint64 {
return s.lastIndex
}
func (s *Snapshot) Indexes() (memdb.ResultIterator, error) {
iter, err := s.tx.Get("index", "id")
if err != nil {
return nil, err
}
return iter, nil
}
// IndexRestore is used to restore an index
func (s *Restore) IndexRestore(idx *IndexEntry) error {
if err := s.tx.Insert("index", idx); err != nil {
return fmt.Errorf("index insert failed: %v", err)
}
return nil
}
// Close performs cleanup of a state snapshot.
func (s *Snapshot) Close() {
s.tx.Abort()

20
agent/consul/txn_endpoint.go
View File

@ -16,13 +16,13 @@ type Txn struct {
// preCheck is used to verify the incoming operations before any further
// processing takes place. This checks things like ACLs.
func (t *Txn) preCheck(acl acl.ACL, ops structs.TxnOps) structs.TxnErrors {
func (t *Txn) preCheck(authorizer acl.Authorizer, ops structs.TxnOps) structs.TxnErrors {
var errors structs.TxnErrors
// Perform the pre-apply checks for any KV operations.
for i, op := range ops {
if op.KV != nil {
ok, err := kvsPreApply(t.srv, acl, op.KV.Verb, &op.KV.DirEnt)
ok, err := kvsPreApply(t.srv, authorizer, op.KV.Verb, &op.KV.DirEnt)
if err != nil {
errors = append(errors, &structs.TxnError{
OpIndex: i,
@ -49,11 +49,11 @@ func (t *Txn) Apply(args *structs.TxnRequest, reply *structs.TxnResponse) error
defer metrics.MeasureSince([]string{"txn", "apply"}, time.Now())
// Run the pre-checks before we send the transaction into Raft.
acl, err := t.srv.resolveToken(args.Token)
authorizer, err := t.srv.ResolveToken(args.Token)
if err != nil {
return err
}
reply.Errors = t.preCheck(acl, args.Ops)
reply.Errors = t.preCheck(authorizer, args.Ops)
if len(reply.Errors) > 0 {
return nil
}
@ -71,8 +71,8 @@ func (t *Txn) Apply(args *structs.TxnRequest, reply *structs.TxnResponse) error
// Convert the return type. This should be a cheap copy since we are
// just taking the two slices.
if txnResp, ok := resp.(structs.TxnResponse); ok {
if acl != nil {
txnResp.Results = FilterTxnResults(acl, txnResp.Results)
if authorizer != nil {
txnResp.Results = FilterTxnResults(authorizer, txnResp.Results)
}
*reply = txnResp
} else {
@ -100,11 +100,11 @@ func (t *Txn) Read(args *structs.TxnReadRequest, reply *structs.TxnReadResponse)
}
// Run the pre-checks before we perform the read.
acl, err := t.srv.resolveToken(args.Token)
authorizer, err := t.srv.ResolveToken(args.Token)
if err != nil {
return err
}
reply.Errors = t.preCheck(acl, args.Ops)
reply.Errors = t.preCheck(authorizer, args.Ops)
if len(reply.Errors) > 0 {
return nil
}
@ -112,8 +112,8 @@ func (t *Txn) Read(args *structs.TxnReadRequest, reply *structs.TxnReadResponse)
// Run the read transaction.
state := t.srv.fsm.State()
reply.Results, reply.Errors = state.TxnRO(args.Ops)
if acl != nil {
reply.Results = FilterTxnResults(acl, reply.Results)
if authorizer != nil {
reply.Results = FilterTxnResults(authorizer, reply.Results)
}
return nil
}

6
agent/consul/txn_endpoint_test.go
View File

@ -158,6 +158,7 @@ func TestTxn_Apply_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -184,7 +185,7 @@ func TestTxn_Apply_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: testListRules,
},
WriteRequest: structs.WriteRequest{Token: "root"},
@ -480,6 +481,7 @@ func TestTxn_Read_ACLDeny(t *testing.T) {
t.Parallel()
dir1, s1 := testServerWithConfig(t, func(c *Config) {
c.ACLDatacenter = "dc1"
c.ACLsEnabled = true
c.ACLMasterToken = "root"
c.ACLDefaultPolicy = "deny"
})
@ -508,7 +510,7 @@ func TestTxn_Read_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: testListRules,
},
WriteRequest: structs.WriteRequest{Token: "root"},

36
agent/consul/util.go
View File

@ -8,6 +8,7 @@ import (
"strconv"
"github.com/hashicorp/consul/agent/metadata"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/go-version"
"github.com/hashicorp/serf/serf"
)
@ -282,3 +283,38 @@ func ServersMeetMinimumVersion(members []serf.Member, minVersion *version.Versio
return true
}
func ServersGetACLMode(members []serf.Member, leader string, datacenter string) (mode structs.ACLMode, leaderMode structs.ACLMode) {
mode = structs.ACLModeEnabled
leaderMode = structs.ACLModeDisabled
for _, member := range members {
if valid, parts := metadata.IsConsulServer(member); valid {
if datacenter != "" && parts.Datacenter != datacenter {
continue
}
if memberAddr := (&net.TCPAddr{IP: member.Addr, Port: parts.Port}).String(); memberAddr == leader {
leaderMode = parts.ACLs
}
switch parts.ACLs {
case structs.ACLModeDisabled:
// anything disabled means we cant enable ACLs
mode = structs.ACLModeDisabled
case structs.ACLModeEnabled:
// do nothing
case structs.ACLModeLegacy:
// This covers legacy mode and older server versions that don't advertise ACL support
if mode != structs.ACLModeDisabled && mode != structs.ACLModeUnknown {
mode = structs.ACLModeLegacy
}
default:
if mode != structs.ACLModeDisabled {
mode = structs.ACLModeUnknown
}
}
}
}
return
}

2
agent/event_endpoint_test.go
View File

@ -70,7 +70,7 @@ func TestEventFire_token(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: testEventPolicy,
},
WriteRequest: structs.WriteRequest{Token: "root"},

4
agent/http.go
View File

@ -260,7 +260,7 @@ func (s *HTTPServer) nodeName() string {
// And then the loop that looks for parameters called "token" does the last
// step to get to the final redacted form.
var (
aclEndpointRE = regexp.MustCompile("^(/v1/acl/[^/]+/)([^?]+)([?]?.*)$")
aclEndpointRE = regexp.MustCompile("^(/v1/acl/(create|update|destroy|info|clone|list)/)([^?]+)([?]?.*)$")
)
// wrap is used to wrap functions to make them more convenient
@ -286,7 +286,7 @@ func (s *HTTPServer) wrap(handler endpoint, methods []string) http.HandlerFunc {
logURL = strings.Replace(logURL, token, "<hidden>", -1)
}
}
logURL = aclEndpointRE.ReplaceAllString(logURL, "$1<hidden>$3")
logURL = aclEndpointRE.ReplaceAllString(logURL, "$1<hidden>$4")
if s.blacklist.Block(req.URL.Path) {
errMsg := "Endpoint is blocked by agent configuration"

9
agent/http_oss.go
View File

@ -11,6 +11,15 @@ func init() {
registerEndpoint("/v1/acl/clone/", []string{"PUT"}, (*HTTPServer).ACLClone)
registerEndpoint("/v1/acl/list", []string{"GET"}, (*HTTPServer).ACLList)
registerEndpoint("/v1/acl/replication", []string{"GET"}, (*HTTPServer).ACLReplicationStatus)
registerEndpoint("/v1/acl/policies", []string{"GET"}, (*HTTPServer).ACLPolicyList)
registerEndpoint("/v1/acl/policy", []string{"PUT"}, (*HTTPServer).ACLPolicyCreate)
registerEndpoint("/v1/acl/policy/", []string{"GET", "PUT", "DELETE"}, (*HTTPServer).ACLPolicyCRUD)
registerEndpoint("/v1/acl/rules/translate", []string{"POST"}, (*HTTPServer).ACLRulesTranslate)
registerEndpoint("/v1/acl/rules/translate/", []string{"GET"}, (*HTTPServer).ACLRulesTranslateLegacyToken)
registerEndpoint("/v1/acl/tokens", []string{"GET"}, (*HTTPServer).ACLTokenList)
registerEndpoint("/v1/acl/token", []string{"PUT"}, (*HTTPServer).ACLTokenCreate)
registerEndpoint("/v1/acl/token/self", []string{"GET"}, (*HTTPServer).ACLTokenSelf)
registerEndpoint("/v1/acl/token/", []string{"GET", "PUT", "DELETE"}, (*HTTPServer).ACLTokenCRUD)
registerEndpoint("/v1/agent/token/", []string{"PUT"}, (*HTTPServer).AgentToken)
registerEndpoint("/v1/agent/self", []string{"GET"}, (*HTTPServer).AgentSelf)
registerEndpoint("/v1/agent/host", []string{"GET"}, (*HTTPServer).AgentHost)

4
agent/local/state_test.go
View File

@ -782,7 +782,7 @@ func TestAgentAntiEntropy_Services_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: testRegisterRules,
},
WriteRequest: structs.WriteRequest{
@ -1129,7 +1129,7 @@ func TestAgentAntiEntropy_Checks_ACLDeny(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: testRegisterRules,
},
WriteRequest: structs.WriteRequest{

10
agent/metadata/server.go
View File

@ -7,6 +7,7 @@ import (
"strconv"
"strings"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/go-version"
"github.com/hashicorp/serf/serf"
)
@ -39,6 +40,7 @@ type Server struct {
Addr net.Addr
Status serf.MemberStatus
NonVoter bool
ACLs structs.ACLMode
// If true, use TLS when connecting to this server
UseTLS bool
@ -92,6 +94,13 @@ func IsConsulServer(m serf.Member) (bool, *Server) {
return false, nil
}
var acls structs.ACLMode
if aclMode, ok := m.Tags["acls"]; ok {
acls = structs.ACLMode(aclMode)
} else {
acls = structs.ACLModeUnknown
}
segmentAddrs := make(map[string]string)
segmentPorts := make(map[string]int)
for name, value := range m.Tags {
@ -163,6 +172,7 @@ func IsConsulServer(m serf.Member) (bool, *Server) {
Status: m.Status,
UseTLS: useTLS,
NonVoter: nonVoter,
ACLs: acls,
}
return true, parts
}

741
agent/structs/acl.go
View File

@ -1,155 +1,698 @@
package structs
import (
"encoding/binary"
"errors"
"fmt"
"hash/fnv"
"sort"
"strings"
"time"
"github.com/hashicorp/consul/acl"
"github.com/hashicorp/consul/sentinel"
"golang.org/x/crypto/blake2b"
)
type ACLMode string
const (
// ACLs are disabled by configuration
ACLModeDisabled ACLMode = "0"
// ACLs are enabled
ACLModeEnabled ACLMode = "1"
// DEPRECATED (ACL-Legacy-Compat) - only needed while legacy ACLs are supported
// ACLs are enabled and using legacy ACLs
ACLModeLegacy ACLMode = "2"
// DEPRECATED (ACL-Legacy-Compat) - only needed while legacy ACLs are supported
// ACLs are assumed enabled but not being advertised
ACLModeUnknown ACLMode = "3"
)
// ACLOp is used in RPCs to encode ACL operations.
type ACLOp string
type ACLTokenIDType string
const (
// ACLBootstrapInit is used to perform a scan for existing tokens which
// will decide whether bootstrapping is allowed for a cluster. This is
// initiated by the leader when it steps up, if necessary.
ACLBootstrapInit = "bootstrap-init"
ACLTokenSecret ACLTokenIDType = "secret"
ACLTokenAccessor ACLTokenIDType = "accessor"
)
// ACLBootstrapNow is used to perform a one-time ACL bootstrap operation on
// a cluster to get the first management token.
ACLBootstrapNow = "bootstrap-now"
type ACLPolicyIDType string
const (
ACLPolicyName ACLPolicyIDType = "name"
ACLPolicyID ACLPolicyIDType = "id"
)
const (
// All policy ids with the first 120 bits set to all zeroes are
// reserved for builtin policies. Policy creation will ensure we
// dont accidentally create them when autogenerating uuids.
// This policy gives unlimited access to everything. Users
// may rename if desired but cannot delete or modify the rules
ACLPolicyGlobalManagementID = "00000000-0000-0000-0000-000000000001"
ACLPolicyGlobalManagement = `
acl = "write"
agent_prefix "" {
policy = "write"
}
event_prefix "" {
policy = "write"
}
key_prefix "" {
policy = "write"
}
keyring = "write"
node_prefix "" {
policy = "write"
}
operator = "write"
query_prefix "" {
policy = "write"
}
service_prefix "" {
policy = "write"
intentions = "write"
}
session_prefix "" {
policy = "write"
}`
// This is the policy ID for anonymous access. This is configurable by the
ACLTokenAnonymousID = "00000000-0000-0000-0000-000000000002"
)
func ACLIDReserved(id string) bool {
return strings.HasPrefix(id, "00000000-0000-0000-0000-0000000000")
}
const (
// ACLSet creates or updates a token.
ACLSet ACLOp = "set"
// ACLForceSet is deprecated, but left for backwards compatibility.
ACLForceSet = "force-set"
// ACLDelete deletes a token.
ACLDelete = "delete"
ACLDelete ACLOp = "delete"
)
// ACLBootstrapNotInitializedErr is returned when a bootstrap is attempted but
// we haven't yet initialized ACL bootstrap. It provides some guidance to
// operators on how to proceed.
var ACLBootstrapNotInitializedErr = errors.New("ACL bootstrap not initialized, need to force a leader election and ensure all Consul servers support this feature")
// ACLBootstrapNotAllowedErr is returned once we know that a bootstrap can no
// longer be done since the cluster was bootstrapped, or a management token
// was created manually.
// longer be done since the cluster was bootstrapped
var ACLBootstrapNotAllowedErr = errors.New("ACL bootstrap no longer allowed")
const (
// ACLTypeClient tokens have rules applied
ACLTypeClient = "client"
// ACLBootstrapInvalidResetIndexErr is returned when bootstrap is requested with a non-zero
// reset index but the index doesn't match the bootstrap index
var ACLBootstrapInvalidResetIndexErr = errors.New("Invalid ACL bootstrap reset index")
// ACLTypeManagement tokens have an always allow policy, so they can
// make other tokens and can access all resources.
ACLTypeManagement = "management"
)
type ACLIdentity interface {
// ID returns a string that can be used for logging and telemetry. This should not
// contain any secret data used for authentication
ID() string
SecretToken() string
PolicyIDs() []string
EmbeddedPolicy() *ACLPolicy
}
// ACL is used to represent a token and its rules
type ACL struct {
ID string
Name string
Type string
Rules string
type ACLTokenPolicyLink struct {
ID string
Name string `hash:"ignore"`
}
type ACLToken struct {
// This is the UUID used for tracking and management purposes
AccessorID string
// This is the UUID used as the api token by clients
SecretID string
// Human readable string to display for the token (Optional)
Description string
// List of policy links - nil/empty for legacy tokens
// Note this is the list of IDs and not the names. Prior to token creation
// the list of policy names gets validated and the policy IDs get stored herein
Policies []ACLTokenPolicyLink
// Type is the V1 Token Type
// DEPRECATED (ACL-Legacy-Compat) - remove once we no longer support v1 ACL compat
// Even though we are going to auto upgrade management tokens we still
// want to be able to have the old APIs operate on the upgraded management tokens
// so this field is being kept to identify legacy tokens even after an auto-upgrade
Type string `json:"-"`
// Rules is the V1 acl rules associated with
// DEPRECATED (ACL-Legacy-Compat) - remove once we no longer support v1 ACL compat
Rules string `json:",omitempty"`
// Whether this token is DC local. This means that it will not be synced
// to the ACL datacenter and replicated to others.
Local bool
// The time when this token was created
CreateTime time.Time `json:",omitempty"`
// Hash of the contents of the token
//
// This is needed mainly for replication purposes. When replicating from
// one DC to another keeping the content Hash will allow us to avoid
// unnecessary calls to the authoritative DC
Hash []byte
// Embedded Raft Metadata
RaftIndex
}
// ACLs is a slice of ACLs.
type ACLs []*ACL
func (t *ACLToken) ID() string {
return t.AccessorID
}
// IsSame checks if one ACL is the same as another, without looking
// at the Raft information (that's why we didn't call it IsEqual). This is
// useful for seeing if an update would be idempotent for all the functional
// parts of the structure.
func (a *ACL) IsSame(other *ACL) bool {
if a.ID != other.ID ||
a.Name != other.Name ||
a.Type != other.Type ||
a.Rules != other.Rules {
return false
func (t *ACLToken) SecretToken() string {
return t.SecretID
}
func (t *ACLToken) PolicyIDs() []string {
var ids []string
for _, link := range t.Policies {
ids = append(ids, link.ID)
}
return ids
}
func (t *ACLToken) EmbeddedPolicy() *ACLPolicy {
// DEPRECATED (ACL-Legacy-Compat)
//
// For legacy tokens with embedded rules this provides a way to map those
// rules to an ACLPolicy. This function can just return nil once legacy
// acl compatibility is no longer needed.
//
// Additionally for management tokens we must embed the policy rules
// as well
policy := &ACLPolicy{}
if t.Rules != "" || t.Type == ACLTokenTypeClient {
hasher := fnv.New128a()
policy.ID = fmt.Sprintf("%x", hasher.Sum([]byte(t.Rules)))
policy.Name = fmt.Sprintf("legacy-policy-%s", policy.ID)
policy.Rules = t.Rules
policy.Syntax = acl.SyntaxLegacy
} else if t.Type == ACLTokenTypeManagement {
hasher := fnv.New128a()
policy.ID = fmt.Sprintf("%x", hasher.Sum([]byte(ACLPolicyGlobalManagement)))
policy.Name = "legacy-management"
policy.Rules = ACLPolicyGlobalManagement
policy.Syntax = acl.SyntaxCurrent
} else {
return nil
}
return true
policy.SetHash(true)
return policy
}
// ACLBootstrap keeps track of whether bootstrapping ACLs is allowed for a
// cluster.
type ACLBootstrap struct {
// AllowBootstrap will only be true if no existing management tokens
// have been found.
AllowBootstrap bool
func (t *ACLToken) SetHash(force bool) []byte {
if force || t.Hash == nil {
// Initialize a 256bit Blake2 hash (32 bytes)
hash, err := blake2b.New256(nil)
if err != nil {
panic(err)
}
RaftIndex
// Write all the user set fields
hash.Write([]byte(t.Description))
hash.Write([]byte(t.Type))
hash.Write([]byte(t.Rules))
if t.Local {
hash.Write([]byte("local"))
} else {
hash.Write([]byte("global"))
}
for _, link := range t.Policies {
hash.Write([]byte(link.ID))
}
// Finalize the hash
hashVal := hash.Sum(nil)
// Set and return the hash
t.Hash = hashVal
}
return t.Hash
}
// ACLRequest is used to create, update or delete an ACL
type ACLRequest struct {
Datacenter string
Op ACLOp
ACL ACL
WriteRequest
func (t *ACLToken) EstimateSize() int {
// 33 = 16 (RaftIndex) + 8 (Hash) + 8 (CreateTime) + 1 (Local)
size := 33 + len(t.AccessorID) + len(t.SecretID) + len(t.Description) + len(t.Type) + len(t.Rules)
for _, link := range t.Policies {
size += len(link.ID) + len(link.Name)
}
return size
}
func (r *ACLRequest) RequestDatacenter() string {
return r.Datacenter
// ACLTokens is a slice of ACLTokens.
type ACLTokens []*ACLToken
type ACLTokenListStub struct {
AccessorID string
Description string
Policies []ACLTokenPolicyLink
Local bool
CreateTime time.Time `json:",omitempty"`
Hash []byte
CreateIndex uint64
ModifyIndex uint64
Legacy bool `json:",omitempty"`
}
// ACLRequests is a list of ACL change requests.
type ACLRequests []*ACLRequest
type ACLTokenListStubs []*ACLTokenListStub
// ACLSpecificRequest is used to request an ACL by ID
type ACLSpecificRequest struct {
Datacenter string
ACL string
QueryOptions
func (token *ACLToken) Stub() *ACLTokenListStub {
return &ACLTokenListStub{
AccessorID: token.AccessorID,
Description: token.Description,
Policies: token.Policies,
Local: token.Local,
CreateTime: token.CreateTime,
Hash: token.Hash,
CreateIndex: token.CreateIndex,
ModifyIndex: token.ModifyIndex,
Legacy: token.Rules != "",
}
}
// RequestDatacenter returns the DC this request is targeted to.
func (r *ACLSpecificRequest) RequestDatacenter() string {
return r.Datacenter
func (tokens ACLTokens) Sort() {
sort.Slice(tokens, func(i, j int) bool {
return tokens[i].AccessorID < tokens[j].AccessorID
})
}
// ACLPolicyRequest is used to request an ACL by ID, conditionally
// filtering on an ID
type ACLPolicyRequest struct {
Datacenter string
ACL string
ETag string
QueryOptions
func (tokens ACLTokenListStubs) Sort() {
sort.Slice(tokens, func(i, j int) bool {
return tokens[i].AccessorID < tokens[j].AccessorID
})
}
// RequestDatacenter returns the DC this request is targeted to.
func (r *ACLPolicyRequest) RequestDatacenter() string {
return r.Datacenter
}
// IndexedACLs has tokens along with the Raft metadata about them.
type IndexedACLs struct {
ACLs ACLs
QueryMeta
}
// ACLPolicy is a policy that can be associated with a token.
type ACLPolicy struct {
ETag string
Parent string
Policy *acl.Policy
TTL time.Duration
QueryMeta
// This is the internal UUID associated with the policy
ID string
// Unique name to reference the policy by.
// - Valid Characters: [a-zA-Z0-9-]
// - Valid Lengths: 1 - 128
Name string
// Human readable description (Optional)
Description string
// The rule set (using the updated rule syntax)
Rules string
// DEPRECATED (ACL-Legacy-Compat) - This is only needed while we support the legacy ACLS
Syntax acl.SyntaxVersion `json:"-"`
// Datacenters that the policy is valid within.
// - No wildcards allowed
// - If empty then the policy is valid within all datacenters
Datacenters []string `json:",omitempty"`
// Hash of the contents of the policy
// This does not take into account the ID (which is immutable)
// nor the raft metadata.
//
// This is needed mainly for replication purposes. When replicating from
// one DC to another keeping the content Hash will allow us to avoid
// unnecessary calls to the authoritative DC
Hash []byte
// Embedded Raft Metadata
RaftIndex `hash:"ignore"`
}
type ACLPolicyListStub struct {
ID string
Name string
Description string
Datacenters []string
Hash []byte
CreateIndex uint64
ModifyIndex uint64
}
func (p *ACLPolicy) Stub() *ACLPolicyListStub {
return &ACLPolicyListStub{
ID: p.ID,
Name: p.Name,
Description: p.Description,
Datacenters: p.Datacenters,
Hash: p.Hash,
CreateIndex: p.CreateIndex,
ModifyIndex: p.ModifyIndex,
}
}
type ACLPolicies []*ACLPolicy
type ACLPolicyListStubs []*ACLPolicyListStub
func (p *ACLPolicy) SetHash(force bool) []byte {
if force || p.Hash == nil {
// Initialize a 256bit Blake2 hash (32 bytes)
hash, err := blake2b.New256(nil)
if err != nil {
panic(err)
}
// Write all the user set fields
hash.Write([]byte(p.Name))
hash.Write([]byte(p.Description))
hash.Write([]byte(p.Rules))
for _, dc := range p.Datacenters {
hash.Write([]byte(dc))
}
// Finalize the hash
hashVal := hash.Sum(nil)
// Set and return the hash
p.Hash = hashVal
}
return p.Hash
}
func (p *ACLPolicy) EstimateSize() int {
// This is just an estimate. There is other data structure overhead
// pointers etc that this does not account for.
// 64 = 36 (uuid) + 16 (RaftIndex) + 8 (Hash) + 4 (Syntax)
size := 64 + len(p.Name) + len(p.Description) + len(p.Rules)
for _, dc := range p.Datacenters {
size += len(dc)
}
return size
}
// ACLPolicyListHash returns a consistent hash for a set of policies.
func (policies ACLPolicies) HashKey() string {
cacheKeyHash, err := blake2b.New256(nil)
if err != nil {
panic(err)
}
for _, policy := range policies {
cacheKeyHash.Write([]byte(policy.ID))
// including the modify index prevents a policy set from being
// cached if one of the policies has changed
binary.Write(cacheKeyHash, binary.BigEndian, policy.ModifyIndex)
}
return fmt.Sprintf("%x", cacheKeyHash.Sum(nil))
}
func (policies ACLPolicies) Sort() {
sort.Slice(policies, func(i, j int) bool {
return policies[i].ID < policies[j].ID
})
}
func (policies ACLPolicyListStubs) Sort() {
sort.Slice(policies, func(i, j int) bool {
return policies[i].ID < policies[j].ID
})
}
func (policies ACLPolicies) resolveWithCache(cache *ACLCaches, sentinel sentinel.Evaluator) ([]*acl.Policy, error) {
// Parse the policies
parsed := make([]*acl.Policy, 0, len(policies))
for _, policy := range policies {
policy.SetHash(false)
cacheKey := fmt.Sprintf("%x", policy.Hash)
cachedPolicy := cache.GetParsedPolicy(cacheKey)
if cachedPolicy != nil {
// policies are content hashed so no need to check the age
parsed = append(parsed, cachedPolicy.Policy)
continue
}
p, err := acl.NewPolicyFromSource(policy.ID, policy.ModifyIndex, policy.Rules, policy.Syntax, sentinel)
if err != nil {
return nil, fmt.Errorf("failed to parse %q: %v", policy.Name, err)
}
cache.PutParsedPolicy(cacheKey, p)
parsed = append(parsed, p)
}
return parsed, nil
}
func (policies ACLPolicies) Compile(parent acl.Authorizer, cache *ACLCaches, sentinel sentinel.Evaluator) (acl.Authorizer, error) {
// Determine the cache key
cacheKey := policies.HashKey()
entry := cache.GetAuthorizer(cacheKey)
if entry != nil {
// the hash key takes into account the policy contents. There is no reason to expire this cache or check its age.
return entry.Authorizer, nil
}
parsed, err := policies.resolveWithCache(cache, sentinel)
if err != nil {
return nil, fmt.Errorf("failed to parse the ACL policies: %v", err)
}
// Create the ACL object
authorizer, err := acl.NewPolicyAuthorizer(parent, parsed, sentinel)
if err != nil {
return nil, fmt.Errorf("failed to construct ACL Authorizer: %v", err)
}
// Update the cache
cache.PutAuthorizer(cacheKey, authorizer)
return authorizer, nil
}
func (policies ACLPolicies) Merge(cache *ACLCaches, sentinel sentinel.Evaluator) (*acl.Policy, error) {
parsed, err := policies.resolveWithCache(cache, sentinel)
if err != nil {
return nil, err
}
return acl.MergePolicies(parsed), nil
}
type ACLReplicationType string
const (
ACLReplicateLegacy ACLReplicationType = "legacy"
ACLReplicatePolicies ACLReplicationType = "policies"
ACLReplicateTokens ACLReplicationType = "tokens"
)
// ACLReplicationStatus provides information about the health of the ACL
// replication system.
type ACLReplicationStatus struct {
Enabled bool
Running bool
SourceDatacenter string
ReplicatedIndex uint64
LastSuccess time.Time
LastError time.Time
Enabled bool
Running bool
SourceDatacenter string
ReplicationType ACLReplicationType
ReplicatedIndex uint64
ReplicatedTokenIndex uint64
LastSuccess time.Time
LastError time.Time
}
// ACLTokenUpsertRequest is used for token creation and update operations
// at the RPC layer
type ACLTokenUpsertRequest struct {
ACLToken ACLToken // Token to manipulate - I really dislike this name but "Token" is taken in the WriteRequest
Datacenter string // The datacenter to perform the request within
WriteRequest
}
func (r *ACLTokenUpsertRequest) RequestDatacenter() string {
return r.Datacenter
}
// ACLTokenReadRequest is used for token read operations at the RPC layer
type ACLTokenReadRequest struct {
TokenID string // id used for the token lookup
TokenIDType ACLTokenIDType // The Type of ID used to lookup the token
Datacenter string // The datacenter to perform the request within
QueryOptions
}
func (r *ACLTokenReadRequest) RequestDatacenter() string {
return r.Datacenter
}
// ACLTokenDeleteRequest is used for token deletion operations at the RPC layer
type ACLTokenDeleteRequest struct {
TokenID string // ID of the token to delete
Datacenter string // The datacenter to perform the request within
WriteRequest
}
func (r *ACLTokenDeleteRequest) RequestDatacenter() string {
return r.Datacenter
}
// ACLTokenListRequest is used for token listing operations at the RPC layer
type ACLTokenListRequest struct {
IncludeLocal bool // Whether local tokens should be included
IncludeGlobal bool // Whether global tokens should be included
Policy string // Policy filter
Datacenter string // The datacenter to perform the request within
QueryOptions
}
func (r *ACLTokenListRequest) RequestDatacenter() string {
return r.Datacenter
}
// ACLTokenListResponse is used to return the secret data free stubs
// of the tokens
type ACLTokenListResponse struct {
Tokens ACLTokenListStubs
QueryMeta
}
// ACLTokenBatchReadRequest is used for reading multiple tokens, this is
// different from the the token list request in that only tokens with the
// the requested ids are returned
type ACLTokenBatchReadRequest struct {
AccessorIDs []string // List of accessor ids to fetch
Datacenter string // The datacenter to perform the request within
QueryOptions
}
func (r *ACLTokenBatchReadRequest) RequestDatacenter() string {
return r.Datacenter
}
// ACLTokenBatchUpsertRequest is used only at the Raft layer
// for batching multiple token creation/update operations
//
// This is particularly useful during token replication and during
// automatic legacy token upgrades.
type ACLTokenBatchUpsertRequest struct {
Tokens ACLTokens
AllowCreate bool
}
// ACLTokenBatchDeleteRequest is used only at the Raft layer
// for batching multiple token deletions.
//
// This is particularly useful during token replication when
// multiple tokens need to be removed from the local DCs state.
type ACLTokenBatchDeleteRequest struct {
TokenIDs []string // Tokens to delete
}
// ACLTokenBootstrapRequest is used only at the Raft layer
// for ACL bootstrapping
//
// The RPC layer will use a generic DCSpecificRequest to indicate
// that bootstrapping must be performed but the actual token
// and the resetIndex will be generated by that RPC endpoint
type ACLTokenBootstrapRequest struct {
Token ACLToken // Token to use for bootstrapping
ResetIndex uint64 // Reset index
}
// ACLTokenResponse returns a single Token + metadata
type ACLTokenResponse struct {
Token *ACLToken
QueryMeta
}
// ACLTokensResponse returns multiple Tokens associated with the same metadata
type ACLTokensResponse struct {
Tokens []*ACLToken
QueryMeta
}
// ACLPolicyUpsertRequest is used at the RPC layer for creation and update requests
type ACLPolicyUpsertRequest struct {
Policy ACLPolicy // The policy to upsert
Datacenter string // The datacenter to perform the request within
WriteRequest
}
func (r *ACLPolicyUpsertRequest) RequestDatacenter() string {
return r.Datacenter
}
// ACLPolicyDeleteRequest is used at the RPC layer deletion requests
type ACLPolicyDeleteRequest struct {
PolicyID string // The id of the policy to delete
Datacenter string // The datacenter to perform the request within
WriteRequest
}
func (r *ACLPolicyDeleteRequest) RequestDatacenter() string {
return r.Datacenter
}
// ACLPolicyReadRequest is used at the RPC layer to perform policy read operations
type ACLPolicyReadRequest struct {
PolicyID string // id used for the policy lookup
Datacenter string // The datacenter to perform the request within
QueryOptions
}
func (r *ACLPolicyReadRequest) RequestDatacenter() string {
return r.Datacenter
}
// ACLPolicyListRequest is used at the RPC layer to request a listing of policies
type ACLPolicyListRequest struct {
DCScope string
Datacenter string // The datacenter to perform the request within
QueryOptions
}
func (r *ACLPolicyListRequest) RequestDatacenter() string {
return r.Datacenter
}
type ACLPolicyListResponse struct {
Policies ACLPolicyListStubs
QueryMeta
}
// ACLPolicyBatchReadRequest is used at the RPC layer to request a subset of
// the policies associated with the token used for retrieval
type ACLPolicyBatchReadRequest struct {
PolicyIDs []string // List of policy ids to fetch
Datacenter string // The datacenter to perform the request within
QueryOptions
}
func (r *ACLPolicyBatchReadRequest) RequestDatacenter() string {
return r.Datacenter
}
// ACLPolicyResponse returns a single policy + metadata
type ACLPolicyResponse struct {
Policy *ACLPolicy
QueryMeta
}
type ACLPoliciesResponse struct {
Policies []*ACLPolicy
QueryMeta
}
// ACLPolicyBatchUpsertRequest is used at the Raft layer for batching
// multiple policy creations and updates
//
// This is particularly useful during replication
type ACLPolicyBatchUpsertRequest struct {
Policies ACLPolicies
}
// ACLPolicyBatchDeleteRequest is used at the Raft layer for batching
// multiple policy deletions
//
// This is particularly useful during replication
type ACLPolicyBatchDeleteRequest struct {
PolicyIDs []string
}

223
agent/structs/acl_cache.go Normal file
View File

@ -0,0 +1,223 @@
package structs
import (
"time"
"github.com/hashicorp/consul/acl"
"github.com/hashicorp/golang-lru"
)
type ACLCachesConfig struct {
Identities int
Policies int
ParsedPolicies int
Authorizers int
}
type ACLCaches struct {
identities *lru.TwoQueueCache // identity id -> structs.ACLIdentity
parsedPolicies *lru.TwoQueueCache // policy content hash -> acl.Policy
policies *lru.TwoQueueCache // policy ID -> ACLPolicy
authorizers *lru.TwoQueueCache // token secret -> acl.Authorizer
}
type IdentityCacheEntry struct {
Identity ACLIdentity
CacheTime time.Time
}
func (e *IdentityCacheEntry) Age() time.Duration {
return time.Since(e.CacheTime)
}
type ParsedPolicyCacheEntry struct {
Policy *acl.Policy
CacheTime time.Time
}
func (e *ParsedPolicyCacheEntry) Age() time.Duration {
return time.Since(e.CacheTime)
}
type PolicyCacheEntry struct {
Policy *ACLPolicy
CacheTime time.Time
}
func (e *PolicyCacheEntry) Age() time.Duration {
return time.Since(e.CacheTime)
}
type AuthorizerCacheEntry struct {
Authorizer acl.Authorizer
CacheTime time.Time
TTL time.Duration
}
func (e *AuthorizerCacheEntry) Age() time.Duration {
return time.Since(e.CacheTime)
}
func NewACLCaches(config *ACLCachesConfig) (*ACLCaches, error) {
cache := &ACLCaches{}
if config != nil && config.Identities > 0 {
identCache, err := lru.New2Q(config.Identities)
if err != nil {
return nil, err
}
cache.identities = identCache
}
if config != nil && config.Policies > 0 {
policyCache, err := lru.New2Q(config.Policies)
if err != nil {
return nil, err
}
cache.policies = policyCache
}
if config != nil && config.ParsedPolicies > 0 {
parsedCache, err := lru.New2Q(config.ParsedPolicies)
if err != nil {
return nil, err
}
cache.parsedPolicies = parsedCache
}
if config != nil && config.Authorizers > 0 {
authCache, err := lru.New2Q(config.Authorizers)
if err != nil {
return nil, err
}
cache.authorizers = authCache
}
return cache, nil
}
// GetIdentity fetches an identity from the cache and returns it
func (c *ACLCaches) GetIdentity(id string) *IdentityCacheEntry {
if c == nil || c.identities == nil {
return nil
}
if raw, ok := c.identities.Get(id); ok {
return raw.(*IdentityCacheEntry)
}
return nil
}
// GetPolicy fetches a policy from the cache and returns it
func (c *ACLCaches) GetPolicy(policyID string) *PolicyCacheEntry {
if c == nil || c.policies == nil {
return nil
}
if raw, ok := c.policies.Get(policyID); ok {
return raw.(*PolicyCacheEntry)
}
return nil
}
// GetPolicy fetches a policy from the cache and returns it
func (c *ACLCaches) GetParsedPolicy(id string) *ParsedPolicyCacheEntry {
if c == nil || c.parsedPolicies == nil {
return nil
}
if raw, ok := c.parsedPolicies.Get(id); ok {
return raw.(*ParsedPolicyCacheEntry)
}
return nil
}
// GetAuthorizer fetches a acl from the cache and returns it
func (c *ACLCaches) GetAuthorizer(id string) *AuthorizerCacheEntry {
if c == nil || c.authorizers == nil {
return nil
}
if raw, ok := c.authorizers.Get(id); ok {
return raw.(*AuthorizerCacheEntry)
}
return nil
}
// PutIdentity adds a new identity to the cache
func (c *ACLCaches) PutIdentity(id string, ident ACLIdentity) {
if c == nil || c.identities == nil {
return
}
c.identities.Add(id, &IdentityCacheEntry{Identity: ident, CacheTime: time.Now()})
}
func (c *ACLCaches) PutPolicy(policyId string, policy *ACLPolicy) {
if c == nil || c.policies == nil {
return
}
c.policies.Add(policyId, &PolicyCacheEntry{Policy: policy, CacheTime: time.Now()})
}
func (c *ACLCaches) PutParsedPolicy(id string, policy *acl.Policy) {
if c == nil || c.parsedPolicies == nil {
return
}
c.parsedPolicies.Add(id, &ParsedPolicyCacheEntry{Policy: policy, CacheTime: time.Now()})
}
func (c *ACLCaches) PutAuthorizer(id string, authorizer acl.Authorizer) {
if c == nil || c.authorizers == nil {
return
}
c.authorizers.Add(id, &AuthorizerCacheEntry{Authorizer: authorizer, CacheTime: time.Now()})
}
func (c *ACLCaches) PutAuthorizerWithTTL(id string, authorizer acl.Authorizer, ttl time.Duration) {
if c == nil || c.authorizers == nil {
return
}
c.authorizers.Add(id, &AuthorizerCacheEntry{Authorizer: authorizer, CacheTime: time.Now(), TTL: ttl})
}
func (c *ACLCaches) RemoveIdentity(id string) {
if c != nil && c.identities != nil {
c.identities.Remove(id)
}
}
func (c *ACLCaches) RemovePolicy(policyID string) {
if c != nil && c.policies != nil {
c.policies.Remove(policyID)
}
}
func (c *ACLCaches) Purge() {
if c != nil {
if c.identities != nil {
c.identities.Purge()
}
if c.policies != nil {
c.policies.Purge()
}
if c.parsedPolicies != nil {
c.parsedPolicies.Purge()
}
if c.authorizers != nil {
c.authorizers.Purge()
}
}
}

105
agent/structs/acl_cache_test.go Normal file
View File

@ -0,0 +1,105 @@
package structs
import (
"testing"
"github.com/hashicorp/consul/acl"
"github.com/stretchr/testify/require"
)
func TestStructs_ACLCaches(t *testing.T) {
t.Parallel()
t.Run("New", func(t *testing.T) {
t.Parallel()
t.Run("Valid Sizes", func(t *testing.T) {
t.Parallel()
// 1 isn't valid due to a bug in golang-lru library
config := ACLCachesConfig{2, 2, 2, 2}
cache, err := NewACLCaches(&config)
require.NoError(t, err)
require.NotNil(t, cache)
require.NotNil(t, cache.identities)
require.NotNil(t, cache.policies)
require.NotNil(t, cache.parsedPolicies)
require.NotNil(t, cache.authorizers)
})
t.Run("Zero Sizes", func(t *testing.T) {
t.Parallel()
// 1 isn't valid due to a bug in golang-lru library
config := ACLCachesConfig{0, 0, 0, 0}
cache, err := NewACLCaches(&config)
require.NoError(t, err)
require.NotNil(t, cache)
require.Nil(t, cache.identities)
require.Nil(t, cache.policies)
require.Nil(t, cache.parsedPolicies)
require.Nil(t, cache.authorizers)
})
})
t.Run("Identities", func(t *testing.T) {
t.Parallel()
// 1 isn't valid due to a bug in golang-lru library
config := ACLCachesConfig{Identities: 4}
cache, err := NewACLCaches(&config)
require.NoError(t, err)
require.NotNil(t, cache)
cache.PutIdentity("foo", &ACLToken{})
entry := cache.GetIdentity("foo")
require.NotNil(t, entry)
require.NotNil(t, entry.Identity)
})
t.Run("Policies", func(t *testing.T) {
t.Parallel()
// 1 isn't valid due to a bug in golang-lru library
config := ACLCachesConfig{Policies: 4}
cache, err := NewACLCaches(&config)
require.NoError(t, err)
require.NotNil(t, cache)
cache.PutPolicy("foo", &ACLPolicy{})
entry := cache.GetPolicy("foo")
require.NotNil(t, entry)
require.NotNil(t, entry.Policy)
})
t.Run("ParsedPolicies", func(t *testing.T) {
t.Parallel()
// 1 isn't valid due to a bug in golang-lru library
config := ACLCachesConfig{ParsedPolicies: 4}
cache, err := NewACLCaches(&config)
require.NoError(t, err)
require.NotNil(t, cache)
cache.PutParsedPolicy("foo", &acl.Policy{})
entry := cache.GetParsedPolicy("foo")
require.NotNil(t, entry)
require.NotNil(t, entry.Policy)
})
t.Run("Authorizers", func(t *testing.T) {
t.Parallel()
// 1 isn't valid due to a bug in golang-lru library
config := ACLCachesConfig{Authorizers: 4}
cache, err := NewACLCaches(&config)
require.NoError(t, err)
require.NotNil(t, cache)
cache.PutAuthorizer("foo", acl.DenyAll())
entry := cache.GetAuthorizer("foo")
require.NotNil(t, entry)
require.NotNil(t, entry.Authorizer)
require.True(t, entry.Authorizer == acl.DenyAll())
})
}

171
agent/structs/acl_legacy.go Normal file
View File

@ -0,0 +1,171 @@
// DEPRECATED (ACL-Legacy-Compat)
//
// Everything within this file is deprecated and related to the original ACL
// implementation. Once support for v1 ACLs are removed this whole file can
// be deleted.
package structs
import (
"errors"
"fmt"
"time"
"github.com/hashicorp/consul/acl"
)
const (
// ACLBootstrapInit is used to perform a scan for existing tokens which
// will decide whether bootstrapping is allowed for a cluster. This is
// initiated by the leader when it steps up, if necessary.
ACLBootstrapInit ACLOp = "bootstrap-init"
// ACLBootstrapNow is used to perform a one-time ACL bootstrap operation on
// a cluster to get the first management token.
ACLBootstrapNow ACLOp = "bootstrap-now"
// ACLForceSet is deprecated, but left for backwards compatibility.
ACLForceSet ACLOp = "force-set"
)
// ACLBootstrapNotInitializedErr is returned when a bootstrap is attempted but
// we haven't yet initialized ACL bootstrap. It provides some guidance to
// operators on how to proceed.
var ACLBootstrapNotInitializedErr = errors.New("ACL bootstrap not initialized, need to force a leader election and ensure all Consul servers support this feature")
const (
// ACLTokenTypeClient tokens have rules applied
ACLTokenTypeClient = "client"
// ACLTokenTypeManagement tokens have an always allow policy, so they can
// make other tokens and can access all resources.
ACLTokenTypeManagement = "management"
// ACLTokenTypeNone
ACLTokenTypeNone = ""
)
// ACL is used to represent a token and its rules
type ACL struct {
ID string
Name string
Type string
Rules string
RaftIndex
}
// ACLs is a slice of ACLs.
type ACLs []*ACL
// Convert does a 1-1 mapping of the ACLCompat structure to its ACLToken
// equivalent. This will NOT fill in the other ACLToken fields or perform any other
// upgrade.
func (a *ACL) Convert() *ACLToken {
return &ACLToken{
AccessorID: "",
SecretID: a.ID,
Description: a.Name,
Policies: nil,
Type: a.Type,
Rules: a.Rules,
Local: false,
RaftIndex: a.RaftIndex,
}
}
// Convert attempts to convert an ACLToken into an ACLCompat.
func (tok *ACLToken) Convert() (*ACL, error) {
if tok.Type == "" {
return nil, fmt.Errorf("Cannot convert ACLToken into compat token")
}
compat := &ACL{
ID: tok.SecretID,
Name: tok.Description,
Type: tok.Type,
Rules: tok.Rules,
RaftIndex: tok.RaftIndex,
}
return compat, nil
}
// IsSame checks if one ACL is the same as another, without looking
// at the Raft information (that's why we didn't call it IsEqual). This is
// useful for seeing if an update would be idempotent for all the functional
// parts of the structure.
func (a *ACL) IsSame(other *ACL) bool {
if a.ID != other.ID ||
a.Name != other.Name ||
a.Type != other.Type ||
a.Rules != other.Rules {
return false
}
return true
}
// ACLRequest is used to create, update or delete an ACL
type ACLRequest struct {
Datacenter string
Op ACLOp
ACL ACL
WriteRequest
}
func (r *ACLRequest) RequestDatacenter() string {
return r.Datacenter
}
// ACLRequests is a list of ACL change requests.
type ACLRequests []*ACLRequest
// ACLSpecificRequest is used to request an ACL by ID
type ACLSpecificRequest struct {
Datacenter string
ACL string
QueryOptions
}
// RequestDatacenter returns the DC this request is targeted to.
func (r *ACLSpecificRequest) RequestDatacenter() string {
return r.Datacenter
}
// IndexedACLs has tokens along with the Raft metadata about them.
type IndexedACLs struct {
ACLs ACLs
QueryMeta
}
// ACLBootstrap keeps track of whether bootstrapping ACLs is allowed for a
// cluster.
type ACLBootstrap struct {
// AllowBootstrap will only be true if no existing management tokens
// have been found.
AllowBootstrap bool
RaftIndex
}
// ACLPolicyResolveLegacyRequest is used to request an ACL by Token SecretID, conditionally
// filtering on an ID
type ACLPolicyResolveLegacyRequest struct {
Datacenter string // The Datacenter the RPC may be sent to
ACL string // The Tokens Secret ID
ETag string // Caching ETag to prevent resending the policy when not needed
QueryOptions
}
// RequestDatacenter returns the DC this request is targeted to.
func (r *ACLPolicyResolveLegacyRequest) RequestDatacenter() string {
return r.Datacenter
}
type ACLPolicyResolveLegacyResponse struct {
ETag string
Parent string
Policy *acl.Policy
TTL time.Duration
QueryMeta
}

140
agent/structs/acl_legacy_test.go Normal file
View File

@ -0,0 +1,140 @@
package structs
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestStructs_ACL_IsSame(t *testing.T) {
acl := &ACL{
ID: "guid",
Name: "An ACL for testing",
Type: "client",
Rules: "service \"\" { policy = \"read\" }",
}
if !acl.IsSame(acl) {
t.Fatalf("should be equal to itself")
}
other := &ACL{
ID: "guid",
Name: "An ACL for testing",
Type: "client",
Rules: "service \"\" { policy = \"read\" }",
RaftIndex: RaftIndex{
CreateIndex: 1,
ModifyIndex: 2,
},
}
if !acl.IsSame(other) || !other.IsSame(acl) {
t.Fatalf("should not care about Raft fields")
}
check := func(twiddle, restore func()) {
if !acl.IsSame(other) || !other.IsSame(acl) {
t.Fatalf("should be the same")
}
twiddle()
if acl.IsSame(other) || other.IsSame(acl) {
t.Fatalf("should not be the same")
}
restore()
if !acl.IsSame(other) || !other.IsSame(acl) {
t.Fatalf("should be the same")
}
}
check(func() { other.ID = "nope" }, func() { other.ID = "guid" })
check(func() { other.Name = "nope" }, func() { other.Name = "An ACL for testing" })
check(func() { other.Type = "management" }, func() { other.Type = "client" })
check(func() { other.Rules = "" }, func() { other.Rules = "service \"\" { policy = \"read\" }" })
}
func TestStructs_ACL_Convert(t *testing.T) {
t.Parallel()
acl := &ACL{
ID: "guid",
Name: "AN ACL for testing",
Type: "client",
Rules: `service "" { policy "read" }`,
}
token := acl.Convert()
require.Equal(t, "", token.AccessorID)
require.Equal(t, acl.ID, token.SecretID)
require.Equal(t, acl.Type, token.Type)
require.Equal(t, acl.Name, token.Description)
require.Nil(t, token.Policies)
require.False(t, token.Local)
require.Equal(t, acl.Rules, token.Rules)
require.Equal(t, acl.CreateIndex, token.CreateIndex)
require.Equal(t, acl.ModifyIndex, token.ModifyIndex)
}
func TestStructs_ACLToken_Convert(t *testing.T) {
t.Parallel()
t.Run("Management", func(t *testing.T) {
t.Parallel()
token := &ACLToken{
AccessorID: "6c4eb178-c7f3-4620-b899-91eb8696c265",
SecretID: "67c29ecd-cabc-42e0-a20e-771e9a1ab70c",
Description: "new token",
Policies: []ACLTokenPolicyLink{
ACLTokenPolicyLink{
ID: ACLPolicyGlobalManagementID,
},
},
Type: ACLTokenTypeManagement,
}
acl, err := token.Convert()
require.NoError(t, err)
require.Equal(t, token.SecretID, acl.ID)
require.Equal(t, token.Type, acl.Type)
require.Equal(t, token.Description, acl.Name)
require.Equal(t, "", acl.Rules)
})
t.Run("Client", func(t *testing.T) {
t.Parallel()
token := &ACLToken{
AccessorID: "6c4eb178-c7f3-4620-b899-91eb8696c265",
SecretID: "67c29ecd-cabc-42e0-a20e-771e9a1ab70c",
Description: "new token",
Policies: nil,
Type: ACLTokenTypeClient,
Rules: `acl = "read"`,
}
acl, err := token.Convert()
require.NoError(t, err)
require.Equal(t, token.SecretID, acl.ID)
require.Equal(t, token.Type, acl.Type)
require.Equal(t, token.Description, acl.Name)
require.Equal(t, token.Rules, acl.Rules)
})
t.Run("Unconvertible", func(t *testing.T) {
t.Parallel()
token := &ACLToken{
AccessorID: "6c4eb178-c7f3-4620-b899-91eb8696c265",
SecretID: "67c29ecd-cabc-42e0-a20e-771e9a1ab70c",
Description: "new token",
Policies: []ACLTokenPolicyLink{
ACLTokenPolicyLink{
ID: ACLPolicyGlobalManagementID,
},
},
}
acl, err := token.Convert()
require.Error(t, err)
require.Nil(t, acl)
})
}

641
agent/structs/acl_test.go
View File

@ -1,52 +1,617 @@
package structs
import (
"fmt"
"strings"
"testing"
"github.com/hashicorp/consul/acl"
"github.com/stretchr/testify/require"
)
func TestStructs_ACL_IsSame(t *testing.T) {
acl := &ACL{
ID: "guid",
Name: "An ACL for testing",
Type: "client",
Rules: "service \"\" { policy = \"read\" }",
}
if !acl.IsSame(acl) {
t.Fatalf("should be equal to itself")
}
func TestStructs_ACLToken_PolicyIDs(t *testing.T) {
t.Parallel()
other := &ACL{
ID: "guid",
Name: "An ACL for testing",
Type: "client",
Rules: "service \"\" { policy = \"read\" }",
RaftIndex: RaftIndex{
CreateIndex: 1,
ModifyIndex: 2,
t.Run("Basic", func(t *testing.T) {
t.Parallel()
token := &ACLToken{
Policies: []ACLTokenPolicyLink{
ACLTokenPolicyLink{
ID: "one",
},
ACLTokenPolicyLink{
ID: "two",
},
ACLTokenPolicyLink{
ID: "three",
},
},
}
policyIDs := token.PolicyIDs()
require.Len(t, policyIDs, 3)
require.Equal(t, "one", policyIDs[0])
require.Equal(t, "two", policyIDs[1])
require.Equal(t, "three", policyIDs[2])
})
t.Run("Legacy Management", func(t *testing.T) {
t.Parallel()
a := &ACL{
ID: "root",
Type: ACLTokenTypeManagement,
Name: "management",
}
token := a.Convert()
policyIDs := token.PolicyIDs()
require.Len(t, policyIDs, 0)
embedded := token.EmbeddedPolicy()
require.NotNil(t, embedded)
require.Equal(t, ACLPolicyGlobalManagement, embedded.Rules)
})
t.Run("No Policies", func(t *testing.T) {
t.Parallel()
token := &ACLToken{}
policyIDs := token.PolicyIDs()
require.Len(t, policyIDs, 0)
})
}
func TestStructs_ACLToken_EmbeddedPolicy(t *testing.T) {
t.Parallel()
t.Run("No Rules", func(t *testing.T) {
t.Parallel()
token := &ACLToken{}
require.Nil(t, token.EmbeddedPolicy())
})
t.Run("Legacy Client", func(t *testing.T) {
t.Parallel()
// None of the other fields should be considered
token := &ACLToken{
Type: ACLTokenTypeClient,
Rules: `acl = "read"`,
}
policy := token.EmbeddedPolicy()
require.NotNil(t, policy)
require.NotEqual(t, "", policy.ID)
require.True(t, strings.HasPrefix(policy.Name, "legacy-policy-"))
require.Equal(t, token.Rules, policy.Rules)
require.Equal(t, policy.Syntax, acl.SyntaxLegacy)
require.NotNil(t, policy.Hash)
require.NotEqual(t, []byte{}, policy.Hash)
})
t.Run("Same Policy for Tokens with same Rules", func(t *testing.T) {
t.Parallel()
token1 := &ACLToken{
AccessorID: "f55b260c-5e05-418e-ab19-d421d1ab4b52",
SecretID: "b2165bac-7006-459b-8a72-7f549f0f06d6",
Description: "token 1",
Type: ACLTokenTypeClient,
Rules: `acl = "read"`,
}
token2 := &ACLToken{
AccessorID: "09d1c059-961a-46bd-a2e4-76adebe35fa5",
SecretID: "65e98e67-9b29-470c-8ffa-7c5a23cc67c8",
Description: "token 2",
Type: ACLTokenTypeClient,
Rules: `acl = "read"`,
}
policy1 := token1.EmbeddedPolicy()
policy2 := token2.EmbeddedPolicy()
require.Equal(t, policy1, policy2)
})
}
func TestStructs_ACLToken_SetHash(t *testing.T) {
t.Parallel()
token := ACLToken{
AccessorID: "09d1c059-961a-46bd-a2e4-76adebe35fa5",
SecretID: "65e98e67-9b29-470c-8ffa-7c5a23cc67c8",
Description: "test",
Policies: []ACLTokenPolicyLink{
ACLTokenPolicyLink{
ID: "one",
},
ACLTokenPolicyLink{
ID: "two",
},
ACLTokenPolicyLink{
ID: "three",
},
},
}
if !acl.IsSame(other) || !other.IsSame(acl) {
t.Fatalf("should not care about Raft fields")
}
check := func(twiddle, restore func()) {
if !acl.IsSame(other) || !other.IsSame(acl) {
t.Fatalf("should be the same")
}
t.Run("Nil Hash - Generate", func(t *testing.T) {
require.Nil(t, token.Hash)
h := token.SetHash(false)
require.NotNil(t, h)
require.NotEqual(t, []byte{}, h)
require.Equal(t, h, token.Hash)
})
twiddle()
if acl.IsSame(other) || other.IsSame(acl) {
t.Fatalf("should not be the same")
}
t.Run("Hash Set - Dont Generate", func(t *testing.T) {
original := token.Hash
h := token.SetHash(false)
require.Equal(t, original, h)
restore()
if !acl.IsSame(other) || !other.IsSame(acl) {
t.Fatalf("should be the same")
}
}
token.Description = "changed"
h = token.SetHash(false)
require.Equal(t, original, h)
})
check(func() { other.ID = "nope" }, func() { other.ID = "guid" })
check(func() { other.Name = "nope" }, func() { other.Name = "An ACL for testing" })
check(func() { other.Type = "management" }, func() { other.Type = "client" })
check(func() { other.Rules = "" }, func() { other.Rules = "service \"\" { policy = \"read\" }" })
t.Run("Hash Set - Generate", func(t *testing.T) {
original := token.Hash
h := token.SetHash(true)
require.NotEqual(t, original, h)
})
}
func TestStructs_ACLToken_EstimateSize(t *testing.T) {
t.Parallel()
// estimated size here should
token := ACLToken{
AccessorID: "09d1c059-961a-46bd-a2e4-76adebe35fa5",
SecretID: "65e98e67-9b29-470c-8ffa-7c5a23cc67c8",
Description: "test",
Policies: []ACLTokenPolicyLink{
ACLTokenPolicyLink{
ID: "one",
},
ACLTokenPolicyLink{
ID: "two",
},
ACLTokenPolicyLink{
ID: "three",
},
},
}
// this test is very contrived. Basically just tests that the
// math is okay and returns the value.
require.Equal(t, 120, token.EstimateSize())
}
func TestStructs_ACLToken_Stub(t *testing.T) {
t.Parallel()
t.Run("Basic", func(t *testing.T) {
t.Parallel()
token := ACLToken{
AccessorID: "09d1c059-961a-46bd-a2e4-76adebe35fa5",
SecretID: "65e98e67-9b29-470c-8ffa-7c5a23cc67c8",
Description: "test",
Policies: []ACLTokenPolicyLink{
ACLTokenPolicyLink{
ID: "one",
},
ACLTokenPolicyLink{
ID: "two",
},
ACLTokenPolicyLink{
ID: "three",
},
},
}
stub := token.Stub()
require.Equal(t, token.AccessorID, stub.AccessorID)
require.Equal(t, token.Description, stub.Description)
require.Equal(t, token.Policies, stub.Policies)
require.Equal(t, token.Local, stub.Local)
require.Equal(t, token.CreateTime, stub.CreateTime)
require.Equal(t, token.Hash, stub.Hash)
require.Equal(t, token.CreateIndex, stub.CreateIndex)
require.Equal(t, token.ModifyIndex, stub.ModifyIndex)
require.False(t, stub.Legacy)
})
t.Run("Legacy", func(t *testing.T) {
t.Parallel()
token := ACLToken{
AccessorID: "09d1c059-961a-46bd-a2e4-76adebe35fa5",
SecretID: "65e98e67-9b29-470c-8ffa-7c5a23cc67c8",
Description: "test",
Type: ACLTokenTypeClient,
Rules: `key "" { policy = "read" }`,
}
stub := token.Stub()
require.Equal(t, token.AccessorID, stub.AccessorID)
require.Equal(t, token.Description, stub.Description)
require.Equal(t, token.Policies, stub.Policies)
require.Equal(t, token.Local, stub.Local)
require.Equal(t, token.CreateTime, stub.CreateTime)
require.Equal(t, token.Hash, stub.Hash)
require.Equal(t, token.CreateIndex, stub.CreateIndex)
require.Equal(t, token.ModifyIndex, stub.ModifyIndex)
require.True(t, stub.Legacy)
})
}
func TestStructs_ACLTokens_Sort(t *testing.T) {
t.Parallel()
tokens := ACLTokens{
&ACLToken{
AccessorID: "9db509a9-c809-48c1-895d-99f845b7a9d5",
},
&ACLToken{
AccessorID: "6bd01084-1695-43b8-898d-b2dd7874754d",
},
&ACLToken{
AccessorID: "614a4cef-9149-4271-b878-7edb1ad661f8",
},
&ACLToken{
AccessorID: "c9dd9980-8d54-472f-9e5e-74c02143e1f4",
},
}
tokens.Sort()
require.Equal(t, tokens[0].AccessorID, "614a4cef-9149-4271-b878-7edb1ad661f8")
require.Equal(t, tokens[1].AccessorID, "6bd01084-1695-43b8-898d-b2dd7874754d")
require.Equal(t, tokens[2].AccessorID, "9db509a9-c809-48c1-895d-99f845b7a9d5")
require.Equal(t, tokens[3].AccessorID, "c9dd9980-8d54-472f-9e5e-74c02143e1f4")
}
func TestStructs_ACLTokenListStubs_Sort(t *testing.T) {
t.Parallel()
tokens := ACLTokenListStubs{
&ACLTokenListStub{
AccessorID: "9db509a9-c809-48c1-895d-99f845b7a9d5",
},
&ACLTokenListStub{
AccessorID: "6bd01084-1695-43b8-898d-b2dd7874754d",
},
&ACLTokenListStub{
AccessorID: "614a4cef-9149-4271-b878-7edb1ad661f8",
},
&ACLTokenListStub{
AccessorID: "c9dd9980-8d54-472f-9e5e-74c02143e1f4",
},
}
tokens.Sort()
require.Equal(t, tokens[0].AccessorID, "614a4cef-9149-4271-b878-7edb1ad661f8")
require.Equal(t, tokens[1].AccessorID, "6bd01084-1695-43b8-898d-b2dd7874754d")
require.Equal(t, tokens[2].AccessorID, "9db509a9-c809-48c1-895d-99f845b7a9d5")
require.Equal(t, tokens[3].AccessorID, "c9dd9980-8d54-472f-9e5e-74c02143e1f4")
}
func TestStructs_ACLPolicy_Stub(t *testing.T) {
t.Parallel()
policy := &ACLPolicy{
ID: "09d1c059-961a-46bd-a2e4-76adebe35fa5",
Name: "test",
Description: "test",
Rules: `acl = "read"`,
}
stub := policy.Stub()
require.Equal(t, policy.ID, stub.ID)
require.Equal(t, policy.Name, stub.Name)
require.Equal(t, policy.Description, stub.Description)
require.Equal(t, policy.Datacenters, stub.Datacenters)
require.Equal(t, policy.Hash, stub.Hash)
require.Equal(t, policy.CreateIndex, stub.CreateIndex)
require.Equal(t, policy.ModifyIndex, stub.ModifyIndex)
}
func TestStructs_ACLPolicy_SetHash(t *testing.T) {
t.Parallel()
policy := &ACLPolicy{
ID: "09d1c059-961a-46bd-a2e4-76adebe35fa5",
Name: "test",
Description: "test",
Rules: `acl = "read"`,
}
t.Run("Nil Hash - Generate", func(t *testing.T) {
require.Nil(t, policy.Hash)
h := policy.SetHash(false)
require.NotNil(t, h)
require.NotEqual(t, []byte{}, h)
require.Equal(t, h, policy.Hash)
})
t.Run("Hash Set - Dont Generate", func(t *testing.T) {
original := policy.Hash
h := policy.SetHash(false)
require.Equal(t, original, h)
policy.Description = "changed"
h = policy.SetHash(false)
require.Equal(t, original, h)
})
t.Run("Hash Set - Generate", func(t *testing.T) {
original := policy.Hash
h := policy.SetHash(true)
require.NotEqual(t, original, h)
})
}
func TestStructs_ACLPolicy_EstimateSize(t *testing.T) {
t.Parallel()
policy := ACLPolicy{
ID: "09d1c059-961a-46bd-a2e4-76adebe35fa5",
Name: "test",
Description: "test",
Rules: `acl = "read"`,
}
// this test is very contrived. Basically just tests that the
// math is okay and returns the value.
require.Equal(t, 84, policy.EstimateSize())
policy.Datacenters = []string{"dc1", "dc2"}
require.Equal(t, 90, policy.EstimateSize())
}
func TestStructs_ACLPolicies_Sort(t *testing.T) {
t.Parallel()
policies := ACLPolicies{
&ACLPolicy{
ID: "9db509a9-c809-48c1-895d-99f845b7a9d5",
},
&ACLPolicy{
ID: "6bd01084-1695-43b8-898d-b2dd7874754d",
},
&ACLPolicy{
ID: "614a4cef-9149-4271-b878-7edb1ad661f8",
},
&ACLPolicy{
ID: "c9dd9980-8d54-472f-9e5e-74c02143e1f4",
},
}
policies.Sort()
require.Equal(t, policies[0].ID, "614a4cef-9149-4271-b878-7edb1ad661f8")
require.Equal(t, policies[1].ID, "6bd01084-1695-43b8-898d-b2dd7874754d")
require.Equal(t, policies[2].ID, "9db509a9-c809-48c1-895d-99f845b7a9d5")
require.Equal(t, policies[3].ID, "c9dd9980-8d54-472f-9e5e-74c02143e1f4")
}
func TestStructs_ACLPolicyListStubs_Sort(t *testing.T) {
t.Parallel()
policies := ACLPolicyListStubs{
&ACLPolicyListStub{
ID: "9db509a9-c809-48c1-895d-99f845b7a9d5",
},
&ACLPolicyListStub{
ID: "6bd01084-1695-43b8-898d-b2dd7874754d",
},
&ACLPolicyListStub{
ID: "614a4cef-9149-4271-b878-7edb1ad661f8",
},
&ACLPolicyListStub{
ID: "c9dd9980-8d54-472f-9e5e-74c02143e1f4",
},
}
policies.Sort()
require.Equal(t, policies[0].ID, "614a4cef-9149-4271-b878-7edb1ad661f8")
require.Equal(t, policies[1].ID, "6bd01084-1695-43b8-898d-b2dd7874754d")
require.Equal(t, policies[2].ID, "9db509a9-c809-48c1-895d-99f845b7a9d5")
require.Equal(t, policies[3].ID, "c9dd9980-8d54-472f-9e5e-74c02143e1f4")
}
func TestStructs_ACLPolicies_resolveWithCache(t *testing.T) {
t.Parallel()
config := ACLCachesConfig{
Identities: 0,
Policies: 0,
ParsedPolicies: 4,
Authorizers: 0,
}
cache, err := NewACLCaches(&config)
require.NoError(t, err)
testPolicies := ACLPolicies{
&ACLPolicy{
ID: "5d5653a1-2c2b-4b36-b083-fc9f1398eb7b",
Name: "policy1",
Description: "policy1",
Rules: `node_prefix "" { policy = "read" }`,
Syntax: acl.SyntaxCurrent,
RaftIndex: RaftIndex{
CreateIndex: 1,
ModifyIndex: 2,
},
},
&ACLPolicy{
ID: "b35541f0-a88a-48da-bc66-43553c60b628",
Name: "policy2",
Description: "policy2",
Rules: `agent_prefix "" { policy = "read" }`,
Syntax: acl.SyntaxCurrent,
RaftIndex: RaftIndex{
CreateIndex: 3,
ModifyIndex: 4,
},
},
&ACLPolicy{
ID: "383abb79-94ca-46c6-89b7-8ecb69046de9",
Name: "policy3",
Description: "policy3",
Rules: `key_prefix "" { policy = "read" }`,
Syntax: acl.SyntaxCurrent,
RaftIndex: RaftIndex{
CreateIndex: 5,
ModifyIndex: 6,
},
},
&ACLPolicy{
ID: "8bf38965-95e5-4e86-9be7-f6070cc0708b",
Name: "policy4",
Description: "policy4",
Rules: `service_prefix "" { policy = "read" }`,
Syntax: acl.SyntaxCurrent,
RaftIndex: RaftIndex{
CreateIndex: 7,
ModifyIndex: 8,
},
},
}
t.Run("Cache Misses", func(t *testing.T) {
policies, err := testPolicies.resolveWithCache(cache, nil)
require.NoError(t, err)
require.Len(t, policies, 4)
for i := range testPolicies {
require.Equal(t, testPolicies[i].ID, policies[i].ID)
require.Equal(t, testPolicies[i].ModifyIndex, policies[i].Revision)
}
})
t.Run("Check Cache", func(t *testing.T) {
for i := range testPolicies {
entry := cache.GetParsedPolicy(fmt.Sprintf("%x", testPolicies[i].Hash))
require.NotNil(t, entry)
require.Equal(t, testPolicies[i].ID, entry.Policy.ID)
require.Equal(t, testPolicies[i].ModifyIndex, entry.Policy.Revision)
// set this to detect using from the cache next time
entry.Policy.Revision = 9999
}
})
t.Run("Cache Hits", func(t *testing.T) {
policies, err := testPolicies.resolveWithCache(cache, nil)
require.NoError(t, err)
require.Len(t, policies, 4)
for i := range testPolicies {
require.Equal(t, testPolicies[i].ID, policies[i].ID)
require.Equal(t, uint64(9999), policies[i].Revision)
}
})
}
func TestStructs_ACLPolicies_Compile(t *testing.T) {
t.Parallel()
config := ACLCachesConfig{
Identities: 0,
Policies: 0,
ParsedPolicies: 4,
Authorizers: 2,
}
cache, err := NewACLCaches(&config)
require.NoError(t, err)
testPolicies := ACLPolicies{
&ACLPolicy{
ID: "5d5653a1-2c2b-4b36-b083-fc9f1398eb7b",
Name: "policy1",
Description: "policy1",
Rules: `node_prefix "" { policy = "read" }`,
Syntax: acl.SyntaxCurrent,
RaftIndex: RaftIndex{
CreateIndex: 1,
ModifyIndex: 2,
},
},
&ACLPolicy{
ID: "b35541f0-a88a-48da-bc66-43553c60b628",
Name: "policy2",
Description: "policy2",
Rules: `agent_prefix "" { policy = "read" }`,
Syntax: acl.SyntaxCurrent,
RaftIndex: RaftIndex{
CreateIndex: 3,
ModifyIndex: 4,
},
},
&ACLPolicy{
ID: "383abb79-94ca-46c6-89b7-8ecb69046de9",
Name: "policy3",
Description: "policy3",
Rules: `key_prefix "" { policy = "read" }`,
Syntax: acl.SyntaxCurrent,
RaftIndex: RaftIndex{
CreateIndex: 5,
ModifyIndex: 6,
},
},
&ACLPolicy{
ID: "8bf38965-95e5-4e86-9be7-f6070cc0708b",
Name: "policy4",
Description: "policy4",
Rules: `service_prefix "" { policy = "read" }`,
Syntax: acl.SyntaxCurrent,
RaftIndex: RaftIndex{
CreateIndex: 7,
ModifyIndex: 8,
},
},
}
t.Run("Cache Miss", func(t *testing.T) {
authz, err := testPolicies.Compile(acl.DenyAll(), cache, nil)
require.NoError(t, err)
require.NotNil(t, authz)
require.True(t, authz.NodeRead("foo"))
require.True(t, authz.AgentRead("foo"))
require.True(t, authz.KeyRead("foo"))
require.True(t, authz.ServiceRead("foo"))
require.False(t, authz.ACLRead())
})
t.Run("Check Cache", func(t *testing.T) {
entry := cache.GetAuthorizer(testPolicies.HashKey())
require.NotNil(t, entry)
authz := entry.Authorizer
require.NotNil(t, authz)
require.True(t, authz.NodeRead("foo"))
require.True(t, authz.AgentRead("foo"))
require.True(t, authz.KeyRead("foo"))
require.True(t, authz.ServiceRead("foo"))
require.False(t, authz.ACLRead())
// setup the cache for the next test
cache.PutAuthorizer(testPolicies.HashKey(), acl.DenyAll())
})
t.Run("Cache Hit", func(t *testing.T) {
authz, err := testPolicies.Compile(acl.DenyAll(), cache, nil)
require.NoError(t, err)
require.NotNil(t, authz)
// we reset the Authorizer in the cache so now everything should be denied
require.False(t, authz.NodeRead("foo"))
require.False(t, authz.AgentRead("foo"))
require.False(t, authz.KeyRead("foo"))
require.False(t, authz.ServiceRead("foo"))
require.False(t, authz.ACLRead())
})
}

15
agent/structs/structs.go
View File

@ -35,18 +35,23 @@ const (
DeregisterRequestType = 1
KVSRequestType = 2
SessionRequestType = 3
ACLRequestType = 4
ACLRequestType = 4 // DEPRECATED (ACL-Legacy-Compat)
TombstoneRequestType = 5
CoordinateBatchUpdateType = 6
PreparedQueryRequestType = 7
TxnRequestType = 8
AutopilotRequestType = 9
AreaRequestType = 10
ACLBootstrapRequestType = 11 // FSM snapshots only.
ACLBootstrapRequestType = 11
IntentionRequestType = 12
ConnectCARequestType = 13
ConnectCAProviderStateType = 14
ConnectCAConfigType = 15 // FSM snapshots only.
IndexRequestType = 16 // FSM snapshots only.
ACLTokenUpsertRequestType = 17
ACLTokenDeleteRequestType = 18
ACLPolicyUpsertRequestType = 19
ACLPolicyDeleteRequestType = 20
)
const (
@ -95,7 +100,7 @@ type RPCInfo interface {
RequestDatacenter() string
IsRead() bool
AllowStaleRead() bool
ACLToken() string
TokenSecret() string
}
// QueryOptions is used to specify various flags for read queries
@ -177,7 +182,7 @@ func (q QueryOptions) AllowStaleRead() bool {
return q.AllowStale
}
func (q QueryOptions) ACLToken() string {
func (q QueryOptions) TokenSecret() string {
return q.Token
}
@ -196,7 +201,7 @@ func (w WriteRequest) AllowStaleRead() bool {
return false
}
func (w WriteRequest) ACLToken() string {
func (w WriteRequest) TokenSecret() string {
return w.Token
}

5
agent/structs/structs_test.go
View File

@ -55,7 +55,10 @@ func TestStructs_Implements(t *testing.T) {
_ RPCInfo = &SessionRequest{}
_ RPCInfo = &SessionSpecificRequest{}
_ RPCInfo = &EventFireRequest{}
_ RPCInfo = &ACLPolicyRequest{}
_ RPCInfo = &ACLPolicyResolveLegacyRequest{}
_ RPCInfo = &ACLPolicyBatchReadRequest{}
_ RPCInfo = &ACLPolicyReadRequest{}
_ RPCInfo = &ACLTokenReadRequest{}
_ RPCInfo = &KeyringRequest{}
_ CompoundResponse = &KeyringResponses{}
)

2
agent/user_event_test.go
View File

@ -195,7 +195,7 @@ func TestUserEventToken(t *testing.T) {
Op: structs.ACLSet,
ACL: structs.ACL{
Name: "User token",
Type: structs.ACLTypeClient,
Type: structs.ACLTokenTypeClient,
Rules: testEventPolicy,
},
WriteRequest: structs.WriteRequest{Token: "root"},

2
agent/xds/server.go
View File

@ -62,7 +62,7 @@ const (
// entirely agent-local and all uses private methods this allows a simple shim
// to be written in the agent package to allow resolving without tightly
// coupling this to the agent.
type ACLResolverFunc func(id string) (acl.ACL, error)
type ACLResolverFunc func(id string) (acl.Authorizer, error)
// ConnectAuthz is the interface the agent needs to expose to be able to re-use
// the authorization logic between both APIs.

16
agent/xds/server_test.go
View File

@ -108,9 +108,9 @@ func (m *testManager) ConnectAuthorize(token string, req *structs.ConnectAuthori
func TestServer_StreamAggregatedResources_BasicProtocol(t *testing.T) {
logger := log.New(os.Stderr, "", log.LstdFlags)
mgr := newTestManager(t)
aclResolve := func(id string) (acl.ACL, error) {
aclResolve := func(id string) (acl.Authorizer, error) {
// Allow all
return acl.RootACL("manage"), nil
return acl.RootAuthorizer("manage"), nil
}
envoy := NewTestEnvoy(t, "web-sidecar-proxy", "")
defer envoy.Close()
@ -570,21 +570,21 @@ func TestServer_StreamAggregatedResources_ACLEnforcment(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
logger := log.New(os.Stderr, "", log.LstdFlags)
mgr := newTestManager(t)
aclResolve := func(id string) (acl.ACL, error) {
aclResolve := func(id string) (acl.Authorizer, error) {
if !tt.defaultDeny {
// Allow all
return acl.RootACL("allow"), nil
return acl.RootAuthorizer("allow"), nil
}
if tt.acl == "" {
// No token and defaultDeny is denied
return acl.RootACL("deny"), nil
return acl.RootAuthorizer("deny"), nil
}
// Ensure the correct token was passed
require.Equal(t, tt.token, id)
// Parse the ACL and enforce it
policy, err := acl.Parse(tt.acl, nil)
policy, err := acl.NewPolicyFromSource("", 0, tt.acl, acl.SyntaxLegacy, nil)
require.NoError(t, err)
return acl.New(acl.RootACL("deny"), policy, nil)
return acl.NewPolicyAuthorizer(acl.RootAuthorizer("deny"), []*acl.Policy{policy}, nil)
}
envoy := NewTestEnvoy(t, "web-sidecar-proxy", tt.token)
defer envoy.Close()
@ -723,7 +723,7 @@ func TestServer_Check(t *testing.T) {
// goroutine is touching this yet.
mgr.authz[token] = tt.authzResult
aclResolve := func(id string) (acl.ACL, error) {
aclResolve := func(id string) (acl.Authorizer, error) {
return nil, nil
}
envoy := NewTestEnvoy(t, "web-sidecar-proxy", token)

364
api/acl.go
View File

@ -1,6 +1,8 @@
package api
import (
"fmt"
"io/ioutil"
"time"
)
@ -12,7 +14,42 @@ const (
ACLManagementType = "management"
)
// ACLEntry is used to represent an ACL entry
type ACLTokenPolicyLink struct {
ID string
Name string
}
// ACLToken represents an ACL Token
type ACLToken struct {
CreateIndex uint64
ModifyIndex uint64
AccessorID string
SecretID string
Description string
Policies []*ACLTokenPolicyLink
Local bool
CreateTime time.Time `json:",omitempty"`
Hash []byte `json:",omitempty"`
// DEPRECATED (ACL-Legacy-Compat)
// Rules will only be present for legacy tokens returned via the new APIs
Rules string `json:",omitempty"`
}
type ACLTokenListEntry struct {
CreateIndex uint64
ModifyIndex uint64
AccessorID string
Description string
Policies []*ACLTokenPolicyLink
Local bool
CreateTime time.Time
Hash []byte
Legacy bool
}
// ACLEntry is used to represent a legacy ACL token
// The legacy tokens are deprecated.
type ACLEntry struct {
CreateIndex uint64
ModifyIndex uint64
@ -32,6 +69,28 @@ type ACLReplicationStatus struct {
LastError time.Time
}
// ACLPolicy represents an ACL Policy.
type ACLPolicy struct {
ID string
Name string
Description string
Rules string
Datacenters []string
Hash []byte
CreateIndex uint64
ModifyIndex uint64
}
type ACLPolicyListEntry struct {
ID string
Name string
Description string
Datacenters []string
Hash []byte
CreateIndex uint64
ModifyIndex uint64
}
// ACL can be used to query the ACL endpoints
type ACL struct {
c *Client
@ -44,20 +103,20 @@ func (c *Client) ACL() *ACL {
// Bootstrap is used to perform a one-time ACL bootstrap operation on a cluster
// to get the first management token.
func (a *ACL) Bootstrap() (string, *WriteMeta, error) {
func (a *ACL) Bootstrap() (*ACLToken, *WriteMeta, error) {
r := a.c.newRequest("PUT", "/v1/acl/bootstrap")
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return "", nil, err
return nil, nil, err
}
defer resp.Body.Close()
wm := &WriteMeta{RequestTime: rtt}
var out struct{ ID string }
var out ACLToken
if err := decodeBody(resp, &out); err != nil {
return "", nil, err
return nil, nil, err
}
return out.ID, wm, nil
return &out, wm, nil
}
// Create is used to generate a new token with the given parameters
@ -191,3 +250,296 @@ func (a *ACL) Replication(q *QueryOptions) (*ACLReplicationStatus, *QueryMeta, e
}
return entries, qm, nil
}
func (a *ACL) TokenCreate(token *ACLToken, q *WriteOptions) (*ACLToken, *WriteMeta, error) {
if token.AccessorID != "" {
return nil, nil, fmt.Errorf("Cannot specify an AccessorID in Token Creation")
}
if token.SecretID != "" {
return nil, nil, fmt.Errorf("Cannot specify a SecretID in Token Creation")
}
r := a.c.newRequest("PUT", "/v1/acl/token")
r.setWriteOptions(q)
r.obj = token
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
wm := &WriteMeta{RequestTime: rtt}
var out ACLToken
if err := decodeBody(resp, &out); err != nil {
return nil, nil, err
}
return &out, wm, nil
}
func (a *ACL) TokenUpdate(token *ACLToken, q *WriteOptions) (*ACLToken, *WriteMeta, error) {
if token.AccessorID == "" {
return nil, nil, fmt.Errorf("Must specify an AccessorID for Token Updating")
}
r := a.c.newRequest("PUT", "/v1/acl/token/"+token.AccessorID)
r.setWriteOptions(q)
r.obj = token
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
wm := &WriteMeta{RequestTime: rtt}
var out ACLToken
if err := decodeBody(resp, &out); err != nil {
return nil, nil, err
}
return &out, wm, nil
}
func (a *ACL) TokenClone(tokenID string, description string, q *WriteOptions) (*ACLToken, *WriteMeta, error) {
if tokenID == "" {
return nil, nil, fmt.Errorf("Must specify a tokenID for Token Cloning")
}
r := a.c.newRequest("PUT", "/v1/acl/token/clone/"+tokenID)
r.setWriteOptions(q)
r.obj = struct{ Description string }{description}
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
wm := &WriteMeta{RequestTime: rtt}
var out ACLToken
if err := decodeBody(resp, &out); err != nil {
return nil, nil, err
}
return &out, wm, nil
}
func (a *ACL) TokenDelete(tokenID string, q *WriteOptions) (*WriteMeta, error) {
r := a.c.newRequest("DELETE", "/v1/acl/token/"+tokenID)
r.setWriteOptions(q)
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return nil, err
}
resp.Body.Close()
wm := &WriteMeta{RequestTime: rtt}
return wm, nil
}
func (a *ACL) TokenRead(tokenID string, q *QueryOptions) (*ACLToken, *QueryMeta, error) {
r := a.c.newRequest("GET", "/v1/acl/token/"+tokenID)
r.setQueryOptions(q)
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
qm := &QueryMeta{}
parseQueryMeta(resp, qm)
qm.RequestTime = rtt
var out ACLToken
if err := decodeBody(resp, &out); err != nil {
return nil, nil, err
}
return &out, qm, nil
}
func (a *ACL) TokenReadSelf(q *QueryOptions) (*ACLToken, *QueryMeta, error) {
r := a.c.newRequest("GET", "/v1/acl/token/self")
r.setQueryOptions(q)
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
qm := &QueryMeta{}
parseQueryMeta(resp, qm)
qm.RequestTime = rtt
var out ACLToken
if err := decodeBody(resp, &out); err != nil {
return nil, nil, err
}
return &out, qm, nil
}
func (a *ACL) TokenList(q *QueryOptions) ([]*ACLTokenListEntry, *QueryMeta, error) {
r := a.c.newRequest("GET", "/v1/acl/tokens")
r.setQueryOptions(q)
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
qm := &QueryMeta{}
parseQueryMeta(resp, qm)
qm.RequestTime = rtt
var entries []*ACLTokenListEntry
if err := decodeBody(resp, &entries); err != nil {
return nil, nil, err
}
return entries, qm, nil
}
// TokenUpgrade performs an almost identical operation as TokenUpdate. The only difference is
// that not all parts of the token must be specified here and the server will patch the token
// with the existing secret id, description etc.
func (a *ACL) TokenUpgrade(token *ACLToken, q *WriteOptions) (*ACLToken, *WriteMeta, error) {
if token.AccessorID == "" {
return nil, nil, fmt.Errorf("Must specify an AccessorID for Token Updating")
}
r := a.c.newRequest("PUT", "/v1/acl/token/upgrade"+token.AccessorID)
r.setWriteOptions(q)
r.obj = token
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
wm := &WriteMeta{RequestTime: rtt}
var out ACLToken
if err := decodeBody(resp, &out); err != nil {
return nil, nil, err
}
return &out, wm, nil
}
func (a *ACL) PolicyCreate(policy *ACLPolicy, q *WriteOptions) (*ACLPolicy, *WriteMeta, error) {
if policy.ID != "" {
return nil, nil, fmt.Errorf("Cannot specify an ID in Policy Creation")
}
r := a.c.newRequest("PUT", "/v1/acl/policy")
r.setWriteOptions(q)
r.obj = policy
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
wm := &WriteMeta{RequestTime: rtt}
var out ACLPolicy
if err := decodeBody(resp, &out); err != nil {
return nil, nil, err
}
return &out, wm, nil
}
func (a *ACL) PolicyUpdate(policy *ACLPolicy, q *WriteOptions) (*ACLPolicy, *WriteMeta, error) {
if policy.ID == "" {
return nil, nil, fmt.Errorf("Must specify an ID in Policy Creation")
}
r := a.c.newRequest("PUT", "/v1/acl/policy/"+policy.ID)
r.setWriteOptions(q)
r.obj = policy
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
wm := &WriteMeta{RequestTime: rtt}
var out ACLPolicy
if err := decodeBody(resp, &out); err != nil {
return nil, nil, err
}
return &out, wm, nil
}
func (a *ACL) PolicyDelete(policyID string, q *WriteOptions) (*WriteMeta, error) {
r := a.c.newRequest("DELETE", "/v1/acl/policy/"+policyID)
r.setWriteOptions(q)
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return nil, err
}
resp.Body.Close()
wm := &WriteMeta{RequestTime: rtt}
return wm, nil
}
func (a *ACL) PolicyRead(policyID string, q *QueryOptions) (*ACLPolicy, *QueryMeta, error) {
r := a.c.newRequest("GET", "/v1/acl/policy/"+policyID)
r.setQueryOptions(q)
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
qm := &QueryMeta{}
parseQueryMeta(resp, qm)
qm.RequestTime = rtt
var out ACLPolicy
if err := decodeBody(resp, &out); err != nil {
return nil, nil, err
}
return &out, qm, nil
}
func (a *ACL) PolicyList(q *QueryOptions) ([]*ACLPolicyListEntry, *QueryMeta, error) {
r := a.c.newRequest("GET", "/v1/acl/policies")
r.setQueryOptions(q)
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
qm := &QueryMeta{}
parseQueryMeta(resp, qm)
qm.RequestTime = rtt
var entries []*ACLPolicyListEntry
if err := decodeBody(resp, &entries); err != nil {
return nil, nil, err
}
return entries, qm, nil
}
func (a *ACL) PolicyTranslate(rules string) (string, error) {
r := a.c.newRequest("POST", "/v1/acl/policy/translate")
r.obj = rules
rtt, resp, err := requireOK(a.c.doRequest(r))
if err != nil {
return "", err
}
defer resp.Body.Close()
qm := &QueryMeta{}
parseQueryMeta(resp, qm)
qm.RequestTime = rtt
ruleBytes, err := ioutil.ReadAll(resp.Body)
if err != nil {
return "", fmt.Errorf("Failed to read translated rule body: %v", err)
}
return string(ruleBytes), nil
}

3
api/acl_test.go
View File

@ -122,7 +122,8 @@ func TestAPI_ACLList(t *testing.T) {
t.Fatalf("err: %v", err)
}
if len(acls) < 2 {
// anon token is a new token
if len(acls) < 1 {
t.Fatalf("bad: %v", acls)
}

3
api/api_test.go
View File

@ -35,8 +35,9 @@ func makeACLClient(t *testing.T) (*Client, *testutil.TestServer) {
return makeClientWithConfig(t, func(clientConfig *Config) {
clientConfig.Token = "root"
}, func(serverConfig *testutil.TestServerConfig) {
serverConfig.PrimaryDatacenter = "dc1"
serverConfig.ACLMasterToken = "root"
serverConfig.ACLDatacenter = "dc1"
serverConfig.ACL.Enabled = true
serverConfig.ACLDefaultPolicy = "deny"
})
}

9
build-support/docker/Consul-Dev.dockerfile
View File

@ -1,12 +1,15 @@
FROM golang:latest as builder
ARG CONSUL_BUILD_IMAGE
FROM ${CONSUL_BUILD_IMAGE}:latest as builder
# FROM golang:latest as builder
ARG GIT_COMMIT
ARG GIT_DIRTY
ARG GIT_DESCRIBE
WORKDIR /go/src/github.com/hashicorp/consul
# WORKDIR /go/src/github.com/hashicorp/consul
ENV CONSUL_DEV=1
ENV COLORIZE=0
Add . /go/src/github.com/hashicorp/consul/
RUN make
RUN make dev
FROM consul:latest

292
build-support/functions/10-util.sh
View File

@ -4,9 +4,9 @@ function err {
tput bold
tput setaf 1
fi
echo "$@" 1>&2
if test "${COLORIZE}" -eq 1
then
tput sgr0
@ -19,9 +19,9 @@ function status {
tput bold
tput setaf 4
fi
echo "$@"
if test "${COLORIZE}" -eq 1
then
tput sgr0
@ -34,13 +34,13 @@ function status_stage {
tput bold
tput setaf 2
fi
echo "$@"
if test "${COLORIZE}" -eq 1
then
tput sgr0
fi
fi
}
function debug {
@ -76,7 +76,7 @@ function is_set {
# Return:
# 0 - is truthy (backwards I know but allows syntax like `if is_set <var>` to work)
# 1 - is not truthy
local val=$(tr '[:upper:]' '[:lower:]' <<< "$1")
case $val in
1 | t | true | y | yes)
@ -95,7 +95,7 @@ function have_gpg_key {
# Return:
# 0 - success (we can use this key for signing)
# * - failure (key cannot be used)
gpg --list-secret-keys $1 > /dev/null 2>&1
return $?
}
@ -114,44 +114,44 @@ function parse_version {
# Notes:
# If the GOTAGS environment variable is present then it is used to determine which
# version file to use for parsing.
local vfile="${1}/version/version.go"
# ensure the version file exists
if ! test -f "${vfile}"
then
err "Error - File not found: ${vfile}"
return 1
fi
local include_release="$2"
local use_git_env="$3"
local omit_version="$4"
local git_version=""
local git_commit=""
if test -z "${include_release}"
then
include_release=true
fi
if test -z "${use_git_env}"
then
use_git_env=true
fi
if is_set "${use_git_env}"
then
git_version="${GIT_DESCRIBE}"
git_commit="${GIT_COMMIT}"
fi
# Get the main version out of the source file
version_main=$(awk '$1 == "Version" && $2 == "=" { gsub(/"/, "", $3); print $3 }' < ${vfile})
release_main=$(awk '$1 == "VersionPrerelease" && $2 == "=" { gsub(/"/, "", $3); print $3 }' < ${vfile})
# try to determine the version if we have build tags
for tag in "$GOTAGS"
do
@ -171,34 +171,34 @@ function parse_version {
then
version="${git_version}"
fi
local rel_ver=""
local rel_ver=""
if is_set "${include_release}"
then
# Default to pre-release from the source
rel_ver="${release_main}"
# When no GIT_DESCRIBE env var is present and no release is in the source then we
# When no GIT_DESCRIBE env var is present and no release is in the source then we
# are definitely in dev mode
if test -z "${git_version}" -a -z "${rel_ver}" && is_set "${use_git_env}"
then
rel_ver="dev"
fi
# Add the release to the version
if test -n "${rel_ver}" -a -n "${git_commit}"
then
rel_ver="${rel_ver} (${git_commit})"
fi
fi
if test -n "${rel_ver}"
then
if is_set "${omit_version}"
then
echo "${rel_ver}" | tr -d "'"
else
echo "${version}-${rel_ver}" | tr -d "'"
echo "${version}-${rel_ver}" | tr -d "'"
fi
return 0
elif ! is_set "${omit_version}"
@ -225,14 +225,14 @@ function get_version {
# In addition to processing the main version.go, version_*.go files will be processed if they have
# a Go build tag that matches the one in the GOTAGS environment variable. This tag processing is
# primitive though and will not match complex build tags in the files with negation etc.
local vers="$VERSION"
if test -z "$vers"
then
# parse the OSS version from version.go
vers="$(parse_version ${1} ${2} ${3})"
fi
if test -z "$vers"
then
return 1
@ -252,20 +252,20 @@ function git_branch {
#
# Notes:
# Echos the current branch to stdout when successful
local gdir="$(pwd)"
if test -d "$1"
then
gdir="$1"
fi
pushd "${gdir}" > /dev/null
local ret=0
local ret=0
local head="$(git status -b --porcelain=v2 | awk '{if ($1 == "#" && $2 =="branch.head") { print $3 }}')" || ret=1
popd > /dev/null
test ${ret} -eq 0 && echo "$head"
return ${ret}
}
@ -280,20 +280,20 @@ function git_upstream {
#
# Notes:
# Echos the current upstream branch to stdout when successful
local gdir="$(pwd)"
if test -d "$1"
then
gdir="$1"
fi
pushd "${gdir}" > /dev/null
local ret=0
local ret=0
local head="$(git status -b --porcelain=v2 | awk '{if ($1 == "#" && $2 =="branch.upstream") { print $3 }}')" || ret=1
popd > /dev/null
test ${ret} -eq 0 && echo "$head"
return ${ret}
}
@ -306,26 +306,26 @@ function git_log_summary {
# 0 - success
# * - failure
#
local gdir="$(pwd)"
if test -d "$1"
then
gdir="$1"
fi
pushd "${gdir}" > /dev/null
local ret=0
local head=$(git_branch) || ret=1
local upstream=$(git_upstream) || ret=1
local rev_range="${head}...${upstream}"
if test ${ret} -eq 0
then
status "Git Changes:"
git log --pretty=oneline ${rev_range} || ret=1
fi
return $ret
}
@ -339,22 +339,22 @@ function git_diff {
# 0 - success
# * - failure
#
local gdir="$(pwd)"
if test -d "$1"
then
gdir="$1"
fi
shift
pushd "${gdir}" > /dev/null
local ret=0
local head=$(git_branch) || ret=1
local upstream=$(git_upstream) || ret=1
if test ${ret} -eq 0
then
status "Git Diff - Paths: $@"
@ -383,27 +383,27 @@ function git_remote_url {
#
# Note:
# The push url for the git remote will be echoed to stdout
if ! test -d "$1"
then
err "ERROR: '$1' is not a directory. git_remote_url must be called with the path to the top level source as the first argument'"
err "ERROR: '$1' is not a directory. git_remote_url must be called with the path to the top level source as the first argument'"
return 1
fi
if test -z "$2"
then
err "ERROR: git_remote_url must be called with a second argument that is the name of the remote"
return 1
fi
local ret=0
pushd "$1" > /dev/null
local url=$(git remote get-url --push $2 2>&1) || ret=1
popd > /dev/null
if test "${ret}" -eq 0
then
echo "${url}"
@ -421,24 +421,24 @@ function find_git_remote {
#
# Note:
# The remote name to use for publishing will be echoed to stdout upon success
if ! test -d "$1"
then
err "ERROR: '$1' is not a directory. find_git_remote must be called with the path to the top level source as the first argument'"
err "ERROR: '$1' is not a directory. find_git_remote must be called with the path to the top level source as the first argument'"
return 1
fi
need_url=$(normalize_git_url "${PUBLISH_GIT_HOST}:${PUBLISH_GIT_REPO}")
debug "Required normalized remote: ${need_url}"
pushd "$1" > /dev/null
local ret=1
for remote in $(git remote)
do
url=$(git remote get-url --push ${remote}) || continue
url=$(normalize_git_url "${url}")
debug "Testing Remote: ${remote}: ${url}"
if test "${url}" == "${need_url}"
then
@ -447,7 +447,7 @@ function find_git_remote {
break
fi
done
popd > /dev/null
return ${ret}
}
@ -472,20 +472,20 @@ function is_git_clean {
# 0 - success
# * - error
#
if ! test -d "$1"
then
err "ERROR: '$1' is not a directory. is_git_clean must be called with the path to a git repo as the first argument'"
err "ERROR: '$1' is not a directory. is_git_clean must be called with the path to a git repo as the first argument'"
return 1
fi
local output_status="$2"
pushd "${1}" > /dev/null
local ret=0
test -z "$(git status --porcelain=v2 2> /dev/null)" || ret=1
if is_set "${output_status}" && test "$ret" -ne 0
then
err "Git repo is not clean"
@ -504,13 +504,13 @@ function update_git_env {
# 0 - success
# * - error
#
if ! test -d "$1"
then
err "ERROR: '$1' is not a directory. is_git_clean must be called with the path to a git repo as the first argument'"
err "ERROR: '$1' is not a directory. is_git_clean must be called with the path to a git repo as the first argument'"
return 1
fi
export GIT_COMMIT=$(git rev-parse --short HEAD)
export GIT_DIRTY=$(test -n "$(git status --porcelain)" && echo "+CHANGES")
export GIT_DESCRIBE=$(git describe --tags --always)
@ -528,35 +528,35 @@ function git_push_ref {
# Returns:
# 0 - success
# * - error
if ! test -d "$1"
then
err "ERROR: '$1' is not a directory. push_git_release must be called with the path to the top level source as the first argument'"
err "ERROR: '$1' is not a directory. push_git_release must be called with the path to the top level source as the first argument'"
return 1
fi
local sdir="$1"
local ret=0
local remote="$3"
# find the correct remote corresponding to the desired repo (basically prevent pushing enterprise to oss or oss to enterprise)
if test -z "${remote}"
then
local remote=$(find_git_remote "${sdir}") || return 1
status "Using git remote: ${remote}"
fi
local ref=""
pushd "${sdir}" > /dev/null
if test -z "$2"
then
# If no git ref was provided we lookup the current local branch and its tracking branch
# It must have a tracking upstream and it must be tracking the sanctioned git remote
local head=$(git_branch "${sdir}") || return 1
local upstream=$(git_upstream "${sdir}") || return 1
# upstream branch for this branch does not track the remote we need to push to
# basically this checks that the upstream (could be something like origin/master) references the correct remote
# if it doesn't then the string modification wont apply and the var will reamin unchanged and equal to itself.
@ -570,7 +570,7 @@ function git_push_ref {
# A git ref was provided - get the full ref and make sure it isn't ambiguous and also to
# be able to determine whether its a branch or tag we are pushing
ref_out=$(git rev-parse --symbolic-full-name "$2" --)
# -ne 2 because it should have the ref on one line followed by a line with '--'
if test "$(wc -l <<< "${ref_out}")" -ne 2
then
@ -578,10 +578,10 @@ function git_push_ref {
debug "${ref_out}"
ret=1
else
ref=$(head -n 1 <<< "${ref_out}")
ref=$(head -n 1 <<< "${ref_out}")
fi
fi
if test ${ret} -eq 0
then
case "${ref}" in
@ -595,16 +595,16 @@ function git_push_ref {
err "ERROR: git_push_ref func is refusing to push ref that isn't a branch or tag"
return 1
esac
if ! git push "${remote}" "${ref}"
then
err "ERROR: Failed to push ${ref} to remote: ${remote}"
ret=1
fi
fi
popd > /dev/null
return $ret
}
@ -617,23 +617,23 @@ function update_version {
# Returns:
# 0 - success
# * - error
if ! test -f "$1"
then
err "ERROR: '$1' is not a regular file. update_version must be called with the path to a go version file"
err "ERROR: '$1' is not a regular file. update_version must be called with the path to a go version file"
return 1
fi
if test -z "$2"
then
err "ERROR: The version specified was empty"
return 1
fi
local vfile="$1"
local version="$2"
local prerelease="$3"
sed_i ${SED_EXT} -e "s/(Version[[:space:]]*=[[:space:]]*)\"[^\"]*\"/\1\"${version}\"/g" -e "s/(VersionPrerelease[[:space:]]*=[[:space:]]*)\"[^\"]*\"/\1\"${prerelease}\"/g" "${vfile}"
return $?
}
@ -647,28 +647,28 @@ function set_changelog_version {
# Returns:
# 0 - success
# * - error
local changelog="${1}/CHANGELOG.md"
local version="$2"
local rel_date="$3"
if ! test -f "${changelog}"
then
err "ERROR: File not found: ${changelog}"
return 1
fi
if test -z "${version}"
then
err "ERROR: Must specify a version to put into the changelog"
return 1
fi
if test -z "${rel_date}"
then
rel_date=$(date +"%B %d, %Y")
fi
sed_i ${SED_EXT} -e "s/## UNRELEASED/## ${version} (${rel_date})/" "${changelog}"
return $?
}
@ -680,15 +680,15 @@ function unset_changelog_version {
# Returns:
# 0 - success
# * - error
local changelog="${1}/CHANGELOG.md"
if ! test -f "${changelog}"
then
err "ERROR: File not found: ${changelog}"
return 1
fi
sed_i ${SED_EXT} -e "1 s/^## [0-9]+\.[0-9]+\.[0-9]+ \([^)]*\)/## UNRELEASED/" "${changelog}"
return $?
}
@ -700,21 +700,21 @@ function add_unreleased_to_changelog {
# Returns:
# 0 - success
# * - error
local changelog="${1}/CHANGELOG.md"
if ! test -f "${changelog}"
then
err "ERROR: File not found: ${changelog}"
return 1
fi
# Check if we are already in unreleased mode
if head -n 1 "${changelog}" | grep -q -c UNRELEASED
then
return 0
fi
local tfile="$(mktemp) -t "CHANGELOG.md_")"
(
echo -e "## UNRELEASED\n" > "${tfile}" &&
@ -732,50 +732,50 @@ function set_release_mode {
# $2 - The version of the release
# $3 - The release date
# $4 - The pre-release version
#
#
#
# Returns:
# 0 - success
# * - error
if ! test -d "$1"
then
err "ERROR: '$1' is not a directory. set_release_mode must be called with the path to a git repo as the first argument"
err "ERROR: '$1' is not a directory. set_release_mode must be called with the path to a git repo as the first argument"
return 1
fi
if test -z "$2"
then
err "ERROR: The version specified was empty"
return 1
fi
local sdir="$1"
local vers="$2"
local rel_date="$(date +"%B %d, %Y")"
if test -n "$3"
then
rel_date="$3"
fi
local changelog_vers="${vers}"
if test -n "$4"
then
changelog_vers="${vers}-$4"
fi
status_stage "==> Updating CHANGELOG.md with release info: ${changelog_vers} (${rel_date})"
set_changelog_version "${sdir}" "${changelog_vers}" "${rel_date}" || return 1
status_stage "==> Updating version/version.go"
if ! update_version "${sdir}/version/version.go" "${vers}" "$4"
then
unset_changelog_version "${sdir}"
return 1
fi
return 0
return 0
}
function set_dev_mode {
@ -785,22 +785,22 @@ function set_dev_mode {
# Returns:
# 0 - success
# * - error
if ! test -d "$1"
then
err "ERROR: '$1' is not a directory. set_dev_mode must be called with the path to a git repo as the first argument'"
err "ERROR: '$1' is not a directory. set_dev_mode must be called with the path to a git repo as the first argument'"
return 1
fi
local sdir="$1"
local vers="$(parse_version "${sdir}" false false)"
status_stage "==> Setting VersionPreRelease back to 'dev'"
update_version "${sdir}/version/version.go" "${vers}" dev || return 1
status_stage "==> Adding new UNRELEASED label in CHANGELOG.md"
add_unreleased_to_changelog "${sdir}" || return 1
return 0
}
@ -811,28 +811,28 @@ function git_staging_empty {
# Returns:
# 0 - success (nothing staged)
# * - error (staged files)
if ! test -d "$1"
then
err "ERROR: '$1' is not a directory. commit_dev_mode must be called with the path to a git repo as the first argument'"
err "ERROR: '$1' is not a directory. commit_dev_mode must be called with the path to a git repo as the first argument'"
return 1
fi
pushd "$1" > /dev/null
declare -i ret=0
for status in $(git status --porcelain=v2 | awk '{print $2}' | cut -b 1)
do
if test "${status}" != "."
then
then
ret=1
break
fi
done
popd > /dev/null
return ${ret}
return ${ret}
}
function commit_dev_mode {
@ -842,31 +842,31 @@ function commit_dev_mode {
# Returns:
# 0 - success
# * - error
if ! test -d "$1"
then
err "ERROR: '$1' is not a directory. commit_dev_mode must be called with the path to a git repo as the first argument'"
err "ERROR: '$1' is not a directory. commit_dev_mode must be called with the path to a git repo as the first argument'"
return 1
fi
status "Checking for previously staged files"
git_staging_empty "$1" || return 1
declare -i ret=0
pushd "$1" > /dev/null
status "Staging CHANGELOG.md and version_*.go files"
git add CHANGELOG.md && git add version/version*.go
ret=$?
if test ${ret} -eq 0
then
status "Adding Commit"
git commit -m "Putting source back into Dev Mode"
ret=$?
ret=$?
fi
popd >/dev/null
return ${ret}
}
@ -879,14 +879,14 @@ function gpg_detach_sign {
# Returns:
# 0 - success
# * - failure
# determine whether the gpg key to use is being overridden
local gpg_key=${HASHICORP_GPG_KEY}
if test -n "$2"
then
gpg_key=$2
fi
gpg --default-key "${gpg_key}" --detach-sig --yes -v "$1"
return $?
}
@ -899,24 +899,24 @@ function shasum_directory {
# Returns:
# 0 - success
# * - failure
if ! test -d "$1"
then
err "ERROR: '$1' is not a directory and shasum_release requires passing a directory as the first argument"
return 1
fi
if test -z "$2"
then
err "ERROR: shasum_release requires a second argument to be the filename to output the shasums to but none was given"
return 1
return 1
fi
pushd $1 > /dev/null
shasum -a256 * > "$2"
ret=$?
popd >/dev/null
return $ret
}
@ -934,7 +934,7 @@ function shasum_directory {
err "ERROR: No such file: '$1'"
return 1
fi
local ui_version=$(sed -n ${SED_EXT} -e 's/.*CONSUL_VERSION%22%3A%22([^%]*)%22%2C%22.*/\1/p' < "$1") || return 1
echo "$ui_version"
return 0

55
command/acl/acl.go Normal file
View File

@ -0,0 +1,55 @@
package acl
import (
"github.com/hashicorp/consul/command/flags"
"github.com/mitchellh/cli"
)
func New() *cmd {
return &cmd{}
}
type cmd struct{}
func (c *cmd) Run(args []string) int {
return cli.RunResultHelp
}
func (c *cmd) Synopsis() string {
return synopsis
}
func (c *cmd) Help() string {
return flags.Usage(help, nil)
}
const synopsis = "Interact with the Consul's ACLs"
const help = `
Usage: consul acl <subcommand> [options] [args]
This command has subcommands for interacting with Consul's ACLs.
Here are some simple examples, and more detailed examples are available
in the subcommands or the documentation.
Bootstrap ACLs:
$ consul acl bootstrap
List all ACL Tokens:
$ consul acl tokens list
Create a new ACL Policy:
$ consul acl policy create “new-policy” \
-description “This is an example policy” \
-datacenter “dc1” \
-datacenter “dc2” \
-rules @rules.hcl
Set the default agent token:
$ consul acl set-agent-token default 0bc6bc46-f25e-4262-b2d9-ffbe1d96be6f
For more examples, ask for subcommand help or view the documentation.
`

175
command/acl/acl_helpers.go Normal file
View File

@ -0,0 +1,175 @@
package acl
import (
"fmt"
"strings"
"github.com/hashicorp/consul/api"
"github.com/mitchellh/cli"
)
func PrintToken(token *api.ACLToken, ui cli.Ui, showMeta bool) {
ui.Info(fmt.Sprintf("AccessorID: %s", token.AccessorID))
ui.Info(fmt.Sprintf("SecretID: %s", token.SecretID))
ui.Info(fmt.Sprintf("Description: %s", token.Description))
ui.Info(fmt.Sprintf("Local: %t", token.Local))
ui.Info(fmt.Sprintf("Create Time: %v", token.CreateTime))
if showMeta {
ui.Info(fmt.Sprintf("Hash: %x", token.Hash))
ui.Info(fmt.Sprintf("Create Index: %d", token.CreateIndex))
ui.Info(fmt.Sprintf("Modify Index: %d", token.ModifyIndex))
}
ui.Info(fmt.Sprintf("Policies:"))
for _, policy := range token.Policies {
ui.Info(fmt.Sprintf(" %s - %s", policy.ID, policy.Name))
}
if token.Rules != "" {
ui.Info(fmt.Sprintf("Rules:"))
ui.Info(token.Rules)
}
}
func PrintTokenListEntry(token *api.ACLTokenListEntry, ui cli.Ui, showMeta bool) {
ui.Info(fmt.Sprintf("AccessorID: %s", token.AccessorID))
ui.Info(fmt.Sprintf("Description: %s", token.Description))
ui.Info(fmt.Sprintf("Local: %t", token.Local))
ui.Info(fmt.Sprintf("Create Time: %v", token.CreateTime))
ui.Info(fmt.Sprintf("Legacy: %t", token.Legacy))
if showMeta {
ui.Info(fmt.Sprintf("Hash: %x", token.Hash))
ui.Info(fmt.Sprintf("Create Index: %d", token.CreateIndex))
ui.Info(fmt.Sprintf("Modify Index: %d", token.ModifyIndex))
}
ui.Info(fmt.Sprintf("Policies:"))
for _, policy := range token.Policies {
ui.Info(fmt.Sprintf(" %s - %s", policy.ID, policy.Name))
}
}
func PrintPolicy(policy *api.ACLPolicy, ui cli.Ui, showMeta bool) {
ui.Info(fmt.Sprintf("ID: %s", policy.ID))
ui.Info(fmt.Sprintf("Name: %s", policy.Name))
ui.Info(fmt.Sprintf("Description: %s", policy.Description))
ui.Info(fmt.Sprintf("Datacenters: %s", strings.Join(policy.Datacenters, ", ")))
if showMeta {
ui.Info(fmt.Sprintf("Hash: %x", policy.Hash))
ui.Info(fmt.Sprintf("Create Index: %d", policy.CreateIndex))
ui.Info(fmt.Sprintf("Modify Index: %d", policy.ModifyIndex))
}
ui.Info(fmt.Sprintf("Rules:"))
ui.Info(policy.Rules)
}
func PrintPolicyListEntry(policy *api.ACLPolicyListEntry, ui cli.Ui, showMeta bool) {
ui.Info(fmt.Sprintf("%s:", policy.Name))
ui.Info(fmt.Sprintf(" ID: %s", policy.ID))
ui.Info(fmt.Sprintf(" Description: %s", policy.Description))
ui.Info(fmt.Sprintf(" Datacenters: %s", strings.Join(policy.Datacenters, ", ")))
if showMeta {
ui.Info(fmt.Sprintf(" Hash: %x", policy.Hash))
ui.Info(fmt.Sprintf(" Create Index: %d", policy.CreateIndex))
ui.Info(fmt.Sprintf(" Modify Index: %d", policy.ModifyIndex))
}
}
func GetTokenIDFromPartial(client *api.Client, partialID string) (string, error) {
// the full UUID string was given
if len(partialID) == 36 {
return partialID, nil
}
tokens, _, err := client.ACL().TokenList(nil)
if err != nil {
return "", err
}
tokenID := ""
for _, token := range tokens {
if strings.HasPrefix(token.AccessorID, partialID) {
if tokenID != "" {
return "", fmt.Errorf("Partial token ID is not unique")
}
tokenID = token.AccessorID
}
}
if tokenID == "" {
return "", fmt.Errorf("No such token ID with prefix: %s", partialID)
}
return tokenID, nil
}
func GetPolicyIDFromPartial(client *api.Client, partialID string) (string, error) {
// The full UUID string was given
if len(partialID) == 36 {
return partialID, nil
}
policies, _, err := client.ACL().PolicyList(nil)
if err != nil {
return "", err
}
policyID := ""
for _, policy := range policies {
if strings.HasPrefix(policy.ID, partialID) {
if policyID != "" {
return "", fmt.Errorf("Partial policy ID is not unique")
}
policyID = policy.ID
}
}
if policyID == "" {
return "", fmt.Errorf("No such policy ID with prefix: %s", partialID)
}
return policyID, nil
}
func GetPolicyIDByName(client *api.Client, name string) (string, error) {
if name == "" {
return "", fmt.Errorf("No name specified")
}
policies, _, err := client.ACL().PolicyList(nil)
if err != nil {
return "", err
}
for _, policy := range policies {
if policy.Name == name {
return policy.ID, nil
}
}
return "", fmt.Errorf("No such policy with name %s", name)
}
func GetRulesFromLegacyToken(client *api.Client, tokenID string, isSecret bool) (string, error) {
var token *api.ACLToken
var err error
if isSecret {
qopts := api.QueryOptions{
Token: tokenID,
}
token, _, err = client.ACL().TokenReadSelf(&qopts)
} else {
token, _, err = client.ACL().TokenRead(tokenID, nil)
}
if err != nil {
return "", fmt.Errorf("Error reading token: %v", err)
}
if token == nil {
return "", fmt.Errorf("Token not found for ID")
}
if token.Rules == "" {
return "", fmt.Errorf("Token is not a legacy token with rules")
}
return token.Rules, nil
}

134
command/acl/agenttokens/agent_tokens.go Normal file
View File

@ -0,0 +1,134 @@
package agenttokens
import (
"flag"
"fmt"
"io"
"github.com/hashicorp/consul/command/flags"
"github.com/hashicorp/consul/command/helpers"
"github.com/mitchellh/cli"
)
func New(ui cli.Ui) *cmd {
c := &cmd{UI: ui}
c.init()
return c
}
type cmd struct {
UI cli.Ui
flags *flag.FlagSet
http *flags.HTTPFlags
help string
testStdin io.Reader
}
func (c *cmd) init() {
c.flags = flag.NewFlagSet("", flag.ContinueOnError)
c.http = &flags.HTTPFlags{}
flags.Merge(c.flags, c.http.ClientFlags())
flags.Merge(c.flags, c.http.ServerFlags())
c.help = flags.Usage(help, c.flags)
}
func (c *cmd) Run(args []string) int {
if err := c.flags.Parse(args); err != nil {
return 1
}
tokenType, token, err := c.dataFromArgs(c.flags.Args())
if err != nil {
c.UI.Error(fmt.Sprintf("Error! %s", err))
return 1
}
client, err := c.http.APIClient()
if err != nil {
c.UI.Error(fmt.Sprintf("Error connecting to Consul Agent: %s", err))
return 1
}
switch tokenType {
case "default":
_, err = client.Agent().UpdateACLToken(token, nil)
case "agent":
_, err = client.Agent().UpdateACLAgentToken(token, nil)
case "master":
_, err = client.Agent().UpdateACLAgentMasterToken(token, nil)
case "replication":
_, err = client.Agent().UpdateACLReplicationToken(token, nil)
default:
c.UI.Error(fmt.Sprintf("Unknown token type"))
return 1
}
if err != nil {
c.UI.Error(fmt.Sprintf("Failed to set ACL token %q: %v", tokenType, err))
return 1
}
c.UI.Info(fmt.Sprintf("ACL token %q set successfully", tokenType))
return 0
}
func (c *cmd) dataFromArgs(args []string) (string, string, error) {
switch len(args) {
case 0:
return "", "", fmt.Errorf("Missing TYPE and TOKEN arguments")
case 1:
switch args[0] {
case "default", "agent", "master", "replication":
return "", "", fmt.Errorf("Missing TOKEN argument")
default:
return "", "", fmt.Errorf("MISSING TYPE argument")
}
case 2:
data, err := helpers.LoadDataSource(args[1], c.testStdin)
if err != nil {
return "", "", err
}
return args[0], data, nil
default:
return "", "", fmt.Errorf("Too many arguments: expected 2 got %d", len(args))
}
}
func (c *cmd) Synopsis() string {
return synopsis
}
func (c *cmd) Help() string {
return flags.Usage(c.help, nil)
}
const synopsis = "Interact with the Consul's ACLs"
const help = `
Usage: consul acl set-agent-token [options] TYPE TOKEN
This command will set the corresponding token for the agent to use.
Note that the tokens uploaded this way are not persisted and if
the agent reloads then the tokens will need to be set again.
Token Types:
default The default token is the token that the agent will use for
both internal agent operations and operations initiated by
the HTTP and DNS interfaces when no specific token is provided.
If not set the agent will use the anonymous token.
agent The token that the agent will use for internal agent operations.
If not given then the default token is used for these operations.
master This sets the token that can be used to access the Agent APIs in
the event that the ACL datacenter cannot be reached.
replication This is the token that the agent will use for replication
operations. This token will need to be configured with read access
to whatever data is being replicated.
Example:
$ consul acl set-agent-token default c4d0f8df-3aba-4ab6-a7a0-35b760dc29a1
`

110
command/acl/agenttokens/agent_tokens_test.go Normal file
View File

@ -0,0 +1,110 @@
package agenttokens
import (
"os"
"strings"
"testing"
"github.com/hashicorp/consul/agent"
"github.com/hashicorp/consul/api"
"github.com/hashicorp/consul/logger"
"github.com/hashicorp/consul/testrpc"
"github.com/hashicorp/consul/testutil"
"github.com/mitchellh/cli"
"github.com/stretchr/testify/assert"
)
func TestAgentTokensCommand_noTabs(t *testing.T) {
t.Parallel()
if strings.ContainsRune(New(cli.NewMockUi()).Help(), '\t') {
t.Fatal("help has tabs")
}
}
func TestAgentTokensCommand(t *testing.T) {
t.Parallel()
assert := assert.New(t)
testDir := testutil.TempDir(t, "acl")
defer os.RemoveAll(testDir)
a := agent.NewTestAgent(t.Name(), `
primary_datacenter = "dc1"
acl {
enabled = true
tokens {
master = "root"
}
}`)
a.Agent.LogWriter = logger.NewLogWriter(512)
defer a.Shutdown()
testrpc.WaitForLeader(t, a.RPC, "dc1")
ui := cli.NewMockUi()
cmd := New(ui)
// Create a token to set
client := a.Client()
token, _, err := client.ACL().TokenCreate(
&api.ACLToken{Description: "test"},
&api.WriteOptions{Token: "root"},
)
assert.NoError(err)
// default token
{
args := []string{
"-http-addr=" + a.HTTPAddr(),
"default",
token.SecretID,
}
code := cmd.Run(args)
assert.Equal(code, 0)
assert.Empty(ui.ErrorWriter.String())
}
// agent token
{
args := []string{
"-http-addr=" + a.HTTPAddr(),
"agent",
token.SecretID,
}
code := cmd.Run(args)
assert.Equal(code, 0)
assert.Empty(ui.ErrorWriter.String())
}
// master token
{
args := []string{
"-http-addr=" + a.HTTPAddr(),
"master",
token.SecretID,
}
code := cmd.Run(args)
assert.Equal(code, 0)
assert.Empty(ui.ErrorWriter.String())
}
// replication token
{
args := []string{
"-http-addr=" + a.HTTPAddr(),
"replication",
token.SecretID,
}
code := cmd.Run(args)
assert.Equal(code, 0)
assert.Empty(ui.ErrorWriter.String())
}
}

72
command/acl/bootstrap/bootstrap.go Normal file
View File

@ -0,0 +1,72 @@
package bootstrap
import (
"flag"
"fmt"
"github.com/hashicorp/consul/command/acl"
"github.com/hashicorp/consul/command/flags"
"github.com/mitchellh/cli"
)
func New(ui cli.Ui) *cmd {
c := &cmd{UI: ui}
c.init()
return c
}
type cmd struct {
UI cli.Ui
flags *flag.FlagSet
http *flags.HTTPFlags
help string
}
func (c *cmd) init() {
c.flags = flag.NewFlagSet("", flag.ContinueOnError)
c.http = &flags.HTTPFlags{}
flags.Merge(c.flags, c.http.ClientFlags())
flags.Merge(c.flags, c.http.ServerFlags())
c.help = flags.Usage(help, c.flags)
}
func (c *cmd) Run(args []string) int {
if err := c.flags.Parse(args); err != nil {
return 1
}
client, err := c.http.APIClient()
if err != nil {
c.UI.Error(fmt.Sprintf("Error connecting to Consul agent: %s", err))
return 1
}
token, _, err := client.ACL().Bootstrap()
if err != nil {
c.UI.Error(fmt.Sprintf("Failed ACL bootstrapping: %v", err))
return 1
}
acl.PrintToken(token, c.UI, false)
return 0
}
func (c *cmd) Synopsis() string {
return synopsis
}
func (c *cmd) Help() string {
return flags.Usage(c.help, nil)
}
const synopsis = "Bootstrap Consul's ACL system"
// TODO (ACL-V2) - maybe embed link to bootstrap reset docs
const help = `
Usage: consul acl bootstrap [options]
The bootstrap command will request Consul to generate a new token with unlimited privileges to use
for management purposes and output its details. This can only be done once and afterwards bootstrapping
will be disabled. If all tokens are lost and you need to bootstrap again you can follow the bootstrap
reset procedure
`

Some files were not shown because too many files have changed in this diff Show More