consul/vendor/github.com/hashicorp/hcl
Matt Keeler 18b29c45c4
New ACLs (#4791)
This PR is almost a complete rewrite of the ACL system within Consul. It brings the features more in line with other HashiCorp products. Obviously there is quite a bit left to do here but most of it is related docs, testing and finishing the last few commands in the CLI. I will update the PR description and check off the todos as I finish them over the next few days/week.
Description

At a high level this PR is mainly to split ACL tokens from Policies and to split the concepts of Authorization from Identities. A lot of this PR is mostly just to support CRUD operations on ACLTokens and ACLPolicies. These in and of themselves are not particularly interesting. The bigger conceptual changes are in how tokens get resolved, how backwards compatibility is handled and the separation of policy from identity which could lead the way to allowing for alternative identity providers.

On the surface and with a new cluster the ACL system will look very similar to that of Nomads. Both have tokens and policies. Both have local tokens. The ACL management APIs for both are very similar. I even ripped off Nomad's ACL bootstrap resetting procedure. There are a few key differences though.

    Nomad requires token and policy replication where Consul only requires policy replication with token replication being opt-in. In Consul local tokens only work with token replication being enabled though.
    All policies in Nomad are globally applicable. In Consul all policies are stored and replicated globally but can be scoped to a subset of the datacenters. This allows for more granular access management.
    Unlike Nomad, Consul has legacy baggage in the form of the original ACL system. The ramifications of this are:
        A server running the new system must still support other clients using the legacy system.
        A client running the new system must be able to use the legacy RPCs when the servers in its datacenter are running the legacy system.
        The primary ACL DC's servers running in legacy mode needs to be a gate that keeps everything else in the entire multi-DC cluster running in legacy mode.

So not only does this PR implement the new ACL system but has a legacy mode built in for when the cluster isn't ready for new ACLs. Also detecting that new ACLs can be used is automatic and requires no configuration on the part of administrators. This process is detailed more in the "Transitioning from Legacy to New ACL Mode" section below.
2018-10-19 12:04:07 -04:00
..
hcl New ACLs (#4791) 2018-10-19 12:04:07 -04:00
json New ACLs (#4791) 2018-10-19 12:04:07 -04:00
test-fixtures Manage dependencies via Godep 2016-02-12 16:50:37 -08:00
LICENSE Manage dependencies via Godep 2016-02-12 16:50:37 -08:00
Makefile New config parser, HCL support, multiple bind addrs (#3480) 2017-09-25 11:40:42 -07:00
README.md New config parser, HCL support, multiple bind addrs (#3480) 2017-09-25 11:40:42 -07:00
appveyor.yml New config parser, HCL support, multiple bind addrs (#3480) 2017-09-25 11:40:42 -07:00
decoder.go New ACLs (#4791) 2018-10-19 12:04:07 -04:00
go.mod New ACLs (#4791) 2018-10-19 12:04:07 -04:00
go.sum New ACLs (#4791) 2018-10-19 12:04:07 -04:00
hcl.go Manage dependencies via Godep 2016-02-12 16:50:37 -08:00
lex.go Updates hashicorp/hcl and hashicorp/hil. 2016-08-09 17:24:13 -07:00
parse.go Updates hashicorp/hcl and hashicorp/hil. 2016-08-09 17:24:13 -07:00

README.md

HCL

GoDoc Build Status

HCL (HashiCorp Configuration Language) is a configuration language built by HashiCorp. The goal of HCL is to build a structured configuration language that is both human and machine friendly for use with command-line tools, but specifically targeted towards DevOps tools, servers, etc.

HCL is also fully JSON compatible. That is, JSON can be used as completely valid input to a system expecting HCL. This helps makes systems interoperable with other systems.

HCL is heavily inspired by libucl, nginx configuration, and others similar.

Why?

A common question when viewing HCL is to ask the question: why not JSON, YAML, etc.?

Prior to HCL, the tools we built at HashiCorp used a variety of configuration languages from full programming languages such as Ruby to complete data structure languages such as JSON. What we learned is that some people wanted human-friendly configuration languages and some people wanted machine-friendly languages.

JSON fits a nice balance in this, but is fairly verbose and most importantly doesn't support comments. With YAML, we found that beginners had a really hard time determining what the actual structure was, and ended up guessing more often than not whether to use a hyphen, colon, etc. in order to represent some configuration key.

Full programming languages such as Ruby enable complex behavior a configuration language shouldn't usually allow, and also forces people to learn some set of Ruby.

Because of this, we decided to create our own configuration language that is JSON-compatible. Our configuration language (HCL) is designed to be written and modified by humans. The API for HCL allows JSON as an input so that it is also machine-friendly (machines can generate JSON instead of trying to generate HCL).

Our goal with HCL is not to alienate other configuration languages. It is instead to provide HCL as a specialized language for our tools, and JSON as the interoperability layer.

Syntax

For a complete grammar, please see the parser itself. A high-level overview of the syntax and grammar is listed here.

  • Single line comments start with # or //

  • Multi-line comments are wrapped in /* and */. Nested block comments are not allowed. A multi-line comment (also known as a block comment) terminates at the first */ found.

  • Values are assigned with the syntax key = value (whitespace doesn't matter). The value can be any primitive: a string, number, boolean, object, or list.

  • Strings are double-quoted and can contain any UTF-8 characters. Example: "Hello, World"

  • Multi-line strings start with <<EOF at the end of a line, and end with EOF on its own line (here documents). Any text may be used in place of EOF. Example:

<<FOO
hello
world
FOO
  • Numbers are assumed to be base 10. If you prefix a number with 0x, it is treated as a hexadecimal. If it is prefixed with 0, it is treated as an octal. Numbers can be in scientific notation: "1e10".

  • Boolean values: true, false

  • Arrays can be made by wrapping it in []. Example: ["foo", "bar", 42]. Arrays can contain primitives, other arrays, and objects. As an alternative, lists of objects can be created with repeated blocks, using this structure:

    service {
        key = "value"
    }
    
    service {
        key = "value"
    }
    

Objects and nested objects are created using the structure shown below:

variable "ami" {
    description = "the AMI to use"
}

This would be equivalent to the following json:

{
  "variable": {
      "ami": {
          "description": "the AMI to use"
        }
    }
}

Thanks

Thanks to:

  • @vstakhov - The original libucl parser and syntax that HCL was based off of.

  • @fatih - The rewritten HCL parser in pure Go (no goyacc) and support for a printer.