505 lines
17 KiB
Markdown
505 lines
17 KiB
Markdown
nim-confutils
|
|
=============
|
|
|
|
[](https://travis-ci.org/status-im/nim-confutils)
|
|
[](https://opensource.org/licenses/Apache-2.0)
|
|
[](https://opensource.org/licenses/MIT)
|
|
|
|
|
|
## Introduction
|
|
|
|
Confutils is a library that aims to solve the configuration problem
|
|
with a holistic approach. The run-time configuration of a program
|
|
is described as a plain Nim object type from which the library
|
|
automatically derives the code for handling command-line options,
|
|
configuration files and other platform-specific sources such as the
|
|
Windows registry.
|
|
|
|
The library focuses on providing a lot of compile-time configurability
|
|
and extensibility with a strong adherence to the DRY principle.
|
|
|
|
Let's illustrate the API with a highly annotated example. Our configuration
|
|
might be described in a separate module looking like this:
|
|
|
|
```nim
|
|
# config.nim
|
|
import
|
|
confutils/defs
|
|
|
|
type
|
|
NimbusConf* = object
|
|
#
|
|
# This is our configuration type.
|
|
#
|
|
# Each field will be considered a configuration option that may appear
|
|
# on the command-line, whitin an environment variable or a configuration
|
|
# file, or elsewhere. Custom pragmas are used to annotate the fields with
|
|
# additional metadata that is used to augment the behavior of the library.
|
|
#
|
|
logLevel* {.
|
|
desc: "Sets the log level",
|
|
defaultValue: LogLevel.INFO.}: LogLevel
|
|
|
|
#
|
|
# This program uses a CLI interface with sub-commands (similar to git).
|
|
#
|
|
# The `StartUpCommand` enum provides the list of available sub-commands,
|
|
# but since we are specifying a default value of `noCommand`, the user
|
|
# can also launch the program without entering any particular command.
|
|
# The default command will also be omitted from help messages.
|
|
#
|
|
# Please note that the `logLevel` option above will be shared by all
|
|
# sub-commands. The rest of the nested options will be relevant only
|
|
# when the designated sub-command is being invoked.
|
|
#
|
|
case cmd* {.
|
|
command
|
|
defaultValue: noCommand.}: StartUpCommand
|
|
|
|
of noCommand:
|
|
dataDir* {.
|
|
desc: "The directory where nimbus will store all blockchain data."
|
|
shortform: "d"
|
|
defaultValue: getConfigDir() / "nimbus".}: DirPath
|
|
|
|
bootstrapNodes* {.
|
|
desc: "Specifies one or more bootstrap nodes to use when connecting to the network."
|
|
longform: "bootstrapNode"
|
|
shortform: "b".}: seq[string]
|
|
|
|
bootstrapNodesFile* {.
|
|
desc: "Specifies a line-delimited file of bootsrap Ethereum network addresses"
|
|
shortform: "f"
|
|
defaultValue: "".}: InputFile
|
|
|
|
tcpPort* {.
|
|
desc: "TCP listening port".}: int
|
|
|
|
udpPort* {.
|
|
desc: "UDP listening port".}: int
|
|
|
|
validators* {.
|
|
required
|
|
desc: "A path to a pair of public and private keys for a validator. " &
|
|
"Nimbus will automatically add the extensions .privkey and .pubkey."
|
|
longform: "validator"
|
|
shortform: "v".}: seq[PrivateValidatorData]
|
|
|
|
stateSnapshot* {.
|
|
desc: "Json file specifying a recent state snapshot"
|
|
shortform: "s".}: Option[BeaconState]
|
|
|
|
of createChain:
|
|
chainStartupData* {.
|
|
desc: ""
|
|
shortform: "c".}: ChainStartupData
|
|
|
|
outputStateFile* {.
|
|
desc: "Output file where to write the initial state snapshot"
|
|
longform: "out"
|
|
shortform: "o".}: OutFilePath
|
|
|
|
StartUpCommand* = enum
|
|
noCommand
|
|
createChain
|
|
|
|
#
|
|
# The configuration can use user-defined types that feature custom
|
|
# command-line parsing and serialization routines.
|
|
#
|
|
PrivateValidatorData* = object
|
|
privKey*: ValidatorPrivKey
|
|
randao*: Randao
|
|
|
|
```
|
|
|
|
Then from our main module, we just need to call `confutils.load` which must be
|
|
given our configuration type as a parameter:
|
|
|
|
```nim
|
|
# main.nim
|
|
import
|
|
confutils, config
|
|
|
|
when isMainModule:
|
|
let conf = NimbusConf.load()
|
|
initDatabase conf.dataDir
|
|
```
|
|
|
|
And that's it - calling `load` with default parameters will first process any
|
|
[command-line options](#handling-of-command-line-options) and then it will
|
|
try to load any missing options from the most appropriate
|
|
[configuration location](#handling-of-environment-variables-and-config-files)
|
|
for the platform. Diagnostic messages will be provided for many simple
|
|
configuration errors and the following help message will be produced
|
|
automatically when calling the program with `program --help`:
|
|
|
|
```
|
|
Usage: beacon_node [OPTIONS] <command>
|
|
|
|
The following options are supported:
|
|
|
|
--logLevel=LogLevel : Sets the log level
|
|
--dataDir=DirPath : The directory where nimbus will store all blockchain data.
|
|
--bootstrapNode=seq[string] : Specifies one or more bootstrap nodes to use when connecting to the network.
|
|
--bootstrapNodesFile=FilePath : Specifies a line-delimited file of bootsrap Ethereum network addresses
|
|
--tcpPort=int : TCP listening port
|
|
--udpPort=int : UDP listening port
|
|
--validator=seq[PrivateValidatorData] : A path to a pair of public and private keys for a validator. Nimbus will automatically add the extensions .privkey and .pubkey.
|
|
--stateSnapshot=Option[BeaconState] : Json file specifying a recent state snapshot
|
|
|
|
Available sub-commands:
|
|
|
|
beacon_node createChain
|
|
|
|
--out=OutFilePath : Output file where to write the initial state snapshot
|
|
|
|
```
|
|
|
|
For simpler CLI utilities, Confutils also provides the following convenience APIs:
|
|
|
|
```nim
|
|
import
|
|
confutils
|
|
|
|
cli do (validators {.
|
|
desc: "number of validators"
|
|
shortform: "v".}: int,
|
|
|
|
outputDir {.
|
|
desc: "output dir to store the generated files"
|
|
shortform: "o"}: OutPath,
|
|
|
|
startupDelay {.
|
|
desc: "delay in seconds before starting the simulation".} = 0):
|
|
|
|
if validators < 64:
|
|
echo "The number of validators must be greater than EPOCH_LENGTH (64)"
|
|
quit(1)
|
|
```
|
|
|
|
```nim
|
|
import
|
|
confutils
|
|
|
|
proc main(foo: string, bar: int) =
|
|
...
|
|
|
|
dispatch(main)
|
|
```
|
|
|
|
Under the hood, using these APIs will result in calling `load` on an anonymous
|
|
configuration type having the same fields as the supplied proc params.
|
|
Any additional arguments given as `cli(args) do ...` and `dispatch(fn, args)`
|
|
will be passed to `load` without modification.
|
|
|
|
This covers the basic usage of the library and the rest of the documentation
|
|
will describe the various ways the default behavior can be tweaked or extended.
|
|
|
|
|
|
## Configuration field pragmas
|
|
|
|
A number of pragmas defined in `confutils/defs` can be attached to the
|
|
configuration fields to control the behavior of the library.
|
|
|
|
```nim
|
|
template desc*(v: string) {.pragma.}
|
|
```
|
|
|
|
A description of the configuration option that will appear in the produced
|
|
help messages.
|
|
|
|
-----------------
|
|
|
|
```nim
|
|
template longform*(v: string) {.pragma.}
|
|
```
|
|
|
|
A long name of the option.
|
|
Typically, it will have to be be specified as `--longOptionName value`.
|
|
See [Handling of command-line options](#handling-of-command-line-options)
|
|
for more details.
|
|
|
|
-----------------
|
|
|
|
```nim
|
|
template shortform*(v: string) {.pragma.}
|
|
|
|
```
|
|
|
|
A short name of the option.
|
|
Typically, it will be required to be specified as `-x value`.
|
|
See [Handling of command-line options](#handling-of-command-line-options)
|
|
for more details.
|
|
|
|
-----------------
|
|
|
|
```nim
|
|
template defaultValue*(v: untyped) {.pragma.}
|
|
```
|
|
|
|
The default value of the option if no value was supplied by the user.
|
|
|
|
-----------------
|
|
|
|
```nim
|
|
template required* {.pragma.}
|
|
```
|
|
|
|
By default, all options without default values are considered required.
|
|
An exception to this rule are all `seq[T]` or `Option[T]` options for
|
|
which the "empty" value can be considered a reasonable default. You can
|
|
also extend this behavior to other user-defined types by providing the
|
|
following overloads:
|
|
|
|
```nim
|
|
template hasDefault*(T: type Foo): bool = true
|
|
template default*(T: type Foo): Foo = Foo(...)
|
|
```
|
|
|
|
The `required` pragma can be applied to fields having such defaultable
|
|
types to make them required.
|
|
|
|
-----------------
|
|
|
|
```nim
|
|
template command* {.pragma.}
|
|
```
|
|
|
|
This must be applied to an enum field that represents a possible sub-command.
|
|
See the section on [sub-commands](#Using-sub-commands) for more details.
|
|
|
|
-----------------
|
|
|
|
```nim
|
|
template argument* {.pragma.}
|
|
```
|
|
|
|
This field represents an argument to the program. If the program expects
|
|
multiple arguments, this pragma can be applied to multiple fields or to
|
|
a single `seq[T]` field depending on the desired behavior.
|
|
|
|
## Configuration field types
|
|
|
|
The `confutils/defs` module provides a number of types frequently used
|
|
for configuration purposes:
|
|
|
|
#### `InputFile`, `InputDir`
|
|
|
|
Confutils will validate that the file/directory exists and that it can
|
|
be read by the current user.
|
|
|
|
#### `ConfigFilePath[Format]`
|
|
|
|
A file system path pointing to a configuration file in the specific format.
|
|
The actual configuration can be loaded by calling `load(path, ConfigType)`.
|
|
When the format is `WindowsRegistry` the path should indicate a registry key.
|
|
|
|
#### `OutPath`
|
|
|
|
A valid path must be given.
|
|
|
|
--------------
|
|
|
|
Furthermore, you can extend the behavior of the library by providing
|
|
overloads such as:
|
|
|
|
```nim
|
|
proc parseCmdArg*(T: type Foo, p: TaintedString): T =
|
|
## This provides parsing and validation for fields having the `Foo` type.
|
|
## You should raise `ConfigurationError` in case of detected problems.
|
|
...
|
|
|
|
proc humaneTypeName*[T](_: type MyList[T]): string =
|
|
## The returned string will be used in the help messages produced by the
|
|
## library to describe the expected type of the configuration option.
|
|
mixin humaneTypeName
|
|
return "list of " & humaneTypeName(T)
|
|
```
|
|
|
|
For config files, Confutils can work with any format supported by the
|
|
[nim-serialization](https://github.com/status-im/nim-serialization/) library
|
|
and it will use the standard serialization routines defined for the field
|
|
types in this format. Fields marked with the `command` or `argument` pragmas
|
|
will be ignored.
|
|
|
|
## Handling of command-line options
|
|
|
|
Confutils includes parsers that can mimic several traditional styles of
|
|
command line interfaces. You can select the parser being used by specifying
|
|
the `CmdParser` option when calling the configuration loading APIs.
|
|
|
|
The default parser of Confutils is called `MixedCmdParser`. It tries to follow
|
|
the [robustness principle](https://en.wikipedia.org/wiki/Robustness_principle)
|
|
by recognizing as many styles of passing command-line switches as possible.
|
|
A prefix of `--` is used to indicate a long option name, while the `-` prefix
|
|
uses the short option name. Multiple short options such as `-a`, `-b` and
|
|
`-c` can be combined into a single `-abc` string. Both the long and the short
|
|
forms can also be prefixed with `/` in the style of Windows utilities. The
|
|
option names are matched in case-insensitive fashion and certain characters
|
|
such as `_` and `-` will be ignored. The values can be separated from the
|
|
option names with a space, colon or an equal sign. `bool` flags default to
|
|
`false` and merely including them in the command line sets them to `true`.
|
|
|
|
Other provided choices are `UnixCmdParser`, `WindowsCmdParser` and `NimCmdParser`
|
|
which are based on more strict grammars following the most established
|
|
tradition of the respective platforms. All of the discussed parsers are
|
|
defined in terms of the lower-level parametric type `CustomCmdParser` that
|
|
can be tweaked further for a more custom behavior.
|
|
|
|
Please note that the choice of `CmdParser` will also affect the formatting
|
|
of the help messages. Please see the definition of the standard [Windows][WIN_CMD]
|
|
or [Posix][POSIX_CMD] command-line help syntax for mode details.
|
|
|
|
[WIN_CMD]: https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/command-line-syntax-key
|
|
[POSIX_CMD]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html
|
|
|
|
### Using sub-commands
|
|
|
|
As seen in the [introduction example](#introduction), Confutils makes it
|
|
easy to create command-line interfaces featuring sub-commands in the style
|
|
of `git` or `nimble`. The structure of the sub-command tree is encoded as
|
|
a Nim case object where the sub-command name is represented by an `enum`
|
|
field having the `command` pragma. Any nested fields will be considered
|
|
options of the particular sub-command. The top-level fields will be shared
|
|
between all sub-commands.
|
|
|
|
For each available choice of command and options, Confutils will automatically
|
|
provide a `help` command and the following additional switches:
|
|
|
|
* `-h` will print a short syntax reminder for the command
|
|
* `--help` will print a full help message (just like the `help` command)
|
|
|
|
## Handling of environment variables and config files
|
|
|
|
After parsing the command line options, the default behavior of Confutils is
|
|
to try to fill any missing options by examining the contents of the environment
|
|
variables plus two per-user and system-wide configuration locations derived from
|
|
the program name. If you want to use Confutils only as a command-line processor
|
|
or a config file parser for example, you can supply an empty/nil value to the
|
|
`cmdLine`, `envTable` or `configFileEnumerator` parameters of the `load` call.
|
|
|
|
More specifically, the `load` call supports the following parameters:
|
|
|
|
#### `cmdLine`, `envTable`
|
|
|
|
The command-line parameters and the environment table of the program.
|
|
By default, these will be obtained through Nim's `os` module.
|
|
|
|
#### `EnvValuesFormat`, `envVarsPrefix`
|
|
|
|
A nim-serialization format used to deserialize the values of environment
|
|
variables. The default format is called `CmdLineFormat` and it uses the
|
|
same `parseCmdArg` calls responsible for parsing the command-line.
|
|
|
|
The names of the environment variables are prefixed by the name of the
|
|
program by default. They are matched in case-insensitive fashion and
|
|
certain characters such as `-` and `_` are ignored.
|
|
|
|
#### `appName`
|
|
|
|
By default, this is obtained by calling Nim's `os.getAppFilename`.
|
|
The specified value is used to determine the default config file location.
|
|
|
|
#### `vendorName`
|
|
|
|
This is equal to `appName` unless overridden.
|
|
The specified value is used to determine the default config file location.
|
|
|
|
#### `configFileEnumerator`
|
|
|
|
A function responsible for returning a sequence of `ConfigFilePath` objects.
|
|
To support heterogenous config file types, you can also return a tuple of
|
|
sequences. The default behavior of Windows is to obtain the configuration
|
|
from the Windows registry by looking at the following keys:
|
|
|
|
```
|
|
HKEY_CURRENT_USER/SOFTWARE/{appVendor}/{appName}/
|
|
HKEY_LOCAL_MACHINE/SOFTWARE/{appVendor}/{appName}/
|
|
```
|
|
|
|
On Posix systems, the default behavior is attempt to load the configuration
|
|
from the following files:
|
|
|
|
```
|
|
/$HOME/.config/{appName}.{ConfigFileFormat.extension}
|
|
/etc/{appName}.{ConfigFileForamt.extension}
|
|
```
|
|
|
|
#### `ConfigFileFormat`
|
|
|
|
A [nim-serialization](https://github.com/status-im/nim-serialization) format
|
|
that will be used by default by Confutils.
|
|
|
|
## Customization of the help messages
|
|
|
|
The `load` call offers few more optional parameters for modifying the
|
|
produced help messages:
|
|
|
|
#### `bannerBeforeHelp`
|
|
|
|
A copyright banner or a similar message that will appear before the
|
|
automatically generated help messages.
|
|
|
|
#### `bannerAfterHelp`
|
|
|
|
A copyright banner or a similar message that will appear after the
|
|
automatically generated help messages.
|
|
|
|
#### `version`
|
|
|
|
If you provide this parameter, Confutils will automatically respond
|
|
to the standard `--version` switch. If sub-commands are used, an
|
|
additional `version` top-level command will be inserted as well.
|
|
|
|
## Compile-time options
|
|
|
|
#### `confutils_colors`
|
|
|
|
This option controls the use of colors appearing in the help messages
|
|
produced by Confutils. Possible values are:
|
|
|
|
- `NativeColors` (used by default)
|
|
|
|
In this mode, Windows builds will produce output suitable for the console
|
|
application in older versions of Windows. On Unix-like systems, this is
|
|
equivalent to specifying `AnsiColors`.
|
|
|
|
- `AnsiColors`
|
|
|
|
Output suitable for terminals supporting the standard ANSI escape codes:
|
|
https://en.wikipedia.org/wiki/ANSI_escape_code
|
|
|
|
This includes most terminal emulators on modern Unix-like systems,
|
|
Windows console replacements such as ConEmu, and the native Console
|
|
and PowerShell applications on Windows 10.
|
|
|
|
- `None` or `NoColors`
|
|
|
|
All output will be colorless.
|
|
|
|
## Contributing
|
|
|
|
The development of Confutils is sponsored by [Status.im](https://status.im/)
|
|
through the use of [GitCoin](https://gitcoin.co/). Please take a look at our
|
|
tracker for any issues having the [bounty][BOUNTIES] tag.
|
|
|
|
When submitting pull requests, please add test cases for any new features
|
|
or fixes and make sure `nimble test` is still able to execute the entire
|
|
test suite successfully.
|
|
|
|
[BOUNTIES]: https://github.com/status-im/nim-confutils/issues?q=is%3Aissue+is%3Aopen+label%3Abounty
|
|
|
|
## License
|
|
|
|
Licensed and distributed under either of
|
|
|
|
* MIT license: [LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT
|
|
|
|
or
|
|
|
|
* Apache License, Version 2.0, ([LICENSE-APACHEv2](LICENSE-APACHEv2) or http://www.apache.org/licenses/LICENSE-2.0)
|
|
|
|
at your option. This file may not be copied, modified, or distributed except according to those terms.
|
|
|