beacon node sim: various improvements (fixes #111) (#156)

* allow running more or fewer validators
* use deterministic key generation for tests to avoid exhausting system
RNG
* update README with simulator docs
* write the data of each validator to separate file, instead of a big
chainstart.json (makes it easier to run different validator counts)
This commit is contained in:
Jacek Sieka 2019-03-07 07:59:28 -06:00 committed by Dustin Brody
parent 5d0de00168
commit 4d55cf8eea
11 changed files with 167 additions and 127 deletions

113
README.md
View File

@ -5,9 +5,13 @@
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
![Stability: experimental](https://img.shields.io/badge/stability-experimental-orange.svg)
An alternative implementation of the Ethereum beacon chain in Nim.
Nimbus beacon chain is a research implementation of the beacon chain component of the upcoming Ethereum Serenity upgrade, aka eth2. See the main [Nimbus](https://github.com/status-im/nimbus/) project for the bigger picture.
Please see [Full Beacon chain](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md) specs and the Ethereum Foundation [reference implementation](https://github.com/ethereum/beacon_chain).
## Related
* [status-im/nimbus](https://github.com/status-im/nimbus/): main Nimbus repository - start here to learn more about the Nimbus eco-system
* [ethereum/eth2.0-specs](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md): Serenity specification that this project implements
* [ethereum/beacon_chain]((https://github.com/ethereum/beacon_chain): reference implementation from the Ethereum foundation
You can check where the beacon chain fits in the Ethereum research ecosystem in the [Status Athenaeum](https://github.com/status-im/athenaeum/blob/b465626cc551e361492e56d32517b2cdadd7493f/ethereum_research_records.json#L38).
@ -15,6 +19,73 @@ You can check where the beacon chain fits in the Ethereum research ecosystem in
The Eth 2.0 test vectors and their generators are available in a [dedicated repository](https://github.com/status-im/eth2-testgen).
## Building and Testing
The beacon chain components require that you have Nim installed - the easiest way to get started is to head over to the main [Nimbus](https://github.com/status-im/nimbus/) repository and follow the build instructions there. Once you're able to build nimbus:
Then:
```bash
# Or wherever you cloned nimbus
cd nimbus
# Prep environment
make
./env.sh bash
# You're now in a shell environment that has the right Nim version available
cd..
git clone https://github.com/status-im/nim-beacon-chain
cd nim-beacon-chain
# Build binaries and run test suite
nimble build
nimble test
```
This should produce some passing tests.
## Beacon node simulation
The beacon node simulation is will create a full peer-to-peer network of beacon nodes and validators, and run the beacon chain in real time. To change network parameters such as shard and validator counts, see [start.sh](tests/simulation/start.sh).
```bash
# Start beacon chain simulation, resuming from the previous state (if any)
./tests/simulation/start.sh
# Clear data from last run and restart simulation with a new genesis block
rf -rf tests/simulation/data ; ./tests/simulation/start.sh
# Run an extra node - by default the network will launch with 9 nodes, each
# hosting 10 validators. The last 10 validators are lazy bums that hid from the
# startup script, but you can command them back to work in a separate terminal
# with:
./tests/simulation/run_node.sh 9
```
You can also separate the output from each beacon node in its own panel, using [multitail](http://www.vanheusden.com/multitail/):
```bash
USE_MULTITAIL="yes" ./tests/simulation/start.sh
```
You can find out more about it in the [development update](https://our.status.im/nimbus-development-update-2018-12-2/).
_Alternatively, fire up our [experimental Vagrant instance with Nim pre-installed](https://our.status.im/setting-up-a-local-vagrant-environment-for-nim-development/) and give us yout feedback about the process!_
## State transition simulation
The state transition simulator can quickly run the Beacon chain state transition function in isolation and output JSON snapshots of the state. The simulation runs without networking and blocks are processed without slot time delays.
```bash
cd research
# build and run state simulator, then display its help - -d:release speeds it
# up substantially, allowing the simulation of longer runs in reasonable time
nim c -d:release -r state_sim --help
```
## Convention
Ethereum Foundation uses:
@ -29,44 +100,6 @@ Nim NEP-1 recommends:
To facilitate collaboration and comparison, Nim-beacon-chain uses the Ethereum Foundation convention.
## Installation
You can install the developement version of the library through nimble with the following command
```
nimble install https://github.com/status-im/nim-beacon-chain@#master
```
## Building and Testing
To try out the implementation, please first make sure you have a [Nim environment configured](https://bitfalls.com/2018/10/09/introduction-into-the-nim-language/).
_Alternatively, fire up our [experimental Vagrant instance with Nim pre-installed](https://our.status.im/setting-up-a-local-vagrant-environment-for-nim-development/) and give us yout feedback about the process!_
Then:
```bash
git clone https://github.com/status-im/nim-beacon-chain
cd nim-beacon-chain
nimble install
nimble test
```
This should produce some passing tests.
Additionally, you can run our simulation which generates a genesis file from some randomly generated validators. It then fires up 10 beacon nodes (each hosting 9 validators) which talk to each other and try to do state transitions. The simulation can be run by executing:
```bash
bash tests/simulation/start.sh
```
You can also separate the output from each beacon node in its own panel, using [multitail](http://www.vanheusden.com/multitail/):
```bash
USE_MULTITAIL="yes" ./tests/simulation/start.sh
```
You can find out more about it in the [development update](https://our.status.im/nimbus-development-update-2018-12-2/).
## License
Licensed and distributed under either of

View File

@ -177,9 +177,8 @@ template findIt(s: openarray, predicate: untyped): int =
res
proc addLocalValidators*(node: BeaconNode) =
for validator in node.config.validators:
for privKey in node.config.validators:
let
privKey = validator.privKey
pubKey = privKey.pubKey()
let idx = node.state.data.validator_registry.findIt(it.pubKey == pubKey)
@ -276,7 +275,7 @@ proc makeAttestation(node: BeaconNode,
beacon_block_root: node.state.blck.root,
epoch_boundary_root: Eth2Digest(), # TODO
shard_block_root: Eth2Digest(), # TODO
latest_crosslink: Crosslink(epoch: state.latest_crosslinks[shard].epoch),
latest_crosslink: state.latest_crosslinks[shard],
justified_epoch: state.justified_epoch,
justified_block_root: justifiedBlockRoot)
@ -632,8 +631,8 @@ when isMainModule:
case config.cmd
of createChain:
createStateSnapshot(
config.chainStartupData, config.genesisOffset,
config.outputStateFile.string)
config.validatorsDir.string, config.numValidators, config.firstValidator,
config.genesisOffset, config.outputStateFile.string)
quit 0
of noCommand:

View File

@ -14,12 +14,6 @@ type
noCommand
createChain
ChainStartupData* = object
validatorDeposits*: seq[Deposit]
PrivateValidatorData* = object
privKey*: ValidatorPrivKey
BeaconNodeConf* = object
logLevel* {.
desc: "Sets the log level",
@ -54,24 +48,32 @@ type
validators* {.
required
desc: "A path to a pair of public and private keys for a validator. " &
"Nimbus will automatically add the extensions .privkey and .pubkey."
desc: "Path to a validator private key, as generated by validator_keygen"
longform: "validator"
shortform: "v".}: seq[PrivateValidatorData]
shortform: "v".}: seq[ValidatorPrivKey]
stateSnapshot* {.
desc: "Json file specifying a recent state snapshot"
shortform: "s".}: Option[BeaconState]
of createChain:
chainStartupData* {.
validatorsDir* {.
desc: "Directory containing validator descriptors named vXXXXXXX.deposit.json"
shortform: "d".}: DirPath
numValidators* {.
desc: ""
shortform: "c".}: ChainStartupData
shortform: "n".}: int
firstValidator* {.
desc: "index of first validator to add to validator list"
shortform: "o"
defaultValue: 0.}: int
genesisOffset* {.
desc: "Seconds from now to add to genesis time"
shortForm: "g"
defaultValue: 0 .}: int
defaultValue: 5 .}: int
outputStateFile* {.
desc: "Output file where to write the initial state snapshot"
@ -98,5 +100,5 @@ template handledAsJsonFilename(T: untyped) {.dirty.} =
return Json.loadFile(string(input), T)
handledAsJsonFilename BeaconState
handledAsJsonFilename ChainStartupData
handledAsJsonFilename PrivateValidatorData
handledAsJsonFilename Deposit
handledAsJsonFilename ValidatorPrivKey

View File

@ -1,5 +1,5 @@
import
chronos,
ospaths, chronos, json_serialization, strformat,
spec/[datatypes, crypto, digest, beaconstate], beacon_chain_db, conf
const
@ -30,13 +30,18 @@ proc obtainTrustedStateSnapshot*(db: BeaconChainDB): Future[BeaconState] {.async
assert(false, "Not implemented")
proc createStateSnapshot*(
startup: ChainStartupData, genesisOffset: int, outFile: string) =
validatorDir: string, numValidators, firstValidator, genesisOffset: int,
outFile: string) =
var deposits: seq[Deposit]
for i in firstValidator..<numValidators:
deposits.add Json.loadFile(validatorDir / &"v{i:07}.deposit.json", Deposit)
let initialState = get_genesis_beacon_state(
startup.validatorDeposits,
deposits,
uint64(int(fastEpochTime() div 1000) + genesisOffset),
Eth1Data(), {})
var vr: Validator
Json.saveFile(outFile, initialState, pretty = true)
echo "Wrote ", outFile

View File

@ -8,38 +8,21 @@ proc writeFile(filename: string, value: auto) =
Json.saveFile(filename, value, pretty = true)
echo "Wrote ", filename
proc genSingleValidator(path: string): (ValidatorPubKey,
ValidatorPrivKey) =
var v: PrivateValidatorData
v.privKey = newPrivKey()
cli do (validators: int = 100000,
outputDir: string = "validators"):
writeFile(path, v)
for i in 0 ..< validators:
# there can apparently be tops 4M validators so we use 7 digits..
let
depositFn = outputDir / &"v{i:07}.deposit.json"
privKeyFn = outputDir / &"v{i:07}.privkey.json"
assert v.privKey != ValidatorPrivKey(), "Private key shouldn't be zero"
return (v.privKey.pubKey(), v.privKey)
if existsFile(depositFn) and existsFile(privKeyFn):
continue
# TODO: Make these more comprehensive and find them a new home
type
Ether* = distinct int64
GWei* = distinct int64
template eth*(x: SomeInteger): Ether = Ether(x)
template gwei*(x: Ether): Gwei = Gwei(int(x) * 1000000000)
cli do (validators: int,
outputDir: string,
startupDelay = 0):
if validators < 64:
echo "The number of validators must be higher than ", SLOTS_PER_EPOCH, " (SLOTS_PER_EPOCH)"
echo "There must be at least one validator assigned per slot."
quit 1
var startupData: ChainStartupData
for i in 1 .. validators:
let (pubKey, privKey) =
genSingleValidator(outputDir / &"validator-{i:02}.json")
let
privKey = makeFakeValidatorPrivKey(i)
pubKey = privKey.pubKey()
let
withdrawalCredentials = makeFakeHash(i)
@ -53,14 +36,17 @@ cli do (validators: int,
0 # TODO - domain
)
startupData.validatorDeposits.add Deposit(
deposit_data: DepositData(
amount: MAX_DEPOSIT_AMOUNT,
timestamp: now(),
deposit_input: DepositInput(
pubkey: pubKey,
proof_of_possession: proofOfPossession,
withdrawal_credentials: withdrawalCredentials)))
let
deposit = Deposit(
deposit_data: DepositData(
amount: MAX_DEPOSIT_AMOUNT,
timestamp: now(),
deposit_input: DepositInput(
pubkey: pubKey,
proof_of_possession: proofOfPossession,
withdrawal_credentials: withdrawalCredentials)))
writeFile(outputDir / "startup.json", startupData)
writeFile(privKeyFn, privKey)
writeFile(depositFn, deposit)
echo "Keys generated by this tool are only for testing!"

View File

@ -106,11 +106,11 @@ cli do(slots = 1945,
for v in scas.committee:
if (rand(r, high(int)).float * attesterRatio).int <= high(int):
if first:
attestation = makeAttestation(state, latest_block_root, v)
attestation = makeAttestation(state, latest_block_root, v, flags)
first = false
else:
attestation.combine(
makeAttestation(state, latest_block_root, v), flags)
makeAttestation(state, latest_block_root, v, flags), flags)
if not first:
# add the attestation if any of the validators attested, as given
@ -138,6 +138,10 @@ cli do(slots = 1945,
echo "All time are ms"
echo &"{\"Average\" :>12}, {\"StdDev\" :>12}, {\"Min\" :>12}, " &
&"{\"Max\" :>12}, {\"Samples\" :>12}, {\"Test\" :>12}"
if not validate:
echo "Validation is turned off meaning that no BLS operations are performed"
for t in Timers:
echo fmtTime(timers[t].mean), fmtTime(timers[t].standardDeviationS),
fmtTime(timers[t].min), fmtTime(timers[t].max), &"{timers[t].n :>12}, ",

View File

@ -1,3 +1,3 @@
data/
startup/
validators/

View File

@ -12,18 +12,22 @@ fi
DATA_DIR=$SIMULATION_DIR/node-${1}
V_PREFIX="$VALIDATORS_DIR/v$(printf '%06d' ${1})"
PORT=$(printf '5%04d' ${1})
$BEACON_NODE_BIN \
--dataDir:$DATA_DIR \
--validator:$STARTUP_DIR/validator-${1}1.json \
--validator:$STARTUP_DIR/validator-${1}2.json \
--validator:$STARTUP_DIR/validator-${1}3.json \
--validator:$STARTUP_DIR/validator-${1}4.json \
--validator:$STARTUP_DIR/validator-${1}5.json \
--validator:$STARTUP_DIR/validator-${1}6.json \
--validator:$STARTUP_DIR/validator-${1}7.json \
--validator:$STARTUP_DIR/validator-${1}8.json \
--validator:$STARTUP_DIR/validator-${1}9.json \
--tcpPort:5000${1} \
--udpPort:5000${1} \
--validator:${V_PREFIX}0.privkey.json \
--validator:${V_PREFIX}1.privkey.json \
--validator:${V_PREFIX}2.privkey.json \
--validator:${V_PREFIX}3.privkey.json \
--validator:${V_PREFIX}4.privkey.json \
--validator:${V_PREFIX}5.privkey.json \
--validator:${V_PREFIX}6.privkey.json \
--validator:${V_PREFIX}7.privkey.json \
--validator:${V_PREFIX}8.privkey.json \
--validator:${V_PREFIX}9.privkey.json \
--tcpPort:$PORT \
--udpPort:$PORT \
--stateSnapshot:$SNAPSHOT_FILE \
$BOOTSTRAP_NODES_FLAG

View File

@ -10,11 +10,12 @@ trap "trap - SIGTERM && kill -- -$$" SIGINT SIGTERM EXIT
# Set a default value for the env vars usually supplied by nimbus Makefile
NUMBER_OF_VALIDATORS=99
NUM_VALIDATORS=${1:-100}
NUM_NODES=${2:-9}
cd $SIM_ROOT
mkdir -p "$SIMULATION_DIR"
mkdir -p "$STARTUP_DIR"
mkdir -p "$VALIDATORS_DIR"
cd $GIT_ROOT
mkdir -p $BUILD_OUTPUTS_DIR
@ -24,12 +25,15 @@ DEFS="-d:SHARD_COUNT=${SHARD_COUNT:-4} " # Spec default: 1024
DEFS+="-d:SLOTS_PER_EPOCH=${SLOTS_PER_EPOCH:-8} " # Spec default: 64
DEFS+="-d:SECONDS_PER_SLOT=${SECONDS_PER_SLOT:-6} " # Spec default: 6
if [ ! -f $STARTUP_FILE ]; then
LAST_VALIDATOR_NUM=$(( $NUM_VALIDATORS - 1 ))
LAST_VALIDATOR="$VALIDATORS_DIR/v$(printf '%07d' $LAST_VALIDATOR_NUM).deposit.json"
if [ ! -f $LAST_VALIDATOR ]; then
if [[ -z "$SKIP_BUILDS" ]]; then
nim c -o:"$VALIDATOR_KEYGEN_BIN" $DEFS -d:release beacon_chain/validator_keygen
fi
$VALIDATOR_KEYGEN_BIN --validators=$NUMBER_OF_VALIDATORS --outputDir="$STARTUP_DIR"
$VALIDATOR_KEYGEN_BIN --validators=$NUM_VALIDATORS --outputDir="$VALIDATORS_DIR"
fi
if [[ -z "$SKIP_BUILDS" ]]; then
@ -38,8 +42,10 @@ fi
if [ ! -f $SNAPSHOT_FILE ]; then
$BEACON_NODE_BIN createChain \
--chainStartupData:$STARTUP_FILE \
--out:$SNAPSHOT_FILE --genesisOffset=5 # Delay in seconds
--validatorsDir:$VALIDATORS_DIR \
--out:$SNAPSHOT_FILE \
--numValidators=$NUM_VALIDATORS \
--genesisOffset=5 # Delay in seconds
fi
# Delete any leftover address files from a previous session
@ -53,7 +59,9 @@ USE_MULTITAIL="${USE_MULTITAIL:-no}" # make it an opt-in
type "$MULTITAIL" &>/dev/null || USE_MULTITAIL="no"
COMMANDS=()
for i in $(seq 0 8); do
LAST_NODE=$(( $NUM_NODES - 1 ))
for i in $(seq 0 $LAST_NODE); do
BOOTSTRAP_NODES_FLAG="--bootstrapNodesFile:$MASTER_NODE_ADDRESS_FILE"
if [[ "$i" == "0" ]]; then

View File

@ -14,8 +14,7 @@ GIT_ROOT="$($PWD_CMD)"
: ${BUILD_OUTPUTS_DIR:="$GIT_ROOT/build"}
SIMULATION_DIR="$SIM_ROOT/data"
STARTUP_DIR="$SIM_ROOT/startup"
STARTUP_FILE="$STARTUP_DIR/startup.json"
VALIDATORS_DIR="$SIM_ROOT/validators"
SNAPSHOT_FILE="$SIMULATION_DIR/state_snapshot.json"
BEACON_NODE_BIN=$BUILD_OUTPUTS_DIR/beacon_node
VALIDATOR_KEYGEN_BIN=$BUILD_OUTPUTS_DIR/validator_keygen

View File

@ -11,7 +11,7 @@ import
../beacon_chain/[beacon_chain_db, extras, ssz, state_transition, validator_pool],
../beacon_chain/spec/[beaconstate, crypto, datatypes, digest, helpers, validator]
func makeValidatorPrivKey(i: int): ValidatorPrivKey =
func makeFakeValidatorPrivKey*(i: int): ValidatorPrivKey =
var i = i + 1 # 0 does not work, as private key...
copyMem(result.x[0].addr, i.addr, min(sizeof(result.x), sizeof(i)))
@ -24,14 +24,14 @@ func hackPrivKey(v: Validator): ValidatorPrivKey =
copyMem(
i.addr, v.withdrawal_credentials.data[0].unsafeAddr,
min(sizeof(v.withdrawal_credentials.data), sizeof(i)))
makeValidatorPrivKey(i)
makeFakeValidatorPrivKey(i)
func makeDeposit(i: int, flags: UpdateFlags): Deposit =
## Ugly hack for now: we stick the private key in withdrawal_credentials
## which means we can repro private key and randao reveal from this data,
## for testing :)
let
privkey = makeValidatorPrivKey(i)
privkey = makeFakeValidatorPrivKey(i)
pubkey = privkey.pubKey()
withdrawal_credentials = makeFakeHash(i)