nimbus-eth2/ncli
tersec 8def2486b0
immutable validator database factoring (#2297)
* initial immutable validator database factoring

* remove changes from chain_dag: this abstraction properly belongs in beacon_chain_db

* add merging mutable/immutable validator portions; individually test database roundtripping of immutable validators and states-sans-immutable-validators

* update test summaries

* use stew/assign2 instead of Nim assignment

* add reading/writing of immutable validators in chaindag

* remove unused import

* replace chunked k/v store of immutable validators with per-row SQL table storage

* use List instead of HashList

* un-stub some ncli_db code so that it uses

* switch HashArray to array; move BeaconStateNoImmutableValidators from datatypes to beacon_chain_db

* begin only-mutable-part state storage

* uncomment some assigns

* work around https://github.com/nim-lang/Nim/issues/17253

* fix most of the issues/oversights; local sim runs again

* fix test suite by adding missing beaconstate field to copy function

* have ncli bench also store immutable validators

* extract some immutable-validator-specific code from the beacon chain db module

* add more rigorous database state roundtripping, with changing validator sets

* adjust ncli_db to use new schema

* simplify putState/getState by moving all immutable validator accounting into beacon state DB

* remove redundant test case and move code to immutable-beacon-chain module

* more efficient, but still brute-force, mutable+immutable validator merging

* reuse BeaconState in getState

* ensure HashList/HashArray caches are cleared when reusing getState buffers; add ncli_db and a unit test to verify this

* HashList.clear() -> HashList.clearCache()

* only copy incrementally necessary immutable validators

* increase strictness of test cases and fix/work around resulting HashList cache invalidation issues

* remove explanatory scaffolding

* allow for storage of full (with all validators) states for backwards/forwards-compatibility

* adjust DbSeq type usage

* store full, with-validators, state every 64 epochs to enable reverting versions

* reduce memory allocation and intermediate objects in state storage codepath

* eliminate allocation/copying through intermediate BeaconStateNoImmutableValidators objects

* skip benchmarking initial genesis-validator-heavy state store

* always store new-style state and sometimes old-style state

* document intent behind BeaconState/Validator type-punnery

* more accurate failure message on SQLite in-memory database initialization failure
2021-03-15 14:11:51 +00:00
..
.gitignore rm ncli_signing_root 2019-12-17 00:06:03 +02:00
README.md update nim-beacon-chain to nimbus-eth2 in beacon_chain/, ncli/, tests/, and README.md (#1843) 2020-10-08 19:02:05 +00:00
e2store.nim e2store: add era format (#2382) 2021-03-15 11:31:39 +01:00
e2store.py e2store: add era format (#2382) 2021-03-15 11:31:39 +01:00
inspector.nim Use setupAddress for better IP and ports configuration 2021-03-06 00:09:09 +02:00
inspector.nim.cfg Move tools (#2370) 2021-03-02 15:02:10 +01:00
logtrace.nim Move tools (#2370) 2021-03-02 15:02:10 +01:00
ncli.nim Reorg (5/5) (#2377) 2021-03-05 14:12:00 +01:00
ncli_db.nim immutable validator database factoring (#2297) 2021-03-15 14:11:51 +00:00
ncli_query.nim 12x speedup on state sim with 100k validators sans BLS by caching get_beacon_proposer_index(...) 2020-06-04 17:07:51 +00:00
ncli_slashing.nim Consolidate modules by areas [part 1] (#2365) 2021-03-02 11:27:45 +01:00
nim.cfg minimize imports, specially for spec - cuts 2/3 of ncli build time (#500) 2019-10-25 12:59:56 +02:00
nimquery.nim Move tools (#2370) 2021-03-02 15:02:10 +01:00

README.md

Introduction

ncli is a set of low level / debugging tools to interact with the nimbus beacon chain specification implementation, simliar to zcli. With it, you explore SSZ, make state transitions and compute hash tree roots.

Tools

  • transition: Perform state transition given a pre-state and a block to apply (both in SSZ format)
  • hash_tree_root: Print tree root of an SSZ object
  • pretty: Pretty-print SSZ object as JSON

Building

Follow the instructions from nimbus-eth2

git clone https://github.com/status-im/nimbus-eth2.git
cd nimbus-eth2
make

Usage

# Build with minimal config
../env.sh nim c -d:const_preset=minimal ncli_transition
# Build with mainnet config
../env.sh nim c -d:const_preset=mainnet ncli_transition

# Run..
./ncli_transition --help