status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/AllTests-mainnet.md

595 lines
36 KiB
Markdown
Raw Normal View History

Add test reports
2020-03-10 04:24:33 +00:00
AllTests-mainnet
===
re-enable mainnet attestation pool tests (#946) * re-enable mainnet attestation pool tests; since https://github.com/status-im/nim-beacon-chain/pull/930 double CI state sim validators * bump deposits spec ref to v0.11.1
2020-04-29 11:44:07 +00:00
## Attestation pool processing [Preset: mainnet]
```diff
+ Attestations may arrive in any order [Preset: mainnet] OK
+ Attestations may overlap, bigger first [Preset: mainnet] OK
+ Attestations may overlap, smaller first [Preset: mainnet] OK
+ Attestations should be combined [Preset: mainnet] OK
Revamp attestation pool This is a revamp of the attestation pool that cleans up several aspects of attestation processing as the network grows larger and block space becomes more precious. The aim is to better exploit the divide between attestation subnets and aggregations by keeping the two kinds separate until it's time to either produce a block or aggregate. This means we're no longer eagerly combining single-vote attestations, but rather wait until the last moment, and then try to add singles to all aggregates, including those coming from the network. Importantly, the branch improves on poor aggregate quality and poor attestation packing in cases where block space is running out. A basic greed scoring mechanism is used to select attestations for blocks - attestations are added based on how much many new votes they bring to the table. * Collect single-vote attestations separately and store these until it's time to make aggregates * Create aggregates based on single-vote attestations * Select _best_ aggregate rather than _first_ aggregate when on aggregation duty * Top up all aggregates with singles when it's time make the attestation cut, thus improving the chances of grabbing the best aggregates out there * Improve aggregation test coverage * Improve bitseq operations * Simplify aggregate signature creation * Make attestation cache temporary instead of storing it in attestation pool - most of the time, blocks are not being produced, no need to keep the data around * Remove redundant aggregate storage that was used only for RPC * Use tables to avoid some linear seeks when looking up attestation data * Fix long cleanup on large slot jumps * Avoid some pointers * Speed up iterating all attestations for a slot (fixes #2490)
2021-04-12 20:25:09 +00:00
+ Can add and retrieve simple attestations [Preset: mainnet] OK
+ Everyone voting for something different [Preset: mainnet] OK
re-enable mainnet attestation pool tests (#946) * re-enable mainnet attestation pool tests; since https://github.com/status-im/nim-beacon-chain/pull/930 double CI state sim validators * bump deposits spec ref to v0.11.1
2020-04-29 11:44:07 +00:00
+ Fork choice returns block with attestation OK
+ Fork choice returns latest block with no attestations OK
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
+ Trying to add a block twice tags the second as an error OK
+ Trying to add a duplicate block from an old pruned epoch is tagged as an error OK
Revamp attestation pool This is a revamp of the attestation pool that cleans up several aspects of attestation processing as the network grows larger and block space becomes more precious. The aim is to better exploit the divide between attestation subnets and aggregations by keeping the two kinds separate until it's time to either produce a block or aggregate. This means we're no longer eagerly combining single-vote attestations, but rather wait until the last moment, and then try to add singles to all aggregates, including those coming from the network. Importantly, the branch improves on poor aggregate quality and poor attestation packing in cases where block space is running out. A basic greed scoring mechanism is used to select attestations for blocks - attestations are added based on how much many new votes they bring to the table. * Collect single-vote attestations separately and store these until it's time to make aggregates * Create aggregates based on single-vote attestations * Select _best_ aggregate rather than _first_ aggregate when on aggregation duty * Top up all aggregates with singles when it's time make the attestation cut, thus improving the chances of grabbing the best aggregates out there * Improve aggregation test coverage * Improve bitseq operations * Simplify aggregate signature creation * Make attestation cache temporary instead of storing it in attestation pool - most of the time, blocks are not being produced, no need to keep the data around * Remove redundant aggregate storage that was used only for RPC * Use tables to avoid some linear seeks when looking up attestation data * Fix long cleanup on large slot jumps * Avoid some pointers * Speed up iterating all attestations for a slot (fixes #2490)
2021-04-12 20:25:09 +00:00
+ Working with aggregates [Preset: mainnet] OK
re-enable mainnet attestation pool tests (#946) * re-enable mainnet attestation pool tests; since https://github.com/status-im/nim-beacon-chain/pull/930 double CI state sim validators * bump deposits spec ref to v0.11.1
2020-04-29 11:44:07 +00:00
```
Revamp attestation pool This is a revamp of the attestation pool that cleans up several aspects of attestation processing as the network grows larger and block space becomes more precious. The aim is to better exploit the divide between attestation subnets and aggregations by keeping the two kinds separate until it's time to either produce a block or aggregate. This means we're no longer eagerly combining single-vote attestations, but rather wait until the last moment, and then try to add singles to all aggregates, including those coming from the network. Importantly, the branch improves on poor aggregate quality and poor attestation packing in cases where block space is running out. A basic greed scoring mechanism is used to select attestations for blocks - attestations are added based on how much many new votes they bring to the table. * Collect single-vote attestations separately and store these until it's time to make aggregates * Create aggregates based on single-vote attestations * Select _best_ aggregate rather than _first_ aggregate when on aggregation duty * Top up all aggregates with singles when it's time make the attestation cut, thus improving the chances of grabbing the best aggregates out there * Improve aggregation test coverage * Improve bitseq operations * Simplify aggregate signature creation * Make attestation cache temporary instead of storing it in attestation pool - most of the time, blocks are not being produced, no need to keep the data around * Remove redundant aggregate storage that was used only for RPC * Use tables to avoid some linear seeks when looking up attestation data * Fix long cleanup on large slot jumps * Avoid some pointers * Speed up iterating all attestations for a slot (fixes #2490)
2021-04-12 20:25:09 +00:00
OK: 11/11 Fail: 0/11 Skip: 0/11
Backfill support for ChainDAG (#3171) In the ChainDAG, 3 block pointers are kept: genesis, tail and head. This PR adds one more block pointer: the backfill block which represents the block that has been backfilled so far. When doing a checkpoint sync, a random block is given as starting point - this is the tail block, and we require that the tail block has a corresponding state. When backfilling, we end up with blocks without corresponding states, hence we cannot use `tail` as a backfill pointer - there is no state. Nonetheless, we need to keep track of where we are in the backfill process between restarts, such that we can answer GetBeaconBlocksByRange requests. This PR adds the basic support for backfill handling - it needs to be integrated with backfill sync, and the REST API needs to be adjusted to take advantage of the new backfilled blocks when responding to certain requests. Future work will also enable moving the tail in either direction: * pruning means moving the tail forward in time and removing states * backwards means recreating past states from genesis, such that intermediate states are recreated step by step all the way to the tail - at that point, tail, genesis and backfill will match up. * backfilling is done when backfill != genesis - later, this will be the WSS checkpoint instead
2021-12-13 13:36:06 +00:00
## Backfill
```diff
+ backfill to genesis OK
+ reload backfill position OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
Add test reports
2020-03-10 04:24:33 +00:00
## Beacon chain DB [Preset: mainnet]
```diff
+ empty database [Preset: mainnet] OK
+ find ancestors [Preset: mainnet] OK
add blockchain_dag altair database reading; add rollback tests (#2683) * add blockchain_dag altair database reading; add rollback tests; fix some unnecessary type conversions * remove debugging scaffolding * proposeSignedBlock() will need to be async for merge; introduce altair types to VC
2021-06-29 15:09:29 +00:00
+ sanity check Altair and cross-fork getState rollback [Preset: mainnet] OK
introduce database support for Altair (#2667) * introduce immutable Altair BeaconState * add database support for Altair blocks and states * add tests for Altair get/put/contains/delete state * enable blockchain_dag Altair state database storing * properly return error on getting missing altair block
2021-06-24 07:11:47 +00:00
+ sanity check Altair blocks [Preset: mainnet] OK
+ sanity check Altair states [Preset: mainnet] OK
+ sanity check Altair states, reusing buffers [Preset: mainnet] OK
rename On{Merge,Bellatrix}BlockAdded and Rollback{Merge,Bellatrix}HashedProc (#3321)
2022-01-26 12:21:29 +00:00
+ sanity check Bellatrix and cross-fork getState rollback [Preset: mainnet] OK
+ sanity check Bellatrix blocks [Preset: mainnet] OK
+ sanity check Bellatrix states [Preset: mainnet] OK
+ sanity check Bellatrix states, reusing buffers [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ sanity check genesis roundtrip [Preset: mainnet] OK
introduce database support for Altair (#2667) * introduce immutable Altair BeaconState * add database support for Altair blocks and states * add tests for Altair get/put/contains/delete state * enable blockchain_dag Altair state database storing * properly return error on getting missing altair block
2021-06-24 07:11:47 +00:00
+ sanity check phase 0 blocks [Preset: mainnet] OK
add blockchain_dag altair database reading; add rollback tests (#2683) * add blockchain_dag altair database reading; add rollback tests; fix some unnecessary type conversions * remove debugging scaffolding * proposeSignedBlock() will need to be async for merge; introduce altair types to VC
2021-06-29 15:09:29 +00:00
+ sanity check phase 0 getState rollback [Preset: mainnet] OK
introduce database support for Altair (#2667) * introduce immutable Altair BeaconState * add database support for Altair blocks and states * add tests for Altair get/put/contains/delete state * enable blockchain_dag Altair state database storing * properly return error on getting missing altair block
2021-06-24 07:11:47 +00:00
+ sanity check phase 0 states [Preset: mainnet] OK
+ sanity check phase 0 states, reusing buffers [Preset: mainnet] OK
initial infrastructure for state diffs (#2087) Initial infrastructure for state diffs
2021-01-18 20:34:41 +00:00
+ sanity check state diff roundtrip [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
```
merge hardfork database support (#2911) * merge hardfork database support * working block_sim * recreate state transition changes
2021-09-30 01:07:24 +00:00
OK: 16/16 Fail: 0/16 Skip: 0/16
Add test reports
2020-03-10 04:24:33 +00:00
## Beacon state [Preset: mainnet]
```diff
rename initialize_beacon_state to initialize_beacon_state_from_eth1 (#2536)
2021-05-04 10:19:11 +00:00
+ Smoke test initialize_beacon_state_from_eth1 [Preset: mainnet] OK
move dependent root computations to `BeaconState` / `EpochRef` (#3478) * fewer deps on `BlockRef` traversal in anticipation of pruning * allows identifying EpochRef:s by their shuffling as a first step of * tighten error handling around missing blocks using the zero hash for signalling "missing block" is fragile and easy to miss - with checkpoint sync now, and pruning in the future, missing blocks become "normal".
2022-03-15 08:24:55 +00:00
+ dependent_root OK
ncli_db: add putState, putBlock (#3096) * ncli_db: add putState, putBlock These tools allow modifying an existing nimbus database for the purpose of recovery or reorg, moving the head, tail and genesis to arbitrary points. * remove potentially expensive `putState` in `BeaconStateDB` * introduce `latest_block_root` which computes the root of the latest applied block from the `latest_block_header` field (instead of passing it in separately) * avoid some unnecessary BeaconState copies during init * discover https://github.com/nim-lang/Nim/issues/19094 * prefer `HashedBeaconState` in a few places to avoid recomputing state root * fetch latest block root from state when creating blocks * harden `get_beacon_proposer_index` against invalid slots and document * move random spec function tests to `test_spec.nim` * avoid unnecessary state root computation before block proposal
2021-11-18 12:02:43 +00:00
+ get_beacon_proposer_index OK
+ latest_block_root OK
fix Nim 1.6 deprecation and unused import warnings (#3834)
2022-07-01 21:52:23 +00:00
+ merklizer state roundtrip OK
More efficient reward data persistance; Address review comments The new format is based on compressed CSV files in two channels: * Detailed per-epoch data * Aggregated "daily" summaries The use of append-only CSV file speeds up significantly the epoch processing speed during data generation. The use of compression results in smaller storage requirements overall. The use of the aggregated files has a very minor cost in both CPU and storage, but leads to near interactive speed for report generation. Other changes: - Implemented support for graceful shut downs to avoid corrupting the saved files. - Fixed a memory leak caused by lacking `StateCache` clean up on each iteration. - Addressed review comments - Moved the rewards and penalties calculation code in a separate module Required invasive changes to existing modules: - The `data` field of the `KeyedBlockRef` type is made public to be used by the validator rewards monitor's Chain DAG update procedure. - The `getForkedBlock` procedure from the `blockchain_dag.nim` module is made public to be used by the validator rewards monitor's Chain DAG update procedure.
2022-01-17 12:58:33 +00:00
+ process_slots OK
Add test reports
2020-03-10 04:24:33 +00:00
```
fix Nim 1.6 deprecation and unused import warnings (#3834)
2022-07-01 21:52:23 +00:00
OK: 6/6 Fail: 0/6 Skip: 0/6
time: spring cleaning (#3262) Time in the beacon chain is expressed relative to the genesis time - this PR creates a `beacon_time` module that collects helpers and utilities for dealing the time units - the new module does not deal with actual wall time (that's remains in `beacon_clock`). Collecting the time related stuff in one place makes it easier to find, avoids some circular imports and allows more easily identifying the code actually needs wall time to operate. * move genesis-time-related functionality into `spec/beacon_time` * avoid using `chronos.Duration` for time differences - it does not support negative values (such as when something happens earlier than it should) * saturate conversions between `FAR_FUTURE_XXX`, so as to avoid overflows * fix delay reporting in validator client so it uses the expected deadline of the slot, not "closest wall slot" * simplify looping over the slots of an epoch * `compute_start_slot_at_epoch` -> `start_slot` * `compute_epoch_at_slot` -> `epoch` A follow-up PR will (likely) introduce saturating arithmetic for the time units - this is merely code moves, renames and fixing of small bugs.
2022-01-11 10:01:54 +00:00
## Beacon time
```diff
+ basics OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Speed up altair block processing 2x (#3115) * Speed up altair block processing >2x Like #3089, this PR drastially speeds up historical REST queries and other long state replays. * cache sync committee validator indices * use ~80mb less memory for validator pubkey mappings * batch-verify sync aggregate signature (fixes #2985) * document sync committee hack with head block vs sync message block * add batch signature verification failure tests Before: ``` ../env.sh nim c -d:release -r ncli_db --db:mainnet_0/db bench --start-slot:-1000 All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 5830.675, 0.000, 5830.675, 5830.675, 1, Initialize DB 0.481, 1.878, 0.215, 59.167, 981, Load block from database 8422.566, 0.000, 8422.566, 8422.566, 1, Load state from database 6.996, 1.678, 0.042, 14.385, 969, Advance slot, non-epoch 93.217, 8.318, 84.192, 122.209, 32, Advance slot, epoch 20.513, 23.665, 11.510, 201.561, 981, Apply block, no slot processing 0.000, 0.000, 0.000, 0.000, 0, Database load 0.000, 0.000, 0.000, 0.000, 0, Database store ``` After: ``` 7081.422, 0.000, 7081.422, 7081.422, 1, Initialize DB 0.553, 2.122, 0.175, 66.692, 981, Load block from database 5439.446, 0.000, 5439.446, 5439.446, 1, Load state from database 6.829, 1.575, 0.043, 12.156, 969, Advance slot, non-epoch 94.716, 2.749, 88.395, 100.026, 32, Advance slot, epoch 11.636, 23.766, 4.889, 205.250, 981, Apply block, no slot processing 0.000, 0.000, 0.000, 0.000, 0, Database load 0.000, 0.000, 0.000, 0.000, 0, Database store ``` * add comment
2021-11-24 12:43:50 +00:00
## Block pool altair processing [Preset: mainnet]
```diff
+ Invalid signatures [Preset: mainnet] OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
## Block pool processing [Preset: mainnet]
```diff
Validator monitoring (#2925) Validator monitoring based on and mostly compatible with the implementation in Lighthouse - tracks additional logs and metrics for specified validators so as to stay on top on performance. The implementation works more or less the following way: * Validator pubkeys are singled out for monitoring - these can be running on the node or not * For every action that the validator takes, we record steps in the process such as messages being seen on the network or published in the API * When the dust settles at the end of an epoch, we report the information from one epoch before that, which coincides with the balances being updated - this is a tradeoff between being correct (waiting for finalization) and providing relevant information in a timely manner)
2021-12-20 19:20:31 +00:00
+ Adding the same block twice returns a Duplicate error [Preset: mainnet] OK
test randao skippping (complements #3837) (#4179)
2022-09-27 07:22:24 +00:00
+ Randao skip and non-skip OK
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
+ Simple block add&get [Preset: mainnet] OK
Prune `BlockRef` on finalization (#3513) Up til now, the block dag has been using `BlockRef`, a structure adapted for a full DAG, to represent all of chain history. This is a correct and simple design, but does not exploit the linearity of the chain once parts of it finalize. By pruning the in-memory `BlockRef` structure at finalization, we save, at the time of writing, a cool ~250mb (or 25%:ish) chunk of memory landing us at a steady state of ~750mb normal memory usage for a validating node. Above all though, we prevent memory usage from growing proportionally with the length of the chain, something that would not be sustainable over time - instead, the steady state memory usage is roughly determined by the validator set size which grows much more slowly. With these changes, the core should remain sustainable memory-wise post-merge all the way to withdrawals (when the validator set is expected to grow). In-memory indices are still used for the "hot" unfinalized portion of the chain - this ensure that consensus performance remains unchanged. What changes is that for historical access, we use a db-based linear slot index which is cache-and-disk-friendly, keeping the cost for accessing historical data at a similar level as before, achieving the savings at no percievable cost to functionality or performance. A nice collateral benefit is the almost-instant startup since we no longer load any large indicies at dag init. The cost of this functionality instead can be found in the complexity of having to deal with two ways of traversing the chain - by `BlockRef` and by slot. * use `BlockId` instead of `BlockRef` where finalized / historical data may be required * simplify clearance pre-advancement * remove dag.finalizedBlocks (~50:ish mb) * remove `getBlockAtSlot` - use `getBlockIdAtSlot` instead * `parent` and `atSlot` for `BlockId` now require a `ChainDAGRef` instance, unlike `BlockRef` traversal * prune `BlockRef` parents on finality (~200:ish mb) * speed up ChainDAG init by not loading finalized history index * mess up light client server error handling - this need revisiting :)
2022-03-17 17:42:56 +00:00
+ basic ops OK
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
+ updateHead updates head and headState [Preset: mainnet] OK
remove StateData (#3507) One more step on the journey to reduce `BlockRef` usage across the codebase - this one gets rid of `StateData` whose job was to keep track of which block was last assigned to a state - these duties have now been taken over by `latest_block_root`, a fairly recent addition that computes this block root from state data (at a small cost that should be insignificant) 99% mechanical change.
2022-03-16 07:20:40 +00:00
+ updateState sanity [Preset: mainnet] OK
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
```
test randao skippping (complements #3837) (#4179)
2022-09-27 07:22:24 +00:00
OK: 6/6 Fail: 0/6 Skip: 0/6
move quarantine outside of chaindag (#3124) * move quarantine outside of chaindag The quarantine has been part of the ChainDAG for the longest time, but this design has a few issues: * the function in which blocks are verified and added to the dag becomes reentrant and therefore difficult to reason about - we're currently using a stateful flag to work around it * quarantined blocks bypass the processing queue leading to a processing stampede * the quarantine flow is unsuitable for orphaned attestations - these should also should be quarantined eventually Instead of processing the quarantine inside ChainDAG, this PR moves re-queueing to `block_processor` which already is responsible for dealing with follow-up work when a block is added to the dag This sets the stage for keeping attestations in the quarantine as well. Also: * make `BlockError` `{.pure.}` * avoid use of `ValidationResult` in block clearance (that's for gossip)
2021-12-06 09:49:01 +00:00
## Block processor [Preset: mainnet]
```diff
+ Reverse order block add & get [Preset: mainnet] OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
## Block quarantine
```diff
+ Unviable smoke test OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
BlockId reform (#3176) * BlockId reform Introduce `BlockId` that helps track a root/slot pair - this prepares the codebase for backfilling and handling out-of-dag blocks * move block dag code to separate module * fix finalised state root in REST event stream * fix finalised head computation on head update, when starting from checkpoint * clean up chaindag init * revert `epochAncestor` change in introduced in #3144 that would return an epoch ancestor from the canoncial history instead of the given history, causing `EpochRef` keys to point to the wrong block
2021-12-09 17:06:21 +00:00
## BlockId and helpers
Add test reports
2020-03-10 04:24:33 +00:00
```diff
BlockId reform (#3176) * BlockId reform Introduce `BlockId` that helps track a root/slot pair - this prepares the codebase for backfilling and handling out-of-dag blocks * move block dag code to separate module * fix finalised state root in REST event stream * fix finalised head computation on head update, when starting from checkpoint * clean up chaindag init * revert `epochAncestor` change in introduced in #3144 that would return an epoch ancestor from the canoncial history instead of the given history, causing `EpochRef` keys to point to the wrong block
2021-12-09 17:06:21 +00:00
+ atSlot sanity OK
+ parent sanity OK
Add test reports
2020-03-10 04:24:33 +00:00
```
fix test results totals (#3168) Updates `AllTests-mainnet.md` output for `make test -j16` execution.
2021-12-07 13:43:06 +00:00
OK: 2/2 Fail: 0/2 Skip: 0/2
BlockId reform (#3176) * BlockId reform Introduce `BlockId` that helps track a root/slot pair - this prepares the codebase for backfilling and handling out-of-dag blocks * move block dag code to separate module * fix finalised state root in REST event stream * fix finalised head computation on head update, when starting from checkpoint * clean up chaindag init * revert `epochAncestor` change in introduced in #3144 that would return an epoch ancestor from the canoncial history instead of the given history, causing `EpochRef` keys to point to the wrong block
2021-12-09 17:06:21 +00:00
## BlockRef and helpers
Add test reports
2020-03-10 04:24:33 +00:00
```diff
BlockId reform (#3176) * BlockId reform Introduce `BlockId` that helps track a root/slot pair - this prepares the codebase for backfilling and handling out-of-dag blocks * move block dag code to separate module * fix finalised state root in REST event stream * fix finalised head computation on head update, when starting from checkpoint * clean up chaindag init * revert `epochAncestor` change in introduced in #3144 that would return an epoch ancestor from the canoncial history instead of the given history, causing `EpochRef` keys to point to the wrong block
2021-12-09 17:06:21 +00:00
+ get_ancestor sanity OK
+ isAncestorOf sanity OK
Add test reports
2020-03-10 04:24:33 +00:00
```
OK: 2/2 Fail: 0/2 Skip: 0/2
BlockId reform (#3176) * BlockId reform Introduce `BlockId` that helps track a root/slot pair - this prepares the codebase for backfilling and handling out-of-dag blocks * move block dag code to separate module * fix finalised state root in REST event stream * fix finalised head computation on head update, when starting from checkpoint * clean up chaindag init * revert `epochAncestor` change in introduced in #3144 that would return an epoch ancestor from the canoncial history instead of the given history, causing `EpochRef` keys to point to the wrong block
2021-12-09 17:06:21 +00:00
## BlockSlot and helpers
```diff
+ atSlot sanity OK
+ parent sanity OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
## DeleteKeys requests [Beacon Node] [Preset: mainnet]
Implementation of the Keymanager API (BETA) https://github.com/ethereum/keymanager-APIs
2021-12-22 12:37:31 +00:00
```diff
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
+ Deleting not existing key [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Header [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Token [Beacon Node] [Preset: mainnet] OK
+ Missing Authorization header [Beacon Node] [Preset: mainnet] OK
Implementation of the Keymanager API (BETA) https://github.com/ethereum/keymanager-APIs
2021-12-22 12:37:31 +00:00
```
Fix #3650 (participation format in BeaconState result is out of spec) (#3776) * Fix #3650 (participation format in BeaconState result is out of spec) * Make EpochParticipationFlags a distinct type
2022-06-20 05:38:56 +00:00
OK: 4/4 Fail: 0/4 Skip: 0/4
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
## DeleteRemoteKeys requests [Beacon Node] [Preset: mainnet]
```diff
+ Deleting existing local key and remote key [Beacon Node] [Preset: mainnet] OK
+ Deleting not existing key [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Header [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Token [Beacon Node] [Preset: mainnet] OK
+ Missing Authorization header [Beacon Node] [Preset: mainnet] OK
```
OK: 5/5 Fail: 0/5 Skip: 0/5
support connecting to peers without bellatrix (#4011) * support connecting to peers without bellatrix Make discovery fork ID aware of scheduled Bellatrix fork to enable connections to peers that don't have Bellatrix scheduled yet. Without this, has peering issues with peers on older SW version. * expand tests with compatibility checks * more exhaustive compatibility checks
2022-08-21 17:36:46 +00:00
## Discovery fork ID
```diff
+ Expected fork IDs OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
discard putative blocks from invalid hardforks
2021-09-08 03:46:33 +00:00
## Diverging hardforks
```diff
+ Non-tail block in common OK
+ Tail block only in common OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
add Bellatrix fork and transition tests; "Ethereum Foundation" -> EF (#3242)
2022-01-05 08:42:56 +00:00
## EF - SSZ generic types
test fixture cleanups (#2976) * "official" -> "scenarios", like the submodule * fewer test binaries - various compile hacks have been improved over time, test suite should follow * remove obsolete bls tests - there are better test vectors in nim-blscurve * remove obsolete mentions of `ssz_testing` * remove obsolete comments about proc vs globals, unittest2 already uses proc's
2021-10-12 11:36:52 +00:00
```diff
Testing basic_vector inputs - invalid Skip
+ Testing basic_vector inputs - valid OK
+ Testing bitlist inputs - invalid OK
+ Testing bitlist inputs - valid OK
Testing bitvector inputs - invalid Skip
+ Testing bitvector inputs - valid OK
+ Testing boolean inputs - invalid OK
+ Testing boolean inputs - valid OK
+ Testing containers inputs - invalid - skipping BitsStruct OK
+ Testing containers inputs - valid - skipping BitsStruct OK
+ Testing uints inputs - invalid OK
+ Testing uints inputs - valid OK
```
OK: 10/12 Fail: 0/12 Skip: 2/12
add Bellatrix fork and transition tests; "Ethereum Foundation" -> EF (#3242)
2022-01-05 08:42:56 +00:00
## Eth1 monitor
```diff
avoid materializing potentially long deposits seq (#3947) When fetching eth1 data and deposits for a new block proposal, the list of deposits from previous eth1 data to the next one is fully loaded into a `seq`. This can potentially be a very long list in active periods. Changing this to an `iterator` saves memory by ensuring that the entire list is no longer materialized; only the `DepositData` roots are needed.
2022-08-12 13:52:06 +00:00
+ Deposits chain OK
add Bellatrix fork and transition tests; "Ethereum Foundation" -> EF (#3242)
2022-01-05 08:42:56 +00:00
+ Rewrite HTTPS Infura URLs OK
+ Roundtrip engine RPC and consensus ExecutionPayload representations OK
```
avoid materializing potentially long deposits seq (#3947) When fetching eth1 data and deposits for a new block proposal, the list of deposits from previous eth1 data to the next one is fully loaded into a `seq`. This can potentially be a very long list in active periods. Changing this to an `iterator` saves memory by ensuring that the entire list is no longer materialized; only the `DepositData` roots are needed.
2022-08-12 13:52:06 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
add Bellatrix fork and transition tests; "Ethereum Foundation" -> EF (#3242)
2022-01-05 08:42:56 +00:00
## Eth2 specific discovery tests
```diff
+ Invalid attnets field OK
+ Subnet query OK
+ Subnet query after ENR update OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
refactor exit pool and gossipsub-subscribe to exit pool message types (#1733) * refactor exit pool and gossipsub-subscribe to all exit pool message types * remove unused loop counter
2020-09-24 17:05:49 +00:00
## Exit pool testing suite
```diff
+ addExitMessage/getAttesterSlashingMessage OK
+ addExitMessage/getProposerSlashingMessage OK
+ addExitMessage/getVoluntaryExitMessage OK
Harden block proposal against expired slashings/exits (#4013) * Harden block proposal against expired slashings/exits When a message is signed in a phase0 domain, it can no longer be validated under bellatrix due to the correct fork no longer being available in the `BeaconState`. To ensure that all slashing/exits are still valid, in this PR we re-run the checks in the state that we're proposing for, thus hardening against both signatures and other changes in the state that might have invalidated the message. * fix same message added multiple times in case of attestation slashing of multiple validators in one go
2022-08-23 15:30:46 +00:00
+ pre-pre-fork voluntary exit OK
refactor exit pool and gossipsub-subscribe to exit pool message types (#1733) * refactor exit pool and gossipsub-subscribe to all exit pool message types * remove unused loop counter
2020-09-24 17:05:49 +00:00
```
Harden block proposal against expired slashings/exits (#4013) * Harden block proposal against expired slashings/exits When a message is signed in a phase0 domain, it can no longer be validated under bellatrix due to the correct fork no longer being available in the `BeaconState`. To ensure that all slashing/exits are still valid, in this PR we re-run the checks in the state that we're proposing for, thus hardening against both signatures and other changes in the state that might have invalidated the message. * fix same message added multiple times in case of attestation slashing of multiple validators in one go
2022-08-23 15:30:46 +00:00
OK: 4/4 Fail: 0/4 Skip: 0/4
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
## Fee recipient management [Beacon Node] [Preset: mainnet]
```diff
+ Configuring the fee recpient [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Header [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Token [Beacon Node] [Preset: mainnet] OK
+ Missing Authorization header [Beacon Node] [Preset: mainnet] OK
+ Obtaining the fee recpient of a missing validator returns 404 [Beacon Node] [Preset: mainn OK
+ Obtaining the fee recpient of an unconfigured validator returns the suggested default [Bea OK
+ Setting the fee recipient on a missing validator creates a record for it [Beacon Node] [Pr OK
```
OK: 7/7 Fail: 0/7 Skip: 0/7
Store finalized block roots in database (3s startup) (#3320) * Store finalized block roots in database (3s startup) When the chain has finalized a checkpoint, the history from that point onwards becomes linear - this is exploited in `.era` files to allow constant-time by-slot lookups. In the database, we can do the same by storing finalized block roots in a simple sparse table indexed by slot, bringing the two representations closer to each other in terms of conceptual layout and performance. Doing so has a number of interesting effects: * mainnet startup time is improved 3-5x (3s on my laptop) * the _first_ startup might take slightly longer as the new index is being built - ~10s on the same laptop * we no longer rely on the beacon block summaries to load the full dag - this is a lot faster because we no longer have to look up each block by parent root * a collateral benefit is that we no longer need to load the full summaries table into memory - we get the RSS benefits of #3164 without the CPU hit. Other random stuff: * simplify forky block generics * fix withManyWrites multiple evaluation * fix validator key cache not being updated properly in chaindag read-only mode * drop pre-altair summaries from `kvstore` * recreate missing summaries from altair+ blocks as well (in case database has lost some to an involuntary restart) * print database startup timings in chaindag load log * avoid allocating superfluos state at startup * use a recursive sql query to load the summaries of the unfinalized blocks
2022-01-30 16:51:04 +00:00
## FinalizedBlocks [Preset: mainnet]
```diff
+ Basic ops [Preset: mainnet] OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
[Ready 1/2] Fork choice rewrite (#865) * initial fork-choice refactor * Add fork_choice test for "no votes" * Initial test with voting: fix handling of unknown validators and parent blocks * Fix tiebreak of votes * Cleanup debugging traces * Complexify the vote test * fakeHash use the bigEndian repr of number + fix tiebreak for good * Stash changes: found critical bug in nimcrypto `==` and var openarray * Passing fork choice tests with varying votes * Add FFG fork choice scenario + fork choice to the test suite * Not sure why lmdb / rocksdb reappeared in rebase * Add sanity checks to .nimble file + integrate fork choice tests to the test DB and test timing * Cleanup debugging echos * nimcrypto fix https://github.com/status-im/nim-beacon-chain/pull/864 as been merged, remove TODO comment * Turn fork choice exception-free * Cleanup "result" to ensure early return is properly used * Add a comment on private/public error code vs Result * result -> results following https://github.com/status-im/nim-beacon-chain/pull/866 * Address comments: - raises: [Defect] doesn't work -> TODO - process_attestation cannot fail - try/except as expression pending Nim v1.2.0 - cleanup TODOs * re-enable all sanity checks * tag no raise for process_attestation * use raises defect everywhere in fork choice and fix process_attestation test
2020-04-09 16:15:00 +00:00
## Fork Choice + Finality [Preset: mainnet]
```diff
+ fork_choice - testing finality #01 OK
+ fork_choice - testing finality #02 OK
+ fork_choice - testing no votes OK
+ fork_choice - testing with votes OK
```
OK: 4/4 Fail: 0/4 Skip: 0/4
merge hardfork database support (#2911) * merge hardfork database support * working block_sim * recreate state transition changes
2021-09-30 01:07:24 +00:00
## Fork id compatibility test
```diff
+ Digest check OK
+ Fork check OK
+ Next fork epoch check OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
Support starting from altair (#3054) * Support starting from altair * hide `finalized-checkpoint-` - they are incomplete and usage may cause crashes * remove genesis detection code (broken, obsolete) * enable starting ChainDAG from altair checkpoints - this is a prerequisite for checkpoint sync (TODO: backfill) * tighten checkpoint state conditions * show error when starting from checkpoint with existing database (not supported) * print rest-compatible JSON in ncli/state_sim * altair/merge support in ncli * more altair/merge support in ncli_db * pre-load header to speed up loading * fix forked block decoding
2021-11-10 11:39:08 +00:00
## Forked SSZ readers
```diff
+ load altair block OK
+ load altair state OK
rename On{Merge,Bellatrix}BlockAdded and Rollback{Merge,Bellatrix}HashedProc (#3321)
2022-01-26 12:21:29 +00:00
+ load bellatrix block OK
+ load bellatrix state OK
Support starting from altair (#3054) * Support starting from altair * hide `finalized-checkpoint-` - they are incomplete and usage may cause crashes * remove genesis detection code (broken, obsolete) * enable starting ChainDAG from altair checkpoints - this is a prerequisite for checkpoint sync (TODO: backfill) * tighten checkpoint state conditions * show error when starting from checkpoint with existing database (not supported) * print rest-compatible JSON in ncli/state_sim * altair/merge support in ncli * more altair/merge support in ncli_db * pre-load header to speed up loading * fix forked block decoding
2021-11-10 11:39:08 +00:00
+ load phase0 block OK
+ load phase0 state OK
+ should raise on unknown data OK
```
OK: 7/7 Fail: 0/7 Skip: 0/7
Merge fork gossip support (#3213) * Merge fork gossip support * index directly by BeaconStateFork and remove debugging log statement
2021-12-21 14:24:23 +00:00
## Gossip fork transition
```diff
+ Gossip fork transition OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
## Gossip validation [Preset: mainnet]
```diff
Harden CommitteeIndex, SubnetId, SyncSubcommitteeIndex (#3259) * Harden CommitteeIndex, SubnetId, SyncSubcommitteeIndex Harden the use of `CommitteeIndex` et al to prevent future issues by using a distinct type, then validating before use in several cases - datatypes in spec are kept simple though so that invalid data still can be read. * fix invalid epoch used in REST `/eth/v1/beacon/states/{state_id}/committees` committee length (could return invalid data) * normalize some variable names * normalize committee index loops * fix `RestAttesterDuty` to use `uint64` for `validator_committee_index` * validate `CommitteeIndex` on ingress in REST API * update rest rules with stricter parsing * better REST serializers * save lots of memory by not using `zip` ...at least a few bytes!
2022-01-08 23:28:49 +00:00
+ Empty committee when no committee for slot OK
Speed up altair block processing 2x (#3115) * Speed up altair block processing >2x Like #3089, this PR drastially speeds up historical REST queries and other long state replays. * cache sync committee validator indices * use ~80mb less memory for validator pubkey mappings * batch-verify sync aggregate signature (fixes #2985) * document sync committee hack with head block vs sync message block * add batch signature verification failure tests Before: ``` ../env.sh nim c -d:release -r ncli_db --db:mainnet_0/db bench --start-slot:-1000 All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 5830.675, 0.000, 5830.675, 5830.675, 1, Initialize DB 0.481, 1.878, 0.215, 59.167, 981, Load block from database 8422.566, 0.000, 8422.566, 8422.566, 1, Load state from database 6.996, 1.678, 0.042, 14.385, 969, Advance slot, non-epoch 93.217, 8.318, 84.192, 122.209, 32, Advance slot, epoch 20.513, 23.665, 11.510, 201.561, 981, Apply block, no slot processing 0.000, 0.000, 0.000, 0.000, 0, Database load 0.000, 0.000, 0.000, 0.000, 0, Database store ``` After: ``` 7081.422, 0.000, 7081.422, 7081.422, 1, Initialize DB 0.553, 2.122, 0.175, 66.692, 981, Load block from database 5439.446, 0.000, 5439.446, 5439.446, 1, Load state from database 6.829, 1.575, 0.043, 12.156, 969, Advance slot, non-epoch 94.716, 2.749, 88.395, 100.026, 32, Advance slot, epoch 11.636, 23.766, 4.889, 205.250, 981, Apply block, no slot processing 0.000, 0.000, 0.000, 0.000, 0, Database load 0.000, 0.000, 0.000, 0.000, 0, Database store ``` * add comment
2021-11-24 12:43:50 +00:00
+ validateAttestation OK
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
```
add sync committee types and some helpers (#2829)
2021-08-28 09:00:00 +00:00
OK: 2/2 Fail: 0/2 Skip: 0/2
handle duplicate pubkeys in sync committee (#2902) When sync committee message handling was introduced in #2830, the edge case of the same validator being selected multiple times as part of a sync subcommittee was not covered. Not handling that edge case makes sync contributions have a lower-than-expected participation rate as each sync validator is only counted up through once per subcommittee. This patch ensures that this edge case is properly covered.
2021-09-28 07:44:20 +00:00
## Gossip validation - Extra
```diff
+ validateSyncCommitteeMessage OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Add test reports
2020-03-10 04:24:33 +00:00
## Honest validator
```diff
detect already-aggregate-voted condition before attestation pool; add is_aggregator tests (#2170) * detect already-aggregate-voted condition before attestation pool; add is_aggregator tests * replace pair of attestation-per-epoch tracking lists with single list and remove Option use * fix attestation condition * use safer type conversions; add more is_aggregator tests
2020-12-14 20:58:32 +00:00
+ General pubsub topics OK
remove some unused imports, add tests for pubsub topics, and subscribe to interop attestations
2020-05-14 11:19:10 +00:00
+ Mainnet attestation topics OK
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
+ isNearSyncCommitteePeriod OK
detect already-aggregate-voted condition before attestation pool; add is_aggregator tests (#2170) * detect already-aggregate-voted condition before attestation pool; add is_aggregator tests * replace pair of attestation-per-epoch tracking lists with single list and remove Option use * fix attestation condition * use safer type conversions; add more is_aggregator tests
2020-12-14 20:58:32 +00:00
+ is_aggregator OK
Add test reports
2020-03-10 04:24:33 +00:00
```
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
OK: 4/4 Fail: 0/4 Skip: 0/4
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
## ImportKeystores requests [Beacon Node] [Preset: mainnet]
Implementation of the Keymanager API (BETA) https://github.com/ethereum/keymanager-APIs
2021-12-22 12:37:31 +00:00
```diff
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
+ ImportKeystores/ListKeystores/DeleteKeystores [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Header [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Token [Beacon Node] [Preset: mainnet] OK
+ Missing Authorization header [Beacon Node] [Preset: mainnet] OK
Implementation of the Keymanager API (BETA) https://github.com/ethereum/keymanager-APIs
2021-12-22 12:37:31 +00:00
```
Fix #3650 (participation format in BeaconState result is out of spec) (#3776) * Fix #3650 (participation format in BeaconState result is out of spec) * Make EpochParticipationFlags a distinct type
2022-06-20 05:38:56 +00:00
OK: 4/4 Fail: 0/4 Skip: 0/4
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
## ImportRemoteKeys/ListRemoteKeys/DeleteRemoteKeys [Beacon Node] [Preset: mainnet]
```diff
+ Importing list of remote keys [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Header [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Token [Beacon Node] [Preset: mainnet] OK
+ Missing Authorization header [Beacon Node] [Preset: mainnet] OK
```
OK: 4/4 Fail: 0/4 Skip: 0/4
re-add test_interop to all_tests and mark several v0.10.1 phase 0 spec references as v0.11.1
2020-04-06 18:55:47 +00:00
## Interop
```diff
+ Interop genesis OK
+ Interop signatures OK
+ Mocked start private key OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
Support for distributed keystores with multiple remotes based on threshold signatures (#3616) Other fixes: * Fix bit rot in the `make prater-dev-deposit` target. * Correct content-type in the responses of the Nimbus signing node * Invalid JSON payload was being sent in the web3signer requests
2022-05-10 00:32:12 +00:00
## Key spliting
```diff
+ k < n OK
+ k == n OK
+ k == n == 100 OK
+ single share OK
```
OK: 4/4 Fail: 0/4 Skip: 0/4
test fixture cleanups (#2976) * "official" -> "scenarios", like the submodule * fewer test binaries - various compile hacks have been improved over time, test suite should follow * remove obsolete bls tests - there are better test vectors in nim-blscurve * remove obsolete mentions of `ssz_testing` * remove obsolete comments about proc vs globals, unittest2 already uses proc's
2021-10-12 11:36:52 +00:00
## KeyStorage testing suite
```diff
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
+ Load Prysm keystore OK
test fixture cleanups (#2976) * "official" -> "scenarios", like the submodule * fewer test binaries - various compile hacks have been improved over time, test suite should follow * remove obsolete bls tests - there are better test vectors in nim-blscurve * remove obsolete mentions of `ssz_testing` * remove obsolete comments about proc vs globals, unittest2 already uses proc's
2021-10-12 11:36:52 +00:00
+ Pbkdf2 errors OK
+ [PBKDF2] Keystore decryption OK
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
+ [PBKDF2] Keystore decryption (requireAllFields, allowUnknownFields) OK
test fixture cleanups (#2976) * "official" -> "scenarios", like the submodule * fewer test binaries - various compile hacks have been improved over time, test suite should follow * remove obsolete bls tests - there are better test vectors in nim-blscurve * remove obsolete mentions of `ssz_testing` * remove obsolete comments about proc vs globals, unittest2 already uses proc's
2021-10-12 11:36:52 +00:00
+ [PBKDF2] Keystore encryption OK
+ [PBKDF2] Network Keystore decryption OK
+ [PBKDF2] Network Keystore encryption OK
+ [SCRYPT] Keystore decryption OK
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
+ [SCRYPT] Keystore decryption (requireAllFields, allowUnknownFields) OK
test fixture cleanups (#2976) * "official" -> "scenarios", like the submodule * fewer test binaries - various compile hacks have been improved over time, test suite should follow * remove obsolete bls tests - there are better test vectors in nim-blscurve * remove obsolete mentions of `ssz_testing` * remove obsolete comments about proc vs globals, unittest2 already uses proc's
2021-10-12 11:36:52 +00:00
+ [SCRYPT] Keystore encryption OK
+ [SCRYPT] Network Keystore decryption OK
+ [SCRYPT] Network Keystore encryption OK
```
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
OK: 12/12 Fail: 0/12 Skip: 0/12
implement v1.2.0 optimistic sync tests (#4174) * implement v1.2.0 optimistic sync tests * Update beacon_chain/consensus_object_pools/blockchain_dag.nim Co-authored-by: Etan Kissling <etan@status.im> * `lvh` -> `latestValidHash` and only invalidate one specific block" * `getEarliestInvalidRoot` -> `getEarliestInvalidBlockRoot`; `defaultEarliestInvalidRoot` -> `defaultEarliestInvalidBlockRoot` Co-authored-by: Etan Kissling <etan@status.im>
2022-09-27 12:11:47 +00:00
## Latest valid hash [Preset: mainnet]
```diff
+ LVH searching OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
add option to collect light client data (#3474) Light clients require full nodes to serve additional data so that they can stay in sync with the network. This patch adds a new launch option `--import-light-client-data` to configure what data to make available. For now, data is only kept in memory; it is not persisted at this time. Note that data is only locally collected, a separate patch is needed to actually make it availble over the network. `--serve-light-client-data` will be used for serving data, but is not functional yet outside tests.
2022-03-11 20:28:10 +00:00
## Light client [Preset: mainnet]
```diff
fix light client data collection for checkpoint sync (#3498) When doing checkpoint sync, collecting light client data of known blocks and states incorrectly assumes that `finalized_checkpoint` information is also known. Hardens collection to only collect finalized checkpoint data after `dag.computeEarliestLightClientSlot`.
2022-03-18 14:47:53 +00:00
+ Init from checkpoint OK
add option to collect light client data (#3474) Light clients require full nodes to serve additional data so that they can stay in sync with the network. This patch adds a new launch option `--import-light-client-data` to configure what data to make available. For now, data is only kept in memory; it is not persisted at this time. Note that data is only locally collected, a separate patch is needed to actually make it availble over the network. `--serve-light-client-data` will be used for serving data, but is not functional yet outside tests.
2022-03-11 20:28:10 +00:00
+ Light client sync OK
+ Pre-Altair OK
```
fix light client data collection for checkpoint sync (#3498) When doing checkpoint sync, collecting light client data of known blocks and states incorrectly assumes that `finalized_checkpoint` information is also known. Hardens collection to only collect finalized checkpoint data after `dag.computeEarliestLightClientSlot`.
2022-03-18 14:47:53 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
## Light client processor [Preset: mainnet]
```diff
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
+ Duplicate bootstrap (Optimistic) [Preset: mainnet] OK
+ Duplicate bootstrap (Strict) [Preset: mainnet] OK
+ Invalid bootstrap (Optimistic) [Preset: mainnet] OK
+ Invalid bootstrap (Strict) [Preset: mainnet] OK
+ Missing bootstrap (finality update) (Optimistic) [Preset: mainnet] OK
+ Missing bootstrap (finality update) (Strict) [Preset: mainnet] OK
+ Missing bootstrap (optimistic update) (Optimistic) [Preset: mainnet] OK
+ Missing bootstrap (optimistic update) (Strict) [Preset: mainnet] OK
+ Missing bootstrap (update) (Optimistic) [Preset: mainnet] OK
+ Missing bootstrap (update) (Strict) [Preset: mainnet] OK
+ Sync (Optimistic) [Preset: mainnet] OK
+ Sync (Strict) [Preset: mainnet] OK
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
```
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
OK: 12/12 Fail: 0/12 Skip: 0/12
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
## ListKeys requests [Beacon Node] [Preset: mainnet]
Implementation of the Keymanager API (BETA) https://github.com/ethereum/keymanager-APIs
2021-12-22 12:37:31 +00:00
```diff
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
+ Correct token provided [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Header [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Token [Beacon Node] [Preset: mainnet] OK
+ Missing Authorization header [Beacon Node] [Preset: mainnet] OK
Implementation of the Keymanager API (BETA) https://github.com/ethereum/keymanager-APIs
2021-12-22 12:37:31 +00:00
```
OK: 4/4 Fail: 0/4 Skip: 0/4
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
## ListRemoteKeys requests [Beacon Node] [Preset: mainnet]
```diff
+ Correct token provided [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Header [Beacon Node] [Preset: mainnet] OK
+ Invalid Authorization Token [Beacon Node] [Preset: mainnet] OK
+ Missing Authorization header [Beacon Node] [Preset: mainnet] OK
```
OK: 4/4 Fail: 0/4 Skip: 0/4
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
## Message signatures
```diff
+ Aggregate and proof signatures OK
+ Attestation signatures OK
+ Deposit signatures OK
+ Slot signatures OK
+ Sync committee message signatures OK
+ Sync committee selection proof signatures OK
+ Sync committee signed contribution and proof signatures OK
+ Voluntary exit signatures OK
```
OK: 8/8 Fail: 0/8 Skip: 0/8
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
## Old database versions [Preset: mainnet]
```diff
+ pre-1.1.0 OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Add test reports
2020-03-10 04:24:33 +00:00
## PeerPool testing suite
```diff
+ Access peers by key test OK
+ Acquire from empty pool OK
+ Acquire/Sorting and consistency test OK
improve slot processing speeds (#1670) about 40% better slot processing times (with LTO enabled) - these don't do BLS but are used heavily during replay (state transition = slot + block transition) tests using a recent medalla state and advancing it 1000 slots: ``` ./ncli slots --preState2:state-302271-3c1dbf19-c1f944bf.ssz --slot:1000 --postState2:xx.ssz ``` pre: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 39.236, 0.000, 39.236, 39.236, 1, Load state from file 0.049, 0.002, 0.046, 0.063, 968, Apply slot 256.504, 81.008, 213.471, 591.902, 32, Apply epoch slot 28.597, 0.000, 28.597, 28.597, 1, Save state to file ``` cast: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 37.079, 0.000, 37.079, 37.079, 1, Load state from file 0.042, 0.002, 0.040, 0.090, 968, Apply slot 215.552, 68.763, 180.155, 500.103, 32, Apply epoch slot 25.106, 0.000, 25.106, 25.106, 1, Save state to file ``` cast+rewards: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 40.049, 0.000, 40.049, 40.049, 1, Load state from file 0.048, 0.001, 0.045, 0.060, 968, Apply slot 164.981, 76.273, 142.099, 477.868, 32, Apply epoch slot 28.498, 0.000, 28.498, 28.498, 1, Save state to file ``` cast+rewards+shr ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 12.898, 0.000, 12.898, 12.898, 1, Load state from file 0.039, 0.002, 0.038, 0.054, 968, Apply slot 139.971, 68.797, 120.088, 428.844, 32, Apply epoch slot 24.761, 0.000, 24.761, 24.761, 1, Save state to file ```
2020-09-16 20:59:33 +00:00
+ Delete peer on release text OK
Add test reports
2020-03-10 04:24:33 +00:00
+ Iterators test OK
+ Peer lifetime test OK
+ Safe/Clear test OK
detect improper 0-length lists of variable-sized objects in SSZ reading (#928) * detect improper 0-length lists of variable-sized objects in SSZ reading
2020-04-23 19:39:23 +00:00
+ Score check test OK
refactor exit pool and gossipsub-subscribe to exit pool message types (#1733) * refactor exit pool and gossipsub-subscribe to all exit pool message types * remove unused loop counter
2020-09-24 17:05:49 +00:00
+ Space tests OK
Add test reports
2020-03-10 04:24:33 +00:00
+ addPeer() test OK
+ addPeerNoWait() test OK
+ deletePeer() test OK
```
refactor exit pool and gossipsub-subscribe to exit pool message types (#1733) * refactor exit pool and gossipsub-subscribe to all exit pool message types * remove unused loop counter
2020-09-24 17:05:49 +00:00
OK: 12/12 Fail: 0/12 Skip: 0/12
Support for distributed keystores with multiple remotes based on threshold signatures (#3616) Other fixes: * Fix bit rot in the `make prater-dev-deposit` target. * Correct content-type in the responses of the Nimbus signing node * Invalid JSON payload was being sent in the web3signer requests
2022-05-10 00:32:12 +00:00
## Remove keystore testing suite
```diff
+ vesion 1 OK
+ vesion 2 many remotes OK
+ vesion 2 single remote OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
## Serialization/deserialization [Beacon Node] [Preset: mainnet]
```diff
+ Deserialization test vectors OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
## Slashing Interchange tests [Preset: mainnet]
```diff
+ Slashing test: duplicate_pubkey_not_slashable.json OK
+ Slashing test: duplicate_pubkey_slashable_attestation.json OK
+ Slashing test: duplicate_pubkey_slashable_block.json OK
+ Slashing test: multiple_interchanges_multiple_validators_repeat_idem.json OK
+ Slashing test: multiple_interchanges_overlapping_validators_merge_stale.json OK
+ Slashing test: multiple_interchanges_overlapping_validators_repeat_idem.json OK
+ Slashing test: multiple_interchanges_single_validator_fail_iff_imported.json OK
+ Slashing test: multiple_interchanges_single_validator_first_surrounds_second.json OK
Minify slashing protection before SQLite (#3393)
2022-03-04 14:43:34 +00:00
+ Slashing test: multiple_interchanges_single_validator_multiple_blocks_out_of_order.json OK
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
+ Slashing test: multiple_interchanges_single_validator_second_surrounds_first.json OK
+ Slashing test: multiple_interchanges_single_validator_single_att_out_of_order.json OK
+ Slashing test: multiple_interchanges_single_validator_single_block_out_of_order.json OK
+ Slashing test: multiple_interchanges_single_validator_single_message_gap.json OK
+ Slashing test: multiple_validators_multiple_blocks_and_attestations.json OK
+ Slashing test: multiple_validators_same_slot_blocks.json OK
+ Slashing test: single_validator_genesis_attestation.json OK
+ Slashing test: single_validator_import_only.json OK
+ Slashing test: single_validator_multiple_block_attempts.json OK
+ Slashing test: single_validator_multiple_blocks_and_attestations.json OK
+ Slashing test: single_validator_out_of_order_attestations.json OK
+ Slashing test: single_validator_out_of_order_blocks.json OK
Minify slashing protection before SQLite (#3393)
2022-03-04 14:43:34 +00:00
Slashing test: single_validator_resign_attestation.json Skip
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
+ Slashing test: single_validator_resign_block.json OK
+ Slashing test: single_validator_single_attestation.json OK
+ Slashing test: single_validator_single_block.json OK
+ Slashing test: single_validator_single_block_and_attestation.json OK
+ Slashing test: single_validator_single_block_and_attestation_signing_root.json OK
+ Slashing test: single_validator_slashable_attestations_double_vote.json OK
+ Slashing test: single_validator_slashable_attestations_surrounded_by_existing.json OK
+ Slashing test: single_validator_slashable_attestations_surrounds_existing.json OK
+ Slashing test: single_validator_slashable_blocks.json OK
+ Slashing test: single_validator_slashable_blocks_no_root.json OK
+ Slashing test: single_validator_source_greater_than_target.json OK
+ Slashing test: single_validator_source_greater_than_target_sensible_iff_minified.json OK
Minify slashing protection before SQLite (#3393)
2022-03-04 14:43:34 +00:00
Slashing test: single_validator_source_greater_than_target_surrounded.json Skip
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
Slashing test: single_validator_source_greater_than_target_surrounding.json Skip
+ Slashing test: single_validator_two_blocks_no_signing_root.json OK
+ Slashing test: wrong_genesis_validators_root.json OK
```
Minify slashing protection before SQLite (#3393)
2022-03-04 14:43:34 +00:00
OK: 35/38 Fail: 0/38 Skip: 3/38
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
## Slashing Protection DB [Preset: mainnet]
```diff
update AllTests-mainnet.md
2020-09-26 21:58:12 +00:00
+ Attestation ordering #1698 OK
Slashing db pruning [Merge only after v2 has been default for 1 noticeable release] (#2452) * Enable slashing DB pruning * integrate slashing DB pruning with onSlotEnd * rebase tests
2021-05-10 14:32:28 +00:00
+ Don't prune the very last attestation(s) even by mistake OK
+ Don't prune the very last block even by mistake OK
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
+ Empty database [Preset: mainnet] OK
Slashing db pruning [Merge only after v2 has been default for 1 noticeable release] (#2452) * Enable slashing DB pruning * integrate slashing DB pruning with onSlotEnd * rebase tests
2021-05-10 14:32:28 +00:00
+ Pruning attestations works OK
+ Pruning blocks works OK
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
+ SP for block proposal - backtracking append OK
+ SP for block proposal - linear append OK
+ SP for same epoch attestation target - linear append OK
+ SP for surrounded attestations OK
+ SP for surrounding attestations OK
update AllTests-mainnet.md
2020-09-26 21:58:12 +00:00
+ Test valid attestation #1699 OK
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
```
Slashing db pruning [Merge only after v2 has been default for 1 noticeable release] (#2452) * Enable slashing DB pruning * integrate slashing DB pruning with onSlotEnd * rebase tests
2021-05-10 14:32:28 +00:00
OK: 12/12 Fail: 0/12 Skip: 0/12
Restore compilation with custom presets (#1309) This also assigns precise types to the constants in the minimal and mainnet presets in order to reduce the chance of compilation errors when custom presets are used (previously, only the custom presets have precisely assigned types for the constants).
2020-07-13 14:44:58 +00:00
## Spec datatypes
```diff
+ Graffiti bytes OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Add test reports
2020-03-10 04:24:33 +00:00
## Spec helpers
```diff
add function to build merkle proofs (#2874) Adds a function that constructs a Merkle proof for a generalized index. This will be used during light client sync to update light clients with a new state (see NEXT_SYNC_COMMITTEE_INDEX / FINALIZED_ROOT_INDEX).
2021-09-29 13:02:34 +00:00
+ build_proof - BeaconState OK
Add test reports
2020-03-10 04:24:33 +00:00
+ integer_squareroot OK
```
bump nim-ssz-serialization to `3db6cc0f282708aca6c290914488edd832971d61` (#3119) This updates `nim-ssz-serialization` to `3db6cc0f282708aca6c290914488edd832971d61`. Notable changes: - Use `uint64` for `GeneralizedIndex` - Add support for building merkle multiproofs
2022-06-26 17:33:06 +00:00
OK: 2/2 Fail: 0/2 Skip: 0/2
disentangle eth2 types from the ssz library (#2785) * reorganize ssz dependencies This PR continues the work in https://github.com/status-im/nimbus-eth2/pull/2646, https://github.com/status-im/nimbus-eth2/pull/2779 as well as past issues with serialization and type, to disentangle SSZ from eth2 and at the same time simplify imports and exports with a structured approach. The principal idea here is that when a library wants to introduce SSZ support, they do so via 3 files: * `ssz_codecs` which imports and reexports `codecs` - this covers the basic byte conversions and ensures no overloads get lost * `xxx_merkleization` imports and exports `merkleization` to specialize and get access to `hash_tree_root` and friends * `xxx_ssz_serialization` imports and exports `ssz_serialization` to specialize ssz for a specific library Those that need to interact with SSZ always import the `xxx_` versions of the modules and never `ssz` itself so as to keep imports simple and safe. This is similar to how the REST / JSON-RPC serializers are structured in that someone wanting to serialize spec types to REST-JSON will import `eth2_rest_serialization` and nothing else. * split up ssz into a core library that is independendent of eth2 types * rename `bytes_reader` to `codec` to highlight that it contains coding and decoding of bytes and native ssz types * remove tricky List init overload that causes compile issues * get rid of top-level ssz import * reenable merkleization tests * move some "standard" json serializers to spec * remove `ValidatorIndex` serialization for now * remove test_ssz_merkleization * add tests for over/underlong byte sequences * fix broken seq[byte] test - seq[byte] is not an SSZ type There are a few things this PR doesn't solve: * like #2646 this PR is weak on how to handle root and other dontSerialize fields that "sometimes" should be computed - the same problem appears in REST / JSON-RPC etc * Fix a build problem on macOS * Another way to fix the macOS builds Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 18:57:58 +00:00
## Specific field types
```diff
+ root update OK
+ roundtrip OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
Restore the sync committee pool pruning and add tests
2021-08-30 01:00:37 +00:00
## Sync committee pool
```diff
+ Aggregating votes OK
+ An empty pool is safe to prune OK
+ An empty pool is safe to prune 2 OK
+ An empty pool is safe to use OK
```
OK: 4/4 Fail: 0/4 Skip: 0/4
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
## SyncManager test suite
```diff
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
+ Process all unviable blocks OK
Backward sync support for SyncManager. (#3131) * Unbundle SyncQueue from sync_manager.nim. Unbundle Peer scores constants to peer_scores.nim. Add Forward/Backward enum. * Further improvements and tests. * Adopt getRewindPoint() and fix MissingParent handler. * Remove unused procedures. Refactor `result` usage. Fix resetWait(). * Add all the tests and fix the issue with rewind point. * Fix get() issue. * Fix flaky tests. * test fixes Co-authored-by: Jacek Sieka <jacek@status.im>
2021-12-08 21:15:29 +00:00
+ [SyncQueue#Backward] Async unordered push test OK
+ [SyncQueue#Backward] Async unordered push with rewind test OK
avoid re-requesting finalized blocks during sync (#3461) When a `beaconBlocksByRange` response advances the `safeSlot`, but later has errors, the sync queue keeps repeating that same request until it is fulfilled without errors. Data up through `safeSlot` is considered to be immutable, i.e., finalized, so re-requesting that data is not useful. By advancing the sync progress in that scenario, those redundant query portions can be avoided. Note, the finalized block _itself_ is always requested, even in the initial request. This behaviour is kept same.
2022-03-15 17:56:56 +00:00
+ [SyncQueue#Backward] Good response with missing values towards end OK
+ [SyncQueue#Backward] Handle out-of-band sync progress advancement OK
Backward sync support for SyncManager. (#3131) * Unbundle SyncQueue from sync_manager.nim. Unbundle Peer scores constants to peer_scores.nim. Add Forward/Backward enum. * Further improvements and tests. * Adopt getRewindPoint() and fix MissingParent handler. * Remove unused procedures. Refactor `result` usage. Fix resetWait(). * Add all the tests and fix the issue with rewind point. * Fix get() issue. * Fix flaky tests. * test fixes Co-authored-by: Jacek Sieka <jacek@status.im>
2021-12-08 21:15:29 +00:00
+ [SyncQueue#Backward] Pass through established limits test OK
+ [SyncQueue#Backward] Smoke test OK
+ [SyncQueue#Backward] Start and finish slots equal OK
+ [SyncQueue#Backward] Two full requests success/fail OK
+ [SyncQueue#Backward] getRewindPoint() test OK
+ [SyncQueue#Forward] Async unordered push test OK
+ [SyncQueue#Forward] Async unordered push with rewind test OK
avoid re-requesting finalized blocks during sync (#3461) When a `beaconBlocksByRange` response advances the `safeSlot`, but later has errors, the sync queue keeps repeating that same request until it is fulfilled without errors. Data up through `safeSlot` is considered to be immutable, i.e., finalized, so re-requesting that data is not useful. By advancing the sync progress in that scenario, those redundant query portions can be avoided. Note, the finalized block _itself_ is always requested, even in the initial request. This behaviour is kept same.
2022-03-15 17:56:56 +00:00
+ [SyncQueue#Forward] Good response with missing values towards end OK
+ [SyncQueue#Forward] Handle out-of-band sync progress advancement OK
Backward sync support for SyncManager. (#3131) * Unbundle SyncQueue from sync_manager.nim. Unbundle Peer scores constants to peer_scores.nim. Add Forward/Backward enum. * Further improvements and tests. * Adopt getRewindPoint() and fix MissingParent handler. * Remove unused procedures. Refactor `result` usage. Fix resetWait(). * Add all the tests and fix the issue with rewind point. * Fix get() issue. * Fix flaky tests. * test fixes Co-authored-by: Jacek Sieka <jacek@status.im>
2021-12-08 21:15:29 +00:00
+ [SyncQueue#Forward] Pass through established limits test OK
+ [SyncQueue#Forward] Smoke test OK
+ [SyncQueue#Forward] Start and finish slots equal OK
+ [SyncQueue#Forward] Two full requests success/fail OK
+ [SyncQueue#Forward] getRewindPoint() test OK
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
+ [SyncQueue] checkResponse() test OK
+ [SyncQueue] contains() test OK
+ [SyncQueue] getLastNonEmptySlot() test OK
+ [SyncQueue] hasEndGap() test OK
```
avoid re-requesting finalized blocks during sync (#3461) When a `beaconBlocksByRange` response advances the `safeSlot`, but later has errors, the sync queue keeps repeating that same request until it is fulfilled without errors. Data up through `safeSlot` is considered to be immutable, i.e., finalized, so re-requesting that data is not useful. By advancing the sync progress in that scenario, those redundant query portions can be avoided. Note, the finalized block _itself_ is always requested, even in the initial request. This behaviour is kept same.
2022-03-15 17:56:56 +00:00
OK: 23/23 Fail: 0/23 Skip: 0/23
Add test reports
2020-03-10 04:24:33 +00:00
## Zero signature sanity checks
```diff
+ SSZ serialization roundtrip of SignedBeaconBlockHeader OK
synchronize AllTests-mainnet.md with new zero signature sanity checks (#2745)
2021-07-28 20:36:23 +00:00
+ Zero signatures cannot be loaded into a BLS signature object OK
+ default initialization of signatures OK
Add test reports
2020-03-10 04:24:33 +00:00
```
synchronize AllTests-mainnet.md with new zero signature sanity checks (#2745)
2021-07-28 20:36:23 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
Add test reports
2020-03-10 04:24:33 +00:00
## [Unit - Spec - Block processing] Deposits [Preset: mainnet]
```diff
+ Deposit at MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Deposit over MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Deposit under MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
Merge devel and resolve the conflicts
2020-07-08 12:36:03 +00:00
+ Invalid deposit at MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
Add test reports
2020-03-10 04:24:33 +00:00
+ Validator top-up OK
```
Merge devel and resolve the conflicts
2020-07-08 12:36:03 +00:00
OK: 5/5 Fail: 0/5 Skip: 0/5
Add test reports
2020-03-10 04:24:33 +00:00
## [Unit - Spec - Epoch processing] Justification and Finalization [Preset: mainnet]
```diff
+ Rule I - 234 finalization with enough support OK
+ Rule I - 234 finalization without support OK
+ Rule II - 23 finalization with enough support OK
+ Rule II - 23 finalization without support OK
+ Rule III - 123 finalization with enough support OK
+ Rule III - 123 finalization without support OK
+ Rule IV - 12 finalization with enough support OK
+ Rule IV - 12 finalization without support OK
```
OK: 8/8 Fail: 0/8 Skip: 0/8
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
## chain DAG finalization tests [Preset: mainnet]
```diff
remove quadratic deposit Merkle tree initialization
2020-08-13 23:37:58 +00:00
+ init with gaps [Preset: mainnet] OK
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
+ orphaned epoch block [Preset: mainnet] OK
+ prune heads on finalization [Preset: mainnet] OK
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
```
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
update AllTests-mainnet.md (#3363)
2022-02-08 14:39:15 +00:00
## createValidatorFiles()
Implementation of the Keymanager API (BETA) https://github.com/ethereum/keymanager-APIs
2021-12-22 12:37:31 +00:00
```diff
update AllTests-mainnet.md (#3363)
2022-02-08 14:39:15 +00:00
+ Add keystore files [LOCAL] OK
+ Add keystore files [REMOTE] OK
+ Add keystore files twice [LOCAL] OK
+ Add keystore files twice [REMOTE] OK
Implement a missing ingnore rule for sync committee contributions (#3941)
2022-08-09 09:52:11 +00:00
+ `createLocalValidatorFiles` with `keystoreDir` without permissions OK
+ `createLocalValidatorFiles` with `secretsDir` without permissions OK
+ `createLocalValidatorFiles` with `validatorsDir` without permissions OK
Implementation of the Keymanager API (BETA) https://github.com/ethereum/keymanager-APIs
2021-12-22 12:37:31 +00:00
+ `createValidatorFiles` with already existing dirs and any error OK
```
update AllTests-mainnet.md (#3363)
2022-02-08 14:39:15 +00:00
OK: 8/8 Fail: 0/8 Skip: 0/8
implement --jwt-secret and HS256 JWT/JWS signing for engine API alpha.7 (#3440)
2022-02-27 16:55:02 +00:00
## engine API authentication
```diff
+ HS256 JWS iat token signing OK
+ HS256 JWS signing OK
+ getIatToken OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
test fixture cleanups (#2976) * "official" -> "scenarios", like the submodule * fewer test binaries - various compile hacks have been improved over time, test suite should follow * remove obsolete bls tests - there are better test vectors in nim-blscurve * remove obsolete mentions of `ssz_testing` * remove obsolete comments about proc vs globals, unittest2 already uses proc's
2021-10-12 11:36:52 +00:00
## eth2.0-deposits-cli compatibility
```diff
+ restoring mnemonic with password OK
+ restoring mnemonic without password OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
update AllTests-mainnet.md (#3363)
2022-02-08 14:39:15 +00:00
## removeValidatorFiles()
Implementation of the Keymanager API (BETA) https://github.com/ethereum/keymanager-APIs
2021-12-22 12:37:31 +00:00
```diff
+ Remove nonexistent validator OK
+ Remove validator files OK
+ Remove validator files twice OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
update AllTests-mainnet.md (#3363)
2022-02-08 14:39:15 +00:00
## removeValidatorFiles() multiple keystore types
```diff
+ Remove [LOCAL] when [LOCAL] is missing OK
+ Remove [LOCAL] when [LOCAL] is present OK
+ Remove [LOCAL] when [REMOTE] is present OK
+ Remove [REMOTE] when [LOCAL] is present OK
+ Remove [REMOTE] when [REMOTE] is missing OK
+ Remove [REMOTE] when [REMOTE] is present OK
```
OK: 6/6 Fail: 0/6 Skip: 0/6
## saveKeystore()
```diff
+ Save [LOCAL] keystore after [LOCAL] keystore with different id OK
+ Save [LOCAL] keystore after [LOCAL] keystore with same id OK
+ Save [LOCAL] keystore after [REMOTE] keystore with different id OK
+ Save [LOCAL] keystore after [REMOTE] keystore with same id OK
+ Save [REMOTE] keystore after [LOCAL] keystore with different id OK
+ Save [REMOTE] keystore after [LOCAL] keystore with same id OK
+ Save [REMOTE] keystore after [REMOTE] keystore with different id OK
+ Save [REMOTE] keystore after [REMOTE] keystore with same id OK
```
OK: 8/8 Fail: 0/8 Skip: 0/8
initial infrastructure for state diffs (#2087) Initial infrastructure for state diffs
2021-01-18 20:34:41 +00:00
## state diff tests [Preset: mainnet]
```diff
+ random slot differences [Preset: mainnet] OK
```
remove delta-encoding from state diff balances (#2397) * remove delta-encoding from state diff balances * switch HashList to List
2021-03-11 05:39:04 +00:00
OK: 1/1 Fail: 0/1 Skip: 0/1
add Altair and Merge rewards tests (#2995)
2021-10-18 14:54:17 +00:00
## subnet tracker
```diff
+ should register stability subnets on attester duties OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
include capella fork version in fork consistency check (#3772) * include capella fork version in fork consistency check * Update tests/test_conf.nim Co-authored-by: Etan Kissling <etan@status.im> Co-authored-by: Etan Kissling <etan@status.im>
2022-06-18 10:05:33 +00:00
## weak-subjectivity-checkpoint
```diff
+ Correct values OK
+ invalid characters in root OK
+ longer root OK
+ missing epoch OK
+ missing root OK
+ missing separator OK
+ negative epoch OK
+ non-number epoch OK
+ shorter root OK
```
OK: 9/9 Fail: 0/9 Skip: 0/9
Add test reports
2020-03-10 04:24:33 +00:00
---TOTAL---
test randao skippping (complements #3837) (#4179)
2022-09-27 07:22:24 +00:00
OK: 327/332 Fail: 0/332 Skip: 5/332