status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/AllTests-mainnet.md

456 lines
29 KiB
Markdown
Raw Normal View History

Add test reports
2020-03-10 04:24:33 +00:00
AllTests-mainnet
===
test fixture cleanups (#2976) * "official" -> "scenarios", like the submodule * fewer test binaries - various compile hacks have been improved over time, test suite should follow * remove obsolete bls tests - there are better test vectors in nim-blscurve * remove obsolete mentions of `ssz_testing` * remove obsolete comments about proc vs globals, unittest2 already uses proc's
2021-10-12 11:36:52 +00:00
##
```diff
+ Slashing test: duplicate_pubkey_not_slashable.json OK
+ Slashing test: multiple_interchanges_single_validator_single_message_gap.json OK
+ Slashing test: multiple_interchanges_single_validator_single_message_out_of_order.json OK
+ Slashing test: multiple_validators_multiple_blocks_and_attestations.json OK
+ Slashing test: multiple_validators_same_slot_blocks.json OK
+ Slashing test: single_validator_genesis_attestation.json OK
+ Slashing test: single_validator_import_only.json OK
+ Slashing test: single_validator_multiple_block_attempts.json OK
+ Slashing test: single_validator_multiple_blocks_and_attestations.json OK
+ Slashing test: single_validator_out_of_order_attestations.json OK
+ Slashing test: single_validator_out_of_order_blocks.json OK
+ Slashing test: single_validator_resign_attestation.json OK
+ Slashing test: single_validator_resign_block.json OK
+ Slashing test: single_validator_single_attestation.json OK
+ Slashing test: single_validator_single_block.json OK
+ Slashing test: single_validator_single_block_and_attestation.json OK
+ Slashing test: single_validator_single_block_and_attestation_signing_root.json OK
+ Slashing test: single_validator_slashable_attestations_double_vote.json OK
+ Slashing test: single_validator_slashable_attestations_surrounded_by_existing.json OK
+ Slashing test: single_validator_slashable_attestations_surrounds_existing.json OK
+ Slashing test: single_validator_slashable_blocks.json OK
+ Slashing test: single_validator_slashable_blocks_no_root.json OK
+ Slashing test: single_validator_source_greater_than_target.json OK
+ Slashing test: single_validator_two_blocks_no_signing_root.json OK
+ Slashing test: wrong_genesis_validators_root.json OK
```
OK: 25/25 Fail: 0/25 Skip: 0/25
re-enable mainnet attestation pool tests (#946) * re-enable mainnet attestation pool tests; since https://github.com/status-im/nim-beacon-chain/pull/930 double CI state sim validators * bump deposits spec ref to v0.11.1
2020-04-29 11:44:07 +00:00
## Attestation pool processing [Preset: mainnet]
```diff
+ Attestations may arrive in any order [Preset: mainnet] OK
+ Attestations may overlap, bigger first [Preset: mainnet] OK
+ Attestations may overlap, smaller first [Preset: mainnet] OK
+ Attestations should be combined [Preset: mainnet] OK
Revamp attestation pool This is a revamp of the attestation pool that cleans up several aspects of attestation processing as the network grows larger and block space becomes more precious. The aim is to better exploit the divide between attestation subnets and aggregations by keeping the two kinds separate until it's time to either produce a block or aggregate. This means we're no longer eagerly combining single-vote attestations, but rather wait until the last moment, and then try to add singles to all aggregates, including those coming from the network. Importantly, the branch improves on poor aggregate quality and poor attestation packing in cases where block space is running out. A basic greed scoring mechanism is used to select attestations for blocks - attestations are added based on how much many new votes they bring to the table. * Collect single-vote attestations separately and store these until it's time to make aggregates * Create aggregates based on single-vote attestations * Select _best_ aggregate rather than _first_ aggregate when on aggregation duty * Top up all aggregates with singles when it's time make the attestation cut, thus improving the chances of grabbing the best aggregates out there * Improve aggregation test coverage * Improve bitseq operations * Simplify aggregate signature creation * Make attestation cache temporary instead of storing it in attestation pool - most of the time, blocks are not being produced, no need to keep the data around * Remove redundant aggregate storage that was used only for RPC * Use tables to avoid some linear seeks when looking up attestation data * Fix long cleanup on large slot jumps * Avoid some pointers * Speed up iterating all attestations for a slot (fixes #2490)
2021-04-12 20:25:09 +00:00
+ Can add and retrieve simple attestations [Preset: mainnet] OK
+ Everyone voting for something different [Preset: mainnet] OK
re-enable mainnet attestation pool tests (#946) * re-enable mainnet attestation pool tests; since https://github.com/status-im/nim-beacon-chain/pull/930 double CI state sim validators * bump deposits spec ref to v0.11.1
2020-04-29 11:44:07 +00:00
+ Fork choice returns block with attestation OK
+ Fork choice returns latest block with no attestations OK
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
+ Trying to add a block twice tags the second as an error OK
+ Trying to add a duplicate block from an old pruned epoch is tagged as an error OK
Revamp attestation pool This is a revamp of the attestation pool that cleans up several aspects of attestation processing as the network grows larger and block space becomes more precious. The aim is to better exploit the divide between attestation subnets and aggregations by keeping the two kinds separate until it's time to either produce a block or aggregate. This means we're no longer eagerly combining single-vote attestations, but rather wait until the last moment, and then try to add singles to all aggregates, including those coming from the network. Importantly, the branch improves on poor aggregate quality and poor attestation packing in cases where block space is running out. A basic greed scoring mechanism is used to select attestations for blocks - attestations are added based on how much many new votes they bring to the table. * Collect single-vote attestations separately and store these until it's time to make aggregates * Create aggregates based on single-vote attestations * Select _best_ aggregate rather than _first_ aggregate when on aggregation duty * Top up all aggregates with singles when it's time make the attestation cut, thus improving the chances of grabbing the best aggregates out there * Improve aggregation test coverage * Improve bitseq operations * Simplify aggregate signature creation * Make attestation cache temporary instead of storing it in attestation pool - most of the time, blocks are not being produced, no need to keep the data around * Remove redundant aggregate storage that was used only for RPC * Use tables to avoid some linear seeks when looking up attestation data * Fix long cleanup on large slot jumps * Avoid some pointers * Speed up iterating all attestations for a slot (fixes #2490)
2021-04-12 20:25:09 +00:00
+ Working with aggregates [Preset: mainnet] OK
re-enable mainnet attestation pool tests (#946) * re-enable mainnet attestation pool tests; since https://github.com/status-im/nim-beacon-chain/pull/930 double CI state sim validators * bump deposits spec ref to v0.11.1
2020-04-29 11:44:07 +00:00
```
Revamp attestation pool This is a revamp of the attestation pool that cleans up several aspects of attestation processing as the network grows larger and block space becomes more precious. The aim is to better exploit the divide between attestation subnets and aggregations by keeping the two kinds separate until it's time to either produce a block or aggregate. This means we're no longer eagerly combining single-vote attestations, but rather wait until the last moment, and then try to add singles to all aggregates, including those coming from the network. Importantly, the branch improves on poor aggregate quality and poor attestation packing in cases where block space is running out. A basic greed scoring mechanism is used to select attestations for blocks - attestations are added based on how much many new votes they bring to the table. * Collect single-vote attestations separately and store these until it's time to make aggregates * Create aggregates based on single-vote attestations * Select _best_ aggregate rather than _first_ aggregate when on aggregation duty * Top up all aggregates with singles when it's time make the attestation cut, thus improving the chances of grabbing the best aggregates out there * Improve aggregation test coverage * Improve bitseq operations * Simplify aggregate signature creation * Make attestation cache temporary instead of storing it in attestation pool - most of the time, blocks are not being produced, no need to keep the data around * Remove redundant aggregate storage that was used only for RPC * Use tables to avoid some linear seeks when looking up attestation data * Fix long cleanup on large slot jumps * Avoid some pointers * Speed up iterating all attestations for a slot (fixes #2490)
2021-04-12 20:25:09 +00:00
OK: 11/11 Fail: 0/11 Skip: 0/11
Add test reports
2020-03-10 04:24:33 +00:00
## Beacon chain DB [Preset: mainnet]
```diff
+ empty database [Preset: mainnet] OK
+ find ancestors [Preset: mainnet] OK
add blockchain_dag altair database reading; add rollback tests (#2683) * add blockchain_dag altair database reading; add rollback tests; fix some unnecessary type conversions * remove debugging scaffolding * proposeSignedBlock() will need to be async for merge; introduce altair types to VC
2021-06-29 15:09:29 +00:00
+ sanity check Altair and cross-fork getState rollback [Preset: mainnet] OK
introduce database support for Altair (#2667) * introduce immutable Altair BeaconState * add database support for Altair blocks and states * add tests for Altair get/put/contains/delete state * enable blockchain_dag Altair state database storing * properly return error on getting missing altair block
2021-06-24 07:11:47 +00:00
+ sanity check Altair blocks [Preset: mainnet] OK
+ sanity check Altair states [Preset: mainnet] OK
+ sanity check Altair states, reusing buffers [Preset: mainnet] OK
merge hardfork database support (#2911) * merge hardfork database support * working block_sim * recreate state transition changes
2021-09-30 01:07:24 +00:00
+ sanity check Merge and cross-fork getState rollback [Preset: mainnet] OK
+ sanity check Merge blocks [Preset: mainnet] OK
+ sanity check Merge states [Preset: mainnet] OK
+ sanity check Merge states, reusing buffers [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ sanity check genesis roundtrip [Preset: mainnet] OK
introduce database support for Altair (#2667) * introduce immutable Altair BeaconState * add database support for Altair blocks and states * add tests for Altair get/put/contains/delete state * enable blockchain_dag Altair state database storing * properly return error on getting missing altair block
2021-06-24 07:11:47 +00:00
+ sanity check phase 0 blocks [Preset: mainnet] OK
add blockchain_dag altair database reading; add rollback tests (#2683) * add blockchain_dag altair database reading; add rollback tests; fix some unnecessary type conversions * remove debugging scaffolding * proposeSignedBlock() will need to be async for merge; introduce altair types to VC
2021-06-29 15:09:29 +00:00
+ sanity check phase 0 getState rollback [Preset: mainnet] OK
introduce database support for Altair (#2667) * introduce immutable Altair BeaconState * add database support for Altair blocks and states * add tests for Altair get/put/contains/delete state * enable blockchain_dag Altair state database storing * properly return error on getting missing altair block
2021-06-24 07:11:47 +00:00
+ sanity check phase 0 states [Preset: mainnet] OK
+ sanity check phase 0 states, reusing buffers [Preset: mainnet] OK
initial infrastructure for state diffs (#2087) Initial infrastructure for state diffs
2021-01-18 20:34:41 +00:00
+ sanity check state diff roundtrip [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
```
merge hardfork database support (#2911) * merge hardfork database support * working block_sim * recreate state transition changes
2021-09-30 01:07:24 +00:00
OK: 16/16 Fail: 0/16 Skip: 0/16
Add test reports
2020-03-10 04:24:33 +00:00
## Beacon state [Preset: mainnet]
```diff
rename initialize_beacon_state to initialize_beacon_state_from_eth1 (#2536)
2021-05-04 10:19:11 +00:00
+ Smoke test initialize_beacon_state_from_eth1 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
## Bit fields
```diff
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
+ isZeros OK
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
+ iterating words OK
+ overlaps OK
Perform attestation check before broadcast (#2550) Currently, we have a bit of a convoluted flow where when sending attestations, we start broadcasting them over gossip then pass them to the attestation validation to include them in the local attestation pool - it should be the other way around: we should be checking attestations _before_ gossipping them - this serves as an additional safety net to ensure that we don't publish junk - this becomes more important when publishing attestations from the API. Also, the REST API was performing its own validation meaning attestations coming from REST would be validated twice - finally, the JSON RPC wasn't pre-validating and would happily broadcast invalid attestations. * Unified attestation production pipeline with the same flow for gossip, locally and API-produced attestations: all are now validated and entered into the pool, then broadcast/republished * Refactor subnet handling with specific SubnetId alias, streamlining where subnets are computed, avoiding the need to pass around the number of active validators * Move some of the subnet handling code to eth2_network * Use BitArray throughout for subnet handling
2021-05-10 07:13:36 +00:00
+ roundtrips BitArray OK
+ roundtrips BitSeq OK
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
```
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
OK: 5/5 Fail: 0/5 Skip: 0/5
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
## Block pool processing [Preset: mainnet]
```diff
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
+ Adding the same block twice returns a Duplicate error [Preset: mainnet] OK
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
+ Reverse order block add & get [Preset: mainnet] OK
+ Simple block add&get [Preset: mainnet] OK
+ getRef returns nil for missing blocks OK
improve slot processing speeds (#1670) about 40% better slot processing times (with LTO enabled) - these don't do BLS but are used heavily during replay (state transition = slot + block transition) tests using a recent medalla state and advancing it 1000 slots: ``` ./ncli slots --preState2:state-302271-3c1dbf19-c1f944bf.ssz --slot:1000 --postState2:xx.ssz ``` pre: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 39.236, 0.000, 39.236, 39.236, 1, Load state from file 0.049, 0.002, 0.046, 0.063, 968, Apply slot 256.504, 81.008, 213.471, 591.902, 32, Apply epoch slot 28.597, 0.000, 28.597, 28.597, 1, Save state to file ``` cast: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 37.079, 0.000, 37.079, 37.079, 1, Load state from file 0.042, 0.002, 0.040, 0.090, 968, Apply slot 215.552, 68.763, 180.155, 500.103, 32, Apply epoch slot 25.106, 0.000, 25.106, 25.106, 1, Save state to file ``` cast+rewards: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 40.049, 0.000, 40.049, 40.049, 1, Load state from file 0.048, 0.001, 0.045, 0.060, 968, Apply slot 164.981, 76.273, 142.099, 477.868, 32, Apply epoch slot 28.498, 0.000, 28.498, 28.498, 1, Save state to file ``` cast+rewards+shr ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 12.898, 0.000, 12.898, 12.898, 1, Load state from file 0.039, 0.002, 0.038, 0.054, 968, Apply slot 139.971, 68.797, 120.088, 428.844, 32, Apply epoch slot 24.761, 0.000, 24.761, 24.761, 1, Save state to file ```
2020-09-16 20:59:33 +00:00
+ loading tail block works [Preset: mainnet] OK
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
+ updateHead updates head and headState [Preset: mainnet] OK
+ updateStateData sanity [Preset: mainnet] OK
```
OK: 7/7 Fail: 0/7 Skip: 0/7
Add test reports
2020-03-10 04:24:33 +00:00
## BlockRef and helpers [Preset: mainnet]
```diff
rework epoch cache referencing * collect all epochrefs in specific blocks to make them easier to find and to avoid lots of small seqs * reuse validator key databases more aggressively by comparing keys * make state cache available from within `withState` * make epochRef available from within onBlockAdded callback * integrate getEpochInfo into block resolution and epoch ref logic such that epochrefs are created when blocks are added to pool or lazily when needed by a getEpochRef * fill state cache better from EpochRef, speeding up replay and validation * store epochRef in specific blocks to make them easier to find and reuse * fix database corruption when state is saved while replaying quarantine * replay slots fully from block pool before processing state * compare bls values more smartly * store epoch state without block applied in database - it's recommended to resync the node! this branch will drastically speed up processing in times of long non-finality, as well as cut memory usage by 10x during the recent medalla madness.
2020-08-18 20:29:33 +00:00
+ epochAncestor sanity [Preset: mainnet] OK
update attestation and block validation to v0.12.2; clean up getAncestorAt()/get_ancestor() (#1417) * update attestation validation to v0.12.2; clean up getAncestorAt()/get_ancestor() * update beacon block validation to v0.12.2
2020-08-03 19:47:42 +00:00
+ get_ancestor sanity [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ isAncestorOf sanity [Preset: mainnet] OK
```
rework epoch cache referencing * collect all epochrefs in specific blocks to make them easier to find and to avoid lots of small seqs * reuse validator key databases more aggressively by comparing keys * make state cache available from within `withState` * make epochRef available from within onBlockAdded callback * integrate getEpochInfo into block resolution and epoch ref logic such that epochrefs are created when blocks are added to pool or lazily when needed by a getEpochRef * fill state cache better from EpochRef, speeding up replay and validation * store epochRef in specific blocks to make them easier to find and reuse * fix database corruption when state is saved while replaying quarantine * replay slots fully from block pool before processing state * compare bls values more smartly * store epoch state without block applied in database - it's recommended to resync the node! this branch will drastically speed up processing in times of long non-finality, as well as cut memory usage by 10x during the recent medalla madness.
2020-08-18 20:29:33 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
Add test reports
2020-03-10 04:24:33 +00:00
## BlockSlot and helpers [Preset: mainnet]
```diff
+ atSlot sanity [Preset: mainnet] OK
+ parent sanity [Preset: mainnet] OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
discard putative blocks from invalid hardforks
2021-09-08 03:46:33 +00:00
## Diverging hardforks
```diff
+ Non-tail block in common OK
+ Tail block only in common OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
## Eth1 monitor
```diff
+ Rewrite HTTPS Infura URLs OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
update some v1.1.0 alpha1 to alpha2 (#2457) * update some v1.1.0 alpha1 to alpha2 * remove unused getDepositMessage overload and move other out of datatypes/base * bump nim-eth2-scenarios to download v1.1.0-alpha.2 test vectors * construct object rather than result
2021-03-29 19:17:48 +00:00
## Eth2 specific discovery tests
```diff
+ Invalid attnets field OK
+ Subnet query OK
+ Subnet query after ENR update OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
test fixture cleanups (#2976) * "official" -> "scenarios", like the submodule * fewer test binaries - various compile hacks have been improved over time, test suite should follow * remove obsolete bls tests - there are better test vectors in nim-blscurve * remove obsolete mentions of `ssz_testing` * remove obsolete comments about proc vs globals, unittest2 already uses proc's
2021-10-12 11:36:52 +00:00
## Ethereum Foundation - SSZ generic types
```diff
Testing basic_vector inputs - invalid Skip
+ Testing basic_vector inputs - valid OK
+ Testing bitlist inputs - invalid OK
+ Testing bitlist inputs - valid OK
Testing bitvector inputs - invalid Skip
+ Testing bitvector inputs - valid OK
+ Testing boolean inputs - invalid OK
+ Testing boolean inputs - valid OK
+ Testing containers inputs - invalid - skipping BitsStruct OK
+ Testing containers inputs - valid - skipping BitsStruct OK
+ Testing uints inputs - invalid OK
+ Testing uints inputs - valid OK
```
OK: 10/12 Fail: 0/12 Skip: 2/12
refactor exit pool and gossipsub-subscribe to exit pool message types (#1733) * refactor exit pool and gossipsub-subscribe to all exit pool message types * remove unused loop counter
2020-09-24 17:05:49 +00:00
## Exit pool testing suite
```diff
+ addExitMessage/getAttesterSlashingMessage OK
+ addExitMessage/getProposerSlashingMessage OK
+ addExitMessage/getVoluntaryExitMessage OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
[Ready 1/2] Fork choice rewrite (#865) * initial fork-choice refactor * Add fork_choice test for "no votes" * Initial test with voting: fix handling of unknown validators and parent blocks * Fix tiebreak of votes * Cleanup debugging traces * Complexify the vote test * fakeHash use the bigEndian repr of number + fix tiebreak for good * Stash changes: found critical bug in nimcrypto `==` and var openarray * Passing fork choice tests with varying votes * Add FFG fork choice scenario + fork choice to the test suite * Not sure why lmdb / rocksdb reappeared in rebase * Add sanity checks to .nimble file + integrate fork choice tests to the test DB and test timing * Cleanup debugging echos * nimcrypto fix https://github.com/status-im/nim-beacon-chain/pull/864 as been merged, remove TODO comment * Turn fork choice exception-free * Cleanup "result" to ensure early return is properly used * Add a comment on private/public error code vs Result * result -> results following https://github.com/status-im/nim-beacon-chain/pull/866 * Address comments: - raises: [Defect] doesn't work -> TODO - process_attestation cannot fail - try/except as expression pending Nim v1.2.0 - cleanup TODOs * re-enable all sanity checks * tag no raise for process_attestation * use raises defect everywhere in fork choice and fix process_attestation test
2020-04-09 16:15:00 +00:00
## Fork Choice + Finality [Preset: mainnet]
```diff
+ fork_choice - testing finality #01 OK
+ fork_choice - testing finality #02 OK
+ fork_choice - testing no votes OK
+ fork_choice - testing with votes OK
```
OK: 4/4 Fail: 0/4 Skip: 0/4
merge hardfork database support (#2911) * merge hardfork database support * working block_sim * recreate state transition changes
2021-09-30 01:07:24 +00:00
## Fork id compatibility test
```diff
+ Digest check OK
+ Fork check OK
+ Next fork epoch check OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
## Gossip validation [Preset: mainnet]
```diff
add sync committee types and some helpers (#2829)
2021-08-28 09:00:00 +00:00
+ Any committee index is valid OK
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
+ Validation sanity OK
```
add sync committee types and some helpers (#2829)
2021-08-28 09:00:00 +00:00
OK: 2/2 Fail: 0/2 Skip: 0/2
handle duplicate pubkeys in sync committee (#2902) When sync committee message handling was introduced in #2830, the edge case of the same validator being selected multiple times as part of a sync subcommittee was not covered. Not handling that edge case makes sync contributions have a lower-than-expected participation rate as each sync validator is only counted up through once per subcommittee. This patch ensures that this edge case is properly covered.
2021-09-28 07:44:20 +00:00
## Gossip validation - Extra
```diff
+ validateSyncCommitteeMessage OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Add test reports
2020-03-10 04:24:33 +00:00
## Honest validator
```diff
detect already-aggregate-voted condition before attestation pool; add is_aggregator tests (#2170) * detect already-aggregate-voted condition before attestation pool; add is_aggregator tests * replace pair of attestation-per-epoch tracking lists with single list and remove Option use * fix attestation condition * use safer type conversions; add more is_aggregator tests
2020-12-14 20:58:32 +00:00
+ General pubsub topics OK
remove some unused imports, add tests for pubsub topics, and subscribe to interop attestations
2020-05-14 11:19:10 +00:00
+ Mainnet attestation topics OK
detect already-aggregate-voted condition before attestation pool; add is_aggregator tests (#2170) * detect already-aggregate-voted condition before attestation pool; add is_aggregator tests * replace pair of attestation-per-epoch tracking lists with single list and remove Option use * fix attestation condition * use safer type conversions; add more is_aggregator tests
2020-12-14 20:58:32 +00:00
+ is_aggregator OK
Add test reports
2020-03-10 04:24:33 +00:00
```
detect already-aggregate-voted condition before attestation pool; add is_aggregator tests (#2170) * detect already-aggregate-voted condition before attestation pool; add is_aggregator tests * replace pair of attestation-per-epoch tracking lists with single list and remove Option use * fix attestation condition * use safer type conversions; add more is_aggregator tests
2020-12-14 20:58:32 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
re-add test_interop to all_tests and mark several v0.10.1 phase 0 spec references as v0.11.1
2020-04-06 18:55:47 +00:00
## Interop
```diff
+ Interop genesis OK
+ Interop signatures OK
+ Mocked start private key OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
test fixture cleanups (#2976) * "official" -> "scenarios", like the submodule * fewer test binaries - various compile hacks have been improved over time, test suite should follow * remove obsolete bls tests - there are better test vectors in nim-blscurve * remove obsolete mentions of `ssz_testing` * remove obsolete comments about proc vs globals, unittest2 already uses proc's
2021-10-12 11:36:52 +00:00
## KeyStorage testing suite
```diff
+ Pbkdf2 errors OK
+ [PBKDF2] Keystore decryption OK
+ [PBKDF2] Keystore encryption OK
+ [PBKDF2] Network Keystore decryption OK
+ [PBKDF2] Network Keystore encryption OK
+ [SCRYPT] Keystore decryption OK
+ [SCRYPT] Keystore encryption OK
+ [SCRYPT] Network Keystore decryption OK
+ [SCRYPT] Network Keystore encryption OK
```
OK: 9/9 Fail: 0/9 Skip: 0/9
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
## Message signatures
```diff
+ Aggregate and proof signatures OK
+ Attestation signatures OK
+ Deposit signatures OK
+ Slot signatures OK
+ Sync committee message signatures OK
+ Sync committee selection proof signatures OK
+ Sync committee signed contribution and proof signatures OK
+ Voluntary exit signatures OK
```
OK: 8/8 Fail: 0/8 Skip: 0/8
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
## Old database versions [Preset: mainnet]
```diff
+ pre-1.1.0 OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Add test reports
2020-03-10 04:24:33 +00:00
## PeerPool testing suite
```diff
+ Access peers by key test OK
+ Acquire from empty pool OK
+ Acquire/Sorting and consistency test OK
improve slot processing speeds (#1670) about 40% better slot processing times (with LTO enabled) - these don't do BLS but are used heavily during replay (state transition = slot + block transition) tests using a recent medalla state and advancing it 1000 slots: ``` ./ncli slots --preState2:state-302271-3c1dbf19-c1f944bf.ssz --slot:1000 --postState2:xx.ssz ``` pre: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 39.236, 0.000, 39.236, 39.236, 1, Load state from file 0.049, 0.002, 0.046, 0.063, 968, Apply slot 256.504, 81.008, 213.471, 591.902, 32, Apply epoch slot 28.597, 0.000, 28.597, 28.597, 1, Save state to file ``` cast: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 37.079, 0.000, 37.079, 37.079, 1, Load state from file 0.042, 0.002, 0.040, 0.090, 968, Apply slot 215.552, 68.763, 180.155, 500.103, 32, Apply epoch slot 25.106, 0.000, 25.106, 25.106, 1, Save state to file ``` cast+rewards: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 40.049, 0.000, 40.049, 40.049, 1, Load state from file 0.048, 0.001, 0.045, 0.060, 968, Apply slot 164.981, 76.273, 142.099, 477.868, 32, Apply epoch slot 28.498, 0.000, 28.498, 28.498, 1, Save state to file ``` cast+rewards+shr ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 12.898, 0.000, 12.898, 12.898, 1, Load state from file 0.039, 0.002, 0.038, 0.054, 968, Apply slot 139.971, 68.797, 120.088, 428.844, 32, Apply epoch slot 24.761, 0.000, 24.761, 24.761, 1, Save state to file ```
2020-09-16 20:59:33 +00:00
+ Delete peer on release text OK
Add test reports
2020-03-10 04:24:33 +00:00
+ Iterators test OK
+ Peer lifetime test OK
+ Safe/Clear test OK
detect improper 0-length lists of variable-sized objects in SSZ reading (#928) * detect improper 0-length lists of variable-sized objects in SSZ reading
2020-04-23 19:39:23 +00:00
+ Score check test OK
refactor exit pool and gossipsub-subscribe to exit pool message types (#1733) * refactor exit pool and gossipsub-subscribe to all exit pool message types * remove unused loop counter
2020-09-24 17:05:49 +00:00
+ Space tests OK
Add test reports
2020-03-10 04:24:33 +00:00
+ addPeer() test OK
+ addPeerNoWait() test OK
+ deletePeer() test OK
```
refactor exit pool and gossipsub-subscribe to exit pool message types (#1733) * refactor exit pool and gossipsub-subscribe to all exit pool message types * remove unused loop counter
2020-09-24 17:05:49 +00:00
OK: 12/12 Fail: 0/12 Skip: 0/12
Add test reports
2020-03-10 04:24:33 +00:00
## SSZ dynamic navigator
```diff
+ navigating fields OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
disentangle eth2 types from the ssz library (#2785) * reorganize ssz dependencies This PR continues the work in https://github.com/status-im/nimbus-eth2/pull/2646, https://github.com/status-im/nimbus-eth2/pull/2779 as well as past issues with serialization and type, to disentangle SSZ from eth2 and at the same time simplify imports and exports with a structured approach. The principal idea here is that when a library wants to introduce SSZ support, they do so via 3 files: * `ssz_codecs` which imports and reexports `codecs` - this covers the basic byte conversions and ensures no overloads get lost * `xxx_merkleization` imports and exports `merkleization` to specialize and get access to `hash_tree_root` and friends * `xxx_ssz_serialization` imports and exports `ssz_serialization` to specialize ssz for a specific library Those that need to interact with SSZ always import the `xxx_` versions of the modules and never `ssz` itself so as to keep imports simple and safe. This is similar to how the REST / JSON-RPC serializers are structured in that someone wanting to serialize spec types to REST-JSON will import `eth2_rest_serialization` and nothing else. * split up ssz into a core library that is independendent of eth2 types * rename `bytes_reader` to `codec` to highlight that it contains coding and decoding of bytes and native ssz types * remove tricky List init overload that causes compile issues * get rid of top-level ssz import * reenable merkleization tests * move some "standard" json serializers to spec * remove `ValidatorIndex` serialization for now * remove test_ssz_merkleization * add tests for over/underlong byte sequences * fix broken seq[byte] test - seq[byte] is not an SSZ type There are a few things this PR doesn't solve: * like #2646 this PR is weak on how to handle root and other dontSerialize fields that "sometimes" should be computed - the same problem appears in REST / JSON-RPC etc * Fix a build problem on macOS * Another way to fix the macOS builds Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 18:57:58 +00:00
## SSZ generic roundtrip tests
```diff
+ case objects OK
+ lists OK
+ objects OK
+ sets OK
+ simple values OK
+ tables OK
+ tuple OK
```
OK: 7/7 Fail: 0/7 Skip: 0/7
Add test reports
2020-03-10 04:24:33 +00:00
## SSZ navigator
```diff
support all basic types
2020-06-02 09:44:51 +00:00
+ basictype OK
Add test reports
2020-03-10 04:24:33 +00:00
+ lists with max size OK
+ simple object fields OK
```
support all basic types
2020-06-02 09:44:51 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
## Slashing Protection DB - Interchange [Preset: mainnet]
```diff
Slashing protection refactor - EIP 3076 (#2094) * Create CLI tool for slashing export * Use SQLite as a DB instead of a KV-store * Keeps v1 and v2 DBs around * Uses the same schema as Lighthouse v1.1.0 * Passes all interchange tests + skeleton of finalization pruning * Removes tests that would violate v5 / minimal slashing DB and MinSlot rules * Migration tool added using low-watermark scheme for faster migration of large number of validators
2021-02-09 15:23:06 +00:00
+ Smoke test - Complete format - Invalid database is refused [Preset: mainnet] OK
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
+ Smoke test - Complete format [Preset: mainnet] OK
```
Slashing protection refactor - EIP 3076 (#2094) * Create CLI tool for slashing export * Use SQLite as a DB instead of a KV-store * Keeps v1 and v2 DBs around * Uses the same schema as Lighthouse v1.1.0 * Passes all interchange tests + skeleton of finalization pruning * Removes tests that would violate v5 / minimal slashing DB and MinSlot rules * Migration tool added using low-watermark scheme for faster migration of large number of validators
2021-02-09 15:23:06 +00:00
OK: 2/2 Fail: 0/2 Skip: 0/2
## Slashing Protection DB - v1 and v2 migration [Preset: mainnet]
```diff
+ Minimal format migration [Preset: mainnet] OK
```
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
OK: 1/1 Fail: 0/1 Skip: 0/1
## Slashing Protection DB [Preset: mainnet]
```diff
update AllTests-mainnet.md
2020-09-26 21:58:12 +00:00
+ Attestation ordering #1698 OK
Slashing db pruning [Merge only after v2 has been default for 1 noticeable release] (#2452) * Enable slashing DB pruning * integrate slashing DB pruning with onSlotEnd * rebase tests
2021-05-10 14:32:28 +00:00
+ Don't prune the very last attestation(s) even by mistake OK
+ Don't prune the very last block even by mistake OK
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
+ Empty database [Preset: mainnet] OK
Slashing db pruning [Merge only after v2 has been default for 1 noticeable release] (#2452) * Enable slashing DB pruning * integrate slashing DB pruning with onSlotEnd * rebase tests
2021-05-10 14:32:28 +00:00
+ Pruning attestations works OK
+ Pruning blocks works OK
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
+ SP for block proposal - backtracking append OK
+ SP for block proposal - linear append OK
+ SP for same epoch attestation target - linear append OK
+ SP for surrounded attestations OK
+ SP for surrounding attestations OK
update AllTests-mainnet.md
2020-09-26 21:58:12 +00:00
+ Test valid attestation #1699 OK
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
```
Slashing db pruning [Merge only after v2 has been default for 1 noticeable release] (#2452) * Enable slashing DB pruning * integrate slashing DB pruning with onSlotEnd * rebase tests
2021-05-10 14:32:28 +00:00
OK: 12/12 Fail: 0/12 Skip: 0/12
Restore compilation with custom presets (#1309) This also assigns precise types to the constants in the minimal and mainnet presets in order to reduce the chance of compilation errors when custom presets are used (previously, only the custom presets have precisely assigned types for the constants).
2020-07-13 14:44:58 +00:00
## Spec datatypes
```diff
+ Graffiti bytes OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Add test reports
2020-03-10 04:24:33 +00:00
## Spec helpers
```diff
add function to build merkle proofs (#2874) Adds a function that constructs a Merkle proof for a generalized index. This will be used during light client sync to update light clients with a new state (see NEXT_SYNC_COMMITTEE_INDEX / FINALIZED_ROOT_INDEX).
2021-09-29 13:02:34 +00:00
+ build_proof - BeaconState OK
support merkle multiproof verification (#2980) This introduces spec-compatible support to verify merkle multiproofs for use, e.g., in light client syncing.
2021-10-14 08:28:22 +00:00
+ get_branch_indices OK
+ get_helper_indices OK
+ get_path_indices OK
Add test reports
2020-03-10 04:24:33 +00:00
+ integer_squareroot OK
support merkle multiproof verification (#2980) This introduces spec-compatible support to verify merkle multiproofs for use, e.g., in light client syncing.
2021-10-14 08:28:22 +00:00
+ verify_merkle_multiproof OK
Add test reports
2020-03-10 04:24:33 +00:00
```
support merkle multiproof verification (#2980) This introduces spec-compatible support to verify merkle multiproofs for use, e.g., in light client syncing.
2021-10-14 08:28:22 +00:00
OK: 6/6 Fail: 0/6 Skip: 0/6
disentangle eth2 types from the ssz library (#2785) * reorganize ssz dependencies This PR continues the work in https://github.com/status-im/nimbus-eth2/pull/2646, https://github.com/status-im/nimbus-eth2/pull/2779 as well as past issues with serialization and type, to disentangle SSZ from eth2 and at the same time simplify imports and exports with a structured approach. The principal idea here is that when a library wants to introduce SSZ support, they do so via 3 files: * `ssz_codecs` which imports and reexports `codecs` - this covers the basic byte conversions and ensures no overloads get lost * `xxx_merkleization` imports and exports `merkleization` to specialize and get access to `hash_tree_root` and friends * `xxx_ssz_serialization` imports and exports `ssz_serialization` to specialize ssz for a specific library Those that need to interact with SSZ always import the `xxx_` versions of the modules and never `ssz` itself so as to keep imports simple and safe. This is similar to how the REST / JSON-RPC serializers are structured in that someone wanting to serialize spec types to REST-JSON will import `eth2_rest_serialization` and nothing else. * split up ssz into a core library that is independendent of eth2 types * rename `bytes_reader` to `codec` to highlight that it contains coding and decoding of bytes and native ssz types * remove tricky List init overload that causes compile issues * get rid of top-level ssz import * reenable merkleization tests * move some "standard" json serializers to spec * remove `ValidatorIndex` serialization for now * remove test_ssz_merkleization * add tests for over/underlong byte sequences * fix broken seq[byte] test - seq[byte] is not an SSZ type There are a few things this PR doesn't solve: * like #2646 this PR is weak on how to handle root and other dontSerialize fields that "sometimes" should be computed - the same problem appears in REST / JSON-RPC etc * Fix a build problem on macOS * Another way to fix the macOS builds Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 18:57:58 +00:00
## Specific field types
```diff
+ root update OK
+ roundtrip OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
Restore the sync committee pool pruning and add tests
2021-08-30 01:00:37 +00:00
## Sync committee pool
```diff
+ Aggregating votes OK
+ An empty pool is safe to prune OK
+ An empty pool is safe to prune 2 OK
+ An empty pool is safe to use OK
```
OK: 4/4 Fail: 0/4 Skip: 0/4
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
## SyncManager test suite
```diff
+ [SyncQueue] Async pending and resetWait() test OK
+ [SyncQueue] Async unordered push start from zero OK
+ [SyncQueue] Async unordered push with not full start from non-zero OK
+ [SyncQueue] Full and incomplete success/fail start from non-zero OK
+ [SyncQueue] Full and incomplete success/fail start from zero OK
+ [SyncQueue] One smart and one stupid + debt split + empty OK
+ [SyncQueue] Smart and stupid success/fail OK
+ [SyncQueue] Start and finish slots equal OK
+ [SyncQueue] Two full requests success/fail OK
+ [SyncQueue] checkResponse() test OK
+ [SyncQueue] contains() test OK
+ [SyncQueue] getLastNonEmptySlot() test OK
use StateData in place of BeaconState outside state transition code (#2551) * use StateData in place of BeaconState outside state transition code * propagate more StateData usage * remove withStateVars().state * wrap get_beacon_committee(BeaconState, ...) as gbc(StateData, ...) * switch makeAttestation() to use StateData * use StateData wrapper/dispatcher for get_committee_count_per_slot() * convert AttestationCache.init(), weak subjectivity functions, and updateValidatorMetrics() * add get_shuffled_active_validator_indices(StateData) and get_block_root_at_slot(StateData) * switch makeAttestationData() to StateData * sync AllTests-mainnet.md after rebase
2021-05-21 09:23:28 +00:00
+ [SyncQueue] getRewindPoint() test OK
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
+ [SyncQueue] hasEndGap() test OK
```
use StateData in place of BeaconState outside state transition code (#2551) * use StateData in place of BeaconState outside state transition code * propagate more StateData usage * remove withStateVars().state * wrap get_beacon_committee(BeaconState, ...) as gbc(StateData, ...) * switch makeAttestation() to use StateData * use StateData wrapper/dispatcher for get_committee_count_per_slot() * convert AttestationCache.init(), weak subjectivity functions, and updateValidatorMetrics() * add get_shuffled_active_validator_indices(StateData) and get_block_root_at_slot(StateData) * switch makeAttestationData() to StateData * sync AllTests-mainnet.md after rebase
2021-05-21 09:23:28 +00:00
OK: 14/14 Fail: 0/14 Skip: 0/14
Add test reports
2020-03-10 04:24:33 +00:00
## Zero signature sanity checks
```diff
+ SSZ serialization roundtrip of SignedBeaconBlockHeader OK
synchronize AllTests-mainnet.md with new zero signature sanity checks (#2745)
2021-07-28 20:36:23 +00:00
+ Zero signatures cannot be loaded into a BLS signature object OK
+ default initialization of signatures OK
Add test reports
2020-03-10 04:24:33 +00:00
```
synchronize AllTests-mainnet.md with new zero signature sanity checks (#2745)
2021-07-28 20:36:23 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
Add test reports
2020-03-10 04:24:33 +00:00
## [Unit - Spec - Block processing] Attestations [Preset: mainnet]
```diff
+ Valid attestation OK
+ Valid attestation from previous epoch OK
```
update test results summary after BLS 0.10.1 update
2020-03-13 14:58:21 +00:00
OK: 2/2 Fail: 0/2 Skip: 0/2
Add test reports
2020-03-10 04:24:33 +00:00
## [Unit - Spec - Block processing] Deposits [Preset: mainnet]
```diff
+ Deposit at MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Deposit over MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Deposit under MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
Merge devel and resolve the conflicts
2020-07-08 12:36:03 +00:00
+ Invalid deposit at MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
Add test reports
2020-03-10 04:24:33 +00:00
+ Validator top-up OK
```
Merge devel and resolve the conflicts
2020-07-08 12:36:03 +00:00
OK: 5/5 Fail: 0/5 Skip: 0/5
Add test reports
2020-03-10 04:24:33 +00:00
## [Unit - Spec - Epoch processing] Justification and Finalization [Preset: mainnet]
```diff
+ Rule I - 234 finalization with enough support OK
+ Rule I - 234 finalization without support OK
+ Rule II - 23 finalization with enough support OK
+ Rule II - 23 finalization without support OK
+ Rule III - 123 finalization with enough support OK
+ Rule III - 123 finalization without support OK
+ Rule IV - 12 finalization with enough support OK
+ Rule IV - 12 finalization without support OK
```
OK: 8/8 Fail: 0/8 Skip: 0/8
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
## chain DAG finalization tests [Preset: mainnet]
```diff
remove quadratic deposit Merkle tree initialization
2020-08-13 23:37:58 +00:00
+ init with gaps [Preset: mainnet] OK
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
+ orphaned epoch block [Preset: mainnet] OK
+ prune heads on finalization [Preset: mainnet] OK
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
```
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
test fixture cleanups (#2976) * "official" -> "scenarios", like the submodule * fewer test binaries - various compile hacks have been improved over time, test suite should follow * remove obsolete bls tests - there are better test vectors in nim-blscurve * remove obsolete mentions of `ssz_testing` * remove obsolete comments about proc vs globals, unittest2 already uses proc's
2021-10-12 11:36:52 +00:00
## eth2.0-deposits-cli compatibility
```diff
+ restoring mnemonic with password OK
+ restoring mnemonic without password OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
wip: cache * cache lists and arrays of complex objects (5x block processing speed on ncli_db) trivial baseline cache that stores tree in flat memory structure
2020-05-28 16:34:59 +00:00
## hash
```diff
+ HashArray OK
sync AllTests-mainnet.md
2021-05-18 07:45:16 +00:00
+ HashList fixed OK
+ HashList variable OK
wip: cache * cache lists and arrays of complex objects (5x block processing speed on ncli_db) trivial baseline cache that stores tree in flat memory structure
2020-05-28 16:34:59 +00:00
```
sync AllTests-mainnet.md
2021-05-18 07:45:16 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
initial infrastructure for state diffs (#2087) Initial infrastructure for state diffs
2021-01-18 20:34:41 +00:00
## state diff tests [Preset: mainnet]
```diff
+ random slot differences [Preset: mainnet] OK
```
remove delta-encoding from state diff balances (#2397) * remove delta-encoding from state diff balances * switch HashList to List
2021-03-11 05:39:04 +00:00
OK: 1/1 Fail: 0/1 Skip: 0/1
add Altair and Merge rewards tests (#2995)
2021-10-18 14:54:17 +00:00
## subnet tracker
```diff
+ should register stability subnets on attester duties OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
disentangle eth2 types from the ssz library (#2785) * reorganize ssz dependencies This PR continues the work in https://github.com/status-im/nimbus-eth2/pull/2646, https://github.com/status-im/nimbus-eth2/pull/2779 as well as past issues with serialization and type, to disentangle SSZ from eth2 and at the same time simplify imports and exports with a structured approach. The principal idea here is that when a library wants to introduce SSZ support, they do so via 3 files: * `ssz_codecs` which imports and reexports `codecs` - this covers the basic byte conversions and ensures no overloads get lost * `xxx_merkleization` imports and exports `merkleization` to specialize and get access to `hash_tree_root` and friends * `xxx_ssz_serialization` imports and exports `ssz_serialization` to specialize ssz for a specific library Those that need to interact with SSZ always import the `xxx_` versions of the modules and never `ssz` itself so as to keep imports simple and safe. This is similar to how the REST / JSON-RPC serializers are structured in that someone wanting to serialize spec types to REST-JSON will import `eth2_rest_serialization` and nothing else. * split up ssz into a core library that is independendent of eth2 types * rename `bytes_reader` to `codec` to highlight that it contains coding and decoding of bytes and native ssz types * remove tricky List init overload that causes compile issues * get rid of top-level ssz import * reenable merkleization tests * move some "standard" json serializers to spec * remove `ValidatorIndex` serialization for now * remove test_ssz_merkleization * add tests for over/underlong byte sequences * fix broken seq[byte] test - seq[byte] is not an SSZ type There are a few things this PR doesn't solve: * like #2646 this PR is weak on how to handle root and other dontSerialize fields that "sometimes" should be computed - the same problem appears in REST / JSON-RPC etc * Fix a build problem on macOS * Another way to fix the macOS builds Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 18:57:58 +00:00
## underlong values
```diff
Overlong SSZ.decode: BitArray[32] Skip
Overlong SSZ.decode: BitList[32] Skip
Overlong SSZ.decode: HashArray[32, system.uint8] Skip
+ Overlong SSZ.decode: HashList[system.uint64, 32] OK
+ Overlong SSZ.decode: List[system.uint64, 32] OK
Overlong SSZ.decode: Simple Skip
enable uint128 / uint256 SSZ serialization tests (#2960) So far, SSZ tests for `uint128` and `uint256` integers were disabled due to "compile-time issues". The code has been adjusted to support those additional test vectors, and the corresponding tests have been enabled.
2021-10-06 06:14:52 +00:00
Overlong SSZ.decode: UInt128 Skip
Overlong SSZ.decode: UInt256 Skip
disentangle eth2 types from the ssz library (#2785) * reorganize ssz dependencies This PR continues the work in https://github.com/status-im/nimbus-eth2/pull/2646, https://github.com/status-im/nimbus-eth2/pull/2779 as well as past issues with serialization and type, to disentangle SSZ from eth2 and at the same time simplify imports and exports with a structured approach. The principal idea here is that when a library wants to introduce SSZ support, they do so via 3 files: * `ssz_codecs` which imports and reexports `codecs` - this covers the basic byte conversions and ensures no overloads get lost * `xxx_merkleization` imports and exports `merkleization` to specialize and get access to `hash_tree_root` and friends * `xxx_ssz_serialization` imports and exports `ssz_serialization` to specialize ssz for a specific library Those that need to interact with SSZ always import the `xxx_` versions of the modules and never `ssz` itself so as to keep imports simple and safe. This is similar to how the REST / JSON-RPC serializers are structured in that someone wanting to serialize spec types to REST-JSON will import `eth2_rest_serialization` and nothing else. * split up ssz into a core library that is independendent of eth2 types * rename `bytes_reader` to `codec` to highlight that it contains coding and decoding of bytes and native ssz types * remove tricky List init overload that causes compile issues * get rid of top-level ssz import * reenable merkleization tests * move some "standard" json serializers to spec * remove `ValidatorIndex` serialization for now * remove test_ssz_merkleization * add tests for over/underlong byte sequences * fix broken seq[byte] test - seq[byte] is not an SSZ type There are a few things this PR doesn't solve: * like #2646 this PR is weak on how to handle root and other dontSerialize fields that "sometimes" should be computed - the same problem appears in REST / JSON-RPC etc * Fix a build problem on macOS * Another way to fix the macOS builds Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 18:57:58 +00:00
Overlong SSZ.decode: array[0..31, byte] Skip
Overlong SSZ.decode: bool Skip
Overlong SSZ.decode: limb_t Skip
Overlong SSZ.decode: uint16 Skip
Overlong SSZ.decode: uint32 Skip
Overlong SSZ.decode: uint8 Skip
+ Overlong readSszBytes: BitArray[32] OK
Overlong readSszBytes: BitList[32] Skip
+ Overlong readSszBytes: HashArray[32, system.uint8] OK
+ Overlong readSszBytes: HashList[system.uint64, 32] OK
+ Overlong readSszBytes: List[system.uint64, 32] OK
Overlong readSszBytes: Simple Skip
enable uint128 / uint256 SSZ serialization tests (#2960) So far, SSZ tests for `uint128` and `uint256` integers were disabled due to "compile-time issues". The code has been adjusted to support those additional test vectors, and the corresponding tests have been enabled.
2021-10-06 06:14:52 +00:00
+ Overlong readSszBytes: UInt128 OK
+ Overlong readSszBytes: UInt256 OK
disentangle eth2 types from the ssz library (#2785) * reorganize ssz dependencies This PR continues the work in https://github.com/status-im/nimbus-eth2/pull/2646, https://github.com/status-im/nimbus-eth2/pull/2779 as well as past issues with serialization and type, to disentangle SSZ from eth2 and at the same time simplify imports and exports with a structured approach. The principal idea here is that when a library wants to introduce SSZ support, they do so via 3 files: * `ssz_codecs` which imports and reexports `codecs` - this covers the basic byte conversions and ensures no overloads get lost * `xxx_merkleization` imports and exports `merkleization` to specialize and get access to `hash_tree_root` and friends * `xxx_ssz_serialization` imports and exports `ssz_serialization` to specialize ssz for a specific library Those that need to interact with SSZ always import the `xxx_` versions of the modules and never `ssz` itself so as to keep imports simple and safe. This is similar to how the REST / JSON-RPC serializers are structured in that someone wanting to serialize spec types to REST-JSON will import `eth2_rest_serialization` and nothing else. * split up ssz into a core library that is independendent of eth2 types * rename `bytes_reader` to `codec` to highlight that it contains coding and decoding of bytes and native ssz types * remove tricky List init overload that causes compile issues * get rid of top-level ssz import * reenable merkleization tests * move some "standard" json serializers to spec * remove `ValidatorIndex` serialization for now * remove test_ssz_merkleization * add tests for over/underlong byte sequences * fix broken seq[byte] test - seq[byte] is not an SSZ type There are a few things this PR doesn't solve: * like #2646 this PR is weak on how to handle root and other dontSerialize fields that "sometimes" should be computed - the same problem appears in REST / JSON-RPC etc * Fix a build problem on macOS * Another way to fix the macOS builds Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 18:57:58 +00:00
+ Overlong readSszBytes: array[0..31, byte] OK
+ Overlong readSszBytes: bool OK
+ Overlong readSszBytes: limb_t OK
+ Overlong readSszBytes: uint16 OK
+ Overlong readSszBytes: uint32 OK
+ Overlong readSszBytes: uint8 OK
+ Underlong SSZ.decode: BitArray[32] OK
+ Underlong SSZ.decode: BitList[32] OK
+ Underlong SSZ.decode: HashArray[32, system.uint8] OK
+ Underlong SSZ.decode: HashList[system.uint64, 32] OK
+ Underlong SSZ.decode: List[system.uint64, 32] OK
+ Underlong SSZ.decode: Simple OK
enable uint128 / uint256 SSZ serialization tests (#2960) So far, SSZ tests for `uint128` and `uint256` integers were disabled due to "compile-time issues". The code has been adjusted to support those additional test vectors, and the corresponding tests have been enabled.
2021-10-06 06:14:52 +00:00
+ Underlong SSZ.decode: UInt128 OK
+ Underlong SSZ.decode: UInt256 OK
disentangle eth2 types from the ssz library (#2785) * reorganize ssz dependencies This PR continues the work in https://github.com/status-im/nimbus-eth2/pull/2646, https://github.com/status-im/nimbus-eth2/pull/2779 as well as past issues with serialization and type, to disentangle SSZ from eth2 and at the same time simplify imports and exports with a structured approach. The principal idea here is that when a library wants to introduce SSZ support, they do so via 3 files: * `ssz_codecs` which imports and reexports `codecs` - this covers the basic byte conversions and ensures no overloads get lost * `xxx_merkleization` imports and exports `merkleization` to specialize and get access to `hash_tree_root` and friends * `xxx_ssz_serialization` imports and exports `ssz_serialization` to specialize ssz for a specific library Those that need to interact with SSZ always import the `xxx_` versions of the modules and never `ssz` itself so as to keep imports simple and safe. This is similar to how the REST / JSON-RPC serializers are structured in that someone wanting to serialize spec types to REST-JSON will import `eth2_rest_serialization` and nothing else. * split up ssz into a core library that is independendent of eth2 types * rename `bytes_reader` to `codec` to highlight that it contains coding and decoding of bytes and native ssz types * remove tricky List init overload that causes compile issues * get rid of top-level ssz import * reenable merkleization tests * move some "standard" json serializers to spec * remove `ValidatorIndex` serialization for now * remove test_ssz_merkleization * add tests for over/underlong byte sequences * fix broken seq[byte] test - seq[byte] is not an SSZ type There are a few things this PR doesn't solve: * like #2646 this PR is weak on how to handle root and other dontSerialize fields that "sometimes" should be computed - the same problem appears in REST / JSON-RPC etc * Fix a build problem on macOS * Another way to fix the macOS builds Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 18:57:58 +00:00
+ Underlong SSZ.decode: array[0..31, byte] OK
+ Underlong SSZ.decode: bool OK
+ Underlong SSZ.decode: limb_t OK
+ Underlong SSZ.decode: uint16 OK
+ Underlong SSZ.decode: uint32 OK
+ Underlong SSZ.decode: uint8 OK
+ Underlong readSszBytes: BitArray[32] OK
+ Underlong readSszBytes: BitList[32] OK
+ Underlong readSszBytes: HashArray[32, system.uint8] OK
+ Underlong readSszBytes: HashList[system.uint64, 32] OK
+ Underlong readSszBytes: List[system.uint64, 32] OK
+ Underlong readSszBytes: Simple OK
enable uint128 / uint256 SSZ serialization tests (#2960) So far, SSZ tests for `uint128` and `uint256` integers were disabled due to "compile-time issues". The code has been adjusted to support those additional test vectors, and the corresponding tests have been enabled.
2021-10-06 06:14:52 +00:00
+ Underlong readSszBytes: UInt128 OK
+ Underlong readSszBytes: UInt256 OK
disentangle eth2 types from the ssz library (#2785) * reorganize ssz dependencies This PR continues the work in https://github.com/status-im/nimbus-eth2/pull/2646, https://github.com/status-im/nimbus-eth2/pull/2779 as well as past issues with serialization and type, to disentangle SSZ from eth2 and at the same time simplify imports and exports with a structured approach. The principal idea here is that when a library wants to introduce SSZ support, they do so via 3 files: * `ssz_codecs` which imports and reexports `codecs` - this covers the basic byte conversions and ensures no overloads get lost * `xxx_merkleization` imports and exports `merkleization` to specialize and get access to `hash_tree_root` and friends * `xxx_ssz_serialization` imports and exports `ssz_serialization` to specialize ssz for a specific library Those that need to interact with SSZ always import the `xxx_` versions of the modules and never `ssz` itself so as to keep imports simple and safe. This is similar to how the REST / JSON-RPC serializers are structured in that someone wanting to serialize spec types to REST-JSON will import `eth2_rest_serialization` and nothing else. * split up ssz into a core library that is independendent of eth2 types * rename `bytes_reader` to `codec` to highlight that it contains coding and decoding of bytes and native ssz types * remove tricky List init overload that causes compile issues * get rid of top-level ssz import * reenable merkleization tests * move some "standard" json serializers to spec * remove `ValidatorIndex` serialization for now * remove test_ssz_merkleization * add tests for over/underlong byte sequences * fix broken seq[byte] test - seq[byte] is not an SSZ type There are a few things this PR doesn't solve: * like #2646 this PR is weak on how to handle root and other dontSerialize fields that "sometimes" should be computed - the same problem appears in REST / JSON-RPC etc * Fix a build problem on macOS * Another way to fix the macOS builds Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 18:57:58 +00:00
+ Underlong readSszBytes: array[0..31, byte] OK
+ Underlong readSszBytes: bool OK
+ Underlong readSszBytes: limb_t OK
+ Underlong readSszBytes: uint16 OK
+ Underlong readSszBytes: uint32 OK
+ Underlong readSszBytes: uint8 OK
```
enable uint128 / uint256 SSZ serialization tests (#2960) So far, SSZ tests for `uint128` and `uint256` integers were disabled due to "compile-time issues". The code has been adjusted to support those additional test vectors, and the corresponding tests have been enabled.
2021-10-06 06:14:52 +00:00
OK: 42/56 Fail: 0/56 Skip: 14/56
Add test reports
2020-03-10 04:24:33 +00:00
---TOTAL---
add Altair and Merge rewards tests (#2995)
2021-10-18 14:54:17 +00:00
OK: 258/274 Fail: 0/274 Skip: 16/274