status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/AllTests-mainnet.md

296 lines
18 KiB
Markdown
Raw Normal View History

Add test reports
2020-03-10 04:24:33 +00:00
AllTests-mainnet
===
re-enable mainnet attestation pool tests (#946) * re-enable mainnet attestation pool tests; since https://github.com/status-im/nim-beacon-chain/pull/930 double CI state sim validators * bump deposits spec ref to v0.11.1
2020-04-29 11:44:07 +00:00
## Attestation pool processing [Preset: mainnet]
```diff
+ Attestations may arrive in any order [Preset: mainnet] OK
+ Attestations may overlap, bigger first [Preset: mainnet] OK
+ Attestations may overlap, smaller first [Preset: mainnet] OK
+ Attestations should be combined [Preset: mainnet] OK
Revamp attestation pool This is a revamp of the attestation pool that cleans up several aspects of attestation processing as the network grows larger and block space becomes more precious. The aim is to better exploit the divide between attestation subnets and aggregations by keeping the two kinds separate until it's time to either produce a block or aggregate. This means we're no longer eagerly combining single-vote attestations, but rather wait until the last moment, and then try to add singles to all aggregates, including those coming from the network. Importantly, the branch improves on poor aggregate quality and poor attestation packing in cases where block space is running out. A basic greed scoring mechanism is used to select attestations for blocks - attestations are added based on how much many new votes they bring to the table. * Collect single-vote attestations separately and store these until it's time to make aggregates * Create aggregates based on single-vote attestations * Select _best_ aggregate rather than _first_ aggregate when on aggregation duty * Top up all aggregates with singles when it's time make the attestation cut, thus improving the chances of grabbing the best aggregates out there * Improve aggregation test coverage * Improve bitseq operations * Simplify aggregate signature creation * Make attestation cache temporary instead of storing it in attestation pool - most of the time, blocks are not being produced, no need to keep the data around * Remove redundant aggregate storage that was used only for RPC * Use tables to avoid some linear seeks when looking up attestation data * Fix long cleanup on large slot jumps * Avoid some pointers * Speed up iterating all attestations for a slot (fixes #2490)
2021-04-12 20:25:09 +00:00
+ Can add and retrieve simple attestations [Preset: mainnet] OK
+ Everyone voting for something different [Preset: mainnet] OK
re-enable mainnet attestation pool tests (#946) * re-enable mainnet attestation pool tests; since https://github.com/status-im/nim-beacon-chain/pull/930 double CI state sim validators * bump deposits spec ref to v0.11.1
2020-04-29 11:44:07 +00:00
+ Fork choice returns block with attestation OK
+ Fork choice returns latest block with no attestations OK
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
+ Trying to add a block twice tags the second as an error OK
+ Trying to add a duplicate block from an old pruned epoch is tagged as an error OK
Revamp attestation pool This is a revamp of the attestation pool that cleans up several aspects of attestation processing as the network grows larger and block space becomes more precious. The aim is to better exploit the divide between attestation subnets and aggregations by keeping the two kinds separate until it's time to either produce a block or aggregate. This means we're no longer eagerly combining single-vote attestations, but rather wait until the last moment, and then try to add singles to all aggregates, including those coming from the network. Importantly, the branch improves on poor aggregate quality and poor attestation packing in cases where block space is running out. A basic greed scoring mechanism is used to select attestations for blocks - attestations are added based on how much many new votes they bring to the table. * Collect single-vote attestations separately and store these until it's time to make aggregates * Create aggregates based on single-vote attestations * Select _best_ aggregate rather than _first_ aggregate when on aggregation duty * Top up all aggregates with singles when it's time make the attestation cut, thus improving the chances of grabbing the best aggregates out there * Improve aggregation test coverage * Improve bitseq operations * Simplify aggregate signature creation * Make attestation cache temporary instead of storing it in attestation pool - most of the time, blocks are not being produced, no need to keep the data around * Remove redundant aggregate storage that was used only for RPC * Use tables to avoid some linear seeks when looking up attestation data * Fix long cleanup on large slot jumps * Avoid some pointers * Speed up iterating all attestations for a slot (fixes #2490)
2021-04-12 20:25:09 +00:00
+ Working with aggregates [Preset: mainnet] OK
re-enable mainnet attestation pool tests (#946) * re-enable mainnet attestation pool tests; since https://github.com/status-im/nim-beacon-chain/pull/930 double CI state sim validators * bump deposits spec ref to v0.11.1
2020-04-29 11:44:07 +00:00
```
Revamp attestation pool This is a revamp of the attestation pool that cleans up several aspects of attestation processing as the network grows larger and block space becomes more precious. The aim is to better exploit the divide between attestation subnets and aggregations by keeping the two kinds separate until it's time to either produce a block or aggregate. This means we're no longer eagerly combining single-vote attestations, but rather wait until the last moment, and then try to add singles to all aggregates, including those coming from the network. Importantly, the branch improves on poor aggregate quality and poor attestation packing in cases where block space is running out. A basic greed scoring mechanism is used to select attestations for blocks - attestations are added based on how much many new votes they bring to the table. * Collect single-vote attestations separately and store these until it's time to make aggregates * Create aggregates based on single-vote attestations * Select _best_ aggregate rather than _first_ aggregate when on aggregation duty * Top up all aggregates with singles when it's time make the attestation cut, thus improving the chances of grabbing the best aggregates out there * Improve aggregation test coverage * Improve bitseq operations * Simplify aggregate signature creation * Make attestation cache temporary instead of storing it in attestation pool - most of the time, blocks are not being produced, no need to keep the data around * Remove redundant aggregate storage that was used only for RPC * Use tables to avoid some linear seeks when looking up attestation data * Fix long cleanup on large slot jumps * Avoid some pointers * Speed up iterating all attestations for a slot (fixes #2490)
2021-04-12 20:25:09 +00:00
OK: 11/11 Fail: 0/11 Skip: 0/11
implement clock disparity for attestation validation (#1568) This implements disparity, resolving a part of https://github.com/status-im/nim-beacon-chain/issues/1367 * make BeaconTime a duration for fractional seconds * factor out attestation/aggregate validation * simplify recording of queued attestations * simplify attestation signature check * fix blocks_received metric * add some trivial validation tests * remove unresolved attestation table - attestations for unknown blocks are dropped instead (cannot verify their signature)
2020-08-27 07:34:12 +00:00
## Attestation validation [Preset: mainnet]
```diff
+ Validation sanity OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Add test reports
2020-03-10 04:24:33 +00:00
## Beacon chain DB [Preset: mainnet]
```diff
+ empty database [Preset: mainnet] OK
+ find ancestors [Preset: mainnet] OK
+ sanity check blocks [Preset: mainnet] OK
immutable validator database factoring (#2297) * initial immutable validator database factoring * remove changes from chain_dag: this abstraction properly belongs in beacon_chain_db * add merging mutable/immutable validator portions; individually test database roundtripping of immutable validators and states-sans-immutable-validators * update test summaries * use stew/assign2 instead of Nim assignment * add reading/writing of immutable validators in chaindag * remove unused import * replace chunked k/v store of immutable validators with per-row SQL table storage * use List instead of HashList * un-stub some ncli_db code so that it uses * switch HashArray to array; move BeaconStateNoImmutableValidators from datatypes to beacon_chain_db * begin only-mutable-part state storage * uncomment some assigns * work around https://github.com/nim-lang/Nim/issues/17253 * fix most of the issues/oversights; local sim runs again * fix test suite by adding missing beaconstate field to copy function * have ncli bench also store immutable validators * extract some immutable-validator-specific code from the beacon chain db module * add more rigorous database state roundtripping, with changing validator sets * adjust ncli_db to use new schema * simplify putState/getState by moving all immutable validator accounting into beacon state DB * remove redundant test case and move code to immutable-beacon-chain module * more efficient, but still brute-force, mutable+immutable validator merging * reuse BeaconState in getState * ensure HashList/HashArray caches are cleared when reusing getState buffers; add ncli_db and a unit test to verify this * HashList.clear() -> HashList.clearCache() * only copy incrementally necessary immutable validators * increase strictness of test cases and fix/work around resulting HashList cache invalidation issues * remove explanatory scaffolding * allow for storage of full (with all validators) states for backwards/forwards-compatibility * adjust DbSeq type usage * store full, with-validators, state every 64 epochs to enable reverting versions * reduce memory allocation and intermediate objects in state storage codepath * eliminate allocation/copying through intermediate BeaconStateNoImmutableValidators objects * skip benchmarking initial genesis-validator-heavy state store * always store new-style state and sometimes old-style state * document intent behind BeaconState/Validator type-punnery * more accurate failure message on SQLite in-memory database initialization failure
2021-03-15 14:11:51 +00:00
+ sanity check full states [Preset: mainnet] OK
+ sanity check full states, reusing buffers [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ sanity check genesis roundtrip [Preset: mainnet] OK
initial infrastructure for state diffs (#2087) Initial infrastructure for state diffs
2021-01-18 20:34:41 +00:00
+ sanity check state diff roundtrip [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ sanity check states [Preset: mainnet] OK
immutable validator database factoring (#2297) * initial immutable validator database factoring * remove changes from chain_dag: this abstraction properly belongs in beacon_chain_db * add merging mutable/immutable validator portions; individually test database roundtripping of immutable validators and states-sans-immutable-validators * update test summaries * use stew/assign2 instead of Nim assignment * add reading/writing of immutable validators in chaindag * remove unused import * replace chunked k/v store of immutable validators with per-row SQL table storage * use List instead of HashList * un-stub some ncli_db code so that it uses * switch HashArray to array; move BeaconStateNoImmutableValidators from datatypes to beacon_chain_db * begin only-mutable-part state storage * uncomment some assigns * work around https://github.com/nim-lang/Nim/issues/17253 * fix most of the issues/oversights; local sim runs again * fix test suite by adding missing beaconstate field to copy function * have ncli bench also store immutable validators * extract some immutable-validator-specific code from the beacon chain db module * add more rigorous database state roundtripping, with changing validator sets * adjust ncli_db to use new schema * simplify putState/getState by moving all immutable validator accounting into beacon state DB * remove redundant test case and move code to immutable-beacon-chain module * more efficient, but still brute-force, mutable+immutable validator merging * reuse BeaconState in getState * ensure HashList/HashArray caches are cleared when reusing getState buffers; add ncli_db and a unit test to verify this * HashList.clear() -> HashList.clearCache() * only copy incrementally necessary immutable validators * increase strictness of test cases and fix/work around resulting HashList cache invalidation issues * remove explanatory scaffolding * allow for storage of full (with all validators) states for backwards/forwards-compatibility * adjust DbSeq type usage * store full, with-validators, state every 64 epochs to enable reverting versions * reduce memory allocation and intermediate objects in state storage codepath * eliminate allocation/copying through intermediate BeaconStateNoImmutableValidators objects * skip benchmarking initial genesis-validator-heavy state store * always store new-style state and sometimes old-style state * document intent behind BeaconState/Validator type-punnery * more accurate failure message on SQLite in-memory database initialization failure
2021-03-15 14:11:51 +00:00
+ sanity check states, reusing buffers [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
```
immutable validator database factoring (#2297) * initial immutable validator database factoring * remove changes from chain_dag: this abstraction properly belongs in beacon_chain_db * add merging mutable/immutable validator portions; individually test database roundtripping of immutable validators and states-sans-immutable-validators * update test summaries * use stew/assign2 instead of Nim assignment * add reading/writing of immutable validators in chaindag * remove unused import * replace chunked k/v store of immutable validators with per-row SQL table storage * use List instead of HashList * un-stub some ncli_db code so that it uses * switch HashArray to array; move BeaconStateNoImmutableValidators from datatypes to beacon_chain_db * begin only-mutable-part state storage * uncomment some assigns * work around https://github.com/nim-lang/Nim/issues/17253 * fix most of the issues/oversights; local sim runs again * fix test suite by adding missing beaconstate field to copy function * have ncli bench also store immutable validators * extract some immutable-validator-specific code from the beacon chain db module * add more rigorous database state roundtripping, with changing validator sets * adjust ncli_db to use new schema * simplify putState/getState by moving all immutable validator accounting into beacon state DB * remove redundant test case and move code to immutable-beacon-chain module * more efficient, but still brute-force, mutable+immutable validator merging * reuse BeaconState in getState * ensure HashList/HashArray caches are cleared when reusing getState buffers; add ncli_db and a unit test to verify this * HashList.clear() -> HashList.clearCache() * only copy incrementally necessary immutable validators * increase strictness of test cases and fix/work around resulting HashList cache invalidation issues * remove explanatory scaffolding * allow for storage of full (with all validators) states for backwards/forwards-compatibility * adjust DbSeq type usage * store full, with-validators, state every 64 epochs to enable reverting versions * reduce memory allocation and intermediate objects in state storage codepath * eliminate allocation/copying through intermediate BeaconStateNoImmutableValidators objects * skip benchmarking initial genesis-validator-heavy state store * always store new-style state and sometimes old-style state * document intent behind BeaconState/Validator type-punnery * more accurate failure message on SQLite in-memory database initialization failure
2021-03-15 14:11:51 +00:00
OK: 9/9 Fail: 0/9 Skip: 0/9
Add test reports
2020-03-10 04:24:33 +00:00
## Beacon node
```diff
+ Compile OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## Beacon state [Preset: mainnet]
```diff
+ Smoke test initialize_beacon_state_from_eth1 [Preset: mainnet] OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
## Block pool processing [Preset: mainnet]
```diff
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
+ Adding the same block twice returns a Duplicate error [Preset: mainnet] OK
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
+ Reverse order block add & get [Preset: mainnet] OK
+ Simple block add&get [Preset: mainnet] OK
+ getRef returns nil for missing blocks OK
improve slot processing speeds (#1670) about 40% better slot processing times (with LTO enabled) - these don't do BLS but are used heavily during replay (state transition = slot + block transition) tests using a recent medalla state and advancing it 1000 slots: ``` ./ncli slots --preState2:state-302271-3c1dbf19-c1f944bf.ssz --slot:1000 --postState2:xx.ssz ``` pre: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 39.236, 0.000, 39.236, 39.236, 1, Load state from file 0.049, 0.002, 0.046, 0.063, 968, Apply slot 256.504, 81.008, 213.471, 591.902, 32, Apply epoch slot 28.597, 0.000, 28.597, 28.597, 1, Save state to file ``` cast: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 37.079, 0.000, 37.079, 37.079, 1, Load state from file 0.042, 0.002, 0.040, 0.090, 968, Apply slot 215.552, 68.763, 180.155, 500.103, 32, Apply epoch slot 25.106, 0.000, 25.106, 25.106, 1, Save state to file ``` cast+rewards: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 40.049, 0.000, 40.049, 40.049, 1, Load state from file 0.048, 0.001, 0.045, 0.060, 968, Apply slot 164.981, 76.273, 142.099, 477.868, 32, Apply epoch slot 28.498, 0.000, 28.498, 28.498, 1, Save state to file ``` cast+rewards+shr ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 12.898, 0.000, 12.898, 12.898, 1, Load state from file 0.039, 0.002, 0.038, 0.054, 968, Apply slot 139.971, 68.797, 120.088, 428.844, 32, Apply epoch slot 24.761, 0.000, 24.761, 24.761, 1, Save state to file ```
2020-09-16 20:59:33 +00:00
+ loading tail block works [Preset: mainnet] OK
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
+ updateHead updates head and headState [Preset: mainnet] OK
+ updateStateData sanity [Preset: mainnet] OK
```
OK: 7/7 Fail: 0/7 Skip: 0/7
Add test reports
2020-03-10 04:24:33 +00:00
## Block processing [Preset: mainnet]
```diff
+ Attestation gets processed at epoch [Preset: mainnet] OK
+ Passes from genesis state, empty block [Preset: mainnet] OK
+ Passes from genesis state, no block [Preset: mainnet] OK
+ Passes through epoch update, empty block [Preset: mainnet] OK
+ Passes through epoch update, no block [Preset: mainnet] OK
```
OK: 5/5 Fail: 0/5 Skip: 0/5
## BlockRef and helpers [Preset: mainnet]
```diff
rework epoch cache referencing * collect all epochrefs in specific blocks to make them easier to find and to avoid lots of small seqs * reuse validator key databases more aggressively by comparing keys * make state cache available from within `withState` * make epochRef available from within onBlockAdded callback * integrate getEpochInfo into block resolution and epoch ref logic such that epochrefs are created when blocks are added to pool or lazily when needed by a getEpochRef * fill state cache better from EpochRef, speeding up replay and validation * store epochRef in specific blocks to make them easier to find and reuse * fix database corruption when state is saved while replaying quarantine * replay slots fully from block pool before processing state * compare bls values more smartly * store epoch state without block applied in database - it's recommended to resync the node! this branch will drastically speed up processing in times of long non-finality, as well as cut memory usage by 10x during the recent medalla madness.
2020-08-18 20:29:33 +00:00
+ epochAncestor sanity [Preset: mainnet] OK
update attestation and block validation to v0.12.2; clean up getAncestorAt()/get_ancestor() (#1417) * update attestation validation to v0.12.2; clean up getAncestorAt()/get_ancestor() * update beacon block validation to v0.12.2
2020-08-03 19:47:42 +00:00
+ get_ancestor sanity [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ isAncestorOf sanity [Preset: mainnet] OK
```
rework epoch cache referencing * collect all epochrefs in specific blocks to make them easier to find and to avoid lots of small seqs * reuse validator key databases more aggressively by comparing keys * make state cache available from within `withState` * make epochRef available from within onBlockAdded callback * integrate getEpochInfo into block resolution and epoch ref logic such that epochrefs are created when blocks are added to pool or lazily when needed by a getEpochRef * fill state cache better from EpochRef, speeding up replay and validation * store epochRef in specific blocks to make them easier to find and reuse * fix database corruption when state is saved while replaying quarantine * replay slots fully from block pool before processing state * compare bls values more smartly * store epoch state without block applied in database - it's recommended to resync the node! this branch will drastically speed up processing in times of long non-finality, as well as cut memory usage by 10x during the recent medalla madness.
2020-08-18 20:29:33 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
Add test reports
2020-03-10 04:24:33 +00:00
## BlockSlot and helpers [Preset: mainnet]
```diff
+ atSlot sanity [Preset: mainnet] OK
+ parent sanity [Preset: mainnet] OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
update some v1.1.0 alpha1 to alpha2 (#2457) * update some v1.1.0 alpha1 to alpha2 * remove unused getDepositMessage overload and move other out of datatypes/base * bump nim-eth2-scenarios to download v1.1.0-alpha.2 test vectors * construct object rather than result
2021-03-29 19:17:48 +00:00
## Eth2 specific discovery tests
```diff
+ Invalid attnets field OK
+ Subnet query OK
+ Subnet query after ENR update OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
refactor exit pool and gossipsub-subscribe to exit pool message types (#1733) * refactor exit pool and gossipsub-subscribe to all exit pool message types * remove unused loop counter
2020-09-24 17:05:49 +00:00
## Exit pool testing suite
```diff
+ addExitMessage/getAttesterSlashingMessage OK
+ addExitMessage/getProposerSlashingMessage OK
+ addExitMessage/getVoluntaryExitMessage OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
[Ready 1/2] Fork choice rewrite (#865) * initial fork-choice refactor * Add fork_choice test for "no votes" * Initial test with voting: fix handling of unknown validators and parent blocks * Fix tiebreak of votes * Cleanup debugging traces * Complexify the vote test * fakeHash use the bigEndian repr of number + fix tiebreak for good * Stash changes: found critical bug in nimcrypto `==` and var openarray * Passing fork choice tests with varying votes * Add FFG fork choice scenario + fork choice to the test suite * Not sure why lmdb / rocksdb reappeared in rebase * Add sanity checks to .nimble file + integrate fork choice tests to the test DB and test timing * Cleanup debugging echos * nimcrypto fix https://github.com/status-im/nim-beacon-chain/pull/864 as been merged, remove TODO comment * Turn fork choice exception-free * Cleanup "result" to ensure early return is properly used * Add a comment on private/public error code vs Result * result -> results following https://github.com/status-im/nim-beacon-chain/pull/866 * Address comments: - raises: [Defect] doesn't work -> TODO - process_attestation cannot fail - try/except as expression pending Nim v1.2.0 - cleanup TODOs * re-enable all sanity checks * tag no raise for process_attestation * use raises defect everywhere in fork choice and fix process_attestation test
2020-04-09 16:15:00 +00:00
## Fork Choice + Finality [Preset: mainnet]
```diff
+ fork_choice - testing finality #01 OK
+ fork_choice - testing finality #02 OK
+ fork_choice - testing no votes OK
+ fork_choice - testing with votes OK
```
OK: 4/4 Fail: 0/4 Skip: 0/4
Add test reports
2020-03-10 04:24:33 +00:00
## Honest validator
```diff
detect already-aggregate-voted condition before attestation pool; add is_aggregator tests (#2170) * detect already-aggregate-voted condition before attestation pool; add is_aggregator tests * replace pair of attestation-per-epoch tracking lists with single list and remove Option use * fix attestation condition * use safer type conversions; add more is_aggregator tests
2020-12-14 20:58:32 +00:00
+ General pubsub topics OK
remove some unused imports, add tests for pubsub topics, and subscribe to interop attestations
2020-05-14 11:19:10 +00:00
+ Mainnet attestation topics OK
detect already-aggregate-voted condition before attestation pool; add is_aggregator tests (#2170) * detect already-aggregate-voted condition before attestation pool; add is_aggregator tests * replace pair of attestation-per-epoch tracking lists with single list and remove Option use * fix attestation condition * use safer type conversions; add more is_aggregator tests
2020-12-14 20:58:32 +00:00
+ is_aggregator OK
Add test reports
2020-03-10 04:24:33 +00:00
```
detect already-aggregate-voted condition before attestation pool; add is_aggregator tests (#2170) * detect already-aggregate-voted condition before attestation pool; add is_aggregator tests * replace pair of attestation-per-epoch tracking lists with single list and remove Option use * fix attestation condition * use safer type conversions; add more is_aggregator tests
2020-12-14 20:58:32 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
re-add test_interop to all_tests and mark several v0.10.1 phase 0 spec references as v0.11.1
2020-04-06 18:55:47 +00:00
## Interop
```diff
+ Interop genesis OK
+ Interop signatures OK
+ Mocked start private key OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
allow compile-time switching between v0.11.3 and v0.12.1
2020-06-09 16:48:51 +00:00
## Official - constants & config [Preset: mainnet]
Add test reports
2020-03-10 04:24:33 +00:00
```diff
+ BASE_REWARD_FACTOR 64 [Preset: mainnet] OK
+ BLS_WITHDRAWAL_PREFIX "0x00" [Preset: mainnet] OK
+ CHURN_LIMIT_QUOTIENT 65536 [Preset: mainnet] OK
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
CONFIG_NAME "mainnet" [Preset: mainnet] Skip
switch mapIt/find to in-place linear search (#1436)
2020-08-04 17:15:27 +00:00
DEPOSIT_CHAIN_ID 1 [Preset: mainnet] Skip
remove v0.12.3 EF test vectors to save 2.7GB of space in CI instances (#2017)
2020-11-13 21:16:56 +00:00
DEPOSIT_CONTRACT_ADDRESS "0x00000000219ab540356cBB839Cbe05303d770 Skip
switch mapIt/find to in-place linear search (#1436)
2020-08-04 17:15:27 +00:00
DEPOSIT_NETWORK_ID 1 [Preset: mainnet] Skip
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ DOMAIN_AGGREGATE_AND_PROOF "0x06000000" [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ DOMAIN_BEACON_ATTESTER "0x01000000" [Preset: mainnet] OK
+ DOMAIN_BEACON_PROPOSER "0x00000000" [Preset: mainnet] OK
+ DOMAIN_DEPOSIT "0x03000000" [Preset: mainnet] OK
+ DOMAIN_RANDAO "0x02000000" [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ DOMAIN_SELECTION_PROOF "0x05000000" [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ DOMAIN_VOLUNTARY_EXIT "0x04000000" [Preset: mainnet] OK
+ EFFECTIVE_BALANCE_INCREMENT 1000000000 [Preset: mainnet] OK
+ EJECTION_BALANCE 16000000000 [Preset: mainnet] OK
remove v0.12.3 EF test vectors to save 2.7GB of space in CI instances (#2017)
2020-11-13 21:16:56 +00:00
+ EPOCHS_PER_ETH1_VOTING_PERIOD 64 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ EPOCHS_PER_HISTORICAL_VECTOR 65536 [Preset: mainnet] OK
+ EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION 256 [Preset: mainnet] OK
+ EPOCHS_PER_SLASHINGS_VECTOR 8192 [Preset: mainnet] OK
remove v0.12.3 EF test vectors to save 2.7GB of space in CI instances (#2017)
2020-11-13 21:16:56 +00:00
ETH1_FOLLOW_DISTANCE 2048 [Preset: mainnet] Skip
GENESIS_DELAY 604800 [Preset: mainnet] Skip
skip skipped constant sanity checks also from markdown reports' perspectives (#1053) * update more spec refs in beacon_chain/spec/presets; skip skipped constant sanity checks also from markdown reports' perspectives * mark skipped as skipped in markdown
2020-05-21 17:56:09 +00:00
GENESIS_FORK_VERSION "0x00000000" [Preset: mainnet] Skip
Add test reports
2020-03-10 04:24:33 +00:00
+ HISTORICAL_ROOTS_LIMIT 16777216 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ HYSTERESIS_DOWNWARD_MULTIPLIER 1 [Preset: mainnet] OK
+ HYSTERESIS_QUOTIENT 4 [Preset: mainnet] OK
+ HYSTERESIS_UPWARD_MULTIPLIER 5 [Preset: mainnet] OK
remove v0.12.3 EF test vectors to save 2.7GB of space in CI instances (#2017)
2020-11-13 21:16:56 +00:00
+ INACTIVITY_PENALTY_QUOTIENT 67108864 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MAX_ATTESTATIONS 128 [Preset: mainnet] OK
initial 0.12.1 beacon chain spec
2020-06-08 18:41:50 +00:00
+ MAX_ATTESTER_SLASHINGS 2 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MAX_COMMITTEES_PER_SLOT 64 [Preset: mainnet] OK
+ MAX_DEPOSITS 16 [Preset: mainnet] OK
+ MAX_EFFECTIVE_BALANCE 32000000000 [Preset: mainnet] OK
+ MAX_PROPOSER_SLASHINGS 16 [Preset: mainnet] OK
+ MAX_SEED_LOOKAHEAD 4 [Preset: mainnet] OK
+ MAX_VALIDATORS_PER_COMMITTEE 2048 [Preset: mainnet] OK
+ MAX_VOLUNTARY_EXITS 16 [Preset: mainnet] OK
+ MIN_ATTESTATION_INCLUSION_DELAY 1 [Preset: mainnet] OK
+ MIN_DEPOSIT_AMOUNT 1000000000 [Preset: mainnet] OK
+ MIN_EPOCHS_TO_INACTIVITY_PENALTY 4 [Preset: mainnet] OK
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
MIN_GENESIS_ACTIVE_VALIDATOR_COUNT 16384 [Preset: mainnet] Skip
remove v0.12.3 EF test vectors to save 2.7GB of space in CI instances (#2017)
2020-11-13 21:16:56 +00:00
MIN_GENESIS_TIME 1606824000 [Preset: mainnet] Skip
Add test reports
2020-03-10 04:24:33 +00:00
+ MIN_PER_EPOCH_CHURN_LIMIT 4 [Preset: mainnet] OK
+ MIN_SEED_LOOKAHEAD 1 [Preset: mainnet] OK
remove v0.12.3 EF test vectors to save 2.7GB of space in CI instances (#2017)
2020-11-13 21:16:56 +00:00
+ MIN_SLASHING_PENALTY_QUOTIENT 128 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MIN_VALIDATOR_WITHDRAWABILITY_DELAY 256 [Preset: mainnet] OK
remove v0.12.3 EF test vectors to save 2.7GB of space in CI instances (#2017)
2020-11-13 21:16:56 +00:00
+ PROPORTIONAL_SLASHING_MULTIPLIER 1 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ PROPOSER_REWARD_QUOTIENT 8 [Preset: mainnet] OK
+ RANDOM_SUBNETS_PER_VALIDATOR 1 [Preset: mainnet] OK
+ SAFE_SLOTS_TO_UPDATE_JUSTIFIED 8 [Preset: mainnet] OK
+ SECONDS_PER_ETH1_BLOCK 14 [Preset: mainnet] OK
+ SECONDS_PER_SLOT 12 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ SHARD_COMMITTEE_PERIOD 256 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ SHUFFLE_ROUND_COUNT 90 [Preset: mainnet] OK
+ SLOTS_PER_EPOCH 32 [Preset: mainnet] OK
+ SLOTS_PER_HISTORICAL_ROOT 8192 [Preset: mainnet] OK
+ TARGET_AGGREGATORS_PER_COMMITTEE 16 [Preset: mainnet] OK
+ TARGET_COMMITTEE_SIZE 128 [Preset: mainnet] OK
+ VALIDATOR_REGISTRY_LIMIT 1099511627776 [Preset: mainnet] OK
+ WHISTLEBLOWER_REWARD_QUOTIENT 512 [Preset: mainnet] OK
```
update v1.0.0-rc.0 preset spec references (#1891) * update ve1.0.0-rc.0 preset spec references * remove runtime preset ETH1_FOLLOW_DISTANCE from preset files; remove two CI build items to try to keep Travis from timing out
2020-10-19 13:53:23 +00:00
OK: 51/60 Fail: 0/60 Skip: 9/60
Add test reports
2020-03-10 04:24:33 +00:00
## PeerPool testing suite
```diff
+ Access peers by key test OK
+ Acquire from empty pool OK
+ Acquire/Sorting and consistency test OK
improve slot processing speeds (#1670) about 40% better slot processing times (with LTO enabled) - these don't do BLS but are used heavily during replay (state transition = slot + block transition) tests using a recent medalla state and advancing it 1000 slots: ``` ./ncli slots --preState2:state-302271-3c1dbf19-c1f944bf.ssz --slot:1000 --postState2:xx.ssz ``` pre: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 39.236, 0.000, 39.236, 39.236, 1, Load state from file 0.049, 0.002, 0.046, 0.063, 968, Apply slot 256.504, 81.008, 213.471, 591.902, 32, Apply epoch slot 28.597, 0.000, 28.597, 28.597, 1, Save state to file ``` cast: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 37.079, 0.000, 37.079, 37.079, 1, Load state from file 0.042, 0.002, 0.040, 0.090, 968, Apply slot 215.552, 68.763, 180.155, 500.103, 32, Apply epoch slot 25.106, 0.000, 25.106, 25.106, 1, Save state to file ``` cast+rewards: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 40.049, 0.000, 40.049, 40.049, 1, Load state from file 0.048, 0.001, 0.045, 0.060, 968, Apply slot 164.981, 76.273, 142.099, 477.868, 32, Apply epoch slot 28.498, 0.000, 28.498, 28.498, 1, Save state to file ``` cast+rewards+shr ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 12.898, 0.000, 12.898, 12.898, 1, Load state from file 0.039, 0.002, 0.038, 0.054, 968, Apply slot 139.971, 68.797, 120.088, 428.844, 32, Apply epoch slot 24.761, 0.000, 24.761, 24.761, 1, Save state to file ```
2020-09-16 20:59:33 +00:00
+ Delete peer on release text OK
Add test reports
2020-03-10 04:24:33 +00:00
+ Iterators test OK
+ Peer lifetime test OK
+ Safe/Clear test OK
detect improper 0-length lists of variable-sized objects in SSZ reading (#928) * detect improper 0-length lists of variable-sized objects in SSZ reading
2020-04-23 19:39:23 +00:00
+ Score check test OK
refactor exit pool and gossipsub-subscribe to exit pool message types (#1733) * refactor exit pool and gossipsub-subscribe to all exit pool message types * remove unused loop counter
2020-09-24 17:05:49 +00:00
+ Space tests OK
Add test reports
2020-03-10 04:24:33 +00:00
+ addPeer() test OK
+ addPeerNoWait() test OK
+ deletePeer() test OK
```
refactor exit pool and gossipsub-subscribe to exit pool message types (#1733) * refactor exit pool and gossipsub-subscribe to all exit pool message types * remove unused loop counter
2020-09-24 17:05:49 +00:00
OK: 12/12 Fail: 0/12 Skip: 0/12
Add test reports
2020-03-10 04:24:33 +00:00
## SSZ dynamic navigator
```diff
+ navigating fields OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## SSZ navigator
```diff
support all basic types
2020-06-02 09:44:51 +00:00
+ basictype OK
Add test reports
2020-03-10 04:24:33 +00:00
+ lists with max size OK
+ simple object fields OK
```
support all basic types
2020-06-02 09:44:51 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
## Slashing Protection DB - Interchange [Preset: mainnet]
```diff
Slashing protection refactor - EIP 3076 (#2094) * Create CLI tool for slashing export * Use SQLite as a DB instead of a KV-store * Keeps v1 and v2 DBs around * Uses the same schema as Lighthouse v1.1.0 * Passes all interchange tests + skeleton of finalization pruning * Removes tests that would violate v5 / minimal slashing DB and MinSlot rules * Migration tool added using low-watermark scheme for faster migration of large number of validators
2021-02-09 15:23:06 +00:00
+ Smoke test - Complete format - Invalid database is refused [Preset: mainnet] OK
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
+ Smoke test - Complete format [Preset: mainnet] OK
```
Slashing protection refactor - EIP 3076 (#2094) * Create CLI tool for slashing export * Use SQLite as a DB instead of a KV-store * Keeps v1 and v2 DBs around * Uses the same schema as Lighthouse v1.1.0 * Passes all interchange tests + skeleton of finalization pruning * Removes tests that would violate v5 / minimal slashing DB and MinSlot rules * Migration tool added using low-watermark scheme for faster migration of large number of validators
2021-02-09 15:23:06 +00:00
OK: 2/2 Fail: 0/2 Skip: 0/2
## Slashing Protection DB - v1 and v2 migration [Preset: mainnet]
```diff
+ Minimal format migration [Preset: mainnet] OK
```
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
OK: 1/1 Fail: 0/1 Skip: 0/1
## Slashing Protection DB [Preset: mainnet]
```diff
update AllTests-mainnet.md
2020-09-26 21:58:12 +00:00
+ Attestation ordering #1698 OK
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
+ Empty database [Preset: mainnet] OK
+ SP for block proposal - backtracking append OK
+ SP for block proposal - linear append OK
+ SP for same epoch attestation target - linear append OK
+ SP for surrounded attestations OK
+ SP for surrounding attestations OK
update AllTests-mainnet.md
2020-09-26 21:58:12 +00:00
+ Test valid attestation #1699 OK
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
```
Slashing protection refactor - EIP 3076 (#2094) * Create CLI tool for slashing export * Use SQLite as a DB instead of a KV-store * Keeps v1 and v2 DBs around * Uses the same schema as Lighthouse v1.1.0 * Passes all interchange tests + skeleton of finalization pruning * Removes tests that would violate v5 / minimal slashing DB and MinSlot rules * Migration tool added using low-watermark scheme for faster migration of large number of validators
2021-02-09 15:23:06 +00:00
OK: 8/8 Fail: 0/8 Skip: 0/8
Restore compilation with custom presets (#1309) This also assigns precise types to the constants in the minimal and mainnet presets in order to reduce the chance of compilation errors when custom presets are used (previously, only the custom presets have precisely assigned types for the constants).
2020-07-13 14:44:58 +00:00
## Spec datatypes
```diff
+ Graffiti bytes OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Add test reports
2020-03-10 04:24:33 +00:00
## Spec helpers
```diff
+ integer_squareroot OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## Sync protocol
```diff
+ Compile OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## Zero signature sanity checks
```diff
+ SSZ serialization roundtrip of SignedBeaconBlockHeader OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## [Unit - Spec - Block processing] Attestations [Preset: mainnet]
```diff
+ Valid attestation OK
+ Valid attestation from previous epoch OK
```
update test results summary after BLS 0.10.1 update
2020-03-13 14:58:21 +00:00
OK: 2/2 Fail: 0/2 Skip: 0/2
Add test reports
2020-03-10 04:24:33 +00:00
## [Unit - Spec - Block processing] Deposits [Preset: mainnet]
```diff
+ Deposit at MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Deposit over MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Deposit under MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
Merge devel and resolve the conflicts
2020-07-08 12:36:03 +00:00
+ Invalid deposit at MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
Add test reports
2020-03-10 04:24:33 +00:00
+ Validator top-up OK
```
Merge devel and resolve the conflicts
2020-07-08 12:36:03 +00:00
OK: 5/5 Fail: 0/5 Skip: 0/5
Add test reports
2020-03-10 04:24:33 +00:00
## [Unit - Spec - Epoch processing] Justification and Finalization [Preset: mainnet]
```diff
+ Rule I - 234 finalization with enough support OK
+ Rule I - 234 finalization without support OK
+ Rule II - 23 finalization with enough support OK
+ Rule II - 23 finalization without support OK
+ Rule III - 123 finalization with enough support OK
+ Rule III - 123 finalization without support OK
+ Rule IV - 12 finalization with enough support OK
+ Rule IV - 12 finalization without support OK
```
OK: 8/8 Fail: 0/8 Skip: 0/8
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
## chain DAG finalization tests [Preset: mainnet]
```diff
remove quadratic deposit Merkle tree initialization
2020-08-13 23:37:58 +00:00
+ init with gaps [Preset: mainnet] OK
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
```
OK: 1/1 Fail: 0/1 Skip: 0/1
wip: cache * cache lists and arrays of complex objects (5x block processing speed on ncli_db) trivial baseline cache that stores tree in flat memory structure
2020-05-28 16:34:59 +00:00
## hash
```diff
+ HashArray OK
Reset cached indices when resetting cache on SSZ read (#2480) * Reset cached indices when resetting cache on SSZ read When deserializing into an existing structure, the cache should be cleared - goes for json also. Also improve error messages.
2021-04-08 10:11:04 +00:00
+ HashList OK
wip: cache * cache lists and arrays of complex objects (5x block processing speed on ncli_db) trivial baseline cache that stores tree in flat memory structure
2020-05-28 16:34:59 +00:00
```
Reset cached indices when resetting cache on SSZ read (#2480) * Reset cached indices when resetting cache on SSZ read When deserializing into an existing structure, the cache should be cleared - goes for json also. Also improve error messages.
2021-04-08 10:11:04 +00:00
OK: 2/2 Fail: 0/2 Skip: 0/2
initial infrastructure for state diffs (#2087) Initial infrastructure for state diffs
2021-01-18 20:34:41 +00:00
## state diff tests [Preset: mainnet]
```diff
+ random slot differences [Preset: mainnet] OK
```
remove delta-encoding from state diff balances (#2397) * remove delta-encoding from state diff balances * switch HashList to List
2021-03-11 05:39:04 +00:00
OK: 1/1 Fail: 0/1 Skip: 0/1
Add test reports
2020-03-10 04:24:33 +00:00
---TOTAL---
Revamp attestation pool This is a revamp of the attestation pool that cleans up several aspects of attestation processing as the network grows larger and block space becomes more precious. The aim is to better exploit the divide between attestation subnets and aggregations by keeping the two kinds separate until it's time to either produce a block or aggregate. This means we're no longer eagerly combining single-vote attestations, but rather wait until the last moment, and then try to add singles to all aggregates, including those coming from the network. Importantly, the branch improves on poor aggregate quality and poor attestation packing in cases where block space is running out. A basic greed scoring mechanism is used to select attestations for blocks - attestations are added based on how much many new votes they bring to the table. * Collect single-vote attestations separately and store these until it's time to make aggregates * Create aggregates based on single-vote attestations * Select _best_ aggregate rather than _first_ aggregate when on aggregation duty * Top up all aggregates with singles when it's time make the attestation cut, thus improving the chances of grabbing the best aggregates out there * Improve aggregation test coverage * Improve bitseq operations * Simplify aggregate signature creation * Make attestation cache temporary instead of storing it in attestation pool - most of the time, blocks are not being produced, no need to keep the data around * Remove redundant aggregate storage that was used only for RPC * Use tables to avoid some linear seeks when looking up attestation data * Fix long cleanup on large slot jumps * Avoid some pointers * Speed up iterating all attestations for a slot (fixes #2490)
2021-04-12 20:25:09 +00:00
OK: 157/166 Fail: 0/166 Skip: 9/166