status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/AllTests-mainnet.md

251 lines
15 KiB
Markdown
Raw Normal View History

Add test reports
2020-03-10 04:24:33 +00:00
AllTests-mainnet
===
re-enable mainnet attestation pool tests (#946) * re-enable mainnet attestation pool tests; since https://github.com/status-im/nim-beacon-chain/pull/930 double CI state sim validators * bump deposits spec ref to v0.11.1
2020-04-29 11:44:07 +00:00
## Attestation pool processing [Preset: mainnet]
```diff
+ Attestations may arrive in any order [Preset: mainnet] OK
+ Attestations may overlap, bigger first [Preset: mainnet] OK
+ Attestations may overlap, smaller first [Preset: mainnet] OK
+ Attestations should be combined [Preset: mainnet] OK
+ Can add and retrieve simple attestation [Preset: mainnet] OK
+ Fork choice returns block with attestation OK
+ Fork choice returns latest block with no attestations OK
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
+ Trying to add a block twice tags the second as an error OK
+ Trying to add a duplicate block from an old pruned epoch is tagged as an error OK
re-enable mainnet attestation pool tests (#946) * re-enable mainnet attestation pool tests; since https://github.com/status-im/nim-beacon-chain/pull/930 double CI state sim validators * bump deposits spec ref to v0.11.1
2020-04-29 11:44:07 +00:00
```
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
OK: 9/9 Fail: 0/9 Skip: 0/9
Add test reports
2020-03-10 04:24:33 +00:00
## Beacon chain DB [Preset: mainnet]
```diff
+ empty database [Preset: mainnet] OK
+ find ancestors [Preset: mainnet] OK
+ sanity check blocks [Preset: mainnet] OK
+ sanity check genesis roundtrip [Preset: mainnet] OK
+ sanity check states [Preset: mainnet] OK
```
OK: 5/5 Fail: 0/5 Skip: 0/5
## Beacon node
```diff
+ Compile OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## Beacon state [Preset: mainnet]
```diff
+ Smoke test initialize_beacon_state_from_eth1 [Preset: mainnet] OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
## Block pool processing [Preset: mainnet]
```diff
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
+ Adding the same block twice returns a Duplicate error [Preset: mainnet] OK
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
+ Reverse order block add & get [Preset: mainnet] OK
+ Simple block add&get [Preset: mainnet] OK
+ getRef returns nil for missing blocks OK
+ loadTailState gets genesis block on first load [Preset: mainnet] OK
+ updateHead updates head and headState [Preset: mainnet] OK
+ updateStateData sanity [Preset: mainnet] OK
```
OK: 7/7 Fail: 0/7 Skip: 0/7
Add test reports
2020-03-10 04:24:33 +00:00
## Block processing [Preset: mainnet]
```diff
+ Attestation gets processed at epoch [Preset: mainnet] OK
+ Passes from genesis state, empty block [Preset: mainnet] OK
+ Passes from genesis state, no block [Preset: mainnet] OK
+ Passes through epoch update, empty block [Preset: mainnet] OK
+ Passes through epoch update, no block [Preset: mainnet] OK
```
OK: 5/5 Fail: 0/5 Skip: 0/5
## BlockRef and helpers [Preset: mainnet]
```diff
update attestation and block validation to v0.12.2; clean up getAncestorAt()/get_ancestor() (#1417) * update attestation validation to v0.12.2; clean up getAncestorAt()/get_ancestor() * update beacon block validation to v0.12.2
2020-08-03 19:47:42 +00:00
+ get_ancestor sanity [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ isAncestorOf sanity [Preset: mainnet] OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
## BlockSlot and helpers [Preset: mainnet]
```diff
+ atSlot sanity [Preset: mainnet] OK
+ parent sanity [Preset: mainnet] OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
[Ready 1/2] Fork choice rewrite (#865) * initial fork-choice refactor * Add fork_choice test for "no votes" * Initial test with voting: fix handling of unknown validators and parent blocks * Fix tiebreak of votes * Cleanup debugging traces * Complexify the vote test * fakeHash use the bigEndian repr of number + fix tiebreak for good * Stash changes: found critical bug in nimcrypto `==` and var openarray * Passing fork choice tests with varying votes * Add FFG fork choice scenario + fork choice to the test suite * Not sure why lmdb / rocksdb reappeared in rebase * Add sanity checks to .nimble file + integrate fork choice tests to the test DB and test timing * Cleanup debugging echos * nimcrypto fix https://github.com/status-im/nim-beacon-chain/pull/864 as been merged, remove TODO comment * Turn fork choice exception-free * Cleanup "result" to ensure early return is properly used * Add a comment on private/public error code vs Result * result -> results following https://github.com/status-im/nim-beacon-chain/pull/866 * Address comments: - raises: [Defect] doesn't work -> TODO - process_attestation cannot fail - try/except as expression pending Nim v1.2.0 - cleanup TODOs * re-enable all sanity checks * tag no raise for process_attestation * use raises defect everywhere in fork choice and fix process_attestation test
2020-04-09 16:15:00 +00:00
## Fork Choice + Finality [Preset: mainnet]
```diff
+ fork_choice - testing finality #01 OK
+ fork_choice - testing finality #02 OK
+ fork_choice - testing no votes OK
+ fork_choice - testing with votes OK
```
OK: 4/4 Fail: 0/4 Skip: 0/4
Add test reports
2020-03-10 04:24:33 +00:00
## Honest validator
```diff
remove some unused imports, add tests for pubsub topics, and subscribe to interop attestations
2020-05-14 11:19:10 +00:00
+ General pubsub topics: OK
+ Mainnet attestation topics OK
Add test reports
2020-03-10 04:24:33 +00:00
```
remove some unused imports, add tests for pubsub topics, and subscribe to interop attestations
2020-05-14 11:19:10 +00:00
OK: 2/2 Fail: 0/2 Skip: 0/2
re-add test_interop to all_tests and mark several v0.10.1 phase 0 spec references as v0.11.1
2020-04-06 18:55:47 +00:00
## Interop
```diff
+ Interop genesis OK
+ Interop signatures OK
+ Mocked start private key OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
Run tests
2020-05-27 15:37:10 +00:00
## Keystore
```diff
+ Pbkdf2 decryption OK
+ Pbkdf2 encryption OK
+ Pbkdf2 errors OK
synchronize test summaries with make test (and do a few small cleanups) (#1415)
2020-08-03 12:07:25 +00:00
+ Scrypt decryption OK
+ Scrypt encryption OK
Run tests
2020-05-27 15:37:10 +00:00
```
synchronize test summaries with make test (and do a few small cleanups) (#1415)
2020-08-03 12:07:25 +00:00
OK: 5/5 Fail: 0/5 Skip: 0/5
post-merge fixes
2020-04-08 21:09:52 +00:00
## Mocking utilities
```diff
+ merkle_minimal OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
allow compile-time switching between v0.11.3 and v0.12.1
2020-06-09 16:48:51 +00:00
## Official - constants & config [Preset: mainnet]
Add test reports
2020-03-10 04:24:33 +00:00
```diff
+ BASE_REWARD_FACTOR 64 [Preset: mainnet] OK
+ BLS_WITHDRAWAL_PREFIX "0x00" [Preset: mainnet] OK
+ CHURN_LIMIT_QUOTIENT 65536 [Preset: mainnet] OK
switch mapIt/find to in-place linear search (#1436)
2020-08-04 17:15:27 +00:00
DEPOSIT_CHAIN_ID 1 [Preset: mainnet] Skip
skip skipped constant sanity checks also from markdown reports' perspectives (#1053) * update more spec refs in beacon_chain/spec/presets; skip skipped constant sanity checks also from markdown reports' perspectives * mark skipped as skipped in markdown
2020-05-21 17:56:09 +00:00
DEPOSIT_CONTRACT_ADDRESS "0x1234567890123456789012345678901234567 Skip
switch mapIt/find to in-place linear search (#1436)
2020-08-04 17:15:27 +00:00
DEPOSIT_NETWORK_ID 1 [Preset: mainnet] Skip
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ DOMAIN_AGGREGATE_AND_PROOF "0x06000000" [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ DOMAIN_BEACON_ATTESTER "0x01000000" [Preset: mainnet] OK
+ DOMAIN_BEACON_PROPOSER "0x00000000" [Preset: mainnet] OK
+ DOMAIN_DEPOSIT "0x03000000" [Preset: mainnet] OK
+ DOMAIN_RANDAO "0x02000000" [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ DOMAIN_SELECTION_PROOF "0x05000000" [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ DOMAIN_VOLUNTARY_EXIT "0x04000000" [Preset: mainnet] OK
+ EFFECTIVE_BALANCE_INCREMENT 1000000000 [Preset: mainnet] OK
+ EJECTION_BALANCE 16000000000 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ EPOCHS_PER_ETH1_VOTING_PERIOD 32 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ EPOCHS_PER_HISTORICAL_VECTOR 65536 [Preset: mainnet] OK
+ EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION 256 [Preset: mainnet] OK
+ EPOCHS_PER_SLASHINGS_VECTOR 8192 [Preset: mainnet] OK
+ ETH1_FOLLOW_DISTANCE 1024 [Preset: mainnet] OK
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
GENESIS_DELAY 172800 [Preset: mainnet] Skip
skip skipped constant sanity checks also from markdown reports' perspectives (#1053) * update more spec refs in beacon_chain/spec/presets; skip skipped constant sanity checks also from markdown reports' perspectives * mark skipped as skipped in markdown
2020-05-21 17:56:09 +00:00
GENESIS_FORK_VERSION "0x00000000" [Preset: mainnet] Skip
Add test reports
2020-03-10 04:24:33 +00:00
+ HISTORICAL_ROOTS_LIMIT 16777216 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ HYSTERESIS_DOWNWARD_MULTIPLIER 1 [Preset: mainnet] OK
+ HYSTERESIS_QUOTIENT 4 [Preset: mainnet] OK
+ HYSTERESIS_UPWARD_MULTIPLIER 5 [Preset: mainnet] OK
initial 0.12.1 beacon chain spec
2020-06-08 18:41:50 +00:00
+ INACTIVITY_PENALTY_QUOTIENT 16777216 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MAX_ATTESTATIONS 128 [Preset: mainnet] OK
initial 0.12.1 beacon chain spec
2020-06-08 18:41:50 +00:00
+ MAX_ATTESTER_SLASHINGS 2 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MAX_COMMITTEES_PER_SLOT 64 [Preset: mainnet] OK
+ MAX_DEPOSITS 16 [Preset: mainnet] OK
+ MAX_EFFECTIVE_BALANCE 32000000000 [Preset: mainnet] OK
+ MAX_EPOCHS_PER_CROSSLINK 64 [Preset: mainnet] OK
+ MAX_PROPOSER_SLASHINGS 16 [Preset: mainnet] OK
+ MAX_SEED_LOOKAHEAD 4 [Preset: mainnet] OK
+ MAX_VALIDATORS_PER_COMMITTEE 2048 [Preset: mainnet] OK
+ MAX_VOLUNTARY_EXITS 16 [Preset: mainnet] OK
+ MIN_ATTESTATION_INCLUSION_DELAY 1 [Preset: mainnet] OK
+ MIN_DEPOSIT_AMOUNT 1000000000 [Preset: mainnet] OK
+ MIN_EPOCHS_TO_INACTIVITY_PENALTY 4 [Preset: mainnet] OK
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
MIN_GENESIS_ACTIVE_VALIDATOR_COUNT 16384 [Preset: mainnet] Skip
MIN_GENESIS_TIME 1578009600 [Preset: mainnet] Skip
Add test reports
2020-03-10 04:24:33 +00:00
+ MIN_PER_EPOCH_CHURN_LIMIT 4 [Preset: mainnet] OK
+ MIN_SEED_LOOKAHEAD 1 [Preset: mainnet] OK
+ MIN_SLASHING_PENALTY_QUOTIENT 32 [Preset: mainnet] OK
+ MIN_VALIDATOR_WITHDRAWABILITY_DELAY 256 [Preset: mainnet] OK
+ PROPOSER_REWARD_QUOTIENT 8 [Preset: mainnet] OK
+ RANDOM_SUBNETS_PER_VALIDATOR 1 [Preset: mainnet] OK
+ SAFE_SLOTS_TO_UPDATE_JUSTIFIED 8 [Preset: mainnet] OK
+ SECONDS_PER_ETH1_BLOCK 14 [Preset: mainnet] OK
+ SECONDS_PER_SLOT 12 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ SHARD_COMMITTEE_PERIOD 256 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ SHUFFLE_ROUND_COUNT 90 [Preset: mainnet] OK
+ SLOTS_PER_EPOCH 32 [Preset: mainnet] OK
+ SLOTS_PER_HISTORICAL_ROOT 8192 [Preset: mainnet] OK
+ TARGET_AGGREGATORS_PER_COMMITTEE 16 [Preset: mainnet] OK
+ TARGET_COMMITTEE_SIZE 128 [Preset: mainnet] OK
+ VALIDATOR_REGISTRY_LIMIT 1099511627776 [Preset: mainnet] OK
+ WHISTLEBLOWER_REWARD_QUOTIENT 512 [Preset: mainnet] OK
```
switch mapIt/find to in-place linear search (#1436)
2020-08-04 17:15:27 +00:00
OK: 52/59 Fail: 0/59 Skip: 7/59
Add test reports
2020-03-10 04:24:33 +00:00
## PeerPool testing suite
```diff
+ Access peers by key test OK
+ Acquire from empty pool OK
+ Acquire/Sorting and consistency test OK
+ Iterators test OK
+ Peer lifetime test OK
+ Safe/Clear test OK
detect improper 0-length lists of variable-sized objects in SSZ reading (#928) * detect improper 0-length lists of variable-sized objects in SSZ reading
2020-04-23 19:39:23 +00:00
+ Score check test OK
Add test reports
2020-03-10 04:24:33 +00:00
+ addPeer() test OK
+ addPeerNoWait() test OK
+ deletePeer() test OK
```
detect improper 0-length lists of variable-sized objects in SSZ reading (#928) * detect improper 0-length lists of variable-sized objects in SSZ reading
2020-04-23 19:39:23 +00:00
OK: 10/10 Fail: 0/10 Skip: 0/10
Add test reports
2020-03-10 04:24:33 +00:00
## SSZ dynamic navigator
```diff
+ navigating fields OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## SSZ navigator
```diff
support all basic types
2020-06-02 09:44:51 +00:00
+ basictype OK
Add test reports
2020-03-10 04:24:33 +00:00
+ lists with max size OK
+ simple object fields OK
```
support all basic types
2020-06-02 09:44:51 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
Restore compilation with custom presets (#1309) This also assigns precise types to the constants in the minimal and mainnet presets in order to reduce the chance of compilation errors when custom presets are used (previously, only the custom presets have precisely assigned types for the constants).
2020-07-13 14:44:58 +00:00
## Spec datatypes
```diff
+ Graffiti bytes OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Add test reports
2020-03-10 04:24:33 +00:00
## Spec helpers
```diff
+ integer_squareroot OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## Sync protocol
```diff
+ Compile OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## Zero signature sanity checks
```diff
+ SSZ serialization roundtrip of SignedBeaconBlockHeader OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## [Unit - Spec - Block processing] Attestations [Preset: mainnet]
```diff
+ Valid attestation OK
+ Valid attestation from previous epoch OK
```
update test results summary after BLS 0.10.1 update
2020-03-13 14:58:21 +00:00
OK: 2/2 Fail: 0/2 Skip: 0/2
Add test reports
2020-03-10 04:24:33 +00:00
## [Unit - Spec - Block processing] Deposits [Preset: mainnet]
```diff
+ Deposit at MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Deposit over MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Deposit under MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
Merge devel and resolve the conflicts
2020-07-08 12:36:03 +00:00
+ Invalid deposit at MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
Add test reports
2020-03-10 04:24:33 +00:00
+ Validator top-up OK
```
Merge devel and resolve the conflicts
2020-07-08 12:36:03 +00:00
OK: 5/5 Fail: 0/5 Skip: 0/5
Add test reports
2020-03-10 04:24:33 +00:00
## [Unit - Spec - Epoch processing] Justification and Finalization [Preset: mainnet]
```diff
+ Rule I - 234 finalization with enough support OK
+ Rule I - 234 finalization without support OK
+ Rule II - 23 finalization with enough support OK
+ Rule II - 23 finalization without support OK
+ Rule III - 123 finalization with enough support OK
+ Rule III - 123 finalization without support OK
+ Rule IV - 12 finalization with enough support OK
+ Rule IV - 12 finalization without support OK
```
OK: 8/8 Fail: 0/8 Skip: 0/8
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
## chain DAG finalization tests [Preset: mainnet]
```diff
+ prune heads on finalization [Preset: mainnet] OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
wip: cache * cache lists and arrays of complex objects (5x block processing speed on ncli_db) trivial baseline cache that stores tree in flat memory structure
2020-05-28 16:34:59 +00:00
## hash
```diff
+ HashArray OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Add test reports
2020-03-10 04:24:33 +00:00
---TOTAL---
switch mapIt/find to in-place linear search (#1436)
2020-08-04 17:15:27 +00:00
OK: 134/141 Fail: 0/141 Skip: 7/141