status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/AllTests-mainnet.md

263 lines
17 KiB
Markdown
Raw Normal View History

Add test reports
2020-03-10 04:24:33 +00:00
AllTests-mainnet
===
re-enable mainnet attestation pool tests (#946) * re-enable mainnet attestation pool tests; since https://github.com/status-im/nim-beacon-chain/pull/930 double CI state sim validators * bump deposits spec ref to v0.11.1
2020-04-29 11:44:07 +00:00
## Attestation pool processing [Preset: mainnet]
```diff
+ Attestations may arrive in any order [Preset: mainnet] OK
+ Attestations may overlap, bigger first [Preset: mainnet] OK
+ Attestations may overlap, smaller first [Preset: mainnet] OK
+ Attestations should be combined [Preset: mainnet] OK
+ Can add and retrieve simple attestation [Preset: mainnet] OK
+ Fork choice returns block with attestation OK
+ Fork choice returns latest block with no attestations OK
```
OK: 7/7 Fail: 0/7 Skip: 0/7
Add test reports
2020-03-10 04:24:33 +00:00
## Beacon chain DB [Preset: mainnet]
```diff
+ empty database [Preset: mainnet] OK
+ find ancestors [Preset: mainnet] OK
+ sanity check blocks [Preset: mainnet] OK
+ sanity check genesis roundtrip [Preset: mainnet] OK
+ sanity check states [Preset: mainnet] OK
```
OK: 5/5 Fail: 0/5 Skip: 0/5
## Beacon node
```diff
+ Compile OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## Beacon state [Preset: mainnet]
```diff
+ Smoke test initialize_beacon_state_from_eth1 [Preset: mainnet] OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
## Block pool processing [Preset: mainnet]
```diff
+ Can add same block twice [Preset: mainnet] OK
+ Reverse order block add & get [Preset: mainnet] OK
+ Simple block add&get [Preset: mainnet] OK
+ getRef returns nil for missing blocks OK
+ loadTailState gets genesis block on first load [Preset: mainnet] OK
+ updateHead updates head and headState [Preset: mainnet] OK
+ updateStateData sanity [Preset: mainnet] OK
```
OK: 7/7 Fail: 0/7 Skip: 0/7
Add test reports
2020-03-10 04:24:33 +00:00
## Block processing [Preset: mainnet]
```diff
+ Attestation gets processed at epoch [Preset: mainnet] OK
+ Passes from genesis state, empty block [Preset: mainnet] OK
+ Passes from genesis state, no block [Preset: mainnet] OK
+ Passes through epoch update, empty block [Preset: mainnet] OK
+ Passes through epoch update, no block [Preset: mainnet] OK
```
OK: 5/5 Fail: 0/5 Skip: 0/5
## BlockRef and helpers [Preset: mainnet]
```diff
+ getAncestorAt sanity [Preset: mainnet] OK
+ isAncestorOf sanity [Preset: mainnet] OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
## BlockSlot and helpers [Preset: mainnet]
```diff
+ atSlot sanity [Preset: mainnet] OK
+ parent sanity [Preset: mainnet] OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
[Ready 1/2] Fork choice rewrite (#865) * initial fork-choice refactor * Add fork_choice test for "no votes" * Initial test with voting: fix handling of unknown validators and parent blocks * Fix tiebreak of votes * Cleanup debugging traces * Complexify the vote test * fakeHash use the bigEndian repr of number + fix tiebreak for good * Stash changes: found critical bug in nimcrypto `==` and var openarray * Passing fork choice tests with varying votes * Add FFG fork choice scenario + fork choice to the test suite * Not sure why lmdb / rocksdb reappeared in rebase * Add sanity checks to .nimble file + integrate fork choice tests to the test DB and test timing * Cleanup debugging echos * nimcrypto fix https://github.com/status-im/nim-beacon-chain/pull/864 as been merged, remove TODO comment * Turn fork choice exception-free * Cleanup "result" to ensure early return is properly used * Add a comment on private/public error code vs Result * result -> results following https://github.com/status-im/nim-beacon-chain/pull/866 * Address comments: - raises: [Defect] doesn't work -> TODO - process_attestation cannot fail - try/except as expression pending Nim v1.2.0 - cleanup TODOs * re-enable all sanity checks * tag no raise for process_attestation * use raises defect everywhere in fork choice and fix process_attestation test
2020-04-09 16:15:00 +00:00
## Fork Choice + Finality [Preset: mainnet]
```diff
+ fork_choice - testing finality #01 OK
+ fork_choice - testing finality #02 OK
+ fork_choice - testing no votes OK
+ fork_choice - testing with votes OK
```
OK: 4/4 Fail: 0/4 Skip: 0/4
Add test reports
2020-03-10 04:24:33 +00:00
## Honest validator
```diff
remove some unused imports, add tests for pubsub topics, and subscribe to interop attestations
2020-05-14 11:19:10 +00:00
+ General pubsub topics: OK
+ Mainnet attestation topics OK
Add test reports
2020-03-10 04:24:33 +00:00
```
remove some unused imports, add tests for pubsub topics, and subscribe to interop attestations
2020-05-14 11:19:10 +00:00
OK: 2/2 Fail: 0/2 Skip: 0/2
re-add test_interop to all_tests and mark several v0.10.1 phase 0 spec references as v0.11.1
2020-04-06 18:55:47 +00:00
## Interop
```diff
+ Interop genesis OK
+ Interop signatures OK
+ Mocked start private key OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
Run tests
2020-05-27 15:37:10 +00:00
## Keystore
```diff
+ Pbkdf2 decryption OK
+ Pbkdf2 encryption OK
+ Pbkdf2 errors OK
```
OK: 3/3 Fail: 0/3 Skip: 0/3
post-merge fixes
2020-04-08 21:09:52 +00:00
## Mocking utilities
```diff
+ merkle_minimal OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
allow compile-time switching between v0.11.3 and v0.12.1
2020-06-09 16:48:51 +00:00
## Official - constants & config [Preset: mainnet]
Add test reports
2020-03-10 04:24:33 +00:00
```diff
+ BASE_REWARD_FACTOR 64 [Preset: mainnet] OK
+ BLS_WITHDRAWAL_PREFIX "0x00" [Preset: mainnet] OK
+ CHURN_LIMIT_QUOTIENT 65536 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ CUSTODY_PERIOD_TO_RANDAO_PADDING 2048 [Preset: mainnet] OK
skip skipped constant sanity checks also from markdown reports' perspectives (#1053) * update more spec refs in beacon_chain/spec/presets; skip skipped constant sanity checks also from markdown reports' perspectives * mark skipped as skipped in markdown
2020-05-21 17:56:09 +00:00
DEPOSIT_CONTRACT_ADDRESS "0x1234567890123456789012345678901234567 Skip
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ DOMAIN_AGGREGATE_AND_PROOF "0x06000000" [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ DOMAIN_BEACON_ATTESTER "0x01000000" [Preset: mainnet] OK
+ DOMAIN_BEACON_PROPOSER "0x00000000" [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ DOMAIN_CUSTODY_BIT_SLASHING "0x83000000" [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ DOMAIN_DEPOSIT "0x03000000" [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ DOMAIN_LIGHT_CLIENT "0x82000000" [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ DOMAIN_RANDAO "0x02000000" [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ DOMAIN_SELECTION_PROOF "0x05000000" [Preset: mainnet] OK
+ DOMAIN_SHARD_COMMITTEE "0x81000000" [Preset: mainnet] OK
+ DOMAIN_SHARD_PROPOSAL "0x80000000" [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ DOMAIN_VOLUNTARY_EXIT "0x04000000" [Preset: mainnet] OK
+ EARLY_DERIVED_SECRET_PENALTY_MAX_FUTURE_EPOCHS 16384 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ EARLY_DERIVED_SECRET_REVEAL_SLOT_REWARD_MULTIPLE 2 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ EFFECTIVE_BALANCE_INCREMENT 1000000000 [Preset: mainnet] OK
+ EJECTION_BALANCE 16000000000 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ EPOCHS_PER_CUSTODY_PERIOD 2048 [Preset: mainnet] OK
+ EPOCHS_PER_ETH1_VOTING_PERIOD 32 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ EPOCHS_PER_HISTORICAL_VECTOR 65536 [Preset: mainnet] OK
+ EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION 256 [Preset: mainnet] OK
+ EPOCHS_PER_SLASHINGS_VECTOR 8192 [Preset: mainnet] OK
+ ETH1_FOLLOW_DISTANCE 1024 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ GASPRICE_ADJUSTMENT_COEFFICIENT 8 [Preset: mainnet] OK
initial 0.12.1 beacon chain spec
2020-06-08 18:41:50 +00:00
+ GENESIS_DELAY 172800 [Preset: mainnet] OK
skip skipped constant sanity checks also from markdown reports' perspectives (#1053) * update more spec refs in beacon_chain/spec/presets; skip skipped constant sanity checks also from markdown reports' perspectives * mark skipped as skipped in markdown
2020-05-21 17:56:09 +00:00
GENESIS_FORK_VERSION "0x00000000" [Preset: mainnet] Skip
Add test reports
2020-03-10 04:24:33 +00:00
+ HISTORICAL_ROOTS_LIMIT 16777216 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ HYSTERESIS_DOWNWARD_MULTIPLIER 1 [Preset: mainnet] OK
+ HYSTERESIS_QUOTIENT 4 [Preset: mainnet] OK
+ HYSTERESIS_UPWARD_MULTIPLIER 5 [Preset: mainnet] OK
initial 0.12.1 beacon chain spec
2020-06-08 18:41:50 +00:00
+ INACTIVITY_PENALTY_QUOTIENT 16777216 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ INITIAL_ACTIVE_SHARDS 64 [Preset: mainnet] OK
+ LIGHT_CLIENT_COMMITTEE_PERIOD 256 [Preset: mainnet] OK
+ LIGHT_CLIENT_COMMITTEE_SIZE 128 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MAX_ATTESTATIONS 128 [Preset: mainnet] OK
initial 0.12.1 beacon chain spec
2020-06-08 18:41:50 +00:00
+ MAX_ATTESTER_SLASHINGS 2 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MAX_COMMITTEES_PER_SLOT 64 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ MAX_CUSTODY_KEY_REVEALS 256 [Preset: mainnet] OK
+ MAX_CUSTODY_SLASHINGS 1 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MAX_DEPOSITS 16 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ MAX_EARLY_DERIVED_SECRET_REVEALS 1 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MAX_EFFECTIVE_BALANCE 32000000000 [Preset: mainnet] OK
+ MAX_EPOCHS_PER_CROSSLINK 64 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ MAX_GASPRICE 16384 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MAX_PROPOSER_SLASHINGS 16 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ MAX_REVEAL_LATENESS_DECREMENT 128 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MAX_SEED_LOOKAHEAD 4 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ MAX_SHARDS 1024 [Preset: mainnet] OK
+ MAX_SHARD_BLOCKS_PER_ATTESTATION 12 [Preset: mainnet] OK
+ MAX_SHARD_BLOCK_CHUNKS 4 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MAX_VALIDATORS_PER_COMMITTEE 2048 [Preset: mainnet] OK
+ MAX_VOLUNTARY_EXITS 16 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ MINOR_REWARD_QUOTIENT 256 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MIN_ATTESTATION_INCLUSION_DELAY 1 [Preset: mainnet] OK
+ MIN_DEPOSIT_AMOUNT 1000000000 [Preset: mainnet] OK
+ MIN_EPOCHS_TO_INACTIVITY_PENALTY 4 [Preset: mainnet] OK
update to spec v0.11.2 to keep compatible with Schlesi (#1036) * update to spec v0.11.2 to keep compatible with Schlesi * update spec/state_transition_epoch spec references to v0.11.2 * bump other spec refs
2020-05-18 23:05:27 +00:00
+ MIN_GASPRICE 8 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ MIN_GENESIS_ACTIVE_VALIDATOR_COUNT 16384 [Preset: mainnet] OK
+ MIN_GENESIS_TIME 1578009600 [Preset: mainnet] OK
+ MIN_PER_EPOCH_CHURN_LIMIT 4 [Preset: mainnet] OK
+ MIN_SEED_LOOKAHEAD 1 [Preset: mainnet] OK
+ MIN_SLASHING_PENALTY_QUOTIENT 32 [Preset: mainnet] OK
+ MIN_VALIDATOR_WITHDRAWABILITY_DELAY 256 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ ONLINE_PERIOD 8 [Preset: mainnet] OK
+ PHASE_1_FORK_VERSION "0x01000000" [Preset: mainnet] OK
initial 0.12.1 beacon chain spec
2020-06-08 18:41:50 +00:00
+ PHASE_1_GENESIS_SLOT 32 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ PROPOSER_REWARD_QUOTIENT 8 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ RANDAO_PENALTY_EPOCHS 2 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ RANDOM_SUBNETS_PER_VALIDATOR 1 [Preset: mainnet] OK
+ SAFE_SLOTS_TO_UPDATE_JUSTIFIED 8 [Preset: mainnet] OK
+ SECONDS_PER_ETH1_BLOCK 14 [Preset: mainnet] OK
+ SECONDS_PER_SLOT 12 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ SHARD_BLOCK_CHUNK_SIZE 262144 [Preset: mainnet] OK
skip skipped constant sanity checks also from markdown reports' perspectives (#1053) * update more spec refs in beacon_chain/spec/presets; skip skipped constant sanity checks also from markdown reports' perspectives * mark skipped as skipped in markdown
2020-05-21 17:56:09 +00:00
SHARD_BLOCK_OFFSETS [1,2,3,5,8,13,21,34,55,89,144,233] [Pres Skip
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ SHARD_COMMITTEE_PERIOD 256 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ SHUFFLE_ROUND_COUNT 90 [Preset: mainnet] OK
+ SLOTS_PER_EPOCH 32 [Preset: mainnet] OK
+ SLOTS_PER_HISTORICAL_ROOT 8192 [Preset: mainnet] OK
+ TARGET_AGGREGATORS_PER_COMMITTEE 16 [Preset: mainnet] OK
+ TARGET_COMMITTEE_SIZE 128 [Preset: mainnet] OK
initial 0.11.0 spec version update
2020-03-14 21:54:45 +00:00
+ TARGET_SHARD_BLOCK_SIZE 196608 [Preset: mainnet] OK
Add test reports
2020-03-10 04:24:33 +00:00
+ VALIDATOR_REGISTRY_LIMIT 1099511627776 [Preset: mainnet] OK
+ WHISTLEBLOWER_REWARD_QUOTIENT 512 [Preset: mainnet] OK
```
skip skipped constant sanity checks also from markdown reports' perspectives (#1053) * update more spec refs in beacon_chain/spec/presets; skip skipped constant sanity checks also from markdown reports' perspectives * mark skipped as skipped in markdown
2020-05-21 17:56:09 +00:00
OK: 83/86 Fail: 0/86 Skip: 3/86
Add test reports
2020-03-10 04:24:33 +00:00
## PeerPool testing suite
```diff
+ Access peers by key test OK
+ Acquire from empty pool OK
+ Acquire/Sorting and consistency test OK
+ Iterators test OK
+ Peer lifetime test OK
+ Safe/Clear test OK
detect improper 0-length lists of variable-sized objects in SSZ reading (#928) * detect improper 0-length lists of variable-sized objects in SSZ reading
2020-04-23 19:39:23 +00:00
+ Score check test OK
Add test reports
2020-03-10 04:24:33 +00:00
+ addPeer() test OK
+ addPeerNoWait() test OK
+ deletePeer() test OK
```
detect improper 0-length lists of variable-sized objects in SSZ reading (#928) * detect improper 0-length lists of variable-sized objects in SSZ reading
2020-04-23 19:39:23 +00:00
OK: 10/10 Fail: 0/10 Skip: 0/10
Add test reports
2020-03-10 04:24:33 +00:00
## SSZ dynamic navigator
```diff
+ navigating fields OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## SSZ navigator
```diff
support all basic types
2020-06-02 09:44:51 +00:00
+ basictype OK
Add test reports
2020-03-10 04:24:33 +00:00
+ lists with max size OK
+ simple object fields OK
```
support all basic types
2020-06-02 09:44:51 +00:00
OK: 3/3 Fail: 0/3 Skip: 0/3
Add test reports
2020-03-10 04:24:33 +00:00
## Spec helpers
```diff
+ integer_squareroot OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## Sync protocol
```diff
+ Compile OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## Zero signature sanity checks
```diff
+ SSZ serialization roundtrip of SignedBeaconBlockHeader OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## [Unit - Spec - Block processing] Attestations [Preset: mainnet]
```diff
+ Valid attestation OK
+ Valid attestation from previous epoch OK
```
update test results summary after BLS 0.10.1 update
2020-03-13 14:58:21 +00:00
OK: 2/2 Fail: 0/2 Skip: 0/2
Add test reports
2020-03-10 04:24:33 +00:00
## [Unit - Spec - Block processing] Deposits [Preset: mainnet]
```diff
+ Deposit at MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Deposit over MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Deposit under MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Validator top-up OK
```
OK: 4/4 Fail: 0/4 Skip: 0/4
## [Unit - Spec - Epoch processing] Justification and Finalization [Preset: mainnet]
```diff
+ Rule I - 234 finalization with enough support OK
+ Rule I - 234 finalization without support OK
+ Rule II - 23 finalization with enough support OK
+ Rule II - 23 finalization without support OK
+ Rule III - 123 finalization with enough support OK
+ Rule III - 123 finalization without support OK
+ Rule IV - 12 finalization with enough support OK
+ Rule IV - 12 finalization without support OK
```
OK: 8/8 Fail: 0/8 Skip: 0/8
wip: cache * cache lists and arrays of complex objects (5x block processing speed on ncli_db) trivial baseline cache that stores tree in flat memory structure
2020-05-28 16:34:59 +00:00
## hash
```diff
+ HashArray OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
Add test reports
2020-03-10 04:24:33 +00:00
---TOTAL---
reorder ssz (#1099) * reorder ssz * split into hash_trees and ssz_serialization, roughly, for hashing and IO * move bitseqs into ssz (from stew) * clean up imports * docs, imports
2020-06-03 13:52:02 +00:00
OK: 158/161 Fail: 0/161 Skip: 3/161