nimbus-eth2/AllTests-mainnet.md
Jacek Sieka 9c2f43ed0e
Speed up altair block processing 2x (#3115)
* Speed up altair block processing >2x

Like #3089, this PR drastially speeds up historical REST queries and
other long state replays.

* cache sync committee validator indices
* use ~80mb less memory for validator pubkey mappings
* batch-verify sync aggregate signature (fixes #2985)
* document sync committee hack with head block vs sync message block
* add batch signature verification failure tests

Before:

```
../env.sh nim c -d:release -r ncli_db --db:mainnet_0/db bench --start-slot:-1000
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    5830.675,        0.000,     5830.675,     5830.675,            1, Initialize DB
       0.481,        1.878,        0.215,       59.167,          981, Load block from database
    8422.566,        0.000,     8422.566,     8422.566,            1, Load state from database
       6.996,        1.678,        0.042,       14.385,          969, Advance slot, non-epoch
      93.217,        8.318,       84.192,      122.209,           32, Advance slot, epoch
      20.513,       23.665,       11.510,      201.561,          981, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

After:

```
    7081.422,        0.000,     7081.422,     7081.422,            1, Initialize DB
       0.553,        2.122,        0.175,       66.692,          981, Load block from database
    5439.446,        0.000,     5439.446,     5439.446,            1, Load state from database
       6.829,        1.575,        0.043,       12.156,          969, Advance slot, non-epoch
      94.716,        2.749,       88.395,      100.026,           32, Advance slot, epoch
      11.636,       23.766,        4.889,      205.250,          981, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

* add comment
2021-11-24 13:43:50 +01:00

23 KiB

AllTests-mainnet

+ Slashing test: duplicate_pubkey_not_slashable.json                                         OK
+ Slashing test: multiple_interchanges_single_validator_single_message_gap.json              OK
+ Slashing test: multiple_interchanges_single_validator_single_message_out_of_order.json     OK
+ Slashing test: multiple_validators_multiple_blocks_and_attestations.json                   OK
+ Slashing test: multiple_validators_same_slot_blocks.json                                   OK
+ Slashing test: single_validator_genesis_attestation.json                                   OK
+ Slashing test: single_validator_import_only.json                                           OK
+ Slashing test: single_validator_multiple_block_attempts.json                               OK
+ Slashing test: single_validator_multiple_blocks_and_attestations.json                      OK
+ Slashing test: single_validator_out_of_order_attestations.json                             OK
+ Slashing test: single_validator_out_of_order_blocks.json                                   OK
+ Slashing test: single_validator_resign_attestation.json                                    OK
+ Slashing test: single_validator_resign_block.json                                          OK
+ Slashing test: single_validator_single_attestation.json                                    OK
+ Slashing test: single_validator_single_block.json                                          OK
+ Slashing test: single_validator_single_block_and_attestation.json                          OK
+ Slashing test: single_validator_single_block_and_attestation_signing_root.json             OK
+ Slashing test: single_validator_slashable_attestations_double_vote.json                    OK
+ Slashing test: single_validator_slashable_attestations_surrounded_by_existing.json         OK
+ Slashing test: single_validator_slashable_attestations_surrounds_existing.json             OK
+ Slashing test: single_validator_slashable_blocks.json                                      OK
+ Slashing test: single_validator_slashable_blocks_no_root.json                              OK
+ Slashing test: single_validator_source_greater_than_target.json                            OK
+ Slashing test: single_validator_two_blocks_no_signing_root.json                            OK
+ Slashing test: wrong_genesis_validators_root.json                                          OK

OK: 25/25 Fail: 0/25 Skip: 0/25

Attestation pool processing [Preset: mainnet]

+ Attestations may arrive in any order [Preset: mainnet]                                     OK
+ Attestations may overlap, bigger first [Preset: mainnet]                                   OK
+ Attestations may overlap, smaller first [Preset: mainnet]                                  OK
+ Attestations should be combined [Preset: mainnet]                                          OK
+ Can add and retrieve simple attestations [Preset: mainnet]                                 OK
+ Everyone voting for something different [Preset: mainnet]                                  OK
+ Fork choice returns block with attestation                                                 OK
+ Fork choice returns latest block with no attestations                                      OK
+ Trying to add a block twice tags the second as an error                                    OK
+ Trying to add a duplicate block from an old pruned epoch is tagged as an error             OK
+ Working with aggregates [Preset: mainnet]                                                  OK

OK: 11/11 Fail: 0/11 Skip: 0/11

Beacon chain DB [Preset: mainnet]

+ empty database [Preset: mainnet]                                                           OK
+ find ancestors [Preset: mainnet]                                                           OK
+ sanity check Altair and cross-fork getState rollback [Preset: mainnet]                     OK
+ sanity check Altair blocks [Preset: mainnet]                                               OK
+ sanity check Altair states [Preset: mainnet]                                               OK
+ sanity check Altair states, reusing buffers [Preset: mainnet]                              OK
+ sanity check Merge and cross-fork getState rollback [Preset: mainnet]                      OK
+ sanity check Merge blocks [Preset: mainnet]                                                OK
+ sanity check Merge states [Preset: mainnet]                                                OK
+ sanity check Merge states, reusing buffers [Preset: mainnet]                               OK
+ sanity check genesis roundtrip [Preset: mainnet]                                           OK
+ sanity check phase 0 blocks [Preset: mainnet]                                              OK
+ sanity check phase 0 getState rollback [Preset: mainnet]                                   OK
+ sanity check phase 0 states [Preset: mainnet]                                              OK
+ sanity check phase 0 states, reusing buffers [Preset: mainnet]                             OK
+ sanity check state diff roundtrip [Preset: mainnet]                                        OK

OK: 16/16 Fail: 0/16 Skip: 0/16

Beacon state [Preset: mainnet]

+ Smoke test initialize_beacon_state_from_eth1 [Preset: mainnet]                             OK
+ get_beacon_proposer_index                                                                  OK
+ latest_block_root                                                                          OK

OK: 3/3 Fail: 0/3 Skip: 0/3

Block pool altair processing [Preset: mainnet]

+ Invalid signatures [Preset: mainnet]                                                       OK

OK: 1/1 Fail: 0/1 Skip: 0/1

Block pool processing [Preset: mainnet]

+ Adding the same block twice returns a Duplicate error [Preset: mainnet]                    OK
+ Reverse order block add & get [Preset: mainnet]                                            OK
+ Simple block add&get [Preset: mainnet]                                                     OK
+ getRef returns nil for missing blocks                                                      OK
+ loading tail block works [Preset: mainnet]                                                 OK
+ updateHead updates head and headState [Preset: mainnet]                                    OK
+ updateStateData sanity [Preset: mainnet]                                                   OK

OK: 7/7 Fail: 0/7 Skip: 0/7

BlockRef and helpers [Preset: mainnet]

+ epochAncestor sanity [Preset: mainnet]                                                     OK
+ get_ancestor sanity [Preset: mainnet]                                                      OK
+ isAncestorOf sanity [Preset: mainnet]                                                      OK

OK: 3/3 Fail: 0/3 Skip: 0/3

BlockSlot and helpers [Preset: mainnet]

+ atSlot sanity [Preset: mainnet]                                                            OK
+ parent sanity [Preset: mainnet]                                                            OK

OK: 2/2 Fail: 0/2 Skip: 0/2

Diverging hardforks

+ Non-tail block in common                                                                   OK
+ Tail block only in common                                                                  OK

OK: 2/2 Fail: 0/2 Skip: 0/2

Eth1 monitor

+ Rewrite HTTPS Infura URLs                                                                  OK

OK: 1/1 Fail: 0/1 Skip: 0/1

Eth2 specific discovery tests

+ Invalid attnets field                                                                      OK
+ Subnet query                                                                               OK
+ Subnet query after ENR update                                                              OK

OK: 3/3 Fail: 0/3 Skip: 0/3

Ethereum Foundation - SSZ generic types

  Testing basic_vector inputs - invalid                                                      Skip
+ Testing basic_vector inputs - valid                                                        OK
+ Testing bitlist      inputs - invalid                                                      OK
+ Testing bitlist      inputs - valid                                                        OK
  Testing bitvector    inputs - invalid                                                      Skip
+ Testing bitvector    inputs - valid                                                        OK
+ Testing boolean      inputs - invalid                                                      OK
+ Testing boolean      inputs - valid                                                        OK
+ Testing containers   inputs - invalid - skipping BitsStruct                                OK
+ Testing containers   inputs - valid - skipping BitsStruct                                  OK
+ Testing uints        inputs - invalid                                                      OK
+ Testing uints        inputs - valid                                                        OK

OK: 10/12 Fail: 0/12 Skip: 2/12

Exit pool testing suite

+ addExitMessage/getAttesterSlashingMessage                                                  OK
+ addExitMessage/getProposerSlashingMessage                                                  OK
+ addExitMessage/getVoluntaryExitMessage                                                     OK

OK: 3/3 Fail: 0/3 Skip: 0/3

Fork Choice + Finality [Preset: mainnet]

+ fork_choice - testing finality #01                                                         OK
+ fork_choice - testing finality #02                                                         OK
+ fork_choice - testing no votes                                                             OK
+ fork_choice - testing with votes                                                           OK

OK: 4/4 Fail: 0/4 Skip: 0/4

Fork id compatibility test

+ Digest check                                                                               OK
+ Fork check                                                                                 OK
+ Next fork epoch check                                                                      OK

OK: 3/3 Fail: 0/3 Skip: 0/3

Forked SSZ readers

+ load altair block                                                                          OK
+ load altair state                                                                          OK
+ load merge block                                                                           OK
+ load merge state                                                                           OK
+ load phase0 block                                                                          OK
+ load phase0 state                                                                          OK
+ should raise on unknown data                                                               OK

OK: 7/7 Fail: 0/7 Skip: 0/7

Gossip validation [Preset: mainnet]

+ Any committee index is valid                                                               OK
+ validateAttestation                                                                        OK

OK: 2/2 Fail: 0/2 Skip: 0/2

Gossip validation - Extra

+ validateSyncCommitteeMessage                                                               OK

OK: 1/1 Fail: 0/1 Skip: 0/1

Honest validator

+ General pubsub topics                                                                      OK
+ Mainnet attestation topics                                                                 OK
+ is_aggregator                                                                              OK

OK: 3/3 Fail: 0/3 Skip: 0/3

Interop

+ Interop genesis                                                                            OK
+ Interop signatures                                                                         OK
+ Mocked start private key                                                                   OK

OK: 3/3 Fail: 0/3 Skip: 0/3

KeyStorage testing suite

+ Pbkdf2 errors                                                                              OK
+ [PBKDF2] Keystore decryption                                                               OK
+ [PBKDF2] Keystore encryption                                                               OK
+ [PBKDF2] Network Keystore decryption                                                       OK
+ [PBKDF2] Network Keystore encryption                                                       OK
+ [SCRYPT] Keystore decryption                                                               OK
+ [SCRYPT] Keystore encryption                                                               OK
+ [SCRYPT] Network Keystore decryption                                                       OK
+ [SCRYPT] Network Keystore encryption                                                       OK

OK: 9/9 Fail: 0/9 Skip: 0/9

Message signatures

+ Aggregate and proof signatures                                                             OK
+ Attestation signatures                                                                     OK
+ Deposit signatures                                                                         OK
+ Slot signatures                                                                            OK
+ Sync committee message signatures                                                          OK
+ Sync committee selection proof signatures                                                  OK
+ Sync committee signed contribution and proof signatures                                    OK
+ Voluntary exit signatures                                                                  OK

OK: 8/8 Fail: 0/8 Skip: 0/8

Old database versions [Preset: mainnet]

+ pre-1.1.0                                                                                  OK

OK: 1/1 Fail: 0/1 Skip: 0/1

PeerPool testing suite

+ Access peers by key test                                                                   OK
+ Acquire from empty pool                                                                    OK
+ Acquire/Sorting and consistency test                                                       OK
+ Delete peer on release text                                                                OK
+ Iterators test                                                                             OK
+ Peer lifetime test                                                                         OK
+ Safe/Clear test                                                                            OK
+ Score check test                                                                           OK
+ Space tests                                                                                OK
+ addPeer() test                                                                             OK
+ addPeerNoWait() test                                                                       OK
+ deletePeer() test                                                                          OK

OK: 12/12 Fail: 0/12 Skip: 0/12

Slashing Protection DB - Interchange [Preset: mainnet]

+ Smoke test - Complete format - Invalid database is refused [Preset: mainnet]               OK
+ Smoke test - Complete format [Preset: mainnet]                                             OK

OK: 2/2 Fail: 0/2 Skip: 0/2

Slashing Protection DB - v1 and v2 migration [Preset: mainnet]

+ Minimal format migration [Preset: mainnet]                                                 OK

OK: 1/1 Fail: 0/1 Skip: 0/1

Slashing Protection DB [Preset: mainnet]

+ Attestation ordering #1698                                                                 OK
+ Don't prune the very last attestation(s) even by mistake                                   OK
+ Don't prune the very last block even by mistake                                            OK
+ Empty database [Preset: mainnet]                                                           OK
+ Pruning attestations works                                                                 OK
+ Pruning blocks works                                                                       OK
+ SP for block proposal - backtracking append                                                OK
+ SP for block proposal - linear append                                                      OK
+ SP for same epoch attestation target - linear append                                       OK
+ SP for surrounded attestations                                                             OK
+ SP for surrounding attestations                                                            OK
+ Test valid attestation #1699                                                               OK

OK: 12/12 Fail: 0/12 Skip: 0/12

Spec datatypes

+ Graffiti bytes                                                                             OK

OK: 1/1 Fail: 0/1 Skip: 0/1

Spec helpers

+ build_proof - BeaconState                                                                  OK
+ get_branch_indices                                                                         OK
+ get_helper_indices                                                                         OK
+ get_path_indices                                                                           OK
+ integer_squareroot                                                                         OK
+ verify_merkle_multiproof                                                                   OK

OK: 6/6 Fail: 0/6 Skip: 0/6

Specific field types

+ root update                                                                                OK
+ roundtrip                                                                                  OK

OK: 2/2 Fail: 0/2 Skip: 0/2

Sync committee pool

+ Aggregating votes                                                                          OK
+ An empty pool is safe to prune                                                             OK
+ An empty pool is safe to prune 2                                                           OK
+ An empty pool is safe to use                                                               OK

OK: 4/4 Fail: 0/4 Skip: 0/4

SyncManager test suite

+ [SyncQueue] Async pending and resetWait() test                                             OK
+ [SyncQueue] Async unordered push start from zero                                           OK
+ [SyncQueue] Async unordered push with not full start from non-zero                         OK
+ [SyncQueue] Full and incomplete success/fail start from non-zero                           OK
+ [SyncQueue] Full and incomplete success/fail start from zero                               OK
+ [SyncQueue] One smart and one stupid + debt split + empty                                  OK
+ [SyncQueue] Smart and stupid success/fail                                                  OK
+ [SyncQueue] Start and finish slots equal                                                   OK
+ [SyncQueue] Two full requests success/fail                                                 OK
+ [SyncQueue] checkResponse() test                                                           OK
+ [SyncQueue] contains() test                                                                OK
+ [SyncQueue] getLastNonEmptySlot() test                                                     OK
+ [SyncQueue] getRewindPoint() test                                                          OK
+ [SyncQueue] hasEndGap() test                                                               OK

OK: 14/14 Fail: 0/14 Skip: 0/14

Zero signature sanity checks

+ SSZ serialization roundtrip of SignedBeaconBlockHeader                                     OK
+ Zero signatures cannot be loaded into a BLS signature object                               OK
+ default initialization of signatures                                                       OK

OK: 3/3 Fail: 0/3 Skip: 0/3

[Unit - Spec - Block processing] Attestations [Preset: mainnet]

+ Valid attestation                                                                          OK
+ Valid attestation from previous epoch                                                      OK

OK: 2/2 Fail: 0/2 Skip: 0/2

[Unit - Spec - Block processing] Deposits [Preset: mainnet]

+ Deposit at MAX_EFFECTIVE_BALANCE balance (32 ETH)                                          OK
+ Deposit over MAX_EFFECTIVE_BALANCE balance (32 ETH)                                        OK
+ Deposit under MAX_EFFECTIVE_BALANCE balance (32 ETH)                                       OK
+ Invalid deposit at MAX_EFFECTIVE_BALANCE balance (32 ETH)                                  OK
+ Validator top-up                                                                           OK

OK: 5/5 Fail: 0/5 Skip: 0/5

[Unit - Spec - Epoch processing] Justification and Finalization [Preset: mainnet]

+  Rule I - 234 finalization with enough support                                             OK
+  Rule I - 234 finalization without support                                                 OK
+  Rule II - 23 finalization with enough support                                             OK
+  Rule II - 23 finalization without support                                                 OK
+  Rule III - 123 finalization with enough support                                           OK
+  Rule III - 123 finalization without support                                               OK
+  Rule IV - 12 finalization with enough support                                             OK
+  Rule IV - 12 finalization without support                                                 OK

OK: 8/8 Fail: 0/8 Skip: 0/8

chain DAG finalization tests [Preset: mainnet]

+ init with gaps [Preset: mainnet]                                                           OK
+ orphaned epoch block [Preset: mainnet]                                                     OK
+ prune heads on finalization [Preset: mainnet]                                              OK

OK: 3/3 Fail: 0/3 Skip: 0/3

eth2.0-deposits-cli compatibility

+ restoring mnemonic with password                                                           OK
+ restoring mnemonic without password                                                        OK

OK: 2/2 Fail: 0/2 Skip: 0/2

state diff tests [Preset: mainnet]

+ random slot differences [Preset: mainnet]                                                  OK

OK: 1/1 Fail: 0/1 Skip: 0/1

subnet tracker

+ should register stability subnets on attester duties                                       OK

OK: 1/1 Fail: 0/1 Skip: 0/1

---TOTAL--- OK: 207/209 Fail: 0/209 Skip: 2/209