nimbus-eth2/tests
Jacek Sieka 9c2f43ed0e
Speed up altair block processing 2x (#3115)
* Speed up altair block processing >2x

Like #3089, this PR drastially speeds up historical REST queries and
other long state replays.

* cache sync committee validator indices
* use ~80mb less memory for validator pubkey mappings
* batch-verify sync aggregate signature (fixes #2985)
* document sync committee hack with head block vs sync message block
* add batch signature verification failure tests

Before:

```
../env.sh nim c -d:release -r ncli_db --db:mainnet_0/db bench --start-slot:-1000
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    5830.675,        0.000,     5830.675,     5830.675,            1, Initialize DB
       0.481,        1.878,        0.215,       59.167,          981, Load block from database
    8422.566,        0.000,     8422.566,     8422.566,            1, Load state from database
       6.996,        1.678,        0.042,       14.385,          969, Advance slot, non-epoch
      93.217,        8.318,       84.192,      122.209,           32, Advance slot, epoch
      20.513,       23.665,       11.510,      201.561,          981, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

After:

```
    7081.422,        0.000,     7081.422,     7081.422,            1, Initialize DB
       0.553,        2.122,        0.175,       66.692,          981, Load block from database
    5439.446,        0.000,     5439.446,     5439.446,            1, Load state from database
       6.829,        1.575,        0.043,       12.156,          969, Advance slot, non-epoch
      94.716,        2.749,       88.395,      100.026,           32, Advance slot, epoch
      11.636,       23.766,        4.889,      205.250,          981, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

* add comment
2021-11-24 13:43:50 +01:00
..
consensus_spec ncli_db: add putState, putBlock (#3096) 2021-11-18 13:02:43 +01:00
fork_choice proc -> func in ncli/, research/, and test/ (#2818) 2021-08-25 14:51:52 +00:00
fuzzing test fixture cleanups (#2976) 2021-10-12 13:36:52 +02:00
helpers Use nim-ssz-serialization module and rm local ssz code 2021-11-10 13:37:24 +02:00
mocking ncli_db: add putState, putBlock (#3096) 2021-11-18 13:02:43 +01:00
simulation /bin/bash -> /usr/bin/env bash (needed on NixOS) 2021-11-15 15:45:01 +02:00
slashing_protection test fixture cleanups (#2976) 2021-10-12 13:36:52 +02:00
spec_block_processing manually-verified v1.1.5 spec URL updates (#3068) 2021-11-09 08:54:59 +00:00
spec_epoch_processing clean up fork enum and field names 2021-10-19 11:06:38 +03:00
all_tests.nim ncli_db: add putState, putBlock (#3096) 2021-11-18 13:02:43 +01:00
all_tests.nim.cfg remove remaining traces of nim-prompt 2021-08-16 21:56:50 +03:00
test_action_tracker.nim register vc duties with subnet tracker (#2949) 2021-10-18 11:11:44 +02:00
test_attestation_pool.nim ncli_db: add putState, putBlock (#3096) 2021-11-18 13:02:43 +01:00
test_beacon_chain_db.nim ncli_db: add putState, putBlock (#3096) 2021-11-18 13:02:43 +01:00
test_block_pool.nim Speed up altair block processing 2x (#3115) 2021-11-24 13:43:50 +01:00
test_datatypes.nim remove false OnBlockAdded dependency on phase0 HashedBeaconState (#2661) 2021-06-21 08:35:24 +00:00
test_discovery.nim odds and ends (#3015) 2021-10-21 15:09:19 +02:00
test_eth1_monitor.nim Add support for HTTPS Web3 providers 2021-11-23 15:56:18 +02:00
test_eth2_ssz_serialization.nim remove `updateRoot` param from SSZ 2021-08-23 16:49:43 +03:00
test_exit_pool.nim Better REST/RPC error messages (#3046) 2021-11-05 17:39:47 +02:00
test_forks.nim Support starting from altair (#3054) 2021-11-10 13:39:08 +02:00
test_gossip_validation.nim Speed up altair block processing 2x (#3115) 2021-11-24 13:43:50 +01:00
test_helpers.nim clean up fork enum and field names 2021-10-19 11:06:38 +03:00
test_honest_validator.nim odds and ends (#3015) 2021-10-21 15:09:19 +02:00
test_interop.nim ncli_db: add putState, putBlock (#3096) 2021-11-18 13:02:43 +01:00
test_keystore.nim unexport various parts of tests/ and remove unused code (#2794) 2021-08-18 13:58:43 +00:00
test_merge_vectors.nim document how to run merge local testnet with Nethermind (#3110) 2021-11-17 20:45:39 +01:00
test_message_signatures.nim always use static keys in tests (#2928) 2021-09-30 17:14:03 +02:00
test_peer_pool.nim proc -> func in ncli/, research/, and test/ (#2818) 2021-08-25 14:51:52 +00:00
test_spec.nim ncli_db: add putState, putBlock (#3096) 2021-11-18 13:02:43 +01:00
test_statediff.nim update statediffs to work with Altair (#3061) 2021-11-09 16:17:57 +00:00
test_sync_committee_pool.nim Bugfix: Avoid the aggregation of duplicate signatures when creating sync committee contributions 2021-11-07 21:41:10 +02:00
test_sync_manager.nim clean up fork enum and field names 2021-10-19 11:06:38 +03:00
test_sync_manager.nim.cfg Split Eth2Processor in prep for batching (#2396) 2021-03-11 11:10:57 +01:00
test_zero_signature.nim update spec references from eth2.0-specs to consensus-specs and to v1.1.0-beta.2 (#2822) 2021-08-26 10:21:52 +02:00
testblockutil.nim ncli_db: add putState, putBlock (#3096) 2021-11-18 13:02:43 +01:00
testdbutil.nim Support starting from altair (#3054) 2021-11-10 13:39:08 +02:00
teststateutil.nim ncli_db: add putState, putBlock (#3096) 2021-11-18 13:02:43 +01:00
testutil.nim forkedbeaconstate_helpers -> forks (#2772) 2021-08-10 22:46:35 +02:00