status-im/nimbus-eth2
2
0
Fork 0
mirror of https://github.com/status-im/nimbus-eth2.git synced 2025-01-10 22:36:01 +00:00
Code Issues Projects Releases Wiki Activity
2,398 Commits 110 Branches 116 Tags 244 MiB
Commit Graph

44 Commits

Author SHA1 Message Date
tersec
6cf7e837ce
refactor exit pool and gossipsub-subscribe to exit pool message types (#1733) 
* refactor exit pool and gossipsub-subscribe to all exit pool message types

* remove unused loop counter
 2020-09-24 19:05:49 +02:00
tersec
3190c695b0
minimal v0.12.3 update (#1716) 2020-09-21 15:58:35 +00:00
Jacek Sieka
dcf8a6b05d
improve slot processing speeds (#1670) 
about 40% better slot processing times (with LTO enabled) - these don't
do BLS but are used
heavily during replay (state transition = slot + block transition)

tests using a recent medalla state and advancing it 1000 slots:

```
./ncli slots --preState2:state-302271-3c1dbf19-c1f944bf.ssz --slot:1000
--postState2:xx.ssz
```
pre:

```

All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
39.236,        0.000,       39.236,       39.236,            1,
Load state from file
0.049,        0.002,        0.046,        0.063,          968,
Apply slot
256.504,       81.008,      213.471,      591.902,           32,
Apply epoch slot
28.597,        0.000,       28.597,       28.597,            1,
Save state to file
```

cast:
```
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
37.079,        0.000,       37.079,       37.079,            1,
Load state from file
0.042,        0.002,        0.040,        0.090,          968,
Apply slot
215.552,       68.763,      180.155,      500.103,           32,
Apply epoch slot
25.106,        0.000,       25.106,       25.106,            1,
Save state to file
```

cast+rewards:
```
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
40.049,        0.000,       40.049,       40.049,            1,
Load state from file
0.048,        0.001,        0.045,        0.060,          968,
Apply slot
164.981,       76.273,      142.099,      477.868,           32,
Apply epoch slot
28.498,        0.000,       28.498,       28.498,            1,
Save state to file
```

cast+rewards+shr
```
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
12.898,        0.000,       12.898,       12.898,            1,
Load state from file
0.039,        0.002,        0.038,        0.054,          968,
Apply slot
139.971,       68.797,      120.088,      428.844,           32,
Apply epoch slot
24.761,        0.000,       24.761,       24.761,            1,
Save state to file

```
 2020-09-16 20:59:33 +00:00
Jacek Sieka
fa1621db46
implement clock disparity for attestation validation (#1568) 
This implements disparity, resolving a part of
https://github.com/status-im/nim-beacon-chain/issues/1367

* make BeaconTime a duration for fractional seconds
* factor out attestation/aggregate validation
* simplify recording of queued attestations
* simplify attestation signature check
* fix blocks_received metric
* add some trivial validation tests
* remove unresolved attestation table - attestations for unknown blocks
are dropped instead (cannot verify their signature)
 2020-08-27 09:34:12 +02:00
Jacek Sieka
46c94a18ba rework epoch cache referencing 
* collect all epochrefs in specific blocks to make them easier to find
and to avoid lots of small seqs
* reuse validator key databases more aggressively by comparing keys
* make state cache available from within `withState`
* make epochRef available from within onBlockAdded callback
* integrate getEpochInfo into block resolution and epoch ref logic such
that epochrefs are created when blocks are added to pool or lazily when
needed by a getEpochRef
* fill state cache better from EpochRef, speeding up replay and
validation
* store epochRef in specific blocks to make them easier to find and
reuse
* fix database corruption when state is saved while replaying quarantine
* replay slots fully from block pool before processing state
* compare bls values more smartly
* store epoch state without block applied in database - it's recommended
to resync the node!

this branch will drastically speed up processing in times of long
non-finality, as well as cut memory usage by 10x during the recent
medalla madness.
 2020-08-19 10:09:06 +03:00
Dustin Brody
3d121d9734 remove quadratic deposit Merkle tree initialization 2020-08-14 12:33:58 +03:00
tersec
a979568d74
switch mapIt/find to in-place linear search (#1436) 2020-08-04 17:15:27 +00:00
tersec
df80071bcf
update attestation and block validation to v0.12.2; clean up getAncestorAt()/get_ancestor() (#1417) 
* update attestation validation to v0.12.2; clean up getAncestorAt()/get_ancestor()

* update beacon block validation to v0.12.2
 2020-08-03 19:47:42 +00:00
tersec
5b13d8f3fa
synchronize test summaries with make test (and do a few small cleanups) (#1415) 2020-08-03 12:07:25 +00:00
Viktor Kirilov
c032366547
removed the BlockPool type and all of the proxy functions around it (#1401) 
* removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type

* fixed the build after the rebase
 2020-07-30 21:18:17 +02:00
tersec
20a2525390
v0.12.2 beacon chain protocol update (#1378) 2020-07-27 12:59:57 +02:00
zah
0be77f9cbc
Restore compilation with custom presets (#1309) 
This also assigns precise types to the constants in the minimal
and mainnet presets in order to reduce the chance of compilation
errors when custom presets are used (previously, only the custom
presets have precisely assigned types for the constants).
 2020-07-13 16:44:58 +02:00
Zahary Karadjov
3ec6a02b12
Merge devel and resolve conflicts 2020-07-10 02:02:40 +03:00
tersec
61b0b5af17
update most remaining non-fork-choice spec refs, updating code where necessary (#1292) 
* update most of the remaining non-fork-choice spec refs, updating code where necessary

* revert presumably harmless compute_signing_root() change, but this way, keep things really unchanged outside inspector
 2020-07-09 11:43:27 +00:00
Mamy Ratsimbazafy
3cdae9f6be
Dual headed fork choice [Revolution] (#1238) 
* Dual headed fork choice

* fix finalizedEpoch not moving

* reduce fork choice verbosity

* Add failing tests due to pruning

* Properly handle duplicate blocks in sync

* test_block_pool also add a test for duplicate blocks

* comments addressing review

* Fix fork choice v2, was missing integrating block proposed

* remove a spurious debug writeStackTrace

* update block_sim

* Use OrderedTable to ensure that we always load parents before children in fork choice

* Load the DAG data in fork choice at init if there is some (can sync witti)

* Cluster of quarantined blocks were not properly added to the fork choice

* Workaround async gcsafe warnings

* Update blockpoool tests

* Do the callback before clearing the quarantine

* Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads

* Make it work with latest devel - Altona readyness

* Add a recovery mechanism when forkchoice desyncs with blockpool

* add the current problematic node to the stack

* Fix rebase indentation bug (but still producing invalid block)

* Fix cache at epoch boundaries and lateBlock addition
 2020-07-09 11:29:32 +02:00
Zahary Karadjov
318b225ccd
Merge devel and resolve the conflicts 2020-07-08 15:36:03 +03:00
Zahary Karadjov
c4af4e2f35
Working test suite with run-time presets 2020-07-08 02:02:14 +03:00
Jacek Sieka
f3e92762e3
add tests for unviable blocks (#1271) 
* add tests for unviable blocks

also enable finalization tests in all test configs - they're plenty fast
now
also fix newClone for non-rvo cases. sigh.

* fixes
 2020-07-01 19:00:14 +02:00
Mamy Ratsimbazafy
902093f57c
Revert "Dual headed fork choice [Reloaded] (#1223)" (#1234) 
This reverts commit 6836d41ebda0ac2181adfd82489f67f968a44aa3.
 2020-06-25 11:36:03 +02:00
Mamy Ratsimbazafy
6836d41ebd
Dual headed fork choice [Reloaded] (#1223) 
* Dual headed fork choice

* fix finalizedEpoch not moving

* reduce fork choice verbosity

* Add failing tests due to pruning

* Properly handle duplicate blocks in sync

* test_block_pool also add a test for duplicate blocks

* comments addressing review
 2020-06-24 20:24:36 +02:00
Dustin Brody
8f7541959a allow compile-time switching between v0.11.3 and v0.12.1 2020-06-09 18:39:05 +00:00
Dustin Brody
cc13153fc6 initial 0.12.1 beacon chain spec 2020-06-09 18:39:05 +00:00
Jacek Sieka
56ffb696be
reorder ssz (#1099) 
* reorder ssz

* split into hash_trees and ssz_serialization, roughly, for hashing and
IO
* move bitseqs into ssz (from stew)
* clean up imports

* docs, imports
 2020-06-03 15:52:02 +02:00
Jacek Sieka
07393c8de6
Ssz cache (#1084) 
* wip: cache

* cache lists and arrays of complex objects (5x block processing speed
on ncli_db)

trivial baseline cache that stores tree in flat memory structure

* support array of uint64

* work around type issues

* more type compiler bug workarounds

* cache balances, more type fixes

* index type

* ncli_db: add validation flag, better ux

* int64 fixes

* test fix

* "oops"

```
647.913,        0.000,      647.913,      647.913,            1,
Initialize DB
0.540,        0.402,        0.340,        9.451,          619,
Load block from database
40.268,        0.000,       40.268,       40.268,            1,
Load state from database
0.498,        0.150,        0.343,        0.930,          596,
Apply block
3.548,       11.005,        0.729,       54.022,           23,
Apply epoch block
```

* support all basic types

* cleanups

* a few more cleanups
 2020-06-03 07:57:17 +02:00
Jacek Sieka
872d7ff493
support all basic types 2020-06-02 11:44:51 +02:00
Dustin Brody
1a6b83a15f cleanups 2020-06-01 19:45:21 +00:00
Jacek Sieka
f1dcee2749
wip: cache 
* cache lists and arrays of complex objects (5x block processing speed
on ncli_db)

trivial baseline cache that stores tree in flat memory structure
 2020-06-01 14:52:50 +02:00
Zed
edfd3eb9ae Run tests 2020-05-29 16:38:55 +03:00
tersec
80b8416305
skip skipped constant sanity checks also from markdown reports' perspectives (#1053) 
* update more spec refs in beacon_chain/spec/presets; skip skipped constant sanity checks also from markdown reports' perspectives

* mark skipped as skipped in markdown
 2020-05-21 19:56:09 +02:00
tersec
8432932c11
update to spec v0.11.2 to keep compatible with Schlesi (#1036) 
* update to spec v0.11.2 to keep compatible with Schlesi

* update spec/state_transition_epoch spec references to v0.11.2

* bump other spec refs
 2020-05-18 23:05:27 +00:00
Dustin Brody
d6c76f6b2f re-enable all but the finalization tests for block pool in mainnet presets 2020-05-14 16:33:11 +00:00
Dustin Brody
57519bebac remove some unused imports, add tests for pubsub topics, and subscribe to interop attestations 2020-05-14 16:41:46 +03:00
tersec
465553f921
re-enable mainnet attestation pool tests (#946) 
* re-enable mainnet attestation pool tests; since https://github.com/status-im/nim-beacon-chain/pull/930 double CI state sim validators

* bump deposits spec ref to v0.11.1
 2020-04-29 13:44:07 +02:00
tersec
7bc18423fa
detect improper 0-length lists of variable-sized objects in SSZ reading (#928) 
* detect improper 0-length lists of variable-sized objects in SSZ reading
 2020-04-23 19:39:23 +00:00
Ștefan Talpalaru
2f74951c04
bump submodules (#912) 2020-04-21 16:54:32 +02:00
Ștefan Talpalaru
b7a32a17ba
bump submodules 
and remove failing syncManagerGroupRecoveryTest
 2020-04-14 18:21:56 +02:00
Mamy Ratsimbazafy
cbc998ed93
[Ready 1/2] Fork choice rewrite (#865) 
* initial fork-choice refactor

* Add fork_choice test for "no votes"

* Initial test with voting: fix handling of unknown validators and parent blocks

* Fix tiebreak of votes

* Cleanup debugging traces

* Complexify the vote test

* fakeHash use the bigEndian repr of number + fix tiebreak for good

* Stash changes: found critical bug in nimcrypto `==` and var openarray

* Passing fork choice tests with varying votes

* Add FFG fork choice scenario + fork choice to the test suite

* Not sure why lmdb / rocksdb reappeared in rebase

* Add sanity checks to .nimble file + integrate fork choice tests to the test DB and test timing

* Cleanup debugging echos

* nimcrypto fix https://github.com/status-im/nim-beacon-chain/pull/864 as been merged, remove TODO comment

* Turn fork choice exception-free

* Cleanup "result" to ensure early return is properly used

* Add a comment on private/public error code vs Result

* result -> results following https://github.com/status-im/nim-beacon-chain/pull/866

* Address comments:
- raises: [Defect] doesn't work -> TODO
- process_attestation cannot fail
- try/except as expression pending Nim v1.2.0
- cleanup TODOs

* re-enable all sanity checks

* tag no raise for process_attestation

* use raises defect everywhere in fork choice and fix process_attestation test
 2020-04-09 18:15:00 +02:00
Ștefan Talpalaru
96431bfd86
post-merge fixes 2020-04-08 23:09:52 +02:00
Dustin Brody
2771deadfc re-add test_interop to all_tests and mark several v0.10.1 phase 0 spec references as v0.11.1 2020-04-07 13:16:55 +03:00
Dustin Brody
e7de8aae20 fix/restore 0.11.1 BeaconState genesis interop: (a) use GENESIS_FORK_VERSION properly in BeaconState init; (b) use GENESIS_FORK_VERSION to be compatible with zcli/zrnt initial eth1 deposits; (c) let List[foo] work with 64-bit ints, so BeaconState.validators SSZ-serializes properly; and (d) update test_interop to use new spec/crypto API 2020-04-07 13:16:55 +03:00
tersec
f5f939bd31
0.11.1 beacon chain spec update (#836) 
* initial 0.11.1 spec commit; no test regressions and finalizes in eth2_network_simulation

* with BLS 0.10/0.11 available, stop skipping attester slashing, proposer slashing, and voluntary exist operations fixture tests

* switch param orders to group state.{fork, genesis_validators_root}; bump spec/datatypes spec version for network purposes

* mark attestation construction and broadcast and some minimal/mainnet constants as 0.11.1-compatible; remove phase 1 sharding constants from minimal which don't exist in that preset
 2020-03-30 11:31:44 +00:00
Dustin Brody
6068dd960b initial 0.11.0 spec version update 2020-03-17 12:06:56 +02:00
Dustin Brody
794b374cf3 update test results summary after BLS 0.10.1 update 2020-03-13 15:17:55 +00:00
Zed
4a0893becb Add test reports 2020-03-13 14:38:59 +00:00