status-im/nimbus-eth2
2
0
Fork 0
mirror of https://github.com/status-im/nimbus-eth2.git synced 2025-01-10 06:16:25 +00:00
Code Issues Projects Releases Wiki Activity
2,474 Commits 109 Branches 116 Tags 244 MiB
Commit Graph

30 Commits

Author SHA1 Message Date
Jacek Sieka
61538fa581 speed up shuffling 
Replace shuffling function with zrnt version - `get_shuffled_seq` in
particular puts more strain on the GC by allocating superfluous seq's
which turns out to have a significant impact on block processing (when
replaying blocks for example) - 4x improvement on non-epoch, 1.5x on
epoch blocks (replay is done without signature checking)

Medalla, first 10k slots - pre:

```
Loaded 68973 blocks, head slot 117077
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
76855.848,        0.000,    76855.848,    76855.848,            1,
Initialize DB
1.073,        0.914,        0.071,       12.454,         7831,
Load block from database
31.382,        0.000,       31.382,       31.382,            1,
Load state from database
85.644,       30.350,        3.056,      466.136,         7519,
Apply block
506.569,       91.129,      130.654,      874.786,          312,
Apply epoch block
```

post:

```
Loaded 68973 blocks, head slot 117077
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
72457.303,        0.000,    72457.303,    72457.303,            1,
Initialize DB
1.015,        0.858,        0.070,       11.231,         7831,
Load block from database
28.983,        0.000,       28.983,       28.983,            1,
Load state from database
21.725,       17.461,        2.659,      393.217,         7519,
Apply block
324.012,       33.954,       45.452,      440.532,          312,
Apply epoch block
```
 2020-08-21 16:05:10 +03:00
tersec
611c5097cc
use cache in process_voluntary_exit() (#1507) 2020-08-14 12:42:59 +00:00
tersec
90708a8287
Prefer converting int` to uint64 and switch foo.len.uint64 to .len64 (#1375) 
* avoid converting from uint64 to int, and where most feasible, int type conversion at all

* .len.uint64 -> .len64

* fix 32-bit compilation

* try keeping state_sim loop variable/bounds as int for 32-bit Azure

* len64 -> lenu64
 2020-07-26 20:55:48 +02:00
tersec
26e893ffc2
restore EpochRef and flush statecaches on epoch transitions (#1312) 
* restore EpochRef and flush statecaches on epoch transitions

* more targeted cache invalidation

* remove get_empty_per_epoch_cache(); implement simpler but still faster get_beacon_proposer_index()/compute_proposer_index() approach; add some abstraction layer for accessing the shuffled validator indices cache

* reduce integer type conversions

* remove most of rest of integer type conversion in compute_proposer_index()
 2020-07-15 12:44:18 +02:00
Zahary Karadjov
318b225ccd
Merge devel and resolve the conflicts 2020-07-08 15:36:03 +03:00
Zahary Karadjov
c4af4e2f35
Working test suite with run-time presets 2020-07-08 02:02:14 +03:00
tersec
a92276d510
adopt Result[void, string] in place of some bool return signatures (#1275) 
* adopt Result[void, string] in place of some bool return signatures

* string -> cstring to reduce memory allocations; ensure all err() strings are constants, with contextual information from higher-level callers

* logScope usage fixes

* homogenize err() reporting convention

* invalid signature in deposit isn't an error
 2020-07-03 17:03:14 +00:00
Jacek Sieka
1301600341
Trusted blocks (#1227) 
* cleanups

* fix ncli state root check flag
* add block dump to ncli_db
* limit ncli_db benchmark length
* tone down finalization logs

* introduce trusted blocks

We only store blocks whose signature we've verified in the database - as
such, there's no need to check it again, and most importantly, no need
to deserialize the signature when loading from database.

50x startup time improvement, 200x block load time improvement.

* fix rewinding when deposits have invalid signature
* speed up ancestor iteration by avoiding copy
* avoid deserializing signatures for trusted data
* load blocks lazily when rewinding (less memory used)

* chronicles workarounds

* document trustedbeaconblock
 2020-06-25 12:23:10 +02:00
tersec
807b920c19
state_transition implements the spec fairly directly (#1220) 2020-06-23 13:54:24 +00:00
Jacek Sieka
56ffb696be
reorder ssz (#1099) 
* reorder ssz

* split into hash_trees and ssz_serialization, roughly, for hashing and
IO
* move bitseqs into ssz (from stew)
* clean up imports

* docs, imports
 2020-06-03 15:52:02 +02:00
tersec
cf8e90615a
More state_transition unification (#953) 
* remove incorrect/obsolete comment; deprecate BeaconState state transition functions

* remove deprecated state_transition(state: var BeaconState)

* add specific workarounds for state_transition() and process_slots() to nfuzz_block() and addTestBlock()
 2020-04-30 18:27:17 +02:00
Jacek Sieka
c74ba5c0c6
ssz: move ref support outside (#943) 
* ssz: move ref support outside

Instead of allocating ref's inside SSZ, move it to separate helper:

* makes `ref` allocations explicit
* less magic inside SSZ
* `ref` in nim generally means reference whereas SSZ was loading as
value - if a type indeed used references it would get copies instead of
references to a single value on roundtrip which is unexpected

TODO: EF tests would benefit from some refactoring since they all do the
same thing practically..

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
 2020-04-29 22:12:07 +02:00
Jacek Sieka
03a147ab8d
avoid state copy in state transition (#930) 
In BlockPool, we keep the head state around, so it's trivial to restore
the temporary state there and keep going as if nothing happened.

This solves 3 problems:
* stack space - the state copy on mainnet is huge
* GC scanning - using stack space for state slows down the GC
significantly
* reckless copying - the copy itself takes a long time

In state_sim, we'll do the same and allocate on heap - this helps a
little with GC - without it, the collection of the temporary strings
created with `toHex` while printing the json dominates the trace.
 2020-04-26 21:13:33 +02:00
Zahary Karadjov
4e9fa51ae9 Introduce BeaconNodeRef and use it in all the right places 2020-04-26 13:04:53 +03:00
Zahary Karadjov
740b76d152 Spec-compliant implementation of Eth1 monitoring; Eth1-enabled local sim 
BEWARE! This commit will trigger a stack overflow during local sim
 2020-04-26 13:04:53 +03:00
Jacek Sieka
7902d070cd prefer endians2 2020-03-05 18:13:28 +02:00
Jacek Sieka
9a3db7a81f
[WIP] Fake bls at runtime (#735) 
* Initial implementation of runtime bls skipping.

Add libnfuzz skipBLSValidation handling, check that it propagates.

* Rename skipBLSValidation -> skipBlsValidation, start skipStateRootValidation
* Replace skipValidation flags with more granular flags.

Also added skipBlockParentRootValidation flag
Mainly replaced with skipBlsValidation but also StateRoot or
BlockParentRootValidation flags where appropriate.

* Adjust interop test to pass when skipping merkle validation.
* Stop skipping validation for mainchain_monitor.
* Remove comment.
* Also skipMerkleValidation for test_beacon_chain_db.
 2020-03-05 13:52:10 +01:00
Nathaniel Jensen
4f6f2f28f3
Implement additional state transition fuzzing harnesses. 2020-01-28 20:51:06 +02:00
Nathaniel Jensen
9aeef775ae Format code. 2020-01-09 17:44:41 +02:00
Nathaniel Jensen
a37fac864f Update exception handling based on discussions. 
ValueErrors identified as associated with logging and are classed as a
failure rather than expected behavior.
 2020-01-09 17:44:41 +02:00
Nathaniel Jensen
e356ae0405 Remove skipValidation for block, block_header handlers. 
As differences are being identified due to it skipping block parent root
validation.
 2020-01-08 13:46:14 +02:00
Nathaniel Jensen
e795234504 Nfuzz to set runtime skipValidation flag where possible. 
Note: this does more than just disable bls - see #407,
so is not strictly consistent with other `beacon-fuzz`
implementations.
 2020-01-08 13:46:14 +02:00
Nathaniel Jensen
a08db4b311 Implement nfuzz_block_header nfuzz_attester_slashing harnesses. 
Add notes where not certain whether the error should crash or return
false.

Update header.
 2020-01-08 13:46:14 +02:00
Nathaniel Jensen
5978f09261 Allow defects and assertions to propagate in fuzzing harnesses. 
Add some slight libnfuzz readme notes.

Adjust exception tagging appropriately.
 2020-01-08 13:46:14 +02:00
Nathaniel Jensen
0d764d87af Raise exception when failing deserialization instead of returning false. 
Fuzzer preprocessing should provide valid ssz.
 2020-01-08 13:46:14 +02:00
kdeme
5599c76d46
Update README.md with const_preset option [skip ci] 2019-12-05 10:07:07 +01:00
kdeme
c4aebe423b
Update README.md with bls verification disable [skip ci] 2019-12-03 16:04:02 +01:00
kdeme
41841c7133 Add libnfuzz documentation [skip ci] 2019-12-03 15:08:03 +02:00
kdeme
5a51676a18 Address review comments 2019-12-03 14:33:47 +02:00
kdeme
3b3a2b10f1 First attempt to have some fuzz tests exported for the fuzzing framework 2019-12-03 14:33:47 +02:00