status-im/nimbus-eth2
2
0
Fork 0
mirror of https://github.com/status-im/nimbus-eth2.git synced 2025-01-10 06:16:25 +00:00
Code Issues Projects Releases Wiki Activity
3,166 Commits 109 Branches 116 Tags 244 MiB
Commit Graph

3166 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
tersec
1bdbf099cc
use IntSet rather than HashSet[ValidatorIndex] (#2267) 
* use IntSet rather than HashSet[ValidatorIndex]

* add bounds check before uint64 -> int conversion

* use intsets in block transitions

* remove superfluous Nim issue explanation/reference
 2021-01-26 12:52:00 +01:00
Dmitriy Ryajov
64e4d74f87
pass max peers to libp2p (#2265) 2021-01-26 10:35:22 +01:00
Mamy Ratsimbazafy
70a03658e3
Block validation flow v2 + Batch (serial) sig verification (#2250) 
* bump nim-blscurve

* Outline the block validation flow

* introduce the SigVerified types, pass the tests

* Split clearance/quarantine to prepare for batch crypto verif

* Add a batch signature collector

* Make clearance use SigVerified block and split verification between crypto and state transition

* Always use signedBeaconBlock for the onBlockAdded callback

* RANDAO signing_root is the epoch instead of the full block

* Support skipping BLS for testing

* Fix compilation of the validator client

* Try to fix strange errors MacOS and Jenkins (Clang, unknown type name br_hmac_drbg_context in stdlib_assertions.nim.c)

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561819858

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828025

* onBlockAdded callback should use TrustedSignedBeaconBlock https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561837261

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828946

* Use the application RNG: https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561815336

* Improve codegen of conversion zero-cost)

* Quick fixes with loadWithCache after #2259 (TODO: graceful error since pubkey validations is now done first in signatures_batch)

* Graceful handle rogue pubkeys and signatures now that those are lazy-loaded
 2021-01-25 20:45:48 +02:00
Jacek Sieka
5ca10d3de2 avoid copying keys for trusted signatures 
this removes `is_valid_indexed_attestion` from performance benchmarks
when replaying blocks
 2021-01-25 19:48:02 +02:00
Jacek Sieka
8f6ce4f88d shorten idle timeout 
this introduces the change more gently, to avoid adverse effects - the
timeout value can subsequently be tuned based on further
experimentation.
 2021-01-25 19:44:05 +02:00
tersec
7d74d3bfbc
only subscribe to subnets when aggregating (#2254) 
Only subscribe to subnets when aggregating
 2021-01-25 19:39:56 +02:00
tersec
8c48d44788
remove superfluous/overly aggressive, timing-dependent assertion (#2264) 2021-01-25 17:52:55 +01:00
Jacek Sieka
5713a3ce4c
performance fixes (#2259) 
* performance fixes

* don't mark tree cache as dirty on read-only List accesses
* store only blob in memory for keys and signatures, parse blob lazily
* compare public keys by blob instead of parsing / converting to raw
* compare Eth2Digest using non-constant-time comparison
* avoid some unnecessary validator copying

This branch will in particular speed up deposit processing which has
been slowing down block replay.

Pre (mainnet, 1600 blocks):

```
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    3450.269,        0.000,     3450.269,     3450.269,            1, Initialize DB
       0.417,        0.822,        0.036,       21.098,         1400, Load block from database
      16.521,        0.000,       16.521,       16.521,            1, Load state from database
      27.906,       50.846,        8.104,     1507.633,         1350, Apply block
      52.617,       37.029,       20.640,      135.938,           50, Apply epoch block
```

Post:

```
    3502.715,        0.000,     3502.715,     3502.715,            1, Initialize DB
       0.080,        0.560,        0.035,       21.015,         1400, Load block from database
      17.595,        0.000,       17.595,       17.595,            1, Load state from database
      15.706,       11.028,        8.300,      107.537,         1350, Apply block
      33.217,       12.622,       17.331,       60.580,           50, Apply epoch block
```

* more perf fixes

* load EpochRef cache into StateCache more aggressively
* point out security concern with public key cache
* reuse proposer index from state when processing block
* avoid genericAssign in a few more places
* don't parse key when signature is unparseable
* fix `==` overload for Eth2Digest
* preallocate validator list when getting active validators
* speed up proposer index calculation a little bit
* reuse cache when replaying blocks in ncli_db
* avoid a few more copying loops

```
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    3279.158,        0.000,     3279.158,     3279.158,            1, Initialize DB
       0.072,        0.357,        0.035,       13.400,         1400, Load block from database
      17.295,        0.000,       17.295,       17.295,            1, Load state from database
       5.918,        9.896,        0.198,       98.028,         1350, Apply block
      15.888,       10.951,        7.902,       39.535,           50, Apply epoch block
       0.000,        0.000,        0.000,        0.000,            0, Database block store
```

* clear full balance cache before processing rewards and penalties

```
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    3947.901,        0.000,     3947.901,     3947.901,            1, Initialize DB
       0.124,        0.506,        0.026,      202.370,       363345, Load block from database
      97.614,        0.000,       97.614,       97.614,            1, Load state from database
       0.186,        0.188,        0.012,       99.561,       357262, Advance slot, non-epoch
      14.161,        5.966,        1.099,      395.511,        11524, Advance slot, epoch
       1.372,        4.170,        0.017,      276.401,       363345, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database block store
```
 2021-01-25 13:04:18 +01:00
Jakub
095b191aee
add cname file to docs to fix domain issues (#2260) 
mdbook has an option called `cname`:
https://rust-lang.github.io/mdBook/format/config.html#html-renderer-options

But it appears to be only included starting from `0.4.3`:
https://github.com/rust-lang/mdBook/blob/master/CHANGELOG.md#mdbook-043

Which means this will be more robust, if uglier.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2021-01-22 17:57:55 +01:00
yslcrypto
173a2fe018 update intro and metrics tutorial 2021-01-22 16:53:49 +01:00
tersec
7f5a26002d
remove some skipBlsValidation usage/handling in tests (#2258) 2021-01-22 14:29:04 +01:00
Zahary Karadjov
7571e74dbd Fix #2252 (consistent typing for the 'delay' log property) 2021-01-22 15:09:21 +02:00
nbc-bump-bot[bot]
f0f292f509
auto-bump nim-libp2p (#2245) 
Co-authored-by: = <dryajov@users.noreply.github.com>
 2021-01-22 08:43:22 +01:00
Zahary Karadjov
960666d1ed
Remove std/random again 2021-01-21 19:39:04 +02:00
Mamy Ratsimbazafy
718feef802
Fix unstable after #2244 (#2255) 2021-01-21 18:27:24 +01:00
cheatfate
69d6ffd990 Remove watchTask(). 2021-01-21 16:22:01 +02:00
Ștefan Talpalaru
9a0cac61c8 only ignore untracked files in submodules 2021-01-21 16:09:28 +02:00
Dustin Brody
a16f5afcd5 pre-emptive duplicate validator detection heuristic 2021-01-21 16:03:02 +02:00
tersec
55ecb61c3a cycle attestation subnets every slot (#2240) 
Cycle attestation subnets every slot
 2021-01-19 19:44:03 +02:00
tersec
921fe5a68f
initial infrastructure for state diffs (#2087) 
Initial infrastructure for state diffs
 2021-01-18 22:34:41 +02:00
Jacek Sieka
c0e11cfba6 shorten idle timeout 2021-01-18 22:27:34 +02:00
Jacek Sieka
76f15302a7 better timesharing in eth2 processor 
* use `idleAsync` to more evenly divide cpu attention when syncing in
particular - this gives networking better latency
* more strict exception handling in eth2_processor
 2021-01-18 22:27:34 +02:00
Kim De Mey
6fabefa76f
Fix discv5 loop in case of no nodes (#2243) 
* Fix discovery loop in case of no peers in routing table

* local testnet: Stop searching when amount of peers in testnet is reached
 2021-01-18 14:13:26 +01:00
tersec
f1bb8a6a1a
don't use logtrace by default in CI (#2241) 2021-01-18 11:42:41 +01:00
tersec
0fce8ad0d7
Revert "only checkpoint every four slots (#2236)" (#2242) 
This reverts commit 7da16f49086fc21ce10766eb0cc00decc9d0fd81.
 2021-01-18 11:02:56 +01:00
Zahary Karadjov
c8c819359c
More clear error message when a validator exit was rejected 2021-01-15 19:40:05 +02:00
nbc-bump-bot[bot]
a0689e4237
auto-bump nim-libp2p (#2239) 
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
 2021-01-15 21:15:58 +09:00
nbc-bump-bot[bot]
bb0a022968
auto-bump nim-libp2p (#2238) 
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
 2021-01-15 15:50:14 +09:00
tersec
7da16f4908
only checkpoint every four slots (#2236) 
* only checkpoint every four slots

* only checkpoint every 16 slots

* every 8 slots

* every 4 slots; 8 seems probably okay, but be a bit conservative
 2021-01-15 05:23:54 +00:00
Giovanni Petrantoni
295e3c9c73
Topics validation and direct peers (#2237) 
* pick the right libp2p branch

* add topics validation
 2021-01-15 04:17:06 +00:00
nbc-bump-bot[bot]
ce64da1fcd
auto-bump nim-libp2p (#2235) 
Co-authored-by: = <arnetheduck@users.noreply.github.com>
 2021-01-15 01:39:09 +00:00
tersec
fa75c477cd
only initially subscribe to relevant attestation subnets (#2231) 2021-01-14 09:43:21 +01:00
Kim De Mey
66d8f317cd
Use async queryRandom instead of inefficient randomNodes to discover nodes (#2211) 2021-01-14 08:58:13 +01:00
Sacha Saint-Leger
67cce36f06
fix metrics guide (#2234) 2021-01-13 14:21:00 +01:00
Sacha Saint-Leger
37c3c3877c
add troubleshooting section on time (#2232) 2021-01-13 11:41:47 +01:00
tersec
62908b0e91
remove backwards compatibility targets (#2223) 2021-01-12 18:00:30 +01:00
tersec
0fad1b6b26
don't special-case zero-validator subnet cycling (#2230) 2021-01-12 17:17:43 +01:00
tersec
dde973e2d4
allow always-on subscription to all attestation subnets when gossiping (#2225) 
* allow always-on subscription to all attestation subnets when gossiping

* in subscribe-all-subnets mode, consider all subnets to be stability subnets for ENR purposes
 2021-01-12 13:43:15 +01:00
nbc-bump-bot[bot]
f5efcc8669
auto-bump nim-libp2p (#2228) 
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
 2021-01-12 04:27:32 +01:00
Giovanni Petrantoni
a3a651b565
always enable topic and aggreate metric topics (#2229) 2021-01-12 04:27:09 +01:00
tersec
6653ce3fc5
don't cancel Jenkins builds in stable, testing, and unstable (#2226) 2021-01-11 19:51:38 +01:00
Kim De Mey
b99a6f3dfd
Warn on Eth1 chain not synced (#2221) 2021-01-11 10:23:09 +01:00
nbc-bump-bot[bot]
be3fac8495
auto-bump nim-libp2p (#2206) 
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
 2021-01-11 09:53:20 +01:00
Mamy Ratsimbazafy
6e2c0e3c17
and don't cancel testing either 2021-01-10 17:34:02 +01:00
Mamy Ratsimbazafy
b13751fb53
Don't cancel stable and unstable builds as well 2021-01-10 15:23:02 +01:00
Zahary Karadjov
87955f2d37
v1.0.6 v1.0.6 2021-01-10 12:23:07 +02:00
Ștefan Talpalaru
3112e98eb8 dist: bootstrap Nim compiler with default number of iterations 
Reducing those iterations results in a buggy compiler that causes
nimbus_beacon_node to fail at runtime with:
"FAT 2021-01-09 21:19:52.417+01:00 Could not obtain PeerID from network key   topics="networking" tid=26805 file=eth2_network.nim:1393"
 2021-01-10 12:09:31 +02:00
Zahary Karadjov
d803f008c0
v1.0.5 (corrected) 2021-01-10 12:02:55 +02:00
Zahary Karadjov
7d97228208
v1.0.5 v1.0.5 2021-01-09 01:14:20 +02:00
asymmetric
51f967a548
scripts: make shebang more portable (#2216) 
Some Linux distros (like NixOS) don't place bash in /bin/bash.
 2021-01-08 11:29:43 +01:00