status-im/nimbus-eth2
2
0
Fork 0
mirror of https://github.com/status-im/nimbus-eth2.git synced 2025-02-14 07:26:25 +00:00
Code Issues Projects Releases Wiki Activity
3,058 Commits 115 Branches 118 Tags 142 MiB
Commit Graph

3058 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
tersec
bdd2ec03e3
fix slashing db interchange test compile warnings (#2311) 2021-02-11 13:54:54 +01:00
nbc-bump-bot[bot]
0419a4122e
auto-bump nim-libp2p (#2309) 
Co-authored-by: = <dryajov@users.noreply.github.com>
 2021-02-11 02:20:39 +00:00
Sacha Saint-Leger
855c3e849d
update intro and binary distribution edits (#2306) 
* update intro and binary distribution edits

* stefan's suggestions
 2021-02-10 17:29:48 +01:00
Jacek Sieka
f012d7060b
better error message on disk / database issues (#2307) 
bumps stew for better result defects as well
 2021-02-10 13:21:06 +01:00
nbc-bump-bot[bot]
c13205035e
auto-bump nim-libp2p (#2305) 
Co-authored-by: = <dryajov@users.noreply.github.com>
 2021-02-10 05:32:19 +00:00
Mamy Ratsimbazafy
03f47c8f2f
Slashing protection refactor - EIP 3076 (#2094) 
* Create CLI tool for slashing export

* Use SQLite as a DB instead of a KV-store

* Keeps v1 and v2 DBs around

* Uses the same schema as Lighthouse v1.1.0

* Passes all interchange tests + skeleton of finalization pruning

* Removes tests that would violate v5 / minimal slashing DB and MinSlot rules

* Migration tool added using low-watermark scheme for faster migration of large number of validators
 2021-02-09 17:23:06 +02:00
Jacek Sieka
9968944329
more callsigs! (#2302) 2021-02-09 10:23:26 +01:00
Giovanni Petrantoni
72b01161c1
populate gossipsub scores (#2091) 
* force pushing to fix unstable base

* increase attestation/aggregate queue sizes

when there are many validators, many aggregates and attestations arrive
every slot - increase the queue size a bit - also do batches on each
idle loop iteration since it's fairly quick

* don't score subnets for now

* wrapping up

* refactor and cleanups

* gossip parameters fixes

* comment fix

Co-authored-by: Jacek Sieka <jacek@status.im>
 2021-02-09 10:20:55 +01:00
Jacek Sieka
8a09286423
tune attestation params (#2301) 
* a little bit more priority to attestation processing
* better implementation for bytes_to_uint64
 2021-02-08 16:13:02 +01:00
yslcrypto
237453ec45 fix api error 2021-02-08 15:00:21 +01:00
Ștefan Talpalaru
979bb39b41 docs: binary distribution internals 2021-02-08 14:53:12 +02:00
nbc-bump-bot[bot]
93cfc2d446
auto-bump nim-libp2p (#2300) 
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
 2021-02-08 08:32:47 +01:00
tersec
8d25663681
remove several IntSet usages in lieu of seq[ValidatorIndex] (#2288) 
* remove several IntSet usages in lieu of seq[ValidatorIndex]

* convert smaller types to larger types

* larger type, again
 2021-02-08 08:27:30 +01:00
Jacek Sieka
ece50c4706
bump nim-json-rpc (fixes https://github.com/status-im/nim-json-rpc/issues/93) (#2299) 2021-02-07 22:30:45 +01:00
nbc-bump-bot[bot]
0b560cc6fb
auto-bump nim-libp2p (#2296) 
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
 2021-02-06 09:50:08 +01:00
Zahary Karadjov
596b8c6009
v1.0.7 v1.0.7 2021-02-04 17:28:54 +02:00
Zahary Karadjov
e0df027c11
Avoid a possible nil dereference 2021-02-04 17:24:34 +02:00
Zahary Karadjov
1394ef37fb
Tentative fix for the freezing Eth1 monitor activity 2021-02-04 17:01:47 +02:00
Ștefan Talpalaru
80c11546ff Windows binary release 
CI: use both cores on GitHub Actions and set timeouts for the local testnet tests
 2021-02-04 10:25:44 +02:00
Dustin Brody
67e4a045a3 simplify doppelganger detection to boolean 2021-02-03 20:55:33 +02:00
Eugene Kabanov
c4c04197c5
Bump nim-http-utils. (#2290) 2021-02-03 19:05:30 +01:00
Ștefan Talpalaru
cadb1aae15
bump submodules (#2279) 
* bump submodules

* undo a couple of bumps (32-bit failure)

* launch_local_testnet.sh: timeout and various fixes

* undo nim-http-utils bump (breaks RPC for VC)
 2021-02-02 19:02:52 +01:00
Giovanni Petrantoni
2cd627f21a
hotfix gossip scoring (#2286) 
* hotfix gossip scoring

* skip gossip scoring parameters validation as they violate for now (but does not matter cos we don't score)

* workaround again gossip validation
 2021-02-02 15:59:18 +01:00
Dustin Brody
707fcd99cb remove unused beacon chain spec and test code 2021-02-02 14:56:38 +02:00
Jacek Sieka
5d3c995d42 json-rpc fixes 
* expose node signatures
* format bitseqs as hex strings
* format trusted sigs as hex strings (same as untrusted)
* reuse rpc client sigs
* include validator index in duties
* move SyncInfo to spec
 2021-02-02 14:55:36 +02:00
Kim De Mey
73d9c2aa53
Add enr auto update cli option and bump nim-eth (#2278) 2021-02-02 09:07:21 +01:00
tersec
6141286547
rename doppelganger protection to doppelganger detection; switch default from warn to stop (#2281) 2021-02-01 12:18:16 +01:00
Zahary Karadjov
fa99c3b417 Fix #2261 
Also bumps Confutils to allow setting the hidden --web3-mode param
(to allow testing the eth1 syncing without validators)
 2021-01-30 01:32:20 +02:00
Dustin Brody
490e4c7666 don't used sorted() in get_indexed_attestation() 2021-01-30 00:23:02 +02:00
Dustin Brody
f7e327234f notice is too spammy for a situation-nominal message 2021-01-30 00:17:54 +02:00
Dustin Brody
281853eee8 rename options and internal structures to doppelgangerFoo and remove probing 2021-01-30 00:17:54 +02:00
Kim De Mey
a3678d6bf0
Bump eth2-testnets for mainnet nimbus bootstrap nodes (#2276) 2021-01-29 14:07:38 +01:00
nbc-bump-bot[bot]
3e89aa0f4f
auto-bump nim-libp2p (#2275) 
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
 2021-01-29 09:48:27 +01:00
Kim De Mey
40a5d44887
Fix selection of bootstrap nodes from metadata (#2273) 
Also removes again the doubling of max peers
 2021-01-29 08:56:02 +01:00
Dmitriy Ryajov
1117993a24
Libp2p connection limits (#2272) 
* max peers gets set as connection limits

* pass max connections
 2021-01-28 07:31:29 +01:00
yslcrypto
6528707e44 add local simulations to the for developers page 2021-01-27 20:38:16 +01:00
yslcrypto
fa07c0f0a1 link to quickstart guide in intro 2021-01-27 19:32:04 +01:00
yslcrypto
3016815860 update quickstart 2021-01-27 19:26:08 +01:00
tersec
4de9875cdf
Revert "pass max peers to libp2p (#2265)" (#2270) 
This reverts commit 64e4d74f87d8e1751078c1c925fb3305a0137ebf.
 2021-01-26 17:53:52 +01:00
Ștefan Talpalaru
20435880db CI: build and publish a Docker image for end-users 
Also disable the log file and log colours for distribution binaries, to
avoid duplicate logs.
 2021-01-26 18:52:59 +02:00
Zahary Karadjov
aa6e93a0cd Remove time.humaneStr and associated code 2021-01-26 18:46:33 +02:00
Jacek Sieka
43c64d32f8 increase attestation/aggregate queue sizes 
when there are many validators, many aggregates and attestations arrive
every slot - increase the queue size a bit - also do batches on each
idle loop iteration since it's fairly quick
 2021-01-26 14:29:47 +02:00
tersec
1bdbf099cc
use IntSet rather than HashSet[ValidatorIndex] (#2267) 
* use IntSet rather than HashSet[ValidatorIndex]

* add bounds check before uint64 -> int conversion

* use intsets in block transitions

* remove superfluous Nim issue explanation/reference
 2021-01-26 12:52:00 +01:00
Dmitriy Ryajov
64e4d74f87
pass max peers to libp2p (#2265) 2021-01-26 10:35:22 +01:00
Mamy Ratsimbazafy
70a03658e3
Block validation flow v2 + Batch (serial) sig verification (#2250) 
* bump nim-blscurve

* Outline the block validation flow

* introduce the SigVerified types, pass the tests

* Split clearance/quarantine to prepare for batch crypto verif

* Add a batch signature collector

* Make clearance use SigVerified block and split verification between crypto and state transition

* Always use signedBeaconBlock for the onBlockAdded callback

* RANDAO signing_root is the epoch instead of the full block

* Support skipping BLS for testing

* Fix compilation of the validator client

* Try to fix strange errors MacOS and Jenkins (Clang, unknown type name br_hmac_drbg_context in stdlib_assertions.nim.c)

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561819858

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828025

* onBlockAdded callback should use TrustedSignedBeaconBlock https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561837261

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828946

* Use the application RNG: https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561815336

* Improve codegen of conversion zero-cost)

* Quick fixes with loadWithCache after #2259 (TODO: graceful error since pubkey validations is now done first in signatures_batch)

* Graceful handle rogue pubkeys and signatures now that those are lazy-loaded
 2021-01-25 20:45:48 +02:00
Jacek Sieka
5ca10d3de2 avoid copying keys for trusted signatures 
this removes `is_valid_indexed_attestion` from performance benchmarks
when replaying blocks
 2021-01-25 19:48:02 +02:00
Jacek Sieka
8f6ce4f88d shorten idle timeout 
this introduces the change more gently, to avoid adverse effects - the
timeout value can subsequently be tuned based on further
experimentation.
 2021-01-25 19:44:05 +02:00
tersec
7d74d3bfbc
only subscribe to subnets when aggregating (#2254) 
Only subscribe to subnets when aggregating
 2021-01-25 19:39:56 +02:00
tersec
8c48d44788
remove superfluous/overly aggressive, timing-dependent assertion (#2264) 2021-01-25 17:52:55 +01:00
Jacek Sieka
5713a3ce4c
performance fixes (#2259) 
* performance fixes

* don't mark tree cache as dirty on read-only List accesses
* store only blob in memory for keys and signatures, parse blob lazily
* compare public keys by blob instead of parsing / converting to raw
* compare Eth2Digest using non-constant-time comparison
* avoid some unnecessary validator copying

This branch will in particular speed up deposit processing which has
been slowing down block replay.

Pre (mainnet, 1600 blocks):

```
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    3450.269,        0.000,     3450.269,     3450.269,            1, Initialize DB
       0.417,        0.822,        0.036,       21.098,         1400, Load block from database
      16.521,        0.000,       16.521,       16.521,            1, Load state from database
      27.906,       50.846,        8.104,     1507.633,         1350, Apply block
      52.617,       37.029,       20.640,      135.938,           50, Apply epoch block
```

Post:

```
    3502.715,        0.000,     3502.715,     3502.715,            1, Initialize DB
       0.080,        0.560,        0.035,       21.015,         1400, Load block from database
      17.595,        0.000,       17.595,       17.595,            1, Load state from database
      15.706,       11.028,        8.300,      107.537,         1350, Apply block
      33.217,       12.622,       17.331,       60.580,           50, Apply epoch block
```

* more perf fixes

* load EpochRef cache into StateCache more aggressively
* point out security concern with public key cache
* reuse proposer index from state when processing block
* avoid genericAssign in a few more places
* don't parse key when signature is unparseable
* fix `==` overload for Eth2Digest
* preallocate validator list when getting active validators
* speed up proposer index calculation a little bit
* reuse cache when replaying blocks in ncli_db
* avoid a few more copying loops

```
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    3279.158,        0.000,     3279.158,     3279.158,            1, Initialize DB
       0.072,        0.357,        0.035,       13.400,         1400, Load block from database
      17.295,        0.000,       17.295,       17.295,            1, Load state from database
       5.918,        9.896,        0.198,       98.028,         1350, Apply block
      15.888,       10.951,        7.902,       39.535,           50, Apply epoch block
       0.000,        0.000,        0.000,        0.000,            0, Database block store
```

* clear full balance cache before processing rewards and penalties

```
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    3947.901,        0.000,     3947.901,     3947.901,            1, Initialize DB
       0.124,        0.506,        0.026,      202.370,       363345, Load block from database
      97.614,        0.000,       97.614,       97.614,            1, Load state from database
       0.186,        0.188,        0.012,       99.561,       357262, Advance slot, non-epoch
      14.161,        5.966,        1.099,      395.511,        11524, Advance slot, epoch
       1.372,        4.170,        0.017,      276.401,       363345, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database block store
```
 2021-01-25 13:04:18 +01:00