status-im/nimbus-eth2
2
0
Fork 0
mirror of https://github.com/status-im/nimbus-eth2.git synced 2025-01-09 13:56:23 +00:00
Code Issues Projects Releases Wiki Activity
6,391 Commits 108 Branches 116 Tags 242 MiB
Commit Graph

468 Commits

Author SHA1 Message Date
cheatfate
80e79aef97 Add TODO comments for missing implementations. 
Change default REST port to use 5052 (Lighthouse).
Add missing checks for maximum amount of validator ids.
 2021-04-09 21:42:13 +03:00
cheatfate
2cf0d3d831 Simplification of POST/api/nimbus/v1/graffiti and GET/api/nimbus/v1/graffiti call implementation. 2021-04-09 21:42:13 +03:00
Zahary Karadjov
5266db3485 Add a POST /api/nimbus/v1/graffiti API 2021-04-09 21:42:13 +03:00
cheatfate
ec48e30ea8 Fix fork/schedule procedure comment. 2021-04-09 21:42:13 +03:00
cheatfate
a1303f3e50 Change json serialization framework. 
Make all calls which are possible.
 2021-04-09 21:42:13 +03:00
cheatfate
b7f36be73c Attempt to find proper readValue. 2021-04-09 21:42:13 +03:00
cheatfate
9de65fa293 Fixing issues after bump. 2021-04-09 21:42:13 +03:00
cheatfate
72695dd62a Rest API initial implementation. 2021-04-09 21:42:13 +03:00
tersec
d3cad92693
remove some BeaconState use and abstract over other uses (#2482) 
* remove some BeaconState use and abstract over other uses

* remove out-of-context comment
 2021-04-08 08:24:25 +00:00
Zahary Karadjov
1af07ad124
Simple RPC for modifying the Graffiti bytes 2021-03-26 19:42:20 +02:00
Jacek Sieka
74732a23fe
json cleanups (#2456) 
* move json-rpc specific marshalling to rpc
* serialize Epoch/Slot with cast to avoid Defect
* avoid a few eth1 deps
* simplify imports
 2021-03-26 15:11:06 +01:00
Jacek Sieka
2695cfa864
EH cleanup (#2455) 
almost 100% raises in nimbus-eth2 now!

* fix some rare exception-related crashes in json-rpc
 2021-03-26 07:52:01 +01:00
Jacek Sieka
3cb31e66b4
set upper bound on EpochRef cache (#2403) 
* set upper bound on EpochRef cache

* max 32 EpochRef instances
* less memory waste in BlockRef by removing EpochRef seq that is mostly
unused (~20mb)
* less memory waste in dag block lookup by not keeping an extra copy of
digest (~70mb)
* fix `==` and `$` for Eth2Digest
* remove `ChainDAG.tmpState` (~50mb?)

all in all, this branch cuts mainnet memory usage by ~160-180mb and puts
limits on EpochRef cache usage - where normally it hovered around 950mb
before, it's now sitting at 600-700mb on my machine.

* docs
 2021-03-17 11:17:15 +01:00
Giovanni Petrantoni
26c56c2800
more debug rpcs (#2385) 
* more debug rpcs

* cleanups

* oops fixing

* add wip change log notes about rpc renaming

* fix wrong breaking changes changelog location
 2021-03-16 08:39:29 +01:00
Joe Clapis
35d9291cfd
Validator APIs now return actual balance instead of effective balance (#2408) 2021-03-15 07:19:04 +00:00
Mamy Ratsimbazafy
de1060e7f3
centralize p2p validation in a single file and address https://github.com/status-im/nimbus-eth2/pull/2377#issuecomment-791313118 (#2383) 2021-03-06 08:32:55 +01:00
Mamy Ratsimbazafy
d47f53cd9d
Reorg (5/5) (#2377) 
* Reorg things left into networking and gossip_processing

* time -> beacon_clock

* fix builds
 2021-03-05 14:12:00 +01:00
Mamy Ratsimbazafy
5d7f9c3a04
Consensus object pools [reorg 4/5] (#2374) 
* Add documentation

* make test doesn't try to build the beacon node :/
 2021-03-04 10:13:44 +01:00
Mamy Ratsimbazafy
2f17ac7b64
Move SSZ, deposit_contracts & eth1_monitor [reorg files 3/5] (#2371) 
* move deposit_contract

* Move SSZ

* fix ssz import in tests

* move also eth1_monitor

* forgot to delete the original

* fix comma [skip ci]

* Fix "make" & tools imports

* Fix import

* Fix import again

* rename deposit_contract -> eth1

* Revert ssz move to subfolder

* path fixes [skip ci]
 2021-03-03 07:23:05 +01:00
Mamy Ratsimbazafy
3276dfc683
Consolidate modules by areas [part 1] (#2365) 
* Move sync in subfolder

* move validator related thingies in validators

* fix binary builds

* update bounds comment [skip ci]
 2021-03-02 11:27:45 +01:00
Mamy Ratsimbazafy
08f063aba9
[RFC - Doc] State of Nimbus block & attestation flows (#2351) 
* Expand documentation on block flow [skip ci]

* address review comments [skip ci]

* Update with GossipFlow out [skip ci]

* LocalBlockProposer -> LocalValidatorDuties +  WeakSubjectivitySync

* First outline of attestation flow

* finish up prose
 2021-03-01 11:22:16 +01:00
tersec
5cab17dc1a
database state storage benchmarking via ncli_db (#2312) 
* database state storage benchmarking via ncli_db

* more cleanups from immutable validator state branch

* unexport some eth2_network constants and remove unused variables/templates

* make two PeerScore constants public
 2021-02-15 17:40:00 +01:00
Jacek Sieka
9968944329
more callsigs! (#2302) 2021-02-09 10:23:26 +01:00
Jacek Sieka
5d3c995d42 json-rpc fixes 
* expose node signatures
* format bitseqs as hex strings
* format trusted sigs as hex strings (same as untrusted)
* reuse rpc client sigs
* include validator index in duties
* move SyncInfo to spec
 2021-02-02 14:55:36 +02:00
Zahary Karadjov
fa99c3b417 Fix #2261 
Also bumps Confutils to allow setting the hidden --web3-mode param
(to allow testing the eth1 syncing without validators)
 2021-01-30 01:32:20 +02:00
tersec
1bdbf099cc
use IntSet rather than HashSet[ValidatorIndex] (#2267) 
* use IntSet rather than HashSet[ValidatorIndex]

* add bounds check before uint64 -> int conversion

* use intsets in block transitions

* remove superfluous Nim issue explanation/reference
 2021-01-26 12:52:00 +01:00
Mamy Ratsimbazafy
70a03658e3
Block validation flow v2 + Batch (serial) sig verification (#2250) 
* bump nim-blscurve

* Outline the block validation flow

* introduce the SigVerified types, pass the tests

* Split clearance/quarantine to prepare for batch crypto verif

* Add a batch signature collector

* Make clearance use SigVerified block and split verification between crypto and state transition

* Always use signedBeaconBlock for the onBlockAdded callback

* RANDAO signing_root is the epoch instead of the full block

* Support skipping BLS for testing

* Fix compilation of the validator client

* Try to fix strange errors MacOS and Jenkins (Clang, unknown type name br_hmac_drbg_context in stdlib_assertions.nim.c)

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561819858

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828025

* onBlockAdded callback should use TrustedSignedBeaconBlock https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561837261

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828946

* Use the application RNG: https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561815336

* Improve codegen of conversion zero-cost)

* Quick fixes with loadWithCache after #2259 (TODO: graceful error since pubkey validations is now done first in signatures_batch)

* Graceful handle rogue pubkeys and signatures now that those are lazy-loaded
 2021-01-25 20:45:48 +02:00
Jacek Sieka
5713a3ce4c
performance fixes (#2259) 
* performance fixes

* don't mark tree cache as dirty on read-only List accesses
* store only blob in memory for keys and signatures, parse blob lazily
* compare public keys by blob instead of parsing / converting to raw
* compare Eth2Digest using non-constant-time comparison
* avoid some unnecessary validator copying

This branch will in particular speed up deposit processing which has
been slowing down block replay.

Pre (mainnet, 1600 blocks):

```
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    3450.269,        0.000,     3450.269,     3450.269,            1, Initialize DB
       0.417,        0.822,        0.036,       21.098,         1400, Load block from database
      16.521,        0.000,       16.521,       16.521,            1, Load state from database
      27.906,       50.846,        8.104,     1507.633,         1350, Apply block
      52.617,       37.029,       20.640,      135.938,           50, Apply epoch block
```

Post:

```
    3502.715,        0.000,     3502.715,     3502.715,            1, Initialize DB
       0.080,        0.560,        0.035,       21.015,         1400, Load block from database
      17.595,        0.000,       17.595,       17.595,            1, Load state from database
      15.706,       11.028,        8.300,      107.537,         1350, Apply block
      33.217,       12.622,       17.331,       60.580,           50, Apply epoch block
```

* more perf fixes

* load EpochRef cache into StateCache more aggressively
* point out security concern with public key cache
* reuse proposer index from state when processing block
* avoid genericAssign in a few more places
* don't parse key when signature is unparseable
* fix `==` overload for Eth2Digest
* preallocate validator list when getting active validators
* speed up proposer index calculation a little bit
* reuse cache when replaying blocks in ncli_db
* avoid a few more copying loops

```
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    3279.158,        0.000,     3279.158,     3279.158,            1, Initialize DB
       0.072,        0.357,        0.035,       13.400,         1400, Load block from database
      17.295,        0.000,       17.295,       17.295,            1, Load state from database
       5.918,        9.896,        0.198,       98.028,         1350, Apply block
      15.888,       10.951,        7.902,       39.535,           50, Apply epoch block
       0.000,        0.000,        0.000,        0.000,            0, Database block store
```

* clear full balance cache before processing rewards and penalties

```
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    3947.901,        0.000,     3947.901,     3947.901,            1, Initialize DB
       0.124,        0.506,        0.026,      202.370,       363345, Load block from database
      97.614,        0.000,       97.614,       97.614,            1, Load state from database
       0.186,        0.188,        0.012,       99.561,       357262, Advance slot, non-epoch
      14.161,        5.966,        1.099,      395.511,        11524, Advance slot, epoch
       1.372,        4.170,        0.017,      276.401,       363345, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database block store
```
 2021-01-25 13:04:18 +01:00
tersec
55ecb61c3a cycle attestation subnets every slot (#2240) 
Cycle attestation subnets every slot
 2021-01-19 19:44:03 +02:00
Zahary Karadjov
338428cbd7 Add Eth1 deposits simulation to block_sim 2021-01-04 13:22:00 +02:00
Giovanni Petrantoni
ed24f60f70
remove async from sub/unsub (#2197) 
* remove await/async from sub/unsub

* fix unsubscribe wrong key (missed _snappy)

* use the right libp2p commit hash

* remove unused async

* fix inspector

* fix subnet calculation in RPC and insert broadcast attestations into node's pool

* unify codepaths to ensure only mostly-checked-to-be-valid attestations enter the pool, even from node's own broadcasts

* update attestation pool tests for new validateAttestation param

Co-authored-by: Dustin Brody <tersec@users.noreply.github.com>
 2020-12-24 09:48:52 +01:00
tersec
97c4f7c5c0
fix subnet calculation in RPC and insert broadcast attestations into node's pool (#2207) 
* fix subnet calculation in RPC and insert broadcast attestations into node's pool

* unify codepaths to ensure only mostly-checked-to-be-valid attestations enter the pool, even from node's own broadcasts

* update attestation pool tests for new validateAttestation param
 2020-12-23 13:59:04 +01:00
tersec
afbaa36ef7
make subnet cycling more robust; use one stability subnet/validator; explicitly represent gossip enabled/disabled (#2201) 
* make subnet cycling more robust; use one stability subnet/validator; explicitly represent gossip enabled/disabled

* fix asymmetry in _snappy being used for subscriptions but not unsubscriptions

* remove redundant comment

* minimal RPC and VC support for infoming BN of subnets

* create and verify slot signatures in RPC interface and VC

* loosen old slot check

* because Slot + uint64 works but uint64 + Slot doesn't

* document assumptions for head state use; don't clear stability subnets; guard against VC not having checked an epoch ahead, fixing a crash; clarify unsigned comparison

* revert unsub fix
 2020-12-22 10:05:36 +01:00
Zahary Karadjov
8ebf9c30b0
More complete reset of the web3 provider on each failure; Fix #2184 2020-12-16 00:21:11 +02:00
Zahary Karadjov
983b3c9fbf Add a 'we3 test' command for verifying the compatibility of a web3 provider 2020-12-10 02:54:58 +02:00
Eugene Kabanov
46c2740097
Documentation for Validators API. (#2147) 
* Recover proper validator API call and remove incorrect one.
Add more examples to API documentation.
 2020-12-07 14:51:14 +02:00
Zahary Karadjov
7363bf3c7c
Fix a logical typo in beacon_api/validator.getStatus 2020-12-03 21:24:01 +02:00
cheatfate
42dd59852b Fix compilation problem. 2020-12-03 11:40:32 +02:00
cheatfate
0904ee5609 get_v1_validator_attestation_data() implementation. 2020-12-03 11:40:32 +02:00
cheatfate
f6285d8381 Fix proposeSignedBlock() become sync. 2020-12-03 11:40:08 +02:00
cheatfate
088b001388 Fix get_v1_beacon_states_stateId_committees_epoch() to use optional arguments properly. 
Optimize get_v1_beacon_pool_attestations() a bit.
 2020-12-03 11:40:08 +02:00
cheatfate
21ef0f3a57 Fixes for get_v1_beacon_states_stateId_validators() and get_v1_beacon_states_stateId_validators_validatorId(). 2020-12-03 11:40:08 +02:00
cheatfate
9afafb54f5 get_v1_beacon_states_stateId_validator_balances() implementation. 2020-12-03 11:40:08 +02:00
cheatfate
5dade7b441 post_v1_beacon_blocks() implementation. 2020-12-03 11:40:08 +02:00
cheatfate
5f392893c0 Change approach for get_v1_beacon_pool_attestations() to use hexadecimal strings. 2020-12-03 11:40:08 +02:00
cheatfate
9e7f8b37c7 get_v1_beacon_pool_attestations() implementation. 2020-12-03 11:40:08 +02:00
cheatfate
7e4ff7a740 Implementation of post_v1_beacon_pool_attester_slashings() and post_v1_beacon_pool_proposer_slashings(). 
Remove doAssert.
 2020-12-03 11:40:08 +02:00
cheatfate
6cae25701b Implementation of beacon_pool_attester_slashings(), beacon_pool_proposer_slashings() and beacon_pool_voluntary_exits(). 2020-12-03 11:40:08 +02:00
Zahary Karadjov
4eaf29875a Less risk for Eth1-induced delays in block proposal 2020-12-01 15:38:50 +02:00
Zahary Karadjov
3c0dfc2fbe Implement the 'deposits exit' command; Remove 'deposits create' 2020-11-29 23:08:07 +02:00