status-im/nimbus-eth2
2
0
Fork 0
mirror of https://github.com/status-im/nimbus-eth2.git synced 2025-01-10 22:36:01 +00:00
Code Issues Projects Releases Wiki Activity
2,943 Commits 110 Branches 116 Tags 244 MiB
Commit Graph

81 Commits

Author SHA1 Message Date
Jacek Sieka
4e191a06ac use stew/leb128 
* 3 varint parsers -> 1
* 1/3 memory usage / allocations when writing stuff to libp2p
 2020-12-15 17:06:15 +02:00
Jacek Sieka
e7f2735271
fix broken metrics during replay (#2090) 
* move metrics out of state transition
* add validator count metric
* remove expensive beacon_current_validators, beacon_previous_validators
metrics (they should be reimplemented with cache), add cheap
beacon_active_validators to approximate
* remove unused validator count metrics
* tidy imports/defects
 2020-11-27 23:16:13 +01:00
Zahary Karadjov
b9e4fef616
Check that the selected data dir is compatible with the selected network 2020-11-25 03:13:58 +02:00
zah
372c9b798c
Fix the corrupted database state on Pyrmont nodes; Add mainnet genesis (#2056) 
* Handle some web3 timeouts better

* Add support for developer .env files

* Eth1 improvements; Mainnet genesis state

Notable changes:

* The deposits table have been removed from the database. The client
  will no longer process all deposits on start-up.

* The network metadata now includes a "state snapshot" of the deposit
  contract. This allows the client to skip syncing deposits made prior
  to the snapshot (i.e. genesis). Suitable metadata added for Pyrmont
  and Mainnet.

* The Eth1 monitor won't be started unless there are validators attached
  to the node.

* The genesis detection code is now optional and disabled by default

* Bugfix: The client should not produce blocks that will fail validation
  when it hasn't downloaded the latest deposits yet

* Bugfix: Work around the database corruption affecting Pyrmont nodes

* Remove metadata for Toledo and Medalla
 2020-11-24 22:21:47 +01:00
tersec
21c4ce8fd4
remove superfluous TODOs/not-really-TODOs, type conversion, imports (#2025) 2020-11-16 17:10:51 +01:00
Zahary Karadjov
b022dc4d1f Use O(n) algorithm in initialize_beacon_state_from_eth1; Avoid unnecessary merkle proofs generation 2020-11-15 21:40:40 +02:00
Zahary Karadjov
9f28a464fb
[skip ci] Update the version number 2020-11-14 23:26:46 +02:00
Zahary Karadjov
389c11743a
Review TODO items and self-assign the most important ones 2020-11-10 20:41:04 +02:00
tersec
271df8b604
bump 1.0.0rc-0 spec refs to 1.0.0 (#1974) 2020-11-09 14:18:55 +00:00
Jacek Sieka
95f5f76180
Datatype cleanup (#1953) 
* clear up spec todo

* test fix

* remove unnecessary toSszType

* type

* one more
 2020-11-04 21:52:47 +00:00
Zahary Karadjov
2f29e3e7f3 Further reduce the number of RPC requests; Significantly faster ETH1 syncing 2020-11-03 23:23:10 +02:00
Zahary Karadjov
14b2d4324d openarray -> openArray 2020-11-03 23:23:10 +02:00
Zahary Karadjov
18639c3eff Don't require requests that might fail on non-archive Geth nodes 2020-11-03 23:23:10 +02:00
Zahary Karadjov
07206ca264 An efficient algorithm for incremental generation of multiple merkle proofs 2020-11-03 23:23:10 +02:00
tersec
3ee2dd8da4
p2p-interface spec ref bump (except non-updated places) (#1862) 2020-10-12 14:37:14 +00:00
tersec
b79e5f8af5
update nim-beacon-chain to nimbus-eth2 in beacon_chain/, ncli/, tests/, and README.md (#1843) 2020-10-08 19:02:05 +00:00
tersec
5354a0a3d5
fix #1585 and restore storeLEBytes on big-endian systems (#1796) 
* fix #1585 and restore storeLEBytes on big-endian systems

* switch to using stew/endians2

* create static endians2-like shim in bitseqs
 2020-10-06 08:00:11 +00:00
tersec
7eaaab908c
fix output of proposer slashing test fixture (#1780) 
* fix output of proposer slashing test fixture

* run make test

* a few more v0.12.3 spec refs
 2020-09-30 13:12:03 +00:00
zah
7c688bffda
Address #1586 (#1596) 2020-09-18 18:34:13 +02:00
Jacek Sieka
dcf8a6b05d
improve slot processing speeds (#1670) 
about 40% better slot processing times (with LTO enabled) - these don't
do BLS but are used
heavily during replay (state transition = slot + block transition)

tests using a recent medalla state and advancing it 1000 slots:

```
./ncli slots --preState2:state-302271-3c1dbf19-c1f944bf.ssz --slot:1000
--postState2:xx.ssz
```
pre:

```

All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
39.236,        0.000,       39.236,       39.236,            1,
Load state from file
0.049,        0.002,        0.046,        0.063,          968,
Apply slot
256.504,       81.008,      213.471,      591.902,           32,
Apply epoch slot
28.597,        0.000,       28.597,       28.597,            1,
Save state to file
```

cast:
```
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
37.079,        0.000,       37.079,       37.079,            1,
Load state from file
0.042,        0.002,        0.040,        0.090,          968,
Apply slot
215.552,       68.763,      180.155,      500.103,           32,
Apply epoch slot
25.106,        0.000,       25.106,       25.106,            1,
Save state to file
```

cast+rewards:
```
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
40.049,        0.000,       40.049,       40.049,            1,
Load state from file
0.048,        0.001,        0.045,        0.060,          968,
Apply slot
164.981,       76.273,      142.099,      477.868,           32,
Apply epoch slot
28.498,        0.000,       28.498,       28.498,            1,
Save state to file
```

cast+rewards+shr
```
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
12.898,        0.000,       12.898,       12.898,            1,
Load state from file
0.039,        0.002,        0.038,        0.054,          968,
Apply slot
139.971,       68.797,      120.088,      428.844,           32,
Apply epoch slot
24.761,        0.000,       24.761,       24.761,            1,
Save state to file

```
 2020-09-16 20:59:33 +00:00
Jacek Sieka
aed57df957
avoid hash tree root calculation when loading blocks from database (#1572) 2020-09-04 08:35:10 +02:00
Jacek Sieka
2081c4f505
avoid unnecessary seq allocations (#1573) 
"sigh"
 2020-08-27 08:32:51 +02:00
Zahary Karadjov
21b5ad3259
make clearn-medalla will preserve your validator files 2020-08-21 21:50:46 +03:00
tersec
b45de824a4
convert 22 v0.12.1 spec refs to v0.12.2 and remove 1 unnecessary spec ref (#1384) 2020-07-28 14:55:46 +00:00
Jacek Sieka
8b01284b0e
cache block hash (#1329) 
hash_tree_root was turning up when running beacon_node, turns out to be
repeated hash_tree_root invocations - this pr brings them back down to
normal.

this PR caches the root of a block in the SignedBeaconBlock object -
this has the potential downside that even invalid blocks will be hashed
(as part of deserialization) - later, one could imagine delaying this
until checks have passed

there's also some cleanup of the `cat=` logs which were applied randomly
and haphazardly, and to a large degree are duplicated by other
information in the log statements - in particular, topics fulfill the
same role
 2020-07-16 15:16:51 +02:00
zah
0be77f9cbc
Restore compilation with custom presets (#1309) 
This also assigns precise types to the constants in the minimal
and mainnet presets in order to reduce the chance of compilation
errors when custom presets are used (previously, only the custom
presets have precisely assigned types for the constants).
 2020-07-13 16:44:58 +02:00
Zahary Karadjov
93b04bc214 Add an option for graffiti customization 2020-07-12 21:01:31 +03:00
tersec
c64737e7f2
implement aggregated attestation receiving/validating (#1272) 
* implement aggregated attestation receiving/validating

* document the conditions without explicit implementations in isValidAggregatedAttestation()
 2020-07-02 16:15:27 +00:00
Zahary Karadjov
a2e28bef27 Further removal of enum support; Bump modules 2020-06-22 20:14:14 +03:00
Jacek Sieka
7e0e4dc327
don't crash on unknown disconnection reason, fix disconnection reason enum (#1208) 2020-06-20 09:24:33 +02:00
Zahary Karadjov
811ba9aacd Fix SSZ issues on 32-bit platforms 2020-06-11 17:40:08 +03:00
Zahary Karadjov
17343442ea Implement more of the KeyStore spec and integrate it in the beacon node 2020-06-11 17:40:08 +03:00
Jacek Sieka
68b5638da4 allow non-power-of-2 limits in hashlist 
fixes "make eth2_network_simulation"

a bit sad: no test coverage except against our own tests
 2020-06-05 14:51:22 +03:00
Jacek Sieka
56ffb696be
reorder ssz (#1099) 
* reorder ssz

* split into hash_trees and ssz_serialization, roughly, for hashing and
IO
* move bitseqs into ssz (from stew)
* clean up imports

* docs, imports
 2020-06-03 15:52:02 +02:00
Jacek Sieka
1d9a58f1e8
a few more cleanups 2020-06-02 22:10:45 +02:00
Jacek Sieka
7e881a4c09
cleanups 2020-06-02 17:17:07 +02:00
Jacek Sieka
872d7ff493
support all basic types 2020-06-02 11:44:51 +02:00
Jacek Sieka
405f460f9b
int64 fixes 2020-06-01 17:35:42 +02:00
Jacek Sieka
2316ac15c3
index type 2020-06-01 14:52:52 +02:00
Jacek Sieka
fec01829c4
cache balances, more type fixes 2020-06-01 14:52:52 +02:00
Jacek Sieka
be92842944
more type compiler bug workarounds 2020-06-01 14:52:52 +02:00
Jacek Sieka
a0b8b818f3
work around type issues 2020-06-01 14:52:52 +02:00
Jacek Sieka
4fc9a84c57
support array of uint64 2020-06-01 14:52:52 +02:00
Jacek Sieka
f1dcee2749
wip: cache 
* cache lists and arrays of complex objects (5x block processing speed
on ncli_db)

trivial baseline cache that stores tree in flat memory structure
 2020-06-01 14:52:50 +02:00
Zahary Karadjov
0bcdabfcdf Detect another invalid SSZ input found through fuzzing 
The first offset of an SSZ object should always have a fixed constant
value. Otherwise, some unused bytes may appear between the fixed portion
and the dynamic portion.

Please note that this fix shutds down the minimal forward compatibility
currently supported by the SSZ format (and thus, the expected behavior
must be clarified in the SSZ spec).
 2020-05-30 14:47:35 +03:00
Zahary Karadjov
1fd87a2255 Remove SszReader.maxObjectSize 2020-05-28 19:15:16 +03:00
Zahary Karadjov
d4fa2c8913 Avoid an unnecessary copy in readSszValue 2020-05-28 19:15:16 +03:00
Jacek Sieka
693bc15919 avoid some RVO bugs 2020-05-28 19:15:16 +03:00
Jacek Sieka
f53b55cbe0 SSZ cleanup 
* be stricter about SSZ length prefix
* compute zeroHash list at compile time
* remove SSZ schema stuff
* move SSZ navigation to ncli
* cleanup a few leftover openArray uses
 2020-05-28 19:15:16 +03:00
Jacek Sieka
0d54073ec9 properly work around nim type bug in ssz 
the cast worked around the bug at compile time by means of casting, but
introduced a runtime error, because the pre-cast type was still being
used during deserialization
 2020-05-26 14:12:42 +03:00