status-im/nimbus-eth2
2
0
Fork 0
mirror of https://github.com/status-im/nimbus-eth2.git synced 2025-02-10 05:35:28 +00:00
Code Issues Projects Releases Wiki Activity
2,286 Commits 116 Branches 117 Tags 135 MiB
Commit Graph

2286 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
tersec
e106549efe
keep REJECT/IGNORE of messages failing validation for libp2p scoring (#1676) 
* keep REJECT/IGNORE status of messages failing validation for libp2p scoring

* fix test suite
 2020-09-18 13:53:09 +02:00
Jacek Sieka
000a0ecc52
initial e2store file format description (#1355) 
This is one way we could organize the flat file storage for blocks - the
alternative would be to not do `type` in the file itself, but have a
single type per file which arguably is simpler but may become annoying.

Another potential restriction would be to require that blocks are
ordered - with this format, it's a little bit more involved to recreate
an index file, and it's easy to accidentally build in assumptions about
the block order in the main data file.
 2020-09-17 23:23:54 +02:00
Sacha Saint-Leger
e4b4b29320
Update medalla.md (#1685) 
minor edits + clarifications
 2020-09-17 20:25:21 +02:00
Sacha Saint-Leger
4c0bdffd4c
Update medalla.md (#1681) 
explain syncv2 string and terminal status bar
 2020-09-17 18:39:49 +02:00
Dmitriy Ryajov
2f89e2ab4e
drop subscribePeer, it's called from pubsub now (#1677) 2020-09-17 11:40:21 +02:00
Sacha Saint-Leger
6bde3b85a7
Book edits: master is now the preferred branch (#1675) 
* Update medalla.md

remove `cd devel`

* Update medalla-troubleshooting.md

edits to make clear that master is now the recommended branch
 2020-09-17 00:26:51 +02:00
Jacek Sieka
dcf8a6b05d
improve slot processing speeds (#1670) 
about 40% better slot processing times (with LTO enabled) - these don't
do BLS but are used
heavily during replay (state transition = slot + block transition)

tests using a recent medalla state and advancing it 1000 slots:

```
./ncli slots --preState2:state-302271-3c1dbf19-c1f944bf.ssz --slot:1000
--postState2:xx.ssz
```
pre:

```

All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
39.236,        0.000,       39.236,       39.236,            1,
Load state from file
0.049,        0.002,        0.046,        0.063,          968,
Apply slot
256.504,       81.008,      213.471,      591.902,           32,
Apply epoch slot
28.597,        0.000,       28.597,       28.597,            1,
Save state to file
```

cast:
```
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
37.079,        0.000,       37.079,       37.079,            1,
Load state from file
0.042,        0.002,        0.040,        0.090,          968,
Apply slot
215.552,       68.763,      180.155,      500.103,           32,
Apply epoch slot
25.106,        0.000,       25.106,       25.106,            1,
Save state to file
```

cast+rewards:
```
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
40.049,        0.000,       40.049,       40.049,            1,
Load state from file
0.048,        0.001,        0.045,        0.060,          968,
Apply slot
164.981,       76.273,      142.099,      477.868,           32,
Apply epoch slot
28.498,        0.000,       28.498,       28.498,            1,
Save state to file
```

cast+rewards+shr
```
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
12.898,        0.000,       12.898,       12.898,            1,
Load state from file
0.039,        0.002,        0.038,        0.054,          968,
Apply slot
139.971,       68.797,      120.088,      428.844,           32,
Apply epoch slot
24.761,        0.000,       24.761,       24.761,            1,
Save state to file

```
 2020-09-16 20:59:33 +00:00
Ștefan Talpalaru
ce85f58d29
better handling of Ctrl+C during make update (#1588) 2020-09-16 17:46:05 +02:00
nbc-bump-bot[bot]
df685f0c12
auto-bump nim-libp2p (#1659) 
Co-authored-by: = <arnetheduck@users.noreply.github.com>
 2020-09-16 14:22:19 +02:00
Mamy Ratsimbazafy
52548f079b
Opt-in Slashing protection + interchange (#1643) 
* Slashing protection + interchange initial commit

* Restrict the when UseSlashingProtection dance in other modules

* Integrate slashing tests in other all_tests

* Add attestation slashing protection support

* Add a message that mention if built with/without slashing protection

* no op the initialization proc

* test slashing protection in Jenkins (temp)

* where to configure NIMFLAGS in Jenkins ...

* Jenkins -> ensure Built with slashing protection

* Add slashing protection complete import

* use Opt.get(otherwise)

* Don't use negation in proc name

* Turn slashing protection on by default
 2020-09-16 13:30:03 +02:00
Eugene Kabanov
6e463257f4
PeerPool fixes. (#1654) 
* Refactor peer_pool.
Fix eth2_network peer counters.
Fix PeerPool do not allow to add more peers when empty space available.

* Remove unused imports.

* Add test for a bug.

* Fix eth2_network disconnect should deletePeer not release.
More PeerPool refactoring.
 2020-09-16 13:00:11 +03:00
tersec
6d8130dc49
close block_sim database; remove code duplication in exit_pool (#1656) 2020-09-16 09:16:23 +02:00
Eugene Kabanov
9abdbdabd8
Fix sync_manager.nim not rewarding peers for good responses. (#1660) 2020-09-16 09:15:06 +02:00
nbc-bump-bot[bot]
88eb0c06c7
auto-bump nim-libp2p (#1606) 
Co-authored-by: = <arnetheduck@users.noreply.github.com>
 2020-09-15 16:17:25 +02:00
tersec
7e10b9850b
enable topic unsubscribing and attestation subnet cycling (#1646) 
* enable topic unsubscribing and attestation subnet cycling

* remove refences to states

* waitFor -> await

* revert exit pool changes for a cleaner PR
 2020-09-15 12:40:43 +00:00
Eugene Kabanov
617c295839
Bump chronos to fix memory leaks. (#1648) 2020-09-15 11:45:00 +02:00
Jacek Sieka
c76305f824
fix some todo (#1645) 
* remove some superfluous gcsafes
* remove getTailState (unused)
* don't store old epochrefs in blocks
* document attestation pool a bit
* remove `pcs =` cruft from log
 2020-09-14 14:50:03 +00:00
tersec
9f21bbd666
[WIP] skeleton of attester slashing pool & validators (#1639) 
* skeleton of attester slashing pool & validators

* add skeleton for proposer slashings and voluntary exits; rename pool to more inclusive exit pool to stay consistent with all three; ensure is initialized by beacon_node so is safe to merge, even if it doesn't do much yet
 2020-09-14 16:26:31 +02:00
Ștefan Talpalaru
60b8905852
reorder startup steps (#1642) 
so the Ctr+C hook and the metrics server go before the potentially slow
`BeaconNode.init()`
 2020-09-14 08:24:31 +02:00
Sacha Saint-Leger
ca2e7e2239
update medalla guide: advanced options (WIP) (#1623) 
* update medalla guide: advanced options

* minor edits
 2020-09-13 23:58:19 +02:00
Sacha Saint-Leger
7d00ca2e86
Update medalla-troubleshooting.md (#1635) 2020-09-13 11:17:27 +02:00
Sacha Saint-Leger
e6c82d1815
Update medalla-troubleshooting.md (#1633) 
Add database pruning subsection
 2020-09-13 10:29:05 +02:00
tersec
aca1a318f2
cleanly close kvstore databases and bump nim-eth (#1630) 
* cleanly close kvstore databases

* close databases for all subcommands and during error conditions
 2020-09-12 05:35:58 +00:00
Kim De Mey
5fff800cf8
Bump nim-eth to use lrucache for discovery sessions (#1622) 2020-09-11 17:46:48 +00:00
Jacek Sieka
eaea3dbc94
remove chronos future tracking from default build (#1628) 
uses a lot of memory, specially during sync
 2020-09-11 17:07:16 +02:00
tersec
48893f1c2e
add ncli_db subcommand to prune database of unnecessary blocks and states (#1593) 
* add ncli_db subcommand to prune database of unnecessary blocks, states, and state roots

* tweak comments

* reduce default aggressiveness in pruning old states

* move copyPrunedDatabase() to ncli_db, as it's not generally useful as part of beacon_chain_db and doesn't use any internal interfaces
 2020-09-11 15:20:34 +02:00
Eugene Kabanov
c7c9b9d5f1
Syncing V2 (#1602) 
* Syncing workers now not bound to peers.
Sync status is now printed in statusbar.

* Add `SyncQueue.outSlot` to statusbar too.

* Add `inRangeEvent` and `rangeAge` parameter.

* Fix rangeAge is not depends on SyncQueue latest slot.
Fix syncManager to start from latest local head slot.

* Add notInRange event.

* Remove suspects field.
 2020-09-11 14:46:01 +02:00
Jacek Sieka
775683cf2e
await validator duties (#1626) 
Validator duties proceed slot-by-slot - we should not start a new
validator duty iteration before the previous one is gone or we might run
into consistency and voting issues
 2020-09-11 09:50:00 +00:00
Jacek Sieka
8a5a261fcd
Quick fix to prune some states, pending smarter state storage (#1624) 
* Quick fix to prune some states, pending smarter state storage

Adverse effects might include slow rewinds - typically the protocol
doesn't ask for pre-finalized states but RPC might

* document issue, add test

* fix cache miss log
 2020-09-11 10:03:50 +02:00
Eugene Kabanov
9db05c484d
Bump chronos to help investigate Index problem. (#1625) 2020-09-11 07:12:12 +02:00
Jacek Sieka
a087909683
fix req/resp protocol (#1621) 
per spec, we must half-close request stream - not doing so may lead to
failure of the other end to start processing our request leading to
timeouts.

In particular, this fixes many sync problems that have been seen on
medalla.

* remove safeClose - close no longer raises
* use per-chunk timeouts in request processing
 2020-09-10 21:40:09 +02:00
Sacha Saint-Leger
697bd23c9b
fix low peer count command typo (#1619) 2020-09-10 12:05:14 +02:00
Jacek Sieka
7c21fa983a
bump libp2p, introduce AsyncQueue BufferStream (#1599) 
* Syncing workers now not bound to peers.
Sync status is now printed in statusbar.

* Add `SyncQueue.outSlot` to statusbar too.

* Add `inRangeEvent` and `rangeAge` parameter.

* Fix rangeAge is not depends on SyncQueue latest slot.
Fix syncManager to start from latest local head slot.

* Add notInRange event.

* bump libp2p, introduce AsyncQueue BufferStream

* bump

Co-authored-by: cheatfate <eugene.kabanov@status.im>
 2020-09-10 08:22:39 +02:00
tersec
67f44d4bd2
use V=1 in Jenkinsfile (#1618) 2020-09-09 14:33:51 +00:00
tersec
5d1ea5e440
match log filenames more precisely for logtrace analysis (#1617) 
* match log filenames more precisely for logtrace analysis
 2020-09-09 13:08:21 +00:00
Dustin Brody
990bc2ea52
remove obsolete comment 2020-09-09 11:00:02 +02:00
tersec
d0de1a49a3
Fix some warnings and hints and partly revert #1610 (#1615) 
* address some XDeclaredButNotUsed, ConvFromXtoItselfNotNeeded, and UnusedImport hints and warnings

* partly revert #1610
 2020-09-08 11:32:43 +00:00
tersec
b3b578501a
mark comments documenting functions as a whole as such (#1613) 2020-09-08 08:54:55 +00:00
Jacek Sieka
dde26f359f
better state cache reuse (#1612) 2020-09-08 09:23:48 +02:00
tersec
3d5f24f14c
stop discarding future epochs; remove a StateCache() construction (#1610) 
* stop discarding non-existent future epochs during epoch state transitions; remove a pointless StateCache() construction in advance_slots()

* update nbench to pass StateCache to process_slots()
 2020-09-07 15:04:33 +00:00
Jacek Sieka
4c3dd853e1
bump libp2p (#1609) 2020-09-07 14:17:15 +02:00
Jacek Sieka
d584591ded
simplify libp2p logging (#1605) 
and a few other small logging fixes
 2020-09-06 10:39:25 +02:00
nbc-bump-bot[bot]
c5f8ef8e40
auto-bump nim-libp2p (#1604) 
Co-authored-by: = <arnetheduck@users.noreply.github.com>
 2020-09-05 08:28:03 +02:00
Jacek Sieka
f61fdf93d3
bump chronos (#1598) 
* bump chronos

this reverts backwards-incompatible changes to CancelledError and
resores other functionality.

chronos metrics are lost and will need to be reimplemented.

* bump
 2020-09-04 21:27:54 +02:00
tersec
456bdc87cd
address issue #1552 (#1601) 2020-09-04 08:39:46 +02:00
Jacek Sieka
aed57df957
avoid hash tree root calculation when loading blocks from database (#1572) 2020-09-04 08:35:10 +02:00
Ștefan Talpalaru
aa3298f0ea
support spaces in $(MAKE) (#1603) 2020-09-04 08:33:37 +02:00
tersec
e4a43f7628
address issue #1580 (#1600) 
* address issue #1580

* Update beacon_chain/spec/beaconstate.nim

Co-authored-by: Jacek Sieka <jacek@status.im>
 2020-09-02 16:16:25 +00:00
Viktor Kirilov
d9f9949ef0 use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process 2020-09-02 12:47:00 +03:00
Viktor Kirilov
65d7787b1e 50/50 bn/vc split for the validator keys ON by default for the testnet scripts 2020-09-01 16:39:07 +03:00