Commit Graph

2900 Commits

Author SHA1 Message Date
tersec 0e6619da69 add setting for benchmarking and profiling of sqlite block storage times (#1575) 2020-09-16 19:21:59 +02:00
Jacek Sieka 2aa9f3e17b log getblocks error 2020-09-16 19:21:59 +02:00
Jacek Sieka 6202ede3d9 implement clock disparity for attestation validation (#1568)
This implements disparity, resolving a part of
https://github.com/status-im/nim-beacon-chain/issues/1367

* make BeaconTime a duration for fractional seconds
* factor out attestation/aggregate validation
* simplify recording of queued attestations
* simplify attestation signature check
* fix blocks_received metric
* add some trivial validation tests
* remove unresolved attestation table - attestations for unknown blocks
are dropped instead (cannot verify their signature)
2020-09-16 19:21:59 +02:00
Jacek Sieka a0bff42016 avoid unnecessary seq allocations (#1573)
"sigh"
2020-09-16 19:21:59 +02:00
Zahary Karadjov 0e64b4e3b3 Update the Nimbus bootstrap nodes 2020-09-16 19:21:59 +02:00
Sacha Saint-Leger 425823ea54 Medalla guide updates + troubleshooting page (#1571)
* book updates

* troubleshooting: introduce devel branch, and link to discord

* add disclaimer and recommendation to fall back to master

* minor edits
2020-09-16 19:21:59 +02:00
Ștefan Talpalaru 484ab55ec0 Jenkins: run benchmarks (#1524)
* Jenkins: run benchmarks

* don't exit the workspace

* try calling the benchmark plugin from the config file

* pass all params

* separate Jenkins job for benchmarks
2020-09-16 19:21:59 +02:00
tersec d0ae51e8a2 harden beacon_pending_deposits metrics calculation against overflow (#1566)
* harden beacon_pending_deposits metrics calculation

* ...

* move beacon_pending_deposits and beacon_processed_deposits_total out of specs and into chain DAG
2020-09-16 19:21:59 +02:00
Eugene Kabanov 5d744066e4 Request manager ignores non-critical errors while processing blocks. (#1569)
* Request manager ignores non-critical errors while processing blocks. Only BlockError.Invalid become critical error.

* Add some comments.
2020-09-16 19:21:59 +02:00
Mamy Ratsimbazafy a29faadace Fork choice - almost free pruning - fix #1534 (#1535)
* initial - cheaper pruning - addresses  #1534

* Pass tests: update offset when pruning, proper handling of pruned parents

* Use options instead of nil for nilable newHead (finalization passing but rootcause not solved)

* First line of defense against stackoverflow in tests

* Fix compute_delta offset after pruning

* Rebase fix - medalla ready

* Remove Option[BlockRef]
2020-09-16 19:21:59 +02:00
Jacek Sieka 1ef98c36ac reuse validator key cache better (#1562)
new key cache can be used for old epochs in the same tree
2020-09-16 19:21:59 +02:00
Ștefan Talpalaru ad08cfe97e bump vendor/nim-metrics (#1565) 2020-09-16 19:21:59 +02:00
Ștefan Talpalaru 7ca375327a bump vendor/nim-blscurve (#1560)
and log the BLS backend
2020-09-16 19:21:59 +02:00
Sacha Saint-Leger 9ac48dddca important book updates (#1561)
* book updates

* save progress

* save progress

* work on deposit data section + edits

* title edit

* minor edits

* edits and clarifications

* add images, launchpad steps, and a note on expected wait time

* add images + edits

* restructure, simplify, edits

* faq corrections

* rework intro: cp

* update intro and faq

* add note on requestiong goerli eth

* typo fix

* add numbers to steps in walletstore guide"

* chapter 8 title: replace Nimbus with NBC

* Update docs/the_nimbus_book/src/medalla.md

Co-authored-by: Jacek Sieka <jacek@status.im>

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-09-16 19:21:59 +02:00
Jacek Sieka 48d19f91a1 add option to disable discv5 (#1509) 2020-09-16 19:21:59 +02:00
Ștefan Talpalaru 646691d079 update Grafana dashboard 2020-09-16 19:21:59 +02:00
Ștefan Talpalaru ce85f58d29
better handling of Ctrl+C during `make update` (#1588) 2020-09-16 17:46:05 +02:00
nbc-bump-bot[bot] df685f0c12
auto-bump nim-libp2p (#1659)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2020-09-16 14:22:19 +02:00
Mamy Ratsimbazafy 52548f079b
Opt-in Slashing protection + interchange (#1643)
* Slashing protection + interchange initial commit

* Restrict the when UseSlashingProtection dance in other modules

* Integrate slashing tests in other all_tests

* Add attestation slashing protection support

* Add a message that mention if built with/without slashing protection

* no op the initialization proc

* test slashing protection in Jenkins (temp)

* where to configure NIMFLAGS in Jenkins ...

* Jenkins -> ensure Built with slashing protection

* Add slashing protection complete import

* use Opt.get(otherwise)

* Don't use negation in proc name

* Turn slashing protection on by default
2020-09-16 13:30:03 +02:00
Eugene Kabanov 6e463257f4
PeerPool fixes. (#1654)
* Refactor peer_pool.
Fix eth2_network peer counters.
Fix PeerPool do not allow to add more peers when empty space available.

* Remove unused imports.

* Add test for a bug.

* Fix eth2_network disconnect should deletePeer not release.
More PeerPool refactoring.
2020-09-16 13:00:11 +03:00
Mamy Ratsimbazafy b5b79bba4c
Phase 2 audit 2020-09-16 10:50:54 +02:00
tersec 6d8130dc49
close block_sim database; remove code duplication in exit_pool (#1656) 2020-09-16 09:16:23 +02:00
Eugene Kabanov 9abdbdabd8
Fix sync_manager.nim not rewarding peers for good responses. (#1660) 2020-09-16 09:15:06 +02:00
nbc-bump-bot[bot] 88eb0c06c7
auto-bump nim-libp2p (#1606)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2020-09-15 16:17:25 +02:00
tersec 7e10b9850b
enable topic unsubscribing and attestation subnet cycling (#1646)
* enable topic unsubscribing and attestation subnet cycling

* remove refences to states

* waitFor -> await

* revert exit pool changes for a cleaner PR
2020-09-15 12:40:43 +00:00
Eugene Kabanov 617c295839
Bump chronos to fix memory leaks. (#1648) 2020-09-15 11:45:00 +02:00
Jacek Sieka c76305f824
fix some todo (#1645)
* remove some superfluous gcsafes
* remove getTailState (unused)
* don't store old epochrefs in blocks
* document attestation pool a bit
* remove `pcs =` cruft from log
2020-09-14 14:50:03 +00:00
tersec 9f21bbd666
[WIP] skeleton of attester slashing pool & validators (#1639)
* skeleton of attester slashing pool & validators

* add skeleton for proposer slashings and voluntary exits; rename pool to more inclusive exit pool to stay consistent with all three; ensure is initialized by beacon_node so is safe to merge, even if it doesn't do much yet
2020-09-14 16:26:31 +02:00
Ștefan Talpalaru 60b8905852
reorder startup steps (#1642)
so the Ctr+C hook and the metrics server go before the potentially slow
`BeaconNode.init()`
2020-09-14 08:24:31 +02:00
Sacha Saint-Leger ca2e7e2239
update medalla guide: advanced options (WIP) (#1623)
* update medalla guide: advanced options

* minor edits
2020-09-13 23:58:19 +02:00
Sacha Saint-Leger 7d00ca2e86
Update medalla-troubleshooting.md (#1635) 2020-09-13 11:17:27 +02:00
Sacha Saint-Leger e6c82d1815
Update medalla-troubleshooting.md (#1633)
Add database pruning subsection
2020-09-13 10:29:05 +02:00
tersec aca1a318f2
cleanly close kvstore databases and bump nim-eth (#1630)
* cleanly close kvstore databases

* close databases for all subcommands and during error conditions
2020-09-12 05:35:58 +00:00
Kim De Mey 5fff800cf8
Bump nim-eth to use lrucache for discovery sessions (#1622) 2020-09-11 17:46:48 +00:00
Jacek Sieka eaea3dbc94
remove chronos future tracking from default build (#1628)
uses a lot of memory, specially during sync
2020-09-11 17:07:16 +02:00
tersec 48893f1c2e
add ncli_db subcommand to prune database of unnecessary blocks and states (#1593)
* add ncli_db subcommand to prune database of unnecessary blocks, states, and state roots

* tweak comments

* reduce default aggressiveness in pruning old states

* move copyPrunedDatabase() to ncli_db, as it's not generally useful as part of beacon_chain_db and doesn't use any internal interfaces
2020-09-11 15:20:34 +02:00
Eugene Kabanov c7c9b9d5f1
Syncing V2 (#1602)
* Syncing workers now not bound to peers.
Sync status is now printed in statusbar.

* Add `SyncQueue.outSlot` to statusbar too.

* Add `inRangeEvent` and `rangeAge` parameter.

* Fix rangeAge is not depends on SyncQueue latest slot.
Fix syncManager to start from latest local head slot.

* Add notInRange event.

* Remove suspects field.
2020-09-11 14:46:01 +02:00
Jacek Sieka 775683cf2e
await validator duties (#1626)
Validator duties proceed slot-by-slot - we should not start a new
validator duty iteration before the previous one is gone or we might run
into consistency and voting issues
2020-09-11 09:50:00 +00:00
Jacek Sieka 8a5a261fcd
Quick fix to prune some states, pending smarter state storage (#1624)
* Quick fix to prune some states, pending smarter state storage

Adverse effects might include slow rewinds - typically the protocol
doesn't ask for pre-finalized states but RPC might

* document issue, add test

* fix cache miss log
2020-09-11 10:03:50 +02:00
Eugene Kabanov 9db05c484d
Bump chronos to help investigate Index problem. (#1625) 2020-09-11 07:12:12 +02:00
Jacek Sieka a087909683
fix req/resp protocol (#1621)
per spec, we must half-close request stream - not doing so may lead to
failure of the other end to start processing our request leading to
timeouts.

In particular, this fixes many sync problems that have been seen on
medalla.

* remove safeClose - close no longer raises
* use per-chunk timeouts in request processing
2020-09-10 21:40:09 +02:00
Sacha Saint-Leger 697bd23c9b
fix low peer count command typo (#1619) 2020-09-10 12:05:14 +02:00
Jacek Sieka 7c21fa983a
bump libp2p, introduce AsyncQueue BufferStream (#1599)
* Syncing workers now not bound to peers.
Sync status is now printed in statusbar.

* Add `SyncQueue.outSlot` to statusbar too.

* Add `inRangeEvent` and `rangeAge` parameter.

* Fix rangeAge is not depends on SyncQueue latest slot.
Fix syncManager to start from latest local head slot.

* Add notInRange event.

* bump libp2p, introduce AsyncQueue BufferStream

* bump

Co-authored-by: cheatfate <eugene.kabanov@status.im>
2020-09-10 08:22:39 +02:00
tersec 67f44d4bd2
use V=1 in Jenkinsfile (#1618) 2020-09-09 14:33:51 +00:00
tersec 5d1ea5e440
match log filenames more precisely for logtrace analysis (#1617)
* match log filenames more precisely for logtrace analysis
2020-09-09 13:08:21 +00:00
Dustin Brody 990bc2ea52
remove obsolete comment 2020-09-09 11:00:02 +02:00
tersec d0de1a49a3
Fix some warnings and hints and partly revert #1610 (#1615)
* address some XDeclaredButNotUsed, ConvFromXtoItselfNotNeeded, and UnusedImport hints and warnings

* partly revert #1610
2020-09-08 11:32:43 +00:00
tersec b3b578501a
mark comments documenting functions as a whole as such (#1613) 2020-09-08 08:54:55 +00:00
Jacek Sieka dde26f359f
better state cache reuse (#1612) 2020-09-08 09:23:48 +02:00
tersec 3d5f24f14c
stop discarding future epochs; remove a StateCache() construction (#1610)
* stop discarding non-existent future epochs during epoch state transitions; remove a pointless StateCache() construction in advance_slots()

* update nbench to pass StateCache to process_slots()
2020-09-07 15:04:33 +00:00