Commit Graph

2253 Commits

Author SHA1 Message Date
Jacek Sieka 7e3f20236f better state cache reuse (#1612) 2020-09-16 19:21:59 +02:00
tersec 1f9cd8ce2b stop discarding future epochs; remove a StateCache() construction (#1610)
* stop discarding non-existent future epochs during epoch state transitions; remove a pointless StateCache() construction in advance_slots()

* update nbench to pass StateCache to process_slots()
2020-09-16 19:21:59 +02:00
Jacek Sieka 1b16ad10b1 bump libp2p (#1609) 2020-09-16 19:21:59 +02:00
Jacek Sieka 625e0912de simplify libp2p logging (#1605)
and a few other small logging fixes
2020-09-16 19:21:59 +02:00
nbc-bump-bot[bot] 5a55ca8f47 auto-bump nim-libp2p (#1604)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2020-09-16 19:21:59 +02:00
Jacek Sieka a095a1e24d bump chronos (#1598)
* bump chronos

this reverts backwards-incompatible changes to CancelledError and
resores other functionality.

chronos metrics are lost and will need to be reimplemented.

* bump
2020-09-16 19:21:59 +02:00
tersec d69ab20d47 address issue #1552 (#1601) 2020-09-16 19:21:59 +02:00
Jacek Sieka 29e56232a7 avoid hash tree root calculation when loading blocks from database (#1572) 2020-09-16 19:21:59 +02:00
Ștefan Talpalaru c2384246bc support spaces in $(MAKE) (#1603) 2020-09-16 19:21:59 +02:00
tersec 368bd7c057 address issue #1580 (#1600)
* address issue #1580

* Update beacon_chain/spec/beaconstate.nim

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-09-16 19:21:59 +02:00
Viktor Kirilov bb83817c2a use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process 2020-09-16 19:21:59 +02:00
Viktor Kirilov 96e1a5d70e 50/50 bn/vc split for the validator keys ON by default for the testnet scripts 2020-09-16 19:21:59 +02:00
nbc-bump-bot[bot] 4a2ac2b46a auto-bump nim-libp2p (#1484)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2020-09-16 19:21:59 +02:00
Viktor Kirilov 9a8e81ceec added the --network=<x> option to the tools for which it matters 2020-09-16 19:21:59 +02:00
Ștefan Talpalaru 3bb2bcdfc9 remove template expansion file from Git control (#1589)
because it changes by simply building the software, creating problems
for beta testers
2020-09-16 19:21:59 +02:00
Jacek Sieka 174e21eb17 perform slashing check before mutation (#1594)
avoid copy of validator as well
2020-09-16 19:21:59 +02:00
tersec 1fefe2bb19 ignore sqlite WAL journals in git; increase logging priority of attestation/block sending (#1590)
* ignore sqlite WAL journal files in git; switch attestation resolved from info to debug

* promote sent attestations/blocks to notice rather than demote resolved attestations/blocks to debug
2020-09-16 19:21:59 +02:00
tersec a52b0436fd bound block quarantine size (#1564)
* bound block quarantine size

* add additional logging for block quarantining

* re-add quarantine.add() call

* remove pre-finalization blocks; add logging for full quarantine

* clear quarantine on chain reorganization

* update block_sim and tests

* update test_attestation_pool
2020-09-16 19:21:59 +02:00
tersec 0a4bbf204b bump nim-eth (#1579) 2020-09-16 19:21:59 +02:00
Zed 3574d1b621 Makefile: add clean-cross task for cross compiling (#1554) 2020-09-16 19:21:59 +02:00
Sacha Saint-Leger 24dae4b04f update troubleshooting medalla (#1578) 2020-09-16 19:21:59 +02:00
Ștefan Talpalaru 76675fc83e fix medalla-dev 2020-09-16 19:21:59 +02:00
Ștefan Talpalaru 80a50d6389 remove BUILD_LOG_LEVEL 2020-09-16 19:21:59 +02:00
tersec 0e6619da69 add setting for benchmarking and profiling of sqlite block storage times (#1575) 2020-09-16 19:21:59 +02:00
Jacek Sieka 2aa9f3e17b log getblocks error 2020-09-16 19:21:59 +02:00
Jacek Sieka 6202ede3d9 implement clock disparity for attestation validation (#1568)
This implements disparity, resolving a part of
https://github.com/status-im/nim-beacon-chain/issues/1367

* make BeaconTime a duration for fractional seconds
* factor out attestation/aggregate validation
* simplify recording of queued attestations
* simplify attestation signature check
* fix blocks_received metric
* add some trivial validation tests
* remove unresolved attestation table - attestations for unknown blocks
are dropped instead (cannot verify their signature)
2020-09-16 19:21:59 +02:00
Jacek Sieka a0bff42016 avoid unnecessary seq allocations (#1573)
"sigh"
2020-09-16 19:21:59 +02:00
Zahary Karadjov 0e64b4e3b3 Update the Nimbus bootstrap nodes 2020-09-16 19:21:59 +02:00
Sacha Saint-Leger 425823ea54 Medalla guide updates + troubleshooting page (#1571)
* book updates

* troubleshooting: introduce devel branch, and link to discord

* add disclaimer and recommendation to fall back to master

* minor edits
2020-09-16 19:21:59 +02:00
Ștefan Talpalaru 484ab55ec0 Jenkins: run benchmarks (#1524)
* Jenkins: run benchmarks

* don't exit the workspace

* try calling the benchmark plugin from the config file

* pass all params

* separate Jenkins job for benchmarks
2020-09-16 19:21:59 +02:00
tersec d0ae51e8a2 harden beacon_pending_deposits metrics calculation against overflow (#1566)
* harden beacon_pending_deposits metrics calculation

* ...

* move beacon_pending_deposits and beacon_processed_deposits_total out of specs and into chain DAG
2020-09-16 19:21:59 +02:00
Eugene Kabanov 5d744066e4 Request manager ignores non-critical errors while processing blocks. (#1569)
* Request manager ignores non-critical errors while processing blocks. Only BlockError.Invalid become critical error.

* Add some comments.
2020-09-16 19:21:59 +02:00
Mamy Ratsimbazafy a29faadace Fork choice - almost free pruning - fix #1534 (#1535)
* initial - cheaper pruning - addresses  #1534

* Pass tests: update offset when pruning, proper handling of pruned parents

* Use options instead of nil for nilable newHead (finalization passing but rootcause not solved)

* First line of defense against stackoverflow in tests

* Fix compute_delta offset after pruning

* Rebase fix - medalla ready

* Remove Option[BlockRef]
2020-09-16 19:21:59 +02:00
Jacek Sieka 1ef98c36ac reuse validator key cache better (#1562)
new key cache can be used for old epochs in the same tree
2020-09-16 19:21:59 +02:00
Ștefan Talpalaru ad08cfe97e bump vendor/nim-metrics (#1565) 2020-09-16 19:21:59 +02:00
Ștefan Talpalaru 7ca375327a bump vendor/nim-blscurve (#1560)
and log the BLS backend
2020-09-16 19:21:59 +02:00
Sacha Saint-Leger 9ac48dddca important book updates (#1561)
* book updates

* save progress

* save progress

* work on deposit data section + edits

* title edit

* minor edits

* edits and clarifications

* add images, launchpad steps, and a note on expected wait time

* add images + edits

* restructure, simplify, edits

* faq corrections

* rework intro: cp

* update intro and faq

* add note on requestiong goerli eth

* typo fix

* add numbers to steps in walletstore guide"

* chapter 8 title: replace Nimbus with NBC

* Update docs/the_nimbus_book/src/medalla.md

Co-authored-by: Jacek Sieka <jacek@status.im>

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-09-16 19:21:59 +02:00
Jacek Sieka 48d19f91a1 add option to disable discv5 (#1509) 2020-09-16 19:21:59 +02:00
Ștefan Talpalaru 646691d079 update Grafana dashboard 2020-09-16 19:21:59 +02:00
Mamy Ratsimbazafy b5b79bba4c
Phase 2 audit 2020-09-16 10:50:54 +02:00
zah a84a8ba192
Merge pull request #1559 from status-im/devel
Merge devel into master
2020-08-22 01:52:44 +03:00
Dustin Brody 95d5736128 don't rely on head updates for topic subscription decision 2020-08-22 01:50:50 +03:00
Dustin Brody cac4fee569 remove waitFor 2020-08-22 01:50:50 +03:00
Dustin Brody d9bf6f450d delay starting gossipsub until near-synchronized 2020-08-22 01:50:50 +03:00
Zahary Karadjov 273123d9c1 Add a CPU_LIMIT option for the Medalla and Altona targets [skip ci]
This can be used to trigger problems such as peer dropping that
manifest only on higher CPU loads.
2020-08-22 01:50:38 +03:00
Zahary Karadjov 8809f8d200 Implement 'wallets restore' and 'wallets list'; Update 'nextAccount' properly after making deposits multiple times 2020-08-22 00:16:18 +03:00
Ștefan Talpalaru 1d1e0217ca bump vendor/nim-nat-traversal 2020-08-22 00:09:45 +03:00
Zahary Karadjov 21b5ad3259
make clearn-medalla will preserve your validator files 2020-08-21 21:50:46 +03:00
Dustin Brody 99e330d014 fix underflow in deposit procesing (#1542) 2020-08-21 19:00:36 +03:00
Jacek Sieka 61538fa581 speed up shuffling
Replace shuffling function with zrnt version - `get_shuffled_seq` in
particular puts more strain on the GC by allocating superfluous seq's
which turns out to have a significant impact on block processing (when
replaying blocks for example) - 4x improvement on non-epoch, 1.5x on
epoch blocks (replay is done without signature checking)

Medalla, first 10k slots - pre:

```
Loaded 68973 blocks, head slot 117077
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
76855.848,        0.000,    76855.848,    76855.848,            1,
Initialize DB
1.073,        0.914,        0.071,       12.454,         7831,
Load block from database
31.382,        0.000,       31.382,       31.382,            1,
Load state from database
85.644,       30.350,        3.056,      466.136,         7519,
Apply block
506.569,       91.129,      130.654,      874.786,          312,
Apply epoch block
```

post:

```
Loaded 68973 blocks, head slot 117077
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
72457.303,        0.000,    72457.303,    72457.303,            1,
Initialize DB
1.015,        0.858,        0.070,       11.231,         7831,
Load block from database
28.983,        0.000,       28.983,       28.983,            1,
Load state from database
21.725,       17.461,        2.659,      393.217,         7519,
Apply block
324.012,       33.954,       45.452,      440.532,          312,
Apply epoch block
```
2020-08-21 16:05:10 +03:00