1621 Commits

Author SHA1 Message Date
Zahary Karadjov
fada11e1e9 Address review comments
* Create a json log named "nbc.log" only when the --dev-build option
  is specified to the connect_to_testnet script.

  This option is specified for the targets "testnet0", "testnet1"
  and "schlesi-dev"
2020-04-28 13:49:30 +03:00
Zahary Karadjov
667bdf066a Bump Chronicles to fix a minor bug; Add a placeholder for disabled topics 2020-04-28 13:49:30 +03:00
Zahary Karadjov
7237a370ba Implement https://github.com/status-im/nim-beacon-chain/issues/872
Please see the newly added 'schlesi-dev' Makefile target.
It demonstrates how the log level can be specified for individual topics.

Additionally, when connecting to testnets like 'schlesi' there will be
two additional log files produced in the working directory:

* json-log.txt
* text-log.txt (in the textblocks format)
2020-04-28 13:49:30 +03:00
Jacek Sieka
28d6cd2524
avoid memory allocations and copies when loading states (#942)
* rolls back some of the ref changes
* adds utility to calculate stack sizes
* works around bugs in nim exception handling and rvo
2020-04-28 10:08:32 +02:00
Zahary Karadjov
80b538452e Updated the custom preset loader to the latest spec; Working makefile target for connecting to the Schlesi testnet 2020-04-27 23:22:54 +03:00
Viktor Kirilov
dcc70c5259 resolved final comment from review 2020-04-27 22:10:35 +03:00
Viktor Kirilov
8ef28c905d fixed commets from review 2020-04-27 22:10:35 +03:00
Viktor Kirilov
5ecba6df49 broadcasting and listening to the '_snappy' topics as well 2020-04-27 22:10:35 +03:00
Jacek Sieka
898df9ba45
kvstore: port to nim-eth (#938) 2020-04-27 18:36:28 +02:00
tersec
7790644e52
remove a pointless hash_tree_root(BeaconState) per node per proposed block (#933)
* remove a pointless hash_tree_root(BeaconBlock)

* use ref with putState
2020-04-27 12:47:49 +02:00
Zahary Karadjov
f0b1fffefc
Fix the connect_to_testnet script 2020-04-27 00:06:22 +03:00
Jacek Sieka
03a147ab8d
avoid state copy in state transition (#930)
In BlockPool, we keep the head state around, so it's trivial to restore
the temporary state there and keep going as if nothing happened.

This solves 3 problems:
* stack space - the state copy on mainnet is huge
* GC scanning - using stack space for state slows down the GC
significantly
* reckless copying - the copy itself takes a long time

In state_sim, we'll do the same and allocate on heap - this helps a
little with GC - without it, the collection of the temporary strings
created with `toHex` while printing the json dominates the trace.
2020-04-26 21:13:33 +02:00
tersec
be475a82d7
catch invalid 0-raw-byte Bitlist SSZ inputs during parsing (#936)
* catch invalid 0-raw-byte Bitlist SSZ inputs during parsing

* bump a couple more spec refs to v0.11.1
2020-04-26 16:26:53 +00:00
Zahary Karadjov
4e9fa51ae9 Introduce BeaconNodeRef and use it in all the right places 2020-04-26 13:04:53 +03:00
Zahary Karadjov
fdcbfdff05 Pass the test suite with a BeaconState ref type 2020-04-26 13:04:53 +03:00
Zahary Karadjov
740b76d152 Spec-compliant implementation of Eth1 monitoring; Eth1-enabled local sim
BEWARE! This commit will trigger a stack overflow during local sim
2020-04-26 13:04:53 +03:00
Ștefan Talpalaru
39a893ea90
use the right preset in "make testnet{0,1}" [skip ci] (#932)
...and stop pretending to handle the preset in "connect_to_testnet.nims".
2020-04-24 17:28:03 +02:00
Jacek Sieka
fe30b24aba
bump (#929) 2020-04-24 11:47:51 +02:00
tersec
65353bab76
add another check for inconsistent aggregation and committee length (#927)
* add another check for inconsistent aggregation and committee length, since ncli_transition bypasses process_attestation(...)/check_attestation(...) and calls almost directly into process_epoch(...)

* bump validator functions to v0.11.1 spec references

* bump some spec references to v0.11.1

* poke
2020-04-24 09:16:40 +02:00
Jacek Sieka
494ffb63ce
eh fixes (#926)
* work around improbable exceptions in metrics / chronos
* fix unnecessary lookup in block pool
2020-04-24 09:16:11 +02:00
tersec
7bc18423fa
detect improper 0-length lists of variable-sized objects in SSZ reading (#928)
* detect improper 0-length lists of variable-sized objects in SSZ reading
2020-04-23 19:39:23 +00:00
Viktor Kirilov
0079242457 again exported the right overloads for serializing hashes properly to json (and not as byte arrays) and updating nim-eth 2020-04-23 20:46:13 +03:00
Viktor Kirilov
4c66999a43 removing the json serialization change - fails to compile with -d:testnet_servers_image and LOG_LEVEL=TRACE and isn't worth the time at the moment 2020-04-23 20:46:13 +03:00
Viktor Kirilov
1575ded589 exported the right overloads for serializing hashes properly to json (and not as byte arrays) 2020-04-23 20:46:13 +03:00
Viktor Kirilov
f3d0ae35f6 added the attestation bitfield to the bootstrap enr 2020-04-23 20:46:13 +03:00
Eugene Kabanov
be89a3c54d
Add "drop by score" ability to PeerPool. (#917)
* Add "drop by score" ability to PeerPool.
Add tests.
Fix syncmanager queue to start from most fresh data.

* Fix endless cycle at the end of syncing process.
2020-04-23 17:31:00 +02:00
tersec
7cd4b0bfae
reduce stack space usage in process_final_updates(...) (#925)
* reduce stack space usage in process_final_updates(...) to avoid fuzzed segfault in https://github.com/status-im/nim-beacon-chain/issues/921

* document motivation behind manually constructing hash_tree_root of a HistoricalBatch
2020-04-23 13:39:38 +02:00
tersec
7e94482409
avoid rewindStateData/state_transition in fast-path (#919)
* avoid rewindStateData/state_transition totally in fast-pathed updateStateData calls

* document motivation behind BlockPool.getStateDataCached(...) and clarify naming
2020-04-23 11:59:29 +02:00
Jacek Sieka
6729d3c032
kvstore: fix raising, be harsher on database errors (#923)
* kvstore: fix raising, be harsher on database errors

* bump stew/serialization
2020-04-23 08:27:35 +02:00
Zahary Karadjov
ae9ca33aca
Disable the faststreams destructors 2020-04-22 21:33:04 +03:00
Jacek Sieka
65ca74c980 req: cap requested blocks better
also cap blocks in roots request
2020-04-22 12:09:26 +03:00
tersec
62e3c250c0
fix warnings (#914) 2020-04-22 07:53:30 +02:00
Jacek Sieka
ed74770451
spec: regulate exceptions (#913)
* spec: regulate exceptions

* a few more simple raises
2020-04-22 07:53:02 +02:00
tersec
bdea75e4c3
Restore all blockpool extended validation checks (#886)
* fix remaining block pool extended validation issues and re-enable first-block-received and block-signature EV checks; enable Merkle validation in beacon_node in eth2_network_simulation; refactor some Merkle proof generation code outside tests/ as a result

* re-enable Merkle validation skipping, since while it works on make eth2_network_simulation, it has issues with local testnet

* tighten already-seen-block blockpool check; move comment closer to conceptually proximate code; queue up maybe-future-valid-blocks as pending to keep libp2p-synchronous interrupt handling time lower

* revert the cleanups, now in a separate PR

* remove the remaining merkle_minimal cleanup remnants, also moved to other PR

* restore PR to only modifying one file after rebasing

* use signatures as summary to compare block contents

* switch signature comparison to be raw byte-wise to ensure no attempts to deserialize it to valid (or not) BLS signatures first
2020-04-21 18:52:53 +02:00
Ștefan Talpalaru
2f74951c04
bump submodules (#912) 2020-04-21 16:54:32 +02:00
Jacek Sieka
da988e4e40 more block range request updates
* handle skip == 0 gracefully
* avoid memory allocation at expense of more complex API
* add more tests
* log block request results
2020-04-21 11:07:27 +03:00
Jacek Sieka
fccce85f0d simplify block request
this matches the intended spec behaviour
2020-04-21 11:07:27 +03:00
Giovanni Petrantoni
b441a373db Bump libp2p submodule 2020-04-21 10:46:21 +09:00
Jacek Sieka
2c2de2b35e
pull out validator duties to simplift onSlot handler (#910) 2020-04-20 19:28:12 +02:00
tersec
8a72ae89b9
fix mainnet finalization (#906)
* fix mainnet finalization and swith eth2_network_simulation to a kind of small-mainnet profile

* Fix slot reference in trace logging

* bump a couple of spec refs from v0.11.0 to v0.11.1

* bump another spec ref to v0.11.1, one more try at Jenkins test vector download CI issue

* fix other slot reference in trace logging and skip past single-block/multi-slot gaps to re-approach from ancestry side by state_transitioning, by requiring exact match on both root hash and slot for fast path

* make more precise the fast path condition

* redo logic to make uniform with BeaconChainDB; fix chronos deprecation warning

* revert not-working replacement of deprecated chronos futures `or`

* switch testnet1 to mainnet
2020-04-20 19:27:52 +02:00
Eugene Kabanov
3d42da90a8
Syncing. (#909) 2020-04-20 16:59:18 +02:00
Ștefan Talpalaru
d01eca677d
Jenkins: update submodules before caching kicks in 2020-04-20 16:34:36 +02:00
Jacek Sieka
edecce1751
disable aggregation when no validators are connected (#908)
* if we don't have validators, don't consider aggregation work
* if we do have validators, don't aggregate when we're out of sync
* when we do aggregate, use a fresh state, and not one from before
sleeping
2020-04-20 14:20:53 +02:00
Jacek Sieka
722c4767fa
travis: ccache (#907) 2020-04-20 14:17:23 +02:00
Ștefan Talpalaru
9974b4bd7b
Jenkins: frak it, we're doing it live 2020-04-19 13:36:11 +02:00
Ștefan Talpalaru
2bb54fc09d
Jenkins fix 2020-04-19 13:22:22 +02:00
Ștefan Talpalaru
3dde68fc20
Jenkins: change "Workspace Cleanup" plugin's strategy 2020-04-19 13:17:02 +02:00
Ștefan Talpalaru
82f28d7718
Jenkins: clean the workspace before starting 2020-04-19 12:48:36 +02:00
Ștefan Talpalaru
a941a4fded
debug Jenkins 2020-04-19 12:41:40 +02:00
Ștefan Talpalaru
1babc00255
new initial submodule update strategy (#904)
* new initial submodule update strategy

* Azure: fix command order
2020-04-19 01:25:21 +02:00