Commit Graph

572 Commits

Author SHA1 Message Date
Jacek Sieka d6f317950c
Compute state root instead of loading from database (#1297)
htr is fast now, so hitting the database to load the state root is no
longer motivated - the more simple code is less vulnerable to database
corruption.
2020-07-10 22:47:39 +02:00
Zahary Karadjov 3ec6a02b12
Merge devel and resolve conflicts 2020-07-10 02:02:40 +03:00
Zahary Karadjov 540b2828b2
Adapt the local sim scripts to use the new run-time presets 2020-07-10 01:08:54 +03:00
tersec 61b0b5af17
update most remaining non-fork-choice spec refs, updating code where necessary (#1292)
* update most of the remaining non-fork-choice spec refs, updating code where necessary

* revert presumably harmless compute_signing_root() change, but this way, keep things really unchanged outside inspector
2020-07-09 11:43:27 +00:00
Mamy Ratsimbazafy 3cdae9f6be
Dual headed fork choice [Revolution] (#1238)
* Dual headed fork choice

* fix finalizedEpoch not moving

* reduce fork choice verbosity

* Add failing tests due to pruning

* Properly handle duplicate blocks in sync

* test_block_pool also add a test for duplicate blocks

* comments addressing review

* Fix fork choice v2, was missing integrating block proposed

* remove a spurious debug writeStackTrace

* update block_sim

* Use OrderedTable to ensure that we always load parents before children in fork choice

* Load the DAG data in fork choice at init if there is some (can sync witti)

* Cluster of quarantined blocks were not properly added to the fork choice

* Workaround async gcsafe warnings

* Update blockpoool tests

* Do the callback before clearing the quarantine

* Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads

* Make it work with latest devel - Altona readyness

* Add a recovery mechanism when forkchoice desyncs with blockpool

* add the current problematic node to the stack

* Fix rebase indentation bug (but still producing invalid block)

* Fix cache at epoch boundaries and lateBlock addition
2020-07-09 11:29:32 +02:00
Zahary Karadjov 318b225ccd
Merge devel and resolve the conflicts 2020-07-08 15:36:03 +03:00
tersec b7d1967449
test for correctly skipping, without failure, deposits with invalid signatures (#1288) 2020-07-08 06:55:23 +02:00
Zahary Karadjov c4af4e2f35
Working test suite with run-time presets 2020-07-08 02:02:14 +03:00
Jacek Sieka 6fe0a623f5
Crypto rng (#1284)
* use bearssl rng throughout

* bump

* bump

* move keygen out of crypto
2020-07-07 17:51:02 +02:00
Eugene Kabanov 293d990d43
Fix PeerPool issue with peers overflow maxPeers setting. (#1285) 2020-07-07 11:49:08 +02:00
tersec a92276d510
adopt Result[void, string] in place of some bool return signatures (#1275)
* adopt Result[void, string] in place of some bool return signatures

* string -> cstring to reduce memory allocations; ensure all err() strings are constants, with contextual information from higher-level callers

* logScope usage fixes

* homogenize err() reporting convention

* invalid signature in deposit isn't an error
2020-07-03 17:03:14 +00:00
tersec c64737e7f2
implement aggregated attestation receiving/validating (#1272)
* implement aggregated attestation receiving/validating

* document the conditions without explicit implementations in isValidAggregatedAttestation()
2020-07-02 16:15:27 +00:00
Jacek Sieka f3e92762e3
add tests for unviable blocks (#1271)
* add tests for unviable blocks

also enable finalization tests in all test configs - they're plenty fast
now
also fix newClone for non-rvo cases. sigh.

* fixes
2020-07-01 19:00:14 +02:00
tersec b4db2ad693
remove v0.11.3 support; add block_sim to CI (#1253)
* remove v0.11.3 support; add block_sim to CI

* rm stray PERSISTENT_COMMITTEE_PERIOD

* remove TopicFilter.InteropAttestations

* bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
Zahary Karadjov 302cef5e90 Attach all local validators before starting the networking 2020-06-29 18:07:41 +03:00
Zahary Karadjov 8140b4458c Working local sim with WAIT_GENESIS=yes 2020-06-29 02:18:48 +03:00
Zahary Karadjov 9cd8a6c730 Working local sim (without waiting genesis) 2020-06-29 02:18:48 +03:00
Zahary Karadjov 8288b568d2
Fix more Tmux issues disovered when testing on a fresh machine [skip ci] 2020-06-25 15:58:13 +03:00
Jacek Sieka 1301600341
Trusted blocks (#1227)
* cleanups

* fix ncli state root check flag
* add block dump to ncli_db
* limit ncli_db benchmark length
* tone down finalization logs

* introduce trusted blocks

We only store blocks whose signature we've verified in the database - as
such, there's no need to check it again, and most importantly, no need
to deserialize the signature when loading from database.

50x startup time improvement, 200x block load time improvement.

* fix rewinding when deposits have invalid signature
* speed up ancestor iteration by avoiding copy
* avoid deserializing signatures for trusted data
* load blocks lazily when rewinding (less memory used)

* chronicles workarounds

* document trustedbeaconblock
2020-06-25 12:23:10 +02:00
Zahary Karadjov b67a506b3f
Fixes for local sim on macOS [skip ci] 2020-06-25 12:42:30 +03:00
Mamy Ratsimbazafy 902093f57c
Revert "Dual headed fork choice [Reloaded] (#1223)" (#1234)
This reverts commit 6836d41ebd.
2020-06-25 11:36:03 +02:00
Mamy Ratsimbazafy 6836d41ebd
Dual headed fork choice [Reloaded] (#1223)
* Dual headed fork choice

* fix finalizedEpoch not moving

* reduce fork choice verbosity

* Add failing tests due to pruning

* Properly handle duplicate blocks in sync

* test_block_pool also add a test for duplicate blocks

* comments addressing review
2020-06-24 20:24:36 +02:00
tersec 807b920c19
state_transition implements the spec fairly directly (#1220) 2020-06-23 13:54:24 +00:00
Zahary Karadjov eb48c05b7f Don't assign validator keys to the bootstrap node; Ganache mode for launch_local_testnet 2020-06-23 07:59:35 +00:00
Zahary Karadjov 7211a99086 Smooth out the differences between Ganache and Infura; Working local sim and Altona target 2020-06-22 17:30:04 +03:00
Zahary Karadjov e65c1f49ce Allow Ganache to bundle more deposits in a single block 2020-06-22 17:30:04 +03:00
Zahary Karadjov ffb613d3c6 Usability enhancements for Tmux
* Press 'q' to quit Tmux
* In case of errors, Tmux will wait for a keypress before closing a pane
2020-06-22 17:30:04 +03:00
Zahary Karadjov 01ea66f24e Some documentation and better defaults 2020-06-22 17:30:04 +03:00
Zahary Karadjov 3d9aff5d23 Fix some logical typos introduced in the rebase 2020-06-22 17:30:04 +03:00
Zahary Karadjov 1def383ad1 Reword the Tmux setup to handle all simulation scenarios in more visible way 2020-06-22 17:30:04 +03:00
Zahary Karadjov e9d68e2f7b Add simulation mode that bootstraps from an Eth1 ganache instance 2020-06-22 17:30:04 +03:00
Zahary Karadjov 7d3160b0b5 Local sim now includes a prometheus server, a Ganache instance and a simulation dashboard server (in Tmux mode) 2020-06-22 17:30:04 +03:00
tersec 7b4e129316
re-add minimal constant checking; organize presets by spec version (#1203)
* re-add minimal preset constant checking; organize presets to better support multiple spec versions

* bump spec ref

* increase Azure timeout to 90 minutes to accomodate Nim compiler building
2020-06-20 09:12:45 +02:00
Viktor Kirilov 72dfe7f578 - updated the validator shell script after the keystore changes
- better logging & retrying requests on the VC side if the BN fails for some reason
- VC now fetches the attestation duties 1 epoch in advance - in the future it will tell the BN to subscribe to the appropriate attestation topics in advance based on that info
- a bunch of other code cleanup & fixes such as better naming for consoles when using multitail, etc.

reviewed in PR #1184 - proper review of the API & VC are pending
2020-06-19 12:21:22 +03:00
tersec dc1a565b3f
support v0.12.1 attestation topics in beacon node/inspector subscribing (#1187)
* support v0.12.1 attestation topics in beacon node and inspector subscribing

* bump is_valid_merkle_branch() spec ref
2020-06-18 15:10:25 +02:00
Eugene Kabanov 4436c85ff7
Forward sync refactoring. (#1191)
* Forward sync refactoring.
Rename Quarantine.pending to Quarantine.orphans.
Removing "old" fields.

* Fix test's FetchRecord.

* Fix `checkResponse` to not allow duplicates in response.
2020-06-18 12:03:36 +02:00
tersec ee9f4a2e3f
remove skipMerkleValidation and skipBlockParentRootValidation (#1197) 2020-06-18 07:56:47 +02:00
Dustin Brody 044ced53be ensure SPEC_VERSION stays synchronized with actual spec tested 2020-06-16 15:43:40 +00:00
Jacek Sieka 60176b8cc1
Revert "Dual headed fork choice (#1163)" (#1181)
This reverts commit 090f06614a.
2020-06-16 09:46:00 +02:00
Jacek Sieka 89e4819ce9
collect signature production and verificaiton in one place (#1179)
* collect signature production and verificaiton in one place

Signatures are made over data and domain - here we collect all such
activities in one place.

Also:
* security: fix cast-before-range-check
* log block/attestation verification consistently
* run block verification based on `getProposer` in its own history
* clean up some unused stuff

* import

* missing raises
2020-06-16 07:45:04 +02:00
Mamy Ratsimbazafy 090f06614a
Dual headed fork choice (#1163)
* Dual headed fork choice

* fix finalizedEpoch not moving

* reduce fork choice verbosity
2020-06-16 00:40:16 +02:00
Ștefan Talpalaru d9b494dff6
run_node.sh: macOS fix 2020-06-13 02:50:35 +02:00
Jacek Sieka 42832cefa8
Small fixes (#1165)
* random fixes

* create dump dir on startup
* don't crash on failure to write dump
* fix a few `uint64` instances being used when indexing arrays - this
should be a compile error but isn't due to compiler bugs
* fix standalone test_block_pool compilation
* add signed block processing in ncli

* reuse cache entry instead of allocating a new one

* allow for small clock disparities when validating blocks
2020-06-12 18:43:20 +02:00
tersec c8f24ae3b8
Remove three skipMerkleValidation usages (#1164)
* remove three skipMerkleValidation usages

* remove a couple obsolete comments/TODOs
2020-06-12 18:03:46 +02:00
Ștefan Talpalaru 33d55fac08
eth2_network_simulation: mkdir + cosmetic changes 2020-06-12 16:37:36 +02:00
Zahary Karadjov cf6a869e9e Address some TODO items; Handle start-up before genesis more properly 2020-06-11 17:40:08 +03:00
Zahary Karadjov fdaf419e41 Address review comments 2020-06-11 17:40:08 +03:00
Zahary Karadjov 37b4739547 Include hash_tree_root in the SSZ fuzzing tests 2020-06-11 17:40:08 +03:00
Zahary Karadjov a8113cf2bc Restore the local sim to a working state 2020-06-11 17:40:08 +03:00
Zahary Karadjov 17343442ea Implement more of the KeyStore spec and integrate it in the beacon node 2020-06-11 17:40:08 +03:00