Commit Graph

258 Commits

Author SHA1 Message Date
Mamy Ratsimbazafy 70a03658e3
Block validation flow v2 + Batch (serial) sig verification (#2250)
* bump nim-blscurve

* Outline the block validation flow

* introduce the SigVerified types, pass the tests

* Split clearance/quarantine to prepare for batch crypto verif

* Add a batch signature collector

* Make clearance use SigVerified block and split verification between crypto and state transition

* Always use signedBeaconBlock for the onBlockAdded callback

* RANDAO signing_root is the epoch instead of the full block

* Support skipping BLS for testing

* Fix compilation of the validator client

* Try to fix strange errors MacOS and Jenkins (Clang, unknown type name br_hmac_drbg_context in stdlib_assertions.nim.c)

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561819858

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828025

* onBlockAdded callback should use TrustedSignedBeaconBlock https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561837261

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828946

* Use the application RNG: https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561815336

* Improve codegen of conversion zero-cost)

* Quick fixes with loadWithCache after #2259 (TODO: graceful error since pubkey validations is now done first in signatures_batch)

* Graceful handle rogue pubkeys and signatures now that those are lazy-loaded
2021-01-25 20:45:48 +02:00
tersec 7d74d3bfbc
only subscribe to subnets when aggregating (#2254)
Only subscribe to subnets when aggregating
2021-01-25 19:39:56 +02:00
Zahary Karadjov 960666d1ed
Remove std/random again 2021-01-21 19:39:04 +02:00
Mamy Ratsimbazafy 718feef802
Fix unstable after #2244 (#2255) 2021-01-21 18:27:24 +01:00
Dustin Brody a16f5afcd5 pre-emptive duplicate validator detection heuristic 2021-01-21 16:03:02 +02:00
tersec 55ecb61c3a cycle attestation subnets every slot (#2240)
Cycle attestation subnets every slot
2021-01-19 19:44:03 +02:00
tersec 0fce8ad0d7
Revert "only checkpoint every four slots (#2236)" (#2242)
This reverts commit 7da16f4908.
2021-01-18 11:02:56 +01:00
Zahary Karadjov c8c819359c
More clear error message when a validator exit was rejected 2021-01-15 19:40:05 +02:00
tersec 7da16f4908
only checkpoint every four slots (#2236)
* only checkpoint every four slots

* only checkpoint every 16 slots

* every 8 slots

* every 4 slots; 8 seems probably okay, but be a bit conservative
2021-01-15 05:23:54 +00:00
Giovanni Petrantoni 295e3c9c73
Topics validation and direct peers (#2237)
* pick the right libp2p branch

* add topics validation
2021-01-15 04:17:06 +00:00
tersec fa75c477cd
only initially subscribe to relevant attestation subnets (#2231) 2021-01-14 09:43:21 +01:00
tersec 0fad1b6b26
don't special-case zero-validator subnet cycling (#2230) 2021-01-12 17:17:43 +01:00
tersec dde973e2d4
allow always-on subscription to all attestation subnets when gossiping (#2225)
* allow always-on subscription to all attestation subnets when gossiping

* in subscribe-all-subnets mode, consider all subnets to be stability subnets for ENR purposes
2021-01-12 13:43:15 +01:00
Giovanni Petrantoni a3a651b565
always enable topic and aggreate metric topics (#2229) 2021-01-12 04:27:09 +01:00
Zahary Karadjov 338428cbd7 Add Eth1 deposits simulation to block_sim 2021-01-04 13:22:00 +02:00
Giovanni Petrantoni ed24f60f70
remove async from sub/unsub (#2197)
* remove await/async from sub/unsub

* fix unsubscribe wrong key (missed _snappy)

* use the right libp2p commit hash

* remove unused async

* fix inspector

* fix subnet calculation in RPC and insert broadcast attestations into node's pool

* unify codepaths to ensure only mostly-checked-to-be-valid attestations enter the pool, even from node's own broadcasts

* update attestation pool tests for new validateAttestation param

Co-authored-by: Dustin Brody <tersec@users.noreply.github.com>
2020-12-24 09:48:52 +01:00
tersec afbaa36ef7
make subnet cycling more robust; use one stability subnet/validator; explicitly represent gossip enabled/disabled (#2201)
* make subnet cycling more robust; use one stability subnet/validator; explicitly represent gossip enabled/disabled

* fix asymmetry in _snappy being used for subscriptions but not unsubscriptions

* remove redundant comment

* minimal RPC and VC support for infoming BN of subnets

* create and verify slot signatures in RPC interface and VC

* loosen old slot check

* because Slot + uint64 works but uint64 + Slot doesn't

* document assumptions for head state use; don't clear stability subnets; guard against VC not having checked an epoch ahead, fixing a crash; clarify unsigned comparison

* revert unsub fix
2020-12-22 10:05:36 +01:00
Jacek Sieka 6c8f630170
Revert "have each validator randomly pick a stability subnet, per spec (#2194)"
This reverts commit 048a67d525.

Fails with:
```
Error: unhandled exception: /data/beacon-node-builds/devel-large/repo/beacon_chain/nimbus_beacon_node.nim(442, 12) `node.attestationSubnets.stabilitySubnets.len == 0`  [AssertionError]
```
2020-12-18 23:04:31 +01:00
Jacek Sieka 0f8a3a5ae8
checkpoint database at end of each slot (#2195)
* checkpoint database at end of each slot

To avoid spending time on synchronizing with the file system while doing
processing, the manual checkpointing mode turns off fsync during
processing and instead checkpoints the database when the slot has ended.

From an sqlite perspecitve, in WAL mode this guaranees database
consistency but may lead to data loss which is fine - anything missing
from the beacon chain database can be recovered on the next startup.

* log sync status and delay in slot start message

* bump
2020-12-18 22:01:24 +01:00
tersec 048a67d525
have each validator randomly pick a stability subnet, per spec (#2194) 2020-12-18 15:46:07 +01:00
Kim De Mey 8cc7effe52
Fix ENR attnets update to only hold persistent subnets (#2193)
* Fix ENR attnets update to only hold persistent subnets

* Use only stability subnet in metadata and enr
2020-12-18 09:50:29 +01:00
Zahary Karadjov 7d95e86c50
Merge branch 'stable' into devel 2020-12-16 22:22:21 +02:00
Jacek Sieka 5d8cdb88c6
update validator metrics on startup 2020-12-16 20:44:48 +02:00
Jacek Sieka de779c7812 update validator metrics on startup 2020-12-16 19:42:19 +02:00
Jacek Sieka 7d5edb4353
use new stew helpers for assignment (#2172)
* bump libp2p (reduces libp2p gossip memory usage to ~1/3)
* use "generic" assign version
2020-12-16 09:37:22 +01:00
Zahary Karadjov 8ebf9c30b0
More complete reset of the web3 provider on each failure; Fix #2184 2020-12-16 00:21:11 +02:00
Ștefan Talpalaru 9daf6be73c
graceful exit on SIGTERM (#2178)
Much easier than convincing all users to change the default signal in
their service definition file to SIGINT.
2020-12-14 16:45:14 +00:00
Jacek Sieka bc977799f6 Log warning when running without metrics support 2020-12-10 17:22:29 +02:00
Zahary Karadjov 983b3c9fbf Add a 'we3 test' command for verifying the compatibility of a web3 provider 2020-12-10 02:54:58 +02:00
Kim De Mey 0ec90b26a5
Update ENR record with metadata attnets at each attestation subnet cycle (#2148) 2020-12-09 10:13:51 +01:00
tersec 8b8b25ddac
always check whether gossip should be enabled in onslotstart (#2162) 2020-12-08 18:11:54 +01:00
Dustin Brody 32a18769e6 remove waitFor in attestation subnet cycling 2020-12-07 14:48:04 +02:00
Ștefan Talpalaru be107df7f1 status bar: actually display it every second, after updating its data 2020-12-03 11:41:40 +02:00
Zahary Karadjov 4feb0a308e Fix #2125 (ETH status bar display); Bump LibP2P 2020-12-02 00:03:59 +02:00
Dustin Brody 68c91d1d1b don't wait until after the first slot to enable gossip 2020-12-01 15:39:03 +02:00
Zahary Karadjov 38f7558e50 Work around a strange codegen issue to fix local sim in CI; Bump LibP2P 2020-12-01 15:38:00 +02:00
Zahary Karadjov 3bdda3dd46
Hotfix: use the mainnet bootstrap nodes without specifying --network=mainnet explicitly 2020-12-01 10:44:30 +02:00
Zahary Karadjov 4328576e18
Hotfix: 'deposits import' was ignoring its arguments in Linux builds 2020-12-01 00:59:57 +02:00
Zahary Karadjov ac9bdde543
Don't rely on a metric value for the ETH display in the status bar 2020-11-29 23:35:39 +02:00
zah cabb07a186 Apply suggestions from code review
Co-authored-by: Sacha Saint-Leger <sacha@status.im>
2020-11-29 23:08:07 +02:00
Zahary Karadjov 3c0dfc2fbe Implement the 'deposits exit' command; Remove 'deposits create' 2020-11-29 23:08:07 +02:00
Zahary Karadjov ae19ab72c0 Implement #2067 2020-11-29 18:27:26 +02:00
Zahary Karadjov bf2673abc4
Restore the ETH display in the status bar 2020-11-28 20:53:51 +02:00
Jacek Sieka e7f2735271
fix broken metrics during replay (#2090)
* move metrics out of state transition
* add validator count metric
* remove expensive beacon_current_validators, beacon_previous_validators
metrics (they should be reimplemented with cache), add cheap
beacon_active_validators to approximate
* remove unused validator count metrics
* tidy imports/defects
2020-11-27 23:16:13 +01:00
tersec 2421d338c1
node.network.metadata.attnets is only secondary source of truth (#2089) 2020-11-27 15:54:13 +01:00
kdeme e69b5ff473 Add a record create and print command 2020-11-25 18:32:59 +02:00
Zahary Karadjov 3594fa2a22
Version 1.0.0-rc1 2020-11-25 03:13:58 +02:00
Zahary Karadjov b9e4fef616
Check that the selected data dir is compatible with the selected network 2020-11-25 03:13:58 +02:00
zah 372c9b798c
Fix the corrupted database state on Pyrmont nodes; Add mainnet genesis (#2056)
* Handle some web3 timeouts better

* Add support for developer .env files

* Eth1 improvements; Mainnet genesis state

Notable changes:

* The deposits table have been removed from the database. The client
  will no longer process all deposits on start-up.

* The network metadata now includes a "state snapshot" of the deposit
  contract. This allows the client to skip syncing deposits made prior
  to the snapshot (i.e. genesis). Suitable metadata added for Pyrmont
  and Mainnet.

* The Eth1 monitor won't be started unless there are validators attached
  to the node.

* The genesis detection code is now optional and disabled by default

* Bugfix: The client should not produce blocks that will fail validation
  when it hasn't downloaded the latest deposits yet

* Bugfix: Work around the database corruption affecting Pyrmont nodes

* Remove metadata for Toledo and Medalla
2020-11-24 22:21:47 +01:00
tersec 54c388b7b4
close slashing protection database (#2050) 2020-11-20 14:23:55 +01:00
Jacek Sieka a6b188bfd4
misc fixes (#2027)
* log when database is loading (to avoid confusion)
* generate network keys later during startup
* fix quarantine not scheduling chain of parents for download and
increase size to one epoch
* log validator count, enr and peerid more clearly on startup
2020-11-16 20:15:43 +01:00
tersec 21c4ce8fd4
remove superfluous TODOs/not-really-TODOs, type conversion, imports (#2025) 2020-11-16 17:10:51 +01:00
tersec e2f161dbf7
fix attestation sending schedules to avoid timing attack (#1995) 2020-11-16 10:44:18 +01:00
Zahary Karadjov b022dc4d1f Use O(n) algorithm in initialize_beacon_state_from_eth1; Avoid unnecessary merkle proofs generation 2020-11-15 21:40:40 +02:00
Zahary Karadjov 17d35e1fd9 Allow the node to start when it fails to initialize the Eth1 monitor
* Avoid hangs when wss:// is specified for a non-secure HTTP server
* Produce an ERROR when the web3 provider is unsupported, but still launch the node
2020-11-12 22:29:43 +02:00
Zahary Karadjov 389c11743a
Review TODO items and self-assign the most important ones 2020-11-10 20:41:04 +02:00
tersec 271df8b604
bump 1.0.0rc-0 spec refs to 1.0.0 (#1974) 2020-11-09 14:18:55 +00:00
Zahary Karadjov e9b9cd75ee Rename binaries; Mimic the original repo layout in the distribution 2020-11-09 11:38:52 +02:00