Commit Graph

1720 Commits

Author SHA1 Message Date
Jacek Sieka 6c8f630170
Revert "have each validator randomly pick a stability subnet, per spec (#2194)"
This reverts commit 048a67d525.

Fails with:
```
Error: unhandled exception: /data/beacon-node-builds/devel-large/repo/beacon_chain/nimbus_beacon_node.nim(442, 12) `node.attestationSubnets.stabilitySubnets.len == 0`  [AssertionError]
```
2020-12-18 23:04:31 +01:00
Jacek Sieka 0f8a3a5ae8
checkpoint database at end of each slot (#2195)
* checkpoint database at end of each slot

To avoid spending time on synchronizing with the file system while doing
processing, the manual checkpointing mode turns off fsync during
processing and instead checkpoints the database when the slot has ended.

From an sqlite perspecitve, in WAL mode this guaranees database
consistency but may lead to data loss which is fine - anything missing
from the beacon chain database can be recovered on the next startup.

* log sync status and delay in slot start message

* bump
2020-12-18 22:01:24 +01:00
Jacek Sieka 452042b17b
msgid formatting, libp2p bump (#2198)
* msgid formatting, libp2p bump

* fix libp2p commit
2020-12-18 18:45:27 +00:00
tersec 048a67d525
have each validator randomly pick a stability subnet, per spec (#2194) 2020-12-18 15:46:07 +01:00
Kim De Mey 8cc7effe52
Fix ENR attnets update to only hold persistent subnets (#2193)
* Fix ENR attnets update to only hold persistent subnets

* Use only stability subnet in metadata and enr
2020-12-18 09:50:29 +01:00
Zahary Karadjov 7d95e86c50
Merge branch 'stable' into devel 2020-12-16 22:22:21 +02:00
Zahary Karadjov f06e6d46a2
v1.0.4 2020-12-16 22:20:45 +02:00
Jacek Sieka 5d8cdb88c6
update validator metrics on startup 2020-12-16 20:44:48 +02:00
Jacek Sieka de779c7812 update validator metrics on startup 2020-12-16 19:42:19 +02:00
Jacek Sieka fea2b712f0
cache not needed to validate exits (#2188) 2020-12-16 15:36:02 +01:00
Jacek Sieka 7d5edb4353
use new stew helpers for assignment (#2172)
* bump libp2p (reduces libp2p gossip memory usage to ~1/3)
* use "generic" assign version
2020-12-16 09:37:22 +01:00
Zahary Karadjov c56677867b
Remove some accidentally left debugging changes 2020-12-16 01:09:19 +02:00
Zahary Karadjov 8ebf9c30b0
More complete reset of the web3 provider on each failure; Fix #2184 2020-12-16 00:21:11 +02:00
Jacek Sieka d59fc8ff5b
readd os import, sometimes used 2020-12-15 16:34:56 +01:00
tersec 91786686d5
don't repeat already-included attestations (#2061)
Don't repeat already-included attestations

Also removes the superfluous (and badly scaling) attestation-cache-eviction
2020-12-15 17:16:10 +02:00
Jacek Sieka 4e191a06ac use stew/leb128
* 3 varint parsers -> 1
* 1/3 memory usage / allocations when writing stuff to libp2p
2020-12-15 17:06:15 +02:00
Jacek Sieka 9211b1ca03 import fixes 2020-12-15 16:19:46 +02:00
tersec 8d1443f03c
detect already-aggregate-voted condition before attestation pool; add is_aggregator tests (#2170)
* detect already-aggregate-voted condition before attestation pool; add is_aggregator tests

* replace pair of attestation-per-epoch tracking lists with single list and remove Option use

* fix attestation condition

* use safer type conversions; add more is_aggregator tests
2020-12-14 20:58:32 +00:00
Ștefan Talpalaru 9daf6be73c
graceful exit on SIGTERM (#2178)
Much easier than convincing all users to change the default signal in
their service definition file to SIGINT.
2020-12-14 16:45:14 +00:00
Giovanni Petrantoni c91091197f
Lower gossipFactor since we default to 160 peers (#2169)
And it might potentially send a bit more IHAVE messages then wanted.
2020-12-11 12:15:34 +01:00
Jacek Sieka bc977799f6 Log warning when running without metrics support 2020-12-10 17:22:29 +02:00
Zahary Karadjov 91741326cc
v1.0.3 2020-12-10 02:56:02 +02:00
Zahary Karadjov 57fdd87a24 Log Eth1 request failures more precisely 2020-12-10 02:54:58 +02:00
Zahary Karadjov 7979fb79d3 Add some useful Eth1 monitor metrics 2020-12-10 02:54:58 +02:00
Zahary Karadjov 983b3c9fbf Add a 'we3 test' command for verifying the compatibility of a web3 provider 2020-12-10 02:54:58 +02:00
Kim De Mey 0ec90b26a5
Update ENR record with metadata attnets at each attestation subnet cycle (#2148) 2020-12-09 10:13:51 +01:00
tersec 8b8b25ddac
always check whether gossip should be enabled in onslotstart (#2162) 2020-12-08 18:11:54 +01:00
Jacek Sieka f10f29d829
avoid some futures and copies in processing pipeline (#2157)
`addLastNoWait` does the same thing with less task scheduling and
copying
2020-12-08 09:59:40 +01:00
Dustin Brody 3b06e0f657 directly construct cstring in err() expressions 2020-12-07 22:24:06 +02:00
tersec 72d00e33b8
always wait for 2/3 into slot for aggregate attestations (#2158) 2020-12-07 20:32:34 +01:00
Eugene Kabanov 81c25ca4cd
Disconnect peers on status handshake timeout. (#2149) 2020-12-07 19:47:07 +01:00
Eugene Kabanov 46c2740097
Documentation for Validators API. (#2147)
* Recover proper validator API call and remove incorrect one.
Add more examples to API documentation.
2020-12-07 14:51:14 +02:00
Dustin Brody 32a18769e6 remove waitFor in attestation subnet cycling 2020-12-07 14:48:04 +02:00
tersec 26ea76bbdf
don't lag aggregated attestations by a slot (#2150)
* don't lag aggregated attestations by a slot

* don't use aggregation topic at all

* use aggregates again, but with aggressively low ATTESTATION_PROPAGATION_SLOT_RANGE; seems to hold on to LH 1.0 nodes

* clean up scaffolding and double ATTESTATION_PROPAGATION_SLOT_RANGE to 16

* increase ATTESTATION_PROPAGATION_SLOT_RANGE to 24

* increase ATTESTATION_PROPAGATION_SLOT_RANGE to 28 and isolate in only used function due to customization; remove TRAILING_DISTANCE machinery
2020-12-07 12:56:49 +01:00
Zahary Karadjov 7363bf3c7c
Fix a logical typo in beacon_api/validator.getStatus 2020-12-03 21:24:01 +02:00
Zahary Karadjov d302c1d8ee
v1.0.2 2020-12-03 19:17:32 +02:00
Zahary Karadjov f4a3a36731
More specific error message when block validation fails 2020-12-03 11:51:42 +02:00
Ștefan Talpalaru be107df7f1 status bar: actually display it every second, after updating its data 2020-12-03 11:41:40 +02:00
cheatfate 42dd59852b Fix compilation problem. 2020-12-03 11:40:32 +02:00
cheatfate 0904ee5609 get_v1_validator_attestation_data() implementation. 2020-12-03 11:40:32 +02:00
cheatfate f6285d8381 Fix proposeSignedBlock() become sync. 2020-12-03 11:40:08 +02:00
cheatfate 088b001388 Fix get_v1_beacon_states_stateId_committees_epoch() to use optional arguments properly.
Optimize get_v1_beacon_pool_attestations() a bit.
2020-12-03 11:40:08 +02:00
cheatfate 21ef0f3a57 Fixes for get_v1_beacon_states_stateId_validators() and get_v1_beacon_states_stateId_validators_validatorId(). 2020-12-03 11:40:08 +02:00
cheatfate 9afafb54f5 get_v1_beacon_states_stateId_validator_balances() implementation. 2020-12-03 11:40:08 +02:00
cheatfate 5dade7b441 post_v1_beacon_blocks() implementation. 2020-12-03 11:40:08 +02:00
cheatfate 5f392893c0 Change approach for get_v1_beacon_pool_attestations() to use hexadecimal strings. 2020-12-03 11:40:08 +02:00
cheatfate 9e7f8b37c7 get_v1_beacon_pool_attestations() implementation. 2020-12-03 11:40:08 +02:00
cheatfate 7e4ff7a740 Implementation of post_v1_beacon_pool_attester_slashings() and post_v1_beacon_pool_proposer_slashings().
Remove doAssert.
2020-12-03 11:40:08 +02:00
cheatfate 6cae25701b Implementation of beacon_pool_attester_slashings(), beacon_pool_proposer_slashings() and beacon_pool_voluntary_exits(). 2020-12-03 11:40:08 +02:00
Zahary Karadjov 027d2547cd Fix a block proposal issue caused by incorrect merkle proofs
The key change here is that `addChunksAndGenMerkleProofs` is called
with all pending deposits instead of just the deposits included in
the block. The later was effectively producing merkle proofs against
a different root.
2020-12-03 11:25:52 +02:00
Zahary Karadjov 43662be3f8
v1.0.1 2020-12-02 14:31:13 +02:00
tersec d5e3712609
log errors in more detail when block production fails (#2135)
* log errors in more detail when block production fails

* pass through block header error

* make everything cstring
2020-12-02 13:23:10 +01:00
Sacha Saint-Leger 94c66f15a9
Wanted peers log message fix (#2114)
* Edit Peer count low wanted peers log

* remove prometheus folder

* remove prometheus.yml

* Add current peers to low peer count log

* change to 25% peer count notice, 12.5% warning

Co-authored-by: kdeme <kim.demey@gmail.com>
2020-12-02 10:47:23 +01:00
Zahary Karadjov 92655f936a Add backoff for the deposit logs requests as well; More debug logs 2020-12-02 01:43:54 +02:00
Zahary Karadjov 7225a6d6ed Retry all web3 requests up to 3 times with exponential backoff 2020-12-02 01:43:54 +02:00
cheatfate f3b8c777a7 Strip whitespace. 2020-12-02 01:43:36 +02:00
Zahary Karadjov 4feb0a308e Fix #2125 (ETH status bar display); Bump LibP2P 2020-12-02 00:03:59 +02:00
Dustin Brody 3e4b49462b detect excessively long graffiti strings 2020-12-01 15:39:46 +02:00
Dustin Brody 68c91d1d1b don't wait until after the first slot to enable gossip 2020-12-01 15:39:03 +02:00
Zahary Karadjov 2fd71a9aac Reduce the rate of requesting deposits to lower the risk of triggering the Infura rate limits 2020-12-01 15:38:50 +02:00
Zahary Karadjov 4eaf29875a Less risk for Eth1-induced delays in block proposal 2020-12-01 15:38:50 +02:00
Zahary Karadjov 38f7558e50 Work around a strange codegen issue to fix local sim in CI; Bump LibP2P 2020-12-01 15:38:00 +02:00
Zahary Karadjov 3bdda3dd46
Hotfix: use the mainnet bootstrap nodes without specifying --network=mainnet explicitly 2020-12-01 10:44:30 +02:00
Zahary Karadjov 4328576e18
Hotfix: 'deposits import' was ignoring its arguments in Linux builds 2020-12-01 00:59:57 +02:00
Zahary Karadjov 8327240458
v1.0.0-stateofus 2020-11-29 23:50:13 +02:00
Zahary Karadjov ac9bdde543
Don't rely on a metric value for the ETH display in the status bar 2020-11-29 23:35:39 +02:00
zah cabb07a186 Apply suggestions from code review
Co-authored-by: Sacha Saint-Leger <sacha@status.im>
2020-11-29 23:08:07 +02:00
Zahary Karadjov 3c0dfc2fbe Implement the 'deposits exit' command; Remove 'deposits create' 2020-11-29 23:08:07 +02:00
Jacek Sieka a2364ce1bc
add getGossipSubPeers RPC (#2107) 2020-11-29 20:07:20 +01:00
Zahary Karadjov ae19ab72c0 Implement #2067 2020-11-29 18:27:26 +02:00
Jacek Sieka c3bfe15a35
increase default max peers (#2106)
* increase default max peers

also avoid reconnection when opening stream as this might induce
a loop

* Use dial without addresses

* dial back max peers a little
2020-11-29 14:43:41 +01:00
cheatfate 7731109320 Remove getSpecPreset() because of crash and add missing calls into api.md 2020-11-29 11:01:55 +02:00
Zahary Karadjov bf2673abc4
Restore the ETH display in the status bar 2020-11-28 20:53:51 +02:00
Jacek Sieka 8685eb7042 add validator balance metrics 2020-11-28 16:22:09 +02:00
Eugene Kabanov db030eb944
Setup agent/proto version identifiers in libp2p. (#2079)
* Supply `nim-libp2p` with `agent-string` and `proto-string` as configured parameters.

* Remove protoString and rng check.

* Fix compilation problem.
2020-11-28 08:00:36 +01:00
Jacek Sieka e7f2735271
fix broken metrics during replay (#2090)
* move metrics out of state transition
* add validator count metric
* remove expensive beacon_current_validators, beacon_previous_validators
metrics (they should be reimplemented with cache), add cheap
beacon_active_validators to approximate
* remove unused validator count metrics
* tidy imports/defects
2020-11-27 23:16:13 +01:00
tersec 2421d338c1
node.network.metadata.attnets is only secondary source of truth (#2089) 2020-11-27 15:54:13 +01:00
Jacek Sieka d16e127daf
Peer stuff (#2084)
* Revert "Revert "Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065)" (#2082)"

This reverts commit 7cc3dc8027.

* fix nil disconnectedFut crash

* fixes

don't resetPeer, it causes peer miscounts

* disconnect disconnecting peers

...when there's a race.

* avoid connection spamming

* never decrease SeenTable timeout
* only recover ENR for known peers

* seen only when really disconnected
2020-11-26 20:23:45 +01:00
Giovanni Petrantoni 9b9aa93cd4
Seenttl by spec (#2083)
* add rpc call to get chronos futures at runtime

* add seenTTL gossip parameter

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-11-26 13:08:22 +01:00
Eugene Kabanov 687cbaf94c
Adjust number of sync workers from 20 to 10. (#2077)
Adjust watch task min-pause-time to 1.minute.
Slowed down pause-time recovery by factor 3/4 instead of 1/2.
2020-11-26 09:19:27 +01:00
Jacek Sieka 7cc3dc8027
Revert "Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065)" (#2082)
This reverts commit d041287a4f.
2020-11-26 09:05:23 +01:00
Eugene Kabanov d041287a4f
Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065)
* Initial commit.

* Fix log lines and compilation error.

* Add get_v1_node_peers() implementation.

* Fix peer's lifetime states.

* Use the most recent multiaddress.

* Fix assign NewPeerScore again.
Fix compilation error with last seen address.
Fix Peer upgraded log line place.

* syncing, health, peer_count, peer_id and fixes for identity.

* Fix compilation problems.

* Move object declaration to callsigs.
Fix identity addresses fields.

* Finish node RPC calls.

* Avoid leak of lifetime future.

* Bump chronos.

* Fix json generator problem.
2020-11-26 08:09:59 +01:00
kdeme e69b5ff473 Add a record create and print command 2020-11-25 18:32:59 +02:00
cheatfate 3a13750734 Try to obtain deposit contract address from config.
Attempt to fix json generation.
2020-11-25 18:30:44 +02:00
cheatfate 68f4ad64a4 Disable exception if depositAddress is not available. 2020-11-25 18:30:44 +02:00
cheatfate af3862da48 Make depositAddress public in eth1_monitor.
Fix representation of config.
Add deposit_contract implementation.
2020-11-25 18:30:44 +02:00
cheatfate 8eafbb4e09 Add config_spec implementation. 2020-11-25 18:30:44 +02:00
Zahary Karadjov 3594fa2a22
Version 1.0.0-rc1 2020-11-25 03:13:58 +02:00
Zahary Karadjov b9e4fef616
Check that the selected data dir is compatible with the selected network 2020-11-25 03:13:58 +02:00
Jacek Sieka e1e4871287 fix time formatting and pre-genesis ops 2020-11-25 01:51:46 +02:00
Zahary Karadjov 151d116f02
Fix off-by-one error in Eth1 pruning 2020-11-25 01:51:17 +02:00
Zahary Karadjov a995cdd3d9
Bugfix: fix an incorrect balance increase during genesis state generation 2020-11-24 23:30:26 +02:00
Zahary Karadjov 594ef713a0
Increase the web3 timeouts 2020-11-24 23:28:20 +02:00
zah 372c9b798c
Fix the corrupted database state on Pyrmont nodes; Add mainnet genesis (#2056)
* Handle some web3 timeouts better

* Add support for developer .env files

* Eth1 improvements; Mainnet genesis state

Notable changes:

* The deposits table have been removed from the database. The client
  will no longer process all deposits on start-up.

* The network metadata now includes a "state snapshot" of the deposit
  contract. This allows the client to skip syncing deposits made prior
  to the snapshot (i.e. genesis). Suitable metadata added for Pyrmont
  and Mainnet.

* The Eth1 monitor won't be started unless there are validators attached
  to the node.

* The genesis detection code is now optional and disabled by default

* Bugfix: The client should not produce blocks that will fail validation
  when it hasn't downloaded the latest deposits yet

* Bugfix: Work around the database corruption affecting Pyrmont nodes

* Remove metadata for Toledo and Medalla
2020-11-24 22:21:47 +01:00
tersec 040093a945
medalla -> pyrmont (#2062) 2020-11-23 17:46:51 +01:00
Giovanni Petrantoni 6f476ca380
Gossip11 params (#2047)
* add rpc call to get chronos futures at runtime

* add eth2 spec gossip parameters

* bump libp2p

* fix url, remove 1.0

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-11-23 10:57:03 +01:00
Jacek Sieka 672915e170
work around long pyrmont startup time (#2060)
* also fixes unnecessary copy/memory alloc when loading DbSeq entries
2020-11-21 18:53:40 +01:00
Zahary Karadjov 00aedddf2a
Version 0.6.6 2020-11-20 22:31:03 +02:00
Jacek Sieka dbcc0686ff delay pruning of cache for finalized epoch (fixes #2049) 2020-11-20 20:57:50 +02:00
Zahary Karadjov 316a19af5f Address review comments 2020-11-20 17:13:51 +02:00