c0e11cfba6
shorten idle timeout
2021-01-18 22:27:34 +02:00
76f15302a7
better timesharing in eth2 processor
...
* use `idleAsync` to more evenly divide cpu attention when syncing in
particular - this gives networking better latency
* more strict exception handling in eth2_processor
2021-01-18 22:27:34 +02:00
6fabefa76f
Fix discv5 loop in case of no nodes ( #2243 )
...
* Fix discovery loop in case of no peers in routing table
* local testnet: Stop searching when amount of peers in testnet is reached
2021-01-18 14:13:26 +01:00
0fce8ad0d7
Revert "only checkpoint every four slots ( #2236 )" ( #2242 )
...
This reverts commit 7da16f4908
2021-01-18 11:02:56 +01:00
c8c819359c
More clear error message when a validator exit was rejected
2021-01-15 19:40:05 +02:00
7da16f4908
only checkpoint every four slots ( #2236 )
...
* only checkpoint every four slots
* only checkpoint every 16 slots
* every 8 slots
* every 4 slots; 8 seems probably okay, but be a bit conservative
2021-01-15 05:23:54 +00:00
295e3c9c73
Topics validation and direct peers ( #2237 )
...
* pick the right libp2p branch
* add topics validation
2021-01-15 04:17:06 +00:00
fa75c477cd
only initially subscribe to relevant attestation subnets ( #2231 )
2021-01-14 09:43:21 +01:00
66d8f317cd
Use async queryRandom instead of inefficient randomNodes to discover nodes ( #2211 )
2021-01-14 08:58:13 +01:00
0fad1b6b26
don't special-case zero-validator subnet cycling ( #2230 )
2021-01-12 17:17:43 +01:00
dde973e2d4
allow always-on subscription to all attestation subnets when gossiping ( #2225 )
...
* allow always-on subscription to all attestation subnets when gossiping
* in subscribe-all-subnets mode, consider all subnets to be stability subnets for ENR purposes
2021-01-12 13:43:15 +01:00
a3a651b565
always enable topic and aggreate metric topics ( #2229 )
2021-01-12 04:27:09 +01:00
b99a6f3dfd
Warn on Eth1 chain not synced ( #2221 )
2021-01-11 10:23:09 +01:00
87955f2d37
v1.0.6
2021-01-10 12:23:07 +02:00
d803f008c0
v1.0.5 (corrected)
2021-01-10 12:02:55 +02:00
bbdd16f81c
increase attestation pool lookback window ( #2215 )
2021-01-07 14:52:02 +01:00
338428cbd7
Add Eth1 deposits simulation to block_sim
2021-01-04 13:22:00 +02:00
ed24f60f70
remove async from sub/unsub ( #2197 )
...
* remove await/async from sub/unsub
* fix unsubscribe wrong key (missed _snappy)
* use the right libp2p commit hash
* remove unused async
* fix inspector
* fix subnet calculation in RPC and insert broadcast attestations into node's pool
* unify codepaths to ensure only mostly-checked-to-be-valid attestations enter the pool, even from node's own broadcasts
* update attestation pool tests for new validateAttestation param
Co-authored-by: Dustin Brody <tersec@users.noreply.github.com>
2020-12-24 09:48:52 +01:00
97c4f7c5c0
fix subnet calculation in RPC and insert broadcast attestations into node's pool ( #2207 )
...
* fix subnet calculation in RPC and insert broadcast attestations into node's pool
* unify codepaths to ensure only mostly-checked-to-be-valid attestations enter the pool, even from node's own broadcasts
* update attestation pool tests for new validateAttestation param
2020-12-23 13:59:04 +01:00
afbaa36ef7
make subnet cycling more robust; use one stability subnet/validator; explicitly represent gossip enabled/disabled ( #2201 )
...
* make subnet cycling more robust; use one stability subnet/validator; explicitly represent gossip enabled/disabled
* fix asymmetry in _snappy being used for subscriptions but not unsubscriptions
* remove redundant comment
* minimal RPC and VC support for infoming BN of subnets
* create and verify slot signatures in RPC interface and VC
* loosen old slot check
* because Slot + uint64 works but uint64 + Slot doesn't
* document assumptions for head state use; don't clear stability subnets; guard against VC not having checked an epoch ahead, fixing a crash; clarify unsigned comparison
* revert unsub fix
2020-12-22 10:05:36 +01:00
6c8f630170
Revert "have each validator randomly pick a stability subnet, per spec ( #2194 )"
...
This reverts commit 048a67d525
2020-12-18 23:04:31 +01:00
0f8a3a5ae8
checkpoint database at end of each slot ( #2195 )
...
* checkpoint database at end of each slot
To avoid spending time on synchronizing with the file system while doing
processing, the manual checkpointing mode turns off fsync during
processing and instead checkpoints the database when the slot has ended.
From an sqlite perspecitve, in WAL mode this guaranees database
consistency but may lead to data loss which is fine - anything missing
from the beacon chain database can be recovered on the next startup.
* log sync status and delay in slot start message
* bump
2020-12-18 22:01:24 +01:00
452042b17b
msgid formatting, libp2p bump ( #2198 )
...
* msgid formatting, libp2p bump
* fix libp2p commit
2020-12-18 18:45:27 +00:00
048a67d525
have each validator randomly pick a stability subnet, per spec ( #2194 )
2020-12-18 15:46:07 +01:00
8cc7effe52
Fix ENR attnets update to only hold persistent subnets ( #2193 )
...
* Fix ENR attnets update to only hold persistent subnets
* Use only stability subnet in metadata and enr
2020-12-18 09:50:29 +01:00
7d95e86c50
Merge branch 'stable' into devel
2020-12-16 22:22:21 +02:00
f06e6d46a2
v1.0.4
2020-12-16 22:20:45 +02:00
5d8cdb88c6
update validator metrics on startup
2020-12-16 20:44:48 +02:00
de779c7812
update validator metrics on startup
2020-12-16 19:42:19 +02:00
fea2b712f0
cache not needed to validate exits ( #2188 )
2020-12-16 15:36:02 +01:00
7d5edb4353
use new stew helpers for assignment ( #2172 )
...
* bump libp2p (reduces libp2p gossip memory usage to ~1/3)
* use "generic" assign version
2020-12-16 09:37:22 +01:00
c56677867b
Remove some accidentally left debugging changes
2020-12-16 01:09:19 +02:00
8ebf9c30b0
More complete reset of the web3 provider on each failure; Fix #2184
2020-12-16 00:21:11 +02:00
d59fc8ff5b
readd os import, sometimes used
2020-12-15 16:34:56 +01:00
91786686d5
don't repeat already-included attestations ( #2061 )
...
Don't repeat already-included attestations
Also removes the superfluous (and badly scaling) attestation-cache-eviction
2020-12-15 17:16:10 +02:00
4e191a06ac
use stew/leb128
...
* 3 varint parsers -> 1
* 1/3 memory usage / allocations when writing stuff to libp2p
2020-12-15 17:06:15 +02:00
9211b1ca03
import fixes
2020-12-15 16:19:46 +02:00
8d1443f03c
detect already-aggregate-voted condition before attestation pool; add is_aggregator tests ( #2170 )
...
* detect already-aggregate-voted condition before attestation pool; add is_aggregator tests
* replace pair of attestation-per-epoch tracking lists with single list and remove Option use
* fix attestation condition
* use safer type conversions; add more is_aggregator tests
2020-12-14 20:58:32 +00:00
9daf6be73c
graceful exit on SIGTERM ( #2178 )
...
Much easier than convincing all users to change the default signal in
their service definition file to SIGINT.
2020-12-14 16:45:14 +00:00
c91091197f
Lower gossipFactor since we default to 160 peers ( #2169 )
...
And it might potentially send a bit more IHAVE messages then wanted.
2020-12-11 12:15:34 +01:00
bc977799f6
Log warning when running without metrics support
2020-12-10 17:22:29 +02:00
91741326cc
v1.0.3
2020-12-10 02:56:02 +02:00
57fdd87a24
Log Eth1 request failures more precisely
2020-12-10 02:54:58 +02:00
7979fb79d3
Add some useful Eth1 monitor metrics
2020-12-10 02:54:58 +02:00
983b3c9fbf
Add a 'we3 test' command for verifying the compatibility of a web3 provider
2020-12-10 02:54:58 +02:00
0ec90b26a5
Update ENR record with metadata attnets at each attestation subnet cycle ( #2148 )
2020-12-09 10:13:51 +01:00
8b8b25ddac
always check whether gossip should be enabled in onslotstart ( #2162 )
2020-12-08 18:11:54 +01:00
f10f29d829
avoid some futures and copies in processing pipeline ( #2157 )
...
`addLastNoWait` does the same thing with less task scheduling and
copying
2020-12-08 09:59:40 +01:00
3b06e0f657
directly construct cstring in err() expressions
2020-12-07 22:24:06 +02:00
72d00e33b8
always wait for 2/3 into slot for aggregate attestations ( #2158 )
2020-12-07 20:32:34 +01:00
81c25ca4cd
Disconnect peers on status handshake timeout. ( #2149 )
2020-12-07 19:47:07 +01:00
46c2740097
Documentation for Validators API. ( #2147 )
...
* Recover proper validator API call and remove incorrect one.
Add more examples to API documentation.
2020-12-07 14:51:14 +02:00
32a18769e6
remove waitFor in attestation subnet cycling
2020-12-07 14:48:04 +02:00
26ea76bbdf
don't lag aggregated attestations by a slot ( #2150 )
...
* don't lag aggregated attestations by a slot
* don't use aggregation topic at all
* use aggregates again, but with aggressively low ATTESTATION_PROPAGATION_SLOT_RANGE; seems to hold on to LH 1.0 nodes
* clean up scaffolding and double ATTESTATION_PROPAGATION_SLOT_RANGE to 16
* increase ATTESTATION_PROPAGATION_SLOT_RANGE to 24
* increase ATTESTATION_PROPAGATION_SLOT_RANGE to 28 and isolate in only used function due to customization; remove TRAILING_DISTANCE machinery
2020-12-07 12:56:49 +01:00
7363bf3c7c
Fix a logical typo in beacon_api/validator.getStatus
2020-12-03 21:24:01 +02:00
d302c1d8ee
v1.0.2
2020-12-03 19:17:32 +02:00
f4a3a36731
More specific error message when block validation fails
2020-12-03 11:51:42 +02:00
be107df7f1
status bar: actually display it every second, after updating its data
2020-12-03 11:41:40 +02:00
42dd59852b
Fix compilation problem.
2020-12-03 11:40:32 +02:00
0904ee5609
get_v1_validator_attestation_data() implementation.
2020-12-03 11:40:32 +02:00
f6285d8381
Fix proposeSignedBlock() become sync.
2020-12-03 11:40:08 +02:00
088b001388
Fix get_v1_beacon_states_stateId_committees_epoch() to use optional arguments properly.
...
Optimize get_v1_beacon_pool_attestations() a bit.
2020-12-03 11:40:08 +02:00
21ef0f3a57
Fixes for get_v1_beacon_states_stateId_validators() and get_v1_beacon_states_stateId_validators_validatorId().
2020-12-03 11:40:08 +02:00
9afafb54f5
get_v1_beacon_states_stateId_validator_balances() implementation.
2020-12-03 11:40:08 +02:00
5dade7b441
post_v1_beacon_blocks() implementation.
2020-12-03 11:40:08 +02:00
5f392893c0
Change approach for get_v1_beacon_pool_attestations() to use hexadecimal strings.
2020-12-03 11:40:08 +02:00
9e7f8b37c7
get_v1_beacon_pool_attestations() implementation.
2020-12-03 11:40:08 +02:00
7e4ff7a740
Implementation of post_v1_beacon_pool_attester_slashings() and post_v1_beacon_pool_proposer_slashings().
...
Remove doAssert.
2020-12-03 11:40:08 +02:00
6cae25701b
Implementation of beacon_pool_attester_slashings(), beacon_pool_proposer_slashings() and beacon_pool_voluntary_exits().
2020-12-03 11:40:08 +02:00
027d2547cd
Fix a block proposal issue caused by incorrect merkle proofs
...
The key change here is that `addChunksAndGenMerkleProofs` is called
with all pending deposits instead of just the deposits included in
the block. The later was effectively producing merkle proofs against
a different root.
2020-12-03 11:25:52 +02:00
43662be3f8
v1.0.1
2020-12-02 14:31:13 +02:00
d5e3712609
log errors in more detail when block production fails ( #2135 )
...
* log errors in more detail when block production fails
* pass through block header error
* make everything cstring
2020-12-02 13:23:10 +01:00
94c66f15a9
Wanted peers log message fix ( #2114 )
...
* Edit Peer count low wanted peers log
* remove prometheus folder
* remove prometheus.yml
* Add current peers to low peer count log
* change to 25% peer count notice, 12.5% warning
Co-authored-by: kdeme <kim.demey@gmail.com>
2020-12-02 10:47:23 +01:00
92655f936a
Add backoff for the deposit logs requests as well; More debug logs
2020-12-02 01:43:54 +02:00
7225a6d6ed
Retry all web3 requests up to 3 times with exponential backoff
2020-12-02 01:43:54 +02:00
f3b8c777a7
Strip whitespace.
2020-12-02 01:43:36 +02:00
4feb0a308e
Fix #2125 (ETH status bar display); Bump LibP2P
2020-12-02 00:03:59 +02:00
3e4b49462b
detect excessively long graffiti strings
2020-12-01 15:39:46 +02:00
68c91d1d1b
don't wait until after the first slot to enable gossip
2020-12-01 15:39:03 +02:00
2fd71a9aac
Reduce the rate of requesting deposits to lower the risk of triggering the Infura rate limits
2020-12-01 15:38:50 +02:00
4eaf29875a
Less risk for Eth1-induced delays in block proposal
2020-12-01 15:38:50 +02:00
38f7558e50
Work around a strange codegen issue to fix local sim in CI; Bump LibP2P
2020-12-01 15:38:00 +02:00
3bdda3dd46
Hotfix: use the mainnet bootstrap nodes without specifying --network=mainnet explicitly
2020-12-01 10:44:30 +02:00
4328576e18
Hotfix: 'deposits import' was ignoring its arguments in Linux builds
2020-12-01 00:59:57 +02:00
8327240458
v1.0.0-stateofus
2020-11-29 23:50:13 +02:00
ac9bdde543
Don't rely on a metric value for the ETH display in the status bar
2020-11-29 23:35:39 +02:00
cabb07a186
Apply suggestions from code review
...
Co-authored-by: Sacha Saint-Leger <sacha@status.im>
2020-11-29 23:08:07 +02:00
3c0dfc2fbe
Implement the 'deposits exit' command; Remove 'deposits create'
2020-11-29 23:08:07 +02:00
a2364ce1bc
add getGossipSubPeers RPC ( #2107 )
2020-11-29 20:07:20 +01:00
ae19ab72c0
Implement #2067
2020-11-29 18:27:26 +02:00
c3bfe15a35
increase default max peers ( #2106 )
...
* increase default max peers
also avoid reconnection when opening stream as this might induce
a loop
* Use dial without addresses
* dial back max peers a little
2020-11-29 14:43:41 +01:00
7731109320
Remove getSpecPreset() because of crash and add missing calls into api.md
2020-11-29 11:01:55 +02:00
bf2673abc4
Restore the ETH display in the status bar
2020-11-28 20:53:51 +02:00
8685eb7042
add validator balance metrics
2020-11-28 16:22:09 +02:00
db030eb944
Setup agent/proto version identifiers in libp2p. ( #2079 )
...
* Supply `nim-libp2p` with `agent-string` and `proto-string` as configured parameters.
* Remove protoString and rng check.
* Fix compilation problem.
2020-11-28 08:00:36 +01:00
e7f2735271
fix broken metrics during replay ( #2090 )
...
* move metrics out of state transition
* add validator count metric
* remove expensive beacon_current_validators, beacon_previous_validators
metrics (they should be reimplemented with cache), add cheap
beacon_active_validators to approximate
* remove unused validator count metrics
* tidy imports/defects
2020-11-27 23:16:13 +01:00
2421d338c1
node.network.metadata.attnets is only secondary source of truth ( #2089 )
2020-11-27 15:54:13 +01:00
d16e127daf
Peer stuff ( #2084 )
...
* Revert "Revert "Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065 )" (#2082 )"
This reverts commit 7cc3dc8027
2020-11-26 20:23:45 +01:00
9b9aa93cd4
Seenttl by spec ( #2083 )
...
* add rpc call to get chronos futures at runtime
* add seenTTL gossip parameter
Co-authored-by: Jacek Sieka <jacek@status.im>
2020-11-26 13:08:22 +01:00
687cbaf94c
Adjust number of sync workers from 20 to 10. ( #2077 )
...
Adjust watch task min-pause-time to 1.minute.
Slowed down pause-time recovery by factor 3/4 instead of 1/2.
2020-11-26 09:19:27 +01:00
7cc3dc8027
Revert "Full "node" RPC calls implementation and fixes to peer lifetime states. ( #2065 )" ( #2082 )
...
This reverts commit d041287a4f
2020-11-26 09:05:23 +01:00
d041287a4f
Full "node" RPC calls implementation and fixes to peer lifetime states. ( #2065 )
...
* Initial commit.
* Fix log lines and compilation error.
* Add get_v1_node_peers() implementation.
* Fix peer's lifetime states.
* Use the most recent multiaddress.
* Fix assign NewPeerScore again.
Fix compilation error with last seen address.
Fix Peer upgraded log line place.
* syncing, health, peer_count, peer_id and fixes for identity.
* Fix compilation problems.
* Move object declaration to callsigs.
Fix identity addresses fields.
* Finish node RPC calls.
* Avoid leak of lifetime future.
* Bump chronos.
* Fix json generator problem.
2020-11-26 08:09:59 +01:00
e69b5ff473
Add a record create and print command
2020-11-25 18:32:59 +02:00
3a13750734
Try to obtain deposit contract address from config.
...
Attempt to fix json generation.
2020-11-25 18:30:44 +02:00
68f4ad64a4
Disable exception if depositAddress is not available.
2020-11-25 18:30:44 +02:00
af3862da48
Make depositAddress public in eth1_monitor.
...
Fix representation of config.
Add deposit_contract implementation.
2020-11-25 18:30:44 +02:00
8eafbb4e09
Add config_spec implementation.
2020-11-25 18:30:44 +02:00
3594fa2a22
Version 1.0.0-rc1
2020-11-25 03:13:58 +02:00
b9e4fef616
Check that the selected data dir is compatible with the selected network
2020-11-25 03:13:58 +02:00
e1e4871287
fix time formatting and pre-genesis ops
2020-11-25 01:51:46 +02:00
151d116f02
Fix off-by-one error in Eth1 pruning
2020-11-25 01:51:17 +02:00
a995cdd3d9
Bugfix: fix an incorrect balance increase during genesis state generation
2020-11-24 23:30:26 +02:00
594ef713a0
Increase the web3 timeouts
2020-11-24 23:28:20 +02:00
372c9b798c
Fix the corrupted database state on Pyrmont nodes; Add mainnet genesis ( #2056 )
...
* Handle some web3 timeouts better
* Add support for developer .env files
* Eth1 improvements; Mainnet genesis state
Notable changes:
* The deposits table have been removed from the database. The client
will no longer process all deposits on start-up.
* The network metadata now includes a "state snapshot" of the deposit
contract. This allows the client to skip syncing deposits made prior
to the snapshot (i.e. genesis). Suitable metadata added for Pyrmont
and Mainnet.
* The Eth1 monitor won't be started unless there are validators attached
to the node.
* The genesis detection code is now optional and disabled by default
* Bugfix: The client should not produce blocks that will fail validation
when it hasn't downloaded the latest deposits yet
* Bugfix: Work around the database corruption affecting Pyrmont nodes
* Remove metadata for Toledo and Medalla
2020-11-24 22:21:47 +01:00
040093a945
medalla -> pyrmont ( #2062 )
2020-11-23 17:46:51 +01:00
6f476ca380
Gossip11 params ( #2047 )
...
* add rpc call to get chronos futures at runtime
* add eth2 spec gossip parameters
* bump libp2p
* fix url, remove 1.0
Co-authored-by: Jacek Sieka <jacek@status.im>
2020-11-23 10:57:03 +01:00
672915e170
work around long pyrmont startup time ( #2060 )
...
* also fixes unnecessary copy/memory alloc when loading DbSeq entries
2020-11-21 18:53:40 +01:00
00aedddf2a
Version 0.6.6
2020-11-20 22:31:03 +02:00
dbcc0686ff
delay pruning of cache for finalized epoch ( fixes #2049 )
2020-11-20 20:57:50 +02:00
316a19af5f
Address review comments
2020-11-20 17:13:51 +02:00
d232f16b40
Add more eth1-related logging
2020-11-20 17:13:51 +02:00
e22248bca1
Tentative fix for the Pyrmont Eth1 voting issue
2020-11-20 17:13:51 +02:00
3c4cf95d0e
remove v0.12.3 and medalla support ( #2038 )
2020-11-20 14:49:49 +01:00
54c388b7b4
close slashing protection database ( #2050 )
2020-11-20 14:23:55 +01:00
1d7fb2ed0c
remove {.inline.} pragmas ( #2033 )
...
* remove {.inline.} pragmas
* re-add inline on bitseqs functions and tweak inlining threshold
* remove macOS/LLVM inlining setting; revert non-init/module-local/tests inline pragma removal
2020-11-20 11:00:22 +01:00
883e18da81
Hotfix: don't crash on web3 provider errors during initialization
2020-11-20 10:26:47 +02:00
771c1d092a
hotfix fork choice score application ( #2041 )
...
Depending on the order of nodes in fork choice, it seems that the break
here could cause some scores not to be applied correctly
2020-11-19 15:11:08 +01:00
e370e80aee
Add more Pyrmont bootstrap nodes
2020-11-18 14:40:45 +02:00
6b6e9b4d77
v0.6.4 Script and instructions for joining Pyrmont; WIP page in the book
2020-11-18 00:27:22 +02:00
11e1a9e8e8
A hacky work-around for a web3 issue that may cause the client to go into a loop of failing requests
2020-11-18 00:27:21 +02:00
ebfacf597c
Review and fix some usages of Future.cancel; Bump Chronos
2020-11-18 00:27:21 +02:00
9c5cef346b
set file and dir permissions
2020-11-17 23:28:26 +02:00
4479c0a9f1
fix some libp2p channel hangs ( #2032 )
...
* fix some libp2p channel hangs
* make sure no concurrent reads are issues by awaiting cancellations
* bumps with fixes
* merge bump
2020-11-17 16:07:05 +01:00
9e716b32bd
address some XDeclaredButNotUsed hints ( #2028 )
2020-11-17 11:14:53 +01:00
a6b188bfd4
misc fixes ( #2027 )
...
* log when database is loading (to avoid confusion)
* generate network keys later during startup
* fix quarantine not scheduling chain of parents for download and
increase size to one epoch
* log validator count, enr and peerid more clearly on startup
2020-11-16 20:15:43 +01:00
21c4ce8fd4
remove superfluous TODOs/not-really-TODOs, type conversion, imports ( #2025 )
2020-11-16 17:10:51 +01:00
19707e89c2
Change random network key log line and include PeerID.
2020-11-16 17:41:23 +02:00
5b011b65ae
add rpc call to get chronos futures at runtime
2020-11-16 12:31:39 +02:00
e2f161dbf7
fix attestation sending schedules to avoid timing attack ( #1995 )
2020-11-16 10:44:18 +01:00
b022dc4d1f
Use O(n) algorithm in initialize_beacon_state_from_eth1; Avoid unnecessary merkle proofs generation
2020-11-15 21:40:40 +02:00
8012102704
Bugfix: the client can miss the genesis event in the absence of new deposits
2020-11-15 21:40:40 +02:00
9f28a464fb
[skip ci] Update the version number
2020-11-14 23:26:46 +02:00
d02d8f8d51
Drop UseDiscv51 flag and only support discovery v5.1
2020-11-14 19:12:30 +02:00
80ca7a2d9f
Implement #1768
2020-11-12 22:29:43 +02:00
17d35e1fd9
Allow the node to start when it fails to initialize the Eth1 monitor
...
* Avoid hangs when wss:// is specified for a non-secure HTTP server
* Produce an ERROR when the web3 provider is unsupported, but still launch the node
2020-11-12 22:29:43 +02:00
5e45e7429e
Rewrite all HTTP(S) web3 URLs to WebSocket URls
2020-11-12 22:29:43 +02:00
87c92ea05f
Address review comments.
...
Remove `-8.0` bucket because `-4.0` is a limit.
2020-11-12 21:52:01 +02:00
dbb058682c
Add beacon_attestation_sent_delay metric.
2020-11-12 21:52:01 +02:00
8ad45de65b
Add makefile targets for Pyrmont
2020-11-12 21:51:15 +02:00
b9e122a60c
address some TODOs ( #2005 )
2020-11-12 19:24:07 +00:00
e035b7d0c9
Fix status rpc request to use RESP_TIMEOUT. ( #2002 )
2020-11-12 15:30:12 +00:00
754d65c159
non-spec aggregate validation check now in spec; update genesis times ( #2001 )
2020-11-12 15:29:32 +00:00
a0621d52f6
bump remaining rc.0 spec ref ( #2003 )
2020-11-12 13:58:41 +00:00
f642f71457
Calculate how much time syncing takes, and show it at statusbar.
2020-11-12 14:16:12 +02:00
332e248d91
rpc: setLogLevel ( #1991 )
...
* rpc: setLogLevel
also change default rpc port to match makefile and manual
* set topic-level formats too
and make feature less secret
2020-11-12 11:46:02 +01:00
43c6c5a0b3
Gossip message id should decompress data ( #1998 )
...
* also fix missing return on empty message
2020-11-12 11:45:28 +01:00
5644cfae8f
increase max db object size ( fixes #1996 )
2020-11-12 08:42:45 +01:00
c5710d97fc
default to gossipsub 1.1 ( #1992 )
2020-11-11 13:30:18 +01:00
389c11743a
Review TODO items and self-assign the most important ones
2020-11-10 20:41:04 +02:00
6e40ca5e15
Use separate cache for getting epochref
...
...else tmpState gets overwritten when producing blocks and requesting
epochRef for finalized state.
2020-11-10 17:40:32 +02:00
ea4d94c65c
Add guard task to monitor synchronization worker Future. ( #1972 )
...
* Add guard task to monitor synchronization worker Future.
* Change log events from debug to warning level.
2020-11-10 14:47:26 +01:00
7eaae89263
keep AllTests-mainnet.md consistent; ensure don't send overly large messages aren't sent ( #1973 )
2020-11-09 18:11:09 +00:00
a010ef2061
update tests from v1.0.0-rc.0 to v1.0.0 ( #1976 )
2020-11-09 17:44:18 +00:00
bd1047b715
0.6.0 release fixes
...
* Updated README
* No double v in nimbus_beacon_node --version
* No md5 checksums
2020-11-09 17:09:49 +02:00
271df8b604
bump 1.0.0rc-0 spec refs to 1.0.0 ( #1974 )
2020-11-09 14:18:55 +00:00
61a3209c24
remove version quotes
2020-11-09 13:37:58 +01:00
94e9ce3c1f
Add metadata for the toledo network
2020-11-09 11:38:52 +02:00
4b603d7f5a
Change the default log level to INFO
2020-11-09 11:38:52 +02:00
e9b9cd75ee
Rename binaries; Mimic the original repo layout in the distribution
2020-11-09 11:38:52 +02:00
bf58915500
release fixes ( #1966 )
2020-11-07 08:46:53 +01:00
d2476b643a
bump version ( #1964 )
...
* bump version
* update version format to "standard" style
* remove zinken/attacknet (both networks deprecated)
* add mainnet warning
2020-11-07 08:45:55 +01:00
fb6ce5c804
Enable discv5.1 for general beacon_node and inspector builds
2020-11-07 01:27:02 +02:00
ac1072b4bb
Add ./run-medalla-beacon-node script
2020-11-07 01:25:02 +02:00
4a710252d8
Check the web3 provider for network compatibility
2020-11-07 01:25:02 +02:00
24f8b7d483
[skip ci] Clean up some accidentally commited code
2020-11-06 11:53:44 +02:00
a8a66fe3f6
Fix #1784 ; Handle Infura HTTPS URLs
2020-11-06 11:15:32 +02:00
f596a24c4c
Mainnet deposit contract monitoring
2020-11-06 11:15:32 +02:00
95f5f76180
Datatype cleanup ( #1953 )
...
* clear up spec todo
* test fix
* remove unnecessary toSszType
* type
* one more
2020-11-04 21:52:47 +00:00
fc7885b27e
Store block summary in database
...
This introcudes a cache for block summaries, useful for instantiating
the block dag on startup, bringing medalla startup times down from
minutes to seconds.
This is something of a temporary band-aid that would be obsoleted by a
finalized block store.
2020-11-04 11:28:55 +02:00
7823c25ba8
Hotfix for a failing assert during block production
2020-11-04 11:06:58 +02:00
36b54fe091
Update eth2_network.nim
...
More informative and friendlier log message (from the user's perspective). Also helps avoid the potential for confusion with the validator key.
2020-11-03 23:23:32 +02:00
6ee80651c7
Rename mainchain_monitor to eth1_monitor
2020-11-03 23:23:10 +02:00
2f29e3e7f3
Further reduce the number of RPC requests; Significantly faster ETH1 syncing
2020-11-03 23:23:10 +02:00
14b2d4324d
openarray -> openArray
2020-11-03 23:23:10 +02:00
18639c3eff
Don't require requests that might fail on non-archive Geth nodes
2020-11-03 23:23:10 +02:00
79f1b6117e
Remove dead code from the beacon node binary; Fix 'make test'
2020-11-03 23:23:10 +02:00
07206ca264
An efficient algorithm for incremental generation of multiple merkle proofs
2020-11-03 23:23:10 +02:00
d913273dcc
Fix #1823
2020-11-03 23:23:10 +02:00
937decb0e9
Revert "Gossip11 post audit ( #1917 )" ( #1947 )
...
This reverts commit 63173ab2c1
2020-11-03 11:45:40 +01:00
9255945fb0
Update mainchain_monitor.nim ( #1945 )
...
Make it clear that this failure concerns the eth1 chain
2020-11-02 21:16:24 +01:00
002a3dfa54
graceful shutdown: move the hook registration ( #1940 )
...
...closer to where it starts having an effect.
2020-11-02 19:02:27 +01:00
2f049dc6e7
Fix init condition for fork choice ( #1941 )
...
... or we choose the wrong head on startup, sometimes.
2020-11-02 18:51:08 +01:00
ca63d48b82
tone down validator exit logging ( #1939 )
...
Validators exiting is normal, no need to scream about it
* avoid reallocating seq on big exit queue
* avoid fetching state cache when updating head (it's rarely needed)
* remove incorrectly implemented live validator counts (avoids memory
allocs)
2020-11-02 18:34:23 +01:00
7294d98ed1
Update validator_duties.nim (Improve node out of sync message) ( #1933 )
...
* Update validator_duties.nim
Improve node out of sync message
* Update validator_duties.nim
2020-11-02 08:52:16 +01:00
67113373f9
Work around BLST build issues
2020-10-30 23:56:04 +02:00
f76679810b
Fix #1855 ; Add support for wallet recovery passwords
2020-10-30 23:56:04 +02:00
63173ab2c1
Gossip11 post audit ( #1917 )
...
* bump libp2p gossip
* use gossip 1.1
* use proper libp2p commit hash
* switch to master libp2p, has audit fixes now
2020-10-30 14:26:35 +00:00
3f5c7c36bc
Some syncing fixes ( #1919 )
...
* Add exponential rewind on MissingParent.
* Try to avoid peers which are useless for syncing.
Fix forward sync restart at proper point.
Fix getLocalWallSlot() to not return slots from the future.
* Fix incorrect logs.
* Fix logging.
Enable peer's status messages log on DEBUG level.
* Fix watch task to monitor operation progress, but not local head progress.
* Add more logging information.
Remove recurring failures detection mechanism.
2020-10-30 13:33:52 +01:00
81f4fe0783
limit fork choice initialization horizon ( #1910 )
...
This helps manage the long startup time on chains that are not
finalizing for long periods of time
2020-10-29 12:09:03 +01:00
020a32ffa4
minor api cleanups ( #1913 )
...
* readd getNodeVersion
* implement get_v1_config_fork_schedule
* fix formatting of several types
2020-10-28 19:51:38 +01:00
19c51e5900
use MAXIMUM_GOSSIP_CLOCK_DISPARITY for BeaconBlock verification ( #1918 )
2020-10-28 14:04:21 +01:00
ff3f8741de
handle fork choice selecting a head unknown to chaindag ( #1907 )
...
* move head slot metric to head selection
* avoid redundant head selection in validator duties
2020-10-28 07:55:36 +00:00
6050eebec0
document not-nil issues
2020-10-27 13:01:45 +01:00
eee01a32d5
Regression fix of eth2_network_simulation on Windows. ( #1900 )
...
* Concentrate all sensitive writeFile/createPath calls in one place.
Fix eth2_network_simulation for Windows.
* Remove artifacts.
* fix import
Co-authored-by: Jacek Sieka <jacek@status.im>
2020-10-27 12:04:17 +01:00
c82ff24b5c
Syncing fixes ( #1909 )
...
* Fix continuous sync queue rewinds on slow PCs.
Fix recurring disconnects on low peer score.
* Calculate average syncing speed, not the current one.
Move speed calculation to different task.
* Address review comments.
2020-10-27 10:25:28 +01:00
ee2ebc96a8
move rpc api to own folder, mimic upstream structure ( #1905 )
...
also implements a few more endpoints
2020-10-27 10:00:57 +01:00
677f099110
BLS signature v4 ( #1912 )
2020-10-27 09:37:04 +01:00
8102d5dc4d
Bump eth2-testnets for more discv5.1 bootstrap nodes on medalla ( #1911 )
...
And improve logging UX a bit by ignoring comments or newlines
2020-10-27 08:44:46 +01:00
756b62fdf5
Bump nim-eth to get UseDiscv51 flag ( #1895 )
...
* Bump nim-eth to get UseDiscv51 flag
* Switch medalla to discovery v5.1, other targets to v5.0
* Bump nim-eth for better discv5.1 logging
* Bump eth2-testnets for updated medalla bootnodes
2020-10-23 18:59:58 +02:00
7c0b4d28d2
speed up reward/penalty calculation
...
Calculating rewards/penalties is slow due to how we compute sets of
attestations validators then use the sets for inclusion checks, to see
who attested. The dominant function during validated block processing /
epoch processing is hash set building and lookup.
This PR inverts the flow by removing the sets and creating a single
large validator status list, then applying all relevant state
attestations, then updating rewards and penalties.
This provides a 10x speedup to epoch processing which in turn speeds up
both empty slot and block processing - for example, on startup, we
replay all non-finalized blocks to prime fork choice - the same when
validating attestations or replaying states on reorg.
2020-10-23 19:23:36 +03:00
499e5ca991
misc memory and perf fixes ( #1899 )
...
* misc memory and perf fixes
* use EpochRef for attestation aggregation
* compress effective balances in memory (medalla unfinalized: 4gb ->
1gb)
* avoid hitting db when rewinding to head or clearance state
* avoid hitting db when blocks can be applied to in-memory state -
speeds up startup considerably
* avoid storing epochref in fork choice
* simplify and speed up beacon block creation flow - avoids state reload
thanks to head rewind optimization
* iterator-based committee and attestation participation help avoid lots
of small memory allocations throughout epoch transition (40% speedup on
epoch processing, for example during startup)
* add constant for threshold
2020-10-22 12:53:33 +02:00
decfd66a78
bump libp2p: seq[byte] message IDs and improve reject/ignore handling ( #1898 )
...
* bump libp2p: use seq[byte] message IDs and improve reject/ignore handling
* bump libp2p to pick up debug message details
2020-10-21 20:03:12 +00:00
9810a31447
Fix #1838 ( #1892 )
2020-10-20 15:01:21 +02:00
a136c2e95a
bump libp2p; integrate pubsub.ValidationResult into extended validation ( #1893 )
2020-10-20 12:31:20 +00:00
e87e0ef179
update to 1.0.0-rc.0 message ID; unexport funcs; enable test in mainnet; remove unused subscribe-with-nondummy-handler function; fix gossip received metric ( #1884 )
2020-10-20 08:54:11 +00:00
c446eb7cff
update v1.0.0-rc.0 preset spec references ( #1891 )
...
* update ve1.0.0-rc.0 preset spec references
* remove runtime preset ETH1_FOLLOW_DISTANCE from preset files; remove two CI build items to try to keep Travis from timing out
2020-10-19 13:53:23 +00:00
5b13a5ba10
update attestation extended validation to v1.0.0-rc.0 ( #1889 )
...
* update attestation extended validation to v1.0.0-rc.0
* attestation block and target must be within same epoch
* remove duplicate attestation epoch/target epoch check
* in spec now
2020-10-19 09:25:06 +00:00
df43b8aa8b
save some more states after all ( #1887 )
...
Don't save states when replaying history, but do save states when
applying new blocks (!)
2020-10-18 15:47:39 +00:00
dbc90e998a
Add Disc v5.1 bootstrap nodes for Zinken
...
Also prepares the Makefiles for supporting Medalla
through a separate binary
2020-10-18 12:44:52 +03:00
d98be078c3
Address #1690 Min password entropy
2020-10-16 19:34:26 +03:00
9899578e1c
Update get_eth1_data from v0.11.1 to v1.0.0-rc
2020-10-16 11:46:53 +03:00
9a543e0af7
partial hotfix for #1879 crash
2020-10-16 11:46:19 +03:00
948d15ea95
Follow-up on #1687 Don't ask for password on invalid keystores
2020-10-16 11:45:39 +03:00
db70f066ba
fixed all crashes which can be triggered through the validator RPC API - fixes #1680
2020-10-15 20:19:43 +03:00
7ed885f144
configuring a writer for the dynamic log output device for the VC binary as well (same as in the BN)
2020-10-15 20:19:10 +03:00
7ad59b1852
Bump nim-stew.
2020-10-15 20:18:33 +03:00
1742a82ca7
Proper fix for writeFile() places.
...
Comment unused/insecure procedure.
2020-10-15 20:18:33 +03:00
84fe294c00
Fix protection of data folder and security files using Windows ACL.
2020-10-15 20:18:33 +03:00
5f6bdc6709
Store all deposit-derived data in memory
2020-10-15 20:15:51 +03:00
7a577b2cef
More tests for getBlockRange
2020-10-15 20:15:51 +03:00
080609eee1
Address #1366 Avoid uint64 overflow in getBlockRange when skipStep is large
2020-10-15 20:15:51 +03:00
2152dc6136
Simplify the mainchain monitor
2020-10-15 20:15:51 +03:00
ce1fda1195
Store the deposits and the immutable validator data in Sqlite
2020-10-15 20:15:51 +03:00
e69af00e3a
Don't penalize flooding peers; Just rate limit them
2020-10-15 20:15:51 +03:00
4d66914f5a
Fix the test suite
2020-10-15 20:15:51 +03:00
f0892ce934
Fix compilation problems on Windows
2020-10-15 20:15:51 +03:00
e6320e5881
Address #1584 Don't keep all deposits in memory (persist them to disk)
2020-10-15 20:15:51 +03:00
bc8acdb9de
Switch to an in-house clearScreen to avoid problems with exception tracking
2020-10-15 20:15:51 +03:00
69e498dc00
Address #1689 and #1517 Usability and security improvements in wallet creation
2020-10-15 20:15:51 +03:00
02a3770803
Adds a fuzzing test for the validator client CLI
...
Also removes an obsolete way to add validators through the command line
2020-10-15 20:15:51 +03:00
8ce0fc3a89
Address #1687
...
This addresses the issues by detecting and rejecting keystores with
incorrect PBKDF2 and SCrypt params. It also bumps the version of
nim-json-serialization to include a bugfix for incorrect parsing
of json files featuring comments.
2020-10-15 20:15:51 +03:00
cd949a2b81
Address #1691 More pervasive handling of Ctrl+C
2020-10-15 20:15:51 +03:00
8a6281aad2
Simple cost model for sync requests; Penalize peers perfoming flooding or invalid requests
2020-10-15 20:15:51 +03:00
6b9419e547
fix db growth on attestation processing ( #1860 )
...
It turns out that we often save lots of states in the database that are
the result of empty slot processing only - here, we make sure to only
save a state if a block follows - this fixes several issues:
* empty slot states are not always pruned leading to state database size
explosion
* storing states is (very) slow which slows down processing in general,
so we should only do it when it's likely to be useful
* attestation processing doesn't get stuck on saving random states that
won't appear in the chain history
2020-10-15 14:28:44 +02:00
4e73d10f86
removed 2 buggy RPC API endpoints - there are 2 well functioning ones in validator_api.nim anyway ( #1867 )
2020-10-15 14:26:59 +02:00
da59f45a90
update spec/datatypes spec refs from v0.12.3 to v1.0.0 ( #1866 )
2020-10-14 10:00:27 +00:00
513ba72b9a
add v1.0.0-rc.0 support behind compile-time flag ( #1852 )
...
* add v1.0.0-rc.0 support behind compile-time flag
* keep runtime presets consistent
2020-10-13 17:21:25 +00:00
3ee2dd8da4
p2p-interface spec ref bump (except non-updated places) ( #1862 )
2020-10-12 14:37:14 +00:00
1994ffe5a0
update 130+ spec references from v0.12.3 to v1.0.0-rc1 ( #1854 )
2020-10-12 08:59:24 +00:00
00a8a68671
Address #1695
...
Better error messages when the beacon node is asked to listen on a
reserved port (0) or an already taken one.
2020-10-09 16:39:03 +03:00
Jacek Sieka
Kim De Mey
tersec
Zahary Karadjov
Giovanni Petrantoni
Ștefan Talpalaru
Eugene Kabanov
Sacha Saint-Leger
Mamy Ratsimbazafy
Viktor Kirilov