c3bfe15a35
increase default max peers ( #2106 )
...
* increase default max peers
also avoid reconnection when opening stream as this might induce
a loop
* Use dial without addresses
* dial back max peers a little
2020-11-29 14:43:41 +01:00
7731109320
Remove getSpecPreset() because of crash and add missing calls into api.md
2020-11-29 11:01:55 +02:00
bf2673abc4
Restore the ETH display in the status bar
2020-11-28 20:53:51 +02:00
8685eb7042
add validator balance metrics
2020-11-28 16:22:09 +02:00
db030eb944
Setup agent/proto version identifiers in libp2p. ( #2079 )
...
* Supply `nim-libp2p` with `agent-string` and `proto-string` as configured parameters.
* Remove protoString and rng check.
* Fix compilation problem.
2020-11-28 08:00:36 +01:00
e7f2735271
fix broken metrics during replay ( #2090 )
...
* move metrics out of state transition
* add validator count metric
* remove expensive beacon_current_validators, beacon_previous_validators
metrics (they should be reimplemented with cache), add cheap
beacon_active_validators to approximate
* remove unused validator count metrics
* tidy imports/defects
2020-11-27 23:16:13 +01:00
2421d338c1
node.network.metadata.attnets is only secondary source of truth ( #2089 )
2020-11-27 15:54:13 +01:00
d16e127daf
Peer stuff ( #2084 )
...
* Revert "Revert "Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065 )" (#2082 )"
This reverts commit 7cc3dc8027
2020-11-26 20:23:45 +01:00
9b9aa93cd4
Seenttl by spec ( #2083 )
...
* add rpc call to get chronos futures at runtime
* add seenTTL gossip parameter
Co-authored-by: Jacek Sieka <jacek@status.im>
2020-11-26 13:08:22 +01:00
687cbaf94c
Adjust number of sync workers from 20 to 10. ( #2077 )
...
Adjust watch task min-pause-time to 1.minute.
Slowed down pause-time recovery by factor 3/4 instead of 1/2.
2020-11-26 09:19:27 +01:00
7cc3dc8027
Revert "Full "node" RPC calls implementation and fixes to peer lifetime states. ( #2065 )" ( #2082 )
...
This reverts commit d041287a4f
2020-11-26 09:05:23 +01:00
d041287a4f
Full "node" RPC calls implementation and fixes to peer lifetime states. ( #2065 )
...
* Initial commit.
* Fix log lines and compilation error.
* Add get_v1_node_peers() implementation.
* Fix peer's lifetime states.
* Use the most recent multiaddress.
* Fix assign NewPeerScore again.
Fix compilation error with last seen address.
Fix Peer upgraded log line place.
* syncing, health, peer_count, peer_id and fixes for identity.
* Fix compilation problems.
* Move object declaration to callsigs.
Fix identity addresses fields.
* Finish node RPC calls.
* Avoid leak of lifetime future.
* Bump chronos.
* Fix json generator problem.
2020-11-26 08:09:59 +01:00
e69b5ff473
Add a record create and print command
2020-11-25 18:32:59 +02:00
3a13750734
Try to obtain deposit contract address from config.
...
Attempt to fix json generation.
2020-11-25 18:30:44 +02:00
68f4ad64a4
Disable exception if depositAddress is not available.
2020-11-25 18:30:44 +02:00
af3862da48
Make depositAddress public in eth1_monitor.
...
Fix representation of config.
Add deposit_contract implementation.
2020-11-25 18:30:44 +02:00
8eafbb4e09
Add config_spec implementation.
2020-11-25 18:30:44 +02:00
3594fa2a22
Version 1.0.0-rc1
2020-11-25 03:13:58 +02:00
b9e4fef616
Check that the selected data dir is compatible with the selected network
2020-11-25 03:13:58 +02:00
e1e4871287
fix time formatting and pre-genesis ops
2020-11-25 01:51:46 +02:00
151d116f02
Fix off-by-one error in Eth1 pruning
2020-11-25 01:51:17 +02:00
a995cdd3d9
Bugfix: fix an incorrect balance increase during genesis state generation
2020-11-24 23:30:26 +02:00
594ef713a0
Increase the web3 timeouts
2020-11-24 23:28:20 +02:00
372c9b798c
Fix the corrupted database state on Pyrmont nodes; Add mainnet genesis ( #2056 )
...
* Handle some web3 timeouts better
* Add support for developer .env files
* Eth1 improvements; Mainnet genesis state
Notable changes:
* The deposits table have been removed from the database. The client
will no longer process all deposits on start-up.
* The network metadata now includes a "state snapshot" of the deposit
contract. This allows the client to skip syncing deposits made prior
to the snapshot (i.e. genesis). Suitable metadata added for Pyrmont
and Mainnet.
* The Eth1 monitor won't be started unless there are validators attached
to the node.
* The genesis detection code is now optional and disabled by default
* Bugfix: The client should not produce blocks that will fail validation
when it hasn't downloaded the latest deposits yet
* Bugfix: Work around the database corruption affecting Pyrmont nodes
* Remove metadata for Toledo and Medalla
2020-11-24 22:21:47 +01:00
040093a945
medalla -> pyrmont ( #2062 )
2020-11-23 17:46:51 +01:00
6f476ca380
Gossip11 params ( #2047 )
...
* add rpc call to get chronos futures at runtime
* add eth2 spec gossip parameters
* bump libp2p
* fix url, remove 1.0
Co-authored-by: Jacek Sieka <jacek@status.im>
2020-11-23 10:57:03 +01:00
672915e170
work around long pyrmont startup time ( #2060 )
...
* also fixes unnecessary copy/memory alloc when loading DbSeq entries
2020-11-21 18:53:40 +01:00
00aedddf2a
Version 0.6.6
2020-11-20 22:31:03 +02:00
dbcc0686ff
delay pruning of cache for finalized epoch ( fixes #2049 )
2020-11-20 20:57:50 +02:00
316a19af5f
Address review comments
2020-11-20 17:13:51 +02:00
d232f16b40
Add more eth1-related logging
2020-11-20 17:13:51 +02:00
e22248bca1
Tentative fix for the Pyrmont Eth1 voting issue
2020-11-20 17:13:51 +02:00
3c4cf95d0e
remove v0.12.3 and medalla support ( #2038 )
2020-11-20 14:49:49 +01:00
54c388b7b4
close slashing protection database ( #2050 )
2020-11-20 14:23:55 +01:00
1d7fb2ed0c
remove {.inline.} pragmas ( #2033 )
...
* remove {.inline.} pragmas
* re-add inline on bitseqs functions and tweak inlining threshold
* remove macOS/LLVM inlining setting; revert non-init/module-local/tests inline pragma removal
2020-11-20 11:00:22 +01:00
883e18da81
Hotfix: don't crash on web3 provider errors during initialization
2020-11-20 10:26:47 +02:00
771c1d092a
hotfix fork choice score application ( #2041 )
...
Depending on the order of nodes in fork choice, it seems that the break
here could cause some scores not to be applied correctly
2020-11-19 15:11:08 +01:00
e370e80aee
Add more Pyrmont bootstrap nodes
2020-11-18 14:40:45 +02:00
6b6e9b4d77
v0.6.4 Script and instructions for joining Pyrmont; WIP page in the book
2020-11-18 00:27:22 +02:00
11e1a9e8e8
A hacky work-around for a web3 issue that may cause the client to go into a loop of failing requests
2020-11-18 00:27:21 +02:00
ebfacf597c
Review and fix some usages of Future.cancel; Bump Chronos
2020-11-18 00:27:21 +02:00
9c5cef346b
set file and dir permissions
2020-11-17 23:28:26 +02:00
4479c0a9f1
fix some libp2p channel hangs ( #2032 )
...
* fix some libp2p channel hangs
* make sure no concurrent reads are issues by awaiting cancellations
* bumps with fixes
* merge bump
2020-11-17 16:07:05 +01:00
9e716b32bd
address some XDeclaredButNotUsed hints ( #2028 )
2020-11-17 11:14:53 +01:00
a6b188bfd4
misc fixes ( #2027 )
...
* log when database is loading (to avoid confusion)
* generate network keys later during startup
* fix quarantine not scheduling chain of parents for download and
increase size to one epoch
* log validator count, enr and peerid more clearly on startup
2020-11-16 20:15:43 +01:00
21c4ce8fd4
remove superfluous TODOs/not-really-TODOs, type conversion, imports ( #2025 )
2020-11-16 17:10:51 +01:00
19707e89c2
Change random network key log line and include PeerID.
2020-11-16 17:41:23 +02:00
5b011b65ae
add rpc call to get chronos futures at runtime
2020-11-16 12:31:39 +02:00
e2f161dbf7
fix attestation sending schedules to avoid timing attack ( #1995 )
2020-11-16 10:44:18 +01:00
b022dc4d1f
Use O(n) algorithm in initialize_beacon_state_from_eth1; Avoid unnecessary merkle proofs generation
2020-11-15 21:40:40 +02:00
8012102704
Bugfix: the client can miss the genesis event in the absence of new deposits
2020-11-15 21:40:40 +02:00
9f28a464fb
[skip ci] Update the version number
2020-11-14 23:26:46 +02:00
d02d8f8d51
Drop UseDiscv51 flag and only support discovery v5.1
2020-11-14 19:12:30 +02:00
80ca7a2d9f
Implement #1768
2020-11-12 22:29:43 +02:00
17d35e1fd9
Allow the node to start when it fails to initialize the Eth1 monitor
...
* Avoid hangs when wss:// is specified for a non-secure HTTP server
* Produce an ERROR when the web3 provider is unsupported, but still launch the node
2020-11-12 22:29:43 +02:00
5e45e7429e
Rewrite all HTTP(S) web3 URLs to WebSocket URls
2020-11-12 22:29:43 +02:00
87c92ea05f
Address review comments.
...
Remove `-8.0` bucket because `-4.0` is a limit.
2020-11-12 21:52:01 +02:00
dbb058682c
Add beacon_attestation_sent_delay metric.
2020-11-12 21:52:01 +02:00
8ad45de65b
Add makefile targets for Pyrmont
2020-11-12 21:51:15 +02:00
b9e122a60c
address some TODOs ( #2005 )
2020-11-12 19:24:07 +00:00
e035b7d0c9
Fix status rpc request to use RESP_TIMEOUT. ( #2002 )
2020-11-12 15:30:12 +00:00
754d65c159
non-spec aggregate validation check now in spec; update genesis times ( #2001 )
2020-11-12 15:29:32 +00:00
a0621d52f6
bump remaining rc.0 spec ref ( #2003 )
2020-11-12 13:58:41 +00:00
f642f71457
Calculate how much time syncing takes, and show it at statusbar.
2020-11-12 14:16:12 +02:00
332e248d91
rpc: setLogLevel ( #1991 )
...
* rpc: setLogLevel
also change default rpc port to match makefile and manual
* set topic-level formats too
and make feature less secret
2020-11-12 11:46:02 +01:00
43c6c5a0b3
Gossip message id should decompress data ( #1998 )
...
* also fix missing return on empty message
2020-11-12 11:45:28 +01:00
5644cfae8f
increase max db object size ( fixes #1996 )
2020-11-12 08:42:45 +01:00
c5710d97fc
default to gossipsub 1.1 ( #1992 )
2020-11-11 13:30:18 +01:00
389c11743a
Review TODO items and self-assign the most important ones
2020-11-10 20:41:04 +02:00
6e40ca5e15
Use separate cache for getting epochref
...
...else tmpState gets overwritten when producing blocks and requesting
epochRef for finalized state.
2020-11-10 17:40:32 +02:00
ea4d94c65c
Add guard task to monitor synchronization worker Future. ( #1972 )
...
* Add guard task to monitor synchronization worker Future.
* Change log events from debug to warning level.
2020-11-10 14:47:26 +01:00
7eaae89263
keep AllTests-mainnet.md consistent; ensure don't send overly large messages aren't sent ( #1973 )
2020-11-09 18:11:09 +00:00
a010ef2061
update tests from v1.0.0-rc.0 to v1.0.0 ( #1976 )
2020-11-09 17:44:18 +00:00
bd1047b715
0.6.0 release fixes
...
* Updated README
* No double v in nimbus_beacon_node --version
* No md5 checksums
2020-11-09 17:09:49 +02:00
271df8b604
bump 1.0.0rc-0 spec refs to 1.0.0 ( #1974 )
2020-11-09 14:18:55 +00:00
61a3209c24
remove version quotes
2020-11-09 13:37:58 +01:00
94e9ce3c1f
Add metadata for the toledo network
2020-11-09 11:38:52 +02:00
4b603d7f5a
Change the default log level to INFO
2020-11-09 11:38:52 +02:00
e9b9cd75ee
Rename binaries; Mimic the original repo layout in the distribution
2020-11-09 11:38:52 +02:00
bf58915500
release fixes ( #1966 )
2020-11-07 08:46:53 +01:00
d2476b643a
bump version ( #1964 )
...
* bump version
* update version format to "standard" style
* remove zinken/attacknet (both networks deprecated)
* add mainnet warning
2020-11-07 08:45:55 +01:00
fb6ce5c804
Enable discv5.1 for general beacon_node and inspector builds
2020-11-07 01:27:02 +02:00
ac1072b4bb
Add ./run-medalla-beacon-node script
2020-11-07 01:25:02 +02:00
4a710252d8
Check the web3 provider for network compatibility
2020-11-07 01:25:02 +02:00
24f8b7d483
[skip ci] Clean up some accidentally commited code
2020-11-06 11:53:44 +02:00
a8a66fe3f6
Fix #1784 ; Handle Infura HTTPS URLs
2020-11-06 11:15:32 +02:00
f596a24c4c
Mainnet deposit contract monitoring
2020-11-06 11:15:32 +02:00
95f5f76180
Datatype cleanup ( #1953 )
...
* clear up spec todo
* test fix
* remove unnecessary toSszType
* type
* one more
2020-11-04 21:52:47 +00:00
fc7885b27e
Store block summary in database
...
This introcudes a cache for block summaries, useful for instantiating
the block dag on startup, bringing medalla startup times down from
minutes to seconds.
This is something of a temporary band-aid that would be obsoleted by a
finalized block store.
2020-11-04 11:28:55 +02:00
7823c25ba8
Hotfix for a failing assert during block production
2020-11-04 11:06:58 +02:00
36b54fe091
Update eth2_network.nim
...
More informative and friendlier log message (from the user's perspective). Also helps avoid the potential for confusion with the validator key.
2020-11-03 23:23:32 +02:00
6ee80651c7
Rename mainchain_monitor to eth1_monitor
2020-11-03 23:23:10 +02:00
2f29e3e7f3
Further reduce the number of RPC requests; Significantly faster ETH1 syncing
2020-11-03 23:23:10 +02:00
14b2d4324d
openarray -> openArray
2020-11-03 23:23:10 +02:00
18639c3eff
Don't require requests that might fail on non-archive Geth nodes
2020-11-03 23:23:10 +02:00
79f1b6117e
Remove dead code from the beacon node binary; Fix 'make test'
2020-11-03 23:23:10 +02:00
07206ca264
An efficient algorithm for incremental generation of multiple merkle proofs
2020-11-03 23:23:10 +02:00
d913273dcc
Fix #1823
2020-11-03 23:23:10 +02:00
937decb0e9
Revert "Gossip11 post audit ( #1917 )" ( #1947 )
...
This reverts commit 63173ab2c1
2020-11-03 11:45:40 +01:00
9255945fb0
Update mainchain_monitor.nim ( #1945 )
...
Make it clear that this failure concerns the eth1 chain
2020-11-02 21:16:24 +01:00
002a3dfa54
graceful shutdown: move the hook registration ( #1940 )
...
...closer to where it starts having an effect.
2020-11-02 19:02:27 +01:00
2f049dc6e7
Fix init condition for fork choice ( #1941 )
...
... or we choose the wrong head on startup, sometimes.
2020-11-02 18:51:08 +01:00
ca63d48b82
tone down validator exit logging ( #1939 )
...
Validators exiting is normal, no need to scream about it
* avoid reallocating seq on big exit queue
* avoid fetching state cache when updating head (it's rarely needed)
* remove incorrectly implemented live validator counts (avoids memory
allocs)
2020-11-02 18:34:23 +01:00
7294d98ed1
Update validator_duties.nim (Improve node out of sync message) ( #1933 )
...
* Update validator_duties.nim
Improve node out of sync message
* Update validator_duties.nim
2020-11-02 08:52:16 +01:00
67113373f9
Work around BLST build issues
2020-10-30 23:56:04 +02:00
f76679810b
Fix #1855 ; Add support for wallet recovery passwords
2020-10-30 23:56:04 +02:00
63173ab2c1
Gossip11 post audit ( #1917 )
...
* bump libp2p gossip
* use gossip 1.1
* use proper libp2p commit hash
* switch to master libp2p, has audit fixes now
2020-10-30 14:26:35 +00:00
3f5c7c36bc
Some syncing fixes ( #1919 )
...
* Add exponential rewind on MissingParent.
* Try to avoid peers which are useless for syncing.
Fix forward sync restart at proper point.
Fix getLocalWallSlot() to not return slots from the future.
* Fix incorrect logs.
* Fix logging.
Enable peer's status messages log on DEBUG level.
* Fix watch task to monitor operation progress, but not local head progress.
* Add more logging information.
Remove recurring failures detection mechanism.
2020-10-30 13:33:52 +01:00
81f4fe0783
limit fork choice initialization horizon ( #1910 )
...
This helps manage the long startup time on chains that are not
finalizing for long periods of time
2020-10-29 12:09:03 +01:00
020a32ffa4
minor api cleanups ( #1913 )
...
* readd getNodeVersion
* implement get_v1_config_fork_schedule
* fix formatting of several types
2020-10-28 19:51:38 +01:00
19c51e5900
use MAXIMUM_GOSSIP_CLOCK_DISPARITY for BeaconBlock verification ( #1918 )
2020-10-28 14:04:21 +01:00
ff3f8741de
handle fork choice selecting a head unknown to chaindag ( #1907 )
...
* move head slot metric to head selection
* avoid redundant head selection in validator duties
2020-10-28 07:55:36 +00:00
6050eebec0
document not-nil issues
2020-10-27 13:01:45 +01:00
eee01a32d5
Regression fix of eth2_network_simulation on Windows. ( #1900 )
...
* Concentrate all sensitive writeFile/createPath calls in one place.
Fix eth2_network_simulation for Windows.
* Remove artifacts.
* fix import
Co-authored-by: Jacek Sieka <jacek@status.im>
2020-10-27 12:04:17 +01:00
c82ff24b5c
Syncing fixes ( #1909 )
...
* Fix continuous sync queue rewinds on slow PCs.
Fix recurring disconnects on low peer score.
* Calculate average syncing speed, not the current one.
Move speed calculation to different task.
* Address review comments.
2020-10-27 10:25:28 +01:00
ee2ebc96a8
move rpc api to own folder, mimic upstream structure ( #1905 )
...
also implements a few more endpoints
2020-10-27 10:00:57 +01:00
677f099110
BLS signature v4 ( #1912 )
2020-10-27 09:37:04 +01:00
8102d5dc4d
Bump eth2-testnets for more discv5.1 bootstrap nodes on medalla ( #1911 )
...
And improve logging UX a bit by ignoring comments or newlines
2020-10-27 08:44:46 +01:00
756b62fdf5
Bump nim-eth to get UseDiscv51 flag ( #1895 )
...
* Bump nim-eth to get UseDiscv51 flag
* Switch medalla to discovery v5.1, other targets to v5.0
* Bump nim-eth for better discv5.1 logging
* Bump eth2-testnets for updated medalla bootnodes
2020-10-23 18:59:58 +02:00
7c0b4d28d2
speed up reward/penalty calculation
...
Calculating rewards/penalties is slow due to how we compute sets of
attestations validators then use the sets for inclusion checks, to see
who attested. The dominant function during validated block processing /
epoch processing is hash set building and lookup.
This PR inverts the flow by removing the sets and creating a single
large validator status list, then applying all relevant state
attestations, then updating rewards and penalties.
This provides a 10x speedup to epoch processing which in turn speeds up
both empty slot and block processing - for example, on startup, we
replay all non-finalized blocks to prime fork choice - the same when
validating attestations or replaying states on reorg.
2020-10-23 19:23:36 +03:00
499e5ca991
misc memory and perf fixes ( #1899 )
...
* misc memory and perf fixes
* use EpochRef for attestation aggregation
* compress effective balances in memory (medalla unfinalized: 4gb ->
1gb)
* avoid hitting db when rewinding to head or clearance state
* avoid hitting db when blocks can be applied to in-memory state -
speeds up startup considerably
* avoid storing epochref in fork choice
* simplify and speed up beacon block creation flow - avoids state reload
thanks to head rewind optimization
* iterator-based committee and attestation participation help avoid lots
of small memory allocations throughout epoch transition (40% speedup on
epoch processing, for example during startup)
* add constant for threshold
2020-10-22 12:53:33 +02:00
decfd66a78
bump libp2p: seq[byte] message IDs and improve reject/ignore handling ( #1898 )
...
* bump libp2p: use seq[byte] message IDs and improve reject/ignore handling
* bump libp2p to pick up debug message details
2020-10-21 20:03:12 +00:00
9810a31447
Fix #1838 ( #1892 )
2020-10-20 15:01:21 +02:00
a136c2e95a
bump libp2p; integrate pubsub.ValidationResult into extended validation ( #1893 )
2020-10-20 12:31:20 +00:00
e87e0ef179
update to 1.0.0-rc.0 message ID; unexport funcs; enable test in mainnet; remove unused subscribe-with-nondummy-handler function; fix gossip received metric ( #1884 )
2020-10-20 08:54:11 +00:00
c446eb7cff
update v1.0.0-rc.0 preset spec references ( #1891 )
...
* update ve1.0.0-rc.0 preset spec references
* remove runtime preset ETH1_FOLLOW_DISTANCE from preset files; remove two CI build items to try to keep Travis from timing out
2020-10-19 13:53:23 +00:00
5b13a5ba10
update attestation extended validation to v1.0.0-rc.0 ( #1889 )
...
* update attestation extended validation to v1.0.0-rc.0
* attestation block and target must be within same epoch
* remove duplicate attestation epoch/target epoch check
* in spec now
2020-10-19 09:25:06 +00:00
df43b8aa8b
save some more states after all ( #1887 )
...
Don't save states when replaying history, but do save states when
applying new blocks (!)
2020-10-18 15:47:39 +00:00
dbc90e998a
Add Disc v5.1 bootstrap nodes for Zinken
...
Also prepares the Makefiles for supporting Medalla
through a separate binary
2020-10-18 12:44:52 +03:00
d98be078c3
Address #1690 Min password entropy
2020-10-16 19:34:26 +03:00
9899578e1c
Update get_eth1_data from v0.11.1 to v1.0.0-rc
2020-10-16 11:46:53 +03:00
9a543e0af7
partial hotfix for #1879 crash
2020-10-16 11:46:19 +03:00
948d15ea95
Follow-up on #1687 Don't ask for password on invalid keystores
2020-10-16 11:45:39 +03:00
db70f066ba
fixed all crashes which can be triggered through the validator RPC API - fixes #1680
2020-10-15 20:19:43 +03:00
7ed885f144
configuring a writer for the dynamic log output device for the VC binary as well (same as in the BN)
2020-10-15 20:19:10 +03:00
7ad59b1852
Bump nim-stew.
2020-10-15 20:18:33 +03:00
1742a82ca7
Proper fix for writeFile() places.
...
Comment unused/insecure procedure.
2020-10-15 20:18:33 +03:00
84fe294c00
Fix protection of data folder and security files using Windows ACL.
2020-10-15 20:18:33 +03:00
5f6bdc6709
Store all deposit-derived data in memory
2020-10-15 20:15:51 +03:00
7a577b2cef
More tests for getBlockRange
2020-10-15 20:15:51 +03:00
080609eee1
Address #1366 Avoid uint64 overflow in getBlockRange when skipStep is large
2020-10-15 20:15:51 +03:00
2152dc6136
Simplify the mainchain monitor
2020-10-15 20:15:51 +03:00
ce1fda1195
Store the deposits and the immutable validator data in Sqlite
2020-10-15 20:15:51 +03:00
e69af00e3a
Don't penalize flooding peers; Just rate limit them
2020-10-15 20:15:51 +03:00
4d66914f5a
Fix the test suite
2020-10-15 20:15:51 +03:00
f0892ce934
Fix compilation problems on Windows
2020-10-15 20:15:51 +03:00
e6320e5881
Address #1584 Don't keep all deposits in memory (persist them to disk)
2020-10-15 20:15:51 +03:00
bc8acdb9de
Switch to an in-house clearScreen to avoid problems with exception tracking
2020-10-15 20:15:51 +03:00
69e498dc00
Address #1689 and #1517 Usability and security improvements in wallet creation
2020-10-15 20:15:51 +03:00
02a3770803
Adds a fuzzing test for the validator client CLI
...
Also removes an obsolete way to add validators through the command line
2020-10-15 20:15:51 +03:00
8ce0fc3a89
Address #1687
...
This addresses the issues by detecting and rejecting keystores with
incorrect PBKDF2 and SCrypt params. It also bumps the version of
nim-json-serialization to include a bugfix for incorrect parsing
of json files featuring comments.
2020-10-15 20:15:51 +03:00
cd949a2b81
Address #1691 More pervasive handling of Ctrl+C
2020-10-15 20:15:51 +03:00
8a6281aad2
Simple cost model for sync requests; Penalize peers perfoming flooding or invalid requests
2020-10-15 20:15:51 +03:00
6b9419e547
fix db growth on attestation processing ( #1860 )
...
It turns out that we often save lots of states in the database that are
the result of empty slot processing only - here, we make sure to only
save a state if a block follows - this fixes several issues:
* empty slot states are not always pruned leading to state database size
explosion
* storing states is (very) slow which slows down processing in general,
so we should only do it when it's likely to be useful
* attestation processing doesn't get stuck on saving random states that
won't appear in the chain history
2020-10-15 14:28:44 +02:00
4e73d10f86
removed 2 buggy RPC API endpoints - there are 2 well functioning ones in validator_api.nim anyway ( #1867 )
2020-10-15 14:26:59 +02:00
da59f45a90
update spec/datatypes spec refs from v0.12.3 to v1.0.0 ( #1866 )
2020-10-14 10:00:27 +00:00
513ba72b9a
add v1.0.0-rc.0 support behind compile-time flag ( #1852 )
...
* add v1.0.0-rc.0 support behind compile-time flag
* keep runtime presets consistent
2020-10-13 17:21:25 +00:00
3ee2dd8da4
p2p-interface spec ref bump (except non-updated places) ( #1862 )
2020-10-12 14:37:14 +00:00
1994ffe5a0
update 130+ spec references from v0.12.3 to v1.0.0-rc1 ( #1854 )
2020-10-12 08:59:24 +00:00
00a8a68671
Address #1695
...
Better error messages when the beacon node is asked to listen on a
reserved port (0) or an already taken one.
2020-10-09 16:39:03 +03:00
d69d89866e
fix logtrace in local network sim ( #1840 )
2020-10-09 10:58:54 +02:00
b79e5f8af5
update nim-beacon-chain to nimbus-eth2 in beacon_chain/, ncli/, tests/, and README.md ( #1843 )
2020-10-08 19:02:05 +00:00
c0aa2f1e58
update process_slashings() to use PROPORTIONAL_SLASHING_MULTIPLIER ( #1841 )
2020-10-08 18:00:07 +00:00
3bd7ab4c20
Do not reward empty responses. ( #1827 )
...
Request status from peers every StatusExpirationTime.
2020-10-08 14:50:48 +02:00
f08f44b9a2
in exit pool, bundle receive messages into beaconblocks ( #1812 )
...
* in exit pool, filter out already-packaged messages; bundle remaining messages into beaconblocks
* filter messages at block construction time
* allow adding up to intended capacity of buffers, beyond per-block limits
* document rationale/design for filtering mechanism
2020-10-07 16:57:21 +00:00
b46c8410a6
Retire Spadina and prepare for the Zinken network
2020-10-07 18:12:21 +03:00
6b9090398e
[JSON-RPC] Prevent silent uint conversion ( #1819 )
...
* Prevent silent uint conversion
* bump json-rpc
2020-10-07 15:02:54 +02:00
deaddc1fc0
Address review comments
2020-10-07 09:32:03 +03:00
aed291128a
Add support for starting from weak subjectivity checkpoints
...
Also removes the `genesis.ssz` file stored in the data folder.
The `medalla-fast-sync` target has been adapted to use the new features.
2020-10-07 09:32:03 +03:00
2c06492407
Address #1728
2020-10-06 15:46:22 +03:00
950bcd0f6d
Address #1729 : Min password length
2020-10-06 15:46:22 +03:00
78953fd9b5
Address #1729 : NFKD Normalization
2020-10-06 15:46:22 +03:00
fec4b5014d
Address #1694
2020-10-06 15:46:22 +03:00
47d4899911
Wallet creation does not allow overwriting an existing wallet
2020-10-06 15:46:22 +03:00
9d23094c68
Address #1692
2020-10-06 15:46:22 +03:00
27fc8bcc80
Address #1538 and do not penalize peers for empty response. ( #1811 )
2020-10-06 14:10:02 +02:00
6bb6b66766
Partial LTO ( #1789 )
...
* initial partial LTO example
* Add select private key related files to ignore LTO
* Apply to keystores
2020-10-06 13:38:18 +02:00
c92d228ec8
Fix non-split attesting protection ( #1804 )
...
* address #1679 - att protection in non-split mode
* AttachedValidator public_key -> pubkey
* Always build with slashing protection
* Remove now unnecessary forwarding impl proc
2020-10-06 10:51:33 +02:00
5354a0a3d5
fix #1585 and restore storeLEBytes on big-endian systems ( #1796 )
...
* fix #1585 and restore storeLEBytes on big-endian systems
* switch to using stew/endians2
* create static endians2-like shim in bitseqs
2020-10-06 08:00:11 +00:00
8c59b92081
fixed #1663 - Interger overflow in compute_start_slot_at_epoch through RPC ( #1810 )
...
* fixed #1663 - Interger overflow in compute_start_slot_at_epoch through RPC
* changed the way the overflow check is done - took the approach from PR #1797 - see the comment in PR #1810 for more details
2020-10-05 21:58:30 +00:00
00a500ef3c
Fix rebase issues.
2020-10-05 22:19:50 +03:00
f091c8d4df
Add chronicles.formatIt for PublicKey.
...
Fix logs.
Rename checkFilePermissions to checkSensitiveFilePermissions.
2020-10-05 22:19:50 +03:00
dd8b73c85c
Remove `keypath` and set version to 1 for network keystorage.
2020-10-05 22:19:50 +03:00
9441e912cb
Merge different places of keyboard input to keyboardCreatePassword() and keyboardGetPassword().
...
Fix Windows problem with english words array.
Add checksum for english words array verification.
2020-10-05 22:19:50 +03:00
868ce14762
Fix compilation issue.
2020-10-05 22:19:50 +03:00
82228fe471
Fix last createDir.
2020-10-05 22:19:50 +03:00
86139839f1
Add permissions checks and handling to wallets and bls keystores.
2020-10-05 22:19:50 +03:00
fc09c2b3d4
Fix compilation problem.
2020-10-05 22:19:50 +03:00
1d76683a8b
Fix os dependency.
2020-10-05 22:19:50 +03:00
20f695515a
Fix localtestnet simulation.
2020-10-05 22:19:50 +03:00
add22a20e1
Update local_testnet and simulation scripts to use netkey-file and insecure-netkey-password.
...
Add more logging
2020-10-05 22:19:50 +03:00
e1182f8000
Add insecure password for automated testing.
...
Fix checkDataDir to run before setupLogging.
2020-10-05 22:19:50 +03:00
e43af4e978
Fix CLI option.
2020-10-05 22:19:50 +03:00
40f2b74f73
Add keystore management and interactive password handling.
2020-10-05 22:19:50 +03:00
ae72c08573
Fix utils problem.
2020-10-05 22:19:50 +03:00
cec4ad2717
Refactor keystore.nim to include network key storage too.
...
Use constant-time hex encoding/decoding procedures.
Add tests for network key storage.
2020-10-05 22:19:50 +03:00
c5c788a9db
Secure network key file and data directory.
2020-10-05 22:19:50 +03:00
d9738b43b3
Revert "Address issue #1538."
...
This reverts commit 1be3fd53b1
2020-10-05 19:25:47 +03:00
1be3fd53b1
Address issue #1538 .
2020-10-05 13:23:25 +03:00
67a93f558b
Address issue #1714 .
2020-10-05 13:22:45 +03:00
99afafecd7
fix quadratic seq assignment in fork choice ( #1805 )
...
this would reallocate the attestation queue on every attestation and
other call to update_time, causing quite the overhead (~10% cpu spent
when gossiping)
2020-10-03 23:43:27 +02:00
b57693ec0d
Logging update ( #1795 )
...
* Fix discovery log message trigger
* Bump chronicles - include https://github.com/status-im/nim-chronicles/pull/89 for better NOTICE/WARNING color
2020-10-03 08:35:45 +02:00
5e95fd7468
Revert "update to v0.12.3 message ID for Spadina launch ( #1762 )" ( #1801 )
...
This reverts commit a2270a5f27
2020-10-02 19:50:21 +00:00
f5340998b9
Add a fuzzing test for the beacon node CLI parser
2020-10-02 12:42:32 +03:00
ee114deb82
fix #1782 by checking for empty attester index sets ( #1786 )
2020-10-01 21:05:22 +02:00
0280d6c73e
Revisiting log levels ( #1788 )
...
* Update log level - https://github.com/status-im/nim-beacon-chain/issues/1779 https://github.com/status-im/nim-beacon-chain/issues/1785
* Address review comments
* Document the logging strategy [skip ci]
2020-10-01 20:56:42 +02:00
13d4831664
fix #1783 by not assuming that an attached validator will attest ( #1787 )
2020-10-01 14:48:24 +00:00
7eaaab908c
fix output of proposer slashing test fixture ( #1780 )
...
* fix output of proposer slashing test fixture
* run make test
* a few more v0.12.3 spec refs
2020-09-30 13:12:03 +00:00
0c921b7bc0
Multinet fixes ( #1774 )
...
* add missing hardcoded preset value
* wip
* wip
2020-09-29 10:12:57 +02:00
f6b8e26567
Fix typos [skip ci] ( #1771 )
2020-09-28 20:43:09 +02:00
2e6fd8976a
Ctrl+C now stops the BN when waiting for genesis ( #1761 )
...
* Ctrl+C now stops the BN when waiting for genesis
* proper shutdown when waiting for genesis if Ctrl+C is used
2020-09-28 17:19:57 +02:00
a2270a5f27
update to v0.12.3 message ID for Spadina launch ( #1762 )
...
* update to v0.12.3 message ID for Spadina launch
* remove base64 import
2020-09-28 17:07:10 +02:00
23bec99341
Let also discovery used listen-address cli option instead of always any address ( #1658 )
2020-09-27 22:00:24 +02:00
c472d5368b
avoid attestation pool copy in check ( #1755 )
2020-09-25 19:51:44 +02:00
94120ad011
Slash protection fixes ( #1757 )
...
* Address #1698
* Add test for #1699
2020-09-25 19:39:06 +02:00
7837646079
anonymize libp2p messages ( #1756 )
...
* anonymize libp2p messages
* bump
2020-09-25 18:40:30 +02:00
b6262bfe46
Bugfix: prevent an error during deposit data generation
2020-09-25 17:11:36 +03:00
1bf8d3af33
Disconnect peers with low score. ( #1747 )
...
* Disconnect peers with low score.
* Change PeerScoreLow value.
* Add spec url for DisconnectionReason.
2020-09-25 15:43:45 +02:00
f96ad87d28
switch another 50+ spec refs from v0.12.2 to v0.12.3 ( #1749 )
2020-09-25 11:52:50 +00:00
6cf7e837ce
refactor exit pool and gossipsub-subscribe to exit pool message types ( #1733 )
...
* refactor exit pool and gossipsub-subscribe to all exit pool message types
* remove unused loop counter
2020-09-24 19:05:49 +02:00
6398a43cc1
update 120+ beacon_chain and validator spec refs from v0.12.2 to v0.12.3 ( #1740 )
2020-09-24 19:04:10 +02:00
b3a9afa0b1
libp2p: limit max gossip writes ( #1739 )
...
* libp2p: limit max gossip writes
* bump
2020-09-24 19:03:17 +02:00
0eb53f2802
avoid unpacking phase 1 tests to reduce Azure CI disk usage ( #1736 )
2020-09-24 17:16:00 +02:00
410a72667b
fixed interop with geth after updating nim-web3
2020-09-24 16:07:57 +03:00
2cd0c3adaa
Fix condition. ( #1734 )
2020-09-24 14:14:29 +02:00
b9deff43ce
fixed build after rebasing on top of the recently introduced slashing protection
2020-09-24 11:31:14 +03:00
e926db22b9
initial implementation of attestation aggregation for the validator API
2020-09-24 11:31:14 +03:00
f0dbebfd3f
avoid storing empty slot states ( #1720 )
...
with the improved empty slot processing, these provide relatively little
benefit, but take up lots of storage that's difficult to free
2020-09-24 09:02:03 +02:00
4e23b0ef23
Fix https://github.com/status-im/nim-beacon-chain/issues/1701 + raise concerns in keystore secrets protection ( #1726 )
2020-09-24 07:27:56 +02:00
08795b3f5d
Fix tight loop at the end of sync process. ( #1731 )
2020-09-23 17:58:02 +02:00
e1c177cdd1
bump libp2p ( #1721 )
...
gossipsub 1.1 can be enabled with -d:nbc_gossipsub_11
2020-09-22 19:34:34 +02:00
73724b2c4b
Add attacknet-beta1-mc-0
2020-09-22 18:51:51 +03:00
10cfd1ec37
Adds the Spedina testnet; Upgrades the spec presets to spec v0.12.3
...
Other changes:
* Removed all references to Altona
* Refactored the Makefile to avoid code duplication
2020-09-22 18:51:51 +03:00
654b8d66bf
Peer management ( #1707 )
...
* addPeer() and addPeerNoWait() now returns PeerStatus, not bool.
Minor refactoring of PeerPool.
Fix tests.
* Refactor PeerPool.
Add lenSpace.
Add tests for lenSpace.
PeerPool.add procedures now return different error codes.
Fix SyncManager break/continue problem.
Fix connectWorker break/continue problem.
Refactor connectWorker and discoveryLoop.
Fix incoming/outgoing blocking problem.
* Refactor discovery loop.
Add checkPeer.
* Fix logic and compilation bugs.
* Adjust position of debugging log.
* Fix issue with maximum peers in PeerPool.
Optimize node record decoding.
* fix discoveryLoop.
* Remove aliases and fix tests using aliases.
2020-09-21 18:02:27 +02:00
3190c695b0
minimal v0.12.3 update ( #1716 )
2020-09-21 15:58:35 +00:00
fc10f5121a
protect against data after initial request
...
spec requires that channel is closed
also, avoid some unnecessary futures
2020-09-18 21:34:07 +03:00
7c688bffda
Address #1586 ( #1596 )
2020-09-18 18:34:13 +02:00
ea4ec6a785
Upgrade blst ( #1682 )
...
* Bump BLST
* Test for https://github.com/supranational/blst/issues/22 regression
* Use SHA256 from BLST + bump nim-blscurve to reenable fno-tree-vectorize
* SHA256 on non-blst platforms import fixes
* import fixes again
* can't prefix with nimcrypto
* address review comment [skip ci]
* {.noInit.} on the digests
2020-09-18 16:55:55 +02:00
e106549efe
keep REJECT/IGNORE of messages failing validation for libp2p scoring ( #1676 )
...
* keep REJECT/IGNORE status of messages failing validation for libp2p scoring
* fix test suite
2020-09-18 13:53:09 +02:00
2f89e2ab4e
drop subscribePeer, it's called from pubsub now ( #1677 )
2020-09-17 11:40:21 +02:00
dcf8a6b05d
improve slot processing speeds ( #1670 )
...
about 40% better slot processing times (with LTO enabled) - these don't
do BLS but are used
heavily during replay (state transition = slot + block transition)
tests using a recent medalla state and advancing it 1000 slots:
```
./ncli slots --preState2:state-302271-3c1dbf19-c1f944bf.ssz --slot:1000
--postState2:xx.ssz
```
pre:
```
All time are ms
Average, StdDev, Min, Max, Samples,
Test
Validation is turned off meaning that no BLS operations are performed
39.236, 0.000, 39.236, 39.236, 1,
Load state from file
0.049, 0.002, 0.046, 0.063, 968,
Apply slot
256.504, 81.008, 213.471, 591.902, 32,
Apply epoch slot
28.597, 0.000, 28.597, 28.597, 1,
Save state to file
```
cast:
```
All time are ms
Average, StdDev, Min, Max, Samples,
Test
Validation is turned off meaning that no BLS operations are performed
37.079, 0.000, 37.079, 37.079, 1,
Load state from file
0.042, 0.002, 0.040, 0.090, 968,
Apply slot
215.552, 68.763, 180.155, 500.103, 32,
Apply epoch slot
25.106, 0.000, 25.106, 25.106, 1,
Save state to file
```
cast+rewards:
```
All time are ms
Average, StdDev, Min, Max, Samples,
Test
Validation is turned off meaning that no BLS operations are performed
40.049, 0.000, 40.049, 40.049, 1,
Load state from file
0.048, 0.001, 0.045, 0.060, 968,
Apply slot
164.981, 76.273, 142.099, 477.868, 32,
Apply epoch slot
28.498, 0.000, 28.498, 28.498, 1,
Save state to file
```
cast+rewards+shr
```
All time are ms
Average, StdDev, Min, Max, Samples,
Test
Validation is turned off meaning that no BLS operations are performed
12.898, 0.000, 12.898, 12.898, 1,
Load state from file
0.039, 0.002, 0.038, 0.054, 968,
Apply slot
139.971, 68.797, 120.088, 428.844, 32,
Apply epoch slot
24.761, 0.000, 24.761, 24.761, 1,
Save state to file
```
2020-09-16 20:59:33 +00:00
52548f079b
Opt-in Slashing protection + interchange ( #1643 )
...
* Slashing protection + interchange initial commit
* Restrict the when UseSlashingProtection dance in other modules
* Integrate slashing tests in other all_tests
* Add attestation slashing protection support
* Add a message that mention if built with/without slashing protection
* no op the initialization proc
* test slashing protection in Jenkins (temp)
* where to configure NIMFLAGS in Jenkins ...
* Jenkins -> ensure Built with slashing protection
* Add slashing protection complete import
* use Opt.get(otherwise)
* Don't use negation in proc name
* Turn slashing protection on by default
2020-09-16 13:30:03 +02:00
6e463257f4
PeerPool fixes. ( #1654 )
...
* Refactor peer_pool.
Fix eth2_network peer counters.
Fix PeerPool do not allow to add more peers when empty space available.
* Remove unused imports.
* Add test for a bug.
* Fix eth2_network disconnect should deletePeer not release.
More PeerPool refactoring.
2020-09-16 13:00:11 +03:00
6d8130dc49
close block_sim database; remove code duplication in exit_pool ( #1656 )
2020-09-16 09:16:23 +02:00
9abdbdabd8
Fix sync_manager.nim not rewarding peers for good responses. ( #1660 )
2020-09-16 09:15:06 +02:00
7e10b9850b
enable topic unsubscribing and attestation subnet cycling ( #1646 )
...
* enable topic unsubscribing and attestation subnet cycling
* remove refences to states
* waitFor -> await
* revert exit pool changes for a cleaner PR
2020-09-15 12:40:43 +00:00
c76305f824
fix some todo ( #1645 )
...
* remove some superfluous gcsafes
* remove getTailState (unused)
* don't store old epochrefs in blocks
* document attestation pool a bit
* remove `pcs =` cruft from log
2020-09-14 14:50:03 +00:00
9f21bbd666
[WIP] skeleton of attester slashing pool & validators ( #1639 )
...
* skeleton of attester slashing pool & validators
* add skeleton for proposer slashings and voluntary exits; rename pool to more inclusive exit pool to stay consistent with all three; ensure is initialized by beacon_node so is safe to merge, even if it doesn't do much yet
2020-09-14 16:26:31 +02:00
60b8905852
reorder startup steps ( #1642 )
...
so the Ctr+C hook and the metrics server go before the potentially slow
`BeaconNode.init()`
2020-09-14 08:24:31 +02:00
aca1a318f2
cleanly close kvstore databases and bump nim-eth ( #1630 )
...
* cleanly close kvstore databases
* close databases for all subcommands and during error conditions
2020-09-12 05:35:58 +00:00
Jacek Sieka
cheatfate
Zahary Karadjov
tersec
Giovanni Petrantoni
kdeme
Ștefan Talpalaru
Sacha Saint-Leger
Mamy Ratsimbazafy
Viktor Kirilov
Giovanni Petrantoni
Dmitriy Ryajov