21c4ce8fd4
remove superfluous TODOs/not-really-TODOs, type conversion, imports ( #2025 )
2020-11-16 17:10:51 +01:00
19707e89c2
Change random network key log line and include PeerID.
2020-11-16 17:41:23 +02:00
5b011b65ae
add rpc call to get chronos futures at runtime
2020-11-16 12:31:39 +02:00
e2f161dbf7
fix attestation sending schedules to avoid timing attack ( #1995 )
2020-11-16 10:44:18 +01:00
b022dc4d1f
Use O(n) algorithm in initialize_beacon_state_from_eth1; Avoid unnecessary merkle proofs generation
2020-11-15 21:40:40 +02:00
8012102704
Bugfix: the client can miss the genesis event in the absence of new deposits
2020-11-15 21:40:40 +02:00
9f28a464fb
[skip ci] Update the version number
2020-11-14 23:26:46 +02:00
d02d8f8d51
Drop UseDiscv51 flag and only support discovery v5.1
2020-11-14 19:12:30 +02:00
80ca7a2d9f
Implement #1768
2020-11-12 22:29:43 +02:00
17d35e1fd9
Allow the node to start when it fails to initialize the Eth1 monitor
...
* Avoid hangs when wss:// is specified for a non-secure HTTP server
* Produce an ERROR when the web3 provider is unsupported, but still launch the node
2020-11-12 22:29:43 +02:00
5e45e7429e
Rewrite all HTTP(S) web3 URLs to WebSocket URls
2020-11-12 22:29:43 +02:00
87c92ea05f
Address review comments.
...
Remove `-8.0` bucket because `-4.0` is a limit.
2020-11-12 21:52:01 +02:00
dbb058682c
Add beacon_attestation_sent_delay metric.
2020-11-12 21:52:01 +02:00
8ad45de65b
Add makefile targets for Pyrmont
2020-11-12 21:51:15 +02:00
b9e122a60c
address some TODOs ( #2005 )
2020-11-12 19:24:07 +00:00
e035b7d0c9
Fix status rpc request to use RESP_TIMEOUT. ( #2002 )
2020-11-12 15:30:12 +00:00
754d65c159
non-spec aggregate validation check now in spec; update genesis times ( #2001 )
2020-11-12 15:29:32 +00:00
a0621d52f6
bump remaining rc.0 spec ref ( #2003 )
2020-11-12 13:58:41 +00:00
f642f71457
Calculate how much time syncing takes, and show it at statusbar.
2020-11-12 14:16:12 +02:00
332e248d91
rpc: setLogLevel ( #1991 )
...
* rpc: setLogLevel
also change default rpc port to match makefile and manual
* set topic-level formats too
and make feature less secret
2020-11-12 11:46:02 +01:00
43c6c5a0b3
Gossip message id should decompress data ( #1998 )
...
* also fix missing return on empty message
2020-11-12 11:45:28 +01:00
5644cfae8f
increase max db object size ( fixes #1996 )
2020-11-12 08:42:45 +01:00
c5710d97fc
default to gossipsub 1.1 ( #1992 )
2020-11-11 13:30:18 +01:00
389c11743a
Review TODO items and self-assign the most important ones
2020-11-10 20:41:04 +02:00
6e40ca5e15
Use separate cache for getting epochref
...
...else tmpState gets overwritten when producing blocks and requesting
epochRef for finalized state.
2020-11-10 17:40:32 +02:00
ea4d94c65c
Add guard task to monitor synchronization worker Future. ( #1972 )
...
* Add guard task to monitor synchronization worker Future.
* Change log events from debug to warning level.
2020-11-10 14:47:26 +01:00
7eaae89263
keep AllTests-mainnet.md consistent; ensure don't send overly large messages aren't sent ( #1973 )
2020-11-09 18:11:09 +00:00
a010ef2061
update tests from v1.0.0-rc.0 to v1.0.0 ( #1976 )
2020-11-09 17:44:18 +00:00
bd1047b715
0.6.0 release fixes
...
* Updated README
* No double v in nimbus_beacon_node --version
* No md5 checksums
2020-11-09 17:09:49 +02:00
271df8b604
bump 1.0.0rc-0 spec refs to 1.0.0 ( #1974 )
2020-11-09 14:18:55 +00:00
61a3209c24
remove version quotes
2020-11-09 13:37:58 +01:00
94e9ce3c1f
Add metadata for the toledo network
2020-11-09 11:38:52 +02:00
4b603d7f5a
Change the default log level to INFO
2020-11-09 11:38:52 +02:00
e9b9cd75ee
Rename binaries; Mimic the original repo layout in the distribution
2020-11-09 11:38:52 +02:00
bf58915500
release fixes ( #1966 )
2020-11-07 08:46:53 +01:00
d2476b643a
bump version ( #1964 )
...
* bump version
* update version format to "standard" style
* remove zinken/attacknet (both networks deprecated)
* add mainnet warning
2020-11-07 08:45:55 +01:00
fb6ce5c804
Enable discv5.1 for general beacon_node and inspector builds
2020-11-07 01:27:02 +02:00
ac1072b4bb
Add ./run-medalla-beacon-node script
2020-11-07 01:25:02 +02:00
4a710252d8
Check the web3 provider for network compatibility
2020-11-07 01:25:02 +02:00
24f8b7d483
[skip ci] Clean up some accidentally commited code
2020-11-06 11:53:44 +02:00
a8a66fe3f6
Fix #1784 ; Handle Infura HTTPS URLs
2020-11-06 11:15:32 +02:00
f596a24c4c
Mainnet deposit contract monitoring
2020-11-06 11:15:32 +02:00
95f5f76180
Datatype cleanup ( #1953 )
...
* clear up spec todo
* test fix
* remove unnecessary toSszType
* type
* one more
2020-11-04 21:52:47 +00:00
fc7885b27e
Store block summary in database
...
This introcudes a cache for block summaries, useful for instantiating
the block dag on startup, bringing medalla startup times down from
minutes to seconds.
This is something of a temporary band-aid that would be obsoleted by a
finalized block store.
2020-11-04 11:28:55 +02:00
7823c25ba8
Hotfix for a failing assert during block production
2020-11-04 11:06:58 +02:00
36b54fe091
Update eth2_network.nim
...
More informative and friendlier log message (from the user's perspective). Also helps avoid the potential for confusion with the validator key.
2020-11-03 23:23:32 +02:00
6ee80651c7
Rename mainchain_monitor to eth1_monitor
2020-11-03 23:23:10 +02:00
2f29e3e7f3
Further reduce the number of RPC requests; Significantly faster ETH1 syncing
2020-11-03 23:23:10 +02:00
14b2d4324d
openarray -> openArray
2020-11-03 23:23:10 +02:00
18639c3eff
Don't require requests that might fail on non-archive Geth nodes
2020-11-03 23:23:10 +02:00
79f1b6117e
Remove dead code from the beacon node binary; Fix 'make test'
2020-11-03 23:23:10 +02:00
07206ca264
An efficient algorithm for incremental generation of multiple merkle proofs
2020-11-03 23:23:10 +02:00
d913273dcc
Fix #1823
2020-11-03 23:23:10 +02:00
937decb0e9
Revert "Gossip11 post audit ( #1917 )" ( #1947 )
...
This reverts commit 63173ab2c1
2020-11-03 11:45:40 +01:00
9255945fb0
Update mainchain_monitor.nim ( #1945 )
...
Make it clear that this failure concerns the eth1 chain
2020-11-02 21:16:24 +01:00
002a3dfa54
graceful shutdown: move the hook registration ( #1940 )
...
...closer to where it starts having an effect.
2020-11-02 19:02:27 +01:00
2f049dc6e7
Fix init condition for fork choice ( #1941 )
...
... or we choose the wrong head on startup, sometimes.
2020-11-02 18:51:08 +01:00
ca63d48b82
tone down validator exit logging ( #1939 )
...
Validators exiting is normal, no need to scream about it
* avoid reallocating seq on big exit queue
* avoid fetching state cache when updating head (it's rarely needed)
* remove incorrectly implemented live validator counts (avoids memory
allocs)
2020-11-02 18:34:23 +01:00
7294d98ed1
Update validator_duties.nim (Improve node out of sync message) ( #1933 )
...
* Update validator_duties.nim
Improve node out of sync message
* Update validator_duties.nim
2020-11-02 08:52:16 +01:00
67113373f9
Work around BLST build issues
2020-10-30 23:56:04 +02:00
f76679810b
Fix #1855 ; Add support for wallet recovery passwords
2020-10-30 23:56:04 +02:00
63173ab2c1
Gossip11 post audit ( #1917 )
...
* bump libp2p gossip
* use gossip 1.1
* use proper libp2p commit hash
* switch to master libp2p, has audit fixes now
2020-10-30 14:26:35 +00:00
3f5c7c36bc
Some syncing fixes ( #1919 )
...
* Add exponential rewind on MissingParent.
* Try to avoid peers which are useless for syncing.
Fix forward sync restart at proper point.
Fix getLocalWallSlot() to not return slots from the future.
* Fix incorrect logs.
* Fix logging.
Enable peer's status messages log on DEBUG level.
* Fix watch task to monitor operation progress, but not local head progress.
* Add more logging information.
Remove recurring failures detection mechanism.
2020-10-30 13:33:52 +01:00
81f4fe0783
limit fork choice initialization horizon ( #1910 )
...
This helps manage the long startup time on chains that are not
finalizing for long periods of time
2020-10-29 12:09:03 +01:00
020a32ffa4
minor api cleanups ( #1913 )
...
* readd getNodeVersion
* implement get_v1_config_fork_schedule
* fix formatting of several types
2020-10-28 19:51:38 +01:00
19c51e5900
use MAXIMUM_GOSSIP_CLOCK_DISPARITY for BeaconBlock verification ( #1918 )
2020-10-28 14:04:21 +01:00
ff3f8741de
handle fork choice selecting a head unknown to chaindag ( #1907 )
...
* move head slot metric to head selection
* avoid redundant head selection in validator duties
2020-10-28 07:55:36 +00:00
6050eebec0
document not-nil issues
2020-10-27 13:01:45 +01:00
eee01a32d5
Regression fix of eth2_network_simulation on Windows. ( #1900 )
...
* Concentrate all sensitive writeFile/createPath calls in one place.
Fix eth2_network_simulation for Windows.
* Remove artifacts.
* fix import
Co-authored-by: Jacek Sieka <jacek@status.im>
2020-10-27 12:04:17 +01:00
c82ff24b5c
Syncing fixes ( #1909 )
...
* Fix continuous sync queue rewinds on slow PCs.
Fix recurring disconnects on low peer score.
* Calculate average syncing speed, not the current one.
Move speed calculation to different task.
* Address review comments.
2020-10-27 10:25:28 +01:00
ee2ebc96a8
move rpc api to own folder, mimic upstream structure ( #1905 )
...
also implements a few more endpoints
2020-10-27 10:00:57 +01:00
677f099110
BLS signature v4 ( #1912 )
2020-10-27 09:37:04 +01:00
8102d5dc4d
Bump eth2-testnets for more discv5.1 bootstrap nodes on medalla ( #1911 )
...
And improve logging UX a bit by ignoring comments or newlines
2020-10-27 08:44:46 +01:00
756b62fdf5
Bump nim-eth to get UseDiscv51 flag ( #1895 )
...
* Bump nim-eth to get UseDiscv51 flag
* Switch medalla to discovery v5.1, other targets to v5.0
* Bump nim-eth for better discv5.1 logging
* Bump eth2-testnets for updated medalla bootnodes
2020-10-23 18:59:58 +02:00
7c0b4d28d2
speed up reward/penalty calculation
...
Calculating rewards/penalties is slow due to how we compute sets of
attestations validators then use the sets for inclusion checks, to see
who attested. The dominant function during validated block processing /
epoch processing is hash set building and lookup.
This PR inverts the flow by removing the sets and creating a single
large validator status list, then applying all relevant state
attestations, then updating rewards and penalties.
This provides a 10x speedup to epoch processing which in turn speeds up
both empty slot and block processing - for example, on startup, we
replay all non-finalized blocks to prime fork choice - the same when
validating attestations or replaying states on reorg.
2020-10-23 19:23:36 +03:00
499e5ca991
misc memory and perf fixes ( #1899 )
...
* misc memory and perf fixes
* use EpochRef for attestation aggregation
* compress effective balances in memory (medalla unfinalized: 4gb ->
1gb)
* avoid hitting db when rewinding to head or clearance state
* avoid hitting db when blocks can be applied to in-memory state -
speeds up startup considerably
* avoid storing epochref in fork choice
* simplify and speed up beacon block creation flow - avoids state reload
thanks to head rewind optimization
* iterator-based committee and attestation participation help avoid lots
of small memory allocations throughout epoch transition (40% speedup on
epoch processing, for example during startup)
* add constant for threshold
2020-10-22 12:53:33 +02:00
decfd66a78
bump libp2p: seq[byte] message IDs and improve reject/ignore handling ( #1898 )
...
* bump libp2p: use seq[byte] message IDs and improve reject/ignore handling
* bump libp2p to pick up debug message details
2020-10-21 20:03:12 +00:00
9810a31447
Fix #1838 ( #1892 )
2020-10-20 15:01:21 +02:00
a136c2e95a
bump libp2p; integrate pubsub.ValidationResult into extended validation ( #1893 )
2020-10-20 12:31:20 +00:00
e87e0ef179
update to 1.0.0-rc.0 message ID; unexport funcs; enable test in mainnet; remove unused subscribe-with-nondummy-handler function; fix gossip received metric ( #1884 )
2020-10-20 08:54:11 +00:00
c446eb7cff
update v1.0.0-rc.0 preset spec references ( #1891 )
...
* update ve1.0.0-rc.0 preset spec references
* remove runtime preset ETH1_FOLLOW_DISTANCE from preset files; remove two CI build items to try to keep Travis from timing out
2020-10-19 13:53:23 +00:00
5b13a5ba10
update attestation extended validation to v1.0.0-rc.0 ( #1889 )
...
* update attestation extended validation to v1.0.0-rc.0
* attestation block and target must be within same epoch
* remove duplicate attestation epoch/target epoch check
* in spec now
2020-10-19 09:25:06 +00:00
df43b8aa8b
save some more states after all ( #1887 )
...
Don't save states when replaying history, but do save states when
applying new blocks (!)
2020-10-18 15:47:39 +00:00
dbc90e998a
Add Disc v5.1 bootstrap nodes for Zinken
...
Also prepares the Makefiles for supporting Medalla
through a separate binary
2020-10-18 12:44:52 +03:00
d98be078c3
Address #1690 Min password entropy
2020-10-16 19:34:26 +03:00
9899578e1c
Update get_eth1_data from v0.11.1 to v1.0.0-rc
2020-10-16 11:46:53 +03:00
9a543e0af7
partial hotfix for #1879 crash
2020-10-16 11:46:19 +03:00
948d15ea95
Follow-up on #1687 Don't ask for password on invalid keystores
2020-10-16 11:45:39 +03:00
db70f066ba
fixed all crashes which can be triggered through the validator RPC API - fixes #1680
2020-10-15 20:19:43 +03:00
7ed885f144
configuring a writer for the dynamic log output device for the VC binary as well (same as in the BN)
2020-10-15 20:19:10 +03:00
7ad59b1852
Bump nim-stew.
2020-10-15 20:18:33 +03:00
1742a82ca7
Proper fix for writeFile() places.
...
Comment unused/insecure procedure.
2020-10-15 20:18:33 +03:00
84fe294c00
Fix protection of data folder and security files using Windows ACL.
2020-10-15 20:18:33 +03:00
5f6bdc6709
Store all deposit-derived data in memory
2020-10-15 20:15:51 +03:00
7a577b2cef
More tests for getBlockRange
2020-10-15 20:15:51 +03:00
080609eee1
Address #1366 Avoid uint64 overflow in getBlockRange when skipStep is large
2020-10-15 20:15:51 +03:00
2152dc6136
Simplify the mainchain monitor
2020-10-15 20:15:51 +03:00
ce1fda1195
Store the deposits and the immutable validator data in Sqlite
2020-10-15 20:15:51 +03:00
e69af00e3a
Don't penalize flooding peers; Just rate limit them
2020-10-15 20:15:51 +03:00
4d66914f5a
Fix the test suite
2020-10-15 20:15:51 +03:00
f0892ce934
Fix compilation problems on Windows
2020-10-15 20:15:51 +03:00
e6320e5881
Address #1584 Don't keep all deposits in memory (persist them to disk)
2020-10-15 20:15:51 +03:00
bc8acdb9de
Switch to an in-house clearScreen to avoid problems with exception tracking
2020-10-15 20:15:51 +03:00
69e498dc00
Address #1689 and #1517 Usability and security improvements in wallet creation
2020-10-15 20:15:51 +03:00
02a3770803
Adds a fuzzing test for the validator client CLI
...
Also removes an obsolete way to add validators through the command line
2020-10-15 20:15:51 +03:00
8ce0fc3a89
Address #1687
...
This addresses the issues by detecting and rejecting keystores with
incorrect PBKDF2 and SCrypt params. It also bumps the version of
nim-json-serialization to include a bugfix for incorrect parsing
of json files featuring comments.
2020-10-15 20:15:51 +03:00
cd949a2b81
Address #1691 More pervasive handling of Ctrl+C
2020-10-15 20:15:51 +03:00
8a6281aad2
Simple cost model for sync requests; Penalize peers perfoming flooding or invalid requests
2020-10-15 20:15:51 +03:00
6b9419e547
fix db growth on attestation processing ( #1860 )
...
It turns out that we often save lots of states in the database that are
the result of empty slot processing only - here, we make sure to only
save a state if a block follows - this fixes several issues:
* empty slot states are not always pruned leading to state database size
explosion
* storing states is (very) slow which slows down processing in general,
so we should only do it when it's likely to be useful
* attestation processing doesn't get stuck on saving random states that
won't appear in the chain history
2020-10-15 14:28:44 +02:00
4e73d10f86
removed 2 buggy RPC API endpoints - there are 2 well functioning ones in validator_api.nim anyway ( #1867 )
2020-10-15 14:26:59 +02:00
da59f45a90
update spec/datatypes spec refs from v0.12.3 to v1.0.0 ( #1866 )
2020-10-14 10:00:27 +00:00
513ba72b9a
add v1.0.0-rc.0 support behind compile-time flag ( #1852 )
...
* add v1.0.0-rc.0 support behind compile-time flag
* keep runtime presets consistent
2020-10-13 17:21:25 +00:00
3ee2dd8da4
p2p-interface spec ref bump (except non-updated places) ( #1862 )
2020-10-12 14:37:14 +00:00
1994ffe5a0
update 130+ spec references from v0.12.3 to v1.0.0-rc1 ( #1854 )
2020-10-12 08:59:24 +00:00
00a8a68671
Address #1695
...
Better error messages when the beacon node is asked to listen on a
reserved port (0) or an already taken one.
2020-10-09 16:39:03 +03:00
d69d89866e
fix logtrace in local network sim ( #1840 )
2020-10-09 10:58:54 +02:00
b79e5f8af5
update nim-beacon-chain to nimbus-eth2 in beacon_chain/, ncli/, tests/, and README.md ( #1843 )
2020-10-08 19:02:05 +00:00
c0aa2f1e58
update process_slashings() to use PROPORTIONAL_SLASHING_MULTIPLIER ( #1841 )
2020-10-08 18:00:07 +00:00
3bd7ab4c20
Do not reward empty responses. ( #1827 )
...
Request status from peers every StatusExpirationTime.
2020-10-08 14:50:48 +02:00
f08f44b9a2
in exit pool, bundle receive messages into beaconblocks ( #1812 )
...
* in exit pool, filter out already-packaged messages; bundle remaining messages into beaconblocks
* filter messages at block construction time
* allow adding up to intended capacity of buffers, beyond per-block limits
* document rationale/design for filtering mechanism
2020-10-07 16:57:21 +00:00
b46c8410a6
Retire Spadina and prepare for the Zinken network
2020-10-07 18:12:21 +03:00
6b9090398e
[JSON-RPC] Prevent silent uint conversion ( #1819 )
...
* Prevent silent uint conversion
* bump json-rpc
2020-10-07 15:02:54 +02:00
deaddc1fc0
Address review comments
2020-10-07 09:32:03 +03:00
aed291128a
Add support for starting from weak subjectivity checkpoints
...
Also removes the `genesis.ssz` file stored in the data folder.
The `medalla-fast-sync` target has been adapted to use the new features.
2020-10-07 09:32:03 +03:00
2c06492407
Address #1728
2020-10-06 15:46:22 +03:00
950bcd0f6d
Address #1729 : Min password length
2020-10-06 15:46:22 +03:00
78953fd9b5
Address #1729 : NFKD Normalization
2020-10-06 15:46:22 +03:00
fec4b5014d
Address #1694
2020-10-06 15:46:22 +03:00
47d4899911
Wallet creation does not allow overwriting an existing wallet
2020-10-06 15:46:22 +03:00
9d23094c68
Address #1692
2020-10-06 15:46:22 +03:00
27fc8bcc80
Address #1538 and do not penalize peers for empty response. ( #1811 )
2020-10-06 14:10:02 +02:00
6bb6b66766
Partial LTO ( #1789 )
...
* initial partial LTO example
* Add select private key related files to ignore LTO
* Apply to keystores
2020-10-06 13:38:18 +02:00
c92d228ec8
Fix non-split attesting protection ( #1804 )
...
* address #1679 - att protection in non-split mode
* AttachedValidator public_key -> pubkey
* Always build with slashing protection
* Remove now unnecessary forwarding impl proc
2020-10-06 10:51:33 +02:00
5354a0a3d5
fix #1585 and restore storeLEBytes on big-endian systems ( #1796 )
...
* fix #1585 and restore storeLEBytes on big-endian systems
* switch to using stew/endians2
* create static endians2-like shim in bitseqs
2020-10-06 08:00:11 +00:00
8c59b92081
fixed #1663 - Interger overflow in compute_start_slot_at_epoch through RPC ( #1810 )
...
* fixed #1663 - Interger overflow in compute_start_slot_at_epoch through RPC
* changed the way the overflow check is done - took the approach from PR #1797 - see the comment in PR #1810 for more details
2020-10-05 21:58:30 +00:00
00a500ef3c
Fix rebase issues.
2020-10-05 22:19:50 +03:00
f091c8d4df
Add chronicles.formatIt for PublicKey.
...
Fix logs.
Rename checkFilePermissions to checkSensitiveFilePermissions.
2020-10-05 22:19:50 +03:00
dd8b73c85c
Remove `keypath` and set version to 1 for network keystorage.
2020-10-05 22:19:50 +03:00
9441e912cb
Merge different places of keyboard input to keyboardCreatePassword() and keyboardGetPassword().
...
Fix Windows problem with english words array.
Add checksum for english words array verification.
2020-10-05 22:19:50 +03:00
868ce14762
Fix compilation issue.
2020-10-05 22:19:50 +03:00
82228fe471
Fix last createDir.
2020-10-05 22:19:50 +03:00
86139839f1
Add permissions checks and handling to wallets and bls keystores.
2020-10-05 22:19:50 +03:00
fc09c2b3d4
Fix compilation problem.
2020-10-05 22:19:50 +03:00
1d76683a8b
Fix os dependency.
2020-10-05 22:19:50 +03:00
20f695515a
Fix localtestnet simulation.
2020-10-05 22:19:50 +03:00
add22a20e1
Update local_testnet and simulation scripts to use netkey-file and insecure-netkey-password.
...
Add more logging
2020-10-05 22:19:50 +03:00
e1182f8000
Add insecure password for automated testing.
...
Fix checkDataDir to run before setupLogging.
2020-10-05 22:19:50 +03:00
e43af4e978
Fix CLI option.
2020-10-05 22:19:50 +03:00
40f2b74f73
Add keystore management and interactive password handling.
2020-10-05 22:19:50 +03:00
ae72c08573
Fix utils problem.
2020-10-05 22:19:50 +03:00
tersec
cheatfate
Jacek Sieka
Zahary Karadjov
kdeme
Ștefan Talpalaru
Sacha Saint-Leger
Giovanni Petrantoni
Mamy Ratsimbazafy
Viktor Kirilov