98 Commits

Author SHA1 Message Date
cheatfate
6052f36fcd Fix #1241. 2020-07-16 13:01:18 +03:00
zah
0be77f9cbc
Restore compilation with custom presets (#1309)
This also assigns precise types to the constants in the minimal
and mainnet presets in order to reduce the chance of compilation
errors when custom presets are used (previously, only the custom
presets have precisely assigned types for the constants).
2020-07-13 16:44:58 +02:00
Jacek Sieka
1301600341
Trusted blocks (#1227)
* cleanups

* fix ncli state root check flag
* add block dump to ncli_db
* limit ncli_db benchmark length
* tone down finalization logs

* introduce trusted blocks

We only store blocks whose signature we've verified in the database - as
such, there's no need to check it again, and most importantly, no need
to deserialize the signature when loading from database.

50x startup time improvement, 200x block load time improvement.

* fix rewinding when deposits have invalid signature
* speed up ancestor iteration by avoiding copy
* avoid deserializing signatures for trusted data
* load blocks lazily when rewinding (less memory used)

* chronicles workarounds

* document trustedbeaconblock
2020-06-25 12:23:10 +02:00
Jacek Sieka
7e0e4dc327
don't crash on unknown disconnection reason, fix disconnection reason enum (#1208) 2020-06-20 09:24:33 +02:00
Jacek Sieka
016cc22173
show peer info on connect (#1155) 2020-06-11 07:14:26 +02:00
Jacek Sieka
56ffb696be
reorder ssz (#1099)
* reorder ssz

* split into hash_trees and ssz_serialization, roughly, for hashing and
IO
* move bitseqs into ssz (from stew)
* clean up imports

* docs, imports
2020-06-03 15:52:02 +02:00
Zahary Karadjov
28128f4d2f Add a handler for the Goodbye message
The lack of body of `goodbye` in sync_protocol.nim was preventing
the respective LibP2P protocol to be mounted and advertised on the
network.

Adding a body fixes that, but I've also made some changes in the
P2P protocol codegen that will prevent the issue from happening
again (no body is now considered the equivalent of having an empty
body).
2020-05-26 22:17:26 +03:00
Zahary Karadjov
833f19e942 Reform the networking layer in order to handle the new stricter SSZ API 2020-05-24 19:00:34 +03:00
Zahary Karadjov
75c1c6a95c Enable Snappy by default (using LibP2P steams for now)
This refactors the newly added Snappy streaming back-ends trying to
make them more similar and to reduce the code duplication to a minimum.
2020-05-13 12:18:42 +03:00
Eugene Kabanov
da0b1a4993
Fix status handling. (#1008)
* Fix status handling.
Add log map of received blocks.

* Fix review comments.
Fix UnusedImport in sync_protocol.nim
2020-05-13 08:37:58 +02:00
tersec
ba1d7e2ed4
switch state cache to use ref statedata objects to limit memory usage (#1007)
* switch state cache to use ref statedata objects to limit memory usage

* more directly initialize ref StateData

* use HashedBeaconState instead of StateData to try to fix memory leak

* switch cache to seq[ref HashedBeaconState]

* remove unused import

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2020-05-12 16:26:58 +00:00
Jacek Sieka
fb2e0ddbec
sync fixes (#1005)
* sync fixes

* fix Status message finalized info
* work around sync starting before initial status exchange
* don't fail block on deposit signature check failure (fixes #989)
* print ForkDigest and Version nicely
* dump incoming blocks
* fix crash when libp2p peer connection is closed
* update chunk size to 16 to work around missing blocks when syncing

* bump libp2p

* bump libp2p

* better deposit skip message
2020-05-11 18:08:52 +00:00
Jacek Sieka
65ca74c980 req: cap requested blocks better
also cap blocks in roots request
2020-04-22 12:09:26 +03:00
Jacek Sieka
da988e4e40 more block range request updates
* handle skip == 0 gracefully
* avoid memory allocation at expense of more complex API
* add more tests
* log block request results
2020-04-21 11:07:27 +03:00
Jacek Sieka
fccce85f0d simplify block request
this matches the intended spec behaviour
2020-04-21 11:07:27 +03:00
Eugene Kabanov
3d42da90a8
Syncing. (#909) 2020-04-20 16:59:18 +02:00
Zahary Karadjov
672f690bf6 Implement most of the v0.11.1 networking changes 2020-04-16 01:01:18 +03:00
Zahary Karadjov
0c018cb68a Mechanically remove all mentions of the daemon from the code 2020-03-23 14:59:33 +02:00
Joao Gabriel Carvalho
91f87b55b4
SignedBeaconBlock in state_transition (#773)
* using SignedBeaconBlock in state_transition
2020-02-29 16:15:44 +01:00
Zahary Karadjov
9e58e22761 [skip ci] remove an obsolete comment 2020-02-20 10:58:38 +02:00
cheatfate
5742299c94 Fix freezes in connection initialization. 2020-02-20 10:58:38 +02:00
Zahary Karadjov
ee7c2c9dff Unify the bootstrap nodes handling code
We no longer discriminate between ENR, MultiAddress or ENode
bootstrap records (all of them are remapped to ENodes).

The discovery loop will stochastically try to reconnect to
accidentally disconnected nodes.
2020-02-18 12:53:49 +02:00
Jacek Sieka
270c25c8b8
drop rlpx support (#679) 2020-01-24 09:32:52 +01:00
Dustin Brody
c824416f56 initial 0.9.3 spec update 2019-12-17 00:06:03 +02:00
Zahary Karadjov
10e6d4842f Local sim finalizes with native LibP2P 2019-12-10 15:02:16 +02:00
Zahary Karadjov
889031453f The beacon node can be compiled with the LibP2P native back-end
This hasn't been tested yet even in local sim.
2019-12-10 15:02:16 +02:00
Ștefan Talpalaru
7682a010a9 remove getCurrentExceptionMsg() 2019-12-02 22:06:42 +02:00
Zahary Karadjov
8b89bbe391 Fix off-by-one error in syncing 2019-11-26 02:45:25 +02:00
Zahary Karadjov
8de4d38e4e More tracing 2019-11-26 02:45:25 +02:00
Jacek Sieka
fd4de5de0f
start untangling beaconnode (#589)
* Move BeaconNode type to its own file (fewer imports)
* disentangle sync protocol/request manager
* fix some old nimisms
* de-fear some logs
* simplify eth1 data production
* add stack tracing to release builds
* drop release compile flag for testnet
2019-11-25 15:36:25 +01:00
Zahary Karadjov
5a93e50b5e Sync-related fixes:
* off-by-one error in the returned range of blocks
* larger request time-outs to deal with non-responsive servers
* fix an unhandled exception when we fail to deliver a response chunk
2019-11-12 23:43:38 +00:00
Zahary Karadjov
f5e9b9a922 Allow running local nodes without attached validators
The number of user nodes is now specified with `USER_NODES`.

To make the instructions more stable, the "numeric id" of the user
nodes will be starting from 0 (so you can always use `run_node.sh 0`
to start a user node).

If you specify a node index above the total number of nodes, you'll
launch a node without any validators attached (this is useful for
testing the sync for example).
2019-11-11 23:29:36 +00:00
Yuriy Glukhov
9b39c792d0 Revert f79f9deeb370af4d35d59969cec087f994cc8756 a0fbf5464a50430f99d8868c9696ea3f91781f19 94a9c51d8ad646e76bfab74d7fc1da5a29d10db7 (#523) 2019-11-06 16:56:54 +02:00
cheatfate
f79f9deeb3
Set wasDialed.
Add more logging information.
2019-11-06 16:41:24 +02:00
Ștefan Talpalaru
a417edb5ff
sync_protocol.nim: better error message [skip ci] 2019-11-05 16:02:26 +01:00
Dustin Brody
63e621c27d
initial 0.9.0 spec sync (#509)
* rename compute_epoch_of_slot(...) to compute_epoch_at_slot(...)

* remove some unnecessary imports; remove some crosslink-related code and tests; complete renaming of compute_epoch_of_slot(...) to compute_epoch_at_slot(...)

* rm more transfer-related code and tests; rm more unnecessary strutils imports

* rm remaining unused imports

* remove useless get_empty_per_epoch_cache(...)/compute_start_slot_of_epoch(...) calls

* rename compute_start_slot_of_epoch(...) to compute_start_slot_at_epoch(...)

* rename ACTIVATION_EXIT_DELAY to MAX_SEED_LOOKAHEAD

* update domain types to 0.9.0

* mark AttesterSlashing, IndexedAttestation, AttestationDataAndCustodyBit, DepositData, BeaconBlockHeader, Fork, integer_squareroot(...), and process_voluntary_exit(...) as 0.9.0

* mark increase_balance(...), decrease_balance(...), get_block_root(...), CheckPoint, Deposit, PendingAttestation, HistoricalBatch, is_active_validator(...), and is_slashable_attestation_data(...) as 0.9.0

* mark compute_activation_exit_epoch(...), bls_verify(...), Validator, get_active_validator_indices(...), get_current_epoch(...), get_total_active_balance(...), and get_previous_epoch(...) as 0.9.0

* mark get_block_root_at_slot(...), ProposerSlashing, get_domain(...), VoluntaryExit, mainnet preset Gwei values, minimal preset max operations, process_block_header(...), and is_slashable_validator(...) as 0.9.0

* mark makeWithdrawalCredentials(...), get_validator_churn_limit(...), get_total_balance(...), is_valid_indexed_attestation(...), bls_aggregate_pubkeys(...), initial genesis value/constants, Attestation, get_randao_mix(...), mainnet preset max operations per block constants, minimal preset Gwei values and time parameters, process_eth1_data(...), get_shuffled_seq(...), compute_committee(...), and process_slots(...) as 0.9.0; partially update get_indexed_attestation(...) to 0.9.0 by removing crosslink refs and associated tests

* mark initiate_validator_exit(...), process_registry_updates(...), BeaconBlock, Eth1Data, compute_domain(...), process_randao(...), process_attester_slashing(...), get_base_reward(...), and process_slot(...) as 0.9.0
2019-10-30 19:41:19 +00:00
Jacek Sieka
ee5d1c1467
minimize imports, specially for spec - cuts 2/3 of ncli build time (#500)
* minimize imports, specially for spec - cuts 2/3 of ncli build time
* ptr_arith->ptrops
2019-10-25 12:59:56 +02:00
Zahary Karadjov
f72a58595f Initial version of status bar 2019-10-07 23:56:34 +09:00
Zahary Karadjov
1555efd9d8
Implement the latest modification of the spec 2019-09-27 19:10:37 +03:00
Zahary Karadjov
cdff79ec6d More sync fixes
* Fix IncompleteData issues brought by the new spec-compliant stream closing
* Fix logic errors in the sync algorithms
2019-09-10 09:49:54 -04:00
Zahary Karadjov
886b92319f Implement the response size limits 2019-09-10 09:49:54 -04:00
Zahary Karadjov
a83aa83644 Working BeaconSync
Changes:

* Do not send separate network packets for response codes and msg
  len prefixes

* Close streams according to the spec

* Implement more timeouts according to the spec

* Make hello requests during syncing to update our knowledge of
  the head block of the other peer.
2019-09-10 09:49:54 -04:00
Zahary Karadjov
4a54fb4103 Cleaned up obsolete BeaconSync code; Added some open questions regarding fetchAncestorBlocks 2019-09-10 09:49:54 -04:00
Zahary Karadjov
9dec05f9c9 Sending chunked responses 2019-09-10 09:49:54 -04:00
Zahary Karadjov
2bbfa8c877 Changes related to the new ETH interop spec
* Hello is no longer a handshake message
  (all handshakes related code was deleted for clarity)

* Deal with the single-parameter inlining defined in the new spec
2019-09-10 09:49:54 -04:00
Jacek Sieka
5f58af2112
fix ssz list, protocol names (#405) 2019-09-08 16:54:31 +02:00
Eugene Kabanov
bb0e9de09d [WIP] Sync which using latest network specification. (#390)
* Fixed getBeaconBlocks() and getRecentBeaconBlocks() to use BlockPool, not db.

* Got the sync_protocol to compiling state; Removed all obsolete RPC calls
2019-09-08 15:08:44 +02:00
zah
93cdb43f1e Integrate nim-metrics and add some metrics from the ETH2 spec (#394) 2019-09-07 19:48:05 +02:00
Zahary Karadjov
f5db2dd7b3
Fix the build with the latest Chronos 2019-08-15 20:43:06 +02:00
Zahary Karadjov
2fca9465e7
Fix the build with RLPx 2019-08-15 20:43:05 +02:00