Commit Graph

90 Commits

Author SHA1 Message Date
Zahary Karadjov 75c1c6a95c Enable Snappy by default (using LibP2P steams for now)
This refactors the newly added Snappy streaming back-ends trying to
make them more similar and to reduce the code duplication to a minimum.
2020-05-13 12:18:42 +03:00
Eugene Kabanov da0b1a4993
Fix status handling. (#1008)
* Fix status handling.
Add log map of received blocks.

* Fix review comments.
Fix UnusedImport in sync_protocol.nim
2020-05-13 08:37:58 +02:00
tersec ba1d7e2ed4
switch state cache to use ref statedata objects to limit memory usage (#1007)
* switch state cache to use ref statedata objects to limit memory usage

* more directly initialize ref StateData

* use HashedBeaconState instead of StateData to try to fix memory leak

* switch cache to seq[ref HashedBeaconState]

* remove unused import

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2020-05-12 16:26:58 +00:00
Jacek Sieka fb2e0ddbec
sync fixes (#1005)
* sync fixes

* fix Status message finalized info
* work around sync starting before initial status exchange
* don't fail block on deposit signature check failure (fixes #989)
* print ForkDigest and Version nicely
* dump incoming blocks
* fix crash when libp2p peer connection is closed
* update chunk size to 16 to work around missing blocks when syncing

* bump libp2p

* bump libp2p

* better deposit skip message
2020-05-11 18:08:52 +00:00
Jacek Sieka 65ca74c980 req: cap requested blocks better
also cap blocks in roots request
2020-04-22 12:09:26 +03:00
Jacek Sieka da988e4e40 more block range request updates
* handle skip == 0 gracefully
* avoid memory allocation at expense of more complex API
* add more tests
* log block request results
2020-04-21 11:07:27 +03:00
Jacek Sieka fccce85f0d simplify block request
this matches the intended spec behaviour
2020-04-21 11:07:27 +03:00
Eugene Kabanov 3d42da90a8
Syncing. (#909) 2020-04-20 16:59:18 +02:00
Zahary Karadjov 672f690bf6 Implement most of the v0.11.1 networking changes 2020-04-16 01:01:18 +03:00
Zahary Karadjov 0c018cb68a Mechanically remove all mentions of the daemon from the code 2020-03-23 14:59:33 +02:00
Joao Gabriel Carvalho 91f87b55b4
SignedBeaconBlock in state_transition (#773)
* using SignedBeaconBlock in state_transition
2020-02-29 16:15:44 +01:00
Zahary Karadjov 9e58e22761 [skip ci] remove an obsolete comment 2020-02-20 10:58:38 +02:00
cheatfate 5742299c94 Fix freezes in connection initialization. 2020-02-20 10:58:38 +02:00
Zahary Karadjov ee7c2c9dff Unify the bootstrap nodes handling code
We no longer discriminate between ENR, MultiAddress or ENode
bootstrap records (all of them are remapped to ENodes).

The discovery loop will stochastically try to reconnect to
accidentally disconnected nodes.
2020-02-18 12:53:49 +02:00
Jacek Sieka 270c25c8b8
drop rlpx support (#679) 2020-01-24 09:32:52 +01:00
Dustin Brody c824416f56 initial 0.9.3 spec update 2019-12-17 00:06:03 +02:00
Zahary Karadjov 10e6d4842f Local sim finalizes with native LibP2P 2019-12-10 15:02:16 +02:00
Zahary Karadjov 889031453f The beacon node can be compiled with the LibP2P native back-end
This hasn't been tested yet even in local sim.
2019-12-10 15:02:16 +02:00
Ștefan Talpalaru 7682a010a9 remove getCurrentExceptionMsg() 2019-12-02 22:06:42 +02:00
Zahary Karadjov 8b89bbe391 Fix off-by-one error in syncing 2019-11-26 02:45:25 +02:00
Zahary Karadjov 8de4d38e4e More tracing 2019-11-26 02:45:25 +02:00
Jacek Sieka fd4de5de0f
start untangling beaconnode (#589)
* Move BeaconNode type to its own file (fewer imports)
* disentangle sync protocol/request manager
* fix some old nimisms
* de-fear some logs
* simplify eth1 data production
* add stack tracing to release builds
* drop release compile flag for testnet
2019-11-25 15:36:25 +01:00
Zahary Karadjov 5a93e50b5e Sync-related fixes:
* off-by-one error in the returned range of blocks
* larger request time-outs to deal with non-responsive servers
* fix an unhandled exception when we fail to deliver a response chunk
2019-11-12 23:43:38 +00:00
Zahary Karadjov f5e9b9a922 Allow running local nodes without attached validators
The number of user nodes is now specified with `USER_NODES`.

To make the instructions more stable, the "numeric id" of the user
nodes will be starting from 0 (so you can always use `run_node.sh 0`
to start a user node).

If you specify a node index above the total number of nodes, you'll
launch a node without any validators attached (this is useful for
testing the sync for example).
2019-11-11 23:29:36 +00:00
Yuriy Glukhov 9b39c792d0 Revert f79f9deeb3 a0fbf5464a 94a9c51d8a (#523) 2019-11-06 16:56:54 +02:00
cheatfate f79f9deeb3
Set wasDialed.
Add more logging information.
2019-11-06 16:41:24 +02:00
Ștefan Talpalaru a417edb5ff
sync_protocol.nim: better error message [skip ci] 2019-11-05 16:02:26 +01:00
Dustin Brody 63e621c27d
initial 0.9.0 spec sync (#509)
* rename compute_epoch_of_slot(...) to compute_epoch_at_slot(...)

* remove some unnecessary imports; remove some crosslink-related code and tests; complete renaming of compute_epoch_of_slot(...) to compute_epoch_at_slot(...)

* rm more transfer-related code and tests; rm more unnecessary strutils imports

* rm remaining unused imports

* remove useless get_empty_per_epoch_cache(...)/compute_start_slot_of_epoch(...) calls

* rename compute_start_slot_of_epoch(...) to compute_start_slot_at_epoch(...)

* rename ACTIVATION_EXIT_DELAY to MAX_SEED_LOOKAHEAD

* update domain types to 0.9.0

* mark AttesterSlashing, IndexedAttestation, AttestationDataAndCustodyBit, DepositData, BeaconBlockHeader, Fork, integer_squareroot(...), and process_voluntary_exit(...) as 0.9.0

* mark increase_balance(...), decrease_balance(...), get_block_root(...), CheckPoint, Deposit, PendingAttestation, HistoricalBatch, is_active_validator(...), and is_slashable_attestation_data(...) as 0.9.0

* mark compute_activation_exit_epoch(...), bls_verify(...), Validator, get_active_validator_indices(...), get_current_epoch(...), get_total_active_balance(...), and get_previous_epoch(...) as 0.9.0

* mark get_block_root_at_slot(...), ProposerSlashing, get_domain(...), VoluntaryExit, mainnet preset Gwei values, minimal preset max operations, process_block_header(...), and is_slashable_validator(...) as 0.9.0

* mark makeWithdrawalCredentials(...), get_validator_churn_limit(...), get_total_balance(...), is_valid_indexed_attestation(...), bls_aggregate_pubkeys(...), initial genesis value/constants, Attestation, get_randao_mix(...), mainnet preset max operations per block constants, minimal preset Gwei values and time parameters, process_eth1_data(...), get_shuffled_seq(...), compute_committee(...), and process_slots(...) as 0.9.0; partially update get_indexed_attestation(...) to 0.9.0 by removing crosslink refs and associated tests

* mark initiate_validator_exit(...), process_registry_updates(...), BeaconBlock, Eth1Data, compute_domain(...), process_randao(...), process_attester_slashing(...), get_base_reward(...), and process_slot(...) as 0.9.0
2019-10-30 19:41:19 +00:00
Jacek Sieka ee5d1c1467
minimize imports, specially for spec - cuts 2/3 of ncli build time (#500)
* minimize imports, specially for spec - cuts 2/3 of ncli build time
* ptr_arith->ptrops
2019-10-25 12:59:56 +02:00
Zahary Karadjov f72a58595f Initial version of status bar 2019-10-07 23:56:34 +09:00
Zahary Karadjov 1555efd9d8
Implement the latest modification of the spec 2019-09-27 19:10:37 +03:00
Zahary Karadjov cdff79ec6d More sync fixes
* Fix IncompleteData issues brought by the new spec-compliant stream closing
* Fix logic errors in the sync algorithms
2019-09-10 09:49:54 -04:00
Zahary Karadjov 886b92319f Implement the response size limits 2019-09-10 09:49:54 -04:00
Zahary Karadjov a83aa83644 Working BeaconSync
Changes:

* Do not send separate network packets for response codes and msg
  len prefixes

* Close streams according to the spec

* Implement more timeouts according to the spec

* Make hello requests during syncing to update our knowledge of
  the head block of the other peer.
2019-09-10 09:49:54 -04:00
Zahary Karadjov 4a54fb4103 Cleaned up obsolete BeaconSync code; Added some open questions regarding fetchAncestorBlocks 2019-09-10 09:49:54 -04:00
Zahary Karadjov 9dec05f9c9 Sending chunked responses 2019-09-10 09:49:54 -04:00
Zahary Karadjov 2bbfa8c877 Changes related to the new ETH interop spec
* Hello is no longer a handshake message
  (all handshakes related code was deleted for clarity)

* Deal with the single-parameter inlining defined in the new spec
2019-09-10 09:49:54 -04:00
Jacek Sieka 5f58af2112
fix ssz list, protocol names (#405) 2019-09-08 16:54:31 +02:00
Eugene Kabanov bb0e9de09d [WIP] Sync which using latest network specification. (#390)
* Fixed getBeaconBlocks() and getRecentBeaconBlocks() to use BlockPool, not db.

* Got the sync_protocol to compiling state; Removed all obsolete RPC calls
2019-09-08 15:08:44 +02:00
zah 93cdb43f1e Integrate nim-metrics and add some metrics from the ETH2 spec (#394) 2019-09-07 19:48:05 +02:00
Zahary Karadjov f5db2dd7b3
Fix the build with the latest Chronos 2019-08-15 20:43:06 +02:00
Zahary Karadjov 2fca9465e7
Fix the build with RLPx 2019-08-15 20:43:05 +02:00
Zahary Karadjov 5ce50b3aca
Implement the latest networking spec
https://github.com/ethereum/eth2.0-specs/pull/1328
2019-08-07 05:22:28 +03:00
Zahary Karadjov 398ea55801
Implement the latest SSZ specification and integrate the official SSZ test suite 2019-08-05 03:18:48 +03:00
Dustin Brody 0a2d09e7fe
Speed up process_crosslinks(...) and get_crosslink_deltas(...) by 10x - 15x in state_sim (#314)
* remove some low-hanging perf silliness from get_winning_crosslink_and_attesting_indices(...) and switch inner/outer loop nesting order to get 10-15x function speedup for 128 and 512 validator cases by avoiding accidentally quadratic behavior, while keeping function signature unchanged and allowing easy ongoing verification of correctness of optimization

* avoid memory allocation with an openarray rather than seq

* update get_crosslink_deltas(...) to 0.8.0

* move non-spec ValidatorSetDeltaFlags from datatypes.nim to sync_protocol; fix mainnet MIN_ATTESTATION_INCLUSION_DELAY preset; update get_attestation_deltas(...) to 0.8.0; for 512 validators, get 90x speedup for get_attestation_deltas(...) from 179s to 2s

* explain purpose of when false portions, so they can be removed once accomplished
2019-07-08 13:19:52 +00:00
Jacek Sieka b9e62a1f6c
std_shims -> stew 2019-07-07 11:53:58 +02:00
Dustin Brody e4321dc4ed Initial frozen phase 0/v0.8.0 spec (#305)
* rename slot_to_epoch(...) to compute_epoch_of_slot(...)

* rename aggregation_bitfield/custody_bitfield fields to aggregation_bits/custody_bits; rename convert_to_indexed(...) to get_indexed_attestation(...); mark BeaconBlockHeader as 0.8.0; update minimal preset MIN_ATTESTATION_INCLUSION_DELAY/time parameters in general to 0.8

* fix beacon node compilation; it used slightly different capitalizations of slot_to_epoch/slotToEpoch

* mark integer_squareroot(...), Deposit, VoluntaryExit, and Transfer as 0.8.0; rename get_epoch_start_slot(...) to compute_start_slot_of_epoch(...)

* add new Domain alias for uint64; rename bls_domain(...) to compute_domain(...), MAX_INDICES_PER_ATTESTATION to MAX_VALIDATORS_PER_COMMITTEE, and SignatureDomain to DomainType; update get_domain(...) to 0.8.0

* update/mark initiate_validator_exit(...) and Crosslink as 0.8.0; rename BeaconState.latest_block_roots -> BeaconState.block_roots; mark HistoricalBatch as 0.8.0
2019-07-01 09:53:42 +02:00
Zahary Karadjov c751285112
Implement all libp2p_native response codes as specified in the latest proposal 2019-06-24 05:34:22 +03:00
Zahary Karadjov e228c2dbcb
Implement the even/odd request ID scheme; Handle more edge cases; Break the cyclic imports 2019-06-24 05:34:22 +03:00
Zahary Karadjov 877b22cfb8
Share more code between the libp2p backends 2019-06-24 05:34:21 +03:00