77 Commits

Author SHA1 Message Date
Zahary Karadjov
ee7c2c9dff Unify the bootstrap nodes handling code
We no longer discriminate between ENR, MultiAddress or ENode
bootstrap records (all of them are remapped to ENodes).

The discovery loop will stochastically try to reconnect to
accidentally disconnected nodes.
2020-02-18 12:53:49 +02:00
Jacek Sieka
270c25c8b8
drop rlpx support (#679) 2020-01-24 09:32:52 +01:00
Dustin Brody
c824416f56 initial 0.9.3 spec update 2019-12-17 00:06:03 +02:00
Zahary Karadjov
10e6d4842f Local sim finalizes with native LibP2P 2019-12-10 15:02:16 +02:00
Zahary Karadjov
889031453f The beacon node can be compiled with the LibP2P native back-end
This hasn't been tested yet even in local sim.
2019-12-10 15:02:16 +02:00
Ștefan Talpalaru
7682a010a9 remove getCurrentExceptionMsg() 2019-12-02 22:06:42 +02:00
Zahary Karadjov
8b89bbe391 Fix off-by-one error in syncing 2019-11-26 02:45:25 +02:00
Zahary Karadjov
8de4d38e4e More tracing 2019-11-26 02:45:25 +02:00
Jacek Sieka
fd4de5de0f
start untangling beaconnode (#589)
* Move BeaconNode type to its own file (fewer imports)
* disentangle sync protocol/request manager
* fix some old nimisms
* de-fear some logs
* simplify eth1 data production
* add stack tracing to release builds
* drop release compile flag for testnet
2019-11-25 15:36:25 +01:00
Zahary Karadjov
5a93e50b5e Sync-related fixes:
* off-by-one error in the returned range of blocks
* larger request time-outs to deal with non-responsive servers
* fix an unhandled exception when we fail to deliver a response chunk
2019-11-12 23:43:38 +00:00
Zahary Karadjov
f5e9b9a922 Allow running local nodes without attached validators
The number of user nodes is now specified with `USER_NODES`.

To make the instructions more stable, the "numeric id" of the user
nodes will be starting from 0 (so you can always use `run_node.sh 0`
to start a user node).

If you specify a node index above the total number of nodes, you'll
launch a node without any validators attached (this is useful for
testing the sync for example).
2019-11-11 23:29:36 +00:00
Yuriy Glukhov
9b39c792d0 Revert f79f9deeb370af4d35d59969cec087f994cc8756 a0fbf5464a50430f99d8868c9696ea3f91781f19 94a9c51d8ad646e76bfab74d7fc1da5a29d10db7 (#523) 2019-11-06 16:56:54 +02:00
cheatfate
f79f9deeb3
Set wasDialed.
Add more logging information.
2019-11-06 16:41:24 +02:00
Ștefan Talpalaru
a417edb5ff
sync_protocol.nim: better error message [skip ci] 2019-11-05 16:02:26 +01:00
Dustin Brody
63e621c27d
initial 0.9.0 spec sync (#509)
* rename compute_epoch_of_slot(...) to compute_epoch_at_slot(...)

* remove some unnecessary imports; remove some crosslink-related code and tests; complete renaming of compute_epoch_of_slot(...) to compute_epoch_at_slot(...)

* rm more transfer-related code and tests; rm more unnecessary strutils imports

* rm remaining unused imports

* remove useless get_empty_per_epoch_cache(...)/compute_start_slot_of_epoch(...) calls

* rename compute_start_slot_of_epoch(...) to compute_start_slot_at_epoch(...)

* rename ACTIVATION_EXIT_DELAY to MAX_SEED_LOOKAHEAD

* update domain types to 0.9.0

* mark AttesterSlashing, IndexedAttestation, AttestationDataAndCustodyBit, DepositData, BeaconBlockHeader, Fork, integer_squareroot(...), and process_voluntary_exit(...) as 0.9.0

* mark increase_balance(...), decrease_balance(...), get_block_root(...), CheckPoint, Deposit, PendingAttestation, HistoricalBatch, is_active_validator(...), and is_slashable_attestation_data(...) as 0.9.0

* mark compute_activation_exit_epoch(...), bls_verify(...), Validator, get_active_validator_indices(...), get_current_epoch(...), get_total_active_balance(...), and get_previous_epoch(...) as 0.9.0

* mark get_block_root_at_slot(...), ProposerSlashing, get_domain(...), VoluntaryExit, mainnet preset Gwei values, minimal preset max operations, process_block_header(...), and is_slashable_validator(...) as 0.9.0

* mark makeWithdrawalCredentials(...), get_validator_churn_limit(...), get_total_balance(...), is_valid_indexed_attestation(...), bls_aggregate_pubkeys(...), initial genesis value/constants, Attestation, get_randao_mix(...), mainnet preset max operations per block constants, minimal preset Gwei values and time parameters, process_eth1_data(...), get_shuffled_seq(...), compute_committee(...), and process_slots(...) as 0.9.0; partially update get_indexed_attestation(...) to 0.9.0 by removing crosslink refs and associated tests

* mark initiate_validator_exit(...), process_registry_updates(...), BeaconBlock, Eth1Data, compute_domain(...), process_randao(...), process_attester_slashing(...), get_base_reward(...), and process_slot(...) as 0.9.0
2019-10-30 19:41:19 +00:00
Jacek Sieka
ee5d1c1467
minimize imports, specially for spec - cuts 2/3 of ncli build time (#500)
* minimize imports, specially for spec - cuts 2/3 of ncli build time
* ptr_arith->ptrops
2019-10-25 12:59:56 +02:00
Zahary Karadjov
f72a58595f Initial version of status bar 2019-10-07 23:56:34 +09:00
Zahary Karadjov
1555efd9d8
Implement the latest modification of the spec 2019-09-27 19:10:37 +03:00
Zahary Karadjov
cdff79ec6d More sync fixes
* Fix IncompleteData issues brought by the new spec-compliant stream closing
* Fix logic errors in the sync algorithms
2019-09-10 09:49:54 -04:00
Zahary Karadjov
886b92319f Implement the response size limits 2019-09-10 09:49:54 -04:00
Zahary Karadjov
a83aa83644 Working BeaconSync
Changes:

* Do not send separate network packets for response codes and msg
  len prefixes

* Close streams according to the spec

* Implement more timeouts according to the spec

* Make hello requests during syncing to update our knowledge of
  the head block of the other peer.
2019-09-10 09:49:54 -04:00
Zahary Karadjov
4a54fb4103 Cleaned up obsolete BeaconSync code; Added some open questions regarding fetchAncestorBlocks 2019-09-10 09:49:54 -04:00
Zahary Karadjov
9dec05f9c9 Sending chunked responses 2019-09-10 09:49:54 -04:00
Zahary Karadjov
2bbfa8c877 Changes related to the new ETH interop spec
* Hello is no longer a handshake message
  (all handshakes related code was deleted for clarity)

* Deal with the single-parameter inlining defined in the new spec
2019-09-10 09:49:54 -04:00
Jacek Sieka
5f58af2112
fix ssz list, protocol names (#405) 2019-09-08 16:54:31 +02:00
Eugene Kabanov
bb0e9de09d [WIP] Sync which using latest network specification. (#390)
* Fixed getBeaconBlocks() and getRecentBeaconBlocks() to use BlockPool, not db.

* Got the sync_protocol to compiling state; Removed all obsolete RPC calls
2019-09-08 15:08:44 +02:00
zah
93cdb43f1e Integrate nim-metrics and add some metrics from the ETH2 spec (#394) 2019-09-07 19:48:05 +02:00
Zahary Karadjov
f5db2dd7b3
Fix the build with the latest Chronos 2019-08-15 20:43:06 +02:00
Zahary Karadjov
2fca9465e7
Fix the build with RLPx 2019-08-15 20:43:05 +02:00
Zahary Karadjov
5ce50b3aca
Implement the latest networking spec
https://github.com/ethereum/eth2.0-specs/pull/1328
2019-08-07 05:22:28 +03:00
Zahary Karadjov
398ea55801
Implement the latest SSZ specification and integrate the official SSZ test suite 2019-08-05 03:18:48 +03:00
Dustin Brody
0a2d09e7fe
Speed up process_crosslinks(...) and get_crosslink_deltas(...) by 10x - 15x in state_sim (#314)
* remove some low-hanging perf silliness from get_winning_crosslink_and_attesting_indices(...) and switch inner/outer loop nesting order to get 10-15x function speedup for 128 and 512 validator cases by avoiding accidentally quadratic behavior, while keeping function signature unchanged and allowing easy ongoing verification of correctness of optimization

* avoid memory allocation with an openarray rather than seq

* update get_crosslink_deltas(...) to 0.8.0

* move non-spec ValidatorSetDeltaFlags from datatypes.nim to sync_protocol; fix mainnet MIN_ATTESTATION_INCLUSION_DELAY preset; update get_attestation_deltas(...) to 0.8.0; for 512 validators, get 90x speedup for get_attestation_deltas(...) from 179s to 2s

* explain purpose of when false portions, so they can be removed once accomplished
2019-07-08 13:19:52 +00:00
Jacek Sieka
b9e62a1f6c
std_shims -> stew 2019-07-07 11:53:58 +02:00
Dustin Brody
e4321dc4ed Initial frozen phase 0/v0.8.0 spec (#305)
* rename slot_to_epoch(...) to compute_epoch_of_slot(...)

* rename aggregation_bitfield/custody_bitfield fields to aggregation_bits/custody_bits; rename convert_to_indexed(...) to get_indexed_attestation(...); mark BeaconBlockHeader as 0.8.0; update minimal preset MIN_ATTESTATION_INCLUSION_DELAY/time parameters in general to 0.8

* fix beacon node compilation; it used slightly different capitalizations of slot_to_epoch/slotToEpoch

* mark integer_squareroot(...), Deposit, VoluntaryExit, and Transfer as 0.8.0; rename get_epoch_start_slot(...) to compute_start_slot_of_epoch(...)

* add new Domain alias for uint64; rename bls_domain(...) to compute_domain(...), MAX_INDICES_PER_ATTESTATION to MAX_VALIDATORS_PER_COMMITTEE, and SignatureDomain to DomainType; update get_domain(...) to 0.8.0

* update/mark initiate_validator_exit(...) and Crosslink as 0.8.0; rename BeaconState.latest_block_roots -> BeaconState.block_roots; mark HistoricalBatch as 0.8.0
2019-07-01 09:53:42 +02:00
Zahary Karadjov
c751285112
Implement all libp2p_native response codes as specified in the latest proposal 2019-06-24 05:34:22 +03:00
Zahary Karadjov
e228c2dbcb
Implement the even/odd request ID scheme; Handle more edge cases; Break the cyclic imports 2019-06-24 05:34:22 +03:00
Zahary Karadjov
877b22cfb8
Share more code between the libp2p backends 2019-06-24 05:34:21 +03:00
Zahary Karadjov
87601a5eae
Share more code between the libp2p backends 2019-06-24 05:34:21 +03:00
Zahary Karadjov
f4a96bc3f3
[WIP] Restore the Lib2P2 builds and implement the latest wire spec
Depends on https://github.com/status-im/nim-eth/pull/54
2019-06-24 05:34:21 +03:00
Dustin Brody
e9180c490e
more 0.7.0 updates (#287)
* update Deposit and BeaconBlockHeader to 0.7.0

* update get_total_balance(...) and get_base_reward(...) to 0.7.0

* rename initial value constant BLS_WITHDRAWAL_PREFIX_BYTE to BLS_WITHDRAWAL_PREFIX

* mark minimal presets as entirely migrated to 0.7.0; update process_rewards_and_penalties(...) to 0.7.0

* specify that targeting 0.7.0; adopt 0.7.0 refactoring of bls_domain/get_domain to update get_domain(...) to 0.7.0; mark that mainnet constant presets and block header processing are 0.7.0-consistent; update process_justification_and_finalization to 0.7.0 to fix potential underflow errors

* address mratsim's comment to copy whole spec comment
2019-06-14 16:21:04 +00:00
Dustin Brody
2a6e64d3f9
0.7.0 updates including semantic changes (#285)
* update get_previous_epoch(...)

* update BeaconBlock to 0.7.0 by renaming previous_block_root field to parent_root
2019-06-14 13:50:47 +00:00
Yuriy Glukhov
10c7920b27 Proto changes to facilitate backward sync (#271)
* Proto changes to facilitate backward sync

* Update to latest spec types in sync proto

* Use blockpool for more straightforward block headers collection

* Added BlockPool.getRef

* Update beacon_chain/sync_protocol.nim

Co-Authored-By: Jacek Sieka <arnetheduck@gmail.com>
2019-06-10 07:13:53 -04:00
Jacek Sieka
bd4893d2e8 head block selection fixes (#259)
* track justified and head blocks correctly in presence of blank slots
* fix new block state root
* keep list of viable heads instead of walking children
* disable json bitfield deserializer (needs a corresponding serializer)
* fix handshake best & finalized root
2019-05-01 12:19:29 +03:00
Yuriy Glukhov
ddcdb63430
Deduplicate blocks in getAncestorBlocks (#254)
* Deduplicate blocks in getAncestorBlocks

* Limit ancestor blocks in getAncestorBlocks
2019-04-26 09:21:46 +03:00
Yuriy Glukhov
b916b1e3e4
Slightly more validations 2019-04-10 15:56:22 +03:00
Yuriy Glukhov
54f894fc7e
Chunked sync second try
This reverts commit cdf9baf5e6a82d3f69762462206f7f8e6d5fea01.
2019-04-10 15:35:52 +03:00
Zahary Karadjov
cdf9baf5e6 Revert "Chunked sync"
This reverts commit c84d930304822a0f4bc9398ca61c8c48ae18745e.
2019-04-09 02:19:33 +03:00
Yuriy Glukhov
c84d930304 Chunked sync 2019-04-09 00:18:53 +03:00
Zahary Karadjov
44d9f7d6c9 Use proper RPC calls when obtaining missing blocks 2019-03-28 16:54:50 +02:00
Zahary Karadjov
406df9d44f More chatty syncing 2019-03-27 18:13:39 +02:00