Commit Graph

483 Commits

Author SHA1 Message Date
Zahary Karadjov 0d37ed7a2a
Fix a problem for the bootstrap nodes
connectToNetwork is executed for the bootstrap node as well, but
it should not quit when peers are not yet present.
2019-06-28 14:55:21 +03:00
Jacek Sieka 1ce459bcdb
Merge commit 'b0d27b9' 2019-06-27 14:52:46 +02:00
Jacek Sieka b0d27b9acc
disambiguate IPPROTO_TCP 2019-06-27 14:52:32 +02:00
Dustin Brody 43a9e904cf
replace get_attestation_slot(...) with 0.7.1 get_attestation_data_slot(...) and update all callers; update AttestationData to be almost 0.7.1, except nonextistent slot field; update Crosslink and get_winning_crosslink_and_attesting_indices(...) to 0.7.1 (#297) 2019-06-25 11:17:08 +00:00
Zahary Karadjov 1eef531259
Attempt to fix the build on Windows 2019-06-25 07:00:31 +03:00
Zahary Karadjov 69f3095fac Remove temp debugging code; Quit if you fail to connect to the network 2019-06-24 19:38:19 +03:00
Zahary Karadjov 31baa77742 Restore building with chronicles_sinks=json 2019-06-24 19:38:19 +03:00
Zahary Karadjov 7d174b548a Switch the testnet1 back-end to libp2p_native
Further simplifications and removal of support for multiple network
back-ends per testnet.
2019-06-24 19:38:19 +03:00
Zahary Karadjov 0116bdaed7 temporary code to debug issues with the testnet bootstrap node 2019-06-24 19:38:19 +03:00
Dustin Brody c7e06374f4
Remove get_crosslink_committees_at_slot and fix research/state_sim (#291)
* migrate attestation pool tests from get_crosslink_committees_at_slot(...) to get_crosslink_committee(...)

* rm obsolete, unused get_crosslink_committees_at_slot_cached(...) and migrate tests/test_state_transition from get_crosslink_committees_at_slot(...) to get_crosslink_committee(...)

* migrate tests/testutil from get_crosslink_committees_at_slot(...) to get_crosslink_committee(...)

* use more pervasive caching infrastructure, initially of compute_committee; remove buggy (and per-index) shuffling to fix research/state_sim, which was noticing validators on multiple shard committees; rm now-unused specific get_attesting_balance_cached

* add some get_active_validator_index caching

* rm obsolete/unused get_attesting_indices_cached

* rm get_crosslink_committees_at_slot(...)

* some more 0.7 changes -- some of which can't be completed pending some data structure updates -- and shard handling changes to offset with epoch start shards
2019-06-24 09:21:56 +00:00
Zahary Karadjov c751285112
Implement all libp2p_native response codes as specified in the latest proposal 2019-06-24 05:34:22 +03:00
Zahary Karadjov 84afb77b27
Restore compilation with libp2p_native after the latest changes in the spec back-end 2019-06-24 05:34:22 +03:00
Zahary Karadjov 44fc05f864
Fix the build for chronicles_sinks=json 2019-06-24 05:34:22 +03:00
Zahary Karadjov 25494aa2fb
Better error message on incompatible validator data when resetting a testnet 2019-06-24 05:34:22 +03:00
Zahary Karadjov e228c2dbcb
Implement the even/odd request ID scheme; Handle more edge cases; Break the cyclic imports 2019-06-24 05:34:22 +03:00
Zahary Karadjov 96e2a02faf
Respect the port configuration and NAT setup when using LibP2P 2019-06-24 05:34:22 +03:00
Zahary Karadjov 15fdf78d6a
[LibP2P] Persistent network key for the bootstrap node 2019-06-24 05:34:22 +03:00
Zahary Karadjov 12e9367f78
Improved error handling; Simple test case for connecting 2 peers 2019-06-24 05:34:22 +03:00
Zahary Karadjov 877b22cfb8
Share more code between the libp2p backends 2019-06-24 05:34:21 +03:00
Zahary Karadjov 87601a5eae
Share more code between the libp2p backends 2019-06-24 05:34:21 +03:00
Zahary Karadjov 3b166be166
Restore compilation in RLPx mode; More libp2p progress 2019-06-24 05:34:21 +03:00
Zahary Karadjov e177d17762
Further simplifications 2019-06-24 05:34:21 +03:00
Zahary Karadjov c060c0fc5d
Simplified the 'spec' back-end by movign more logic into the shared DSL module 2019-06-24 05:34:21 +03:00
Zahary Karadjov ea4690d567
A new P2P backend implementing the libp2p networking according to the official spec 2019-06-24 05:34:21 +03:00
Zahary Karadjov f4a96bc3f3
[WIP] Restore the Lib2P2 builds and implement the latest wire spec
Depends on https://github.com/status-im/nim-eth/pull/54
2019-06-24 05:34:21 +03:00
Dustin Brody 13381c8fea
update everything https://github.com/ethereum/eth2.0-specs/releases/tag/v0.7.1 leaves unchanged from v0.7.0 to v0.7.1 (#290) 2019-06-18 15:34:30 +00:00
Dustin Brody e9180c490e
more 0.7.0 updates (#287)
* update Deposit and BeaconBlockHeader to 0.7.0

* update get_total_balance(...) and get_base_reward(...) to 0.7.0

* rename initial value constant BLS_WITHDRAWAL_PREFIX_BYTE to BLS_WITHDRAWAL_PREFIX

* mark minimal presets as entirely migrated to 0.7.0; update process_rewards_and_penalties(...) to 0.7.0

* specify that targeting 0.7.0; adopt 0.7.0 refactoring of bls_domain/get_domain to update get_domain(...) to 0.7.0; mark that mainnet constant presets and block header processing are 0.7.0-consistent; update process_justification_and_finalization to 0.7.0 to fix potential underflow errors

* address mratsim's comment to copy whole spec comment
2019-06-14 16:21:04 +00:00
Dustin Brody 2a6e64d3f9
0.7.0 updates including semantic changes (#285)
* update get_previous_epoch(...)

* update BeaconBlock to 0.7.0 by renaming previous_block_root field to parent_root
2019-06-14 13:50:47 +00:00
Dustin Brody e2496567b3 More safe 0.7.0 updates (#283)
* complete marking unchanged-in-0.7.0 beaconstate functions

* complete marking unchanged-in-0.7.0 spec refs functions in helpers.nim and time.nim: get_randao_mix(...), bytes_to_int(...), and description of beacon chain processing apropos time

* complete remaining feasible 0.7.0 updates which lack externally visible effect
2019-06-14 09:50:14 +02:00
Dustin Brody 6357bc2a98 0.7.0 updates which don't change semantics; mostly comment changes (#281)
* 0.7.0 updates which don't change semantics; mostly comment changes

* mark get_attesting_indices, bls_verify, ProposerSlashing, BeaconBlockBody, deposit contract constants, state list length constants, compute_committee, get_crosslink_committee, is_slashable_attestation_data, processAttesterSlashings as 0.7.0; rename BASE_REWARD_QUOTIENT to BASE_REWARD_FACTOR

* complete marking unchanged-in-0.7.0 datatypes as such; a few more have trivial field renamings, etc
2019-06-13 09:40:58 +02:00
Dustin Brody d400650eeb
0.6.3 updates (#279)
* update IndexedAttestation, verify_slashable_attestation/verify_indexed_attestation, and attester slashing processing to 0.6.3

* rm debug scaffolding

* rename Attestation.aggregate_signature -> Attestation.signature; convert various references to AttestationData.slot to get_attestation_slot; implement convert_to_indexed; update checkAttestation and processAttestations to 0.6.3; remove spurious assertion in beacon node related to invalid attestations

* replace 0.5 get_winning_root_and_participants with 0.6 get_winning_crosslink_and_attesting_indices; update process_crosslinks to 0.6.3

* mark both remaining 0.6.0 spec implementations as 0.6.3

* clear out all remaining spec version 0.6.1 refs

* GENESIS_SLOT and GENESIS_EPOCH are 0

* rm 0.5 get_attestation_participants in favor of 0.6 get_attesting_indices

* address mratsim's comment

* time can be equal to genesis

* fix invalid block assertions; those were essentially spurious

* allow toBeaconTime to handle time before genesis (in accordance with now(...) which states time can exist before GENESIS)
2019-06-12 07:48:49 +00:00
Yuriy Glukhov 10c7920b27 Proto changes to facilitate backward sync (#271)
* Proto changes to facilitate backward sync

* Update to latest spec types in sync proto

* Use blockpool for more straightforward block headers collection

* Added BlockPool.getRef

* Update beacon_chain/sync_protocol.nim

Co-Authored-By: Jacek Sieka <arnetheduck@gmail.com>
2019-06-10 07:13:53 -04:00
Dustin Brody 252819a84d
bump everything that's not one of the 3 or so specific changes in 0.6.3 from 0.6.2 to 0.6.3 if already at 0.6.2 (#277) 2019-06-05 10:25:30 +00:00
Dustin Brody 2e1515b107
0.6.2 updates (#275)
* update process_justification_and_finalization to 0.6.2; mark AttesterSlashing as 0.6.2

* replace get_effective_balance(...) with state.validator_registry[idx].effective_balance; rm get_effective_balance, process_ejections, should_update_validator_registry, update_validator_registry, and update_registry_and_shuffling_data; update get_total_balance to 0.6.2; implement process_registry_updates

* rm exit_validator; implement is_slashable_attestation_data; partly update processAttesterSlashings

* mark HistoricalBatch and Eth1Data as 0.6.2; implement get_shard_delta(...); replace 0.5 finish_epoch_update with 0.6 process_final_updates

* mark increase_balance, decrease_balance, get_delayed_activation_exit_epoch, bls_aggregate_pubkeys, bls_verify_multiple, Attestation, Transfer, slot_to_epoch, Crosslink, get_current_epoch, int_to_bytes*, various constants, processEth1Data, processTransfers, and verifyStateRoot as 0.6.2; rm is_double_vote and is_surround_vote

* mark get_bitfield_bit, verify_bitfield, ProposerSlashing, DepositData, VoluntaryExit, PendingAttestation, Fork, integer_squareroot, get_epoch_start_slot, is_active_validator, generate_seed, some constants to 0.6.2; rename MIN_PENALTY_QUOTIENT to MIN_SLASHING_PENALTY_QUOTIENT

* rm get_previous_total_balance, get_current_epoch_boundary_attestations, get_previous_epoch_boundary_attestations, and get_previous_epoch_matching_head_attestations

* update BeaconState to 0.6.2; simplify legacy get_crosslink_committees_at_slot infrastructure a bit by noting that registry_change is always false; reimplment 0.5 get_crosslink_committees_at_slot in terms of 0.6 get_crosslink_committee

* mark process_deposit(...), get_block_root_at_slot(...), get_block_root(...), Deposit, BeaconBlockHeader, BeaconBlockBody, hash(...), get_active_index_root(...), various constants, get_shard_delta(...), get_epoch_start_shard(...), get_crosslink_committee(...), processRandao(...), processVoluntaryExits(...), cacheState(...) as 0.6.2

* rm removed-since-0.5 split(...), is_power_of_2(...), get_shuffling(...); rm 0.5 versions of get_active_validator_indices and get_epoch_committee_count; add a few tests for integer_squareroot

* mark bytes_to_int(...) and advanceState(...) as 0.6.2

* rm 0.5 get_attesting_indices; update get_attesting_balance to 0.6.2

* another tiny commit to poke AppVeyor to maybe not timeout at connecting to GitHub partway through CI: mark get_churn_limit(...), initiate_validator_exit(...), and Validator as 0.6.2

* mark get_attestation_slot(...), AttestationDataAndCustodyBit, and BeaconBlock as 0.6.2
2019-06-03 10:31:04 +00:00
Jacek Sieka 16fb9e8d11 Add support for dealing with overlapping attestations (#262) 2019-06-03 08:26:38 +00:00
Dustin Brody 6a47ca3c46
More 0.6.1 updates (#274)
* implement get_churn_limit from 0.6.1; update initiate_validator_exit and get_previous_epoch to 0.6.1; remove obsoleted/non-spec reduce_balance, replaced by decrease_balance; mark BeaconBlock as 0.6.1

* rename get_block_root to get_block_root_at_slot and introduce epoch-oriented get_block_root; implement get_attestation_slot, get_matching_source_attestations, get_matching_target_attestations, get_matching_head_attestations, 0.6.1 get_attesting_indices, get_unslashed_attesting_indices, get_attestation_deltas, process_rewards_and_penalties; rm get_inactivity_penalty

* update Validator and processVoluntaryExits to 0.6.1; rm obsolete inclusion_slots/inclusion_distances

* rm removed activate_validator; mark DepositData, misc values, Gwei values, Randao processing, state caching as 0.6.1; update GENESIS_SLOT to 64, since it doesn't quite yet work at 0

* mark BeaconBlockHeader as 0.6.1; update BeaconBlockBody to 0.6.1; rename WHISTLEBLOWER_REWARD_QUOTIENT to WHISTLEBLOWING_REWARD_QUOTIENT; update slash_validator to 0.6.1

* implement 0.6.2 is_slashable_validator; update processProposerSlashings to 0.6.2; mark state caching as 0.6.2

* mark get_total_active_balance and process_slashings as 0.6.2; update get_base_reward to 0.6.2

* rm prepare_validator_for_withdrawal, process_exit_queue; mark mainnet misc constants as 0.6.2; update process block header to 0.6.2

* address mratsim's code review comment
2019-05-29 10:08:56 +00:00
Yuriy Glukhov f679677995 Fixed signing_root hack (#272) 2019-05-28 08:44:25 +00:00
Mamy Ratsimbazafy 6b1f0e816c
Introduce constant presets (#269)
* Initial move constants to mainnet preset

* Bump fixtures to include shuffling with minimal preset

* Add minimal preset with shuffling test

* Add minimal preset to the full test suite

* use preset() everywhere
2019-05-27 14:48:13 +02:00
Dustin Brody 46cfce652d
initial new get_beacon_proposer_index and dependencies (get_shuffled_index, get_shard_delta, get_epoch_start_shard, get_crosslink_committee) (#266)
* initial new get_beacon_proposer_index and dependencies (get_shuffled_index, get_shard_delta, get_epoch_start_shard, get_crosslink_committee)

* more 0.6.1 updates

* clean up debugEchos and fix validator keygen to create correct indices
2019-05-23 11:13:02 +00:00
Zahary Karadjov 8da71715eb Bump the beacon_node version; Make the simulation script compatible with macOS
The shell expansion syntax used in the run_node script was not handled properly
by the bash shell shipped with macOS. In particular, it was not able to handle
range expressions involving interpolated vars:

cp foo/{$FIRST_INDEX..$LAST_INDEX} ...
2019-05-14 16:46:43 +03:00
Yuriy Glukhov a0d4c3432f Crude request parallelization 2019-05-11 22:47:53 +03:00
Mamy Ratsimbazafy e5047d93a4 [Tests] SHA256 + official shuffling vectors (#263)
* Change digests to SHA2-256 (from Keccak256)

* Fix sha256 digest initialization - pass shuffling test

* Add comments + remove old shuffling tests

* Remove stale tree hashing tests

* Small toOpenArray optim + notes on in-place hashing

* Revert "Revert "Update test repo""

This reverts commit f385467cd3.

* Revert "Revert "Fix SIGFPE on shuffling for 0""

This reverts commit 226d380f87.

* Revert "Revert "Implement shuffling test (pending typedesc and shuffling algo fix)""

This reverts commit 813cb6fbb8.

* withEth2hash templates now needs ctx.init()

* Use init-update-finish to avoid burnMem
2019-05-10 08:14:01 +00:00
Dustin Brody d977c96ae1 0.6.1 - signing_root, deposits, validator indices and committee count (#261)
* signed_root -> signing_root

* implement new process_deposit; update timing parameters to 0.6.1; update Deposit to 0.6.1 and remove DepositInput

* update get_active_validator_indices and get_epoch_committee_count to 0.6.1; rm get_current_epoch_committee_count, get_previous_epoch_committee_count, and get_next_epoch_committee_count; bump state_sim default validator count a bit more

* re-introduce 0.5.1-ish get_active_validator_indices, get_epoch_committee_count as scaffolding for still-0.5.1ish shuffling
2019-05-09 14:27:37 +02:00
Mamy André-Ratsimbazafy 226d380f87
Revert "Fix SIGFPE on shuffling for 0"
This reverts commit 8cfbfdf565.
2019-05-09 12:07:09 +02:00
Mamy André-Ratsimbazafy 8cfbfdf565
Fix SIGFPE on shuffling for 0 2019-05-09 12:01:01 +02:00
Jacek Sieka 17fa499755 cache state root a bit more aggressively (#260)
needed to be able to run state sim, too slow otherwise
2019-05-04 14:10:45 +00:00
Jacek Sieka bd4893d2e8 head block selection fixes (#259)
* track justified and head blocks correctly in presence of blank slots
* fix new block state root
* keep list of viable heads instead of walking children
* disable json bitfield deserializer (needs a corresponding serializer)
* fix handshake best & finalized root
2019-05-01 12:19:29 +03:00
Dustin Brody 46b4154ce8 begin 0.6.0 (#256)
* begin 0.6.0: new get_domain/increase_balance/reduce_balance, BeaconState.validator_balances -> BeaconState.balances, some renamed constants, transaction processing changes, SlashableAttestation field name changes, 0.6.0 get_beacon_proposer_index always uses given state's slot, update tests subrepo

* mark get_bitfield_bit/bls_verify_multiple/stat-list-lengths/is_active_validator/is_surround_vote/slot_to_epoch/int_to_bytes/etc as unchanged in 0.6.0; rm Eth1DataVote/maybe_reset_eth1_period and thus adjust expected tree hash test results

* mark verify_bitfield/bls_verify/deposit-contract/VoluntaryExit/PendingAttestation/Historicalbatch/Fork as 0.6.0; update DOMAIN_BEACON_BLOCK to DOMAIN_BEACON_PROPOSER

* update Crosslink to 0.6.0 (also requires tree hashing test result change, so isolate in individual commit)

* mark verify_merkle_branch/get_delayed_activation_exit_epoch/ProposerSlashing/Attestation/AttestationDataAndCustodyBit/hash/integer_squareroot/get_epoch_start_slot/is_double_vote/get_randao_mix/generate_seed as 0.6.0; update reward and penalty quotients; SlashableAttestation -> IndexedAttestation; rm get_fork_version; ATTESTATION_INCLUSION_REWARD_QUOTIENT -> PROPOSER_REWARD_QUOTIENT
2019-04-29 10:48:30 -06:00
Mamy Ratsimbazafy 5784b2d7f7
Full EF state tests parsing (#242)
* Load full state test + Add json-serialization for Bitfield

* Implement empty_block_transition test (but test is missing the end state)

* Use the provided empty block instead of mocking one

* Add failing block signing test

* Tests that can't be passed now are now "for information" + indent the "information"/hash given
2019-04-29 18:10:01 +02:00
Jacek Sieka c7e6d39279 fix statedata fields not being updated correctly (#255) 2019-04-29 11:14:22 +03:00
Yuriy Glukhov 61ec6fa167 Move incoming block from unresolved to pending if it can't be applied (#253)
* Move resolved block from unresolved to pending if it can't be applied

* Renamed unresolved to missing
2019-04-26 10:38:56 -06:00
Yuriy Glukhov ddcdb63430
Deduplicate blocks in getAncestorBlocks (#254)
* Deduplicate blocks in getAncestorBlocks

* Limit ancestor blocks in getAncestorBlocks
2019-04-26 09:21:46 +03:00
Ștefan Talpalaru 53012b9a25
NAT traversal 2019-04-18 15:48:17 +02:00
Yuriy Glukhov b916b1e3e4
Slightly more validations 2019-04-10 15:56:22 +03:00
Yuriy Glukhov 54f894fc7e
Chunked sync second try
This reverts commit cdf9baf5e6.
2019-04-10 15:35:52 +03:00
Zahary Karadjov cdf9baf5e6 Revert "Chunked sync"
This reverts commit c84d930304.
2019-04-09 02:19:33 +03:00
Zahary Karadjov ba48fcd165 Slightly better error messages during testnet upgrades; Upgrade testnet0 to 1000 validators 2019-04-09 00:36:06 +03:00
Yuriy Glukhov c84d930304 Chunked sync 2019-04-09 00:18:53 +03:00
Jacek Sieka 9a2f3176f7 guard node.state (#237)
* fix node.state being used unsafely across async callbacks (fixes #230)
* node.state -> node.stateCache
* fix attestation validation not to use asserts
* re-enable attestation pool tests
* prepare for resizing validator registry
2019-04-06 09:46:07 +02:00
Dustin Brody c53de3e550 Caching updates/refactoring & state_sim defaults updates (#235)
* rm now-superceded shuffling cache (shuffling is only called from get_crosslinks), which was badly architected due to trying to exist in state; rm one more vestige of previous light-client regime (one more to go, from datatypes)

* fix wrong shuffling list size (active validator size, not validator size) to make consistent with 0.5.1 (will be inconsistent with testnet0); fix typo and change defaults in state_sim

* doAssert a couple of constant relationships necessary to avoid underflow; rm non-spec, unused helper function get_new_recent_block_roots

* refactor separate crosslink_committee_cache and winning_root_participants_cache(s) into StateData object; remove last vestige of previous shuffling cache

* separate out caching parts of StateData to new StateCache object
2019-04-05 08:18:13 -06:00
Jacek Sieka 605dd0a0e9 Some speedups (#226)
* ssz: avoid memory allocations

* a bit fishy with the 32-item stack.. this should be a smallvector

* digest: avoid another burnmem

* avoid a few allocations by using iterator
2019-04-03 15:46:22 +00:00
Yuriy Glukhov ff7adcefbd Persist head block 2019-04-02 15:35:09 +03:00
Ștefan Talpalaru f6eb6af310
downloadFile(): print command in the error message
so people can figure out that they don't have "curl" installed
2019-04-01 20:41:35 +02:00
zah e16d8faee2 Switch to a two-steps build process (#228) 2019-03-29 10:44:19 -06:00
Zahary Karadjov 6689d0a642 Remove some deprecation warnings 2019-03-29 16:45:38 +02:00
Zahary Karadjov 8e0025035c all validators are free for picking by default 2019-03-29 15:38:59 +02:00
Dustin Brody 6cbf355c5a state_transition part of get_attesting_indices/get_attesting_indices_cached perf tweaks 2019-03-28 23:31:30 +02:00
Dustin Brody d74fa2027a remove redundant conversions 2019-03-28 23:31:30 +02:00
cheatfate 3117a4481e Remove unneeded initialization. 2019-03-28 23:24:50 +02:00
cheatfate 6f4c7b775c Remove one-line hash to avoid burnMem.
Remove unnecessary copyMem()s.
2019-03-28 23:24:50 +02:00
Jacek Sieka 201b079fe3 avoid bad blocks by filtering included attestations
* fix epoch violation during initial attestation validation
* fill in attestation fields (finalizes again!)
2019-03-28 21:10:08 +02:00
Zahary Karadjov 44d9f7d6c9 Use proper RPC calls when obtaining missing blocks 2019-03-28 16:54:50 +02:00
Zahary Karadjov f23ee4ccae Fix a wrong assert 2019-03-28 16:54:50 +02:00
Zahary Karadjov e0d8bf9e99 Logging in preInit works just fine locally; pushing to test on the CI 2019-03-28 16:54:50 +02:00
Zahary Karadjov 6d4470a8c8 Simplified gossipsub.broadcast further 2019-03-28 16:54:50 +02:00
Zahary Karadjov dd72218f48 Integrate `traceAsyncErrors`
You'll need the latest version of nim-eth
2019-03-28 16:54:50 +02:00
Yuriy Glukhov 497f88c20d Increase minPeers in PeerPool to PeerPool default (10) 2019-03-28 16:35:17 +02:00
Jacek Sieka f9e0418b5b fetch multiple blocks at a time
* avoid crash on invalid block production (fixes #209)
* fetch blocks every second, roughly
* fix start.sh param order
* run beacon node sim at slower pace
2019-03-28 10:42:16 +02:00
Dustin Brody 58b6174654 smallish speedup -- also use cache in get_crosslink_deltas 2019-03-28 10:40:20 +02:00
Dustin Brody 6f481f8812 another ~2x speedup in epoch processing 2019-03-28 10:40:20 +02:00
Dustin Brody e1ba820486 clean up caching 2019-03-28 10:40:20 +02:00
Dustin Brody d84aa45a22 share crosslink committees cache across process_crosslinks and compute_crosslinks_deltas 2019-03-28 10:40:20 +02:00
Dustin Brody 2c566df411 rm debug message 2019-03-28 10:40:20 +02:00
Dustin Brody ced4dbe859 4x speedup on epoch processing for 64 validators 2019-03-28 10:40:20 +02:00
Jacek Sieka 3ad6053298 Use block, slot tuple to idenfiy fork
this is the beginning of tracking block-slots more precisely, so we can
the justification epoch slot bug.

* avoid asyncDiscard (swallows assertions)
* fix attestation delay
* fix several state root cache bugs
* introduce workaround for genesis epoch spec issue
2019-03-28 10:32:09 +02:00
Zahary Karadjov 92c38d6d48 Don't log the backwards sync requests 2019-03-27 18:32:33 +02:00
Zahary Karadjov 2efcffe6ac Fix an issue for completely fresh installations 2019-03-27 18:29:02 +02:00
Zahary Karadjov 406df9d44f More chatty syncing 2019-03-27 18:13:39 +02:00
Zahary Karadjov b3d23c052c Switch to 30 seconds slot duration; UX improvements
* On the testnet site, the first N instead of the last N validators
  will be considered reserved for users

* The number of validators is consistently named `totalValidators`
  in all commands

* Proper keys are generated by default in validator_keygen
2019-03-27 18:13:39 +02:00
Zahary Karadjov a3ce9d17ec Expose the default port 2019-03-27 18:13:39 +02:00
Zahary Karadjov 6629897567 We don't need the more complicated port setup anymore 2019-03-27 18:13:39 +02:00
Jacek Sieka edd9826464
some ignores 2019-03-26 19:41:34 -06:00
Jacek Sieka 81c66fbce0
use signed_root as canonical block root (#211)
* no need to pass prev_block_root when updating state
* some Slot fixes
* fix `hash_tree_root` for Slot, Epoch
* detect missing hash_tree_root type support
* remove some <0.5 state checks
2019-03-26 19:32:35 -06:00
Zahary Karadjov c0fa229254 Improved error handling in the networking code 2019-03-26 17:40:20 +02:00
Zahary Karadjov d94d4f2606 Dumbed down the validator key import procedure
The previous idempotent scheme proved to be too slow, making it
harder to quickly re-assign validators on the servers.
2019-03-26 17:37:31 +02:00
Zahary Karadjov 5b1caaff41 more detailed error message when you connect to an incompatible testnet 2019-03-26 17:37:31 +02:00
Mamy André-Ratsimbazafy 8273345a2c fix comment 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy 8b5e18b7a4 Magic fix to #209 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy 2fafa8b1a1 Add TODO comment about loggable stacktraces and open https://github.com/status-im/nim-chronicles/issues/47 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy 57c693c88f Only check for epoch underflow just before peer communication
During state processing epochs can underflow (hence why we have a non-zero arbitrary genesis epoch)
2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy e189c1d3cd Crash before making attestation when replaying state pass genesis 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy d1f75cdfef Improve distinction between debugging stacktraces and crash/exception stacktraces 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy e8160c2126 more logs to investigate epoch underflow/overflow https://github.com/status-im/nim-beacon-chain/issues/207 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy 9b520d3949 address comment 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy 808c5edfc0 Add more fork choice logs 2019-03-26 17:35:09 +02:00
Jacek Sieka e7b36c4389 add beacon clock and rework timing
* have regular slot tick, decide on work to do lazily
* use beacon clock instance that will eventually sync time
* skip work when falling behind
* add one more state cache for justified state (since that's used in
fork choice)
* remove old sync cruft
* single timer per slot, serializing block production and attestation
2019-03-26 13:21:48 +02:00
Yuriy Glukhov e4c10a31c9 Use latest state from the DB in sync. Fixed off by one in block roots (#203) 2019-03-25 14:52:23 -06:00
Zahary Karadjov 70cb84ef93 Fix the build with the latest nim-eth 2019-03-25 19:57:05 +02:00
Jacek Sieka 1b0e67c88c
ssz: update to 0.5.1:ish (#202)
* ssz: update to 0.5.1:ish
* slightly fewer seq allocations
* still a lot of potential for optimization
* fixes #174
* ssz: avoid reallocating leaves (logN merkle impl)
2019-03-25 10:46:31 -06:00
Dustin Brody 6b1527ccd5 Almost final 0.5.1 spec updates (#201)
* update processExits to 0.5.1

* mark process_deposit, get_empty_block as 0.5.1; update get_genesis_beacon_state to 0.5.1; implement processDeposits

* hotfix via https://github.com/ethereum/eth2.0-specs/pull/821
2019-03-24 12:21:23 -06:00
Dustin Brody 9f55b4646c
More 0.5.1 spec updates (#195)
* rm gone-in-0.5.0 Proposal, verifyBlockSignature, and slot check which moved to spec function processBlockHeader

* mark get_attestation_participants and get_epoch_committee_count as 0.5.1; finish updating processAttestations to 0.5.1; add kludgy workaround for bug relating to get_winning_roots_etc using crosslink_data_root as index when we have that as ZERO_HASH for all leading to it confusing attesters on different shards; rm BeaconState.latest_attestations, which splits into previous_epoch_attestations and current_epoch_attestations

* Fix CI due to removed latest_attestations field
2019-03-22 18:33:12 +00:00
Zahary Karadjov 0a027e410a Address review concerns and make some steps towards json logging 2019-03-22 17:04:17 +02:00
Zahary Karadjov 06301458bf Don't add the 'node' log property when connecting to the testnet 2019-03-22 17:04:17 +02:00
Mamy André-Ratsimbazafy ca49ae9b5a
Report connected peers before fork choice 2019-03-22 12:33:10 +01:00
Yuriy Glukhov 894816fa0c Fixed crash caused by findAncestorBySlot (#196)
closes #193
2019-03-22 11:57:19 +01:00
Dustin Brody 94c3b005fa
https://github.com/ethereum/eth2.0-specs/pull/816 hotfix (#194) 2019-03-21 16:02:50 +00:00
Dustin Brody 1bb2859aca
implement bulk of 0.5.0 state transition function (#192) 2019-03-21 15:31:36 +00:00
Jacek Sieka ee89ef1c79
cleanups (#189)
* add simple bitfield type (fixes #19)
* fix bit endianess to match spec
* consolidate attestation and block logging
* cruft cleanup
* run optimizer on tests, speeds up build
2019-03-20 14:01:48 -06:00
Dustin Brody 13c7f7fcec begin refactoring 0.5.0 epoch transition (#185)
* begin refactoring 0.5.0 epoch transition at justification and finalization

* update processTransfers to 0.5.0 spec

* update get_crosslink_committees_at_slot to 0.5.0 except for noted workaround for pre-0.5.0 spec bug related to epoch boundary conditions, to be flipped in more coordinated way with other epoch processing changes

* mark ejection processing as 0.5.0

* 0.5.0 epoch update finalization

* rm BEACON_CHAIN_SHARD_NUMBER to complete updating miscellaneous constants to 0.5.0

* mark verify_slashable_attestation as 0.5.0

* update process_attester_slashing to 0.5.0

* update process_slashings(...) to 0.5.0 and mark process_exit_queue as 0.5.0

* refactor epoch processing by implementing 0.5.0's update_registry_and_shuffling_data(...)
2019-03-19 20:19:46 -06:00
Zahary Karadjov 6d4284877d Tentative fix for self-connection discovery issues 2019-03-20 03:18:59 +02:00
Zahary Karadjov 6ec8ffe0ff Much better error reporting on snapshot deserialization problems 2019-03-20 02:05:10 +02:00
Zahary Karadjov b454501595 handle missing validators dir during key import 2019-03-19 23:04:22 +02:00
Zahary Karadjov 42ca6afae2 Helpful scripts for resetting the testnets 2019-03-19 21:50:22 +02:00
Zahary Karadjov 6a35d3584d Automate the creation of the network metadata files
With these changes, running a simulation is very close to running
an actual testnet. Some checks have been added in the client to
make sure you are not connecting to an incompatible network (e.g.
a network running with a different number of shards).
2019-03-19 20:27:54 +02:00
Jacek Sieka 8bab6fd51f
Add TODO for local bind ip 2019-03-19 11:19:27 -06:00
Jacek Sieka d7f972a47e allow setting advertised external ip
* detect it in start.sh
2019-03-19 10:50:09 +02:00
Dustin Brody 2bc632c29b
AttestationData.justified epoch -> AttestationData.source_epoch and AttestationData.latest_crosslink -> AttestationData.previous_crosslink; implement verify_merkle_branch and update process_deposit to use it; rm BeaconState.batched_block_roots; Transfer.{from, to} -> Transfer.{sender, recipient}; update Fork data type and code to use bytes4; implement int_to_bytes8 (close to metaprogramming time); adjust signed_root and all callers to use 0.5.0 calling signature; processDepositRoot -> processEth1Data (#182) 2019-03-18 15:42:42 +00:00
Zahary Karadjov 6bb38095c9 Command-line and data storage handling for testnets 2019-03-18 12:45:29 +02:00
Dustin Brody 798197879b
Revert "AttestationData.justified epoch -> AttestationData.source_epoch and AttestationData.latest_crosslink -> AttestationData.previous_crosslink; implement verify_merkle_branch and update process_deposit to use it; rm BeaconState.batched_block_roots; Transfer.{from, to} -> Transfer.{sender, recipient}; update Fork data type and code to use bytes4; implement int_to_bytes8 (close to metaprogramming time); adjust signed_root and all callers to use 0.5.0 calling signature; processDepositRoot -> processEth1Data"
This reverts commit 18f1d000a1.
2019-03-17 15:24:50 -07:00
Dustin Brody 18f1d000a1
AttestationData.justified epoch -> AttestationData.source_epoch and AttestationData.latest_crosslink -> AttestationData.previous_crosslink; implement verify_merkle_branch and update process_deposit to use it; rm BeaconState.batched_block_roots; Transfer.{from, to} -> Transfer.{sender, recipient}; update Fork data type and code to use bytes4; implement int_to_bytes8 (close to metaprogramming time); adjust signed_root and all callers to use 0.5.0 calling signature; processDepositRoot -> processEth1Data 2019-03-17 15:17:40 -07:00
Dustin Brody f36c2d86dc begin 0.5.0 spec update (#179)
* begin 0.5.0 spec update: parent_root -> previous_block_root, BeaconState.justified_epoch -> BeaconState.current_justified_epoch, DOMAIN_PROPOSAL -> DOMAIN_BEACON_BLOCK, temporarily rename BeaconBlockHeader to BeaconBlockHeaderRLP to allow for gradual re-merging without disrupting RLP; mark a few unchanged functions and data types, implement get_temporary_block_header/get_empty_block/should_update_validator_registry/processBlockHeader/cacheState; update a few others

* a dozen or so more trivial mostly comment changes, finding more unchanged parts of 0.5.0

* several more trivial changes; goal is to reduce noise around the upcoming substantial changes (epoch processing, etc)
2019-03-16 13:52:37 -06:00
Jacek Sieka 1cb8ae9004
state processing fixes (#177)
* remove some redundant state updates
* when attesting late, use correct state / head
* don't send out obsolete attestations
* don't propose obsolete blocks
* remove some more resundant state updates :)
* simplify block logging (experimental)
* document fork choice division
* fix some Slot / Epoch conversion warnings
2019-03-14 07:33:56 -06:00
Ștefan Talpalaru 9ff1eb4ac8
Merge branch 'master' of github.com:status-im/nim-beacon-chain 2019-03-14 00:05:12 +01:00
Ștefan Talpalaru de295619be
assert() -> doAssert() 2019-03-14 00:04:43 +01:00
Jacek Sieka b0f4034060
implement fork choice (#175)
* keep track of a finalized block
* keep track of all justified blocks
* use naive spec version of LMD ghost
* cache slot number and a few more things in BlockRef
* keep track of the latest vote of each validator
* depend less on the state of node.state (it's a cache, effectively)
2019-03-13 16:59:20 -06:00
Zahary Karadjov 562eafe124 Finish the Spec/Epoch types refactoring 2019-03-13 23:23:01 +02:00
Dustin Brody 1479bae22f implement distinct Epoch type and continue 0.4.0 spec update (#173)
* implement distinct Epoch type
* update two more spec 0.4.0 markers, leaving just deposit processing
2019-03-12 19:46:44 -06:00
Dustin Brody 50a0948bc0 Mostly convert to using distinct Slot type (#172) 2019-03-12 16:21:32 -06:00
Mamy André-Ratsimbazafy 6dea18c140
Squashed commit of the following:
commit bed6909f33
Author: Mamy André-Ratsimbazafy <mamy_github@numforge.co>
Date:   Mon Mar 11 12:22:35 2019 +0100

    Cleanup imports

commit 241f62ed8d
Author: Mamy André-Ratsimbazafy <mamy_github@numforge.co>
Date:   Mon Mar 11 12:06:00 2019 +0100

    Group beacon types to prevent recursive imports
2019-03-12 16:03:14 +01:00
Zahary Karadjov 5d186ef93c Fix the build of serialized_sized 2019-03-12 15:38:58 +02:00
Zahary Karadjov 0206496663 Fix a build error 2019-03-12 15:14:30 +02:00
Zahary Karadjov 903cb8a8b5 A LibP2P-based networking stack;
To enable it, comment out the 'withLibp2p' line in nim.cfg

The history was squashed in order to remove an accidentally
commited binary file.

Other changes:

* SSZ was adapted to use the common serialization framework

* gossibsup.subscribe is not using async handlers at the moment
  and this allowed me to simplify it
2019-03-12 13:36:55 +02:00
Dustin Brody 434ba5727f finish all but 3 items for 0.4.0 spec update (#167) 2019-03-12 10:59:43 +01:00
Yuriy Glukhov b587455e7b Use BlockPool for slot->blocks mapping (#166) 2019-03-11 09:38:36 -06:00
Dustin Brody 13de015038 more 0.4.0 spec updates: renamings, continue Proposal updates, and note several more functions which were verified not to have changed in 0.4.0 (#163) 2019-03-11 09:33:24 -06:00
Dustin Brody 89aa3ec53f
fix keccak256 digest comments 2019-03-09 12:34:08 -08:00
Jacek Sieka aec31b1126 Updates for finalizing chain! (#162)
* set epoch_boundary_root - chain finalizes!
* fix slotStart to offset GENESIS_SLOT
* work around bug that will be fixed by
https://github.com/ethereum/eth2.0-specs/pull/732
* compile with debug info (there was a GC-related crash in C land)
2019-03-09 04:23:14 +00:00
Dustin Brody 0b47acae37
some comment changes noting further 0.4.0-spec-update progress, but no meaningful semantic changes (#161) 2019-03-08 18:23:42 +00:00
Dustin Brody 1afdcda62d
spec version 0.4.0 update: shard_block_root -> crosslink_data_root; switch from validator.slashed_epoch -> validator.slashed approach; a couple assert -> doAssert; note fix for underflow-prone spec changes in checkAttestation; GENESIS_SLOT change from 2^63 to 2^32; refactor Proposer and ProposalSlashing structure; add signed_root; remove duplicate slashValidator (#159)
* spec version 0.4.0 update: shard_block_root -> crosslink_data_root; switch from validator.slashed_epoch -> validator.slashed approach; a couple assert -> doAssert; note fix for underflow-prone spec changes in checkAttestation; GENESIS_SLOT change from 2^63 to 2^32; refactor Proposer and ProposalSlashing structure; add signed_root; remove duplicate slashValidator

* re-apply shard_block_root -> crosslink_data_root

* remove incorrect humaneSlotNum

* add (run-time only, alas) sanity check on signed_root
2019-03-08 17:44:31 +00:00
Jacek Sieka 6bcefc0e42
verify blocks before storing in block pool / database (#158)
* fix state db lookup typo
* fix randao reveal slot when proposing blocks
* only store blocks that can be applied to a state
* store state at every epoch boundary (yes, needs pruning!)
* split out state advancement function when there's no block
* default state sim to 0.9 attestation ratio
2019-03-08 10:40:17 -06:00