Commit Graph

4314 Commits

Author SHA1 Message Date
Zahary Karadjov 398ea55801
Implement the latest SSZ specification and integrate the official SSZ test suite 2019-08-05 03:18:48 +03:00
Yuriy Glukhov 46c9e99754 Cleanup web3 api usage 2019-07-31 16:39:26 +03:00
Yuriy Glukhov 24cbc0710f Make web3 way of getting genesis optional 2019-07-31 16:39:26 +03:00
Yuriy Glukhov 68c4cbf078 Updated deposit contract 2019-07-31 16:39:26 +03:00
Yuriy Glukhov 958e4cd3b4 Getting genesis through web3 2019-07-31 16:39:26 +03:00
Mamy André-Ratsimbazafy 3e622644e7
Remove committed binaries 2019-07-21 17:23:05 +02:00
Zahary Karadjov 33dfc8b2d5
Fix the compilation when json logging is used 2019-07-16 13:40:24 +03:00
Dustin Brody b049dded88
summarize why previous epoch attestations can be recapped where they are 2019-07-16 11:16:04 +02:00
Dustin Brody f6fd089ae7
Add non-attesting-validators instrumentation in process_justification_and_finalization(...) (#323)
* update process_justifications_and_finalizations(...) to 0.8.1
2019-07-16 07:34:11 +00:00
Mamy Ratsimbazafy d1d5497233
v0.7.x-v0.8.1 state transition refactor part2 (#322)
* State sim compiles and run

* Works on libp2p testnet1

* Successfully update the state transition tests

* naive update of attestation pool (failing attstations can arrive in any order)

* Remove an official assert to allow attestation reordering
2019-07-15 23:10:40 +02:00
Dustin Brody 712038fe21
mark applicable 0.8.0-compliant functions and data structures as 0.8.1 and update a couple of other comments (#321) 2019-07-15 12:57:18 +00:00
Dustin Brody c92806b950
include both static and dynamic check for invariants to ensure that get_crosslink_committee(...) can access all committees by ensure that SHARD_COUNT >= SLOTS_PER_EPOCH (#318)
* include both static and dynamic check for invariants to ensure that get_crosslink_committee(...) can access all committees by ensure that SHARD_COUNT >= SLOTS_PER_EPOCH

* add state_sim to CI, to ensure that it also doesn't break SHARD_COUNT/SLOTS_PER_EPOCH assumptions, along with just in general not breaking
2019-07-13 15:58:01 +00:00
Ștefan Talpalaru 2768a3e89e
Ctrl+C handling and:
- added a "quitProc" for shutting down the p2pd process, after seeing it
  survive an unhandled exception (this required a global var for the
  DaemonApi instance)
- Makefile testnet0/1 targets now depend on the "p2pd" binary
- prefixed the testnet command line in the build script message with
  "../../env.sh" so it can find the superproject's "p2pd"
2019-07-11 04:36:07 +02:00
Dustin Brody 7bc2b81e18
~2x state_sim speedup via additional caching in get_crosslink_committee(...) (#316) 2019-07-10 14:00:28 +00:00
Dustin Brody dfa062db7d 0.8.0, continued (#315)
* rename get_genesis_beacon_state(...) to initialize_beacon_state_from_eth1(...); remove unused vestiges get_temporary_block_header(...) and get_empty_block(...); partly align initialize_beacon_state_from_eth1(...) with 0.8.0 version; implement CompactCommittee object type; update process_final_updates(...) to 0.8.0

* mark get_shard_delta(...) as 0.8.0

* mark get_total_active_balance(...) and epoch transition helper functions as 0.8.0

* move FORK_CHOICE_BALANCE_INCREMENT, not in spec anymore, to sole user in fork_choice
2019-07-10 14:23:02 +02:00
Dustin Brody 0a2d09e7fe
Speed up process_crosslinks(...) and get_crosslink_deltas(...) by 10x - 15x in state_sim (#314)
* remove some low-hanging perf silliness from get_winning_crosslink_and_attesting_indices(...) and switch inner/outer loop nesting order to get 10-15x function speedup for 128 and 512 validator cases by avoiding accidentally quadratic behavior, while keeping function signature unchanged and allowing easy ongoing verification of correctness of optimization

* avoid memory allocation with an openarray rather than seq

* update get_crosslink_deltas(...) to 0.8.0

* move non-spec ValidatorSetDeltaFlags from datatypes.nim to sync_protocol; fix mainnet MIN_ATTESTATION_INCLUSION_DELAY preset; update get_attestation_deltas(...) to 0.8.0; for 512 validators, get 90x speedup for get_attestation_deltas(...) from 179s to 2s

* explain purpose of when false portions, so they can be removed once accomplished
2019-07-08 13:19:52 +00:00
Jacek Sieka b9e62a1f6c
std_shims -> stew 2019-07-07 11:53:58 +02:00
Dustin Brody 68be98cafa
More 0.8.0 updates (#311)
* replace BeaconState.finalized_{epoch,root} with BeaconState.finalized_checkpoint; rename get_delayed_activation_exit_epoch(...) to compute_activation_exit_epoch(...) and mark as 0.8.0; update get_churn_limit(...)/get_validator_churn_limit(...) to 0.8.0; update process_registry_updates(...) to 0.8.0

* update process_crosslinks(...) to 0.8.0; mark compute_start_slot_of_epoch(...) and get_committee_count(...) as 0.8.0

* mark Fork, is_slashable_validator(...), and get_beacon_proposer_index(...) as 0.8.0

* rename LATEST_SLASHED_EXIT_LENGTH to EPOCHS_PER_SLASHINGS_VECTOR; update process_slashings(...) to 0.8.0; remove pointless type conversion warning in get_previous_epoch(...)

* convert remaining references to finalized_epoch to finalized_checkpoint.epoch

* update slash_validator(...) to 0.8.0; mark inital value, Gwei, and time constants as 0.8.0; mark hash(...) and processBlockHeader(...) as 0.8.0

* rename WHISTLEBLOWING_REWARD_QUOTIENT to WHISTLEBLOWER_REWARD_QUOTIENT; rename LATEST_ACTIVE_INDEX_ROOTS_LENGTH to EPOCHS_PER_HISTORICAL_VECTOR (randao will also get merged into this); remove get_active_index_root(...); mark time parameter, signature domain types, and max operations per block constants as 0.8.0; update rewards and penalties constants to 0.8.0

* update is_valid_indexed_attestation(...) to 0.8.0; mark process_slot(...) as 0.8.0

* replace BeaconState.{current,previous}_justified_{epoch,root} with BeaconState.{current,previous}_justified_checkpoint
2019-07-05 08:30:05 +00:00
Dustin Brody 5198930503 add start_epoch consistent with rest of crosslink 2019-07-03 10:41:42 +03:00
Dustin Brody 8a5e5334d6 re-implement attestation checking in 0.8.0 form and rename relevant function from checkAttestation(...) to spec's process_attestation(...), though there's a slight difference in how they're factored; remove a couple of pointless type conversion warnings; rename validate_indexed_attestation(...) to is_valid_indexed_attestation(...); remove verify_bitfield(...) not in 0.8.0; update AttestationData to 0.8.0 by using Checkpoint data structure; rename MAX_CROSSLINK_EPOCHS to MAX_EPOCHS_PER_CROSSLINK 2019-07-03 10:41:42 +03:00
Dustin Brody d7905351eb update get_crosslink_committee(...) to 0.8.0 and fix https://github.com/status-im/nim-beacon-chain/issues/307 2019-07-02 19:52:39 +03:00
Zahary Karadjov bf037eab5e
Tentative bugfix: Reset testnet was producing a genesis state without validators 2019-07-01 16:20:55 +03:00
Dustin Brody 95dea4c165 mark get_attestation_data_slot(...), ProposerSlashing, AttesterSlashing, DepositData, BeaconBlock, Eth1Data, AttestationDataAndCustodyBit, and is_slashable_attestation_data(...) as 0.8.0; add Checkpoint object type 2019-07-01 14:44:57 +03:00
Dustin Brody fa39eb82c4 do most of 0.8.0 BeaconState updating, in particular field renamings, re-orderings, etc; rm now-unnecessary genesis block creation aspects, towards the near-stub version in current spec as mostfields are 0/default/etc values 2019-07-01 14:44:57 +03:00
Dustin Brody fb601bd5b8 rename BeaconState.validator_registry to BeaconState.validators 2019-07-01 14:44:57 +03:00
Dustin Brody 2ba14bc9af update get_previous_epoch(...) to 0.8.0 and mark compute_committee(...) as 0.8.0 2019-07-01 14:44:57 +03:00
Dustin Brody dc98ffb09b update Validator to 0.8.0; update/refactor deposit contract/etc to constants category; mark process_rewards_and_penalties(...) as 0.8.0 2019-07-01 14:44:57 +03:00
Dustin Brody 607a7de5eb rename get_epoch_start_shard(...) to 0.8.0 get_start_shard(...) 2019-07-01 14:44:57 +03:00
Dustin Brody e4321dc4ed Initial frozen phase 0/v0.8.0 spec (#305)
* rename slot_to_epoch(...) to compute_epoch_of_slot(...)

* rename aggregation_bitfield/custody_bitfield fields to aggregation_bits/custody_bits; rename convert_to_indexed(...) to get_indexed_attestation(...); mark BeaconBlockHeader as 0.8.0; update minimal preset MIN_ATTESTATION_INCLUSION_DELAY/time parameters in general to 0.8

* fix beacon node compilation; it used slightly different capitalizations of slot_to_epoch/slotToEpoch

* mark integer_squareroot(...), Deposit, VoluntaryExit, and Transfer as 0.8.0; rename get_epoch_start_slot(...) to compute_start_slot_of_epoch(...)

* add new Domain alias for uint64; rename bls_domain(...) to compute_domain(...), MAX_INDICES_PER_ATTESTATION to MAX_VALIDATORS_PER_COMMITTEE, and SignatureDomain to DomainType; update get_domain(...) to 0.8.0

* update/mark initiate_validator_exit(...) and Crosslink as 0.8.0; rename BeaconState.latest_block_roots -> BeaconState.block_roots; mark HistoricalBatch as 0.8.0
2019-07-01 09:53:42 +02:00
Dustin Brody 7356905f95
remove get_epoch_start_slot(...) kludge and update to work exactly as 0.7.1 specifies; remove unused get_attestation_participants_cached(...); update AttestationData to 0.7.1 (being the last data structure, I believe, remaining for such; remove potentially spurious/certainly not-in-spec assertion from compute_committee(...); fix state sim to work with new get_epoch_start_slot/AttestationData/etc setup where it can't stuff all shards' attestations from same slot in the same MIN_ATTESTATION_INCLUSION_DELAY rotating/circular buffer of seq[Attestation]s without more involved shuffling of shard/slot calculation order; fix attestation pool testing to be consistent with get_epoch_start_slot(...) (#302) 2019-06-29 09:17:24 +00:00
Mamy Ratsimbazafy 214860cb88 Split state_transition (#301) 2019-06-28 13:44:44 +00:00
Zahary Karadjov 0d37ed7a2a
Fix a problem for the bootstrap nodes
connectToNetwork is executed for the bootstrap node as well, but
it should not quit when peers are not yet present.
2019-06-28 14:55:21 +03:00
Jacek Sieka 1ce459bcdb
Merge commit 'b0d27b9' 2019-06-27 14:52:46 +02:00
Jacek Sieka b0d27b9acc
disambiguate IPPROTO_TCP 2019-06-27 14:52:32 +02:00
Dustin Brody 43a9e904cf
replace get_attestation_slot(...) with 0.7.1 get_attestation_data_slot(...) and update all callers; update AttestationData to be almost 0.7.1, except nonextistent slot field; update Crosslink and get_winning_crosslink_and_attesting_indices(...) to 0.7.1 (#297) 2019-06-25 11:17:08 +00:00
Zahary Karadjov 1eef531259
Attempt to fix the build on Windows 2019-06-25 07:00:31 +03:00
Zahary Karadjov 69f3095fac Remove temp debugging code; Quit if you fail to connect to the network 2019-06-24 19:38:19 +03:00
Zahary Karadjov 31baa77742 Restore building with chronicles_sinks=json 2019-06-24 19:38:19 +03:00
Zahary Karadjov 7d174b548a Switch the testnet1 back-end to libp2p_native
Further simplifications and removal of support for multiple network
back-ends per testnet.
2019-06-24 19:38:19 +03:00
Zahary Karadjov 0116bdaed7 temporary code to debug issues with the testnet bootstrap node 2019-06-24 19:38:19 +03:00
Dustin Brody c7e06374f4
Remove get_crosslink_committees_at_slot and fix research/state_sim (#291)
* migrate attestation pool tests from get_crosslink_committees_at_slot(...) to get_crosslink_committee(...)

* rm obsolete, unused get_crosslink_committees_at_slot_cached(...) and migrate tests/test_state_transition from get_crosslink_committees_at_slot(...) to get_crosslink_committee(...)

* migrate tests/testutil from get_crosslink_committees_at_slot(...) to get_crosslink_committee(...)

* use more pervasive caching infrastructure, initially of compute_committee; remove buggy (and per-index) shuffling to fix research/state_sim, which was noticing validators on multiple shard committees; rm now-unused specific get_attesting_balance_cached

* add some get_active_validator_index caching

* rm obsolete/unused get_attesting_indices_cached

* rm get_crosslink_committees_at_slot(...)

* some more 0.7 changes -- some of which can't be completed pending some data structure updates -- and shard handling changes to offset with epoch start shards
2019-06-24 09:21:56 +00:00
Zahary Karadjov c751285112
Implement all libp2p_native response codes as specified in the latest proposal 2019-06-24 05:34:22 +03:00
Zahary Karadjov 84afb77b27
Restore compilation with libp2p_native after the latest changes in the spec back-end 2019-06-24 05:34:22 +03:00
Zahary Karadjov 44fc05f864
Fix the build for chronicles_sinks=json 2019-06-24 05:34:22 +03:00
Zahary Karadjov 25494aa2fb
Better error message on incompatible validator data when resetting a testnet 2019-06-24 05:34:22 +03:00
Zahary Karadjov e228c2dbcb
Implement the even/odd request ID scheme; Handle more edge cases; Break the cyclic imports 2019-06-24 05:34:22 +03:00
Zahary Karadjov 96e2a02faf
Respect the port configuration and NAT setup when using LibP2P 2019-06-24 05:34:22 +03:00
Zahary Karadjov 15fdf78d6a
[LibP2P] Persistent network key for the bootstrap node 2019-06-24 05:34:22 +03:00
Zahary Karadjov 12e9367f78
Improved error handling; Simple test case for connecting 2 peers 2019-06-24 05:34:22 +03:00
Zahary Karadjov 877b22cfb8
Share more code between the libp2p backends 2019-06-24 05:34:21 +03:00
Zahary Karadjov 87601a5eae
Share more code between the libp2p backends 2019-06-24 05:34:21 +03:00
Zahary Karadjov 3b166be166
Restore compilation in RLPx mode; More libp2p progress 2019-06-24 05:34:21 +03:00
Zahary Karadjov e177d17762
Further simplifications 2019-06-24 05:34:21 +03:00
Zahary Karadjov c060c0fc5d
Simplified the 'spec' back-end by movign more logic into the shared DSL module 2019-06-24 05:34:21 +03:00
Zahary Karadjov ea4690d567
A new P2P backend implementing the libp2p networking according to the official spec 2019-06-24 05:34:21 +03:00
Zahary Karadjov f4a96bc3f3
[WIP] Restore the Lib2P2 builds and implement the latest wire spec
Depends on https://github.com/status-im/nim-eth/pull/54
2019-06-24 05:34:21 +03:00
Dustin Brody 13381c8fea
update everything https://github.com/ethereum/eth2.0-specs/releases/tag/v0.7.1 leaves unchanged from v0.7.0 to v0.7.1 (#290) 2019-06-18 15:34:30 +00:00
Dustin Brody e9180c490e
more 0.7.0 updates (#287)
* update Deposit and BeaconBlockHeader to 0.7.0

* update get_total_balance(...) and get_base_reward(...) to 0.7.0

* rename initial value constant BLS_WITHDRAWAL_PREFIX_BYTE to BLS_WITHDRAWAL_PREFIX

* mark minimal presets as entirely migrated to 0.7.0; update process_rewards_and_penalties(...) to 0.7.0

* specify that targeting 0.7.0; adopt 0.7.0 refactoring of bls_domain/get_domain to update get_domain(...) to 0.7.0; mark that mainnet constant presets and block header processing are 0.7.0-consistent; update process_justification_and_finalization to 0.7.0 to fix potential underflow errors

* address mratsim's comment to copy whole spec comment
2019-06-14 16:21:04 +00:00
Dustin Brody 2a6e64d3f9
0.7.0 updates including semantic changes (#285)
* update get_previous_epoch(...)

* update BeaconBlock to 0.7.0 by renaming previous_block_root field to parent_root
2019-06-14 13:50:47 +00:00
Dustin Brody e2496567b3 More safe 0.7.0 updates (#283)
* complete marking unchanged-in-0.7.0 beaconstate functions

* complete marking unchanged-in-0.7.0 spec refs functions in helpers.nim and time.nim: get_randao_mix(...), bytes_to_int(...), and description of beacon chain processing apropos time

* complete remaining feasible 0.7.0 updates which lack externally visible effect
2019-06-14 09:50:14 +02:00
Dustin Brody 6357bc2a98 0.7.0 updates which don't change semantics; mostly comment changes (#281)
* 0.7.0 updates which don't change semantics; mostly comment changes

* mark get_attesting_indices, bls_verify, ProposerSlashing, BeaconBlockBody, deposit contract constants, state list length constants, compute_committee, get_crosslink_committee, is_slashable_attestation_data, processAttesterSlashings as 0.7.0; rename BASE_REWARD_QUOTIENT to BASE_REWARD_FACTOR

* complete marking unchanged-in-0.7.0 datatypes as such; a few more have trivial field renamings, etc
2019-06-13 09:40:58 +02:00
Dustin Brody d400650eeb
0.6.3 updates (#279)
* update IndexedAttestation, verify_slashable_attestation/verify_indexed_attestation, and attester slashing processing to 0.6.3

* rm debug scaffolding

* rename Attestation.aggregate_signature -> Attestation.signature; convert various references to AttestationData.slot to get_attestation_slot; implement convert_to_indexed; update checkAttestation and processAttestations to 0.6.3; remove spurious assertion in beacon node related to invalid attestations

* replace 0.5 get_winning_root_and_participants with 0.6 get_winning_crosslink_and_attesting_indices; update process_crosslinks to 0.6.3

* mark both remaining 0.6.0 spec implementations as 0.6.3

* clear out all remaining spec version 0.6.1 refs

* GENESIS_SLOT and GENESIS_EPOCH are 0

* rm 0.5 get_attestation_participants in favor of 0.6 get_attesting_indices

* address mratsim's comment

* time can be equal to genesis

* fix invalid block assertions; those were essentially spurious

* allow toBeaconTime to handle time before genesis (in accordance with now(...) which states time can exist before GENESIS)
2019-06-12 07:48:49 +00:00
Yuriy Glukhov 10c7920b27 Proto changes to facilitate backward sync (#271)
* Proto changes to facilitate backward sync

* Update to latest spec types in sync proto

* Use blockpool for more straightforward block headers collection

* Added BlockPool.getRef

* Update beacon_chain/sync_protocol.nim

Co-Authored-By: Jacek Sieka <arnetheduck@gmail.com>
2019-06-10 07:13:53 -04:00
Dustin Brody 252819a84d
bump everything that's not one of the 3 or so specific changes in 0.6.3 from 0.6.2 to 0.6.3 if already at 0.6.2 (#277) 2019-06-05 10:25:30 +00:00
Dustin Brody 2e1515b107
0.6.2 updates (#275)
* update process_justification_and_finalization to 0.6.2; mark AttesterSlashing as 0.6.2

* replace get_effective_balance(...) with state.validator_registry[idx].effective_balance; rm get_effective_balance, process_ejections, should_update_validator_registry, update_validator_registry, and update_registry_and_shuffling_data; update get_total_balance to 0.6.2; implement process_registry_updates

* rm exit_validator; implement is_slashable_attestation_data; partly update processAttesterSlashings

* mark HistoricalBatch and Eth1Data as 0.6.2; implement get_shard_delta(...); replace 0.5 finish_epoch_update with 0.6 process_final_updates

* mark increase_balance, decrease_balance, get_delayed_activation_exit_epoch, bls_aggregate_pubkeys, bls_verify_multiple, Attestation, Transfer, slot_to_epoch, Crosslink, get_current_epoch, int_to_bytes*, various constants, processEth1Data, processTransfers, and verifyStateRoot as 0.6.2; rm is_double_vote and is_surround_vote

* mark get_bitfield_bit, verify_bitfield, ProposerSlashing, DepositData, VoluntaryExit, PendingAttestation, Fork, integer_squareroot, get_epoch_start_slot, is_active_validator, generate_seed, some constants to 0.6.2; rename MIN_PENALTY_QUOTIENT to MIN_SLASHING_PENALTY_QUOTIENT

* rm get_previous_total_balance, get_current_epoch_boundary_attestations, get_previous_epoch_boundary_attestations, and get_previous_epoch_matching_head_attestations

* update BeaconState to 0.6.2; simplify legacy get_crosslink_committees_at_slot infrastructure a bit by noting that registry_change is always false; reimplment 0.5 get_crosslink_committees_at_slot in terms of 0.6 get_crosslink_committee

* mark process_deposit(...), get_block_root_at_slot(...), get_block_root(...), Deposit, BeaconBlockHeader, BeaconBlockBody, hash(...), get_active_index_root(...), various constants, get_shard_delta(...), get_epoch_start_shard(...), get_crosslink_committee(...), processRandao(...), processVoluntaryExits(...), cacheState(...) as 0.6.2

* rm removed-since-0.5 split(...), is_power_of_2(...), get_shuffling(...); rm 0.5 versions of get_active_validator_indices and get_epoch_committee_count; add a few tests for integer_squareroot

* mark bytes_to_int(...) and advanceState(...) as 0.6.2

* rm 0.5 get_attesting_indices; update get_attesting_balance to 0.6.2

* another tiny commit to poke AppVeyor to maybe not timeout at connecting to GitHub partway through CI: mark get_churn_limit(...), initiate_validator_exit(...), and Validator as 0.6.2

* mark get_attestation_slot(...), AttestationDataAndCustodyBit, and BeaconBlock as 0.6.2
2019-06-03 10:31:04 +00:00
Jacek Sieka 16fb9e8d11 Add support for dealing with overlapping attestations (#262) 2019-06-03 08:26:38 +00:00
Dustin Brody 6a47ca3c46
More 0.6.1 updates (#274)
* implement get_churn_limit from 0.6.1; update initiate_validator_exit and get_previous_epoch to 0.6.1; remove obsoleted/non-spec reduce_balance, replaced by decrease_balance; mark BeaconBlock as 0.6.1

* rename get_block_root to get_block_root_at_slot and introduce epoch-oriented get_block_root; implement get_attestation_slot, get_matching_source_attestations, get_matching_target_attestations, get_matching_head_attestations, 0.6.1 get_attesting_indices, get_unslashed_attesting_indices, get_attestation_deltas, process_rewards_and_penalties; rm get_inactivity_penalty

* update Validator and processVoluntaryExits to 0.6.1; rm obsolete inclusion_slots/inclusion_distances

* rm removed activate_validator; mark DepositData, misc values, Gwei values, Randao processing, state caching as 0.6.1; update GENESIS_SLOT to 64, since it doesn't quite yet work at 0

* mark BeaconBlockHeader as 0.6.1; update BeaconBlockBody to 0.6.1; rename WHISTLEBLOWER_REWARD_QUOTIENT to WHISTLEBLOWING_REWARD_QUOTIENT; update slash_validator to 0.6.1

* implement 0.6.2 is_slashable_validator; update processProposerSlashings to 0.6.2; mark state caching as 0.6.2

* mark get_total_active_balance and process_slashings as 0.6.2; update get_base_reward to 0.6.2

* rm prepare_validator_for_withdrawal, process_exit_queue; mark mainnet misc constants as 0.6.2; update process block header to 0.6.2

* address mratsim's code review comment
2019-05-29 10:08:56 +00:00
Yuriy Glukhov f679677995 Fixed signing_root hack (#272) 2019-05-28 08:44:25 +00:00
Mamy Ratsimbazafy 6b1f0e816c
Introduce constant presets (#269)
* Initial move constants to mainnet preset

* Bump fixtures to include shuffling with minimal preset

* Add minimal preset with shuffling test

* Add minimal preset to the full test suite

* use preset() everywhere
2019-05-27 14:48:13 +02:00
Dustin Brody 46cfce652d
initial new get_beacon_proposer_index and dependencies (get_shuffled_index, get_shard_delta, get_epoch_start_shard, get_crosslink_committee) (#266)
* initial new get_beacon_proposer_index and dependencies (get_shuffled_index, get_shard_delta, get_epoch_start_shard, get_crosslink_committee)

* more 0.6.1 updates

* clean up debugEchos and fix validator keygen to create correct indices
2019-05-23 11:13:02 +00:00
Zahary Karadjov 8da71715eb Bump the beacon_node version; Make the simulation script compatible with macOS
The shell expansion syntax used in the run_node script was not handled properly
by the bash shell shipped with macOS. In particular, it was not able to handle
range expressions involving interpolated vars:

cp foo/{$FIRST_INDEX..$LAST_INDEX} ...
2019-05-14 16:46:43 +03:00
Yuriy Glukhov a0d4c3432f Crude request parallelization 2019-05-11 22:47:53 +03:00
Mamy Ratsimbazafy e5047d93a4 [Tests] SHA256 + official shuffling vectors (#263)
* Change digests to SHA2-256 (from Keccak256)

* Fix sha256 digest initialization - pass shuffling test

* Add comments + remove old shuffling tests

* Remove stale tree hashing tests

* Small toOpenArray optim + notes on in-place hashing

* Revert "Revert "Update test repo""

This reverts commit f385467cd3.

* Revert "Revert "Fix SIGFPE on shuffling for 0""

This reverts commit 226d380f87.

* Revert "Revert "Implement shuffling test (pending typedesc and shuffling algo fix)""

This reverts commit 813cb6fbb8.

* withEth2hash templates now needs ctx.init()

* Use init-update-finish to avoid burnMem
2019-05-10 08:14:01 +00:00
Dustin Brody d977c96ae1 0.6.1 - signing_root, deposits, validator indices and committee count (#261)
* signed_root -> signing_root

* implement new process_deposit; update timing parameters to 0.6.1; update Deposit to 0.6.1 and remove DepositInput

* update get_active_validator_indices and get_epoch_committee_count to 0.6.1; rm get_current_epoch_committee_count, get_previous_epoch_committee_count, and get_next_epoch_committee_count; bump state_sim default validator count a bit more

* re-introduce 0.5.1-ish get_active_validator_indices, get_epoch_committee_count as scaffolding for still-0.5.1ish shuffling
2019-05-09 14:27:37 +02:00
Mamy André-Ratsimbazafy 226d380f87
Revert "Fix SIGFPE on shuffling for 0"
This reverts commit 8cfbfdf565.
2019-05-09 12:07:09 +02:00
Mamy André-Ratsimbazafy 8cfbfdf565
Fix SIGFPE on shuffling for 0 2019-05-09 12:01:01 +02:00
Jacek Sieka 17fa499755 cache state root a bit more aggressively (#260)
needed to be able to run state sim, too slow otherwise
2019-05-04 14:10:45 +00:00
Jacek Sieka bd4893d2e8 head block selection fixes (#259)
* track justified and head blocks correctly in presence of blank slots
* fix new block state root
* keep list of viable heads instead of walking children
* disable json bitfield deserializer (needs a corresponding serializer)
* fix handshake best & finalized root
2019-05-01 12:19:29 +03:00
Dustin Brody 46b4154ce8 begin 0.6.0 (#256)
* begin 0.6.0: new get_domain/increase_balance/reduce_balance, BeaconState.validator_balances -> BeaconState.balances, some renamed constants, transaction processing changes, SlashableAttestation field name changes, 0.6.0 get_beacon_proposer_index always uses given state's slot, update tests subrepo

* mark get_bitfield_bit/bls_verify_multiple/stat-list-lengths/is_active_validator/is_surround_vote/slot_to_epoch/int_to_bytes/etc as unchanged in 0.6.0; rm Eth1DataVote/maybe_reset_eth1_period and thus adjust expected tree hash test results

* mark verify_bitfield/bls_verify/deposit-contract/VoluntaryExit/PendingAttestation/Historicalbatch/Fork as 0.6.0; update DOMAIN_BEACON_BLOCK to DOMAIN_BEACON_PROPOSER

* update Crosslink to 0.6.0 (also requires tree hashing test result change, so isolate in individual commit)

* mark verify_merkle_branch/get_delayed_activation_exit_epoch/ProposerSlashing/Attestation/AttestationDataAndCustodyBit/hash/integer_squareroot/get_epoch_start_slot/is_double_vote/get_randao_mix/generate_seed as 0.6.0; update reward and penalty quotients; SlashableAttestation -> IndexedAttestation; rm get_fork_version; ATTESTATION_INCLUSION_REWARD_QUOTIENT -> PROPOSER_REWARD_QUOTIENT
2019-04-29 10:48:30 -06:00
Mamy Ratsimbazafy 5784b2d7f7
Full EF state tests parsing (#242)
* Load full state test + Add json-serialization for Bitfield

* Implement empty_block_transition test (but test is missing the end state)

* Use the provided empty block instead of mocking one

* Add failing block signing test

* Tests that can't be passed now are now "for information" + indent the "information"/hash given
2019-04-29 18:10:01 +02:00
Jacek Sieka c7e6d39279 fix statedata fields not being updated correctly (#255) 2019-04-29 11:14:22 +03:00
Yuriy Glukhov 61ec6fa167 Move incoming block from unresolved to pending if it can't be applied (#253)
* Move resolved block from unresolved to pending if it can't be applied

* Renamed unresolved to missing
2019-04-26 10:38:56 -06:00
Yuriy Glukhov ddcdb63430
Deduplicate blocks in getAncestorBlocks (#254)
* Deduplicate blocks in getAncestorBlocks

* Limit ancestor blocks in getAncestorBlocks
2019-04-26 09:21:46 +03:00
Ștefan Talpalaru 53012b9a25
NAT traversal 2019-04-18 15:48:17 +02:00
Yuriy Glukhov b916b1e3e4
Slightly more validations 2019-04-10 15:56:22 +03:00
Yuriy Glukhov 54f894fc7e
Chunked sync second try
This reverts commit cdf9baf5e6.
2019-04-10 15:35:52 +03:00
Zahary Karadjov cdf9baf5e6 Revert "Chunked sync"
This reverts commit c84d930304.
2019-04-09 02:19:33 +03:00
Zahary Karadjov ba48fcd165 Slightly better error messages during testnet upgrades; Upgrade testnet0 to 1000 validators 2019-04-09 00:36:06 +03:00
Yuriy Glukhov c84d930304 Chunked sync 2019-04-09 00:18:53 +03:00
Jacek Sieka 9a2f3176f7 guard node.state (#237)
* fix node.state being used unsafely across async callbacks (fixes #230)
* node.state -> node.stateCache
* fix attestation validation not to use asserts
* re-enable attestation pool tests
* prepare for resizing validator registry
2019-04-06 09:46:07 +02:00
Dustin Brody c53de3e550 Caching updates/refactoring & state_sim defaults updates (#235)
* rm now-superceded shuffling cache (shuffling is only called from get_crosslinks), which was badly architected due to trying to exist in state; rm one more vestige of previous light-client regime (one more to go, from datatypes)

* fix wrong shuffling list size (active validator size, not validator size) to make consistent with 0.5.1 (will be inconsistent with testnet0); fix typo and change defaults in state_sim

* doAssert a couple of constant relationships necessary to avoid underflow; rm non-spec, unused helper function get_new_recent_block_roots

* refactor separate crosslink_committee_cache and winning_root_participants_cache(s) into StateData object; remove last vestige of previous shuffling cache

* separate out caching parts of StateData to new StateCache object
2019-04-05 08:18:13 -06:00
Jacek Sieka 605dd0a0e9 Some speedups (#226)
* ssz: avoid memory allocations

* a bit fishy with the 32-item stack.. this should be a smallvector

* digest: avoid another burnmem

* avoid a few allocations by using iterator
2019-04-03 15:46:22 +00:00
Yuriy Glukhov ff7adcefbd Persist head block 2019-04-02 15:35:09 +03:00
Ștefan Talpalaru f6eb6af310
downloadFile(): print command in the error message
so people can figure out that they don't have "curl" installed
2019-04-01 20:41:35 +02:00
zah e16d8faee2 Switch to a two-steps build process (#228) 2019-03-29 10:44:19 -06:00
Zahary Karadjov 6689d0a642 Remove some deprecation warnings 2019-03-29 16:45:38 +02:00
Zahary Karadjov 8e0025035c all validators are free for picking by default 2019-03-29 15:38:59 +02:00
Dustin Brody 6cbf355c5a state_transition part of get_attesting_indices/get_attesting_indices_cached perf tweaks 2019-03-28 23:31:30 +02:00
Dustin Brody d74fa2027a remove redundant conversions 2019-03-28 23:31:30 +02:00
cheatfate 3117a4481e Remove unneeded initialization. 2019-03-28 23:24:50 +02:00
cheatfate 6f4c7b775c Remove one-line hash to avoid burnMem.
Remove unnecessary copyMem()s.
2019-03-28 23:24:50 +02:00
Jacek Sieka 201b079fe3 avoid bad blocks by filtering included attestations
* fix epoch violation during initial attestation validation
* fill in attestation fields (finalizes again!)
2019-03-28 21:10:08 +02:00
Zahary Karadjov 44d9f7d6c9 Use proper RPC calls when obtaining missing blocks 2019-03-28 16:54:50 +02:00
Zahary Karadjov f23ee4ccae Fix a wrong assert 2019-03-28 16:54:50 +02:00
Zahary Karadjov e0d8bf9e99 Logging in preInit works just fine locally; pushing to test on the CI 2019-03-28 16:54:50 +02:00
Zahary Karadjov 6d4470a8c8 Simplified gossipsub.broadcast further 2019-03-28 16:54:50 +02:00
Zahary Karadjov dd72218f48 Integrate `traceAsyncErrors`
You'll need the latest version of nim-eth
2019-03-28 16:54:50 +02:00
Yuriy Glukhov 497f88c20d Increase minPeers in PeerPool to PeerPool default (10) 2019-03-28 16:35:17 +02:00
Jacek Sieka f9e0418b5b fetch multiple blocks at a time
* avoid crash on invalid block production (fixes #209)
* fetch blocks every second, roughly
* fix start.sh param order
* run beacon node sim at slower pace
2019-03-28 10:42:16 +02:00
Dustin Brody 58b6174654 smallish speedup -- also use cache in get_crosslink_deltas 2019-03-28 10:40:20 +02:00
Dustin Brody 6f481f8812 another ~2x speedup in epoch processing 2019-03-28 10:40:20 +02:00
Dustin Brody e1ba820486 clean up caching 2019-03-28 10:40:20 +02:00
Dustin Brody d84aa45a22 share crosslink committees cache across process_crosslinks and compute_crosslinks_deltas 2019-03-28 10:40:20 +02:00
Dustin Brody 2c566df411 rm debug message 2019-03-28 10:40:20 +02:00
Dustin Brody ced4dbe859 4x speedup on epoch processing for 64 validators 2019-03-28 10:40:20 +02:00
Jacek Sieka 3ad6053298 Use block, slot tuple to idenfiy fork
this is the beginning of tracking block-slots more precisely, so we can
the justification epoch slot bug.

* avoid asyncDiscard (swallows assertions)
* fix attestation delay
* fix several state root cache bugs
* introduce workaround for genesis epoch spec issue
2019-03-28 10:32:09 +02:00
Zahary Karadjov 92c38d6d48 Don't log the backwards sync requests 2019-03-27 18:32:33 +02:00
Zahary Karadjov 2efcffe6ac Fix an issue for completely fresh installations 2019-03-27 18:29:02 +02:00
Zahary Karadjov 406df9d44f More chatty syncing 2019-03-27 18:13:39 +02:00
Zahary Karadjov b3d23c052c Switch to 30 seconds slot duration; UX improvements
* On the testnet site, the first N instead of the last N validators
  will be considered reserved for users

* The number of validators is consistently named `totalValidators`
  in all commands

* Proper keys are generated by default in validator_keygen
2019-03-27 18:13:39 +02:00
Zahary Karadjov a3ce9d17ec Expose the default port 2019-03-27 18:13:39 +02:00
Zahary Karadjov 6629897567 We don't need the more complicated port setup anymore 2019-03-27 18:13:39 +02:00
Jacek Sieka edd9826464
some ignores 2019-03-26 19:41:34 -06:00
Jacek Sieka 81c66fbce0
use signed_root as canonical block root (#211)
* no need to pass prev_block_root when updating state
* some Slot fixes
* fix `hash_tree_root` for Slot, Epoch
* detect missing hash_tree_root type support
* remove some <0.5 state checks
2019-03-26 19:32:35 -06:00
Zahary Karadjov c0fa229254 Improved error handling in the networking code 2019-03-26 17:40:20 +02:00
Zahary Karadjov d94d4f2606 Dumbed down the validator key import procedure
The previous idempotent scheme proved to be too slow, making it
harder to quickly re-assign validators on the servers.
2019-03-26 17:37:31 +02:00
Zahary Karadjov 5b1caaff41 more detailed error message when you connect to an incompatible testnet 2019-03-26 17:37:31 +02:00
Mamy André-Ratsimbazafy 8273345a2c fix comment 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy 8b5e18b7a4 Magic fix to #209 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy 2fafa8b1a1 Add TODO comment about loggable stacktraces and open https://github.com/status-im/nim-chronicles/issues/47 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy 57c693c88f Only check for epoch underflow just before peer communication
During state processing epochs can underflow (hence why we have a non-zero arbitrary genesis epoch)
2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy e189c1d3cd Crash before making attestation when replaying state pass genesis 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy d1f75cdfef Improve distinction between debugging stacktraces and crash/exception stacktraces 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy e8160c2126 more logs to investigate epoch underflow/overflow https://github.com/status-im/nim-beacon-chain/issues/207 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy 9b520d3949 address comment 2019-03-26 17:35:09 +02:00
Mamy André-Ratsimbazafy 808c5edfc0 Add more fork choice logs 2019-03-26 17:35:09 +02:00
Jacek Sieka e7b36c4389 add beacon clock and rework timing
* have regular slot tick, decide on work to do lazily
* use beacon clock instance that will eventually sync time
* skip work when falling behind
* add one more state cache for justified state (since that's used in
fork choice)
* remove old sync cruft
* single timer per slot, serializing block production and attestation
2019-03-26 13:21:48 +02:00
Yuriy Glukhov e4c10a31c9 Use latest state from the DB in sync. Fixed off by one in block roots (#203) 2019-03-25 14:52:23 -06:00
Zahary Karadjov 70cb84ef93 Fix the build with the latest nim-eth 2019-03-25 19:57:05 +02:00
Jacek Sieka 1b0e67c88c
ssz: update to 0.5.1:ish (#202)
* ssz: update to 0.5.1:ish
* slightly fewer seq allocations
* still a lot of potential for optimization
* fixes #174
* ssz: avoid reallocating leaves (logN merkle impl)
2019-03-25 10:46:31 -06:00
Dustin Brody 6b1527ccd5 Almost final 0.5.1 spec updates (#201)
* update processExits to 0.5.1

* mark process_deposit, get_empty_block as 0.5.1; update get_genesis_beacon_state to 0.5.1; implement processDeposits

* hotfix via https://github.com/ethereum/eth2.0-specs/pull/821
2019-03-24 12:21:23 -06:00
Dustin Brody 9f55b4646c
More 0.5.1 spec updates (#195)
* rm gone-in-0.5.0 Proposal, verifyBlockSignature, and slot check which moved to spec function processBlockHeader

* mark get_attestation_participants and get_epoch_committee_count as 0.5.1; finish updating processAttestations to 0.5.1; add kludgy workaround for bug relating to get_winning_roots_etc using crosslink_data_root as index when we have that as ZERO_HASH for all leading to it confusing attesters on different shards; rm BeaconState.latest_attestations, which splits into previous_epoch_attestations and current_epoch_attestations

* Fix CI due to removed latest_attestations field
2019-03-22 18:33:12 +00:00
Zahary Karadjov 0a027e410a Address review concerns and make some steps towards json logging 2019-03-22 17:04:17 +02:00
Zahary Karadjov 06301458bf Don't add the 'node' log property when connecting to the testnet 2019-03-22 17:04:17 +02:00
Mamy André-Ratsimbazafy ca49ae9b5a
Report connected peers before fork choice 2019-03-22 12:33:10 +01:00
Yuriy Glukhov 894816fa0c Fixed crash caused by findAncestorBySlot (#196)
closes #193
2019-03-22 11:57:19 +01:00
Dustin Brody 94c3b005fa
https://github.com/ethereum/eth2.0-specs/pull/816 hotfix (#194) 2019-03-21 16:02:50 +00:00
Dustin Brody 1bb2859aca
implement bulk of 0.5.0 state transition function (#192) 2019-03-21 15:31:36 +00:00
Jacek Sieka ee89ef1c79
cleanups (#189)
* add simple bitfield type (fixes #19)
* fix bit endianess to match spec
* consolidate attestation and block logging
* cruft cleanup
* run optimizer on tests, speeds up build
2019-03-20 14:01:48 -06:00
Dustin Brody 13c7f7fcec begin refactoring 0.5.0 epoch transition (#185)
* begin refactoring 0.5.0 epoch transition at justification and finalization

* update processTransfers to 0.5.0 spec

* update get_crosslink_committees_at_slot to 0.5.0 except for noted workaround for pre-0.5.0 spec bug related to epoch boundary conditions, to be flipped in more coordinated way with other epoch processing changes

* mark ejection processing as 0.5.0

* 0.5.0 epoch update finalization

* rm BEACON_CHAIN_SHARD_NUMBER to complete updating miscellaneous constants to 0.5.0

* mark verify_slashable_attestation as 0.5.0

* update process_attester_slashing to 0.5.0

* update process_slashings(...) to 0.5.0 and mark process_exit_queue as 0.5.0

* refactor epoch processing by implementing 0.5.0's update_registry_and_shuffling_data(...)
2019-03-19 20:19:46 -06:00
Zahary Karadjov 6d4284877d Tentative fix for self-connection discovery issues 2019-03-20 03:18:59 +02:00
Zahary Karadjov 6ec8ffe0ff Much better error reporting on snapshot deserialization problems 2019-03-20 02:05:10 +02:00
Zahary Karadjov b454501595 handle missing validators dir during key import 2019-03-19 23:04:22 +02:00
Zahary Karadjov 42ca6afae2 Helpful scripts for resetting the testnets 2019-03-19 21:50:22 +02:00
Zahary Karadjov 6a35d3584d Automate the creation of the network metadata files
With these changes, running a simulation is very close to running
an actual testnet. Some checks have been added in the client to
make sure you are not connecting to an incompatible network (e.g.
a network running with a different number of shards).
2019-03-19 20:27:54 +02:00
Jacek Sieka 8bab6fd51f
Add TODO for local bind ip 2019-03-19 11:19:27 -06:00
Jacek Sieka d7f972a47e allow setting advertised external ip
* detect it in start.sh
2019-03-19 10:50:09 +02:00
Dustin Brody 2bc632c29b
AttestationData.justified epoch -> AttestationData.source_epoch and AttestationData.latest_crosslink -> AttestationData.previous_crosslink; implement verify_merkle_branch and update process_deposit to use it; rm BeaconState.batched_block_roots; Transfer.{from, to} -> Transfer.{sender, recipient}; update Fork data type and code to use bytes4; implement int_to_bytes8 (close to metaprogramming time); adjust signed_root and all callers to use 0.5.0 calling signature; processDepositRoot -> processEth1Data (#182) 2019-03-18 15:42:42 +00:00
Zahary Karadjov 6bb38095c9 Command-line and data storage handling for testnets 2019-03-18 12:45:29 +02:00
Dustin Brody 798197879b
Revert "AttestationData.justified epoch -> AttestationData.source_epoch and AttestationData.latest_crosslink -> AttestationData.previous_crosslink; implement verify_merkle_branch and update process_deposit to use it; rm BeaconState.batched_block_roots; Transfer.{from, to} -> Transfer.{sender, recipient}; update Fork data type and code to use bytes4; implement int_to_bytes8 (close to metaprogramming time); adjust signed_root and all callers to use 0.5.0 calling signature; processDepositRoot -> processEth1Data"
This reverts commit 18f1d000a1.
2019-03-17 15:24:50 -07:00
Dustin Brody 18f1d000a1
AttestationData.justified epoch -> AttestationData.source_epoch and AttestationData.latest_crosslink -> AttestationData.previous_crosslink; implement verify_merkle_branch and update process_deposit to use it; rm BeaconState.batched_block_roots; Transfer.{from, to} -> Transfer.{sender, recipient}; update Fork data type and code to use bytes4; implement int_to_bytes8 (close to metaprogramming time); adjust signed_root and all callers to use 0.5.0 calling signature; processDepositRoot -> processEth1Data 2019-03-17 15:17:40 -07:00
Dustin Brody f36c2d86dc begin 0.5.0 spec update (#179)
* begin 0.5.0 spec update: parent_root -> previous_block_root, BeaconState.justified_epoch -> BeaconState.current_justified_epoch, DOMAIN_PROPOSAL -> DOMAIN_BEACON_BLOCK, temporarily rename BeaconBlockHeader to BeaconBlockHeaderRLP to allow for gradual re-merging without disrupting RLP; mark a few unchanged functions and data types, implement get_temporary_block_header/get_empty_block/should_update_validator_registry/processBlockHeader/cacheState; update a few others

* a dozen or so more trivial mostly comment changes, finding more unchanged parts of 0.5.0

* several more trivial changes; goal is to reduce noise around the upcoming substantial changes (epoch processing, etc)
2019-03-16 13:52:37 -06:00
Jacek Sieka 1cb8ae9004
state processing fixes (#177)
* remove some redundant state updates
* when attesting late, use correct state / head
* don't send out obsolete attestations
* don't propose obsolete blocks
* remove some more resundant state updates :)
* simplify block logging (experimental)
* document fork choice division
* fix some Slot / Epoch conversion warnings
2019-03-14 07:33:56 -06:00
Ștefan Talpalaru 9ff1eb4ac8
Merge branch 'master' of github.com:status-im/nim-beacon-chain 2019-03-14 00:05:12 +01:00
Ștefan Talpalaru de295619be
assert() -> doAssert() 2019-03-14 00:04:43 +01:00
Jacek Sieka b0f4034060
implement fork choice (#175)
* keep track of a finalized block
* keep track of all justified blocks
* use naive spec version of LMD ghost
* cache slot number and a few more things in BlockRef
* keep track of the latest vote of each validator
* depend less on the state of node.state (it's a cache, effectively)
2019-03-13 16:59:20 -06:00
Zahary Karadjov 562eafe124 Finish the Spec/Epoch types refactoring 2019-03-13 23:23:01 +02:00
Dustin Brody 1479bae22f implement distinct Epoch type and continue 0.4.0 spec update (#173)
* implement distinct Epoch type
* update two more spec 0.4.0 markers, leaving just deposit processing
2019-03-12 19:46:44 -06:00
Dustin Brody 50a0948bc0 Mostly convert to using distinct Slot type (#172) 2019-03-12 16:21:32 -06:00
Mamy André-Ratsimbazafy 6dea18c140
Squashed commit of the following:
commit bed6909f33
Author: Mamy André-Ratsimbazafy <mamy_github@numforge.co>
Date:   Mon Mar 11 12:22:35 2019 +0100

    Cleanup imports

commit 241f62ed8d
Author: Mamy André-Ratsimbazafy <mamy_github@numforge.co>
Date:   Mon Mar 11 12:06:00 2019 +0100

    Group beacon types to prevent recursive imports
2019-03-12 16:03:14 +01:00
Zahary Karadjov 5d186ef93c Fix the build of serialized_sized 2019-03-12 15:38:58 +02:00
Zahary Karadjov 0206496663 Fix a build error 2019-03-12 15:14:30 +02:00
Zahary Karadjov 903cb8a8b5 A LibP2P-based networking stack;
To enable it, comment out the 'withLibp2p' line in nim.cfg

The history was squashed in order to remove an accidentally
commited binary file.

Other changes:

* SSZ was adapted to use the common serialization framework

* gossibsup.subscribe is not using async handlers at the moment
  and this allowed me to simplify it
2019-03-12 13:36:55 +02:00
Dustin Brody 434ba5727f finish all but 3 items for 0.4.0 spec update (#167) 2019-03-12 10:59:43 +01:00
Yuriy Glukhov b587455e7b Use BlockPool for slot->blocks mapping (#166) 2019-03-11 09:38:36 -06:00
Dustin Brody 13de015038 more 0.4.0 spec updates: renamings, continue Proposal updates, and note several more functions which were verified not to have changed in 0.4.0 (#163) 2019-03-11 09:33:24 -06:00
Dustin Brody 89aa3ec53f
fix keccak256 digest comments 2019-03-09 12:34:08 -08:00
Jacek Sieka aec31b1126 Updates for finalizing chain! (#162)
* set epoch_boundary_root - chain finalizes!
* fix slotStart to offset GENESIS_SLOT
* work around bug that will be fixed by
https://github.com/ethereum/eth2.0-specs/pull/732
* compile with debug info (there was a GC-related crash in C land)
2019-03-09 04:23:14 +00:00
Dustin Brody 0b47acae37
some comment changes noting further 0.4.0-spec-update progress, but no meaningful semantic changes (#161) 2019-03-08 18:23:42 +00:00
Dustin Brody 1afdcda62d
spec version 0.4.0 update: shard_block_root -> crosslink_data_root; switch from validator.slashed_epoch -> validator.slashed approach; a couple assert -> doAssert; note fix for underflow-prone spec changes in checkAttestation; GENESIS_SLOT change from 2^63 to 2^32; refactor Proposer and ProposalSlashing structure; add signed_root; remove duplicate slashValidator (#159)
* spec version 0.4.0 update: shard_block_root -> crosslink_data_root; switch from validator.slashed_epoch -> validator.slashed approach; a couple assert -> doAssert; note fix for underflow-prone spec changes in checkAttestation; GENESIS_SLOT change from 2^63 to 2^32; refactor Proposer and ProposalSlashing structure; add signed_root; remove duplicate slashValidator

* re-apply shard_block_root -> crosslink_data_root

* remove incorrect humaneSlotNum

* add (run-time only, alas) sanity check on signed_root
2019-03-08 17:44:31 +00:00
Jacek Sieka 6bcefc0e42
verify blocks before storing in block pool / database (#158)
* fix state db lookup typo
* fix randao reveal slot when proposing blocks
* only store blocks that can be applied to a state
* store state at every epoch boundary (yes, needs pruning!)
* split out state advancement function when there's no block
* default state sim to 0.9 attestation ratio
2019-03-08 10:40:17 -06:00
Dustin Brody 72749f4d04
processEpoch 5x faster, total 2x faster, with 16k validators (#157)
* processEpoch 5x faster, total 2x faster, with 16k validators

* don't recompute active_validator_indices as much

* remove duplicate process_ejections function
2019-03-07 16:53:44 +00:00
Jacek Sieka 4d55cf8eea beacon node sim: various improvements (fixes #111) (#156)
* allow running more or fewer validators
* use deterministic key generation for tests to avoid exhausting system
RNG
* update README with simulator docs
* write the data of each validator to separate file, instead of a big
chainstart.json (makes it easier to run different validator counts)
2019-03-07 13:59:28 +00:00
Dustin Brody 5d0de00168 transform a few quadratic algorithms from processEpoch into linear or otherwise subquadratic algorithms (#155) 2019-03-06 08:21:13 -06:00
Dustin Brody b8402e9809
speed up epoch processing by caching shuffling results (#153)
* speed up epoch processing by caching shuffling results
2019-03-05 19:10:36 +00:00
Yuriy Glukhov 38f48753b2 Comments on #117 addressed (#154) 2019-03-05 07:55:09 -06:00
Yuriy Glukhov 21e4debda0
Merge pull request #117 from status-im/sync2
Initial full sync impl
2019-03-04 21:05:24 +02:00
Yuriy Glukhov 9ea0bf4326
Comments addressed 2019-03-04 13:50:26 +02:00
Dustin Brody 81d5becc7b Switch hash functions, prevent underflow & verify that shuffling works (#149)
* prevent balance underflow, per spec implementation note

* verify that permutation shuffing works: add YAML dependency to be used for ingesting test cases; switch from blake2 to keccak256 to match EF test cases; remove inefficient naive-spec-version of shuffling algorithm now that protolambda's can be tested directly
2019-03-01 17:50:01 -06:00
Yuriy Glukhov 4c31f50889
Comments addressed 2019-03-01 22:09:20 +02:00
Yuriy Glukhov bed6510da3
Initial full sync impl 2019-03-01 19:39:38 +02:00
Zahary Karadjov 68a39a21be Mysterious fix for the mysterious build issue 2019-03-01 18:51:37 +02:00
Dustin Brody 794f6aed89
fix https://github.com/status-im/nim-beacon-chain/issues/145 (#147) 2019-03-01 14:39:17 +00:00
Jacek Sieka ddb3a9cd94
Revert "Add some missing RLP serialization routines (#144)" (#146)
This reverts commit 859aa327cb.
2019-03-01 07:34:37 -06:00
zah 859aa327cb Add some missing RLP serialization routines (#144)
* Add some missing RLP serialization routines
2019-02-28 15:50:21 -06:00
Dustin Brody 1cfea240ed Complete scan through v0.3.0 spec (#141)
* Complete scan through v0.3.0 spec; remaining known discreptancy is in bls_verify_multiple

* switch remaining slot-misuses of get_active_validator_indices to epochs

* spec workaround

* finish first 0.3.0 pass by getting slow, naive version of bls_verify_multiple running
2019-02-28 15:24:43 -06:00
Jacek Sieka 125231d321
add initial block pool (#139)
* implement in-memory block graph
* store tail block in database
* resolve unknown parents by syncing them from peers
* introduce concept of resolved blocks and attestations - those that
follow minimal protocol rules
* update state head lazily
* log more stuff
* shortHash -> shortLog
* start 9/10 beacon nodes by default, last can be started manually
* see also #134
* fix start.sh epoch length
2019-02-28 15:21:29 -06:00
Mamy Ratsimbazafy 20f99db058 remove unused epoch template (#142) 2019-02-28 08:17:57 -06:00
Yuriy Glukhov 5b60c3b45f Enable discovery service regardless bootnodes are provided 2019-02-28 09:48:33 +02:00
Dustin Brody 34e6149f89
fix crosslink epoch refs to begin with GENESIS_EPOCH; more minor epoch processing tweaks/unstubbed-code (#137)
* fix crosslink epoch refs to begin with GENESIS_EPOCH; more minor epoch processing tweaks/unstubbed-code

* remove obsolete TODO and use humanEpochNum
2019-02-27 13:58:07 +00:00
Dustin Brody 0305432a87 implement and call process_slashings and process_exit_queue from processEpoch; update a few more epoch processing details to be consistent with 0.3.0; add rest of final upates per epoch (#136) 2019-02-26 10:54:08 +01:00
Dustin Brody 60e6a9fedc assert relationship between constants necessary to avoid underflow 2019-02-25 14:58:04 +00:00
Dustin Brody eda43b5b1c adjust copyright comment form 2019-02-25 14:58:04 +00:00
Dustin Brody 6dab867848 cleanup 2019-02-25 14:58:04 +00:00
Dustin Brody 8baab40ea6 much faster shuffling based on observations from https://github.com/protolambda/eth2-shuffle enough that it's not easy to get exact speedup numbers 2019-02-25 14:58:04 +00:00
Dustin Brody f77016af68 switch sense of bls_verify check 2019-02-25 14:58:04 +00:00
Dustin Brody de2cad0e12 switch from legacy pre-spec sum_effective_balances(...) to get_total_balance(...) 2019-02-25 14:58:04 +00:00
Dustin Brody a42601e29e initial epoch processing update switching from slot-oriented to epoch-oriented, using newer helper functions 2019-02-25 14:58:04 +00:00
Dustin Brody fb3bbfccaa small 0.3.0 adjustments; comment updates/clarifications; add Transfer processing 2019-02-25 14:58:04 +00:00
Yuriy Glukhov a62d25ab98
Merge pull request #131 from status-im/state-replay
initial state replay implementation
2019-02-22 18:04:48 +02:00
Dustin Brody 2d52d5cbfe convert some asserts to doAsserts to keep them in release mode builds; rename get_initial_beacon_state to get_genesis_beacon_state to track spec; switch target spec version to 0.3.0; switch references to penalize_validator to slash_validator/slashValidator to track spec; make some function returns safer by omitting 'return' (#132)
* convert some asserts to doAsserts to keep them in release mode builds; rename get_initial_beacon_state to get_genesis_beacon_state to track spec; switch target spec version to 0.3.0; switch references to penalize_validator to slash_validator/slashValidator to track spec; make some function returns safer by omitting 'return'

* 2x shuffling speedup by hoisting pivot calculations per https://github.com/protolambda/eth2-shuffle
2019-02-22 10:56:45 +01:00
Jacek Sieka 4c2d212781
updates
* comment on last finalized state
* use explicitly named proc's instead of type tricks
2019-02-21 15:38:26 -06:00
Jacek Sieka 1d9c91d230
cleanups
* add ancestor getter to db layer, and use
* remove head block call
* add trivial beacon chain db test
2019-02-21 11:20:50 -06:00
Jacek Sieka 2d307e2257
initial state replay implementation
* fix initial attestation pool on reordered attestations
* simplify db layer api
* load head block from database on startup, then load state
  * significantly changes database format
* move subscriptions to separate proc's
* implement block replay from historical state
* avoid rescheduling epoch actions on block receipt (why?)
* make sure genesis block is created and used
* relax initial state sim parameters a bit
2019-02-20 22:42:17 -06:00
Dustin Brody 1d13007627 More spec v0.3.0 notations and updates (#129)
* tighten get_active_index_root epoch assertion bound

* implement merkle_root

* rm process_penalties_and_exits; add slash_validator

* rm rm'd and unused constant
2019-02-20 14:35:27 -06:00
Dustin Brody 132f1147a2 spec 0.3.0 updates: EPOCH_LENGTH -> SLOTS_PER_EPOCH, SlotNumber -> Slot, EpochNumber -> Epoch (#127)
* spec 0.3.0 updates: EPOCH_LENGTH -> SLOTS_PER_EPOCH, SlotNumber -> Slot, EpochNumber -> Epoch
2019-02-19 19:33:58 -06:00
Jacek Sieka 4670d6c98a naive attestation pool, in preparation of fork choice integration (#125)
* move attestation pool to separate file
* combine attestations lazily when needed
* advance state when there's a gap while attesting
* compile beacon node with optimizations - it's tooo slow right now
* log when unable to keep up
2019-02-20 00:35:02 +01:00
Dustin Brody 1269e001ad get most 0.3.0 spec renamings (#126)
* get most 0.3.0 spec renamings
2019-02-19 17:07:56 -06:00
Jacek Sieka c64cd2f275 a few simple SlotNumber cleanups 2019-02-18 22:43:33 +02:00
Mamy Ratsimbazafy 8e48bac58b
[WIP] Fix some issues after introducing "humaneEpochNumber" (#118)
* Fix some issues after introducing "humaneEpochNumber" + use slot_duration to 4

* Address humane comments

* Fix comment
2019-02-18 16:58:34 +01:00
Jacek Sieka 0e9cc20cfe beacon node simplifications (#116)
* remove previous randao scheme code
* fix some epoch logging
* move genesis time selection to state generation
2019-02-15 17:33:32 +01:00
Dustin Brody d3f97c1d5c
Merge pull request #115 from status-im/nus
complete switching to v0.2.0 RANDAO scheme, with verification enabled
2019-02-15 10:57:14 +00:00
Dustin Brody 6bef36d418 remove more old RANDAO vestiges 2019-02-14 14:03:45 -08:00
Dustin Brody 8fbf87aa8c complete switching to v0.2.0 RANDAO scheme, with verification enabled 2019-02-14 13:41:04 -08:00
Jacek Sieka a71424d768
add more config options and log compile parameters (#114)
default sim runs with smaller chain now
2019-02-14 13:32:33 -06:00
Mamy Ratsimbazafy c2a52d7fc5 Allow faster slots (#112)
* Allow compile time configurable slot time
2019-02-14 13:15:10 +01:00
Dustin Brody 55ac8d337b some v0.2.0 updates and fixes a bug that should enable simulation to continue longer (#110)
* some v0.2.0 updates and fixes a bug that should enable simulation to continue longer
2019-02-14 04:16:50 +01:00
Dustin Brody 31a4c7255f more v0.2.0 spec updates (#107)
* switch to v0.2.0 shuffling scheme

* fix some >80 char lines; add get_total_balance helper function (used in epoch processing, etc); note intended removal of repeat_hash and users thereof; add some int_to_bytes and bytes_to_int helpers; more underflow comments; add Gwei type; adjust generate_seed to include additional entropy source; note egregious inefficiencies

* minor code cleanup

* move some general helper functions to helpers.nim and clarify little-endian comments
2019-02-13 11:26:32 +01:00
Mamy Ratsimbazafy b60fa131f1
Fix simulation regression due to custody_bitfield (#108) 2019-02-12 23:50:02 +01:00
Mamy Ratsimbazafy 899c8e5df9
Bitfield debugging asserts (#106)
* debugging asserts

* Ensure initialization of custody_bitfield for mock Attestations
2019-02-12 18:07:44 +01:00
Yuriy Glukhov 56a48cc60f Fixed epoch actions (#105)
* Fixed epoch actions

* debugging asserts

* Revert "debugging asserts"

This reverts commit 9a11f5a7d8.
2019-02-12 16:56:58 +01:00
Dustin Brody 433e2f78c2
Merge pull request #104 from status-im/oug
Spec updates for 0.2.0
2019-02-11 15:48:00 +00:00
Dustin Brody f1cbc4a9dc adjust indentation 2019-02-11 16:32:22 +01:00
Mamy Ratsimbazafy f7feb9a6c9
Delay validators and add more log in crosslink asserts (#103) 2019-02-11 16:29:21 +01:00
Dustin Brody b03ac3427f split out get_previous_epoch into helper function 2019-02-11 16:10:46 +01:00
Dustin Brody d44999abc5 latest_crosslink_root -> latest_crosslink; adjust checkAttestation; add SHUFFLE_ROUND_COUNT and FORK_CHOICE_BALANCE_INCREMENT constants; some additional casts because a few places where ValidatorIndex was used are now uint64; remove some old commented-out lines of code from shardcommittee transition 2019-02-11 15:32:27 +01:00
Dustin Brody 4747477160 more checking that various functions match v0.1; some variable/etc renaming; fix is_double_vote to check epoch, not slot equivalence; finish verify_slashable_attestation; adjust is_surround_vote logic (#100) 2019-02-08 00:07:15 +01:00
Zahary Karadjov c567bc410f
Use human-readable names for the slot numbers 2019-02-07 22:14:08 +01:00
Yuriy Glukhov f1f1beb703 Fixed block proposal signature domain 2019-02-07 21:55:44 +01:00
mratsim 1e574cf75a Hacky (?) fix #97 assert fails 2019-02-07 21:22:46 +01:00
Dustin Brody 072c1607fd implement new RANDAO reveals (#92)
* implement new RANDAO reveals
2019-02-07 21:13:10 +01:00
Dustin Brody f2545318db get_crosslink_committees_at_slot updates (#95) 2019-02-07 20:27:13 +01:00
Dustin Brody 2794181889 Spec updates (#94)
* s/slot_included/inclusion_slot/; s/participation_bitfield/aggregation_bitfield/; rearrange some type definitions to match spec order

* a couple more Eth1Data-related spec syncs
2019-02-07 17:55:48 +01:00
Zahary Karadjov 31abd31d8a Fix an int conversion issue 2019-02-07 17:53:02 +01:00
Zahary Karadjov dfcd22519b
Fix for getting the current epoch number 2019-02-07 16:40:00 +01:00
Dustin Brody bc44744acd rm unused work_pool module (#90)
* rm unused work_pool module

* preserve comment from Danny about signature aggregation
2019-02-07 16:28:59 +01:00
Jacek Sieka ed8f7ab475
schedule epoch actions for all 64 slots in an epoch 2019-02-07 13:18:00 +01:00
Jacek Sieka 976e1e1f10
spec: remove unused code, fix state_sim speed 2019-02-07 13:03:02 +01:00
cheatfate 20e7ca48b5 Fix `validator not in registry` and `$` bugs. 2019-02-07 12:51:21 +02:00
Dustin Brody 2d71b998c7 Spec update to v0.1 (#88)
* update based on https://github.com/ethereum/eth2.0-specs/pull/483
* more spec updates
2019-02-07 11:43:21 +01:00
Dustin Brody 788b093284
Merge pull request #87 from status-im/vdy
Spec updates
2019-02-06 22:52:36 +00:00
Dustin Brody 74cb86ec21 more spec changes 2019-02-06 21:37:21 +01:00
Mamy Ratsimbazafy 224f22d377
Fix regression from #80 on validator_keygen (#86) 2019-02-06 20:58:18 +01:00
Jacek Sieka 337aa83f1a
spec: fix off-by-one, allows state_sim to continue 2019-02-06 20:53:05 +01:00
Jacek Sieka 2f96c4bade
spec: remove ShardCommittee remnants, add CrosslinkCommittee helper 2019-02-06 20:37:25 +01:00
Mamy Ratsimbazafy 11da02c34f
Asyncdispatch2 rebranding to chronos (#84)
* Asyncdispatch2 rebranding to chronos

* nim-eth and nim-chronos part of nimble packages

* fix package name
2019-02-06 18:56:04 +01:00
Dustin Brody 7cf13c22f0 fix up some non-spec things that interfere with state_sim with attestations 2019-02-06 13:31:41 +01:00
Yuriy Glukhov 7f1bddb267 Use nim-eth (#82) 2019-02-05 20:21:18 +01:00
Dustin Brody d1af069c55 Spec update (#83)
* rm GWEI_PER_ETH

* update EJECTION_BALANCE; rm MAX_CASPER_VOTES/replace with MAX_INDICES_PER_SLASHABLE_VOTE; modify GENESIS_SLOT

* update various constants from slots to epochs; MAX_CASPER_SLASHINGS -> MAX_ATTESTER_SLASHINGS

* AttesterSlashing field renamings; SlashableVote -> SlashableAttestation; add DOMAIN_RANDAO

* rm aggregate_signature_poc_[01]_indices

* centralize some bitfield functions
2019-02-05 19:40:27 +01:00
Ștefan Talpalaru 20e02d5615 enable "--threads:on" for tests (#75)
* enable "--threads:on" for tests

* disable Chronicles colours on Windows

* delete forLoopMacros-related config
2019-02-05 18:48:34 +01:00
Mamy Ratsimbazafy e5d152c6d6
Update BLS to the latest scheme. (#80)
* Attempt to switch to nim-blscurve.

* "Fix" BLS test for new tests

* Missing domain param in validator keygen
2019-02-05 17:13:29 +01:00
Yuriy Glukhov bbd94185a4 State storage predicate moved to beacon_node, split persistBlock to persistBlock and persistState 2019-02-04 21:44:37 +01:00
Yuriy Glukhov 1606bc6f92 Cosmetics 2019-02-04 21:44:37 +01:00
Yuriy Glukhov 917c48d9a5 More persistance 2019-02-04 21:44:37 +01:00
Dustin Brody dda24619e8 several mechanical renamings 2019-01-30 19:30:34 +02:00
Dustin Brody 68ca7850f4 rm min_empty_validator_index, ZERO_BALANCE_VALIDATOR_TTL, Validator.latest_status_change_slot; switch Fork.fork_slot to Fork.epoch; begin removing Validator.randao_layers; now that https://github.com/ethereum/eth2.0-specs/issues/352 is fixed, test out for more slots; get process_deposit consistent with current spec 2019-01-30 00:41:07 -08:00
Dustin Brody 8dadbd3257 fix spurious assertion per https://github.com/ethereum/eth2.0-specs/issues/515 2019-01-28 20:49:20 -08:00
Dustin Brody 5d0e1c026d remove Validator.{custody_commitment,latest_custody_reseed_slot,penultimate_custody_reseed_slot} 2019-01-28 20:34:44 -08:00
Dustin Brody ad3de5fde9 Uint24 -> ValidatorIndex; Fork.pre_fork_version -> Fork.previous_version; Fork.post_fork_version -> Fork.current_version 2019-01-28 20:15:00 -08:00
Dustin Brody 4574178836 epochs replace slots as main first-class entity; several mechanical renamings (ForkData -> Fork; .fork_data -> .fork; remove stub custody types/code; some type aliasing to keep slots and epochs more enforceably separate; fix test_state_transition.nim assumption that GENESIS_SLOT == 0 2019-01-28 19:22:22 -08:00
Dustin Brody b46183047f remove BeaconState.shard_committees_at_slots in favor of crosslink committees; validator_registry_latest_change_slot -> validator_registry_update_slot mechanical renaming 2019-01-28 23:43:55 +02:00
Dustin Brody f85088651c Revert "remove BeaconState.shard_committees_at_slots in favor of crosslink committees; validator_registry_latest_change_slot -> validator_registry_update_slot mechanical renaming"
This reverts commit 2222a8e222.
2019-01-26 11:17:22 -08:00
Dustin Brody 2222a8e222 remove BeaconState.shard_committees_at_slots in favor of crosslink committees; validator_registry_latest_change_slot -> validator_registry_update_slot mechanical renaming 2019-01-26 11:15:33 -08:00
Bruno Škvorc 7071021dc5
Typo fix 2019-01-25 23:13:41 +01:00
Jacek Sieka e0a257406e
clean up and add some logging (#70) 2019-01-25 11:35:22 -06:00
zah a0712e691a Replace the usage of cligen with confutils (#68) 2019-01-23 12:45:15 -06:00
Dustin Brody 53952dd0c9
Merge pull request #66 from status-im/wcc
Spec update
2019-01-23 18:34:00 +00:00
Dustin Brody 3877ef263c remove return to gain better error messages 2019-01-23 09:53:51 -08:00
Jacek Sieka 63f7302a41
Update beacon_chain/spec/beaconstate.nim
Co-Authored-By: tersec <tersec@users.noreply.github.com>
2019-01-23 17:51:59 +00:00
Dustin Brody a11dcdfe64 switch from get_shuffling_prev to get_shuffling; remove some also-removed-in-spec calls and temporarily/ugly-bridge a still-useful reference to shard committees until they're completely removed 2019-01-22 17:50:51 -08:00
Dustin Brody 21b8bd7f71 new shuffling routine and adjust tests for it 2019-01-22 14:36:31 -08:00
Dustin Brody d728f113ac move existing shuffling routine out of way and keep existing references to it 2019-01-22 11:17:34 -08:00
Jacek Sieka 523a990adc
ssz: switch to little-endian
https://github.com/ethereum/eth2.0-specs/pull/139
2019-01-22 11:56:01 -06:00
Zahary Karadjov a9c026b957 More logging close to the code that fails often 2019-01-21 21:43:08 +02:00
Dustin Brody eed9ea8f0a Spec updates (#64)
* s/CrosslinkRecord/Crosslink/

* {MIN,MAX}_DEPOSIT to {MIN,MAX}_DEPOSIT_AMOUNT; rm SHARD_PERSISTENT_COMMITTEE_CHANGE_PERIOD; per https://github.com/ethereum/eth2.0-specs/issues/378 and 35adc9c61a rm persistent_committees, persistent_committee_reassignments, and ShardReassignmentRecord

* complete switchover from status/ValidatorStatusCodes to status_flags/INITIATED_EXIT/WITHDRAWABLE validator representation
2019-01-21 19:26:58 +01:00
Dustin Brody 476052b314 deposit root to Eth1Data transition 2019-01-18 12:47:30 +02:00
Dustin Brody c788d72857 s/POW_RECEIPT_ROOT_VOTING_PERIOD/ETH1_DATA_VOTING_PERIOD/; rm COLLECTIVE_PENALTY_CALCULATION_PERIOD, ACTIVE_PENDING_EXIT, EXITED_WITH_PENALTY 2019-01-18 12:47:30 +02:00
Dustin Brody 1fe5ba07b8 update initiate_validator_exit and exit_validator helper functions to current spec; remove spec-removed EXITED_WITHOUT_PENALTY 2019-01-18 12:47:30 +02:00
Dustin Brody 9dcf37aad6 adjust get_shuffling and get_active_validator_indices to use slot, per spec 2019-01-18 12:47:30 +02:00
Dustin Brody f30b4f822e ValidatorRecord -> Validator 2019-01-18 12:47:30 +02:00
Dustin Brody 38c2cc72fa poc -> custody mechanical name changes 2019-01-18 12:47:30 +02:00
Zahary Karadjov d753332c54 Fix the build of the validator_keygen 2019-01-17 14:38:27 +02:00
Zahary Karadjov 43ae45fe7e Add --logLevel command-line flag; Fix running the start.sh script from the beacon-chain repo 2019-01-17 14:38:27 +02:00
Yuriy Glukhov 7e4d264887 Back the db by rocksdb backend 2019-01-17 14:38:16 +02:00
Dustin Brody ce72c1b4bc re-disable assertion creationg failures in research/state_sim; hadn't been set up to run before, either 2019-01-17 00:14:55 -08:00
Dustin Brody 6f103a7bcd switch to new penalty calculation infrastructure 2019-01-16 06:50:54 -08:00
Dustin Brody 7e3aa4dda2 initial process_penalties_and_exits 2019-01-16 05:39:34 -08:00
Dustin Brody 44edf400d2 rm PENDING_ACTIVATION 2019-01-16 05:24:58 -08:00
Dustin Brody 2ff973e46f add some helper functions from new spec; fix some comments; more plumbing; start removing some of the status-code refs in favor of the new status bitfield 2019-01-16 04:57:49 -08:00
Dustin Brody bc65693eae stop routing through update_validator_status 2019-01-16 04:20:44 -08:00
Dustin Brody 5823091501 some more fields, plumbing, comment fixes 2019-01-16 03:39:16 -08:00
Dustin Brody cb1096c581 add some helper functions, change epoch processing, add some new constants 2019-01-16 03:07:41 -08:00
Dustin Brody d101c117fd stage a few more working changes from the bunch 2019-01-16 02:42:40 -08:00
Dustin Brody 1bf876f5c1 mechanical renamings 2019-01-16 02:21:06 -08:00
Zahary Karadjov 7608fd76a3 Make the code compile in Nim 0.19.0 2019-01-12 13:57:50 +02:00
mratsim c3d6961eed Remove rebase hash leftovers 2019-01-12 13:57:50 +02:00
mratsim 0a080e4f27 Remove the vote count closure, unlike the spec we use a count table 2019-01-12 13:57:50 +02:00
mratsim 423f632606 Fork choice compiles (but untested) 2019-01-12 13:57:50 +02:00
mratsim f0e3de6b19 Depart from spec "attestation_targets" 2019-01-12 13:57:50 +02:00
mratsim 4a6e694268 Add BeaconChainDB 2019-01-12 13:57:50 +02:00
mratsim 8a68bbb26c Integrate fork choice helpers into skeleton 2019-01-12 13:57:50 +02:00
mratsim e3d8bc9695 Sync with latest specs
get_parent - https://github.com/ethereum/eth2.0-specs/pull/250
AttestationData - ac600d39

https://github.com/ethereum/eth2.0-specs/issues/285 and 5b35b6c2
2019-01-12 13:57:50 +02:00
mratsim 4c3cae5519 Move fork choice research out of beacon chain 2019-01-12 13:57:50 +02:00
mratsim 12c6e2f46b Implement spec helpers for lmd ghost 2019-01-12 13:57:50 +02:00
Zahary Karadjov a586087472 Stable network simulation 2019-01-09 10:45:33 +02:00
Zahary Karadjov 1c30214e9a Reach block 64 in the network simulation
* Slightly better logging
* Handling of incoming blocks and missing blocks
2019-01-07 14:59:52 +02:00
Zahary Karadjov 8f9a5441f1 More reliable network simulation
You'll need the latest versions of nim-eth-p2p, nim-serialization
and nim-json-serialization.

Before starting the simulation script, make sure to delete any previous
json files from the simulation folder:

```
rm tests/simulation/*.json
tests/simulation/start.sh
```

This should survive the creation of few blocks before diying with a
block validation error.
2019-01-05 14:35:47 +02:00
Jacek Sieka a0aa230c45 spec updates
* skip validation in a few more cases
* more renames
* add attestations to state sim
2018-12-28 02:32:41 +02:00
Jacek Sieka 12a819c110
spec updates (#48)
* spec updates
* balances move out to separate seq
* bunch of placeholders for proof-of-custody / phase1
* fix inclusion distance adjustment
* modify state in-place in `updateState` (tests spent over 80% time
copying state! now it's down to 25-50)
* document several conditions and conversations
* some renames here and there to follow spec
2018-12-27 14:14:37 -06:00
Dustin Brody 68644517da initial rough commit of a work/attestation pool (#46)
* initial rough commit of a work/attestation pool

* add kludgy participation_bitfield handling, some error handling, and document future improvments from spec update #47

* rename addAttestation to add for Nim consistency
2018-12-26 13:17:30 +01:00
Jacek Sieka eb369cee4e spec updates
* first attestation created!
* add hash_tree_root_final that returns an Eth2Digest
* hits the first real blocking spec bug :(
2018-12-24 10:08:11 +01:00
Jacek Sieka 04314589ff
spec updates (#45)
* spec updates

* random small updates
* ssz no longer sorts by field, fix enum serialization
* rewire block processing a little to avoid a few state copies
* add a state simulation tool that writes out jsons
2018-12-21 16:37:46 -06:00
Zahary Karadjov abb199d6dc Chain creation and network simulation start script 2018-12-19 15:39:00 +02:00
Jacek Sieka 0b0c66ebd9 spec updates
* spec-following renames
* more documentation
* simplify casper slashing
* fix block creation flow / logic
2018-12-19 14:48:25 +02:00
Jacek Sieka 400ac83aa0
BLSVerify > bls_verify 2018-12-17 13:36:17 -06:00
Jacek Sieka 142aa8ca8e ssz: finish implementation (#42)
* ssz: finish implementation

* add object support, simplify implementation
* fix extra round of hashing in tree_hash_root

* ssz: cleanups

* work around Nim range bug for Uint24, cleanups
2018-12-17 19:03:53 +01:00
Dustin Brody 5dc06f4496 switch some proc to func, add some infrastructure, update based on spec name changes (#40)
* switch some proc to func, add some infrastructure, update based on spec name changes

* typo
2018-12-15 10:32:36 -06:00
Danny Ryan aa5129b90e simplify randao layers tests 2018-12-15 01:17:59 +02:00
Danny Ryan b1114a94de fix double incrementing of proposer.randao_layers 2018-12-15 01:17:59 +02:00
Jacek Sieka 35ac0c4ddc
spec updates (#39)
* some minor renames
* fix some leftover block slot references (replaced by state slot)
* fix some more uint underflows
* update epoch calculations (reward et al)
* now works past 100 epochs at least with empty blocks (very slow)
2018-12-14 10:12:39 -06:00
Jacek Sieka 44bb13ae46
spec updates (#37)
* use repeat_hash from spec in randao
* add some logging to state processing
* export crypto string converters in spec insulation layer
* enable block signature verification
* ssz: add support for arrays, remove custom data type code
* correctly handle previous and new block
* add trivial block processing tests
* prefer iterative repeat_hash
* state transition mostly done
* handle most specials and slashings
2018-12-13 10:00:55 -06:00
Yuriy Glukhov a7ee8a71bb Sign block proposal and attestation 2018-12-13 12:58:05 +02:00
Dustin Brody 34f7254c61 fix build 2018-12-12 16:10:46 -08:00
Dustin Brody 4b1197bc17 fix one compile error 2018-12-12 13:44:18 -08:00
tersec d48f056be9
Merge pull request #33 from status-im/gossip_sub
Faux implementation of GossipSub based on RLPx
2018-12-12 21:36:43 +00:00
Yuriy Glukhov 5776393539 Fixed compilation error with newer nim 2018-12-12 12:36:20 +01:00
Jacek Sieka 5b35b6c2dc
hash -> root
ethereum/eth2.0-specs#285
2018-12-11 15:53:18 -06:00
Jacek Sieka 7ea51d5b0b
spec updates (#34)
* spec updates

* make several constants uint64 to help minimize casting
* document data type woes - will have to revisit these
* change comment style on fields and constants to make room for better
comments
* add BLSVerify and BLSAddPbkeys facades to insulate spec code from
milagro
* fix proof of possession type
* drop explicitly ordered container fields from ssz - there's an issue
open to sort this out before committing it to the spec
2018-12-11 11:55:45 -06:00
Zahary Karadjov c71f89e211 Faux implementation of GossipSub based on RLPx 2018-12-10 14:06:54 +02:00
Yuriy Glukhov 10ed2bd5b9 Comments addressed, cosmetics 2018-12-09 11:17:54 +02:00
Yuriy Glukhov 61f81fb0c4 Some bits of randao 2018-12-09 11:17:54 +02:00
Jacek Sieka e8ec26debb
spec updates
* bunch of renames
* bunch of new constants
* more long variable names!
2018-12-05 20:07:04 -06:00
Jacek Sieka eba4f18f71
Merge remote-tracking branch 'origin/master' into epoch-state-2 2018-12-05 11:58:01 -06:00
Yuriy Glukhov f12b679880 addLocalValidators impl 2018-12-05 18:58:28 +02:00
Jacek Sieka 94756cc5a2
add is_power_of_2 implementation 2018-12-05 07:19:57 -06:00
Jacek Sieka 4b41010610
add more epoch processing
* document several parts
* almost complete epoch state processing
2018-12-04 12:45:30 -06:00
Jacek Sieka ac600d398b
spec updates
* lots of renames
* add some epoch processing
2018-12-03 15:41:24 -06:00
tersec 0141c84fe5 data type updates from spec (#25)
* data type updates from spec
2018-12-03 11:46:22 -06:00
Jacek Sieka 6669e698b5
spec updates
* more work on block processing
* fix some unsigned issues
* fix ssz issue when last chunk is not full
* work around empty const seq compiler bug
* XXX -> TODO
2018-11-29 16:16:07 -06:00
Jacek Sieka a661e5afd8
call out inefficiencies and quirks for future consideration 2018-11-29 12:18:12 -06:00
Jacek Sieka 4d6b068697
beacon state updates
* add `on_startup` genesis state constructor
* update state transition to use ssz
* update validator adder
2018-11-28 23:38:44 -06:00
Zahary Karadjov e4d1bebf4d Get the code to compiling state 2018-11-29 03:17:57 +02:00
Zahary Karadjov 32ebcb007a Add more flesh to the skeleton :) 2018-11-28 22:55:31 +02:00
Zahary Karadjov 4920c0a357 work-in-progress beacon node skeleton (not compiling at the moment) 2018-11-28 22:55:04 +02:00
Jacek Sieka 53f819689e
fix import 2018-11-28 14:01:30 -06:00
Jacek Sieka b01c51b3e2
move spec stuff to spec directory
helps maintain sanity in the face of spec changes
2018-11-28 13:49:03 -06:00
Dustin Brody e63b452181 assorted fixes; some of algorithms aren't 100% updated yet, but want to get data structures in place 2018-11-27 20:23:27 -08:00
Jacek Sieka 548b6922ba
introduce Eth2Hash, Eth2Digest and friends (#22, fixes #3)
* introduce Eth2Hash, Eth2Digest and friends
2018-11-27 17:10:09 -06:00
Zahary Karadjov 2137b084b2 work-around a generics compilation issue 2018-11-26 18:41:03 +02:00
Jacek Sieka da22f2065f
docs & comments
* link to resources for shuffling
* describe get_beacon_proposer
* hints for solving init issue
2018-11-26 09:44:49 -06:00
Jacek Sieka d1839dad9a
more small fixes
* move consts to top
* move some functions to validator.nim
* add some validator functoin smoke tests
2018-11-23 16:44:43 -06:00
Jacek Sieka 3ce94623e3
spec updates 2018-11-23 13:42:47 -06:00
Mamy Ratsimbazafy 9989f2fe56
"Stub" hashSSZ function for BeaconBlock (#18)
* FIx hash_ssz in the generic case

* Add a stub hash function for BeaconBlocks
2018-11-22 11:17:05 +01:00
tersec 0fe8c6762c update data structures (#17) 2018-11-20 18:52:14 -06:00
Jacek Sieka 7e243d57cd
ssz: documentation updates, fix fromBytesSSZ type constraint 2018-11-20 11:35:11 -06:00
Jacek Sieka 6df3ec952b
hash_ssz: simplify per latest spec update
* update per https://github.com/ethereum/eth2.0-specs/pull/140
* join with ssz to form a single ssz module (like spec)
2018-11-20 10:43:09 -06:00
Yuriy Glukhov 7e630bf8bf Fixed compilation errors. Reenabled test 2018-11-19 19:09:39 +02:00
tersec eff23ba182
Merge pull request #14 from status-im/HelperSpecUpdates
update some helper functions to current spec, along with a couple of data types that needed tweaking/fixing
2018-11-16 22:57:54 +00:00
Jacek Sieka 949b24702c
hash_ssz: updates (#13)
* rename from tree_ssz
* add special cases for some types based on beacon chain code
* add smoke test
* avoids some trivial allocations, but the big one remains - the
temporary
chunk buffer is still heap allocated
* update to handle zero-length buffer case same as latest spec
2018-11-16 11:04:21 -06:00
Dustin Brody d0fd1b4eb7 update some helper functions to current spec, along with a couple of data types that needed tweaking/fixing 2018-11-15 16:54:36 -08:00
Dustin Brody 8493949456 update ShardAndCommittee and CrosslinkRecord 2018-11-15 10:20:27 -08:00
Dustin Brody b2d93f75c1 update ValidatorRecord 2018-11-15 10:16:09 -08:00
Dustin Brody 150d0b6055 rm CrystallizedState, add ShardReassignmentRecord, add BeaconState 2018-11-15 10:01:54 -08:00
Dustin Brody 9ea109ba4c rm ActiveState 2018-11-15 08:47:56 -08:00
Dustin Brody 6a95e2c664 add AttestationSignedData 2018-11-15 08:44:52 -08:00
Dustin Brody 0b45eb5b74 update AttestationRecord 2018-11-15 08:35:25 -08:00
mratsim 572b856dfd empty start at true 2018-11-13 19:19:10 +01:00
mratsim 3935d33457 Merge branch 'bls-scheme2' of https://github.com/status-im/nim-beacon-chain into bls-scheme2 2018-11-13 19:16:03 +01:00
mratsim b8a17522c0 Fix agg_pubkey initialization 2018-11-13 19:15:57 +01:00
Mamy Ratsimbazafy 45fb4f83a2
Merge branch 'master' into bls-scheme2 2018-11-12 17:18:14 +01:00
mratsim 6a3c7d6aa5 Update for bls scheme 2 2018-11-12 10:13:24 +01:00
mratsim dab75a76a2 comment out aggregated sig + make SSZ compile again 2018-11-12 09:46:16 +01:00
Jacek Sieka 786337e930
tree_ssz: naive implementation of treeHash
as seen in https://github.com/ethereum/eth2.0-specs/pull/120
2018-11-11 19:21:47 -06:00
Jacek Sieka 5bdcee6536
document spec links 2018-11-11 19:19:58 -06:00
tersec f389c454d6 first part of spec updating; some constants, structure changes, data type changes from int to uint (#9)
* first part of spec updating; some constants, structure changes, data type changes from int to uint

* Update beacon_chain/datatypes.nim

Co-Authored-By: tersec <tersec@users.noreply.github.com>
2018-11-09 10:34:30 +01:00
Jacek Sieka 577598cca7 ssz: fix buffer overflows, explicit error reporting (#7)
* ssz: fix buffer overflows, explicit error reporting

* ssz: harden against unaligned data
2018-10-08 12:56:34 +02:00
mratsim 46e8affee0 Delete outdated per_block_processing.nim 2018-10-03 13:15:17 +02:00
Mamy Ratsimbazafy f60d578fb3
Signature aggregation in block processing (#5)
* signature-aggreg-in-block-processing

* Message is a Blake2 hash not simple concat

* Address comments:

- Range change in parent_hashes computation
- Slot not beind module Cycles anymore
- justified_slot part of the hash

Not addressed: using parent.slot_number instead of block.slot_number
2018-10-02 18:09:11 +02:00
mratsim 99a394e084 Config CI + use Milagro BLS signatures 2018-09-26 18:26:39 +02:00
mratsim ca4082b316 Update to latest spec 2018-09-24 17:44:48 +02:00
mratsim 6ddab7e9c0 basic SimpleSerialize + tests 2018-09-20 17:45:02 +02:00
mratsim a42a0ad851 Fork choice compile and run 🔥 2018-09-12 15:41:49 +02:00
mratsim acf322ea7b Fix tables initialization issues 2018-09-11 14:02:45 +02:00
mratsim 3bc75711d5 Finish network simulator 2018-09-11 12:24:46 +02:00
mratsim 03b1f1a4ba Networksim additions + fix agents and normal_distribution types 2018-09-10 12:35:11 +02:00
mratsim 32dbdc860b Mention thread-safety 2018-09-06 18:16:11 +02:00
mratsim 61af4c33c7 Add a normal distribution generator for latency simulations 2018-09-06 12:31:20 +02:00
mratsim e049d9e106 Finish fork choice rule base procs 2018-09-05 12:03:52 +02:00
mratsim 3d76e238e7 Fix slot and proposer type, add initBlock proc 2018-09-04 18:09:28 +02:00
mratsim 0561bf70b7 Switch to proper time datatypes 2018-09-04 17:27:53 +02:00
mratsim 1cb28aecc8 Implement signature and block fork choide rule 2018-09-04 16:22:10 +02:00
mratsim 1f1328d019 Make sense of Ghost node Python spec dynamic polymorphism 2018-09-04 14:48:32 +02:00
mratsim f7e876471e initial impl of fork choice rules. Main types sorted out 2018-09-03 18:42:22 +02:00
mratsim e403e96144 indices_for_slot renaming - aligning to 01abccfdbb 2018-08-24 10:20:15 +02:00
mratsim 14bb9b6efd Revamp per block processing / state transition 2018-08-21 18:21:45 +02:00
mratsim 0b560f7307 Update helpers get_indices_for_slot and get_block_hash following spec fixes, see https://github.com/ethereum/beacon_chain/pull/62 2018-08-21 14:45:35 +02:00
mratsim 9f0fc6817b Update helpers to latest sped, highlight spec bugs 2018-08-17 18:21:25 +02:00
mratsim f203859316 Use Ethereum Foundation naming convention (snake_case + MACRO_CASE) 2018-08-17 18:21:10 +02:00
mratsim f8b2e790e3 sync with latest spec, closes #2 2018-08-16 12:02:59 +02:00
mratsim d65d714c3b Stub Partial crosslink records checks 2018-07-23 18:36:02 +02:00
mratsim 585072ae15 Add getCutoffs helper (buggy) 2018-07-23 16:22:37 +02:00
mratsim 4c6665832b Add the pseudo renadom shuffle 2018-07-23 14:58:41 +02:00
mratsim 2438aaada8 Update everything to latest spec change from July 21 2018-07-23 12:15:56 +02:00
mratsim 65dbb086d1 Add the types used in the beacon chain 2018-07-20 20:18:56 +02:00