154 Commits

Author SHA1 Message Date
Mamy Ratsimbazafy
2a4c4df7c3
Bls official tests (#268)
* Update fixtures with BLS and SSZ generic

* Stash changes following https://github.com/status-im/nim-serialization/issues/4

* Add Private to Public key conversion

* Add message signing tests

* Add aggregate signatures and public keys

* Add BLS to test suite
2019-05-23 09:41:17 +02:00
Zahary Karadjov
8da71715eb Bump the beacon_node version; Make the simulation script compatible with macOS
The shell expansion syntax used in the run_node script was not handled properly
by the bash shell shipped with macOS. In particular, it was not able to handle
range expressions involving interpolated vars:

cp foo/{$FIRST_INDEX..$LAST_INDEX} ...
2019-05-14 16:46:43 +03:00
Mamy Ratsimbazafy
e5047d93a4 [Tests] SHA256 + official shuffling vectors (#263)
* Change digests to SHA2-256 (from Keccak256)

* Fix sha256 digest initialization - pass shuffling test

* Add comments + remove old shuffling tests

* Remove stale tree hashing tests

* Small toOpenArray optim + notes on in-place hashing

* Revert "Revert "Update test repo""

This reverts commit f385467cd3312058ec0ccf70be83e80f400a41c5.

* Revert "Revert "Fix SIGFPE on shuffling for 0""

This reverts commit 226d380f8733663c033edee94fdd8427c4ddb2a5.

* Revert "Revert "Implement shuffling test (pending typedesc and shuffling algo fix)""

This reverts commit 813cb6fbb8cdbb768a894c35173c3adc1e5d6c59.

* withEth2hash templates now needs ctx.init()

* Use init-update-finish to avoid burnMem
2019-05-10 08:14:01 +00:00
Dustin Brody
d977c96ae1 0.6.1 - signing_root, deposits, validator indices and committee count (#261)
* signed_root -> signing_root

* implement new process_deposit; update timing parameters to 0.6.1; update Deposit to 0.6.1 and remove DepositInput

* update get_active_validator_indices and get_epoch_committee_count to 0.6.1; rm get_current_epoch_committee_count, get_previous_epoch_committee_count, and get_next_epoch_committee_count; bump state_sim default validator count a bit more

* re-introduce 0.5.1-ish get_active_validator_indices, get_epoch_committee_count as scaffolding for still-0.5.1ish shuffling
2019-05-09 14:27:37 +02:00
Mamy André-Ratsimbazafy
f385467cd3
Revert "Update test repo"
This reverts commit d7d594eee93030c3906bfc40e4100a4304e0c2aa.
2019-05-09 12:07:10 +02:00
Mamy André-Ratsimbazafy
813cb6fbb8
Revert "Implement shuffling test (pending typedesc and shuffling algo fix)"
This reverts commit ad9f64387302513c7a8ac43d97e3ad04201d5e7d.
2019-05-09 12:06:45 +02:00
Mamy André-Ratsimbazafy
ad9f643873
Implement shuffling test (pending typedesc and shuffling algo fix) 2019-05-09 12:02:52 +02:00
Mamy André-Ratsimbazafy
d7d594eee9
Update test repo 2019-05-09 12:00:40 +02:00
Jacek Sieka
17fa499755 cache state root a bit more aggressively (#260)
needed to be able to run state sim, too slow otherwise
2019-05-04 14:10:45 +00:00
Dustin Brody
46b4154ce8 begin 0.6.0 (#256)
* begin 0.6.0: new get_domain/increase_balance/reduce_balance, BeaconState.validator_balances -> BeaconState.balances, some renamed constants, transaction processing changes, SlashableAttestation field name changes, 0.6.0 get_beacon_proposer_index always uses given state's slot, update tests subrepo

* mark get_bitfield_bit/bls_verify_multiple/stat-list-lengths/is_active_validator/is_surround_vote/slot_to_epoch/int_to_bytes/etc as unchanged in 0.6.0; rm Eth1DataVote/maybe_reset_eth1_period and thus adjust expected tree hash test results

* mark verify_bitfield/bls_verify/deposit-contract/VoluntaryExit/PendingAttestation/Historicalbatch/Fork as 0.6.0; update DOMAIN_BEACON_BLOCK to DOMAIN_BEACON_PROPOSER

* update Crosslink to 0.6.0 (also requires tree hashing test result change, so isolate in individual commit)

* mark verify_merkle_branch/get_delayed_activation_exit_epoch/ProposerSlashing/Attestation/AttestationDataAndCustodyBit/hash/integer_squareroot/get_epoch_start_slot/is_double_vote/get_randao_mix/generate_seed as 0.6.0; update reward and penalty quotients; SlashableAttestation -> IndexedAttestation; rm get_fork_version; ATTESTATION_INCLUSION_REWARD_QUOTIENT -> PROPOSER_REWARD_QUOTIENT
2019-04-29 10:48:30 -06:00
Mamy Ratsimbazafy
5784b2d7f7
Full EF state tests parsing (#242)
* Load full state test + Add json-serialization for Bitfield

* Implement empty_block_transition test (but test is missing the end state)

* Use the provided empty block instead of mocking one

* Add failing block signing test

* Tests that can't be passed now are now "for information" + indent the "information"/hash given
2019-04-29 18:10:01 +02:00
Jacek Sieka
c7e6d39279 fix statedata fields not being updated correctly (#255) 2019-04-29 11:14:22 +03:00
Yuriy Glukhov
61ec6fa167 Move incoming block from unresolved to pending if it can't be applied (#253)
* Move resolved block from unresolved to pending if it can't be applied

* Renamed unresolved to missing
2019-04-26 10:38:56 -06:00
Ștefan Talpalaru
53012b9a25
NAT traversal 2019-04-18 15:48:17 +02:00
cheatfate
0f8dd374d9
Fix official tests path. 2019-04-12 01:25:22 +03:00
Mamy Ratsimbazafy
ad1803bdd6
Submodule official tests (#238)
* Remove eth2.0-tests submodule

* Remove files

* Add submodule https://github.com/status-im/nim-eth2-official-tests

* Update tests path

* update submodule version

* Move beacon-chain specific datastructures out of the submodule
2019-04-08 10:29:44 +02:00
Dustin Brody
c53de3e550 Caching updates/refactoring & state_sim defaults updates (#235)
* rm now-superceded shuffling cache (shuffling is only called from get_crosslinks), which was badly architected due to trying to exist in state; rm one more vestige of previous light-client regime (one more to go, from datatypes)

* fix wrong shuffling list size (active validator size, not validator size) to make consistent with 0.5.1 (will be inconsistent with testnet0); fix typo and change defaults in state_sim

* doAssert a couple of constant relationships necessary to avoid underflow; rm non-spec, unused helper function get_new_recent_block_roots

* refactor separate crosslink_committee_cache and winning_root_participants_cache(s) into StateData object; remove last vestige of previous shuffling cache

* separate out caching parts of StateData to new StateCache object
2019-04-05 08:18:13 -06:00
Jacek Sieka
605dd0a0e9 Some speedups (#226)
* ssz: avoid memory allocations

* a bit fishy with the 32-item stack.. this should be a smallvector

* digest: avoid another burnmem

* avoid a few allocations by using iterator
2019-04-03 15:46:22 +00:00
Mamy Ratsimbazafy
ad133a0222
Official state tests - parsing and loading beacon state (#227)
* initial commit of official state tests

* sanity check fixture

* Parsing official state test is mostly working
(Except BLS signature)

* Successfully load state test

* Use json-serialization instead of json and display deserialized and from scratch beacon state hashes

* Add official state test as a smoke parsing test
2019-04-02 16:50:24 +02:00
Zahary Karadjov
82282928c3 Allow setting the number of missing nodes 2019-03-29 11:32:25 +02:00
Zahary Karadjov
c32a833bf7 Support for running a local simulation with arbitrary number of validators 2019-03-29 11:32:25 +02:00
Jacek Sieka
2f86bf21c8
attestation test: fix compile & disable temporarily 2019-03-28 13:15:09 -06:00
Ștefan Talpalaru
f8fbe0ff3b Makefile that only works in a Git submodule (#210)
- updated README.md
- beacon_chain.nimble: accept compilation flags as params
- nim.cfg: enable `--opt:speed` at the top level
- simulation scripts: allow overriding GIT_ROOT and do some refactoring
2019-03-28 09:18:59 -06:00
Zahary Karadjov
5f4581240e Aha, the logging issue depended on options set in start.sh; Worked-around 2019-03-28 16:54:50 +02:00
Jacek Sieka
f9e0418b5b fetch multiple blocks at a time
* avoid crash on invalid block production (fixes #209)
* fetch blocks every second, roughly
* fix start.sh param order
* run beacon node sim at slower pace
2019-03-28 10:42:16 +02:00
Jacek Sieka
3ad6053298 Use block, slot tuple to idenfiy fork
this is the beginning of tracking block-slots more precisely, so we can
the justification epoch slot bug.

* avoid asyncDiscard (swallows assertions)
* fix attestation delay
* fix several state root cache bugs
* introduce workaround for genesis epoch spec issue
2019-03-28 10:32:09 +02:00
Zahary Karadjov
b3d23c052c Switch to 30 seconds slot duration; UX improvements
* On the testnet site, the first N instead of the last N validators
  will be considered reserved for users

* The number of validators is consistently named `totalValidators`
  in all commands

* Proper keys are generated by default in validator_keygen
2019-03-27 18:13:39 +02:00
Jacek Sieka
81c66fbce0
use signed_root as canonical block root (#211)
* no need to pass prev_block_root when updating state
* some Slot fixes
* fix `hash_tree_root` for Slot, Epoch
* detect missing hash_tree_root type support
* remove some <0.5 state checks
2019-03-26 19:32:35 -06:00
Zahary Karadjov
803a0570be deleted an accidentally commited submodule 2019-03-26 18:42:02 +02:00
Mamy André-Ratsimbazafy
808c5edfc0 Add more fork choice logs 2019-03-26 17:35:09 +02:00
Jacek Sieka
1b0e67c88c
ssz: update to 0.5.1:ish (#202)
* ssz: update to 0.5.1:ish
* slightly fewer seq allocations
* still a lot of potential for optimization
* fixes #174
* ssz: avoid reallocating leaves (logN merkle impl)
2019-03-25 10:46:31 -06:00
Dustin Brody
9f55b4646c
More 0.5.1 spec updates (#195)
* rm gone-in-0.5.0 Proposal, verifyBlockSignature, and slot check which moved to spec function processBlockHeader

* mark get_attestation_participants and get_epoch_committee_count as 0.5.1; finish updating processAttestations to 0.5.1; add kludgy workaround for bug relating to get_winning_roots_etc using crosslink_data_root as index when we have that as ZERO_HASH for all leading to it confusing attesters on different shards; rm BeaconState.latest_attestations, which splits into previous_epoch_attestations and current_epoch_attestations

* Fix CI due to removed latest_attestations field
2019-03-22 18:33:12 +00:00
Zahary Karadjov
0a027e410a Address review concerns and make some steps towards json logging 2019-03-22 17:04:17 +02:00
Jacek Sieka
ee89ef1c79
cleanups (#189)
* add simple bitfield type (fixes #19)
* fix bit endianess to match spec
* consolidate attestation and block logging
* cruft cleanup
* run optimizer on tests, speeds up build
2019-03-20 14:01:48 -06:00
Dustin Brody
13c7f7fcec begin refactoring 0.5.0 epoch transition (#185)
* begin refactoring 0.5.0 epoch transition at justification and finalization

* update processTransfers to 0.5.0 spec

* update get_crosslink_committees_at_slot to 0.5.0 except for noted workaround for pre-0.5.0 spec bug related to epoch boundary conditions, to be flipped in more coordinated way with other epoch processing changes

* mark ejection processing as 0.5.0

* 0.5.0 epoch update finalization

* rm BEACON_CHAIN_SHARD_NUMBER to complete updating miscellaneous constants to 0.5.0

* mark verify_slashable_attestation as 0.5.0

* update process_attester_slashing to 0.5.0

* update process_slashings(...) to 0.5.0 and mark process_exit_queue as 0.5.0

* refactor epoch processing by implementing 0.5.0's update_registry_and_shuffling_data(...)
2019-03-19 20:19:46 -06:00
Zahary Karadjov
9930fa5bef Fix a wrong bootstrap port 2019-03-20 03:34:04 +02:00
Jacek Sieka
71670ca9c9
make nat setting optional when running sim 2019-03-19 18:12:54 -06:00
Zahary Karadjov
6a35d3584d Automate the creation of the network metadata files
With these changes, running a simulation is very close to running
an actual testnet. Some checks have been added in the client to
make sure you are not connecting to an incompatible network (e.g.
a network running with a different number of shards).
2019-03-19 20:27:54 +02:00
Jacek Sieka
d7f972a47e allow setting advertised external ip
* detect it in start.sh
2019-03-19 10:50:09 +02:00
Dustin Brody
2bc632c29b
AttestationData.justified epoch -> AttestationData.source_epoch and AttestationData.latest_crosslink -> AttestationData.previous_crosslink; implement verify_merkle_branch and update process_deposit to use it; rm BeaconState.batched_block_roots; Transfer.{from, to} -> Transfer.{sender, recipient}; update Fork data type and code to use bytes4; implement int_to_bytes8 (close to metaprogramming time); adjust signed_root and all callers to use 0.5.0 calling signature; processDepositRoot -> processEth1Data (#182) 2019-03-18 15:42:42 +00:00
Zahary Karadjov
6bb38095c9 Command-line and data storage handling for testnets 2019-03-18 12:45:29 +02:00
Dustin Brody
798197879b
Revert "AttestationData.justified epoch -> AttestationData.source_epoch and AttestationData.latest_crosslink -> AttestationData.previous_crosslink; implement verify_merkle_branch and update process_deposit to use it; rm BeaconState.batched_block_roots; Transfer.{from, to} -> Transfer.{sender, recipient}; update Fork data type and code to use bytes4; implement int_to_bytes8 (close to metaprogramming time); adjust signed_root and all callers to use 0.5.0 calling signature; processDepositRoot -> processEth1Data"
This reverts commit 18f1d000a19cd88db6ea832af74d2b489b730cfb.
2019-03-17 15:24:50 -07:00
Dustin Brody
18f1d000a1
AttestationData.justified epoch -> AttestationData.source_epoch and AttestationData.latest_crosslink -> AttestationData.previous_crosslink; implement verify_merkle_branch and update process_deposit to use it; rm BeaconState.batched_block_roots; Transfer.{from, to} -> Transfer.{sender, recipient}; update Fork data type and code to use bytes4; implement int_to_bytes8 (close to metaprogramming time); adjust signed_root and all callers to use 0.5.0 calling signature; processDepositRoot -> processEth1Data 2019-03-17 15:17:40 -07:00
Dustin Brody
f36c2d86dc begin 0.5.0 spec update (#179)
* begin 0.5.0 spec update: parent_root -> previous_block_root, BeaconState.justified_epoch -> BeaconState.current_justified_epoch, DOMAIN_PROPOSAL -> DOMAIN_BEACON_BLOCK, temporarily rename BeaconBlockHeader to BeaconBlockHeaderRLP to allow for gradual re-merging without disrupting RLP; mark a few unchanged functions and data types, implement get_temporary_block_header/get_empty_block/should_update_validator_registry/processBlockHeader/cacheState; update a few others

* a dozen or so more trivial mostly comment changes, finding more unchanged parts of 0.5.0

* several more trivial changes; goal is to reduce noise around the upcoming substantial changes (epoch processing, etc)
2019-03-16 13:52:37 -06:00
Ștefan Talpalaru
9ff1eb4ac8
Merge branch 'master' of github.com:status-im/nim-beacon-chain 2019-03-14 00:05:12 +01:00
Ștefan Talpalaru
de295619be
assert() -> doAssert() 2019-03-14 00:04:43 +01:00
Jacek Sieka
b0f4034060
implement fork choice (#175)
* keep track of a finalized block
* keep track of all justified blocks
* use naive spec version of LMD ghost
* cache slot number and a few more things in BlockRef
* keep track of the latest vote of each validator
* depend less on the state of node.state (it's a cache, effectively)
2019-03-13 16:59:20 -06:00
Bruno Škvorc
e69702bab5 Use environment variables (#176)
Passing along no-name command line arguments when using make (from Nimbus) would need make target revamps that are a bit clumsy AFAIK, so using environment variables in the spirit of USE_MULTITAIL to get the number of validators and nodes makes more sense to me.

This lets us run the sim from nim-beacon-chain with

```bash
VALIDATORS=512 NODES=100 USE_MULTITAIL="yes" tests/simulation/start.sh
```

and from Nimbus with:

```bash
VALIDATORS=512 NODES=100 USE_MULTITAIL="yes" make eth2_network_simulation
```

Also makes for a more verbose launch process.
2019-03-13 06:44:30 -06:00
Dustin Brody
1479bae22f implement distinct Epoch type and continue 0.4.0 spec update (#173)
* implement distinct Epoch type
* update two more spec 0.4.0 markers, leaving just deposit processing
2019-03-12 19:46:44 -06:00
Dustin Brody
50a0948bc0 Mostly convert to using distinct Slot type (#172) 2019-03-12 16:21:32 -06:00