Commit Graph

47 Commits

Author SHA1 Message Date
tersec 1d7fb2ed0c
remove {.inline.} pragmas (#2033)
* remove {.inline.} pragmas

* re-add inline on bitseqs functions and tweak inlining threshold

* remove macOS/LLVM inlining setting; revert non-init/module-local/tests inline pragma removal
2020-11-20 11:00:22 +01:00
Zahary Karadjov 14b2d4324d openarray -> openArray 2020-11-03 23:23:10 +02:00
Jacek Sieka b246d51ac1 test fix 2020-10-23 19:23:36 +03:00
Jacek Sieka 61538fa581 speed up shuffling
Replace shuffling function with zrnt version - `get_shuffled_seq` in
particular puts more strain on the GC by allocating superfluous seq's
which turns out to have a significant impact on block processing (when
replaying blocks for example) - 4x improvement on non-epoch, 1.5x on
epoch blocks (replay is done without signature checking)

Medalla, first 10k slots - pre:

```
Loaded 68973 blocks, head slot 117077
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
76855.848,        0.000,    76855.848,    76855.848,            1,
Initialize DB
1.073,        0.914,        0.071,       12.454,         7831,
Load block from database
31.382,        0.000,       31.382,       31.382,            1,
Load state from database
85.644,       30.350,        3.056,      466.136,         7519,
Apply block
506.569,       91.129,      130.654,      874.786,          312,
Apply epoch block
```

post:

```
Loaded 68973 blocks, head slot 117077
All time are ms
Average,       StdDev,          Min,          Max,      Samples,
Test
Validation is turned off meaning that no BLS operations are performed
72457.303,        0.000,    72457.303,    72457.303,            1,
Initialize DB
1.015,        0.858,        0.070,       11.231,         7831,
Load block from database
28.983,        0.000,       28.983,       28.983,            1,
Load state from database
21.725,       17.461,        2.659,      393.217,         7519,
Apply block
324.012,       33.954,       45.452,      440.532,          312,
Apply epoch block
```
2020-08-21 16:05:10 +03:00
Dustin Brody 044ced53be ensure SPEC_VERSION stays synchronized with actual spec tested 2020-06-16 15:43:40 +00:00
Dustin Brody 8f7541959a allow compile-time switching between v0.11.3 and v0.12.1 2020-06-09 18:39:05 +00:00
Dustin Brody cc13153fc6 initial 0.12.1 beacon chain spec 2020-06-09 18:39:05 +00:00
Zahary Karadjov 1fd87a2255 Remove SszReader.maxObjectSize 2020-05-28 19:15:16 +03:00
Jacek Sieka 693bc15919 avoid some RVO bugs 2020-05-28 19:15:16 +03:00
Jacek Sieka f53b55cbe0 SSZ cleanup
* be stricter about SSZ length prefix
* compute zeroHash list at compile time
* remove SSZ schema stuff
* move SSZ navigation to ncli
* cleanup a few leftover openArray uses
2020-05-28 19:15:16 +03:00
Dustin Brody 6c941b1ff7 spec v0.11.3 update 2020-05-19 17:16:55 +00:00
tersec 8432932c11
update to spec v0.11.2 to keep compatible with Schlesi (#1036)
* update to spec v0.11.2 to keep compatible with Schlesi

* update spec/state_transition_epoch spec references to v0.11.2

* bump other spec refs
2020-05-18 23:05:27 +00:00
Zahary Karadjov 7e846a0bce Implement sszSize 2020-05-13 12:18:42 +03:00
tersec f5f939bd31
0.11.1 beacon chain spec update (#836)
* initial 0.11.1 spec commit; no test regressions and finalizes in eth2_network_simulation

* with BLS 0.10/0.11 available, stop skipping attester slashing, proposer slashing, and voluntary exist operations fixture tests

* switch param orders to group state.{fork, genesis_validators_root}; bump spec/datatypes spec version for network purposes

* mark attestation construction and broadcast and some minimal/mainnet constants as 0.11.1-compatible; remove phase 1 sharding constants from minimal which don't exist in that preset
2020-03-30 11:31:44 +00:00
Dustin Brody 6068dd960b initial 0.11.0 spec version update 2020-03-17 12:06:56 +02:00
Mamy André-Ratsimbazafy 37446302b2 Use the new nim-eth2-scenarios path for the test vectors 2020-02-17 15:26:11 +00:00
Mamy Ratsimbazafy 1ba2cebd85 v0.10.1 (test vectors + unchanged parts) (#696)
* update to 0.10.1

* SSZ Generic and nbench uses the v0.10.1 fixtures

* Tests + spec links: v0.10.0 -> v0.10.1

* Add v0.10.1 TODO in get_latest_attesting_balance (forkchoice)

* SSZ Bytes are now ByteList

* Remove nim-result submodules that was leftover/added by mistake in the branch
2020-01-27 10:56:32 +00:00
Dustin Brody 8eaebf7163 update to spec version 0.10.0 2020-01-14 18:17:16 +00:00
Dustin Brody 3e7d56a66f update test suite to 0.9.4 vectors 2019-12-20 16:37:36 +00:00
Dustin Brody c824416f56 initial 0.9.3 spec update 2019-12-17 00:06:03 +02:00
Dustin Brody 1494bcc262 replace links to Apache and MIT licenses with HTTPS versions (#592) 2019-11-25 16:30:02 +01:00
Mamy Ratsimbazafy 22adccc917
support List in reportDiff and comment out BLS and shuffling tests (#585)
* support List in report diff

* Json tests were removed but walkDirRec lull-ed us into thinking we were passing them :/
2019-11-24 11:46:45 +01:00
Dustin Brody edfd65fd5d stop download 0.9.1 test vectors; finish 0.9.2 update, primarily via fixing preset constants; remove a couple 0.19.6 shims 2019-11-22 16:41:43 +00:00
Dustin Brody 8e259368a3 flip all but 3 test fixtures to 0.9.1 test vectors 2019-11-12 14:49:26 +00:00
Dustin Brody 275fb22c00 flip targeted spec version and ssz test fixtures to 0.9.0; almost completely update BeaconState to 0.9.0; update minimal and mainnet preset constants to 0.9.0; update process_proposer_slashing(...), get_attestation_deltas(...), and get_matching_head_attestations(...) to 0.9.0; mark process_final_updates(...) as 0.9.0 2019-11-08 13:23:08 +00:00
Dustin Brody 55dac3e36f re-enable some tests on Win64 since Nim 1.0.2 fixes https://github.com/nim-lang/Nim/issues/12186 (#513) 2019-11-01 09:46:50 +01:00
Mamy Ratsimbazafy 753d5af447
Test vectors: use tarballs, allow multiple spec versions for progressive spec updates (#514)
* update test submodule [skip ci]

* rename the process_lfs script to setup_official_tests [skip ci]

* clarify that the existing LFS is for json files [skip ci]

* Update again to use download/unpack function library

* update the test downloading script [skip ci]

* Update EF fixtures with new test path

* fix symlink order

* Fix relative:absolute symlink issue

* add symlink existence check

* setup_official_tests.sh: refactoring
2019-11-01 09:44:16 +01:00
Mamy Ratsimbazafy 88ec9fd4be "industrialize" skipping win64 CI (#441)
* "industrialize" skipping win64 CI
- reintroduce actually working tests
- skip BLS priv_to_pub (tested in blscurve + broken by #440 and #435)

* missing echo

* try/except OSError doesn't seem to work in unittest so plain ignore + move x64 first for fail fast

* ignore transfers completely in WIn64
2019-09-12 11:33:07 +00:00
Mamy Ratsimbazafy 297e9079d4 V0.8.3 tests part2 - BLS + Shuffling + State transition epoch (#372)
* Update BLS fixtures to 0.8.3

* Bump fixtures with shuffling with minimal preset

* Update shuffling tests

* parseTest generic over file format (Json or SSZ)

* Initial crosslink parsing commit for debugging Nim crash

* Workaround https://github.com/status-im/nim-beacon-chain/issues/369

* Crosslink test works for minimal - https://github.com/status-im/nim-beacon-chain/issues/369 is back on mainnet

* Use ref objects to workaround https://github.com/status-im/nim-beacon-chain/issues/369

* Generalize state transition epoch test to all epoch tests

* Fix slashing (potential uint64 overflow in previous spec)

* Add a state debugging macro to deeply inspect the wrong fields

* make reportDiff visible

* Improve the debug state macro for containers
2019-09-03 03:12:09 +00:00
Mamy Ratsimbazafy 1ec4c5aef7
Prepare for v0.8.3 tests: submodules + LFS script + Appveyor updates (#368)
* Update fixtures to 0.8.3

* v0.8.1 tests use new test path

* LFS caching: handle multiple versions of json_tests

* Enable paths > 260 characters in appveyor

* Appveyor: use git clone core.longpaths + the env variable at startup

* improve process_lfs.sh and cleanup submodules
2019-08-30 17:50:46 +02:00
Mamy André-Ratsimbazafy 9cfc153620
Remove obselete mocking helper skeleton 2019-08-29 00:36:37 +02:00
Zahary Karadjov 398ea55801
Implement the latest SSZ specification and integrate the official SSZ test suite 2019-08-05 03:18:48 +03:00
Dustin Brody dfa062db7d 0.8.0, continued (#315)
* rename get_genesis_beacon_state(...) to initialize_beacon_state_from_eth1(...); remove unused vestiges get_temporary_block_header(...) and get_empty_block(...); partly align initialize_beacon_state_from_eth1(...) with 0.8.0 version; implement CompactCommittee object type; update process_final_updates(...) to 0.8.0

* mark get_shard_delta(...) as 0.8.0

* mark get_total_active_balance(...) and epoch transition helper functions as 0.8.0

* move FORK_CHOICE_BALANCE_INCREMENT, not in spec anymore, to sole user in fork_choice
2019-07-10 14:23:02 +02:00
Jacek Sieka b9e62a1f6c
std_shims -> stew 2019-07-07 11:53:58 +02:00
Dustin Brody 68be98cafa
More 0.8.0 updates (#311)
* replace BeaconState.finalized_{epoch,root} with BeaconState.finalized_checkpoint; rename get_delayed_activation_exit_epoch(...) to compute_activation_exit_epoch(...) and mark as 0.8.0; update get_churn_limit(...)/get_validator_churn_limit(...) to 0.8.0; update process_registry_updates(...) to 0.8.0

* update process_crosslinks(...) to 0.8.0; mark compute_start_slot_of_epoch(...) and get_committee_count(...) as 0.8.0

* mark Fork, is_slashable_validator(...), and get_beacon_proposer_index(...) as 0.8.0

* rename LATEST_SLASHED_EXIT_LENGTH to EPOCHS_PER_SLASHINGS_VECTOR; update process_slashings(...) to 0.8.0; remove pointless type conversion warning in get_previous_epoch(...)

* convert remaining references to finalized_epoch to finalized_checkpoint.epoch

* update slash_validator(...) to 0.8.0; mark inital value, Gwei, and time constants as 0.8.0; mark hash(...) and processBlockHeader(...) as 0.8.0

* rename WHISTLEBLOWING_REWARD_QUOTIENT to WHISTLEBLOWER_REWARD_QUOTIENT; rename LATEST_ACTIVE_INDEX_ROOTS_LENGTH to EPOCHS_PER_HISTORICAL_VECTOR (randao will also get merged into this); remove get_active_index_root(...); mark time parameter, signature domain types, and max operations per block constants as 0.8.0; update rewards and penalties constants to 0.8.0

* update is_valid_indexed_attestation(...) to 0.8.0; mark process_slot(...) as 0.8.0

* replace BeaconState.{current,previous}_justified_{epoch,root} with BeaconState.{current,previous}_justified_checkpoint
2019-07-05 08:30:05 +00:00
Dustin Brody e4321dc4ed Initial frozen phase 0/v0.8.0 spec (#305)
* rename slot_to_epoch(...) to compute_epoch_of_slot(...)

* rename aggregation_bitfield/custody_bitfield fields to aggregation_bits/custody_bits; rename convert_to_indexed(...) to get_indexed_attestation(...); mark BeaconBlockHeader as 0.8.0; update minimal preset MIN_ATTESTATION_INCLUSION_DELAY/time parameters in general to 0.8

* fix beacon node compilation; it used slightly different capitalizations of slot_to_epoch/slotToEpoch

* mark integer_squareroot(...), Deposit, VoluntaryExit, and Transfer as 0.8.0; rename get_epoch_start_slot(...) to compute_start_slot_of_epoch(...)

* add new Domain alias for uint64; rename bls_domain(...) to compute_domain(...), MAX_INDICES_PER_ATTESTATION to MAX_VALIDATORS_PER_COMMITTEE, and SignatureDomain to DomainType; update get_domain(...) to 0.8.0

* update/mark initiate_validator_exit(...) and Crosslink as 0.8.0; rename BeaconState.latest_block_roots -> BeaconState.block_roots; mark HistoricalBatch as 0.8.0
2019-07-01 09:53:42 +02:00
Mamy Ratsimbazafy 0926ebf8fe [Tests] SSZ unsigned integer + test suite genericity (#284)
* Generics over tests (https://github.com/status-im/nim-serialization/issues/4, https://github.com/status-im/nim-serialization/issues/5, https://github.com/nim-lang/Nim/issues/11225)

* Skeleton of SSZ uint tests

* Check all primitive uint types

* Add deserialization test. "wrong length" skipped due to https://github.com/status-im/nim-beacon-chain/issues/280

* Move test types to their specific test files

* Stint also sometimes throws an AssertionError for invalid ranges

* Add debug path for Access denied issue in Appveyor 64-bit (https://ci.appveyor.com/project/nimbus/nim-beacon-chain/builds/25278666/job/fs8q0bcluvj2gdor#L866)

* indent the Appveyor debug info
2019-06-17 09:29:23 +00:00
Dustin Brody e9180c490e
more 0.7.0 updates (#287)
* update Deposit and BeaconBlockHeader to 0.7.0

* update get_total_balance(...) and get_base_reward(...) to 0.7.0

* rename initial value constant BLS_WITHDRAWAL_PREFIX_BYTE to BLS_WITHDRAWAL_PREFIX

* mark minimal presets as entirely migrated to 0.7.0; update process_rewards_and_penalties(...) to 0.7.0

* specify that targeting 0.7.0; adopt 0.7.0 refactoring of bls_domain/get_domain to update get_domain(...) to 0.7.0; mark that mainnet constant presets and block header processing are 0.7.0-consistent; update process_justification_and_finalization to 0.7.0 to fix potential underflow errors

* address mratsim's comment to copy whole spec comment
2019-06-14 16:21:04 +00:00
Dustin Brody 6357bc2a98 0.7.0 updates which don't change semantics; mostly comment changes (#281)
* 0.7.0 updates which don't change semantics; mostly comment changes

* mark get_attesting_indices, bls_verify, ProposerSlashing, BeaconBlockBody, deposit contract constants, state list length constants, compute_committee, get_crosslink_committee, is_slashable_attestation_data, processAttesterSlashings as 0.7.0; rename BASE_REWARD_QUOTIENT to BASE_REWARD_FACTOR

* complete marking unchanged-in-0.7.0 datatypes as such; a few more have trivial field renamings, etc
2019-06-13 09:40:58 +02:00
Dustin Brody 2e1515b107
0.6.2 updates (#275)
* update process_justification_and_finalization to 0.6.2; mark AttesterSlashing as 0.6.2

* replace get_effective_balance(...) with state.validator_registry[idx].effective_balance; rm get_effective_balance, process_ejections, should_update_validator_registry, update_validator_registry, and update_registry_and_shuffling_data; update get_total_balance to 0.6.2; implement process_registry_updates

* rm exit_validator; implement is_slashable_attestation_data; partly update processAttesterSlashings

* mark HistoricalBatch and Eth1Data as 0.6.2; implement get_shard_delta(...); replace 0.5 finish_epoch_update with 0.6 process_final_updates

* mark increase_balance, decrease_balance, get_delayed_activation_exit_epoch, bls_aggregate_pubkeys, bls_verify_multiple, Attestation, Transfer, slot_to_epoch, Crosslink, get_current_epoch, int_to_bytes*, various constants, processEth1Data, processTransfers, and verifyStateRoot as 0.6.2; rm is_double_vote and is_surround_vote

* mark get_bitfield_bit, verify_bitfield, ProposerSlashing, DepositData, VoluntaryExit, PendingAttestation, Fork, integer_squareroot, get_epoch_start_slot, is_active_validator, generate_seed, some constants to 0.6.2; rename MIN_PENALTY_QUOTIENT to MIN_SLASHING_PENALTY_QUOTIENT

* rm get_previous_total_balance, get_current_epoch_boundary_attestations, get_previous_epoch_boundary_attestations, and get_previous_epoch_matching_head_attestations

* update BeaconState to 0.6.2; simplify legacy get_crosslink_committees_at_slot infrastructure a bit by noting that registry_change is always false; reimplment 0.5 get_crosslink_committees_at_slot in terms of 0.6 get_crosslink_committee

* mark process_deposit(...), get_block_root_at_slot(...), get_block_root(...), Deposit, BeaconBlockHeader, BeaconBlockBody, hash(...), get_active_index_root(...), various constants, get_shard_delta(...), get_epoch_start_shard(...), get_crosslink_committee(...), processRandao(...), processVoluntaryExits(...), cacheState(...) as 0.6.2

* rm removed-since-0.5 split(...), is_power_of_2(...), get_shuffling(...); rm 0.5 versions of get_active_validator_indices and get_epoch_committee_count; add a few tests for integer_squareroot

* mark bytes_to_int(...) and advanceState(...) as 0.6.2

* rm 0.5 get_attesting_indices; update get_attesting_balance to 0.6.2

* another tiny commit to poke AppVeyor to maybe not timeout at connecting to GitHub partway through CI: mark get_churn_limit(...), initiate_validator_exit(...), and Validator as 0.6.2

* mark get_attestation_slot(...), AttestationDataAndCustodyBit, and BeaconBlock as 0.6.2
2019-06-03 10:31:04 +00:00
Dustin Brody 6a47ca3c46
More 0.6.1 updates (#274)
* implement get_churn_limit from 0.6.1; update initiate_validator_exit and get_previous_epoch to 0.6.1; remove obsoleted/non-spec reduce_balance, replaced by decrease_balance; mark BeaconBlock as 0.6.1

* rename get_block_root to get_block_root_at_slot and introduce epoch-oriented get_block_root; implement get_attestation_slot, get_matching_source_attestations, get_matching_target_attestations, get_matching_head_attestations, 0.6.1 get_attesting_indices, get_unslashed_attesting_indices, get_attestation_deltas, process_rewards_and_penalties; rm get_inactivity_penalty

* update Validator and processVoluntaryExits to 0.6.1; rm obsolete inclusion_slots/inclusion_distances

* rm removed activate_validator; mark DepositData, misc values, Gwei values, Randao processing, state caching as 0.6.1; update GENESIS_SLOT to 64, since it doesn't quite yet work at 0

* mark BeaconBlockHeader as 0.6.1; update BeaconBlockBody to 0.6.1; rename WHISTLEBLOWER_REWARD_QUOTIENT to WHISTLEBLOWING_REWARD_QUOTIENT; update slash_validator to 0.6.1

* implement 0.6.2 is_slashable_validator; update processProposerSlashings to 0.6.2; mark state caching as 0.6.2

* mark get_total_active_balance and process_slashings as 0.6.2; update get_base_reward to 0.6.2

* rm prepare_validator_for_withdrawal, process_exit_queue; mark mainnet misc constants as 0.6.2; update process block header to 0.6.2

* address mratsim's code review comment
2019-05-29 10:08:56 +00:00
Mamy Ratsimbazafy 2a4c4df7c3
Bls official tests (#268)
* Update fixtures with BLS and SSZ generic

* Stash changes following https://github.com/status-im/nim-serialization/issues/4

* Add Private to Public key conversion

* Add message signing tests

* Add aggregate signatures and public keys

* Add BLS to test suite
2019-05-23 09:41:17 +02:00
Mamy Ratsimbazafy e5047d93a4 [Tests] SHA256 + official shuffling vectors (#263)
* Change digests to SHA2-256 (from Keccak256)

* Fix sha256 digest initialization - pass shuffling test

* Add comments + remove old shuffling tests

* Remove stale tree hashing tests

* Small toOpenArray optim + notes on in-place hashing

* Revert "Revert "Update test repo""

This reverts commit f385467cd3.

* Revert "Revert "Fix SIGFPE on shuffling for 0""

This reverts commit 226d380f87.

* Revert "Revert "Implement shuffling test (pending typedesc and shuffling algo fix)""

This reverts commit 813cb6fbb8.

* withEth2hash templates now needs ctx.init()

* Use init-update-finish to avoid burnMem
2019-05-10 08:14:01 +00:00
Mamy André-Ratsimbazafy 813cb6fbb8
Revert "Implement shuffling test (pending typedesc and shuffling algo fix)"
This reverts commit ad9f643873.
2019-05-09 12:06:45 +02:00
Mamy André-Ratsimbazafy ad9f643873
Implement shuffling test (pending typedesc and shuffling algo fix) 2019-05-09 12:02:52 +02:00
Mamy Ratsimbazafy ad1803bdd6
Submodule official tests (#238)
* Remove eth2.0-tests submodule

* Remove files

* Add submodule https://github.com/status-im/nim-eth2-official-tests

* Update tests path

* update submodule version

* Move beacon-chain specific datastructures out of the submodule
2019-04-08 10:29:44 +02:00
Mamy Ratsimbazafy ad133a0222
Official state tests - parsing and loading beacon state (#227)
* initial commit of official state tests

* sanity check fixture

* Parsing official state test is mostly working
(Except BLS signature)

* Successfully load state test

* Use json-serialization instead of json and display deserialized and from scratch beacon state hashes

* Add official state test as a smoke parsing test
2019-04-02 16:50:24 +02:00