status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/tests/consensus_spec/test_fixture_rewards.nim

129 lines
5.0 KiB
Nim
Raw Normal View History

fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
# beacon_chain
avoid `-Present` in copyrights (#2935) Replaced non-conventional `-Present` form in copyright headers with the more appropriate `-2021` version that is already used by the majority of the code base. Copyright header should indicate the years during which the specific file was modified.
2021-09-30 16:28:02 +00:00
# Copyright (c) 2020-2021 Status Research & Development GmbH
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
# Licensed and distributed under either of
# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
# at your option. This file may not be copied, modified, or distributed except according to those terms.
{.used.}
import
# Standard library
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
os,
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
# Utilities
stew/results,
# Beacon chain internals
rework spec imports (#2779) The spec imports are a mess to work with, so this branch cleans them up a bit to ensure that we avoid generic sandwitches and that importing stuff generally becomes easier. * reexport crypto/digest/presets because these are part of the public symbol set of the rest of the spec types * don't export `merge` types from `base` - this causes circular deps * fix circular deps in `ssz/spec_types` - this is the first step in disentangling ssz from spec * be explicit about phase0 vs altair - longer term, `altair` will become the "natural" type set, then merge and so on, so no point in giving `phase0` special preferential treatment
2021-08-12 13:08:20 +00:00
../../beacon_chain/spec/[validator, helpers, state_transition_epoch],
reward accounting for altair+ (#2981) Similar to the existing `RewardInfo`, this PR adds the infrastructure needed to export epoch processing information from altair+. Because accounting is done somewhat differently, the PR uses a fork-specific object to extrct the information in order to make the cost on the spec side low. * RewardInfo -> EpochInfo, ForkedEpochInfo * use array for computing new sync committee * avoid repeated total active balance computations in block processing * simplify proposer index check * simplify epoch transition tests * pre-compute base increment and reuse in epoch processing, and a few other small optimizations This PR introduces the type and does the heavy lifting in terms of refactoring - the tools that use the accounting will need separate PR:s (as well as refinements to the exportred information)
2021-10-13 14:24:36 +00:00
../../beacon_chain/spec/datatypes/phase0,
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
# Test utilities
../testutil,
./fixtures_utils
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
const
RewardsDirBase = SszTestsDir/const_preset/"phase0"/"rewards"
RewardsDirBasic = RewardsDirBase/"basic"/"pyspec_tests"
RewardsDirLeak = RewardsDirBase/"leak"/"pyspec_tests"
RewardsDirRandom = RewardsDirBase/"random"/"pyspec_tests"
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
update spec references from eth2.0-specs to consensus-specs and to v1.1.0-beta.2 (#2822)
2021-08-26 08:21:52 +00:00
# https://github.com/ethereum/consensus-specs/tree/v1.0.1/tests/formats/rewards#rewards-tests
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
type Deltas = object
rewards: List[uint64, Limit VALIDATOR_REGISTRY_LIMIT]
penalties: List[uint64, Limit VALIDATOR_REGISTRY_LIMIT]
add attestation stats tool to ncli_db (#2539) This also makes future efforts to provide metrics and logs for attestation efficiency easier * Export rewards from epoch transition * Use less memory for reward calculation (bool -> set[enum], field alignment) * Reuse reward memory when replaying, avoiding spike * Allow replaying any range in ncli_db benchmark
2021-05-07 11:36:21 +00:00
func add(v: var Deltas, idx: int, delta: RewardDelta) =
speed up reward/penalty calculation Calculating rewards/penalties is slow due to how we compute sets of attestations validators then use the sets for inclusion checks, to see who attested. The dominant function during validated block processing / epoch processing is hash set building and lookup. This PR inverts the flow by removing the sets and creating a single large validator status list, then applying all relevant state attestations, then updating rewards and penalties. This provides a 10x speedup to epoch processing which in turn speeds up both empty slot and block processing - for example, on startup, we replay all non-finalized blocks to prime fork choice - the same when validating attestations or replaying states on reorg.
2020-10-22 11:08:46 +00:00
v.rewards[idx] += delta.rewards
v.penalties[idx] += delta.penalties
func init(T: type Deltas, len: int): T =
Reset cached indices when resetting cache on SSZ read (#2480) * Reset cached indices when resetting cache on SSZ read When deserializing into an existing structure, the cache should be cleared - goes for json also. Also improve error messages.
2021-04-08 10:11:04 +00:00
if not result.rewards.setLen(len):
raiseAssert "setLen"
if not result.penalties.setLen(len):
raiseAssert "setLen"
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
proc runTest(rewardsDir, identifier: string) =
let testDir = rewardsDir / identifier
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
proc `testImpl _ rewards _ identifier`() =
Introduce unittest2 and junit reports (#2522) * Introduce unittest2 and junit reports * fix XML path * don't combine multiple CI runs * fixup * public combined report also Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 16:41:02 +00:00
test "Rewards" & " - " & identifier & preset():
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
var
rework spec imports (#2779) The spec imports are a mess to work with, so this branch cleans them up a bit to ensure that we avoid generic sandwitches and that importing stuff generally becomes easier. * reexport crypto/digest/presets because these are part of the public symbol set of the rest of the spec types * don't export `merge` types from `base` - this causes circular deps * fix circular deps in `ssz/spec_types` - this is the first step in disentangling ssz from spec * be explicit about phase0 vs altair - longer term, `altair` will become the "natural" type set, then merge and so on, so no point in giving `phase0` special preferential treatment
2021-08-12 13:08:20 +00:00
state = newClone(parseTest(testDir/"pre.ssz_snappy", SSZ, phase0.BeaconState))
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
cache = StateCache()
let
use v1.1.0 phase 0 test vectors (#2592) * use v1.1.0 phase 0 test vectors * re-export process_final_updates()
2021-05-24 08:42:40 +00:00
sourceDeltas =
parseTest(testDir/"source_deltas.ssz_snappy", SSZ, Deltas)
targetDeltas =
parseTest(testDir/"target_deltas.ssz_snappy", SSZ, Deltas)
headDeltas = parseTest(testDir/"head_deltas.ssz_snappy", SSZ, Deltas)
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
inclusionDelayDeltas =
use v1.1.0 phase 0 test vectors (#2592) * use v1.1.0 phase 0 test vectors * re-export process_final_updates()
2021-05-24 08:42:40 +00:00
parseTest(testDir/"inclusion_delay_deltas.ssz_snappy", SSZ, Deltas)
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
inactivityPenaltyDeltas =
use v1.1.0 phase 0 test vectors (#2592) * use v1.1.0 phase 0 test vectors * re-export process_final_updates()
2021-05-24 08:42:40 +00:00
parseTest(testDir/"inactivity_penalty_deltas.ssz_snappy", SSZ, Deltas)
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
speed up reward/penalty calculation Calculating rewards/penalties is slow due to how we compute sets of attestations validators then use the sets for inclusion checks, to see who attested. The dominant function during validated block processing / epoch processing is hash set building and lookup. This PR inverts the flow by removing the sets and creating a single large validator status list, then applying all relevant state attestations, then updating rewards and penalties. This provides a 10x speedup to epoch processing which in turn speeds up both empty slot and block processing - for example, on startup, we replay all non-finalized blocks to prime fork choice - the same when validating attestations or replaying states on reorg.
2020-10-22 11:08:46 +00:00
var
reward accounting for altair+ (#2981) Similar to the existing `RewardInfo`, this PR adds the infrastructure needed to export epoch processing information from altair+. Because accounting is done somewhat differently, the PR uses a fork-specific object to extrct the information in order to make the cost on the spec side low. * RewardInfo -> EpochInfo, ForkedEpochInfo * use array for computing new sync committee * avoid repeated total active balance computations in block processing * simplify proposer index check * simplify epoch transition tests * pre-compute base increment and reuse in epoch processing, and a few other small optimizations This PR introduces the type and does the heavy lifting in terms of refactoring - the tools that use the accounting will need separate PR:s (as well as refinements to the exportred information)
2021-10-13 14:24:36 +00:00
info: phase0.EpochInfo
speed up reward/penalty calculation Calculating rewards/penalties is slow due to how we compute sets of attestations validators then use the sets for inclusion checks, to see who attested. The dominant function during validated block processing / epoch processing is hash set building and lookup. This PR inverts the flow by removing the sets and creating a single large validator status list, then applying all relevant state attestations, then updating rewards and penalties. This provides a 10x speedup to epoch processing which in turn speeds up both empty slot and block processing - for example, on startup, we replay all non-finalized blocks to prime fork choice - the same when validating attestations or replaying states on reorg.
2020-10-22 11:08:46 +00:00
finality_delay = (state[].get_previous_epoch() - state[].finalized_checkpoint.epoch)
reward accounting for altair+ (#2981) Similar to the existing `RewardInfo`, this PR adds the infrastructure needed to export epoch processing information from altair+. Because accounting is done somewhat differently, the PR uses a fork-specific object to extrct the information in order to make the cost on the spec side low. * RewardInfo -> EpochInfo, ForkedEpochInfo * use array for computing new sync committee * avoid repeated total active balance computations in block processing * simplify proposer index check * simplify epoch transition tests * pre-compute base increment and reuse in epoch processing, and a few other small optimizations This PR introduces the type and does the heavy lifting in terms of refactoring - the tools that use the accounting will need separate PR:s (as well as refinements to the exportred information)
2021-10-13 14:24:36 +00:00
info.init(state[])
info.process_attestations(state[], cache)
speed up reward/penalty calculation Calculating rewards/penalties is slow due to how we compute sets of attestations validators then use the sets for inclusion checks, to see who attested. The dominant function during validated block processing / epoch processing is hash set building and lookup. This PR inverts the flow by removing the sets and creating a single large validator status list, then applying all relevant state attestations, then updating rewards and penalties. This provides a 10x speedup to epoch processing which in turn speeds up both empty slot and block processing - for example, on startup, we replay all non-finalized blocks to prime fork choice - the same when validating attestations or replaying states on reorg.
2020-10-22 11:08:46 +00:00
let
reward accounting for altair+ (#2981) Similar to the existing `RewardInfo`, this PR adds the infrastructure needed to export epoch processing information from altair+. Because accounting is done somewhat differently, the PR uses a fork-specific object to extrct the information in order to make the cost on the spec side low. * RewardInfo -> EpochInfo, ForkedEpochInfo * use array for computing new sync committee * avoid repeated total active balance computations in block processing * simplify proposer index check * simplify epoch transition tests * pre-compute base increment and reuse in epoch processing, and a few other small optimizations This PR introduces the type and does the heavy lifting in terms of refactoring - the tools that use the accounting will need separate PR:s (as well as refinements to the exportred information)
2021-10-13 14:24:36 +00:00
total_balance = info.total_balances.current_epoch
speed up reward/penalty calculation Calculating rewards/penalties is slow due to how we compute sets of attestations validators then use the sets for inclusion checks, to see who attested. The dominant function during validated block processing / epoch processing is hash set building and lookup. This PR inverts the flow by removing the sets and creating a single large validator status list, then applying all relevant state attestations, then updating rewards and penalties. This provides a 10x speedup to epoch processing which in turn speeds up both empty slot and block processing - for example, on startup, we replay all non-finalized blocks to prime fork choice - the same when validating attestations or replaying states on reorg.
2020-10-22 11:08:46 +00:00
total_balance_sqrt = integer_squareroot(total_balance)
var
sourceDeltas2 = Deltas.init(state[].validators.len)
targetDeltas2 = Deltas.init(state[].validators.len)
headDeltas2 = Deltas.init(state[].validators.len)
inclusionDelayDeltas2 = Deltas.init(state[].validators.len)
inactivityPenaltyDeltas2 = Deltas.init(state[].validators.len)
reward accounting for altair+ (#2981) Similar to the existing `RewardInfo`, this PR adds the infrastructure needed to export epoch processing information from altair+. Because accounting is done somewhat differently, the PR uses a fork-specific object to extrct the information in order to make the cost on the spec side low. * RewardInfo -> EpochInfo, ForkedEpochInfo * use array for computing new sync committee * avoid repeated total active balance computations in block processing * simplify proposer index check * simplify epoch transition tests * pre-compute base increment and reuse in epoch processing, and a few other small optimizations This PR introduces the type and does the heavy lifting in terms of refactoring - the tools that use the accounting will need separate PR:s (as well as refinements to the exportred information)
2021-10-13 14:24:36 +00:00
for index, validator in info.statuses.mpairs():
speed up reward/penalty calculation Calculating rewards/penalties is slow due to how we compute sets of attestations validators then use the sets for inclusion checks, to see who attested. The dominant function during validated block processing / epoch processing is hash set building and lookup. This PR inverts the flow by removing the sets and creating a single large validator status list, then applying all relevant state attestations, then updating rewards and penalties. This provides a 10x speedup to epoch processing which in turn speeds up both empty slot and block processing - for example, on startup, we replay all non-finalized blocks to prime fork choice - the same when validating attestations or replaying states on reorg.
2020-10-22 11:08:46 +00:00
if not is_eligible_validator(validator):
continue
let
base_reward = get_base_reward_sqrt(
state[], index.ValidatorIndex, total_balance_sqrt)
sourceDeltas2.add(index, get_source_delta(
reward accounting for altair+ (#2981) Similar to the existing `RewardInfo`, this PR adds the infrastructure needed to export epoch processing information from altair+. Because accounting is done somewhat differently, the PR uses a fork-specific object to extrct the information in order to make the cost on the spec side low. * RewardInfo -> EpochInfo, ForkedEpochInfo * use array for computing new sync committee * avoid repeated total active balance computations in block processing * simplify proposer index check * simplify epoch transition tests * pre-compute base increment and reuse in epoch processing, and a few other small optimizations This PR introduces the type and does the heavy lifting in terms of refactoring - the tools that use the accounting will need separate PR:s (as well as refinements to the exportred information)
2021-10-13 14:24:36 +00:00
validator, base_reward, info.total_balances, finality_delay))
speed up reward/penalty calculation Calculating rewards/penalties is slow due to how we compute sets of attestations validators then use the sets for inclusion checks, to see who attested. The dominant function during validated block processing / epoch processing is hash set building and lookup. This PR inverts the flow by removing the sets and creating a single large validator status list, then applying all relevant state attestations, then updating rewards and penalties. This provides a 10x speedup to epoch processing which in turn speeds up both empty slot and block processing - for example, on startup, we replay all non-finalized blocks to prime fork choice - the same when validating attestations or replaying states on reorg.
2020-10-22 11:08:46 +00:00
targetDeltas2.add(index, get_target_delta(
reward accounting for altair+ (#2981) Similar to the existing `RewardInfo`, this PR adds the infrastructure needed to export epoch processing information from altair+. Because accounting is done somewhat differently, the PR uses a fork-specific object to extrct the information in order to make the cost on the spec side low. * RewardInfo -> EpochInfo, ForkedEpochInfo * use array for computing new sync committee * avoid repeated total active balance computations in block processing * simplify proposer index check * simplify epoch transition tests * pre-compute base increment and reuse in epoch processing, and a few other small optimizations This PR introduces the type and does the heavy lifting in terms of refactoring - the tools that use the accounting will need separate PR:s (as well as refinements to the exportred information)
2021-10-13 14:24:36 +00:00
validator, base_reward, info.total_balances, finality_delay))
speed up reward/penalty calculation Calculating rewards/penalties is slow due to how we compute sets of attestations validators then use the sets for inclusion checks, to see who attested. The dominant function during validated block processing / epoch processing is hash set building and lookup. This PR inverts the flow by removing the sets and creating a single large validator status list, then applying all relevant state attestations, then updating rewards and penalties. This provides a 10x speedup to epoch processing which in turn speeds up both empty slot and block processing - for example, on startup, we replay all non-finalized blocks to prime fork choice - the same when validating attestations or replaying states on reorg.
2020-10-22 11:08:46 +00:00
headDeltas2.add(index, get_head_delta(
reward accounting for altair+ (#2981) Similar to the existing `RewardInfo`, this PR adds the infrastructure needed to export epoch processing information from altair+. Because accounting is done somewhat differently, the PR uses a fork-specific object to extrct the information in order to make the cost on the spec side low. * RewardInfo -> EpochInfo, ForkedEpochInfo * use array for computing new sync committee * avoid repeated total active balance computations in block processing * simplify proposer index check * simplify epoch transition tests * pre-compute base increment and reuse in epoch processing, and a few other small optimizations This PR introduces the type and does the heavy lifting in terms of refactoring - the tools that use the accounting will need separate PR:s (as well as refinements to the exportred information)
2021-10-13 14:24:36 +00:00
validator, base_reward, info.total_balances, finality_delay))
speed up reward/penalty calculation Calculating rewards/penalties is slow due to how we compute sets of attestations validators then use the sets for inclusion checks, to see who attested. The dominant function during validated block processing / epoch processing is hash set building and lookup. This PR inverts the flow by removing the sets and creating a single large validator status list, then applying all relevant state attestations, then updating rewards and penalties. This provides a 10x speedup to epoch processing which in turn speeds up both empty slot and block processing - for example, on startup, we replay all non-finalized blocks to prime fork choice - the same when validating attestations or replaying states on reorg.
2020-10-22 11:08:46 +00:00
let
(inclusion_delay_delta, proposer_delta) =
get_inclusion_delay_delta(validator, base_reward)
inclusionDelayDeltas2.add(index, inclusion_delay_delta)
inactivityPenaltyDeltas2.add(index, get_inactivity_penalty_delta(
validator, base_reward, finality_delay))
if proposer_delta.isSome:
let proposer_index = proposer_delta.get()[0]
inclusionDelayDeltas2.add(proposer_index.int, proposer_delta.get()[1])
improve slot processing speeds (#1670) about 40% better slot processing times (with LTO enabled) - these don't do BLS but are used heavily during replay (state transition = slot + block transition) tests using a recent medalla state and advancing it 1000 slots: ``` ./ncli slots --preState2:state-302271-3c1dbf19-c1f944bf.ssz --slot:1000 --postState2:xx.ssz ``` pre: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 39.236, 0.000, 39.236, 39.236, 1, Load state from file 0.049, 0.002, 0.046, 0.063, 968, Apply slot 256.504, 81.008, 213.471, 591.902, 32, Apply epoch slot 28.597, 0.000, 28.597, 28.597, 1, Save state to file ``` cast: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 37.079, 0.000, 37.079, 37.079, 1, Load state from file 0.042, 0.002, 0.040, 0.090, 968, Apply slot 215.552, 68.763, 180.155, 500.103, 32, Apply epoch slot 25.106, 0.000, 25.106, 25.106, 1, Save state to file ``` cast+rewards: ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 40.049, 0.000, 40.049, 40.049, 1, Load state from file 0.048, 0.001, 0.045, 0.060, 968, Apply slot 164.981, 76.273, 142.099, 477.868, 32, Apply epoch slot 28.498, 0.000, 28.498, 28.498, 1, Save state to file ``` cast+rewards+shr ``` All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 12.898, 0.000, 12.898, 12.898, 1, Load state from file 0.039, 0.002, 0.038, 0.054, 968, Apply slot 139.971, 68.797, 120.088, 428.844, 32, Apply epoch slot 24.761, 0.000, 24.761, 24.761, 1, Save state to file ```
2020-09-16 20:59:33 +00:00
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
check:
speed up reward/penalty calculation Calculating rewards/penalties is slow due to how we compute sets of attestations validators then use the sets for inclusion checks, to see who attested. The dominant function during validated block processing / epoch processing is hash set building and lookup. This PR inverts the flow by removing the sets and creating a single large validator status list, then applying all relevant state attestations, then updating rewards and penalties. This provides a 10x speedup to epoch processing which in turn speeds up both empty slot and block processing - for example, on startup, we replay all non-finalized blocks to prime fork choice - the same when validating attestations or replaying states on reorg.
2020-10-22 11:08:46 +00:00
sourceDeltas.rewards.asSeq == sourceDeltas2.rewards.asSeq
sourceDeltas.penalties.asSeq == sourceDeltas2.penalties.asSeq
targetDeltas.rewards.asSeq == targetDeltas2.rewards.asSeq
targetDeltas.penalties.asSeq == targetDeltas2.penalties.asSeq
headDeltas.rewards.asSeq == headDeltas2.rewards.asSeq
headDeltas.penalties.asSeq == headDeltas2.penalties.asSeq
inclusionDelayDeltas.rewards.asSeq == inclusionDelayDeltas2.rewards.asSeq
inclusionDelayDeltas.penalties.asSeq == inclusionDelayDeltas2.penalties.asSeq
inactivityPenaltyDeltas.rewards.asSeq == inactivityPenaltyDeltas2.rewards.asSeq
inactivityPenaltyDeltas.penalties.asSeq == inactivityPenaltyDeltas2.penalties.asSeq
fix get_unslashed_attesting_indices() and add official EF rewards tests for it (#1514)
2020-08-17 01:09:27 +00:00
`testImpl _ rewards _ identifier`()
add random tests and rename "Official" to "Ethereum Foundation" (#2826) * add random tests and rename "Official" to "Ethereum Foundation" * checkDir = true covers dirExists(...) * invalidate CI EF fixtures cache * more correct cache invalidation
2021-08-27 06:30:30 +00:00
suite "Ethereum Foundation - Rewards " & preset():
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
for rewardsDir in [RewardsDirBasic, RewardsDirLeak, RewardsDirRandom]:
add checkDir = true to walkDir() across tests (#2694) * add checkDir = true to walkDir() across tests
2021-07-02 17:33:59 +00:00
for kind, path in walkDir(rewardsDir, relative = true, checkDir = true):
minimal v0.12.3 update (#1716)
2020-09-21 15:58:35 +00:00
runTest(rewardsDir, path)