status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/beacon_chain/spec/helpers.nim

479 lines
20 KiB
Nim
Raw Normal View History

Add the pseudo renadom shuffle
2018-07-23 12:58:41 +00:00
# beacon_chain
spec ref URL & copyright year updates (#4467)
2023-01-06 16:28:46 +00:00
# Copyright (c) 2018-2023 Status Research & Development GmbH
Add the pseudo renadom shuffle
2018-07-23 12:58:41 +00:00
# Licensed and distributed under either of
replace links to Apache and MIT licenses with HTTPS versions (#592)
2019-11-25 15:30:02 +00:00
# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
Add the pseudo renadom shuffle
2018-07-23 12:58:41 +00:00
# at your option. This file may not be copied, modified, or distributed except according to those terms.
move spec stuff to spec directory helps maintain sanity in the face of spec changes
2018-11-28 19:49:03 +00:00
# Uncategorized helper functions from the spec
remove Nim 1.2-compatible `push raise`s and update copyright notice years (#4528)
2023-01-20 14:14:37 +00:00
{.push raises: [].}
spec: regulate exceptions (#913) * spec: regulate exceptions * a few more simple raises
2020-04-22 05:53:02 +00:00
Domain is now array[8, byte] instead of uint64
2019-09-25 17:07:08 +00:00
import
support merkle multiproof verification (#2980) This introduces spec-compatible support to verify merkle multiproofs for use, e.g., in light client syncing.
2021-10-14 08:28:22 +00:00
# Status libraries
add `num_active_participants` helpers for `SyncAggregate` (#4478) Introduce `num_active_participants` helper function to reduce visibility of low-level `countOnes` function and reduce code duplication.
2023-01-10 11:26:25 +00:00
stew/[byteutils, endians2, objects, saturation_arith],
fork choice cleanups (#3849) Removes a few extra-ambitious templates to make `self` updates explicit, and moves the `FinalityCheckpoints` type from `base` to `helpers` as it is an additional Nimbus specific type not defined by spec.
2022-07-10 15:26:29 +00:00
chronicles,
remove empty block fallback now that capella's on mainnnet (#4821) * remove empty block fallback now that capella's on mainnnet * build_empty_execution_payload is only testing infrastructure now * update error message
2023-04-18 09:21:15 +00:00
eth/common/[eth_types, eth_types_rlp],
bump `nim-eth`, extend empty block fallback for Capella (#4357) Implements `emptyPayloadToBlockHeader` for Capella. https://github.com/status-im/nim-eth/pull/562
2022-11-28 13:41:25 +00:00
eth/rlp, eth/trie/[db, hexary],
Domain is now array[8, byte] instead of uint64
2019-09-25 17:07:08 +00:00
# Internal
altair fork handling cleanups (#3050) * fix stack overflow crash in REST/debug/getStateV2 * introduce `ForkyXxx` for generic type matching of `Xxx` across branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) - `Some` is already used for other types of type classes * consolidate function naming in BeaconChainDB, use some generics * import `forks.nim` from other spec modules and move `Forked*` helpers around to resolve circular imports * remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout (less data shuffling between the types) * fix several cases of states being stored on stack in tests, causing random failures on some platforms * remove reading json support from ncli - this should be ported to the rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 07:34:34 +00:00
"."/[eth2_merkleization, forks, ssz_codec]
rework spec imports (#2779) The spec imports are a mess to work with, so this branch cleans them up a bit to ensure that we avoid generic sandwitches and that importing stuff generally becomes easier. * reexport crypto/digest/presets because these are part of the public symbol set of the rest of the spec types * don't export `merge` types from `base` - this causes circular deps * fix circular deps in `ssz/spec_types` - this is the first step in disentangling ssz from spec * be explicit about phase0 vs altair - longer term, `altair` will become the "natural" type set, then merge and so on, so no point in giving `phase0` special preferential treatment
2021-08-12 13:08:20 +00:00
disentangle eth2 types from the ssz library (#2785) * reorganize ssz dependencies This PR continues the work in https://github.com/status-im/nimbus-eth2/pull/2646, https://github.com/status-im/nimbus-eth2/pull/2779 as well as past issues with serialization and type, to disentangle SSZ from eth2 and at the same time simplify imports and exports with a structured approach. The principal idea here is that when a library wants to introduce SSZ support, they do so via 3 files: * `ssz_codecs` which imports and reexports `codecs` - this covers the basic byte conversions and ensures no overloads get lost * `xxx_merkleization` imports and exports `merkleization` to specialize and get access to `hash_tree_root` and friends * `xxx_ssz_serialization` imports and exports `ssz_serialization` to specialize ssz for a specific library Those that need to interact with SSZ always import the `xxx_` versions of the modules and never `ssz` itself so as to keep imports simple and safe. This is similar to how the REST / JSON-RPC serializers are structured in that someone wanting to serialize spec types to REST-JSON will import `eth2_rest_serialization` and nothing else. * split up ssz into a core library that is independendent of eth2 types * rename `bytes_reader` to `codec` to highlight that it contains coding and decoding of bytes and native ssz types * remove tricky List init overload that causes compile issues * get rid of top-level ssz import * reenable merkleization tests * move some "standard" json serializers to spec * remove `ValidatorIndex` serialization for now * remove test_ssz_merkleization * add tests for over/underlong byte sequences * fix broken seq[byte] test - seq[byte] is not an SSZ type There are a few things this PR doesn't solve: * like #2646 this PR is weak on how to handle root and other dontSerialize fields that "sometimes" should be computed - the same problem appears in REST / JSON-RPC etc * Fix a build problem on macOS * Another way to fix the macOS builds Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 18:57:58 +00:00
# TODO although eth2_merkleization already exports ssz_codec, *sometimes* code
remove capella and deneb empty execution payload fallbacks (#4613)
2023-02-13 17:15:16 +00:00
# fails to compile if the export is not done here also. Exporting rlp avoids a
# generics sandwich where rlp/writer.append() is not seen, by a caller outside
# this module via compute_execution_block_hash() called from block_processor.
disentangle eth2 types from the ssz library (#2785) * reorganize ssz dependencies This PR continues the work in https://github.com/status-im/nimbus-eth2/pull/2646, https://github.com/status-im/nimbus-eth2/pull/2779 as well as past issues with serialization and type, to disentangle SSZ from eth2 and at the same time simplify imports and exports with a structured approach. The principal idea here is that when a library wants to introduce SSZ support, they do so via 3 files: * `ssz_codecs` which imports and reexports `codecs` - this covers the basic byte conversions and ensures no overloads get lost * `xxx_merkleization` imports and exports `merkleization` to specialize and get access to `hash_tree_root` and friends * `xxx_ssz_serialization` imports and exports `ssz_serialization` to specialize ssz for a specific library Those that need to interact with SSZ always import the `xxx_` versions of the modules and never `ssz` itself so as to keep imports simple and safe. This is similar to how the REST / JSON-RPC serializers are structured in that someone wanting to serialize spec types to REST-JSON will import `eth2_rest_serialization` and nothing else. * split up ssz into a core library that is independendent of eth2 types * rename `bytes_reader` to `codec` to highlight that it contains coding and decoding of bytes and native ssz types * remove tricky List init overload that causes compile issues * get rid of top-level ssz import * reenable merkleization tests * move some "standard" json serializers to spec * remove `ValidatorIndex` serialization for now * remove test_ssz_merkleization * add tests for over/underlong byte sequences * fix broken seq[byte] test - seq[byte] is not an SSZ type There are a few things this PR doesn't solve: * like #2646 this PR is weak on how to handle root and other dontSerialize fields that "sometimes" should be computed - the same problem appears in REST / JSON-RPC etc * Fix a build problem on macOS * Another way to fix the macOS builds Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 18:57:58 +00:00
export
remove capella and deneb empty execution payload fallbacks (#4613)
2023-02-13 17:15:16 +00:00
eth2_merkleization, forks, rlp, ssz_codec
Add the pseudo renadom shuffle
2018-07-23 12:58:41 +00:00
update weak subjectivity calculations (#4923) Weak subjectivity logic (`--weak-subjectivity-checkpoint`) was outdated. Updated to latest specs, also taking into account total active balance. See https://github.com/ethereum/consensus-specs/pull/2190
2023-05-10 08:23:59 +00:00
func toEther*(gwei: Gwei): Ether =
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/phase0/weak-subjectivity.md#constants
update weak subjectivity calculations (#4923) Weak subjectivity logic (`--weak-subjectivity-checkpoint`) was outdated. Updated to latest specs, also taking into account total active balance. See https://github.com/ethereum/consensus-specs/pull/2190
2023-05-10 08:23:59 +00:00
const ETH_TO_GWEI = 1_000_000_000
(gwei div ETH_TO_GWEI).Ether
Fix the block header computation when proposing an empty execution payload (#3991) * Fix the block header computation when proposing an empty execution payload * Spec compliant base fee calculation when producing empty payloads
2022-08-19 10:28:42 +00:00
type
add helpers for processing EL block header to `libnimbus_lc.a` (#5199) To obtain the correct `transactions_root` and `withdrawals_root`, it is necessary to process execution block header. Light client updates don't contain the correct MPT roots.
2023-08-07 12:23:44 +00:00
ExecutionHash256* = eth_types.Hash256
add helpers for processing transactions to `libnimbus_lc.a` (#5269) It is useful to verify transactions data against `transactionsRoot`. Add corresponding functionality to the light client library.
2023-08-25 09:29:39 +00:00
ExecutionTransaction* = eth_types.Transaction
add helpers for processing receipts to `libnimbus_lc.a` (#5360) Similar to the existing helpers for processing transactions, extend `libnimbus_lc.a` with support for processing receipts as well.
2023-08-28 15:56:40 +00:00
ExecutionReceipt* = eth_types.Receipt
add helpers for processing withdrawals to `libnimbus_lc.a` (#5374) Similar to the existing helpers for processing transactions / receipts, extend `libnimbus_lc.a` with support for processing withdrawals as well.
2023-09-04 18:44:03 +00:00
ExecutionWithdrawal* = eth_types.Withdrawal
add helpers for processing EL block header to `libnimbus_lc.a` (#5199) To obtain the correct `transactions_root` and `withdrawals_root`, it is necessary to process execution block header. Light client updates don't contain the correct MPT roots.
2023-08-07 12:23:44 +00:00
ExecutionBlockHeader* = eth_types.BlockHeader
Fix the block header computation when proposing an empty execution payload (#3991) * Fix the block header computation when proposing an empty execution payload * Spec compliant base fee calculation when producing empty payloads
2022-08-19 10:28:42 +00:00
FinalityCheckpoints* = object
justified*: Checkpoint
finalized*: Checkpoint
fork choice cleanups (#3849) Removes a few extra-ambitious templates to make `self` updates explicit, and moves the `FinalityCheckpoints` type from `base` to `helpers` as it is an additional Nimbus specific type not defined by spec.
2022-07-10 15:26:29 +00:00
func shortLog*(v: FinalityCheckpoints): auto =
(
justified: shortLog(v.justified),
finalized: shortLog(v.finalized)
)
chronicles.formatIt FinalityCheckpoints: it.shortLog
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/phase0/beacon-chain.md#integer_squareroot
spec updates * spec-following renames * more documentation * simplify casper slashing * fix block creation flow / logic
2018-12-19 04:36:10 +00:00
func integer_squareroot*(n: SomeInteger): SomeInteger =
mark comments documenting functions as a whole as such (#1613)
2020-09-08 08:54:55 +00:00
## Return the largest integer ``x`` such that ``x**2 <= n``.
begin 0.5.0 spec update (#179) * begin 0.5.0 spec update: parent_root -> previous_block_root, BeaconState.justified_epoch -> BeaconState.current_justified_epoch, DOMAIN_PROPOSAL -> DOMAIN_BEACON_BLOCK, temporarily rename BeaconBlockHeader to BeaconBlockHeaderRLP to allow for gradual re-merging without disrupting RLP; mark a few unchanged functions and data types, implement get_temporary_block_header/get_empty_block/should_update_validator_registry/processBlockHeader/cacheState; update a few others * a dozen or so more trivial mostly comment changes, finding more unchanged parts of 0.5.0 * several more trivial changes; goal is to reduce noise around the upcoming substantial changes (epoch processing, etc)
2019-03-16 19:52:37 +00:00
doAssert n >= 0'u64
spec updates * lots of renames * add some epoch processing
2018-12-03 21:41:24 +00:00
var
x = n
y = (x + 1) div 2
while y < x:
x = y
y = (x + n div x) div 2
x
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/phase0/beacon-chain.md#is_active_validator
spec 0.3.0 updates: EPOCH_LENGTH -> SLOTS_PER_EPOCH, SlotNumber -> Slot, EpochNumber -> Epoch (#127) * spec 0.3.0 updates: EPOCH_LENGTH -> SLOTS_PER_EPOCH, SlotNumber -> Slot, EpochNumber -> Epoch
2019-02-20 01:33:58 +00:00
func is_active_validator*(validator: Validator, epoch: Epoch): bool =
bump more consensus-specs refs to v1.3.0 (with comments) (#4901) Bump some more consensus-specs links to v1.3.0, and also align comments with latest specs where they differ, and fix typos.
2023-05-05 20:37:56 +00:00
## Check if ``validator`` is active.
epochs replace slots as main first-class entity; several mechanical renamings (ForkData -> Fork; .fork_data -> .fork; remove stub custody types/code; some type aliasing to keep slots and epochs more enforceably separate; fix test_state_transition.nim assumption that GENESIS_SLOT == 0
2019-01-29 03:22:22 +00:00
validator.activation_epoch <= epoch and epoch < validator.exit_epoch
add some helper functions, change epoch processing, add some new constants
2019-01-16 11:07:41 +00:00
Validator monitoring (#2925) Validator monitoring based on and mostly compatible with the implementation in Lighthouse - tracks additional logs and metrics for specified validators so as to stay on top on performance. The implementation works more or less the following way: * Validator pubkeys are singled out for monitoring - these can be running on the node or not * For every action that the validator takes, we record steps in the process such as messages being seen on the network or published in the API * When the dust settles at the end of an epoch, we report the information from one epoch before that, which coincides with the balances being updated - this is a tradeoff between being correct (waiting for finalization) and providing relevant information in a timely manner)
2021-12-20 19:20:31 +00:00
func is_exited_validator*(validator: Validator, epoch: Epoch): bool =
bump more consensus-specs refs to v1.3.0 (with comments) (#4901) Bump some more consensus-specs links to v1.3.0, and also align comments with latest specs where they differ, and fix typos.
2023-05-05 20:37:56 +00:00
## Check if ``validator`` is exited.
Validator monitoring (#2925) Validator monitoring based on and mostly compatible with the implementation in Lighthouse - tracks additional logs and metrics for specified validators so as to stay on top on performance. The implementation works more or less the following way: * Validator pubkeys are singled out for monitoring - these can be running on the node or not * For every action that the validator takes, we record steps in the process such as messages being seen on the network or published in the API * When the dust settles at the end of an epoch, we report the information from one epoch before that, which coincides with the balances being updated - this is a tradeoff between being correct (waiting for finalization) and providing relevant information in a timely manner)
2021-12-20 19:20:31 +00:00
validator.exit_epoch <= epoch
func is_withdrawable_validator*(validator: Validator, epoch: Epoch): bool =
epoch >= validator.withdrawable_epoch
automated v1.4.0-alpha.2 to v1.4.0-alpha.3 consensus spec URL updates (#5065)
2023-06-13 14:03:49 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-alpha.3/specs/phase0/beacon-chain.md#get_active_validator_indices
altair fork handling cleanups (#3050) * fix stack overflow crash in REST/debug/getStateV2 * introduce `ForkyXxx` for generic type matching of `Xxx` across branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) - `Some` is already used for other types of type classes * consolidate function naming in BeaconChainDB, use some generics * import `forks.nim` from other spec modules and move `Forked*` helpers around to resolve circular imports * remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout (less data shuffling between the types) * fix several cases of states being stored on stack in tests, causing random failures on some platforms * remove reading json support from ncli - this should be ported to the rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 07:34:34 +00:00
iterator get_active_validator_indices*(state: ForkyBeaconState, epoch: Epoch):
double altair epoch slot processing speed (#2811) * approximately double altair epoch processing speed * don't redundantly clear state balances cache * only invalidate altair state balances cache once in process_rewards_and_penalties()
2021-08-24 20:09:03 +00:00
ValidatorIndex =
document and clean up `ValidatorIndex` usage (#3651) * document static vs dynamic range checking requirements * add `vindices` iterator to iterate over valid validator indices in a state * clean up spec comments in general * fixup Co-authored-by: tersec <tersec@users.noreply.github.com>
2022-05-23 23:39:08 +00:00
for vidx in state.validators.vindices:
if is_active_validator(state.validators[vidx], epoch):
yield vidx
double altair epoch slot processing speed (#2811) * approximately double altair epoch processing speed * don't redundantly clear state balances cache * only invalidate altair state balances cache once in process_rewards_and_penalties()
2021-08-24 20:09:03 +00:00
altair fork handling cleanups (#3050) * fix stack overflow crash in REST/debug/getStateV2 * introduce `ForkyXxx` for generic type matching of `Xxx` across branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) - `Some` is already used for other types of type classes * consolidate function naming in BeaconChainDB, use some generics * import `forks.nim` from other spec modules and move `Forked*` helpers around to resolve circular imports * remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout (less data shuffling between the types) * fix several cases of states being stored on stack in tests, causing random failures on some platforms * remove reading json support from ncli - this should be ported to the rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 07:34:34 +00:00
func get_active_validator_indices*(state: ForkyBeaconState, epoch: Epoch):
0.6.1 - signing_root, deposits, validator indices and committee count (#261) * signed_root -> signing_root * implement new process_deposit; update timing parameters to 0.6.1; update Deposit to 0.6.1 and remove DepositInput * update get_active_validator_indices and get_epoch_committee_count to 0.6.1; rm get_current_epoch_committee_count, get_previous_epoch_committee_count, and get_next_epoch_committee_count; bump state_sim default validator count a bit more * re-introduce 0.5.1-ish get_active_validator_indices, get_epoch_committee_count as scaffolding for still-0.5.1ish shuffling
2019-05-09 12:27:37 +00:00
seq[ValidatorIndex] =
mark comments documenting functions as a whole as such (#1613)
2020-09-08 08:54:55 +00:00
## Return the sequence of active validator indices at ``epoch``.
double altair epoch slot processing speed (#2811) * approximately double altair epoch processing speed * don't redundantly clear state balances cache * only invalidate altair state balances cache once in process_rewards_and_penalties()
2021-08-24 20:09:03 +00:00
var res = newSeqOfCap[ValidatorIndex](state.validators.len)
document and clean up `ValidatorIndex` usage (#3651) * document static vs dynamic range checking requirements * add `vindices` iterator to iterate over valid validator indices in a state * clean up spec comments in general * fixup Co-authored-by: tersec <tersec@users.noreply.github.com>
2022-05-23 23:39:08 +00:00
for vidx in get_active_validator_indices(state, epoch):
res.add vidx
double altair epoch slot processing speed (#2811) * approximately double altair epoch processing speed * don't redundantly clear state balances cache * only invalidate altair state balances cache once in process_rewards_and_penalties()
2021-08-24 20:09:03 +00:00
res
0.6.1 - signing_root, deposits, validator indices and committee count (#261) * signed_root -> signing_root * implement new process_deposit; update timing parameters to 0.6.1; update Deposit to 0.6.1 and remove DepositInput * update get_active_validator_indices and get_epoch_committee_count to 0.6.1; rm get_current_epoch_committee_count, get_previous_epoch_committee_count, and get_next_epoch_committee_count; bump state_sim default validator count a bit more * re-introduce 0.5.1-ish get_active_validator_indices, get_epoch_committee_count as scaffolding for still-0.5.1ish shuffling
2019-05-09 12:27:37 +00:00
altair fork handling cleanups (#3050) * fix stack overflow crash in REST/debug/getStateV2 * introduce `ForkyXxx` for generic type matching of `Xxx` across branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) - `Some` is already used for other types of type classes * consolidate function naming in BeaconChainDB, use some generics * import `forks.nim` from other spec modules and move `Forked*` helpers around to resolve circular imports * remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout (less data shuffling between the types) * fix several cases of states being stored on stack in tests, causing random failures on some platforms * remove reading json support from ncli - this should be ported to the rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 07:34:34 +00:00
func get_active_validator_indices_len*(state: ForkyBeaconState, epoch: Epoch):
proposed structure for altair (#2323) * proposed structure for hf1 * refactor datatypes.nim into datatypes/{base, phase0, hf1}.nim * hf1 is Altair * some syncing with alpha 2 * adjust epoch processing to disambiguate access to RewardFlags * relocate StateData to stay consistent with meaning phase 0 StateData * passes v1.1.0 alpha 5 SSZ consensus object tests * Altair block header test fixtures work * fix slash_validator() so that Altair attester slashings, proposer slashings, and voluntary exit textures work * deposit operation Altair test fixtures work * slot sanity and all but a couple epoch transition tests switched to Altair * attestation Altair test fixtures work * Altair block sanity test fixtures work * add working altair sync committee tests * improve workarounds for sum-types-across-modules Nim bug; incorporate SignedBeaconBlock root reconstuction to SSZ byte reader
2021-05-28 15:25:58 +00:00
uint64 =
document and clean up `ValidatorIndex` usage (#3651) * document static vs dynamic range checking requirements * add `vindices` iterator to iterate over valid validator indices in a state * clean up spec comments in general * fixup Co-authored-by: tersec <tersec@users.noreply.github.com>
2022-05-23 23:39:08 +00:00
for vidx in state.validators.vindices:
fix unnecessary HashList/HashArray cache invalidation (#3660) * SSZ `[]` -> `mitem` * `[]` -> `item` immutable access via mutable instance cannot rely on template overloading, and `[]` cannot be a `func` because of special seq handling in compiler.
2022-05-30 13:30:42 +00:00
if is_active_validator(state.validators.item(vidx), epoch):
Add support for starting from weak subjectivity checkpoints Also removes the `genesis.ssz` file stored in the data folder. The `medalla-fast-sync` target has been adapted to use the new features.
2020-09-22 20:42:42 +00:00
inc result
altair fork handling cleanups (#3050) * fix stack overflow crash in REST/debug/getStateV2 * introduce `ForkyXxx` for generic type matching of `Xxx` across branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) - `Some` is already used for other types of type classes * consolidate function naming in BeaconChainDB, use some generics * import `forks.nim` from other spec modules and move `Forked*` helpers around to resolve circular imports * remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout (less data shuffling between the types) * fix several cases of states being stored on stack in tests, causing random failures on some platforms * remove reading json support from ncli - this should be ported to the rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 07:34:34 +00:00
func get_active_validator_indices_len*(
state: ForkedHashedBeaconState; epoch: Epoch): uint64 =
withState(state):
allow accessing withState forky state via `forkyState` (#4026)
2022-08-26 14:14:18 +00:00
get_active_validator_indices_len(forkyState.data, epoch)
altair fork handling cleanups (#3050) * fix stack overflow crash in REST/debug/getStateV2 * introduce `ForkyXxx` for generic type matching of `Xxx` across branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) - `Some` is already used for other types of type classes * consolidate function naming in BeaconChainDB, use some generics * import `forks.nim` from other spec modules and move `Forked*` helpers around to resolve circular imports * remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout (less data shuffling between the types) * fix several cases of states being stored on stack in tests, causing random failures on some platforms * remove reading json support from ncli - this should be ported to the rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 07:34:34 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/phase0/beacon-chain.md#get_current_epoch
altair fork handling cleanups (#3050) * fix stack overflow crash in REST/debug/getStateV2 * introduce `ForkyXxx` for generic type matching of `Xxx` across branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) - `Some` is already used for other types of type classes * consolidate function naming in BeaconChainDB, use some generics * import `forks.nim` from other spec modules and move `Forked*` helpers around to resolve circular imports * remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout (less data shuffling between the types) * fix several cases of states being stored on stack in tests, causing random failures on some platforms * remove reading json support from ncli - this should be ported to the rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 07:34:34 +00:00
func get_current_epoch*(state: ForkyBeaconState): Epoch =
mark comments documenting functions as a whole as such (#1613)
2020-09-08 08:54:55 +00:00
## Return the current epoch.
time: spring cleaning (#3262) Time in the beacon chain is expressed relative to the genesis time - this PR creates a `beacon_time` module that collects helpers and utilities for dealing the time units - the new module does not deal with actual wall time (that's remains in `beacon_clock`). Collecting the time related stuff in one place makes it easier to find, avoids some circular imports and allows more easily identifying the code actually needs wall time to operate. * move genesis-time-related functionality into `spec/beacon_time` * avoid using `chronos.Duration` for time differences - it does not support negative values (such as when something happens earlier than it should) * saturate conversions between `FAR_FUTURE_XXX`, so as to avoid overflows * fix delay reporting in validator client so it uses the expected deadline of the slot, not "closest wall slot" * simplify looping over the slots of an epoch * `compute_start_slot_at_epoch` -> `start_slot` * `compute_epoch_at_slot` -> `epoch` A follow-up PR will (likely) introduce saturating arithmetic for the time units - this is merely code moves, renames and fixing of small bugs.
2022-01-11 10:01:54 +00:00
state.slot.epoch
epochs replace slots as main first-class entity; several mechanical renamings (ForkData -> Fork; .fork_data -> .fork; remove stub custody types/code; some type aliasing to keep slots and epochs more enforceably separate; fix test_state_transition.nim assumption that GENESIS_SLOT == 0
2019-01-29 03:22:22 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/phase0/beacon-chain.md#get_current_epoch
altair fork handling cleanups (#3050) * fix stack overflow crash in REST/debug/getStateV2 * introduce `ForkyXxx` for generic type matching of `Xxx` across branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) - `Some` is already used for other types of type classes * consolidate function naming in BeaconChainDB, use some generics * import `forks.nim` from other spec modules and move `Forked*` helpers around to resolve circular imports * remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout (less data shuffling between the types) * fix several cases of states being stored on stack in tests, causing random failures on some platforms * remove reading json support from ncli - this should be ported to the rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 07:34:34 +00:00
func get_current_epoch*(state: ForkedHashedBeaconState): Epoch =
## Return the current epoch.
allow accessing withState forky state via `forkyState` (#4026)
2022-08-26 14:14:18 +00:00
withState(state): get_current_epoch(forkyState.data)
altair fork handling cleanups (#3050) * fix stack overflow crash in REST/debug/getStateV2 * introduce `ForkyXxx` for generic type matching of `Xxx` across branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) - `Some` is already used for other types of type classes * consolidate function naming in BeaconChainDB, use some generics * import `forks.nim` from other spec modules and move `Forked*` helpers around to resolve circular imports * remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout (less data shuffling between the types) * fix several cases of states being stored on stack in tests, causing random failures on some platforms * remove reading json support from ncli - this should be ported to the rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 07:34:34 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/phase0/beacon-chain.md#get_previous_epoch
group justified and finalized `Checkpoint` (#3841) The justified and finalized `Checkpoint` are frequently passed around together. This introduces a new `FinalityCheckpoint` data structure that combines them into one. Due to the large usage of this structure in fork choice, also took this opportunity to update fork choice tests to the latest v1.2.0-rc.1 spec. Many additional tests enabled, some need more work, e.g. EL mock blocks. Also implemented `discard_equivocations` which was skipped in #3661, and improved code reuse across fork choice logic while at it.
2022-07-06 10:33:02 +00:00
func get_previous_epoch*(
state: ForkyBeaconState | ForkedHashedBeaconState): Epoch =
## Return the previous epoch (unless the current epoch is ``GENESIS_EPOCH``).
get_previous_epoch(get_current_epoch(state))
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/phase0/beacon-chain.md#get_randao_mix
altair fork handling cleanups (#3050) * fix stack overflow crash in REST/debug/getStateV2 * introduce `ForkyXxx` for generic type matching of `Xxx` across branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) - `Some` is already used for other types of type classes * consolidate function naming in BeaconChainDB, use some generics * import `forks.nim` from other spec modules and move `Forked*` helpers around to resolve circular imports * remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout (less data shuffling between the types) * fix several cases of states being stored on stack in tests, causing random failures on some platforms * remove reading json support from ncli - this should be ported to the rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 07:34:34 +00:00
func get_randao_mix*(state: ForkyBeaconState, epoch: Epoch): Eth2Digest =
bump more consensus-specs refs to v1.3.0 (with comments) (#4901) Bump some more consensus-specs links to v1.3.0, and also align comments with latest specs where they differ, and fix typos.
2023-05-05 20:37:56 +00:00
## Return the randao mix at a recent ``epoch``.
Restore compilation with custom presets (#1309) This also assigns precise types to the constants in the minimal and mainnet presets in order to reduce the chance of compilation errors when custom presets are used (previously, only the custom presets have precisely assigned types for the constants).
2020-07-13 14:44:58 +00:00
state.randao_mixes[epoch mod EPOCHS_PER_HISTORICAL_VECTOR]
epochs replace slots as main first-class entity; several mechanical renamings (ForkData -> Fork; .fork_data -> .fork; remove stub custody types/code; some type aliasing to keep slots and epochs more enforceably separate; fix test_state_transition.nim assumption that GENESIS_SLOT == 0
2019-01-29 03:22:22 +00:00
eip4844 block/slot sanity and transition tests (#4399)
2022-12-08 02:07:41 +00:00
func bytes_to_uint32*(data: openArray[byte]): uint32 =
doAssert data.len == 4
# Little-endian data representation
uint32.fromBytesLE(data)
openarray -> openArray
2020-10-28 18:35:31 +00:00
func bytes_to_uint64*(data: openArray[byte]): uint64 =
assert() -> doAssert()
2019-03-13 23:04:43 +00:00
doAssert data.len == 8
more v0.2.0 spec updates (#107) * switch to v0.2.0 shuffling scheme * fix some >80 char lines; add get_total_balance helper function (used in epoch processing, etc); note intended removal of repeat_hash and users thereof; add some int_to_bytes and bytes_to_int helpers; more underflow comments; add Gwei type; adjust generate_seed to include additional entropy source; note egregious inefficiencies * minor code cleanup * move some general helper functions to helpers.nim and clarify little-endian comments
2019-02-13 10:26:32 +00:00
# Little-endian data representation
tune attestation params (#2301) * a little bit more priority to attestation processing * better implementation for bytes_to_uint64
2021-02-08 15:13:02 +00:00
uint64.fromBytesLE(data)
more v0.2.0 spec updates (#107) * switch to v0.2.0 shuffling scheme * fix some >80 char lines; add get_total_balance helper function (used in epoch processing, etc); note intended removal of repeat_hash and users thereof; add some int_to_bytes and bytes_to_int helpers; more underflow comments; add Gwei type; adjust generate_seed to include additional entropy source; note egregious inefficiencies * minor code cleanup * move some general helper functions to helpers.nim and clarify little-endian comments
2019-02-13 10:26:32 +00:00
REST cleanups (#3255) * REST cleanups * reject out-of-range committee requests * print all hex values as lower-case * allow requesting state information by head state root * turn `DomainType` into array (follow spec) * `uint_to_bytesXX` -> `uint_to_bytes` (follow spec) * fix wrong dependent root in `/eth/v1/validator/duties/proposer/` * update documentation - `--subscribe-all-subnets` is no longer needed when using the REST interface with validator clients * more fixes * common helpers for dependent block * remove test rules obsoleted by more strict epoch tests * fix trailing commas * Update docs/the_nimbus_book/src/rest-api.md * Update docs/the_nimbus_book/src/rest-api.md Co-authored-by: sacha <sacha@status.im>
2022-01-08 20:06:34 +00:00
func uint_to_bytes*(x: uint64): array[8, byte] = toBytesLE(x)
func uint_to_bytes*(x: uint32): array[4, byte] = toBytesLE(x)
func uint_to_bytes*(x: uint16): array[2, byte] = toBytesLE(x)
func uint_to_bytes*(x: uint8): array[1, byte] = toBytesLE(x)
more v0.2.0 spec updates (#107) * switch to v0.2.0 shuffling scheme * fix some >80 char lines; add get_total_balance helper function (used in epoch processing, etc); note intended removal of repeat_hash and users thereof; add some int_to_bytes and bytes_to_int helpers; more underflow comments; add Gwei type; adjust generate_seed to include additional entropy source; note egregious inefficiencies * minor code cleanup * move some general helper functions to helpers.nim and clarify little-endian comments
2019-02-13 10:26:32 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/phase0/beacon-chain.md#compute_domain
interop updates * add interop launcher scripts * stick validator_keygen into beacon_node * fix lmd ghost slot number on missing block * use mocked eth1data when producing blocks * use bls public key method for withdrawal credentials * fix deposit domain * prefer lowercase for a bunch of toHex * build simulation binary in data folder to avoid data types confusion
2019-09-01 15:02:49 +00:00
func compute_domain*(
domain_type: DomainType,
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
fork_version: Version,
Bump modules to take advantage of the new Json format flavors support Since quite a lot of additional procs were now compiled as generics, this lead to compiler bugs that had to be worked-around: * The `Domain` type was renamed to `Eth2Domain` to avoid compilation errors due to conflicts with `nativesockets.Domain`. Similarly, `eth2_network.KeyPair` was renamed to `NetKeyPair`. * A new more robust version of `hexToByteArray` was added to stew
2021-03-19 02:22:45 +00:00
genesis_validators_root: Eth2Digest = ZERO_HASH): Eth2Domain =
mark comments documenting functions as a whole as such (#1613)
2020-09-08 08:54:55 +00:00
## Return the domain for the ``domain_type`` and ``fork_version``.
MEV boost script for ropsten and sepolia builder network (#3851) * MEV boost script for ropsten and sepolia builder network * networkInfo not a global variable
2022-07-12 17:50:12 +00:00
#
remove Nim 1.2 workarounds and `TODO`s (#4533) * clean up some Nim 1.2 workarounds * re-add notes about JS backend * another proc/noSideEffect -> func * revert ncli/ncli_common.nim changes; 19969 evidently wasn't backported to 1.6
2023-01-30 19:21:51 +00:00
# TODO toOpenArray can't be used from JavaScript backend
MEV boost script for ropsten and sepolia builder network (#3851) * MEV boost script for ropsten and sepolia builder network * networkInfo not a global variable
2022-07-12 17:50:12 +00:00
# https://github.com/nim-lang/Nim/issues/15952
complete (except for get_domain(...)) 0.11.0 beacon chain spec update (#822) * complete (except for get_domain(...)) 0.11.0 beacon chain spec update * mark compute_start_slot_at_epoch(...), is_active_validator(...), compute_signing_root(...), and get_seed(...) as 0.11.0
2020-03-22 16:03:07 +00:00
let fork_data_root =
compute_fork_data_root(fork_version, genesis_validators_root)
REST cleanups (#3255) * REST cleanups * reject out-of-range committee requests * print all hex values as lower-case * allow requesting state information by head state root * turn `DomainType` into array (follow spec) * `uint_to_bytesXX` -> `uint_to_bytes` (follow spec) * fix wrong dependent root in `/eth/v1/validator/duties/proposer/` * update documentation - `--subscribe-all-subnets` is no longer needed when using the REST interface with validator clients * more fixes * common helpers for dependent block * remove test rules obsoleted by more strict epoch tests * fix trailing commas * Update docs/the_nimbus_book/src/rest-api.md * Update docs/the_nimbus_book/src/rest-api.md Co-authored-by: sacha <sacha@status.im>
2022-01-08 20:06:34 +00:00
result[0..3] = domain_type.data
avoid unnecessary seq allocations (#1573) "sigh"
2020-08-27 06:32:51 +00:00
result[4..31] = fork_data_root.data.toOpenArray(0, 27)
more 0.7.0 updates (#287) * update Deposit and BeaconBlockHeader to 0.7.0 * update get_total_balance(...) and get_base_reward(...) to 0.7.0 * rename initial value constant BLS_WITHDRAWAL_PREFIX_BYTE to BLS_WITHDRAWAL_PREFIX * mark minimal presets as entirely migrated to 0.7.0; update process_rewards_and_penalties(...) to 0.7.0 * specify that targeting 0.7.0; adopt 0.7.0 refactoring of bls_domain/get_domain to update get_domain(...) to 0.7.0; mark that mainnet constant presets and block header processing are 0.7.0-consistent; update process_justification_and_finalization to 0.7.0 to fix potential underflow errors * address mratsim's comment to copy whole spec comment
2019-06-14 16:21:04 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/phase0/beacon-chain.md#get_domain
AttestationData.justified epoch -> AttestationData.source_epoch and AttestationData.latest_crosslink -> AttestationData.previous_crosslink; implement verify_merkle_branch and update process_deposit to use it; rm BeaconState.batched_block_roots; Transfer.{from, to} -> Transfer.{sender, recipient}; update Fork data type and code to use bytes4; implement int_to_bytes8 (close to metaprogramming time); adjust signed_root and all callers to use 0.5.0 calling signature; processDepositRoot -> processEth1Data (#182)
2019-03-18 15:42:42 +00:00
func get_domain*(
Bump modules to take advantage of the new Json format flavors support Since quite a lot of additional procs were now compiled as generics, this lead to compiler bugs that had to be worked-around: * The `Domain` type was renamed to `Eth2Domain` to avoid compilation errors due to conflicts with `nativesockets.Domain`. Similarly, `eth2_network.KeyPair` was renamed to `NetKeyPair`. * A new more robust version of `hexToByteArray` was added to stew
2021-03-19 02:22:45 +00:00
fork: Fork,
domain_type: DomainType,
epoch: Epoch,
genesis_validators_root: Eth2Digest): Eth2Domain =
begin 0.6.0 (#256) * begin 0.6.0: new get_domain/increase_balance/reduce_balance, BeaconState.validator_balances -> BeaconState.balances, some renamed constants, transaction processing changes, SlashableAttestation field name changes, 0.6.0 get_beacon_proposer_index always uses given state's slot, update tests subrepo * mark get_bitfield_bit/bls_verify_multiple/stat-list-lengths/is_active_validator/is_surround_vote/slot_to_epoch/int_to_bytes/etc as unchanged in 0.6.0; rm Eth1DataVote/maybe_reset_eth1_period and thus adjust expected tree hash test results * mark verify_bitfield/bls_verify/deposit-contract/VoluntaryExit/PendingAttestation/Historicalbatch/Fork as 0.6.0; update DOMAIN_BEACON_BLOCK to DOMAIN_BEACON_PROPOSER * update Crosslink to 0.6.0 (also requires tree hashing test result change, so isolate in individual commit) * mark verify_merkle_branch/get_delayed_activation_exit_epoch/ProposerSlashing/Attestation/AttestationDataAndCustodyBit/hash/integer_squareroot/get_epoch_start_slot/is_double_vote/get_randao_mix/generate_seed as 0.6.0; update reward and penalty quotients; SlashableAttestation -> IndexedAttestation; rm get_fork_version; ATTESTATION_INCLUSION_REWARD_QUOTIENT -> PROPOSER_REWARD_QUOTIENT
2019-04-29 16:48:30 +00:00
## Return the signature domain (fork version concatenated with domain type)
## of a message.
0.11.1 beacon chain spec update (#836) * initial 0.11.1 spec commit; no test regressions and finalizes in eth2_network_simulation * with BLS 0.10/0.11 available, stop skipping attester slashing, proposer slashing, and voluntary exist operations fixture tests * switch param orders to group state.{fork, genesis_validators_root}; bump spec/datatypes spec version for network purposes * mark attestation construction and broadcast and some minimal/mainnet constants as 0.11.1-compatible; remove phase 1 sharding constants from minimal which don't exist in that preset
2020-03-30 11:31:44 +00:00
let fork_version =
if epoch < fork.epoch:
fork.previous_version
else:
fork.current_version
compute_domain(domain_type, fork_version, genesis_validators_root)
AttestationData.justified epoch -> AttestationData.source_epoch and AttestationData.latest_crosslink -> AttestationData.previous_crosslink; implement verify_merkle_branch and update process_deposit to use it; rm BeaconState.batched_block_roots; Transfer.{from, to} -> Transfer.{sender, recipient}; update Fork data type and code to use bytes4; implement int_to_bytes8 (close to metaprogramming time); adjust signed_root and all callers to use 0.5.0 calling signature; processDepositRoot -> processEth1Data (#182)
2019-03-18 15:42:42 +00:00
simplify validator api (#573) * avoid passing whole state to validator api (easier when it goes out-of-process) * clean up some cruft
2019-11-21 09:57:59 +00:00
func get_domain*(
altair fork handling cleanups (#3050) * fix stack overflow crash in REST/debug/getStateV2 * introduce `ForkyXxx` for generic type matching of `Xxx` across branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) - `Some` is already used for other types of type classes * consolidate function naming in BeaconChainDB, use some generics * import `forks.nim` from other spec modules and move `Forked*` helpers around to resolve circular imports * remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout (less data shuffling between the types) * fix several cases of states being stored on stack in tests, causing random failures on some platforms * remove reading json support from ncli - this should be ported to the rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 07:34:34 +00:00
state: ForkyBeaconState, domain_type: DomainType, epoch: Epoch): Eth2Domain =
simplify validator api (#573) * avoid passing whole state to validator api (easier when it goes out-of-process) * clean up some cruft
2019-11-21 09:57:59 +00:00
## Return the signature domain (fork version concatenated with domain type)
## of a message.
Pass the test suite with a BeaconState ref type
2020-04-22 23:35:55 +00:00
get_domain(state.fork, domain_type, epoch, state.genesis_validators_root)
begin 0.6.0 (#256) * begin 0.6.0: new get_domain/increase_balance/reduce_balance, BeaconState.validator_balances -> BeaconState.balances, some renamed constants, transaction processing changes, SlashableAttestation field name changes, 0.6.0 get_beacon_proposer_index always uses given state's slot, update tests subrepo * mark get_bitfield_bit/bls_verify_multiple/stat-list-lengths/is_active_validator/is_surround_vote/slot_to_epoch/int_to_bytes/etc as unchanged in 0.6.0; rm Eth1DataVote/maybe_reset_eth1_period and thus adjust expected tree hash test results * mark verify_bitfield/bls_verify/deposit-contract/VoluntaryExit/PendingAttestation/Historicalbatch/Fork as 0.6.0; update DOMAIN_BEACON_BLOCK to DOMAIN_BEACON_PROPOSER * update Crosslink to 0.6.0 (also requires tree hashing test result change, so isolate in individual commit) * mark verify_merkle_branch/get_delayed_activation_exit_epoch/ProposerSlashing/Attestation/AttestationDataAndCustodyBit/hash/integer_squareroot/get_epoch_start_slot/is_double_vote/get_randao_mix/generate_seed as 0.6.0; update reward and penalty quotients; SlashableAttestation -> IndexedAttestation; rm get_fork_version; ATTESTATION_INCLUSION_REWARD_QUOTIENT -> PROPOSER_REWARD_QUOTIENT
2019-04-29 16:48:30 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/phase0/beacon-chain.md#compute_signing_root
Bump modules to take advantage of the new Json format flavors support Since quite a lot of additional procs were now compiled as generics, this lead to compiler bugs that had to be worked-around: * The `Domain` type was renamed to `Eth2Domain` to avoid compilation errors due to conflicts with `nativesockets.Domain`. Similarly, `eth2_network.KeyPair` was renamed to `NetKeyPair`. * A new more robust version of `hexToByteArray` was added to stew
2021-03-19 02:22:45 +00:00
func compute_signing_root*(ssz_object: auto, domain: Eth2Domain): Eth2Digest =
bump more consensus-specs refs to v1.3.0 (with comments) (#4901) Bump some more consensus-specs links to v1.3.0, and also align comments with latest specs where they differ, and fix typos.
2023-05-05 20:37:56 +00:00
## Return the signing root for the corresponding signing data.
remove v0.11.3 support; add block_sim to CI (#1253) * remove v0.11.3 support; add block_sim to CI * rm stray PERSISTENT_COMMITTEE_PERIOD * remove TopicFilter.InteropAttestations * bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
let domain_wrapped_object = SigningData(
object_root: hash_tree_root(ssz_object),
domain: domain
)
implement compute_signing_root() and mark some functions as 0.10.1-compatible
2020-01-30 14:03:26 +00:00
hash_tree_root(domain_wrapped_object)
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/phase0/beacon-chain.md#get_seed
Revert "Revert "accelerate `getShufflingRef` (#4911)" (#4958)" This reverts commit 748be8b67ba8598340766cabcce784cc9b0f353e.
2023-05-15 15:41:30 +00:00
func get_seed*(
state: ForkyBeaconState, epoch: Epoch, domain_type: DomainType,
mix: Eth2Digest): Eth2Digest =
mark comments documenting functions as a whole as such (#1613)
2020-09-08 08:54:55 +00:00
## Return the seed at ``epoch``.
update get_seed(...) and get_beacon_proposer_index(...) to 0.9.0, implement compute_proposer_index(...), and render 3 more test fixtures working (#532) * update get_seed(...) and get_beacon_proposer_index(...) to 0.9.0, implement compute_proposer_index(...), and render 3 more test fixtures working * rm stray Crosslink reference which prevented static SSZ tests from building * remove references to removed tests in attestations test fixture; add minimal-preset block sanity test, plus all but one of mainnet tests for block sanity to transition fixtures
2019-11-10 00:03:41 +00:00
var seed_input : array[4+8+32, byte]
REST cleanups (#3255) * REST cleanups * reject out-of-range committee requests * print all hex values as lower-case * allow requesting state information by head state root * turn `DomainType` into array (follow spec) * `uint_to_bytesXX` -> `uint_to_bytes` (follow spec) * fix wrong dependent root in `/eth/v1/validator/duties/proposer/` * update documentation - `--subscribe-all-subnets` is no longer needed when using the REST interface with validator clients * more fixes * common helpers for dependent block * remove test rules obsoleted by more strict epoch tests * fix trailing commas * Update docs/the_nimbus_book/src/rest-api.md * Update docs/the_nimbus_book/src/rest-api.md Co-authored-by: sacha <sacha@status.im>
2022-01-08 20:06:34 +00:00
seed_input[0..3] = domain_type.data
seed_input[4..11] = uint_to_bytes(epoch.uint64)
Revert "Revert "accelerate `getShufflingRef` (#4911)" (#4958)" This reverts commit 748be8b67ba8598340766cabcce784cc9b0f353e.
2023-05-15 15:41:30 +00:00
seed_input[12..43] = mix.data
eth2hash -> eth2digest hash in nim is the insecure hash-map helper - might as well use `digest` consistently to mark the difference
2020-06-16 12:16:43 +00:00
eth2digest(seed_input)
add Altair domains and participation flags; clean up imports (#2462)
2021-04-04 16:24:45 +00:00
Revert "Revert "accelerate `getShufflingRef` (#4911)" (#4958)" This reverts commit 748be8b67ba8598340766cabcce784cc9b0f353e.
2023-05-15 15:41:30 +00:00
func get_seed*(state: ForkyBeaconState, epoch: Epoch, domain_type: DomainType):
Eth2Digest =
# Detect potential underflow
static: doAssert EPOCHS_PER_HISTORICAL_VECTOR > MIN_SEED_LOOKAHEAD
let mix = get_randao_mix(state, # Avoid underflow
epoch + EPOCHS_PER_HISTORICAL_VECTOR - MIN_SEED_LOOKAHEAD - 1)
state.get_seed(epoch, domain_type, mix)
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/altair/beacon-chain.md#add_flag
reduce memory allocations during state transition (#5235) This PR removes a few hundred thousand temporary seq allocations during state transition - in particular, the flag seq was allocated per validator while committees are computed per attestation.
2023-08-02 23:03:40 +00:00
func add_flag*(flags: ParticipationFlags, flag_index: TimelyFlag): ParticipationFlags =
let flag = ParticipationFlags(1'u8 shl ord(flag_index))
add Altair domains and participation flags; clean up imports (#2462)
2021-04-04 16:24:45 +00:00
flags or flag
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/altair/beacon-chain.md#has_flag
reduce memory allocations during state transition (#5235) This PR removes a few hundred thousand temporary seq allocations during state transition - in particular, the flag seq was allocated per validator while committees are computed per attestation.
2023-08-02 23:03:40 +00:00
func has_flag*(flags: ParticipationFlags, flag_index: TimelyFlag): bool =
let flag = ParticipationFlags(1'u8 shl ord(flag_index))
add Altair domains and participation flags; clean up imports (#2462)
2021-04-04 16:24:45 +00:00
(flags and flag) == flag
Light client sync spec functions
2021-09-08 16:57:00 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/altair/light-client/sync-protocol.md#is_sync_committee_update
LC data fork cleanup (#4506) Distinguish between those code locations that need to be updated on each light client data format change, and those others that should generally be fine, as long as a valid light client object is processed. The former are tagged with static assert for `LightClientDataFork.high`. The latter are changed to `lcDataFork > LightClientDataFork.None` to indicate that they depend only on presence of any valid object. Also bundled a few minor cleanups and fixes. Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 21:19:50 +00:00
template is_sync_committee_update*(update: SomeForkyLightClientUpdate): bool =
when update is SomeForkyLightClientUpdateWithSyncCommittee:
update.next_sync_committee_branch !=
further reduce stack size of LC helpers for Nim 2.0 (#5401) The `is_next_sync_committee_known` helper allocates a fresh `SyncCommittee` in the caller and `nimZeroMem`s it on each use. Use `static` to compare against a compile-time zeroed copy instead. This also should help reduce stack size far enough to link with Nim 2.0.
2023-09-06 14:05:12 +00:00
static(default(typeof(update.next_sync_committee_branch)))
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
else:
false
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/altair/light-client/sync-protocol.md#is_finality_update
LC data fork cleanup (#4506) Distinguish between those code locations that need to be updated on each light client data format change, and those others that should generally be fine, as long as a valid light client object is processed. The former are tagged with static assert for `LightClientDataFork.high`. The latter are changed to `lcDataFork > LightClientDataFork.None` to indicate that they depend only on presence of any valid object. Also bundled a few minor cleanups and fixes. Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 21:19:50 +00:00
template is_finality_update*(update: SomeForkyLightClientUpdate): bool =
when update is SomeForkyLightClientUpdateWithFinality:
further reduce stack size of LC helpers for Nim 2.0 (#5401) The `is_next_sync_committee_known` helper allocates a fresh `SyncCommittee` in the caller and `nimZeroMem`s it on each use. Use `static` to compare against a compile-time zeroed copy instead. This also should help reduce stack size far enough to link with Nim 2.0.
2023-09-06 14:05:12 +00:00
update.finality_branch !=
static(default(typeof(update.finality_branch)))
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
else:
false
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/altair/light-client/sync-protocol.md#is_next_sync_committee_known
LC data fork cleanup (#4506) Distinguish between those code locations that need to be updated on each light client data format change, and those others that should generally be fine, as long as a valid light client object is processed. The former are tagged with static assert for `LightClientDataFork.high`. The latter are changed to `lcDataFork > LightClientDataFork.None` to indicate that they depend only on presence of any valid object. Also bundled a few minor cleanups and fixes. Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 21:19:50 +00:00
template is_next_sync_committee_known*(store: ForkyLightClientStore): bool =
further reduce stack size of LC helpers for Nim 2.0 (#5401) The `is_next_sync_committee_known` helper allocates a fresh `SyncCommittee` in the caller and `nimZeroMem`s it on each use. Use `static` to compare against a compile-time zeroed copy instead. This also should help reduce stack size far enough to link with Nim 2.0.
2023-09-06 14:05:12 +00:00
store.next_sync_committee !=
static(default(typeof(store.next_sync_committee)))
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/altair/light-client/sync-protocol.md#get_safety_threshold
LC data fork cleanup (#4506) Distinguish between those code locations that need to be updated on each light client data format change, and those others that should generally be fine, as long as a valid light client object is processed. The former are tagged with static assert for `LightClientDataFork.high`. The latter are changed to `lcDataFork > LightClientDataFork.None` to indicate that they depend only on presence of any valid object. Also bundled a few minor cleanups and fixes. Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 21:19:50 +00:00
func get_safety_threshold*(store: ForkyLightClientStore): uint64 =
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
max(
store.previous_max_active_participants,
store.current_max_active_participants
) div 2
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/altair/light-client/sync-protocol.md#is_better_update
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
type LightClientUpdateMetadata* = object
attested_slot*, finalized_slot*, signature_slot*: Slot
has_sync_committee*, has_finality*: bool
num_active_participants*: uint64
LC data fork cleanup (#4506) Distinguish between those code locations that need to be updated on each light client data format change, and those others that should generally be fine, as long as a valid light client object is processed. The former are tagged with static assert for `LightClientDataFork.high`. The latter are changed to `lcDataFork > LightClientDataFork.None` to indicate that they depend only on presence of any valid object. Also bundled a few minor cleanups and fixes. Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 21:19:50 +00:00
func toMeta*(update: SomeForkyLightClientUpdate): LightClientUpdateMetadata =
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
var meta {.noinit.}: LightClientUpdateMetadata
meta.attested_slot =
add `LightClientHeader` wrapper (#4481) The light client data structures were changed to accommodate additional fields in future forks (e.g., to also hold execution data). There is a minor change to the JSON serialization, where the `header` properties are now nested inside a `LightClientHeader`. The SSZ serialization remains compatible. See https://github.com/ethereum/consensus-specs/pull/3190 and https://github.com/ethereum/beacon-APIs/pull/287
2023-01-13 15:46:35 +00:00
update.attested_header.beacon.slot
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
meta.finalized_slot =
LC data fork cleanup (#4506) Distinguish between those code locations that need to be updated on each light client data format change, and those others that should generally be fine, as long as a valid light client object is processed. The former are tagged with static assert for `LightClientDataFork.high`. The latter are changed to `lcDataFork > LightClientDataFork.None` to indicate that they depend only on presence of any valid object. Also bundled a few minor cleanups and fixes. Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 21:19:50 +00:00
when update is SomeForkyLightClientUpdateWithFinality:
add `LightClientHeader` wrapper (#4481) The light client data structures were changed to accommodate additional fields in future forks (e.g., to also hold execution data). There is a minor change to the JSON serialization, where the `header` properties are now nested inside a `LightClientHeader`. The SSZ serialization remains compatible. See https://github.com/ethereum/consensus-specs/pull/3190 and https://github.com/ethereum/beacon-APIs/pull/287
2023-01-13 15:46:35 +00:00
update.finalized_header.beacon.slot
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
else:
GENESIS_SLOT
meta.signature_slot =
update.signature_slot
meta.has_sync_committee =
LC data fork cleanup (#4506) Distinguish between those code locations that need to be updated on each light client data format change, and those others that should generally be fine, as long as a valid light client object is processed. The former are tagged with static assert for `LightClientDataFork.high`. The latter are changed to `lcDataFork > LightClientDataFork.None` to indicate that they depend only on presence of any valid object. Also bundled a few minor cleanups and fixes. Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 21:19:50 +00:00
when update is SomeForkyLightClientUpdateWithSyncCommittee:
update.is_sync_committee_update
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
else:
false
meta.has_finality =
LC data fork cleanup (#4506) Distinguish between those code locations that need to be updated on each light client data format change, and those others that should generally be fine, as long as a valid light client object is processed. The former are tagged with static assert for `LightClientDataFork.high`. The latter are changed to `lcDataFork > LightClientDataFork.None` to indicate that they depend only on presence of any valid object. Also bundled a few minor cleanups and fixes. Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 21:19:50 +00:00
when update is SomeForkyLightClientUpdateWithFinality:
update.is_finality_update
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
else:
false
meta.num_active_participants =
add `num_active_participants` helpers for `SyncAggregate` (#4478) Introduce `num_active_participants` helper function to reduce visibility of low-level `countOnes` function and reduce code duplication.
2023-01-10 11:26:25 +00:00
update.sync_aggregate.num_active_participants.uint64
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
meta
make LC data fork aware (#4493) In a future fork, light client data will be extended with execution info to support more use cases. To anticipate such an upgrade, introduce `Forky` and `Forked` types, and ready the database schema. Because the mapping of sync committee periods to fork versions is not necessarily unique (fork schedule not in sync with period boundaries), an additional column is added to `period` -> `LightClientUpdate` table.
2023-01-12 17:11:38 +00:00
template toMeta*(update: ForkedLightClientUpdate): LightClientUpdateMetadata =
withForkyUpdate(update):
LC data fork cleanup (#4506) Distinguish between those code locations that need to be updated on each light client data format change, and those others that should generally be fine, as long as a valid light client object is processed. The former are tagged with static assert for `LightClientDataFork.high`. The latter are changed to `lcDataFork > LightClientDataFork.None` to indicate that they depend only on presence of any valid object. Also bundled a few minor cleanups and fixes. Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 21:19:50 +00:00
when lcDataFork > LightClientDataFork.None:
make LC data fork aware (#4493) In a future fork, light client data will be extended with execution info to support more use cases. To anticipate such an upgrade, introduce `Forky` and `Forked` types, and ready the database schema. Because the mapping of sync committee periods to fork versions is not necessarily unique (fork schedule not in sync with period boundaries), an additional column is added to `period` -> `LightClientUpdate` table.
2023-01-12 17:11:38 +00:00
forkyUpdate.toMeta()
else:
default(LightClientUpdateMetadata)
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
func is_better_data*(new_meta, old_meta: LightClientUpdateMetadata): bool =
# Compare supermajority (> 2/3) sync committee participation
const max_active_participants = SYNC_COMMITTEE_SIZE.uint64
let
new_has_supermajority =
new_meta.num_active_participants * 3 >= max_active_participants * 2
old_has_supermajority =
old_meta.num_active_participants * 3 >= max_active_participants * 2
if new_has_supermajority != old_has_supermajority:
return new_has_supermajority > old_has_supermajority
if not new_has_supermajority:
if new_meta.num_active_participants != old_meta.num_active_participants:
return new_meta.num_active_participants > old_meta.num_active_participants
# Compare presence of relevant sync committee
let
new_has_relevant_sync_committee = new_meta.has_sync_committee and
new_meta.attested_slot.sync_committee_period ==
new_meta.signature_slot.sync_committee_period
old_has_relevant_sync_committee = old_meta.has_sync_committee and
old_meta.attested_slot.sync_committee_period ==
old_meta.signature_slot.sync_committee_period
if new_has_relevant_sync_committee != old_has_relevant_sync_committee:
return new_has_relevant_sync_committee > old_has_relevant_sync_committee
# Compare indication of any finality
if new_meta.has_finality != old_meta.has_finality:
return new_meta.has_finality > old_meta.has_finality
# Compare sync committee finality
if new_meta.has_finality:
let
new_has_sync_committee_finality =
new_meta.finalized_slot.sync_committee_period ==
new_meta.attested_slot.sync_committee_period
old_has_sync_committee_finality =
old_meta.finalized_slot.sync_committee_period ==
old_meta.attested_slot.sync_committee_period
if new_has_sync_committee_finality != old_has_sync_committee_finality:
return new_has_sync_committee_finality > old_has_sync_committee_finality
# Tiebreaker 1: Sync committee participation beyond supermajority
if new_meta.num_active_participants != old_meta.num_active_participants:
return new_meta.num_active_participants > old_meta.num_active_participants
# Tiebreaker 2: Prefer older data (fewer changes to best data)
new_meta.attested_slot < old_meta.attested_slot
make LC data fork aware (#4493) In a future fork, light client data will be extended with execution info to support more use cases. To anticipate such an upgrade, introduce `Forky` and `Forked` types, and ready the database schema. Because the mapping of sync committee periods to fork versions is not necessarily unique (fork schedule not in sync with period boundaries), an additional column is added to `period` -> `LightClientUpdate` table.
2023-01-12 17:11:38 +00:00
template is_better_update*[
LC data fork cleanup (#4506) Distinguish between those code locations that need to be updated on each light client data format change, and those others that should generally be fine, as long as a valid light client object is processed. The former are tagged with static assert for `LightClientDataFork.high`. The latter are changed to `lcDataFork > LightClientDataFork.None` to indicate that they depend only on presence of any valid object. Also bundled a few minor cleanups and fixes. Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 21:19:50 +00:00
A, B: SomeForkyLightClientUpdate | ForkedLightClientUpdate](
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
new_update: A, old_update: B): bool =
is_better_data(toMeta(new_update), toMeta(old_update))
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/altair/light-client/p2p-interface.md#getlightclientbootstrap
LC data fork cleanup (#4506) Distinguish between those code locations that need to be updated on each light client data format change, and those others that should generally be fine, as long as a valid light client object is processed. The former are tagged with static assert for `LightClientDataFork.high`. The latter are changed to `lcDataFork > LightClientDataFork.None` to indicate that they depend only on presence of any valid object. Also bundled a few minor cleanups and fixes. Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 21:19:50 +00:00
func contextEpoch*(bootstrap: ForkyLightClientBootstrap): Epoch =
add `LightClientHeader` wrapper (#4481) The light client data structures were changed to accommodate additional fields in future forks (e.g., to also hold execution data). There is a minor change to the JSON serialization, where the `header` properties are now nested inside a `LightClientHeader`. The SSZ serialization remains compatible. See https://github.com/ethereum/consensus-specs/pull/3190 and https://github.com/ethereum/beacon-APIs/pull/287
2023-01-13 15:46:35 +00:00
bootstrap.header.beacon.slot.epoch
update for latest LC REST proposal (#4213) Implements the latest proposal for providing LC data via REST, as of https://github.com/ethereum/beacon-APIs/pull/247 with a v0 suffix. Requests: - `/eth/v0/beacon/light_client/bootstrap/{block_root}` - `/eth/v0/beacon/light_client/updates?start_period={start_period}&count={count}` - `/eth/v0/beacon/light_client/finality_update` - `/eth/v0/beacon/light_client/optimistic_update` HTTP Server-Sent Events (SSE): - `light_client_finality_update_v0` - `light_client_optimistic_update_v0`
2022-10-04 11:38:09 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/altair/light-client/p2p-interface.md#lightclientupdatesbyrange
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/altair/light-client/p2p-interface.md#getlightclientfinalityupdate
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/altair/light-client/p2p-interface.md#getlightclientoptimisticupdate
LC data fork cleanup (#4506) Distinguish between those code locations that need to be updated on each light client data format change, and those others that should generally be fine, as long as a valid light client object is processed. The former are tagged with static assert for `LightClientDataFork.high`. The latter are changed to `lcDataFork > LightClientDataFork.None` to indicate that they depend only on presence of any valid object. Also bundled a few minor cleanups and fixes. Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 21:19:50 +00:00
func contextEpoch*(update: SomeForkyLightClientUpdate): Epoch =
add `LightClientHeader` wrapper (#4481) The light client data structures were changed to accommodate additional fields in future forks (e.g., to also hold execution data). There is a minor change to the JSON serialization, where the `header` properties are now nested inside a `LightClientHeader`. The SSZ serialization remains compatible. See https://github.com/ethereum/consensus-specs/pull/3190 and https://github.com/ethereum/beacon-APIs/pull/287
2023-01-13 15:46:35 +00:00
update.attested_header.beacon.slot.epoch
update for latest LC REST proposal (#4213) Implements the latest proposal for providing LC data via REST, as of https://github.com/ethereum/beacon-APIs/pull/247 with a v0 suffix. Requests: - `/eth/v0/beacon/light_client/bootstrap/{block_root}` - `/eth/v0/beacon/light_client/updates?start_period={start_period}&count={count}` - `/eth/v0/beacon/light_client/finality_update` - `/eth/v0/beacon/light_client/optimistic_update` HTTP Server-Sent Events (SSE): - `light_client_finality_update_v0` - `light_client_optimistic_update_v0`
2022-10-04 11:38:09 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/bellatrix/beacon-chain.md#is_merge_transition_complete
capella test vector support (#4261)
2022-10-27 06:29:24 +00:00
func is_merge_transition_complete*(
switch 3 modules in beacon_chain/spec/ from eip4844 to deneb (#4651)
2023-02-22 13:10:00 +00:00
state: bellatrix.BeaconState | capella.BeaconState | deneb.BeaconState): bool =
capella test vector support (#4261)
2022-10-27 06:29:24 +00:00
const defaultExecutionPayloadHeader =
default(typeof(state.latest_execution_payload_header))
avoid nimZeroMem and stack usage in is_merge_transition_complete and is_merge_transition_block (#3399)
2022-02-16 07:16:01 +00:00
state.latest_execution_payload_header != defaultExecutionPayloadHeader
implement forked merge state/block support (#2890) * implement forked state/block support * merge support for containsOrphan; import cleanup; 80-column lines * add merge block header operations and slot sanity fixture * add epoch state transition tests; implement is_valid_gas_limit(), is_merge_block(), is_execution_enabled(), and compute_timestamp_at_slot() * implement process_execution_payload() and add merge deposit operations tests * add merge block sanity tests * add merge case to syncCommitteeParticipants * v1.1.0-beta.5 updates * reduce getTestStates-based memory usage; don't try to REST-serialize ExecutionPayload transactions without underlying support * add execution payload tests; switch var to let in tests/official/
2021-09-27 14:22:58 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/sync/optimistic.md#helpers
optimistic sync (#3793) * optimistic sync * flag that initially loaded blocks from database might need execution block root filled in * return optimistic status in REST calls * refactor blockslot pruning * ensure beacon_blocks_by_{root,range} do not provide optimistic blocks * handle forkchoice head being pre-merge with block being postmerge * re-enable blocking head updates on validator duties * fix is_optimistic_candidate_block per spec; don't crash with nil future * fix is_optimistic_candidate_block per spec; don't crash with nil future * mark blocks sans execution payloads valid during head update
2022-07-04 20:35:33 +00:00
func is_execution_block*(blck: SomeForkyBeaconBlock): bool =
convert remaining `toFork` -> `kind` for consistency (#5462) We currently have multiple ways to obtain `ConsensusFork` or `LcDataFork` from a forky type. Rename `toFork` to `kind` for a consistent API naming.
2023-09-27 15:10:28 +00:00
when typeof(blck).kind >= ConsensusFork.Bellatrix:
optimistic sync (#3793) * optimistic sync * flag that initially loaded blocks from database might need execution block root filled in * return optimistic status in REST calls * refactor blockslot pruning * ensure beacon_blocks_by_{root,range} do not provide optimistic blocks * handle forkchoice head being pre-merge with block being postmerge * re-enable blocking head updates on validator duties * fix is_optimistic_candidate_block per spec; don't crash with nil future * fix is_optimistic_candidate_block per spec; don't crash with nil future * mark blocks sans execution payloads valid during head update
2022-07-04 20:35:33 +00:00
const defaultExecutionPayload =
default(typeof(blck.body.execution_payload))
blck.body.execution_payload != defaultExecutionPayload
else:
false
keep track of latest blocks for optimistic sync (#3715) When launched with `--light-client-enable` the latest blocks are fetched and optimistic candidate blocks are passed to a callback (log for now). This helps accelerate syncing in the future (optimistic sync).
2022-06-10 14:16:37 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/bellatrix/beacon-chain.md#is_merge_transition_block
v1.1.6 spec updates (#3206)
2021-12-17 06:56:33 +00:00
func is_merge_transition_block(
switch 3 modules in beacon_chain/spec/ from eip4844 to deneb (#4651)
2023-02-22 13:10:00 +00:00
state: bellatrix.BeaconState | capella.BeaconState | deneb.BeaconState,
initial migration from spec/datatypes/{merge => bellatrix} (#3249)
2022-01-06 11:25:35 +00:00
body: bellatrix.BeaconBlockBody | bellatrix.TrustedBeaconBlockBody |
run capella block sanity tests in CI (#4292)
2022-11-07 18:37:48 +00:00
bellatrix.SigVerifiedBeaconBlockBody |
capella.BeaconBlockBody | capella.TrustedBeaconBlockBody |
eip4844 block/slot sanity and transition tests (#4399)
2022-12-08 02:07:41 +00:00
capella.SigVerifiedBeaconBlockBody |
switch 3 modules in beacon_chain/spec/ from eip4844 to deneb (#4651)
2023-02-22 13:10:00 +00:00
deneb.BeaconBlockBody | deneb.TrustedBeaconBlockBody |
deneb.SigVerifiedBeaconBlockBody): bool =
run capella block sanity tests in CI (#4292)
2022-11-07 18:37:48 +00:00
const defaultExecutionPayload = default(typeof(body.execution_payload))
v1.1.6 spec updates (minor, mostly URLs) (#3197)
2021-12-14 21:02:29 +00:00
not is_merge_transition_complete(state) and
run capella block sanity tests in CI (#4292)
2022-11-07 18:37:48 +00:00
body.execution_payload != defaultExecutionPayload
implement forked merge state/block support (#2890) * implement forked state/block support * merge support for containsOrphan; import cleanup; 80-column lines * add merge block header operations and slot sanity fixture * add epoch state transition tests; implement is_valid_gas_limit(), is_merge_block(), is_execution_enabled(), and compute_timestamp_at_slot() * implement process_execution_payload() and add merge deposit operations tests * add merge block sanity tests * add merge case to syncCommitteeParticipants * v1.1.0-beta.5 updates * reduce getTestStates-based memory usage; don't try to REST-serialize ExecutionPayload transactions without underlying support * add execution payload tests; switch var to let in tests/official/
2021-09-27 14:22:58 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/bellatrix/beacon-chain.md#is_execution_enabled
implement forked merge state/block support (#2890) * implement forked state/block support * merge support for containsOrphan; import cleanup; 80-column lines * add merge block header operations and slot sanity fixture * add epoch state transition tests; implement is_valid_gas_limit(), is_merge_block(), is_execution_enabled(), and compute_timestamp_at_slot() * implement process_execution_payload() and add merge deposit operations tests * add merge block sanity tests * add merge case to syncCommitteeParticipants * v1.1.0-beta.5 updates * reduce getTestStates-based memory usage; don't try to REST-serialize ExecutionPayload transactions without underlying support * add execution payload tests; switch var to let in tests/official/
2021-09-27 14:22:58 +00:00
func is_execution_enabled*(
switch 3 modules in beacon_chain/spec/ from eip4844 to deneb (#4651)
2023-02-22 13:10:00 +00:00
state: bellatrix.BeaconState | capella.BeaconState | deneb.BeaconState,
initial migration from spec/datatypes/{merge => bellatrix} (#3249)
2022-01-06 11:25:35 +00:00
body: bellatrix.BeaconBlockBody | bellatrix.TrustedBeaconBlockBody |
run capella block sanity tests in CI (#4292)
2022-11-07 18:37:48 +00:00
bellatrix.SigVerifiedBeaconBlockBody |
capella.BeaconBlockBody | capella.TrustedBeaconBlockBody |
eip4844 block/slot sanity and transition tests (#4399)
2022-12-08 02:07:41 +00:00
capella.SigVerifiedBeaconBlockBody |
switch 3 modules in beacon_chain/spec/ from eip4844 to deneb (#4651)
2023-02-22 13:10:00 +00:00
deneb.BeaconBlockBody | deneb.TrustedBeaconBlockBody |
deneb.SigVerifiedBeaconBlockBody): bool =
v1.1.6 spec updates (#3206)
2021-12-17 06:56:33 +00:00
is_merge_transition_block(state, body) or is_merge_transition_complete(state)
implement forked merge state/block support (#2890) * implement forked state/block support * merge support for containsOrphan; import cleanup; 80-column lines * add merge block header operations and slot sanity fixture * add epoch state transition tests; implement is_valid_gas_limit(), is_merge_block(), is_execution_enabled(), and compute_timestamp_at_slot() * implement process_execution_payload() and add merge deposit operations tests * add merge block sanity tests * add merge case to syncCommitteeParticipants * v1.1.0-beta.5 updates * reduce getTestStates-based memory usage; don't try to REST-serialize ExecutionPayload transactions without underlying support * add execution payload tests; switch var to let in tests/official/
2021-09-27 14:22:58 +00:00
automated consensus spec URL updating to v1.4.0-beta.2 (#5453)
2023-09-21 18:06:51 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.2/specs/bellatrix/beacon-chain.md#compute_timestamp_at_slot
altair fork handling cleanups (#3050) * fix stack overflow crash in REST/debug/getStateV2 * introduce `ForkyXxx` for generic type matching of `Xxx` across branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) - `Some` is already used for other types of type classes * consolidate function naming in BeaconChainDB, use some generics * import `forks.nim` from other spec modules and move `Forked*` helpers around to resolve circular imports * remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout (less data shuffling between the types) * fix several cases of states being stored on stack in tests, causing random failures on some platforms * remove reading json support from ncli - this should be ported to the rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 07:34:34 +00:00
func compute_timestamp_at_slot*(state: ForkyBeaconState, slot: Slot): uint64 =
implement forked merge state/block support (#2890) * implement forked state/block support * merge support for containsOrphan; import cleanup; 80-column lines * add merge block header operations and slot sanity fixture * add epoch state transition tests; implement is_valid_gas_limit(), is_merge_block(), is_execution_enabled(), and compute_timestamp_at_slot() * implement process_execution_payload() and add merge deposit operations tests * add merge block sanity tests * add merge case to syncCommitteeParticipants * v1.1.0-beta.5 updates * reduce getTestStates-based memory usage; don't try to REST-serialize ExecutionPayload transactions without underlying support * add execution payload tests; switch var to let in tests/official/
2021-09-27 14:22:58 +00:00
# Note: This function is unsafe with respect to overflows and underflows.
let slots_since_genesis = slot - GENESIS_SLOT
state.genesis_time + slots_since_genesis * SECONDS_PER_SLOT
post-merge Bellatrix block proposals (#3570) * post-merge Bellatrix block proposals * tolerate running without an Eth1Monitor better * remove obsolete comment * use correct empty receipts root * handle invalid CLI parameters in parseCmdArg overloads
2022-04-14 20:15:34 +00:00
validate EL block hash in EL simulation (#4420) When simulating EL with `--optimistic` flag, perform block hash check.
2022-12-20 08:24:33 +00:00
proc computeTransactionsTrieRoot*(
add helpers for processing EL block header to `libnimbus_lc.a` (#5199) To obtain the correct `transactions_root` and `withdrawals_root`, it is necessary to process execution block header. Light client updates don't contain the correct MPT roots.
2023-08-07 12:23:44 +00:00
payload: ForkyExecutionPayload): ExecutionHash256 =
validate EL block hash in EL simulation (#4420) When simulating EL with `--optimistic` flag, perform block hash check.
2022-12-20 08:24:33 +00:00
if payload.transactions.len == 0:
return EMPTY_ROOT_HASH
var tr = initHexaryTrie(newMemoryDB())
for i, transaction in payload.transactions:
try:
tr.put(rlp.encode(i), distinctBase(transaction)) # Already RLP encoded
except RlpError as exc:
doAssert false, "HexaryTrie.put failed: " & $exc.msg
tr.rootHash()
bump `nim-eth`, extend empty block fallback for Capella (#4357) Implements `emptyPayloadToBlockHeader` for Capella. https://github.com/status-im/nim-eth/pull/562
2022-11-28 13:41:25 +00:00
func toExecutionWithdrawal*(
withdrawal: capella.Withdrawal): ExecutionWithdrawal =
ExecutionWithdrawal(
index: withdrawal.index,
bump `nim-eth`, extend empty block fallback for EIP4844 (#4425) Implements `emptyPayloadToBlockHeader` for EIP-4844. https://github.com/status-im/nim-eth/pull/570
2022-12-20 19:00:56 +00:00
validatorIndex: withdrawal.validator_index,
bump `nim-eth`, extend empty block fallback for Capella (#4357) Implements `emptyPayloadToBlockHeader` for Capella. https://github.com/status-im/nim-eth/pull/562
2022-11-28 13:41:25 +00:00
address: EthAddress withdrawal.address.data,
execution/engine withdrawals amount in uint64 gwei (#4509)
2023-01-14 17:26:57 +00:00
amount: withdrawal.amount)
bump `nim-eth`, extend empty block fallback for Capella (#4357) Implements `emptyPayloadToBlockHeader` for Capella. https://github.com/status-im/nim-eth/pull/562
2022-11-28 13:41:25 +00:00
# https://eips.ethereum.org/EIPS/eip-4895
proc computeWithdrawalsTrieRoot*(
add helpers for processing EL block header to `libnimbus_lc.a` (#5199) To obtain the correct `transactions_root` and `withdrawals_root`, it is necessary to process execution block header. Light client updates don't contain the correct MPT roots.
2023-08-07 12:23:44 +00:00
payload: capella.ExecutionPayload | deneb.ExecutionPayload
): ExecutionHash256 =
bump `nim-eth`, extend empty block fallback for Capella (#4357) Implements `emptyPayloadToBlockHeader` for Capella. https://github.com/status-im/nim-eth/pull/562
2022-11-28 13:41:25 +00:00
if payload.withdrawals.len == 0:
return EMPTY_ROOT_HASH
var tr = initHexaryTrie(newMemoryDB())
for i, withdrawal in payload.withdrawals:
try:
tr.put(rlp.encode(i), rlp.encode(toExecutionWithdrawal(withdrawal)))
except RlpError as exc:
doAssert false, "HexaryTrie.put failed: " & $exc.msg
tr.rootHash()
EIP-4788/parentBeaconBlockRoot fixes (#5318)
2023-08-19 08:38:17 +00:00
proc blockToBlockHeader*(blck: ForkyBeaconBlock): ExecutionBlockHeader =
template payload: auto = blck.body.execution_payload
bump `nim-eth`, extend empty block fallback for Capella (#4357) Implements `emptyPayloadToBlockHeader` for Capella. https://github.com/status-im/nim-eth/pull/562
2022-11-28 13:41:25 +00:00
static: # `GasInt` is signed. We only use it for hashing.
doAssert sizeof(GasInt) == sizeof(payload.gas_limit)
doAssert sizeof(GasInt) == sizeof(payload.gas_used)
let
validate EL block hash in EL simulation (#4420) When simulating EL with `--optimistic` flag, perform block hash check.
2022-12-20 08:24:33 +00:00
txRoot = payload.computeTransactionsTrieRoot()
bump `nim-eth`, extend empty block fallback for Capella (#4357) Implements `emptyPayloadToBlockHeader` for Capella. https://github.com/status-im/nim-eth/pull/562
2022-11-28 13:41:25 +00:00
withdrawalsRoot =
convert remaining `toFork` -> `kind` for consistency (#5462) We currently have multiple ways to obtain `ConsensusFork` or `LcDataFork` from a forky type. Rename `toFork` to `kind` for a consistent API naming.
2023-09-27 15:10:28 +00:00
when typeof(payload).kind >= ConsensusFork.Capella:
bump `nim-eth`, extend empty block fallback for EIP4844 (#4425) Implements `emptyPayloadToBlockHeader` for EIP-4844. https://github.com/status-im/nim-eth/pull/570
2022-12-20 19:00:56 +00:00
some payload.computeWithdrawalsTrieRoot()
else:
add helpers for processing EL block header to `libnimbus_lc.a` (#5199) To obtain the correct `transactions_root` and `withdrawals_root`, it is necessary to process execution block header. Light client updates don't contain the correct MPT roots.
2023-08-07 12:23:44 +00:00
none(ExecutionHash256)
Rename "data gas" to "blob gas" (#5216) * Rename data gas to blob gas * Update vendor/nim-eth and vendor/nim-web3
2023-08-02 22:07:57 +00:00
blobGasUsed =
convert remaining `toFork` -> `kind` for consistency (#5462) We currently have multiple ways to obtain `ConsensusFork` or `LcDataFork` from a forky type. Rename `toFork` to `kind` for a consistent API naming.
2023-09-27 15:10:28 +00:00
when typeof(payload).kind >= ConsensusFork.Deneb:
Rename "data gas" to "blob gas" (#5216) * Rename data gas to blob gas * Update vendor/nim-eth and vendor/nim-web3
2023-08-02 22:07:57 +00:00
some payload.blob_gas_used
use v1.4.0-alpha.1 consensus spec test vectors (#5026)
2023-06-03 21:55:08 +00:00
else:
none(uint64)
Rename "data gas" to "blob gas" (#5216) * Rename data gas to blob gas * Update vendor/nim-eth and vendor/nim-web3
2023-08-02 22:07:57 +00:00
excessBlobGas =
convert remaining `toFork` -> `kind` for consistency (#5462) We currently have multiple ways to obtain `ConsensusFork` or `LcDataFork` from a forky type. Rename `toFork` to `kind` for a consistent API naming.
2023-09-27 15:10:28 +00:00
when typeof(payload).kind >= ConsensusFork.Deneb:
Rename "data gas" to "blob gas" (#5216) * Rename data gas to blob gas * Update vendor/nim-eth and vendor/nim-web3
2023-08-02 22:07:57 +00:00
some payload.excess_blob_gas
bump `nim-eth`, extend empty block fallback for Capella (#4357) Implements `emptyPayloadToBlockHeader` for Capella. https://github.com/status-im/nim-eth/pull/562
2022-11-28 13:41:25 +00:00
else:
use v1.4.0-alpha.1 consensus spec test vectors (#5026)
2023-06-03 21:55:08 +00:00
none(uint64)
EIP-4788/parentBeaconBlockRoot fixes (#5318)
2023-08-19 08:38:17 +00:00
parentBeaconBlockRoot =
convert remaining `toFork` -> `kind` for consistency (#5462) We currently have multiple ways to obtain `ConsensusFork` or `LcDataFork` from a forky type. Rename `toFork` to `kind` for a consistent API naming.
2023-09-27 15:10:28 +00:00
when typeof(payload).kind >= ConsensusFork.Deneb:
EIP-4788/parentBeaconBlockRoot fixes (#5318)
2023-08-19 08:38:17 +00:00
some ExecutionHash256(data: blck.parent_root.data)
else:
none(ExecutionHash256)
bump `nim-eth`, extend empty block fallback for Capella (#4357) Implements `emptyPayloadToBlockHeader` for Capella. https://github.com/status-im/nim-eth/pull/562
2022-11-28 13:41:25 +00:00
Fix the block header computation when proposing an empty execution payload (#3991) * Fix the block header computation when proposing an empty execution payload * Spec compliant base fee calculation when producing empty payloads
2022-08-19 10:28:42 +00:00
ExecutionBlockHeader(
EIP-4788/parentBeaconBlockRoot fixes (#5318)
2023-08-19 08:38:17 +00:00
parentHash : payload.parent_hash,
ommersHash : EMPTY_UNCLE_HASH,
coinbase : EthAddress payload.fee_recipient.data,
stateRoot : payload.state_root,
txRoot : txRoot,
receiptRoot : payload.receipts_root,
bloom : payload.logs_bloom.data,
difficulty : default(DifficultyInt),
blockNumber : payload.block_number.u256,
gasLimit : cast[GasInt](payload.gas_limit),
gasUsed : cast[GasInt](payload.gas_used),
Bump nim-eth: Change block timestamp from std.Time to distinct uint64 (#5510) * Bump nim-eth: Change block timestamp from std.Time to distinct uint64 Also change tx.maxFeePerBlobGas from GasInt to UInt256 following Cancun latest spec * Fix EthTime.now from func to proc due to sideeffects
2023-10-18 00:37:57 +00:00
timestamp : EthTime(int64.saturate payload.timestamp),
EIP-4788/parentBeaconBlockRoot fixes (#5318)
2023-08-19 08:38:17 +00:00
extraData : payload.extra_data.asSeq,
mixDigest : payload.prev_randao, # EIP-4399 `mixDigest` -> `prevRandao`
nonce : default(BlockNonce),
fee : some payload.base_fee_per_gas,
withdrawalsRoot : withdrawalsRoot,
blobGasUsed : blobGasUsed, # EIP-4844
excessBlobGas : excessBlobGas, # EIP-4844
parentBeaconBlockRoot : parentBeaconBlockRoot) # EIP-4788
proc compute_execution_block_hash*(blck: ForkyBeaconBlock): Eth2Digest =
rlpHash blockToBlockHeader(blck)