status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/beacon_chain/spec/signatures.nim

333 lines
13 KiB
Nim
Raw Normal View History

collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
# beacon_chain
initial migration from spec/datatypes/{merge => bellatrix} (#3249)
2022-01-06 11:25:35 +00:00
# Copyright (c) 2018-2022 Status Research & Development GmbH
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
# Licensed and distributed under either of
# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
# at your option. This file may not be copied, modified, or distributed except according to those terms.
{.push raises: [Defect].}
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
## Signature production and verification for spec types - for every type of
## signature, there are 3 functions:
## * `compute_*_signing_root` - reduce message to the data that will be signed
## * `get_*_signature` - sign the signing root with a private key
## * `verify_*_signature` - verify a signature produced by `get_*_signature`
##
## See also `signatures_batch` for batch verification versions of these
## functions.
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
import
initial migration from spec/datatypes/{merge => bellatrix} (#3249)
2022-01-06 11:25:35 +00:00
./datatypes/[phase0, altair, bellatrix], ./helpers, ./eth2_merkleization
rework spec imports (#2779) The spec imports are a mess to work with, so this branch cleans them up a bit to ensure that we avoid generic sandwitches and that importing stuff generally becomes easier. * reexport crypto/digest/presets because these are part of the public symbol set of the rest of the spec types * don't export `merge` types from `base` - this causes circular deps * fix circular deps in `ssz/spec_types` - this is the first step in disentangling ssz from spec * be explicit about phase0 vs altair - longer term, `altair` will become the "natural" type set, then merge and so on, so no point in giving `phase0` special preferential treatment
2021-08-12 13:08:20 +00:00
export phase0, altair
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
template withTrust(sig: SomeSig, body: untyped): bool =
when sig is TrustedSig:
true
else:
body
update some v1.1.0 alpha1 to alpha2 (#2457) * update some v1.1.0 alpha1 to alpha2 * remove unused getDepositMessage overload and move other out of datatypes/base * bump nim-eth2-scenarios to download v1.1.0-alpha.2 test vectors * construct object rather than result
2021-03-29 19:17:48 +00:00
func getDepositMessage(depositData: DepositData): DepositMessage =
DepositMessage(
pubkey: depositData.pubkey,
amount: depositData.amount,
withdrawal_credentials: depositData.withdrawal_credentials)
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
func compute_slot_signing_root*(
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
fork: Fork, genesis_validators_root: Eth2Digest, slot: Slot
): Eth2Digest =
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
let
time: spring cleaning (#3262) Time in the beacon chain is expressed relative to the genesis time - this PR creates a `beacon_time` module that collects helpers and utilities for dealing the time units - the new module does not deal with actual wall time (that's remains in `beacon_clock`). Collecting the time related stuff in one place makes it easier to find, avoids some circular imports and allows more easily identifying the code actually needs wall time to operate. * move genesis-time-related functionality into `spec/beacon_time` * avoid using `chronos.Duration` for time differences - it does not support negative values (such as when something happens earlier than it should) * saturate conversions between `FAR_FUTURE_XXX`, so as to avoid overflows * fix delay reporting in validator client so it uses the expected deadline of the slot, not "closest wall slot" * simplify looping over the slots of an epoch * `compute_start_slot_at_epoch` -> `start_slot` * `compute_epoch_at_slot` -> `epoch` A follow-up PR will (likely) introduce saturating arithmetic for the time units - this is merely code moves, renames and fixing of small bugs.
2022-01-11 10:01:54 +00:00
epoch = epoch(slot)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
domain = get_domain(
fork, DOMAIN_SELECTION_PROOF, epoch, genesis_validators_root)
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
compute_signing_root(slot, domain)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
automated CL spec ref URL updates from v1.1.9 to v1.1.10 (#3455)
2022-03-02 10:00:21 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.10/specs/phase0/validator.md#aggregation-selection
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
func get_slot_signature*(
fork: Fork, genesis_validators_root: Eth2Digest, slot: Slot,
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
privkey: ValidatorPrivKey): CookedSig =
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
let signing_root = compute_slot_signing_root(
fork, genesis_validators_root, slot)
enable `styleCheck:usages` (#3573) Some upstream repos still need fixes, but this gets us close enough that style hints can be enabled by default. In general, "canonical" spellings are preferred even if they violate nep-1 - this applies in particular to spec-related stuff like `genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
blsSign(privkey, signing_root.data)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
lazily initialize validator public keys, which were a third or more of sync CPU usage
2020-07-23 14:38:28 +00:00
proc verify_slot_signature*(
implement aggregated attestation receiving/validating (#1272) * implement aggregated attestation receiving/validating * document the conditions without explicit implementations in isValidAggregatedAttestation()
2020-07-02 16:15:27 +00:00
fork: Fork, genesis_validators_root: Eth2Digest, slot: Slot,
singe validator key cache Instead of keeping a validator key list per EpochRef, this PR introduces a single shared validator key list in ChainDAG, and cleans up some other ChainDAG and key-related issues. The PR does not introduce the validator key list in the state transition - this is because we batch-check all signatures before entering the spec code, thus the spec code never hits the cache. A future refactor should _probably_ remove the threadvar altogether. There's a few other small fixes in here that make the flow easier to read: * fix `var ChainDAGRef` -> `ChainDAGRef` * fix `var QuarantineRef` -> `QuarantineRef` * consistent `dag` variable name * avoid using threadvar pubkey cache in most cases * better error messages in batch signature checking
2021-06-01 11:13:40 +00:00
pubkey: ValidatorPubKey | CookedPubKey, signature: SomeSig): bool =
implement aggregated attestation receiving/validating (#1272) * implement aggregated attestation receiving/validating * document the conditions without explicit implementations in isValidAggregatedAttestation()
2020-07-02 16:15:27 +00:00
withTrust(signature):
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
let signing_root = compute_slot_signing_root(
fork, genesis_validators_root, slot)
implement aggregated attestation receiving/validating (#1272) * implement aggregated attestation receiving/validating * document the conditions without explicit implementations in isValidAggregatedAttestation()
2020-07-02 16:15:27 +00:00
blsVerify(pubkey, signing_root.data, signature)
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
func compute_epoch_signing_root*(
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
fork: Fork, genesis_validators_root: Eth2Digest, epoch: Epoch
): Eth2Digest =
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
let domain = get_domain(fork, DOMAIN_RANDAO, epoch, genesis_validators_root)
compute_signing_root(epoch, domain)
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
automated CL spec ref URL updates from v1.1.9 to v1.1.10 (#3455)
2022-03-02 10:00:21 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.10/specs/phase0/validator.md#randao-reveal
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
func get_epoch_signature*(
fork: Fork, genesis_validators_root: Eth2Digest, epoch: Epoch,
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
privkey: ValidatorPrivKey): CookedSig =
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
let signing_root = compute_epoch_signing_root(
fork, genesis_validators_root, epoch)
enable `styleCheck:usages` (#3573) Some upstream repos still need fixes, but this gets us close enough that style hints can be enabled by default. In general, "canonical" spellings are preferred even if they violate nep-1 - this applies in particular to spec-related stuff like `genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
blsSign(privkey, signing_root.data)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
lazily initialize validator public keys, which were a third or more of sync CPU usage
2020-07-23 14:38:28 +00:00
proc verify_epoch_signature*(
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
fork: Fork, genesis_validators_root: Eth2Digest, epoch: Epoch,
singe validator key cache Instead of keeping a validator key list per EpochRef, this PR introduces a single shared validator key list in ChainDAG, and cleans up some other ChainDAG and key-related issues. The PR does not introduce the validator key list in the state transition - this is because we batch-check all signatures before entering the spec code, thus the spec code never hits the cache. A future refactor should _probably_ remove the threadvar altogether. There's a few other small fixes in here that make the flow easier to read: * fix `var ChainDAGRef` -> `ChainDAGRef` * fix `var QuarantineRef` -> `QuarantineRef` * consistent `dag` variable name * avoid using threadvar pubkey cache in most cases * better error messages in batch signature checking
2021-06-01 11:13:40 +00:00
pubkey: ValidatorPubKey | CookedPubKey, signature: SomeSig): bool =
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
withTrust(signature):
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
let signing_root = compute_epoch_signing_root(
fork, genesis_validators_root, epoch)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
blsVerify(pubkey, signing_root.data, signature)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
func compute_block_signing_root*(
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
fork: Fork, genesis_validators_root: Eth2Digest, slot: Slot,
Store finalized block roots in database (3s startup) (#3320) * Store finalized block roots in database (3s startup) When the chain has finalized a checkpoint, the history from that point onwards becomes linear - this is exploited in `.era` files to allow constant-time by-slot lookups. In the database, we can do the same by storing finalized block roots in a simple sparse table indexed by slot, bringing the two representations closer to each other in terms of conceptual layout and performance. Doing so has a number of interesting effects: * mainnet startup time is improved 3-5x (3s on my laptop) * the _first_ startup might take slightly longer as the new index is being built - ~10s on the same laptop * we no longer rely on the beacon block summaries to load the full dag - this is a lot faster because we no longer have to look up each block by parent root * a collateral benefit is that we no longer need to load the full summaries table into memory - we get the RSS benefits of #3164 without the CPU hit. Other random stuff: * simplify forky block generics * fix withManyWrites multiple evaluation * fix validator key cache not being updated properly in chaindag read-only mode * drop pre-altair summaries from `kvstore` * recreate missing summaries from altair+ blocks as well (in case database has lost some to an involuntary restart) * print database startup timings in chaindag load log * avoid allocating superfluos state at startup * use a recursive sql query to load the summaries of the unfinalized blocks
2022-01-30 16:51:04 +00:00
blck: Eth2Digest | SomeForkyBeaconBlock | BeaconBlockHeader): Eth2Digest =
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
let
time: spring cleaning (#3262) Time in the beacon chain is expressed relative to the genesis time - this PR creates a `beacon_time` module that collects helpers and utilities for dealing the time units - the new module does not deal with actual wall time (that's remains in `beacon_clock`). Collecting the time related stuff in one place makes it easier to find, avoids some circular imports and allows more easily identifying the code actually needs wall time to operate. * move genesis-time-related functionality into `spec/beacon_time` * avoid using `chronos.Duration` for time differences - it does not support negative values (such as when something happens earlier than it should) * saturate conversions between `FAR_FUTURE_XXX`, so as to avoid overflows * fix delay reporting in validator client so it uses the expected deadline of the slot, not "closest wall slot" * simplify looping over the slots of an epoch * `compute_start_slot_at_epoch` -> `start_slot` * `compute_epoch_at_slot` -> `epoch` A follow-up PR will (likely) introduce saturating arithmetic for the time units - this is merely code moves, renames and fixing of small bugs.
2022-01-11 10:01:54 +00:00
epoch = epoch(slot)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
domain = get_domain(
fork, DOMAIN_BEACON_PROPOSER, epoch, genesis_validators_root)
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
compute_signing_root(blck, domain)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
automated CL spec ref URL updates from v1.1.9 to v1.1.10 (#3455)
2022-03-02 10:00:21 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.10/specs/phase0/validator.md#signature
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
func get_block_signature*(
fork: Fork, genesis_validators_root: Eth2Digest, slot: Slot,
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
root: Eth2Digest, privkey: ValidatorPrivKey): CookedSig =
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
let signing_root = compute_block_signing_root(
fork, genesis_validators_root, slot, root)
enable `styleCheck:usages` (#3573) Some upstream repos still need fixes, but this gets us close enough that style hints can be enabled by default. In general, "canonical" spellings are preferred even if they violate nep-1 - this applies in particular to spec-related stuff like `genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
blsSign(privkey, signing_root.data)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
lazily initialize validator public keys, which were a third or more of sync CPU usage
2020-07-23 14:38:28 +00:00
proc verify_block_signature*(
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
fork: Fork, genesis_validators_root: Eth2Digest, slot: Slot,
Store finalized block roots in database (3s startup) (#3320) * Store finalized block roots in database (3s startup) When the chain has finalized a checkpoint, the history from that point onwards becomes linear - this is exploited in `.era` files to allow constant-time by-slot lookups. In the database, we can do the same by storing finalized block roots in a simple sparse table indexed by slot, bringing the two representations closer to each other in terms of conceptual layout and performance. Doing so has a number of interesting effects: * mainnet startup time is improved 3-5x (3s on my laptop) * the _first_ startup might take slightly longer as the new index is being built - ~10s on the same laptop * we no longer rely on the beacon block summaries to load the full dag - this is a lot faster because we no longer have to look up each block by parent root * a collateral benefit is that we no longer need to load the full summaries table into memory - we get the RSS benefits of #3164 without the CPU hit. Other random stuff: * simplify forky block generics * fix withManyWrites multiple evaluation * fix validator key cache not being updated properly in chaindag read-only mode * drop pre-altair summaries from `kvstore` * recreate missing summaries from altair+ blocks as well (in case database has lost some to an involuntary restart) * print database startup timings in chaindag load log * avoid allocating superfluos state at startup * use a recursive sql query to load the summaries of the unfinalized blocks
2022-01-30 16:51:04 +00:00
blck: Eth2Digest | SomeForkyBeaconBlock | BeaconBlockHeader,
singe validator key cache Instead of keeping a validator key list per EpochRef, this PR introduces a single shared validator key list in ChainDAG, and cleans up some other ChainDAG and key-related issues. The PR does not introduce the validator key list in the state transition - this is because we batch-check all signatures before entering the spec code, thus the spec code never hits the cache. A future refactor should _probably_ remove the threadvar altogether. There's a few other small fixes in here that make the flow easier to read: * fix `var ChainDAGRef` -> `ChainDAGRef` * fix `var QuarantineRef` -> `QuarantineRef` * consistent `dag` variable name * avoid using threadvar pubkey cache in most cases * better error messages in batch signature checking
2021-06-01 11:13:40 +00:00
pubkey: ValidatorPubKey | CookedPubKey, signature: SomeSig): bool =
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
withTrust(signature):
let
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
signing_root = compute_block_signing_root(
fork, genesis_validators_root, slot, blck)
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
blsVerify(pubkey, signing_root.data, signature)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
func compute_aggregate_and_proof_signing_root*(
fork: Fork, genesis_validators_root: Eth2Digest,
aggregate_and_proof: AggregateAndProof): Eth2Digest =
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
let
time: spring cleaning (#3262) Time in the beacon chain is expressed relative to the genesis time - this PR creates a `beacon_time` module that collects helpers and utilities for dealing the time units - the new module does not deal with actual wall time (that's remains in `beacon_clock`). Collecting the time related stuff in one place makes it easier to find, avoids some circular imports and allows more easily identifying the code actually needs wall time to operate. * move genesis-time-related functionality into `spec/beacon_time` * avoid using `chronos.Duration` for time differences - it does not support negative values (such as when something happens earlier than it should) * saturate conversions between `FAR_FUTURE_XXX`, so as to avoid overflows * fix delay reporting in validator client so it uses the expected deadline of the slot, not "closest wall slot" * simplify looping over the slots of an epoch * `compute_start_slot_at_epoch` -> `start_slot` * `compute_epoch_at_slot` -> `epoch` A follow-up PR will (likely) introduce saturating arithmetic for the time units - this is merely code moves, renames and fixing of small bugs.
2022-01-11 10:01:54 +00:00
epoch = epoch(aggregate_and_proof.aggregate.data.slot)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
domain = get_domain(
fork, DOMAIN_AGGREGATE_AND_PROOF, epoch, genesis_validators_root)
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
compute_signing_root(aggregate_and_proof, domain)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
automated CL spec ref URL updates from v1.1.9 to v1.1.10 (#3455)
2022-03-02 10:00:21 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.10/specs/phase0/validator.md#broadcast-aggregate
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
func get_aggregate_and_proof_signature*(fork: Fork, genesis_validators_root: Eth2Digest,
aggregate_and_proof: AggregateAndProof,
enable `styleCheck:usages` (#3573) Some upstream repos still need fixes, but this gets us close enough that style hints can be enabled by default. In general, "canonical" spellings are preferred even if they violate nep-1 - this applies in particular to spec-related stuff like `genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
privkey: ValidatorPrivKey): CookedSig =
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
let signing_root = compute_aggregate_and_proof_signing_root(
fork, genesis_validators_root, aggregate_and_proof)
enable `styleCheck:usages` (#3573) Some upstream repos still need fixes, but this gets us close enough that style hints can be enabled by default. In general, "canonical" spellings are preferred even if they violate nep-1 - this applies in particular to spec-related stuff like `genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
blsSign(privkey, signing_root.data)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
singe validator key cache Instead of keeping a validator key list per EpochRef, this PR introduces a single shared validator key list in ChainDAG, and cleans up some other ChainDAG and key-related issues. The PR does not introduce the validator key list in the state transition - this is because we batch-check all signatures before entering the spec code, thus the spec code never hits the cache. A future refactor should _probably_ remove the threadvar altogether. There's a few other small fixes in here that make the flow easier to read: * fix `var ChainDAGRef` -> `ChainDAGRef` * fix `var QuarantineRef` -> `QuarantineRef` * consistent `dag` variable name * avoid using threadvar pubkey cache in most cases * better error messages in batch signature checking
2021-06-01 11:13:40 +00:00
proc verify_aggregate_and_proof_signature*(
fork: Fork, genesis_validators_root: Eth2Digest,
aggregate_and_proof: AggregateAndProof,
pubkey: ValidatorPubKey | CookedPubKey, signature: SomeSig): bool =
implement aggregated attestation receiving/validating (#1272) * implement aggregated attestation receiving/validating * document the conditions without explicit implementations in isValidAggregatedAttestation()
2020-07-02 16:15:27 +00:00
withTrust(signature):
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
let signing_root = compute_aggregate_and_proof_signing_root(
fork, genesis_validators_root, aggregate_and_proof)
implement aggregated attestation receiving/validating (#1272) * implement aggregated attestation receiving/validating * document the conditions without explicit implementations in isValidAggregatedAttestation()
2020-07-02 16:15:27 +00:00
Implementation of the Keymanager API (BETA) https://github.com/ethereum/keymanager-APIs
2021-12-22 12:37:31 +00:00
blsVerify(pubkey, signing_root.data, signature)
implement aggregated attestation receiving/validating (#1272) * implement aggregated attestation receiving/validating * document the conditions without explicit implementations in isValidAggregatedAttestation()
2020-07-02 16:15:27 +00:00
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
func compute_attestation_signing_root*(
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
fork: Fork, genesis_validators_root: Eth2Digest,
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
attestation_data: AttestationData): Eth2Digest =
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
let
epoch = attestation_data.target.epoch
domain = get_domain(
fork, DOMAIN_BEACON_ATTESTER, epoch, genesis_validators_root)
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
compute_signing_root(attestation_data, domain)
automated CL spec ref URL updates from v1.1.9 to v1.1.10 (#3455)
2022-03-02 10:00:21 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.10/specs/phase0/validator.md#aggregate-signature
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
func get_attestation_signature*(
fork: Fork, genesis_validators_root: Eth2Digest,
attestation_data: AttestationData,
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
privkey: ValidatorPrivKey): CookedSig =
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
let signing_root = compute_attestation_signing_root(
fork, genesis_validators_root, attestation_data)
enable `styleCheck:usages` (#3573) Some upstream repos still need fixes, but this gets us close enough that style hints can be enabled by default. In general, "canonical" spellings are preferred even if they violate nep-1 - this applies in particular to spec-related stuff like `genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
blsSign(privkey, signing_root.data)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
lazily initialize validator public keys, which were a third or more of sync CPU usage
2020-07-23 14:38:28 +00:00
proc verify_attestation_signature*(
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
fork: Fork, genesis_validators_root: Eth2Digest,
attestation_data: AttestationData,
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
pubkeys: auto, signature: SomeSig): bool =
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
withTrust(signature):
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
let signing_root = compute_attestation_signing_root(
fork, genesis_validators_root, attestation_data)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
blsFastAggregateVerify(pubkeys, signing_root.data, signature)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
func compute_deposit_signing_root*(
version: Version,
deposit_message: DepositMessage): Eth2Digest =
let
# Fork-agnostic domain since deposits are valid across forks
domain = compute_domain(DOMAIN_DEPOSIT, version)
compute_signing_root(deposit_message, domain)
automated CL spec ref URL updates from v1.1.9 to v1.1.10 (#3455)
2022-03-02 10:00:21 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.10/specs/phase0/beacon-chain.md#deposits
Implement split preset/config support (#2710) * Implement split preset/config support This is the initial bulk refactor to introduce runtime config values in a number of places, somewhat replacing the existing mechanism of loading network metadata. It still needs more work, this is the initial refactor that introduces runtime configuration in some of the places that need it. The PR changes the way presets and constants work, to match the spec. In particular, a "preset" now refers to the compile-time configuration while a "cfg" or "RuntimeConfig" is the dynamic part. A single binary can support either mainnet or minimal, but not both. Support for other presets has been removed completely (can be readded, in case there's need). There's a number of outstanding tasks: * `SECONDS_PER_SLOT` still needs fixing * loading custom runtime configs needs redoing * checking constants against YAML file * yeerongpilly support `build/nimbus_beacon_node --network=yeerongpilly --discv5:no --log-level=DEBUG` * load fork epoch from config * fix fork digest sent in status * nicer error string for request failures * fix tools * one more * fixup * fixup * fixup * use "standard" network definition folder in local testnet Files are loaded from their standard locations, including genesis etc, to conform to the format used in the `eth2-networks` repo. * fix launch scripts, allow unknown config values * fix base config of rest test * cleanups * bundle mainnet config using common loader * fix spec links and names * only include supported preset in binary * drop yeerongpilly, add altair-devnet-0, support boot_enr.yaml
2021-07-12 13:01:38 +00:00
func get_deposit_signature*(preset: RuntimeConfig,
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
deposit: DepositData,
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
privkey: ValidatorPrivKey): CookedSig =
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
let signing_root = compute_deposit_signing_root(
preset.GENESIS_FORK_VERSION, deposit.getDepositMessage())
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
enable `styleCheck:usages` (#3573) Some upstream repos still need fixes, but this gets us close enough that style hints can be enabled by default. In general, "canonical" spellings are preferred even if they violate nep-1 - this applies in particular to spec-related stuff like `genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
blsSign(privkey, signing_root.data)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
Remote signing client/server. (#3077)
2021-11-30 01:20:21 +00:00
func get_deposit_signature*(message: DepositMessage, version: Version,
privkey: ValidatorPrivKey): CookedSig =
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
let signing_root = compute_deposit_signing_root(version, message)
Remote signing client/server. (#3077)
2021-11-30 01:20:21 +00:00
blsSign(privkey, signing_root.data)
Implement split preset/config support (#2710) * Implement split preset/config support This is the initial bulk refactor to introduce runtime config values in a number of places, somewhat replacing the existing mechanism of loading network metadata. It still needs more work, this is the initial refactor that introduces runtime configuration in some of the places that need it. The PR changes the way presets and constants work, to match the spec. In particular, a "preset" now refers to the compile-time configuration while a "cfg" or "RuntimeConfig" is the dynamic part. A single binary can support either mainnet or minimal, but not both. Support for other presets has been removed completely (can be readded, in case there's need). There's a number of outstanding tasks: * `SECONDS_PER_SLOT` still needs fixing * loading custom runtime configs needs redoing * checking constants against YAML file * yeerongpilly support `build/nimbus_beacon_node --network=yeerongpilly --discv5:no --log-level=DEBUG` * load fork epoch from config * fix fork digest sent in status * nicer error string for request failures * fix tools * one more * fixup * fixup * fixup * use "standard" network definition folder in local testnet Files are loaded from their standard locations, including genesis etc, to conform to the format used in the `eth2-networks` repo. * fix launch scripts, allow unknown config values * fix base config of rest test * cleanups * bundle mainnet config using common loader * fix spec links and names * only include supported preset in binary * drop yeerongpilly, add altair-devnet-0, support boot_enr.yaml
2021-07-12 13:01:38 +00:00
proc verify_deposit_signature*(preset: RuntimeConfig,
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
deposit: DepositData): bool =
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
let
deposit_message = deposit.getDepositMessage()
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
signing_root = compute_deposit_signing_root(
preset.GENESIS_FORK_VERSION, deposit_message)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
blsVerify(deposit.pubkey, signing_root.data, deposit.signature)
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
func compute_voluntary_exit_signing_root*(
fork: Fork, genesis_validators_root: Eth2Digest,
voluntary_exit: VoluntaryExit): Eth2Digest =
let
epoch = voluntary_exit.epoch
domain = get_domain(
fork, DOMAIN_VOLUNTARY_EXIT, epoch, genesis_validators_root)
compute_signing_root(voluntary_exit, domain)
Implement the 'deposits exit' command; Remove 'deposits create'
2020-11-27 19:48:33 +00:00
func get_voluntary_exit_signature*(
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
fork: Fork, genesis_validators_root: Eth2Digest,
Implement the 'deposits exit' command; Remove 'deposits create'
2020-11-27 19:48:33 +00:00
voluntary_exit: VoluntaryExit,
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
privkey: ValidatorPrivKey): CookedSig =
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
let signing_root = compute_voluntary_exit_signing_root(
fork, genesis_validators_root, voluntary_exit)
Implement the 'deposits exit' command; Remove 'deposits create'
2020-11-27 19:48:33 +00:00
enable `styleCheck:usages` (#3573) Some upstream repos still need fixes, but this gets us close enough that style hints can be enabled by default. In general, "canonical" spellings are preferred even if they violate nep-1 - this applies in particular to spec-related stuff like `genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
blsSign(privkey, signing_root.data)
Implement the 'deposits exit' command; Remove 'deposits create'
2020-11-27 19:48:33 +00:00
lazily initialize validator public keys, which were a third or more of sync CPU usage
2020-07-23 14:38:28 +00:00
proc verify_voluntary_exit_signature*(
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
fork: Fork, genesis_validators_root: Eth2Digest,
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
voluntary_exit: VoluntaryExit,
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
pubkey: ValidatorPubKey | CookedPubKey, signature: SomeSig): bool =
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
withTrust(signature):
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
let signing_root = compute_voluntary_exit_signing_root(
fork, genesis_validators_root, voluntary_exit)
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
blsVerify(pubkey, signing_root.data, signature)
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
automated CL spec ref URL updates from v1.1.9 to v1.1.10 (#3455)
2022-03-02 10:00:21 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.10/specs/altair/validator.md#prepare-sync-committee-message
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
func compute_sync_committee_message_signing_root*(
fork: Fork, genesis_validators_root: Eth2Digest,
slot: Slot, beacon_block_root: Eth2Digest): Eth2Digest =
let domain = get_domain(
fork, DOMAIN_SYNC_COMMITTEE, slot.epoch, genesis_validators_root)
compute_signing_root(beacon_block_root, domain)
Remote signing client/server. (#3077)
2021-11-30 01:20:21 +00:00
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
func get_sync_committee_message_signature*(
fork: Fork, genesis_validators_root: Eth2Digest,
slot: Slot, beacon_block_root: Eth2Digest,
privkey: ValidatorPrivKey): CookedSig =
let signing_root = compute_sync_committee_message_signing_root(
fork, genesis_validators_root, slot, beacon_block_root)
Remote signing client/server. (#3077)
2021-11-30 01:20:21 +00:00
blsSign(privkey, signing_root.data)
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
proc verify_sync_committee_message_signature*(
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
fork: Fork, genesis_validators_root: Eth2Digest,
slot: Slot, beacon_block_root: Eth2Digest,
pubkey: ValidatorPubKey | CookedPubKey, signature: SomeSig): bool =
let signing_root = compute_sync_committee_message_signing_root(
fork, genesis_validators_root, slot, beacon_block_root)
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
blsVerify(pubkey, signing_root.data, signature)
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
proc verify_sync_committee_signature*(
fork: Fork, genesis_validators_root: Eth2Digest,
slot: Slot, beacon_block_root: Eth2Digest,
pubkeys: auto, signature: SomeSig): bool =
let signing_root = compute_sync_committee_message_signing_root(
fork, genesis_validators_root, slot, beacon_block_root)
blsFastAggregateVerify(pubkeys, signing_root.data, signature)
automated CL spec ref URL updates from v1.1.9 to v1.1.10 (#3455)
2022-03-02 10:00:21 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.10/specs/altair/validator.md#aggregation-selection
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
func compute_sync_committee_selection_proof_signing_root*(
fork: Fork, genesis_validators_root: Eth2Digest,
slot: Slot, subcommittee_index: uint64): Eth2Digest =
let
domain = get_domain(fork, DOMAIN_SYNC_COMMITTEE_SELECTION_PROOF,
slot.epoch, genesis_validators_root)
signing_data = SyncAggregatorSelectionData(
slot: slot,
subcommittee_index: subcommittee_index)
compute_signing_root(signing_data, domain)
func get_sync_committee_selection_proof*(
fork: Fork, genesis_validators_root: Eth2Digest,
slot: Slot, subcommittee_index: uint64,
privkey: ValidatorPrivKey): CookedSig =
let signing_root = compute_sync_committee_selection_proof_signing_root(
fork, genesis_validators_root, slot, subcommittee_index)
blsSign(privkey, signing_root.data)
proc verify_sync_committee_selection_proof*(
fork: Fork, genesis_validators_root: Eth2Digest,
slot: Slot, subcommittee_index: uint64,
pubkey: ValidatorPubKey | CookedPubKey, signature: SomeSig): bool =
withTrust(signature):
let signing_root = compute_sync_committee_selection_proof_signing_root(
fork, genesis_validators_root, slot, subcommittee_index)
blsVerify(pubkey, signing_root.data, signature)
automated CL spec ref URL updates from v1.1.9 to v1.1.10 (#3455)
2022-03-02 10:00:21 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.10/specs/altair/validator.md#signature
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
func compute_contribution_and_proof_signing_root*(
fork: Fork, genesis_validators_root: Eth2Digest,
msg: ContributionAndProof): Eth2Digest =
let domain = get_domain(fork, DOMAIN_CONTRIBUTION_AND_PROOF,
msg.contribution.slot.epoch,
genesis_validators_root)
compute_signing_root(msg, domain)
proc get_contribution_and_proof_signature*(
fork: Fork, genesis_validators_root: Eth2Digest,
msg: ContributionAndProof,
privkey: ValidatorPrivKey): CookedSig =
let signing_root = compute_contribution_and_proof_signing_root(
fork, genesis_validators_root, msg)
blsSign(privkey, signing_root.data)
automated CL spec ref URL updates from v1.1.9 to v1.1.10 (#3455)
2022-03-02 10:00:21 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.10/specs/altair/validator.md#aggregation-selection
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
proc is_sync_committee_aggregator*(signature: ValidatorSig): bool =
let
signatureDigest = eth2digest(signature.blob)
modulo = max(1'u64, (SYNC_COMMITTEE_SIZE div SYNC_COMMITTEE_SUBNET_COUNT) div TARGET_AGGREGATORS_PER_SYNC_SUBCOMMITTEE)
bytes_to_uint64(signatureDigest.data.toOpenArray(0, 7)) mod modulo == 0
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
proc verify_contribution_and_proof_signature*(
fork: Fork, genesis_validators_root: Eth2Digest,
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
msg: ContributionAndProof,
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
pubkey: ValidatorPubKey | CookedPubKey, signature: SomeSig): bool =
let signing_root = compute_contribution_and_proof_signing_root(
fork, genesis_validators_root, msg)
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
batch-verify sync messages for a small perf boost (#3151) * batch-verify sync messages for a small perf boost Generally reuses the same structure as attestation and aggregate verification * normalize `signatures` and `signature_batch` to use the same pattern of verification * normalize parameter names, order etc for signature stuff in general * avoid calling `blsSign` directly - instead, go through `signatures` consistently
2021-12-09 12:56:54 +00:00
blsVerify(pubkey, signing_root.data, signature)