status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/beacon_chain/spec/signatures.nim

293 lines
12 KiB
Nim
Raw Normal View History

collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
# beacon_chain
bump spec refs from v1.0.0 to v1.0.1 and update copyright years (#2357)
2021-02-25 13:37:22 +00:00
# Copyright (c) 2018-2021 Status Research & Development GmbH
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
# Licensed and distributed under either of
# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
# at your option. This file may not be copied, modified, or distributed except according to those terms.
{.push raises: [Defect].}
import
implement forked merge state/block support (#2890) * implement forked state/block support * merge support for containsOrphan; import cleanup; 80-column lines * add merge block header operations and slot sanity fixture * add epoch state transition tests; implement is_valid_gas_limit(), is_merge_block(), is_execution_enabled(), and compute_timestamp_at_slot() * implement process_execution_payload() and add merge deposit operations tests * add merge block sanity tests * add merge case to syncCommitteeParticipants * v1.1.0-beta.5 updates * reduce getTestStates-based memory usage; don't try to REST-serialize ExecutionPayload transactions without underlying support * add execution payload tests; switch var to let in tests/official/
2021-09-27 14:22:58 +00:00
./datatypes/[phase0, altair, merge], ./helpers, ./eth2_merkleization
rework spec imports (#2779) The spec imports are a mess to work with, so this branch cleans them up a bit to ensure that we avoid generic sandwitches and that importing stuff generally becomes easier. * reexport crypto/digest/presets because these are part of the public symbol set of the rest of the spec types * don't export `merge` types from `base` - this causes circular deps * fix circular deps in `ssz/spec_types` - this is the first step in disentangling ssz from spec * be explicit about phase0 vs altair - longer term, `altair` will become the "natural" type set, then merge and so on, so no point in giving `phase0` special preferential treatment
2021-08-12 13:08:20 +00:00
export phase0, altair
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
template withTrust(sig: SomeSig, body: untyped): bool =
when sig is TrustedSig:
true
else:
body
update some v1.1.0 alpha1 to alpha2 (#2457) * update some v1.1.0 alpha1 to alpha2 * remove unused getDepositMessage overload and move other out of datatypes/base * bump nim-eth2-scenarios to download v1.1.0-alpha.2 test vectors * construct object rather than result
2021-03-29 19:17:48 +00:00
func getDepositMessage(depositData: DepositData): DepositMessage =
DepositMessage(
pubkey: depositData.pubkey,
amount: depositData.amount,
withdrawal_credentials: depositData.withdrawal_credentials)
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
func compute_slot_root*(
fork: Fork, genesis_validators_root: Eth2Digest, slot: Slot
): Eth2Digest =
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
let
epoch = compute_epoch_at_slot(slot)
domain = get_domain(
fork, DOMAIN_SELECTION_PROOF, epoch, genesis_validators_root)
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
compute_signing_root(slot, domain)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
manually-verified v1.1.5 spec URL updates (#3068)
2021-11-09 08:54:59 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/validator.md#aggregation-selection
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
func get_slot_signature*(
fork: Fork, genesis_validators_root: Eth2Digest, slot: Slot,
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
privkey: ValidatorPrivKey): CookedSig =
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
blsSign(privKey, compute_slot_root(fork, genesis_validators_root, slot).data)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
lazily initialize validator public keys, which were a third or more of sync CPU usage
2020-07-23 14:38:28 +00:00
proc verify_slot_signature*(
implement aggregated attestation receiving/validating (#1272) * implement aggregated attestation receiving/validating * document the conditions without explicit implementations in isValidAggregatedAttestation()
2020-07-02 16:15:27 +00:00
fork: Fork, genesis_validators_root: Eth2Digest, slot: Slot,
singe validator key cache Instead of keeping a validator key list per EpochRef, this PR introduces a single shared validator key list in ChainDAG, and cleans up some other ChainDAG and key-related issues. The PR does not introduce the validator key list in the state transition - this is because we batch-check all signatures before entering the spec code, thus the spec code never hits the cache. A future refactor should _probably_ remove the threadvar altogether. There's a few other small fixes in here that make the flow easier to read: * fix `var ChainDAGRef` -> `ChainDAGRef` * fix `var QuarantineRef` -> `QuarantineRef` * consistent `dag` variable name * avoid using threadvar pubkey cache in most cases * better error messages in batch signature checking
2021-06-01 11:13:40 +00:00
pubkey: ValidatorPubKey | CookedPubKey, signature: SomeSig): bool =
implement aggregated attestation receiving/validating (#1272) * implement aggregated attestation receiving/validating * document the conditions without explicit implementations in isValidAggregatedAttestation()
2020-07-02 16:15:27 +00:00
withTrust(signature):
let
epoch = compute_epoch_at_slot(slot)
domain = get_domain(
fork, DOMAIN_SELECTION_PROOF, epoch, genesis_validators_root)
signing_root = compute_signing_root(slot, domain)
blsVerify(pubkey, signing_root.data, signature)
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
func compute_epoch_root*(
fork: Fork, genesis_validators_root: Eth2Digest, epoch: Epoch
): Eth2Digest =
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
let domain = get_domain(fork, DOMAIN_RANDAO, epoch, genesis_validators_root)
compute_signing_root(epoch, domain)
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/validator.md#randao-reveal
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
func get_epoch_signature*(
fork: Fork, genesis_validators_root: Eth2Digest, epoch: Epoch,
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
privkey: ValidatorPrivKey): CookedSig =
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
blsSign(privKey, compute_epoch_root(fork, genesis_validators_root, epoch).data)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
lazily initialize validator public keys, which were a third or more of sync CPU usage
2020-07-23 14:38:28 +00:00
proc verify_epoch_signature*(
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
fork: Fork, genesis_validators_root: Eth2Digest, epoch: Epoch,
singe validator key cache Instead of keeping a validator key list per EpochRef, this PR introduces a single shared validator key list in ChainDAG, and cleans up some other ChainDAG and key-related issues. The PR does not introduce the validator key list in the state transition - this is because we batch-check all signatures before entering the spec code, thus the spec code never hits the cache. A future refactor should _probably_ remove the threadvar altogether. There's a few other small fixes in here that make the flow easier to read: * fix `var ChainDAGRef` -> `ChainDAGRef` * fix `var QuarantineRef` -> `QuarantineRef` * consistent `dag` variable name * avoid using threadvar pubkey cache in most cases * better error messages in batch signature checking
2021-06-01 11:13:40 +00:00
pubkey: ValidatorPubKey | CookedPubKey, signature: SomeSig): bool =
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
withTrust(signature):
let
domain = get_domain(fork, DOMAIN_RANDAO, epoch, genesis_validators_root)
signing_root = compute_signing_root(epoch, domain)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
blsVerify(pubkey, signing_root.data, signature)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
func compute_block_root*(
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
fork: Fork, genesis_validators_root: Eth2Digest, slot: Slot,
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
root: Eth2Digest): Eth2Digest =
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
let
epoch = compute_epoch_at_slot(slot)
domain = get_domain(
fork, DOMAIN_BEACON_PROPOSER, epoch, genesis_validators_root)
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
compute_signing_root(root, domain)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/validator.md#signature
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
func get_block_signature*(
fork: Fork, genesis_validators_root: Eth2Digest, slot: Slot,
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
root: Eth2Digest, privkey: ValidatorPrivKey): CookedSig =
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
blsSign(privKey, compute_block_root(fork, genesis_validators_root, slot, root).data)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
lazily initialize validator public keys, which were a third or more of sync CPU usage
2020-07-23 14:38:28 +00:00
proc verify_block_signature*(
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
fork: Fork, genesis_validators_root: Eth2Digest, slot: Slot,
proposed structure for altair (#2323) * proposed structure for hf1 * refactor datatypes.nim into datatypes/{base, phase0, hf1}.nim * hf1 is Altair * some syncing with alpha 2 * adjust epoch processing to disambiguate access to RewardFlags * relocate StateData to stay consistent with meaning phase 0 StateData * passes v1.1.0 alpha 5 SSZ consensus object tests * Altair block header test fixtures work * fix slash_validator() so that Altair attester slashings, proposer slashings, and voluntary exit textures work * deposit operation Altair test fixtures work * slot sanity and all but a couple epoch transition tests switched to Altair * attestation Altair test fixtures work * Altair block sanity test fixtures work * add working altair sync committee tests * improve workarounds for sum-types-across-modules Nim bug; incorporate SignedBeaconBlock root reconstuction to SSZ byte reader
2021-05-28 15:25:58 +00:00
blck: Eth2Digest | SomeSomeBeaconBlock | BeaconBlockHeader,
singe validator key cache Instead of keeping a validator key list per EpochRef, this PR introduces a single shared validator key list in ChainDAG, and cleans up some other ChainDAG and key-related issues. The PR does not introduce the validator key list in the state transition - this is because we batch-check all signatures before entering the spec code, thus the spec code never hits the cache. A future refactor should _probably_ remove the threadvar altogether. There's a few other small fixes in here that make the flow easier to read: * fix `var ChainDAGRef` -> `ChainDAGRef` * fix `var QuarantineRef` -> `QuarantineRef` * consistent `dag` variable name * avoid using threadvar pubkey cache in most cases * better error messages in batch signature checking
2021-06-01 11:13:40 +00:00
pubkey: ValidatorPubKey | CookedPubKey, signature: SomeSig): bool =
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
withTrust(signature):
let
epoch = compute_epoch_at_slot(slot)
domain = get_domain(
fork, DOMAIN_BEACON_PROPOSER, epoch, genesis_validators_root)
signing_root = compute_signing_root(blck, domain)
blsVerify(pubKey, signing_root.data, signature)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
func compute_aggregate_and_proof_root*(fork: Fork, genesis_validators_root: Eth2Digest,
aggregate_and_proof: AggregateAndProof,
): Eth2Digest =
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
let
epoch = compute_epoch_at_slot(aggregate_and_proof.aggregate.data.slot)
domain = get_domain(
fork, DOMAIN_AGGREGATE_AND_PROOF, epoch, genesis_validators_root)
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
compute_signing_root(aggregate_and_proof, domain)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
manually-verified v1.1.5 spec URL updates (#3068)
2021-11-09 08:54:59 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/validator.md#broadcast-aggregate
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
func get_aggregate_and_proof_signature*(fork: Fork, genesis_validators_root: Eth2Digest,
aggregate_and_proof: AggregateAndProof,
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
privKey: ValidatorPrivKey): CookedSig =
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
blsSign(privKey, compute_aggregate_and_proof_root(fork, genesis_validators_root,
aggregate_and_proof).data)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
singe validator key cache Instead of keeping a validator key list per EpochRef, this PR introduces a single shared validator key list in ChainDAG, and cleans up some other ChainDAG and key-related issues. The PR does not introduce the validator key list in the state transition - this is because we batch-check all signatures before entering the spec code, thus the spec code never hits the cache. A future refactor should _probably_ remove the threadvar altogether. There's a few other small fixes in here that make the flow easier to read: * fix `var ChainDAGRef` -> `ChainDAGRef` * fix `var QuarantineRef` -> `QuarantineRef` * consistent `dag` variable name * avoid using threadvar pubkey cache in most cases * better error messages in batch signature checking
2021-06-01 11:13:40 +00:00
proc verify_aggregate_and_proof_signature*(
fork: Fork, genesis_validators_root: Eth2Digest,
aggregate_and_proof: AggregateAndProof,
pubkey: ValidatorPubKey | CookedPubKey, signature: SomeSig): bool =
implement aggregated attestation receiving/validating (#1272) * implement aggregated attestation receiving/validating * document the conditions without explicit implementations in isValidAggregatedAttestation()
2020-07-02 16:15:27 +00:00
withTrust(signature):
let
epoch = compute_epoch_at_slot(aggregate_and_proof.aggregate.data.slot)
domain = get_domain(
fork, DOMAIN_AGGREGATE_AND_PROOF, epoch, genesis_validators_root)
signing_root = compute_signing_root(aggregate_and_proof, domain)
blsVerify(pubKey, signing_root.data, signature)
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
func compute_attestation_root*(
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
fork: Fork, genesis_validators_root: Eth2Digest,
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
attestation_data: AttestationData
): Eth2Digest =
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
let
epoch = attestation_data.target.epoch
domain = get_domain(
fork, DOMAIN_BEACON_ATTESTER, epoch, genesis_validators_root)
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
compute_signing_root(attestation_data, domain)
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/altair/validator.md#prepare-sync-committee-message
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
func sync_committee_msg_signing_root*(
fork: Fork, epoch: Epoch,
genesis_validators_root: Eth2Digest,
block_root: Eth2Digest): Eth2Digest =
let domain = get_domain(fork, DOMAIN_SYNC_COMMITTEE, epoch, genesis_validators_root)
compute_signing_root(block_root, domain)
update 22 spec URLs to v1.1.5 (#3111)
2021-11-18 08:08:00 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/altair/validator.md#signature
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
func contribution_and_proof_signing_root*(
fork: Fork,
genesis_validators_root: Eth2Digest,
msg: ContributionAndProof): Eth2Digest =
let domain = get_domain(fork, DOMAIN_CONTRIBUTION_AND_PROOF,
msg.contribution.slot.epoch,
genesis_validators_root)
compute_signing_root(msg, domain)
a dozen spec URL updates to v1.1.5 (#3078)
2021-11-10 08:12:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/altair/validator.md#aggregation-selection
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
proc sync_committee_selection_proof_signing_root*(
fork: Fork,
genesis_validators_root: Eth2Digest,
slot: Slot,
subcommittee_index: uint64): Eth2Digest =
let
domain = get_domain(fork, DOMAIN_SYNC_COMMITTEE_SELECTION_PROOF,
slot.epoch, genesis_validators_root)
signing_data = SyncAggregatorSelectionData(
slot: slot,
subcommittee_index: subcommittee_index)
compute_signing_root(signing_data, domain)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/validator.md#aggregate-signature
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
func get_attestation_signature*(
fork: Fork, genesis_validators_root: Eth2Digest,
attestation_data: AttestationData,
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
privkey: ValidatorPrivKey): CookedSig =
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
blsSign(privKey, compute_attestation_root(fork, genesis_validators_root,
attestation_data).data)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
lazily initialize validator public keys, which were a third or more of sync CPU usage
2020-07-23 14:38:28 +00:00
proc verify_attestation_signature*(
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
fork: Fork, genesis_validators_root: Eth2Digest,
attestation_data: AttestationData,
singe validator key cache Instead of keeping a validator key list per EpochRef, this PR introduces a single shared validator key list in ChainDAG, and cleans up some other ChainDAG and key-related issues. The PR does not introduce the validator key list in the state transition - this is because we batch-check all signatures before entering the spec code, thus the spec code never hits the cache. A future refactor should _probably_ remove the threadvar altogether. There's a few other small fixes in here that make the flow easier to read: * fix `var ChainDAGRef` -> `ChainDAGRef` * fix `var QuarantineRef` -> `QuarantineRef` * consistent `dag` variable name * avoid using threadvar pubkey cache in most cases * better error messages in batch signature checking
2021-06-01 11:13:40 +00:00
pubkeys: auto,
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
signature: SomeSig): bool =
withTrust(signature):
let
epoch = attestation_data.target.epoch
domain = get_domain(
fork, DOMAIN_BEACON_ATTESTER, epoch, genesis_validators_root)
signing_root = compute_signing_root(attestation_data, domain)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
blsFastAggregateVerify(pubkeys, signing_root.data, signature)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/beacon-chain.md#deposits
Implement split preset/config support (#2710) * Implement split preset/config support This is the initial bulk refactor to introduce runtime config values in a number of places, somewhat replacing the existing mechanism of loading network metadata. It still needs more work, this is the initial refactor that introduces runtime configuration in some of the places that need it. The PR changes the way presets and constants work, to match the spec. In particular, a "preset" now refers to the compile-time configuration while a "cfg" or "RuntimeConfig" is the dynamic part. A single binary can support either mainnet or minimal, but not both. Support for other presets has been removed completely (can be readded, in case there's need). There's a number of outstanding tasks: * `SECONDS_PER_SLOT` still needs fixing * loading custom runtime configs needs redoing * checking constants against YAML file * yeerongpilly support `build/nimbus_beacon_node --network=yeerongpilly --discv5:no --log-level=DEBUG` * load fork epoch from config * fix fork digest sent in status * nicer error string for request failures * fix tools * one more * fixup * fixup * fixup * use "standard" network definition folder in local testnet Files are loaded from their standard locations, including genesis etc, to conform to the format used in the `eth2-networks` repo. * fix launch scripts, allow unknown config values * fix base config of rest test * cleanups * bundle mainnet config using common loader * fix spec links and names * only include supported preset in binary * drop yeerongpilly, add altair-devnet-0, support boot_enr.yaml
2021-07-12 13:01:38 +00:00
func get_deposit_signature*(preset: RuntimeConfig,
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
deposit: DepositData,
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
privkey: ValidatorPrivKey): CookedSig =
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
let
deposit_message = deposit.getDepositMessage()
# Fork-agnostic domain since deposits are valid across forks
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
domain = compute_domain(DOMAIN_DEPOSIT, preset.GENESIS_FORK_VERSION)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
signing_root = compute_signing_root(deposit_message, domain)
blsSign(privKey, signing_root.data)
Implement split preset/config support (#2710) * Implement split preset/config support This is the initial bulk refactor to introduce runtime config values in a number of places, somewhat replacing the existing mechanism of loading network metadata. It still needs more work, this is the initial refactor that introduces runtime configuration in some of the places that need it. The PR changes the way presets and constants work, to match the spec. In particular, a "preset" now refers to the compile-time configuration while a "cfg" or "RuntimeConfig" is the dynamic part. A single binary can support either mainnet or minimal, but not both. Support for other presets has been removed completely (can be readded, in case there's need). There's a number of outstanding tasks: * `SECONDS_PER_SLOT` still needs fixing * loading custom runtime configs needs redoing * checking constants against YAML file * yeerongpilly support `build/nimbus_beacon_node --network=yeerongpilly --discv5:no --log-level=DEBUG` * load fork epoch from config * fix fork digest sent in status * nicer error string for request failures * fix tools * one more * fixup * fixup * fixup * use "standard" network definition folder in local testnet Files are loaded from their standard locations, including genesis etc, to conform to the format used in the `eth2-networks` repo. * fix launch scripts, allow unknown config values * fix base config of rest test * cleanups * bundle mainnet config using common loader * fix spec links and names * only include supported preset in binary * drop yeerongpilly, add altair-devnet-0, support boot_enr.yaml
2021-07-12 13:01:38 +00:00
proc verify_deposit_signature*(preset: RuntimeConfig,
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
deposit: DepositData): bool =
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
let
deposit_message = deposit.getDepositMessage()
# Fork-agnostic domain since deposits are valid across forks
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
domain = compute_domain(DOMAIN_DEPOSIT, preset.GENESIS_FORK_VERSION)
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
signing_root = compute_signing_root(deposit_message, domain)
blsVerify(deposit.pubkey, signing_root.data, deposit.signature)
Implement the 'deposits exit' command; Remove 'deposits create'
2020-11-27 19:48:33 +00:00
func get_voluntary_exit_signature*(
fork: Fork,
genesis_validators_root: Eth2Digest,
voluntary_exit: VoluntaryExit,
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
privkey: ValidatorPrivKey): CookedSig =
Implement the 'deposits exit' command; Remove 'deposits create'
2020-11-27 19:48:33 +00:00
let
domain = get_domain(
fork, DOMAIN_VOLUNTARY_EXIT, voluntary_exit.epoch, genesis_validators_root)
signing_root = compute_signing_root(voluntary_exit, domain)
blsSign(privKey, signing_root.data)
lazily initialize validator public keys, which were a third or more of sync CPU usage
2020-07-23 14:38:28 +00:00
proc verify_voluntary_exit_signature*(
Implement the 'deposits exit' command; Remove 'deposits create'
2020-11-27 19:48:33 +00:00
fork: Fork,
genesis_validators_root: Eth2Digest,
collect signature production and verificaiton in one place (#1179) * collect signature production and verificaiton in one place Signatures are made over data and domain - here we collect all such activities in one place. Also: * security: fix cast-before-range-check * log block/attestation verification consistently * run block verification based on `getProposer` in its own history * clean up some unused stuff * import * missing raises
2020-06-16 05:45:04 +00:00
voluntary_exit: VoluntaryExit,
Implement the 'deposits exit' command; Remove 'deposits create'
2020-11-27 19:48:33 +00:00
pubkey: ValidatorPubKey,
signature: SomeSig): bool =
Trusted blocks (#1227) * cleanups * fix ncli state root check flag * add block dump to ncli_db * limit ncli_db benchmark length * tone down finalization logs * introduce trusted blocks We only store blocks whose signature we've verified in the database - as such, there's no need to check it again, and most importantly, no need to deserialize the signature when loading from database. 50x startup time improvement, 200x block load time improvement. * fix rewinding when deposits have invalid signature * speed up ancestor iteration by avoiding copy * avoid deserializing signatures for trusted data * load blocks lazily when rewinding (less memory used) * chronicles workarounds * document trustedbeaconblock
2020-06-25 10:23:10 +00:00
withTrust(signature):
let
domain = get_domain(
fork, DOMAIN_VOLUNTARY_EXIT, voluntary_exit.epoch, genesis_validators_root)
signing_root = compute_signing_root(voluntary_exit, domain)
blsVerify(pubkey, signing_root.data, signature)
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
proc verify_sync_committee_message_signature*(
epoch: Epoch,
beacon_block_root: Eth2Digest,
fork: Fork,
genesis_validators_root: Eth2Digest,
pubkey: CookedPubKey,
signature: CookedSig): bool =
let
domain = get_domain(
fork, DOMAIN_SYNC_COMMITTEE, epoch, genesis_validators_root)
signing_root = compute_signing_root(beacon_block_root, domain)
blsVerify(pubkey, signing_root.data, signature)
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/altair/validator.md#aggregation-selection
sync committee/aggregate signature signing and verification (#2784) * sync committee/aggregate signature signing and verification * add message signature tests
2021-08-17 08:07:17 +00:00
proc is_sync_committee_aggregator*(signature: ValidatorSig): bool =
let
signatureDigest = eth2digest(signature.blob)
modulo = max(1'u64, (SYNC_COMMITTEE_SIZE div SYNC_COMMITTEE_SUBNET_COUNT) div TARGET_AGGREGATORS_PER_SYNC_SUBCOMMITTEE)
bytes_to_uint64(signatureDigest.data.toOpenArray(0, 7)) mod modulo == 0
proc verify_signed_contribution_and_proof_signature*(
msg: SignedContributionAndProof,
fork: Fork,
genesis_validators_root: Eth2Digest,
pubkey: ValidatorPubKey | CookedPubKey): bool =
let
domain = get_domain(
fork, DOMAIN_CONTRIBUTION_AND_PROOF, msg.message.contribution.slot.epoch, genesis_validators_root)
signing_root = compute_signing_root(msg.message, domain)
blsVerify(pubkey, signing_root.data, msg.signature)
proc verify_selection_proof_signature*(
msg: ContributionAndProof,
fork: Fork,
genesis_validators_root: Eth2Digest,
pubkey: ValidatorPubKey | CookedPubKey): bool =
let
slot = msg.contribution.slot
domain = get_domain(
fork, DOMAIN_SYNC_COMMITTEE_SELECTION_PROOF, slot.epoch, genesis_validators_root)
signing_data = SyncAggregatorSelectionData(
slot: slot,
subcommittee_index: msg.contribution.subcommittee_index)
signing_root = compute_signing_root(signing_data, domain)
blsVerify(pubkey, signing_root.data, msg.selection_proof)