nimbus-eth2/beacon_chain/el/merkle_minimal.nim

44 lines
1.7 KiB
Nim
Raw Normal View History

v0.8.1 tests refactor (#326) * Introduce new mocking proc to replace: - makeFakeValidatorPrivKey - hackPrivKey - getNextBeaconProposerIndex - addBlock - makeBlock * Add comments on datastructure unsynced with the spec * Add merkle tree constructor and initial mocking for deposits (missing merkle proofs) * [Mock] Implement sparse merkle tree and merkle proof builder * [Mocking] Genesis deposits * Add compact_committees_roots init + mock genesis state * [Tests] Add first deposit test using the new mocking procedures * [Tests -deposits] add at and over 32 ETH deposit tests * [Tests - deposits] Add test for validator top-up * [Tests -deposits] Mention the TODO to test for invalid conditions * [Tests] Add stub to test "is_valid_genesis_state" * [Merkle proofs] Implement round-trip checks * Deactivate roundtrips test * SSZ - use EF convention for hash_tree_root / hashTreeRoot * [Tests - Attestation] Attestation mocking + initial test * Add mocking + 3 new tests for valid attestations + mention future invalid attestation tests * Add crosslinks test (1 failing to attestations in block being duplicated in state transition) * Single attestation crosslink test - workaround https://github.com/status-im/nim-beacon-chain/issues/361 * Add test for failed crosslink penalty * Rebase fixes + add refactored tests to test suite * justif-finalization helpers first batch * Add 234 finalization tests * Fix justif test, Rule I 234 finalization does not happen with sufficient support. (Also unittest check template does not fail properly in some cases) * Add tests for all finalization rules * Properly delete nim-byteutils following https://github.com/status-im/nim-beacon-chain/commit/c91727e7e5f9c7a95a634055e94e9d3bcb6cc41a#diff-7c3613dba5171cb6027c67835dd3b9d4 * use digest helper for deposit root
2019-08-28 12:07:00 +00:00
# beacon_chain
# Copyright (c) 2018-2023 Status Research & Development GmbH
v0.8.1 tests refactor (#326) * Introduce new mocking proc to replace: - makeFakeValidatorPrivKey - hackPrivKey - getNextBeaconProposerIndex - addBlock - makeBlock * Add comments on datastructure unsynced with the spec * Add merkle tree constructor and initial mocking for deposits (missing merkle proofs) * [Mock] Implement sparse merkle tree and merkle proof builder * [Mocking] Genesis deposits * Add compact_committees_roots init + mock genesis state * [Tests] Add first deposit test using the new mocking procedures * [Tests -deposits] add at and over 32 ETH deposit tests * [Tests - deposits] Add test for validator top-up * [Tests -deposits] Mention the TODO to test for invalid conditions * [Tests] Add stub to test "is_valid_genesis_state" * [Merkle proofs] Implement round-trip checks * Deactivate roundtrips test * SSZ - use EF convention for hash_tree_root / hashTreeRoot * [Tests - Attestation] Attestation mocking + initial test * Add mocking + 3 new tests for valid attestations + mention future invalid attestation tests * Add crosslinks test (1 failing to attestations in block being duplicated in state transition) * Single attestation crosslink test - workaround https://github.com/status-im/nim-beacon-chain/issues/361 * Add test for failed crosslink penalty * Rebase fixes + add refactored tests to test suite * justif-finalization helpers first batch * Add 234 finalization tests * Fix justif test, Rule I 234 finalization does not happen with sufficient support. (Also unittest check template does not fail properly in some cases) * Add tests for all finalization rules * Properly delete nim-byteutils following https://github.com/status-im/nim-beacon-chain/commit/c91727e7e5f9c7a95a634055e94e9d3bcb6cc41a#diff-7c3613dba5171cb6027c67835dd3b9d4 * use digest helper for deposit root
2019-08-28 12:07:00 +00:00
# Licensed and distributed under either of
# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
v0.8.1 tests refactor (#326) * Introduce new mocking proc to replace: - makeFakeValidatorPrivKey - hackPrivKey - getNextBeaconProposerIndex - addBlock - makeBlock * Add comments on datastructure unsynced with the spec * Add merkle tree constructor and initial mocking for deposits (missing merkle proofs) * [Mock] Implement sparse merkle tree and merkle proof builder * [Mocking] Genesis deposits * Add compact_committees_roots init + mock genesis state * [Tests] Add first deposit test using the new mocking procedures * [Tests -deposits] add at and over 32 ETH deposit tests * [Tests - deposits] Add test for validator top-up * [Tests -deposits] Mention the TODO to test for invalid conditions * [Tests] Add stub to test "is_valid_genesis_state" * [Merkle proofs] Implement round-trip checks * Deactivate roundtrips test * SSZ - use EF convention for hash_tree_root / hashTreeRoot * [Tests - Attestation] Attestation mocking + initial test * Add mocking + 3 new tests for valid attestations + mention future invalid attestation tests * Add crosslinks test (1 failing to attestations in block being duplicated in state transition) * Single attestation crosslink test - workaround https://github.com/status-im/nim-beacon-chain/issues/361 * Add test for failed crosslink penalty * Rebase fixes + add refactored tests to test suite * justif-finalization helpers first batch * Add 234 finalization tests * Fix justif test, Rule I 234 finalization does not happen with sufficient support. (Also unittest check template does not fail properly in some cases) * Add tests for all finalization rules * Properly delete nim-byteutils following https://github.com/status-im/nim-beacon-chain/commit/c91727e7e5f9c7a95a634055e94e9d3bcb6cc41a#diff-7c3613dba5171cb6027c67835dd3b9d4 * use digest helper for deposit root
2019-08-28 12:07:00 +00:00
# at your option. This file may not be copied, modified, or distributed except according to those terms.
{.push raises: [].}
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-alpha.3/tests/core/pyspec/eth2spec/utils/merkle_minimal.py
v0.8.1 tests refactor (#326) * Introduce new mocking proc to replace: - makeFakeValidatorPrivKey - hackPrivKey - getNextBeaconProposerIndex - addBlock - makeBlock * Add comments on datastructure unsynced with the spec * Add merkle tree constructor and initial mocking for deposits (missing merkle proofs) * [Mock] Implement sparse merkle tree and merkle proof builder * [Mocking] Genesis deposits * Add compact_committees_roots init + mock genesis state * [Tests] Add first deposit test using the new mocking procedures * [Tests -deposits] add at and over 32 ETH deposit tests * [Tests - deposits] Add test for validator top-up * [Tests -deposits] Mention the TODO to test for invalid conditions * [Tests] Add stub to test "is_valid_genesis_state" * [Merkle proofs] Implement round-trip checks * Deactivate roundtrips test * SSZ - use EF convention for hash_tree_root / hashTreeRoot * [Tests - Attestation] Attestation mocking + initial test * Add mocking + 3 new tests for valid attestations + mention future invalid attestation tests * Add crosslinks test (1 failing to attestations in block being duplicated in state transition) * Single attestation crosslink test - workaround https://github.com/status-im/nim-beacon-chain/issues/361 * Add test for failed crosslink penalty * Rebase fixes + add refactored tests to test suite * justif-finalization helpers first batch * Add 234 finalization tests * Fix justif test, Rule I 234 finalization does not happen with sufficient support. (Also unittest check template does not fail properly in some cases) * Add tests for all finalization rules * Properly delete nim-byteutils following https://github.com/status-im/nim-beacon-chain/commit/c91727e7e5f9c7a95a634055e94e9d3bcb6cc41a#diff-7c3613dba5171cb6027c67835dd3b9d4 * use digest helper for deposit root
2019-08-28 12:07:00 +00:00
# Merkle tree helpers
# ---------------------------------------------------------------
import
sequtils,
stew/endians2,
v0.8.1 tests refactor (#326) * Introduce new mocking proc to replace: - makeFakeValidatorPrivKey - hackPrivKey - getNextBeaconProposerIndex - addBlock - makeBlock * Add comments on datastructure unsynced with the spec * Add merkle tree constructor and initial mocking for deposits (missing merkle proofs) * [Mock] Implement sparse merkle tree and merkle proof builder * [Mocking] Genesis deposits * Add compact_committees_roots init + mock genesis state * [Tests] Add first deposit test using the new mocking procedures * [Tests -deposits] add at and over 32 ETH deposit tests * [Tests - deposits] Add test for validator top-up * [Tests -deposits] Mention the TODO to test for invalid conditions * [Tests] Add stub to test "is_valid_genesis_state" * [Merkle proofs] Implement round-trip checks * Deactivate roundtrips test * SSZ - use EF convention for hash_tree_root / hashTreeRoot * [Tests - Attestation] Attestation mocking + initial test * Add mocking + 3 new tests for valid attestations + mention future invalid attestation tests * Add crosslinks test (1 failing to attestations in block being duplicated in state transition) * Single attestation crosslink test - workaround https://github.com/status-im/nim-beacon-chain/issues/361 * Add test for failed crosslink penalty * Rebase fixes + add refactored tests to test suite * justif-finalization helpers first batch * Add 234 finalization tests * Fix justif test, Rule I 234 finalization does not happen with sufficient support. (Also unittest check template does not fail properly in some cases) * Add tests for all finalization rules * Properly delete nim-byteutils following https://github.com/status-im/nim-beacon-chain/commit/c91727e7e5f9c7a95a634055e94e9d3bcb6cc41a#diff-7c3613dba5171cb6027c67835dd3b9d4 * use digest helper for deposit root
2019-08-28 12:07:00 +00:00
# Specs
disentangle eth2 types from the ssz library (#2785) * reorganize ssz dependencies This PR continues the work in https://github.com/status-im/nimbus-eth2/pull/2646, https://github.com/status-im/nimbus-eth2/pull/2779 as well as past issues with serialization and type, to disentangle SSZ from eth2 and at the same time simplify imports and exports with a structured approach. The principal idea here is that when a library wants to introduce SSZ support, they do so via 3 files: * `ssz_codecs` which imports and reexports `codecs` - this covers the basic byte conversions and ensures no overloads get lost * `xxx_merkleization` imports and exports `merkleization` to specialize and get access to `hash_tree_root` and friends * `xxx_ssz_serialization` imports and exports `ssz_serialization` to specialize ssz for a specific library Those that need to interact with SSZ always import the `xxx_` versions of the modules and never `ssz` itself so as to keep imports simple and safe. This is similar to how the REST / JSON-RPC serializers are structured in that someone wanting to serialize spec types to REST-JSON will import `eth2_rest_serialization` and nothing else. * split up ssz into a core library that is independendent of eth2 types * rename `bytes_reader` to `codec` to highlight that it contains coding and decoding of bytes and native ssz types * remove tricky List init overload that causes compile issues * get rid of top-level ssz import * reenable merkleization tests * move some "standard" json serializers to spec * remove `ValidatorIndex` serialization for now * remove test_ssz_merkleization * add tests for over/underlong byte sequences * fix broken seq[byte] test - seq[byte] is not an SSZ type There are a few things this PR doesn't solve: * like #2646 this PR is weak on how to handle root and other dontSerialize fields that "sometimes" should be computed - the same problem appears in REST / JSON-RPC etc * Fix a build problem on macOS * Another way to fix the macOS builds Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 18:57:58 +00:00
../spec/[eth2_merkleization, digest],
../spec/datatypes/base
v0.8.1 tests refactor (#326) * Introduce new mocking proc to replace: - makeFakeValidatorPrivKey - hackPrivKey - getNextBeaconProposerIndex - addBlock - makeBlock * Add comments on datastructure unsynced with the spec * Add merkle tree constructor and initial mocking for deposits (missing merkle proofs) * [Mock] Implement sparse merkle tree and merkle proof builder * [Mocking] Genesis deposits * Add compact_committees_roots init + mock genesis state * [Tests] Add first deposit test using the new mocking procedures * [Tests -deposits] add at and over 32 ETH deposit tests * [Tests - deposits] Add test for validator top-up * [Tests -deposits] Mention the TODO to test for invalid conditions * [Tests] Add stub to test "is_valid_genesis_state" * [Merkle proofs] Implement round-trip checks * Deactivate roundtrips test * SSZ - use EF convention for hash_tree_root / hashTreeRoot * [Tests - Attestation] Attestation mocking + initial test * Add mocking + 3 new tests for valid attestations + mention future invalid attestation tests * Add crosslinks test (1 failing to attestations in block being duplicated in state transition) * Single attestation crosslink test - workaround https://github.com/status-im/nim-beacon-chain/issues/361 * Add test for failed crosslink penalty * Rebase fixes + add refactored tests to test suite * justif-finalization helpers first batch * Add 234 finalization tests * Fix justif test, Rule I 234 finalization does not happen with sufficient support. (Also unittest check template does not fail properly in some cases) * Add tests for all finalization rules * Properly delete nim-byteutils following https://github.com/status-im/nim-beacon-chain/commit/c91727e7e5f9c7a95a634055e94e9d3bcb6cc41a#diff-7c3613dba5171cb6027c67835dd3b9d4 * use digest helper for deposit root
2019-08-28 12:07:00 +00:00
template getProof*(
proofs: seq[Eth2Digest], idxParam: int): openArray[Eth2Digest] =
let
idx = idxParam
## TODO: It's surprising that we have to do +1 here.
## It seems that `depositContractLimit` is set too high.
startIdx = idx * (DEPOSIT_CONTRACT_TREE_DEPTH + 1)
endIdx = startIdx + DEPOSIT_CONTRACT_TREE_DEPTH - 1
proofs.toOpenArray(startIdx, endIdx)
func attachMerkleProofs*(deposits: var openArray[Deposit]): Eth2Digest =
let depositsRoots = mapIt(deposits, hash_tree_root(it.data))
var merkleizer = createMerkleizer(DEPOSIT_CONTRACT_LIMIT)
let proofs = merkleizer.addChunksAndGenMerkleProofs(depositsRoots)
for i in 0 ..< depositsRoots.len:
deposits[i].proof[0 ..< DEPOSIT_CONTRACT_TREE_DEPTH] = getProof(proofs, i)
deposits[i].proof[DEPOSIT_CONTRACT_TREE_DEPTH] = default(Eth2Digest)
deposits[i].proof[DEPOSIT_CONTRACT_TREE_DEPTH].data[0..7] =
toBytesLE deposits.lenu64
mixInLength(merkleizer.getFinalHash(), deposits.len)