* reorganize ssz dependencies This PR continues the work in https://github.com/status-im/nimbus-eth2/pull/2646, https://github.com/status-im/nimbus-eth2/pull/2779 as well as past issues with serialization and type, to disentangle SSZ from eth2 and at the same time simplify imports and exports with a structured approach. The principal idea here is that when a library wants to introduce SSZ support, they do so via 3 files: * `ssz_codecs` which imports and reexports `codecs` - this covers the basic byte conversions and ensures no overloads get lost * `xxx_merkleization` imports and exports `merkleization` to specialize and get access to `hash_tree_root` and friends * `xxx_ssz_serialization` imports and exports `ssz_serialization` to specialize ssz for a specific library Those that need to interact with SSZ always import the `xxx_` versions of the modules and never `ssz` itself so as to keep imports simple and safe. This is similar to how the REST / JSON-RPC serializers are structured in that someone wanting to serialize spec types to REST-JSON will import `eth2_rest_serialization` and nothing else. * split up ssz into a core library that is independendent of eth2 types * rename `bytes_reader` to `codec` to highlight that it contains coding and decoding of bytes and native ssz types * remove tricky List init overload that causes compile issues * get rid of top-level ssz import * reenable merkleization tests * move some "standard" json serializers to spec * remove `ValidatorIndex` serialization for now * remove test_ssz_merkleization * add tests for over/underlong byte sequences * fix broken seq[byte] test - seq[byte] is not an SSZ type There are a few things this PR doesn't solve: * like #2646 this PR is weak on how to handle root and other dontSerialize fields that "sometimes" should be computed - the same problem appears in REST / JSON-RPC etc * Fix a build problem on macOS * Another way to fix the macOS builds Co-authored-by: Zahary Karadjov <zahary@gmail.com>
21 KiB
AllTests-mainnet
Attestation pool processing [Preset: mainnet]
+ Attestations may arrive in any order [Preset: mainnet] OK
+ Attestations may overlap, bigger first [Preset: mainnet] OK
+ Attestations may overlap, smaller first [Preset: mainnet] OK
+ Attestations should be combined [Preset: mainnet] OK
+ Can add and retrieve simple attestations [Preset: mainnet] OK
+ Everyone voting for something different [Preset: mainnet] OK
+ Fork choice returns block with attestation OK
+ Fork choice returns latest block with no attestations OK
+ Trying to add a block twice tags the second as an error OK
+ Trying to add a duplicate block from an old pruned epoch is tagged as an error OK
+ Working with aggregates [Preset: mainnet] OK
OK: 11/11 Fail: 0/11 Skip: 0/11
Beacon chain DB [Preset: mainnet]
+ empty database [Preset: mainnet] OK
+ find ancestors [Preset: mainnet] OK
+ sanity check Altair and cross-fork getState rollback [Preset: mainnet] OK
+ sanity check Altair blocks [Preset: mainnet] OK
+ sanity check Altair states [Preset: mainnet] OK
+ sanity check Altair states, reusing buffers [Preset: mainnet] OK
+ sanity check genesis roundtrip [Preset: mainnet] OK
+ sanity check phase 0 blocks [Preset: mainnet] OK
+ sanity check phase 0 getState rollback [Preset: mainnet] OK
+ sanity check phase 0 states [Preset: mainnet] OK
+ sanity check phase 0 states, reusing buffers [Preset: mainnet] OK
+ sanity check state diff roundtrip [Preset: mainnet] OK
OK: 12/12 Fail: 0/12 Skip: 0/12
Beacon state [Preset: mainnet]
+ Smoke test initialize_beacon_state_from_eth1 [Preset: mainnet] OK
OK: 1/1 Fail: 0/1 Skip: 0/1
Bit fields
+ isZeros OK
+ iterating words OK
+ overlaps OK
+ roundtrips BitArray OK
+ roundtrips BitSeq OK
OK: 5/5 Fail: 0/5 Skip: 0/5
Block pool processing [Preset: mainnet]
+ Adding the same block twice returns a Duplicate error [Preset: mainnet] OK
+ Reverse order block add & get [Preset: mainnet] OK
+ Simple block add&get [Preset: mainnet] OK
+ getRef returns nil for missing blocks OK
+ loading tail block works [Preset: mainnet] OK
+ updateHead updates head and headState [Preset: mainnet] OK
+ updateStateData sanity [Preset: mainnet] OK
OK: 7/7 Fail: 0/7 Skip: 0/7
BlockRef and helpers [Preset: mainnet]
+ epochAncestor sanity [Preset: mainnet] OK
+ get_ancestor sanity [Preset: mainnet] OK
+ isAncestorOf sanity [Preset: mainnet] OK
OK: 3/3 Fail: 0/3 Skip: 0/3
BlockSlot and helpers [Preset: mainnet]
+ atSlot sanity [Preset: mainnet] OK
+ parent sanity [Preset: mainnet] OK
OK: 2/2 Fail: 0/2 Skip: 0/2
Eth1 monitor
+ Rewrite HTTPS Infura URLs OK
OK: 1/1 Fail: 0/1 Skip: 0/1
Eth2 specific discovery tests
+ Invalid attnets field OK
+ Subnet query OK
+ Subnet query after ENR update OK
OK: 3/3 Fail: 0/3 Skip: 0/3
Exit pool testing suite
+ addExitMessage/getAttesterSlashingMessage OK
+ addExitMessage/getProposerSlashingMessage OK
+ addExitMessage/getVoluntaryExitMessage OK
OK: 3/3 Fail: 0/3 Skip: 0/3
Fork Choice + Finality [Preset: mainnet]
+ fork_choice - testing finality #01 OK
+ fork_choice - testing finality #02 OK
+ fork_choice - testing no votes OK
+ fork_choice - testing with votes OK
OK: 4/4 Fail: 0/4 Skip: 0/4
Gossip validation [Preset: mainnet]
+ Validation sanity OK
OK: 1/1 Fail: 0/1 Skip: 0/1
Honest validator
+ General pubsub topics OK
+ Mainnet attestation topics OK
+ is_aggregator OK
OK: 3/3 Fail: 0/3 Skip: 0/3
Interop
+ Interop genesis OK
+ Interop signatures OK
+ Mocked start private key OK
OK: 3/3 Fail: 0/3 Skip: 0/3
Message signatures
+ Aggregate and proof signatures OK
+ Attestation signatures OK
+ Deposit signatures OK
+ Slot signatures OK
+ Sync committee message signatures OK
+ Sync committee selection proof signatures OK
+ Sync committee signed contribution and proof signatures OK
+ Voluntary exit signatures OK
OK: 8/8 Fail: 0/8 Skip: 0/8
Old database versions [Preset: mainnet]
+ pre-1.1.0 OK
OK: 1/1 Fail: 0/1 Skip: 0/1
PeerPool testing suite
+ Access peers by key test OK
+ Acquire from empty pool OK
+ Acquire/Sorting and consistency test OK
+ Delete peer on release text OK
+ Iterators test OK
+ Peer lifetime test OK
+ Safe/Clear test OK
+ Score check test OK
+ Space tests OK
+ addPeer() test OK
+ addPeerNoWait() test OK
+ deletePeer() test OK
OK: 12/12 Fail: 0/12 Skip: 0/12
SSZ dynamic navigator
+ navigating fields OK
OK: 1/1 Fail: 0/1 Skip: 0/1
SSZ generic roundtrip tests
+ case objects OK
+ lists OK
+ objects OK
+ sets OK
+ simple values OK
+ tables OK
+ tuple OK
OK: 7/7 Fail: 0/7 Skip: 0/7
SSZ navigator
+ basictype OK
+ lists with max size OK
+ simple object fields OK
OK: 3/3 Fail: 0/3 Skip: 0/3
Slashing Protection DB - Interchange [Preset: mainnet]
+ Smoke test - Complete format - Invalid database is refused [Preset: mainnet] OK
+ Smoke test - Complete format [Preset: mainnet] OK
OK: 2/2 Fail: 0/2 Skip: 0/2
Slashing Protection DB - v1 and v2 migration [Preset: mainnet]
+ Minimal format migration [Preset: mainnet] OK
OK: 1/1 Fail: 0/1 Skip: 0/1
Slashing Protection DB [Preset: mainnet]
+ Attestation ordering #1698 OK
+ Don't prune the very last attestation(s) even by mistake OK
+ Don't prune the very last block even by mistake OK
+ Empty database [Preset: mainnet] OK
+ Pruning attestations works OK
+ Pruning blocks works OK
+ SP for block proposal - backtracking append OK
+ SP for block proposal - linear append OK
+ SP for same epoch attestation target - linear append OK
+ SP for surrounded attestations OK
+ SP for surrounding attestations OK
+ Test valid attestation #1699 OK
OK: 12/12 Fail: 0/12 Skip: 0/12
Spec datatypes
+ Graffiti bytes OK
OK: 1/1 Fail: 0/1 Skip: 0/1
Spec helpers
+ integer_squareroot OK
OK: 1/1 Fail: 0/1 Skip: 0/1
Specific field types
+ root update OK
+ roundtrip OK
OK: 2/2 Fail: 0/2 Skip: 0/2
SyncManager test suite
+ [SyncQueue] Async pending and resetWait() test OK
+ [SyncQueue] Async unordered push start from zero OK
+ [SyncQueue] Async unordered push with not full start from non-zero OK
+ [SyncQueue] Full and incomplete success/fail start from non-zero OK
+ [SyncQueue] Full and incomplete success/fail start from zero OK
+ [SyncQueue] One smart and one stupid + debt split + empty OK
+ [SyncQueue] Smart and stupid success/fail OK
+ [SyncQueue] Start and finish slots equal OK
+ [SyncQueue] Two full requests success/fail OK
+ [SyncQueue] checkResponse() test OK
+ [SyncQueue] contains() test OK
+ [SyncQueue] getLastNonEmptySlot() test OK
+ [SyncQueue] getRewindPoint() test OK
+ [SyncQueue] hasEndGap() test OK
OK: 14/14 Fail: 0/14 Skip: 0/14
Zero signature sanity checks
+ SSZ serialization roundtrip of SignedBeaconBlockHeader OK
+ Zero signatures cannot be loaded into a BLS signature object OK
+ default initialization of signatures OK
OK: 3/3 Fail: 0/3 Skip: 0/3
[Unit - Spec - Block processing] Attestations [Preset: mainnet]
+ Valid attestation OK
+ Valid attestation from previous epoch OK
OK: 2/2 Fail: 0/2 Skip: 0/2
[Unit - Spec - Block processing] Deposits [Preset: mainnet]
+ Deposit at MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Deposit over MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Deposit under MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Invalid deposit at MAX_EFFECTIVE_BALANCE balance (32 ETH) OK
+ Validator top-up OK
OK: 5/5 Fail: 0/5 Skip: 0/5
[Unit - Spec - Epoch processing] Justification and Finalization [Preset: mainnet]
+ Rule I - 234 finalization with enough support OK
+ Rule I - 234 finalization without support OK
+ Rule II - 23 finalization with enough support OK
+ Rule II - 23 finalization without support OK
+ Rule III - 123 finalization with enough support OK
+ Rule III - 123 finalization without support OK
+ Rule IV - 12 finalization with enough support OK
+ Rule IV - 12 finalization without support OK
OK: 8/8 Fail: 0/8 Skip: 0/8
chain DAG finalization tests [Preset: mainnet]
+ init with gaps [Preset: mainnet] OK
+ orphaned epoch block [Preset: mainnet] OK
+ prune heads on finalization [Preset: mainnet] OK
OK: 3/3 Fail: 0/3 Skip: 0/3
hash
+ HashArray OK
+ HashList fixed OK
+ HashList variable OK
OK: 3/3 Fail: 0/3 Skip: 0/3
state diff tests [Preset: mainnet]
+ random slot differences [Preset: mainnet] OK
OK: 1/1 Fail: 0/1 Skip: 0/1
underlong values
Overlong SSZ.decode: BitArray[32] Skip
Overlong SSZ.decode: BitList[32] Skip
Overlong SSZ.decode: HashArray[32, system.uint8] Skip
+ Overlong SSZ.decode: HashList[system.uint64, 32] OK
+ Overlong SSZ.decode: List[system.uint64, 32] OK
Overlong SSZ.decode: Simple Skip
Overlong SSZ.decode: array[0..31, byte] Skip
Overlong SSZ.decode: bool Skip
Overlong SSZ.decode: limb_t Skip
Overlong SSZ.decode: uint16 Skip
Overlong SSZ.decode: uint32 Skip
Overlong SSZ.decode: uint8 Skip
+ Overlong readSszBytes: BitArray[32] OK
Overlong readSszBytes: BitList[32] Skip
+ Overlong readSszBytes: HashArray[32, system.uint8] OK
+ Overlong readSszBytes: HashList[system.uint64, 32] OK
+ Overlong readSszBytes: List[system.uint64, 32] OK
Overlong readSszBytes: Simple Skip
+ Overlong readSszBytes: array[0..31, byte] OK
+ Overlong readSszBytes: bool OK
+ Overlong readSszBytes: limb_t OK
+ Overlong readSszBytes: uint16 OK
+ Overlong readSszBytes: uint32 OK
+ Overlong readSszBytes: uint8 OK
+ Underlong SSZ.decode: BitArray[32] OK
+ Underlong SSZ.decode: BitList[32] OK
+ Underlong SSZ.decode: HashArray[32, system.uint8] OK
+ Underlong SSZ.decode: HashList[system.uint64, 32] OK
+ Underlong SSZ.decode: List[system.uint64, 32] OK
+ Underlong SSZ.decode: Simple OK
+ Underlong SSZ.decode: array[0..31, byte] OK
+ Underlong SSZ.decode: bool OK
+ Underlong SSZ.decode: limb_t OK
+ Underlong SSZ.decode: uint16 OK
+ Underlong SSZ.decode: uint32 OK
+ Underlong SSZ.decode: uint8 OK
+ Underlong readSszBytes: BitArray[32] OK
+ Underlong readSszBytes: BitList[32] OK
+ Underlong readSszBytes: HashArray[32, system.uint8] OK
+ Underlong readSszBytes: HashList[system.uint64, 32] OK
+ Underlong readSszBytes: List[system.uint64, 32] OK
+ Underlong readSszBytes: Simple OK
+ Underlong readSszBytes: array[0..31, byte] OK
+ Underlong readSszBytes: bool OK
+ Underlong readSszBytes: limb_t OK
+ Underlong readSszBytes: uint16 OK
+ Underlong readSszBytes: uint32 OK
+ Underlong readSszBytes: uint8 OK
OK: 36/48 Fail: 0/48 Skip: 12/48
---TOTAL--- OK: 185/197 Fail: 0/197 Skip: 12/197