nim-eth2-official-tests

This repo reformats the official Ethereum 2 tests to a format suitable for Nimbus/nim-beacon-chain.

Cloning the repo

Due to upstream usage of Git LFS to store the large test vectors, there is an extra step after cloning the repo:

git clone https://github.com/status-im/nim-eth2-official-tests
cd nim-eth2-official-tests
git lfs install
git submodule update --init --recursive

Usage in nim-beacon-chain

This repository is meant to be submoduled in nim-beacon-chain/tests/official/ with the fixtures name.

From the nim-beacon-chain project folder:

git submodule add https://github.com/status-im/nim-eth2-official-tests ./tests/official/fixtures

This repository allows use to workaround the current following limitations:

• no YAML support in nim-serialization library which allows well-tested serialization and deserialization into and from Ethereum types.
• NimYAML uses int by default for numerals and cannot deserialize 18446744073709551615 (2^64-1), the FAR_FUTURE_SLOT constant.
• Eth2.0 tests currently uses an invalid BLS signature https://github.com/ethereum/eth2.0-tests/issues/27
• All those workarounds requires an intermediate reformatted JSON file, but the tests are huge (100k+ lines) and will cause review issues in the main repo.

Only state tests are worked on at the moment.

State tests conversion.

Note on serialization hacks:

FAR_FUTURE_SLOT (18446744073709551615)

The FAR_FUTURE_SLOT (18446744073709551615) has been rewritten as a string in the YAML file as it's 2^64-1 and Nim by default try to parse it into a int64 (which can represents up to 2^63-1).

The YAML file is then converted to JSON for easy input to the json serialization/deserialization with beacon chain type support.

"18446744073709551615" is then replaced again by uint64 18446744073709551615.

Compressed signature

In latest_block_header field, the signatures and randao_reveals are "0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000" but that is not a valid compressed BLS signature, the zero signature should be: "0xc00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"

Commands

Example commands to generate the tests and apply the proper workarounds on UNIX systems. To be used from this repo root directory. Please double-check the commands.

# Out of place replace
sed 's/18446744073709551615/"18446744073709551615"/g' eth2.0-tests/state/sanity-check_default-config_100-vals.yaml > json_tests/state/sanity-check_default-config_100-vals.yaml
# Create Nim build directory if it doesn't exist
mkdir -p build
# Compile conversion utility
nim c -r -o:build/yamlToJson fixtures_utils.nim # Will convert "json_tests/state/sanity-check_default-config_100-vals-first_test.yaml" by default
# In-place replaces
sed -i 's/"18446744073709551615"/18446744073709551615/g' json_tests/state/sanity-check_default-config_100-vals.json
sed -i 's/"randao_reveal":"0x00000000/"randao_reveal":"0xc0000000/g' json_tests/state/sanity-check_default-config_100-vals.json
sed -i 's/"signature":"0x00000000/"signature":"0xc0000000/g' json_tests/state/sanity-check_default-config_100-vals.json
sed -i 's/"proof_of_possession":"0x00000000/"proof_of_possession":"0xc0000000/g' json_tests/state/sanity-check_default-config_100-vals.json
sed -i 's/"aggregate_signature":"0x00000000/"aggregate_signature":"0xc0000000/g' json_tests/state/sanity-check_default-config_100-vals.json
# Interactive delete
rm -i json_tests/state/sanity-check_default-config_100-vals.yaml