Move test-generators to specs repo
Co-authored-by: Chih Cheng Liang <chihchengliang@gmail.com> Co-authored-by: Danny Ryan <dannyjryan@gmail.com> Co-authored-by: Dmitrii Shmatko <leodex23@gmail.com> Co-authored-by: Jannik Luhn <jannik@brainbot.com> Co-authored-by: Paul Hauner <paul@paulhauner.com> Co-authored-by: protolambda <proto@protolambda.com>
This commit is contained in:
protolambda
parent
53e528e56c
commit
64f012b276
|
|
@ -0,0 +1,151 @@
|
|||
# Eth2.0 Test Generators
|
||||
|
||||
This directory of contains all the generators for YAML tests, consumed by Eth 2.0 client implementations.
|
||||
|
||||
Any issues with the generators and/or generated tests should be filed
|
||||
in the repository that hosts the generator outputs, here: [ethereum/eth2.0-tests](https://github.com/ethereum/eth2.0-tests/).
|
||||
|
||||
Whenever a release is made, the new tests are automatically built and
|
||||
[eth2TestGenBot](https://github.com/eth2TestGenBot) commits the changes to the test repository.
|
||||
|
||||
## How to run generators
|
||||
|
||||
pre-requisites:
|
||||
- Python 3 installed
|
||||
- PIP 3
|
||||
- GNU make
|
||||
|
||||
### Cleaning
|
||||
|
||||
This removes the existing virtual environments (`/test_generators/.venvs/`), and generated tests (`/yaml_tests/`).
|
||||
|
||||
```bash
|
||||
make clean
|
||||
```
|
||||
|
||||
### Running all test generators
|
||||
|
||||
This runs all the generators.
|
||||
|
||||
```bash
|
||||
make all
|
||||
```
|
||||
|
||||
### Running a single generator
|
||||
|
||||
The make file auto-detects generators in the `test_generators/` directory,
|
||||
and provides a tests-gen target for each generator, see example.
|
||||
|
||||
```bash
|
||||
make ./tests/shuffling/
|
||||
```
|
||||
|
||||
## Developing a generator
|
||||
|
||||
Simply open up the generator (not all at once) of choice in your favorite IDE/editor, and run:
|
||||
|
||||
```bash
|
||||
# Create a virtual environment (any venv/.venv/.venvs is git-ignored)
|
||||
python3 -m venv .venv
|
||||
# Activate the venv, this is where dependencies are installed for the generator
|
||||
. .venv/bin/activate
|
||||
```
|
||||
|
||||
Now that you have a virtual environment, write your generator.
|
||||
It's recommended to extend the base-generator.
|
||||
|
||||
Create a `requirements.txt` in the root of your generator directory:
|
||||
```
|
||||
eth-utils==1.4.1
|
||||
../test_libs/gen_helpers
|
||||
```
|
||||
|
||||
Install all the necessary requirements (re-run when you add more):
|
||||
```bash
|
||||
pip3 install -r requirements.txt --user
|
||||
```
|
||||
|
||||
And write your initial test generator, extending the base generator:
|
||||
|
||||
Write a `main.py` file, here's an example:
|
||||
|
||||
```python
|
||||
from gen_base import gen_runner, gen_suite, gen_typing
|
||||
|
||||
from eth_utils import (
|
||||
to_dict, to_tuple
|
||||
)
|
||||
|
||||
|
||||
@to_dict
|
||||
def bar_test_case(v: int):
|
||||
yield "bar_v", v
|
||||
yield "bar_v_plus_1", v + 1
|
||||
yield "bar_list", list(range(v))
|
||||
|
||||
|
||||
@to_tuple
|
||||
def generate_bar_test_cases():
|
||||
for i in range(10):
|
||||
yield bar_test_case(i)
|
||||
|
||||
|
||||
def bar_test_suite() -> gen_typing.TestSuite:
|
||||
return gen_suite.render_suite(
|
||||
title="bar_minimal",
|
||||
summary="Minimal example suite, testing bar.",
|
||||
fork="v0.5.1",
|
||||
config="minimal",
|
||||
test_cases=generate_bar_test_cases())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
gen_runner.run_generator("foo", [bar_test_suite])
|
||||
|
||||
```
|
||||
|
||||
Recommendations:
|
||||
- you can have more than just 1 generator, e.g. ` gen_runner.run_generator("foo", [bar_test_suite, abc_test_suite, example_test_suite])`
|
||||
- you can concatenate lists of test cases, if you don't want to split it up in suites.
|
||||
- you can split your suite generators into different python files/packages, good for code organization.
|
||||
- use config "minimal" for performance. But also implement a suite with the default config where necessary
|
||||
- the test-generator accepts `--output` and `--force` (overwrite output)
|
||||
|
||||
## How to add a new test generator
|
||||
|
||||
In order to add a new test generator that builds `New Tests`:
|
||||
|
||||
1. Create a new directory `new_tests`, within the `test_generators` directory.
|
||||
Note that `new_tests` is also the name of the directory in which the tests will appear in the tests repository later.
|
||||
2. Your generator is assumed to have a `requirements.txt` file,
|
||||
with any dependencies it may need. Leave it empty if your generator has none.
|
||||
3. Your generator is assumed to have a `main.py` file in its root.
|
||||
By adding the base generator to your requirements, you can make a generator really easily. See docs below.
|
||||
4. Your generator is called with `-o some/file/path/for_testing/can/be_anything`.
|
||||
The base generator helps you handle this; you only have to define suite headers,
|
||||
and a list of tests for each suite you generate.
|
||||
5. Finally, add any linting or testing commands to the
|
||||
[circleci config file](https://github.com/ethereum/eth2.0-test-generators/blob/master/.circleci/config.yml)
|
||||
if desired to increase code quality.
|
||||
|
||||
Note: you do not have to change the makefile.
|
||||
However, if necessary (e.g. not using python, or mixing in other languages), submit an issue, and it can be a special case.
|
||||
Do note that generators should be easy to maintain, lean, and based on the spec.
|
||||
|
||||
All of this should be done in a pull request to the master branch.
|
||||
|
||||
To deploy new tests to the testing repository:
|
||||
|
||||
1. Create a release tag with a new version number on Github.
|
||||
2. Increment either the:
|
||||
- major version, to indicate a change in the general testing format
|
||||
- minor version, if a new test generator has been added
|
||||
- path version, in other cases.
|
||||
|
||||
## How to remove a test generator
|
||||
|
||||
If a test generator is not needed anymore, undo the steps described above and make a new release:
|
||||
|
||||
1. remove the generator folder
|
||||
2. remove the generated tests in the `eth2.0-tests` repository by opening a PR there.
|
||||
3. make a new release
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
# BLS Test Generator
|
||||
|
||||
Explanation of BLS12-381 type hierarchy
|
||||
The base unit is bytes48 of which only 381 bits are used
|
||||
|
||||
- FQ: uint381 modulo field modulus
|
||||
- FQ2: (FQ, FQ)
|
||||
- G2: (FQ2, FQ2, FQ2)
|
||||
|
||||
## Resources
|
||||
|
||||
- [Eth2.0 spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/bls_signature.md)
|
||||
- [Finite Field Arithmetic](http://www.springeronline.com/sgw/cda/pageitems/document/cda_downloaddocument/0,11996,0-0-45-110359-0,00.pdf)
|
||||
- Chapter 2 of [Elliptic Curve Cryptography](http://cacr.uwaterloo.ca/ecc/). Darrel Hankerson, Alfred Menezes, and Scott Vanstone
|
||||
- [Zcash BLS parameters](https://github.com/zkcrypto/pairing/tree/master/src/bls12_381)
|
||||
- [Trinity implementation](https://github.com/ethereum/trinity/blob/master/eth2/_utils/bls.py)
|
||||
|
||||
## Comments
|
||||
|
||||
Compared to Zcash, Ethereum specs always requires the compressed form (c_flag / most significant bit always set).
|
||||
|
|
@ -0,0 +1,192 @@
|
|||
"""
|
||||
BLS test vectors generator
|
||||
Usage:
|
||||
"python tgen_bls path/to/output.yml"
|
||||
"""
|
||||
|
||||
# Standard library
|
||||
import sys
|
||||
from typing import Tuple
|
||||
|
||||
# Third-party
|
||||
import yaml
|
||||
|
||||
# Ethereum
|
||||
from eth_utils import int_to_big_endian, big_endian_to_int
|
||||
|
||||
# Local imports
|
||||
from py_ecc import bls
|
||||
|
||||
|
||||
def int_to_hex(n: int) -> str:
|
||||
return '0x' + int_to_big_endian(n).hex()
|
||||
|
||||
|
||||
def hex_to_int(x: str) -> int:
|
||||
return int(x, 16)
|
||||
|
||||
|
||||
# Note: even though a domain is only an uint64,
|
||||
# To avoid issues with YAML parsers that are limited to 53-bit (JS language limit)
|
||||
# It is serialized as an hex string as well.
|
||||
DOMAINS = [
|
||||
0,
|
||||
1,
|
||||
1234,
|
||||
2**32-1,
|
||||
2**64-1
|
||||
]
|
||||
|
||||
MESSAGES = [
|
||||
b'\x00' * 32,
|
||||
b'\x56' * 32,
|
||||
b'\xab' * 32,
|
||||
]
|
||||
|
||||
PRIVKEYS = [
|
||||
# Curve order is 256 so private keys are 32 bytes at most.
|
||||
# Also not all integers is a valid private key, so using pre-generated keys
|
||||
hex_to_int('0x00000000000000000000000000000000263dbd792f5b1be47ed85f8938c0f29586af0d3ac7b977f21c278fe1462040e3'),
|
||||
hex_to_int('0x0000000000000000000000000000000047b8192d77bf871b62e87859d653922725724a5c031afeabc60bcef5ff665138'),
|
||||
hex_to_int('0x00000000000000000000000000000000328388aff0d4a5b7dc9205abd374e7e98f3cd9f3418edb4eafda5fb16473d216'),
|
||||
]
|
||||
|
||||
|
||||
def hash_message(msg: bytes,
|
||||
domain: int) ->Tuple[Tuple[str, str], Tuple[str, str], Tuple[str, str]]:
|
||||
"""
|
||||
Hash message
|
||||
Input:
|
||||
- Message as bytes
|
||||
- domain as uint64
|
||||
Output:
|
||||
- Message hash as a G2 point
|
||||
"""
|
||||
return [
|
||||
[
|
||||
int_to_hex(fq2.coeffs[0]),
|
||||
int_to_hex(fq2.coeffs[1]),
|
||||
]
|
||||
for fq2 in bls.utils.hash_to_G2(msg, domain)
|
||||
]
|
||||
|
||||
|
||||
def hash_message_compressed(msg: bytes, domain: int) -> Tuple[str, str]:
|
||||
"""
|
||||
Hash message
|
||||
Input:
|
||||
- Message as bytes
|
||||
- domain as uint64
|
||||
Output:
|
||||
- Message hash as a compressed G2 point
|
||||
"""
|
||||
z1, z2 = bls.utils.compress_G2(bls.utils.hash_to_G2(msg, domain))
|
||||
return [int_to_hex(z1), int_to_hex(z2)]
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
||||
# Order not preserved - https://github.com/yaml/pyyaml/issues/110
|
||||
metadata = {
|
||||
'title': 'BLS signature and aggregation tests',
|
||||
'summary': 'Test vectors for BLS signature',
|
||||
'test_suite': 'bls',
|
||||
'fork': 'phase0-0.5.0',
|
||||
}
|
||||
|
||||
case01_message_hash_G2_uncompressed = []
|
||||
for msg in MESSAGES:
|
||||
for domain in DOMAINS:
|
||||
case01_message_hash_G2_uncompressed.append({
|
||||
'input': {'message': '0x' + msg.hex(), 'domain': int_to_hex(domain)},
|
||||
'output': hash_message(msg, domain)
|
||||
})
|
||||
|
||||
case02_message_hash_G2_compressed = []
|
||||
for msg in MESSAGES:
|
||||
for domain in DOMAINS:
|
||||
case02_message_hash_G2_compressed.append({
|
||||
'input': {'message': '0x' + msg.hex(), 'domain': int_to_hex(domain)},
|
||||
'output': hash_message_compressed(msg, domain)
|
||||
})
|
||||
|
||||
case03_private_to_public_key = []
|
||||
# Used in later cases
|
||||
pubkeys = [bls.privtopub(privkey) for privkey in PRIVKEYS]
|
||||
# Used in public key aggregation
|
||||
pubkeys_serial = ['0x' + pubkey.hex() for pubkey in pubkeys]
|
||||
case03_private_to_public_key = [
|
||||
{
|
||||
'input': int_to_hex(privkey),
|
||||
'output': pubkey_serial,
|
||||
}
|
||||
for privkey, pubkey_serial in zip(PRIVKEYS, pubkeys_serial)
|
||||
]
|
||||
|
||||
case04_sign_messages = []
|
||||
sigs = [] # used in verify
|
||||
for privkey in PRIVKEYS:
|
||||
for message in MESSAGES:
|
||||
for domain in DOMAINS:
|
||||
sig = bls.sign(message, privkey, domain)
|
||||
case04_sign_messages.append({
|
||||
'input': {
|
||||
'privkey': int_to_hex(privkey),
|
||||
'message': '0x' + message.hex(),
|
||||
'domain': int_to_hex(domain)
|
||||
},
|
||||
'output': '0x' + sig.hex()
|
||||
})
|
||||
sigs.append(sig)
|
||||
|
||||
# TODO: case05_verify_messages: Verify messages signed in case04
|
||||
# It takes too long, empty for now
|
||||
|
||||
case06_aggregate_sigs = []
|
||||
for domain in DOMAINS:
|
||||
for message in MESSAGES:
|
||||
sigs = []
|
||||
for privkey in PRIVKEYS:
|
||||
sig = bls.sign(message, privkey, domain)
|
||||
sigs.append(sig)
|
||||
case06_aggregate_sigs.append({
|
||||
'input': ['0x' + sig.hex() for sig in sigs],
|
||||
'output': '0x' + bls.aggregate_signatures(sigs).hex(),
|
||||
})
|
||||
|
||||
case07_aggregate_pubkeys = [
|
||||
{
|
||||
'input': pubkeys_serial,
|
||||
'output': '0x' + bls.aggregate_pubkeys(pubkeys).hex(),
|
||||
}
|
||||
]
|
||||
|
||||
# TODO
|
||||
# Aggregate verify
|
||||
|
||||
# TODO
|
||||
# Proof-of-possession
|
||||
|
||||
with open(sys.argv[2] + "test_bls.yml", 'w') as outfile:
|
||||
# Dump at top level
|
||||
yaml.dump(metadata, outfile, default_flow_style=False)
|
||||
# default_flow_style will unravel "ValidatorRecord" and "committee" line,
|
||||
# exploding file size
|
||||
yaml.dump(
|
||||
{'case01_message_hash_G2_uncompressed': case01_message_hash_G2_uncompressed},
|
||||
outfile,
|
||||
)
|
||||
yaml.dump(
|
||||
{'case02_message_hash_G2_compressed': case02_message_hash_G2_compressed},
|
||||
outfile,
|
||||
)
|
||||
yaml.dump(
|
||||
{'case03_private_to_public_key': case03_private_to_public_key},
|
||||
outfile,
|
||||
)
|
||||
yaml.dump({'case04_sign_messages': case04_sign_messages}, outfile)
|
||||
|
||||
# Too time consuming to generate
|
||||
# yaml.dump({'case05_verify_messages': case05_verify_messages}, outfile)
|
||||
yaml.dump({'case06_aggregate_sigs': case06_aggregate_sigs}, outfile)
|
||||
yaml.dump({'case07_aggregate_pubkeys': case07_aggregate_pubkeys}, outfile)
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
py-ecc==1.6.0
|
||||
PyYAML==4.2b1
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
# Shuffling Test Generator
|
||||
|
||||
```
|
||||
2018 Status Research & Development GmbH
|
||||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
|
||||
|
||||
This work uses public domain work under CC0 from the Ethereum Foundation
|
||||
https://github.com/ethereum/eth2.0-specs
|
||||
```
|
||||
|
||||
|
||||
This file implements a test vectors generator for the shuffling algorithm described in the Ethereum
|
||||
[specs](https://github.com/ethereum/eth2.0-specs/blob/2983e68f0305551083fac7fcf9330c1fc9da3411/specs/core/0_beacon-chain.md#get_new_shuffling)
|
||||
|
||||
Utilizes 'swap or not' shuffling found in [An Enciphering Scheme Based on a Card Shuffle](https://link.springer.com/content/pdf/10.1007%2F978-3-642-32009-5_1.pdf).
|
||||
See the `Generalized domain` algorithm on page 3.
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
SLOTS_PER_EPOCH = 2**6 # 64 slots, 6.4 minutes
|
||||
FAR_FUTURE_EPOCH = 2**64 - 1 # uint64 max
|
||||
SHARD_COUNT = 2**10 # 1024
|
||||
TARGET_COMMITTEE_SIZE = 2**7 # 128 validators
|
||||
ACTIVATION_EXIT_DELAY = 2**2 # 4 epochs
|
||||
SHUFFLE_ROUND_COUNT = 90
|
||||
|
|
@ -0,0 +1,95 @@
|
|||
from typing import Any, List, NewType
|
||||
|
||||
from constants import SLOTS_PER_EPOCH, SHARD_COUNT, TARGET_COMMITTEE_SIZE, SHUFFLE_ROUND_COUNT
|
||||
from utils import hash
|
||||
from yaml_objects import Validator
|
||||
|
||||
Epoch = NewType("Epoch", int)
|
||||
ValidatorIndex = NewType("ValidatorIndex", int)
|
||||
Bytes32 = NewType("Bytes32", bytes)
|
||||
|
||||
|
||||
def int_to_bytes1(x):
|
||||
return x.to_bytes(1, 'little')
|
||||
|
||||
|
||||
def int_to_bytes4(x):
|
||||
return x.to_bytes(4, 'little')
|
||||
|
||||
|
||||
def bytes_to_int(data: bytes) -> int:
|
||||
return int.from_bytes(data, 'little')
|
||||
|
||||
|
||||
def is_active_validator(validator: Validator, epoch: Epoch) -> bool:
|
||||
"""
|
||||
Check if ``validator`` is active.
|
||||
"""
|
||||
return validator.activation_epoch <= epoch < validator.exit_epoch
|
||||
|
||||
|
||||
def get_active_validator_indices(validators: List[Validator], epoch: Epoch) -> List[ValidatorIndex]:
|
||||
"""
|
||||
Get indices of active validators from ``validators``.
|
||||
"""
|
||||
return [i for i, v in enumerate(validators) if is_active_validator(v, epoch)]
|
||||
|
||||
|
||||
def split(values: List[Any], split_count: int) -> List[List[Any]]:
|
||||
"""
|
||||
Splits ``values`` into ``split_count`` pieces.
|
||||
"""
|
||||
list_length = len(values)
|
||||
return [
|
||||
values[(list_length * i // split_count): (list_length * (i + 1) // split_count)]
|
||||
for i in range(split_count)
|
||||
]
|
||||
|
||||
|
||||
def get_epoch_committee_count(active_validator_count: int) -> int:
|
||||
"""
|
||||
Return the number of committees in one epoch.
|
||||
"""
|
||||
return max(
|
||||
1,
|
||||
min(
|
||||
SHARD_COUNT // SLOTS_PER_EPOCH,
|
||||
active_validator_count // SLOTS_PER_EPOCH // TARGET_COMMITTEE_SIZE,
|
||||
)
|
||||
) * SLOTS_PER_EPOCH
|
||||
|
||||
|
||||
def get_permuted_index(index: int, list_size: int, seed: Bytes32) -> int:
|
||||
"""
|
||||
Return `p(index)` in a pseudorandom permutation `p` of `0...list_size-1` with ``seed`` as entropy.
|
||||
|
||||
Utilizes 'swap or not' shuffling found in
|
||||
https://link.springer.com/content/pdf/10.1007%2F978-3-642-32009-5_1.pdf
|
||||
See the 'generalized domain' algorithm on page 3.
|
||||
"""
|
||||
for round in range(SHUFFLE_ROUND_COUNT):
|
||||
pivot = bytes_to_int(hash(seed + int_to_bytes1(round))[0:8]) % list_size
|
||||
flip = (pivot - index) % list_size
|
||||
position = max(index, flip)
|
||||
source = hash(seed + int_to_bytes1(round) + int_to_bytes4(position // 256))
|
||||
byte = source[(position % 256) // 8]
|
||||
bit = (byte >> (position % 8)) % 2
|
||||
index = flip if bit else index
|
||||
|
||||
return index
|
||||
|
||||
|
||||
def get_shuffling(seed: Bytes32,
|
||||
validators: List[Validator],
|
||||
epoch: Epoch) -> List[List[ValidatorIndex]]:
|
||||
"""
|
||||
Shuffle active validators and split into crosslink committees.
|
||||
Return a list of committees (each a list of validator indices).
|
||||
"""
|
||||
# Shuffle active validator indices
|
||||
active_validator_indices = get_active_validator_indices(validators, epoch)
|
||||
length = len(active_validator_indices)
|
||||
shuffled_indices = [active_validator_indices[get_permuted_index(i, length, seed)] for i in range(length)]
|
||||
|
||||
# Split the shuffled active validator indices
|
||||
return split(shuffled_indices, get_epoch_committee_count(length))
|
||||
|
|
@ -0,0 +1,160 @@
|
|||
import random
|
||||
import sys
|
||||
import os
|
||||
|
||||
import yaml
|
||||
|
||||
from constants import ACTIVATION_EXIT_DELAY, FAR_FUTURE_EPOCH
|
||||
from core_helpers import get_shuffling
|
||||
from yaml_objects import Validator
|
||||
|
||||
|
||||
def noop(self, *args, **kw):
|
||||
# Prevent !!str or !!binary tags
|
||||
pass
|
||||
|
||||
|
||||
yaml.emitter.Emitter.process_tag = noop
|
||||
|
||||
|
||||
EPOCH = 1000 # The epoch, also a mean for the normal distribution
|
||||
|
||||
# Standard deviation, around 8% validators will activate or exit within
|
||||
# ENTRY_EXIT_DELAY inclusive from EPOCH thus creating an edge case for validator
|
||||
# shuffling
|
||||
RAND_EPOCH_STD = 35
|
||||
MAX_EXIT_EPOCH = 5000 # Maximum exit_epoch for easier reading
|
||||
|
||||
|
||||
def active_exited_validators_generator():
|
||||
"""
|
||||
Random cases with variety of validator's activity status
|
||||
"""
|
||||
# Order not preserved - https://github.com/yaml/pyyaml/issues/110
|
||||
metadata = {
|
||||
'title': 'Shuffling Algorithm Tests 1',
|
||||
'summary': 'Test vectors for validator shuffling with different validator\'s activity status.'
|
||||
' Note: only relevant validator fields are defined.',
|
||||
'test_suite': 'shuffle',
|
||||
'fork': 'phase0-0.5.0',
|
||||
}
|
||||
|
||||
# Config
|
||||
random.seed(int("0xEF00BEAC", 16))
|
||||
num_cases = 10
|
||||
|
||||
test_cases = []
|
||||
|
||||
for case in range(num_cases):
|
||||
seedhash = bytes(random.randint(0, 255) for byte in range(32))
|
||||
idx_max = random.randint(128, 512)
|
||||
|
||||
validators = []
|
||||
for idx in range(idx_max):
|
||||
v = Validator(original_index=idx)
|
||||
# 4/5 of all validators are active
|
||||
if random.random() < 0.8:
|
||||
# Choose a normally distributed epoch number
|
||||
rand_epoch = round(random.gauss(EPOCH, RAND_EPOCH_STD))
|
||||
|
||||
# for 1/2 of *active* validators rand_epoch is the activation epoch
|
||||
if random.random() < 0.5:
|
||||
v.activation_epoch = rand_epoch
|
||||
|
||||
# 1/4 of active validators will exit in forseeable future
|
||||
if random.random() < 0.5:
|
||||
v.exit_epoch = random.randint(
|
||||
rand_epoch + ACTIVATION_EXIT_DELAY + 1, MAX_EXIT_EPOCH)
|
||||
# 1/4 of active validators in theory remain in the set indefinitely
|
||||
else:
|
||||
v.exit_epoch = FAR_FUTURE_EPOCH
|
||||
# for the other active 1/2 rand_epoch is the exit epoch
|
||||
else:
|
||||
v.activation_epoch = random.randint(
|
||||
0, rand_epoch - ACTIVATION_EXIT_DELAY)
|
||||
v.exit_epoch = rand_epoch
|
||||
|
||||
# The remaining 1/5 of all validators is not activated
|
||||
else:
|
||||
v.activation_epoch = FAR_FUTURE_EPOCH
|
||||
v.exit_epoch = FAR_FUTURE_EPOCH
|
||||
|
||||
validators.append(v)
|
||||
|
||||
input_ = {
|
||||
'validators': validators,
|
||||
'epoch': EPOCH
|
||||
}
|
||||
output = get_shuffling(
|
||||
seedhash, validators, input_['epoch'])
|
||||
|
||||
test_cases.append({
|
||||
'seed': '0x' + seedhash.hex(), 'input': input_, 'output': output
|
||||
})
|
||||
|
||||
return {
|
||||
'metadata': metadata,
|
||||
'filename': 'test_vector_shuffling.yml',
|
||||
'test_cases': test_cases
|
||||
}
|
||||
|
||||
|
||||
def validators_set_size_variety_generator():
|
||||
"""
|
||||
Different validator set size cases, inspired by removed manual `permutated_index` tests
|
||||
https://github.com/ethereum/eth2.0-test-generators/tree/bcd9ab2933d9f696901d1dfda0828061e9d3093f/permutated_index
|
||||
"""
|
||||
# Order not preserved - https://github.com/yaml/pyyaml/issues/110
|
||||
metadata = {
|
||||
'title': 'Shuffling Algorithm Tests 2',
|
||||
'summary': 'Test vectors for validator shuffling with different validator\'s set size.'
|
||||
' Note: only relevant validator fields are defined.',
|
||||
'test_suite': 'shuffle',
|
||||
'fork': 'tchaikovsky',
|
||||
'version': 1.0
|
||||
}
|
||||
|
||||
# Config
|
||||
random.seed(int("0xEF00BEAC", 16))
|
||||
|
||||
test_cases = []
|
||||
|
||||
seedhash = bytes(random.randint(0, 255) for byte in range(32))
|
||||
idx_max = 4096
|
||||
set_sizes = [1, 2, 3, 1024, idx_max]
|
||||
|
||||
for size in set_sizes:
|
||||
validators = []
|
||||
for idx in range(size):
|
||||
v = Validator(original_index=idx)
|
||||
v.activation_epoch = EPOCH
|
||||
v.exit_epoch = FAR_FUTURE_EPOCH
|
||||
validators.append(v)
|
||||
input_ = {
|
||||
'validators': validators,
|
||||
'epoch': EPOCH
|
||||
}
|
||||
output = get_shuffling(
|
||||
seedhash, validators, input_['epoch'])
|
||||
|
||||
test_cases.append({
|
||||
'seed': '0x' + seedhash.hex(), 'input': input_, 'output': output
|
||||
})
|
||||
|
||||
return {
|
||||
'metadata': metadata,
|
||||
'filename': 'shuffling_set_size.yml',
|
||||
'test_cases': test_cases
|
||||
}
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
output_dir = sys.argv[2]
|
||||
for generator in [active_exited_validators_generator, validators_set_size_variety_generator]:
|
||||
result = generator()
|
||||
filename = os.path.join(output_dir, result['filename'])
|
||||
with open(filename, 'w') as outfile:
|
||||
# Dump at top level
|
||||
yaml.dump(result['metadata'], outfile, default_flow_style=False)
|
||||
# default_flow_style will unravel "ValidatorRecord" and "committee" line, exploding file size
|
||||
yaml.dump({'test_cases': result['test_cases']}, outfile)
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
eth-hash[pycryptodome]==0.2.0
|
||||
eth-typing==2.0.0
|
||||
eth-utils==1.4.1
|
||||
PyYAML==4.2b1
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
from eth_typing import Hash32
|
||||
from eth_utils import keccak
|
||||
|
||||
|
||||
def hash(x: bytes) -> Hash32:
|
||||
return keccak(x)
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
|
||||
class Validator(yaml.YAMLObject):
|
||||
"""
|
||||
A validator stub containing only the fields relevant for get_shuffling()
|
||||
"""
|
||||
fields = {
|
||||
'activation_epoch': 'uint64',
|
||||
'exit_epoch': 'uint64',
|
||||
# Extra index field to ease testing/debugging
|
||||
'original_index': 'uint64',
|
||||
}
|
||||
|
||||
def __init__(self, **kwargs):
|
||||
for k in self.fields.keys():
|
||||
setattr(self, k, kwargs.get(k))
|
||||
|
||||
def __setattr__(self, name: str, value: Any) -> None:
|
||||
super().__setattr__(name, value)
|
||||
|
||||
def __getattribute__(self, name: str) -> Any:
|
||||
return super().__getattribute__(name)
|
||||
|
|
@ -0,0 +1,84 @@
|
|||
import argparse
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
from ruamel.yaml import (
|
||||
YAML,
|
||||
)
|
||||
|
||||
from uint_test_generators import (
|
||||
generate_uint_bounds_test,
|
||||
generate_uint_random_test,
|
||||
generate_uint_wrong_length_test,
|
||||
)
|
||||
|
||||
test_generators = [
|
||||
generate_uint_random_test,
|
||||
generate_uint_wrong_length_test,
|
||||
generate_uint_bounds_test,
|
||||
]
|
||||
|
||||
|
||||
def make_filename_for_test(test):
|
||||
title = test["title"]
|
||||
filename = title.lower().replace(" ", "_") + ".yaml"
|
||||
return pathlib.Path(filename)
|
||||
|
||||
|
||||
def validate_output_dir(path_str):
|
||||
path = pathlib.Path(path_str)
|
||||
|
||||
if not path.exists():
|
||||
raise argparse.ArgumentTypeError("Output directory must exist")
|
||||
|
||||
if not path.is_dir():
|
||||
raise argparse.ArgumentTypeError("Output path must lead to a directory")
|
||||
|
||||
return path
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="gen-ssz-tests",
|
||||
description="Generate YAML test files for SSZ and tree hashing",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-o",
|
||||
"--output-dir",
|
||||
dest="output_dir",
|
||||
required=True,
|
||||
type=validate_output_dir,
|
||||
help="directory into which the generated YAML files will be dumped"
|
||||
)
|
||||
parser.add_argument(
|
||||
"-f",
|
||||
"--force",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="if set overwrite test files if they exist",
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
args = parser.parse_args()
|
||||
output_dir = args.output_dir
|
||||
if not args.force:
|
||||
file_mode = "x"
|
||||
else:
|
||||
file_mode = "w"
|
||||
|
||||
yaml = YAML(pure=True)
|
||||
|
||||
print(f"generating {len(test_generators)} test files...")
|
||||
for test_generator in test_generators:
|
||||
test = test_generator()
|
||||
|
||||
filename = make_filename_for_test(test)
|
||||
path = output_dir / filename
|
||||
|
||||
try:
|
||||
with path.open(file_mode) as f:
|
||||
yaml.dump(test, f)
|
||||
except IOError as e:
|
||||
sys.exit(f'Error when dumping test "{test["title"]}" ({e})')
|
||||
|
||||
print("done.")
|
||||
|
|
@ -0,0 +1,102 @@
|
|||
from collections.abc import (
|
||||
Mapping,
|
||||
Sequence,
|
||||
)
|
||||
|
||||
from eth_utils import (
|
||||
encode_hex,
|
||||
to_dict,
|
||||
)
|
||||
|
||||
from ssz.sedes import (
|
||||
BaseSedes,
|
||||
Boolean,
|
||||
Bytes,
|
||||
BytesN,
|
||||
Container,
|
||||
List,
|
||||
UInt,
|
||||
)
|
||||
|
||||
|
||||
def render_value(value):
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
elif isinstance(value, int):
|
||||
return str(value)
|
||||
elif isinstance(value, bytes):
|
||||
return encode_hex(value)
|
||||
elif isinstance(value, Sequence):
|
||||
return tuple(render_value(element) for element in value)
|
||||
elif isinstance(value, Mapping):
|
||||
return render_dict_value(value)
|
||||
else:
|
||||
raise ValueError(f"Cannot render value {value}")
|
||||
|
||||
|
||||
@to_dict
|
||||
def render_dict_value(value):
|
||||
for key, value in value.items():
|
||||
yield key, render_value(value)
|
||||
|
||||
|
||||
def render_type_definition(sedes):
|
||||
if isinstance(sedes, Boolean):
|
||||
return "bool"
|
||||
|
||||
elif isinstance(sedes, UInt):
|
||||
return f"uint{sedes.length * 8}"
|
||||
|
||||
elif isinstance(sedes, BytesN):
|
||||
return f"bytes{sedes.length}"
|
||||
|
||||
elif isinstance(sedes, Bytes):
|
||||
return f"bytes"
|
||||
|
||||
elif isinstance(sedes, List):
|
||||
return [render_type_definition(sedes.element_sedes)]
|
||||
|
||||
elif isinstance(sedes, Container):
|
||||
return {
|
||||
field_name: render_type_definition(field_sedes)
|
||||
for field_name, field_sedes in sedes.fields
|
||||
}
|
||||
|
||||
elif isinstance(sedes, BaseSedes):
|
||||
raise Exception("Unreachable: All sedes types have been checked")
|
||||
|
||||
else:
|
||||
raise TypeError("Expected BaseSedes")
|
||||
|
||||
|
||||
@to_dict
|
||||
def render_test_case(*, sedes, valid, value=None, serial=None, description=None, tags=None):
|
||||
value_and_serial_given = value is not None and serial is not None
|
||||
if valid:
|
||||
if not value_and_serial_given:
|
||||
raise ValueError("For valid test cases, both value and ssz must be present")
|
||||
else:
|
||||
if value_and_serial_given:
|
||||
raise ValueError("For invalid test cases, either value or ssz must not be present")
|
||||
|
||||
if tags is None:
|
||||
tags = []
|
||||
|
||||
yield "type", render_type_definition(sedes)
|
||||
yield "valid", valid
|
||||
if value is not None:
|
||||
yield "value", render_value(value)
|
||||
if serial is not None:
|
||||
yield "ssz", encode_hex(serial)
|
||||
if description is not None:
|
||||
yield description
|
||||
yield "tags", tags
|
||||
|
||||
|
||||
@to_dict
|
||||
def render_test(*, title, summary, fork, test_cases):
|
||||
yield "title", title,
|
||||
if summary is not None:
|
||||
yield "summary", summary
|
||||
yield "fork", fork
|
||||
yield "test_cases", test_cases
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
ruamel.yaml==0.15.87
|
||||
ssz==0.1.0a2
|
||||
|
|
@ -0,0 +1,132 @@
|
|||
import random
|
||||
|
||||
from eth_utils import (
|
||||
to_tuple,
|
||||
)
|
||||
|
||||
import ssz
|
||||
from ssz.sedes import (
|
||||
UInt,
|
||||
)
|
||||
from renderers import (
|
||||
render_test,
|
||||
render_test_case,
|
||||
)
|
||||
|
||||
random.seed(0)
|
||||
|
||||
|
||||
BIT_SIZES = [i for i in range(8, 512 + 1, 8)]
|
||||
RANDOM_TEST_CASES_PER_BIT_SIZE = 10
|
||||
RANDOM_TEST_CASES_PER_LENGTH = 3
|
||||
|
||||
|
||||
def get_random_bytes(length):
|
||||
return bytes(random.randint(0, 255) for _ in range(length))
|
||||
|
||||
|
||||
def generate_uint_bounds_test():
|
||||
test_cases = generate_uint_bounds_test_cases() + generate_uint_out_of_bounds_test_cases()
|
||||
|
||||
return render_test(
|
||||
title="UInt Bounds",
|
||||
summary="Integers right at or beyond the bounds of the allowed value range",
|
||||
fork="phase0-0.2.0",
|
||||
test_cases=test_cases,
|
||||
)
|
||||
|
||||
|
||||
def generate_uint_random_test():
|
||||
test_cases = generate_random_uint_test_cases()
|
||||
|
||||
return render_test(
|
||||
title="UInt Random",
|
||||
summary="Random integers chosen uniformly over the allowed value range",
|
||||
fork="phase0-0.2.0",
|
||||
test_cases=test_cases,
|
||||
)
|
||||
|
||||
|
||||
def generate_uint_wrong_length_test():
|
||||
test_cases = generate_uint_wrong_length_test_cases()
|
||||
|
||||
return render_test(
|
||||
title="UInt Wrong Length",
|
||||
summary="Serialized integers that are too short or too long",
|
||||
fork="phase0-0.2.0",
|
||||
test_cases=test_cases,
|
||||
)
|
||||
|
||||
|
||||
@to_tuple
|
||||
def generate_random_uint_test_cases():
|
||||
for bit_size in BIT_SIZES:
|
||||
sedes = UInt(bit_size)
|
||||
|
||||
for _ in range(RANDOM_TEST_CASES_PER_BIT_SIZE):
|
||||
value = random.randrange(0, 2 ** bit_size)
|
||||
serial = ssz.encode(value, sedes)
|
||||
# note that we need to create the tags in each loop cycle, otherwise ruamel will use
|
||||
# YAML references which makes the resulting file harder to read
|
||||
tags = tuple(["atomic", "uint", "random"])
|
||||
yield render_test_case(
|
||||
sedes=sedes,
|
||||
valid=True,
|
||||
value=value,
|
||||
serial=serial,
|
||||
tags=tags,
|
||||
)
|
||||
|
||||
|
||||
@to_tuple
|
||||
def generate_uint_wrong_length_test_cases():
|
||||
for bit_size in BIT_SIZES:
|
||||
sedes = UInt(bit_size)
|
||||
lengths = sorted({
|
||||
0,
|
||||
sedes.length // 2,
|
||||
sedes.length - 1,
|
||||
sedes.length + 1,
|
||||
sedes.length * 2,
|
||||
})
|
||||
for length in lengths:
|
||||
for _ in range(RANDOM_TEST_CASES_PER_LENGTH):
|
||||
tags = tuple(["atomic", "uint", "wrong_length"])
|
||||
yield render_test_case(
|
||||
sedes=sedes,
|
||||
valid=False,
|
||||
serial=get_random_bytes(length),
|
||||
tags=tags,
|
||||
)
|
||||
|
||||
|
||||
@to_tuple
|
||||
def generate_uint_bounds_test_cases():
|
||||
common_tags = ("atomic", "uint")
|
||||
for bit_size in BIT_SIZES:
|
||||
sedes = UInt(bit_size)
|
||||
|
||||
for value, tag in ((0, "uint_lower_bound"), (2 ** bit_size - 1, "uint_upper_bound")):
|
||||
serial = ssz.encode(value, sedes)
|
||||
yield render_test_case(
|
||||
sedes=sedes,
|
||||
valid=True,
|
||||
value=value,
|
||||
serial=serial,
|
||||
tags=common_tags + (tag,),
|
||||
)
|
||||
|
||||
|
||||
@to_tuple
|
||||
def generate_uint_out_of_bounds_test_cases():
|
||||
common_tags = ("atomic", "uint")
|
||||
for bit_size in BIT_SIZES:
|
||||
sedes = UInt(bit_size)
|
||||
|
||||
for value, tag in ((-1, "uint_underflow"), (2 ** bit_size, "uint_overflow")):
|
||||
yield render_test_case(
|
||||
sedes=sedes,
|
||||
valid=False,
|
||||
value=value,
|
||||
tags=common_tags + (tag,),
|
||||
)
|
||||
Loading…
Reference in New Issue