minor naming tweaks, document BLS and deposit test formats

specs/test_formats/README.md

@@ -77,6 +77,12 @@ The way this list of forks is interpreted, is up to the test-runner:
 State-transition test suites may want to just declare forks that are being covered in the test suite,
  whereas shuffling test suites may want to declare a list of forks to test the shuffling algorithm for individually.
 
+Test-formats specify the following `forks` interpretation rules:
+
+- `collective`: the test suite applies to all specified forks, and only needs to run once
+- `individual`: the test suite should be ran against every fork
+- more types may be specified with future test types.
+
 ### Test completeness
 
 Tests should be independent of any sync-data. If one wants to run a test, the input data should be available from the YAML.

specs/test_formats/bls/README.md

@@ -0,0 +1,15 @@
+# BLS test suite
+
+A test suite for BLS. Primarily geared towards verifying the *integration* of any BLS library.
+We do not recommend to roll your own crypto, or use an untested BLS library.
+
+The BLS test suite runner has the following handlers:
+
+- [`aggregate_pubkeys`](./aggregate_pubkeys.md)
+- [`aggregate_sigs`](./aggregate_sigs.md)
+- [`msg_hash_g2_compressed`](./msg_hash_g2_compressed.md)
+- [`msg_hash_g2_uncompressed`](./msg_hash_g2_uncompressed.md)
+- [`priv_to_pub`](./priv_to_pub.md)
+- [`sign_msg`](./sign_msg.md)
+
+Note: signature-verification and aggregate-verify test cases are not yet supported.

specs/test_formats/bls/aggregate_pubkeys.md

@@ -0,0 +1,21 @@
+# Test format: BLS pubkey aggregation
+
+A BLS pubkey aggregation combines a series of pubkeys into a single pubkey.
+
+## Test case format
+
+```yaml
+input: List[BLS Pubkey] -- list of input BLS pubkeys
+output: BLS Pubkey -- expected output, single BLS pubkey
+```
+
+`BLS Pubkey` here is encoded as a string: hexadecimal encoding of 48 bytes (96 nibbles), prefixed with `0x`.
+
+
+## Condition
+
+The `aggregate_pubkeys` handler should aggregate the keys in the `input`, and the result should match the expected `output`.
+
+## Forks
+
+Forks-interpretation: `collective` 

specs/test_formats/bls/aggregate_sigs.md

@@ -0,0 +1,21 @@
+# Test format: BLS signature aggregation
+
+A BLS signature aggregation combines a series of signatures into a single signature.
+
+## Test case format
+
+```yaml
+input: List[BLS Signature] -- list of input BLS signatures
+output: BLS Signature -- expected output, single BLS signature
+```
+
+`BLS Signature` here is encoded as a string: hexadecimal encoding of 96 bytes (192 nibbles), prefixed with `0x`.
+
+
+## Condition
+
+The `aggregate_sigs` handler should aggregate the signatures in the `input`, and the result should match the expected `output`.
+
+## Forks
+
+Forks-interpretation: `collective` 

specs/test_formats/bls/msg_hash_g2_compressed.md

@@ -0,0 +1,23 @@
+# Test format: BLS hash-compressed
+
+A BLS compressed-hash to G2.
+
+## Test case format
+
+```yaml
+input: 
+  message: bytes32,
+  domain: bytes -- any number
+output: List[bytes48] -- length of two
+```
+
+All byte(s) fields are encoded as strings, hexadecimal encoding, prefixed with `0x`
+
+
+## Condition
+
+The `msg_hash_g2_compressed` handler should hash the `message`, with the given `domain`, to G2 with compression, and the result should match the expected `output`.
+
+## Forks
+
+Forks-interpretation: `collective` 

specs/test_formats/bls/msg_hash_g2_uncompressed.md

@@ -0,0 +1,23 @@
+# Test format: BLS hash-uncompressed
+
+A BLS uncompressed-hash to G2. 
+
+## Test case format
+
+```yaml
+input: 
+  message: bytes32,
+  domain: bytes -- any number
+output: List[List[bytes48]] -- 3 lists, each a length of two
+```
+
+All byte(s) fields are encoded as strings, hexadecimal encoding, prefixed with `0x`
+
+
+## Condition
+
+The `msg_hash_g2_uncompressed` handler should hash the `message`, with the given `domain`, to G2, without compression, and the result should match the expected `output`.
+
+## Forks
+
+Forks-interpretation: `collective` 

specs/test_formats/bls/priv_to_pub.md

@@ -0,0 +1,21 @@
+# Test format: BLS private key to pubkey
+
+A BLS private key to public key conversion.
+
+## Test case format
+
+```yaml
+input: bytes32 -- the private key
+output: bytes48 -- the public key
+```
+
+All byte(s) fields are encoded as strings, hexadecimal encoding, prefixed with `0x`
+
+
+## Condition
+
+The `priv_to_pub` handler should compute the public key for the given private key `input`, and the result should match the expected `output`.
+
+## Forks
+
+Forks-interpretation: `collective` 

specs/test_formats/bls/sign_msg.md

@@ -0,0 +1,24 @@
+# Test format: BLS sign message
+
+Message signing with BLS should produce a signature. 
+
+## Test case format
+
+```yaml
+input:
+  privkey: bytes32 -- the private key used for signing
+  message: bytes32 -- input message to sign (a hash)
+  domain: bytes    -- BLS domain
+output: bytes96    -- expected signature
+```
+
+All byte(s) fields are encoded as strings, hexadecimal encoding, prefixed with `0x`
+
+
+## Condition
+
+The `sign_msg` handler should sign the given `message`, with `domain`, using the given `privkey`, and the result should match the expected `output`.
+
+## Forks
+
+Forks-interpretation: `collective` 

specs/test_formats/operations/README.md

@@ -0,0 +1,4 @@
+# Operations test suite
+
+The operations test suite
+

specs/test_formats/operations/deposits.md

@@ -0,0 +1,26 @@
+# Test format: Deposit operations
+
+A deposit is a form of an operation (or "transaction"), modifying the state.
+
+## Test case format
+
+```yaml
+case: string      -- description of test case, purely for debugging purposes
+pre: BeaconState  -- state before applying the deposit
+deposit: Deposit  -- the deposit
+post: BeaconState -- state after applying the deposit. No value if deposit processing is aborted.
+```
+
+## Condition
+
+A `deposits` handler of the `operations` should process these cases, 
+ calling the implementation of the `process_deposit(state, deposit)` functionality described in the spec.
+The resulting state should match the expected `post` state, or no change if the `post` state is left blank.
+
+## Forks
+
+Forks-interpretation: `collective` 
+
+Pre and post state contain slot numbers, and are time sensitive. 
+Additional tests will be added for future forks to cover fork-specific behavior based on input data 
+ (including suites with deposits on fork transition blocks, covering multiple forks)

test_generators/bls/main.py

@@ -160,7 +160,7 @@ def case07_aggregate_pubkeys():
 
 
 def bls_msg_hash_uncompressed_suite(configs_path: str) -> gen_typing.TestSuiteOutput:
-    return ("g2_uncompressed", "msg_hash_uncompressed", gen_suite.render_suite(
+    return ("g2_uncompressed", "msg_hash_g2_uncompressed", gen_suite.render_suite(
         title="BLS G2 Uncompressed msg hash",
         summary="BLS G2 Uncompressed msg hash",
         forks_timeline="mainnet",
@@ -171,7 +171,7 @@ def bls_msg_hash_uncompressed_suite(configs_path: str) -> gen_typing.TestSuiteOu
 
 
 def bls_msg_hash_compressed_suite(configs_path: str) -> gen_typing.TestSuiteOutput:
-    return ("g2_compressed", "msg_hash_compressed", gen_suite.render_suite(
+    return ("g2_compressed", "msg_hash_g2_compressed", gen_suite.render_suite(
         title="BLS G2 Compressed msg hash",
         summary="BLS G2 Compressed msg hash",
         forks_timeline="mainnet",

test_generators/operations/main.py

@@ -1,6 +1,6 @@
 from gen_base import gen_runner
 
-from deposit import mini_deposits_suite, full_deposits_suite
+from deposits import mini_deposits_suite, full_deposits_suite
 
 if __name__ == "__main__":
     gen_runner.run_generator("operations", [