update test format doc and SSZ-static format docs

specs/test_formats/README.md

@@ -49,22 +49,20 @@ Test formats:
 
 ## Glossary
 
-- `generator`: a program that outputs one or more `suite` files.
-  - A generator should only output one `type` of test.
-  - A generator is free to output multiple `suite` files, optionally with different `handler`s.
-- `type`: the specialization of one single `generator`.
-- `suite`: a YAML file with:
-  - a header: describes the `suite`, and defines what the `suite` is for
-  - a list of test cases
+- `generator`: a program that outputs one or more test-cases, each organized into a `config > runner > handler > suite` hierarchy.
+- `config`: tests are grouped by configuration used for spec presets. In addition to the standard configurations, 
+  `general` may be used as a catch-all for tests not restricted to one configuration. (E.g. BLS).
+- `type`: the specialization of one single `generator`. E.g. epoch processing.
 - `runner`: where a generator is a *"producer"*, this is the *"consumer"*.
   - A `runner` focuses on *only one* `type`, and each type has *only one* `runner`.
-- `handler`: a `runner` may be too limited sometimes, you may have a `suite` with a specific focus that requires a different format.
+- `handler`: a `runner` may be too limited sometimes, you may have a set of tests with a specific focus that requires a different format.
   To facilitate this, you specify a `handler`: the runner can deal with the format by using the specified handler.
-  Using a `handler` in a `runner` is optional.
-- `case`: a test case, an entry in the `test_cases` list of a `suite`. A case can be anything in general, 
-  but its format should be well-defined in the documentation corresponding to the `type` (and `handler`).\
-  A test has the same exact configuration and fork context as the other entries in the `case` list of its `suite`.
-
+- `suite`: a directory containing test cases that are coherent. Each `suite` under the same `handler` shares the same format.
+  This is an organizational/cosmetic hierarchy layer.
+- `case`: a test case, a directory in a `suite`. A case can be anything in general, 
+  but its format should be well-defined in the documentation corresponding to the `type` (and `handler`).
+- `case part`: a test case consists of different files, possibly in different formats, to facilitate the specific test case format better.
+  Optionally, a `meta.yaml` is included to declare meta-data for the test, e.g. BLS requirements. 
 
 ## Test format philosophy
 
@@ -121,10 +119,12 @@ The well known bls/shuffling/ssz_static/operations/epoch_processing/etc. Handler
 ### `<test handler name>/`
 
 Specialization within category. All suites in here will have the same test case format.
+Using a `handler` in a `runner` is optional. A `core` (or other generic) handler may be used if the `runner` does not have different formats.
 
 ### `<test suite name>/`
 
-Suites are split up. Suite size does not change memory bounds, and makes lookups of particular tests fast to find and load.
+Suites are split up. Suite size (i.e. the amount of tests) does not change the maximum memory requirement, as test cases can be loaded one by one.
+This also makes filtered sets of tests fast and easy to load.
 
 ### `<test case>/`
 

specs/test_formats/ssz_static/README.md

@@ -3,6 +3,6 @@
 This set of test-suites provides static testing for SSZ:
  to instantiate just the known Eth 2.0 SSZ types from binary data.
 
-This series of tests is based on the spec-maintained `minimal_ssz.py`, i.e. fully consistent with the SSZ spec. 
+This series of tests is based on the spec-maintained `eth2spec/utils/ssz/ssz_impl.py`, i.e. fully consistent with the SSZ spec. 
 
 Test format documentation can be found here: [core test format](./core.md).

specs/test_formats/ssz_static/core.md

@@ -4,28 +4,51 @@ The goal of this type is to provide clients with a solid reference for how the k
 Each object described in the Phase 0 spec is covered.
 This is important, as many of the clients aiming to serialize/deserialize objects directly into structs/classes
 do not support (or have alternatives for) generic SSZ encoding/decoding.
+
 This test-format ensures these direct serializations are covered.
 
+Note that this test suite does not cover the invalid-encoding case:
+ SSZ implementations should be hardened against invalid inputs with the other SSZ tests as guide, along with fuzzing.
+
 ## Test case format
 
+Each SSZ type is a `handler`, since the format is semantically different: the type of the data is different.
+
+One can iterate over the handlers, and select the type based on the handler name.
+Suites are then the same format, but each specialized in one randomization mode.
+Some randomization modes may only produce a single test case (e.g. the all-zeroes case).
+
+The output parts are: `meta.yaml`, `serialized.ssz`, `value.yaml`
+
+### `meta.yaml`
+
+For non-container SSZ type:
+
 ```yaml
-SomeObjectName:        -- key, object name, formatted as in spec. E.g. "BeaconBlock".
-    value: dynamic     -- the YAML-encoded value, of the type specified by type_name.
-    serialized: bytes  -- string, SSZ-serialized data, hex encoded, with prefix 0x
-    root: bytes32      -- string, hash-tree-root of the value, hex encoded, with prefix 0x
-    signing_root: bytes32 -- string, signing-root of the value, hex encoded, with prefix 0x. Optional, present if type contains ``signature`` field
+root: bytes32         -- string, hash-tree-root of the value, hex encoded, with prefix 0x
+signing_root: bytes32 -- string, signing-root of the value, hex encoded, with prefix 0x. 
+                           Optional, present if type is a container and ends with a ``signature`` field.
 ```
 
+### `serialized.ssz`
+
+The raw encoded bytes.
+
+### `value.yaml`
+
+The same value as `serialized.ssz`, represented as YAML.
+
+
 ## Condition
 
 A test-runner can implement the following assertions:
 - Serialization: After parsing the `value`, SSZ-serialize it: the output should match `serialized`
-- Hash-tree-root: After parsing the `value`, Hash-tree-root it: the output should match `root`
-    - Optionally also check signing-root, if present.
+- Hash-tree-root: After parsing the `value` (or deserializing `serialized`), Hash-tree-root it: the output should match `root`
+    - Optionally also check `signing_root`, if present.
 - Deserialization: SSZ-deserialize the `serialized` value, and see if it matches the parsed `value`
 
-## References
 
+## References
 
 **`serialized`**—[SSZ serialization](../../simple-serialize.md#serialization)   
 **`root`**—[hash_tree_root](../../simple-serialize.md#merkleization) function