# General test format [WIP]

This document defines the general YAML format to which all tests should conform. Testing specifications in Eth2.0 are still a work in progress. _Expect breaking changes_

## ToC

* [About](#about)
* [YAML Fields](#yaml-fields)
* [Example test suite](#example-test-suite)

## About
Ethereum 2.0 uses YAML as the format for all cross client tests. This document describes at a high level the general format to which all test files should conform.

The particular formats of specific types of tests (test suites) are defined in separate documents.

## YAML fields
`title` _(required)_

`summary` _(optional)_

`test_suite` _(required)_ string defining the test suite to which the test cases conform

`fork` _(required)_ production release versioning

`version` _(required)_ version for particular test document

`test_cases` _(required)_ list of test cases each of which is formatted to conform to the `test_case` standard defined by `test_suite`. All test cases have optional `name` and `description` string fields.

## Example test suite
`shuffle` is a test suite that defines test cases for the `shuffle()` helper function defined in the `beacon-chain` spec.

Test cases that conform to the `shuffle` test suite have the following fields:

* `input` _(required)_ the list of items passed into `shuffle()`
* `output` _(required)_ the expected list returned by `shuffle()`
* `seed` _(required)_ the seed of entropy passed into `shuffle()`

As for all test cases, `name` and `description` are optional string fields.

The following is a sample YAML document for the `shuffle` test suite:

```yaml
title: Shuffling Algorithm Tests
summary: Test vectors for shuffling a list based upon a seed using `shuffle`
test_suite: shuffle
fork: tchaikovsky
version: 1.0

test_cases:
- input: []
  output: []
  seed: !!binary ""
- name: boring_list
  description: List with a single element, 0
  input: [0]
  output: [0]
  seed: !!binary ""
- input: [255]
  output: [255]
  seed: !!binary ""
- input: [4, 6, 2, 6, 1, 4, 6, 2, 1, 5]
  output: [1, 6, 4, 1, 6, 6, 2, 2, 4, 5]
  seed: !!binary ""
- input: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
  output: [4, 7, 10, 13, 3, 1, 2, 9, 12, 6, 11, 8, 5]
  seed: !!binary ""
- input: [65, 6, 2, 6, 1, 4, 6, 2, 1, 5]
  output: [6, 65, 2, 5, 4, 2, 6, 6, 1, 1]
  seed: !!binary |
    JlAYJ5H2j8g7PLiPHZI/rTS1uAvKiieOrifPN6Moso0=
```