status-im
/
eth2.0-specs
mirror of https://github.com/status-im/eth2.0-specs.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
eth2.0-specs/specs/test-format.md

2.3 KiB
Raw Blame History

General test format [WIP]

This document defines the general YAML format to which all tests should conform. Testing specifications in Eth2.0 are still a work in progress. Expect breaking changes

ToC

About

Ethereum 2.0 uses YAML as the format for all cross client tests. This document describes at a high level the general format to which all test files should conform.

The particular formats of specific types of tests (test suites) are defined in separate documents.

YAML fields

title (required)

summary (optional)

test_suite (required) string defining the test suite to which the test cases conform

fork (required) production release versioning

version (required) version for particular test document

test_cases (required) list of test cases each of which is formatted to conform to the test_case standard defined by test_suite. All test cases have optional name and description string fields.

Example test suite

shuffle is a test suite that defines test cases for the shuffle() helper function defined in the beacon-chain spec.

Test cases that conform to the shuffle test suite have the following fields:

  • input (required) the list of items passed into shuffle()
  • output (required) the expected list returned by shuffle()
  • seed (required) the seed of entropy passed into shuffle()

As for all test cases, name and description are optional string fields.

The following is a sample YAML document for the shuffle test suite:

title: Shuffling Algorithm Tests
summary: Test vectors for shuffling a list based upon a seed using `shuffle`
test_suite: shuffle
fork: tchaikovsky
version: 1.0

test_cases:
- input: []
  output: []
  seed: !!binary ""
- name: boring_list
  description: List with a single element, 0
  input: [0]
  output: [0]
  seed: !!binary ""
- input: [255]
  output: [255]
  seed: !!binary ""
- input: [4, 6, 2, 6, 1, 4, 6, 2, 1, 5]
  output: [1, 6, 4, 1, 6, 6, 2, 2, 4, 5]
  seed: !!binary ""
- input: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
  output: [4, 7, 10, 13, 3, 1, 2, 9, 12, 6, 11, 8, 5]
  seed: !!binary ""
- input: [65, 6, 2, 6, 1, 4, 6, 2, 1, 5]
  output: [6, 65, 2, 5, 4, 2, 6, 6, 1, 1]
  seed: !!binary |
    JlAYJ5H2j8g7PLiPHZI/rTS1uAvKiieOrifPN6Moso0=