Attestation changes + persistent committee changes (#1294)

* Minimal attestation simplification

* minor fix

* Make the tests pass

* Decrease `PLACEHOLDER`, Use `compute_epoch_of_shard_slot`

* Fix proposer signature name and use get_seed() to calculate current_shuffling_seed

* Fix linter error

* Add the WIP `test_is_valid_shard_block`

* Add `get_shard_block_attester_committee`

* Simplified committee selection

* Added some helpers and simplified

* Update specs/core/1_shard-data-chains.md

* Update 1_shard-data-chains.md

* Simplified switchover epochs, changed block structure, changed crosslink structure

* Update 1_shard-data-chains.md

* Moved balance dependency to proposer selection

* Update specs/core/1_shard-data-chains.md

Co-Authored-By: Danny Ryan <dannyjryan@gmail.com>

* Update specs/core/1_shard-data-chains.md

Co-Authored-By: Danny Ryan <dannyjryan@gmail.com>

* Update specs/core/1_shard-data-chains.md

Co-Authored-By: Danny Ryan <dannyjryan@gmail.com>

* Update specs/core/1_shard-data-chains.md

Co-Authored-By: Danny Ryan <dannyjryan@gmail.com>

* Update specs/core/1_shard-data-chains.md

Co-Authored-By: Danny Ryan <dannyjryan@gmail.com>

* Update specs/core/1_shard-data-chains.md

Co-Authored-By: Danny Ryan <dannyjryan@gmail.com>

* Update specs/core/1_shard-data-chains.md

Co-Authored-By: Danny Ryan <dannyjryan@gmail.com>

* Update specs/core/1_shard-data-chains.md

* Fixed shard header flattening

* Update specs/core/1_shard-data-chains.md

* Minor fixes

* Update specs/core/1_shard-data-chains.md

* Update specs/core/1_shard-data-chains.md

Co-Authored-By: Hsiao-Wei Wang <hwwang156@gmail.com>

* cleanup testing and lint

* return none if not active validators in persistent committee

* only allow active validators as shard proposer

scripts/build_spec.py

@@ -48,11 +48,10 @@ from dataclasses import (
 from eth2spec.utils.ssz.ssz_impl import (
     hash_tree_root,
     signing_root,
-    serialize,
     is_empty,
 )
 from eth2spec.utils.ssz.ssz_typing import (
-    bit, boolean, Container, List, Vector, Bytes, uint64,
+    uint64, bit, boolean, Container, List, Vector, Bytes, BytesN,
     Bytes1, Bytes4, Bytes8, Bytes32, Bytes48, Bytes96, Bitlist, Bitvector,
 )
 from eth2spec.utils.bls import (

specs/core/1_shard-data-chains.md

@@ -9,6 +9,7 @@
 - [Ethereum 2.0 Phase 1 -- Shard Data Chains](#ethereum-20-phase-1----shard-data-chains)
     - [Table of contents](#table-of-contents)
     - [Introduction](#introduction)
+    - [Custom types](#custom-types)
     - [Configuration](#configuration)
         - [Misc](#misc)
         - [Initial values](#initial-values)
@@ -16,21 +17,25 @@
         - [Signature domain types](#signature-domain-types)
         - [TODO PLACEHOLDER](#todo-placeholder)
     - [Data structures](#data-structures)
-        - [`ShardBlockBody`](#shardblockbody)
-        - [`ShardAttestation`](#shardattestation)
-        - [`ShardBlock`](#shardblock)
         - [`ShardBlockHeader`](#shardblockheader)
+        - [`ShardBlock`](#shardblock)
+        - [`ShardBlockSignatures`](#shardblocksignatures)
+        - [`ShardBlockCore`](#shardblockcore)
+        - [`ExtendedShardBlockCore`](#extendedshardblockcore)
     - [Helper functions](#helper-functions)
+        - [`compute_epoch_of_shard_slot`](#compute_epoch_of_shard_slot)
+        - [`compute_slot_of_shard_slot`](#compute_slot_of_shard_slot)
+        - [`get_shard_period_start_epoch`](#get_shard_period_start_epoch)
         - [`get_period_committee`](#get_period_committee)
-        - [`get_switchover_epoch`](#get_switchover_epoch)
         - [`get_persistent_committee`](#get_persistent_committee)
-        - [`get_shard_proposer_index`](#get_shard_proposer_index)
+        - [`get_shard_block_proposer_index`](#get_shard_block_proposer_index)
+        - [`get_shard_block_attester_committee`](#get_shard_block_attester_committee)
         - [`get_shard_header`](#get_shard_header)
-        - [`verify_shard_attestation_signature`](#verify_shard_attestation_signature)
+        - [`pad`](#pad)
+        - [`flatten_shard_header`](#flatten_shard_header)
         - [`compute_crosslink_data_root`](#compute_crosslink_data_root)
     - [Object validity](#object-validity)
         - [Shard blocks](#shard-blocks)
-        - [Shard attestations](#shard-attestations)
         - [Beacon attestations](#beacon-attestations)
     - [Shard fork choice rule](#shard-fork-choice-rule)
 
@@ -40,14 +45,24 @@
 
 This document describes the shard data layer and the shard fork choice rule in Phase 1 of Ethereum 2.0.
 
+## Custom types
+
+We define the following Python custom types for type hinting and readability:
+
+| Name | SSZ equivalent | Description |
+| - | - | - |
+| `ShardSlot` | `uint64` | a slot number in shard chain |
+
 ## Configuration
 
 ### Misc
 
 | Name | Value |
 | - | - |
-| `BYTES_PER_SHARD_BLOCK_BODY` | `2**14` (= 16,384) |
+| `SHARD_HEADER_SIZE` | `2**9` (= 512) |
-| `MAX_SHARD_ATTESTIONS` | `2**4` (= 16) |
+| `SHARD_BLOCK_SIZE_LIMIT` | `2**16` (= 65,536) |
+| `SHARD_SLOTS_PER_BEACON_SLOT` | `2**1` (= 2) |
+| `MAX_PERSISTENT_COMMITTEE_SIZE` | `2**7` (= 128) |
 
 ### Initial values
 
@@ -61,7 +76,8 @@ This document describes the shard data layer and the shard fork choice rule in P
 
 | Name | Value | Unit | Duration |
 | - | - | :-: | :-: |
-| `CROSSLINK_LOOKBACK` | `2**0` (= 1) | epochs  | 6.2 minutes |
+| `CROSSLINK_LOOKBACK` | `2**0` (= 1) | epochs | 6.4 minutes |
+| `EPOCHS_PER_SHARD_PERIOD` | `2**8` (= 256) | epochs | ~27 hours |
 
 ### Signature domain types
 
@@ -76,85 +92,102 @@ The following types are defined, mapping into `DomainType` (little endian):
 
 | Name | Value |
 | - | - |
-| `PLACEHOLDER` | `2**32` |
+| `PLACEHOLDER` | `2**3` |
 
 ## Data structures
 
-### `ShardBlockBody`
+_Note: the shard block header structure is carefully designed so that all of the values have the same depth in a hash tree implementation, so `hash_tree_root(SSZ_partial(x)) == hash_tree_root(x)` (using the "left-to-right leaves" scheme [here](https://github.com/ethereum/eth2.0-specs/issues/1303)), which allows shard block headers to look like an SSZ object when in the crosslink structure. This is done by balancing it so that 7 or 8 items are on the left side (the "core") and two 96-byte (ie. 3*2 = 6 chunk) items are on the right side. Change with care._
+
+### `ShardBlockHeader`
 
 ```python
-class ShardBlockBody(Container):
+class ShardBlockHeader(Container):
-    data: Vector[Bytes[PLACEHOLDER], BYTES_PER_SHARD_BLOCK_BODY]
+    core: ShardBlockCore
-```
+    signatures: ShardBlockSignatures
-
-### `ShardAttestation`
-
-```python
-class ShardAttestation(Container):
-    class data(Container):
-        slot: Slot
-        shard: Shard
-        shard_block_root: Hash
-    aggregation_bits: Bitlist[PLACEHOLDER]
-    aggregate_signature: BLSSignature
 ```
 
 ### `ShardBlock`
 
 ```python
 class ShardBlock(Container):
-    slot: Slot
+    core: ExtendedShardBlockCore
-    shard: Shard
+    signatures: ShardBlockSignatures
-    beacon_chain_root: Hash
-    parent_root: Hash
-    data: ShardBlockBody
-    state_root: Hash
-    attestations: List[ShardAttestation, PLACEHOLDER]
-    signature: BLSSignature
 ```
 
-### `ShardBlockHeader`
+### `ShardBlockSignatures`
 
 ```python
-class ShardBlockHeader(Container):
+class ShardBlockSignatures(Container):
-    slot: Slot
+    attestation_signature: BLSSignature
-    shard: Shard
+    proposer_signature: BLSSignature
+```
+
+### `ShardBlockCore`
+
+```python
+class ShardBlockCore(Container):
+    slot: ShardSlot
     beacon_chain_root: Hash
     parent_root: Hash
-    body_root: Hash
+    data_root: Hash
     state_root: Hash
-    attestations: List[ShardAttestation, PLACEHOLDER]
+    total_bytes: uint64
-    signature: BLSSignature
+    attester_bitfield: Bitvector[MAX_PERSISTENT_COMMITTEE_SIZE * 2]
+```
+
+### `ExtendedShardBlockCore`
+
+```python
+class ExtendedShardBlockCore(Container):
+    slot: ShardSlot
+    beacon_chain_root: Hash
+    parent_root: Hash
+    data: Bytes[SHARD_BLOCK_SIZE_LIMIT - SHARD_HEADER_SIZE]
+    state_root: Hash
+    total_bytes: uint64
+    attester_bitfield: Bitvector[MAX_PERSISTENT_COMMITTEE_SIZE * 2]
 ```
 
 ## Helper functions
 
+### `compute_slot_of_shard_slot`
+
+```python
+def compute_slot_of_shard_slot(slot: ShardSlot) -> Epoch:
+    return Epoch(slot // SHARD_SLOTS_PER_BEACON_SLOT)
+```
+
+### `compute_epoch_of_shard_slot`
+
+```python
+def compute_epoch_of_shard_slot(slot: ShardSlot) -> Epoch:
+    return Epoch(slot // SHARD_SLOTS_PER_BEACON_SLOT // SLOTS_PER_EPOCH)
+```
+
+### `get_shard_period_start_epoch`
+
+```python
+def get_shard_period_start_epoch(epoch: Epoch, lookback: Epoch=Epoch(0)) -> Epoch:
+    return Epoch(epoch - (epoch % EPOCHS_PER_SHARD_PERIOD) - lookback * EPOCHS_PER_SHARD_PERIOD)
+```
+
 ### `get_period_committee`
 
 ```python
 def get_period_committee(state: BeaconState,
                          epoch: Epoch,
-                         shard: Shard,
+                         shard: Shard) -> List[ValidatorIndex, MAX_PERSISTENT_COMMITTEE_SIZE]:
-                         index: uint64,
-                         count: uint64) -> Sequence[ValidatorIndex]:
     """
     Return committee for a period. Used to construct persistent committees.
     """
-    return compute_committee(
+    full_committee = compute_committee(
         indices=get_active_validator_indices(state, epoch),
         seed=get_seed(state, epoch),
-        index=shard * count + index,
+        index=shard,
-        count=SHARD_COUNT * count,
+        count=SHARD_COUNT,
     )
-```
 
-### `get_switchover_epoch`
+    return full_committee[:MAX_PERSISTENT_COMMITTEE_SIZE]
-
-```python
-def get_switchover_epoch(state: BeaconState, epoch: Epoch, index: ValidatorIndex) -> int:
-    earlier_start_epoch = Epoch(epoch - (epoch % PERSISTENT_COMMITTEE_PERIOD) - PERSISTENT_COMMITTEE_PERIOD * 2)
-    return (bytes_to_int(hash(get_seed(state, earlier_start_epoch) + int_to_bytes(index, length=3)[0:8]))
-            % PERSISTENT_COMMITTEE_PERIOD)
 ```
 
 ### `get_persistent_committee`
@@ -162,52 +195,47 @@ def get_switchover_epoch(state: BeaconState, epoch: Epoch, index: ValidatorIndex
 ```python
 def get_persistent_committee(state: BeaconState,
                              shard: Shard,
-                             slot: Slot) -> Sequence[ValidatorIndex]:
+                             slot: ShardSlot) -> Sequence[ValidatorIndex]:
     """
     Return the persistent committee for the given ``shard`` at the given ``slot``.
     """
-    epoch = compute_epoch_of_slot(slot)
+    epoch = compute_epoch_of_shard_slot(slot)
-    earlier_start_epoch = Epoch(epoch - (epoch % PERSISTENT_COMMITTEE_PERIOD) - PERSISTENT_COMMITTEE_PERIOD * 2)
-    later_start_epoch = Epoch(epoch - (epoch % PERSISTENT_COMMITTEE_PERIOD) - PERSISTENT_COMMITTEE_PERIOD)
 
-    committee_count = max(
+    earlier_committee = get_period_committee(state, get_shard_period_start_epoch(epoch, lookback=Epoch(2)), shard)
-        len(get_active_validator_indices(state, earlier_start_epoch)) //
+    later_committee = get_period_committee(state, get_shard_period_start_epoch(epoch, lookback=Epoch(1)), shard)
-        (SHARD_COUNT * TARGET_COMMITTEE_SIZE),
-        len(get_active_validator_indices(state, later_start_epoch)) //
-        (SHARD_COUNT * TARGET_COMMITTEE_SIZE),
-    ) + 1
-
-    index = slot % committee_count
-    earlier_committee = get_period_committee(state, earlier_start_epoch, shard, index, committee_count)
-    later_committee = get_period_committee(state, later_start_epoch, shard, index, committee_count)
 
     # Take not-yet-cycled-out validators from earlier committee and already-cycled-in validators from
     # later committee; return a sorted list of the union of the two, deduplicated
-    return sorted(list(set(
+    return sorted(set(
-        [i for i in earlier_committee if epoch % PERSISTENT_COMMITTEE_PERIOD < get_switchover_epoch(state, epoch, i)]
+        [i for i in earlier_committee if epoch % EPOCHS_PER_SHARD_PERIOD < i % EPOCHS_PER_SHARD_PERIOD]
-        + [i for i in later_committee if epoch % PERSISTENT_COMMITTEE_PERIOD >= get_switchover_epoch(state, epoch, i)]
+        + [i for i in later_committee if epoch % EPOCHS_PER_SHARD_PERIOD >= i % EPOCHS_PER_SHARD_PERIOD]
-    )))
+    ))
 ```
 
-### `get_shard_proposer_index`
+### `get_shard_block_proposer_index`
 
 ```python
-def get_shard_proposer_index(state: BeaconState,
+def get_shard_block_proposer_index(state: BeaconState,
                                    shard: Shard,
-                             slot: Slot) -> Optional[ValidatorIndex]:
+                                   slot: ShardSlot) -> Optional[ValidatorIndex]:
     # Randomly shift persistent committee
     persistent_committee = list(get_persistent_committee(state, shard, slot))
-    seed = hash(state.current_shuffling_seed + int_to_bytes(shard, length=8) + int_to_bytes(slot, length=8))
+    current_epoch = get_current_epoch(state)
-    random_index = bytes_to_int(seed[0:8]) % len(persistent_committee)
-    persistent_committee = persistent_committee[random_index:] + persistent_committee[:random_index]
 
-    # Search for an active proposer
+    active_indices = [i for i in persistent_committee if is_active_validator(state.validators[i], current_epoch)]
-    for index in persistent_committee:
+    if not any(active_indices):
-        if is_active_validator(state.validators[index], get_current_epoch(state)):
-            return index
-
-    # No block can be proposed if no validator is active
         return None
+
+    MAX_RANDOM_BYTE = 2**8 - 1
+    seed = hash(get_seed(state, current_epoch) + int_to_bytes(shard, length=8) + int_to_bytes(slot, length=8))
+    i = 0
+    while True:
+        candidate_index = active_indices[(slot + i) % len(active_indices)]
+        random_byte = hash(seed + int_to_bytes(i // 32, length=8))[i % 32]
+        effective_balance = state.validators[candidate_index].effective_balance
+        if effective_balance * MAX_RANDOM_BYTE >= MAX_EFFECTIVE_BALANCE * random_byte:
+            return ValidatorIndex(candidate_index)
+        i += 1
 ```
 
 ### `get_shard_header`
@@ -215,35 +243,49 @@ def get_shard_proposer_index(state: BeaconState,
 ```python
 def get_shard_header(block: ShardBlock) -> ShardBlockHeader:
     return ShardBlockHeader(
-        slot=block.slot,
+        core=ShardBlockCore(
-        shard=block.shard,
+            slot=block.core.slot,
-        beacon_chain_root=block.beacon_chain_root,
+            beacon_chain_root=block.core.beacon_chain_root,
-        parent_root=block.parent_root,
+            parent_root=block.core.parent_root,
-        body_root=hash_tree_root(block.body),
+            data_root=hash_tree_root(block.core.data),
-        state_root=block.state_root,
+            state_root=block.core.state_root,
-        attestations=block.attestations,
+            total_bytes=block.core.total_bytes,
-        signature=block.signature,
+            attester_bitfield=block.core.attester_bitfield
+        ),
+        signatures=block.signatures
     )
 ```
 
-### `verify_shard_attestation_signature`
+### `pad`
 
 ```python
-def verify_shard_attestation_signature(state: BeaconState,
+def pad(x: bytes, length: int) -> bytes:
-                                       attestation: ShardAttestation) -> None:
+    assert len(x) <= length
-    data = attestation.data
+    return x + b'\x00' * (length - len(x))
-    persistent_committee = get_persistent_committee(state, data.shard, data.slot)
+```
-    pubkeys = []
+
-    for i, index in enumerate(persistent_committee):
+### `flatten_shard_header`
-        if attestation.aggregation_bits[i]:
+
-            validator = state.validators[index]
+```python
-            assert is_active_validator(validator, get_current_epoch(state))
+def flatten_shard_header(header: ShardBlockHeader) -> Bytes[SHARD_HEADER_SIZE]:
-            pubkeys.append(validator.pubkey)
+    """
-    assert bls_verify(
+    Converts a shard block header into a flat object with the same hash tree root. Used
-        pubkey=bls_aggregate_pubkeys(pubkeys),
+    in the crosslink construction.
-        message_hash=data.shard_block_root,
+    """
-        signature=attestation.aggregate_signature,
+    committee_size = len(header.core.attester_bitfield)
-        domain=get_domain(state, DOMAIN_SHARD_ATTESTER, compute_epoch_of_slot(data.slot))
+    attester_bits = [header.core.attester_bitfield[i] if i < committee_size else 0 for i in range(256)]
+    attester_bytes = bytes([sum([attester_bits[i + j] << j for j in range(8)]) for i in range(0, 256, 8)])
+    return (
+        pad(int_to_bytes(header.core.slot, length=8), 32) +
+        header.core.beacon_chain_root +
+        header.core.parent_root +
+        header.core.data_root +
+        header.core.state_root +
+        pad(int_to_bytes(header.core.total_bytes, length=8), 32) +
+        attester_bytes +
+        b'\x00' * 32 +
+        pad(header.signatures.attestation_signature, 128) +
+        pad(header.signatures.proposer_signature, 128)
     )
 ```
 
@@ -251,32 +293,10 @@ def verify_shard_attestation_signature(state: BeaconState,
 
 ```python
 def compute_crosslink_data_root(blocks: Sequence[ShardBlock]) -> Hash:
-    def is_power_of_two(value: uint64) -> bool:
+    header = b''.join([flatten_shard_header(get_shard_header(block)) for block in blocks])
-        return (value > 0) and (value & (value - 1) == 0)
+    footer = b''.join([block.core.data for block in blocks])
-
+    MAX_SIZE = SHARD_BLOCK_SIZE_LIMIT * SHARD_SLOTS_PER_BEACON_SLOT * SLOTS_PER_EPOCH * MAX_EPOCHS_PER_CROSSLINK
-    def pad_to_power_of_2(values: MutableSequence[bytes]) -> Sequence[bytes]:
+    return hash_tree_root(BytesN[MAX_SIZE](pad(header + footer, MAX_SIZE)))
-        while not is_power_of_two(len(values)):
-            values.append(b'\x00' * BYTES_PER_SHARD_BLOCK_BODY)
-        return values
-
-    def hash_tree_root_of_bytes(data: bytes) -> Hash:
-        return hash_tree_root([data[i:i + 32] for i in range(0, len(data), 32)])
-
-    def zpad(data: bytes, length: uint64) -> bytes:
-        return data + b'\x00' * (length - len(data))
-
-    return hash(
-        # TODO untested code.
-        #  Need to either pass a typed list to hash-tree-root, or merkleize_chunks(values, pad_to=2**x)
-        hash_tree_root(pad_to_power_of_2([
-            hash_tree_root_of_bytes(
-                zpad(serialize(get_shard_header(block)), BYTES_PER_SHARD_BLOCK_BODY)
-            ) for block in blocks
-        ]))
-        + hash_tree_root(pad_to_power_of_2([
-            hash_tree_root_of_bytes(block.body) for block in blocks
-        ]))
-    )
 ```
 
 ## Object validity
@@ -287,12 +307,14 @@ Let:
 
 - `beacon_blocks` be the `BeaconBlock` list such that `beacon_blocks[slot]` is the canonical `BeaconBlock` at slot `slot`
 - `beacon_state` be the canonical `BeaconState` after processing `beacon_blocks[-1]`
+- `shard` is the shard ID
 - `valid_shard_blocks` be the list of valid `ShardBlock`, recursively defined
 - `candidate` be a candidate `ShardBlock` for which validity is to be determined by running `is_valid_shard_block`
 
 ```python
-def is_valid_shard_block(beacon_blocks: Sequence[BeaconBlock],
+def is_valid_shard_block(beacon_state: BeaconState,
-                         beacon_state: BeaconState,
+                         beacon_blocks: Sequence[BeaconBlock],
+                         shard: Shard,
                          valid_shard_blocks: Sequence[ShardBlock],
                          candidate: ShardBlock) -> bool:
     # Check if block is already determined valid
@@ -301,80 +323,56 @@ def is_valid_shard_block(beacon_blocks: Sequence[BeaconBlock],
             return True
 
     # Check slot number
-    assert candidate.slot >= PHASE_1_FORK_SLOT
+    assert compute_slot_of_shard_slot(candidate.core.slot) >= PHASE_1_FORK_SLOT
-
-    # Check shard number
-    assert candidate.shard <= SHARD_COUNT
 
     # Check beacon block
-    beacon_block = beacon_blocks[candidate.slot]
+    beacon_block_slot = compute_start_slot_of_epoch(compute_epoch_of_shard_slot(candidate.core.slot))
-    assert candidate.beacon_block_root == signing_root(beacon_block)
+    beacon_block = beacon_blocks[beacon_block_slot]
-    assert beacon_block.slot <= candidate.slot
+    assert candidate.core.beacon_block_root == signing_root(beacon_block)
+    assert beacon_block.slot <= candidate.core.slot
 
     # Check state root
-    assert candidate.state_root == Hash()  # [to be removed in phase 2]
+    assert candidate.core.state_root == Hash()  # [to be removed in phase 2]
 
     # Check parent block
-    if candidate.slot == PHASE_1_FORK_SLOT:
+    if candidate.core.parent_root != Hash():
-        assert candidate.parent_root == Hash()
-    else:
         parent_block = next(
-            (block for block in valid_shard_blocks if signing_root(block) == candidate.parent_root),
+            (block for block in valid_shard_blocks if hash_tree_root(block.core) == candidate.core.parent_root),
             None
         )
         assert parent_block is not None
-        assert parent_block.shard == candidate.shard
+        assert parent_block.core.slot < candidate.core.slot
-        assert parent_block.slot < candidate.slot
+        parent_beacon_block_slot = compute_start_slot_of_epoch(compute_epoch_of_shard_slot(parent_block.core.slot))
-        assert signing_root(beacon_blocks[parent_block.slot]) == parent_block.beacon_chain_root
+        assert signing_root(beacon_blocks[parent_beacon_block_slot]) == parent_block.core.beacon_chain_root
 
     # Check attestations
-    assert len(candidate.attestations) <= MAX_SHARD_ATTESTIONS
+    attester_committee = get_persistent_committee(beacon_state, shard, block.core.slot)
-    for _, attestation in enumerate(candidate.attestations):
+    pubkeys = []
-        assert max(GENESIS_SHARD_SLOT, candidate.slot - SLOTS_PER_EPOCH) <= attestation.data.slot
+    for i, index in enumerate(attester_committee):
-        assert attestation.data.slot <= candidate.slot - MIN_ATTESTATION_INCLUSION_DELAY
+        if block.core.attester_bitfield[i]:
-        assert attestation.data.crosslink.shard == candidate.shard
+            pubkeys.append(beacon_state.validators[index].pubkey)
-        verify_shard_attestation_signature(beacon_state, attestation)
+    for i in range(len(attester_committee), MAX_PERSISTENT_COMMITTEE_SIZE * 2):
+        assert block.attester_bitfield[i] is False
+    assert bls_verify(
+        pubkey=bls_aggregate_pubkeys(pubkeys),
+        message_hash=candidate.core.parent_root,
+        signature=candidate.signatures.attestation_signature,
+        domain=get_domain(beacon_state, DOMAIN_SHARD_ATTESTER, compute_epoch_of_shard_slot(candidate.core.slot))
+    )
 
-    # Check signature
+    # Check proposer
-    proposer_index = get_shard_proposer_index(beacon_state, candidate.shard, candidate.slot)
+    proposer_index = get_shard_block_proposer_index(beacon_state, shard, candidate.core.slot)
     assert proposer_index is not None
     assert bls_verify(
         pubkey=beacon_state.validators[proposer_index].pubkey,
-        message_hash=signing_root(candidate),
+        message_hash=hash_tree_root(candidate.core),
-        signature=candidate.signature,
+        signature=candidate.signatures.proposer_signature,
-        domain=get_domain(beacon_state, DOMAIN_SHARD_PROPOSER, compute_epoch_of_slot(candidate.slot)),
+        domain=get_domain(beacon_state, DOMAIN_SHARD_PROPOSER, compute_epoch_of_shard_slot(candidate.core.slot)),
     )
 
     return True
 ```
 
-### Shard attestations
-
-Let:
-
-- `valid_shard_blocks` be the list of valid `ShardBlock`
-- `beacon_state` be the canonical `BeaconState`
-- `candidate` be a candidate `ShardAttestation` for which validity is to be determined by running `is_valid_shard_attestation`
-
-```python
-def is_valid_shard_attestation(valid_shard_blocks: Sequence[ShardBlock],
-                               beacon_state: BeaconState,
-                               candidate: ShardAttestation) -> bool:
-    # Check shard block
-    shard_block = next(
-        (block for block in valid_shard_blocks if signing_root(block) == candidate.data.shard_block_root),
-        None,
-    )
-    assert shard_block is not None
-    assert shard_block.slot == candidate.data.slot
-    assert shard_block.shard == candidate.data.shard
-
-    # Check signature
-    verify_shard_attestation_signature(beacon_state, candidate)
-
-    return True
-```
-
 ### Beacon attestations
 
 Let:

test_libs/pyspec/eth2spec/test/helpers/phase1/shard_block.py

@@ -0,0 +1,47 @@
+from eth2spec.test.helpers.keys import privkeys
+from eth2spec.utils.bls import (
+    bls_sign,
+    only_with_bls,
+)
+from eth2spec.utils.ssz.ssz_impl import (
+    signing_root,
+)
+
+
+@only_with_bls()
+def sign_shard_block(spec, state, block, shard, proposer_index=None):
+    if proposer_index is None:
+        proposer_index = spec.get_shard_block_proposer_index(state, shard, block.core.slot)
+
+    privkey = privkeys[proposer_index]
+
+    block.signatures.proposer_signature = bls_sign(
+        message_hash=signing_root(block),
+        privkey=privkey,
+        domain=spec.get_domain(
+            state,
+            spec.DOMAIN_SHARD_PROPOSER,
+            spec.compute_epoch_of_shard_slot(block.core.slot),
+        )
+    )
+
+
+def build_empty_shard_block(spec, state, slot, shard, parent_root, signed=False):
+    if slot is None:
+        slot = state.slot
+    block = spec.ShardBlock(
+        core=spec.ExtendedShardBlockCore(
+            slot=slot,
+            beacon_chain_root=state.block_roots[state.slot % spec.SLOTS_PER_HISTORICAL_ROOT],
+            parent_root=parent_root,
+        ),
+        signatures=spec.ShardBlockSignatures(
+            attestation_signature=b'\x12' * 96,
+            proposer_signature=b'\x25' * 96,
+        )
+    )
+
+    if signed:
+        sign_shard_block(spec, state, block, shard)
+
+    return block

test_libs/pyspec/eth2spec/test/phase_1/shard_data_chain/test_shard_block.py

@@ -0,0 +1,26 @@
+from eth2spec.test.helpers.phase1.shard_block import (
+    build_empty_shard_block,
+)
+from eth2spec.test.context import (
+    with_all_phases_except,
+    spec_state_test,
+    always_bls,
+)
+
+
+@with_all_phases_except(['phase0'])
+@always_bls
+@spec_state_test
+def test_is_valid_shard_block(spec, state):
+    block = build_empty_shard_block(
+        spec,
+        state,
+        slot=spec.Slot(spec.PERSISTENT_COMMITTEE_PERIOD * 100),
+        shard=spec.Shard(1),
+        parent_root=spec.Hash(),
+        signed=True,
+    )
+
+    # TODO: test `is_valid_shard_block`
+
+    yield 'blocks', (block,)