eth2.0-specs/specs/phase1/custody-game.md

24 KiB

Ethereum 2.0 Phase 1 -- Custody Game

Notice: This document is a work-in-progress for researchers and implementers.

Table of contents

Introduction

This document details the beacon chain additions and changes in Phase 1 of Ethereum 2.0 to support the shard data custody game, building upon the Phase 0 specification.

Constants

Misc

Name Value Unit
CUSTODY_PRIME 2 ** 256 - 189 -
CUSTODY_SECRETS 3 -
BYTES_PER_CUSTODY_ATOM 32 bytes
CUSTODY_PROBABILITY_EXPONENT 10 -

Configuration

Time parameters

Name Value Unit Duration
RANDAO_PENALTY_EPOCHS 2**1 (= 2) epochs 12.8 minutes
EARLY_DERIVED_SECRET_PENALTY_MAX_FUTURE_EPOCHS 2**15 (= 32,768) epochs ~146 days
EPOCHS_PER_CUSTODY_PERIOD 2**14 (= 16,384) epochs ~73 days
CUSTODY_PERIOD_TO_RANDAO_PADDING 2**11 (= 2,048) epochs ~9 days
MAX_CHUNK_CHALLENGE_DELAY 2**15 (= 32,768) epochs ~146 days
CHUNK_RESPONSE_DEADLINE 2**14 (= 16,384) epochs ~73 days

Max operations per block

Name Value
MAX_CUSTODY_CHUNK_CHALLENGE_RECORDS 2**20 (= 1,048,576)
MAX_CUSTODY_KEY_REVEALS 2**8 (= 256)
MAX_EARLY_DERIVED_SECRET_REVEALS 2**0 (= 1)
MAX_CUSTODY_CHUNK_CHALLENGES 2**2 (= 4)
MAX_CUSTODY_CHUNK_CHALLENGE_RESPONSES 2**4 (= 16)
MAX_CUSTODY_SLASHINGS 2**0 (= 1)

Reward and penalty quotients

Name Value
EARLY_DERIVED_SECRET_REVEAL_SLOT_REWARD_MULTIPLE 2**1 (= 2)
MINOR_REWARD_QUOTIENT 2**8 (= 256)

Signature domain types

The following types are defined, mapping into DomainType (little endian):

Name Value
DOMAIN_CUSTODY_BIT_SLASHING DomainType('0x83000000')

Data structures

New Beacon Chain operations

CustodyChunkChallenge

class CustodyChunkChallenge(Container):
    responder_index: ValidatorIndex
    shard_transition: ShardTransition
    attestation: Attestation
    data_index: uint64
    chunk_index: uint64

CustodyChunkChallengeRecord

class CustodyChunkChallengeRecord(Container):
    challenge_index: uint64
    challenger_index: ValidatorIndex
    responder_index: ValidatorIndex
    inclusion_epoch: Epoch
    data_root: Root
    chunk_index: uint64

CustodyChunkResponse

class CustodyChunkResponse(Container):
    challenge_index: uint64
    chunk_index: uint64
    chunk: ByteVector[BYTES_PER_CUSTODY_CHUNK]
    branch: Vector[Root, CUSTODY_RESPONSE_DEPTH]

CustodySlashing

class CustodySlashing(Container):
    # (Attestation.data.shard_transition_root as ShardTransition).shard_data_roots[data_index] is the root of the data.
    data_index: uint64
    malefactor_index: ValidatorIndex
    malefactor_secret: BLSSignature
    whistleblower_index: ValidatorIndex
    shard_transition: ShardTransition
    attestation: Attestation
    data: ByteList[MAX_SHARD_BLOCK_SIZE]

SignedCustodySlashing

class SignedCustodySlashing(Container):
    message: CustodySlashing
    signature: BLSSignature

CustodyKeyReveal

class CustodyKeyReveal(Container):
    # Index of the validator whose key is being revealed
    revealer_index: ValidatorIndex
    # Reveal (masked signature)
    reveal: BLSSignature

EarlyDerivedSecretReveal

Represents an early (punishable) reveal of one of the derived secrets, where derived secrets are RANDAO reveals and custody reveals (both are part of the same domain).

class EarlyDerivedSecretReveal(Container):
    # Index of the validator whose key is being revealed
    revealed_index: ValidatorIndex
    # RANDAO epoch of the key that is being revealed
    epoch: Epoch
    # Reveal (masked signature)
    reveal: BLSSignature
    # Index of the validator who revealed (whistleblower)
    masker_index: ValidatorIndex
    # Mask used to hide the actual reveal signature (prevent reveal from being stolen)
    mask: Bytes32

Helpers

replace_empty_or_append

def replace_empty_or_append(l: List, new_element: Any) -> int:
    for i in range(len(l)):
        if l[i] == type(new_element)():
            l[i] = new_element
            return i
    l.append(new_element)
    return len(l) - 1

legendre_bit

Returns the Legendre symbol (a/q) normalizes as a bit (i.e. ((a/q) + 1) // 2). In a production implementation, a well-optimized library (e.g. GMP) should be used for this.

def legendre_bit(a: int, q: int) -> int:
    if a >= q:
        return legendre_bit(a % q, q)
    if a == 0:
        return 0
    assert(q > a > 0 and q % 2 == 1)
    t = 1
    n = q
    while a != 0:
        while a % 2 == 0:
            a //= 2
            r = n % 8
            if r == 3 or r == 5:
                t = -t
        a, n = n, a
        if a % 4 == n % 4 == 3:
            t = -t
        a %= n
    if n == 1:
        return (t + 1) // 2
    else:
        return 0

get_custody_atoms

Given one set of data, return the custody atoms: each atom will be combined with one legendre bit.

def get_custody_atoms(bytez: bytes) -> Sequence[bytes]:
    bytez += b'\x00' * (-len(bytez) % BYTES_PER_CUSTODY_ATOM)  # right-padding
    return [bytez[i:i + BYTES_PER_CUSTODY_ATOM]
            for i in range(0, len(bytez), BYTES_PER_CUSTODY_ATOM)]

get_custody_secrets

Extract the custody secrets from the signature

def get_custody_secrets(key: BLSSignature) -> Sequence[int]:
    full_G2_element = bls.signature_to_G2(key)
    signature = full_G2_element[0].coeffs
    signature_bytes = b"".join(x.to_bytes(48, "little") for x in signature)
    secrets = [int.from_bytes(signature_bytes[i:i + BYTES_PER_CUSTODY_ATOM], "little")
               for i in range(0, len(signature_bytes), 32)]
    return secrets

universal_hash_function

def universal_hash_function(data_chunks: Sequence[bytes], secrets: Sequence[int]) -> int:
    n = len(data_chunks)
    return (
        sum(
            secrets[i % CUSTODY_SECRETS]**i * int.from_bytes(atom, "little") % CUSTODY_PRIME
            for i, atom in enumerate(data_chunks)
        ) + secrets[n % CUSTODY_SECRETS]**n
    ) % CUSTODY_PRIME

compute_custody_bit

def compute_custody_bit(key: BLSSignature, data: ByteList[MAX_SHARD_BLOCK_SIZE]) -> bit:
    custody_atoms = get_custody_atoms(data)
    secrets = get_custody_secrets(key)
    uhf = universal_hash_function(custody_atoms, secrets)
    legendre_bits = [legendre_bit(uhf + secrets[0] + i, CUSTODY_PRIME) for i in range(CUSTODY_PROBABILITY_EXPONENT)]
    return all(legendre_bits)

get_randao_epoch_for_custody_period

def get_randao_epoch_for_custody_period(period: uint64, validator_index: ValidatorIndex) -> Epoch:
    next_period_start = (period + 1) * EPOCHS_PER_CUSTODY_PERIOD - validator_index % EPOCHS_PER_CUSTODY_PERIOD
    return Epoch(next_period_start + CUSTODY_PERIOD_TO_RANDAO_PADDING)

get_custody_period_for_validator

def get_custody_period_for_validator(validator_index: ValidatorIndex, epoch: Epoch) -> int:
    '''
    Return the reveal period for a given validator.
    '''
    return (epoch + validator_index % EPOCHS_PER_CUSTODY_PERIOD) // EPOCHS_PER_CUSTODY_PERIOD

Per-block processing

Custody Game Operations

def process_custody_game_operations(state: BeaconState, body: BeaconBlockBody) -> None:
    def for_ops(operations: Sequence[Any], fn: Callable[[BeaconState, Any], None]) -> None:
        for operation in operations:
            fn(state, operation)

    for_ops(body.chunk_challenges, process_chunk_challenge)
    for_ops(body.chunk_challenge_responses, process_chunk_challenge)
    for_ops(body.custody_key_reveals, process_custody_key_reveal)
    for_ops(body.early_derived_secret_reveals, process_early_derived_secret_reveal)
    for_ops(body.custody_slashings, process_custody_slashing)

Chunk challenges

def process_chunk_challenge(state: BeaconState, challenge: CustodyChunkChallenge) -> None:
    # Verify the attestation
    assert is_valid_indexed_attestation(state, get_indexed_attestation(state, challenge.attestation))
    # Verify it is not too late to challenge the attestation
    max_attestation_challenge_epoch = challenge.attestation.data.target.epoch + MAX_CHUNK_CHALLENGE_DELAY
    assert get_current_epoch(state) <= max_attestation_challenge_epoch
    # Verify it is not too late to challenge the responder
    responder = state.validators[challenge.responder_index]
    if responder.exit_epoch < FAR_FUTURE_EPOCH:
        assert get_current_epoch(state) <= responder.exit_epoch + MAX_CHUNK_CHALLENGE_DELAY
    # Verify responder is slashable
    assert is_slashable_validator(responder, get_current_epoch(state))
    # Verify the responder participated in the attestation
    attesters = get_attesting_indices(state, challenge.attestation.data, challenge.attestation.aggregation_bits)
    assert challenge.responder_index in attesters
    # Verify shard transition is correctly given
    assert hash_tree_root(challenge.shard_transition) == challenge.attestation.data.shard_transition_root
    data_root = challenge.shard_transition.shard_data_roots[challenge.data_index]
    # Verify the challenge is not a duplicate
    for record in state.custody_chunk_challenge_records:
        assert (
            record.data_root != data_root or
            record.chunk_index != challenge.chunk_index
        )
    # Verify depth
    shard_block_length = challenge.shard_transition.shard_block_lengths[challenge.data_index]
    transition_chunks = (shard_block_length + BYTES_PER_CUSTODY_CHUNK - 1) // BYTES_PER_CUSTODY_CHUNK
    assert challenge.chunk_index < transition_chunks
    # Add new chunk challenge record
    new_record = CustodyChunkChallengeRecord(
        challenge_index=state.custody_chunk_challenge_index,
        challenger_index=get_beacon_proposer_index(state),
        responder_index=challenge.responder_index,
        inclusion_epoch=get_current_epoch(state),
        data_root=challenge.shard_transition.shard_data_roots[challenge.data_index],
        chunk_index=challenge.chunk_index,
    )
    replace_empty_or_append(state.custody_chunk_challenge_records, new_record)

    state.custody_chunk_challenge_index += 1
    # Postpone responder withdrawability
    responder.withdrawable_epoch = FAR_FUTURE_EPOCH

Custody chunk response

def process_chunk_challenge_response(state: BeaconState,
                                     response: CustodyChunkResponse) -> None:
    # Get matching challenge (if any) from records
    matching_challenges = [
        record for record in state.custody_chunk_challenge_records
        if record.challenge_index == response.challenge_index
    ]
    assert len(matching_challenges) == 1
    challenge = matching_challenges[0]
    # Verify chunk index
    assert response.chunk_index == challenge.chunk_index
    # Verify the chunk matches the crosslink data root
    assert is_valid_merkle_branch(
        leaf=hash_tree_root(response.chunk),
        branch=response.branch,
        depth=CUSTODY_RESPONSE_DEPTH,
        index=response.chunk_index,
        root=challenge.data_root,
    )
    # Clear the challenge
    index_in_records = state.custody_chunk_challenge_records.index(challenge)
    state.custody_chunk_challenge_records[index_in_records] = CustodyChunkChallengeRecord()
    # Reward the proposer
    proposer_index = get_beacon_proposer_index(state)
    increase_balance(state, proposer_index, Gwei(get_base_reward(state, proposer_index) // MINOR_REWARD_QUOTIENT))

Custody key reveals

def process_custody_key_reveal(state: BeaconState, reveal: CustodyKeyReveal) -> None:
    """
    Process ``CustodyKeyReveal`` operation.
    Note that this function mutates ``state``.
    """
    revealer = state.validators[reveal.revealer_index]
    epoch_to_sign = get_randao_epoch_for_custody_period(revealer.next_custody_secret_to_reveal, reveal.revealer_index)

    custody_reveal_period = get_custody_period_for_validator(reveal.revealer_index, get_current_epoch(state))
    # Only past custody periods can be revealed, except after exiting the exit period can be revealed
    is_past_reveal = revealer.next_custody_secret_to_reveal < custody_reveal_period
    is_exited = revealer.exit_epoch <= get_current_epoch(state)
    is_exit_period_reveal = (
        revealer.next_custody_secret_to_reveal
        == get_custody_period_for_validator(reveal.revealer_index, revealer.exit_epoch - 1)
    )
    assert is_past_reveal or (is_exited and is_exit_period_reveal)

    # Revealed validator is active or exited, but not withdrawn
    assert is_slashable_validator(revealer, get_current_epoch(state))

    # Verify signature
    domain = get_domain(state, DOMAIN_RANDAO, epoch_to_sign)
    signing_root = compute_signing_root(epoch_to_sign, domain)
    assert bls.Verify(revealer.pubkey, signing_root, reveal.reveal)

    # Process reveal
    if is_exited and is_exit_period_reveal:
        revealer.all_custody_secrets_revealed_epoch = get_current_epoch(state)
    revealer.next_custody_secret_to_reveal += 1

    # Reward Block Proposer
    proposer_index = get_beacon_proposer_index(state)
    increase_balance(
        state,
        proposer_index,
        Gwei(get_base_reward(state, reveal.revealer_index) // MINOR_REWARD_QUOTIENT)
    )

Early derived secret reveals

def process_early_derived_secret_reveal(state: BeaconState, reveal: EarlyDerivedSecretReveal) -> None:
    """
    Process ``EarlyDerivedSecretReveal`` operation.
    Note that this function mutates ``state``.
    """
    revealed_validator = state.validators[reveal.revealed_index]
    derived_secret_location = reveal.epoch % EARLY_DERIVED_SECRET_PENALTY_MAX_FUTURE_EPOCHS

    assert reveal.epoch >= get_current_epoch(state) + RANDAO_PENALTY_EPOCHS
    assert reveal.epoch < get_current_epoch(state) + EARLY_DERIVED_SECRET_PENALTY_MAX_FUTURE_EPOCHS
    assert not revealed_validator.slashed
    assert reveal.revealed_index not in state.exposed_derived_secrets[derived_secret_location]

    # Verify signature correctness
    masker = state.validators[reveal.masker_index]
    pubkeys = [revealed_validator.pubkey, masker.pubkey]

    domain = get_domain(state, DOMAIN_RANDAO, reveal.epoch)
    signing_roots = [compute_signing_root(root, domain) for root in [hash_tree_root(reveal.epoch), reveal.mask]]
    assert bls.AggregateVerify(pubkeys, signing_roots, reveal.reveal)

    if reveal.epoch >= get_current_epoch(state) + CUSTODY_PERIOD_TO_RANDAO_PADDING:
        # Full slashing when the secret was revealed so early it may be a valid custody
        # round key
        slash_validator(state, reveal.revealed_index, reveal.masker_index)
    else:
        # Only a small penalty proportional to proposer slot reward for RANDAO reveal
        # that does not interfere with the custody period
        # The penalty is proportional to the max proposer reward

        # Calculate penalty
        max_proposer_slot_reward = (
            get_base_reward(state, reveal.revealed_index)
            * SLOTS_PER_EPOCH
            // len(get_active_validator_indices(state, get_current_epoch(state)))
            // PROPOSER_REWARD_QUOTIENT
        )
        penalty = Gwei(
            max_proposer_slot_reward
            * EARLY_DERIVED_SECRET_REVEAL_SLOT_REWARD_MULTIPLE
            * (len(state.exposed_derived_secrets[derived_secret_location]) + 1)
        )

        # Apply penalty
        proposer_index = get_beacon_proposer_index(state)
        whistleblower_index = reveal.masker_index
        whistleblowing_reward = Gwei(penalty // WHISTLEBLOWER_REWARD_QUOTIENT)
        proposer_reward = Gwei(whistleblowing_reward // PROPOSER_REWARD_QUOTIENT)
        increase_balance(state, proposer_index, proposer_reward)
        increase_balance(state, whistleblower_index, whistleblowing_reward - proposer_reward)
        decrease_balance(state, reveal.revealed_index, penalty)

        # Mark this derived secret as exposed so validator cannot be punished repeatedly
        state.exposed_derived_secrets[derived_secret_location].append(reveal.revealed_index)

Custody Slashings

def process_custody_slashing(state: BeaconState, signed_custody_slashing: SignedCustodySlashing) -> None:
    custody_slashing = signed_custody_slashing.message
    attestation = custody_slashing.attestation

    # Any signed custody-slashing should result in at least one slashing.
    # If the custody bits are valid, then the claim itself is slashed.
    malefactor = state.validators[custody_slashing.malefactor_index] 
    whistleblower = state.validators[custody_slashing.whistleblower_index]
    domain = get_domain(state, DOMAIN_CUSTODY_BIT_SLASHING, get_current_epoch(state))
    signing_root = compute_signing_root(custody_slashing, domain)
    assert bls.Verify(whistleblower.pubkey, signing_root, signed_custody_slashing.signature)
    # Verify that the whistleblower is slashable
    assert is_slashable_validator(whistleblower, get_current_epoch(state))
    # Verify that the claimed malefactor is slashable
    assert is_slashable_validator(malefactor, get_current_epoch(state))

    # Verify the attestation
    assert is_valid_indexed_attestation(state, get_indexed_attestation(state, attestation))

    # TODO: can do a single combined merkle proof of data being attested.
    # Verify the shard transition is indeed attested by the attestation
    shard_transition = custody_slashing.shard_transition
    assert hash_tree_root(shard_transition) == attestation.data.shard_transition_root
    # Verify that the provided data matches the shard-transition
    assert (
        custody_slashing.data.get_backing().get_left().merkle_root()
        == shard_transition.shard_data_roots[custody_slashing.data_index]
    )
    assert len(custody_slashing.data) == shard_transition.shard_block_lengths[custody_slashing.data_index]

    # Verify existence and participation of claimed malefactor
    attesters = get_attesting_indices(state, attestation.data, attestation.aggregation_bits)
    assert custody_slashing.malefactor_index in attesters
    
    # Verify the malefactor custody key
    epoch_to_sign = get_randao_epoch_for_custody_period(
        get_custody_period_for_validator(custody_slashing.malefactor_index, attestation.data.target.epoch),
        custody_slashing.malefactor_index,
    )
    domain = get_domain(state, DOMAIN_RANDAO, epoch_to_sign)
    signing_root = compute_signing_root(epoch_to_sign, domain)
    assert bls.Verify(malefactor.pubkey, signing_root, custody_slashing.malefactor_secret)

    # Compute the custody bit
    computed_custody_bit = compute_custody_bit(custody_slashing.malefactor_secret, custody_slashing.data)

    # Verify the claim
    if computed_custody_bit == 1:
        # Slash the malefactor, reward the other committee members
        slash_validator(state, custody_slashing.malefactor_index)
        committee = get_beacon_committee(state, attestation.data.slot, attestation.data.index)
        others_count = len(committee) - 1
        whistleblower_reward = Gwei(malefactor.effective_balance // WHISTLEBLOWER_REWARD_QUOTIENT // others_count)
        for attester_index in attesters:
            if attester_index != custody_slashing.malefactor_index:
                increase_balance(state, attester_index, whistleblower_reward)
        # No special whisteblower reward: it is expected to be an attester. Others are free to slash too however. 
    else:
        # The claim was false, the custody bit was correct. Slash the whistleblower that induced this work.
        slash_validator(state, custody_slashing.whistleblower_index)

Per-epoch processing

Handling of reveal deadlines

def process_reveal_deadlines(state: BeaconState) -> None:
    epoch = get_current_epoch(state)
    for index, validator in enumerate(state.validators):
        deadline = validator.next_custody_secret_to_reveal + 1
        if get_custody_period_for_validator(ValidatorIndex(index), epoch) > deadline:
            slash_validator(state, ValidatorIndex(index))
def process_challenge_deadlines(state: BeaconState) -> None:
    for custody_chunk_challenge in state.custody_chunk_challenge_records:
        if get_current_epoch(state) > custody_chunk_challenge.inclusion_epoch + EPOCHS_PER_CUSTODY_PERIOD:
            slash_validator(state, custody_chunk_challenge.responder_index, custody_chunk_challenge.challenger_index)
            index_in_records = state.custody_chunk_challenge_records.index(custody_chunk_challenge)
            state.custody_chunk_challenge_records[index_in_records] = CustodyChunkChallengeRecord()

Final updates

def process_custody_final_updates(state: BeaconState) -> None:
    # Clean up exposed RANDAO key reveals
    state.exposed_derived_secrets[get_current_epoch(state) % EARLY_DERIVED_SECRET_PENALTY_MAX_FUTURE_EPOCHS] = []

    # Reset withdrawable epochs if challenge records are empty
    records = state.custody_chunk_challenge_records
    validator_indices_in_records = set([record.responder_index for record in records])
    for index, validator in enumerate(state.validators):
        if validator.exit_epoch != FAR_FUTURE_EPOCH:
            not_all_secrets_are_revealed = validator.all_custody_secrets_revealed_epoch == FAR_FUTURE_EPOCH
            if index in validator_indices_in_records or not_all_secrets_are_revealed:
                # Delay withdrawable epochs if challenge records are not empty or not all
                # custody secrets revealed
                validator.withdrawable_epoch = FAR_FUTURE_EPOCH
            else:
                # Reset withdrawable epochs if challenge records are empty and all secrets are revealed
                if validator.withdrawable_epoch == FAR_FUTURE_EPOCH:
                    validator.withdrawable_epoch = Epoch(validator.all_custody_secrets_revealed_epoch
                                                         + MIN_VALIDATOR_WITHDRAWABILITY_DELAY)