24 KiB

Raw Blame History

Ethereum 2.0 Phase 1 -- Custody Game

Notice: This document is a work-in-progress for researchers and implementers.

Introduction
Constants
- Misc
Configuration
Data structures
- New Beacon Chain operations
Helpers
Per-block processing
- Custody Game Operations
Per-epoch processing
- Handling of reveal deadlines
- Final updates

Introduction

This document details the beacon chain additions and changes in Phase 1 of Ethereum 2.0 to support the shard data custody game, building upon the Phase 0 specification.

Constants

Misc

Name	Value	Unit
`CUSTODY_PRIME`	`int(2 ** 256 - 189)`	-
`CUSTODY_SECRETS`	`uint64(3)`	-
`BYTES_PER_CUSTODY_ATOM`	`uint64(32)`	bytes
`CUSTODY_PROBABILITY_EXPONENT`	`uint64(10)`	-

Configuration

Time parameters

Name	Value	Unit	Duration
`RANDAO_PENALTY_EPOCHS`	`uint64(2**1)` (= 2)	epochs	12.8 minutes
`EARLY_DERIVED_SECRET_PENALTY_MAX_FUTURE_EPOCHS`	`uint64(2**15)` (= 32,768)	epochs	~146 days
`EPOCHS_PER_CUSTODY_PERIOD`	`uint64(2**14)` (= 16,384)	epochs	~73 days
`CUSTODY_PERIOD_TO_RANDAO_PADDING`	`uint64(2**11)` (= 2,048)	epochs	~9 days
`MAX_CHUNK_CHALLENGE_DELAY`	`uint64(2**15)` (= 32,768)	epochs	~146 days

Max operations per block

Name	Value
`MAX_CUSTODY_CHUNK_CHALLENGE_RECORDS`	`uint64(2**20)` (= 1,048,576)
`MAX_CUSTODY_KEY_REVEALS`	`uint64(2**8)` (= 256)
`MAX_EARLY_DERIVED_SECRET_REVEALS`	`uint64(2**0)` (= 1)
`MAX_CUSTODY_CHUNK_CHALLENGES`	`uint64(2**2)` (= 4)
`MAX_CUSTODY_CHUNK_CHALLENGE_RESPONSES`	`uint64(2**4)` (= 16)
`MAX_CUSTODY_SLASHINGS`	`uint64(2**0)` (= 1)

Reward and penalty quotients

Name	Value
`EARLY_DERIVED_SECRET_REVEAL_SLOT_REWARD_MULTIPLE`	`uint64(2**1)` (= 2)
`MINOR_REWARD_QUOTIENT`	`uint64(2**8)` (= 256)

Data structures

New Beacon Chain operations

`CustodyChunkChallenge`

class CustodyChunkChallenge(Container):
    responder_index: ValidatorIndex
    shard_transition: ShardTransition
    attestation: Attestation
    data_index: uint64
    chunk_index: uint64

`CustodyChunkChallengeRecord`

class CustodyChunkChallengeRecord(Container):
    challenge_index: uint64
    challenger_index: ValidatorIndex
    responder_index: ValidatorIndex
    inclusion_epoch: Epoch
    data_root: Root
    chunk_index: uint64

`CustodyChunkResponse`

class CustodyChunkResponse(Container):
    challenge_index: uint64
    chunk_index: uint64
    chunk: ByteVector[BYTES_PER_CUSTODY_CHUNK]
    branch: Vector[Root, CUSTODY_RESPONSE_DEPTH + 1]

`CustodySlashing`

class CustodySlashing(Container):
    # (Attestation.data.shard_transition_root as ShardTransition).shard_data_roots[data_index] is the root of the data.
    data_index: uint64
    malefactor_index: ValidatorIndex
    malefactor_secret: BLSSignature
    whistleblower_index: ValidatorIndex
    shard_transition: ShardTransition
    attestation: Attestation
    data: ByteList[MAX_SHARD_BLOCK_SIZE]

`SignedCustodySlashing`

class SignedCustodySlashing(Container):
    message: CustodySlashing
    signature: BLSSignature

`CustodyKeyReveal`

class CustodyKeyReveal(Container):
    # Index of the validator whose key is being revealed
    revealer_index: ValidatorIndex
    # Reveal (masked signature)
    reveal: BLSSignature

`EarlyDerivedSecretReveal`

Represents an early (punishable) reveal of one of the derived secrets, where derived secrets are RANDAO reveals and custody reveals (both are part of the same domain).

class EarlyDerivedSecretReveal(Container):
    # Index of the validator whose key is being revealed
    revealed_index: ValidatorIndex
    # RANDAO epoch of the key that is being revealed
    epoch: Epoch
    # Reveal (masked signature)
    reveal: BLSSignature
    # Index of the validator who revealed (whistleblower)
    masker_index: ValidatorIndex
    # Mask used to hide the actual reveal signature (prevent reveal from being stolen)
    mask: Bytes32

Helpers

`replace_empty_or_append`

def replace_empty_or_append(l: List, new_element: Any) -> int:
    for i in range(len(l)):
        if l[i] == type(new_element)():
            l[i] = new_element
            return i
    l.append(new_element)
    return len(l) - 1

`legendre_bit`

Returns the Legendre symbol (a/q) normalizes as a bit (i.e. ((a/q) + 1) // 2). In a production implementation, a well-optimized library (e.g. GMP) should be used for this.

def legendre_bit(a: int, q: int) -> int:
    if a >= q:
        return legendre_bit(a % q, q)
    if a == 0:
        return 0
    assert(q > a > 0 and q % 2 == 1)
    t = 1
    n = q
    while a != 0:
        while a % 2 == 0:
            a //= 2
            r = n % 8
            if r == 3 or r == 5:
                t = -t
        a, n = n, a
        if a % 4 == n % 4 == 3:
            t = -t
        a %= n
    if n == 1:
        return (t + 1) // 2
    else:
        return 0

`get_custody_atoms`

Given one set of data, return the custody atoms: each atom will be combined with one legendre bit.

def get_custody_atoms(bytez: bytes) -> Sequence[bytes]:
    length_remainder = len(bytez) % BYTES_PER_CUSTODY_ATOM
    bytez += b'\x00' * ((BYTES_PER_CUSTODY_ATOM - length_remainder) % BYTES_PER_CUSTODY_ATOM)  # right-padding
    return [
        bytez[i:i + BYTES_PER_CUSTODY_ATOM]
        for i in range(0, len(bytez), BYTES_PER_CUSTODY_ATOM)
    ]

`get_custody_secrets`

Extract the custody secrets from the signature

def get_custody_secrets(key: BLSSignature) -> Sequence[int]:
    full_G2_element = bls.signature_to_G2(key)
    signature = full_G2_element[0].coeffs
    signature_bytes = b"".join(x.to_bytes(48, "little") for x in signature)
    secrets = [int.from_bytes(signature_bytes[i:i + BYTES_PER_CUSTODY_ATOM], "little")
               for i in range(0, len(signature_bytes), 32)]
    return secrets

`universal_hash_function`

def universal_hash_function(data_chunks: Sequence[bytes], secrets: Sequence[int]) -> int:
    n = len(data_chunks)
    return (
        sum(
            secrets[i % CUSTODY_SECRETS]**i * int.from_bytes(atom, "little") % CUSTODY_PRIME
            for i, atom in enumerate(data_chunks)
        ) + secrets[n % CUSTODY_SECRETS]**n
    ) % CUSTODY_PRIME

`compute_custody_bit`

def compute_custody_bit(key: BLSSignature, data: ByteList) -> bit:
    custody_atoms = get_custody_atoms(data)
    secrets = get_custody_secrets(key)
    uhf = universal_hash_function(custody_atoms, secrets)
    legendre_bits = [legendre_bit(uhf + secrets[0] + i, CUSTODY_PRIME) for i in range(CUSTODY_PROBABILITY_EXPONENT)]
    return bit(all(legendre_bits))

`get_randao_epoch_for_custody_period`

def get_randao_epoch_for_custody_period(period: uint64, validator_index: ValidatorIndex) -> Epoch:
    next_period_start = (period + 1) * EPOCHS_PER_CUSTODY_PERIOD - validator_index % EPOCHS_PER_CUSTODY_PERIOD
    return Epoch(next_period_start + CUSTODY_PERIOD_TO_RANDAO_PADDING)

`get_custody_period_for_validator`

def get_custody_period_for_validator(validator_index: ValidatorIndex, epoch: Epoch) -> uint64:
    '''
    Return the reveal period for a given validator.
    '''
    return (epoch + validator_index % EPOCHS_PER_CUSTODY_PERIOD) // EPOCHS_PER_CUSTODY_PERIOD

Per-block processing

Custody Game Operations

def process_custody_game_operations(state: BeaconState, body: BeaconBlockBody) -> None:
    def for_ops(operations: Sequence[Any], fn: Callable[[BeaconState, Any], None]) -> None:
        for operation in operations:
            fn(state, operation)

    for_ops(body.chunk_challenges, process_chunk_challenge)
    for_ops(body.chunk_challenge_responses, process_chunk_challenge_response)
    for_ops(body.custody_key_reveals, process_custody_key_reveal)
    for_ops(body.early_derived_secret_reveals, process_early_derived_secret_reveal)
    for_ops(body.custody_slashings, process_custody_slashing)

Chunk challenges

def process_chunk_challenge(state: BeaconState, challenge: CustodyChunkChallenge) -> None:
    # Verify the attestation
    assert is_valid_indexed_attestation(state, get_indexed_attestation(state, challenge.attestation))
    # Verify it is not too late to challenge the attestation
    max_attestation_challenge_epoch = Epoch(challenge.attestation.data.target.epoch + MAX_CHUNK_CHALLENGE_DELAY)
    assert get_current_epoch(state) <= max_attestation_challenge_epoch
    # Verify it is not too late to challenge the responder
    responder = state.validators[challenge.responder_index]
    if responder.exit_epoch < FAR_FUTURE_EPOCH:
        assert get_current_epoch(state) <= responder.exit_epoch + MAX_CHUNK_CHALLENGE_DELAY
    # Verify responder is slashable
    assert is_slashable_validator(responder, get_current_epoch(state))
    # Verify the responder participated in the attestation
    attesters = get_attesting_indices(state, challenge.attestation.data, challenge.attestation.aggregation_bits)
    assert challenge.responder_index in attesters
    # Verify shard transition is correctly given
    assert hash_tree_root(challenge.shard_transition) == challenge.attestation.data.shard_transition_root
    data_root = challenge.shard_transition.shard_data_roots[challenge.data_index]
    # Verify the challenge is not a duplicate
    for record in state.custody_chunk_challenge_records:
        assert (
            record.data_root != data_root or
            record.chunk_index != challenge.chunk_index
        )
    # Verify depth
    shard_block_length = challenge.shard_transition.shard_block_lengths[challenge.data_index]
    transition_chunks = (shard_block_length + BYTES_PER_CUSTODY_CHUNK - 1) // BYTES_PER_CUSTODY_CHUNK
    assert challenge.chunk_index < transition_chunks
    # Add new chunk challenge record
    new_record = CustodyChunkChallengeRecord(
        challenge_index=state.custody_chunk_challenge_index,
        challenger_index=get_beacon_proposer_index(state),
        responder_index=challenge.responder_index,
        inclusion_epoch=get_current_epoch(state),
        data_root=challenge.shard_transition.shard_data_roots[challenge.data_index],
        chunk_index=challenge.chunk_index,
    )
    replace_empty_or_append(state.custody_chunk_challenge_records, new_record)

    state.custody_chunk_challenge_index += 1
    # Postpone responder withdrawability
    responder.withdrawable_epoch = FAR_FUTURE_EPOCH

Custody chunk response

def process_chunk_challenge_response(state: BeaconState,
                                     response: CustodyChunkResponse) -> None:
    # Get matching challenge (if any) from records
    matching_challenges = [
        record for record in state.custody_chunk_challenge_records
        if record.challenge_index == response.challenge_index
    ]
    assert len(matching_challenges) == 1
    challenge = matching_challenges[0]
    # Verify chunk index
    assert response.chunk_index == challenge.chunk_index
    # Verify the chunk matches the crosslink data root
    assert is_valid_merkle_branch(
        leaf=hash_tree_root(response.chunk),
        branch=response.branch,
        depth=CUSTODY_RESPONSE_DEPTH + 1,  # Add 1 for the List length mix-in
        index=response.chunk_index,
        root=challenge.data_root,
    )
    # Clear the challenge
    index_in_records = state.custody_chunk_challenge_records.index(challenge)
    state.custody_chunk_challenge_records[index_in_records] = CustodyChunkChallengeRecord()
    # Reward the proposer
    proposer_index = get_beacon_proposer_index(state)
    increase_balance(state, proposer_index, Gwei(get_base_reward(state, proposer_index) // MINOR_REWARD_QUOTIENT))

Custody key reveals

def process_custody_key_reveal(state: BeaconState, reveal: CustodyKeyReveal) -> None:
    """
    Process ``CustodyKeyReveal`` operation.
    Note that this function mutates ``state``.
    """
    revealer = state.validators[reveal.revealer_index]
    epoch_to_sign = get_randao_epoch_for_custody_period(revealer.next_custody_secret_to_reveal, reveal.revealer_index)

    custody_reveal_period = get_custody_period_for_validator(reveal.revealer_index, get_current_epoch(state))
    # Only past custody periods can be revealed, except after exiting the exit period can be revealed
    is_past_reveal = revealer.next_custody_secret_to_reveal < custody_reveal_period
    is_exited = revealer.exit_epoch <= get_current_epoch(state)
    is_exit_period_reveal = (
        revealer.next_custody_secret_to_reveal
        == get_custody_period_for_validator(reveal.revealer_index, revealer.exit_epoch - 1)
    )
    assert is_past_reveal or (is_exited and is_exit_period_reveal)

    # Revealed validator is active or exited, but not withdrawn
    assert is_slashable_validator(revealer, get_current_epoch(state))

    # Verify signature
    domain = get_domain(state, DOMAIN_RANDAO, epoch_to_sign)
    signing_root = compute_signing_root(epoch_to_sign, domain)
    assert bls.Verify(revealer.pubkey, signing_root, reveal.reveal)

    # Process reveal
    if is_exited and is_exit_period_reveal:
        revealer.all_custody_secrets_revealed_epoch = get_current_epoch(state)
    revealer.next_custody_secret_to_reveal += 1

    # Reward Block Proposer
    proposer_index = get_beacon_proposer_index(state)
    increase_balance(
        state,
        proposer_index,
        Gwei(get_base_reward(state, reveal.revealer_index) // MINOR_REWARD_QUOTIENT)
    )

Early derived secret reveals

def process_early_derived_secret_reveal(state: BeaconState, reveal: EarlyDerivedSecretReveal) -> None:
    """
    Process ``EarlyDerivedSecretReveal`` operation.
    Note that this function mutates ``state``.
    """
    revealed_validator = state.validators[reveal.revealed_index]
    derived_secret_location = uint64(reveal.epoch % EARLY_DERIVED_SECRET_PENALTY_MAX_FUTURE_EPOCHS)

    assert reveal.epoch >= get_current_epoch(state) + RANDAO_PENALTY_EPOCHS
    assert reveal.epoch < get_current_epoch(state) + EARLY_DERIVED_SECRET_PENALTY_MAX_FUTURE_EPOCHS
    assert not revealed_validator.slashed
    assert reveal.revealed_index not in state.exposed_derived_secrets[derived_secret_location]

    # Verify signature correctness
    masker = state.validators[reveal.masker_index]
    pubkeys = [revealed_validator.pubkey, masker.pubkey]

    domain = get_domain(state, DOMAIN_RANDAO, reveal.epoch)
    signing_roots = [compute_signing_root(root, domain) for root in [hash_tree_root(reveal.epoch), reveal.mask]]
    assert bls.AggregateVerify(pubkeys, signing_roots, reveal.reveal)

    if reveal.epoch >= get_current_epoch(state) + CUSTODY_PERIOD_TO_RANDAO_PADDING:
        # Full slashing when the secret was revealed so early it may be a valid custody
        # round key
        slash_validator(state, reveal.revealed_index, reveal.masker_index)
    else:
        # Only a small penalty proportional to proposer slot reward for RANDAO reveal
        # that does not interfere with the custody period
        # The penalty is proportional to the max proposer reward

        # Calculate penalty
        max_proposer_slot_reward = (
            get_base_reward(state, reveal.revealed_index)
            * SLOTS_PER_EPOCH
            // len(get_active_validator_indices(state, get_current_epoch(state)))
            // PROPOSER_REWARD_QUOTIENT
        )
        penalty = Gwei(
            max_proposer_slot_reward
            * EARLY_DERIVED_SECRET_REVEAL_SLOT_REWARD_MULTIPLE
            * (len(state.exposed_derived_secrets[derived_secret_location]) + 1)
        )

        # Apply penalty
        proposer_index = get_beacon_proposer_index(state)
        whistleblower_index = reveal.masker_index
        whistleblowing_reward = Gwei(penalty // WHISTLEBLOWER_REWARD_QUOTIENT)
        proposer_reward = Gwei(whistleblowing_reward // PROPOSER_REWARD_QUOTIENT)
        increase_balance(state, proposer_index, proposer_reward)
        increase_balance(state, whistleblower_index, whistleblowing_reward - proposer_reward)
        decrease_balance(state, reveal.revealed_index, penalty)

        # Mark this derived secret as exposed so validator cannot be punished repeatedly
        state.exposed_derived_secrets[derived_secret_location].append(reveal.revealed_index)

Custody Slashings

def process_custody_slashing(state: BeaconState, signed_custody_slashing: SignedCustodySlashing) -> None:
    custody_slashing = signed_custody_slashing.message
    attestation = custody_slashing.attestation

    # Any signed custody-slashing should result in at least one slashing.
    # If the custody bits are valid, then the claim itself is slashed.
    malefactor = state.validators[custody_slashing.malefactor_index] 
    whistleblower = state.validators[custody_slashing.whistleblower_index]
    domain = get_domain(state, DOMAIN_CUSTODY_BIT_SLASHING, get_current_epoch(state))
    signing_root = compute_signing_root(custody_slashing, domain)
    assert bls.Verify(whistleblower.pubkey, signing_root, signed_custody_slashing.signature)
    # Verify that the whistleblower is slashable
    assert is_slashable_validator(whistleblower, get_current_epoch(state))
    # Verify that the claimed malefactor is slashable
    assert is_slashable_validator(malefactor, get_current_epoch(state))

    # Verify the attestation
    assert is_valid_indexed_attestation(state, get_indexed_attestation(state, attestation))

    # TODO: can do a single combined merkle proof of data being attested.
    # Verify the shard transition is indeed attested by the attestation
    shard_transition = custody_slashing.shard_transition
    assert hash_tree_root(shard_transition) == attestation.data.shard_transition_root
    # Verify that the provided data matches the shard-transition
    assert len(custody_slashing.data) == shard_transition.shard_block_lengths[custody_slashing.data_index]
    assert hash_tree_root(custody_slashing.data) == shard_transition.shard_data_roots[custody_slashing.data_index]
    # Verify existence and participation of claimed malefactor
    attesters = get_attesting_indices(state, attestation.data, attestation.aggregation_bits)
    assert custody_slashing.malefactor_index in attesters
    
    # Verify the malefactor custody key
    epoch_to_sign = get_randao_epoch_for_custody_period(
        get_custody_period_for_validator(custody_slashing.malefactor_index, attestation.data.target.epoch),
        custody_slashing.malefactor_index,
    )
    domain = get_domain(state, DOMAIN_RANDAO, epoch_to_sign)
    signing_root = compute_signing_root(epoch_to_sign, domain)
    assert bls.Verify(malefactor.pubkey, signing_root, custody_slashing.malefactor_secret)

    # Compute the custody bit
    computed_custody_bit = compute_custody_bit(custody_slashing.malefactor_secret, custody_slashing.data)

    # Verify the claim
    if computed_custody_bit == 1:
        # Slash the malefactor, reward the other committee members
        slash_validator(state, custody_slashing.malefactor_index)
        committee = get_beacon_committee(state, attestation.data.slot, attestation.data.index)
        others_count = len(committee) - 1
        whistleblower_reward = Gwei(malefactor.effective_balance // WHISTLEBLOWER_REWARD_QUOTIENT // others_count)
        for attester_index in attesters:
            if attester_index != custody_slashing.malefactor_index:
                increase_balance(state, attester_index, whistleblower_reward)
        # No special whisteblower reward: it is expected to be an attester. Others are free to slash too however. 
    else:
        # The claim was false, the custody bit was correct. Slash the whistleblower that induced this work.
        slash_validator(state, custody_slashing.whistleblower_index)

Per-epoch processing

Handling of reveal deadlines

def process_reveal_deadlines(state: BeaconState) -> None:
    epoch = get_current_epoch(state)
    for index, validator in enumerate(state.validators):
        deadline = validator.next_custody_secret_to_reveal + 1
        if get_custody_period_for_validator(ValidatorIndex(index), epoch) > deadline:
            slash_validator(state, ValidatorIndex(index))

def process_challenge_deadlines(state: BeaconState) -> None:
    for custody_chunk_challenge in state.custody_chunk_challenge_records:
        if get_current_epoch(state) > custody_chunk_challenge.inclusion_epoch + EPOCHS_PER_CUSTODY_PERIOD:
            slash_validator(state, custody_chunk_challenge.responder_index, custody_chunk_challenge.challenger_index)
            index_in_records = state.custody_chunk_challenge_records.index(custody_chunk_challenge)
            state.custody_chunk_challenge_records[index_in_records] = CustodyChunkChallengeRecord()

Final updates

def process_custody_final_updates(state: BeaconState) -> None:
    # Clean up exposed RANDAO key reveals
    state.exposed_derived_secrets[get_current_epoch(state) % EARLY_DERIVED_SECRET_PENALTY_MAX_FUTURE_EPOCHS] = []

    # Reset withdrawable epochs if challenge records are empty
    records = state.custody_chunk_challenge_records
    validator_indices_in_records = set(record.responder_index for record in records)  # non-duplicate
    for index, validator in enumerate(state.validators):
        if validator.exit_epoch != FAR_FUTURE_EPOCH:
            not_all_secrets_are_revealed = validator.all_custody_secrets_revealed_epoch == FAR_FUTURE_EPOCH
            if ValidatorIndex(index) in validator_indices_in_records or not_all_secrets_are_revealed:
                # Delay withdrawable epochs if challenge records are not empty or not all
                # custody secrets revealed
                validator.withdrawable_epoch = FAR_FUTURE_EPOCH
            else:
                # Reset withdrawable epochs if challenge records are empty and all secrets are revealed
                if validator.withdrawable_epoch == FAR_FUTURE_EPOCH:
                    validator.withdrawable_epoch = Epoch(validator.all_custody_secrets_revealed_epoch
                                                         + MIN_VALIDATOR_WITHDRAWABILITY_DELAY)

24 KiB Raw Blame History

Ethereum 2.0 Phase 1 -- Custody Game

Table of contents