status-im
/
eth2.0-specs
mirror of https://github.com/status-im/eth2.0-specs.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
eth2.0-specs/specs/phase1/beacon-chain.md

25 KiB
Raw Blame History

Ethereum 2.0 Phase 1 -- The Beacon Chain with Shards

Notice: This document is a work-in-progress for researchers and implementers.

Table of contents

Introduction

This document describes the extensions made to the Phase 0 design of The Beacon Chain to support data sharding, based on the ideas here and more broadly here, using Kate commitments to commit to data to remove any need for fraud proofs (and hence, safety-critical synchrony assumptions) in the design.

Custom types

We define the following Python custom types for type hinting and readability:

Name SSZ equivalent Description
Shard uint64 A shard number
BLSCommitment bytes48 A G1 curve point

Configuration

Misc

Name Value Notes
MAX_SHARDS uint64(2**10) (= 1024) Theoretical max shard count (used to determine data structure sizes)
INITIAL_ACTIVE_SHARDS uint64(2**6) (= 64) Initial shard count
GASPRICE_ADJUSTMENT_COEFFICIENT uint64(2**3) (= 8) Gasprice may decrease/increase by at most exp(1 / this value) per epoch
MAX_SHARD_HEADERS_PER_SHARD 4
MAX_SHARD_HEADERS MAX_SHARDS * MAX_SHARD_HEADERS_PER_SHARD
PRIMITIVE_ROOT_OF_UNITY 5 Primitive root of unity of the BLS12_381 (inner) modulus
DATA_AVAILABILITY_INVERSE_CODING_RATE 2**1 (=2) Factor by which samples are extended for data availability encoding

Shard block configs

Name Value Notes
POINTS_PER_SAMPLE uint64(2**3) (= 8) 31 * 8 = 248 bytes
MAX_SAMPLES_PER_BLOCK uint64(2**11) (= 2,048) 248 * 2,048 = 507,904 bytes
TARGET_SAMPLES_PER_BLOCK uint64(2**10) (= 1,024) 248 * 1,024 = 253,952 bytes

Precomputed size verification points

Name Value
G2_SETUP Type List[G2]. The G2-side trusted setup [G, G*s, G*s**2....]; note that the first point is the generator.
ROOT_OF_UNITY pow(PRIMITIVE_ROOT_OF_UNITY, (MODULUS - 1) // (MAX_SAMPLES_PER_BLOCK * POINTS_PER_SAMPLE, MODULUS)
SIZE_CHECK_POINTS Type List[G2, MAX_SAMPLES_PER_BLOCK + 1]; TO BE COMPUTED

These points are the G2-side Kate commitments to product[a in i...next_power_of_two(i)] (X ** POINTS_PER_SAMPLE - w ** (reverse_bit_order(a, MAX_SAMPLES_PER_BLOCK * DATA_AVAILABILITY_INVERSE_CODING_RATE) * POINTS_PER_SAMPLE)) for each i in [0...MAX_SAMPLES_PER_BLOCK], where w = ROOT_OF_UNITY. They are used to verify block size proofs. They can be computed with a one-time O(N**2/log(N)) calculation using fast-linear-combinations in G2.

Gwei values

Name Value Unit Description
MAX_GASPRICE Gwei(2**33) (= 8,589,934,592) Gwei Max gasprice charged for a TARGET-sized shard block
MIN_GASPRICE Gwei(2**3) (= 8) Gwei Min gasprice charged for a TARGET-sized shard block

Time parameters

Name Value Unit Duration
SHARD_COMMITTEE_PERIOD Epoch(2**8) (= 256) epochs ~27 hours

Domain types

Name Value
DOMAIN_SHARD_HEADER DomainType('0x80000000')

Updated containers

The following containers have updated definitions in Phase 1.

AttestationData

class AttestationData(Container):
    slot: Slot
    index: CommitteeIndex
    # LMD GHOST vote
    beacon_block_root: Root
    # FFG vote
    source: Checkpoint
    target: Checkpoint
    # Shard header root
    shard_header_root: Root

BeaconBlock

class BeaconBlock(phase0.BeaconBlock):
    shard_headers: List[Signed[ShardHeader], MAX_SHARD_HEADERS]

BeaconState

class BeaconState(phase0.BeaconState):
    # Updated fields
    previous_epoch_attestations: List[PendingAttestation, MAX_ATTESTATIONS * SLOTS_PER_EPOCH]
    current_epoch_attestations: List[PendingAttestation, MAX_ATTESTATIONS * SLOTS_PER_EPOCH]
    # New fields
    current_epoch_pending_shard_headers: List[PendingShardHeader, MAX_SHARD_HEADERS * SLOTS_PER_EPOCH]
    previous_epoch_pending_shard_headers: List[PendingShardHeader, MAX_SHARD_HEADERS * SLOTS_PER_EPOCH]
    most_recent_confirmed_commitments: Vector[Vector[DataCommitment, SLOTS_PER_EPOCH], MAX_SHARDS]
    shard_gasprice: uint64
    current_epoch_start_shard: Shard

New containers

The following containers are new in Phase 1.

DataCommitment

class DataCommitment(Container):
    # Kate commitment to the data
    point: BLSCommitment
    # Length of the data in samples
    length: length

ShardHeader

class ShardHeader(Container):
    # Slot and shard that this header is intended for
    slot: Slot
    shard: Shard
    # The actual data commitment
    commitment: DataCommitment
    # Proof of the length (more precisely, proof that values at
    # positions >= the length all equal zero)
    length_proof: BLSCommitment

PendingShardHeader

class PendingShardHeader(Container):
    # Slot and shard that this header is intended for
    slot: uint64
    shard: Shard
    # Kate commitment to the data
    commitment: BLSCommitment
    # hash_tree_root of the ShardHeader (stored so that attestations
    # can be checked against it)
    root: Hash
    # Length of the data in samples
    length: uint64
    # Who voted for the header
    votes: Bitlist[MAX_COMMITTEE_SIZE]
    # Has this header been confirmed?
    confirmed: bool

Helper functions

Misc

next_power_of_two

def next_power_of_two(x):
    return 2 ** ((x - 1).bit_length())

reverse_bit_order

def reverse_bit_order(n, order):
    """
    Reverse the bit order of an integer n
    """
    assert is_power_of_two(order)
    return int(('{:0' + str(order.bit_length() - 1) + 'b}').format(n)[::-1], 2)

compute_previous_slot

def compute_previous_slot(slot: Slot) -> Slot:
    if slot > 0:
        return Slot(slot - 1)
    else:
        return Slot(0)

compute_shard_from_committee_index

def compute_shard_from_committee_index(state: BeaconState, index: CommitteeIndex, slot: Slot) -> Shard:
    active_shards = get_active_shard_count(state)
    return Shard((index + get_start_shard(state, slot)) % active_shards)

compute_updated_gasprice

def compute_updated_gasprice(prev_gasprice: Gwei, shard_block_length: uint64, adjustment_quotient: uint64) -> Gwei:
    if shard_block_length > TARGET_SAMPLES_PER_BLOCK:
        delta = max(1, prev_gasprice * (shard_block_length - TARGET_SAMPLES_PER_BLOCK)
                       // TARGET_SAMPLES_PER_BLOCK // adjustment_quotient)
        return min(prev_gasprice + delta, MAX_GASPRICE)
    else:
        delta = max(1, prev_gasprice * (TARGET_SAMPLES_PER_BLOCK - shard_block_length)
                       // TARGET_SAMPLES_PER_BLOCK // adjustment_quotient)
        return max(prev_gasprice, MIN_GASPRICE + delta) - delta

compute_committee_source_epoch

def compute_committee_source_epoch(epoch: Epoch, period: uint64) -> Epoch:
    """
    Return the source epoch for computing the committee.
    """
    source_epoch = Epoch(epoch - epoch % period)
    if source_epoch >= period:
        source_epoch -= period  # `period` epochs lookahead
    return source_epoch

Beacon state accessors

Updated get_committee_count_per_slot

def get_committee_count_per_slot(state: BeaconState, epoch: Epoch) -> uint64:
    """
    Return the number of committees in each slot for the given ``epoch``.
    """
    return max(uint64(1), min(
        get_active_shard_count(state, epoch),
        uint64(len(get_active_validator_indices(state, epoch))) // SLOTS_PER_EPOCH // TARGET_COMMITTEE_SIZE,
    ))

get_active_shard_count

def get_active_shard_count(state: BeaconState, epoch: Epoch) -> uint64:
    """
    Return the number of active shards.
    Note that this puts an upper bound on the number of committees per slot.
    """
    return INITIAL_ACTIVE_SHARDS

get_shard_committee

def get_shard_committee(beacon_state: BeaconState, epoch: Epoch, shard: Shard) -> Sequence[ValidatorIndex]:
    """
    Return the shard committee of the given ``epoch`` of the given ``shard``.
    """
    source_epoch = compute_committee_source_epoch(epoch, SHARD_COMMITTEE_PERIOD)
    active_validator_indices = get_active_validator_indices(beacon_state, source_epoch)
    seed = get_seed(beacon_state, source_epoch, DOMAIN_SHARD_COMMITTEE)
    return compute_committee(
        indices=active_validator_indices,
        seed=seed,
        index=shard,
        count=get_active_shard_count(beacon_state, epoch),
    )

compute_proposer_index

Updated version to get a proposer index that will only allow proposers with a certain minimum balance, ensuring that the balance is always sufficient to cover gas costs.

def compute_proposer_index(state: BeaconState, indices: Sequence[ValidatorIndex], seed: Bytes32, min_effective_balance: GWei = GWei(0)) -> ValidatorIndex:
    """
    Return from ``indices`` a random index sampled by effective balance.
    """
    assert len(indices) > 0
    MAX_RANDOM_BYTE = 2**8 - 1
    i = uint64(0)
    total = uint64(len(indices))
    while True:
        candidate_index = indices[compute_shuffled_index(i % total, total, seed)]
        random_byte = hash(seed + uint_to_bytes(uint64(i // 32)))[i % 32]
        effective_balance = state.validators[candidate_index].effective_balance
        if effective_balance * MAX_RANDOM_BYTE >= MAX_EFFECTIVE_BALANCE * random_byte and effective_balance > min_effective_balance:
            return candidate_index
        i += 1

get_shard_proposer_index

def get_shard_proposer_index(beacon_state: BeaconState, slot: Slot, shard: Shard) -> ValidatorIndex:
    """
    Return the proposer's index of shard block at ``slot``.
    """
    epoch = compute_epoch_at_slot(slot)
    committee = get_shard_committee(beacon_state, epoch, shard)
    seed = hash(get_seed(state, epoch, DOMAIN_BEACON_PROPOSER) + uint_to_bytes(state.slot))
    EFFECTIVE_BALANCE_MAX_DOWNWARD_DEVIATION = EFFECTIVE_BALANCE_INCREMENT - EFFECTIVE_BALANCE_INCREMENT * HYSTERESIS_DOWNWARD_MULTIPLIER // HYSTERESIS_QUOTIENT

    return compute_proposer_index(state, committee, seed,
        state.shard_gasprice * MAX_SAMPLES_PER_BLOCK // TARGET_SAMPLES_PER_BLOCK + EFFECTIVE_BALANCE_MAX_DOWNWARD_DEVIATION)

get_start_shard

def get_start_shard(state: BeaconState, slot: Slot) -> Shard:
    """
    Return the start shard at ``slot``.
    """
    current_epoch_start_slot = compute_start_slot_at_epoch(get_current_epoch(state))
    shard = state.current_epoch_start_shard
    if slot > current_epoch_start_slot:
        # Current epoch or the next epoch lookahead
        for _slot in range(current_epoch_start_slot, slot):
            committee_count = get_committee_count_per_slot(state, compute_epoch_at_slot(Slot(_slot)))
            active_shard_count = get_active_shard_count(state, compute_epoch_at_slot(Slot(_slot)))
            shard = (shard + committee_count) % active_shard_count
        return Shard(shard)
    elif slot < current_epoch_start_slot:
        # Previous epoch
        for _slot in list(range(slot, current_epoch_start_slot))[::-1]:
            committee_count = get_committee_count_per_slot(state, compute_epoch_at_slot(Slot(_slot)))
            active_shard_count = get_active_shard_count(state, compute_epoch_at_slot(Slot(_slot)))
            # Ensure positive
            shard = (shard + active_shard_count - committee_count) % active_shard_count
    return Shard(shard)

Block processing

def process_block(state: BeaconState, block: BeaconBlock) -> None:
    process_block_header(state, block)
    process_randao(state, block.body)
    process_eth1_data(state, block.body)
    process_light_client_aggregate(state, block.body)
    process_operations(state, block.body)

Operations

def process_operations(state: BeaconState, body: BeaconBlockBody) -> None:
    # Verify that outstanding deposits are processed up to the maximum number of deposits
    assert len(body.deposits) == min(MAX_DEPOSITS, state.eth1_data.deposit_count - state.eth1_deposit_index)

    def for_ops(operations: Sequence[Any], fn: Callable[[BeaconState, Any], None]) -> None:
        for operation in operations:
            fn(state, operation)

    for_ops(body.proposer_slashings, process_proposer_slashing)
    for_ops(body.attester_slashings, process_attester_slashing)
    # New attestation processing
    for_ops(body.attestations, process_attestation)
    for_ops(body.deposits, process_deposit)
    for_ops(body.voluntary_exits, process_voluntary_exit)
    # Limit is dynamic based on active shard count
    assert len(body.shard_headers) <= MAX_SHARD_HEADERS_PER_SHARD * get_active_shard_count(state, get_current_epoch(state))
    for_ops(body.shard_headers, process_shard_header)

    # See custody game spec.
    process_custody_game_operations(state, body)

    process_shard_transitions(state, body.shard_transitions, body.attestations)

    # TODO process_operations(body.shard_receipt_proofs, process_shard_receipt_proofs)

New Attestation processing

Updated process_attestation

def process_attestation(state: BeaconState, attestation: Attestation) -> None:
    phase0.process_attestation(state, attestation)
    update_pending_votes(
        state=state,
        attestation: Attestation,
        root=,
        aggregation_bits=attestation.aggregation_bits
    )

update_pending_votes

def update_pending_votes(state: BeaconState,
                         attestation: Attestation) -> None:
    if compute_epoch_at_slot(slot) == get_current_epoch(state):
        pending_headers = state.current_epoch_pending_shard_headers
    else:
        pending_headers = state.previous_epoch_pending_shard_headers
    # Create or update the PendingShardHeader object
    pending_header = None
    for header in pending_headers:
        if header.root == attestation.data.shard_header_root:
            pending_header = header
    assert pending_header is not None
    assert pending_header.slot == attestation.data.slot + 1
    assert pending_header.shard == compute_shard_from_committee_index(
        state,
        attestation.data.index,
        attestation.data.slot
    )
    pending_header.votes = bitwise_or(
        pending_header.votes,
        attestation.aggregation_bits
    )

    # Check if the PendingShardHeader is eligible for expedited confirmation
    # Requirement 1: nothing else confirmed
    all_candidates = [
        c for c in pending_headers if
        (c.slot, c.shard) == (pending_header.slot, pending_header.shard)
    ]
    if True not in [c.confirmed for c in all_candidates]:
        # Requirement 2: >= 2/3 of balance attesting
        participants = get_attesting_indices(state, attestationg.data, pending_commitment.votes)
        participants_balance = get_total_balance(state, participants)
        full_committee = get_beacon_committee(state, attestationg.data.slot, attestationg.data.shard)
        full_committee_balance = get_total_balance(state, full_committee)
        if participants_balance * 3 > full_committee_balance * 2:
            pending_header.confirmed = True

process_shard_header

def process_shard_header(state: BeaconState,
                         signed_header: Signed[ShardDataHeader]) -> None:
    header = signed_header.message
    header_root = hash_tree_root(header)
    # Verify signature
    signer_index = get_shard_proposer_index(state, header.slot, header.shard)
    assert bls.Verify(
        state.validators[signer_index].pubkey,
        compute_signing_root(header, get_domain(state, DOMAIN_SHARD_HEADER)),
        signed_header.signature
    )
    # Verify length of the header, and simultaneously verify degree.
    assert (
        bls.Pairing(header.length_proof, SIZE_CHECK_POINTS[header.commitment.length]) ==
        bls.Pairing(header.commitment.point, G2_SETUP[-header.commitment.length]))
    )
    # Get the correct pending header list
    if compute_epoch_at_slot(header.slot) == get_current_epoch(state):
        pending_headers = state.current_epoch_pending_shard_headers
    else:
        pending_headers = state.previous_epoch_pending_shard_headers
        
    # Check that this header is not yet in the pending list
    for pending_header in pending_headers:
        assert header_root != pending_header.root
    # Include it in the pending list
    committee_length = len(get_beacon_committee(state, header.slot, header.shard))
    pending_headers.append(PendingShardHeader(
        slot=header.slot,
        shard=header.shard,
        commitment=header.commitment,
        root=header_root,
        votes=Bitlist[MAX_COMMITTEE_SIZE]([0] * committee_length),
        confirmed=False
    ))

The length-and-degree proof works as follows. For a block B with length l (so l nonzero values in [0... - 1]), the length proof is the commitment to the polynomial (B(X) / Z(X)) * (X**(MAX_DEGREE + 1 - l)), where Z is the minimal polynomial that is zero over ROOT_OF_UNITY ** [l...next_power_of_two(l) - 1] (see SIZE_CHECK_POINTS above) and MAX_DEGREE the the maximum power of s available in the setup, which is MAX_DEGREE = len(G2_SETUP) - 1. The goal is to ensure that a proof can only be constructed if (i) B / Z is itself non-fractional, meaning that B is a multiple of Z, and (ii) deg(B) < next_power_of_two(l) (there are not hidden higher-order terms in the polynomial, which would thwart reconstruction).

The length proof will have the degree of (B(X) / Z(X)) * X**(MAX_DEGREE + 1 - l), so deg(B) - (next_power_of_two(l) - l) + MAX_DEGREE + 1 - l, simplified to deg(B) - next_power_of_two(l) + MAX_DEGREE + 1. Because it's only possible to commit to polynomials with degree <= MAX_DEGREE, it's only possible to generate the proof if this expression is less than or equal to MAX_DEGREE, meaning that deg(B) must be strictly less than next_power_of_two(l).

Shard transition processing

Epoch transition

This epoch transition overrides the phase0 epoch transition:

def process_epoch(state: BeaconState) -> None:
    process_justification_and_finalization(state)
    process_rewards_and_penalties(state)
    process_registry_updates(state)

    # Proof of custody
    process_reveal_deadlines(state)
    process_challenge_deadlines(state)

    process_slashings(state)

    # Sharding
    process_pending_headers(state)
    charge_confirmed_header_fees(state)
    reset_pending_headers(state)

    # Final updates
    # Phase 0
    process_eth1_data_reset(state)
    process_effective_balance_updates(state)
    process_slashings_reset(state)
    process_randao_mixes_reset(state)
    process_historical_roots_update(state)
    process_participation_record_updates(state)
    # Proof of custody
    process_custody_final_updates(state)
    # Update current_epoch_start_shard
    state.current_epoch_start_shard = get_start_shard(state, Slot(state.slot + 1))

Pending headers


def process_pending_headers(state: BeaconState):
    for slot in range(SLOTS_PER_EPOCH):
        for shard in range(SHARD_COUNT):
            # Pending headers for this (slot, shard) combo
            candidates = [
                c for c in state.previous_epoch_pending_shard_headers if
                (c.slot, c.shard) == (slot, shard)
            ]
            if True not in [c.confirmed for c in candidates]:
                # The entire committee (and its balance)
                full_committee = get_beacon_committee(state, slot, shard)
                full_committee_balance = get_total_balance(state, full_committee)
                # The set of voters who voted for each header
                # (and their total balances)
                voting_sets = [
                    [v for i, v in enumerate(full_committee) if c.votes[i]]
                    for c in candidates
                ]
                voting_balances = [
                    get_total_balance(state, voters)
                    for voters in voting_sets
                ]
                # Get the index with the most total balance voting for them.
                # NOTE: if two choices get exactly the same voting balance,
                # the candidate earlier in the list wins
                if max(voting_balances) > 0:
                    winning_index = voting_balances.index(max(voting_balances))
                else:
                    # If no votes, zero wins
                    winning_index = [c.root for c in candidates].index(Root())
                candidates[winning_index].confirmed = True
    for slot in range(SLOTS_PER_EPOCH):
        for shard in range(SHARD_COUNT):
            state.most_recent_confirmed_commitments[shard][slot] = DataCommitment()
    for c in state.previous_epoch_pending_shard_headers:
        if c.confirmed:
            state.most_recent_confirmed_commitments[c.shard][c.slot % SLOTS_PER_EPOCH] = c.commitment

def charge_confirmed_header_fees(state: BeaconState) -> None:
    new_gasprice = state.shard_gasprice
    adjustment_quotient = get_active_shard_count(state) * SLOTS_PER_EPOCH * GASPRICE_ADJUSTMENT_COEFFICIENT
    for slot in range(SLOTS_PER_EPOCH):
        for shard in range(SHARD_COUNT):
            confirmed_candidates = [
                c for c in state.previous_epoch_pending_shard_headers if
                (c.slot, c.shard, c.confirmed) == (slot, shard, True)
            ]
            if confirmed_candidates:
                candidate = confirmed_candidates[0]
                # Charge EIP 1559 fee
                proposer = get_shard_proposer(state, slot, shard)
                fee = (
                    (state.shard_gasprice * candidates[i].commitment.length) //
                    TARGET_SAMPLES_PER_BLOCK
                )
                decrease_balance(state, proposer, fee)
                new_gasprice = compute_updated_gasprice(
                    new_gasprice,
                    candidates[i].commitment.length,
                    adjustment_quotient
                )
    state.shard_gasprice = new_gasprice

def reset_pending_headers(state: BeaconState):
    state.previous_epoch_pending_shard_headers = state.current_epoch_pending_shard_headers
    shards = [
        compute_shard_from_committee_index(state, index, slot)
        for i in range()
        state,
        attestation.data.index,
        attestation.data.slot
    )
    state.current_epoch_pending_shard_headers = []
    # Add dummy "empty" PendingAttestations
    # (default to vote for if no shard header available)
    for slot in range(SLOTS_IN_EPOCH):
        for index in range(get_committee_count_per_slot(get_current_epoch(state))):
            shard = compute_shard_from_committee_index(state, index, slot)
            committee_length = len(get_beacon_committee(
                state,
                header.slot,
                header.shard
            ))
            state.current_epoch_pending_shard_headers.append(PendingShardHeader(
                slot=slot,
                shard=shard,
                commitment=DataCommitment(),
                root=Root(),
                votes=Bitlist[MAX_COMMITTEE_SIZE]([0] * committee_length),
                confirmed=False
            ))

Custody game updates

process_reveal_deadlines, process_challenge_deadlines and process_custody_final_updates are defined in the Custody Game spec.