26 KiB
26 KiB
Ethereum 2.0 Phase 1 -- The Beacon Chain with Shards
Notice: This document is a work-in-progress for researchers and implementers.
Table of contents
- Introduction
- Custom types
- Configuration
- Updated containers
- New containers
- Helper functions
Introduction
This document describes the extensions made to the Phase 0 design of The Beacon Chain to support data sharding, based on the ideas here and more broadly here, using Kate commitments to commit to data to remove any need for fraud proofs (and hence, safety-critical synchrony assumptions) in the design.
Custom types
We define the following Python custom types for type hinting and readability:
| Name | SSZ equivalent | Description |
|---|---|---|
Shard |
uint64 |
A shard number |
BLSCommitment |
bytes48 |
A G1 curve point |
Configuration
Misc
| Name | Value | Notes |
|---|---|---|
MAX_SHARDS |
uint64(2**10) (= 1024) |
Theoretical max shard count (used to determine data structure sizes) |
INITIAL_ACTIVE_SHARDS |
uint64(2**6) (= 64) |
Initial shard count |
GASPRICE_ADJUSTMENT_COEFFICIENT |
uint64(2**3) (= 8) |
Gasprice may decrease/increase by at most exp(1 / this value) per epoch |
MAX_SHARD_HEADERS_PER_SHARD |
4 |
|
MAX_SHARD_HEADERS |
MAX_SHARDS * MAX_SHARD_HEADERS_PER_SHARD |
|
PRIMITIVE_ROOT_OF_UNITY |
5 |
Primitive root of unity of the BLS12_381 (inner) modulus |
DATA_AVAILABILITY_INVERSE_CODING_RATE |
2**1 (=2) |
Factor by which samples are extended for data availability encoding |
Shard block configs
| Name | Value | Notes |
|---|---|---|
POINTS_PER_SAMPLE |
uint64(2**3) (= 8) |
31 * 8 = 248 bytes |
MAX_SAMPLES_PER_BLOCK |
uint64(2**11) (= 2,048) |
248 * 2,048 = 507,904 bytes |
TARGET_SAMPLES_PER_BLOCK |
uint64(2**10) (= 1,024) |
248 * 1,024 = 253,952 bytes |
Precomputed size verification points
| Name | Value |
|---|---|
G2_SETUP |
Type List[G2]. The G2-side trusted setup [G, G*s, G*s**2....]; note that the first point is the generator. |
MODULUS |
0x73eda753299d7d483339d80809a1d80553bda402fffe5bfeffffffff00000001 (curve order of BLS12_381) |
ROOT_OF_UNITY |
pow(PRIMITIVE_ROOT_OF_UNITY, (MODULUS - 1) // (MAX_SAMPLES_PER_BLOCK * POINTS_PER_SAMPLE, MODULUS) |
SIZE_CHECK_POINTS |
Type List[G2, MAX_SAMPLES_PER_BLOCK + 1]; TO BE COMPUTED |
These points are the G2-side Kate commitments to product[a in i...next_power_of_two(i)] (X ** POINTS_PER_SAMPLE - w ** (reverse_bit_order(a, MAX_SAMPLES_PER_BLOCK * DATA_AVAILABILITY_INVERSE_CODING_RATE) * POINTS_PER_SAMPLE)) for each i in [0...MAX_SAMPLES_PER_BLOCK], where w = ROOT_OF_UNITY. They are used to verify block size proofs. They can be computed with a one-time O(N**2/log(N)) calculation using fast-linear-combinations in G2.
Gwei values
| Name | Value | Unit | Description |
|---|---|---|---|
MAX_GASPRICE |
Gwei(2**33) (= 8,589,934,592) |
Gwei | Max gasprice charged for a TARGET-sized shard block |
MIN_GASPRICE |
Gwei(2**3) (= 8) |
Gwei | Min gasprice charged for a TARGET-sized shard block |
Time parameters
| Name | Value | Unit | Duration |
|---|---|---|---|
SHARD_COMMITTEE_PERIOD |
Epoch(2**8) (= 256) |
epochs | ~27 hours |
Domain types
| Name | Value |
|---|---|
DOMAIN_SHARD_HEADER |
DomainType('0x80000000') |
Updated containers
The following containers have updated definitions in Phase 1.
AttestationData
class AttestationData(Container):
slot: Slot
index: CommitteeIndex
# LMD GHOST vote
beacon_block_root: Root
# FFG vote
source: Checkpoint
target: Checkpoint
# Shard header root
shard_header_root: Root
BeaconBlock
class BeaconBlock(phase0.BeaconBlock):
shard_headers: List[Signed[ShardHeader], MAX_SHARD_HEADERS]
BeaconState
class BeaconState(phase0.BeaconState):
# Updated fields
previous_epoch_attestations: List[PendingAttestation, MAX_ATTESTATIONS * SLOTS_PER_EPOCH]
current_epoch_attestations: List[PendingAttestation, MAX_ATTESTATIONS * SLOTS_PER_EPOCH]
# New fields
current_epoch_pending_shard_headers: List[PendingShardHeader, MAX_SHARD_HEADERS * SLOTS_PER_EPOCH]
previous_epoch_pending_shard_headers: List[PendingShardHeader, MAX_SHARD_HEADERS * SLOTS_PER_EPOCH]
most_recent_confirmed_commitments: Vector[Vector[DataCommitment, SLOTS_PER_EPOCH], MAX_SHARDS]
shard_gasprice: uint64
current_epoch_start_shard: Shard
New containers
The following containers are new in Phase 1.
DataCommitment
class DataCommitment(Container):
# Kate commitment to the data
point: BLSCommitment
# Length of the data in samples
length: uint64
ShardHeader
class ShardHeader(Container):
# Slot and shard that this header is intended for
slot: Slot
shard: Shard
# The actual data commitment
commitment: DataCommitment
# Proof of the length (more precisely, proof that values at
# positions >= the length all equal zero)
length_proof: BLSCommitment
PendingShardHeader
class PendingShardHeader(Container):
# Slot and shard that this header is intended for
slot: uint64
shard: Shard
# Kate commitment to the data
commitment: BLSCommitment
# hash_tree_root of the ShardHeader (stored so that attestations
# can be checked against it)
root: Hash
# Length of the data in samples
length: uint64
# Who voted for the header
votes: Bitlist[MAX_COMMITTEE_SIZE]
# Has this header been confirmed?
confirmed: bool
Helper functions
Misc
next_power_of_two
def next_power_of_two(x):
return 2 ** ((x - 1).bit_length())
reverse_bit_order
def reverse_bit_order(n, order):
"""
Reverse the bit order of an integer n
"""
assert is_power_of_two(order)
return int(('{:0' + str(order.bit_length() - 1) + 'b}').format(n)[::-1], 2)
compute_previous_slot
def compute_previous_slot(slot: Slot) -> Slot:
if slot > 0:
return Slot(slot - 1)
else:
return Slot(0)
compute_shard_from_committee_index
def compute_shard_from_committee_index(state: BeaconState, index: CommitteeIndex, slot: Slot) -> Shard:
active_shards = get_active_shard_count(state)
return Shard((index + get_start_shard(state, slot)) % active_shards)
compute_updated_gasprice
def compute_updated_gasprice(prev_gasprice: Gwei, shard_block_length: uint64, adjustment_quotient: uint64) -> Gwei:
if shard_block_length > TARGET_SAMPLES_PER_BLOCK:
delta = max(1, prev_gasprice * (shard_block_length - TARGET_SAMPLES_PER_BLOCK)
// TARGET_SAMPLES_PER_BLOCK // adjustment_quotient)
return min(prev_gasprice + delta, MAX_GASPRICE)
else:
delta = max(1, prev_gasprice * (TARGET_SAMPLES_PER_BLOCK - shard_block_length)
// TARGET_SAMPLES_PER_BLOCK // adjustment_quotient)
return max(prev_gasprice, MIN_GASPRICE + delta) - delta
compute_committee_source_epoch
def compute_committee_source_epoch(epoch: Epoch, period: uint64) -> Epoch:
"""
Return the source epoch for computing the committee.
"""
source_epoch = Epoch(epoch - epoch % period)
if source_epoch >= period:
source_epoch -= period # `period` epochs lookahead
return source_epoch
Beacon state accessors
Updated get_committee_count_per_slot
def get_committee_count_per_slot(state: BeaconState, epoch: Epoch) -> uint64:
"""
Return the number of committees in each slot for the given ``epoch``.
"""
return max(uint64(1), min(
get_active_shard_count(state, epoch),
uint64(len(get_active_validator_indices(state, epoch))) // SLOTS_PER_EPOCH // TARGET_COMMITTEE_SIZE,
))
get_active_shard_count
def get_active_shard_count(state: BeaconState, epoch: Epoch) -> uint64:
"""
Return the number of active shards.
Note that this puts an upper bound on the number of committees per slot.
"""
return INITIAL_ACTIVE_SHARDS
get_shard_committee
def get_shard_committee(beacon_state: BeaconState, epoch: Epoch, shard: Shard) -> Sequence[ValidatorIndex]:
"""
Return the shard committee of the given ``epoch`` of the given ``shard``.
"""
source_epoch = compute_committee_source_epoch(epoch, SHARD_COMMITTEE_PERIOD)
active_validator_indices = get_active_validator_indices(beacon_state, source_epoch)
seed = get_seed(beacon_state, source_epoch, DOMAIN_SHARD_COMMITTEE)
return compute_committee(
indices=active_validator_indices,
seed=seed,
index=shard,
count=get_active_shard_count(beacon_state, epoch),
)
compute_proposer_index
Updated version to get a proposer index that will only allow proposers with a certain minimum balance, ensuring that the balance is always sufficient to cover gas costs.
def compute_proposer_index(state: BeaconState, indices: Sequence[ValidatorIndex], seed: Bytes32, min_effective_balance: GWei = GWei(0)) -> ValidatorIndex:
"""
Return from ``indices`` a random index sampled by effective balance.
"""
assert len(indices) > 0
MAX_RANDOM_BYTE = 2**8 - 1
i = uint64(0)
total = uint64(len(indices))
while True:
candidate_index = indices[compute_shuffled_index(i % total, total, seed)]
random_byte = hash(seed + uint_to_bytes(uint64(i // 32)))[i % 32]
effective_balance = state.validators[candidate_index].effective_balance
if effective_balance * MAX_RANDOM_BYTE >= MAX_EFFECTIVE_BALANCE * random_byte and effective_balance > min_effective_balance:
return candidate_index
i += 1
get_shard_proposer_index
def get_shard_proposer_index(beacon_state: BeaconState, slot: Slot, shard: Shard) -> ValidatorIndex:
"""
Return the proposer's index of shard block at ``slot``.
"""
epoch = compute_epoch_at_slot(slot)
committee = get_shard_committee(beacon_state, epoch, shard)
seed = hash(get_seed(state, epoch, DOMAIN_BEACON_PROPOSER) + uint_to_bytes(state.slot))
EFFECTIVE_BALANCE_MAX_DOWNWARD_DEVIATION = EFFECTIVE_BALANCE_INCREMENT - EFFECTIVE_BALANCE_INCREMENT * HYSTERESIS_DOWNWARD_MULTIPLIER // HYSTERESIS_QUOTIENT
return compute_proposer_index(state, committee, seed,
state.shard_gasprice * MAX_SAMPLES_PER_BLOCK // TARGET_SAMPLES_PER_BLOCK + EFFECTIVE_BALANCE_MAX_DOWNWARD_DEVIATION)
get_start_shard
def get_start_shard(state: BeaconState, slot: Slot) -> Shard:
"""
Return the start shard at ``slot``.
"""
current_epoch_start_slot = compute_start_slot_at_epoch(get_current_epoch(state))
shard = state.current_epoch_start_shard
if slot > current_epoch_start_slot:
# Current epoch or the next epoch lookahead
for _slot in range(current_epoch_start_slot, slot):
committee_count = get_committee_count_per_slot(state, compute_epoch_at_slot(Slot(_slot)))
active_shard_count = get_active_shard_count(state, compute_epoch_at_slot(Slot(_slot)))
shard = (shard + committee_count) % active_shard_count
return Shard(shard)
elif slot < current_epoch_start_slot:
# Previous epoch
for _slot in list(range(slot, current_epoch_start_slot))[::-1]:
committee_count = get_committee_count_per_slot(state, compute_epoch_at_slot(Slot(_slot)))
active_shard_count = get_active_shard_count(state, compute_epoch_at_slot(Slot(_slot)))
# Ensure positive
shard = (shard + active_shard_count - committee_count) % active_shard_count
return Shard(shard)
Block processing
def process_block(state: BeaconState, block: BeaconBlock) -> None:
process_block_header(state, block)
process_randao(state, block.body)
process_eth1_data(state, block.body)
process_light_client_aggregate(state, block.body)
process_operations(state, block.body)
Operations
def process_operations(state: BeaconState, body: BeaconBlockBody) -> None:
# Verify that outstanding deposits are processed up to the maximum number of deposits
assert len(body.deposits) == min(MAX_DEPOSITS, state.eth1_data.deposit_count - state.eth1_deposit_index)
def for_ops(operations: Sequence[Any], fn: Callable[[BeaconState, Any], None]) -> None:
for operation in operations:
fn(state, operation)
for_ops(body.proposer_slashings, process_proposer_slashing)
for_ops(body.attester_slashings, process_attester_slashing)
# New attestation processing
for_ops(body.attestations, process_attestation)
for_ops(body.deposits, process_deposit)
for_ops(body.voluntary_exits, process_voluntary_exit)
# Limit is dynamic based on active shard count
assert len(body.shard_headers) <= MAX_SHARD_HEADERS_PER_SHARD * get_active_shard_count(state, get_current_epoch(state))
for_ops(body.shard_headers, process_shard_header)
# See custody game spec.
process_custody_game_operations(state, body)
process_shard_transitions(state, body.shard_transitions, body.attestations)
# TODO process_operations(body.shard_receipt_proofs, process_shard_receipt_proofs)
New Attestation processing
Updated process_attestation
def process_attestation(state: BeaconState, attestation: Attestation) -> None:
phase0.process_attestation(state, attestation)
update_pending_votes(
state=state,
attestation: Attestation,
root=,
aggregation_bits=attestation.aggregation_bits
)
update_pending_votes
def update_pending_votes(state: BeaconState,
attestation: Attestation) -> None:
if compute_epoch_at_slot(slot) == get_current_epoch(state):
pending_headers = state.current_epoch_pending_shard_headers
else:
pending_headers = state.previous_epoch_pending_shard_headers
# Create or update the PendingShardHeader object
pending_header = None
for header in pending_headers:
if header.root == attestation.data.shard_header_root:
pending_header = header
assert pending_header is not None
assert pending_header.slot == attestation.data.slot + 1
assert pending_header.shard == compute_shard_from_committee_index(
state,
attestation.data.index,
attestation.data.slot
)
pending_header.votes = bitwise_or(
pending_header.votes,
attestation.aggregation_bits
)
# Check if the PendingShardHeader is eligible for expedited confirmation
# Requirement 1: nothing else confirmed
all_candidates = [
c for c in pending_headers if
(c.slot, c.shard) == (pending_header.slot, pending_header.shard)
]
if True not in [c.confirmed for c in all_candidates]:
# Requirement 2: >= 2/3 of balance attesting
participants = get_attesting_indices(state, attestationg.data, pending_commitment.votes)
participants_balance = get_total_balance(state, participants)
full_committee = get_beacon_committee(state, attestationg.data.slot, attestationg.data.shard)
full_committee_balance = get_total_balance(state, full_committee)
if participants_balance * 3 > full_committee_balance * 2:
pending_header.confirmed = True
process_shard_header
def process_shard_header(state: BeaconState,
signed_header: Signed[ShardDataHeader]) -> None:
header = signed_header.message
header_root = hash_tree_root(header)
# Verify signature
signer_index = get_shard_proposer_index(state, header.slot, header.shard)
assert bls.Verify(
state.validators[signer_index].pubkey,
compute_signing_root(header, get_domain(state, DOMAIN_SHARD_HEADER)),
signed_header.signature
)
# Verify length of the header, and simultaneously verify degree.
assert (
bls.Pairing(header.length_proof, SIZE_CHECK_POINTS[header.commitment.length]) ==
bls.Pairing(header.commitment.point, G2_SETUP[-header.commitment.length]))
)
# Get the correct pending header list
if compute_epoch_at_slot(header.slot) == get_current_epoch(state):
pending_headers = state.current_epoch_pending_shard_headers
else:
pending_headers = state.previous_epoch_pending_shard_headers
# Check that this header is not yet in the pending list
for pending_header in pending_headers:
assert header_root != pending_header.root
# Include it in the pending list
committee_length = len(get_beacon_committee(state, header.slot, header.shard))
pending_headers.append(PendingShardHeader(
slot=header.slot,
shard=header.shard,
commitment=header.commitment,
root=header_root,
votes=Bitlist[MAX_VALIDATORS_PER_COMMITTEE]([0] * committee_length),
confirmed=False
))
The length-and-degree proof works as follows. For a block B with length l (so l nonzero values in [0... - 1]), the length proof is the commitment to the polynomial (B(X) / Z(X)) * (X**(MAX_DEGREE + 1 - l)), where Z is the minimal polynomial that is zero over ROOT_OF_UNITY ** [l...next_power_of_two(l) - 1] (see SIZE_CHECK_POINTS above) and MAX_DEGREE the the maximum power of s available in the setup, which is MAX_DEGREE = len(G2_SETUP) - 1. The goal is to ensure that a proof can only be constructed if (i) B / Z is itself non-fractional, meaning that B is a multiple of Z, and (ii) deg(B) < next_power_of_two(l) (there are not hidden higher-order terms in the polynomial, which would thwart reconstruction).
The length proof will have the degree of (B(X) / Z(X)) * X**(MAX_DEGREE + 1 - l), so deg(B) - (next_power_of_two(l) - l) + MAX_DEGREE + 1 - l, simplified to deg(B) - next_power_of_two(l) + MAX_DEGREE + 1. Because it's only possible to commit to polynomials with degree <= MAX_DEGREE, it's only possible to generate the proof if this expression is less than or equal to MAX_DEGREE, meaning that deg(B) must be strictly less than next_power_of_two(l).
Shard transition processing
Epoch transition
This epoch transition overrides the phase0 epoch transition:
def process_epoch(state: BeaconState) -> None:
process_justification_and_finalization(state)
process_rewards_and_penalties(state)
process_registry_updates(state)
# Proof of custody
process_reveal_deadlines(state)
process_challenge_deadlines(state)
process_slashings(state)
# Sharding
process_pending_headers(state)
charge_confirmed_header_fees(state)
reset_pending_headers(state)
# Final updates
# Phase 0
process_eth1_data_reset(state)
process_effective_balance_updates(state)
process_slashings_reset(state)
process_randao_mixes_reset(state)
process_historical_roots_update(state)
process_participation_record_updates(state)
# Proof of custody
process_custody_final_updates(state)
# Update current_epoch_start_shard
state.current_epoch_start_shard = get_start_shard(state, Slot(state.slot + 1))
Pending headers
def process_pending_headers(state: BeaconState):
for slot in range(SLOTS_PER_EPOCH):
for shard in range(SHARD_COUNT):
# Pending headers for this (slot, shard) combo
candidates = [
c for c in state.previous_epoch_pending_shard_headers if
(c.slot, c.shard) == (slot, shard)
]
if True not in [c.confirmed for c in candidates]:
# The entire committee (and its balance)
full_committee = get_beacon_committee(state, slot, shard)
full_committee_balance = get_total_balance(state, full_committee)
# The set of voters who voted for each header
# (and their total balances)
voting_sets = [
[v for i, v in enumerate(full_committee) if c.votes[i]]
for c in candidates
]
voting_balances = [
get_total_balance(state, voters)
for voters in voting_sets
]
# Get the index with the most total balance voting for them.
# NOTE: if two choices get exactly the same voting balance,
# the candidate earlier in the list wins
if max(voting_balances) > 0:
winning_index = voting_balances.index(max(voting_balances))
else:
# If no votes, zero wins
winning_index = [c.root for c in candidates].index(Root())
candidates[winning_index].confirmed = True
for slot in range(SLOTS_PER_EPOCH):
for shard in range(SHARD_COUNT):
state.most_recent_confirmed_commitments[shard][slot] = DataCommitment()
for c in state.previous_epoch_pending_shard_headers:
if c.confirmed:
state.most_recent_confirmed_commitments[c.shard][c.slot % SLOTS_PER_EPOCH] = c.commitment
def charge_confirmed_header_fees(state: BeaconState) -> None:
new_gasprice = state.shard_gasprice
adjustment_quotient = get_active_shard_count(state) * SLOTS_PER_EPOCH * GASPRICE_ADJUSTMENT_COEFFICIENT
for slot in range(SLOTS_PER_EPOCH):
for shard in range(SHARD_COUNT):
confirmed_candidates = [
c for c in state.previous_epoch_pending_shard_headers if
(c.slot, c.shard, c.confirmed) == (slot, shard, True)
]
if confirmed_candidates:
candidate = confirmed_candidates[0]
# Charge EIP 1559 fee
proposer = get_shard_proposer(state, slot, shard)
fee = (
(state.shard_gasprice * candidates[i].commitment.length) //
TARGET_SAMPLES_PER_BLOCK
)
decrease_balance(state, proposer, fee)
new_gasprice = compute_updated_gasprice(
new_gasprice,
candidates[i].commitment.length,
adjustment_quotient
)
state.shard_gasprice = new_gasprice
def reset_pending_headers(state: BeaconState):
state.previous_epoch_pending_shard_headers = state.current_epoch_pending_shard_headers
shards = [
compute_shard_from_committee_index(state, index, slot)
for i in range()
state,
attestation.data.index,
attestation.data.slot
)
state.current_epoch_pending_shard_headers = []
# Add dummy "empty" PendingAttestations
# (default to vote for if no shard header available)
for slot in range(SLOTS_IN_EPOCH):
for index in range(get_committee_count_per_slot(get_current_epoch(state))):
shard = compute_shard_from_committee_index(state, index, slot)
committee_length = len(get_beacon_committee(
state,
header.slot,
header.shard
))
state.current_epoch_pending_shard_headers.append(PendingShardHeader(
slot=slot,
shard=shard,
commitment=DataCommitment(),
root=Root(),
votes=Bitlist[MAX_VALIDATORS_PER_COMMITTEE]([0] * committee_length),
confirmed=False
))
Custody game updates
process_reveal_deadlines, process_challenge_deadlines and process_custody_final_updates are defined in the Custody Game spec.