Updated phase 1: branch and early subkey reveal challenges (#587)

* Updated phase 1: branch challenges

* Removed unnecessary line

* Added early subkey reveal slashing

* Revealing during the active period is still revealing early

* Added....

* Machinery for publishing old subkeys
* Inability to withdraw until you published all subkeys
* After a validator exits the queue there's still a minimum 1-day delay before they can withdraw (in the future this delay will be used as an opportunity to start a PoC challenge game)

* Update 1_shard-data-chains.md

* formatting

* minor edits

* Added masking scheme for reveals

Secure under the aggregate extraction infeasibility assumption described on pages 11-12 of https://crypto.stanford.edu/~dabo/pubs/papers/aggreg.pdf

* Added rewards going to challengers

* Add ToC and reorg the constant tables

* Remove tags

* fix constant formatting

* normalize domain constants in phase 1

* Update 1_shard-data-chains.md

* Update 1_shard-data-chains.md

* Update 1_shard-data-chains.md

* Added transition logic

* Fix ToC

* Fix ToC

* Adjusted for #615

* Added more helpers

* epoch -> slot

* fix some type hints

* clean up `get_attestation_merkle_depth`
This commit is contained in:
vbuterin 2019-02-19 05:26:35 -06:00 committed by GitHub
parent f0b562054f
commit 2540f55c13
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 376 additions and 13 deletions

View File

@ -1,9 +1,57 @@
# Ethereum 2.0 Phase 1 -- Shard Data Chains
###### tags: `spec`, `eth2.0`, `casper`, `sharding`
**NOTICE**: This document is a work-in-progress for researchers and implementers. It reflects recent spec changes and takes precedence over the [Python proof-of-concept implementation](https://github.com/ethereum/beacon_chain).
## Table of contents
<!-- TOC -->
- [Ethereum 2.0 Phase 1 -- Shard Data Chains](#ethereum-20-phase-1----shard-data-chains)
- [Table of contents](#table-of-contents)
- [Introduction](#introduction)
- [Terminology](#terminology)
- [Constants](#constants)
- [Misc](#misc)
- [Time parameters](#time-parameters)
- [Max operations per block](#max-operations-per-block)
- [Signature domains](#signature-domains)
- [Helper functions](#helper-functions)
- [get_split_offset](#get_split_offset)
- [get_shuffled_committee](#get_shuffled_committee)
- [get_persistent_committee](#get_persistent_committee)
- [get_shard_proposer_index](#get_shard_proposer_index)
- [Data Structures](#data-structures)
- [Shard chain blocks](#shard-chain-blocks)
- [Shard block processing](#shard-block-processing)
- [Verifying shard block data](#verifying-shard-block-data)
- [Verifying a crosslink](#verifying-a-crosslink)
- [Shard block fork choice rule](#shard-block-fork-choice-rule)
- [Updates to the beacon chain](#updates-to-the-beacon-chain)
- [Data structures](#data-structures)
- [`Validator`](#validator)
- [`BeaconBlockBody`](#beaconblockbody)
- [`BranchChallenge`](#branchchallenge)
- [`BranchResponse`](#branchresponse)
- [`BranchChallengeRecord`](#branchchallengerecord)
- [`SubkeyReveal`](#subkeyreveal)
- [Helpers](#helpers)
- [`get_attestation_merkle_depth`](#get_attestation_merkle_depth)
- [`epoch_to_custody_period`](#epoch_to_custody_period)
- [`slot_to_custody_period`](#slot_to_custody_period)
- [`get_current_custody_period`](#get_current_custody_period)
- [`verify_custody_subkey_reveal`](#verify_custody_subkey_reveal)
- [`prepare_validator_for_withdrawal`](#prepare_validator_for_withdrawal)
- [`penalize_validator`](#penalize_validator)
- [Per-slot processing](#per-slot-processing)
- [Operations](#operations)
- [Branch challenges](#branch-challenges)
- [Branch responses](#branch-responses)
- [Subkey reveals](#subkey-reveals)
- [Per-epoch processing](#per-epoch-processing)
- [One-time phase 1 initiation transition](#one-time-phase-1-initiation-transition)
<!-- /TOC -->
### Introduction
This document represents the specification for Phase 1 of Ethereum 2.0 -- Shard Data Chains. Phase 1 depends on the implementation of [Phase 0 -- The Beacon Chain](0_beacon-chain.md).
@ -16,19 +64,39 @@ Ethereum 2.0 consists of a central beacon chain along with `SHARD_COUNT` shard c
Phase 1 depends upon all of the constants defined in [Phase 0](0_beacon-chain.md#constants) in addition to the following:
| Constant | Value | Unit | Approximation |
|-------------------------------|------------------|--------|---------------|
| `SHARD_CHUNK_SIZE` | 2**5 (= 32) | bytes | |
| `SHARD_BLOCK_SIZE` | 2**14 (= 16,384) | bytes | |
| `CROSSLINK_LOOKBACK` | 2**5 (= 32) | slots | |
#### Misc
| Name | Value | Unit |
|-------------------------------|------------------|--------|
| `SHARD_CHUNK_SIZE` | 2**5 (= 32) | bytes |
| `SHARD_BLOCK_SIZE` | 2**14 (= 16,384) | bytes |
| `MINOR_REWARD_QUOTIENT` | 2**8 (= 256) | |
#### Time parameters
| Name | Value | Unit | Duration |
| - | - | :-: | :-: |
| `CROSSLINK_LOOKBACK` | 2**5 (= 32) | slots | 3.2 minutes |
| `MAX_BRANCH_CHALLENGE_DELAY` | 2**11 (= 2,048) | epochs | 9 days |
| `CUSTODY_PERIOD_LENGTH` | 2**11 (= 2,048) | epochs | 9 days |
| `PERSISTENT_COMMITTEE_PERIOD` | 2**11 (= 2,048) | epochs | 9 days |
| `CHALLENGE_RESPONSE_DEADLINE` | 2**14 (= 16,384) | epochs | 73 days |
### Flags, domains, etc.
#### Max operations per block
| Constant | Value |
| Name | Value |
|-------------------------------|---------------|
| `MAX_BRANCH_CHALLENGES` | 2**2 (= 4) |
| `MAX_BRANCH_RESPONSES` | 2**4 (= 16) |
| `MAX_EARLY_SUBKEY_REVEALS` | 2**4 (= 16) |
#### Signature domains
| Name | Value |
|------------------------|-----------------|
| `SHARD_PROPOSER_DOMAIN`| 129 |
| `SHARD_ATTESTER_DOMAIN`| 130 |
| `DOMAIN_SHARD_PROPOSER`| 129 |
| `DOMAIN_SHARD_ATTESTER`| 130 |
| `DOMAIN_CUSTODY_SUBKEY`| 131 |
## Helper functions
@ -159,9 +227,9 @@ To validate a block header on shard `shard_block.shard_id`, compute as follows:
* Let `proposer_index = get_shard_proposer_index(state, shard_block.shard_id, shard_block.slot)`.
* Verify that `proposer_index` is not `None`.
* Let `msg` be the `shard_block` but with `shard_block.signature` set to `[0, 0]`.
* Verify that `bls_verify(pubkey=validators[proposer_index].pubkey, message_hash=hash(msg), signature=shard_block.signature, domain=get_domain(state, slot_to_epoch(shard_block.slot), SHARD_PROPOSER_DOMAIN))` passes.
* Verify that `bls_verify(pubkey=validators[proposer_index].pubkey, message_hash=hash(msg), signature=shard_block.signature, domain=get_domain(state, slot_to_epoch(shard_block.slot), DOMAIN_SHARD_PROPOSER))` passes.
* Let `group_public_key = bls_aggregate_pubkeys([state.validators[index].pubkey for i, index in enumerate(persistent_committee) if get_bitfield_bit(shard_block.participation_bitfield, i) is True])`.
* Verify that `bls_verify(pubkey=group_public_key, message_hash=shard_block.parent_root, sig=shard_block.aggregate_signature, domain=get_domain(state, slot_to_epoch(shard_block.slot), SHARD_ATTESTER_DOMAIN))` passes.
* Verify that `bls_verify(pubkey=group_public_key, message_hash=shard_block.parent_root, sig=shard_block.aggregate_signature, domain=get_domain(state, slot_to_epoch(shard_block.slot), DOMAIN_SHARD_ATTESTER))` passes.
### Verifying shard block data
@ -222,3 +290,298 @@ The `shard_chain_commitment` is only valid if it equals `compute_commitment(head
### Shard block fork choice rule
The fork choice rule for any shard is LMD GHOST using the shard chain attestations of the persistent committee and the beacon chain attestations of the crosslink committee currently assigned to that shard, but instead of being rooted in the genesis it is rooted in the block referenced in the most recent accepted crosslink (ie. `state.crosslinks[shard].shard_block_root`). Only blocks whose `beacon_chain_ref` is the block in the main beacon chain at the specified `slot` should be considered (if the beacon chain skips a slot, then the block at that slot is considered to be the block in the beacon chain at the highest slot lower than a slot).
# Updates to the beacon chain
## Data structures
### `Validator`
Add member values to the end of the `Validator` object:
```python
'open_branch_challenges': [BranchChallengeRecord],
'next_subkey_to_reveal': 'uint64',
'reveal_max_periods_late': 'uint64',
```
And the initializers:
```python
'open_branch_challenges': [],
'next_subkey_to_reveal': get_current_custody_period(state),
'reveal_max_periods_late': 0,
```
### `BeaconBlockBody`
Add member values to the `BeaconBlockBody` structure:
```python
'branch_challenges': [BranchChallenge],
'branch_responses': [BranchResponse],
'subkey_reveals': [SubkeyReveal],
```
And initialize to the following:
```python
'branch_challenges': [],
'branch_responses': [],
'subkey_reveals': [],
```
### `BranchChallenge`
Define a `BranchChallenge` as follows:
```python
{
'responder_index': 'uint64',
'data_index': 'uint64',
'attestation': SlashableAttestation,
}
```
### `BranchResponse`
Define a `BranchResponse` as follows:
```python
{
'responder_index': 'uint64',
'data': 'bytes32',
'branch': ['bytes32'],
'data_index': 'uint64',
'root': 'bytes32',
}
```
### `BranchChallengeRecord`
Define a `BranchChallengeRecord` as follows:
```python
{
'challenger_index': 'uint64',
'root': 'bytes32',
'depth': 'uint64',
'inclusion_epoch': 'uint64',
'data_index': 'uint64',
}
```
### `SubkeyReveal`
Define a `SubkeyReveal` as follows:
```python
{
'validator_index': 'uint64',
'period': 'uint64',
'subkey': 'bytes96',
'mask': 'bytes32',
'revealer_index': 'uint64'
}
```
## Helpers
### `get_attestation_merkle_depth`
```python
def get_attestation_merkle_depth(attestation: Attestation) -> int:
start_epoch = attestation.data.latest_crosslink.epoch
end_epoch = slot_to_epoch(attestation.data.slot)
chunks_per_slot = SHARD_BLOCK_SIZE // 32
chunks = (end_epoch - start_epoch) * EPOCH_LENGTH * chunks_per_slot
return log2(next_power_of_two(chunks))
```
### `epoch_to_custody_period`
```python
def epoch_to_custody_period(epoch: Epoch) -> int:
return epoch // CUSTODY_PERIOD_LENGTH
```
### `slot_to_custody_period`
```python
def slot_to_custody_period(slot: Slot) -> int:
return epoch_to_custody_period(slot_to_epoch(slot))
```
### `get_current_custody_period`
```python
def get_current_custody_period(state: BeaconState) -> int:
return epoch_to_custody_period(get_current_epoch(state))
```
### `verify_custody_subkey_reveal`
```python
def verify_custody_subkey_reveal(pubkey: bytes48,
subkey: bytes96,
mask: bytes32,
mask_pubkey: bytes48,
period: int) -> bool:
# Legitimate reveal: checking that the provided value actually is the subkey
if mask == ZERO_HASH:
pubkeys=[pubkey]
message_hashes=[hash(int_to_bytes8(period))]
# Punitive early reveal: checking that the provided value is a valid masked subkey
# (masking done to prevent "stealing the reward" from a whistleblower by block proposers)
# Secure under the aggregate extraction infeasibility assumption described on page 11-12
# of https://crypto.stanford.edu/~dabo/pubs/papers/aggreg.pdf
else:
pubkeys=[pubkey, mask_pubkey]
message_hashes=[hash(int_to_bytes8(period)), mask]
return bls_multi_verify(
pubkeys=pubkeys,
message_hashes=message_hashes,
signature=subkey,
domain=get_domain(
fork=state.fork,
epoch=period * CUSTODY_PERIOD_LENGTH,
domain_type=DOMAIN_CUSTODY_SUBKEY,
)
)
```
### `penalize_validator`
Change the definition of `penalize_validator` as follows:
```python
def penalize_validator(state: BeaconState, index: ValidatorIndex, whistleblower_index=None:ValidatorIndex) -> None:
"""
Penalize the validator of the given ``index``.
Note that this function mutates ``state``.
"""
exit_validator(state, index)
validator = state.validator_registry[index]
state.latest_penalized_balances[get_current_epoch(state) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index)
block_proposer_index = get_beacon_proposer_index(state, state.slot)
whistleblower_reward = get_effective_balance(state, index) // WHISTLEBLOWER_REWARD_QUOTIENT
if whistleblower_index is None:
state.validator_balances[block_proposer_index] += whistleblower_reward
else:
state.validator_balances[whistleblower_index] += (
whistleblower_reward * INCLUDER_REWARD_QUOTIENT / (INCLUDER_REWARD_QUOTIENT + 1)
)
state.validator_balances[block_proposer_index] += whistleblower_reward / (INCLUDER_REWARD_QUOTIENT + 1)
state.validator_balances[index] -= whistleblower_reward
validator.penalized_epoch = get_current_epoch(state)
validator.withdrawable_epoch = get_current_epoch(state) + LATEST_PENALIZED_EXIT_LENGTH
```
The only change is that this introduces the possibility of a penalization where the "whistleblower" that takes credit is NOT the block proposer.
## Per-slot processing
### Operations
Add the following operations to the per-slot processing, in order the given below and _after_ all other operations (specifically, right after exits).
#### Branch challenges
Verify that `len(block.body.branch_challenges) <= MAX_BRANCH_CHALLENGES`.
For each `challenge` in `block.body.branch_challenges`:
* Verify that `slot_to_epoch(challenge.attestation.data.slot) >= get_current_epoch(state) - MAX_BRANCH_CHALLENGE_DELAY`.
* Verify that `state.validator_registry[responder_index].exit_epoch >= get_current_epoch(state) - MAX_BRANCH_CHALLENGE_DELAY`.
* Verify that `verify_slashable_attestation(state, challenge.attestation)` returns `True`.
* Verify that `challenge.responder_index` is in `challenge.attestation.validator_indices`.
* Let `depth = get_attestation_merkle_depth(challenge.attestation)`. Verify that `challenge.data_index < 2**depth`.
* Verify that there does not exist a `BranchChallengeRecord` in `state.validator_registry[challenge.responder_index].open_branch_challenges` with `root == challenge.attestation.data.shard_chain_commitment` and `data_index == data_index`.
* Append to `state.validator_registry[challenge.responder_index].open_branch_challenges` the object `BranchChallengeRecord(challenger_index=get_beacon_proposer_index(state, state.slot), root=challenge.attestation.data.shard_chain_commitment, depth=depth, inclusion_epoch=get_current_epoch(state), data_index=data_index)`.
**Invariant**: the `open_branch_challenges` array will always stay sorted in order of `inclusion_epoch`.
#### Branch responses
Verify that `len(block.body.branch_responses) <= MAX_BRANCH_RESPONSES`.
For each `response` in `block.body.branch_responses`:
* Find the `BranchChallengeRecord` in `state.validator_registry[response.responder_index].open_branch_challenges` whose (`root`, `data_index`) match the (`root`, `data_index`) of the `response`. Verify that one such record exists (it is not possible for there to be more than one), call it `record`.
* Verify that `verify_merkle_branch(leaf=response.data, branch=response.branch, depth=record.depth, index=record.data_index, root=record.root)` is True.
* Verify that `get_current_epoch(state) >= record.inclusion_epoch + ENTRY_EXIT_DELAY`.
* Remove the `record` from `state.validator_registry[response.responder_index].open_branch_challenges`
* Determine the proposer `proposer_index = get_beacon_proposer_index(state, state.slot)` and set `state.validator_balances[proposer_index] += base_reward(state, index) // MINOR_REWARD_QUOTIENT`.
#### Subkey reveals
Verify that `len(block.body.early_subkey_reveals) <= MAX_EARLY_SUBKEY_REVEALS`.
For each `reveal` in `block.body.early_subkey_reveals`:
* Verify that `verify_custody_subkey_reveal(state.validator_registry[reveal.validator_index].pubkey, reveal.subkey, reveal.period, reveal.mask, state.validator_registry[reveal.revealer_index].pubkey)` returns `True`.
* Let `is_early_reveal = reveal.period > get_current_custody_period(state) or (reveal.period == get_current_custody_period(state) and state.validator_registry[reveal.validator_index].exit_epoch > get_current_epoch(state))` (ie. either the reveal is of a future period, or it's of the current period and the validator is still active)
* Verify that one of the following is true:
* (i) `is_early_reveal` is `True`
* (ii) `is_early_reveal` is `False` and `reveal.period == state.validator_registry[reveal.validator_index].next_subkey_to_reveal` (revealing a past subkey, or a current subkey for a validator that has exited) and `reveal.mask == ZERO_HASH`
In case (i):
* Verify that `state.validator_registry[reveal.validator_index].penalized_epoch > get_current_epoch(state).
* Run `penalize_validator(state, reveal.validator_index, reveal.revealer_index)`.
* Set `state.validator_balances[reveal.revealer_index] += base_reward(state, index) // MINOR_REWARD_QUOTIENT`
In case (ii):
* Determine the proposer `proposer_index = get_beacon_proposer_index(state, state.slot)` and set `state.validator_balances[proposer_index] += base_reward(state, index) // MINOR_REWARD_QUOTIENT`.
* Set `state.validator_registry[reveal.validator_index].next_subkey_to_reveal += 1`
* Set `state.validator_registry[reveal.validator_index].reveal_max_periods_late = max(state.validator_registry[reveal.validator_index].reveal_max_periods_late, get_current_period(state) - reveal.period)`.
## Per-epoch processing
Add the following loop immediately below the `process_ejections` loop:
```python
def process_challenge_absences(state: BeaconState) -> None:
"""
Iterate through the validator registry
and penalize validators with balance that did not answer challenges.
"""
for index, validator in enumerate(state.validator_registry):
if len(validator.open_branch_challenges) > 0 and get_current_epoch(state) > validator.open_branch_challenges[0].inclusion_epoch + CHALLENGE_RESPONSE_DEADLINE:
penalize_validator(state, index, validator.open_branch_challenges[0].challenger_index)
```
In `process_penalties_and_exits`, change the definition of `eligible` to the following (note that it is not a pure function because `state` is declared in the surrounding scope):
```python
def eligible(index):
validator = state.validator_registry[index]
# Cannot exit if there are still open branch challenges
if len(validator.open_branch_challenges) > 0:
return False
# Cannot exit if you have not revealed all of your subkeys
elif validator.next_subkey_to_reveal <= epoch_to_custody_period(validator.exit_epoch):
return False
# Cannot exit if you already have
elif validator.withdrawable_epoch < FAR_FUTURE_EPOCH:
return False
# Return minimum time
else:
return current_epoch >= validator.exit_epoch + MIN_VALIDATOR_WITHDRAWAL_EPOCHS
```
## One-time phase 1 initiation transition
Run the following on the fork block after per-slot processing and before per-block and per-epoch processing.
For all `validator` in `ValidatorRegistry`, update it to the new format and fill the new member values with:
```python
'open_branch_challenges': [],
'next_subkey_to_reveal': get_current_custody_period(state),
'reveal_max_periods_late': 0,
```