status-im
/
eth2.0-specs
mirror of https://github.com/status-im/eth2.0-specs.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge branch 'master' into honest-validator

This commit is contained in:
Danny Ryan 2019-01-17 11:00:51 -06:00
parent 40d4ec33cb 4a5018e544
commit 8cdb8d9334
No known key found for this signature in database
GPG Key ID: 2765A792E42CE07A
2 changed files with 189 additions and 198 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

282
specs/core/0_beacon-chain.md
View File

@ -43,12 +43,13 @@
- [`ProposalSignedData`](#proposalsigneddata)
- [Beacon chain state](#beacon-chain-state)
- [`BeaconState`](#beaconstate)
- [`ValidatorRecord`](#validatorrecord)
- [`CrosslinkRecord`](#crosslinkrecord)
- [`DepositRootVote`](#depositrootvote)
- [`PendingAttestationRecord`](#pendingattestationrecord)
- [`ForkData`](#forkdata)
- [`Validator`](#validator)
- [`Crosslink`](#crosslink)
- [`PendingAttestation`](#pendingattestation)
- [`Fork`](#fork)
- [`ValidatorRegistryDeltaBlock`](#validatorregistrydeltablock)
- [`Eth1Data`](#eth1data)
- [`Eth1DataVote`](#eth1datavote)
- [Ethereum 1.0 deposit contract](#ethereum-10-deposit-contract)
- [Deposit arguments](#deposit-arguments)
- [Withdrawal credentials](#withdrawal-credentials)
@ -96,7 +97,7 @@
- [Slot](#slot)
- [Proposer signature](#proposer-signature)
- [RANDAO](#randao)
- [Deposit root](#deposit-root)
- [Eth1 data](#eth1-data)
- [Operations](#operations)
- [Proposer slashings](#proposer-slashings-1)
- [Casper slashings](#casper-slashings-1)
@ -106,7 +107,7 @@
- [Custody](#custody)
- [Per-epoch processing](#per-epoch-processing)
- [Helpers](#helpers)
- [Deposit roots](#deposit-roots)
- [Eth1 data](#eth1-data-1)
- [Justification](#justification)
- [Crosslinks](#crosslinks)
- [Rewards and penalties](#rewards-and-penalties)
@ -161,9 +162,8 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
| - | - | :-: |
| `SHARD_COUNT` | `2**10` (= 1,024) | shards |
| `TARGET_COMMITTEE_SIZE` | `2**7` (= 128) | [validators](#dfn-validator) |
| `EJECTION_BALANCE` | `2**4` (= 16) | ETH |
| `EJECTION_BALANCE` | `2**4 * 1e9` (= 16,000,000,000) | Gwei |
| `MAX_BALANCE_CHURN_QUOTIENT` | `2**5` (= 32) | - |
| `GWEI_PER_ETH` | `10**9` | Gwei/ETH |
| `BEACON_CHAIN_SHARD_NUMBER` | `2**64 - 1` | - |
| `MAX_CASPER_VOTES` | `2**10` (= 1,024) | votes |
| `LATEST_BLOCK_ROOTS_LENGTH` | `2**13` (= 8,192) | block roots |
@ -179,8 +179,8 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
| - | - | :-: |
| `DEPOSIT_CONTRACT_ADDRESS` | **TBD** |
| `DEPOSIT_CONTRACT_TREE_DEPTH` | `2**5` (= 32) | - |
| `MIN_DEPOSIT` | `2**0` (= 1) | ETH |
| `MAX_DEPOSIT` | `2**5` (= 32) | ETH |
| `MIN_DEPOSIT_AMOUNT` | `2**0 * 1e9` (= 1,000,000,000) | Gwei |
| `MAX_DEPOSIT_AMOUNT` | `2**5 * 1e9` (= 32,000,000,000) | Gwei |
### Initial values
@ -203,14 +203,14 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
| `EPOCH_LENGTH` | `2**6` (= 64) | slots | 6.4 minutes |
| `SEED_LOOKAHEAD` | `2**6` (= 64) | slots | 6.4 minutes |
| `ENTRY_EXIT_DELAY` | `2**8` (= 256) | slots | 25.6 minutes |
| `DEPOSIT_ROOT_VOTING_PERIOD` | `2**10` (= 1,024) | slots | ~1.7 hours |
| `ETH1_DATA_VOTING_PERIOD` | `2**10` (= 1,024) | slots | ~1.7 hours |
| `MIN_VALIDATOR_WITHDRAWAL_TIME` | `2**14` (= 16,384) | slots | ~27 hours |
### Reward and penalty quotients
| Name | Value |
| - | - |
| `BASE_REWARD_QUOTIENT` | `2**10` (= 1,024) |
| `BASE_REWARD_QUOTIENT` | `2**5` (= 32) |
| `WHISTLEBLOWER_REWARD_QUOTIENT` | `2**9` (= 512) |
| `INCLUDER_REWARD_QUOTIENT` | `2**3` (= 8) |
| `INACTIVITY_PENALTY_QUOTIENT` | `2**24` (= 16,777,216) |
@ -310,8 +310,8 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
{
# Attestation data
'data': AttestationData,
# Attester participation bitfield
'participation_bitfield': 'bytes',
# Attester aggregation bitfield
'aggregation_bitfield': 'bytes',
# Custody bitfield
'custody_bitfield': 'bytes',
# BLS aggregate signature
@ -424,7 +424,7 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
'parent_root': 'hash32',
'state_root': 'hash32',
'randao_reveal': 'hash32',
'deposit_root': 'hash32',
'eth1_data': Eth1Data,
'signature': ['uint384'],
## Body ##
@ -471,12 +471,12 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
# Misc
'slot': 'uint64',
'genesis_time': 'uint64',
'fork_data': ForkData, # For versioning hard forks
'fork': Fork, # For versioning hard forks
# Validator registry
'validator_registry': [ValidatorRecord],
'validator_registry': [Validator],
'validator_balances': ['uint64'],
'validator_registry_latest_change_slot': 'uint64',
'validator_registry_update_slot': 'uint64',
'validator_registry_exit_count': 'uint64',
'validator_registry_delta_chain_tip': 'hash32', # For light clients to track deltas
@ -500,19 +500,19 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
'finalized_slot': 'uint64',
# Recent state
'latest_crosslinks': [CrosslinkRecord],
'latest_crosslinks': [Crosslink],
'latest_block_roots': ['hash32'], # Needed to process attestations, older to newer
'latest_penalized_exit_balances': ['uint64'], # Balances penalized at every withdrawal period
'latest_attestations': [PendingAttestationRecord],
'latest_penalized_balances': ['uint64'], # Balances penalized at every withdrawal period
'latest_attestations': [PendingAttestation],
'batched_block_roots': ['hash32'],
# Ethereum 1.0 deposit root
'latest_deposit_root': 'hash32',
'deposit_root_votes': [DepositRootVote],
# Ethereum 1.0 chain data
'latest_eth1_data': Eth1Data,
'eth1_data_votes': [Eth1DataVote],
}
```
#### `ValidatorRecord`
#### `Validator`
```python
{
@ -545,7 +545,7 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
}
```
#### `CrosslinkRecord`
#### `Crosslink`
```python
{
@ -556,25 +556,14 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
}
```
#### `DepositRootVote`
```python
{
# Deposit root
'deposit_root': 'hash32',
# Vote count
'vote_count': 'uint64',
}
```
#### `PendingAttestationRecord`
#### `PendingAttestation`
```python
{
# Signed data
'data': AttestationData,
# Attester participation bitfield
'participation_bitfield': 'bytes',
# Attester aggregation bitfield
'aggregation_bitfield': 'bytes',
# Custody bitfield
'custody_bitfield': 'bytes',
# Slot the attestation was included
@ -582,16 +571,16 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
}
```
#### `ForkData`
#### `Fork`
```python
{
# Previous fork version
'pre_fork_version': 'uint64',
# Post fork version
'post_fork_version': 'uint64',
'previous_version': 'uint64',
# Current fork version
'current_version': 'uint64',
# Fork slot number
'fork_slot': 'uint64',
'slot': 'uint64',
}
```
@ -599,11 +588,33 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
```python
{
latest_registry_delta_root: 'hash32',
validator_index: 'uint24',
pubkey: 'uint384',
slot: 'uint64',
flag: 'uint64',
'latest_registry_delta_root': 'hash32',
'validator_index': 'uint24',
'pubkey': 'uint384',
'slot': 'uint64',
'flag': 'uint64',
}
```
#### `Eth1Data`
```python
{
# Root of the deposit tree
'deposit_root': 'hash32',
# Block hash
'block_hash': 'hash32',
}
```
#### `Eth1DataVote`
```python
{
# Data being voted for
'eth1_data': Eth1Data,
# Vote count
'vote_count': 'uint64',
}
```
@ -626,14 +637,14 @@ The private key corresponding to `withdrawal_pubkey` will be required to initiat
### `Deposit` logs
Every Ethereum 1.0 deposit, of size between `MIN_DEPOSIT` and `MAX_DEPOSIT`, emits a `Deposit` log for consumption by the beacon chain. The deposit contract does little validation, pushing most of the validator onboarding logic to the beacon chain. In particular, the proof of possession (a BLS signature) is not verified by the deposit contract.
Every Ethereum 1.0 deposit, of size between `MIN_DEPOSIT_AMOUNT` and `MAX_DEPOSIT_AMOUNT`, emits a `Deposit` log for consumption by the beacon chain. The deposit contract does little validation, pushing most of the validator onboarding logic to the beacon chain. In particular, the proof of possession (a BLS12 signature) is not verified by the deposit contract.
### `ChainStart` log
When sufficiently many full deposits have been made the deposit contract emits the `ChainStart` log. The beacon chain state may then be initialized by calling the `get_initial_beacon_state` function (defined below) where:
* `genesis_time` equals `time` in the `ChainStart` log
* `latest_deposit_root` equals `deposit_root` in the `ChainStart` log
* `latest_eth1_data.deposit_root` equals `deposit_root` in the `ChainStart` log, and `latest_eth1_data.block_hash` equals the hash of the block that included the log
* `initial_validator_deposits` is a list of `Deposit` objects built according to the `Deposit` logs up to the deposit that triggered the `ChainStart` log, processed in the order in which they were emitted (oldest to newest)
### Vyper code
@ -641,8 +652,8 @@ When sufficiently many full deposits have been made the deposit contract emits t
```python
## compiled with v0.1.0-beta.6 ##
MIN_DEPOSIT: constant(uint256) = 1 # ETH
MAX_DEPOSIT: constant(uint256) = 32 # ETH
MIN_DEPOSIT_AMOUNT: constant(uint256) = 1000000000 # Gwei
MAX_DEPOSIT_AMOUNT: constant(uint256) = 32000000000 # Gwei
GWEI_PER_ETH: constant(uint256) = 1000000000 # 10**9
CHAIN_START_FULL_DEPOSIT_THRESHOLD: constant(uint256) = 16384 # 2**14
DEPOSIT_CONTRACT_TREE_DEPTH: constant(uint256) = 32
@ -659,13 +670,13 @@ full_deposit_count: uint256
@payable
@public
def deposit(deposit_input: bytes[2048]):
assert msg.value >= as_wei_value(MIN_DEPOSIT, "ether")
assert msg.value <= as_wei_value(MAX_DEPOSIT, "ether")
assert msg.value >= as_wei_value(MIN_DEPOSIT_AMOUNT, "gwei")
assert msg.value <= as_wei_value(MAX_DEPOSIT_AMOUNT, "gwei")
index: uint256 = self.deposit_count + TWO_TO_POWER_OF_TREE_DEPTH
msg_gwei_bytes8: bytes[8] = slice(concat("", convert(msg.value / GWEI_PER_ETH, bytes32)), start=24, len=8)
timestamp_bytes8: bytes[8] = slice(concat("", convert(block.timestamp, bytes32)), start=24, len=8)
deposit_data: bytes[2064] = concat(msg_gwei_bytes8, timestamp_bytes8, deposit_input)
deposit_amount: bytes[8] = slice(concat("", convert(msg.value / GWEI_PER_ETH, bytes32)), start=24, len=8)
deposit_timestamp: bytes[8] = slice(concat("", convert(block.timestamp, bytes32)), start=24, len=8)
deposit_data: bytes[2064] = concat(deposit_amount, deposit_timestamp, deposit_input)
merkle_tree_index: bytes[8] = slice(concat("", convert(index, bytes32)), start=24, len=8)
log.Deposit(self.deposit_tree[1], deposit_data, merkle_tree_index)
@ -677,12 +688,12 @@ def deposit(deposit_input: bytes[2048]):
self.deposit_tree[index] = sha3(concat(self.deposit_tree[index * 2], self.deposit_tree[index * 2 + 1]))
self.deposit_count += 1
if msg.value == as_wei_value(MAX_DEPOSIT, "ether"):
if msg.value == as_wei_value(MAX_DEPOSIT_AMOUNT, "gwei"):
self.full_deposit_count += 1
if self.full_deposit_count == CHAIN_START_FULL_DEPOSIT_THRESHOLD:
timestamp_day_boundary: uint256 = as_unitless_number(block.timestamp) - as_unitless_number(block.timestamp) % SECONDS_PER_DAY + SECONDS_PER_DAY
timestamp_day_boundary_bytes8: bytes[8] = slice(concat("", convert(timestamp_day_boundary, bytes32)), start=24, len=8)
log.ChainStart(self.deposit_tree[1], timestamp_day_boundary_bytes8)
chainstart_time: bytes[8] = slice(concat("", convert(timestamp_day_boundary, bytes32)), start=24, len=8)
log.ChainStart(self.deposit_tree[1], chainstart_time)
@public
@constant
@ -714,7 +725,7 @@ For a beacon chain block, `block`, to be processed by a node, the following cond
* The parent block with root `block.parent_root` has been processed and accepted.
* The node has processed its `state` up to slot, `block.slot - 1`.
* An Ethereum 1.0 block pointed to by the `state.latest_deposit_root` has been processed and accepted.
* An Ethereum 1.0 block pointed to by the `state.latest_eth1_data.block_hash` has been processed and accepted.
* The node's local clock time is greater than or equal to `state.genesis_time + block.slot * SLOT_DURATION`.
If these conditions are not met, the client should delay processing the beacon block until the conditions are all satisfied.
@ -777,7 +788,7 @@ Note: We aim to migrate to a S[T/N]ARK-friendly hash function in a future Ethere
#### `is_active_validator`
```python
def is_active_validator(validator: ValidatorRecord, slot: int) -> bool:
def is_active_validator(validator: Validator, slot: int) -> bool:
"""
Checks if ``validator`` is active.
"""
@ -787,7 +798,7 @@ def is_active_validator(validator: ValidatorRecord, slot: int) -> bool:
#### `get_active_validator_indices`
```python
def get_active_validator_indices(validators: [ValidatorRecord], slot: int) -> List[int]:
def get_active_validator_indices(validators: [Validator], slot: int) -> List[int]:
"""
Gets indices of active validators from ``validators``.
"""
@ -850,7 +861,7 @@ def shuffle(values: List[Any], seed: Hash32) -> List[Any]:
#### `split`
```python
def split(values: List[Any], split_count: int) -> List[Any]:
def split(values: List[Any], split_count: int) -> List[List[Any]]:
"""
Splits ``values`` into ``split_count`` pieces.
"""
@ -878,7 +889,7 @@ def get_committee_count_per_slot(active_validator_count: int) -> int:
```python
def get_shuffling(seed: Hash32,
validators: List[ValidatorRecord],
validators: List[Validator],
slot: int) -> List[List[int]]
"""
Shuffles ``validators`` into crosslink committees seeded by ``seed`` and ``slot``.
@ -1025,9 +1036,9 @@ def merkle_root(values: List[Hash32]) -> Hash32:
```python
def get_attestation_participants(state: BeaconState,
attestation_data: AttestationData,
participation_bitfield: bytes) -> List[int]:
aggregation_bitfield: bytes) -> List[int]:
"""
Returns the participant indices at for the ``attestation_data`` and ``participation_bitfield``.
Returns the participant indices at for the ``attestation_data`` and ``aggregation_bitfield``.
"""
# Find the committee in the list with the desired shard
@ -1035,13 +1046,13 @@ def get_attestation_participants(state: BeaconState,
assert attestation_data.shard in [shard for _, shard in crosslink_committees]
crosslink_committee = [committee for committee, shard in crosslink_committees if shard == attestation_data.shard][0]
assert len(participation_bitfield) == (len(committee) + 7) // 8
assert len(aggregation_bitfield) == (len(committee) + 7) // 8
# Find the participating attesters in the committee
participants = []
for i, validator_index in enumerate(crosslink_committee):
participation_bit = (participation_bitfield[i//8] >> (7 - (i % 8))) % 2
if participation_bit == 1:
aggregation_bit = (aggregation_bitfield[i // 8] >> (7 - (i % 8))) % 2
if aggregation_bit == 1:
participants.append(validator_index)
return participants
```
@ -1057,28 +1068,28 @@ def get_effective_balance(state: State, index: int) -> int:
"""
Returns the effective balance (also known as "balance at stake") for a ``validator`` with the given ``index``.
"""
return min(state.validator_balances[index], MAX_DEPOSIT * GWEI_PER_ETH)
return min(state.validator_balances[index], MAX_DEPOSIT_AMOUNT)
```
#### `get_fork_version`
```python
def get_fork_version(fork_data: ForkData,
def get_fork_version(fork: Fork,
slot: int) -> int:
if slot < fork_data.fork_slot:
return fork_data.pre_fork_version
if slot < fork.slot:
return fork.previous_version
else:
return fork_data.post_fork_version
return fork.current_version
```
#### `get_domain`
```python
def get_domain(fork_data: ForkData,
def get_domain(fork: Fork,
slot: int,
domain_type: int) -> int:
return get_fork_version(
fork_data,
fork,
slot
) * 2**32 + domain_type
```
@ -1092,8 +1103,8 @@ def verify_slashable_vote_data(state: BeaconState, vote_data: SlashableVoteData)
return bls_verify_multiple(
pubkeys=[
bls_aggregate_pubkey([state.validators[i].pubkey for i in vote_data.custody_bit_0_indices]),
bls_aggregate_pubkey([state.validators[i].pubkey for i in vote_data.custody_bit_1_indices]),
bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in vote_data.custody_bit_0_indices]),
bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in vote_data.custody_bit_1_indices]),
],
messages=[
hash_tree_root(AttestationDataAndCustodyBit(vote_data.data, False)),
@ -1101,7 +1112,7 @@ def verify_slashable_vote_data(state: BeaconState, vote_data: SlashableVoteData)
],
signature=vote_data.aggregate_signature,
domain=get_domain(
state.fork_data,
state.fork,
state.slot,
DOMAIN_ATTESTATION,
),
@ -1184,7 +1195,10 @@ A valid block with slot `GENESIS_SLOT` (a "genesis block") has the following val
parent_root=ZERO_HASH,
state_root=STARTUP_STATE_ROOT,
randao_reveal=ZERO_HASH,
deposit_root=ZERO_HASH,
eth1_data=Eth1Data(
deposit_root=ZERO_HASH,
block_hash=ZERO_HASH
),
signature=EMPTY_SIGNATURE,
body=BeaconBlockBody(
proposer_slashings=[],
@ -1204,21 +1218,21 @@ A valid block with slot `GENESIS_SLOT` (a "genesis block") has the following val
```python
def get_initial_beacon_state(initial_validator_deposits: List[Deposit],
genesis_time: int,
latest_deposit_root: Hash32) -> BeaconState:
latest_eth1_data: Eth1Data) -> BeaconState:
state = BeaconState(
# Misc
slot=GENESIS_SLOT,
genesis_time=genesis_time,
fork_data=ForkData(
pre_fork_version=GENESIS_FORK_VERSION,
post_fork_version=GENESIS_FORK_VERSION,
fork_slot=GENESIS_SLOT,
fork=Fork(
previous_version=GENESIS_FORK_VERSION,
current_version=GENESIS_FORK_VERSION,
slot=GENESIS_SLOT,
),
# Validator registry
validator_registry=[],
validator_balances=[],
validator_registry_latest_change_slot=GENESIS_SLOT,
validator_registry_update_slot=GENESIS_SLOT,
validator_registry_exit_count=0,
validator_registry_delta_chain_tip=ZERO_HASH,
@ -1242,15 +1256,15 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit],
finalized_slot=GENESIS_SLOT,
# Recent state
latest_crosslinks=[CrosslinkRecord(slot=GENESIS_SLOT, shard_block_root=ZERO_HASH) for _ in range(SHARD_COUNT)],
latest_crosslinks=[Crosslink(slot=GENESIS_SLOT, shard_block_root=ZERO_HASH) for _ in range(SHARD_COUNT)],
latest_block_roots=[ZERO_HASH for _ in range(LATEST_BLOCK_ROOTS_LENGTH)],
latest_penalized_exit_balances=[0 for _ in range(LATEST_PENALIZED_EXIT_LENGTH)],
latest_penalized_balances=[0 for _ in range(LATEST_PENALIZED_EXIT_LENGTH)],
latest_attestations=[],
batched_block_roots=[],
# Deposit root
latest_deposit_root=latest_deposit_root,
deposit_root_votes=[],
# Ethereum 1.0 chain data
latest_eth1_data=latest_eth1_data,
eth1_data_votes=[],
)
# Process initial deposits
@ -1267,7 +1281,7 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit],
# Process initial activations
for validator_index, _ in enumerate(state.validator_registry):
if get_effective_balance(state, validator_index) >= MAX_DEPOSIT * GWEI_PER_ETH:
if get_effective_balance(state, validator_index) >= MAX_DEPOSIT_AMOUNT:
activate_validator(state, validator_index, True)
return state
@ -1297,7 +1311,7 @@ def validate_proof_of_possession(state: BeaconState,
message=hash_tree_root(proof_of_possession_data),
signature=proof_of_possession,
domain=get_domain(
state.fork_data,
state.fork,
state.slot,
DOMAIN_DEPOSIT,
)
@ -1332,7 +1346,7 @@ def process_deposit(state: BeaconState,
if pubkey not in validator_pubkeys:
# Add new validator
validator = ValidatorRecord(
validator = Validator(
pubkey=pubkey,
withdrawal_credentials=withdrawal_credentials,
randao_commitment=randao_commitment,
@ -1413,7 +1427,7 @@ def exit_validator(state: BeaconState, index: int) -> None:
def penalize_validator(state: BeaconState, index: int) -> None:
exit_validator(state, index)
validator = state.validator_registry[index]
state.latest_penalized_exit_balances[(state.slot // EPOCH_LENGTH) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index)
state.latest_penalized_balances[(state.slot // EPOCH_LENGTH) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index)
whistleblower_index = get_beacon_proposer_index(state, state.slot)
whistleblower_reward = get_effective_balance(state, index) // WHISTLEBLOWER_REWARD_QUOTIENT
@ -1456,7 +1470,7 @@ Below are the processing steps that happen at every `block`.
* Let `block_without_signature_root` be the `hash_tree_root` of `block` where `block.signature` is set to `EMPTY_SIGNATURE`.
* Let `proposal_root = hash_tree_root(ProposalSignedData(state.slot, BEACON_CHAIN_SHARD_NUMBER, block_without_signature_root))`.
* Verify that `bls_verify(pubkey=state.validator_registry[get_beacon_proposer_index(state, state.slot)].pubkey, message=proposal_root, signature=block.signature, domain=get_domain(state.fork_data, state.slot, DOMAIN_PROPOSAL))`.
* Verify that `bls_verify(pubkey=state.validator_registry[get_beacon_proposer_index(state, state.slot)].pubkey, message=proposal_root, signature=block.signature, domain=get_domain(state.fork, state.slot, DOMAIN_PROPOSAL))`.
### RANDAO
@ -1467,10 +1481,10 @@ Below are the processing steps that happen at every `block`.
* Set `proposer.randao_commitment = block.randao_reveal`.
* Set `proposer.randao_layers = 0`.
### Deposit root
### Eth1 data
* If `block.deposit_root` is `deposit_root_vote.deposit_root` for some `deposit_root_vote` in `state.deposit_root_votes`, set `deposit_root_vote.vote_count += 1`.
* Otherwise, append to `state.deposit_root_votes` a new `DepositRootVote(deposit_root=block.deposit_root, vote_count=1)`.
* If `block.eth1_data` equals `eth1_data_vote.eth1_data` for some `eth1_data_vote` in `state.eth1_data_votes`, set `eth1_data_vote.vote_count += 1`.
* Otherwise, append to `state.eth1_data_votes` a new `Eth1DataVote(eth1_data=block.eth1_data, vote_count=1)`.
### Operations
@ -1485,8 +1499,8 @@ For each `proposer_slashing` in `block.body.proposer_slashings`:
* Verify that `proposer_slashing.proposal_data_1.shard == proposer_slashing.proposal_data_2.shard`.
* Verify that `proposer_slashing.proposal_data_1.block_root != proposer_slashing.proposal_data_2.block_root`.
* Verify that `proposer.penalized_slot > state.slot`.
* Verify that `bls_verify(pubkey=proposer.pubkey, message=hash_tree_root(proposer_slashing.proposal_data_1), signature=proposer_slashing.proposal_signature_1, domain=get_domain(state.fork_data, proposer_slashing.proposal_data_1.slot, DOMAIN_PROPOSAL))`.
* Verify that `bls_verify(pubkey=proposer.pubkey, message=hash_tree_root(proposer_slashing.proposal_data_2), signature=proposer_slashing.proposal_signature_2, domain=get_domain(state.fork_data, proposer_slashing.proposal_data_2.slot, DOMAIN_PROPOSAL))`.
* Verify that `bls_verify(pubkey=proposer.pubkey, message=hash_tree_root(proposer_slashing.proposal_data_1), signature=proposer_slashing.proposal_signature_1, domain=get_domain(state.fork, proposer_slashing.proposal_data_1.slot, DOMAIN_PROPOSAL))`.
* Verify that `bls_verify(pubkey=proposer.pubkey, message=hash_tree_root(proposer_slashing.proposal_data_2), signature=proposer_slashing.proposal_signature_2, domain=get_domain(state.fork, proposer_slashing.proposal_data_2.slot, DOMAIN_PROPOSAL))`.
* Run `penalize_validator(state, proposer_slashing.proposer_index)`.
#### Casper slashings
@ -1518,11 +1532,11 @@ For each `attestation` in `block.body.attestations`:
* Verify that `attestation.data.justified_block_root` is equal to `get_block_root(state, attestation.data.justified_slot)`.
* Verify that either `attestation.data.latest_crosslink_root` or `attestation.data.shard_block_root` equals `state.latest_crosslinks[shard].shard_block_root`.
* `aggregate_signature` verification:
* Let `participants = get_attestation_participants(state, attestation.data, attestation.participation_bitfield)`.
* Let `participants = get_attestation_participants(state, attestation.data, attestation.aggregation_bitfield)`.
* Let `group_public_key = bls_aggregate_pubkeys([state.validator_registry[v].pubkey for v in participants])`.
* Verify that `bls_verify(pubkey=group_public_key, message=hash_tree_root(AttestationDataAndCustodyBit(attestation.data, False)), signature=attestation.aggregate_signature, domain=get_domain(state.fork_data, attestation.data.slot, DOMAIN_ATTESTATION))`.
* Verify that `bls_verify(pubkey=group_public_key, message=hash_tree_root(AttestationDataAndCustodyBit(attestation.data, False)), signature=attestation.aggregate_signature, domain=get_domain(state.fork, attestation.data.slot, DOMAIN_ATTESTATION))`.
* [TO BE REMOVED IN PHASE 1] Verify that `attestation.data.shard_block_root == ZERO_HASH`.
* Append `PendingAttestationRecord(data=attestation.data, participation_bitfield=attestation.participation_bitfield, custody_bitfield=attestation.custody_bitfield, slot_included=state.slot)` to `state.latest_attestations`.
* Append `PendingAttestation(data=attestation.data, aggregation_bitfield=attestation.aggregation_bitfield, custody_bitfield=attestation.custody_bitfield, slot_included=state.slot)` to `state.latest_attestations`.
#### Deposits
@ -1534,7 +1548,7 @@ Verify that `len(block.body.deposits) <= MAX_DEPOSITS`.
For each `deposit` in `block.body.deposits`:
* Let `serialized_deposit_data` be the serialized form of `deposit.deposit_data`. It should be 8 bytes for `deposit_data.amount` followed by 8 bytes for `deposit_data.timestamp` and then the `DepositInput` bytes. That is, it should match `deposit_data` in the [Ethereum 1.0 deposit contract](#ethereum-10-deposit-contract) of which the hash was placed into the Merkle tree.
* Verify that `verify_merkle_branch(hash(serialized_deposit_data), deposit.branch, DEPOSIT_CONTRACT_TREE_DEPTH, deposit.index, state.latest_deposit_root)` is `True`.
* Verify that `verify_merkle_branch(hash(serialized_deposit_data), deposit.branch, DEPOSIT_CONTRACT_TREE_DEPTH, deposit.index, state.latest_eth1_data.deposit_root)` is `True`.
```python
def verify_merkle_branch(leaf: Hash32, branch: [Hash32], depth: int, index: int, root: Hash32) -> bool:
@ -1570,7 +1584,7 @@ For each `exit` in `block.body.exits`:
* Let `validator = state.validator_registry[exit.validator_index]`.
* Verify that `validator.exit_slot > state.slot + ENTRY_EXIT_DELAY`.
* Verify that `state.slot >= exit.slot`.
* Verify that `bls_verify(pubkey=validator.pubkey, message=ZERO_HASH, signature=exit.signature, domain=get_domain(state.fork_data, exit.slot, DOMAIN_EXIT))`.
* Verify that `bls_verify(pubkey=validator.pubkey, message=ZERO_HASH, signature=exit.signature, domain=get_domain(state.fork, exit.slot, DOMAIN_EXIT))`.
* Run `initiate_validator_exit(state, exit.validator_index)`.
#### Custody
@ -1593,25 +1607,25 @@ All [validators](#dfn-validator):
* Let `current_epoch_attestations = [a for a in state.latest_attestations if state.slot - EPOCH_LENGTH <= a.data.slot < state.slot]`. (Note: this is the set of attestations of slots in the epoch `state.slot-EPOCH_LENGTH...state.slot-1`, _not_ attestations that got included in the chain during the epoch `state.slot-EPOCH_LENGTH...state.slot-1`.)
* Validators justifying the epoch boundary block at the start of the current epoch:
* Let `current_epoch_boundary_attestations = [a for a in current_epoch_attestations if a.data.epoch_boundary_root == get_block_root(state, state.slot-EPOCH_LENGTH) and a.justified_slot == state.justified_slot]`.
* Let `current_epoch_boundary_attester_indices` be the union of the [validator](#dfn-validator) index sets given by `[get_attestation_participants(state, a.data, a.participation_bitfield) for a in current_epoch_boundary_attestations]`.
* Let `current_epoch_boundary_attester_indices` be the union of the [validator](#dfn-validator) index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in current_epoch_boundary_attestations]`.
* Let `current_epoch_boundary_attesting_balance = sum([get_effective_balance(state, i) for i in current_epoch_boundary_attester_indices])`.
[Validators](#dfn-Validator) attesting during the previous epoch:
* Validators that made an attestation during the previous epoch:
* Let `previous_epoch_attestations = [a for a in state.latest_attestations if state.slot - 2 * EPOCH_LENGTH <= a.slot < state.slot - EPOCH_LENGTH]`.
* Let `previous_epoch_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.participation_bitfield) for a in previous_epoch_attestations]`.
* Let `previous_epoch_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_attestations]`.
* Validators targeting the previous justified slot:
* Let `previous_epoch_justified_attestations = [a for a in current_epoch_attestations + previous_epoch_attestations if a.justified_slot == state.previous_justified_slot]`.
* Let `previous_epoch_justified_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.participation_bitfield) for a in previous_epoch_justified_attestations]`.
* Let `previous_epoch_justified_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_justified_attestations]`.
* Let `previous_epoch_justified_attesting_balance = sum([get_effective_balance(state, i) for i in previous_epoch_justified_attester_indices])`.
* Validators justifying the epoch boundary block at the start of the previous epoch:
* Let `previous_epoch_boundary_attestations = [a for a in previous_epoch_justified_attestations if a.epoch_boundary_root == get_block_root(state, state.slot - 2 * EPOCH_LENGTH)]`.
* Let `previous_epoch_boundary_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.participation_bitfield) for a in previous_epoch_boundary_attestations]`.
* Let `previous_epoch_boundary_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_boundary_attestations]`.
* Let `previous_epoch_boundary_attesting_balance = sum([get_effective_balance(state, i) for i in previous_epoch_boundary_attester_indices])`.
* Validators attesting to the expected beacon chain head during the previous epoch:
* Let `previous_epoch_head_attestations = [a for a in previous_epoch_attestations if a.beacon_block_root == get_block_root(state, a.slot)]`.
* Let `previous_epoch_head_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.participation_bitfield) for a in previous_epoch_head_attestations]`.
* Let `previous_epoch_head_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_head_attestations]`.
* Let `previous_epoch_head_attesting_balance = sum([get_effective_balance(state, i) for i in previous_epoch_head_attester_indices])`.
**Note**: `previous_epoch_boundary_attesting_balance` balance might be marginally different than `current_epoch_boundary_attesting_balance` during the previous epoch transition. Due to the tight bound on validator churn each epoch and small per-epoch rewards/penalties, the potential balance difference is very low and only marginally affects consensus safety.
@ -1619,7 +1633,7 @@ All [validators](#dfn-validator):
For every `slot in range(state.slot - 2 * EPOCH_LENGTH, state.slot)`, let `crosslink_committee_at_slot = get_crosslink_committees_at_slot(slot)`. For every `(crosslink_committee, shard)` in `crosslink_committee_at_slot`, compute:
* Let `shard_block_root` be `state.latest_crosslinks[shard].shard_block_root`
* Let `attesting_validator_indices(crosslink_committee, shard_block_root)` be the union of the [validator](#dfn-validator) index sets given by `[get_attestation_participants(state, a.data, a.participation_bitfield) for a in current_epoch_attestations + previous_epoch_attestations if a.shard == shard and a.shard_block_root == shard_block_root]`.
* Let `attesting_validator_indices(crosslink_committee, shard_block_root)` be the union of the [validator](#dfn-validator) index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in current_epoch_attestations + previous_epoch_attestations if a.shard == shard and a.shard_block_root == shard_block_root]`.
* Let `winning_root(crosslink_committee)` be equal to the value of `shard_block_root` such that `sum([get_effective_balance(state, i) for i in attesting_validator_indices(crosslink_committee, shard_block_root)])` is maximized (ties broken by favoring lower `shard_block_root` values).
* Let `attesting_validators(crosslink_committee)` be equal to `attesting_validator_indices(crosslink_committee, winning_root(crosslink_committee))` for convenience.
* Let `total_attesting_balance(crosslink_committee) = sum([get_effective_balance(state, i) for i in attesting_validators(crosslink_committee)])`.
@ -1627,15 +1641,15 @@ For every `slot in range(state.slot - 2 * EPOCH_LENGTH, state.slot)`, let `cross
Define the following helpers to process attestation inclusion rewards and inclusion distance reward/penalty. For every attestation `a` in `previous_epoch_attestations`:
* Let `inclusion_slot(state, index) = a.slot_included` for the attestation `a` where `index` is in `get_attestation_participants(state, a.data, a.participation_bitfield)`.
* Let `inclusion_slot(state, index) = a.slot_included` for the attestation `a` where `index` is in `get_attestation_participants(state, a.data, a.aggregation_bitfield)`.
* Let `inclusion_distance(state, index) = a.slot_included - a.data.slot` where `a` is the above attestation.
### Deposit roots
### Eth1 data
If `state.slot % DEPOSIT_ROOT_VOTING_PERIOD == 0`:
If `state.slot % ETH1_DATA_VOTING_PERIOD == 0`:
* Set `state.latest_deposit_root = deposit_root_vote.deposit_root` if `deposit_root_vote.vote_count * 2 > DEPOSIT_ROOT_VOTING_PERIOD` for some `deposit_root_vote` in `state.deposit_root_votes`.
* Set `state.deposit_root_votes = []`.
* Set `state.latest_eth1_data = eth1_data_vote.data` if `eth1_data_vote.vote_count * 2 > ETH1_DATA_VOTING_PERIOD` for some `eth1_data_vote` in `state.eth1_data_votes`.
* Set `state.eth1_data_votes = []`.
### Justification
@ -1654,13 +1668,13 @@ Set `state.finalized_slot = state.previous_justified_slot` if any of the followi
For every `slot in range(state.slot - 2 * EPOCH_LENGTH, state.slot)`, let `crosslink_committee_at_slot = get_crosslink_committees_at_slot(slot)`. For every `(crosslink_committee, shard)` in `crosslink_committee_at_slot`, compute:
* Set `state.latest_crosslinks[shard] = CrosslinkRecord(slot=state.slot, shard_block_root=winning_root(crosslink_committee))` if `3 * total_attesting_balance(crosslink_committee) >= 2 * total_balance(crosslink_committee)`.
* Set `state.latest_crosslinks[shard] = Crosslink(slot=state.slot, shard_block_root=winning_root(crosslink_committee))` if `3 * total_attesting_balance(crosslink_committee) >= 2 * total_balance(crosslink_committee)`.
### Rewards and penalties
First, we define some additional helpers:
* Let `base_reward_quotient = BASE_REWARD_QUOTIENT * integer_squareroot(total_balance // GWEI_PER_ETH)`.
* Let `base_reward_quotient = integer_squareroot(total_balance) // BASE_REWARD_QUOTIENT`.
* Let `base_reward(state, index) = get_effective_balance(state, index) // base_reward_quotient // 5` for any validator with the given `index`.
* Let `inactivity_penalty(state, index, epochs_since_finality) = base_reward(state, index) + get_effective_balance(state, index) * epochs_since_finality // INACTIVITY_PENALTY_QUOTIENT // 2` for any validator with the given `index`.
@ -1713,8 +1727,8 @@ def process_ejections(state: BeaconState) -> None:
Iterate through the validator registry
and eject active validators with balance below ``EJECTION_BALANCE``.
"""
for index in active_validator_indices(state.validator_registry):
if state.validator_balances[index] < EJECTION_BALANCE * GWEI_PER_ETH:
for index in get_active_validator_indices(state.validator_registry, state.slot):
if state.validator_balances[index] < EJECTION_BALANCE:
exit_validator(state, index)
```
@ -1722,8 +1736,8 @@ def process_ejections(state: BeaconState) -> None:
If the following are satisfied:
* `state.finalized_slot > state.validator_registry_latest_change_slot`
* `state.latest_crosslinks[shard].slot > state.validator_registry_latest_change_slot` for every shard number `shard` in `[(state.current_epoch_start_shard + i) % SHARD_COUNT for i in range(get_current_epoch_committee_count_per_slot(state) * EPOCH_LENGTH)]` (that is, for every shard in the current committees)
* `state.finalized_slot > state.validator_registry_update_slot`
* `state.latest_crosslinks[shard].slot > state.validator_registry_update_slot` for every shard number `shard` in `[(state.current_epoch_start_shard + i) % SHARD_COUNT for i in range(get_current_epoch_committee_count_per_slot(state) * EPOCH_LENGTH)]` (that is, for every shard in the current committees)
update the validator registry and associated fields by running
@ -1740,14 +1754,14 @@ def update_validator_registry(state: BeaconState) -> None:
# The maximum balance churn in Gwei (for deposits and exits separately)
max_balance_churn = max(
MAX_DEPOSIT * GWEI_PER_ETH,
MAX_DEPOSIT_AMOUNT,
total_balance // (2 * MAX_BALANCE_CHURN_QUOTIENT)
)
# Activate validators within the allowable balance churn
balance_churn = 0
for index, validator in enumerate(state.validator_registry):
if validator.activation_slot > state.slot + ENTRY_EXIT_DELAY and state.validator_balances[index] >= MAX_DEPOSIT * GWEI_PER_ETH:
if validator.activation_slot > state.slot + ENTRY_EXIT_DELAY and state.validator_balances[index] >= MAX_DEPOSIT_AMOUNT:
# Check the balance churn would be within the allowance
balance_churn += get_effective_balance(state, index)
if balance_churn > max_balance_churn:
@ -1768,7 +1782,7 @@ def update_validator_registry(state: BeaconState) -> None:
# Exit validator
exit_validator(state, index)
state.validator_registry_latest_change_slot = state.slot
state.validator_registry_update_slot = state.slot
```
and perform the following updates:
@ -1784,7 +1798,7 @@ If a validator registry update does _not_ happen do the following:
* Set `state.previous_epoch_calculation_slot = state.current_epoch_calculation_slot`
* Set `state.previous_epoch_start_shard = state.current_epoch_start_shard`
* Let `epochs_since_last_registry_change = (state.slot - state.validator_registry_latest_change_slot) // EPOCH_LENGTH`.
* Let `epochs_since_last_registry_change = (state.slot - state.validator_registry_update_slot) // EPOCH_LENGTH`.
* If `epochs_since_last_registry_change` is an exact power of 2, set `state.current_epoch_calculation_slot = state.slot` and `state.current_epoch_randao_mix = state.latest_randao_mixes[(state.current_epoch_calculation_slot - SEED_LOOKAHEAD) % LATEST_RANDAO_MIXES_LENGTH]`. Note that `state.current_epoch_start_shard` is left unchanged.
Regardless of whether or not a validator set change happens, run the following:
@ -1799,8 +1813,8 @@ def process_penalties_and_exits(state: BeaconState) -> None:
for index, validator in enumerate(state.validator_registry):
if (state.slot // EPOCH_LENGTH) == (validator.penalized_slot // EPOCH_LENGTH) + LATEST_PENALIZED_EXIT_LENGTH // 2:
e = (state.slot // EPOCH_LENGTH) % LATEST_PENALIZED_EXIT_LENGTH
total_at_start = state.latest_penalized_exit_balances[(e + 1) % LATEST_PENALIZED_EXIT_LENGTH]
total_at_end = state.latest_penalized_exit_balances[e]
total_at_start = state.latest_penalized_balances[(e + 1) % LATEST_PENALIZED_EXIT_LENGTH]
total_at_end = state.latest_penalized_balances[e]
total_penalties = total_at_end - total_at_start
penalty = get_effective_balance(state, index) * min(total_penalties * 3, total_balance) // total_balance
state.validator_balances[index] -= penalty
@ -1826,7 +1840,7 @@ def process_penalties_and_exits(state: BeaconState) -> None:
### Final updates
* Let `e = state.slot // EPOCH_LENGTH`. Set `state.latest_penalized_exit_balances[(e+1) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_exit_balances[e % LATEST_PENALIZED_EXIT_LENGTH]`
* Let `e = state.slot // EPOCH_LENGTH`. Set `state.latest_penalized_balances[(e+1) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[e % LATEST_PENALIZED_EXIT_LENGTH]`
* Remove any `attestation` in `state.latest_attestations` such that `attestation.data.slot < state.slot - EPOCH_LENGTH`.
## State root processing

105
specs/simple-serialize.md
View File

@ -15,20 +15,24 @@ deserializing objects and data types.
+ [Serialize/Encode](#serializeencode)
- [uint](#uint)
- [Bool](#bool)
- [Address](#address)
- [Hash](#hash)
- [Bytes](#bytes)
- [bytesN](#bytesn)
- [bytes](#bytes-1)
- [List/Vectors](#listvectors)
- [Container](#container)
+ [Deserialize/Decode](#deserializedecode)
- [uint](#uint-1)
- [Bool](#bool-1)
- [Address](#address-1)
- [Hash](#hash-1)
- [Bytes](#bytes-1)
- [Bytes](#bytes-2)
- [bytesN](#bytesn-1)
- [bytes](#bytes-1)
- [List/Vectors](#listvectors-1)
- [Container](#container-1)
+ [Tree Hash](#tree-hash)
- [`uint8`..`uint256`, `bool`, `bytes1`..`bytes32`](#uint8uint256-bool-bytes1bytes32)
- [`uint264`..`uintN`, `bytes`, `bytes33`..`bytesN`](#uint264uintn-bytes-bytes33bytesn)
- [List/Vectors](#listvectors-2)
- [Container](#container-2)
* [Implementations](#implementations)
## About
@ -43,9 +47,9 @@ overhead.
| Term | Definition |
|:-------------|:-----------------------------------------------------------------------------------------------|
| `big` | Big Endian |
| `byte_order` | Specifies [endianness:](https://en.wikipedia.org/wiki/Endianness) Big Endian or Little Endian. |
| `len` | Length/Number of Bytes. |
| `little` | Little endian. |
| `byte_order` | Specifies [endianness](https://en.wikipedia.org/wiki/Endianness): big endian or little endian. |
| `len` | Length/number of bytes. |
| `to_bytes` | Convert to bytes. Should take parameters ``size`` and ``byte_order``. |
| `from_bytes` | Convert from bytes to object. Should take ``bytes`` and ``byte_order``. |
| `value` | The value to serialize. |
@ -58,7 +62,6 @@ overhead.
| `LENGTH_BYTES` | 4 | Number of bytes used for the length added before a variable-length serialized object. |
| `SSZ_CHUNK_SIZE` | 128 | Number of bytes for the chunk size of the Merkle tree leaf. |
## Overview
### Serialize/Encode
@ -72,7 +75,7 @@ overhead.
Convert directly to bytes the size of the int. (e.g. ``uint16 = 2 bytes``)
All integers are serialized as **big endian**.
All integers are serialized as **little endian**.
| Check to perform | Code |
|:-----------------------|:----------------------|
@ -81,7 +84,7 @@ All integers are serialized as **big endian**.
```python
assert(int_size % 8 == 0)
buffer_size = int_size / 8
return value.to_bytes(buffer_size, 'big')
return value.to_bytes(buffer_size, 'little')
```
#### Bool
@ -97,32 +100,18 @@ assert(value in (True, False))
return b'\x01' if value is True else b'\x00'
```
#### Bytes
#### Address
The `address` should already come as a hash/byte format. Ensure that length is **20**.
| Check to perform | Code |
|:-----------------------|:---------------------|
| Length is correct (20) | ``len(value) == 20`` |
```python
assert( len(value) == 20 )
return value
```
#### Hash
| Hash Type | Usage |
|:---------:|:------------------------------------------------|
| `hashN` | Hash of arbitrary byte length `N`. |
| Bytes Type | Usage |
|:---------:|:------------------------------------|
| `bytesN` | Explicit length `N` bytes data. |
| `bytes` | Bytes data with arbitrary length. |
##### bytesN
| Checks to perform | Code |
|:---------------------------------------|:---------------------|
| Length in bytes is correct for `hashN` | ``len(value) == N`` |
##### hashN
| Length in bytes is correct for `bytesN` | ``len(value) == N`` |
```python
assert(len(value) == N)
@ -130,8 +119,7 @@ assert(len(value) == N)
return value
```
#### Bytes
##### bytes
For general `bytes` type:
1. Get the length/number of bytes; Encode into a `4-byte` integer.
2. Append the value to the length and return: ``[ length_bytes ] + [ value_bytes ]``
@ -142,7 +130,7 @@ For general `bytes` type:
```python
assert(len(value) < 2**32)
byte_length = (len(value)).to_bytes(LENGTH_BYTES, 'big')
byte_length = (len(value)).to_bytes(LENGTH_BYTES, 'little')
return byte_length + value
```
@ -156,7 +144,7 @@ Lists are a collection of elements of the same homogeneous type.
1. Get the number of raw bytes to serialize: it is ``len(list) * sizeof(element)``.
* Encode that as a `4-byte` **big endian** `uint32`.
* Encode that as a `4-byte` **little endian** `uint32`.
2. Append the elements in a packed manner.
* *Note on efficiency*: consider using a container that does not need to iterate over all elements to get its length. For example Python lists, C++ vectors or Rust Vec.
@ -172,7 +160,7 @@ for item in value:
assert(len(serialized_list_string) < 2**32)
serialized_len = (len(serialized_list_string).to_bytes(LENGTH_BYTES, 'big'))
serialized_len = (len(serialized_list_string).to_bytes(LENGTH_BYTES, 'little'))
return serialized_len + serialized_list_string
```
@ -181,7 +169,7 @@ return serialized_len + serialized_list_string
A container represents a heterogenous, associative collection of key-value pairs. Each pair is referred to as a `field`. To get the value for a given field, you supply the key which is a symbol unique to the container referred to as the field's `name`. The container data type is analogous to the `struct` type found in many languages like C or Go.
To serialize a container, obtain the list of its field's names in the specified order. For each field name in this list, obtain the corresponding value and serialize it. Tightly pack the complete set of serialized values in the same order as the field names into a buffer. Calculate the size of this buffer of serialized bytes and encode as a `4-byte` **big endian** `uint32`. Prepend the encoded length to the buffer. The result of this concatenation is the final serialized value of the container.
To serialize a container, obtain the list of its field's names in the specified order. For each field name in this list, obtain the corresponding value and serialize it. Tightly pack the complete set of serialized values in the same order as the field names into a buffer. Calculate the size of this buffer of serialized bytes and encode as a `4-byte` **little endian** `uint32`. Prepend the encoded length to the buffer. The result of this concatenation is the final serialized value of the container.
| Check to perform | Code |
@ -194,7 +182,7 @@ To serialize:
2. For each name in the list, obtain the corresponding value from the container and serialize it. Place this serialized value into a buffer. The serialized values should be tightly packed.
3. Get the number of raw bytes in the serialized buffer. Encode that number as a `4-byte` **big endian** `uint32`.
3. Get the number of raw bytes in the serialized buffer. Encode that number as a `4-byte` **little endian** `uint32`.
4. Prepend the length to the serialized buffer.
@ -220,7 +208,7 @@ for field_name in get_field_names(typ):
assert(len(serialized_buffer) < 2**32)
serialized_len = (len(serialized_buffer).to_bytes(LENGTH_BYTES, 'big'))
serialized_len = (len(serialized_buffer).to_bytes(LENGTH_BYTES, 'little'))
return serialized_len + serialized_buffer
```
@ -245,13 +233,13 @@ At each step, the following checks should be made:
Convert directly from bytes into integer utilising the number of bytes the same
size as the integer length. (e.g. ``uint16 == 2 bytes``)
All integers are interpreted as **big endian**.
All integers are interpreted as **little endian**.
```python
byte_length = int_size / 8
new_index = current_index + byte_length
assert(len(rawbytes) >= new_index)
return int.from_bytes(rawbytes[current_index:current_index+byte_length], 'big'), new_index
return int.from_bytes(rawbytes[current_index:current_index+byte_length], 'little'), new_index
```
#### Bool
@ -263,19 +251,9 @@ assert rawbytes in (b'\x00', b'\x01')
return True if rawbytes == b'\x01' else False
```
#### Address
#### Bytes
Return the 20-byte deserialized address.
```python
assert(len(rawbytes) >= current_index + 20)
new_index = current_index + 20
return rawbytes[current_index:current_index+20], new_index
```
#### Hash
##### hashN
##### bytesN
Return the `N` bytes.
@ -285,7 +263,7 @@ new_index = current_index + N
return rawbytes[current_index:current_index+N], new_index
```
#### Bytes
##### bytes
Get the length of the bytes, return the bytes.
@ -296,7 +274,7 @@ Get the length of the bytes, return the bytes.
```python
assert(len(rawbytes) > current_index + LENGTH_BYTES)
bytes_length = int.from_bytes(rawbytes[current_index:current_index + LENGTH_BYTES], 'big')
bytes_length = int.from_bytes(rawbytes[current_index:current_index + LENGTH_BYTES], 'little')
bytes_start = current_index + LENGTH_BYTES
bytes_end = bytes_start + bytes_length
@ -322,7 +300,7 @@ entire length of the list.
```python
assert(len(rawbytes) > current_index + LENGTH_BYTES)
total_length = int.from_bytes(rawbytes[current_index:current_index + LENGTH_BYTES], 'big')
total_length = int.from_bytes(rawbytes[current_index:current_index + LENGTH_BYTES], 'little')
new_index = current_index + LENGTH_BYTES + total_length
assert(len(rawbytes) >= new_index)
item_index = current_index + LENGTH_BYTES
@ -375,7 +353,7 @@ container = Container()
typ = type(container)
assert(len(rawbytes) > current_index + LENGTH_BYTES)
total_length = int.from_bytes(rawbytes[current_index:current_index + LENGTH_BYTES], 'big')
total_length = int.from_bytes(rawbytes[current_index:current_index + LENGTH_BYTES], 'little')
new_index = current_index + LENGTH_BYTES + total_length
assert(len(rawbytes) >= new_index)
item_index = current_index + LENGTH_BYTES
@ -394,11 +372,11 @@ The below `hash_tree_root` algorithm is defined recursively in the case of lists
Refer to [the helper function `hash`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#hash) of Phase 0 of the [Eth2.0 specs](https://github.com/ethereum/eth2.0-specs) for a definition of the hash function used below, `hash(x)`.
#### `uint8`..`uint256`, `bool`, `address`, `hash1`..`hash32`
#### `uint8`..`uint256`, `bool`, `bytes1`..`bytes32`
Return the serialization of the value.
#### `uint264`..`uintN`, `bytes`, `hash33`..`hashN`
#### `uint264`..`uintN`, `bytes`, `bytes33`..`bytesN`
Return the hash of the serialization of the value.
@ -410,7 +388,7 @@ First, we define some helpers and then the Merkle tree function.
# Merkle tree hash of a list of homogenous, non-empty items
def merkle_hash(lst):
# Store length of list (to compensate for non-bijectiveness of padding)
datalen = len(lst).to_bytes(32, 'big')
datalen = len(lst).to_bytes(32, 'little')
if len(lst) == 0:
# Handle empty list case
@ -443,7 +421,6 @@ return merkle_hash([hash_tree_root(item) for item in value])
Where the inner `hash_tree_root` is a recursive application of the tree-hashing function (returning less than 32 bytes for short single values).
#### Container
Recursively tree hash the values in the container in the same order as the fields, and return the hash of the concatenation of the results.
@ -452,18 +429,18 @@ Recursively tree hash the values in the container in the same order as the field
return hash(b''.join([hash_tree_root(getattr(x, field)) for field in value.fields]))
```
## Implementations
| Language | Implementation | Description |
|:--------:|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------|
| Python | [ https://github.com/ethereum/beacon_chain/blob/master/ssz/ssz.py ](https://github.com/ethereum/beacon_chain/blob/master/ssz/ssz.py) | Beacon chain reference implementation written in Python. |
| Python | [ https://github.com/ethereum/py-ssz ](https://github.com/ethereum/py-ssz) | Python implementation of SSZ |
| Rust | [ https://github.com/sigp/lighthouse/tree/master/beacon_chain/utils/ssz ](https://github.com/sigp/lighthouse/tree/master/beacon_chain/utils/ssz) | Lighthouse (Rust Ethereum 2.0 Node) maintained SSZ. |
| Nim | [ https://github.com/status-im/nim-beacon-chain/blob/master/beacon_chain/ssz.nim ](https://github.com/status-im/nim-beacon-chain/blob/master/beacon_chain/ssz.nim) | Nim Implementation maintained SSZ. |
| Rust | [ https://github.com/paritytech/shasper/tree/master/util/ssz ](https://github.com/paritytech/shasper/tree/master/util/ssz) | Shasper implementation of SSZ maintained by ParityTech. |
| Javascript | [ https://github.com/ChainSafeSystems/ssz-js/blob/master/src/index.js ](https://github.com/ChainSafeSystems/ssz-js/blob/master/src/index.js) | Javascript Implementation maintained SSZ |
| Java | [ https://www.github.com/ConsenSys/cava/tree/master/ssz ](https://www.github.com/ConsenSys/cava/tree/master/ssz) | SSZ Java library part of the Cava suite |
| Go | [ https://github.com/prysmaticlabs/prysm/tree/master/shared/ssz ](https://github.com/prysmaticlabs/prysm/tree/master/shared/ssz) | Go implementation of SSZ mantained by Prysmatic Labs |
| Swift | [ https://github.com/yeeth/SimpleSerialize.swift ](https://github.com/yeeth/SimpleSerialize.swift) | Swift implementation maintained SSZ |
## Copyright