From bc53f0e386ce822278fe9b429a6d8b8589138e8d Mon Sep 17 00:00:00 2001 From: vbuterin Date: Thu, 22 Nov 2018 08:24:20 -0500 Subject: [PATCH 1/8] Fork choice rule Added the LMD GHOST fork choice rule. --- specs/core/0_beacon-chain.md | 28 +++++++++++++++++++++++----- 1 file changed, 23 insertions(+), 5 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 9ac39501c..423d26f1b 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -337,15 +337,33 @@ Beacon block production is significantly different because of the proof of stake ### Beacon chain fork choice rule -The beacon chain uses the Casper FFG fork choice rule of "favor the chain containing the highest-slot-number justified block". To choose between chains that are all descended from the same justified block, the chain uses "immediate message driven GHOST" (IMD GHOST) to choose the head of the chain. +The beacon chain uses a hybrid fork choice rule that combines together FFG justification/finality with latest-message-driven GHOST. -For a description see: **https://ethresear.ch/t/beacon-chain-casper-ffg-rpj-mini-spec/2760** +The following is a description of how to calculate the head at any given point in time. -For an implementation with a network simulator see: **https://github.com/ethereum/research/blob/master/clock_disparity/ghost_node.py** +1. Set `finalized_head` to the finalized block with the highest slot number. +2. Set `justified_head` be the descendant of `finalized_head` that is justified, and has been known to be justified for at least `SLOT_DURATION * CYCLE_LENGTH` seconds, with the highest slot number. +3. Return `LMD_GHOST(store, justified_head)`, where `store` contains all attestations and blocks that the validator has received. -Here's an example of its working (green is finalized blocks, yellow is justified, grey is attestations): +Let `get_most_recent_attestation_target(store, v)` be a function that returns the block that a validator attested to in the attestation in `store` that has the highest slot number. Let `get_ancestor(store, block, slot)` be the function that returns the ancestor of the given block at the given slot; it could be defined recursively as `def get_ancestor(store, block, slot): return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. LMD GHOST is defined as follows: -![](https://vitalik.ca/files/RPJ.png) +```python +def lmd_ghost(store, start): + validators = [start.state.validators[i] for i in range(get_active_validators(start.state.validators, start.slot))] + latest_message_targets = [get_most_recent_attestation_target(store, v) for v in validators] + head = start + while 1: + c = get_children(head) + if len(c) == 0: + return head + + def get_vote_count(block): + return len([t for t in latest_message_targets if get_ancestor(store, t, block.slot) == block]) + + head = max(c, key=get_vote_count) +``` + +Note that the above is a definition, not an optimal implementation; implementations that determine the head in logarithmic time are possible. ## Beacon chain state transition function From f3094e179e1523e3b8bde078ca3231dfc04729c7 Mon Sep 17 00:00:00 2001 From: vbuterin Date: Fri, 23 Nov 2018 08:54:24 -0500 Subject: [PATCH 2/8] Resolved Justin's questions --- specs/core/0_beacon-chain.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 423d26f1b..ae067f588 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -341,11 +341,11 @@ The beacon chain uses a hybrid fork choice rule that combines together FFG justi The following is a description of how to calculate the head at any given point in time. -1. Set `finalized_head` to the finalized block with the highest slot number. -2. Set `justified_head` be the descendant of `finalized_head` that is justified, and has been known to be justified for at least `SLOT_DURATION * CYCLE_LENGTH` seconds, with the highest slot number. -3. Return `LMD_GHOST(store, justified_head)`, where `store` contains all attestations and blocks that the validator has received. +1. Set `finalized_head` to the finalized block with the highest slot number (as in, the block B with the highest slot number such that there exists a descendant of B the processing of which set B as finalized) +2. Set `justified_head` be the descendant of `finalized_head` that is justified, and has been known to be justified for at least `SLOT_DURATION * CYCLE_LENGTH` seconds, with the highest slot number (as in, the block B with the highest slot number such that there exists a descendant of B the processing of which set B as justified; if no such block exists, then it is just set to the `finalized_head`) +3. Return `LMD_GHOST(store, justified_head)`, where `store` contains all attestations and blocks that the validator has received, including those that have not been included in any chain. -Let `get_most_recent_attestation_target(store, v)` be a function that returns the block that a validator attested to in the attestation in `store` that has the highest slot number. Let `get_ancestor(store, block, slot)` be the function that returns the ancestor of the given block at the given slot; it could be defined recursively as `def get_ancestor(store, block, slot): return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. LMD GHOST is defined as follows: +Let `get_most_recent_attestation_target(store, v)` be a function that returns the block that a validator attested to in the attestation in `store` with the highest slot number; only attestations pointing to blocks that the validator has verified (including recursively verifying ancestors) are considered. If more than one attestation exists in the same slot, `get_most_recent_attestation_target` should return the attestation that the client learned about earlier. Let `get_ancestor(store, block, slot)` be the function that returns the ancestor of the given block at the given slot; it could be defined recursively as `def get_ancestor(store, block, slot): return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. LMD GHOST is defined as follows: ```python def lmd_ghost(store, start): From 92981e9714e0a8f6d62c90af35544edfacdbf0b4 Mon Sep 17 00:00:00 2001 From: Justin Date: Fri, 23 Nov 2018 14:46:25 +0000 Subject: [PATCH 3/8] Started cleaning up fork choice rule definition --- specs/core/0_beacon-chain.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index ae067f588..53845a3ba 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -337,15 +337,15 @@ Beacon block production is significantly different because of the proof of stake ### Beacon chain fork choice rule -The beacon chain uses a hybrid fork choice rule that combines together FFG justification/finality with latest-message-driven GHOST. +The beacon chain fork choice rule is a hybrid that combines justification and finality with Latest Message Driven (LMD) GHOST. At any point in time a validator `v` calculates the head as follows. -The following is a description of how to calculate the head at any given point in time. - -1. Set `finalized_head` to the finalized block with the highest slot number (as in, the block B with the highest slot number such that there exists a descendant of B the processing of which set B as finalized) -2. Set `justified_head` be the descendant of `finalized_head` that is justified, and has been known to be justified for at least `SLOT_DURATION * CYCLE_LENGTH` seconds, with the highest slot number (as in, the block B with the highest slot number such that there exists a descendant of B the processing of which set B as justified; if no such block exists, then it is just set to the `finalized_head`) -3. Return `LMD_GHOST(store, justified_head)`, where `store` contains all attestations and blocks that the validator has received, including those that have not been included in any chain. - -Let `get_most_recent_attestation_target(store, v)` be a function that returns the block that a validator attested to in the attestation in `store` with the highest slot number; only attestations pointing to blocks that the validator has verified (including recursively verifying ancestors) are considered. If more than one attestation exists in the same slot, `get_most_recent_attestation_target` should return the attestation that the client learned about earlier. Let `get_ancestor(store, block, slot)` be the function that returns the ancestor of the given block at the given slot; it could be defined recursively as `def get_ancestor(store, block, slot): return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. LMD GHOST is defined as follows: +* Let `store` be the set of all attestations and blocks that the validator `v` has verified (in particular, ancestors must be recursively verified), including attestations not included in any chain. +* Let `finalized_head` be the finalized block with the highest slot number. (A block `B` is finalized if there is a descendant of `B` in `store` the processing of which sets `B` as finalized.) +* Let `justified_head` be the descendant of `finalized_head` with the highest slot number that has been justified for at least `CYCLE_LENGTH` slots. (A block `B` is justified is there is a descendant of `B` in `store` the processing of which sets `B` as justified.) If no such descendant exists set `justified_head` to `finalized_head`. +* Let `get_most_recent_attestation(store, validator)` be the attestation in `store` from `validator` with the highest slot number. If several such attestations exist use the one the validator `v` verified first. +* Let `get_most_recent_attestation_target(store, validator)` be the target block in `get_most_recent_attestation(store, validator)`. +* Let `get_ancestor(store, block, slot)` be the ancestor of `block` with slot number `slot`. The `get_ancestor` function can be defined recursively as `def get_ancestor(store, block, slot): return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. +* The head is `lmd_ghost(store, justified_head)` where the function `lmd_ghost` is defined below. Note that the implementation below is suboptimal; there are implementations that compute the head in logarithmic time. ```python def lmd_ghost(store, start): @@ -363,8 +363,6 @@ def lmd_ghost(store, start): head = max(c, key=get_vote_count) ``` -Note that the above is a definition, not an optimal implementation; implementations that determine the head in logarithmic time are possible. - ## Beacon chain state transition function We now define the state transition function. At the high level, the state transition is made up of two parts: From 361bcd6be493d4933461cdf32aba9b33641845e8 Mon Sep 17 00:00:00 2001 From: Justin Date: Fri, 23 Nov 2018 15:23:22 +0000 Subject: [PATCH 4/8] More polishing to fork choice rule --- specs/core/0_beacon-chain.md | 28 +++++++++++++--------------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 53845a3ba..7f4522360 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -337,30 +337,29 @@ Beacon block production is significantly different because of the proof of stake ### Beacon chain fork choice rule -The beacon chain fork choice rule is a hybrid that combines justification and finality with Latest Message Driven (LMD) GHOST. At any point in time a validator `v` calculates the head as follows. +The beacon chain fork choice rule is a hybrid that combines justification and finality with Latest Message Driven (LMD) Greediest Heaviest Observed SubTree (GHOST). At any point in time a validator `v` calculates the beacon chain head as follows. -* Let `store` be the set of all attestations and blocks that the validator `v` has verified (in particular, ancestors must be recursively verified), including attestations not included in any chain. +* Let `store` be the set of attestations and blocks that the validator `v` has observed and verified (in particular, block ancestors must be recursively verified). Attestations not part of any chain are still included in `store`. * Let `finalized_head` be the finalized block with the highest slot number. (A block `B` is finalized if there is a descendant of `B` in `store` the processing of which sets `B` as finalized.) * Let `justified_head` be the descendant of `finalized_head` with the highest slot number that has been justified for at least `CYCLE_LENGTH` slots. (A block `B` is justified is there is a descendant of `B` in `store` the processing of which sets `B` as justified.) If no such descendant exists set `justified_head` to `finalized_head`. -* Let `get_most_recent_attestation(store, validator)` be the attestation in `store` from `validator` with the highest slot number. If several such attestations exist use the one the validator `v` verified first. -* Let `get_most_recent_attestation_target(store, validator)` be the target block in `get_most_recent_attestation(store, validator)`. * Let `get_ancestor(store, block, slot)` be the ancestor of `block` with slot number `slot`. The `get_ancestor` function can be defined recursively as `def get_ancestor(store, block, slot): return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. +* Let `get_latest_attestation(store, validator)` be the attestation in `store` from `validator` with the highest slot number. If several such attestations exist use the one the validator `v` verified first. +* Let `get_latest_attestation_target(store, validator)` be the target block in the attestation `get_latest_attestation(store, validator)`. * The head is `lmd_ghost(store, justified_head)` where the function `lmd_ghost` is defined below. Note that the implementation below is suboptimal; there are implementations that compute the head in logarithmic time. ```python def lmd_ghost(store, start): - validators = [start.state.validators[i] for i in range(get_active_validators(start.state.validators, start.slot))] - latest_message_targets = [get_most_recent_attestation_target(store, v) for v in validators] + active_validators = [start.state.validators[i] for i in range(get_active_validators(start.state.validators, start.slot))] + latest_attestation_targets = [get_latest_attestation_target(store, validator) for validator in active_validators] + def get_vote_count(block): + return len([target for target in latest_attestation_targets if get_ancestor(store, target, block.slot) == block]) + head = start while 1: - c = get_children(head) - if len(c) == 0: - return head - - def get_vote_count(block): - return len([t for t in latest_message_targets if get_ancestor(store, t, block.slot) == block]) - - head = max(c, key=get_vote_count) + children = get_children(head) + if len(children) == 0: + return head + head = max(children, key=get_vote_count) ``` ## Beacon chain state transition function @@ -1187,7 +1186,6 @@ Note: This spec is ~65% complete. **Possible modifications and additions** -* [ ] Replace the IMD fork choice rule with LMD * [ ] Homogenise types to `uint64` ([PR 36](https://github.com/ethereum/eth2.0-specs/pull/36)) * [ ] Reduce the slot duration to 8 seconds * [ ] Allow for the delayed inclusion of aggregated signatures From b76eae8a71d700ee6bd88867ba21790e6a18dfb9 Mon Sep 17 00:00:00 2001 From: Justin Date: Fri, 23 Nov 2018 15:37:37 +0000 Subject: [PATCH 5/8] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 7f4522360..820c20edc 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -337,22 +337,23 @@ Beacon block production is significantly different because of the proof of stake ### Beacon chain fork choice rule -The beacon chain fork choice rule is a hybrid that combines justification and finality with Latest Message Driven (LMD) Greediest Heaviest Observed SubTree (GHOST). At any point in time a validator `v` calculates the beacon chain head as follows. +The beacon chain fork choice rule is a hybrid that combines justification and finality with Latest Message Driven (LMD) Greediest Heaviest Observed SubTree (GHOST). At any point in time a validator `v` subjectively calculates the beacon chain head as follows. * Let `store` be the set of attestations and blocks that the validator `v` has observed and verified (in particular, block ancestors must be recursively verified). Attestations not part of any chain are still included in `store`. * Let `finalized_head` be the finalized block with the highest slot number. (A block `B` is finalized if there is a descendant of `B` in `store` the processing of which sets `B` as finalized.) * Let `justified_head` be the descendant of `finalized_head` with the highest slot number that has been justified for at least `CYCLE_LENGTH` slots. (A block `B` is justified is there is a descendant of `B` in `store` the processing of which sets `B` as justified.) If no such descendant exists set `justified_head` to `finalized_head`. * Let `get_ancestor(store, block, slot)` be the ancestor of `block` with slot number `slot`. The `get_ancestor` function can be defined recursively as `def get_ancestor(store, block, slot): return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. -* Let `get_latest_attestation(store, validator)` be the attestation in `store` from `validator` with the highest slot number. If several such attestations exist use the one the validator `v` verified first. +* Let `get_latest_attestation(store, validator)` be the attestation with the highest slot number in `store` from `validator`. If several such attestations exist use the one the validator `v` observed first. * Let `get_latest_attestation_target(store, validator)` be the target block in the attestation `get_latest_attestation(store, validator)`. * The head is `lmd_ghost(store, justified_head)` where the function `lmd_ghost` is defined below. Note that the implementation below is suboptimal; there are implementations that compute the head in logarithmic time. ```python def lmd_ghost(store, start): - active_validators = [start.state.validators[i] for i in range(get_active_validators(start.state.validators, start.slot))] - latest_attestation_targets = [get_latest_attestation_target(store, validator) for validator in active_validators] + validators = start.state.validators + active_validators = [validators[i] for i in range(get_active_validators(validators, start.slot))] + attestation_targets = [get_latest_attestation_target(store, validator) for validator in active_validators] def get_vote_count(block): - return len([target for target in latest_attestation_targets if get_ancestor(store, target, block.slot) == block]) + return len([target for target in attestation_targets if get_ancestor(store, target, block.slot) == block]) head = start while 1: From ea31ff3cea51830f8693770f69dab8b2ab49b45e Mon Sep 17 00:00:00 2001 From: Justin Date: Fri, 23 Nov 2018 16:05:30 +0000 Subject: [PATCH 6/8] Fix minor bug in `ghost_lmd` --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 820c20edc..ba92b601b 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -350,7 +350,7 @@ The beacon chain fork choice rule is a hybrid that combines justification and fi ```python def lmd_ghost(store, start): validators = start.state.validators - active_validators = [validators[i] for i in range(get_active_validators(validators, start.slot))] + active_validators = [validators[i] for i in range(get_active_validator_indices(validators, start.slot))] attestation_targets = [get_latest_attestation_target(store, validator) for validator in active_validators] def get_vote_count(block): return len([target for target in attestation_targets if get_ancestor(store, target, block.slot) == block]) From ee40888d2eebd0157ff518a367b98b0a4c8b185b Mon Sep 17 00:00:00 2001 From: vbuterin Date: Fri, 23 Nov 2018 15:01:15 -0500 Subject: [PATCH 7/8] Fix to fixes --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index ba92b601b..ca80daa09 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -350,7 +350,7 @@ The beacon chain fork choice rule is a hybrid that combines justification and fi ```python def lmd_ghost(store, start): validators = start.state.validators - active_validators = [validators[i] for i in range(get_active_validator_indices(validators, start.slot))] + active_validators = [validators[i] for i in get_active_validator_indices(validators, start.slot)] attestation_targets = [get_latest_attestation_target(store, validator) for validator in active_validators] def get_vote_count(block): return len([target for target in attestation_targets if get_ancestor(store, target, block.slot) == block]) From 545e35e4b1bc12439a3a9a4c20c36aa6a2bd8360 Mon Sep 17 00:00:00 2001 From: vbuterin Date: Fri, 23 Nov 2018 15:10:03 -0500 Subject: [PATCH 8/8] Clarified "logarithmic" --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index ca80daa09..72e733eed 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -345,7 +345,7 @@ The beacon chain fork choice rule is a hybrid that combines justification and fi * Let `get_ancestor(store, block, slot)` be the ancestor of `block` with slot number `slot`. The `get_ancestor` function can be defined recursively as `def get_ancestor(store, block, slot): return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. * Let `get_latest_attestation(store, validator)` be the attestation with the highest slot number in `store` from `validator`. If several such attestations exist use the one the validator `v` observed first. * Let `get_latest_attestation_target(store, validator)` be the target block in the attestation `get_latest_attestation(store, validator)`. -* The head is `lmd_ghost(store, justified_head)` where the function `lmd_ghost` is defined below. Note that the implementation below is suboptimal; there are implementations that compute the head in logarithmic time. +* The head is `lmd_ghost(store, justified_head)` where the function `lmd_ghost` is defined below. Note that the implementation below is suboptimal; there are implementations that compute the head in time logarithmic in slot count. ```python def lmd_ghost(store, start):