diff --git a/README.md b/README.md index 3f58b3dc4..83dbbd3fe 100644 --- a/README.md +++ b/README.md @@ -59,7 +59,7 @@ The following are the broad design goals for Ethereum 2.0: * [Design Rationale](https://notes.ethereum.org/s/rkhCgQteN#) * [Phase 0 Onboarding Document](https://notes.ethereum.org/s/Bkn3zpwxB) -* [Gasper paper](https://github.com/ethereum/research/blob/master/papers/ffg%2Bghost/paper.pdf) +* [Gasper paper](https://arxiv.org/abs/2003.03052) ## For spec contributors @@ -67,4 +67,3 @@ The following are the broad design goals for Ethereum 2.0: Documentation on the different components used during spec writing can be found here: * [YAML Test Generators](tests/generators/README.md) * [Executable Python Spec, with Py-tests](tests/core/pyspec/README.md) - diff --git a/configs/mainnet.yaml b/configs/mainnet.yaml index 74f062d9b..46977f087 100644 --- a/configs/mainnet.yaml +++ b/configs/mainnet.yaml @@ -21,6 +21,12 @@ SHUFFLE_ROUND_COUNT: 90 MIN_GENESIS_ACTIVE_VALIDATOR_COUNT: 16384 # Jan 3, 2020 MIN_GENESIS_TIME: 1578009600 +# 4 +HYSTERESIS_QUOTIENT: 4 +# 1 (minus 0.25) +HYSTERESIS_DOWNWARD_MULTIPLIER: 1 +# 5 (plus 1.25) +HYSTERESIS_UPWARD_MULTIPLIER: 5 # Fork Choice diff --git a/configs/minimal.yaml b/configs/minimal.yaml index 42c63e301..202da8237 100644 --- a/configs/minimal.yaml +++ b/configs/minimal.yaml @@ -20,6 +20,13 @@ SHUFFLE_ROUND_COUNT: 10 MIN_GENESIS_ACTIVE_VALIDATOR_COUNT: 64 # Jan 3, 2020 MIN_GENESIS_TIME: 1578009600 +# 4 +HYSTERESIS_QUOTIENT: 4 +# 1 (minus 0.25) +HYSTERESIS_DOWNWARD_MULTIPLIER: 1 +# 5 (plus 1.25) +HYSTERESIS_UPWARD_MULTIPLIER: 5 + # Fork Choice diff --git a/specs/phase0/beacon-chain.md b/specs/phase0/beacon-chain.md index cd1d765fe..a90a8aa1a 100644 --- a/specs/phase0/beacon-chain.md +++ b/specs/phase0/beacon-chain.md @@ -183,6 +183,10 @@ The following values are (non-configurable) constants used throughout the specif | `SHUFFLE_ROUND_COUNT` | `90` | | `MIN_GENESIS_ACTIVE_VALIDATOR_COUNT` | `2**14` (= 16,384) | | `MIN_GENESIS_TIME` | `1578009600` (Jan 3, 2020) | +| `HYSTERESIS_QUOTIENT` | `4` | +| `HYSTERESIS_DOWNWARD_MULTIPLIER` | `1` | +| `HYSTERESIS_UPWARD_MULTIPLIER` | `5` | + - For the safety of committees, `TARGET_COMMITTEE_SIZE` exceeds [the recommended minimum committee size of 111](http://web.archive.org/web/20190504131341/https://vitalik.ca/files/Ithaca201807_Sharding.pdf); with sufficient active validators (at least `SLOTS_PER_EPOCH * TARGET_COMMITTEE_SIZE`), the shuffling algorithm ensures committee sizes of at least `TARGET_COMMITTEE_SIZE`. (Unbiasable randomness with a Verifiable Delay Function (VDF) will improve committee robustness and lower the safe minimum committee size.) @@ -1404,8 +1408,13 @@ def process_final_updates(state: BeaconState) -> None: # Update effective balances with hysteresis for index, validator in enumerate(state.validators): balance = state.balances[index] - HALF_INCREMENT = EFFECTIVE_BALANCE_INCREMENT // 2 - if balance < validator.effective_balance or validator.effective_balance + 3 * HALF_INCREMENT < balance: + HYSTERESIS_INCREMENT = EFFECTIVE_BALANCE_INCREMENT // HYSTERESIS_QUOTIENT + DOWNWARD_THRESHOLD = HYSTERESIS_INCREMENT * HYSTERESIS_DOWNWARD_MULTIPLIER + UPWARD_THRESHOLD = HYSTERESIS_INCREMENT * HYSTERESIS_UPWARD_MULTIPLIER + if ( + balance + DOWNWARD_THRESHOLD < validator.effective_balance + or validator.effective_balance + UPWARD_THRESHOLD < balance + ): validator.effective_balance = min(balance - balance % EFFECTIVE_BALANCE_INCREMENT, MAX_EFFECTIVE_BALANCE) # Reset slashings state.slashings[next_epoch % EPOCHS_PER_SLASHINGS_VECTOR] = Gwei(0) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index 7ad8759d3..8bde2b80a 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -419,9 +419,37 @@ The token of the negotiated protocol ID specifies the type of encoding to be use #### SSZ-encoding strategy (with or without Snappy) -The [SimpleSerialize (SSZ) specification](../../ssz/simple-serialize.md) outlines how objects are SSZ-encoded. If the Snappy variant is selected, we feed the serialized form to the Snappy compressor on encoding. The inverse happens on decoding. +The [SimpleSerialize (SSZ) specification](../../ssz/simple-serialize.md) outlines how objects are SSZ-encoded. -**Encoding-dependent header:** Req/Resp protocols using the `ssz` or `ssz_snappy` encoding strategies MUST prefix all encoded and compressed (if applicable) payloads with an unsigned [protobuf varint](https://developers.google.com/protocol-buffers/docs/encoding#varints). +If the Snappy variant is selected, we feed the serialized form of the object to the Snappy compressor on encoding. The inverse happens on decoding. + +Snappy has two formats: "block" and "frames" (streaming). To support large requests and response chunks, snappy-framing is used. + +Since snappy frame contents [have a maximum size of `65536` bytes](https://github.com/google/snappy/blob/master/framing_format.txt#L104) + and frame headers are just `identifier (1) + checksum (4)` bytes, the expected buffering of a single frame is acceptable. + +**Encoding-dependent header:** Req/Resp protocols using the `ssz` or `ssz_snappy` encoding strategies MUST encode the length of the raw SSZ bytes, encoded as an unsigned [protobuf varint](https://developers.google.com/protocol-buffers/docs/encoding#varints). + +*Writing*: By first computing and writing the SSZ byte length, the SSZ encoder can then directly write the chunk contents to the stream. +If Snappy is applied, it can be passed through a buffered Snappy writer to compress frame by frame. + +*Reading*: After reading the expected SSZ byte length, the SSZ decoder can directly read the contents from the stream. +If snappy is applied, it can be passed through a buffered Snappy reader to decompress frame by frame. + +A reader SHOULD NOT read more than `max_encoded_len(n)` bytes after reading the SSZ length prefix `n` from the header. +- For `ssz` this is: `n` +- For `ssz_snappy` this is: `32 + n + n // 6`. This is considered the [worst-case compression result](https://github.com/google/snappy/blob/537f4ad6240e586970fe554614542e9717df7902/snappy.cc#L98) by Snappy. + +A reader SHOULD consider the following cases as invalid input: +- A SSZ length prefix that, compared against the SSZ type information (vector lengths, list limits, integer sizes, etc.), is: + - Smaller than the expected minimum serialized length. + - Bigger than the expected maximum serialized length. +- Any remaining bytes, after having read the `n` SSZ bytes. An EOF is expected. +- An early EOF, before fully reading the declared length prefix worth of SSZ bytes. + +In case of an invalid input, a reader MUST: +- From requests: send back an error message, response code `InvalidRequest`. The request itself is ignored. +- From responses: ignore the response, the response MUST be considered bad server behavior. All messages that contain only a single field MUST be encoded directly as the type of that field and MUST NOT be encoded as an SSZ container. @@ -845,23 +873,14 @@ Requests are segregated by protocol ID to: We are using single-use streams where each stream is closed at the end of the message. Thus, libp2p transparently handles message delimiting in the underlying stream. libp2p streams are full-duplex, and each party is responsible for closing their write side (like in TCP). We can therefore use stream closure to mark the end of the request and response independently. -Nevertheless, messages are still length-prefixed—this is now being considered for removal. - -Advantages of length-prefixing include: - -* Reader can prepare a correctly sized buffer before reading message +Nevertheless, in the case of `ssz` and `ssz_snappy`, messages are still length-prefixed with the length of the underlying data: +* A basic reader can prepare a correctly sized buffer before reading the message +* A more advanced reader can stream-decode SSZ given the length of the SSZ data. * Alignment with protocols like gRPC over HTTP/2 that prefix with length -* Sanity checking of stream closure / message length +* Sanity checking of message length, and enabling much stricter message length limiting based on SSZ type information, + to provide even more DOS protection than the global message length already does. E.g. a small `Status` message does not nearly require `MAX_CHUNK_SIZE` bytes. -Disadvantages include: - -* Redundant methods of message delimiting—both stream end marker and length prefix -* Harder to stream as length must be known up-front -* Additional code path required to verify length - -In some protocols, adding a length prefix serves as a form of DoS protection against very long messages, allowing the client to abort if an overlong message is about to be sent. In this protocol, we are globally limiting message sizes using `MAX_CHUNK_SIZE`, thus the length prefix does not afford any additional protection. - -[Protobuf varint](https://developers.google.com/protocol-buffers/docs/encoding#varints) is an efficient technique to encode variable-length ints. Instead of reserving a fixed-size field of as many bytes as necessary to convey the maximum possible value, this field is elastic in exchange for 1-bit overhead per byte. +[Protobuf varint](https://developers.google.com/protocol-buffers/docs/encoding#varints) is an efficient technique to encode variable-length (unsigned here) ints. Instead of reserving a fixed-size field of as many bytes as necessary to convey the maximum possible value, this field is elastic in exchange for 1-bit overhead per byte. ### Why do we version protocol strings with ordinals instead of semver? diff --git a/specs/phase1/custody-game.md b/specs/phase1/custody-game.md index 121f91f97..af3aadc96 100644 --- a/specs/phase1/custody-game.md +++ b/specs/phase1/custody-game.md @@ -48,8 +48,8 @@ This document details the beacon chain additions and changes in Phase 1 of Ether ### Misc | Name | Value | Unit | -| - | - | -| `BLS12_381_Q` | `4002409555221667393417789825735904156556882819939007885332058136124031650490837864442687629129015664037894272559787` | +| - | - | - | +| `BLS12_381_Q` | `4002409555221667393417789825735904156556882819939007885332058136124031650490837864442687629129015664037894272559787` | - | | `BYTES_PER_CUSTODY_ATOM` | `48` | bytes | ## Configuration diff --git a/tests/core/pyspec/eth2spec/test/phase_0/epoch_processing/test_process_final_updates.py b/tests/core/pyspec/eth2spec/test/phase_0/epoch_processing/test_process_final_updates.py index 58882a44f..d959ce16e 100644 --- a/tests/core/pyspec/eth2spec/test/phase_0/epoch_processing/test_process_final_updates.py +++ b/tests/core/pyspec/eth2spec/test/phase_0/epoch_processing/test_process_final_updates.py @@ -51,19 +51,25 @@ def test_effective_balance_hysteresis(spec, state): max = spec.MAX_EFFECTIVE_BALANCE min = spec.EJECTION_BALANCE inc = spec.EFFECTIVE_BALANCE_INCREMENT - half_inc = inc // 2 + div = spec.HYSTERESIS_QUOTIENT + hys_inc = inc // div + down = spec.HYSTERESIS_DOWNWARD_MULTIPLIER + up = spec.HYSTERESIS_UPWARD_MULTIPLIER cases = [ (max, max, max, "as-is"), - (max, max - 1, max - inc, "round down, step lower"), + (max, max - 1, max, "round up"), (max, max + 1, max, "round down"), + (max, max - down * hys_inc, max, "lower balance, but not low enough"), + (max, max - down * hys_inc - 1, max - inc, "lower balance, step down"), + (max, max + (up * hys_inc) + 1, max, "already at max, as is"), (max, max - inc, max - inc, "exactly 1 step lower"), - (max, max - inc - 1, max - (2 * inc), "just 1 over 1 step lower"), + (max, max - inc - 1, max - (2 * inc), "past 1 step lower, double step"), (max, max - inc + 1, max - inc, "close to 1 step lower"), - (min, min + (half_inc * 3), min, "bigger balance, but not high enough"), - (min, min + (half_inc * 3) + 1, min + inc, "bigger balance, high enough, but small step"), - (min, min + (half_inc * 4) - 1, min + inc, "bigger balance, high enough, close to double step"), - (min, min + (half_inc * 4), min + (2 * inc), "exact two step balance increment"), - (min, min + (half_inc * 4) + 1, min + (2 * inc), "over two steps, round down"), + (min, min + (hys_inc * up), min, "bigger balance, but not high enough"), + (min, min + (hys_inc * up) + 1, min + inc, "bigger balance, high enough, but small step"), + (min, min + (hys_inc * div * 2) - 1, min + inc, "bigger balance, high enough, close to double step"), + (min, min + (hys_inc * div * 2), min + (2 * inc), "exact two step balance increment"), + (min, min + (hys_inc * div * 2) + 1, min + (2 * inc), "over two steps, round down"), ] current_epoch = spec.get_current_epoch(state) for i, (pre_eff, bal, _, _) in enumerate(cases):