From a365fcb03b28727f1427b53f64e75db87f6dd0fe Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 6 Oct 2020 14:42:23 -0600 Subject: [PATCH 1/2] fix message-id issues --- specs/phase0/p2p-interface.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index b42237731..c50ab362a 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -243,11 +243,21 @@ Each gossipsub [message](https://github.com/libp2p/go-libp2p-pubsub/blob/master/ Clients MUST reject (fail validation) messages that are over this size limit. Likewise, clients MUST NOT emit or propagate messages larger than this limit. -The `message-id` of a gossipsub message MUST be the first 8 bytes of the SHA-256 hash of the message data, i.e.: +The `message-id` of a gossipsub message MUST be the following 20 byte value computed from the message data: +* If `message.data` can be snappy decompressed: + * Let `message-id` be the first 20 bytes of the `SHA256` hash of the snappy decompressed message data, + i.e. `SHA256(snappy_decompress(message.data))[:20]`. + * Then set the most significant bit of the left-most byte of `message-id` to 1, + i.e. `message-id[0] |= (0x01 << 7)`. +* Otherwise: + * Let `message-id` be the first 20 bytes of the `SHA256` hash of the raw message data, + i.e. `SHA256(message.data)[:20]`. + * Then set the most significant bit of the left-most byte of `message-id` to 0, + i.e. `message-id[0] &= (0xFF >> 1)`. -```python - message-id: SHA256(message.data)[0:8] -``` +*Note*: The above logic handles two exceptional cases: +(1) multiple snappy `data` can decompress to the same value, +and (2) some message `data` can fail to snappy decompress altogether. The payload is carried in the `data` field of a gossipsub message, and varies depending on the topic: From 6287875baf53e7a062cb6246e728ff7d1fd3d84a Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 7 Oct 2020 11:57:25 -0600 Subject: [PATCH 2/2] use domain byte to isolate message-id domains --- specs/phase0/p2p-interface.md | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index c50ab362a..c9416f328 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -178,6 +178,9 @@ This section outlines constants that are used in this spec. | `RESP_TIMEOUT` | `10s` | The maximum time for complete response transfer. | | `ATTESTATION_PROPAGATION_SLOT_RANGE` | `32` | The maximum number of slots during which an attestation can be propagated. | | `MAXIMUM_GOSSIP_CLOCK_DISPARITY` | `500ms` | The maximum milliseconds of clock disparity assumed between honest nodes. | +| `MESSAGE_DOMAIN_INVALID_SNAPPY` | `0x00000000` | 4-byte domain for gossip message-id isolation of *invalid* snappy messages | +| `MESSAGE_DOMAIN_VALID_SNAPPY` | `0x01000000` | 4-byte domain for gossip message-id isolation of *valid* snappy messages | + ## MetaData @@ -244,16 +247,12 @@ Clients MUST reject (fail validation) messages that are over this size limit. Likewise, clients MUST NOT emit or propagate messages larger than this limit. The `message-id` of a gossipsub message MUST be the following 20 byte value computed from the message data: -* If `message.data` can be snappy decompressed: - * Let `message-id` be the first 20 bytes of the `SHA256` hash of the snappy decompressed message data, - i.e. `SHA256(snappy_decompress(message.data))[:20]`. - * Then set the most significant bit of the left-most byte of `message-id` to 1, - i.e. `message-id[0] |= (0x01 << 7)`. -* Otherwise: - * Let `message-id` be the first 20 bytes of the `SHA256` hash of the raw message data, - i.e. `SHA256(message.data)[:20]`. - * Then set the most significant bit of the left-most byte of `message-id` to 0, - i.e. `message-id[0] &= (0xFF >> 1)`. +* If `message.data` has a valid snappy decompression, set `message-id` to the first 20 bytes of the `SHA256` hash of + the concatenation of `MESSAGE_DOMAIN_VALID_SNAPPY` with the snappy decompressed message data, + i.e. `SHA256(MESSAGE_DOMAIN_VALID_SNAPPY + snappy_decompress(message.data))[:20]`. +* Otherwise, set `message-id` to the first 20 bytes of the `SHA256` hash of + the concatenation of `MESSAGE_DOMAIN_INVALID_SNAPPY` with the raw message data, + i.e. `SHA256(MESSAGE_DOMAIN_INVALID_SNAPPY + message.data)[:20]`. *Note*: The above logic handles two exceptional cases: (1) multiple snappy `data` can decompress to the same value,