status-im
/
eth2.0-specs
mirror of https://github.com/status-im/eth2.0-specs.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge pull request #481 from ChihChengLiang/fix-container-ssz 

Fix #479, make `item_index == new_index` a check to perform
This commit is contained in:
Hsiao-Wei Wang 2019-01-24 18:46:12 +08:00 committed by GitHub
parent 847746286b 45c064a2d6
commit 25ee768b42
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 18 additions and 13 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
specs/simple-serialize.md
View File

@ -43,17 +43,19 @@ protocol for use in the Ethereum 2.0 Beacon Chain.
The core feature of `ssz` is the simplicity of the serialization with low The core feature of `ssz` is the simplicity of the serialization with low
overhead. overhead.
## Terminology ## Variables and Functions
| Term | Definition | | Term | Definition |
|:-------------|:-----------------------------------------------------------------------------------------------| |:-------------|:-----------------------------------------------------------------------------------------------|
| `little` | Little endian. | | `little` | Little endian. |
| `byte_order` | Specifies [endianness](https://en.wikipedia.org/wiki/Endianness): big endian or little endian. | | `byteorder` | Specifies [endianness](https://en.wikipedia.org/wiki/Endianness): big endian or little endian. |
| `len` | Length/number of bytes. | | `len` | Length/number of bytes. |
| `to_bytes` | Convert to bytes. Should take parameters ``size`` and ``byte_order``. | | `to_bytes` | Convert to bytes. Should take parameters ``size`` and ``byteorder``. |
| `from_bytes` | Convert from bytes to object. Should take ``bytes`` and ``byte_order``. | | `from_bytes` | Convert from bytes to object. Should take ``bytes`` and ``byteorder``. |
| `value` | The value to serialize. | | `value` | The value to serialize. |
| `rawbytes` | Raw serialized bytes. | | `rawbytes` | Raw serialized bytes. |
| `deserialized_object` | The deserialized data in the data structure of your programming language. |
| `new_index` | An index to keep track the latest position where the `rawbytes` have been deserialized. |
## Constants ## Constants
@ -72,7 +74,6 @@ overhead.
|:---------:|:-----------------------------------------------------------| |:---------:|:-----------------------------------------------------------|
| `uintN` | Type of `N` bits unsigned integer, where ``N % 8 == 0``. | | `uintN` | Type of `N` bits unsigned integer, where ``N % 8 == 0``. |
Convert directly to bytes the size of the int. (e.g. ``uint16 = 2 bytes``) Convert directly to bytes the size of the int. (e.g. ``uint16 = 2 bytes``)
All integers are serialized as **little endian**. All integers are serialized as **little endian**.
@ -142,7 +143,6 @@ Lists are a collection of elements of the same homogeneous type.
|:--------------------------------------------|:----------------------------| |:--------------------------------------------|:----------------------------|
| Length of serialized list fits into 4 bytes | ``len(serialized) < 2**32`` | | Length of serialized list fits into 4 bytes | ``len(serialized) < 2**32`` |
1. Get the number of raw bytes to serialize: it is ``len(list) * sizeof(element)``. 1. Get the number of raw bytes to serialize: it is ``len(list) * sizeof(element)``.
* Encode that as a `4-byte` **little endian** `uint32`. * Encode that as a `4-byte` **little endian** `uint32`.
2. Append the elements in a packed manner. 2. Append the elements in a packed manner.
@ -171,7 +171,6 @@ A container represents a heterogenous, associative collection of key-value pairs
To serialize a container, obtain the list of its field's names in the specified order. For each field name in this list, obtain the corresponding value and serialize it. Tightly pack the complete set of serialized values in the same order as the field names into a buffer. Calculate the size of this buffer of serialized bytes and encode as a `4-byte` **little endian** `uint32`. Prepend the encoded length to the buffer. The result of this concatenation is the final serialized value of the container. To serialize a container, obtain the list of its field's names in the specified order. For each field name in this list, obtain the corresponding value and serialize it. Tightly pack the complete set of serialized values in the same order as the field names into a buffer. Calculate the size of this buffer of serialized bytes and encode as a `4-byte` **little endian** `uint32`. Prepend the encoded length to the buffer. The result of this concatenation is the final serialized value of the container.
| Check to perform | Code | | Check to perform | Code |
|:--------------------------------------------|:----------------------------| |:--------------------------------------------|:----------------------------|
| Length of serialized fields fits into 4 bytes | ``len(serialized) < 2**32`` | | Length of serialized fields fits into 4 bytes | ``len(serialized) < 2**32`` |
@ -219,14 +218,21 @@ The decoding requires knowledge of the type of the item to be decoded. When
performing decoding on an entire serialized string, it also requires knowledge performing decoding on an entire serialized string, it also requires knowledge
of the order in which the objects have been serialized. of the order in which the objects have been serialized.
Note: Each return will provide ``deserialized_object, new_index`` keeping track Note: Each return will provide:
of the new index. - `deserialized_object`
- `new_index`
At each step, the following checks should be made: At each step, the following checks should be made:
| Check to perform | Check | | Check to perform | Check |
|:-------------------------|:-----------------------------------------------------------| |:-------------------------|:-----------------------------------------------------------|
| Ensure sufficient length | ``length(rawbytes) >= current_index + deserialize_length`` | | Ensure sufficient length | ``len(rawbytes) >= current_index + deserialize_length`` |
At the final step, the following checks should be made:
| Check to perform | Check |
|:-------------------------|:-------------------------------------|
| Ensure no extra length | `new_index == len(rawbytes)` |
#### uint #### uint
@ -295,7 +301,7 @@ entire length of the list.
| Check to perform | code | | Check to perform | code |
|:------------------------------------------|:----------------------------------------------------------------| |:------------------------------------------|:----------------------------------------------------------------|
| rawbytes has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | | ``rawbytes`` has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` |
| list is not greater than serialized bytes | ``len(rawbytes) > current_index + LENGTH_BYTES + total_length`` | | list is not greater than serialized bytes | ``len(rawbytes) > current_index + LENGTH_BYTES + total_length`` |
```python ```python
@ -323,7 +329,7 @@ Instantiate a container with the full set of deserialized data, matching each me
| Check to perform | code | | Check to perform | code |
|:------------------------------------------|:----------------------------------------------------------------| |:------------------------------------------|:----------------------------------------------------------------|
| rawbytes has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | | ``rawbytes`` has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` |
| list is not greater than serialized bytes | ``len(rawbytes) > current_index + LENGTH_BYTES + total_length`` | | list is not greater than serialized bytes | ``len(rawbytes) > current_index + LENGTH_BYTES + total_length`` |
To deserialize: To deserialize:
@ -442,6 +448,5 @@ return hash(b''.join([hash_tree_root(getattr(x, field)) for field in value.field
| Go | [ https://github.com/prysmaticlabs/prysm/tree/master/shared/ssz ](https://github.com/prysmaticlabs/prysm/tree/master/shared/ssz) | Go implementation of SSZ mantained by Prysmatic Labs | | Go | [ https://github.com/prysmaticlabs/prysm/tree/master/shared/ssz ](https://github.com/prysmaticlabs/prysm/tree/master/shared/ssz) | Go implementation of SSZ mantained by Prysmatic Labs |
| Swift | [ https://github.com/yeeth/SimpleSerialize.swift ](https://github.com/yeeth/SimpleSerialize.swift) | Swift implementation maintained SSZ | | Swift | [ https://github.com/yeeth/SimpleSerialize.swift ](https://github.com/yeeth/SimpleSerialize.swift) | Swift implementation maintained SSZ |
## Copyright ## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).