mirror of https://github.com/status-im/nim-eth.git
Common utilities for Ethereum
https://nimbus.status.im
00c91a1dca
Root encoding is on the hot path for block verification both in the consensus (when syncing) and execution clients and oddly consititutes a significant part of resource usage even though it is not that much work. While the trie code is capable of producing a transaction root and similar feats, it turns out that it is quite inefficient - even for small work loads. This PR brings in a helper for the specific use case of building tries of lists of values whose key is the RLP-encoded index of the item. As it happens, such keys follow a particular structure where items end up "almost" sorted, with the exception for the item at index 0 which gets encoded as `[0x80]`, ie the empty list, thus moving it to a new location. Armed with this knowledge and the understanding that inserting ordered items into a trie easily can be done with a simple recursion, this PR brings a ~100x improvement in CPU usage (360ms vs 33s) and a ~50x reduction in memory usage (70mb vs >3gb!) for the simple test of encoding 1000000 keys. In part, the memory usage reduction is due to a trick where the hash of the item is computed as the item is being added instead of storing it in the value. There are further reductions possible such as maintaining a hasher per level instead of storing hash values as well as using a direct-to-hash rlp encoder. |
||
|---|---|---|
| .github/workflows | ||
| doc | ||
| eth | ||
| tests | ||
| tools | ||
| .gitignore | ||
| LICENSE-APACHEv2 | ||
| LICENSE-MIT | ||
| README.md | ||
| config.nims | ||
| eth.nimble | ||
| nim.cfg | ||
README.md
eth - Eth Common Library
Introduction
Ethereum-related utilities written in Nim. Includes things like Bloom filters, private/public key utilities, RLP, devp2p, and more.
Docs
Prerequisites
- Nim & Nimble
Building & Testing
# Install required modules
nimble install
# Run full test suite
nimble test
You can also run specific parts of the test suite, e.g.:
# Test p2p functionality
nimble test_p2p
# Test rlp functionality
nimble test_rlp
Fuzzing
Next to the test suite, there are also several fuzzing test cases available. How these can be run is explained in the fuzzing readme.
License
Licensed and distributed under either of
- MIT license: LICENSE-MIT or http://opensource.org/licenses/MIT
or
- Apache License, Version 2.0, (LICENSE-APACHEv2 or http://www.apache.org/licenses/LICENSE-2.0)
at your option. This file may not be copied, modified, or distributed except according to those terms.