status-im/nimbus-eth1
2
0
Fork 0
mirror of https://github.com/status-im/nimbus-eth1.git synced 2025-01-12 21:34:33 +00:00
Code Issues Projects Releases Wiki Activity

Initial Home page

zah 2018-02-26 22:30:39 +02:00
commit 859affa7d7
1 changed files with 449 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

449
Nimbus-Roadmap.md Normal file

@ -0,0 +1,449 @@
This document attempts to specify a list of high-level components of the
Nimbus project that can be developed and tested in isolation. It tries to
identify the dependencies between the development tasks, so the implementation
efforts can be scheduled in parallel where appropriate. Please keep in mind
that the list is not exhaustive and further planning is necessary before all
tasks are specified in the same level of detail.
## Development tasks:
### 1. Select an AES implementation
The AES library should support both AES256 in CTR mode (as per [the RLPx spec][1])
and GCM mode (used in [Whisper v6][2])
Suggested library: Crypto++
[1]: https://github.com/ethereum/devp2p/blob/master/rlpx.md
[2]: https://gist.github.com/gluk256/cbdf8d1f05c72c2273cc42bce905388a
### 2. Implement ECIES
Required for the [encyprted handshake][3]. Based on libsecp256k1.
Reference implementation:
- https://github.com/ethereum/py-evm/blob/master/p2p/ecies.py
Depends on:
* [Select an AES implementation](#select-an-aes-implementation)
[3]: https://github.com/ethereum/devp2p/blob/master/rlpx.md#encrypted-handshake
### 3. Select Scrypt library (optional)
Both cpp-ethereum and Parity also use Scrypt in their local keystore code.
It doesn't seem mandatory for the implementation.
### 4. Ethereum Bloom Filter
Bloom filters are used for efficient monitoring of the transaction
logs/events, stored in the receipt trie of the blockchain, for topic
selection in Whisper and some other places.
Reference implementation:
- https://github.com/ethereum/eth-bloom
Existing Nim library:
- https://github.com/zielmicha/nim-bloom
### 5. Ethereum key file
The key file stores the keys associated with one or more accounts in
password-protected encrypted form.
Reference implementation:
- https://github.com/ethereum/eth-keyfile
Sub-tasks:
* [ ] Safe storage
* [ ] Key generation / passphrase recovery
* [ ] Use native APIs on mobile platforms
Depends on:
* [Select an AES implementation](#select-an-aes-implementation)
### 6. JSON-RPC Server
Development already started. This will be an ongoing effort and the
list of supported calls should increase as other parts of the system
are developed.
Reference information:
- https://github.com/ethereum/wiki/wiki/JSON-RPC
### 7. Node discovery
- https://github.com/ethereum/devp2p/blob/master/rlpx.md#node-discovery
This uses a modified and stripped down version of [Kademlia][kad].
Kademlia us also used in Swarm for resource discovery and some of the
logic for maintaining peer connection tables can be shared between the
two implementations.
In the sharded blockchan, each shard is represented by a separate P2P
network. This is implemented by running the same node discovery protocol,
but with a different peer connection port:
- https://github.com/ethereum/py-evm/issues/196
In the future, Ethereum may support alternative transport protocols
(such as lib2p2). To enable this, there is new version 5 of the node
discovery protocol in development, which allow the nodes to advertise
alternative connection methods (ENR):
- https://github.com/ethereum/go-ethereum/tree/master/p2p/discv5
- https://github.com/fjl/EIPs/blob/f80eb12f22f178c81dc2a6de4be0c73cd302b73b/EIPS/eip-778.md
Sub-tasks:
- https://github.com/ethereum/devp2p/blob/master/rlpx.md#node-discovery-1
[kad]: https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf
### 8. UPnP support
The standard Ethereum protocols use TCP to communicate between peers.
Hosts with private IP addresses must set up port redirection on their
routers. UPnP can be used to automate this.
### 9. Authenticated handshake
https://github.com/ethereum/devp2p/blob/master/rlpx.md#encrypted-handshake
Sub-tasks:
* [ ] Authenticate previously known nodes with an old session key
* [ ] Authenticate newly discovered nodes
* [ ] Support RLPx sub-protocol negotiation in the authentication message (capabilities)
* [ ] Derive shared secrets from handshake
Depends on:
* [Select an AES implementation](#select-an-aes-implementation)
* [Implement ECIES](#implement-ecies)
### 10. RLPx sub-protocols
The Ethereum wire protocols are transport-agnostic. Our framework should
allow running them over the TCP connections established after node discovery
or through connections established with libp2p or uTP in the future.
It should also be possible to define unit tests involving multiple agents
simulated within a single process (bypassing the network stack).
**Issues:**
None of the protocols currently supports using NAT traversal for directly
establishing P2P connections. This may be important for supporting Video
and Audio calls in Status. We can solve this by establishing such connections
in an out-of-band protocol (e.g. by linking WebRTC or something similar) or
by contributing EIPs for the existing protocols.
Sub-tasks:
* [ ] SubProtocol registry
* [ ] Active protocol map (shared objects)
* [ ] Message dispatcher
* [ ] Subprotocol definition DSL
* [ ] Protocol negotiation
* [ ] Per-subprotocol send queue
* [ ] Chunking
* [ ] Peer reputation tracking (with persistence)
* [ ] Flow control (optional)
* [ ] Pre-sending PoW priority increase
### 11. The new binary trie
Replaces the old hexary trie in the new blockchain shards.
Sub-tasks:
* [ ] Add support for partial tries
* [ ] Support for obtaining and verifying Merkle proofs
Reference implementations:
- https://github.com/ethereum/research/blob/master/trie_research/new_bintrie.py
- https://github.com/ethereum/py-trie/blob/master/trie/binary.py
### 12. New parametric Blockchain definition
There should be sufficient parametrization to allow:
* [ ] The Chain is instantiated with a Trie type (either Binary or Hexary)
* [ ] Per-account database key handling (single layer trie)
* [ ] Different header structure (in the Sharded chains, header creation is also tracked in the VMC)
* [ ] Different transaction fields
* [ ] Different VM behavior (i.e. CALL opcode, the other new opcodes)
* [ ] per-transaction witness (account access lists):
https://github.com/ethereum/sharding/blob/develop/docs/doc.md#access-list
https://github.com/ethereum/py-evm/issues/195
The new shard chain uses a new account creation method:
- https://github.com/ethereum/py-evm/issues/203
- https://ethresear.ch/t/tradeoffs-in-account-abstraction-proposals/263
- https://github.com/ethereum/EIPs/blob/bd136e662fca4154787b44cded8d2a29b993be66/EIPS/abstraction.md
... which requires 3 new opcodes to be added:
- SIGHASH
- PAYGAS
- CREATE2
Current outstanding tasks:
* [ ] Stateless and stateful execution with roll-back
* [ ] Maintenance of a receipt trie
* [ ] Block validation
* [ ] Precompiled contracts
* [ ] Transaction pool
Depends on:
* [Bloom filters](#ethereum-bloom-filter)
* [Partial tries](#the-new-binary-trie)
Extra tasks:
* VM execution tracing
### 13. Blockchain Sync Algorithms
Fast sync uses the standard Ethereum sub-protocol.
Parity's warp sync is based on a custom server and protocol developed
by the Parity team.
The sharding client features an even faster stateless sync protocol
that aims to obtain only the latest block headers and to perform a
randomized verification that the blockchain data is available upon
request:
- https://github.com/ethereum/research/wiki/A-note-on-data-availability-and-erasure-coding
Depends on:
* [RLPx Sub-protocols](#rlpx-sub-protocols)
* Functional blockchain storage
* The stateless sync depends on [LES](#light-clients) and the [Binary Trie](#the-new-binary-trie)
### 14. Known contract abstraction / DApp framework
Make it easier to interact with known contracts such as Casper,
the Sharding VMC, etc. Each such contract has a well-known address
and a particular API that can be expressed in standard Nim.
Libraries such as web3.js and web.py offer similar functionality.
Sub-tasks:
* [ ] Nim-types-to-ABI-types translation and contract FFI DSL.
* [ ] Transaction creation logic
* [ ] Event monitoring logic
* [ ] Provide high-level information about the contract for the UI layer (NatSpec, etc)
Depends on:
* [Bloom filters](#ethereum-bloom-filter)
* [RLPx Sub-protocols](#rlpx-sub-protocols)
### 15. Light Clients
The light client operates without storing the full chain locally.
Most operations depends on Asynchronous APIs that will downloads
parts of the chain on-demand. Light clients can also execute DApps
by monitoring the new blocks for events of interest.
- https://github.com/ethereum/wiki/wiki/Light-client-protocol
- https://github.com/paritytech/parity/wiki/Light-Client
Sub-tasks:
* [ ] Events monitoring
* [ ] State-fetching APIs
* [ ] Flow control
* [ ] Fetch queue
Depends on:
* [Partial tries](#the-new-binary-trie)
* [Bloom filters](#ethereum-bloom-filter)
* [RLPx Sub-protocols](#rlpx-sub-protocols)
* DApp framework
### 16. Stateless Clients
- https://ethresear.ch/t/the-stateless-client-concept/172
- https://github.com/ethereum/sharding/blob/develop/docs/doc.md#stateless-clients
Sub-tasks:
* [ ] Transaction state witness extraction
* [ ] Stateless sync
* [ ] State objects and "pure" VM execution
* [ ] Track updates to all keys in the witness in an in-memory store.
Depends on:
* [LES Implementation](#light-clients)
* [Partial tries](#the-new-binary-trie)
### 17. Casper daemon
- https://arxiv.org/pdf/1710.09437.pdf
- https://github.com/ethereum/research/wiki/Casper-Version-1-Implementation-Guide
- https://github.com/ethereum/casper
- https://github.com/ethereum/wiki/wiki/Proof-of-Stake-FAQ
Sub-tasks:
* [ ] Implement the new Fork-choice rule
* [ ] Monitor epochs, send prepares and commits when appropriate.
* [ ] Avoid triggering the slashing conditions by maintaing a local database of past actions.
* [ ] Provide an UI for adding deposits, tracking revenue and withdrawing funds.
Depends on:
* [Known contract abstraction](#known-contract-abstraction-dapp-framework)
### 18. Sharding daemon
- https://github.com/ethereum/sharding/blob/develop/docs/doc.md
- https://github.com/ethereum/wiki/wiki/Sharding-FAQ
PY-EVM Implementation Roadmap:
- https://github.com/ethereum/py-evm/issues/190
Sub-tasks:
* [ ] Shard monitoring/switching
* [ ] Block/Collation creation
* [ ] New account creation logic
* [ ] New VM opcodes
* [ ] Provide an UI for adding deposits, tracking revenue and withdrawing funds.
Depends on:
* [ ] Binary Trie, Overlayed/Tracked DB back-end.
(used to keep track of modifications created by a transaction and to model the transaction Witness data)
* [ ] Stateless Clients
* [ ] Parametric chain implementation
* [ ] [Known contract abstraction](#known-contract-abstraction-dapp-framework)
### 19. Etehreum name service resolver
- https://docs.ens.domains/en/latest/
Sub-tasks:
* [ ] Resolve names by recursive ENS contract application
Depends on:
* [ ] Functional Blockchain repica / VM
* [ ] [Known contract abstraction](#known-contract-abstraction-dapp-framework)
### 20. Swarm
- http://swarm-guide.readthedocs.io/en/latest/introduction.html
Sub-tasks:
* [ ] Kademlia search
* [ ] File chunks upload / download manager
* [ ] Hierarchical hash verification
* [ ] Virtual file system based on manifests
* [ ] PSS (messaging over swarm)
* [ ] NAT Traversal brokerage over PSS (e.g. for WebRTC)
* [ ] Light mode of operation, relay nodes
* [ ] SWAP, SWEAR, SWINDLE, chequebooks
Depends on:
* Reusable Kademlia routing library
* [ENS resolver](#ethereum-name-service-resolver)
* [RLPx Sub-protocols](#rlpx-sub-protocols)
* DApp framework
### 21. Whisper
While Swarm seems a better fit for Status's needs, we may also implement
the Whisper protocol, which is currently at version 6:
- https://gist.github.com/gluk256/cbdf8d1f05c72c2273cc42bce905388a
Overview information (previous versions):
- https://github.com/ethereum/wiki/wiki/Whisper
- https://github.com/ethereum/go-ethereum/wiki/Whisper
Sub-tasks:
* [ ] Envelope encryption
* [ ] Mail Server
* [ ] Filter table
* [ ] Onion routing (Darkness)
Depends on:
* [RLPx Sub-protocols](#rlpx-sub-protocols)
* [Bloom filters](#ethereum-bloom-filter)
### 22. eWASM VM
Sub-tasks:
* [ ] Integrate Hera
* [ ] Extract all extern methods in a module shared between the VMs
Depends on:
* Parametric chain
### 23. Integrate libsnark
Developers can take advantage of the homomorphic encryption and the
zero-knowledge proofs available in libsnark to implement currencies
and other smart contracts with stronger privacy (similar to zcash).
Ethereum supports this by offering the operations from libsnark as
precompiled contracts.
- https://blog.ethereum.org/2016/12/05/zksnarks-in-a-nutshell/
Depends on:
* [ ] Precompiled contracts
### 24. Support IPFS
No concrete plans yet.
### 25. Command-line interface, Configuration, Local directory structure, Build Targets, Web Interface
Sub-tasks:
* [ ] Provide options for launching the various daemons and modes of Nimbus
* [ ] Define a common scheme for specifying parameters (as command-line switches, as env vars, in configuration files, etc)
* [ ] Provide abstractions over local file system usage (e.g. XDG, AppData, Android and iOS specific APIs)
* [ ] Provide compile-time defines for turning off features of Nimbus (i.e. allow the lite clients to omit most of the code)
* [ ] Test example applications using Nimbus as a library
* [ ] Define various UIs for Nimbus (Web UI, GUI, etc)
* [ ] Android build / CI test suite
* [ ] iOS build / CI test suite
### 26. Nim smart contract development
Define a Nim DSL for creating smart contracts.
Sub-tasks:
* [ ] Support creating library contracts
* [ ] Compile Nim to Solidity ASM or Vyper LLL
* [ ] Compile Nim to eWASM (either through the C back-end or through LLVM)
* [ ] Enhance security through static code analysis / formal methods
### 27. Fuzzing framework
Test the implementation of all building block libraries such as RLP,
the Tries, the VM, etc through a fuzzer such as afl-fuzz. Teach the
fuzzer about Nim.