523 lines
106 KiB
HTML
523 lines
106 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en-US">
|
|
<head>
|
|
<meta charset="utf-8">
|
|
<meta name="viewport" content="width=device-width,initial-scale=1">
|
|
<title>Eth | Nimbus Libraries</title>
|
|
<meta name="generator" content="VuePress 1.8.2">
|
|
<link rel="icon" href="/assets/img/logo.png">
|
|
<meta name="description" content="Ethereum 2.0 utilities and more">
|
|
|
|
<link rel="preload" href="/assets/css/0.styles.74783a08.css" as="style"><link rel="preload" href="/assets/js/app.b6b895fe.js" as="script"><link rel="preload" href="/assets/js/2.fd7e9e0b.js" as="script"><link rel="preload" href="/assets/js/11.b620ddcc.js" as="script"><link rel="prefetch" href="/assets/js/10.a7e5f96d.js"><link rel="prefetch" href="/assets/js/12.cf1bd068.js"><link rel="prefetch" href="/assets/js/13.9dfac9d9.js"><link rel="prefetch" href="/assets/js/14.1ce3f7fe.js"><link rel="prefetch" href="/assets/js/3.c5869ccf.js"><link rel="prefetch" href="/assets/js/4.b9b777bd.js"><link rel="prefetch" href="/assets/js/5.073c37d9.js"><link rel="prefetch" href="/assets/js/6.7db49fe2.js"><link rel="prefetch" href="/assets/js/7.852d0869.js"><link rel="prefetch" href="/assets/js/8.b8ee59fd.js"><link rel="prefetch" href="/assets/js/9.249d32d4.js">
|
|
<link rel="stylesheet" href="/assets/css/0.styles.74783a08.css">
|
|
</head>
|
|
<body>
|
|
<div id="app" data-server-rendered="true"><div class="theme-container"><header class="navbar"><div class="sidebar-button"><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" role="img" viewBox="0 0 448 512" class="icon"><path fill="currentColor" d="M436 124H12c-6.627 0-12-5.373-12-12V80c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12z"></path></svg></div> <a href="/" class="home-link router-link-active"><img src="/assets/img/logo.png" alt="Nimbus Libraries" class="logo"> <span class="site-name can-hide">Nimbus Libraries</span></a> <div class="links"><div class="search-box"><input aria-label="Search" autocomplete="off" spellcheck="false" value=""> <!----></div> <nav class="nav-links can-hide"><div class="nav-item"><a href="/lib/nim-libp2p/" class="nav-link">
|
|
libp2p
|
|
</a></div><div class="nav-item"><a href="/lib/nim-chronicles/" class="nav-link">
|
|
Chronicles
|
|
</a></div><div class="nav-item"><a href="/lib/nimcrypto/" class="nav-link">
|
|
Nimcrypto
|
|
</a></div><div class="nav-item"><a href="/lib/nim-chronos/" class="nav-link">
|
|
Chronos
|
|
</a></div><div class="nav-item"><a href="/lib/nim-eth/" aria-current="page" class="nav-link router-link-exact-active router-link-active">
|
|
Eth
|
|
</a></div><div class="nav-item"><a href="/lib/nim-stew/" class="nav-link">
|
|
Stew
|
|
</a></div> <!----></nav></div></header> <div class="sidebar-mask"></div> <aside class="sidebar"><nav class="nav-links"><div class="nav-item"><a href="/lib/nim-libp2p/" class="nav-link">
|
|
libp2p
|
|
</a></div><div class="nav-item"><a href="/lib/nim-chronicles/" class="nav-link">
|
|
Chronicles
|
|
</a></div><div class="nav-item"><a href="/lib/nimcrypto/" class="nav-link">
|
|
Nimcrypto
|
|
</a></div><div class="nav-item"><a href="/lib/nim-chronos/" class="nav-link">
|
|
Chronos
|
|
</a></div><div class="nav-item"><a href="/lib/nim-eth/" aria-current="page" class="nav-link router-link-exact-active router-link-active">
|
|
Eth
|
|
</a></div><div class="nav-item"><a href="/lib/nim-stew/" class="nav-link">
|
|
Stew
|
|
</a></div> <!----></nav> <ul class="sidebar-links"><li><section class="sidebar-group depth-0"><p class="sidebar-heading open"><span>Eth</span> <!----></p> <ul class="sidebar-links sidebar-group-items"><li><a href="/lib/nim-eth/#rlp" class="sidebar-link">rlp</a><ul class="sidebar-sub-headers"><li class="sidebar-sub-header"><a href="/lib/nim-eth/#introduction" class="sidebar-link">Introduction</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#reading-rlp-data" class="sidebar-link">Reading RLP data</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#streaming-api" class="sidebar-link">Streaming API</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#dom-api" class="sidebar-link">DOM API</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#creating-rlp-data" class="sidebar-link">Creating RLP data</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#object-serialization" class="sidebar-link">Object serialization</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#contributing-testing" class="sidebar-link">Contributing / Testing</a></li></ul></li><li><a href="/lib/nim-eth/#p2p" class="sidebar-link">p2p</a><ul class="sidebar-sub-headers"><li class="sidebar-sub-header"><a href="/lib/nim-eth/#introduction-2" class="sidebar-link">Introduction</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#connecting-to-the-ethereum-network" class="sidebar-link">Connecting to the Ethereum network</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#communicating-with-peers-using-rlpx" class="sidebar-link">Communicating with Peers using RLPx</a></li></ul></li><li><a href="/lib/nim-eth/#keys" class="sidebar-link">keys</a><ul class="sidebar-sub-headers"></ul></li><li><a href="/lib/nim-eth/#keyfile" class="sidebar-link">keyfile</a><ul class="sidebar-sub-headers"><li class="sidebar-sub-header"><a href="/lib/nim-eth/#introduction-3" class="sidebar-link">Introduction</a></li></ul></li><li><a href="/lib/nim-eth/#trie" class="sidebar-link">trie</a><ul class="sidebar-sub-headers"></ul></li><li><a href="/lib/nim-eth/#nim-implementation-of-the-ethereum-trie-structure" class="sidebar-link">Nim Implementation of the Ethereum Trie structure</a><ul class="sidebar-sub-headers"><li class="sidebar-sub-header"><a href="/lib/nim-eth/#hexary-trie" class="sidebar-link">Hexary Trie</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#binary-trie" class="sidebar-link">Binary Trie</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#examples" class="sidebar-link">Examples</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#the-truth-behind-a-lie" class="sidebar-link">The truth behind a lie</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#the-branch-utils" class="sidebar-link">The branch utils</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#remember-the-lie" class="sidebar-link">Remember the lie?</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#sparse-merkle-trie" class="sidebar-link">Sparse Merkle Trie</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#examples-2" class="sidebar-link">Examples</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#merkle-proofing" class="sidebar-link">Merkle Proofing</a></li></ul></li><li><a href="/lib/nim-eth/#bloom-an-ethereum-bloom-filter" class="sidebar-link">bloom: an Ethereum Bloom Filter</a><ul class="sidebar-sub-headers"></ul></li><li><a href="/lib/nim-eth/#introduction-4" class="sidebar-link">Introduction</a><ul class="sidebar-sub-headers"></ul></li><li><a href="/lib/nim-eth/#description" class="sidebar-link">Description</a><ul class="sidebar-sub-headers"></ul></li><li><a href="/lib/nim-eth/#usage" class="sidebar-link">Usage</a><ul class="sidebar-sub-headers"></ul></li><li><a href="/lib/nim-eth/#node-discovery-protocol-v5" class="sidebar-link">Node Discovery Protocol v5</a><ul class="sidebar-sub-headers"><li class="sidebar-sub-header"><a href="/lib/nim-eth/#introduction-5" class="sidebar-link">Introduction</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#how-to-use" class="sidebar-link">How to use</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#test-suite" class="sidebar-link">Test suite</a></li><li class="sidebar-sub-header"><a href="/lib/nim-eth/#dcli" class="sidebar-link">dcli</a></li></ul></li><li><a href="/lib/nim-eth/#prerequisites" class="sidebar-link">Prerequisites</a><ul class="sidebar-sub-headers"></ul></li><li><a href="/lib/nim-eth/#building-testing" class="sidebar-link">Building & Testing</a><ul class="sidebar-sub-headers"></ul></li><li><a href="/lib/nim-eth/#fuzzing" class="sidebar-link">Fuzzing</a><ul class="sidebar-sub-headers"></ul></li></ul></section></li></ul> </aside> <main class="page"> <div class="theme-default-content content__default"><h1 id="eth"><a href="#eth" class="header-anchor">#</a> Eth</h1> <p>Ethereum-related utilities written in Nim. Includes things like Bloom filters, private/public key utilities, RLP, devp2p, and more.</p> <h2 id="rlp"><a href="#rlp" class="header-anchor">#</a> rlp</h2> <h3 id="introduction"><a href="#introduction" class="header-anchor">#</a> Introduction</h3> <p>A Nim implementation of the Recursive Length Prefix encoding (RLP) as specified
|
|
in the Ethereum's <a href="https://ethereum.github.io/yellowpaper/paper.pdf" target="_blank" rel="noopener noreferrer">Yellow Paper<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>
|
|
and <a href="https://github.com/ethereum/wiki/wiki/RLP" target="_blank" rel="noopener noreferrer">Wiki<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>.</p> <h3 id="reading-rlp-data"><a href="#reading-rlp-data" class="header-anchor">#</a> Reading RLP data</h3> <p>The <code>Rlp</code> type provided by this library represents a cursor over an RLP-encoded
|
|
byte stream.</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">proc</span> <span class="token function">rlpFromBytes<span class="token operator">*</span></span><span class="token punctuation">(</span>data<span class="token operator">:</span> openArray<span class="token punctuation">[</span>byte<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token operator">:</span> Rlp
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br></div></div><h3 id="streaming-api"><a href="#streaming-api" class="header-anchor">#</a> Streaming API</h3> <p>Once created, the <code>Rlp</code> object will offer procs such as <code>isList</code>, <code>isBlob</code>,
|
|
<code>getType</code>, <code>listLen</code>, <code>blobLen</code> to determine the type of the value under
|
|
the cursor. The contents of blobs can be extracted with procs such as
|
|
<code>toString</code>, <code>toBytes</code> and <code>toInt</code> without advancing the cursor.</p> <p>Lists can be traversed with the standard <code>items</code> iterator, which will advance
|
|
the cursor to each sub-item position and yield the <code>Rlp</code> object at that point.
|
|
As an alternative, <code>listElem</code> can return a new <code>Rlp</code> object adjusted to a
|
|
particular sub-item position without advancing the original cursor.
|
|
Keep in mind that copying <code>Rlp</code> objects is cheap and you can create as many
|
|
cursors pointing to different positions in the RLP stream as necessary.</p> <p><code>skipElem</code> will advance the cursor to the next position in the current list.
|
|
<code>hasData</code> will indicate that there are no more bytes in the stream that can
|
|
be consumed.</p> <p>Another way to extract data from the stream is through the universal <code>read</code>
|
|
proc that accepts a type as a parameter. You can pass any supported type
|
|
such as <code>string</code>, <code>int</code>, <code>seq[T]</code>, etc, including composite user-defined
|
|
types (see <a href="#object-serialization">Object Serialization</a>). The cursor
|
|
will be advanced just past the end of the consumed object.</p> <p>The <code>toXX</code> and <code>read</code> family of procs may raise a <code>RlpTypeMismatch</code> in case
|
|
of type mismatch with the stream contents under the cursor. A corrupted
|
|
RLP stream or an attemp to read past the stream end will be signaled
|
|
with the <code>MalformedRlpError</code> exception. If the RLP stream includes data
|
|
that cannot be processed on the current platform (e.g. an integer value
|
|
that is too large), the library will raise an <code>UnsupportedRlpError</code> exception.</p> <h3 id="dom-api"><a href="#dom-api" class="header-anchor">#</a> DOM API</h3> <p>Calling <code>Rlp.toNodes</code> at any position within the stream will return a tree
|
|
of <code>RlpNode</code> objects representing the collection of values starting at that
|
|
position:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">type</span>
|
|
RlpNodeType<span class="token operator">*</span> <span class="token operator">=</span> <span class="token keyword">enum</span>
|
|
rlpBlob
|
|
rlpList
|
|
|
|
RlpNode<span class="token operator">*</span> <span class="token operator">=</span> <span class="token keyword">object</span>
|
|
<span class="token keyword">case</span> kind<span class="token operator">*:</span> RlpNodeType
|
|
<span class="token operator">of</span> rlpBlob<span class="token operator">:</span>
|
|
bytes<span class="token operator">*:</span> seq<span class="token punctuation">[</span>byte<span class="token punctuation">]</span>
|
|
<span class="token operator">of</span> rlpList<span class="token operator">:</span>
|
|
elems<span class="token operator">*:</span> seq<span class="token punctuation">[</span>RlpNode<span class="token punctuation">]</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br><span class="line-number">10</span><br><span class="line-number">11</span><br></div></div><p>As a short-cut, you can also call <code>decode</code> directly on a byte sequence to
|
|
avoid creating a <code>Rlp</code> object when obtaining the nodes.
|
|
For debugging purposes, you can also create a human readable representation
|
|
of the Rlp nodes by calling the <code>inspect</code> proc:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">proc</span> <span class="token function">inspect<span class="token operator">*</span></span><span class="token punctuation">(</span>self<span class="token operator">:</span> Rlp<span class="token punctuation">,</span> indent <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token operator">:</span> string
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br></div></div><h3 id="creating-rlp-data"><a href="#creating-rlp-data" class="header-anchor">#</a> Creating RLP data</h3> <p>The <code>RlpWriter</code> type can be used to encode RLP data. Instances are created
|
|
with the <code>initRlpWriter</code> proc. This should be followed by one or more calls
|
|
to <code>append</code> which is overloaded to accept arbitrary values. Finally, you can
|
|
call <code>finish</code> to obtain the final <code>seq[byte]</code>.</p> <p>If the end result should be a RLP list of particular length, you can replace
|
|
the initial call to <code>initRlpWriter</code> with <code>initRlpList(n)</code>. Calling <code>finish</code>
|
|
before writing the sufficient number of elements will then result in an assertion failure.</p> <p>As an alternative short-cut, you can also call <code>encode</code> on an arbitrary value
|
|
(including sequences and user-defined types) to execute all of the steps at
|
|
once and directly obtain the final RLP bytes. <code>encodeList(varargs)</code> is another
|
|
short-cut for creating RLP lists.</p> <h3 id="object-serialization"><a href="#object-serialization" class="header-anchor">#</a> Object serialization</h3> <p>As previously explained, generic procs such as <code>read</code>, <code>append</code>, <code>encode</code> and
|
|
<code>decode</code> can be used with arbitrary used-defined object types. By default, the
|
|
library will serialize all of the fields of the object using the <code>fields</code>
|
|
iterator, but you can also include only a subset of the fields or modify the
|
|
order of serialization or by employing the <code>rlpIgnore</code> pragma or by using the
|
|
<code>rlpFields</code> macro:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">macro</span> <span class="token function">rlpFields<span class="token operator">*</span></span><span class="token punctuation">(</span>T<span class="token operator">:</span> typedesc<span class="token punctuation">,</span> fields<span class="token operator">:</span> varargs<span class="token punctuation">[</span>untyped<span class="token punctuation">]</span><span class="token punctuation">)</span>
|
|
|
|
<span class="token comment">## example usage:</span>
|
|
|
|
<span class="token keyword">type</span>
|
|
Transaction <span class="token operator">=</span> <span class="token keyword">object</span>
|
|
amount<span class="token operator">:</span> int
|
|
time<span class="token operator">:</span> DateTime
|
|
sender<span class="token operator">:</span> string
|
|
receiver<span class="token operator">:</span> string
|
|
|
|
rlpFields Transaction<span class="token punctuation">,</span>
|
|
sender<span class="token punctuation">,</span> receiver<span class="token punctuation">,</span> amount
|
|
|
|
<span class="token operator">...</span>
|
|
|
|
<span class="token keyword">var</span> t1 <span class="token operator">=</span> rlp<span class="token operator">.</span><span class="token function">read</span><span class="token punctuation">(</span>Transaction<span class="token punctuation">)</span>
|
|
<span class="token keyword">var</span> bytes <span class="token operator">=</span> <span class="token function">encode</span><span class="token punctuation">(</span>t1<span class="token punctuation">)</span>
|
|
<span class="token keyword">var</span> t2 <span class="token operator">=</span> bytes<span class="token operator">.</span><span class="token function">decode</span><span class="token punctuation">(</span>Transaction<span class="token punctuation">)</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br><span class="line-number">10</span><br><span class="line-number">11</span><br><span class="line-number">12</span><br><span class="line-number">13</span><br><span class="line-number">14</span><br><span class="line-number">15</span><br><span class="line-number">16</span><br><span class="line-number">17</span><br><span class="line-number">18</span><br><span class="line-number">19</span><br></div></div><p>By default, sub-fields within objects are wrapped in RLP lists. You can avoid this
|
|
behavior by adding the custom pragma <code>rlpInline</code> on a particular field. In rare
|
|
circumstances, you may need to serialize the same field type differently depending
|
|
on the enclosing object type. You can use the <code>rlpCustomSerialization</code> pragma to
|
|
achieve this.</p> <h3 id="contributing-testing"><a href="#contributing-testing" class="header-anchor">#</a> Contributing / Testing</h3> <p>To test the correctness of any modifications to the library, please execute
|
|
<code>nimble test_rlp</code> at the root of the repo.</p> <h2 id="p2p"><a href="#p2p" class="header-anchor">#</a> p2p</h2> <h3 id="introduction-2"><a href="#introduction-2" class="header-anchor">#</a> Introduction</h3> <p>This library implements the DevP2P family of networking protocols used
|
|
in the Ethereum world.</p> <h3 id="connecting-to-the-ethereum-network"><a href="#connecting-to-the-ethereum-network" class="header-anchor">#</a> Connecting to the Ethereum network</h3> <p>A connection to the Ethereum network can be created by instantiating
|
|
the <code>EthereumNode</code> type:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">proc</span> <span class="token function">newEthereumNode<span class="token operator">*</span></span><span class="token punctuation">(</span>keys<span class="token operator">:</span> KeyPair<span class="token punctuation">,</span>
|
|
listeningAddress<span class="token operator">:</span> Address<span class="token punctuation">,</span>
|
|
networkId<span class="token operator">:</span> uint<span class="token punctuation">,</span>
|
|
chain<span class="token operator">:</span> AbstractChainDB<span class="token punctuation">,</span>
|
|
clientId <span class="token operator">=</span> <span class="token string">"nim-eth-p2p"</span><span class="token punctuation">,</span>
|
|
addAllCapabilities <span class="token operator">=</span> true<span class="token punctuation">)</span><span class="token operator">:</span> EthereumNode <span class="token operator">=</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br></div></div><h4 id="parameters"><a href="#parameters" class="header-anchor">#</a> Parameters:</h4> <p><code>keys</code>:
|
|
A pair of public and private keys used to authenticate the node
|
|
on the network and to determine its node ID.
|
|
See the <a href="/lib/nim-eth/keys.html">keys</a>
|
|
library for utilities that will help you generate and manage
|
|
such keys.</p> <p><code>listeningAddress</code>:
|
|
The network interface and port where your client will be
|
|
accepting incoming connections.</p> <p><code>networkId</code>:
|
|
The Ethereum network ID. The client will disconnect immediately
|
|
from any peers who don't use the same network.</p> <p><code>chain</code>:
|
|
An abstract instance of the Ethereum blockchain associated
|
|
with the node. This library allows you to plug any instance
|
|
conforming to the abstract interface defined in the
|
|
<a href="https://github.com/status-im/nim-eth-common" target="_blank" rel="noopener noreferrer">eth_common<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>
|
|
package.</p> <p><code>clientId</code>:
|
|
A name used to identify the software package connecting
|
|
to the network (i.e. similar to the <code>User-Agent</code> string
|
|
in a browser).</p> <p><code>addAllCapabilities</code>:
|
|
By default, the node will support all RPLx protocols imported in
|
|
your project. You can specify <code>false</code> if you prefer to create a
|
|
node with a more limited set of protocols. Use one or more calls
|
|
to <code>node.addCapability</code> to specify the desired set:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code>node<span class="token operator">.</span><span class="token function">addCapability</span><span class="token punctuation">(</span>eth<span class="token punctuation">)</span>
|
|
node<span class="token operator">.</span><span class="token function">addCapability</span><span class="token punctuation">(</span>shh<span class="token punctuation">)</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br></div></div><p>Each supplied protocol identifier is a name of a protocol introduced
|
|
by the <code>p2pProtocol</code> macro discussed later in this document.</p> <p>Instantiating an <code>EthereumNode</code> does not immediately connect you to
|
|
the network. To start the connection process, call <code>node.connectToNetwork</code>:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">proc</span> <span class="token function">connectToNetwork<span class="token operator">*</span></span><span class="token punctuation">(</span>node<span class="token operator">:</span> <span class="token keyword">var</span> EthereumNode<span class="token punctuation">,</span>
|
|
bootstrapNodes<span class="token operator">:</span> openarray<span class="token punctuation">[</span>ENode<span class="token punctuation">]</span><span class="token punctuation">,</span>
|
|
startListening <span class="token operator">=</span> true<span class="token punctuation">,</span>
|
|
enableDiscovery <span class="token operator">=</span> true<span class="token punctuation">)</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br></div></div><p>The <code>EthereumNode</code> will automatically find and maintain a pool of peers
|
|
using the Ethereum node discovery protocol. You can access the pool as
|
|
<code>node.peers</code>.</p> <h3 id="communicating-with-peers-using-rlpx"><a href="#communicating-with-peers-using-rlpx" class="header-anchor">#</a> Communicating with Peers using RLPx</h3> <p><a href="https://github.com/ethereum/devp2p/blob/master/rlpx.md" target="_blank" rel="noopener noreferrer">RLPx<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> is the
|
|
high-level protocol for exchanging messages between peers in the Ethereum
|
|
network. Most of the client code of this library should not be concerned
|
|
with the implementation details of the underlying protocols and should use
|
|
the high-level APIs described in this section.</p> <p>The RLPx protocols are defined as a collection of strongly-typed messages,
|
|
which are grouped into sub-protocols multiplexed over the same TCP connection.</p> <p>This library represents each such message as a regular Nim function call
|
|
over the <code>Peer</code> object. Certain messages act only as notifications, while
|
|
others fit the request/response pattern.</p> <p>To understand more about how messages are defined and used, let's look at
|
|
the definition of a RLPx protocol:</p> <h4 id="rlpx-sub-protocols"><a href="#rlpx-sub-protocols" class="header-anchor">#</a> RLPx sub-protocols</h4> <p>The sub-protocols are defined with the <code>p2pProtocol</code> macro. It will accept
|
|
a short identifier for the protocol and the current protocol version:</p> <p>Here is how the <a href="https://github.com/ethereum/devp2p/blob/master/rlpx.md#p2p-capability" target="_blank" rel="noopener noreferrer">DevP2P wire protocol<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> might look like:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code>p2pProtocol <span class="token function">DevP2P</span><span class="token punctuation">(</span>version <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">,</span> rlpxName <span class="token operator">=</span> <span class="token string">"p2p"</span><span class="token punctuation">)</span><span class="token operator">:</span>
|
|
<span class="token keyword">proc</span> <span class="token function">hello</span><span class="token punctuation">(</span>peer<span class="token operator">:</span> Peer<span class="token punctuation">,</span>
|
|
version<span class="token operator">:</span> uint<span class="token punctuation">,</span>
|
|
clientId<span class="token operator">:</span> string<span class="token punctuation">,</span>
|
|
capabilities<span class="token operator">:</span> openarray<span class="token punctuation">[</span>Capability<span class="token punctuation">]</span><span class="token punctuation">,</span>
|
|
listenPort<span class="token operator">:</span> uint<span class="token punctuation">,</span>
|
|
nodeId<span class="token operator">:</span> P2PNodeId<span class="token punctuation">)</span> <span class="token operator">=</span>
|
|
peer<span class="token operator">.</span>id <span class="token operator">=</span> nodeId
|
|
|
|
<span class="token keyword">proc</span> <span class="token function">disconnect</span><span class="token punctuation">(</span>peer<span class="token operator">:</span> Peer<span class="token punctuation">,</span> reason<span class="token operator">:</span> DisconnectionReason<span class="token punctuation">)</span>
|
|
|
|
<span class="token keyword">proc</span> <span class="token function">ping</span><span class="token punctuation">(</span>peer<span class="token operator">:</span> Peer<span class="token punctuation">)</span> <span class="token operator">=</span>
|
|
await peer<span class="token operator">.</span><span class="token function">pong</span><span class="token punctuation">(</span><span class="token punctuation">)</span>
|
|
|
|
<span class="token keyword">proc</span> <span class="token function">pong</span><span class="token punctuation">(</span>peer<span class="token operator">:</span> Peer<span class="token punctuation">)</span> <span class="token operator">=</span>
|
|
echo <span class="token string">"received pong from "</span><span class="token punctuation">,</span> peer<span class="token operator">.</span>id
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br><span class="line-number">10</span><br><span class="line-number">11</span><br><span class="line-number">12</span><br><span class="line-number">13</span><br><span class="line-number">14</span><br><span class="line-number">15</span><br><span class="line-number">16</span><br></div></div><p>As seen in the example above, a protocol definition determines both the
|
|
available messages that can be sent to another peer (e.g. as in <code>peer.pong()</code>)
|
|
and the asynchronous code responsible for handling the incoming messages.</p> <h4 id="protocol-state"><a href="#protocol-state" class="header-anchor">#</a> Protocol state</h4> <p>The protocol implementations are expected to maintain a state and to act
|
|
like a state machine handling the incoming messages. You are allowed to
|
|
define an arbitrary state type that can be specified in the <code>peerState</code>
|
|
protocol option. Later, instances of the state object can be obtained
|
|
though the <code>state</code> pseudo-field of the <code>Peer</code> object:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">type</span> AbcPeerState <span class="token operator">=</span> <span class="token keyword">object</span>
|
|
receivedMsgsCount<span class="token operator">:</span> int
|
|
|
|
p2pProtocol <span class="token function">abc</span><span class="token punctuation">(</span>version <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span>
|
|
peerState <span class="token operator">=</span> AbcPeerState<span class="token punctuation">)</span><span class="token operator">:</span>
|
|
|
|
<span class="token keyword">proc</span> <span class="token function">incomingMessage</span><span class="token punctuation">(</span>p<span class="token operator">:</span> Peer<span class="token punctuation">)</span> <span class="token operator">=</span>
|
|
p<span class="token operator">.</span>state<span class="token operator">.</span>receivedMsgsCount <span class="token operator">+=</span> <span class="token number">1</span>
|
|
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br></div></div><p>Besides the per-peer state demonstrated above, there is also support
|
|
for maintaining a network-wide state. It's enabled by specifying the
|
|
<code>networkState</code> option of the protocol and the state object can be obtained
|
|
through accessor of the same name.</p> <p>The state objects are initialized to zero by default, but you can modify
|
|
this behaviour by overriding the following procs for your state types:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">proc</span> <span class="token function">initProtocolState<span class="token operator">*</span></span><span class="token punctuation">(</span>state<span class="token operator">:</span> MyPeerState<span class="token punctuation">,</span> p<span class="token operator">:</span> Peer<span class="token punctuation">)</span>
|
|
<span class="token keyword">proc</span> <span class="token function">initProtocolState<span class="token operator">*</span></span><span class="token punctuation">(</span>state<span class="token operator">:</span> MyNetworkState<span class="token punctuation">,</span> n<span class="token operator">:</span> EthereumNode<span class="token punctuation">)</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br></div></div><p>Sometimes, you'll need to access the state of another protocol.
|
|
To do this, specify the protocol identifier to the <code>state</code> accessors:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code> echo <span class="token string">"ABC protocol messages: "</span><span class="token punctuation">,</span> peer<span class="token operator">.</span><span class="token function">state</span><span class="token punctuation">(</span>abc<span class="token punctuation">)</span><span class="token operator">.</span>receivedMsgCount
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br></div></div><p>While the state machine approach may be a particularly robust way of
|
|
implementing sub-protocols (it is more amenable to proving the correctness
|
|
of the implementation through formal verification methods), sometimes it may
|
|
be more convenient to use more imperative style of communication where the
|
|
code is able to wait for a particular response after sending a particular
|
|
request. The library provides two mechanisms for achieving this:</p> <h4 id="waiting-particular-messages-with-nextmsg"><a href="#waiting-particular-messages-with-nextmsg" class="header-anchor">#</a> Waiting particular messages with <code>nextMsg</code></h4> <p>The <code>nextMsg</code> helper proc can be used to pause the execution of an async
|
|
proc until a particular incoming message from a peer arrives:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">proc</span> <span class="token function">helloExample</span><span class="token punctuation">(</span>peer<span class="token operator">:</span> Peer<span class="token punctuation">)</span> <span class="token operator">=</span>
|
|
<span class="token operator">...</span>
|
|
<span class="token comment"># send a hello message</span>
|
|
await peer<span class="token operator">.</span><span class="token function">hello</span><span class="token punctuation">(</span><span class="token operator">..</span><span class="token punctuation">.)</span>
|
|
|
|
<span class="token comment"># wait for a matching hello response, might want to add a timeout here</span>
|
|
<span class="token keyword">let</span> response <span class="token operator">=</span> await peer<span class="token operator">.</span><span class="token function">nextMsg</span><span class="token punctuation">(</span>p2p<span class="token operator">.</span>hello<span class="token punctuation">)</span>
|
|
echo response<span class="token operator">.</span>clientId <span class="token comment"># print the name of the Ethereum client</span>
|
|
<span class="token comment"># used by the other peer (Geth, Parity, Nimbus, etc)</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br></div></div><p>There are few things to note in the above example:</p> <ol><li><p>The <code>p2pProtocol</code> definition created a pseudo-variable named after the
|
|
protocol holding various properties of the protocol.</p></li> <li><p>Each message defined in the protocol received a corresponding type name,
|
|
matching the message name (e.g. <code>p2p.hello</code>). This type will have fields
|
|
matching the parameter names of the message. If the messages has <code>openarray</code>
|
|
params, these will be remapped to <code>seq</code> types.</p></li></ol> <p>If the designated messages also has an attached handler, the future returned
|
|
by <code>nextMsg</code> will be resolved only after the handler has been fully executed
|
|
(so you can count on any side effects produced by the handler to have taken
|
|
place). If there are multiple outstanding calls to <code>nextMsg</code>, they will
|
|
complete together. Any other messages received in the meantime will still
|
|
be dispatched to their respective handlers.</p> <p>Please also note that the <code>p2pProtocol</code> macro will make this <code>helloExample</code> proc
|
|
<code>async</code>. Practically see it as <code>proc helloExample(peer: Peer) {.async.}</code>, and
|
|
thus never use <code>waitFor</code>, but rather <code>await</code> inside this proc.</p> <p>For implementing protocol handshakes with <code>nextMsg</code> there are specific helpers
|
|
which are explained <a href="https://github.com/status-im/nim-eth/blob/master/doc/p2p.md#implementing-handshakes-and-reacting-to-other-events" target="_blank" rel="noopener noreferrer">below<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>.</p> <h4 id="requestresponse-pairs"><a href="#requestresponse-pairs" class="header-anchor">#</a> <code>requestResponse</code> pairs</h4> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code>p2pProtocol <span class="token function">les</span><span class="token punctuation">(</span>version <span class="token operator">=</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token operator">:</span>
|
|
<span class="token operator">...</span>
|
|
|
|
requestResponse<span class="token operator">:</span>
|
|
<span class="token keyword">proc</span> <span class="token function">getProofs</span><span class="token punctuation">(</span>p<span class="token operator">:</span> Peer<span class="token punctuation">,</span> proofs<span class="token operator">:</span> openarray<span class="token punctuation">[</span>ProofRequest<span class="token punctuation">]</span><span class="token punctuation">)</span>
|
|
<span class="token keyword">proc</span> <span class="token function">proofs</span><span class="token punctuation">(</span>p<span class="token operator">:</span> Peer<span class="token punctuation">,</span> BV<span class="token operator">:</span> uint<span class="token punctuation">,</span> proofs<span class="token operator">:</span> openarray<span class="token punctuation">[</span>Blob<span class="token punctuation">]</span><span class="token punctuation">)</span>
|
|
|
|
<span class="token operator">...</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br></div></div><p>Two or more messages within the protocol may be grouped into a
|
|
<code>requestResponse</code> block. The last message in the group is assumed
|
|
to be the response while all other messages are considered requests.</p> <p>When a request message is sent, the return type will be a <code>Future</code>
|
|
that will be completed once the response is received. Please note
|
|
that there is a mandatory timeout parameter, so the actual return
|
|
type is <code>Future[Option[MessageType]]</code>. The <code>timeout</code> parameter can
|
|
be specified for each individual call and the default value can be
|
|
overridden on the level of individual message, or the entire protocol:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code>p2pProtocol <span class="token function">abc</span><span class="token punctuation">(</span>version <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">,</span>
|
|
useRequestIds <span class="token operator">=</span> false<span class="token punctuation">,</span>
|
|
timeout <span class="token operator">=</span> <span class="token number">5000</span><span class="token punctuation">)</span><span class="token operator">:</span> <span class="token comment"># value in milliseconds</span>
|
|
requestResponse<span class="token operator">:</span>
|
|
<span class="token keyword">proc</span> <span class="token function">myReq</span><span class="token punctuation">(</span>dataId<span class="token operator">:</span> int<span class="token punctuation">,</span> timeout <span class="token operator">=</span> <span class="token number">3000</span><span class="token punctuation">)</span>
|
|
<span class="token keyword">proc</span> <span class="token function">myRes</span><span class="token punctuation">(</span>data<span class="token operator">:</span> string<span class="token punctuation">)</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br></div></div><p>By default, the library will take care of inserting a hidden <code>reqId</code>
|
|
parameter as used in the <a href="https://github.com/zsfelfoldi/go-ethereum/wiki/Light-Ethereum-Subprotocol-%28LES%29" target="_blank" rel="noopener noreferrer">LES protocol<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>,
|
|
but you can disable this behavior by overriding the protocol setting
|
|
<code>useRequestIds</code>.</p> <h4 id="implementing-handshakes-and-reacting-to-other-events"><a href="#implementing-handshakes-and-reacting-to-other-events" class="header-anchor">#</a> Implementing handshakes and reacting to other events</h4> <p>Besides message definitions and implementations, a protocol specification may
|
|
also include handlers for certain important events such as newly connected
|
|
peers or misbehaving or disconnecting peers:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code>p2pProtocol <span class="token function">foo</span><span class="token punctuation">(</span>version <span class="token operator">=</span> fooVersion<span class="token punctuation">)</span><span class="token operator">:</span>
|
|
onPeerConnected <span class="token keyword">do</span> <span class="token punctuation">(</span>peer<span class="token operator">:</span> Peer<span class="token punctuation">)</span><span class="token operator">:</span>
|
|
<span class="token keyword">let</span> m <span class="token operator">=</span> await peer<span class="token operator">.</span><span class="token function">status</span><span class="token punctuation">(</span>fooVersion<span class="token punctuation">,</span>
|
|
timeout <span class="token operator">=</span> chronos<span class="token operator">.</span><span class="token function">milliseconds</span><span class="token punctuation">(</span><span class="token number">5000</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
|
|
|
|
<span class="token keyword">if</span> m<span class="token operator">.</span>protocolVersion <span class="token operator">==</span> fooVersion<span class="token operator">:</span>
|
|
debug <span class="token string">"Foo peer"</span><span class="token punctuation">,</span> peer<span class="token punctuation">,</span> fooVersion
|
|
<span class="token keyword">else</span><span class="token operator">:</span>
|
|
<span class="token keyword">raise</span> <span class="token function">newException</span><span class="token punctuation">(</span>UselessPeerError<span class="token punctuation">,</span> <span class="token string">"Incompatible Foo version"</span><span class="token punctuation">)</span>
|
|
|
|
onPeerDisconnected <span class="token keyword">do</span> <span class="token punctuation">(</span>peer<span class="token operator">:</span> Peer<span class="token punctuation">,</span> reason<span class="token operator">:</span> DisconnectionReason<span class="token punctuation">)</span><span class="token operator">:</span>
|
|
debug <span class="token string">"peer disconnected"</span><span class="token punctuation">,</span> peer
|
|
|
|
handshake<span class="token operator">:</span>
|
|
<span class="token keyword">proc</span> <span class="token function">status</span><span class="token punctuation">(</span>peer<span class="token operator">:</span> Peer<span class="token punctuation">,</span>
|
|
protocolVersion<span class="token operator">:</span> uint<span class="token punctuation">)</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br><span class="line-number">10</span><br><span class="line-number">11</span><br><span class="line-number">12</span><br><span class="line-number">13</span><br><span class="line-number">14</span><br><span class="line-number">15</span><br><span class="line-number">16</span><br></div></div><p>For handshake messages, where the same type of message needs to be send to and
|
|
received from the peer, a <code>handshake</code> helper is introduced, as you can see in
|
|
the code example above.</p> <p>Thanks to the <code>handshake</code> helper the <code>status</code> message will both be send, and be
|
|
awaited for receival from the peer, with the defined timeout. In case no <code>status</code>
|
|
message is received within the defined timeout, an error will be raised which
|
|
will result in a disconnect from the peer.</p> <p><strong>Note:</strong> Be aware that if currently one of the subprotocol <code>onPeerConnected</code>
|
|
calls fails, the client will be disconnected as <code>UselessPeer</code> but no
|
|
<code>onPeerDisconnect</code> calls are run.</p> <h4 id="checking-the-other-peer-s-supported-sub-protocols"><a href="#checking-the-other-peer-s-supported-sub-protocols" class="header-anchor">#</a> Checking the other peer's supported sub-protocols</h4> <p>Upon establishing a connection, RLPx will automatically negotiate the list of
|
|
mutually supported protocols by the peers. To check whether a particular peer
|
|
supports a particular sub-protocol, use the following code:</p> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">if</span> peer<span class="token operator">.</span><span class="token function">supports</span><span class="token punctuation">(</span>les<span class="token punctuation">)</span><span class="token operator">:</span> <span class="token comment"># `les` is the identifier of the light clients sub-protocol</span>
|
|
peer<span class="token operator">.</span><span class="token function">getReceipts</span><span class="token punctuation">(</span><span class="token function">nextReqId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">neededReceipts</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
|
|
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br></div></div><h2 id="keys"><a href="#keys" class="header-anchor">#</a> keys</h2> <p>This library is a Nim re-implementation of <a href="https://github.com/ethereum/eth-keys" target="_blank" rel="noopener noreferrer">eth-keys<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>: the common API for working with Ethereum's public and private keys, signatures, and addresses.</p> <p>By default, Nim eth-keys uses Bitcoin's <a href="https://github.com/bitcoin-core/secp256k1" target="_blank" rel="noopener noreferrer">libsecp256k1<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> as a backend. Make sure libsecp256k1 is available on your system.</p> <p>An experimental pure Nim backend (Warning ⚠: do not use in production) is available with the compilation switch <code>-d:backend_native</code></p> <h2 id="keyfile"><a href="#keyfile" class="header-anchor">#</a> keyfile</h2> <h3 id="introduction-3"><a href="#introduction-3" class="header-anchor">#</a> Introduction</h3> <p>This library is a Nim reimplementation of <a href="https://github.com/ethereum/eth-keyfile" target="_blank" rel="noopener noreferrer">ethereum/eth-keyfile<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>, which is used to create and load Ethereum <code>keyfile</code> format and the tools for handling the format and for storing private keys. Currently, the library supports only the PBKDF2 method and does not support the Scrypt method.</p> <h2 id="trie"><a href="#trie" class="header-anchor">#</a> trie</h2> <h2 id="nim-implementation-of-the-ethereum-trie-structure"><a href="#nim-implementation-of-the-ethereum-trie-structure" class="header-anchor">#</a> Nim Implementation of the Ethereum Trie structure</h2> <h3 id="hexary-trie"><a href="#hexary-trie" class="header-anchor">#</a> Hexary Trie</h3> <h3 id="binary-trie"><a href="#binary-trie" class="header-anchor">#</a> Binary Trie</h3> <p>Binary-trie is a dictionary-like data structure to store key-value pair.
|
|
Much like it's sibling Hexary-trie, the key-value pair will be stored into key-value flat-db.
|
|
The primary difference with Hexary-trie is, each node of Binary-trie only consist of one or two child,
|
|
while Hexary-trie node can contains up to 16 or 17 child-nodes.</p> <p>Unlike Hexary-trie, Binary-trie store it's data into flat-db without using rlp encoding.
|
|
Binary-trie store its value using simple <strong>Node-Types</strong> encoding.
|
|
The encoded-node will be hashed by keccak_256 and the hash value will be the key to flat-db.
|
|
Each entry in the flat-db will looks like:</p> <table><thead><tr><th>key</th> <th>value</th></tr></thead> <tbody><tr><td>32-bytes-keccak-hash</td> <td>encoded-node(KV or BRANCH or LEAF encoded)</td></tr></tbody></table> <h4 id="node-types"><a href="#node-types" class="header-anchor">#</a> Node-Types</h4> <ul><li>KV = [0, encoded-key-path, 32 bytes hash of child]</li> <li>BRANCH = [1, 32 bytes hash of left child, 32 bytes hash of right child]</li> <li>LEAF = [2, value]</li></ul> <p>The KV node can have BRANCH node or LEAF node as it's child, but cannot a KV node.
|
|
The internal algorithm will merge a KV(parent)->KV(child) into one KV node.
|
|
Every KV node contains encoded keypath to reduce the number of blank nodes.</p> <p>The BRANCH node can have KV, BRANCH, or LEAF node as it's children.</p> <p>The LEAF node is the terminal node, it contains the value of a key.</p> <h4 id="encoded-key-path"><a href="#encoded-key-path" class="header-anchor">#</a> encoded-key-path</h4> <p>While Hexary-trie encode the path using Hex-Prefix encoding, Binary-trie
|
|
encode the path using binary encoding, the scheme looks like this table below.</p> <div class="language-text line-numbers-mode"><pre class="language-text"><code> |--------- odd --------|
|
|
00mm yyyy xxxx xxxx xxxx xxxx
|
|
|------ even -----|
|
|
1000 00mm yyyy xxxx xxxx xxxx
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br></div></div><table><thead><tr><th>symbol</th> <th>explanation</th></tr></thead> <tbody><tr><td>xxxx</td> <td>nibble of binary keypath in bits, 0 = left, 1 = right</td></tr> <tr><td>yyyy</td> <td>nibble contains 0-3 bits padding + binary keypath</td></tr> <tr><td>mm</td> <td>number of binary keypath bits modulo 4 (0-3)</td></tr> <tr><td>00</td> <td>zero zero prefix</td></tr> <tr><td>1000</td> <td>even numbered nibbles prefix</td></tr></tbody></table> <p>if there is no padding, then yyyy bit sequence is absent, mm also zero.
|
|
yyyy = mm bits + padding bits must be 4 bits length.</p> <h4 id="the-api"><a href="#the-api" class="header-anchor">#</a> The API</h4> <p>The primary API for Binary-trie is <code>set</code> and <code>get</code>.</p> <ul><li>set(key, value) --- <em>store a value associated with a key</em></li> <li>get(key): value --- <em>get a value using a key</em></li></ul> <p>Both <code>key</code> and <code>value</code> are of <code>seq[byte]</code> type. And they cannot have zero length.</p> <p>Getting a non-existent key will return zero length seq[byte].</p> <p>Binary-trie also provide dictionary syntax API for <code>set</code> and <code>get</code>.</p> <ul><li>trie[key] = value -- same as <code>set</code></li> <li>value = trie[key] -- same as <code>get</code></li> <li>contains(key) a.k.a. <code>in</code> operator</li></ul> <p>Additional APIs are:</p> <ul><li>exists(key) -- returns <code>bool</code>, to check key-value existence -- same as contains</li> <li>delete(key) -- remove a key-value from the trie</li> <li>deleteSubtrie(key) -- remove a key-value from the trie plus all of it's subtrie
|
|
that starts with the same key prefix</li> <li>rootNode() -- get root node</li> <li>rootNode(node) -- replace the root node</li> <li>getRootHash(): <code>KeccakHash</code> with <code>seq[byte]</code> type</li> <li>getDB(): <code>DB</code> -- get flat-db pointer</li></ul> <p>Constructor API:</p> <ul><li>initBinaryTrie(DB, rootHash[optional]) -- rootHash has <code>seq[byte]</code> or KeccakHash type</li> <li>init(BinaryTrie, DB, rootHash[optional])</li></ul> <p>Normally you would not set the rootHash when constructing an empty Binary-trie.
|
|
Setting the rootHash occured in a scenario where you have a populated DB
|
|
with existing trie structure and you know the rootHash,
|
|
and then you want to continue/resume the trie operations.</p> <h3 id="examples"><a href="#examples" class="header-anchor">#</a> Examples</h3> <div class="language-Nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">import</span>
|
|
eth<span class="token operator">/</span>trie<span class="token operator">/</span><span class="token punctuation">[</span>db<span class="token punctuation">,</span> binary<span class="token punctuation">,</span> utils<span class="token punctuation">]</span>
|
|
|
|
<span class="token keyword">var</span> db <span class="token operator">=</span> <span class="token function">newMemoryDB</span><span class="token punctuation">(</span><span class="token punctuation">)</span>
|
|
<span class="token keyword">var</span> trie <span class="token operator">=</span> <span class="token function">initBinaryTrie</span><span class="token punctuation">(</span>db<span class="token punctuation">)</span>
|
|
trie<span class="token operator">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">"key1"</span><span class="token punctuation">,</span> <span class="token string">"value1"</span><span class="token punctuation">)</span>
|
|
trie<span class="token operator">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">"key2"</span><span class="token punctuation">,</span> <span class="token string">"value2"</span><span class="token punctuation">)</span>
|
|
doAssert trie<span class="token operator">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token string">"key1"</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">"value1"</span><span class="token operator">.</span>toBytes
|
|
doAssert trie<span class="token operator">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token string">"key2"</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">"value2"</span><span class="token operator">.</span>toBytes
|
|
|
|
<span class="token comment">## delete all subtrie with key prefixes "key"</span>
|
|
trie<span class="token operator">.</span><span class="token function">deleteSubtrie</span><span class="token punctuation">(</span><span class="token string">"key"</span><span class="token punctuation">)</span>
|
|
doAssert trie<span class="token operator">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token string">"key1"</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token punctuation">[</span><span class="token punctuation">]</span>
|
|
doAssert trie<span class="token operator">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token string">"key2"</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">]</span>
|
|
|
|
trie<span class="token punctuation">[</span><span class="token string">"moon"</span><span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token string">"sun"</span>
|
|
doAssert <span class="token string">"moon"</span> <span class="token operator">in</span> trie
|
|
doAssert trie<span class="token punctuation">[</span><span class="token string">"moon"</span><span class="token punctuation">]</span> <span class="token operator">==</span> <span class="token string">"sun"</span><span class="token operator">.</span>toBytes
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br><span class="line-number">10</span><br><span class="line-number">11</span><br><span class="line-number">12</span><br><span class="line-number">13</span><br><span class="line-number">14</span><br><span class="line-number">15</span><br><span class="line-number">16</span><br><span class="line-number">17</span><br><span class="line-number">18</span><br></div></div><p>Remember, <code>set</code> and <code>get</code> are trie operations. A single <code>set</code> operation may invoke
|
|
more than one store/lookup operation into the underlying DB. The same is also happened to <code>get</code> operation,
|
|
it could do more than one flat-db lookup before it return the requested value.</p> <h3 id="the-truth-behind-a-lie"><a href="#the-truth-behind-a-lie" class="header-anchor">#</a> The truth behind a lie</h3> <p>What kind of lie? actually, <code>delete</code> and <code>deleteSubtrie</code> doesn't remove the
|
|
'deleted' node from the underlying DB. It only make the node inaccessible
|
|
from the user of the trie. The same also happened if you update the value of a key,
|
|
the old value node is not removed from the underlying DB.
|
|
A more subtle lie also happened when you add new entrie into the trie using <code>set</code> operation.
|
|
The previous hash of affected branch become obsolete and replaced by new hash,
|
|
the old hash become inaccessible to the user.
|
|
You may think that is a waste of storage space.
|
|
Luckily, we also provide some utilities to deal with this situation, the branch utils.</p> <h3 id="the-branch-utils"><a href="#the-branch-utils" class="header-anchor">#</a> The branch utils</h3> <p>The branch utils consist of these API:</p> <ul><li>checkIfBranchExist(DB; rootHash; keyPrefix): bool</li> <li>getBranch(DB; rootHash; key): branch</li> <li>isValidBranch(branch, rootHash, key, value): bool</li> <li>getWitness(DB; nodeHash; key): branch</li> <li>getTrieNodes(DB; nodeHash): branch</li></ul> <p><code>keyPrefix</code>, <code>key</code>, and <code>value</code> are bytes container with length greater than zero.
|
|
They can be openArray[byte].</p> <p><code>rootHash</code> and <code>nodeHash</code> also bytes container,
|
|
but they have constraint: must be 32 bytes in length, and it must be a keccak_256 hash value.</p> <p><code>branch</code> is a list of nodes, or in this case a <code>seq[seq[byte]]</code>.
|
|
A list? yes, the structure is stored along with the encoded node.
|
|
Therefore a list is enough to reconstruct the entire trie/branch.</p> <div class="language-Nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">import</span>
|
|
eth<span class="token operator">/</span>trie<span class="token operator">/</span><span class="token punctuation">[</span>db<span class="token punctuation">,</span> binary<span class="token punctuation">,</span> utils<span class="token punctuation">]</span>
|
|
|
|
<span class="token keyword">var</span> db <span class="token operator">=</span> <span class="token function">newMemoryDB</span><span class="token punctuation">(</span><span class="token punctuation">)</span>
|
|
<span class="token keyword">var</span> trie <span class="token operator">=</span> <span class="token function">initBinaryTrie</span><span class="token punctuation">(</span>db<span class="token punctuation">)</span>
|
|
trie<span class="token operator">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">"key1"</span><span class="token punctuation">,</span> <span class="token string">"value1"</span><span class="token punctuation">)</span>
|
|
trie<span class="token operator">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">"key2"</span><span class="token punctuation">,</span> <span class="token string">"value2"</span><span class="token punctuation">)</span>
|
|
|
|
doAssert <span class="token function">checkIfBranchExist</span><span class="token punctuation">(</span>db<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"key"</span><span class="token punctuation">)</span> <span class="token operator">==</span> true
|
|
doAssert <span class="token function">checkIfBranchExist</span><span class="token punctuation">(</span>db<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"key1"</span><span class="token punctuation">)</span> <span class="token operator">==</span> true
|
|
doAssert <span class="token function">checkIfBranchExist</span><span class="token punctuation">(</span>db<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"ken"</span><span class="token punctuation">)</span> <span class="token operator">==</span> false
|
|
doAssert <span class="token function">checkIfBranchExist</span><span class="token punctuation">(</span>db<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"key123"</span><span class="token punctuation">)</span> <span class="token operator">==</span> false
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br><span class="line-number">10</span><br><span class="line-number">11</span><br><span class="line-number">12</span><br></div></div><p>The tree will looks like:</p> <div class="language-text line-numbers-mode"><pre class="language-text"><code> root ---> A(kvnode, *common key prefix*)
|
|
|
|
|
|
|
|
|
|
|
B(branchnode)
|
|
/ \
|
|
/ \
|
|
/ \
|
|
C1(kvnode, *remain kepath*) C2(kvnode, *remain kepath*)
|
|
| |
|
|
| |
|
|
| |
|
|
D1(leafnode, b'value1') D2(leafnode, b'value2')
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br><span class="line-number">10</span><br><span class="line-number">11</span><br><span class="line-number">12</span><br><span class="line-number">13</span><br></div></div><div class="language-Nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">var</span> branchA <span class="token operator">=</span> <span class="token function">getBranch</span><span class="token punctuation">(</span>db<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"key1"</span><span class="token punctuation">)</span>
|
|
<span class="token comment">## ==> [A, B, C1, D1]</span>
|
|
|
|
<span class="token keyword">var</span> branchB <span class="token operator">=</span> <span class="token function">getBranch</span><span class="token punctuation">(</span>db<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"key2"</span><span class="token punctuation">)</span>
|
|
<span class="token comment">## ==> [A, B, C2, D2]</span>
|
|
|
|
doAssert <span class="token function">isValidBranch</span><span class="token punctuation">(</span>branchA<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"key1"</span><span class="token punctuation">,</span> <span class="token string">"value1"</span><span class="token punctuation">)</span> <span class="token operator">==</span> true
|
|
<span class="token comment">## wrong key, return zero bytes</span>
|
|
doAssert <span class="token function">isValidBranch</span><span class="token punctuation">(</span>branchA<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"key5"</span><span class="token punctuation">,</span> <span class="token string">""</span><span class="token punctuation">)</span> <span class="token operator">==</span> true
|
|
|
|
doAssert <span class="token function">isValidBranch</span><span class="token punctuation">(</span>branchB<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"key1"</span><span class="token punctuation">,</span> <span class="token string">"value1"</span><span class="token punctuation">)</span> <span class="token comment"># InvalidNode</span>
|
|
|
|
<span class="token keyword">var</span> x <span class="token operator">=</span> <span class="token function">getBranch</span><span class="token punctuation">(</span>db<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"key"</span><span class="token punctuation">)</span>
|
|
<span class="token comment">## ==> [A]</span>
|
|
|
|
x <span class="token operator">=</span> <span class="token function">getBranch</span><span class="token punctuation">(</span>db<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"key123"</span><span class="token punctuation">)</span> <span class="token comment"># InvalidKeyError</span>
|
|
x <span class="token operator">=</span> <span class="token function">getBranch</span><span class="token punctuation">(</span>db<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"key5"</span><span class="token punctuation">)</span> <span class="token comment"># there is still branch for non-exist key</span>
|
|
<span class="token comment">## ==> [A]</span>
|
|
|
|
<span class="token keyword">var</span> branch <span class="token operator">=</span> <span class="token function">getWitness</span><span class="token punctuation">(</span>db<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"key1"</span><span class="token punctuation">)</span>
|
|
<span class="token comment">## equivalent to `getBranch(db, trie.getRootHash(), "key1")`</span>
|
|
<span class="token comment">## ==> [A, B, C1, D1]</span>
|
|
|
|
branch <span class="token operator">=</span> <span class="token function">getWitness</span><span class="token punctuation">(</span>db<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"key"</span><span class="token punctuation">)</span>
|
|
<span class="token comment">## this will include additional nodes of "key2"</span>
|
|
<span class="token comment">## ==> [A, B, C1, D1, C2, D2]</span>
|
|
|
|
<span class="token keyword">var</span> wholeTrie <span class="token operator">=</span> <span class="token function">getWitness</span><span class="token punctuation">(</span>db<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">""</span><span class="token punctuation">)</span>
|
|
<span class="token comment">## this will return the whole trie</span>
|
|
<span class="token comment">## ==> [A, B, C1, D1, C2, D2]</span>
|
|
|
|
<span class="token keyword">var</span> node <span class="token operator">=</span> branch<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span> <span class="token comment"># B</span>
|
|
<span class="token keyword">let</span> nodeHash <span class="token operator">=</span> keccak256<span class="token operator">.</span><span class="token function">digest</span><span class="token punctuation">(</span>node<span class="token operator">.</span>baseAddr<span class="token punctuation">,</span> <span class="token function">uint</span><span class="token punctuation">(</span>node<span class="token operator">.</span>len<span class="token punctuation">)</span><span class="token punctuation">)</span>
|
|
<span class="token keyword">var</span> nodes <span class="token operator">=</span> <span class="token function">getTrieNodes</span><span class="token punctuation">(</span>db<span class="token punctuation">,</span> nodeHash<span class="token punctuation">)</span>
|
|
doAssert nodes<span class="token operator">.</span>len <span class="token operator">==</span> wholeTrie<span class="token operator">.</span>len <span class="token operator">-</span> <span class="token number">1</span>
|
|
<span class="token comment">## ==> [B, C1, D1, C2, D2]</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br><span class="line-number">10</span><br><span class="line-number">11</span><br><span class="line-number">12</span><br><span class="line-number">13</span><br><span class="line-number">14</span><br><span class="line-number">15</span><br><span class="line-number">16</span><br><span class="line-number">17</span><br><span class="line-number">18</span><br><span class="line-number">19</span><br><span class="line-number">20</span><br><span class="line-number">21</span><br><span class="line-number">22</span><br><span class="line-number">23</span><br><span class="line-number">24</span><br><span class="line-number">25</span><br><span class="line-number">26</span><br><span class="line-number">27</span><br><span class="line-number">28</span><br><span class="line-number">29</span><br><span class="line-number">30</span><br><span class="line-number">31</span><br><span class="line-number">32</span><br><span class="line-number">33</span><br><span class="line-number">34</span><br><span class="line-number">35</span><br><span class="line-number">36</span><br></div></div><h3 id="remember-the-lie"><a href="#remember-the-lie" class="header-anchor">#</a> Remember the lie?</h3> <p>Because trie <code>delete</code>, <code>deleteSubtrie</code> and <code>set</code> operation create inaccessible nodes in the underlying DB,
|
|
we need to remove them if necessary. We already see that <code>wholeTrie = getWitness(db, trie.getRootHash(), "")</code>
|
|
will return the whole trie, a list of accessible nodes.
|
|
Then we can write the clean tree into a new DB instance to replace the old one.</p> <h3 id="sparse-merkle-trie"><a href="#sparse-merkle-trie" class="header-anchor">#</a> Sparse Merkle Trie</h3> <p>Sparse Merkle Trie(SMT) is a variant of Binary Trie which uses binary encoding to
|
|
represent path during trie travelsal. When Binary Trie uses three types of node,
|
|
SMT only use one type of node without any additional special encoding to store it's key-path.</p> <p>Actually, it doesn't even store it's key-path anywhere like Binary Trie,
|
|
the key-path is stored implicitly in the trie structure during key-value insertion.</p> <p>Because the key-path is not encoded in any special ways, the bits can be extracted directly from
|
|
the key without any conversion.</p> <p>However, the key restricted to a fixed length because the algorithm demand a fixed height trie
|
|
to works properly. In this case, the trie height is limited to 160 level,
|
|
or the key is of fixed length 20 bytes (8 bits x 20 = 160).</p> <p>To be able to use variable length key, the algorithm can be adapted slightly using hashed key before
|
|
constructing the binary key-path. For example, if using keccak256 as the hashing function,
|
|
then the height of the tree will be 256, but the key itself can be any length.</p> <h4 id="the-api-2"><a href="#the-api-2" class="header-anchor">#</a> The API</h4> <p>The primary API for Binary-trie is <code>set</code> and <code>get</code>.</p> <ul><li>set(key, value, rootHash[optional]) --- <em>store a value associated with a key</em></li> <li>get(key, rootHash[optional]): value --- <em>get a value using a key</em></li></ul> <p>Both <code>key</code> and <code>value</code> are of <code>BytesRange</code> type. And they cannot have zero length.
|
|
You can also use convenience API <code>get</code> and <code>set</code> which accepts
|
|
<code>Bytes</code> or <code>string</code> (a <code>string</code> is conceptually wrong in this context
|
|
and may costlier than a <code>BytesRange</code>, but it is good for testing purpose).</p> <p>rootHash is an optional parameter. When used, <code>get</code> will get a key from specific root,
|
|
and <code>set</code> will also set a key at specific root.</p> <p>Getting a non-existent key will return zero length BytesRange or a zeroBytesRange.</p> <p>Sparse Merkle Trie also provide dictionary syntax API for <code>set</code> and <code>get</code>.</p> <ul><li>trie[key] = value -- same as <code>set</code></li> <li>value = trie[key] -- same as <code>get</code></li> <li>contains(key) a.k.a. <code>in</code> operator</li></ul> <p>Additional APIs are:</p> <ul><li>exists(key) -- returns <code>bool</code>, to check key-value existence -- same as contains</li> <li>delete(key) -- remove a key-value from the trie</li> <li>getRootHash(): <code>KeccakHash</code> with <code>BytesRange</code> type</li> <li>getDB(): <code>DB</code> -- get flat-db pointer</li> <li>prove(key, rootHash[optional]): proof -- useful for merkling</li></ul> <p>Constructor API:</p> <ul><li>initSparseBinaryTrie(DB, rootHash[optional])</li> <li>init(SparseBinaryTrie, DB, rootHash[optional])</li></ul> <p>Normally you would not set the rootHash when constructing an empty Sparse Merkle Trie.
|
|
Setting the rootHash occured in a scenario where you have a populated DB
|
|
with existing trie structure and you know the rootHash,
|
|
and then you want to continue/resume the trie operations.</p> <h3 id="examples-2"><a href="#examples-2" class="header-anchor">#</a> Examples</h3> <div class="language-Nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">import</span>
|
|
eth<span class="token operator">/</span>trie<span class="token operator">/</span><span class="token punctuation">[</span>db<span class="token punctuation">,</span> sparse_binary<span class="token punctuation">,</span> utils<span class="token punctuation">]</span>
|
|
|
|
<span class="token keyword">var</span>
|
|
db <span class="token operator">=</span> <span class="token function">newMemoryDB</span><span class="token punctuation">(</span><span class="token punctuation">)</span>
|
|
trie <span class="token operator">=</span> <span class="token function">initSparseMerkleTrie</span><span class="token punctuation">(</span>db<span class="token punctuation">)</span>
|
|
|
|
<span class="token keyword">let</span>
|
|
key1 <span class="token operator">=</span> <span class="token string">"01234567890123456789"</span>
|
|
key2 <span class="token operator">=</span> <span class="token string">"abcdefghijklmnopqrst"</span>
|
|
|
|
trie<span class="token operator">.</span><span class="token function">set</span><span class="token punctuation">(</span>key1<span class="token punctuation">,</span> <span class="token string">"value1"</span><span class="token punctuation">)</span>
|
|
trie<span class="token operator">.</span><span class="token function">set</span><span class="token punctuation">(</span>key2<span class="token punctuation">,</span> <span class="token string">"value2"</span><span class="token punctuation">)</span>
|
|
doAssert trie<span class="token operator">.</span><span class="token function">get</span><span class="token punctuation">(</span>key1<span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">"value1"</span><span class="token operator">.</span>toBytes
|
|
doAssert trie<span class="token operator">.</span><span class="token function">get</span><span class="token punctuation">(</span>key2<span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">"value2"</span><span class="token operator">.</span>toBytes
|
|
|
|
trie<span class="token operator">.</span><span class="token function">delete</span><span class="token punctuation">(</span>key1<span class="token punctuation">)</span>
|
|
doAssert trie<span class="token operator">.</span><span class="token function">get</span><span class="token punctuation">(</span>key1<span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token punctuation">[</span><span class="token punctuation">]</span>
|
|
|
|
trie<span class="token operator">.</span><span class="token function">delete</span><span class="token punctuation">(</span>key2<span class="token punctuation">)</span>
|
|
doAssert trie<span class="token punctuation">[</span>key2<span class="token punctuation">]</span> <span class="token operator">==</span> <span class="token punctuation">[</span><span class="token punctuation">]</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br><span class="line-number">10</span><br><span class="line-number">11</span><br><span class="line-number">12</span><br><span class="line-number">13</span><br><span class="line-number">14</span><br><span class="line-number">15</span><br><span class="line-number">16</span><br><span class="line-number">17</span><br><span class="line-number">18</span><br><span class="line-number">19</span><br><span class="line-number">20</span><br><span class="line-number">21</span><br></div></div><p>Remember, <code>set</code> and <code>get</code> are trie operations. A single <code>set</code> operation may invoke
|
|
more than one store/lookup operation into the underlying DB. The same is also happened to <code>get</code> operation,
|
|
it could do more than one flat-db lookup before it return the requested value.
|
|
While Binary Trie perform a variable numbers of lookup and store operations, Sparse Merkle Trie
|
|
will do constant numbers of lookup and store operations each <code>get</code> and <code>set</code> operation.</p> <h3 id="merkle-proofing"><a href="#merkle-proofing" class="header-anchor">#</a> Merkle Proofing</h3> <p>Using <code>prove</code> dan <code>verifyProof</code> API, we can do some merkling with SMT.</p> <div class="language-Nim line-numbers-mode"><pre class="language-nim"><code> <span class="token keyword">let</span>
|
|
value1 <span class="token operator">=</span> <span class="token string">"hello world"</span>
|
|
badValue <span class="token operator">=</span> <span class="token string">"bad value"</span>
|
|
|
|
trie<span class="token punctuation">[</span>key1<span class="token punctuation">]</span> <span class="token operator">=</span> value1
|
|
<span class="token keyword">var</span> proof <span class="token operator">=</span> trie<span class="token operator">.</span><span class="token function">prove</span><span class="token punctuation">(</span>key1<span class="token punctuation">)</span>
|
|
|
|
doAssert <span class="token function">verifyProof</span><span class="token punctuation">(</span>proof<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> key1<span class="token punctuation">,</span> value1<span class="token punctuation">)</span> <span class="token operator">==</span> true
|
|
doAssert <span class="token function">verifyProof</span><span class="token punctuation">(</span>proof<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> key1<span class="token punctuation">,</span> badValue<span class="token punctuation">)</span> <span class="token operator">==</span> false
|
|
doAssert <span class="token function">verifyProof</span><span class="token punctuation">(</span>proof<span class="token punctuation">,</span> trie<span class="token operator">.</span><span class="token function">getRootHash</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> key2<span class="token punctuation">,</span> value1<span class="token punctuation">)</span> <span class="token operator">==</span> false
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br><span class="line-number">10</span><br></div></div><h2 id="bloom-an-ethereum-bloom-filter"><a href="#bloom-an-ethereum-bloom-filter" class="header-anchor">#</a> bloom: an Ethereum Bloom Filter</h2> <h2 id="introduction-4"><a href="#introduction-4" class="header-anchor">#</a> Introduction</h2> <p>A Nim implementation of the bloom filter used by Ethereum.</p> <h2 id="description"><a href="#description" class="header-anchor">#</a> Description</h2> <p><a href="https://en.wikipedia.org/wiki/Bloom_filter" target="_blank" rel="noopener noreferrer">Bloom filters<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> are data structures that use hash functions to test whether an element is a member of a set. They work like other data structures but are probabilistic in nature: that is, they allow false positive matches but not false negative. Bloom filters use less storage space than other data structures.</p> <p>Ethereum bloom filters are implemented with the Keccak-256 cryptographic hash function.</p> <p>To see the bloom filter used in the context of Ethereum, please refer to the <a href="https://ethereum.github.io/yellowpaper/paper.pdf" target="_blank" rel="noopener noreferrer">Ethereum Yellow Paper<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>.</p> <h2 id="usage"><a href="#usage" class="header-anchor">#</a> Usage</h2> <div class="language-nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">import</span> eth<span class="token operator">/</span>bloom<span class="token punctuation">,</span> stint
|
|
<span class="token keyword">var</span> f<span class="token operator">:</span> BloomFilter
|
|
f<span class="token operator">.</span><span class="token function">incl</span><span class="token punctuation">(</span><span class="token string">"test1"</span><span class="token punctuation">)</span>
|
|
<span class="token function">doAssert</span><span class="token punctuation">(</span><span class="token string">"test1"</span> <span class="token operator">in</span> f<span class="token punctuation">)</span>
|
|
<span class="token function">doAssert</span><span class="token punctuation">(</span><span class="token string">"test2"</span> <span class="token operator">notin</span> f<span class="token punctuation">)</span>
|
|
f<span class="token operator">.</span><span class="token function">incl</span><span class="token punctuation">(</span><span class="token string">"test2"</span><span class="token punctuation">)</span>
|
|
<span class="token function">doAssert</span><span class="token punctuation">(</span><span class="token string">"test2"</span> <span class="token operator">in</span> f<span class="token punctuation">)</span>
|
|
<span class="token function">doAssert</span><span class="token punctuation">(</span>f<span class="token operator">.</span>value<span class="token operator">.</span>toHex <span class="token operator">==</span> <span class="token string">"80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000200000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000040000000000000000000000000000000000000000000000000000000000000000000"</span><span class="token punctuation">)</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br></div></div><h2 id="node-discovery-protocol-v5"><a href="#node-discovery-protocol-v5" class="header-anchor">#</a> Node Discovery Protocol v5</h2> <h3 id="introduction-5"><a href="#introduction-5" class="header-anchor">#</a> Introduction</h3> <p>The <code>eth/p2p/discoveryv5</code> directory holds a Nim implementation of the
|
|
<a href="https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md" target="_blank" rel="noopener noreferrer">Node Discovery Protocol v5.1<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>.</p> <p>The implemented specification is Protocol version v5.1.</p> <p>This implementation does not support "Topic Advertisement" yet as this part of
|
|
the specification is not complete.</p> <p>The implementation relies on other modules in the <code>eth</code> package, namely: <code>keys</code>,
|
|
<code>rlp</code> and <code>async_utils</code>.</p> <h3 id="how-to-use"><a href="#how-to-use" class="header-anchor">#</a> How to use</h3> <div class="language-Nim line-numbers-mode"><pre class="language-nim"><code><span class="token keyword">let</span>
|
|
rng <span class="token operator">=</span> keys<span class="token operator">.</span>newRng
|
|
privKey <span class="token operator">=</span> PrivateKey<span class="token operator">.</span><span class="token function">random</span><span class="token punctuation">(</span>rng<span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
|
|
<span class="token punctuation">(</span>ip<span class="token punctuation">,</span> tcpPort<span class="token punctuation">,</span> udpPort<span class="token punctuation">)</span> <span class="token operator">=</span> <span class="token function">setupNat</span><span class="token punctuation">(</span>config<span class="token punctuation">)</span> <span class="token comment"># Or fill in external IP/ports manually</span>
|
|
d <span class="token operator">=</span> <span class="token function">newProtocol</span><span class="token punctuation">(</span>privKey<span class="token punctuation">,</span> ip<span class="token punctuation">,</span> tcpPort<span class="token punctuation">,</span> udpPort<span class="token punctuation">,</span> rng <span class="token operator">=</span> rng<span class="token punctuation">)</span>
|
|
|
|
d<span class="token operator">.</span><span class="token function">open</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token comment"># Start listening</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br></div></div><p>This will initialize the <code>Protocol</code> and start listening. However, as no
|
|
bootstrap nodes were passed in the <code>newProtocol</code> call, the created ENR will need
|
|
to be advertised somehow ("out of band"), so that the node can become known to
|
|
other nodes in the network.</p> <p>To initialize with a bootnode or a set of bootnodes, the ENRs need to be passed
|
|
as parameter in <code>newProtocol</code>.</p> <div class="language-Nim line-numbers-mode"><pre class="language-nim"><code>d <span class="token operator">=</span> <span class="token function">newProtocol</span><span class="token punctuation">(</span>privKey<span class="token punctuation">,</span> ip<span class="token punctuation">,</span> tcpPort<span class="token punctuation">,</span> udpPort<span class="token punctuation">,</span>
|
|
bootstrapRecords <span class="token operator">=</span> bootnodes<span class="token punctuation">)</span>
|
|
d<span class="token operator">.</span><span class="token function">open</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token comment"># Start listening and add bootstrap nodes to the routing table.</span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br></div></div><p>Next there are two ways to run the protocol.</p> <p>One can call <code>d.start()</code> and two loops will be started:</p> <ol><li>Refresh loop</li> <li>Revalidation loop</li></ol> <p>The first loop will at specific interval do a query with a random <code>NodeId</code> if no
|
|
manual queries were done for more than that interval period.
|
|
This query will add discovered nodes to the routing table.
|
|
The second loop will at random ranged interval send a ping to the least recently
|
|
seen node in a random bucket. This is to keep the routing table cleared of
|
|
unreachable/dead nodes.</p> <p>Now within the application, manual queries or lookups can be done, for which
|
|
the discovered nodes can be used. Nodes discovered during this process will be
|
|
attempted to be added to the routing table. One can use the <code>query</code>, <code>queryRandom</code>
|
|
or <code>lookup</code> calls for this. <code>randomNodes</code> can also be used to find nodes,
|
|
but this will only look into the current routing table and not actively
|
|
search for nodes on the network.</p> <p>Or, one can decide not to run <code>d.start()</code> and do this manually within its
|
|
application by using the available calls:</p> <ul><li><code>query</code>, <code>queryRandom</code> or <code>lookup</code> for discovering more peers.</li> <li><code>revalidateNode</code> or directly <code>ping</code> for revalidating nodes.</li></ul> <p>Of course, in either scenario, lookups can still be done for actually finding a
|
|
specific node. There is a <code>resolve</code> call that can help with this, it will first
|
|
look in the local routing table and if it finds the node it will try to contact
|
|
the node directly to check if the ENR is up to date. If any of this fail a
|
|
<code>lookup</code> will be done.</p> <h3 id="test-suite"><a href="#test-suite" class="header-anchor">#</a> Test suite</h3> <p>To run the test suite specifically for discovery v5 related (discovery v5 + its
|
|
nim-eth dependencies) tests, one can run following command:</p> <div class="language-sh line-numbers-mode"><pre class="language-sh"><code><span class="token comment">## Install required modules</span>
|
|
nimble <span class="token function">install</span>
|
|
<span class="token comment">## Run only discovery v5 related test suite</span>
|
|
nimble tests_discv5_full
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br></div></div><h3 id="dcli"><a href="#dcli" class="header-anchor">#</a> dcli</h3> <p>This is a small command line application that allows you to run a discovery
|
|
node. It also has the options to do a <code>ping</code> or <code>findNode</code> request to a specific
|
|
node, by providing its ENR.</p> <h4 id="example-usage"><a href="#example-usage" class="header-anchor">#</a> Example usage</h4> <div class="language-sh line-numbers-mode"><pre class="language-sh"><code><span class="token comment">## Install required modules</span>
|
|
<span class="token comment">## Make sure you have the latest modules, do NOT trust nimble on this.</span>
|
|
nimble <span class="token function">install</span>
|
|
<span class="token comment">## Build dcli</span>
|
|
nim c -d:chronicles_log_level:trace -d:release --threads:on eth/p2p/discoveryv5/dcli
|
|
<span class="token comment">## See all options</span>
|
|
./eth/p2p/discoveryv5/dcli --help
|
|
<span class="token comment">## Ping another node</span>
|
|
./eth/p2p/discoveryv5/dcli <span class="token function">ping</span> enr:<span class="token operator"><</span>base64 encoding of ENR<span class="token operator">></span>
|
|
<span class="token comment">## Run discovery node</span>
|
|
./eth/p2p/discoveryv5/dcli --log-level:debug --bootnode:enr:<span class="token operator"><</span>base64 encoding of ENR<span class="token operator">></span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br><span class="line-number">10</span><br><span class="line-number">11</span><br></div></div><h4 id="metrics"><a href="#metrics" class="header-anchor">#</a> Metrics</h4> <p>To run dcli with metrics enabled provide the <code>metrics</code> flag:</p> <div class="language-sh line-numbers-mode"><pre class="language-sh"><code><span class="token comment">## Run dcli with metrics</span>
|
|
./eth/p2p/discoveryv5/dcli --metrics --bootnode:enr:<span class="token operator"><</span>base64 encoding of ENR<span class="token operator">></span>
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br></div></div><p>You can now see the metrics at http://localhost:8008/metrics. Or use e.g.
|
|
Prometheus to grab the data.</p> <h2 id="prerequisites"><a href="#prerequisites" class="header-anchor">#</a> Prerequisites</h2> <ul><li>Nim & Nimble</li> <li>RocksDB, SQLite, LMDB (required for the trie backend tests)</li></ul> <p>E.g. on Ubuntu one can run:</p> <div class="language- line-numbers-mode"><pre class="language-text"><code>apt install -y librocksdb-dev liblmdb-dev sqlite3
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br></div></div><h2 id="building-testing"><a href="#building-testing" class="header-anchor">#</a> Building & Testing</h2> <div class="language- line-numbers-mode"><pre class="language-text"><code># Install required modules
|
|
nimble install
|
|
# Run full test suite
|
|
nimble test
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br></div></div><p>You can also run specific parts of the test suite, e.g.:</p> <div class="language- line-numbers-mode"><pre class="language-text"><code># Test p2p functionality
|
|
nimble test_p2p
|
|
# Test rlp functionality
|
|
nimble test_rlp
|
|
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br></div></div><h2 id="fuzzing"><a href="#fuzzing" class="header-anchor">#</a> Fuzzing</h2> <p>Next to the test suite, there are also several fuzzing test cases available.
|
|
How these can be run is explained in the <a href="https://github.com/status-im/nim-eth/blob/master/tests/fuzzing/readme.md" target="_blank" rel="noopener noreferrer">fuzzing readme<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>.</p></div> <footer class="page-edit"><!----> <!----></footer> <!----> </main></div><div class="global-ui"></div></div>
|
|
<script src="/assets/js/app.b6b895fe.js" defer></script><script src="/assets/js/2.fd7e9e0b.js" defer></script><script src="/assets/js/11.b620ddcc.js" defer></script>
|
|
</body>
|
|
</html>
|