Fluffy state bridge docs (#2632)

* Don't proxy implemented state JSON-RPC endpoints. Fix minor issue in testnet script.

* Add docs covering usage of the Fluffy state bridge.

fluffy/docs/the_fluffy_book/docs/adding-documentation.md

@@ -9,6 +9,7 @@ All the documentation related files can be found under the `./fluffy/docs/the_fl
 ## How to test and add documentation changes
 
 - Install `mkdocs`
+- Install Material for MkDocs by running `pip install mkdocs-material`.
 - Make your changes to the documentation
 - Run `mkdocs serve` from the `./fluffy/docs/the_fluffy_book` directory and test your changes. Alter as required.
 - Push your changes to a PR on nimbus-eth1

fluffy/docs/the_fluffy_book/docs/history-content-bridging.md

@@ -1,19 +1,19 @@
-# Bridging content: Portal history network
+# Bridging content into the Portal history network
 
-## From content bridges
+## Seeding from content bridges
 
 ### Seeding history data with the `portal_bridge`
 
 #### Step 1: Run a Portal client
 
-Run a Portal client with the Portal JSON-RPC API enabled, e.g. fluffy:
+Run a Portal client with the Portal JSON-RPC API enabled, e.g. Fluffy:
 
 ```bash
 ./build/fluffy --rpc --storage-capacity:0
 ```
 
 > Note: The `--storage-capacity:0` option is not required, but it is added here
-for the use case where the node its only focus is on gossiping content from the
+for the use case where the node's only focus is on gossiping content from the
 `portal_bridge`.
 
 #### Step 2: Run an EL client

fluffy/docs/the_fluffy_book/docs/state-content-bridging.md

@@ -0,0 +1,71 @@
+# Bridging content into the Portal state network
+
+## Seeding from content bridges
+
+### Seeding state data with the `portal_bridge`
+
+
+#### Step 1: Run a Portal client
+
+Run a Portal client with the Portal JSON-RPC API enabled (e.g. Fluffy) and enable the `state` subnetwork:
+
+```bash
+./build/fluffy --rpc --storage-capacity:0 --portal-subnetworks:state
+```
+
+> Note: The `--storage-capacity:0` option is not required, but it is added here
+for the use case where the node's only focus is on gossiping content from the
+`portal_bridge`.
+
+
+#### Step 2: Run an EL client (archive node) that supports `trace_replayBlockTransactions`
+
+The `portal_bridge` needs access to the EL JSON-RPC API, either through a local
+Ethereum client or via a web3 provider.
+
+Currently the portal state bridge requires access to the following EL JSON-RPC APIs:
+
+- `eth_getBlockByNumber`
+- `eth_getUncleByBlockNumberAndIndex`
+- `trace_replayBlockTransactions`
+
+`trace_replayBlockTransactions` is a non-standard endpoint that is only implemented
+by some EL clients (e.g. Erigon, Reth and Besu at the time of writing). The bridge uses the
+`stateDiff` trace option parameter to collect the state updates for each block of
+transactions as it syncs and builds the state from genesis onwards. Since access to the
+state is required in order to build these state diffs, an EL archive node is required
+to ensure that the state is available for all the historical blocks being synced.
+
+
+#### Step 3: Run the Portal bridge in state mode
+
+Build & run the `portal_bridge`:
+```bash
+make portal_bridge
+
+WEB3_URL="ws://127.0.0.1:8548" # Replace with your provider.
+./build/portal_bridge state --web3-url:${WEB3_URL} --start-block=1 --gossip-workers=2
+```
+
+> Note: A WebSocket connection to the web3 provider is recommended to improve the
+speed of the syncing process.
+
+The `--start-block` parameter can be used to start syncing from a specific block number.
+The bridge stores a local copy of the collected state diffs in it's database so that
+the bridge can be restarted and continue syncing from a specific block number. The
+first time the bridge is started it will/must start syncing from the genesis block
+so you should initially set the `--start-block` option to 1.
+
+The `--gossip-workers` parameter can be used to set the number of workers that will
+gossip the portal state data into the portal state subnetwork. Each worker handles
+gossipping the state for a single block and the workers gossip the data concurrently.
+It is recommended to increase the number of workers in order to increase the speed
+and throughput of the gossiping process up until the point where Fluffy is unable
+keep up.
+
+The optional `--verify-gossip` parameter can be used to verify that the state data has
+successfully been gossipped and is available on the portal state network. When this
+option is enabled the bridge will lookup every bit of state data and verify that it
+is available. If any state data is missing, the bridge will retry gossipping the
+data for that block. Using this option will slow down the process but will ensure that
+the state data is available for every block.

fluffy/docs/the_fluffy_book/mkdocs.yml

@@ -73,6 +73,7 @@ nav:
     - Portal bridge:
       - 'history-content-bridging.md'
       - 'beacon-content-bridging.md'
+      - 'state-content-bridging.md'
     - 'db_pruning.md'
 
   - Developers:

fluffy/rpc/rpc_eth_api.nim

@@ -144,7 +144,7 @@ proc installEthApiHandlers*(
 
   rpcServerWithProxy.registerProxyMethod("eth_feeHistory")
 
-  rpcServerWithProxy.registerProxyMethod("eth_getBalance")
+  #rpcServerWithProxy.registerProxyMethod("eth_getBalance")
 
   # rpcServerWithProxy.registerProxyMethod("eth_getBlockByHash")
 
@@ -154,7 +154,7 @@ proc installEthApiHandlers*(
 
   rpcServerWithProxy.registerProxyMethod("eth_getBlockTransactionCountByNumber")
 
-  rpcServerWithProxy.registerProxyMethod("eth_getCode")
+  #rpcServerWithProxy.registerProxyMethod("eth_getCode")
 
   rpcServerWithProxy.registerProxyMethod("eth_getRawTransactionByHash")
 
@@ -162,7 +162,7 @@ proc installEthApiHandlers*(
 
   rpcServerWithProxy.registerProxyMethod("eth_getRawTransactionByBlockNumberAndIndex")
 
-  rpcServerWithProxy.registerProxyMethod("eth_getStorageAt")
+  #rpcServerWithProxy.registerProxyMethod("eth_getStorageAt")
 
   rpcServerWithProxy.registerProxyMethod("eth_getTransactionByBlockHashAndIndex")
 
@@ -170,7 +170,7 @@ proc installEthApiHandlers*(
 
   rpcServerWithProxy.registerProxyMethod("eth_getTransactionByHash")
 
-  rpcServerWithProxy.registerProxyMethod("eth_getTransactionCount")
+  #rpcServerWithProxy.registerProxyMethod("eth_getTransactionCount")
 
   rpcServerWithProxy.registerProxyMethod("eth_getTransactionReceipt")
 
@@ -182,7 +182,7 @@ proc installEthApiHandlers*(
 
   rpcServerWithProxy.registerProxyMethod("eth_getUncleCountByBlockNumber")
 
-  rpcServerWithProxy.registerProxyMethod("eth_getProof")
+  #rpcServerWithProxy.registerProxyMethod("eth_getProof")
 
   rpcServerWithProxy.registerProxyMethod("eth_sendRawTransaction")
 

fluffy/scripts/launch_local_testnet.sh

@@ -181,7 +181,7 @@ if [[ $# != 0 ]]; then
   shift $#
 fi
 
-if [[ $((NUM_NODES)) < 3 ]]; then
+if [[ $((NUM_NODES)) -lt 3 ]]; then
   echo "--nodes is less than minimum of 3. Must have at least 3 nodes in order for the network to be stable."
   exit 1
 fi