diff --git a/.github/workflows/fluffy_docs.yml b/.github/workflows/fluffy_docs.yml new file mode 100644 index 000000000..fa0af9fce --- /dev/null +++ b/.github/workflows/fluffy_docs.yml @@ -0,0 +1,32 @@ +name: fluffy docs CI + +on: + push: + branches: + - master + paths: + - '.github/workflows/fluffy_docs.yml' + - 'fluffy/docs/**' + +permissions: + contents: write +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v4 + with: + python-version: 3.x + - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV + - uses: actions/cache@v3 + with: + key: mkdocs-material-${{ env.cache_id }} + path: .cache + restore-keys: | + mkdocs-material- + - run: pip install mkdocs-material + - name: Run mkdocs github deploy + working-directory: ./fluffy/docs/the_fluffy_book/ + run: | + mkdocs gh-deploy --force \ No newline at end of file diff --git a/fluffy/README.md b/fluffy/README.md index d0b670291..e8cb04e19 100644 --- a/fluffy/README.md +++ b/fluffy/README.md @@ -121,26 +121,6 @@ Find more details on the usage and workings of the local testnet script Follow the steps outlined [here](../README.md#windows) to build fluffy on Windows. -## Development tools and documentation - -The fluffy directory also holds several tools to help development of the Portal -networks. - -Command to build the tools: - -```bash -make fluffy-tools -j6 -``` - -Additional documention on the tools or on what you can use them for: - -- [eth_data_exporter](./docs/eth_data_exporter.md): tool to extract content from -EL or CL and prepare it as Portal content and content keys. -- [Content seeding](./docs/content_seeding.md): Documentation on how to retrieve & generate history data and how to seed it into the network -- [Manual protocol interop testing](./docs/protocol_interop.md): commands on how to manually test the discv5 and Portal protocol request and responses -- [Local testnet script](./docs/local_testnet.md): Documentation on the local testnet script and how to use it - - ## The basics for developers When working on this repository, you can run the `env.sh` script to run a diff --git a/fluffy/docs/the_fluffy_book/docs/beacon-content-bridging.md b/fluffy/docs/the_fluffy_book/docs/beacon-content-bridging.md new file mode 100644 index 000000000..cc3d90529 --- /dev/null +++ b/fluffy/docs/the_fluffy_book/docs/beacon-content-bridging.md @@ -0,0 +1,25 @@ +# Bridging content into the Portal beacon network + +## Seeding from content bridges + +Run a Fluffy node with the JSON-RPC API enabled. + +```bash +./build/fluffy --rpc +``` + +Build & run the `beacon_chain_bridge`: +```bash +make beacon_chain_bridge + +TRUSTED_BLOCK_ROOT=0x1234567890123456789012345678901234567890123456789012345678901234 # Replace with trusted block root. +# --rest-url = access to beacon node API, default http://127.0.0.1:5052 +./build/beacon_chain_bridge --trusted-block-root:${TRUSTED_BLOCK_ROOT} --rest-url:http://127.0.0.1:5052 +``` + +The `beacon_chain_bridge` will connect to Fluffy node over the JSON-RPC +interface and start gossiping an `LightClientBootstrap` for +given trusted block root and gossip backfill `LightClientUpdate`s. + +Next, it will gossip a new `LightClientOptimisticUpdate`, +`LightClientFinalityUpdate` and `LightClientUpdate` as they become available. diff --git a/fluffy/docs/eth_data_exporter.md b/fluffy/docs/the_fluffy_book/docs/eth-data-exporter.md similarity index 76% rename from fluffy/docs/eth_data_exporter.md rename to fluffy/docs/the_fluffy_book/docs/eth-data-exporter.md index 5dd29b890..713cb9fe0 100644 --- a/fluffy/docs/eth_data_exporter.md +++ b/fluffy/docs/the_fluffy_book/docs/eth-data-exporter.md @@ -1,4 +1,6 @@ -# eth_data_exporter +# Exporting Ethereum content for Portal + +## eth_data_exporter The `eth_data_exporter` is a tool to extract content from Ethereum EL or CL and prepare it as Portal content and content keys. @@ -9,12 +11,14 @@ Currently the `history` and the `beacon` networks are supported. Example commands: ```bash +# Build the tool +make eth_data_exporter # See the different commands and options ./build/eth_data_exporter --help ``` ```bash -# Request of Beacon Chain Light Client Updates and export into the Portal +# Request of `BeaconLightClientUpdate`s and export into the Portal # network supported format ./build/eth_data_exporter beacon exportLCUpdates --rest-url:http://testing.mainnet.beacon-api.nimbus.team --start-period:816 --count:4 ``` diff --git a/fluffy/docs/content_seeding.md b/fluffy/docs/the_fluffy_book/docs/history-content-bridging.md similarity index 81% rename from fluffy/docs/content_seeding.md rename to fluffy/docs/the_fluffy_book/docs/history-content-bridging.md index 0d19c6ef5..9db309526 100644 --- a/fluffy/docs/content_seeding.md +++ b/fluffy/docs/the_fluffy_book/docs/history-content-bridging.md @@ -1,8 +1,38 @@ -# Seeding data into the Portal history network +# Bridging content: Portal history network -## Seeding from locally stored history data +## From content bridges -### Building and seeding epoch accumulators into the Portal history network +### Seeding post-merge block data through the `beacon_lc_bridge` + +Run a Fluffy node with the JSON-RPC API enabled. + +```bash +./build/fluffy --rpc +``` + +Build & run the `beacon_lc_bridge`: +```bash +make beacon_lc_bridge + +TRUSTED_BLOCK_ROOT=0x1234567890123456789012345678901234567890123456789012345678901234 # Replace with trusted block root. +./build/beacon_lc_bridge --trusted-block-root=${TRUSTED_BLOCK_ROOT} +``` + +The `beacon_lc_bridge` will start with the consensus light client sync follow +beacon block gossip. Once it is synced, the execution payload of new beacon +blocks will be extracted and injected in the Portal network as execution headers +and blocks. + +> Note: The execution headers will come without a proof for now. + +The injection into the Portal network is done via the +`portal_historyGossip` JSON-RPC endpoint of the running Fluffy node. + +> Note: Backfilling of block bodies and headers is not yet supported. + +## From locally stored block data + +### Building and seeding epoch accumulators #### Step 1: Building the epoch accumulators 1. Set-up access to an Ethereum JSON-RPC endpoint (e.g. local geth instance) @@ -12,7 +42,7 @@ that can serve the data. *.e2s files arranged per epoch (8192 blocks): ```bash -make fluffy-tools +make eth_data_exporter ./build/eth_data_exporter history exportEpochHeaders --data-dir:"./user_data_dir/" ``` @@ -31,7 +61,7 @@ Run Fluffy and trigger the propagation of data with the `portal_history_propagateEpochAccumulators` JSON-RPC API call: ```bash -./build/fluffy --rpc --table-ip-limit:1024 --bucket-ip-limit:24 +./build/fluffy --rpc # From another terminal curl -s -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","id":"1","method":"portal_history_propagateEpochAccumulators","params":["./user_data_dir/"]}' http://localhost:8545 | jq @@ -44,15 +74,16 @@ accumulators are available on the history network: Make sure you still have a fluffy instance running, if not run: ```bash -./build/fluffy --rpc --table-ip-limit:1024 --bucket-ip-limit:24 +./build/fluffy --rpc ``` Run the `content_verifier` tool and see if all epoch accumulators are found: ```bash +make content_verifier ./build/content_verifier ``` -### Downloading & seeding block data into the Portal network +### Downloading & seeding block data 1. Set-up access to an Ethereum JSON-RPC endpoint (e.g. local geth instance) that can serve the data. @@ -61,7 +92,7 @@ JSON-RPC endpoint into the format which is suitable for reading data into Fluffy client and propagating into the network: ```bash -make fluffy-tools +make eth_data_exporter ./build/eth_data_exporter history exportBlockData--initial-block:1 --end-block:10 --data-dir:"/user_data_dir/" ``` @@ -73,38 +104,8 @@ This will store blocks 1 to 10 into a json file located at `portal_history_propagate` JSON-RPC API call: ```bash -./build/fluffy --table-ip-limit:1024 --bucket-ip-limit:24 --log-level:info --rpc +./build/fluffy --rpc # From another shell curl -s -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","id":"1","method":"portal_history_propagate","params":["./user_data_dir/eth-history-data.json"]}' http://localhost:8545 | jq ``` - -## Seeding from content bridges - -### Seeding post-merge block headers and bodies through the beacon chain bridge - -Run a Fluffy node with the JSON-RPC API enabled. - -```bash -./build/fluffy --rpc --table-ip-limit:1024 --bucket-ip-limit:24 -``` - -Build & run the `beacon_lc_bridge`: -```bash -make beacon_lc_bridge - -TRUSTED_BLOCK_ROOT=0x1234567890123456789012345678901234567890123456789012345678901234 # Replace this -./build/beacon_lc_bridge --trusted-block-root=${TRUSTED_BLOCK_ROOT} -``` - -The `beacon_lc_bridge` will start with the consensus light client sync follow -beacon block gossip. Once it is synced, the execution payload of new beacon -blocks will be extracted and injected in the Portal network as execution headers -and blocks. - -> Note: The execution headers will come without a proof for now. - -The injection into the Portal network is done via the -`portal_historyGossip` JSON-RPC endpoint of the running Fluffy node. - -> Note: Backfilling of block bodies and headers is not yet supported. diff --git a/fluffy/docs/the_fluffy_book/docs/index.md b/fluffy/docs/the_fluffy_book/docs/index.md new file mode 100644 index 000000000..0204c2772 --- /dev/null +++ b/fluffy/docs/the_fluffy_book/docs/index.md @@ -0,0 +1,42 @@ +# The Nimbus Fluffy Guide + +Fluffy is the Nimbus client implementation of the +[Portal network specifications](https://github.com/ethereum/portal-network-specs/blob/master/portal-network.md). + +The Portal Network aims to deliver a reliable, sync-free, and decentralized +access to the Ethereum blockchain. The network can be used by a light client to +get access to Ethereum data and as such become a drop-in replacement for full +nodes by providing that data through the existing +[Ethereum JSON RPC Execution API](https://github.com/ethereum/execution-apis). + +This book describes how to build, run and monitor the Fluffy client, and how to +use and test its currently implemented functionality. + +To quickly get your Fluffy node up and running, follow the quickstart page: + + - [Quickstart for Linux / macOS users](./quick-start.md) + - [Quickstart for Windows users](./quick-start-windows.md) + +# Development status +The Portal Network is a project still in research phase. +This client is thus still experimental. + +However, the Portal history and Portal beacon sub-networks are already operational and can be tested on the public testnet or in a local testnet. + +## Get in touch + +Need help with anything? +Join us on [Status](https://join.status.im/nimbus-general) and [Discord](https://discord.gg/9dWwPnG). + +## Donate + +If you'd like to contribute to Nimbus development: + +* Our donation address is [`0xDeb4A0e8d9a8dB30a9f53AF2dCc9Eb27060c6557`](https://etherscan.io/address/0xDeb4A0e8d9a8dB30a9f53AF2dCc9Eb27060c6557) +* We're also listed on [GitCoin](https://gitcoin.co/grants/137/nimbus-2) + +## Disclaimer + +This documentation assumes Nimbus Fluffy is in its ideal state. +The project is still under heavy development. +Please submit a [Github issue](https://github.com/status-im/nimbus-eth1/issues) if you come across a problem. \ No newline at end of file diff --git a/fluffy/docs/protocol_interop.md b/fluffy/docs/the_fluffy_book/docs/protocol-interop-testing.md similarity index 85% rename from fluffy/docs/protocol_interop.md rename to fluffy/docs/the_fluffy_book/docs/protocol-interop-testing.md index c2411ba7c..76acca66f 100644 --- a/fluffy/docs/protocol_interop.md +++ b/fluffy/docs/the_fluffy_book/docs/protocol-interop-testing.md @@ -1,17 +1,18 @@ -# Testing Client Protocol Interoperability -This document shows some commands that can be used to test the individual -protocol messages per network (Discovery v5 and Portal networks). +# Protocol Interoperability Testing +This document shows a set of commands that can be used to test the individual +protocol messages per network (Discovery v5 and Portal networks), e.g. to test +client protocol interoperability. -Two ways are explained, the first by keeping a node running and interacting -with it through the JSON-RPC service. The second by running cli applications +Two ways are explained, the first, by keeping a node running and interacting +with it through the JSON-RPC service. The second, by running cli applications that attempt to send 1 specific message and then shutdown. -The first is more powerful and complete, the second might be easier to do some -quick testing. +The first is more powerful and complete, the second one might be easier to do +some quick testing. -## Run Fluffy and test protocol messages launched via JSON-RPC API +## Run Fluffy and test protocol messages via JSON-RPC API -First build Fluffy as explained [here](../README.md#build-fluffy-client). +First build Fluffy as explained [here](./quick-start.md#build-the-fluffy-client). Next run it with the JSON-RPC server enabled: ```bash @@ -59,7 +60,7 @@ curl -s -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","id":"1 > The `portal_state_` prefix can be replaced for testing other networks such as `portal_history_`. -## Test Discovery and Portal Wire protocol messages through cli tools +## Test Discovery and Portal Wire protocol messages with cli tools ### Testing Discovery v5 Layer: dcli @@ -90,7 +91,7 @@ and ENR. ```bash # Build portalcli -make fluffy-tools +make portalcli ``` With the `portalcli` tool you can test the individual Portal wire protocol @@ -107,7 +108,7 @@ messages, e.g.: # Test Portal wire FindContent, should print the returned content ./build/portalcli findcontent enr: -# Default the State network is tested, but you can provide another protocol id +# Default the history network is tested, but you can provide another protocol id ./build/portalcli ping enr: --protocol-id:0x500B ``` diff --git a/fluffy/docs/the_fluffy_book/docs/quick-start-docker.md b/fluffy/docs/the_fluffy_book/docs/quick-start-docker.md new file mode 100644 index 000000000..f182a0ac6 --- /dev/null +++ b/fluffy/docs/the_fluffy_book/docs/quick-start-docker.md @@ -0,0 +1,10 @@ +# Quick start - Docker + +This page takes you through the steps of getting the Fluffy Portal node running +on the public network by use of the [public Docker image](https://hub.docker.com/r/statusim/nimbus-fluffy/tags). + +The docker image gets currently rebuild from latest master every night. + +## Steps + +To be added. \ No newline at end of file diff --git a/fluffy/docs/the_fluffy_book/docs/quick-start-windows.md b/fluffy/docs/the_fluffy_book/docs/quick-start-windows.md new file mode 100644 index 000000000..3bc145bf0 --- /dev/null +++ b/fluffy/docs/the_fluffy_book/docs/quick-start-windows.md @@ -0,0 +1,73 @@ +# Quick start - Windows + +This page takes you through the steps of getting the Fluffy Portal node running +on the public network. + +The guide assumes Windows is being used. For Linux/macOS users follow this +[tutorial](./quick-start.md). + +!!! notice + Running Fluffy on Windows is more experimental and less tested! + +## Steps + +### Prerequisites +- Install & setup Mingw-w64: + - Download Mingw-w64 for your architecture using the "[MinGW-W64 Online + Installer](https://sourceforge.net/projects/mingw-w64/files/)" (first link + under the directory listing). + - Run it and select your architecture in the setup + menu ("i686" on 32-bit, "x86\_64" on 64-bit), set the threads to "win32" and + the exceptions to "dwarf" on 32-bit and "seh" on 64-bit. Change the + installation directory to "C:\mingw-w64". + - Add it to your system PATH in "My Computer"/"This PC" -> Properties -> + Advanced system settings -> Environment Variables -> Path -> Edit -> New -> C:\mingw-w64\mingw64\bin (it's "C:\mingw-w64\mingw32\bin" on 32-bit) + +- Install [cmake](https://cmake.org/). + +- Install [Git for Windows](https://gitforwindows.org/) and use a "Git Bash" +shell to clone and build Fluffy in the next steps. + +### Build the Fluffy client +```bash +git clone git@github.com:status-im/nimbus-eth1.git +cd nimbus-eth1 +mingw32-make fluffy + +# Test if binary was successfully build by running the help command. +./build/fluffy --help +``` + +### Run a Fluffy client on the public testnet + +```bash +# Connect to the Portal testnet bootstrap nodes and enable the JSON-RPC APIs +./build/fluffy --rpc +``` + +### Try requesting a execution layer block from the network + +The Portal testnet is slowly being filled up with historical data through bridge +nodes. Because of this, more recent history data is more likely to be available. +This can be tested by using the `eth_getBlockByHash` JSON-RPC from the +[execution JSON-RPC API](https://ethereum.github.io/execution-apis/api-documentation/). + +```bash +# Get the hash of a block from your favorite block explorer, e.g.: +BLOCKHASH=0x34eea44911b19f9aa8c72f69bdcbda3ed933b11a940511b6f3f58a87427231fb # Replace this to the block hash of your choice +# Run this command to get the block: +curl -s -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","id":"1","method":"eth_getBlockByHash","params":["'${BLOCKHASH}'", true]}' http://localhost:8545 | jq +``` + +### Update and rebuild the Fluffy client +In order to stay up to date you can pull the latest version from our master +branch. There are currently released versions tagged. + +```bash +# From the nimbus-eth1 repository +git pull +# To bring the git submodules up to date +mingw32-make update + +mingw32-make fluffy +``` diff --git a/fluffy/docs/the_fluffy_book/docs/quick-start.md b/fluffy/docs/the_fluffy_book/docs/quick-start.md new file mode 100644 index 000000000..d68e9d64d --- /dev/null +++ b/fluffy/docs/the_fluffy_book/docs/quick-start.md @@ -0,0 +1,56 @@ +# Quick start - Linux/macOS + +This page takes you through the steps of getting the Fluffy Portal node running +on the public network. + +The guide assumes Linux or macOS is being used. For Windows users follow this +[tutorial](./quick-start-windows.md). + +## Steps + +### Prerequisites +- GNU Make, Bash and the usual POSIX utilities. Git 2.9.4 or newer. + +### Build the Fluffy client +```bash +git clone git@github.com:status-im/nimbus-eth1.git +cd nimbus-eth1 +make fluffy + +# Test if binary was successfully build by running the help command. +./build/fluffy --help +``` + +### Run a Fluffy client on the public testnet + +```bash +# Connect to the Portal testnet bootstrap nodes and enable the JSON-RPC APIs +./build/fluffy --rpc +``` + +### Try requesting a execution layer block from the network + +The Portal testnet is slowly being filled up with historical data through bridge +nodes. Because of this, more recent history data is more likely to be available. +This can be tested by using the `eth_getBlockByHash` JSON-RPC from the +[execution JSON-RPC API](https://ethereum.github.io/execution-apis/api-documentation/). + +``` +# Get the hash of a block from your favorite block explorer, e.g.: +BLOCKHASH=0x34eea44911b19f9aa8c72f69bdcbda3ed933b11a940511b6f3f58a87427231fb # Replace this to the block hash of your choice +# Run this command to get the block: +curl -s -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","id":"1","method":"eth_getBlockByHash","params":["'${BLOCKHASH}'", true]}' http://localhost:8545 | jq +``` + +### Update and rebuild the Fluffy client +In order to stay up to date you can pull the latest version from our master +branch. There are currently released versions tagged. + +```bash +# From the nimbus-eth1 repository +git pull +# To bring the git submodules up to date +make update + +make fluffy +``` diff --git a/fluffy/docs/the_fluffy_book/docs/run-local-testnet.md b/fluffy/docs/the_fluffy_book/docs/run-local-testnet.md new file mode 100644 index 000000000..97aec1cd1 --- /dev/null +++ b/fluffy/docs/the_fluffy_book/docs/run-local-testnet.md @@ -0,0 +1,31 @@ +# Running a local testnet + +To easily start a local testnet you can use the `launch_local_testnet.sh` +script. +This script allows you to start `n` amount of nodes and then run several actions +on them through the JSON-RPC API. + +## Run the local testnet script + +```bash +# Run the script, default start 64 nodes +./fluffy/scripts/launch_local_testnet.sh +# Run the script with 16 nodes +./fluffy/scripts/launch_local_testnet.sh -n 16 + +# See the script help +./fluffy/scripts/launch_local_testnet.sh --help +``` + +The nodes will be started and all nodes will use `node0` as bootstrap node. + +The `data-dir`s and logs of each node can be found in `./local_testnet_data/`. + +You can manually start extra nodes that connect to the network by providing +any of the running nodes their ENR. + +E.g. to manually add a Fluffy node to the local testnet run: + +```bash +./build/fluffy --rpc --network:none --udp-port:9010 --nat:extip:127.0.0.1 --bootstrap-node:`cat ./local_testnet_data/node0/fluffy_node.enr` +``` diff --git a/fluffy/docs/the_fluffy_book/docs/testnet-beacon-network.md b/fluffy/docs/the_fluffy_book/docs/testnet-beacon-network.md new file mode 100644 index 000000000..3650ccc2e --- /dev/null +++ b/fluffy/docs/the_fluffy_book/docs/testnet-beacon-network.md @@ -0,0 +1,46 @@ +# Testing beacon network on local testnet + +This section explains how one can set up a local testnet together with a beacon +network bridge in order to test if all nodes can do the beacon light client sync +and stay up to date with the latest head of the chain. + +To accomodate this, the `launch_local_testnet.sh` script has the option to +launch the Fluffy `beacon_chain_bridge` automatically and connect it to `node0` +of the local tesnet. + +## Run the local testnet script with bridge + +The `launch_local_testnet.sh` script must be launched with the +`--trusted-block-root` cli option. +The individual nodes will be started with this `trusted-block-root` and each +node will try to start sync from this block root. + +Run the following command to launch the network with the `beacon_chain_bridge` +activated. + +```bash +TRUSTED_BLOCK_ROOT=0x1234567890123456789012345678901234567890123456789012345678901234 # Replace with trusted block root. + +# Run the script, start 8 nodes + beacon_chain_bridge +./fluffy/scripts/launch_local_testnet.sh -n8 --trusted-block-root ${TRUSTED_BLOCK_ROOT}--beacon-chain-bridge +``` + +## Run the local testnet script and launch the bridge manually + +To have control over when to start or restart the `beacon_chain_bridge` on can +also control the bridge manually, e.g. start the testnet: + +```bash +TRUSTED_BLOCK_ROOT=0x1234567890123456789012345678901234567890123456789012345678901234 # Replace with trusted block root. + +# Run the script, start 8 nodes +./fluffy/scripts/launch_local_testnet.sh -n8 --trusted-block-root ${TRUSTED_BLOCK_ROOT} +``` + +Next, start the `beacon_chain_bridge` + +```bash +# --rpc-port 10000 = default node0 +# --rest-url = access to beacon node API, default http://127.0.0.1:5052 +./build/beacon_chain_bridge --trusted-block-root:${TRUSTED_BLOCK_ROOT} --rest-url:http://127.0.0.1:5052 --backfill-amount:128 --rpc-port:10000 +``` diff --git a/fluffy/docs/local_testnet.md b/fluffy/docs/the_fluffy_book/docs/testnet-history-network.md similarity index 55% rename from fluffy/docs/local_testnet.md rename to fluffy/docs/the_fluffy_book/docs/testnet-history-network.md index d2d5ca34b..b4cb16d61 100644 --- a/fluffy/docs/local_testnet.md +++ b/fluffy/docs/the_fluffy_book/docs/testnet-history-network.md @@ -1,57 +1,45 @@ -# Using the local testnet script -There is a local testnet script in `./fluffy/scripts/` folder that allows you -to start `n` amount of nodes and then run several actions on them through -the JSON-RPC API. +# Testing history network on local testnet -## Run the local testnet script +There is an automated test for the Portal history network integrated in the +`launch_local_testnet.sh` script. + +The `test_portal_testnet` binary can be run from within this script and do a +set of actions on the nodes through the JSON-RPC API. When that is finished, all +nodes will be killed. + +## Run the local testnet script with history network test ```bash -# Run the script -./fluffy/scripts/launch_local_testnet.sh -# Run the script with 128 nodes -./fluffy/scripts/launch_local_testnet.sh -n 128 - -# See the script help -./fluffy/scripts/launch_local_testnet.sh --help +# Run the script, default start 64 nodes and run history tests +./fluffy/scripts/launch_local_testnet.sh --run-tests ``` -The nodes will be started and all use node 0 as bootstrap node. -Default, the `test_portal_testnet` binary will be run and do a set of actions -on the nodes through the JSON-RPC API. When that is finished, all nodes will be -killed. - -To run the nodes without testing and keep them alive for manual testing: -```bash -./fluffy/scripts/launch_local_testnet.sh --enable-htop -``` - -## Details of the test +## Details of the `test_portal_testnet` test ### Initial set-up -Following inital steps are done to set up the Discovery v5 network and the +Following initial steps are done to set up the Discovery v5 network and the Portal networks: + 1. Nodes join the network by providing them all with one and the same - bootstrap node at start-up. TODO: The amount of bootstrap nodes should become - configurable. + bootstrap node at start-up. 2. Attempt to add the ENRs of all the nodes to each node its routing table: This is done in order to quickly simulate a network that has all the nodes -propagated around. A custom JSON-RPC call is used for this. + propagated around. The JSON-RPC `portal_historyAddEnr` is used for this. 3. Select, at random, a node id of one of the nodes. Let every node do a lookup for this node id. This is done to validate that every node can successfully lookup a specific node in the DHT. -### Sharing the data around +### Data propagation test -How the content gets shared around: +How the content gets shared around and tested: 1. First one node (the bootstrap node) will get triggered by a JSON-RPC call - to load a data file that contains `n` amount of blocks (Currently - `[header, txs, uncles]`, should probably become + to load a data file that contains `n` amount of blocks ( `[header, [txs, uncles], receipts]`), and to propagate these over the network. - This is done by doing offer request to `x` (=6 currently) neighbours to that - content its id. This is practically the neighborhood gossip at work, but then + This is done by doing offer request to `x` (= 8) neighbours to that content + its id. This is practically the neighborhood gossip at work, but then initiated on the read of each block in the provided data file. 2. Next, the nodes that accepted and received the content will do the same neighborhood gossip mechanism with the received content. And so on, until no diff --git a/fluffy/docs/the_fluffy_book/mkdocs.yml b/fluffy/docs/the_fluffy_book/mkdocs.yml new file mode 100644 index 000000000..a1f57924d --- /dev/null +++ b/fluffy/docs/the_fluffy_book/mkdocs.yml @@ -0,0 +1,62 @@ +# Nimbus Fluffy book +# Copyright (c) 2023 Status Research & Development GmbH +# Licensed and distributed under either of +# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT). +# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0). +# at your option. This file may not be copied, modified, or distributed except according to those terms. + +site_name: The Nimbus Fluffy Guide +theme: + name: material + features: + - navigation.expand + - navigation.tabs + - navigation.top + - content.tabs.link + palette: + scheme: default + primary: orange + accent: amber + +# Support urls previously used by mdbook +use_directory_urls: false +repo_url: https://github.com/status-im/nimbus-eth1 +edit_uri: edit/unstable/docs/the_nimbus_book/src + +markdown_extensions: + - admonition + - meta + - pymdownx.details + - pymdownx.highlight: + anchor_linenums: true + - pymdownx.inlinehilite + - pymdownx.snippets + - pymdownx.superfences + - pymdownx.tabbed: + alternate_style: true + - tables + - toc: + toc_depth: 3 + permalink: "#" + +nav: + - Home: + - 'index.md' + + - Getting started: + - 'quick-start.md' + - 'quick-start-windows.md' + - 'quick-start-docker.md' + + - How-to: + # - 'install.md' + # - 'build.md' + - Portal bridge: + - 'history-content-bridging.md' + - 'beacon-content-bridging.md' + - Local testnet: + - 'run-local-testnet.md' + - 'testnet-history-network.md' + - 'testnet-beacon-network.md' + - 'protocol-interop-testing.md' + - 'eth-data-exporter.md'