Version 23.3.0

CHANGELOG.md

@@ -1,3 +1,78 @@
+2023-03-11 v23.3.0
+==================
+
+Nimbus `v23.3.0` is `low-urgency` upgrade bringing full support for the upcoming Capella hard-fork on the Goerli testnet. Keep an eye out for future mainnet releases!
+
+### Improvements
+
+* You can now increase the resilience of your setup and eliminate any downtime during upgrade procedures of the execution client software by allowing your beacon node to manage multiple execution clients. To enable this mode, just specify multiple URLs through the `--el` option (alias of `--web3-url`) when starting your beacon node:
+
+  ```sh
+  ./run-mainnet-beacon-node.sh \
+    --el=http://127.0.0.1:8551 \
+    --el=ws://other:8551 \
+    --jwt-secret=/tmp/jwtsecret
+  ```
+
+  As long as only one of execution clients remains operational and fully synced, Nimbus will keep performing all validator duties.
+
+  If you use this mode with different execution client implementations, Nimbus will act as an execution layer consensus violation detector, preventing the publishing of blocks that may trigger a catastrophic partitioning in the network.
+
+  https://github.com/status-im/nimbus-eth2/pull/4465
+  https://nimbus.guide/eth1.html
+
+* The metrics `engine_api_responses`, `engine_api_request_duration_seconds` and `engine_api_timeouts` provide statistics about latency and response status codes for all requests sent to each individual execution layer URL:
+
+  https://github.com/status-im/nimbus-eth2/pull/4465
+  https://github.com/status-im/nimbus-eth2/pull/4707
+
+* Nimbus will now attempt to connect to a locally running execution client even when the options `--el` and `--jwt-secret` are not specified. This is made possible by the following proposed standard:
+
+  https://github.com/ethereum/execution-apis/pull/302
+
+  Please note that the standard hasn't been implemented in any execution client yet.
+
+* Nimbus now support the latest version of the Builder API, adding support for the Capella hard-fork:
+
+  https://github.com/status-im/nimbus-eth2/pull/4643
+
+* Improved diagnostic messages and more spec-compliant behavior of the Nimbus validator client when being paired with a non-synced or optimistically synced beacon nodes:
+
+  https://github.com/status-im/nimbus-eth2/pull/4643
+  https://github.com/status-im/nimbus-eth2/pull/4657
+  https://github.com/status-im/nimbus-eth2/pull/4673
+
+* The Sqlite3 database engine has been upgraded to version 3.40.1:
+
+  https://github.com/status-im/nimbus-eth2/pull/4649
+
+### Fixes
+
+* The doppelganger detection now acts safer after a period of lost network connectivity
+
+  https://github.com/status-im/nimbus-eth2/pull/4616
+
+* The doppelganger detection now acts safer in the presence of out-of-order responses from the beacon node:
+
+  https://github.com/status-im/nimbus-eth2/pull/4691
+
+* Nimbus can now export ERA files for the Sepolia network:
+
+  https://github.com/status-im/nimbus-eth2/pull/4689
+
+* The `--history=prune` mode will no longer interfere with serving light client data for the full retention period as mandated by the spec:
+
+  https://github.com/status-im/nimbus-eth2/pull/4702
+
+* Nimbus now downloads a longer range of recent execution blocks in order to avoid potential situations where our `Eth1Data` votes fail to agree with the honest majority in the network:
+
+  https://github.com/status-im/nimbus-eth2/pull/4588
+
+* Nimbus has addressed a potential interruption of deposit syncing when connected to Geth over WebSocket:
+
+  https://github.com/status-im/nimbus-eth2/pull/4708
+
+
 2023-02-16 v23.2.0
 ==================
 
@@ -27,7 +102,7 @@ Capella hard-fork on the Sepolia testnet. Keep an eye out for future mainnet rel
   https://github.com/status-im/nimbus-eth2/pull/4591
 
 * The Keymanager API now supports all `gas_limit` end-points:
-  https://ethereum.github.io/keymanager-APIs/#/Gas%20Limit  
+  https://ethereum.github.io/keymanager-APIs/#/Gas%20Limit
   https://github.com/status-im/nimbus-eth2/pull/4612
 
 * Nimbus serves light client updates up to the retention period mandated by

beacon_chain/version.nim

@@ -17,7 +17,7 @@ when not defined(nimscript):
 
 const
   versionMajor* = 23
-  versionMinor* = 2
+  versionMinor* = 3
   versionBuild* = 0
 
   versionBlob* = "stateofus" # Single word - ends up in the default graffiti

docs/Makefile

@@ -1,7 +1,7 @@
 mkdocs: mkdocs/bin/mkdocs
 
 mkdocs/bin/mkdocs:
-	test -d mkdocs || python -m venv mkdocs
+	test -d mkdocs || python3 -m venv mkdocs
 	. mkdocs/bin/activate; pip install -r requirements.txt
 
 compile:

docs/the_nimbus_book/src/eth1.md

@@ -8,7 +8,7 @@ Nimbus has been tested all major execution clients - see the [execution client c
     You need to run your own execution client - relying on third-party services such as Infura, Alchemy and Pocket is no longer possible. Sharing the same execution client between multiple beacon nodes is not supported.
 
 !!! info
-    Syncing an execution client may take hours or even days, depending on your hardware! The backup providers will be synced only when the primary becomes unavailable, which may lead to a small gap in validation duties - this limitation may be lifted in future versions.
+    Syncing an execution client may take hours or even days, depending on your hardware!
 
 ## Steps
 
@@ -28,7 +28,7 @@ Select an execution client and install it, configuring it such that that the aut
 
     #### 2. Start Geth
 
-    Once you have geth installed, make sure to enable the JSON-RPC WebSocket interface when running geth, along with the options for creating an [autheticated RPC endpoint](https://geth.ethereum.org/docs/interface/consensus-clients):
+    Once you have geth installed, make sure to enable the [authenticated JSON-RPC interface](https://geth.ethereum.org/docs/getting-started/consensus-clients) when running geth:
 
     === "Mainnet"
         ```
@@ -40,21 +40,6 @@ Select an execution client and install it, configuring it such that that the aut
         geth --goerli --authrpc.addr localhost --authrpc.port 8551 --authrpc.vhosts localhost --authrpc.jwtsecret /tmp/jwtsecret
         ```
 
-    #### 3. Leave Geth running
-
-    Let it syns - it may take anywhere between a few hours and a couple of days.
-
-    You'll know Geth has finished syncing, when you start seeing logs that look like the following:
-
-    ```
-    INFO [05-29|01:16:05] Imported new chain segment               blocks=1 txs=3   mgas=0.065  elapsed=5.885ms   mgasps=11.038  number=3785445 hash=553d9e…fc4547
-    INFO [05-29|01:16:10] Imported new chain segment               blocks=1 txs=0   mgas=0.000  elapsed=5.447ms   mgasps=0.000   number=3785446 hash=5e3e7d…bd4afd
-    INFO [05-29|01:16:10] Imported new chain segment               blocks=1 txs=1   mgas=0.021  elapsed=7.382ms   mgasps=2.845   number=3785447 hash=39986c…dd2a01
-    INFO [05-29|01:16:14] Imported new chain segment               blocks=1 txs=11  mgas=1.135  elapsed=22.281ms  mgasps=50.943  number=3785444 hash=277bb9…623d8c
-    ```
-
-    Geth accepts connections from the localhost interface (`127.0.0.1`), with default authenticated RPC port `8551`. This means that your default Web3 provider URL should be: `http://127.0.0.1:8551`
-
 === "Nethermind"
 
     See the [Getting started](https://docs.nethermind.io/nethermind/first-steps-with-nethermind/getting-started) guide to set up Nethermind.
@@ -81,30 +66,49 @@ It is safe to start the beacon node even if the execution client is not yet full
 
 ### 3. Pass the URL and JWT secret to Nimbus
 
-The `--web3-url` option informs the beacon node how to connect to the execution client - both `http://` and `ws://` URLs are supported.
+The `--el` option informs the beacon node how to connect to the execution client - both `http://` and `ws://` URLs are supported.
+
+!!! info
+    By default, the execution client accepts connections on the localhost interface (`127.0.0.1`), with default authenticated RPC port `8551`. When the `--el` option is not explicitly specified, Nimbus will assume that the execution client is running on the same machine with such default settings.
 
 Once started, the execution client will create a file containing a JWT secret token. The token file is needed for Nimbus to authenticate itself with the execution client and perform trusted operations. You will need to pass the path to the token file to Nimbus together with the web3 URL.
 
 === "Mainnet"
     ```sh
     ./run-mainnet-beacon-node.sh \
-      --web3-url=http://127.0.0.1:8551 \
+      --el=http://127.0.0.1:8551 \
       --jwt-secret=/tmp/jwtsecret
     ```
 
 === "Prater"
     ```sh
     ./run-prater-beacon-node.sh \
-      --web3-url=http://127.0.0.1:8551 \
+      --el=http://127.0.0.1:8551 \
       --jwt-secret=/tmp/jwtsecret
     ```
 
-!!! tip "Multiple execution clients"
+!!! info
-    You can pass one or more `--web3-url` parameters to the node as long as they share JWT secret. Any additional web3 URLs will be used for backup, should the first one become unavailable:
+    When the `--jwt-secret` option is not specified and the execution client is running on the same machine under default setting, Nimbus may be able to connect successfully to it by using the default secret value `0xd4e56740f876aef8c010b86a40d5f56745a118d0906a34e69aec8c0db1cb8fa3`. This is a [proposed standard protocol](https://github.com/ethereum/execution-apis/pull/302) that aims to simplify the required user configuration, but it's not yet adopted by all execution clients.
 
-    ```sh
+## Advanced setups
-    ./run-mainnet-beacon-node.sh \
+
-      --web3-url=http://127.0.0.1:8551 \
+### Running multiple execution clients
-      --web3-url=ws://other:8551 \
+
-      --jwt-secret=/tmp/jwtsecret
+You can increase the resilience of your setup and eliminate any downtime during upgrade procedure of the execution client software by allowing your beacon node to manage multiple execution clients. To enable this mode, just specify multiple URLs through the `--el` option when starting your beacon node:
-    ```
+
+```sh
+./run-mainnet-beacon-node.sh \
+  --el=http://127.0.0.1:8551 \
+  --el=ws://other:8551 \
+  --jwt-secret=/tmp/jwtsecret
+```
+
+!!! tip
+    You can use a different secret for each connection by specifying `jwtSecret` or `jwtSecretFile` as a query parameter in the anchor section of the URL (e.g. `http://127.0.0.1:8551/#jwtSecret=0x12345...` or `http://127.0.0.1:8551/#jwtSecretFile=/tmp/jwtsecret`).
+
+As long as only one of execution clients remains operational and fully synced, Nimbus will keep performing all validator duties.
+
+!!! tip
+    To carry out an upgrade procedure without any downtime, just restart the execution clients one by one, waiting for each instance to re-sync before moving to the next one.
+
+If you use this mode with different execution client implementations, Nimbus will act as an execution layer consensus violation detector, preventing the publishing of blocks that may trigger a catastrophic partitioning in the network.

docs/the_nimbus_book/src/options.md

@@ -34,9 +34,10 @@ The following options are available:
      --validators-dir          A directory containing validator keystores.
      --secrets-dir             A directory containing validator keystore passwords.
      --wallets-dir             A directory containing wallet files.
-     --web3-url                One or more execution layer Web3 provider URLs.
+     --web3-url                One or more execution layer Engine API URLs.
-     --optimistic              Run the node in optimistic mode, allowing it to optimistically sync without an
+     --el                      One or more execution layer Engine API URLs.
-                               execution client [=false].
+     --no-el                   Don't use an EL. The node will remain optimistically synced and won't be able to
+                               perform validator duties [=false].
      --non-interactive         Do not display interative prompts. Quit on missing configuration.
      --netkey-file             Source of network (secp256k1) private key file (random|<path>) [=random].
      --insecure-netkey-password  Use pre-generated INSECURE password for network private key file [=false].
@@ -60,6 +61,7 @@ The following options are available:
                                this functionality [=false].
      --weak-subjectivity-checkpoint  Weak subjectivity checkpoint in the format block_root:epoch_number.
      --finalized-checkpoint-state  SSZ file specifying a recent finalized state.
+     --finalized-deposit-tree-snapshot  SSZ file specifying a recent finalized EIP-4881 deposit tree snapshot.
      --node-name               A name for this node that will appear in the logs. If you set this to 'auto', a
                                persistent automatically generated ID will be selected for each --data-dir
                                folder.
@@ -81,7 +83,7 @@ The following options are available:
      --rest-request-timeout    The number of seconds to wait until complete REST request will be received
                                [=infinite].
      --rest-max-body-size      Maximum size of REST request body (kilobytes) [=16384].
-     --rest-max-headers-size   Maximum size of REST request headers (kilobytes) [=64].
+     --rest-max-headers-size   Maximum size of REST request headers (kilobytes) [=128].
      --keymanager              Enable the REST keymanager API [=false].
      --keymanager-port         Listening port for the REST keymanager API [=5052].
      --keymanager-address      Listening port for the REST keymanager API [=127.0.0.1].
@@ -106,15 +108,17 @@ The following options are available:
                                a validator with the same index (a doppelganger), before sending an attestation
                                itself. This protects against slashing (due to double-voting) but means you will
                                miss two attestations when restarting. [=true].
-     --validator-monitor-auto  Automatically monitor locally active validators (BETA) [=false].
+     --validator-monitor-auto  Monitor validator activity automatically for validators active on this beacon
+                               node [=true].
      --validator-monitor-pubkey  One or more validators to monitor - works best when --subscribe-all-subnets is
-                               enabled (BETA).
+                               enabled.
-     --validator-monitor-totals  Publish metrics to single 'totals' label for better collection performance when
+     --validator-monitor-details  Publish detailed metrics for each validator individually - may incur significant
-                               monitoring many validators (BETA) [=false].
+                               overhead with large numbers of validators [=false].
      --suggested-fee-recipient  Suggested fee recipient.
-     --suggested-gas-limit     Suggested gas limit [=30000000].     
+     --suggested-gas-limit     Suggested gas limit [=defaultGasLimit].
      --payload-builder         Enable external payload builder [=false].
      --payload-builder-url     Payload builder URL.
+     --history                 Retention strategy for historical data (archive/prune) [=HistoryMode.Archive].
 
 ...
 ```