Commit Graph

6767 Commits

Author SHA1 Message Date
Etan Kissling 2a1bf9944d
bump `nim-json-rpc` to `9c65df305390ab01a13788af5aea78cd62dceb2a` (#5923)
- Client also handle error message if id is null
- Client pass meaningful error to newFut when processMessage failed
- Refactoring: extract rpc handler from HTTP and WebSocket server
- Use pragma push/pop pair to disable warning
- http server better exception handling
- Fix CI badge url
- Upgrade github actions to v4
- Revert "Fix CI badge url"
- HttpAuthHook use async raises
- Move CancelledError handling to outer try/except of RpcWebsocketServer
- Implement RPC batch call both in servers and clients
- v0.4.0
- Should compile if chronicles log turned on
- Add framework to support more optional types
- v0.4.2
- test refc in CI in Nim 2.0 and later
- use non-EOL macOS version for GitHub Actions CI
- avoid failing uninitialized `Future`
- Improve batch call example and wrapper comments
- Fix ws and socket client error handling and add test to #212
- Add build test with chronicles to json enabled
2024-02-20 21:27:06 +01:00
Etan Kissling eefe8841ef
bump `nim-stew` to `1662762c0144854db60632e4115fe596ffa67fca` (#5921)
- Add push raises to shims/macros module
- rename `RangeError` -> `RangeDefect` in test name
2024-02-20 21:21:09 +01:00
Etan Kissling 88045a91cd
rename new timing metrics, as `_total` suffix is implicit (#5917)
* track latest duration instead of total in new timing metrics

Change `db_checkpoint_seconds` and `state_replay_seconds` metrics to
record the latest duration instead of the total. `nim-metrics` already
synthesizes a `_total` metric from these implicitly.

* still have to use inc, metrics only synthesizes the name not the sum

* prefix with `beacon_dag`
2024-02-20 20:34:41 +01:00
Etan Kissling 9286eb6795
bump `nim-kzg4844` to `5eba921b1079376403b8b8e1b44c70248eabbc70` (#5922)
- bump `csources` to `v0.4.1`
- test refc in CI in Nim 2.0 and later
- bump `csources` to `v0.4.2`
2024-02-20 18:45:00 +00:00
Etan Kissling 6fe183f0de
bump `nim-websock` to `ed89f58624ba6482225bb6a3169cefd7e632f498` (#5920)
- Reduce compiler warnings when using new chronos
- Upgrade github action to v3
- Fix autobahn tests
- Reenable tests
- Restrict autobahn tests run to merged PR only
- Upgrade github action setup-python to v3 and up/dn artifact to v3
- Reduce compiler warnings
- Fix github actions badge url
- Reduce compiler warnings
- Upgrade to github actions v4
- Revert "Fix github actions badge url"
- Downgrade upload artifact action to v3
- use non-EOL macOS version for GitHub Actions CI
2024-02-20 17:12:09 +00:00
Etan Kissling 8b494c2f26
bump `nim-chronicles` to `ab3ab545be0b550cca1c2529f7e97fbebf5eba81` (#5918)
- Upgrade to github actions v4
- Silence declared but unused warnings when there is no loggging code generated
2024-02-20 15:21:28 +01:00
Etan Kissling 74eeb0b93c
bump `nim-blscurve` to `1002533ad205fa71d60d6fd8f8c283b9352a5314` (#5916)
- update instructions for tracking upstream MIRACL Core
- bump `bls12-381-tests` to `v0.1.2`
- bump `miracl-core` to `0f67878bee7c4108405deb2b0b5e4e58d1ae30fc`
- test refc in CI in Nim 2.0 and later
- rename `milagro.nims` -> `miracl.nims`
- rename `milagro.nim` -> `miracl.nim`
- rename `milagro(Path|_func)` -> `miracl(Path|_func)`
- rename `milagro` references -> `miracl` in documentation
2024-02-20 13:35:22 +01:00
Etan Kissling ba7549a66e
bump `nim-bearssl` to `7640cc6aa859f6d4ad657c892f14c76dc1d34bd5` (#5915)
- test refc in CI in Nim 2.0 and later
2024-02-20 13:31:36 +01:00
tersec ffbc8d1466
refactor epoch state transition to facilitate individual validator balance change calculations (#5910) 2024-02-20 05:14:52 +00:00
Jacek Sieka 8d465a7d8c
vmon: Missed block metric (#5913)
Validator monitoring gained 2 new metrics for tracking when blocks are
included or not on the head chain.

Similar to attestations, if the block is produced in epoch N, reporting
will use the state when switching to epoch N+2 to do the reporting (so
as to reasonably stabilise the block inclusion in the face of reorgs).
2024-02-20 06:40:18 +02:00
tersec 87ae60f780
search for validator indices backwards while processing deposits (#5914) 2024-02-20 06:34:57 +02:00
Zahary Karadjov 7fe43fc204
Version v24.2.1 2024-02-20 05:49:56 +02:00
Etan Kissling ae12b71ce0
bump `holesky` to `60997dd7e876a67775a7f2d53b852e75bbb2a630` (#5912)
- Add Lodestar holesky bootnode ENRs
2024-02-19 19:27:10 +01:00
tersec 28f69ccc0a
add Prater/Goerli deprecation notice (#5898) 2024-02-19 10:09:39 +00:00
Etan Kissling 698a802168
extend linter to enforce exception tracking is turned on (#5909)
Status Nim style mandates `{.push raises: []}.` at start of modules.
Add a CI task to ensure exceptions keep getting properly tracked.

- https://status-im.github.io/nim-style-guide/errors.exceptions.html
- https://github.com/status-im/nim-eth/pull/614#discussion_r1220906149
2024-02-19 11:02:36 +01:00
Etan Kissling 92197ce690
add metric for database checkpoint duration (#5897)
Database checkpointing can take seconds, e.g., while Geth is syncing.
Add a debug log + metric for it, and also info log if it takes longer
than 250ms, same as for the existing `State replayed` log. If the log
shows up for a user while the system is not overloaded, it may point
to slow disk speed or thermal issue.
2024-02-19 11:00:11 +01:00
Etan Kissling 5fdb06fcd1
extend `ncli` exception handling (#5905)
Make raised exceptions explicit in `ncli_common.nim`, and handle more of
them in `ncli_db.nim` to have better UX when directories cannot be read
or file names do not parse against the expected format.
2024-02-19 10:56:19 +01:00
Etan Kissling e04e95167d
avoid `read`/`readError` in favor of `value`/`error` (#5904)
In VC logic, bump 3 remaining uses of `readError`/`read` to use
`error`/`value` instead. The surrounding logic guarantees success.
2024-02-19 10:52:35 +01:00
Etan Kissling 4fc1550d0f
add `{.push raises: [].}` to recently modified files (#5908)
Status Nim style mandates `{.push raises: []}.` at start of modules.
Ensure that's the case so that exceptions are properly tracked.

- https://status-im.github.io/nim-style-guide/errors.exceptions.html
- https://github.com/status-im/nim-eth/pull/614#discussion_r1220906149
2024-02-18 01:16:49 +00:00
Etan Kissling bf882887bd
`noCancel` in `test_gossip_validation` (#5907)
To avoid spurious warnings, mark futures that are not cancelled by test
as such when waiting for them. Variable rename is for max line length.
2024-02-18 00:40:15 +00:00
Etan Kissling e7d2ee7cb6
exception handling in `block_sim` (#5906)
There were a few missing exception handlers in the `block_sim` dev tool.
Add them now, and also remove `strformat` dependency for modernization.
2024-02-17 23:27:49 +00:00
Etan Kissling 71444a371a
annotate `test_fixture_fork_choice` with `{.raises.}` (#5903)
The fork choice test fixture is one of the more complex ones. Annotate
its functions with `{.raises.}` to make exception flow explicit.
2024-02-17 22:52:11 +01:00
Etan Kissling 9cd6b270df
rename reference of `RangeError` to `RangeDefect` (#5902)
`RangeError` got deprecated in favor of `RangeDefect`. Update handbook.
2024-02-17 20:29:06 +01:00
Etan Kissling 403568cd2b
modernize `simutils` (#5901)
`simutils.nim` is quite outdated w.r.t. code style. Apply the following:

- Use string concatenation instead of `strformat` for simple cases
- Catch `IOError` and `SerializationError` when loading/saving SSZ files
- Catch `ValueError` for remaining `strformat` usage
- Consistently use `chronicles` in `loadGenesis`
2024-02-17 20:23:26 +01:00
Etan Kissling 30b7c6153f
handle `Exception` during `EraFile.verify` (#5900)
`Taskpool.new()` is marked as `{.raises: [Exception].}`. Catch this.
2024-02-17 18:19:30 +01:00
Etan Kissling 3b5056db92
update `nim-blscurve` link to follow renamed file (#5899)
`milagro.nim` was renamed to `miracl.nim`. Point to the new filename
from auditors handbook.

- https://github.com/status-im/nim-blscurve/pull/166
2024-02-17 12:44:33 +01:00
Jacek Sieka b5089ebf70
log elmanager timeouts (#5895)
Also:

* remove some unused metrics
* simplify execution payload fetching flow
2024-02-17 10:15:02 +01:00
tersec e410fe0052
https://github.com/ethereum/consensus-specs/pull/3600 (#5896) 2024-02-17 09:02:50 +00:00
tersec 7b5815358e
test macOS with Nim 2.0 in CI and use non-EOL macOS version (#5890) 2024-02-16 04:53:13 +00:00
tersec ea29e0afc8
use 1.4.0-beta.7-hotfix consensus spec test vectors (#5894) 2024-02-16 04:49:18 +00:00
Eugene Kabanov 211f53582f
Increase buffer sizes to 16348. Which should bring some fresh air to el_manager. (#5893) 2024-02-15 07:01:15 +01:00
Eugene Kabanov 535d599c57
Bump chronos to fix REST server become unresponsive at some cases. (#5892) 2024-02-14 20:31:16 +01:00
tersec 52c538fb3c
stop calling exchangeTransitionConfiguration (#5889) 2024-02-14 10:01:08 +00:00
tersec 384d40fc0b
remove requirement for REST JSON serialization tests to hash signedbeaconblock (#5887) 2024-02-13 17:30:03 +00:00
Etan Kissling 81b849a2eb
bump `gnosis-chain-configs` to `b02e5dd0bc61f123fa28d027cf95d47ebe2ae05d` (#5885)
- Schedule deneb
2024-02-13 12:07:22 +01:00
Jacek Sieka afdfe302f3
state loading optimizations (#5881)
* compute post-merge randao mix without loading state
* avoid copying state on shuffling computation and compute epochref
* speed up state copy for block production
2024-02-12 15:58:55 +01:00
Peter Straus ec710bd90e
fix: broken links in docs (#5880)
* fix: broken link to consensus api

* fix: broken link to divio.com
2024-02-10 14:58:29 +00:00
tersec 8240c1bf34
use decimal representations of engine and builder bid values (#5879) 2024-02-10 05:13:00 +01:00
tersec 134774e00d
ensure reason field logging consistently uses string type (#5878) 2024-02-10 03:50:31 +01:00
tersec a4680cb7fa
refactor addHeadBlock() to research/ and tests/ helper (#5874)
* refactor addHeadBlock() to research/ and tests/ helper

* rm now-dead code
2024-02-09 23:46:51 +00:00
Etan Kissling 9593ef74b8
do not cache zero block hash if block unavailable (#5865)
With checkpoint sync, the checkpoint block is typically unavailable at
the start, and only backfilled later. To avoid treating it as having
zero hash, execution disabled in some contexts, wrap the result of
`loadExecutionBlockHash` in `Opt` and handle block hash being unknown.

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
2024-02-09 22:10:38 +00:00
Etan Kissling 7c53841cd8
Revert "Revert "fix checkpoint block potentially not getting backfilled into DB (#5863)" (#5871)" (#5875)
This reverts commit 1575478b72.
2024-02-09 20:44:54 +01:00
Etan Kissling f2d92729a2
reduce verbosity of `Got request for pre-backfill slot` (#5876)
When syncing, we log a notice each time someone asks us for a block that
we haven't backfilled yet. This is quite verbose and not unexpected,
because the status message does not allow indicating backfill progress.
2024-02-09 20:32:31 +01:00
tersec 1575478b72
Revert "fix checkpoint block potentially not getting backfilled into DB (#5863)" (#5871)
This reverts commit 65e6f892de.
2024-02-09 12:49:07 +00:00
Etan Kissling 65e6f892de
fix checkpoint block potentially not getting backfilled into DB (#5863)
When using checkpoint sync, only checkpoint state is available, block is
not downloaded and backfilled later.

`dag.backfill` tracks latest filled `slot`, and latest `parent_root` for
which no block has been synced yet.

In checkpoint sync, this assumption is broken, because there, the start
`dag.backfill.slot` is set based on checkpoint state slot, and the block
is also not available.

However, sync manager in backward mode also requests `dag.backfill.slot`
and `block_clearance` then backfills the checkpoint block once it is
synced. But, there is no guarantee that a peer ever sends us that block.
They could send us all parent blocks and solely omit the checkpoint
block itself. In that situation, we would accept the parent blocks and
advance `dag.backfill`, and subsequently never request the checkpoint
block again, resulting in gap inside blocks DB that is never filled.

To mitigate that, the assumption is restored that `dag.backfill.slot`
is the latest filled `slot`, and `dag.backfill.parent_root` is the next
block that needs to be synced. By setting `slot` to `tail.slot + 1` and
`parent_root` to `tail.root`, we put a fake summary into `dag.backfill`
so that `block_clearance` only proceeds once checkpoint block exists.
2024-02-09 11:20:36 +01:00
Etan Kissling 4266e16835
allow `getBlockIdAtSlot` to answer queries from available states (#5869)
After checkpoint sync, historical block IDs cannot yet be queried.
However, they are needed to compute dependent roots of `ShufflingRef`.
To allow lookup, enable `getBlockIdAtSlot` to answer from compatible
states in memory; as long as they descend from the finalized checkpoint
and the requested slot is sufficiently recent, `block_roots` contains
everything to recover `BlockSlotId` up to `SLOTS_PER_HISTORICAL_ROOT`.
This is similar to how `attester_dependent_root` etc. are computed.

This accelerates the first couple minutes of checkpoint sync on Mainnet,
especially the time until finality advances past the synced checkpoint.
2024-02-09 11:13:00 +01:00
Etan Kissling 91cf50a5ad
bump `geth` to `v1.13.2` (#5870)
- https://github.com/ethereum/go-ethereum/releases/tag/v1.13.12
2024-02-09 11:04:18 +01:00
tersec 642774e596
unrevert rest of https://github.com/status-im/nimbus-eth2/pull/5765 (#5867)
* unrevert rest of https://github.com/status-im/nimbus-eth2/pull/5765

* rm stray e2store docs changes

* reduce diff

* fix indent

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
2024-02-09 09:35:41 +01:00
Kim De Mey dca444bea7
Split era specific code from e2s specific code (#5866) 2024-02-09 08:59:36 +01:00
Etan Kissling a746063a61
bump `eth2-networks` to `934c948e69205dcf2deb87e4ae6cc140c335f94d` (#5868)
- Schedule Deneb for Mainnet
2024-02-08 19:18:35 +00:00