314 Commits

Author SHA1 Message Date
tersec
b56a671122
fix most ConvFromXtoItselfNotNeeded hints and unhide remaining ones (#6307) 2024-05-22 13:56:37 +02:00
tersec
1c3aaa7be2
add (Signed)AggregateAndProof SSZ tests (#6285) 2024-05-14 13:51:06 +02:00
tersec
137d969f1d
configure block_sim to start with Deneb genesis (#6274) 2024-05-08 06:38:49 +00:00
tersec
1b30dcc165
initial electra attestation pool changes; electra block_sim (#6255) 2024-05-07 15:01:51 +00:00
tersec
87452374e4
add Electra SSZ object test fixture (#6225) 2024-04-22 09:00:38 +00:00
tersec
603c83522e
explicitly refer to phase0.{Attestation,TrustedAttestation} rather than sans module name (#6214) 2024-04-17 20:44:29 +00:00
Etan Kissling
41a3b62fe2
update wss_sim for Bellatrix, Capella and Deneb (#6170)
The `wss_sim` was not properly maintained since Bellatrix. The missing
functionality is now added, including:

- Bellatrix: Connect to an EL for execution payload production
- Capella: Correct withdrawals processing, is mandatory to do
- Deneb: Dump blob sidecars into the output directory

See https://ethresear.ch/t/insecura-my-consensus-for-the-pyrmont-network/11833
2024-04-08 15:28:46 +02:00
tersec
937cc62b85
block_sim runs electra (#6181) 2024-04-07 09:58:11 +02:00
tersec
0b6f75b88b
remove some debugRaiseAsserts and fill in actual Electra functionality (#6172) 2024-04-04 05:17:31 +02:00
tersec
7fa32b7f02
add Electra to ConsensusFork enum (#6169)
* add Electra to ConsensusFork enum

* fix gnosis check
2024-04-03 16:43:43 +02:00
Etan Kissling
9ad8ea0f7a
fix missing space in block_sim output (#6134)
In #5906, the `strformat` use was replaced with simply passing the parts
of the string as `varargs` to `echo`. A space got lost from output as
part of the transformation. Re-add it.
2024-03-25 20:26:14 +01:00
Etan Kissling
5d42859176
make Gwei distinct (#6090)
#6087 introduced a subtle change to `nim-web3` resulting in `Gwei` to be
serialized differently than before. Using a `distinct` type for `Gwei`
improves type safety and avoids such problems in the future.
2024-03-19 14:22:07 +01:00
Etan Kissling
1dd2c939ac
bump nim-web3 to 80c7aa6de2a26c57fa1f06ad47f3ac6058e6545b (#6088)
- Add writeValue for BlockNumber
- make `BlockNumber` `distinct`
2024-03-19 14:21:47 +01:00
Etan Kissling
50a43f397f
rename DepositTreeSnapshot -> DepositContractSnapshot (#6036)
EIP-4881 was never correctly implemented, the `DepositTreeSnapshot`
structure has nothing to do with its actual definition. Reflect that
by renaming the type to a Nimbus-specific `DepositContractSnapshot`,
so that an actual EIP-4881 implementation can use the correct names.

- https://eips.ethereum.org/EIPS/eip-4881#specification

Notably, `DepositTreeSnapshot` contains a compressed sequence in
`finalized`, only containing the minimally required intermediate roots.

That also explains the incorrect REST response reported in #5508.

The non-canonical representation was introduced in #4303 and is also
persisted in the database. We'll have to maintain it for a while.
2024-03-07 18:42:52 +01:00
tersec
b4db011daf
enable --styleCheck:error in ncli/ and research/ (#6014) 2024-03-02 06:25:17 +01:00
tersec
84034c0379
rm Capella builder API-related remote signer support (#6003) 2024-03-01 05:30:09 +00:00
tersec
d7e9ff2a98
fix some stylecheck consistency Name hints in ncli/ and research/ (#6004) 2024-03-01 05:59:34 +01:00
Etan Kissling
f5e9d3ffe4
add trivial {.push raises: [].} to research and test folders (#5988)
Annotate the `research` and `test` files for which no further changes
are needed to successfully compile them, to not interfere with periodic
tasks such as spec reference bumps.
2024-02-29 13:24:08 +00:00
Etan Kissling
078f4ac347
assert exceptions that statically cannot occur in test code (#5989)
There are some spurious exceptions that statically cannot occur.
Add `raiseAssert` for these.
2024-02-28 20:15:49 +00:00
tersec
a4f4a35845
Revert "initial Electra support skeleton" (#5955)
* Revert "initial Electra support skeleton (#5946)"

This reverts commit d09bf3b587bc4f0c91b8e2f58884665a0ae80e34.

* Update test_signing_node.nim
2024-02-25 19:42:44 +00:00
tersec
d09bf3b587
initial Electra support skeleton (#5946) 2024-02-24 13:44:15 +00:00
tersec
e865817d44
add some push raises, ElectraBeaconStateNoImmutableValidators, and ELECTRA_FORK_VERSION/EPOCH (#5949) 2024-02-24 05:08:22 +00:00
Etan Kissling
e7d2ee7cb6
exception handling in block_sim (#5906)
There were a few missing exception handlers in the `block_sim` dev tool.
Add them now, and also remove `strformat` dependency for modernization.
2024-02-17 23:27:49 +00:00
Etan Kissling
403568cd2b
modernize simutils (#5901)
`simutils.nim` is quite outdated w.r.t. code style. Apply the following:

- Use string concatenation instead of `strformat` for simple cases
- Catch `IOError` and `SerializationError` when loading/saving SSZ files
- Catch `ValueError` for remaining `strformat` usage
- Consistently use `chronicles` in `loadGenesis`
2024-02-17 20:23:26 +01:00
tersec
a4680cb7fa
refactor addHeadBlock() to research/ and tests/ helper (#5874)
* refactor addHeadBlock() to research/ and tests/ helper

* rm now-dead code
2024-02-09 23:46:51 +00:00
xiaolou86
af6affb5b3
Fix typos (#5835)
* Fix typos

* Update copyright year
2024-01-29 09:37:38 +00:00
tersec
7fd8beb418
rm unused code in {ncli,research,tests}/ (#5809) 2024-01-21 07:55:03 +01:00
Jacek Sieka
d5785677a8
split out eth1chain into its own module (#5768)
reduces import junk in some places - more could be done here
2024-01-17 15:26:16 +01:00
andri lim
017f9f1103
bump nim-web3 to bf1ce869b5f76d9744444b248f6f5c6c782fedc7 (#5722)
Etan Kissling (2):
  remove unused `skip0xPrefix`
  keep the internal count helper

Will (1):
  Bugfix/nully values (#61)

Yuriy Glukhov (5):
  Contract constructor support
  Fixed compilation error in exec function
  Added string encoding
  Fixed source->from field of EthCall
  More flexibility to contract DSL, Async contract caller

jangko (5):
  Reduce compiler warnings when using Nim v2
  Migrate to json-serialization
  Add tests of json rpc marshalled types
  Resolve contract_dsl ambiguity
  Event handler passing around JsonString instead of JsonNode
  Share encoder between json-rpc and chronicles (#119)
  Simplify generic constraint of rpc and chronicles encoders
  Feature/execution api spec (#69)
  v0.3.0

bump nim-json-rpc to a6475e49b26d3afc58aaa3d67621c94eafef8efb
coffeepots (1):
  Use nim-json-serialization for RPCs (#172)

jangko (10):
  Add copyright to source file
  Remove StringOfJson
  Fix optional parameter parsing fails in rpc macro with generics
  Rename jrpc_sys module back to jsonmarshal
  Reenable test hhtps
  Add test for createRpcSigsFromNim and createSingleRpcSig
  Let the OS choose the port for tests
  Add onProcessMessage hook to client
  Fix example in the README.md
  Move errors module back to json_rpc folder
  Upgrade rpc router internals (#178)
  RPC server handle null return value correctly
  v0.3.0

kdeme (1):
  Add example test case that currently fails the Option parsing
2024-01-13 08:36:17 +07:00
Jacek Sieka
62cbdeefc5
verify genesis_time more strictly (fixes #1667) (#5694)
Bogus values lead to crashes down the line when timers overflow
2024-01-06 15:26:56 +01:00
Etan Kissling
508f3b6368
add missing std/ prefix to more imports (#5696)
Bumping some `std` imports to explicitly include the prefix.
Also add explicit `./` prefix for local directory imports.
2024-01-06 07:18:28 +01:00
Jacek Sieka
3f525acb87
chronos: bump (#5684)
This PR causes a few new warnings to appear - these are harmless but
will need addressing separately as they span several libraries.

* new asyncraises syntax
* asyncraises support in several modules
* `sink` usage reduces spurious copying
* `?` compatiblity for `async` + `results`
* remove `-d:chronosStrictException` (obsolete)
2024-01-05 10:08:38 +01:00
andri lim
15147cccb1
Bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354 (#5664)
* bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354

also bump json-rpc to a8731e91bc336d930ac66f985d3b88ed7cf2a7d7
2023-12-12 22:15:00 +07:00
tersec
1791b0ee0a
fix XDeclaredButNotUsed warnings (#5638) 2023-12-04 21:14:35 +00:00
Etan Kissling
e7bc41e005
blck --> forkyBlck when using withBlck / withStateAndBlck (#5451)
For symmetry with `forkyState` when using `withState`, and to avoid
problems with shadowing of `blck` when using `withBlck` in `template`,
also rename the injected `blck` to `forkyBlck`.

- https://github.com/nim-lang/Nim/issues/22698
2023-09-21 12:49:14 +02:00
tersec
295c3e2e41
rm state_sim; capella genesis for block_sim (#5331)
* rm state_sim; capella genesis for block_sim

* copyright year
2023-08-21 09:10:15 +00:00
Jacek Sieka
3c9fc49411
reduce ncli/research imports (#5275)
* reduce ncli/research imports

avoids ssl, most of chronos etc, speeds up compiles

* copyright
2023-08-10 14:52:49 +02:00
Jacek Sieka
b8a32419b8
async batch verification (+40% sig verification throughput) (#5176)
* async batch verification

When batch verification is done, the main thread is blocked reducing
concurrency.

With this PR, the new thread signalling primitive in chronos is used to
offload the full batch verification process to a separate thread
allowing the main threads to continue async operations while the other
threads verify signatures.

Similar to previous behavior, the number of ongoing batch verifications
is capped to prevent runaway resource usage.

In addition to the asynchronous processing, 3 addition changes help
drive throughput:

* A loop is used for batch accumulation: this prevents a stampede of
small batches in eager mode where both the eager and the scheduled batch
runner would pick batches off the queue, prematurely picking "fresh"
batches off the queue
* An additional small wait is introduced for small batches - this helps
create slightly larger batches which make better used of the increased
concurrency
* Up to 2 batches are scheduled to the threadpool during high pressure,
reducing startup latency for the threads

Together, these changes increase attestation verification throughput
under load up to 30%.

* fixup

* Update submodules

* fix blst build issues (and a PIC warning)

* bump

---------

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2023-08-03 11:36:45 +03:00
Etan Kissling
2722778ce5
reduce nim-eth dependencies just for RNG (#5099)
We have several modules that import `nim-eth` for the sole purpose of
its `keys.newRng` function. This function is meanwhile a simple wrapper
around `nim-bearssl`'s `HmacDrbgContext.new()`, so the import doesn't
really serve a use anymore. Replace `keys.newRng` with the direct call
to reduce `nim-eth` imports.
2023-06-19 22:43:50 +00:00
tersec
38a6706509
initial version of mock builder relay (#5034) 2023-06-06 15:21:15 +00:00
Etan Kissling
40e89937c5
segregate sync committee messages by period / fork (#4953)
`SyncCommitteeMsgPool` grouped messages by their `beacon_block_root`.
This is problematic around sync committee period boundaries and forks.
Around sync committee period boundaries, members from both the current
and next sync committee may sign the same `beacon_block_root`; mixing
the signatures from both committees together is a mistake. Likewise,
around fork transitions, the `signing_root` changes, so those messages
also need to be segregated.
2023-05-17 07:55:55 +03:00
henridf
573228ffa0
Rename eth1/ -> el/ and eth1_monitor.nim -> el_monitor.nim (#4944) 2023-05-15 05:05:12 +00:00
Jacek Sieka
34a81f56c3
bump web3 (#4919)
and update fakeee
2023-05-09 22:51:57 +00:00
Etan Kissling
445ece1157
align MockPrivKeys with EF keys (#4866)
Back then, Milagro interop used offset 1000 for mock BLS keys.
Meanwhile, interop code was removed and multi client testnets are there.
EF tests use an offset of 1 for mock BLS keys. This patch aligns our
implementation to also use offset of 1, potentially making debugging of
state differences a bit easier (but, ultimately, low impact).

Furthermore, simulation files are now emitted into a subdirectory
to have less chunk in the repo root directory, and to avoid conflicts
where a cached file with offset 1000 runs against tests expecting 1.

See https://github.com/status-im/nimbus-eth2/pull/2928/files#r719266863
2023-04-27 12:17:19 +00:00
tersec
228e10f1d9
update engine API URLs from v1.0.0-beta.2 to beta.3 (#4828) 2023-04-17 20:11:28 +00:00
tersec
c31fbc3977
remove most std/options imports (#4778) 2023-03-31 20:46:47 +00:00
Etan Kissling
ad118cd354
rename stateFork > consensusFork (#4718)
Just the variable, not yet `lcDataForkAtStateFork` / `atStateFork`.

- Shorten comment in `light_client.nim` to keep line width
- Do not rename `stateFork` mention in `runProposalForkchoiceUpdated`.
- Do not rename `stateFork` in `getStateField(dag.headState, fork)`

Rest is just a mechanical mass replace
2023-03-11 00:35:52 +00:00
Etan Kissling
969c6f73ae
misc local EIP4844 > Deneb bumps (#4717)
* misc local `EIP4844` > `Deneb` bumps

* fix
2023-03-11 00:28:19 +00:00
tersec
a47f0b054e
finish eip4844 to deneb module rename (#4705) 2023-03-09 01:34:17 +01:00
tersec
63b1b0840f
5 more modules of eip4844.foo to deneb.foo renames (#4698) 2023-03-06 18:45:52 +00:00