4362 Commits

Author SHA1 Message Date
Jacek Sieka
8690a03af7
Fix poor eth_getLogs performance (fixes #3033) (#3040)
* Fix poor eth_getLogs performance (fixes #3033)

* don't recompute txhash in inner log loop (!)
* filter logs before computing hashes

* copyright
2025-01-30 19:38:24 +00:00
bhartnett
e03a9c3172
Update nim-rocksdb to v9.10.0.0 (#3035) 2025-01-30 19:34:27 +01:00
Jordan Hrycaj
bc0620f6ca
Provide global sync progress, supersedes activation indicator (#3039) 2025-01-29 16:20:25 +00:00
pmmiranda
3e1f5e0b5f
Improved logging (#3001)
* Improved logging using nimbus-eth2 options

- log file support removal
- auto detection of tty, colour support and related.

* Added EthTime serializer (implemented on eth_types_json_serialization)

* added json chronicles logging support
 for some eth, web3 and confutils types

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2025-01-29 15:04:27 +00:00
Jordan Hrycaj
b9e1c94b99
Beacon sync provide activation status globally (#3038)
* Cosmetics, rename some functions, add/update comments

* Provide globally accessible syncer status query function
2025-01-29 12:31:06 +00:00
Jordan Hrycaj
8bc8094413
Beacon sync fix stalling while block processing (#3037)
* Correct docu

why:
  `T` is mentioned on the metrics table but not explained

* Update state sync ticker

why:
  Print last named state for debugging unexpected states.

* Rename `nec_consensus_head`=> `nec_sync_consensus_head`

why:
  This variable is syncer local, derived from what would be vaguely be
  the consensus head. In fact, at some point it is the consensus head
  but often will keep that value while the consensus head advances.

* Handle legit system state when block processing is cancelled

why:
  This state context was previously missing. It happens with problematic
  blocks (i.e. corrupt or missing.) Rather than trying to remedy the batch
  queue, all will be cancelled and the batch queue rebuilt from scratch.

* Update block queue with unexpectedly missing blocks

why:
  Concurrently serving `RPC` requests might cause a reset of the `FC`
  module data area. This in turn might produce a gap between expected `FC`
  module top and the beginning of the already downloaded blocks list.

  Currently this led to a deadlock situation because the missing blocks
  were never downloaded by the syncer, neither installed into `FC` module
  via `RFC`.

* Fix copyright year
2025-01-29 11:30:40 +00:00
kdeme
cdd71cac2e
Portal: Remove pruneDeprecatedAccumulatorRecords (#3031)
This was added to cleanup database from a content type that was
removed.

After ping extensions upgrade all active nodes must have been
updated to remain compatible and thus must have ran this pruning
code.
2025-01-29 10:20:29 +01:00
andri lim
13961fbae6
Bump nim-eth to 254be32672d0ffbddd110e21ecb55c32f9e90b02 (#3030)
* Bump nim-eth to 254be32672d0ffbddd110e21ecb55c32f9e90b02

* Bump nim-eth to e589cc0288b139509a19f3a27cd0cf0368ca0834
2025-01-29 13:53:47 +07:00
bhartnett
1b78aea40d
Bump nim-rocksdb (#3034) 2025-01-29 11:46:11 +08:00
andri lim
de80eb4214
Bump nim-stew to 687d1b4ab1a91e6cc9c92e4fd4d98bec7874c259 (#3032) 2025-01-29 08:48:03 +07:00
bhartnett
f78f82e8c9
Fluffy: Randomize gossip in neighborhoodGossip (#3029) 2025-01-29 08:49:37 +08:00
kdeme
28f9b792d9
Minimal implementation of Portal ping payload extensions spec (#3010)
* Minimal implementation of Portal ping payload extensions spec

* Remove serialization layer for CustomPayload (spec change)
2025-01-27 19:26:23 +01:00
andri lim
c4bc2a4eea
devnet-6: Update EIP-7840: Add BaseFeeUpdateFraction (#3022)
* devnet-6: Update EIP-7840: Add BaseFeeUpdateFraction

* Fix missing baseFeeUpdateFraction fallback

* Fix and add test
2025-01-27 16:20:39 +00:00
Advaita Saha
c16fc30358
Fix engine api misconceptions (#3025)
* fix shanghai misconception

* incorporate spec changes

* shanghai only check

* ForkchoiceUpdatedV2 after Cancun with beacon root field must return INVALID_PAYLOAD_ATTRIBUTES

* better exception positioning

* beter error msg
2025-01-27 19:07:03 +05:30
kdeme
c24335ce29
Bump nph formatter in CI to v0.6.1 (#3024) 2025-01-27 13:07:34 +01:00
kdeme
de80acdf79
Bump portal-spec-test (test-vectors) to latest (#3023) 2025-01-27 10:54:44 +00:00
andri lim
c11f20f8b0
devnet-6: Update EIP-7702: update EXTCODE* opcodes to act on full 'delegation designator' (#3021) 2025-01-25 14:32:39 +00:00
tersec
49523c5d8f
rm unused pre-1559 gas limit update calculation (#3011)
* rm unused pre-1559 gas limit update calculation

* wiki links 404
2025-01-25 17:36:43 +07:00
Advaita Saha
8372736600
custom genesis config fallback case for blobs (#3020)
* clear re-org misconception

* fcuV3 allow shanghai head

* add fallback blobcount for custom network

* fix tests
2025-01-24 22:49:11 +00:00
kdeme
b917be54ab
Fix wrong Result error usage in eth_data_exporter (#3018) 2025-01-24 08:36:28 +01:00
kdeme
9a5fd34dac
Portal history: Add ephemeral content type (#3017)
This does not yet enable the offer/accept and storage of the
ephemeral headers as it requires database changes.
Follow-up PR for that functionality.
2025-01-23 21:45:51 +01:00
Advaita Saha
f973b86dd8
nrpc re-org handling (#2994)
* reorg handling

* formatting fix

* fix validation condition

* eip-7685 compatible

* more exception handling

* remove strict checking
2025-01-23 01:51:44 +01:00
Jordan Hrycaj
184af027dc
Beacon sync metrics managemnt update (#3016)
* Sync scheduler provides an independent `ticker` loop process

why:
  Can be used to update `metrics` and for debug logging. While an event
  driven solution would stall if there are no events at the moment (e.g.
  when the syncer hibernates, the `ticker` will run regardless.

* Use `runTicker()` loop interface alike for updating ticker

why:
  Not event driven anymore so it will not stall when the syncer
  hibernates.

* Re-implement logging ticker by running it within the `runTicker()` driver

why:
  Simplifies implementation

* Re-name metrics variable to better fit into the current naming schemes

* Fix copyright header
2025-01-22 10:12:50 +00:00
kdeme
4b893eb88e
Discard received data on uTP content stream timeout (#3014)
Discard the received data on uTP content stream read timeout.
Before the data was still added to the queue and being processed
and should normally fail in validation. However as we know not all
data got read it should not even move to the validation step.

Added however a FIN send after the timeout instead of the delayed
socket clean-up which does not make much sense in that scenario
either. Basically either be nice and still send a FIN or just
destroy the socket immediatly.
2025-01-21 17:44:02 +01:00
andri lim
c62024ea24
devnet-5: Add EIP-7623: Increase calldata cost (#3015)
* devnet5: Add EIP-7623: Increase calldata cost

* Fix copyright year
2025-01-21 11:26:23 +00:00
kdeme
234487a8cb
Portal JSON-RPC: add eth_blockNumber and fix eth_getBlockByNumber (#3013)
- add eth_blockNumber
- fix eth_getBlockByNumber: latest vs safe fix.
2025-01-21 09:44:13 +01:00
bhartnett
285b6de4bc
Fluffy: Cleanup unused code from PortalProtocol (#3006) 2025-01-20 20:57:18 +08:00
Jordan Hrycaj
3aae33d6cc
Beacon sync maintenance update (#3012)
* Force metrics update when peers vanish

why:
  After that there might be reduced activity so that the next metrics
  update is delayed.

* Update comments (code cosmetics)

* Tidy up nano-sleep wait directives to an `update.nim`-function

* Fix copyright year
2025-01-20 10:30:04 +00:00
bhartnett
5b74335977
Update nim-rocksdb to v9.8.4.0 (#2997) 2025-01-17 18:23:24 +01:00
Advaita Saha
bf57c8e0e4
fix copyright lint failure (#3008) 2025-01-17 16:29:26 +05:30
kdeme
fa147f90ed
Portal JSON-RPC: minor clean-up (#3000)
Minor cleanup/refactor + add a basic TraceObject when returning
locally found content.
2025-01-17 11:57:15 +01:00
andri lim
aa85d6a61c
Only hashing immutable part of network configuration for dataDir ID (#2955)
* Only hashing immutable part of network configuration for dataDir ID

* Add debug- prefix to rewrite-datadir-id
2025-01-17 15:33:15 +05:30
andri lim
ac053bf4c8
devnet-5: Add EIP-7840: Add blob schedule to EL config files (#3005)
* devnet-5: Add EIP-7840: Add blob schedule to EL config files

* Fix test_transaction_json

* Add missing blobSchedule to common/helpers
2025-01-16 05:10:52 +00:00
bhartnett
19ea82bf50
Fluffy: State bridge clean shutdown (#2996)
* Implement clean shutdown for state bridge.

* Handle shutdown in history and beacon bridges.
2025-01-16 08:48:10 +08:00
andri lim
232a9ad247
devnet-5: Update EIP-7702: Remove delegation behavior of EXTCODE* (#2871) 2025-01-15 14:30:26 +00:00
andri lim
5c02b88f82
devnet-5: Update EIP-2935: bring up to date with sys contract impl (#3003)
* devnet-5: Update EIP-2935: bring up to date with sys contract impl

* Fix copyright year
2025-01-15 02:42:20 +00:00
andri lim
265d794583
devnet-5: Move EIP-7702 Authorization validation to authority func (#2999)
* Move EIP-7702 Authorization validation to authority func

If the authorization is invalid the transaction itself is still valid,
the invalid authorization will be skipped.

* Fix copyright year
2025-01-15 02:18:25 +00:00
andri lim
288ee28077
Integrate PoS payload attributes into txPool (#2998)
* Integrate PoS payload attributes into txPool

* Fix test_txpool
2025-01-14 13:30:56 +00:00
bhartnett
6e83a48969
Fluffy: Refactor and organize state bridge code (#2992) 2025-01-13 16:49:35 +08:00
tersec
67a45b0a7f
rm references to unused nimbus-eth1-blobs, unused imports (#2993) 2025-01-13 09:45:12 +07:00
Siddarth Kumar
419e0193e9
blocks-import-script: swap --csv with --markdown (#2985)
This PR adds `--markdown-output` support to blocks import script.
This PR also removes `--csv-output` flag.
2025-01-10 14:17:30 +05:30
bhartnett
58d447564a
Fluffy: Improve testnet script parameters (#2989) 2025-01-10 08:48:49 +08:00
bhartnett
ec41a8d6fe
Fluffy: Add additional cli parameters for configuring the WebSocket server (#2988)
* Add additional rpc parameters for configuring the WebSocket server.
2025-01-10 08:37:44 +08:00
Advaita Saha
23fa3b8e1e
Logging Fixes for INFO and DEBUG level (#2984)
* logs summarized

* fix copyright year

* add topics for logs

* fix copyright year

* bring syncer logs to info & debug level

* fix debug dockerfile

* fix: copyright error

* shift txpool logs to debug and introduce logs in rpc

* after header bring block download to info level

* comments for finalization summary of logs

* change literals to meaningful names

* remove unwanted data from userfacing logs

* include target logs

* remove control

* fix capitalization

* complete txpool
2025-01-09 12:33:29 +05:30
bhartnett
a5194846af
Fluffy: Support gossiping content to multiple portal client endpoints in the state bridge (#2987) 2025-01-09 11:42:16 +08:00
bhartnett
13a58abf5c
Fluffy: Improve state bridge gossip retry mechanism and logging (#2986)
- Minor refactor and cleanup of gossip retry and logging.

- Wait time before verifying the gossip for a block is now proportional to the number of offers per block.

- Don't retry gossipping content after finding it in the network. When retrying gossip of a block, only the offers not yet found in the network will be re-sent.
2025-01-08 15:19:37 +08:00
andri lim
0ffc17d153
Merge test_txpool and test_txpool2 (#2983)
* Merge test_txpool and test_txpool2

* Fix copyright year
2025-01-05 08:05:29 +00:00
Siddarth Kumar
7672f0b7c0
blocks-import-script: add --csv-output flag (#2982)
This commit adds --csv-output support to blocks import script.

Github renders CSVs as tables and this addition would be useful in nimbus-eth1-benchmark repo to make this comparison easy to render.
2025-01-05 00:56:00 +05:30
bhartnett
e33ab7d468
Fluffy: Minor refactor, cleanup and improve variable names in JSON-RPC handlers (#2981) 2025-01-03 15:24:12 +08:00
bhartnett
7c347e1a2a
Fluffy: Update portal_*PutContent JSON-RPC endpoints to return metadata (#2980)
* Update portal_*PutContent JSON-RPC endpoints to return metadata as defined in the portal specs.
2025-01-02 16:28:12 +08:00