Commit Graph

1711 Commits

Author SHA1 Message Date
Jacek Sieka 9f879406f3
append instead of reallocate in blobify (#2277)
...otherwise, we get lots and lots of temporary allocations of seq's
2024-06-01 17:13:24 +02:00
andri lim f9765e617b
Early exit from some of CoreDbRef functions if nothing to do (#2259)
* Early exit from some of CoreDbRef functions if nothing to do

* More exits

* persistReceipts early exit if nothing to do
2024-06-01 16:54:02 +02:00
andri lim 1565c57ae6
Fix engine_forkchoiceUpdated bug (#2274)
* fCU Ignore update to old head

* Only enable terminal PoW block conditions for fCUV1

* Typo fix: TDD -> TTD

* Add link to Shanghai fCUV2 specification

* Add link to Paris fCUV1 specification
2024-06-01 13:26:12 +00:00
andri lim 0f0ac0bc14
Only produce debugging metadata when nimbusDumpDebuggingMetaData is defined (#2275) 2024-06-01 12:34:28 +00:00
tersec cfbbcda4f7
rm unused Nim modules (#2270) 2024-06-01 17:49:46 +07:00
Jordan Hrycaj bda760f41d
Run coredb without journal (#2266)
* Add persistent last state stamp feature

why:
  This allows to run `CoreDb` without journal

* Start `CoreDb` without journal

* Remove journal related functions from `CoredDb`
2024-05-31 17:32:22 +00:00
tersec b72ebca2db
remove snap sync from git working tree (#2265) 2024-05-31 19:14:50 +02:00
Jordan Hrycaj 483b2d8ef4
Fix block-number last saved block state (#2263)
why:
  Typo. That worked the wrong way in the unit tests because the
  tests always store the last item separately with extended
  logging.

Co-authored-by: jordan <jordan@dry.pudding>
2024-05-31 09:43:31 +00:00
andri lim c18aa8e6f7
Move --trusted-setup-file under ETHEREUM OPTIONS section instead under noCmd (#2257) 2024-05-31 09:36:25 +02:00
Jacek Sieka a375720c16
import: read from era files (#2254)
This PR extends the `nimbus import` command to also allow reading from
era files - this command allows creating or topping up an existing
database with data coming from era files instead of network sync.

* add `--era1-dir` and `--max-blocks` options to command line
* make `persistBlocks` report basic stats like transactions and gas
* improve error reporting in several API
* allow importing multiple RLP files in one go
* clean up logging options to match nimbus-eth2
* make sure database is closed properly on shutdown
2024-05-31 09:13:56 +02:00
tersec 01aeb3ad97
fix Dencun fcU being always rejected as invalid; rm newPayload fork validation hack (#2253) 2024-05-30 22:30:40 +02:00
tersec 828fd63348
rm Miracl and remaining i386 (32-bit) build support (#2250) 2024-05-30 21:08:09 +02:00
Jordan Hrycaj 0f430c70fd
Aristo avoid storage trie update race conditions (#2251)
* Update TDD suite logger output format choices

why:
  New format is not practical for TDD as it just dumps data across a wide
  range (considerably larder than 80 columns.)

  So the new format can be turned on by function argument.

* Update unit tests samples configuration

why:
  Slightly changed the way to find the `era1` directory

* Remove compiler warnings (fix deprecated expressions and phrases)

* Update `Aristo` debugging tools

* Always update the `storageID` field of account leaf vertices

why:
  Storage tries are weekly linked to an account leaf object in that
  the `storageID` field is updated by the application.

  Previously, `Aristo` verified that leaf objects make sense when passed
  to the database. As a consequence
  * the database was inconsistent for a short while
  * the burden for correctness was all on the application which led
    to delayed error handling which is hard to debug.

  So `Aristo` will internally update the account leaf objects so that
  there are no race conditions due to the storage trie handling

* Aristo: Let `stow()`/`persist()` bail out unless there is a `VertexID(1)`

why:
  The journal and filter logic depends on the hash of the `VertexID(1)`
  which is commonly known as the state root. This implies that all
  changes to the database are somehow related to that.

* Make sure that a `Ledger` account does not overwrite the storage trie reference

why:
  Due to the abstraction of a sub-trie (now referred to as column with a
  hash describing its state) there was a weakness in the `Aristo` handler
  where an account leaf could be overwritten though changing the validity
  of the database. This has been changed and the database will now reject
  such changes.

  This patch fixes the behaviour on the application layer. In particular,
  the column handle returned by the `CoreDb` needs to be updated by
  the `Aristo` database state. This mitigates the problem that a storage
  trie might have vanished or re-apperaed with a different vertex ID.

* Fix sub-trie deletion test

why:
  Was originally hinged on `VertexID(1)` which cannot be wholesale
  deleted anymore after the last Aristo update. Also, running with
  `VertexID(2)` needs an artificial `VertexID(1)` for making `stow()`
  or `persist()` work.

* Cosmetics

* Activate `test_generalstate_json`

* Temporarily `deactivate test_tracer_json`

* Fix copyright header

---------

Co-authored-by: jordan <jordan@dry.pudding>
Co-authored-by: Jacek Sieka <jacek@status.im>
2024-05-30 17:48:38 +00:00
Jacek Sieka 919242c98e
results: use canonical import (#2248) 2024-05-30 14:54:03 +02:00
andri lim 198e9c6c98
Remove more goerli data (#2249) 2024-05-30 11:49:47 +00:00
andri lim 0a07425112
Cleanup unused raises in evm/state and other obsolete informations (#2243) 2024-05-30 09:03:54 +00:00
tersec 6a9ca40cc8
use --styleCheck:error (#2242)
* use --styleCheck:error

* rename evm/types.nim statusCode to evmStatus
2024-05-30 09:01:07 +00:00
tersec 674394b924
fix import path; force refc memory management even with Nim 2.0+ (#2241) 2024-05-29 20:47:06 +02:00
tersec c466edfd8d
change an Aristo function name to avoid Nim stdlib ambiguity (#2240) 2024-05-29 15:08:00 +00:00
andri lim eaf3d9897e
Simplify AccountsLedgerRef complexity (#2239) 2024-05-29 13:06:49 +02:00
andri lim 74cc3b6127
Implement engine_getClientVersionV1 (#2233)
* Implement engine_getClientVersionV1

* full git revision string

* Limit GitRevisionString to 8 chars

* Fixes

* Debug windows CI

* debug windows ci

* produce git revision using -C

* try not to delete .git folder in windows ci

* Harden GitRevision procuration

* Add double quotes to git -C param

* Escape sourcePath

* Remove double quotes from git -C param
2024-05-29 09:20:50 +02:00
andri lim abf1e58ed4
Remove eth/66 and eth/67 wire protocol (#2238) 2024-05-29 09:20:25 +02:00
andri lim 5ca2bf80b0
Remove legacy sync (#2237) 2024-05-29 06:58:33 +02:00
tersec 709200f62a
remove expicit PoW support from tx pool; tighten tx pool exceptions specs (#2235) 2024-05-28 20:26:51 +02:00
Jordan Hrycaj 3a62250d04
Make test op memory work again (#2236)
* Remove crufty `pruneTrie` arguments

* Replaced legacy `distinct_trie` logic by new `ledger` functionality

why:
  The module `distinct_trie` is supported by `Aristo` in trivial cases.

* Activate `test_op_memory`
2024-05-28 14:24:10 +00:00
Jacek Sieka 08e98eb385
restore a few tests, cleanup (#2234)
* remove `compensateLegacySetup`, `localDbOnly`
* enable trivially fixable tests
2024-05-28 14:49:35 +02:00
Etan Kissling 741fcca9b6
Workaround for disallowed transaction superseding (#2181)
* Workaround for disallowed transaction superseding

The transaction spammer from Kurtosis keeps spamming transactions with
the same nonce because report 'pending' account nocne based on 'latest'
rather than actually considering the mempool. To avoid errors from
rejecting those, disable the required gas price bump when replacing
a transaction with a new nonce.

* Lint

* Skip superseding negative test
2024-05-28 13:38:57 +07:00
Etan Kissling b7a7745e24
Workaround for 0 gas price estimation (#2180)
* Workaround for 0 gas price estimation

The transaction spammer from Kurtosis estimates the gas price of its
transactions using 'eth_gasPrice'. Our implementation returns 0 when
no transactions have been executed yet, not taking into account the
EIP-1559 block base fee. Force a hardcoded minimum for now to unstuck.

* Adjust tests to cover new minimum gas fee

* Skip gas price test
2024-05-28 13:36:38 +07:00
tersec ca60b13e6a
rm clique/mining remnants; rm unused code (#2232) 2024-05-28 07:10:10 +02:00
tersec f932c8df22
rm some proof of work mining support (#2227) 2024-05-27 21:49:07 +02:00
andri lim eb50dcc2f9
Noop in AccountsLedger.getCode if the codehash is EMPTY_CODE_HASH (#2229) 2024-05-27 10:26:26 +02:00
andri lim 5db6ee07fb
Fix accounts_ledger savePoint conversion (#2226) 2024-05-26 17:42:02 +07:00
Jacek Sieka 9c3de888a4
era: simplify, instant startup (#2218)
This PR exploits structural properties of era files to simplify the
implementation and in particular remove the need to load all era file
indicies at startup which may be slow (due to archival storage residing
on slow drives)
2024-05-26 08:24:13 +02:00
tersec 75ff54fef3
rm --terminal-total-difficulty command-line option (#2221) 2024-05-26 08:15:18 +02:00
tersec 34ac68990f
fix warnings around unused imports of std/algorithm; proc -> func (#2220) 2024-05-25 21:01:28 +02:00
tersec e895c0baeb
rm Clique consensus method support and Goerli network (#2219)
* rm Clique consensus method support and Goerli network

* rm a few more SealingEngineRef and GoerliNets
2024-05-25 16:12:14 +02:00
Jacek Sieka 0a49833d69
avoid a few more copies (#2215) 2024-05-24 11:27:17 +02:00
Jacek Sieka f38c5e631e
trivial memory-based speedups (#2205)
* trivial memory-based speedups

* HashKey becomes non-ref
* use openArray instead of seq in lots of places
* avoid sequtils.reversed when unnecessary
* add basic perf stats to test_coredb

* copyright
2024-05-23 17:37:51 +02:00
Jacek Sieka fe296213cf
les: remove (#2211)
LES was [removed from
geth](https://github.com/ethereum/go-ethereum/pull/28586) and is
generally no longer supported on the network.
2024-05-23 16:53:51 +02:00
Jordan Hrycaj 2c322054e0
Attempt to roll back stateless mode implementation in a single PR (#2209)
* Attempt to roll back stateless mode implementation in a single PR

why:
+ Stateless mode is not fully working and in the way
+ Single PR should make it feasible to investigate for a possible
  re-implementation

* Fix copyright year

* Fix annotation for exception (evmc mode)
2024-05-22 21:01:19 +00:00
Jordan Hrycaj 9cc6e5a3aa
Aristo resume off line syncing on pre loaded database (#2203)
* Update some docu & messages

* Remove cruft from the ledger modules

* Must not overwrite genesis data on an initialised database

why:
  This will overwrite the global state of the Aristo single state DB.
  Otherwise resuming at the last synced state becomes impossible.

* Provide latest block number from journal

why:
  This relates the global state of the DB directly to the corresponding
  block number.

* Implemented unit test providing DB pre-load and resume
2024-05-22 13:41:14 +00:00
Jordan Hrycaj 2629d412d7
CoreDb+Aristo: Update tracer (#2201)
why:
  When deleting accounts while restoring the previous state, storage tries
  must be deleted first. Otherwise a `DelDanglingStoTrie` error will occur
  when trying to delete an account which refers to an active storage trie.
2024-05-21 12:51:06 +00:00
Jordan Hrycaj bc8e3c02ee
Fix missing disabling of `eth` service (#2199)
why:
  Otherwise it might crash on a not fully initialised call back function
  when most other functions are disabled during the early sync state.
2024-05-20 15:02:36 +00:00
Jordan Hrycaj de0388919f
Unified mode for undumping gzip-ed or era1-ed encoded block dumps (#2198)
ackn:
  Built on Daniel's work
2024-05-20 13:59:18 +00:00
Jordan Hrycaj ee9aea171d
Culling legacy DB and accounts cache (#2197)
details:
+ Compiles nimbus all_tests
+ Failing tests have been commented out
2024-05-20 10:17:51 +00:00
andri lim 38eaebc5c7
TxPool: rank calculator now include profit from blob fee (#2196)
* TxPool: rank calculator now include profit from blob fee

* Fix copyright year
2024-05-20 14:30:48 +07:00
andri lim 33ac53217f
Check invalid ancestor in engine-api and beacon-sync (#2192) 2024-05-17 08:38:46 +07:00
Jacek Sieka 293ce28e4d
remove geth_db (#2189)
doesn't look like it's used
2024-05-16 22:00:20 +07:00
andri lim 8767bbd10a
Fix engine simulator and improve logging (#2188)
* Fix engine simulator and improve logging

* Fix engine simulator genesis loader
2024-05-15 23:22:03 +07:00
Etan Kissling c4c37302b1
Introduce wrapper type for EIP-4844 transactions (#2177)
* Introduce wrapper type for EIP-4844 transactions

EIP-4844 blob sidecars are a concept that only exists in the mempool.
After inclusion of a transaction into an execution block, only the
versioned hash within the transaction remains. To improve type safety,
replace the `Transaction.networkPayload` member with a wrapper type
`PooledTransaction` that is used in contexts where blob sidecars exist.

* Bump nimbus-eth2 to 87605d08a7f9cfc3b223bd32143e93a6cdf351ac

* IPv6 'listen-address' in `nimbus_verified_proxy`

* Bump nim-libp2p to 21cbe3a91a70811522554e89e6a791172cebfef2

* Fix beacon_lc_bridge payload conversion and conf.listenAddress type

* Change nimbus_verified_proxy.asExecutionData param to SomeExecutionPayload

* Rerun nph to fix asExecutionData style format

* nimbus_verified_proxy listenAddress

* Use PooledTransaction in nimbus-eth1 tests

---------

Co-authored-by: jangko <jangko128@gmail.com>
2024-05-15 10:07:59 +07:00