Commit Graph

3907 Commits

Author SHA1 Message Date
Jordan Hrycaj 0f430c70fd
Aristo avoid storage trie update race conditions (#2251)
* Update TDD suite logger output format choices

why:
  New format is not practical for TDD as it just dumps data across a wide
  range (considerably larder than 80 columns.)

  So the new format can be turned on by function argument.

* Update unit tests samples configuration

why:
  Slightly changed the way to find the `era1` directory

* Remove compiler warnings (fix deprecated expressions and phrases)

* Update `Aristo` debugging tools

* Always update the `storageID` field of account leaf vertices

why:
  Storage tries are weekly linked to an account leaf object in that
  the `storageID` field is updated by the application.

  Previously, `Aristo` verified that leaf objects make sense when passed
  to the database. As a consequence
  * the database was inconsistent for a short while
  * the burden for correctness was all on the application which led
    to delayed error handling which is hard to debug.

  So `Aristo` will internally update the account leaf objects so that
  there are no race conditions due to the storage trie handling

* Aristo: Let `stow()`/`persist()` bail out unless there is a `VertexID(1)`

why:
  The journal and filter logic depends on the hash of the `VertexID(1)`
  which is commonly known as the state root. This implies that all
  changes to the database are somehow related to that.

* Make sure that a `Ledger` account does not overwrite the storage trie reference

why:
  Due to the abstraction of a sub-trie (now referred to as column with a
  hash describing its state) there was a weakness in the `Aristo` handler
  where an account leaf could be overwritten though changing the validity
  of the database. This has been changed and the database will now reject
  such changes.

  This patch fixes the behaviour on the application layer. In particular,
  the column handle returned by the `CoreDb` needs to be updated by
  the `Aristo` database state. This mitigates the problem that a storage
  trie might have vanished or re-apperaed with a different vertex ID.

* Fix sub-trie deletion test

why:
  Was originally hinged on `VertexID(1)` which cannot be wholesale
  deleted anymore after the last Aristo update. Also, running with
  `VertexID(2)` needs an artificial `VertexID(1)` for making `stow()`
  or `persist()` work.

* Cosmetics

* Activate `test_generalstate_json`

* Temporarily `deactivate test_tracer_json`

* Fix copyright header

---------

Co-authored-by: jordan <jordan@dry.pudding>
Co-authored-by: Jacek Sieka <jacek@status.im>
2024-05-30 17:48:38 +00:00
Kim De Mey d814d84b9b
Use --styleCheck:error for Fluffy + fixes (#2247)
* Use --styleCheck:error for Fluffy + fixes

There seems to be a clash between the names of an object field and
a proc here. As workaround names of the procs are changed.

* Fix case style for rpc client calls in test_portal_testnet

* Fix style case for utp test
2024-05-30 18:12:28 +02:00
Jacek Sieka 919242c98e
results: use canonical import (#2248) 2024-05-30 14:54:03 +02:00
andri lim 198e9c6c98
Remove more goerli data (#2249) 2024-05-30 11:49:47 +00:00
andri lim 27ea571c8f
Enable test_txpool2 independetly from test_txpool (#2246) 2024-05-30 12:11:41 +02:00
andri lim 0a07425112
Cleanup unused raises in evm/state and other obsolete informations (#2243) 2024-05-30 09:03:54 +00:00
tersec 6a9ca40cc8
use --styleCheck:error (#2242)
* use --styleCheck:error

* rename evm/types.nim statusCode to evmStatus
2024-05-30 09:01:07 +00:00
web3-developer 1eb170e686
Fluffy state network tests (#2245)
* Add state network get content test.

* Completed state network get content tests.

* Completed state gossip offer content test.

* Improve state network offer content test.

* Completed state gossip offer test.

* Rename tests.
2024-05-30 17:00:57 +08:00
andri lim 4b8219a0dd
Enable test_state_db (#2244)
* Make test_state_db compilable

* Enable test_state_db
2024-05-30 07:44:52 +02:00
tersec 674394b924
fix import path; force refc memory management even with Nim 2.0+ (#2241) 2024-05-29 20:47:06 +02:00
tersec c466edfd8d
change an Aristo function name to avoid Nim stdlib ambiguity (#2240) 2024-05-29 15:08:00 +00:00
andri lim eaf3d9897e
Simplify AccountsLedgerRef complexity (#2239) 2024-05-29 13:06:49 +02:00
Jacek Sieka 9d9aa738db
general state test: don't swallow asserts 2024-05-29 11:28:01 +02:00
andri lim 74cc3b6127
Implement engine_getClientVersionV1 (#2233)
* Implement engine_getClientVersionV1

* full git revision string

* Limit GitRevisionString to 8 chars

* Fixes

* Debug windows CI

* debug windows ci

* produce git revision using -C

* try not to delete .git folder in windows ci

* Harden GitRevision procuration

* Add double quotes to git -C param

* Escape sourcePath

* Remove double quotes from git -C param
2024-05-29 09:20:50 +02:00
andri lim abf1e58ed4
Remove eth/66 and eth/67 wire protocol (#2238) 2024-05-29 09:20:25 +02:00
andri lim 5ca2bf80b0
Remove legacy sync (#2237) 2024-05-29 06:58:33 +02:00
tersec 709200f62a
remove expicit PoW support from tx pool; tighten tx pool exceptions specs (#2235) 2024-05-28 20:26:51 +02:00
Jordan Hrycaj 3a62250d04
Make test op memory work again (#2236)
* Remove crufty `pruneTrie` arguments

* Replaced legacy `distinct_trie` logic by new `ledger` functionality

why:
  The module `distinct_trie` is supported by `Aristo` in trivial cases.

* Activate `test_op_memory`
2024-05-28 14:24:10 +00:00
Jacek Sieka 08e98eb385
restore a few tests, cleanup (#2234)
* remove `compensateLegacySetup`, `localDbOnly`
* enable trivially fixable tests
2024-05-28 14:49:35 +02:00
Etan Kissling 741fcca9b6
Workaround for disallowed transaction superseding (#2181)
* Workaround for disallowed transaction superseding

The transaction spammer from Kurtosis keeps spamming transactions with
the same nonce because report 'pending' account nocne based on 'latest'
rather than actually considering the mempool. To avoid errors from
rejecting those, disable the required gas price bump when replacing
a transaction with a new nonce.

* Lint

* Skip superseding negative test
2024-05-28 13:38:57 +07:00
Etan Kissling b7a7745e24
Workaround for 0 gas price estimation (#2180)
* Workaround for 0 gas price estimation

The transaction spammer from Kurtosis estimates the gas price of its
transactions using 'eth_gasPrice'. Our implementation returns 0 when
no transactions have been executed yet, not taking into account the
EIP-1559 block base fee. Force a hardcoded minimum for now to unstuck.

* Adjust tests to cover new minimum gas fee

* Skip gas price test
2024-05-28 13:36:38 +07:00
tersec ca60b13e6a
rm clique/mining remnants; rm unused code (#2232) 2024-05-28 07:10:10 +02:00
web3-developer 9354cb8411
Fluffy recursive gossip improvements, fixes and tests. (#2231)
* Refactor get parent gossip code and add Nibbles helper function.

* Add logging to state gossip.

* Unit test recursive gossip using state gossip getParent functions.

* Add recursive gossip genesis json test and fix bug in state gossip getParent.

* Add Nibbles len function.
2024-05-28 08:45:30 +08:00
tersec f932c8df22
rm some proof of work mining support (#2227) 2024-05-27 21:49:07 +02:00
andri lim eb50dcc2f9
Noop in AccountsLedger.getCode if the codehash is EMPTY_CODE_HASH (#2229) 2024-05-27 10:26:26 +02:00
andri lim 5db6ee07fb
Fix accounts_ledger savePoint conversion (#2226) 2024-05-26 17:42:02 +07:00
Jacek Sieka a649e633ae
eth: bump (#2225)
* new rlp code
* binary sparse tries removed
2024-05-26 10:56:17 +02:00
andri lim 9c3dfc93a7
Bump nim-web3 to 46b4b4c8bc42013196a6290ee53029131f97931a (#2224)
* Bump nim-web3 to 46b4b4c8bc42013196a6290ee53029131f97931a

* Fix beacon_lc_bridge and nimbus_verified_proxy
2024-05-26 14:13:59 +07:00
Jacek Sieka 9c3de888a4
era: simplify, instant startup (#2218)
This PR exploits structural properties of era files to simplify the
implementation and in particular remove the need to load all era file
indicies at startup which may be slow (due to archival storage residing
on slow drives)
2024-05-26 08:24:13 +02:00
tersec 75ff54fef3
rm --terminal-total-difficulty command-line option (#2221) 2024-05-26 08:15:18 +02:00
andri lim e6d5a791f5
Bump nim-stint to 9d2b382c5dc34f0d6bbd93b2a5d65dde85067e0f (#2223) 2024-05-26 10:48:59 +07:00
andri lim d4866d5003
Bump nim-json-serialization to 4d0b0662ed960ab2c5a1ddbd08f77048bac13ae7 (#2222) 2024-05-26 10:48:45 +07:00
tersec 34ac68990f
fix warnings around unused imports of std/algorithm; proc -> func (#2220) 2024-05-25 21:01:28 +02:00
tersec e895c0baeb
rm Clique consensus method support and Goerli network (#2219)
* rm Clique consensus method support and Goerli network

* rm a few more SealingEngineRef and GoerliNets
2024-05-25 16:12:14 +02:00
Kim De Mey 72912626a2
Several spec fixes for HistoricalSummaries + add test-vector test (#2217) 2024-05-24 23:15:04 +02:00
andri lim 1d4eb78bcf
Disable t8n log output (#2216) 2024-05-24 22:48:57 +07:00
Jacek Sieka 0a49833d69
avoid a few more copies (#2215) 2024-05-24 11:27:17 +02:00
web3-developer 771d6fd3e8
Implementation of Fluffy state network gossip. (#2210)
* Add toContentKey helper functions.

* Implement typed lookup functions for getting account and storage trie nodes and bytecode.

* Re-implementation of recursive gossip.

* Offer validation improvements. Don't allow key end in Nibbles for leafs as defined in the spec.

* Minor refactor to validation and add addition bytecode check.
2024-05-24 15:49:51 +08:00
Jacek Sieka f38c5e631e
trivial memory-based speedups (#2205)
* trivial memory-based speedups

* HashKey becomes non-ref
* use openArray instead of seq in lots of places
* avoid sequtils.reversed when unnecessary
* add basic perf stats to test_coredb

* copyright
2024-05-23 17:37:51 +02:00
Jacek Sieka fe296213cf
les: remove (#2211)
LES was [removed from
geth](https://github.com/ethereum/go-ethereum/pull/28586) and is
generally no longer supported on the network.
2024-05-23 16:53:51 +02:00
Jordan Hrycaj 2c322054e0
Attempt to roll back stateless mode implementation in a single PR (#2209)
* Attempt to roll back stateless mode implementation in a single PR

why:
+ Stateless mode is not fully working and in the way
+ Single PR should make it feasible to investigate for a possible
  re-implementation

* Fix copyright year

* Fix annotation for exception (evmc mode)
2024-05-22 21:01:19 +00:00
Kim De Mey 95089f103a
Bump portal-spec-tests for additional header validation tests (#2208) 2024-05-22 19:23:18 +02:00
Kim De Mey ba2dd76f30
Rename the Portal testnet0 to mainnet (#2204)
Also adds the new testnet as option, but this needs to be still
implemented by adding testnet bootstrap nodes and protocol ids.
2024-05-22 17:10:31 +02:00
Jordan Hrycaj 9cc6e5a3aa
Aristo resume off line syncing on pre loaded database (#2203)
* Update some docu & messages

* Remove cruft from the ledger modules

* Must not overwrite genesis data on an initialised database

why:
  This will overwrite the global state of the Aristo single state DB.
  Otherwise resuming at the last synced state becomes impossible.

* Provide latest block number from journal

why:
  This relates the global state of the DB directly to the corresponding
  block number.

* Implemented unit test providing DB pre-load and resume
2024-05-22 13:41:14 +00:00
Jacek Sieka 308bd51bbd
era: speed up index reading (#2202) 2024-05-22 09:27:23 +02:00
web3-developer e566e89d03
Fluffy state network refactor (#2200)
* Add additional getTrieProof test.

* Refactor state network: Removed variant objects for values and split state_content into multiple files.
2024-05-22 09:22:07 +08:00
Jordan Hrycaj 2629d412d7
CoreDb+Aristo: Update tracer (#2201)
why:
  When deleting accounts while restoring the previous state, storage tries
  must be deleted first. Otherwise a `DelDanglingStoTrie` error will occur
  when trying to delete an account which refers to an active storage trie.
2024-05-21 12:51:06 +00:00
Jordan Hrycaj bc8e3c02ee
Fix missing disabling of `eth` service (#2199)
why:
  Otherwise it might crash on a not fully initialised call back function
  when most other functions are disabled during the early sync state.
2024-05-20 15:02:36 +00:00
Jordan Hrycaj de0388919f
Unified mode for undumping gzip-ed or era1-ed encoded block dumps (#2198)
ackn:
  Built on Daniel's work
2024-05-20 13:59:18 +00:00
Jordan Hrycaj ee9aea171d
Culling legacy DB and accounts cache (#2197)
details:
+ Compiles nimbus all_tests
+ Failing tests have been commented out
2024-05-20 10:17:51 +00:00