Commit Graph

77 Commits

Author SHA1 Message Date
andri lim 99ff8dc876
Fix t8n: blobGasUsed exceeds allowance issue (#2407)
* Fix t8n: blobGasUsed exceeds allowance issue

* Put blobGasUsed validation into transaction precessing pipeline
2024-06-24 07:56:24 +02:00
Jacek Sieka 768307d91d
Cache code and invalid jump destination tables (fixes #2268) (#2404)
It is common for many accounts to share the same code - at the database
level, code is stored by hash meaning only one copy exists per unique
program but when loaded in memory, a copy is made for each account.

Further, every time we execute the code, it must be scanned for invalid
jump destinations which slows down EVM exeuction.

Finally, the extcodesize call causes code to be loaded even if only the
size is needed.

This PR improves on all these points by introducing a shared
CodeBytesRef type whose code section is immutable and that can be shared
between accounts. Further, a dedicated `len` API call is added so that
the EXTCODESIZE opcode can operate without polluting the GC and code
cache, for cases where only the size is requested - rocksdb will in this
case cache the code itself in the row cache meaning that lookup of the
code itself remains fast when length is asked for first.

With 16k code entries, there's a 90% hit rate which goes up to 99%
during the 2.3M attack - the cache significantly lowers memory
consumption and execution time not only during this event but across the
board.
2024-06-21 09:44:10 +02:00
andri lim 83f6f89869
Add t8n debugging tool and fix EVM regression (#2386)
- fix blockNumber overflow in blockHash op code
- reenable 3 test cases of test_blockchain_json
- fix t8n crash when creating invalid tracer stream
2024-06-19 08:58:08 +07:00
andri lim 61a809cf4d
Remove EVM indirect imports and unused EVM errors (#2370)
Those indirect imports are used when there was two EVMs.
2024-06-17 09:56:39 +02:00
andri lim 5a18537450
Bump nim-eth, nim-web3, nimbus-eth2 (#2344)
* Bump nim-eth, nim-web3, nimbus-eth2

- Replace std.Option with results.Opt
- Fields name changes

* More fixes

* Fix Portal stream async raises and portal testnet Opt usage

* Bump eth + nimbus-eth2 + more fixes related to eth_types changes

* Fix in utp test app and nimbus-eth2 bump

* Fix test_blockchain_json rebase conflict

* Fix EVMC block_timestamp conversion plus commentary

---------

Co-authored-by: kdeme <kim.demey@gmail.com>
2024-06-14 14:31:08 +07:00
andri lim b3a5c67532
Remove exceptions from EVM (#2314)
* Remove exception from evm memory

* Remove exception from gas meter

* Remove exception from stack

* Remove exception from precompiles

* Remove exception from gas_costs

* Remove exception from op handlers

* Remove exception from op dispatcher

* Remove exception from call_evm

* Remove exception from EVM

* Fix tools and tests

* Remove exception from EVMC

* fix evmc

* Fix evmc

* Remove remnants of async evm stuff

* Remove superflous error handling

* Proc to func

* Fix errors detected by CI

* Fix EVM op call stack usage

* REmove exception handling from getVmState

* Better error message instead of just doAssert

* Remove unused validation

* Remove superflous catchRaise

* Use results.expect instead of unsafeValue
2024-06-07 15:24:32 +07:00
Jacek Sieka 7f76586214
Speed up account ledger a little (#2279)
`persist` is a hotspot when processing blocks because it is run at least
once per transaction and loops over the entire account cache every time.

Here, we introduce an extra `dirty` map that keeps track of all accounts
that need checking during `persist` which fixes the immediate
inefficiency, though probably this could benefit from a more thorough
review - we also get rid of the unused clearCache flag - we start with
a fresh cache on every fresh vmState.

* avoid unnecessary code hash comparisons
* avoid unnecessary copies when iterating
* use EMPTY_CODE_HASH throughout for code hash comparison
2024-06-02 21:21:29 +02:00
andri lim 324610ef05
Add test vector to evmstate (#2255) 2024-05-31 11:03:11 +07:00
Jacek Sieka 919242c98e
results: use canonical import (#2248) 2024-05-30 14:54:03 +02:00
andri lim 1d4eb78bcf
Disable t8n log output (#2216) 2024-05-24 22:48:57 +07:00
Jordan Hrycaj ee9aea171d
Culling legacy DB and accounts cache (#2197)
details:
+ Compiles nimbus all_tests
+ Failing tests have been commented out
2024-05-20 10:17:51 +00:00
andri lim d29c9fae17
Update t8n test vectors after EIP-7610 update (#2169)
* Skip call family gas cost overflow check if refund is negative

* Update t8n test vectors after EIP-7610 update
2024-05-08 09:27:37 +07:00
andri lim c41206be39
Fix styles and reduce compiler warnings (#2086)
* Fix styles and reduce compiler warnings

* Fix copyright year
2024-03-20 14:35:38 +07:00
andri lim d830692b72
Bump nim-evmc: add push raises: [] (#2051) 2024-02-24 09:38:50 +07:00
andri lim 3a4da2cdb7
Bump eth_tests to 5bf1fff257c0662bc526c27d7546d470025b3239 (#2047) 2024-02-21 23:04:44 +07:00
andri lim 7b4ef814ea
Bump eth_tests and related fixes (#2042)
* Bump eth_tests and related fixes

* Fix tests
2024-02-21 16:14:20 +07:00
Jordan Hrycaj b623909c44
Ledger activate unified accounts cache wrapper (#1939)
* Activate `LedgerRef` wrapper for `AccountsCache`

details:
  `accounts_cache.nim` methods are indirectly processed by the wrapper
  methods from `ledger.nim`.

  This works for all sources except `test_state_db.nim` where the source
  `accounts_cache.nim` is included (rather than imported) in order to
  access objects privy to the very source.

* Provide facility to switch to a preselected `LedgerRef` type

details:
  Can be set as suggestion when initialising `CommonRef`

* Update `CoreDb` test suite for better time tracking

details:
+ Allow time logging by pre-defined block intervals
+ Print `CoreDb`/`Ledger`profiling results (if enabled)
2023-12-12 19:12:56 +00:00
jangko b0000eed8b
Add check copyright year linter to CI 2023-11-01 10:41:20 +07:00
jangko 0ee448c1eb
Fix ForkID calculation 2023-10-25 11:10:01 +07:00
jangko 35ee171013
Fix t8n response cancun fields: rename currentBlobGasUsed to blobGasUsed 2023-10-24 21:22:49 +07:00
jangko 20db41878d
Unify dumpAccounts implementation for tools and debugging 2023-10-19 13:02:12 +07:00
andri lim 04c7ed8ec9
Change EthTime from std.Time to distinct uint64 (#1820)
* Change EthTime from std.Time to distinct uint64

* Bump nimbus-eth2
2023-10-18 09:16:11 +07:00
andri lim 12f494e669
Only one place to declare gwei (#1802)
* Only one place to declare gwei

* Fix weiAmount overflow
2023-10-04 10:47:18 +07:00
andri lim dc1dcfb206
EIP-4844: Fix excessBlobGas handling (#1771)
* EIP-4844: Fix excessBlobGas handling

* Change vmState.difficulty to vmState.difficultyOrPrevRandao
2023-09-24 22:25:41 +07:00
jangko 2d2def9a8d
Fix blobGas fields handling in evmstate 2023-09-22 07:42:58 +07:00
jangko 7ee862ab3f
Fix blobGas fields handling in t8n 2023-09-22 07:42:43 +07:00
jangko 47fae1bb71
Add EVM tracer test case for EIP-2929 opcodes 2023-08-31 09:49:51 +07:00
andri lim 96fb355efe
Implement EIP-4788: Beacon block root in the EVM (#1722)
* Implement EIP-4788: Beacon block root in the EVM

* EIP-4788: Fix genesis.parentBeaconBlockRoot initialization
2023-08-30 23:29:48 +07:00
jangko 0b0d478966
Allow EVM dispatcher to use different GasCostKind for each fork
Why?
Some opcodes such as labeled EIP-2929 changed their behavior from fixed
gas cost to dynamic gas cost.

This changes together with #1715 and #1717 will make the new EVM tracer
to produce trace result identical to geth.
2023-08-30 21:08:35 +07:00
jangko 80aec9ccd9
Fix EVM tracer: capture exception properly
Also fix EVM to support new tracer
2023-08-28 14:26:43 +07:00
andri lim 91704cd3ae
Add current computation to each EVM tracer interface (#1712) 2023-08-25 16:07:20 +07:00
jangko 849c4bc785
Fix EVM tracer producing wrong order of CALL family
Also fix t8n tool when given json txs with no v,r,s fields.
v,r,s field can be subtituted by "secretKey" field.
2023-08-23 17:15:34 +07:00
andri lim fd79c5c264
Fix EVM tracer crash bug when serializing nil stack (#1697)
* Fix EVM tracer crash bug when serializing nil stack

* Fix t8n tracer doc

add following description to reflect new functionality
  `stdout` - into the stdout output.
  `stderr` - into the stderr output.
  <file>   - into the file <file>-<txIndex>.jsonl.
  none     - output.basedir/trace-<txIndex>-<txhash>.jsonl.
2023-08-20 11:15:11 +07:00
andri lim bdaeedb09f
rename data gas to blob gas (#1659)
* rename data gas to blob gas

* bump more submodules
* extend evmc tx_context with EIP-4844 blob_hashes
2023-08-04 19:43:30 +07:00
Jordan Hrycaj 221e6c9e2f
Unified database frontend integration (#1670)
* Nimbus folder environment update

details:
* Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder.
* The `nimbus` program does not compile yet as it needs the updates
  in the parallel `stateless` sub-folder.

* Stateless environment update

details:
* Integrated `CoreDbRef` for the sources in the `stateless` sub-folder.
* The `nimbus` program compiles now.

* Premix environment update

details:
* Integrated `CoreDbRef` for the sources in the `premix` sub-folder.

* Fluffy environment update

details:
* Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder.

* Tools environment update

details:
* Integrated `CoreDbRef` for the sources in the `tools` sub-folder.

* Nodocker environment update

details:
* Integrated `CoreDbRef` for the sources in the
  `hive_integration/nodocker` sub-folder.

* Tests environment update

details:
* Integrated `CoreDbRef` for the sources in the `tests` sub-folder.
* The unit tests compile and run cleanly now.

* Generalise `CoreDbRef` to any `select_backend` supported database

why:
  Generalisation was just missed due to overcoming some compiler oddity
  which was tied to rocksdb for testing.

* Suppress compiler warning for `newChainDB()`

why:
  Warning was added to this function which must be wrapped so that
  any `CatchableError` is re-raised as `Defect`.

* Split off persistent `CoreDbRef` constructor into separate file

why:
  This allows to compile a memory only database version without linking
  the backend library.

* Use memory `CoreDbRef` database by default

detail:
 Persistent DB constructor needs to import `db/core_db/persistent

why:
 Most tests use memory DB anyway. This avoids linking `-lrocksdb` or
 any other backend by default.

* fix `toLegacyBackend()` availability check

why:
  got garbled after memory/persistent split.

* Clarify raw access to MPT for snap sync handler

why:
  Logically, `kvt` is not the raw access for the hexary trie (although
  this holds for the legacy database)
2023-08-04 12:10:09 +01:00
jangko 4d207e49ce
Implement plugable EVM tracer
available tracers:
- Legacy tracer
- new Json tracer
2023-08-03 16:45:39 +07:00
jangko 9be5df90cf
update tools build instruction 2023-08-01 09:25:18 +07:00
jangko 3aff8d0d61
fix EIP-4844 bugs in genesis, evmstate and blobFee 2023-07-21 06:34:56 +07:00
jangko ab79cfc775
fix blob tx limit 2023-07-20 20:20:41 +07:00
jangko 1eca772a08
hive: pyspec simulator implementation 2023-07-19 20:33:02 +07:00
andri lim 26a8759c34
implementation of EIP-4844: Shard Blob Transactions (#1440)
* EIP-4844: add pointEvaluation precompiled contract

* EIP-4844: validate transaction and block header

* EIP-4844: implement DataHash Op Code

* EIP-4844: txPool support excessDataGas calculation

* EIP-4844: make sure tx produce correct txHash

* EIP-4844: node should not automatically broadcast blob tx to it's peers

* EIP-4844: add test cases

* EIP-4844: add EIP-4844 support to t8n tool

* EIP-4844: update nim-eth to branch eip-4844

* fix t8n transaction decoding

* add t8n test data

* EIP-4844: fix blobHash opcode

* disable blobHash test when evmc_enable
2023-06-24 20:56:44 +07:00
jangko 10aabd8ca1
bubble up validate transaction error message 2023-06-12 12:58:53 +07:00
jangko 2871dbfddf
remove unused vmState param from t8n tool 2023-05-11 14:36:32 +07:00
jangko 4e58f9d79a
modexp precompile allow arbitrary input length 2023-05-11 07:21:03 +07:00
jangko 918c1309c8
fix processTransaction's gasLimit 2023-04-19 12:37:18 +07:00
jangko 12aea42c00
fix t8n's EthAddress parser 2023-04-17 21:32:02 +07:00
jangko a31db89e0e
fix t8n tool bug and accounts cache bug
t8n: a silly bug contract address generator, should use original
tx nonce instead of read the nonce from sender address in state db.
Although in EVM contract address generated by reading nonce from state db
is correct, outside EVM that nonce value might have been modified,
thus generating incorrect contract address.

accounts cache: when clearing account storage, the originalValue
cache is not cleared, only the storageRoot set to empty storage root,
this will cause getStorage and getCommitedStorage return wrong value
if the originalValue cache contains old value.
2023-03-23 11:07:30 +07:00
jangko 0b97f5a2e6
t8n tools support withdrawals records 2023-03-21 18:36:22 +07:00
jangko 2a3c67b4be
refactor touched accounts, selfdestruct, and log
simplify EVM and delegete those things to accounts cache.
also no more manual state clearing, accounts cache will be
responsible for both collecting touched account and perform
state clearing.
2023-03-21 08:14:06 +07:00
jangko 633f135c72
align accounts cache with EIP158/161
Some nomenclature used in accounts cache are not what described
in EIP158/161, therefore causing confusion and introduce bugs.
Now it should be fixed.
2023-03-18 18:13:06 +07:00