Commit Graph

3253 Commits

Author SHA1 Message Date
andri lim fd79c5c264
Fix EVM tracer crash bug when serializing nil stack (#1697)
* Fix EVM tracer crash bug when serializing nil stack

* Fix t8n tracer doc

add following description to reflect new functionality
  `stdout` - into the stdout output.
  `stderr` - into the stderr output.
  <file>   - into the file <file>-<txIndex>.jsonl.
  none     - output.basedir/trace-<txIndex>-<txhash>.jsonl.
2023-08-20 11:15:11 +07:00
andri lim bf65378006
Bump nim-web3 and nim-eth: Add EIP-4788 parentBeaconBlockRoot field (#1696) 2023-08-19 16:33:41 +07:00
Jordan Hrycaj 4c9141ffac
Aristo db implement filter serialisation for storage (#1695)
* Remove concept of empty/blind filters

why:
  Not needed. A non-existent filter is is coded as a nil reference.

* Slightly generalised backend iterators

why:
 * VertexID as key for the ID generator state makes no sense
 * there will be more tables addressed by non-VertexID keys

* Store serialised/blobified vertices on memory backend

why:
  This is more in line with the RocksDB backend so more appropriate
  for testing when comparing behaviour. For a speedy memory database,
  a backend-less variant should be used.

* Drop the `Aristo` prefix from names `AristoLayerRef`, etc.

* Suppress compiler warning

why:
  duplicate imports

* Add filter serialisation transcoder

why:
  Will be used as storage format
2023-08-18 20:46:55 +01:00
jangko df9c73cf98
fix test_merge 2023-08-18 14:49:11 +07:00
jangko 4fd53156f5
Fix engine api: getPayload V2 and V3 now returns correct blockValue 2023-08-18 14:25:11 +07:00
jangko 467e6fffa6
Implement EIP-6780: SELFDESTRUCT only in same transaction 2023-08-18 10:22:34 +07:00
jangko dd9e181acc
EIP-1153: completing transient storage on EVMC side 2023-08-18 10:19:34 +07:00
Jordan Hrycaj 3078c207ca
Aristo db implement distributed backend access (#1688)
* Fix hashing algorithm

why:
  Particular case where a sub-tree is on the backend, linked by an
  Extension vertex to the top level.

* Update backend verification to report `dirty` top layer

* Implement distributed merge of backend filters

* Implement distributed backend access management

details:
  Implemented and tested as described in chapter 5 of the `README.md`
  file.
2023-08-17 14:42:01 +01:00
jangko 3f0506b5bc
fix getPayloadBodiesByRange upper limit 2023-08-17 14:02:53 +07:00
jangko c6e7ac1647
fix fluffy rpc_eth_api tx object initialization 2023-08-17 14:02:48 +07:00
jangko 7c2e1ad3f3
Engine-api: fix getPayloadBodiesByHashV1 and getPayloadBodiesByRangeV1 bugs 2023-08-17 11:33:35 +07:00
jangko 53d7165f6d
bump web3: Change withdrawals field of ExecutionPayloadBodyV1 become optional 2023-08-17 11:31:43 +07:00
jangko 5644749de9
Fix bugs discovered by engine api simulator 2023-08-17 11:08:01 +07:00
jangko 2e85e6b2de
remove toPayloadAttributesV1OrPayloadAttributesV2 2023-08-17 07:31:22 +07:00
jangko 07fd4e9b50
Implement engine_getPayloadBodiesByRangeV1 2023-08-17 07:27:34 +07:00
jangko 629970dab8
bump nim-web3: add engine_getPayloadBodiesByRangeV1 callsig 2023-08-17 06:46:28 +07:00
jangko 0c1236756d
EIP-4895: add withdrawal processing in txpool 2023-08-16 21:28:03 +07:00
jangko a19168dcef
rpc: fix missing EIP-4895 and EIP-4844 fields of marshalled objects 2023-08-16 17:20:52 +07:00
Jordan Hrycaj 8f21cf48a8
Distributed access architecture specs (#1680)
why:
  Manage access to different MPTs via the same database (makes sense
  only if the MPTs are not too different.)
2023-08-14 19:39:10 +01:00
Jordan Hrycaj 01fe172738
Aristo db integrate hashify into tx (#1679)
* Renamed type `NoneBackendRef` => `VoidBackendRef`

* Clarify names: `BE=filter+backend` and `UBE=backend (unfiltered)`

why:
  Most functions used full names as `getVtxUnfilteredBackend()` or
  `getKeyBackend()`. After defining abbreviations (and its meaning) it
   seems easier to use `getVtxUBE()` and `getKeyBE()`.

* Integrate `hashify()` process into transaction logic

why:
  Is now transparent unless explicitly controlled.

details:
  Cache changes imply setting a `dirty` flag which in turn triggers
  `hashify()` processing in transaction and `pack()` directives.

* Removed `aristo_tx.exec()` directive

why:
  Inconsistent implementation, functionality will be provided with a
  different paradigm.
2023-08-11 18:23:57 +01:00
Jordan Hrycaj 09fabd04eb
Aristo db use filter betw backend and tx cache (#1678)
* Provide deep copy for each transaction layer

why:
  Localising changes. Selective deep copy was just overlooked.

* Generalise vertex ID generator state reorg function `vidReorg()`

why:
  makes it somewhat easier to handle when saving layers.

* Provide dummy back end descriptor `NoneBackendRef`

* Optional read-only filter between backend and transaction cache

why:
  Some staging area for accumulating changes to the backend DB. This
  will eventually be an access layer for emulating a backend with
  multiple/historic state roots.

* Re-factor `persistent()` with filter between backend/tx-cache => `stow()`

why:
  The filter provides an abstraction from the physically stored data on
  disk. So, there can be several MPT instances using the same disk data
  with different state roots. Of course, all the MPT instances should
  not differ too much for practical reasons :).

TODO:
  Filter administration tools need to be provided.
2023-08-10 21:01:28 +01:00
jangko a7db7b9101
add engine API V3 of Cancun 2023-08-08 21:41:40 +07:00
jangko ec17a5a348
engine_api: add getPayloadV3 and newPayloadV3 to echangeCapabilities
also add echangeCapabilities test to hive sim
2023-08-08 10:50:28 +07:00
jangko e9630a9200
fix engine_exchangeTransitionConfigurationV1 bug 2023-08-08 10:49:50 +07:00
jangko 7514cfc63f
increase jwt auth max time drift to 60 seconds
per engine api spec v1.0.0.beta.3
2023-08-08 08:26:08 +07:00
Jordan Hrycaj 71c91e2280
Aristo db refactor tx paradim (#1674)
* Better error handling

why:
  Bail out on some error as early as possible before any changes.

* Implement `fetch()` as opposite of `merge()`

rationale:
  In the `Aristo` realm, the action named `fetch()` and `merge()` indicate
  leaf value related actions on the MPT, while actions `get()` and `put()`
   handle vertex or hash key related operations that constitute the MPT.

* Re-factor `merge()` prototypes

why:
  The most used variant of `merge()` should have the simplest prototype.

* Persistent DB constructor needs to import `aristo/aristo_init/persistent`

why:
  Most applications use memory DB anyway. This avoids linking `-lrocksdb`
  or any other back end libraries by default.

* Re-factor transaction module

why:
  Got the paradigm wrong. The transaction descriptor did replace the
  database one but should be handled separately.
2023-08-07 18:45:23 +01:00
jangko 1788d27dfe
add ttd passed field to chain config 2023-08-06 14:55:11 +07:00
jangko 7ec298d49d
fix block import 2023-08-06 10:42:28 +07:00
andri lim bdaeedb09f
rename data gas to blob gas (#1659)
* rename data gas to blob gas

* bump more submodules
* extend evmc tx_context with EIP-4844 blob_hashes
2023-08-04 19:43:30 +07:00
Jordan Hrycaj 221e6c9e2f
Unified database frontend integration (#1670)
* Nimbus folder environment update

details:
* Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder.
* The `nimbus` program does not compile yet as it needs the updates
  in the parallel `stateless` sub-folder.

* Stateless environment update

details:
* Integrated `CoreDbRef` for the sources in the `stateless` sub-folder.
* The `nimbus` program compiles now.

* Premix environment update

details:
* Integrated `CoreDbRef` for the sources in the `premix` sub-folder.

* Fluffy environment update

details:
* Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder.

* Tools environment update

details:
* Integrated `CoreDbRef` for the sources in the `tools` sub-folder.

* Nodocker environment update

details:
* Integrated `CoreDbRef` for the sources in the
  `hive_integration/nodocker` sub-folder.

* Tests environment update

details:
* Integrated `CoreDbRef` for the sources in the `tests` sub-folder.
* The unit tests compile and run cleanly now.

* Generalise `CoreDbRef` to any `select_backend` supported database

why:
  Generalisation was just missed due to overcoming some compiler oddity
  which was tied to rocksdb for testing.

* Suppress compiler warning for `newChainDB()`

why:
  Warning was added to this function which must be wrapped so that
  any `CatchableError` is re-raised as `Defect`.

* Split off persistent `CoreDbRef` constructor into separate file

why:
  This allows to compile a memory only database version without linking
  the backend library.

* Use memory `CoreDbRef` database by default

detail:
 Persistent DB constructor needs to import `db/core_db/persistent

why:
 Most tests use memory DB anyway. This avoids linking `-lrocksdb` or
 any other backend by default.

* fix `toLegacyBackend()` availability check

why:
  got garbled after memory/persistent split.

* Clarify raw access to MPT for snap sync handler

why:
  Logically, `kvt` is not the raw access for the hexary trie (although
  this holds for the legacy database)
2023-08-04 12:10:09 +01:00
jangko 70d718119e
fix txpool usage at the beginning of process 2023-08-04 10:59:12 +07:00
jangko 4d207e49ce
Implement plugable EVM tracer
available tracers:
- Legacy tracer
- new Json tracer
2023-08-03 16:45:39 +07:00
jangko b6337e9b66
fix modexp bug 2023-08-03 12:10:38 +07:00
Jordan Hrycaj d3d172edee
Replace `CoreDbRef` oo-methods by closures (#1668)
why:
* Resolves some compiler coughing when it bails out on persitent
  db constructor inside `test()` caluses (works perfectly outside.)
* API looks cleaner and better to maintain for the price of slightly
  more work at the backend
2023-08-02 21:46:41 +01:00
jangko dc5907c30c
fix slow operations in evm 2023-08-01 16:05:39 +07:00
jangko 9be5df90cf
update tools build instruction 2023-08-01 09:25:18 +07:00
Jordan Hrycaj 322f1c2e9e
Unified database frontend (#1661)
* Remove 32bit os support from `custom_network` unit test

also:
* Fix compilation annoyance #1648
* Fix unit test on Kiln (changed `merge` logic?)

* Hide unused sources do not compile

why:
* Get them out of the way before major update
* Import and function prototype mismatch -- maybe some changes got out
  of scope.

* Re-implemented `db_chain` as `core_db`

why:
  Hiding `TrieDatabaseRef` and `HexaryTrie` by default allows to replace
  the current db wrapper by some other one, e.g. Aristo

* Support compiler exception warnings for CoreDbRef base methods.

* Allow `pairs()` iterator on all memory based key-value tables

why:
  Previously only available for capture recorder.

* Backport `chain_db.nim` changes into its re-implementation `core_apps.nim`

* Fix exception annotation
2023-07-31 14:43:38 +01:00
jangko 12faf4bdb2
bump eth-tests 2023-07-31 12:58:53 +07:00
jangko 81b7a8c682
graphql: add Shanghai and Cancun fields 2023-07-29 20:16:03 +07:00
Kim De Mey 31cff910b1
Rename beacon_chain_bridge to beacon_lc_bridge (#1655) 2023-07-25 20:53:31 +02:00
Kim De Mey c20892f805
Allow starting beacon network in local testnet and add lc bridge (#1654)
Add the option to start the fluffy nodes with the beacon network
activated and allow for running a beacon lc bridge in the local
testnet script.
2023-07-25 14:52:44 +02:00
jangko f02c20c1f9
fix EIP-4844: txpool missing dataGasUsed when generate block header 2023-07-25 12:28:17 +07:00
jangko 7229bca033
fix consensus-sim genesis parser 2023-07-25 12:00:59 +07:00
jangko 18587c3abe
use axplicit 127.0.0.1 instead of localhost string
on windows, using "localhost" for rpc test is very slow.
both pyspec_sim and engine_sim will need more than one hour.
while on linux and macos only few minutes.
2023-07-24 21:50:39 +07:00
Kim De Mey edc87b702b
Launch Fluffy builds directly from make to avoid compile issue (#1646)
* Launch Fluffy builds directly from make to avoid compile issue

Without this change, builds on latest macos fails when ulimit is
not set to 1024. But it will still cause libbacktrace error to occur
when launching the binaries so it would be still advised to
set it to 1024.

* Fix fluffy local testnet for some macOS systems

And some additional improvements to the script + run the fluffy
nodes at INFO log-level to speed-up the testing time.

* Split up fluffy tests in separate targets

This way the two test binaries can be build and ran
concurrently.
2023-07-21 16:30:22 +02:00
jangko 3aff8d0d61
fix EIP-4844 bugs in genesis, evmstate and blobFee 2023-07-21 06:34:56 +07:00
jangko ab79cfc775
fix blob tx limit 2023-07-20 20:20:41 +07:00
jangko 1eca772a08
hive: pyspec simulator implementation 2023-07-19 20:33:02 +07:00
jangko 2446c22138
pyspec sim: add test cases 2023-07-19 08:28:49 +07:00
Kim De Mey f950e0aac8
Remove unused variable in local testnet script (#1641) 2023-07-13 13:43:50 +02:00