Commit Graph

215 Commits

Author SHA1 Message Date
Etan Kissling c4c37302b1
Introduce wrapper type for EIP-4844 transactions (#2177)
* Introduce wrapper type for EIP-4844 transactions

EIP-4844 blob sidecars are a concept that only exists in the mempool.
After inclusion of a transaction into an execution block, only the
versioned hash within the transaction remains. To improve type safety,
replace the `Transaction.networkPayload` member with a wrapper type
`PooledTransaction` that is used in contexts where blob sidecars exist.

* Bump nimbus-eth2 to 87605d08a7f9cfc3b223bd32143e93a6cdf351ac

* IPv6 'listen-address' in `nimbus_verified_proxy`

* Bump nim-libp2p to 21cbe3a91a70811522554e89e6a791172cebfef2

* Fix beacon_lc_bridge payload conversion and conf.listenAddress type

* Change nimbus_verified_proxy.asExecutionData param to SomeExecutionPayload

* Rerun nph to fix asExecutionData style format

* nimbus_verified_proxy listenAddress

* Use PooledTransaction in nimbus-eth1 tests

---------

Co-authored-by: jangko <jangko128@gmail.com>
2024-05-15 10:07:59 +07:00
andri lim 4078cb14c5
Engine api simulator: fix payload customizer (#2143)
* Engine api simulator: fix payload customizer

* Move versionedHashes validation before blockHash validation in newPayload

* More descriptive error message

* Fix Web3Hash to eth.Hash256 conversion
2024-04-21 21:44:05 +07:00
andri lim 6694e240d7
Repositioning blob hash validation in newPayload of engine API (#2141) 2024-04-20 02:43:13 +07:00
jangko e2c873b16f
Fix engine API and engine API simulator according to latest spec. 2024-04-18 14:55:49 +07:00
Jordan Hrycaj e8eb3268f5
Generalise prune mode option 4 different db models (#2139)
* Update README

* Nimbus-main: replaced `PruneMode` options by `ChainDbMode` options

details:
  For the legacy database, this changes the phrase
  - `conf.pruneMode == PruneMode.Full` to the expression
  + `conf.chainDbMode == ChainDbMode.Prune`.

* Fix issues moaned about by NIM compiler

* Fix copyright year
2024-04-17 18:09:55 +00:00
jangko 6c1afe1127
Engine API: forkChoiceUpdated V2 And V3 should return invalidPayloadAttributes error instead of invalidParams error upon receiving erroneous payload attributes 2024-04-17 15:45:39 +07:00
andri lim e713f3c287
Implement eth_feeHistory (#2130)
* Implement eth_feeHistory

* Fix copyright year
2024-04-16 08:02:42 +07:00
andri lim fd9bb28b20
Bump nim-web3 to 9e370474fb16039860d074535dd1649f1c2307b2 (#2112) 2024-03-28 14:16:40 +07:00
andri lim 33976e8875
Replace invalidParams error with invalidForkchoiceState error in forkchoiceUpdated handler (#2106) 2024-03-26 09:31:21 +07:00
Snoppy 4848fd19d5
chore: fix typos (#2103) 2024-03-25 09:14:47 +01:00
andri lim eb67e78fde
Implement RPC methods: debug_getRawBlock, debug_getRawHeader, debug_g… (#2098)
* Implement RPC methods: debug_getRawBlock, debug_getRawHeader, debug_getRawReceipts, debug_getRawTransaction

* Fix engine api sim when calling setupDebugRpc
2024-03-22 23:40:16 +07:00
andri lim 9cd80800df
Load Kzg trusted setup at the start of consensus sim (#2096) 2024-03-22 12:49:26 +07:00
andri lim 7ea6d719d9
Implement RPC method eth_getAccessList (#2091)
* Implement RPC method eth_getAccessList

* Fix comment
2024-03-21 18:24:32 +07:00
andri lim 30277be1f3
Bump nim-web3 to 285d97c2b05bbe2a13dab4b52ea878157fb1a1a1 (#2088)
* Bump nim-web3 to 285d97c2b05bbe2a13dab4b52ea878157fb1a1a1

Unify EthCall/EthSend into TransactionArgs (#138)

* bump ssz-serialization

* Fix BlockNumber conversion

* Bump ssz-serialization: Restrict toSszType usage to non SszType in readSszBytes (#81)
2024-03-21 08:05:22 +07:00
andri lim c41206be39
Fix styles and reduce compiler warnings (#2086)
* Fix styles and reduce compiler warnings

* Fix copyright year
2024-03-20 14:35:38 +07:00
Kim De Mey 1159b0114e
Add portal_bridge history mode (#2067)
Portal bridge mode for following latest and injecting the latest
block data into the Portal network.
2024-03-11 18:20:29 +01:00
Jordan Hrycaj a02a915039
Provide public default db symbol (#2050)
* CoreDb: Provide default db backend symbols

why:
  Handy for running `Aristo` against standard tests

note:
  These defaults are currently set to legacy DB types. The must be
  enabled manually in `db/core_db.nim`.

* Provide `Aristo` for macro assembler related tests

caveat:
  Some tests use `initStorageTrie()` which lets `Aristo` bail out. The
  test need to run on `distinct_ledgers` (or something like) rather than
  `distinct_tries`.

* Tests: Misc modules that can run on `Aristo` as well

* NoHive: Module that can run on `Aristo` as well

* Fix copyright year

* ditto
2024-02-23 09:17:24 +00:00
andri lim 52aa87c539
Align with EIP-4844: rename getBlobGasPrice to getBlobBaseFee (#2043) 2024-02-21 16:14:34 +07:00
andri lim 7b4ef814ea
Bump eth_tests and related fixes (#2042)
* Bump eth_tests and related fixes

* Fix tests
2024-02-21 16:14:20 +07:00
andri lim 966adcb124
Prepare source code for nim v2 CI (#2028)
* Prepare source code for nim v2 CI

* Fix copyright year
2024-02-15 09:57:05 +07:00
andri lim 8a40521cbe
Fix 'trustedSetupFile' is not accessible for type NimbusConf.cmd == import (#2008) 2024-02-04 22:45:54 +07:00
andri lim a441ec3cb1
Remove obsolete select_backend and fix simulators CI (#2007)
* Remove obsolete select_backend

* Fix copyright year
2024-02-04 21:28:20 +07:00
andri lim c635e160d9
Implement combo http server for rpc, engine_api, and graphql services (#1992)
* Combo HTTP server implementation

* Use json flavor for jwt_auth decoder
2024-01-29 20:20:04 +07:00
Kim De Mey dbc1ae86e2
Vendor bumps + related fixes + warning fixes (#1985)
- Vendor bump of stew, nim-eth, chronos, nimbus-eth2 and libp2p
- Bump related fixes + fixes of deprecation warnings
- Several other warnings fixed.
2024-01-24 16:28:03 +01:00
jangko fb2aa1b6f9
Fix engine_sim regression 2024-01-15 13:03:29 +07:00
jangko 3e21281d12
Bump nim-web3 and others
Bump nim-json-rpc and nimbus-eth2 too.
Reason: both nim-json-rpc and nim-web3 migrate from
stdlib/json to nim-json-serialization
2024-01-14 10:41:23 +07:00
jangko 5fd54961e3
Bump nim-web3: remove rpc types duplicates 2023-12-13 07:58:12 +07:00
Jordan Hrycaj b623909c44
Ledger activate unified accounts cache wrapper (#1939)
* Activate `LedgerRef` wrapper for `AccountsCache`

details:
  `accounts_cache.nim` methods are indirectly processed by the wrapper
  methods from `ledger.nim`.

  This works for all sources except `test_state_db.nim` where the source
  `accounts_cache.nim` is included (rather than imported) in order to
  access objects privy to the very source.

* Provide facility to switch to a preselected `LedgerRef` type

details:
  Can be set as suggestion when initialising `CommonRef`

* Update `CoreDb` test suite for better time tracking

details:
+ Allow time logging by pre-defined block intervals
+ Print `CoreDb`/`Ledger`profiling results (if enabled)
2023-12-12 19:12:56 +00:00
jangko 9cf3c71a57
Fix Engine API simulator 2023-11-05 14:46:26 +07:00
jangko 7de6199ba3
Engine API: Fix latestValidHash value when invalid timestamp detected 2023-11-05 10:52:27 +07:00
jangko 19f313d891
More Engine API tests 2023-11-03 16:58:22 +07:00
jangko 25bc8e4b22
Add blobGasPrice and blobGasUsed to ReceiptObject of RPC 2023-11-02 11:38:07 +07:00
jangko 69254e614f
More Engine API tests 2023-11-01 19:55:28 +07:00
jangko 682b160777
Fix TxSender of engine api simulator 2023-11-01 17:30:47 +07:00
jangko b0000eed8b
Add check copyright year linter to CI 2023-11-01 10:41:20 +07:00
jangko 5d50bb9a2b
Add Engine API generic tests 2023-10-31 10:18:37 +07:00
jangko d5c6c1bbf0
Pyspec sim: run test using stable release test vector 2023-10-31 09:52:31 +07:00
andri lim 5bfdcd0d27
Engine API: rearrange version and fork validation in fcU and newPayload (#1848) 2023-10-24 11:30:48 +07:00
andri lim 77289c7795
More cancun tests (#1843)
* Engine API simulator: More Cancun tests

* Fix Cancun validation in Engine API and TxPool
2023-10-23 20:59:57 +07:00
andri lim 4f6cdab641
Engine API simulator: pass first Cancun test (#1838) 2023-10-22 14:05:20 +07:00
andri lim ec08907f05
hive sim: Prepare Cancun engine API test framework (#1809) 2023-10-19 10:28:52 +07:00
andri lim 04c7ed8ec9
Change EthTime from std.Time to distinct uint64 (#1820)
* Change EthTime from std.Time to distinct uint64

* Bump nimbus-eth2
2023-10-18 09:16:11 +07:00
andri lim 744e4f9739
bump web3 and add shouldOverrideBuilder field to GetPayloadV3Response (#1807)
* bump web3 and add shouldOverrideBuilder field to GetPayloadV3Response

* Resolve ambiguous identifier error
2023-10-05 20:54:51 +07:00
andri lim 34b1e29ac9
Fix test_blockchain_json and pyspec simulator for Cancun (#1805)
* Fix test_blockchain_json and pyspec simulator for Cancun

* Preserve applyDeletes comments

* Fix redefinition error

* Move test_macro to tests folder
2023-10-05 10:04:12 +07:00
andri lim 12f494e669
Only one place to declare gwei (#1802)
* Only one place to declare gwei

* Fix weiAmount overflow
2023-10-04 10:47:18 +07:00
Kim De Mey 1bb389ad4e
Use admin prefix for nodeInfo and addPeer JSON-RPCs (#1799) 2023-10-03 14:44:08 +02:00
jangko 5754a9079b
Simplify pyspec simulator 2023-09-30 21:31:57 +07:00
jangko 369a54d62a
Add makeNextWithdrawals to engine api simulator 2023-09-30 20:16:45 +07:00
andri lim 501d8a369a
Add BeaconSync reorg tests (#1782)
* Add BeaconSync reorg tests

* Fix redefinition error in tx_sender.nim
2023-09-30 19:20:29 +07:00
andri lim e8d59bc7a7
Working prototype of beacon downloader (#1780)
* Working prototype of beacon downloader

* Use KeyedQueue for setSyncTarget queue

* Convert MergeTracker and PayloadQueue to non ref object
2023-09-28 13:20:12 +07:00
andri lim 9cce31d921
Prepare pyspec sim for Cancun (#1773) 2023-09-25 06:53:33 +07:00
andri lim b7f0be5468
Fix Windows simulators CI (#1743)
* update simulator cache key

* enable libbacktrace in hive simulators
2023-09-10 21:13:09 +07:00
andri lim 948c94763c
Bump nim-eth: Add closeWait to EthereumNode (#1742) 2023-09-09 13:54:58 +07:00
andri lim 348a9aea6f
Close eth node after usage (#1741) 2023-09-08 21:37:50 +07:00
jangko 5fb0fc65ba
Implement beacon sync stub
- Prepare a test env for beacon sync in engine api simulator.
- Wiring beacon sync to the rest of subsystems.
2023-09-07 08:49:31 +07:00
jangko 6139152143
Add multiple clients support to engine api simulator 2023-09-06 18:51:02 +07:00
jangko 7a1fe5707c
Refactor engine api and cleanup web3 types conversion 2023-08-30 10:42:46 +07:00
jangko 820525d78c
Fix outdated EIP-4844 constants 2023-08-24 12:11:19 +07:00
jangko 8773eb609d
Fix engine api client ambiguity in engine api sim 2023-08-24 11:34:12 +07:00
jangko 38c0c34331
hive: Engine api simulator overhaul 2023-08-22 21:19:43 +07:00
jangko ec17a5a348
engine_api: add getPayloadV3 and newPayloadV3 to echangeCapabilities
also add echangeCapabilities test to hive sim
2023-08-08 10:50:28 +07:00
jangko 7514cfc63f
increase jwt auth max time drift to 60 seconds
per engine api spec v1.0.0.beta.3
2023-08-08 08:26:08 +07:00
jangko 7ec298d49d
fix block import 2023-08-06 10:42:28 +07:00
andri lim bdaeedb09f
rename data gas to blob gas (#1659)
* rename data gas to blob gas

* bump more submodules
* extend evmc tx_context with EIP-4844 blob_hashes
2023-08-04 19:43:30 +07:00
Jordan Hrycaj 221e6c9e2f
Unified database frontend integration (#1670)
* Nimbus folder environment update

details:
* Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder.
* The `nimbus` program does not compile yet as it needs the updates
  in the parallel `stateless` sub-folder.

* Stateless environment update

details:
* Integrated `CoreDbRef` for the sources in the `stateless` sub-folder.
* The `nimbus` program compiles now.

* Premix environment update

details:
* Integrated `CoreDbRef` for the sources in the `premix` sub-folder.

* Fluffy environment update

details:
* Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder.

* Tools environment update

details:
* Integrated `CoreDbRef` for the sources in the `tools` sub-folder.

* Nodocker environment update

details:
* Integrated `CoreDbRef` for the sources in the
  `hive_integration/nodocker` sub-folder.

* Tests environment update

details:
* Integrated `CoreDbRef` for the sources in the `tests` sub-folder.
* The unit tests compile and run cleanly now.

* Generalise `CoreDbRef` to any `select_backend` supported database

why:
  Generalisation was just missed due to overcoming some compiler oddity
  which was tied to rocksdb for testing.

* Suppress compiler warning for `newChainDB()`

why:
  Warning was added to this function which must be wrapped so that
  any `CatchableError` is re-raised as `Defect`.

* Split off persistent `CoreDbRef` constructor into separate file

why:
  This allows to compile a memory only database version without linking
  the backend library.

* Use memory `CoreDbRef` database by default

detail:
 Persistent DB constructor needs to import `db/core_db/persistent

why:
 Most tests use memory DB anyway. This avoids linking `-lrocksdb` or
 any other backend by default.

* fix `toLegacyBackend()` availability check

why:
  got garbled after memory/persistent split.

* Clarify raw access to MPT for snap sync handler

why:
  Logically, `kvt` is not the raw access for the hexary trie (although
  this holds for the legacy database)
2023-08-04 12:10:09 +01:00
jangko 70d718119e
fix txpool usage at the beginning of process 2023-08-04 10:59:12 +07:00
jangko 81b7a8c682
graphql: add Shanghai and Cancun fields 2023-07-29 20:16:03 +07:00
jangko 7229bca033
fix consensus-sim genesis parser 2023-07-25 12:00:59 +07:00
jangko 18587c3abe
use axplicit 127.0.0.1 instead of localhost string
on windows, using "localhost" for rpc test is very slow.
both pyspec_sim and engine_sim will need more than one hour.
while on linux and macos only few minutes.
2023-07-24 21:50:39 +07:00
jangko 1eca772a08
hive: pyspec simulator implementation 2023-07-19 20:33:02 +07:00
jangko 2446c22138
pyspec sim: add test cases 2023-07-19 08:28:49 +07:00
jangko 217068790d
update graphql simulator test vector 2023-07-07 14:40:48 +07:00
jangko dac1454a15
consensus-sim: importing blocks individually instead of concatenating them.
This is a workaround following hive example, because one of the test case
withdrawalsAmountBounds.json, have bad blocks between good blocks.
And that bad blocks contains big int too big to fit in uint64 of
withdrawal amount field.

Clients who still importing concatenated blocks cannot pass all tests.
2023-06-27 07:30:05 +07:00
jangko ff1a45e095
fix shanghai withdrawal validation
previously, the withdrawal validation is in process_block only,
but the one in persist block, which is also used in synchronizer
is not validated properly.
2023-06-26 07:46:09 +07:00
jangko f8c1a7f0a8
fix rpc_sim receipt unmarshalling 2023-06-25 13:52:31 +07:00
Adam Spitz d8a1adacaa
More work on withdrawals (#1482)
* Part of EIP-4895: add withdrawals processing to block processing.

* Refactoring: extracted the engine API handler bodies into procs.

Intending to implement the V2 versions next. (I need the bodies to be
in separate procs so that multiple versions can use them.)

* Working on Engine API changes for Shanghai.

* Updated nim-web3, resolved ambiguity in Hash256 type.

* Updated nim-eth3 to point to master, now that I've merged that.

* I'm confused about what's going on with engine_client.

But let's try resolving this Hash256 ambiguity.

* Still trying to fix this conflict with the Hash256 types.

* Does this work now that nimbus-eth2 has been updated?

* Corrected blockValue in getPayload responses back to UInt256.

c834f67a37

* Working on getting the withdrawals-related tests to pass.

* Fixing more of those Hash256 ambiguities.

(I'm not sure why the nim-web3 library introduced a conflicting type
named Hash256, but right now I just want to get this code to compile again.)

* Bumped a couple of libraries to fix some error messages.

* Needed to get "make fluffy-tools" to pass, too.

* Getting "make nimbus_verified_proxy" to build.
2023-03-09 18:40:55 -05:00
Kim De Mey 8523d8b40e
Bump submodules and fix related issues (#1448)
* Updated to the latest nim-eth, nim-rocksdb, nim-web3

* Bump nimbus-eth2 module and fix related issues

Temporarily disabling Portal beacon light client network as it is
a lot of copy pasted code that did not yet take into account
forks. This will require a bigger rework and was not yet tested
in an actual network anyhow.

* More nimbus fixes after module bumps

---------

Co-authored-by: Adam Spitz <adamspitz@status.im>
Co-authored-by: jangko <jangko128@gmail.com>
2023-01-27 15:57:48 +01:00
jangko 13e5b1a76b
fix persistHeaderToDb, allow POS block header to be canonical 2022-12-10 09:02:30 +07:00
jangko 94a94c5b65 implement better hardfork management 2022-12-02 13:51:42 +07:00
jangko 66439d69ca
unify chain config parser of t8n, bc test, and consensus simulator 2022-11-25 12:26:29 +07:00
jangko 6b28b64395
update hive statistics 2022-11-18 11:58:04 +07:00
jangko 43f4b99a1b
disable NewBlockHashes and NewBlock of eth wire handler after POS transition
fix #1133
2022-11-14 16:17:34 +07:00
Kim De Mey 74a83c1229
Bump nim-eth and remove all now unneeded p2p related imports (#1273) 2022-10-20 10:34:59 +02:00
jangko 16bc2de1cf
update EF test fixtures and fixes to pass all tests 2022-10-03 16:41:32 +07:00
Jacek Sieka c2ed731fa5
eth: adapt to smaller eth_types (#1210) 2022-09-03 20:15:35 +02:00
jangko fe54f93ab6
bump nim eth
better method not implemented message of AbstractChainDB
2022-08-22 20:11:36 +07:00
jangko 11908c78ec
engine api test: handle JsonRpcError for each client rpc call 2022-08-05 09:51:16 +07:00
jangko 21140191da
attempt to fix flaky simulator test 2022-08-02 11:48:59 +07:00
jangko 2c0a8fa26d
update to supported macOS in Github CI 2022-07-30 08:42:44 +07:00
jangko eb972fd598
attemp to fix simulator ci 2022-07-24 15:47:06 +07:00
jangko a48f69a89a
hive: engine simulator add jwt auth test cases 2022-07-18 16:56:44 +07:00
jangko 2b4baff8ec
move block validation from execution payload generator to engine api 2022-07-04 20:36:30 +07:00
jangko ce098bf88f
fix engine api test 2022-07-04 19:32:14 +07:00
Kim De Mey fb12793482
Bump nim-eth and nim-bearssl and accompanying fixes (#1150) 2022-07-04 09:38:02 +02:00
jangko 7f0bc71b65
add invalidMissingAncestorReOrg test case 2022-06-27 20:18:54 +07:00
jangko 4a50b00c37
rpc: change getStorageAt param type to HexDataStr
if using HexQuantityStr, it will fail when given hex string with leading zeros
2022-06-27 17:47:59 +07:00
jangko 71ac6b7de5
engine-api-test: fix transactionReorg test case 2022-06-27 13:01:32 +07:00
jangko b80eca0718
json-rpc: able to query finalized block and safe block header
engine-api:
- store safe block hash and finalized block hash

engine-api test:
- fix test case related to safe block hash and finalized block hash
2022-06-27 11:15:54 +07:00
jangko 8dce76c49f
add engine api test fixtures: PoW chain rlp 2022-06-22 08:10:10 +07:00
jangko d07ef2ee56
fix engine api and angine api test 2022-06-22 08:10:01 +07:00