Commit Graph

92 Commits

Author SHA1 Message Date
Jordan Hrycaj ee9aea171d
Culling legacy DB and accounts cache (#2197)
details:
+ Compiles nimbus all_tests
+ Failing tests have been commented out
2024-05-20 10:17:51 +00:00
web3-developer a147ff7553
Fix 'value out of range' RangeDefect in block witness generation. (#2072)
* Fix 'value out of range' RangeDefect caused by large/expensive blocks/transactions during DOS period.

* Clear witness cache in AccountCache persist.

* Revert previous fix and force clear cache after processing each block.

* Revert clear cache in process block.
2024-03-19 15:41:40 +08:00
andri lim 6ff2edc416
Fix styles (#2046)
* Fix styles

* Fix copyright year
2024-02-21 23:04:59 +07:00
andri lim bea558740f
Reduce compiler warnings (#2030)
* Reduce compiler warnings

* Reduce compiler warnings in test code
2024-02-16 16:08:07 +07:00
andri lim 966adcb124
Prepare source code for nim v2 CI (#2028)
* Prepare source code for nim v2 CI

* Fix copyright year
2024-02-15 09:57:05 +07:00
web3-developer 93fb4c8f59
More witness fixes (#2009)
* Update experimental rpc test to do further validation on proof responses.

* Enable zero value storage slots in the witness cache data so that proofs will be returned when a storage slot is updated to zero. Refactor and simplify implementation of getProofs endpoint.

* Improve test validation.

* Minor fixes and added test to track the changes introduced in every block using a local state.

* Refactor and cleanup test.

* Comments added to test and account cache fixes applied to account ledger.

* Return updated storage slots even when storage is empty and add test to build.

* Fix copyright and remove incorrect depth check during witness building in writeShortRlp assertion.
2024-02-09 12:09:02 +08:00
web3-developer 54644fa3cb
Witness Generation Bug Fixes (#1981)
* Return slots when verifying witness regardless of code length.

* Prevent AccountCache WitnessData codeTouched from being reset.

* Default to using {wfNoFlag} for witness flags in tests to allow running with data from before EIP170.

* Add additional json files to experimental JSON RPC test.

* Use HTTP RPC server in tests.

* Add test to check that block witness contains bytecode.
2024-01-25 01:18:45 +08:00
web3-developer 48630ccb03
Experimental RPC endpoints for generating block witnesses (#1977)
* Completed draft implementation of witness JSON-RPC endpoints for portal network bridge.

* Updated Nimbus RPC configuration to support enabling experimental endpoints.

* Moved witness verification tests.

* Added json test for getProof.

* Added main procs to new tests to fix test suite.

* Added getBlockWitness test to blockchain json test suite.

* Added tests for experimental RPC endpoints and improved the API to support returning state proofs from before or after block execution.

* Correctly rollback transaction in getBlockWitness proc.
2024-01-22 17:11:37 +08:00
web3-developer 84e7c680dd
Initial implementation of block witness verification. (#1958)
* Remove support for validation of missing leafs.

* Move proof types into separate file and borrow required procs.

* Added state proof types.

* Completed initial implementation of witness verification.

* Reject empty proofs.

* Added tests for verification of block witnesses.
2024-01-09 23:09:02 +08:00
Jordan Hrycaj b623909c44
Ledger activate unified accounts cache wrapper (#1939)
* Activate `LedgerRef` wrapper for `AccountsCache`

details:
  `accounts_cache.nim` methods are indirectly processed by the wrapper
  methods from `ledger.nim`.

  This works for all sources except `test_state_db.nim` where the source
  `accounts_cache.nim` is included (rather than imported) in order to
  access objects privy to the very source.

* Provide facility to switch to a preselected `LedgerRef` type

details:
  Can be set as suggestion when initialising `CommonRef`

* Update `CoreDb` test suite for better time tracking

details:
+ Allow time logging by pre-defined block intervals
+ Print `CoreDb`/`Ledger`profiling results (if enabled)
2023-12-12 19:12:56 +00:00
jangko b0000eed8b
Add check copyright year linter to CI 2023-11-01 10:41:20 +07:00
Jordan Hrycaj 221e6c9e2f
Unified database frontend integration (#1670)
* Nimbus folder environment update

details:
* Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder.
* The `nimbus` program does not compile yet as it needs the updates
  in the parallel `stateless` sub-folder.

* Stateless environment update

details:
* Integrated `CoreDbRef` for the sources in the `stateless` sub-folder.
* The `nimbus` program compiles now.

* Premix environment update

details:
* Integrated `CoreDbRef` for the sources in the `premix` sub-folder.

* Fluffy environment update

details:
* Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder.

* Tools environment update

details:
* Integrated `CoreDbRef` for the sources in the `tools` sub-folder.

* Nodocker environment update

details:
* Integrated `CoreDbRef` for the sources in the
  `hive_integration/nodocker` sub-folder.

* Tests environment update

details:
* Integrated `CoreDbRef` for the sources in the `tests` sub-folder.
* The unit tests compile and run cleanly now.

* Generalise `CoreDbRef` to any `select_backend` supported database

why:
  Generalisation was just missed due to overcoming some compiler oddity
  which was tied to rocksdb for testing.

* Suppress compiler warning for `newChainDB()`

why:
  Warning was added to this function which must be wrapped so that
  any `CatchableError` is re-raised as `Defect`.

* Split off persistent `CoreDbRef` constructor into separate file

why:
  This allows to compile a memory only database version without linking
  the backend library.

* Use memory `CoreDbRef` database by default

detail:
 Persistent DB constructor needs to import `db/core_db/persistent

why:
 Most tests use memory DB anyway. This avoids linking `-lrocksdb` or
 any other backend by default.

* fix `toLegacyBackend()` availability check

why:
  got garbled after memory/persistent split.

* Clarify raw access to MPT for snap sync handler

why:
  Logically, `kvt` is not the raw access for the hexary trie (although
  this holds for the legacy database)
2023-08-04 12:10:09 +01:00
Jordan Hrycaj 322f1c2e9e
Unified database frontend (#1661)
* Remove 32bit os support from `custom_network` unit test

also:
* Fix compilation annoyance #1648
* Fix unit test on Kiln (changed `merge` logic?)

* Hide unused sources do not compile

why:
* Get them out of the way before major update
* Import and function prototype mismatch -- maybe some changes got out
  of scope.

* Re-implemented `db_chain` as `core_db`

why:
  Hiding `TrieDatabaseRef` and `HexaryTrie` by default allows to replace
  the current db wrapper by some other one, e.g. Aristo

* Support compiler exception warnings for CoreDbRef base methods.

* Allow `pairs()` iterator on all memory based key-value tables

why:
  Previously only available for capture recorder.

* Backport `chain_db.nim` changes into its re-implementation `core_apps.nim`

* Fix exception annotation
2023-07-31 14:43:38 +01:00
jangko 8700d8b1e1
reduce compiler warnings 2023-06-12 12:58:53 +07:00
jangko 4e332868b7
reduce compiler warnings 2023-05-16 11:15:10 +07:00
Kim De Mey 408394a2bd
Bump nim-eth and remove unneeded Defect raises (#1575) 2023-05-10 18:04:35 +02:00
Adam Spitz c921135e2e
Refactored to introduce distinct types AccountsTrie and StorageTrie. (#1490)
This is in preparation for working on stateless mode; it's useful to
have a clearer and more type-safe interface for accessing accounts
versus slots.
2023-03-10 13:42:37 -05:00
Kim De Mey 73e93f1f11
Reduce Nim 1.6 compiler warnings & hints
* Reduce Nim 1.6 compiler warnings/hints for Fluffy and Nimbus proxy

Mostly raises Defect removals, TaintedString removal and some
unnecessary imports.

Also updating the copyright years alongside.

* Further reduce Nim 1.6 compiler warnings/hints for Nimbus
2023-01-31 13:38:08 +01:00
Jordan Hrycaj 6b9f3c9ac5
Silence compiler gossip after nim upgrade cont1 (#1455)
* Silence some compiler gossip -- part 5, common

details:
  Mostly removing redundant imports and `Defect` tracer after switch
  to nim 1.6

* Silence some compiler gossip -- part 6, db, rpc, utils

details:
  Mostly removing redundant imports and `Defect` tracer after switch
  to nim 1.6

* Silence some compiler gossip -- part 7, randomly collected source files

details:
  Mostly removing redundant imports and `Defect` tracer after switch
  to nim 1.6

* Silence some compiler gossip -- part 8, assorted tests

details:
  Mostly removing redundant imports and `Defect` tracer after switch
  to nim 1.6

* Clique update

why:
  More impossible exceptions (undoes temporary fix from previous PR)
2023-01-31 01:32:17 +00:00
jangko 94a94c5b65 implement better hardfork management 2022-12-02 13:51:42 +07:00
Jacek Sieka c2ed731fa5
eth: adapt to smaller eth_types (#1210) 2022-09-03 20:15:35 +02:00
Jamie Lokier 242dfdd5ac
Bugfix: Off by 1 in EIP-170 code size checks in `stateless`
Fixes an off by 1 error where `EIP170_CODE_SIZE_LIMIT` was being treated as the
lowest invalid value by EVM code, but the highest valid value by witness code.

To remove confusion, this is renamed to `EIP170_MAX_CODE_SIZE` with value
0x6000, which matches the name (`MAX_CODE_SIZE`) and value used for this limit
in [EIP-170](https://eips.ethereum.org/EIPS/eip-170).

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-10-19 10:30:53 +01:00
jangko 3d468a7a4c
fixes path pointing to eth_tests 2021-01-06 21:45:48 +07:00
jangko 69787ed96a
fixes test_block_witness 2021-01-06 18:47:44 +07:00
jangko 165f9fea2e
reduce warnings 2020-07-21 13:15:06 +07:00
jangko c8871bd00c
fix test_block_witness 2020-07-09 08:25:18 +07:00
jangko a2de3067a9
add test for short rlp in block witness 2020-07-08 23:17:48 +07:00
jangko 883f03b732
update block witness docs 2020-07-08 21:14:19 +07:00
jangko 0d0bc35137
block witness spec changes: explicit short rlp encoding 2020-07-08 21:05:20 +07:00
jangko 50816f2ebb
implement block witness data collection in accounts_cache 2020-06-03 20:50:13 +07:00
jangko 7d062e171b block witness: add test for unwanted multiple identical keys when constructing witness from trie 2020-06-02 12:27:13 +03:00
jangko 57e9bea4ec
add more test for invalid input 2020-05-24 11:41:20 +07:00
jangko e130cab959
remove unused artefact 2020-05-24 11:40:01 +07:00
jangko 7c0d24a648
add instruction comment for test_fuzz 2020-05-20 12:41:05 +07:00
jangko ed3434f38f
fix test_witness_json wrong code arrangement 2020-05-20 11:36:09 +07:00
jangko 0f9b31a958
add fuzz test tool 2020-05-19 11:34:40 +07:00
jangko 6c583c8e32
block witness: add more tests contain invalid data 2020-05-19 09:27:53 +07:00
jangko 612192cf86
block witness: switch uint encoding to uvarint LEB128 encoding 2020-05-16 13:18:05 +07:00
jangko 61c60abdf2
block witness new spec impl 2020-05-14 11:09:01 +07:00
jangko af28985f6e
fix error message in tests 2020-05-14 06:58:34 +07:00
andri lim 758ebabd87
working test_witness_json prototype 2020-05-08 12:16:24 +07:00
andri lim 3cfabeeb3d
optimize multi keys of branch node algorithm 2020-05-08 10:18:41 +07:00
andri lim 798de46ca7
fix json_from_tree 2020-05-07 22:12:34 +07:00
andri lim 36c93aacb1
update developers guide 2020-05-07 22:03:20 +07:00
andri lim bac4913a5a
tidying up branch node code 2020-05-07 22:03:00 +07:00
andri lim ba4360ba34
fix multikeys algorithm with multiple nibbles common prefix 2020-05-07 21:36:51 +07:00
andri lim 57355c98fd
add multi keys developers guide 2020-05-07 17:51:17 +07:00
andri lim 12b6d805cf
fix some invalid doAssert 2020-05-07 11:28:11 +07:00
andri lim 238b440040
json test generator prototype 2020-05-06 12:34:54 +07:00
andri lim ea92951fc6
add missing export marker 2020-05-06 11:47:33 +07:00