Commit Graph

182 Commits

Author SHA1 Message Date
Jordan Hrycaj 134fe26997
Store proved snap accounts (#1145)
* Relocated `IntervalSets` to nim-stew repo

* Accumulate accounts on temporary kv-DB

why:
  Explore the data as returned from snap/1. Will be converted to a
  `eth/db` next.

details:
  Verify and accumulate per/state-root accounts downloaded via snap.

also:
  Some unit tests

* Replace `Table` by `TrieDatabaseRef` for accounts accumulator

* update ticker statistics

details:
  mean/variance based counter update

* allow persistent db for proved accounts

* rebase, and globally activate unit test

* fix statistics
2022-07-01 12:42:17 +01:00
jangko e274b347ae
add haveBlockAndState to BaseChainDB and use it in engine api 2022-06-30 16:11:43 +07:00
KonradStaniec fdb048ed21
Add getLogs implementation (#1143)
* Add eth_getLogs json-rpc endpoint in Fluffy and Nimbus
2022-06-29 17:44:08 +02:00
jangko 4a50b00c37
rpc: change getStorageAt param type to HexDataStr
if using HexQuantityStr, it will fail when given hex string with leading zeros
2022-06-27 17:47:59 +07:00
jangko b80eca0718
json-rpc: able to query finalized block and safe block header
engine-api:
- store safe block hash and finalized block hash

engine-api test:
- fix test case related to safe block hash and finalized block hash
2022-06-27 11:15:54 +07:00
jangko 69a1000d77
more engine api test 2022-06-15 15:56:45 +07:00
jangko 6fcd63cb58
modify hive simulators to run in CI 2022-04-21 12:01:18 +07:00
jangko f2f204293e
first step into styleCheck fixes 2022-04-14 08:39:50 +07:00
Jacek Sieka 1d6a9951d6
move rocksdb support to eth1 (#927)
* move rocksdb support to eth1

only used here / causes unnecessary build deps
2022-04-06 07:28:19 +02:00
jangko c704626ea8
mark the new chain as canonical when calling db.setHead
the new chain marked as canonical is from the new head down to
forking point. most of the time, the forking point is the parent
itself.
2022-03-26 13:23:08 +07:00
jangko e628c8dc88
temporary fix db.setHead before actual fix 2022-03-25 14:01:32 +07:00
jangko d41a3dedf0
write block ttd in `persistHeaderToDbWithoutSetHead` 2022-03-08 21:47:51 +07:00
jangko f782327fcf
reimplement engine API rpc kiln spec v2 2022-03-03 11:28:18 +07:00
jangko 73e28694b5
move ttd from vm/state to chain_db 2022-03-03 08:58:55 +07:00
jangko 9a1c8fc779
refactor genesis.nim 2022-03-03 08:28:00 +07:00
jangko 28cdfcaf6b
fix EIP-4399 'random' opcode
- fix previous implementation of EIP-4399
- now `random` opcode can be used with evmc_enabled
2022-02-08 20:23:40 +07:00
Zahary Karadjov 137eb97766 Initial implementation of the merge spec
Includes a simple test harness for the merge interop M1 milestone

This aims to enable connecting nimbus-eth2 to nimbus-eth1 within
the testing protocol described here:

https://github.com/status-im/nimbus-eth2/blob/amphora-merge-interop/docs/interop_merge.md

To execute the work-in-progress test, please run:

In terminal 1:
tests/amphora/launch-nimbus.sh

In terminal 2:
tests/amphora/check-merge-test-vectors.sh
2022-01-24 09:44:39 +02:00
Jordan Hrycaj 261c0b51a7
Redesign of BaseVMState descriptor (#923)
* Redesign of BaseVMState descriptor

why:
  BaseVMState provides an environment for executing transactions. The
  current descriptor also provides data that cannot generally be known
  within the execution environment, e.g. the total gasUsed which is
  available not before after all transactions have finished.

  Also, the BaseVMState constructor has been replaced by a constructor
  that does not need pre-initialised input of the account database.

also:
  Previous constructor and some fields are provided with a deprecated
  annotation (producing a lot of noise.)

* Replace legacy directives in production sources

* Replace legacy directives in unit test sources

* fix CI (missing premix update)

* Remove legacy directives

* chase CI problem

* rebased

* Re-introduce 'AccountsCache' constructor optimisation for 'BaseVmState' re-initialisation

why:
  Constructing a new 'AccountsCache' descriptor can be avoided sometimes
  when the current state root is properly positioned already. Such a
  feature existed already as the update function 'initStateDB()' for the
  'BaseChanDB' where the accounts cache was linked into this desctiptor.

  The function 'initStateDB()' was removed and re-implemented into the
  'BaseVmState' constructor without optimisation. The old version was of
  restricted use as a wrong accounts cache state would unconditionally
  throw an exception rather than conceptually ask for a remedy.

  The optimised 'BaseVmState' re-initialisation has been implemented for
  the 'persistBlocks()' function.

also:
  moved some test helpers to 'test/replay' folder

* Remove unused & undocumented fields from Chain descriptor

why:
  Reduces attack surface in general & improves reading the code.
2022-01-18 16:19:32 +00:00
jangko baf508f6ae
move stateDB from VMState to chainDB
previously, every time the VMState was created, it will also create
new stateDB, and this action will nullify the advantages of cached accounts.

the new changes will conserve the accounts cache if the executed blocks
are contiguous. if not the stateDB need to be reinited.

this changes also allow rpcCallEvm and rpcEstimateGas executed properly
using current stateDB instead of creating new one each time they are called.
2021-10-28 18:57:08 +07:00
jangko 44394d9ffd
fixes nimbus evm tracer, add missing networkParams when constructing chainDB 2021-09-26 10:45:52 +07:00
jangko a3badea928
config: fix new config based on input from jamie and zahary 2021-09-18 17:34:51 +07:00
jangko 69f2a0f95a
config: replace stdlib parseOpt with nim-confutils
fixes #581
2021-09-18 17:34:46 +07:00
jangko 48d497580a
config: remove last instance of getConfiguration usage from nimbus code
this is a preparation for migration to confutils based config
although there is still some getConfiguration usage in tests code
it will be removed after new config arrived
2021-09-08 21:25:14 +07:00
Kim De Mey e37bafd47e
Add a first, simple, content network test (#760)
* Add a first, simple, content network test

* Use sqlite as nimbus_db_backend for fluffy tests

* Fix backend selection for sqlite
2021-07-15 15:12:33 +02:00
Jordan Hrycaj 491149c6d5 Eip225 clique/PoA consensus protocol
details:
  formal port from go-lang sources, compiles but will not do anything
  useful yet
2021-06-17 08:03:57 +01:00
jangko a0d10f5728
drop PublicNetwork enum usage and replace it with NetworkId
we cannot limit the `--networkid` switch to values available in
`PublicNetwork` enum. it should able to accept very wide range of
custom NetworkId.
2021-05-20 14:04:16 +07:00
jangko f2491e6307
fixes crappy custom genesis and chain config parser
instead of using stdlib/json, now we switch to json_serialization
the result is much tidier code and more robust when parsing
optional fields.

fixes #635
2021-05-13 16:04:08 +07:00
jangko f6a0e4bcbd
fixes wrong usage of `chainId` in places where it should be networkId
fixes #643
2021-05-12 09:45:09 +07:00
jangko 82d4f6a2d4
fixes getReceipts proc signature in db_chain.nim
instead of using header as input param, now getReceipts using
receiptRoot hash, the intention is clearer and less data passed around
when we only using receiptRoot instead of whole block header.
2021-04-24 10:51:05 +07:00
jangko 1728dd8d54
fix getTransactionCount bug in db_chain.nim
getTransactionCount will always return 0 because the txCount
variable is not returned
2021-04-24 10:47:37 +07:00
jangko 3db535aa39
EIP2929 implementation 2021-01-11 14:56:42 +07:00
jangko f2b483d6ad
access list implementation 2021-01-11 14:54:55 +07:00
jangko f987e86562
implement more eth rpc 2020-07-30 14:21:11 +07:00
jangko c9802edfce
setup block and state env for more complex eth rpc tests 2020-07-28 23:48:45 +07:00
jangko f82dff64fa
implement more eth rpc and keystore management 2020-07-23 14:54:32 +07:00
jangko 336efdb0c3
implement web3, net, and some eth namespace rpc 2020-07-22 23:57:55 +07:00
jangko 165f9fea2e
reduce warnings 2020-07-21 13:15:06 +07:00
jangko 9d0b399213
fix accounts_cache: move overlayStorage to originalStorage when not clearCache 2020-06-20 18:45:09 +07:00
jangko bd7e1fe2e5
reuse VMState and AccountsCache for better performance 2020-06-18 13:16:38 +07:00
jangko eabacb0a33
allow nimbus to read geth database 2020-06-18 13:16:25 +07:00
jangko 50816f2ebb
implement block witness data collection in accounts_cache 2020-06-03 20:50:13 +07:00
jangko 844071033a
fix at various places related to missing accounts_cache.persist call 2020-06-01 13:45:32 +07:00
jangko 8a4c8c6273
fix buggy isAlive from accounts_cache 2020-06-01 10:54:25 +07:00
jangko 71514a0a66
replace state_db with accounts_cache 2020-05-30 10:14:59 +07:00
andri lim c9e49bf68a
fix accounts cache init API 2020-04-29 12:00:44 +07:00
Jacek Sieka ed7dbab70f db: use kvstore from nim-eth 2020-04-27 18:40:45 +03:00
Jacek Sieka 4ade5797ee
rlp: don't use ranges / experimental features (#495) 2020-04-20 20:12:44 +02:00
andri lim edc19076e3 accounts cache: read-only operations should not create new account 2020-02-27 14:38:42 +02:00
andri lim 2fbabd25a4 implement aleth/geth/parity compatibility mode -- 100% pass test 2020-02-20 09:08:44 +02:00
andri lim 16a938d3fa fixes clearStorage bug 2020-01-20 18:36:58 +02:00
andri lim 5f24757cd7 use 'withValue' when retrieving account's storage 2020-01-19 19:17:37 +02:00
andri lim cf0d4481f3 [skip ci] rename 'isExists' to 'exists' 2020-01-19 19:17:37 +02:00
andri lim 727b477fca [skip ci] add accounts cache test 2020-01-19 19:17:37 +02:00
andri lim 5795bf52e0 [skip ci] implement accounts_cache 2020-01-19 19:17:37 +02:00
andri lim 1ffb992674 multi root state trie implementation 2019-12-10 12:34:48 +02:00
andri lim 97e89b3afe implement 'getCommittedStorage' 2019-11-14 10:11:35 +00:00
andri lim c4dcf5da30
reduce compiler warnings 2019-11-13 21:49:39 +07:00
andri lim 61f340ae87 implement EIP2200's SSTORE and gas cost 2019-11-12 15:51:48 +00:00
andri lim 3ad465d169 implement block validation and friends 2019-09-26 12:33:43 +03:00
Jacek Sieka 2763bd0dd5
std_shims -> stew 2019-07-07 12:12:01 +02:00
andri lim 4a7f58c7dc implement constantinople bitwise opcode 2019-05-13 10:26:28 +03:00
andri lim 85d9bd3958
implement EIP158, GST -4 +81 2019-04-17 11:03:52 +07:00
andri lim 8a9e4aba80
apply dao fork changes 2019-04-07 06:53:26 +07:00
andri lim f2f92f5608
fix block 1.487.668 problem: getScore int overflow 2019-04-05 16:29:05 +07:00
Ștefan Talpalaru a67edd693a
assert() -> doAssert() 2019-03-13 22:36:54 +01:00
andri lim e2311d798b
fix recurring symbol collision problem 2019-03-07 22:53:09 +07:00
andri lim 200f93a3b7 refactor incNonce 2019-02-27 13:30:18 +02:00
andri lim 2c032ad1ab refactor utils 2019-02-27 13:30:18 +02:00
andri lim 282824cd53 zah's request: add missing comments 2019-02-18 13:16:04 +02:00
andri lim c53e7fa07c fixes #228 2019-02-15 17:20:51 +02:00
andri lim 7c6c7dbe3a 'accountExist' to 'accountExists' 2019-02-06 18:42:03 +01:00
andri lim 4cc0ef427c add ccountExist and isDeadAccount to AccountStateDB 2019-02-06 18:42:03 +01:00
Yuriy Glukhov 481c6cf4ed Use nim-eth (#224) 2019-02-05 20:15:50 +01:00
andri lim 0c887b3711 move initializeEmptyDb to genesis.nim 2019-01-15 15:30:25 +02:00
Yuriy Glukhov 58ad0e472c Moved db backends to eth_trie 2019-01-13 15:31:59 +02:00
Ștefan Talpalaru 89044b84e7 update Nimbus header and refine db backend picking
- dynamically generated copyright year interval
- added the db backend to the header
- documented the db-backend-changing define, made it case insensitive
  and ensured wrong values would trigger compilation errors
2019-01-07 01:21:29 +02:00
andri lim 281b7dffac implement debug_setHead 2019-01-06 13:17:08 +02:00
andri lim e78fb72ef6 fixes #188 2019-01-06 13:16:45 +02:00
andri lim 041ed689aa fix silly typo mistake 2019-01-06 11:43:38 +02:00
andri lim 96b18cfce3 add select_backend module 2019-01-06 11:43:38 +02:00
andri lim 0487720769 fix capturedb bug 2019-01-06 11:43:38 +02:00
andri lim ab0fb7fdf0 fix getScore 32bit bug 2019-01-06 11:43:38 +02:00
andri lim 5fc134ac2a add lmdb backend 2019-01-06 11:43:38 +02:00
andri lim a6bf970b1b some cleanup 2019-01-06 11:43:38 +02:00
Ștefan Talpalaru 048a43b2f1
better error handling
- added some useful `chronicles` defines in the top-level nim.cfg
- converted some assert() calls to error codes to avoid a reported
  segfault with -d:release
- the recoverable errors are being recovered from, in persistWorkItem()
- the only case that's a show stopper raises an exception
2018-12-17 23:36:38 +01:00
andri lim 9c0bb70ba7 working makeReceipt 2018-12-16 15:38:51 +02:00
andri lim ad1202f98a initial make_receipt 2018-12-16 15:38:51 +02:00
andri lim 7a3ddfec87 correcting account storage retrieval 2018-12-05 22:01:29 +07:00
andri lim c954dc7d62 fix tests 2018-12-05 22:01:29 +07:00
andri lim 5509c66cce implement transaction tracer 2018-12-05 22:01:28 +07:00
andri lim a098285a7b put prune switch into usage 2018-12-05 21:58:34 +07:00
andri lim 21070d510d add TransactionKey to persistTransactions and add more fat to debug_traceTransaction 2018-11-28 09:25:36 +07:00
andri lim 161d6adbc7 implement persistTransactions and getBlockBody 2018-11-24 08:00:58 +07:00
Dustin Brody d97cc50afc fix eth_trie/constants missing compile error 2018-10-19 07:44:08 -07:00
Zahary Karadjov 41adca1abb Minimal changes required for building the LES branch 2018-10-16 03:10:01 +03:00
Zahary Karadjov 343cc4fa43 Populate the persistent databases with the empty RLP key.
Also implements transactional block persistence. Two issues
in the transaction processing code have been discovered that
might affect other usages such as the CALL instruction.

The main fix gets us past block 49000.

You may need to clean up your database.
2018-10-05 03:36:48 +03:00
Zahary Karadjov 74f6f0b738 Deal with the refactoring in status-im/nim-rlp#20 2018-09-26 12:58:23 +03:00
Zahary Karadjov 5c0fd16448 Fix an issue with the Sqlite3 tests (the in-memory DB was not properly used) 2018-09-25 16:01:38 +03:00
Zahary Karadjov 4e323df363 Implement a --datadir command-line options
By default, the database files will be written in the
platform-specific application data folder:

$HOME/AppData/Roaming/Nimbus/DB
$HOME/Library/Application Support/Nimbus/DB
$HOME/.cache/nimbus/db
2018-09-25 02:06:20 +03:00
tersec 690ce2cf97
Partly fix state_db getCode for 20 new working GeneralStateTests (#156)
* partly fix state_db getCode to get 20 new working GeneralStateTests; remove 2 functions which existed as workarounds; switch all remaining setBalance calls in GeneralStateTestRunner to subBalance (addBalance calls already changed)

* two of the 20 new tests don't work in 32-bit builds
2018-09-20 21:36:57 +00:00