2019-02-01 09:25:10 +00:00
|
|
|
# Nimbus
|
Core db and aristo updates for destructor and tx logic (#1894)
* Disable `TransactionID` related functions from `state_db.nim`
why:
Functions `getCommittedStorage()` and `updateOriginalRoot()` from
the `state_db` module are nowhere used. The emulation of a legacy
`TransactionID` type functionality is administratively expensive to
provide by `Aristo` (the legacy DB version is only partially
implemented, anyway).
As there is no other place where `TransactionID`s are used, they will
not be provided by the `Aristo` variant of the `CoreDb`. For the
legacy DB API, nothing will change.
* Fix copyright headers in source code
* Get rid of compiler warning
* Update Aristo code, remove unused `merge()` variant, export `hashify()`
why:
Adapt to upcoming `CoreDb` wrapper
* Remove synced tx feature from `Aristo`
why:
+ This feature allowed to synchronise transaction methods like begin,
commit, and rollback for a group of descriptors.
+ The feature is over engineered and not needed for `CoreDb`, neither
is it complete (some convergence features missing.)
* Add debugging helpers to `Kvt`
also:
Update database iterator, add count variable yield argument similar
to `Aristo`.
* Provide optional destructors for `CoreDb` API
why;
For the upcoming Aristo wrapper, this allows to control when certain
smart destruction and update can take place. The auto destructor works
fine in general when the storage/cache strategy is known and acceptable
when creating descriptors.
* Add update option for `CoreDb` API function `hash()`
why;
The hash function is typically used to get the state root of the MPT.
Due to lazy hashing, this might be not available on the `Aristo` DB.
So the `update` function asks for re-hashing the gurrent state changes
if needed.
* Update API tracking log mode: `info` => `debug
* Use shared `Kvt` descriptor in new Ledger API
why:
No need to create a new descriptor all the time
2023-11-16 19:35:03 +00:00
|
|
|
# Copyright (c) 2018-2023 Status Research & Development GmbH
|
2019-02-01 09:25:10 +00:00
|
|
|
# Licensed under either of
|
|
|
|
|
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
|
|
|
|
|
# * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
|
|
|
|
|
# at your option. This file may not be copied, modified, or distributed except according to those terms.
|
|
|
|
|
|
2023-06-12 04:29:03 +00:00
|
|
|
import
|
2023-08-04 11:10:09 +00:00
|
|
|
eth/trie/trie_defs,
|
|
|
|
|
stew/[byteutils, endians2],
|
2023-06-12 04:29:03 +00:00
|
|
|
unittest2,
|
Optional accounts cache module for creating genesis (#1897)
* Split off `ReadOnlyStateDB` from `AccountStateDB` from `state_db.nim`
why:
Apart from testing, applications use `ReadOnlyStateDB` as an easy
way to access the accounts ledger. This is well supported by the
`Aristo` db, but writable mode is only parially supported.
The writable AccountStateDB` object for modifying accounts is not
used by production code.
So, for lecgacy and testing apps, the full support of the previous
`AccountStateDB` is now enabled by `import db/state_db/read_write`
and the `import db/state_db` provides read-only mode.
* Encapsulate `AccountStateDB` as `GenesisLedgerRef` or genesis creation
why:
`AccountStateDB` has poor support for `Aristo` and is not widely used
in favour of `AccountsLedger` (which will be abstracted as `ledger`.)
Currently, using other than the `AccountStateDB` ledgers within the
`GenesisLedgerRef` wrapper is experimental and test only. Eventually,
the wrapper should disappear so that the `Ledger` object (which
encapsulates `AccountsCache` and `AccountsLedger`) will prevail.
* For the `Ledger`, provide access to raw accounts `MPT`
why:
This gives to the `CoreDbMptRef` descriptor from the `CoreDb` (which is
the legacy version of CoreDxMptRef`.) For the new `ledger` API, the
accounts are based on the `CoreDxMAccRef` descriptor which uses a
particular sub-system for accounts while legacy applications use the
`CoreDbPhkRef` equivalent of the `SecureHexaryTrie`.
The only place where this feature will currently be used is the
`genesis.nim` source file.
* Fix `Aristo` bugs, missing boundary checks, typos, etc.
* Verify root vertex in `MPT` and account constructors
why:
Was missing so far, in particular the accounts constructor must
verify `VertexID(1)
* Fix include file
2023-11-20 11:51:43 +00:00
|
|
|
../nimbus/db/state_db/read_write
|
2019-02-01 09:25:10 +00:00
|
|
|
|
2023-11-08 16:52:25 +00:00
|
|
|
include ../nimbus/db/ledger/accounts_cache
|
2020-01-07 12:49:42 +00:00
|
|
|
|
2020-02-26 07:30:19 +00:00
|
|
|
func initAddr(z: int): EthAddress =
|
2020-12-09 05:24:37 +00:00
|
|
|
const L = sizeof(result)
|
|
|
|
|
result[L-sizeof(uint32)..^1] = toBytesBE(z.uint32)
|
2020-02-26 07:30:19 +00:00
|
|
|
|
2019-09-21 05:45:23 +00:00
|
|
|
proc stateDBMain*() =
|
|
|
|
|
suite "Account State DB":
|
2019-05-30 16:42:55 +00:00
|
|
|
setup:
|
2020-07-21 06:15:06 +00:00
|
|
|
const emptyAcc {.used.} = newAccount()
|
2020-02-26 07:30:19 +00:00
|
|
|
|
2019-05-30 16:42:55 +00:00
|
|
|
var
|
2023-08-04 11:10:09 +00:00
|
|
|
memDB = newCoreDbRef LegacyDbMemory
|
|
|
|
|
acDB {.used.} = newCoreDbRef LegacyDbMemory
|
|
|
|
|
trie = memDB.mptPrune()
|
2020-07-21 06:15:06 +00:00
|
|
|
stateDB {.used.} = newAccountStateDB(memDB, trie.rootHash, true)
|
|
|
|
|
address {.used.} = hexToByteArray[20]("0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6")
|
|
|
|
|
code {.used.} = hexToSeqByte("0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6")
|
|
|
|
|
rootHash {.used.} : KeccakHash
|
2019-09-21 05:45:23 +00:00
|
|
|
|
|
|
|
|
test "accountExists and isDeadAccount":
|
|
|
|
|
check stateDB.accountExists(address) == false
|
|
|
|
|
check stateDB.isDeadAccount(address) == true
|
|
|
|
|
|
|
|
|
|
var acc = stateDB.getAccount(address)
|
|
|
|
|
acc.balance = 1000.u256
|
|
|
|
|
stateDB.setAccount(address, acc)
|
|
|
|
|
|
|
|
|
|
check stateDB.accountExists(address) == true
|
|
|
|
|
check stateDB.isDeadAccount(address) == false
|
|
|
|
|
|
|
|
|
|
acc.balance = 0.u256
|
|
|
|
|
acc.nonce = 1
|
|
|
|
|
stateDB.setAccount(address, acc)
|
|
|
|
|
check stateDB.isDeadAccount(address) == false
|
|
|
|
|
|
2020-02-26 07:30:19 +00:00
|
|
|
stateDB.setCode(address, code)
|
2019-09-21 05:45:23 +00:00
|
|
|
stateDB.setNonce(address, 0)
|
|
|
|
|
check stateDB.isDeadAccount(address) == false
|
|
|
|
|
|
2020-04-20 18:12:44 +00:00
|
|
|
stateDB.setCode(address, [])
|
2019-09-21 05:45:23 +00:00
|
|
|
check stateDB.isDeadAccount(address) == true
|
|
|
|
|
check stateDB.accountExists(address) == true
|
2020-01-07 12:49:42 +00:00
|
|
|
|
|
|
|
|
test "clone storage":
|
|
|
|
|
var x = RefAccount(
|
2023-04-24 20:59:38 +00:00
|
|
|
overlayStorage: initTable[UInt256, UInt256](),
|
2020-01-07 12:49:42 +00:00
|
|
|
originalStorage: newTable[UInt256, UInt256]()
|
|
|
|
|
)
|
|
|
|
|
|
2023-04-24 20:59:38 +00:00
|
|
|
x.overlayStorage[10.u256] = 11.u256
|
|
|
|
|
x.overlayStorage[11.u256] = 12.u256
|
2020-01-07 12:49:42 +00:00
|
|
|
|
|
|
|
|
x.originalStorage[10.u256] = 11.u256
|
|
|
|
|
x.originalStorage[11.u256] = 12.u256
|
|
|
|
|
|
|
|
|
|
var y = x.clone(cloneStorage = true)
|
2023-04-24 20:59:38 +00:00
|
|
|
y.overlayStorage[12.u256] = 13.u256
|
2020-01-07 12:49:42 +00:00
|
|
|
y.originalStorage[12.u256] = 13.u256
|
|
|
|
|
|
|
|
|
|
check 12.u256 notin x.overlayStorage
|
|
|
|
|
check 12.u256 in y.overlayStorage
|
|
|
|
|
|
|
|
|
|
check x.overlayStorage.len == 2
|
|
|
|
|
check y.overlayStorage.len == 3
|
|
|
|
|
|
|
|
|
|
check 12.u256 in x.originalStorage
|
|
|
|
|
check 12.u256 in y.originalStorage
|
|
|
|
|
|
|
|
|
|
check x.originalStorage.len == 3
|
|
|
|
|
check y.originalStorage.len == 3
|
|
|
|
|
|
|
|
|
|
test "accounts cache":
|
2020-02-26 07:30:19 +00:00
|
|
|
var ac = init(AccountsCache, acDB, emptyRlpHash, true)
|
2020-01-07 12:49:42 +00:00
|
|
|
var addr1 = initAddr(1)
|
|
|
|
|
|
|
|
|
|
check ac.isDeadAccount(addr1) == true
|
|
|
|
|
check ac.accountExists(addr1) == false
|
|
|
|
|
check ac.hasCodeOrNonce(addr1) == false
|
|
|
|
|
|
|
|
|
|
ac.setBalance(addr1, 1000.u256)
|
|
|
|
|
check ac.getBalance(addr1) == 1000.u256
|
|
|
|
|
ac.subBalance(addr1, 100.u256)
|
|
|
|
|
check ac.getBalance(addr1) == 900.u256
|
|
|
|
|
ac.addBalance(addr1, 200.u256)
|
|
|
|
|
check ac.getBalance(addr1) == 1100.u256
|
|
|
|
|
|
|
|
|
|
ac.setNonce(addr1, 1)
|
|
|
|
|
check ac.getNonce(addr1) == 1
|
|
|
|
|
ac.incNonce(addr1)
|
|
|
|
|
check ac.getNonce(addr1) == 2
|
|
|
|
|
|
|
|
|
|
ac.setCode(addr1, code)
|
|
|
|
|
check ac.getCode(addr1) == code
|
|
|
|
|
|
|
|
|
|
ac.setStorage(addr1, 1.u256, 10.u256)
|
|
|
|
|
check ac.getStorage(addr1, 1.u256) == 10.u256
|
|
|
|
|
check ac.getCommittedStorage(addr1, 1.u256) == 0.u256
|
|
|
|
|
|
|
|
|
|
check ac.hasCodeOrNonce(addr1) == true
|
|
|
|
|
check ac.getCodeSize(addr1) == code.len
|
2020-02-26 07:30:19 +00:00
|
|
|
|
|
|
|
|
ac.persist()
|
|
|
|
|
rootHash = ac.rootHash
|
|
|
|
|
|
|
|
|
|
var db = newAccountStateDB(memDB, emptyRlpHash, true)
|
|
|
|
|
db.setBalance(addr1, 1100.u256)
|
|
|
|
|
db.setNonce(addr1, 2)
|
|
|
|
|
db.setCode(addr1, code)
|
|
|
|
|
db.setStorage(addr1, 1.u256, 10.u256)
|
|
|
|
|
check rootHash == db.rootHash
|
|
|
|
|
|
|
|
|
|
test "accounts cache readonly operations":
|
|
|
|
|
# use previous hash
|
|
|
|
|
var ac = init(AccountsCache, acDB, rootHash, true)
|
|
|
|
|
var addr2 = initAddr(2)
|
|
|
|
|
|
|
|
|
|
check ac.getCodeHash(addr2) == emptyAcc.codeHash
|
|
|
|
|
check ac.getBalance(addr2) == emptyAcc.balance
|
|
|
|
|
check ac.getNonce(addr2) == emptyAcc.nonce
|
2020-04-20 18:12:44 +00:00
|
|
|
check ac.getCode(addr2) == []
|
2020-02-26 07:30:19 +00:00
|
|
|
check ac.getCodeSize(addr2) == 0
|
|
|
|
|
check ac.getCommittedStorage(addr2, 1.u256) == 0.u256
|
|
|
|
|
check ac.getStorage(addr2, 1.u256) == 0.u256
|
|
|
|
|
check ac.hasCodeOrNonce(addr2) == false
|
|
|
|
|
check ac.accountExists(addr2) == false
|
|
|
|
|
check ac.isDeadAccount(addr2) == true
|
|
|
|
|
|
|
|
|
|
ac.persist()
|
|
|
|
|
# readonly operations should not modify
|
|
|
|
|
# state trie at all
|
|
|
|
|
check ac.rootHash == rootHash
|
|
|
|
|
|
2020-04-29 05:00:44 +00:00
|
|
|
test "accounts cache code retrieval after persist called":
|
|
|
|
|
var ac = init(AccountsCache, acDB)
|
|
|
|
|
var addr2 = initAddr(2)
|
|
|
|
|
ac.setCode(addr2, code)
|
|
|
|
|
ac.persist()
|
2020-07-21 06:15:06 +00:00
|
|
|
check ac.getCode(addr2) == code
|
2022-09-03 18:15:35 +00:00
|
|
|
let key = contractHashKey(keccakHash(code))
|
2023-08-04 11:10:09 +00:00
|
|
|
check acDB.kvt.get(key.toOpenArray) == code
|
2020-07-21 06:15:06 +00:00
|
|
|
|
2020-12-09 05:24:37 +00:00
|
|
|
test "accessList operations":
|
|
|
|
|
proc verifyAddrs(ac: AccountsCache, addrs: varargs[int]): bool =
|
|
|
|
|
for c in addrs:
|
|
|
|
|
if not ac.inAccessList(c.initAddr):
|
|
|
|
|
return false
|
|
|
|
|
true
|
|
|
|
|
|
|
|
|
|
proc verifySlots(ac: AccountsCache, address: int, slots: varargs[int]): bool =
|
|
|
|
|
let a = address.initAddr
|
|
|
|
|
if not ac.inAccessList(a):
|
|
|
|
|
return false
|
|
|
|
|
|
|
|
|
|
for c in slots:
|
|
|
|
|
if not ac.inAccessList(a, c.u256):
|
|
|
|
|
return false
|
|
|
|
|
true
|
|
|
|
|
|
2023-12-12 19:12:56 +00:00
|
|
|
proc accessList(ac: AccountsCache, address: int) {.inline.} =
|
2020-12-09 05:24:37 +00:00
|
|
|
ac.accessList(address.initAddr)
|
|
|
|
|
|
2023-12-12 19:12:56 +00:00
|
|
|
proc accessList(ac: AccountsCache, address, slot: int) {.inline.} =
|
2020-12-09 05:24:37 +00:00
|
|
|
ac.accessList(address.initAddr, slot.u256)
|
|
|
|
|
|
|
|
|
|
var ac = init(AccountsCache, acDB)
|
|
|
|
|
|
|
|
|
|
ac.accessList(0xaa)
|
|
|
|
|
ac.accessList(0xbb, 0x01)
|
|
|
|
|
ac.accessList(0xbb, 0x02)
|
|
|
|
|
check ac.verifyAddrs(0xaa, 0xbb)
|
|
|
|
|
check ac.verifySlots(0xbb, 0x01, 0x02)
|
|
|
|
|
check ac.verifySlots(0xaa, 0x01) == false
|
|
|
|
|
check ac.verifySlots(0xaa, 0x02) == false
|
|
|
|
|
|
|
|
|
|
var sp = ac.beginSavepoint
|
|
|
|
|
# some new ones
|
|
|
|
|
ac.accessList(0xbb, 0x03)
|
|
|
|
|
ac.accessList(0xaa, 0x01)
|
|
|
|
|
ac.accessList(0xcc, 0x01)
|
|
|
|
|
ac.accessList(0xcc)
|
|
|
|
|
|
|
|
|
|
check ac.verifyAddrs(0xaa, 0xbb, 0xcc)
|
|
|
|
|
check ac.verifySlots(0xaa, 0x01)
|
|
|
|
|
check ac.verifySlots(0xbb, 0x01, 0x02, 0x03)
|
|
|
|
|
check ac.verifySlots(0xcc, 0x01)
|
|
|
|
|
|
|
|
|
|
ac.rollback(sp)
|
|
|
|
|
check ac.verifyAddrs(0xaa, 0xbb)
|
|
|
|
|
check ac.verifyAddrs(0xcc) == false
|
|
|
|
|
check ac.verifySlots(0xcc, 0x01) == false
|
|
|
|
|
|
|
|
|
|
sp = ac.beginSavepoint
|
|
|
|
|
ac.accessList(0xbb, 0x03)
|
|
|
|
|
ac.accessList(0xaa, 0x01)
|
|
|
|
|
ac.accessList(0xcc, 0x01)
|
|
|
|
|
ac.accessList(0xcc)
|
|
|
|
|
ac.accessList(0xdd, 0x04)
|
|
|
|
|
ac.commit(sp)
|
|
|
|
|
|
|
|
|
|
check ac.verifyAddrs(0xaa, 0xbb, 0xcc)
|
|
|
|
|
check ac.verifySlots(0xaa, 0x01)
|
|
|
|
|
check ac.verifySlots(0xbb, 0x01, 0x02, 0x03)
|
|
|
|
|
check ac.verifySlots(0xcc, 0x01)
|
|
|
|
|
check ac.verifySlots(0xdd, 0x04)
|
|
|
|
|
|
2023-03-21 13:27:12 +00:00
|
|
|
test "transient storage operations":
|
|
|
|
|
var ac = init(AccountsCache, acDB)
|
|
|
|
|
|
|
|
|
|
proc tStore(ac: AccountsCache, address, slot, val: int) =
|
|
|
|
|
ac.setTransientStorage(address.initAddr, slot.u256, val.u256)
|
|
|
|
|
|
|
|
|
|
proc tLoad(ac: AccountsCache, address, slot: int): UInt256 =
|
|
|
|
|
ac.getTransientStorage(address.initAddr, slot.u256)
|
|
|
|
|
|
|
|
|
|
proc vts(ac: AccountsCache, address, slot, val: int): bool =
|
|
|
|
|
ac.tLoad(address, slot) == val.u256
|
|
|
|
|
|
|
|
|
|
ac.tStore(0xaa, 3, 66)
|
|
|
|
|
ac.tStore(0xbb, 1, 33)
|
|
|
|
|
ac.tStore(0xbb, 2, 99)
|
|
|
|
|
|
|
|
|
|
check ac.vts(0xaa, 3, 66)
|
|
|
|
|
check ac.vts(0xbb, 1, 33)
|
|
|
|
|
check ac.vts(0xbb, 2, 99)
|
|
|
|
|
check ac.vts(0xaa, 1, 33) == false
|
|
|
|
|
check ac.vts(0xbb, 1, 66) == false
|
|
|
|
|
|
|
|
|
|
var sp = ac.beginSavepoint
|
|
|
|
|
# some new ones
|
|
|
|
|
ac.tStore(0xaa, 3, 77)
|
|
|
|
|
ac.tStore(0xbb, 1, 55)
|
|
|
|
|
ac.tStore(0xcc, 7, 88)
|
|
|
|
|
|
|
|
|
|
check ac.vts(0xaa, 3, 77)
|
|
|
|
|
check ac.vts(0xbb, 1, 55)
|
|
|
|
|
check ac.vts(0xcc, 7, 88)
|
|
|
|
|
|
|
|
|
|
check ac.vts(0xaa, 3, 66) == false
|
|
|
|
|
check ac.vts(0xbb, 1, 33) == false
|
|
|
|
|
check ac.vts(0xbb, 2, 99)
|
|
|
|
|
|
|
|
|
|
ac.rollback(sp)
|
|
|
|
|
check ac.vts(0xaa, 3, 66)
|
|
|
|
|
check ac.vts(0xbb, 1, 33)
|
|
|
|
|
check ac.vts(0xbb, 2, 99)
|
|
|
|
|
check ac.vts(0xcc, 7, 88) == false
|
|
|
|
|
|
|
|
|
|
sp = ac.beginSavepoint
|
|
|
|
|
ac.tStore(0xaa, 3, 44)
|
|
|
|
|
ac.tStore(0xaa, 4, 55)
|
|
|
|
|
ac.tStore(0xbb, 1, 22)
|
|
|
|
|
ac.tStore(0xdd, 2, 66)
|
|
|
|
|
|
|
|
|
|
ac.commit(sp)
|
|
|
|
|
check ac.vts(0xaa, 3, 44)
|
|
|
|
|
check ac.vts(0xaa, 4, 55)
|
|
|
|
|
check ac.vts(0xbb, 1, 22)
|
|
|
|
|
check ac.vts(0xbb, 1, 55) == false
|
|
|
|
|
check ac.vts(0xbb, 2, 99)
|
|
|
|
|
check ac.vts(0xcc, 7, 88) == false
|
|
|
|
|
check ac.vts(0xdd, 2, 66)
|
|
|
|
|
|
|
|
|
|
ac.clearTransientStorage()
|
|
|
|
|
check ac.vts(0xaa, 3, 44) == false
|
|
|
|
|
check ac.vts(0xaa, 4, 55) == false
|
|
|
|
|
check ac.vts(0xbb, 1, 22) == false
|
|
|
|
|
check ac.vts(0xbb, 1, 55) == false
|
|
|
|
|
check ac.vts(0xbb, 2, 99) == false
|
|
|
|
|
check ac.vts(0xcc, 7, 88) == false
|
|
|
|
|
check ac.vts(0xdd, 2, 66) == false
|
|
|
|
|
|
2020-01-07 12:49:42 +00:00
|
|
|
when isMainModule:
|
|
|
|
|
stateDBMain()
|
