mirror of
https://github.com/status-im/nimbus-eth1.git
synced 2025-02-04 00:05:22 +00:00
9521582005
An instance of `CoreDbMptRef` is created for and stored in every account - when we are processing blocks and have many accounts in memory, this closure environment takes up hundreds of mb of memory (around block 5M, it is the 4:th largest memory consumer!) - incidentally, this also removes a circular reference in the setup that causes the `AristoCodeDbMptRef` to linger in memory much longer than it has to which is the core reason why it takes so much. The real solution here is to remove the methods indirection entirely, but this PR provides relief until that has been done. Similar treatment is given to some of the other core api functions to avoid circulars there too.
Core database replacement wrapper object
This wrapper replaces the TrieDatabaseRef and its derivatives by the new object CoreDbRef.
Relations to current TrieDatabaseRef implementation
Here are some incomplete translations for objects and constructors.
Object types:
| Legacy notation | CoreDbRef based replacement |
|---|---|
| ChainDB | (don't use/avoid) |
| ChainDbRef | CoreDbRef |
| TrieDatabaseRef | CoreDbKvtRef |
| HexaryTrie | CoreDbMptRef |
| SecureHexaryTrie | CoreDbPhkRef |
| DbTransaction | CoreDbTxRef |
| TransactionID | CoreDbTxID |
Constructors:
| Legacy notation | CoreDbRef based replacement |
|---|---|
| trieDB newChainDB("..") | newCoreDbRef(LegacyDbPersistent,"..") |
| newMemoryDB() | newCoreDbRef(LegacyDbMemory) |
| -- | |
| initHexaryTrie(db,..) | db.mpt(..) (no pruning) |
| db.mptPrune(..) (w/pruning true/false) | |
| -- | |
| initSecureHexaryTrie(db,..) | db.phk(..) (no pruning) |
| db.phkPrune(..) (w/pruning true/false) | |
| -- | |
| newCaptureDB(db,memDB) | db.capture() (see below) |
Usage of the replacement wrapper
Objects pedigree:
CoreDbRef -- base descriptor
| | |
| | +--- CoreDbCtxRef -- MPT context descriptor
| | | |
| | | +-- CoreDbMptRef -- hexary trie instance
| | | | : :
| | | +-- CoreDbMptRef -- hexary trie instance
| | |
| | |
| | +---- CoreDbPhkRef -- pre-hashed key hexary trie instance
| | | : :
| | +---- CoreDbPhkRef -- pre-hashed key hexary trie instance
| |
| |
| +------ CoreDbKvtRef -- single static key-value table
|
|
+-------- CoreDbCaptRef -- tracer support descriptor
Instantiating legacy standard database object descriptors works as follows:
let
db = newCoreDbRef(..) # new base descriptor
mpt = db.mpt(..) # hexary trie/Merkle Patricia Tree
phk = db.phk(..) # pre-hashed key hexary trie/MPT
kvt = db.kvt # key-value table
Tracer support setup by hiding the current CoreDbRef behind a replacement:
let
capture = db.capture()
db = capture.recorder # use the recorder in place of db
...
for key,value in capture.recorder.kvt:
... # process recorded data