* Update TDD suite logger output format choices
why:
New format is not practical for TDD as it just dumps data across a wide
range (considerably larder than 80 columns.)
So the new format can be turned on by function argument.
* Update unit tests samples configuration
why:
Slightly changed the way to find the `era1` directory
* Remove compiler warnings (fix deprecated expressions and phrases)
* Update `Aristo` debugging tools
* Always update the `storageID` field of account leaf vertices
why:
Storage tries are weekly linked to an account leaf object in that
the `storageID` field is updated by the application.
Previously, `Aristo` verified that leaf objects make sense when passed
to the database. As a consequence
* the database was inconsistent for a short while
* the burden for correctness was all on the application which led
to delayed error handling which is hard to debug.
So `Aristo` will internally update the account leaf objects so that
there are no race conditions due to the storage trie handling
* Aristo: Let `stow()`/`persist()` bail out unless there is a `VertexID(1)`
why:
The journal and filter logic depends on the hash of the `VertexID(1)`
which is commonly known as the state root. This implies that all
changes to the database are somehow related to that.
* Make sure that a `Ledger` account does not overwrite the storage trie reference
why:
Due to the abstraction of a sub-trie (now referred to as column with a
hash describing its state) there was a weakness in the `Aristo` handler
where an account leaf could be overwritten though changing the validity
of the database. This has been changed and the database will now reject
such changes.
This patch fixes the behaviour on the application layer. In particular,
the column handle returned by the `CoreDb` needs to be updated by
the `Aristo` database state. This mitigates the problem that a storage
trie might have vanished or re-apperaed with a different vertex ID.
* Fix sub-trie deletion test
why:
Was originally hinged on `VertexID(1)` which cannot be wholesale
deleted anymore after the last Aristo update. Also, running with
`VertexID(2)` needs an artificial `VertexID(1)` for making `stow()`
or `persist()` work.
* Cosmetics
* Activate `test_generalstate_json`
* Temporarily `deactivate test_tracer_json`
* Fix copyright header
---------
Co-authored-by: jordan <jordan@dry.pudding>
Co-authored-by: Jacek Sieka <jacek@status.im>
* Implement engine_getClientVersionV1
* full git revision string
* Limit GitRevisionString to 8 chars
* Fixes
* Debug windows CI
* debug windows ci
* produce git revision using -C
* try not to delete .git folder in windows ci
* Harden GitRevision procuration
* Add double quotes to git -C param
* Escape sourcePath
* Remove double quotes from git -C param
* Remove crufty `pruneTrie` arguments
* Replaced legacy `distinct_trie` logic by new `ledger` functionality
why:
The module `distinct_trie` is supported by `Aristo` in trivial cases.
* Activate `test_op_memory`
* Workaround for disallowed transaction superseding
The transaction spammer from Kurtosis keeps spamming transactions with
the same nonce because report 'pending' account nocne based on 'latest'
rather than actually considering the mempool. To avoid errors from
rejecting those, disable the required gas price bump when replacing
a transaction with a new nonce.
* Lint
* Skip superseding negative test
* Workaround for 0 gas price estimation
The transaction spammer from Kurtosis estimates the gas price of its
transactions using 'eth_gasPrice'. Our implementation returns 0 when
no transactions have been executed yet, not taking into account the
EIP-1559 block base fee. Force a hardcoded minimum for now to unstuck.
* Adjust tests to cover new minimum gas fee
* Skip gas price test
This PR exploits structural properties of era files to simplify the
implementation and in particular remove the need to load all era file
indicies at startup which may be slow (due to archival storage residing
on slow drives)
* Attempt to roll back stateless mode implementation in a single PR
why:
+ Stateless mode is not fully working and in the way
+ Single PR should make it feasible to investigate for a possible
re-implementation
* Fix copyright year
* Fix annotation for exception (evmc mode)
* Update some docu & messages
* Remove cruft from the ledger modules
* Must not overwrite genesis data on an initialised database
why:
This will overwrite the global state of the Aristo single state DB.
Otherwise resuming at the last synced state becomes impossible.
* Provide latest block number from journal
why:
This relates the global state of the DB directly to the corresponding
block number.
* Implemented unit test providing DB pre-load and resume
why:
When deleting accounts while restoring the previous state, storage tries
must be deleted first. Otherwise a `DelDanglingStoTrie` error will occur
when trying to delete an account which refers to an active storage trie.
* Introduce wrapper type for EIP-4844 transactions
EIP-4844 blob sidecars are a concept that only exists in the mempool.
After inclusion of a transaction into an execution block, only the
versioned hash within the transaction remains. To improve type safety,
replace the `Transaction.networkPayload` member with a wrapper type
`PooledTransaction` that is used in contexts where blob sidecars exist.
* Bump nimbus-eth2 to 87605d08a7f9cfc3b223bd32143e93a6cdf351ac
* IPv6 'listen-address' in `nimbus_verified_proxy`
* Bump nim-libp2p to 21cbe3a91a70811522554e89e6a791172cebfef2
* Fix beacon_lc_bridge payload conversion and conf.listenAddress type
* Change nimbus_verified_proxy.asExecutionData param to SomeExecutionPayload
* Rerun nph to fix asExecutionData style format
* nimbus_verified_proxy listenAddress
* Use PooledTransaction in nimbus-eth1 tests
---------
Co-authored-by: jangko <jangko128@gmail.com>
The transaction spammer from Kurtosis requests the nonce value of its
account based on 'pending' tag. As we lack that implementation, go with
the next best answer from the 'latest' tag. This does not include info
about transactions from the mempool that have not yet been executed.
* Aristo: Generalise alien/guest interface for piggiback on database
* Aristo: Code cosmetics
* CoreDb+Kvt: Update transaction API
why:
Use single addressable function `forkTx(backLevel: int)` as used
in `Aristo`. So `Kvt` can be synced simultaneously to `Aristo`.
also:
Refactored `kvt_tx.nim` in a similar fashion to `Aristo`.
* Kvt: Replace `LayerDelta` object by reference
why:
Will be needed when introducing filters
* Kvt: Remodel backend filter facility similar to `Aristo`
why:
This allows to operate on several KVT instances simultaneously.
* CoreDb+Kvt: Fix on-disk storage
why:
Overlooked name change: `stow()` => `persist()` for permanent storage
* Fix copyright headers
* Aristo: Rename journal related sources and functions
why:
Previously, the naming was hinged on the phrases `fifo`, `filter` etc.
which reflect the inner workings of cascaded filters. This was
unfortunate for reading/understanding the source code for actions where
the focus is the journal as a whole.
* Aristo: Fix buffer overflow (path length truncating error)
* Aristo: Tighten `hikeUp()` stop check, update error code
why:
Detect dangling vertex links. These are legit with `snap` sync
processing but not with regular processing.
* Aristo: Raise assert in regular mode `merge()` at a dangling link/edge
why:
With `snap` sync processing, partial trees are ok and can be amended.
Not so in regular mode.
Previously there was only a debug message when a non-legit dangling edge
was encountered.
* Aristo: Make sure that vertices are copied before modification
why:
Otherwise vertices from lower layers might also be modified
* Aristo: Fix relaxed mode for validity checker `check()`
* Remove cruft
* Aristo: Update API for transaction handling
details:
+ Split `aristo_tx.nim` into sub-modules
+ Split `forkWith()` into `findTx()` + `forkTx()`
+ Removed `forkTop()`, `forkBase()` (now superseded by new `forkTx()`)
* CoreDb+Aristo: Fix initialiser (missing methods)
* Aristo: Allow to define/set `FilterID` for journal filter records
why:
After some changes, the `FilterID` is isomorphic to the `BlockNumber`
scalar (well, the first 2^64 entries of a `BlockNumber`.)
The needed change for `FilterID` is that the `FilterID(0)` value is
valid part of the `FilterID` scalar. A non-valid `FilterID` entry is
represented by `none(FilterID)`.
* Aristo: Split off function `persist()` as persistent version of `stow()`
why:
In production, `stow(persistent=false,..)` is currently unused. So,
using `persist()` rather than `stow(persistent=true,..)` improves
readability and is better to maintain.
* CoreDb+Aristo: Store block numbers in journal records
why:
This makes journal records searchable by block numbers
* Aristo: Rename some journal related functions
why:
The name *journal* is more appropriate to api functions than something
with *fifo* or *filter*.
* CoreDb+Aristo: Update last/oldest journal state retrieval
* CoreDb+Aristo: Register block number with state root in journal
why:
No need anymore for extra lookup table `stRootToBlockNum` which maps
a storage root -> block number.
* Aristo: Remove unused function `getFilUbe()` from api
* CoreDb: Remove now unused virtual table `stRootToBlockNum`
why:
Was used to map a state root to a block number. This functionality
is now embedded into the recovery journal backend.
* Turn of API tracking (will fail on `fluffy`)
* Aristo: Code cosmetics, e.g. update some CamelCase names
* CoreDb+Aristo: Provide oldest known state root implied
details:
The Aristo journal allows to recover earlier but not all state roots.
* Aristo: Fix journal backward index operator, e.g. `[^1]`
* Aristo: Fix journal updater
why:
The `fifosStore()` store function slightly misinterpreted the update
instructions when translation is to database `put()` functions. The
effect was that the journal was ever growing due to stale entries which
were never deleted.
* CoreDb+Aristo: Provide utils for purging stale data from the KVT
details:
See earlier patch, not all state roots are available. This patch
provides a mapping from some state root to a block number and allows to
remove all KVT data related to a particular block number
* Aristo+Kvt: Implement a clean up schedule for expired data in KVT
why:
For a single state ledger like `Aristo`, there is only a limited
backlog of states. So KVT data (i.e. headers etc.) are cleaned up
regularly
* Fix copyright year
* Aristo+Kvt: Better RocksDB profiling
why:
Providing more detailed information, mainly for `Aristo`
* Aristo: Renamed journal `stats()` to `capacity()`
why:
`Stats()` was a misnomer
* Aristo: Provide backend read caches for key and vertex IDs
why:
Dedicated LRU caching for particular types gives a throughput advantage.
The sizes of the LRU queues used for caching are currently constant
but might be adjusted at a later time.
* Fix copyright year
tersec
Jordan Hrycaj
Jacek Sieka
andri lim
Etan Kissling