* Fix 'value out of range' RangeDefect caused by large/expensive blocks/transactions during DOS period.
* Clear witness cache in AccountCache persist.
* Revert previous fix and force clear cache after processing each block.
* Revert clear cache in process block.
* CoreDb+Ledger: Update logging
why:
Use symbol `api` rather than `ctx` because the latter will be used
as name for particular objects
* CoreDb: Remove cruft
* CoreDb: Remove `TxID` support
why:
It is nowhere used and ugly implemented. The upcoming context layer
will be a cleaner alternative to use, instead should this particular
functionality be needed.
* CoreDb: Rearrange base methods in source code for better reading
* CoreDb+Aristo: Update API closures for better reading & maintenance
* CoreDb: Implement context layer for MPT
why:
On `Aristo` the context layer allows to manage different views on
the same backend database. This is an abstraction of the legacy
hexary trie which can be localised on a particular root nose.
details:
The `ctx` context provides the state (equiv. to state root) of the
database for MPT and account descriptors.
* Fix Copyright headers
why:
Ignoring `nil` objects was handy for a while but eventually led to
lazy programming which in turn led to double destructor calls for
the rocks-db.
* Aristo+Kvt: Fix backend `dup()` function in api setup
why:
Backend object is subject to an inheritance cascade which was not
taken care of, before. Only the base object was duplicated.
* Kvt: Simplify DB clone/peers management
* Aristo: Simplify DB clone/peers management
* Aristo: Adjust unit test for working with memory DB only
why:
This currently causes some memory corruption persumably in the
`libc` background layer.
* CoredDb+Kvt: Simplify API for KVT
why:
Simplified storage models (was over engineered) for better performance
and code maintenance.
* CoredDb+Aristo: Simplify API for `Aristo`
why:
Only single database state needed here. Accessing a similar state will
be implemented from outside this module using a context layer. This
gives better performance and improves code maintenance.
* Fix Copyright headers
* CoreDb: Turn off API tracking
why:
CI would ot go through. Was accidentally turned on.
* Add new RocksNamespaceRef type and remove backups and readonly support from RocksDb KvStore.
* Bump nim-rocksdb to fc2ba4a836b6b47ae1b17d1c45801c7e06585e19
* Fix tests.
* Fix copyright notice.
* Aristo: Remove cruft
* Prettifying profile statistics printing & source code cosmetics
* Aristo/Kvt: API tools update
* CoreDb: Corrections, mostly API related
* CoreDb: Changed the tracer `logDb()` result
why:
Only the key-value table is used from the logger database
* CoreDb: Update legacy tracer
why:
The `contains()` directive did not correspond to `0 < get().len`
also:
Copy `CoreDb` meta settings like `trackLegaApi`, `trackNewApi`, etc. to
overlay tracer descriptor
* CoreDb: Extend/update tracer API
why:
Get ready for accommodating `Aristo` tracer
* Fix missing import
why:
Some CI compilers might have (cached?) a different NIM patch level
* Ditto
* Aristo/Kvt: Provide function hooks APIs
why:
These APIs can be used for installing tracers, profiling functoinality,
and other niceties on the databases.
* Aristo: Provide optional API profiling
details:
It basically is a re-implementation of the `CoreDb` profiling
implementation
* Kvt: Provide optional API profiling similar to `Aristo`
* CoreDb: Re-implementing profiling using `aristo_profile`
* Ledger: Re-implementing profiling using `aristo_profile`
* CoreDb: Update unit tests for maintainability
* update copyright dates
* CoreDb: Provide default db backend symbols
why:
Handy for running `Aristo` against standard tests
note:
These defaults are currently set to legacy DB types. The must be
enabled manually in `db/core_db.nim`.
* Provide `Aristo` for macro assembler related tests
caveat:
Some tests use `initStorageTrie()` which lets `Aristo` bail out. The
test need to run on `distinct_ledgers` (or something like) rather than
`distinct_tries`.
* Tests: Misc modules that can run on `Aristo` as well
* NoHive: Module that can run on `Aristo` as well
* Fix copyright year
* ditto
* Aristo: Reorg `hashify()` using different schedule algorithm
why:
Directly calculating the search tree top down from the roots turns
out to be faster than using the cached structures left over by `merge()`
and `delete()`.
Time gains is short of 20%
* Aristo: Remove `lTab[]` leaf entry object type
why:
Not used anymore. It was previously needed to build the schedule for
`hashify()`.
* Aristo: Avoid unnecessary re-org of the vertex ID recycling list
why:
This list can become quite large so a heuristic is employed whether
it makes sense to re-org.
Also, re-org check is only done by `delete()` functions.
* Aristo: Remove key/reverse lookup table from tx layers
why:
It is ignored except for handling proof nodes and costs unnecessary
run time resources.
This feature was originally needed to accommodate the mental transition
from the legacy MPT to the `Aristo` trie :).
* Fix copyright year
* CoreDb: update test suite
* Aristo: Simplify reverse key map
why:
The reverse key map `pAmk: (root,key) -> {vid,..}` as been simplified to
`pAmk: key -> {vid,..}` as the state `root` domain argument is not used,
anymore
* Aristo: Remove `HashLabel` object type and replace it by `HashKey`
why:
The `HashLabel` object attaches a root hash to a hash key. This is
nowhere used, anymore.
* Fix copyright
* Aristo: Update schedule runner for `hashify()`
why:
Width-first schedule walker overlooked resolvable vertex and ended in
a deadlock situation
* CoreDb+Aristo: Update error code for fringe condition
* Ledger: Remove redundant extra MPT hashing statements
why:
These statement were used for troubleshooting
* CoreDb: Couch medicine for legacy DB backend
why:
MPT will occasionally enter inconsistent states after deletion, some
dirty fix is randomly added to mitgate that.
* Added procs to get and store block witness in db and add generate-witness cli flag.
* Completed initial implementation of block witness storage.
* Added test to verify witness is persisted to db after call to persistBlock.
* Update getBlockWitness to return witness using Result type.
* Make generate witness parameter hidden.
* CoreDb: Test module with additional sample selector cmd line options
* Aristo: Do not automatically remove a storage trie with the account
why:
This is an unnecessary side effect. Rather than using an automatism, a
a storage root must be deleted manually.
* Aristo: Can handle stale storage root vertex IDs as empty IDs.
why:
This is currently needed for the ledger API supporting both, a legacy
and the `Aristo` database backend.
This feature can be disabled at compile time by re-setting the
`LOOSE_STORAGE_TRIE_COUPLING` flag in the `aristo_constants` module.
* CoreDb+Aristo: Flush/delete storage trie when deleting account
why:
On either backend, a deleted account leave a dangling storage trie on
the database.
For consistency nn the legacy backend, storage tries must not be
deleted as they might be shared by several accounts whereas on `Aristo`
they are always unique.
* Update experimental rpc test to do further validation on proof responses.
* Enable zero value storage slots in the witness cache data so that proofs will be returned when a storage slot is updated to zero. Refactor and simplify implementation of getProofs endpoint.
* Improve test validation.
* Minor fixes and added test to track the changes introduced in every block using a local state.
* Refactor and cleanup test.
* Comments added to test and account cache fixes applied to account ledger.
* Return updated storage slots even when storage is empty and add test to build.
* Fix copyright and remove incorrect depth check during witness building in writeShortRlp assertion.
* Aristo: Update error return code
why:
Failing of `Aristo` function `delete()` might fail because there is
no such data item on the db. This must return a single error code
as is done with `fetch()`.
* Ledger: Better error handling
why:
The `expect()` clauses have been replaced by raising asserts indicating
the error from the database backend.
Also, `delete()` failures are legitimate if the item to delete does not
exist.
* Aristo: Delete function must always leave a label on DB for `hashify()`
why:
The `hashify()` uses the labels left bu `merge()` and `delete()` to
compile (and optimise) a scheduler for subsequent hashing.
Originally, the labels were not used for deleted entries and `delete()`
still had some edge case where the deletion label was not properly
handled.
* Aristo: Update `hashify()` scheduler, remove buggy optimisation
why:
Was left over from version without virtual state roots which did not
know about account payload leaf vertices referring to storage roots.
* Aristo: Label storage trie account in `delete()` similar to `merge()`
details;
The `delete()` function applied to a non-static state root (assumed
to be a storage root) will check the payload of an accounts leaf
and mark its Merkle keys to be re-checked when runninh `hashify()`
* Aristo: Clean up and re-org recycled vertex IDs in `hashify()`
why:
Re-organising the recycled vertex IDs list intends to reduce the size of the
list.
This list is organised as a LIFO (or stack.) By reorganising it in a way
so that the least vertex ID numbers are on top, the list will be kept
smaller as observed on some examples (less than 30%.)
* CoreDb: Accept storage trie deletion requests in non-initialised state
why:
Due to lazy initialisation, the root vertex ID might not yet exist. So
the `Aristo` database handlers would reject this call with an error and
this condition needs to be handled by the API (which realises the lazy
feature.)
* Cosmetics & code massage, prettify logging
* fix missing import
* CoreDb: Improve API and API tracking
why:
Now logs state roots where appropriate
* CoreDb: re-implement `CoreDbVidRef` => `CoreDbTrieRef`
why:
Instead of a root vertex ID wrapper, the purpose of this object type
has been upgrades to a sub-trie prototype.
caveat:
`Aristo` backend not fully functional, yet.
* CoreDb: Update `Aristo` backend
why:
Supports virtual sub-tries
* CoreDb: Account address tracking for `StorageTrie` virtual tries
details:
Supported with API tracking/logging
* CoreDb: Keep account address in payload object
why:
No need to provide extra address argument for `merge()`, also
provides tracking possibility for account debugging.
* Ledger: Update new API for `Aristo` specific storage trie handling
* CoreDb+Ledger: Update unit tests
* Fix copyright headers
* Aristo: Update unit test suite
* Aristo/Kvt: Fix iterators
why:
Generic iterators were not properly updated after backend change
* Aristo: Add sub-trie deletion functionality
why:
For storage tries linked to an account payload vertex ID, a the
whole storage trie needs to be deleted with the account.
* Aristo: Reserve vertex ID numbers for static custom state roots
why:
Static custom state roots may be controlled by an application,
e.g. for a receipt or a transaction root. The `Aristo` functions
are agnostic of what the static state roots are when different
from the internal tree vertex ID 1.
details;
The `merge()` function applied to a non-static state root (assumed
to be a storage root) will check the payload of an accounts leaf
and mark its Merkle keys to be re-checked.
* Aristo: Correct error code symbol
* Aristo: Update error code symbols
* Aristo: Code cosmetics/comments
* Aristo: Fix hashify schedule calculator
why:
Had a tendency to stop early leaving an incomplete job
* Add premix persist tool to Makefile to build as separate binary.
* Fix rootBytes.len > 0 assertion defect in HexaryTrie by adding call to com.db.compensateLegacySetup().
* Run hardForkTransition before executing transactions in getBlockWitness.
* Improve logging and add help message.
* Retry requests for block data.
* Nimbus light client integration with status-go
* Add cleanup code, address review comments
* Disable metrics for libverifproxy only
* Update confutils
* missing import
* build proxy in tests
* more build stuff
* namespace make vars
* export NimMain for windows
* reduce dependency on Nim compiler in header file
* copyright
---------
Co-authored-by: Vitaliy Vlasov <siphiuel@protonmail.com>
Co-authored-by: Jacek Sieka <jacek@status.im>
* Return slots when verifying witness regardless of code length.
* Prevent AccountCache WitnessData codeTouched from being reset.
* Default to using {wfNoFlag} for witness flags in tests to allow running with data from before EIP170.
* Add additional json files to experimental JSON RPC test.
* Use HTTP RPC server in tests.
* Add test to check that block witness contains bytecode.
* Completed draft implementation of witness JSON-RPC endpoints for portal network bridge.
* Updated Nimbus RPC configuration to support enabling experimental endpoints.
* Moved witness verification tests.
* Added json test for getProof.
* Added main procs to new tests to fix test suite.
* Added getBlockWitness test to blockchain json test suite.
* Added tests for experimental RPC endpoints and improved the API to support returning state proofs from before or after block execution.
* Correctly rollback transaction in getBlockWitness proc.
web3-developer
Jordan Hrycaj
andri lim
Agnish Ghosh
Vit∀ly Vlasov
Kim De Mey
Guido Vranken