Includes a simple test harness for the merge interop M1 milestone
This aims to enable connecting nimbus-eth2 to nimbus-eth1 within
the testing protocol described here:
https://github.com/status-im/nimbus-eth2/blob/amphora-merge-interop/docs/interop_merge.md
To execute the work-in-progress test, please run:
In terminal 1:
tests/amphora/launch-nimbus.sh
In terminal 2:
tests/amphora/check-merge-test-vectors.sh
details:
For documentation, see comments in the file tx_pool.nim.
For prettified manual pages run 'make docs' in the nimbus directory and
point your web browser to the newly created 'docs' directory.
why:
Previous version was based on lru_cache which is ugly. This module is
based on the stew/keyed_queue library module.
other:
There are still some other modules rely on lru_cache which should be
removed.
Add the new [Arrow Glacier fork](https://eips.ethereum.org/EIPS/eip-4345).
Only the difficulty calculation is changed, but as a new fork it still affects
a number of places in the code.
To the best of my knowledge the change is only scheduled on Mainnet.
In addition:
- The fork date comments in `chain_config.nim` have been checked against the
real networks, set consistently in UTC instead of random timezones, and made
neater. Maybe we'll keep these when transferring config to a file someday.
- It's added to forkid hash tests (EIP-2124/EIP-2364), of course.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
* PoW wrapper for verification & mining
why:
It eases data management of per-Epoch lookup tables. Also some unit
tests show limits of usefulness on non-specialised machines for
mining besides developing tests.
details:
For PoW verification, this patch provides a pretty wrapper hiding the
details of the ethash/Hashimoto lookup cache management.
For mining on my development system without special hardware, the
underlying ethash functions are prohibitively slow. It takes
* ~20 minutes to prepare the full ethash/Hashimoto lookup dataset
* a second to run ~25k nonce tests (in the mining loop)
The mining part might be of some use for generating test data for
the tx-pool, though.
* Using PowRef as replacement for EpochHashCache + hashimotoLight()
* Fix typo (CI failed)
why:
was below log level when testing locally
* fix canonical naming
* re-shuffled Clique functions
why:
Due to the port from the go-sources, the interface logic is not optimal
for nimbus. The main visible function is currently snapshot() and most
of the _procurement_ of this function result has been moved to a
sub-directory.
* run eip-225 Clique test against p2p/chain.persistBlocks()
why:
Previously, loading the test block chains was fugdged with the purpose
only to fill the database. As it is now clear how nimbus works on
Goerli, the same can be achieved with a more realistic scenario.
details:
Eventually these tests will be pre-cursor to the reply tests for the
Goerli chain supporting TDD approach with more simple cases.
* fix exception annotations for executor module
why:
needed for exception tracking
details:
main annoyance are vmState methods (in state.nim) which potentially
throw a base level Exception (a proc would only throws CatchableError)
* split p2p/chain into sub-modules and fix exception annotations
why:
make space for implementing PoA stuff
* provide over-loadable Clique PRNG
why:
There is a PRNG provided for generating reproducible number sequences.
The functions which employ the PRNG to generate time slots were ported
ported from the go-implementation. They are currently unused.
* implement trusted signer assembly in p2p/chain.persistBlocks()
details:
* PoA processing moved there at the end of a transaction. Currently,
there is no action (eg. transaction rollback) if this fails.
* The unit tests with staged blocks work ok. In particular, there should
be tests with to-be-rejected blocks.
* TODO: 1.Optimise throughput/cache handling; 2.Verify headers
* fix statement cast in pool.nim
* added table features to LRU cache
why:
Clique uses the LRU cache using a mixture of volatile online items
from the LRU cache and database checkpoints for hard synchronisation.
For performance, Clique needs more table like features.
details:
First, last, and query key added, as well as efficient random delete
added. Also key-item pair iterator added for debugging.
* re-factored LRU snapshot caching
why:
Caching was sub-optimal (aka. bonkers) in that it skipped over memory
caches in many cases and so mostly rebuild the snapshot from the
last on-disk checkpoint.
details;
The LRU snapshot toValue() handler has been moved into the module
clique_snapshot. This is for the fact that toValue() is not supposed
to see the whole LRU cache database. So there must be a higher layer
working with the the whole LRU cache and the on-disk checkpoint
database.
also:
some clean up
todo:
The code still assumes that the block headers are valid in itself. This
is particular important when an epoch header (aka re-sync header) is
processed as it must contain the PoA result of all previous headers.
So blocks need to be verified when they come in before used for PoA
processing.
* fix some snapshot cache fringe cases
why:
Must not index empty sequences in clique_snapshot module
Transaction and BlockHeader already updated in nim-eth repo
to support EIP-1559
EIP-1559 header validation and gasLimit validation
already implemented in previous commit
This commit deals with block validation:
- Effective gasPrice per EIP-1559
- new miner reward based on priorityFee
why:
previously, table data were stored with the table iterator. while
loading a table with permuted entries will always reconstruct equivalent
tables (in the sense of `==`), serialisation data are not comparable.
this patch produces always the same serialised data for equivalent
tables.
why:
source-local unit tests would hardly be triggered by github CI as rightly
criticised with the last patch.
details:
source-local unit tests have been moved to tests folder.
this version also contains rlp serialisation code so rlp encode/decode
will apply tranparently. this is not needed in p2p/validate but will be
useful with the clique protocol.
why:
to be used in Clique consensus protocol which suggests 4k cache entries.
the previous implementation used OrderTable[] which has complexity O(n)
for deleting entries.
instead of using stdlib/json, now we switch to json_serialization
the result is much tidier code and more robust when parsing
optional fields.
fixes#635
* Delete utilities: address, padding, bytes
* Remove bigEndianToInt
* Fix C compiler error: member reference base type 'char' is not a structure or union
* fix todo comment
Merge note: currently cannot compile due to `quasiBoolean` (#63). This will be solved by https://github.com/status-im/nimbus/pull/65
----
* Implemented most of the stubbed out state handling instructions
The code compiles, but still fails at the moment due to incorrect
initialization of the VM. Don't merge yet. More commits will be
pushed in the coming days.
* Fixed crash
* trie put and del are void now
* getBlockTransactionData and getReceipts
* Working code for extcodesize0.json
* fix origin.json
* fix calldatasize1
* fix calldataloadSizeTooHighPartial
* fix calldataloadSizeTooHigh
* more efficient PushX implementation
* fix and, or, xor
Some of the other clients don't bother to store block numbers as
256-bit integers (64 bits or even 32 bit are enough).
The changes here are based on helpers introduced in eth_common
that try to abstract away all conversion steps in the code between
VM words and block numbers in variables in fields. Unfortunately,
this turn out to be more work than anticipated and I've postponed
finishing the refactoring for now (Block numbers are still 256 bit).
* Move and cleanup interpreter files - prepare for redesign of VM
* fix call comment aobut recursive dependencies
* memory: use a template again and avoid (?) a cstring-> string conversion
* Fix stack test regression
* Fix recursive dependency on logging_ops, test_vm_json compiles but regression :/
* Fix signextend regression
* Fix 3 signed test and sha3 test
* Decoupling op logic and gas - introduce gasometer, rework opcode declaration
* Remove gas constants for gas opcode computation
* Remove gas constants for precompiled contracts
* make vm_types compile
* Make opcode, call and computation compile
* Distinguish between dynamic and complex gas costs, fix arithmetic
* Fix context and sha3
* update memory and storage ops
* Log opcode uses memory expansion code
* update/stub system_ops with gas costs
* Make test compile. Deactivate stub test_vm
* all tests compiles, opcode fails due to https://github.com/nim-lang/Nim/issues/8007 (const object variant in tables reset at runtime)
* Create an enum without holes - workaround: https://github.com/nim-lang/Nim/issues/8007
* Use arrays instead of tables for GasCosts, remove some unused imports - passing all basic tests!
* Make test_vm_json compile
* Fix test_vm_json - workaround https://github.com/nim-lang/Nim/issues/8015
* fix memory expansion cost bug
* Remove leftover special handling from before GckMemExpansion
* cleanup outdated comment, better align =
* Fix sha3 gas cost not taking memory expansion into account
* Improve gas error reporting of test_vm_json
* Fix gas computation regression due to mem expansion
* mass replace for memExpansion->RequestedMemSize was too eager
* fix log gas cost (no tests :/)
* missed a static FeeSchedule
* static as expression is fickle
jangko
Zahary Karadjov
Jordan Hrycaj
Jamie Lokier
Yuriy Glukhov
coffeepots
Mamy Ratsimbazafy
Yuriy Glukhov