* Nimbus folder environment update
details:
* Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder.
* The `nimbus` program does not compile yet as it needs the updates
in the parallel `stateless` sub-folder.
* Stateless environment update
details:
* Integrated `CoreDbRef` for the sources in the `stateless` sub-folder.
* The `nimbus` program compiles now.
* Premix environment update
details:
* Integrated `CoreDbRef` for the sources in the `premix` sub-folder.
* Fluffy environment update
details:
* Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder.
* Tools environment update
details:
* Integrated `CoreDbRef` for the sources in the `tools` sub-folder.
* Nodocker environment update
details:
* Integrated `CoreDbRef` for the sources in the
`hive_integration/nodocker` sub-folder.
* Tests environment update
details:
* Integrated `CoreDbRef` for the sources in the `tests` sub-folder.
* The unit tests compile and run cleanly now.
* Generalise `CoreDbRef` to any `select_backend` supported database
why:
Generalisation was just missed due to overcoming some compiler oddity
which was tied to rocksdb for testing.
* Suppress compiler warning for `newChainDB()`
why:
Warning was added to this function which must be wrapped so that
any `CatchableError` is re-raised as `Defect`.
* Split off persistent `CoreDbRef` constructor into separate file
why:
This allows to compile a memory only database version without linking
the backend library.
* Use memory `CoreDbRef` database by default
detail:
Persistent DB constructor needs to import `db/core_db/persistent
why:
Most tests use memory DB anyway. This avoids linking `-lrocksdb` or
any other backend by default.
* fix `toLegacyBackend()` availability check
why:
got garbled after memory/persistent split.
* Clarify raw access to MPT for snap sync handler
why:
Logically, `kvt` is not the raw access for the hexary trie (although
this holds for the legacy database)
on windows, using "localhost" for rpc test is very slow.
both pyspec_sim and engine_sim will need more than one hour.
while on linux and macos only few minutes.
This is a workaround following hive example, because one of the test case
withdrawalsAmountBounds.json, have bad blocks between good blocks.
And that bad blocks contains big int too big to fit in uint64 of
withdrawal amount field.
Clients who still importing concatenated blocks cannot pass all tests.
previously, the withdrawal validation is in process_block only,
but the one in persist block, which is also used in synchronizer
is not validated properly.
* Part of EIP-4895: add withdrawals processing to block processing.
* Refactoring: extracted the engine API handler bodies into procs.
Intending to implement the V2 versions next. (I need the bodies to be
in separate procs so that multiple versions can use them.)
* Working on Engine API changes for Shanghai.
* Updated nim-web3, resolved ambiguity in Hash256 type.
* Updated nim-eth3 to point to master, now that I've merged that.
* I'm confused about what's going on with engine_client.
But let's try resolving this Hash256 ambiguity.
* Still trying to fix this conflict with the Hash256 types.
* Does this work now that nimbus-eth2 has been updated?
* Corrected blockValue in getPayload responses back to UInt256.
c834f67a37
* Working on getting the withdrawals-related tests to pass.
* Fixing more of those Hash256 ambiguities.
(I'm not sure why the nim-web3 library introduced a conflicting type
named Hash256, but right now I just want to get this code to compile again.)
* Bumped a couple of libraries to fix some error messages.
* Needed to get "make fluffy-tools" to pass, too.
* Getting "make nimbus_verified_proxy" to build.
* Updated to the latest nim-eth, nim-rocksdb, nim-web3
* Bump nimbus-eth2 module and fix related issues
Temporarily disabling Portal beacon light client network as it is
a lot of copy pasted code that did not yet take into account
forks. This will require a bigger rework and was not yet tested
in an actual network anyhow.
* More nimbus fixes after module bumps
---------
Co-authored-by: Adam Spitz <adamspitz@status.im>
Co-authored-by: jangko <jangko128@gmail.com>
* Use type name eth and snap (rather than snap1)
* Prettified snap/eth handler trace messages
* Regrouped sync sources
details:
Snap storage related sources are moved to common directory.
Option --new-sync renamed to --snap-sync
also:
Normalised logging for secondary/non-protocol handlers.
* Merge protocol wrapper files => protocol.nim
details:
Merge wrapper sync/protocol_ethxx.nim and sync/protocol_snapxx.nim
into single file snap/protocol.nim
* Comments cosmetics
* Similar start logic for blockchain_sync.nim and sync/snap.nim
* Renamed p2p/blockchain_sync.nim -> sync/fast.nim
new hive features introduces breaking changes we need to address:
- expose engine api port 8551 in docker script
- export enode.sh file to hive-bin folder
- add terminalTotalDifficulty to mapper.jq
* Activate wire protocol eth/66
and:
Disentangle protocol_eth66.nim from import sections
why:
Importing the protocol_eth66 module is not necessary. There is
no need to know too many details of the underlying wire protocol. All
that is needed will be exported by blockchain_sync.nim.
* fixes, and rebase
* Update nimbus/p2p/blockchain_sync.nim
Co-authored-by: Kim De Mey <kim.demey@gmail.com>
* Fixes and rebase
Co-authored-by: Kim De Mey <kim.demey@gmail.com>
Adds changes to Nimbus Hive support for the new
[Arrow Glacier fork](https://eips.ethereum.org/EIPS/eip-4345).
While here:
- Fix typo in `nimbus.sh`:
`HIVE_FORK_MUIRGLACIER` => `HIVE_FORK_MUIR_GLACIER` (just a comment).
- Add `muirGlacierBlock` to the JSON generated in `extract_consensus_data.nim`.
This makes it symmetric with the JSON parsed in `mapper.jq`.
- Removed "At5" network names which are not used by any of the test suite.
These are `ByzantiumToConstantinopleAt5`, `ConstantinopleFixToIstanbulAt5`
and `IstanbulToBerlinAt5`.
The motivation for removing these instead of systematically including all
possibilities was that I realised `LondonToArrowGlacierAt5` does not appear
anywhere in the current test suite, even though `ArrowGlacier` does. As each
section in the code is rather large already, I thought it cleaner to not add
this one, and keep only the ones the test suite actually uses.
This also now better matches the code in `test_blockchain_json.nim`.
- Sorted `HomesteadToDaoAt5` before `HomesteadToEIP150At5` because the DAO
fork happened earlier than EIP-150 in real life.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
this is a preparation for migration to confutils based config
although there is still some getConfiguration usage in tests code
it will be removed after new config arrived
This preparation is needed for subsequent
EIPs included in London.
- Add London to Fork enum
- Block number to fork
- Parsing London fork in chain config
- Prepare gas costs table for London
- Prepare EVM opcode dispatcher for London
- Block rewards for London
- Prepare hive script for London
* Re-adjust canonical head to parent of block to be inserted
why:
of the failing tests that remain to be solved, 30 of those will succeed
if the canonical database chain head is cleverly adjusted -- yes, it
looks like a hack, indeed.
details:
at the moment, this hack works for the non-hive tests only and is
triggered by a boolean argument passed on to the chain.persistBlocks()
method.
* Use parent instead of canonical head for block to be inserted
why:
side chains need to be inserted typically somewhere before the
canonical head.
details:
the previous _hack_ was unnecessary and removed, it was inspired by
some verification in persistBlocks() which explicitly referenced the
canonical head (which now might or might not refer to the newly inserted
header.)
* remove unnecessary code + comment
additional flags we use:
- [x] HIVE_NETWORK_ID network ID number to use for the eth protocol
- [x] HIVE_LOGLEVEL client loglevel (0-5)
- [x] HIVE_GRAPHQL_ENABLED enables graphql on port 8545
hive always set 'HIVE_FORK_DAO_VOTE' to '1'.
set 'daoForkSupport' to 'true' if only 'HIVE_FORK_DAO_BLOCK'
is set by hive, not depends only on 'HIVE_FORK_DAO_VOTE'.
This is because 'nimbus-eth1' internal is different compared to 'geth'.
now we have hive simulators written in nim:
- ethereum/consensus
- ethereum/graphql
Using these simulators, we can debug test cases without have to run
hive or docker.
this changes is required due to recent #654.
custom genesis and chain config parser are fixed
and the genesis fields are grouped into "genesis"
field, similar with chain config fields,
they are grouped in "config" field.
trying Jacek suggestion in #591, I added nat setting to
nimbus-eth1 hive shell script. visible difference after adding
this flag is the nat library not complaining about
"the gateway does not support nat-pmp" anymore.
but the slow startup time described in #591 is yet to be measured
again, although I already see improvement when executing
ethereum/consensus category in hive.
- Explain some of the prerequisites in more detail (Go version).
- Practical issues around Go version and Docker configuration to make it work.
- Make the first example be one that works with nimbus-eth1.
(Instead of giving an obscure error message that looks like the user's Docker
setup is broken).
- Rename to `README.md` per convention.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
andri lim
Jordan Hrycaj
Adam Spitz
Kim De Mey
Jacek Sieka
Jamie Lokier