- Add new content + content key functionality for header by number
- Remove EpochRecords from the network
- Add pruning call for the EpochRecords + required deprecated
functionality
- Adjust getBlock and getBlockHashByNumber to make use of the
new functionality instead
- Delete content_verifier as it was only verifying the now
deprecated EpochRecord
This avoid restarting the node always with a full radius, which
causes the node the be bombarded with offers which it later has
to delete anyhow.
In order to implement this functionality, several changes were
made as the radius needed to move from the Portal wire protocol
current location to the contentDB and beaconDB, which is
conceptually more correct anyhow.
So radius is now part of the database objects and a handler is
used in the portal wire protocol to access its value.
* Return default values when account, slot or code doesn't exist.
* Handle case when storage doesn't exist due to account not existing or being a non contract account.
* Update state network to use addressHash instead of address in contract trie and contract code content keys.
* Fix path calculation bug in getParent when working with extension nodes.
* Bump portal spec tests repo.
* Finish updating tests due to portal test vector changes.
* Update Fluffy state bridge to use addressHash.
* Update Fluffy book with correct commands for running portal hive tests.
* Use RPC batching to send offer requests and filter out duplicate offers.
* Lookup offers after gossip to check if gossip successful.
* Use multiple workers for gossiping offers.
* Update Fluffy state network logging.
* Use single RPC calls instead of batching.
* Update cli parameters.
* Fix bug in contract trie offer building.
* Create block offers queue and collect account preimages.
* Implement iterators to return account and storage proofs and bytecode from updatedCaches.
* Implement building offers from proofs.
* Refactor BlockDataRef type to only include required fields.
* Store block data in database.
* Improve state diff types.
* Implement start state backfill from specific block.
* Record last persisted block number in database.
* Persist account preimages in db.
* Apply state updates for DAO hard fork.
* Implement state gossip of block offers via portal JSON RPC.
- Add basic validation for LC bootstrap gossip, validating either
by trusted block root (only 1) when not synced, or by comparing
with the header of the latest finality update when synced.
- Update portal_bridge beacon to also gossip bootstraps into the
network on each end of epoch.
The 3 proofs can be reworked to two proofs as we can use the
BeaconBlock directly instead of BeaconBlockHeader and
BeaconBlockBody. This is possible because the HTR of the
BeaconBlock is the same as the one of the BeaconBlockHeader.
This results in 32 bytes less as an intermediate hash can be
removed. But more importantly looks more clean and compact in
structure and code.
- EpochAccumulator got renamed to EpochRecord
- MasterAccumulator is not HistoricalHashesAccumulator
- The List size for the accumulator got a different maximum which
also result in a different encoding and HTR
* Bump nim-eth, nim-web3, nimbus-eth2
- Replace std.Option with results.Opt
- Fields name changes
* More fixes
* Fix Portal stream async raises and portal testnet Opt usage
* Bump eth + nimbus-eth2 + more fixes related to eth_types changes
* Fix in utp test app and nimbus-eth2 bump
* Fix test_blockchain_json rebase conflict
* Fix EVMC block_timestamp conversion plus commentary
---------
Co-authored-by: kdeme <kim.demey@gmail.com>
* Updates to Fluffy book for hive tests.
* Add support to disable state root checks for state content from the command line.
* Update portal_stateStore endpoint to support decoding offer and storing retrieval value.
* Started implementation of state endpoints.
* Add rpc calls and server stubs.
* Initial implementation of getAccountProof and getStorageProof.
* Refactor validation to use toAccount utils functions.
* Add state endpoints tests.
`initTable` is obsolete since nim 0.19 and can introduce significant
memory overhead while providing no benefit (since the table will be
grown to the default initial size on first use anyway).
In particular, aristo layers will not necessarily use all tables they
initialize, for exampe when many empty accounts are being created.
* Use --styleCheck:error for Fluffy + fixes
There seems to be a clash between the names of an object field and
a proc here. As workaround names of the procs are changed.
* Fix case style for rpc client calls in test_portal_testnet
* Fix style case for utp test
* Refactor get parent gossip code and add Nibbles helper function.
* Add logging to state gossip.
* Unit test recursive gossip using state gossip getParent functions.
* Add recursive gossip genesis json test and fix bug in state gossip getParent.
* Add Nibbles len function.
* Add toContentKey helper functions.
* Implement typed lookup functions for getting account and storage trie nodes and bytecode.
* Re-implementation of recursive gossip.
* Offer validation improvements. Don't allow key end in Nibbles for leafs as defined in the spec.
* Minor refactor to validation and add addition bytecode check.
* Implemented offer validation interface.
* Implement high level offer validation steps.
* Completed state validation tests.
* Update validation to use result type.
* Update state proof verification tests to test offer verification.
* Query history network to get state root by block hash.
* Fix state network test and remove usage of CoreDb.
* Fix state network gossip test and PR comment updates.
* Add trieproof state validation tests and fix for short nodes.
* Add retrieval content validation to state network.
* Complete state validation tests for retrieval content.
* Improve naming functions and parameters.
- Add post Bellatrix Beacon block proof exporting to
eth_data_exporter
- Rework the Bellatrix Beacon block proof exports to us yaml
format
- Move some common code of the two different beacon block proofs
to beacon_chain_block_proof_common module
- Move + add to yaml utils to be usable not just for the tests but
also for eth data exporter
* Remove references to CoreDb in Fluffy code.
* Add missing import.
* Remove coredb imports and use dynamic server port in tests.
* Fix formatting using nph.
* Add nph check to fluffy CI lint
* Add a section on nph usage in the fluffy.guide
* Update copyright years for altered files
* Avoid chained methods formatting style in db code
* Update nph in CI to v0.5
* Remove leftover commented import
* Move comment to avoid nph turning complex list into simple list (nph bug)
* Update nph in CI to v0.5.1
* Formatting fluffy with nph v0.5.1
This would currently manifest sometimes in state gossip test in
CI or locally. But could occur on fluffy binary in case state
network is enable and data is gossiped.
* Initial implementation of era1 + export + verify
Initial implementation of era1 + export and verify command in the
existing eth_data_exporter.
* Quick verify version to test with geth created era1 files
* Add Era1File object and getter for specific block tuple
* Add getAccumulatorRoot and update AccumulatorRoot type name
+ clean-up of some comments.
* Implement BlockIndex based verify + additional helper calls
+ fix bug for reading last (incomplete) Era1 file
* Fix init BlockIndex offsets for last era + extra check on reading
* Remove duplicated e2store code
* Return slots when verifying witness regardless of code length.
* Prevent AccountCache WitnessData codeTouched from being reset.
* Default to using {wfNoFlag} for witness flags in tests to allow running with data from before EIP170.
* Add additional json files to experimental JSON RPC test.
* Use HTTP RPC server in tests.
* Add test to check that block witness contains bytecode.
* Completed draft implementation of witness JSON-RPC endpoints for portal network bridge.
* Updated Nimbus RPC configuration to support enabling experimental endpoints.
* Moved witness verification tests.
* Added json test for getProof.
* Added main procs to new tests to fix test suite.
* Added getBlockWitness test to blockchain json test suite.
* Added tests for experimental RPC endpoints and improved the API to support returning state proofs from before or after block execution.
* Correctly rollback transaction in getBlockWitness proc.
* Remove support for validation of missing leafs.
* Move proof types into separate file and borrow required procs.
* Added state proof types.
* Completed initial implementation of witness verification.
* Reject empty proofs.
* Added tests for verification of block witnesses.
* Initial implementation of account and storage proof verification for the portal state network.
* Completed initial state proof verification loop test.
* Completed proof verification tests.
* Minor updates based on PR feedback and comments.
* Add state proof verification test to test runner.
- The current setting of 0.25 was very big. Set to 0.5, which is
potentially still large. This change did expose some issues with
the current implementation and especially testing.
- General clean-up, renaming for consistency, and
re-ordering/restructuring/deletion of some code.
- Fixed several typos
- ...
This would lead to sending an AcceptMessage with a zeroed out
connection id but still potentially bits set in the bitlist.
Requesting nodes could try to open up an uTP connection that
would never be accepted on the responder side.
Kim De Mey
web3-developer
andri lim
Miran
Jacek Sieka
BiKangNing
Daniel Sobol