The radius sort performance improvement in content lookups was
not implemented in the trace version.
Also cleanup some part of the logging related to uTP connection
setup.
There is an assertion hitting due to the additon of an iterator
that deletes items from the sequence while iteratting over it.
Before the keepIf helper was used that has different code for
doing this similar work.
startedAtMs is the time passed since UNIX epoch in milliseconds.
Cannot use a monotime clock for that. As we want to keep mono
for other Moment.now() usage, imported epochTime from std/times
instead.
- Move any validation related code to new file
- Move any type conversion related code to new file
- Prepare validation code for adding different type of canonical
proofs
- Prepare for validation code for json-rpc api
- several other clean-ups and renames
Sort of a first pass as the validation code specifically can
use some further changes.
* Remove getSszDecoded from ContentDb.
* Update ContentDb get to use onData callback to reduce copies.
* Use templates for helper procs in ContentDb.
* Add contains handler to portal protocol.
* Improve performance of DbGetHandler.
* switch to Nim v2.0.12
* fix LruCache capitalization for styleCheck
* KzgProof/KzgCommitment for styleCheck
* TxEip4844 for styleCheck
* styleCheck issues in nimbus/beacon/payload_conv.nim
* ENode for styleCheck
* isOk for styleCheck
* some more styleCheck fixes
* more styleCheck fixes
---------
Co-authored-by: jangko <jangko128@gmail.com>
* Set offer processing logs to debug level and add offer counts to state network.
* Use metrics instead of int counters and remove from logs.
* More logging improvements. Make decoding and validation failures use error log level.
* Add protocol_id to metrics.
* Make content request retries configurable and add to state network and history network.
* Set retries to 1. Use uint to prevent negative values. Make contentRequestRetries a debug parameter.
* Cache content after lookups in history network.
* Cleanup config in PortalProtocol.
* Use local content lookup in history network to enable using cache.
* Enable content cache for state network.
* Update state json-rpc endpoints to return local content which uses cache if enabled.
* Add content cache metrics.
* Make content cache configurable.
* Add content cache tests.
* Add validation functions to be used in state portal rpc.
* Add validation to remaining state portal rpc methods.
* Lookup local content in recursiveFindContent rpc methods.
* portal_stateFindContent and portal_stateOffer no longer store in db.
Portal beacon LC sync can be started now from a provided trusted
block root or, in case it has been running before, from a
previously verified and stored LC bootstrap.
This required altering the the beacon db on how the bootstraps
are stored.
These create only confusion as if they are actual different types
and it is within their usage already clear what they are about
because of the name of the variable or the function.
They are also nowhere aliased like this in any of the Portal
specification.
* bump nimbus-build-system to use Nim v2.0.10
* 2.0.10 fixes
* fluffy linting
* make trivial change which should trigger whole-nimbus+fluffy rebuild/ci
* Nim v2.0.10 chronicles.error/macros.error ambiguity workaround
* another contentType enum specifier
* fluffy linting
* Fix eth/common & web3 related deprecation warnings for fluffy
This commit uses the new types in the new eth/common/ structure
to remove deprecation warnings.
It is however more than just a mass replace as also all places
where eth/common or eth/common/eth_types or eth/common/eth_types_rlp
got imported have been revised and adjusted to a better per submodule
based import.
There are still a bunch of toMDigest deprecation warnings but that
convertor is not needed for fluffy code anymore so in theory it
should not be used (bug?). It seems to still get imported via export
leaks ffrom imported nimbus code I think.
* Address review comments
* Remove two more unused eth/common imports
This is a minimal set of changes to make things work with the new types
in nim-eth - this is the minimal PR that merely resolves
incompatibilities while the full change set would include more cleanup
and migration.
The revalidateMax value is lowered to have a quicker ramp up of
the radiusCache + to keep it healthier.
The defaultMaxGossipNodes value is lowered because with the
current value a Nodes lookup is triggered almost always.
This value is dependant on the content replication value. This
is dependant on the network (and subnetwork) because of the amount
of nodes and their radius/storage capacity.
This bug would have as effect that our radius cache would not get
filled by any outgoing pings, causing:
- Node lookups to always be occurring on NH gossip
- POKEs to much more rarely
Also add metrics for the amount of offers done via POKE mechanism.
* Make stop functions wait for completion before return.
* Implement graceful shutdown.
* Shutdown rpc and metric servers if enabled.
* Move metrics and rpc servers out of PortalNode.
* Enable state network by default. Create status log loop for state and beacon networks. Create status log loop for portal node. Implement stop functions.
* Improve state endpoint genesis test and cover cases when accounts, code and slots doesn't exist.
* Refactor state endpoints to support returning partial proofs.
* Implement getProofs in state endpoints.
* Add tests for getProofs and improve code.
* Implement eth_getProof JSON-RPC api in Fluffy.
bhartnett
Kim De Mey
Kim De Mey
tersec
Jacek Sieka
web3-developer