* simplify state fork access pattern
* fixes
* unsafeAddr needs to be dereferenced outside of case for best effect
* remove hash_tree_root of ForkedXxx (doesn't make sense)
* simplify state transition
* fix vc
* readd hash_tree_root(forkedbeaconblock)
* readd htr(fhbs) as well
...and add some protections to not hash the wrong items elsewhere
There are still a few cases with manual loops through sync subcommittees
even though an `allSyncCommittees` iterator exists. Adjusted the few
remaining instances to also use the iterator instead.
Currently there is a mix of comment formats in the various preset files.
Altair presets have a leading header line, Phase0 presets don't.
Furthermore, the Altair `minimal` header refers to `mainnet` (#2710).
Updated all of the header lines to match the spec.
The current `getStateField` implementation fails at run-time when called
on a post-Altair state. This is improved by replacing the `if` structure
with a `case` expression, which is checked for exhaustive coverage at
compile time. Care is taken to preserve the `unsafeAddr` optimization.
* Placing callbacks into strategic places.
* Initial events call implementation.
* Post rebase fixes.
* Change addSyncContribution() implementation.
* Add `attestation-sent` event.
Remove gcsafe, raises from callbacks implementations.
Move `attestation-received` fire at the end of attestation processing.
* Address review comments.
The template `message` extracts fork specific data from a forked block.
However, its current implementation does not compile due to each fork
returning a different type, i.e., the generated `case` code attempts to
return different types from each branch. As the template is unused, it
is safely removed for now.
* add merge SSZ consensus object tests
* add merge attestations, attester slashing, proposer slashing, and voluntary exit test fixtures
* throw catchable exception rather than assert on invalid SingleMemberUnion selector
* hash_tree_root can assert, because it's trusted data by that point
* re-assert on writing, since also trusted data
* beta.4 update
* implement upgrade_to_merge()
Other changes:
* Add server getBlockV2(), and produceBlockV2().
* Add getBlockV2() to REST test suite.
* Add client getBlockV2(), and produceBlockV2().
* Fix URLs in comments.
* Add some primitives and fix some issues in forks.nim.
* Switch `validator_client` to V2 calls usage.
* Bump `chronos` with imports fixes.
* Bump `nim-json-serialization` for `requireAllFields`.
* construct all unslashed, participating balances in one validator scan
* remove altair benchmarking setup for block_sim
* revert a benchmarking change from proc to func
* remove more benchmarking func/proc tweaks
* re-add asSeq, which is necessary because unsafeAddr
* ... except for the block_sim benchmark part
* approximately double altair epoch processing speed
* don't redundantly clear state balances cache
* only invalidate altair state balances cache once in process_rewards_and_penalties()
* cleanups
* use ForkedTrustedSignedBeaconBlock.ionit where appropriate
* move `is_aggregator` to `spec/`
* use `errReject` in a few more places
* update enr fork id when time is auspicious
* use network broadcast functions
* Return Ignore for aggregate signature validation timeouts
...consistently between aggregates and attestations.
* clean up some more reject/ignore rules
* shorten texts a bit
* errReject->checkedReject, use err helpers throughout
* get rid of quarantine in exitpool as well
* Fix getForkSchedule call.
Create cache of all configuration endpoints at node startup.
Add prepareJsonResponse() call to create cached responses.
Mark all procedures with `raises`.
* Add getForkSchedule to VC.
Fix getForkSchedule return type for API.
More `raises` annotations.
Fix VC fork_service.nim.
* Use `push raises` instead of inline `raises`.
* Improvements for REST API aggregated attestations and attestations processing.
* Rename eth2_network.sendXXX procedures to eth2_network.broadcastXXX.
Add broadcastBeaconBlock() and broadcastAggregateAndProof().
Fix links to specification in REST API declarations.
Add implementation for v2 getStateV2().
Add validator_duties.sendXXX procedures which not only broadcast data, but also validate it.
Fix JSON-RPC/REST to use new validator_duties.sendXXX procedures instead of own implementations.
* Fix validator_client online nodes count incorrect value.
Fix aggregate and proof attestation could be sent too late.
* Adding timeout for block wait in attestations processing.
Fix compilation errors.
* Attempt to debug aggregate and proofs.
* Fix Beacon AIP to use `sendAttestation`.
Add link comment to produceBlockV2.
* Add debug logs before publish operation for blocks, attestations and aggregated attestations.
Fix attestations publishing issue.
* logging fixes
`indexInCommnittee` already logged in attestation
Co-authored-by: Jacek Sieka <jacek@status.im>
* reorganize ssz dependencies
This PR continues the work in
https://github.com/status-im/nimbus-eth2/pull/2646,
https://github.com/status-im/nimbus-eth2/pull/2779 as well as past
issues with serialization and type, to disentangle SSZ from eth2 and at
the same time simplify imports and exports with a structured approach.
The principal idea here is that when a library wants to introduce SSZ
support, they do so via 3 files:
* `ssz_codecs` which imports and reexports `codecs` - this covers the
basic byte conversions and ensures no overloads get lost
* `xxx_merkleization` imports and exports `merkleization` to specialize
and get access to `hash_tree_root` and friends
* `xxx_ssz_serialization` imports and exports `ssz_serialization` to
specialize ssz for a specific library
Those that need to interact with SSZ always import the `xxx_` versions
of the modules and never `ssz` itself so as to keep imports simple and
safe.
This is similar to how the REST / JSON-RPC serializers are structured in
that someone wanting to serialize spec types to REST-JSON will import
`eth2_rest_serialization` and nothing else.
* split up ssz into a core library that is independendent of eth2 types
* rename `bytes_reader` to `codec` to highlight that it contains coding
and decoding of bytes and native ssz types
* remove tricky List init overload that causes compile issues
* get rid of top-level ssz import
* reenable merkleization tests
* move some "standard" json serializers to spec
* remove `ValidatorIndex` serialization for now
* remove test_ssz_merkleization
* add tests for over/underlong byte sequences
* fix broken seq[byte] test - seq[byte] is not an SSZ type
There are a few things this PR doesn't solve:
* like #2646 this PR is weak on how to handle root and other
dontSerialize fields that "sometimes" should be computed - the same
problem appears in REST / JSON-RPC etc
* Fix a build problem on macOS
* Another way to fix the macOS builds
Co-authored-by: Zahary Karadjov <zahary@gmail.com>
The spec imports are a mess to work with, so this branch cleans them up
a bit to ensure that we avoid generic sandwitches and that importing
stuff generally becomes easier.
* reexport crypto/digest/presets because these are part of the public
symbol set of the rest of the spec types
* don't export `merge` types from `base` - this causes circular deps
* fix circular deps in `ssz/spec_types` - this is the first step in
disentangling ssz from spec
* be explicit about phase0 vs altair - longer term, `altair` will become
the "natural" type set, then merge and so on, so no point in giving
`phase0` special preferential treatment
Simpler module name for stuff that covers forks
* check that runtime config matches database state
* also include some assorted altair cleanups
* use "standard" genesis fork in local testnet to work around missing
runtime config support
This refactoring puts the JSON-RPC and REST APIs on more equal footing
by renaming and moving things around, creating a separation between
client and server, and documenting what they are - the aim is to have a
simple-to-use base to start from when developing API clients, as well as
make it easier to navigate the code when looking for the legacy JSON-RPC
interface vs the new REST API.
* move REST client, serialization and supporting types to spec/eth2_apis
* REST stuff now starts with `rest_`, JSON-RPC stuff starts with `rpc_`,
more or less
* simplify imports such that there's a simple module to import for both
server and client
* map REST type and proc names to yaml spec more closely - in
particular, reuse operation and type names in `rest_types` to make
comparisons against spec more easy
* cleaner separation between client and server modules - modules common
between server and client such as `rest_types` and serialization move to
the spec folder - this allows the client to be built with less knowledge
about server internals
* some whole-file copies from altair branch
* rpc/node_api and rpc/node_rest_api also need to be copied
* remove new sync committee-related functionality
Jacek Sieka
Etan Kissling
tersec
Eugene Kabanov
Zahary Karadjov