Corrects a regression from #5998 that led to crashes in #6046.
In `trustedNodeSync` mode, the config does not contain genesis keys,
so attempting to load from them is a `Defect`.
Fix the `/eth/v1/beacon/deposit_snapshot` API to produce proper EIP-4881
compatible `DepositTreeSnapshot` responses. The endpoint used to expose
a Nimbus-specific database internal format.
Also fix trusted node sync to consume properly formatted EIP-4881 data
with `--with-deposit-snapshot`, and `--finalized-deposit-tree-snapshot`
beacon node launch option to use the EIP-4881 data. Further ensure that
`ncli_testnet` produces EIP-4881 formatted data for interoperability.
- fix conversion error with `or` on futures with `{.async: (raises: []).}`
- Fix `or` deadlock issue.
- Ensure that `OwnCancelSchedule` flag will not be removed from `wait()` and `withTimeout()`.
- Add missing parts of defaults buffer size increase.
- Avoid `ValueError` effect in varargs `race`/`one`
- fix circular reference in timer
- Ensure `transp.reader` is reset to `nil` on error
This reverts commit 21601f59e2.
Reason: `/eth/v1/beacon/light_client/updates?start_period=0&count=128`
provides malformed data. Reverting fixes it locally.
Download JSON and SSZ, then use this script to validate (Holesky).
Need to export some symbols for parsing.
```nim
import
stew/io2,
./beacon_chain/spec/[digest, forks, helpers, presets],
./beacon_chain/spec/eth2_apis/[eth2_rest_serialization, rest_light_client_calls]
let jsonBytes = io2.readAllBytes("/Users/etan/Downloads/json_updates.json").get()
let
(cfg, unknowns) = readRuntimeConfig("/Users/etan/Documents/Repos/nimbus-eth2/vendor/holesky/custom_config_data/config.yaml")
genesis_validators_root = Eth2Digest.fromHex("0x9143aa7c615a7f7115e2b6aac319c03529df8242ae705fba9df39b79c59fa8b1")
forkDigests = newClone ForkDigests.init(cfg, genesis_validators_root)
let updates = seq[ForkedLightClientUpdate].decodeJsonLightClientObjects(jsonBytes, cfg, forkDigests)
var entries = newSeqOfCap[RestVersioned[ForkedLightClientUpdate]](updates.len)
for update in updates:
let
contextEpoch = withForkyUpdate(update):
when lcDataFork > LightClientDataFork.None:
forkyUpdate.contextEpoch
else:
continue
contextFork = cfg.consensusForkAtEpoch(contextEpoch)
entries.add RestVersioned[ForkedLightClientUpdate](
data: update,
jsonVersion: contextFork,
sszContext: forkDigests[].atconsensusFork(contextFork))
let res =
block:
var default: seq[byte]
try:
var stream = memoryOutput()
for e in entries:
withForkyUpdate(e.data):
when lcDataFork > LightClientDataFork.None:
var cursor = stream.delayFixedSizeWrite(sizeof(uint64))
let initPos = stream.pos
stream.write e.sszContext.data
var writer = SszWriter.init(stream)
writer.writeValue forkyUpdate
cursor.finalWrite (stream.pos - initPos).uint64.toBytesLE()
stream.getOutput(seq[byte])
except SerializationError:
default
except IOError:
default
let sszBytes = io2.readAllBytes("/Users/etan/Downloads/ssz_updates.ssz").get()
let sszUpdates = seq[ForkedLightClientUpdate].decodeSszLightClientObjects(res, cfg, forkDigests)
doAssert updates.len == sszUpdates.len
for i in 0 ..< updates.len:
doAssert updates[i].kind == sszUpdates[i].kind
withForkyUpdate(updates[i]):
when lcDataFork > LightClientDataFork.None:
doAssert forkyUpdate == sszUpdates[i].forky(lcDataFork)
doAssert sszBytes == res
```
EIP-4881 was never correctly implemented, the `DepositTreeSnapshot`
structure has nothing to do with its actual definition. Reflect that
by renaming the type to a Nimbus-specific `DepositContractSnapshot`,
so that an actual EIP-4881 implementation can use the correct names.
- https://eips.ethereum.org/EIPS/eip-4881#specification
Notably, `DepositTreeSnapshot` contains a compressed sequence in
`finalized`, only containing the minimally required intermediate roots.
That also explains the incorrect REST response reported in #5508.
The non-canonical representation was introduced in #4303 and is also
persisted in the database. We'll have to maintain it for a while.
`nextActionWait` currently shows `n/a` if only proposal is scheduled
but no attestation, e.g., attestation was already made for current
epoch and validator is exiting next epoch so doesn't have another
attestation lined up. It's an edge case but it's still more correct
to also log `nextActionWait` if only proposal is scheduled.
When using `--external-beacon-api-url`, one has to accompany it with
either `--trusted-block-root` or `--trusted-state-root`. If neither is
specified, we can fallback to a deeply finalized noncontroversial block
root. For networks that started post Altair, e.g., Holesky, the genesis
block root fulfills that requirement, as in, it is implicitly trusted.
Therefore, if only `--external-beacon-api-url` is provided without any
`--trusted-block-root` or `--trusted-state-root`, use genesis block root
if it is a viable starting point (post-Altair).
```
build/nimbus_beacon_node \
--network=holesky \
--data-dir="$HOME/Downloads/nimbus/data/holesky" \
"--external-beacon-api-url=http://unstable.holesky.beacon-api.nimbus.team" \
--tcp-port=9010 --udp-port=9010 \
--rest --log-level=DEBUG \
--no-el
```
Daily CI uses an outdated Nim 1.6 because it uses `origin/version-1-6`
which is not maintained very regularly. Pull from `upstream/version-1-6`
instead, same as in `ci.yml`, and also make sure that `--mm:refc` is
turned on for `upstream/devel`.
Update `generate_makefile` to produce sensible error message in all
exceptional situations, instead of just some of them. These may surface
as one of the first things to someone getting started with the repo.
Annotate the `research` and `test` files for which no further changes
are needed to successfully compile them, to not interfere with periodic
tasks such as spec reference bumps.
`scanf` apparently has both a `bool` return as well as raising random
exceptions depending on what functions get called by the `macro`.
To make this explicit, catch the `ValueError` from the generated
`parseInt` call, to separate `scanf` behaviour from the actual SSZ
test logic. In the end, it mostly doesn't matter as there are some
`doAssert wasMatched` on the next line (not everywhere though).
But it still makes the `scanf` internals explicit, so is clearer.
Add `{.raises.}` annotations to `tests` files where needed to enable
`{.push raises: [].}`. Avoids interfering with periodic changes such as
spec version bumps, and avoids special casing folders when editing.
The effort to maintain `{.raises.}` is trivial after the initial round.
`test_peer_pool` is a bit different from the other tests as it uses
`closureScope` which doesn't play well with `{.push raises: [].}`.
Define an overload instead that allows passing `{.raises.}` to the
`template`. This allows using `unittest2`'s exception handler without
having to refactor the test.
Etan Kissling
tersec
Jacek Sieka
omahs