Commit Graph

564 Commits

Author SHA1 Message Date
Zahary Karadjov ba59dd85cd Support for fallback web3 providers; Fix resource leaks on Eth1 monitor restarts 2021-04-08 12:46:42 +03:00
tersec 5f5e3e43c3
bump num-eth2-scenarios from v1.1.0-alpha.2 to v1.1.0-alpha3 (#2481) 2021-04-07 13:52:17 +00:00
Jacek Sieka adec9d878e
bump chronos (#2476) 2021-04-06 13:58:36 +02:00
tersec 49a5667288
update some v1.1.0 alpha1 to alpha2 (#2457)
* update some v1.1.0 alpha1 to alpha2

* remove unused getDepositMessage overload and move other out of datatypes/base

* bump nim-eth2-scenarios to download v1.1.0-alpha.2 test vectors

* construct object rather than result
2021-03-29 19:17:48 +00:00
Jacek Sieka 74732a23fe
json cleanups (#2456)
* move json-rpc specific marshalling to rpc
* serialize Epoch/Slot with cast to avoid Defect
* avoid a few eth1 deps
* simplify imports
2021-03-26 15:11:06 +01:00
Jacek Sieka 2695cfa864
EH cleanup (#2455)
almost 100% raises in nimbus-eth2 now!

* fix some rare exception-related crashes in json-rpc
2021-03-26 07:52:01 +01:00
Zahary Karadjov 2eacfc4685 Bump modules to take advantage of the new Json format flavors support
Since quite a lot of additional procs were now compiled as generics, this lead to compiler bugs that had
to be worked-around:

* The `Domain` type was renamed to `Eth2Domain` to avoid compilation errors
  due to conflicts with `nativesockets.Domain`.
  Similarly, `eth2_network.KeyPair` was renamed to `NetKeyPair`.

* A new more robust version of `hexToByteArray` was added to stew
2021-03-25 09:37:35 +02:00
Jacek Sieka 8b76ceed52
Fix minor exception effect issues (#2448)
Makes code compatible with
https://github.com/status-im/nim-chronos/pull/166 without requiring it.
2021-03-24 17:20:55 +01:00
Zahary Karadjov a727fc8e52
Merge stable into unstable 2021-03-24 03:24:20 +02:00
Zahary Karadjov 8391d3f83e
v1.0.12 2021-03-24 03:23:14 +02:00
Zahary Karadjov 5fd82ddaf5
Merge stable into unstable 2021-03-22 23:13:05 +02:00
Jacek Sieka d1357801e7
bumps 2021-03-22 20:53:27 +01:00
Zahary Karadjov af0d302398
Use the latest bootstrap nodes for Prater 2021-03-22 18:37:32 +02:00
Jacek Sieka 3743df6434
introduce unittest2 (#2437)
This commit introduces `unittest2`, driven by the need to fix several
long-standing issues with `unittest`, in particular exception handling,
test isolation, timing and others.

Upgrading to `unittest2` generally requires updating the unittest import
and potentially fixing the code to support the new `one-proc-per-test`
setup.

* nim-unittest2 has been moved from Stefans repository to status-im -
https://github.com/status-im/nim-unittest2/pull/2 fixes some exception
handling issues and brings it in line with latest nim devel - a notable
difference to unittest is that each test is run inside a proc which
gives it a fresh stack and removes a lot of globals, meaning tests
should run under conditions more similar to "normal" code and interfere
with each other less - this is a crucial first step to improving the
testing experience in general, that stefan implemented for the
multithreading support
* the multithreading in unittest2 is optional, and disabled by default
* nim-testutils contains utilities for fuzzing, as well as other
"large:ish" features that have dependencies -
https://github.com/status-im/nim-testutils/pull/27 pulls in unittest2 so
unittest2 can be kept "low-dependency"
* nim-chronos needs the above unittest changes to test its own new,
stricter exception handling -
https://github.com/status-im/nim-chronos/pull/166 - the strictess
introduced is expected to increase with time
2021-03-22 08:21:22 +01:00
Jacek Sieka 01fe564e46
chronos-based metrics (#2432)
This opens up the road for removing `-d:insecure` for metrics.
2021-03-19 10:36:38 +01:00
Zahary Karadjov c519c6012e
Merge branch 'stable' into unstable 2021-03-15 21:52:12 +02:00
Zahary Karadjov f3fc551ea7
Support for the prater testnet 2021-03-15 21:51:24 +02:00
nbc-bump-bot[bot] 2201480bd9
auto-bump nim-libp2p (#2409)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-03-15 17:39:10 +00:00
Jacek Sieka eb830c7968
Merge remote-tracking branch 'origin/stable' into unstable 2021-03-15 17:29:43 +01:00
Jacek Sieka f5180e0840
Revert "Nim-1.2.10" (#2411)
This reverts commit 12e15a0b8d.
2021-03-15 16:26:41 +01:00
Ștefan Talpalaru c5035c6eca dist: enable libbacktrace on ARM and ARM64
(and make sure it will work on Windows, when enabled)
2021-03-15 14:38:39 +02:00
Ștefan Talpalaru 12e15a0b8d Nim-1.2.10 2021-03-15 14:29:48 +02:00
nbc-bump-bot[bot] 4fcff4cd27
auto-bump nim-libp2p (#2406)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-03-15 07:36:36 +00:00
Zahary Karadjov 77ee210762
v1.0.10 2021-03-10 23:33:37 +02:00
zah 33c7f264bc
Bump news to fix resource leaks (#2388) 2021-03-10 12:22:41 +00:00
nbc-bump-bot[bot] a88d17dc04
auto-bump nim-libp2p (#2389)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2021-03-09 14:28:28 +01:00
Mamy André-Ratsimbazafy d425aabce2 BLST v0.3.3 2021-03-08 14:43:48 +02:00
kdeme 5f750f84b4 Use setupAddress for better IP and ports configuration 2021-03-06 00:09:09 +02:00
kdeme 769225b591 Bump nimbus-build-system to use nim-1-2-6-RC1-hotfixed
This should practically solve the segfaults we have been seeing in
issue #2134
2021-03-04 23:00:44 +02:00
nbc-bump-bot[bot] 9408a9af1c
auto-bump nim-libp2p (#2373)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2021-03-04 10:14:00 +01:00
nbc-bump-bot[bot] 965972dd0b
auto-bump nim-libp2p (#2360)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-03-01 10:26:16 +00:00
nbc-bump-bot[bot] 037a3e8946
auto-bump nim-libp2p (#2359)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-02-26 09:43:35 +00:00
Dustin Brody f14e7babb6 update eth2 specs to version v1.0.1 2021-02-25 14:21:59 +02:00
Ștefan Talpalaru 16abf2989b bump NimYAML 2021-02-25 14:20:26 +02:00
Ștefan Talpalaru 22620afa8f bump NimYAML for Nim-1.4 compatibility 2021-02-25 14:20:26 +02:00
nbc-bump-bot[bot] 45087f36bd
auto-bump nim-libp2p (#2350)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-02-25 11:25:00 +00:00
Eugene Kabanov 8478345619
Bump nim-bearssl to include latest fixes in submodule repository and cacert.pem. (#2354) 2021-02-24 19:22:40 +01:00
Zahary Karadjov 3f6834cce7
Fix a crash occurring with --slashing-db-kind=both 2021-02-22 21:05:08 +02:00
Zahary Karadjov 7e9648e52c
Use the latest Teku bootstrap nodes 2021-02-22 17:16:53 +02:00
Zahary Karadjov 5f924c0038 Bump modules; Compile with DEBUG logs by default 2021-02-19 08:44:13 +02:00
Kim De Mey aa29be288e
Bump eth2-testnets for updated LH bootstrap nodes (#2322) 2021-02-15 17:56:34 +01:00
Jacek Sieka 946608b572
bump json-rpc (#2321) 2021-02-15 13:47:57 +01:00
nbc-bump-bot[bot] ad66ba507e
auto-bump nim-libp2p (#2317)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-02-13 06:16:39 +00:00
nbc-bump-bot[bot] 8192015b86
auto-bump nim-libp2p (#2315)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-02-12 05:53:38 +00:00
nbc-bump-bot[bot] 0419a4122e
auto-bump nim-libp2p (#2309)
Co-authored-by: = <dryajov@users.noreply.github.com>
2021-02-11 02:20:39 +00:00
Jacek Sieka f012d7060b
better error message on disk / database issues (#2307)
bumps stew for better result defects as well
2021-02-10 13:21:06 +01:00
nbc-bump-bot[bot] c13205035e
auto-bump nim-libp2p (#2305)
Co-authored-by: = <dryajov@users.noreply.github.com>
2021-02-10 05:32:19 +00:00
Mamy Ratsimbazafy 03f47c8f2f
Slashing protection refactor - EIP 3076 (#2094)
* Create CLI tool for slashing export

* Use SQLite as a DB instead of a KV-store

* Keeps v1 and v2 DBs around

* Uses the same schema as Lighthouse v1.1.0

* Passes all interchange tests + skeleton of finalization pruning

* Removes tests that would violate v5 / minimal slashing DB and MinSlot rules

* Migration tool added using low-watermark scheme for faster migration of large number of validators
2021-02-09 17:23:06 +02:00
nbc-bump-bot[bot] 93cfc2d446
auto-bump nim-libp2p (#2300)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-02-08 08:32:47 +01:00
Jacek Sieka ece50c4706
bump nim-json-rpc (fixes https://github.com/status-im/nim-json-rpc/issues/93) (#2299) 2021-02-07 22:30:45 +01:00
nbc-bump-bot[bot] 0b560cc6fb
auto-bump nim-libp2p (#2296)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-02-06 09:50:08 +01:00
Zahary Karadjov 1394ef37fb
Tentative fix for the freezing Eth1 monitor activity 2021-02-04 17:01:47 +02:00
Ștefan Talpalaru 80c11546ff Windows binary release
CI: use both cores on GitHub Actions and set timeouts for the local testnet tests
2021-02-04 10:25:44 +02:00
Eugene Kabanov c4c04197c5
Bump nim-http-utils. (#2290) 2021-02-03 19:05:30 +01:00
Ștefan Talpalaru cadb1aae15
bump submodules (#2279)
* bump submodules

* undo a couple of bumps (32-bit failure)

* launch_local_testnet.sh: timeout and various fixes

* undo nim-http-utils bump (breaks RPC for VC)
2021-02-02 19:02:52 +01:00
Kim De Mey 73d9c2aa53
Add enr auto update cli option and bump nim-eth (#2278) 2021-02-02 09:07:21 +01:00
Zahary Karadjov fa99c3b417 Fix #2261
Also bumps Confutils to allow setting the hidden --web3-mode param
(to allow testing the eth1 syncing without validators)
2021-01-30 01:32:20 +02:00
Kim De Mey a3678d6bf0
Bump eth2-testnets for mainnet nimbus bootstrap nodes (#2276) 2021-01-29 14:07:38 +01:00
nbc-bump-bot[bot] 3e89aa0f4f
auto-bump nim-libp2p (#2275)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-01-29 09:48:27 +01:00
Dmitriy Ryajov 1117993a24
Libp2p connection limits (#2272)
* max peers gets set as connection limits

* pass max connections
2021-01-28 07:31:29 +01:00
tersec 4de9875cdf
Revert "pass max peers to libp2p (#2265)" (#2270)
This reverts commit 64e4d74f87.
2021-01-26 17:53:52 +01:00
Dmitriy Ryajov 64e4d74f87
pass max peers to libp2p (#2265) 2021-01-26 10:35:22 +01:00
Mamy Ratsimbazafy 70a03658e3
Block validation flow v2 + Batch (serial) sig verification (#2250)
* bump nim-blscurve

* Outline the block validation flow

* introduce the SigVerified types, pass the tests

* Split clearance/quarantine to prepare for batch crypto verif

* Add a batch signature collector

* Make clearance use SigVerified block and split verification between crypto and state transition

* Always use signedBeaconBlock for the onBlockAdded callback

* RANDAO signing_root is the epoch instead of the full block

* Support skipping BLS for testing

* Fix compilation of the validator client

* Try to fix strange errors MacOS and Jenkins (Clang, unknown type name br_hmac_drbg_context in stdlib_assertions.nim.c)

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561819858

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828025

* onBlockAdded callback should use TrustedSignedBeaconBlock https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561837261

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828946

* Use the application RNG: https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561815336

* Improve codegen of conversion zero-cost)

* Quick fixes with loadWithCache after #2259 (TODO: graceful error since pubkey validations is now done first in signatures_batch)

* Graceful handle rogue pubkeys and signatures now that those are lazy-loaded
2021-01-25 20:45:48 +02:00
nbc-bump-bot[bot] f0f292f509
auto-bump nim-libp2p (#2245)
Co-authored-by: = <dryajov@users.noreply.github.com>
2021-01-22 08:43:22 +01:00
Ștefan Talpalaru 9a0cac61c8 only ignore untracked files in submodules 2021-01-21 16:09:28 +02:00
Jacek Sieka 76f15302a7 better timesharing in eth2 processor
* use `idleAsync` to more evenly divide cpu attention when syncing in
particular - this gives networking better latency
* more strict exception handling in eth2_processor
2021-01-18 22:27:34 +02:00
nbc-bump-bot[bot] a0689e4237
auto-bump nim-libp2p (#2239)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-01-15 21:15:58 +09:00
nbc-bump-bot[bot] bb0a022968
auto-bump nim-libp2p (#2238)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-01-15 15:50:14 +09:00
Giovanni Petrantoni 295e3c9c73
Topics validation and direct peers (#2237)
* pick the right libp2p branch

* add topics validation
2021-01-15 04:17:06 +00:00
nbc-bump-bot[bot] ce64da1fcd
auto-bump nim-libp2p (#2235)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2021-01-15 01:39:09 +00:00
Kim De Mey 66d8f317cd
Use async queryRandom instead of inefficient randomNodes to discover nodes (#2211) 2021-01-14 08:58:13 +01:00
nbc-bump-bot[bot] f5efcc8669
auto-bump nim-libp2p (#2228)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-01-12 04:27:32 +01:00
nbc-bump-bot[bot] be3fac8495
auto-bump nim-libp2p (#2206)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-01-11 09:53:20 +01:00
Jacek Sieka 93b434d0ac
bumps (#2214)
* bumps

several modules pinned to merged branches

* faststreams master broken, go back
2021-01-07 13:12:37 +01:00
Ștefan Talpalaru e58a355a1c
CI release: ARM and ARM64 builds (#2213) 2021-01-07 10:19:29 +01:00
Jacek Sieka 52ca12c79f
bump nat, avoid one more finalizer (#2209) 2020-12-31 15:58:31 +01:00
Giovanni Petrantoni ed24f60f70
remove async from sub/unsub (#2197)
* remove await/async from sub/unsub

* fix unsubscribe wrong key (missed _snappy)

* use the right libp2p commit hash

* remove unused async

* fix inspector

* fix subnet calculation in RPC and insert broadcast attestations into node's pool

* unify codepaths to ensure only mostly-checked-to-be-valid attestations enter the pool, even from node's own broadcasts

* update attestation pool tests for new validateAttestation param

Co-authored-by: Dustin Brody <tersec@users.noreply.github.com>
2020-12-24 09:48:52 +01:00
Jacek Sieka 0f8a3a5ae8
checkpoint database at end of each slot (#2195)
* checkpoint database at end of each slot

To avoid spending time on synchronizing with the file system while doing
processing, the manual checkpointing mode turns off fsync during
processing and instead checkpoints the database when the slot has ended.

From an sqlite perspecitve, in WAL mode this guaranees database
consistency but may lead to data loss which is fine - anything missing
from the beacon chain database can be recovered on the next startup.

* log sync status and delay in slot start message

* bump
2020-12-18 22:01:24 +01:00
Jacek Sieka 452042b17b
msgid formatting, libp2p bump (#2198)
* msgid formatting, libp2p bump

* fix libp2p commit
2020-12-18 18:45:27 +00:00
Jacek Sieka ca2a09d424 bump secp 2020-12-16 19:41:12 +02:00
Ștefan Talpalaru ff23ca27c2
bump vendor/nimbus-benchmarking 2020-12-16 14:11:11 +01:00
Jacek Sieka b0086c5d99
bump libp2p (#2187)
* bump libp2p

* also stew
2020-12-16 09:36:51 +01:00
Jacek Sieka 4e191a06ac use stew/leb128
* 3 varint parsers -> 1
* 1/3 memory usage / allocations when writing stuff to libp2p
2020-12-15 17:06:15 +02:00
Ștefan Talpalaru e52c88386b
build warnings (#2179) 2020-12-14 16:45:31 +00:00
Kim De Mey f49fcb9df3
Bump nim-eth for discovery v5 ip limits (#2085) 2020-12-11 14:35:08 +01:00
Ștefan Talpalaru e59437c5e2
new Make target: "benchmarks" (#2174)
It runs some benchmarks, collects the output and generates HTML charts.
2020-12-11 12:12:43 +01:00
Mamy Ratsimbazafy 2b3b219c77
Bump blst: perf + 32-bit platforms support + pure C fallback + drop Miracl fallback tests (#2152)
* bump BLST

* Cached subgroup checks

* Bump BLST - BLST now has pure C fallback for all arch (Miracl unneeded)

* Drop testing Miracl fallback in Nimbus

* Charting uncharted waters: 32-bit should imply -mno-adx
2020-12-10 10:18:07 +01:00
Jacek Sieka c4d9d5f64e
bump nim-stew (no change) 2020-12-09 17:24:25 +01:00
Jacek Sieka 0b33183341
bumps (#2154)
* avoid dangling cstring returns in Result, fixing some low-risk crash
potential
* bump libp2p for connection race fix and a few cleanups
2020-12-07 17:09:05 +01:00
Zahary Karadjov 4feb0a308e Fix #2125 (ETH status bar display); Bump LibP2P 2020-12-02 00:03:59 +02:00
Zahary Karadjov 4eaf29875a Less risk for Eth1-induced delays in block proposal 2020-12-01 15:38:50 +02:00
Zahary Karadjov 38f7558e50 Work around a strange codegen issue to fix local sim in CI; Bump LibP2P 2020-12-01 15:38:00 +02:00
Jacek Sieka 7e4fd325b5
bump nim-libp2p 2020-11-29 14:47:19 +01:00
= e1fb2d98fb auto-bump nim-libp2p 2020-11-28 09:20:02 +00:00
Jacek Sieka d16e127daf
Peer stuff (#2084)
* Revert "Revert "Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065)" (#2082)"

This reverts commit 7cc3dc8027.

* fix nil disconnectedFut crash

* fixes

don't resetPeer, it causes peer miscounts

* disconnect disconnecting peers

...when there's a race.

* avoid connection spamming

* never decrease SeenTable timeout
* only recover ENR for known peers

* seen only when really disconnected
2020-11-26 20:23:45 +01:00
Giovanni Petrantoni 9b9aa93cd4
Seenttl by spec (#2083)
* add rpc call to get chronos futures at runtime

* add seenTTL gossip parameter

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-11-26 13:08:22 +01:00
Jacek Sieka 7cc3dc8027
Revert "Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065)" (#2082)
This reverts commit d041287a4f.
2020-11-26 09:05:23 +01:00
Eugene Kabanov d041287a4f
Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065)
* Initial commit.

* Fix log lines and compilation error.

* Add get_v1_node_peers() implementation.

* Fix peer's lifetime states.

* Use the most recent multiaddress.

* Fix assign NewPeerScore again.
Fix compilation error with last seen address.
Fix Peer upgraded log line place.

* syncing, health, peer_count, peer_id and fixes for identity.

* Fix compilation problems.

* Move object declaration to callsigs.
Fix identity addresses fields.

* Finish node RPC calls.

* Avoid leak of lifetime future.

* Bump chronos.

* Fix json generator problem.
2020-11-26 08:09:59 +01:00
Zahary Karadjov 7fdafa65ee
New bootstrap nodes for Mainnet 2020-11-25 21:27:40 +02:00
zah 372c9b798c
Fix the corrupted database state on Pyrmont nodes; Add mainnet genesis (#2056)
* Handle some web3 timeouts better

* Add support for developer .env files

* Eth1 improvements; Mainnet genesis state

Notable changes:

* The deposits table have been removed from the database. The client
  will no longer process all deposits on start-up.

* The network metadata now includes a "state snapshot" of the deposit
  contract. This allows the client to skip syncing deposits made prior
  to the snapshot (i.e. genesis). Suitable metadata added for Pyrmont
  and Mainnet.

* The Eth1 monitor won't be started unless there are validators attached
  to the node.

* The genesis detection code is now optional and disabled by default

* Bugfix: The client should not produce blocks that will fail validation
  when it hasn't downloaded the latest deposits yet

* Bugfix: Work around the database corruption affecting Pyrmont nodes

* Remove metadata for Toledo and Medalla
2020-11-24 22:21:47 +01:00