Commit Graph

3148 Commits

Author SHA1 Message Date
tersec 8d7792e6e9
add Altair domains and participation flags; clean up imports (#2462) 2021-04-04 16:24:45 +00:00
Mamy Ratsimbazafy 6b13cdce36
Batch attestations (#2439)
* batch attestations

* Fixes (but now need to investigate the chronos 0 .. 4095 crash similar to https://github.com/status-im/nimbus-eth2/issues/1518

* Try to remove the processing loop to no avail :/

* batch aggregates

* use resultsBuffer size for triggering deadline schedule

* pass attestation pool tests

* Introduce async gossip validators. May fix the 4096 bug (reentrancy issue?) (similar to sync unknown blocks #1518)

* Put logging at debug level, add speed info

* remove unnecessary batch info when it is known to be one

* downgrade some logs to trace level

* better comments [skip ci]

* Address most review comments

* only use ref for async proc

* fix exceptions in eth2_network

* update async exceptions in gossip_validation

* eth2_network 2nd pass

* change to sleepAsync

* Update beacon_chain/gossip_processing/batch_validation.nim

Co-authored-by: Jacek Sieka <jacek@status.im>

Co-authored-by: Jacek Sieka <jacek@status.im>
2021-04-02 16:36:43 +02:00
Jacek Sieka f821bc878e
Remove `-d:insecure` compile option (#2468)
With metrics running on top of chronos, the metrics server no longer
needs to be compiled in conditionally - it remains disabled by default.
2021-04-01 14:44:11 +02:00
Jacek Sieka e746cb09c2
Fix block number request (#2470)
leading 0's not tolerated
2021-04-01 14:43:03 +02:00
Jacek Sieka 8f441588cd
Add a few help defaults, format all of them the same (#2284) (#2467)
No support in confutils, so so this code simply hardcodes it
2021-04-01 12:28:57 +00:00
Jacek Sieka 3cd7cebc7c
Fix block dag pruning frequency (#2469)
Should always prune after finality change but not more than once
2021-04-01 13:26:17 +02:00
Sacha Saint-Leger 981e25a99b
Nimbus book: update homepage (#2466)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* Update homepage
2021-03-31 14:18:20 +02:00
Sacha Saint-Leger 0355c27194
Nimbus book: add Prater page (#2465)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* add prater testnet page
2021-03-31 13:59:12 +02:00
tersec bd8b60f8c8
use epochref for get_committee_assignments() to avoid repeated shuffling (#2463)
* use epochref for get_committee_assignments() to avoid repeated shuffling

* remove unnecessary imports and StateCache() construction
2021-03-30 15:01:47 +00:00
Mamy Ratsimbazafy a9938a2067
Fix pruning time display (#2461)
* Fix pruning time display

* remove import
2021-03-30 09:40:28 +02:00
Jacek Sieka 714620b13f
script for finding slow logs 2021-03-30 09:39:16 +02:00
tersec 49a5667288
update some v1.1.0 alpha1 to alpha2 (#2457)
* update some v1.1.0 alpha1 to alpha2

* remove unused getDepositMessage overload and move other out of datatypes/base

* bump nim-eth2-scenarios to download v1.1.0-alpha.2 test vectors

* construct object rather than result
2021-03-29 19:17:48 +00:00
Zahary Karadjov 1af07ad124
Simple RPC for modifying the Graffiti bytes 2021-03-26 19:42:20 +02:00
Jacek Sieka 74732a23fe
json cleanups (#2456)
* move json-rpc specific marshalling to rpc
* serialize Epoch/Slot with cast to avoid Defect
* avoid a few eth1 deps
* simplify imports
2021-03-26 15:11:06 +01:00
Jacek Sieka 2695cfa864
EH cleanup (#2455)
almost 100% raises in nimbus-eth2 now!

* fix some rare exception-related crashes in json-rpc
2021-03-26 07:52:01 +01:00
Mamy Ratsimbazafy 596cf3d674
Fix comment about backpressure handling in verif queue [skip ci] (#2453) 2021-03-25 15:09:53 +01:00
Zahary Karadjov 2eacfc4685 Bump modules to take advantage of the new Json format flavors support
Since quite a lot of additional procs were now compiled as generics, this lead to compiler bugs that had
to be worked-around:

* The `Domain` type was renamed to `Eth2Domain` to avoid compilation errors
  due to conflicts with `nativesockets.Domain`.
  Similarly, `eth2_network.KeyPair` was renamed to `NetKeyPair`.

* A new more robust version of `hexToByteArray` was added to stew
2021-03-25 09:37:35 +02:00
Jacek Sieka 8b76ceed52
Fix minor exception effect issues (#2448)
Makes code compatible with
https://github.com/status-im/nim-chronos/pull/166 without requiring it.
2021-03-24 17:20:55 +01:00
Kim De Mey a4a2c1c0e1
Add discovery query with forkId and attnets filter (#2420) 2021-03-24 11:48:53 +01:00
tersec 36311bfc05
incorporate proposals into nextActionWait; switch some proc to func (#2438) 2021-03-24 10:05:04 +00:00
Zahary Karadjov a727fc8e52
Merge stable into unstable 2021-03-24 03:24:20 +02:00
Zahary Karadjov 8391d3f83e
v1.0.12 2021-03-24 03:23:14 +02:00
kdeme adcae8d416 Avoid dialing peers that will be connection limited by libp2p 2021-03-23 14:46:52 +02:00
tersec 3076f5c3b6
rm std/random from beacon_chain and rm attestation timing randomness (#2442)
* remove added attestation timing randomness

* remove os/random from rest of beacon_chain, primarily deposit_contract

* remove scaffolding

* randomize std/random seed in beacon node and validator client

* use CSPRNG to more securely seed std/random
2021-03-23 06:57:10 +00:00
Giovanni Petrantoni 9558946df4
Expose gossip direct peers (#2246)
* Expose gossip direct peers

* direct-peer fixes

* add explicit info about direct peer

* build fixes wip
2021-03-23 15:10:17 +09:00
Zahary Karadjov 5fd82ddaf5
Merge stable into unstable 2021-03-22 23:13:05 +02:00
Jacek Sieka d1357801e7
bumps 2021-03-22 20:53:27 +01:00
Zahary Karadjov 25695d1e4f
v1.0.11 2021-03-22 19:25:34 +02:00
tersec 85289269d1
increase block proposal speed with many validators (#2423)
* increase block proposal speed with many validators

* document CookedSig rationale
2021-03-22 19:02:27 +02:00
Sacha Saint-Leger 5362d966ad
Nimbus guide: Rocket Pool edits (#2446)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* update joe's guide

* incorporate pineapple's feedback
2021-03-22 17:57:22 +01:00
Zahary Karadjov af0d302398
Use the latest bootstrap nodes for Prater 2021-03-22 18:37:32 +02:00
Ștefan Talpalaru b2c4441834
book: document "logrotate" (#2390)
* book: document "logrotate"

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* rewording

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

Co-authored-by: Sacha Saint-Leger <sacha@status.im>
2021-03-22 16:40:26 +01:00
tersec dfd99ec943
Altair (HF1/v1.1.0) minimal and mainnet presets/constants (#2444)
* Altair mainnet & minimal presets

* std/math not used
2021-03-22 14:44:45 +00:00
Mentor Palokaj 895fe4baf1
Make it clear that the default port for ETH2 is 9000 udp/tcp (#2421)
* Add default port info

* Add default port info

* Edit CLI and docs to reflect 9000 port

* Apply suggestions from code review

Co-authored-by: Jacek Sieka <arnetheduck@gmail.com>
2021-03-22 08:22:56 +01:00
Jacek Sieka 3743df6434
introduce unittest2 (#2437)
This commit introduces `unittest2`, driven by the need to fix several
long-standing issues with `unittest`, in particular exception handling,
test isolation, timing and others.

Upgrading to `unittest2` generally requires updating the unittest import
and potentially fixing the code to support the new `one-proc-per-test`
setup.

* nim-unittest2 has been moved from Stefans repository to status-im -
https://github.com/status-im/nim-unittest2/pull/2 fixes some exception
handling issues and brings it in line with latest nim devel - a notable
difference to unittest is that each test is run inside a proc which
gives it a fresh stack and removes a lot of globals, meaning tests
should run under conditions more similar to "normal" code and interfere
with each other less - this is a crucial first step to improving the
testing experience in general, that stefan implemented for the
multithreading support
* the multithreading in unittest2 is optional, and disabled by default
* nim-testutils contains utilities for fuzzing, as well as other
"large:ish" features that have dependencies -
https://github.com/status-im/nim-testutils/pull/27 pulls in unittest2 so
unittest2 can be kept "low-dependency"
* nim-chronos needs the above unittest changes to test its own new,
stricter exception handling -
https://github.com/status-im/nim-chronos/pull/166 - the strictess
introduced is expected to increase with time
2021-03-22 08:21:22 +01:00
tersec baa43ae7f7
add stricter ABI-compatibility checks for punned types (#2434)
* add stricter ABI-compatibility checks for punned types

* enforce type isomorphism for each pair of cast types to eliminate potential safety hole
2021-03-19 12:30:20 +00:00
Jacek Sieka 01fe564e46
chronos-based metrics (#2432)
This opens up the road for removing `-d:insecure` for metrics.
2021-03-19 10:36:38 +01:00
tersec 97850741a0
remove unused imports in slashing prtection (#2436) 2021-03-19 08:26:02 +00:00
tersec 21b2367bf1
Remove Travis and Azure build statuses from README.md (#2433) 2021-03-18 21:50:59 +00:00
Dustin Brody a3a6aa68e8
check if Nim-caching version also works 2021-03-17 19:58:33 +01:00
Dustin Brody 732a437fa8
maybe fix Jenkins syntax 2021-03-17 19:45:27 +01:00
Dustin Brody 51079770b9
disable Nim compiler cache 2021-03-17 19:41:02 +01:00
tersec 9ff6118c28
clear nimcache on make update (#2426)
* test clearing nimcache

* don't be verbose
2021-03-17 17:29:34 +00:00
Sacha Saint-Leger 756c82671d
Update rp-quick-start.md 2021-03-17 17:12:32 +01:00
Sacha Saint-Leger b9f3fc4ceb
Rocket Pool guide (#2425)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* Rocket Pool guide
2021-03-17 15:33:28 +01:00
tersec b059cb42c5
increase block proposal speed with many validators (#2423)
* increase block proposal speed with many validators

* document CookedSig rationale
2021-03-17 13:35:59 +00:00
Jacek Sieka 3cb31e66b4
set upper bound on EpochRef cache (#2403)
* set upper bound on EpochRef cache

* max 32 EpochRef instances
* less memory waste in BlockRef by removing EpochRef seq that is mostly
unused (~20mb)
* less memory waste in dag block lookup by not keeping an extra copy of
digest (~70mb)
* fix `==` and `$` for Eth2Digest
* remove `ChainDAG.tmpState` (~50mb?)

all in all, this branch cuts mainnet memory usage by ~160-180mb and puts
limits on EpochRef cache usage - where normally it hovered around 950mb
before, it's now sitting at 600-700mb on my machine.

* docs
2021-03-17 11:17:15 +01:00
Kim De Mey 5e3770e994
Remove unused test (#2424) 2021-03-17 06:36:48 +00:00
Mamy Ratsimbazafy 6e38d474cc
Add pruning timings (#2422) 2021-03-17 07:30:16 +01:00
Ștefan Talpalaru 683edbff7a
restore terminal echoing after pressing Ctrl+C at a password prompt (#2412) 2021-03-16 08:06:45 +00:00