Commit Graph

3431 Commits

Author SHA1 Message Date
Mamy Ratsimbazafy e6b559a35a
Slashing db pruning [Merge only after v2 has been default for 1 noticeable release] (#2452)
* Enable slashing DB pruning

* integrate slashing DB pruning with onSlotEnd

* rebase tests
2021-05-10 16:32:28 +02:00
tersec f9b964ca5d
bump nim-eth2-scenarios for merge test vectors (#2553) 2021-05-10 13:56:31 +00:00
Jacek Sieka 867d8f3223
Perform attestation check before broadcast (#2550)
Currently, we have a bit of a convoluted flow where when sending
attestations, we start broadcasting them over gossip then pass them to
the attestation validation to include them in the local attestation pool
- it should be the other way around: we should be checking attestations
_before_ gossipping them - this serves as an additional safety net to
ensure that we don't publish junk - this becomes more important when
publishing attestations from the API.

Also, the REST API was performing its own validation meaning
attestations coming from REST would be validated twice - finally, the
JSON RPC wasn't pre-validating and would happily broadcast invalid
attestations.

* Unified attestation production pipeline with the same flow for gossip,
locally and API-produced attestations: all are now validated and entered
into the pool, then broadcast/republished
* Refactor subnet handling with specific SubnetId alias, streamlining
where subnets are computed, avoiding the need to pass around the number
of active validators
* Move some of the subnet handling code to eth2_network
* Use BitArray throughout for subnet handling
2021-05-10 09:13:36 +02:00
tersec 39da640beb
use getStateField() in ncli_db (#2549) 2021-05-07 13:14:20 +00:00
Jacek Sieka 646923c3dd
add attestation stats tool to ncli_db (#2539)
This also makes future efforts to provide metrics and logs for
attestation efficiency easier

* Export rewards from epoch transition
* Use less memory for reward calculation (bool -> set[enum], field
alignment)
* Reuse reward memory when replaying, avoiding spike
* Allow replaying any range in ncli_db benchmark
2021-05-07 13:36:21 +02:00
Jacek Sieka 5cd5da74c4
Gossipsub unsubscribe fixes (fixes #2540) (#2545) 2021-05-07 06:31:28 +02:00
Jacek Sieka 01b404f43d
bump chronos (#2544) 2021-05-06 18:20:37 +02:00
tersec 1d6c8ee9ab
store full state 4x less often (#2542) 2021-05-06 07:36:18 +02:00
tersec dd43a2c3b0
bump nim-eth2-scenarios to get merge SSZ test vectors (#2541) 2021-05-05 15:35:36 +00:00
Ștefan Talpalaru 2018a55c51
bump a few submodules (#2529) 2021-05-05 08:55:39 +02:00
Jacek Sieka 427c0f307c
avoid extraneous hash root calculation (#2537)
When applying a block, we'll currently compute a state root for the
state after slot processing but before block processing - this is
unnecessary when a block is being applied because the intermediate state
root is never observed.
2021-05-05 08:54:21 +02:00
Jacek Sieka 4d74c742da
move ENRForkID into `spec` (#2538)
* move ENRForkID into `spec`

also get rid of strformat in topic formation and fix some case
discrepancies

* also move `Eth2Metadata`
2021-05-04 17:28:48 +02:00
Jacek Sieka efdf759cc0
avoid some slashing protection queries (#2528)
This PR reduces the number of database queries for slashing protection
from 5 reads and 1 write to 2 reads and 1 write in the optimistic case.

In the process, it removes user-level support for writing the database
in the version 1 format in order to simplify the code flow, and prevent
code rot. In particular, the v1 format was not covered by any unit tests
and has no advantages over v2. The concrete code to read and write it
remains for now, in particular to support upgrades from v1 to v2.

The branch also removes the use of concepts which doesn't work with
checked exceptions - in particular, this highlights code that both
raises exceptions and returns error codes, which could be cleaned up in
the future.

* Cache internal validator ID
* Rely on unique index to check for trivial duplicate votes
* Combine two surround vote queries into one
* Combine API for checking and registering slashing into single function

The slashing DB is normally not a bottleneck, but may become one with
high attached validator counts.
2021-05-04 15:17:28 +02:00
tersec 290b889ce6
non-intrusive, novel portions of merge (#2535) 2021-05-04 11:54:19 +00:00
tersec e0f4d28116
rename initialize_beacon_state to initialize_beacon_state_from_eth1 (#2536) 2021-05-04 12:19:11 +02:00
Ștefan Talpalaru 51fde853d5
Merge branch 'stable' into unstable 2021-05-03 15:26:56 +02:00
Zahary Karadjov f369682b3f
v1.2.2 2021-05-03 11:13:12 +03:00
Chris Hobcroft a3b9990beb
Fixed confusing typo (#2533)
"One or more" appears to be a better description than "One of more"
2021-05-02 17:01:49 +02:00
Zahary Karadjov faebe3470c Don't execute CI builds (Github actions) when no build inputs are changed 2021-05-01 10:24:46 +03:00
Zahary Karadjov e60dd8e4f6 Stronger validation for third-party Eth1Data votes 2021-05-01 10:24:31 +03:00
Jacek Sieka 9d031a8446
Update README links 2021-04-29 10:46:09 +02:00
Jacek Sieka b899878a23
Update README.md
Fix links, clarify scope
2021-04-29 10:43:32 +02:00
Ștefan Talpalaru bac1a64bf6
Jenkins: disable tools/tests parallelism (#2526)
New REST test depends on a tool and, if it's missing, it builds it by
itself, thus interacting dangerously with a parallel "Tools" stage.

We give up on that parallelism because it's unlikely to provide relevant
gains, now that `make test` starts by building all test binaries.
2021-04-29 03:28:11 +02:00
Jacek Sieka ce49da6c0a
Introduce unittest2 and junit reports (#2522)
* Introduce unittest2 and junit reports

* fix XML path

* don't combine multiple CI runs

* fixup

* public combined report also

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 18:41:02 +02:00
yslcrypto 4cdbc17e39 remove references to devel 2021-04-28 16:06:00 +02:00
yslcrypto ec1aa622b0 Merge branch 'unstable' of github.com:status-im/nim-beacon-chain into unstable 2021-04-28 16:00:02 +02:00
0xmiel 7af4f7c665
Guide updates (#2525)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* update binaries page

* rm extra emacs generated files

* remove extra emacs generated file

* fix windows typ

* fix eth1 title

* page on adding additional validator
2021-04-28 15:55:43 +02:00
yslcrypto a42b3e1f9c Merge branch 'unstable' of github.com:status-im/nim-beacon-chain into unstable 2021-04-28 13:39:46 +02:00
nbc-bump-bot[bot] d4cda02c97
auto-bump nim-libp2p (#2523)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-04-28 09:57:25 +02:00
Dustin Brody 7f42d38219 rename initialize_beacon_state{_from_eth1,}; suppress warnings when doppelganger detection disabled 2021-04-28 00:12:41 +03:00
cheatfate 18da0f8928 Bump nim-chronos to fix issues in httpserver.
Bump nim-http-utils to fix issues in httpserver.
2021-04-28 00:00:00 +03:00
Eugene Kabanov 0c3302b826
REST API test framework and tests. (#2492)
* REST API test framework and tests.

* Fix ValidatorIndex tests to properly handle int32, but not uint32 values.

* Fix tests to follow latest REST fixes.

* refactor restapi.sh

and add it to the test suite

* Fix issues.
Add delay timeout which is required.

* Fix restapi.sh script for Windows.

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-27 23:46:24 +03:00
Jacek Sieka 7dba1b37dd
remove attestation/aggregate queue (#2519)
With the introduction of batching and lazy attestation aggregation, it
no longer makes sense to enqueue attestations between the signature
check and adding them to the attestation pool - this only takes up
valuable CPU without any real benefit.

* add successfully validated attestations to attestion pool directly
* avoid copying participant list around for single-vote attestations,
pass single validator index instead
* release decompressed gossip memory earlier, specially during async
message validation
* use cooked signatures in a few more places to avoid reloads and errors
* remove some Defect-raising versions of signature-loading
* release decompressed data memory before validating message
2021-04-26 22:39:44 +02:00
nbc-bump-bot[bot] 8a3c070337
auto-bump nim-libp2p (#2517)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-04-23 12:54:06 +02:00
yslcrypto 70607f8b49 Merge branch 'unstable' of github.com:status-im/nim-beacon-chain into unstable 2021-04-22 13:12:48 +02:00
Sacha Saint-Leger 7e31798649
Nimbus guide simplifications (#2516)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* simplify web3 provider instructions

* add donation address
2021-04-22 13:12:24 +02:00
yslcrypto e73aafffc3 Merge branch 'unstable' of github.com:status-im/nim-beacon-chain into unstable 2021-04-22 12:25:48 +02:00
Sacha Saint-Leger bbb8866c09
Update README.md
Add donation address
2021-04-21 11:39:30 +02:00
Zahary Karadjov 5c7194bad0
More release notes tweaks 2021-04-20 20:06:40 +03:00
Zahary Karadjov feec2c2b65
Final version of the 1.2.1 release notes 2021-04-20 18:53:38 +03:00
Zahary Karadjov 8cd570fc52
Merge branch 'stable' into unstable 2021-04-20 18:17:54 +03:00
Zahary Karadjov 2faaf391a0
v1.2.1 2021-04-20 16:55:37 +03:00
Jacek Sieka 54d6884c89
fix sync issue when upgrading from 1.1.0-inited db
This patch writes a full genesis state to `kvstore` if one was missing,
which fixes 1.2.0 restarting sync when upgrading from 1.1.0, or when
downgrading to a pre-1.1.0 release.
2021-04-20 16:55:18 +03:00
Zahary Karadjov 9b57535f33
Merge branch 'stable' into unstable 2021-04-19 19:55:20 +03:00
Zahary Karadjov e1a8049ed5
v1.2.0 2021-04-19 19:54:24 +03:00
Sacha Saint-Leger 4bbd9717ed
Update README.md 2021-04-19 14:38:45 +02:00
Ștefan Talpalaru 5b32f7ca17 Nim-1.2.12 2021-04-19 10:53:58 +03:00
Ștefan Talpalaru 0b8919f500
we now have metrics support in distrib. bins 2021-04-18 18:37:30 +02:00
Jacek Sieka daf98e4330 fix committee vs subnet confusion 2021-04-18 14:17:45 +03:00
Jacek Sieka 4959da78a8
mem fixes / libp2p (#2510)
* mem fixes / libp2p

* bump
2021-04-18 12:30:25 +02:00