Commit Graph

3300 Commits

Author SHA1 Message Date
Ștefan Talpalaru dd84ecd520
CI cron: debug fixture download 2021-05-20 18:28:03 +02:00
Ștefan Talpalaru 57259cf0d9
CI cron: debug fixture download 2021-05-20 18:21:16 +02:00
Ștefan Talpalaru 876a4cd684
CI cron: try a different expression syntax 2021-05-20 17:52:28 +02:00
Zahary Karadjov eb55fe2419
v1.3.0 2021-05-19 09:25:22 +03:00
Ștefan Talpalaru 06dca32786
CI: daily cron job testing 3 Nim branches 2021-05-18 21:17:23 +03:00
Zahary Karadjov 6b14d33ad6
Bump Confutils
This fixes an issue with the --bootstrap-file parameter which was
accidentally considered mandatory due to an upstream issue.
2021-05-18 11:02:01 +03:00
Zahary Karadjov 2cb1396969
Log the slashing DB pruning time 2021-05-17 21:42:28 +03:00
Ștefan Talpalaru 1f1367d09a
macOS binary distribution (both AMD64 and ARM64) 2021-05-17 21:42:28 +03:00
Zahary Karadjov f6a1f602b4
Handle changes in the latest version of Confutils 2021-05-17 21:42:28 +03:00
Zahary Karadjov b9924214ab
Better error-handling for the slashingdb import/export feature
* Error when specifying an invalid --data-dir (or --validator-dir)
* Error when entering an invalid validator public key (e.g. invalid hex value)
* Warning when attempting to export a validator not present in the local database

Some unnecessary remains of the v1 mode has been removed as well
2021-05-17 21:42:23 +03:00
Zahary Karadjov 5fadaa247c
Remove ncli_slashing 2021-05-17 18:17:42 +03:00
Zahary Karadjov 5c313b958e
Simplify the slashing db import/export CLI 2021-05-17 17:12:03 +03:00
Mamy André-Ratsimbazafy dacc508992
slashing import integrated in NBC 2021-05-16 21:48:38 +03:00
Mamy André-Ratsimbazafy 0574531c43
add restore from slashing DB 2021-05-16 21:45:24 +03:00
Jacek Sieka 2c257438a0 Upgrade database schema
This is a minimal performance hotfix for the kvstore table, such that
new databases get a kvstore table with rowid, fixing the most urgent
performance problem we have with pruning.
2021-05-14 20:08:07 +03:00
Jacek Sieka 895ccd1c95 clean up imports (#2557) 2021-05-14 20:08:07 +03:00
Zahary Karadjov 0a91c3c592
Prepare for the v1.3.0 release (more to come) 2021-05-12 16:28:10 +03:00
Jacek Sieka 0a477eb2d3
fix subnet logic (#2555)
This PR decreases the lead subscription time which should help
decrease bandwidth usage and CPU making the subscription for future
aggregation happen a bit later. There's room for more tuning here,
probably.

* fix missing negation from in #2550
* fix silly bitarray issues
* decrease subnet lead subscription time
* log all subnet switching source data
* rename subnet trackers to refer to stability and aggregate subnets
* more tests
2021-05-11 22:03:40 +02:00
Mamy Ratsimbazafy 149ff49c8e
Remove correlated queries in finalization pruning, all use indexes (#2554) 2021-05-11 10:41:37 +02:00
Mamy Ratsimbazafy e6b559a35a
Slashing db pruning [Merge only after v2 has been default for 1 noticeable release] (#2452)
* Enable slashing DB pruning

* integrate slashing DB pruning with onSlotEnd

* rebase tests
2021-05-10 16:32:28 +02:00
tersec f9b964ca5d
bump nim-eth2-scenarios for merge test vectors (#2553) 2021-05-10 13:56:31 +00:00
Jacek Sieka 867d8f3223
Perform attestation check before broadcast (#2550)
Currently, we have a bit of a convoluted flow where when sending
attestations, we start broadcasting them over gossip then pass them to
the attestation validation to include them in the local attestation pool
- it should be the other way around: we should be checking attestations
_before_ gossipping them - this serves as an additional safety net to
ensure that we don't publish junk - this becomes more important when
publishing attestations from the API.

Also, the REST API was performing its own validation meaning
attestations coming from REST would be validated twice - finally, the
JSON RPC wasn't pre-validating and would happily broadcast invalid
attestations.

* Unified attestation production pipeline with the same flow for gossip,
locally and API-produced attestations: all are now validated and entered
into the pool, then broadcast/republished
* Refactor subnet handling with specific SubnetId alias, streamlining
where subnets are computed, avoiding the need to pass around the number
of active validators
* Move some of the subnet handling code to eth2_network
* Use BitArray throughout for subnet handling
2021-05-10 09:13:36 +02:00
tersec 39da640beb
use getStateField() in ncli_db (#2549) 2021-05-07 13:14:20 +00:00
Jacek Sieka 646923c3dd
add attestation stats tool to ncli_db (#2539)
This also makes future efforts to provide metrics and logs for
attestation efficiency easier

* Export rewards from epoch transition
* Use less memory for reward calculation (bool -> set[enum], field
alignment)
* Reuse reward memory when replaying, avoiding spike
* Allow replaying any range in ncli_db benchmark
2021-05-07 13:36:21 +02:00
Jacek Sieka 5cd5da74c4
Gossipsub unsubscribe fixes (fixes #2540) (#2545) 2021-05-07 06:31:28 +02:00
Jacek Sieka 01b404f43d
bump chronos (#2544) 2021-05-06 18:20:37 +02:00
tersec 1d6c8ee9ab
store full state 4x less often (#2542) 2021-05-06 07:36:18 +02:00
tersec dd43a2c3b0
bump nim-eth2-scenarios to get merge SSZ test vectors (#2541) 2021-05-05 15:35:36 +00:00
Ștefan Talpalaru 2018a55c51
bump a few submodules (#2529) 2021-05-05 08:55:39 +02:00
Jacek Sieka 427c0f307c
avoid extraneous hash root calculation (#2537)
When applying a block, we'll currently compute a state root for the
state after slot processing but before block processing - this is
unnecessary when a block is being applied because the intermediate state
root is never observed.
2021-05-05 08:54:21 +02:00
Jacek Sieka 4d74c742da
move ENRForkID into `spec` (#2538)
* move ENRForkID into `spec`

also get rid of strformat in topic formation and fix some case
discrepancies

* also move `Eth2Metadata`
2021-05-04 17:28:48 +02:00
Jacek Sieka efdf759cc0
avoid some slashing protection queries (#2528)
This PR reduces the number of database queries for slashing protection
from 5 reads and 1 write to 2 reads and 1 write in the optimistic case.

In the process, it removes user-level support for writing the database
in the version 1 format in order to simplify the code flow, and prevent
code rot. In particular, the v1 format was not covered by any unit tests
and has no advantages over v2. The concrete code to read and write it
remains for now, in particular to support upgrades from v1 to v2.

The branch also removes the use of concepts which doesn't work with
checked exceptions - in particular, this highlights code that both
raises exceptions and returns error codes, which could be cleaned up in
the future.

* Cache internal validator ID
* Rely on unique index to check for trivial duplicate votes
* Combine two surround vote queries into one
* Combine API for checking and registering slashing into single function

The slashing DB is normally not a bottleneck, but may become one with
high attached validator counts.
2021-05-04 15:17:28 +02:00
tersec 290b889ce6
non-intrusive, novel portions of merge (#2535) 2021-05-04 11:54:19 +00:00
tersec e0f4d28116
rename initialize_beacon_state to initialize_beacon_state_from_eth1 (#2536) 2021-05-04 12:19:11 +02:00
Ștefan Talpalaru 51fde853d5
Merge branch 'stable' into unstable 2021-05-03 15:26:56 +02:00
Zahary Karadjov f369682b3f
v1.2.2 2021-05-03 11:13:12 +03:00
Chris Hobcroft a3b9990beb
Fixed confusing typo (#2533)
"One or more" appears to be a better description than "One of more"
2021-05-02 17:01:49 +02:00
Zahary Karadjov faebe3470c Don't execute CI builds (Github actions) when no build inputs are changed 2021-05-01 10:24:46 +03:00
Zahary Karadjov e60dd8e4f6 Stronger validation for third-party Eth1Data votes 2021-05-01 10:24:31 +03:00
Jacek Sieka 9d031a8446
Update README links 2021-04-29 10:46:09 +02:00
Jacek Sieka b899878a23
Update README.md
Fix links, clarify scope
2021-04-29 10:43:32 +02:00
Ștefan Talpalaru bac1a64bf6
Jenkins: disable tools/tests parallelism (#2526)
New REST test depends on a tool and, if it's missing, it builds it by
itself, thus interacting dangerously with a parallel "Tools" stage.

We give up on that parallelism because it's unlikely to provide relevant
gains, now that `make test` starts by building all test binaries.
2021-04-29 03:28:11 +02:00
Jacek Sieka ce49da6c0a
Introduce unittest2 and junit reports (#2522)
* Introduce unittest2 and junit reports

* fix XML path

* don't combine multiple CI runs

* fixup

* public combined report also

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 18:41:02 +02:00
yslcrypto 4cdbc17e39 remove references to devel 2021-04-28 16:06:00 +02:00
yslcrypto ec1aa622b0 Merge branch 'unstable' of github.com:status-im/nim-beacon-chain into unstable 2021-04-28 16:00:02 +02:00
0xmiel 7af4f7c665
Guide updates (#2525)
* update nimbus book readme

* Revert "update nimbus book readme"

This reverts commit d568f0f7a3.

* update binaries page

* rm extra emacs generated files

* remove extra emacs generated file

* fix windows typ

* fix eth1 title

* page on adding additional validator
2021-04-28 15:55:43 +02:00
yslcrypto a42b3e1f9c Merge branch 'unstable' of github.com:status-im/nim-beacon-chain into unstable 2021-04-28 13:39:46 +02:00
nbc-bump-bot[bot] d4cda02c97
auto-bump nim-libp2p (#2523)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-04-28 09:57:25 +02:00
Dustin Brody 7f42d38219 rename initialize_beacon_state{_from_eth1,}; suppress warnings when doppelganger detection disabled 2021-04-28 00:12:41 +03:00
cheatfate 18da0f8928 Bump nim-chronos to fix issues in httpserver.
Bump nim-http-utils to fix issues in httpserver.
2021-04-28 00:00:00 +03:00