2
0
mirror of https://github.com/status-im/nimbus-eth2.git synced 2025-01-12 07:14:20 +00:00

678 Commits

Author SHA1 Message Date
Jacek Sieka
d583e8e4ac
Store finalized block roots in database (3s startup) ()
* Store finalized block roots in database (3s startup)

When the chain has finalized a checkpoint, the history from that point
onwards becomes linear - this is exploited in `.era` files to allow
constant-time by-slot lookups.

In the database, we can do the same by storing finalized block roots in
a simple sparse table indexed by slot, bringing the two representations
closer to each other in terms of conceptual layout and performance.

Doing so has a number of interesting effects:

* mainnet startup time is improved 3-5x (3s on my laptop)
* the _first_ startup might take slightly longer as the new index is
being built - ~10s on the same laptop
* we no longer rely on the beacon block summaries to load the full dag -
this is a lot faster because we no longer have to look up each block by
parent root
* a collateral benefit is that we no longer need to load the full
summaries table into memory - we get the RSS benefits of  without
the CPU hit.

Other random stuff:

* simplify forky block generics
* fix withManyWrites multiple evaluation
* fix validator key cache not being updated properly in chaindag
read-only mode
* drop pre-altair summaries from `kvstore`
* recreate missing summaries from altair+ blocks as well (in case
database has lost some to an involuntary restart)
* print database startup timings in chaindag load log
* avoid allocating superfluos state at startup
* use a recursive sql query to load the summaries of the unfinalized
blocks
2022-01-30 18:51:04 +02:00
Emil
0051af430b Put application/json as a higher preference than application/octet-stream 2022-01-30 18:50:14 +02:00
tersec
60bf5b8bf4
use v1.1.9 test vectors () 2022-01-28 22:47:48 +00:00
Ștefan Talpalaru
fdb76b8e0b
bump nimbus-build-system () 2022-01-26 17:11:28 +01:00
Ștefan Talpalaru
47a6f9950f
bump nimbus-build-system () 2022-01-26 14:15:53 +01:00
Mamy Ratsimbazafy
9e9ccf4a1f
Slashing prot interchange tests v5.2.1 ()
* initial support for minification and new interchange tests. Removal of v1 and v1 migration.

* Synthetic attestations: SQLite3 requires one statement/query per prepared statement

* Fix DB import interrupted if no attestation was found

* Skip test relying on undocumented test behavior (https://github.com/eth-clients/slashing-protection-interchange-tests/pull/12#issuecomment-1011158701)

* Skip test relying on unclear minification behavior:
creating an invalid minified attestation with source > target or setting target = max(source, target)

* remove DB v1 and update submodule

* Apply suggestions from code review

Co-authored-by: Jacek Sieka <jacek@status.im>

Co-authored-by: Jacek Sieka <jacek@status.im>
2022-01-20 17:14:06 +01:00
Jacek Sieka
836f6984bb
move state_transition to Result ()
* better error messages in api
* avoid `BlockData` copies when replaying blocks
2022-01-17 12:19:58 +01:00
Zahary Karadjov
af614b6a85 bump nim-json-rpc 2022-01-16 18:26:21 +02:00
tersec
142e0ea163
bump nim-web3 to use enums for engine API types () 2022-01-14 14:11:42 +01:00
Jacek Sieka
7e1cdce8fc
stew: bump ()
result fixes
2022-01-12 07:09:23 +00:00
Jacek Sieka
aab302aa9c
bumps ()
result + serialization cleanups
2022-01-11 09:35:20 +00:00
Jacek Sieka
6f7e0e3393
REST cleanups ()
* REST cleanups

* reject out-of-range committee requests
* print all hex values as lower-case
* allow requesting state information by head state root
* turn `DomainType` into array (follow spec)
* `uint_to_bytesXX` -> `uint_to_bytes` (follow spec)
* fix wrong dependent root in `/eth/v1/validator/duties/proposer/`
* update documentation - `--subscribe-all-subnets` is no longer needed
when using the REST interface with validator clients
* more fixes
* common helpers for dependent block
* remove test rules obsoleted by more strict epoch tests
* fix trailing commas

* Update docs/the_nimbus_book/src/rest-api.md
* Update docs/the_nimbus_book/src/rest-api.md

Co-authored-by: sacha <sacha@status.im>
2022-01-08 22:06:34 +02:00
Tanguy
7a0119ac32
Bump libp2p () 2022-01-05 17:33:23 +01:00
tersec
cd77377375
add Bellatrix fork and transition tests; "Ethereum Foundation" -> EF () 2022-01-05 09:42:56 +01:00
Zahary Karadjov
54d0d588b1 Implementation of the Keymanager API (BETA)
https://github.com/ethereum/keymanager-APIs
2022-01-04 18:51:45 +02:00
tersec
3c63a78c01
use v1.1.8 test vectors () 2022-01-03 17:43:00 +00:00
Jacek Sieka
61b6fc1016
3x speedup in snappy compression ()
* 3x speedup in snappy compression

oh, the wonders of `copyMem` in `endians2` - speeds up all kinds of
operations like database stores, sending gossip etc.

* endian usage fixes
2022-01-03 18:17:10 +01:00
tersec
8be1699014
use v1.1.7 test vectors ()
* use v1.1.7 test vectors
2022-01-03 13:06:14 +00:00
Mamy Ratsimbazafy
873b0f1128
Bump BLST, improve pubkey deserialization speed by 20% to 2x faster. improve signature deserialization by 5% to 4x faster. ()
Nice!
2022-01-02 16:48:14 +01:00
Jacek Sieka
6b60a774e0
Lazy aggregated batch verification ()
A novel optimisation for attestation and sync committee message
validation: when batching, we look for signatures of the same message
and aggregate these before batch-validating: this results in up to 60%
fewer signature verifications on a busy server, leading to a significant
reduction in CPU usage.

* increase batch size slightly which helps finding more aggregates
* add metrics for batch verification efficiency
* use simple `blsVerify` when there is only one signature to verify in
the batch, avoiding the RNG
2021-12-29 15:28:40 +01:00
Tanguy
4a72def1d5
Bump libp2p () 2021-12-17 12:39:24 +01:00
Ștefan Talpalaru
9311d0a887
Nim-1.2.16 () 2021-12-17 00:52:49 +01:00
Etan Kissling
dc1562665f
bump nim-ssz-serialization to 3cd8d2d6b80bde0ce7f25609cb5cb9fc37852fe2 ()
This updates `nim-ssz-serialization` to
`3cd8d2d6b80bde0ce7f25609cb5cb9fc37852fe2`.

Notable changes:
- Serialization of object variant (case object) to/from SSZ Union.
- int -> int64 fix in hashTreeRootCached
2021-12-16 17:26:54 +00:00
Etan Kissling
f68aa9218f
bump nim-confutils to 6a56d01381f434d5fbcc61b6e497b9409155bcbc ()
This updates `nim-confutils` to
`6a56d01381f434d5fbcc61b6e497b9409155bcbc`.

Notable changes:
- feature: separator text when displaying help
- feature: multiple lines long description
- feature: add ignore property in addition to hidden
- add compile time check to detect duplicate abbr and duplicate name
2021-12-15 09:27:04 +00:00
Etan Kissling
6f75262a03
bump nim-snappy to 16cce7d07ce8be2cfc760d86744b8b54cd0808a9 ()
This updates `nim-snappy` to `16cce7d07ce8be2cfc760d86744b8b54cd0808a9`.

Notable changes:
- avoid unnecessary compression of short payloads
2021-12-14 18:50:24 +00:00
Etan Kissling
d023613045
bump nim-eth to 5655bd035cfd7319c6f2e60b7fdefef65a057939 ()
This updates `nim-eth` to `5655bd035cfd7319c6f2e60b7fdefef65a057939`.

Notable changes:
- db: Allow Sqlite keystores to be used in read-only mode
- net: avoid allocation in hash(ValidIpAddress)
- net: Remove hashData usage on objects
- p2p: reject WHOAREYOU packets with non-empty message
- p2p: Adjust logging when node is not reachable but enrAutoUpdate is on
- p2p: Allow a node to self resolve
- p2p: Fix logDistance for BE arch and remove toBytes for NodeId
- p2p: Export discovery routing table and its buckets nodes
- ssz: remove outdated and incorrect SSZ code
- utp: Various updates and fixes
2021-12-14 17:27:46 +00:00
tersec
f88bdc5d14
bump nim-web3 to properly decode TypedTransactions () 2021-12-13 15:50:18 +00:00
Tanguy
4be7e800f2
Bump chronos (Improve ram usage) () 2021-12-10 13:25:43 +01:00
Jacek Sieka
6f077ce82c
chronos: fast path for writes (and a bunch of other changes) () 2021-12-08 14:47:04 +01:00
tersec
aeb90a1585
bump nim-web3 to adjust engine API field names for alpha.5 () 2021-12-07 14:30:16 +00:00
tersec
8d7df05f2e
Revert "Bump chronos and presto. ()" ()
This reverts commit 4c90b82d9f211a8aad374f63c24472a47edf52c4.
2021-12-04 15:52:28 +00:00
Eugene Kabanov
4c90b82d9f
Bump chronos and presto. ()
* Add some indicators to help fixing issue.

* Bump presto to help debugging.

* Fix compilation problems in presto.

* Fix SIGSEGV.

* Bump latest changes in chronos and presto.
Fix rare cases in validator_client.

* Use proper commits for chronos and presto.
2021-12-04 14:26:16 +00:00
zah
74c63ed740
More efficient implementation of the 'POST beacon_committee_subscriptions' API () 2021-12-03 17:04:58 +02:00
nbc-bump-bot[bot]
fa2cf028e2
auto-bump nim-libp2p ()
Co-authored-by: = <Menduist@users.noreply.github.com>
2021-12-03 15:03:34 +01:00
tersec
61fb458f89
use v1.1.6 test vectors () 2021-12-01 12:55:42 +00:00
Ștefan Talpalaru
5e50297f95
bump nimbus-build-system () 2021-12-01 05:31:22 +01:00
Zahary Karadjov
872b88db50
Hotfix: non-crashing version of nim-json-rpc/news 2021-11-30 02:51:35 +02:00
zah
bc6a6a8b8f
Switch back to the news package in nim-json-rpc () 2021-11-29 21:47:31 +01:00
Zahary Karadjov
88c623e250 Add support for HTTPS Web3 providers 2021-11-23 15:56:18 +02:00
Ștefan Talpalaru
2d986c5346
bump nimbus-build-system () 2021-11-23 02:43:04 +01:00
Ștefan Talpalaru
4639b04b2a
bump nimbus-build-system () 2021-11-22 21:21:13 +01:00
Tanguy
4bcdccab9a
Bump libp2p () 2021-11-18 10:57:27 +01:00
Eugene Kabanov
eda42f4a89
Bump nim-chronos. () 2021-11-15 18:34:07 +00:00
tersec
fde73b5a70
bump nim-web3 to switch payload ID from uint64 to 8-byte buffer () 2021-11-15 14:55:03 +00:00
Zahary Karadjov
5e62df5ad5 /bin/bash -> /usr/bin/env bash (needed on NixOS) 2021-11-15 15:45:01 +02:00
tersec
97ad5d49b0
kintsugi merge vector tests () 2021-11-10 13:41:02 +02:00
kdeme
03a70fbf36 Use nim-ssz-serialization module and rm local ssz code 2021-11-10 13:37:24 +02:00
Ștefan Talpalaru
782318d3fc
Nim-1.2.14 ()
* Nim-1.2.14

* bump submodules
2021-11-10 11:39:43 +01:00
Zahary Karadjov
66d60e47ae Bump Chronicles 2021-11-09 13:33:53 +02:00
tersec
b76326f4b8
use v1.1.5 consensus spec test vectors () 2021-11-09 06:46:24 +00:00