Commit Graph

1004 Commits

Author SHA1 Message Date
Eugene Kabanov 4fb95d000d
REST server fixes and improvements. (#5422)
* Move from Option[T] to Opt[T] usage.

* Add `finalized` flag.

* Fix compilation issue.

* Http415 error code for some REST API calls.
Introduce more comprehensive error reporting for block calls.
Deprecate decodeEthConsensusVersion() function.

* Bump http-utils.

* Fix copyright year.

* Fix serialization issue.

* Address review comments.

* Post rebase fixes.
2023-09-27 16:45:33 +02:00
Jacek Sieka 96e05e4f37
stint: bump (#5459)
fixes 32-bit compilation
2023-09-25 21:08:48 +02:00
Etan Kissling 5150505e66
bump `nim-ssz-serialization` to `a5767c1b4992dca3e5bd72557aadd4df0f183bf8` (#5457)
- nimble/ci updates
- use common super-type in `raisesssz` pragma
2023-09-22 23:44:57 +02:00
Etan Kissling 273f1d34cd
implement EIP-7514 for Deneb: Add Max Epoch Churn Limit (#5442)
* implement EIP-7514 for Deneb: Add Max Epoch Churn Limit

Cap activations per epoch according to EIP-7514:

- https://eips.ethereum.org/EIPS/eip-7514
- https://github.com/ethereum/consensus-specs/pull/3499

* apply proposer boost to first block in case of equivocation

Implement spec changes to fork choice; this only affects equivocation
when multiple blocks are signed for the same slot. Regular operation
is not changed.

- https://github.com/ethereum/consensus-specs/pull/3352

* bump test vectors to v1.4.0-beta.2-hotfix

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-09-20 13:39:56 +02:00
Etan Kissling eadd0d69ce
bump `nim-kzg4844` to `6c406c70ad8cde6ab56320158dd969beb45322ef` (#5448)
- bump `csources` to `v0.3.1`
2023-09-19 23:29:50 +00:00
Eugene Kabanov 521e7f535c
Bump chronos & presto. (#5394)
* Bump chronos & presto.

* Increase finalization timeouts for 10 minutes.

* Return Jenkins timeouts back to normal.
Bump chronos.

* Bump presto.
2023-09-19 08:01:58 +02:00
Etan Kissling 63440766ef
exclude 'public-keys/all.txt' file from checkout (#5439)
The 'vendor/holesky/public-keys/all.txt' file is not used by Nimbus,
but sometimes fails to download due to payment issues in `@eth-clients`.
Skipping the download of that file should reduce that risk.

```
Downloading public-keys/all.txt (240 MB)
Error downloading object: public-keys/all.txt (ba977b5): Smudge error: Error downloading public-keys/all.txt (ba977b5b1da180914c934623fce187860e1b54cff939e6208533b2cb5f589e07): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
```

* bump nbs: Patch libnatpmp.a rule to support llvm-mingw in targets.mk

* bump nbs: Allow passing config to submodule initialization in `update-common`
2023-09-18 20:24:13 +02:00
Tanguy 4918a4e2e0
Fix direct peers (#5427)
* Fix direct peers

* Support ENRs in DP, use DP in local testnet

* fix docs

* bump libp2p
2023-09-15 18:45:55 +00:00
Eugene Kabanov aec953e4da
VC: Fix VC must not crash if beacon node address could not be resolved. (#5388)
* Fix VC should not crash, if beacon node URL could not be resolved.

* Bump chronos.

* Update .gitmodules.
2023-09-15 07:34:32 +02:00
andri lim 5c88e74c08
Bump stint to v2.0: new array backend (#5113)
* bump stint to v2.0: new array backend

* Fix missing isEven in libnimbus_lc.nim

* bump nim-stint: compiles with arm64
2023-09-13 01:16:04 +07:00
Zahary Karadjov 6b1ab81046
Support both snappy-encoded and pure SSZ genesis states 2023-09-12 01:52:15 +03:00
Jacek Sieka 4d8e972e70
snappy: bump (#5417) 2023-09-11 12:49:54 +02:00
zah 53589b5a7d
Add metadata for the Holesky network (#5337)
* Add metadata for the Holesky network

* Add copyright banner to the new Nim module

* Working version

* Bump Chronos to fix downloading from Github
* Add checksum check of the downloaded file
* Clean up debugging code and obsolete imports
2023-09-08 08:53:27 +03:00
Tanguy c5a79a6f8f
Add IDontWant metrics (#5405)
* Add IDontWant metrics

* update .gitmodules
2023-09-07 15:13:04 +02:00
Etan Kissling 99507e19cd
bump `nim-ssz-serialization` to `53428359a6759ac34bb4b16acb234f8477153b8f` (#5398)
- convert func to iterator
- Fix compiler complaining type mismatch for byte array
- workaround for Nim 2.0 issue with the template
2023-09-06 08:51:19 +00:00
Jacek Sieka 6fab826487
bump unittest2, clean up suite names (#5384)
Suite names were not being used because `test` has to have access to it
during instantiation - this PR cleans things up a little while at the
same time upgrading unittest2.
2023-09-04 13:05:11 +00:00
tersec 8725595328
bump nim-ssz-serialization (#5378) 2023-09-01 11:08:52 +00:00
Jacek Sieka 6f59c6c46d
nimcrypto: bump (#5376)
aligns `MDigest`
2023-08-31 13:30:59 +00:00
Etan Kissling 1fbf371826
bump `nim-web3` to `70f22351c44b849a321b78b94dc0f9cd4ac9a5e1` (#5373)
- export `WithdrawalObject`
2023-08-30 23:24:10 +00:00
Etan Kissling 060e89a07d
update `nim-web3` to `91113bef0a3fef277782bffa83d143214bd680f4` (#5359)
- update `ReceiptObject` and `LogObject`
2023-08-26 00:32:49 +00:00
Etan Kissling 2172e248aa
bump `nim-web3` to `ab3dfabea24dda4771afcb9c0df1607c0eb75602` (#5351)
- add `{.raises.}` annotation to `writeValue`
- add EIP-4844 fields to `TransactionObject`
2023-08-25 00:16:15 +00:00
Tanguy 36413c8f21
Bump libp2p for gossipsub improvements (#5229)
* Bump libp2p

* Add bandwidthEstimate param

* Fix typo and type

* Add config for light client

* Fix unit

* Add debug prefix

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
Co-authored-by: diegomrsantos <diegomrsantos@gmail.com>
Co-authored-by: Diego <diego@status.im>
2023-08-24 16:04:19 +02:00
Jacek Sieka 0043a07c6f
NimYAML: bump to 1.1.0 (#5346)
We were using a status-based fork with additional Nim 1.2 support -
should not needed anymore so this PR bumps it to a clean upstream
version
2023-08-24 16:01:34 +02:00
Etan Kissling d28facf4c4
bump `nim-json-rpc` to `60c4c9b5f28b530d5d89fd14c337af1d86390a82` (#5341)
- fix router example
- fix ws server param type
- improve RPC client logging
2023-08-23 19:36:32 +03:00
Etan Kissling cf3fd7d811
bump `nim-confutils` to `674c9e4c8e0cad2b7193cc9a59c12d39a397750f` (#5330)
- explicitly initialize `result`
2023-08-20 12:46:45 +02:00
Etan Kissling 474d9a3f20
bump `nim-confutils` to `d784b823be28532aadd77af5c983b6ba18cf33d8` (#5329)
- fail on error while processing `secondarySources`
- add `{.raises.}` annotation to `writeValue`
2023-08-20 10:10:15 +00:00
Etan Kissling b4b40fa7d2
bump `nim-chronicles` / `nim-json-serialization` (#5319)
* bump `nim-chronicles` to `71556f84af16cd275a8ae4c2390060481c6fedf7`

- remove Nim 1.2 and 1.4 support; test Nim 2.0 in CI
- suppress some more `IOError` when logging

* bump `nim-json-serialization` to `85b7ea093cb85ee4f433a617b97571bd709d30df`

- add `readValue` implementations for case objects
- add `{.raises.}` annotation to `writeValue`
2023-08-20 02:24:03 +00:00
Etan Kissling a2f85e4b67
bump `nim-ssz-serialization` to `947f39fd34f3e73dc5b9465600e84ab98e848a56` (#5327)
- remove SingleMemberUnion read/write support
- note upstream Nim tracking bug
- remove rest of SingleMemberUnion
- remove spurious `{.raises.}`
2023-08-20 01:26:13 +02:00
Etan Kissling ad60ba027d
bump `nim-eth` to `894ec07d9f977a5ef9487630b71764df9163982f` (#5325)
- fix `{.raises.}` annotation to `writeValue`
2023-08-19 20:34:29 +00:00
Etan Kissling 82037fab54
bump `nim-serialization` to `4bdbc29e54fe54049950e352bb969aab97173b35` (#5322)
- add `{.raises.}` annotation to `writeValue`
2023-08-19 14:17:27 +00:00
tersec 6fd4983f6b
EIP-4788/parentBeaconBlockRoot fixes (#5318) 2023-08-19 08:38:17 +00:00
Jacek Sieka 5b879e3bc2
blscurve: bump for cleanups (#5311) 2023-08-18 01:01:03 +00:00
Jacek Sieka a150bc93a6
blst: 0.3.11 (#5300)
```
Essential changes:

    security bugfix in Go bindings
    run-time switch in portable build
    no-std support [exercised with limited stack size of 56K]
    serde support
blst_miller_loop_n interface, with parallelized Rust and Go
counterparts
    optional blst_fr_pentaroot subroutine
    Emscripten bindings
    hardened security on non-asm platforms
```
2023-08-16 18:13:58 +00:00
Jacek Sieka ddddfc41aa
bearssl: bump to 0.2.1 (#5298) 2023-08-16 15:42:01 +00:00
Jacek Sieka b39e71335d
ssz: bump for 10% hash_tree_root perf increase, json loading fix (#5292)
* move file loading out of timed ncli section to focus benchmark on
deserialization rather than disk loading
* JSON loading fix included in bump, but does not practically happen
since we generally don't load hashlist from JSON
2023-08-15 13:28:33 +00:00
Eugene Kabanov 255131d331
Bump chronos. (#5281)
Add current/last connection query to the connection information.
2023-08-15 13:14:34 +02:00
Jacek Sieka 3c9fc49411
reduce ncli/research imports (#5275)
* reduce ncli/research imports

avoids ssl, most of chronos etc, speeds up compiles

* copyright
2023-08-10 14:52:49 +02:00
Jacek Sieka a154a42205
bump ssz (small speedup) (#5282) 2023-08-09 19:40:57 +00:00
tersec 8e398001fe
bump nimcrypto to f118a5f86b04fbfd388b584d749f93bb3795f88f (#5279) 2023-08-09 06:40:03 +00:00
Etan Kissling 2b57a221de
bump `nim-blscurve` to `337ab727eb8a38a0a71725f6f01ce55563f355c0` (#5274)
- bump blst
- add sig aggregation bench
- allow subtracting from aggregate for when participation is high
- add `{.push: raises: [].}` annotation
2023-08-08 12:20:08 +00:00
andri lim e194d7b4ab
bump ssz-serialization to 4edfffe2b8d338444ef0b754954fba3b9fe78b9d (#5266) 2023-08-07 07:45:34 +00:00
Etan Kissling f0774564a8
bump `nim-web3` to `04f56c593a035af1b7bebdc726543a2a73826412` (#5241)
- fix `TransactionObject` type up through mainnet
- fix `Quantity` parsing on 32-bit platform
2023-08-03 17:18:57 +00:00
Jacek Sieka b8a32419b8
async batch verification (+40% sig verification throughput) (#5176)
* async batch verification

When batch verification is done, the main thread is blocked reducing
concurrency.

With this PR, the new thread signalling primitive in chronos is used to
offload the full batch verification process to a separate thread
allowing the main threads to continue async operations while the other
threads verify signatures.

Similar to previous behavior, the number of ongoing batch verifications
is capped to prevent runaway resource usage.

In addition to the asynchronous processing, 3 addition changes help
drive throughput:

* A loop is used for batch accumulation: this prevents a stampede of
small batches in eager mode where both the eager and the scheduled batch
runner would pick batches off the queue, prematurely picking "fresh"
batches off the queue
* An additional small wait is introduced for small batches - this helps
create slightly larger batches which make better used of the increased
concurrency
* Up to 2 batches are scheduled to the threadpool during high pressure,
reducing startup latency for the threads

Together, these changes increase attestation verification throughput
under load up to 30%.

* fixup

* Update submodules

* fix blst build issues (and a PIC warning)

* bump

---------

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2023-08-03 11:36:45 +03:00
tersec 0a4036baee
bump nim-eth to d217d309eb12e39d2a642feae1d9d27157b88352 (#5237) 2023-08-03 02:13:36 +00:00
henridf 28194468c9
Rename "data gas" to "blob gas" (#5216)
* Rename data gas to blob gas

* Update vendor/nim-eth and vendor/nim-web3
2023-08-02 22:07:57 +00:00
Etan Kissling 32e8bfe911
bump `nim-confutils` to `dbe8d61f7fbb1cb9d74b38f9012406c8071aa9e9` (#5228)
- return empty completion lines on unsupported `COMP_POINT`
- require `secondarySources` to be `gcsafe`
2023-08-01 01:23:55 +00:00
Etan Kissling 66160d986f
bump `gnosis-chain-configs` to `b90374a1c63703db8235fcdb65aff2e909bc42b5` (#5227)
- fix: bootstrap_nodes.txt
2023-08-01 01:23:46 +00:00
tersec 909a7c2d07
use v1.4.0-beta.1 consensus spec tests (#5230) 2023-07-31 21:51:06 +00:00
Jacek Sieka e8379389e7
speed up state/block loading (#5207)
* speed up state/block loading

When loading blocks and states from db/era, we currently redundantly
check their CRC32 - for a state, this costs 50ms of loading time
presently (110mb uncompressed size) on a decent laptop.

* remove `maxDecompressedDbRecordSize` - not actually used on recent
data since we store the framed format - also, we're in luck: we blew
past the limit quite some time ago
* fix obsolete exception-based error checking
* avoid `zeroMem` when reading from era store

see https://github.com/status-im/nim-snappy/pull/22 for benchmarks

* bump snappy
2023-07-26 10:47:46 +03:00
Eugene Kabanov df80ae68fa
REST server metrics and connections API endpoints. (#5193)
* Add new REST endpoints to monitor REST server connections and new chronos metrics.

* Bump head versions of chronos and presto.

* Bump chronos with regression fix.

* Remove outdated tests which was supposed to test pipeline mode.

* Disable pipeline mode in resttest.

* Update copyright year.

* Upgrade test_signing_node to start use AsyncProcess instead of std library's osproc.
Bump chronos to check graceful shutdown.

* Update AllTests.

* Bump chronos.
2023-07-21 16:54:09 +02:00