Commit Graph

18 Commits

Author SHA1 Message Date
Jacek Sieka 8723a79225
add era dir to make_states 2024-08-12 14:49:32 +02:00
Jacek Sieka 3d3831dde8
Small cleanups (#2435)
* avoid costly hike memory allocations for operations that don't need to
re-traverse it
* avoid unnecessary state checks (which might trigger unwanted state
root computations)
* disable optimize-for-hits due to the MPT no longer being complete at
all times
2024-07-01 14:07:39 +02:00
Jacek Sieka 55ebd70d1e
stats: interpolate, remove some broken stats 2024-06-29 06:36:35 +02:00
tersec 2aaab1cb4a
fix Dependabot alerts (#2375) 2024-06-17 15:30:43 +02:00
Jacek Sieka 189a20bbae
Avoid recomputing hashes when persisting data (#2350) 2024-06-14 07:10:00 +02:00
Jacek Sieka eb041abba7
avoid unnecessary memory allocations and lookups (#2334)
* use `withValue` instead of `hasKey` + `[]`
* avoid `@` et al
* parse database data inside `onData` instead of making seq then parsing
2024-06-11 11:38:58 +02:00
web3-developer db8c5b90bd
Cleanup stateless and block witness code. (#2295)
* Cleanup unneeded stateless and block witness code. Keeping MultiKeys which is used in the eth_getProofsByBlockNumber RPC endpoint which is needed for the Fluffy state network bridge.

* Rename generateWitness flag to collectWitnessData to better describe what the flag does. We only collect the keys of the touched accounts and storage slots but no block witness generation is supported for now.

* Move remaining stateless code into nimbus directory.

* Add vmstate parameter to ChainRef to fix test.

* Exclude *.in from check copyright year

---------

Co-authored-by: jangko <jangko128@gmail.com>
2024-06-08 15:05:00 +07:00
Jacek Sieka 32c51b14a4
keccak: improve perf a little (#2321)
* avoid `burnMem`
* avoid zeroing buffers
* work around `when nimvm` issue
2024-06-07 16:48:27 +00:00
Jacek Sieka ce80ed79a5
fix display in early blocks and time avg 2024-06-07 12:29:34 +02:00
Jacek Sieka c5b3081828
eth: bump (#2308)
* eth: bump

Speed up basic operations like hashing and creating RLP:s - up to 25%
improvement in certain block ranges!

```
876729c.csv /data/nimbus_stats/stats-20240605_2204-ed4f6221.csv
stats-20240605_2000-c876729c.csv vs stats-20240605_2204-ed4f6221.csv
                       bps_x   bps_y     tps_x        tps_y    bpsd    tpsd    timed
block_number
(500001, 888889]    1,017.72  996.07  1,784.96  1742.438676  -2.72%  -2.72%    3.31%
(888889, 1277778]     528.00  536.30  2,159.79  2198.781046   1.69%   1.69%   -1.44%
(1277778, 1666667]    324.29  317.78  2,064.48  2008.106377  -2.82%  -2.82%    3.33%
(1666667, 2055556]    253.87  258.74  1,840.94  1872.935273   1.67%   1.67%   -1.39%
(2055556, 2444445]    175.79  178.66  1,340.61  1363.248939   0.93%   0.93%   -0.74%
(2444445, 2833334]    137.27  159.74    958.75  1113.323757  14.24%  14.24%  -10.69%
(2833334, 3222223]    170.48  228.63  1,272.70  1704.047195  34.41%  34.41%  -25.17%
(3222223, 3611112]    127.49  125.48  1,572.39  1548.835791  -1.19%  -1.19%    1.47%
(3611112, 4000001]     37.25   40.42  1,100.65  1184.740493   9.58%   9.58%   -7.04%

blocks: 3501696, baseline: 11h59m40s, contender: 11h21m38s
bpsd (mean): 6.18%
tpsd (mean): 6.18%
Time (sum): -38m1s, -4.26%

bpsd = blocks per sec diff (+), tpsd = txs per sec diff, timed = time to process diff (-)
+ = more is better, - = less is better
```

* ignore gitignore
2024-06-06 23:39:09 +00:00
Jacek Sieka e9f2608cd0
no cr for scripts 2024-06-06 14:38:58 +02:00
Jacek Sieka 0c6c84f2ce
Script for comparing csv outputs from block import 2024-06-06 14:33:49 +02:00
Jacek Sieka a375720c16
import: read from era files (#2254)
This PR extends the `nimbus import` command to also allow reading from
era files - this command allows creating or topping up an existing
database with data coming from era files instead of network sync.

* add `--era1-dir` and `--max-blocks` options to command line
* make `persistBlocks` report basic stats like transactions and gas
* improve error reporting in several API
* allow importing multiple RLP files in one go
* clean up logging options to match nimbus-eth2
* make sure database is closed properly on shutdown
2024-05-31 09:13:56 +02:00
Jordan Hrycaj de0388919f
Unified mode for undumping gzip-ed or era1-ed encoded block dumps (#2198)
ackn:
  Built on Daniel's work
2024-05-20 13:59:18 +00:00
Kim De Mey d3a706c229
Replace status-im/portal-spec-tests with ethereum fork version (#2097)
- The fluffy test vector repo got forked (well, copied rather) to
become the official one under ethereum github org, so we change
to that repo now and archive ours.
- Our repo also stored accumulator / historical_roots, replace
that with a new repo which is only for network configs.
- Several changes needed to be made due to test vectors that got
updated + some of them got changed to / are yaml format instead of
json.
2024-03-22 11:28:44 +01:00
jangko b0000eed8b
Add check copyright year linter to CI 2023-11-01 10:41:20 +07:00
jangko 2f6b4de3e9 ci: fix nightly build 2023-02-23 18:34:04 +07:00
Ștefan Talpalaru 51bc1cf87f
dist: precompiled binaries and Docker images (#1015)
* dist: precompiled binaries and Docker images

The builds are reproducible, the binaries are portable and statically link librocksdb.

This took some patching. Upstream PR: https://github.com/facebook/rocksdb/pull/9752

32-bit ARM is missing as a target because two different GCC versions
fail with an ICE when trying to cross-compile RocksDB. Using Clang
instead is too much trouble for a platform that nobody should be using
anyway.

(Clang doesn't come with its own target headers and libraries, can't be
easily convinced to use the ones from GCC, so it needs an fs image from
a 32-bit ARM distro - at which point I stopped caring).

* CI: disable reproducibility test
2022-03-27 13:21:15 +02:00