7d7e26d45f
Experimental bulk loader tests ( #1187 )
...
why:
Rocksdb bulk loading might provide a slight advantage when loading
larger data sets into the system
2022-08-12 16:42:07 +01:00
f0cd340163
Adhere to transport limits in seed methods ( #1186 )
...
* Add means to limit offered content to fit talkreq
* Add test for history network limits
* Change seed method api to return num of items offered
2022-08-09 14:32:41 +02:00
4f8d10f338
Bump nim-eth ( #1185 )
2022-08-06 10:11:36 +02:00
11908c78ec
engine api test: handle JsonRpcError for each client rpc call
2022-08-05 09:51:16 +07:00
5f0e89a41e
Snap accounts bulk import preparer ( #1183 )
...
* Provided common scheduler API, applied to `full` sync
* Use hexary trie as storage for proofs_db records
also:
+ Store metadata with account for keeping track of account state
+ add iterator over accounts
* Common scheduler API applied to `snap` sync
* Prepare for accounts bulk import
details:
+ Added some ad-hoc checks for proving accounts data received from the
snap/1 (will be replaced by proper database version when ready)
+ Added code that dumps some of the received snap/1 data into a file
(turned of by default, see `worker_desc.nim`)
2022-08-04 09:04:30 +01:00
71f9e37482
Add getting block by number through accumulator ( #1182 )
2022-08-04 08:34:53 +02:00
8117032c67
fix chain config parser regression
...
fixes #1180
2022-08-02 16:41:53 +07:00
21140191da
attempt to fix flaky simulator test
2022-08-02 11:48:59 +07:00
50af402f59
Add verification of block data by use of accumulator ( #1178 )
2022-08-01 21:00:21 +02:00
a5d4759bfd
enhance net-key command line option to accept random, hex, and path
...
- previously it only accept hex
- fix #587
2022-07-30 08:46:11 +07:00
a087d65542
reduce test suite time consumption
2022-07-30 08:43:15 +07:00
2c0a8fa26d
update to supported macOS in Github CI
2022-07-30 08:42:44 +07:00
a1253c67bd
Fix testnet tests ( #1176 )
...
* Testnet improvements
Increase timeout for reading
Add more logs
Offer endpoint can fail due to talkReq timeout, to avoid
test failure, retry it few times until success.
2022-07-29 14:24:07 +02:00
e3cabaff7f
Add bulk seeding to multiple peers ( #1170 )
...
* Add bulk seeding to multiple peers
2022-07-26 13:14:56 +02:00
eb972fd598
attemp to fix simulator ci
2022-07-24 15:47:06 +07:00
e3cf999033
fix .nimble dependency
2022-07-22 09:05:42 +07:00
73b628491d
Clique snapshots reorg ( #1169 )
...
* Add persistent snapshot size logging
why:
Suspecting too much space used
snapshot statistic:
[..]
blockNumber=2214912 nSnaps=2236 snapsTotal=1.14m
blockNumber=2215936 nSnaps=2237 snapsTotal=1.14m
[..]
Persisting blocks fromBlock=2216449 toBlock=2216640
36458496 datadir-nimbus-goerlish/data/nimbus/
* Replace legacy `lru_cache` by `keyed_queue`
why:
`keyed_queue` generalises `lru_cache`
snapshot statistic:
[..]
blockNumber=2234368 nSnaps=2259 snapsTotal=1.15m
blockNumber=2235392 nSnaps=2260 snapsTotal=1.15m
[..]
Persisting blocks fromBlock=2235649 toBlock=2235840
37627288 datadir-nimbus-goerlish/data/nimbus/
* Increase persistent snapshot storage interval by 300%
snapshot statistic:
[..]
blockNumber=2232320 nSnaps=620 snapsTotal=0.30m
blockNumber=2236416 nSnaps=621 snapsTotal=0.30m
[..]
Persisting blocks fromBlock=2237185 toBlock=2237376
37627288 datadir-nimbus-goerlish/data/nimbus/
* Cull legacy debugging environment for clique
why:
Chronicles provides a better choice (when properly set up)
2022-07-21 19:16:28 +01:00
5d98f68c09
Sync update to work with sepolia reorgs ( #1168 )
...
* Error return in `persistBlocks()` on initial `VmState` roblem
why:
previously threw an exception
* Updated sync mode option
why:
using enum rather than bool => space for more
* Added sync mode `full`, re-factued legacy sync
also:
rebased
* Fix typo (crashes `pesistBlocks()` otherwise)
also:
rebase to master
* Reduce log ticker noise by suppressing duplicate messages
* Clarify staged queue overflow handling
why:
backtrack/re-org mode in `stageItem()` should be detected by both,
the global indicator or the work item where it might have moved into.
also:
rebased
2022-07-21 13:14:41 +01:00
aa945f1ed9
Additional seed endpoints ( #1164 )
...
* Additional seed endpoints
2022-07-20 12:46:42 +02:00
4721fc7a54
add CORS support for HTTP local services(json-rpc, engine-api, graphql)
...
There is also a stub for websocket CORS handler. We still need to add
non immediate response management to websock.
2022-07-19 16:12:36 +07:00
a48f69a89a
hive: engine simulator add jwt auth test cases
2022-07-18 16:56:44 +07:00
76e438da2a
bump submodule: json-rpc and graphql
...
this is an update to support jwt auth and cors hook
2022-07-18 16:56:44 +07:00
e6938af437
apply jwt auth to rpcHttpServer and update jwt auth of rpcWebsocketServer
...
fixes #967
2022-07-18 16:56:44 +07:00
6cfaaf5b45
bump json-rpc, secp256k1, blscurve, and accompanying fixes
2022-07-12 20:01:02 +07:00
5abf05e455
Add validation for offered block bodies and receipts ( #1158 )
...
Also use a AsyncQueue now to handle offered data instead of the
network specific validation callbacks.
2022-07-11 16:29:16 +02:00
87ea6ffb92
Bump most of the submodules ( #1156 )
2022-07-08 20:28:31 +02:00
1a5e2e53dd
treat `persistBlocks()` exception as failure return code, unless `Defect` ( #1155 )
2022-07-06 16:00:51 +01:00
f29cbebb9d
Move block data test files to one location ( #1154 )
2022-07-05 16:14:52 +02:00
facd7342fb
Add network retries for fluffy block requests ( #1153 )
...
- Add retries when the network request failed on validation of
block header, body or receipts.
- Shuffle the first set of neighbours that the request is send to
in order to not always hit the same peer first for the same content
- Some general clean-up
2022-07-05 14:42:55 +02:00
2b4baff8ec
move block validation from execution payload generator to engine api
2022-07-04 20:36:30 +07:00
709d8ef255
fix markCanonicalChain bug
2022-07-04 19:32:14 +07:00
ce098bf88f
fix engine api test
2022-07-04 19:32:14 +07:00
1509dea39d
tidy up engine api
2022-07-04 19:32:13 +07:00
feda401e85
remove unused entry ( #1151 )
...
why:
temporary value, `nim-eth` master did not compile cleanly
2022-07-04 12:47:15 +01:00
fb12793482
Bump nim-eth and nim-bearssl and accompanying fixes ( #1150 )
2022-07-04 09:38:02 +02:00
0068e3b427
Bump some modules to fix breakage from #1145 ( #1149 )
2022-07-01 22:46:36 +02:00
7293a54e58
Added sepolia specs ( #1148 )
...
* Added sepolia specs
* temporarily avoid latest `master` branch from `nim-eth1`
why:
Currently does not cleanly compile after the `bearssl` split api update.
2022-07-01 21:16:26 +01:00
33015ea4a3
Adjust to the newly specified SSZ types for Block Body and Receipts ( #1147 )
2022-07-01 21:51:51 +02:00
134fe26997
Store proved snap accounts ( #1145 )
...
* Relocated `IntervalSets` to nim-stew repo
* Accumulate accounts on temporary kv-DB
why:
Explore the data as returned from snap/1. Will be converted to a
`eth/db` next.
details:
Verify and accumulate per/state-root accounts downloaded via snap.
also:
Some unit tests
* Replace `Table` by `TrieDatabaseRef` for accounts accumulator
* update ticker statistics
details:
mean/variance based counter update
* allow persistent db for proved accounts
* rebase, and globally activate unit test
* fix statistics
2022-07-01 12:42:17 +01:00
e274b347ae
add haveBlockAndState to BaseChainDB and use it in engine api
2022-06-30 16:11:43 +07:00
fdb048ed21
Add getLogs implementation ( #1143 )
...
* Add eth_getLogs json-rpc endpoint in Fluffy and Nimbus
2022-06-29 17:44:08 +02:00
7f0bc71b65
add invalidMissingAncestorReOrg test case
2022-06-27 20:18:54 +07:00
4a50b00c37
rpc: change getStorageAt param type to HexDataStr
...
if using HexQuantityStr, it will fail when given hex string with leading zeros
2022-06-27 17:47:59 +07:00
3f99ece6c5
rpc: change receipt status field type to HexQuantityStr
2022-06-27 13:01:32 +07:00
71ac6b7de5
engine-api-test: fix transactionReorg test case
2022-06-27 13:01:32 +07:00
b80eca0718
json-rpc: able to query finalized block and safe block header
...
engine-api:
- store safe block hash and finalized block hash
engine-api test:
- fix test case related to safe block hash and finalized block hash
2022-06-27 11:15:54 +07:00
cad74db423
Support for multiple content items over Portal stream ( #1090 )
...
Each content item send gets prefix with varint (max u32) allowing
to send multiple content items as requested by offer/accept flow.
2022-06-24 15:35:31 +02:00
d4d4e8c28f
Add accumulator buildProof, verifyProof and verifyHeader ( #1132 )
2022-06-23 21:00:59 +02:00
bf511b091e
Docs: Add separate section for seeding history data ( #1140 )
...
- Move section of eth_data_exporter out of default way to run
Fluffy
- Update section to explain full flow of seeding history data
into the network
2022-06-23 14:14:31 +02:00
589ba8367e
Adjust Grafana dashboard to represent the one used for the fleet ( #1139 )
...
This means adding the instance and container variables which can
be used to select the metrics of the specific fleet node.
2022-06-22 17:22:07 +02:00
Jordan Hrycaj
KonradStaniec
jangko
Kim De Mey