Commit Graph

111 Commits

Author SHA1 Message Date
KonradStaniec 181243b6c4
Make portal_historyOffer spec compliant (#1259)
* Make portal_historyOffer spec compliant
2022-10-11 12:10:54 +02:00
Kim De Mey 78f7de1344
Make accumulator finite at merge block (#1256)
- Let accumulator finish its last pre merge epoch (hash_tree_root
on incomplete epoch).
- Adjust code to use isPreMerge and remove isCurrentEpoch
- Split up tests to a set that runs with a mainnet merge block
number and a set that runs with a testing value.
2022-10-10 12:59:55 +02:00
Kim De Mey 4e08f774d5
Fix issue where invalid / not validated would get gossiped (#1247)
Also requires us to split header data propagation from block body
and receipts propagation as the now fixed bug would allow for more
data to be gossiped even when data does not get validated (which
requires the headers).
2022-09-29 08:42:54 +02:00
Kim De Mey 1bd4911e78
Remove ChaindId from BlockKey and update test vectors (#1242) 2022-09-28 09:09:38 +02:00
Kim De Mey 87f28458a5
StyleCheck fixes for Fluffy and lcproxy (#1220)
Also add the compiler options in the nim.cfg of Fluffy and
lcproxy (only visible with direct nimble usage).
2022-09-10 21:00:27 +02:00
Kim De Mey 011f44abea
Split parsing and seeding functionality for history data (#1218) 2022-09-09 21:21:48 +02:00
Kim De Mey 621c6a31a7
Fix bug in inCurrentEpoch and improve accumulator related tests (#1217)
* Fix bug in inCurrentEpoch and improve accumulator related tests

- Fix negative wraparound / underflow in inCurrentEpoch
- Add tests in accumulator tests to verify the above
- Add header offer tests with accumulator that does and doesn't
contain historical epochs
- Additional clean-up of history tests
- enable canonicalVerify in the tests
2022-09-09 13:12:09 +02:00
Jacek Sieka c2ed731fa5
eth: adapt to smaller eth_types (#1210) 2022-09-03 20:15:35 +02:00
Kim De Mey 5355f4e73a
A bit of consistency clean-up in accumulator code (#1201) 2022-08-24 22:12:56 +02:00
KonradStaniec 88263e664b
Clean up (#1197) 2022-08-22 12:23:26 +02:00
KonradStaniec 5f2117d081
Refactor portal stream (#1191) 2022-08-17 09:32:06 +02:00
KonradStaniec f0cd340163
Adhere to transport limits in seed methods (#1186)
* Add means to limit offered content to fit talkreq

* Add test for history network limits

* Change seed method api to return num of items offered
2022-08-09 14:32:41 +02:00
KonradStaniec 71f9e37482
Add getting block by number through accumulator (#1182) 2022-08-04 08:34:53 +02:00
Kim De Mey 50af402f59
Add verification of block data by use of accumulator (#1178) 2022-08-01 21:00:21 +02:00
KonradStaniec a1253c67bd
Fix testnet tests (#1176)
* Testnet improvements

Increase timeout for reading
Add more logs
Offer endpoint can fail due to talkReq timeout, to avoid
test failure, retry it few times until success.
2022-07-29 14:24:07 +02:00
KonradStaniec e3cabaff7f
Add bulk seeding to multiple peers (#1170)
* Add bulk seeding to multiple peers
2022-07-26 13:14:56 +02:00
KonradStaniec aa945f1ed9
Additional seed endpoints (#1164)
* Additional seed endpoints
2022-07-20 12:46:42 +02:00
Kim De Mey 5abf05e455
Add validation for offered block bodies and receipts (#1158)
Also use a AsyncQueue now to handle offered data instead of the
network specific validation callbacks.
2022-07-11 16:29:16 +02:00
Kim De Mey facd7342fb
Add network retries for fluffy block requests (#1153)
- Add retries when the network request failed on validation of
block header, body or receipts.
- Shuffle the first set of neighbours that the request is send to
in order to not always hit the same peer first for the same content
- Some general clean-up
2022-07-05 14:42:55 +02:00
Kim De Mey fb12793482
Bump nim-eth and nim-bearssl and accompanying fixes (#1150) 2022-07-04 09:38:02 +02:00
Kim De Mey 33015ea4a3
Adjust to the newly specified SSZ types for Block Body and Receipts (#1147) 2022-07-01 21:51:51 +02:00
KonradStaniec fdb048ed21
Add getLogs implementation (#1143)
* Add eth_getLogs json-rpc endpoint in Fluffy and Nimbus
2022-06-29 17:44:08 +02:00
Kim De Mey cad74db423
Support for multiple content items over Portal stream (#1090)
Each content item send gets prefix with varint (max u32) allowing
to send multiple content items as requested by offer/accept flow.
2022-06-24 15:35:31 +02:00
Kim De Mey d4d4e8c28f
Add accumulator buildProof, verifyProof and verifyHeader (#1132) 2022-06-23 21:00:59 +02:00
Kim De Mey de2051dcf8
Add Accumulator build helper calls and refactors (#1127)
- Move the accumulator definitions to a history accumulator file
- Add accumulator build helper calls + temporary database
- Add a header gossip content key encoding test
- Refactor & some cleanup
2022-06-16 08:50:29 +02:00
Kim De Mey 6325712aa6
Add accumulator content keys for history network (#1123) 2022-06-14 23:38:34 +02:00
KonradStaniec b23789d107
Add getting receipts from history network (#1118)
* Add getting receipts from history network
2022-06-10 12:24:53 +02:00
Kim De Mey 6d8b25a5f5
Update header accumulator test to start from Mainnet genesis (#1111) 2022-06-01 15:21:22 +02:00
KonradStaniec 5c78fe64e4
Fix utp connection leak on cancel (#1107)
* Fix utp connection leak on cancel
2022-05-31 14:28:02 +02:00
Kim De Mey f219c69840
Rework of Portal json-rpc debug API and related functionality (#1102)
- More consistent naming in calls used by the JSON-RPC debug API
- Use storeContent everywhere to make sure content is only stored
if in range
- Remove populateHistoryDb subcommand and replace by JSON-RPC call
storeContent
- Remove some whitespace and fix some indentations
2022-05-24 13:27:22 +02:00
Kim De Mey 553be51217
Change radius-config option to radius and allow also logRadius (#1100) 2022-05-23 23:23:24 +02:00
Kim De Mey a69b16abff
Cleanup some imports and fix some warnings in the process (#1096) 2022-05-22 22:44:15 +02:00
Kim De Mey 6e05c7588e
Add Header Accumulator for the Header Gossip Network (#1094) 2022-05-18 22:49:35 +02:00
KonradStaniec 6a9d875fe7
Add pruning metrics (#1093) 2022-05-17 12:47:24 +02:00
KonradStaniec 4e8c5f292e
Dynamic radius adjustments (#1079)
* Add config for node radius

* Adjust radius when deleting content
2022-05-12 18:04:37 +02:00
KonradStaniec d3f231e3da
Add put method which preserves max size (#1065)
* Add put method which preserves max size
2022-05-09 17:18:57 +02:00
Kim De Mey 69366e1880
Improve selecting of nodes in neighborhood gossip (#1072)
Allow also concurrent neighborhood gossip jobs when seeding data
into the network.
Update Grafana dashboard for two additional metrics regarding
lookups in neighborhood gossip.
2022-05-07 13:50:16 +02:00
Kim De Mey 7bd9afa465
Bump nim-eth for max discv5 packet size (#1070) 2022-05-03 09:18:33 +02:00
Kim De Mey 99eea47dff
Fix clean-up socket in FindContent on cancel (#1067) 2022-04-20 18:46:00 +02:00
Kim De Mey 991edc1711
Fix log for block body network fetching (#1066) 2022-04-15 16:10:05 +02:00
Kim De Mey f37dde1177
Fix two possible index errors on Portal wire protocol (#1061) 2022-04-13 17:42:32 +02:00
Kim De Mey 9626655557
Few more log improvements for Fluffy (#1060) 2022-04-13 11:17:07 +02:00
Kim De Mey d337806301
Improve logs and loglevels, and general cleanup (#1059) 2022-04-13 07:56:01 +02:00
Kim De Mey 0fba19b81a
Improvements to the propagation and seeding of data (#1058)
* Improvements to the propagation and seeding of data

- Use a lookup for nodes selection in neighborhoodGossip
- Rework populate db code and add `propagateBlockHistoryDb` call
and portal_history__propagateBlock json-rpc call
- Small adjustment to blockwalk

* Avoid storing out-of-range data in the propagate db calls
2022-04-12 15:49:19 +02:00
Kim De Mey 0195d5830b
Fix portal_content_keys_offered metric regression (#1057) 2022-04-12 10:59:41 +02:00
Kim De Mey ed38ed494f
Add validation handler to Portal wire protocol (#1055)
- Validation handler allows for network specific validation of
the content
- Added also pruning of the allowed connections on PortalStream
2022-04-11 19:42:38 +02:00
KonradStaniec eb84eb0854
Improve content offer (#1050)
* Improve content offer
2022-04-11 11:25:36 +02:00
KonradStaniec 0365d030fb
Increase packet size of utp over discv5 (#1042)
* Increase packet size of utp over discv5
2022-04-07 13:54:49 +02:00
KonradStaniec 203d8e8b70
Inital poke implementation (#1037)
* Initial poke implementation
2022-04-06 13:47:23 +02:00
Kim De Mey 9d656e99c4
Add concurrency to the content offers of neighborhoodGossip proc (#1027)
* Add concurrency to the content offers of neighborhoodGossip proc

And remove some whitespace

* Remove more whitespace and adjust for 80 char line limit

* Update fluffy grafana dashboard to include gossip offer results
2022-04-01 18:01:50 +02:00