Kim De Mey
5355f4e73a
A bit of consistency clean-up in accumulator code ( #1201 )
2022-08-24 22:12:56 +02:00
KonradStaniec
88263e664b
Clean up ( #1197 )
2022-08-22 12:23:26 +02:00
KonradStaniec
5f2117d081
Refactor portal stream ( #1191 )
2022-08-17 09:32:06 +02:00
KonradStaniec
f0cd340163
Adhere to transport limits in seed methods ( #1186 )
...
* Add means to limit offered content to fit talkreq
* Add test for history network limits
* Change seed method api to return num of items offered
2022-08-09 14:32:41 +02:00
KonradStaniec
71f9e37482
Add getting block by number through accumulator ( #1182 )
2022-08-04 08:34:53 +02:00
Kim De Mey
50af402f59
Add verification of block data by use of accumulator ( #1178 )
2022-08-01 21:00:21 +02:00
KonradStaniec
a1253c67bd
Fix testnet tests ( #1176 )
...
* Testnet improvements
Increase timeout for reading
Add more logs
Offer endpoint can fail due to talkReq timeout, to avoid
test failure, retry it few times until success.
2022-07-29 14:24:07 +02:00
KonradStaniec
e3cabaff7f
Add bulk seeding to multiple peers ( #1170 )
...
* Add bulk seeding to multiple peers
2022-07-26 13:14:56 +02:00
KonradStaniec
aa945f1ed9
Additional seed endpoints ( #1164 )
...
* Additional seed endpoints
2022-07-20 12:46:42 +02:00
Kim De Mey
5abf05e455
Add validation for offered block bodies and receipts ( #1158 )
...
Also use a AsyncQueue now to handle offered data instead of the
network specific validation callbacks.
2022-07-11 16:29:16 +02:00
Kim De Mey
facd7342fb
Add network retries for fluffy block requests ( #1153 )
...
- Add retries when the network request failed on validation of
block header, body or receipts.
- Shuffle the first set of neighbours that the request is send to
in order to not always hit the same peer first for the same content
- Some general clean-up
2022-07-05 14:42:55 +02:00
Kim De Mey
fb12793482
Bump nim-eth and nim-bearssl and accompanying fixes ( #1150 )
2022-07-04 09:38:02 +02:00
Kim De Mey
33015ea4a3
Adjust to the newly specified SSZ types for Block Body and Receipts ( #1147 )
2022-07-01 21:51:51 +02:00
KonradStaniec
fdb048ed21
Add getLogs implementation ( #1143 )
...
* Add eth_getLogs json-rpc endpoint in Fluffy and Nimbus
2022-06-29 17:44:08 +02:00
Kim De Mey
cad74db423
Support for multiple content items over Portal stream ( #1090 )
...
Each content item send gets prefix with varint (max u32) allowing
to send multiple content items as requested by offer/accept flow.
2022-06-24 15:35:31 +02:00
Kim De Mey
d4d4e8c28f
Add accumulator buildProof, verifyProof and verifyHeader ( #1132 )
2022-06-23 21:00:59 +02:00
Kim De Mey
de2051dcf8
Add Accumulator build helper calls and refactors ( #1127 )
...
- Move the accumulator definitions to a history accumulator file
- Add accumulator build helper calls + temporary database
- Add a header gossip content key encoding test
- Refactor & some cleanup
2022-06-16 08:50:29 +02:00
Kim De Mey
6325712aa6
Add accumulator content keys for history network ( #1123 )
2022-06-14 23:38:34 +02:00
KonradStaniec
b23789d107
Add getting receipts from history network ( #1118 )
...
* Add getting receipts from history network
2022-06-10 12:24:53 +02:00
Kim De Mey
6d8b25a5f5
Update header accumulator test to start from Mainnet genesis ( #1111 )
2022-06-01 15:21:22 +02:00
KonradStaniec
5c78fe64e4
Fix utp connection leak on cancel ( #1107 )
...
* Fix utp connection leak on cancel
2022-05-31 14:28:02 +02:00
Kim De Mey
f219c69840
Rework of Portal json-rpc debug API and related functionality ( #1102 )
...
- More consistent naming in calls used by the JSON-RPC debug API
- Use storeContent everywhere to make sure content is only stored
if in range
- Remove populateHistoryDb subcommand and replace by JSON-RPC call
storeContent
- Remove some whitespace and fix some indentations
2022-05-24 13:27:22 +02:00
Kim De Mey
553be51217
Change radius-config option to radius and allow also logRadius ( #1100 )
2022-05-23 23:23:24 +02:00
Kim De Mey
a69b16abff
Cleanup some imports and fix some warnings in the process ( #1096 )
2022-05-22 22:44:15 +02:00
Kim De Mey
6e05c7588e
Add Header Accumulator for the Header Gossip Network ( #1094 )
2022-05-18 22:49:35 +02:00
KonradStaniec
6a9d875fe7
Add pruning metrics ( #1093 )
2022-05-17 12:47:24 +02:00
KonradStaniec
4e8c5f292e
Dynamic radius adjustments ( #1079 )
...
* Add config for node radius
* Adjust radius when deleting content
2022-05-12 18:04:37 +02:00
KonradStaniec
d3f231e3da
Add put method which preserves max size ( #1065 )
...
* Add put method which preserves max size
2022-05-09 17:18:57 +02:00
Kim De Mey
69366e1880
Improve selecting of nodes in neighborhood gossip ( #1072 )
...
Allow also concurrent neighborhood gossip jobs when seeding data
into the network.
Update Grafana dashboard for two additional metrics regarding
lookups in neighborhood gossip.
2022-05-07 13:50:16 +02:00
Kim De Mey
7bd9afa465
Bump nim-eth for max discv5 packet size ( #1070 )
2022-05-03 09:18:33 +02:00
Kim De Mey
99eea47dff
Fix clean-up socket in FindContent on cancel ( #1067 )
2022-04-20 18:46:00 +02:00
Kim De Mey
991edc1711
Fix log for block body network fetching ( #1066 )
2022-04-15 16:10:05 +02:00
Kim De Mey
f37dde1177
Fix two possible index errors on Portal wire protocol ( #1061 )
2022-04-13 17:42:32 +02:00
Kim De Mey
9626655557
Few more log improvements for Fluffy ( #1060 )
2022-04-13 11:17:07 +02:00
Kim De Mey
d337806301
Improve logs and loglevels, and general cleanup ( #1059 )
2022-04-13 07:56:01 +02:00
Kim De Mey
0fba19b81a
Improvements to the propagation and seeding of data ( #1058 )
...
* Improvements to the propagation and seeding of data
- Use a lookup for nodes selection in neighborhoodGossip
- Rework populate db code and add `propagateBlockHistoryDb` call
and portal_history__propagateBlock json-rpc call
- Small adjustment to blockwalk
* Avoid storing out-of-range data in the propagate db calls
2022-04-12 15:49:19 +02:00
Kim De Mey
0195d5830b
Fix portal_content_keys_offered metric regression ( #1057 )
2022-04-12 10:59:41 +02:00
Kim De Mey
ed38ed494f
Add validation handler to Portal wire protocol ( #1055 )
...
- Validation handler allows for network specific validation of
the content
- Added also pruning of the allowed connections on PortalStream
2022-04-11 19:42:38 +02:00
KonradStaniec
eb84eb0854
Improve content offer ( #1050 )
...
* Improve content offer
2022-04-11 11:25:36 +02:00
KonradStaniec
0365d030fb
Increase packet size of utp over discv5 ( #1042 )
...
* Increase packet size of utp over discv5
2022-04-07 13:54:49 +02:00
KonradStaniec
203d8e8b70
Inital poke implementation ( #1037 )
...
* Initial poke implementation
2022-04-06 13:47:23 +02:00
Kim De Mey
9d656e99c4
Add concurrency to the content offers of neighborhoodGossip proc ( #1027 )
...
* Add concurrency to the content offers of neighborhoodGossip proc
And remove some whitespace
* Remove more whitespace and adjust for 80 char line limit
* Update fluffy grafana dashboard to include gossip offer results
2022-04-01 18:01:50 +02:00
KonradStaniec
4cb4f50bf3
Refactor connection process ( #1016 )
...
* Refactor connection process
2022-03-30 07:12:39 +02:00
Kim De Mey
89023782f3
Let Portal wire protocol adhere to the discv5 packet size limits ( #1008 )
...
- Truncate returned number of ENRs in Portal Nodes message to fit
the discv5 max. packet size
- Truncate returned number of ENRs in Portal Content message to
fit the discv5 max. packet size
- Use more detailed packet size calculation for max content payload
in Portal Content message.
- Add metrics to track how many ENRs get packed in the Portal
Nodes and Content messages.
2022-03-24 21:06:13 +01:00
Kim De Mey
bde07054d1
Add metrics to the Portal wire protocol ( #1006 )
2022-03-23 16:32:59 +01:00
Kim De Mey
74127644db
Adjust findNodes proc naming for consistency ( #1002 )
2022-03-19 08:54:42 +01:00
Kim De Mey
aff785ab48
Avoid opening an uTP stream when no content was accepted ( #1001 )
...
When the returned content key bitlist of an accept message is all
zeroes, neither side should try to set up an uTP stream.
2022-03-18 18:29:06 +01:00
Kim De Mey
08aa20a791
Fix error when dataDir does not exist ( #1000 )
...
- When dataDir does not exist, attempt to create the dir
- Some additional logging improvements
- Whitespace clean-up
2022-03-18 13:06:57 +01:00
Kim De Mey
b14dfea553
Improve portal wire logging by adding protocolId ( #997 )
2022-03-17 13:19:36 +01:00
KonradStaniec
b951139af9
Add basic validation of history network ( #983 )
...
* Add basic validation of history network
* Save only valid objects to db
2022-03-02 15:29:12 +01:00
