cad74db423
Support for multiple content items over Portal stream ( #1090 )
...
Each content item send gets prefix with varint (max u32) allowing
to send multiple content items as requested by offer/accept flow.
2022-06-24 15:35:31 +02:00
d4d4e8c28f
Add accumulator buildProof, verifyProof and verifyHeader ( #1132 )
2022-06-23 21:00:59 +02:00
bf511b091e
Docs: Add separate section for seeding history data ( #1140 )
...
- Move section of eth_data_exporter out of default way to run
Fluffy
- Update section to explain full flow of seeding history data
into the network
2022-06-23 14:14:31 +02:00
589ba8367e
Adjust Grafana dashboard to represent the one used for the fleet ( #1139 )
...
This means adding the instance and container variables which can
be used to select the metrics of the specific fleet node.
2022-06-22 17:22:07 +02:00
196428dcf5
Use one ws connection to transfer data from source ( #1136 )
2022-06-22 08:50:58 +02:00
489b975189
Add seed db file format ( #1126 )
...
* Add seed db file format
2022-06-20 16:52:48 +02:00
de2051dcf8
Add Accumulator build helper calls and refactors ( #1127 )
...
- Move the accumulator definitions to a history accumulator file
- Add accumulator build helper calls + temporary database
- Add a header gossip content key encoding test
- Refactor & some cleanup
2022-06-16 08:50:29 +02:00
6325712aa6
Add accumulator content keys for history network ( #1123 )
2022-06-14 23:38:34 +02:00
49bdaa112e
Increase wait time for content propagation to avoid CI failures ( #1122 )
2022-06-14 14:23:07 +02:00
b23789d107
Add getting receipts from history network ( #1118 )
...
* Add getting receipts from history network
2022-06-10 12:24:53 +02:00
730013bde3
Change test data and propagation format ( #1115 )
...
* Change test data and propagation format
* Add tool to download data from local node
2022-06-08 15:14:01 +02:00
0776f35e0c
Use queries with custom xor function ( #1113 )
...
* Use queries with custom xor function
2022-06-03 13:44:42 +02:00
6d8b25a5f5
Update header accumulator test to start from Mainnet genesis ( #1111 )
2022-06-01 15:21:22 +02:00
7216160ad0
Update Ultralight bootstrap nodes for testnet ( #1110 )
2022-06-01 08:52:54 +02:00
5c78fe64e4
Fix utp connection leak on cancel ( #1107 )
...
* Fix utp connection leak on cancel
2022-05-31 14:28:02 +02:00
af10e8f179
Do not use vacuum when pruning ( #1103 )
...
* Do not use vacuum when pruning
2022-05-26 08:26:08 +02:00
f219c69840
Rework of Portal json-rpc debug API and related functionality ( #1102 )
...
- More consistent naming in calls used by the JSON-RPC debug API
- Use storeContent everywhere to make sure content is only stored
if in range
- Remove populateHistoryDb subcommand and replace by JSON-RPC call
storeContent
- Remove some whitespace and fix some indentations
2022-05-24 13:27:22 +02:00
553be51217
Change radius-config option to radius and allow also logRadius ( #1100 )
2022-05-23 23:23:24 +02:00
a69b16abff
Cleanup some imports and fix some warnings in the process ( #1096 )
2022-05-22 22:44:15 +02:00
6e05c7588e
Add Header Accumulator for the Header Gossip Network ( #1094 )
2022-05-18 22:49:35 +02:00
6a9d875fe7
Add pruning metrics ( #1093 )
2022-05-17 12:47:24 +02:00
56c3e31efc
Refactor and clean-up of Portal wire protocol test ( #1089 )
2022-05-16 13:17:42 +02:00
4e8c5f292e
Dynamic radius adjustments ( #1079 )
...
* Add config for node radius
* Adjust radius when deleting content
2022-05-12 18:04:37 +02:00
d3f231e3da
Add put method which preserves max size ( #1065 )
...
* Add put method which preserves max size
2022-05-09 17:18:57 +02:00
69366e1880
Improve selecting of nodes in neighborhood gossip ( #1072 )
...
Allow also concurrent neighborhood gossip jobs when seeding data
into the network.
Update Grafana dashboard for two additional metrics regarding
lookups in neighborhood gossip.
2022-05-07 13:50:16 +02:00
7bd9afa465
Bump nim-eth for max discv5 packet size ( #1070 )
2022-05-03 09:18:33 +02:00
99eea47dff
Fix clean-up socket in FindContent on cancel ( #1067 )
2022-04-20 18:46:00 +02:00
991edc1711
Fix log for block body network fetching ( #1066 )
2022-04-15 16:10:05 +02:00
f37dde1177
Fix two possible index errors on Portal wire protocol ( #1061 )
2022-04-13 17:42:32 +02:00
3d818ad8ee
Update Fluffy README.md with metrics information
2022-04-13 16:11:01 +02:00
9626655557
Few more log improvements for Fluffy ( #1060 )
2022-04-13 11:17:07 +02:00
d337806301
Improve logs and loglevels, and general cleanup ( #1059 )
2022-04-13 07:56:01 +02:00
0fba19b81a
Improvements to the propagation and seeding of data ( #1058 )
...
* Improvements to the propagation and seeding of data
- Use a lookup for nodes selection in neighborhoodGossip
- Rework populate db code and add `propagateBlockHistoryDb` call
and portal_history__propagateBlock json-rpc call
- Small adjustment to blockwalk
* Avoid storing out-of-range data in the propagate db calls
2022-04-12 15:49:19 +02:00
0195d5830b
Fix portal_content_keys_offered metric regression ( #1057 )
2022-04-12 10:59:41 +02:00
ed38ed494f
Add validation handler to Portal wire protocol ( #1055 )
...
- Validation handler allows for network specific validation of
the content
- Added also pruning of the allowed connections on PortalStream
2022-04-11 19:42:38 +02:00
82aab2a404
Bump chronos and json-rpc and avoid http conn re-use for blockwalk ( #1053 )
...
Currently re-using http connections for the json-rpc requests
seems to slow down the requests considerably. Avoid this by
forcing a close after each requests in the blockwalk tool.
2022-04-11 19:39:45 +02:00
eb84eb0854
Improve content offer ( #1050 )
...
* Improve content offer
2022-04-11 11:25:36 +02:00
0365d030fb
Increase packet size of utp over discv5 ( #1042 )
...
* Increase packet size of utp over discv5
2022-04-07 13:54:49 +02:00
203d8e8b70
Inital poke implementation ( #1037 )
...
* Initial poke implementation
2022-04-06 13:47:23 +02:00
b3570fae6a
Add api to get n furthest elements from db ( #1026 )
...
* Add api to get n furthest elements from db
2022-04-03 15:14:44 +02:00
6a90937d4b
Add Trin and Ultralight bootstrap nodes to Fluffy ( #1028 )
2022-04-01 20:37:38 +02:00
9d656e99c4
Add concurrency to the content offers of neighborhoodGossip proc ( #1027 )
...
* Add concurrency to the content offers of neighborhoodGossip proc
And remove some whitespace
* Remove more whitespace and adjust for 80 char line limit
* Update fluffy grafana dashboard to include gossip offer results
2022-04-01 18:01:50 +02:00
8c3f210526
Add blockwalk tool to test range of blocks on Portal network ( #1020 )
2022-03-30 08:55:38 +02:00
4cb4f50bf3
Refactor connection process ( #1016 )
...
* Refactor connection process
2022-03-30 07:12:39 +02:00
392a540eb5
Remove unused import of config to avoid select_backend db import ( #1018 )
...
* Remove unused import of config to avoid select_backend db import
- Importing nimbus-eth1 config.nim causes import of select_backend
which will default cause an import of kvstore_rocksdb and thus a
require rocksdb. Remove unused one to avoid rocksdb dependency
for Fluffy.
- Remove some whitespace in bridge_client (to make fluffy CI
trigger for sure).
* Use specific cache keys for fluffy CI workflow
* Disable Fluffy CI reproducibility test
2022-03-29 15:51:48 +02:00
97c0cea767
Add grafana dashboard for Fluffy metrics ( #1012 )
2022-03-28 09:43:55 +02:00
89023782f3
Let Portal wire protocol adhere to the discv5 packet size limits ( #1008 )
...
- Truncate returned number of ENRs in Portal Nodes message to fit
the discv5 max. packet size
- Truncate returned number of ENRs in Portal Content message to
fit the discv5 max. packet size
- Use more detailed packet size calculation for max content payload
in Portal Content message.
- Add metrics to track how many ENRs get packed in the Portal
Nodes and Content messages.
2022-03-24 21:06:13 +01:00
bde07054d1
Add metrics to the Portal wire protocol ( #1006 )
2022-03-23 16:32:59 +01:00
74127644db
Adjust findNodes proc naming for consistency ( #1002 )
2022-03-19 08:54:42 +01:00
aff785ab48
Avoid opening an uTP stream when no content was accepted ( #1001 )
...
When the returned content key bitlist of an accept message is all
zeroes, neither side should try to set up an uTP stream.
2022-03-18 18:29:06 +01:00
Kim De Mey
KonradStaniec