Commit Graph

268 Commits

Author SHA1 Message Date
Simon-Pierre Vivier 650a9487e1
feat: waku rendezvous wrapper (#2962) 2024-12-09 15:22:36 -05:00
Simon-Pierre Vivier 2ab9c3d363
feat: remove Waku Sync 1.0 & Negentropy (#3185) 2024-11-29 09:09:41 -05:00
NagyZoltanPeter 8368ff006d
chore: Partial version bumps for v0.34.0-rc.1 (#3172)
* Bumps for v0.34.0-rc.1 - partial bumping - libp2p
* Avoid importing quic and ngtcp2 dependencies through tests
* libp2p 1.7.1, fixes RendezVous construction
2024-11-10 09:27:04 +01:00
gabrielmer 400d7a54f6
feat: running periodicaly peer exchange if discv5 is disabled (#3150) 2024-10-30 12:51:04 +02:00
gabrielmer b3656d6eea
chore: naming connectPeer procedure (#3157) 2024-10-29 18:37:07 +02:00
gabrielmer 99ac68447c
fix: linting error (#3156) 2024-10-29 15:36:21 +02:00
fryorcraken 956fde6ebc
feat: change latency buckets (#3153) 2024-10-29 14:39:12 +11:00
Ivan FB cfde7eea82
chore: Circuit relay (#3112)
* undo apt install libpcre (not circuit-relay related.)
* nat.nim: protect against possible exceptions when calling getExternalIP
* new external CLI argument, isRelayClient
* waku factory change to mount circuit hop proto by default
* waku_node: move autonat_service to a separate module
2024-10-28 09:17:46 +01:00
NagyZoltanPeter 268e7e66d0
chore: easy setup fleets for lpt (#3125)
* Added bootstrap peer exchange discovery option for easy setup ltp
* Extended with PX discovery, auto-dial of PX cap peers, added switching service peers if failed with original
* Added peer-exchange, found capable peers test, metrics on peer stability and availability, dashboard adjustments
* Updated and actualized README.md for liteprotocoltester
* Created jenkinsfile for liteprotocoltester deployment
* Fixed dial exception during lightpublish
* Add configuration for requesting and testing peer exchange peers
* Extended examples added to Readme
* Added metrics port configurability
---------

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
2024-10-25 22:59:02 +02:00
Ivan FB 3dc3fc8e5a
peer_manager: prevent too intense loop when no peers connected (#3130) 2024-10-22 20:09:25 +02:00
Simon-Pierre Vivier bdb31cc114
added randomness to peer selection (#3123) 2024-10-16 15:18:47 -04:00
gabrielmer 25da810257
chore: adding to libwaku dial and disconnect by peerIds (#3111) 2024-10-15 15:32:02 +03:00
richΛrd c861fa9f75
chore: remove warnings (#3106)
- Removes deprecation and unused import warnings for libwaku
- Removes unused imports
- Adds .base. pragma to `SubscriptionObserver.onSubscribe`
- Uses casting for uint to enums conversions
- Bumps nim-chronicles
2024-10-10 08:40:09 -04:00
gabrielmer 1d2b910f54
fix: remove spammy log (#3091) 2024-10-04 17:11:40 +03:00
Darshan K 0f8e874000
refactor: re-arrange function based on responsibility of peer-manager (#3086) 2024-10-04 15:23:20 +05:30
gabrielmer eb2bbae665
fix: out connections leak (#3077) 2024-10-03 12:37:22 +03:00
Ivan FB 5875ed63af
chore: Optimize store (#3061)
* use messages_lookup to retrieve timestamps
* deep refactoring in db_postgres for better use of async approach
2024-10-01 23:36:03 +02:00
gabrielmer 8b0884c7b5
fix: rejecting excess relay connections (#3065) 2024-09-27 19:35:18 +03:00
Darshan K 729e63f53d
refactor: wrap peer store (#3051)
Encapsulate peerstore with wakupeerstore
2024-09-27 18:16:46 +05:30
Ivan FB 711e7db1e9
chore: test peer connection management (#3049)
* Make some useful consts public, add some utils.
* Implement various utilities.
* peer_manager reconnectPeers enhancements

---------

Co-authored-by: Álex Cabeza Romero <alex93cabeza@gmail.com>
2024-09-24 18:20:29 +02:00
NagyZoltanPeter 0a7f16a332
chore: rate limit peer exchange protocol, enhanced response status in RPC (#3035)
* Enhanced peer-ex protocol - added rate limiting, added response status and desc to the rpc

* Better error result handling for PeerEx request, adjusted tests

* Refactored RateLimit configuration option for better CLI UX - now possible to set separate limits per protocol. Adjusted mountings. Added and adjusted tests

* Fix libwaku due to changes of error return type of fetchPeerExchangePeers

* Fix rate limit setting tests due to changed defaults

* Introduce new gauge to help dasboard effectively show current rate limit applied for protocol

* Adjust timeing in filter rate limit test to let macos CI test run ok.

* Address review findings, namings, error logs, removed left-overs

* Changes to reflect latest spec agreement and changes. PeerExchange RPC is changed the now respond structure will contain status_code and status_desc.
2024-09-18 15:58:07 +02:00
gabrielmer 51391aa2ac
adding a dynamic sleep interval in the connectivity loop (#3031) 2024-09-12 22:49:47 +02:00
fryorcraken ce9a8c468a
chore: use submodule nph in CI to check lint (#3027) 2024-09-11 11:51:42 +10:00
gabrielmer a3cd2a1a92
chore: deprecating pubsub topic (#2997) 2024-09-10 15:07:12 -06:00
Ivan FB e8a49b76b2
chore: Better timing and requestId detail for slower store db queries (#2994)
* Better timing and requestId detail for store db queries slower than two seconds
* Adapt tests and client to allow sending custom store requestId
2024-08-29 22:56:14 +02:00
Ivan FB 0f68274c85
chore: delivery monitor for store v3 reliability protocol (#2977)
- Use of observer observable pattern to inform delivery_monitor about subscription state
- send_monitor becomes a publish observer of lightpush and relay
- deliver monitor add more protection against possible crash and better logs
- creating a separate proc in store client for delivery monitor
2024-08-27 16:49:46 +02:00
gabrielmer e8bce67d76
chore: logging received message info via onValidated observer (#2973) 2024-08-19 14:13:28 +02:00
gabrielmer 225b5e7260
avoid using the msg key in chronicles (#2970) 2024-08-14 16:40:08 +02:00
Simon-Pierre Vivier 2cc86c51da
feat: Nwaku Sync (#2403)
* feat: Waku Sync Protocol

* feat: state machine (#2656)

* feat: pruning storage mehcanism (#2673)

* feat: message transfer mechanism & tests (#2688)

* update docker files

* added ENR filed for sync & misc. fixes

* adding new sync range param & fixes

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
2024-08-13 07:27:34 -04:00
gabrielmer 54b5222222
pruning excess in relay connections (#2965) 2024-08-12 17:59:11 +02:00
Ivan FB d4e8a0dab6
chore: Simplification of store legacy code (#2931) 2024-07-30 14:05:23 +02:00
Simon-Pierre Vivier aed2a1130e
feat: store resume (#2919)
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-07-30 07:23:39 -04:00
Simon-Pierre Vivier b4618f98ed
chore: add peer filtering by cluster for waku peer exchange (#2932) 2024-07-29 15:53:43 -04:00
Simon-Pierre Vivier e4e01fabfe
fix: add back waku discv5 metrics (#2927)
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-07-26 16:18:14 -04:00
Darshan K 08973add7e
fix: update and shift unittest (#2934)
* fix: update and shift location of unit test
2024-07-26 16:57:34 +05:30
Darshan K 8d107b0ded
fix: handle rln-relay-message-limit (#2867)
* fix: enforcing rln-contract max message limit and resolve early
2024-07-22 22:28:45 +05:30
NagyZoltanPeter ba418ab5ba
feat: DOS protection of non relay protocols - rate limit phase3 (#2897)
* DOS protection of non relay protocols - rate limit phase3:
- Enhanced TokenBucket to be able to add compensation tokens based on previous usage percentage,
- per peer rate limiter 'PeerRateLimier' applied on waku_filter_v2 with opinionated default of acceptable request rate
- Add traffic metrics to filter message push
- RequestRateLimiter added to combine simple token bucket limiting of request numbers but consider per peer usage over time and prevent some peers to over use the service
  (although currently rule violating peers will not be disconnected by this time only their requests will get not served)
- TimedMap utility created (inspired and taken from libp2p TimedCache) which serves as forgiving feature for peers had been overusing the service.
- Added more tests
- Fix rebase issues
- Applied new RequestRateLimiter for store and legacy_store and lightpush
* Incorporate review comments, typos, file/class naming and placement changes.
* Add issue link reference of the original issue with nim-chronos TokenBucket
* Make TimedEntry of TimedMap private and not mixable with similar named in libp2p
* Fix review comments, renamings, const instead of values and more comments.
2024-07-16 15:46:21 +02:00
gabrielmer 8578fb0c3e
chore: improving logging under debugDiscv5 flag (#2899) 2024-07-15 10:55:31 +03:00
Simon-Pierre Vivier f54ba10bc7
chore(archive): archive and drivers refactor (#2761)
* queue driver refactor (#2753)
* chore(archive): archive refactor (#2752)
* chore(archive): sqlite driver refactor (#2754)
* chore(archive): postgres driver refactor (#2755)
* chore(archive): renaming & copies (#2751)
* posgres legacy: stop using the storedAt field
* migration script 6: we still need the id column
  The id column is needed because it contains the message digest
  which is used in store v2, and we need to keep support to
  store v2 for a while
* legacy archive: set target migration version to 6
* waku_node: try to use wakuLegacyArchive if wakuArchive is nil
* node_factory, waku_node: mount legacy and future store simultaneously
  We want the nwaku node to simultaneously support store-v2
  requests and store-v3 requests.
  Only the legacy archive is in charge of archiving messages, and the
  archived information is suitable to fulfill both store-v2 and
  store-v3 needs.
* postgres_driver: adding temporary code until store-v2 is removed

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan Folgueira Bande <ivansete@status.im>
2024-07-12 18:19:12 +02:00
gabrielmer e1518cf9ff
chore: deprecating named sharding (#2723) 2024-07-09 18:36:12 +03:00
gabrielmer d0980eba4c
setting connectivity loop interval to 30 seconds (#2878) 2024-07-09 17:33:18 +03:00
Ivan FB fd6a71cdd7
chore: Bump dependencies for v0.31.0 (#2885)
* bump_dependencies.md: add nim-results dependency
* change imports stew/results to results
* switching to Nim 2.0.8
* waku.nimble: reflect the requirement nim 1.6.0 to 2.0.8
  Adding --mm:refc as nim 2.0 enables a new garbage collector that we're
  not yet ready to support
* adapt waku code to Nim 2.0
* gcsafe adaptations because Nim 2.0 is more strict
2024-07-09 13:14:28 +02:00
gabrielmer cae0c7e378
chore: saving agent and protoVersion in peerStore (#2860) 2024-07-01 13:29:14 +02:00
Darshan K 31c632e427
chore: unit test for duplicate message push (#2852)
* chore: add unit test for testing duplicate message push with timedcache

* chore: update according to better naming convention

---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2024-06-28 18:16:06 +05:30
Darshan K 9bd8c33aee
chore: remove all pre-nim-1.6 deadcode from codebase (#2857) 2024-06-28 16:04:57 +05:30
gabrielmer 974b8a39ac
chore: adding discv5 logs (#2811) 2024-06-26 14:25:58 +02:00
gabrielmer 01050138c6
fix: only set disconnect time on left event (#2831) 2024-06-24 10:20:09 +02:00
gabrielmer 41bc582823
stop connecting to out peers until target is reached (#2823) 2024-06-20 12:16:15 +02:00
gabrielmer cda18f96ca
fix: update peers ENRs in peer store in case they are updated (#2818) 2024-06-19 17:29:55 +02:00
gabrielmer 93e9ba22aa
fix: revert "chore: adding observers for message logging (#2800)" (#2815) 2024-06-17 13:14:05 +02:00