* Enhanced peer-ex protocol - added rate limiting, added response status and desc to the rpc
* Better error result handling for PeerEx request, adjusted tests
* Refactored RateLimit configuration option for better CLI UX - now possible to set separate limits per protocol. Adjusted mountings. Added and adjusted tests
* Fix libwaku due to changes of error return type of fetchPeerExchangePeers
* Fix rate limit setting tests due to changed defaults
* Introduce new gauge to help dasboard effectively show current rate limit applied for protocol
* Adjust timeing in filter rate limit test to let macos CI test run ok.
* Address review findings, namings, error logs, removed left-overs
* Changes to reflect latest spec agreement and changes. PeerExchange RPC is changed the now respond structure will contain status_code and status_desc.
* DOS protection of non relay protocols - rate limit phase3:
- Enhanced TokenBucket to be able to add compensation tokens based on previous usage percentage,
- per peer rate limiter 'PeerRateLimier' applied on waku_filter_v2 with opinionated default of acceptable request rate
- Add traffic metrics to filter message push
- RequestRateLimiter added to combine simple token bucket limiting of request numbers but consider per peer usage over time and prevent some peers to over use the service
(although currently rule violating peers will not be disconnected by this time only their requests will get not served)
- TimedMap utility created (inspired and taken from libp2p TimedCache) which serves as forgiving feature for peers had been overusing the service.
- Added more tests
- Fix rebase issues
- Applied new RequestRateLimiter for store and legacy_store and lightpush
* Incorporate review comments, typos, file/class naming and placement changes.
* Add issue link reference of the original issue with nim-chronos TokenBucket
* Make TimedEntry of TimedMap private and not mixable with similar named in libp2p
* Fix review comments, renamings, const instead of values and more comments.
* queue driver refactor (#2753)
* chore(archive): archive refactor (#2752)
* chore(archive): sqlite driver refactor (#2754)
* chore(archive): postgres driver refactor (#2755)
* chore(archive): renaming & copies (#2751)
* posgres legacy: stop using the storedAt field
* migration script 6: we still need the id column
The id column is needed because it contains the message digest
which is used in store v2, and we need to keep support to
store v2 for a while
* legacy archive: set target migration version to 6
* waku_node: try to use wakuLegacyArchive if wakuArchive is nil
* node_factory, waku_node: mount legacy and future store simultaneously
We want the nwaku node to simultaneously support store-v2
requests and store-v3 requests.
Only the legacy archive is in charge of archiving messages, and the
archived information is suitable to fulfill both store-v2 and
store-v3 needs.
* postgres_driver: adding temporary code until store-v2 is removed
---------
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan Folgueira Bande <ivansete@status.im>
* chore: update logs with topic discription & debug msg
* chore: update unit according to error msg
* chore: update rest unit according to error msg
* chore: add content-topic with debug msg
* feat: Added simple, configurable rate limit for lightpush and store-query
Adjust lightpush rest response to rate limit, added tests ann some fixes
Add rest store query test for rate limit checks and proper error response
Update apps/wakunode2/external_config.nim
Move chronos/tokenbucket to nwaku codebasee with limited and fixed feature set
Add meterics counter to lightpush rate limits
Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
* chore: remove deprecated legacy filter protocol
* fix: do not use legacy import in test
* fix: remove legacy test references
* fix: more test fixes, starting filter client
* fix: sigh. more references to remove.
* fix: fix dereferencing error
* fix: fix merge mess up
* fix: sigh. merge tool used tabs.
* fix: more peer manager tests needed fixing
---------
Co-authored-by: Hanno Cornelius <hanno@status.im>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
NagyZoltanPeter
gabrielmer
Simon-Pierre Vivier
Darshan K
Ivan FB
Ivan Folgueira Bande
Aaryamann Challani
Alvaro Revuelta
Álex Cabeza Romero