72 Commits

Author SHA1 Message Date
Simon-Pierre Vivier
682981f967 feat: remove Waku Sync 1.0 & Negentropy (#3185) 2024-11-29 09:09:41 -05:00
gabrielmer
f3af7fa37e chore: naming connectPeer procedure (#3157) 2024-10-29 18:37:07 +02:00
Darshan K
785cf2e9d9 refactor: wrap peer store (#3051)
Encapsulate peerstore with wakupeerstore
2024-09-27 18:16:46 +05:30
fryorcraken
723b009b20 chore: use submodule nph in CI to check lint (#3027) 2024-09-11 11:51:42 +10:00
gabrielmer
43bea3c476 chore: deprecating pubsub topic (#2997) 2024-09-10 15:07:12 -06:00
Ivan FB
5f2d87ec71 chore: Bump dependencies for v0.33 (#3017) 2024-09-09 10:45:14 +02:00
NagyZoltanPeter
3212459f77 chore: liteprotocoltester for simulation and for fleets (#2813)
* Added phase 2 - waku-simulatior integration in README.md

* Enhancement on statistics reports, added list of sent messages with hash, fixed latency calculations

* Enable standalonde running liteprotocoltester agains any waku network/fleet

* Fix missing env vars on run_tester_node.sh

* Adjustment on log levels, fix REST initialization

* Added standalon docker image build, fine tune duplicate detection and logging.

* Adjustments for waku-simulator runs

* Extended liteprotocoltester README.md with docker build

* Fix test inside docker service node connectivity failure

* Update apps/liteprotocoltester/README.md

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Explain minLatency calculation in code comment

---------

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-08-21 14:54:18 +02:00
gabrielmer
4b9bee99a8 chore: logging received message info via onValidated observer (#2973) 2024-08-19 14:13:28 +02:00
gabrielmer
1f3162ae5f avoid using the msg key in chronicles (#2970) 2024-08-14 16:40:08 +02:00
Simon-Pierre Vivier
301ce8068c feat: Nwaku Sync (#2403)
* feat: Waku Sync Protocol

* feat: state machine (#2656)

* feat: pruning storage mehcanism (#2673)

* feat: message transfer mechanism & tests (#2688)

* update docker files

* added ENR filed for sync & misc. fixes

* adding new sync range param & fixes

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
2024-08-13 07:27:34 -04:00
Sasha
1d850e43dc chore: return all connected peers from REST API (#2923)
* Remove the condition of gathering connected peers with relay and user req/resp protocols.
* Return PeerExchange protocol support of connected nodes with /admin/peers
* Added test for checking return of PeerExchange mounted protocol of connected peer by GET /admin/peers

---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2024-07-23 12:58:56 +02:00
gabrielmer
aa9c30655c chore: adding lint job to the CI (#2925) 2024-07-23 13:57:24 +03:00
NagyZoltanPeter
ca634ef3ba feat: DOS protection of non relay protocols - rate limit phase3 (#2897)
* DOS protection of non relay protocols - rate limit phase3:
- Enhanced TokenBucket to be able to add compensation tokens based on previous usage percentage,
- per peer rate limiter 'PeerRateLimier' applied on waku_filter_v2 with opinionated default of acceptable request rate
- Add traffic metrics to filter message push
- RequestRateLimiter added to combine simple token bucket limiting of request numbers but consider per peer usage over time and prevent some peers to over use the service
  (although currently rule violating peers will not be disconnected by this time only their requests will get not served)
- TimedMap utility created (inspired and taken from libp2p TimedCache) which serves as forgiving feature for peers had been overusing the service.
- Added more tests
- Fix rebase issues
- Applied new RequestRateLimiter for store and legacy_store and lightpush
* Incorporate review comments, typos, file/class naming and placement changes.
* Add issue link reference of the original issue with nim-chronos TokenBucket
* Make TimedEntry of TimedMap private and not mixable with similar named in libp2p
* Fix review comments, renamings, const instead of values and more comments.
2024-07-16 15:46:21 +02:00
Ivan FB
13316201f7 chore: Bump dependencies for v0.31.0 (#2885)
* bump_dependencies.md: add nim-results dependency
* change imports stew/results to results
* switching to Nim 2.0.8
* waku.nimble: reflect the requirement nim 1.6.0 to 2.0.8
  Adding --mm:refc as nim 2.0 enables a new garbage collector that we're
  not yet ready to support
* adapt waku code to Nim 2.0
* gcsafe adaptations because Nim 2.0 is more strict
2024-07-09 13:14:28 +02:00
Darshan K
7ad9722ecf chore: remove all pre-nim-1.6 deadcode from codebase (#2857) 2024-06-28 16:04:57 +05:30
gabrielmer
67c6b2142c chore: adding origin to peers admin endpoint (#2848) 2024-06-26 17:59:12 +02:00
Simon-Pierre Vivier
6ab1084d46 fix: invalid cursor returning messages (#2724)
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-05-27 10:54:10 -04:00
Simon-Pierre Vivier
2db1cf3fcb fix: store v3 bug fixes (#2718) 2024-05-23 08:01:52 -04:00
Ivan FB
044bbf8b40 standardize store types by using camel case instead of snake case (#2709) 2024-05-17 16:56:54 +02:00
Ivan FB
652fc172d4 chore: generic change to reduce the number of compilation warnings (#2696) 2024-05-16 22:29:11 +02:00
NagyZoltanPeter
e028362086 feat: Added flexible rate limit checks for store, legacy store and lightpush (#2668)
* Added flexible rate limit checks for store, legacy store and lightpush. Also added rate and traffic metrics.

* Fix chat2 after WakuLegacyStoreCodec rename

* Update waku/common/ratelimit.nim

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Update waku/common/ratelimit.nim

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Update waku/waku_store_legacy/protocol.nim

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Fix review findings, added limit to debug logs

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-05-09 20:07:49 +02:00
Simon-Pierre Vivier
3e51b9d904 fix: message cache removal crash (#2682) 2024-05-09 10:38:55 -04:00
Ivan FB
f65eead529 refactor: simplify app.nim and move discovery items to appropriate modules (#2657) 2024-05-01 21:13:08 +02:00
Simon-Pierre Vivier
db72e2b823 fix: store v3 validate cursor & remove messages (#2636) 2024-05-01 14:47:06 -04:00
Ivan FB
8f9bcef2c4 rest/store/types: contentTopic -> content_topic (#2646) 2024-04-29 16:19:07 +02:00
NagyZoltanPeter
ebcabd8ed0 chore: Enabling to use a full node for lightpush via rest api without lightpush client configured (#2626)
* Enabling to use a full node for lightpush via rest api without light push client configured
2024-04-26 12:42:47 +02:00
Simon-Pierre Vivier
665d9e3a06 feat: store v3 (#2431) 2024-04-25 09:09:52 -04:00
Ivan FB
963d79aee7 refactor: addition of waku_api/rest/builder.nim and reduce app.nim (#2623) 2024-04-24 15:59:50 +02:00
NagyZoltanPeter
daa88019d0 chore: Separation of node health and initialization state from rln_relay (#2612)
* Separation of node health and initialization state from rln_relay status. Make (only) health endpoint avail early and install others in the last stage of node setup.

* Proper json report from /health, adjusted and fixed test, added convenient script for checking node health

* Stop wakunode2 if configured rest server cannot be started

* Fix wakuRlnRelay protocol existence check

* Fix typo

* Removed unused imports from touched files.

* Added missing /health test for all
2024-04-23 18:53:18 +02:00
kaichao
4f63fa9f88 fix: create options api for cors preflight request (#2598) 2024-04-18 18:29:50 +08:00
Ivan FB
790b708d11 refactor: start moving discovery modules to waku/discovery (#2587) 2024-04-17 21:48:20 +02:00
NagyZoltanPeter
559531749b feat: Added simple, configurable rate limit for lightpush and store-query (#2390)
* feat: Added simple, configurable rate limit for lightpush and store-query
Adjust lightpush rest response to rate limit, added tests ann some fixes
Add rest store query test for rate limit checks and proper error response
Update apps/wakunode2/external_config.nim
Move chronos/tokenbucket to nwaku codebasee with limited and fixed feature set
Add meterics counter to lightpush rate limits

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
2024-04-15 15:28:35 +02:00
gabrielmer
d171add431 chore: bumping chronicles version (#2583) 2024-04-15 10:59:37 +03:00
Ivan FB
b92e67ef41 fix: rest store: content_topic -> contentTopic in the response (#2584) 2024-04-12 17:47:32 +02:00
Alvaro Revuelta
c432c1bfcc chore: remove deprecated legacy filter protocol (#2507)
* chore: remove deprecated legacy filter protocol

* fix: do not use legacy import in test

* fix: remove legacy test references

* fix: more test fixes, starting filter client

* fix: sigh. more references to remove.

* fix: fix dereferencing error

* fix: fix merge mess up

* fix: sigh. merge tool used tabs.

* fix: more peer manager tests needed fixing

---------

Co-authored-by: Hanno Cornelius <hanno@status.im>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2024-03-25 18:07:56 +00:00
richΛrd
6eaf94f323 fix(rest): filter/v2/subscriptions response (#2529) 2024-03-18 18:21:06 -04:00
Ivan FB
cf6298ca1f Generic re-style with nph 0.5.1 (#2396) 2024-03-16 00:08:47 +01:00
richΛrd
b42f2802c1 feat(rest): add support to ephemeral field (#2525) 2024-03-13 08:49:21 -04:00
Alvaro Revuelta
813d0b207c fix: enable autosharding in any cluster (#2505) 2024-03-13 10:58:13 +01:00
Aaryamann Challani
6d74aa08a9 chore(rln-relay-v2): wakunode testing + improvements (#2501)
* chore(rln-relay-v2): additional testing

* fix: bump librln to v0.4.2 for v2

* fix: catch possible error from the copyFrom

* ci: rename step title for rln-version
2024-03-12 16:20:30 +05:30
Ivan FB
1f2e5065d3 rest: rm openapi defs. they are in https://github.com/waku-org/waku-rest-api (#2520) 2024-03-08 19:43:55 +01:00
NagyZoltanPeter
70b7224336 Removed json-rpc leftovers (#2503) 2024-03-05 15:51:43 +01:00
Alvaro Revuelta
a8769955f0 chore: remove json rpc (#2416) 2024-02-29 23:35:27 +01:00
NagyZoltanPeter
7885ce0c9e chore: Implemented CORS handling for nwaku REST server (#2470)
* Add allowOrigin configuration for wakunode and WakuRestServer
Update nim-presto to the latest master that contains middleware support
Rework Rest Server in waku to utilize chronos' and presto's new middleware design and added proper CORS handling.
Added cors tests and fixes

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-02-29 09:48:14 +01:00
gabrielmer
9ed0dde494 feat: supporting meta field in WakuMessage (#2384) 2024-02-14 17:29:59 +02:00
Aaryamann Challani
3842584558 feat(rln-relay-v2): nonce/messageId manager (#2413)
* feat(rln-relay-v2): nonce/messageId manager

* fix: simplify
2024-02-13 10:18:02 +05:30
gabrielmer
b413122a73 feat: running validators in /relay/v1/auto/messages/{topic} (#2394) 2024-02-05 10:24:54 +02:00
gabrielmer
72c67363ec feat: running validators in /relay/v1/messages/{pubsubTopic} (#2373) 2024-02-01 18:16:10 +01:00
Ivan FB
7aa50c8b09 REST store: get msgs from self node when store is mounted and no peerAddr is passed (#2387)
A node that handles REST-Store requests normally acts as a 
Store-client and therefore it retrieved the messages from another
Store-node.
With these changes, we allow a node with Store mounted, to retrieve
its messages. In other words, the node can act as a Store-server of
its messages.

* test_rest_store.nim: add a new test to validate that the self-node can
retrieve its messages to the REST client.

* rest/store/client.nim: add new proc to allow making a GET store
request without peerAddr.

* rest/store/handle.nim: add logic to handle requests that don't
provide peerAddr but the self/local node has Store mounted. In this case,
the self/local node will retrieve its locally stored messages.

* waku_store/self_req_handler.nim: logic to handle "store" requests
allowing the REST-store node to act as a Store-server node. The
'self_req_handler.nim' helps to bypass the store protocol and directly
retrieve the messages from the local/self node. I added this logic in
a separate file from 'protocol.nim' because it doesn't participate in
any libp2p communication.

* waku_store/protocol.nim: make 'queryHandler' attribute public so that
it can be used from the 'self_req_handler.nim' module.
2024-01-31 17:43:59 +01:00
Simon-Pierre Vivier
cfd62e495c feat: shard aware relay peer management (#2332)
note that this feature is behind a config flag. `--relay-shard-manager`
2024-01-30 07:28:21 -05:00