688 Commits

Author SHA1 Message Date
Ivan FB
236547ec7d chore: Better timing and requestId detail for slower store db queries (#2994)
* Better timing and requestId detail for store db queries slower than two seconds
* Adapt tests and client to allow sending custom store requestId
2024-08-29 22:56:14 +02:00
gabrielmer
3f641dff60 chore: deprecating protected topics in favor of protected shards (#2983) 2024-08-19 12:56:22 +02:00
gabrielmer
90b4dc89ff chore: rename NsPubsubTopic (#2974) 2024-08-19 11:29:35 +02:00
gabrielmer
f094c671ca chore: updating dependencies for release 0.32.0 (#2971) 2024-08-14 16:38:31 +02:00
Simon-Pierre Vivier
301ce8068c feat: Nwaku Sync (#2403)
* feat: Waku Sync Protocol

* feat: state machine (#2656)

* feat: pruning storage mehcanism (#2673)

* feat: message transfer mechanism & tests (#2688)

* update docker files

* added ENR filed for sync & misc. fixes

* adding new sync range param & fixes

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
2024-08-13 07:27:34 -04:00
Ivan FB
696587fdac chore: Optimize hash queries with lookup table (#2933)
* Upgrade Postgres schema to add messages_lookup table
* Perform optimized query for messageHash-only queries
2024-08-08 21:46:08 +02:00
fcc11a7cd9 chore: replace statusim.net instances with status.im (#2941)
Use of `statusim.net` domain been deprecated since March:
https://github.com/status-im/infra-shards/commit/7df38c14

Also adjust test to match enr with multiaddresses.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-08-05 12:57:43 +02:00
Álex
1ab665ce2c test(rln): Implement rln tests (#2639)
* Implement tests.
* Clean coding.
2024-08-02 16:43:22 +02:00
Ivan FB
1ce87c49a8 lightpush better feedback in case the lightpush service node does not have peers (#2951) 2024-08-02 09:45:05 +02:00
Ivan FB
f9552e133b chore: Simplification of store legacy code (#2931) 2024-07-30 14:05:23 +02:00
Simon-Pierre Vivier
2134ad76b4 feat: store resume (#2919)
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-07-30 07:23:39 -04:00
Simon-Pierre Vivier
7fdabe5ad9 chore: add peer filtering by cluster for waku peer exchange (#2932) 2024-07-29 15:53:43 -04:00
Darshan K
48a79d3012 fix: update and shift unittest (#2934)
* fix: update and shift location of unit test
2024-07-26 16:57:34 +05:30
Sasha
1d850e43dc chore: return all connected peers from REST API (#2923)
* Remove the condition of gathering connected peers with relay and user req/resp protocols.
* Return PeerExchange protocol support of connected nodes with /admin/peers
* Added test for checking return of PeerExchange mounted protocol of connected peer by GET /admin/peers

---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2024-07-23 12:58:56 +02:00
gabrielmer
aa9c30655c chore: adding lint job to the CI (#2925) 2024-07-23 13:57:24 +03:00
Darshan K
ad6f6c6bac fix: handle rln-relay-message-limit (#2867)
* fix: enforcing rln-contract max message limit and resolve early
2024-07-22 22:28:45 +05:30
NagyZoltanPeter
ca634ef3ba feat: DOS protection of non relay protocols - rate limit phase3 (#2897)
* DOS protection of non relay protocols - rate limit phase3:
- Enhanced TokenBucket to be able to add compensation tokens based on previous usage percentage,
- per peer rate limiter 'PeerRateLimier' applied on waku_filter_v2 with opinionated default of acceptable request rate
- Add traffic metrics to filter message push
- RequestRateLimiter added to combine simple token bucket limiting of request numbers but consider per peer usage over time and prevent some peers to over use the service
  (although currently rule violating peers will not be disconnected by this time only their requests will get not served)
- TimedMap utility created (inspired and taken from libp2p TimedCache) which serves as forgiving feature for peers had been overusing the service.
- Added more tests
- Fix rebase issues
- Applied new RequestRateLimiter for store and legacy_store and lightpush
* Incorporate review comments, typos, file/class naming and placement changes.
* Add issue link reference of the original issue with nim-chronos TokenBucket
* Make TimedEntry of TimedMap private and not mixable with similar named in libp2p
* Fix review comments, renamings, const instead of values and more comments.
2024-07-16 15:46:21 +02:00
Simon-Pierre Vivier
d60ff3e0e6 chore(archive): archive and drivers refactor (#2761)
* queue driver refactor (#2753)
* chore(archive): archive refactor (#2752)
* chore(archive): sqlite driver refactor (#2754)
* chore(archive): postgres driver refactor (#2755)
* chore(archive): renaming & copies (#2751)
* posgres legacy: stop using the storedAt field
* migration script 6: we still need the id column
  The id column is needed because it contains the message digest
  which is used in store v2, and we need to keep support to
  store v2 for a while
* legacy archive: set target migration version to 6
* waku_node: try to use wakuLegacyArchive if wakuArchive is nil
* node_factory, waku_node: mount legacy and future store simultaneously
  We want the nwaku node to simultaneously support store-v2
  requests and store-v3 requests.
  Only the legacy archive is in charge of archiving messages, and the
  archived information is suitable to fulfill both store-v2 and
  store-v3 needs.
* postgres_driver: adding temporary code until store-v2 is removed

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan Folgueira Bande <ivansete@status.im>
2024-07-12 18:19:12 +02:00
Aaryamann Challani
6ef45927f1 fix(rln_keystore_generator): improve error handling for unrecoverable failure (#2881) 2024-07-10 19:12:49 +02:00
gabrielmer
1071ffc6c8 chore: deprecating named sharding (#2723) 2024-07-09 18:36:12 +03:00
Ivan FB
13316201f7 chore: Bump dependencies for v0.31.0 (#2885)
* bump_dependencies.md: add nim-results dependency
* change imports stew/results to results
* switching to Nim 2.0.8
* waku.nimble: reflect the requirement nim 1.6.0 to 2.0.8
  Adding --mm:refc as nim 2.0 enables a new garbage collector that we're
  not yet ready to support
* adapt waku code to Nim 2.0
* gcsafe adaptations because Nim 2.0 is more strict
2024-07-09 13:14:28 +02:00
Darshan K
8da2a9c0a5 chore: refactor relative path to better absolute (#2861) 2024-07-06 00:03:38 +02:00
Ivan Folgueira Bande
b574636e72 Merge branch 'release/v0.30' into update-master-from-release-v0.30 2024-07-02 10:16:40 +02:00
gabrielmer
3e8094a4ef chore: saving agent and protoVersion in peerStore (#2860) 2024-07-01 13:29:14 +02:00
Darshan K
d41280cc7a chore: unit test for duplicate message push (#2852)
* chore: add unit test for testing duplicate message push with timedcache

* chore: update according to better naming convention

---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2024-06-28 18:16:06 +05:30
Darshan K
7ad9722ecf chore: remove all pre-nim-1.6 deadcode from codebase (#2857) 2024-06-28 16:04:57 +05:30
Aaryamann Challani
e093af4c12 chore(rln-relay): add chain-id flag to wakunode and restrict usage if mismatches rpc provider (#2858) 2024-06-28 11:19:16 +02:00
Alvaro Revuelta
393d65faf9 chore: add TWN parameters for RLNv2 (#2843) 2024-06-27 11:45:21 +02:00
gabrielmer
67c6b2142c chore: adding origin to peers admin endpoint (#2848) 2024-06-26 17:59:12 +02:00
Ivan FB
75ed5d14e3 chore: archive.nim - increase the max limit of content topics per query to 100 (#2846) 2024-06-26 12:24:15 +02:00
Aaryamann Challani
ec3d02a028 fix(rln-relay): clear nullifier log only if length is over max epoch gap (#2836) 2024-06-24 13:29:56 +02:00
Darshan K
944b044e93 chore: better descriptive log (#2826)
* chore: update logs with topic discription & debug msg

* chore: update unit according to error msg

* chore: update rest unit according to error msg

* chore: add content-topic with debug msg
2024-06-20 18:38:55 +05:30
Aaryamann Challani
aa16002a4e feat(rlnv2): clean fork of rlnv2 (#2828)
* chore(rlnv2): contract interface changes (#2770)
* fix: tests
* fix: remove stuint[32]
* chore(submodule): update zerokit submodule to v0.5.1 (#2782)
* fix: remove cond comp for lightpush test
* fix: ci and nonceManager
2024-06-20 14:55:50 +02:00
Aaryamann Challani
7e4f18cda7 feat(rlnv2): clean fork of rlnv2 (#2828)
* chore(rlnv2): contract interface changes (#2770)
* fix: tests
* fix: remove stuint[32]
* chore(submodule): update zerokit submodule to v0.5.1 (#2782)
* fix: remove cond comp for lightpush test
* fix: ci and nonceManager
2024-06-20 11:35:21 +02:00
Akhil
0b97106cbe feat: RLN proofs as a lightpush service (#2768) 2024-06-13 21:10:00 +04:00
gabrielmer
767e89d5f1 added message to failed assert (#2805) 2024-06-12 22:27:10 +02:00
NagyZoltanPeter
641aa48696 Use random ports in rest tests instead of fixed ones (#2804) 2024-06-12 15:07:33 +02:00
gabrielmer
49372441c5 fix: increase on chain group manager starting balance (#2795) 2024-06-10 14:31:16 +02:00
Ivan FB
21a96cea7d postgres partitions: ensure the partition ranges are o'clock (#2776)
Also, skip the error "partition already exists" because that happens
when multiple nodes interact with the same database.
2024-06-05 17:45:38 +02:00
gabrielmer
3fd715cfc2 unifying clusterId to be uint16 (#2777) 2024-06-05 15:32:35 +02:00
Ivan FB
8a51db68bd test_waku_store.nim: add logs to better analyse uncertain flaky tests (#2740) 2024-05-29 10:05:07 +02:00
Prem Chaitanya Prathi
b13a11ad7c fix: move postgres related tests under linux conditional 2024-05-28 17:24:22 +05:30
Simon-Pierre Vivier
6ab1084d46 fix: invalid cursor returning messages (#2724)
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-05-27 10:54:10 -04:00
Simon-Pierre Vivier
2db1cf3fcb fix: store v3 bug fixes (#2718) 2024-05-23 08:01:52 -04:00
Ivan FB
652fc172d4 chore: generic change to reduce the number of compilation warnings (#2696) 2024-05-16 22:29:11 +02:00
Ivan FB
103ed86021 test_client: simple sleep to try avoid macos CI test failures (#2707) 2024-05-16 18:04:04 +02:00
Ivan FB
6df7df2bb2 test_client: add nil error handling after serverSwitch.start() clientSwitch.start() (#2703) 2024-05-15 12:36:17 +02:00
Akhil
41cdb92be7 feat: Added message size check before relay for lightpush (#2695) 2024-05-15 14:13:13 +04:00
Ivan FB
0b2859ed5a chore: move code from wakunode2 to a more generic place, waku (#2670)
* testlib/wakunode.nim: not use cluster-id == 1 to avoid test rln by default
2024-05-13 17:45:48 +02:00
Álex Cabeza Romero
d3a2c4e76d test(sharding): Implement sharding tests (#2603)
* Implement sharding tests.
2024-05-13 17:43:14 +02:00