687 Commits

Author SHA1 Message Date
gabrielmer
dfd42a7cb1
chore: updating dependencies for release 0.32.0 (#2971) 2024-08-14 16:38:31 +02:00
Simon-Pierre Vivier
2cc86c51da
feat: Nwaku Sync (#2403)
* feat: Waku Sync Protocol

* feat: state machine (#2656)

* feat: pruning storage mehcanism (#2673)

* feat: message transfer mechanism & tests (#2688)

* update docker files

* added ENR filed for sync & misc. fixes

* adding new sync range param & fixes

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
2024-08-13 07:27:34 -04:00
Ivan FB
4d47aa655f
chore: Optimize hash queries with lookup table (#2933)
* Upgrade Postgres schema to add messages_lookup table
* Perform optimized query for messageHash-only queries
2024-08-08 21:46:08 +02:00
f534549a1e
chore: replace statusim.net instances with status.im (#2941)
Use of `statusim.net` domain been deprecated since March:
https://github.com/status-im/infra-shards/commit/7df38c14

Also adjust test to match enr with multiaddresses.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-08-05 12:57:43 +02:00
Álex
a3fa175054
test(rln): Implement rln tests (#2639)
* Implement tests.
* Clean coding.
2024-08-02 16:43:22 +02:00
Ivan FB
ebda56dd90
lightpush better feedback in case the lightpush service node does not have peers (#2951) 2024-08-02 09:45:05 +02:00
Ivan FB
d4e8a0dab6
chore: Simplification of store legacy code (#2931) 2024-07-30 14:05:23 +02:00
Simon-Pierre Vivier
aed2a1130e
feat: store resume (#2919)
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-07-30 07:23:39 -04:00
Simon-Pierre Vivier
b4618f98ed
chore: add peer filtering by cluster for waku peer exchange (#2932) 2024-07-29 15:53:43 -04:00
Darshan K
08973add7e
fix: update and shift unittest (#2934)
* fix: update and shift location of unit test
2024-07-26 16:57:34 +05:30
Sasha
a29eca77a0
chore: return all connected peers from REST API (#2923)
* Remove the condition of gathering connected peers with relay and user req/resp protocols.
* Return PeerExchange protocol support of connected nodes with /admin/peers
* Added test for checking return of PeerExchange mounted protocol of connected peer by GET /admin/peers

---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2024-07-23 12:58:56 +02:00
gabrielmer
086cc8edd2
chore: adding lint job to the CI (#2925) 2024-07-23 13:57:24 +03:00
Darshan K
8d107b0ded
fix: handle rln-relay-message-limit (#2867)
* fix: enforcing rln-contract max message limit and resolve early
2024-07-22 22:28:45 +05:30
NagyZoltanPeter
ba418ab5ba
feat: DOS protection of non relay protocols - rate limit phase3 (#2897)
* DOS protection of non relay protocols - rate limit phase3:
- Enhanced TokenBucket to be able to add compensation tokens based on previous usage percentage,
- per peer rate limiter 'PeerRateLimier' applied on waku_filter_v2 with opinionated default of acceptable request rate
- Add traffic metrics to filter message push
- RequestRateLimiter added to combine simple token bucket limiting of request numbers but consider per peer usage over time and prevent some peers to over use the service
  (although currently rule violating peers will not be disconnected by this time only their requests will get not served)
- TimedMap utility created (inspired and taken from libp2p TimedCache) which serves as forgiving feature for peers had been overusing the service.
- Added more tests
- Fix rebase issues
- Applied new RequestRateLimiter for store and legacy_store and lightpush
* Incorporate review comments, typos, file/class naming and placement changes.
* Add issue link reference of the original issue with nim-chronos TokenBucket
* Make TimedEntry of TimedMap private and not mixable with similar named in libp2p
* Fix review comments, renamings, const instead of values and more comments.
2024-07-16 15:46:21 +02:00
Simon-Pierre Vivier
f54ba10bc7
chore(archive): archive and drivers refactor (#2761)
* queue driver refactor (#2753)
* chore(archive): archive refactor (#2752)
* chore(archive): sqlite driver refactor (#2754)
* chore(archive): postgres driver refactor (#2755)
* chore(archive): renaming & copies (#2751)
* posgres legacy: stop using the storedAt field
* migration script 6: we still need the id column
  The id column is needed because it contains the message digest
  which is used in store v2, and we need to keep support to
  store v2 for a while
* legacy archive: set target migration version to 6
* waku_node: try to use wakuLegacyArchive if wakuArchive is nil
* node_factory, waku_node: mount legacy and future store simultaneously
  We want the nwaku node to simultaneously support store-v2
  requests and store-v3 requests.
  Only the legacy archive is in charge of archiving messages, and the
  archived information is suitable to fulfill both store-v2 and
  store-v3 needs.
* postgres_driver: adding temporary code until store-v2 is removed

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan Folgueira Bande <ivansete@status.im>
2024-07-12 18:19:12 +02:00
Aaryamann Challani
1c9eb27415
fix(rln_keystore_generator): improve error handling for unrecoverable failure (#2881) 2024-07-10 19:12:49 +02:00
gabrielmer
e1518cf9ff
chore: deprecating named sharding (#2723) 2024-07-09 18:36:12 +03:00
Ivan FB
fd6a71cdd7
chore: Bump dependencies for v0.31.0 (#2885)
* bump_dependencies.md: add nim-results dependency
* change imports stew/results to results
* switching to Nim 2.0.8
* waku.nimble: reflect the requirement nim 1.6.0 to 2.0.8
  Adding --mm:refc as nim 2.0 enables a new garbage collector that we're
  not yet ready to support
* adapt waku code to Nim 2.0
* gcsafe adaptations because Nim 2.0 is more strict
2024-07-09 13:14:28 +02:00
Darshan K
8bfad3ab45
chore: refactor relative path to better absolute (#2861) 2024-07-06 00:03:38 +02:00
Ivan Folgueira Bande
519abbebf0
Merge branch 'release/v0.30' into update-master-from-release-v0.30 2024-07-02 10:16:40 +02:00
gabrielmer
cae0c7e378
chore: saving agent and protoVersion in peerStore (#2860) 2024-07-01 13:29:14 +02:00
Darshan K
31c632e427
chore: unit test for duplicate message push (#2852)
* chore: add unit test for testing duplicate message push with timedcache

* chore: update according to better naming convention

---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2024-06-28 18:16:06 +05:30
Darshan K
9bd8c33aee
chore: remove all pre-nim-1.6 deadcode from codebase (#2857) 2024-06-28 16:04:57 +05:30
Aaryamann Challani
05356ab97f
chore(rln-relay): add chain-id flag to wakunode and restrict usage if mismatches rpc provider (#2858) 2024-06-28 11:19:16 +02:00
Alvaro Revuelta
7b97c8003c
chore: add TWN parameters for RLNv2 (#2843) 2024-06-27 11:45:21 +02:00
gabrielmer
7205f95cff
chore: adding origin to peers admin endpoint (#2848) 2024-06-26 17:59:12 +02:00
Ivan FB
a05fa06919
chore: archive.nim - increase the max limit of content topics per query to 100 (#2846) 2024-06-26 12:24:15 +02:00
Aaryamann Challani
c56785327a
fix(rln-relay): clear nullifier log only if length is over max epoch gap (#2836) 2024-06-24 13:29:56 +02:00
Darshan K
94947a8504
chore: better descriptive log (#2826)
* chore: update logs with topic discription & debug msg

* chore: update unit according to error msg

* chore: update rest unit according to error msg

* chore: add content-topic with debug msg
2024-06-20 18:38:55 +05:30
Aaryamann Challani
a02832fe12
feat(rlnv2): clean fork of rlnv2 (#2828)
* chore(rlnv2): contract interface changes (#2770)
* fix: tests
* fix: remove stuint[32]
* chore(submodule): update zerokit submodule to v0.5.1 (#2782)
* fix: remove cond comp for lightpush test
* fix: ci and nonceManager
2024-06-20 14:55:50 +02:00
Aaryamann Challani
dd3c13582e
feat(rlnv2): clean fork of rlnv2 (#2828)
* chore(rlnv2): contract interface changes (#2770)
* fix: tests
* fix: remove stuint[32]
* chore(submodule): update zerokit submodule to v0.5.1 (#2782)
* fix: remove cond comp for lightpush test
* fix: ci and nonceManager
2024-06-20 11:35:21 +02:00
Akhil
0561e5bd45
feat: RLN proofs as a lightpush service (#2768) 2024-06-13 21:10:00 +04:00
gabrielmer
15d578ad87
added message to failed assert (#2805) 2024-06-12 22:27:10 +02:00
NagyZoltanPeter
f501436d21
Use random ports in rest tests instead of fixed ones (#2804) 2024-06-12 15:07:33 +02:00
gabrielmer
e72bb7e7b5
fix: increase on chain group manager starting balance (#2795) 2024-06-10 14:31:16 +02:00
Ivan FB
24661ea969
postgres partitions: ensure the partition ranges are o'clock (#2776)
Also, skip the error "partition already exists" because that happens
when multiple nodes interact with the same database.
2024-06-05 17:45:38 +02:00
gabrielmer
21a4b5bd4a
unifying clusterId to be uint16 (#2777) 2024-06-05 15:32:35 +02:00
Ivan FB
25a1aa9231
test_waku_store.nim: add logs to better analyse uncertain flaky tests (#2740) 2024-05-29 10:05:07 +02:00
Prem Chaitanya Prathi
57ecb3e090
fix: move postgres related tests under linux conditional 2024-05-28 17:24:22 +05:30
Simon-Pierre Vivier
a65b13fcb8
fix: invalid cursor returning messages (#2724)
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-05-27 10:54:10 -04:00
Simon-Pierre Vivier
4a6ec468db
fix: store v3 bug fixes (#2718) 2024-05-23 08:01:52 -04:00
Ivan FB
78132dc12e
chore: generic change to reduce the number of compilation warnings (#2696) 2024-05-16 22:29:11 +02:00
Ivan FB
f0f94412a0
test_client: simple sleep to try avoid macos CI test failures (#2707) 2024-05-16 18:04:04 +02:00
Ivan FB
5b1fbb26c4
test_client: add nil error handling after serverSwitch.start() clientSwitch.start() (#2703) 2024-05-15 12:36:17 +02:00
Akhil
9dfdfa2774
feat: Added message size check before relay for lightpush (#2695) 2024-05-15 14:13:13 +04:00
Ivan FB
840e012294
chore: move code from wakunode2 to a more generic place, waku (#2670)
* testlib/wakunode.nim: not use cluster-id == 1 to avoid test rln by default
2024-05-13 17:45:48 +02:00
Álex Cabeza Romero
6c3ad50455
test(sharding): Implement sharding tests (#2603)
* Implement sharding tests.
2024-05-13 17:43:14 +02:00
Álex Cabeza Romero
49dd6c1989
test(peer-and-connection-management): Implement tests (#2566)
* Implement peer and connection management tests.
* Fix multiple peers added on initialisation.
* Remove clusterId parameter from newTestWakuNode.
2024-05-13 17:25:44 +02:00
Ivan FB
8451cf8e1b
chore: Postgres enhance get oldest timestamp (#2687)
* postgres: consider also the existing paritions when getting oldest timestamp
* test_driver_postgres_query: adapt test to oldest timestamp
2024-05-10 18:31:01 +02:00
gabrielmer
a37c9ba911
fix: use await instead of waitFor in async tests (#2690) 2024-05-10 14:13:58 +02:00