635 Commits

Author SHA1 Message Date
NagyZoltanPeter
e028362086 feat: Added flexible rate limit checks for store, legacy store and lightpush (#2668)
* Added flexible rate limit checks for store, legacy store and lightpush. Also added rate and traffic metrics.

* Fix chat2 after WakuLegacyStoreCodec rename

* Update waku/common/ratelimit.nim

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Update waku/common/ratelimit.nim

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Update waku/waku_store_legacy/protocol.nim

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Fix review findings, added limit to debug logs

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-05-09 20:07:49 +02:00
Simon-Pierre Vivier
3e51b9d904 fix: message cache removal crash (#2682) 2024-05-09 10:38:55 -04:00
gabrielmer
42b250b9e9 adding wait after starting node to avoid segfault (#2686) 2024-05-09 11:31:58 +02:00
Simon-Pierre Vivier
9ae579b714 feat: store v3 return pubsub topics (#2676) 2024-05-08 15:35:56 -04:00
gabrielmer
9be3221b5d feat: supporting meta field in store (#2609) 2024-05-06 10:20:21 +02:00
Ivan FB
1d35ca970f refactor: big refactor to add waku component in libwaku instead of onlu waku node (#2658) 2024-05-03 14:07:15 +02:00
Simon-Pierre Vivier
db72e2b823 fix: store v3 validate cursor & remove messages (#2636) 2024-05-01 14:47:06 -04:00
Aaryamann Challani
44703f2608 fix(waku_keystore): sigsegv on different appInfo (#2654)
* fix(waku_keystore): sigsegv on different appInfo

* fix: field specific errors

* fix: more verbose error logs
2024-05-01 23:05:22 +05:30
Aaryamann Challani
8ada4e06f1 fix(rln-relay): persist metadata every batch during initial sync (#2649)
* fix(rln-relay): persist metadata every batch during initial sync

* fix: test

* Apply suggestions from code review

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* patch: isOkOr template

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-04-30 18:52:47 +05:30
Ivan FB
7bab843003 refactor: metrics server. Simplify app.nim module (#2650) 2024-04-30 15:07:17 +02:00
Aaryamann Challani
7dd7531a4a chore(rln-relay): resultify rln-relay 1/n (#2607)
* chore(rln-relay): resultify rln-relay 1/n

* fix: v2 too

* fix: for static group manager

* fix: cleanup, make PR digestable

* fix: remove resultified retry wrapper

* fix: cleanup

* fix: cleanup
2024-04-26 11:53:58 +02:00
Simon-Pierre Vivier
665d9e3a06 feat: store v3 (#2431) 2024-04-25 09:09:52 -04:00
Ivan FB
963d79aee7 refactor: addition of waku_api/rest/builder.nim and reduce app.nim (#2623) 2024-04-24 15:59:50 +02:00
NagyZoltanPeter
daa88019d0 chore: Separation of node health and initialization state from rln_relay (#2612)
* Separation of node health and initialization state from rln_relay status. Make (only) health endpoint avail early and install others in the last stage of node setup.

* Proper json report from /health, adjusted and fixed test, added convenient script for checking node health

* Stop wakunode2 if configured rest server cannot be started

* Fix wakuRlnRelay protocol existence check

* Fix typo

* Removed unused imports from touched files.

* Added missing /health test for all
2024-04-23 18:53:18 +02:00
Prem Chaitanya Prathi
9045af9363 fix: don't use WakuMessageSize in req/resp protocols (#2601)
* fix: don't use WakuMessageSize in req/resp protocols
2024-04-20 09:10:52 +05:30
Ivan FB
dc7d036074 refactor: move app.nim and networks_config.nim to waku/factory (#2608) 2024-04-19 20:03:36 +02:00
kaichao
4f63fa9f88 fix: create options api for cors preflight request (#2598) 2024-04-18 18:29:50 +08:00
Ivan FB
1dc7224c48 fix: node restart test issue (#2576)
* test_protocol.nim: enhance test reboot and connect

- Is not necessary to start the node if the switch object has been
already started
- Enable an existing "Relay can receive messages after reboot and
reconnect" test
- Explicit reconnect to peer in "Relay can receive messages after reboot
and reconnect" test

* tests/waku_relay/utils: avoid starting the proto again in newTestSwitch proc
With that, we avoid double start of the protocol.

* bump nim-libp2p
2024-04-18 11:20:39 +02:00
Ivan FB
790b708d11 refactor: start moving discovery modules to waku/discovery (#2587) 2024-04-17 21:48:20 +02:00
NagyZoltanPeter
559531749b feat: Added simple, configurable rate limit for lightpush and store-query (#2390)
* feat: Added simple, configurable rate limit for lightpush and store-query
Adjust lightpush rest response to rate limit, added tests ann some fixes
Add rest store query test for rate limit checks and proper error response
Update apps/wakunode2/external_config.nim
Move chronos/tokenbucket to nwaku codebasee with limited and fixed feature set
Add meterics counter to lightpush rate limits

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
2024-04-15 15:28:35 +02:00
Alvaro Revuelta
7a36ce836d fix: fix regresion + remove deprecated flag (#2556) 2024-03-26 19:44:55 +01:00
Sergei Tikhomirov
e7fb8b5fb1 feat(incentivization): add codec for eligibility proof and status (#2419)
* incentivization: add codec for eligibility proofs

* add codec for eligibility proof and eligibility status

* address minor comments

* make status code mandatory in eligibility status
2024-03-26 18:25:42 +01:00
Alvaro Revuelta
c432c1bfcc chore: remove deprecated legacy filter protocol (#2507)
* chore: remove deprecated legacy filter protocol

* fix: do not use legacy import in test

* fix: remove legacy test references

* fix: more test fixes, starting filter client

* fix: sigh. more references to remove.

* fix: fix dereferencing error

* fix: fix merge mess up

* fix: sigh. merge tool used tabs.

* fix: more peer manager tests needed fixing

---------

Co-authored-by: Hanno Cornelius <hanno@status.im>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2024-03-25 18:07:56 +00:00
Anton Iakimov
40b687c0a5 chore: switch wakuv2 to waku fleet (#2519)
See https://github.com/status-im/infra-nim-waku/issues/91
2024-03-20 16:28:00 +01:00
Álex Cabeza Romero
279d0dfa7f fix(2491): Fix metadata protocol disconnecting light nodes (#2533)
* Fix metadata protocol disconnecting light nodes.
* Implement test cases.
2024-03-19 16:18:52 +01:00
richΛrd
6eaf94f323 fix(rest): filter/v2/subscriptions response (#2529) 2024-03-18 18:21:06 -04:00
Ivan FB
588530d5c7 chore: Better postgres duplicate insert (#2535)
* postgres_driver: add ON CONFLICT DO NOTHING in the insert statement
* test_driver_postgres: adapt test to ON CONFLICT DO NOTHING
  The insert does not fail when duplicate, it returns a positive response
  when doing 'put' of a duplicated row. The test is adapted so that
  we just check that the number of messages doesn't grow after
  trying to add a duplicated row.
2024-03-18 15:59:45 +01:00
Ivan FB
cf6298ca1f Generic re-style with nph 0.5.1 (#2396) 2024-03-16 00:08:47 +01:00
Álex Cabeza Romero
297838b145 test(discv5): Implement discv5 tests (#2487)
* Implement discv5 tests.
2024-03-14 19:01:13 +01:00
Álex Cabeza Romero
8e13bfbb65 test(peer-exchange): Implement peer exchange tests (#2464)
* Implement peer exchange tests.
* Refactor, and remove duplicated tests.
* feat(wakunode): Resultify fetch peer exchange peers (#2486)
2024-03-14 17:48:09 +01:00
richΛrd
b42f2802c1 feat(rest): add support to ephemeral field (#2525) 2024-03-13 08:49:21 -04:00
Alvaro Revuelta
813d0b207c fix: enable autosharding in any cluster (#2505) 2024-03-13 10:58:13 +01:00
gabrielmer
72d9ed5b0b chore: adding node factory tests (#2524) 2024-03-12 10:12:44 -05:00
Simon-Pierre Vivier
430708ccc6 feat: archive update for store v3 (#2451) 2024-03-12 07:51:03 -04:00
Aaryamann Challani
6d74aa08a9 chore(rln-relay-v2): wakunode testing + improvements (#2501)
* chore(rln-relay-v2): additional testing

* fix: bump librln to v0.4.2 for v2

* fix: catch possible error from the copyFrom

* ci: rename step title for rln-version
2024-03-12 16:20:30 +05:30
gabrielmer
92051e95d2 chore: migrating logic from wakunode2.nim to node_factory.nim (#2504) 2024-03-08 16:46:42 -06:00
Aaryamann Challani
d2f30df8c7 fix(rln-relay): handle empty metadata returned by getMetadata proc (#2516)
* fix(rln-relay): silence error on startup when metadata is not found

* chore: fix fetching value from option

* fix: clarity of returned opt
2024-03-08 19:36:22 +05:30
Ivan FB
132bb0bbf2 feat: Postgres partition implementation (#2506)
* postgres: first step to implement partition management
* postgres_driver: use of times.now().toTime().toUnix() instead of Moment.now()
* postgres migrations: set new version to 2
* test_driver_postgres: use of assert instead of require and avoid using times.now()
* postgres_driver: better implementation of the reset method with partitions
* Remove createMessageTable, init, and deleteMessageTable procs
* postgres: ensure we use the version 15.4 in tests
* postgres_driver.nim: enhance debug logs partition addition
* ci.yml: ensure logs are printed without colors
* postgres_driver: starting the loop factory in an asynchronous task
* postgres_driver: log partition name and size when removing a partition
2024-03-06 20:50:22 +01:00
Aaryamann Challani
dc6381264f fix(rln-relay): make nullifier log abide by epoch ordering (#2508)
* fix(rln-relay): nullifier log abide by epoch ordering

* fix: cleaner hasKey method, test

* chore: idiomatic usage of results, error handling

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-03-06 23:59:07 +05:30
NagyZoltanPeter
70b7224336 Removed json-rpc leftovers (#2503) 2024-03-05 15:51:43 +01:00
Aaryamann Challani
d3e01495b8 chore(rln-relay-v2): added tests for static rln-relay-v2 (#2484)
* chore(rln-relay-v2): added tests for onchain rln-relay-v2

* chore(rln-relay): added tests for static rln-relay-v2

* Update waku/waku_rln_relay/group_manager/static/group_manager.nim

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* fix: split lines

* fix: remove redundant require

* fix: remove redundant require

* fix: bad await

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-03-04 17:41:33 +05:30
gabrielmer
8cf2f78b6c chore: moving node initialization code to node_factory.nim (#2479) 2024-03-02 18:59:53 -06:00
Simon-Pierre Vivier
4638756aef fix: time on 32 bits architecture (#2492)
authored-by: Emil Ivanichkov <emil.ivanichkov@gmail.com>
2024-03-01 07:58:45 -05:00
Ivan FB
75521122a4 chore: Postgres migrations (#2477)
* Add postgres_driver/migrations.nim
* Postgres and archive logic adaptation to the migration implementation
* libwaku: adapt node_lifecycle_request.nim to migration refactoring
* test_app.nim: add more detail for test that only fails in CI
* postgres migrations: store the migration scripts inside the resulting wakunode binary instead of external .sql files.
2024-03-01 12:05:27 +01:00
Aaryamann Challani
545d9aee99 chore(rln-relay-v2): added tests for onchain rln-relay-v2 (#2482)
* chore(rln-relay-v2): added tests for onchain rln-relay-v2

* Update tests/waku_rln_relay/test_rln_group_manager_onchain.nim

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-03-01 14:15:40 +05:30
Alvaro Revuelta
a8769955f0 chore: remove json rpc (#2416) 2024-02-29 23:35:27 +01:00
NagyZoltanPeter
7885ce0c9e chore: Implemented CORS handling for nwaku REST server (#2470)
* Add allowOrigin configuration for wakunode and WakuRestServer
Update nim-presto to the latest master that contains middleware support
Rework Rest Server in waku to utilize chronos' and presto's new middleware design and added proper CORS handling.
Added cors tests and fixes

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-02-29 09:48:14 +01:00
Alvaro Revuelta
cd9ebde02f chore: remove rln epoch hardcoding (#2483) 2024-02-28 17:19:20 +01:00
Ivan FB
0887a344a4 refactor: new proc to foster different size retention policy implementations (#2463)
* new proc to foster different size retention policy implementations
  The new proc, decreaseDatabaseSize, will have different implementations
  per each driver. For example, in future commits we will implement a size
  retention policy thanks to partitions management, in Postgres.
* RetentionPolicy: use of new instead of init for ref object types
* waku_archive: fix signatures in decreaseDatabaseSize methods
* retention_policy_size: minor cleanup of comments and imports
2024-02-22 16:55:37 +01:00
Aaryamann Challani
2f89fdeee9 chore(rln-relay): use anvil instead of ganache in onchain tests (#2449)
* chore(rln-relay): use anvil instead of ganache in onchain tests

* fix: incl rustup in makefile
2024-02-22 16:59:13 +05:30