58 Commits

Author SHA1 Message Date
Ivan FB
c738841d43 chore: message.nim - set max message size to 150KiB according to spec (#2298)
* message.nim: set max message size to 150KiB according to spec

Using KiB instead of KB because that seems more aligned with
the actual default defined in nim-libp2p (1024 * 1024)

Spec details: https://rfc.vac.dev/spec/64/#message-size

* test_protocol.nim: align test to current WakuMessage limit
* test_waku_client.nim: adapt test to MaxWakuMessageSize change
* make maxMessageSize configurable for wakunode2
* wakunode2 app now accepts max-num-bytes-msg-size with KiB, KB, or B units
* testlib/wakunode.nim: set maxMessageSize: "1024 KiB"
* test_waku_client.nim: remove duplicate check in "Valid Payload Sizes"
* set DefaultMaxWakuMessageSizeStr as the only source of truth
* external_config.nim: rename max-num-bytes-msg-size -> max-msg-size
2024-01-03 13:11:50 +01:00
Ivan FB
ce567acb62 ip colocation is parameterizable. If set to 0, it is disabled (#2323)
The "ip colocation" concept refers to the maximum allowed peers
from the same IP address. For example, we allow disabling this limit when the
node works behind a reverse proxy.
2024-01-02 14:01:18 +01:00
Ivan FB
cb59623466 fix: Revert "feat: shard aware peer management (#2151)" (#2312)
This reverts commit dc1d6ce4bf7390e23b73d96634ff87ca9341e129.

We need to revert this commit because
the waku-simulator stopped working. i.e. the nodes couldn't establish
connections among them: 054ba9e33f

Also, the following js-waku test fails due to this commit:
"same cluster, different shard: nodes connect"

* waku_lightpush/protocol.nim: minor changes to make it compile after revert
2023-12-20 15:23:41 +01:00
Ivan FB
7c692cc313 chore: vendor bump for 0.23.0 (#2274)
* on_chain/group_manager: use .async: (raises:[Exception]).
* bump nim-dnsdisc
* update nim-chronos to the latest state
* chat2.nim: catch any possible exception when stopping
* chat2bridge.nim: make it to compile after vendor bump
* ValidIpAddress (deprecated) -> IpAddress
* vendor/nim-libp2p additional bump
* libwaku: adapt to vendor bump
* testlib/wakunode.nim: adapt to vendor bump (ValidIpAddress -> IpAddress)
* waku_node: avoid throwing any exception from stop*(node: WakuNode)
* test_confutils_envvar.nim: ValidIpAddress -> IpAddress
* test_jsonrpc_store: capture exception
* test_rln*: handling exceptions
* adaptation to make test_rln_* to work properly
* signature enhancement of group_manager methods
2023-12-14 07:16:39 +01:00
Simon-Pierre Vivier
dc1d6ce4bf feat: shard aware peer management (#2151) 2023-12-07 06:48:28 -05:00
Simon-Pierre Vivier
ffc39e1f55 chore(rest): refactor message cache (#2221) 2023-11-28 07:21:41 -05:00
Ivan FB
75cbea86cb chore: postres driver allow setting the max number of connection from a parameter (#2246)
* postres driver: allow setting the max number of connections from a parameter
2023-11-24 16:21:22 +01:00
Prem Chaitanya Prathi
76fa222816 fix: admin REST API to be enabled only if config is set (#2218) 2023-11-24 14:43:20 +05:30
Simon-Pierre Vivier
0be13a356f chore: mics. improvements to cluster id and shards setup (#2187) 2023-11-21 15:15:39 -05:00
Simon-Pierre Vivier
5078ae2430 feat: REST APIs discovery handlers (#2109) 2023-10-27 15:43:54 -04:00
NagyZoltanPeter
b6ea215d71 Pull new version of nim-presto that implements RestServer' new error handler callback (#2144)
Added rest request error handler to capture calls on not installed endpoints
better, more descriptive error message returned.
2023-10-27 16:31:57 +02:00
gabrielmer
f7ed781257 feat: implementing port 0 support (#2125) 2023-10-27 10:11:47 +03:00
NagyZoltanPeter
9448b343d2 Fix wrong install of filter rest api - now installed when using filter client mode and filter client protocol is mounted correctly (#2133) 2023-10-18 11:47:47 +02:00
NagyZoltanPeter
577ab0db72 feat: /admin rest api endpoint (#2094)
/admin rest api implementation and tests
* open api doc
* Add rest admin test to all tests
* Enrich /admin get peers interface, group protocols by peers in response
2023-10-05 14:00:09 +02:00
Simon-Pierre Vivier
9cfde68fdf feat: Autosharding API for RELAY subscriptions (#1983) 2023-09-26 07:33:52 -04:00
Simon-Pierre Vivier
091cb5d899 refactor: Moved external APIs out of node (#2069) 2023-09-22 09:36:46 -04:00
NagyZoltanPeter
971b59c44a feat: lightpush rest api (#2052)
RestApi Lightpush endpoint implemented
* Openapi definition for lightpush rest api
* Update waku/node/rest/lightpush/handlers.nim
* Fix install handler naming, added negative test cases and fixes for restapi lightpush
* Fix error handling in lightpush rest handler
* Fix main success case - relay message that comes with pushRequest
* Fix rest relay serdes test with RelayWakuMessage validation changes
* Checking response message tests
2023-09-22 13:46:55 +02:00
NagyZoltanPeter
dd86da3247 feat: HTTP REST API: Filter support v2 (#1890)
Filter v2 rest api support implemented 
Filter rest api documentation updated with v1 and v2 interface support.
Separated legacy filter rest interface
Fix code and tests of v2 Filter rest api
Filter v2 message push test added
Applied autoshard to Filter V2
Redesigned FilterPushHandling, code style, catch up apps and tests with filter v2 interface changes
Rename of FilterV1SubscriptionsRequest to FilterLegacySubscribeRequest, fix broken chat2 app, fix tests
Changed Filter v2 push handler subscription to simple register
Separate node's filterUnsubscribe and filterUnsubscribeAll
2023-09-14 21:28:57 +02:00
Aaryamann Challani
75fd99e5a4 feat(rln-relay): removed rln from experimental 🚀 (#2001)
* feat(rln-relay): removed rln from experimental 🚀

* fix(waku_node): if rln-relay is nil, return true

* chore(tests): mark rest health as skipped
2023-09-11 12:02:31 +05:30
NagyZoltanPeter
d7cd41d78c feat: Rest endoint /health for rln (#2011)
* Rest endoint /health added

* Remove dev-debug echo

* Changed not ready message

* Update waku/node/rest/health/handlers.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* Various fixes on PR foundings, added comments

---------

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
2023-09-08 11:19:47 +02:00
Ivan Folgueira Bande
399c2bca1e Adding healtcheck and reconnection mechanism to the postgres archive driver (#1997)
It starts an asynchronous infinite task that checks the connectivity
with the database. In case of error, the postgres_healthcheck task
tries to reconnect for a while, and if it determines that the connection
cannot be resumed, then it invokes a callback indicating that
situation. For the case of the `wakunode2` app, this callback
quits the application itself and adds a log trace indicating
the connectivity issue with the database.
2023-09-06 19:16:37 +02:00
Aaryamann Challani
7c4ce7361b chore(rln-relay): clean up nullifier table every MaxEpochGap (#1994) 2023-09-06 13:48:02 +05:30
Simon-Pierre Vivier
d4decc2fc5 fix: --topic should be ignore when using --pubsub-topic or --content-topic (#1977) 2023-08-31 16:13:45 -04:00
Aaryamann Challani
eeff44c9f0 fix(rln-relay): modify keystore credentials logic (#1956)
* fix(rln-relay): modify keystore credentials logic

fix: bump version

* Update waku/waku_rln_relay/group_manager/on_chain/group_manager.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* Update tests/waku_rln_relay/test_waku_rln_relay.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* Update waku/waku_keystore/protocol_types.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* fix: greatly improve error handling

* fix: display proc and appropriate assert

---------

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
2023-08-29 17:46:21 +05:30
Simon-Pierre Vivier
0f0031eba9 feat(discv5): topic subscriptions update discv5 filter predicate (#1918) 2023-08-23 11:50:59 -04:00
Simon-Pierre Vivier
a3538599aa feat: topic subscriptions updates discv5 ENR (#1875) 2023-08-23 09:53:17 -04:00
Aaryamann Challani
de0a40972b fix(rln-relay): waitFor startup, otherwise valid proofs will be marked invalid (#1920) 2023-08-22 17:47:06 +05:30
Aaryamann Challani
a2519906c6 fix(rln-relay): remove registration capability (#1916) 2023-08-22 11:49:18 +05:30
Alvaro Revuelta
6320ffb28a chore(rln): run rln in all relay pubsubtopics + remove cli flags (#1917) 2023-08-21 08:55:34 +02:00
Simon-Pierre Vivier
51fee9d744 feat: update various protocols to autoshard (#1857)
* feat: update FILTER & LIGHTPUSH to autoshard
2023-08-17 08:11:18 -04:00
Simon-Pierre Vivier
1be8171fde fix: bring back default topic in config (#1902)
* fix: bring back default topic in config

* fix

* Fix a bug

* Fix enr creation
2023-08-11 17:28:24 +02:00
Hanno Cornelius
5e2e1b6bd0 chore: remove references to v2 (#1898)
* chore: remove references to v2

* fix: lingering rln-relay import path
2023-08-09 18:11:50 +01:00
Hanno Cornelius
0acf71176b chore: remove Waku v1 and wakubridge code (#1874)
* chore: remove v1 code

* chore: deprecate support for v1 compatibility from JSON-RPC API

* chore: remove v1 tests from JSON-RPC suite

* chore: remove wakubridge code

* chore: remove v1 compatibility functions

* chore: remove support for v1 payloads from chat2 app

* chore: remove v1 from CI processes

* fix: lingering references to private API functions

* fix: remove v1 compat from chat2

* fix: some more lingering references in tests
2023-08-07 15:11:46 +01:00
NagyZoltanPeter
cdc5423a86 feat: Rest API interface for legacy (v1) filter service. (#1851)
* Added Rest API interface for legacy (v1) filter service with tests.
2023-08-04 11:34:22 +02:00
Simon-Pierre Vivier
ac8ab37e7b feat: autosharding content topics in config (#1856)
- added content topic to config.
- updated ENR building to include content topics.
2023-08-01 16:01:44 -04:00
Aaryamann Challani
baaa9c7ec2 chore(rln-relay): verify proofs based on bandwidth usage (#1844)
* chore(rln-relay): Verify proofs based on bandwidth usage

* fix(rln-relay): make default threshold 0, for backwards compat

* fix(rln-relay): add unit test for bandwidth cutoff

* fix(rln-relay): rlnRelayBandwidthTreshold option
2023-07-07 17:28:37 +05:30
Alvaro Revuelta
fb74863f32 feat: limit relay connections below max conns (#1813) 2023-07-04 13:31:18 +02:00
Ivan Folgueira Bande
94db4b081f chore(refactor): Move record creation & fix libwaku compilation (#1833)
* Move record creation & fix libwaku

* app,external_config,internal_config,libwaku,sharding: refactorgin to make it compile

---------

Co-authored-by: SionoiS <simvivier@status.im>
2023-06-29 21:59:53 +02:00
Ivan Folgueira Bande
b5920a5c24 refactor of internal_config to avoid passing a 'WakuNodeConf' item (#1832)
This change is needed so that the `libwaku.nim` code gets simpler. The
`libwaku.nim` can't import the 'WakuNodeConf' type because this is
related to the wakunode2 app, and the library shouldn't know about it.
2023-06-29 11:01:14 +02:00
Simon-Pierre Vivier
ea5af1a935 chore: discv5 re-org clean-up (#1823)
- Remove discv5 from node and it's builder
- Require a record to init a node (was previously using discv5 record).
- Fix various tests that assumed nodes with discv5.
- Fix ENR creation
- Fix wakubridge
- Fix networkmonitor
2023-06-28 08:57:10 -04:00
Ivan Folgueira Bande
a7c775da65 fix(discv5): Fixing issue that prevented the wakunode2 from starting (#1829)
* Fixing issue that prevented the wakunode2 from starting

The issue was introduced in PR#1818.

Before this commit, the `wakunode2` app crashed with the next error:

ERR 2023-06-27 15:57:27.268+00:00 5/7 Starting node and protocols failed
topics="wakunode main" tid=1 file=wakunode2.nim:92 error="failed to
start waku discovery v5: "

* fix tests accordingly

* publisher.nim, subscriber.nim: fix
2023-06-27 15:16:59 -04:00
Simon-Pierre Vivier
00a818b96c chore: move discv5 out of node. (#1818)
- Refactor discv5 start, stop & loop.
- Fix tests.
2023-06-27 09:50:11 -04:00
Ivan Folgueira Bande
32ba79a920 refactor(archive): Moving waku archive logic from app.nim to the archive module (#1817)
* Refactoring the Waku Archive. Simplifying the app.nim

This change is needed to accommodate the further PRs where we will integrate Postgres in `wakunode2`.
2023-06-27 13:24:31 +02:00
Alvaro Revuelta
a67bf22e61 chore: add peer manager config to builder (#1816) 2023-06-23 15:30:28 +02:00
Simon-Pierre Vivier
887a5c76b2 chore: discv5 re-org setup (#1815)
Key, netconfig, enr are created at App initialization and config files has been renamed.
2023-06-22 16:58:14 -04:00
Ivan Folgueira Bande
12c6fd3df7 refactor(databases): Creation of the databases folder to keep the logic for sqlite and postgres (#1811)
* Refactoring in sqlite and postgres. Creation of the databases folder.
2023-06-22 11:27:40 +02:00
Ivan Folgueira Bande
c25c94cdd8 Deprecation of the 'topics' param (#1806) 2023-06-20 13:14:17 +02:00
Simon-Pierre Vivier
e5bca6ab29 sharded topic validation & ENR update (#1795)
Add sharded topic validation and update the node ENR accordingly.
2023-06-19 18:16:25 -04:00
Aaryamann Challani
0ed1070cef feat(rln-relay): resume onchain sync from persisted tree db (#1805)
* feat(rln-relay): resume onchain sync from persisted tree db

* chore(rln-relay): bump zerokit
2023-06-19 15:21:18 +05:30
Aaryamann Challani
107cb8119f feat(rln-relay): pass in index to keystore credentials (#1777)
* feat(rln-relay): pass in index to keystore credentials

* fix(rln-relay): s/MembershipIndex/uint
2023-06-06 23:13:05 +05:30