754 Commits

Author SHA1 Message Date
Ivan FB
336fbf8b64 fix: relay unsubscribe (#3422)
* waku_relay protocol fix unsubscribe and remove topic validator
* simplify subscription and avoid unneeded code
* tests adaptations
* call wakuRelay.subscribe only in one place within waku_node
2025-06-02 22:02:49 +02:00
NagyZoltanPeter
a39bcff6dc feat: Extend node /health REST endpoint with all protocol's state (#3419)
* Extend ndoe /health REST endpoint with all protocol's state

* Added check for Rendezvous peers availability

* Fine tune filter, added client protocols to health report

* Fix /health endpoint test

* Add explanatory description for state NOT_READY

* Fix formattings

* Apply suggestions from code review

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Apply code style changes and extended test

* Fix formatting

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2025-06-02 17:21:09 +02:00
gabrielmer
5e22ea18b6 chore: don't return error on double relay subscription/unsubscription (#3429) 2025-05-29 12:05:48 +02:00
Ivan FB
9f68c83fed chore: bump dependencies for v0.36 (#3410)
* properly pass userMessageLimit to OnchainGroupManager
* waku.nimble 2.2.4 Nim compiler
* rm stew/shims/net import
* change ValidIpAddress.init with parseIpAddress
* fix serialize for zerokit
* group_manager: separate if statements
* protocol_types: add encode UInt32 with zeros up to 32 bytes
* windows build: skip libunwind build and rm libunwind.a inlcusion step
* bump nph to overcome the compilation issues with 2.2.x
* bump nim-libp2p to v1.10.1
2025-05-26 21:58:02 +02:00
Darshan K
39e65dea28 fix: timestamp based validation (#3406) 2025-05-26 17:56:29 +05:30
Ivan FB
42ab866f2c chore: allow multiple rln eth clients (#3402)
* use of multiple Eth clients instead of just one
* config_chat2 enhance param comment
* group_manager: raise exception if could not connect to any of the eth clients
2025-05-12 10:57:13 +02:00
Darshan K
d86babac3a feat: deprecate sync / local merkle tree (#3312) 2025-05-09 05:37:58 +05:30
fryorcraken
cc66c7fe78 chore!: separate internal and CLI configurations (#3357)
Split `WakuNodeConfig` object for better separation of concerns and to introduce a tree-like structure to configuration.

* fix: ensure twn cluster conf is still applied when clusterId=1
* test: remove usage of `WakuNodeConf`
* Remove macro, split builder files, remove wakunodeconf from tests
* rm network_conf_builder module as it is not used

---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
Co-authored-by: Ivan Folgueira Bande <ivansete@status.im>
2025-05-07 23:05:35 +02:00
Ivan FB
6bc05efc02 chore: Avoid double relay subscription (#3396)
* make sure subscribe once to every topic in waku_node
* start suggest use of removeValidator in waku_relay/protocol. Commented until libp2p updated.
2025-05-05 22:57:20 +02:00
NagyZoltanPeter
98c3979119 chore: return all peers from rest admin (#3395)
* Updated version of getting peers by /admin endpoints
2025-04-25 15:36:41 +02:00
Ivan FB
2d6e5ef9ad chore: rln_relay simplify code a little (#3392) 2025-04-25 14:52:37 +02:00
Simon-Pierre Vivier
0c63ce4e9b feat: refactor waku sync DOS protection (#3391) 2025-04-24 09:07:21 -04:00
NagyZoltanPeter
8394c15a1a fix: bad HttpCode conversion, add missing lightpush v3 rest api tests (#3389)
* Fix bad HttpCode conversion, add missing lightpush v3 rest api tests
2025-04-24 08:36:30 +02:00
NagyZoltanPeter
ab8a30d3d6 chore: extended /admin/v1 RESP API with different option to look at current connected/relay/mesh state of the node (#3382)
* Extended /admin/v1 RESP API with different option to look at current connected/relay/mesh state of the node
* Added score information for peer info retrievals
2025-04-24 08:36:02 +02:00
Ivan FB
5ae526ce4f chore: Timestamp now in publish (#3373)
* Ensure timestamp is always set in WakuMessage when publishing
2025-04-17 13:03:56 +02:00
Miran
ed0474ade3 chore: fix unused and deprecated imports (#3368) 2025-04-11 18:20:23 +03:00
Ivan FB
3098b117d3 chore: skip two flaky tests (#3364) 2025-04-10 00:28:25 +02:00
Ivan FB
75b8838fbf chore: retrieve protocols in new added peer from discv5 (#3354)
* add new unit test to validate that any peer can be retrieved
* add new discv5 test and better peer store management
* wakuPeerStore -> switch.peerStore
* simplify waku_peer_store, better logs and peer_manager enhancements
2025-04-07 12:24:03 +02:00
fryorcraken
63cff2ab42 test: fix preset tests (#3351) 2025-04-01 11:57:30 +02:00
fryorcraken
58f76ce467 feat: introduce preset option (#3346)
* feat: introduce `preset` option

Overwriting config from cluster-id will be deprecated as a second step.

* doc: improve preset doc

* Change `default` preset to `twn`
2025-04-01 09:28:18 +11:00
NagyZoltanPeter
fe8327627e Make /debug endpoints /version and /info available as root (#3333) 2025-03-27 11:56:44 +01:00
Simon-Pierre Vivier
7dbc1fe061 fix: apply latest nph formating (#3334) 2025-03-19 11:30:47 -04:00
NagyZoltanPeter
8b927b92d2 fix:lightppush v3 not returning relayed peers count (#3329)
* Fixing lightpush publish logs stated it was legacy lightpush, also fix on wrong mounted protocol check.
* Fix lightpush v3 success report did not embed relayed peer count
* Fix test that missed the case of non returning relayed peers
2025-03-18 09:36:58 +01:00
Simon-Pierre Vivier
bf1a0dc42c fix: waku sync 2.0 codecs ENR support (#3326) 2025-03-14 12:01:11 -04:00
NagyZoltanPeter
dcf09dd365 feat: lightpush v3 (#3279)
* Separate new lightpush protocol
New RPC defined
Rename al occurence of old lightpush to legacy lightpush, fix rest tests of lightpush
New lightpush protocol added back
Setup new lightpush protocol, mounting and rest api for it

	modified:   apps/chat2/chat2.nim
	modified:   tests/node/test_wakunode_lightpush.nim
	modified:   tests/node/test_wakunode_sharding.nim
	modified:   tests/test_peer_manager.nim
	modified:   tests/test_wakunode_lightpush.nim
	renamed:    tests/waku_lightpush/lightpush_utils.nim -> tests/waku_lightpush_legacy/lightpush_utils.nim
	renamed:    tests/waku_lightpush/test_all.nim -> tests/waku_lightpush_legacy/test_all.nim
	renamed:    tests/waku_lightpush/test_client.nim -> tests/waku_lightpush_legacy/test_client.nim
	renamed:    tests/waku_lightpush/test_ratelimit.nim -> tests/waku_lightpush_legacy/test_ratelimit.nim
	modified:   tests/wakunode_rest/test_all.nim
	renamed:    tests/wakunode_rest/test_rest_lightpush.nim -> tests/wakunode_rest/test_rest_lightpush_legacy.nim
	modified:   waku/factory/node_factory.nim
	modified:   waku/node/waku_node.nim
	modified:   waku/waku_api/rest/admin/handlers.nim
	modified:   waku/waku_api/rest/builder.nim
	new file:   waku/waku_api/rest/legacy_lightpush/client.nim
	new file:   waku/waku_api/rest/legacy_lightpush/handlers.nim
	new file:   waku/waku_api/rest/legacy_lightpush/types.nim
	modified:   waku/waku_api/rest/lightpush/client.nim
	modified:   waku/waku_api/rest/lightpush/handlers.nim
	modified:   waku/waku_api/rest/lightpush/types.nim
	modified:   waku/waku_core/codecs.nim
	modified:   waku/waku_lightpush.nim
	modified:   waku/waku_lightpush/callbacks.nim
	modified:   waku/waku_lightpush/client.nim
	modified:   waku/waku_lightpush/common.nim
	modified:   waku/waku_lightpush/protocol.nim
	modified:   waku/waku_lightpush/rpc.nim
	modified:   waku/waku_lightpush/rpc_codec.nim
	modified:   waku/waku_lightpush/self_req_handler.nim
	new file:   waku/waku_lightpush_legacy.nim
	renamed:    waku/waku_lightpush/README.md -> waku/waku_lightpush_legacy/README.md
	new file:   waku/waku_lightpush_legacy/callbacks.nim
	new file:   waku/waku_lightpush_legacy/client.nim
	new file:   waku/waku_lightpush_legacy/common.nim
	new file:   waku/waku_lightpush_legacy/protocol.nim
	new file:   waku/waku_lightpush_legacy/protocol_metrics.nim
	new file:   waku/waku_lightpush_legacy/rpc.nim
	new file:   waku/waku_lightpush_legacy/rpc_codec.nim
	new file:   waku/waku_lightpush_legacy/self_req_handler.nim

Adapt to non-invasive libp2p observers

cherry pick latest lightpush (v1) changes into legacy lightpush code after rebase to latest master

Fix vendor dependencies from origin/master after failed rebase of them

Adjust examples, test to new lightpush - keep using of legacy

Fixup error code mappings

Fix REST admin interface with distinct legacy and new lightpush

Fix lightpush v2 tests

* Utilize new publishEx interface of pubsub libp2p

* Adapt to latest libp2p pubslih design changes. publish returns an outcome as Result error.

* Fix review findings

* Fix tests, re-added lost one

* Fix rebase

* Apply suggestions from code review

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Addressing review comments

* Fix incentivization tests

* Fix build failed on libwaku

* Change new lightpush endpoint version to 3 instead of 2. Noticed that old and new lightpush metrics can cause trouble in monitoring dashboards so decided to give new name as v3 for the new lightpush metrics and change legacy ones back - temporarly till old lightpush will be decommissioned

* Fixing flaky test with rate limit timing

* Fixing logscope of lightpush and legacy lightpush

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2025-03-05 12:07:56 +01:00
Ivan FB
564b6466a8 chore: better implementation to properly convert database query metrics (#3314) 2025-03-04 19:14:31 +01:00
NagyZoltanPeter
92f893987f chore: remove flaky test debug logs from rln and store tests (#3303)
* chore: remove flaky test debug logs from rln tests
* Remove flaky test logs from store and legacy store tests
2025-02-28 15:36:50 +01:00
Sergei Tikhomirov
fb55ed0b70 feat: incentivization PoC: client-side reputation system basics (#3293)
* chore: rename test file for eligibility tests

* add reputation manager

* add simple boolean reputation with dummy response

* set default reputation to true

* use reputation indicator term; remove unnecessary updateReputation

* use PushResponse in reputation manager

* add custom type for reputation

* add reputation update from response quality

* encode reputation indicators as Option[bool]
2025-02-20 16:07:21 +01:00
Ivan FB
9bb567eb0e chore: better proof handling in REST (#3286)
* better proof handling in REST
2025-02-14 11:14:38 +01:00
Ivan FB
f65bea0f8e Revert "chore: waku_archive add protection against queries longer than 24h" (#3278)
This reverts commit 401402368d9075f93692d180cb30156785eed5a8.
2025-02-06 17:44:12 +01:00
Sergei Tikhomirov
79846e8433 feat: incentivization POC: add double-spend check for txid-based eligibility (#3264)
* add double-spend check for txid-based eligibility

* Apply suggestions from code review

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* split assert into two in double-spending test

* remove unnecessary import

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2025-01-31 18:53:46 +01:00
Ivan FB
80291abc9a chore: filter remove all subscription from a peer that is leaving (#3267)
* waku/waku_filter_v2/protocol.nim keeps track of the filter-client connections in Table[PeerId, Connection]
* waku/waku_filter_v2/protocol.nim starts listening for peer-left events in order to completely remove the previous Connection instance. Also, a new Connection is added when the filter-service starts publishing to its peers.

---------
    
Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2025-01-31 17:01:55 +01:00
Simon-Pierre Vivier
d8aaa93df9 feat: waku sync shard matching check (#3259) 2025-01-30 08:46:34 -05:00
gabrielmer
8867fd6fa9 chore: sending msg hash as string on libwaku message event (#3234) 2025-01-30 10:15:31 +02:00
Ivan FB
d9e79022fe fix: filter - enhancements in subscription management (#3198)
* waku_filter_v2: idiomatic way run periodic subscription manager
* filter subscriptions: add more debug logs
* filter: make sure the custom start and stop procs are called
* make sure filter protocol is started if it is mounted
* filter: dial push connection on subscribe only
* reduce max num filter peers from 1000 to 100
* adapt filter tests
* waku_peer_exchange protocol remove temporary debug logs
2025-01-28 15:37:33 +01:00
Ivan FB
c01a21e01f chore: bump dependencies for v0.35 (#3255)
Changes:
	modified:   .gitmodules
	modified:   tests/waku_discv5/utils.nim
	modified:   tests/waku_enr/utils.nim
	modified:   tests/waku_rln_relay/test_rln_group_manager_onchain.nim
	modified:   tests/waku_rln_relay/utils.nim
	modified:   tests/waku_rln_relay/utils_onchain.nim

        modified:   vendor/nim-chronicles
	modified:   vendor/nim-eth
	modified:   vendor/nim-http-utils
	modified:   vendor/nim-json-rpc
	modified:   vendor/nim-json-serialization
	modified:   vendor/nim-libp2p - 1.8.0!
	modified:   vendor/nim-metrics
	new file:   vendor/nim-minilru
	modified:   vendor/nim-nat-traversal
	modified:   vendor/nim-presto
	modified:   vendor/nim-secp256k1
	modified:   vendor/nim-serialization
	modified:   vendor/nim-stew
	modified:   vendor/nim-taskpools
	modified:   vendor/nim-testutils
	modified:   vendor/nim-toml-serialization
	modified:   vendor/nim-unicodedb
	modified:   vendor/nim-unittest2
	modified:   vendor/nim-web3 - from distinct branch that solves Ethereum ABI issue.
	modified:   vendor/nim-websock
	modified:   vendor/nim-zlib
	modified:   vendor/nimcrypto
	modified:   waku.nimble

        modified:   waku/common/enr/builder.nim
	modified:   waku/common/enr/typed_record.nim
	modified:   waku/common/utils/nat.nim
	modified:   waku/discovery/waku_discv5.nim
	modified:   waku/waku_rln_relay/conversion_utils.nim
	modified:   waku/waku_rln_relay/group_manager/on_chain/group_manager.nim
	modified:   waku/waku_rln_relay/rln/wrappers.nim
	modified:   waku/waku_rln_relay/rln_relay.nim

* Eliminate C compilation issue with chat2bridge due to an overcomplicating import from json_rpc instead of using std/json
* Adapt ENR Record handling to new interface of nim-eth
* Fix chrash in group_manager on_chain
* Fix signature of register and MemberRegister to UInt256, check transaction success in register
* Upgrade json-rpc and serialization
* Update to match latest enr and nat interface
* Using of extracted result of contract macro - with necessary adaption
* Bump nim-chornicles, nim-libp2p, nimcrypto
* Bump nim-web3, nim-eth and deps - on_chain/group_manager.nim adaption
* Added status-im/nim-minilru submodule required by latest nim-eth
Fixing tests.
* group_manager: adapt smart contract param types
* update web3 vendor
* bump vendors for v0.35.0
* protobuf.nim: fix compilation error after nim-libp2p bump
* changes to make it compile after rebase from master
---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2025-01-28 10:04:34 +01:00
Ivan FB
401402368d chore: waku_archive add protection against queries longer than 24h (#3256)
* store test adaptations because the tests were using future times
* waku_store_sync.nim, test_waku_archive, test_rln_group_manager_onchain.nim nph (unrelated change)
2025-01-27 10:44:59 +01:00
Simon-Pierre Vivier
7031607b58 feat: waku store sync 2.0 config & setup (#3217) 2025-01-24 11:46:11 -05:00
Simon-Pierre Vivier
2630b88b41 feat: waku store sync 2.0 protocols & tests (#3216) 2025-01-23 16:13:26 -05:00
Simon-Pierre Vivier
f550c76eb1 feat: waku store sync 2.0 storage & tests (#3215) 2025-01-23 10:39:23 -05:00
Simon-Pierre Vivier
c1b9257948 feat: waku store sync 2.0 common types & codec (#3213) 2025-01-22 11:08:23 -05:00
Sergei Tikhomirov
fdfc48c923 feat: add txhash-based eligibility checks for incentivization PoC (#3166)
Implement data structures and tests for checking transaction eligibility based on tx hash. This work will be continues in future PRs. All code added in this PR is only used in tests.

* feat: add simple txid-based eligibility check with hard-coded params (#3166)

* use new proc to generate eligibility status

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>

* minor fixes

* add comments to clarify eligibility definition

* use Address.fromHex conversion from eth-web3

* move isEligible to common

* refactor: avoid result and unnecesary branching

* define const for simple transfer gas usage

* avoid unnecessary parentheses

* chore: run nph linter manually

* refactor, move all hard-coded constants to tests

* use Result type in eligibility tests

* use standard method of error handling

* make try-block smaller

* add a try-block in case of connection failure to web3 provider

* make queries to web3 provider in parallel

* move Web3 provider RPC URL into env variable

* remove unused import

* rename functions

* use await in async proc

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>

* add timeout to tx receipt query

* parallelize queries for tx and txreceipt

* make test txids non public

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* use assert in txid i13n test

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* use parentheses when calling verb-methods without arguments

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* remove unused import

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* use init for stack-allocated objects

* add txReceipt error message to error

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* introduce eligibility manager

* [WIP] use Anvil for eligibility testing

* add eligibility test with contract deployment tx

* add eligibility test with contract call

* add asyncSetup and asyncTeardown for eligibility tests

* minor refactor

* refactor tests for onchain group manager with asyncSetup and asyncTeardown

* minor refactor

* remove unnecessary defer in asyncTeardown

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* remove unnecessary call in test (moved to asyncTeardown)

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* add comment justidying the use of discard

* rename file txid_proof to eligibility_manager

---------

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2025-01-22 11:16:49 +01:00
Darshan K
dd1a70bdb7 chore: capping mechanism for relay and service connections (#3184) 2025-01-21 11:29:52 +05:30
gabrielmer
4d9e11f16b chore: adding extra migration to sqlite and improving error message (#3240) 2025-01-16 16:10:28 +01:00
Ivan FB
202c2785ca test: test_wakunode_rln_relay use waitForNullifierLog in all tests avoid flaky (#3227) 2025-01-06 13:47:17 +01:00
Simon-Pierre Vivier
ae013e1928 feat: waku rendezvous wrapper (#2962) 2024-12-09 15:22:36 -05:00
clonefetch
b5edf6db98 chore: fix some typos in comment (#3201)
Signed-off-by: clonefetch <c0217@outlook.com>
2024-12-09 14:45:26 +01:00
Darshan K
4f77bb21d1 chore: add two metrics and panal (#3181) 2024-12-04 17:11:41 +05:30
gabrielmer
ad03b22413 feat: making dns discovery async (#3175) 2024-12-03 14:39:37 +01:00
Simon-Pierre Vivier
682981f967 feat: remove Waku Sync 1.0 & Negentropy (#3185) 2024-11-29 09:09:41 -05:00