732 Commits

Author SHA1 Message Date
NagyZoltanPeter
8b927b92d2 fix:lightppush v3 not returning relayed peers count (#3329)
* Fixing lightpush publish logs stated it was legacy lightpush, also fix on wrong mounted protocol check.
* Fix lightpush v3 success report did not embed relayed peer count
* Fix test that missed the case of non returning relayed peers
2025-03-18 09:36:58 +01:00
Simon-Pierre Vivier
bf1a0dc42c fix: waku sync 2.0 codecs ENR support (#3326) 2025-03-14 12:01:11 -04:00
NagyZoltanPeter
dcf09dd365 feat: lightpush v3 (#3279)
* Separate new lightpush protocol
New RPC defined
Rename al occurence of old lightpush to legacy lightpush, fix rest tests of lightpush
New lightpush protocol added back
Setup new lightpush protocol, mounting and rest api for it

	modified:   apps/chat2/chat2.nim
	modified:   tests/node/test_wakunode_lightpush.nim
	modified:   tests/node/test_wakunode_sharding.nim
	modified:   tests/test_peer_manager.nim
	modified:   tests/test_wakunode_lightpush.nim
	renamed:    tests/waku_lightpush/lightpush_utils.nim -> tests/waku_lightpush_legacy/lightpush_utils.nim
	renamed:    tests/waku_lightpush/test_all.nim -> tests/waku_lightpush_legacy/test_all.nim
	renamed:    tests/waku_lightpush/test_client.nim -> tests/waku_lightpush_legacy/test_client.nim
	renamed:    tests/waku_lightpush/test_ratelimit.nim -> tests/waku_lightpush_legacy/test_ratelimit.nim
	modified:   tests/wakunode_rest/test_all.nim
	renamed:    tests/wakunode_rest/test_rest_lightpush.nim -> tests/wakunode_rest/test_rest_lightpush_legacy.nim
	modified:   waku/factory/node_factory.nim
	modified:   waku/node/waku_node.nim
	modified:   waku/waku_api/rest/admin/handlers.nim
	modified:   waku/waku_api/rest/builder.nim
	new file:   waku/waku_api/rest/legacy_lightpush/client.nim
	new file:   waku/waku_api/rest/legacy_lightpush/handlers.nim
	new file:   waku/waku_api/rest/legacy_lightpush/types.nim
	modified:   waku/waku_api/rest/lightpush/client.nim
	modified:   waku/waku_api/rest/lightpush/handlers.nim
	modified:   waku/waku_api/rest/lightpush/types.nim
	modified:   waku/waku_core/codecs.nim
	modified:   waku/waku_lightpush.nim
	modified:   waku/waku_lightpush/callbacks.nim
	modified:   waku/waku_lightpush/client.nim
	modified:   waku/waku_lightpush/common.nim
	modified:   waku/waku_lightpush/protocol.nim
	modified:   waku/waku_lightpush/rpc.nim
	modified:   waku/waku_lightpush/rpc_codec.nim
	modified:   waku/waku_lightpush/self_req_handler.nim
	new file:   waku/waku_lightpush_legacy.nim
	renamed:    waku/waku_lightpush/README.md -> waku/waku_lightpush_legacy/README.md
	new file:   waku/waku_lightpush_legacy/callbacks.nim
	new file:   waku/waku_lightpush_legacy/client.nim
	new file:   waku/waku_lightpush_legacy/common.nim
	new file:   waku/waku_lightpush_legacy/protocol.nim
	new file:   waku/waku_lightpush_legacy/protocol_metrics.nim
	new file:   waku/waku_lightpush_legacy/rpc.nim
	new file:   waku/waku_lightpush_legacy/rpc_codec.nim
	new file:   waku/waku_lightpush_legacy/self_req_handler.nim

Adapt to non-invasive libp2p observers

cherry pick latest lightpush (v1) changes into legacy lightpush code after rebase to latest master

Fix vendor dependencies from origin/master after failed rebase of them

Adjust examples, test to new lightpush - keep using of legacy

Fixup error code mappings

Fix REST admin interface with distinct legacy and new lightpush

Fix lightpush v2 tests

* Utilize new publishEx interface of pubsub libp2p

* Adapt to latest libp2p pubslih design changes. publish returns an outcome as Result error.

* Fix review findings

* Fix tests, re-added lost one

* Fix rebase

* Apply suggestions from code review

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Addressing review comments

* Fix incentivization tests

* Fix build failed on libwaku

* Change new lightpush endpoint version to 3 instead of 2. Noticed that old and new lightpush metrics can cause trouble in monitoring dashboards so decided to give new name as v3 for the new lightpush metrics and change legacy ones back - temporarly till old lightpush will be decommissioned

* Fixing flaky test with rate limit timing

* Fixing logscope of lightpush and legacy lightpush

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2025-03-05 12:07:56 +01:00
Ivan FB
564b6466a8 chore: better implementation to properly convert database query metrics (#3314) 2025-03-04 19:14:31 +01:00
NagyZoltanPeter
92f893987f chore: remove flaky test debug logs from rln and store tests (#3303)
* chore: remove flaky test debug logs from rln tests
* Remove flaky test logs from store and legacy store tests
2025-02-28 15:36:50 +01:00
Sergei Tikhomirov
fb55ed0b70 feat: incentivization PoC: client-side reputation system basics (#3293)
* chore: rename test file for eligibility tests

* add reputation manager

* add simple boolean reputation with dummy response

* set default reputation to true

* use reputation indicator term; remove unnecessary updateReputation

* use PushResponse in reputation manager

* add custom type for reputation

* add reputation update from response quality

* encode reputation indicators as Option[bool]
2025-02-20 16:07:21 +01:00
Ivan FB
9bb567eb0e chore: better proof handling in REST (#3286)
* better proof handling in REST
2025-02-14 11:14:38 +01:00
Ivan FB
f65bea0f8e Revert "chore: waku_archive add protection against queries longer than 24h" (#3278)
This reverts commit 401402368d9075f93692d180cb30156785eed5a8.
2025-02-06 17:44:12 +01:00
Sergei Tikhomirov
79846e8433 feat: incentivization POC: add double-spend check for txid-based eligibility (#3264)
* add double-spend check for txid-based eligibility

* Apply suggestions from code review

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* split assert into two in double-spending test

* remove unnecessary import

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2025-01-31 18:53:46 +01:00
Ivan FB
80291abc9a chore: filter remove all subscription from a peer that is leaving (#3267)
* waku/waku_filter_v2/protocol.nim keeps track of the filter-client connections in Table[PeerId, Connection]
* waku/waku_filter_v2/protocol.nim starts listening for peer-left events in order to completely remove the previous Connection instance. Also, a new Connection is added when the filter-service starts publishing to its peers.

---------
    
Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2025-01-31 17:01:55 +01:00
Simon-Pierre Vivier
d8aaa93df9 feat: waku sync shard matching check (#3259) 2025-01-30 08:46:34 -05:00
gabrielmer
8867fd6fa9 chore: sending msg hash as string on libwaku message event (#3234) 2025-01-30 10:15:31 +02:00
Ivan FB
d9e79022fe fix: filter - enhancements in subscription management (#3198)
* waku_filter_v2: idiomatic way run periodic subscription manager
* filter subscriptions: add more debug logs
* filter: make sure the custom start and stop procs are called
* make sure filter protocol is started if it is mounted
* filter: dial push connection on subscribe only
* reduce max num filter peers from 1000 to 100
* adapt filter tests
* waku_peer_exchange protocol remove temporary debug logs
2025-01-28 15:37:33 +01:00
Ivan FB
c01a21e01f chore: bump dependencies for v0.35 (#3255)
Changes:
	modified:   .gitmodules
	modified:   tests/waku_discv5/utils.nim
	modified:   tests/waku_enr/utils.nim
	modified:   tests/waku_rln_relay/test_rln_group_manager_onchain.nim
	modified:   tests/waku_rln_relay/utils.nim
	modified:   tests/waku_rln_relay/utils_onchain.nim

        modified:   vendor/nim-chronicles
	modified:   vendor/nim-eth
	modified:   vendor/nim-http-utils
	modified:   vendor/nim-json-rpc
	modified:   vendor/nim-json-serialization
	modified:   vendor/nim-libp2p - 1.8.0!
	modified:   vendor/nim-metrics
	new file:   vendor/nim-minilru
	modified:   vendor/nim-nat-traversal
	modified:   vendor/nim-presto
	modified:   vendor/nim-secp256k1
	modified:   vendor/nim-serialization
	modified:   vendor/nim-stew
	modified:   vendor/nim-taskpools
	modified:   vendor/nim-testutils
	modified:   vendor/nim-toml-serialization
	modified:   vendor/nim-unicodedb
	modified:   vendor/nim-unittest2
	modified:   vendor/nim-web3 - from distinct branch that solves Ethereum ABI issue.
	modified:   vendor/nim-websock
	modified:   vendor/nim-zlib
	modified:   vendor/nimcrypto
	modified:   waku.nimble

        modified:   waku/common/enr/builder.nim
	modified:   waku/common/enr/typed_record.nim
	modified:   waku/common/utils/nat.nim
	modified:   waku/discovery/waku_discv5.nim
	modified:   waku/waku_rln_relay/conversion_utils.nim
	modified:   waku/waku_rln_relay/group_manager/on_chain/group_manager.nim
	modified:   waku/waku_rln_relay/rln/wrappers.nim
	modified:   waku/waku_rln_relay/rln_relay.nim

* Eliminate C compilation issue with chat2bridge due to an overcomplicating import from json_rpc instead of using std/json
* Adapt ENR Record handling to new interface of nim-eth
* Fix chrash in group_manager on_chain
* Fix signature of register and MemberRegister to UInt256, check transaction success in register
* Upgrade json-rpc and serialization
* Update to match latest enr and nat interface
* Using of extracted result of contract macro - with necessary adaption
* Bump nim-chornicles, nim-libp2p, nimcrypto
* Bump nim-web3, nim-eth and deps - on_chain/group_manager.nim adaption
* Added status-im/nim-minilru submodule required by latest nim-eth
Fixing tests.
* group_manager: adapt smart contract param types
* update web3 vendor
* bump vendors for v0.35.0
* protobuf.nim: fix compilation error after nim-libp2p bump
* changes to make it compile after rebase from master
---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2025-01-28 10:04:34 +01:00
Ivan FB
401402368d chore: waku_archive add protection against queries longer than 24h (#3256)
* store test adaptations because the tests were using future times
* waku_store_sync.nim, test_waku_archive, test_rln_group_manager_onchain.nim nph (unrelated change)
2025-01-27 10:44:59 +01:00
Simon-Pierre Vivier
7031607b58 feat: waku store sync 2.0 config & setup (#3217) 2025-01-24 11:46:11 -05:00
Simon-Pierre Vivier
2630b88b41 feat: waku store sync 2.0 protocols & tests (#3216) 2025-01-23 16:13:26 -05:00
Simon-Pierre Vivier
f550c76eb1 feat: waku store sync 2.0 storage & tests (#3215) 2025-01-23 10:39:23 -05:00
Simon-Pierre Vivier
c1b9257948 feat: waku store sync 2.0 common types & codec (#3213) 2025-01-22 11:08:23 -05:00
Sergei Tikhomirov
fdfc48c923 feat: add txhash-based eligibility checks for incentivization PoC (#3166)
Implement data structures and tests for checking transaction eligibility based on tx hash. This work will be continues in future PRs. All code added in this PR is only used in tests.

* feat: add simple txid-based eligibility check with hard-coded params (#3166)

* use new proc to generate eligibility status

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>

* minor fixes

* add comments to clarify eligibility definition

* use Address.fromHex conversion from eth-web3

* move isEligible to common

* refactor: avoid result and unnecesary branching

* define const for simple transfer gas usage

* avoid unnecessary parentheses

* chore: run nph linter manually

* refactor, move all hard-coded constants to tests

* use Result type in eligibility tests

* use standard method of error handling

* make try-block smaller

* add a try-block in case of connection failure to web3 provider

* make queries to web3 provider in parallel

* move Web3 provider RPC URL into env variable

* remove unused import

* rename functions

* use await in async proc

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>

* add timeout to tx receipt query

* parallelize queries for tx and txreceipt

* make test txids non public

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* use assert in txid i13n test

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* use parentheses when calling verb-methods without arguments

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* remove unused import

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* use init for stack-allocated objects

* add txReceipt error message to error

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* introduce eligibility manager

* [WIP] use Anvil for eligibility testing

* add eligibility test with contract deployment tx

* add eligibility test with contract call

* add asyncSetup and asyncTeardown for eligibility tests

* minor refactor

* refactor tests for onchain group manager with asyncSetup and asyncTeardown

* minor refactor

* remove unnecessary defer in asyncTeardown

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* remove unnecessary call in test (moved to asyncTeardown)

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* add comment justidying the use of discard

* rename file txid_proof to eligibility_manager

---------

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2025-01-22 11:16:49 +01:00
Darshan K
dd1a70bdb7 chore: capping mechanism for relay and service connections (#3184) 2025-01-21 11:29:52 +05:30
gabrielmer
4d9e11f16b chore: adding extra migration to sqlite and improving error message (#3240) 2025-01-16 16:10:28 +01:00
Ivan FB
202c2785ca test: test_wakunode_rln_relay use waitForNullifierLog in all tests avoid flaky (#3227) 2025-01-06 13:47:17 +01:00
Simon-Pierre Vivier
ae013e1928 feat: waku rendezvous wrapper (#2962) 2024-12-09 15:22:36 -05:00
clonefetch
b5edf6db98 chore: fix some typos in comment (#3201)
Signed-off-by: clonefetch <c0217@outlook.com>
2024-12-09 14:45:26 +01:00
Darshan K
4f77bb21d1 chore: add two metrics and panal (#3181) 2024-12-04 17:11:41 +05:30
gabrielmer
ad03b22413 feat: making dns discovery async (#3175) 2024-12-03 14:39:37 +01:00
Simon-Pierre Vivier
682981f967 feat: remove Waku Sync 1.0 & Negentropy (#3185) 2024-11-29 09:09:41 -05:00
Darshan K
7de94c5c2c chore: flaky rln test (#3173) 2024-11-26 13:03:23 +05:30
NagyZoltanPeter
0d3b70fa16 chore: Partial version bumps for v0.34.0-rc.1 (#3172)
* Bumps for v0.34.0-rc.1 - partial bumping - libp2p
* Avoid importing quic and ngtcp2 dependencies through tests
* libp2p 1.7.1, fixes RendezVous construction
2024-11-10 09:27:04 +01:00
gabrielmer
460be6e5a6 feat: running periodicaly peer exchange if discv5 is disabled (#3150) 2024-10-30 12:51:04 +02:00
gabrielmer
f3af7fa37e chore: naming connectPeer procedure (#3157) 2024-10-29 18:37:07 +02:00
Ivan FB
3786ce12e2 chore: Circuit relay (#3112)
* undo apt install libpcre (not circuit-relay related.)
* nat.nim: protect against possible exceptions when calling getExternalIP
* new external CLI argument, isRelayClient
* waku factory change to mount circuit hop proto by default
* waku_node: move autonat_service to a separate module
2024-10-28 09:17:46 +01:00
gabrielmer
223ca1db75 chore: saving peers enr capabilities (#3127) 2024-10-24 15:31:04 +03:00
gabrielmer
d964b66146 chore: improving and temporarily skipping flaky rln test (#3094) 2024-10-07 18:02:06 +03:00
Ivan FB
e128385e69 chore: Optimize store (#3061)
* use messages_lookup to retrieve timestamps
* deep refactoring in db_postgres for better use of async approach
2024-10-01 23:36:03 +02:00
Darshan K
785cf2e9d9 refactor: wrap peer store (#3051)
Encapsulate peerstore with wakupeerstore
2024-09-27 18:16:46 +05:30
Ivan FB
8c8eea4b67 chore: test peer connection management (#3049)
* Make some useful consts public, add some utils.
* Implement various utilities.
* peer_manager reconnectPeers enhancements

---------

Co-authored-by: Álex Cabeza Romero <alex93cabeza@gmail.com>
2024-09-24 18:20:29 +02:00
NagyZoltanPeter
e7ae1a0382 chore: rate limit peer exchange protocol, enhanced response status in RPC (#3035)
* Enhanced peer-ex protocol - added rate limiting, added response status and desc to the rpc

* Better error result handling for PeerEx request, adjusted tests

* Refactored RateLimit configuration option for better CLI UX - now possible to set separate limits per protocol. Adjusted mountings. Added and adjusted tests

* Fix libwaku due to changes of error return type of fetchPeerExchangePeers

* Fix rate limit setting tests due to changed defaults

* Introduce new gauge to help dasboard effectively show current rate limit applied for protocol

* Adjust timeing in filter rate limit test to let macos CI test run ok.

* Address review findings, namings, error logs, removed left-overs

* Changes to reflect latest spec agreement and changes. PeerExchange RPC is changed the now respond structure will contain status_code and status_desc.
2024-09-18 15:58:07 +02:00
gabrielmer
36df0fd838 fix: setting up node with modified config (#3036) 2024-09-16 16:30:38 +03:00
Ivan FB
b1fd3ef204 test: avoid too verbose rln test (#3029) 2024-09-11 10:22:00 +02:00
fryorcraken
723b009b20 chore: use submodule nph in CI to check lint (#3027) 2024-09-11 11:51:42 +10:00
gabrielmer
43bea3c476 chore: deprecating pubsub topic (#2997) 2024-09-10 15:07:12 -06:00
Ivan FB
f34a044ccf chore: lightpush - error metric less variable by only setting a fixed string (#3020) 2024-09-10 17:30:09 +02:00
Ivan FB
236547ec7d chore: Better timing and requestId detail for slower store db queries (#2994)
* Better timing and requestId detail for store db queries slower than two seconds
* Adapt tests and client to allow sending custom store requestId
2024-08-29 22:56:14 +02:00
gabrielmer
3f641dff60 chore: deprecating protected topics in favor of protected shards (#2983) 2024-08-19 12:56:22 +02:00
gabrielmer
90b4dc89ff chore: rename NsPubsubTopic (#2974) 2024-08-19 11:29:35 +02:00
gabrielmer
f094c671ca chore: updating dependencies for release 0.32.0 (#2971) 2024-08-14 16:38:31 +02:00
Simon-Pierre Vivier
301ce8068c feat: Nwaku Sync (#2403)
* feat: Waku Sync Protocol

* feat: state machine (#2656)

* feat: pruning storage mehcanism (#2673)

* feat: message transfer mechanism & tests (#2688)

* update docker files

* added ENR filed for sync & misc. fixes

* adding new sync range param & fixes

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
2024-08-13 07:27:34 -04:00
Ivan FB
696587fdac chore: Optimize hash queries with lookup table (#2933)
* Upgrade Postgres schema to add messages_lookup table
* Perform optimized query for messageHash-only queries
2024-08-08 21:46:08 +02:00