736 Commits

Author SHA1 Message Date
NagyZoltanPeter
e62664cca0
Make /debug endpoints /version and /info available as root (#3333) 2025-03-27 11:56:44 +01:00
Simon-Pierre Vivier
77105a6c2e
fix: apply latest nph formating (#3334) 2025-03-19 11:30:47 -04:00
NagyZoltanPeter
a3584caf5e
fix:lightppush v3 not returning relayed peers count (#3329)
* Fixing lightpush publish logs stated it was legacy lightpush, also fix on wrong mounted protocol check.
* Fix lightpush v3 success report did not embed relayed peer count
* Fix test that missed the case of non returning relayed peers
2025-03-18 09:36:58 +01:00
Simon-Pierre Vivier
bf735e7779
fix: waku sync 2.0 codecs ENR support (#3326) 2025-03-14 12:01:11 -04:00
NagyZoltanPeter
e0b563ffe5
feat: lightpush v3 (#3279)
* Separate new lightpush protocol
New RPC defined
Rename al occurence of old lightpush to legacy lightpush, fix rest tests of lightpush
New lightpush protocol added back
Setup new lightpush protocol, mounting and rest api for it

	modified:   apps/chat2/chat2.nim
	modified:   tests/node/test_wakunode_lightpush.nim
	modified:   tests/node/test_wakunode_sharding.nim
	modified:   tests/test_peer_manager.nim
	modified:   tests/test_wakunode_lightpush.nim
	renamed:    tests/waku_lightpush/lightpush_utils.nim -> tests/waku_lightpush_legacy/lightpush_utils.nim
	renamed:    tests/waku_lightpush/test_all.nim -> tests/waku_lightpush_legacy/test_all.nim
	renamed:    tests/waku_lightpush/test_client.nim -> tests/waku_lightpush_legacy/test_client.nim
	renamed:    tests/waku_lightpush/test_ratelimit.nim -> tests/waku_lightpush_legacy/test_ratelimit.nim
	modified:   tests/wakunode_rest/test_all.nim
	renamed:    tests/wakunode_rest/test_rest_lightpush.nim -> tests/wakunode_rest/test_rest_lightpush_legacy.nim
	modified:   waku/factory/node_factory.nim
	modified:   waku/node/waku_node.nim
	modified:   waku/waku_api/rest/admin/handlers.nim
	modified:   waku/waku_api/rest/builder.nim
	new file:   waku/waku_api/rest/legacy_lightpush/client.nim
	new file:   waku/waku_api/rest/legacy_lightpush/handlers.nim
	new file:   waku/waku_api/rest/legacy_lightpush/types.nim
	modified:   waku/waku_api/rest/lightpush/client.nim
	modified:   waku/waku_api/rest/lightpush/handlers.nim
	modified:   waku/waku_api/rest/lightpush/types.nim
	modified:   waku/waku_core/codecs.nim
	modified:   waku/waku_lightpush.nim
	modified:   waku/waku_lightpush/callbacks.nim
	modified:   waku/waku_lightpush/client.nim
	modified:   waku/waku_lightpush/common.nim
	modified:   waku/waku_lightpush/protocol.nim
	modified:   waku/waku_lightpush/rpc.nim
	modified:   waku/waku_lightpush/rpc_codec.nim
	modified:   waku/waku_lightpush/self_req_handler.nim
	new file:   waku/waku_lightpush_legacy.nim
	renamed:    waku/waku_lightpush/README.md -> waku/waku_lightpush_legacy/README.md
	new file:   waku/waku_lightpush_legacy/callbacks.nim
	new file:   waku/waku_lightpush_legacy/client.nim
	new file:   waku/waku_lightpush_legacy/common.nim
	new file:   waku/waku_lightpush_legacy/protocol.nim
	new file:   waku/waku_lightpush_legacy/protocol_metrics.nim
	new file:   waku/waku_lightpush_legacy/rpc.nim
	new file:   waku/waku_lightpush_legacy/rpc_codec.nim
	new file:   waku/waku_lightpush_legacy/self_req_handler.nim

Adapt to non-invasive libp2p observers

cherry pick latest lightpush (v1) changes into legacy lightpush code after rebase to latest master

Fix vendor dependencies from origin/master after failed rebase of them

Adjust examples, test to new lightpush - keep using of legacy

Fixup error code mappings

Fix REST admin interface with distinct legacy and new lightpush

Fix lightpush v2 tests

* Utilize new publishEx interface of pubsub libp2p

* Adapt to latest libp2p pubslih design changes. publish returns an outcome as Result error.

* Fix review findings

* Fix tests, re-added lost one

* Fix rebase

* Apply suggestions from code review

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Addressing review comments

* Fix incentivization tests

* Fix build failed on libwaku

* Change new lightpush endpoint version to 3 instead of 2. Noticed that old and new lightpush metrics can cause trouble in monitoring dashboards so decided to give new name as v3 for the new lightpush metrics and change legacy ones back - temporarly till old lightpush will be decommissioned

* Fixing flaky test with rate limit timing

* Fixing logscope of lightpush and legacy lightpush

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2025-03-05 12:07:56 +01:00
Ivan FB
c07e278d82
chore: better implementation to properly convert database query metrics (#3314) 2025-03-04 19:14:31 +01:00
NagyZoltanPeter
f08ba90906
chore: remove flaky test debug logs from rln and store tests (#3303)
* chore: remove flaky test debug logs from rln tests
* Remove flaky test logs from store and legacy store tests
2025-02-28 15:36:50 +01:00
Sergei Tikhomirov
a4e92a3d02
feat: incentivization PoC: client-side reputation system basics (#3293)
* chore: rename test file for eligibility tests

* add reputation manager

* add simple boolean reputation with dummy response

* set default reputation to true

* use reputation indicator term; remove unnecessary updateReputation

* use PushResponse in reputation manager

* add custom type for reputation

* add reputation update from response quality

* encode reputation indicators as Option[bool]
2025-02-20 16:07:21 +01:00
Ivan FB
091024b376
chore: better proof handling in REST (#3286)
* better proof handling in REST
2025-02-14 11:14:38 +01:00
Ivan FB
d391357ed7
Revert "chore: waku_archive add protection against queries longer than 24h" (#3278)
This reverts commit a4d71718a250018fbae7b287400b1d6b18f04bee.
2025-02-06 17:44:12 +01:00
Sergei Tikhomirov
c142994772
feat: incentivization POC: add double-spend check for txid-based eligibility (#3264)
* add double-spend check for txid-based eligibility

* Apply suggestions from code review

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* split assert into two in double-spending test

* remove unnecessary import

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2025-01-31 18:53:46 +01:00
Ivan FB
46747fd496
chore: filter remove all subscription from a peer that is leaving (#3267)
* waku/waku_filter_v2/protocol.nim keeps track of the filter-client connections in Table[PeerId, Connection]
* waku/waku_filter_v2/protocol.nim starts listening for peer-left events in order to completely remove the previous Connection instance. Also, a new Connection is added when the filter-service starts publishing to its peers.

---------
    
Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2025-01-31 17:01:55 +01:00
Simon-Pierre Vivier
42fd6b8278
feat: waku sync shard matching check (#3259) 2025-01-30 08:46:34 -05:00
gabrielmer
9c209b4c3b
chore: sending msg hash as string on libwaku message event (#3234) 2025-01-30 10:15:31 +02:00
Ivan FB
287e9b12ca
fix: filter - enhancements in subscription management (#3198)
* waku_filter_v2: idiomatic way run periodic subscription manager
* filter subscriptions: add more debug logs
* filter: make sure the custom start and stop procs are called
* make sure filter protocol is started if it is mounted
* filter: dial push connection on subscribe only
* reduce max num filter peers from 1000 to 100
* adapt filter tests
* waku_peer_exchange protocol remove temporary debug logs
2025-01-28 15:37:33 +01:00
Ivan FB
f8a233381e
chore: bump dependencies for v0.35 (#3255)
Changes:
	modified:   .gitmodules
	modified:   tests/waku_discv5/utils.nim
	modified:   tests/waku_enr/utils.nim
	modified:   tests/waku_rln_relay/test_rln_group_manager_onchain.nim
	modified:   tests/waku_rln_relay/utils.nim
	modified:   tests/waku_rln_relay/utils_onchain.nim

        modified:   vendor/nim-chronicles
	modified:   vendor/nim-eth
	modified:   vendor/nim-http-utils
	modified:   vendor/nim-json-rpc
	modified:   vendor/nim-json-serialization
	modified:   vendor/nim-libp2p - 1.8.0!
	modified:   vendor/nim-metrics
	new file:   vendor/nim-minilru
	modified:   vendor/nim-nat-traversal
	modified:   vendor/nim-presto
	modified:   vendor/nim-secp256k1
	modified:   vendor/nim-serialization
	modified:   vendor/nim-stew
	modified:   vendor/nim-taskpools
	modified:   vendor/nim-testutils
	modified:   vendor/nim-toml-serialization
	modified:   vendor/nim-unicodedb
	modified:   vendor/nim-unittest2
	modified:   vendor/nim-web3 - from distinct branch that solves Ethereum ABI issue.
	modified:   vendor/nim-websock
	modified:   vendor/nim-zlib
	modified:   vendor/nimcrypto
	modified:   waku.nimble

        modified:   waku/common/enr/builder.nim
	modified:   waku/common/enr/typed_record.nim
	modified:   waku/common/utils/nat.nim
	modified:   waku/discovery/waku_discv5.nim
	modified:   waku/waku_rln_relay/conversion_utils.nim
	modified:   waku/waku_rln_relay/group_manager/on_chain/group_manager.nim
	modified:   waku/waku_rln_relay/rln/wrappers.nim
	modified:   waku/waku_rln_relay/rln_relay.nim

* Eliminate C compilation issue with chat2bridge due to an overcomplicating import from json_rpc instead of using std/json
* Adapt ENR Record handling to new interface of nim-eth
* Fix chrash in group_manager on_chain
* Fix signature of register and MemberRegister to UInt256, check transaction success in register
* Upgrade json-rpc and serialization
* Update to match latest enr and nat interface
* Using of extracted result of contract macro - with necessary adaption
* Bump nim-chornicles, nim-libp2p, nimcrypto
* Bump nim-web3, nim-eth and deps - on_chain/group_manager.nim adaption
* Added status-im/nim-minilru submodule required by latest nim-eth
Fixing tests.
* group_manager: adapt smart contract param types
* update web3 vendor
* bump vendors for v0.35.0
* protobuf.nim: fix compilation error after nim-libp2p bump
* changes to make it compile after rebase from master
---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2025-01-28 10:04:34 +01:00
Ivan FB
a4d71718a2
chore: waku_archive add protection against queries longer than 24h (#3256)
* store test adaptations because the tests were using future times
* waku_store_sync.nim, test_waku_archive, test_rln_group_manager_onchain.nim nph (unrelated change)
2025-01-27 10:44:59 +01:00
Simon-Pierre Vivier
7f64dc03aa
feat: waku store sync 2.0 config & setup (#3217) 2025-01-24 11:46:11 -05:00
Simon-Pierre Vivier
6ee494d902
feat: waku store sync 2.0 protocols & tests (#3216) 2025-01-23 16:13:26 -05:00
Simon-Pierre Vivier
54a7a68754
feat: waku store sync 2.0 storage & tests (#3215) 2025-01-23 10:39:23 -05:00
Simon-Pierre Vivier
29fda2dab7
feat: waku store sync 2.0 common types & codec (#3213) 2025-01-22 11:08:23 -05:00
Sergei Tikhomirov
505ec84ce8
feat: add txhash-based eligibility checks for incentivization PoC (#3166)
Implement data structures and tests for checking transaction eligibility based on tx hash. This work will be continues in future PRs. All code added in this PR is only used in tests.

* feat: add simple txid-based eligibility check with hard-coded params (#3166)

* use new proc to generate eligibility status

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>

* minor fixes

* add comments to clarify eligibility definition

* use Address.fromHex conversion from eth-web3

* move isEligible to common

* refactor: avoid result and unnecesary branching

* define const for simple transfer gas usage

* avoid unnecessary parentheses

* chore: run nph linter manually

* refactor, move all hard-coded constants to tests

* use Result type in eligibility tests

* use standard method of error handling

* make try-block smaller

* add a try-block in case of connection failure to web3 provider

* make queries to web3 provider in parallel

* move Web3 provider RPC URL into env variable

* remove unused import

* rename functions

* use await in async proc

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>

* add timeout to tx receipt query

* parallelize queries for tx and txreceipt

* make test txids non public

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* use assert in txid i13n test

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* use parentheses when calling verb-methods without arguments

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* remove unused import

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* use init for stack-allocated objects

* add txReceipt error message to error

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* introduce eligibility manager

* [WIP] use Anvil for eligibility testing

* add eligibility test with contract deployment tx

* add eligibility test with contract call

* add asyncSetup and asyncTeardown for eligibility tests

* minor refactor

* refactor tests for onchain group manager with asyncSetup and asyncTeardown

* minor refactor

* remove unnecessary defer in asyncTeardown

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* remove unnecessary call in test (moved to asyncTeardown)

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* add comment justidying the use of discard

* rename file txid_proof to eligibility_manager

---------

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2025-01-22 11:16:49 +01:00
Darshan K
2942782f95
chore: capping mechanism for relay and service connections (#3184) 2025-01-21 11:29:52 +05:30
gabrielmer
bfd60ceab4
chore: adding extra migration to sqlite and improving error message (#3240) 2025-01-16 16:10:28 +01:00
Ivan FB
bb5988e51d
test: test_wakunode_rln_relay use waitForNullifierLog in all tests avoid flaky (#3227) 2025-01-06 13:47:17 +01:00
Simon-Pierre Vivier
650a9487e1
feat: waku rendezvous wrapper (#2962) 2024-12-09 15:22:36 -05:00
clonefetch
8f2bd39f23
chore: fix some typos in comment (#3201)
Signed-off-by: clonefetch <c0217@outlook.com>
2024-12-09 14:45:26 +01:00
Darshan K
1b532e8ab9
chore: add two metrics and panal (#3181) 2024-12-04 17:11:41 +05:30
gabrielmer
d7d00bfd79
feat: making dns discovery async (#3175) 2024-12-03 14:39:37 +01:00
Simon-Pierre Vivier
2ab9c3d363
feat: remove Waku Sync 1.0 & Negentropy (#3185) 2024-11-29 09:09:41 -05:00
Darshan K
bebd3dd62c
chore: flaky rln test (#3173) 2024-11-26 13:03:23 +05:30
NagyZoltanPeter
8368ff006d
chore: Partial version bumps for v0.34.0-rc.1 (#3172)
* Bumps for v0.34.0-rc.1 - partial bumping - libp2p
* Avoid importing quic and ngtcp2 dependencies through tests
* libp2p 1.7.1, fixes RendezVous construction
2024-11-10 09:27:04 +01:00
gabrielmer
400d7a54f6
feat: running periodicaly peer exchange if discv5 is disabled (#3150) 2024-10-30 12:51:04 +02:00
gabrielmer
b3656d6eea
chore: naming connectPeer procedure (#3157) 2024-10-29 18:37:07 +02:00
Ivan FB
cfde7eea82
chore: Circuit relay (#3112)
* undo apt install libpcre (not circuit-relay related.)
* nat.nim: protect against possible exceptions when calling getExternalIP
* new external CLI argument, isRelayClient
* waku factory change to mount circuit hop proto by default
* waku_node: move autonat_service to a separate module
2024-10-28 09:17:46 +01:00
gabrielmer
69d9524fa4
chore: saving peers enr capabilities (#3127) 2024-10-24 15:31:04 +03:00
gabrielmer
a6ed80a55d
chore: improving and temporarily skipping flaky rln test (#3094) 2024-10-07 18:02:06 +03:00
Ivan FB
5875ed63af
chore: Optimize store (#3061)
* use messages_lookup to retrieve timestamps
* deep refactoring in db_postgres for better use of async approach
2024-10-01 23:36:03 +02:00
Darshan K
729e63f53d
refactor: wrap peer store (#3051)
Encapsulate peerstore with wakupeerstore
2024-09-27 18:16:46 +05:30
Ivan FB
711e7db1e9
chore: test peer connection management (#3049)
* Make some useful consts public, add some utils.
* Implement various utilities.
* peer_manager reconnectPeers enhancements

---------

Co-authored-by: Álex Cabeza Romero <alex93cabeza@gmail.com>
2024-09-24 18:20:29 +02:00
NagyZoltanPeter
0a7f16a332
chore: rate limit peer exchange protocol, enhanced response status in RPC (#3035)
* Enhanced peer-ex protocol - added rate limiting, added response status and desc to the rpc

* Better error result handling for PeerEx request, adjusted tests

* Refactored RateLimit configuration option for better CLI UX - now possible to set separate limits per protocol. Adjusted mountings. Added and adjusted tests

* Fix libwaku due to changes of error return type of fetchPeerExchangePeers

* Fix rate limit setting tests due to changed defaults

* Introduce new gauge to help dasboard effectively show current rate limit applied for protocol

* Adjust timeing in filter rate limit test to let macos CI test run ok.

* Address review findings, namings, error logs, removed left-overs

* Changes to reflect latest spec agreement and changes. PeerExchange RPC is changed the now respond structure will contain status_code and status_desc.
2024-09-18 15:58:07 +02:00
gabrielmer
8f28992599
fix: setting up node with modified config (#3036) 2024-09-16 16:30:38 +03:00
Ivan FB
3ccb6cdf61
test: avoid too verbose rln test (#3029) 2024-09-11 10:22:00 +02:00
fryorcraken
ce9a8c468a
chore: use submodule nph in CI to check lint (#3027) 2024-09-11 11:51:42 +10:00
gabrielmer
a3cd2a1a92
chore: deprecating pubsub topic (#2997) 2024-09-10 15:07:12 -06:00
Ivan FB
d3e6717a60
chore: lightpush - error metric less variable by only setting a fixed string (#3020) 2024-09-10 17:30:09 +02:00
Ivan FB
e8a49b76b2
chore: Better timing and requestId detail for slower store db queries (#2994)
* Better timing and requestId detail for store db queries slower than two seconds
* Adapt tests and client to allow sending custom store requestId
2024-08-29 22:56:14 +02:00
gabrielmer
e51ffe0759
chore: deprecating protected topics in favor of protected shards (#2983) 2024-08-19 12:56:22 +02:00
gabrielmer
67439057fb
chore: rename NsPubsubTopic (#2974) 2024-08-19 11:29:35 +02:00
gabrielmer
dfd42a7cb1
chore: updating dependencies for release 0.32.0 (#2971) 2024-08-14 16:38:31 +02:00