715 Commits

Author SHA1 Message Date
NagyZoltanPeter
baae8b418a
Fix signature of register and MemberRegister to UInt256, check transaction success in register 2024-12-12 11:02:35 +01:00
NagyZoltanPeter
4b0be366f4
Fix chrash in group_manager on_chain 2024-12-12 11:02:35 +01:00
NagyZoltanPeter
b2a00a8021
Adapt ENR Record handling to new interface of nim-eth 2024-12-12 11:02:34 +01:00
NagyZoltanPeter
53e4e7b95a
Bump nim-chornicles, nim-libp2p, nimcrypto
Bump nim-web3, nim-eth and deps - on_chain/group_manager.nim adaption
Added status-im/nim-minilru submodule required by latest nim-eth
Added nim-quic and nim-ngtcp2 as new deps of nim-libp2p
Fixing tests.

 Changes:
	modified:   .gitmodules
	modified:   tests/waku_discv5/utils.nim
	modified:   tests/waku_enr/utils.nim
	modified:   tests/waku_rln_relay/test_rln_group_manager_onchain.nim
	modified:   tests/waku_rln_relay/utils.nim
	modified:   tests/waku_rln_relay/utils_onchain.nim

        modified:   vendor/nim-chronicles
	modified:   vendor/nim-eth
	modified:   vendor/nim-http-utils
	modified:   vendor/nim-json-rpc
	modified:   vendor/nim-json-serialization
	modified:   vendor/nim-libp2p
	modified:   vendor/nim-metrics
	new file:   vendor/nim-minilru
	modified:   vendor/nim-nat-traversal
	new file:   vendor/nim-ngtcp2
	modified:   vendor/nim-presto
	new file:   vendor/nim-quic
	modified:   vendor/nim-secp256k1
	modified:   vendor/nim-serialization
	modified:   vendor/nim-stew
	modified:   vendor/nim-taskpools
	modified:   vendor/nim-testutils
	modified:   vendor/nim-toml-serialization
	modified:   vendor/nim-unicodedb
	modified:   vendor/nim-unittest2
	modified:   vendor/nim-web3
	modified:   vendor/nim-websock
	modified:   vendor/nim-zlib
	modified:   vendor/nimcrypto
	modified:   waku.nimble

        modified:   waku/common/enr/builder.nim
	modified:   waku/common/enr/typed_record.nim
	modified:   waku/common/utils/nat.nim
	modified:   waku/discovery/waku_discv5.nim
	modified:   waku/waku_rln_relay/conversion_utils.nim
	modified:   waku/waku_rln_relay/group_manager/on_chain/group_manager.nim
	modified:   waku/waku_rln_relay/rln/wrappers.nim
	modified:   waku/waku_rln_relay/rln_relay.nim
2024-12-12 11:02:33 +01:00
Simon-Pierre Vivier
650a9487e1
feat: waku rendezvous wrapper (#2962) 2024-12-09 15:22:36 -05:00
clonefetch
8f2bd39f23
chore: fix some typos in comment (#3201)
Signed-off-by: clonefetch <c0217@outlook.com>
2024-12-09 14:45:26 +01:00
Darshan K
1b532e8ab9
chore: add two metrics and panal (#3181) 2024-12-04 17:11:41 +05:30
gabrielmer
d7d00bfd79
feat: making dns discovery async (#3175) 2024-12-03 14:39:37 +01:00
Simon-Pierre Vivier
2ab9c3d363
feat: remove Waku Sync 1.0 & Negentropy (#3185) 2024-11-29 09:09:41 -05:00
Darshan K
bebd3dd62c
chore: flaky rln test (#3173) 2024-11-26 13:03:23 +05:30
NagyZoltanPeter
8368ff006d
chore: Partial version bumps for v0.34.0-rc.1 (#3172)
* Bumps for v0.34.0-rc.1 - partial bumping - libp2p
* Avoid importing quic and ngtcp2 dependencies through tests
* libp2p 1.7.1, fixes RendezVous construction
2024-11-10 09:27:04 +01:00
gabrielmer
400d7a54f6
feat: running periodicaly peer exchange if discv5 is disabled (#3150) 2024-10-30 12:51:04 +02:00
gabrielmer
b3656d6eea
chore: naming connectPeer procedure (#3157) 2024-10-29 18:37:07 +02:00
Ivan FB
cfde7eea82
chore: Circuit relay (#3112)
* undo apt install libpcre (not circuit-relay related.)
* nat.nim: protect against possible exceptions when calling getExternalIP
* new external CLI argument, isRelayClient
* waku factory change to mount circuit hop proto by default
* waku_node: move autonat_service to a separate module
2024-10-28 09:17:46 +01:00
gabrielmer
69d9524fa4
chore: saving peers enr capabilities (#3127) 2024-10-24 15:31:04 +03:00
gabrielmer
a6ed80a55d
chore: improving and temporarily skipping flaky rln test (#3094) 2024-10-07 18:02:06 +03:00
Ivan FB
5875ed63af
chore: Optimize store (#3061)
* use messages_lookup to retrieve timestamps
* deep refactoring in db_postgres for better use of async approach
2024-10-01 23:36:03 +02:00
Darshan K
729e63f53d
refactor: wrap peer store (#3051)
Encapsulate peerstore with wakupeerstore
2024-09-27 18:16:46 +05:30
Ivan FB
711e7db1e9
chore: test peer connection management (#3049)
* Make some useful consts public, add some utils.
* Implement various utilities.
* peer_manager reconnectPeers enhancements

---------

Co-authored-by: Álex Cabeza Romero <alex93cabeza@gmail.com>
2024-09-24 18:20:29 +02:00
NagyZoltanPeter
0a7f16a332
chore: rate limit peer exchange protocol, enhanced response status in RPC (#3035)
* Enhanced peer-ex protocol - added rate limiting, added response status and desc to the rpc

* Better error result handling for PeerEx request, adjusted tests

* Refactored RateLimit configuration option for better CLI UX - now possible to set separate limits per protocol. Adjusted mountings. Added and adjusted tests

* Fix libwaku due to changes of error return type of fetchPeerExchangePeers

* Fix rate limit setting tests due to changed defaults

* Introduce new gauge to help dasboard effectively show current rate limit applied for protocol

* Adjust timeing in filter rate limit test to let macos CI test run ok.

* Address review findings, namings, error logs, removed left-overs

* Changes to reflect latest spec agreement and changes. PeerExchange RPC is changed the now respond structure will contain status_code and status_desc.
2024-09-18 15:58:07 +02:00
gabrielmer
8f28992599
fix: setting up node with modified config (#3036) 2024-09-16 16:30:38 +03:00
Ivan FB
3ccb6cdf61
test: avoid too verbose rln test (#3029) 2024-09-11 10:22:00 +02:00
fryorcraken
ce9a8c468a
chore: use submodule nph in CI to check lint (#3027) 2024-09-11 11:51:42 +10:00
gabrielmer
a3cd2a1a92
chore: deprecating pubsub topic (#2997) 2024-09-10 15:07:12 -06:00
Ivan FB
d3e6717a60
chore: lightpush - error metric less variable by only setting a fixed string (#3020) 2024-09-10 17:30:09 +02:00
Ivan FB
e8a49b76b2
chore: Better timing and requestId detail for slower store db queries (#2994)
* Better timing and requestId detail for store db queries slower than two seconds
* Adapt tests and client to allow sending custom store requestId
2024-08-29 22:56:14 +02:00
gabrielmer
e51ffe0759
chore: deprecating protected topics in favor of protected shards (#2983) 2024-08-19 12:56:22 +02:00
gabrielmer
67439057fb
chore: rename NsPubsubTopic (#2974) 2024-08-19 11:29:35 +02:00
gabrielmer
dfd42a7cb1
chore: updating dependencies for release 0.32.0 (#2971) 2024-08-14 16:38:31 +02:00
Simon-Pierre Vivier
2cc86c51da
feat: Nwaku Sync (#2403)
* feat: Waku Sync Protocol

* feat: state machine (#2656)

* feat: pruning storage mehcanism (#2673)

* feat: message transfer mechanism & tests (#2688)

* update docker files

* added ENR filed for sync & misc. fixes

* adding new sync range param & fixes

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
2024-08-13 07:27:34 -04:00
Ivan FB
4d47aa655f
chore: Optimize hash queries with lookup table (#2933)
* Upgrade Postgres schema to add messages_lookup table
* Perform optimized query for messageHash-only queries
2024-08-08 21:46:08 +02:00
f534549a1e
chore: replace statusim.net instances with status.im (#2941)
Use of `statusim.net` domain been deprecated since March:
https://github.com/status-im/infra-shards/commit/7df38c14

Also adjust test to match enr with multiaddresses.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-08-05 12:57:43 +02:00
Álex
a3fa175054
test(rln): Implement rln tests (#2639)
* Implement tests.
* Clean coding.
2024-08-02 16:43:22 +02:00
Ivan FB
ebda56dd90
lightpush better feedback in case the lightpush service node does not have peers (#2951) 2024-08-02 09:45:05 +02:00
Ivan FB
d4e8a0dab6
chore: Simplification of store legacy code (#2931) 2024-07-30 14:05:23 +02:00
Simon-Pierre Vivier
aed2a1130e
feat: store resume (#2919)
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-07-30 07:23:39 -04:00
Simon-Pierre Vivier
b4618f98ed
chore: add peer filtering by cluster for waku peer exchange (#2932) 2024-07-29 15:53:43 -04:00
Darshan K
08973add7e
fix: update and shift unittest (#2934)
* fix: update and shift location of unit test
2024-07-26 16:57:34 +05:30
Sasha
a29eca77a0
chore: return all connected peers from REST API (#2923)
* Remove the condition of gathering connected peers with relay and user req/resp protocols.
* Return PeerExchange protocol support of connected nodes with /admin/peers
* Added test for checking return of PeerExchange mounted protocol of connected peer by GET /admin/peers

---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2024-07-23 12:58:56 +02:00
gabrielmer
086cc8edd2
chore: adding lint job to the CI (#2925) 2024-07-23 13:57:24 +03:00
Darshan K
8d107b0ded
fix: handle rln-relay-message-limit (#2867)
* fix: enforcing rln-contract max message limit and resolve early
2024-07-22 22:28:45 +05:30
NagyZoltanPeter
ba418ab5ba
feat: DOS protection of non relay protocols - rate limit phase3 (#2897)
* DOS protection of non relay protocols - rate limit phase3:
- Enhanced TokenBucket to be able to add compensation tokens based on previous usage percentage,
- per peer rate limiter 'PeerRateLimier' applied on waku_filter_v2 with opinionated default of acceptable request rate
- Add traffic metrics to filter message push
- RequestRateLimiter added to combine simple token bucket limiting of request numbers but consider per peer usage over time and prevent some peers to over use the service
  (although currently rule violating peers will not be disconnected by this time only their requests will get not served)
- TimedMap utility created (inspired and taken from libp2p TimedCache) which serves as forgiving feature for peers had been overusing the service.
- Added more tests
- Fix rebase issues
- Applied new RequestRateLimiter for store and legacy_store and lightpush
* Incorporate review comments, typos, file/class naming and placement changes.
* Add issue link reference of the original issue with nim-chronos TokenBucket
* Make TimedEntry of TimedMap private and not mixable with similar named in libp2p
* Fix review comments, renamings, const instead of values and more comments.
2024-07-16 15:46:21 +02:00
Simon-Pierre Vivier
f54ba10bc7
chore(archive): archive and drivers refactor (#2761)
* queue driver refactor (#2753)
* chore(archive): archive refactor (#2752)
* chore(archive): sqlite driver refactor (#2754)
* chore(archive): postgres driver refactor (#2755)
* chore(archive): renaming & copies (#2751)
* posgres legacy: stop using the storedAt field
* migration script 6: we still need the id column
  The id column is needed because it contains the message digest
  which is used in store v2, and we need to keep support to
  store v2 for a while
* legacy archive: set target migration version to 6
* waku_node: try to use wakuLegacyArchive if wakuArchive is nil
* node_factory, waku_node: mount legacy and future store simultaneously
  We want the nwaku node to simultaneously support store-v2
  requests and store-v3 requests.
  Only the legacy archive is in charge of archiving messages, and the
  archived information is suitable to fulfill both store-v2 and
  store-v3 needs.
* postgres_driver: adding temporary code until store-v2 is removed

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan Folgueira Bande <ivansete@status.im>
2024-07-12 18:19:12 +02:00
Aaryamann Challani
1c9eb27415
fix(rln_keystore_generator): improve error handling for unrecoverable failure (#2881) 2024-07-10 19:12:49 +02:00
gabrielmer
e1518cf9ff
chore: deprecating named sharding (#2723) 2024-07-09 18:36:12 +03:00
Ivan FB
fd6a71cdd7
chore: Bump dependencies for v0.31.0 (#2885)
* bump_dependencies.md: add nim-results dependency
* change imports stew/results to results
* switching to Nim 2.0.8
* waku.nimble: reflect the requirement nim 1.6.0 to 2.0.8
  Adding --mm:refc as nim 2.0 enables a new garbage collector that we're
  not yet ready to support
* adapt waku code to Nim 2.0
* gcsafe adaptations because Nim 2.0 is more strict
2024-07-09 13:14:28 +02:00
Darshan K
8bfad3ab45
chore: refactor relative path to better absolute (#2861) 2024-07-06 00:03:38 +02:00
Ivan Folgueira Bande
519abbebf0
Merge branch 'release/v0.30' into update-master-from-release-v0.30 2024-07-02 10:16:40 +02:00
gabrielmer
cae0c7e378
chore: saving agent and protoVersion in peerStore (#2860) 2024-07-01 13:29:14 +02:00
Darshan K
31c632e427
chore: unit test for duplicate message push (#2852)
* chore: add unit test for testing duplicate message push with timedcache

* chore: update according to better naming convention

---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2024-06-28 18:16:06 +05:30