258 Commits

Author SHA1 Message Date
gabrielmer
a1901a044e chore: deprecating dnsDiscovery flag (#3305) 2025-02-24 22:06:48 +02:00
Ivan FB
c01a21e01f chore: bump dependencies for v0.35 (#3255)
Changes:
	modified:   .gitmodules
	modified:   tests/waku_discv5/utils.nim
	modified:   tests/waku_enr/utils.nim
	modified:   tests/waku_rln_relay/test_rln_group_manager_onchain.nim
	modified:   tests/waku_rln_relay/utils.nim
	modified:   tests/waku_rln_relay/utils_onchain.nim

        modified:   vendor/nim-chronicles
	modified:   vendor/nim-eth
	modified:   vendor/nim-http-utils
	modified:   vendor/nim-json-rpc
	modified:   vendor/nim-json-serialization
	modified:   vendor/nim-libp2p - 1.8.0!
	modified:   vendor/nim-metrics
	new file:   vendor/nim-minilru
	modified:   vendor/nim-nat-traversal
	modified:   vendor/nim-presto
	modified:   vendor/nim-secp256k1
	modified:   vendor/nim-serialization
	modified:   vendor/nim-stew
	modified:   vendor/nim-taskpools
	modified:   vendor/nim-testutils
	modified:   vendor/nim-toml-serialization
	modified:   vendor/nim-unicodedb
	modified:   vendor/nim-unittest2
	modified:   vendor/nim-web3 - from distinct branch that solves Ethereum ABI issue.
	modified:   vendor/nim-websock
	modified:   vendor/nim-zlib
	modified:   vendor/nimcrypto
	modified:   waku.nimble

        modified:   waku/common/enr/builder.nim
	modified:   waku/common/enr/typed_record.nim
	modified:   waku/common/utils/nat.nim
	modified:   waku/discovery/waku_discv5.nim
	modified:   waku/waku_rln_relay/conversion_utils.nim
	modified:   waku/waku_rln_relay/group_manager/on_chain/group_manager.nim
	modified:   waku/waku_rln_relay/rln/wrappers.nim
	modified:   waku/waku_rln_relay/rln_relay.nim

* Eliminate C compilation issue with chat2bridge due to an overcomplicating import from json_rpc instead of using std/json
* Adapt ENR Record handling to new interface of nim-eth
* Fix chrash in group_manager on_chain
* Fix signature of register and MemberRegister to UInt256, check transaction success in register
* Upgrade json-rpc and serialization
* Update to match latest enr and nat interface
* Using of extracted result of contract macro - with necessary adaption
* Bump nim-chornicles, nim-libp2p, nimcrypto
* Bump nim-web3, nim-eth and deps - on_chain/group_manager.nim adaption
* Added status-im/nim-minilru submodule required by latest nim-eth
Fixing tests.
* group_manager: adapt smart contract param types
* update web3 vendor
* bump vendors for v0.35.0
* protobuf.nim: fix compilation error after nim-libp2p bump
* changes to make it compile after rebase from master
---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2025-01-28 10:04:34 +01:00
Darshan K
dd1a70bdb7 chore: capping mechanism for relay and service connections (#3184) 2025-01-21 11:29:52 +05:30
NagyZoltanPeter
7731dfad32 chore: Bump nimbus and nim to latest available - nim-2.0.12 (#3188)
* Bump nimbus and nim to latest available - nim-2.0.12
* Fix name collision of templates of result.nim and nwaku serdes.nim - unrecognizedFieldWarning
2024-12-10 14:42:54 +01:00
clonefetch
b5edf6db98 chore: fix some typos in comment (#3201)
Signed-off-by: clonefetch <c0217@outlook.com>
2024-12-09 14:45:26 +01:00
NagyZoltanPeter
a0c468b4d5 fix: lite-protocol-tester receiver exit check (#3187)
* Fix receiver exit criteria, not let it wait forever in some cases, added a timely check from the last arrived message
* Extend dial and service usage failure metrics with agent string to reveal service nodes origins
* Adjusted infra testing content topic to be unique in the system
* Extend error logs with peer's agent string, fix exit criteria
* Add informative log for not waiting for more messages
* Add unknown as default for empty agent identifier
* better explain exit logic of receiver
* Address review comment - checking for last message arrival return Optional Moment instead of result - better explains what is happening.
2024-12-07 01:22:50 +01:00
gabrielmer
ad03b22413 feat: making dns discovery async (#3175) 2024-12-03 14:39:37 +01:00
NagyZoltanPeter
f35a6f10a6 chore: add supervisor for lpt infra (#3176)
* Adding lpt-runner script and assemble into liteprotocoltester image - to ease infra deployment
* Add supervisor that can run lpt continously in infra environment, infra.env defines defaults for run, in case image tag of lpt docker image is deploy it will build a specific image for infra deployment.
* Added message latency metrics
* DELAY_MESSAGES to MESSAGE_INTERVAL renaming
* Adjust name of START_PUBLISHING_AFTER
* Extend lpt readme with how to use make to build dockerized image and notice about infra deployment
* As fixed in discussion, we will control infra testing by built in predefined test setup
* Prevent peer switch in case using fixed service peers
2024-11-26 20:42:27 +01:00
NagyZoltanPeter
293682b57d Adding lpt-runner script and assemble into liteprotocoltester image - to ease infra deployment (#3158) 2024-11-26 15:54:29 +01:00
Ivan FB
76354df9bf chore: libwaku - better error handling and better waku thread destroy handling (#3167) 2024-11-08 14:59:02 +07:00
fryorcraken
89addda4b0 feat: change latency buckets (#3153) 2024-10-29 14:39:12 +11:00
Ivan FB
3786ce12e2 chore: Circuit relay (#3112)
* undo apt install libpcre (not circuit-relay related.)
* nat.nim: protect against possible exceptions when calling getExternalIP
* new external CLI argument, isRelayClient
* waku factory change to mount circuit hop proto by default
* waku_node: move autonat_service to a separate module
2024-10-28 09:17:46 +01:00
NagyZoltanPeter
e5f7a8f776 chore: easy setup fleets for lpt (#3125)
* Added bootstrap peer exchange discovery option for easy setup ltp
* Extended with PX discovery, auto-dial of PX cap peers, added switching service peers if failed with original
* Added peer-exchange, found capable peers test, metrics on peer stability and availability, dashboard adjustments
* Updated and actualized README.md for liteprotocoltester
* Created jenkinsfile for liteprotocoltester deployment
* Fixed dial exception during lightpublish
* Add configuration for requesting and testing peer exchange peers
* Extended examples added to Readme
* Added metrics port configurability
---------

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
2024-10-25 22:59:02 +02:00
Marko Burčul
2198d78fc6 sonda: adapt setup for deployment (#3151)
Referenced issue: https://github.com/status-im/infra-hq/issues/135

Signed-off-by: markoburcul <marko@status.im>
2024-10-25 16:31:59 +02:00
Vaclav Pavlin
9d2b931567 chore(networkmonitor): add missing field on RlnRelay init, set default for num of shard (#3136) 2024-10-23 10:23:25 +02:00
Darshan K
785cf2e9d9 refactor: wrap peer store (#3051)
Encapsulate peerstore with wakupeerstore
2024-09-27 18:16:46 +05:30
NagyZoltanPeter
6dfefc5e42 chore: Switch libnegentropy library build from shared to static linkage (#3041)
* Switch libnegentropy library build from shared to static linkage

* Update negentropy with -fPIC compile option that is necessary for libwaku build

* Bump waku-org/negentropy to the latest on master to incorporate merged static build of libnegentropy
2024-09-18 14:34:50 +02:00
gabrielmer
36df0fd838 fix: setting up node with modified config (#3036) 2024-09-16 16:30:38 +03:00
fryorcraken
723b009b20 chore: use submodule nph in CI to check lint (#3027) 2024-09-11 11:51:42 +10:00
gabrielmer
43bea3c476 chore: deprecating pubsub topic (#2997) 2024-09-10 15:07:12 -06:00
NagyZoltanPeter
930d2a8b04 chore: Added metrics to liteprotocoltester (#3002)
* Added metrics to tests
Fix liteprotocoltester docker files with libnegentropy COPY
docker compose with waku-sim simulation now having test performance dashboard and localhost:3033

Mention dashboard in Readme

* Update apps/liteprotocoltester/statistics.nim

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* indent fix, more stable finding of service/bootstrap nodes, pre-set for TWN

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-09-04 08:35:51 +02:00
NagyZoltanPeter
3212459f77 chore: liteprotocoltester for simulation and for fleets (#2813)
* Added phase 2 - waku-simulatior integration in README.md

* Enhancement on statistics reports, added list of sent messages with hash, fixed latency calculations

* Enable standalonde running liteprotocoltester agains any waku network/fleet

* Fix missing env vars on run_tester_node.sh

* Adjustment on log levels, fix REST initialization

* Added standalon docker image build, fine tune duplicate detection and logging.

* Adjustments for waku-simulator runs

* Extended liteprotocoltester README.md with docker build

* Fix test inside docker service node connectivity failure

* Update apps/liteprotocoltester/README.md

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Explain minLatency calculation in code comment

---------

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-08-21 14:54:18 +02:00
Ivan FB
3e6b2ea683 networkmonitor: fix compilation issue (#2964) 2024-08-08 20:11:51 +02:00
Simon-Pierre Vivier
6b22823b64 feat: misc. updates for discovery network analysis (#2930)
added metrics, a way to start without RLN and a new avg latency algorithm
2024-08-07 14:58:28 -04:00
fcc11a7cd9 chore: replace statusim.net instances with status.im (#2941)
Use of `statusim.net` domain been deprecated since March:
https://github.com/status-im/infra-shards/commit/7df38c14

Also adjust test to match enr with multiaddresses.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-08-05 12:57:43 +02:00
Prem Chaitanya Prathi
2614d93566 fix: network monitor improvements (#2939) 2024-07-30 16:56:49 +03:00
gabrielmer
6ca28cd74d including UTC time in logs and logging timestamp (#2926) 2024-07-26 12:45:44 +02:00
gabrielmer
d9a48321d2 chore: improve sonda dashboard (#2918) 2024-07-19 14:31:23 +03:00
Simon-Pierre Vivier
d60ff3e0e6 chore(archive): archive and drivers refactor (#2761)
* queue driver refactor (#2753)
* chore(archive): archive refactor (#2752)
* chore(archive): sqlite driver refactor (#2754)
* chore(archive): postgres driver refactor (#2755)
* chore(archive): renaming & copies (#2751)
* posgres legacy: stop using the storedAt field
* migration script 6: we still need the id column
  The id column is needed because it contains the message digest
  which is used in store v2, and we need to keep support to
  store v2 for a while
* legacy archive: set target migration version to 6
* waku_node: try to use wakuLegacyArchive if wakuArchive is nil
* node_factory, waku_node: mount legacy and future store simultaneously
  We want the nwaku node to simultaneously support store-v2
  requests and store-v3 requests.
  Only the legacy archive is in charge of archiving messages, and the
  archived information is suitable to fulfill both store-v2 and
  store-v3 needs.
* postgres_driver: adding temporary code until store-v2 is removed

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan Folgueira Bande <ivansete@status.im>
2024-07-12 18:19:12 +02:00
gabrielmer
49d6adca3b feat: sonda tool (#2893) 2024-07-12 11:05:13 +03:00
gabrielmer
1071ffc6c8 chore: deprecating named sharding (#2723) 2024-07-09 18:36:12 +03:00
Ivan FB
13316201f7 chore: Bump dependencies for v0.31.0 (#2885)
* bump_dependencies.md: add nim-results dependency
* change imports stew/results to results
* switching to Nim 2.0.8
* waku.nimble: reflect the requirement nim 1.6.0 to 2.0.8
  Adding --mm:refc as nim 2.0 enables a new garbage collector that we're
  not yet ready to support
* adapt waku code to Nim 2.0
* gcsafe adaptations because Nim 2.0 is more strict
2024-07-09 13:14:28 +02:00
Darshan K
8da2a9c0a5 chore: refactor relative path to better absolute (#2861) 2024-07-06 00:03:38 +02:00
Darshan K
7ad9722ecf chore: remove all pre-nim-1.6 deadcode from codebase (#2857) 2024-06-28 16:04:57 +05:30
NagyZoltanPeter
35509ed4fd feat: Added proper per shard bandwidth metric calculation (#2851)
* Added proper per shard bandwidth metric calculation and proper logging of in/out messages
Changed rate limit metrics for dashboard
Updated monitoring dashboard for bw and rate metrics
2024-06-28 02:48:29 +02:00
Aaryamann Challani
7e4f18cda7 feat(rlnv2): clean fork of rlnv2 (#2828)
* chore(rlnv2): contract interface changes (#2770)
* fix: tests
* fix: remove stuint[32]
* chore(submodule): update zerokit submodule to v0.5.1 (#2782)
* fix: remove cond comp for lightpush test
* fix: ci and nonceManager
2024-06-20 11:35:21 +02:00
Darshan K
88d25755db fix: mount metadata in wakucanary (#2793)
* chore: integrate cluster id and shards to waku node.
2024-06-14 18:29:42 +05:30
Ivan FB
f958512e46 chore: Minor enhancements (#2789)
* archive.nim: reduce the database report interval from 1 to 30 min
  This counts the number of rows with "select count(1) from messages"
  which is quite intense and we shouldn't run it every minute
* aside cleanup
2024-06-09 23:09:23 +02:00
gabrielmer
3fd715cfc2 unifying clusterId to be uint16 (#2777) 2024-06-05 15:32:35 +02:00
gabrielmer
b9d0525663 chore: Improving liteprotocolteseter stats (#2750) 2024-06-04 22:57:16 +02:00
Vaclav Pavlin
b1f74e5776 chore(wakucanary): fix fitler protocol, add storev3 (#2735) 2024-05-27 23:13:59 +02:00
NagyZoltanPeter
16748a986a LiteProtocolTester application and docker compose bundle setup. (#2706)
faster image build with copy from pre-built binary
   cluster-id to 0 
   Added README.md documentation
2024-05-21 23:03:33 +02:00
Ivan FB
ab3b7df42e chore: libwaku - allow to properly set the log level in libwaku and unify a little (#2708)
* waku.nimble: set properly chronicles compilation flags for static libwaku
* adapt examples to new log setup
2024-05-17 16:28:54 +02:00
Ivan FB
652fc172d4 chore: generic change to reduce the number of compilation warnings (#2696) 2024-05-16 22:29:11 +02:00
Ivan FB
0b2859ed5a chore: move code from wakunode2 to a more generic place, waku (#2670)
* testlib/wakunode.nim: not use cluster-id == 1 to avoid test rln by default
2024-05-13 17:45:48 +02:00
NagyZoltanPeter
e028362086 feat: Added flexible rate limit checks for store, legacy store and lightpush (#2668)
* Added flexible rate limit checks for store, legacy store and lightpush. Also added rate and traffic metrics.

* Fix chat2 after WakuLegacyStoreCodec rename

* Update waku/common/ratelimit.nim

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Update waku/common/ratelimit.nim

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Update waku/waku_store_legacy/protocol.nim

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Fix review findings, added limit to debug logs

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-05-09 20:07:49 +02:00
Ivan FB
1d35ca970f refactor: big refactor to add waku component in libwaku instead of onlu waku node (#2658) 2024-05-03 14:07:15 +02:00
Ivan FB
7bab843003 refactor: metrics server. Simplify app.nim module (#2650) 2024-04-30 15:07:17 +02:00
Simon-Pierre Vivier
665d9e3a06 feat: store v3 (#2431) 2024-04-25 09:09:52 -04:00
Ivan FB
963d79aee7 refactor: addition of waku_api/rest/builder.nim and reduce app.nim (#2623) 2024-04-24 15:59:50 +02:00