1292 Commits

Author SHA1 Message Date
Ivan FB
e58b5c15c8 chore: Better database query logs and logarithmic scale in grafana store panels (#3048) 2024-09-20 22:47:15 +03:00
Ivan FB
96cc2f1b39 chore: Better database query logs and logarithmic scale in grafana store panels (#3048) 2024-09-20 17:43:56 +02:00
Ivan FB
a16c64ad45 chore: extending store metrics (#3042)
* adding query_metrics module
* update fleet-dashboard with new store panels for better timing insight
2024-09-20 15:43:00 +03:00
Ivan FB
3921036ced sharding: reduce log level for a too spammy message (#3045) 2024-09-20 15:42:46 +03:00
Ivan FB
1f768cb3e8 chore: extending store metrics (#3042)
* adding query_metrics module
* update fleet-dashboard with new store panels for better timing insight
2024-09-20 13:23:53 +02:00
Ivan FB
d5ff611a5e sharding: reduce log level for a too spammy message (#3045) 2024-09-20 13:22:10 +02:00
NagyZoltanPeter
e7ae1a0382 chore: rate limit peer exchange protocol, enhanced response status in RPC (#3035)
* Enhanced peer-ex protocol - added rate limiting, added response status and desc to the rpc

* Better error result handling for PeerEx request, adjusted tests

* Refactored RateLimit configuration option for better CLI UX - now possible to set separate limits per protocol. Adjusted mountings. Added and adjusted tests

* Fix libwaku due to changes of error return type of fetchPeerExchangePeers

* Fix rate limit setting tests due to changed defaults

* Introduce new gauge to help dasboard effectively show current rate limit applied for protocol

* Adjust timeing in filter rate limit test to let macos CI test run ok.

* Address review findings, namings, error logs, removed left-overs

* Changes to reflect latest spec agreement and changes. PeerExchange RPC is changed the now respond structure will contain status_code and status_desc.
2024-09-18 15:58:07 +02:00
NagyZoltanPeter
6dfefc5e42 chore: Switch libnegentropy library build from shared to static linkage (#3041)
* Switch libnegentropy library build from shared to static linkage

* Update negentropy with -fPIC compile option that is necessary for libwaku build

* Bump waku-org/negentropy to the latest on master to incorporate merged static build of libnegentropy
2024-09-18 14:34:50 +02:00
gabrielmer
36df0fd838 fix: setting up node with modified config (#3036) 2024-09-16 16:30:38 +03:00
gabrielmer
2a596f4c77 adding a dynamic sleep interval in the connectivity loop (#3031) 2024-09-12 22:49:47 +02:00
Ivan FB
004b56e422 chore: libwaku - extending the library with peer_manager and peer_exchange features (#3026)
* libwaku: get peerids by protocol and peer exchange request
2024-09-11 10:13:54 +02:00
fryorcraken
723b009b20 chore: use submodule nph in CI to check lint (#3027) 2024-09-11 11:51:42 +10:00
gabrielmer
43bea3c476 chore: deprecating pubsub topic (#2997) 2024-09-10 15:07:12 -06:00
Ivan FB
f34a044ccf chore: lightpush - error metric less variable by only setting a fixed string (#3020) 2024-09-10 17:30:09 +02:00
Ivan FB
aefb7fb73d fix: get back health check for postgres legacy (#3010) 2024-09-10 15:07:53 +02:00
Ivan FB
5f2d87ec71 chore: Bump dependencies for v0.33 (#3017) 2024-09-09 10:45:14 +02:00
Ivan FB
50e15746d1 chore: enhance libpq management (#3015)
* db_postgres: register pg socket fd to chronos better data available awareness
* waku_store protocol: better logs to track time and new store metrics
2024-09-06 11:33:15 +02:00
Simon-Pierre Vivier
ed866321a0 chore: per limit split of PostgreSQL queries (#3008) 2024-09-04 10:17:28 -04:00
Ivan FB
236547ec7d chore: Better timing and requestId detail for slower store db queries (#2994)
* Better timing and requestId detail for store db queries slower than two seconds
* Adapt tests and client to allow sending custom store requestId
2024-08-29 22:56:14 +02:00
Ivan FB
97019896ac chore: remove unused setting from external_config.nim (#3004) 2024-08-29 17:54:37 +02:00
Ivan FB
6eff20507a libwaku: exposing more features (#3003)
- Allow to start or store discv5
- Expose lightpush request operation
- Expose list of connected and mesh peers
- Expose store client
2024-08-29 14:29:02 +02:00
Simon-Pierre Vivier
853025284b fix waku sync config defaults (#3001) 2024-08-28 10:26:38 -04:00
Ivan FB
f1db75262b chore: delivery monitor for store v3 reliability protocol (#2977)
- Use of observer observable pattern to inform delivery_monitor about subscription state
- send_monitor becomes a publish observer of lightpush and relay
- deliver monitor add more protection against possible crash and better logs
- creating a separate proc in store client for delivery monitor
2024-08-27 16:49:46 +02:00
Darshan K
fd653ef0da fix: peer-exchange issue (#2889) 2024-08-23 23:31:30 +05:30
Ivan FB
2e8f2f0076 chore: ANALYZE messages query should be performed regularly (#2986)
---------

Co-authored-by: Richard Ramos <info@richardramos.me>
2024-08-21 19:17:08 +02:00
NagyZoltanPeter
b68cc07261 Distinction between gross/net trafic in bandwidth per shard metric, added bandwidths and request rate panels to single node and fleet dashboards (#2920) 2024-08-21 17:10:29 +02:00
NagyZoltanPeter
3212459f77 chore: liteprotocoltester for simulation and for fleets (#2813)
* Added phase 2 - waku-simulatior integration in README.md

* Enhancement on statistics reports, added list of sent messages with hash, fixed latency calculations

* Enable standalonde running liteprotocoltester agains any waku network/fleet

* Fix missing env vars on run_tester_node.sh

* Adjustment on log levels, fix REST initialization

* Added standalon docker image build, fine tune duplicate detection and logging.

* Adjustments for waku-simulator runs

* Extended liteprotocoltester README.md with docker build

* Fix test inside docker service node connectivity failure

* Update apps/liteprotocoltester/README.md

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>

* Explain minLatency calculation in code comment

---------

Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-08-21 14:54:18 +02:00
gabrielmer
4b9bee99a8 chore: logging received message info via onValidated observer (#2973) 2024-08-19 14:13:28 +02:00
gabrielmer
3f641dff60 chore: deprecating protected topics in favor of protected shards (#2983) 2024-08-19 12:56:22 +02:00
gabrielmer
90b4dc89ff chore: rename NsPubsubTopic (#2974) 2024-08-19 11:29:35 +02:00
Ivan FB
73fbe5c337 postgres_driver limit max num hashes to 100 (#2976) 2024-08-19 11:12:31 +02:00
richΛrd
4a89875a36 chore: print WakuMessageHash as hex strings (#2969) 2024-08-14 21:04:20 +02:00
gabrielmer
1f3162ae5f avoid using the msg key in chronicles (#2970) 2024-08-14 16:40:08 +02:00
Simon-Pierre Vivier
301ce8068c feat: Nwaku Sync (#2403)
* feat: Waku Sync Protocol

* feat: state machine (#2656)

* feat: pruning storage mehcanism (#2673)

* feat: message transfer mechanism & tests (#2688)

* update docker files

* added ENR filed for sync & misc. fixes

* adding new sync range param & fixes

---------

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
2024-08-13 07:27:34 -04:00
gabrielmer
459221f93d pruning excess in relay connections (#2965) 2024-08-12 17:59:11 +02:00
Ivan FB
6ae46c5fff faster retention policy when error and use of detach finalize when needed (#2966) 2024-08-12 10:47:01 +02:00
richΛrd
71946b911f fix: return on insert error (#2956) 2024-08-11 21:35:04 -04:00
Ivan FB
696587fdac chore: Optimize hash queries with lookup table (#2933)
* Upgrade Postgres schema to add messages_lookup table
* Perform optimized query for messageHash-only queries
2024-08-08 21:46:08 +02:00
Aaryamann Challani
b839b1c81f chore(keystore): verbose error message when credential is not found (#2943) 2024-08-07 11:57:03 +02:00
Darshan K
6748142f29 chore: upgrade peer exchange mounting (#2953) 2024-08-06 13:27:25 +05:30
Ivan FB
1ce87c49a8 lightpush better feedback in case the lightpush service node does not have peers (#2951) 2024-08-02 09:45:05 +02:00
gabrielmer
f10a604764 don't start node when it's discv5 only (#2947) 2024-08-01 23:28:00 +03:00
Prem Chaitanya Prathi
2614d93566 fix: network monitor improvements (#2939) 2024-07-30 16:56:49 +03:00
Ivan FB
f9552e133b chore: Simplification of store legacy code (#2931) 2024-07-30 14:05:23 +02:00
Simon-Pierre Vivier
2134ad76b4 feat: store resume (#2919)
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-07-30 07:23:39 -04:00
Simon-Pierre Vivier
7fdabe5ad9 chore: add peer filtering by cluster for waku peer exchange (#2932) 2024-07-29 15:53:43 -04:00
Simon-Pierre Vivier
43e66939b5 fix: add back waku discv5 metrics (#2927)
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-07-26 16:18:14 -04:00
Darshan K
48a79d3012 fix: update and shift unittest (#2934)
* fix: update and shift location of unit test
2024-07-26 16:57:34 +05:30
Sasha
1d850e43dc chore: return all connected peers from REST API (#2923)
* Remove the condition of gathering connected peers with relay and user req/resp protocols.
* Return PeerExchange protocol support of connected nodes with /admin/peers
* Added test for checking return of PeerExchange mounted protocol of connected peer by GET /admin/peers

---------

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2024-07-23 12:58:56 +02:00
gabrielmer
aa9c30655c chore: adding lint job to the CI (#2925) 2024-07-23 13:57:24 +03:00