2378 Commits

Author SHA1 Message Date
Fabiana Cecin
0f1a3a951b
Update logos_delivery/messaging/messaging_client.nim
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2026-06-22 06:22:57 -03:00
Fabiana Cecin
488266888f
fix: harden LogosDelivery facade lifecycle
* MessagingClient.new: return a Result via a new newMessagingClient
   factory instead of quit(QuitFailure) on a recoverable SendService
   failure; no longer aborts the FFI host (restores master's behaviour).
   Callers (facade, FFI mount, tests) updated to the factory.
 * LogosDelivery.stop(): best-effort run-to-completion; stop and nil every
   component even if a prior one fails, aggregating errors (was: early
   return that leaked the messaging client and the node).
 * LogosDelivery.getNodeInfo(): nil-guard self.waku like the sibling
   getters (was: nil-deref before start / after stop).
2026-06-19 12:09:36 -03:00
NagyZoltanPeter
43452e200b
fix: update nix dependencies 2026-06-19 13:51:44 +02:00
NagyZoltanPeter
47f3194dfd
rename event in test 2026-06-19 12:51:00 +02:00
NagyZoltanPeter
15c4e3b918
nph fix 2026-06-19 12:07:28 +02:00
NagyZoltanPeter
4af5ee86fd
Fixes after rebase 2026-06-19 11:58:13 +02:00
NagyZoltanPeter
3006eefdbe
set nim-brokers to v3.1.4 fixed version 2026-06-19 10:21:19 +02:00
NagyZoltanPeter
6bd8873094
Wire KernelInterface onto Waku; non-API broker conversion
Squash of the kernel-wiring work (d86f0651..535fe2c9):
- Don't exercise FFI of Brokers
- Fix tests and examples to compile; verify product unchanged
- bump nim-brokers to v3.1.3
- Events dropListeners changed to async
- WIP + finalize KernelInterface <-> Waku wiring
2026-06-19 08:50:18 +02:00
NagyZoltanPeter
8456245714
Structured Broker API: interface/impl split + nim-brokers integration
Squash of the initial structured-API POC work (1d684bf8..c37dcf84):
- WIP: structured Broker API interface + impl (Logos Delivery facade)
- Restructured folders; interfaces under logos_delivery/api; MessagingClient
  and ReliableChannelManager reworked as BrokerInterface/BrokerImplement
- API types elevated under logos_delivery/api/types.nim
- onelogosdelivery target; wakunode2 / FFI lib build fixes
- nim-brokers integration + version bumps; git_version into FFI lib version
- compile fixes for tests
2026-06-19 08:25:25 +02:00
Simon-Pierre Vivier
a73035e28d
feat: integrate service discovery (#3947)
---------

Co-authored-by: Fabiana Cecin <fabiana@waku.org>
2026-06-18 12:51:27 -04:00
Fabiana Cecin
03efe6766c
Messaging backfill from store only when regaining connectivity (#3957)
* ConnectionStatus transition to connected now trigger store query
* Query period computed over period in which the node was disconnected
* remove periodic 5min store query
* add connection status edge-triggered test case
* refactor RecvService test suite

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2026-06-18 13:23:49 -03:00
Tanya S
02ad7783bf
Add tests for merkle root and proof updates (#3950)
* make generateProof async and add ensureFreshMerkleProofPath

* Use Wakumessage.new()

* Add trigger for client side only merkleproofcache updates

* full decoupling of updateRoots and merkleproofcache update

* Add tests for on demand merkle path updates

* debugging WIP

* fix for getTransactionReceipt endless loop

* clean up group manager logs

* Remove unused code from rebase mistake

* Improve Anvil proc for RLN testing

* fixing from rebase

* Fix message ref in tests and clean up logs in utils_onchain

* Clean up more logs

* Change root update log to trace

* Increase approved token amount for RLN registration testing

* Reduce comments in utils_onchain

* simplify benchmarks test result output

* Add test names

* remove duplicated tests for test_rln_groupmanager

* Trim group_manager tests

* Remove long test for local window of roots and improve amvil test performance

* Optimise tests for group_manager
2026-06-18 14:50:04 +02:00
Ivan FB
066838aa03
chore: address deferred SDS-handler review comments from #3942 (#3960) 2026-06-17 20:28:43 +02:00
Fabiana Cecin
c3090fb62f
feat: QUIC transport support (#3951)
* additive quic transport, off by default (--quic-support)
* add QuicConf + QuicConfBuilder, --quic-support / --quic-port flags
* net_config announces quic-v1 host/ext/dns4 addrs, adds quic-v1 to enr multiaddrs
* newWakuSwitch mounts quic transport when a quic addr is set
* toRemotePeerInfo: quic from enr multiaddrs ext, sorted quic-first
* BoundPorts.quic, read back the bound quic port at start (handles --quic-port=0)
* skip auto quic addr when operator supplies one via --ext-multiaddr
* tests: nodes dual-stack by default, tcp-only where single transport asserted
* tests: drop hardcoded ephemeral ports (port 0) to fix quic-churn bind flakes
* use setupNat to discover NAT-mapped UDP port when QUIC enabled
2026-06-17 07:55:45 -03:00
Darshan
7e98489a24
feat: wire ReliabilityManager into the Reliable Channel (#3942) 2026-06-16 10:40:59 +05:30
Fabiana Cecin
6837ae0c1f
feat: bump nim-libp2p to v2.0.0 (#3929)
* bump nim-libp2p pin to v2.0.0 tag
* bump json_rpc to v0.6.1, lsquic to v0.5.1, boringssl to v0.0.8 (latest tags)
* add libp2p_mix dep; repoint libp2p/protocols/mix -> libp2p_mix
* pin nimble.lock: websock / protobuf_serialization / npeg / jwt
* Makefile: add -d:libp2p_quic_support
* regenerate nix/deps.nix (adds libp2p_mix, refreshes pins)
* migrate rng ref HmacDrbgContext -> libp2p Rng across prod/channels/tests (interface-only; same DRBG)
* waku_switch: TransportConfig factory; unified 2.0.0 connection limits (withMaxInOut, withMaxConnections); local MaxConnections
* waku_relay/rendezvous/discv5/kademlia: v2.0.0 API (rng, config, ServiceDiscovery rename)
* call Service.setup() on post-build switch services (2.0.0 split setup/start)
* drop libp2p/utils/semaphore -> chronos AsyncSemaphore
* add logos_delivery/waku/compat/option_valueor shim (Option[T] valueOr/withValue, dropped upstream)
* add std/options where a transitive re-export was removed
* add newStandardSwitch shim (libp2p removed it in 2.0.0); mounts yamux+mplex to match prod muxer
* PeerId.random(rng); common.rng()/crypto.newRng(); hoist shared rng (instantiation cleanup)
* update expectations for 2.0.0 defaults: DEFAULT_PROTOCOLS += /ipfs/id/push/1.0.0; agent "nim-libp2p"
* drop relay reboot/reconnect test (asserted a Switch restart capability that is simply not supported)
* fix up a few tests that were flaking on MacOS (libp2p upgrade may have exposed these)
2026-06-15 09:56:15 -03:00
Ivan FB
54c890856f
feat: unify libs into a single liblogosdelivery (#3949) 2026-06-15 13:03:44 +02:00
Darshan
9289ba6220
chore: update release issue templates to logos fleets (#3952) 2026-06-13 10:48:29 +05:30
Tanya S
22040b739f
feat: decouple merkle path and on-demand strategy (#3940)
* make validateRoots async

* add on-demand refreshRoots functionality

* Move max rootsrefresh time to constants

* make generateProof async and add ensureFreshMerkleProofPath

* Update to match code format and linting

* Use Wakumessage.new()

* Add trigger for client side only merkleproofcache updates

* full decoupling of updateRoots and merkleproofcache update

* Fix isNil check format

* Move moment check to top of roots and merkle path update procs

* Update PathCheckMinInterval

* Add tests for on demand merkle path updates

* Replace appendRLNProof and use message.toRLNSignal

* Fix linting

* Remove commented code

* Remove more old commented code

* Fix formatting and simplifications
2026-06-12 12:22:34 +02:00
Fabiana Cecin
b7c2cee2c9
fix(pr-lint): grant the comment job pull-requests:write (#3944) 2026-06-11 10:31:53 +02:00
Fabiana Cecin
d712da5a54
accept CLI option names in JSON config (#3943) 2026-06-11 10:31:33 +02:00
Fabiana Cecin
41b5c4906f
feat: Improve config (v2) (#3925)
* rename NetworkConf -> NetworkPresetConf and related procs/vars
* Rewrite applyNetworkPresetConf to apply user-set fields over preset fields
* New dedicated parser for configJson
* Fix tests to use actual extract JSON nodeconf parser
* Change all confbuilder defaults from literal values to DefaultXXX consts
* Change int/bool WakuNodeConf fields to Option to get user intent w/o sentinels
* Make Option CLI default-value help mention defaults now owned by confbuilder
* Document CLI defaults that differ from confbuilder defaults
* Fix agent-string builder default deviating from CLI default
* Add WakuConfBuilder.enforceSecurityConstraints()
* Fail on RLN user preset overrides instead of drop-and-continue
* Add regression tests for initial set of conf constraints
* fix kademliaDiscoveryConfBuilder.build() enable/disable kad logic
* Misc refactors, fixes
* Add tests

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2026-06-10 09:09:22 -03:00
Tanya S
362c35f2fb
Install specific foundry anvil version directly (#3937)
* Install specific foundry anvil version directly

* clean verbose comments
2026-06-09 14:25:13 +02:00
Ivan FB
c7350abb58
clean waku_noise because it is not used in prod code (#3934) 2026-06-09 10:32:42 +02:00
Darshan
faa6741311
fix: build zerokit rln from source via fixed nixpkgs, drop prebuilt fetch (#3930) 2026-06-08 18:42:26 +05:30
Ivan FB
3b03ca29b1
refactor: introduce proper logos_delivery layers folder structure (#3935)
Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2026-06-08 13:37:53 +02:00
Ivan FB
38d951a2fd
Rename kernel_api dir to waku_node and tidy node module layout (#3927) 2026-06-04 23:06:54 +02:00
Ivan FB
6fd0f9c079
ci: fix Windows build hang on re-downloading nimble deps (#3920) 2026-06-04 21:17:43 +02:00
Fabiana Cecin
f833ded209
Clean separation between ReliableChannelManager, MessagingClient, and kernel/core (#3918)
* Convert DeliveryService into optionally mountable MessagingClient
* Move SubscriptionManager to core layer (WakuNode)
* Ensure libwaku kernel_api/ still works (deprecated; removal pending)
* Create node_types.nim to allow WakuNode to compose subsystems cleanly
* Create node_telemetry.nim to centralize Prometheus types
* Remove unnecessary "ptr Waku" / "addr waku" indirection
* Rename Waku.startWaku -> Waku.start for upcoming Waku rename
* Write complete proc surface for SubscriptionManager (all intents expressible)
* Rename edgeFilterHealthLoop -> edgeFilterConnectionLoop ("Health" means monitoring)
* logosdelivery_start_node calls mountMessagingClient then starts
* libwaku and wakunode2 do not mount messagingClient
* Improve edge filter peer cleanup on disconnect
* misc refactors/moves, improvements, fixes

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2026-06-04 15:53:27 -03:00
Tanya S
86e424c82c
chore: retrieve cache of merkle roots from RLN contract (#3903)
* Only add new roots, not all received

* Fix error in removing recent roots not checking AcceptableWindowSize

* fix merging

* more merging fixes

* merge fixes

* add test for updated merkle roots window

* add pr re-add gauge for proof-generation-duration-seconds

* Decrease AcceptableRootWindowSize for testing

* debug spam log

* linting

* start trackRootChanges call loop immediately

* Fix 5s delay trackRootChanges

* set rpcDelay for root tracking to 10s

* add default params to sendEthCallWithParams

* improve recents roots retrieval and logs

* Use updateRecentRoots to track root changes

* simplify updateRecentRoots

* set root polling to 15s

* set rpc poll delay to 30s

* set acceptablerootwindowsize and root poll delay

* Improve test 'should fetch history correctly' for root cache

* Make root cache handling more efficient

* add contract root cache size as constant and function use fix

* updateRecentRoots comments update

* Update group_manager and tests

* fix linting
2026-06-04 16:02:25 +02:00
NagyZoltanPeter
deb6929670
feat: introduce SDS persistency glue (#3913)
* persistency: follow nim-sds 0.3.0 snapshot persistence contract
nim-sds 0.3.0 replaced the ~14 fine-grained per-row Persistence callbacks
with a 5-proc snapshot model (saveChannelMeta / updateHistory / loadChannel
/ dropChannel / setRetrievalHint), all returning Future[Result[...]].

Rewrite waku/persistency/sds_persistency.nim accordingly:
- ChannelMeta is stored as one blob per channel; the message log as
  append/evict rows. Categories collapse from 7 to 2 (sds.meta, sds.log).
- Blob transform uses nim-sds' own codecs: snapshot_codec (schema-versioned
  protobuf) for ChannelMeta, the SDS wire codec for SdsMessage log rows. The
  generic payload_codec/BlobCodec path is retired (removed payload_codec.nim
  and test_blob_codec.nim).
- setRetrievalHint is a deliberate no-op: persisted hints are never read back
  (loadChannel/ChannelMeta carry none; hints are supplied live via the
  onRetrievalHint provider). The closure stays because the field is required.
- Fix the module import spelling (srcDir="sds" => bare module paths), which
  the previous adapter got wrong and never compiled against the locked deps.

Add tests/persistency/test_sds_persistency.nim (round-trip, empty-load,
evict, drop) replacing test_blob_codec in test_all. Full persistency suite
passes 74/74 under both refc and ORC.
* Bump to latest nim-sds and nim-brokers 3.1.1
* Update with latest nim-sds changes - removal of setRetrievalHints - not needed
2026-06-04 10:53:02 +02:00
Ivan FB
4099ff2638
Pin nim-ffi to v0.1.3 in waku.nimble (#3928) 2026-06-03 19:30:51 +02:00
NagyZoltanPeter
64a0ed7d96
Add helper nimble task to ease nph formatting on branch/pr's changed nim files -> nimble nphchanges (#3926) 2026-06-02 18:25:51 +02:00
NagyZoltanPeter
b593d16d11
tools: add sync-nimble-lock.sh to cross-check waku.nimble pins into nimble.lock (#3924)
Adds a portable (macOS bash 3.2 / Linux) helper that detects git-URL pinned
`requires` in waku.nimble which changed vs a git base ref (default HEAD) and
updates ONLY those nimble.lock entries — version, vcsRevision and the sha1
checksum — leaving every other entry byte-for-byte untouched.

It does not run `nimble lock` (which rewrites the whole file). The sha1 is
computed directly, reproducing nimble's algorithm from
src/nimblepkg/checksums.nim (git ls-files -> sort -> SHA1 over path +
symlink-target/file-bytes). Resolves tags to commits via git rev-parse and
guards against invalid commit hashes (e.g. a stray leading character).

Dry-run by default (exit 1 on drift); --apply writes; --base REF to compare
against another ref. Requires git + python3; nimble not required.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 18:25:21 +02:00
Ivan FB
8b0e21fada
enhance reliable channel segment states (#3919) 2026-06-02 14:16:13 +02:00
Ivan FB
5bc1ad63a7
ci: pass -d:disableMarchNative to avoid secp256k1 build failures (#3916) 2026-05-29 22:28:15 +02:00
Ivan FB
c5b24e21da
better pending segments management (#3914)
Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2026-05-29 22:24:46 +02:00
Darshan
bb23ee64af
feat: fetch prebuilt zerokit rln, fall back to source build (#3915) 2026-05-29 23:53:38 +05:30
Ivan FB
2447ce9e73
disable js-waku from ci (#3917) 2026-05-29 08:11:41 +02:00
Ivan FB
74057c6622
start basic reliable channel folder (#3886) nightly 2026-05-27 23:05:20 +02:00
Darshan
5e262badf7
chore: fixing daily ci (#3878) 2026-05-27 23:58:30 +05:30
Tanya S
8b53e64379
Remove makefile target update (#3897)
* Remove makefile target update

* fix: set execute permission on install_nimble.sh

* improve install_nim script

* skip second nim install on Windows

* fix path check in install-nim

* Makefile workfile reordering
2026-05-27 10:40:54 +02:00
Ivan FB
79dda63758
Recover wakucanary in nix output (#3892) 2026-05-26 16:22:10 +02:00
Fabiana Cecin
4b80c7762d
Point CLAUDE.md to (updated) AGENTS.md
* Point CLAUDE.md to (updated) AGENTS.md
* add cluster id, shard id, autosharding
* add in-flight testing section

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2026-05-25 18:22:04 -03:00
Igor Sirotin
c738c7b65e
fix: accept port 0 in JSON config (ephemeral port support) (#3895)
* chore: pin confutils to merged upstream commit

status-im/nim-confutils#146 is merged; move the confutils pin from the
PR fork back to status-im/nim-confutils master (36f3115). Content is
identical to the fork commit, so nimble sha1 and nix sha256 are unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 14:32:54 +01:00
Darshan
67eebe3a02
ci: add daily rln simulator e2e workflow (#3885) 2026-05-22 17:15:31 +05:30
Tanya S
04ef12ccf3
Update and Improve READMEs (#3894)
* Update READMEs for new nimble build system

* Move recommended nim versions out of prerequisites
2026-05-22 13:23:58 +02:00
Igor Sirotin
bdd562ecc6
feat(nix): expose cargoHash-corrected librln as packages.rln (#3902)
PR #3899 fixes zerokit v2.0.2's stale cargoHash, but only via an internal
`let` binding consumed by liblogosdelivery. Downstream consumers (e.g.
logos-delivery-module) that need librln still pull zerokit's rln package
directly and hit the stale hash.

Expose that corrected derivation as `packages.<system>.rln` so consumers can
bundle the exact same librln this build links, instead of overriding the
cargoHash themselves.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 11:51:52 +01:00
5ff734aa56
chore: simplify zerokit cargoHash fix (#3899)
Replacing the whole package is unnecessary. Just hash can be fixed.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2026-05-21 21:46:10 +01:00
Igor Sirotin
29a77dcf4d
feat: add logos.test fleet preset (#3900)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 18:29:33 +01:00