2381 Commits

Author SHA1 Message Date
Ivan FB
89474e7236
Expose reliable-channel ops in the stable C header (#3851)
The library already ships as a single .so with a tiered header surface
(liblogosdelivery.h = stable Messaging/Reliable-Channels, liblogosdelivery_kernel.h
= advanced Kernel). Per that tiering, the reliable-channel ops belong on the
stable surface, so declare channel_create / channel_send / channel_close in
liblogosdelivery.h and document the channel lifecycle events delivered through
the event callback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 15:43:39 +02:00
Ivan FB
369ad1b430
Add reliable-channel FFI ops + events (nim-ffi v0.1.3)
Expose the reliable-channel layer through the v0.1.3 FFI:
- channel_create / channel_send / channel_close call the
  ReliableChannelManager api (createReliableChannel / send / closeChannel),
  marshalling channel id + base64 payload + ephemeral by hand
- channel message received / sent / errored are surfaced by listening to the
  channel-layer broker events in start_node and forwarding them through
  callEventCallback (received payload base64-encoded), dropped in stop_node

Stays on nim-ffi v0.1.3 (no typed/CBOR rewrite).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 15:27:55 +02:00
Ivan FB
aca652008a
Reshape per-layer API into api/ folders and thin the FFI over them
Each layer now separates its constructible core from its public surface:

  - core module (waku.nim / messaging_client.nim /
    reliable_channel_manager.nim): the type plus new/start/stop and the
    private construction helpers.
  - api/ folder: one module per differentiated set of operations
    (waku: topics/relay/filter/lightpush/store/peer_manager/discovery/
    debug/health) plus an events surface.

The waku api is reshaped to be the complete operation surface the C
bindings need, so the library no longer reaches into node internals:
relayPublish returns the message hash, relaySubscribe takes an optional
handler, filter/lightpush auto-select the service peer, connectedPeersInfo
returns structured data, pingPeer honours the timeout, plus
relayNumPeersInMesh / relayNumConnectedPeers / isOnline. library/ is now a
thin C-ABI shim: each {.ffi.} proc only marshals cstring/JSON/callbacks and
delegates to ctx.myLib[].waku.<op> (or messagingClient.<op>).
app_callbacks re-exports the modules defining its handler types, which the
included FFI files previously relied on by leakage.

Events move next to the surface that owns them, with each dependency kept
pointing the right way:

  - waku/events/ relocated under waku/api/events/.
  - channel events live in channels/api/events.nim.
  - the four messaging-level message events move to messaging/api/events;
    MessageSeenEvent stays in waku because it is emitted by waku core, so
    moving it would make waku depend on the messaging layer.
  - delivery_events renamed to filter_subscribe_events to match the
    OnFilterSubscribe/Unsubscribe events it actually declares.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 13:00:18 +02:00
NagyZoltanPeter
6d35800fce
Chore: api shape phase2 (#3974)
* Move waku.nim from waku/factory to under waku/
* remove unused
* Realize Kernel API in scope of Waku class
* Refactor waku/api into messaging_client, waku/api/types and api_conf into logos_delivery/api
* Make liblogosdelivery and wakunode2 compile, remove waku/api.nim as it was just a import orchestrator
* make test compile and run
* Reconcile master's new send tests to LogosDelivery API after rebase

master commits #3965/#3669-followup added two test cases (Edge lightpush
delivery #3847, store-validation timeout) written against the removed
waku/api.nim createNode helper. Rewrite them to the LogosDelivery shape:
createNode -> LogosDelivery.new, node.node -> node.waku.node,
node.brokerCtx -> node.waku.brokerCtx, node.send -> node.messagingClient.send,
and drop the now-implicit mountMessagingClient calls (LogosDelivery.new
mounts the client internally).
2026-06-25 09:27:01 +02:00
Fabiana Cecin
56b92cfd71
chore: Fix .md docs (fix broken links, add QUIC notes) (#3988) 2026-06-25 09:11:52 +02:00
Ivan FB
5c20f48ab3
chore: build C/C++ examples on macOS via POSIX getopt (#3982) 2026-06-24 14:44:44 +02:00
Tanya S
b279533214
Apply LogosDelivery Orchestrator changes to api_send tests (#3976) 2026-06-24 10:43:16 +02:00
NagyZoltanPeter
8501d051a1
fix - Cap store checks on propagated messages by MessagingClient (#3965)
* Frame send service store checks and cap to 1 min per task

* Add test for store not verify case
2026-06-23 12:22:16 +02:00
Ivan FB
5309ce294b
fix: retry send tasks stuck in Entry state 2026-06-23 11:39:35 +02:00
Ivan FB
1a3b3204fa
feat: add LogosDelivery orchestrator as project entry point for FFI (#3970) 2026-06-23 01:20:09 +02:00
NagyZoltanPeter
2fe7e1c373
Workaround MacOS build issue caused by between new bearssl 0.2.9 and crhonos 2.2.4 (#3969) 2026-06-22 10:01:54 +02:00
Ivan FB
319c0c68fe
chore: simplify release process — merge deploy into prepare_release (#3964)
Co-authored-by: Igor Sirotin <sirotin@status.im>
2026-06-19 15:29:33 +02:00
Simon-Pierre Vivier
a73035e28d
feat: integrate service discovery (#3947)
---------

Co-authored-by: Fabiana Cecin <fabiana@waku.org>
2026-06-18 12:51:27 -04:00
Fabiana Cecin
03efe6766c
Messaging backfill from store only when regaining connectivity (#3957)
* ConnectionStatus transition to connected now trigger store query
* Query period computed over period in which the node was disconnected
* remove periodic 5min store query
* add connection status edge-triggered test case
* refactor RecvService test suite

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2026-06-18 13:23:49 -03:00
Tanya S
02ad7783bf
Add tests for merkle root and proof updates (#3950)
* make generateProof async and add ensureFreshMerkleProofPath

* Use Wakumessage.new()

* Add trigger for client side only merkleproofcache updates

* full decoupling of updateRoots and merkleproofcache update

* Add tests for on demand merkle path updates

* debugging WIP

* fix for getTransactionReceipt endless loop

* clean up group manager logs

* Remove unused code from rebase mistake

* Improve Anvil proc for RLN testing

* fixing from rebase

* Fix message ref in tests and clean up logs in utils_onchain

* Clean up more logs

* Change root update log to trace

* Increase approved token amount for RLN registration testing

* Reduce comments in utils_onchain

* simplify benchmarks test result output

* Add test names

* remove duplicated tests for test_rln_groupmanager

* Trim group_manager tests

* Remove long test for local window of roots and improve amvil test performance

* Optimise tests for group_manager
2026-06-18 14:50:04 +02:00
Ivan FB
066838aa03
chore: address deferred SDS-handler review comments from #3942 (#3960) 2026-06-17 20:28:43 +02:00
Fabiana Cecin
c3090fb62f
feat: QUIC transport support (#3951)
* additive quic transport, off by default (--quic-support)
* add QuicConf + QuicConfBuilder, --quic-support / --quic-port flags
* net_config announces quic-v1 host/ext/dns4 addrs, adds quic-v1 to enr multiaddrs
* newWakuSwitch mounts quic transport when a quic addr is set
* toRemotePeerInfo: quic from enr multiaddrs ext, sorted quic-first
* BoundPorts.quic, read back the bound quic port at start (handles --quic-port=0)
* skip auto quic addr when operator supplies one via --ext-multiaddr
* tests: nodes dual-stack by default, tcp-only where single transport asserted
* tests: drop hardcoded ephemeral ports (port 0) to fix quic-churn bind flakes
* use setupNat to discover NAT-mapped UDP port when QUIC enabled
2026-06-17 07:55:45 -03:00
Darshan
7e98489a24
feat: wire ReliabilityManager into the Reliable Channel (#3942) 2026-06-16 10:40:59 +05:30
Fabiana Cecin
6837ae0c1f
feat: bump nim-libp2p to v2.0.0 (#3929)
* bump nim-libp2p pin to v2.0.0 tag
* bump json_rpc to v0.6.1, lsquic to v0.5.1, boringssl to v0.0.8 (latest tags)
* add libp2p_mix dep; repoint libp2p/protocols/mix -> libp2p_mix
* pin nimble.lock: websock / protobuf_serialization / npeg / jwt
* Makefile: add -d:libp2p_quic_support
* regenerate nix/deps.nix (adds libp2p_mix, refreshes pins)
* migrate rng ref HmacDrbgContext -> libp2p Rng across prod/channels/tests (interface-only; same DRBG)
* waku_switch: TransportConfig factory; unified 2.0.0 connection limits (withMaxInOut, withMaxConnections); local MaxConnections
* waku_relay/rendezvous/discv5/kademlia: v2.0.0 API (rng, config, ServiceDiscovery rename)
* call Service.setup() on post-build switch services (2.0.0 split setup/start)
* drop libp2p/utils/semaphore -> chronos AsyncSemaphore
* add logos_delivery/waku/compat/option_valueor shim (Option[T] valueOr/withValue, dropped upstream)
* add std/options where a transitive re-export was removed
* add newStandardSwitch shim (libp2p removed it in 2.0.0); mounts yamux+mplex to match prod muxer
* PeerId.random(rng); common.rng()/crypto.newRng(); hoist shared rng (instantiation cleanup)
* update expectations for 2.0.0 defaults: DEFAULT_PROTOCOLS += /ipfs/id/push/1.0.0; agent "nim-libp2p"
* drop relay reboot/reconnect test (asserted a Switch restart capability that is simply not supported)
* fix up a few tests that were flaking on MacOS (libp2p upgrade may have exposed these)
2026-06-15 09:56:15 -03:00
Ivan FB
54c890856f
feat: unify libs into a single liblogosdelivery (#3949) 2026-06-15 13:03:44 +02:00
Darshan
9289ba6220
chore: update release issue templates to logos fleets (#3952) 2026-06-13 10:48:29 +05:30
Tanya S
22040b739f
feat: decouple merkle path and on-demand strategy (#3940)
* make validateRoots async

* add on-demand refreshRoots functionality

* Move max rootsrefresh time to constants

* make generateProof async and add ensureFreshMerkleProofPath

* Update to match code format and linting

* Use Wakumessage.new()

* Add trigger for client side only merkleproofcache updates

* full decoupling of updateRoots and merkleproofcache update

* Fix isNil check format

* Move moment check to top of roots and merkle path update procs

* Update PathCheckMinInterval

* Add tests for on demand merkle path updates

* Replace appendRLNProof and use message.toRLNSignal

* Fix linting

* Remove commented code

* Remove more old commented code

* Fix formatting and simplifications
2026-06-12 12:22:34 +02:00
Fabiana Cecin
b7c2cee2c9
fix(pr-lint): grant the comment job pull-requests:write (#3944) 2026-06-11 10:31:53 +02:00
Fabiana Cecin
d712da5a54
accept CLI option names in JSON config (#3943) 2026-06-11 10:31:33 +02:00
Fabiana Cecin
41b5c4906f
feat: Improve config (v2) (#3925)
* rename NetworkConf -> NetworkPresetConf and related procs/vars
* Rewrite applyNetworkPresetConf to apply user-set fields over preset fields
* New dedicated parser for configJson
* Fix tests to use actual extract JSON nodeconf parser
* Change all confbuilder defaults from literal values to DefaultXXX consts
* Change int/bool WakuNodeConf fields to Option to get user intent w/o sentinels
* Make Option CLI default-value help mention defaults now owned by confbuilder
* Document CLI defaults that differ from confbuilder defaults
* Fix agent-string builder default deviating from CLI default
* Add WakuConfBuilder.enforceSecurityConstraints()
* Fail on RLN user preset overrides instead of drop-and-continue
* Add regression tests for initial set of conf constraints
* fix kademliaDiscoveryConfBuilder.build() enable/disable kad logic
* Misc refactors, fixes
* Add tests

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2026-06-10 09:09:22 -03:00
Tanya S
362c35f2fb
Install specific foundry anvil version directly (#3937)
* Install specific foundry anvil version directly

* clean verbose comments
2026-06-09 14:25:13 +02:00
Ivan FB
c7350abb58
clean waku_noise because it is not used in prod code (#3934) 2026-06-09 10:32:42 +02:00
Darshan
faa6741311
fix: build zerokit rln from source via fixed nixpkgs, drop prebuilt fetch (#3930) 2026-06-08 18:42:26 +05:30
Ivan FB
3b03ca29b1
refactor: introduce proper logos_delivery layers folder structure (#3935)
Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2026-06-08 13:37:53 +02:00
Ivan FB
38d951a2fd
Rename kernel_api dir to waku_node and tidy node module layout (#3927) 2026-06-04 23:06:54 +02:00
Ivan FB
6fd0f9c079
ci: fix Windows build hang on re-downloading nimble deps (#3920) 2026-06-04 21:17:43 +02:00
Fabiana Cecin
f833ded209
Clean separation between ReliableChannelManager, MessagingClient, and kernel/core (#3918)
* Convert DeliveryService into optionally mountable MessagingClient
* Move SubscriptionManager to core layer (WakuNode)
* Ensure libwaku kernel_api/ still works (deprecated; removal pending)
* Create node_types.nim to allow WakuNode to compose subsystems cleanly
* Create node_telemetry.nim to centralize Prometheus types
* Remove unnecessary "ptr Waku" / "addr waku" indirection
* Rename Waku.startWaku -> Waku.start for upcoming Waku rename
* Write complete proc surface for SubscriptionManager (all intents expressible)
* Rename edgeFilterHealthLoop -> edgeFilterConnectionLoop ("Health" means monitoring)
* logosdelivery_start_node calls mountMessagingClient then starts
* libwaku and wakunode2 do not mount messagingClient
* Improve edge filter peer cleanup on disconnect
* misc refactors/moves, improvements, fixes

Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2026-06-04 15:53:27 -03:00
Tanya S
86e424c82c
chore: retrieve cache of merkle roots from RLN contract (#3903)
* Only add new roots, not all received

* Fix error in removing recent roots not checking AcceptableWindowSize

* fix merging

* more merging fixes

* merge fixes

* add test for updated merkle roots window

* add pr re-add gauge for proof-generation-duration-seconds

* Decrease AcceptableRootWindowSize for testing

* debug spam log

* linting

* start trackRootChanges call loop immediately

* Fix 5s delay trackRootChanges

* set rpcDelay for root tracking to 10s

* add default params to sendEthCallWithParams

* improve recents roots retrieval and logs

* Use updateRecentRoots to track root changes

* simplify updateRecentRoots

* set root polling to 15s

* set rpc poll delay to 30s

* set acceptablerootwindowsize and root poll delay

* Improve test 'should fetch history correctly' for root cache

* Make root cache handling more efficient

* add contract root cache size as constant and function use fix

* updateRecentRoots comments update

* Update group_manager and tests

* fix linting
2026-06-04 16:02:25 +02:00
NagyZoltanPeter
deb6929670
feat: introduce SDS persistency glue (#3913)
* persistency: follow nim-sds 0.3.0 snapshot persistence contract
nim-sds 0.3.0 replaced the ~14 fine-grained per-row Persistence callbacks
with a 5-proc snapshot model (saveChannelMeta / updateHistory / loadChannel
/ dropChannel / setRetrievalHint), all returning Future[Result[...]].

Rewrite waku/persistency/sds_persistency.nim accordingly:
- ChannelMeta is stored as one blob per channel; the message log as
  append/evict rows. Categories collapse from 7 to 2 (sds.meta, sds.log).
- Blob transform uses nim-sds' own codecs: snapshot_codec (schema-versioned
  protobuf) for ChannelMeta, the SDS wire codec for SdsMessage log rows. The
  generic payload_codec/BlobCodec path is retired (removed payload_codec.nim
  and test_blob_codec.nim).
- setRetrievalHint is a deliberate no-op: persisted hints are never read back
  (loadChannel/ChannelMeta carry none; hints are supplied live via the
  onRetrievalHint provider). The closure stays because the field is required.
- Fix the module import spelling (srcDir="sds" => bare module paths), which
  the previous adapter got wrong and never compiled against the locked deps.

Add tests/persistency/test_sds_persistency.nim (round-trip, empty-load,
evict, drop) replacing test_blob_codec in test_all. Full persistency suite
passes 74/74 under both refc and ORC.
* Bump to latest nim-sds and nim-brokers 3.1.1
* Update with latest nim-sds changes - removal of setRetrievalHints - not needed
2026-06-04 10:53:02 +02:00
Ivan FB
4099ff2638
Pin nim-ffi to v0.1.3 in waku.nimble (#3928) 2026-06-03 19:30:51 +02:00
NagyZoltanPeter
64a0ed7d96
Add helper nimble task to ease nph formatting on branch/pr's changed nim files -> nimble nphchanges (#3926) 2026-06-02 18:25:51 +02:00
NagyZoltanPeter
b593d16d11
tools: add sync-nimble-lock.sh to cross-check waku.nimble pins into nimble.lock (#3924)
Adds a portable (macOS bash 3.2 / Linux) helper that detects git-URL pinned
`requires` in waku.nimble which changed vs a git base ref (default HEAD) and
updates ONLY those nimble.lock entries — version, vcsRevision and the sha1
checksum — leaving every other entry byte-for-byte untouched.

It does not run `nimble lock` (which rewrites the whole file). The sha1 is
computed directly, reproducing nimble's algorithm from
src/nimblepkg/checksums.nim (git ls-files -> sort -> SHA1 over path +
symlink-target/file-bytes). Resolves tags to commits via git rev-parse and
guards against invalid commit hashes (e.g. a stray leading character).

Dry-run by default (exit 1 on drift); --apply writes; --base REF to compare
against another ref. Requires git + python3; nimble not required.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 18:25:21 +02:00
Ivan FB
8b0e21fada
enhance reliable channel segment states (#3919) 2026-06-02 14:16:13 +02:00
Ivan FB
5bc1ad63a7
ci: pass -d:disableMarchNative to avoid secp256k1 build failures (#3916) 2026-05-29 22:28:15 +02:00
Ivan FB
c5b24e21da
better pending segments management (#3914)
Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2026-05-29 22:24:46 +02:00
Darshan
bb23ee64af
feat: fetch prebuilt zerokit rln, fall back to source build (#3915) 2026-05-29 23:53:38 +05:30
Ivan FB
2447ce9e73
disable js-waku from ci (#3917) 2026-05-29 08:11:41 +02:00
Ivan FB
74057c6622
start basic reliable channel folder (#3886) nightly 2026-05-27 23:05:20 +02:00
Darshan
5e262badf7
chore: fixing daily ci (#3878) 2026-05-27 23:58:30 +05:30
Tanya S
8b53e64379
Remove makefile target update (#3897)
* Remove makefile target update

* fix: set execute permission on install_nimble.sh

* improve install_nim script

* skip second nim install on Windows

* fix path check in install-nim

* Makefile workfile reordering
2026-05-27 10:40:54 +02:00
Ivan FB
79dda63758
Recover wakucanary in nix output (#3892) 2026-05-26 16:22:10 +02:00
Fabiana Cecin
4b80c7762d
Point CLAUDE.md to (updated) AGENTS.md
* Point CLAUDE.md to (updated) AGENTS.md
* add cluster id, shard id, autosharding
* add in-flight testing section

Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
2026-05-25 18:22:04 -03:00
Igor Sirotin
c738c7b65e
fix: accept port 0 in JSON config (ephemeral port support) (#3895)
* chore: pin confutils to merged upstream commit

status-im/nim-confutils#146 is merged; move the confutils pin from the
PR fork back to status-im/nim-confutils master (36f3115). Content is
identical to the fork commit, so nimble sha1 and nix sha256 are unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 14:32:54 +01:00
Darshan
67eebe3a02
ci: add daily rln simulator e2e workflow (#3885) 2026-05-22 17:15:31 +05:30
Tanya S
04ef12ccf3
Update and Improve READMEs (#3894)
* Update READMEs for new nimble build system

* Move recommended nim versions out of prerequisites
2026-05-22 13:23:58 +02:00