* chore: drop rate-limit stage from reliable channel send pipeline
Collapses the outgoing pipeline to `segmentation -> sds -> encryption ->
dispatch` by folding the encrypt-and-dispatch tail of `onReadyToSend`
directly into `send()`. Removes the `RateLimitManager` field, its
constructor param, the `ReadyToSendEvent` listener, and the
`awaitingDispatch` accounting that only existed to bridge the event-bus
hop between `send()` and `onReadyToSend`.
The `rate_limit_manager.nim` module itself is untouched — it will be
relocated to the messaging layer (co-located with RLN) in a follow-up
commit, where per-epoch admission actually belongs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: relocate RateLimitManager to messaging layer
Moves rate_limit_manager.nim from `logos_delivery/channels/` to
`logos_delivery/messaging/rate_limit_manager/` — the correct owner
now that admission sits alongside RLN in the messaging layer instead
of being fanned out to per-channel event listeners.
The API surface changes:
- `enqueueToSend` + `ReadyToSendEvent` (broker-based fan-out to a
ReliableChannel listener) is replaced by `admit(msg): Future[Result[
void, RateLimitError]]`. Callers now branch directly on the result
instead of subscribing to an event.
- `channelId`, `SdsChannelID` and the SDS import are dropped — the
messaging layer has no notion of channels; SDS was a channels-layer
concern that only survived on this type because of the old broker
fan-out.
- `brokerCtx` is dropped for the same reason.
Epoch config (`epochPeriodSec`), the wall-clock `currentEpochStart`,
`queue`, `dequeueReady`, and `resetEpoch` are preserved exactly as the
original owner designed them — this refactor is intentionally scoped
to the API surface, not the epoch mechanism.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: meter SendService transmissions through RateLimitManager
Wires the relocated RateLimitManager into the messaging layer at the
transmission stage rather than the API entry point:
- MessagingClientConf gains a rateLimit: RateLimitConfig field
(defaulting to DefaultEpochPeriodSec / DefaultMessagesPerEpoch), and
MessagingClient.new hands the constructed manager to SendService.
- SendService consults admit() before the first transmission of a
task, both in send() and in the retry loop. Re-publishes of an
already-propagated message (firstPropagatedTime set) skip admission:
they resend the same bytes, which reuse the same RLN proof/nullifier
and consume no fresh epoch slot.
- An over-budget task parks in the task cache as NextRoundRetry; the
service loop re-admits it as the epoch budget frees up. The skeleton
admit() is a pass-through, so behaviour is unchanged today.
Gating transmissions instead of MessagingClient.send keeps SDS repair
rebroadcasts free of API-entry rejection (SDS decides that a repair is
needed; the transmission scheduler decides when it fits the budget)
while every wire transmission still draws from one node-wide budget --
which network-side RLN enforcement applies to repairs regardless of
any local bypass. It is also where RLN proof attachment must happen,
since proofs bind to the epoch current at transmission time.
Adds tests/messaging/test_rate_limit_manager.nim covering the current
disabled + enabled pass-through behaviour, wired into
all_tests_waku.nim.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: make MessagingClientConf.rateLimit reachable, warn on dead channel knobs
Addresses two review findings on the move PR:
- `rateLimit` was a plain `RateLimitConfig`, but `merge` only copies
`Opt` fields (`when oField is Opt`), so every override path
(`LogosDeliveryConf.init`, JSON `messagingOverrides`) silently dropped
it and the field was always taken from `base` — unsettable by any
caller. Make it `Opt[RateLimitConfig]` like every other field;
`MessagingClient.new` falls back to `DefaultRateLimitConfig` (new
const, rate limiting disabled) when unset. Adds a merge test.
- The channel-level knobs (`rateLimitEnabled` /
`rateLimitEpochPeriodSec` / `rateLimitMessagesPerEpoch`) are still
parsed but unread since rate limiting moved to the messaging client,
so setting them was silently ignored. `ReliableChannelManager.new`
now logs a deprecation warning when any is set. Full removal remains a
follow-up API decision.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* WIP logosdeliverynode app initial commit
* WIP - extra cli option
* WIP: messaging client REST endpoints
* Add event poll for messaging rest with cache mechanism
* Messaging rest tests
* test: assert 404 via raw string client in messaging REST test
presto's typed REST client raises RestDecodingError when it cannot decode
a non-2xx text error body into the response type. Add a RestResponse[string]
stub (messagingGetSendEventsByIdRawV1) and point the "already-polled id ->
404" assertion at it, matching the relay REST test pattern.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* remove customized cli args as confutils has no support for it
* Introduce --entry-layer and re-introduce --mode flags into cli args, applied new driver into LogosDelivery + tests
* Add messaging REST client test
* Add docker image build of logosdeliverynode for CI builds
* Fix tests
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Refactor Messaging REST API to better match Messaging Send and Receive APIs
* chore: migrate messaging REST API to Opt[T]
Follow-up to the rebase onto master's repo-wide Option[T] -> Opt[T] change
(#4035). Converts the code this branch adds to the new convention:
- messaging/rest_api/types.nim: MessagingJsonEnvelope fields to Opt[T],
Opt.some/Opt.none, and json_serialization/pkg/results instead of
json_serialization/std/options.
- tests: WakuNodeConf.clusterId is now Opt[uint16]; DTO fields are Opt.
`Option[ContentBody]` in the handlers is presto's own API and stays as-is.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* move MessagingClientConf to api/conf/messaging_conf
* move ReliableChannelManagerConf to new api/conf/channels_conf
* impl modules now import their conf module, not the reverse
* logos_delivery_conf uses channels_conf, not reliable_channel_manager
* cleanup import lists
* Rename WakuMode to LogosDeliveryMode, add Fleet mode
* KernelConf (which starts only kernel) is now a distinct WakuNodeConf (starts messaging, for tests)
* Add LogosDelivery.new(kernelConf) (kernel-only)
* Add LogosDelivery.new(kernelConf, messagingOverrides, channelsOverrides) (full stack)
* LogosDeliveryConf.messagingOverrides/channelsOverrides are Option and mount each layer if present
* Add kernel-only LogosDeliveryConf.init(kernelConf)
* Parse {"mode": "fleet", "kernelConf": {...}} in parseLogosDeliveryConf
* Generalize parseOverrides to parse over caller-supplied defaults
* Add requireMessaging/requireChannels FFI guards and apply them
* Make start/stop skip layers that were not mounted (no config)
* Rename toKernelConf to toWakuNodeConf
* Add tests to test_conf.nim
* Misc additions/refactors
* remove --mode from the CLI
* move WakuMode to the messaging layer
* expose store backend (db url, max connections) and a remote store node on the messaging surface
* wakunode2 with no flags now runs as a full service node (store still opt-in)
* add rateLimitMessagesPerEpoch
* channel rate-limiting auto-enables if epochPeriodSec or messagesPerEpoch is set
* fix JSON conf parser to be generic (works over all config types)
* messaging config = mode + preset + messagingOverrides + channelsOverrides
* add full messaging plus selective kernel config options to MessagingClientConf
* mode (Core/Edge) expands to kernel protocol flags in the messaging layer
* create_node parses the messaging config, drops the flat WakuNodeConf JSON entrypoint
* wire channelsOverrides (segmentation/SDS/rate-limit) into channel creation
* fix liblogosdelivery.h comments and README for the new config shape
* messaging conf tests: switch names, reject-unknown, set-twice, field->kernel
* add kernel log-level, log-format, nodekey to the messaging surface
* Port 0 (ephemeral) default for messaging entry points
* KernelConf alias for WakuNodeConf
* rewrite the FFI examples to the new config shape
* C/C++ examples use preset status.prod
* drop operator-only confs from the examples
* remove duplicate tests & misc test fixes
* Delete p2pReliability from Kernel (Waku) resolver and config (keep preset definition)
* Delete NodeConfig API (deprecation completed by p2pReliability removal from kernel)
* Rename test_messaging_conf.nim to test_conf.nim (tests Logos Delivery config in general)
* Rename messaging_conf_json.nim to logos_delivery_conf_json.nim
* Add logos_delivery_conf.nim (defines LogosDeliveryConf aggregate)
* misc docs/comments cleanups
* squash original commits and rebase to master: removed sendHandler and replaced by MessagingSend BrokerRequest
* Needed to adapt to new messaging source structure to avoid cyclic imports
* Relocate MessagingSend request broker and eliminate api/requests
* fix ci fails
* Reshape per-layer API into api/ folders and thin the FFI over them
Each layer now separates its constructible core from its public surface:
- core module (waku.nim / messaging_client.nim /
reliable_channel_manager.nim): the type plus new/start/stop and the
private construction helpers.
- api/ folder: one module per differentiated set of operations
(waku: topics/relay/filter/lightpush/store/peer_manager/discovery/
debug/health) plus an events surface.
The waku api is reshaped to be the complete operation surface the C
bindings need, so the library no longer reaches into node internals:
relayPublish returns the message hash, relaySubscribe takes an optional
handler, filter/lightpush auto-select the service peer, connectedPeersInfo
returns structured data, pingPeer honours the timeout, plus
relayNumPeersInMesh / relayNumConnectedPeers / isOnline. library/ is now a
thin C-ABI shim: each {.ffi.} proc only marshals cstring/JSON/callbacks and
delegates to ctx.myLib[].waku.<op> (or messagingClient.<op>).
app_callbacks re-exports the modules defining its handler types, which the
included FFI files previously relied on by leakage.
Events move next to the surface that owns them, with each dependency kept
pointing the right way:
- waku/events/ relocated under waku/api/events/.
- channel events live in channels/api/events.nim.
- the four messaging-level message events move to messaging/api/events;
MessageSeenEvent stays in waku because it is emitted by waku core, so
moving it would make waku depend on the messaging layer.
- delivery_events renamed to filter_subscribe_events to match the
OnFilterSubscribe/Unsubscribe events it actually declares.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add reliable-channel FFI ops + events (nim-ffi v0.1.3)
Expose the reliable-channel layer through the v0.1.3 FFI:
- channel_create / channel_send / channel_close call the
ReliableChannelManager api (createReliableChannel / send / closeChannel),
marshalling channel id + base64 payload + ephemeral by hand
- channel message received / sent / errored are surfaced by listening to the
channel-layer broker events in start_node and forwarding them through
callEventCallback (received payload base64-encoded), dropped in stop_node
Stays on nim-ffi v0.1.3 (no typed/CBOR rewrite).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Expose reliable-channel ops in the stable C header (#3851)
The library already ships as a single .so with a tiered header surface
(liblogosdelivery.h = stable Messaging/Reliable-Channels, liblogosdelivery_kernel.h
= advanced Kernel). Per that tiering, the reliable-channel ops belong on the
stable surface, so declare channel_create / channel_send / channel_close in
liblogosdelivery.h and document the channel lifecycle events delivered through
the event callback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Graft PR#3975 interface layer onto decomposed foundation (events deduped)
Add IKernel/IMessagingClient/IReliableChannelManager/ILogosDelivery interface
classes under logos_delivery/api/. The EventBroker types PR#3975 hoisted into
these files already exist in PR#3989's decomposed */api/events/ modules, so the
interface files re-export those modules instead of redefining the types
(avoids 8 duplicate EventBroker definitions). api/types.nim kept at the
foundation version (ChannelId stays in channels/types.nim, which the decomposed
modules import).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Wire impl classes to interfaces (inherit; relocate SendHandler)
- Waku : IKernel, MessagingClient : IMessagingClient,
ReliableChannelManager : IReliableChannelManager.
- The operation procs already live in PR#3989's decomposed */api/ modules and
stay as plain procs (nothing dispatches through the interface types, so no
method-ization is needed).
- SendHandler now lives in reliable_channel_manager_api.nim (its PR#3975 home);
removed the duplicate from reliable_channel.nim, which re-exports the
interface module so channels/api/{channel_lifecycle,send} still see it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Wire LogosDelivery to ILogosDelivery orchestrator interface
LogosDelivery : ILogosDelivery; start/stop/isOnline become method overrides.
Peripheral PR#3975 edits (lightpush/store clients, self_req_handlers,
statistics) are import-reorg artifacts of deleting waku/utils/requests.nim,
which the decomposed structure keeps -- so they are intentionally not ported.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Dedup EventConnectionStatusChange (re-export from health_events)
9th duplicate EventBroker type: defined in both logos_delivery_api.nim and the
decomposed waku/api/events/health_events.nim. The interface file now re-exports
it. liblogosdelivery builds clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Move events back into interface-class source files (restore #3975 placement)
Reverses the earlier dedup-by-re-export: event TYPE definitions now live in the
interface classes, and the emptied decomposed event files are removed.
- MessageSeenEvent -> logos_delivery/api/kernel_api.nim
- Message{Sent,Error,Propagated,Received}Event -> api/messaging_client_api.nim
- ChannelMessage{Received,Sent,Error}Event -> api/reliable_channel_manager_api.nim
- EventConnectionStatusChange -> api/logos_delivery_api.nim
Deleted (became empty after the move):
- logos_delivery/waku/api/events/message_events.nim
- logos_delivery/messaging/api/events.nim
- logos_delivery/channels/api/events.nim
health_events.nim keeps its two remaining events (content/shard topic health).
Rewiring: each layer re-exports its interface module (waku->kernel_api,
messaging_client->messaging_client_api, reliable_channel->reliable_channel_manager_api,
which also re-exports messaging_client_api). Deep emitters/listeners
(subscription_manager, waku_node, waku_node/relay, node_health_monitor,
recv_service, send_service) import the owning interface module directly.
kernel_api stays below node level (types/topics/message/store-common) so the
node->kernel_api imports are acyclic. liblogosdelivery builds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* nph formatting
---------
Co-authored-by: Ivan FB <ivansete@status.im>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Move waku.nim from waku/factory to under waku/
* remove unused
* Realize Kernel API in scope of Waku class
* Refactor waku/api into messaging_client, waku/api/types and api_conf into logos_delivery/api
* Make liblogosdelivery and wakunode2 compile, remove waku/api.nim as it was just a import orchestrator
* make test compile and run
* Reconcile master's new send tests to LogosDelivery API after rebase
master commits #3965/#3669-followup added two test cases (Edge lightpush
delivery #3847, store-validation timeout) written against the removed
waku/api.nim createNode helper. Rewrite them to the LogosDelivery shape:
createNode -> LogosDelivery.new, node.node -> node.waku.node,
node.brokerCtx -> node.waku.brokerCtx, node.send -> node.messagingClient.send,
and drop the now-implicit mountMessagingClient calls (LogosDelivery.new
mounts the client internally).
