diff --git a/api.md b/api.md index 6fe5c01d9..c8d508b7e 100644 --- a/api.md +++ b/api.md @@ -1,10 +1,9 @@ # API design and consistency -Users need to be able to run a Delivery node, run their own fleets, or join existing networks, or simply send and receive messages over existing network. +We have a few interfaces to configure a Delivery node: CLI, library (Nim / C-bindings), REST. We should agree which is for which role, and what each looks like. -We have a few APIs: CLI, library (nim / C-bindings), REST. We should define which API is for which use case, and which API should support which features. +## Discussions -There's been a few discussions about this, e.g.: - https://github.com/logos-messaging/logos-delivery/issues/3795 - https://github.com/logos-messaging/logos-delivery/pull/3828 - https://github.com/logos-messaging/logos-delivery/pull/3828#issuecomment-4317261531 @@ -12,78 +11,97 @@ There's been a few discussions about this, e.g.: - https://github.com/logos-messaging/logos-delivery/pull/3828#issuecomment-4353802313 - https://github.com/logos-co/logos-delivery-module/issues/18 -I would like us to come up to an agreement and implement it. +## Roles -## User roles +- **App developer** — sends/receives messages on a known network. +- **Node operator** — runs a fleet node 24/7. +- **Tester (DST / QA)** — drives custom configs, often many nodes per host. -There're 3 main roles of `logos-delivery` users, which we need to cover: -1. Node Operator - - Run a custom node (== run a fleet node) - - Join existing network -2. App developer - - Send/Receive messages on existing network -3. Tester - - Run a custom node - - Instruct the node to Send/Receive messages with certain protocols +## Requirements -## Ways to configure a node +- **No breaking changes.** Soft deprecations are fine. +- **Good defaults.** App developers should get a working node from a one-liner. +- **Flexibility for node operators.** Every preset/mode-controlled value must be overridable by an explicit field (#3795). -1. CLI -2. Library API -3. Logos Core (effectively same as [2]) +## Principles -## User journeys +1. Every node starts from a `WakuNodeConf`. Every interface must produce one deterministically: -### I am developing an application and want to send/receive messages over Logos Testnet + ```mermaid + flowchart LR + CLI["CLI flags
--preset, --tcp-port, --relay, ..."] -->|build| Conf + Msg["Library: Messaging API
preset, mode, overrides"] -->|build| Conf + Full["Library:
full WakuNodeConf"] -->|pass| Conf + Conf[WakuNodeConf] --> Waku["Waku.new(conf)"] + ``` +2. Different interfaces don't need identical argument lists. Each is shaped for its role. +3. Developer gets a shortcut: `(preset, mode, overrides?)`. They never assemble a full `WakuNodeConf`. +4. Operator gets the full `WakuNodeConf` surface. -This should be done with Messaging API, i.e. a high-level API, which hides all complexity and customization. +## Problems -| Parameter | Value | +1. **No `(preset, mode)` entry point.** The library only has `createNode(conf: WakuNodeConf)`: + https://github.com/logos-messaging/logos-delivery/blob/75864a705ea0b913d517a5f3640747f8709e9e53/waku/api/api.nim#L16 + and the C FFI mirror at: + https://github.com/logos-messaging/logos-delivery/blob/75864a705ea0b913d517a5f3640747f8709e9e53/liblogosdelivery/logos_delivery_api/node_api.nim#L94-L96 + The Messaging API shape doesn't exist. +2. **`logosdelivery_create_node(configJson)` mixes Messaging-API and full-CLI fields in the same blob.** A caller can pass `"preset": "twn"` next to any of ~80 `WakuNodeConf` fields. +3. **`--preset` overrides aren't uniform.** Some preset-controlled fields silently win, some warn, some get overridden. Behaviour differs per field. +4. **Cluster-id implicitly selects a preset.** [`cli_args.nim:926-935`](tools/confutils/cli_args.nim#L926-L935) infers presets from `--cluster-id=1` / `--cluster-id=2` and only warns. Implicit and confusing. + +> See also [`ffi-libraries.md`](ffi-libraries.md) for the separate question of consolidating the two C FFI libraries. + +## Proposal + +### Library + +Two entry points, both produce a `WakuNodeConf` and call the same `Waku.new(...)`: + +| Entry point | Audience | |-|-| -| preset | `logos.test` | -| mode | Core / Edge | - -### I want to run a node 24/7 to support Logos Testnet - -| Parameter | Value | -|-|-| -| preset | `logos.test` | -| nat | ... | -| relay, lightpush, store | ... | - -### I need to host a fleet of custom configuration nodes for my own network - -In this case, user should manually configure all of the parameters: -- Network: cluster id, number of shards in network, max message size -- Connectivity: nat, external addresses -- Protocols: relay, lightpush, store -- RLN configuration: cred path, eth client address, chain id, etc. - -| Parameter | Value | -|-|-| -| cluster id, number of shards in network, max message size | | -| nat | ... | -| relay, lightpush, store | ... | - -> [!NOTE] -> No use of `preset`, only explicit list of parameters -> -> **Q:** Why can't we use `preset` in this case? -> **A:** Node should be configurable by Infra, not by changing `logos-delivery` source code, rebuilding and re-deploying. - -## API - -1. CLI -2. Library (Nim, C-bindings) -3. REST API (to be deprecated, use Logos Core module API instead, exposed to REST or Python) - -### Logos Core - -Ideally, in all use cases, Logos Core with `logos-delivery-module` should be used. Not CLI, not REST API. -This means that Library itself should expose enough API. - -So probably `createNode` should have 2 signatures: `createNode(preset, mode)` and `createNode(config)` (I think it's already like this). +| `createNode(preset, mode, overrides?)` | Developer (Messaging API) | +| `createNode(conf: WakuNodeConf)` | Tester, advanced tooling | ### CLI -But for now (and amybe forever), we'll need to keep supporting CLI. It needs to be cleaned up. Options like `--mode` should not appear there, because `mode` is a notion of Messaging API, while CLI doesn't expose Messaging API capabilities. \ No newline at end of file +Full `WakuNodeConf` surface. `--preset` is a shortcut for network-level params; explicit flags override. + +**`--mode` should be removed.** Today it is purely a protocol-toggle shortcut ([`cli_args.nim:1126-1144`](tools/confutils/cli_args.nim#L1126-L1144)) — six `withRelay` / `withLightPush` / `withFilter…` / `withDiscv5` / `withPeerExchange` / `withRendezvous` calls plus a rate-limit default. Nothing else. So it overlaps with the explicit protocol flags an operator already uses, and it doesn't carry any of the broader meaning the Messaging API's `mode` is supposed to have. + +Keep it on the CLI only if DST/QA actually depend on the shortcut. If they do, this should be the documented reason. If they don't, drop it. + +## Code changes + +1. **Add `(preset, mode, overrides?)` to the library Messaging API.** + - In [`waku/api/api.nim`](waku/api/api.nim): a new `createNode` overload that builds a `WakuNodeConf` from `(preset, mode, overrides)` and delegates to the existing one. + - Mirror in [`liblogosdelivery`](liblogosdelivery/logos_delivery_api/node_api.nim) — either a dedicated FFI call or a documented Messaging-API JSON shape (`{"preset": "...", "mode": "...", "overrides": {...}}`). + - Define `WakuNodeConfOverrides` (likely `Option[T]` per field, derived from `WakuNodeConf`). + +2. **Audit preset overrides.** For every field set by [`NetworkConf`](waku/factory/networks_config.nim#L20-L36), confirm: explicit override wins; warning is logged; resulting config validated. + +3. **Drop cluster-id → preset auto-mapping.** Soft-deprecate first ([`cli_args.nim:926-935`](tools/confutils/cli_args.nim#L926-L935)), remove in a later release. + +
+What --preset sets + +See [`waku/factory/networks_config.nim`](waku/factory/networks_config.nim). A preset fills in: `entryNodes`, `clusterId`, `numShardsInCluster`, `maxMessageSize`, RLN config (contract, chain id, epoch, message limit), discv5 bootstrap, kad bootstrap, `mix`, `p2pReliability`. + +
+ +
+What mode means + +**In CLI today** ([`cli_args.nim:1126-1144`](tools/confutils/cli_args.nim#L1126-L1144)) + +A protocol-toggle shortcut. Nothing more. +- `Core` / `Edge` / `noMode` + +**What `mode` should mean (Messaging API)** + +A developer-facing role/profile. The app developer says "my app is a Core participant" or "my app is an Edge consumer", and the library translates that into a coherent set of defaults. But also, Messaging API runs background routines, which is not the case when one would use existing `--mode` from CLI. + +This is why `mode` doesn't fit the CLI: an operator wiring a fleet by hand picks each of those values explicitly. And it should not be look like they're using anything close to Messaging API. + +And note that in Messaging API `noMode` is impossible. + +
diff --git a/ffi-libraries.md b/ffi-libraries.md new file mode 100644 index 000000000..28d398bbc --- /dev/null +++ b/ffi-libraries.md @@ -0,0 +1,99 @@ +# FFI library consolidation + +We have two C FFI libraries built from the same source tree: + +- [`liblogosdelivery`](liblogosdelivery/) — high-level Messaging API. +- [`library/libwaku`](library/) — low-level kernel API (per-protocol). + +This doc proposes merging them into one library with a tiered surface, and uses that to answer the open Store-access question on the 2026 GA roadmap. + +## Discussions + +- https://github.com/logos-messaging/logos-delivery/pull/3714#discussion_r2773830458 — original split rationale, plus naming side-point. +- https://roadmap.logos.co/messaging/roadmap/milestones/2026-messaging-api-general-availability — Store access for Status. + +## Context + +The split was deliberate ([thread above](https://github.com/logos-messaging/logos-delivery/pull/3714#discussion_r2773830458)): + +> The aim is to avoid any further messaging consumers to use `libwaku` (kernel API) so that they are not tempted to use it. + +> We do not want to promise anything about KernelAPI outward, that is open for change. So we would like to lead out libwaku slowly and let Status/Chat/anybody only depend on this new API layer. + +Counter-position from the same thread (Igor): + +> Why does it have to be a separate library? I thought we want to just add the API into existing `libwaku`. Then it would be up to the users which API to use — low- or high-level. + +With the [plan](https://roadmap.logos.co/messaging/roadmap/milestones/2026-status-logos-delivery-integration) of using Messaging API in Status, we are ought to expose Store API from the same node, because Status needs Store for offline history, community descriptions, profiles, missed messages. Store was intentionally excluded from the Messaging API, so the intuitive solution is to expose Store next to it. This is mentioned in [Messagin API GA milestone](https://roadmap.logos.co/messaging/roadmap/milestones/2026-messaging-api-general-availability). + +## Requirements + +- Messaging API consumers (e.g. Status) can access Store API from the same node they created using Messaging API. +- The default surface stays minimal — Messaging API is the front door for developers. + +## Principles + +1. **One library.** Two FFI plumbings duplicated across `createNode`, lifecycle, FFI context, JSON parsing is maintenance debt with no upside. +2. **Tiered surface inside the library.** Messaging API is the supported, recommended, stable surface. Kernel calls are the advanced surface — usable when explicitly needed (Store for Status; per-protocol calls for advanced tooling), not the default. Tiering is expressed via separate C headers, not a separate library and not a longer symbol prefix (see [Proposal](#proposal)). +3. **"Don't promise stability" is a documentation concern, not a separate-library concern.** A consumer that wants the kernel API just adds another link line today — the split doesn't actually prevent misuse. Headers, naming, and docs do. +4. **We MAY control what we expose to Logos Core**. Not all library functions have to be exposed to Logos Core module API, and this is also where we could control what users have access to. + +## Problems + +1. **Status needs Store, Messaging API excludes it** \ + The split forces a false choice: pollute the Messaging API with Store (compromises minimalism), or push Status to `libwaku` (defeats the "one library for messaging consumers" promise). +1. **Two libraries means two nodes** \ + Each FFI library creates its own `Waku` instance via `Waku.new(...)`. A consumer that wanted *both* the Messaging API and a kernel call (e.g. Store) couldn't just link both `.so`s — they'd be running two independent libp2p stacks. +2. **Duplicated plumbing** \ + Both libraries implement: FFI context, JSON config parsing, lifecycle, etc. + +## + +## Proposal + +Merge into a single library — keep [`liblogosdelivery`](liblogosdelivery/) as the host, retire `library/libwaku`. + +Answering + +All symbols share the same `logosdelivery_` prefix. Tiering is expressed by splitting the C header: + +| Header | Tier | Audience | Stability promise | +|-|-|-|-| +| `liblogosdelivery.h` | Messaging API | Default — app developers, Logos Core | Stable, supported | +| `liblogosdelivery_kernel.h` | Kernel / advanced | Status (Store), advanced tooling | "Use at your own risk", may change | + +Both headers expose symbols from the same `.so`, so they share node lifecycle, FFI context, JSON config parsing — no duplication, single `Waku` instance. + +This is a common C pattern (SDL, OpenSSL, POSIX). The "advanced / unsupported" signal comes from `#include "liblogosdelivery_kernel.h"` — the consumer makes a deliberate choice to opt in. Symbol names stay short: `logosdelivery_store_query(...)` instead of `logosdelivery_kernel_store_query(...)`. + +This answers the 2026 GA roadmap's open question: **low-level access lives in the same library, behind a separate header**. Status `#include`s `liblogosdelivery_kernel.h`, calls `logosdelivery_store_query(...)`, and shares the existing node. + +### Naming side-note + +Filipe's point in #3714: drop "waku" from public surfaces, keep it for internals / unsupported things. The merged library is a natural moment to apply this — pick a final public prefix (`logosdelivery_`, `lm_`, …) and stick with it. Open. + +## Code changes + +1. **Split [`liblogosdelivery.h`](liblogosdelivery/liblogosdelivery.h) into two headers.** Keep the existing one as the Messaging API surface. Add `liblogosdelivery_kernel.h` for the advanced surface. Both pull symbols from the same `.so`. Hand-authored — Nim's `exportc` doesn't auto-split. +2. **Move kernel calls inside `liblogosdelivery`.** Bring every per-protocol call from [`library/kernel_api/`](library/kernel_api/) (peer manager, discovery, ping, debug, relay, store, lightpush, filter) under `liblogosdelivery/`. All keep the `logosdelivery_` prefix; declarations land in `liblogosdelivery_kernel.h`. +3. **Add Store access** under the kernel header. Required by Status for the 2026 GA milestone. +4. **Unify `createNode` and JSON parsing.** Pick the `liblogosdelivery` semantics (case-insensitive, reject-unknown). Resolve the `restServerConf` divergence — strip it for FFI, keep it for CLI. +5. **Unify lifecycle plumbing.** One `FFIContext`, one set of `start_node` / `stop_node` / `destroy`. Shared between both headers. +6. **Migrate `libwaku` consumers** one by one. Each moves to the merged library; their kernel calls map to the new declarations in `liblogosdelivery_kernel.h`. +7. **Remove `library/libwaku`** once nothing references it. Non-breaking by sequencing. + +
+Surface comparison + +`liblogosdelivery` (high-level, today): +- `logosdelivery_create_node` / `start_node` / `stop_node` / `destroy` +- `logosdelivery_subscribe` / `unsubscribe` +- `logosdelivery_send` — returns `requestId` +- `logosdelivery_set_event_callback` — events: `message_sent`, `message_propagated`, `message_received`, `message_error`, `connection_status_change` +- `logosdelivery_get_available_node_info_ids` / `get_node_info` / `get_available_configs` + +`library/libwaku` (low-level, today): +- `waku_new` / `waku_destroy` +- per-protocol modules included from [`libwaku.nim:19-28`](library/libwaku.nim#L19-L28): `peer_manager_api`, `discovery_api`, `node_lifecycle_api`, `debug_node_api`, `ping_api`, `relay_api`, `store_api`, `lightpush_api`, `filter_api`. + +