mirror of
https://github.com/logos-co/logos-delivery-module.git
synced 2026-08-27 12:01:09 +00:00
docs: document the three createNode config shapes (#79)
* docs: document createNode config shapes for the three user roles Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: drop port defaulting, make the storage-path default shape-aware The injected top-level keys forced every config through the legacy flat parser and broke the layered shapes ('kernel entry layer takes only kernelConf'). Ports now follow upstream defaults; the per-instance storage path goes inside kernelConf when the config carries one. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: place the storage-path default in messagingOverrides for structured configs Requires logos-delivery#4083 (localStoragePath on MessagingClientConf). Pins logos-delivery to that branch; re-lock to master once it merges. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: tighten config-defaults comments Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: re-lock logos-delivery to master with #4083 and #4084 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: route wrapper-less layered configs through messagingOverrides A plain {mode, preset} config previously fell to the legacy flat parser because the injected top-level storage path counts as a bare kernel field. Creating messagingOverrides for it keeps the storage path set and gives layered configs the structured defaults (ephemeral p2p ports). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Igor Sirotin
co-authored by
Claude Fable 5
Claude Fable 5
parent
ee8d231fa3
commit
531e61e11c
@@ -103,50 +103,50 @@ The delivery module provides the following API methods (all synchronous, all ret
|
||||
|
||||
### Node Configuration (`createNode`)
|
||||
|
||||
`createNode` accepts a **flat** JSON object whose keys correspond to `WakuNodeConf`
|
||||
field names (camelCase) from
|
||||
[logos-delivery](https://github.com/logos-messaging/logos-delivery).
|
||||
Unknown keys are silently ignored. Every field has a built-in default, so only
|
||||
values that differ from defaults need to be supplied.
|
||||
The JSON config is passed verbatim to
|
||||
[logos-delivery](https://github.com/logos-messaging/logos-delivery), which owns
|
||||
the grammar (`parseLogosDeliveryConf`). `entryLayer` selects how much of the
|
||||
stack is mounted: `"kernel"` (transport node only), `"messaging"` (+ messaging
|
||||
client), `"channels"` (+ reliable channels, the default).
|
||||
|
||||
#### Commonly used keys
|
||||
Three typical shapes:
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|----------------------|------------------|------------|------------------------------------------|
|
||||
| `mode` | string | `"noMode"` | `"Core"`, `"Edge"`, or `"noMode"` |
|
||||
| `preset` | string | `""` | Network preset (`"logos.test"`, `"logos.dev"`, `"twn"`) |
|
||||
| `clusterId` | number (uint16) | `0` | Cluster identifier |
|
||||
| `entryNodes` | array of string | `[]` | Bootstrap peers (enrtree / multiaddress) |
|
||||
| `relay` | boolean | `false` | Enable relay protocol |
|
||||
| `rlnRelay` | boolean | `false` | Enable RLN rate-limit nullifier |
|
||||
| `tcpPort` | number (uint16) | `60000` | P2P TCP listen port |
|
||||
| `numShardsInNetwork` | number (uint16) | `1` | Auto-sharding shard count |
|
||||
| `logLevel` | string | `"INFO"` | `"TRACE"`, `"DEBUG"`, `"INFO"`, `"WARN"` |
|
||||
| `logFormat` | string | `"TEXT"` | `"TEXT"` or `"JSON"` |
|
||||
| `maxMessageSize` | string | `"150KiB"` | Maximum message payload size |
|
||||
**App developer** — full stack (default `entryLayer`). `preset` picks the
|
||||
network (`"logos.test"`, `"logos.dev"`, `"twn"`), `mode` picks the protocol
|
||||
flags (`"Core"` = relay node, `"Edge"` = light node). Optional
|
||||
`messagingOverrides` / `channelsOverrides` objects override per-layer defaults:
|
||||
|
||||
#### Presets
|
||||
```json
|
||||
{ "mode": "Core", "preset": "logos.test" }
|
||||
```
|
||||
|
||||
Using a `preset` populates cluster ID, entry nodes, sharding, RLN, and other
|
||||
network-specific defaults automatically. Individual keys supplied alongside a
|
||||
preset override the preset values.
|
||||
|
||||
- `"logos.test"` – Logos Test fleet (the default for running a node; mix
|
||||
enabled, p2pReliability on, auto-shards, built-in bootstrap nodes).
|
||||
- `"logos.dev"` – Logos Dev Network (cluster 2, mix enabled, p2pReliability on,
|
||||
8 auto-shards, built-in bootstrap nodes).
|
||||
- `"twn"` – The RLN-protected Waku Network (cluster 1).
|
||||
|
||||
Minimal example using the default `logos.test` preset:
|
||||
**Node operator** — kernel-only service node on a public network. `mode` is not
|
||||
applied on this layer, so protocol flags are set explicitly in `kernelConf`:
|
||||
|
||||
```json
|
||||
{
|
||||
"logLevel": "INFO",
|
||||
"mode": "Core",
|
||||
"preset": "logos.test"
|
||||
"entryLayer": "kernel",
|
||||
"kernelConf": { "preset": "logos.test", "relay": true }
|
||||
}
|
||||
```
|
||||
|
||||
**Network hoster** — kernel-only node on a self-hosted network; `kernelConf` is
|
||||
a raw `WakuNodeConf` used as-is:
|
||||
|
||||
```json
|
||||
{
|
||||
"entryLayer": "kernel",
|
||||
"kernelConf": { "clusterId": 42, "relay": true, "entryNodes": ["/dns4/…"] }
|
||||
}
|
||||
```
|
||||
|
||||
On kernel-only nodes `send` / `subscribe` / `channel*` fail with "node has no
|
||||
messaging client" / "no reliable channel manager"; `getNodeInfo`, `storeQuery`
|
||||
and metrics keep working.
|
||||
|
||||
The pre-layered flat shape (bare `WakuNodeConf` keys at top level) still parses
|
||||
and boots the full stack.
|
||||
|
||||
### Content Topics
|
||||
|
||||
Content topics identify message channels for publishing and subscribing. Use a
|
||||
|
||||
Generated
+4
-4
@@ -3824,11 +3824,11 @@
|
||||
"zerokit": "zerokit"
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1785353753,
|
||||
"narHash": "sha256-+hMflwMeA5aRhNV+XHhWqeXCYfpjHbcmXFFlfks7tnw=",
|
||||
"lastModified": 1785443689,
|
||||
"narHash": "sha256-MHPA2HSHjK6xeo+a1GrDQK80IBMmBShv7W10jZmeykI=",
|
||||
"ref": "refs/heads/master",
|
||||
"rev": "ed8e881c1d36af2d033de49289fa392fc6b7a092",
|
||||
"revCount": 2419,
|
||||
"rev": "f8b036594ea2a36b529e10b584b7d2851a3ac5c8",
|
||||
"revCount": 2425,
|
||||
"submodules": true,
|
||||
"type": "git",
|
||||
"url": "https://github.com/logos-messaging/logos-delivery"
|
||||
|
||||
@@ -207,37 +207,46 @@ void DeliveryModuleImpl::event_callback(int callerRet, const char* msg, size_t l
|
||||
}
|
||||
}
|
||||
|
||||
// True when cfgObj already carries one of `names`. The upstream JSON conf
|
||||
// parser keys fields case-insensitively and matches either the Nim field name
|
||||
// or its CLI `name:` pragma, so a caller may legitimately spell a key several
|
||||
// ways; matching the same way keeps us from overriding their value.
|
||||
static bool containsAnyKey(const nlohmann::json& cfgObj,
|
||||
std::initializer_list<const char*> names)
|
||||
static std::string toLowerCopy(std::string s)
|
||||
{
|
||||
for (auto& c : s) c = static_cast<char>(std::tolower(static_cast<unsigned char>(c)));
|
||||
return s;
|
||||
}
|
||||
|
||||
// Case-insensitive key lookup, matching keys the same way as the upstream
|
||||
// conf parser. Returns the key as spelled in the config.
|
||||
static std::optional<std::string> findKey(const nlohmann::json& cfgObj,
|
||||
std::initializer_list<const char*> names)
|
||||
{
|
||||
for (const auto& entry : cfgObj.items()) {
|
||||
std::string key = entry.key();
|
||||
for (auto& c : key) c = static_cast<char>(std::tolower(static_cast<unsigned char>(c)));
|
||||
const std::string key = toLowerCopy(entry.key());
|
||||
for (const char* name : names) {
|
||||
if (key == name) return true;
|
||||
if (key == name) return entry.key();
|
||||
}
|
||||
}
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
// True when the config is the legacy flat shape: any top-level key besides the
|
||||
// ones the layered parser consumes marks a bare WakuNodeConf field.
|
||||
static bool isFlatShape(const nlohmann::json& cfgObj)
|
||||
{
|
||||
for (const auto& entry : cfgObj.items()) {
|
||||
const std::string key = toLowerCopy(entry.key());
|
||||
if (key != "entrylayer" && key != "mode" && key != "preset"
|
||||
&& key != "kernelconf" && key != "messagingoverrides"
|
||||
&& key != "channelsoverrides") {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// Default every listening port (tcpPort, discv5UdpPort, restPort,
|
||||
// metricsServerPort, websocketPort) to 0 so the OS assigns an ephemeral port
|
||||
// when the caller did not pin a specific value. Caller-supplied ports are
|
||||
// preserved so fleet configs that pin ports keep working. logos-delivery now
|
||||
// accepts port 0 (status-im/nim-confutils#146), which makes this work.
|
||||
// See logos-delivery-module#18.
|
||||
//
|
||||
// Also default the node's storage directory to the per-instance path the host
|
||||
// provisions for this module. logos-delivery otherwise falls back to "./data"
|
||||
// (persistency.nim DefaultStoragePath), which is relative to the process
|
||||
// working directory and therefore identical for every instance launched from
|
||||
// it — side-by-side instances would share one SQLite file. The path is empty
|
||||
// when the module runs outside a host that provisions persistence (unit tests
|
||||
// constructing the impl directly), in which case upstream's default stands.
|
||||
// Defaults the node's storage directory to the host's per-instance path, so
|
||||
// side-by-side instances don't share upstream's cwd-relative "./data". The
|
||||
// path goes where each config shape accepts it: kernelConf when present,
|
||||
// messagingOverrides (created if needed) for the layered shapes, top level
|
||||
// for the legacy flat shape.
|
||||
static std::optional<std::string> applyConfigDefaults(const std::string& cfg,
|
||||
const std::string& persistencePath)
|
||||
{
|
||||
@@ -254,21 +263,29 @@ static std::optional<std::string> applyConfigDefaults(const std::string& cfg,
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
for (const char* portKey : {
|
||||
"tcpPort",
|
||||
"discv5UdpPort",
|
||||
"restPort",
|
||||
"metricsServerPort",
|
||||
"websocketPort",
|
||||
}) {
|
||||
if (!cfgObj.contains(portKey)) {
|
||||
cfgObj[portKey] = 0;
|
||||
if (!persistencePath.empty()) {
|
||||
nlohmann::json* target = &cfgObj;
|
||||
const auto entryLayerKey = findKey(cfgObj, {"entrylayer"});
|
||||
const bool kernelEntry = entryLayerKey && cfgObj[*entryLayerKey].is_string()
|
||||
&& toLowerCopy(cfgObj[*entryLayerKey].get<std::string>()) == "kernel";
|
||||
if (auto kernelConfKey = findKey(cfgObj, {"kernelconf"});
|
||||
kernelConfKey && cfgObj[*kernelConfKey].is_object()) {
|
||||
target = &cfgObj[*kernelConfKey];
|
||||
} else if (kernelEntry) {
|
||||
// Kernel entry without a kernelConf object: leave the config
|
||||
// untouched for the parser to reject.
|
||||
target = nullptr;
|
||||
} else if (!isFlatShape(cfgObj)) {
|
||||
auto overridesKey = findKey(cfgObj, {"messagingoverrides"});
|
||||
if (!overridesKey) {
|
||||
cfgObj["messagingOverrides"] = nlohmann::json::object();
|
||||
overridesKey = "messagingOverrides";
|
||||
}
|
||||
target = cfgObj[*overridesKey].is_object() ? &cfgObj[*overridesKey] : nullptr;
|
||||
}
|
||||
if (target && !findKey(*target, {"localstoragepath", "local-storage-path"})) {
|
||||
(*target)["localStoragePath"] = persistencePath + "/data";
|
||||
}
|
||||
}
|
||||
|
||||
if (!persistencePath.empty()
|
||||
&& !containsAnyKey(cfgObj, {"localstoragepath", "local-storage-path"})) {
|
||||
cfgObj["localStoragePath"] = persistencePath + "/data";
|
||||
}
|
||||
|
||||
return cfgObj.dump();
|
||||
|
||||
@@ -50,64 +50,52 @@ public:
|
||||
~DeliveryModuleImpl();
|
||||
|
||||
/**
|
||||
* @brief Creates a liblogosdelivery node from a WakuNodeConf JSON document.
|
||||
* @brief Creates a liblogosdelivery node from a JSON configuration.
|
||||
*
|
||||
* The JSON is parsed by logos-delivery (liblogosdelivery folder) side and maps to
|
||||
* `WakuNodeConf` from `tools/confutils/cli_args.nim`
|
||||
* (https://github.com/logos-messaging/logos-delivery).
|
||||
* The JSON passes through to logos-delivery verbatim; `parseLogosDeliveryConf`
|
||||
* (https://github.com/logos-messaging/logos-delivery) owns the grammar.
|
||||
* `entryLayer` selects how much of the stack is mounted:
|
||||
* - `"kernel"` — transport node only
|
||||
* - `"messaging"` — kernel + messaging client
|
||||
* - `"channels"` — kernel + messaging + reliable channels (default)
|
||||
*
|
||||
* The configuration is a **flat** JSON object whose keys correspond to
|
||||
* `WakuNodeConf` Nim field names (camelCase). Unknown keys are silently
|
||||
* ignored. Every field has a built-in default, so only the values that
|
||||
* differ from defaults need to be supplied.
|
||||
* Three typical shapes:
|
||||
*
|
||||
* ## Commonly used keys
|
||||
* | Key | Type | Default | Description |
|
||||
* |----------------------|------------------|------------|---------------------------------------------|
|
||||
* | `mode` | string | `"noMode"` | `"Core"`, `"Edge"`, or `"noMode"` |
|
||||
* | `preset` | string | `""` | Network preset (`"twn"`, `"logos.dev"`, …) |
|
||||
* | `clusterId` | number (uint16) | `0` | Cluster identifier |
|
||||
* | `entryNodes` | array of string | `[]` | Bootstrap peers (enrtree / multiaddress) |
|
||||
* | `relay` | boolean | `false` | Enable relay protocol |
|
||||
* | `rlnRelay` | boolean | `false` | Enable RLN rate-limit nullifier |
|
||||
* | `tcpPort` | number (uint16) | `60000` | P2P TCP listen port |
|
||||
* | `numShardsInNetwork` | number (uint16) | `1` | Auto-sharding shard count |
|
||||
* | `logLevel` | string | `"INFO"` | `"TRACE"`, `"DEBUG"`, `"INFO"`, `"WARN"`, … |
|
||||
* | `logFormat` | string | `"TEXT"` | `"TEXT"` or `"JSON"` |
|
||||
* | `maxMessageSize` | string | `"150KiB"` | Maximum message payload size |
|
||||
* **App developer** — full stack (default `entryLayer`). `preset` picks the
|
||||
* network (`"logos.test"`, `"logos.dev"`, `"twn"`), `mode` picks the protocol
|
||||
* flags (`"Core"` = relay node, `"Edge"` = light node). Optional
|
||||
* `messagingOverrides` / `channelsOverrides` objects override per-layer
|
||||
* defaults:
|
||||
* @code{.json}
|
||||
* { "mode": "Core", "preset": "logos.test" }
|
||||
* @endcode
|
||||
*
|
||||
* ## Presets
|
||||
* Using a `preset` populates cluster ID, entry nodes, sharding, RLN, and
|
||||
* other network-specific defaults automatically. Individual keys supplied
|
||||
* alongside a preset override the preset values.
|
||||
* - `"twn"` – The RLN-protected Waku Network (cluster 1).
|
||||
* - `"logos.dev"` – Logos Dev Network (cluster 2, mix enabled,
|
||||
* p2pReliability on, 8 auto-shards, built-in bootstrap nodes).
|
||||
*
|
||||
* Minimal `logos.dev` example:
|
||||
* **Node operator** — kernel-only service node on a public network. `mode`
|
||||
* is not applied on this layer, so protocol flags are set explicitly in
|
||||
* `kernelConf`:
|
||||
* @code{.json}
|
||||
* {
|
||||
* "logLevel": "INFO",
|
||||
* "mode": "Core",
|
||||
* "preset": "logos.dev"
|
||||
* "entryLayer": "kernel",
|
||||
* "kernelConf": { "preset": "logos.test", "relay": true }
|
||||
* }
|
||||
* @endcode
|
||||
*
|
||||
* Full override example:
|
||||
* **Network hoster** — kernel-only node on a self-hosted network;
|
||||
* `kernelConf` is a raw `WakuNodeConf` used as-is:
|
||||
* @code{.json}
|
||||
* {
|
||||
* "mode": "Core",
|
||||
* "clusterId": 42,
|
||||
* "entryNodes": ["enrtree://TREE@nodes.example.com"],
|
||||
* "relay": true,
|
||||
* "tcpPort": 60000,
|
||||
* "numShardsInNetwork": 8,
|
||||
* "maxMessageSize": "150KiB",
|
||||
* "logLevel": "INFO",
|
||||
* "logFormat": "TEXT"
|
||||
* "entryLayer": "kernel",
|
||||
* "kernelConf": { "clusterId": 42, "relay": true, "entryNodes": ["/dns4/…"] }
|
||||
* }
|
||||
* @endcode
|
||||
*
|
||||
* On kernel-only nodes `send` / `subscribe` / `channel*` fail with "node has
|
||||
* no messaging client" / "no reliable channel manager"; `getNodeInfo`,
|
||||
* `storeQuery` and metrics keep working.
|
||||
*
|
||||
* The pre-layered flat shape (bare `WakuNodeConf` keys at top level) still
|
||||
* parses and boots the full stack.
|
||||
*
|
||||
* @param cfg UTF-8 JSON payload string.
|
||||
* @return `true` if context creation succeeds and callback returns `RET_OK`,
|
||||
* otherwise `false`.
|
||||
|
||||
Reference in New Issue
Block a user