logos-messaging/logos-delivery
1
0
Fork 0
mirror of https://github.com/logos-messaging/logos-delivery.git synced 2026-07-21 04:00:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into chat2disco

Browse Source
This commit is contained in:
Simon-Pierre Vivier 2026-07-10 12:00:52 -04:00 committed by GitHub
commit 123503f6f0
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
219 changed files with 5374 additions and 4712 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

55
.github/ISSUE_TEMPLATE/deploy_release.md vendored
View File

@ -1,55 +0,0 @@
---
name: Deploy Release
about: Execute tasks for deploying a new version in a fleet
title: 'Deploy release vX.X.X in logos.test and/or status.prod fleet'
labels: deploy-release
assignees: ''
---
<!--
Add appropriate release number and adjust the target fleet in the title!
-->
### Link to the Release PR
<!--
Kindly add a link to the release PR where we have a sign-off from QA. At this time, that release PR should be already merged.
-->
### Items to complete, in order
<!--
You can release into either logos.test, status.prod, or both. Both cases require coordination with Infra Team.
logos.test must be considered a prod fleet as it is used by external parties.
For both status.prod it is crucial to coordinate such deployment with Status Team.
The following points should be followed in order.
-->
- [ ] Receive sign-off from DST.
- [ ] Inform DST team about what are the expectations for this release. For example, if we expect higher, same or lower bandwidth consumption. Or a new protocol appears, etc.
- [ ] Ask DST to add a comment approving this deployment and add a link to the analysis report.
- [ ] Deploy to logos.test
- [ ] Coordinate with Infra Team about possible changes in CI behavior
- [ ] Update logos.test with the corresponding deployment job in [Jenkins](https://ci.infra.status.im/) (ask the Infra Team if you don't have access or can't find the job).
- [ ] Deploy to status.prod
- [ ] Coordinate with Infra Team about possible changes in CI behavior
- [ ] Ask Status admin to add a comment approving that this deployment to happen now.
- [ ] Update status.prod with [this deployment job](https://ci.infra.status.im/job/nim-waku/job/deploy-status-prod/).
- [ ] Update infra config
- [ ] Submit PRs into infra repos to adjust deprecated or changed arguments (review CHANGELOG.md for that release). And confirm the fleet can run after that. This requires coordination with infra team.
### Reference Links
- [Release process](https://github.com/logos-messaging/logos-delivery/blob/master/docs/contributors/release-process.md)
- [Release notes](https://github.com/logos-messaging/logos-delivery/blob/master/CHANGELOG.md)
- [Infra-role-logos-node](https://github.com/status-im/infra-role-logos-node)
- [Infra-logos](https://github.com/status-im/infra-logos)
- [Infra-Status](https://github.com/status-im/infra-status)
- [Jenkins](https://ci.infra.status.im/)
- [Fleets](https://fleets.logos.co/)
- [Harbor](https://harbor.status.im/harbor/projects/32/repositories/logos-node/artifacts-tab)
- [Kibana](https://kibana.infra.status.im/app/)

106
.github/ISSUE_TEMPLATE/prepare_release.md vendored
View File

@ -1,6 +1,6 @@
---
name: Prepare Release
about: Execute tasks for the creation and publishing of a new full release
about: Execute tasks for the creation, validation, deployment and publishing of a new full release
title: 'Prepare release 0.0.0'
labels: full-release
assignees: ''
@ -8,64 +8,81 @@ assignees: ''
---
<!--
Add appropriate release number to title!
For detailed info on the release process refer to https://github.com/logos-messaging/logos-delivery/blob/master/docs/contributors/release-process.md
Add appropriate release number to title
-->
### Items to complete
### Items to complete, in order
All items below are to be completed by the owner of the given release.
#### Initial validation
- [ ] Confirm waku.test has been running with new commits on each merge into master (see Grafana boards.)
- [ ] Search [Kibana logs](https://kibana.infra.status.im/app/discover) since the last release for crashes or errors in waku.test that we didn't detect in CI.
- Most relevant search queries: `(fleet: "waku.test" AND message: "SIGSEGV")`, `(fleet: "waku.test" AND message: "exception")`, `(fleet: "waku.test" AND message: "error")`.
- Detect unexpected reboots.
- Document any crashes or errors found and fix them before proceeding.
- [ ] Create release branch with major and minor only ( e.g. release/v0.X ) if it doesn't exist.
- [ ] Update the `version` field in `logos_delivery.nimble` to match the release version (e.g. `version = "0.X.0"`) **and merge it before assigning any tag** - the `release-assets` workflow gates artifact build/upload.
### Create the release candidate
- [ ] Create release branch with major and minor only ( e.g. release/v0.X ).
- [ ] In release branch, update the `version` field in `logos_delivery.nimble` to match the release version (e.g. `version = "0.X.0"`).
- [ ] Assign release candidate tag to the release branch HEAD (e.g. `v0.X.0-rc.0`, `v0.X.0-rc.1`, ... `v0.X.0-rc.N`).
- [ ] Generate and edit release notes in CHANGELOG.md.
- [ ] **Validation of release candidate**
#### Validate the release candidate
- [ ] **Automated testing**
- [ ] Ensure all the unit tests (specifically logos-messaging-js tests) are green against the release candidate.
- [ ] 1. Essential test
- [ ] Ensure all unit tests are green
- [ ] Get interop tests results from QA
- [ ] **QA testing**
- [ ] Ask QA to run their available tests against the release candidate.
- [ ] **Logos fleet testing**
- [ ] Deploy the release candidate to `logos.dev` fleet.
- [ ] 2. Fleet validation (pre-requisite: 1. Can be done in parallel with 3.)
- [ ] Deploy the release candidate to the waku.test fleet.
- Start the deployment job in [Jenkins](https://ci.infra.status.im/) and wait for it to finish (Jenkins access required; ask the infra team if you don't have it).
- After completion, disable fleet so that daily CI does not override your release candidate.
- Verify at https://fleets.logos.co/ that the fleet is locked to the release candidate image.
- Confirm the container image exists on [Harbor](https://harbor.status.im/harbor/projects/32/repositories/logos-node/artifacts-tab).
- [ ] Search [Kibana logs](https://kibana.infra.status.im/app/discover) from the previous month (since the last release was deployed) for possible crashes or errors in `logos.dev`.
- Set time range to "Last 30 days" (or since last release).
- Most relevant search query: `(fleet: "logos.dev" AND message: "SIGSEGV")`, `(fleet: "logos.dev" AND message: "exception")`, `(fleet: "logos.dev" AND message: "error")`.
- Document any crashes or errors found.
- [ ] Ask QA to perform tests against `logos.dev`, if any. Then, after that, review Kibana for possible issues or unexpected restart.
- [ ] Enable the `logos.dev` fleet again to resume auto-deployment of the latest `master` commit.
- After completion, disable the fleet so that daily CI does not override your release candidate.
- Verify at https://fleets.logos.co/ that the fleet is locked to the release candidate image.
- [ ] Ask QA to run tests against waku.test and attach a screenshot as evidence.
- [ ] Re-enable the waku.test fleet to resume auto-deployment of the latest master commit.
<!-- In the future, automate `waku.test` crash detection so `master` stays continuously green and we can cut a release easier. -->
- [ ] **Status testing**
- [ ] Get QA approval to deploy a new version in `status.staging`.
- [ ] Deploy release candidate to `status.staging`.
- [ ] Perform [sanity check](https://www.notion.so/How-to-test-Nwaku-on-Status-12c6e4b9bf06420ca868bd199129b425) and log results as comments in this issue.
- [ ] Connect 2 instances to `status.staging` fleet, one in relay mode, the other one in light client mode.
- 1:1 Chats with each other.
- Send and receive messages in a community.
- Close one instance, send messages with second instance, reopen first instance and confirm messages sent while offline are retrieved from store.
- [ ] Perform checks based on _end user impact_
- [ ] Inform other (Waku and Status) CCs to point their instances to `status.staging` for a few days. Ping Status colleagues on their Discord server or in the [Status community](https://status.app/c/G3kAAMSQtb05kog3aGbr3kiaxN4tF5xy4BAGEkkLwILk2z3GcoYlm5hSJXGn7J3laft-tnTwDWmYJ18dP_3bgX96dqr_8E3qKAvxDf3NrrCMUBp4R9EYkQez9XSM4486mXoC3mIln2zc-TNdvjdfL9eHVZ-mGgs=#zQ3shZeEJqTC1xhGUjxuS4rtHSrhJ8vUYp64v6qWkLpvdy9L9) (this is not a blocking point.)
- [ ] Ask QA to perform sanity checks (as described above) and checks based on _end user impact_; specify the version being tested
- [ ] Ask QA or infra to run the automated Status e2e tests against `status.staging`
- [ ] Get other CCs' sign-off: they should comment on this PR, e.g., "Used the app for a week, no problem." If problems are reported, resolve them and create a new RC.
- [ ] 3. DST sign-off (pre-requisite: 1)
- [ ] Inform the DST team about the expectations for this release. For example, if we expect higher, same or lower bandwidth consumption, or a new protocol appears, etc.
- [ ] Ask DST to add a comment approving this release and add a summary analysis report.
- [ ] **Proceed with release**
- [ ] 4. Status testing (pre-requisite: 1, 2, 3)
- [ ] Bump logos-delivery dependency in [logos-delivery-go-bindings](https://github.com/logos-messaging/logos-delivery-go-bindings) and make sure all tests work.
- [ ] Submit a PR on [status-go](https://github.com/status-im/status-go/blob/1f9061064587e1167e32d965d5a6f2b745324d5e/tests-functional/docker-compose.waku.yml#L3) bumping logos-delivery to this release candidate.
- [ ] Submit a PR on [status-app](https://github.com/status-im/status-app/blob/3639e28374ca3c2158ac2dac6af35dbce439b10a/docker-compose.waku.yml#L3) bumping logos-delivery to this release candidate.
- [ ] Both PRs must be merged before the release is considered created.
- [ ] Deploy the release candidate in status.staging. That may alert about needed infra changes.
- [ ] Assign a final release tag (`v0.X.0`) to the same commit that contains the validated release-candidate tag (e.g. `git tag -as v0.X.0 -m "final release."`).
- [ ] 5. Submit a PR against release/v0.X with CHANGELOG.md updates (pre-requisite: all previous.)
#### Complete the release
- [ ] Assign a final release tag (`v0.X.0`) to the same commit that contains the latest release-candidate tag (e.g. `git tag -as v0.X.0 -m "final release."`).
- [ ] Update [logos-delivery-compose](https://github.com/logos-messaging/logos-delivery-compose) and [logos-delivery-simulator](https://github.com/logos-messaging/logos-delivery-simulator) according to the new release.
- [ ] Bump logos-delivery dependency in [logos-delivery-rust-bindings](https://github.com/logos-messaging/logos-delivery-rust-bindings) and make sure all examples and tests work.
- [ ] Bump logos-delivery dependency in [logos-delivery-go-bindings](https://github.com/logos-messaging/logos-delivery-go-bindings) and make sure all tests work.
- [ ] Create GitHub release (https://github.com/logos-messaging/logos-delivery/releases).
- [ ] Submit a PR to merge the release branch back to `master`. Make sure you use the option "Merge pull request (Create a merge commit)" to perform the merge. Ping repo admin if this option is not available.
- [ ] Create a deployment issue with the recently created release.
- [ ] Merge release branch into master
- [ ] Create a temporary branch from the release branch. This is needed in case we need to rebase that branch without impacting the release branch. Notice that the release branch should live forever.
- [ ] Submit a PR from temporary branch to master. Make sure you use the option "Merge pull request (Create a merge commit)" to perform the merge. Ping repo admin if this option is not available.
- [ ] Adjust the status-go and status-app `docker-compose.waku.yml` files (the ones bumped above) so they use the final tag instead of the release candidate one.
#### Deployment
- [ ] Deploy to waku.sandbox
- [ ] Confirm the release candidate ran properly in waku.test with the current infra config.
- [ ] If it did NOT (e.g. CLI params changed): submit PRs to the infra repos adjusting the deprecated or changed arguments (review CHANGELOG.md for that release), add links to them, and wait until they are merged. Infra changes are deployed by the infra team, so this requires coordination with them.
- [ ] If it ran fine with the current config, no infra PR is needed.
- [ ] Deploy:
- [ ] Coordinate with the Infra Team about the deployment timing.
- [ ] Update waku.sandbox with [this deployment job](https://ci.infra.status.im/job/nim-waku/job/deploy-waku-sandbox/).
- [ ] Confirm the fleet runs properly after the deployment.
- [ ] Deploy to status.prod
- [ ] Confirm the release candidate ran properly in status.staging with the current infra config.
- [ ] If it did NOT (e.g. CLI params changed): submit PRs to the infra repos adjusting the deprecated or changed arguments (review CHANGELOG.md for that release), add links to them, and wait until they are merged. Infra changes are deployed by the infra team, so this requires coordination with them.
- [ ] If it ran fine with the current config, no infra PR is needed.
- [ ] Deploy:
- [ ] Ask the Status admin to add a comment approving that this deployment happen now.
- [ ] Update status.prod with [this deployment job](https://ci.infra.status.im/job/nim-waku/job/deploy-status-prod/).
- [ ] Confirm the fleet runs properly after the deployment.
### Links
@ -73,6 +90,7 @@ All items below are to be completed by the owner of the given release.
- [Release notes](https://github.com/logos-messaging/logos-delivery/blob/master/CHANGELOG.md)
- [Fleet ownership](https://www.notion.so/Fleet-Ownership-7532aad8896d46599abac3c274189741?pvs=4#d2d2f0fe4b3c429fbd860a1d64f89a64)
- [Infra-logos](https://github.com/status-im/infra-logos)
- [Infra-Status](https://github.com/status-im/infra-status)
- [Jenkins](https://ci.infra.status.im/)
- [Fleets](https://fleets.logos.co/)
- [Harbor](https://harbor.status.im/harbor/projects/32/repositories/logos-node/artifacts-tab)

34
.github/workflows/ci.yml vendored
View File

@ -101,7 +101,25 @@ jobs:
touch nimbledeps/.nimble-setup
- name: Build binaries
run: make V=1 all
# -j1: `all` builds wakunode2 and liblogosdelivery, each via `nimble
# <task>`. Under -j2 the two nimble invocations re-resolve git deps
# concurrently and clobber each other in the shared ~/.nimble/pkgcache
# (e.g. "destination already exists", "could not lock config file"),
# failing the build. Serializing the targets removes the race; Nim's
# own --parallelBuild still parallelizes compilation. Mirrors the test
# job, which already forces -j1.
run: make V=1 -j1 all
# Publish the freshly built shared library so the interop-tests workflow
# can run the wrapper suite against it without rebuilding. Only the Linux
# build is uploaded since the interop runners are ubuntu-latest.
- name: Upload liblogosdelivery.so
if: matrix.os == 'ubuntu-22.04'
uses: actions/upload-artifact@v4
with:
name: liblogosdelivery
path: build/liblogosdelivery.so
retention-days: 1
build-windows:
needs: changes
@ -118,7 +136,7 @@ jobs:
matrix:
os: [ubuntu-22.04, macos-15]
runs-on: ${{ matrix.os }}
timeout-minutes: 45
timeout-minutes: 60
name: test-${{ matrix.os }}
steps:
@ -177,14 +195,24 @@ jobs:
uses: ./.github/workflows/container-image.yml
secrets: inherit
# Docker-based interop smoke tests. Gated only on the docker image so they
# still run on docker-only PRs (no liblogosdelivery build required).
nwaku-nwaku-interop-tests:
needs: build-docker-image
uses: logos-messaging/logos-delivery-interop-tests/.github/workflows/nim_waku_PR.yml@SMOKE_TEST_STABLE
uses: logos-messaging/logos-delivery-interop-tests/.github/workflows/nim_waku_PR.yml@SMOKE_TEST_2026.06.25
with:
node_nwaku: ${{ needs.build-docker-image.outputs.image }}
secrets: inherit
# Send API E2E tests run the wrapper suite against the liblogosdelivery.so
# built by the `build` job (downloaded as the `liblogosdelivery` artifact).
# Gated on `build`, so it is skipped on docker-only PRs where no lib is built.
send-api-e2e-tests:
needs: build
uses: logos-messaging/logos-delivery-interop-tests/.github/workflows/send_api_e2e_PR.yml@SMOKE_TEST_2026.06.25
secrets: inherit
lint:
name: "Lint"
runs-on: ubuntu-22.04

7
README.md
View File

@ -11,7 +11,7 @@ private communications.
- Examples.
- Various tests of above.
For more details see the [source code](waku/README.md)
For more details see the [source code](logos_delivery/waku/README.md)
## How to Build & Run ( Linux, MacOS & WSL )
@ -48,11 +48,14 @@ make wakunode2 NIMFLAGS="-d:chronicles_colors:none -d:disableMarchNative"
# Run with DNS bootstrapping
./build/wakunode2 --dns-discovery --dns-discovery-url=DNS_BOOTSTRAP_NODE_URL
# Run with the QUIC transport enabled
./build/wakunode2 --quic-support=true
# See available command line options
./build/wakunode2 --help
```
To join the network, you need to know the address of at least one bootstrap node.
Please refer to the [Waku README](https://github.com/logos-messaging/logos-delivery/blob/master/waku/README.md) for more information.
Please refer to the [Waku README](https://github.com/logos-messaging/logos-delivery/blob/master/logos_delivery/waku/README.md) for more information.
For more on how to run `wakunode2`, refer to:
- [Run using binaries](https://docs.waku.org/run-node/build-source)

10
apps/benchmarks/benchmarks.nim
View File

@ -3,11 +3,11 @@ import
import
logos_delivery/waku/[
waku_rln_relay/protocol_types,
waku_rln_relay/rln,
waku_rln_relay,
waku_rln_relay/conversion_utils,
waku_rln_relay/group_manager/on_chain/group_manager,
rln/protocol_types,
rln/bindings,
rln,
rln/conversion_utils,
rln/group_manager/on_chain/group_manager,
],
tests/waku_rln_relay/utils_onchain

26
apps/chat2/chat2.nim
View File

@ -48,7 +48,7 @@ import
./config_chat2
import libp2p/protocols/pubsub/rpc/messages, libp2p/protocols/pubsub/pubsub
import ../../logos_delivery/waku/waku_rln_relay
import ../../logos_delivery/waku/rln
const Help = """
Commands: /[?|help|connect|nick|exit]
@ -189,11 +189,11 @@ proc publish(c: Chat, line: string) =
timestamp: getNanosecondTime(time),
)
if not isNil(c.node.wakuRlnRelay):
if not isNil(c.node.rln):
# for future version when we support more than one rln protected content topic,
# we should check the message content topic as well
let proofRes =
waitFor c.node.wakuRlnRelay.generateRLNProof(message.toRLNSignal(), float64(time))
waitFor c.node.rln.generateRLNProof(message.toRLNSignal(), float64(time))
if proofRes.isErr():
info "could not append rate limit proof to the message"
else:
@ -204,13 +204,13 @@ proc publish(c: Chat, line: string) =
return
# TODO move it to log after dogfooding
let msgEpoch = fromEpoch(proof.epoch)
if fromEpoch(c.node.wakuRlnRelay.lastEpoch) == msgEpoch:
if fromEpoch(c.node.rln.lastEpoch) == msgEpoch:
echo "--rln epoch: ",
msgEpoch, " ⚠️ message rate violation! you are spamming the network!"
else:
echo "--rln epoch: ", msgEpoch
# update the last epoch
c.node.wakuRlnRelay.lastEpoch = proof.epoch
c.node.rln.lastEpoch = proof.epoch
try:
if not c.node.wakuLegacyLightPush.isNil():
@ -521,7 +521,7 @@ proc processInput(rfd: AsyncFD, rng: crypto.Rng) {.async.} =
topic = DefaultPubsubTopic, error = error
if conf.rlnRelay:
info "WakuRLNRelay is enabled"
info "Rln is enabled"
proc spamHandler(wakuMessage: WakuMessage) {.gcsafe, closure.} =
info "spam handler is called"
@ -538,24 +538,22 @@ proc processInput(rfd: AsyncFD, rng: crypto.Rng) {.async.} =
chainId: UInt256.fromBytesBE(conf.rlnRelayChainId.toBytesBE()),
ethClientUrls: conf.ethClientUrls.mapIt(string(it)),
creds: some(
RlnRelayCreds(
path: conf.rlnRelayCredPath, password: conf.rlnRelayCredPassword
)
RlnCreds(path: conf.rlnRelayCredPath, password: conf.rlnRelayCredPassword)
),
userMessageLimit: conf.rlnRelayUserMessageLimit,
epochSizeSec: conf.rlnEpochSizeSec,
)
waitFor node.mountRlnRelay(rlnConf, spamHandler = some(spamHandler))
waitFor node.setRlnValidator(rlnConf, spamHandler = some(spamHandler))
let membershipIndex = node.wakuRlnRelay.groupManager.membershipIndex.get()
let identityCredential = node.wakuRlnRelay.groupManager.idCredentials.get()
let membershipIndex = node.rln.groupManager.membershipIndex.get()
let identityCredential = node.rln.groupManager.idCredentials.get()
echo "your membership index is: ", membershipIndex
echo "your rln identity commitment key is: ",
identityCredential.idCommitment.inHex()
else:
info "WakuRLNRelay is disabled"
echo "WakuRLNRelay is disabled, please enable it by passing in the --rln-relay flag"
info "Rln is disabled"
echo "Rln is disabled, please enable it by passing in the --rln-relay flag"
if conf.metricsLogging:
startMetricsLog()

2
apps/chat2mix/chat2mix.nim
View File

@ -56,7 +56,7 @@ import
./config_chat2mix
import libp2p/protocols/pubsub/rpc/messages, libp2p/protocols/pubsub/pubsub
import ../../logos_delivery/waku/waku_rln_relay
import ../../logos_delivery/waku/rln
logScope:
topics = "chat2 mix"

2
apps/liteprotocoltester/liteprotocoltester.nim
View File

@ -14,7 +14,7 @@ import
logos_delivery/waku/[
common/enr,
common/logging,
factory/waku as waku_factory,
waku as waku_factory,
waku_node,
node/waku_metrics,
node/peer_manager,

6
apps/networkmonitor/networkmonitor.nim
View File

@ -25,7 +25,7 @@ import
discovery/waku_discv5,
discovery/waku_dnsdisc,
waku_relay,
waku_rln_relay,
rln,
factory/builder,
factory/networks_config,
],
@ -614,12 +614,12 @@ when isMainModule:
ethContractAddress: conf.rlnRelayEthContractAddress,
ethClientUrls: conf.ethClientUrls.mapIt(string(it)),
epochSizeSec: conf.rlnEpochSizeSec,
creds: none(RlnRelayCreds),
creds: none(RlnCreds),
onFatalErrorAction: onFatalErrorAction,
)
try:
waitFor node.mountRlnRelay(rlnConf)
waitFor node.setRlnValidator(rlnConf)
except CatchableError:
error "failed to setup RLN", error = getCurrentExceptionMsg()
quit(QuitFailure)

2
apps/wakunode2/wakunode2.nim
View File

@ -11,7 +11,7 @@ import
../../tools/[rln_keystore_generator/rln_keystore_generator, confutils/cli_args],
logos_delivery/waku/[
common/logging,
factory/waku,
waku,
node/health_monitor,
rest_api/endpoint/builder as rest_server_builder,
waku_core/message/default_values,

21
config.nims
View File

@ -63,6 +63,27 @@ else:
switch("passC", "-mno-avx512f")
switch("passL", "-mno-avx512f")
when defined(macosx):
# TEMPORARY (added 2026-06-19) — remove once nim-chronos is fixed/bumped upstream.
#
# macOS / Apple Clang 16+ promotes -Wincompatible-function-pointer-types to a
# DEFAULT error (not gated behind -Werror). It fires on a dependency↔dependency
# call in nim-chronos — none of our code is involved:
# chronos/streams/tlsstream.nim:718 ctx.setdest(itemAppend, ...)
# chronos declares the PEM callback `itemAppend(ctx, pbytes: pointer, nbytes)`
# while bearssl's `br_pem_decoder_setdest` expects `pbytes: const void*`. The
# callback only READS the buffer (copyMem from pbytes), so the missing `const`
# is benign — we demote the diagnostic back to a warning instead of failing.
#
# Why it appeared now (not a change on our side): the `const` was introduced in
# nim-bearssl v0.2.9, published 2026-06-19 11:22 UTC. CI does not honor the
# locked bearssl (nimble.lock pins 0.2.8) and fresh-resolves to latest, so any
# run after 11:22 UTC picks up 0.2.9 and hits this. Earlier-running PRs that
# resolved 0.2.8 are green only by timing; re-running them now reproduces it.
#
# Proper fix: chronos should declare `itemAppend`'s `pbytes` as `const pointer`.
switch("passC", "-Wno-error=incompatible-function-pointer-types")
--threads:
on
--opt:

20
docs/operators/how-to/configure-quic.md Normal file
View File

@ -0,0 +1,20 @@
# Configure QUIC transport
QUIC is a UDP-based transport. Enabling it allows peers to connect to your node over QUIC, in addition to the default TCP transport.
To enable QUIC, use the `--quic-support` option.
Note, the default port for QUIC is 60000.
```shell
wakunode2 --quic-support=true
```
To listen on a different UDP port, use `--quic-port`:
```shell
wakunode2 --quic-support=true --quic-port=<port>
```
QUIC runs alongside the existing TCP transport. The node keeps listening on TCP and announces a `/udp/<port>/quic-v1` address in its ENR, so peers that support QUIC can connect over it while others continue to use TCP.
If you restrict the node's announced addresses with `--ext-multiaddr-only`, the QUIC address is no longer announced automatically. In that case, include the QUIC multiaddr in `--ext-multiaddr` yourself, for example `/ip4/<ip>/udp/<port>/quic-v1`.

5
docs/operators/how-to/configure.md
View File

@ -132,5 +132,6 @@ This is an index of tutorials explaining how to configure your nwaku node for di
4. [Configure store protocol and message store](./configure-store.md)
5. [Generate and configure a node key](./configure-key.md)
6. [Configure websocket transport](./configure-websocket.md)
7. [Run nwaku with rate limiting enabled](./run-with-rln.md)
8. [Configure a REST API node](./configure-rest-api.md)
7. [Configure QUIC transport](./configure-quic.md)
8. [Run nwaku with rate limiting enabled](./run-with-rln.md)
9. [Configure a REST API node](./configure-rest-api.md)

6
docs/operators/how-to/run.md
View File

@ -56,6 +56,12 @@ and websocket address
/ip4/0.0.0.0/tcp/8000/ws/p2p/16Uiu2HAkzjwwgEAXfeGNMKFPSpc6vGBRqCdTLG5q3Gmk2v4pQw7H
```
If your node is running with QUIC enabled, the log also includes a QUIC address
```
/ip4/0.0.0.0/udp/60000/quic-v1/p2p/16Uiu2HAkzjwwgEAXfeGNMKFPSpc6vGBRqCdTLG5q3Gmk2v4pQw7H
```
You can also query a running node for its listening addresses
using the REST API.

14
examples/api_example/api_example.nim
View File

@ -7,7 +7,7 @@ type CliArgs = object
defaultValue: "", desc: "ETH RPC Endpoint, if passed, RLN is enabled"
.}: string
proc periodicSender(w: Waku): Future[void] {.async.} =
proc periodicSender(logos: LogosDelivery): Future[void] {.async.} =
let sentListener = MessageSentEvent.listen(
proc(event: MessageSentEvent) {.async: (raises: []).} =
echo "Message sent with request ID: ",
@ -45,7 +45,7 @@ proc periodicSender(w: Waku): Future[void] {.async.} =
payload = "Hello Waku! Message number: " & $counter,
)
let sendRequestId = (await w.send(envelope)).valueOr:
let sendRequestId = (await logos.messagingClient.send(envelope)).valueOr:
echo "Failed to send message: ", error
quit(QuitFailure)
@ -75,16 +75,12 @@ when isMainModule:
conf.preset = "twn"
conf.ethClientUrls = @[EthRpcUrl(args.ethRpcEndpoint)]
# Create the node using the library API's createNode function
let node = (waitFor createNode(conf)).valueOr:
# Create the full Logos Messaging stack (Waku + messaging + channels)
let node = (waitFor LogosDelivery.new(conf)).valueOr:
echo "Failed to create node: ", error
quit(QuitFailure)
echo("Waku node created successfully!")
node.mountMessagingClient().isOkOr:
echo "Failed to mount messaging: ", error
quit(QuitFailure)
echo("Logos Messaging node created successfully!")
# Start the node
(waitFor node.start()).isOkOr:

125
examples/cbindings/waku_example.c
View File

@ -1,7 +1,7 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <argp.h>
#include <getopt.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdint.h>
@ -50,12 +50,6 @@ struct ConfigNode
int relay;
char peers[2048];
int store;
char storeNode[2048];
char storeRetentionPolicy[64];
char storeDbUrl[256];
int storeVacuum;
int storeDbMigration;
int storeMaxNumDbConnections;
};
// libwaku Context
@ -64,51 +58,35 @@ void *ctx;
// For the case of C language we don't need to store a particular userData
void *userData = NULL;
// Arguments parsing
static char doc[] = "\nC example that shows how to use the waku library.";
static char args_doc[] = "";
static struct argp_option options[] = {
{"host", 'h', "HOST", 0, "IP to listen for for LibP2P traffic. (default: \"0.0.0.0\")"},
{"port", 'p', "PORT", 0, "TCP listening port. (default: \"60000\")"},
{"key", 'k', "KEY", 0, "P2P node private key as 64 char hex string."},
{"relay", 'r', "RELAY", 0, "Enable relay protocol: 1 or 0. (default: 1)"},
{"peers", 'a', "PEERS", 0, "Comma-separated list of peer-multiaddress to connect\
to. (default: \"\") e.g. \"/ip4/127.0.0.1/tcp/60001/p2p/16Uiu2HAmVFXtAfSj4EiR7mL2KvL4EE2wztuQgUSBoj2Jx2KeXFLN\""},
{0}};
static error_t parse_opt(int key, char *arg, struct argp_state *state)
// Arguments parsing. Uses POSIX getopt so the example builds on glibc and on
// macOS/BSD alike (argp is a GNU libc extension not available everywhere).
static void parse_args(int argc, char **argv, struct ConfigNode *cfgNode)
{
struct ConfigNode *cfgNode = state->input;
switch (key)
int opt;
while ((opt = getopt(argc, argv, "h:p:k:r:a:")) != -1)
{
case 'h':
snprintf(cfgNode->host, 128, "%s", arg);
break;
case 'p':
cfgNode->port = atoi(arg);
break;
case 'k':
snprintf(cfgNode->key, 128, "%s", arg);
break;
case 'r':
cfgNode->relay = atoi(arg);
break;
case 'a':
snprintf(cfgNode->peers, 2048, "%s", arg);
break;
case ARGP_KEY_ARG:
if (state->arg_num >= 1) /* Too many arguments. */
argp_usage(state);
break;
case ARGP_KEY_END:
break;
default:
return ARGP_ERR_UNKNOWN;
switch (opt)
{
case 'h':
snprintf(cfgNode->host, 128, "%s", optarg);
break;
case 'p':
cfgNode->port = atoi(optarg);
break;
case 'k':
snprintf(cfgNode->key, 128, "%s", optarg);
break;
case 'r':
cfgNode->relay = atoi(optarg);
break;
case 'a':
snprintf(cfgNode->peers, 2048, "%s", optarg);
break;
default:
printf("Wrong parameters\n");
exit(1);
}
}
return 0;
}
void signal_cond()
@ -119,8 +97,6 @@ void signal_cond()
pthread_mutex_unlock(&mutex);
}
static struct argp argp = {options, parse_opt, args_doc, doc, 0, 0, 0};
void event_handler(int callerRet, const char *msg, size_t len, void *userData)
{
if (callerRet == RET_ERR)
@ -308,47 +284,26 @@ int main(int argc, char **argv)
cfgNode.relay = 1;
cfgNode.store = 0;
snprintf(cfgNode.storeNode, 2048, "");
snprintf(cfgNode.storeRetentionPolicy, 64, "time:6000000");
snprintf(cfgNode.storeDbUrl, 256, "postgres://postgres:test123@localhost:5432/postgres");
cfgNode.storeVacuum = 0;
cfgNode.storeDbMigration = 0;
cfgNode.storeMaxNumDbConnections = 30;
if (argp_parse(&argp, argc, argv, 0, 0, &cfgNode) == ARGP_ERR_UNKNOWN)
{
show_help_and_exit();
}
parse_args(argc, argv, &cfgNode);
char jsonConfig[5000];
snprintf(jsonConfig, 5000, "{ \
\"clusterId\": 16, \
\"shards\": [ 1, 32, 64, 128, 256 ], \
\"numShardsInNetwork\": 257, \
\"listenAddress\": \"%s\", \
\"tcpPort\": %d, \
\"relay\": %s, \
\"store\": %s, \
\"storeMessageDbUrl\": \"%s\", \
\"storeMessageRetentionPolicy\": \"%s\", \
\"storeMaxNumDbConnections\": %d , \
\"logLevel\": \"DEBUG\", \
\"discv5Discovery\": true, \
\"discv5BootstrapNodes\": \
[\"enr:-QEKuED9AJm2HGgrRpVaJY2nj68ao_QiPeUT43sK-aRM7sMJ6R4G11OSDOwnvVacgN1sTw-K7soC5dzHDFZgZkHU0u-XAYJpZIJ2NIJpcISnYxMvim11bHRpYWRkcnO4WgAqNiVib290LTAxLmRvLWFtczMuc3RhdHVzLnByb2Quc3RhdHVzLmltBnZfACw2JWJvb3QtMDEuZG8tYW1zMy5zdGF0dXMucHJvZC5zdGF0dXMuaW0GAbveA4Jyc40AEAUAAQAgAEAAgAEAiXNlY3AyNTZrMaEC3rRtFQSgc24uWewzXaxTY8hDAHB8sgnxr9k8Rjb5GeSDdGNwgnZfg3VkcIIjKIV3YWt1Mg0\", \"enr:-QEcuED7ww5vo2rKc1pyBp7fubBUH-8STHEZHo7InjVjLblEVyDGkjdTI9VdqmYQOn95vuQH-Htku17WSTzEufx-Wg4mAYJpZIJ2NIJpcIQihw1Xim11bHRpYWRkcnO4bAAzNi5ib290LTAxLmdjLXVzLWNlbnRyYWwxLWEuc3RhdHVzLnByb2Quc3RhdHVzLmltBnZfADU2LmJvb3QtMDEuZ2MtdXMtY2VudHJhbDEtYS5zdGF0dXMucHJvZC5zdGF0dXMuaW0GAbveA4Jyc40AEAUAAQAgAEAAgAEAiXNlY3AyNTZrMaECxjqgDQ0WyRSOilYU32DA5k_XNlDis3m1VdXkK9xM6kODdGNwgnZfg3VkcIIjKIV3YWt1Mg0\", \"enr:-QEcuEAoShWGyN66wwusE3Ri8hXBaIkoHZHybUB8cCPv5v3ypEf9OCg4cfslJxZFANl90s-jmMOugLUyBx4EfOBNJ6_VAYJpZIJ2NIJpcIQI2hdMim11bHRpYWRkcnO4bAAzNi5ib290LTAxLmFjLWNuLWhvbmdrb25nLWMuc3RhdHVzLnByb2Quc3RhdHVzLmltBnZfADU2LmJvb3QtMDEuYWMtY24taG9uZ2tvbmctYy5zdGF0dXMucHJvZC5zdGF0dXMuaW0GAbveA4Jyc40AEAUAAQAgAEAAgAEAiXNlY3AyNTZrMaEDP7CbRk-YKJwOFFM4Z9ney0GPc7WPJaCwGkpNRyla7mCDdGNwgnZfg3VkcIIjKIV3YWt1Mg0\"], \
\"discv5UdpPort\": 9999, \
\"dnsDiscoveryUrl\": \"enrtree://AMOJVZX4V6EXP7NTJPMAYJYST2QP6AJXYW76IU6VGJS7UVSNDYZG4@boot.prod.status.nodes.status.im\", \
\"dnsDiscoveryNameServers\": [\"8.8.8.8\", \"1.0.0.1\"] \
\"mode\": \"Core\", \
\"preset\": \"status.prod\", \
\"messagingOverrides\": { \
\"listen-address\": \"%s\", \
\"tcp-port\": %d, \
\"store\": %s, \
\"log-level\": \"DEBUG\", \
\"discv5-udp-port\": 9999 \
} \
}",
cfgNode.host,
cfgNode.port,
cfgNode.relay ? "true" : "false",
cfgNode.store ? "true" : "false",
cfgNode.storeDbUrl,
cfgNode.storeRetentionPolicy,
cfgNode.storeMaxNumDbConnections);
cfgNode.store ? "true" : "false");
ctx = waku_new(jsonConfig, event_handler, userData);
ctx = logosdelivery_create_node(jsonConfig, event_handler, userData);
waitForCallback();
WAKU_CALL(waku_default_pubsub_topic(ctx, print_default_pubsub_topic, userData));
@ -359,7 +314,7 @@ int main(int argc, char **argv)
logosdelivery_set_event_callback(ctx, on_event_received, userData);
waku_start(ctx, event_handler, userData);
logosdelivery_start_node(ctx, event_handler, userData);
waitForCallback();
WAKU_CALL(waku_listen_addresses(ctx, event_handler, userData));

103
examples/cpp/waku.cpp
View File

@ -1,7 +1,7 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <argp.h>
#include <getopt.h>
#include <signal.h>
#include <unistd.h>
#include <fcntl.h>
@ -60,51 +60,35 @@ struct ConfigNode
char peers[2048];
};
// Arguments parsing
static char doc[] = "\nC example that shows how to use the waku library.";
static char args_doc[] = "";
static struct argp_option options[] = {
{"host", 'h', "HOST", 0, "IP to listen for for LibP2P traffic. (default: \"0.0.0.0\")"},
{"port", 'p', "PORT", 0, "TCP listening port. (default: \"60000\")"},
{"key", 'k', "KEY", 0, "P2P node private key as 64 char hex string."},
{"relay", 'r', "RELAY", 0, "Enable relay protocol: 1 or 0. (default: 1)"},
{"peers", 'a', "PEERS", 0, "Comma-separated list of peer-multiaddress to connect\
to. (default: \"\") e.g. \"/ip4/127.0.0.1/tcp/60001/p2p/16Uiu2HAmVFXtAfSj4EiR7mL2KvL4EE2wztuQgUSBoj2Jx2KeXFLN\""},
{0}};
static error_t parse_opt(int key, char *arg, struct argp_state *state)
// Arguments parsing. Uses POSIX getopt so the example builds on glibc and on
// macOS/BSD alike (argp is a GNU libc extension not available everywhere).
static void parse_args(int argc, char **argv, struct ConfigNode *cfgNode)
{
struct ConfigNode *cfgNode = (ConfigNode *)state->input;
switch (key)
int opt;
while ((opt = getopt(argc, argv, "h:p:k:r:a:")) != -1)
{
case 'h':
snprintf(cfgNode->host, 128, "%s", arg);
break;
case 'p':
cfgNode->port = atoi(arg);
break;
case 'k':
snprintf(cfgNode->key, 128, "%s", arg);
break;
case 'r':
cfgNode->relay = atoi(arg);
break;
case 'a':
snprintf(cfgNode->peers, 2048, "%s", arg);
break;
case ARGP_KEY_ARG:
if (state->arg_num >= 1) /* Too many arguments. */
argp_usage(state);
break;
case ARGP_KEY_END:
break;
default:
return ARGP_ERR_UNKNOWN;
switch (opt)
{
case 'h':
snprintf(cfgNode->host, 128, "%s", optarg);
break;
case 'p':
cfgNode->port = atoi(optarg);
break;
case 'k':
snprintf(cfgNode->key, 128, "%s", optarg);
break;
case 'r':
cfgNode->relay = atoi(optarg);
break;
case 'a':
snprintf(cfgNode->peers, 2048, "%s", optarg);
break;
default:
printf("Wrong parameters\n");
exit(1);
}
}
return 0;
}
void event_handler(const char *msg, size_t len)
@ -129,8 +113,6 @@ auto cify(F &&f)
};
}
static struct argp argp = {options, parse_opt, args_doc, doc, 0, 0, 0};
// Beginning of UI program logic
enum PROGRAM_STATE
@ -254,33 +236,26 @@ int main(int argc, char **argv)
cfgNode.port = 60000;
cfgNode.relay = 1;
if (argp_parse(&argp, argc, argv, 0, 0, &cfgNode) == ARGP_ERR_UNKNOWN)
{
show_help_and_exit();
}
parse_args(argc, argv, &cfgNode);
char jsonConfig[2048];
snprintf(jsonConfig, 2048, "{ \
\"host\": \"%s\", \
\"port\": %d, \
\"relay\": true, \
\"clusterId\": 16, \
\"shards\": [ 1, 32, 64, 128, 256 ], \
\"logLevel\": \"FATAL\", \
\"discv5Discovery\": true, \
\"discv5BootstrapNodes\": \
[\"enr:-QESuEB4Dchgjn7gfAvwB00CxTA-nGiyk-aALI-H4dYSZD3rUk7bZHmP8d2U6xDiQ2vZffpo45Jp7zKNdnwDUx6g4o6XAYJpZIJ2NIJpcIRA4VDAim11bHRpYWRkcnO4XAArNiZub2RlLTAxLmRvLWFtczMud2FrdS5zYW5kYm94LnN0YXR1cy5pbQZ2XwAtNiZub2RlLTAxLmRvLWFtczMud2FrdS5zYW5kYm94LnN0YXR1cy5pbQYfQN4DgnJzkwABCAAAAAEAAgADAAQABQAGAAeJc2VjcDI1NmsxoQOvD3S3jUNICsrOILlmhENiWAMmMVlAl6-Q8wRB7hidY4N0Y3CCdl-DdWRwgiMohXdha3UyDw\", \"enr:-QEkuEBIkb8q8_mrorHndoXH9t5N6ZfD-jehQCrYeoJDPHqT0l0wyaONa2-piRQsi3oVKAzDShDVeoQhy0uwN1xbZfPZAYJpZIJ2NIJpcIQiQlleim11bHRpYWRkcnO4bgA0Ni9ub2RlLTAxLmdjLXVzLWNlbnRyYWwxLWEud2FrdS5zYW5kYm94LnN0YXR1cy5pbQZ2XwA2Ni9ub2RlLTAxLmdjLXVzLWNlbnRyYWwxLWEud2FrdS5zYW5kYm94LnN0YXR1cy5pbQYfQN4DgnJzkwABCAAAAAEAAgADAAQABQAGAAeJc2VjcDI1NmsxoQKnGt-GSgqPSf3IAPM7bFgTlpczpMZZLF3geeoNNsxzSoN0Y3CCdl-DdWRwgiMohXdha3UyDw\"], \
\"discv5UdpPort\": 9999, \
\"dnsDiscoveryUrl\": \"enrtree://AMOJVZX4V6EXP7NTJPMAYJYST2QP6AJXYW76IU6VGJS7UVSNDYZG4@boot.prod.status.nodes.status.im\", \
\"dnsDiscoveryNameServers\": [\"8.8.8.8\", \"1.0.0.1\"] \
\"mode\": \"Core\", \
\"preset\": \"status.prod\", \
\"messagingOverrides\": { \
\"listen-address\": \"%s\", \
\"tcp-port\": %d, \
\"log-level\": \"FATAL\", \
\"discv5-udp-port\": 9999 \
} \
}",
cfgNode.host,
cfgNode.port);
void *ctx =
waku_new(jsonConfig,
logosdelivery_create_node(jsonConfig,
cify([](const char *msg, size_t len)
{ std::cout << "waku_new feedback: " << msg << std::endl; }),
{ std::cout << "logosdelivery_create_node feedback: " << msg << std::endl; }),
nullptr);
waitForCallback();
@ -317,7 +292,7 @@ int main(int argc, char **argv)
{ event_handler(msg, len); }),
nullptr);
WAKU_CALL(waku_start(ctx,
WAKU_CALL(logosdelivery_start_node(ctx,
cify([&](const char *msg, size_t len)
{ event_handler(msg, len); }),
nullptr));

34
examples/golang/waku.go
View File

@ -71,20 +71,20 @@ package main
static void* cGoWakuNew(const char* configJson, void* resp) {
// We pass NULL because we are not interested in retrieving data from this callback
void* ret = waku_new(configJson, (FFICallBack) callback, resp);
void* ret = logosdelivery_create_node(configJson, (FFICallBack) callback, resp);
return ret;
}
static void cGoWakuStart(void* wakuCtx, void* resp) {
WAKU_CALL(waku_start(wakuCtx, (FFICallBack) callback, resp));
WAKU_CALL(logosdelivery_start_node(wakuCtx, (FFICallBack) callback, resp));
}
static void cGoWakuStop(void* wakuCtx, void* resp) {
WAKU_CALL(waku_stop(wakuCtx, (FFICallBack) callback, resp));
WAKU_CALL(logosdelivery_stop_node(wakuCtx, (FFICallBack) callback, resp));
}
static void cGoWakuDestroy(void* wakuCtx, void* resp) {
WAKU_CALL(waku_destroy(wakuCtx, (FFICallBack) callback, resp));
WAKU_CALL(logosdelivery_destroy(wakuCtx, (FFICallBack) callback, resp));
}
static void cGoWakuStartDiscV5(void* wakuCtx, void* resp) {
@ -293,12 +293,16 @@ type WakuMessageHash = string
type WakuPubsubTopic = string
type WakuContentTopic = string
type MessagingOverrides struct {
ListenAddress string `json:"listen-address,omitempty"`
Port int `json:"tcp-port,omitempty"`
NodeKey string `json:"nodekey,omitempty"`
LogLevel string `json:"log-level,omitempty"`
}
type WakuConfig struct {
Host string `json:"host,omitempty"`
Port int `json:"port,omitempty"`
NodeKey string `json:"key,omitempty"`
EnableRelay bool `json:"relay"`
LogLevel string `json:"logLevel"`
Mode string `json:"mode"`
MessagingOverrides MessagingOverrides `json:"messagingOverrides"`
}
type WakuNode struct {
@ -563,11 +567,13 @@ func (self *WakuNode) WakuGetMyENR() (string, error) {
func main() {
config := WakuConfig{
Host: "0.0.0.0",
Port: 30304,
NodeKey: "11d0dcea28e86f81937a3bd1163473c7fbc0a0db54fd72914849bc47bdf78710",
EnableRelay: true,
LogLevel: "DEBUG",
Mode: "Core",
MessagingOverrides: MessagingOverrides{
ListenAddress: "0.0.0.0",
Port: 30304,
NodeKey: "11d0dcea28e86f81937a3bd1163473c7fbc0a0db54fd72914849bc47bdf78710",
LogLevel: "DEBUG",
},
}
node, err := WakuNew(config)

39
examples/ios/WakuExample/WakuNode.swift
View File

@ -240,11 +240,11 @@ actor WakuActor {
print("[WakuActor] Unsubscribed from filter")
// Stop
_ = await self.callWakuSync { waku_stop(ctxToStop, WakuActor.syncCallback, $0) }
_ = await self.callWakuSync { logosdelivery_stop_node(ctxToStop, WakuActor.syncCallback, $0) }
print("[WakuActor] Node stopped")
// Destroy
_ = await self.callWakuSync { waku_destroy(ctxToStop, WakuActor.syncCallback, $0) }
_ = await self.callWakuSync { logosdelivery_destroy(ctxToStop, WakuActor.syncCallback, $0) }
print("[WakuActor] Node destroyed")
}
}
@ -300,32 +300,27 @@ actor WakuActor {
private func initializeNode() async -> Bool {
let config = """
{
"tcpPort": 60000,
"clusterId": 1,
"shards": [0],
"relay": false,
"lightpush": true,
"filter": true,
"logLevel": "DEBUG",
"discv5Discovery": true,
"discv5BootstrapNodes": [
"enr:-QESuEB4Dchgjn7gfAvwB00CxTA-nGiyk-aALI-H4dYSZD3rUk7bZHmP8d2U6xDiQ2vZffpo45Jp7zKNdnwDUx6g4o6XAYJpZIJ2NIJpcIRA4VDAim11bHRpYWRkcnO4XAArNiZub2RlLTAxLmRvLWFtczMud2FrdS5zYW5kYm94LnN0YXR1cy5pbQZ2XwAtNiZub2RlLTAxLmRvLWFtczMud2FrdS5zYW5kYm94LnN0YXR1cy5pbQYfQN4DgnJzkwABCAAAAAEAAgADAAQABQAGAAeJc2VjcDI1NmsxoQOvD3S3jUNICsrOILlmhENiWAMmMVlAl6-Q8wRB7hidY4N0Y3CCdl-DdWRwgiMohXdha3UyDw",
"enr:-QEkuEBIkb8q8_mrorHndoXH9t5N6ZfD-jehQCrYeoJDPHqT0l0wyaONa2-piRQsi3oVKAzDShDVeoQhy0uwN1xbZfPZAYJpZIJ2NIJpcIQiQlleim11bHRpYWRkcnO4bgA0Ni9ub2RlLTAxLmdjLXVzLWNlbnRyYWwxLWEud2FrdS5zYW5kYm94LnN0YXR1cy5pbQZ2XwA2Ni9ub2RlLTAxLmdjLXVzLWNlbnRyYWwxLWEud2FrdS5zYW5kYm94LnN0YXR1cy5pbQYfQN4DgnJzkwABCAAAAAEAAgADAAQABQAGAAeJc2VjcDI1NmsxoQKnGt-GSgqPSf3IAPM7bFgTlpczpMZZLF3geeoNNsxzSoN0Y3CCdl-DdWRwgiMohXdha3UyDw"
],
"discv5UdpPort": 9999,
"dnsDiscovery": true,
"dnsDiscoveryUrl": "enrtree://AOGYWMBYOUIMOENHXCHILPKY3ZRFEULMFI4DOM442QSZ73TT2A7VI@test.waku.nodes.status.im",
"dnsDiscoveryNameServers": ["8.8.8.8", "1.0.0.1"]
"mode": "Edge",
"messagingOverrides": {
"tcp-port": 60000,
"cluster-id": 1,
"log-level": "DEBUG",
"entry-node": [
"enr:-QESuEB4Dchgjn7gfAvwB00CxTA-nGiyk-aALI-H4dYSZD3rUk7bZHmP8d2U6xDiQ2vZffpo45Jp7zKNdnwDUx6g4o6XAYJpZIJ2NIJpcIRA4VDAim11bHRpYWRkcnO4XAArNiZub2RlLTAxLmRvLWFtczMud2FrdS5zYW5kYm94LnN0YXR1cy5pbQZ2XwAtNiZub2RlLTAxLmRvLWFtczMud2FrdS5zYW5kYm94LnN0YXR1cy5pbQYfQN4DgnJzkwABCAAAAAEAAgADAAQABQAGAAeJc2VjcDI1NmsxoQOvD3S3jUNICsrOILlmhENiWAMmMVlAl6-Q8wRB7hidY4N0Y3CCdl-DdWRwgiMohXdha3UyDw",
"enr:-QEkuEBIkb8q8_mrorHndoXH9t5N6ZfD-jehQCrYeoJDPHqT0l0wyaONa2-piRQsi3oVKAzDShDVeoQhy0uwN1xbZfPZAYJpZIJ2NIJpcIQiQlleim11bHRpYWRkcnO4bgA0Ni9ub2RlLTAxLmdjLXVzLWNlbnRyYWwxLWEud2FrdS5zYW5kYm94LnN0YXR1cy5pbQZ2XwA2Ni9ub2RlLTAxLmdjLXVzLWNlbnRyYWwxLWEud2FrdS5zYW5kYm94LnN0YXR1cy5pbQYfQN4DgnJzkwABCAAAAAEAAgADAAQABQAGAAeJc2VjcDI1NmsxoQKnGt-GSgqPSf3IAPM7bFgTlpczpMZZLF3geeoNNsxzSoN0Y3CCdl-DdWRwgiMohXdha3UyDw"
],
"discv5-udp-port": 9999
}
}
"""
// Create node - waku_new is special, it returns the context directly
// Create node - logosdelivery_create_node is special, it returns the context directly
let createResult = await withCheckedContinuation { (continuation: CheckedContinuation<(ctx: UnsafeMutableRawPointer?, success: Bool, result: String?), Never>) in
let callbackCtx = CallbackContext()
let userDataPtr = Unmanaged.passRetained(callbackCtx).toOpaque()
// Set up a simple callback for waku_new
let newCtx = waku_new(config, { ret, msg, len, userData in
// Set up a simple callback for logosdelivery_create_node
let newCtx = logosdelivery_create_node(config, { ret, msg, len, userData in
guard let userData = userData else { return }
let context = Unmanaged<CallbackContext>.fromOpaque(userData).takeUnretainedValue()
context.success = (ret == RET_OK)
@ -354,7 +349,7 @@ actor WakuActor {
// Start node
let startResult = await callWakuSync { userData in
waku_start(self.ctx, WakuActor.syncCallback, userData)
logosdelivery_start_node(self.ctx, WakuActor.syncCallback, userData)
}
guard startResult.success else {

11
examples/mobile/App.tsx
View File

@ -162,10 +162,13 @@ function App(): React.JSX.Element {
const onClickNew = async () => {
const config = {
host: '0.0.0.0',
port: 42342,
key: '1122334455667788990011223344556677889900112233445566778899000022',
relay: true,
mode: 'Core',
messagingOverrides: {
'listen-address': '0.0.0.0',
'tcp-port': 42342,
nodekey:
'1122334455667788990011223344556677889900112233445566778899000022',
},
};
waku = await WakuFactory.createInstance(config);
};

8
examples/mobile/android/app/src/main/java/com/mobile/WakuModule.kt
View File

@ -123,7 +123,7 @@ class WakuModule(reactContext: ReactApplicationContext) : ReactContextBaseJavaMo
val configStr = stringifyReadableMap(config)
val response = wakuNew(configStr)
if (response.error) {
promise.reject("waku_new", response.errorMessage)
promise.reject("logosdelivery_create_node", response.errorMessage)
} else {
// With this we just indicate to waku_ffi that we have registered a
// closure, for this wakuPtr. Later once a message is received the
@ -140,7 +140,7 @@ class WakuModule(reactContext: ReactApplicationContext) : ReactContextBaseJavaMo
val wakuPtr = BigInteger(ctx).toLong()
val response = wakuStart(wakuPtr)
if (response.error) {
promise.reject("waku_start", response.message)
promise.reject("logosdelivery_start_node", response.message)
} else {
promise.resolve(null)
}
@ -162,7 +162,7 @@ class WakuModule(reactContext: ReactApplicationContext) : ReactContextBaseJavaMo
val wakuPtr = BigInteger(ctx).toLong()
val response = wakuStop(wakuPtr)
if (response.error) {
promise.reject("waku_stop", response.message)
promise.reject("logosdelivery_stop_node", response.message)
} else {
promise.resolve(null)
}
@ -173,7 +173,7 @@ class WakuModule(reactContext: ReactApplicationContext) : ReactContextBaseJavaMo
val wakuPtr = BigInteger(ctx).toLong()
val response = wakuDestroy(wakuPtr)
if (response.error) {
promise.reject("waku_destroy", response.message)
promise.reject("logosdelivery_destroy", response.message)
} else {
promise.resolve(null)
}

8
examples/mobile/android/app/src/main/jni/waku_ffi.c
View File

@ -184,7 +184,7 @@ jobject Java_com_mobile_WakuModule_wakuNew(JNIEnv *env, jobject thiz,
jstring configJson) {
const char *config = (*env)->GetStringUTFChars(env, configJson, 0);
cb_result *result = NULL;
void *wakuPtr = waku_new(config, on_response, (void *)&result);
void *wakuPtr = logosdelivery_create_node(config, on_response, (void *)&result);
jobject response = to_jni_ptr(env, result, wakuPtr);
(*env)->ReleaseStringUTFChars(env, configJson, config);
free_cb_result(result);
@ -194,7 +194,7 @@ jobject Java_com_mobile_WakuModule_wakuNew(JNIEnv *env, jobject thiz,
jobject Java_com_mobile_WakuModule_wakuStart(JNIEnv *env, jobject thiz,
jlong wakuPtr) {
cb_result *result = NULL;
waku_start((void *)wakuPtr, on_response, &result);
logosdelivery_start_node((void *)wakuPtr, on_response, &result);
jobject response = to_jni_result(env, result);
free_cb_result(result);
return response;
@ -212,7 +212,7 @@ jobject Java_com_mobile_WakuModule_wakuVersion(JNIEnv *env, jobject thiz,
jobject Java_com_mobile_WakuModule_wakuStop(JNIEnv *env, jobject thiz,
jlong wakuPtr) {
cb_result *result = NULL;
waku_stop((void *)wakuPtr, on_response, &result);
logosdelivery_stop_node((void *)wakuPtr, on_response, &result);
jobject response = to_jni_result(env, result);
free_cb_result(result);
return response;
@ -221,7 +221,7 @@ jobject Java_com_mobile_WakuModule_wakuStop(JNIEnv *env, jobject thiz,
jobject Java_com_mobile_WakuModule_wakuDestroy(JNIEnv *env, jobject thiz,
jlong wakuPtr) {
cb_result *result = NULL;
waku_destroy((void *)wakuPtr, on_response, &result);
logosdelivery_destroy((void *)wakuPtr, on_response, &result);
jobject response = to_jni_result(env, result);
free_cb_result(result);
return response;

12
examples/nodejs/waku.js
View File

@ -17,11 +17,13 @@ function create_random_string(length) {
var wakuMod = require('bindings')('waku');
var cfg = `{
"host": "0.0.0.0",
"port": 60001,
"key": "364d111d729a6eb6d3e6113e163f017b5ef03a6f94c9b5b7bb1bb36fa5cb07a9",
"relay": true
"logLevel": "DEBUG"
"mode": "Core",
"messagingOverrides": {
"listen-address": "0.0.0.0",
"tcp-port": 60001,
"nodekey": "364d111d729a6eb6d3e6113e163f017b5ef03a6f94c9b5b7bb1bb36fa5cb07a9",
"log-level": "DEBUG"
}
}`
function event_handler(event) {

4
examples/nodejs/waku_addon.c
View File

@ -200,7 +200,7 @@ static napi_value WakuNew(napi_env env, napi_callback_info info) {
str_size = str_size + 1;
napi_get_value_string_utf8(env, args[0], jsonConfig, str_size, &str_size_read);
ctx = waku_new(jsonConfig, event_handler, userData);
ctx = logosdelivery_create_node(jsonConfig, event_handler, userData);
free(jsonConfig);
@ -290,7 +290,7 @@ static napi_value WakuSetEventCallback(napi_env env, napi_callback_info info) {
}
static napi_value WakuStart(napi_env env, napi_callback_info info) {
waku_start(ctx, event_handler, userData);
logosdelivery_start_node(ctx, event_handler, userData);
return NULL;
}

31
examples/python/waku.py
View File

@ -53,31 +53,32 @@ parser.add_argument('--peer', dest='peer', default="",
args = parser.parse_args()
# The next 'json_config' is the item passed to the 'waku_new'.
# The next 'json_config' is the item passed to the 'logosdelivery_create_node'.
json_config = "{ \
\"host\": \"%s\", \
\"port\": %d, \
\"key\": \"%s\", \
\"relay\": %s ,\
\"logLevel\": \"DEBUG\" \
\"mode\": \"Core\", \
\"messagingOverrides\": { \
\"listen-address\": \"%s\", \
\"tcp-port\": %d, \
\"nodekey\": \"%s\", \
\"log-level\": \"DEBUG\" \
} \
}" % (args.host,
int(args.port),
args.key,
"true" if args.relay else "false")
args.key)
callback_type = ctypes.CFUNCTYPE(None, ctypes.c_int, ctypes.c_char_p, ctypes.c_size_t)
# Node creation
libwaku.waku_new.restype = ctypes.c_void_p
libwaku.waku_new.argtypes = [ctypes.c_char_p,
libwaku.logosdelivery_create_node.restype = ctypes.c_void_p
libwaku.logosdelivery_create_node.argtypes = [ctypes.c_char_p,
callback_type,
ctypes.c_void_p]
ctx = libwaku.waku_new(bytes(json_config, 'utf-8'),
ctx = libwaku.logosdelivery_create_node(bytes(json_config, 'utf-8'),
callback_type(
#onErrCb
lambda ret, msg, len:
print("Error calling waku_new: %s",
print("Error calling logosdelivery_create_node: %s",
msg.decode('utf-8'))
),
ctypes.c_void_p(0))
@ -115,12 +116,12 @@ libwaku.logosdelivery_set_event_callback.argtypes = [callback_type, ctypes.c_voi
libwaku.logosdelivery_set_event_callback(callback, ctypes.c_void_p(0))
# Start the node
libwaku.waku_start.argtypes = [ctypes.c_void_p,
libwaku.logosdelivery_start_node.argtypes = [ctypes.c_void_p,
callback_type,
ctypes.c_void_p]
libwaku.waku_start(ctx,
libwaku.logosdelivery_start_node(ctx,
callback_type(lambda ret, msg, len:
print("Error in waku_start: %s" %
print("Error in logosdelivery_start_node: %s" %
msg.decode('utf-8'))),
ctypes.c_void_p(0))

22
examples/qt/main_qt.cpp
View File

@ -17,18 +17,16 @@ int main(int argc, char *argv[]) {
QString jsonConfig = R"(
{
"tcpPort": 60000,
"relay": true,
"logLevel": "TRACE",
"discv5Discovery": true,
"discv5BootstrapNodes": [
"enr:-QESuEB4Dchgjn7gfAvwB00CxTA-nGiyk-aALI-H4dYSZD3rUk7bZHmP8d2U6xDiQ2vZffpo45Jp7zKNdnwDUx6g4o6XAYJpZIJ2NIJpcIRA4VDAim11bHRpYWRkcnO4XAArNiZub2RlLTAxLmRvLWFtczMud2FrdS5zYW5kYm94LnN0YXR1cy5pbQZ2XwAtNiZub2RlLTAxLmRvLWFtczMud2FrdS5zYW5kYm94LnN0YXR1cy5pbQYfQN4DgnJzkwABCAAAAAEAAgADAAQABQAGAAeJc2VjcDI1NmsxoQOvD3S3jUNICsrOILlmhENiWAMmMVlAl6-Q8wRB7hidY4N0Y3CCdl-DdWRwgiMohXdha3UyDw",
"enr:-QEkuEBIkb8q8_mrorHndoXH9t5N6ZfD-jehQCrYeoJDPHqT0l0wyaONa2-piRQsi3oVKAzDShDVeoQhy0uwN1xbZfPZAYJpZIJ2NIJpcIQiQlleim11bHRpYWRkcnO4bgA0Ni9ub2RlLTAxLmdjLXVzLWNlbnRyYWwxLWEud2FrdS5zYW5kYm94LnN0YXR1cy5pbQZ2XwA2Ni9ub2RlLTAxLmdjLXVzLWNlbnRyYWwxLWEud2FrdS5zYW5kYm94LnN0YXR1cy5pbQYfQN4DgnJzkwABCAAAAAEAAgADAAQABQAGAAeJc2VjcDI1NmsxoQKnGt-GSgqPSf3IAPM7bFgTlpczpMZZLF3geeoNNsxzSoN0Y3CCdl-DdWRwgiMohXdha3UyDw"
],
"discv5UdpPort": 9999,
"dnsDiscovery": true,
"dnsDiscoveryUrl": "enrtree://AOGYWMBYOUIMOENHXCHILPKY3ZRFEULMFI4DOM442QSZ73TT2A7VI@test.waku.nodes.status.im",
"dnsDiscoveryNameServers": ["8.8.8.8", "1.0.0.1"]
"mode": "Core",
"messagingOverrides": {
"tcp-port": 60000,
"log-level": "TRACE",
"entry-node": [
"enr:-QESuEB4Dchgjn7gfAvwB00CxTA-nGiyk-aALI-H4dYSZD3rUk7bZHmP8d2U6xDiQ2vZffpo45Jp7zKNdnwDUx6g4o6XAYJpZIJ2NIJpcIRA4VDAim11bHRpYWRkcnO4XAArNiZub2RlLTAxLmRvLWFtczMud2FrdS5zYW5kYm94LnN0YXR1cy5pbQZ2XwAtNiZub2RlLTAxLmRvLWFtczMud2FrdS5zYW5kYm94LnN0YXR1cy5pbQYfQN4DgnJzkwABCAAAAAEAAgADAAQABQAGAAeJc2VjcDI1NmsxoQOvD3S3jUNICsrOILlmhENiWAMmMVlAl6-Q8wRB7hidY4N0Y3CCdl-DdWRwgiMohXdha3UyDw",
"enr:-QEkuEBIkb8q8_mrorHndoXH9t5N6ZfD-jehQCrYeoJDPHqT0l0wyaONa2-piRQsi3oVKAzDShDVeoQhy0uwN1xbZfPZAYJpZIJ2NIJpcIQiQlleim11bHRpYWRkcnO4bgA0Ni9ub2RlLTAxLmdjLXVzLWNlbnRyYWwxLWEud2FrdS5zYW5kYm94LnN0YXR1cy5pbQZ2XwA2Ni9ub2RlLTAxLmdjLXVzLWNlbnRyYWwxLWEud2FrdS5zYW5kYm94LnN0YXR1cy5pbQYfQN4DgnJzkwABCAAAAAEAAgADAAQABQAGAAeJc2VjcDI1NmsxoQKnGt-GSgqPSf3IAPM7bFgTlpczpMZZLF3geeoNNsxzSoN0Y3CCdl-DdWRwgiMohXdha3UyDw"
],
"discv5-udp-port": 9999
}
}
)";

6
examples/qt/waku_handler.h
View File

@ -25,7 +25,7 @@ public:
WakuHandler() : QObject(), ctx(nullptr) {}
void initialize(const QString& jsonConfig, WakuCallBack event_handler, void* userData) {
ctx = waku_new(jsonConfig.toUtf8().constData(), WakuCallBack(event_handler), userData);
ctx = logosdelivery_create_node(jsonConfig.toUtf8().constData(), WakuCallBack(event_handler), userData);
logosdelivery_set_event_callback(ctx, on_event_received, userData);
qDebug() << "Waku context initialized, ready to start.";
@ -33,7 +33,7 @@ public:
Q_INVOKABLE void start() {
if (ctx) {
waku_start(ctx, event_handler, nullptr);
logosdelivery_start_node(ctx, event_handler, nullptr);
qDebug() << "Waku start called with event_handler and userData.";
} else {
qDebug() << "Context is not initialized in start.";
@ -42,7 +42,7 @@ public:
Q_INVOKABLE void stop() {
if (ctx) {
waku_stop(ctx, event_handler, nullptr);
logosdelivery_stop_node(ctx, event_handler, nullptr);
qDebug() << "Waku stop called with event_handler and userData.";
} else {
qDebug() << "Context is not initialized in stop.";

24
examples/rust/src/main.rs
View File

@ -6,7 +6,7 @@ use std::{slice, thread, time};
pub type FFICallBack = unsafe extern "C" fn(c_int, *const c_char, usize, *const c_void);
extern "C" {
pub fn waku_new(
pub fn logosdelivery_create_node(
config_json: *const u8,
cb: FFICallBack,
user_data: *const c_void,
@ -14,7 +14,7 @@ extern "C" {
pub fn waku_version(ctx: *const c_void, cb: FFICallBack, user_data: *const c_void) -> c_int;
pub fn waku_start(ctx: *const c_void, cb: FFICallBack, user_data: *const c_void) -> c_int;
pub fn logosdelivery_start_node(ctx: *const c_void, cb: FFICallBack, user_data: *const c_void) -> c_int;
pub fn waku_default_pubsub_topic(
ctx: *mut c_void,
@ -50,21 +50,23 @@ where
fn main() {
let config_json = "\
{ \
\"host\": \"127.0.0.1\",\
\"port\": 60000, \
\"key\": \"0d714a1fada214dead6dc9c7274581ec20ff292451866e7d6d677dc818e8ccd2\", \
\"relay\": true ,\
\"logLevel\": \"DEBUG\"
\"mode\": \"Core\",\
\"messagingOverrides\": { \
\"listen-address\": \"127.0.0.1\",\
\"tcp-port\": 60000, \
\"nodekey\": \"0d714a1fada214dead6dc9c7274581ec20ff292451866e7d6d677dc818e8ccd2\", \
\"log-level\": \"DEBUG\"
}
}";
unsafe {
// Create the waku node
let closure = |ret: i32, data: &str| {
println!("Ret {ret}. waku_new closure called {data}");
println!("Ret {ret}. logosdelivery_create_node closure called {data}");
};
let cb = get_trampoline(&closure);
let config_json_str = CString::new(config_json).unwrap();
let ctx = waku_new(
let ctx = logosdelivery_create_node(
config_json_str.as_ptr() as *const u8,
cb,
&closure as *const _ as *const c_void,
@ -99,10 +101,10 @@ fn main() {
// Start the Waku node
let closure = |ret: i32, data: &str| {
println!("Ret {ret}. waku_start closure called {data}");
println!("Ret {ret}. logosdelivery_start_node closure called {data}");
};
let cb = get_trampoline(&closure);
let _ret = waku_start(ctx, cb, &closure as *const _ as *const c_void);
let _ret = logosdelivery_start_node(ctx, cb, &closure as *const _ as *const c_void);
}
loop {

2
examples/wakustealthcommitments/node_spec.nim
View File

@ -1,7 +1,7 @@
{.push raises: [].}
import tools/confutils/cli_args
import logos_delivery/waku/[common/logging, factory/[waku, networks_config]]
import logos_delivery/waku/[common/logging, waku, factory/networks_config]
import
std/[options, strutils, os, sequtils],
chronicles,

6
examples/wakustealthcommitments/stealth_commitment_protocol.nim
View File

@ -2,7 +2,7 @@
import
results,
logos_delivery/waku/[common/logging, waku_node, waku_rln_relay],
logos_delivery/waku/[common/logging, waku_node, rln],
./erc_5564_interface as StealthCommitmentFFI,
./node_spec,
./wire_spec
@ -47,9 +47,7 @@ proc sendThruWaku*(
)
message.proof = (
await self.waku.node.wakuRlnRelay.generateRLNProof(
message.toRLNSignal(), float64(time)
)
await self.waku.node.rln.generateRLNProof(message.toRLNSignal(), float64(time))
).valueOr:
return err("could not append rate limit proof to the message: " & error)

28
library/README.md
View File

@ -33,15 +33,31 @@ void *logosdelivery_create_node(
{
"mode": "Core",
"preset": "logos.dev",
"listenAddress": "0.0.0.0",
"tcpPort": 60000,
"discv5UdpPort": 9000
"messagingOverrides": {
"listen-address": "0.0.0.0",
"tcp-port": 60000,
"discv5-udp-port": 9000
}
}
```
Configuration uses flat field names matching `WakuNodeConf` in `tools/confutils/cli_args.nim`.
Use `"preset"` to select a network preset (e.g., `"twn"`, `"logos.dev"`) which auto-configures
entry nodes, cluster ID, sharding, and other network-specific settings.
The configuration object has four optional top-level keys: `mode` (`"Core"` or
`"Edge"`, defaults to `"Core"`), `preset`, `messagingOverrides` (per-field node
config overrides), and `channelsOverrides` (reliable-channel overrides).
Override keys accept the config field name or its CLI switch name (e.g.
`"clusterId"` or `"cluster-id"`); unknown keys are rejected.
Use `"preset"` to select a network preset (e.g., `"twn"`, `"logos.dev"`,
`"status.prod"`) which auto-configures entry nodes, cluster ID, sharding, and
other network-specific settings.
Available presets:
| Preset | Cluster ID | RLN | Sharding | Network |
| --- | --- | --- | --- | --- |
| `twn` | 1 | on | auto (8 shards) | The Waku Network |
| `logos.dev` | 2 | off | auto (8 shards) | Logos Dev Network |
| `logos.test` | 2 | off | auto (8 shards) | Logos Test Network |
| `status.prod` | 16 | off | auto (1 shard) | Status Production Network |
#### `logosdelivery_start_node`
Starts the node.

85
library/channels_api/channel_api.nim Normal file
View File

@ -0,0 +1,85 @@
import std/json
import chronos, results, ffi
import
logos_delivery/waku/common/base64,
logos_delivery,
logos_delivery/waku/waku_core/topics/content_topic,
logos_delivery/api/types,
../declare_lib
proc logosdelivery_channel_create(
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
channelIdStr: cstring,
contentTopicStr: cstring,
senderIdStr: cstring,
) {.ffi.} =
requireInitializedNode(ctx, "ChannelCreate"):
return err(errMsg)
requireChannels(ctx, "ChannelCreate"):
return err(errMsg)
let id = ctx.myLib[].reliableChannelManager.createReliableChannel(
ChannelId($channelIdStr),
ContentTopic($contentTopicStr),
SdsParticipantID($senderIdStr),
).valueOr:
return err("ChannelCreate failed: " & $error)
return ok(string(id))
proc logosdelivery_channel_send(
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
channelIdStr: cstring,
messageJson: cstring,
) {.ffi.} =
## `messageJson` carries `{ "payload": <base64>, "ephemeral": <bool> }`.
requireInitializedNode(ctx, "ChannelSend"):
return err(errMsg)
requireChannels(ctx, "ChannelSend"):
return err(errMsg)
var jsonNode: JsonNode
try:
jsonNode = parseJson($messageJson)
except Exception as e:
return err("Failed to parse channel message JSON: " & e.msg)
if not jsonNode.hasKey("payload"):
return err("Missing payload field")
let payload = base64.decode(Base64String(jsonNode["payload"].getStr())).valueOr:
return err("invalid payload format: " & error)
let ephemeral = jsonNode.getOrDefault("ephemeral").getBool(false)
let requestId = (
await ctx.myLib[].reliableChannelManager.send(
ChannelId($channelIdStr), payload, ephemeral
)
).valueOr:
return err("ChannelSend failed: " & $error)
return ok($requestId)
proc logosdelivery_channel_close(
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
channelIdStr: cstring,
) {.ffi.} =
requireInitializedNode(ctx, "ChannelClose"):
return err(errMsg)
requireChannels(ctx, "ChannelClose"):
return err(errMsg)
(await ctx.myLib[].reliableChannelManager.closeChannel(ChannelId($channelIdStr))).isOkOr:
return err("ChannelClose failed: " & $error)
return ok("")

25
library/declare_lib.nim
View File

@ -1,6 +1,7 @@
import ffi
import std/locks
import logos_delivery/waku/factory/waku
import results
import logos_delivery
declareLibrary("logosdelivery")
@ -8,7 +9,7 @@ var eventCallbackLock: Lock
initLock(eventCallbackLock)
template requireInitializedNode*(
ctx: ptr FFIContext[Waku], opName: string, onError: untyped
ctx: ptr FFIContext[LogosDelivery], opName: string, onError: untyped
) =
if isNil(ctx):
let errMsg {.inject.} = opName & " failed: invalid context"
@ -17,8 +18,26 @@ template requireInitializedNode*(
let errMsg {.inject.} = opName & " failed: node is not initialized"
onError
template requireMessaging*(
ctx: ptr FFIContext[LogosDelivery], opName: string, onError: untyped
) =
## Use after `requireInitializedNode`. Fails if the node has no messaging client
## (a kernel-only / fleet node).
ctx.myLib[].ensureMessaging().isOkOr:
let errMsg {.inject.} = opName & " failed: " & error
onError
template requireChannels*(
ctx: ptr FFIContext[LogosDelivery], opName: string, onError: untyped
) =
## Use after `requireInitializedNode`. Fails if the node has no reliable channel
## manager (a kernel-only / fleet node).
ctx.myLib[].ensureChannels().isOkOr:
let errMsg {.inject.} = opName & " failed: " & error
onError
proc logosdelivery_set_event_callback(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.dynlib, exportc, cdecl.} =
if isNil(ctx):
echo "error: invalid context in logosdelivery_set_event_callback"

2
library/events/json_connection_status_change_event.nim
View File

@ -2,7 +2,7 @@
import system, std/json
import ./json_base_event
import ../../logos_delivery/waku/api/types
import ../../logos_delivery/api/types
type JsonConnectionStatusChangeEvent* = ref object of JsonEvent
status*: ConnectionStatus

10
library/examples/logosdelivery_example.c
View File

@ -129,12 +129,14 @@ void simple_callback(int ret, const char *msg, size_t len, void *userData) {
int main() {
printf("=== Logos Messaging API (LMAPI) Example ===\n\n");
// Configuration JSON using WakuNodeConf field names (flat structure).
// Field names match Nim identifiers from WakuNodeConf in tools/confutils/cli_args.nim.
// Layered messaging config: {mode, preset, messagingOverrides, channelsOverrides}.
// Override keys are MessagingClientConf field or CLI switch names.
const char *config = "{"
"\"logLevel\": \"INFO\","
"\"mode\": \"Core\","
"\"preset\": \"logos.dev\""
"\"preset\": \"logos.dev\","
"\"messagingOverrides\": {"
"\"log-level\": \"INFO\""
"}"
"}";
printf("1. Creating node...\n");

61
library/kernel_api/debug_node_api.nim
View File

@ -1,53 +1,46 @@
import std/json
import
chronicles,
chronos,
results,
eth/p2p/discoveryv5/enr,
strutils,
libp2p/peerid,
metrics,
ffi
import
logos_delivery/waku/factory/waku,
logos_delivery/waku/node/waku_node,
logos_delivery/waku/node/health_monitor,
library/declare_lib
proc getMultiaddresses(node: WakuNode): seq[string] =
return node.info().listenAddresses
proc getMetrics(): string =
{.gcsafe.}:
return defaultRegistry.toText() ## defaultRegistry is {.global.} in metrics module
import std/strutils
import chronos, results, ffi
import logos_delivery, library/declare_lib
proc waku_version(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
return ok(WakuNodeVersionString)
let v = (await ctx.myLib[].waku.version()).valueOr:
return err(error)
return ok(v)
proc waku_listen_addresses(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
## returns a comma-separated string of the listen addresses
return ok(ctx.myLib[].node.getMultiaddresses().join(","))
let addrs = (await ctx.myLib[].waku.listenAddresses()).valueOr:
return err(error)
return ok(addrs.join(","))
proc waku_get_my_enr(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
return ok(ctx.myLib[].node.enr.toURI())
let enrUri = (await ctx.myLib[].waku.myEnr()).valueOr:
return err(error)
return ok(enrUri)
proc waku_get_my_peerid(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
return ok($ctx.myLib[].node.peerId())
let peerId = (await ctx.myLib[].waku.myPeerId()).valueOr:
return err(error)
return ok(peerId)
proc waku_get_metrics(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
return ok(getMetrics())
let m = (await ctx.myLib[].waku.metrics()).valueOr:
return err(error)
return ok(m)
proc waku_is_online(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
return ok($ctx.myLib[].healthMonitor.onlineMonitor.amIOnline())
let online = (await ctx.myLib[].waku.isOnline()).valueOr:
return err(error)
return ok($online)

79
library/kernel_api/discovery_api.nim
View File

@ -1,96 +1,59 @@
import logos_delivery/waku/compat/option_valueor
import std/json
import chronos, chronicles, results, strutils, libp2p/multiaddress, ffi
import
logos_delivery/waku/factory/waku,
logos_delivery/waku/discovery/waku_dnsdisc,
logos_delivery/waku/discovery/waku_discv5,
logos_delivery/waku/waku_core/peers,
logos_delivery/waku/waku_node,
library/declare_lib
proc retrieveBootstrapNodes(
enrTreeUrl: string, ipDnsServer: string
): Future[Result[seq[string], string]] {.async.} =
let dnsNameServers = @[parseIpAddress(ipDnsServer)]
let discoveredPeers: seq[RemotePeerInfo] = (
await retrieveDynamicBootstrapNodes(enrTreeUrl, dnsNameServers)
).valueOr:
return err("failed discovering peers from DNS: " & $error)
var multiAddresses = newSeq[string]()
for discPeer in discoveredPeers:
for address in discPeer.addrs:
multiAddresses.add($address & "/p2p/" & $discPeer)
return ok(multiAddresses)
proc updateDiscv5BootstrapNodes(nodes: string, waku: Waku): Result[void, string] =
waku.wakuDiscv5.updateBootstrapRecords(nodes).isOkOr:
return err("error in updateDiscv5BootstrapNodes: " & $error)
return ok()
proc performPeerExchangeRequestTo*(
numPeers: uint64, waku: Waku
): Future[Result[int, string]] {.async.} =
let numPeersRecv = (await waku.node.fetchPeerExchangePeers(numPeers)).valueOr:
return err($error)
return ok(numPeersRecv)
import std/strutils
import chronos, chronicles, results, ffi
import logos_delivery, library/declare_lib
proc waku_discv5_update_bootnodes(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
bootnodes: cstring,
) {.ffi.} =
## Updates the bootnode list used for discovering new peers via DiscoveryV5
## bootnodes - JSON array containing the bootnode ENRs i.e. `["enr:...", "enr:..."]`
updateDiscv5BootstrapNodes($bootnodes, ctx.myLib[]).isOkOr:
(await ctx.myLib[].waku.discv5UpdateBootnodes($bootnodes)).isOkOr:
error "UPDATE_DISCV5_BOOTSTRAP_NODES failed", error = error
return err($error)
return err(error)
return ok("discovery request processed correctly")
proc waku_dns_discovery(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
enrTreeUrl: cstring,
nameDnsServer: cstring,
timeoutMs: cint,
) {.ffi.} =
let nodes = (await retrieveBootstrapNodes($enrTreeUrl, $nameDnsServer)).valueOr:
let nodes = (
await ctx.myLib[].waku.dnsDiscovery($enrTreeUrl, $nameDnsServer, int(timeoutMs))
).valueOr:
error "GET_BOOTSTRAP_NODES failed", error = error
return err($error)
return err(error)
## returns a comma-separated string of bootstrap nodes' multiaddresses
return ok(nodes.join(","))
proc waku_start_discv5(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
(await ctx.myLib[].wakuDiscv5.start()).isOkOr:
(await ctx.myLib[].waku.startDiscv5()).isOkOr:
error "START_DISCV5 failed", error = error
return err("error starting discv5: " & $error)
return err(error)
return ok("discv5 started correctly")
proc waku_stop_discv5(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
await ctx.myLib[].wakuDiscv5.stop()
(await ctx.myLib[].waku.stopDiscv5()).isOkOr:
error "STOP_DISCV5 failed", error = error
return err(error)
return ok("discv5 stopped correctly")
proc waku_peer_exchange_request(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
numPeers: uint64,
) {.ffi.} =
let numValidPeers = (await performPeerExchangeRequestTo(numPeers, ctx.myLib[])).valueOr:
let numValidPeers = (await ctx.myLib[].waku.peerExchangeRequest(numPeers)).valueOr:
error "waku_peer_exchange_request failed", error = error
return err("failed peer exchange: " & $error)
return err(error)
return ok($numValidPeers)

86
library/kernel_api/node_lifecycle_api.nim
View File

@ -1,86 +0,0 @@
import logos_delivery/waku/compat/option_valueor
import std/[options, json, strutils, net]
import chronos, chronicles, results, confutils, confutils/std/net, ffi
import
logos_delivery/waku/node/peer_manager/peer_manager,
tools/confutils/cli_args,
logos_delivery/waku/factory/waku,
logos_delivery/waku/factory/node_factory,
logos_delivery/waku/factory/app_callbacks,
logos_delivery/waku/rest_api/endpoint/builder,
library/declare_lib
proc createWaku(
configJson: cstring, appCallbacks: AppCallbacks = nil
): Future[Result[Waku, string]] {.async.} =
var conf = defaultWakuNodeConf().valueOr:
return err("Failed creating node: " & error)
var errorResp: string
var jsonNode: JsonNode
try:
jsonNode = parseJson($configJson)
except Exception:
return err(
"exception in createWaku when calling parseJson: " & getCurrentExceptionMsg() &
" configJson string: " & $configJson
)
for confField, confValue in fieldPairs(conf):
if jsonNode.contains(confField):
# Make sure string doesn't contain the leading or trailing " character
let formattedString = ($jsonNode[confField]).strip(chars = {'\"'})
# Override conf field with the value set in the json-string
try:
confValue = parseCmdArg(typeof(confValue), formattedString)
except Exception:
return err(
"exception in createWaku when parsing configuration. exc: " &
getCurrentExceptionMsg() & ". string that could not be parsed: " &
formattedString & ". expected type: " & $typeof(confValue)
)
# Don't send relay app callbacks if relay is disabled
if not conf.relay and not appCallbacks.isNil():
appCallbacks.relayHandler = nil
appCallbacks.topicHealthChangeHandler = nil
# TODO: Convert `confJson` directly to `WakuConf`
var wakuConf = conf.toWakuConf().valueOr:
return err("Configuration error: " & $error)
wakuConf.restServerConf = none(RestServerConf) ## don't want REST in libwaku
let wakuRes = (await Waku.new(wakuConf, appCallbacks)).valueOr:
error "waku initialization failed", error = error
return err("Failed setting up Waku: " & $error)
return ok(wakuRes)
registerReqFFI(CreateNodeWithCallbacksRequest, ctx: ptr FFIContext[Waku]):
proc(
configJson: cstring, appCallbacks: AppCallbacks
): Future[Result[string, string]] {.async.} =
ctx.myLib[] = (await createWaku(configJson, cast[AppCallbacks](appCallbacks))).valueOr:
error "CreateNodeWithCallbacksRequest failed", error = error
return err($error)
return ok("")
proc waku_start(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
) {.ffi.} =
(await ctx.myLib[].start()).isOkOr:
error "START_NODE failed", error = error
return err("failed to start: " & $error)
return ok("")
proc waku_stop(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
) {.ffi.} =
(await ctx.myLib[].stop()).isOkOr:
error "STOP_NODE failed", error = error
return err("failed to stop: " & $error)
return ok("")

110
library/kernel_api/peer_manager_api.nim
View File

@ -1,128 +1,104 @@
import logos_delivery/waku/compat/option_valueor
import std/[sequtils, strutils, tables]
import chronicles, chronos, results, options, json, ffi
import
logos_delivery/waku/factory/waku,
logos_delivery/waku/node/waku_node,
logos_delivery/waku/node/peer_manager,
library/declare_lib
import std/[strutils, tables, json]
import chronicles, chronos, results, ffi
import logos_delivery, library/declare_lib
type PeerInfo = object
protocols: seq[string]
addresses: seq[string]
proc waku_get_peerids_from_peerstore(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
## returns a comma-separated string of peerIDs
let peerIDs =
ctx.myLib[].node.peerManager.switch.peerStore.peers().mapIt($it.peerId).join(",")
return ok(peerIDs)
let peerIds = (await ctx.myLib[].waku.peerIdsFromPeerstore()).valueOr:
return err(error)
return ok(peerIds.join(","))
proc waku_connect(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
peerMultiAddr: cstring,
timeoutMs: cuint,
) {.ffi.} =
let peers = ($peerMultiAddr).split(",").mapIt(strip(it))
await ctx.myLib[].node.connectToNodes(peers, source = "static")
let peers = ($peerMultiAddr).split(",")
(await ctx.myLib[].waku.connect(peers, uint32(timeoutMs))).isOkOr:
return err(error)
return ok("")
proc waku_disconnect_peer_by_id(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer, peerId: cstring
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
peerId: cstring,
) {.ffi.} =
let pId = PeerId.init($peerId).valueOr:
error "DISCONNECT_PEER_BY_ID failed", error = $error
return err($error)
await ctx.myLib[].node.peerManager.disconnectNode(pId)
(await ctx.myLib[].waku.disconnectPeerById($peerId)).isOkOr:
error "DISCONNECT_PEER_BY_ID failed", error = error
return err(error)
return ok("")
proc waku_disconnect_all_peers(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
await ctx.myLib[].node.peerManager.disconnectAllPeers()
(await ctx.myLib[].waku.disconnectAllPeers()).isOkOr:
return err(error)
return ok("")
proc waku_dial_peer(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
peerMultiAddr: cstring,
protocol: cstring,
timeoutMs: cuint,
) {.ffi.} =
let remotePeerInfo = parsePeerInfo($peerMultiAddr).valueOr:
error "DIAL_PEER failed", error = $error
return err($error)
let conn = await ctx.myLib[].node.peerManager.dialPeer(remotePeerInfo, $protocol)
if conn.isNone():
let msg = "failed dialing peer"
error "DIAL_PEER failed", error = msg, peerId = $remotePeerInfo.peerId
return err(msg)
(await ctx.myLib[].waku.dialPeer($peerMultiAddr, $protocol, int(timeoutMs))).isOkOr:
error "DIAL_PEER failed", error = error
return err(error)
return ok("")
proc waku_dial_peer_by_id(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
peerId: cstring,
protocol: cstring,
timeoutMs: cuint,
) {.ffi.} =
let pId = PeerId.init($peerId).valueOr:
error "DIAL_PEER_BY_ID failed", error = $error
return err($error)
let conn = await ctx.myLib[].node.peerManager.dialPeer(pId, $protocol)
if conn.isNone():
let msg = "failed dialing peer"
error "DIAL_PEER_BY_ID failed", error = msg, peerId = $peerId
return err(msg)
(await ctx.myLib[].waku.dialPeerById($peerId, $protocol, int(timeoutMs))).isOkOr:
error "DIAL_PEER_BY_ID failed", error = error
return err(error)
return ok("")
proc waku_get_connected_peers_info(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
## returns a JSON string mapping peerIDs to objects with protocols and addresses
let peers = (await ctx.myLib[].waku.connectedPeersInfo()).valueOr:
return err(error)
var peersMap = initTable[string, PeerInfo]()
let peers = ctx.myLib[].node.peerManager.switch.peerStore.peers().filterIt(
it.connectedness == Connected
)
# Build a map of peer IDs to peer info objects
for peer in peers:
let peerIdStr = $peer.peerId
peersMap[peerIdStr] =
PeerInfo(protocols: peer.protocols, addresses: peer.addrs.mapIt($it))
peersMap[peer.peerId] =
PeerInfo(protocols: peer.protocols, addresses: peer.addresses)
# Convert the map to JSON string
let jsonObj = %*peersMap
let jsonStr = $jsonObj
return ok(jsonStr)
return ok($(%*peersMap))
proc waku_get_connected_peers(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
## returns a comma-separated string of peerIDs
let
(inPeerIds, outPeerIds) = ctx.myLib[].node.peerManager.connectedPeers()
connectedPeerids = concat(inPeerIds, outPeerIds)
return ok(connectedPeerids.mapIt($it).join(","))
let peerIds = (await ctx.myLib[].waku.connectedPeers()).valueOr:
return err(error)
return ok(peerIds.join(","))
proc waku_get_peerids_by_protocol(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
protocol: cstring,
) {.ffi.} =
## returns a comma-separated string of peerIDs that mount the given protocol
let connectedPeers = ctx.myLib[].node.peerManager.switch.peerStore
.peers($protocol)
.filterIt(it.connectedness == Connected)
.mapIt($it.peerId)
.join(",")
return ok(connectedPeers)
let peerIds = (await ctx.myLib[].waku.peerIdsByProtocol($protocol)).valueOr:
return err(error)
return ok(peerIds.join(","))

42
library/kernel_api/ping_api.nim
View File

@ -1,45 +1,13 @@
import std/[json, strutils]
import chronos, results, ffi
import libp2p/[protocols/ping, switch, multiaddress, multicodec]
import
logos_delivery/waku/[factory/waku, waku_core/peers, node/waku_node],
library/declare_lib
import logos_delivery, library/declare_lib
proc waku_ping_peer(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
peerAddr: cstring,
timeoutMs: cuint,
) {.ffi.} =
let peerInfo = peers.parsePeerInfo(($peerAddr).split(",")).valueOr:
return err("PingRequest failed to parse peer addr: " & $error)
let timeout = chronos.milliseconds(timeoutMs)
proc ping(): Future[Result[Duration, string]] {.async, gcsafe.} =
try:
let conn =
await ctx.myLib[].node.switch.dial(peerInfo.peerId, peerInfo.addrs, PingCodec)
defer:
await conn.close()
let pingRTT = await ctx.myLib[].node.libp2pPing.ping(conn)
if pingRTT == 0.nanos:
return err("could not ping peer: rtt-0")
return ok(pingRTT)
except CatchableError as exc:
return err("could not ping peer: " & exc.msg)
let pingFuture = ping()
let pingRTT: Duration =
if timeout == chronos.milliseconds(0): # No timeout expected
(await pingFuture).valueOr:
return err("ping failed, no timeout expected: " & error)
else:
let timedOut = not (await pingFuture.withTimeout(timeout))
if timedOut:
return err("ping timed out")
pingFuture.read().valueOr:
return err("failed to read ping future: " & error)
return ok($(pingRTT.nanos))
let rttNanos = (await ctx.myLib[].waku.pingPeer($peerAddr, int(timeoutMs))).valueOr:
return err(error)
return ok($rttNanos)

102
library/kernel_api/protocols/filter_api.nim
View File

@ -1,109 +1,57 @@
import logos_delivery/waku/compat/option_valueor
import options, std/[strutils, sequtils]
import std/[strutils, sequtils]
import chronicles, chronos, results, ffi
import
logos_delivery/waku/waku_filter_v2/client,
logos_delivery,
logos_delivery/waku/waku_core/message/message,
logos_delivery/waku/factory/waku,
logos_delivery/waku/waku_relay,
logos_delivery/waku/waku_filter_v2/common,
logos_delivery/waku/waku_core/subscription/push_handler,
logos_delivery/waku/node/peer_manager/peer_manager,
logos_delivery/waku/waku_node,
logos_delivery/waku/waku_core/topics/pubsub_topic,
logos_delivery/waku/waku_core/topics/content_topic,
library/events/json_message_event,
library/declare_lib
const FilterOpTimeout = 5.seconds
proc checkFilterClientMounted(waku: Waku): Result[string, string] =
if waku.node.wakuFilterClient.isNil():
let errorMsg = "wakuFilterClient is not mounted"
error "fail filter process", error = errorMsg
return err(errorMsg)
return ok("")
proc waku_filter_subscribe(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
pubSubTopic: cstring,
contentTopics: cstring,
) {.ffi.} =
proc onReceivedMessage(ctx: ptr FFIContext): WakuRelayHandler =
proc onReceivedMessage(ctx: ptr FFIContext[LogosDelivery]): FilterPushHandler =
return proc(pubsubTopic: PubsubTopic, msg: WakuMessage) {.async.} =
callEventCallback(ctx, "onReceivedMessage"):
$JsonMessageEvent.new(pubsubTopic, msg)
checkFilterClientMounted(ctx.myLib[]).isOkOr:
return err($error)
var filterPushEventCallback = FilterPushHandler(onReceivedMessage(ctx))
ctx.myLib[].node.wakuFilterClient.registerPushHandler(filterPushEventCallback)
let peer = ctx.myLib[].node.peerManager.selectPeer(WakuFilterSubscribeCodec).valueOr:
let errorMsg = "could not find peer with WakuFilterSubscribeCodec when subscribing"
error "fail filter subscribe", error = errorMsg
return err(errorMsg)
let subFut = ctx.myLib[].node.filterSubscribe(
some(PubsubTopic($pubsubTopic)),
($contentTopics).split(",").mapIt(ContentTopic(it)),
peer,
)
if not await subFut.withTimeout(FilterOpTimeout):
let errorMsg = "filter subscription timed out"
error "fail filter unsubscribe", error = errorMsg
return err(errorMsg)
(
await ctx.myLib[].waku.filterSubscribe(
PubsubTopic($pubSubTopic),
($contentTopics).split(",").mapIt(ContentTopic(it)),
FilterPushHandler(onReceivedMessage(ctx)),
)
).isOkOr:
error "fail filter subscribe", error = error
return err(error)
return ok("")
proc waku_filter_unsubscribe(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
pubSubTopic: cstring,
contentTopics: cstring,
) {.ffi.} =
checkFilterClientMounted(ctx.myLib[]).isOkOr:
return err($error)
let peer = ctx.myLib[].node.peerManager.selectPeer(WakuFilterSubscribeCodec).valueOr:
let errorMsg =
"could not find peer with WakuFilterSubscribeCodec when unsubscribing"
error "fail filter process", error = errorMsg
return err(errorMsg)
let subFut = ctx.myLib[].node.filterUnsubscribe(
some(PubsubTopic($pubsubTopic)),
($contentTopics).split(",").mapIt(ContentTopic(it)),
peer,
)
if not await subFut.withTimeout(FilterOpTimeout):
let errorMsg = "filter un-subscription timed out"
error "fail filter unsubscribe", error = errorMsg
return err(errorMsg)
(
await ctx.myLib[].waku.filterUnsubscribe(
PubsubTopic($pubSubTopic), ($contentTopics).split(",").mapIt(ContentTopic(it))
)
).isOkOr:
error "fail filter unsubscribe", error = error
return err(error)
return ok("")
proc waku_filter_unsubscribe_all(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
checkFilterClientMounted(ctx.myLib[]).isOkOr:
return err($error)
let peer = ctx.myLib[].node.peerManager.selectPeer(WakuFilterSubscribeCodec).valueOr:
let errorMsg =
"could not find peer with WakuFilterSubscribeCodec when unsubscribing all"
error "fail filter unsubscribe all", error = errorMsg
return err(errorMsg)
let unsubFut = ctx.myLib[].node.filterUnsubscribeAll(peer)
if not await unsubFut.withTimeout(FilterOpTimeout):
let errorMsg = "filter un-subscription all timed out"
error "fail filter unsubscribe all", error = errorMsg
return err(errorMsg)
(await ctx.myLib[].waku.filterUnsubscribeAll()).isOkOr:
error "fail filter unsubscribe all", error = error
return err(error)
return ok("")

28
library/kernel_api/protocols/lightpush_api.nim
View File

@ -1,29 +1,19 @@
import logos_delivery/waku/compat/option_valueor
import options, std/[json, strformat]
import std/[json, strformat]
import chronicles, chronos, results, ffi
import
logos_delivery/waku/waku_core/message/message,
logos_delivery/waku/waku_core/codecs,
logos_delivery/waku/factory/waku,
logos_delivery,
logos_delivery/waku/waku_core/message,
logos_delivery/waku/waku_core/topics/pubsub_topic,
logos_delivery/waku/waku_lightpush_legacy/client,
logos_delivery/waku/node/peer_manager/peer_manager,
library/events/json_message_event,
library/declare_lib
proc waku_lightpush_publish(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
pubSubTopic: cstring,
jsonWakuMessage: cstring,
) {.ffi.} =
if ctx.myLib[].node.wakuLightpushClient.isNil():
let errorMsg = "LightpushRequest waku.node.wakuLightpushClient is nil"
error "PUBLISH failed", error = errorMsg
return err(errorMsg)
var jsonMessage: JsonMessage
try:
let jsonContent = parseJson($jsonWakuMessage)
@ -35,18 +25,10 @@ proc waku_lightpush_publish(
let msg = json_message_event.toWakuMessage(jsonMessage).valueOr:
return err("Problem building the WakuMessage: " & $error)
let peerOpt = ctx.myLib[].node.peerManager.selectPeer(WakuLightPushCodec)
if peerOpt.isNone():
let errorMsg = "failed to lightpublish message, no suitable remote peers"
error "PUBLISH failed", error = errorMsg
return err(errorMsg)
let msgHashHex = (
await ctx.myLib[].node.wakuLegacyLightpushClient.publish(
$pubsubTopic, msg, peer = peerOpt.get()
)
await ctx.myLib[].waku.lightpushPublish(PubsubTopic($pubSubTopic), msg)
).valueOr:
error "PUBLISH failed", error = error
return err($error)
return err(error)
return ok(msgHashHex)

129
library/kernel_api/protocols/relay_api.nim
View File

@ -1,68 +1,61 @@
import logos_delivery/waku/compat/option_valueor
import std/[net, sequtils, strutils, json], strformat
import chronicles, chronos, stew/byteutils, results, ffi
import std/[strutils, json]
import chronicles, chronos, results, ffi
import
logos_delivery/waku/waku_core/message/message,
logos_delivery/waku/factory/[validator_signed, waku],
tools/confutils/cli_args,
logos_delivery/waku/waku_core/message,
logos_delivery,
logos_delivery/waku/waku_core/topics/pubsub_topic,
logos_delivery/waku/waku_core/topics,
logos_delivery/waku/node/waku_node/relay,
logos_delivery/waku/waku_core/message,
logos_delivery/waku/waku_relay/protocol,
logos_delivery/waku/node/peer_manager,
library/events/json_message_event,
library/declare_lib
proc waku_relay_get_peers_in_mesh(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
pubSubTopic: cstring,
) {.ffi.} =
let meshPeers = ctx.myLib[].node.wakuRelay.getPeersInMesh($pubsubTopic).valueOr:
let peers = (await ctx.myLib[].waku.relayPeersInMesh(PubsubTopic($pubSubTopic))).valueOr:
error "LIST_MESH_PEERS failed", error = error
return err($error)
return err(error)
## returns a comma-separated string of peerIDs
return ok(meshPeers.mapIt($it).join(","))
return ok(peers.join(","))
proc waku_relay_get_num_peers_in_mesh(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
pubSubTopic: cstring,
) {.ffi.} =
let numPeersInMesh = ctx.myLib[].node.wakuRelay.getNumPeersInMesh($pubsubTopic).valueOr:
let n = (await ctx.myLib[].waku.relayNumPeersInMesh(PubsubTopic($pubSubTopic))).valueOr:
error "NUM_MESH_PEERS failed", error = error
return err($error)
return ok($numPeersInMesh)
return err(error)
return ok($n)
proc waku_relay_get_connected_peers(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
pubSubTopic: cstring,
) {.ffi.} =
## Returns the list of all connected peers to an specific pubsub topic
let connPeers = ctx.myLib[].node.wakuRelay.getConnectedPeers($pubsubTopic).valueOr:
let peers = (await ctx.myLib[].waku.relayConnectedPeers(PubsubTopic($pubSubTopic))).valueOr:
error "LIST_CONNECTED_PEERS failed", error = error
return err($error)
## returns a comma-separated string of peerIDs
return ok(connPeers.mapIt($it).join(","))
return err(error)
return ok(peers.join(","))
proc waku_relay_get_num_connected_peers(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
pubSubTopic: cstring,
) {.ffi.} =
let numConnPeers = ctx.myLib[].node.wakuRelay.getNumConnectedPeers($pubsubTopic).valueOr:
let n = (await ctx.myLib[].waku.relayNumConnectedPeers(PubsubTopic($pubSubTopic))).valueOr:
error "NUM_CONNECTED_PEERS failed", error = error
return err($error)
return ok($numConnPeers)
return err(error)
return ok($n)
proc waku_relay_add_protected_shard(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
clusterId: cint,
@ -70,87 +63,82 @@ proc waku_relay_add_protected_shard(
publicKey: cstring,
) {.ffi.} =
## Protects a shard with a public key
try:
let relayShard = RelayShard(clusterId: uint16(clusterId), shardId: uint16(shardId))
let protectedShard = ProtectedShard.parseCmdArg($relayShard & ":" & $publicKey)
ctx.myLib[].node.wakuRelay.addSignedShardsValidator(
@[protectedShard], uint16(clusterId)
(
await ctx.myLib[].waku.relayAddProtectedShard(
uint16(clusterId), uint16(shardId), $publicKey
)
except ValueError as exc:
return err("ERROR in waku_relay_add_protected_shard: " & exc.msg)
).isOkOr:
return err(error)
return ok("")
proc waku_relay_subscribe(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
pubSubTopic: cstring,
) {.ffi.} =
echo "Subscribing to topic: " & $pubSubTopic & " ..."
proc onReceivedMessage(ctx: ptr FFIContext[Waku]): WakuRelayHandler =
proc onReceivedMessage(ctx: ptr FFIContext[LogosDelivery]): WakuRelayHandler =
return proc(pubsubTopic: PubsubTopic, msg: WakuMessage) {.async.} =
callEventCallback(ctx, "onReceivedMessage"):
$JsonMessageEvent.new(pubsubTopic, msg)
var cb = onReceivedMessage(ctx)
ctx.myLib[].node.subscribe(
(kind: SubscriptionKind.PubsubSub, topic: $pubsubTopic),
handler = WakuRelayHandler(cb),
(
await ctx.myLib[].waku.relaySubscribe(
PubsubTopic($pubSubTopic), WakuRelayHandler(onReceivedMessage(ctx))
)
).isOkOr:
error "SUBSCRIBE failed", error = error
return err($error)
return err(error)
return ok("")
proc waku_relay_unsubscribe(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
pubSubTopic: cstring,
) {.ffi.} =
ctx.myLib[].node.unsubscribe((kind: SubscriptionKind.PubsubSub, topic: $pubsubTopic)).isOkOr:
(await ctx.myLib[].waku.relayUnsubscribe(PubsubTopic($pubSubTopic))).isOkOr:
error "UNSUBSCRIBE failed", error = error
return err($error)
return err(error)
return ok("")
proc waku_relay_publish(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
pubSubTopic: cstring,
jsonWakuMessage: cstring,
timeoutMs: cuint,
) {.ffi.} =
var
# https://rfc.vac.dev/spec/36/#extern-char-waku_relay_publishchar-messagejson-char-pubsubtopic-int-timeoutms
jsonMessage: JsonMessage
var jsonMessage: JsonMessage
try:
let jsonContent = parseJson($jsonWakuMessage)
jsonMessage = JsonMessage.fromJsonNode(jsonContent).valueOr:
raise newException(JsonParsingError, $error)
except JsonParsingError as exc:
return err(fmt"Error parsing json message: {exc.msg}")
return err("Error parsing json message: " & exc.msg)
let msg = json_message_event.toWakuMessage(jsonMessage).valueOr:
return err("Problem building the WakuMessage: " & $error)
(await ctx.myLib[].node.wakuRelay.publish($pubsubTopic, msg)).isOkOr:
let msgHash = (
await ctx.myLib[].waku.relayPublish(
PubsubTopic($pubSubTopic), msg, uint32(timeoutMs)
)
).valueOr:
error "PUBLISH failed", error = error
return err($error)
let msgHash = computeMessageHash($pubSubTopic, msg).to0xHex
return err(error)
return ok(msgHash)
proc waku_default_pubsub_topic(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
# https://rfc.vac.dev/spec/36/#extern-char-waku_default_pubsub_topic
return ok(DefaultPubsubTopic)
let topic = (await ctx.myLib[].waku.defaultPubsubTopic()).valueOr:
return err(error)
return ok(string(topic))
proc waku_content_topic(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
appName: cstring,
@ -158,15 +146,20 @@ proc waku_content_topic(
contentTopicName: cstring,
encoding: cstring,
) {.ffi.} =
# https://rfc.vac.dev/spec/36/#extern-char-waku_content_topicchar-applicationname-unsigned-int-applicationversion-char-contenttopicname-char-encoding
return ok(fmt"/{$appName}/{$appVersion}/{$contentTopicName}/{$encoding}")
let topic = (
await ctx.myLib[].waku.buildContentTopic(
$appName, uint32(appVersion), $contentTopicName, $encoding
)
).valueOr:
return err(error)
return ok(string(topic))
proc waku_pubsub_topic(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
topicName: cstring,
) {.ffi.} =
# https://rfc.vac.dev/spec/36/#extern-char-waku_pubsub_topicchar-name-char-encoding
return ok(fmt"/waku/2/{$topicName}")
let topic = (await ctx.myLib[].waku.buildPubsubTopic($topicName)).valueOr:
return err(error)
return ok(string(topic))

18
library/kernel_api/protocols/store_api.nim
View File

@ -1,13 +1,10 @@
import logos_delivery/waku/compat/option_valueor
import std/[json, sugar, strutils, options]
import chronos, chronicles, results, stew/byteutils, ffi
import std/[json, sugar, options]
import chronos, chronicles, results, ffi
import
logos_delivery/waku/factory/waku,
logos_delivery,
library/utils,
logos_delivery/waku/waku_core/peers,
logos_delivery/waku/waku_core/message/digest,
logos_delivery/waku/waku_store/common,
logos_delivery/waku/waku_store/client,
logos_delivery/waku/common/paging,
library/declare_lib
@ -68,7 +65,7 @@ func fromJsonNode(jsonContent: JsonNode): Result[StoreQueryRequest, string] =
)
proc waku_store_query(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
jsonQuery: cstring,
@ -83,13 +80,10 @@ proc waku_store_query(
let storeQueryRequest = ?fromJsonNode(jsonContentRes.get())
let peer = peers.parsePeerInfo(($peerAddr).split(",")).valueOr:
return err("StoreRequest failed to parse peer addr: " & $error)
let queryResponse = (
await ctx.myLib[].node.wakuStoreClient.query(storeQueryRequest, peer)
await ctx.myLib[].waku.storeQuery(storeQueryRequest, $peerAddr, int(timeoutMs))
).valueOr:
return err("StoreRequest failed store query: " & $error)
return err("StoreRequest failed store query: " & error)
let res = $(%*(queryResponse.toHex()))
return ok(res) ## returning the response in json format

41
library/liblogosdelivery.h
View File

@ -22,9 +22,14 @@ extern "C"
// Creates a new instance of the node from the given configuration JSON.
// Returns a pointer to the Context needed by the rest of the API functions.
// Configuration should be in JSON format using WakuNodeConf field names.
// Field names match Nim identifiers from WakuNodeConf (camelCase).
// Example: {"mode": "Core", "clusterId": 42, "relay": true}
// The configuration is a JSON object with these optional keys:
// "mode": "Core" | "Edge" (messaging role; defaults to "Core")
// "preset": "<network preset>" (e.g. "twn")
// "messagingOverrides": { ... } (per-field messaging config overrides)
// "channelsOverrides": { ... } (per-field reliable-channel overrides)
// Override keys accept the config field name or its CLI switch name (e.g.
// "clusterId" or "cluster-id"). Unknown keys are rejected.
// Example: {"mode":"Core","messagingOverrides":{"cluster-id":42,"log-level":"INFO"}}
void *logosdelivery_create_node(
const char *configJson,
FFICallBack callback,
@ -71,6 +76,36 @@ extern "C"
void *userData,
const char *messageJson);
// --- Reliable Channels API (stable surface) ---
// Create a reliable channel. Returns the channel id.
int logosdelivery_channel_create(void *ctx,
FFICallBack callback,
void *userData,
const char *channelId,
const char *contentTopic,
const char *senderId);
// Send a message on a reliable channel.
// messageJson: { "payload": "base64-encoded-payload", "ephemeral": false }
// Returns a request ID that can be used to track delivery.
int logosdelivery_channel_send(void *ctx,
FFICallBack callback,
void *userData,
const char *channelId,
const char *messageJson);
// Close a reliable channel: stops its SDS loops; persisted state survives, so
// re-creating the channel restores it.
int logosdelivery_channel_close(void *ctx,
FFICallBack callback,
void *userData,
const char *channelId);
// Channel lifecycle events are delivered through the event callback set via
// logosdelivery_set_event_callback: "onChannelMessageReceived" (payload
// base64-encoded), "onChannelMessageSent", "onChannelMessageError".
// Sets a callback that will be invoked whenever an event occurs.
// It is crucial that the passed callback is fast, non-blocking and potentially thread-safe.
void logosdelivery_set_event_callback(void *ctx,

93
library/liblogosdelivery.nim
View File

@ -5,14 +5,11 @@ import
logos_delivery/waku/waku_core/message/message,
logos_delivery/waku/waku_core/topics/pubsub_topic,
logos_delivery/waku/waku_relay,
logos_delivery/waku/factory/waku,
logos_delivery,
logos_delivery/waku/waku,
logos_delivery/waku/node/waku_node,
logos_delivery/waku/node/health_monitor/health_status,
../logos_delivery/waku/factory/app_callbacks,
./events/json_message_event,
./events/json_topic_health_change_event,
./events/json_connection_change_event,
./events/json_connection_status_change_event,
./declare_lib
################################################################################
@ -24,89 +21,15 @@ include
./logos_delivery_api/debug_api,
./kernel_api/peer_manager_api,
./kernel_api/discovery_api,
./kernel_api/node_lifecycle_api,
./kernel_api/debug_node_api,
./kernel_api/ping_api,
./kernel_api/protocols/relay_api,
./kernel_api/protocols/store_api,
./kernel_api/protocols/lightpush_api,
./kernel_api/protocols/filter_api
./kernel_api/protocols/filter_api,
./channels_api/channel_api
################################################################################
### Exported procs (former libwaku API)
proc waku_new(
configJson: cstring, callback: FFICallback, userData: pointer
): pointer {.dynlib, exportc, cdecl.} =
initializeLibrary()
## Creates a new instance of the WakuNode.
if isNil(callback):
echo "error: missing callback in waku_new"
return nil
## Create the Waku thread that will keep waiting for req from the main thread.
var ctx = ffi.createFFIContext[Waku]().valueOr:
let msg = "Error in createFFIContext: " & $error
callback(RET_ERR, unsafeAddr msg[0], cast[csize_t](len(msg)), userData)
return nil
ctx.userData = userData
proc onReceivedMessage(ctx: ptr FFIContext): WakuRelayHandler =
return proc(pubsubTopic: PubsubTopic, msg: WakuMessage) {.async.} =
callEventCallback(ctx, "onReceivedMessage"):
$JsonMessageEvent.new(pubsubTopic, msg)
proc onTopicHealthChange(ctx: ptr FFIContext): TopicHealthChangeHandler =
return proc(pubsubTopic: PubsubTopic, topicHealth: TopicHealth) {.async.} =
callEventCallback(ctx, "onTopicHealthChange"):
$JsonTopicHealthChangeEvent.new(pubsubTopic, topicHealth)
proc onConnectionChange(ctx: ptr FFIContext): ConnectionChangeHandler =
return proc(peerId: PeerId, peerEvent: PeerEventKind) {.async.} =
callEventCallback(ctx, "onConnectionChange"):
$JsonConnectionChangeEvent.new($peerId, peerEvent)
proc onConnectionStatusChange(ctx: ptr FFIContext): ConnectionStatusChangeHandler =
return proc(status: ConnectionStatus) {.async.} =
callEventCallback(ctx, "onConnectionStatusChange"):
$JsonConnectionStatusChangeEvent.new(status)
let appCallbacks = AppCallbacks(
relayHandler: onReceivedMessage(ctx),
topicHealthChangeHandler: onTopicHealthChange(ctx),
connectionChangeHandler: onConnectionChange(ctx),
connectionStatusChangeHandler: onConnectionStatusChange(ctx),
)
ffi.sendRequestToFFIThread(
ctx,
CreateNodeWithCallbacksRequest.ffiNewReq(
callback, userData, configJson, appCallbacks
),
).isOkOr:
let msg = "error in sendRequestToFFIThread: " & $error
callback(RET_ERR, unsafeAddr msg[0], cast[csize_t](len(msg)), userData)
return nil
return ctx
proc waku_destroy(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
): cint {.dynlib, exportc, cdecl.} =
initializeLibrary()
checkParams(ctx, callback, userData)
ffi.destroyFFIContext(ctx).isOkOr:
let msg = "libwaku error: " & $error
callback(RET_ERR, unsafeAddr msg[0], cast[csize_t](len(msg)), userData)
return RET_ERR
## always need to invoke the callback although we don't retrieve value to the caller
callback(RET_OK, nil, 0, userData)
return RET_OK
# ### End of exported procs
# ################################################################################
# Node lifecycle (create / start / stop / destroy) is unified under the stable
# logosdelivery_* surface in ./logos_delivery_api/node_api. The former
# waku_new / waku_start / waku_stop / waku_destroy entry points were removed to
# avoid maintaining two parallel node-lifecycle APIs.

25
library/liblogosdelivery_kernel.h
View File

@ -26,26 +26,11 @@ extern "C"
{
#endif
// Creates a new instance of the waku node.
// Sets up the waku node from the given configuration.
// Returns a pointer to the Context needed by the rest of the API functions.
void *waku_new(
const char *configJson,
FFICallBack callback,
void *userData);
int waku_start(void *ctx,
FFICallBack callback,
void *userData);
int waku_stop(void *ctx,
FFICallBack callback,
void *userData);
// Destroys an instance of a waku node created with waku_new
int waku_destroy(void *ctx,
FFICallBack callback,
void *userData);
// NOTE: node lifecycle (create / start / stop / destroy) is unified and lives
// only in the stable header. Use logosdelivery_create_node,
// logosdelivery_start_node, logosdelivery_stop_node and logosdelivery_destroy
// (declared in liblogosdelivery.h, included above) regardless of whether you
// drive the node through the messaging surface or this kernel API.
int waku_version(void *ctx,
FFICallBack callback,

10
library/logos_delivery_api/debug_api.nim
View File

@ -3,17 +3,17 @@ import logos_delivery/waku/factory/waku_state_info
import tools/confutils/[cli_args, config_option_meta]
proc logosdelivery_get_available_node_info_ids(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
## Returns the list of all available node info item ids that
## can be queried with `get_node_info_item`.
requireInitializedNode(ctx, "GetNodeInfoIds"):
return err(errMsg)
return ok($ctx.myLib[].stateInfo.getAllPossibleInfoItemIds())
return ok($ctx.myLib[].waku.stateInfo.getAllPossibleInfoItemIds())
proc logosdelivery_get_node_info(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
nodeInfoId: cstring,
@ -28,10 +28,10 @@ proc logosdelivery_get_node_info(
except ValueError:
return err("Invalid node info id: " & $nodeInfoId)
return ok(ctx.myLib[].stateInfo.getNodeInfoItem(infoItemIdEnum))
return ok(ctx.myLib[].waku.stateInfo.getNodeInfoItem(infoItemIdEnum))
proc logosdelivery_get_available_configs(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
## Returns information about the accepted config items.
requireInitializedNode(ctx, "GetAvailableConfigs"):

27
library/logos_delivery_api/messaging_api.nim
View File

@ -3,13 +3,13 @@ import chronos, results, ffi
import stew/byteutils
import
logos_delivery/waku/common/base64,
logos_delivery/waku/factory/waku,
logos_delivery/waku/waku,
logos_delivery/waku/waku_core/topics/content_topic,
logos_delivery/waku/api/[api, types],
logos_delivery/api/types,
../declare_lib
proc logosdelivery_subscribe(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
contentTopicStr: cstring,
@ -17,17 +17,20 @@ proc logosdelivery_subscribe(
requireInitializedNode(ctx, "Subscribe"):
return err(errMsg)
requireMessaging(ctx, "Subscribe"):
return err(errMsg)
# ContentTopic is just a string type alias
let contentTopic = ContentTopic($contentTopicStr)
(await api.subscribe(ctx.myLib[], contentTopic)).isOkOr:
(await ctx.myLib[].messagingClient.subscribe(contentTopic)).isOkOr:
let errMsg = $error
return err("Subscribe failed: " & errMsg)
return ok("")
proc logosdelivery_unsubscribe(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
contentTopicStr: cstring,
@ -35,17 +38,20 @@ proc logosdelivery_unsubscribe(
requireInitializedNode(ctx, "Unsubscribe"):
return err(errMsg)
requireMessaging(ctx, "Unsubscribe"):
return err(errMsg)
# ContentTopic is just a string type alias
let contentTopic = ContentTopic($contentTopicStr)
api.unsubscribe(ctx.myLib[], contentTopic).isOkOr:
ctx.myLib[].messagingClient.unsubscribe(contentTopic).isOkOr:
let errMsg = $error
return err("Unsubscribe failed: " & errMsg)
return ok("")
proc logosdelivery_send(
ctx: ptr FFIContext[Waku],
ctx: ptr FFIContext[LogosDelivery],
callback: FFICallBack,
userData: pointer,
messageJson: cstring,
@ -53,6 +59,9 @@ proc logosdelivery_send(
requireInitializedNode(ctx, "Send"):
return err(errMsg)
requireMessaging(ctx, "Send"):
return err(errMsg)
## Parse the message JSON and send the message
var jsonNode: JsonNode
try:
@ -83,8 +92,8 @@ proc logosdelivery_send(
contentTopic = contentTopic, payload = payload, ephemeral = ephemeral
)
# Send the message
let requestId = (await api.send(ctx.myLib[], envelope)).valueOr:
# Send the message via the messaging layer's own API.
let requestId = (await ctx.myLib[].messagingClient.send(envelope)).valueOr:
let errMsg = $error
return err("Send failed: " & errMsg)

123
library/logos_delivery_api/node_api.nim
View File

@ -1,11 +1,13 @@
import std/json
import chronos, chronicles, results, ffi
import logos_delivery/waku/common/base64
import
logos_delivery/waku/factory/waku,
logos_delivery,
logos_delivery/waku/node/waku_node,
logos_delivery/waku/api/[api, types],
logos_delivery/waku/events/[message_events, health_events],
tools/confutils/conf_from_json,
logos_delivery/api/types,
logos_delivery/waku/api/events/health_events,
logos_delivery/waku/api/events/peer_events,
logos_delivery/api/conf/logos_delivery_conf_json,
../declare_lib,
../json_event
@ -13,14 +15,14 @@ import
proc `%`*(id: RequestId): JsonNode =
%($id)
registerReqFFI(CreateNodeRequest, ctx: ptr FFIContext[Waku]):
registerReqFFI(CreateNodeRequest, ctx: ptr FFIContext[LogosDelivery]):
proc(configJson: cstring): Future[Result[string, string]] {.async.} =
let conf = parseNodeConfFromJson($configJson).valueOr:
error "Failed to assemble WakuNodeConf from JSON",
let conf = parseLogosDeliveryConf($configJson).valueOr:
error "Failed to parse Logos Delivery configuration JSON",
error = error, configJson = $configJson
return err("failed parseNodeConfFromJson " & error)
return err("failed parseLogosDeliveryConf " & error)
ctx.myLib[] = (await api.createNode(conf)).valueOr:
ctx.myLib[] = (await LogosDelivery.new(conf)).valueOr:
let errMsg = $error
chronicles.error "CreateNodeRequest failed", err = errMsg
return err(errMsg)
@ -28,7 +30,7 @@ registerReqFFI(CreateNodeRequest, ctx: ptr FFIContext[Waku]):
return ok("")
proc logosdelivery_destroy(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
): cint {.dynlib, exportc, cdecl.} =
initializeLibrary()
checkParams(ctx, callback, userData)
@ -52,7 +54,7 @@ proc logosdelivery_create_node(
echo "error: missing callback in logosdelivery_create_node"
return nil
var ctx = ffi.createFFIContext[Waku]().valueOr:
var ctx = ffi.createFFIContext[LogosDelivery]().valueOr:
let msg = "Error in createFFIContext: " & $error
callback(RET_ERR, unsafeAddr msg[0], cast[csize_t](len(msg)), userData)
return nil
@ -73,14 +75,14 @@ proc logosdelivery_create_node(
return ctx
proc logosdelivery_start_node(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
requireInitializedNode(ctx, "START_NODE"):
return err(errMsg)
# setting up outgoing event listeners
let sentListener = MessageSentEvent.listen(
ctx.myLib[].brokerCtx,
ctx.myLib[].waku.brokerCtx,
proc(event: MessageSentEvent) {.async: (raises: []).} =
callEventCallback(ctx, "onMessageSent"):
$newJsonEvent("message_sent", event),
@ -89,7 +91,7 @@ proc logosdelivery_start_node(
return err("MessageSentEvent.listen failed: " & $error)
let errorListener = MessageErrorEvent.listen(
ctx.myLib[].brokerCtx,
ctx.myLib[].waku.brokerCtx,
proc(event: MessageErrorEvent) {.async: (raises: []).} =
callEventCallback(ctx, "onMessageError"):
$newJsonEvent("message_error", event),
@ -98,7 +100,7 @@ proc logosdelivery_start_node(
return err("MessageErrorEvent.listen failed: " & $error)
let propagatedListener = MessagePropagatedEvent.listen(
ctx.myLib[].brokerCtx,
ctx.myLib[].waku.brokerCtx,
proc(event: MessagePropagatedEvent) {.async: (raises: []).} =
callEventCallback(ctx, "onMessagePropagated"):
$newJsonEvent("message_propagated", event),
@ -107,7 +109,7 @@ proc logosdelivery_start_node(
return err("MessagePropagatedEvent.listen failed: " & $error)
let receivedListener = MessageReceivedEvent.listen(
ctx.myLib[].brokerCtx,
ctx.myLib[].waku.brokerCtx,
proc(event: MessageReceivedEvent) {.async: (raises: []).} =
callEventCallback(ctx, "onMessageReceived"):
$newJsonEvent("message_received", event),
@ -116,7 +118,7 @@ proc logosdelivery_start_node(
return err("MessageReceivedEvent.listen failed: " & $error)
let ConnectionStatusChangeListener = EventConnectionStatusChange.listen(
ctx.myLib[].brokerCtx,
ctx.myLib[].waku.brokerCtx,
proc(event: EventConnectionStatusChange) {.async: (raises: []).} =
callEventCallback(ctx, "onConnectionStatusChange"):
$newJsonEvent("connection_status_change", event),
@ -124,15 +126,69 @@ proc logosdelivery_start_node(
chronicles.error "ConnectionStatusChange.listen failed", err = $error
return err("ConnectionStatusChange.listen failed: " & $error)
ctx.myLib[].mountMessagingClient().isOkOr:
let errMsg = $error
chronicles.error "mountMessagingClient failed", error = errMsg
return err("failed to mount messaging: " & errMsg)
let shardTopicHealthListener = EventShardTopicHealthChange.listen(
ctx.myLib[].waku.brokerCtx,
proc(event: EventShardTopicHealthChange) {.async: (raises: []).} =
callEventCallback(ctx, "onTopicHealthChange"):
$(
%*{
"eventType": "relay_topic_health_change",
"pubsubTopic": $event.topic,
"topicHealth": $event.health,
}
),
).valueOr:
chronicles.error "EventShardTopicHealthChange.listen failed", err = $error
return err("EventShardTopicHealthChange.listen failed: " & $error)
ctx.myLib[].mountReliableChannelManager().isOkOr:
let errMsg = $error
chronicles.error "mountReliableChannelManager failed", err = errMsg
return err("failed to mount reliable channel manager: " & errMsg)
let peerEventListener = WakuPeerEvent.listen(
ctx.myLib[].waku.brokerCtx,
proc(event: WakuPeerEvent) {.async: (raises: []).} =
callEventCallback(ctx, "onConnectionChange"):
$(
%*{
"eventType": "connection_change",
"peerId": $event.peerId,
"peerEvent": $event.kind,
}
),
).valueOr:
chronicles.error "WakuPeerEvent.listen failed", err = $error
return err("WakuPeerEvent.listen failed: " & $error)
let channelReceivedListener = ChannelMessageReceivedEvent.listen(
ctx.myLib[].waku.brokerCtx,
proc(event: ChannelMessageReceivedEvent) {.async: (raises: []).} =
callEventCallback(ctx, "onChannelMessageReceived"):
$(
%*{
"eventType": "channel_message_received",
"channelId": string(event.channelId),
"senderId": $event.senderId,
"payload": string(base64.encode(event.payload)),
}
),
).valueOr:
chronicles.error "ChannelMessageReceivedEvent.listen failed", err = $error
return err("ChannelMessageReceivedEvent.listen failed: " & $error)
let channelSentListener = ChannelMessageSentEvent.listen(
ctx.myLib[].waku.brokerCtx,
proc(event: ChannelMessageSentEvent) {.async: (raises: []).} =
callEventCallback(ctx, "onChannelMessageSent"):
$newJsonEvent("channel_message_sent", event),
).valueOr:
chronicles.error "ChannelMessageSentEvent.listen failed", err = $error
return err("ChannelMessageSentEvent.listen failed: " & $error)
let channelErrorListener = ChannelMessageErrorEvent.listen(
ctx.myLib[].waku.brokerCtx,
proc(event: ChannelMessageErrorEvent) {.async: (raises: []).} =
callEventCallback(ctx, "onChannelMessageError"):
$newJsonEvent("channel_message_error", event),
).valueOr:
chronicles.error "ChannelMessageErrorEvent.listen failed", err = $error
return err("ChannelMessageErrorEvent.listen failed: " & $error)
(await ctx.myLib[].start()).isOkOr:
let errMsg = $error
@ -141,16 +197,21 @@ proc logosdelivery_start_node(
return ok("")
proc logosdelivery_stop_node(
ctx: ptr FFIContext[Waku], callback: FFICallBack, userData: pointer
ctx: ptr FFIContext[LogosDelivery], callback: FFICallBack, userData: pointer
) {.ffi.} =
requireInitializedNode(ctx, "STOP_NODE"):
return err(errMsg)
await MessageErrorEvent.dropAllListeners(ctx.myLib[].brokerCtx)
await MessageSentEvent.dropAllListeners(ctx.myLib[].brokerCtx)
await MessagePropagatedEvent.dropAllListeners(ctx.myLib[].brokerCtx)
await MessageReceivedEvent.dropAllListeners(ctx.myLib[].brokerCtx)
await EventConnectionStatusChange.dropAllListeners(ctx.myLib[].brokerCtx)
await MessageErrorEvent.dropAllListeners(ctx.myLib[].waku.brokerCtx)
await MessageSentEvent.dropAllListeners(ctx.myLib[].waku.brokerCtx)
await MessagePropagatedEvent.dropAllListeners(ctx.myLib[].waku.brokerCtx)
await MessageReceivedEvent.dropAllListeners(ctx.myLib[].waku.brokerCtx)
await EventConnectionStatusChange.dropAllListeners(ctx.myLib[].waku.brokerCtx)
await EventShardTopicHealthChange.dropAllListeners(ctx.myLib[].waku.brokerCtx)
await WakuPeerEvent.dropAllListeners(ctx.myLib[].waku.brokerCtx)
await ChannelMessageReceivedEvent.dropAllListeners(ctx.myLib[].waku.brokerCtx)
await ChannelMessageSentEvent.dropAllListeners(ctx.myLib[].waku.brokerCtx)
await ChannelMessageErrorEvent.dropAllListeners(ctx.myLib[].waku.brokerCtx)
(await ctx.myLib[].stop()).isOkOr:
let errMsg = $error

21
logos_delivery.nim
View File

@ -1,10 +1,15 @@
## Main module for using nwaku as a Nimble library
## Package entry point for using Logos Messaging as a Nimble library.
##
## This module re-exports the public API for creating and managing Waku nodes
## when using nwaku as a library dependency.
## This root module is a thin aggregator, following the standard Nimble layout:
## the implementation lives under `./logos_delivery/`, and importing the package
## name re-exports `LogosDelivery` together with every per-layer public API.
##
## See `logos_delivery/logos_delivery.nim` for `LogosDelivery`, the pure
## concentrator that owns one instance of each API layer
##
## Waku <- MessagingClient <- ReliableChannelManager
##
## and drives their shared `new` / `start` / `stop` lifecycle.
import logos_delivery/waku/api
export api
import logos_delivery/waku/factory/waku
export waku
import ./logos_delivery/logos_delivery as logos_delivery_impl
export logos_delivery_impl

2
logos_delivery.nimble
View File

@ -65,7 +65,7 @@ requires "https://github.com/logos-messaging/nim-ffi#v0.1.3"
requires "https://github.com/logos-messaging/nim-sds.git#b12f5ee07c5b764303b51fb948b32a4ade1de3b5"
requires "https://github.com/NagyZoltanPeter/nim-brokers.git#v3.1.1"
requires "https://github.com/NagyZoltanPeter/nim-brokers.git#v3.1.4"
requires "https://github.com/vacp2p/nim-lsquic.git#v0.5.1"
requires "https://github.com/vacp2p/nim-jwt.git#057ec95eb5af0eea9c49bfe9025b3312c95dc5f2"

15
logos_delivery/api/conf/channels_conf.nim Normal file
View File

@ -0,0 +1,15 @@
import std/options
type ReliableChannelManagerConf* = object
## All-`Option` partial; unset fields fall back to `createReliableChannel` defaults.
segmentationEnableReedSolomon*: Option[bool]
## Add Reed-Solomon parity segments for recovery of lost segments.
segmentationSegmentSizeBytes*: Option[int] ## Maximum segment size in bytes.
sdsAcknowledgementTimeoutMs*: Option[int]
## Time to wait before retransmitting an unacknowledged message.
sdsMaxRetransmissions*: Option[int]
## Maximum retransmission attempts before delivery fails.
sdsCausalHistorySize*: Option[int] ## Number of message ids kept in causal history.
rateLimitEnabled*: Option[bool] ## Enable rate limiting.
rateLimitEpochPeriodSec*: Option[int] ## Rate-limit epoch length in seconds.
rateLimitMessagesPerEpoch*: Option[int] ## Messages allowed per rate-limit epoch.

6
logos_delivery/api/conf/kernel_conf.nim Normal file
View File

@ -0,0 +1,6 @@
import tools/confutils/cli_args
export cli_args
type KernelConf* = distinct WakuNodeConf
## Raw kernel config; distinct so `new(KernelConf)` doesn't collide with the
## full-stack `new(WakuNodeConf)`.

39
logos_delivery/api/conf/logos_delivery_conf.nim Normal file
View File

@ -0,0 +1,39 @@
{.push raises: [].}
import std/options
import results
import logos_delivery/api/conf/messaging_conf
import logos_delivery/api/conf/channels_conf
export options, messaging_conf, channels_conf
type LogosDeliveryConf* = object
## Aggregates the per-layer config objects. A layer is mounted iff its config
## is present.
kernelConf*: KernelConf
messagingConf*: Option[MessagingClientConf]
channelsConf*: Option[ReliableChannelManagerConf]
proc init*(T: type LogosDeliveryConf, kernelConf: KernelConf): LogosDeliveryConf =
return LogosDeliveryConf(kernelConf: kernelConf)
proc init*(
T: type LogosDeliveryConf,
mode: LogosDeliveryMode,
preset: string,
messagingOverrides: MessagingClientConf,
channelsOverrides: ReliableChannelManagerConf,
): ConfResult[LogosDeliveryConf] =
let merged = merge(?resolvePreset(preset), messagingOverrides)
var kernelConf = ?toWakuNodeConf(merged, mode)
kernelConf.preset = preset
return ok(
LogosDeliveryConf(
kernelConf: KernelConf(kernelConf),
messagingConf: some(merged),
channelsConf: some(channelsOverrides),
)
)
{.pop.}

163
logos_delivery/api/conf/logos_delivery_conf_json.nim Normal file
View File

@ -0,0 +1,163 @@
{.push raises: [].}
import std/[json, options, strutils, tables]
import results
import tools/confutils/conf_from_json
import logos_delivery/api/conf/logos_delivery_conf
const
# Lowercased, since `collectJsonFields` keys the object case-insensitively.
KeyMode = "mode"
KeyPreset = "preset"
KeyKernelConf = "kernelconf"
KeyMessagingOverrides = "messagingoverrides"
KeyChannelsOverrides = "channelsoverrides"
# [Legacy flat JSON config] Keys that left the kernel and so must be lifted out of
# a flat blob before the WakuNodeConf walker sees them (it would reject them).
KeyReliabilityEnabled = "reliabilityenabled"
KeyReliability = "reliability"
proc parseMode(s: string): Result[LogosDeliveryMode, string] =
case s.strip().toLowerAscii()
of "core":
return ok(LogosDeliveryMode.Core)
of "edge":
return ok(LogosDeliveryMode.Edge)
of "fleet":
return ok(LogosDeliveryMode.Fleet)
else:
return err("invalid mode: '" & s & "' (expected 'Core', 'Edge' or 'Fleet')")
proc parseOverrides[T](defaults: T, node: JsonNode, label: string): Result[T, string] =
## Parse the JSON object `node` as overrides on top of `defaults`.
if node.kind != JObject:
return err(label & " must be a JSON object")
var fields = ?collectJsonFields(node)
var conf = defaults
?applyJsonFieldsToConf(
conf,
fields,
"Failed to parse " & label & " field",
"Unrecognized " & label & " option(s) found",
)
return ok(conf)
proc parseFlatConf(
mode: LogosDeliveryMode, topJsonNode: var Table[string, (string, JsonNode)]
): ConfResult[LogosDeliveryConf] =
## [Legacy flat JSON config] Flat shape: a blob of `WakuNodeConf` fields. `mode`
## expands to protocol flags over raw kernel defaults, `reliabilityEnabled` routes
## to the messaging conf (it left the kernel), and the rest parses as a
## `WakuNodeConf`. Full stack. Delete this proc and its call site to drop support.
var messaging = MessagingClientConf()
var reliabilityFields: Table[string, (string, JsonNode)]
for key in [KeyReliabilityEnabled, KeyReliability]:
if topJsonNode.hasKey(key):
reliabilityFields[key] = topJsonNode.getOrDefault(key)
topJsonNode.del(key)
if reliabilityFields.len > 0:
?applyJsonFieldsToConf(
messaging, reliabilityFields, "Failed to parse reliability field",
"Unrecognized reliability option(s) found",
)
# [Legacy flat JSON config] The blob is a raw WakuNodeConf, exactly as the
# pre-refactor flat create_node parsed it: start from the kernel defaults and apply
# the mode's protocol flags (the kernel no longer owns `mode`, so we expand it here,
# like the old kernel builder did), then let explicit fields override.
var kernel = ?defaultWakuNodeConf()
?applyMode(kernel, mode)
?applyJsonFieldsToConf(
kernel, topJsonNode, "Failed to parse config field",
"Unrecognized configuration option(s) found",
)
# [Legacy flat JSON config] Reliability is resolved from the preset by the messaging
# layer (the kernel no longer carries it), so a flat blob's `preset` must lift it
# here to stay faithful to master. An explicit reliability in the blob still wins.
if kernel.preset.len > 0:
messaging = merge(?resolvePreset(kernel.preset), messaging)
return ok(
LogosDeliveryConf(
kernelConf: KernelConf(kernel),
messagingConf: some(messaging),
channelsConf: some(ReliableChannelManagerConf()),
)
)
proc parseLogosDeliveryConf*(jsonStr: string): ConfResult[LogosDeliveryConf] =
var node: JsonNode
try:
node = parseJson(jsonStr)
except CatchableError as e:
return err("invalid JSON: " & e.msg)
if node.kind != JObject:
return err("configuration JSON must be an object")
var top = ?collectJsonFields(node)
var mode = LogosDeliveryMode.Core
if top.hasKey(KeyMode):
let (_, v) = top.getOrDefault(KeyMode)
if v.kind != JString:
return err("mode must be a string")
mode = ?parseMode(v.getStr())
top.del(KeyMode)
if mode == LogosDeliveryMode.Fleet:
# Kernel-only: a raw kernelConf and no upper layers.
if not top.hasKey(KeyKernelConf):
return err("fleet mode requires a 'kernelConf' object")
let (_, v) = top.getOrDefault(KeyKernelConf)
let kernel = ?parseOverrides(?defaultWakuNodeConf(), v, "kernelConf")
top.del(KeyKernelConf)
if top.len > 0:
return
err(unknownKeysError(top, "fleet mode takes only 'kernelConf'; unexpected"))
return ok(LogosDeliveryConf.init(KernelConf(kernel)))
# [Legacy flat JSON config] A wrapper key marks our structured shape. Otherwise any
# leftover top-level key besides `preset` (mode is already consumed) is a bare
# kernel field -> flat blob. Delete this block to drop flat-shape support.
let hasWrapper =
top.hasKey(KeyMessagingOverrides) or top.hasKey(KeyChannelsOverrides) or
top.hasKey(KeyKernelConf)
if not hasWrapper:
var bareField = false
for k in top.keys:
if k != KeyPreset:
bareField = true
break
if bareField:
return parseFlatConf(mode, top)
var preset = ""
var messagingOverrides = MessagingClientConf()
var channelsOverrides = ReliableChannelManagerConf()
if top.hasKey(KeyPreset):
let (_, v) = top.getOrDefault(KeyPreset)
if v.kind != JString:
return err("preset must be a string")
preset = v.getStr().strip()
top.del(KeyPreset)
if top.hasKey(KeyMessagingOverrides):
let (_, v) = top.getOrDefault(KeyMessagingOverrides)
messagingOverrides = ?parseOverrides(MessagingClientConf(), v, "messagingOverrides")
top.del(KeyMessagingOverrides)
if top.hasKey(KeyChannelsOverrides):
let (_, v) = top.getOrDefault(KeyChannelsOverrides)
channelsOverrides =
?parseOverrides(ReliableChannelManagerConf(), v, "channelsOverrides")
top.del(KeyChannelsOverrides)
if top.len > 0:
return err(unknownKeysError(top, "Unrecognized configuration option(s) found"))
return LogosDeliveryConf.init(mode, preset, messagingOverrides, channelsOverrides)
{.pop.}

147
logos_delivery/api/conf/messaging_conf.nim Normal file
View File

@ -0,0 +1,147 @@
import std/options
import std/net
import results
import libp2p/crypto/crypto
import logos_delivery/api/conf/kernel_conf
import logos_delivery/waku/common/logging
import logos_delivery/waku/factory/networks_config
export kernel_conf
type LogosDeliveryMode* {.pure.} = enum
Edge # client-only node
Core # full service node
Fleet # kernel-only node from a raw kernel config
type MessagingClientConf* = object
clusterId* {.name: "cluster-id".}: Option[uint16] ## Network cluster id.
numShardsInCluster* {.name: "num-shards-in-network".}: Option[uint16]
## Number of shards in the cluster.
p2pTcpPort* {.name: "tcp-port".}: Option[Port] ## TCP listening port.
discv5UdpPort* {.name: "discv5-udp-port".}: Option[Port] ## discv5 UDP port.
websocketSupport* {.name: "websocket-support".}: Option[bool]
## Enable the websocket transport.
websocketPort* {.name: "websocket-port".}: Option[Port] ## Websocket listening port.
quicSupport* {.name: "quic-support".}: Option[bool] ## Enable the QUIC transport.
quicPort* {.name: "quic-port".}: Option[Port] ## QUIC (UDP) listening port.
listenIpv4* {.name: "listen-address".}: Option[IpAddress] ## Inbound bind address.
maxMessageSize* {.name: "max-msg-size".}: Option[string]
## Maximum accepted message size (e.g. "150 KiB").
entryNodes* {.name: "entry-node".}: Option[seq[string]]
## Bootstrap / connectivity nodes (enrtree or multiaddr).
ethRpcEndpoints* {.name: "rln-relay-eth-client-address".}: Option[seq[EthRpcUrl]]
## Ethereum RPC endpoints (required for RLN validation); multiple for fail-over.
rlnContractAddress* {.name: "rln-relay-eth-contract-address".}: Option[string]
## RLN contract address; when set, RLN validation is enabled.
rlnChainId* {.name: "rln-relay-chain-id".}: Option[uint]
## Chain id the RLN contract is deployed on.
rlnEpochSizeSec* {.name: "rln-relay-epoch-sec".}: Option[uint]
## RLN epoch size, in seconds.
reliabilityEnabled* {.name: "reliability".}: Option[bool]
## Enable store-based send reliability.
store*: Option[bool] ## Enable the store protocol.
storenode* {.name: "storenode".}: Option[string]
storeMessageDbUrl* {.name: "store-message-db-url".}: Option[string]
## Database connection URL for the store service's persistent storage.
storeMessageRetentionPolicy* {.name: "store-message-retention-policy".}:
Option[string] ## Store retention policy (e.g. "time:3600;size:1GB").
storeMaxNumDbConnections* {.name: "store-max-num-db-connections".}: Option[int]
## Maximum number of simultaneous store database connections.
logLevel* {.name: "log-level".}: Option[logging.LogLevel]
## Process log level (TRACE..FATAL); applied by the kernel on node creation.
logFormat* {.name: "log-format".}: Option[logging.LogFormat]
## Process log format (TEXT or JSON); applied by the kernel on node creation.
nodeKey* {.name: "nodekey".}: Option[crypto.PrivateKey]
## P2P node private key (64-char hex): stable identity / peerId across restarts.
proc applyMode*(conf: var WakuNodeConf, mode: LogosDeliveryMode): ConfResult[void] =
## Sets the protocol flags implied by the mode.
case mode
of LogosDeliveryMode.Core:
conf.relay = true
conf.filter = true
conf.lightpush = true
conf.discv5Discovery = some(true)
conf.peerExchange = true
conf.rendezvous = true
of LogosDeliveryMode.Edge:
conf.peerExchange = true
conf.relay = false
conf.filter = false
conf.lightpush = false
conf.store = false
of LogosDeliveryMode.Fleet:
return
err("fleet mode takes a raw kernel config; use LogosDelivery.new(kernelConf)")
return ok()
proc toWakuNodeConf*(
self: MessagingClientConf, mode: LogosDeliveryMode
): ConfResult[WakuNodeConf] =
## Mode sets the protocol flags; set fields map to their kernel counterpart.
var conf = ?defaultWakuNodeConf()
?applyMode(conf, mode)
if self.store.isSome():
conf.store = self.store.get()
if self.storeMessageDbUrl.isSome():
conf.storeMessageDbUrl = self.storeMessageDbUrl.get()
if self.storeMessageRetentionPolicy.isSome():
conf.storeMessageRetentionPolicy = self.storeMessageRetentionPolicy.get()
if self.storeMaxNumDbConnections.isSome():
conf.storeMaxNumDbConnections = self.storeMaxNumDbConnections.get()
if self.storenode.isSome():
conf.storenode = self.storenode.get()
if self.clusterId.isSome():
conf.clusterId = self.clusterId
if self.numShardsInCluster.isSome():
conf.numShardsInNetwork = self.numShardsInCluster.get()
if self.listenIpv4.isSome():
conf.listenAddress = self.listenIpv4.get()
if self.maxMessageSize.isSome():
conf.maxMessageSize = self.maxMessageSize.get()
if self.entryNodes.isSome():
conf.entryNodes = self.entryNodes.get()
if self.ethRpcEndpoints.isSome():
conf.ethClientUrls = self.ethRpcEndpoints.get()
if self.rlnContractAddress.isSome():
conf.rlnRelayEthContractAddress = self.rlnContractAddress.get()
conf.rlnRelay = some(true)
if self.rlnChainId.isSome():
conf.rlnRelayChainId = self.rlnChainId.get()
if self.rlnEpochSizeSec.isSome():
conf.rlnEpochSizeSec = some(self.rlnEpochSizeSec.get().uint64)
if self.logLevel.isSome():
conf.logLevel = self.logLevel.get()
if self.logFormat.isSome():
conf.logFormat = self.logFormat.get()
if self.nodeKey.isSome():
conf.nodekey = self.nodeKey
conf.tcpPort = self.p2pTcpPort.get(Port(0))
conf.discv5UdpPort = self.discv5UdpPort.get(Port(0))
conf.websocketPort = self.websocketPort.get(Port(0))
conf.quicPort = self.quicPort.get(Port(0))
conf.websocketSupport = self.websocketSupport.get(false)
conf.quicSupport = self.quicSupport.get(true)
return ok(conf)
proc merge*(base, overrides: MessagingClientConf): MessagingClientConf =
var m = base
for _, mField, oField in fieldPairs(m, overrides):
when oField is Option:
if oField.isSome():
mField = oField
return m
proc resolvePreset*(preset: string): ConfResult[MessagingClientConf] =
## Preset to messaging-only fields. Kernel-mirrored fields stay unset; the
## kernel resolves those from `conf.preset`.
let npcOpt = ?toNetworkPresetConf(preset, none(uint16))
if npcOpt.isNone():
return ok(MessagingClientConf())
let npc = npcOpt.get()
return ok(MessagingClientConf(reliabilityEnabled: some(npc.p2pReliability)))

18
logos_delivery/api/events/kernel_events.nim Normal file
View File

@ -0,0 +1,18 @@
import brokers/event_broker
import logos_delivery/api/types
import logos_delivery/waku/waku_core/topics/pubsub_topic
import logos_delivery/waku/waku_core/message
export event_broker, pubsub_topic, message
EventBroker:
# Internal event emitted when a message arrives from the network via any protocol
type MessageSeenEvent* = object
topic*: PubsubTopic
message*: WakuMessage
# Emitted by the health monitor when overall node connectivity changes.
EventBroker:
type EventConnectionStatusChange* = object
connectionStatus*: ConnectionStatus

12
logos_delivery/waku/events/message_events.nim → logos_delivery/api/events/messaging_client_events.nim
View File

@ -1,6 +1,8 @@
import brokers/event_broker
import logos_delivery/waku/[api/types, waku_core/message, waku_core/topics]
export types
import logos_delivery/api/types as api_types
export event_broker, api_types
EventBroker:
# Event emitted when a message is sent to the network
@ -26,9 +28,3 @@ EventBroker:
type MessageReceivedEvent* = object
messageHash*: string
message*: WakuMessage
EventBroker:
# Internal event emitted when a message arrives from the network via any protocol
type MessageSeenEvent* = object
topic*: PubsubTopic
message*: WakuMessage

16
logos_delivery/channels/events.nim → logos_delivery/api/events/reliable_channel_manager_events.nim
View File

@ -1,20 +1,8 @@
## Reliable Channel event types emitted to API consumers.
##
## Lifecycle events for individual segments (sent / propagated / errored)
## are the same as the network-level ones the MessagingClient already
## emits — `requestId` is shared across layers — so we just re-export
## `waku/events/message_events` and avoid declaring duplicates.
##
## Only the channel-level `MessageReceivedEvent` carries data that has
## no analogue in the lower layer (reassembled application payload,
## senderId, channelId), so it lives here.
import logos_delivery/waku/events/message_events as waku_message_events
import brokers/event_broker
import ./types as channel_types
import logos_delivery/channels/types as channel_types
export waku_message_events, channel_types, event_broker
export event_broker, channel_types
EventBroker:
type ChannelMessageReceivedEvent* = object

81
logos_delivery/api/kernel_api.nim Normal file
View File

@ -0,0 +1,81 @@
import chronos, results
import logos_delivery/api/types as api_types
import logos_delivery/waku/waku_core/topics/pubsub_topic
import logos_delivery/waku/waku_core/message
import logos_delivery/waku/waku_core/subscription/push_handler
import logos_delivery/waku/waku_store/common as store_types
export api_types, pubsub_topic, store_types
# Structural API contract for the Kernel surface, implemented by `Waku`
# (ops in `waku/api/*`).
type KernelApi* = concept w
# --- topic construction ---
buildContentTopic(
w, appName = string, appVersion = uint32, name = string, encoding = string
) is Future[Result[ContentTopic, string]]
buildPubsubTopic(w, topicName = string) is Future[Result[PubsubTopic, string]]
defaultPubsubTopic(w) is Future[Result[PubsubTopic, string]]
# --- relay ---
relayPublish(w, pubsubTopic = PubsubTopic, message = WakuMessage, timeoutMs = uint32) is
Future[Result[string, string]]
relaySubscribe(w, pubsubTopic = PubsubTopic) is Future[Result[bool, string]]
relayUnsubscribe(w, pubsubTopic = PubsubTopic) is Future[Result[bool, string]]
relayAddProtectedShard(w, clusterId = uint16, shardId = uint16, publicKey = string) is
Future[Result[bool, string]]
relayConnectedPeers(w, pubsubTopic = PubsubTopic) is
Future[Result[seq[string], string]]
relayPeersInMesh(w, pubsubTopic = PubsubTopic) is Future[Result[seq[string], string]]
relayNumPeersInMesh(w, pubsubTopic = PubsubTopic) is Future[Result[int, string]]
relayNumConnectedPeers(w, pubsubTopic = PubsubTopic) is Future[Result[int, string]]
# --- filter ---
filterSubscribe(
w,
pubsubTopic = PubsubTopic,
contentTopics = seq[ContentTopic],
pushHandler = FilterPushHandler,
) is Future[Result[bool, string]]
filterUnsubscribe(w, pubsubTopic = PubsubTopic, contentTopics = seq[ContentTopic]) is
Future[Result[bool, string]]
filterUnsubscribeAll(w) is Future[Result[bool, string]]
# --- lightpush ---
lightpushPublish(w, pubsubTopic = PubsubTopic, message = WakuMessage) is
Future[Result[string, string]]
# --- store ---
storeQuery(w, request = StoreQueryRequest, peer = string, timeoutMs = int) is
Future[Result[StoreQueryResponse, string]]
# --- peer management ---
connect(w, peers = seq[string], timeoutMs = uint32) is Future[Result[bool, string]]
disconnectPeerById(w, peerId = string) is Future[Result[bool, string]]
disconnectAllPeers(w) is Future[Result[bool, string]]
dialPeer(w, peerAddr = string, protocol = string, timeoutMs = int) is
Future[Result[bool, string]]
dialPeerById(w, peerId = string, protocol = string, timeoutMs = int) is
Future[Result[bool, string]]
peerIdsFromPeerstore(w) is Future[Result[seq[string], string]]
connectedPeersInfo(w) is Future[Result[seq[PeerConnInfo], string]]
connectedPeers(w) is Future[Result[seq[string], string]]
peerIdsByProtocol(w, protocol = string) is Future[Result[seq[string], string]]
# --- discovery ---
dnsDiscovery(w, enrTreeUrl = string, nameServer = string, timeoutMs = int) is
Future[Result[seq[string], string]]
discv5UpdateBootnodes(w, bootnodes = string) is Future[Result[bool, string]]
startDiscv5(w) is Future[Result[bool, string]]
stopDiscv5(w) is Future[Result[bool, string]]
peerExchangeRequest(w, numPeers = uint64) is Future[Result[int, string]]
# --- debug / info ---
version(w) is Future[Result[string, string]]
listenAddresses(w) is Future[Result[seq[string], string]]
myEnr(w) is Future[Result[string, string]]
myPeerId(w) is Future[Result[string, string]]
metrics(w) is Future[Result[string, string]]
isOnline(w) is Future[Result[bool, string]]
pingPeer(w, peerAddr = string, timeoutMs = int) is Future[Result[int64, string]]

11
logos_delivery/api/logos_delivery_api.nim Normal file
View File

@ -0,0 +1,11 @@
import chronos, results
# Structural API contract for the top-level entry point, implemented by
# `LogosDelivery` (the aggregate owning one instance of each API layer).
type LogosDeliveryApi* = concept ld
# --- lifecycle ---
start(ld) is Future[Result[void, string]]
stop(ld) is Future[Result[void, string]]
# --- health ---
isOnline(ld) is Future[Result[bool, string]]

18
logos_delivery/api/messaging_client_api.nim Normal file
View File

@ -0,0 +1,18 @@
import chronos, results
import brokers/[request_broker, broker_context]
import logos_delivery/api/types as api_types
export api_types, request_broker, broker_context
# Structural API contract for a messaging client (ops in `messaging/api/*`).
type MessagingApi* = concept c
subscribe(c, contentTopic = ContentTopic) is Future[Result[void, string]]
unsubscribe(c, contentTopic = ContentTopic) is Result[void, string]
send(c, envelope = MessageEnvelope) is Future[Result[RequestId, string]]
# Semi detached MessagingClient send interface.
# Can be used without MessagingClient, under the same context of MessagingClient instance.
RequestBroker:
proc MessagingSend(
envelope: MessageEnvelope
): Future[Result[RequestId, string]] {.async.}

15
logos_delivery/api/reliable_channel_manager_api.nim Normal file
View File

@ -0,0 +1,15 @@
import chronos, results
import logos_delivery/api/types as api_types
import logos_delivery/channels/types as channel_types
export api_types, channel_types
# Structural API contract for the reliable-channel surface (ops in `channels/api/*`).
type ReliableChannelApi* = concept c
createReliableChannel(
c, channelId = ChannelId, contentTopic = ContentTopic, senderId = SdsParticipantID
) is Result[ChannelId, string]
closeChannel(c, channelId = ChannelId) is Future[Result[void, string]]
send(c, channelId = ChannelId, appPayload = seq[byte]) is
Future[Result[RequestId, string]]

8
logos_delivery/waku/api/types.nim → logos_delivery/api/types.nim
View File

@ -6,7 +6,8 @@ import bearssl/rand, std/times, chronos
import stew/byteutils
import logos_delivery/waku/utils/requests as request_utils
import logos_delivery/waku/waku_core/[topics/content_topic, message/message, time]
import logos_delivery/waku/requests/requests
export content_topic, message
type
MessageEnvelope* = object
@ -25,6 +26,11 @@ type
PartiallyConnected
Connected
PeerConnInfo* = object ## structured connected-peer info for the api boundary
peerId*: string
protocols*: seq[string]
addresses*: seq[string]
proc new*(T: typedesc[RequestId], rng: crypto.Rng): T =
## Generate a new RequestId using the provided RNG.
RequestId(request_utils.generateRequestId(rng))

87
logos_delivery/channels/api/channel_lifecycle.nim Normal file
View File

@ -0,0 +1,87 @@
## Reliable Channel layer API — channel lifecycle
## (createReliableChannel / closeChannel).
import std/[options, tables]
import results, chronos, chronicles
import logos_delivery/api/types
import logos_delivery/channels/reliable_channel_manager
import logos_delivery/channels/reliable_channel
import logos_delivery/waku/persistency/sds_persistency
# ReliableChannel, config and wire-version markers.
export reliable_channel
const SdsJobId = "sds"
## One persistency job shared by every channel's SDS state; rows are
## keyed by channelId.
proc sdsPersistence(): Option[Persistence] =
## SDS backend from the Persistency singleton; memory-only fallback when
## it is unavailable (e.g. unit tests).
let p = Persistency.instance().valueOr:
info "SDS persistence disabled, running memory-only", reason = $error
return none(Persistence)
let job = p.openJob(SdsJobId).valueOr:
warn "SDS persistence disabled, could not open persistency job",
jobId = SdsJobId, reason = $error
return none(Persistence)
return some(newSdsPersistence(job))
proc createReliableChannel*(
self: ReliableChannelManager,
channelId: ChannelId,
contentTopic: ContentTopic,
senderId: SdsParticipantID,
): Result[ChannelId, string] =
## Encryption and egress providers must be installed (or `setNoopEncryption()`)
## before traffic flows on the channel.
if self.channels.hasKey(channelId):
return err("channel already exists: " & channelId)
let cc = self.conf
let segConfig = SegmentationConfig(
segmentSizeBytes: cc.segmentationSegmentSizeBytes.get(DefaultSegmentSizeBytes),
enableReedSolomon: cc.segmentationEnableReedSolomon.get(false),
persistence: nil,
)
let sdsConfig = SdsConfig(
acknowledgementTimeoutMs:
cc.sdsAcknowledgementTimeoutMs.get(DefaultAcknowledgementTimeoutMs),
maxRetransmissions: cc.sdsMaxRetransmissions.get(DefaultMaxRetransmissions),
causalHistorySize: cc.sdsCausalHistorySize.get(DefaultCausalHistorySize),
persistence: sdsPersistence(),
)
let rateConfig = RateLimitConfig(
# Setting a rate-limit parameter implies enabling; an explicit
# rateLimitEnabled still wins.
enabled: cc.rateLimitEnabled.get(
cc.rateLimitEpochPeriodSec.isSome() or cc.rateLimitMessagesPerEpoch.isSome()
),
epochPeriodSec: cc.rateLimitEpochPeriodSec.get(DefaultEpochPeriodSec),
messagesPerEpoch: cc.rateLimitMessagesPerEpoch.get(DefaultMessagesPerEpoch),
)
let chn = ReliableChannel.new(
channelId = channelId,
contentTopic = contentTopic,
senderId = senderId,
segConfig = segConfig,
sdsConfig = sdsConfig,
rateConfig = rateConfig,
brokerCtx = self.brokerCtx,
)
self.channels[channelId] = chn
return ok(channelId)
proc closeChannel*(
self: ReliableChannelManager, channelId: ChannelId
): Future[Result[void, string]] {.async: (raises: []).} =
## Stops the channel's SDS loops and releases the channel. Persisted SDS
## state survives, so re-creating the channel restores it.
let chn = self.channels.getOrDefault(channelId)
if chn.isNil():
return err("unknown channel: " & channelId)
self.channels.del(channelId)
await chn.stop()
return ok()

21
logos_delivery/channels/api/send.nim Normal file
View File

@ -0,0 +1,21 @@
## Reliable Channel layer API — channel send operation.
import std/tables
import results, chronos
import logos_delivery/api/types
import logos_delivery/channels/reliable_channel_manager
import logos_delivery/channels/reliable_channel
proc send*(
self: ReliableChannelManager,
channelId: ChannelId,
appPayload: seq[byte],
ephemeral: bool = false,
): Future[Result[RequestId, string]] {.async: (raises: []).} =
## Spec-level entry point. Looks the channel up by id and delegates
## to `ReliableChannel.send`, which exposes the visible pipeline
## segmentation -> sds -> rate_limit_manager -> encryption.
let chn = self.channels.getOrDefault(channelId)
if chn.isNil():
return err("unknown channel: " & channelId)
return await chn.send(appPayload, ephemeral)

48
logos_delivery/channels/reliable_channel.nim
View File

@ -21,19 +21,22 @@ import bearssl/rand
import stew/byteutils
import libp2p/crypto/crypto as libp2p_crypto
import logos_delivery/waku/api/types
import logos_delivery/messaging/delivery_service/send_service
import logos_delivery/api/types
import logos_delivery/api/reliable_channel_manager_api
import logos_delivery/api/events/messaging_client_events
import logos_delivery/api/messaging_client_api
import logos_delivery/api/events/reliable_channel_manager_events
import logos_delivery/messaging/messaging_client
import logos_delivery/waku/waku_core/topics
import ./events
import ./segmentation/segmentation
import ./scalable_data_sync/scalable_data_sync
import ./rate_limit_manager/rate_limit_manager
import ./encryption/encryption
export
types, send_service, events, segmentation, scalable_data_sync, rate_limit_manager,
encryption
types, reliable_channel_manager_api, segmentation, scalable_data_sync,
rate_limit_manager, encryption
const LipWireReliableChannelVersion* = "RELIABLE-CHANNEL-API/1"
## Wire-format spec marker for the Reliable Channel layer, as defined
@ -44,14 +47,6 @@ const LipWireReliableChannelVersion* = "RELIABLE-CHANNEL-API/1"
## on breaking on-the-wire changes; implementations pin one version.
type
SendHandler* = proc(envelope: MessageEnvelope): Future[Result[RequestId, string]] {.
async: (raises: [CatchableError]), gcsafe
.}
## Egress dispatch boundary. Typically wraps `MessagingClient.send`;
## tests inject a fake that records calls and returns canned
## `RequestId`s so the send state machine can be exercised end-to-end
## without a network.
MessagePersistence {.pure.} = enum
Persistent
Ephemeral
@ -90,7 +85,6 @@ type
## Spec-defined public type. Fields are private so callers cannot
## mutate internals and break invariants. Getters are added below
## for the few values consumers may need.
sendHandler: SendHandler
channelId: ChannelId
contentTopic: ContentTopic
senderId: SdsParticipantID
@ -135,7 +129,7 @@ proc tryFinalizeChannelReq(self: ReliableChannel, channelReqId: RequestId) =
## and the total number of confirmed + failed segments equals the total expected segments.
## Therefore, the channel-level request is removed from `self.channelReqs`
## and the appropriate final event is emitted.
##
##
let state = self.channelReqs.getOrDefault(channelReqId)
if state.totalExpectedSegments == 0:
## Either already finalized (and removed) or never inserted.
@ -269,14 +263,7 @@ proc onReadyToSend(
meta: LipWireReliableChannelVersion.toBytes(),
)
## `sendHandler` is not annotated `(raises: [])`, but this listener is.
## Convert any raise to a Result error so the state machine handles
## both failure modes (Result.err and exception) through one path.
let sendRes =
try:
await self.sendHandler(envelope)
except CatchableError as e:
Result[RequestId, string].err("messaging send raised: " & e.msg)
let sendRes = await MessagingSend.request(self.brokerCtx, envelope)
let messagingReqId = sendRes.valueOr:
MessageErrorEvent.emit(
@ -366,14 +353,9 @@ proc dispatchRepair(self: ReliableChannel, wire: seq[byte]) {.async: (raises: []
meta: LipWireReliableChannelVersion.toBytes(),
)
let sendRes =
try:
await self.sendHandler(envelope)
except CatchableError as e:
Result[RequestId, string].err("messaging send raised: " & e.msg)
if sendRes.isErr():
(await MessagingSend.request(self.brokerCtx, envelope)).isOkOr:
debug "SDS repair rebroadcast dropped: dispatch failed",
channelId = self.channelId, error = sendRes.error
channelId = self.channelId, error = error
proc onMessageReceived(
self: ReliableChannel, messageHash: string, payload: seq[byte]
@ -421,7 +403,6 @@ proc onMessageReceived(
proc new*(
T: type ReliableChannel,
sendHandler: SendHandler,
channelId: ChannelId,
contentTopic: ContentTopic,
senderId: SdsParticipantID,
@ -436,12 +417,7 @@ proc new*(
## should be wiring up. Encryption is delegated to the `Encrypt`/
## `Decrypt` request brokers, so the channel keeps no per-instance
## encryption state either.
##
## `sendHandler` is the egress dispatch. The owning `ReliableChannelManager`
## typically constructs it as a closure over `MessagingClient.send`. Tests
## pass a fake to drive the send state machine without touching the network.
let chn = T(
sendHandler: sendHandler,
channelId: channelId,
contentTopic: contentTopic,
senderId: senderId,

128
logos_delivery/channels/reliable_channel_manager.nim
View File

@ -5,7 +5,7 @@
##
## See: https://lip.logos.co/messaging/raw/reliable-channel-api.html
import std/[options, tables]
import std/tables
import results
import chronos
import chronicles
@ -13,44 +13,28 @@ import stew/byteutils
import brokers/broker_context
import logos_delivery/waku/events/message_events as waku_message_events
import logos_delivery/messaging/messaging_client
import logos_delivery/waku/waku_core/topics
import logos_delivery/waku/persistency/sds_persistency
import logos_delivery/api/types
import logos_delivery/api/reliable_channel_manager_api
import logos_delivery/api/conf/channels_conf
import ./reliable_channel
import ./encryption/noop_encryption
export reliable_channel
export reliable_channel, channels_conf
const SdsJobId = "sds"
## One persistency job shared by every channel's SDS state; rows are
## keyed by channelId.
type ReliableChannelManager* = ref object
channels: Table[ChannelId, ReliableChannel]
messagingClient: MessagingClient ## Borrowed from the owning `Waku`.
sendHandler: SendHandler
## Default egress dispatch for channels created through this manager.
## Constructed at mount time as a closure over `MessagingClient.send`
## so the channel layer itself stays callable-only.
brokerCtx: BrokerContext
type ReliableChannelManager* = ref object ## Implements `ReliableChannelApi`.
channels*: Table[ChannelId, ReliableChannel] ## read by `channels/api.nim`
conf*: ReliableChannelManagerConf
brokerCtx*: BrokerContext
proc new*(
T: type ReliableChannelManager,
messagingClient: MessagingClient,
sendHandler: SendHandler,
conf: ReliableChannelManagerConf,
brokerCtx: BrokerContext = globalBrokerContext(),
): Result[T, string] =
if messagingClient.isNil():
return err("messaging client is required")
if sendHandler.isNil():
return err("sendHandler is required")
return ok(
T(
channels: initTable[ChannelId, ReliableChannel](),
messagingClient: messagingClient,
sendHandler: sendHandler,
conf: conf,
brokerCtx: brokerCtx,
)
)
@ -68,96 +52,6 @@ proc stop*(self: ReliableChannelManager) {.async.} =
await chn.stop()
self.channels.clear()
proc sdsPersistence(): Option[Persistence] =
## SDS backend from the Persistency singleton; memory-only fallback when
## it is unavailable (e.g. unit tests).
let p = Persistency.instance().valueOr:
info "SDS persistence disabled, running memory-only", reason = $error
return none(Persistence)
let job = p.openJob(SdsJobId).valueOr:
warn "SDS persistence disabled, could not open persistency job",
jobId = SdsJobId, reason = $error
return none(Persistence)
return some(newSdsPersistence(job))
proc createReliableChannel*(
self: ReliableChannelManager,
channelId: ChannelId,
contentTopic: ContentTopic,
senderId: SdsParticipantID,
sendHandler: SendHandler = nil,
): Result[ChannelId, string] =
## Spec entry point. The `sendHandler` and `rng` the channel needs are
## sourced from the owning `ReliableChannelManager` rather than passed
## per call. Encryption is wired up through the `Encrypt`/`Decrypt`
## request brokers — the application installs its own providers
## (or `setNoopEncryption()`) before traffic flows.
##
## Segmentation, SDS and rate-limit configs will eventually be read
## from the node's `NodeConfig`. Defaults for now.
##
## `sendHandler` defaults to the manager's default (constructed at mount
## from `MessagingClient.send`); tests pass a fake to bypass the network.
if self.channels.hasKey(channelId):
return err("channel already exists: " & channelId)
let segConfig = SegmentationConfig(
segmentSizeBytes: DefaultSegmentSizeBytes,
enableReedSolomon: false,
persistence: nil,
)
let sdsConfig = SdsConfig(
acknowledgementTimeoutMs: DefaultAcknowledgementTimeoutMs,
maxRetransmissions: DefaultMaxRetransmissions,
causalHistorySize: DefaultCausalHistorySize,
persistence: sdsPersistence(),
)
let rateConfig = RateLimitConfig(
epochPeriodSec: DefaultEpochPeriodSec, messagesPerEpoch: DefaultMessagesPerEpoch
)
let effectiveSendHandler = if sendHandler.isNil(): self.sendHandler else: sendHandler
let chn = ReliableChannel.new(
sendHandler = effectiveSendHandler,
channelId = channelId,
contentTopic = contentTopic,
senderId = senderId,
segConfig = segConfig,
sdsConfig = sdsConfig,
rateConfig = rateConfig,
brokerCtx = self.brokerCtx,
)
self.channels[channelId] = chn
return ok(channelId)
proc closeChannel*(
self: ReliableChannelManager, channelId: ChannelId
): Future[Result[void, string]] {.async: (raises: []).} =
## Stops the channel's SDS loops and releases the channel. Persisted SDS
## state survives, so re-creating the channel restores it.
let chn = self.channels.getOrDefault(channelId)
if chn.isNil():
return err("unknown channel: " & channelId)
self.channels.del(channelId)
await chn.stop()
return ok()
proc send*(
self: ReliableChannelManager,
channelId: ChannelId,
appPayload: seq[byte],
ephemeral: bool = false,
): Future[Result[RequestId, string]] {.async: (raises: []).} =
## Spec-level entry point. Looks the channel up by id and delegates
## to `ReliableChannel.send`, which exposes the visible pipeline
## segmentation -> sds -> rate_limit_manager -> encryption.
let chn = self.channels.getOrDefault(channelId)
if chn.isNil():
return err("unknown channel: " & channelId)
return await chn.send(appPayload, ephemeral)
## Inbound messages are not handed to the manager by direct call. Each
## `ReliableChannel` installs its own `MessageReceivedEvent` listener
## in `ReliableChannel.new`, filters by spec marker and `contentTopic`,

2
logos_delivery/channels/types.nim
View File

@ -1,7 +1,7 @@
## Core identifier types for the Reliable Channel API.
import std/hashes
import logos_delivery/waku/api/types as api_types
import logos_delivery/api/types as api_types
import ./scalable_data_sync/scalable_data_sync

210
logos_delivery/logos_delivery.nim Normal file
View File

@ -0,0 +1,210 @@
## `LogosDelivery` is the project entry point. It is a pure concentrator: it
## owns exactly one instance of each API layer
##
## Waku <- MessagingClient <- ReliableChannelManager
##
## and chains them together (each layer drives the one below it). Every layer
## keeps its own, separate public API — `LogosDelivery` only wires them up and
## drives the shared `new` / `start` / `stop` lifecycle.
{.push raises: [].}
import std/options
import results, chronos, chronicles
# Each layer has a core module (type + new/start/stop) and an api/ folder whose
# modules each implement a differentiated set of operations, plus an events
# surface. The concentrator re-exports them so library consumers get the full
# surface from `import logos_delivery`. (The per-layer `events` modules share a
# stem, so they are imported under aliases.)
# Waku layer
import logos_delivery/waku/waku
export waku
import
logos_delivery/waku/api/[
topics, relay, subscriptions, filter, lightpush, store, peer_manager, discovery,
debug, health, ping,
]
export
topics, relay, subscriptions, filter, lightpush, store, peer_manager, discovery,
debug, health, ping
# Kernel event surface (`MessageSeenEvent`) plus the remaining waku health events.
import logos_delivery/api/events/kernel_events
export kernel_events
import logos_delivery/waku/api/events/health_events
export health_events
# Messaging layer
import logos_delivery/messaging/[messaging_client, messaging_client_lifecycle]
export messaging_client
import logos_delivery/messaging/api/[subscription, send]
export subscription, send
import logos_delivery/api/events/messaging_client_events
export messaging_client_events
import logos_delivery/api/conf/messaging_conf
export messaging_conf
# Reliable Channel layer
import logos_delivery/channels/reliable_channel_manager
export reliable_channel_manager
import logos_delivery/channels/api/channel_lifecycle
export channel_lifecycle
import logos_delivery/channels/api/send as channel_send
export channel_send
import logos_delivery/api/events/reliable_channel_manager_events
export reliable_channel_manager_events
import logos_delivery/api/logos_delivery_api
export logos_delivery_api
import logos_delivery/waku/factory/waku_conf
import logos_delivery/waku/factory/app_callbacks
import tools/confutils/cli_args
import logos_delivery/waku/node/health_monitor/online_monitor
import logos_delivery/api/conf/logos_delivery_conf
export logos_delivery_conf
logScope:
topics = "logosdelivery"
type LogosDelivery* = ref object ## Entry point. Holds one instance of each API layer.
waku*: Waku
messagingClient*: MessagingClient
reliableChannelManager*: ReliableChannelManager
proc new*(
T: type LogosDelivery, conf: LogosDeliveryConf, appCallbacks: AppCallbacks = nil
): Future[Result[LogosDelivery, string]] {.async.} =
## Builds the stack bottom-up from a resolved per-layer config; each layer is
## mounted iff its config is present.
let wakuConf = WakuNodeConf(conf.kernelConf).toWakuConf().valueOr:
return err("failed to handle the configuration: " & error)
let waku = (await Waku.new(wakuConf, appCallbacks)).valueOr:
return err("failed to create Waku: " & error)
let messagingClient =
if conf.messagingConf.isSome():
MessagingClient.new(conf.messagingConf.get(), waku).valueOr:
return err("failed to create MessagingClient: " & error)
else:
nil
let reliableChannelManager =
if conf.channelsConf.isSome():
ReliableChannelManager.new(conf.channelsConf.get(), waku.brokerCtx).valueOr:
return err("failed to create ReliableChannelManager: " & error)
else:
nil
return ok(
LogosDelivery(
waku: waku,
messagingClient: messagingClient,
reliableChannelManager: reliableChannelManager,
)
)
proc new*(
T: type LogosDelivery, conf: WakuNodeConf, appCallbacks: AppCallbacks = nil
): Future[Result[LogosDelivery, string]] {.async.} =
## Builds the full stack from a kernel `WakuNodeConf`.
return await LogosDelivery.new(
LogosDeliveryConf(
kernelConf: KernelConf(conf),
messagingConf: some(MessagingClientConf()),
channelsConf: some(ReliableChannelManagerConf()),
),
appCallbacks,
)
proc new*(
T: type LogosDelivery, kernelConf: KernelConf, appCallbacks: AppCallbacks = nil
): Future[Result[LogosDelivery, string]] {.async.} =
## Fleet mode: mounts the kernel only from a raw `KernelConf`; no messaging client,
## no channel manager.
return await LogosDelivery.new(LogosDeliveryConf.init(kernelConf), appCallbacks)
proc new*(
T: type LogosDelivery,
kernelConf: KernelConf,
messagingOverrides: MessagingClientConf,
channelsOverrides: ReliableChannelManagerConf,
appCallbacks: AppCallbacks = nil,
): Future[Result[LogosDelivery, string]] {.async.} =
## Full stack: kernel + messaging + channels. Messaging is never skipped; a
## kernel-only node uses `new(kernelConf)` instead.
return await LogosDelivery.new(
LogosDeliveryConf(
kernelConf: kernelConf,
messagingConf: some(messagingOverrides),
channelsConf: some(channelsOverrides),
),
appCallbacks,
)
proc new*(
T: type LogosDelivery,
mode: LogosDeliveryMode = LogosDeliveryMode.Core,
preset: string = "",
messagingOverrides: MessagingClientConf = MessagingClientConf(),
channelsOverrides: ReliableChannelManagerConf = ReliableChannelManagerConf(),
appCallbacks: AppCallbacks = nil,
): Future[Result[LogosDelivery, string]] {.async.} =
## Messaging entry point (app dev). Builds the full stack from preset, mode and overrides.
let conf = LogosDeliveryConf.init(mode, preset, messagingOverrides, channelsOverrides).valueOr:
return err("failed to synthesize configuration: " & error)
return await LogosDelivery.new(conf, appCallbacks)
proc start*(self: LogosDelivery): Future[Result[void, string]] {.async.} =
## Starts each present layer bottom-up: transport, then messaging, then channels.
if self.waku.isNil():
return err("Waku node is not initialized")
(await self.waku.start()).isOkOr:
return err("failed to start Waku: " & error)
if not self.messagingClient.isNil():
self.messagingClient.start().isOkOr:
return err("failed to start MessagingClient: " & error)
if not self.reliableChannelManager.isNil():
self.reliableChannelManager.start().isOkOr:
return err("failed to start ReliableChannelManager: " & error)
return ok()
proc stop*(self: LogosDelivery): Future[Result[void, string]] {.async.} =
## Stops in reverse order so higher layers drain before their dependencies.
if not self.reliableChannelManager.isNil():
await self.reliableChannelManager.stop()
if not self.messagingClient.isNil():
await self.messagingClient.stop()
if not self.waku.isNil():
(await self.waku.stop()).isOkOr:
return err("failed to stop Waku: " & error)
return ok()
proc isOnline*(self: LogosDelivery): Future[Result[bool, string]] {.async.} =
if self.waku.isNil():
return err("Waku node is not initialized")
return await self.waku.isOnline()
proc ensureMessaging*(self: LogosDelivery): Result[void, string] =
## Fails if the node has no messaging client (a kernel-only / fleet node).
if self.isNil() or self.messagingClient.isNil():
return err("node has no messaging client (kernel-only/fleet node)")
ok()
proc ensureChannels*(self: LogosDelivery): Result[void, string] =
## Fails if the node has no reliable channel manager (a kernel-only / fleet node).
if self.isNil() or self.reliableChannelManager.isNil():
return err("node has no reliable channel manager (kernel-only/fleet node)")
ok()
# Compile-time check that each concrete type satisfies its API concept.
static:
doAssert Waku is KernelApi
doAssert MessagingClient is MessagingApi
doAssert ReliableChannelManager is ReliableChannelApi
doAssert LogosDelivery is LogosDeliveryApi

35
logos_delivery/messaging/api/send.nim Normal file
View File

@ -0,0 +1,35 @@
## Messaging layer API — send operation.
import results, chronos, chronicles
import logos_delivery/api/types
import logos_delivery/api/events/messaging_client_events
import logos_delivery/messaging/messaging_client
import logos_delivery/waku/waku
import logos_delivery/waku/api/subscriptions
import logos_delivery/messaging/delivery_service/send_service
import logos_delivery/messaging/delivery_service/send_service/delivery_task
proc send*(
self: MessagingClient, envelope: MessageEnvelope
): Future[Result[RequestId, string]] {.async.} =
## High-level messaging API send. Auto-subscribes to the content topic
## (so the local node sees its own gossipsub broadcast), builds a
## `DeliveryTask`, and hands it to the send service. Returns the request
## id the caller can correlate with `MessageSentEvent` / `MessageErrorEvent`.
?self.checkApiAvailability()
let isSubbed = self.waku.isSubscribed(envelope.contentTopic).valueOr(false)
if not isSubbed:
info "Auto-subscribing to topic on send", contentTopic = envelope.contentTopic
self.waku.subscribe(envelope.contentTopic).isOkOr:
warn "Failed to auto-subscribe", error = error
return err("Failed to auto-subscribe before sending: " & error)
let requestId = RequestId.new(self.waku.rng)
let deliveryTask = DeliveryTask.new(requestId, envelope, self.waku.brokerCtx).valueOr:
return err("MessagingClient.send: Failed to create delivery task: " & error)
asyncSpawn self.sendService.send(deliveryTask)
return ok(requestId)

19
logos_delivery/messaging/api/subscription.nim Normal file
View File

@ -0,0 +1,19 @@
## Messaging layer API — subscription operations.
import results, chronos
import logos_delivery/api/types
import logos_delivery/messaging/messaging_client
import logos_delivery/waku/waku
import logos_delivery/waku/api/subscriptions
proc subscribe*(
self: MessagingClient, contentTopic: ContentTopic
): Future[Result[void, string]] {.async.} =
?self.checkApiAvailability()
return self.waku.subscribe(contentTopic)
proc unsubscribe*(
self: MessagingClient, contentTopic: ContentTopic
): Result[void, string] =
?self.checkApiAvailability()
return self.waku.unsubscribe(contentTopic)

38
logos_delivery/messaging/delivery_service/recv_service/recv_service.nim
View File

@ -7,17 +7,13 @@ import std/[tables, sequtils, options, sets]
import chronos, chronicles, libp2p/utility
import brokers/broker_context
import
logos_delivery/waku/[
waku_core,
waku_core/topics,
waku_store/client,
waku_store/common,
waku_filter_v2/client,
events/message_events,
events/health_events,
waku_node,
node/subscription_manager,
]
logos_delivery/waku/[waku_core, waku_core/topics, waku_store/common],
logos_delivery/waku/waku,
logos_delivery/waku/api/[store, subscriptions]
import
logos_delivery/api/events/kernel_events,
# MessageSeenEvent + EventConnectionStatusChange
logos_delivery/api/events/messaging_client_events # MessageReceivedEvent
const MaxMessageLife = chronos.minutes(7) ## Max time we will keep track of rx messages
@ -36,7 +32,7 @@ type RecvMessage = object
type RecvService* = ref object of RootObj
brokerCtx: BrokerContext
node: WakuNode
waku: Waku
seenMsgListener: MessageSeenEventListener
connStatusListener: EventConnectionStatusChangeListener
@ -58,7 +54,7 @@ proc getMissingMsgsFromStore(
self: RecvService, msgHashes: seq[WakuMessageHash]
): Future[Result[seq[TupleHashAndMsg], string]] {.async.} =
let storeResp: StoreQueryResponse = (
await self.node.wakuStoreClient.queryToAny(
await self.waku.storeQueryToAny(
StoreQueryRequest(includeData: true, messageHashes: msgHashes)
)
).valueOr:
@ -83,9 +79,7 @@ proc processIncomingMessage(
## or if the message is a duplicate (recently-seen). Otherwise, save it as
## recently-seen, emit a MessageReceivedEvent, and return true.
if not self.node.subscriptionManager.isContentSubscribed(
pubsubTopic, message.contentTopic
):
if not self.waku.isContentSubscribed(pubsubTopic, message.contentTopic):
trace "skipping message as I am not subscribed",
shard = pubsubTopic, contentTopic = message.contentTopic
return false
@ -106,16 +100,16 @@ proc processIncomingMessage(
proc checkStore*(self: RecvService) {.async.} =
## Checks the store for messages that were not received directly and
## delivers them via MessageReceivedEvent.
if self.node.wakuStoreClient.isNil():
if not self.waku.isStoreMounted():
error "recv service has no store client mounted, skipping store check"
return
self.endTimeToCheck = getNowInNanosecondTime()
## query store and deliver new recovered messages per subscribed topic
for pubsubTopic, contentTopics in self.node.subscriptionManager.subscribedContentTopics:
for (pubsubTopic, contentTopics) in self.waku.subscribedContentTopics():
let storeResp: StoreQueryResponse = (
await self.node.wakuStoreClient.queryToAny(
await self.waku.storeQueryToAny(
StoreQueryRequest(
includeData: false,
pubsubTopic: some(pubsubTopic),
@ -169,14 +163,14 @@ proc onConnectionStatusChange(self: RecvService, status: ConnectionStatus) =
info "recv service backfilling missed messages after coming back online"
self.backfillHandler = self.checkStore()
proc new*(T: typedesc[RecvService], node: WakuNode): T =
proc new*(T: typedesc[RecvService], waku: Waku): T =
## The storeClient will help to acquire any possible missed messages
let now = getNowInNanosecondTime()
var recvService = RecvService(
node: node,
waku: waku,
startTimeToCheck: now,
brokerCtx: node.brokerCtx,
brokerCtx: waku.brokerCtx,
recentReceivedMsgs: @[],
)

13
logos_delivery/messaging/delivery_service/send_service/delivery_task.nim
View File

@ -3,7 +3,7 @@ import std/[options, times], chronos
import brokers/broker_context
import
logos_delivery/waku/waku_core,
logos_delivery/waku/api/types,
logos_delivery/api/types,
logos_delivery/waku/requests/node_requests
type DeliveryState* {.pure.} = enum
@ -24,6 +24,9 @@ type DeliveryTask* = ref object
tryCount*: int
state*: DeliveryState
deliveryTime*: Moment
firstPropagatedTime*: Option[Moment]
## Set once on the first successful propagation; never reset on re-publish.
## Anchors the store-validation time cap (see propagationAge).
propagateEventEmitted*: bool
errorDesc*: string
@ -74,5 +77,13 @@ proc deliveryAge*(self: DeliveryTask): timer.Duration =
else:
ZeroDuration
proc propagationAge*(self: DeliveryTask): timer.Duration =
## Time elapsed since the message was first successfully propagated.
## Stable across re-publishes; ZeroDuration until first propagation.
if self.firstPropagatedTime.isSome():
timer.Moment.now() - self.firstPropagatedTime.get()
else:
ZeroDuration
proc isEphemeral*(self: DeliveryTask): bool =
return self.msg.ephemeral

34
logos_delivery/messaging/delivery_service/send_service/lightpush_processor.nim
View File

@ -2,10 +2,8 @@ import logos_delivery/waku/compat/option_valueor
import chronicles, chronos, results
import std/options
import brokers/broker_context
import
logos_delivery/waku/node/peer_manager,
logos_delivery/waku/waku_core,
logos_delivery/waku/waku_lightpush/[common, client, rpc]
import logos_delivery/waku/waku_core, logos_delivery/waku/waku
import logos_delivery/waku/api/publish
import ./[delivery_task, send_processor]
@ -13,27 +11,17 @@ logScope:
topics = "send service lightpush processor"
type LightpushSendProcessor* = ref object of BaseSendProcessor
peerManager: PeerManager
lightpushClient: WakuLightPushClient
waku: Waku
proc new*(
T: typedesc[LightpushSendProcessor],
peerManager: PeerManager,
lightpushClient: WakuLightPushClient,
brokerCtx: BrokerContext,
T: typedesc[LightpushSendProcessor], waku: Waku, brokerCtx: BrokerContext
): T =
return
T(peerManager: peerManager, lightpushClient: lightpushClient, brokerCtx: brokerCtx)
proc isLightpushPeerAvailable(
self: LightpushSendProcessor, pubsubTopic: PubsubTopic
): bool =
return self.peerManager.selectPeer(WakuLightPushCodec, some(pubsubTopic)).isSome()
return T(waku: waku, brokerCtx: brokerCtx)
method isValidProcessor*(
self: LightpushSendProcessor, task: DeliveryTask
): bool {.gcsafe.} =
return self.isLightpushPeerAvailable(task.pubsubTopic)
return self.waku.lightpushPeerAvailable(task.pubsubTopic)
method sendImpl*(
self: LightpushSendProcessor, task: DeliveryTask
@ -44,14 +32,8 @@ method sendImpl*(
msgHash = task.msgHash.to0xHex(),
tryCount = task.tryCount
let peer = self.peerManager.selectPeer(WakuLightPushCodec, some(task.pubsubTopic)).valueOr:
debug "No peer available for Lightpush, request pushed back for next round",
requestId = task.requestId
task.state = DeliveryState.NextRoundRetry
return
let numLightpushServers = (
await self.lightpushClient.publish(some(task.pubsubTopic), task.msg, peer)
await self.waku.lightpushPublishToAny(task.pubsubTopic, task.msg)
).valueOr:
error "LightpushSendProcessor.sendImpl failed", error = error.desc.get($error.code)
case error.code
@ -71,6 +53,8 @@ method sendImpl*(
requestId = task.requestId, msgHash = task.msgHash.to0xHex()
task.state = DeliveryState.SuccessfullyPropagated
task.deliveryTime = Moment.now()
if task.firstPropagatedTime.isNone():
task.firstPropagatedTime = some(Moment.now())
# TODO: with a simple retry processor it might be more accurate to say `Sent`
else:
# Controversial state, publish says ok but no peer. It should not happen.

4
logos_delivery/messaging/delivery_service/send_service/relay_processor.nim
View File

@ -4,7 +4,7 @@ import chronos, chronicles
import brokers/broker_context
import logos_delivery/waku/[waku_core], logos_delivery/waku/waku_lightpush/[common, rpc]
import logos_delivery/waku/requests/health_requests
import logos_delivery/waku/api/types
import logos_delivery/api/types
import ./[delivery_task, send_processor]
logScope:
@ -76,6 +76,8 @@ method sendImpl*(self: RelaySendProcessor, task: DeliveryTask) {.async.} =
noOfPeers = noOfPublishedPeers
task.state = DeliveryState.SuccessfullyPropagated
task.deliveryTime = Moment.now()
if task.firstPropagatedTime.isNone():
task.firstPropagatedTime = some(Moment.now())
else:
# It shall not happen, but still covering it
task.state = self.fallbackStateToSet

7
logos_delivery/messaging/delivery_service/send_service/send_processor.nim
View File

@ -32,5 +32,10 @@ method process*(
currentProcessor = currentProcessor.fallbackProcessor
keepTrying = task.state == DeliveryState.FallbackRetry
if task.state == DeliveryState.FallbackRetry:
# A task still in `FallbackRetry` exhausted the chain without delivering, and
# one still in `Entry` was never attempted because no processor had a usable
# peer yet (e.g. a lightpush peer that finishes registering right after the
# first send). Both must be queued for the next round so the service loop
# retries them; otherwise the task would sit untouched until it ages out.
if task.state == DeliveryState.FallbackRetry or task.state == DeliveryState.Entry:
task.state = DeliveryState.NextRoundRetry

98
logos_delivery/messaging/delivery_service/send_service/send_service.nim
View File

@ -7,19 +7,10 @@ import chronos, chronicles, libp2p/utility
import brokers/broker_context
import
./[send_processor, relay_processor, lightpush_processor, delivery_task],
logos_delivery/waku/[
waku_core,
node/waku_node,
node/subscription_manager,
node/peer_manager,
waku_store/client,
waku_store/common,
waku_relay/protocol,
waku_rln_relay/rln_relay,
waku_lightpush/client,
waku_lightpush/callbacks,
events/message_events,
]
logos_delivery/waku/[waku_core, waku_store/common],
logos_delivery/waku/waku,
logos_delivery/waku/api/[store, subscriptions, publish]
import logos_delivery/api/events/messaging_client_events
logScope:
topics = "send service"
@ -57,18 +48,15 @@ type SendService* = ref object of RootObj
serviceLoopHandle: Future[void] ## handle that allows to stop the async task
sendProcessor: BaseSendProcessor
node: WakuNode
waku: Waku
checkStoreForMessages: bool
lastStoreCheckTime: Moment ## throttles store validation queries to ArchiveTime cadence
proc setupSendProcessorChain(
peerManager: PeerManager,
lightpushClient: WakuLightPushClient,
relay: WakuRelay,
rlnRelay: WakuRLNRelay,
brokerCtx: BrokerContext,
waku: Waku, brokerCtx: BrokerContext
): Result[BaseSendProcessor, string] =
let isRelayAvail = not relay.isNil()
let isLightPushAvail = not lightpushClient.isNil()
let isRelayAvail = waku.hasRelay()
let isLightPushAvail = waku.hasLightpush()
if not isRelayAvail and not isLightPushAvail:
return err("No valid send processor found for the delivery task")
@ -76,16 +64,10 @@ proc setupSendProcessorChain(
var processors = newSeq[BaseSendProcessor]()
if isRelayAvail:
let rln: Option[WakuRLNRelay] =
if rlnRelay.isNil():
none[WakuRLNRelay]()
else:
some(rlnRelay)
let publishProc = getRelayPushHandler(relay, rln)
let publishProc = waku.relayPushHandler()
processors.add(RelaySendProcessor.new(isLightPushAvail, publishProc, brokerCtx))
if isLightPushAvail:
processors.add(LightpushSendProcessor.new(peerManager, lightpushClient, brokerCtx))
processors.add(LightpushSendProcessor.new(waku, brokerCtx))
var currentProcessor: BaseSendProcessor = processors[0]
for i in 1 ..< processors.len:
@ -96,27 +78,26 @@ proc setupSendProcessorChain(
return ok(processors[0])
proc new*(
T: typedesc[SendService], preferP2PReliability: bool, w: WakuNode
T: typedesc[SendService], preferP2PReliability: bool, waku: Waku
): Result[T, string] =
if w.wakuRelay.isNil() and w.wakuLightpushClient.isNil():
if not waku.hasRelay() and not waku.hasLightpush():
return err(
"Could not create SendService. wakuRelay or wakuLightpushClient should be set"
)
let checkStoreForMessages = preferP2PReliability and not w.wakuStoreClient.isNil()
let checkStoreForMessages = preferP2PReliability and waku.isStoreMounted()
let sendProcessorChain = setupSendProcessorChain(
w.peerManager, w.wakuLightPushClient, w.wakuRelay, w.wakuRlnRelay, w.brokerCtx
).valueOr:
let sendProcessorChain = setupSendProcessorChain(waku, waku.brokerCtx).valueOr:
return err("failed to setup SendProcessorChain: " & $error)
let sendService = SendService(
brokerCtx: w.brokerCtx,
brokerCtx: waku.brokerCtx,
taskCache: newSeq[DeliveryTask](),
serviceLoopHandle: nil,
sendProcessor: sendProcessorChain,
node: w,
waku: waku,
checkStoreForMessages: checkStoreForMessages,
lastStoreCheckTime: Moment.now(),
)
return ok(sendService)
@ -125,7 +106,7 @@ proc addTask(self: SendService, task: DeliveryTask) =
self.taskCache.addUnique(task)
proc isStorePeerAvailable*(sendService: SendService): bool =
return sendService.node.peerManager.selectPeer(WakuStoreCodec).isSome()
return sendService.waku.hasStorePeer()
proc checkMsgsInStore(self: SendService, tasksToValidate: seq[DeliveryTask]) {.async.} =
if tasksToValidate.len() == 0:
@ -140,7 +121,7 @@ proc checkMsgsInStore(self: SendService, tasksToValidate: seq[DeliveryTask]) {.a
# TODO: confirm hash format for store query!!!
let storeResp: StoreQueryResponse = (
await self.node.wakuStoreClient.queryToAny(
await self.waku.storeQueryToAny(
StoreQueryRequest(includeData: false, messageHashes: hashesToValidate)
)
).valueOr:
@ -163,11 +144,20 @@ proc checkStoredMessages(self: SendService) {.async.} =
if not self.checkStoreForMessages:
return
# Throttle store queries so they run at most every ArchiveTime (3s), regardless
# of the 1s service loop cadence.
if Moment.now() - self.lastStoreCheckTime < ArchiveTime:
return
let tasksToValidate = self.taskCache.filterIt(
it.state == DeliveryState.SuccessfullyPropagated and it.deliveryAge() > ArchiveTime and
not it.isEphemeral()
it.state == DeliveryState.SuccessfullyPropagated and
it.propagationAge() > ArchiveTime and not it.isEphemeral()
)
if tasksToValidate.len() == 0:
return
self.lastStoreCheckTime = Moment.now()
await self.checkMsgsInStore(tasksToValidate)
proc reportTaskResult(self: SendService, task: DeliveryTask) =
@ -200,7 +190,10 @@ proc reportTaskResult(self: SendService, task: DeliveryTask) =
# rest of the states are intermediate and does not translate to event
discard
if task.messageAge() > MaxTimeInCache:
# Only tasks that never propagated are reported as hard send failures here.
# Propagated-but-not-store-validated tasks are handled (warn + drop, no event)
# in evaluateAndCleanUp.
if task.firstPropagatedTime.isNone() and task.messageAge() > MaxTimeInCache:
error "Failed to send message",
requestId = task.requestId,
msgHash = task.msgHash.to0xHex(),
@ -229,6 +222,25 @@ proc evaluateAndCleanUp(self: SendService) =
)
)
# Store validation timed out: the message was propagated but never confirmed in a
# store node within MaxTimeInCache (measured from first propagation). Warn and drop
# without emitting an app event.
for task in self.taskCache:
if task.firstPropagatedTime.isSome() and
task.state != DeliveryState.SuccessfullyValidated and
task.propagationAge() > MaxTimeInCache:
warn "Message propagated but not validated by a store node within time window; stop trying.",
requestId = task.requestId,
msgHash = task.msgHash.to0xHex(),
propagationAge = task.propagationAge()
self.taskCache.keepItIf(
not (
it.firstPropagatedTime.isSome() and it.state != DeliveryState.SuccessfullyValidated and
it.propagationAge() > MaxTimeInCache
)
)
proc trySendMessages(self: SendService) {.async.} =
let tasksToSend = self.taskCache.filterIt(it.state == DeliveryState.NextRoundRetry)
@ -259,7 +271,7 @@ proc send*(self: SendService, task: DeliveryTask) {.async.} =
info "SendService.send: processing delivery task",
requestId = task.requestId, msgHash = task.msgHash.to0xHex()
self.node.subscriptionManager.subscribe(task.msg.contentTopic).isOkOr:
self.waku.subscribe(task.msg.contentTopic).isOkOr:
error "SendService.send: failed to subscribe to content topic",
contentTopic = task.msg.contentTopic, error = error

83
logos_delivery/messaging/messaging_client.nim
View File

@ -1,59 +1,44 @@
import results, chronos
import chronicles
## Messaging layer core: the `MessagingClient` type plus its construction and
## lifecycle. The public operations (subscribe / unsubscribe / send) live in
## `messaging/api.nim`.
import std/options
import results, chronos, chronicles
import
logos_delivery/waku/api/types,
logos_delivery/waku/node/[waku_node, subscription_manager],
logos_delivery/messaging/delivery_service/[recv_service, send_service],
logos_delivery/messaging/delivery_service/send_service/delivery_task
logos_delivery/api/conf/messaging_conf,
logos_delivery/api/messaging_client_api,
logos_delivery/waku/waku,
logos_delivery/waku/factory/conf_builder/waku_conf_builder,
logos_delivery/messaging/delivery_service/[recv_service, send_service]
export messaging_client_api, messaging_conf
type MessagingClient* = ref object
node: WakuNode
brokerCtx*: BrokerContext
waku*: Waku ## The Waku kernel this layer drives; read by `messaging/api/*`.
sendService*: SendService
recvService*: RecvService
started: bool
started*: bool
proc new*(
T: type MessagingClient, useP2PReliability: bool, node: WakuNode
T: type MessagingClient, conf: MessagingClientConf, waku: Waku
): Result[T, string] =
let sendService = ?SendService.new(useP2PReliability, node)
let recvService = RecvService.new(node)
ok(T(node: node, sendService: sendService, recvService: recvService))
## The messaging layer chains onto Waku: it drives the underlying Waku kernel
## for transport while exposing its own send/recv API.
let reliability = conf.reliabilityEnabled.get(DefaultP2pReliability)
let sendService = ?SendService.new(reliability, waku)
let recvService = RecvService.new(waku)
return ok(
T(
waku: waku,
sendService: sendService,
recvService: recvService,
brokerCtx: waku.brokerCtx,
)
)
proc start*(self: MessagingClient): Result[void, string] =
if self.started:
return ok()
self.recvService.startRecvService()
self.sendService.startSendService()
self.started = true
ok()
proc checkApiAvailability*(self: MessagingClient): Result[void, string] =
## Shared guard for the api operation module.
if self.isNil():
return err("MessagingClient is not initialized")
proc stop*(self: MessagingClient) {.async.} =
if not self.started:
return
await self.sendService.stopSendService()
await self.recvService.stopRecvService()
self.started = false
proc send*(
self: MessagingClient, envelope: MessageEnvelope
): Future[Result[RequestId, string]] {.async.} =
## High-level messaging API send. Auto-subscribes to the content topic
## (so the local node sees its own gossipsub broadcast), builds a
## `DeliveryTask`, and hands it to the send service. Returns the request
## id the caller can correlate with `MessageSentEvent` / `MessageErrorEvent`.
let isSubbed =
self.node.subscriptionManager.isSubscribed(envelope.contentTopic).valueOr(false)
if not isSubbed:
info "Auto-subscribing to topic on send", contentTopic = envelope.contentTopic
self.node.subscriptionManager.subscribe(envelope.contentTopic).isOkOr:
warn "Failed to auto-subscribe", error = error
return err("Failed to auto-subscribe before sending: " & error)
let requestId = RequestId.new(self.node.rng)
let deliveryTask = DeliveryTask.new(requestId, envelope, self.node.brokerCtx).valueOr:
return err("MessagingClient.send: Failed to create delivery task: " & error)
asyncSpawn self.sendService.send(deliveryTask)
return ok(requestId)
return ok()

37
logos_delivery/messaging/messaging_client_lifecycle.nim Normal file
View File

@ -0,0 +1,37 @@
## Messaging layer core: the `MessagingClient` type plus its construction and
## lifecycle. The public operations (subscribe / unsubscribe / send) live in
## `messaging/api.nim`.
import results, chronos
import chronicles
import
logos_delivery/messaging/messaging_client,
logos_delivery/messaging/api/send,
logos_delivery/api/messaging_client_api,
logos_delivery/waku/waku,
logos_delivery/messaging/delivery_service/[recv_service, send_service]
# Surfaces the messaging API interface (and its Message* events) to consumers.
export messaging_client
proc start*(self: MessagingClient): Result[void, string] =
if self.started:
return ok()
self.recvService.startRecvService()
self.sendService.startSendService()
?MessagingSend.setProvider(
self.brokerCtx,
proc(envelope: MessageEnvelope): Future[Result[RequestId, string]] {.async.} =
return await self.send(envelope),
)
self.started = true
ok()
proc stop*(self: MessagingClient) {.async.} =
if not self.started:
return
MessagingSend.clearProvider(self.brokerCtx)
await self.sendService.stopSendService()
await self.recvService.stopRecvService()
self.started = false

16
logos_delivery/waku/README.md
View File

@ -232,4 +232,20 @@ However, they can be used for local testing purposes:
mkdir -p ./ssl_dir/
openssl req -x509 -newkey rsa:4096 -keyout ./ssl_dir/key.pem -out ./ssl_dir/cert.pem -sha256 -nodes
wakunode2 --websocket-secure-support=true --websocket-secure-key-path="./ssl_dir/key.pem" --websocket-secure-cert-path="./ssl_dir/cert.pem"
```
## Enabling QUIC
QUIC is a UDP-based transport that peers can use to connect to your node.
The default port for QUIC is 60000.
```shell
wakunode2 --quic-support=true
```
To listen on a different UDP port, use `--quic-port`:
```shell
wakunode2 --quic-support=true --quic-port=<port>
```

5
logos_delivery/waku/api.nim
View File

@ -1,5 +0,0 @@
import ./api/[api, api_conf]
import ./events/message_events
import tools/confutils/entry_nodes
export api, api_conf, entry_nodes, message_events

56
logos_delivery/waku/api/api.nim
View File

@ -1,56 +0,0 @@
import logos_delivery/waku/compat/option_valueor
import std/[net, options]
import chronicles, chronos, libp2p/peerid, results
import logos_delivery/waku/factory/waku
import logos_delivery/messaging/messaging_client
import logos_delivery/waku/[requests/health_requests, waku_core, waku_node]
import logos_delivery/messaging/delivery_service/send_service
import logos_delivery/waku/node/subscription_manager
import libp2p/peerid
import tools/confutils/cli_args
import ./[api_conf, types]
export cli_args
logScope:
topics = "api"
proc createNode*(conf: WakuNodeConf): Future[Result[Waku, string]] {.async.} =
let wakuConf = conf.toWakuConf().valueOr:
return err("Failed to handle the configuration: " & error)
## We are not defining app callbacks at node creation
let wakuRes = (await Waku.new(wakuConf)).valueOr:
error "waku initialization failed", error = error
return err("Failed setting up Waku: " & $error)
return ok(wakuRes)
proc checkApiAvailability(w: Waku): Result[void, string] =
if w.isNil():
return err("Waku node is not initialized")
# TODO: Conciliate request-bouncing health checks here with unit testing.
# (For now, better to just allow all sends and rely on retries.)
return ok()
proc subscribe*(
w: Waku, contentTopic: ContentTopic
): Future[Result[void, string]] {.async.} =
?checkApiAvailability(w)
return w.node.subscriptionManager.subscribe(contentTopic)
proc unsubscribe*(w: Waku, contentTopic: ContentTopic): Result[void, string] =
?checkApiAvailability(w)
return w.node.subscriptionManager.unsubscribe(contentTopic)
proc send*(
w: Waku, envelope: MessageEnvelope
): Future[Result[RequestId, string]] {.async.} =
?checkApiAvailability(w)
return await w.messagingClient.send(envelope)

534
logos_delivery/waku/api/api_conf.nim
View File

@ -1,534 +0,0 @@
import logos_delivery/waku/compat/option_valueor
import std/[net, options]
import results
import json_serialization, json_serialization/std/options as json_options
import
logos_delivery/waku/common/utils/parse_size_units,
logos_delivery/waku/common/logging,
logos_delivery/waku/factory/waku_conf,
logos_delivery/waku/factory/conf_builder/conf_builder,
logos_delivery/waku/factory/networks_config,
tools/confutils/entry_nodes
export json_serialization, json_options
type AutoShardingConfig* = object
numShardsInCluster*: uint16
type RlnConfig* = object
contractAddress*: string
chainId*: uint
epochSizeSec*: uint64
type NetworkingConfig* = object
listenIpv4*: string
p2pTcpPort*: uint16
discv5UdpPort*: uint16
type MessageValidation* = object
maxMessageSize*: string # Accepts formats like "150 KiB", "1500 B"
rlnConfig*: Option[RlnConfig]
type ProtocolsConfig* = object
entryNodes: seq[string]
staticStoreNodes: seq[string]
clusterId: uint16
autoShardingConfig: AutoShardingConfig
messageValidation: MessageValidation
const DefaultNetworkingConfig* =
NetworkingConfig(listenIpv4: "0.0.0.0", p2pTcpPort: 60000, discv5UdpPort: 9000)
const DefaultAutoShardingConfig* = AutoShardingConfig(numShardsInCluster: 1)
const DefaultMessageValidation* =
MessageValidation(maxMessageSize: "150 KiB", rlnConfig: none(RlnConfig))
proc init*(
T: typedesc[ProtocolsConfig],
entryNodes: seq[string],
staticStoreNodes: seq[string] = @[],
clusterId: uint16,
autoShardingConfig: AutoShardingConfig = DefaultAutoShardingConfig,
messageValidation: MessageValidation = DefaultMessageValidation,
): T =
return T(
entryNodes: entryNodes,
staticStoreNodes: staticStoreNodes,
clusterId: clusterId,
autoShardingConfig: autoShardingConfig,
messageValidation: messageValidation,
)
const TheWakuNetworkPreset* = ProtocolsConfig(
entryNodes: @[
"enrtree://AIRVQ5DDA4FFWLRBCHJWUWOO6X6S4ZTZ5B667LQ6AJU6PEYDLRD5O@sandbox.waku.nodes.status.im"
],
staticStoreNodes: @[],
clusterId: 1,
autoShardingConfig: AutoShardingConfig(numShardsInCluster: 8),
messageValidation: MessageValidation(
maxMessageSize: "150 KiB",
rlnConfig: some(
RlnConfig(
contractAddress: "0xB9cd878C90E49F797B4431fBF4fb333108CB90e6",
chainId: 59141,
epochSizeSec: 600, # 10 minutes
)
),
),
)
type WakuMode* {.pure.} = enum
Edge
Core
type NodeConfig* {.
requiresInit, deprecated: "Use WakuNodeConf from tools/confutils/cli_args instead"
.} = object
mode: WakuMode
protocolsConfig: ProtocolsConfig
networkingConfig: NetworkingConfig
ethRpcEndpoints: seq[string]
p2pReliability: bool
logLevel: LogLevel
logFormat: LogFormat
proc init*(
T: typedesc[NodeConfig],
mode: WakuMode = WakuMode.Core,
protocolsConfig: ProtocolsConfig = TheWakuNetworkPreset,
networkingConfig: NetworkingConfig = DefaultNetworkingConfig,
ethRpcEndpoints: seq[string] = @[],
p2pReliability: bool = false,
logLevel: LogLevel = LogLevel.INFO,
logFormat: LogFormat = LogFormat.TEXT,
): T =
return T(
mode: mode,
protocolsConfig: protocolsConfig,
networkingConfig: networkingConfig,
ethRpcEndpoints: ethRpcEndpoints,
p2pReliability: p2pReliability,
logLevel: logLevel,
logFormat: logFormat,
)
# -- Getters for ProtocolsConfig (private fields) - used for testing --
proc entryNodes*(c: ProtocolsConfig): seq[string] =
c.entryNodes
proc staticStoreNodes*(c: ProtocolsConfig): seq[string] =
c.staticStoreNodes
proc clusterId*(c: ProtocolsConfig): uint16 =
c.clusterId
proc autoShardingConfig*(c: ProtocolsConfig): AutoShardingConfig =
c.autoShardingConfig
proc messageValidation*(c: ProtocolsConfig): MessageValidation =
c.messageValidation
# -- Getters for NodeConfig (private fields) - used for testing --
proc mode*(c: NodeConfig): WakuMode =
c.mode
proc protocolsConfig*(c: NodeConfig): ProtocolsConfig =
c.protocolsConfig
proc networkingConfig*(c: NodeConfig): NetworkingConfig =
c.networkingConfig
proc ethRpcEndpoints*(c: NodeConfig): seq[string] =
c.ethRpcEndpoints
proc p2pReliability*(c: NodeConfig): bool =
c.p2pReliability
proc logLevel*(c: NodeConfig): LogLevel =
c.logLevel
proc logFormat*(c: NodeConfig): LogFormat =
c.logFormat
proc toWakuConf*(
nodeConfig: NodeConfig
): Result[WakuConf, string] {.deprecated: "Use WakuNodeConf.toWakuConf instead".} =
var b = WakuConfBuilder.init()
# Apply log configuration
b.withLogLevel(nodeConfig.logLevel)
b.withLogFormat(nodeConfig.logFormat)
# Apply networking configuration
let networkingConfig = nodeConfig.networkingConfig
let ip = parseIpAddress(networkingConfig.listenIpv4)
b.withP2pListenAddress(ip)
b.withP2pTcpPort(networkingConfig.p2pTcpPort)
b.discv5Conf.withUdpPort(networkingConfig.discv5UdpPort)
case nodeConfig.mode
of Core:
b.withRelay(true)
# Metadata is always mounted
b.filterServiceConf.withEnabled(true)
b.filterServiceConf.withMaxPeersToServe(20)
b.withLightPush(true)
b.discv5Conf.withEnabled(true)
b.withPeerExchange(true)
b.withRendezvous(true)
# TODO: fix store as client usage
b.rateLimitConf.withRateLimits(@["filter:100/1s", "lightpush:5/1s", "px:5/1s"])
of Edge:
# All client side protocols are mounted by default
# Peer exchange client is always enabled and start_node will start the px loop
# Metadata is always mounted
b.withPeerExchange(true)
# switch off all service side protocols and relay
b.withRelay(false)
b.filterServiceConf.withEnabled(false)
b.withLightPush(false)
b.storeServiceConf.withEnabled(false)
# Leave discv5 and rendezvous for user choice
## Network Conf
let protocolsConfig = nodeConfig.protocolsConfig
# Set cluster ID
b.withClusterId(protocolsConfig.clusterId)
# Set sharding configuration
b.withShardingConf(ShardingConfKind.AutoSharding)
let autoShardingConfig = protocolsConfig.autoShardingConfig
b.withNumShardsInCluster(autoShardingConfig.numShardsInCluster)
# Process entry nodes - supports enrtree:, enr:, and multiaddress formats
if protocolsConfig.entryNodes.len > 0:
let (enrTreeUrls, bootstrapEnrs, staticNodesFromEntry) = processEntryNodes(
protocolsConfig.entryNodes
).valueOr:
return err("Failed to process entry nodes: " & error)
# Set ENRTree URLs for DNS discovery
if enrTreeUrls.len > 0:
for url in enrTreeUrls:
b.dnsDiscoveryConf.withEnrTreeUrl(url)
b.dnsDiscoveryconf.withNameServers(
@[parseIpAddress("1.1.1.1"), parseIpAddress("1.0.0.1")]
)
# Set ENR records as bootstrap nodes for discv5
if bootstrapEnrs.len > 0:
b.discv5Conf.withBootstrapNodes(bootstrapEnrs)
# Add static nodes (multiaddrs and those extracted from ENR entries)
if staticNodesFromEntry.len > 0:
b.withStaticNodes(staticNodesFromEntry)
# TODO: verify behaviour
# Set static store nodes
if protocolsConfig.staticStoreNodes.len > 0:
b.withStaticNodes(protocolsConfig.staticStoreNodes)
# Set message validation
let msgValidation = protocolsConfig.messageValidation
let maxSizeBytes = parseMsgSize(msgValidation.maxMessageSize).valueOr:
return err("Failed to parse max message size: " & error)
b.withMaxMessageSize(maxSizeBytes)
# Set RLN config if provided
if msgValidation.rlnConfig.isSome():
let rlnConfig = msgValidation.rlnConfig.get()
b.rlnRelayConf.withEnabled(true)
b.rlnRelayConf.withEthContractAddress(rlnConfig.contractAddress)
b.rlnRelayConf.withChainId(rlnConfig.chainId)
b.rlnRelayConf.withEpochSizeSec(rlnConfig.epochSizeSec)
b.rlnRelayConf.withDynamic(true)
b.rlnRelayConf.withEthClientUrls(nodeConfig.ethRpcEndpoints)
# TODO: we should get rid of those two
b.rlnRelayconf.withUserMessageLimit(100)
## Various configurations
b.withNatStrategy("any")
b.withP2PReliability(nodeConfig.p2pReliability)
let wakuConf = b.build().valueOr:
return err("Failed to build configuration: " & error)
wakuConf.validate().isOkOr:
return err("Failed to validate configuration: " & error)
return ok(wakuConf)
# ---- JSON serialization (writeValue / readValue) ----
# ---------- AutoShardingConfig ----------
proc writeValue*(w: var JsonWriter, val: AutoShardingConfig) {.raises: [IOError].} =
w.beginRecord()
w.writeField("numShardsInCluster", val.numShardsInCluster)
w.endRecord()
proc readValue*(
r: var JsonReader, val: var AutoShardingConfig
) {.raises: [SerializationError, IOError].} =
var numShardsInCluster: Option[uint16]
for fieldName in readObjectFields(r):
case fieldName
of "numShardsInCluster":
numShardsInCluster = some(r.readValue(uint16))
else:
r.raiseUnexpectedField(fieldName, "AutoShardingConfig")
if numShardsInCluster.isNone():
r.raiseUnexpectedValue("Missing required field 'numShardsInCluster'")
val = AutoShardingConfig(numShardsInCluster: numShardsInCluster.get())
# ---------- RlnConfig ----------
proc writeValue*(w: var JsonWriter, val: RlnConfig) {.raises: [IOError].} =
w.beginRecord()
w.writeField("contractAddress", val.contractAddress)
w.writeField("chainId", val.chainId)
w.writeField("epochSizeSec", val.epochSizeSec)
w.endRecord()
proc readValue*(
r: var JsonReader, val: var RlnConfig
) {.raises: [SerializationError, IOError].} =
var
contractAddress: Option[string]
chainId: Option[uint]
epochSizeSec: Option[uint64]
for fieldName in readObjectFields(r):
case fieldName
of "contractAddress":
contractAddress = some(r.readValue(string))
of "chainId":
chainId = some(r.readValue(uint))
of "epochSizeSec":
epochSizeSec = some(r.readValue(uint64))
else:
r.raiseUnexpectedField(fieldName, "RlnConfig")
if contractAddress.isNone():
r.raiseUnexpectedValue("Missing required field 'contractAddress'")
if chainId.isNone():
r.raiseUnexpectedValue("Missing required field 'chainId'")
if epochSizeSec.isNone():
r.raiseUnexpectedValue("Missing required field 'epochSizeSec'")
val = RlnConfig(
contractAddress: contractAddress.get(),
chainId: chainId.get(),
epochSizeSec: epochSizeSec.get(),
)
# ---------- NetworkingConfig ----------
proc writeValue*(w: var JsonWriter, val: NetworkingConfig) {.raises: [IOError].} =
w.beginRecord()
w.writeField("listenIpv4", val.listenIpv4)
w.writeField("p2pTcpPort", val.p2pTcpPort)
w.writeField("discv5UdpPort", val.discv5UdpPort)
w.endRecord()
proc readValue*(
r: var JsonReader, val: var NetworkingConfig
) {.raises: [SerializationError, IOError].} =
var
listenIpv4: Option[string]
p2pTcpPort: Option[uint16]
discv5UdpPort: Option[uint16]
for fieldName in readObjectFields(r):
case fieldName
of "listenIpv4":
listenIpv4 = some(r.readValue(string))
of "p2pTcpPort":
p2pTcpPort = some(r.readValue(uint16))
of "discv5UdpPort":
discv5UdpPort = some(r.readValue(uint16))
else:
r.raiseUnexpectedField(fieldName, "NetworkingConfig")
if listenIpv4.isNone():
r.raiseUnexpectedValue("Missing required field 'listenIpv4'")
if p2pTcpPort.isNone():
r.raiseUnexpectedValue("Missing required field 'p2pTcpPort'")
if discv5UdpPort.isNone():
r.raiseUnexpectedValue("Missing required field 'discv5UdpPort'")
val = NetworkingConfig(
listenIpv4: listenIpv4.get(),
p2pTcpPort: p2pTcpPort.get(),
discv5UdpPort: discv5UdpPort.get(),
)
# ---------- MessageValidation ----------
proc writeValue*(w: var JsonWriter, val: MessageValidation) {.raises: [IOError].} =
w.beginRecord()
w.writeField("maxMessageSize", val.maxMessageSize)
w.writeField("rlnConfig", val.rlnConfig)
w.endRecord()
proc readValue*(
r: var JsonReader, val: var MessageValidation
) {.raises: [SerializationError, IOError].} =
var
maxMessageSize: Option[string]
rlnConfig: Option[Option[RlnConfig]]
for fieldName in readObjectFields(r):
case fieldName
of "maxMessageSize":
maxMessageSize = some(r.readValue(string))
of "rlnConfig":
rlnConfig = some(r.readValue(Option[RlnConfig]))
else:
r.raiseUnexpectedField(fieldName, "MessageValidation")
if maxMessageSize.isNone():
r.raiseUnexpectedValue("Missing required field 'maxMessageSize'")
val = MessageValidation(
maxMessageSize: maxMessageSize.get(), rlnConfig: rlnConfig.get(none(RlnConfig))
)
# ---------- ProtocolsConfig ----------
proc writeValue*(w: var JsonWriter, val: ProtocolsConfig) {.raises: [IOError].} =
w.beginRecord()
w.writeField("entryNodes", val.entryNodes)
w.writeField("staticStoreNodes", val.staticStoreNodes)
w.writeField("clusterId", val.clusterId)
w.writeField("autoShardingConfig", val.autoShardingConfig)
w.writeField("messageValidation", val.messageValidation)
w.endRecord()
proc readValue*(
r: var JsonReader, val: var ProtocolsConfig
) {.raises: [SerializationError, IOError].} =
var
entryNodes: Option[seq[string]]
staticStoreNodes: Option[seq[string]]
clusterId: Option[uint16]
autoShardingConfig: Option[AutoShardingConfig]
messageValidation: Option[MessageValidation]
for fieldName in readObjectFields(r):
case fieldName
of "entryNodes":
entryNodes = some(r.readValue(seq[string]))
of "staticStoreNodes":
staticStoreNodes = some(r.readValue(seq[string]))
of "clusterId":
clusterId = some(r.readValue(uint16))
of "autoShardingConfig":
autoShardingConfig = some(r.readValue(AutoShardingConfig))
of "messageValidation":
messageValidation = some(r.readValue(MessageValidation))
else:
r.raiseUnexpectedField(fieldName, "ProtocolsConfig")
if entryNodes.isNone():
r.raiseUnexpectedValue("Missing required field 'entryNodes'")
if clusterId.isNone():
r.raiseUnexpectedValue("Missing required field 'clusterId'")
val = ProtocolsConfig.init(
entryNodes = entryNodes.get(),
staticStoreNodes = staticStoreNodes.get(@[]),
clusterId = clusterId.get(),
autoShardingConfig = autoShardingConfig.get(DefaultAutoShardingConfig),
messageValidation = messageValidation.get(DefaultMessageValidation),
)
# ---------- NodeConfig ----------
proc writeValue*(w: var JsonWriter, val: NodeConfig) {.raises: [IOError].} =
w.beginRecord()
w.writeField("mode", val.mode)
w.writeField("protocolsConfig", val.protocolsConfig)
w.writeField("networkingConfig", val.networkingConfig)
w.writeField("ethRpcEndpoints", val.ethRpcEndpoints)
w.writeField("p2pReliability", val.p2pReliability)
w.writeField("logLevel", val.logLevel)
w.writeField("logFormat", val.logFormat)
w.endRecord()
proc readValue*(
r: var JsonReader, val: var NodeConfig
) {.raises: [SerializationError, IOError].} =
var
mode: Option[WakuMode]
protocolsConfig: Option[ProtocolsConfig]
networkingConfig: Option[NetworkingConfig]
ethRpcEndpoints: Option[seq[string]]
p2pReliability: Option[bool]
logLevel: Option[LogLevel]
logFormat: Option[LogFormat]
for fieldName in readObjectFields(r):
case fieldName
of "mode":
mode = some(r.readValue(WakuMode))
of "protocolsConfig":
protocolsConfig = some(r.readValue(ProtocolsConfig))
of "networkingConfig":
networkingConfig = some(r.readValue(NetworkingConfig))
of "ethRpcEndpoints":
ethRpcEndpoints = some(r.readValue(seq[string]))
of "p2pReliability":
p2pReliability = some(r.readValue(bool))
of "logLevel":
logLevel = some(r.readValue(LogLevel))
of "logFormat":
logFormat = some(r.readValue(LogFormat))
else:
r.raiseUnexpectedField(fieldName, "NodeConfig")
val = NodeConfig.init(
mode = mode.get(WakuMode.Core),
protocolsConfig = protocolsConfig.get(TheWakuNetworkPreset),
networkingConfig = networkingConfig.get(DefaultNetworkingConfig),
ethRpcEndpoints = ethRpcEndpoints.get(@[]),
p2pReliability = p2pReliability.get(false),
logLevel = logLevel.get(LogLevel.INFO),
logFormat = logFormat.get(LogFormat.TEXT),
)
# ---------- Decode helper ----------
# Json.decode returns T via `result`, which conflicts with {.requiresInit.}
# on Nim 2.x. This helper avoids the issue by using readValue into a var.
proc decodeNodeConfigFromJson*(
jsonStr: string
): NodeConfig {.
raises: [SerializationError],
deprecated: "Use WakuNodeConf with fieldPairs-based JSON parsing instead"
.} =
var val = NodeConfig.init() # default-initialized
try:
var stream = unsafeMemoryInput(jsonStr)
var reader = (JsonReader[DefaultFlavor].init(stream))
reader.readValue(val)
except IOError as err:
raise (ref SerializationError)(msg: err.msg)
return val

36
logos_delivery/waku/api/debug.nim Normal file
View File

@ -0,0 +1,36 @@
## Waku layer API — debug / info operations.
{.push raises: [].}
import results, chronos, chronicles, metrics
import eth/p2p/discoveryv5/enr
import logos_delivery/waku/waku
import logos_delivery/waku/[waku_core, node/waku_node]
proc version*(self: Waku): Future[Result[string, string]] {.async.} =
return ok(WakuNodeVersionString)
proc listenAddresses*(self: Waku): Future[Result[seq[string], string]] {.async.} =
try:
return ok(self.node.info().listenAddresses)
except CatchableError as e:
return err(e.msg)
proc myEnr*(self: Waku): Future[Result[string, string]] {.async.} =
try:
return ok(self.node.enr.toURI())
except CatchableError as e:
return err(e.msg)
proc myPeerId*(self: Waku): Future[Result[string, string]] {.async.} =
try:
return ok($self.node.peerId())
except CatchableError as e:
return err(e.msg)
proc metrics*(self: Waku): Future[Result[string, string]] {.async.} =
{.gcsafe.}:
try:
return ok(defaultRegistry.toText())
except CatchableError as e:
return err(e.msg)

76
logos_delivery/waku/api/discovery.nim Normal file
View File

@ -0,0 +1,76 @@
## Waku layer API — discovery operations (DNS, discv5, peer exchange).
{.push raises: [].}
import std/[net, sequtils]
import results, chronos, chronicles
import logos_delivery/waku/waku
import
logos_delivery/waku/[
waku_core,
node/waku_node,
node/waku_node/peer_exchange,
discovery/waku_dnsdisc,
discovery/waku_discv5,
]
proc dnsDiscovery*(
self: Waku, enrTreeUrl: string, nameServer: string, timeoutMs: int
): Future[Result[seq[string], string]] {.async.} =
try:
let dnsNameServers = @[parseIpAddress(nameServer)]
let discoveredPeers = (
await retrieveDynamicBootstrapNodes(enrTreeUrl, dnsNameServers)
).valueOr:
return err("failed discovering peers from DNS: " & $error)
var multiAddresses = newSeq[string]()
for discPeer in discoveredPeers:
for address in discPeer.addrs:
multiAddresses.add($address & "/p2p/" & $discPeer)
return ok(multiAddresses)
except CatchableError as e:
return err(e.msg)
proc discv5UpdateBootnodes*(
self: Waku, bootnodes: string
): Future[Result[bool, string]] {.async.} =
## `bootnodes` is a JSON array of ENRs, e.g. `["enr:...", "enr:..."]`.
try:
if self.wakuDiscv5.isNil():
return err("discv5 not started")
self.wakuDiscv5.updateBootstrapRecords(bootnodes).isOkOr:
return err("error in discv5UpdateBootnodes: " & $error)
return ok(true)
except CatchableError as e:
return err(e.msg)
proc startDiscv5*(self: Waku): Future[Result[bool, string]] {.async.} =
try:
if self.wakuDiscv5.isNil():
return err("discv5 not started")
(await self.wakuDiscv5.start()).isOkOr:
return err("error starting discv5: " & $error)
return ok(true)
except CatchableError as e:
return err(e.msg)
proc stopDiscv5*(self: Waku): Future[Result[bool, string]] {.async.} =
try:
if self.wakuDiscv5.isNil():
return err("discv5 not started")
await self.wakuDiscv5.stop()
return ok(true)
except CatchableError as e:
return err(e.msg)
proc peerExchangeRequest*(
self: Waku, numPeers: uint64
): Future[Result[int, string]] {.async.} =
try:
let numPeersRecv = (await self.node.fetchPeerExchangePeers(numPeers)).valueOr:
return err("failed peer exchange: " & $error)
return ok(numPeersRecv)
except CatchableError as e:
return err(e.msg)

0
logos_delivery/waku/events/discovery_events.nim → logos_delivery/waku/api/events/discovery_events.nim
View File

3
logos_delivery/waku/api/events/events.nim Normal file
View File

@ -0,0 +1,3 @@
import ./[filter_subscribe_events, health_events, peer_events, discovery_events]
export filter_subscribe_events, health_events, peer_events, discovery_events

0
logos_delivery/waku/events/delivery_events.nim → logos_delivery/waku/api/events/filter_subscribe_events.nim
View File

9
logos_delivery/waku/events/health_events.nim → logos_delivery/waku/api/events/health_events.nim
View File

@ -1,15 +1,10 @@
import brokers/event_broker
import logos_delivery/waku/api/types
import logos_delivery/api/types
import logos_delivery/waku/node/health_monitor/[protocol_health, topic_health]
import logos_delivery/waku/waku_core/topics
export protocol_health, topic_health
# Notify health changes to node connectivity
EventBroker:
type EventConnectionStatusChange* = object
connectionStatus*: ConnectionStatus
export event_broker, protocol_health, topic_health
# Notify health changes to a subscribed topic
# TODO: emit content topic health change events when subscribe/unsubscribe

0
logos_delivery/waku/events/peer_events.nim → logos_delivery/waku/api/events/peer_events.nim
View File

88
logos_delivery/waku/api/filter.nim Normal file
View File

@ -0,0 +1,88 @@
## Waku layer API — filter (light client) operations.
import logos_delivery/waku/compat/option_valueor
{.push raises: [].}
import std/options
import results, chronos, chronicles
import logos_delivery/waku/waku
import
logos_delivery/waku/[
waku_core,
waku_core/subscription/push_handler,
node/waku_node,
node/waku_node/filter,
node/peer_manager,
waku_filter_v2/client,
waku_filter_v2/common,
]
const FilterOpTimeout = 5.seconds
proc filterSubscribe*(
self: Waku,
pubsubTopic: PubsubTopic,
contentTopics: seq[ContentTopic],
pushHandler: FilterPushHandler,
): Future[Result[bool, string]] {.async.} =
## Registers `pushHandler` for incoming filtered messages, selects a filter
## service peer, and subscribes.
try:
if self.node.wakuFilterClient.isNil():
return err("wakuFilterClient is not mounted")
self.node.wakuFilterClient.registerPushHandler(pushHandler)
let peer = self.node.peerManager.selectPeer(WakuFilterSubscribeCodec).valueOr:
return err("could not find peer with WakuFilterSubscribeCodec when subscribing")
let subFut = self.node.filterSubscribe(some(pubsubTopic), contentTopics, peer)
if not await subFut.withTimeout(FilterOpTimeout):
return err("filter subscription timed out")
subFut.read().isOkOr:
return err($error)
return ok(true)
except CatchableError as e:
return err(e.msg)
proc filterUnsubscribe*(
self: Waku, pubsubTopic: PubsubTopic, contentTopics: seq[ContentTopic]
): Future[Result[bool, string]] {.async.} =
## Selects a filter service peer and unsubscribes the given content topics.
try:
if self.node.wakuFilterClient.isNil():
return err("wakuFilterClient is not mounted")
let peer = self.node.peerManager.selectPeer(WakuFilterSubscribeCodec).valueOr:
return err("could not find peer with WakuFilterSubscribeCodec when unsubscribing")
let unsubFut = self.node.filterUnsubscribe(some(pubsubTopic), contentTopics, peer)
if not await unsubFut.withTimeout(FilterOpTimeout):
return err("filter un-subscription timed out")
unsubFut.read().isOkOr:
return err($error)
return ok(true)
except CatchableError as e:
return err(e.msg)
proc filterUnsubscribeAll*(self: Waku): Future[Result[bool, string]] {.async.} =
## Selects a filter service peer and unsubscribes from everything.
try:
if self.node.wakuFilterClient.isNil():
return err("wakuFilterClient is not mounted")
let peer = self.node.peerManager.selectPeer(WakuFilterSubscribeCodec).valueOr:
return
err("could not find peer with WakuFilterSubscribeCodec when unsubscribing all")
let unsubFut = self.node.filterUnsubscribeAll(peer)
if not await unsubFut.withTimeout(FilterOpTimeout):
return err("filter un-subscription all timed out")
unsubFut.read().isOkOr:
return err($error)
return ok(true)
except CatchableError as e:
return err(e.msg)

13
logos_delivery/waku/api/health.nim Normal file
View File

@ -0,0 +1,13 @@
## Waku layer API — health / connectivity.
{.push raises: [].}
import results, chronos, chronicles
import logos_delivery/waku/waku
import logos_delivery/waku/[node/health_monitor, node/health_monitor/online_monitor]
proc isOnline*(self: Waku): Future[Result[bool, string]] {.async.} =
try:
return ok(self.healthMonitor.onlineMonitor.amIOnline())
except CatchableError as e:
return err(e.msg)

37
logos_delivery/waku/api/lightpush.nim Normal file
View File

@ -0,0 +1,37 @@
## Waku layer API — lightpush (light client publish) operations.
import logos_delivery/waku/compat/option_valueor
{.push raises: [].}
import results, chronos, chronicles
import logos_delivery/waku/waku
import
logos_delivery/waku/[
waku_core,
waku_core/codecs,
node/waku_node,
node/peer_manager,
waku_lightpush_legacy/client,
]
proc lightpushPublish*(
self: Waku, pubsubTopic: PubsubTopic, message: WakuMessage
): Future[Result[string, string]] {.async.} =
## Selects a lightpush service peer and publishes; returns the message hash.
try:
if self.node.wakuLegacyLightpushClient.isNil():
return err("wakuLegacyLightpushClient is not mounted")
let remotePeer = self.node.peerManager.selectPeer(WakuLightPushCodec).valueOr:
return err("failed to lightpublish message, no suitable remote peers")
let msgHashHex = (
await self.node.wakuLegacyLightpushClient.publish(
pubsubTopic, message, remotePeer
)
).valueOr:
return err($error)
return ok(msgHashHex)
except CatchableError as e:
return err(e.msg)

109
logos_delivery/waku/api/peer_manager.nim Normal file
View File

@ -0,0 +1,109 @@
## Waku layer API — peer management operations.
{.push raises: [].}
import std/[options, sequtils, strutils]
import results, chronos, chronicles
import libp2p/[peerid, peerstore]
import logos_delivery/waku/waku
import logos_delivery/waku/[waku_core, node/waku_node, node/peer_manager]
# `PeerConnInfo` is defined in `api/types` (surfaced here via `import waku`).
proc connect*(
self: Waku, peers: seq[string], timeoutMs: uint32
): Future[Result[bool, string]] {.async.} =
try:
await self.node.connectToNodes(peers.mapIt(strip(it)), source = "static")
return ok(true)
except CatchableError as e:
return err(e.msg)
proc disconnectPeerById*(
self: Waku, peerId: string
): Future[Result[bool, string]] {.async.} =
try:
let pId = PeerId.init(peerId).valueOr:
return err($error)
await self.node.peerManager.disconnectNode(pId)
return ok(true)
except CatchableError as e:
return err(e.msg)
proc disconnectAllPeers*(self: Waku): Future[Result[bool, string]] {.async.} =
try:
await self.node.peerManager.disconnectAllPeers()
return ok(true)
except CatchableError as e:
return err(e.msg)
proc dialPeer*(
self: Waku, peerAddr: string, protocol: string, timeoutMs: int
): Future[Result[bool, string]] {.async.} =
try:
let remotePeerInfo = parsePeerInfo(peerAddr).valueOr:
return err($error)
let conn = await self.node.peerManager.dialPeer(remotePeerInfo, protocol)
if conn.isNone():
return err("failed dialing peer")
return ok(true)
except CatchableError as e:
return err(e.msg)
proc dialPeerById*(
self: Waku, peerId: string, protocol: string, timeoutMs: int
): Future[Result[bool, string]] {.async.} =
try:
let pId = PeerId.init(peerId).valueOr:
return err($error)
let conn = await self.node.peerManager.dialPeer(pId, protocol)
if conn.isNone():
return err("failed dialing peer")
return ok(true)
except CatchableError as e:
return err(e.msg)
proc peerIdsFromPeerstore*(self: Waku): Future[Result[seq[string], string]] {.async.} =
try:
return ok(self.node.peerManager.switch.peerStore.peers().mapIt($it.peerId))
except CatchableError as e:
return err(e.msg)
proc connectedPeersInfo*(
self: Waku
): Future[Result[seq[PeerConnInfo], string]] {.async.} =
## Structured info (protocols, addresses) for every connected peer.
try:
var infos: seq[PeerConnInfo]
for peer in self.node.peerManager.switch.peerStore.peers():
if peer.connectedness == Connected:
infos.add(
PeerConnInfo(
peerId: $peer.peerId,
protocols: peer.protocols,
addresses: peer.addrs.mapIt($it),
)
)
return ok(infos)
except CatchableError as e:
return err(e.msg)
proc connectedPeers*(self: Waku): Future[Result[seq[string], string]] {.async.} =
try:
let (inPeerIds, outPeerIds) = self.node.peerManager.connectedPeers()
return ok(concat(inPeerIds, outPeerIds).mapIt($it))
except CatchableError as e:
return err(e.msg)
proc peerIdsByProtocol*(
self: Waku, protocol: string
): Future[Result[seq[string], string]] {.async.} =
try:
return ok(
self.node.peerManager.switch.peerStore
.peers(protocol)
.filterIt(it.connectedness == Connected)
.mapIt($it.peerId)
)
except CatchableError as e:
return err(e.msg)

45
logos_delivery/waku/api/ping.nim Normal file
View File

@ -0,0 +1,45 @@
## Waku layer API — ping operation.
{.push raises: [].}
import results, chronos, chronicles
import libp2p/protocols/ping
import libp2p/switch
import logos_delivery/waku/waku
import logos_delivery/waku/[waku_core, node/waku_node, node/waku_node/ping]
proc pingPeer*(
self: Waku, peerAddr: string, timeoutMs: int
): Future[Result[int64, string]] {.async.} =
## Pings the peer; `timeoutMs <= 0` means no timeout. Returns RTT in nanos.
try:
let peerInfo = parsePeerInfo(peerAddr).valueOr:
return err("pingPeer failed to parse peer addr: " & $error)
proc doPing(): Future[Result[Duration, string]] {.async.} =
try:
let conn =
await self.node.switch.dial(peerInfo.peerId, peerInfo.addrs, PingCodec)
defer:
await conn.close()
let rtt = await self.node.libp2pPing.ping(conn)
if rtt == 0.nanos:
return err("could not ping peer: rtt-0")
return ok(rtt)
except CatchableError as e:
return err("could not ping peer: " & e.msg)
let pingFut = doPing()
let rtt: Duration =
if timeoutMs <= 0:
(await pingFut).valueOr:
return err(error)
else:
if not await pingFut.withTimeout(chronos.milliseconds(timeoutMs)):
return err("ping timed out")
pingFut.read().valueOr:
return err(error)
return ok(rtt.nanos)
except CatchableError as e:
return err(e.msg)

60
logos_delivery/waku/api/publish.nim Normal file
View File

@ -0,0 +1,60 @@
## Waku layer API — message publish primitives used by the messaging send
## pipeline.
##
## Unlike `relay.nim`/`lightpush.nim`, these preserve the rich
## `WakuLightPushResult` (status code + description) that the send processors
## branch on for their retry decisions, and expose relay/lightpush availability
## so the messaging layer never inspects `waku.node` directly.
import logos_delivery/waku/compat/option_valueor
{.push raises: [].}
import std/options
import results, chronos
import logos_delivery/waku/waku
import
logos_delivery/waku/[
waku_core,
node/waku_node,
node/peer_manager,
waku_relay/protocol,
rln,
waku_lightpush/common,
waku_lightpush/rpc,
waku_lightpush/client,
waku_lightpush/callbacks,
]
# WakuLightPushResult, PushMessageHandler, LightPushErrorCode (common) plus the
# LightPushStatusCode `$`/`==` the send processors branch on (rpc).
export common, rpc
proc hasRelay*(self: Waku): bool =
## True if relay (gossipsub publishing) is mounted.
return not self.node.wakuRelay.isNil()
proc hasLightpush*(self: Waku): bool =
## True if a lightpush client is mounted.
return not self.node.wakuLightpushClient.isNil()
proc relayPushHandler*(self: Waku): PushMessageHandler =
## Builds the relay publish handler used by the send pipeline. Caller
## ensures relay is mounted. RLN proof generation is handled client-side
## in (legacy)lightpushPublish; this handler only validates and republishes.
return getRelayPushHandler(self.node.wakuRelay)
proc lightpushPeerAvailable*(self: Waku, shard: PubsubTopic): bool =
## True if a lightpush service peer is available for `shard`.
return self.node.peerManager.selectPeer(WakuLightPushCodec, some(shard)).isSome()
proc lightpushPublishToAny*(
self: Waku, shard: PubsubTopic, message: WakuMessage
): Future[WakuLightPushResult] {.async.} =
## Selects a lightpush service peer for `shard` and publishes `message`.
## Returns SERVICE_NOT_AVAILABLE when no peer is available.
let peer = self.node.peerManager.selectPeer(WakuLightPushCodec, some(shard)).valueOr:
return lightpushResultServiceUnavailable("no lightpush peer available for shard")
try:
return await self.node.wakuLightpushClient.publish(some(shard), message, peer)
except CatchableError as e:
return lightpushResultInternalError(e.msg)

132
logos_delivery/waku/api/relay.nim Normal file
View File

@ -0,0 +1,132 @@
## Waku layer API — relay (gossipsub) operations.
{.push raises: [].}
import std/sequtils
import results, chronos, chronicles, secp256k1, stew/byteutils
import logos_delivery/waku/waku
import
logos_delivery/waku/[
waku_core,
node/waku_node,
node/waku_node/relay,
node/subscription_manager,
waku_relay/protocol,
factory/waku_conf,
factory/validator_signed,
]
proc relayPublish*(
self: Waku, pubsubTopic: PubsubTopic, message: WakuMessage, timeoutMs: uint32
): Future[Result[string, string]] {.async.} =
## Publishes `message` and returns its message hash (0x-hex).
try:
if self.node.wakuRelay.isNil():
return err("relayPublish: WakuRelay not mounted")
(await self.node.wakuRelay.publish(pubsubTopic, message)).isOkOr:
return err($error)
return ok(computeMessageHash(pubsubTopic, message).to0xHex)
except CatchableError as e:
return err(e.msg)
proc relaySubscribe*(
self: Waku,
pubsubTopic: PubsubTopic,
handler: WakuRelayHandler = WakuRelayHandler(nil),
): Future[Result[bool, string]] {.async.} =
## Subscribes to `pubsubTopic`. `handler` (optional) is invoked per message;
## pass nil to subscribe without a message callback.
try:
if self.node.wakuRelay.isNil():
return err("relaySubscribe: WakuRelay not mounted")
self.node.subscribe((kind: SubscriptionKind.PubsubSub, topic: pubsubTopic), handler).isOkOr:
return err($error)
return ok(true)
except CatchableError as e:
return err(e.msg)
proc relayUnsubscribe*(
self: Waku, pubsubTopic: PubsubTopic
): Future[Result[bool, string]] {.async.} =
try:
if self.node.wakuRelay.isNil():
return err("relayUnsubscribe: WakuRelay not mounted")
self.node.unsubscribe((kind: SubscriptionKind.PubsubSub, topic: pubsubTopic)).isOkOr:
return err($error)
return ok(true)
except CatchableError as e:
return err(e.msg)
proc relayAddProtectedShard*(
self: Waku, clusterId: uint16, shardId: uint16, publicKey: string
): Future[Result[bool, string]] {.async.} =
try:
if self.node.wakuRelay.isNil():
return err("relayAddProtectedShard: WakuRelay not mounted")
let pubKey = SkPublicKey.fromHex(publicKey).valueOr:
return err("relayAddProtectedShard: invalid public key: " & $error)
let protectedShard = ProtectedShard(shard: shardId, key: pubKey)
self.node.wakuRelay.addSignedShardsValidator(@[protectedShard], clusterId)
return ok(true)
except CatchableError as e:
return err(e.msg)
proc relayConnectedPeers*(
self: Waku, pubsubTopic: PubsubTopic
): Future[Result[seq[string], string]] {.async.} =
try:
if self.node.wakuRelay.isNil():
return err("relayConnectedPeers: WakuRelay not mounted")
let connPeers = self.node.wakuRelay.getConnectedPeers(pubsubTopic).valueOr:
return err($error)
return ok(connPeers.mapIt($it))
except CatchableError as e:
return err(e.msg)
proc relayPeersInMesh*(
self: Waku, pubsubTopic: PubsubTopic
): Future[Result[seq[string], string]] {.async.} =
try:
if self.node.wakuRelay.isNil():
return err("relayPeersInMesh: WakuRelay not mounted")
let meshPeers = self.node.wakuRelay.getPeersInMesh(pubsubTopic).valueOr:
return err($error)
return ok(meshPeers.mapIt($it))
except CatchableError as e:
return err(e.msg)
proc relayNumPeersInMesh*(
self: Waku, pubsubTopic: PubsubTopic
): Future[Result[int, string]] {.async.} =
try:
if self.node.wakuRelay.isNil():
return err("relayNumPeersInMesh: WakuRelay not mounted")
let n = self.node.wakuRelay.getNumPeersInMesh(pubsubTopic).valueOr:
return err($error)
return ok(n)
except CatchableError as e:
return err(e.msg)
proc relayNumConnectedPeers*(
self: Waku, pubsubTopic: PubsubTopic
): Future[Result[int, string]] {.async.} =
try:
if self.node.wakuRelay.isNil():
return err("relayNumConnectedPeers: WakuRelay not mounted")
let n = self.node.wakuRelay.getNumConnectedPeers(pubsubTopic).valueOr:
return err($error)
return ok(n)
except CatchableError as e:
return err(e.msg)

46
logos_delivery/waku/api/send_api.md
View File

@ -1,46 +0,0 @@
# SEND API
**THIS IS TO BE REMOVED BEFORE PR MERGE**
This document collects logic and todo's around the Send API.
## Overview
Send api hides the complex logic of using raw protocols for reliable message delivery.
The delivery method is chosen based on the node configuration and actual availabilities of peers.
## Delivery task
Each message send request is bundled into a task that not just holds the composed message but also the state of the delivery.
## Delivery methods
Depending on the configuration and the availability of store client protocol + actual configured and/or discovered store nodes:
- P2PReliability validation - checking network store node whether the message is reached at least a store node.
- Simple retry until message is propagated to the network
- Relay says >0 peers as publish result
- LightpushClient returns with success
Depending on node config:
- Relay
- Lightpush
These methods are used in combination to achieve the best reliability.
Fallback mechanism is used to switch between methods if the current one fails.
Relay+StoreCheck -> Relay+simple retry -> Lightpush+StoreCheck -> Lightpush simple retry -> Error
Combination is dynamically chosen on node configuration. Levels can be skipped depending on actual connectivity.
Actual connectivity is checked:
- Relay's topic health check - at least dLow peers in the mesh for the topic
- Store nodes availability - at least one store service node is available in peer manager
- Lightpush client availability - at least one lightpush service node is available in peer manager
## Delivery processing
At every send request, each task is tried to be delivered right away.
Any further retries and store check is done as a background task in a loop with predefined intervals.
Each task is set for a maximum number of retries and/or maximum time to live.
In each round of store check and retry send tasks are selected based on their state.
The state is updated based on the result of the delivery method.

Some files were not shown because too many files have changed in this diff Show More