osmaczko
a58b5ffe71
* feat: migrate chat_module from a C++ plugin to a Rust SDK module Reimplement chat_module as a Rust module on the logos-rust-sdk LIDL model, replacing the hand-written C++ plugin that wrapped liblogoschat directly. The contract rust-lib/chat_module.lidl (interface: cdylib) is now the single source of truth: logos-module-builder runs logos-lidl-gen over it to generate the C-ABI provider scaffold (the ChatModule trait, dispatch, the logos_module_* exports, and the emit_* event emitters), which src/lib.rs include!s and implements. The module pushes six typed events (message_received, message_sent, conversation_created/updated/deleted, delivery_state_changed) over the lp_* IPC event channel; consumers subscribe via on_<event>() instead of polling. Delivery is consumed through the generated typed dependency client rather than untyped string-keyed calls. init() bootstraps delivery off the Qt dispatch thread: it runs the fallible local setup (open, load_state) and registers the event subscriptions first, then fires start_async and chains the content-topic subscribe in its completion callback, so the dispatch thread is never blocked and a failed ChatClient::open can no longer orphan a started node. Readiness is surfaced as a delivery_state_changed event reaching `online`, and publish() hands the envelope to delivery's async send so a slow accept handshake never stalls the UI. Persistence write failures are surfaced instead of swallowed. The C++ plugin, its CMake test harness, and the liblogoschat mocks/stubs are removed; metadata.json now declares codegen.rust over the crate. * build: build the Rust staticlib via Nix and Cargo nix build .#chat_module drives the whole build through logos-module-builder (mkLogosModule): it generates the provider scaffold, fetches the Cargo deps from rust-lib/Cargo.lock, compiles the staticlib, and links it into the Qt plugin via CMakeLists.txt. No build.rs, no manual hash bookkeeping. Two build inputs the crate needs are gitignored because the builder produces them from a single pinned logos-rust-sdk rev (so the generator and the SDK can never skew): rust-lib/generated/provider_gen.rs (the include!d scaffold) and the staged SDK source the crate path-deps at ../logos-rust-sdk-src. nix run .#generate materialises both in the working tree so a bare cargo build works in rust-lib/. logos-delivery-module is pinned past v0.1.2 to the commit carrying the zerokit/RLN nix build fix, so a cache-miss build does not hit the crates.io 403. * ci: lint, build and test the Rust module Three jobs: nix build .#chat_module is the hermetic build gate (matrix OS); a lint job runs rustfmt and clippy (-D warnings); a cargo job runs the unit tests (matrix OS). The lint and cargo jobs call nix run .#generate first to materialise the generated scaffold and the SDK source, then run cargo in rust-lib/ directly. Generated code is excluded from both rustfmt and clippy: it lives behind an include! inside a module that opts out of warnings, so machine output never fails the lint. * docs: document the Rust module build and API Rewrite README for the Rust module: the nix build entry point, the nix run .#generate step for a bare cargo build, the LIDL contract and codegen, the six IPC events, and init() bring-up. Refresh the e2e README's CI and setup notes. * chore: address review feedback - bump logos-module-builder to the thread-safe inter-module-calls SDK; init no longer needs the Qt event-loop thread, so drop that caveat from the README and the module/inbound docs - flatten start_delivery_bootstrap's nested callbacks into create/start/ subscribe steps - name the conversation in the steady-state "not found" error - note the installation-name override is superseded once Accounts land
logos-chat-module e2e tests
TODO — legacy suite, migrate. This harness drives the pre-migration surface: the retired Nim
liblogoschat(fromlogos-messaging/logos-chat, no longer used) over the old c-ffi/polling path, and it is out of CI. It should be rewritten against the current logos-rust-sdk / LIDLchat_module(typed dependency client + lp_*-pushed events) and re-enabled.
Two-user end-to-end chat check. Runs against built chat_module + liblogoschat
inside two LogoscoreDockerDaemon containers, plus a third wakuorg/nwaku
container as a static bootstrap-node. Per PR: red on regressions in
chat_module / liblogoschat / waku-stack.
Mock-based GTest unit tests live in ../tests/. This is a separate suite that
runs the real stack.
Character names follow logos-messaging/specs:informational/chat_cast.md:
Saro = sender (initiator), Raya = recipient.
Architecture
Three containers in a shared docker network (172.30.0.0/16):
flowchart LR
subgraph net["docker network "logoschat-e2e-<uuid>""]
boot["nwaku-bootstrap<br/>172.30.0.10:60000<br/>deterministic ENR"]
saro["logoscore-saro<br/>port=60002<br/>staticPeers=[bootENR]"]
raya["logoscore-raya<br/>port=60003<br/>staticPeers=[bootENR]"]
end
saro <-->|"/vac/waku/relay/2.0.0"| boot
raya <-->|"/vac/waku/relay/2.0.0"| boot
saro <-.->|"pubsub /waku/2/rs/2/1"| raya
Saro and Raya find each other through the bootstrap-node's gossipsub mesh.
Both chat configs are generated on the fly in conftest.py::chat_user_factory
with the bootstrap ENR injected into staticPeers. The bootstrap's ENR is
not committed — it's read live from its REST API (/debug/v1/info) on each
test session, so an nwaku version upgrade doesn't silently invalidate a
stale fixture.
The bootstrap nodekey IS committed (fixtures/bootstrap-nodekey.txt) — it
fixes peerId across runs. Do not reuse this nodekey on public waku
networks; the peerId derived from it will collide with our test-node and
confuse external peer-discovery. Regenerate via openssl rand -hex 32 if
you fork this test setup.
Packaging note
We use requirements.txt instead of pyproject.toml because this is a test
harness, not a publishable package. The framework + pytest install via
pip install -r requirements.txt. Helpers and constants live in an importable
libs/ package that the tests import via from libs.helpers import ... /
from libs.constants import ... — no installable module needed.
Both pinned dependencies (logos-integration-test-framework and logoscore)
live in public repos — pip install works without auth, no PAT needed.
Prerequisites
dockeravailable on host.LOGOSCORE_IMAGEenv:logoscore:smoke-portablebuilt locally fromlogos-co/logos-logoscore-py/tests/docker_smoke/Dockerfile(one-time build).LOGOS_MODULES_DIRenv: path to a directory containingchat_module/manifest.json(i.e. output ofnix build .#install-portableon Linux, or.#installon macOS for local single-host mode — see «Local development» below).
If any of these is missing, all e2e tests skip with a clear reason.
Running locally — Linux (CI mode)
# 1. Build chat-module in install-portable layout.
nix build .#install-portable
export LOGOS_MODULES_DIR=$PWD/result/modules
# 2. Build the logoscore docker image (one-time).
git clone https://github.com/logos-co/logos-logoscore-py.git /tmp/logoscore-py
cd /tmp/logoscore-py && git checkout aa45db52
bash tests/docker_smoke/build_smoke_image.sh
export LOGOSCORE_IMAGE=logoscore:smoke-portable
cd -
# 3. Pull nwaku.
docker pull wakuorg/nwaku:v0.38.0
# 4. Install + run tests.
cd tests/e2e
pip install -r requirements.txt
pytest -v
Running locally — macOS
Local docker-mode on macOS (arm64) requires a Linux remote builder for
nix build .#install-portable (the chat_module_plugin.so Linux artifact),
which we don't ship setup for. Use the CI to validate locally during
shape-2 rollout. macOS host-process mode (Saro as a host logoscore daemon
- nwaku in docker with
127.0.0.1:60000mapping) is captured in the POC notebook but not inconftest.py— it's a TODO for step 2b.
Configuration schema
Chat configs are generated by make_chat_config(name, port, bootstrap_enr)
(in libs/helpers.py) and follow the schema understood by liblogoschat's
chat_new (see library/liblogoschat.h and library/api/client_api.nim in
logos-messaging/logos-chat):
| Field | Type | Description |
|---|---|---|
name |
string | Identity name. Note: chat_get_id returns this string, NOT a libp2p peerId. |
port |
int | Waku TCP port. Each chat container binds its own. |
clusterId |
int | Waku cluster id. Must be 2 to match the bootstrap's --preset=logos.dev. |
shardId |
int | Waku shard id. Must be 1 to match the bootstrap's --shard=1. |
staticPeers |
string[] | List of bootstrap ENRs. Only ENR strings are accepted; multiaddrs do NOT work (waku_client.nim). |
Important
If you change
--presetor--shardof the nwaku bootstrap, updateCHAT_CLUSTER_ID/CHAT_SHARD_IDinlibs/constants.pyin lockstep. The pubsub topic/waku/2/rs/{clusterId}/{shardId}must match across all three containers, or messages won't propagate.
Troubleshooting
Container logs on failure
E2E_LOG_DIR env (default /tmp) controls where _save_logs callbacks write
<container_name>.log files.
CI
Not in CI — see the top banner. This legacy suite is disabled pending a
rewrite against the logos-rust-sdk / LIDL chat_module contract.
.github/workflows/ci.yml runs only lint, cargo, and nix; there is no
e2e-tests job. Run it locally per «Running locally» above.