mirror of
https://github.com/logos-co/logos-delivery-module.git
synced 2026-08-27 12:01:09 +00:00
* test: add Python e2e suite driving delivery_module over RPC Adds tests/e2e/: a pytest suite that boots logoscore daemons in docker, loads delivery_module, and drives it over RPC — exercising the plugin-load + codegen + RPC + two-node network-delivery paths the in-process C++ tests can't. Reuses logos-integration-test-framework / logoscore, pinned to chat-module's known-good SHAs. Single-daemon: load/createNode/start/stop lifecycle, createNode-twice rejection, and query + subscribe/unsubscribe round-trips. Two-daemon (direct staticnodes peering, relay-only single shard): sender-side messagePropagated is the hard gate; messageReceived is xfail until the receive path and event-payload shape are confirmed on a green run. CI: new e2e-tests job (needs build-and-test, ubuntu-latest) building .#install-portable plus the logoscore smoke image and CLI, with a cachix pre-warm step added to build-and-test. * test(e2e): add RPC-reply-wedge diagnostic probes Opt-in (E2E_PROBES=1) probe suite that isolates which factor wedges a delivery node's RPC reply channel after a watched send: event emission alone, the watcher subscription alone, messagePropagated transmission, or the loopback messageReceived (vector<uint8_t>). Each probe runs on a fresh two-node pair so a wedge can't pollute later probes. Enabled in the e2e CI job; set E2E_QT_DEBUG=1 to also capture Qt RemoteObjects transport logs. * update(README): clarify module packaging and add build reproducibility note * Revert "update(README): clarify module packaging and add build reproducibility note" This reverts commit5aed6447a9. * Revert "test(e2e): add RPC-reply-wedge diagnostic probes" This reverts commitef1f541358. * test(e2e): mark propagation and messageReceived tests xfail due to known issues (#59) * ci: make e2e-tests non-gating while build bug #58 is open The module build is reproducible per source tree but some trees produce a plugin whose start() never returns (#58), which fails the solo lifecycle tests at start. xfail can't cover that (it hits setup), so allow the e2e job to fail without blocking the PR until #58 is fixed. * test(e2e),ci: drop narrative/decorative comments per repo style Remove explanatory comment blocks, section-banner comments, and redundant inline rationale across the e2e suite and the CI workflow; keep only linter directives and a single footgun one-liner. No behaviour change. * test(e2e): refactor node interaction to use LogosDelivery API Refactor message delivery tests and helpers for clarity using the new `LogosDelivery` class. Replaced low-level RPC calls with higher-level methods (`subscribe`, `send`, `watch`, etc.), streamlining node interaction and removing redundant logic. Updated lifecycle tests for consistency. No behavior change. * ci(e2e): capture gdb thread backtraces when start wedges (#58) Non-gating diagnostic in the e2e-tests job: boot a solo logoscore daemon, call start on a background thread, and if it wedges attach gdb to every container process for thread backtraces, /proc maps and kernel stacks. Uploaded with the e2e-test-logs artifact. * ci(e2e): fix #58 wedge detection so gdb actually fires The client RPC errors (exit 4) ~20s before the daemon-side start() releases its 30s semaphore, so a populated result no longer means success. Classify any non-success outcome as wedged, poll to detect it early, and attach gdb to every container process in parallel to freeze them before the semaphore releases. * debug(#58): log start callback path and capture daemon log past 30s Add fprintf in callApiRetVoid at callback entry, after invoke, and on the semaphore timeout to show whether liblogosdelivery ever invokes our completion callback (upstream) vs a userData/map miss (adapter). Keep the wedge probe daemon alive past the 30s CALLBACK_TIMEOUT so the start outcome is captured. * debug(#58): add pre-invoke positive control and inspect loaded plugin Log unconditionally right before invoke() so a bad build distinguishes 'start_node did not return' from 'the loaded plugin lacks the instrumentation'. On a wedge the probe now strings/sha256 the loaded delivery_module_plugin.so for the instrumentation literals and ldd's the bundled libs. * debug(#58): force stderr unbuffered so start-path logs are reliable Prior run showed the 'about to invoke' line (which provably executed — the node started right after) missing from the captured log, making log-line absence useless as evidence. setvbuf(_IONBF) at module init + fflush after each instrumented fprintf so the next bad build's log is trustworthy. * debug(#58): probe RPC channel after a start wedge start() returns and no thread is stuck (kernel stacks), so the wedge is a lost RPC reply, not a hang. On a wedge, issue follow-up version/getAvailableConfigs calls: if they also time out the whole QtRO reply channel is dead (matches the sync-reply-wedge bug); if they succeed only start's reply was lost. * debug(#58): revert useless stderr instrumentation The fprintf/setvbuf tracing added to localize #58 is dead weight — logoscore's stderr capture drops adapter log lines during the node-startup flood, so the log-based method never worked. Revert it (src/ is now net-zero vs master) and drop the diagnostic's now-moot instrumentation-strings check. The gdb + kernel stacks + RPC-channel probe in wedge_gdb.py are what actually localized the bug and stay. * ci(e2e): bump logoscore daemon+CLI past the QtRO reply-wedge fix (#58) Root-caused #58/#59 to the pre-fix ModuleProxy in the e2e daemon's SDK: it emits module events directly on the caller's (module FFI) thread, racing QtRO source serialization against an in-flight reply and permanently wedging the sync-reply channel. Fixed upstream in logos-cpp-sdk d77c3dd (marshal events onto the source thread) + logos-protocol #7 (defer async completion off the QtRO read stack). Our daemon was pinned 6 commits before it. Bump the smoke-image daemon (logoscore-py aa45db52 -> 71e00381) and the CLI (logoscore-cli 5a1cf746 -> a9e18455) to a post-split closure carrying both fixes. The Python client already floats logoscore@master, so no requirements change is needed. * test(e2e): un-xfail two-node delivery tests; drop the #58 wedge diagnostic The daemon-pin bump (901c525) fixed the QtRO reply-wedge behind #58/#59, so the three two-node delivery tests now pass (propagation both directions + receive). Un-xfail them and delete the temporary gdb wedge diagnostic + its CI step; revert the artifact upload to failure-only. Keep continue-on-error on e2e-tests for now to gather a flake baseline on the real cross-node delivery before it gates. * test(e2e): rename lifecycle test for clarity --------- Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>
151 lines
5.4 KiB
Python
151 lines
5.4 KiB
Python
"""Fixtures for the delivery_module e2e suite.
|
|
|
|
Two daemon flavours:
|
|
* `solo_daemon` (function scope) — one fresh logoscore container per test, used
|
|
by the single-node lifecycle/query tests. Fresh because createNode is
|
|
once-per-context: a started node can't be re-created, so tests can't share one.
|
|
* `node_a` / `node_b` (module scope) — two daemons in a shared docker network,
|
|
peered directly via staticnodes (node A's container-routable multiaddr), used
|
|
by the two-node delivery tests.
|
|
|
|
Every prerequisite (docker, the logoscore image, the built module layout) is
|
|
gated by `_e2e_env_or_skip`, so the suite skips cleanly when run without setup.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import os
|
|
import subprocess
|
|
import sys
|
|
import time
|
|
import uuid
|
|
from collections.abc import Callable, Iterator
|
|
from contextlib import ExitStack
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
from libs.constants import NETWORK_SUBNET
|
|
from libs.helpers import DeliveryNode, make_delivery_config, node_multiaddr, setup_delivery_node
|
|
|
|
MESH_STABILIZATION_S = 12.0
|
|
|
|
|
|
def _docker_logs(container_name: str) -> str:
|
|
r = subprocess.run(["docker", "logs", container_name], capture_output=True, text=True)
|
|
return (r.stdout or "") + (r.stderr or "")
|
|
|
|
|
|
def _save_logs(container_name: str) -> None:
|
|
log_dir = Path(os.environ.get("E2E_LOG_DIR", "/tmp"))
|
|
log_dir.mkdir(parents=True, exist_ok=True)
|
|
try:
|
|
(log_dir / f"{container_name}.log").write_text(_docker_logs(container_name))
|
|
except OSError as e:
|
|
sys.stderr.write(f"warning: failed to save log for {container_name}: {e}\n")
|
|
|
|
|
|
@pytest.fixture(scope="session")
|
|
def _e2e_env_or_skip() -> tuple[str, Path]:
|
|
"""Single gate: skip unless docker, the logoscore image, and the built
|
|
delivery_module layout are all present."""
|
|
from logoscore import docker_available, image_present # noqa: PLC0415
|
|
|
|
if not docker_available():
|
|
pytest.skip("docker not available")
|
|
image = os.environ.get("LOGOSCORE_IMAGE")
|
|
if not image:
|
|
pytest.skip("LOGOSCORE_IMAGE not set")
|
|
if not image_present(image):
|
|
pytest.skip(f"LOGOSCORE_IMAGE={image!r} not present locally")
|
|
modules_dir_env = os.environ.get("LOGOS_MODULES_DIR")
|
|
if not modules_dir_env:
|
|
pytest.skip("LOGOS_MODULES_DIR not set")
|
|
modules_dir = Path(modules_dir_env)
|
|
if not (modules_dir / "delivery_module" / "manifest.json").is_file():
|
|
pytest.skip(
|
|
f"LOGOS_MODULES_DIR={modules_dir} doesn't contain "
|
|
"delivery_module/manifest.json — did you run `nix build .#install-portable`?"
|
|
)
|
|
return image, modules_dir
|
|
|
|
|
|
@pytest.fixture(scope="session")
|
|
def shared_docker_network(_e2e_env_or_skip: tuple[str, Path]) -> Iterator[str]:
|
|
name = f"logosdelivery-e2e-{uuid.uuid4().hex[:8]}"
|
|
r = subprocess.run(
|
|
["docker", "network", "create", "--subnet", NETWORK_SUBNET, name],
|
|
capture_output=True, text=True,
|
|
)
|
|
if r.returncode != 0:
|
|
pytest.fail(f"failed to create docker network {name!r}: {r.stderr.strip()}")
|
|
try:
|
|
yield name
|
|
finally:
|
|
subprocess.run(["docker", "network", "rm", name], capture_output=True)
|
|
|
|
|
|
def _new_daemon(image: str, modules_dir: Path, network: str, label: str):
|
|
from logoscore import LogoscoreDockerDaemon # noqa: PLC0415
|
|
|
|
container_name = f"logoscore-{label.lower()}-{uuid.uuid4().hex[:8]}"
|
|
daemon = LogoscoreDockerDaemon(
|
|
image=image,
|
|
modules_dir=modules_dir,
|
|
container_name=container_name,
|
|
network=network,
|
|
startup_timeout=60.0,
|
|
extra_args=["--verbose"],
|
|
)
|
|
return daemon, container_name
|
|
|
|
|
|
@pytest.fixture
|
|
def solo_daemon(_e2e_env_or_skip: tuple[str, Path], shared_docker_network: str) -> Iterator:
|
|
"""A fresh, started logoscore container — module mounted, nothing loaded yet.
|
|
The test drives load_module/createNode/start/stop itself."""
|
|
image, modules_dir = _e2e_env_or_skip
|
|
daemon, container_name = _new_daemon(image, modules_dir, shared_docker_network, "solo")
|
|
try:
|
|
daemon.start()
|
|
yield daemon
|
|
finally:
|
|
_save_logs(container_name)
|
|
daemon.stop()
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def delivery_node_factory(
|
|
_e2e_env_or_skip: tuple[str, Path],
|
|
shared_docker_network: str,
|
|
) -> Iterator[Callable[[str, str], DeliveryNode]]:
|
|
"""Factory spawning a fully set-up (load+createNode+start) DeliveryNode per
|
|
call. All teardown + log capture is registered on one ExitStack closed when
|
|
the module's tests finish."""
|
|
image, modules_dir = _e2e_env_or_skip
|
|
|
|
with ExitStack() as stack:
|
|
def _create(label: str, config_json: str) -> DeliveryNode:
|
|
daemon, container_name = _new_daemon(image, modules_dir, shared_docker_network, label)
|
|
stack.enter_context(daemon)
|
|
stack.callback(_save_logs, container_name)
|
|
return setup_delivery_node(daemon, config_json, label)
|
|
|
|
yield _create
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def node_a(delivery_node_factory: Callable[[str, str], DeliveryNode]) -> DeliveryNode:
|
|
return delivery_node_factory("A", make_delivery_config())
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def node_b(
|
|
delivery_node_factory: Callable[[str, str], DeliveryNode],
|
|
node_a: DeliveryNode,
|
|
) -> DeliveryNode:
|
|
peer = node_multiaddr(node_a)
|
|
node = delivery_node_factory("B", make_delivery_config(static_peers=[peer]))
|
|
time.sleep(MESH_STABILIZATION_S)
|
|
return node
|