logos-messaging/logos-messaging-interop-tests

mirror of https://github.com/logos-messaging/logos-messaging-interop-tests.git synced 2026-05-18 08:19:32 +00:00

* add test s17

* Add temp changes

* Add s17 positive / negative scenarios

* add S19

* Add S06 relay-only test and fix wrapper helpers (#173)

* - Add S06 relay-only test case for testing message propagation without a store.
- Update `wrapper_helpers` for clearer event type handling and type annotations (`Optional[...]` usage).
- Simplify `get_node_multiaddr` to retrieve addresses via `get_node_info_raw`.
- Refactor `wrappers_manager` to adjust bindings path to `vendor` directory and add `get_node_info_raw` method.
- Update `.gitignore` to exclude `store.sqlite3*`.

* Refactor S06 relay-only test: replace try-finally blocks with context managers for clarity and conciseness.

* Migrate S06 relay-only test to `test_send_e2e.py` and refactor with `StepsCommon` for reusability.

---------

Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

* Modify S19 test

* Adding S21

* Fix review comments

* Adding S22/S23

* Adding S24

* Add S26

* Add S30

* Add S31

* Improve `wait_for_event` loop logic and add `assert_event_invariants` helper (#178)

- Refactored the `wait_for_event` function for clarity and to ensure proper deadline handling within the loop.
- Introduced `assert_event_invariants` to validate per-request event properties, enforcing invariants like correct `requestId`, no duplicate terminal events, and proper timing between `Propagated` and `Sent`.
- Added tests for `assert_event_invariants` enforcement in `S14` and `S15` lightpush scenarios.

Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

* Add S07 and S10 send API tests with event invariants helper  (#176)

* Add `assert_event_invariants` to enforce per-request event constraints and integrate into relevant tests

* Integrate `assert_event_invariants` into edge and store tests

* Remove redundant comments from `test_send_e2e.py`

---------

Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

* Fix some tests

* Add S02/S12 send API tests and PR CI pipeline (#174)

* Add tests for auto-subscribe on first send and isolated sender with no peers

* Add PR CI workflow with tiered test strategy

- pr_tests.yml: build job with cache, wrapper-tests, smoke-tests,
  and label-triggered full-suite
- test_common.yml: add deploy_allure/send_discord inputs so PR runs
  skip reporting side effects
- Add docker_required marker to S19 (needs Docker, excluded from
  wrapper-only CI job)
- Register docker_required marker in pytest.ini

* Document PR CI test workflows in README

* Refine PR CI test strategy:
- Exclude `docker_required` tests from smoke set in `pr_tests.yml`.
- Add `wait_for_connected` helper for connection state checks.
- Update S19 test to dynamically create and clean up the store node setup.
- General simplifications and improved test stability.

* Add `wait_for_connected` assertion to ensure sender connection state before propagation test

* Refine tests and CI workflows:
- Replace `ERROR_TIMEOUT_S` with `ERROR_AFTER_CACHE_EXPIRY_TIMEOUT_S` in `test_send_e2e.py`.
- Adjust timeout assertion for better clarity and accuracy.
- Update `pr_tests.yml` to add retries (`--reruns`) and ignore wrapper tests in smoke tests.
- Change `test_common.yml` default Discord reporting to `false`.

* Normalize `portsshift` to `portsShift` in `test_send_e2e.py` configuration definitions.

---------

Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

* Add relay-to-lightpush fallback integration tests (S08/S09) (#180)

Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

* Ignore S19

* fix s26

* Ignore s20 / s31 for errors

* Change image name

* fix xfail syntax error

* rename test file

* FIx flaky tests

* comment the skipped tests

* Fix review comments

* revert tag in yml in latest

* commenting lightpush

* Modify the PR

* Fix the ports conflict

* Modify S20

* fix portsshift option

* remove the /true from yml to allow errors to exist

* Modify the yml to continue on error

* First set of review comments

* adding xfail mark for failed tests

* address review comments about xfail

* cleanup unused lines

* event collector fix

* Address review comment about delay constant

* fix the timeout review comment

* Add assert_event_invariants

* enhance comment on S26 test

* mark the waku tests as docker_required

* Mark `test_s10_edge_lightpush_propagation` as xfail due to broken lightpush peer discovery.

* Mark `test_s15_lightpush_retryable_error_then_recovery` as xfail due to broken lightpush peer discovery.

---------

Co-authored-by: Egor Rachkovskii <32649334+at0m1x19@users.noreply.github.com>
Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

2026-05-11 15:53:18 +02:00

3.7 KiB

Raw Blame History

logos-messaging-interop-tests

Logos Messaging end‑to‑end (e2e) interoperability test framework for the Waku v2 protocol. It exercises multiple clients (logos-messaging-nim, js‑waku, go‑waku…) in realistic network topologies and reports results via Allure.

Setup & contribution

# Use sparse checkout since the repo has large history
git clone --depth=1 git@github.com:logos-messaging/logos-messaging-interop-tests.git
cd logos-messaging-interop-tests

# create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate

# install python dependencies + prepare git hooks
pip install -r requirements.txt
pre-commit install

Tip

– You can override any default variable defined in src/env_vars.py either • by exporting it before the pytest call, or • by creating a .env file at the repository root.

Running tests locally

Run one specific test:

pytest -k test_unsubscribe_from_some_content_topics

Run an entire test class / suite:

pytest -k TestRelaySubscribe

All usual pytest selectors (-k, -m, -q, etc.) work.

Waku logs can be found in log/docker folder while test log can be seen either in the terminal or in the log folder.

Continuous Integration (CI)

Daily build on nwaku:latest

Every day the workflow nim_waku_daily.yml triggers against the image wakuorg/nwaku:latest.

To launch it manually:

Open https://github.com/logos-messaging/logos-messaging-interop-tests/actions/workflows/nim_waku_daily.yml.
Click ► Run workflow.
Pick the branch you want to test (defaults to master) and press Run workflow.

PR tests

Every push to a pull request triggers pr_tests.yml which runs:

Build — compiles liblogosdelivery.so (cached by submodule commit hash).
Wrapper tests — all tests under tests/wrappers_tests/ that don't require Docker (~5 min).
Smoke tests — pytest -m smoke with Docker nodes (~10 min).

To run the full test suite (18 shards, same as daily) on a PR, add the label full-test to the pull request. The full suite will start automatically.

On‑demand matrix against custom logos-messaging-nim versions

Use interop_tests.yml when you need to test a PR or a historical image:

Open https://github.com/logos-messaging/logos-messaging-interop-tests/actions/workflows/interop_tests.yml.
Press ► Run workflow and choose the branch.
In the workflow inputs field set the nwaku_image you want, e.g. wakuorg/nwaku:v0.32.0.

Viewing the results

When the job finishes GitHub will display an Allure Report link in the run summary.
The bot also posts the same link in the Waku / test‑reports Discord channel.

Updating the CI job used from logos-messaging-nim

In the logos-messaging-nim repository itself the file .github/workflows/test_PR_image.yml pins the interop test version to SMOKE_TEST_STABLE.

To update it, move the SMOKE_TEST_STABLE tag to point to the desired commit in waku-interop-tests.

License

Licensed under either of:

MIT License – see LICENSE-MIT or http://opensource.org/licenses/MIT
Apache License 2.0 – see LICENSE-APACHE-v2 or http://www.apache.org/licenses/LICENSE-2.0

at your option.

3.7 KiB Raw Blame History Unescape Escape