mirror of
https://github.com/logos-messaging/logos-messaging-interop-tests.git
synced 2026-05-18 16:29:59 +00:00
4ba30a8aff
* add test s17 * Add temp changes * Add s17 positive / negative scenarios * add S19 * Add S06 relay-only test and fix wrapper helpers (#173) * - Add S06 relay-only test case for testing message propagation without a store. - Update `wrapper_helpers` for clearer event type handling and type annotations (`Optional[...]` usage). - Simplify `get_node_multiaddr` to retrieve addresses via `get_node_info_raw`. - Refactor `wrappers_manager` to adjust bindings path to `vendor` directory and add `get_node_info_raw` method. - Update `.gitignore` to exclude `store.sqlite3*`. * Refactor S06 relay-only test: replace try-finally blocks with context managers for clarity and conciseness. * Migrate S06 relay-only test to `test_send_e2e.py` and refactor with `StepsCommon` for reusability. --------- Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Modify S19 test * Adding S21 * Fix review comments * Adding S22/S23 * Adding S24 * Add S26 * Add S30 * Add S31 * Improve `wait_for_event` loop logic and add `assert_event_invariants` helper (#178) - Refactored the `wait_for_event` function for clarity and to ensure proper deadline handling within the loop. - Introduced `assert_event_invariants` to validate per-request event properties, enforcing invariants like correct `requestId`, no duplicate terminal events, and proper timing between `Propagated` and `Sent`. - Added tests for `assert_event_invariants` enforcement in `S14` and `S15` lightpush scenarios. Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Add S07 and S10 send API tests with event invariants helper (#176) * Add `assert_event_invariants` to enforce per-request event constraints and integrate into relevant tests * Integrate `assert_event_invariants` into edge and store tests * Remove redundant comments from `test_send_e2e.py` --------- Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Fix some tests * Add S02/S12 send API tests and PR CI pipeline (#174) * Add tests for auto-subscribe on first send and isolated sender with no peers * Add PR CI workflow with tiered test strategy - pr_tests.yml: build job with cache, wrapper-tests, smoke-tests, and label-triggered full-suite - test_common.yml: add deploy_allure/send_discord inputs so PR runs skip reporting side effects - Add docker_required marker to S19 (needs Docker, excluded from wrapper-only CI job) - Register docker_required marker in pytest.ini * Document PR CI test workflows in README * Refine PR CI test strategy: - Exclude `docker_required` tests from smoke set in `pr_tests.yml`. - Add `wait_for_connected` helper for connection state checks. - Update S19 test to dynamically create and clean up the store node setup. - General simplifications and improved test stability. * Add `wait_for_connected` assertion to ensure sender connection state before propagation test * Refine tests and CI workflows: - Replace `ERROR_TIMEOUT_S` with `ERROR_AFTER_CACHE_EXPIRY_TIMEOUT_S` in `test_send_e2e.py`. - Adjust timeout assertion for better clarity and accuracy. - Update `pr_tests.yml` to add retries (`--reruns`) and ignore wrapper tests in smoke tests. - Change `test_common.yml` default Discord reporting to `false`. * Normalize `portsshift` to `portsShift` in `test_send_e2e.py` configuration definitions. --------- Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Add relay-to-lightpush fallback integration tests (S08/S09) (#180) Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Ignore S19 * fix s26 * Ignore s20 / s31 for errors * Change image name * fix xfail syntax error * rename test file * FIx flaky tests * comment the skipped tests * Fix review comments * revert tag in yml in latest * commenting lightpush * Modify the PR * Fix the ports conflict * Modify S20 * fix portsshift option * remove the /true from yml to allow errors to exist * Modify the yml to continue on error * First set of review comments * adding xfail mark for failed tests * address review comments about xfail * cleanup unused lines * event collector fix * Address review comment about delay constant * fix the timeout review comment * Add assert_event_invariants * enhance comment on S26 test * mark the waku tests as docker_required * Add S01 * add S01 second scenario * Add S03 * Add S04 * Adding S11 * modify s11 scenario to pass * Adding test S05 * Adding the new tests in part3 file * Fix the yml file error * Add the new test file to the PR job * bump logos-delivery-python-bindings to include destroy_keep_ctx * modify the S01 test * mark S01 with xfail * mark the second S01 test as xfail too * use skip instead of xfail * comment the skip line to try S01 again * restore the xfail mark again * remove the wrapped text code from test file * Changing the test files names * skip S01 again * removed extra comments * Update logos-delivery-python-bindings submodule --------- Co-authored-by: Egor Rachkovskii <32649334+at0m1x19@users.noreply.github.com> Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>
logos-messaging-interop-tests
Logos Messaging end‑to‑end (e2e) interoperability test framework for the Waku v2 protocol. It exercises multiple clients (logos-messaging-nim, js‑waku, go‑waku…) in realistic network topologies and reports results via Allure.
Setup & contribution
# Use sparse checkout since the repo has large history
git clone --depth=1 git@github.com:logos-messaging/logos-messaging-interop-tests.git
cd logos-messaging-interop-tests
# create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate
# install python dependencies + prepare git hooks
pip install -r requirements.txt
pre-commit install
Tip
– You can override any default variable defined in
src/env_vars.pyeither • by exporting it before thepytestcall, or • by creating a.envfile at the repository root.
Running tests locally
Run one specific test:
pytest -k test_unsubscribe_from_some_content_topics
Run an entire test class / suite:
pytest -k TestRelaySubscribe
All usual pytest selectors (-k, -m, -q, etc.) work.
Waku logs can be found in log/docker folder while test log can be seen either in the terminal or in the log folder.
Continuous Integration (CI)
Daily build on nwaku:latest
Every day the workflow nim_waku_daily.yml triggers against the image wakuorg/nwaku:latest.
To launch it manually:
- Open https://github.com/logos-messaging/logos-messaging-interop-tests/actions/workflows/nim_waku_daily.yml.
- Click ► Run workflow.
- Pick the branch you want to test (defaults to
master) and press Run workflow.
PR tests
Every push to a pull request triggers pr_tests.yml which runs:
- Build — compiles
liblogosdelivery.so(cached by submodule commit hash). - Wrapper tests — all tests under
tests/wrappers_tests/that don't require Docker (~5 min). - Smoke tests —
pytest -m smokewith Docker nodes (~10 min).
To run the full test suite (18 shards, same as daily) on a PR, add the label full-test to the pull request. The full suite will start automatically.
On‑demand matrix against custom logos-messaging-nim versions
Use interop_tests.yml when you need to test a PR or a historical image:
- Open https://github.com/logos-messaging/logos-messaging-interop-tests/actions/workflows/interop_tests.yml.
- Press ► Run workflow and choose the branch.
- In the workflow inputs field set the
nwaku_imageyou want, e.g.wakuorg/nwaku:v0.32.0.
Viewing the results
- When the job finishes GitHub will display an Allure Report link in the run summary.
- The bot also posts the same link in the Waku / test‑reports Discord channel.
Updating the CI job used from logos-messaging-nim
In the logos-messaging-nim repository itself the file .github/workflows/test_PR_image.yml pins the interop test version to SMOKE_TEST_STABLE.
To update it, move the SMOKE_TEST_STABLE tag to point to the desired commit in waku-interop-tests.
License
Licensed under either of:
- MIT License – see LICENSE-MIT or http://opensource.org/licenses/MIT
- Apache License 2.0 – see LICENSE-APACHE-v2 or http://www.apache.org/licenses/LICENSE-2.0
at your option.