mirror of
https://github.com/logos-messaging/logos-messaging-interop-tests.git
synced 2026-05-19 00:39:25 +00:00
11197db624
* add test s17 * Add temp changes * Add s17 positive / negative scenarios * add S19 * Add S06 relay-only test and fix wrapper helpers (#173) * - Add S06 relay-only test case for testing message propagation without a store. - Update `wrapper_helpers` for clearer event type handling and type annotations (`Optional[...]` usage). - Simplify `get_node_multiaddr` to retrieve addresses via `get_node_info_raw`. - Refactor `wrappers_manager` to adjust bindings path to `vendor` directory and add `get_node_info_raw` method. - Update `.gitignore` to exclude `store.sqlite3*`. * Refactor S06 relay-only test: replace try-finally blocks with context managers for clarity and conciseness. * Migrate S06 relay-only test to `test_send_e2e.py` and refactor with `StepsCommon` for reusability. --------- Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Modify S19 test * Adding S21 * Fix review comments * Adding S22/S23 * Adding S24 * Add S26 * Add S30 * Add S31 * Improve `wait_for_event` loop logic and add `assert_event_invariants` helper (#178) - Refactored the `wait_for_event` function for clarity and to ensure proper deadline handling within the loop. - Introduced `assert_event_invariants` to validate per-request event properties, enforcing invariants like correct `requestId`, no duplicate terminal events, and proper timing between `Propagated` and `Sent`. - Added tests for `assert_event_invariants` enforcement in `S14` and `S15` lightpush scenarios. Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Add S07 and S10 send API tests with event invariants helper (#176) * Add `assert_event_invariants` to enforce per-request event constraints and integrate into relevant tests * Integrate `assert_event_invariants` into edge and store tests * Remove redundant comments from `test_send_e2e.py` --------- Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Fix some tests * Add S02/S12 send API tests and PR CI pipeline (#174) * Add tests for auto-subscribe on first send and isolated sender with no peers * Add PR CI workflow with tiered test strategy - pr_tests.yml: build job with cache, wrapper-tests, smoke-tests, and label-triggered full-suite - test_common.yml: add deploy_allure/send_discord inputs so PR runs skip reporting side effects - Add docker_required marker to S19 (needs Docker, excluded from wrapper-only CI job) - Register docker_required marker in pytest.ini * Document PR CI test workflows in README * Refine PR CI test strategy: - Exclude `docker_required` tests from smoke set in `pr_tests.yml`. - Add `wait_for_connected` helper for connection state checks. - Update S19 test to dynamically create and clean up the store node setup. - General simplifications and improved test stability. * Add `wait_for_connected` assertion to ensure sender connection state before propagation test * Refine tests and CI workflows: - Replace `ERROR_TIMEOUT_S` with `ERROR_AFTER_CACHE_EXPIRY_TIMEOUT_S` in `test_send_e2e.py`. - Adjust timeout assertion for better clarity and accuracy. - Update `pr_tests.yml` to add retries (`--reruns`) and ignore wrapper tests in smoke tests. - Change `test_common.yml` default Discord reporting to `false`. * Normalize `portsshift` to `portsShift` in `test_send_e2e.py` configuration definitions. --------- Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Add relay-to-lightpush fallback integration tests (S08/S09) (#180) Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Ignore S19 * fix s26 * Ignore s20 / s31 for errors * Change image name * fix xfail syntax error * rename test file * FIx flaky tests * comment the skipped tests * Fix review comments * revert tag in yml in latest * commenting lightpush * Modify the PR * Fix the ports conflict * Modify S20 * fix portsshift option * remove the /true from yml to allow errors to exist * Modify the yml to continue on error * First set of review comments * adding xfail mark for failed tests * address review comments about xfail * cleanup unused lines * event collector fix * Address review comment about delay constant * fix the timeout review comment * Add assert_event_invariants * enhance comment on S26 test * mark the waku tests as docker_required * Mark `test_s10_edge_lightpush_propagation` as xfail due to broken lightpush peer discovery. * Mark `test_s15_lightpush_retryable_error_then_recovery` as xfail due to broken lightpush peer discovery. --------- Co-authored-by: Egor Rachkovskii <32649334+at0m1x19@users.noreply.github.com> Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>
92 lines
3.7 KiB
Markdown
92 lines
3.7 KiB
Markdown
# logos-messaging-interop-tests
|
||
|
||
Logos Messaging end‑to‑end (e2e) interoperability test framework for the [Waku v2 protocol](https://rfc.vac.dev/spec/10/). It exercises multiple clients (logos-messaging-nim, js‑waku, go‑waku…) in realistic network topologies and reports results via Allure.
|
||
|
||
## Setup & contribution
|
||
|
||
```bash
|
||
# Use sparse checkout since the repo has large history
|
||
git clone --depth=1 git@github.com:logos-messaging/logos-messaging-interop-tests.git
|
||
cd logos-messaging-interop-tests
|
||
|
||
# create and activate a virtual environment
|
||
python -m venv .venv
|
||
source .venv/bin/activate
|
||
|
||
# install python dependencies + prepare git hooks
|
||
pip install -r requirements.txt
|
||
pre-commit install
|
||
```
|
||
|
||
> **Tip** – You can override any default variable defined in `src/env_vars.py` either
|
||
> • by exporting it before the `pytest` call, or
|
||
> • by creating a `.env` file at the repository root.
|
||
|
||
## Running tests locally
|
||
|
||
Run **one specific test**:
|
||
|
||
```bash
|
||
pytest -k test_unsubscribe_from_some_content_topics
|
||
```
|
||
|
||
Run **an entire test class / suite**:
|
||
|
||
```bash
|
||
pytest -k TestRelaySubscribe
|
||
```
|
||
|
||
All usual [pytest](https://docs.pytest.org/) selectors (`-k`, `-m`, `-q`, etc.) work.
|
||
|
||
Waku logs can be found in `log/docker` folder while test log can be seen either in the terminal or in the `log` folder.
|
||
|
||
## Continuous Integration (CI)
|
||
|
||
### Daily build on *nwaku\:latest*
|
||
|
||
Every day the workflow **nim\_waku\_daily.yml** triggers against the image `wakuorg/nwaku:latest`.
|
||
|
||
To launch it manually:
|
||
|
||
1. Open [https://github.com/logos-messaging/logos-messaging-interop-tests/actions/workflows/nim\_waku\_daily.yml](https://github.com/logos-messaging/logos-messaging-interop-tests/actions/workflows/nim_waku_daily.yml).
|
||
2. Click **► Run workflow**.
|
||
3. Pick the branch you want to test (defaults to `master`) and press **Run workflow**.
|
||
|
||
### PR tests
|
||
|
||
Every push to a pull request triggers **pr\_tests.yml** which runs:
|
||
|
||
1. **Build** — compiles `liblogosdelivery.so` (cached by submodule commit hash).
|
||
2. **Wrapper tests** — all tests under `tests/wrappers_tests/` that don't require Docker (~5 min).
|
||
3. **Smoke tests** — `pytest -m smoke` with Docker nodes (~10 min).
|
||
|
||
To run the **full test suite** (18 shards, same as daily) on a PR, add the label **`full-test`** to the pull request. The full suite will start automatically.
|
||
|
||
### On‑demand matrix against custom *logos-messaging-nim* versions
|
||
|
||
Use **interop\_tests.yml** when you need to test a PR or a historical image:
|
||
|
||
1. Open [https://github.com/logos-messaging/logos-messaging-interop-tests/actions/workflows/interop\_tests.yml](https://github.com/logos-messaging/logos-messaging-interop-tests/actions/workflows/interop_tests.yml).
|
||
2. Press **► Run workflow** and choose the branch.
|
||
3. In the *workflow inputs* field set the `nwaku_image` you want, e.g. `wakuorg/nwaku:v0.32.0`.
|
||
|
||
### Viewing the results
|
||
|
||
* When the job finishes GitHub will display an **Allure Report** link in the run summary.
|
||
* The bot also posts the same link in the **Waku / test‑reports** Discord channel.
|
||
|
||
### Updating the CI job used from *logos-messaging-nim*
|
||
|
||
In the **logos-messaging-nim** repository itself the file `.github/workflows/test_PR_image.yml` pins the interop test version to `SMOKE_TEST_STABLE`.
|
||
|
||
To update it, move the `SMOKE_TEST_STABLE` tag to point to the desired commit in `waku-interop-tests`.
|
||
|
||
## License
|
||
|
||
Licensed under either of:
|
||
|
||
* **MIT License** – see [LICENSE-MIT](https://github.com/waku-org/js-waku/blob/master/LICENSE-MIT) or [http://opensource.org/licenses/MIT](http://opensource.org/licenses/MIT)
|
||
* **Apache License 2.0** – see [LICENSE-APACHE-v2](https://github.com/waku-org/js-waku/blob/master/LICENSE-APACHE-v2) or [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)
|
||
|
||
at your option.
|