AYAHASSAN287 4ba30a8aff
e2e part3 (#181)
* add test s17

* Add temp changes

* Add s17 positive / negative scenarios

* add S19

* Add S06 relay-only test and fix wrapper helpers (#173)

* - Add S06 relay-only test case for testing message propagation without a store.
- Update `wrapper_helpers` for clearer event type handling and type annotations (`Optional[...]` usage).
- Simplify `get_node_multiaddr` to retrieve addresses via `get_node_info_raw`.
- Refactor `wrappers_manager` to adjust bindings path to `vendor` directory and add `get_node_info_raw` method.
- Update `.gitignore` to exclude `store.sqlite3*`.

* Refactor S06 relay-only test: replace try-finally blocks with context managers for clarity and conciseness.

* Migrate S06 relay-only test to `test_send_e2e.py` and refactor with `StepsCommon` for reusability.

---------

Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

* Modify S19 test

* Adding S21

* Fix review comments

* Adding S22/S23

* Adding S24

* Add S26

* Add S30

* Add S31

* Improve `wait_for_event` loop logic and add `assert_event_invariants` helper (#178)

- Refactored the `wait_for_event` function for clarity and to ensure proper deadline handling within the loop.
- Introduced `assert_event_invariants` to validate per-request event properties, enforcing invariants like correct `requestId`, no duplicate terminal events, and proper timing between `Propagated` and `Sent`.
- Added tests for `assert_event_invariants` enforcement in `S14` and `S15` lightpush scenarios.

Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

* Add S07 and S10 send API tests with event invariants helper  (#176)

* Add `assert_event_invariants` to enforce per-request event constraints and integrate into relevant tests

* Integrate `assert_event_invariants` into edge and store tests

* Remove redundant comments from `test_send_e2e.py`

---------

Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

* Fix some tests

* Add S02/S12 send API tests and PR CI pipeline (#174)

* Add tests for auto-subscribe on first send and isolated sender with no peers

* Add PR CI workflow with tiered test strategy

- pr_tests.yml: build job with cache, wrapper-tests, smoke-tests,
  and label-triggered full-suite
- test_common.yml: add deploy_allure/send_discord inputs so PR runs
  skip reporting side effects
- Add docker_required marker to S19 (needs Docker, excluded from
  wrapper-only CI job)
- Register docker_required marker in pytest.ini

* Document PR CI test workflows in README

* Refine PR CI test strategy:
- Exclude `docker_required` tests from smoke set in `pr_tests.yml`.
- Add `wait_for_connected` helper for connection state checks.
- Update S19 test to dynamically create and clean up the store node setup.
- General simplifications and improved test stability.

* Add `wait_for_connected` assertion to ensure sender connection state before propagation test

* Refine tests and CI workflows:
- Replace `ERROR_TIMEOUT_S` with `ERROR_AFTER_CACHE_EXPIRY_TIMEOUT_S` in `test_send_e2e.py`.
- Adjust timeout assertion for better clarity and accuracy.
- Update `pr_tests.yml` to add retries (`--reruns`) and ignore wrapper tests in smoke tests.
- Change `test_common.yml` default Discord reporting to `false`.

* Normalize `portsshift` to `portsShift` in `test_send_e2e.py` configuration definitions.

---------

Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

* Add relay-to-lightpush fallback integration tests (S08/S09) (#180)

Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>

* Ignore S19

* fix s26

* Ignore s20 / s31 for errors

* Change image name

* fix xfail syntax error

* rename test file

* FIx flaky tests

* comment the skipped tests

* Fix review comments

* revert tag in yml in latest

* commenting lightpush

* Modify the PR

* Fix the ports conflict

* Modify S20

* fix portsshift option

* remove the /true from yml to allow errors to exist

* Modify the yml to continue on error

* First set of review comments

* adding xfail mark for failed tests

* address review comments about xfail

* cleanup unused lines

* event collector fix

* Address review comment about delay constant

* fix the timeout review comment

* Add assert_event_invariants

* enhance comment on S26 test

* mark the waku tests as docker_required

* Add S01

* add S01 second scenario

* Add S03

* Add S04

* Adding S11

* modify s11 scenario to pass

* Adding test S05

* Adding the new tests in part3 file

* Fix the yml file error

* Add the new test file to the PR job

* bump logos-delivery-python-bindings to include destroy_keep_ctx

* modify the S01 test

* mark S01 with xfail

* mark the second S01 test as xfail too

* use skip instead of xfail

* comment the skip line to try S01 again

* restore the xfail mark again

* remove the wrapped text code from test file

* Changing  the test files names

* skip S01 again

* removed extra comments

* Update logos-delivery-python-bindings submodule

---------

Co-authored-by: Egor Rachkovskii <32649334+at0m1x19@users.noreply.github.com>
Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>
2026-05-14 14:48:14 +02:00

276 lines
8.4 KiB
YAML

name: PR Tests
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
on:
pull_request:
types: [opened, synchronize, reopened, labeled]
paths:
- "src/**"
- "tests/**"
- "vendor/**"
- "requirements.txt"
- "pytest.ini"
- ".github/workflows/pr_tests.yml"
push:
branches: [master]
paths:
- "vendor/**"
workflow_dispatch:
inputs:
run_full_suite:
description: "Run the full test suite (18 shards)"
required: false
default: false
type: boolean
jobs:
build:
name: Build liblogosdelivery
runs-on: ubuntu-latest
timeout-minutes: 45
if: >-
github.event.action != 'labeled' ||
github.event.label.name == 'full-test'
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Compute cache key
id: cache-key
run: |
BINDINGS_HASH=$(git rev-parse HEAD:vendor/logos-delivery-python-bindings)
DELIVERY_HASH=$(git -C vendor/logos-delivery-python-bindings rev-parse HEAD:vendor/logos-delivery)
echo "key=liblogosdelivery-${{ runner.os }}-nim2.2.4-${BINDINGS_HASH}-${DELIVERY_HASH}" >> "$GITHUB_OUTPUT"
- name: Cache liblogosdelivery.so
id: cache-lib
uses: actions/cache@v4
with:
path: vendor/logos-delivery-python-bindings/lib/liblogosdelivery.so
key: ${{ steps.cache-key.outputs.key }}
- name: Remove unwanted software
if: steps.cache-lib.outputs.cache-hit != 'true'
uses: ./.github/actions/prune-vm
- name: Install system deps
if: steps.cache-lib.outputs.cache-hit != 'true'
run: |
sudo apt-get update
sudo apt-get install -y \
util-linux \
iproute2 \
sudo \
ca-certificates \
curl \
make \
gcc \
g++
- name: Install Nim 2.2.4
if: steps.cache-lib.outputs.cache-hit != 'true'
run: |
set -euo pipefail
curl https://nim-lang.org/choosenim/init.sh -sSf | sh -s -- -y
echo "$HOME/.nimble/bin" >> "$GITHUB_PATH"
export PATH="$HOME/.nimble/bin:$PATH"
choosenim 2.2.4
nim --version
nimble --version
- name: Build liblogosdelivery.so
if: steps.cache-lib.outputs.cache-hit != 'true'
run: |
set -euo pipefail
export PATH="$HOME/.nimble/bin:$PATH"
BINDINGS_DIR="$(pwd)/vendor/logos-delivery-python-bindings"
DELIVERY_DIR="$BINDINGS_DIR/vendor/logos-delivery"
mkdir -p "$BINDINGS_DIR/lib"
cd "$DELIVERY_DIR"
ln -sf waku.nimble waku.nims
nimble install -y
make setup
make liblogosdelivery
SO_PATH="$(find . -type f -name 'liblogosdelivery.so' | head -n 1)"
if [ -z "$SO_PATH" ]; then
echo "liblogosdelivery.so was not built"
exit 1
fi
cp "$SO_PATH" "$BINDINGS_DIR/lib/liblogosdelivery.so"
echo "Built library:"
ls -l "$BINDINGS_DIR/lib/liblogosdelivery.so"
- name: Upload library artifact
uses: actions/upload-artifact@v4
with:
name: liblogosdelivery
path: vendor/logos-delivery-python-bindings/lib/liblogosdelivery.so
retention-days: 1
wrapper-tests:
name: Wrapper Tests
runs-on: ubuntu-latest
needs: [build]
timeout-minutes: 45
if: >-
github.event_name != 'push' &&
(github.event.action != 'labeled' || github.event.label.name == 'full-test')
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- uses: actions/setup-python@v4
with:
python-version: "3.12"
cache: "pip"
- run: pip install -r requirements.txt
- name: Download liblogosdelivery.so
uses: actions/download-artifact@v4
with:
name: liblogosdelivery
path: vendor/logos-delivery-python-bindings/lib/
- name: Run wrapper tests - basic life cycle
continue-on-error: true
env:
PYTHONPATH: ${{ github.workspace }}/vendor/logos-delivery-python-bindings/waku
run: |
pytest tests/wrappers_tests/test_basic_life_cycle.py \
-m "not docker_required" \
--reruns 2 \
--junit-xml=wrapper-results-basic.xml
- name: Run wrapper tests - send handle and subscription
continue-on-error: true
env:
PYTHONPATH: ${{ github.workspace }}/vendor/logos-delivery-python-bindings/waku
run: |
pytest tests/wrappers_tests/test_send_handle_and_subscription.py \
-m "not docker_required" \
--reruns 2 \
--junit-xml=wrapper-results-send-handle-and-subscription.xml
- name: Run wrapper tests - send relay propagation
continue-on-error: true
env:
PYTHONPATH: ${{ github.workspace }}/vendor/logos-delivery-python-bindings/waku
run: |
pytest tests/wrappers_tests/test_send_relay_propagation.py \
-m "not docker_required" \
--reruns 2 \
--junit-xml=wrapper-results-send-relay-propagation.xml
- name: Run wrapper tests - send lightpush and edge
continue-on-error: true
env:
PYTHONPATH: ${{ github.workspace }}/vendor/logos-delivery-python-bindings/waku
run: |
pytest tests/wrappers_tests/test_send_lightpush_and_edge.py \
-m "not docker_required" \
--reruns 2 \
--junit-xml=wrapper-results-send-lightpush-and-edge.xml
- name: Run wrapper tests - send errors and concurrency
continue-on-error: true
env:
PYTHONPATH: ${{ github.workspace }}/vendor/logos-delivery-python-bindings/waku
run: |
pytest tests/wrappers_tests/test_send_errors_and_concurrency.py \
-m "not docker_required" \
--reruns 2 \
--junit-xml=wrapper-results-send-errors-and-concurrency.xml
- name: Test Report
if: always()
uses: dorny/test-reporter@95058abb17504553158e70e2c058fe1fda4392c2
with:
name: Wrapper Test Results
path: wrapper-results-*.xml
reporter: java-junit
use-actions-summary: "true"
smoke-tests:
name: Smoke Tests
runs-on: ubuntu-latest
needs: [build]
timeout-minutes: 30
if: >-
github.event_name != 'push' &&
(github.event.action != 'labeled' || github.event.label.name == 'full-test')
env:
NODE_1: "wakuorg/nwaku:latest"
NODE_2: "wakuorg/nwaku:latest"
ADDITIONAL_NODES: "wakuorg/nwaku:latest,wakuorg/nwaku:latest,wakuorg/nwaku:latest"
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- uses: actions/setup-python@v4
with:
python-version: "3.12"
cache: "pip"
- run: pip install -r requirements.txt
- name: Download liblogosdelivery.so
uses: actions/download-artifact@v4
with:
name: liblogosdelivery
path: vendor/logos-delivery-python-bindings/lib/
- name: Run smoke tests
env:
PYTHONPATH: ${{ github.workspace }}/vendor/logos-delivery-python-bindings/waku
run: |
pytest -m "smoke and not docker_required" \
--ignore=vendor/logos-delivery-python-bindings/tests \
--ignore=tests/wrappers_tests \
--reruns 1 \
-n 4 \
--dist=loadgroup \
--junit-xml=smoke-results.xml
- name: Test Report
if: always()
uses: dorny/test-reporter@95058abb17504553158e70e2c058fe1fda4392c2
with:
name: Smoke Test Results
path: smoke-results.xml
reporter: java-junit
use-actions-summary: "true"
full-suite:
name: Full Suite
if: >-
github.event_name != 'push' &&
(contains(github.event.pull_request.labels.*.name, 'full-test') ||
github.event.inputs.run_full_suite == 'true')
uses: ./.github/workflows/test_common.yml
secrets: inherit
with:
node1: "wakuorg/nwaku:latest"
node2: "wakuorg/nwaku:latest"
additional_nodes: "wakuorg/nwaku:latest,wakuorg/nwaku:latest,wakuorg/nwaku:latest"
caller: "pr"
deploy_allure: false
send_discord: false