mirror of
https://github.com/logos-messaging/logos-messaging-interop-tests.git
synced 2026-05-18 16:29:59 +00:00
* add test s17 * Add temp changes * Add s17 positive / negative scenarios * add S19 * Add S06 relay-only test and fix wrapper helpers (#173) * - Add S06 relay-only test case for testing message propagation without a store. - Update `wrapper_helpers` for clearer event type handling and type annotations (`Optional[...]` usage). - Simplify `get_node_multiaddr` to retrieve addresses via `get_node_info_raw`. - Refactor `wrappers_manager` to adjust bindings path to `vendor` directory and add `get_node_info_raw` method. - Update `.gitignore` to exclude `store.sqlite3*`. * Refactor S06 relay-only test: replace try-finally blocks with context managers for clarity and conciseness. * Migrate S06 relay-only test to `test_send_e2e.py` and refactor with `StepsCommon` for reusability. --------- Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Modify S19 test * Adding S21 * Fix review comments * Adding S22/S23 * Adding S24 * Add S26 * Add S30 * Add S31 * Improve `wait_for_event` loop logic and add `assert_event_invariants` helper (#178) - Refactored the `wait_for_event` function for clarity and to ensure proper deadline handling within the loop. - Introduced `assert_event_invariants` to validate per-request event properties, enforcing invariants like correct `requestId`, no duplicate terminal events, and proper timing between `Propagated` and `Sent`. - Added tests for `assert_event_invariants` enforcement in `S14` and `S15` lightpush scenarios. Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Add S07 and S10 send API tests with event invariants helper (#176) * Add `assert_event_invariants` to enforce per-request event constraints and integrate into relevant tests * Integrate `assert_event_invariants` into edge and store tests * Remove redundant comments from `test_send_e2e.py` --------- Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Fix some tests * Add S02/S12 send API tests and PR CI pipeline (#174) * Add tests for auto-subscribe on first send and isolated sender with no peers * Add PR CI workflow with tiered test strategy - pr_tests.yml: build job with cache, wrapper-tests, smoke-tests, and label-triggered full-suite - test_common.yml: add deploy_allure/send_discord inputs so PR runs skip reporting side effects - Add docker_required marker to S19 (needs Docker, excluded from wrapper-only CI job) - Register docker_required marker in pytest.ini * Document PR CI test workflows in README * Refine PR CI test strategy: - Exclude `docker_required` tests from smoke set in `pr_tests.yml`. - Add `wait_for_connected` helper for connection state checks. - Update S19 test to dynamically create and clean up the store node setup. - General simplifications and improved test stability. * Add `wait_for_connected` assertion to ensure sender connection state before propagation test * Refine tests and CI workflows: - Replace `ERROR_TIMEOUT_S` with `ERROR_AFTER_CACHE_EXPIRY_TIMEOUT_S` in `test_send_e2e.py`. - Adjust timeout assertion for better clarity and accuracy. - Update `pr_tests.yml` to add retries (`--reruns`) and ignore wrapper tests in smoke tests. - Change `test_common.yml` default Discord reporting to `false`. * Normalize `portsshift` to `portsShift` in `test_send_e2e.py` configuration definitions. --------- Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Add relay-to-lightpush fallback integration tests (S08/S09) (#180) Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im> * Ignore S19 * fix s26 * Ignore s20 / s31 for errors * Change image name * fix xfail syntax error * rename test file * FIx flaky tests * comment the skipped tests * Fix review comments * revert tag in yml in latest * commenting lightpush * Modify the PR * Fix the ports conflict * Modify S20 * fix portsshift option * remove the /true from yml to allow errors to exist * Modify the yml to continue on error * First set of review comments * adding xfail mark for failed tests * address review comments about xfail * cleanup unused lines * event collector fix * Address review comment about delay constant * fix the timeout review comment * Add assert_event_invariants * enhance comment on S26 test * mark the waku tests as docker_required * Add S01 * add S01 second scenario * Add S03 * Add S04 * Adding S11 * modify s11 scenario to pass * Adding test S05 * Adding the new tests in part3 file * Fix the yml file error * Add the new test file to the PR job * bump logos-delivery-python-bindings to include destroy_keep_ctx * modify the S01 test * mark S01 with xfail * mark the second S01 test as xfail too * use skip instead of xfail * comment the skip line to try S01 again * restore the xfail mark again * remove the wrapped text code from test file * Changing the test files names * skip S01 again * removed extra comments * Update logos-delivery-python-bindings submodule --------- Co-authored-by: Egor Rachkovskii <32649334+at0m1x19@users.noreply.github.com> Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>
276 lines
8.4 KiB
YAML
276 lines
8.4 KiB
YAML
name: PR Tests
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
on:
|
|
pull_request:
|
|
types: [opened, synchronize, reopened, labeled]
|
|
paths:
|
|
- "src/**"
|
|
- "tests/**"
|
|
- "vendor/**"
|
|
- "requirements.txt"
|
|
- "pytest.ini"
|
|
- ".github/workflows/pr_tests.yml"
|
|
push:
|
|
branches: [master]
|
|
paths:
|
|
- "vendor/**"
|
|
workflow_dispatch:
|
|
inputs:
|
|
run_full_suite:
|
|
description: "Run the full test suite (18 shards)"
|
|
required: false
|
|
default: false
|
|
type: boolean
|
|
|
|
jobs:
|
|
build:
|
|
name: Build liblogosdelivery
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 45
|
|
if: >-
|
|
github.event.action != 'labeled' ||
|
|
github.event.label.name == 'full-test'
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
submodules: recursive
|
|
|
|
- name: Compute cache key
|
|
id: cache-key
|
|
run: |
|
|
BINDINGS_HASH=$(git rev-parse HEAD:vendor/logos-delivery-python-bindings)
|
|
DELIVERY_HASH=$(git -C vendor/logos-delivery-python-bindings rev-parse HEAD:vendor/logos-delivery)
|
|
echo "key=liblogosdelivery-${{ runner.os }}-nim2.2.4-${BINDINGS_HASH}-${DELIVERY_HASH}" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Cache liblogosdelivery.so
|
|
id: cache-lib
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: vendor/logos-delivery-python-bindings/lib/liblogosdelivery.so
|
|
key: ${{ steps.cache-key.outputs.key }}
|
|
|
|
- name: Remove unwanted software
|
|
if: steps.cache-lib.outputs.cache-hit != 'true'
|
|
uses: ./.github/actions/prune-vm
|
|
|
|
- name: Install system deps
|
|
if: steps.cache-lib.outputs.cache-hit != 'true'
|
|
run: |
|
|
sudo apt-get update
|
|
sudo apt-get install -y \
|
|
util-linux \
|
|
iproute2 \
|
|
sudo \
|
|
ca-certificates \
|
|
curl \
|
|
make \
|
|
gcc \
|
|
g++
|
|
|
|
- name: Install Nim 2.2.4
|
|
if: steps.cache-lib.outputs.cache-hit != 'true'
|
|
run: |
|
|
set -euo pipefail
|
|
curl https://nim-lang.org/choosenim/init.sh -sSf | sh -s -- -y
|
|
echo "$HOME/.nimble/bin" >> "$GITHUB_PATH"
|
|
export PATH="$HOME/.nimble/bin:$PATH"
|
|
choosenim 2.2.4
|
|
nim --version
|
|
nimble --version
|
|
|
|
- name: Build liblogosdelivery.so
|
|
if: steps.cache-lib.outputs.cache-hit != 'true'
|
|
run: |
|
|
set -euo pipefail
|
|
|
|
export PATH="$HOME/.nimble/bin:$PATH"
|
|
|
|
BINDINGS_DIR="$(pwd)/vendor/logos-delivery-python-bindings"
|
|
DELIVERY_DIR="$BINDINGS_DIR/vendor/logos-delivery"
|
|
|
|
mkdir -p "$BINDINGS_DIR/lib"
|
|
|
|
cd "$DELIVERY_DIR"
|
|
|
|
ln -sf waku.nimble waku.nims
|
|
|
|
nimble install -y
|
|
|
|
make setup
|
|
|
|
make liblogosdelivery
|
|
|
|
SO_PATH="$(find . -type f -name 'liblogosdelivery.so' | head -n 1)"
|
|
|
|
if [ -z "$SO_PATH" ]; then
|
|
echo "liblogosdelivery.so was not built"
|
|
exit 1
|
|
fi
|
|
|
|
cp "$SO_PATH" "$BINDINGS_DIR/lib/liblogosdelivery.so"
|
|
|
|
echo "Built library:"
|
|
ls -l "$BINDINGS_DIR/lib/liblogosdelivery.so"
|
|
|
|
- name: Upload library artifact
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: liblogosdelivery
|
|
path: vendor/logos-delivery-python-bindings/lib/liblogosdelivery.so
|
|
retention-days: 1
|
|
|
|
wrapper-tests:
|
|
name: Wrapper Tests
|
|
runs-on: ubuntu-latest
|
|
needs: [build]
|
|
timeout-minutes: 45
|
|
if: >-
|
|
github.event_name != 'push' &&
|
|
(github.event.action != 'labeled' || github.event.label.name == 'full-test')
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
submodules: recursive
|
|
|
|
- uses: actions/setup-python@v4
|
|
with:
|
|
python-version: "3.12"
|
|
cache: "pip"
|
|
|
|
- run: pip install -r requirements.txt
|
|
|
|
- name: Download liblogosdelivery.so
|
|
uses: actions/download-artifact@v4
|
|
with:
|
|
name: liblogosdelivery
|
|
path: vendor/logos-delivery-python-bindings/lib/
|
|
|
|
- name: Run wrapper tests - basic life cycle
|
|
continue-on-error: true
|
|
env:
|
|
PYTHONPATH: ${{ github.workspace }}/vendor/logos-delivery-python-bindings/waku
|
|
run: |
|
|
pytest tests/wrappers_tests/test_basic_life_cycle.py \
|
|
-m "not docker_required" \
|
|
--reruns 2 \
|
|
--junit-xml=wrapper-results-basic.xml
|
|
|
|
- name: Run wrapper tests - send handle and subscription
|
|
continue-on-error: true
|
|
env:
|
|
PYTHONPATH: ${{ github.workspace }}/vendor/logos-delivery-python-bindings/waku
|
|
run: |
|
|
pytest tests/wrappers_tests/test_send_handle_and_subscription.py \
|
|
-m "not docker_required" \
|
|
--reruns 2 \
|
|
--junit-xml=wrapper-results-send-handle-and-subscription.xml
|
|
|
|
- name: Run wrapper tests - send relay propagation
|
|
continue-on-error: true
|
|
env:
|
|
PYTHONPATH: ${{ github.workspace }}/vendor/logos-delivery-python-bindings/waku
|
|
run: |
|
|
pytest tests/wrappers_tests/test_send_relay_propagation.py \
|
|
-m "not docker_required" \
|
|
--reruns 2 \
|
|
--junit-xml=wrapper-results-send-relay-propagation.xml
|
|
|
|
- name: Run wrapper tests - send lightpush and edge
|
|
continue-on-error: true
|
|
env:
|
|
PYTHONPATH: ${{ github.workspace }}/vendor/logos-delivery-python-bindings/waku
|
|
run: |
|
|
pytest tests/wrappers_tests/test_send_lightpush_and_edge.py \
|
|
-m "not docker_required" \
|
|
--reruns 2 \
|
|
--junit-xml=wrapper-results-send-lightpush-and-edge.xml
|
|
|
|
- name: Run wrapper tests - send errors and concurrency
|
|
continue-on-error: true
|
|
env:
|
|
PYTHONPATH: ${{ github.workspace }}/vendor/logos-delivery-python-bindings/waku
|
|
run: |
|
|
pytest tests/wrappers_tests/test_send_errors_and_concurrency.py \
|
|
-m "not docker_required" \
|
|
--reruns 2 \
|
|
--junit-xml=wrapper-results-send-errors-and-concurrency.xml
|
|
|
|
- name: Test Report
|
|
if: always()
|
|
uses: dorny/test-reporter@95058abb17504553158e70e2c058fe1fda4392c2
|
|
with:
|
|
name: Wrapper Test Results
|
|
path: wrapper-results-*.xml
|
|
reporter: java-junit
|
|
use-actions-summary: "true"
|
|
|
|
smoke-tests:
|
|
name: Smoke Tests
|
|
runs-on: ubuntu-latest
|
|
needs: [build]
|
|
timeout-minutes: 30
|
|
if: >-
|
|
github.event_name != 'push' &&
|
|
(github.event.action != 'labeled' || github.event.label.name == 'full-test')
|
|
env:
|
|
NODE_1: "wakuorg/nwaku:latest"
|
|
NODE_2: "wakuorg/nwaku:latest"
|
|
ADDITIONAL_NODES: "wakuorg/nwaku:latest,wakuorg/nwaku:latest,wakuorg/nwaku:latest"
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
submodules: recursive
|
|
|
|
- uses: actions/setup-python@v4
|
|
with:
|
|
python-version: "3.12"
|
|
cache: "pip"
|
|
|
|
- run: pip install -r requirements.txt
|
|
|
|
- name: Download liblogosdelivery.so
|
|
uses: actions/download-artifact@v4
|
|
with:
|
|
name: liblogosdelivery
|
|
path: vendor/logos-delivery-python-bindings/lib/
|
|
|
|
- name: Run smoke tests
|
|
env:
|
|
PYTHONPATH: ${{ github.workspace }}/vendor/logos-delivery-python-bindings/waku
|
|
run: |
|
|
pytest -m "smoke and not docker_required" \
|
|
--ignore=vendor/logos-delivery-python-bindings/tests \
|
|
--ignore=tests/wrappers_tests \
|
|
--reruns 1 \
|
|
-n 4 \
|
|
--dist=loadgroup \
|
|
--junit-xml=smoke-results.xml
|
|
|
|
- name: Test Report
|
|
if: always()
|
|
uses: dorny/test-reporter@95058abb17504553158e70e2c058fe1fda4392c2
|
|
with:
|
|
name: Smoke Test Results
|
|
path: smoke-results.xml
|
|
reporter: java-junit
|
|
use-actions-summary: "true"
|
|
|
|
full-suite:
|
|
name: Full Suite
|
|
if: >-
|
|
github.event_name != 'push' &&
|
|
(contains(github.event.pull_request.labels.*.name, 'full-test') ||
|
|
github.event.inputs.run_full_suite == 'true')
|
|
uses: ./.github/workflows/test_common.yml
|
|
secrets: inherit
|
|
with:
|
|
node1: "wakuorg/nwaku:latest"
|
|
node2: "wakuorg/nwaku:latest"
|
|
additional_nodes: "wakuorg/nwaku:latest,wakuorg/nwaku:latest,wakuorg/nwaku:latest"
|
|
caller: "pr"
|
|
deploy_allure: false
|
|
send_discord: false |