Initial commit - implements runner for liteprotocoltester image deployed from nwaku

.env

@@ -0,0 +1,54 @@
+#LiteProtocolTester run configurator
+
+# Docker image of lite-protocol-tester if different than used default
+#LPT_IMAGE=
+
+# Define number of publisher and receiver nodes to run.
+NUM_PUBLISHER_NODES=1
+NUM_RECEIVER_NODES=1
+
+
+# Can add some seconds delay before SENDER starts publishing
+# Useful to let RECEIVER nodes to setup and subscribe ahead of expected messages being sent.
+START_PUBLISHING_AFTER=60
+
+# Define number of messages to be sent by each publisher node.
+# Set 0 for infinite number of messages
+NUM_MESSAGES=100
+
+# ms delay between messages - when setting this consider RLN if applied to the network use will use.
+DELAY_MESSAGES=1000
+
+# Define message size range in bytes - publisher node will randomly generate message size between these values.
+MIN_MESSAGE_SIZE=15Kb
+MAX_MESSAGE_SIZE=145Kb
+
+
+# You can define service peers for publishers and receivers if you know ahead the exact peers. Specify either with multiaddress or ENR.
+#LIGHTPUSH_SERVICE_PEER=
+#FILTER_SERVICE_PEER=
+
+# ...or you can define bootstrap to locate service peers on the network, specify either with multiaddress or ENR.
+#LIGHTPUSH_BOOTSTRAP=
+#FILTER_BOOTSTRAP=
+
+# Predefined values for different fleets - or set according to your needs
+## for wakusim
+#PUBSUB=/waku/2/rs/66/0
+#CONTENT_TOPIC=/tester/2/light-pubsub-test/wakusim
+#CLUSTER_ID=66
+
+## for status.prod
+#PUBSUB=/waku/2/rs/16/32
+#CONTENT_TOPIC=/tester/2/light-pubsub-test/fleet
+#CLUSTER_ID=16
+
+## for status.staging
+#PUBSUB=/waku/2/rs/16/32
+#CONTENT_TOPIC=/tester/2/light-pubsub-test/fleet
+#CLUSTER_ID=16
+
+## for TWN/waku.sandbox
+#PUBSUB=/waku/2/rs/1/4
+#CONTENT_TOPIC=/tester/2/light-pubsub-test/twn
+#CLUSTER_ID=1

README.md

@@ -1,2 +1,61 @@
-# lpt-runner
+# Waku - Lite Protocol Tester runner
-NWaku Lite-protocol-tester runner
+
+NWaku Lite-protocol-tester runner that intends to ease the process of running tester applications on different conditions.
+It uses docker compose to set up the environment for lite-protocol-tester with dashboard addon.
+
+## Lite Protocol Tester
+
+This tool is settled in nwaku repository under `apps/liteprotocoltester` directory.
+https://github.com/waku-org/nwaku/tree/master/apps/liteprotocoltester
+
+Tool is available as docker image at `wakuorg/liteprotocoltester:latest`.
+
+The aim is to test selected waku network's reliability on message delivery, specifically message delivery using lightpush and filter services.
+
+## Usage
+
+```bash
+git clone https://github.com/waku-org/lpt-runner.git
+cd lpt-runner
+
+# check Reame.md for more information
+# edit .env file to your needs
+
+docker compose up -d
+
+# navigate localhost:3033 to see the lite-protocol-tester dashboard
+```
+
+#### Test monitoring
+
+Navigate to http://localhost:3033 to see the lite-protocol-tester dashboard.
+
+## Configuration
+
+### Environment variables for docker compose runs
+
+|   Variable     | Description | Default |
+| ---: | :--- | :--- |
+| NUM_PUBLISHER_NODES | Number of publisher node copies to run - this can extend stressful testing of the network | 1 |
+| NUM_RECEIVER_NODES | Number of receiver node copies to run - to widen reliability test of the network | 1 |
+| NUM_MESSAGES   | Number of message to publish, 0 means infinite | 120 |
+| DELAY_MESSAGES | Frequency of messages in milliseconds | 1000 |
+| PUBSUB | Used pubsub_topic for testing | /waku/2/rs/66/0 |
+| CONTENT_TOPIC  | content_topic for testing | /tester/1/light-pubsub-example/proto |
+| CLUSTER_ID  | cluster_id of the network | 16 |
+| START_PUBLISHING_AFTER | Delay in seconds before starting to publish to let service node connected | 5 |
+| MIN_MESSAGE_SIZE | Minimum message size in bytes | 1KiB |
+| MAX_MESSAGE_SIZE | Maximum message size in bytes | 120KiB |
+| LIGHTPUSH_SERVICE_PEER | Lightpush service node's address |  |
+| FILTER_SERVICE_PEER | Filter service node's adress |  |
+| LIGHTPUSH_BOOTSTRAP | Alternative to directly specify service peer, bootstrap node is used to gather possible lightpush peer randomly from the network. |  |
+| FILTER_BOOTSTRAP | Alternative to directly specify service peer, bootstrap node is used to gather possible filter peer randomly from the network. |  |
+
+### Specifying peer addresses
+
+Service node or bootstrap addresses can be specified in multiadress or ENR form.
+
+### Using bootstrap nodes
+
+There are multiple benefits of using bootstrap nodes. By using them liteprotocoltester will use Peer Exchange protocol to get possible peers from the network that are capable to serve as service peers for testing. Additionally it will test dial them to verify their connectivity - this will be reported in the logs and on dashboard metrics.
+Also by using bootstrap node and peer exchange discovery, litprotocoltester will be able to simulate service peer switch in case of failures. There are built in tresholds for service peer failures during test and service peer can be switched during the test. Also these service peer failures are reported, thus extening network reliability measures.

docker-compose.yml

@@ -0,0 +1,115 @@
+version: "3.7"
+x-logging: &logging
+  logging:
+    driver: json-file
+    options:
+      max-size: 1000m
+
+# Environment variable definitions
+x-test-running-conditions: &test_running_conditions
+  NUM_MESSAGES: ${NUM_MESSAGES:-120}
+  DELAY_MESSAGES: "${DELAY_MESSAGES:-1000}"
+  PUBSUB: ${PUBSUB:-/waku/2/rs/66/0}
+  CONTENT_TOPIC: ${CONTENT_TOPIC:-/tester/2/light-pubsub-test/wakusim}
+  CLUSTER_ID: ${CLUSTER_ID:-66}
+  MIN_MESSAGE_SIZE: ${MIN_MESSAGE_SIZE:-1Kb}
+  MAX_MESSAGE_SIZE: ${MAX_MESSAGE_SIZE:-150Kb}
+  START_PUBLISHING_AFTER: ${START_PUBLISHING_AFTER:-60}  # seconds
+  NUM_PUBLISHER_NODES: ${NUM_PUBLISHER_NODES:-1}
+  NUM_RECEIVER_NODES: ${NUM_RECEIVER_NODES:-1}
+  LIGHTPUSH_SERVICE_PEER: ${LIGHTPUSH_SERVICE_PEER:-}
+  LIGHTPUSH_BOOTSTRAP: ${LIGHTPUSH_BOOTSTRAP:-}
+  FILTER_SERVICE_PEER: ${FILTER_SERVICE_PEER:-}
+  FILTER_BOOTSTRAP: ${FILTER_BOOTSTRAP:-}
+
+
+# Services definitions
+services:
+  publishernode:
+    image: ${LPT_IMAGE:-harbor.status.im/wakuorg/liteprotocoltester:latest}
+    deploy:
+      replicas: ${NUM_PUBLISHER_NODES:-1}
+    # ports:
+      # - 30304:30304/tcp
+      # - 30304:30304/udp
+      # - 9005:9005/udp
+      # - 127.0.0.1:8003:8003
+      # - 80:80 #Let's Encrypt
+      # - 8000:8000/tcp #WSS
+      # - 127.0.0.1:8646:8646
+    <<:
+      - *logging
+    environment:
+      DOMAIN: ${DOMAIN:-}
+      <<:
+        - *test_running_conditions
+    volumes:
+      - ./run_tester_node.sh:/opt/run_tester_node.sh:Z
+    entrypoint: sh
+    command:
+      - /opt/run_tester_node.sh
+      - /usr/bin/liteprotocoltester
+      - SENDER
+      
+  receivernode:
+    image: ${LPT_IMAGE:-harbor.status.im/wakuorg/liteprotocoltester:latest}
+    deploy:
+      replicas: ${NUM_RECEIVER_NODES:-1}
+    # ports:
+      # - 30304:30304/tcp
+      # - 30304:30304/udp
+      # - 9005:9005/udp
+      # - 127.0.0.1:8003:8003
+      # - 80:80 #Let's Encrypt
+      # - 8000:8000/tcp #WSS
+      # - 127.0.0.1:8647:8647
+    <<:
+      - *logging
+    environment:
+      DOMAIN: ${DOMAIN:-}
+      <<:
+        - *test_running_conditions
+    volumes:
+      - ./run_tester_node.sh:/opt/run_tester_node.sh:Z
+    entrypoint: sh
+    command:
+      - /opt/run_tester_node.sh
+      - /usr/bin/liteprotocoltester
+      - RECEIVER
+    depends_on:
+      - publishernode
+        
+  # We have prometheus and grafana defined in waku-simulator already
+  prometheus:
+    image: docker.io/prom/prometheus:latest
+    volumes:
+      - ./monitoring/prometheus-config.yml:/etc/prometheus/prometheus.yml:Z
+    command:
+      - --config.file=/etc/prometheus/prometheus.yml
+      - --web.listen-address=:9099
+    # ports:
+    #   - 127.0.0.1:9090:9090
+    restart: on-failure:3
+    depends_on:
+      - publishernode
+      - receivernode
+
+  grafana:
+    image: docker.io/grafana/grafana:latest
+    env_file:
+      - ./monitoring/configuration/grafana-plugins.env
+    volumes:
+      - ./monitoring/configuration/grafana.ini:/etc/grafana/grafana.ini:Z
+      - ./monitoring/configuration/dashboards.yaml:/etc/grafana/provisioning/dashboards/dashboards.yaml:Z
+      - ./monitoring/configuration/datasources.yaml:/etc/grafana/provisioning/datasources/datasources.yaml:Z
+      - ./monitoring/configuration/dashboards:/var/lib/grafana/dashboards/:Z
+      - ./monitoring/configuration/customizations/custom-logo.svg:/usr/share/grafana/public/img/grafana_icon.svg:Z
+      - ./monitoring/configuration/customizations/custom-logo.svg:/usr/share/grafana/public/img/grafana_typelogo.svg:Z
+      - ./monitoring/configuration/customizations/custom-logo.png:/usr/share/grafana/public/img/fav32.png:Z
+    ports:
+      - 0.0.0.0:3033:3033
+    restart: on-failure:3
+    depends_on:
+      - prometheus
+
+

monitoring/configuration/dashboards.yaml

@@ -0,0 +1,9 @@
+apiVersion: 1
+
+providers:
+- name: 'Prometheus'
+  orgId: 1
+  folder: ''
+  type: file
+  options:
+    path: /var/lib/grafana/dashboards

monitoring/configuration/datasources.yaml

@@ -0,0 +1,11 @@
+apiVersion: 1
+
+datasources:
+  - name: Prometheus
+    type: prometheus
+    access: proxy
+    org_id: 1
+    url: http://prometheus:9099
+    is_default: true
+    version: 1
+    editable: true

monitoring/configuration/grafana-plugins.env

@@ -0,0 +1,2 @@
+#GF_INSTALL_PLUGINS=grafana-worldmap-panel,grafana-piechart-panel,digrich-bubblechart-panel,yesoreyeram-boomtheme-panel,briangann-gauge-panel,jdbranham-diagram-panel,agenty-flowcharting-panel,citilogics-geoloop-panel,savantly-heatmap-panel,mtanda-histogram-panel,pierosavi-imageit-panel,michaeldmoore-multistat-panel,zuburqan-parity-report-panel,natel-plotly-panel,bessler-pictureit-panel,grafana-polystat-panel,corpglory-progresslist-panel,snuids-radar-panel,fzakaria-simple-config.config.annotations-datasource,vonage-status-panel,snuids-trafficlights-panel,pr0ps-trackmap-panel,alexandra-trackmap-panel,btplc-trend-box-panel
+GF_INSTALL_PLUGINS=grafana-worldmap-panel,grafana-piechart-panel,yesoreyeram-boomtheme-panel,briangann-gauge-panel,pierosavi-imageit-panel,bessler-pictureit-panel,vonage-status-panel

monitoring/configuration/grafana.ini

@@ -0,0 +1,53 @@
+instance_name = liteprotocoltester dashboard
+
+;[dashboards.json]
+;enabled = true
+;path = /home/git/grafana/grafana-dashboards/dashboards
+
+[server]
+http_port = 3033  
+
+#################################### Auth ##########################
+[auth]
+disable_login_form = false
+
+#################################### Anonymous Auth ##########################
+[auth.anonymous]
+# enable anonymous access
+enabled = true
+
+# specify organization name that should be used for unauthenticated users
+;org_name = Public
+
+# specify role for unauthenticated users
+org_role = Admin
+; org_role = Viewer
+
+;[security]
+;admin_user = ocr
+;admin_password = ocr
+
+;[users]
+# disable user signup / registration
+;allow_sign_up = false
+
+# Set to true to automatically assign new users to the default organization (id 1)
+;auto_assign_org = true
+
+# Default role new users will be automatically assigned (if disabled above is set to true)
+;auto_assign_org_role = Viewer
+
+#################################### SMTP / Emailing ##########################
+;[smtp]
+;enabled = false
+;host = localhost:25
+;user =
+;password =
+;cert_file =
+;key_file =
+;skip_verify = false
+;from_address = admin@grafana.localhost
+
+;[emails]
+;welcome_email_on_sign_up = false
+

monitoring/prometheus-config.yml

@@ -0,0 +1,35 @@
+global:
+  scrape_interval: 15s
+  evaluation_interval: 15s
+  external_labels:
+    monitor: "Monitoring"
+
+scrape_configs:
+  - job_name: "liteprotocoltester"
+    static_configs:
+    - targets: ["lpt-runner-publishernode:8003",
+                "lpt-runner-publishernode-1:8003",
+                "lpt-runner-publishernode-2:8003",
+                "lpt-runner-publishernode-3:8003",
+                "lpt-runner-publishernode-4:8003",
+                "lpt-runner-publishernode-5:8003",
+                "lpt-runner-publishernode-6:8003",
+                "lpt-runner-receivernode:8003",
+                "lpt-runner-receivernode-1:8003",
+                "lpt-runner-receivernode-2:8003",
+                "lpt-runner-receivernode-3:8003",
+                "lpt-runner-receivernode-4:8003",
+                "lpt-runner-receivernode-5:8003",
+                "lpt-runner-receivernode-6:8003",
+                "publishernode-1:8003",
+                "publishernode-2:8003",
+                "publishernode-3:8003",
+                "publishernode-4:8003",
+                "publishernode-5:8003",
+                "publishernode-6:8003",
+                "receivernode-1:8003",
+                "receivernode-2:8003",
+                "receivernode-3:8003",
+                "receivernode-4:8003",
+                "receivernode-5:8003",
+                "receivernode-6:8003",]

run_tester_node.sh

@@ -0,0 +1,101 @@
+#!/bin/sh
+
+#set -x
+#echo "$@"
+
+if test -f .env; then
+  echo "Using .env file"
+  . $(pwd)/.env
+fi
+
+
+echo "I am a lite-protocol-tester node"
+
+BINARY_PATH=$1
+
+if [ ! -x "${BINARY_PATH}" ]; then
+  echo "Invalid binary path '${BINARY_PATH}'. Failing"
+  exit 1
+fi
+
+if [ "${2}" = "--help" ]; then
+  echo "You might want to check nwaku/apps/liteprotocoltester/README.md"
+  exec "${BINARY_PATH}" --help
+  exit 0
+fi
+
+FUNCTION=$2
+if [ "${FUNCTION}" = "SENDER" ]; then
+  FUNCTION=--test-func=SENDER
+  SERIVCE_NODE_ADDR=${LIGHTPUSH_SERVICE_PEER:-${LIGHTPUSH_BOOTSTRAP:-}}
+  NODE_ARG=${LIGHTPUSH_SERVICE_PEER:+--service-node="${LIGHTPUSH_SERVICE_PEER}"}
+  NODE_ARG=${NODE_ARG:---bootstrap-node="${LIGHTPUSH_BOOTSTRAP}"}
+fi
+
+if [ "${FUNCTION}" = "RECEIVER" ]; then
+  FUNCTION=--test-func=RECEIVER
+  SERIVCE_NODE_ADDR=${FILTER_SERVICE_PEER:-${FILTER_BOOTSTRAP:-}}
+  NODE_ARG=${FILTER_SERVICE_PEER:+--service-node="${FILTER_SERVICE_PEER}"}
+  NODE_ARG=${NODE_ARG:---bootstrap-node="${FILTER_BOOTSTRAP}"}
+fi
+
+if [ -z "${SERIVCE_NODE_ADDR}" ]; then  
+  echo "Service/Bootsrap node peer_id or enr is not provided. Failing"
+  exit 1
+fi
+
+MY_EXT_IP=$(wget -qO- --no-check-certificate https://api4.ipify.org)
+
+if [ -n "${PUBSUB}" ]; then
+    PUBSUB=--pubsub-topic="${PUBSUB}"
+else
+    PUBSUB=--pubsub-topic="/waku/2/rs/66/0"
+fi
+
+if [ -n "${CONTENT_TOPIC}" ]; then
+    CONTENT_TOPIC=--content-topic="${CONTENT_TOPIC}"
+fi
+
+if [ -n "${CLUSTER_ID}" ]; then
+    CLUSTER_ID=--cluster-id="${CLUSTER_ID}"
+fi
+
+if [ -n "${START_PUBLISHING_AFTER}" ]; then
+    START_PUBLISHING_AFTER=--start-publishing-after="${START_PUBLISHING_AFTER}"
+fi
+
+if [ -n "${MIN_MESSAGE_SIZE}" ]; then
+    MIN_MESSAGE_SIZE=--min-test-msg-size="${MIN_MESSAGE_SIZE}"
+fi
+
+if [ -n "${MAX_MESSAGE_SIZE}" ]; then
+    MAX_MESSAGE_SIZE=--max-test-msg-size="${MAX_MESSAGE_SIZE}"
+fi
+
+
+if [ -n "${NUM_MESSAGES}" ]; then
+    NUM_MESSAGES=--num-messages="${NUM_MESSAGES}"
+fi
+
+if [ -n "${DELAY_MESSAGES}" ]; then
+    DELAY_MESSAGES=--delay-messages="${DELAY_MESSAGES}"
+fi
+
+echo "Running binary: ${BINARY_PATH}"
+echo "Node function is: ${FUNCTION}"
+echo "Using service/bootstrap node as: ${NODE_ARG}"
+echo "My external IP: ${MY_EXT_IP}"
+
+exec "${BINARY_PATH}"\
+      --log-level=INFO\
+      --nat=extip:${MY_EXT_IP}\
+      ${NODE_ARG}\
+      ${DELAY_MESSAGES}\
+      ${NUM_MESSAGES}\
+      ${PUBSUB}\
+      ${CONTENT_TOPIC}\
+      ${CLUSTER_ID}\
+      ${FUNCTION}\
+      ${START_PUBLISHING_AFTER}\
+      ${MIN_MESSAGE_SIZE}\
+      ${MAX_MESSAGE_SIZE}