chore: RLN registration support (#16)
* chore: parameters and volumes for RLN relay * chore: add startup test for RLN * fix: remove redundant rln_enabled from WakuNode * fix: - and _ magic in rln parameters * fix: key 'eth_testnet_key' -> 'eth_client_private_key' * fix: rln-register_only -> rln-register-only * fix: use extend instead of append for the volumes list * fix: use absolute path for the volumes names - mark volumes shared across containers * fix: remove :z attr from volumes * fix: remove filename from volume path * fix: remove request for ENR * test: plain RLN registration * fix: remove subscribe * fix: remove ENR related params * test: add run_container func to Docker Mananger * fix: remove run_container func from Docker Mananger - no need for exec * fix: pass exec commands instead of specialized docker exec func * fix: exclude RLN arguments from others * fix: separate RLN volumes by implementation * test: registration with nwaku * test: registration over existing credentials * test: add RLN Relay node startup * test: RLN credentials reuse for relay node startup * fix: clean up unnecessary commands * fix: clean up unnecessary commands for non RLN relay too * test: regression by sending one valid message * fix: add forgotten call to get enr_uri * fix: check RLN credentials set properly * fix: parenthesis in eval expression * fix: better check for RLN credentials * fix: update to new flags - gowaku - separate private key for go/nwaku * test: register RLN * fix: RLN credentials check * fix: remove enable rln-relay flag from registration * test: reorder commands to check go-waku registration * fix: restructure if statements for RLN registration * fix: different set of flag for RLN registration and operation * fix: forgotten "eth" in eth-contract-address * fix: remove redundant None from get_env_var call * fix: refactor rln credentials parsing from start function * fix: missed second return value * test: call to parse_rln_credentials * fix: remove redundant self in call to parse_rln_credentials * fix: remove rln related values if not valid * fix: refactored to accept multiple private keys - JSON source for RLN credentials - removed go-waku support for RLN * test: register RLN with 2 nwaku nodes * fix: missing open file, key errors * fix: return effective keystore_path * fix: cleanup unused env variables * fix: cleanup unused env variables from relay * fix: wait longer for credentials file to be written to disk * test: remove waiting for credentials file to be written to disk * fix: add select_private_key() * fix: merge parse_rln_credentials and parse_rln_registration_credentials * test: wait for filesystem cache * test: try with os.sync to flush cache * test: stop container to clear cache * fix: 15 sec wait + stop container to clear cache * test: RLN registration test with fixture and cred file check * test: added allure.step - RLN registration for single node - corresponding test to register all keys * fix: f-strings in the test * fix: sync naming for register_main_rln_relay_nodes * fix: add WARN message to log unset and expected RLN credentials * fix: pytest.skip added to exit tests when non nwaku image is used * fix: swap NODE2 for DEFAULT_NWAKU * fix: let rln_credential_store_ready to raise exception for retry * fix: let register_rln to raise exception too, when credential fine still not available * fix: remove container stop * fix: tune down retry timeouts * fix: remove unnecessary f-string * test: reduce unnecessary variables init * test: undo reduce unnecessary variables init * test: increase timeout for rln_credential_store_ready * test: refactor kwargs handling into sanitize_docker_flags * fix: created RLN registration check - changed rln_credential_store_ready to function * fix: delete unnecessary ports from register_rln init * fix: remove wait for registration entirely * test: RLN_CREDENTIALS env var example * fix: check_rln_registration to raise exception * fix: use f-string at check_rln_registration * fix: add gen_step_id function * fix: RLN_CREDENTIALS as loaded from .env * fix: RLN_CREDENTIALS example, skipping test if not set * test: RLN with actions * fix: tune up RLN timeouts for Github runners * test: filesystem write to / * fix: change RLN data to be stored at working directory * fix: catch exception instead of "if not" - print out container log * fix: wrap logs into debug msg * fix: print log file * fix: remove additional logging * fix: exit RLN cred parsing sooner when not used - delete proper keys from default_args * fix: Allure report * fix: Discord notifications * fix: remove f-string * fix: delete test workflow
This commit is contained in:
Roman Zajic
GitHub
parent
dc25c6e51b
commit
8cb44f1201
|
|
@ -29,6 +29,7 @@ env:
|
|||
NODE_2: ${{ inputs.node2 }}
|
||||
ADDITIONAL_NODES: ${{ inputs.additional_nodes }}
|
||||
CALLER: ${{ inputs.caller || 'manual' }}
|
||||
RLN_CREDENTIALS: ${{ secrets.RLN_CREDENTIALS }}
|
||||
|
||||
jobs:
|
||||
|
||||
|
|
|
|||
|
|
@ -28,3 +28,7 @@ GATEWAY = get_env_var("GATEWAY", "172.18.0.1")
|
|||
RUNNING_IN_CI = get_env_var("CI")
|
||||
NODEKEY = get_env_var("NODEKEY", "30348dd51465150e04a5d9d932c72864c8967f806cce60b5d26afeca1e77eb68")
|
||||
API_REQUEST_TIMEOUT = get_env_var("API_REQUEST_TIMEOUT", 10)
|
||||
RLN_CREDENTIALS = get_env_var("RLN_CREDENTIALS")
|
||||
|
||||
# example for .env file
|
||||
# RLN_CREDENTIALS = {"rln-relay-cred-password": "password", "rln-relay-eth-client-address": "wss://sepolia.infura.io/ws/v3/api_key", "rln-relay-eth-contract-address": "0xF471d71E9b1455bBF4b85d475afb9BB0954A29c4", "rln-relay-eth-private-key-1": "1111111111111111111111111111111111111111111111111111111111111111", "rln-relay-eth-private-key-2": "1111111111111111111111111111111111111111111111111111111111111111"}
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
import uuid
|
||||
from datetime import datetime
|
||||
from time import sleep
|
||||
from src.libs.custom_logger import get_custom_logger
|
||||
import os
|
||||
|
|
@ -32,3 +34,7 @@ def attach_allure_file(file):
|
|||
def delay(num_seconds):
|
||||
logger.debug(f"Sleeping for {num_seconds} seconds")
|
||||
sleep(num_seconds)
|
||||
|
||||
|
||||
def gen_step_id():
|
||||
return f"{datetime.now().strftime('%Y-%m-%d_%H-%M-%S')}__{str(uuid.uuid4())}"
|
||||
|
|
|
|||
|
|
@ -31,17 +31,22 @@ class DockerManager:
|
|||
logger.debug(f"Network {network_name} created")
|
||||
return network
|
||||
|
||||
def start_container(self, image_name, ports, args, log_path, container_ip):
|
||||
def start_container(self, image_name, ports, args, log_path, container_ip, volumes):
|
||||
cli_args = []
|
||||
for key, value in args.items():
|
||||
if isinstance(value, list): # Check if value is a list
|
||||
cli_args.extend([f"--{key}={item}" for item in value]) # Add a command for each item in the list
|
||||
elif value is None:
|
||||
cli_args.append(f"{key}") # Add simple command as it is passed in the key
|
||||
else:
|
||||
cli_args.append(f"--{key}={value}") # Add a single command
|
||||
|
||||
port_bindings = {f"{port}/tcp": ("", port) for port in ports}
|
||||
logger.debug(f"Starting container with image {image_name}")
|
||||
logger.debug(f"Using args {cli_args}")
|
||||
container = self._client.containers.run(image_name, command=cli_args, ports=port_bindings, detach=True, remove=True, auto_remove=True)
|
||||
container = self._client.containers.run(
|
||||
image_name, command=cli_args, ports=port_bindings, detach=True, remove=True, auto_remove=True, volumes=volumes
|
||||
)
|
||||
|
||||
network = self._client.networks.get(NETWORK_NAME)
|
||||
network.connect(container, ipv4_address=container_ip)
|
||||
|
|
|
|||
|
|
@ -1,4 +1,8 @@
|
|||
import errno
|
||||
import json
|
||||
import os
|
||||
import string
|
||||
|
||||
import pytest
|
||||
import requests
|
||||
from src.libs.common import delay
|
||||
|
|
@ -12,6 +16,31 @@ from src.data_storage import DS
|
|||
logger = get_custom_logger(__name__)
|
||||
|
||||
|
||||
def select_private_key(prv_keys, key_id):
|
||||
for key in prv_keys:
|
||||
if key.endswith(key_id):
|
||||
return key
|
||||
|
||||
raise ValueError("No matching key was found")
|
||||
|
||||
|
||||
def sanitize_docker_flags(input_flags):
|
||||
output_flags = {}
|
||||
for key, value in input_flags.items():
|
||||
key = key.replace("_", "-")
|
||||
output_flags[key] = value
|
||||
|
||||
return output_flags
|
||||
|
||||
|
||||
@retry(stop=stop_after_delay(120), wait=wait_fixed(0.5), reraise=True)
|
||||
def rln_credential_store_ready(creds_file_path):
|
||||
if os.path.exists(creds_file_path):
|
||||
return True
|
||||
else:
|
||||
raise FileNotFoundError(errno.ENOENT, os.strerror(errno.ENOENT), creds_file_path)
|
||||
|
||||
|
||||
class WakuNode:
|
||||
def __init__(self, docker_image, docker_log_prefix=""):
|
||||
self._image_name = docker_image
|
||||
|
|
@ -32,6 +61,7 @@ class WakuNode:
|
|||
self._discv5_port = self._ports[3]
|
||||
self._metrics_port = self._ports[4]
|
||||
self._api = REST(self._rest_port)
|
||||
self._volumes = []
|
||||
|
||||
default_args = {
|
||||
"listen-address": "0.0.0.0",
|
||||
|
|
@ -49,6 +79,8 @@ class WakuNode:
|
|||
"peer-exchange": "true",
|
||||
"discv5-discovery": "true",
|
||||
"cluster-id": "0",
|
||||
"rln-creds-id": None,
|
||||
"rln-creds-source": None,
|
||||
}
|
||||
|
||||
if self.is_gowaku():
|
||||
|
|
@ -70,11 +102,23 @@ class WakuNode:
|
|||
else:
|
||||
raise NotImplementedError("Not implemented for this node type")
|
||||
|
||||
for key, value in kwargs.items():
|
||||
key = key.replace("_", "-")
|
||||
default_args[key] = value
|
||||
default_args.update(sanitize_docker_flags(kwargs))
|
||||
|
||||
rln_args, rln_creds_set, keystore_path = self.parse_rln_credentials(default_args, False)
|
||||
|
||||
del default_args["rln-creds-id"]
|
||||
del default_args["rln-creds-source"]
|
||||
|
||||
if rln_creds_set:
|
||||
rln_credential_store_ready(keystore_path)
|
||||
default_args.update(rln_args)
|
||||
else:
|
||||
logger.info(f"RLN credentials not set or credential store not available, starting without RLN")
|
||||
|
||||
self._container = self._docker_manager.start_container(
|
||||
self._docker_manager.image, self._ports, default_args, self._log_path, self._ext_ip, self._volumes
|
||||
)
|
||||
|
||||
self._container = self._docker_manager.start_container(self._docker_manager.image, self._ports, default_args, self._log_path, self._ext_ip)
|
||||
logger.debug(f"Started container from image {self._image_name}. REST: {self._rest_port}")
|
||||
DS.waku_nodes.append(self)
|
||||
delay(1) # if we fire requests to soon after starting the node will sometimes fail to start correctly
|
||||
|
|
@ -84,6 +128,41 @@ class WakuNode:
|
|||
logger.error(f"REST service did not become ready in time: {ex}")
|
||||
raise
|
||||
|
||||
@retry(stop=stop_after_delay(250), wait=wait_fixed(0.1), reraise=True)
|
||||
def register_rln(self, **kwargs):
|
||||
logger.debug("Registering RLN credentials...")
|
||||
self._docker_manager.create_network()
|
||||
self._ext_ip = self._docker_manager.generate_random_ext_ip()
|
||||
self._ports = self._docker_manager.generate_ports()
|
||||
self._rest_port = self._ports[0]
|
||||
self._api = REST(self._rest_port)
|
||||
self._volumes = []
|
||||
|
||||
default_args = {
|
||||
"rln-creds-id": None,
|
||||
"rln-creds-source": None,
|
||||
}
|
||||
|
||||
default_args.update(sanitize_docker_flags(kwargs))
|
||||
|
||||
rln_args, rln_creds_set, keystore_path = self.parse_rln_credentials(default_args, True)
|
||||
|
||||
if rln_creds_set:
|
||||
self._container = self._docker_manager.start_container(
|
||||
self._docker_manager.image, self._ports, rln_args, self._log_path, self._ext_ip, self._volumes
|
||||
)
|
||||
|
||||
logger.debug(f"Executed container from image {self._image_name}. REST: {self._rest_port} to register RLN")
|
||||
|
||||
logger.debug(f"Waiting for keystore {keystore_path}")
|
||||
try:
|
||||
rln_credential_store_ready(keystore_path)
|
||||
except Exception as ex:
|
||||
logger.error(f"File {keystore_path} with RLN credentials did not become available in time {ex}")
|
||||
raise
|
||||
else:
|
||||
logger.warn("RLN credentials not set, no action performed")
|
||||
|
||||
@retry(stop=stop_after_delay(5), wait=wait_fixed(0.1), reraise=True)
|
||||
def stop(self):
|
||||
if self._container:
|
||||
|
|
@ -186,3 +265,63 @@ class WakuNode:
|
|||
|
||||
def is_gowaku(self):
|
||||
return "go-waku" in self.image
|
||||
|
||||
def parse_rln_credentials(self, default_args, is_registration):
|
||||
rln_args = {}
|
||||
keystore_path = None
|
||||
|
||||
rln_creds_source = default_args["rln-creds-source"]
|
||||
selected_id = default_args["rln-creds-id"]
|
||||
|
||||
if rln_creds_source is None or selected_id is None:
|
||||
logger.debug(f"RLN credentials were not set")
|
||||
return rln_args, False, keystore_path
|
||||
|
||||
imported_creds = json.loads(rln_creds_source)
|
||||
|
||||
if len(imported_creds) < 4 or any(value is None for value in imported_creds.values()):
|
||||
logger.warn(f"One or more of required RLN credentials were not set properly")
|
||||
return rln_args, False, keystore_path
|
||||
|
||||
eth_private_key = select_private_key(imported_creds, selected_id)
|
||||
|
||||
current_working_directory = os.getcwd()
|
||||
|
||||
if self.is_nwaku():
|
||||
if is_registration:
|
||||
rln_args.update(
|
||||
{
|
||||
"generateRlnKeystore": None,
|
||||
"--execute": None,
|
||||
}
|
||||
)
|
||||
else:
|
||||
rln_args.update(
|
||||
{
|
||||
"rln-relay": "true",
|
||||
}
|
||||
)
|
||||
|
||||
rln_args.update(
|
||||
{
|
||||
"rln-relay-cred-path": "/keystore/keystore.json",
|
||||
"rln-relay-cred-password": imported_creds["rln-relay-cred-password"],
|
||||
"rln-relay-eth-client-address": imported_creds["rln-relay-eth-client-address"],
|
||||
"rln-relay-eth-contract-address": imported_creds["rln-relay-eth-contract-address"],
|
||||
"rln-relay-eth-private-key": imported_creds[eth_private_key],
|
||||
}
|
||||
)
|
||||
|
||||
keystore_path = current_working_directory + "/keystore_" + selected_id + "/keystore.json"
|
||||
|
||||
self._volumes.extend(
|
||||
[
|
||||
current_working_directory + "/rln_tree_" + selected_id + ":/etc/rln_tree",
|
||||
current_working_directory + "/keystore_" + selected_id + ":/keystore",
|
||||
]
|
||||
)
|
||||
|
||||
else:
|
||||
raise NotImplementedError("Not implemented for type other than Nim Waku ")
|
||||
|
||||
return rln_args, True, keystore_path
|
||||
|
|
|
|||
|
|
@ -1,12 +1,24 @@
|
|||
import inspect
|
||||
import os
|
||||
from datetime import datetime
|
||||
from uuid import uuid4
|
||||
|
||||
from src.libs.custom_logger import get_custom_logger
|
||||
from time import time
|
||||
import pytest
|
||||
import allure
|
||||
from src.libs.common import to_base64, delay
|
||||
from src.libs.common import to_base64, delay, gen_step_id
|
||||
from src.node.waku_message import WakuMessage
|
||||
from src.env_vars import NODE_1, NODE_2, ADDITIONAL_NODES, NODEKEY, RUNNING_IN_CI
|
||||
from src.node.waku_node import WakuNode
|
||||
from src.env_vars import (
|
||||
NODE_1,
|
||||
NODE_2,
|
||||
ADDITIONAL_NODES,
|
||||
NODEKEY,
|
||||
RUNNING_IN_CI,
|
||||
DEFAULT_NWAKU,
|
||||
RLN_CREDENTIALS,
|
||||
)
|
||||
from src.node.waku_node import WakuNode, rln_credential_store_ready
|
||||
from tenacity import retry, stop_after_delay, wait_fixed
|
||||
from src.test_data import VALID_PUBSUB_TOPICS
|
||||
|
||||
|
|
@ -34,6 +46,25 @@ class StepsRelay:
|
|||
self.node2.start(relay="true", discv5_bootstrap_node=self.enr_uri)
|
||||
self.main_nodes.extend([self.node1, self.node2])
|
||||
|
||||
@pytest.fixture(scope="function")
|
||||
def register_main_rln_relay_nodes(self, request):
|
||||
logger.debug(f"Registering RLN credentials: {inspect.currentframe().f_code.co_name}")
|
||||
self.node1 = WakuNode(DEFAULT_NWAKU, f"node1_{request.cls.test_id}")
|
||||
self.node1.register_rln(rln_creds_source=RLN_CREDENTIALS, rln_creds_id="1")
|
||||
self.node2 = WakuNode(DEFAULT_NWAKU, f"node2_{request.cls.test_id}")
|
||||
self.node2.register_rln(rln_creds_source=RLN_CREDENTIALS, rln_creds_id="2")
|
||||
self.main_nodes.extend([self.node1, self.node2])
|
||||
|
||||
@pytest.fixture(scope="function")
|
||||
def setup_main_rln_relay_nodes(self, request):
|
||||
logger.debug(f"Running fixture setup: {inspect.currentframe().f_code.co_name}")
|
||||
self.node1 = WakuNode(DEFAULT_NWAKU, f"node1_{request.cls.test_id}")
|
||||
self.node1.start(relay="true", nodekey=NODEKEY, rln_creds_source=RLN_CREDENTIALS, rln_creds_id="1")
|
||||
self.enr_uri = self.node1.get_enr_uri()
|
||||
self.node2 = WakuNode(DEFAULT_NWAKU, f"node2_{request.cls.test_id}")
|
||||
self.node2.start(relay="true", discv5_bootstrap_node=self.enr_uri, rln_creds_source=RLN_CREDENTIALS, rln_creds_id="2")
|
||||
self.main_nodes.extend([self.node1, self.node2])
|
||||
|
||||
@pytest.fixture(scope="function")
|
||||
def setup_optional_relay_nodes(self, request):
|
||||
logger.debug(f"Running fixture setup: {inspect.currentframe().f_code.co_name}")
|
||||
|
|
@ -128,3 +159,19 @@ class StepsRelay:
|
|||
def subscribe_and_publish_with_retry(self, node_list, pubsub_topic_list):
|
||||
self.ensure_relay_subscriptions_on_nodes(node_list, pubsub_topic_list)
|
||||
self.check_published_message_reaches_relay_peer()
|
||||
|
||||
@allure.step
|
||||
def register_rln_single_node(self, **kwargs):
|
||||
logger.debug("Registering RLN credentials for single node")
|
||||
self.node1 = WakuNode(DEFAULT_NWAKU, f"node1_{gen_step_id()}")
|
||||
self.node1.register_rln(rln_creds_source=kwargs["rln_creds_source"], rln_creds_id=kwargs["rln_creds_id"])
|
||||
|
||||
@allure.step
|
||||
def check_rln_registration(self, key_id):
|
||||
current_working_directory = os.getcwd()
|
||||
creds_file_path = f"{current_working_directory}/keystore_{key_id}/keystore.json"
|
||||
try:
|
||||
rln_credential_store_ready(creds_file_path)
|
||||
except Exception as ex:
|
||||
logger.error(f"Credentials at {creds_file_path} not available: {ex}")
|
||||
raise
|
||||
|
|
|
|||
|
|
@ -0,0 +1,24 @@
|
|||
import os
|
||||
import pytest
|
||||
|
||||
from src.env_vars import RLN_CREDENTIALS
|
||||
from src.libs.custom_logger import get_custom_logger
|
||||
from src.steps.relay import StepsRelay
|
||||
|
||||
logger = get_custom_logger(__name__)
|
||||
|
||||
|
||||
@pytest.mark.usefixtures()
|
||||
class TestRelayRLN(StepsRelay):
|
||||
def test_register_rln(self):
|
||||
logger.debug("Running register RLN test for main relay nodes")
|
||||
key_stores_found = 0
|
||||
|
||||
if RLN_CREDENTIALS is None:
|
||||
pytest.skip("RLN_CREDENTIALS not set, skipping test")
|
||||
|
||||
for k in range(1, 6):
|
||||
self.register_rln_single_node(rln_creds_source=RLN_CREDENTIALS, rln_creds_id=f"{k}")
|
||||
self.check_rln_registration(k)
|
||||
key_stores_found += 1
|
||||
assert key_stores_found == 5, f"Invalid number of RLN keystores found, expected 5 found {key_stores_found}"
|
||||
Loading…
Reference in New Issue