486 lines
99 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_relay_packet_loss_correlated_vs_uncorrelated with id: 2026-03-18_04-41-50__31edff42-d781-473e-b368-1749273557a3
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-03-18_04-41-50__31edff42-d781-473e-b368-1749273557a3__wakuorg_nwaku:latest.log
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node2_2026-03-18_04-41-50__31edff42-d781-473e-b368-1749273557a3__wakuorg_nwaku:latest.log
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node3_2026-03-18_04-41-50__31edff42-d781-473e-b368-1749273557a3__wakuorg_nwaku:latest.log
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node4_2026-03-18_04-41-50__31edff42-d781-473e-b368-1749273557a3__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.177.141
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['36133', '36134', '36135', '36136', '36137']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 36133:36133 -p 36134:36134 -p 36135:36135 -p 36136:36136 -p 36137:36137 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=36135 --rest-port=36133 --tcp-port=36134 --discv5-udp-port=36136 --rest-address=0.0.0.0 --nat=extip:172.18.177.141 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=4e540ddfb6ddf2a0187b2a647d5332e51506baecb8bd5e2c4b3faac2dee0be2c --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=36137 --metrics-logging=true --relay=true
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 224df8334fac. Exiting log stream.
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.177.141 waku c63b34f283f0b788ec2adc64dd1cd30327cd95309958de8dc026f037361c4218
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID c63b34f283f0. Setting up logs at ./log/docker/node1_2026-03-18_04-41-50__31edff42-d781-473e-b368-1749273557a3__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 36133
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container ab3582f568bc. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:36133/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:36133/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.177.141/tcp/36134/p2p/16Uiu2HAmR6TBfhnC6ifPdzjTnPCFRxrmw7X2qEdrpBpo1UAePSfp","/ip4/172.18.177.141/tcp/36135/ws/p2p/16Uiu2HAmR6TBfhnC6ifPdzjTnPCFRxrmw7X2qEdrpBpo1UAePSfp"],"enrUri":"enr:-L24QCyl7l6ReT6B5ZqGbRKl-sA8gFKv1YwdfJoX_NJMR6SpUeF-T4nLa7BCgZh2rzFZiOqLtv1MfTNTWOy-UzbCVW8CgmlkgnY0gmlwhKwSsY2KbXVsdGlhZGRyc5YACASsErGNBo0mAAoErBKxjQaNJ90DgnJzhQADAQAAiXNlY3AyNTZrMaEDuNIa0eQiQeYUMh9ScxHhAYIQLeSwmxabpBaOdlaVo2-DdGNwgo0mg3VkcIKNKIV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.187.166
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['63590', '63591', '63592', '63593', '63594']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 63590:63590 -p 63591:63591 -p 63592:63592 -p 63593:63593 -p 63594:63594 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=63592 --rest-port=63590 --tcp-port=63591 --discv5-udp-port=63593 --rest-address=0.0.0.0 --nat=extip:172.18.187.166 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=c596446d3dbffb51d9acf5d1d123bd7cedfdbe9a7c52ebf0af18afe520ad2ce7 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=63594 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QCyl7l6ReT6B5ZqGbRKl-sA8gFKv1YwdfJoX_NJMR6SpUeF-T4nLa7BCgZh2rzFZiOqLtv1MfTNTWOy-UzbCVW8CgmlkgnY0gmlwhKwSsY2KbXVsdGlhZGRyc5YACASsErGNBo0mAAoErBKxjQaNJ90DgnJzhQADAQAAiXNlY3AyNTZrMaEDuNIa0eQiQeYUMh9ScxHhAYIQLeSwmxabpBaOdlaVo2-DdGNwgo0mg3VkcIKNKIV3YWt1MgE
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.187.166 waku 791deee44077b5e3fd875ea7223a37b9160ff04d51285b12ab5a981f8042670c
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 791deee44077. Setting up logs at ./log/docker/node2_2026-03-18_04-41-50__31edff42-d781-473e-b368-1749273557a3__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 63590
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:63590/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:63590/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.187.166/tcp/63591/p2p/16Uiu2HAmL2bWNCPwMfwMPpAu8UL29gpYomcaGtgUmdS3ts7xknaA","/ip4/172.18.187.166/tcp/63592/ws/p2p/16Uiu2HAmL2bWNCPwMfwMPpAu8UL29gpYomcaGtgUmdS3ts7xknaA"],"enrUri":"enr:-L24QIYwoDxQpL00WdxmdAfTct3IOuvK2ShB1wLXBsA9ZteWIRiL5gWj17kd2u2IB40RTDD24zFnfmK6o__GZb4fmAICgmlkgnY0gmlwhKwSu6aKbXVsdGlhZGRyc5YACASsErumBvhnAAoErBK7pgb4aN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDbYrBzawo7pVvF4KgEX1Le2N45Tp2BrCZuNntmEOYp7-DdGNwgvhng3VkcIL4aYV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.84.123
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['3524', '3525', '3526', '3527', '3528']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 3524:3524 -p 3525:3525 -p 3526:3526 -p 3527:3527 -p 3528:3528 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=3526 --rest-port=3524 --tcp-port=3525 --discv5-udp-port=3527 --rest-address=0.0.0.0 --nat=extip:172.18.84.123 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=823f8eb0008397deae3cfcea03c54fc76fdbfbee6dcad29afac49363a155a5cf --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=3528 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QIYwoDxQpL00WdxmdAfTct3IOuvK2ShB1wLXBsA9ZteWIRiL5gWj17kd2u2IB40RTDD24zFnfmK6o__GZb4fmAICgmlkgnY0gmlwhKwSu6aKbXVsdGlhZGRyc5YACASsErumBvhnAAoErBK7pgb4aN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDbYrBzawo7pVvF4KgEX1Le2N45Tp2BrCZuNntmEOYp7-DdGNwgvhng3VkcIL4aYV3YWt1MgE
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.84.123 waku 2a3a236f73014491426f3faeef743806997373a986966842b225b7ab7719451e
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 2a3a236f7301. Setting up logs at ./log/docker/node3_2026-03-18_04-41-50__31edff42-d781-473e-b368-1749273557a3__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 3524
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:3524/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:3524/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.84.123/tcp/3525/p2p/16Uiu2HAkxjmf6BeiUCxmJwXpTuzEUy6hZVyyYkkNAH43XEVjkLxh","/ip4/172.18.84.123/tcp/3526/ws/p2p/16Uiu2HAkxjmf6BeiUCxmJwXpTuzEUy6hZVyyYkkNAH43XEVjkLxh"],"enrUri":"enr:-L24QKU2t2JRFYu1j5i4xDOcPWx20McrWVW_16silD9zIxryKfQCHgGuLuoJOz-Zyn7aegEXiFlvV4gBaB2HM43Dc-ECgmlkgnY0gmlwhKwSVHuKbXVsdGlhZGRyc5YACASsElR7Bg3FAAoErBJUewYNxt0DgnJzhQADAQAAiXNlY3AyNTZrMaECMTXvWImTHEDIP6gr2wUNCE-sqqDBoiSpIDgH-8bRAWKDdGNwgg3Fg3VkcIINx4V3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.87.136
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['23973', '23974', '23975', '23976', '23977']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 23973:23973 -p 23974:23974 -p 23975:23975 -p 23976:23976 -p 23977:23977 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=23975 --rest-port=23973 --tcp-port=23974 --discv5-udp-port=23976 --rest-address=0.0.0.0 --nat=extip:172.18.87.136 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=cc07886ce1b776dee87e2ac5dbde7a9507c5e48c1200dbd5cb8dca23c9abbfbb --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=23977 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QKU2t2JRFYu1j5i4xDOcPWx20McrWVW_16silD9zIxryKfQCHgGuLuoJOz-Zyn7aegEXiFlvV4gBaB2HM43Dc-ECgmlkgnY0gmlwhKwSVHuKbXVsdGlhZGRyc5YACASsElR7Bg3FAAoErBJUewYNxt0DgnJzhQADAQAAiXNlY3AyNTZrMaECMTXvWImTHEDIP6gr2wUNCE-sqqDBoiSpIDgH-8bRAWKDdGNwgg3Fg3VkcIINx4V3YWt1MgE
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.87.136 waku 287d951fa3a3f4b21ff393995b5f589d56ec675005cc3ab9d89b8bac8a78fa45
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 287d951fa3a3. Setting up logs at ./log/docker/node4_2026-03-18_04-41-50__31edff42-d781-473e-b368-1749273557a3__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 23973
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:23973/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:23973/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.87.136/tcp/23974/p2p/16Uiu2HAm7FbhXB7rnuNAhkFYxAsVHWFCk4B2txFj6h9dVBaMyi5r","/ip4/172.18.87.136/tcp/23975/ws/p2p/16Uiu2HAm7FbhXB7rnuNAhkFYxAsVHWFCk4B2txFj6h9dVBaMyi5r"],"enrUri":"enr:-L24QJz3WqqfkQ6MmIG5EctnvzagA1Gmn5XdwsXFbRXKIOGXPKNB7zJAIBru0DN2fcxOPHk4jsi891EzNXuCmmMNIvsCgmlkgnY0gmlwhKwSV4iKbXVsdGlhZGRyc5YACASsEleIBl2mAAoErBJXiAZdp90DgnJzhQADAQAAiXNlY3AyNTZrMaECr7eYULOsO7LS1IMpBQUDYU84_hcHzLJXw0GAprf34_2DdGNwgl2mg3VkcIJdqIV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:36133/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:63590/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:3524/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:23973/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:36133/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.187.166/tcp/34498/p2p/16Uiu2HAmL2bWNCPwMfwMPpAu8UL29gpYomcaGtgUmdS3ts7xknaA","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"UnknownOrigin"}]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:63590/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.177.141/tcp/36134/p2p/16Uiu2HAmR6TBfhnC6ifPdzjTnPCFRxrmw7X2qEdrpBpo1UAePSfp","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"Discv5"},{"multiaddr":"/ip4/172.18.84.123/tcp/39048/p2p/16Uiu2HAkxjmf6BeiUCxmJwXpTuzEUy6hZVyyYkkNAH43XEVjkLxh","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"UnknownOrigin"}]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:3524/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.87.136/tcp/43914/p2p/16Uiu2HAm7FbhXB7rnuNAhkFYxAsVHWFCk4B2txFj6h9dVBaMyi5r","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.187.166/tcp/63591/p2p/16Uiu2HAmL2bWNCPwMfwMPpAu8UL29gpYomcaGtgUmdS3ts7xknaA","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"Discv5"}]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:23973/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.84.123/tcp/3525/p2p/16Uiu2HAkxjmf6BeiUCxmJwXpTuzEUy6hZVyyYkkNAH43XEVjkLxh","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"Discv5"}]'
DEBUG  src.libs.common:common.py:47 Sleeping for 20 seconds
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '8326', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '8326', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'loss', '50.0%']
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '8326', '-n', 'tc', '-s', 'qdisc', 'show', 'dev', 'eth0']
DEBUG  src.steps.network_conditions:network_conditions.py:71 tc -s qdisc show dev eth0:
qdisc netem 8002: root refcnt 5 limit 1000 loss 50%
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:23973/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:36133/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:36133/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:36133/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:36133/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:36133/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.libs.common:common.py:47 Sleeping for 30.0 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:23973/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773808935661074107,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773808938351687752,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773808938362514398,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773808945967613207,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773808947003094703,"ephemeral":false,"proof":""}]'
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '8326', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '8326', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '8326', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'loss', '50.0%', '75.0%']
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:23973/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:36133/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id c63b34f283f0
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 791deee44077
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 2a3a236f7301
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container c63b34f283f0. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 287d951fa3a3
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 791deee44077. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:73 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:74 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-03-18_04-41-50__31edff42-d781-473e-b368-1749273557a3__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-03-18_04-41-50__31edff42-d781-473e-b368-1749273557a3__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node4_2026-03-18_04-41-50__31edff42-d781-473e-b368-1749273557a3__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node3_2026-03-18_04-41-50__31edff42-d781-473e-b368-1749273557a3__wakuorg_nwaku:latest.log
DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_relay_packet_loss_correlated_vs_uncorrelated with id: 2026-03-18_04-43-21__56097d9a-4a30-42ae-912f-fcc56a53182b
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-03-18_04-43-21__56097d9a-4a30-42ae-912f-fcc56a53182b__wakuorg_nwaku:latest.log
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node2_2026-03-18_04-43-21__56097d9a-4a30-42ae-912f-fcc56a53182b__wakuorg_nwaku:latest.log
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node3_2026-03-18_04-43-21__56097d9a-4a30-42ae-912f-fcc56a53182b__wakuorg_nwaku:latest.log
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node4_2026-03-18_04-43-21__56097d9a-4a30-42ae-912f-fcc56a53182b__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.180.127
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['34912', '34913', '34914', '34915', '34916']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 34912:34912 -p 34913:34913 -p 34914:34914 -p 34915:34915 -p 34916:34916 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=34914 --rest-port=34912 --tcp-port=34913 --discv5-udp-port=34915 --rest-address=0.0.0.0 --nat=extip:172.18.180.127 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=cc4fd4ee1bb753efaaa9b0edfabbae7ffdec042afffeb4bbbddecb5a43ea1a6c --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=34916 --metrics-logging=true --relay=true
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 2a3a236f7301. Exiting log stream.
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.180.127 waku 2aeac63a87c275a4958e377e772d1e51baf765625a784c0c8373942f0cda3d8c
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 2aeac63a87c2. Setting up logs at ./log/docker/node1_2026-03-18_04-43-21__56097d9a-4a30-42ae-912f-fcc56a53182b__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 34912
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 287d951fa3a3. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:34912/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:34912/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.180.127/tcp/34913/p2p/16Uiu2HAkzj6ujXeg2NGbo12S5fELGTwjyx7dkokzwiYhLrVXvHUb","/ip4/172.18.180.127/tcp/34914/ws/p2p/16Uiu2HAkzj6ujXeg2NGbo12S5fELGTwjyx7dkokzwiYhLrVXvHUb"],"enrUri":"enr:-L24QATbDsToJWPuO2pzyksdNHdF1_1SbU1BMVq0eLFp4htyO0K0D3zDnPv6kxSNa_breVFniZBwcsmJ_kc4g6hOviICgmlkgnY0gmlwhKwStH-KbXVsdGlhZGRyc5YACASsErR_BohhAAoErBK0fwaIYt0DgnJzhQADAQAAiXNlY3AyNTZrMaECTsF_ML-gJB4B52DX9maHQusoHrfgq-jo5jeNLZbt0-iDdGNwgohhg3VkcIKIY4V3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.176.40
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['55576', '55577', '55578', '55579', '55580']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 55576:55576 -p 55577:55577 -p 55578:55578 -p 55579:55579 -p 55580:55580 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=55578 --rest-port=55576 --tcp-port=55577 --discv5-udp-port=55579 --rest-address=0.0.0.0 --nat=extip:172.18.176.40 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=fafae0aff1daedefdc43faf9c3f7fe3dffcb1a6fd3fc7d8daa0e4d7afb9e4f71 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=55580 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QATbDsToJWPuO2pzyksdNHdF1_1SbU1BMVq0eLFp4htyO0K0D3zDnPv6kxSNa_breVFniZBwcsmJ_kc4g6hOviICgmlkgnY0gmlwhKwStH-KbXVsdGlhZGRyc5YACASsErR_BohhAAoErBK0fwaIYt0DgnJzhQADAQAAiXNlY3AyNTZrMaECTsF_ML-gJB4B52DX9maHQusoHrfgq-jo5jeNLZbt0-iDdGNwgohhg3VkcIKIY4V3YWt1MgE
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.176.40 waku 1c250a42fefe45e108cd4b01d270c98b58ca5f2257b7d91f6ee97bf4cddcdf39
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 1c250a42fefe. Setting up logs at ./log/docker/node2_2026-03-18_04-43-21__56097d9a-4a30-42ae-912f-fcc56a53182b__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 55576
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:55576/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:55576/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.176.40/tcp/55577/p2p/16Uiu2HAm3pftx5wdjfxFksCbtHLAT9vA8HHnkfZ7D7bjNY9P331o","/ip4/172.18.176.40/tcp/55578/ws/p2p/16Uiu2HAm3pftx5wdjfxFksCbtHLAT9vA8HHnkfZ7D7bjNY9P331o"],"enrUri":"enr:-L24QLuAyI-FcR-R64atlp4LsAUy39YacOKHVFQcE6VDSomAWHxBh7t-O6m486oVND5o_CwuPQtVPdcnoz2sTEhXjtACgmlkgnY0gmlwhKwSsCiKbXVsdGlhZGRyc5YACASsErAoBtkZAAoErBKwKAbZGt0DgnJzhQADAQAAiXNlY3AyNTZrMaECfMHCdZT4kSGri-tuatjQikeJam2kAxPx-ySo6gUp72aDdGNwgtkZg3VkcILZG4V3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.251.238
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['43977', '43978', '43979', '43980', '43981']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 43977:43977 -p 43978:43978 -p 43979:43979 -p 43980:43980 -p 43981:43981 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=43979 --rest-port=43977 --tcp-port=43978 --discv5-udp-port=43980 --rest-address=0.0.0.0 --nat=extip:172.18.251.238 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=ac75eafae5b6ababc1de47a9ca4abc9757dfe85d626c6da13bae040ca06da8ac --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=43981 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QLuAyI-FcR-R64atlp4LsAUy39YacOKHVFQcE6VDSomAWHxBh7t-O6m486oVND5o_CwuPQtVPdcnoz2sTEhXjtACgmlkgnY0gmlwhKwSsCiKbXVsdGlhZGRyc5YACASsErAoBtkZAAoErBKwKAbZGt0DgnJzhQADAQAAiXNlY3AyNTZrMaECfMHCdZT4kSGri-tuatjQikeJam2kAxPx-ySo6gUp72aDdGNwgtkZg3VkcILZG4V3YWt1MgE
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.251.238 waku 2db3861ddadaed88319500706129a92e5fe93b3746f556570e253845f715413b
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 2db3861ddada. Setting up logs at ./log/docker/node3_2026-03-18_04-43-21__56097d9a-4a30-42ae-912f-fcc56a53182b__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 43977
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43977/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43977/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.251.238/tcp/43978/p2p/16Uiu2HAkuz43KRNx3be5Viq4s462xGe3pzvvLtJgr3PPsNvN44Vy","/ip4/172.18.251.238/tcp/43979/ws/p2p/16Uiu2HAkuz43KRNx3be5Viq4s462xGe3pzvvLtJgr3PPsNvN44Vy"],"enrUri":"enr:-L24QLMtkzT2uci_ZoyU_c8OwqRTMPnh2jyXsvT9N--nqlP1flgUVzB8B7IuOtDKS7gdhDnWx6t14tF29CV2qe7zljMCgmlkgnY0gmlwhKwS--6KbXVsdGlhZGRyc5YACASsEvvuBqvKAAoErBL77gary90DgnJzhQADAQAAiXNlY3AyNTZrMaECCEuHs8k9wVlvO7cg_6naFEDBATIBowpAu882yPjWaySDdGNwgqvKg3VkcIKrzIV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.150.146
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['41151', '41152', '41153', '41154', '41155']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 41151:41151 -p 41152:41152 -p 41153:41153 -p 41154:41154 -p 41155:41155 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=41153 --rest-port=41151 --tcp-port=41152 --discv5-udp-port=41154 --rest-address=0.0.0.0 --nat=extip:172.18.150.146 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=0e8aa4cee4c0ddd1bab799edc24e30a12e2e2e36ffd6cae2500bccec87fece7c --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=41155 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QLMtkzT2uci_ZoyU_c8OwqRTMPnh2jyXsvT9N--nqlP1flgUVzB8B7IuOtDKS7gdhDnWx6t14tF29CV2qe7zljMCgmlkgnY0gmlwhKwS--6KbXVsdGlhZGRyc5YACASsEvvuBqvKAAoErBL77gary90DgnJzhQADAQAAiXNlY3AyNTZrMaECCEuHs8k9wVlvO7cg_6naFEDBATIBowpAu882yPjWaySDdGNwgqvKg3VkcIKrzIV3YWt1MgE
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.150.146 waku 25d57fa71f6fe4450472aa8d3256db73e0fecdd882370560d10defbf44af79af
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 25d57fa71f6f. Setting up logs at ./log/docker/node4_2026-03-18_04-43-21__56097d9a-4a30-42ae-912f-fcc56a53182b__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 41151
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:41151/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:41151/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.150.146/tcp/41152/p2p/16Uiu2HAmUF6mrmaDvrUJ6aUk13RTbFkdMa9PD7BQwehuEJzuVftX","/ip4/172.18.150.146/tcp/41153/ws/p2p/16Uiu2HAmUF6mrmaDvrUJ6aUk13RTbFkdMa9PD7BQwehuEJzuVftX"],"enrUri":"enr:-L24QPsFGr1ogPXEhV0iEhmuebTcUeTOj4jDDYN1qIIvyVVmbxwOS8PsTkE4n8RRWSYYr9RQChPjF5GjZNIllrE22_cCgmlkgnY0gmlwhKwSlpKKbXVsdGlhZGRyc5YACASsEpaSBqDAAAoErBKWkgagwd0DgnJzhQADAQAAiXNlY3AyNTZrMaED55xQiVkFUc89jjrw7_Oml2M8uIf2ALheMWMjn6VpvgSDdGNwgqDAg3VkcIKgwoV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:34912/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:55576/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:43977/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:41151/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:34912/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.176.40/tcp/34644/p2p/16Uiu2HAm3pftx5wdjfxFksCbtHLAT9vA8HHnkfZ7D7bjNY9P331o","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"UnknownOrigin"}]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:55576/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.180.127/tcp/34913/p2p/16Uiu2HAkzj6ujXeg2NGbo12S5fELGTwjyx7dkokzwiYhLrVXvHUb","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"Discv5"},{"multiaddr":"/ip4/172.18.251.238/tcp/60500/p2p/16Uiu2HAkuz43KRNx3be5Viq4s462xGe3pzvvLtJgr3PPsNvN44Vy","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"UnknownOrigin"}]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43977/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.150.146/tcp/57158/p2p/16Uiu2HAmUF6mrmaDvrUJ6aUk13RTbFkdMa9PD7BQwehuEJzuVftX","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.176.40/tcp/55577/p2p/16Uiu2HAm3pftx5wdjfxFksCbtHLAT9vA8HHnkfZ7D7bjNY9P331o","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"Discv5"}]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:41151/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.251.238/tcp/43978/p2p/16Uiu2HAkuz43KRNx3be5Viq4s462xGe3pzvvLtJgr3PPsNvN44Vy","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"Discv5"}]'
DEBUG  src.libs.common:common.py:47 Sleeping for 20 seconds
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '10177', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '10177', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'loss', '50.0%']
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '10177', '-n', 'tc', '-s', 'qdisc', 'show', 'dev', 'eth0']
DEBUG  src.steps.network_conditions:network_conditions.py:71 tc -s qdisc show dev eth0:
qdisc netem 8004: root refcnt 5 limit 1000 loss 50%
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:41151/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:34912/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:34912/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:34912/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:34912/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:34912/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.libs.common:common.py:47 Sleeping for 30.0 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:41151/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773809026483409063,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773809026695537011,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773809029199628691,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773809030471655274,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773809030486055639,"ephemeral":false,"proof":""}]'
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '10177', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '10177', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '10177', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'loss', '50.0%', '75.0%']
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:41151/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:34912/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:34912/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 2aeac63a87c2
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 1c250a42fefe
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 2db3861ddada
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 2aeac63a87c2. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 25d57fa71f6f
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 1c250a42fefe. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 2db3861ddada. Exiting log stream.
DEBUG  tests.conftest:conftest.py:73 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:74 Test failed, attempting to attach logs to the allure reports
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-03-18_04-43-21__56097d9a-4a30-42ae-912f-fcc56a53182b__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node3_2026-03-18_04-43-21__56097d9a-4a30-42ae-912f-fcc56a53182b__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node4_2026-03-18_04-43-21__56097d9a-4a30-42ae-912f-fcc56a53182b__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-03-18_04-43-21__56097d9a-4a30-42ae-912f-fcc56a53182b__wakuorg_nwaku:latest.log
DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_relay_packet_loss_correlated_vs_uncorrelated with id: 2026-03-18_04-45-03__2a724c8e-aeb9-4e17-8c4c-5d702405157f
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-03-18_04-45-03__2a724c8e-aeb9-4e17-8c4c-5d702405157f__wakuorg_nwaku:latest.log
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node2_2026-03-18_04-45-03__2a724c8e-aeb9-4e17-8c4c-5d702405157f__wakuorg_nwaku:latest.log
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node3_2026-03-18_04-45-03__2a724c8e-aeb9-4e17-8c4c-5d702405157f__wakuorg_nwaku:latest.log
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node4_2026-03-18_04-45-03__2a724c8e-aeb9-4e17-8c4c-5d702405157f__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.1.179
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['40890', '40891', '40892', '40893', '40894']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 40890:40890 -p 40891:40891 -p 40892:40892 -p 40893:40893 -p 40894:40894 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=40892 --rest-port=40890 --tcp-port=40891 --discv5-udp-port=40893 --rest-address=0.0.0.0 --nat=extip:172.18.1.179 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=9ae5eca1da36a7ccd6c1bc6b6fac07f60a6acef6d7aea0f13dd1cb4d56958af9 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=40894 --metrics-logging=true --relay=true
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.1.179 waku 9224a614d404e08d9fe06680598ace5e5f18a99b4ade0294d24f867ccc209bd0
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 9224a614d404. Setting up logs at ./log/docker/node1_2026-03-18_04-45-03__2a724c8e-aeb9-4e17-8c4c-5d702405157f__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 40890
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 25d57fa71f6f. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:40890/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:40890/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.1.179/tcp/40891/p2p/16Uiu2HAmQy8ayZW4AniZc699GWMyvzdQmWdWsBiJrw1XfBNQruDN","/ip4/172.18.1.179/tcp/40892/ws/p2p/16Uiu2HAmQy8ayZW4AniZc699GWMyvzdQmWdWsBiJrw1XfBNQruDN"],"enrUri":"enr:-L24QOBllqdPIIGf3HJiuTUyVv5Z6qLiMBdjrKhA8RJB2vLoV9FnCfZ0WG5cfaV8KpzAW_MaZ4n_1XigBBQiGHO-8hoCgmlkgnY0gmlwhKwSAbOKbXVsdGlhZGRyc5YACASsEgGzBp-7AAoErBIBswafvN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDtvIClxR7ZMJYSS1nTJ4FjAcqCFxA6V1P-EDRaQ7rndWDdGNwgp-7g3VkcIKfvYV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.93.78
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['59799', '59800', '59801', '59802', '59803']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 59799:59799 -p 59800:59800 -p 59801:59801 -p 59802:59802 -p 59803:59803 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=59801 --rest-port=59799 --tcp-port=59800 --discv5-udp-port=59802 --rest-address=0.0.0.0 --nat=extip:172.18.93.78 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=cfe6fb69b0bdcd805b3f752a47d5d1bd555e0b2386fee5cb4fe4e1ffd0f1a504 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=59803 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QOBllqdPIIGf3HJiuTUyVv5Z6qLiMBdjrKhA8RJB2vLoV9FnCfZ0WG5cfaV8KpzAW_MaZ4n_1XigBBQiGHO-8hoCgmlkgnY0gmlwhKwSAbOKbXVsdGlhZGRyc5YACASsEgGzBp-7AAoErBIBswafvN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDtvIClxR7ZMJYSS1nTJ4FjAcqCFxA6V1P-EDRaQ7rndWDdGNwgp-7g3VkcIKfvYV3YWt1MgE
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.93.78 waku 9ea21438a50d3cf6dbd1b26302abe75f8c29b79710c315f51c58928ef87ef744
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 9ea21438a50d. Setting up logs at ./log/docker/node2_2026-03-18_04-45-03__2a724c8e-aeb9-4e17-8c4c-5d702405157f__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 59799
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:59799/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:59799/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.93.78/tcp/59800/p2p/16Uiu2HAmCQCnqNBTP1cdb9Ek3qqjxP3xK8iSpqFQynKdXp6UDtHS","/ip4/172.18.93.78/tcp/59801/ws/p2p/16Uiu2HAmCQCnqNBTP1cdb9Ek3qqjxP3xK8iSpqFQynKdXp6UDtHS"],"enrUri":"enr:-L24QMhw_eFawsgzf4aD4Y95ENC-qcTv5MnaYgTMPuHkKtfrcJBjI1K6p2dNKmDCxLFUTF9gQKe5WptEb0Dxx3Ky31ACgmlkgnY0gmlwhKwSXU6KbXVsdGlhZGRyc5YACASsEl1OBumYAAoErBJdTgbpmd0DgnJzhQADAQAAiXNlY3AyNTZrMaEC_DZas3JB6QUM6d3G3i363Q0AKweEXMUZswR1BOXvJVWDdGNwgumYg3VkcILpmoV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.120.123
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['62758', '62759', '62760', '62761', '62762']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 62758:62758 -p 62759:62759 -p 62760:62760 -p 62761:62761 -p 62762:62762 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=62760 --rest-port=62758 --tcp-port=62759 --discv5-udp-port=62761 --rest-address=0.0.0.0 --nat=extip:172.18.120.123 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=fd5553e985cdcac5a9ebb02f613cd33e0fdfe880bba786dc71f6893cc86bbc81 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=62762 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QMhw_eFawsgzf4aD4Y95ENC-qcTv5MnaYgTMPuHkKtfrcJBjI1K6p2dNKmDCxLFUTF9gQKe5WptEb0Dxx3Ky31ACgmlkgnY0gmlwhKwSXU6KbXVsdGlhZGRyc5YACASsEl1OBumYAAoErBJdTgbpmd0DgnJzhQADAQAAiXNlY3AyNTZrMaEC_DZas3JB6QUM6d3G3i363Q0AKweEXMUZswR1BOXvJVWDdGNwgumYg3VkcILpmoV3YWt1MgE
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.120.123 waku 8697c7585c9cb92ee609f07a230745723a23875f1e7fcf3b66fc364b149d9049
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 8697c7585c9c. Setting up logs at ./log/docker/node3_2026-03-18_04-45-03__2a724c8e-aeb9-4e17-8c4c-5d702405157f__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 62758
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:62758/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:62758/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.120.123/tcp/62759/p2p/16Uiu2HAmSKC7v2odPWp5PhcXh5GdDJpeshcRYj4MBRVvPxD2C6BS","/ip4/172.18.120.123/tcp/62760/ws/p2p/16Uiu2HAmSKC7v2odPWp5PhcXh5GdDJpeshcRYj4MBRVvPxD2C6BS"],"enrUri":"enr:-L24QL4dJD4dTTTYM8xdTOJAt54pOJsYtuzHWXqyKPgP-vU9Nv7_-j9ybqto_hEnKFCQr7fANhXyv-ebLap44pYdv-ECgmlkgnY0gmlwhKwSeHuKbXVsdGlhZGRyc5YACASsEnh7BvUnAAoErBJ4ewb1KN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDyvFO-VGDHWkE2QmJf2v5wajI8l2ZqqHaHru_7dzo0BmDdGNwgvUng3VkcIL1KYV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.88.75
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['38877', '38878', '38879', '38880', '38881']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 38877:38877 -p 38878:38878 -p 38879:38879 -p 38880:38880 -p 38881:38881 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=38879 --rest-port=38877 --tcp-port=38878 --discv5-udp-port=38880 --rest-address=0.0.0.0 --nat=extip:172.18.88.75 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=3fedc3deca034951d094fbeb4ace6243ad11a37c15a81ad37fa6faee737cda93 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=38881 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QL4dJD4dTTTYM8xdTOJAt54pOJsYtuzHWXqyKPgP-vU9Nv7_-j9ybqto_hEnKFCQr7fANhXyv-ebLap44pYdv-ECgmlkgnY0gmlwhKwSeHuKbXVsdGlhZGRyc5YACASsEnh7BvUnAAoErBJ4ewb1KN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDyvFO-VGDHWkE2QmJf2v5wajI8l2ZqqHaHru_7dzo0BmDdGNwgvUng3VkcIL1KYV3YWt1MgE
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.88.75 waku fe40c3b47818889dbebc27550afbd59ca4e103556dfa50d35e37364357782aeb
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID fe40c3b47818. Setting up logs at ./log/docker/node4_2026-03-18_04-45-03__2a724c8e-aeb9-4e17-8c4c-5d702405157f__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 38877
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38877/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38877/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.88.75/tcp/38878/p2p/16Uiu2HAm8XCrKCnDehbzZuiNod87GZUPWp7xtwvPVXAGRvvGXw7h","/ip4/172.18.88.75/tcp/38879/ws/p2p/16Uiu2HAm8XCrKCnDehbzZuiNod87GZUPWp7xtwvPVXAGRvvGXw7h"],"enrUri":"enr:-L24QIJzg-ULgyJNOYqD3EBTikLop5gj8sLCCjCBorlMyjfzT_pNwcVxqsuRqQHq7hC5RHgK-mSZMU5Evn4XbFDXD24CgmlkgnY0gmlwhKwSWEuKbXVsdGlhZGRyc5YACASsElhLBpfeAAoErBJYSwaX390DgnJzhQADAQAAiXNlY3AyNTZrMaECwpK9dzYIXzWRFpDnEO6zMAxmyK7SpqOj7wOh002hDhyDdGNwgpfeg3VkcIKX4IV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40890/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:59799/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:62758/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:38877/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:40890/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.93.78/tcp/48994/p2p/16Uiu2HAmCQCnqNBTP1cdb9Ek3qqjxP3xK8iSpqFQynKdXp6UDtHS","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"UnknownOrigin"}]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:59799/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.120.123/tcp/58770/p2p/16Uiu2HAmSKC7v2odPWp5PhcXh5GdDJpeshcRYj4MBRVvPxD2C6BS","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.1.179/tcp/40891/p2p/16Uiu2HAmQy8ayZW4AniZc699GWMyvzdQmWdWsBiJrw1XfBNQruDN","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"Discv5"}]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:62758/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.88.75/tcp/51760/p2p/16Uiu2HAm8XCrKCnDehbzZuiNod87GZUPWp7xtwvPVXAGRvvGXw7h","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"UnknownOrigin"},{"multiaddr":"/ip4/172.18.93.78/tcp/59800/p2p/16Uiu2HAmCQCnqNBTP1cdb9Ek3qqjxP3xK8iSpqFQynKdXp6UDtHS","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"Discv5"}]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38877/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.120.123/tcp/62759/p2p/16Uiu2HAmSKC7v2odPWp5PhcXh5GdDJpeshcRYj4MBRVvPxD2C6BS","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/relay/2.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.38.0-rc.0-2-g85a7bf","origin":"Discv5"}]'
DEBUG  src.libs.common:common.py:47 Sleeping for 20 seconds
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '12005', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '12005', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'loss', '50.0%']
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '12005', '-n', 'tc', '-s', 'qdisc', 'show', 'dev', 'eth0']
DEBUG  src.steps.network_conditions:network_conditions.py:71 tc -s qdisc show dev eth0:
qdisc netem 8006: root refcnt 5 limit 1000 loss 50%
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38877/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40890/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40890/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40890/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40890/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40890/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.libs.common:common.py:47 Sleeping for 30.0 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38877/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773809128571170424,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773809128590275995,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773809129863316989,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773809129874960899,"ephemeral":false,"proof":""},{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1773809132975807423,"ephemeral":false,"proof":""}]'
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '12005', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '12005', '-n', 'tc', 'qdisc', 'del', 'dev', 'eth0', 'root']
INFO  src.steps.network_conditions:network_conditions.py:22 TC exec: ['sudo', '-n', 'nsenter', '-t', '12005', '-n', 'tc', 'qdisc', 'add', 'dev', 'eth0', 'root', 'netem', 'loss', '50.0%', '75.0%']
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:38877/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[]'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40890/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40890/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40890/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40890/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 9224a614d404
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 9ea21438a50d
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 8697c7585c9c
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 9224a614d404. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id fe40c3b47818
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 9ea21438a50d. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  tests.conftest:conftest.py:73 Running fixture teardown: attach_logs_on_fail
DEBUG  tests.conftest:conftest.py:74 Test failed, attempting to attach logs to the allure reports
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 8697c7585c9c. Exiting log stream.
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node2_2026-03-18_04-45-03__2a724c8e-aeb9-4e17-8c4c-5d702405157f__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node3_2026-03-18_04-45-03__2a724c8e-aeb9-4e17-8c4c-5d702405157f__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node1_2026-03-18_04-45-03__2a724c8e-aeb9-4e17-8c4c-5d702405157f__wakuorg_nwaku:latest.log
DEBUG  src.libs.common:common.py:42 Attaching file ./log/docker/node4_2026-03-18_04-45-03__2a724c8e-aeb9-4e17-8c4c-5d702405157f__wakuorg_nwaku:latest.log