192 lines
35 KiB
Plaintext

2026-03-05 04:33:14.941 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-05 04:33:14.941 DEBUG [tests.conftest] Running test: test_first_node_to_start_publishes with id: 2026-03-05_04-33-14__a9d87193-71d9-45a9-b2dd-91b3866fcf1e
2026-03-05 04:33:14.942 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-05 04:33:14.942 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-05 04:33:14.943 DEBUG [src.steps.relay] Running fixture setup: setup_main_relay_nodes
2026-03-05 04:33:14.950 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:33:14.950 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-05_04-33-14__a9d87193-71d9-45a9-b2dd-91b3866fcf1e__wakuorg_nwaku:latest.log
2026-03-05 04:33:14.950 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:33:14.950 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:33:14.952 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:33:14.952 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.119.59
2026-03-05 04:33:14.952 DEBUG [src.node.docker_mananger] Generated ports ['33022', '33023', '33024', '33025', '33026']
2026-03-05 04:33:14.952 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:33:14.952 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:33:14.953 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:33:14.953 DEBUG [src.node.docker_mananger] docker run -i -t -p 33022:33022 -p 33023:33023 -p 33024:33024 -p 33025:33025 -p 33026:33026 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=33024 --rest-port=33022 --tcp-port=33023 --discv5-udp-port=33025 --rest-address=0.0.0.0 --nat=extip:172.18.119.59 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=c7ef0b5b5b1d7fc5ede85ed1d60e46eebfcddc0b0b1d5f76a8bdb2059abb1f0b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=33026 --metrics-logging=true --relay=true
2026-03-05 04:33:15.147 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.119.59 waku 11fb96cd5e4f61e5c1bea53c431b59dcb74429e10141a71ea3c91d284bbfe988
2026-03-05 04:33:15.181 DEBUG [src.node.docker_mananger] Container started with ID 11fb96cd5e4f. Setting up logs at ./log/docker/node1_2026-03-05_04-33-14__a9d87193-71d9-45a9-b2dd-91b3866fcf1e__wakuorg_nwaku:latest.log
2026-03-05 04:33:15.184 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 33022
2026-03-05 04:33:15.185 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:33:15.188 ERROR [src.node.docker_mananger] Max retries reached for container 950f662154ac. Exiting log stream.
2026-03-05 04:33:15.725 ERROR [src.node.docker_mananger] Max retries reached for container 9a301e59d68a. Exiting log stream.
2026-03-05 04:33:16.185 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33022/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:16.188 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:33:16.188 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:33:16.188 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33022/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:16.191 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.119.59/tcp/33023/p2p/16Uiu2HAkyYEc8JixCsDfFzV1owQ6KtuotgDbnA7acgE3TRR5MD5f","/ip4/172.18.119.59/tcp/33024/ws/p2p/16Uiu2HAkyYEc8JixCsDfFzV1owQ6KtuotgDbnA7acgE3TRR5MD5f"],"enrUri":"enr:-L24QC5WvvyK5m-lQH1rzxM_0VzrIaSaWExvOLQeweZMMvfHTdexgbaiT1a53CrVTUZ1GgnX4AI0oV1BmYixxEE_dmoCgmlkgnY0gmlwhKwSdzuKbXVsdGlhZGRyc5YACASsEnc7BoD_AAoErBJ3OwaBAN0DgnJzhQADAQAAiXNlY3AyNTZrMaECPR0fllwnk1IMXM8PJZ0wMfzSbMY5y-TTmI5QrgTaRB6DdGNwgoD_g3VkcIKBAYV3YWt1MgE"}'
2026-03-05 04:33:16.191 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:33:16.197 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:33:16.198 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-05_04-33-14__a9d87193-71d9-45a9-b2dd-91b3866fcf1e__wakuorg_nwaku:latest.log
2026-03-05 04:33:16.198 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:33:16.198 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:33:16.199 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:33:16.199 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.135.170
2026-03-05 04:33:16.200 DEBUG [src.node.docker_mananger] Generated ports ['42136', '42137', '42138', '42139', '42140']
2026-03-05 04:33:16.200 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:33:16.200 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:33:16.200 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:33:16.200 DEBUG [src.node.docker_mananger] docker run -i -t -p 42136:42136 -p 42137:42137 -p 42138:42138 -p 42139:42139 -p 42140:42140 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=42138 --rest-port=42136 --tcp-port=42137 --discv5-udp-port=42139 --rest-address=0.0.0.0 --nat=extip:172.18.135.170 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=e24efc2fad4adfb9b01e8afceefcaff71c58ef8bafbdede42bda361e8dfab5d6 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=42140 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QC5WvvyK5m-lQH1rzxM_0VzrIaSaWExvOLQeweZMMvfHTdexgbaiT1a53CrVTUZ1GgnX4AI0oV1BmYixxEE_dmoCgmlkgnY0gmlwhKwSdzuKbXVsdGlhZGRyc5YACASsEnc7BoD_AAoErBJ3OwaBAN0DgnJzhQADAQAAiXNlY3AyNTZrMaECPR0fllwnk1IMXM8PJZ0wMfzSbMY5y-TTmI5QrgTaRB6DdGNwgoD_g3VkcIKBAYV3YWt1MgE
2026-03-05 04:33:16.398 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.135.170 waku c8342bc651846edb486e9b98a87b761df3b12c1adeeafd9e19802d397dfd8fc0
2026-03-05 04:33:16.431 DEBUG [src.node.docker_mananger] Container started with ID c8342bc65184. Setting up logs at ./log/docker/node2_2026-03-05_04-33-14__a9d87193-71d9-45a9-b2dd-91b3866fcf1e__wakuorg_nwaku:latest.log
2026-03-05 04:33:16.431 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 42136
2026-03-05 04:33:16.431 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:33:17.432 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42136/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:17.454 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:33:17.454 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:33:17.455 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42136/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:17.461 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.135.170/tcp/42137/p2p/16Uiu2HAm677NB9PeBm4YDWA5rLqGDegMCk7VDBWw2nUtCxXxV3Rp","/ip4/172.18.135.170/tcp/42138/ws/p2p/16Uiu2HAm677NB9PeBm4YDWA5rLqGDegMCk7VDBWw2nUtCxXxV3Rp"],"enrUri":"enr:-L24QGBC1gWhw3McfD3a_CV7ZQhE48GgRRvE-a9fIbIXgVDKAZYl2-ONlecklK-JZl9ZMxSK7L3so4pwKaLUP33MQLsCgmlkgnY0gmlwhKwSh6qKbXVsdGlhZGRyc5YACASsEoeqBqSZAAoErBKHqgakmt0DgnJzhQADAQAAiXNlY3AyNTZrMaECnq85iPgX-LoAMdGiYe9UGpxdLQIzAU15op6Y14MZa7eDdGNwgqSZg3VkcIKkm4V3YWt1MgE"}'
2026-03-05 04:33:17.461 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:33:17.462 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:42136/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.119.59/tcp/33023/p2p/16Uiu2HAkyYEc8JixCsDfFzV1owQ6KtuotgDbnA7acgE3TRR5MD5f"]'
2026-03-05 04:33:17.467 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:17.468 DEBUG [src.steps.relay] Running fixture setup: setup_optional_relay_nodes
2026-03-05 04:33:17.474 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:33:17.474 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-05_04-33-14__a9d87193-71d9-45a9-b2dd-91b3866fcf1e__wakuorg_nwaku:latest.log
2026-03-05 04:33:17.474 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:33:17.475 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:33:17.476 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:33:17.476 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.111.142
2026-03-05 04:33:17.476 DEBUG [src.node.docker_mananger] Generated ports ['23850', '23851', '23852', '23853', '23854']
2026-03-05 04:33:17.476 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:33:17.476 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:33:17.477 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:33:17.477 DEBUG [src.node.docker_mananger] docker run -i -t -p 23850:23850 -p 23851:23851 -p 23852:23852 -p 23853:23853 -p 23854:23854 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=23852 --rest-port=23850 --tcp-port=23851 --discv5-udp-port=23853 --rest-address=0.0.0.0 --nat=extip:172.18.111.142 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=c1156f04f8f55a42a0abe3dd6290b48a375efaa3d8ab77a0ce99d410fefcbbda --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=23854 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QC5WvvyK5m-lQH1rzxM_0VzrIaSaWExvOLQeweZMMvfHTdexgbaiT1a53CrVTUZ1GgnX4AI0oV1BmYixxEE_dmoCgmlkgnY0gmlwhKwSdzuKbXVsdGlhZGRyc5YACASsEnc7BoD_AAoErBJ3OwaBAN0DgnJzhQADAQAAiXNlY3AyNTZrMaECPR0fllwnk1IMXM8PJZ0wMfzSbMY5y-TTmI5QrgTaRB6DdGNwgoD_g3VkcIKBAYV3YWt1MgE
2026-03-05 04:33:17.666 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.111.142 waku 23bcec481fc737b9c53c4cbf957975dccc2e940ec370442f40d8ee4f9f6a4046
2026-03-05 04:33:17.700 DEBUG [src.node.docker_mananger] Container started with ID 23bcec481fc7. Setting up logs at ./log/docker/node3_2026-03-05_04-33-14__a9d87193-71d9-45a9-b2dd-91b3866fcf1e__wakuorg_nwaku:latest.log
2026-03-05 04:33:17.700 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 23850
2026-03-05 04:33:17.700 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:33:18.701 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23850/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:18.720 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:33:18.721 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:33:18.725 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23850/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:18.729 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.111.142/tcp/23851/p2p/16Uiu2HAmBCs6X79D8uqHXevvCJZ7zxVmJgzTp1qdQ8bKtw5XH5TS","/ip4/172.18.111.142/tcp/23852/ws/p2p/16Uiu2HAmBCs6X79D8uqHXevvCJZ7zxVmJgzTp1qdQ8bKtw5XH5TS"],"enrUri":"enr:-L24QHWQzqm8FqwAuMroB1qe_z60oM4FUL_lWFSNpGGoirSfT9VANy5I__I6hUb7YUv5_yk5edrn0ZB4Axt4t76SdBYCgmlkgnY0gmlwhKwSb46KbXVsdGlhZGRyc5YACASsEm-OBl0rAAoErBJvjgZdLN0DgnJzhQADAQAAiXNlY3AyNTZrMaEC6nMCWSrZkONnqDWZOsLm2I4dqEzDc31L--TmcrnmXm2DdGNwgl0rg3VkcIJdLYV3YWt1MgE"}'
2026-03-05 04:33:18.729 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:33:18.729 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23850/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.119.59/tcp/33023/p2p/16Uiu2HAkyYEc8JixCsDfFzV1owQ6KtuotgDbnA7acgE3TRR5MD5f"]'
2026-03-05 04:33:18.734 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:18.740 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:33:18.741 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-05_04-33-14__a9d87193-71d9-45a9-b2dd-91b3866fcf1e__wakuorg_nwaku:latest.log
2026-03-05 04:33:18.741 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:33:18.741 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:33:18.742 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:33:18.742 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.123.1
2026-03-05 04:33:18.743 DEBUG [src.node.docker_mananger] Generated ports ['38732', '38733', '38734', '38735', '38736']
2026-03-05 04:33:18.743 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:33:18.743 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:33:18.743 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:33:18.743 DEBUG [src.node.docker_mananger] docker run -i -t -p 38732:38732 -p 38733:38733 -p 38734:38734 -p 38735:38735 -p 38736:38736 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=38734 --rest-port=38732 --tcp-port=38733 --discv5-udp-port=38735 --rest-address=0.0.0.0 --nat=extip:172.18.123.1 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=1eab6d6e0b81385bc1d9d5acd8acff831fc87bfbd7ff3894cd748bbedfc4aa8e --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=38736 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QC5WvvyK5m-lQH1rzxM_0VzrIaSaWExvOLQeweZMMvfHTdexgbaiT1a53CrVTUZ1GgnX4AI0oV1BmYixxEE_dmoCgmlkgnY0gmlwhKwSdzuKbXVsdGlhZGRyc5YACASsEnc7BoD_AAoErBJ3OwaBAN0DgnJzhQADAQAAiXNlY3AyNTZrMaECPR0fllwnk1IMXM8PJZ0wMfzSbMY5y-TTmI5QrgTaRB6DdGNwgoD_g3VkcIKBAYV3YWt1MgE
2026-03-05 04:33:18.948 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.123.1 waku fc07cc333537ceb0e84478c5f0a8fd28632a91b8d1eef73f20a1a71e1a3d22dd
2026-03-05 04:33:18.983 DEBUG [src.node.docker_mananger] Container started with ID fc07cc333537. Setting up logs at ./log/docker/node4_2026-03-05_04-33-14__a9d87193-71d9-45a9-b2dd-91b3866fcf1e__wakuorg_nwaku:latest.log
2026-03-05 04:33:18.983 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 38732
2026-03-05 04:33:18.983 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:33:19.984 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38732/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:19.998 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:33:19.998 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:33:19.999 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38732/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:20.005 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.123.1/tcp/38733/p2p/16Uiu2HAmGPCiKnjtevn1upxCFdi1ApqZweMEUZckvD8kq8R1hyrB","/ip4/172.18.123.1/tcp/38734/ws/p2p/16Uiu2HAmGPCiKnjtevn1upxCFdi1ApqZweMEUZckvD8kq8R1hyrB"],"enrUri":"enr:-L24QD9_PfPzqDZ5TbfKr-_S79KCjwHvRSDB7DFDQWzHAEwZRaLpyzVuLdgY_63U2vUM1j0hImyEiSN8fvBCs_i9loMCgmlkgnY0gmlwhKwSewGKbXVsdGlhZGRyc5YACASsEnsBBpdNAAoErBJ7AQaXTt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDN2Nu6y_jwnp3NSG_vxUwm07ivjadQjgaYfon3CredQSDdGNwgpdNg3VkcIKXT4V3YWt1MgE"}'
2026-03-05 04:33:20.007 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:33:20.008 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38732/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.119.59/tcp/33023/p2p/16Uiu2HAkyYEc8JixCsDfFzV1owQ6KtuotgDbnA7acgE3TRR5MD5f"]'
2026-03-05 04:33:20.012 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:20.018 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-05 04:33:20.019 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-03-05_04-33-14__a9d87193-71d9-45a9-b2dd-91b3866fcf1e__wakuorg_nwaku:latest.log
2026-03-05 04:33:20.019 DEBUG [src.node.waku_node] Starting Node...
2026-03-05 04:33:20.019 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-05 04:33:20.020 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-05 04:33:20.020 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.186.237
2026-03-05 04:33:20.020 DEBUG [src.node.docker_mananger] Generated ports ['38539', '38540', '38541', '38542', '38543']
2026-03-05 04:33:20.021 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-05 04:33:20.021 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-05 04:33:20.021 DEBUG [src.node.waku_node] Using volumes []
2026-03-05 04:33:20.021 DEBUG [src.node.docker_mananger] docker run -i -t -p 38539:38539 -p 38540:38540 -p 38541:38541 -p 38542:38542 -p 38543:38543 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=38541 --rest-port=38539 --tcp-port=38540 --discv5-udp-port=38542 --rest-address=0.0.0.0 --nat=extip:172.18.186.237 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=83aded0dd1957ce306df2e6e4bf7bdecd0bfe30dc5fe6dbc1655f37b0c8bd537 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=38543 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QC5WvvyK5m-lQH1rzxM_0VzrIaSaWExvOLQeweZMMvfHTdexgbaiT1a53CrVTUZ1GgnX4AI0oV1BmYixxEE_dmoCgmlkgnY0gmlwhKwSdzuKbXVsdGlhZGRyc5YACASsEnc7BoD_AAoErBJ3OwaBAN0DgnJzhQADAQAAiXNlY3AyNTZrMaECPR0fllwnk1IMXM8PJZ0wMfzSbMY5y-TTmI5QrgTaRB6DdGNwgoD_g3VkcIKBAYV3YWt1MgE
2026-03-05 04:33:20.216 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.186.237 waku d9c3f28841653a115e2e450a6948b1b498ece9dce9fbd5a951e6916a992f42ba
2026-03-05 04:33:20.254 DEBUG [src.node.docker_mananger] Container started with ID d9c3f2884165. Setting up logs at ./log/docker/node5_2026-03-05_04-33-14__a9d87193-71d9-45a9-b2dd-91b3866fcf1e__wakuorg_nwaku:latest.log
2026-03-05 04:33:20.255 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 38539
2026-03-05 04:33:20.255 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-05 04:33:21.257 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38539/health" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:21.269 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-05 04:33:21.269 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-05 04:33:21.269 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38539/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:21.277 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.186.237/tcp/38540/p2p/16Uiu2HAkxP6n6nb2FEJkzPfd32nfHSJEvZZwxWTNrYWQmzuhamWd","/ip4/172.18.186.237/tcp/38541/ws/p2p/16Uiu2HAkxP6n6nb2FEJkzPfd32nfHSJEvZZwxWTNrYWQmzuhamWd"],"enrUri":"enr:-L24QAH145Wk82CpqPGg19sAd3xRW-x166RnJBSbxLadbk2VLY1xdIVGq1gg7ogMl-V9UVNCdFONJP2Mj6zeXd5A058CgmlkgnY0gmlwhKwSuu2KbXVsdGlhZGRyc5YACASsErrtBpaMAAoErBK67QaWjd0DgnJzhQADAQAAiXNlY3AyNTZrMaECK-pbpTBEkwcAL5xI1Oa8VJM3qlsOHv7Y2d0LoDCbZk6DdGNwgpaMg3VkcIKWjoV3YWt1MgE"}'
2026-03-05 04:33:21.277 INFO [src.node.waku_node] REST service is ready !!
2026-03-05 04:33:21.278 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38539/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.119.59/tcp/33023/p2p/16Uiu2HAkyYEc8JixCsDfFzV1owQ6KtuotgDbnA7acgE3TRR5MD5f"]'
2026-03-05 04:33:21.281 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:21.281 DEBUG [src.steps.relay] Running fixture setup: subscribe_main_relay_nodes
2026-03-05 04:33:21.282 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:33022/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-05 04:33:21.289 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:21.291 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:42136/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-05 04:33:21.298 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:21.299 DEBUG [src.steps.relay] Running fixture setup: subscribe_optional_relay_nodes
2026-03-05 04:33:21.299 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:23850/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-05 04:33:21.306 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:21.306 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38732/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-05 04:33:21.313 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:21.313 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38539/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-05 04:33:21.320 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:21.322 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:33022/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 04:33:21.332 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:21.333 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-05 04:33:21.434 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-05 04:33:21.435 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33022/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:21.437 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772685201321886464,"ephemeral":false,"proof":""}]'
2026-03-05 04:33:21.438 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-05 04:33:21.439 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42136/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:21.441 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772685201321886464,"ephemeral":false,"proof":""}]'
2026-03-05 04:33:21.442 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-03-05 04:33:21.442 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23850/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:21.444 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772685201321886464,"ephemeral":false,"proof":""}]'
2026-03-05 04:33:21.446 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-03-05 04:33:21.446 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38732/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:21.448 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772685201321886464,"ephemeral":false,"proof":""}]'
2026-03-05 04:33:21.450 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-03-05 04:33:21.450 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38539/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:21.452 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772685201321886464,"ephemeral":false,"proof":""}]'
2026-03-05 04:33:21.454 INFO [src.steps.relay] WARM UP successful!!
2026-03-05 04:33:21.455 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:33022/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-05 04:33:21.465 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-05 04:33:21.465 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-05 04:33:21.566 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-05 04:33:21.566 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:33022/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:21.569 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772685201455427671,"ephemeral":false,"proof":""}]'
2026-03-05 04:33:21.570 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-05 04:33:21.571 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:42136/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:21.573 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772685201455427671,"ephemeral":false,"proof":""}]'
2026-03-05 04:33:21.574 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-03-05 04:33:21.574 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:23850/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:21.576 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772685201455427671,"ephemeral":false,"proof":""}]'
2026-03-05 04:33:21.579 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-03-05 04:33:21.579 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38732/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:21.581 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772685201455427671,"ephemeral":false,"proof":""}]'
2026-03-05 04:33:21.582 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-03-05 04:33:21.582 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38539/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-05 04:33:21.584 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1772685201455427671,"ephemeral":false,"proof":""}]'
2026-03-05 04:33:21.588 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-05 04:33:21.589 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-05 04:33:21.589 DEBUG [src.node.waku_node] Stopping container with id 11fb96cd5e4f
2026-03-05 04:33:22.186 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:33:22.186 DEBUG [src.node.waku_node] Stopping container with id c8342bc65184
2026-03-05 04:33:22.734 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:33:22.734 DEBUG [src.node.waku_node] Stopping container with id 23bcec481fc7
2026-03-05 04:33:23.060 ERROR [src.node.docker_mananger] Max retries reached for container 11fb96cd5e4f. Exiting log stream.
2026-03-05 04:33:23.296 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:33:23.296 DEBUG [src.node.waku_node] Stopping container with id fc07cc333537
2026-03-05 04:33:23.546 ERROR [src.node.docker_mananger] Max retries reached for container c8342bc65184. Exiting log stream.
2026-03-05 04:33:23.910 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:33:23.911 DEBUG [src.node.waku_node] Stopping container with id d9c3f2884165
2026-03-05 04:33:24.116 ERROR [src.node.docker_mananger] Max retries reached for container 23bcec481fc7. Exiting log stream.
2026-03-05 04:33:24.460 DEBUG [src.node.waku_node] Container stopped.
2026-03-05 04:33:24.463 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-05 04:33:24.495 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:33:24.501 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:33:24.506 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:33:24.512 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-05 04:33:24.517 DEBUG [src.node.docker_mananger] No errors found in the waku logs.