192 lines
35 KiB
Plaintext

2026-03-21 04:31:53.038 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-21 04:31:53.038 DEBUG [tests.conftest] Running test: test_first_node_to_start_publishes with id: 2026-03-21_04-31-53__d9b4c9aa-5651-4d7c-bf7a-694efd4b5f45
2026-03-21 04:31:53.038 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-21 04:31:53.039 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-21 04:31:53.039 DEBUG [src.steps.relay] Running fixture setup: setup_main_relay_nodes
2026-03-21 04:31:53.047 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-21 04:31:53.047 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-21_04-31-53__d9b4c9aa-5651-4d7c-bf7a-694efd4b5f45__wakuorg_nwaku:latest.log
2026-03-21 04:31:53.047 DEBUG [src.node.waku_node] Starting Node...
2026-03-21 04:31:53.047 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-21 04:31:53.048 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-21 04:31:53.049 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.161.177
2026-03-21 04:31:53.049 DEBUG [src.node.docker_mananger] Generated ports ['4918', '4919', '4920', '4921', '4922']
2026-03-21 04:31:53.049 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-21 04:31:53.049 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-21 04:31:53.049 DEBUG [src.node.waku_node] Using volumes []
2026-03-21 04:31:53.049 DEBUG [src.node.docker_mananger] docker run -i -t -p 4918:4918 -p 4919:4919 -p 4920:4920 -p 4921:4921 -p 4922:4922 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=4920 --rest-port=4918 --tcp-port=4919 --discv5-udp-port=4921 --rest-address=0.0.0.0 --nat=extip:172.18.161.177 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=dcb3d17cec219ea856092197be1a8d66db9e06f05c44bc812cc4a53dd6354aea --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=4922 --metrics-logging=true --relay=true
2026-03-21 04:31:53.246 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.161.177 waku dbb38ed026c4a648c04735b99a49f4d15bff8b43d9b59dfcae781c98a743a11a
2026-03-21 04:31:53.283 DEBUG [src.node.docker_mananger] Container started with ID dbb38ed026c4. Setting up logs at ./log/docker/node1_2026-03-21_04-31-53__d9b4c9aa-5651-4d7c-bf7a-694efd4b5f45__wakuorg_nwaku:latest.log
2026-03-21 04:31:53.283 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 4918
2026-03-21 04:31:53.284 ERROR [src.node.docker_mananger] Max retries reached for container 6ac706a35e7d. Exiting log stream.
2026-03-21 04:31:53.284 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-21 04:31:53.823 ERROR [src.node.docker_mananger] Max retries reached for container 43644d550388. Exiting log stream.
2026-03-21 04:31:54.285 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4918/health" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:54.288 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-21 04:31:54.288 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-21 04:31:54.289 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4918/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:54.291 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.161.177/tcp/4919/p2p/16Uiu2HAmJQGnXthsAfQ4e4i865aMKjCFFchg1ij12mcARXhgyDnB","/ip4/172.18.161.177/tcp/4920/ws/p2p/16Uiu2HAmJQGnXthsAfQ4e4i865aMKjCFFchg1ij12mcARXhgyDnB"],"enrUri":"enr:-L24QOrpLrtIHr0efl7u8aW_gGDIBJInOTSo5CaXbAJ5i99GPuU_foGp8A6-OHDO-YrL4O_xHoNZWIQjtppZdTgCnxACgmlkgnY0gmlwhKwSobGKbXVsdGlhZGRyc5YACASsEqGxBhM3AAoErBKhsQYTON0DgnJzhQADAQAAiXNlY3AyNTZrMaEDVWD-KCK6le17kjj7mxqC6Dv_PVUlpoTfCp0ik5qNxZyDdGNwghM3g3VkcIITOYV3YWt1MgE"}'
2026-03-21 04:31:54.291 INFO [src.node.waku_node] REST service is ready !!
2026-03-21 04:31:54.298 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-21 04:31:54.298 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-21_04-31-53__d9b4c9aa-5651-4d7c-bf7a-694efd4b5f45__wakuorg_nwaku:latest.log
2026-03-21 04:31:54.298 DEBUG [src.node.waku_node] Starting Node...
2026-03-21 04:31:54.298 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-21 04:31:54.299 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-21 04:31:54.299 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.0.66
2026-03-21 04:31:54.300 DEBUG [src.node.docker_mananger] Generated ports ['9626', '9627', '9628', '9629', '9630']
2026-03-21 04:31:54.300 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-21 04:31:54.300 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-21 04:31:54.300 DEBUG [src.node.waku_node] Using volumes []
2026-03-21 04:31:54.300 DEBUG [src.node.docker_mananger] docker run -i -t -p 9626:9626 -p 9627:9627 -p 9628:9628 -p 9629:9629 -p 9630:9630 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=9628 --rest-port=9626 --tcp-port=9627 --discv5-udp-port=9629 --rest-address=0.0.0.0 --nat=extip:172.18.0.66 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=87b08b19b8d8964ecd70d8cd8b69a8eaeaed1916f9c12ceb8149d7ddf1be9c55 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=9630 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QOrpLrtIHr0efl7u8aW_gGDIBJInOTSo5CaXbAJ5i99GPuU_foGp8A6-OHDO-YrL4O_xHoNZWIQjtppZdTgCnxACgmlkgnY0gmlwhKwSobGKbXVsdGlhZGRyc5YACASsEqGxBhM3AAoErBKhsQYTON0DgnJzhQADAQAAiXNlY3AyNTZrMaEDVWD-KCK6le17kjj7mxqC6Dv_PVUlpoTfCp0ik5qNxZyDdGNwghM3g3VkcIITOYV3YWt1MgE
2026-03-21 04:31:54.498 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.0.66 waku a886e1a6e213ffed312e70ad25dca4e98b31a66ea667796dc33c2f0c2cc3976d
2026-03-21 04:31:54.531 DEBUG [src.node.docker_mananger] Container started with ID a886e1a6e213. Setting up logs at ./log/docker/node2_2026-03-21_04-31-53__d9b4c9aa-5651-4d7c-bf7a-694efd4b5f45__wakuorg_nwaku:latest.log
2026-03-21 04:31:54.532 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 9626
2026-03-21 04:31:54.532 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-21 04:31:55.533 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9626/health" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:55.548 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-21 04:31:55.549 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-21 04:31:55.549 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9626/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:55.557 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.0.66/tcp/9627/p2p/16Uiu2HAmNYPy1K9ynum9oUiWF2PCCEyr7f5WSExTzs9DNjDF319q","/ip4/172.18.0.66/tcp/9628/ws/p2p/16Uiu2HAmNYPy1K9ynum9oUiWF2PCCEyr7f5WSExTzs9DNjDF319q"],"enrUri":"enr:-L24QH5GP7V2F0xeNpDBaW15LF_9cL9sy_PVU6By0JbKLnpMMQFbTU62rxqVBsbN-d252YCQrWvg8lbjcxS36jQKkgYCgmlkgnY0gmlwhKwSAEKKbXVsdGlhZGRyc5YACASsEgBCBiWbAAoErBIAQgYlnN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDkuSBjbxJ2g39u1zcLqmXHwijz_4b10Go-gE9f8KVP7CDdGNwgiWbg3VkcIIlnYV3YWt1MgE"}'
2026-03-21 04:31:55.559 INFO [src.node.waku_node] REST service is ready !!
2026-03-21 04:31:55.562 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:9626/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.161.177/tcp/4919/p2p/16Uiu2HAmJQGnXthsAfQ4e4i865aMKjCFFchg1ij12mcARXhgyDnB"]'
2026-03-21 04:31:55.564 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-21 04:31:55.565 DEBUG [src.steps.relay] Running fixture setup: setup_optional_relay_nodes
2026-03-21 04:31:55.571 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-21 04:31:55.572 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-21_04-31-53__d9b4c9aa-5651-4d7c-bf7a-694efd4b5f45__wakuorg_nwaku:latest.log
2026-03-21 04:31:55.572 DEBUG [src.node.waku_node] Starting Node...
2026-03-21 04:31:55.572 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-21 04:31:55.573 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-21 04:31:55.573 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.167.176
2026-03-21 04:31:55.573 DEBUG [src.node.docker_mananger] Generated ports ['52866', '52867', '52868', '52869', '52870']
2026-03-21 04:31:55.574 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-21 04:31:55.574 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-21 04:31:55.574 DEBUG [src.node.waku_node] Using volumes []
2026-03-21 04:31:55.574 DEBUG [src.node.docker_mananger] docker run -i -t -p 52866:52866 -p 52867:52867 -p 52868:52868 -p 52869:52869 -p 52870:52870 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=52868 --rest-port=52866 --tcp-port=52867 --discv5-udp-port=52869 --rest-address=0.0.0.0 --nat=extip:172.18.167.176 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=bbc8a61d9f77f82fd33244f98bf69ff0c1beddaf4ecae4cb02c46d18d05014af --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=52870 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QOrpLrtIHr0efl7u8aW_gGDIBJInOTSo5CaXbAJ5i99GPuU_foGp8A6-OHDO-YrL4O_xHoNZWIQjtppZdTgCnxACgmlkgnY0gmlwhKwSobGKbXVsdGlhZGRyc5YACASsEqGxBhM3AAoErBKhsQYTON0DgnJzhQADAQAAiXNlY3AyNTZrMaEDVWD-KCK6le17kjj7mxqC6Dv_PVUlpoTfCp0ik5qNxZyDdGNwghM3g3VkcIITOYV3YWt1MgE
2026-03-21 04:31:55.773 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.167.176 waku 87b18408674eb50eddefaee0c836e12d5aa1bd415f7edee9d82a7dd5e6bdd345
2026-03-21 04:31:55.807 DEBUG [src.node.docker_mananger] Container started with ID 87b18408674e. Setting up logs at ./log/docker/node3_2026-03-21_04-31-53__d9b4c9aa-5651-4d7c-bf7a-694efd4b5f45__wakuorg_nwaku:latest.log
2026-03-21 04:31:55.807 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 52866
2026-03-21 04:31:55.808 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-21 04:31:56.809 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52866/health" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:56.822 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-21 04:31:56.822 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-21 04:31:56.823 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52866/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:56.829 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.167.176/tcp/52867/p2p/16Uiu2HAmLUvKZt8Q9ryizGqUyeUcCa78JtF2GnXHAV9GdNMCQA6s","/ip4/172.18.167.176/tcp/52868/ws/p2p/16Uiu2HAmLUvKZt8Q9ryizGqUyeUcCa78JtF2GnXHAV9GdNMCQA6s"],"enrUri":"enr:-L24QGz1pw7E5t2LhaAKZOPrdG08quHiZ-f7Rl2I7w-ygU6cTrvynCtIQwPZMwYyK0jiRz8rK4LqCTbS5l20UGGB4BICgmlkgnY0gmlwhKwSp7CKbXVsdGlhZGRyc5YACASsEqewBs6DAAoErBKnsAbOhN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDdEkhhM99rn-N1Xhu2MW5dh02v5okPGFybvU-A8iIxaCDdGNwgs6Dg3VkcILOhYV3YWt1MgE"}'
2026-03-21 04:31:56.829 INFO [src.node.waku_node] REST service is ready !!
2026-03-21 04:31:56.831 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:52866/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.161.177/tcp/4919/p2p/16Uiu2HAmJQGnXthsAfQ4e4i865aMKjCFFchg1ij12mcARXhgyDnB"]'
2026-03-21 04:31:56.835 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-21 04:31:56.841 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-21 04:31:56.842 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-21_04-31-53__d9b4c9aa-5651-4d7c-bf7a-694efd4b5f45__wakuorg_nwaku:latest.log
2026-03-21 04:31:56.842 DEBUG [src.node.waku_node] Starting Node...
2026-03-21 04:31:56.842 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-21 04:31:56.843 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-21 04:31:56.843 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.57.51
2026-03-21 04:31:56.843 DEBUG [src.node.docker_mananger] Generated ports ['64754', '64755', '64756', '64757', '64758']
2026-03-21 04:31:56.844 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-21 04:31:56.844 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-21 04:31:56.844 DEBUG [src.node.waku_node] Using volumes []
2026-03-21 04:31:56.844 DEBUG [src.node.docker_mananger] docker run -i -t -p 64754:64754 -p 64755:64755 -p 64756:64756 -p 64757:64757 -p 64758:64758 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=64756 --rest-port=64754 --tcp-port=64755 --discv5-udp-port=64757 --rest-address=0.0.0.0 --nat=extip:172.18.57.51 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=7ecb648dcae6bb6dc74d5e4ac7f0b7cc8ece3d2bff7fba8fa702e995ee5aeabb --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=64758 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QOrpLrtIHr0efl7u8aW_gGDIBJInOTSo5CaXbAJ5i99GPuU_foGp8A6-OHDO-YrL4O_xHoNZWIQjtppZdTgCnxACgmlkgnY0gmlwhKwSobGKbXVsdGlhZGRyc5YACASsEqGxBhM3AAoErBKhsQYTON0DgnJzhQADAQAAiXNlY3AyNTZrMaEDVWD-KCK6le17kjj7mxqC6Dv_PVUlpoTfCp0ik5qNxZyDdGNwghM3g3VkcIITOYV3YWt1MgE
2026-03-21 04:31:57.039 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.57.51 waku 525b08470cfee48ce4eb0e860a905b694f0a792962d9118b3f1caa212b8f0013
2026-03-21 04:31:57.076 DEBUG [src.node.docker_mananger] Container started with ID 525b08470cfe. Setting up logs at ./log/docker/node4_2026-03-21_04-31-53__d9b4c9aa-5651-4d7c-bf7a-694efd4b5f45__wakuorg_nwaku:latest.log
2026-03-21 04:31:57.077 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 64754
2026-03-21 04:31:57.077 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-21 04:31:58.078 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:64754/health" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:58.099 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-21 04:31:58.100 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-21 04:31:58.100 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:64754/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:58.104 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.57.51/tcp/64755/p2p/16Uiu2HAmVHDH1AdrKgcDDgztCZ6Lk6vDp8ejXM4n5PYzRJkbAvSY","/ip4/172.18.57.51/tcp/64756/ws/p2p/16Uiu2HAmVHDH1AdrKgcDDgztCZ6Lk6vDp8ejXM4n5PYzRJkbAvSY"],"enrUri":"enr:-L24QM92kNBfepy5kfU2IT3yirLjnbK8ZVoIEuPZlCmVSWRbQyXCfxKgxnhGLdSqotMYQG4VumB_77l3ne6PL915L4wCgmlkgnY0gmlwhKwSOTOKbXVsdGlhZGRyc5YACASsEjkzBvzzAAoErBI5Mwb89N0DgnJzhQADAQAAiXNlY3AyNTZrMaED9wKD-i_FhX2W4g-gdzY5udmgEg-k0dgIWhbq4SJ_FOWDdGNwgvzzg3VkcIL89YV3YWt1MgE"}'
2026-03-21 04:31:58.104 INFO [src.node.waku_node] REST service is ready !!
2026-03-21 04:31:58.105 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:64754/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.161.177/tcp/4919/p2p/16Uiu2HAmJQGnXthsAfQ4e4i865aMKjCFFchg1ij12mcARXhgyDnB"]'
2026-03-21 04:31:58.110 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-21 04:31:58.117 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-21 04:31:58.118 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-03-21_04-31-53__d9b4c9aa-5651-4d7c-bf7a-694efd4b5f45__wakuorg_nwaku:latest.log
2026-03-21 04:31:58.118 DEBUG [src.node.waku_node] Starting Node...
2026-03-21 04:31:58.118 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-21 04:31:58.119 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-21 04:31:58.119 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.247.187
2026-03-21 04:31:58.120 DEBUG [src.node.docker_mananger] Generated ports ['41756', '41757', '41758', '41759', '41760']
2026-03-21 04:31:58.120 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-21 04:31:58.120 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-21 04:31:58.120 DEBUG [src.node.waku_node] Using volumes []
2026-03-21 04:31:58.120 DEBUG [src.node.docker_mananger] docker run -i -t -p 41756:41756 -p 41757:41757 -p 41758:41758 -p 41759:41759 -p 41760:41760 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=41758 --rest-port=41756 --tcp-port=41757 --discv5-udp-port=41759 --rest-address=0.0.0.0 --nat=extip:172.18.247.187 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=d0cccfbf5ec9828998a79f78e51d655a5b13bff64a1e1ded9e0a2fe764dc5bae --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=41760 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QOrpLrtIHr0efl7u8aW_gGDIBJInOTSo5CaXbAJ5i99GPuU_foGp8A6-OHDO-YrL4O_xHoNZWIQjtppZdTgCnxACgmlkgnY0gmlwhKwSobGKbXVsdGlhZGRyc5YACASsEqGxBhM3AAoErBKhsQYTON0DgnJzhQADAQAAiXNlY3AyNTZrMaEDVWD-KCK6le17kjj7mxqC6Dv_PVUlpoTfCp0ik5qNxZyDdGNwghM3g3VkcIITOYV3YWt1MgE
2026-03-21 04:31:58.326 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.247.187 waku be710ed6a7d671772a608a84fbb6b6da0715ee89569a79ae76a73009fc9f6154
2026-03-21 04:31:58.359 DEBUG [src.node.docker_mananger] Container started with ID be710ed6a7d6. Setting up logs at ./log/docker/node5_2026-03-21_04-31-53__d9b4c9aa-5651-4d7c-bf7a-694efd4b5f45__wakuorg_nwaku:latest.log
2026-03-21 04:31:58.359 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 41756
2026-03-21 04:31:58.360 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-21 04:31:59.361 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:41756/health" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:59.382 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-21 04:31:59.385 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-21 04:31:59.385 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:41756/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:59.392 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.247.187/tcp/41757/p2p/16Uiu2HAmFDro8eRtbxrBVpV9aQEXqHEfMuDZ9Jjsk2scK5tMkCDM","/ip4/172.18.247.187/tcp/41758/ws/p2p/16Uiu2HAmFDro8eRtbxrBVpV9aQEXqHEfMuDZ9Jjsk2scK5tMkCDM"],"enrUri":"enr:-L24QA7mrVc6Cl4WzI5BUVqPb4mFrLmyoyhNzjkWoAxaUJ0AG4nx8dX7qhr6YlJZHqNT8JOIy1DwRoThsvVYcU4twMkCgmlkgnY0gmlwhKwS97uKbXVsdGlhZGRyc5YACASsEve7BqMdAAoErBL3uwajHt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDJiL_rHGLbLkZSoE5NlNdrzmqwiq_XBYtzRdHYE2RNTCDdGNwgqMdg3VkcIKjH4V3YWt1MgE"}'
2026-03-21 04:31:59.393 INFO [src.node.waku_node] REST service is ready !!
2026-03-21 04:31:59.394 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:41756/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.161.177/tcp/4919/p2p/16Uiu2HAmJQGnXthsAfQ4e4i865aMKjCFFchg1ij12mcARXhgyDnB"]'
2026-03-21 04:31:59.397 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-21 04:31:59.398 DEBUG [src.steps.relay] Running fixture setup: subscribe_main_relay_nodes
2026-03-21 04:31:59.398 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:4918/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-21 04:31:59.406 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-21 04:31:59.406 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:9626/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-21 04:31:59.414 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-21 04:31:59.415 DEBUG [src.steps.relay] Running fixture setup: subscribe_optional_relay_nodes
2026-03-21 04:31:59.416 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:52866/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-21 04:31:59.422 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-21 04:31:59.423 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:64754/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-21 04:31:59.430 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-21 04:31:59.430 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:41756/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-21 04:31:59.435 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-21 04:31:59.437 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:4918/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-21 04:31:59.451 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-21 04:31:59.454 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-21 04:31:59.554 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-21 04:31:59.554 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4918/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:59.557 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1774067519437467904,"ephemeral":false,"proof":""}]'
2026-03-21 04:31:59.559 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-21 04:31:59.559 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9626/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:59.562 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1774067519437467904,"ephemeral":false,"proof":""}]'
2026-03-21 04:31:59.563 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-03-21 04:31:59.564 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52866/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:59.566 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1774067519437467904,"ephemeral":false,"proof":""}]'
2026-03-21 04:31:59.568 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-03-21 04:31:59.568 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:64754/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:59.570 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1774067519437467904,"ephemeral":false,"proof":""}]'
2026-03-21 04:31:59.572 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-03-21 04:31:59.572 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:41756/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:59.574 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1774067519437467904,"ephemeral":false,"proof":""}]'
2026-03-21 04:31:59.576 INFO [src.steps.relay] WARM UP successful!!
2026-03-21 04:31:59.577 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:4918/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-03-21 04:31:59.589 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-21 04:31:59.590 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-21 04:31:59.690 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-03-21 04:31:59.690 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:4918/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:59.693 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1774067519577159805,"ephemeral":false,"proof":""}]'
2026-03-21 04:31:59.695 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-03-21 04:31:59.695 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:9626/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:59.697 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1774067519577159805,"ephemeral":false,"proof":""}]'
2026-03-21 04:31:59.699 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-03-21 04:31:59.699 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:52866/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:59.701 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1774067519577159805,"ephemeral":false,"proof":""}]'
2026-03-21 04:31:59.702 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-03-21 04:31:59.703 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:64754/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:59.705 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1774067519577159805,"ephemeral":false,"proof":""}]'
2026-03-21 04:31:59.707 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-03-21 04:31:59.707 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:41756/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-03-21 04:31:59.710 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1774067519577159805,"ephemeral":false,"proof":""}]'
2026-03-21 04:31:59.713 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-21 04:31:59.714 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-21 04:31:59.714 DEBUG [src.node.waku_node] Stopping container with id dbb38ed026c4
2026-03-21 04:32:00.308 DEBUG [src.node.waku_node] Container stopped.
2026-03-21 04:32:00.309 DEBUG [src.node.waku_node] Stopping container with id a886e1a6e213
2026-03-21 04:32:00.863 DEBUG [src.node.waku_node] Container stopped.
2026-03-21 04:32:00.863 DEBUG [src.node.waku_node] Stopping container with id 87b18408674e
2026-03-21 04:32:01.122 ERROR [src.node.docker_mananger] Max retries reached for container dbb38ed026c4. Exiting log stream.
2026-03-21 04:32:01.431 DEBUG [src.node.waku_node] Container stopped.
2026-03-21 04:32:01.431 DEBUG [src.node.waku_node] Stopping container with id 525b08470cfe
2026-03-21 04:32:01.676 ERROR [src.node.docker_mananger] Max retries reached for container a886e1a6e213. Exiting log stream.
2026-03-21 04:32:02.011 DEBUG [src.node.waku_node] Container stopped.
2026-03-21 04:32:02.012 DEBUG [src.node.waku_node] Stopping container with id be710ed6a7d6
2026-03-21 04:32:02.245 ERROR [src.node.docker_mananger] Max retries reached for container 87b18408674e. Exiting log stream.
2026-03-21 04:32:02.581 DEBUG [src.node.waku_node] Container stopped.
2026-03-21 04:32:02.582 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-21 04:32:02.600 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-21 04:32:02.606 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-21 04:32:02.613 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-21 04:32:02.620 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-21 04:32:02.630 DEBUG [src.node.docker_mananger] No errors found in the waku logs.