190 lines
35 KiB
Plaintext

2026-01-29 04:44:19.358 DEBUG [tests.conftest] Running fixture setup: test_id
2026-01-29 04:44:19.359 DEBUG [tests.conftest] Running test: test_first_node_to_start_publishes with id: 2026-01-29_04-44-19__91526413-5e89-46b5-9e50-5714f74883da
2026-01-29 04:44:19.359 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-01-29 04:44:19.359 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-01-29 04:44:19.360 DEBUG [src.steps.relay] Running fixture setup: setup_main_relay_nodes
2026-01-29 04:44:19.366 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-29 04:44:19.366 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-01-29_04-44-19__91526413-5e89-46b5-9e50-5714f74883da__wakuorg_nwaku:latest.log
2026-01-29 04:44:19.367 DEBUG [src.node.waku_node] Starting Node...
2026-01-29 04:44:19.367 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-29 04:44:19.368 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-29 04:44:19.368 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.27.227
2026-01-29 04:44:19.368 DEBUG [src.node.docker_mananger] Generated ports ['6662', '6663', '6664', '6665', '6666']
2026-01-29 04:44:19.368 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-29 04:44:19.369 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-29 04:44:19.369 DEBUG [src.node.waku_node] Using volumes []
2026-01-29 04:44:19.369 DEBUG [src.node.docker_mananger] docker run -i -t -p 6662:6662 -p 6663:6663 -p 6664:6664 -p 6665:6665 -p 6666:6666 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=6664 --rest-port=6662 --tcp-port=6663 --discv5-udp-port=6665 --rest-address=0.0.0.0 --nat=extip:172.18.27.227 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=06bd1f9527f66dbbf564e1f2fa6a44a6df04e2f5a0fe53de36e205866dac6edd --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=6666 --metrics-logging=true --relay=true
2026-01-29 04:44:19.567 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.27.227 waku d9ebd698eb832de8ddce0d441991667d4216a73987d4e207525f1d65ed733ff6
2026-01-29 04:44:19.603 DEBUG [src.node.docker_mananger] Container started with ID d9ebd698eb83. Setting up logs at ./log/docker/node1_2026-01-29_04-44-19__91526413-5e89-46b5-9e50-5714f74883da__wakuorg_nwaku:latest.log
2026-01-29 04:44:19.604 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 6662
2026-01-29 04:44:19.604 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-29 04:44:20.605 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6662/health" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:20.608 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-29 04:44:20.608 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-29 04:44:20.608 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6662/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:20.610 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.27.227/tcp/6663/p2p/16Uiu2HAmKYL6Myx4M7H9m5TJ4oDqzguJs81aCe4AMTP6DQEXKudW","/ip4/172.18.27.227/tcp/6664/ws/p2p/16Uiu2HAmKYL6Myx4M7H9m5TJ4oDqzguJs81aCe4AMTP6DQEXKudW"],"enrUri":"enr:-L24QG6Sj87ZAn9TaW1cbMucfBSMOfHwlSza3wWV0EdYdYfmVE_QXlX38qTZOlTNrkPLMB3TYmvX5q2swJBhdOH95TMCgmlkgnY0gmlwhKwSG-OKbXVsdGlhZGRyc5YACASsEhvjBhoHAAoErBIb4wYaCN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDZk0Q3fbqet2bn1e6uV_bYL6ShkvbqOsW8rmtlIJhUiWDdGNwghoHg3VkcIIaCYV3YWt1MgE"}'
2026-01-29 04:44:20.611 INFO [src.node.waku_node] REST service is ready !!
2026-01-29 04:44:20.617 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-29 04:44:20.617 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-01-29_04-44-19__91526413-5e89-46b5-9e50-5714f74883da__wakuorg_nwaku:latest.log
2026-01-29 04:44:20.617 DEBUG [src.node.waku_node] Starting Node...
2026-01-29 04:44:20.617 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-29 04:44:20.619 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-29 04:44:20.619 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.95.30
2026-01-29 04:44:20.619 DEBUG [src.node.docker_mananger] Generated ports ['7386', '7387', '7388', '7389', '7390']
2026-01-29 04:44:20.619 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-29 04:44:20.619 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-29 04:44:20.620 DEBUG [src.node.waku_node] Using volumes []
2026-01-29 04:44:20.620 DEBUG [src.node.docker_mananger] docker run -i -t -p 7386:7386 -p 7387:7387 -p 7388:7388 -p 7389:7389 -p 7390:7390 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=7388 --rest-port=7386 --tcp-port=7387 --discv5-udp-port=7389 --rest-address=0.0.0.0 --nat=extip:172.18.95.30 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=3f2df9f2dadeca3cad639ac871aabf392445ce8a56df1d4e6dcbaf3dcc17c0de --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=7390 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QG6Sj87ZAn9TaW1cbMucfBSMOfHwlSza3wWV0EdYdYfmVE_QXlX38qTZOlTNrkPLMB3TYmvX5q2swJBhdOH95TMCgmlkgnY0gmlwhKwSG-OKbXVsdGlhZGRyc5YACASsEhvjBhoHAAoErBIb4wYaCN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDZk0Q3fbqet2bn1e6uV_bYL6ShkvbqOsW8rmtlIJhUiWDdGNwghoHg3VkcIIaCYV3YWt1MgE
2026-01-29 04:44:20.810 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.95.30 waku 121861a64f01c14364839697a6b05cf97eb81355b3497a6d906bd05a6e24df5e
2026-01-29 04:44:20.839 DEBUG [src.node.docker_mananger] Container started with ID 121861a64f01. Setting up logs at ./log/docker/node2_2026-01-29_04-44-19__91526413-5e89-46b5-9e50-5714f74883da__wakuorg_nwaku:latest.log
2026-01-29 04:44:20.839 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 7386
2026-01-29 04:44:20.839 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-29 04:44:21.841 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7386/health" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:21.854 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-29 04:44:21.856 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-29 04:44:21.858 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7386/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:21.861 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.95.30/tcp/7387/p2p/16Uiu2HAm4EQqDpeU6vwj1L2SznJaAqSwZHpvWMw8bfedUvYc9ZD2","/ip4/172.18.95.30/tcp/7388/ws/p2p/16Uiu2HAm4EQqDpeU6vwj1L2SznJaAqSwZHpvWMw8bfedUvYc9ZD2"],"enrUri":"enr:-L24QFmrDUtnR6zt7Gf788Z_AZenU1WCUVwfkbNvfQU5JtLQG6OjYKxOf-p866edpTwO2gKX_zqOflXf2QFUIiAnODECgmlkgnY0gmlwhKwSXx6KbXVsdGlhZGRyc5YACASsEl8eBhzbAAoErBJfHgYc3N0DgnJzhQADAQAAiXNlY3AyNTZrMaECgtaqcJBS8X31WLVVLw-nFmNY5nhxNgokEBN-o8kcJMmDdGNwghzbg3VkcIIc3YV3YWt1MgE"}'
2026-01-29 04:44:21.863 INFO [src.node.waku_node] REST service is ready !!
2026-01-29 04:44:21.864 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7386/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.27.227/tcp/6663/p2p/16Uiu2HAmKYL6Myx4M7H9m5TJ4oDqzguJs81aCe4AMTP6DQEXKudW"]'
2026-01-29 04:44:21.867 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-29 04:44:21.868 DEBUG [src.steps.relay] Running fixture setup: setup_optional_relay_nodes
2026-01-29 04:44:21.874 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-29 04:44:21.874 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-01-29_04-44-19__91526413-5e89-46b5-9e50-5714f74883da__wakuorg_nwaku:latest.log
2026-01-29 04:44:21.874 DEBUG [src.node.waku_node] Starting Node...
2026-01-29 04:44:21.874 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-29 04:44:21.876 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-29 04:44:21.876 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.233.43
2026-01-29 04:44:21.876 DEBUG [src.node.docker_mananger] Generated ports ['63834', '63835', '63836', '63837', '63838']
2026-01-29 04:44:21.876 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-29 04:44:21.876 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-29 04:44:21.876 DEBUG [src.node.waku_node] Using volumes []
2026-01-29 04:44:21.876 DEBUG [src.node.docker_mananger] docker run -i -t -p 63834:63834 -p 63835:63835 -p 63836:63836 -p 63837:63837 -p 63838:63838 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=63836 --rest-port=63834 --tcp-port=63835 --discv5-udp-port=63837 --rest-address=0.0.0.0 --nat=extip:172.18.233.43 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=5e9ef7afd8daad73e4e838a1fde7b217eca0ef2a63caae96fa1bf66686ccd6d4 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=63838 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QG6Sj87ZAn9TaW1cbMucfBSMOfHwlSza3wWV0EdYdYfmVE_QXlX38qTZOlTNrkPLMB3TYmvX5q2swJBhdOH95TMCgmlkgnY0gmlwhKwSG-OKbXVsdGlhZGRyc5YACASsEhvjBhoHAAoErBIb4wYaCN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDZk0Q3fbqet2bn1e6uV_bYL6ShkvbqOsW8rmtlIJhUiWDdGNwghoHg3VkcIIaCYV3YWt1MgE
2026-01-29 04:44:22.067 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.233.43 waku 5efe753c59d22b5b5e5e24ce3bab081faf2d978544435641c43bc3122920f283
2026-01-29 04:44:22.097 DEBUG [src.node.docker_mananger] Container started with ID 5efe753c59d2. Setting up logs at ./log/docker/node3_2026-01-29_04-44-19__91526413-5e89-46b5-9e50-5714f74883da__wakuorg_nwaku:latest.log
2026-01-29 04:44:22.098 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 63834
2026-01-29 04:44:22.098 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-29 04:44:23.098 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63834/health" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:23.110 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-29 04:44:23.111 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-29 04:44:23.111 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63834/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:23.115 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.233.43/tcp/63835/p2p/16Uiu2HAm92Z8gjy5UkEoGCFUuc2y5h9e7joXdGTNoUGoXFXfcTR3","/ip4/172.18.233.43/tcp/63836/ws/p2p/16Uiu2HAm92Z8gjy5UkEoGCFUuc2y5h9e7joXdGTNoUGoXFXfcTR3"],"enrUri":"enr:-L24QCXcipAlWfK9oeAW3RhrozA-_8vl32Gc4bX2VIxiOxFOHAFvYflv7ba5ltMnuUntvsZtbrRMJCuQEWTmLUb5QG4CgmlkgnY0gmlwhKwS6SuKbXVsdGlhZGRyc5YACASsEukrBvlbAAoErBLpKwb5XN0DgnJzhQADAQAAiXNlY3AyNTZrMaECyheETIvU4YQuO_1yNCP1tz4Gpo6KavEJ2Epm9OxhejKDdGNwgvlbg3VkcIL5XYV3YWt1MgE"}'
2026-01-29 04:44:23.115 INFO [src.node.waku_node] REST service is ready !!
2026-01-29 04:44:23.116 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63834/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.27.227/tcp/6663/p2p/16Uiu2HAmKYL6Myx4M7H9m5TJ4oDqzguJs81aCe4AMTP6DQEXKudW"]'
2026-01-29 04:44:23.121 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-29 04:44:23.127 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-29 04:44:23.127 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-01-29_04-44-19__91526413-5e89-46b5-9e50-5714f74883da__wakuorg_nwaku:latest.log
2026-01-29 04:44:23.127 DEBUG [src.node.waku_node] Starting Node...
2026-01-29 04:44:23.127 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-29 04:44:23.129 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-29 04:44:23.129 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.134.55
2026-01-29 04:44:23.129 DEBUG [src.node.docker_mananger] Generated ports ['19002', '19003', '19004', '19005', '19006']
2026-01-29 04:44:23.129 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-29 04:44:23.129 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-29 04:44:23.129 DEBUG [src.node.waku_node] Using volumes []
2026-01-29 04:44:23.129 DEBUG [src.node.docker_mananger] docker run -i -t -p 19002:19002 -p 19003:19003 -p 19004:19004 -p 19005:19005 -p 19006:19006 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=19004 --rest-port=19002 --tcp-port=19003 --discv5-udp-port=19005 --rest-address=0.0.0.0 --nat=extip:172.18.134.55 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=ea514694faefdac0db8454fc2fdc8a6e2ede88c3dd88aaff1dd27a6eddbfebe4 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=19006 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QG6Sj87ZAn9TaW1cbMucfBSMOfHwlSza3wWV0EdYdYfmVE_QXlX38qTZOlTNrkPLMB3TYmvX5q2swJBhdOH95TMCgmlkgnY0gmlwhKwSG-OKbXVsdGlhZGRyc5YACASsEhvjBhoHAAoErBIb4wYaCN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDZk0Q3fbqet2bn1e6uV_bYL6ShkvbqOsW8rmtlIJhUiWDdGNwghoHg3VkcIIaCYV3YWt1MgE
2026-01-29 04:44:23.322 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.134.55 waku 202c3e2af5fbd9426a4cf899779b55476fafc8e570cbde6cbc212187fa87ba74
2026-01-29 04:44:23.352 DEBUG [src.node.docker_mananger] Container started with ID 202c3e2af5fb. Setting up logs at ./log/docker/node4_2026-01-29_04-44-19__91526413-5e89-46b5-9e50-5714f74883da__wakuorg_nwaku:latest.log
2026-01-29 04:44:23.353 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 19002
2026-01-29 04:44:23.353 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-29 04:44:24.353 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19002/health" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:24.363 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-29 04:44:24.363 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-29 04:44:24.363 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19002/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:24.369 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.134.55/tcp/19003/p2p/16Uiu2HAmN7YzntYXy7jxEbLDZowFYP5G19Lqk8hZKQvzzpg63YJh","/ip4/172.18.134.55/tcp/19004/ws/p2p/16Uiu2HAmN7YzntYXy7jxEbLDZowFYP5G19Lqk8hZKQvzzpg63YJh"],"enrUri":"enr:-L24QKhVBUEascLH-jepHj1MZdOaNOPKpPqCbA8JkFj_9ja3dLgKmaT_3AmUzKnh1x-6Hqb6oYPq6pjdK__WFwu6sNICgmlkgnY0gmlwhKwShjeKbXVsdGlhZGRyc5YACASsEoY3Bko7AAoErBKGNwZKPN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDjIcybdKMD9D943__Vp_sIb6L7rzricmGHq69OJ-W_56DdGNwgko7g3VkcIJKPYV3YWt1MgE"}'
2026-01-29 04:44:24.369 INFO [src.node.waku_node] REST service is ready !!
2026-01-29 04:44:24.370 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19002/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.27.227/tcp/6663/p2p/16Uiu2HAmKYL6Myx4M7H9m5TJ4oDqzguJs81aCe4AMTP6DQEXKudW"]'
2026-01-29 04:44:24.372 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-29 04:44:24.378 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-01-29 04:44:24.378 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-01-29_04-44-19__91526413-5e89-46b5-9e50-5714f74883da__wakuorg_nwaku:latest.log
2026-01-29 04:44:24.378 DEBUG [src.node.waku_node] Starting Node...
2026-01-29 04:44:24.378 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-01-29 04:44:24.380 DEBUG [src.node.docker_mananger] Network waku already exists
2026-01-29 04:44:24.380 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.121.161
2026-01-29 04:44:24.380 DEBUG [src.node.docker_mananger] Generated ports ['13858', '13859', '13860', '13861', '13862']
2026-01-29 04:44:24.380 DEBUG [src.node.waku_node] RLN credentials were not set
2026-01-29 04:44:24.380 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-01-29 04:44:24.381 DEBUG [src.node.waku_node] Using volumes []
2026-01-29 04:44:24.381 DEBUG [src.node.docker_mananger] docker run -i -t -p 13858:13858 -p 13859:13859 -p 13860:13860 -p 13861:13861 -p 13862:13862 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=13860 --rest-port=13858 --tcp-port=13859 --discv5-udp-port=13861 --rest-address=0.0.0.0 --nat=extip:172.18.121.161 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=2c0f7db85eaef225dcfaea5ca06f83a0b8b03ff5c6e934eb24c4cccb5a28fa17 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=13862 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QG6Sj87ZAn9TaW1cbMucfBSMOfHwlSza3wWV0EdYdYfmVE_QXlX38qTZOlTNrkPLMB3TYmvX5q2swJBhdOH95TMCgmlkgnY0gmlwhKwSG-OKbXVsdGlhZGRyc5YACASsEhvjBhoHAAoErBIb4wYaCN0DgnJzhQADAQAAiXNlY3AyNTZrMaEDZk0Q3fbqet2bn1e6uV_bYL6ShkvbqOsW8rmtlIJhUiWDdGNwghoHg3VkcIIaCYV3YWt1MgE
2026-01-29 04:44:24.572 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.121.161 waku ad6d3f725d83b97944e8096052add081b2c5038e45ae71b6c1968284802379c0
2026-01-29 04:44:24.602 DEBUG [src.node.docker_mananger] Container started with ID ad6d3f725d83. Setting up logs at ./log/docker/node5_2026-01-29_04-44-19__91526413-5e89-46b5-9e50-5714f74883da__wakuorg_nwaku:latest.log
2026-01-29 04:44:24.602 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 13858
2026-01-29 04:44:24.602 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-01-29 04:44:25.604 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13858/health" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:25.615 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-01-29 04:44:25.615 INFO [src.node.waku_node] Node protocols are initialized !!
2026-01-29 04:44:25.616 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13858/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:25.620 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.121.161/tcp/13859/p2p/16Uiu2HAm6mZfaayc3bacyFa4yLhtgeaJmRfPd1j1rAALk49pxTtM","/ip4/172.18.121.161/tcp/13860/ws/p2p/16Uiu2HAm6mZfaayc3bacyFa4yLhtgeaJmRfPd1j1rAALk49pxTtM"],"enrUri":"enr:-L24QF465LqiHRIb0N9eO9gLlGlMRct1-5ytSYBl0USkOaDVWKvoLNmhiLiW3qy5TI9jgBHAlajmxcQnjHL9Bcy8Ck0CgmlkgnY0gmlwhKwSeaGKbXVsdGlhZGRyc5YACASsEnmhBjYjAAoErBJ5oQY2JN0DgnJzhQADAQAAiXNlY3AyNTZrMaECqIkId7rtU_0bzam5GM2OK_3968UslcPh-Yb6tM93khKDdGNwgjYjg3VkcII2JYV3YWt1MgE"}'
2026-01-29 04:44:25.621 INFO [src.node.waku_node] REST service is ready !!
2026-01-29 04:44:25.623 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13858/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.27.227/tcp/6663/p2p/16Uiu2HAmKYL6Myx4M7H9m5TJ4oDqzguJs81aCe4AMTP6DQEXKudW"]'
2026-01-29 04:44:25.626 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-29 04:44:25.626 DEBUG [src.steps.relay] Running fixture setup: subscribe_main_relay_nodes
2026-01-29 04:44:25.627 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6662/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-29 04:44:25.635 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-29 04:44:25.635 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7386/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-29 04:44:25.640 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-29 04:44:25.641 DEBUG [src.steps.relay] Running fixture setup: subscribe_optional_relay_nodes
2026-01-29 04:44:25.641 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63834/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-29 04:44:25.645 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-29 04:44:25.646 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19002/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-29 04:44:25.651 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-29 04:44:25.652 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:13858/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-01-29 04:44:25.657 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-29 04:44:25.658 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6662/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-01-29 04:44:25.667 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-29 04:44:25.667 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-01-29 04:44:25.767 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-01-29 04:44:25.768 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6662/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:25.770 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1769661865658237440,"ephemeral":false,"proof":""}]'
2026-01-29 04:44:25.771 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-01-29 04:44:25.771 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7386/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:25.774 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1769661865658237440,"ephemeral":false,"proof":""}]'
2026-01-29 04:44:25.775 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-01-29 04:44:25.775 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63834/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:25.778 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1769661865658237440,"ephemeral":false,"proof":""}]'
2026-01-29 04:44:25.779 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-01-29 04:44:25.780 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19002/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:25.782 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1769661865658237440,"ephemeral":false,"proof":""}]'
2026-01-29 04:44:25.783 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-01-29 04:44:25.783 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13858/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:25.785 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1769661865658237440,"ephemeral":false,"proof":""}]'
2026-01-29 04:44:25.787 INFO [src.steps.relay] WARM UP successful!!
2026-01-29 04:44:25.788 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6662/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-01-29 04:44:25.794 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-01-29 04:44:25.797 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-01-29 04:44:25.897 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-01-29 04:44:25.897 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6662/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:25.900 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1769661865788182265,"ephemeral":false,"proof":""}]'
2026-01-29 04:44:25.901 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-01-29 04:44:25.901 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7386/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:25.903 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1769661865788182265,"ephemeral":false,"proof":""}]'
2026-01-29 04:44:25.905 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-01-29 04:44:25.905 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63834/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:25.907 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1769661865788182265,"ephemeral":false,"proof":""}]'
2026-01-29 04:44:25.908 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-01-29 04:44:25.908 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19002/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:25.910 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1769661865788182265,"ephemeral":false,"proof":""}]'
2026-01-29 04:44:25.912 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-01-29 04:44:25.912 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:13858/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-01-29 04:44:25.914 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1769661865788182265,"ephemeral":false,"proof":""}]'
2026-01-29 04:44:25.918 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-01-29 04:44:25.919 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-01-29 04:44:25.919 DEBUG [src.node.waku_node] Stopping container with id d9ebd698eb83
2026-01-29 04:44:26.472 DEBUG [src.node.waku_node] Container stopped.
2026-01-29 04:44:26.472 DEBUG [src.node.waku_node] Stopping container with id 121861a64f01
2026-01-29 04:44:26.985 DEBUG [src.node.waku_node] Container stopped.
2026-01-29 04:44:26.985 DEBUG [src.node.waku_node] Stopping container with id 5efe753c59d2
2026-01-29 04:44:27.286 ERROR [src.node.docker_mananger] Max retries reached for container d9ebd698eb83. Exiting log stream.
2026-01-29 04:44:27.514 DEBUG [src.node.waku_node] Container stopped.
2026-01-29 04:44:27.514 DEBUG [src.node.waku_node] Stopping container with id 202c3e2af5fb
2026-01-29 04:44:27.797 ERROR [src.node.docker_mananger] Max retries reached for container 121861a64f01. Exiting log stream.
2026-01-29 04:44:28.025 DEBUG [src.node.waku_node] Container stopped.
2026-01-29 04:44:28.025 DEBUG [src.node.waku_node] Stopping container with id ad6d3f725d83
2026-01-29 04:44:28.326 ERROR [src.node.docker_mananger] Max retries reached for container 5efe753c59d2. Exiting log stream.
2026-01-29 04:44:28.538 DEBUG [src.node.waku_node] Container stopped.
2026-01-29 04:44:28.540 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-01-29 04:44:28.556 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-29 04:44:28.562 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-29 04:44:28.568 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-29 04:44:28.573 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-01-29 04:44:28.580 DEBUG [src.node.docker_mananger] No errors found in the waku logs.