308 lines
58 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_auto_shard_relay_10_nwaku_nodes with id: 2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.239.32
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['11542', '11543', '11544', '11545', '11546']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 11542:11542 -p 11543:11543 -p 11544:11544 -p 11545:11545 -p 11546:11546 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=11544 --rest-port=11542 --tcp-port=11543 --discv5-udp-port=11545 --rest-address=0.0.0.0 --nat=extip:172.18.239.32 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=1afef2aac34af3b0ccbed0e9aa7b19ffbed1c58a67d5eb1d772beda2316a2338 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=11546 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.239.32 waku 6df6d1dd6ee1924b404c9054e6da551e9f5c5404bc75e94b20853850072be3cf
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 6df6d1dd6ee1. Setting up logs at ./log/docker/node1_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 11542
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 6a166737b345. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:11542/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"READY"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:11542/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.239.32/tcp/11543/p2p/16Uiu2HAmS4vKpNgPoHsLUeqXyvLkdwqqMPsHcxEwio7WKRV9NsKX","/ip4/172.18.239.32/tcp/11544/ws/p2p/16Uiu2HAmS4vKpNgPoHsLUeqXyvLkdwqqMPsHcxEwio7WKRV9NsKX"],"enrUri":"enr:-L24QNGbzJKZmg78tF7o-FrXxXDlgKESxAgrdX0sYqkmKMjtBMrMBkVNl7sjzz8zafgXz5_3E1-MF5MI1EWf5ottxEgCgmlkgnY0gmlwhKwS7yCKbXVsdGlhZGRyc5YACASsEu8gBi0XAAoErBLvIAYtGN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDx0lRZMrEuw1QZ3T277IjMDqK2i_HLI_o6e4GTHVxrIKDdGNwgi0Xg3VkcIItGYV3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node3_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.177.14
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['10307', '10308', '10309', '10310', '10311']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 10307:10307 -p 10308:10308 -p 10309:10309 -p 10310:10310 -p 10311:10311 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=10309 --rest-port=10307 --tcp-port=10308 --discv5-udp-port=10310 --rest-address=0.0.0.0 --nat=extip:172.18.177.14 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=d1ddbefae4aab0d971e444efa1a0a36ea316ebd8dd48fc4fc0893d4dea0f37cd --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=10311 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNGbzJKZmg78tF7o-FrXxXDlgKESxAgrdX0sYqkmKMjtBMrMBkVNl7sjzz8zafgXz5_3E1-MF5MI1EWf5ottxEgCgmlkgnY0gmlwhKwS7yCKbXVsdGlhZGRyc5YACASsEu8gBi0XAAoErBLvIAYtGN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDx0lRZMrEuw1QZ3T277IjMDqK2i_HLI_o6e4GTHVxrIKDdGNwgi0Xg3VkcIItGYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.177.14 waku b65f7cb0b3370d4e84667f0c6a807ca9c31e5aaa2eb824ee8a1aa7363e54ca35
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID b65f7cb0b337. Setting up logs at ./log/docker/node3_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 10307
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:10307/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:10307/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:10307/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:10307/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:10307/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.177.14/tcp/10308/p2p/16Uiu2HAmN6qpZPNJri1VbDFCmsgK4ZbEH1jgx9HiXjGPMEXS8qMD","/ip4/172.18.177.14/tcp/10309/ws/p2p/16Uiu2HAmN6qpZPNJri1VbDFCmsgK4ZbEH1jgx9HiXjGPMEXS8qMD"],"enrUri":"enr:-L24QFcgFgjtB6Iz8AO9nUdnFbvDTCqyaYxhmW5nuOteueYpEpTBl67DONgydsm-W_mEwKSdfNifCLvkQjAr7lF6IYQCgmlkgnY0gmlwhKwSsQ6KbXVsdGlhZGRyc5YACASsErEOBihEAAoErBKxDgYoRd0DgnJzhQACAQAAiXNlY3AyNTZrMaEDjFijfsJUh2_ZHMsvNVnmLZgO4Wl79wrCv8RgYcYQaPyDdGNwgihEg3VkcIIoRoV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:10307/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.239.32/tcp/11543/p2p/16Uiu2HAmS4vKpNgPoHsLUeqXyvLkdwqqMPsHcxEwio7WKRV9NsKX"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node4_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.81.3
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['48859', '48860', '48861', '48862', '48863']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 48859:48859 -p 48860:48860 -p 48861:48861 -p 48862:48862 -p 48863:48863 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=48861 --rest-port=48859 --tcp-port=48860 --discv5-udp-port=48862 --rest-address=0.0.0.0 --nat=extip:172.18.81.3 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=5aedad3c51cdbba6c5dae3013a286fe83cc21a50ce603f6c3b6070b23c84fb68 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=48863 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNGbzJKZmg78tF7o-FrXxXDlgKESxAgrdX0sYqkmKMjtBMrMBkVNl7sjzz8zafgXz5_3E1-MF5MI1EWf5ottxEgCgmlkgnY0gmlwhKwS7yCKbXVsdGlhZGRyc5YACASsEu8gBi0XAAoErBLvIAYtGN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDx0lRZMrEuw1QZ3T277IjMDqK2i_HLI_o6e4GTHVxrIKDdGNwgi0Xg3VkcIItGYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.81.3 waku 5af89b33fe81825e3a9c7be1341e0917f03576d1b6508eeb735b5e57ebee8100
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 5af89b33fe81. Setting up logs at ./log/docker/node4_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 48859
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:48859/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:48859/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.81.3/tcp/48860/p2p/16Uiu2HAm86MB8wfYfdFs4RLWiTogUmyp7s5Dj7gHP53CQKgHmGU7","/ip4/172.18.81.3/tcp/48861/ws/p2p/16Uiu2HAm86MB8wfYfdFs4RLWiTogUmyp7s5Dj7gHP53CQKgHmGU7"],"enrUri":"enr:-L24QPLW4rIKFazLf2VibpUIfmY-vMjIUt_gakNH_nqmsiHfMVNqJJgusu6QKjvU9ORjo9MXCWqa5xiBJwhXbCAx-VUCgmlkgnY0gmlwhKwSUQOKbXVsdGlhZGRyc5YACASsElEDBr7cAAoErBJRAwa-3d0DgnJzhQACAQAAiXNlY3AyNTZrMaECvDSh6KjSQt5CisJZYUBFpQ8a9kjcq9ck9kSiSUQyh4CDdGNwgr7cg3VkcIK-3oV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:48859/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.239.32/tcp/11543/p2p/16Uiu2HAmS4vKpNgPoHsLUeqXyvLkdwqqMPsHcxEwio7WKRV9NsKX"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node5_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.149.6
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['21051', '21052', '21053', '21054', '21055']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 21051:21051 -p 21052:21052 -p 21053:21053 -p 21054:21054 -p 21055:21055 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=21053 --rest-port=21051 --tcp-port=21052 --discv5-udp-port=21054 --rest-address=0.0.0.0 --nat=extip:172.18.149.6 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=f1daa21a511ee536fb121e2ec13d2cfad9daabe7e0a0b7b9ecfddad59fe391cf --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=21055 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNGbzJKZmg78tF7o-FrXxXDlgKESxAgrdX0sYqkmKMjtBMrMBkVNl7sjzz8zafgXz5_3E1-MF5MI1EWf5ottxEgCgmlkgnY0gmlwhKwS7yCKbXVsdGlhZGRyc5YACASsEu8gBi0XAAoErBLvIAYtGN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDx0lRZMrEuw1QZ3T277IjMDqK2i_HLI_o6e4GTHVxrIKDdGNwgi0Xg3VkcIItGYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.149.6 waku 8d199e22d52b3ffc5fc459f42d608cd0f200c6d0ef99027cac1d18f67fedd33c
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 8d199e22d52b. Setting up logs at ./log/docker/node5_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 21051
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21051/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21051/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.149.6/tcp/21052/p2p/16Uiu2HAkxxhww3C1uVUFq51bHUpK2X1Ye5PWzUZjymZFMdPz1TSQ","/ip4/172.18.149.6/tcp/21053/ws/p2p/16Uiu2HAkxxhww3C1uVUFq51bHUpK2X1Ye5PWzUZjymZFMdPz1TSQ"],"enrUri":"enr:-L24QGEYirduij4Nr746d4SeLxEGAqv3e5F1IO0p8oChg0VbD91P4SYsTDdq3xAbJcRxCoJcAI9bb2w6KBdxWv3zN8cCgmlkgnY0gmlwhKwSlQaKbXVsdGlhZGRyc5YACASsEpUGBlI8AAoErBKVBgZSPd0DgnJzhQACAQAAiXNlY3AyNTZrMaECNIZKfiTps-W2aBcxvrJd0jDgN1fMOaAznY7LNyS_ELmDdGNwglI8g3VkcIJSPoV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:21051/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.239.32/tcp/11543/p2p/16Uiu2HAmS4vKpNgPoHsLUeqXyvLkdwqqMPsHcxEwio7WKRV9NsKX"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node6_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.9.160
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['62344', '62345', '62346', '62347', '62348']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 62344:62344 -p 62345:62345 -p 62346:62346 -p 62347:62347 -p 62348:62348 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=62346 --rest-port=62344 --tcp-port=62345 --discv5-udp-port=62347 --rest-address=0.0.0.0 --nat=extip:172.18.9.160 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=346afb045d24337caddacbad33cd355c146fd5cfa97a2337fdda16a9b7bbefc0 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=62348 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNGbzJKZmg78tF7o-FrXxXDlgKESxAgrdX0sYqkmKMjtBMrMBkVNl7sjzz8zafgXz5_3E1-MF5MI1EWf5ottxEgCgmlkgnY0gmlwhKwS7yCKbXVsdGlhZGRyc5YACASsEu8gBi0XAAoErBLvIAYtGN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDx0lRZMrEuw1QZ3T277IjMDqK2i_HLI_o6e4GTHVxrIKDdGNwgi0Xg3VkcIItGYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.9.160 waku 2fb0ce939ebe8e861b05d2c3c2256e89b22cc01ca58f2003e339f50ab354be80
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 2fb0ce939ebe. Setting up logs at ./log/docker/node6_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 62344
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:62344/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:62344/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.9.160/tcp/62345/p2p/16Uiu2HAmKgTzkc4LAqjf5LKdVr4EyJFgACs9fny4ozfyeFcamNmZ","/ip4/172.18.9.160/tcp/62346/ws/p2p/16Uiu2HAmKgTzkc4LAqjf5LKdVr4EyJFgACs9fny4ozfyeFcamNmZ"],"enrUri":"enr:-L24QKi29dlXcH12RGsdgVMNeImIMDFYA72A_9rc_u8Uw6AFKzlteRnCTCoyOC8DyxRcNtXgc1VHHggXHBp6fR9_ztECgmlkgnY0gmlwhKwSCaCKbXVsdGlhZGRyc5YACASsEgmgBvOJAAoErBIJoAbzit0DgnJzhQACAQAAiXNlY3AyNTZrMaEDaGKmDCYUFpknfpL_7V4sGoUIZaBiTaPyYR--EL7b9NyDdGNwgvOJg3VkcILzi4V3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:62344/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.239.32/tcp/11543/p2p/16Uiu2HAmS4vKpNgPoHsLUeqXyvLkdwqqMPsHcxEwio7WKRV9NsKX"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node7_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.143.38
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['43457', '43458', '43459', '43460', '43461']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 43457:43457 -p 43458:43458 -p 43459:43459 -p 43460:43460 -p 43461:43461 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=43459 --rest-port=43457 --tcp-port=43458 --discv5-udp-port=43460 --rest-address=0.0.0.0 --nat=extip:172.18.143.38 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=7996bdc1f2c0654694c4dffa3fef1c2bfeaedfbc2d11f1e59cd875ffdcfdaea4 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=43461 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNGbzJKZmg78tF7o-FrXxXDlgKESxAgrdX0sYqkmKMjtBMrMBkVNl7sjzz8zafgXz5_3E1-MF5MI1EWf5ottxEgCgmlkgnY0gmlwhKwS7yCKbXVsdGlhZGRyc5YACASsEu8gBi0XAAoErBLvIAYtGN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDx0lRZMrEuw1QZ3T277IjMDqK2i_HLI_o6e4GTHVxrIKDdGNwgi0Xg3VkcIItGYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.143.38 waku 9d5417423c55d9de75f7b15f903458646973d43ac6c51e61e4d4ac747922d6e3
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 9d5417423c55. Setting up logs at ./log/docker/node7_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 43457
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43457/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43457/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43457/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43457/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.143.38/tcp/43458/p2p/16Uiu2HAmR6TPZ38LhbZMYMcTDQXrpPn7jJsSdWGwxzM8pSANLstQ","/ip4/172.18.143.38/tcp/43459/ws/p2p/16Uiu2HAmR6TPZ38LhbZMYMcTDQXrpPn7jJsSdWGwxzM8pSANLstQ"],"enrUri":"enr:-L24QChPvfNR3jYiKS1nrT5oad2epvyhZs_lhbm9w7B0YyFPX0nCQDtYhfoF2O9F9olEZo6iURJmrG9zIxDmA5tzEyQCgmlkgnY0gmlwhKwSjyaKbXVsdGlhZGRyc5YACASsEo8mBqnCAAoErBKPJgapw90DgnJzhQACAQAAiXNlY3AyNTZrMaEDuNJWInL2RWVoMIlUjC-KLv7ZSmLG9IYvFKztI_SyV1WDdGNwgqnCg3VkcIKpxIV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:43457/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.239.32/tcp/11543/p2p/16Uiu2HAmS4vKpNgPoHsLUeqXyvLkdwqqMPsHcxEwio7WKRV9NsKX"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node8_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.172.246
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['10328', '10329', '10330', '10331', '10332']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 10328:10328 -p 10329:10329 -p 10330:10330 -p 10331:10331 -p 10332:10332 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=10330 --rest-port=10328 --tcp-port=10329 --discv5-udp-port=10331 --rest-address=0.0.0.0 --nat=extip:172.18.172.246 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=5ed9aa231ab8ba56be5bb0e7dfcb4a30f5ecdfca800fdf838e4ae8faf6fdee95 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=10332 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNGbzJKZmg78tF7o-FrXxXDlgKESxAgrdX0sYqkmKMjtBMrMBkVNl7sjzz8zafgXz5_3E1-MF5MI1EWf5ottxEgCgmlkgnY0gmlwhKwS7yCKbXVsdGlhZGRyc5YACASsEu8gBi0XAAoErBLvIAYtGN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDx0lRZMrEuw1QZ3T277IjMDqK2i_HLI_o6e4GTHVxrIKDdGNwgi0Xg3VkcIItGYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.172.246 waku 051de1eeaa115636035347b68a8f29ab2ca2940c725efe434e5cdcf9a0d25be3
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 051de1eeaa11. Setting up logs at ./log/docker/node8_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 10328
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:10328/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:10328/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.172.246/tcp/10329/p2p/16Uiu2HAm1ppz3AH8XyiSZaUe7Fx8XnKe4t7ez4x1p2m76S6ccrHD","/ip4/172.18.172.246/tcp/10330/ws/p2p/16Uiu2HAm1ppz3AH8XyiSZaUe7Fx8XnKe4t7ez4x1p2m76S6ccrHD"],"enrUri":"enr:-L24QHj3vEY-X5nK7q4bllib-7aLmybK4VjK4Sz8kOdPjn1kE3W6vvgzn1OX_onK9UctR4ZJAYanSokut5XO-rFTNMMCgmlkgnY0gmlwhKwSrPaKbXVsdGlhZGRyc5YACASsEqz2BihZAAoErBKs9gYoWt0DgnJzhQACAQAAiXNlY3AyNTZrMaECXxSpSMQGn2UEiDEAloruYQR4qmkbZLWZdtwR0H7b2xiDdGNwgihZg3VkcIIoW4V3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:10328/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.239.32/tcp/11543/p2p/16Uiu2HAmS4vKpNgPoHsLUeqXyvLkdwqqMPsHcxEwio7WKRV9NsKX"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node9_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.57.213
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['47388', '47389', '47390', '47391', '47392']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 47388:47388 -p 47389:47389 -p 47390:47390 -p 47391:47391 -p 47392:47392 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=47390 --rest-port=47388 --tcp-port=47389 --discv5-udp-port=47391 --rest-address=0.0.0.0 --nat=extip:172.18.57.213 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=307cbdae483de22292a3bbe2f63bbf7b593d2d5fb061ed8ed0cc7e8a5caad921 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=47392 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNGbzJKZmg78tF7o-FrXxXDlgKESxAgrdX0sYqkmKMjtBMrMBkVNl7sjzz8zafgXz5_3E1-MF5MI1EWf5ottxEgCgmlkgnY0gmlwhKwS7yCKbXVsdGlhZGRyc5YACASsEu8gBi0XAAoErBLvIAYtGN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDx0lRZMrEuw1QZ3T277IjMDqK2i_HLI_o6e4GTHVxrIKDdGNwgi0Xg3VkcIItGYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.57.213 waku 8fb61d01d78468373431c7d1731f13627c215a1b22f21e0b9fd12076d389a2b5
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 8fb61d01d784. Setting up logs at ./log/docker/node9_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 47388
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:47388/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"INITIALIZING","connectionStatus":"Disconnected","protocolsHealth":[]}'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:47388/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:47388/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.57.213/tcp/47389/p2p/16Uiu2HAkzyeG2DGN3Laa2i2r8sZ5n4VDj1PPuZ1c7thPMCbpjrhj","/ip4/172.18.57.213/tcp/47390/ws/p2p/16Uiu2HAkzyeG2DGN3Laa2i2r8sZ5n4VDj1PPuZ1c7thPMCbpjrhj"],"enrUri":"enr:-L24QKGYGbkLa_KDAX7xg_YD_A9XzcLVnxzoTMOIV1D0b-lfX4zvYj9qAd0zFoyoyomob9ppmeMt_68kmUgfEapIGYUCgmlkgnY0gmlwhKwSOdWKbXVsdGlhZGRyc5YACASsEjnVBrkdAAoErBI51Qa5Ht0DgnJzhQACAQAAiXNlY3AyNTZrMaECUnsTUsQsMJisSAiQ-SmiTtobJHfuSBvXiyKvCnRaJZ6DdGNwgrkdg3VkcIK5H4V3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:47388/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.239.32/tcp/11543/p2p/16Uiu2HAmS4vKpNgPoHsLUeqXyvLkdwqqMPsHcxEwio7WKRV9NsKX"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node10_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.101.245
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['7914', '7915', '7916', '7917', '7918']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 7914:7914 -p 7915:7915 -p 7916:7916 -p 7917:7917 -p 7918:7918 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=7916 --rest-port=7914 --tcp-port=7915 --discv5-udp-port=7917 --rest-address=0.0.0.0 --nat=extip:172.18.101.245 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=8069d93324c9aff19ba0837febe7a66cdfca0f9aa5ff2638cb60bf7193f041dd --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=7918 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QNGbzJKZmg78tF7o-FrXxXDlgKESxAgrdX0sYqkmKMjtBMrMBkVNl7sjzz8zafgXz5_3E1-MF5MI1EWf5ottxEgCgmlkgnY0gmlwhKwS7yCKbXVsdGlhZGRyc5YACASsEu8gBi0XAAoErBLvIAYtGN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDx0lRZMrEuw1QZ3T277IjMDqK2i_HLI_o6e4GTHVxrIKDdGNwgi0Xg3VkcIItGYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.101.245 waku a2d77171f9f4917b6e9261372cd2ba759649434c35d40de754e25763c50c79e0
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID a2d77171f9f4. Setting up logs at ./log/docker/node10_2026-03-29_04-54-30__be6879cc-d916-4ef7-979a-586d74336389__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 7914
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:7914/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"PartiallyConnected","protocolsHealth":[{"Relay":"READY"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"READY"},{"Legacy Lightpush Client":"READY"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"READY"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:7914/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.101.245/tcp/7915/p2p/16Uiu2HAkwDsF6DTYcJ3NTigV9b8L8aYNCezhjJVdQMF7v5Y6Gn16","/ip4/172.18.101.245/tcp/7916/ws/p2p/16Uiu2HAkwDsF6DTYcJ3NTigV9b8L8aYNCezhjJVdQMF7v5Y6Gn16"],"enrUri":"enr:-L24QHUThJYUbFCAlqEc35nThDBU3mBeZSnIg4_d9pUxP5A8SKZuSayt8u9SfiDGicV6UJZypbMCBXkIQxI6EmIePgMCgmlkgnY0gmlwhKwSZfWKbXVsdGlhZGRyc5YACASsEmX1Bh7rAAoErBJl9QYe7N0DgnJzhQACAQAAiXNlY3AyNTZrMaECGrEj3aaKKhK3mMHST3PkT2mC1Q9PZGlwq8eRB08iluGDdGNwgh7rg3VkcIIe7YV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:7914/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.239.32/tcp/11543/p2p/16Uiu2HAmS4vKpNgPoHsLUeqXyvLkdwqqMPsHcxEwio7WKRV9NsKX"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:11542/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:10307/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:48859/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:21051/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:62344/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:43457/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:10328/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:47388/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:7914/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:11542/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.libs.common:common.py:47 Sleeping for 0.1 seconds
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:11542/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774760082602865593,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:10307/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774760082602865593,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:48859/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774760082602865593,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21051/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774760082602865593,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:62344/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774760082602865593,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_6:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:43457/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774760082602865593,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_7:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:10328/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774760082602865593,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_8:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:47388/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774760082602865593,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_9:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:7914/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1774760082602865593,"ephemeral":false,"proof":""}]'
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 6df6d1dd6ee1
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id b65f7cb0b337
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 5af89b33fe81
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 6df6d1dd6ee1. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 8d199e22d52b
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container b65f7cb0b337. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 2fb0ce939ebe
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 5af89b33fe81. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 9d5417423c55
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 8d199e22d52b. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 051de1eeaa11
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 2fb0ce939ebe. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 8fb61d01d784
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 9d5417423c55. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id a2d77171f9f4
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 051de1eeaa11. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.