296 lines
58 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

DEBUG  tests.conftest:conftest.py:51 Running fixture setup: test_id
DEBUG  tests.conftest:conftest.py:57 Running test: test_auto_shard_relay_10_nwaku_nodes with id: 2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece
DEBUG  src.steps.common:common.py:19 Running fixture setup: common_setup
DEBUG  src.steps.relay:relay.py:28 Running fixture setup: relay_setup
DEBUG  src.steps.sharding:sharding.py:30 Running fixture setup: sharding_setup
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node1_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.182.12
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['22314', '22315', '22316', '22317', '22318']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 22314:22314 -p 22315:22315 -p 22316:22316 -p 22317:22317 -p 22318:22318 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=22316 --rest-port=22314 --tcp-port=22315 --discv5-udp-port=22317 --rest-address=0.0.0.0 --nat=extip:172.18.182.12 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=6a7e693ac75abc61cf4a2e0cd556e6bef4fc1f02e92af7bb2eb33d90bea9c01a --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=22318 --metrics-logging=true --relay=true --filter=true --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.182.12 waku 8dda342b0db19e25e64b353a105a6ce4020a82dc11c8bb5ea0e58260c0c9c009
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 8dda342b0db1. Setting up logs at ./log/docker/node1_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 22314
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 1d3656ed512e. Exiting log stream.
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:22314/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_READY","desc":"Relay is not ready, filter will not be able to sort out messages"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:22314/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.182.12/tcp/22315/p2p/16Uiu2HAkwtVjYpDmM3mVpsaxQSxx3u9fHPMWgKG8ccTn8VFVCAcU","/ip4/172.18.182.12/tcp/22316/ws/p2p/16Uiu2HAkwtVjYpDmM3mVpsaxQSxx3u9fHPMWgKG8ccTn8VFVCAcU"],"enrUri":"enr:-L24QPakrikF28_0fb5JUW_AxYdUtmFoadudadzIDMRPZ971OwtbocAKy0tdr-whQUOKibkE2Q5Bh55_rIukYTdZWowCgmlkgnY0gmlwhKwStgyKbXVsdGlhZGRyc5YACASsErYMBlcrAAoErBK2DAZXLN0DgnJzhQACAQAAiXNlY3AyNTZrMaECJJZ4ka7XSRr-Upe_K771J63nBYdetGpKfL6NQb9rWAWDdGNwglcrg3VkcIJXLYV3YWt1MgU"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node3_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.136.105
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['21101', '21102', '21103', '21104', '21105']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 21101:21101 -p 21102:21102 -p 21103:21103 -p 21104:21104 -p 21105:21105 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=21103 --rest-port=21101 --tcp-port=21102 --discv5-udp-port=21104 --rest-address=0.0.0.0 --nat=extip:172.18.136.105 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=3fdfbd2e0ba27aaf96eea53cfd2dbe9f5dbb37db9a0f2594fbdc3d75f5a9cff2 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=21105 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPakrikF28_0fb5JUW_AxYdUtmFoadudadzIDMRPZ971OwtbocAKy0tdr-whQUOKibkE2Q5Bh55_rIukYTdZWowCgmlkgnY0gmlwhKwStgyKbXVsdGlhZGRyc5YACASsErYMBlcrAAoErBK2DAZXLN0DgnJzhQACAQAAiXNlY3AyNTZrMaECJJZ4ka7XSRr-Upe_K771J63nBYdetGpKfL6NQb9rWAWDdGNwglcrg3VkcIJXLYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.136.105 waku 9c1da11c9ebbd062d5c648a14d314c8302d5c57908b3481521e9e8ac8fef0d3b
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 9c1da11c9ebb. Setting up logs at ./log/docker/node3_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 21101
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21101/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21101/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.136.105/tcp/21102/p2p/16Uiu2HAmCWxKJS4WKqN4K5uNDx1dYcL9etK2MDXEcnWvsjj7BPr9","/ip4/172.18.136.105/tcp/21103/ws/p2p/16Uiu2HAmCWxKJS4WKqN4K5uNDx1dYcL9etK2MDXEcnWvsjj7BPr9"],"enrUri":"enr:-L24QKxxdV5dAgy6iqyF7pOahfOyp2s9pcnNU6Y_mg6Rm6JXKWfquHpecQlCQzdgdDhWXTSWW2kwklyelbVfgskYodACgmlkgnY0gmlwhKwSiGmKbXVsdGlhZGRyc5YACASsEohpBlJuAAoErBKIaQZSb90DgnJzhQACAQAAiXNlY3AyNTZrMaEC_fENqKOGFFNqOSYMuHeBsKy5hKU4X69JcaHzjR5vYuqDdGNwglJug3VkcIJScIV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:21101/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.182.12/tcp/22315/p2p/16Uiu2HAkwtVjYpDmM3mVpsaxQSxx3u9fHPMWgKG8ccTn8VFVCAcU"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node4_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.126.244
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['55234', '55235', '55236', '55237', '55238']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 55234:55234 -p 55235:55235 -p 55236:55236 -p 55237:55237 -p 55238:55238 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=55236 --rest-port=55234 --tcp-port=55235 --discv5-udp-port=55237 --rest-address=0.0.0.0 --nat=extip:172.18.126.244 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=adb9f5d5ada0cd5eb8ca64ba20b733f99d5ad0fec18b9dede414fcbf77db12d3 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=55238 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPakrikF28_0fb5JUW_AxYdUtmFoadudadzIDMRPZ971OwtbocAKy0tdr-whQUOKibkE2Q5Bh55_rIukYTdZWowCgmlkgnY0gmlwhKwStgyKbXVsdGlhZGRyc5YACASsErYMBlcrAAoErBK2DAZXLN0DgnJzhQACAQAAiXNlY3AyNTZrMaECJJZ4ka7XSRr-Upe_K771J63nBYdetGpKfL6NQb9rWAWDdGNwglcrg3VkcIJXLYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.126.244 waku 9ef7d46ddd8601db87b80b84775bbf4e84cf89787705c87db600333ef0f1648a
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 9ef7d46ddd86. Setting up logs at ./log/docker/node4_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 55234
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:55234/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:55234/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.126.244/tcp/55235/p2p/16Uiu2HAmMzyb1GZf9Sptu5QqLtFXDM266yFkx8jQ6eY5TaxsYVK6","/ip4/172.18.126.244/tcp/55236/ws/p2p/16Uiu2HAmMzyb1GZf9Sptu5QqLtFXDM266yFkx8jQ6eY5TaxsYVK6"],"enrUri":"enr:-L24QI_GTDZbygd8aFKzOWv5vIYLANeK1lyU2w8caqhDP78GOyWYWRnDHFNy88DyKXw___d9autnRDO_Drnx-TXd5_wCgmlkgnY0gmlwhKwSfvSKbXVsdGlhZGRyc5YACASsEn70BtfDAAoErBJ-9AbXxN0DgnJzhQACAQAAiXNlY3AyNTZrMaEDitfvaSu2G2Ta8LJrIFvWO6smtsgcsksv8lwWGxRfBqGDdGNwgtfDg3VkcILXxYV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:55234/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.182.12/tcp/22315/p2p/16Uiu2HAkwtVjYpDmM3mVpsaxQSxx3u9fHPMWgKG8ccTn8VFVCAcU"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node5_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.58.157
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['1028', '1029', '1030', '1031', '1032']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 1028:1028 -p 1029:1029 -p 1030:1030 -p 1031:1031 -p 1032:1032 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=1030 --rest-port=1028 --tcp-port=1029 --discv5-udp-port=1031 --rest-address=0.0.0.0 --nat=extip:172.18.58.157 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=9eedfdc5d4c0c525bfc06c9265b7d75ccdedf1ab5fc93a47b35d8cafda4c9cff --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=1032 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPakrikF28_0fb5JUW_AxYdUtmFoadudadzIDMRPZ971OwtbocAKy0tdr-whQUOKibkE2Q5Bh55_rIukYTdZWowCgmlkgnY0gmlwhKwStgyKbXVsdGlhZGRyc5YACASsErYMBlcrAAoErBK2DAZXLN0DgnJzhQACAQAAiXNlY3AyNTZrMaECJJZ4ka7XSRr-Upe_K771J63nBYdetGpKfL6NQb9rWAWDdGNwglcrg3VkcIJXLYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.58.157 waku 2995ef8fb89416e0b5c071b55c0182f359f56d21feb032680fe6aa15fa904b2c
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 2995ef8fb894. Setting up logs at ./log/docker/node5_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 1028
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:1028/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:1028/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.58.157/tcp/1029/p2p/16Uiu2HAkuyj7w42FpbTRZgrVp8Wzp3F1HahbKzmRUqibYX4mgUF7","/ip4/172.18.58.157/tcp/1030/ws/p2p/16Uiu2HAkuyj7w42FpbTRZgrVp8Wzp3F1HahbKzmRUqibYX4mgUF7"],"enrUri":"enr:-L24QDkE0ZFWAVCNFcuwthZ9eZNP-aULDQa18Z5bGN9vNS3hchaarteNirkgh4-pfqLBSBjhf9ENvzr2jg5V46zTnVcCgmlkgnY0gmlwhKwSOp2KbXVsdGlhZGRyc5YACASsEjqdBgQFAAoErBI6nQYEBt0DgnJzhQACAQAAiXNlY3AyNTZrMaECCDYi_RwVk6Raz7KzFBn6Jiis6S-G6qR7j93pIW4jJraDdGNwggQFg3VkcIIEB4V3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:1028/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.182.12/tcp/22315/p2p/16Uiu2HAkwtVjYpDmM3mVpsaxQSxx3u9fHPMWgKG8ccTn8VFVCAcU"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node6_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.44.172
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['42092', '42093', '42094', '42095', '42096']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 42092:42092 -p 42093:42093 -p 42094:42094 -p 42095:42095 -p 42096:42096 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=42094 --rest-port=42092 --tcp-port=42093 --discv5-udp-port=42095 --rest-address=0.0.0.0 --nat=extip:172.18.44.172 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=52fb6fbcc9b5cda95dbfd786a5ce499ba6b358bcbf6dbdc6ef0cfe8323ecb71c --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=42096 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPakrikF28_0fb5JUW_AxYdUtmFoadudadzIDMRPZ971OwtbocAKy0tdr-whQUOKibkE2Q5Bh55_rIukYTdZWowCgmlkgnY0gmlwhKwStgyKbXVsdGlhZGRyc5YACASsErYMBlcrAAoErBK2DAZXLN0DgnJzhQACAQAAiXNlY3AyNTZrMaECJJZ4ka7XSRr-Upe_K771J63nBYdetGpKfL6NQb9rWAWDdGNwglcrg3VkcIJXLYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.44.172 waku 1970fc24e1adc670c9670f735546c08fee8da9a85a5a703e6c64f8500a70d79f
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 1970fc24e1ad. Setting up logs at ./log/docker/node6_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 42092
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:42092/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:42092/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.44.172/tcp/42093/p2p/16Uiu2HAm8unhjNeNqwCVU2hydx3LUTDbGmzrrrVeNpXyHfDMVxy3","/ip4/172.18.44.172/tcp/42094/ws/p2p/16Uiu2HAm8unhjNeNqwCVU2hydx3LUTDbGmzrrrVeNpXyHfDMVxy3"],"enrUri":"enr:-L24QAweKBvyqFRlYTgPTnFSoGVrdgdSKBHrv3d2ZOBenJFCcB7Gl4wGLps4Pm7UTrsbdKM2UooCscmHkBmb8YCmEKkCgmlkgnY0gmlwhKwSLKyKbXVsdGlhZGRyc5YACASsEiysBqRtAAoErBIsrAakbt0DgnJzhQACAQAAiXNlY3AyNTZrMaECyFvLY9wdPCWgtErL2cWWd82b9OMVAEsrjfzRBavQEg6DdGNwgqRtg3VkcIKkb4V3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:42092/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.182.12/tcp/22315/p2p/16Uiu2HAkwtVjYpDmM3mVpsaxQSxx3u9fHPMWgKG8ccTn8VFVCAcU"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node7_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.37.165
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['40322', '40323', '40324', '40325', '40326']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 40322:40322 -p 40323:40323 -p 40324:40324 -p 40325:40325 -p 40326:40326 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=40324 --rest-port=40322 --tcp-port=40323 --discv5-udp-port=40325 --rest-address=0.0.0.0 --nat=extip:172.18.37.165 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=ecbca2832d85afe331e6f5ebb3c6ed7f55eeecc461aa6ef95d2f5c94fa0e0e65 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=40326 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPakrikF28_0fb5JUW_AxYdUtmFoadudadzIDMRPZ971OwtbocAKy0tdr-whQUOKibkE2Q5Bh55_rIukYTdZWowCgmlkgnY0gmlwhKwStgyKbXVsdGlhZGRyc5YACASsErYMBlcrAAoErBK2DAZXLN0DgnJzhQACAQAAiXNlY3AyNTZrMaECJJZ4ka7XSRr-Upe_K771J63nBYdetGpKfL6NQb9rWAWDdGNwglcrg3VkcIJXLYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.37.165 waku d2c1e00bc8413bea3ece47e126b783f1b2646fa54428f88a821436f046c170ab
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID d2c1e00bc841. Setting up logs at ./log/docker/node7_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 40322
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:40322/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:40322/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.37.165/tcp/40323/p2p/16Uiu2HAm2pmy25c8wK6T18mo2TUkYtPEmaBv7h32epzJ2NRnkYCz","/ip4/172.18.37.165/tcp/40324/ws/p2p/16Uiu2HAm2pmy25c8wK6T18mo2TUkYtPEmaBv7h32epzJ2NRnkYCz"],"enrUri":"enr:-L24QOZ_nXHU-JDAO5cc_zgk0XUeudqgXXdMSwU8efKZkGRoSxYnmEcRP-DUOHB9juxB04pLjy_r1izVIa3rWkY-PZYCgmlkgnY0gmlwhKwSJaWKbXVsdGlhZGRyc5YACASsEiWlBp2DAAoErBIlpQadhN0DgnJzhQACAQAAiXNlY3AyNTZrMaECbezvrGCa7QbUtlTJedJ8V1vtjlIjWccTRsUMFxoO7luDdGNwgp2Dg3VkcIKdhYV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40322/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.182.12/tcp/22315/p2p/16Uiu2HAkwtVjYpDmM3mVpsaxQSxx3u9fHPMWgKG8ccTn8VFVCAcU"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node8_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.25.8
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['42695', '42696', '42697', '42698', '42699']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 42695:42695 -p 42696:42696 -p 42697:42697 -p 42698:42698 -p 42699:42699 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=42697 --rest-port=42695 --tcp-port=42696 --discv5-udp-port=42698 --rest-address=0.0.0.0 --nat=extip:172.18.25.8 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=fa04bfdd97b29eefe0af7307e19debe84af8ee4a3f0b9fe33abc578ebb524cc2 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=42699 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPakrikF28_0fb5JUW_AxYdUtmFoadudadzIDMRPZ971OwtbocAKy0tdr-whQUOKibkE2Q5Bh55_rIukYTdZWowCgmlkgnY0gmlwhKwStgyKbXVsdGlhZGRyc5YACASsErYMBlcrAAoErBK2DAZXLN0DgnJzhQACAQAAiXNlY3AyNTZrMaECJJZ4ka7XSRr-Upe_K771J63nBYdetGpKfL6NQb9rWAWDdGNwglcrg3VkcIJXLYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.25.8 waku 61cd0275bb8e26ea7870edfdbb1714f637c2bb0dc27109ea11b3ee92df76e8ce
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 61cd0275bb8e. Setting up logs at ./log/docker/node8_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 42695
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:42695/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:42695/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.25.8/tcp/42696/p2p/16Uiu2HAm7PTzgTWqPjr1UZDtJBS3sxpoposUrT12i291q8TuhvPW","/ip4/172.18.25.8/tcp/42697/ws/p2p/16Uiu2HAm7PTzgTWqPjr1UZDtJBS3sxpoposUrT12i291q8TuhvPW"],"enrUri":"enr:-L24QHmDh3t1gdB1pcMy3SJU8frZj7XMhb6xwRcUItNaFPFvI952jNwa5ufE5CL6Z89FYRr9Qp_RtR2cmpONqCJKXy4CgmlkgnY0gmlwhKwSGQiKbXVsdGlhZGRyc5YACASsEhkIBqbIAAoErBIZCAamyd0DgnJzhQACAQAAiXNlY3AyNTZrMaECsbuHxZE5v9ZTDeA4yCreojp48p3jc23ufyijPyzzXQ2DdGNwgqbIg3VkcIKmyoV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:42695/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.182.12/tcp/22315/p2p/16Uiu2HAkwtVjYpDmM3mVpsaxQSxx3u9fHPMWgKG8ccTn8VFVCAcU"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node9_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.37.119
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['21270', '21271', '21272', '21273', '21274']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 21270:21270 -p 21271:21271 -p 21272:21272 -p 21273:21273 -p 21274:21274 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=21272 --rest-port=21270 --tcp-port=21271 --discv5-udp-port=21273 --rest-address=0.0.0.0 --nat=extip:172.18.37.119 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=e6757b6ca6d630bfcc2d347af670c7ba7fdd9454784deaaaffc8d79cc08a9777 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=21274 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPakrikF28_0fb5JUW_AxYdUtmFoadudadzIDMRPZ971OwtbocAKy0tdr-whQUOKibkE2Q5Bh55_rIukYTdZWowCgmlkgnY0gmlwhKwStgyKbXVsdGlhZGRyc5YACASsErYMBlcrAAoErBK2DAZXLN0DgnJzhQACAQAAiXNlY3AyNTZrMaECJJZ4ka7XSRr-Upe_K771J63nBYdetGpKfL6NQb9rWAWDdGNwglcrg3VkcIJXLYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.37.119 waku 925d76b6769c62d1649f31c9e4b19f1f51186d669494ca6357a58ed8cb48aeb6
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 925d76b6769c. Setting up logs at ./log/docker/node9_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 21270
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21270/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21270/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.37.119/tcp/21271/p2p/16Uiu2HAkxweM6o5nxh5fWE1QwPKAF2VLbXv6Fvj73MRH7LibE15i","/ip4/172.18.37.119/tcp/21272/ws/p2p/16Uiu2HAkxweM6o5nxh5fWE1QwPKAF2VLbXv6Fvj73MRH7LibE15i"],"enrUri":"enr:-L24QO0igFwn2zQjCIzGBoNE5bmGhg-mAl83VFhYzMYXuJ04cuKlQWPGGntmOItVNaPLKMz4lCyTFCF6K6cQMg4AMP8CgmlkgnY0gmlwhKwSJXeKbXVsdGlhZGRyc5YACASsEiV3BlMXAAoErBIldwZTGN0DgnJzhQACAQAAiXNlY3AyNTZrMaECNECjlzBkOyU4LUkaqeL3QqtvFr3UfVb7EeNkdPxinZmDdGNwglMXg3VkcIJTGYV3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:21270/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.182.12/tcp/22315/p2p/16Uiu2HAkwtVjYpDmM3mVpsaxQSxx3u9fHPMWgKG8ccTn8VFVCAcU"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.node.docker_mananger:docker_mananger.py:19 Docker client initialized with image wakuorg/nwaku:latest
DEBUG  src.node.waku_node:waku_node.py:86 WakuNode instance initialized with log path ./log/docker/node10_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:90 Starting Node...
DEBUG  src.node.docker_mananger:docker_mananger.py:22 Attempting to create or retrieve network waku
DEBUG  src.node.docker_mananger:docker_mananger.py:25 Network waku already exists
DEBUG  src.node.docker_mananger:docker_mananger.py:108 Generated random external IP 172.18.208.104
DEBUG  src.node.docker_mananger:docker_mananger.py:101 Generated ports ['50116', '50117', '50118', '50119', '50120']
DEBUG  src.node.waku_node:waku_node.py:439 RLN credentials were not set
INFO  src.node.waku_node:waku_node.py:176 RLN credentials not set or credential store not available, starting without RLN
DEBUG  src.node.waku_node:waku_node.py:178 Using volumes []
DEBUG  src.node.docker_mananger:docker_mananger.py:49 docker run -i -t -p 50116:50116 -p 50117:50117 -p 50118:50118 -p 50119:50119 -p 50120:50120 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=50118 --rest-port=50116 --tcp-port=50117 --discv5-udp-port=50119 --rest-address=0.0.0.0 --nat=extip:172.18.208.104 --peer-exchange=true --discv5-discovery=true --cluster-id=2 --nodekey=a894c3ded3febaff68eddaf9fecefd23fe311cf4de5add59c4f6332e5e6f2331 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=50120 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QPakrikF28_0fb5JUW_AxYdUtmFoadudadzIDMRPZ971OwtbocAKy0tdr-whQUOKibkE2Q5Bh55_rIukYTdZWowCgmlkgnY0gmlwhKwStgyKbXVsdGlhZGRyc5YACASsErYMBlcrAAoErBK2DAZXLN0DgnJzhQACAQAAiXNlY3AyNTZrMaECJJZ4ka7XSRr-Upe_K771J63nBYdetGpKfL6NQb9rWAWDdGNwglcrg3VkcIJXLYV3YWt1MgU --content-topic=/myapp/1/latest/proto
DEBUG  src.node.docker_mananger:docker_mananger.py:55 docker network connect --ip 172.18.208.104 waku 0d2a2e7487cd057a26df7a3e5f1d4fde69813d6c6e5610cf369b29a94c2d94ce
DEBUG  src.node.docker_mananger:docker_mananger.py:58 Container started with ID 0d2a2e7487cd. Setting up logs at ./log/docker/node10_2026-02-27_04-37-57__473b6e5c-868e-4560-94d1-9a07b0f9fece__wakuorg_nwaku:latest.log
DEBUG  src.node.waku_node:waku_node.py:190 Started container from image wakuorg/nwaku:latest. REST: 50116
DEBUG  src.libs.common:common.py:47 Sleeping for 1 seconds
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:50116/health" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
INFO  src.node.waku_node:waku_node.py:287 Node protocols are initialized !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:50116/debug/v1/info" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.208.104/tcp/50117/p2p/16Uiu2HAm5otZRRpeHm2UAVuMM2ZtxNDpNXYykmrbMrfXy31UPzBL","/ip4/172.18.208.104/tcp/50118/ws/p2p/16Uiu2HAm5otZRRpeHm2UAVuMM2ZtxNDpNXYykmrbMrfXy31UPzBL"],"enrUri":"enr:-L24QMcSyEeJL1IVZUOVf51Cj6EFaxGLYa43BxDHyjZYW-VpH6r2PxnwgJdbk2_9KnspjXGpha3EJQS5ZL_jE4eUAPsCgmlkgnY0gmlwhKwS0GiKbXVsdGlhZGRyc5YACASsEtBoBsPFAAoErBLQaAbDxt0DgnJzhQACAQAAiXNlY3AyNTZrMaECmkXe_70piudkJuI06FLamvTbgBWJNN7vZye6a7WjBfuDdGNwgsPFg3VkcILDx4V3YWt1MgE"}'
INFO  src.node.waku_node:waku_node.py:292 REST service is ready !!
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:50116/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.182.12/tcp/22315/p2p/16Uiu2HAkwtVjYpDmM3mVpsaxQSxx3u9fHPMWgKG8ccTn8VFVCAcU"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:22314/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:21101/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:55234/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:1028/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:42092/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:40322/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:42695/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:21270/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:50116/relay/v1/auto/subscriptions" -H "Content-Type: application/json" -d '["/myapp/1/latest/proto"]'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X POST "http://127.0.0.1:22314/relay/v1/auto/messages" -H "Content-Type: application/json" -d '{"payload": "U2hhcmRpbmcgd29ya3MhIQ==", "contentTopic": "/myapp/1/latest/proto", "timestamp": '$(date +%s%N)'}'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'OK'
DEBUG  src.libs.common:common.py:47 Sleeping for 0.1 seconds
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:22314/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772167089543038796,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21101/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772167089543038796,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:55234/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772167089543038796,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:1028/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772167089543038796,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:42092/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772167089543038796,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_6:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:40322/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772167089543038796,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_7:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:42695/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772167089543038796,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_8:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:21270/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772167089543038796,"ephemeral":false,"proof":""}]'
DEBUG  src.steps.sharding:sharding.py:158 Checking that peer NODE_9:wakuorg/nwaku:latest can find the published message
INFO  src.node.api_clients.base_client:base_client.py:37 curl -v -X GET "http://127.0.0.1:50116/relay/v1/auto/messages/%2Fmyapp%2F1%2Flatest%2Fproto" -H "Content-Type: application/json" -d 'None'
INFO  src.node.api_clients.base_client:base_client.py:22 Response status code: 200. Response content: b'[{"payload":"U2hhcmRpbmcgd29ya3MhIQ==","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772167089543038796,"ephemeral":false,"proof":""}]'
DEBUG  tests.conftest:conftest.py:59 Running fixture teardown: test_setup
DEBUG  tests.conftest:conftest.py:83 Running fixture teardown: close_open_nodes
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 8dda342b0db1
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 9c1da11c9ebb
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 9ef7d46ddd86
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 8dda342b0db1. Exiting log stream.
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 9c1da11c9ebb. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 2995ef8fb894
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 9ef7d46ddd86. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 1970fc24e1ad
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id d2c1e00bc841
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 2995ef8fb894. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 61cd0275bb8e
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 1970fc24e1ad. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 925d76b6769c
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container d2c1e00bc841. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  src.node.waku_node:waku_node.py:234 Stopping container with id 0d2a2e7487cd
ERROR  src.node.docker_mananger:docker_mananger.py:89 Max retries reached for container 61cd0275bb8e. Exiting log stream.
DEBUG  src.node.waku_node:waku_node.py:241 Container stopped.
DEBUG  tests.conftest:conftest.py:98 Running fixture teardown: check_waku_log_errors
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.
DEBUG  src.node.docker_mananger:docker_mananger.py:144 No errors found in the waku logs.