2026-02-25 20:22:16 +00:00

188 lines
34 KiB
Plaintext

2026-02-25 20:07:24.985 DEBUG [tests.conftest] Running fixture setup: test_id
2026-02-25 20:07:24.985 DEBUG [tests.conftest] Running test: test_multiple_receiving_nodes with id: 2026-02-25_20-07-24__444c2685-f18f-4542-85aa-4c59f64c2ea8
2026-02-25 20:07:24.986 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-02-25 20:07:24.986 DEBUG [src.steps.light_push] Running fixture setup: light_push_setup
2026-02-25 20:07:24.992 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-25 20:07:24.992 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node1_2026-02-25_20-07-24__444c2685-f18f-4542-85aa-4c59f64c2ea8__wakuorg_nwaku:latest.log
2026-02-25 20:07:24.992 DEBUG [src.node.waku_node] Starting Node...
2026-02-25 20:07:24.992 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-25 20:07:24.994 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-25 20:07:24.994 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.70.145
2026-02-25 20:07:24.994 DEBUG [src.node.docker_mananger] Generated ports ['31683', '31684', '31685', '31686', '31687']
2026-02-25 20:07:24.994 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-25 20:07:24.994 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-25 20:07:24.994 DEBUG [src.node.waku_node] Using volumes []
2026-02-25 20:07:24.994 DEBUG [src.node.docker_mananger] docker run -i -t -p 31683:31683 -p 31684:31684 -p 31685:31685 -p 31686:31686 -p 31687:31687 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=31685 --rest-port=31683 --tcp-port=31684 --discv5-udp-port=31686 --rest-address=0.0.0.0 --nat=extip:172.18.70.145 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=52e3c4c5fa48847b5cd334c0fa15b5bb0dcdd0dad4ebfe6baaabae8bbbafabd3 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=31687 --metrics-logging=true --lightpush=true --relay=true
2026-02-25 20:07:25.133 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.70.145 waku 36f3dde8dd6c5fc286afca3e2275e87dca589fb279a59b84082304960e04c99c
2026-02-25 20:07:25.159 DEBUG [src.node.docker_mananger] Container started with ID 36f3dde8dd6c. Setting up logs at ./log/docker/receiving_node1_2026-02-25_20-07-24__444c2685-f18f-4542-85aa-4c59f64c2ea8__wakuorg_nwaku:latest.log
2026-02-25 20:07:25.160 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 31683
2026-02-25 20:07:25.161 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-25 20:07:25.312 ERROR [src.node.docker_mananger] Max retries reached for container 868cffcd54a1. Exiting log stream.
2026-02-25 20:07:25.759 ERROR [src.node.docker_mananger] Max retries reached for container ad1de50bd933. Exiting log stream.
2026-02-25 20:07:26.163 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:31683/health" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:26.166 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-25 20:07:26.166 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-25 20:07:26.166 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:31683/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:26.168 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.70.145/tcp/31684/p2p/16Uiu2HAkvnShHSA69ZNXRQxhmhDc9NJPHS4EDPYwgZThomriFiuK","/ip4/172.18.70.145/tcp/31685/ws/p2p/16Uiu2HAkvnShHSA69ZNXRQxhmhDc9NJPHS4EDPYwgZThomriFiuK"],"enrUri":"enr:-L24QITJDqR2dSL6QULfQhGZAOEjarpcaQy_hdvnAoJePyrTY9khvkGcBaTSuiGlpjKUMfgI1OFLP2vZotDFssBOjYACgmlkgnY0gmlwhKwSRpGKbXVsdGlhZGRyc5YACASsEkaRBnvEAAoErBJGkQZ7xd0DgnJzhQADAQAAiXNlY3AyNTZrMaECFC3c6HM5g6Wzt-S7XsvpBdHu3MmtXGsk4o4bB3TFaf6DdGNwgnvEg3VkcIJ7xoV3YWt1Mgk"}'
2026-02-25 20:07:26.168 INFO [src.node.waku_node] REST service is ready !!
2026-02-25 20:07:26.175 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-25 20:07:26.175 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node2_2026-02-25_20-07-24__444c2685-f18f-4542-85aa-4c59f64c2ea8__wakuorg_nwaku:latest.log
2026-02-25 20:07:26.175 DEBUG [src.node.waku_node] Starting Node...
2026-02-25 20:07:26.175 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-25 20:07:26.176 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-25 20:07:26.176 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.238.93
2026-02-25 20:07:26.177 DEBUG [src.node.docker_mananger] Generated ports ['7018', '7019', '7020', '7021', '7022']
2026-02-25 20:07:26.177 DEBUG [src.node.waku_node] Cluster id was resolved to: 3
2026-02-25 20:07:26.177 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-02-25 20:07:26.177 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-25 20:07:26.177 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-25 20:07:26.177 DEBUG [src.node.waku_node] Using volumes []
2026-02-25 20:07:26.177 DEBUG [src.node.docker_mananger] docker run -i -t -p 7018:7018 -p 7019:7019 -p 7020:7020 -p 7021:7021 -p 7022:7022 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=7020 --rest-port=7018 --tcp-port=7019 --discv5-udp-port=7021 --rest-address=0.0.0.0 --nat=extip:172.18.238.93 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=363a6c1c43a35bec28cdae20ef4c0decd72cd10789fd22b4b9fe82a2ef0f33ae --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=7022 --metrics-logging=true --lightpush=true --relay=true
2026-02-25 20:07:26.331 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.238.93 waku 589f8d93ac942772b226d0989b9b608f9e1836b6cfc5f362c389033bd2b7051b
2026-02-25 20:07:26.355 DEBUG [src.node.docker_mananger] Container started with ID 589f8d93ac94. Setting up logs at ./log/docker/receiving_node2_2026-02-25_20-07-24__444c2685-f18f-4542-85aa-4c59f64c2ea8__wakuorg_nwaku:latest.log
2026-02-25 20:07:26.355 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 7018
2026-02-25 20:07:26.356 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-25 20:07:27.356 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7018/health" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:27.359 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-25 20:07:27.359 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-25 20:07:27.359 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7018/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:27.361 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.238.93/tcp/7019/p2p/16Uiu2HAm8ZXvvApGqZUQwEGYMNetQLToEsLu3qrU6ytZT7LvM3QW","/ip4/172.18.238.93/tcp/7020/ws/p2p/16Uiu2HAm8ZXvvApGqZUQwEGYMNetQLToEsLu3qrU6ytZT7LvM3QW"],"enrUri":"enr:-L24QDyyvOqZNbGCcYb-wBeEdOJZFoDHORfurv53HiNqOI7EAnYydmvuj5922grYbyqIP8Pep_WdGpH1iXUnNCTjCl4CgmlkgnY0gmlwhKwS7l2KbXVsdGlhZGRyc5YACASsEu5dBhtrAAoErBLuXQYbbN0DgnJzhQADAQAAiXNlY3AyNTZrMaECwyt5jcDryXoX2GM_IoGOfeeecNCHr-n4sSRTYPrkFmuDdGNwghtrg3VkcIIbbYV3YWt1Mgk"}'
2026-02-25 20:07:27.361 INFO [src.node.waku_node] REST service is ready !!
2026-02-25 20:07:27.362 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7018/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.70.145/tcp/31684/p2p/16Uiu2HAkvnShHSA69ZNXRQxhmhDc9NJPHS4EDPYwgZThomriFiuK"]'
2026-02-25 20:07:27.393 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:27.401 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-25 20:07:27.401 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node3_2026-02-25_20-07-24__444c2685-f18f-4542-85aa-4c59f64c2ea8__wakuorg_nwaku:latest.log
2026-02-25 20:07:27.401 DEBUG [src.node.waku_node] Starting Node...
2026-02-25 20:07:27.401 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-25 20:07:27.402 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-25 20:07:27.402 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.66.159
2026-02-25 20:07:27.403 DEBUG [src.node.docker_mananger] Generated ports ['30825', '30826', '30827', '30828', '30829']
2026-02-25 20:07:27.403 DEBUG [src.node.waku_node] Cluster id was resolved to: 3
2026-02-25 20:07:27.403 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-02-25 20:07:27.403 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-25 20:07:27.403 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-25 20:07:27.403 DEBUG [src.node.waku_node] Using volumes []
2026-02-25 20:07:27.403 DEBUG [src.node.docker_mananger] docker run -i -t -p 30825:30825 -p 30826:30826 -p 30827:30827 -p 30828:30828 -p 30829:30829 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=30827 --rest-port=30825 --tcp-port=30826 --discv5-udp-port=30828 --rest-address=0.0.0.0 --nat=extip:172.18.66.159 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=696510febcf9f92f76c6cc9b056ae4d1edb0cdf4ff91bacb7aaaa3edfaebdc51 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=30829 --metrics-logging=true --lightpush=true --relay=true
2026-02-25 20:07:27.555 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.66.159 waku c0896c64395b565217aa0a506a73e913c3b1760a9ae7d56934e9bdc961abfc89
2026-02-25 20:07:27.579 DEBUG [src.node.docker_mananger] Container started with ID c0896c64395b. Setting up logs at ./log/docker/receiving_node3_2026-02-25_20-07-24__444c2685-f18f-4542-85aa-4c59f64c2ea8__wakuorg_nwaku:latest.log
2026-02-25 20:07:27.579 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 30825
2026-02-25 20:07:27.579 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-25 20:07:28.580 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30825/health" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:28.583 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-25 20:07:28.583 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-25 20:07:28.583 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30825/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:28.586 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.66.159/tcp/30826/p2p/16Uiu2HAmCJGFaUuYtYZcBBNToJPbntsTbvLSYf5CrM2wGNPMPKL9","/ip4/172.18.66.159/tcp/30827/ws/p2p/16Uiu2HAmCJGFaUuYtYZcBBNToJPbntsTbvLSYf5CrM2wGNPMPKL9"],"enrUri":"enr:-L24QBAxKKuX9046FYKjeSYYV_bnlFtB1jqzNVwq9qL98vflBUIr4z8eDcL0_DLO-PwSCleo1wK-qXa_Nerc7hqx8f0CgmlkgnY0gmlwhKwSQp-KbXVsdGlhZGRyc5YACASsEkKfBnhqAAoErBJCnwZ4a90DgnJzhQADAQAAiXNlY3AyNTZrMaEC-rDIbTJ049aohACkSBRP1cA2VfS1hvEofhaE4PFFyg6DdGNwgnhqg3VkcIJ4bIV3YWt1Mgk"}'
2026-02-25 20:07:28.586 INFO [src.node.waku_node] REST service is ready !!
2026-02-25 20:07:28.586 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30825/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.70.145/tcp/31684/p2p/16Uiu2HAkvnShHSA69ZNXRQxhmhDc9NJPHS4EDPYwgZThomriFiuK"]'
2026-02-25 20:07:28.622 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:28.623 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30825/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.238.93/tcp/7019/p2p/16Uiu2HAm8ZXvvApGqZUQwEGYMNetQLToEsLu3qrU6ytZT7LvM3QW"]'
2026-02-25 20:07:28.645 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:28.656 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-25 20:07:28.656 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node4_2026-02-25_20-07-24__444c2685-f18f-4542-85aa-4c59f64c2ea8__wakuorg_nwaku:latest.log
2026-02-25 20:07:28.657 DEBUG [src.node.waku_node] Starting Node...
2026-02-25 20:07:28.659 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-25 20:07:28.662 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-25 20:07:28.662 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.177.135
2026-02-25 20:07:28.662 DEBUG [src.node.docker_mananger] Generated ports ['17160', '17161', '17162', '17163', '17164']
2026-02-25 20:07:28.662 DEBUG [src.node.waku_node] Cluster id was resolved to: 3
2026-02-25 20:07:28.662 DEBUG [src.node.waku_node] Removing pubsub-topic from nwaku args
2026-02-25 20:07:28.663 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-25 20:07:28.663 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-25 20:07:28.663 DEBUG [src.node.waku_node] Using volumes []
2026-02-25 20:07:28.663 DEBUG [src.node.docker_mananger] docker run -i -t -p 17160:17160 -p 17161:17161 -p 17162:17162 -p 17163:17163 -p 17164:17164 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=17162 --rest-port=17160 --tcp-port=17161 --discv5-udp-port=17163 --rest-address=0.0.0.0 --nat=extip:172.18.177.135 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=ac533466bbdcbf3beea594dba1ce98fcdf2602deabbf263053627baaf6fcec8b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=17164 --metrics-logging=true --lightpush=true --relay=true
2026-02-25 20:07:28.814 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.177.135 waku a8d6af138eaed88fe8cf09a3b4c944e83e3d0c30e0531b493f5713d6dd545bab
2026-02-25 20:07:28.840 DEBUG [src.node.docker_mananger] Container started with ID a8d6af138eae. Setting up logs at ./log/docker/receiving_node4_2026-02-25_20-07-24__444c2685-f18f-4542-85aa-4c59f64c2ea8__wakuorg_nwaku:latest.log
2026-02-25 20:07:28.840 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 17160
2026-02-25 20:07:28.840 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-25 20:07:29.841 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:17160/health" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:29.843 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-25 20:07:29.844 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-25 20:07:29.844 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:17160/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:29.846 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.177.135/tcp/17161/p2p/16Uiu2HAmLr62Cu1u5o7n1QpjScxKxVWQQRkn241buBhysXsPuGYw","/ip4/172.18.177.135/tcp/17162/ws/p2p/16Uiu2HAmLr62Cu1u5o7n1QpjScxKxVWQQRkn241buBhysXsPuGYw"],"enrUri":"enr:-L24QKARTW6ntn1CU7gO7GGRThEsooNHBbf19uUpFCmtcxfBV-GSNAcbqlfCW3lrFHHUUTFEoLbuFTpF5H07r7cnM9gCgmlkgnY0gmlwhKwSsYeKbXVsdGlhZGRyc5YACASsErGHBkMJAAoErBKxhwZDCt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDebVL5vXeKJ8WqAc95Z4seK3qkVLz-bxqGagSTtrr7ZiDdGNwgkMJg3VkcIJDC4V3YWt1Mgk"}'
2026-02-25 20:07:29.846 INFO [src.node.waku_node] REST service is ready !!
2026-02-25 20:07:29.846 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:17160/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.70.145/tcp/31684/p2p/16Uiu2HAkvnShHSA69ZNXRQxhmhDc9NJPHS4EDPYwgZThomriFiuK"]'
2026-02-25 20:07:29.883 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:29.884 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:17160/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.238.93/tcp/7019/p2p/16Uiu2HAm8ZXvvApGqZUQwEGYMNetQLToEsLu3qrU6ytZT7LvM3QW"]'
2026-02-25 20:07:29.911 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:29.911 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:17160/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.66.159/tcp/30826/p2p/16Uiu2HAmCJGFaUuYtYZcBBNToJPbntsTbvLSYf5CrM2wGNPMPKL9"]'
2026-02-25 20:07:29.932 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:29.943 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-25 20:07:29.943 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node1_2026-02-25_20-07-24__444c2685-f18f-4542-85aa-4c59f64c2ea8__wakuorg_nwaku:latest.log
2026-02-25 20:07:29.943 DEBUG [src.node.waku_node] Starting Node...
2026-02-25 20:07:29.943 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-25 20:07:29.945 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-25 20:07:29.945 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.32.158
2026-02-25 20:07:29.945 DEBUG [src.node.docker_mananger] Generated ports ['11469', '11470', '11471', '11472', '11473']
2026-02-25 20:07:29.945 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-25 20:07:29.945 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-25 20:07:29.945 DEBUG [src.node.waku_node] Using volumes []
2026-02-25 20:07:29.946 DEBUG [src.node.docker_mananger] docker run -i -t -p 11469:11469 -p 11470:11470 -p 11471:11471 -p 11472:11472 -p 11473:11473 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=11471 --rest-port=11469 --tcp-port=11470 --discv5-udp-port=11472 --rest-address=0.0.0.0 --nat=extip:172.18.32.158 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=9eeadca190df484a28a2b7b6a2e275af2e293cbbc0f4d8ebfe4f75a7a3a1cc1a --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=11473 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QITJDqR2dSL6QULfQhGZAOEjarpcaQy_hdvnAoJePyrTY9khvkGcBaTSuiGlpjKUMfgI1OFLP2vZotDFssBOjYACgmlkgnY0gmlwhKwSRpGKbXVsdGlhZGRyc5YACASsEkaRBnvEAAoErBJGkQZ7xd0DgnJzhQADAQAAiXNlY3AyNTZrMaECFC3c6HM5g6Wzt-S7XsvpBdHu3MmtXGsk4o4bB3TFaf6DdGNwgnvEg3VkcIJ7xoV3YWt1Mgk --lightpushnode=/ip4/172.18.70.145/tcp/31684/p2p/16Uiu2HAkvnShHSA69ZNXRQxhmhDc9NJPHS4EDPYwgZThomriFiuK --lightpush=true --relay=true
2026-02-25 20:07:30.104 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.32.158 waku 1f7b4568cc719144a813ce5c28b2e75952d6b753bda3165005509e9acbf80e3f
2026-02-25 20:07:30.130 DEBUG [src.node.docker_mananger] Container started with ID 1f7b4568cc71. Setting up logs at ./log/docker/lightpush_node1_2026-02-25_20-07-24__444c2685-f18f-4542-85aa-4c59f64c2ea8__wakuorg_nwaku:latest.log
2026-02-25 20:07:30.130 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 11469
2026-02-25 20:07:30.130 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-25 20:07:31.131 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:11469/health" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:31.133 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-25 20:07:31.133 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-25 20:07:31.134 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:11469/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:31.136 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.32.158/tcp/11470/p2p/16Uiu2HAkwNRADSUYhfJJoNxJ3cfqhM1gLBrJX9EFKsXF9E8YgYpr","/ip4/172.18.32.158/tcp/11471/ws/p2p/16Uiu2HAkwNRADSUYhfJJoNxJ3cfqhM1gLBrJX9EFKsXF9E8YgYpr"],"enrUri":"enr:-L24QMJrCEko2NipQmK3dAahn77sESOEyU6zHeNl8K4FyXSmZhUdO7p_Lz1HGtUK1DxVd27TpFLX3DVIsk1gORt3PoICgmlkgnY0gmlwhKwSIJ6KbXVsdGlhZGRyc5YACASsEiCeBizOAAoErBIgngYsz90DgnJzhQADAQAAiXNlY3AyNTZrMaECHOHfwUIt4ZOD0gSqKnUhWTthvPtjR-y6VbJ4bCs8YduDdGNwgizOg3VkcIIs0IV3YWt1Mgk"}'
2026-02-25 20:07:31.136 INFO [src.node.waku_node] REST service is ready !!
2026-02-25 20:07:31.136 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:11469/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.70.145/tcp/31684/p2p/16Uiu2HAkvnShHSA69ZNXRQxhmhDc9NJPHS4EDPYwgZThomriFiuK"]'
2026-02-25 20:07:31.174 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:31.174 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:11469/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.238.93/tcp/7019/p2p/16Uiu2HAm8ZXvvApGqZUQwEGYMNetQLToEsLu3qrU6ytZT7LvM3QW"]'
2026-02-25 20:07:31.193 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:31.194 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:11469/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.66.159/tcp/30826/p2p/16Uiu2HAmCJGFaUuYtYZcBBNToJPbntsTbvLSYf5CrM2wGNPMPKL9"]'
2026-02-25 20:07:31.218 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:31.219 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:11469/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.177.135/tcp/17161/p2p/16Uiu2HAmLr62Cu1u5o7n1QpjScxKxVWQQRkn241buBhysXsPuGYw"]'
2026-02-25 20:07:31.253 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:31.254 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:31683/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-25 20:07:31.256 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:31.256 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7018/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-25 20:07:31.258 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:31.259 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:30825/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-25 20:07:31.261 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:31.261 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:17160/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-25 20:07:31.263 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:31.263 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:11469/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-25 20:07:31.265 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:31.266 DEBUG [src.steps.light_push] Lightpushing message
2026-02-25 20:07:31.266 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:11469/lightpush/v1/message" -H "Content-Type: application/json" -d '{"pubsubTopic": "/waku/2/rs/3/0", "message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1772050051265967194}}'
2026-02-25 20:07:31.293 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-25 20:07:31.293 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-02-25 20:07:31.399 DEBUG [src.steps.light_push] Checking that peer NODE_1:wakuorg/nwaku:latest can find the lightpushed message
2026-02-25 20:07:31.400 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:31683/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:31.402 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772050051265967194,"ephemeral":false,"proof":""}]'
2026-02-25 20:07:31.403 DEBUG [src.steps.light_push] Checking that peer NODE_2:wakuorg/nwaku:latest can find the lightpushed message
2026-02-25 20:07:31.403 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7018/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:31.405 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772050051265967194,"ephemeral":false,"proof":""}]'
2026-02-25 20:07:31.406 DEBUG [src.steps.light_push] Checking that peer NODE_3:wakuorg/nwaku:latest can find the lightpushed message
2026-02-25 20:07:31.406 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:30825/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:31.408 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772050051265967194,"ephemeral":false,"proof":""}]'
2026-02-25 20:07:31.409 DEBUG [src.steps.light_push] Checking that peer NODE_4:wakuorg/nwaku:latest can find the lightpushed message
2026-02-25 20:07:31.410 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:17160/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:31.412 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772050051265967194,"ephemeral":false,"proof":""}]'
2026-02-25 20:07:31.413 DEBUG [src.steps.light_push] Checking that peer NODE_5:wakuorg/nwaku:latest can find the lightpushed message
2026-02-25 20:07:31.413 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:11469/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-25 20:07:31.415 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772050051265967194,"ephemeral":false,"proof":""}]'
2026-02-25 20:07:31.417 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-02-25 20:07:31.418 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-02-25 20:07:31.418 DEBUG [src.node.waku_node] Stopping container with id 36f3dde8dd6c
2026-02-25 20:07:31.914 DEBUG [src.node.waku_node] Container stopped.
2026-02-25 20:07:31.915 DEBUG [src.node.waku_node] Stopping container with id 589f8d93ac94
2026-02-25 20:07:32.389 DEBUG [src.node.waku_node] Container stopped.
2026-02-25 20:07:32.389 DEBUG [src.node.waku_node] Stopping container with id c0896c64395b
2026-02-25 20:07:32.726 ERROR [src.node.docker_mananger] Max retries reached for container 36f3dde8dd6c. Exiting log stream.
2026-02-25 20:07:32.875 DEBUG [src.node.waku_node] Container stopped.
2026-02-25 20:07:32.876 DEBUG [src.node.waku_node] Stopping container with id a8d6af138eae
2026-02-25 20:07:33.201 ERROR [src.node.docker_mananger] Max retries reached for container 589f8d93ac94. Exiting log stream.
2026-02-25 20:07:33.356 DEBUG [src.node.waku_node] Container stopped.
2026-02-25 20:07:33.356 DEBUG [src.node.waku_node] Stopping container with id 1f7b4568cc71
2026-02-25 20:07:33.686 ERROR [src.node.docker_mananger] Max retries reached for container c0896c64395b. Exiting log stream.
2026-02-25 20:07:33.810 DEBUG [src.node.waku_node] Container stopped.
2026-02-25 20:07:33.812 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-02-25 20:07:33.826 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-25 20:07:33.839 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-25 20:07:33.852 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-25 20:07:33.864 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-25 20:07:33.878 DEBUG [src.node.docker_mananger] No errors found in the waku logs.