2026-02-28 20:41:24 +00:00

276 lines
52 KiB
Plaintext

2026-02-28 20:22:26.059 DEBUG [tests.conftest] Running fixture setup: test_id
2026-02-28 20:22:26.060 DEBUG [tests.conftest] Running test: test_multiple_lightpush_nodes with id: 2026-02-28_20-22-26__4ec97691-e888-4f2a-be87-8e16869aa005
2026-02-28 20:22:26.060 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-02-28 20:22:26.061 DEBUG [src.steps.light_push] Running fixture setup: light_push_setup
2026-02-28 20:22:26.071 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-28 20:22:26.072 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node1_2026-02-28_20-22-26__4ec97691-e888-4f2a-be87-8e16869aa005__wakuorg_nwaku:latest.log
2026-02-28 20:22:26.072 DEBUG [src.node.waku_node] Starting Node...
2026-02-28 20:22:26.072 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-28 20:22:26.074 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-28 20:22:26.075 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.43.65
2026-02-28 20:22:26.075 DEBUG [src.node.docker_mananger] Generated ports ['10414', '10415', '10416', '10417', '10418']
2026-02-28 20:22:26.075 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-28 20:22:26.075 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-28 20:22:26.075 DEBUG [src.node.waku_node] Using volumes []
2026-02-28 20:22:26.076 DEBUG [src.node.docker_mananger] docker run -i -t -p 10414:10414 -p 10415:10415 -p 10416:10416 -p 10417:10417 -p 10418:10418 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=10416 --rest-port=10414 --tcp-port=10415 --discv5-udp-port=10417 --rest-address=0.0.0.0 --nat=extip:172.18.43.65 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=305cf06cd36fead955b0cffe33e954fead92b5fcc5c95bedad9cfa77f8dd1b35 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=10418 --metrics-logging=true --lightpush=true --relay=true
2026-02-28 20:22:26.267 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.43.65 waku ddd672ec66beb7d8c2bdd1f801d35d128c22743a32fb8cb90e03100789a389f6
2026-02-28 20:22:26.292 ERROR [src.node.docker_mananger] Max retries reached for container 28d15ac15fec. Exiting log stream.
2026-02-28 20:22:26.307 DEBUG [src.node.docker_mananger] Container started with ID ddd672ec66be. Setting up logs at ./log/docker/receiving_node1_2026-02-28_20-22-26__4ec97691-e888-4f2a-be87-8e16869aa005__wakuorg_nwaku:latest.log
2026-02-28 20:22:26.308 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 10414
2026-02-28 20:22:26.308 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-28 20:22:26.892 ERROR [src.node.docker_mananger] Max retries reached for container 7743bbcc146e. Exiting log stream.
2026-02-28 20:22:27.308 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10414/health" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:27.311 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-28 20:22:27.311 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-28 20:22:27.312 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10414/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:27.314 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.43.65/tcp/10415/p2p/16Uiu2HAkv3vRxjf4LnDhMLrqFoBaQnF4fHtCofWaCPJaqawAaVuj","/ip4/172.18.43.65/tcp/10416/ws/p2p/16Uiu2HAkv3vRxjf4LnDhMLrqFoBaQnF4fHtCofWaCPJaqawAaVuj"],"enrUri":"enr:-L24QFJKV3gCAm5vEw-Xa2ySy52YE8zZU1mhmVUgqN2Srb1td_3lWZw7ABmvXN7fR8tozPtj4wLdy7Vp_nhbtHfKIXsCgmlkgnY0gmlwhKwSK0GKbXVsdGlhZGRyc5YACASsEitBBiivAAoErBIrQQYosN0DgnJzhQADAQAAiXNlY3AyNTZrMaECCUk_xp49KDqD9KjIJoZU8Ia-gQdbgFdHQ99kQflJYZqDdGNwgiivg3VkcIIosYV3YWt1Mgk"}'
2026-02-28 20:22:27.314 INFO [src.node.waku_node] REST service is ready !!
2026-02-28 20:22:27.321 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-28 20:22:27.321 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node2_2026-02-28_20-22-26__4ec97691-e888-4f2a-be87-8e16869aa005__wakuorg_nwaku:latest.log
2026-02-28 20:22:27.321 DEBUG [src.node.waku_node] Starting Node...
2026-02-28 20:22:27.321 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-28 20:22:27.323 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-28 20:22:27.323 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.49.192
2026-02-28 20:22:27.323 DEBUG [src.node.docker_mananger] Generated ports ['36052', '36053', '36054', '36055', '36056']
2026-02-28 20:22:27.323 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-28 20:22:27.323 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-28 20:22:27.323 DEBUG [src.node.waku_node] Using volumes []
2026-02-28 20:22:27.324 DEBUG [src.node.docker_mananger] docker run -i -t -p 36052:36052 -p 36053:36053 -p 36054:36054 -p 36055:36055 -p 36056:36056 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=36054 --rest-port=36052 --tcp-port=36053 --discv5-udp-port=36055 --rest-address=0.0.0.0 --nat=extip:172.18.49.192 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=dcecffa05273de4ceb1d93cee2fbb7489bc95fea5ff0844326630a420043bad4 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=36056 --metrics-logging=true --lightpush=false --relay=true
2026-02-28 20:22:27.523 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.49.192 waku 0483be9e08cf5313d1c0622b38bd74f1c743b6c48d11e6b4e7d768bdb1ba3f84
2026-02-28 20:22:27.561 DEBUG [src.node.docker_mananger] Container started with ID 0483be9e08cf. Setting up logs at ./log/docker/receiving_node2_2026-02-28_20-22-26__4ec97691-e888-4f2a-be87-8e16869aa005__wakuorg_nwaku:latest.log
2026-02-28 20:22:27.561 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 36052
2026-02-28 20:22:27.562 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-28 20:22:28.563 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36052/health" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:28.566 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-28 20:22:28.566 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-28 20:22:28.566 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36052/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:28.569 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.49.192/tcp/36053/p2p/16Uiu2HAmHNTUWuH4gHgLo7mAfn8ZQuYzFqyGzkXgncwFPHVka1UZ","/ip4/172.18.49.192/tcp/36054/ws/p2p/16Uiu2HAmHNTUWuH4gHgLo7mAfn8ZQuYzFqyGzkXgncwFPHVka1UZ"],"enrUri":"enr:-L24QJPecHaCvgWr7RHEn8qtQA-5yeu7YRr4NngXrpUotfiaVF2ElZfwe8jVX4_qVHABo47XcEgSpZDbkIBx2_LsqrACgmlkgnY0gmlwhKwSMcCKbXVsdGlhZGRyc5YACASsEjHABozVAAoErBIxwAaM1t0DgnJzhQADAQAAiXNlY3AyNTZrMaEDRg47ErX3kjczgkzsvoUOqBIO4W1OefmhYGkos69wrZaDdGNwgozVg3VkcIKM14V3YWt1MgE"}'
2026-02-28 20:22:28.569 INFO [src.node.waku_node] REST service is ready !!
2026-02-28 20:22:28.569 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:36052/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.43.65/tcp/10415/p2p/16Uiu2HAkv3vRxjf4LnDhMLrqFoBaQnF4fHtCofWaCPJaqawAaVuj"]'
2026-02-28 20:22:28.604 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:28.620 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-28 20:22:28.621 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node1_2026-02-28_20-22-26__4ec97691-e888-4f2a-be87-8e16869aa005__wakuorg_nwaku:latest.log
2026-02-28 20:22:28.621 DEBUG [src.node.waku_node] Starting Node...
2026-02-28 20:22:28.621 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-28 20:22:28.622 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-28 20:22:28.622 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.95.79
2026-02-28 20:22:28.623 DEBUG [src.node.docker_mananger] Generated ports ['15921', '15922', '15923', '15924', '15925']
2026-02-28 20:22:28.623 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-28 20:22:28.623 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-28 20:22:28.623 DEBUG [src.node.waku_node] Using volumes []
2026-02-28 20:22:28.623 DEBUG [src.node.docker_mananger] docker run -i -t -p 15921:15921 -p 15922:15922 -p 15923:15923 -p 15924:15924 -p 15925:15925 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=15923 --rest-port=15921 --tcp-port=15922 --discv5-udp-port=15924 --rest-address=0.0.0.0 --nat=extip:172.18.95.79 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=b3bbabeeba7c047dbdf5c9a8b04029332c9546ae01bcdabadeaff98f6e2477df --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=15925 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QFJKV3gCAm5vEw-Xa2ySy52YE8zZU1mhmVUgqN2Srb1td_3lWZw7ABmvXN7fR8tozPtj4wLdy7Vp_nhbtHfKIXsCgmlkgnY0gmlwhKwSK0GKbXVsdGlhZGRyc5YACASsEitBBiivAAoErBIrQQYosN0DgnJzhQADAQAAiXNlY3AyNTZrMaECCUk_xp49KDqD9KjIJoZU8Ia-gQdbgFdHQ99kQflJYZqDdGNwgiivg3VkcIIosYV3YWt1Mgk --lightpushnode=/ip4/172.18.43.65/tcp/10415/p2p/16Uiu2HAkv3vRxjf4LnDhMLrqFoBaQnF4fHtCofWaCPJaqawAaVuj --lightpush=true --relay=true
2026-02-28 20:22:28.825 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.95.79 waku 51a87fa191351292d3ae20b6aa21bb26a1865aa4b251e3da709485cb0cf98186
2026-02-28 20:22:28.865 DEBUG [src.node.docker_mananger] Container started with ID 51a87fa19135. Setting up logs at ./log/docker/lightpush_node1_2026-02-28_20-22-26__4ec97691-e888-4f2a-be87-8e16869aa005__wakuorg_nwaku:latest.log
2026-02-28 20:22:28.865 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 15921
2026-02-28 20:22:28.866 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-28 20:22:29.867 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15921/health" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:29.875 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-28 20:22:29.876 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-28 20:22:29.877 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15921/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:29.885 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.95.79/tcp/15922/p2p/16Uiu2HAm5J6Kuo8Bsj6CkwVVaUYxXybm51BUeFFDnR7osVm6X3en","/ip4/172.18.95.79/tcp/15923/ws/p2p/16Uiu2HAm5J6Kuo8Bsj6CkwVVaUYxXybm51BUeFFDnR7osVm6X3en"],"enrUri":"enr:-L24QADKPiGxP6jbg01M_yJdhD_8vkuKidpidbSVCc6OgS8RMhBwA3WWECSH3B2YTy2oSxmlSuRGPGuTMfpxKiKlYrsCgmlkgnY0gmlwhKwSX0-KbXVsdGlhZGRyc5YACASsEl9PBj4yAAoErBJfTwY-M90DgnJzhQADAQAAiXNlY3AyNTZrMaECkqPAelyeZqgYSpw8XcLdHffw8DJclc2-BpqXc8XrOleDdGNwgj4yg3VkcII-NIV3YWt1Mgk"}'
2026-02-28 20:22:29.886 INFO [src.node.waku_node] REST service is ready !!
2026-02-28 20:22:29.886 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:15921/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.43.65/tcp/10415/p2p/16Uiu2HAkv3vRxjf4LnDhMLrqFoBaQnF4fHtCofWaCPJaqawAaVuj"]'
2026-02-28 20:22:29.911 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:29.912 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:15921/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.49.192/tcp/36053/p2p/16Uiu2HAmHNTUWuH4gHgLo7mAfn8ZQuYzFqyGzkXgncwFPHVka1UZ"]'
2026-02-28 20:22:29.938 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:29.950 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-28 20:22:29.950 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node2_2026-02-28_20-22-26__4ec97691-e888-4f2a-be87-8e16869aa005__wakuorg_nwaku:latest.log
2026-02-28 20:22:29.950 DEBUG [src.node.waku_node] Starting Node...
2026-02-28 20:22:29.951 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-28 20:22:29.954 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-28 20:22:29.954 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.114.43
2026-02-28 20:22:29.955 DEBUG [src.node.docker_mananger] Generated ports ['6204', '6205', '6206', '6207', '6208']
2026-02-28 20:22:29.956 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-28 20:22:29.956 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-28 20:22:29.956 DEBUG [src.node.waku_node] Using volumes []
2026-02-28 20:22:29.957 DEBUG [src.node.docker_mananger] docker run -i -t -p 6204:6204 -p 6205:6205 -p 6206:6206 -p 6207:6207 -p 6208:6208 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=6206 --rest-port=6204 --tcp-port=6205 --discv5-udp-port=6207 --rest-address=0.0.0.0 --nat=extip:172.18.114.43 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=c7fedadbaac0a6eeeadd8c065df1b4560b2c08e358140b99f8bfc2fbb29f91cc --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=6208 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QFJKV3gCAm5vEw-Xa2ySy52YE8zZU1mhmVUgqN2Srb1td_3lWZw7ABmvXN7fR8tozPtj4wLdy7Vp_nhbtHfKIXsCgmlkgnY0gmlwhKwSK0GKbXVsdGlhZGRyc5YACASsEitBBiivAAoErBIrQQYosN0DgnJzhQADAQAAiXNlY3AyNTZrMaECCUk_xp49KDqD9KjIJoZU8Ia-gQdbgFdHQ99kQflJYZqDdGNwgiivg3VkcIIosYV3YWt1Mgk --lightpushnode=/ip4/172.18.43.65/tcp/10415/p2p/16Uiu2HAkv3vRxjf4LnDhMLrqFoBaQnF4fHtCofWaCPJaqawAaVuj --lightpush=true --relay=true
2026-02-28 20:22:30.173 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.114.43 waku 49fbfc951b0cc199f0d0df344548381ec7d8f28db5b8b144a6f577c4e396dd0e
2026-02-28 20:22:30.209 DEBUG [src.node.docker_mananger] Container started with ID 49fbfc951b0c. Setting up logs at ./log/docker/lightpush_node2_2026-02-28_20-22-26__4ec97691-e888-4f2a-be87-8e16869aa005__wakuorg_nwaku:latest.log
2026-02-28 20:22:30.209 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 6204
2026-02-28 20:22:30.210 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-28 20:22:31.210 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6204/health" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:31.218 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-28 20:22:31.222 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-28 20:22:31.222 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6204/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:31.230 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.114.43/tcp/6205/p2p/16Uiu2HAmDQvJbxq3MAEYYeo3yYM5hzCw8N6ARBUVzda7iKu8Bp2A","/ip4/172.18.114.43/tcp/6206/ws/p2p/16Uiu2HAmDQvJbxq3MAEYYeo3yYM5hzCw8N6ARBUVzda7iKu8Bp2A"],"enrUri":"enr:-L24QGCdRyYCGd00MEX8mudka6gAGUoJ-zcDLjGYcslqoG5eF31C20emkhYhXpJm_NrCXIN9EHnRvphf39JgjPa1BjgCgmlkgnY0gmlwhKwSciuKbXVsdGlhZGRyc5YACASsEnIrBhg9AAoErBJyKwYYPt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDC0D6_JURS5xSQ9cdkz7R8jBQGRAkppi9ZmdS0Rc3Zl-DdGNwghg9g3VkcIIYP4V3YWt1Mgk"}'
2026-02-28 20:22:31.231 INFO [src.node.waku_node] REST service is ready !!
2026-02-28 20:22:31.232 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6204/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.43.65/tcp/10415/p2p/16Uiu2HAkv3vRxjf4LnDhMLrqFoBaQnF4fHtCofWaCPJaqawAaVuj"]'
2026-02-28 20:22:31.254 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:31.255 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6204/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.49.192/tcp/36053/p2p/16Uiu2HAmHNTUWuH4gHgLo7mAfn8ZQuYzFqyGzkXgncwFPHVka1UZ"]'
2026-02-28 20:22:31.284 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:31.297 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-28 20:22:31.298 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node3_2026-02-28_20-22-26__4ec97691-e888-4f2a-be87-8e16869aa005__wakuorg_nwaku:latest.log
2026-02-28 20:22:31.298 DEBUG [src.node.waku_node] Starting Node...
2026-02-28 20:22:31.298 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-28 20:22:31.300 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-28 20:22:31.300 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.123.102
2026-02-28 20:22:31.301 DEBUG [src.node.docker_mananger] Generated ports ['60677', '60678', '60679', '60680', '60681']
2026-02-28 20:22:31.301 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-28 20:22:31.301 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-28 20:22:31.301 DEBUG [src.node.waku_node] Using volumes []
2026-02-28 20:22:31.301 DEBUG [src.node.docker_mananger] docker run -i -t -p 60677:60677 -p 60678:60678 -p 60679:60679 -p 60680:60680 -p 60681:60681 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=60679 --rest-port=60677 --tcp-port=60678 --discv5-udp-port=60680 --rest-address=0.0.0.0 --nat=extip:172.18.123.102 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=50ba568a328dd7b136d85c5efd1ee01ea04b090d1c6b72f2dcd363293e5824d7 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=60681 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QFJKV3gCAm5vEw-Xa2ySy52YE8zZU1mhmVUgqN2Srb1td_3lWZw7ABmvXN7fR8tozPtj4wLdy7Vp_nhbtHfKIXsCgmlkgnY0gmlwhKwSK0GKbXVsdGlhZGRyc5YACASsEitBBiivAAoErBIrQQYosN0DgnJzhQADAQAAiXNlY3AyNTZrMaECCUk_xp49KDqD9KjIJoZU8Ia-gQdbgFdHQ99kQflJYZqDdGNwgiivg3VkcIIosYV3YWt1Mgk --lightpushnode=/ip4/172.18.43.65/tcp/10415/p2p/16Uiu2HAkv3vRxjf4LnDhMLrqFoBaQnF4fHtCofWaCPJaqawAaVuj --lightpush=true --relay=true
2026-02-28 20:22:31.501 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.123.102 waku b349f402000a6afe59d0e5d7ebcb2460ceeb4c110692c4af16aae77270abd0b4
2026-02-28 20:22:31.536 DEBUG [src.node.docker_mananger] Container started with ID b349f402000a. Setting up logs at ./log/docker/lightpush_node3_2026-02-28_20-22-26__4ec97691-e888-4f2a-be87-8e16869aa005__wakuorg_nwaku:latest.log
2026-02-28 20:22:31.536 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 60677
2026-02-28 20:22:31.536 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-28 20:22:32.537 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60677/health" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:32.540 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-28 20:22:32.540 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-28 20:22:32.540 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60677/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:32.542 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.123.102/tcp/60678/p2p/16Uiu2HAmUzyyAsFuXmG9xrvPHQvzYwqP5NET4wwzy7JbNfscrD8Y","/ip4/172.18.123.102/tcp/60679/ws/p2p/16Uiu2HAmUzyyAsFuXmG9xrvPHQvzYwqP5NET4wwzy7JbNfscrD8Y"],"enrUri":"enr:-L24QBHs8rYZrU8d3Y3iUuqQBWDIP5UKPsWGr_iL7oMBtKbuLNhsiNh0tHyo8xREEce31Tw9DbDdSiJ1mlUDqF0e8A8CgmlkgnY0gmlwhKwSe2aKbXVsdGlhZGRyc5YACASsEntmBu0GAAoErBJ7ZgbtB90DgnJzhQADAQAAiXNlY3AyNTZrMaED8totAQMe1KpnjYaPKVZJVe9Hu6gONU9tP85hbxqNsX2DdGNwgu0Gg3VkcILtCIV3YWt1Mgk"}'
2026-02-28 20:22:32.543 INFO [src.node.waku_node] REST service is ready !!
2026-02-28 20:22:32.543 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60677/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.43.65/tcp/10415/p2p/16Uiu2HAkv3vRxjf4LnDhMLrqFoBaQnF4fHtCofWaCPJaqawAaVuj"]'
2026-02-28 20:22:32.582 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:32.583 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60677/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.49.192/tcp/36053/p2p/16Uiu2HAmHNTUWuH4gHgLo7mAfn8ZQuYzFqyGzkXgncwFPHVka1UZ"]'
2026-02-28 20:22:32.611 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:32.626 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-28 20:22:32.628 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node4_2026-02-28_20-22-26__4ec97691-e888-4f2a-be87-8e16869aa005__wakuorg_nwaku:latest.log
2026-02-28 20:22:32.628 DEBUG [src.node.waku_node] Starting Node...
2026-02-28 20:22:32.628 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-28 20:22:32.632 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-28 20:22:32.632 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.7.193
2026-02-28 20:22:32.632 DEBUG [src.node.docker_mananger] Generated ports ['15275', '15276', '15277', '15278', '15279']
2026-02-28 20:22:32.632 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-28 20:22:32.633 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-28 20:22:32.633 DEBUG [src.node.waku_node] Using volumes []
2026-02-28 20:22:32.633 DEBUG [src.node.docker_mananger] docker run -i -t -p 15275:15275 -p 15276:15276 -p 15277:15277 -p 15278:15278 -p 15279:15279 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=15277 --rest-port=15275 --tcp-port=15276 --discv5-udp-port=15278 --rest-address=0.0.0.0 --nat=extip:172.18.7.193 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=9eabdacb84ee511c8c344ce729da4bcdefcdedfb87a2f1cb9f39ab4cd635d5b3 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=15279 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QFJKV3gCAm5vEw-Xa2ySy52YE8zZU1mhmVUgqN2Srb1td_3lWZw7ABmvXN7fR8tozPtj4wLdy7Vp_nhbtHfKIXsCgmlkgnY0gmlwhKwSK0GKbXVsdGlhZGRyc5YACASsEitBBiivAAoErBIrQQYosN0DgnJzhQADAQAAiXNlY3AyNTZrMaECCUk_xp49KDqD9KjIJoZU8Ia-gQdbgFdHQ99kQflJYZqDdGNwgiivg3VkcIIosYV3YWt1Mgk --lightpushnode=/ip4/172.18.43.65/tcp/10415/p2p/16Uiu2HAkv3vRxjf4LnDhMLrqFoBaQnF4fHtCofWaCPJaqawAaVuj --lightpush=true --relay=true
2026-02-28 20:22:32.831 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.7.193 waku c29c706502735b86f7be564f2001c9e0b2388dcb2cd9f364f61689ff97822033
2026-02-28 20:22:32.872 DEBUG [src.node.docker_mananger] Container started with ID c29c70650273. Setting up logs at ./log/docker/lightpush_node4_2026-02-28_20-22-26__4ec97691-e888-4f2a-be87-8e16869aa005__wakuorg_nwaku:latest.log
2026-02-28 20:22:32.872 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 15275
2026-02-28 20:22:32.872 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-28 20:22:33.873 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15275/health" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:33.876 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-02-28 20:22:33.876 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-28 20:22:33.876 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15275/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:33.879 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.7.193/tcp/15276/p2p/16Uiu2HAmQBzwz1ZE5yBbUaWEfCrW96vaSAqKSyd93Fx3DwpdtQPq","/ip4/172.18.7.193/tcp/15277/ws/p2p/16Uiu2HAmQBzwz1ZE5yBbUaWEfCrW96vaSAqKSyd93Fx3DwpdtQPq"],"enrUri":"enr:-L24QJC4rqP2t5F1VmZGK-P3V9BYq2-bGmu7pi4lyj-6Ox92UuIOnvD40McBQhdaQ_VwhwEnisJJn7Rj7iF6w9ybC6sCgmlkgnY0gmlwhKwSB8GKbXVsdGlhZGRyc5YACASsEgfBBjusAAoErBIHwQY7rd0DgnJzhQADAQAAiXNlY3AyNTZrMaEDq2I81PGCu78INat1U0DNGRNl0d0BApagkhu5VdUuh4CDdGNwgjusg3VkcII7roV3YWt1Mgk"}'
2026-02-28 20:22:33.879 INFO [src.node.waku_node] REST service is ready !!
2026-02-28 20:22:33.879 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:15275/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.43.65/tcp/10415/p2p/16Uiu2HAkv3vRxjf4LnDhMLrqFoBaQnF4fHtCofWaCPJaqawAaVuj"]'
2026-02-28 20:22:33.922 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:33.924 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:15275/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.49.192/tcp/36053/p2p/16Uiu2HAmHNTUWuH4gHgLo7mAfn8ZQuYzFqyGzkXgncwFPHVka1UZ"]'
2026-02-28 20:22:33.949 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:33.950 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:10414/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-28 20:22:33.957 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:33.957 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:36052/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-28 20:22:33.965 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:33.965 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:15921/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-28 20:22:33.975 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:33.976 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6204/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-28 20:22:33.978 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:33.978 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60677/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-28 20:22:33.981 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:33.981 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:15275/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-02-28 20:22:33.983 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:33.984 DEBUG [src.steps.light_push] Lightpushing message
2026-02-28 20:22:33.984 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:15921/lightpush/v1/message" -H "Content-Type: application/json" -d '{"pubsubTopic": "/waku/2/rs/3/0", "message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1772310153984065801}}'
2026-02-28 20:22:34.002 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:34.002 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-02-28 20:22:34.103 DEBUG [src.steps.light_push] Checking that peer NODE_1:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.103 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10414/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.106 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310153984065801,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.107 DEBUG [src.steps.light_push] Checking that peer NODE_2:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.107 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36052/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.110 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310153984065801,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.111 DEBUG [src.steps.light_push] Checking that peer NODE_3:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.111 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15921/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.114 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310153984065801,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.115 DEBUG [src.steps.light_push] Checking that peer NODE_4:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.116 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6204/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.118 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310153984065801,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.119 DEBUG [src.steps.light_push] Checking that peer NODE_5:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.119 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60677/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.122 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310153984065801,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.123 DEBUG [src.steps.light_push] Checking that peer NODE_6:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.123 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15275/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.125 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310153984065801,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.127 DEBUG [src.steps.light_push] Lightpushing message
2026-02-28 20:22:34.127 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:6204/lightpush/v1/message" -H "Content-Type: application/json" -d '{"pubsubTopic": "/waku/2/rs/3/0", "message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1772310154127510582}}'
2026-02-28 20:22:34.144 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:34.147 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-02-28 20:22:34.250 DEBUG [src.steps.light_push] Checking that peer NODE_1:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.251 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10414/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.253 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154127510582,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.255 DEBUG [src.steps.light_push] Checking that peer NODE_2:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.255 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36052/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.257 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154127510582,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.259 DEBUG [src.steps.light_push] Checking that peer NODE_3:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.259 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15921/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.261 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154127510582,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.263 DEBUG [src.steps.light_push] Checking that peer NODE_4:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.263 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6204/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.265 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154127510582,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.266 DEBUG [src.steps.light_push] Checking that peer NODE_5:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.266 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60677/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.269 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154127510582,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.270 DEBUG [src.steps.light_push] Checking that peer NODE_6:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.270 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15275/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.272 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154127510582,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.274 DEBUG [src.steps.light_push] Lightpushing message
2026-02-28 20:22:34.274 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:60677/lightpush/v1/message" -H "Content-Type: application/json" -d '{"pubsubTopic": "/waku/2/rs/3/0", "message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1772310154274443858}}'
2026-02-28 20:22:34.294 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:34.296 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-02-28 20:22:34.396 DEBUG [src.steps.light_push] Checking that peer NODE_1:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.397 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10414/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.399 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154274443858,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.401 DEBUG [src.steps.light_push] Checking that peer NODE_2:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.401 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36052/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.403 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154274443858,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.405 DEBUG [src.steps.light_push] Checking that peer NODE_3:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.405 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15921/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.407 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154274443858,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.408 DEBUG [src.steps.light_push] Checking that peer NODE_4:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.409 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6204/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.411 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154274443858,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.412 DEBUG [src.steps.light_push] Checking that peer NODE_5:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.412 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60677/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.415 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154274443858,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.416 DEBUG [src.steps.light_push] Checking that peer NODE_6:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.416 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15275/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.418 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154274443858,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.420 DEBUG [src.steps.light_push] Lightpushing message
2026-02-28 20:22:34.420 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:15275/lightpush/v1/message" -H "Content-Type: application/json" -d '{"pubsubTopic": "/waku/2/rs/3/0", "message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1772310154420558635}}'
2026-02-28 20:22:34.436 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-28 20:22:34.440 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-02-28 20:22:34.543 DEBUG [src.steps.light_push] Checking that peer NODE_1:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.543 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:10414/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.546 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154420558635,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.548 DEBUG [src.steps.light_push] Checking that peer NODE_2:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.548 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36052/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.550 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154420558635,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.551 DEBUG [src.steps.light_push] Checking that peer NODE_3:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.551 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15921/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.554 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154420558635,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.555 DEBUG [src.steps.light_push] Checking that peer NODE_4:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.555 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:6204/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.557 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154420558635,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.559 DEBUG [src.steps.light_push] Checking that peer NODE_5:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.559 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:60677/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.561 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154420558635,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.563 DEBUG [src.steps.light_push] Checking that peer NODE_6:wakuorg/nwaku:latest can find the lightpushed message
2026-02-28 20:22:34.563 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:15275/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-02-28 20:22:34.565 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772310154420558635,"ephemeral":false,"proof":""}]'
2026-02-28 20:22:34.568 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-02-28 20:22:34.569 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-02-28 20:22:34.569 DEBUG [src.node.waku_node] Stopping container with id ddd672ec66be
2026-02-28 20:22:35.253 DEBUG [src.node.waku_node] Container stopped.
2026-02-28 20:22:35.253 DEBUG [src.node.waku_node] Stopping container with id 0483be9e08cf
2026-02-28 20:22:35.874 DEBUG [src.node.waku_node] Container stopped.
2026-02-28 20:22:35.876 DEBUG [src.node.waku_node] Stopping container with id 51a87fa19135
2026-02-28 20:22:36.167 ERROR [src.node.docker_mananger] Max retries reached for container ddd672ec66be. Exiting log stream.
2026-02-28 20:22:36.511 DEBUG [src.node.waku_node] Container stopped.
2026-02-28 20:22:36.511 DEBUG [src.node.waku_node] Stopping container with id 49fbfc951b0c
2026-02-28 20:22:36.686 ERROR [src.node.docker_mananger] Max retries reached for container 0483be9e08cf. Exiting log stream.
2026-02-28 20:22:37.110 DEBUG [src.node.waku_node] Container stopped.
2026-02-28 20:22:37.112 DEBUG [src.node.waku_node] Stopping container with id b349f402000a
2026-02-28 20:22:37.371 ERROR [src.node.docker_mananger] Max retries reached for container 51a87fa19135. Exiting log stream.
2026-02-28 20:22:37.642 DEBUG [src.node.waku_node] Container stopped.
2026-02-28 20:22:37.642 DEBUG [src.node.waku_node] Stopping container with id c29c70650273
2026-02-28 20:22:37.924 ERROR [src.node.docker_mananger] Max retries reached for container 49fbfc951b0c. Exiting log stream.
2026-02-28 20:22:38.238 DEBUG [src.node.waku_node] Container stopped.
2026-02-28 20:22:38.242 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-02-28 20:22:38.282 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-28 20:22:38.304 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-28 20:22:38.324 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-28 20:22:38.334 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-28 20:22:38.343 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-28 20:22:38.352 DEBUG [src.node.docker_mananger] No errors found in the waku logs.