276 lines
52 KiB
Plaintext

2026-03-03 04:34:08.890 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-03 04:34:08.891 DEBUG [tests.conftest] Running test: test_multiple_lightpush_nodes with id: 2026-03-03_04-34-08__122f4e67-8808-42ef-a998-fc9f785c29d3
2026-03-03 04:34:08.891 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-03 04:34:08.891 DEBUG [src.steps.light_push] Running fixture setup: light_push_setup
2026-03-03 04:34:08.900 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:34:08.901 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node1_2026-03-03_04-34-08__122f4e67-8808-42ef-a998-fc9f785c29d3__wakuorg_nwaku:latest.log
2026-03-03 04:34:08.901 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:34:08.901 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:34:08.902 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:34:08.903 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.12.187
2026-03-03 04:34:08.903 DEBUG [src.node.docker_mananger] Generated ports ['2115', '2116', '2117', '2118', '2119']
2026-03-03 04:34:08.903 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:34:08.903 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:34:08.903 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:34:08.903 DEBUG [src.node.docker_mananger] docker run -i -t -p 2115:2115 -p 2116:2116 -p 2117:2117 -p 2118:2118 -p 2119:2119 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=2117 --rest-port=2115 --tcp-port=2116 --discv5-udp-port=2118 --rest-address=0.0.0.0 --nat=extip:172.18.12.187 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=6eafbc44adfbba0cb1d9adf541fc6dedaeebfc20946ae5b1ad3f0e3adb48844a --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=2119 --metrics-logging=true --lightpush=true --relay=true
2026-03-03 04:34:09.114 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.12.187 waku d8f08dcdd2bf8df396c9a5c37cefa4cbb70be5a9227faaf56b9bacbd3981f83f
2026-03-03 04:34:09.136 ERROR [src.node.docker_mananger] Max retries reached for container 823001493f01. Exiting log stream.
2026-03-03 04:34:09.151 DEBUG [src.node.docker_mananger] Container started with ID d8f08dcdd2bf. Setting up logs at ./log/docker/receiving_node1_2026-03-03_04-34-08__122f4e67-8808-42ef-a998-fc9f785c29d3__wakuorg_nwaku:latest.log
2026-03-03 04:34:09.151 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 2115
2026-03-03 04:34:09.152 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:34:09.692 ERROR [src.node.docker_mananger] Max retries reached for container c669e100589a. Exiting log stream.
2026-03-03 04:34:10.153 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2115/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:10.156 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:34:10.157 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:34:10.157 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2115/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:10.159 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.12.187/tcp/2116/p2p/16Uiu2HAm8CdR6vMF9ZnmqaThHmtBTtsn3NMhCXn1V9BrwudAxKSH","/ip4/172.18.12.187/tcp/2117/ws/p2p/16Uiu2HAm8CdR6vMF9ZnmqaThHmtBTtsn3NMhCXn1V9BrwudAxKSH"],"enrUri":"enr:-L24QPY6gnNvzVet58J03RJVF7sixeK2dQ_qpulv2gdiJq5RCO0XM3MxquCTOXT_8LMARfu852uTzSspBZxi-xcUENYCgmlkgnY0gmlwhKwSDLuKbXVsdGlhZGRyc5YACASsEgy7BghEAAoErBIMuwYIRd0DgnJzhQADAQAAiXNlY3AyNTZrMaECvdB7FtPiT7gbZeBNdt6OGkJO9KG98xY4xatfYNKBQeqDdGNwgghEg3VkcIIIRoV3YWt1Mgk"}'
2026-03-03 04:34:10.159 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:34:10.167 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:34:10.167 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/receiving_node2_2026-03-03_04-34-08__122f4e67-8808-42ef-a998-fc9f785c29d3__wakuorg_nwaku:latest.log
2026-03-03 04:34:10.167 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:34:10.167 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:34:10.169 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:34:10.169 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.210.138
2026-03-03 04:34:10.169 DEBUG [src.node.docker_mananger] Generated ports ['63833', '63834', '63835', '63836', '63837']
2026-03-03 04:34:10.169 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:34:10.169 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:34:10.169 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:34:10.170 DEBUG [src.node.docker_mananger] docker run -i -t -p 63833:63833 -p 63834:63834 -p 63835:63835 -p 63836:63836 -p 63837:63837 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=63835 --rest-port=63833 --tcp-port=63834 --discv5-udp-port=63836 --rest-address=0.0.0.0 --nat=extip:172.18.210.138 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=4b1ff5acc9cd7c32524da8330efb65a91d50b8ac93d2dbaab49db49e2b64bcf1 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=63837 --metrics-logging=true --lightpush=false --relay=true
2026-03-03 04:34:10.371 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.210.138 waku ca473dc8a9ce61f5bf65770ffc4ed156b4d5aaa319c75c0d09c0e932b293106c
2026-03-03 04:34:10.406 DEBUG [src.node.docker_mananger] Container started with ID ca473dc8a9ce. Setting up logs at ./log/docker/receiving_node2_2026-03-03_04-34-08__122f4e67-8808-42ef-a998-fc9f785c29d3__wakuorg_nwaku:latest.log
2026-03-03 04:34:10.407 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 63833
2026-03-03 04:34:10.407 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:34:11.407 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63833/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:11.410 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:34:11.410 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:34:11.410 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63833/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:11.413 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.210.138/tcp/63834/p2p/16Uiu2HAmCzknth5yxpFNMaqttFJa74vpNdJiHXU1oBkfRBjYJYG3","/ip4/172.18.210.138/tcp/63835/ws/p2p/16Uiu2HAmCzknth5yxpFNMaqttFJa74vpNdJiHXU1oBkfRBjYJYG3"],"enrUri":"enr:-L24QELdldRAJAC5q7yZTAL8vTDMXOgatFSCF89oK_MuBCuHAe49JlntP9CwMABadVh2jkl-dEfyNQOriYXfwW5gjFYCgmlkgnY0gmlwhKwS0oqKbXVsdGlhZGRyc5YACASsEtKKBvlaAAoErBLSigb5W90DgnJzhQADAQAAiXNlY3AyNTZrMaEDBRBJEsG1MxIYCgrvovFiM6B5gqMPfCgxjleZ4BeJzxyDdGNwgvlag3VkcIL5XIV3YWt1MgE"}'
2026-03-03 04:34:11.413 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:34:11.414 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63833/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.12.187/tcp/2116/p2p/16Uiu2HAm8CdR6vMF9ZnmqaThHmtBTtsn3NMhCXn1V9BrwudAxKSH"]'
2026-03-03 04:34:11.451 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:11.461 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:34:11.461 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node1_2026-03-03_04-34-08__122f4e67-8808-42ef-a998-fc9f785c29d3__wakuorg_nwaku:latest.log
2026-03-03 04:34:11.461 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:34:11.461 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:34:11.463 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:34:11.463 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.252.8
2026-03-03 04:34:11.463 DEBUG [src.node.docker_mananger] Generated ports ['24696', '24697', '24698', '24699', '24700']
2026-03-03 04:34:11.463 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:34:11.463 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:34:11.464 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:34:11.464 DEBUG [src.node.docker_mananger] docker run -i -t -p 24696:24696 -p 24697:24697 -p 24698:24698 -p 24699:24699 -p 24700:24700 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=24698 --rest-port=24696 --tcp-port=24697 --discv5-udp-port=24699 --rest-address=0.0.0.0 --nat=extip:172.18.252.8 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=cfe58864ab8ef0eeee692b78adbcad85bfc4576431cfcef2e28ea6ad9aa1f32d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=24700 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QPY6gnNvzVet58J03RJVF7sixeK2dQ_qpulv2gdiJq5RCO0XM3MxquCTOXT_8LMARfu852uTzSspBZxi-xcUENYCgmlkgnY0gmlwhKwSDLuKbXVsdGlhZGRyc5YACASsEgy7BghEAAoErBIMuwYIRd0DgnJzhQADAQAAiXNlY3AyNTZrMaECvdB7FtPiT7gbZeBNdt6OGkJO9KG98xY4xatfYNKBQeqDdGNwgghEg3VkcIIIRoV3YWt1Mgk --lightpushnode=/ip4/172.18.12.187/tcp/2116/p2p/16Uiu2HAm8CdR6vMF9ZnmqaThHmtBTtsn3NMhCXn1V9BrwudAxKSH --lightpush=true --relay=true
2026-03-03 04:34:11.662 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.252.8 waku eb6a1900f6db3f0e7b1dcb12383ac2eef65313707c5e7108538d7c5bdf0e0921
2026-03-03 04:34:11.699 DEBUG [src.node.docker_mananger] Container started with ID eb6a1900f6db. Setting up logs at ./log/docker/lightpush_node1_2026-03-03_04-34-08__122f4e67-8808-42ef-a998-fc9f785c29d3__wakuorg_nwaku:latest.log
2026-03-03 04:34:11.699 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 24696
2026-03-03 04:34:11.700 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:34:12.701 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24696/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:12.714 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:34:12.716 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:34:12.717 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24696/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:12.732 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.252.8/tcp/24697/p2p/16Uiu2HAmAE3hD7Ch2Dy4UgDfbYY8UgQNXVoLMDFtpKLZ5qdJ55ax","/ip4/172.18.252.8/tcp/24698/ws/p2p/16Uiu2HAmAE3hD7Ch2Dy4UgDfbYY8UgQNXVoLMDFtpKLZ5qdJ55ax"],"enrUri":"enr:-L24QH_PT9fkEBG2Jta-xx_BY-QvFfz8-VjnR9YhWqp0S85seH2WQAkJiV93t_lo9JCzFsEIhltu9_L-gn6u6Ldt9RMCgmlkgnY0gmlwhKwS_AiKbXVsdGlhZGRyc5YACASsEvwIBmB5AAoErBL8CAZget0DgnJzhQADAQAAiXNlY3AyNTZrMaEC2-Ti6LL5XFKTkK_OHatzxm9L5tuGvgRwtHowMpIfAXGDdGNwgmB5g3VkcIJge4V3YWt1Mgk"}'
2026-03-03 04:34:12.734 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:34:12.736 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:24696/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.12.187/tcp/2116/p2p/16Uiu2HAm8CdR6vMF9ZnmqaThHmtBTtsn3NMhCXn1V9BrwudAxKSH"]'
2026-03-03 04:34:12.744 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:12.744 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:24696/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.210.138/tcp/63834/p2p/16Uiu2HAmCzknth5yxpFNMaqttFJa74vpNdJiHXU1oBkfRBjYJYG3"]'
2026-03-03 04:34:12.773 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:12.784 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:34:12.784 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node2_2026-03-03_04-34-08__122f4e67-8808-42ef-a998-fc9f785c29d3__wakuorg_nwaku:latest.log
2026-03-03 04:34:12.784 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:34:12.784 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:34:12.786 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:34:12.786 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.199.238
2026-03-03 04:34:12.786 DEBUG [src.node.docker_mananger] Generated ports ['50051', '50052', '50053', '50054', '50055']
2026-03-03 04:34:12.786 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:34:12.786 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:34:12.787 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:34:12.787 DEBUG [src.node.docker_mananger] docker run -i -t -p 50051:50051 -p 50052:50052 -p 50053:50053 -p 50054:50054 -p 50055:50055 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=50053 --rest-port=50051 --tcp-port=50052 --discv5-udp-port=50054 --rest-address=0.0.0.0 --nat=extip:172.18.199.238 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=caaceb195caaadbc6de1a07dee7df5dcaabcec28e67b801e4faf956a41d8be0d --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=50055 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QPY6gnNvzVet58J03RJVF7sixeK2dQ_qpulv2gdiJq5RCO0XM3MxquCTOXT_8LMARfu852uTzSspBZxi-xcUENYCgmlkgnY0gmlwhKwSDLuKbXVsdGlhZGRyc5YACASsEgy7BghEAAoErBIMuwYIRd0DgnJzhQADAQAAiXNlY3AyNTZrMaECvdB7FtPiT7gbZeBNdt6OGkJO9KG98xY4xatfYNKBQeqDdGNwgghEg3VkcIIIRoV3YWt1Mgk --lightpushnode=/ip4/172.18.12.187/tcp/2116/p2p/16Uiu2HAm8CdR6vMF9ZnmqaThHmtBTtsn3NMhCXn1V9BrwudAxKSH --lightpush=true --relay=true
2026-03-03 04:34:12.997 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.199.238 waku effd6f27ae985bf38ad21ab327333c4bb05bbd1aabd5073048bd4ac127653bbc
2026-03-03 04:34:13.030 DEBUG [src.node.docker_mananger] Container started with ID effd6f27ae98. Setting up logs at ./log/docker/lightpush_node2_2026-03-03_04-34-08__122f4e67-8808-42ef-a998-fc9f785c29d3__wakuorg_nwaku:latest.log
2026-03-03 04:34:13.030 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 50051
2026-03-03 04:34:13.031 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:34:14.031 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50051/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:14.034 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:34:14.034 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:34:14.034 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50051/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:14.037 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.199.238/tcp/50052/p2p/16Uiu2HAm2Fj585gNq7nfhsoLjFeihFPt8Zi5y3Z3ZU2vKgGyx8dd","/ip4/172.18.199.238/tcp/50053/ws/p2p/16Uiu2HAm2Fj585gNq7nfhsoLjFeihFPt8Zi5y3Z3ZU2vKgGyx8dd"],"enrUri":"enr:-L24QB5wOZDRwBs6dXRPXhz0SVnOem14_SV6HnRUHfPPcs7qEDlyOPG4S-38g_8FzSBUEt-_v67gzla5egbcmHIa09ICgmlkgnY0gmlwhKwSx-6KbXVsdGlhZGRyc5YACASsEsfuBsOEAAoErBLH7gbDhd0DgnJzhQADAQAAiXNlY3AyNTZrMaECZXV_FBVx89qdbabEPU1tmBZhK6tVC3gL4rKMO1M9roCDdGNwgsOEg3VkcILDhoV3YWt1Mgk"}'
2026-03-03 04:34:14.037 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:34:14.037 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:50051/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.12.187/tcp/2116/p2p/16Uiu2HAm8CdR6vMF9ZnmqaThHmtBTtsn3NMhCXn1V9BrwudAxKSH"]'
2026-03-03 04:34:14.078 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:14.083 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:50051/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.210.138/tcp/63834/p2p/16Uiu2HAmCzknth5yxpFNMaqttFJa74vpNdJiHXU1oBkfRBjYJYG3"]'
2026-03-03 04:34:14.114 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:14.124 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:34:14.124 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node3_2026-03-03_04-34-08__122f4e67-8808-42ef-a998-fc9f785c29d3__wakuorg_nwaku:latest.log
2026-03-03 04:34:14.124 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:34:14.124 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:34:14.126 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:34:14.126 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.74.174
2026-03-03 04:34:14.127 DEBUG [src.node.docker_mananger] Generated ports ['32719', '32720', '32721', '32722', '32723']
2026-03-03 04:34:14.127 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:34:14.127 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:34:14.127 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:34:14.128 DEBUG [src.node.docker_mananger] docker run -i -t -p 32719:32719 -p 32720:32720 -p 32721:32721 -p 32722:32722 -p 32723:32723 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=32721 --rest-port=32719 --tcp-port=32720 --discv5-udp-port=32722 --rest-address=0.0.0.0 --nat=extip:172.18.74.174 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=1f78453ee0b7075d41bfde245ddad1b38ca31b4a2debf7be7f1c6ebcee5a706b --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=32723 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QPY6gnNvzVet58J03RJVF7sixeK2dQ_qpulv2gdiJq5RCO0XM3MxquCTOXT_8LMARfu852uTzSspBZxi-xcUENYCgmlkgnY0gmlwhKwSDLuKbXVsdGlhZGRyc5YACASsEgy7BghEAAoErBIMuwYIRd0DgnJzhQADAQAAiXNlY3AyNTZrMaECvdB7FtPiT7gbZeBNdt6OGkJO9KG98xY4xatfYNKBQeqDdGNwgghEg3VkcIIIRoV3YWt1Mgk --lightpushnode=/ip4/172.18.12.187/tcp/2116/p2p/16Uiu2HAm8CdR6vMF9ZnmqaThHmtBTtsn3NMhCXn1V9BrwudAxKSH --lightpush=true --relay=true
2026-03-03 04:34:14.327 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.74.174 waku eb3b3392d86467665b43f2cf9e97f02aad8a84b05c99e1e89f621a31bce5f9c1
2026-03-03 04:34:14.364 DEBUG [src.node.docker_mananger] Container started with ID eb3b3392d864. Setting up logs at ./log/docker/lightpush_node3_2026-03-03_04-34-08__122f4e67-8808-42ef-a998-fc9f785c29d3__wakuorg_nwaku:latest.log
2026-03-03 04:34:14.365 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 32719
2026-03-03 04:34:14.365 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:34:15.365 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32719/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:15.368 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:34:15.368 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:34:15.368 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32719/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:15.371 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.74.174/tcp/32720/p2p/16Uiu2HAm4xfLPWx6SojYHzFiwpLhYboRXM2ignMw6gd4LNu9c6Ec","/ip4/172.18.74.174/tcp/32721/ws/p2p/16Uiu2HAm4xfLPWx6SojYHzFiwpLhYboRXM2ignMw6gd4LNu9c6Ec"],"enrUri":"enr:-L24QCziQQDB9-k_WBusPGB26EDR7UdakYyzfazE1MVs33n4P8YZBgT262sk7fNmLrJyZ_2cmFxSXVMxT-cBulpW2yQCgmlkgnY0gmlwhKwSSq6KbXVsdGlhZGRyc5YACASsEkquBn_QAAoErBJKrgZ_0d0DgnJzhQADAQAAiXNlY3AyNTZrMaECjal1Tgvg-5bK5eNycHfDpdYn8CTt4h0ccS7L035g_wGDdGNwgn_Qg3VkcIJ_0oV3YWt1Mgk"}'
2026-03-03 04:34:15.371 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:34:15.372 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32719/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.12.187/tcp/2116/p2p/16Uiu2HAm8CdR6vMF9ZnmqaThHmtBTtsn3NMhCXn1V9BrwudAxKSH"]'
2026-03-03 04:34:15.409 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:15.410 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32719/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.210.138/tcp/63834/p2p/16Uiu2HAmCzknth5yxpFNMaqttFJa74vpNdJiHXU1oBkfRBjYJYG3"]'
2026-03-03 04:34:15.440 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:15.453 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-03 04:34:15.455 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/lightpush_node4_2026-03-03_04-34-08__122f4e67-8808-42ef-a998-fc9f785c29d3__wakuorg_nwaku:latest.log
2026-03-03 04:34:15.455 DEBUG [src.node.waku_node] Starting Node...
2026-03-03 04:34:15.456 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-03 04:34:15.459 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-03 04:34:15.459 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.148.35
2026-03-03 04:34:15.460 DEBUG [src.node.docker_mananger] Generated ports ['38081', '38082', '38083', '38084', '38085']
2026-03-03 04:34:15.460 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-03 04:34:15.460 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-03 04:34:15.462 DEBUG [src.node.waku_node] Using volumes []
2026-03-03 04:34:15.463 DEBUG [src.node.docker_mananger] docker run -i -t -p 38081:38081 -p 38082:38082 -p 38083:38083 -p 38084:38084 -p 38085:38085 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=38083 --rest-port=38081 --tcp-port=38082 --discv5-udp-port=38084 --rest-address=0.0.0.0 --nat=extip:172.18.148.35 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=d918a2d8d7498ffe093fdf8c93f39d9c58aa588e22d8fb3fcae43f1eeac14eb2 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=38085 --metrics-logging=true --discv5-bootstrap-node=enr:-L24QPY6gnNvzVet58J03RJVF7sixeK2dQ_qpulv2gdiJq5RCO0XM3MxquCTOXT_8LMARfu852uTzSspBZxi-xcUENYCgmlkgnY0gmlwhKwSDLuKbXVsdGlhZGRyc5YACASsEgy7BghEAAoErBIMuwYIRd0DgnJzhQADAQAAiXNlY3AyNTZrMaECvdB7FtPiT7gbZeBNdt6OGkJO9KG98xY4xatfYNKBQeqDdGNwgghEg3VkcIIIRoV3YWt1Mgk --lightpushnode=/ip4/172.18.12.187/tcp/2116/p2p/16Uiu2HAm8CdR6vMF9ZnmqaThHmtBTtsn3NMhCXn1V9BrwudAxKSH --lightpush=true --relay=true
2026-03-03 04:34:15.669 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.148.35 waku aaef8453ace82fdf33f4530f4f12307484130d10a42a35cc7215fd4ace90ceff
2026-03-03 04:34:15.708 DEBUG [src.node.docker_mananger] Container started with ID aaef8453ace8. Setting up logs at ./log/docker/lightpush_node4_2026-03-03_04-34-08__122f4e67-8808-42ef-a998-fc9f785c29d3__wakuorg_nwaku:latest.log
2026-03-03 04:34:15.709 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 38081
2026-03-03 04:34:15.709 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-03 04:34:16.710 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38081/health" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:16.712 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Legacy Lightpush":"NOT_READY","desc":"Node has no relay peers to fullfill push requests"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-03 04:34:16.713 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-03 04:34:16.713 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38081/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:16.715 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.148.35/tcp/38082/p2p/16Uiu2HAmHYm8y2HmQg1S86eeNtaTQBDX6Z4VHzMva6X8Cv4CWQ1j","/ip4/172.18.148.35/tcp/38083/ws/p2p/16Uiu2HAmHYm8y2HmQg1S86eeNtaTQBDX6Z4VHzMva6X8Cv4CWQ1j"],"enrUri":"enr:-L24QBf78SDSvw5PwMopX9GdvOj5KXvNPPQ77NPH4pT0Yd5TBMlUI8TXhiHVBWY6dhFfv8IELBKdx7jVDvY-EVttCYECgmlkgnY0gmlwhKwSlCOKbXVsdGlhZGRyc5YACASsEpQjBpTCAAoErBKUIwaUw90DgnJzhQADAQAAiXNlY3AyNTZrMaEDSLICqQnPPPH5rCIp9zfgtXUo5qwfCojWazTT5kZB8z6DdGNwgpTCg3VkcIKUxIV3YWt1Mgk"}'
2026-03-03 04:34:16.715 INFO [src.node.waku_node] REST service is ready !!
2026-03-03 04:34:16.716 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38081/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.12.187/tcp/2116/p2p/16Uiu2HAm8CdR6vMF9ZnmqaThHmtBTtsn3NMhCXn1V9BrwudAxKSH"]'
2026-03-03 04:34:16.757 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:16.757 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38081/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.210.138/tcp/63834/p2p/16Uiu2HAmCzknth5yxpFNMaqttFJa74vpNdJiHXU1oBkfRBjYJYG3"]'
2026-03-03 04:34:16.785 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:16.786 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:2115/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-03 04:34:16.792 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:16.793 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63833/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-03 04:34:16.800 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:16.800 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:24696/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-03 04:34:16.806 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:16.807 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:50051/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-03 04:34:16.810 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:16.811 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32719/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-03 04:34:16.813 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:16.813 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38081/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/0"]'
2026-03-03 04:34:16.816 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:16.816 DEBUG [src.steps.light_push] Lightpushing message
2026-03-03 04:34:16.816 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:24696/lightpush/v1/message" -H "Content-Type: application/json" -d '{"pubsubTopic": "/waku/2/rs/3/0", "message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1772512456816700859}}'
2026-03-03 04:34:16.835 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:16.837 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-03 04:34:16.941 DEBUG [src.steps.light_push] Checking that peer NODE_1:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:16.942 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2115/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:16.944 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512456816700859,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:16.945 DEBUG [src.steps.light_push] Checking that peer NODE_2:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:16.946 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63833/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:16.948 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512456816700859,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:16.949 DEBUG [src.steps.light_push] Checking that peer NODE_3:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:16.949 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24696/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:16.951 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512456816700859,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:16.953 DEBUG [src.steps.light_push] Checking that peer NODE_4:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:16.953 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50051/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:16.955 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512456816700859,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:16.957 DEBUG [src.steps.light_push] Checking that peer NODE_5:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:16.957 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32719/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:16.959 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512456816700859,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:16.960 DEBUG [src.steps.light_push] Checking that peer NODE_6:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:16.960 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38081/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:16.963 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512456816700859,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:16.964 DEBUG [src.steps.light_push] Lightpushing message
2026-03-03 04:34:16.965 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:50051/lightpush/v1/message" -H "Content-Type: application/json" -d '{"pubsubTopic": "/waku/2/rs/3/0", "message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1772512456964863718}}'
2026-03-03 04:34:16.987 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:16.987 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-03 04:34:17.088 DEBUG [src.steps.light_push] Checking that peer NODE_1:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.088 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2115/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.091 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512456964863718,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.092 DEBUG [src.steps.light_push] Checking that peer NODE_2:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.092 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63833/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.095 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512456964863718,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.096 DEBUG [src.steps.light_push] Checking that peer NODE_3:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.096 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24696/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.098 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512456964863718,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.100 DEBUG [src.steps.light_push] Checking that peer NODE_4:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.100 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50051/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.102 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512456964863718,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.104 DEBUG [src.steps.light_push] Checking that peer NODE_5:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.104 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32719/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.106 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512456964863718,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.107 DEBUG [src.steps.light_push] Checking that peer NODE_6:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.107 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38081/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.109 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512456964863718,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.111 DEBUG [src.steps.light_push] Lightpushing message
2026-03-03 04:34:17.111 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:32719/lightpush/v1/message" -H "Content-Type: application/json" -d '{"pubsubTopic": "/waku/2/rs/3/0", "message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1772512457111391871}}'
2026-03-03 04:34:17.126 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:17.128 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-03 04:34:17.229 DEBUG [src.steps.light_push] Checking that peer NODE_1:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.230 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2115/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.233 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512457111391871,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.234 DEBUG [src.steps.light_push] Checking that peer NODE_2:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.234 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63833/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.237 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512457111391871,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.238 DEBUG [src.steps.light_push] Checking that peer NODE_3:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.238 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24696/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.240 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512457111391871,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.242 DEBUG [src.steps.light_push] Checking that peer NODE_4:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.242 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50051/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.244 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512457111391871,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.246 DEBUG [src.steps.light_push] Checking that peer NODE_5:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.246 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32719/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.248 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512457111391871,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.249 DEBUG [src.steps.light_push] Checking that peer NODE_6:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.249 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38081/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.252 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512457111391871,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.254 DEBUG [src.steps.light_push] Lightpushing message
2026-03-03 04:34:17.254 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38081/lightpush/v1/message" -H "Content-Type: application/json" -d '{"pubsubTopic": "/waku/2/rs/3/0", "message": {"payload": "TGlnaHQgcHVzaCB3b3JrcyEh", "contentTopic": "/myapp/1/latest/proto", "timestamp": 1772512457254177924}}'
2026-03-03 04:34:17.276 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-03 04:34:17.277 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-03-03 04:34:17.377 DEBUG [src.steps.light_push] Checking that peer NODE_1:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.377 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2115/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.380 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512457254177924,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.381 DEBUG [src.steps.light_push] Checking that peer NODE_2:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.382 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63833/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.384 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512457254177924,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.385 DEBUG [src.steps.light_push] Checking that peer NODE_3:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.386 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24696/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.388 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512457254177924,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.389 DEBUG [src.steps.light_push] Checking that peer NODE_4:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.389 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50051/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.392 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512457254177924,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.393 DEBUG [src.steps.light_push] Checking that peer NODE_5:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.393 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:32719/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.396 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512457254177924,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.397 DEBUG [src.steps.light_push] Checking that peer NODE_6:wakuorg/nwaku:latest can find the lightpushed message
2026-03-03 04:34:17.397 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38081/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F0" -H "Content-Type: application/json" -d 'None'
2026-03-03 04:34:17.400 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"TGlnaHQgcHVzaCB3b3JrcyEh","contentTopic":"/myapp/1/latest/proto","version":0,"timestamp":1772512457254177924,"ephemeral":false,"proof":""}]'
2026-03-03 04:34:17.402 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-03 04:34:17.403 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-03 04:34:17.403 DEBUG [src.node.waku_node] Stopping container with id d8f08dcdd2bf
2026-03-03 04:34:18.061 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:34:18.062 DEBUG [src.node.waku_node] Stopping container with id ca473dc8a9ce
2026-03-03 04:34:18.671 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:34:18.671 DEBUG [src.node.waku_node] Stopping container with id eb6a1900f6db
2026-03-03 04:34:18.980 ERROR [src.node.docker_mananger] Max retries reached for container d8f08dcdd2bf. Exiting log stream.
2026-03-03 04:34:19.261 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:34:19.263 DEBUG [src.node.waku_node] Stopping container with id effd6f27ae98
2026-03-03 04:34:19.580 ERROR [src.node.docker_mananger] Max retries reached for container ca473dc8a9ce. Exiting log stream.
2026-03-03 04:34:19.829 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:34:19.830 DEBUG [src.node.waku_node] Stopping container with id eb3b3392d864
2026-03-03 04:34:20.075 ERROR [src.node.docker_mananger] Max retries reached for container eb6a1900f6db. Exiting log stream.
2026-03-03 04:34:20.402 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:34:20.403 DEBUG [src.node.waku_node] Stopping container with id aaef8453ace8
2026-03-03 04:34:20.647 ERROR [src.node.docker_mananger] Max retries reached for container effd6f27ae98. Exiting log stream.
2026-03-03 04:34:20.967 DEBUG [src.node.waku_node] Container stopped.
2026-03-03 04:34:20.969 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-03 04:34:21.009 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-03 04:34:21.048 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-03 04:34:21.059 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-03 04:34:21.069 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-03 04:34:21.078 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-03 04:34:21.087 DEBUG [src.node.docker_mananger] No errors found in the waku logs.