162 lines
30 KiB
Plaintext

2026-03-13 04:32:37.781 DEBUG [tests.conftest] Running fixture setup: test_id
2026-03-13 04:32:37.782 DEBUG [tests.conftest] Running test: test_store_no_peer_selected with id: 2026-03-13_04-32-37__beea50b0-8cb8-4f51-bdc9-8521835a9347
2026-03-13 04:32:37.782 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-03-13 04:32:37.782 DEBUG [src.steps.filter] Running fixture setup: filter_setup
2026-03-13 04:32:37.783 DEBUG [src.steps.light_push] Running fixture setup: light_push_setup
2026-03-13 04:32:37.792 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-13 04:32:37.793 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-03-13_04-32-37__beea50b0-8cb8-4f51-bdc9-8521835a9347__wakuorg_nwaku:latest.log
2026-03-13 04:32:37.799 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-13 04:32:37.799 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-03-13_04-32-37__beea50b0-8cb8-4f51-bdc9-8521835a9347__wakuorg_nwaku:latest.log
2026-03-13 04:32:37.806 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-13 04:32:37.806 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-13_04-32-37__beea50b0-8cb8-4f51-bdc9-8521835a9347__wakuorg_nwaku:latest.log
2026-03-13 04:32:37.807 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-03-13 04:32:37.807 DEBUG [src.steps.store] Running fixture setup: store_setup
2026-03-13 04:32:37.808 DEBUG [tests.e2e.test_e2e] Start 5 nodes
2026-03-13 04:32:37.814 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-13 04:32:37.814 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-03-13_04-32-37__beea50b0-8cb8-4f51-bdc9-8521835a9347__wakuorg_nwaku:latest.log
2026-03-13 04:32:37.820 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-13 04:32:37.820 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-03-13_04-32-37__beea50b0-8cb8-4f51-bdc9-8521835a9347__wakuorg_nwaku:latest.log
2026-03-13 04:32:37.827 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-03-13 04:32:37.827 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-03-13_04-32-37__beea50b0-8cb8-4f51-bdc9-8521835a9347__wakuorg_nwaku:latest.log
2026-03-13 04:32:37.827 DEBUG [src.node.waku_node] Starting Node...
2026-03-13 04:32:37.827 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-13 04:32:37.873 DEBUG [src.node.docker_mananger] Network waku created
2026-03-13 04:32:37.873 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.163.12
2026-03-13 04:32:37.873 DEBUG [src.node.docker_mananger] Generated ports ['63173', '63174', '63175', '63176', '63177']
2026-03-13 04:32:37.874 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-13 04:32:37.874 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-13 04:32:37.874 DEBUG [src.node.waku_node] Using volumes []
2026-03-13 04:32:37.874 DEBUG [src.node.docker_mananger] docker run -i -t -p 63173:63173 -p 63174:63174 -p 63175:63175 -p 63176:63176 -p 63177:63177 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=63175 --rest-port=63173 --tcp-port=63174 --discv5-udp-port=63176 --rest-address=0.0.0.0 --nat=extip:172.18.163.12 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=dbe9c45bc57bf52cadea325cbb815cd70eb4faba30eecea5ae12fba4a6baaab4 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=63177 --metrics-logging=true --relay=true --store=true
2026-03-13 04:32:40.675 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.163.12 waku e4258df5a90d2da235e88902b72b1f6ce3a69802e84e849ef55cb945a02b8d92
2026-03-13 04:32:40.712 DEBUG [src.node.docker_mananger] Container started with ID e4258df5a90d. Setting up logs at ./log/docker/node1_2026-03-13_04-32-37__beea50b0-8cb8-4f51-bdc9-8521835a9347__wakuorg_nwaku:latest.log
2026-03-13 04:32:40.713 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 63173
2026-03-13 04:32:40.714 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-13 04:32:41.715 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63173/health" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:32:41.719 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-13 04:32:41.719 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-13 04:32:41.719 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63173/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:32:41.722 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.163.12/tcp/63174/p2p/16Uiu2HAmBV77o2BUKZxnvPwQRBFwanyouak9e9QxjWX23nFfms84","/ip4/172.18.163.12/tcp/63175/ws/p2p/16Uiu2HAmBV77o2BUKZxnvPwQRBFwanyouak9e9QxjWX23nFfms84"],"enrUri":"enr:-L24QK6qfFH_9opOqZLA0isM801nk8Q6XwA9oRnsyhtNnxSuKZX0GpAXKQTEQ4fa8Jx3veo7AIq0cjSWHyRrBzRqprsCgmlkgnY0gmlwhKwSowyKbXVsdGlhZGRyc5YACASsEqMMBvbGAAoErBKjDAb2x90DgnJzhQADAQAAiXNlY3AyNTZrMaEC7pwoJOIdbmsUmGQDy7w8mqfIkWAjdEBTmX2g49tLn-GDdGNwgvbGg3VkcIL2yIV3YWt1MgM"}'
2026-03-13 04:32:41.722 INFO [src.node.waku_node] REST service is ready !!
2026-03-13 04:32:41.722 DEBUG [src.node.waku_node] Starting Node...
2026-03-13 04:32:41.722 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-13 04:32:41.724 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-13 04:32:41.724 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.229.24
2026-03-13 04:32:41.724 DEBUG [src.node.docker_mananger] Generated ports ['51412', '51413', '51414', '51415', '51416']
2026-03-13 04:32:41.725 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-13 04:32:41.725 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-13 04:32:41.725 DEBUG [src.node.waku_node] Using volumes []
2026-03-13 04:32:41.725 DEBUG [src.node.docker_mananger] docker run -i -t -p 51412:51412 -p 51413:51413 -p 51414:51414 -p 51415:51415 -p 51416:51416 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=51414 --rest-port=51412 --tcp-port=51413 --discv5-udp-port=51415 --rest-address=0.0.0.0 --nat=extip:172.18.229.24 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=390957fcadb893a32d8cffe4e2d4ce14e346b6bb6e18acffbb05fe564be7c60e --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=51416 --metrics-logging=true --store=false --relay=false --discv5-bootstrap-node=enr:-L24QK6qfFH_9opOqZLA0isM801nk8Q6XwA9oRnsyhtNnxSuKZX0GpAXKQTEQ4fa8Jx3veo7AIq0cjSWHyRrBzRqprsCgmlkgnY0gmlwhKwSowyKbXVsdGlhZGRyc5YACASsEqMMBvbGAAoErBKjDAb2x90DgnJzhQADAQAAiXNlY3AyNTZrMaEC7pwoJOIdbmsUmGQDy7w8mqfIkWAjdEBTmX2g49tLn-GDdGNwgvbGg3VkcIL2yIV3YWt1MgM
2026-03-13 04:32:41.928 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.229.24 waku 5d9423bfd310b7e8e330ca3ef8d94ae54aeee4cb4e4a4ab4b175132e4c7522ec
2026-03-13 04:32:41.964 DEBUG [src.node.docker_mananger] Container started with ID 5d9423bfd310. Setting up logs at ./log/docker/node2_2026-03-13_04-32-37__beea50b0-8cb8-4f51-bdc9-8521835a9347__wakuorg_nwaku:latest.log
2026-03-13 04:32:41.965 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 51412
2026-03-13 04:32:41.966 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-13 04:32:42.966 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:51412/health" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:32:42.969 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-13 04:32:42.970 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-13 04:32:42.970 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:51412/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:32:42.972 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.229.24/tcp/51413/p2p/16Uiu2HAm7BLdn9ULFsi797ZojSFww6pWFDDWqpgJQQ1faCkDtJFL","/ip4/172.18.229.24/tcp/51414/ws/p2p/16Uiu2HAm7BLdn9ULFsi797ZojSFww6pWFDDWqpgJQQ1faCkDtJFL"],"enrUri":"enr:-L24QIcLPx9nwqgkx0UKlr7qzPbps_RdVDGoN0sLzyLBGfp4eIIptP0UZH16t-QXamfcJ1Go7C-EtakX8sEWZTf3zMgCgmlkgnY0gmlwhKwS5RiKbXVsdGlhZGRyc5YACASsEuUYBsjVAAoErBLlGAbI1t0DgnJzhQADAQAAiXNlY3AyNTZrMaECrqA8_YpXOVJo0bCc-ffc7WcI14n-ir1oOW08faBC4HuDdGNwgsjVg3VkcILI14V3YWt1MgA"}'
2026-03-13 04:32:42.972 INFO [src.node.waku_node] REST service is ready !!
2026-03-13 04:32:42.973 DEBUG [src.node.waku_node] Starting Node...
2026-03-13 04:32:42.973 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-13 04:32:42.974 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-13 04:32:42.974 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.241.179
2026-03-13 04:32:42.974 DEBUG [src.node.docker_mananger] Generated ports ['27359', '27360', '27361', '27362', '27363']
2026-03-13 04:32:42.975 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-13 04:32:42.975 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-13 04:32:42.975 DEBUG [src.node.waku_node] Using volumes []
2026-03-13 04:32:42.975 DEBUG [src.node.docker_mananger] docker run -i -t -p 27359:27359 -p 27360:27360 -p 27361:27361 -p 27362:27362 -p 27363:27363 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=27361 --rest-port=27359 --tcp-port=27360 --discv5-udp-port=27362 --rest-address=0.0.0.0 --nat=extip:172.18.241.179 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=96b0fc577018724ca5a0ed3cb5d03ae27bbceacec50abdd8fc1112ff68befb51 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=27363 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QIcLPx9nwqgkx0UKlr7qzPbps_RdVDGoN0sLzyLBGfp4eIIptP0UZH16t-QXamfcJ1Go7C-EtakX8sEWZTf3zMgCgmlkgnY0gmlwhKwS5RiKbXVsdGlhZGRyc5YACASsEuUYBsjVAAoErBLlGAbI1t0DgnJzhQADAQAAiXNlY3AyNTZrMaECrqA8_YpXOVJo0bCc-ffc7WcI14n-ir1oOW08faBC4HuDdGNwgsjVg3VkcILI14V3YWt1MgA
2026-03-13 04:32:43.191 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.241.179 waku a54cf903387811e4469abdfd0cc5b7c1924df75a9f2ac0ea71f27cb42ed58baa
2026-03-13 04:32:43.228 DEBUG [src.node.docker_mananger] Container started with ID a54cf9033878. Setting up logs at ./log/docker/node3_2026-03-13_04-32-37__beea50b0-8cb8-4f51-bdc9-8521835a9347__wakuorg_nwaku:latest.log
2026-03-13 04:32:43.228 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 27359
2026-03-13 04:32:43.228 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-13 04:32:44.230 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27359/health" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:32:44.233 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-13 04:32:44.233 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-13 04:32:44.233 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27359/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:32:44.236 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.241.179/tcp/27360/p2p/16Uiu2HAkzLrHevJsKiFBKK6uSN5wPqrFp11nxqdLgWsHoT62UALo","/ip4/172.18.241.179/tcp/27361/ws/p2p/16Uiu2HAkzLrHevJsKiFBKK6uSN5wPqrFp11nxqdLgWsHoT62UALo"],"enrUri":"enr:-L24QEWIwrPrCFz5yJ6LAS7ls-fg0IGdkEGKv86TRbQGcCDYVKAFzQB8dv4I8kXehcYhe3eioWTQUcjQqO3_OnF7u3ACgmlkgnY0gmlwhKwS8bOKbXVsdGlhZGRyc5YACASsEvGzBmrgAAoErBLxswZq4d0DgnJzhQADAQAAiXNlY3AyNTZrMaECSQ4vkgqKi1DM-UqersdQ5IUedPPQ95gVpGH6wAIp26iDdGNwgmrgg3VkcIJq4oV3YWt1MgA"}'
2026-03-13 04:32:44.236 INFO [src.node.waku_node] REST service is ready !!
2026-03-13 04:32:44.236 DEBUG [src.node.waku_node] Starting Node...
2026-03-13 04:32:44.236 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-13 04:32:44.237 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-13 04:32:44.238 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.54.205
2026-03-13 04:32:44.238 DEBUG [src.node.docker_mananger] Generated ports ['20175', '20176', '20177', '20178', '20179']
2026-03-13 04:32:44.238 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-13 04:32:44.238 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-13 04:32:44.238 DEBUG [src.node.waku_node] Using volumes []
2026-03-13 04:32:44.238 DEBUG [src.node.docker_mananger] docker run -i -t -p 20175:20175 -p 20176:20176 -p 20177:20177 -p 20178:20178 -p 20179:20179 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=20177 --rest-port=20175 --tcp-port=20176 --discv5-udp-port=20178 --rest-address=0.0.0.0 --nat=extip:172.18.54.205 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=a19fa2baac2affad27309b7cb20f6cd0026d124af2b6d4eef10552abd353ef59 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=20179 --metrics-logging=true --relay=true --store=false --discv5-bootstrap-node=enr:-L24QEWIwrPrCFz5yJ6LAS7ls-fg0IGdkEGKv86TRbQGcCDYVKAFzQB8dv4I8kXehcYhe3eioWTQUcjQqO3_OnF7u3ACgmlkgnY0gmlwhKwS8bOKbXVsdGlhZGRyc5YACASsEvGzBmrgAAoErBLxswZq4d0DgnJzhQADAQAAiXNlY3AyNTZrMaECSQ4vkgqKi1DM-UqersdQ5IUedPPQ95gVpGH6wAIp26iDdGNwgmrgg3VkcIJq4oV3YWt1MgA
2026-03-13 04:32:44.439 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.54.205 waku 49c2628b40dd4521542f8f65ab9df8d1e23d18144f405ac47b9b87f31fabc362
2026-03-13 04:32:44.474 DEBUG [src.node.docker_mananger] Container started with ID 49c2628b40dd. Setting up logs at ./log/docker/node3_2026-03-13_04-32-37__beea50b0-8cb8-4f51-bdc9-8521835a9347__wakuorg_nwaku:latest.log
2026-03-13 04:32:44.474 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 20175
2026-03-13 04:32:44.474 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-13 04:32:45.477 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:20175/health" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:32:45.498 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-13 04:32:45.498 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-13 04:32:45.499 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:20175/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:32:45.508 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.54.205/tcp/20176/p2p/16Uiu2HAkuqvk3qUHq2ejtkebFVKzXXq8gwhoW4drMDCKBvcRHwQp","/ip4/172.18.54.205/tcp/20177/ws/p2p/16Uiu2HAkuqvk3qUHq2ejtkebFVKzXXq8gwhoW4drMDCKBvcRHwQp"],"enrUri":"enr:-L24QK5V0RKapQVVYFgmsI4w3QTmQhJNAvEdGXS0VTIpCvVUfzk7zHBnQ6lF-kxVsNJOwEGJkbgO8xiQhhnKTwMZ_j4CgmlkgnY0gmlwhKwSNs2KbXVsdGlhZGRyc5YACASsEjbNBk7QAAoErBI2zQZO0d0DgnJzhQADAQAAiXNlY3AyNTZrMaECBjahzTsfllrnJvNdf5cfeB4sSymri1QUskLVGODcwB2DdGNwgk7Qg3VkcIJO0oV3YWt1MgE"}'
2026-03-13 04:32:45.510 INFO [src.node.waku_node] REST service is ready !!
2026-03-13 04:32:45.511 DEBUG [src.node.waku_node] Starting Node...
2026-03-13 04:32:45.511 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-03-13 04:32:45.514 DEBUG [src.node.docker_mananger] Network waku already exists
2026-03-13 04:32:45.517 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.254.235
2026-03-13 04:32:45.517 DEBUG [src.node.docker_mananger] Generated ports ['50607', '50608', '50609', '50610', '50611']
2026-03-13 04:32:45.517 DEBUG [src.node.waku_node] RLN credentials were not set
2026-03-13 04:32:45.517 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-03-13 04:32:45.517 DEBUG [src.node.waku_node] Using volumes []
2026-03-13 04:32:45.518 DEBUG [src.node.docker_mananger] docker run -i -t -p 50607:50607 -p 50608:50608 -p 50609:50609 -p 50610:50610 -p 50611:50611 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=50609 --rest-port=50607 --tcp-port=50608 --discv5-udp-port=50610 --rest-address=0.0.0.0 --nat=extip:172.18.254.235 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=66a40b12dc3b843c3ad2aaa6bbaaee1c2cba2a85c05fef250817ee6a19aba0bd --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=50611 --metrics-logging=true --relay=false --store=false --discv5-bootstrap-node=enr:-L24QK5V0RKapQVVYFgmsI4w3QTmQhJNAvEdGXS0VTIpCvVUfzk7zHBnQ6lF-kxVsNJOwEGJkbgO8xiQhhnKTwMZ_j4CgmlkgnY0gmlwhKwSNs2KbXVsdGlhZGRyc5YACASsEjbNBk7QAAoErBI2zQZO0d0DgnJzhQADAQAAiXNlY3AyNTZrMaECBjahzTsfllrnJvNdf5cfeB4sSymri1QUskLVGODcwB2DdGNwgk7Qg3VkcIJO0oV3YWt1MgE
2026-03-13 04:32:45.721 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.254.235 waku 323ddb4760204018675e3b8c2c865424739e83972cc1925ff9016602d82e3a1c
2026-03-13 04:32:45.757 DEBUG [src.node.docker_mananger] Container started with ID 323ddb476020. Setting up logs at ./log/docker/node4_2026-03-13_04-32-37__beea50b0-8cb8-4f51-bdc9-8521835a9347__wakuorg_nwaku:latest.log
2026-03-13 04:32:45.757 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 50607
2026-03-13 04:32:45.758 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-03-13 04:32:46.758 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50607/health" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:32:46.761 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","connectionStatus":"Disconnected","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"},{"Rln Relay":"NOT_MOUNTED"}]}'
2026-03-13 04:32:46.761 INFO [src.node.waku_node] Node protocols are initialized !!
2026-03-13 04:32:46.761 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:50607/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:32:46.764 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.254.235/tcp/50608/p2p/16Uiu2HAmBqK5JXeqK4RJ6SS9uvspWMZEb2mpoUsgzoYAhSeRfRLD","/ip4/172.18.254.235/tcp/50609/ws/p2p/16Uiu2HAmBqK5JXeqK4RJ6SS9uvspWMZEb2mpoUsgzoYAhSeRfRLD"],"enrUri":"enr:-L24QLnFMY2rB20Wq42VfKfZUQegOGTEfs9ujOvLaB1eRtFyZf0MI5FIbUPBv5YatvWp6uanFb9n8WIyWVr0mC4LRA0CgmlkgnY0gmlwhKwS_uuKbXVsdGlhZGRyc5YACASsEv7rBsWwAAoErBL-6wbFsd0DgnJzhQADAQAAiXNlY3AyNTZrMaEC88lK-ZoM353ecDaDpochXJqO7o1NXM4WY2rVgdr2-oqDdGNwgsWwg3VkcILFsoV3YWt1MgA"}'
2026-03-13 04:32:46.764 INFO [src.node.waku_node] REST service is ready !!
2026-03-13 04:32:46.764 DEBUG [tests.e2e.test_e2e] Add 3 peer nodes to node3
2026-03-13 04:32:46.764 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:27359/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.163.12/tcp/63174/p2p/16Uiu2HAmBV77o2BUKZxnvPwQRBFwanyouak9e9QxjWX23nFfms84"]'
2026-03-13 04:32:46.783 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:32:46.784 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:27359/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.229.24/tcp/51413/p2p/16Uiu2HAm7BLdn9ULFsi797ZojSFww6pWFDDWqpgJQQ1faCkDtJFL"]'
2026-03-13 04:32:46.804 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:32:46.805 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:27359/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.54.205/tcp/20176/p2p/16Uiu2HAkuqvk3qUHq2ejtkebFVKzXXq8gwhoW4drMDCKBvcRHwQp"]'
2026-03-13 04:32:46.814 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:32:46.815 DEBUG [tests.e2e.test_e2e] Subscribe nodes 1,2 to relay on pubsubtopic /waku/2/rs/3/1
2026-03-13 04:32:46.816 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:20175/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-13 04:32:46.834 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:32:46.837 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:63173/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-03-13 04:32:46.842 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:32:46.843 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:63173/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:32:46.846 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.241.179/tcp/36250/p2p/16Uiu2HAkzLrHevJsKiFBKK6uSN5wPqrFp11nxqdLgWsHoT62UALo","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"UnknownOrigin"}]'
2026-03-13 04:32:46.847 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:20175/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:32:46.850 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.241.179/tcp/27360/p2p/16Uiu2HAkzLrHevJsKiFBKK6uSN5wPqrFp11nxqdLgWsHoT62UALo","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"logos-delivery-v0.36.0-164-g0ad551","origin":"Discv5"}]'
2026-03-13 04:32:46.850 DEBUG [src.libs.common] Sleeping for 30 seconds
2026-03-13 04:33:16.851 DEBUG [tests.e2e.test_e2e] Node1 publish message
2026-03-13 04:33:16.852 DEBUG [src.steps.store] Relaying message
2026-03-13 04:33:16.852 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:20175/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "RmlsdGVyIHdvcmtzISE=", "contentTopic": "/test/1/waku-filter/proto", "timestamp": '$(date +%s%N)'}'
2026-03-13 04:33:16.858 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-03-13 04:33:16.858 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2026-03-13 04:33:17.059 DEBUG [tests.e2e.test_e2e] Check if node3 can inquiry stored message without stor peer specified
2026-03-13 04:33:17.059 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:27359/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F1&contentTopics=%2Ftest%2F1%2Fwaku-filter%2Fproto&pageSize=5&ascending=true" -H "Content-Type: application/json" -d 'None'
2026-03-13 04:33:17.069 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"f45139e02a2352298c79","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x6bcf198f5dd4b766ef56be9d6be1e0a8717b1c0fdf8f7e9e5b8ef693221c84aa"}]}'
2026-03-13 04:33:17.071 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-03-13 04:33:17.072 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-03-13 04:33:17.072 DEBUG [src.node.waku_node] Stopping container with id e4258df5a90d
2026-03-13 04:33:17.615 DEBUG [src.node.waku_node] Container stopped.
2026-03-13 04:33:17.617 DEBUG [src.node.waku_node] Stopping container with id 5d9423bfd310
2026-03-13 04:33:18.212 DEBUG [src.node.waku_node] Container stopped.
2026-03-13 04:33:18.214 DEBUG [src.node.waku_node] Stopping container with id a54cf9033878
2026-03-13 04:33:18.524 ERROR [src.node.docker_mananger] Max retries reached for container e4258df5a90d. Exiting log stream.
2026-03-13 04:33:18.768 DEBUG [src.node.waku_node] Container stopped.
2026-03-13 04:33:18.769 DEBUG [src.node.waku_node] Stopping container with id 49c2628b40dd
2026-03-13 04:33:19.029 ERROR [src.node.docker_mananger] Max retries reached for container 5d9423bfd310. Exiting log stream.
2026-03-13 04:33:19.324 DEBUG [src.node.waku_node] Container stopped.
2026-03-13 04:33:19.326 DEBUG [src.node.waku_node] Stopping container with id 323ddb476020
2026-03-13 04:33:19.679 ERROR [src.node.docker_mananger] Max retries reached for container a54cf9033878. Exiting log stream.
2026-03-13 04:33:19.874 DEBUG [src.node.waku_node] Container stopped.
2026-03-13 04:33:19.876 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-03-13 04:33:19.920 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-13 04:33:19.929 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-13 04:33:19.976 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-13 04:33:20.024 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-03-13 04:33:20.026 DEBUG [src.node.docker_mananger] No errors found in the waku logs.