162 lines
30 KiB
Plaintext

2025-12-10 04:12:25.752 DEBUG [tests.conftest] Running fixture setup: test_id
2025-12-10 04:12:25.752 DEBUG [tests.conftest] Running test: test_store_no_peer_selected with id: 2025-12-10_04-12-25__d4fb63a7-03d3-4cee-a924-eb5db8d4ef6c
2025-12-10 04:12:25.753 DEBUG [src.steps.common] Running fixture setup: common_setup
2025-12-10 04:12:25.753 DEBUG [src.steps.filter] Running fixture setup: filter_setup
2025-12-10 04:12:25.753 DEBUG [src.steps.light_push] Running fixture setup: light_push_setup
2025-12-10 04:12:25.763 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-10 04:12:25.764 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2025-12-10_04-12-25__d4fb63a7-03d3-4cee-a924-eb5db8d4ef6c__wakuorg_nwaku:latest.log
2025-12-10 04:12:25.770 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-10 04:12:25.770 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2025-12-10_04-12-25__d4fb63a7-03d3-4cee-a924-eb5db8d4ef6c__wakuorg_nwaku:latest.log
2025-12-10 04:12:25.775 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-10 04:12:25.775 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2025-12-10_04-12-25__d4fb63a7-03d3-4cee-a924-eb5db8d4ef6c__wakuorg_nwaku:latest.log
2025-12-10 04:12:25.776 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2025-12-10 04:12:25.776 DEBUG [src.steps.store] Running fixture setup: store_setup
2025-12-10 04:12:25.777 DEBUG [tests.e2e.test_e2e] Start 5 nodes
2025-12-10 04:12:25.783 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-10 04:12:25.783 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2025-12-10_04-12-25__d4fb63a7-03d3-4cee-a924-eb5db8d4ef6c__wakuorg_nwaku:latest.log
2025-12-10 04:12:25.788 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-10 04:12:25.789 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2025-12-10_04-12-25__d4fb63a7-03d3-4cee-a924-eb5db8d4ef6c__wakuorg_nwaku:latest.log
2025-12-10 04:12:25.794 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2025-12-10 04:12:25.795 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2025-12-10_04-12-25__d4fb63a7-03d3-4cee-a924-eb5db8d4ef6c__wakuorg_nwaku:latest.log
2025-12-10 04:12:25.795 DEBUG [src.node.waku_node] Starting Node...
2025-12-10 04:12:25.795 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-10 04:12:25.836 DEBUG [src.node.docker_mananger] Network waku created
2025-12-10 04:12:25.836 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.101.193
2025-12-10 04:12:25.836 DEBUG [src.node.docker_mananger] Generated ports ['36055', '36056', '36057', '36058', '36059']
2025-12-10 04:12:25.837 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-10 04:12:25.837 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-10 04:12:25.837 DEBUG [src.node.waku_node] Using volumes []
2025-12-10 04:12:25.837 DEBUG [src.node.docker_mananger] docker run -i -t -p 36055:36055 -p 36056:36056 -p 36057:36057 -p 36058:36058 -p 36059:36059 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=36057 --rest-port=36055 --tcp-port=36056 --discv5-udp-port=36058 --rest-address=0.0.0.0 --nat=extip:172.18.101.193 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=006dd825a085a4cb7d9a11709d9e767d3bd0fbd80aaf42340bb9bb36d1f2a3e8 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=36059 --metrics-logging=true --relay=true --store=true
2025-12-10 04:12:28.035 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.101.193 waku aa7e12501ff986869d36229c30cc987a1a364b77f4a7d26b6c5603e64e4cdfe4
2025-12-10 04:12:28.087 DEBUG [src.node.docker_mananger] Container started with ID aa7e12501ff9. Setting up logs at ./log/docker/node1_2025-12-10_04-12-25__d4fb63a7-03d3-4cee-a924-eb5db8d4ef6c__wakuorg_nwaku:latest.log
2025-12-10 04:12:28.087 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 36055
2025-12-10 04:12:28.088 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-10 04:12:29.088 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36055/health" -H "Content-Type: application/json" -d 'None'
2025-12-10 04:12:29.091 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"READY"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-10 04:12:29.092 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-10 04:12:29.092 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36055/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-10 04:12:29.094 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.101.193/tcp/36056/p2p/16Uiu2HAmJZZ2Zna9LqqNFfZTwXgxXf644zFFjffDZ4V75hsUDHG8","/ip4/172.18.101.193/tcp/36057/ws/p2p/16Uiu2HAmJZZ2Zna9LqqNFfZTwXgxXf644zFFjffDZ4V75hsUDHG8"],"enrUri":"enr:-L24QPQ4qiaJRfuwrnGiXUsxcfi8krQ_fsLXrOm0KncuSrt8HDQxhTz4-tLNTVDLpknMlR6k8cb6wkcoZ9kkOREpdWgCgmlkgnY0gmlwhKwSZcGKbXVsdGlhZGRyc5YACASsEmXBBozYAAoErBJlwQaM2d0DgnJzhQADAQAAiXNlY3AyNTZrMaEDV8GVvEvdwoxBNIsv_5moLd1lgjtLjL8feP4Zw-y3932DdGNwgozYg3VkcIKM2oV3YWt1MgM"}'
2025-12-10 04:12:29.094 INFO [src.node.waku_node] REST service is ready !!
2025-12-10 04:12:29.095 DEBUG [src.node.waku_node] Starting Node...
2025-12-10 04:12:29.095 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-10 04:12:29.096 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-10 04:12:29.096 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.243.231
2025-12-10 04:12:29.096 DEBUG [src.node.docker_mananger] Generated ports ['56539', '56540', '56541', '56542', '56543']
2025-12-10 04:12:29.096 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-10 04:12:29.097 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-10 04:12:29.097 DEBUG [src.node.waku_node] Using volumes []
2025-12-10 04:12:29.097 DEBUG [src.node.docker_mananger] docker run -i -t -p 56539:56539 -p 56540:56540 -p 56541:56541 -p 56542:56542 -p 56543:56543 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=56541 --rest-port=56539 --tcp-port=56540 --discv5-udp-port=56542 --rest-address=0.0.0.0 --nat=extip:172.18.243.231 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=a2e7ced13fea4bb84aa0bf3d6df4a2c6bb1dd19c07c1b2e0b87ad30dc9f4dada --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=56543 --metrics-logging=true --store=false --relay=false --discv5-bootstrap-node=enr:-L24QPQ4qiaJRfuwrnGiXUsxcfi8krQ_fsLXrOm0KncuSrt8HDQxhTz4-tLNTVDLpknMlR6k8cb6wkcoZ9kkOREpdWgCgmlkgnY0gmlwhKwSZcGKbXVsdGlhZGRyc5YACASsEmXBBozYAAoErBJlwQaM2d0DgnJzhQADAQAAiXNlY3AyNTZrMaEDV8GVvEvdwoxBNIsv_5moLd1lgjtLjL8feP4Zw-y3932DdGNwgozYg3VkcIKM2oV3YWt1MgM
2025-12-10 04:12:29.279 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.243.231 waku d1398ae5eb53eae5393347f1424fe5971eb4efb6834410e7822d98c0847675a3
2025-12-10 04:12:29.309 DEBUG [src.node.docker_mananger] Container started with ID d1398ae5eb53. Setting up logs at ./log/docker/node2_2025-12-10_04-12-25__d4fb63a7-03d3-4cee-a924-eb5db8d4ef6c__wakuorg_nwaku:latest.log
2025-12-10 04:12:29.309 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 56539
2025-12-10 04:12:29.310 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-10 04:12:30.311 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56539/health" -H "Content-Type: application/json" -d 'None'
2025-12-10 04:12:30.315 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"READY"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-10 04:12:30.315 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-10 04:12:30.315 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:56539/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-10 04:12:30.318 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.243.231/tcp/56540/p2p/16Uiu2HAmFu8YMamqbE5Hybvrs96d5RL86Z7ZEqV5djoE5ebV9WTJ","/ip4/172.18.243.231/tcp/56541/ws/p2p/16Uiu2HAmFu8YMamqbE5Hybvrs96d5RL86Z7ZEqV5djoE5ebV9WTJ"],"enrUri":"enr:-L24QCUpv8eMrmZcvqsIM4U845o5GC8bwSUTAcshoI7Hm7WwE3tqfCOUqfd1MGz01CTI68ZHosfZkbonQ3vnjoUKVk4CgmlkgnY0gmlwhKwS8-eKbXVsdGlhZGRyc5YACASsEvPnBtzcAAoErBLz5wbc3d0DgnJzhQADAQAAiXNlY3AyNTZrMaEDMDJ0Fg_tagZMv7xmAKoAR33HAeFXYD4ZGni-iKb0c4mDdGNwgtzcg3VkcILc3oV3YWt1MgA"}'
2025-12-10 04:12:30.318 INFO [src.node.waku_node] REST service is ready !!
2025-12-10 04:12:30.318 DEBUG [src.node.waku_node] Starting Node...
2025-12-10 04:12:30.318 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-10 04:12:30.320 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-10 04:12:30.320 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.26.142
2025-12-10 04:12:30.320 DEBUG [src.node.docker_mananger] Generated ports ['38038', '38039', '38040', '38041', '38042']
2025-12-10 04:12:30.320 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-10 04:12:30.321 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-10 04:12:30.321 DEBUG [src.node.waku_node] Using volumes []
2025-12-10 04:12:30.321 DEBUG [src.node.docker_mananger] docker run -i -t -p 38038:38038 -p 38039:38039 -p 38040:38040 -p 38041:38041 -p 38042:38042 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=38040 --rest-port=38038 --tcp-port=38039 --discv5-udp-port=38041 --rest-address=0.0.0.0 --nat=extip:172.18.26.142 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=fe6af4f4140baaffd073f24fa7b9bd8ed2afce0ef4da3f941ad8b0ea8456bbfd --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=38042 --metrics-logging=true --relay=false --discv5-bootstrap-node=enr:-L24QCUpv8eMrmZcvqsIM4U845o5GC8bwSUTAcshoI7Hm7WwE3tqfCOUqfd1MGz01CTI68ZHosfZkbonQ3vnjoUKVk4CgmlkgnY0gmlwhKwS8-eKbXVsdGlhZGRyc5YACASsEvPnBtzcAAoErBLz5wbc3d0DgnJzhQADAQAAiXNlY3AyNTZrMaEDMDJ0Fg_tagZMv7xmAKoAR33HAeFXYD4ZGni-iKb0c4mDdGNwgtzcg3VkcILc3oV3YWt1MgA
2025-12-10 04:12:30.515 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.26.142 waku 766009b3c1c0810b3b48b84f9f4a91118c085c072d68332388a1fd97e8520176
2025-12-10 04:12:30.543 DEBUG [src.node.docker_mananger] Container started with ID 766009b3c1c0. Setting up logs at ./log/docker/node3_2025-12-10_04-12-25__d4fb63a7-03d3-4cee-a924-eb5db8d4ef6c__wakuorg_nwaku:latest.log
2025-12-10 04:12:30.543 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 38038
2025-12-10 04:12:30.543 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-10 04:12:31.544 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38038/health" -H "Content-Type: application/json" -d 'None'
2025-12-10 04:12:31.548 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-10 04:12:31.548 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-10 04:12:31.548 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38038/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-10 04:12:31.550 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.26.142/tcp/38039/p2p/16Uiu2HAm4hsMJsYXq1hss6i1eM7aRVtJnEUsXQZH6yVVMKM99Hog","/ip4/172.18.26.142/tcp/38040/ws/p2p/16Uiu2HAm4hsMJsYXq1hss6i1eM7aRVtJnEUsXQZH6yVVMKM99Hog"],"enrUri":"enr:-L24QBjJpCvgOuenjyDyg2CQ0Tni9T2AxhtmZOBCb-0C6XVGNU1FdbRJSnPsmQZuDTs0aeHzHzER3TstJ_avNc6o8z0CgmlkgnY0gmlwhKwSGo6KbXVsdGlhZGRyc5YACASsEhqOBpSXAAoErBIajgaUmN0DgnJzhQADAQAAiXNlY3AyNTZrMaECid9VHEohQelODAFQZDUClxCarHPyNiDUzh6p07wYpZODdGNwgpSXg3VkcIKUmYV3YWt1MgA"}'
2025-12-10 04:12:31.550 INFO [src.node.waku_node] REST service is ready !!
2025-12-10 04:12:31.551 DEBUG [src.node.waku_node] Starting Node...
2025-12-10 04:12:31.551 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-10 04:12:31.552 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-10 04:12:31.552 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.153.57
2025-12-10 04:12:31.552 DEBUG [src.node.docker_mananger] Generated ports ['2186', '2187', '2188', '2189', '2190']
2025-12-10 04:12:31.552 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-10 04:12:31.552 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-10 04:12:31.553 DEBUG [src.node.waku_node] Using volumes []
2025-12-10 04:12:31.553 DEBUG [src.node.docker_mananger] docker run -i -t -p 2186:2186 -p 2187:2187 -p 2188:2188 -p 2189:2189 -p 2190:2190 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=2188 --rest-port=2186 --tcp-port=2187 --discv5-udp-port=2189 --rest-address=0.0.0.0 --nat=extip:172.18.153.57 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=4bcd02c85af75ffcf5a1aee7c5efb398ff3c4ec0a17ff02cbc69eadecdbd1e60 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=2190 --metrics-logging=true --relay=true --store=false --discv5-bootstrap-node=enr:-L24QBjJpCvgOuenjyDyg2CQ0Tni9T2AxhtmZOBCb-0C6XVGNU1FdbRJSnPsmQZuDTs0aeHzHzER3TstJ_avNc6o8z0CgmlkgnY0gmlwhKwSGo6KbXVsdGlhZGRyc5YACASsEhqOBpSXAAoErBIajgaUmN0DgnJzhQADAQAAiXNlY3AyNTZrMaECid9VHEohQelODAFQZDUClxCarHPyNiDUzh6p07wYpZODdGNwgpSXg3VkcIKUmYV3YWt1MgA
2025-12-10 04:12:31.748 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.153.57 waku dde9c2a8749ff0e16096ddf7aecfa2603d5f47b1698642625c42a315c94d6746
2025-12-10 04:12:31.782 DEBUG [src.node.docker_mananger] Container started with ID dde9c2a8749f. Setting up logs at ./log/docker/node3_2025-12-10_04-12-25__d4fb63a7-03d3-4cee-a924-eb5db8d4ef6c__wakuorg_nwaku:latest.log
2025-12-10 04:12:31.783 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 2186
2025-12-10 04:12:31.784 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-10 04:12:32.784 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2186/health" -H "Content-Type: application/json" -d 'None'
2025-12-10 04:12:32.790 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-10 04:12:32.791 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-10 04:12:32.791 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2186/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-10 04:12:32.795 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.153.57/tcp/2187/p2p/16Uiu2HAm2NY4SMr7wU33GZ8cNMRDjwAgPUVqneQbie5RWkL6cAFQ","/ip4/172.18.153.57/tcp/2188/ws/p2p/16Uiu2HAm2NY4SMr7wU33GZ8cNMRDjwAgPUVqneQbie5RWkL6cAFQ"],"enrUri":"enr:-L24QOgfxpI0aKoOtaYG6EFadsGA3er3KSEY2XJR4wcHZ3-9MW2rpsXydF34oqGphJheXLHWN30p-Bm3njJIAq3aZ1MCgmlkgnY0gmlwhKwSmTmKbXVsdGlhZGRyc5YACASsEpk5BgiLAAoErBKZOQYIjN0DgnJzhQADAQAAiXNlY3AyNTZrMaECZzQcaLpaM6PGceACy8NUpw3IGYmvuincHPll6HhPIm-DdGNwggiLg3VkcIIIjYV3YWt1MgE"}'
2025-12-10 04:12:32.796 INFO [src.node.waku_node] REST service is ready !!
2025-12-10 04:12:32.796 DEBUG [src.node.waku_node] Starting Node...
2025-12-10 04:12:32.798 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2025-12-10 04:12:32.801 DEBUG [src.node.docker_mananger] Network waku already exists
2025-12-10 04:12:32.802 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.72.1
2025-12-10 04:12:32.803 DEBUG [src.node.docker_mananger] Generated ports ['54716', '54717', '54718', '54719', '54720']
2025-12-10 04:12:32.803 DEBUG [src.node.waku_node] RLN credentials were not set
2025-12-10 04:12:32.803 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2025-12-10 04:12:32.803 DEBUG [src.node.waku_node] Using volumes []
2025-12-10 04:12:32.803 DEBUG [src.node.docker_mananger] docker run -i -t -p 54716:54716 -p 54717:54717 -p 54718:54718 -p 54719:54719 -p 54720:54720 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=54718 --rest-port=54716 --tcp-port=54717 --discv5-udp-port=54719 --rest-address=0.0.0.0 --nat=extip:172.18.72.1 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=5fac2fb584aaed4e6d9f00dfc9fbc33ad0cd4cbacd854dc28db37c06f003d618 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=54720 --metrics-logging=true --relay=false --store=false --discv5-bootstrap-node=enr:-L24QOgfxpI0aKoOtaYG6EFadsGA3er3KSEY2XJR4wcHZ3-9MW2rpsXydF34oqGphJheXLHWN30p-Bm3njJIAq3aZ1MCgmlkgnY0gmlwhKwSmTmKbXVsdGlhZGRyc5YACASsEpk5BgiLAAoErBKZOQYIjN0DgnJzhQADAQAAiXNlY3AyNTZrMaECZzQcaLpaM6PGceACy8NUpw3IGYmvuincHPll6HhPIm-DdGNwggiLg3VkcIIIjYV3YWt1MgE
2025-12-10 04:12:32.999 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.72.1 waku 0ed22764f7199a99cffc883e96c0f8ec706bb613756ed775f99120f6012c3833
2025-12-10 04:12:33.031 DEBUG [src.node.docker_mananger] Container started with ID 0ed22764f719. Setting up logs at ./log/docker/node4_2025-12-10_04-12-25__d4fb63a7-03d3-4cee-a924-eb5db8d4ef6c__wakuorg_nwaku:latest.log
2025-12-10 04:12:33.031 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 54716
2025-12-10 04:12:33.032 DEBUG [src.libs.common] Sleeping for 1 seconds
2025-12-10 04:12:34.032 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:54716/health" -H "Content-Type: application/json" -d 'None'
2025-12-10 04:12:34.036 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_MOUNTED"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2025-12-10 04:12:34.036 INFO [src.node.waku_node] Node protocols are initialized !!
2025-12-10 04:12:34.036 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:54716/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2025-12-10 04:12:34.038 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.72.1/tcp/54717/p2p/16Uiu2HAmLQTRD24bn2KX4h3LYnFHvBdH1YZpZNHHmeojF9jk7b71","/ip4/172.18.72.1/tcp/54718/ws/p2p/16Uiu2HAmLQTRD24bn2KX4h3LYnFHvBdH1YZpZNHHmeojF9jk7b71"],"enrUri":"enr:-L24QDJwIzD1t7tfKLqlvyM07IyHQZwr8bnGt3XSkASlmDg5WzVXihSBODic2FS4tZcagCMI8gMTKhKFw4waAtMA3mcCgmlkgnY0gmlwhKwSSAGKbXVsdGlhZGRyc5YACASsEkgBBtW9AAoErBJIAQbVvt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDcyRjfvU9NnkawiiI3lVLOagHr4u8yrT8pES2MXzNKgSDdGNwgtW9g3VkcILVv4V3YWt1MgA"}'
2025-12-10 04:12:34.039 INFO [src.node.waku_node] REST service is ready !!
2025-12-10 04:12:34.039 DEBUG [tests.e2e.test_e2e] Add 3 peer nodes to node3
2025-12-10 04:12:34.039 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38038/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.101.193/tcp/36056/p2p/16Uiu2HAmJZZ2Zna9LqqNFfZTwXgxXf644zFFjffDZ4V75hsUDHG8"]'
2025-12-10 04:12:34.059 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-10 04:12:34.060 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38038/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.243.231/tcp/56540/p2p/16Uiu2HAmFu8YMamqbE5Hybvrs96d5RL86Z7ZEqV5djoE5ebV9WTJ"]'
2025-12-10 04:12:34.086 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-10 04:12:34.086 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:38038/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.153.57/tcp/2187/p2p/16Uiu2HAm2NY4SMr7wU33GZ8cNMRDjwAgPUVqneQbie5RWkL6cAFQ"]'
2025-12-10 04:12:34.092 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-10 04:12:34.093 DEBUG [tests.e2e.test_e2e] Subscribe nodes 1,2 to relay on pubsubtopic /waku/2/rs/3/1
2025-12-10 04:12:34.094 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:2186/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2025-12-10 04:12:34.099 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-10 04:12:34.100 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:36055/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2025-12-10 04:12:34.106 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-10 04:12:34.106 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:36055/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2025-12-10 04:12:34.109 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.26.142/tcp/58968/p2p/16Uiu2HAm4hsMJsYXq1hss6i1eM7aRVtJnEUsXQZH6yVVMKM99Hog","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[],"connected":"Connected","agent":"nwaku-v0.36.0-112-g12952d","origin":"UnknownOrigin"}]'
2025-12-10 04:12:34.109 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:2186/admin/v1/peers" -H "Content-Type: application/json" -d 'None'
2025-12-10 04:12:34.111 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"multiaddr":"/ip4/172.18.26.142/tcp/38039/p2p/16Uiu2HAm4hsMJsYXq1hss6i1eM7aRVtJnEUsXQZH6yVVMKM99Hog","protocols":["/ipfs/id/1.0.0","/libp2p/autonat/1.0.0","/libp2p/circuit/relay/0.2.0/hop","/vac/waku/metadata/1.0.0","/vac/waku/rendezvous/1.0.0","/ipfs/ping/1.0.0","/vac/waku/filter-push/2.0.0-beta1","/vac/waku/peer-exchange/2.0.0-alpha1"],"shards":[0],"connected":"Connected","agent":"nwaku-v0.36.0-112-g12952d","origin":"Discv5"}]'
2025-12-10 04:12:34.111 DEBUG [src.libs.common] Sleeping for 30 seconds
2025-12-10 04:13:04.111 DEBUG [tests.e2e.test_e2e] Node1 publish message
2025-12-10 04:13:04.112 DEBUG [src.steps.store] Relaying message
2025-12-10 04:13:04.112 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:2186/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "RmlsdGVyIHdvcmtzISE=", "contentTopic": "/test/1/waku-filter/proto", "timestamp": '$(date +%s%N)'}'
2025-12-10 04:13:04.116 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2025-12-10 04:13:04.117 DEBUG [src.libs.common] Sleeping for 0.2 seconds
2025-12-10 04:13:04.317 DEBUG [tests.e2e.test_e2e] Check if node3 can inquiry stored message without stor peer specified
2025-12-10 04:13:04.318 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:38038/store/v3/messages?pubsubTopic=%2Fwaku%2F2%2Frs%2F3%2F1&contentTopics=%2Ftest%2F1%2Fwaku-filter%2Fproto&pageSize=5&ascending=true" -H "Content-Type: application/json" -d 'None'
2025-12-10 04:13:04.326 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"requestId":"897a71a3e3d5be8b78c8","statusCode":200,"statusDesc":"OK","messages":[{"messageHash":"0x556adb3790b4efb310581cdcb0c67a621712a99109d646f929615bcb27510469"}]}'
2025-12-10 04:13:04.328 DEBUG [tests.conftest] Running fixture teardown: test_setup
2025-12-10 04:13:04.329 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2025-12-10 04:13:04.329 DEBUG [src.node.waku_node] Stopping container with id aa7e12501ff9
2025-12-10 04:13:04.860 DEBUG [src.node.waku_node] Container stopped.
2025-12-10 04:13:04.860 DEBUG [src.node.waku_node] Stopping container with id d1398ae5eb53
2025-12-10 04:13:05.426 DEBUG [src.node.waku_node] Container stopped.
2025-12-10 04:13:05.427 DEBUG [src.node.waku_node] Stopping container with id 766009b3c1c0
2025-12-10 04:13:05.673 ERROR [src.node.docker_mananger] Max retries reached for container aa7e12501ff9. Exiting log stream.
2025-12-10 04:13:05.952 DEBUG [src.node.waku_node] Container stopped.
2025-12-10 04:13:05.953 DEBUG [src.node.waku_node] Stopping container with id dde9c2a8749f
2025-12-10 04:13:06.239 ERROR [src.node.docker_mananger] Max retries reached for container d1398ae5eb53. Exiting log stream.
2025-12-10 04:13:06.482 DEBUG [src.node.waku_node] Container stopped.
2025-12-10 04:13:06.483 DEBUG [src.node.waku_node] Stopping container with id 0ed22764f719
2025-12-10 04:13:06.765 ERROR [src.node.docker_mananger] Max retries reached for container 766009b3c1c0. Exiting log stream.
2025-12-10 04:13:07.038 DEBUG [src.node.waku_node] Container stopped.
2025-12-10 04:13:07.039 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2025-12-10 04:13:07.060 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2025-12-10 04:13:07.069 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2025-12-10 04:13:07.092 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2025-12-10 04:13:07.116 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2025-12-10 04:13:07.119 DEBUG [src.node.docker_mananger] No errors found in the waku logs.