2026-02-04 17:29:28 +00:00

192 lines
35 KiB
Plaintext

2026-02-04 17:11:30.669 DEBUG [tests.conftest] Running fixture setup: test_id
2026-02-04 17:11:30.670 DEBUG [tests.conftest] Running test: test_first_node_to_start_publishes with id: 2026-02-04_17-11-30__cfb03cff-5136-4744-8d1d-a78fbd4c9666
2026-02-04 17:11:30.670 DEBUG [src.steps.common] Running fixture setup: common_setup
2026-02-04 17:11:30.670 DEBUG [src.steps.relay] Running fixture setup: relay_setup
2026-02-04 17:11:30.671 DEBUG [src.steps.relay] Running fixture setup: setup_main_relay_nodes
2026-02-04 17:11:30.678 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-04 17:11:30.679 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node1_2026-02-04_17-11-30__cfb03cff-5136-4744-8d1d-a78fbd4c9666__wakuorg_nwaku:latest.log
2026-02-04 17:11:30.679 DEBUG [src.node.waku_node] Starting Node...
2026-02-04 17:11:30.679 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-04 17:11:30.680 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-04 17:11:30.681 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.220.114
2026-02-04 17:11:30.681 DEBUG [src.node.docker_mananger] Generated ports ['24480', '24481', '24482', '24483', '24484']
2026-02-04 17:11:30.681 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-04 17:11:30.681 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-04 17:11:30.681 DEBUG [src.node.waku_node] Using volumes []
2026-02-04 17:11:30.681 DEBUG [src.node.docker_mananger] docker run -i -t -p 24480:24480 -p 24481:24481 -p 24482:24482 -p 24483:24483 -p 24484:24484 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=24482 --rest-port=24480 --tcp-port=24481 --discv5-udp-port=24483 --rest-address=0.0.0.0 --nat=extip:172.18.220.114 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=214a8ebb4cd25adef7fef1dc65ee83af1b2dd711df85fac5bc7cd296cb19de25 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=24484 --metrics-logging=true --relay=true
2026-02-04 17:11:30.870 ERROR [src.node.docker_mananger] Max retries reached for container 012cb4cd7e8e. Exiting log stream.
2026-02-04 17:11:30.872 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.220.114 waku 9e8c4f8cb1bf8c84beb30f917275d5e748e5a07c2cbf26c7f5d77182587e489c
2026-02-04 17:11:30.907 DEBUG [src.node.docker_mananger] Container started with ID 9e8c4f8cb1bf. Setting up logs at ./log/docker/node1_2026-02-04_17-11-30__cfb03cff-5136-4744-8d1d-a78fbd4c9666__wakuorg_nwaku:latest.log
2026-02-04 17:11:30.909 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 24480
2026-02-04 17:11:30.909 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-04 17:11:31.460 ERROR [src.node.docker_mananger] Max retries reached for container 9257e35980f4. Exiting log stream.
2026-02-04 17:11:31.911 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24480/health" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:31.914 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-02-04 17:11:31.915 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-04 17:11:31.915 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24480/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:31.917 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.220.114/tcp/24481/p2p/16Uiu2HAm1reKhfQAjxPqH3hT69iwz2HapbvcHgVTbJZC4V8XArJu","/ip4/172.18.220.114/tcp/24482/ws/p2p/16Uiu2HAm1reKhfQAjxPqH3hT69iwz2HapbvcHgVTbJZC4V8XArJu"],"enrUri":"enr:-L24QN05RYybRfUBzISochpw035PldswfSpEDBK1Sgiz1LVfc-M47pjzHFau5ZriSg0sQo1z5k4a9LhEkLf-wyKZg68CgmlkgnY0gmlwhKwS3HKKbXVsdGlhZGRyc5YACASsEtxyBl-hAAoErBLccgZfot0DgnJzhQADAQAAiXNlY3AyNTZrMaECX4vEu7YUcyQ8-Qc2aR_YmSi1tmfa4Tb2gwR4S5q1gBqDdGNwgl-hg3VkcIJfo4V3YWt1MgE"}'
2026-02-04 17:11:31.917 INFO [src.node.waku_node] REST service is ready !!
2026-02-04 17:11:31.924 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-04 17:11:31.924 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node2_2026-02-04_17-11-30__cfb03cff-5136-4744-8d1d-a78fbd4c9666__wakuorg_nwaku:latest.log
2026-02-04 17:11:31.925 DEBUG [src.node.waku_node] Starting Node...
2026-02-04 17:11:31.925 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-04 17:11:31.926 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-04 17:11:31.926 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.157.181
2026-02-04 17:11:31.926 DEBUG [src.node.docker_mananger] Generated ports ['20402', '20403', '20404', '20405', '20406']
2026-02-04 17:11:31.927 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-04 17:11:31.927 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-04 17:11:31.927 DEBUG [src.node.waku_node] Using volumes []
2026-02-04 17:11:31.927 DEBUG [src.node.docker_mananger] docker run -i -t -p 20402:20402 -p 20403:20403 -p 20404:20404 -p 20405:20405 -p 20406:20406 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=20404 --rest-port=20402 --tcp-port=20403 --discv5-udp-port=20405 --rest-address=0.0.0.0 --nat=extip:172.18.157.181 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=e8be9605000aaf3a1d961d2d795458fba6af51de5a5ffe03f7afef820dfcd219 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=20406 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QN05RYybRfUBzISochpw035PldswfSpEDBK1Sgiz1LVfc-M47pjzHFau5ZriSg0sQo1z5k4a9LhEkLf-wyKZg68CgmlkgnY0gmlwhKwS3HKKbXVsdGlhZGRyc5YACASsEtxyBl-hAAoErBLccgZfot0DgnJzhQADAQAAiXNlY3AyNTZrMaECX4vEu7YUcyQ8-Qc2aR_YmSi1tmfa4Tb2gwR4S5q1gBqDdGNwgl-hg3VkcIJfo4V3YWt1MgE
2026-02-04 17:11:32.107 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.157.181 waku 1af8ad733c284af70d7eb45140a1e32b7a1be63d8ffe290e7f0a179d641b570d
2026-02-04 17:11:32.143 DEBUG [src.node.docker_mananger] Container started with ID 1af8ad733c28. Setting up logs at ./log/docker/node2_2026-02-04_17-11-30__cfb03cff-5136-4744-8d1d-a78fbd4c9666__wakuorg_nwaku:latest.log
2026-02-04 17:11:32.144 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 20402
2026-02-04 17:11:32.144 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-04 17:11:33.145 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:20402/health" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:33.150 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-02-04 17:11:33.151 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-04 17:11:33.151 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:20402/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:33.157 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.157.181/tcp/20403/p2p/16Uiu2HAky6H1TSFo78SNXeHjTi1eojCrmhaU3d3PpHViGcJqLnbA","/ip4/172.18.157.181/tcp/20404/ws/p2p/16Uiu2HAky6H1TSFo78SNXeHjTi1eojCrmhaU3d3PpHViGcJqLnbA"],"enrUri":"enr:-L24QIj2G5rDWTOGGj5Ln1xP4K2o5hGKepOXGHFJYkTqU1lBOlxRvV2lJYHrOxVxYQVYhfATj8O8Vqslk2AJofNFfuACgmlkgnY0gmlwhKwSnbWKbXVsdGlhZGRyc5YACASsEp21Bk-zAAoErBKdtQZPtN0DgnJzhQADAQAAiXNlY3AyNTZrMaECNna9Ewi_M6n8fssoTu8_WC65a35B22qEVDESaU-ke-mDdGNwgk-zg3VkcIJPtYV3YWt1MgE"}'
2026-02-04 17:11:33.158 INFO [src.node.waku_node] REST service is ready !!
2026-02-04 17:11:33.159 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:20402/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.220.114/tcp/24481/p2p/16Uiu2HAm1reKhfQAjxPqH3hT69iwz2HapbvcHgVTbJZC4V8XArJu"]'
2026-02-04 17:11:33.162 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-04 17:11:33.163 DEBUG [src.steps.relay] Running fixture setup: setup_optional_relay_nodes
2026-02-04 17:11:33.169 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-04 17:11:33.169 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node3_2026-02-04_17-11-30__cfb03cff-5136-4744-8d1d-a78fbd4c9666__wakuorg_nwaku:latest.log
2026-02-04 17:11:33.169 DEBUG [src.node.waku_node] Starting Node...
2026-02-04 17:11:33.169 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-04 17:11:33.170 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-04 17:11:33.171 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.231.71
2026-02-04 17:11:33.171 DEBUG [src.node.docker_mananger] Generated ports ['7208', '7209', '7210', '7211', '7212']
2026-02-04 17:11:33.171 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-04 17:11:33.171 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-04 17:11:33.171 DEBUG [src.node.waku_node] Using volumes []
2026-02-04 17:11:33.171 DEBUG [src.node.docker_mananger] docker run -i -t -p 7208:7208 -p 7209:7209 -p 7210:7210 -p 7211:7211 -p 7212:7212 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=7210 --rest-port=7208 --tcp-port=7209 --discv5-udp-port=7211 --rest-address=0.0.0.0 --nat=extip:172.18.231.71 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=1ba4e825f06c4fcc7cd0d67fa3bf65bbcaad715cdda62fde06743888bfacf3cf --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=7212 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QN05RYybRfUBzISochpw035PldswfSpEDBK1Sgiz1LVfc-M47pjzHFau5ZriSg0sQo1z5k4a9LhEkLf-wyKZg68CgmlkgnY0gmlwhKwS3HKKbXVsdGlhZGRyc5YACASsEtxyBl-hAAoErBLccgZfot0DgnJzhQADAQAAiXNlY3AyNTZrMaECX4vEu7YUcyQ8-Qc2aR_YmSi1tmfa4Tb2gwR4S5q1gBqDdGNwgl-hg3VkcIJfo4V3YWt1MgE
2026-02-04 17:11:33.363 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.231.71 waku 2bd93aadb6e9ae16e8bcbc5f906b51974c2033dbc96c827b4bd0263d54123508
2026-02-04 17:11:33.393 DEBUG [src.node.docker_mananger] Container started with ID 2bd93aadb6e9. Setting up logs at ./log/docker/node3_2026-02-04_17-11-30__cfb03cff-5136-4744-8d1d-a78fbd4c9666__wakuorg_nwaku:latest.log
2026-02-04 17:11:33.393 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 7208
2026-02-04 17:11:33.393 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-04 17:11:34.393 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7208/health" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:34.407 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-02-04 17:11:34.408 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-04 17:11:34.410 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7208/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:34.417 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.231.71/tcp/7209/p2p/16Uiu2HAmNpks7nEXjQn61yhSxhTpPWJovtK9KDzPPM2J2T8Wiv3H","/ip4/172.18.231.71/tcp/7210/ws/p2p/16Uiu2HAmNpks7nEXjQn61yhSxhTpPWJovtK9KDzPPM2J2T8Wiv3H"],"enrUri":"enr:-L24QI7O9Omddvpy-4VUD24fqbfYMolwB49Nv1wO_aZu19qGcyJfRlQvr3kc7H2jBF4IsxCZCJXIq0erGgjZdViUBwUCgmlkgnY0gmlwhKwS50eKbXVsdGlhZGRyc5YACASsEudHBhwpAAoErBLnRwYcKt0DgnJzhQADAQAAiXNlY3AyNTZrMaEDlxVt0o-xU2w2AzmikHAha2IBZKMx4JujEKVlyTvK05CDdGNwghwpg3VkcIIcK4V3YWt1MgE"}'
2026-02-04 17:11:34.417 INFO [src.node.waku_node] REST service is ready !!
2026-02-04 17:11:34.417 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7208/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.220.114/tcp/24481/p2p/16Uiu2HAm1reKhfQAjxPqH3hT69iwz2HapbvcHgVTbJZC4V8XArJu"]'
2026-02-04 17:11:34.420 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-04 17:11:34.427 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-04 17:11:34.427 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node4_2026-02-04_17-11-30__cfb03cff-5136-4744-8d1d-a78fbd4c9666__wakuorg_nwaku:latest.log
2026-02-04 17:11:34.427 DEBUG [src.node.waku_node] Starting Node...
2026-02-04 17:11:34.428 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-04 17:11:34.429 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-04 17:11:34.429 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.135.19
2026-02-04 17:11:34.429 DEBUG [src.node.docker_mananger] Generated ports ['47754', '47755', '47756', '47757', '47758']
2026-02-04 17:11:34.429 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-04 17:11:34.430 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-04 17:11:34.430 DEBUG [src.node.waku_node] Using volumes []
2026-02-04 17:11:34.430 DEBUG [src.node.docker_mananger] docker run -i -t -p 47754:47754 -p 47755:47755 -p 47756:47756 -p 47757:47757 -p 47758:47758 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=47756 --rest-port=47754 --tcp-port=47755 --discv5-udp-port=47757 --rest-address=0.0.0.0 --nat=extip:172.18.135.19 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=f15cbea0bbf0b5bef2a57f6897f84ae86eaebc9da27eaa7fff31f3a7daec6c00 --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=47758 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QN05RYybRfUBzISochpw035PldswfSpEDBK1Sgiz1LVfc-M47pjzHFau5ZriSg0sQo1z5k4a9LhEkLf-wyKZg68CgmlkgnY0gmlwhKwS3HKKbXVsdGlhZGRyc5YACASsEtxyBl-hAAoErBLccgZfot0DgnJzhQADAQAAiXNlY3AyNTZrMaECX4vEu7YUcyQ8-Qc2aR_YmSi1tmfa4Tb2gwR4S5q1gBqDdGNwgl-hg3VkcIJfo4V3YWt1MgE
2026-02-04 17:11:34.636 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.135.19 waku 5f71989be6ebecd370f085d28edce03e5e46bad7e095a799646978c93c9dede1
2026-02-04 17:11:34.669 DEBUG [src.node.docker_mananger] Container started with ID 5f71989be6eb. Setting up logs at ./log/docker/node4_2026-02-04_17-11-30__cfb03cff-5136-4744-8d1d-a78fbd4c9666__wakuorg_nwaku:latest.log
2026-02-04 17:11:34.669 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 47754
2026-02-04 17:11:34.669 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-04 17:11:35.670 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:47754/health" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:35.678 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-02-04 17:11:35.680 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-04 17:11:35.680 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:47754/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:35.684 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.135.19/tcp/47755/p2p/16Uiu2HAkupWcTZ29exzWZnrkQkdR7B9FZm6aUiKjbTGM7hGMaWhg","/ip4/172.18.135.19/tcp/47756/ws/p2p/16Uiu2HAkupWcTZ29exzWZnrkQkdR7B9FZm6aUiKjbTGM7hGMaWhg"],"enrUri":"enr:-L24QOuN7znFL0tUzHs31G7ScjBChd7Q3gLrnk5z_NCeitjGV5A_1XSGOwz-gYQT6I3gwotHvAx1QZEV78-CMygvmCkCgmlkgnY0gmlwhKwShxOKbXVsdGlhZGRyc5YACASsEocTBrqLAAoErBKHEwa6jN0DgnJzhQADAQAAiXNlY3AyNTZrMaECBdnEL9FBL6960CeV7oxYuJjUqWso23my30WKm4yJuhODdGNwgrqLg3VkcIK6jYV3YWt1MgE"}'
2026-02-04 17:11:35.685 INFO [src.node.waku_node] REST service is ready !!
2026-02-04 17:11:35.686 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:47754/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.220.114/tcp/24481/p2p/16Uiu2HAm1reKhfQAjxPqH3hT69iwz2HapbvcHgVTbJZC4V8XArJu"]'
2026-02-04 17:11:35.690 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-04 17:11:35.697 DEBUG [src.node.docker_mananger] Docker client initialized with image wakuorg/nwaku:latest
2026-02-04 17:11:35.697 DEBUG [src.node.waku_node] WakuNode instance initialized with log path ./log/docker/node5_2026-02-04_17-11-30__cfb03cff-5136-4744-8d1d-a78fbd4c9666__wakuorg_nwaku:latest.log
2026-02-04 17:11:35.697 DEBUG [src.node.waku_node] Starting Node...
2026-02-04 17:11:35.697 DEBUG [src.node.docker_mananger] Attempting to create or retrieve network waku
2026-02-04 17:11:35.699 DEBUG [src.node.docker_mananger] Network waku already exists
2026-02-04 17:11:35.699 DEBUG [src.node.docker_mananger] Generated random external IP 172.18.158.237
2026-02-04 17:11:35.699 DEBUG [src.node.docker_mananger] Generated ports ['19365', '19366', '19367', '19368', '19369']
2026-02-04 17:11:35.700 DEBUG [src.node.waku_node] RLN credentials were not set
2026-02-04 17:11:35.700 INFO [src.node.waku_node] RLN credentials not set or credential store not available, starting without RLN
2026-02-04 17:11:35.700 DEBUG [src.node.waku_node] Using volumes []
2026-02-04 17:11:35.700 DEBUG [src.node.docker_mananger] docker run -i -t -p 19365:19365 -p 19366:19366 -p 19367:19367 -p 19368:19368 -p 19369:19369 wakuorg/nwaku:latest --listen-address=0.0.0.0 --rest=true --rest-admin=true --websocket-support=true --log-level=TRACE --rest-relay-cache-capacity=100 --websocket-port=19367 --rest-port=19365 --tcp-port=19366 --discv5-udp-port=19368 --rest-address=0.0.0.0 --nat=extip:172.18.158.237 --peer-exchange=true --discv5-discovery=true --cluster-id=3 --nodekey=fa4d28afedfca1cedebbbd938ca38f92ee0a04f9d5fbbc3cffb392c8b513b0af --shard=0 --metrics-server=true --metrics-server-address=0.0.0.0 --metrics-server-port=19369 --metrics-logging=true --relay=true --discv5-bootstrap-node=enr:-L24QN05RYybRfUBzISochpw035PldswfSpEDBK1Sgiz1LVfc-M47pjzHFau5ZriSg0sQo1z5k4a9LhEkLf-wyKZg68CgmlkgnY0gmlwhKwS3HKKbXVsdGlhZGRyc5YACASsEtxyBl-hAAoErBLccgZfot0DgnJzhQADAQAAiXNlY3AyNTZrMaECX4vEu7YUcyQ8-Qc2aR_YmSi1tmfa4Tb2gwR4S5q1gBqDdGNwgl-hg3VkcIJfo4V3YWt1MgE
2026-02-04 17:11:35.894 DEBUG [src.node.docker_mananger] docker network connect --ip 172.18.158.237 waku faef6d648621e362988a1c77dee7518bbff9e6371f9bb3fe481796217ec9722b
2026-02-04 17:11:35.929 DEBUG [src.node.docker_mananger] Container started with ID faef6d648621. Setting up logs at ./log/docker/node5_2026-02-04_17-11-30__cfb03cff-5136-4744-8d1d-a78fbd4c9666__wakuorg_nwaku:latest.log
2026-02-04 17:11:35.930 DEBUG [src.node.waku_node] Started container from image wakuorg/nwaku:latest. REST: 19365
2026-02-04 17:11:35.930 DEBUG [src.libs.common] Sleeping for 1 seconds
2026-02-04 17:11:36.931 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19365/health" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:36.937 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"nodeHealth":"READY","protocolsHealth":[{"Relay":"NOT_READY","desc":"No connected peers"},{"Rln Relay":"NOT_MOUNTED"},{"Lightpush":"NOT_MOUNTED"},{"Legacy Lightpush":"NOT_MOUNTED"},{"Filter":"NOT_MOUNTED"},{"Store":"NOT_MOUNTED"},{"Legacy Store":"NOT_MOUNTED"},{"Peer Exchange":"READY"},{"Rendezvous":"NOT_READY","desc":"No Rendezvous peers are available yet"},{"Mix":"NOT_MOUNTED"},{"Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Legacy Lightpush Client":"NOT_READY","desc":"No Lightpush service peer available yet"},{"Store Client":"NOT_READY","desc":"No Store service peer available yet, neither Store service set up for the node"},{"Legacy Store Client":"NOT_READY","desc":"No Legacy Store service peers are available yet, neither Store service set up for the node"},{"Filter Client":"NOT_READY","desc":"No Filter service peer available yet"}]}'
2026-02-04 17:11:36.938 INFO [src.node.waku_node] Node protocols are initialized !!
2026-02-04 17:11:36.940 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19365/debug/v1/info" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:36.945 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'{"listenAddresses":["/ip4/172.18.158.237/tcp/19366/p2p/16Uiu2HAmBb8pWNFJKaGgwNLYAapGfxx8b6TXCJ7KGdQyVKVXjRjv","/ip4/172.18.158.237/tcp/19367/ws/p2p/16Uiu2HAmBb8pWNFJKaGgwNLYAapGfxx8b6TXCJ7KGdQyVKVXjRjv"],"enrUri":"enr:-L24QE_GNhlqnAVvhZ2BgNQtSuehUM4LXwEGdA5PXz5YAvU9eW1Q_oucmXzX_A-PgIKj_VHxjH1iZ0t4fSDKdB5TO1YCgmlkgnY0gmlwhKwSnu2KbXVsdGlhZGRyc5YACASsEp7tBkumAAoErBKe7QZLp90DgnJzhQADAQAAiXNlY3AyNTZrMaEC8CeQ6Yswb052rgGGFfBJBLEi55G5Bw4P7R-TSGxzAYmDdGNwgkumg3VkcIJLqIV3YWt1MgE"}'
2026-02-04 17:11:36.946 INFO [src.node.waku_node] REST service is ready !!
2026-02-04 17:11:36.947 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19365/admin/v1/peers" -H "Content-Type: application/json" -d '["/ip4/172.18.220.114/tcp/24481/p2p/16Uiu2HAm1reKhfQAjxPqH3hT69iwz2HapbvcHgVTbJZC4V8XArJu"]'
2026-02-04 17:11:36.951 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-04 17:11:36.951 DEBUG [src.steps.relay] Running fixture setup: subscribe_main_relay_nodes
2026-02-04 17:11:36.952 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:24480/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-02-04 17:11:36.959 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-04 17:11:36.960 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:20402/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-02-04 17:11:36.966 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-04 17:11:36.966 DEBUG [src.steps.relay] Running fixture setup: subscribe_optional_relay_nodes
2026-02-04 17:11:36.967 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:7208/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-02-04 17:11:36.971 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-04 17:11:36.971 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:47754/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-02-04 17:11:36.978 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-04 17:11:36.978 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:19365/relay/v1/subscriptions" -H "Content-Type: application/json" -d '["/waku/2/rs/3/1"]'
2026-02-04 17:11:36.983 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-04 17:11:36.984 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:24480/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-02-04 17:11:36.993 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-04 17:11:36.993 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-02-04 17:11:37.095 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-02-04 17:11:37.095 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24480/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:37.098 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1770225096984209152,"ephemeral":false,"proof":""}]'
2026-02-04 17:11:37.099 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-02-04 17:11:37.099 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:20402/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:37.102 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1770225096984209152,"ephemeral":false,"proof":""}]'
2026-02-04 17:11:37.103 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-02-04 17:11:37.104 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7208/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:37.106 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1770225096984209152,"ephemeral":false,"proof":""}]'
2026-02-04 17:11:37.108 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-02-04 17:11:37.108 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:47754/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:37.110 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1770225096984209152,"ephemeral":false,"proof":""}]'
2026-02-04 17:11:37.112 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-02-04 17:11:37.112 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19365/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:37.115 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1770225096984209152,"ephemeral":false,"proof":""}]'
2026-02-04 17:11:37.116 INFO [src.steps.relay] WARM UP successful!!
2026-02-04 17:11:37.118 INFO [src.node.api_clients.base_client] curl -v -X POST "http://127.0.0.1:24480/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d '{"payload": "UmVsYXkgd29ya3MhIQ==", "contentTopic": "/test/1/waku-relay/proto", "timestamp": '$(date +%s%N)'}'
2026-02-04 17:11:37.126 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'OK'
2026-02-04 17:11:37.128 DEBUG [src.libs.common] Sleeping for 0.1 seconds
2026-02-04 17:11:37.228 DEBUG [src.steps.relay] Checking that peer NODE_1:wakuorg/nwaku:latest can find the published message
2026-02-04 17:11:37.229 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:24480/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:37.232 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1770225097118083081,"ephemeral":false,"proof":""}]'
2026-02-04 17:11:37.233 DEBUG [src.steps.relay] Checking that peer NODE_2:wakuorg/nwaku:latest can find the published message
2026-02-04 17:11:37.233 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:20402/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:37.236 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1770225097118083081,"ephemeral":false,"proof":""}]'
2026-02-04 17:11:37.237 DEBUG [src.steps.relay] Checking that peer NODE_3:wakuorg/nwaku:latest can find the published message
2026-02-04 17:11:37.237 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:7208/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:37.240 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1770225097118083081,"ephemeral":false,"proof":""}]'
2026-02-04 17:11:37.241 DEBUG [src.steps.relay] Checking that peer NODE_4:wakuorg/nwaku:latest can find the published message
2026-02-04 17:11:37.241 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:47754/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:37.244 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1770225097118083081,"ephemeral":false,"proof":""}]'
2026-02-04 17:11:37.246 DEBUG [src.steps.relay] Checking that peer NODE_5:wakuorg/nwaku:latest can find the published message
2026-02-04 17:11:37.246 INFO [src.node.api_clients.base_client] curl -v -X GET "http://127.0.0.1:19365/relay/v1/messages/%2Fwaku%2F2%2Frs%2F3%2F1" -H "Content-Type: application/json" -d 'None'
2026-02-04 17:11:37.248 INFO [src.node.api_clients.base_client] Response status code: 200. Response content: b'[{"payload":"UmVsYXkgd29ya3MhIQ==","contentTopic":"/test/1/waku-relay/proto","version":0,"timestamp":1770225097118083081,"ephemeral":false,"proof":""}]'
2026-02-04 17:11:37.252 DEBUG [tests.conftest] Running fixture teardown: test_setup
2026-02-04 17:11:37.253 DEBUG [tests.conftest] Running fixture teardown: close_open_nodes
2026-02-04 17:11:37.254 DEBUG [src.node.waku_node] Stopping container with id 9e8c4f8cb1bf
2026-02-04 17:11:37.819 DEBUG [src.node.waku_node] Container stopped.
2026-02-04 17:11:37.819 DEBUG [src.node.waku_node] Stopping container with id 1af8ad733c28
2026-02-04 17:11:38.374 DEBUG [src.node.waku_node] Container stopped.
2026-02-04 17:11:38.375 DEBUG [src.node.waku_node] Stopping container with id 2bd93aadb6e9
2026-02-04 17:11:38.632 ERROR [src.node.docker_mananger] Max retries reached for container 9e8c4f8cb1bf. Exiting log stream.
2026-02-04 17:11:38.931 DEBUG [src.node.waku_node] Container stopped.
2026-02-04 17:11:38.933 DEBUG [src.node.waku_node] Stopping container with id 5f71989be6eb
2026-02-04 17:11:39.185 ERROR [src.node.docker_mananger] Max retries reached for container 1af8ad733c28. Exiting log stream.
2026-02-04 17:11:39.476 DEBUG [src.node.waku_node] Container stopped.
2026-02-04 17:11:39.476 DEBUG [src.node.waku_node] Stopping container with id faef6d648621
2026-02-04 17:11:39.745 ERROR [src.node.docker_mananger] Max retries reached for container 2bd93aadb6e9. Exiting log stream.
2026-02-04 17:11:40.000 DEBUG [src.node.waku_node] Container stopped.
2026-02-04 17:11:40.002 DEBUG [tests.conftest] Running fixture teardown: check_waku_log_errors
2026-02-04 17:11:40.017 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-04 17:11:40.024 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-04 17:11:40.030 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-04 17:11:40.035 DEBUG [src.node.docker_mananger] No errors found in the waku logs.
2026-02-04 17:11:40.041 DEBUG [src.node.docker_mananger] No errors found in the waku logs.